Microsoft has added a small new button next to the open New tab button in Internet Explorer that lets you quickly open the Microsoft Edge browser. If it gets in your way you can easily remove it.
Hide the button that opens Microsoft Edge in IE
Using Internet Options
Open Internet Options > Advanced tab.
Under Browsing, select the Hide the button (next to the New Tab button) that opens Microsoft Edge setting and click Apply.
Restart IE and see.
Close Internet Explorer. Next, type gpedit.msc in Start Search and hit Enter to open the Local Group Policy Editor.
Now navigate to the following setting:
User Configuration\Administrative Templates\Windows Components\Internet Explorer\Internet Settings\Advanced Settings\Browsing\
Now in the right pane double-click on the following setting to open its configuration box:
Hide the button (next to the New Tab button) that opens Microsoft Edge
This policy setting allows you to manage if users can see the button (next to the New Tab button) that opens Microsoft Edge. If you enable this policy setting, the button to open Microsoft Edge from Internet Explorer will be hidden. If you disable this policy setting, the button to open Microsoft Edge from Internet Explorer will be shown. If you do not configure this policy setting, the button to open Microsoft Edge from Internet Explorer can be configured by the user.
Double click it and change it from Not configured to Enabled.
Now open and check your Internet Explorer!
Using Registry Editor
If your version of Windows 10 does not ship with the Group Policy Editor, Run regedit to open the Registry Editor. Copy-paste this path in the address bar of the Registry Editor and hit Enter:
In the right pane change the value of HideNewEdgeButton from 0 (zero) to 1 (one).
If this data value is not present, you will have to create it. To do this, right-click in the right-side > New > DWORD (32-bit) Value and name it HideNewEdgeButtonwith. Give it a value of 1 (one).
The button to open Microsoft Edge will have been removed.
Thanks for the tip Joey P and George Gi.