Microsoft has made its Copilot Vision AI assistant available for free within the Edge browser, as announced by Mustafa Suleyman, CEO of Microsoft AI. This development enables users to leverage the AI’s capabilities to interpret screen content.
Copilot Vision offers a “talk-based experience,” allowing users to interact with it through spoken commands and receive responses from Copilot. According to Suleyman, once enabled, Copilot Vision can “literally see what you see on screen” and guide users through various tasks. For instance, it can assist users in following a recipe while cooking or help with job application materials by decoding job descriptions and providing customized interview preparation or cover letter brainstorming.
When activated, Copilot Vision may highlight portions of the screen to help users find relevant information, as stated on a Microsoft support page. However, it is essential to note that Copilot Vision does not click links or perform actions on behalf of the user. While the feature is now available for free within Edge, broader system-wide Copilot Vision capabilities remain exclusive to Copilot Pro subscribers. These subscribers can access Vision in other applications, such as Photoshop, video editing software, or games like Minecraft.
To opt into Copilot Vision, users need to visit a specific Microsoft website within the Edge browser. Upon doing so, they should receive a prompt to enable the feature. Once enabled, users can initiate a Vision session by opening the Copilot sidebar, clicking the microphone icon, and waiting for the session to start, indicated by a chime and a change in the browser’s color. However, the writer’s experience was marred by technical issues, possibly due to using an underpowered laptop, which prevented the controls from loading properly.
According to Microsoft, during a Copilot Vision session, the company logs Copilot’s responses to the user but does not collect user inputs, images, or page content. Users can terminate a Vision session either by ending it through the controls or by closing the browser window.
