Microsoft’s new Copilot Vision can ‘see’ your apps on Windows

Microsoft is launching Copilot Vision for Windows today, an extension of its AI assistant that lets you share your screen or apps so that Copilot can do things like coach you through using Adobe Photoshop features or analyze the photos and webpages you’re looking at.

“When you choose to enable it, Copilot Vision can see what you see and talk to you about it in real time,” says the Copilot team. “It acts as your second set of eyes, able to analyze content, help when you’re lost, provide insights, and answer your questions as you go.”

Microsoft first started testing Copilot Vision in Windows earlier this year, and now it’s launching free in the US for both Windows 10 and Windows 11 users. You’ll no longer need a Copilot Pro subscription, but it’s still part of Microsoft’s experimental Copilot Labs AI initiatives.

Copilot Vision on Windows works within the Copilot app and can be activated through a glasses icon that lets you select a browser window or app that you want to share with the AI — much like sharing your screen or apps in a Microsoft Teams meeting. It’s an opt-in experience, where you’ll have to specifically share your apps for Copilot Vision to work; it’s not like Recall, where it’s continuously taking snapshots of your screen once it’s enabled.

I haven’t been able to fully test Copilot Vision on Windows because Microsoft is limiting the experience to the US, but I did get a brief hands-on with it at Microsoft’s 50th anniversary event in April. Once you share an app, Copilot is then able to highlight parts of your screen to guide you and help answer questions. You can also use Copilot Vision on iOS and Android.

Update, June 12th: Article updated to reflect that Copilot Vision doesn’t require Copilot Pro.