Microsoft has announced the Fall Moments update for its Windows 11 operating system, introducing a set of features designed to integrate artificial intelligence capabilities into every supported personal computer. The update centers on two primary additions, Copilot Voice and Copilot Vision, which function together to create a more interactive AI environment for users.
The first feature, Copilot Voice, enables users to interact with their computer through voice commands and natural conversation, a method of interaction similar to that of other conversational AI tools. The second core feature is Copilot Vision, which allows the computer to perceive and interpret its surroundings through a camera. Copilot Vision can also analyze the content displayed on a user’s screen, enabling it to understand context and perform actions based on what it “sees.”
As part of this update, the AI assistant can be activated by voice using the phrase “Hey, Copilot.” Once activated, users can ask questions, search for information, or direct the assistant to execute specific tasks. The Copilot Vision functionality allows users to share their desktop or individual applications directly with the AI. In return, Copilot can provide insights or answer questions related to the on-screen content.
The update also introduces a guidance feature that Microsoft calls Highlights. This tool is designed to assist users who are unfamiliar with certain computer operations. By pointing to an area on the screen and asking for help, users can receive step-by-step instructions from the AI on how to complete a task. Microsoft has noted this feature is particularly intended to benefit users with lower levels of technological familiarity. The Highlights functionality is available within the operating system and inside specific applications.
In conjunction with these features, the Copilot typing interface has been redesigned and integrated directly into the Windows 11 taskbar. This change aims to make text-based interaction with the AI assistant more accessible.
Within the Microsoft 365 application suite, which includes PowerPoint, Excel, and Word, Copilot Vision has been enhanced to understand the full context of a user’s work. With explicit user permission, the AI can view the screen and offer assistance on tasks such as designing a presentation or analyzing spreadsheet data.
Microsoft is also expanding Copilot Actions, a feature that allows an intelligent agent to perform multi-step tasks for the user. Previously limited to browser-based actions like booking a hotel or finding flights, this capability is now being extended to the Windows 11 Insiders program. The expansion allows Copilot Actions to operate within the local operating system, performing tasks in File Explorer or searching for files on the computer. Users can describe a desired outcome, and the AI will work to execute it. This connects to a broader Microsoft initiative to link its AI with services like OneDrive and external platforms such as Google Gmail, with the ability to export task results to other applications.
Additionally, Microsoft announced a collaboration with the task-agent tool Manus. This partnership will enable the tool to perform complex operations on a user’s local files, contingent on receiving permission. The technical basis for this collaboration is a shared infrastructure built by Microsoft and Manus’ parent company, a third-party firm based in Shanghai, utilizing Azure servers enhanced with AI capabilities.
To address security and privacy, the update includes new control levels described as “security fences.” Microsoft states that users will have full control over the new features and can disable or suspend Copilot Actions at any time. The company also commits to providing users with full transparency regarding the AI’s actions, with updates provided at each stage of a task. Referencing lessons from the recent “recall fiasco,” Microsoft asserts that it is continuously performing comprehensive checks and collecting organizational feedback to improve its security and privacy mechanisms.
