Google Rolls Out Application Control by Gemini on Mac

Google is starting today to roll out local or web application control in Gemini on Mac, in American English, to Google AI Plus, Pro, and Ultra subscribers. The assistant can perform clicks and enter text to carry out multi-step operations.
How to Enable Application Control
To request this mode, the query must include the mention @Computer use. The Customize > Connectors menu features applications identified by the label "Desktop," including Blender and Autodesk Fusion. These are in addition to the integrations with Git and Obsidian.
Use Cases for Creation, Forms, and Reselling
Google presents this capability as a time-saver for tasks spread across software and websites that lack connectivity with each other. In creation, the company mentions both batch photo colorization and the setup of complex 3D scenes. It also proposes extracting weights, dimensions, and addresses from local PDFs, then transferring them into various shipping quote fields.
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
Another scenario put forward by Google involves gathering indicators and live-updated dashboard snapshots covering multiple regions, then producing a Google Slides presentation. For clothing resale, the assistant could examine photos stored in a local folder, search for prices, log item characteristics, and prepare multiple listings.
An Isolated Window and Pauses for Sensitive Actions
Until this update, Task, previously known as Gemini Spark, could only access local files in conjunction with connected applications like Gmail or Google Calendar. The assistant can now navigate menus or fill out forms in Mac software just like in web applications.
Operations take place in a separate and isolated window, in the background; their progress is visible live in the conversation. The cursor and main screen remain available for other activities. The user selects the authorized applications and can interrupt the execution with a click at any time.
Google plans for pauses during connection or payment steps. Validation is required for sensitive operations.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.