Brief IA
›

Google Rolls Out Application Control by Gemini on Mac

AI Tools·Tom Levy·

Google Rolls Out Application Control by Gemini on Mac

Google Rolls Out Application Control by Gemini on Mac
Key Takeaways
1Gemini on Mac can perform clicks and enter text in local or web applications.
2The rollout begins in American English for Google AI Plus, Pro, and Ultra subscribers.
3Users can choose the allowed applications and can stop the assistant at any time.
Why it matters : Gemini can support steps in software and websites that do not have a connection between them.

Google is starting today to roll out local or web application control in Gemini on Mac, in American English, to Google AI Plus, Pro, and Ultra subscribers. The assistant can perform clicks and enter text to carry out multi-step operations.

How to Enable Application Control

To request this mode, the query must include the mention @Computer use. The Customize > Connectors menu features applications identified by the label "Desktop," including Blender and Autodesk Fusion. These are in addition to the integrations with Git and Obsidian.

Use Cases for Creation, Forms, and Reselling

Google presents this capability as a time-saver for tasks spread across software and websites that lack connectivity with each other. In creation, the company mentions both batch photo colorization and the setup of complex 3D scenes. It also proposes extracting weights, dimensions, and addresses from local PDFs, then transferring them into various shipping quote fields.

La lettre du soirGratuit · 19 h

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Another scenario put forward by Google involves gathering indicators and live-updated dashboard snapshots covering multiple regions, then producing a Google Slides presentation. For clothing resale, the assistant could examine photos stored in a local folder, search for prices, log item characteristics, and prepare multiple listings.

An Isolated Window and Pauses for Sensitive Actions

Until this update, Task, previously known as Gemini Spark, could only access local files in conjunction with connected applications like Gmail or Google Calendar. The assistant can now navigate menus or fill out forms in Mac software just like in web applications.

Operations take place in a separate and isolated window, in the background; their progress is visible live in the conversation. The cursor and main screen remain available for other activities. The user selects the authorized applications and can interrupt the execution with a click at any time.

Google plans for pauses during connection or payment steps. Validation is required for sensitive operations.

Brief IA — L'actualité IA en français

L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.