Claude from Anthropic: the AI that learns by observing your screen

⚡

Key Takeaways

1Anthropic has introduced a feature that allows Claude to learn by recording users' screens.

2The "Record a skill" function aims to automate repetitive tasks by capturing work processes.

3Available to paying subscribers, this option raises security concerns regarding sensitive data.

💡Why it matters — This innovation could transform the way professionals automate their tasks, but it presents challenges related to data privacy.

Claude by Anthropic: The AI That Learns by Observing Your Screen

Anthropic, a company specializing in artificial intelligence, has recently launched an innovative feature for its tool Claude. This new capability allows the AI to learn users' work methods by directly observing their screens. However, this learning method raises questions about its relevance for all users.

A New Way to Teach Claude

The feature introduced by Anthropic enables Claude to record users' screens to learn work processes. This option, called "Record a skill," is an addition to the existing "Cowork" tools available for paid subscribers. With this feature, users can now demonstrate to the AI how to perform tasks by recording their on-screen actions, including clicks and inputs, while adding voice comments. However, Anthropic warns against including passwords, secret keys, or confidential information in these recordings.

Once the recording is complete, Claude analyzes the demonstration and generates what Anthropic refers to as a "repeatable skill." This means that the AI can automate the process without requiring a detailed textual description, thereby simplifying the task for the user.

Towards Automation of Repetitive Tasks

The primary goal of this feature is to facilitate the automation of repetitive tasks. For example, it can be used for report creation that requires aggregating data from multiple sources. A user could record the necessary steps to extract and organize information from different files, and Claude would then be able to reproduce this sequence automatically. This feature is accessible to subscribers of the Max, Pro, and Team plans, but no information has been provided regarding potential availability for free users.

Access Conditions and Limitations

Several subscribers reported the appearance of the option shortly after its announcement. To use it, one must access the "Cowork" tab and select "Record a skill" before starting the recording. However, Anthropic has not specified the number of allowed uses based on the subscription plan. A Reddit user, known by the pseudonym neveralone59, mentioned that the function was available once every five hours with a Pro subscription. It remains to be determined whether the recording duration or the complexity of tasks impacts this timeframe.

A Step Forward for iOS Developers

In parallel, Anthropic has also introduced a new feature for iOS app developers. Integrated into the "Claude Code" application for Mac, this option allows emulating the behavior of a service via a virtual simulator. This provides developers with the opportunity to test applications intended for iPhone and iPad in a simulated environment, thus facilitating the development and debugging of their applications.