Apple and Qualcomm: AI Under Control for Enhanced Security

⚡

Key Takeaways

1Apple and Qualcomm are developing AI assistants with limitations to ensure user safety.

2These assistants can navigate applications and perform tasks, but require confirmation for sensitive actions.

3The AI systems are designed to protect privacy by limiting access to data and incorporating security checks.

💡Why it matters — These measures aim to prevent financial and privacy risks associated with the increasing autonomy of AI.

Next-Generation AI Assistants

In the field of artificial intelligence, Apple and chip manufacturers like Qualcomm are developing next-generation AI assistants. These systems are designed to perform a variety of tasks, ranging from navigating applications to managing services. However, preliminary reports indicate that these assistants are intentionally equipped with limitations to ensure user safety.

According to Tom’s Guide, these assistants can perform actions such as booking services or posting content in applications. For example, during a private beta test, an agentic system was able to navigate an application workflow until it reached a payment screen. At this point, it requested user confirmation before finalizing the action, illustrating the "human in the loop" model.

User Control

AI agents are designed with built-in approval checkpoints. For sensitive actions, particularly those involving payments or account modifications, user confirmation is required before any finalization. This approach ensures that the system cannot perform actions that the user has not explicitly requested.

This method is already in use in banking applications, where confirmation is necessary for transfers. Now, this concept is being applied to AI-driven actions across various services, thereby enhancing user security and trust.

Access Limits and Privacy

Another layer of control lies in restricting the AI's access to applications and data. Rather than allowing unlimited access, companies are establishing clear boundaries on the applications with which the AI can interact and when actions can be triggered.

In practice, this means that the AI can draft a purchase or prepare a reservation, but it cannot finalize them without user approval. Furthermore, the system cannot move freely across all services without prior authorization. This feature aims to protect user privacy by keeping data on the device, thus eliminating the need to send sensitive information to external servers.

Enhanced Security in Payments

In the payments sector, AI systems are designed to work with partners that already have strict rules in place. For example, payment provider services are integrated to provide secure authentication before transactions are finalized. Although these security measures are still under development, they add an additional layer of oversight.

Existing systems can set transaction limits or require additional verification, ensuring that sensitive actions are monitored and controlled.

Towards Controlled Autonomy

As AI gains the ability to perform actions, the associated risks also increase. Errors can lead to financial losses or data exposure. To manage these risks, companies are implementing multi-level controls, including user approval and secure infrastructure.

This approach could influence the future development of agentic AI. Rather than aiming for total autonomy, companies seem to favor controlled environments where risks can be better managed. This could shape the way agentic AI evolves in the short term, focusing on security and control.