Tencent: AI Must Perform Tasks to Be a Colleague

Tencent: AI Must Accomplish Tasks to Be a Colleague

A study argues that AI systems will not become reliable colleagues until they complete entire tasks in persistent work environments, rather than merely generating responses. The key lies in "reusable skills."

A research team from Tencent's Youtu Lab and several Chinese universities outlines the transition "from chatbot to digital colleague" along two dimensions in a new research paper: cognitive core and tool-assisted task execution.

The central question is no longer how a model produces a better response, but how it reliably transforms an intention into completed work, the researchers assert. The goal shifts from reactive responses to executing delegated tasks.

The paper traces the evolution of large language models through five stages, from basic chatbot to autonomous digital colleague.

From Quick Responses to Slow Thinking

In the era of chatbots, models primarily generated text quickly. They stored linguistic patterns and facts in their parameters, then crafted responses all at once, token by token, following the most probable continuation without checking intermediate steps or seeking solutions.

Thinking LLMs invest additional resources at the time of inference, exploring solution paths, verifying intermediate steps, and correcting errors before the final response.

The era of thinking LLMs, initiated by OpenAI and Deepseek, dedicates more computational power at the moment of responding. These models produce long chains of thought, verify intermediate steps, and learn through reinforcement learning to search for and self-correct. Only verifiable solutions receive rewards. The researchers present this as a transition from fast, intuitive thinking to slow, deliberate reflection, borrowing the framework from psychologist Daniel Kahneman.

From Tool Calls to Work Environments

First-generation agents could call APIs, write code, and browse the web, but they remained fragile. The researchers identify four structural bottlenecks: agents perceived their environment only in fragments, tool calls left no lasting state, unexpected behavior broke them, and they rarely completed tasks.

In the OpenClaw era, the model operates in a persistent and secure workspace with files, terminals, reusable skills, and verification loops until verifiable completion.

The OpenClaw era is where the environment itself becomes persistent. Files, sessions, logs, browsers, permissions, and skills survive throughout the workflow. The paper cites OpenHands and SWE-agent, which integrate agents into controlled development environments.

Workspace and Skill as the Missing Link

The central argument of the paper is that the combination of workspace and skill enables a true leap in performance. A workspace provides state, storage, and consequences, while a skill bundles operational knowledge into reusable packages. Agent Skills from Anthropic already formalize this model in the form of folders containing a SKILL.md file with instructions, scripts, and resources.

The workspace plus skill approach groups recurring workflows into versioned skill packages instead of reinventing each task through ad-hoc prompts.

According to the researchers, skills are not prompts, nor are they traditional tools. They lie between the model's reasoning and execution in the workspace, allowing organizations to capture know-how in a modular, testable, and portable form. However, the authors also warn that reusable procedures can become obsolete, adapt to specific workflows, or become vectors for attack.

Why Training and Evaluation Must Change

This shift also transforms how these systems are trained and evaluated. Chatbots learned from instruction-response pairs and were rated on the accuracy of their responses. Workspace-based systems learn from state-action-observation trajectories. Success no longer depends on plausible responses, the researchers argue, but on task closure: whether the system brings the target environment to a verifiable final state.

Evaluation shifts from scoring final responses to verifying the entire process and confirmed task completion.

Benchmarks such as SWE-bench, OSWorld, and WebArena require reproducible starting states, executable tools, trajectory logs, and final state checks. GPT-4 initially completed only 14% of the tasks in WebArena, highlighting how realistic web environments are far removed from static Q&A scenarios.

Security Becomes an Operational Concern

Persistent workspaces also expand the attack surface. Agents hold credentials, local files, identity tokens, and communication channels. Projects like OpenClaw PRISM and ClawGuard aim to establish permissions, provenance tracking, and audit logs as real-time security guarantees. Data sovereignty is equally important, the authors argue, as workspace agents observe sensitive repositories, internal documents, and intermediate results that could later become memories, skills, or training data.

As autonomy increases, errors become more durable and harder to undo. Open issues include rollback, governance, and persistent memory.

The authors acknowledge that the workspace plus skill combination is not a complete solution. Skills can overfit, and workspaces can fill up with obsolete files and broken artifacts. A reliable deployment, the researchers argue, requires skill lifecycle management, workspace hygiene, permission checks, sandboxing, rollback, and trajectory-based evaluation.

Towards Future Systems

The paper outlines future systems that learn from their own execution traces and gradually improve as digital colleagues.

A recent survey conducted by Meta, Stanford, and the University of Illinois Urbana-Champaign made a similar argument from a different angle: the performance of autonomous systems depends less on the underlying model than on the software that surrounds it. This "harness" groups tools, sandbox execution environments, and verification mechanisms.

The "skill" part of this argument becomes complicated in practice, according to a recent assessment by Vercel. It revealed that coding agents did not even call a provided skill system 56% of the time, while a compressed documentation index embedded in a AGENTS.md file achieved 100% success. The skill system capped at 79%. A passive context, always present, outperformed the active retrieval of skills, tipping the balance in favor of the workspace.