Microsoft Research: AI Degrades 25% of Long Documents

Artificial intelligence systems, when faced with complex and long-duration tasks, can seriously compromise the integrity of the documents they process. A recent study from Microsoft Research highlights this issue, revealing that large language models (LLMs) can silently corrupt up to 25% of business documents during lengthy tasks. Even the most advanced models, such as Gemini 3.1 Pro, Claude 4.6 Opus, and GPT 5.4, are not spared, showing similar degradation over extended workflows. Other models may fail even more severely, with an average across all models hovering around 50%. However, Python stands out by achieving a fidelity threshold of 98%.

When an AI has to manage thousands of pages or hours of audio transcription, it must be capable of processing a massive amount of information simultaneously. This long context is particularly prone to errors, as indicated by the Microsoft Research study. The reliability of LLMs significantly decreases with the complexity and length of tasks. Fortunately, there are various solutions to minimize risk and ensure the reliability of AI systems.

Breaking Down Complex Tasks

To mitigate these degradations, the study recommends breaking complex tasks into verifiable micro-tasks. Each additional slice of 1,000 tokens results in an average degradation of 3.6% after 20 interactions. By breaking down the work, each step becomes more manageable and verifiable, reducing errors. A 2025 study demonstrates that this approach can improve model accuracy by 9% to 40% compared to a standard method. In the case of agentic systems, each agent works on a small part of the problem, reducing the risk of overall error. For example, during onboarding, the inspecting agent checks for the presence of administrative files in a folder, while the "HR secretary" agent takes the inspector's raw report and transforms it into a friendly message.

Human Verification and RAG

Integrating human feedback loops, or "human-in-the-loop," is also crucial. These mechanisms allow for the verification of response accuracy at each step. The study "Applications, challenges, and future directions of human-in-the-loop learning" highlights that human-in-the-loop systems can achieve high precision in reliability detection (0.95). When requesting the creation of a document, one might ask the AI to draft a version that requests review and validation at various stages. For the agentic system, "checkpoints" can be inserted into the Python code so that a human review ensures the quality of the response.

Moreover, the use of RAG (Retrieval Augmented Generation) helps provide recent and relevant information, thereby minimizing model hallucinations. If their training data is outdated or incomplete, LLMs can invent responses. The Microsoft Research study shows that LLMs can forget key information as the context lengthens. In this context, providing recent and reliable information from its own databases can be wise. In the case of RAG, only relevant information is provided to the LLM as context for generating its response, significantly reducing hallucinations.

Logging Interactions

Finally, implementing detailed logging of AI interactions is essential for identifying and correcting silent corruptions. Recording each request, response, and human validation allows for auditing performance and continuously improving the system. According to an analysis of best practices in AI debugging in 2025, AI-based debugging tools saw their problem resolution rates improve from 4.4% in 2023 to 69.1% in 2025. Although this figure cannot be directly attributed to logging alone, detailed logging enables these tools to function effectively. Platforms like Datadog LLM Observability, LangSmith, or Arize Phoenix offer solutions for monitoring and resolving issues in AI systems throughout their lifecycle. Interactions with the AI can also be recorded by integrating a logging system into the application. Each call to the AI, every processed data point, and each human validation are timestamped and recorded, allowing for performance audits, error pattern identification, and continuous system improvement.