Agentic RAG: Revolutionizing Research with Iterative AI

Agentic RAG: A New Era for Information Retrieval

Applications of Large Language Models (LLM)

Agentic RAG offers an innovative approach to information retrieval, transforming the process into a continuous loop of searching, reading, and decision-making. This method relies on a minimal implementation of the OpenAI Agents SDK, emphasizing iterative interaction with data.

The application we are developing is a RAG (Retrieval-Augmented Generation) application. The method seems straightforward: slice the data, integrate it, retrieve it, and then formulate a response. However, in practice, challenges quickly multiply. Similarity search may identify closely worded formulations without providing relevant information, and crucial elements may be omitted from the retrieved context due to inadequate ranking or inappropriate chunking of data pieces.

With limited context, large language models (LLMs) have little room to compensate. So, why not make retrieval iterative? What if the model could not only search and read but also decide whether it has enough evidence before continuing its search? This approach could even do without vector integrations at the outset. This is where the concept of Agentic RAG comes into play.

In this article, we will explore how to build a minimalist agentic RAG workflow using the OpenAI Agents SDK. We will see how the agent performs searching, reading, and anchoring its response iteratively. Finally, we will take a step back to discuss the elements to consider when developing a viable agentic RAG solution.

1. Case Study: Answering a Policy Question with Agentic RAG

To illustrate the potential of Agentic RAG, we built an agent specialized in corporate policies, capable of interacting with a collection of relevant documents.

1.1 Curation of the Document Collection

We created a series of six synthetic corporate policy documents, each in markdown format. Each document includes a title, an effective date, a summary, and the full text of the policy.

These documents cover six common areas of corporate policies:

• approval_matrix.md: Details of approval levels for business travel decisions, effective from July 1, 2025.
• conference_guidelines.md: Rules for attending external events, applicable from May 15, 2025.
• faq.md: Informal answers to frequently asked questions about travel, valid from September 1, 2025.
• policy_updates_2026.md: Updates on lodging, conference travel, and the approval timeline for 2026, effective from January 1, 2026.
• remote_work_policy.md: Rules for remote work, effective from February 1, 2026.
• travel_policy.md: Standard rules for booking travel, including flights, lodging, meals, and transportation, applicable from March 1, 2025.

Answering a policy question may require information from multiple documents, allowing us to observe the desired agentic behavior. The complete synthetic documents and the implementation notebook for agentic RAG are available for consultation.

1.2 Defining the Agent

To set up the agent, we used the OpenAI Agents SDK. The agent is defined simply:

# pip install openai-agents
from agents import Agent
name="Policy Research Assistant",
instructions=INSTRUCTIONS,
model="[gpt](/glossaire/gpt)-5.4",
tools=[list_docs, search_docs, read_doc],

Two key elements need to be specified: the agent's instructions and the tools at its disposal.

The instructions determine the desired search behavior:

# Note: This instruction is iterated with the AI
INSTRUCTIONS = """
You are a careful internal policy research assistant.
[Search Behavior]
Answer employee questions about policies using the document tools.
Find enough relevant evidence to support the answer.
Keep conclusions anchored in the policy documents.
[Expected Output]
First, give a direct answer.
Then briefly explain the evidence.
Cite the file names of the documents used for each important claim.
"""

For this case study, the agent can access the documents only through three predefined tools:

• A tool to get a quick overview of the available documents:

def list_docs() -> list[dict]:
    """Lists the available policy documents without returning their text."""
    "doc_name": doc["doc_name"],
    "title": doc["title"],
    "effective": doc["effective"],
    "summary": doc["summary"],
    for doc in docs.values()

• A keyword search tool that compares each query with chunks of paragraphs:

def search_docs(query: str) -> list[dict]:
    """Searches the policy documents and returns the top three short excerpts."""
    query_tokens = tokenize(query)
    for chunk in chunks:
        score = len(query_tokens & chunk["tokens"])
        scored.append((score, chunk))
    scored.sort(key=lambda item: item[0], reverse=True)
    for score, chunk in scored[:3]:
        snippet = chunk["text"].replace("\n", " ")
        if len(snippet) > 420:
            snippet = snippet[:417].rstrip() + "..."
        results.append({
            "doc_name": chunk["doc_name"],
            "title": chunk["title"],
            "section": chunk["section"],
            "snippet": snippet,
            "score": round(score, 2),

• A tool to open a document by its file name:

def read_doc(doc_name: str) -> str:
    """Reads a policy document by file name."""
    if doc_name not in docs:
        valid = ", ".join(sorted(docs))
        return f"Unknown document: {doc_name}. Valid documents: {valid}"
    return docs[doc_name]["text"]

This is how the complete RAG agent is constructed.

1.3 Executing a Policy Question

We tested the agent with a concrete question:

"I am attending a conference in Berlin. The conference organizer lists an official hotel, but the nightly rate exceeds the normal hotel cap. Can I book this hotel, and what approval do I need before booking?"

The agent was executed with:

from agents import Runner
result = await Runner.run(agent, PROMPT, max_turns=12)

The agent provided the correct answer: the employee can book the official conference hotel if there is a valid business reason. This information was extracted from conference_guidelines.md.

Regarding approval, the agent first identified that approval was necessary due to the rate exceeding the normal cap, then specified the required approval conditions. The agent used travel_policy.md, approval_matrix.md, and policy_updates_2026.md to support its answer, which perfectly matched our expectations.

The most fascinating part lies in the trace, which reveals the agent's thought process. To visualize this trace:

for item in result.new_items:
    print(type(item).__name__, item)

result.new_items contains the intermediate tool calls and outputs generated by the agent. In our execution, the agent first used search_docs() with keywords like "conference hotel," "hotel cap," "approval," and "Berlin." It then consulted list_docs() to review the available documents before opening the relevant files with read_doc(). Only after gathering this information did it formulate its final response.

This is precisely the agentic loop we sought to observe.

3. Considerations Before Building an Agentic RAG

The case study we just explored is only an introduction. To develop an effective agentic RAG solution, it is crucial to address the 5 following questions:

• Q1: What freedom should the agent have? A common approach is to expose a few carefully selected tools, as in our case study, where the agent is limited to these tools for conducting its research. This simplifies control, testing, and auditing.

  However, it is also possible to give the agent broader access, including a shell and a file system. This way, the agent can execute scripts to search and analyze files, or even perform additional data processing autonomously.

  This model offers more power but also increases risks and makes the agent's behavior less predictable.

  For most RAG applications, it is advisable to start with selected tools and then consider broader access only if the task complexity justifies it.

• Q2: Should the agent only search for plain text? Most RAG projects start with plain text, such as PDFs, wiki pages, or manuals. This is a good foundation.

  However, in practice, it is often useful to derive a knowledge layer above the raw texts. These knowledge artifacts can include metadata, summaries, inter-document links, or even a complete knowledge graph.

  These artifacts help the agent navigate the corpus, while the raw texts remain the source of truth.

• Q3: Do we still need integrations? Agentic RAG does not necessarily mean abandoning integrations.

  Vector integrations remain an effective way to find semantically relevant texts, often outperforming simple keyword searches. In agentic RAG, retrieval becomes an "action" that the agent can undertake. This "action" can be powered by a retriever based on integrations, keywords, or a hybrid of both.

  Integrations can therefore still be useful, serving as support for the agent's search tool.

• Q4: Should one agent handle everything? The simplest agentic RAG setup involves a single agent performing searching, reading, and responding.

  However, as the task becomes more complex, it may be wise to divide the work among multiple agents, adopting a multi-agent strategy.

  You can distribute the work by role, for example, separating planning, retrieval, and writing functions. Each agent then focuses on a specific task, optimizing the overall process.

  You can also divide by source type, with each agent equipped with custom tools to focus on a particular type of source.

  Keep in mind that a multi-agent setup adds coordination complexity, with no guarantee of superior performance compared to a single-agent setup. Empirical testing is essential.

• Q5: Should we always use agentic RAG? Not necessarily.

  Just because agentic RAG is trending does not mean it should be adopted by default.

  While agentic RAG offers more flexibility, it comes at a cost, particularly in terms of latency, token costs, and less predictable agent behavior.

  It is wise to start with simple solutions and then add agentic loops only when the question truly requires iterative retrieval.