JDN Innovates with a RAG Chatbot Using OpenAI's AgentKit

The JDN has embarked on developing an innovative chatbot using OpenAI's AgentKit framework, an initiative aimed at making the design of generative AI chatbots accessible to a wider audience. Launched in October 2025, AgentKit is an SDK that allows the creation of AI systems with minimal coding, thanks to its add-ons. Among these modules, Agent Builder offers a visual canvas for designing multi-step workflows, while ChatKit provides easily integrable chatbots on a web page. The JDN was captivated by the idea of using this tool to guide its readers in choosing a generative AI model, relying on a comparator listing over 155 models.

The Principle of ChatKit

ChatKit is based on a three-step architecture. The first step involves designing an agentic workflow via Agent Builder, OpenAI's visual interface that allows for configuring the chatbot's business logic based on the N8N principle. Next, a backend server is set up to create ChatKit sessions and exchange authentication tokens with the OpenAI API. Finally, the chat widget is integrated into the website using a few lines of JavaScript and React. This architecture enables the deployment of a conversational chatbot without having to develop the user interface from scratch, as OpenAI manages the hosting and scalability of the backend.

On paper, the architecture seems clear: the client-side chat widget communicates with OpenAI's ChatKit servers, which handle inference, file storage, and conversation history. In between, an intermediary server secures exchanges by creating sessions and managing authentication.

Creating the Workflow in Agent Builder

To create the interactive chatbot, the JDN begins by designing the agent's workflow in Agent Builder. The specifications are relatively simple: develop a chatbot capable of advising readers on the best model suited to their specific needs, referencing the database of models. This database includes parameters such as name, input modality, output modality, cost (estimated on a scale from ???? to ????????????), open-source or proprietary nature, fine-tuning capability, and scores on key benchmarks like MMLU, LiveCodeBench, SWE Bench Verified, and AIME 2025.

Agent Builder operates with modules. The JDN starts by adding a Guardrails module that filters user requests: moderating content, reducing the risk of jailbreak, detecting NSFW texts, and blocking prompt injections, among others. Once the guardrails are passed, three outcomes are possible: either the request is validated, it fails, or an error occurs. In the latter two cases, the chatbot informs the user that it cannot process the request. When the guardrails are successfully navigated, the request enters a classification module (Classify) that determines if the user's question pertains to an AI model. If so, it is directed to the main AI agent, equipped with RAG and the database, to formulate an appropriate recommendation. Otherwise, the user is politely informed that the chatbot is dedicated to advising on AI models and is invited to rephrase their question.

RAG or Data in the System Prompt?

For the logic of the AI agent, the JDN questioned the use of RAG or the direct addition of data from the model guide table into the system prompt. Both methods have their advantages and disadvantages. Injecting the table data into the prompt is the simplest approach. However, this method quickly shows its limits as the volume of data increases: higher costs and latency, more complex maintenance during guide updates, and, most importantly, an increased risk of inaccurate responses when the model must reason over several hundred lines simultaneously. This approach could have seen its costs partially reduced through the prompt caching mechanism, which allows for the shared processing of an identical system prompt across multiple requests, but it does not resolve the reliability limits.

RAG adds a slight complexity, but it is quite manageable: the table needs to be vectorized, and the add-on module must be added in Agent Builder. For each update, the file must be re-vectorized with the updated data. The effort is worthwhile to maximize the model's efficiency. The JDN thus chose to adopt a RAG approach. To do this, they accessed OpenAI's vector storage tool, the Vector Store. Currently, the tool does not allow for vectorizing tabular data in CSV or XLSX (Excel) format: it only handles unstructured text files. The JDN opted to convert the table into JSON (using a prompt and Claude) to obtain a .txt file filled with JSON. The JSON format was chosen because it maintains an explicit and readable structure of the table fields (model, price, modalities, etc.) while remaining compatible with the vectorization of unstructured text files.

Back in Agent Builder, an AI agent module is created to allow the model to respond based on the user request. The module uses File Search by providing the ID of the previously created vector. The model will then respond using the data from the model guide file.

For the model choice, the JDN decided to go with GPT-5.3-nano instead of GPT-4o for cost and future scalability reasons. The model is one of the cheapest and should be supported by OpenAI for a few more months. It has minimal reasoning capabilities that will allow the JDN, if desired later, to use web search quite effectively to respond to user requests with greater precision (for example, citing JDN articles as sources). The "low" reasoning level is activated, the minimum required to use RAG with this model.

The prompt is crafted based on a manually written base, then improved by Claude for better formatting. The prompt precisely defines the editorial identity of the agent, its advisory mission, its scope of expertise, its conversational style rules, and the methodology to follow for qualifying a need before any recommendation. It is worth noting that during the various iterations, each micro-change in the prompt impacted all instructions. Benchmarking and retesting are necessary with each new version.

Structure of Our Prompt:

• Role/Expertise
• Conversational Style
• Main Objective
• Differentiating Added Value
• Permissions (Scope)
• Privacy Rules

[STYLE RULES]

• Mandatory format (prose/list/other)
• Formal prohibitions
• Token/word limit

[DATA USAGE]

• Sources (RAG/web/internal)
• Specific rating (stars, scores)
• Mandatory verification

[FUNDAMENTAL RULE]

• Critical workflow (e.g., questions before recommendation)
• Exceptions to the workflow

[CONVERSATIONAL APPROACH]

• Adaptation logic (vague→precise)
• Topics to explore
• Number of questions/interactions
• Decision hierarchy (ordered criteria)
• Triage logic
• Result presentation format
• Proposal after main action
• Bad vs Good (2-3 concrete cases)
• Missing data
• Extraction attempts

For security reasons, the current version of the prompt in production cannot be disclosed.

Customizing the Chatbot's Appearance

Once the business logic of the bot is configured in Agent Builder, it's time to customize the chatbot's appearance. OpenAI offers a very user-friendly interface in its ChatKit playground. It is possible to configure the theme in night or day mode, the general color of the send button, the font and font size used, the main tagline, the starting phrases (the examples), the option to upload files or not, and many other parameters.

Once the graphic features are validated, the code can be retrieved by clicking on the code tag. The entire interface is thus pre-configured in the code. The GUI and business logic are now set up, and we then think about deployment, the most technical part for non-experts.

Deployment on Development Server

As previously explained, OpenAI's ChatKit SDK cannot operate solely in the browser. The reason is simple: your OpenAI API key, which is linked to your billing, must never appear in client-side code. Otherwise, any curious user could open their browser's developer tools, retrieve this key, and use it at your expense. To prevent this, OpenAI enforces a security mechanism: you must have an intermediary server that holds your API key and, for each new conversation, contacts OpenAI to create a session. OpenAI then returns a temporary token, valid only for that exchange, which the server transmits to the browser. The ChatKit widget uses this ephemeral token to function, and your main API key never circulates on the client side.

To set up this architecture, the JDN ordered a VPS server from OVH, the lowest price option, for a few euros per month.