RAG in Production: Architecture, Hybrid Retrieval, and Continuous Evaluation

Putting a RAG system online is not just about querying a vector index. An architecture separates offline ingestion from online service and maintains a feedback loop, while a .NET and Blazor example demonstrates how to structure outputs and display citations to validate the interface. Beyond the prototype, retrieval combines keywords and vectors, graphs, semantic re-referencing, and agentic planning. This is accompanied by continuous evaluation with metrics dedicated to retrieval, robustness, and relevance.

The architecture isolates ingestion and adds a feedback loop

A production-oriented architecture separates offline ingestion from online retrieval and generation, and retains a feedback loop to evolve the system. An implementation example supports this approach with a .NET and Blazor stack. In this configuration, structured outputs and citations accompany each response to facilitate rendering and validation on the user interface side.

Retrieval goes beyond a single vector query

A single vector query is not sufficient to cover the variety of needs. The combination of a hybrid search mixing keywords and vectors, including reciprocal rank fusion (RRF), enhances relevance. For relationship-rich questions, adding graph-based retrieval is essential, complemented by semantic re-referencing of results. When the demand requires multiple steps, agentic planning of queries takes over from fixed schemas. These techniques sit alongside other practical models such as context windows or project/session-based retrieval.

From POC to production: ingestion pipeline and dedicated measurement

The classic prototype—ingestion, embeddings, vector index, chunk retrieval, and model call—demonstrates the idea but shows its limits as the corpus grows, queries become more complex, or responses aggregate multiple sources. In a production environment, ensuring the reliability of context becomes the priority, which requires a robust ingestion pipeline, including parsing, segmentation, metadata management, and rigorous vectorization. This constraint implies ongoing evaluation tracking and appropriate observability, with separate indicators for retrieval performance, robustness, and response relevance.