Large language models (LLMs) are highly capable, but they are not reliable on their own in the enterprise world. Language models tend to hallucinate, and they are not only deprived of new or proprietary information inputs but are also inefficient in areas such as governance, traceability, and expenditure management. Retrieval-Augmented Generation (RAG) came to the fore as an effective approach to anchor model responses to external knowledge sources. There is a tendency among various teams to consider RAG as a single pattern of implementation.
Something I quickly discovered is that RAG is not one architecture, but several. Indeed, a system that is adequate for a simple “search assistance” scenario is not sufficient for scenarios involving multi-step reasoning, tool execution, or multiple data sources. It is important to treat different RAG architectures differently in order to avoid fragile or overly engineered systems that are difficult to run in production environments.