Retrieval-Augmented Generation as an Epistemic Architecture: A Theoretical Account of Grounding, Provenance and Hallucination Reduction

Abstract

Retrieval-augmented generation (RAG) is widely presented as the remedy for hallucination in large language models: by retrieving documents and conditioning generation on them, the system is said to answer from evidence rather than from memory. This paper offers a theoretical account of what RAG actually does to the epistemic status of a model's output, and argues that the remedy framing is misleading in a specific and consequential way. Drawing on the survey literature on hallucination and on RAG, on the evaluation literature, and on empirical studies of reference accuracy in retrieval-backed tools, the paper models a RAG system as a provenance chain with five links, query, retrieval, selection, generation and attribution, and shows that each link can fail independently. RAG does not eliminate the mechanism that produces hallucination, which is generation without regard to truth; it relocates error from the parametric fabrication of content to four other sites: retrieval error, knowledge conflict in which the model overrides retrieved evidence, misattribution in which a real source is cited for a claim it does not support, and corpus contamination in which the retrieved evidence is itself false or adversarial. The paper distinguishes grounding from truth and citation from support, derives six propositions about when RAG outputs can be trusted, and specifies the architectural properties, verbatim provenance, abstention on retrieval failure, conflict handling, attribution-fidelity evaluation and corpus integrity, under which a RAG system functions as an epistemic architecture rather than as a more persuasive generator. Implications for deployment with local corpora in low-resource settings and a research agenda close the paper.