GraphRAG: How AI Answers Questions Hidden Across Many Documents

Standard Retrieval Augmented Generation (RAG) operates through a straightforward pipeline where documents are split into token chunks and transformed into vectors via an embedding model. These vectors are stored in a vector index to facilitate proximity-based semantic search against incoming query vectors. During query execution, the closest document chunks are retrieved and included in the language model prompt to generate grounded answers. The entire workflow relies on the core assumption that queries and their corresponding answers share lexical or semantic resemblance.

The assumption that questions resemble their answers holds for local queries but breaks down for global queries that require reasoning across an entire corpus. For global queries, standard vector retrieval surfaces documents with superficial vocabulary overlap rather than synthesizing information distributed across hundreds of files. Microsoft's evaluations show that increasing context windows up to 64,000 tokens does not bridge this gap in comprehensiveness, diversity, and source quality compared to GraphRAG. What is commonly diagnosed as model hallucination is often actually a retrieval issue where fluent text is generated from irrelevant retrieved context.

Knowledge graphs enhance retrieval and answer quality by modeling relationships between entities across documents rather than treating text chunks independently. By aggregating extracted entities and typed relations from multiple sources, knowledge graphs create multi-hop paths connecting disparate information that no single document holds. Real-world implementations, such as LinkedIn's support ticket system presented at SIGIR 2024, demonstrate significant production gains including higher retrieval rank and reduced resolution times. Modern GraphRAG architectures typically combine lexical graphs of document structures with entity graphs of domain concepts to query across both.

Constructing a knowledge graph from raw documents requires a multi-phase pipeline where the majority of computation and expense is concentrated in graph extraction and merging. Microsoft's indexing workflow involves text chunking, LLM extraction of entities and relationships, merging and summarizing redundant descriptions, community clustering, and embedding into a vector store. The two LLM passes across a corpus represent approximately 75 percent of total indexing costs. To mitigate this expense, alternatives like FastGraphRAG substitute LLMs with traditional NLP techniques, though this trades cost efficiency for increased noise.

GraphRAG enables answering whole-collection questions by executing hierarchical Leiden clustering across an entity graph to partition it into multi-level communities. For each community across every level, a language model pre-generates a community report detailing key entities, relationships, and claims during the indexing phase. Choosing the appropriate hierarchy level determines the balance between answer thoroughness and operational costs in time and tokens.

GraphRAG supports multiple retrieval query modes tailored to different question types across a knowledge graph and report hierarchy. Local search identifies specific entry-point entities and expands along five parallel directions—text units, community reports, neighbors, relationships, and covariates—before ranking and packing them into a context window. Global search operates over community reports using a map-reduce process to answer broad, corpus-wide questions without relying on specific entity lookups. Additionally, GraphRAG provides DRIFT search, which combines community reports and follow-up local searches, as well as a basic search mode using standard top-k vector retrieval.

GraphRAG significantly redistributes retrieval-augmented generation costs compared to standard RAG, incurring heavy indexing expenses due to multiple language model passes and community summarization. Updating the index with new documents creates an ongoing operational commitment as hierarchies shift and material is re-summarized. Microsoft developed LazyGraphRAG to address this by replacing LLM-based indexing with traditional NLP, which reduces indexing cost to match vector RAG and lowers query cost by over 700x while maintaining global-query quality. Nevertheless, full GraphRAG remains valuable for its human-readable community reports and excels in comprehensiveness and diversity, though its faithfulness matches standard RAG rather than eliminating hallucinations.

Agentic RAG addresses the limitations of fixed retrieval pipelines by using a language model to dynamically classify queries and select tailored retrieval strategies, such as vector search, SQL, or web search. LlamaIndex illustrates a two-layer routing design where a composite retriever selects an index and an auto-routed mode chooses the retrieval method within that index. While this flexibility improves accuracy across diverse questions, it introduces latency, higher per-query costs, and debugging hurdles from routing errors. The paradigm marks an evolutionary step beyond basic RAG, advanced RAG, and GraphRAG.

GraphRAG addresses the shortcomings of standard vector retrieval for global queries by extracting a knowledge graph and generating hierarchical community summaries across a corpus. Graph extraction is computationally expensive, driving around 75 percent of total indexing costs, which has motivated lower-cost alternatives like LazyGraphRAG. While local search expands from matched graph entities and vector RAG remains better suited for local queries, global search performs map-reduce operations over pre-generated community reports. Consequently, agentic retrieval frameworks can dynamically pick between vector and graph-based strategies depending on query needs.