Enterprise Knowledge Base (01): Why RAG Is Just the Starting Point

Starting With a Real Failure

A technical team spent two months building a RAG knowledge base: PDF parsing, chunking, vectorizing, storing in Pinecone, wired to GPT-4 for answer generation. Post-launch Recall@5 = 0.87 — looks solid.

Then came the first real user question:

"We evaluated the Acme vendor before — what was the conclusion, and how does it compare to the options we're considering now?"

The RAG system had no answer.

Not because the technology was broken. Because this question has no complete answer in any single document — it requires finding the original evaluation report, understanding the conclusion, and doing comparative reasoning against current documents. Vector similarity matches local semantic relevance, not cross-document reasoning chains.

This failure wasn't a fluke. It exposed the fundamental gap between enterprise knowledge bases and ordinary document search.

Four Real Challenges in Enterprise Knowledge Bases

Before looking at technology options, get the problem statement right. Most enterprise knowledge base challenges aren't retrieval algorithm problems — they're data problems.

Challenge 1: Bad data quality

The real state of enterprise documents:

• PDFs are scanned images with 5-15% OCR error rates
• Word documents embed tables and charts that can't be parsed
• The same concept goes by three different names across departments
• Version chaos — the latest version isn't where it's most visible

These are pre-conditions for any RAG framework. Vectorizing a document full of OCR errors produces systematically biased retrieval results.

Challenge 2: Fragmented knowledge

Enterprise knowledge is scattered across silos:

• Design documents in Confluence
• Meeting notes in Slack or Teams
• Key decisions buried in email threads
• Implementation details in code comments
• Expertise that was never written down at all

RAG can index documents, but it can't automatically organize these fragments into structured knowledge. Questions like "how did we handle that requirement last time" are often unanswerable without explicit connections.

Challenge 3: Mixed modalities

Enterprise documents are far from plain text:

• Technical architecture diagrams (PNG/SVG)
• Data analysis reports with charts and Excel tables
• Product screenshots documenting operational steps
• Video meeting recordings

Classic RAG only indexes text — everything else is a black box.

Challenge 4: Knowledge decay

Document half-lives:

• API documentation: stale the moment a new version ships
• Process specifications: invalidated by every reorg
• Technical research reports: potentially obsolete after 6 months

A knowledge base that isn't updated returns outdated answers. But building a system that knows when to update the knowledge base is itself an engineering problem.

What RAG Can and Can't Do

Classic RAG works like this:

User question → vectorize → find similar chunks in vector store
→ feed similar chunks to LLM → LLM generates answer

It's fundamentally semantic relevance matching + context augmentation. It works well when:

✅ The answer lives in one chunk of text (single-hop query)
✅ Documents are high quality, chunking is sensible, terminology is consistent
✅ The user's phrasing is close to how the documents are written

It fails when:

❌ Answering requires reasoning across multiple documents (multi-hop)
❌ The question is about entity relationships ("what's the history between A Corp and B Corp?")
❌ The answer isn't in the knowledge base at all — but the LLM will fabricate one anyway
❌ Summarizing across a large document set ("what are all our authentication policies?")

These three failure modes drove the next generation of RAG evolution.

Four Generations of Knowledge Base Technology

Generation 1: Classic Vector RAG

Representative projects: QAnything, LightRAG (vector mode)
Core idea: vectorize → similarity matching → LLM generation
Key improvements: QAnything's BCEmbedding + Rerank two-stage pipeline improves precision; LightRAG adds keyword sparse retrieval for hybrid search

QAnything two-stage retrieval:
Stage 1: Vector retrieval (Top-K coarse recall)
Stage 2: BCEReranker precision ranking (returns Top-N)

Good fit: High-quality documents, mostly single-hop factual queries, cost-sensitive deployments.
Poor fit: Complex reasoning, relationship queries, large-scale multi-hop problems.

Generation 2: Graph-Augmented RAG

Representative projects: GraphRAG (Microsoft), HippoRAG
Core idea: Alongside vector indexing, build an entity relationship graph and use graph traversal to answer relational questions

GraphRAG's core mechanism is community detection: extract entities (people, organizations, concepts) from documents into a knowledge graph, use the Leiden algorithm to find natural clusters (communities), and route different query types through different paths:

GraphRAG query modes:
  Global search → community summaries → global questions ("what is this corpus about overall")
  Local search  → entity neighborhoods → local questions ("what are the details about X")

HippoRAG takes a different approach, closer to human memory: modeled after the hippocampus associative memory mechanism, storing key concepts as mutually-activating nodes. Multi-hop reasoning propagates through node activation rather than re-running search.

HippoRAG 2 (arXiv:2502.14802) shows significant improvements over classic RAG on multi-hop benchmarks (MuSiQue, 2Wiki, HotpotQA) while consuming substantially fewer indexing resources than GraphRAG.

Good fit: Dense entity relationships, frequent multi-hop reasoning, questions that need "global awareness."
Poor fit: Knowledge graph construction is expensive and slow to update — high cost for high-velocity content.

Generation 3: Hypergraph RAG

Representative project: HyperGraphRAG (NeurIPS 2025)
Core idea: Traditional knowledge graphs model binary relations (A → B). Hypergraphs can represent n-ary relations (A, B, and C all participating in the same event/concept simultaneously)

Normal knowledge graph: Alice --[participated_in]--> Project X, Bob --[participated_in]--> Project X
Hypergraph: {Alice, Bob, Carol} --[jointly_delivered]--> Project X

The advantage: directly encoding multi-party relationships, rather than decomposing "three people did this together" into multiple binary relations and then reassembling them. For queries involving complex multi-party collaboration or multi-constraint reasoning, hypergraphs deliver more complete context.

Good fit: High relational complexity, dense multi-party events and decisions, contexts where academic rigor matters.
Status: NeurIPS 2025 paper — primarily research-stage, limited production deployments.

Generation 4: Agent-native Knowledge Systems

Representative project: GBrain (Garry Tan, current CEO of Y Combinator — his personal production system)
Core idea: The knowledge base is no longer a passive retrieval tool but a continuously learning, self-correcting, actively synthesizing "brain"

GBrain's design principles:

Synthesis, not retrieval: The response isn't "here are 10 relevant chunks" — it's "here's the synthesized answer from these sources, and here's what the brain doesn't know yet." The gap analysis is what changes how you use the system.

Self-wiring knowledge graph: Every page write automatically extracts entity references and creates typed edges (attended, works_at, invested_in, founded, advises) — zero LLM calls per write. Queries like "who works at Acme AI?" are answered through graph traversal, not vector search.

Continuously running knowledge agents: 66 cron jobs run continuously, ingesting new content (meetings, emails, tweets), enriching entity data, fixing citation errors, consolidating duplicate memories. In Garry Tan's words:

"My agent works while I sleep. I wake up smarter than when I went to bed — and so will you."

Real production data: 146,646 pages, P@5 = 49.1%, R@5 = 97.9% — +31.4 points P@5 over vector-only RAG.

Good fit: Personal or team knowledge management, scenarios requiring continuous learning and self-updating, long-term memory layer for AI agents.
Limitations: Requires persistent background processes, higher infrastructure requirements — better suited as a team platform than a lightweight tool.

The Complete Technology Map

Enterprise knowledge base technology landscape
│
├── Retrieval-Augmented (RAG variants)
│   ├── Dense Vector Retrieval
│   │   └── QAnything, LightRAG (vector mode)
│   ├── Sparse + Dense Hybrid (BM25 + Dense)
│   │   └── QAnything v2, most enterprise deployments
│   └── Rerank-Augmented
│       └── QAnything BCEReranker
│
├── Graph-Augmented
│   ├── Entity graph + community detection
│   │   └── GraphRAG (Microsoft)
│   ├── Neural associative graph (hippocampus model)
│   │   └── HippoRAG
│   ├── Hypergraph (n-ary relations)
│   │   └── HyperGraphRAG (NeurIPS 2025)
│   └── Lightweight graph + vector hybrid
│       └── LightRAG (graph mode)
│
├── Multimodal Knowledge
│   └── RAG-Anything (text + image + table + audio + video)
│
└── Agent-native Knowledge
    └── GBrain (synthesis, graph traversal, gap analysis, continuous learning)

What This Series Does

With the technology map in place, the selection question becomes: which generation's capability covers your business scenario?

This series isn't a survey paper — it's engineering benchmarks:

• Build the unified test set first (done — see Series 00)
• Benchmark each major framework against the same 89 questions, reporting RAGAS four metrics + boundary refusal rate + P90 latency
• End with a cross-framework comparison and selection decision framework

Part 1 benchmark plan (7 articles):

Part 2 methodology (5 articles): Knowledge distillation, selection, data governance, deployment architecture, evaluation tuning.
Part 3 case studies (as available): Real project deployment processes.

One expectation to calibrate upfront:

This series won't tell you "XXX is the best RAG framework," because "best" depends on your scenario, data quality, and team operations capability. What this series gives you is: understand where each framework is strong, where it's weak, and which category your scenario falls into.

Follow me at dongqi.dev for more LLM engineering content.

At PrimeSkills, we help engineering teams select and deploy enterprise knowledge base systems — from evaluation frameworks to production deployment. If you're building a knowledge base for your team, reach out.

primeskills.dev