Retrieval-Augmented Generation (RAG) is the gold standard for grounding LLMs in proprietary enterprise data. Rather than fine-tuning a model on company documents, RAG retrieves relevant document chunks and injects them into the prompt window dynamically.
However, poorly designed RAG pipelines suffer from retrieval degradation, lost context, and contradictory answers. Here is how we architect production-grade RAG systems at Ascenta.
The Chunking Strategy: Context Over Length
Naive RAG pipelines chunk text strictly by fixed character length (e.g. 500 characters). This frequently cuts sentences in half or separates table headers from data rows.
The Hierarchical Parent-Document Pattern:
- Child Chunks (200 tokens): Used strictly for vector similarity search to maximize semantic match precision.
- Parent Document (1,000 tokens): When a child chunk matches a query, the system retrieves the entire parent section to provide complete situational context to the generator LLM.
Hybrid Search: Dense Vectors + Sparse BM25
Vector embeddings excel at semantic similarity, but fail on exact alphanumeric queries (e.g., serial numbers, client tax IDs, part numbers).
A production RAG engine must implement Hybrid Search:
- Dense Vectors (e.g., text-embedding-3-small): Captures conceptual meaning.
- Sparse Search (BM25 or PostgreSQL tsvector): Guarantees exact keyword retrieval.
- Reciprocal Rank Fusion (RRF): Merges both result sets before LLM synthesis.
Explore our CleanData AI case study to see our production RAG infrastructure in action, or consult our engineering team.