Why 90% of enterprise RAG chatbots fail—and the hybrid pgvector architecture that fixes them

Almost every company we talk to has tried building an internal "Chat with your docs" prototype. Someone uploads a hundred PDFs into a hosted vector index, wires up an LLM prompt, and demos it on a clean sample question. Everyone claps.
Then real employees or paying customers try it—and the bot confidently invents a refund clause from an expired 2023 memo, misses an exact invoice number, or leaks a document from another client's folder.
The gap between a weekend vector demo and a trusted Retrieval-Augmented Generation (RAG) system is not the choice of LLM. It is retrieval architecture, data hygiene, and evaluation.
The 4 reasons naive RAG breaks in production
- Pure semantic search misses exact identifiers: Dense vector embeddings are great at capturing meaning ("how do I cancel?") and terrible at exact matches (
SKU-4920-X, error codeERR_502, or a specific contract clause number). - Blind character-count chunking destroys context: Splitting a PDF every 500 tokens slices tables in half and separates column headers from the numbers underneath them.
- Zero tenant or role filtering: A separate vector database often strips away the relational permissions stored in your primary SQL database—creating an instant security risk.
- No automated evaluation harness: Teams tweak a prompt or swap an embedding model without measuring whether recall or faithfulness actually improved or regressed.
The production RAG blueprint we build
Across platforms like and custom enterprise knowledge engines, our and squads rely on four proven architectural pillars.
Have a question about this article — or your own project?
Ask us anything about the approach above, or tell us what you're building. You'll hear back from an engineer, not a sales script.