RAG for Industrial Documentation: What Actually Works

Why industrial docs break standard RAG

The standard RAG tutorial has you chunking a PDF into 512-token blocks, embedding them, sticking them in a vector store, and calling it done. That works fine for a company wiki or a knowledge base full of prose. It falls apart on industrial documentation — and the failures are specific enough to be worth understanding before you write a line of code.

Here's what makes manufacturing documents different:

• Dense tables and spec sheets. A pump specification PDF is 60% tables — flow rates, pressure ratings, torque values, part numbers. Naive chunking splits tables mid-row, destroys their structure, and produces embeddings that represent garbled nonsense.
• Procedural sequences that can't be reordered. Lockout/tagout procedures, startup sequences, calibration steps — these are ordered lists where step 7 is meaningless without steps 1-6. Retrieving step 7 in isolation and asking an LLM to answer a question about it is actively dangerous.
• Cross-references everywhere. "Refer to Section 4.2.3 for torque specifications." A chunk of text with that sentence is useless without the context it's pointing to. Standard RAG doesn't follow references — it retrieves the chunk and stops.
• Safety-critical content. An LLM that confidently answers a question about electrical isolation procedures based on a poorly-retrieved chunk, without surfacing the relevant warnings, is a liability. The failure mode isn't "wrong answer" — it's "wrong answer someone acts on."
• Data sovereignty requirements. Many manufacturers will not send their process documentation to a third-party API. Their SOPs contain trade secrets, proprietary formulations, or competitive IP. The default assumption has to be on-premises or private cloud.

I built two RAG systems against this problem set — The Guide (document intelligence for operators) and QuoteForge (semantic search over historical RFPs for estimators). Different domains, same core challenges. Here's what I learned.

The chunking problem

Chunk size and strategy is the most underrated decision in a RAG system. Get it wrong and no amount of prompt engineering rescues you.

What doesn't work

Fixed-size token chunking — splitting every 512 tokens with 50-token overlap — is the tutorial default. It's fine for prose-heavy documents where ideas span paragraphs. For technical manuals it destroys structure constantly: it splits tables, breaks numbered lists mid-step, and separates warnings from the procedures they apply to.

What does work: structure-aware chunking

Industrial documents have structure. Use it. The approach I landed on:

1. Parse the document structure first. Extract headings, sections, tables, and lists as distinct elements before chunking. A PDF parser that gives you raw text loses this — use one that preserves structure (PyMuPDF for PDFs, python-docx for Word files).
2. Chunk at section boundaries, not token counts. A section headed "3.4 Electrical Isolation Procedure" is a natural unit. Keep it together, even if it's 800 tokens.
3. Keep tables intact. A table that gets split is worse than no table at all. If a table is too large, chunk it by row groups — but never split mid-row.
4. Prepend context to every chunk. The retriever doesn't know what document a chunk came from. Prepend the document title, section heading, and page number to every chunk before embedding. This sounds obvious; most implementations skip it.

import fitz  # PyMuPDF
from dataclasses import dataclass

@dataclass
class Chunk:
    content: str
    source_doc: str
    section: str
    page: int
    chunk_type: str   # "prose", "table", "procedure"

def chunk_document(pdf_path: str, doc_name: str) -> list[Chunk]:
    doc = fitz.open(pdf_path)
    chunks = []
    current_section = "Introduction"

    for page_num, page in enumerate(doc, start=1):
        blocks = page.get_text("dict")["blocks"]

        for block in blocks:
            if block["type"] == 0:  # text block
                text = " ".join(
                    span["text"]
                    for line in block["lines"]
                    for span in line["spans"]
                ).strip()

                if not text:
                    continue

                # Detect section headings by font size
                avg_size = sum(
                    span["size"]
                    for line in block["lines"]
                    for span in line["spans"]
                ) / max(1, sum(len(line["spans"]) for line in block["lines"]))

                if avg_size > 13:  # heading threshold — tune per document
                    current_section = text
                    continue

                # Build contextual prefix for every chunk
                context_prefix = (
                    f"Document: {doc_name}\n"
                    f"Section: {current_section}\n"
                    f"Page: {page_num}\n\n"
                )

                chunks.append(Chunk(
                    content=context_prefix + text,
                    source_doc=doc_name,
                    section=current_section,
                    page=page_num,
                    chunk_type="prose",
                ))

    return chunks

Self-hosted embeddings with Hugging Face TEI

You have two options for embeddings: a hosted API (OpenAI, Cohere) or a self-hosted model. For most projects the hosted API is fine. For manufacturing customers, it's often a non-starter — you're sending document content to a third-party endpoint, and that document might contain proprietary process information.

Hugging Face Text Embeddings Inference (TEI) is the right answer here. It's a high-performance inference server for embedding models, ships as a Docker image, and runs on CPU or GPU. You get consistent embeddings that don't change when a provider silently upgrades their model, and nothing leaves your infrastructure.

services:
  embeddings:
    image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
    ports:
      - "8081:80"
    volumes:
      - ./models:/data
    command: --model-id BAAI/bge-base-en-v1.5 --port 80
    environment:
      - HUGGING_FACE_HUB_TOKEN=${HF_TOKEN}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:80/health"]
      interval: 30s
      timeout: 10s
      retries: 3

I use BAAI/bge-base-en-v1.5 — solid performance, 768 dimensions, runs comfortably on CPU for document indexing workloads. For real-time retrieval on CPU you'll get ~50ms per query, which is fine. If you need lower latency, switch to bge-small-en-v1.5 (384 dims, faster) or add a GPU.

import httpx
from app.config import settings

EMBEDDING_DIM = 768  # BAAI/bge-base-en-v1.5

async def embed_texts(texts: list[str]) -> list[list[float]]:
    """
    Call the TEI service to embed a batch of texts.
    Returns a list of embedding vectors.
    """
    async with httpx.AsyncClient(timeout=30.0) as client:
        response = await client.post(
            f"{settings.EMBEDDING_SERVICE_URL}/embed",
            json={"inputs": texts, "normalize": True},
        )
        response.raise_for_status()
        return response.json()

async def embed_query(query: str) -> list[float]:
    """Embed a single search query."""
    vectors = await embed_texts([query])
    return vectors[0]

Hybrid search: semantic + keyword

Pure semantic search has a failure mode that bites hard in technical documentation: exact-match recall for part numbers, model codes, and technical terms.

If a technician asks "what's the torque spec for the M12 bolt on the 3500-XR pump?" — semantic search will retrieve chunks about torque in general, bolts in general, pumps in general. The model number "3500-XR" is a token that carries no semantic meaning. Keyword search (BM25) finds it instantly.

The solution is hybrid search: run both, then combine the rankings.

I use Reciprocal Rank Fusion (RRF) to merge the two result lists. It's simple, parameter-free, and consistently outperforms weighted score combinations in practice.

from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession

RRF_K = 60  # standard RRF constant

async def hybrid_search(
    query: str,
    query_embedding: list[float],
    db: AsyncSession,
    top_k: int = 8,
) -> list[dict]:
    """
    Hybrid search: semantic vector search + BM25 keyword search,
    merged with Reciprocal Rank Fusion.
    """
    sql = text("""
        WITH semantic AS (
            SELECT id, content, source_doc, section, page,
                   ROW_NUMBER() OVER (ORDER BY embedding <-> :embedding) AS rank
            FROM document_chunks
            ORDER BY embedding <-> :embedding
            LIMIT 20
        ),
        keyword AS (
            SELECT id, content, source_doc, section, page,
                   ROW_NUMBER() OVER (
                       ORDER BY ts_rank(search_vector, plainto_tsquery('english', :query)) DESC
                   ) AS rank
            FROM document_chunks
            WHERE search_vector @@ plainto_tsquery('english', :query)
            ORDER BY rank
            LIMIT 20
        ),
        rrf AS (
            SELECT
                COALESCE(s.id, k.id) AS id,
                COALESCE(s.content, k.content) AS content,
                COALESCE(s.source_doc, k.source_doc) AS source_doc,
                COALESCE(s.section, k.section) AS section,
                COALESCE(s.page, k.page) AS page,
                COALESCE(1.0 / (:k + s.rank), 0) +
                COALESCE(1.0 / (:k + k.rank), 0) AS rrf_score
            FROM semantic s
            FULL OUTER JOIN keyword k ON s.id = k.id
        )
        SELECT * FROM rrf ORDER BY rrf_score DESC LIMIT :top_k
    """)

    result = await db.execute(sql, {
        "embedding": query_embedding,
        "query": query,
        "k": RRF_K,
        "top_k": top_k,
    })

    return [dict(row._mapping) for row in result.fetchall()]

pgvector in practice

pgvector turns PostgreSQL into a vector database. No separate Pinecone account, no Weaviate instance — your embeddings live in the same database as everything else, with full SQL expressiveness for filtering and joining.

-- Enable extensions
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;  -- needed for GIN text indexing

-- Document chunks table
CREATE TABLE document_chunks (
    id          SERIAL PRIMARY KEY,
    source_doc  TEXT NOT NULL,
    section     TEXT,
    page        INTEGER,
    chunk_type  TEXT DEFAULT 'prose',
    content     TEXT NOT NULL,
    embedding   vector(768),          -- BAAI/bge-base-en-v1.5 dimensions
    search_vector tsvector            -- for BM25 keyword search
);

-- Vector index (IVFFlat — good for up to a few million chunks)
CREATE INDEX idx_chunks_embedding
    ON document_chunks
    USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 100);

-- Full-text search index
CREATE INDEX idx_chunks_fts
    ON document_chunks USING GIN (search_vector);

-- Keep search_vector updated automatically
CREATE TRIGGER update_search_vector
    BEFORE INSERT OR UPDATE ON document_chunks
    FOR EACH ROW EXECUTE FUNCTION
    tsvector_update_trigger(search_vector, 'pg_catalog.english', content);

Citation-aware retrieval

This is the feature that actually makes the system usable in a manufacturing context. An LLM that answers "what's the max operating pressure?" without telling you which document and which section that answer came from is not a tool an experienced engineer will trust.

Citation-aware retrieval means: every retrieved chunk carries its provenance, and you instruct the LLM to cite its sources in the answer.

import anthropic
from app.config import settings

SYSTEM_PROMPT = """You are a technical documentation assistant for manufacturing operations.

When answering questions:
1. Base your answer ONLY on the provided document excerpts
2. Cite your sources inline using [Doc: {source}, Section: {section}, p.{page}] format
3. If the excerpts don't contain enough information to answer, say so explicitly
4. If the question involves safety procedures, always include relevant warnings from the excerpts
5. Do not invent specifications, part numbers, or procedures

Accuracy matters more than completeness. A partial answer with clear citations is better
than a confident answer without them."""

async def answer_question(
    query: str,
    chunks: list[dict],
    client: anthropic.AsyncAnthropic,
) -> dict:
    # Format retrieved chunks with their provenance
    context_parts = []
    for i, chunk in enumerate(chunks, start=1):
        context_parts.append(
            f"[Excerpt {i}]\n"
            f"Source: {chunk['source_doc']}, "
            f"Section: {chunk['section']}, "
            f"Page: {chunk['page']}\n"
            f"{chunk['content']}"
        )
    context = "\n\n---\n\n".join(context_parts)

    message = await client.messages.create(
        model=settings.LLM_MODEL,
        max_tokens=512,
        system=SYSTEM_PROMPT,
        messages=[{
            "role": "user",
            "content": (
                f"Question: {query}\n\n"
                f"Document excerpts:\n\n{context}"
            )
        }]
    )

    answer_text = message.content[0].text

    # Extract which sources were actually cited
    cited_sources = [
        {"doc": c["source_doc"], "section": c["section"], "page": c["page"]}
        for c in chunks
        if c["source_doc"] in answer_text
    ]

    return {
        "answer": answer_text,
        "sources": cited_sources,
        "chunks_retrieved": len(chunks),
    }

The key instruction in the system prompt is "if the excerpts don't contain enough information, say so explicitly." Without this, LLMs hallucinate plausible-sounding specs when the retrieved context is thin. For a knowledge base about pump pressures and torque values, a hallucinated spec is worse than no answer at all.

Safety disclaimers

This is the part most RAG tutorials skip entirely because it doesn't come up for SaaS documentation. It comes up constantly for manufacturing.

The pattern I use: tag certain document sections as safety-critical during ingestion (lockout/tagout procedures, electrical isolation, chemical handling). At retrieval time, if any retrieved chunk is tagged safety-critical, or if the query matches a safety-related pattern, append the relevant safety sections to the context automatically and instruct the LLM to surface them.

import re

SAFETY_QUERY_PATTERNS = [
    r'\b(lockout|tagout|loto)\b',
    r'\b(electrical|voltage|energize|de-energize)\b',
    r'\b(hazardous|chemical|flammable|explosive)\b',
    r'\b(confined space|oxygen deficient)\b',
    r'\bhow to (disconnect|remove|bypass|override)\b',
]

SAFETY_DISCLAIMER = (
    "\n\n⚠️ SAFETY NOTICE: This information relates to safety-critical procedures. "
    "Always follow your facility's lockout/tagout program and applicable regulations "
    "(OSHA 29 CFR 1910.147) before performing any maintenance. "
    "This system provides reference information only — consult qualified personnel "
    "before proceeding with any safety-critical work."
)

def requires_safety_context(query: str, chunks: list[dict]) -> bool:
    """Returns True if the query or retrieved chunks touch safety-critical content."""
    query_lower = query.lower()
    for pattern in SAFETY_QUERY_PATTERNS:
        if re.search(pattern, query_lower, re.IGNORECASE):
            return True
    return any(c.get("chunk_type") == "safety" for c in chunks)

def add_safety_disclaimer(answer: str, query: str, chunks: list[dict]) -> str:
    if requires_safety_context(query, chunks):
        return answer + SAFETY_DISCLAIMER
    return answer

BYOM architecture

BYOM — Bring Your Own Model — is the architectural pattern that makes this deployable to actual manufacturing customers. The idea: your application is LLM-agnostic. The customer configures which provider to use; you support at minimum Azure OpenAI and AWS Bedrock, because enterprise manufacturing customers are on one of those two clouds.

This matters for two reasons:

1. Data residency. Azure OpenAI with the data privacy addendum keeps data in the customer's Azure tenant. AWS Bedrock keeps it in their VPC. Neither routes through a shared API endpoint.
2. Procurement reality. Many large manufacturers already have enterprise agreements with Azure or AWS. Letting them use that existing spend removes a procurement blocker.

from abc import ABC, abstractmethod
from app.config import settings

class LLMClient(ABC):
    @abstractmethod
    async def complete(self, system: str, user: str, max_tokens: int = 512) -> str:
        ...

class AzureOpenAIClient(LLMClient):
    def __init__(self):
        from openai import AsyncAzureOpenAI
        self.client = AsyncAzureOpenAI(
            azure_endpoint=settings.AZURE_OPENAI_ENDPOINT,
            api_key=settings.AZURE_OPENAI_API_KEY,
            api_version="2024-02-01",
        )

    async def complete(self, system: str, user: str, max_tokens: int = 512) -> str:
        response = await self.client.chat.completions.create(
            model=settings.AZURE_OPENAI_DEPLOYMENT,
            messages=[
                {"role": "system", "content": system},
                {"role": "user", "content": user},
            ],
            max_tokens=max_tokens,
        )
        return response.choices[0].message.content

class BedrockClient(LLMClient):
    def __init__(self):
        import boto3, json
        self.client = boto3.client("bedrock-runtime", region_name=settings.AWS_REGION)
        self.model_id = settings.BEDROCK_MODEL_ID

    async def complete(self, system: str, user: str, max_tokens: int = 512) -> str:
        import asyncio, json
        body = json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": max_tokens,
            "system": system,
            "messages": [{"role": "user", "content": user}],
        })
        loop = asyncio.get_event_loop()
        response = await loop.run_in_executor(
            None,
            lambda: self.client.invoke_model(modelId=self.model_id, body=body)
        )
        return json.loads(response["body"].read())["content"][0]["text"]

def get_llm_client() -> LLMClient:
    """Factory — reads LLM_PROVIDER from environment."""
    provider = settings.LLM_PROVIDER.lower()
    if provider == "azure":
        return AzureOpenAIClient()
    elif provider == "bedrock":
        return BedrockClient()
    raise ValueError(f"Unknown LLM provider: {provider}")

Swapping providers is a one-line environment variable change. No code changes, no redeployment of logic. For on-premises deployments, you add an OllamaClient that points at a local inference server — same interface, same prompts, different endpoint.

What I'd do differently

Invest more in document ingestion pipeline quality. The retrieval quality ceiling is set by how well you parsed and chunked the documents. I underinvested here early on and paid for it in retrieval precision. A week spent on a robust ingestion pipeline — handling PDFs, Word docs, HTML manuals, and structured data sheets correctly — is worth more than a week spent tuning prompts.

Add a reranker. Vector search returns the most semantically similar chunks, not necessarily the most useful ones for answering the specific question. A cross-encoder reranker (e.g., cross-encoder/ms-marco-MiniLM-L-6-v2 via Hugging Face) scores each retrieved chunk against the query directly and reorders them. It adds latency but meaningfully improves answer quality for complex questions.

Evaluate with real queries, not synthetic benchmarks. I built an evaluation set using LLM-generated questions. Those questions look like what an LLM thinks a technician would ask, not what actual technicians ask. The gap is significant. Get five real queries from real users in the first week and use those as your eval set instead.

Chunk overlap matters less than you think. I spent time tuning chunk overlap (the number of tokens shared between adjacent chunks to preserve context across boundaries). In practice, the context prefix approach — prepending document title, section, and page to every chunk — does more work than overlap tuning. If the retriever pulls the right chunk, the prefix tells the LLM where it came from. That's what you actually need.

The Guide is a commercial product and the full source isn't public, but this architecture is complete enough to build from. The QuoteForge repo — which uses the same hybrid search approach for RFP data — has some public pieces if you want to see code in context.

Questions or pushback? Email me or find me on LinkedIn.

Next in this series: MCP as an Enterprise Integration Primitive — why Model Context Protocol is the missing layer between AI agents and the legacy systems that run manufacturing, and how to build adapters that hold up in production.