Base de connaissances

A document Q&A system lets users ask natural language questions about a specific set of documents, with answers grounded in that actual content. This guide covers building one using LangChain on a self-hosted VPS.

The Core Architecture

This is a specific, common application of the RAG pattern (see How to Deploy a RAG (Retrieval-Augmented Generation) Pipeline on a VPS) — documents are chunked and embedded, relevant chunks are retrieved based on the question, and an LLM generates an answer grounded in that retrieved context.

Step 1 — Install LangChain and Dependencies

pip install langchain langchain-community sentence-transformers qdrant-client --break-system-packages

Step 2 — Load and Chunk Your Documents

from langchain.document_loaders import DirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

loader = DirectoryLoader('./documents', glob="**/*.pdf")
documents = loader.load()

splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = splitter.split_documents(documents)

See How to Build a Semantic Search Engine with Embeddings for the underlying chunking/embedding concepts — chunk size affects both retrieval precision and how much context the LLM receives per retrieved chunk.

Step 3 — Generate Embeddings and Store in a Vector Database

from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import Qdrant

embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
vectorstore = Qdrant.from_documents(chunks, embeddings, url="http://localhost:6333", collection_name="docs")

See How to Set Up a Vector Database (Qdrant) for AI Applications for the base vector database setup.

Step 4 — Set Up the Question-Answering Chain

from langchain.chains import RetrievalQA
from langchain.llms import Ollama

llm = Ollama(model="llama3")
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=vectorstore.as_retriever(search_kwargs={"k": 4}),
)

See How to Install Ollama and Run Local LLMs on a VPS for the self-hosted LLM backend — keeps the entire pipeline self-hosted, without third-party API dependency for the generation step.

Step 5 — Ask Questions

response = qa_chain.run("What is the refund policy described in these documents?")
print(response)

Improving Answer Quality with Better Prompting

from langchain.prompts import PromptTemplate

prompt_template = """
Answer the question based only on the following context. If the answer isn't in the context, say so.

Context: {context}
Question: {question}
"""

Explicitly instructing the model to answer only from provided context, and to acknowledge when information isn't present, reduces hallucination compared to an unconstrained prompt.

Adding Source Citations to Answers

qa_chain = RetrievalQA.from_chain_type(
    llm=llm, retriever=vectorstore.as_retriever(), return_source_documents=True
)

Returning source documents alongside the answer lets users verify the answer against original content — important for trust and verification, particularly for any use case with genuine accuracy stakes.

Handling Documents That Update Over Time

Establish a process for re-indexing when source documents change — a Q&A system answering from stale, outdated document content provides misleading results, similar to the general concern in How to Build a Semantic Search Engine with Embeddings.

Common Errors

Answers seem confidently wrong (hallucinated) despite correct retrieval — verify the retrieved context actually contains the needed information (a retrieval quality issue, not generation), and strengthen your prompt's instruction to stay grounded in provided context rather than the model's general training knowledge.

Continue Reading

• How to Deploy a RAG (Retrieval-Augmented Generation) Pipeline on a VPS
• How to Build a Semantic Search Engine with Embeddings
• How to Build a Simple AI Chatbot with LangChain on a VPS

Browse more articles in AI & Machine Learning on a VPS.