Knowledgebase

LangChain is a framework that simplifies building applications powered by language models — handling prompt management, conversation memory, and integration with tools like vector databases. This guide covers building a basic conversational chatbot.

Prerequisites

• Ollama installed and running (see How to Install Ollama and Run Local LLMs on a VPS)
• Python installed

Step 1 — Install LangChain

pip install langchain langchain-community --break-system-packages

Step 2 — Create a Basic Conversational Chain

nano chatbot.py

from langchain_community.llms import Ollama
from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationChain

llm = Ollama(model="llama3.2")
memory = ConversationBufferMemory()
conversation = ConversationChain(llm=llm, memory=memory)

while True:
    user_input = input("You: ")
    if user_input.lower() in ["exit", "quit"]:
        break
    response = conversation.predict(input=user_input)
    print(f"Bot: {response}")

Step 3 — Run the Chatbot

python3 chatbot.py

The conversation memory means the chatbot retains context from earlier in the same session, enabling natural follow-up questions.

Adding a System Prompt (Defining the Bot's Personality/Role)

from langchain.prompts import PromptTemplate

template = (
    "You are a helpful assistant for a VPS hosting company.
"
    "Answer questions concisely and accurately.

"
    "Conversation history:
"
    "{history}
"
    "Human: {input}
"
    "AI:"
)

prompt = PromptTemplate(input_variables=["history", "input"], template=template)
conversation = ConversationChain(llm=llm, memory=memory, prompt=prompt)

Limiting Conversation Memory (Avoiding Ever-Growing Context)

from langchain.memory import ConversationBufferWindowMemory

memory = ConversationBufferWindowMemory(k=5)

k=5 keeps only the last 5 exchanges, preventing the context from growing indefinitely and slowing down or degrading responses over a long conversation.

Wrapping the Chatbot in a Web API

See How to Serve a Machine Learning Model with FastAPI to expose this as an HTTP endpoint usable by a web or mobile frontend, rather than only a command-line interface.

Adding Document Knowledge (RAG)

For a chatbot that can answer questions about your own specific documents rather than just general knowledge, combine this with retrieval — see How to Deploy a RAG (Retrieval-Augmented Generation) Pipeline on a VPS.

Handling Multiple Concurrent Users

Each user needs their own separate memory instance to avoid conversations bleeding into each other — key memory objects by session/user ID rather than using a single shared memory object across all requests.

Common Errors

Responses ignore earlier conversation context — verify the memory object is actually being reused across calls (not recreated on every request), and that the prompt template correctly includes the history variable.

Very slow responses as conversation grows — longer accumulated context takes more time to process; consider windowed memory (shown above) to cap context size.

Continue Reading

• How to Install Ollama and Run Local LLMs on a VPS
• How to Deploy a RAG (Retrieval-Augmented Generation) Pipeline on a VPS
• How to Serve a Machine Learning Model with FastAPI

Browse more articles in AI & Machine Learning on a VPS.