Ollama makes running open-source large language models locally straightforward — handling model downloading, quantization, and serving through a simple API, without the manual complexity of raw model files.
Prerequisites
- Ubuntu 22.04/24.04 VPS: 8 GB RAM minimum (more for larger models), 4+ vCPU recommended
- See VPS Requirements for Running AI and Machine Learning Workloads for detailed sizing guidance
Step 1 — Install Ollama
curl -fsSL https://ollama.com/install.sh | shStep 2 — Verify the Service Is Running
sudo systemctl status ollamaStep 3 — Download and Run a Model
ollama run llama3.2This downloads the model (several GB, so allow time) and drops you into an interactive chat prompt once ready.
Step 4 — Choose an Appropriately-Sized Model
| Model Size | Approximate RAM Needed | Suitable For |
|---|---|---|
| ~3B parameters | 4-6 GB | Smaller VPS, basic tasks |
| ~7-8B parameters | 8-10 GB | General-purpose use, good balance |
| ~13B+ parameters | 16 GB+ | Higher quality, needs a larger VPS |
Step 5 — Use Ollama's API from Your Application
curl http://localhost:11434/api/generate -d '{
"model": "llama3.2",
"prompt": "Explain what a VPS is in one sentence."
}'Step 6 — Restrict Ollama to Localhost (Default, Recommended)
By default, Ollama only listens on localhost — keep it this way unless you have a specific, secured reason to expose it externally, since it has no built-in authentication.
Exposing Ollama Securely (If Genuinely Needed)
If external access is required, put Ollama behind an authenticated reverse proxy (Nginx with basic auth, or an application layer handling authentication) rather than exposing its raw API directly — see Nginx as a Reverse Proxy for Node.js/Docker Apps for the underlying pattern.
Managing Downloaded Models
ollama listollama rm MODEL_NAMEBuilding a Web Interface for Ollama
See How to Set Up a Private ChatGPT-Style Interface with Open WebUI for a browser-based chat interface connected to your Ollama installation.
Common Errors
"model requires more system memory than available" — choose a smaller model, or add swap space as a (slower) stopgap — see How to Add Swap Space on a Linux VPS.
Very slow responses — expected on CPU-only hardware, especially for larger models; this is a fundamental hardware limitation, not a misconfiguration.
Best Practices
- Start with a smaller model and only move to larger ones if quality genuinely requires it
- Never expose Ollama's API directly to the internet without an authentication layer in front
- Monitor RAM usage closely when experimenting with different model sizes
Continue Reading
- VPS Requirements for Running AI and Machine Learning Workloads
- How to Set Up a Private ChatGPT-Style Interface with Open WebUI
- How to Deploy a RAG (Retrieval-Augmented Generation) Pipeline on a VPS
Browse more articles in AI & Machine Learning on a VPS.
