How to Install Ollama and Run Local LLMs on a VPS

Vidensdatabase

Ollama makes running open-source large language models locally straightforward — handling model downloading, quantization, and serving through a simple API, without the manual complexity of raw model files.

Prerequisites

Ubuntu 22.04/24.04 VPS: 8 GB RAM minimum (more for larger models), 4+ vCPU recommended
See VPS Requirements for Running AI and Machine Learning Workloads for detailed sizing guidance

Step 1 — Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Step 2 — Verify the Service Is Running

sudo systemctl status ollama

Step 3 — Download and Run a Model

ollama run llama3.2

This downloads the model (several GB, so allow time) and drops you into an interactive chat prompt once ready.

Step 4 — Choose an Appropriately-Sized Model

Model Size	Approximate RAM Needed	Suitable For
~3B parameters	4-6 GB	Smaller VPS, basic tasks
~7-8B parameters	8-10 GB	General-purpose use, good balance
~13B+ parameters	16 GB+	Higher quality, needs a larger VPS

Step 5 — Use Ollama's API from Your Application

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Explain what a VPS is in one sentence."
}'

Step 6 — Restrict Ollama to Localhost (Default, Recommended)

By default, Ollama only listens on localhost — keep it this way unless you have a specific, secured reason to expose it externally, since it has no built-in authentication.

Exposing Ollama Securely (If Genuinely Needed)

If external access is required, put Ollama behind an authenticated reverse proxy (Nginx with basic auth, or an application layer handling authentication) rather than exposing its raw API directly — see Nginx as a Reverse Proxy for Node.js/Docker Apps for the underlying pattern.

Managing Downloaded Models

ollama list

ollama rm MODEL_NAME

Building a Web Interface for Ollama

See How to Set Up a Private ChatGPT-Style Interface with Open WebUI for a browser-based chat interface connected to your Ollama installation.

Common Errors

"model requires more system memory than available" — choose a smaller model, or add swap space as a (slower) stopgap — see How to Add Swap Space on a Linux VPS.

Very slow responses — expected on CPU-only hardware, especially for larger models; this is a fundamental hardware limitation, not a misconfiguration.

Best Practices

Start with a smaller model and only move to larger ones if quality genuinely requires it
Never expose Ollama's API directly to the internet without an authentication layer in front
Monitor RAM usage closely when experimenting with different model sizes

Continue Reading

Browse more articles in AI & Machine Learning on a VPS.

Vidensdatabase

Kategorier

Kategorier