Before installing any AI tooling, it's worth understanding what a VPS can and can't realistically handle for AI/ML workloads — expectations here matter more than with most other VPS use cases.
CPU vs GPU: The Core Distinction
Most VPS plans are CPU-only. This is perfectly fine for many practical AI tasks (running smaller language models, basic inference, embeddings, lightweight image classification) but will be significantly slower than GPU-accelerated hardware for large model training or heavy image/video generation.
What Runs Well on a CPU-Only VPS
- Smaller open-source LLMs (7B parameters or fewer, quantized) via Ollama — see How to Install Ollama and Run Local LLMs on a VPS
- Text embeddings and vector search for retrieval-augmented generation (RAG)
- Lightweight classification and NLP tasks
- API-based AI features (calling external AI APIs from your application, rather than running models locally)
What Genuinely Needs a GPU
- Training models from scratch (as opposed to using pre-trained ones)
- Large language models (13B+ parameters) at reasonable response speed
- Image generation (Stable Diffusion and similar) at practical speed
- Video generation or heavy computer vision workloads
RAM Requirements
Even CPU-based inference needs substantial RAM — a quantized 7B parameter model typically needs at least 8 GB RAM, with more headroom needed for larger models or concurrent requests. See How to Choose the Right VPS Plan for Your Workload for general sizing guidance, adjusted upward for AI workloads specifically.
Storage Considerations
AI models themselves can be large (several GB to tens of GB per model) — factor this into your storage planning beyond typical application needs, especially if you plan to experiment with multiple models.
Setting Realistic Expectations
A CPU-only VPS running a local LLM will respond noticeably slower than a cloud AI API or GPU-equipped hardware — acceptable for personal projects, prototyping, or lower-traffic use cases, but not necessarily suitable for a high-traffic production AI feature requiring instant responses at scale.
When to Use an API Instead of Self-Hosting
For production applications needing fast, reliable AI responses at scale, calling a managed AI API is often more practical than self-hosting on a CPU-only VPS — self-hosting makes the most sense for privacy-sensitive data, cost control at high volume, or genuine learning/experimentation purposes.
Quantization: Making Models Fit Smaller Hardware
Quantization reduces a model's precision (and therefore its size and compute requirements) with some quality trade-off — tools like Ollama handle this automatically, making meaningfully larger models practical on modest CPU-only hardware than would otherwise be possible.
Common Errors
Model runs but responses are extremely slow — expected behavior for larger models on CPU-only hardware; try a smaller or more aggressively quantized model, or verify sufficient RAM is actually available (swapping to disk causes severe slowdowns).
"Out of memory" when loading a model — the model's RAM requirement exceeds available VPS memory; choose a smaller model or upgrade your plan.
FAQ
Can I add a GPU to an existing VPS?
Generally no — GPU acceleration requires a plan specifically provisioned with GPU access from the start; check with your provider about GPU-specific VPS options if your workload genuinely requires one.
Continue Reading
- How to Install Ollama and Run Local LLMs on a VPS
- GPU vs CPU VPS: What You Actually Need for AI Workloads
- How to Choose the Right VPS Plan for Your Workload
Browse more articles in AI & Machine Learning on a VPS.
