How to Install LocalAI as an OpenAI-Compatible API Alternative

Kunnskapsbase

LocalAI provides a drop-in, OpenAI-API-compatible endpoint backed by open-source models running locally — letting you point existing applications built for OpenAI's API at your own self-hosted server with minimal code changes.

Why Use an OpenAI-Compatible API

If you're building an application that already uses OpenAI-style API calls, LocalAI lets you swap in self-hosted, open-source models without rewriting your application's integration code — simply changing the API endpoint and model name in most cases.

Prerequisites

Docker installed
A VPS with adequate RAM for your chosen model (see VPS Requirements for Running AI and Machine Learning Workloads)

Step 1 — Run LocalAI with Docker

docker run -d \
  --name localai \
  --restart unless-stopped \
  -p 8080:8080 \
  -v localai-models:/build/models \
  localai/localai:latest

Step 2 — Verify It's Running

curl http://localhost:8080/readyz

Step 3 — Install a Model

curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{
  "id": "[email protected]"
}'

LocalAI includes a model gallery, or you can manually place compatible GGUF model files in the mounted models volume.

Step 4 — Make an OpenAI-Compatible Request

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Step 5 — Point an Existing Application at LocalAI

Most OpenAI SDK libraries let you override the base URL:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="not-needed-but-required-by-sdk"
)

Existing application code using the OpenAI SDK typically works with only this configuration change.

Securing LocalAI

LocalAI doesn't include authentication by default — place it behind a reverse proxy with API key validation or network-level restrictions if it needs to be reachable beyond localhost.

Available Capabilities Beyond Chat

LocalAI also supports embeddings, image generation, and audio transcription endpoints, depending on which models you install — check the current LocalAI documentation for the full compatibility matrix as it evolves.

Performance Expectations

Like any locally-run model on a CPU-only VPS, expect meaningfully slower response times than a cloud AI API — appropriate for cost control, privacy, or experimentation rather than high-throughput production use unless running on substantial hardware.

Common Errors

"model not found" — verify the model was actually installed successfully and the name in your request exactly matches the installed model's identifier.

Requests time out — larger models on limited CPU resources can take a long time to respond; increase client-side timeout settings accordingly, or use a smaller model.

Continue Reading

Browse more articles in AI & Machine Learning on a VPS.

Kunnskapsbase

Kategorier

Kategorier