Skip to main content
Local model providers run entirely on your own hardware. Manifest detects the running server, fetches the model list, and routes requests to http://localhost:<port> like any other provider. No API key, no network egress, no per-token cost.

​
Supported runtimes

All three speak OpenAI-compatible /v1/chat/completions and accept any GGUF model file.

​
Start the server

​
Connect to Manifest

1

Open the Routing page

In the dashboard, click the runtime tile (Ollama, LM Studio, or llama.cpp).
2

Confirm the server is reachable

Manifest probes http://localhost:<default-port>/v1/models. If the probe succeeds, every loaded model appears for routing.
3

Pin a model to a tier

Open your default or a custom tier and pick a local model as the primary. You can mix local and cloud models in the same fallback chain.

​
Running Manifest in Docker

If you self-host Manifest in Docker, the container can’t reach a local server bound to 127.0.0.1 on the host. Two of the three runtimes default to loopback and need an explicit override:
Either flip the GUI toggle (LM Studio → ⚙ Developer → Serve on Local Network) or rebind from the CLI:
LM Studio remembers the last --bind, so this is one-time setup.
Inside the Manifest container, the host is reachable as host.docker.internal. Manifest sets this automatically when probing local providers.

​
Cost & privacy

Mix local and cloud in one chain: set a local model as your default for day-to-day calls, and fall back to a cloud model when the local server is offline.