Skip to main content
The Manifest LLM Gateway connects the models and subscriptions you already use behind one OpenAI- and Anthropic-compatible endpoint. Send auto to apply your routing rules and fallbacks, or name a model to send the request there directly. Either way, you can cap what each harness spends and see the cost of every request.

​
Key features

All your providers

Connect API keys, subscriptions you already pay for, local models, or any custom endpoint.

LLM Gateway

Send auto and your routing rules pick the model, with fallbacks when one fails. Limits cap what a harness can spend.

Autofix

Repair a malformed request and resend it, instead of failing on something a retry can’t fix.

Observability

See what every harness spent, which models served it, and what failed.

Request logs

Read back the full request and response body of any provider attempt, as a conversation.

CLI

Create harnesses, connect providers, set routing, and read the request log from the terminal or a script.

​
Cloud or self-hosted?

The gateway is open source and runs two ways. Both speak the same API, so pointing a client at one or the other is a URL change.

LLM Gateway Cloud

Runs on our servers. Sign up, connect a provider, and you’re routing. Recommended if you’re starting out.

Self-hosting

Runs on infrastructure you control: Docker on your own machine, a managed platform like Railway, your AWS or GCP account, or a server panel.
On Cloud there is nothing to install or configure on a server. You sign up, connect your providers, and point your harness at the gateway URL. Routing, limits, Autofix, and alerts are settings in the dashboard. The environment variables throughout these docs apply only to a self-hosted gateway. Cloud is also subject to plan limits: the free plan includes 10,000 routed requests per calendar month, shown by the usage meter in the dashboard sidebar and reset at 00:00 UTC on the 1st. And it can’t reach local models running on your machine. A self-hosted instance has neither restriction.

​
Your first request

Sign up at app.manifest.build and a guided setup walks you through it. Step one creates a harness. Then connect a provider, and give your default tier a model to route to. The last step shows the gateway address and the harness key, which starts with mnfst_. A harness you create later reuses the providers your workspace already has.
The LLM Gateway guided setup on step one, Create your first harness, with the five setup steps listed in the sidebar

The guided setup after signing up: step one creates your first harness.

auto asks the gateway to route the request. The request appears on the Requests page with the model that served it and its cost.
Install the gateway first. Self-hosted covers Docker and the managed platforms. Your instance starts empty. First request takes you from there to a routed request against http://localhost:2099/v1.