Manifest LLM Gateway

LLM calls that don't break

An open-source LLM gateway that autofixes your bad requests on the fly.

Use the cloud Self-host it

GitHub 7.5k stars Documentation

Trusted by engineers who ship at

OpenAI Compatible
Bring your own key
Custom providers
Local models

USAGE

Who uses the gateway?

Our LLM gateway is open source. It helps builders and teams create reliable agents and workflows, and manage their inference consumption.

AI builders

Connect your subscription plans, BYOK providers and local models with your harness in seconds. Customize your routing and maximize your plans.

Teams & startups

Manage your inference consumption and settings, and track every request and token through a beautiful interface.

Platforms & providers

Don't let avoidable client errors harm your users' experience and your revenue. Help them connect smoothly to models.

MANIFEST AUTOFIX

5% of LLM API calls fail in production. Most of those errors are avoidable.

Ensure zero downtime by fixing issues on the fly with Manifest Autofix. Our agent repairs the request before it ever comes back to yours. Available through the LLM gateway or standalone.

Source: Datadog, State of AI Engineering

A failed request is checked against known issues, patched (an AI fix is created if the issue is new) and replayed as a success

Everything you need to keep your agents running

Model fallbacks

Automatically retry on another provider when one goes down.

Full body logs

Inspect the complete request and response body for every call.

Autofix

Broken requests get repaired on the fly, before your agent sees the error.

Self-host

Run the whole gateway on your infrastructure or your machine. Free and open source.

Local models

Route to Ollama, LM Studio, llama.cpp or any local server. Decide what goes to the cloud and what stays on your machine.

Bring your own key

Bring your own keys for every provider and pay usage directly. The gateway takes API keys from all popular providers and OAuth tokens for subscriptions.

Easy parameter setup

Set the right model and parameters per route without touching the code. Set AI model parameters visually.

Cost visualization

See every dollar spent, broken down by agent, key and provider.

OPEN SOURCE

Built in the open

Our LLM gateway is fully open source. Use our cloud version for easy onboarding or our self-hosted version based on Docker. We encourage you to participate in this project.