We are deprecating our rule-based routing

Action required: If you are using complexity routing or specificity routing tiers, set up the default tier or custom routing tiers to your favorite models before September 1st, 2026.

Manifest LLM gateway started with rule-based complexity routing, later joined by a specificity routing set of rules. Today we are deprecating it as it provided mixed results for our users.

For now, users with complexity or specificity tiers activated will still continue with their setup, but we recommend moving to default tier or custom tiers. On September 1st, we will definitively remove those features and we will map the “standard” tier onto the “default” tier and remove the task-specificity routing and other complexity tiers configuration.

Why rule-based routing is not enough

Rule-based routing is a very good deal: its programmatic nature allows routing with almost 0 latency and without any cost. However, there are many flaws in it: semantic rules are not enough to ensure a quality routing for several reasons.

Some harnesses like OpenClaw or Hermes often send a huge system prompt that can be enough to categorize all messages as complex. The abundance of new 3rd party harnesses with different system prompts makes it impossible to cover every case. Many users complained about that.

Moreover, rule-based routing only applies to prompts written in English, leaving aside all of our users using their tools with other languages. In any case, static rules are not powerful enough to capture and understand the complexity of human language.

A credible alternative would be AI-powered routing: an AI router with a specialized model that is able to understand requests and classify them. However, this adds extra latency and cost to all requests and most users do not need that kind of system.

Isolating requests with custom routing

Isolating the requests by passing custom HTTP parameters is a very good alternative for those who can, mostly AI apps / SDK users. Here is how it works with OpenAI SDK in TypeScript:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://app.manifest.build/v1",
  apiKey: "mnfst_XXXX",
  defaultHeaders: { "x-manifest-tier": "custom-value" }, // Replace key and value by your own values, static or dynamic.
});

const response = await client.responses.create({
  model: "auto",
  input: "Hello",
  store: false,
});

The code snippet above will call the model attached to the custom tier x-manifest-tier: custom-value. You can create as many custom tiers as you want and attach a different model and parameters to it.

Our direction

Manifest continues to act as an Open Source LLM gateway with highly configurable routing tiers. We want to focus on making sure our users can use their subscription and providers with all the 3rd party harnesses they want.

All the other features continue to exist: fallbacks, model parameters, custom routing, observability and especially letting you connect every tool with your favorite providers. Happy hacking!