Skip to main content
Your application searches the web with Tavily, extracts a document with AWS Textract and synthesises speech with Azure. None of these is an LLM, so none of them goes through Bifrost’s provider routing, yet each one spends money, needs a credential and should land on the same budgets as the model calls next to it. A Relay puts Bifrost on that path. You register the upstream API once with its credential, list the operations callers may use and the price of each, and callers send their requests to /relay/{name}/{path} with a virtual key instead of the upstream’s secret.

​
What a relay does on every call

A relay is not a provider. There is no key selection, no routing rule, no model catalog and no per-model budget on this path. The cost is charged to the caller’s overall budgets: virtual key, team, customer and, in Enterprise, business unit and project.

​
Where relays fit

​
Start with one relay

  1. Create a relay with its base URL and credential.
  2. Add routes for the operations callers may use, or import them from an OpenAPI document.
  3. Set the pricing and check it with Test Pricing.
  4. Grant a virtual key access to the relay.
  5. Send a request to /relay/{name}/{path} and read Relay Logs.

Create a relay

Name, base URL, credential, extra headers, network and proxy settings.

Routes and header matchers

Which operations are relayed, counted, streamed or refused.

Usage mapping and pricing

The cost formula, every mapping field and worked examples.

Virtual key access

Grants, budgets and rate limits on the relay path.

Streaming

When a reply is streamed and how a stream is priced.

Relay Logs

Find a call, read its cost breakdown and latency split.
Relays are managed through the config store. When the config store is disabled, no relay is served, the /api/relays management API is not registered (it answers 404), and relays declared in config.json are not loaded.