Skip to main content

​
Overview

Passthrough integrations let you call provider-native API paths and payloads through Bifrost without route-level request/response conversion. When you use passthrough endpoints, the request still flows through Bifrost core logic. You keep Bifrost features such as logging and observability while sending provider-native paths and bodies.

​
Endpoints

  • /openai_passthrough Default provider: openai
  • /anthropic_passthrough Default provider: anthropic
  • /azure_passthrough Default provider: azure
  • /genai_passthrough Default provider: gemini (with automatic Vertex detection for clients configured to use Vertex)

​
How It Works

  1. Send your request to a passthrough endpoint (OpenAI, Anthropic, Azure, or GenAI passthrough).
  2. The integration strips the passthrough prefix and forwards the remaining provider-native path/body.
  3. Bifrost handles provider execution through core inference and plugin pipelines.
  4. Response status, headers, and body are returned as passthrough output (for both stream and non-stream requests).

​
Provider Selection Rules

​
OpenAI Passthrough

  • Uses openai as the default provider.

​
Anthropic Passthrough

  • Uses anthropic as the default provider.

​
Azure Passthrough

  • Uses azure as the default provider.
  • Requires an Azure key with endpoint configured. api-version is injected automatically:
    • Key config api_version takes priority (consistent with how auth is handled).
    • Falls back to any api-version the client supplied in the query string.

​
GenAI Passthrough

  • Uses gemini by default.
  • Automatically switches to vertex when Vertex patterns are detected, such as:
    • URL path containing /projects/{PROJECT_ID}/locations/{LOCATION}/
    • Request body model containing a Vertex resource path
    • OAuth token pattern typically used for Vertex (Bearer ya29...)

​
Usage Examples

​
OpenAI Passthrough

import openai

client = openai.OpenAI(
    base_url="http://localhost:8080/openai_passthrough/v1",
    api_key="dummy-key"
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "hello from passthrough"}]
)

print(response.choices[0].message.content)

​
Anthropic Passthrough

import anthropic

client = anthropic.Anthropic(
    base_url="http://localhost:8080/anthropic_passthrough",
    api_key="dummy-key"
)

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    messages=[{"role": "user", "content": "hello from passthrough"}]
)

print(response.content[0].text)

​
Azure Passthrough

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="http://localhost:8080/azure_passthrough",
    api_key="dummy-key",
    api_version="2024-10-21",  # overridden by key config api_version if set
)

response = client.chat.completions.create(
    model="gpt-4o",  # your Azure deployment name
    messages=[{"role": "user", "content": "hello from azure passthrough"}]
)

print(response.choices[0].message.content)

​
GenAI Passthrough (Gemini)

from google import genai
from google.genai.types import HttpOptions

client = genai.Client(
    api_key="dummy-key",
    http_options=HttpOptions(base_url="http://localhost:8080/genai_passthrough")
)

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="hello from passthrough"
)

print(response.text)

​
GenAI Passthrough (Vertex-style request)

from google import genai
from google.genai.types import HttpOptions

client = genai.Client(
    vertexai=True,
    api_key="dummy-key",
    http_options=HttpOptions(base_url="http://localhost:8080/genai_passthrough")
)

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="hello from vertex passthrough"
)

print(response.text)

​
Notes

  • Use passthrough when you need a provider endpoint that is not directly supported by Bifrost integration routes yet.
  • For Azure passthrough, auth headers (api-key, x-api-key, OAuth token) are always sourced from the Bifrost key config and never forwarded from the client request.