Skip to main content

​
Overview

Bifrost provides complete OpenAI API compatibility through protocol adaptation. The integration handles request transformation, response normalization, and error mapping between OpenAI’s API specification and Bifrost’s internal processing pipeline. This integration enables you to utilize Bifrost’s features like governance, load balancing, semantic caching, multi-provider support, and more, all while preserving your existing OpenAI SDK-based architecture. Endpoint: /openai

​
Setup

  • Python
  • JavaScript
import openai

# Configure client to use Bifrost
client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="dummy-key"  # Keys handled by Bifrost
)

# Make requests as usual
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

​
Provider/Model Usage Examples

Use multiple providers through the same OpenAI SDK format by prefixing model names with the provider:
  • Python
  • JavaScript
import openai

client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="dummy-key"
)

# OpenAI models (default)
openai_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello from OpenAI!"}]
)

# Anthropic models via OpenAI SDK format
anthropic_response = client.chat.completions.create(
    model="anthropic/claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello from Claude!"}]
)

# Google Vertex models via OpenAI SDK format
vertex_response = client.chat.completions.create(
    model="vertex/gemini-pro",
    messages=[{"role": "user", "content": "Hello from Gemini!"}]
)

# Azure OpenAI models
azure_response = client.chat.completions.create(
    model="azure/gpt-4o",
    messages=[{"role": "user", "content": "Hello from Azure!"}]
)

# Local Ollama models
ollama_response = client.chat.completions.create(
    model="ollama/llama3.1:8b",
    messages=[{"role": "user", "content": "Hello from Ollama!"}]
)

​
Adding Custom Headers

Pass custom headers required by Bifrost plugins (like governance, telemetry, etc.):
  • Python
  • JavaScript
import openai

client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="dummy-key",
    default_headers={
        "x-bf-vk": "vk_12345",  # Virtual key for governance
        "x-bf-user-id": "user_789",  # User identification
        "x-bf-team-id": "team_456",  # Team identification
        "x-bf-trace-id": "trace_abc123",  # Request tracing
    }
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello with custom headers!"}]
)

​
Using Direct Keys

Pass API keys directly in requests to bypass Bifrost’s load balancing. You can pass any provider’s API key (OpenAI, Anthropic, Mistral, etc.) since Bifrost only looks for Authorization or x-api-key headers. This requires the Allow Direct API keys option to be enabled in Bifrost configuration.
Learn more: See Quickstart Configuration for enabling direct API key usage.
  • Python
  • JavaScript
import openai

# Using OpenAI's API key directly
client_with_direct_key = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="sk-your-openai-key"  # OpenAI's API key works
)

openai_response = client_with_direct_key.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello from GPT!"}]
)

# Or pass different provider keys per request
client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="dummy-key"
)

# Use OpenAI key for GPT models
openai_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    extra_headers={
        "Authorization": "Bearer sk-your-openai-key"
    }
)

# Use Anthropic key for Claude models
anthropic_response = client.chat.completions.create(
    model="anthropic/claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello Claude!"}],
    extra_headers={
        "x-api-key": "sk-ant-your-anthropic-key"
    }
)
For Azure OpenAI, you can use the AzureOpenAI client and point it to Bifrost integration endpoint. The x-bf-azure-endpoint header is required to specify your Azure OpenAI resource endpoint.
  • Python
  • JavaScript
from openai import AzureOpenAI

azure_client = AzureOpenAI(
    api_key="your-azure-api-key",
    api_version="2024-02-01",
    azure_endpoint="http://localhost:8080/openai",  # Point to Bifrost
    default_headers={
        "x-bf-azure-endpoint": "https://your-resource.openai.azure.com"
    }
)

azure_response = azure_client.chat.completions.create(
    model="gpt-4-deployment",  # Your deployment name
    messages=[{"role": "user", "content": "Hello from Azure!"}]
)

print(azure_response.choices[0].message.content)

​
Supported Features

The OpenAI integration supports all features that are available in both the OpenAI SDK and Bifrost core functionality. If the OpenAI SDK supports a feature and Bifrost supports it, the integration will work seamlessly. 😄

​
Next Steps