LiteLLM SDK - Bifrost

Since LiteLLM already provides multi-provider abstraction, Bifrost adds enterprise features like governance, semantic caching, MCP tools, observability, etc, on top of your existing setup. Endpoint: /litellm

Provider Compatibility: This integration only works for AI providers that both LiteLLM and Bifrost support. If you’re using a provider specific to LiteLLM that Bifrost doesn’t support (or vice versa), those requests will fail.

Setup

Python

from litellm import completion

# Configure client to use Bifrost
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm"  # Point to Bifrost
)

print(response.choices[0].message.content)

Provider/Model Usage Examples

Your existing LiteLLM provider switching works unchanged through Bifrost:

Python

from litellm import completion

# All your existing LiteLLM patterns work the same
base_url = "http://localhost:8080/litellm"

# OpenAI models
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url=base_url
)

# Anthropic models  
anthropic_response = completion(
    model="claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url=base_url
)

# Google models
google_response = completion(
    model="gemini/gemini-1.5-flash",
    messages=[{"role": "user", "content": "Hello Gemini!"}],
    base_url=base_url
)

# Azure models
azure_response = completion(
    model="azure/gpt-4o",
    messages=[{"role": "user", "content": "Hello Azure!"}],
    base_url=base_url
)

Adding Custom Headers

Add Bifrost-specific headers for governance and tracking:

Python

from litellm import completion

# Add custom headers for Bifrost features
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-bf-vk": "<YOUR-BIFROST-VIRTUAL-KEY>",          # Replace with your actual Bifrost virtual key.
    }
)

print(response.choices[0].message.content)

Supported Features

The LiteLLM integration supports all features that are available in both the LiteLLM SDK and Bifrost core functionality. Your existing LiteLLM code works seamlessly with Bifrost’s enterprise features. 😄

Next Steps

Governance Features - Virtual keys and team management
Semantic Caching - Intelligent response caching
Configuration - Provider setup and API key management