Skip to main content

​
Compatibility Transformations

The LiteLLM compatibility plugin provides two transformations:
  1. Text-to-Chat Conversion - Automatically converts text completion requests to chat completion format for models that only support chat APIs
When either transformation is applied, responses include extra_fields.litellm_compat: true.

​
1. Text-to-Chat Conversion

Many modern AI models (like GPT-3.5-turbo, GPT-4, Claude, etc.) only support the chat completion API and don’t have native text completion endpoints. LiteLLM compatibility mode automatically handles this by:
  1. Checking if the model supports text completion natively (using the model catalog)
  2. If not supported, converting your text prompt to chat message format
  3. Calling the chat completion endpoint internally
  4. Transforming the response back to text completion format
  5. Returning content in choices[0].text instead of choices[0].message.content
Smart Conversion: The conversion only happens when the model doesn’t support text completions natively. If a model has native text completion support (like OpenAI’s davinci models), Bifrost uses the text completion endpoint directly without any conversion.
This allows you to use a unified text completion interface across all providers, even those that only support chat completions.

​
How It Works

When LiteLLM compatibility is enabled and you make a text completion request, Bifrost first checks if the model supports text completion: Request Transformation:
  • Your text prompt becomes a user message: {"role": "user", "content": "your prompt"}
  • Parameters like max_tokens, temperature, top_p are mapped to chat equivalents
  • Fallbacks are preserved
Response Transformation:
  • choices[0].message.content → choices[0].text
  • object: "chat.completion" → object: "text_completion"
  • Usage statistics and metadata are preserved

​
Enabling LiteLLM Compatibility

  1. Open the Bifrost dashboard
  2. Navigate to Settings → Client Configuration
  3. Enable LiteLLM Fallbacks
  4. Save your configuration

​
Supported Providers

LiteLLM compatibility mode works with any provider that supports chat completions but lacks native text completion support:
ProviderNative Text CompletionLiteLLM Fallback
OpenAI (GPT-4, GPT-3.5-turbo)NoYes
Anthropic (Claude)NoYes
GroqNoYes
GeminiNoYes
MistralNoYes
BedrockVaries by modelYes

​
Behavior Details

Model Capability Detection:
  • Bifrost uses the model catalog to check if a model supports text completion
  • If the model has a “completion” mode in its pricing data, it supports text completion
  • Conversion only happens when the model lacks native text completion support

​
Transformations Reference

​
Transformation 1: Text-to-Chat Conversion

Applies to: Text completion requests on chat-only models
PhaseOriginalTransformed
RequestText prompt (string)Chat message with role: "user"
RequestArray promptsConcatenated into text content blocks
Requesttext_completion request typechat_completion request type
Requestmax_tokens, temperature, top_pMapped to chat equivalents
Responsechoices[0].message.contentchoices[0].text
Responseobject: "chat.completion"object: "text_completion"

​
Metadata Set on Transformed Responses

When either transformation is applied:
  • extra_fields.litellm_compat: Set to true
  • extra_fields.provider: The provider that handled the request
  • extra_fields.request_type: Reflects the original request type
  • extra_fields.model_requested: The originally requested model

​
Error Handling

When errors occur on transformed requests:
  • extra_fields.litellm_compat is set to true
  • Original request type and model are preserved in error metadata

​
What’s Preserved

  • Model selection and fallback chain
  • Temperature, top_p, max_tokens, and other generation parameters
  • Stop sequences and frequency/presence penalties
  • Usage statistics and token counts

​
When to Use This

Good Use Cases:
  • Migrating from LiteLLM to Bifrost without code changes
  • Maintaining backward compatibility with text completion interfaces
  • Using a unified API across providers with different capabilities
Consider Alternatives When:
  • You need chat-specific features (system messages, conversation history)
  • You want explicit control over message formatting
  • Performance is critical (direct chat requests avoid conversion overhead)