​

Overview

• Role remapping - “assistant” → “model”, system messages integrated into main flow
• Message grouping - Consecutive tool responses merged into single user message
• Parameter renaming - e.g., max_completion_tokens → maxOutputTokens, stop → stopSequences
• Function call handling - Tool call ID preservation and thought signature support
• Content modality - Support for text, images, video, code execution, and thought content
• Thinking/Reasoning - Thinking configuration mapped to Bifrost reasoning structure

​

Supported Operations

​

1. Chat Completions

​

Request Parameters

​

Parameter Mapping

​

Dropped Parameters

​

Extra Parameters

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-2.0-flash",
    "messages": [{"role": "user", "content": "Hello"}],
    "top_k": 40,
    "stop_sequences": ["###"]
  }'

resp, err := client.ChatCompletionRequest(ctx, &schemas.BifrostChatRequest{
    Provider: schemas.Gemini,
    Model:    "gemini-2.0-flash",
    Input:    messages,
    Params: &schemas.ChatParameters{
        ExtraParams: map[string]interface{}{
            "top_k": 40,
            "stop_sequences": []string{"###"},
        },
    },
})

​

Reasoning / Thinking

​

Parameter Mapping

• reasoning.effort → thinkingConfig.thinkingLevel (“low” → LOW, “high” → HIGH)
• reasoning.max_tokens → thinkingConfig.thinkingBudget (token budget for thinking)
• reasoning parameter triggers thinkingConfig.includeThoughts = true

​

Supported Thinking Levels

• "low" / "minimal" → LOW
• "medium" / "high" → HIGH
• null or unspecified → Based on max_tokens: -1 (dynamic), 0 (disabled), or specific budget

​

Example

// Request
{"reasoning": {"effort": "high", "max_tokens": 10000}}

// Gemini conversion
{"thinkingConfig": {"includeThoughts": true, "thinkingLevel": "HIGH", "thinkingBudget": 10000}}

​

Message Conversion

​

Critical Caveats

• Role remapping: “assistant” → “model”, “system” → part of user/model content flow
• Consecutive tool responses: Tool response messages merged into single user message with function response parts
• Content flattening: Multi-part content in single message preserved as parts array

​

Image Conversion

• URL images: {type: "image_url", image_url: {url: "..."}} → {type: "image", source: {type: "url", url: "..."}}
• Base64 images: Data URL → {type: "image", source: {type: "base64", media_type: "image/png", ...}}
• Video content: Preserved with metadata (fps, start/end offset)

​

Tool Conversion

• function.name → functionDeclarations.name (preserved)
• function.parameters → functionDeclarations.parameters (Schema format)
• function.description → functionDeclarations.description
• function.strict → Dropped (not supported by Gemini)

​

Tool Choice Mapping

​

Response Conversion

​

Field Mapping

• finishReason → finish_reason:
  • STOP → stop
  • MAX_TOKENS → length
  • SAFETY, RECITATION, LANGUAGE, BLOCKLIST, PROHIBITED_CONTENT, SPII, IMAGE_SAFETY → content_filter
  • MALFORMED_FUNCTION_CALL, UNEXPECTED_TOOL_CALL → tool_calls
• candidates[0].content.parts[0].text → choices[0].message.content (if single text block)
• candidates[0].content.parts[].functionCall → choices[0].message.tool_calls
• promptTokenCount → usage.prompt_tokens
• candidatesTokenCount → usage.completion_tokens
• totalTokenCount → usage.total_tokens
• cachedContentTokenCount → usage.prompt_tokens_details.cached_tokens
• thoughtsTokenCount → usage.completion_tokens_details.reasoning_tokens
• Thought content (from text parts with thought: true) → reasoning field in stream deltas
• Function call args (map) → JSON string arguments

​

Streaming

• Streaming responses contain deltas in delta.content (text), delta.reasoning (thoughts), delta.toolCalls (function calls)
• Function responses appear as text content in the delta
• finish_reason only set on final chunk
• Usage metadata only included in final chunk

​

2. Responses API

​

Request Parameters

​

Parameter Mapping

​

Extra Parameters

curl -X POST http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-2.0-flash",
    "input": "Hello, how are you?",
    "instructions": "You are a helpful assistant.",
    "top_k": 40
  }'

resp, err := client.ResponsesRequest(ctx, &schemas.BifrostResponsesRequest{
    Provider: schemas.Gemini,
    Model:    "gemini-2.0-flash",
    Input:    messages,
    Params: &schemas.ResponsesParameters{
        Instructions: schemas.Ptr("You are a helpful assistant."),
        ExtraParams: map[string]interface{}{
            "top_k": 40,
        },
    },
})

​

Input & Instructions

• Input: String wrapped as user message or array converted to messages
• Instructions: Becomes system instruction (single text block)

​

Tool Support

• Computer tools auto-configured (if specified in Bifrost request)
• Function-based tools always enabled

​

Response Conversion

• finishReason → status: STOP/MAX_TOKENS/other → completed | SAFETY → incomplete
• Output items conversion:
  • Text parts → message field
  • Function calls → function_call field
  • Thought content → reasoning field
• Usage fields preserved with cache tokens mapped to *_tokens_details.cached_tokens

​

Streaming

• content_part.added emitted for text and reasoning parts
• Item IDs generated as msg_{responseID}_item_{outputIndex}

​

3. Speech (Text-to-Speech)

​

Request Parameters

​

Voice Configuration

{
  "generationConfig": {
    "responseModalities": ["AUDIO"],
    "speechConfig": {
      "voiceConfig": {
        "prebuiltVoiceConfig": {
          "voiceName": "Chant-Female"
        }
      }
    }
  }
}

{
  "generationConfig": {
    "responseModalities": ["AUDIO"],
    "speechConfig": {
      "multiSpeakerVoiceConfig": {
        "speakerVoiceConfigs": [
          {
            "speaker": "Character A",
            "voiceConfig": {
              "prebuiltVoiceConfig": {
                "voiceName": "Chant-Female"
              }
            }
          }
        ]
      }
    }
  }
}

​

Response Conversion

• Audio data extracted from candidates[0].content.parts[].inlineData
• Format conversion: Gemini returns PCM audio (s16le, 24kHz, mono)
• Auto-conversion: PCM → WAV when response_format: "wav" (default)
• Raw audio returned if response_format is omitted or empty string

​

Supported Voices

• Chant-Female - Female voice
• Chant-Male - Male voice
• Additional voices depend on model capabilities

​

4. Transcriptions (Speech-to-Text)

​

Request Parameters

​

Audio Input Handling

{
  "contents": [
    {
      "parts": [
        {
          "text": "<prompt text>"
        },
        {
          "inlineData": {
            "mimeType": "audio/wav",
            "data": "<base64-encoded-audio>"
          }
        }
      ]
    }
  ]
}

​

Extra Parameters

curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-2.0-flash",
    "file": "<binary-audio-data>",
    "prompt": "Transcribe this audio in the original language."
  }'

resp, err := client.TranscriptionRequest(ctx, &schemas.BifrostTranscriptionRequest{
    Provider: schemas.Gemini,
    Model:    "gemini-2.0-flash",
    Input: &schemas.TranscriptionInput{
        File: audioBytes,
    },
    Params: &schemas.TranscriptionParameters{
        Prompt: schemas.Ptr("Transcribe this audio."),
        ExtraParams: map[string]interface{}{
            "safety_settings": [...],
        },
    },
})

​

Response Conversion

• Transcribed text extracted from candidates[0].content.parts[].text
• task set to "transcribe"
• Usage metadata mapped:
  • promptTokenCount → input_tokens
  • candidatesTokenCount → output_tokens
  • totalTokenCount → total_tokens

​

5. Embeddings

• input → requests[0].content.parts[0].text (single text joins arrays with space)
• dimensions → outputDimensionality
• Extra task type and title via extra_params

• embeddings[].values → Bifrost embedding array
• metadata.billableCharacterCount → Usage prompt tokens (fallback)
• Token counts extracted from usage metadata

​

6. Batch API

• POST /v1beta/batchJobs - Create
• GET /v1beta/batchJobs?pageSize={limit}&pageToken={token} - List
• GET /v1beta/batchJobs/{batch_id} - Retrieve
• POST /v1beta/batchJobs/{batch_id}:cancel - Cancel

• Status mapping: BATCH_STATE_PENDING/BATCH_STATE_RUNNING → in_progress, BATCH_STATE_SUCCEEDED → completed, BATCH_STATE_FAILED → failed, BATCH_STATE_CANCELLING → cancelling, BATCH_STATE_CANCELLED → cancelled, BATCH_STATE_EXPIRED → expired
• Inline responses: Array in dest.inlinedResponses
• File-based responses: JSONL file in dest.fileName

​

7. Files API

• name → id
• displayName → filename
• sizeBytes → size_bytes
• mimeType → content_type
• createTime (RFC3339) → Converted to Unix timestamp

• POST /upload/storage/v1beta/files - Upload
• GET /v1beta/files?limit={limit}&pageToken={token} (cursor pagination)
• GET /v1beta/files/{file_id} - Retrieve
• DELETE /v1beta/files/{file_id} - Delete
• GET /v1beta/files/{file_id}/content - Download

​

8. List Models

• name (remove “models/” prefix) → id (add “gemini/” prefix)
• displayName → name
• description → description
• inputTokenLimit → max_input_tokens
• outputTokenLimit → max_output_tokens
• Context length = inputTokenLimit + outputTokenLimit

​

Content Type Support

​

Caveats