​

Overview

• Model family detection - Automatic routing based on model ID to handle family-specific parameters
• Parameter renaming - e.g., max_completion_tokens → maxTokens, stop → stopSequences
• Reasoning transformation - reasoning parameters mapped to model-specific thinking/reasoning structures (Anthropic, Nova)
• Tool restructuring - Function definitions converted to Bedrock’s ToolConfig format
• Message conversion - System message extraction, tool message grouping, image format adaptation (base64 only)
• AWS authentication - Automatic SigV4 request signing with credential chain support
• Structured output - response_format converted to specialized tool definitions
• Service tier & guardrails - Support for Bedrock-specific performance and safety configurations

​

Model Family Support

​

Supported Operations

​

1. Chat Completions

​

Request Parameters

​

Parameter Mapping

​

Dropped Parameters

​

Extra Parameters

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",
    "messages": [{"role": "user", "content": "Hello"}],
    "guardrailConfig": {
      "guardrailIdentifier": "guardrail-id",
      "guardrailVersion": "1",
      "trace": "enabled"
    },
    "performanceConfig": {
      "latency": "optimized"
    }
  }'

resp, err := client.ChatCompletionRequest(ctx, &schemas.BifrostChatRequest{
    Provider: schemas.Bedrock,
    Model:    "anthropic.claude-3-5-sonnet-20241022-v2:0",
    Input:    messages,
    Params: &schemas.ChatParameters{
        ExtraParams: map[string]interface{}{
            "guardrailConfig": map[string]interface{}{
                "guardrailIdentifier": "guardrail-id",
                "guardrailVersion": "1",
                "trace": "enabled",
            },
            "performanceConfig": map[string]interface{}{
                "latency": "optimized",
            },
        },
    },
})

• guardrailConfig - Bedrock guardrail configuration with guardrailIdentifier, guardrailVersion, trace
• performanceConfig - Performance optimization with latency (“optimized” or “standard”)
• additionalModelRequestFieldPaths - Pass-through for model-specific fields not in standard schema
• promptVariables - Variables for prompt templates (if using prompt caching)
• requestMetadata - Custom metadata for request tracking

​

Cache Control

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "This context will be cached",
            "cache_control": {"type": "ephemeral"}
          }
        ]
      }
    ],
    "system": [
      {
        "type": "text",
        "text": "You are a helpful assistant",
        "cache_control": {"type": "ephemeral"}
      }
    ]
  }'

resp, err := client.ChatCompletionRequest(ctx, &schemas.BifrostChatRequest{
    Provider: schemas.Bedrock,
    Model:    "anthropic.claude-3-5-sonnet-20241022-v2:0",
    Input: []schemas.ChatMessage{
        {
            Role: schemas.ChatMessageRoleUser,
            Content: &schemas.ChatMessageContent{
                ContentBlocks: []schemas.ChatMessageContentBlock{
                    {
                        Text: schemas.Ptr("This context will be cached"),
                        CacheControl: &schemas.CacheControl{
                            Type: schemas.Ptr("ephemeral"),
                        },
                    },
                },
            },
        },
    },
    SystemMessages: []schemas.ChatMessage{
        {
            Role: schemas.ChatMessageRoleSystem,
            Content: &schemas.ChatMessageContent{
                ContentBlocks: []schemas.ChatMessageContentBlock{
                    {
                        Text: schemas.Ptr("You are a helpful assistant"),
                        CacheControl: &schemas.CacheControl{
                            Type: schemas.Ptr("ephemeral"),
                        },
                    },
                },
            },
        },
    },
})

​

Reasoning / Thinking

​

Anthropic Claude Models

• reasoning.effort → thinkingConfig.type = "enabled" (always enabled when reasoning present)
• reasoning.max_tokens → thinkingConfig.budgetTokens (token budget for thinking)

• Minimum budget: 1024 tokens required; requests below this fail with error
• Dynamic budget: -1 is converted to 1024 automatically

// Request
{"reasoning": {"effort": "high", "max_tokens": 2048}}

// Bedrock conversion
{"thinkingConfig": {"type": "enabled", "budgetTokens": 2048}}

​

Anthropic Nova Models

• reasoning.effort → reasoningConfig.thinkingLevel (“low” → low, “high” → high)
• reasoning.max_tokens → Max reasoning tokens (affects inference configuration)

// Request
{"reasoning": {"effort": "high", "max_tokens": 10000}}

// Bedrock conversion
{"reasoningConfig": {"type": "enabled", "thinkingLevel": "high"}}

​

Message Conversion

​

Critical Caveats

• System message extraction: System messages are removed from messages array and placed in separate system field
• Tool message grouping: Consecutive tool messages are merged into single user message with tool result content blocks
• Image format: Only base64/data URI supported; remote image URLs are not supported by Bedrock Converse API
• Document support: PDF, CSV, DOC, DOCX, XLS, XLSX, HTML, TXT, MD formats supported

​

Image Conversion

• Base64 images: Data URL → {type: "image", source: {type: "base64", mediaType: "image/png", data: "..."}}
• URL images: ❌ Not supported - Will fail if attempted
• Documents: Converted to document content blocks with MIME types

​

Cache Control Locations

• System content blocks (entire system message)
• User message content blocks (specific parts)
• Tool definitions within tool configuration

​

Tool Conversion

• function.name → name (preserved)
• function.parameters → inputSchema (Schema format)
• function.strict → Dropped (not supported by Bedrock)

​

Tool Choice Mapping

​

Tool Call Handling

• Bifrost → Bedrock: Tool call arguments converted from JSON object to input field
• Bedrock → Bifrost: Tool use results with toolUseId, converted back to Bifrost format
• Tool results: Merged consecutive tool messages into single user message

​

Structured Output

// Request with structured output
{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "response",
      "schema": {
        "type": "object",
        "properties": {
          "name": {"type": "string"},
          "age": {"type": "number"}
        }
      }
    }
  }
}

// Bedrock conversion (internal)
{
  "tools": [{
    "name": "bf_so_response",
    "description": "Structured output tool",
    "inputSchema": {
      "type": "object",
      "properties": {...}
    }
  }],
  "toolChoice": {"type": "tool", "name": "bf_so_response"}
}

// Response extraction
// Tool use input is extracted and returned as contentStr

​

Response Conversion

​

Field Mapping

• stopReason → finish_reason: endTurn/stopSequence → stop, maxTokens → length, toolUse → tool_calls
• usage.inputTokens → prompt_tokens | usage.outputTokens → completion_tokens
• Cache tokens: cacheReadInputTokens, cacheWriteInputTokens → prompt_tokens_details/completion_tokens_details
• reasoning/thinking blocks → reasoning_details with index, type, text, and signature
• Tool call input (object) → arguments (JSON string)

​

Structured Output Response

• Tool call with name bf_so_* is treated as structured output
• input object is extracted and returned as contentStr
• Removed from toolCalls array

​

Streaming

​

Chat Completions Streaming

1. Initial message role: contentBlockIndex and role information
2. Content block starts: toolUse blocks with toolUseId, name
3. Content block deltas:
   • Text delta: Incremental text content
   • Tool use delta: Accumulated tool call arguments (JSON)
   • Reasoning delta: Reasoning text and optional signature
4. Message completion: stopReason and final token counts
5. Usage metrics: Token counts, cached tokens, performance metrics

• Each Bedrock streaming event → Multiple Bifrost chunks as needed
• Tool arguments accumulated across deltas and emitted on block end
• Reasoning content emitted with signature if present

​

Text Completion Streaming

​

Responses API Streaming

• response.created
• response.in_progress
• content_part.start
• content_part.delta
• content_part.done
• function_call_arguments.delta
• function_call_arguments.done
• output_item.done

• Tool arguments accumulated across deltas
• Content block indices mapped to output indices
• Synthetic events emitted for text/reasoning content

​

2. Responses API

​

Request Parameters

​

Parameter Mapping

​

Extra Parameters

curl -X POST http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",
    "input": "Hello, how are you?",
    "stop": ["###"]
  }'

resp, err := client.ResponsesRequest(ctx, &schemas.BifrostResponsesRequest{
    Provider: schemas.Bedrock,
    Model:    "anthropic.claude-3-5-sonnet-20241022-v2:0",
    Input:    messages,
    Params: &schemas.ResponsesParameters{
        ExtraParams: map[string]interface{}{
            "stop": []string{"###"},
        },
    },
})

​

Input & Instructions

• Input: String wrapped as user message or array converted to messages
• Instructions: Becomes system message (same extraction as Chat Completions)
• Cache control: Supported on instructions (system) and input messages

​

Response Conversion

• stopReason → status: endTurn/stopSequence → completed, maxTokens → incomplete
• usage.inputTokens/usage.outputTokens preserved with cache tokens → *_tokens_details.cached_tokens
• Output items: text → message | toolUse → function_call | thinking → reasoning

​

Streaming

​

3. Text Completions (Legacy)

• Claude models: Uses Anthropic’s /v1/complete format with prompt wrapping
  • prompt auto-wrapped with \n\nHuman: {prompt}\n\nAssistant:
  • max_tokens → max_tokens_to_sample
  • temperature, top_p direct pass-through
  • top_k, stop via extra_params
• Mistral models: Uses standard format
  • max_tokens → max_tokens
  • temperature, top_p direct pass-through
  • stop → stop

• Claude: completion → choices[0].text
• Mistral: outputs[].text → choices[] (supports multiple)
• stopReason → finish_reason

​

4. Embeddings

​

Request Parameters

​

Parameter Mapping

• No dimension customization
• Fixed output size per model version

• Reuses Cohere format conversion
• Similar parameter mapping to standard Cohere

​

Response Conversion

• Titan: embedding → single embedding vector
• Cohere: Reuses Cohere response format with embeddings array
• usage.inputTokens → usage.prompt_tokens

​

5. Batch API

• POST /batch - Create batch
• GET /batch - List batches
• GET /batch/{batch_id} - Retrieve batch
• POST /batch/{batch_id}/cancel - Cancel batch

​

6. Files API

• id (file ID)
• filename
• size_bytes (from S3 object size)
• created_at (Unix timestamp from S3 LastModified)
• mime_type (derived from content or explicitly set)

• POST /v1/files - Upload
• GET /v1/files - List (cursor pagination)
• GET /v1/files/{file_id} - Retrieve metadata
• DELETE /v1/files/{file_id} - Delete
• GET /v1/files/{file_id}/content - Download content

​

7. List Models

• id (model name with deployment prefix if applicable)
• display_name → name
• created_at (Unix timestamp)

• Region-based model filtering
• Deployment mapping from configuration
• Model allowlist support (allowed_models config)

​

8. AWS Authentication & Configuration

​

Setup & Configuration

​

Endpoints

• Runtime API: bedrock-runtime.{region}.amazonaws.com/model/{path}
• Control Plane: bedrock.{region}.amazonaws.com (list models)
• Batch API: Via bedrock-runtime

​

9. Error Handling

type BifrostError struct {
    IsBifrostError bool
    StatusCode     *int
    Error: {
        Type:    string    // Error classification
        Message: string    // Human-readable message
        Error:   error     // Underlying error
    }
}

• Context cancellation → RequestCancelled
• Request timeout → ErrProviderRequestTimedOut
• Streaming errors → Sent via channel with stream end indicator
• Response unmarshalling → ErrProviderResponseUnmarshal

​

Caveats