Skip to main content
2.6.0

🔒 Security

  • Go 1.27.4 - Security release with fixes to net/http (a possible HTTP server crash, CVE-2026-97054), cgo, the go command, the compiler and the linker. Bifrost now builds with Go 1.27.4, and the Docker images (Dockerfile, Dockerfile.local, Dockerfile.redhat) move to golang:1.27.4-alpine3.24. Custom Go plugins must be rebuilt with Go 1.27.4 to load.
  • Dependency security updates - golang.org/x/net 0.62.0 (CVE-2026-78661 excessive resource use parsing HTML, CVE-2026-97045 incorrect HTML rendering), github.com/klauspost/compress 1.20.3 (CVE-2026-116591 crash processing compressed data) and github.com/andybalholm/brotli 1.2.8 (CVE-2026-111497 excessive memory use decompressing data).

✨ Features

  • Model Routers - Named model routers classify each request into SIMPLE, MEDIUM or COMPLEX with a semantic, decision-model or LLM classifier, and send each tier to its own weighted provider/model/key targets with fallbacks. A router takes the requests its routing matches (global, team, customer, virtual key or user scope, an optional CEL condition and a priority) after the routing rules run. Routers are managed from the new Routers section in the dashboard, under /api/routing/model-routers, or in config.json, and request logs record which router routed each request. The existing Complexity Router config keeps filling complexity_tier for routing rules unchanged, and the router page warns about routing rules that still read complexity_tier
  • Complexity Router Phrase Suggestions - A model router can sample its traffic, cluster it on a schedule within a spend budget and suggest phrases for each tier. The suggestions view shows their impact on past traffic, adds accepted phrases to the router, backfills its window from existing logs, and can classify a test statement against the router’s tiers. Runs can also be started on demand through the API. The dashboard view is behind a feature flag
  • Bifrost MCP Server - POST /v1/mcp serves Bifrost’s own HTTP API as MCP tools, so MCP clients such as Claude Code can manage a gateway and call its inference routes. Every tool call runs through the same auth and permission checks as a direct /api request. Configured with client.bifrost_url, client.bifrost_mcp_allow_writes (reads only by default) and framework.pricing.openapi_url, or BIFROST_MCP_SPEC for air-gapped installs
  • Warp Management Actions - Warp answers questions about the gateway’s configuration and can create resources through the Bifrost MCP. Every write ends the turn with a confirmation card that shows the tool, its arguments and whether it is destructive, and runs only after the user confirms. Writes are off unless enabled in Warp settings. New endpoints shared with Warp: GET /api/session/me, user limit lookups, GET /api/logs/search (semantic search) and GET /api/logs/{id}/trace
  • Claude Code Gateway Sign-In - Claude Code can sign in to Bifrost with /login (Cloud gateway) at <issuer_url>/gateway/claude-code: device-code sign-in through the OAuth consent page (virtual key, user or anonymous), silent token refresh, Messages inference resolved to the approved virtual key, managed settings, and optional forwarding of Claude Code’s OTLP telemetry to your collectors, stamped with the signed-in virtual key, team, customer or user. The consent page requires the code shown in the terminal and can deny a sign-in, users with no access are refused, and enabling a gateway or changing its managed settings needs an admin session. Configured under oauth2_server_config.gateways.claude_code or on the new Settings → Gateways page, independent of the MCP auth mode
    The gateway reserves the /gateway route prefix. A server.base_path whose first segment is gateway is refused at startup, so move such a deployment to another prefix before upgrading.
  • Realtime Speech and Live Sessions - /v1/realtime serves Gemini Live on the Gemini API and Vertex AI, ElevenLabs Scribe and Soniox real-time transcription (intent=transcription). GenAI SDK clients in Vertex mode can open Live sessions on /genai. A new WebSocket at /v1/audio/speech/stream streams text in and audio back while the text is still arriving, billed once per session and logged with the full spoken text
  • ElevenLabs Drop-In Routes - ElevenLabs SDKs and raw clients can point at {gateway}/elevenlabs and send a Bifrost virtual key in xi-api-key: text-to-speech (with and without timestamps), the model list filtered to the key’s models, and the native Scribe and text-to-dialogue WebSockets, with governance, logging and cost tracking. /elevenlabs_passthrough forwards any other ElevenLabs route
  • Vertex Text-to-Speech - Vertex serves /v1/audio/speech through Cloud Text-to-Speech (Chirp 3 HD and other Google voices) and through Gemini TTS models, including streaming for Gemini TTS
  • Soniox Provider - New soniox provider for speech-to-text (async jobs behind /v1/audio/transcriptions), text-to-speech with streaming, and real-time transcription
  • Baseten Provider - New baseten provider for Model APIs and dedicated deployments: chat, Responses, embeddings and transcription, /v1/decisions through System One, image generation on dedicated deployments (including the dashboard’s /predict URL), and rerank on dedicated BEI deployments
  • Kimchi Provider - New kimchi provider for chat, Responses, embeddings and list models (thanks @arnasnn!)
  • MachGen Provider - New machgen provider for MachGen’s image and video models: text-to-image, image edit, and async video generation with retrieve and download, settled and priced by the video job sweeper. /v1/models lists the models each key can use, with their task types. Set up from the UI or providers.machgen in config.json (thanks @nishkarsh-machgen!)
  • Sarvam Token Factory v2 - Sarvam chat and model listing move to Token Factory v2, and Sarvam Prism serves /v1/decisions (thanks @vikashmaddi-sarvam!)
    sarvam-30b is no longer served. Use sarvam-105b.
  • Gemini Decisions - Gemini decision models (gemini-decision-*) serve /v1/decisions natively, and a new input_file part carries PDFs, audio and video. Empty or mistyped input is rejected before the call (thanks @Xytronix!)
  • Regex Key Aliases - A key alias can be a regex: pattern over the requested model, with a model_id template that reuses its capture groups (for example regex:(claude-.*) → global.anthropic.$1), so one row aliases a whole model family. The model catalog resolves pattern aliases, and the key deployments table shows pattern rows
  • Datasheet Overrides - Model pricing and capabilities can be edited from a datasheet editor under Models, through new override endpoints or datasheet_overrides in config.json. Overrides are stored separately and survive every upstream datasheet sync
  • Usage Estimates for Cancelled Streams - A chat, text or Responses stream that is cancelled or times out has its missing usage estimated with a tokenizer from the output already received, and priced like reported usage. Estimated fields are marked in usage.estimation, and provider-reported fields are never replaced
  • Bedrock Native Structured Output - Bedrock Converse sends json_schema response formats through native outputConfig for models that support it, and falls back to the synthetic tool for other models, for schemas Claude cannot accept, and once more if Bedrock rejects the native schema. The supports_native_structured_output datasheet field overrides the default per model
  • Custom Tools on Every Provider - Custom (freeform) tools such as Codex’s apply_patch are sent as function tools to every model without native custom tool support, and the model’s calls come back to the client as custom tool calls on both Chat Completions and Responses, streaming included. Models marked supports_custom_tools keep the native form
  • Codex WebSocket Through ChatGPT Sign-In - /chatgpt_passthrough/backend-api/codex/responses accepts the WebSocket Codex opens with supports_websockets = true and relays it to ChatGPT with the caller’s ChatGPT token. Each turn is checked against the virtual key and logged and billed like a turn sent over HTTP. A request without a ChatGPT token gets a 426, which moves Codex to HTTP right away. The Codex model picker lists only the models the caller’s virtual key allows
  • MCP Gateway Settings - slug_code_mode turns Code Mode on or off for one client’s /mcp/<slug> endpoint only. mcp_client_metadata_url lets Bifrost present a self-hosted Client ID Metadata Document when upstream authorization servers cannot reach it. mcp_server_max_protocol_version caps the MCP protocol version Bifrost’s own /mcp serves, applied without a restart
  • Configurable OpenAPI MCP Spec Size - The OpenAPI MCP spec size limit rises from 5 MiB to 32 MiB by default and is set with mcp_openapi_max_spec_mb (up to 256)
  • MCP Logs Dashboard - MCP logs show a metric strip with sparklines and previous-period trends, a latency histogram and cancelled counts, readable error cards with formatted upstream JSON, and user, team and customer filters
  • Stop Reason Column - The logs table has an optional stop reason column (thanks @Constantine3!)
  • Trace Dimensions on Child Spans - An OTEL profile with apply_trace_dimensions_to_child_spans set merges the request’s x-bf-dim-* dimensions onto every exported span (LLM call, plugin, retry, fallback, MCP tool), not only the root HTTP span. Span attributes win on conflict, and path, method and keys in reserved attribute namespaces (gen_ai., http., bifrost., …) are skipped (thanks @devin-williams!)
  • Provider Queue Wait Metric - New bifrost_provider_queue_wait_seconds histogram records how long an attempt waits for a provider worker
  • opencode Sessions - Requests from opencode carry a stable x-opencode-session header per virtual key, resolved from the client’s headers (thanks @sstreichan!)
  • Agent Tracing and Log Previews - Agent traces are easier to follow, and log rows for agent-loop requests preview the latest tool call or tool result instead of a blank message
  • Anthropic Thinking Block Binding - thinking.block_binding is forwarded to the Claude API with its beta header
  • Fable 5.2 Fast Mode on Vertex - "speed": "fast" is sent to Vertex for claude-fable-5-2
  • Databricks Scoped OAuth Secrets - Databricks service principals whose OAuth secret is limited to specific scopes work without configuration: Bifrost retries the token request with the scopes its routes need, and a scope 403 is reported as the provider’s own message
  • Virtual MCPs on Virtual Keys - The virtual key details sheet lists the Virtual MCPs assigned to the key
  • FIPS-Approved Key Derivation - In FIPS 140-3 mode, the encryption data key is derived with PBKDF2-HMAC-SHA-256 and passwords are hashed with PBKDF2-HMAC-SHA-256. Standard builds keep writing the earlier formats, and both formats stay readable, so a rolling deploy is safe

🐞 Fixed

  • Mid-Conversation System Reminders - Claude Code’s trailing system reminder now keeps one form (native or <system-reminder>) across turns, decided from the messages actually sent. On Bedrock InvokeModel the changing form had invalidated the prompt cache every turn, cutting cached input and raising cost. The <system-reminder> envelope also drops its trailing newline, and the cache breakpoint stays on the marked block. Native placement is now controlled by the supports_native_mid_conversation_system datasheet field
  • Vertex Prompt Cache Misses - Vertex Claude request bodies always end with tools, system, messages, so turn-to-turn cache reads no longer miss when a client sends messages first
  • Cached Tokens Counted Twice - Anthropic streaming usage, on chat and Responses streams and on the passthrough route, counts each cached prompt token once, including upstreams such as kimi-code that report the cache split on message_delta (thanks @is911 and @wangkanai!)
  • Prompt Cache on Aliases - Prompt cache auto-injection checks an alias’s canonical model, so Bedrock application inference profile aliases for Claude get cache markers (thanks @bardrop!)
  • prompt_cache_options - prompt_cache_options reaches only the models that accept it, instead of failing requests to other providers
  • Anthropic Deferred Web Search - A deferred web search is replayed without a made-up web_search_tool_result, and its late result stays in the next assistant message on replay, Responses and streams, so neither the continuation nor later turns 400
  • Anthropic MCP History on Chat Providers - Messages requests whose history holds mcp_tool_use / mcp_tool_result or server-tool blocks no longer send a roleless message to OpenAI-compatible providers, and Responses items with no chat equivalent are dropped on the Responses-to-Chat fallback (thanks @1AbdullahButt!)
  • Bedrock Structured Output Streams - /anthropic/v1/messages with output_config.format on Bedrock Claude streams one well-formed text block, and a plain-text answer from a model that skips the synthetic tool is no longer lost
  • Bedrock base_url - Bedrock honours network_config.base_url for Converse, InvokeModel, the OpenAI-compatible runtime, Nova Sonic realtime and ApplyGuardrail, and its HTTP client applies the same dial-time address policy as other providers
  • Bedrock Beta Headers - A caller’s anthropic_beta on Bedrock Converse is merged with the betas Bifrost derives instead of replacing them, so fine-grained tool streaming and server-tool betas keep working (thanks @fus3r!)
  • Bedrock tool_reference Blocks - Bedrock InvokeModel keeps tool_reference blocks inside a client tool_result, so deferred tools are expanded (thanks @kunalgoesbyken!)
  • Bedrock Computer Tools - Claude Responses requests with a computer or computer_use_preview tool are served through InvokeModel instead of losing the tool, and replayed tool_use blocks keep toolset_name (thanks @davidelct!)
  • Vertex Anthropic Betas - Vertex receives Anthropic betas in the anthropic-beta header, including on CountTokens, instead of a body field it rejects
  • Bedrock Mantle OpenAI Models - Unsupported effort-dependent sampling parameters are stripped for OpenAI models on Bedrock Mantle, as they are on OpenAI
  • Logprobs Across Formats - Logprobs survive conversion between Chat Completions and Responses, streaming and non-streaming
  • Gemini Function Call IDs - Gemini function calls without an ID get unique IDs, so parallel calls to the same tool pair with the right responses
  • Nested Custom Tool Names - Chat custom tools keep the name and description nested under custom
  • Gemini Image Output Format - output_format and output_compression map to Gemini image output options on Vertex, and the Gemini API no longer receives options it rejects (thanks @carlreid!)
  • Mistral Reasoning Effort - reasoning_effort follows each Mistral model’s ladder instead of being rewritten to high
  • ElevenLabs Timestamps - Speech with_timestamps returns character times instead of null (thanks @abmorrill4!)
    Breaking for Go SDK users: SpeechAlignment.CharStartTimesMs / CharEndTimesMs are renamed to CharacterStartTimesSeconds / CharacterEndTimesSeconds, and the JSON fields char_*_times_ms become character_*_times_seconds, in seconds.
  • ElevenLabs WAV and Scribe Access - Non-streaming wav speech returns a real WAV file, and Scribe transcription works with wildcard virtual keys
  • Empty Transcriptions - Transcribing silence with text, srt or vtt output returns empty text instead of a 500
  • Unsupported Streaming Speech Models - Streaming speech on OpenAI tts-1 and tts-1-hd returns a 400 instead of a 500
  • GenAI Audio by URL - A /genai request with an audio fileData URL is answered as a multimodal prompt instead of being rejected as a transcription
  • Vertex Model Listing - Model listing on us and eu keys uses the same regional endpoint host as inference, and token errors show the underlying cause
  • allowed_models and Key Aliases - A virtual key that allows a model also allows a key alias that resolves to it (thanks @aharshit123456!)
  • Unprefixed Models on Keyless Providers - Catalog-based provider resolution honours SkipKeySelection, so bare Claude Code model names work on the Anthropic passthrough (thanks @cgtarmenta!)
  • Model Parameters After URL Sync - Stored model parameters stay in effect after the background model-parameters URL sync (thanks @Constantine3!)
  • Deterministic Config Hashes - Provider config hashes no longer change with map order, so API-added keys are not dropped on restart (thanks @rojinebrahimi!)
  • Seedance Video Tokens - Seedance video settlements record their output tokens, so logs show them and virtual key token limits count them
  • Active Requests Gauge Leak - A stream cancelled before the upstream’s first byte no longer leaves bifrost_active_requests raised or its log row stuck in processing
  • Upstream Latency - bifrost_upstream_latency_seconds no longer includes provider queue wait, retry backoff or other plugins’ hooks (thanks @sam-klaviyo!)
    Dashboards built on bifrost_upstream_latency_seconds will show lower values after upgrading. Queue wait is now in bifrost_provider_queue_wait_seconds.
  • Large File Uploads - Multipart uploads are streamed to temp files and on to the provider instead of being held in memory, on the Files API (OpenAI, Azure), passthrough routes and transport plugins. Body size limits and their 413 still apply
  • Virtual Key Budgets - Updating the budget of a config.json virtual key no longer fails with invalid reset duration format (thanks @High-PO!). Moving a quarterly budget to another window, or updating a team’s or customer’s rate-limit maximum alone, no longer fails with a 500, and a budget window added with Reset Usage starts at zero
  • Team and Customer Saves - Saving a customer or team no longer rewrites its teams, virtual keys or customer, and a team’s virtual keys are charged to its new customer right after the change
  • Duplicate Provider Configs - A virtual key naming the same provider twice is refused with a 400
  • MCP OAuth Consent - The consent page no longer gets stuck when an expired SSO session is refreshed mid-flow, and abandoned client registrations are kept long enough for clients to retry
  • OAuth Grant Expiry - The OAuth Grants page uses the configured access token TTL
  • OpenAPI MCP Tool Arguments - OpenAPI-derived MCP tools give every argument a name clients accept, so they are no longer dropped
  • A2A Agents - Valid A2A v1 agent cards register, native HTTP+JSON errors are preserved, and Agent Logs show Task artifacts and history
  • Log Retention on Postgres - Retention deletes shrink their batch when they hit the statement timeout instead of failing on the same rows every pass, and retention tables get tighter autovacuum settings so they stop growing between vacuums
  • ClickHouse Hybrid Storage - Object uploads update has_object in batches, which stops memory growth and dropped uploads
  • Log Filters - Relay Logs filter values are read from the last 30 days, and MCP log team and customer filters match calls recorded under several of them
  • Encoded Request Paths - Transport plugins see the raw encoded path, so an encoded traversal cannot disguise the route
  • Plugin Downloads - URL-based plugins download only from hosts listed in BIFROST_PLUGIN_URL_ALLOWLIST, checked on every redirect
    The allowlist is empty by default, so no plugin URL is fetched until its host is listed. Set BIFROST_PLUGIN_URL_ALLOWLIST (comma-separated hosts, *.example.com or *) before upgrading if you load plugins from a URL.
  • Dashboard - Log headers and the date picker fit on mobile, the upgrading screen appears only when the server confirms a chunk is gone, and the datasheet editor hides entries without a provider

🔧 Maintenance

  • Dependency Upgrades - brotli 1.2.8, A2A SDK 2.8.0 and Dependabot updates

🗄️ Database Migrations

This release adds columns to the logs table (logs_add_model_routing_columns) and the agent_logs table (agent_logs_add_caller_correlation_id_column). Bifrost handles concurrent access on the application side, but on large log tables the change still takes locks and adds load. Deploy during a low-traffic window.
  • add_usage_estimation_tokenizers_json_column - Adds the nullable tokenizer settings column to config_client for usage estimation. Reversible: rolls back by dropping the column.
  • add_datasheet_overrides_table - Creates governance_datasheet_overrides. Non-reversible: dropping it would delete user datasheet edits. The table is additive and older binaries ignore it.
  • add_databricks_scopes_column - Adds databricks_scopes to config_keys. Reversible: rolls back by dropping the column.
  • add_model_routing_suggestion_schedules_table - Creates the phrase suggestion schedules table. Reversible: rolls back by dropping the table.
  • add_model_routers_table - Creates model_routers. Non-reversible: dropping it would delete every model router.
  • add_model_router_targets_and_routing - Creates model_router_targets and adds routing columns to model_routers. Non-reversible: dropping it would delete every router’s targets.
  • add_model_router_suggestions_tier_columns - Adds suggestion tier columns to model_routers. Reversible: rolls back by dropping the columns.
  • add_mcp_client_server_type_on_slug_column - Adds the nullable slug_code_mode setting to config_mcp_clients. Reversible: rolls back by dropping the column.
  • add_mcp_client_metadata_url_columns - Adds config_client.mcp_client_metadata_url and oauth_configs.client_metadata_document. Non-reversible: dropping them would discard the hosted document URL and which OAuth configs use Bifrost’s document.
  • add_client_config_mcp_openapi_max_spec_mb_column - Adds mcp_openapi_max_spec_mb to config_client. Reversible: rolls back by dropping the column.
  • add_client_config_bifrost_url_and_bifrost_mcp_columns - Adds the Bifrost URL and Bifrost MCP settings to config_client. Reversible: rolls back by dropping the columns.
  • add_framework_config_openapi_url_column - Adds the OpenAPI URL to the framework config. Reversible: rolls back by dropping the column.
  • add_warp_management_writes_column - Adds management_writes to the Warp config. Reversible: without the column Warp offers reads only.
  • add_client_config_entity_access_profile_mode_column - Adds entity_access_profile_mode (default union) to config_client. Reversible: rolls back by dropping the column.
  • add_budget_business_unit_id_column - Adds a nullable business_unit_id owner column to governance_budgets. Reversible: rolls back by dropping the column.
  • add_budget_business_unit_id_index - Builds the business_unit_id index concurrently on PostgreSQL. Reversible: rolls back by dropping the index.
  • add_oauth2_authorize_requests_device_code_columns - Adds the nullable device_code_hash and user_code_hash columns to oauth2_authorize_requests, with a bounded lock wait on PostgreSQL. Reversible: only short-lived sign-in state is dropped.
  • add_oauth2_authorize_requests_device_code_indexes - Builds their unique indexes concurrently on PostgreSQL, so writes are never blocked. Reversible: rolls back by dropping the indexes.
  • add_client_config_mcp_server_max_protocol_version_column - Adds mcp_server_max_protocol_version to config_client. Reversible: without the column /mcp serves every version, as before.
  • agent_logs_add_caller_correlation_id_column - Adds caller_correlation_id to agent_logs. Non-reversible once any row records a value: the rollback refuses while one does.
  • logs_add_model_routing_suggestion_tables - Creates the log-store tables for phrase suggestions. Reversible: rolls back by dropping the tables.
  • logs_add_model_routing_columns - Adds model router columns to logs. Non-reversible once any log records a router: the router a request was routed by cannot be recaptured, so the rollback refuses while one does.
  • logs_tune_retention_autovacuum - Sets autovacuum_vacuum_scale_factor=0.01 and autovacuum_vacuum_threshold=10000 on the Postgres retention tables. Reversible: rolls back by resetting the options.
All new migrations add tables, nullable columns or indexes and never rewrite existing rows, so they are safe for a rolling deploy.

🐙 Closed GitHub Issues

  • #123 - Files API Support
  • #3770 - Propagate x-bf-dim-* headers to all spans in a trace, not just the root HTTP span
  • #4965 - [Bug]: Bedrock never activates fine-grained-tool-streaming - tool.eager_input_streaming is silently dropped
  • #5272 - [Bug] Bedrock provider ignores Helm network_config.base_url (and AWS_ENDPOINT_URL_*), falls back to default public runtime URL
  • #5354 - [Bug]: Anthropic streaming usage double-counts cache-read tokens (~2× prompt_tokens inflation)
  • #5510 - [Bug]: Anthropic passthrough double-counts cache-read tokens for Kimi-compatible streams
  • #5792 - [Bug]: Anthropic passthrough rejects bare (unprefixed) Claude Code model names with “could not auto resolve a provider”
  • #6406 - Vertex: prompt cache mostly misses because messages is serialized before system/tools
  • #6815 - [Feature]: forward x-opencode-session on opencode requests
  • #7963 - [Feature]: Claude Code gateway sign-in (/login → Cloud gateway) for Bifrost
  • #8073 - Custom model-parameters rows stop applying after the background model-parameters URL sync
  • #8094 - [Bug]: Provider config hash is non-deterministic when maps have 2+ entries; API-added keys dropped on restart
  • #8467 - [Bug]: ElevenLabs Scribe is blocked by wildcard virtual-key permissions despite appearing in Models
  • #8976 - [Bug]: /bedrock/model//invoke drops tool_reference blocks inside tool_result, so deferred tools are never expanded
  • #9276 - [Feature]: Add Kimchi as a first-class provider
  • #9579 - [Bug]: bifrost_upstream_latency_seconds includes provider queue wait
  • #9591 - [Bug]: /anthropic/v1/messages injects web_search_tool_result after a deferred server_tool_use, so the mixed-tool continuation 400s (tool_use without tool_result)
  • #9721 - [Bug]: Anthropic→OpenAI conversion drops role on messages containing mcp_tool_use/mcp_tool_result blocks, causing 400 from all OpenAI-compatible providers
  • #9731 - Mistral: every reasoning_effort is rewritten to “high” (Mistral now accepts low…max per model)
  • #9788 - [Bug]: ElevenLabs speech with_timestamps returns null char_start_times_ms / char_end_times_ms
  • #9846 - [Feature]: Add option to show stop reason as a column.
  • #10026 - [Bug]: prompt_cache auto_inject checks r.Model, not the canonical model
  • #10132 - [Bug]: Updating the budget of a config.json virtual key fails with “invalid reset duration format: ”
1.16.0
  • feat: baseten - new provider for Model APIs and dedicated deployments: chat, Responses, embeddings, transcription, decisions through System One, image generation on dedicated deployments (including /predict URLs) and rerank on BEI deployments
  • feat: kimchi - new provider for chat, Responses, embeddings and list models (thanks @arnasnn!)
  • feat: machgen - new provider for image generation, image edit and async video generation (submit, retrieve, download), with model listing (thanks @nishkarsh-machgen!)
  • feat: soniox - new provider for async speech-to-text, text-to-speech with streaming, speech sessions and real-time transcription
  • feat: sarvam - chat and model listing move to Token Factory v2, and Sarvam Prism serves decisions (thanks @vikashmaddi-sarvam!)
    sarvam-30b is no longer served. Use sarvam-105b.
  • feat: gemini - native decisions for gemini-decision-* models and an input_file part for PDFs, audio and video; empty or mistyped input is rejected before the call (thanks @Xytronix!)
  • feat: realtime - Gemini Live on the Gemini API and Vertex, Vertex Live through the GenAI native protocol, ElevenLabs Scribe and Soniox real-time transcription
  • feat: speech sessions - SpeechSessionProvider interface for incremental text-to-speech over a WebSocket, implemented by ElevenLabs and Soniox
  • feat: elevenlabs - passthrough and native WebSocket dialing for Scribe and text-to-dialogue, raw response kept for with-timestamps
  • feat: vertex - text-to-speech through Cloud Text-to-Speech (Chirp) and Gemini TTS models
  • feat: regex key aliases - regex: alias names over the requested model with model_id templates that reuse capture groups
  • feat: usage estimation - cancelled or timed-out chat, text and Responses streams get missing usage estimated with a tokenizer, marked in usage.estimation
  • feat: bedrock - native outputConfig structured output for supported models, with a one-time retry on the synthetic tool when Bedrock rejects the schema; supports_native_structured_output datasheet field
  • feat: custom tools - custom (freeform) tools are rewritten as function tools for every model without native support and restored as custom calls on chat and Responses, streaming included
  • feat: anthropic - forward thinking.block_binding with its beta header on the Claude API
  • feat: anthropic - fast mode for claude-fable-5-2 on Vertex
  • feat: databricks - OAuth M2M secrets limited to specific scopes work through a scope fallback, and scope errors show the provider’s message
  • feat: opencode - stable x-opencode-session header per virtual key, resolved from client headers (thanks @sstreichan!)
  • feat: files - Files API uploads stream from a FileSource instead of a byte slice on OpenAI and Azure
  • feat: websocket stream hooks return the pre-hook request, so raw relays can forward plugin edits
  • feat: bifrost_provider_queue_wait_seconds timing on the attempt window
  • fix: anthropic - mid-conversation system reminders keep one form across turns, decided from the emitted messages, with placement controlled by the supports_native_mid_conversation_system datasheet field; the <system-reminder> envelope drops its trailing newline and the cache breakpoint stays on the marked block
  • fix: vertex - Claude request bodies always end with tools, system, messages so prompt cache reads hit
  • fix: anthropic - streamed usage counts each cached prompt token once on chat, Responses and passthrough streams (thanks @is911 and @wangkanai!)
  • fix: prompt cache auto-injection checks an alias’s canonical model (thanks @bardrop!)
  • fix: prompt_cache_options reaches only models that accept it
  • fix: anthropic - a deferred web search is replayed without a made-up web_search_tool_result, and its late result stays in place on replay, Responses and streams
  • fix: anthropic - history with MCP or server-tool blocks no longer sends a roleless message to OpenAI-compatible providers; Responses items with no chat equivalent are dropped on the Responses-to-Chat fallback (thanks @1AbdullahButt!)
  • fix: bedrock - /anthropic/v1/messages with output_config.format streams one well-formed text block, and plain-text answers are no longer lost
  • fix: bedrock - honour network_config.base_url and apply the dial-time address policy to the net/http client
  • fix: bedrock - a caller’s anthropic_beta on Converse is merged with derived betas (thanks @fus3r!)
  • fix: bedrock - InvokeModel keeps tool_reference blocks inside a client tool_result (thanks @kunalgoesbyken!)
  • fix: bedrock - Claude Responses requests with computer tools go through InvokeModel, and replayed tool_use blocks keep toolset_name (thanks @davidelct!)
  • fix: vertex - Anthropic betas are sent in the anthropic-beta header, including on CountTokens
  • fix: bedrock mantle - unsupported effort-dependent sampling parameters are stripped for OpenAI models
  • fix: logprobs survive conversion between Chat Completions and Responses
  • fix: gemini - function calls without an ID get unique IDs and pair with responses in order
  • fix: chat custom tools keep the name and description nested under custom
  • fix: gemini - image output_format and output_compression map to Vertex image output options, and the Gemini API no longer receives options it rejects (thanks @carlreid!)
  • fix: mistral - reasoning_effort follows the model’s datasheet ladder
  • fix: elevenlabs - speech with_timestamps returns character times instead of null (thanks @abmorrill4!)
    Breaking for Go SDK users: SpeechAlignment.CharStartTimesMs / CharEndTimesMs are renamed to CharacterStartTimesSeconds / CharacterEndTimesSeconds (JSON char_*_times_ms → character_*_times_seconds).
  • fix: elevenlabs - non-streaming wav speech returns a real WAV file
  • fix: openai - empty text, srt and vtt transcriptions are valid responses
  • fix: openai - streaming speech on tts-1 and tts-1-hd returns a 400
  • fix: gemini - audio fileData parts are no longer treated as transcription
  • fix: vertex - model listing on us and eu keys uses the regional endpoint host, and token errors show the underlying cause
  • fix: streams cancelled before the upstream’s first byte run their terminal post-hooks
  • chore: brotli 1.2.8 and dependency updates
  • chore: upgraded Go to 1.27.4
1.13.0
  • feat: model routers - model_routers and model_router_targets tables with config.json support, and the log-store tables and columns for phrase suggestions and routed requests
  • feat: client gateways - oauth2_server_config.gateways and device-code columns on oauth2_authorize_requests for Claude Code gateway sign-in
  • feat: datasheet overrides - governance_datasheet_overrides table merged over the synced datasheet, surviving upstream sync
  • feat: model catalog resolves regex: key aliases
  • feat: MachGen video jobs are settled and priced by the video job sweeper
  • feat: openapimcp - spec size limit rises to 32 MiB by default and is configurable with mcp_openapi_max_spec_mb
  • feat: warp - management reads and creates through the Bifrost MCP, with confirmation before writes, and model router search filters
  • feat: MCP settings - slug_code_mode, mcp_client_metadata_url and mcp_server_max_protocol_version storage
  • feat: encrypt - PBKDF2-HMAC-SHA-256 key derivation and password hashing in FIPS 140-3 mode; both formats stay readable
  • feat: usage estimation settings and cost for estimated tokens
  • feat: MCP log stats with cancelled counts, a latency histogram and previous-period comparison, and user, team and customer filter data
  • fix: logstore - Postgres retention deletes shrink their batch on statement timeouts, and retention tables get tighter autovacuum settings
  • fix: logstore - ClickHouse hybrid storage updates has_object in batches
  • fix: logstore - Relay Logs filter values read the last 30 days, and MCP log team and customer filters match JSON array columns
  • fix: logstore - log list previews fall back to the latest tool call or result
  • fix: logstore - include stop reason in log list and session queries (thanks @Constantine3!)
  • fix: modelcatalog - allowed_models accepts a key alias that resolves to an allowed model (thanks @aharshit123456!)
  • fix: modelcatalog - stored model parameters survive the URL sync (thanks @Constantine3!)
  • fix: modelcatalog - ElevenLabs Scribe works with wildcard virtual keys
  • fix: configstore - config hashes are deterministic for map fields (thanks @rojinebrahimi!)
  • fix: configstore - customer and team saves no longer rewrite their associations
  • fix: jobaccounting - Seedance video settlements record output tokens
  • fix: openapimcp - every tool argument gets a name clients accept
  • chore: upgraded Go to 1.27.4 and core to v1.16.0
0.5.7
  • fix: Baseten decision models are served natively instead of through Responses emulation
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.12.3
  • feat: the Codex passthrough model list is filtered to the caller’s allowed models
  • feat: virtual keys are read from the xi-api-key header for ElevenLabs clients
  • fix: updating the budget of a config.json virtual key no longer fails with invalid reset duration format (thanks @High-PO!)
  • fix: a team’s cached virtual keys follow its new customer
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.6.18
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.12.3
  • feat: speech sessions are logged with their full input text
  • feat: streamed multipart passthrough uploads are summarized with each part’s content type
  • fix: log previews for agent-loop requests use the latest tool call or result
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.7.18
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.6.18
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.1.18
  • fix: honor SkipKeySelection before catalog-based provider resolution, so bare model names work on keyless passthrough providers (thanks @cgtarmenta!)
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.6.8
  • feat: optional apply_trace_dimensions_to_child_spans profile flag merges x-bf-dim-* dimensions onto every exported span; span attributes win on conflict, reserved keys are skipped (thanks @devin-williams!)
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.1.18
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.4.0
  • feat: model routers - named routers classify requests into tiers and route each tier to weighted targets with fallbacks, by scope, CEL condition and priority
  • feat: complexity router phrase suggestions - sampled request capture, scheduled clustering within a budget, impact replay, add-phrases and backfill from logs, and statement classification
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.7.8
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0
1.9.8
  • feat: bifrost_provider_queue_wait_seconds histogram for time spent waiting for a provider worker
  • fix: bifrost_upstream_latency_seconds excludes provider queue wait (thanks @sam-klaviyo!)
  • fix: bifrost_active_requests no longer leaks on streams cancelled before headers
  • chore: upgraded Go to 1.27.4, core to v1.16.0 and framework to v1.13.0