v2.2.1

​

✨ Features

• Virtual Key Assignees and Expanded Search - The virtual key list resolves assigned_user for every row via a single batch lookup, and VK search matches team, customer and user names in addition to key fields, so keys can be found by who owns them (#7225, #7226)
• User Attribution in Prometheus Metrics - The telemetry plugin’s metrics export includes user id and user name labels, so per-user usage and error rates can be graphed and alerted on directly (#7267)
• Prompt Cache Breakpoints Capability - A SupportsPromptCacheBreakpoints model cap with a name-based fallback for OpenRouter gates prompt_cache_breakpoint forwarding, so OpenRouter’s non-Claude models stop rejecting requests that carry it (#7261, #7262)
• MCP Observed Latency - Native (observed) MCP tool-call latency from observed_latency_ms is surfaced in the logs duration column and detail sheet, kept clearly distinct from true execution time and never fabricating a synthetic start timestamp on the timeline (#7271)

​

🐞 Fixed

• Resource ID Path Injection - Caller-supplied resource IDs (batch, file, video, container, response IDs, cached content names) were interpolated into outbound URL paths unvalidated; a crafted ID with ../, ?, #, percent-encoded bytes or control characters could redirect the request to an unintended upstream endpoint. A central EscapeResourceID helper validates and escapes every such ID (#7307, #7310)
• Bedrock S3 SSRF - A caller-supplied s3:// file ID or s3_bucket param could control the upstream host, since S3 virtual-hosted URLs are built as https://{bucket}.{s3host}/{key}; a bucket like 127.0.0.1:PORT# opened a SigV4-signed TLS connection to a caller-chosen host. Bucket names are now validated with a DNS-compatible regex (#7315)
• Caller-Forged Billing Idempotency - The billing-idempotency key was (RequestID, AttemptNumber), and RequestID may come from the caller’s x-request-id header, so two unrelated requests sharing a deliberately chosen ID collided and the second was silently never charged. An internally minted BillingNonce is mixed into the key, making it unforgeable (#7303)
• Abandoned Non-Streaming Request Hung Forever - The worker kept a ctx.Done() arm on an already-claimed delivery send, a regression from the #6972 delivery fix, so a non-streaming caller could hang indefinitely (#7313)
• Repeated Empty Thinking Blocks - Streaming chat chunks emitted empty reasoning/message fields on every content delta, which clients rendered as repeated empty thinking blocks (#7318)
• Claude Code Thread Continuations Failed Behind Key Rotation - Claude Code’s server-side conversation threads are bound to the upstream account that creates them, and Bifrost’s per-request key selection, retries and fallbacks cannot keep a continuation on that account, so thread: {"type": "continue"} requests failed with thread_not_found (about half the time on a two-key config). The Anthropic integration now declares itself stateless: continuations are refused before the provider call with a 400 whose details.error_code is thread_unsupported_request, which makes the client resend the turn in full and stop sending the thread field for the rest of the session, and the provider raw-body path strips thread from create requests so no orphaned thread state accumulates upstream. Token counting is never refused (#7274)
• Gemini Flash-Lite Minimal Thinking Promoted to Low - Normalized gemini-3.1-flash-lite requests silently promoted minimal thinking to low on both Gemini and Vertex; the text model’s four supported levels are now registered, preserving the image variant’s separate restrictions (#7288) (thanks @Javtor!)
• Config.json Virtual Key Limits Broke Under UI Edits - VK rate limits and budgets created via the config.json standalone-limits flow and then edited through the UI produced duplicate, conflicting ownership records (standalone budgets owned directly by the VK, plus orphaned UUID rate-limit rows created by the UI). Migration migrate_vk_standalone_limits_to_model_configs consolidates ownership into VK-scoped model configs while preserving usage counters, and the write paths stop creating the divergent rows (#7291)
• Bedrock Service Tier Rejections - Service tier forwarding for Bedrock (Converse and Mantle paths) sent whatever tier was requested; it is now gated on explicit model capability metadata, failing closed when none exists, so models that do not support the requested tier stop rejecting the request (#7266)
• Anthropic Root-Level Tool Schema Compositions - Anthropic rejects oneOf/anyOf/allOf at the root of a tool’s input_schema but accepts them inside properties; root-level compositions are now rewritten into a flat object schema before dispatch, unblocking tools like Codex’s automation_update (#7265)
• GenAI Streaming TTS - Speech stream chunks routed through the /genai integration had no converter registered; ToGeminiSpeechStreamResponse now serves streaming TTS, and the streaming router returns a clean error instead of panicking when any stream converter is missing (#7248)
• xAI Usage and Cost - xAI reports visible and reasoning completion tokens separately; completion_tokens now folds reasoning in so prompt_tokens + completion_tokens = total_tokens holds, and streaming cost normalization preserves xAI’s authoritative cost_in_usd_ticks to the terminal usage chunk instead of falling back to catalog pricing (#7245, #7250)
• OpenAI-Only search_content_types Forwarded Everywhere - search_content_types on web_search tools is an OpenAI-specific extension; it is now gated behind a per-provider capability check so Bedrock and other OpenAI-compatible backends receive a clean web_search tool without the field (#7244)
• Claude Code diagnostics Field Rejected by Non-Native Providers - Claude Code sends diagnostics.previous_message_id on every request; on the typed-sanitizer path Bedrock, Vertex and Azure returned 400 diagnostics: Extra inputs are not permitted. The field is stripped for providers that do not support it (#7243)
• Unreadable Bedrock Error Logs - AWS returns errors in a flat {"message": ..., "__type": ...} shape the shared Anthropic/OpenAI parsers never looked at, so Bedrock errors were logged with no human-readable reason; the root-level message now seeds BifrostError, and the logs UI falls back to showing the raw provider error body when no message could be extracted (#7221, #7222)
• Anthropic container Param Dropped - The string-form container param on /anthropic/v1/messages was silently dropped, so container reuse provisioned a fresh container every time; it is now carried through the round trip (#5829) (thanks @AdityaPainuli!)
• Responses API Finish Reason Missing From OTEL Spans - gen_ai.response.finish_reason / finish_reasons are now emitted on the llm.call span for /v1/responses requests, so refusals are visible to OTEL consumers, matching the chat path (#7205) (thanks @bkfl-notai!)
• Cohere Fallback Response Shape - The Cohere-compatible route returned the raw Bifrost normalized response when a fallback served the request from a non-Cohere provider, which the Cohere SDK failed to parse; responses are now converted to the Cohere v2 shape (message, finish_reason) (#7198)
• Skill Serving Race - Fixed a race condition in the skill serving handler
• Log and Dashboard Label Truncation - Long model and provider labels in the logs table and dashboard charts truncate from the start so the distinctive suffix stays visible, the logs model column is wider, the search field border and icon spacing are cleaned up, and key picker options are keyed by id so duplicate labels stop highlighting together (#7215, #7216, #7217, #7269, #7297)

​

🗄️ Database Migrations

• migrate_vk_standalone_limits_to_model_configs - Consolidates virtual key standalone budgets and rate limits into VK-scoped model configs: standalone budgets are re-pointed to the VK’s top-level model config (created if missing, usage preserved), orphaned duplicate UUID rate-limit rows created by the UI are deleted with the model config re-pointed to the canonical config.json row, and vk.rate_limit_id is cleared.

​

🐙 Closed GitHub Issues

• #123 - Files API Support
• #5707 - string-form container param on /anthropic/v1/messages is silently dropped - container reuse provisions a fresh container
• #7204 - Responses API path never sets gen_ai.response.finish_reason in OTEL traces (refusals invisible to observability)
• #7287 - Gemini 3.1 Flash-Lite minimal thinking is silently promoted to low
• #7294 - Streaming chat chunks emit empty reasoning/message fields on every content delta, causing repeated thinking blocks
• #7308 - non-streaming caller hangs forever - worker keeps a ctx.Done() arm on a claimed delivery send (regression from #6972)

• fix: POST /v1/realtime/client_secrets mints a Bifrost-issued ephemeral token instead of returning the provider’s token (e.g. OpenAI ek_...) verbatim, which let a client connect to the provider directly and bypass governance, metering and attribution; virtual key credentials are validated before a realtime session is admitted, so a nonexistent VK can no longer open an upstream provider session (#7317)
• fix: streaming chat chunks no longer emit empty reasoning/message fields on every content delta, which made clients render repeated empty thinking blocks (#7318)
• fix: an abandoned non-streaming request no longer hangs its caller forever; the worker kept a ctx.Done() arm on an already-claimed delivery send, a regression from the #6972 delivery fix (#7313)
• fix: caller-supplied s3:// file IDs and s3_bucket params on the Bedrock provider are validated with a DNS-compatible bucket regex before URL construction, closing an SSRF where a bucket like 127.0.0.1:PORT# steered the SigV4-signed request to a caller-chosen host (#7315)
• fix: resource IDs (batch, file, video, container, response IDs and cached content names) are validated and escaped through a central EscapeResourceID helper before being interpolated into outbound URL paths, so IDs carrying ../, ?, #, percent-encoded bytes or control characters can no longer redirect requests to unintended upstream endpoints (#7307)
• fix: an internally minted BillingNonce is mixed into the billing-idempotency key, which was previously (RequestID, AttemptNumber) alone; since x-request-id is caller-supplied, two unrelated requests sharing a forged ID collided on the key and the second was never charged (#7303)
• fix: Bedrock service tier forwarding is gated on explicit model capability metadata (failing closed when none exists) on both Converse and Mantle paths, instead of sending whatever tier was requested and getting rejected by models that do not support it (#7266)
• fix: root-level oneOf/anyOf/allOf in a custom tool’s input_schema are normalized into a flat object schema before reaching Anthropic, which rejects composition keywords at the schema root but accepts them inside properties; unblocks tools like Codex’s automation_update (#7265)
• fix: speech stream chunks have a GenAI converter (ToGeminiSpeechStreamResponse), enabling streaming TTS through the /genai integration, and the streaming router returns a clean error instead of panicking when a stream converter is missing (#7248)
• feat: SupportsPromptCacheBreakpoints model cap with a name-based fallback for OpenRouter, so prompt_cache_breakpoint is only forwarded to models that accept it (#7261, #7262)
• fix: xAI completion_tokens folds reasoning_tokens into the total so prompt_tokens + completion_tokens = total_tokens holds, and streaming cost normalization preserves xAI’s authoritative cost_in_usd_ticks through to the terminal usage chunk instead of falling back to catalog pricing (#7245, #7250)
• fix: search_content_types on web_search tools is an OpenAI-specific extension and is now gated behind a per-provider capability check, so Bedrock and other OpenAI-compatible backends receive a clean web_search tool without the field (#7244)
• fix: the diagnostics field Claude Code sends on every request is stripped for providers that reject it (Bedrock, Vertex, Azure return 400 diagnostics: Extra inputs are not permitted on the typed sanitizer path) (#7243)
• fix: AWS Bedrock’s flat error shape {"message": ..., "__type": ...} seeds the root-level message into BifrostError, so Bedrock errors served through the shared Anthropic/OpenAI handlers are logged with a human-readable reason (#7222)
• [fix]: the string-form container param is carried through the /v1/messages round trip instead of being silently dropped, so container reuse stops provisioning a fresh container (#5829) (thanks @AdityaPainuli!)
• fix: the Cohere-compatible route converts fallback responses from non-Cohere providers into the Cohere v2 shape (message, finish_reason) via ToCohereChatResponse, instead of returning the raw Bifrost normalized response the Cohere SDK cannot parse (#7198)
• fix: clearer error message in the Anthropic integration (#7264)
• [fix]: preserve Gemini 3.1 Flash-Lite minimal thinking on Gemini and Vertex @Javtor
• fix: the Anthropic raw-body request path strips the server-side thread field before the wire on both messages and count_tokens (thread state is bound to the creating account and cannot survive Bifrost’s per-request key selection, retries, or fallbacks; the HTTP transport refuses continuations outright), AnthropicMessageErrorStruct gains an optional details object with error_code so error envelopes can carry the machine-readable codes clients key their recovery on, and LargePayloadMetadata gains ThreadType so the refusal also works when large-payload mode skips body parsing (populated by the enterprise metadata extractor)
• [fix]: MCP tool schemas keep one property order across tool syncs @dougcalobrisi
• feat: WhiteList and BlackList entries that start with regex: are RE2 patterns. MatchEntry compiles them once as a case-insensitive full match, Validate refuses a pattern that is empty, * or does not compile, and the list-models pipeline never surfaces a pattern as a model. Plain entries keep their exact, case-insensitive match
• fix: fasthttp-level stale-connection retries no longer multiply max_retries. contextTransport.RoundTrip reports a failure before response headers on a freshly dialed socket with retry=false, so network.StaleConnectionRetryIfErr only walks past pooled keep-alive sockets the upstream closed while idle; an upstream that closes a fresh connection without answering now costs exactly one attempt of Bifrost’s own retry budget instead of up to four. The retry backoff also ends as soon as the request context is cancelled, so a worker is freed immediately instead of after up to retry_backoff_max (#7035)
• [fix]: Gemini provider - preserve inline image and audio data in chat completions responses @Atharva-Kanherkar
• [fix]: route opencode-zen Responses calls through /v1/chat/completions, which its upstream does not serve on /v1/responses @miguelchico
• feat: the namespace-tool names a provider reserves for its own server tools (web, image_gen, browser, python on Bedrock and Bedrock Mantle) are now read from the datasheet row reserved_tool_namespaces (new ModelCapabilities field and ModelCaps.ReservedToolNamespaces) for the base provider and canonical model; a non-empty row replaces the hardcoded list, an absent or empty row keeps it, so any OpenAI-wire provider can reserve names through a row without a code change
• fix: Codex >= 0.147 wraps its default tools in a namespace literally named functions (openai/codex#37022), which Bedrock Mantle reserves and rejects with User-defined namespace 'functions' collides with an existing tool namespace. Core now unwraps that namespace to top-level tools, unprefixed, for every provider before dispatch; Codex treats the bare name and the explicit functions namespace as the same tool, so no mapping back is needed; a description on the functions namespace is prepended to each hoisted member, as flattening does
• fix: Responses namespace tools are now flattened in core for every provider whose wire does not support the type (Anthropic, Gemini, Vertex, Bedrock Converse, DeepSeek, and every OpenAI-compatible third party), with nested functions renamed to <namespace>__<function> so two namespaces that share a function name no longer collide into an upstream Tool names must be unique 400; returned function_call items are mapped back to the bare name plus namespace (unary and streaming), prior-turn calls carrying namespace and tool_choice names are re-aliased to match, and a name that is still duplicated after flattening or a tool_choice that matches several namespaces is rejected with a clear 400 before reaching the provider. Bedrock answers namespace support from its own surface resolver via the new optional ResponsesNamespaceToolProvider interface. A datasheet row supports_namespace_tools (new ModelCapabilities field and ModelCaps.SupportsNamespaceTools) overrides the per-provider default for a (provider, model) pair; with no row the default applies. A row can only narrow within what the wire can carry: the Anthropic Messages API (Anthropic, Claude on Azure or Bedrock Mantle), the Gemini API (Gemini, Vertex) and Bedrock Converse have no namespace container, so they answer false regardless of the row and always flatten. Flattened names honour the target wire’s documented tool-name limit, 64 characters of [A-Za-z0-9_-] for OpenAI-compatible wires, Bedrock Converse and Fireworks, 128 for Anthropic, and 128 with . and : allowed for Gemini and Vertex, overridable per model through the datasheet row tool_name_max_length (a row below 10 cannot hold the hashed form and is ignored); a longer alias becomes an 8-hex hash prefix plus the function name, deterministically, so history and tool_choice re-alias to the same string. The alias map travels on the prepared request (BifrostResponsesRequest.NamespaceToolAliases) and is applied to that attempt’s response, unary and streaming; nothing is kept on the request context or in process-wide state (#7048)
• fix: drop namespace tools whose name Amazon Bedrock reserves (web, image_gen, browser, python) on the Bedrock and Bedrock Mantle Responses paths instead of forwarding them into a tools.namespace collision 400; a dropped Codex web namespace becomes the hosted web_search tool on Bedrock Mantle
• fix: Bedrock Mantle chat streaming no longer drops the usage-only chunk that arrives after finish_reason; ProviderSendsDoneMarker now treats bedrock_mantle (and the legacy Mantle route under the bedrock key) as sending [DONE], so streamed usage and cost are recorded (#7065)
• fix: default_request_timeout_in_seconds now bounds the wait for response headers on streaming requests, and cancelling a request now closes the upstream socket. Every fasthttp client is driven through a Bifrost-owned RoundTripper (core/providers/utils/roundtripper.go) that applies the client’s read/write timeouts and the request context to the request write and the header wait, then lifts the socket deadline once headers are parsed so stream_idle_timeout_in_seconds remains the only bound on the body. BuildStreamingClient and BuildLargeResponseClient no longer zero the timeouts. An upstream that accepts the connection and never answers now fails with 504 RequestTimedOut after the request timeout and the request’s fallbacks are used, instead of pinning the provider worker until the upstream closes the socket; MakeRequestWithContext no longer leaves a background client.Do running until ReadTimeout after its context is cancelled (#7034)
• fix: unary large-response downloads (FinalizeResponseWithLargeDetection) now bound every body read with stream_idle_timeout_in_seconds and close the upstream socket on request cancellation, mirroring the streamed passthrough path; a stall during the prefetch surfaces as 504 RequestTimedOut instead of pinning the provider worker, and a stall while the transport drains the LargeResponseReader fails the read with the idle-timeout error instead of blocking the writer indefinitely (#7104)
• fix: gzip-encoded unary responses are now classified for large-response mode by decompressed size; Content-Length describes the compressed bytes, so a body that was small on the wire but large once decompressed used to be materialized in full past large_response_threshold (#7104)
• fix: an OpenAI-compatible upstream that omits [DONE] and then goes silent after finish_reason no longer fails the stream when stream_idle_timeout_in_seconds fires. The chat and text completion read loops now treat an idle timeout after a terminal signal as a parked upstream, mark the stream parked so the connection is abandoned rather than drained, and synthesize the final chunk with the buffered finish_reason; a stall before finish_reason still surfaces as the idle-timeout error (#7108)
• fix: a streamed upstream that drops the connection mid-body is again reported as the retryable 502 provider closed the stream before sending a completion marker error instead of a generic Error reading stream: unexpected EOF. The Bifrost round tripper’s chunked decoder surfaced the drop as io.ErrUnexpectedEOF, which no provider read loop treats as end of stream; it now reports the plain io.EOF fasthttp always did and discards the half-read connection (#7104 follow-up)
• fix: fallbacks for image edit, image variation and video edit requests now reach the configured fallback provider and model. prepareFallbackRequest had no arm for those three types, so the shallow request copy kept the primary’s sub-request pointer and the “fallback” attempt was routed back to the primary while RoutingInfo, the x-bifrost-routing-info-* headers, the log row and the fallback_index metric label reported it as a fallback. The helper now also verifies the prepared request targets the fallback provider and model and skips the fallback with a warning otherwise, so a future request type added without an arm fails loudly instead of silently re-running the primary (#6966)
• fix: route Bedrock Claude requests that carry a compact_20260112 edit to InvokeModel / InvokeModelWithResponseStream with the native Anthropic Messages body, so server-side compaction works on bedrock/ models, including keys that pin an inference-profile ARN. AWS documents compaction as unsupported on the Converse API, which previously received the edit and silently ignored it (#6825)
• feat: serve Anthropic tool search (tool_search_tool_*, defer_loading) on bedrock/ Claude models by routing those requests to InvokeModel / InvokeModelWithResponseStream, the only Bedrock API AWS allows it on; CountTokens counts such requests with the same InvokeModel body via the invokeModel input
• fix: a reasoning.context value the target model does not accept is now dropped on the OpenAI and Azure Responses path, so all_turns on the original gpt-5 family incl. gpt-5-pro, gpt-5.1 to gpt-5.3, and the o-series runs under the model’s own current_turn default instead of failing with Unsupported value: 'all_turns' is not supported with the 'gpt-5-pro' model; gpt-5.4, gpt-5.5 and gpt-5.6 keep it. The accepted values come from the datasheet row supported_reasoning_contexts (new ModelCapabilities field and ModelCaps.SupportedReasoningContexts), falling back to a name-based default of auto and current_turn everywhere plus all_turns on gpt-5.4+
• fix: mid-conversation role: "system" messages are now inlined in place as <system-reminder> user turns for every model family on every converter that has a top-level system field: Bedrock Converse (Responses and Chat Completions), Gemini Chat Completions (systemInstruction), and the Anthropic wire shape used by DeepSeek, Fireworks and SGL. Previously only Claude on Bedrock/Anthropic inlined; everything else hoisted each reminder into the top-level system block. Claude Code appends a trailing <total_tokens> system reminder after every turn, so the hoisted block grew the front of the prompt each turn and prefix-based caches (Bedrock implicit caching for global.openai.gpt-5.6-luna on Converse, Gemini implicit caching, DeepSeek context caching) reported a full cache write and zero cache reads on every turn
• [fix]: vLLM key selection resolves the requested alias through key.Aliases per key before comparing it with vllm_key_config.model_name, so a valid key is no longer rejected when the request used an alias; the same public alias can map to different physical model IDs across vLLM instances, and allow and block checks keep matching the original user-facing alias @Constantine3
• [fix]: Gemini Responses requests keep top_k, frequency_penalty, presence_penalty, stop_sequences and media_resolution across retries and fallbacks. convertParamsToGenerationConfigResponses deleted those keys from ExtraParams while mapping them into generationConfig, and the conversion runs once per attempt on the same request, so every attempt after the first reached the provider without them; the outbound ExtraParams is now built through a filtered copy instead of mutating the request @VictorRequenaMaisa
• feat: wait_for_usage on custom_provider_config keeps the OpenAI-compatible chat and text completion read loops open past finish_reason until the trailing usage-only frame arrives, so a custom provider that also sets does_not_send_done_marker no longer synthesizes a zero-usage terminal chunk and records the request at zero tokens and zero cost. WaitForUsage is carried on BifrostContextKeyWaitForUsage with the same set-or-clear discipline as DoesNotSendDoneMarker, so it never leaks onto a fallback provider that did not declare it, and termination stays bounded by the usage chunk, two consecutive post-finish heartbeat comments, EOF, or stream_idle_timeout_in_seconds (#7143)
• fix: ExtraFields.RawResponse now carries role-only, finish-only and usage-only SSE frames on the OpenAI-compatible chat stream. Those frames never entered the chunk-forwarding branch, so their bytes were discarded from the reconstructed raw_response, which made the captured audit trail irreconcilable against a provider invoice because the usage frame carries the token counts Bifrost bills from. A pendingRawFrames buffer accumulates them and drains onto the next forwarded chunk or the synthetic terminal chunk; the Responses-over-Chat fallback no longer stamps one upstream frame onto every derived event, and delta.refusal and delta.annotations are added to the forwarding predicate so refusals and streamed URL citations are no longer dropped entirely (#7144)
• fix: Anthropic server-side tool search survives the Bedrock-native invoke ingress end to end. tool_search_tool_* was dropped during the Converse-shaped intermediate conversion, leaving the egress predicate blind to the tool and routing the request to Converse where server-side tool search is unavailable; the tool is now carried as an ingress-only marker (json:"-", never reaching a Converse body) and the neutral tool is rebuilt in ToBifrostResponsesRequest along with defer_loading. Results are returned as a server_tool_use plus tool_search_tool_result pair rather than a client tool_use, which Anthropic rejects when the caller echoes a tool_result for the srvtoolu_ ID on the next turn, on both the unary and streaming paths; BedrockContentBlock.UnmarshalJSON gained server_tool_use and tool_search_tool_result cases so replayed turns round-trip unchanged instead of falling through to empty structs; and the Anthropic-native response path carries the same blocks (#7155)
• fix: native Gemini generateContent image requests routed through /genai no longer return HTTP 200 while silently ignoring part of the request. isImageEditRequest only inspected contents[0].parts[0], so a request with the prompt text before the inlineData part, the ordering in Google’s own REST edit sample, was misclassified as image generation: the image never reached Vertex and the model invented a picture from the prompt alone. Detection now scans every part across every content. imageConfig.aspectRatio is preserved as a typed aspect_ratio param that both outbound converters prefer, instead of being folded into a WxH size string that collapsed any ratio outside 1:1, 3:4, 4:3, 9:16 and 16:9 back to square (#7173)
• fix: Gemini Part gained mediaResolution, the per-part override for generationConfig.mediaResolution. Part.UnmarshalJSON decodes into a closed alias, so the key was discarded before any conversion ran and per-part image and PDF tokenization fell back to the model default; an ULTRA_HIGH image billed about 21k prompt tokens through /genai instead of about 22.1k direct. Both the mediaResolution and media_resolution spellings are parsed, the value round-trips through ResponsesMessageContentBlock and is reconstructed on the outbound part, and it is stripped on the OpenAI wire path where the field is unknown (#7156)
• fix: image blocks inside a Bedrock Converse toolResult are hoisted out and re-inserted immediately after the last tool result in the same message, with a placeholder text block left behind so the now image-free result is not rejected for being empty. Some Bedrock-hosted models, notably the OpenAI and Grok families, reject nested tool-result images on Converse even though they accept images in tool output via the Responses API. BedrockModelSupportsToolResultImages supplies the name-based default and the datasheet field supports_converse_tool_result_images (new ModelCapabilities field and ModelCaps.SupportsConverseToolResultImages) overrides it per row (#7150)
• feat: bifrost_error_requests_total gains a normalized error_type label drawn from a closed, prefix-structured vocabulary (caller_*, policy_*, provider_*, bifrost_*, _OTHER), so a 429 from a governance rate limit is distinguishable from a 429 from an upstream provider, a 403 from a blocked model from one from a bad key, and a 503 Bifrost shed under queue pressure from an upstream overload. ClassifyErrorType resolves a declared ExtraFields.ErrorType first, then Bifrost’s own string markers, then the upstream status code; it deliberately ignores the provider’s own error.type and error.code strings, which disagree across providers for the same condition. Values are prefixed by fault domain so an alarm expression can match a family with a single regex (#7141)
• fix: Azure Foundry deployments of Fireworks-hosted models are no longer silently capped at 4096 output tokens on /openai/v1/responses, where Microsoft routes those models through chat completions internally. The Azure provider checks the model’s datasheet supported_endpoints and, when /v1/responses is absent, transparently serves both Responses and ResponsesStream through /openai/v1/chat/completions and converts the result back to the Responses shape; a model with an explicit /v1/responses entry or no datasheet row at all is unchanged. Separately, ToAnthropicResponsesResponse now follows StopReason > IncompleteDetails > tool_use inference > end_turn, so a turn truncated by the output-token cap on any OpenAI-shaped Responses provider reports stop_reason: max_tokens instead of hiding the truncation as end_turn (#6782)
• fix: DeepSeek chat requests move max_completion_tokens into max_tokens on the wire. DeepSeek’s chat-completions endpoint only recognizes the legacy field and silently ignores max_completion_tokens, so the limit had no effect; this matches the behaviour already in place for Opencode and Ollama (#7131)
• feat: guardrail redaction gains an identity-based transformation path for provider-native request and response bodies, so exact provider-managed rewrites such as Model Armor and Bedrock land in the correct native JSON field even when the same text value appears in several fields. TextTargetID, TextTargetIDForIndex and TextRewrite identify each guardrail-visible field by a stable ID rather than by content value; RawRequestBodyTextTransformer and RawResponseTextTransformer let integrations register handlers on the request context; and rewriteRawJSONTextTargets validates target uniqueness, verifies each Original value before patching and re-reads the written value after, so stale or mismatched normalized text cannot silently rewrite the wrong field (#7049)
• feat: guardrail redaction covers LLM tool-call arguments, Chat function arguments, Responses function arguments and custom-tool input, across the Anthropic streaming and non-streaming paths. input_json_delta events are inspected and rewritten alongside text_delta, reading and writing delta.partial_json; collectAnthropicArgumentStringPaths recursively collects string-valued paths inside a tool_use block’s input without touching tool names, IDs or definitions; BifrostResponsesStreamResponse gained Input to carry the full custom-tool payload on custom_tool_call_input.done; and DeepCopyResponsesMessage deep-copies ResponsesCustomToolCall so a copied message cannot mutate the original (#6977)
• fix: an abandoned non-streaming request, one whose caller context was already cancelled by the time the upstream finished, is billed and logged deterministically instead of about half the time. The worker’s delivery select had two simultaneously ready cases, a send into a cap-1 channel and ctx.Done(), and Go picks uniformly among ready cases, so the terminal post-hooks that record billing and finalize the log row were skipped roughly 50% of the time. requestWorker now checks req.Context.Err() before the select on both the error and success paths and calls billAbandonedTerminal directly, keeping the 5-second timer guard inside the select for a caller that leaves between the check and the send (#6972)
• fix: provider response headers are filtered by classifier, not only by a fixed map of 28 exact names, so a credential-named header outside that map is no longer re-served to the inference caller in the HTTP response and extra_fields.provider_response_headers. The extractors now also consult schemas.IsSensitiveHeader, which matches credential names by substring and suffix and already recognized cf-access-* and x-amzn-oidc-*; a fixed list cannot enumerate the space when network_config.extra_headers exists to carry custom authentication headers and some upstreams echo request headers back (#7120) @Atharva-Kanherkar
• [fix]: a plugin that returns an incomplete BifrostError whose nested Error field is nil no longer panics the request worker. The fallback decision helper nil-checks fallbackErr, guards access to Error.Type and reads the message through the nil-safe GetErrorString() helper, continuing to the next fallback when allowed; behaviour for cancelled requests and AllowFallbacks == false is unchanged (#6967) @Constantine3
• [fix]: clearCtxForFallback now clears BifrostContextKeyProviderResponseHeaders. Providers set that key from their own HTTP response before the status check so error paths can forward it, and when a fallback attempt failed pre-flight, key selection failing for the fallback provider, a plugin short-circuiting it, or the queue retiring, nothing overwrote the key and the primary’s headers survived onto the fallback’s error response. A client could receive a response attributed to provider B carrying provider A’s Retry-After and x-ratelimit-remaining-*, and wait according to a limit belonging to a provider that never served the request (#6973) @Huang-404-Q
• [fix]: untitled document blocks get unique names on the Bedrock Converse path. Every untitled block was named the literal document, and Converse rejects duplicate document names, so any request carrying two or more untitled documents failed unconditionally with ValidationException: Messages can't contain duplicate document names. A per-request namer threaded through the content-block conversion disambiguates with numeric suffixes (document, document-2, document-3) and suffixes explicitly titled documents only on an actual collision; the Responses input replay path gets the same treatment scoped to its content-block list (#7003) @Huang-404-Q
• [fix]: BifrostResponsesStreamResponse.Item is omitempty, so Responses stream events that carry no item payload no longer serialize "item": null. Strict OpenAI Responses clients reject those as invalid stream frames, which broke streamed /v1/responses usage entirely; response.output_item.added and response.output_item.done still emit the item object @ReStranger
• feat: SecretVar.RedactedIfSecret() returns a plain clone when the value is a literal and delegates to Redacted() when it is env-var or vault-backed, so regions and service URLs (Azure endpoint, Vertex/Bedrock/Bedrock Mantle region, vLLM/Ollama/SGL/Databricks URL, MCP connection string) are readable in the UI instead of being masked as if they were credentials. It always returns a fresh pointer, so the redacted copy never aliases the live config, and it replaces the ad-hoc IsFromSecret() guard blocks that tried to replicate this inline (#7085)
• feat: GA realtime transcription is served for OpenAI and Azure over both WebSocket and WebRTC with normal Bifrost authentication, routing, governance, guardrails, logging and transcription-aware pricing. Unlike a normal realtime session, which names its routing model in the connect URL, a GA transcription session carries only intent=transcription and delivers the model later in session.update under audio.input.transcription.model (or in the initial multipart /v1/realtime/calls request on WebRTC), so routing now resolves from the nested transcription model while realtime connection and turn semantics are preserved (#7089)
• fix: Claude Code replaying a prior assistant message through POST /anthropic/v1/messages at a Bedrock Mantle model is no longer rejected with hundreds of validation errors. The Bedrock-grouped ingress converter tagged user and system input text as output_text, though only input_text is valid on input messages, and omitted the required status on replayed assistant output messages, so Mantle’s strict OpenAI-compatible validator refused the whole request. Replayed assistant messages now carry Status: "completed" and input text blocks are tagged input_text. Bedrock requests with no explicit max_tokens also populate it from the model’s known capacity instead of truncating silently on large context windows (#7074)
• fix: Anthropic server tools such as web_search_20250305 are stripped before a request reaches Fireworks, vLLM or SGLang on their Anthropic-compatible endpoints, which return 400 for tools executed by Anthropic-operated infrastructure that does not exist on third-party hosts; clients whose built-in web search is always on, such as Codex, hit this on every request. ProviderFeatures entries for Fireworks, VLLM and SGL declare all server-tool flags off so the existing validators strip them, StripUnsupportedServerToolsFromRawBody mirrors that on the raw-body passthrough path, the caller’s function tools are kept, and the drops are reported on the response’s DroppedUnsupportedTools instead of failing the call (#7090)
• fix: guardrailConfig reaches Bedrock’s OpenAI-compatible chat/completions and responses surfaces, streaming and non-streaming, which apply guardrails through request headers rather than the body field Converse uses, so a configured guardrail was previously ignored there with no error. withGuardrailHeaders maps guardrailIdentifier, guardrailVersion and optional trace to X-Amzn-Bedrock-GuardrailIdentifier, X-Amzn-Bedrock-GuardrailVersion and X-Amzn-Bedrock-Trace, then deletes the key from ExtraParams so it is not also emitted into the body. A half-formed config with only one of identifier or version is left untouched rather than sent, since both are required upstream, and the base header map is cloned rather than mutated (#7095)
• feat: use_openai_endpoints on Bedrock keys and aliases routes chat completions and responses through Bedrock’s OpenAI-compatible /openai/v1 endpoints instead of Converse, for models that support them, mirroring the existing use_anthropic_endpoints pattern. It is opt-in by design: Converse carries Bedrock Guardrails, performanceConfig and requestMetadata that the OpenAI-compatible surface silently ignores, so diverting automatically could stop a guardrail from being enforced with no visible error. ResolveUseOpenAIEndpoints gives the alias value precedence over the key, and the narrow runtimeServesResponses check is replaced by a general runtimeServesOpenAIAPI that takes a BedrockAPI discriminator; chat completions, streaming and non-streaming, gained the runtime path that previously existed only for Responses (#7073)
• fix: Bedrock runtime models that serve the Responses API are routed to it through a dedicated surface resolver rather than falling back to Converse (#7071)
• fix: Bedrock Converse document blocks always ship their content base64-encoded through source.bytes, never through source.text, which Converse rejects unless citations are explicitly enabled. Plain text formats (text/plain, text/markdown, text/csv, text/html) previously took the text branch and failed with DocumentSource must set one of the following keys: bytes, s3Location; the branch is removed for data URLs, percent-encoded payloads and the non-data-URL file_data path alike (#7072)
• fix: the Bedrock Mantle base path is resolved from the model capabilities datasheet instead of hard-coded string matching in two packages. Mantle serves each model on exactly one of v1 or openai/v1 and returns a 400 on the other, and the previous matching covered only generations up to GPT-5 and Gemma 4, so GPT-6 and any future closed-generation model silently fell through to the wrong path. ResolveBedrockMantleBasePath applies family-name detection as a fallback and defers to the new BedrockMantleBasePath field on ModelCapabilities, so a new generation needs a datasheet row rather than a code change (#7077)
• fix: image_generation_call items where OpenAI emits action as a bare JSON string, such as "generate", decode correctly. ResponsesToolMessageActionStruct.UnmarshalJSON immediately peeked at a .type field, which cannot be read from a JSON string, so the whole decode failed with failed to peek at type field; that silently dropped the response.output_item.done and response.completed events carrying the image, leaving the stream without a terminal event and surfacing as a bogus provider closed the stream truncation error. The action is now tried as a bare string before falling back to the object type-peek, and ResponsesImageGenerationCall keeps the background, output_format, quality, revised_prompt and size settings OpenAI echoes back on completed items (#7060)

• fix: virtual key rate limits and budgets created via the config.json standalone-limits flow and later edited through the UI no longer produce duplicate, conflicting ownership records; migration migrate_vk_standalone_limits_to_model_configs consolidates ownership into VK-scoped model configs, preserving usage counters, and the write paths stop creating the divergent rows (#7291)
• feat: virtual key listing batch-resolves assignees (assigned_user) instead of resolving per row, and VK search matches team and customer names in addition to key fields (#7225)
• chore: OTEL dependency updates (#7311)
• fix: Responses API requests emit gen_ai.response.finish_reason / gen_ai.response.finish_reasons on the OTEL llm.call span. PopulateResponsesResponseAttributes never copied the top-level stop_reason into the span attributes, so refusals on /v1/responses were invisible to OTEL consumers even though the stop reason reached the Logs DB; it now emits finish_reasons like the chat path and the tracer derives the singular key from it (#7204)
• feat: model allow and block lists accept regex: entries. The provider-key aggregate keeps an exact name next to a pattern that also covers it, the catalog allow check tries a pattern against the bare name and provider/model, and listings skip pattern entries
• fix: ClickHouse log store deletes no longer run as heavyweight ALTER TABLE ... DELETE mutations. The retention cleaner issued one such mutation per 100 rows, each rewriting the whole current-month part, and the once-a-minute stale-processing sweeps issued one per table unconditionally, filling replica disks in minutes. Every delete on the ClickHouse store (retention sweep, Flush/FlushMCPToolLogs, UI log deletes, async job and webhook delivery expiry) is now a single lightweight DELETE FROM ... WHERE per run, skipped entirely when nothing matches. The table TTL derived from logs_store.retention_days is now reconciled on every startup with MODIFY TTL (metadata only), so changing the value reaches existing tables; 0 leaves an existing TTL untouched (#7098)
• feat: access profiles and governance projects reference Virtual MCPs through virtual_mcp_name, so config files are portable across environments instead of carrying database-assigned integer IDs. Names resolve to stored records on startup and a name that matches nothing is refused; virtual_mcp_id is deprecated, still accepted, and wins when both are set. mcp_configs ({ mcp_client_id, tools_to_execute }) replaces the mcp_servers and mcp_tool_overrides include-exclude model with a single allowlist, where ["*"] grants all tools including future ones, [] grants none, and a named list grants only those; the old keys are deprecated, still accepted, and folded into the new shape at load time (#7181)
• feat: MCPToolLog records governance entity names beside their IDs, so MCP tool logs carry the same attribution shape the logs table has instead of rendering raw UUIDs in the dashboard. user_name, team_name, customer_name and business_unit_name stop being gorm:"-" transients and become storage; the multi-valued team_ids/team_names, customer_ids/customer_names and business_unit_ids/business_unit_names sets are stored as index-aligned JSON arrays; and budget_ids and rate_limit_ids are recorded id-only, as in logs. Names are written from the request context at ingestion through the new MCPToolLog.ApplyGovernance in framework/logstore/governance.go, with nothing resolved on read. Added by migration mcp_tool_logs_add_governance_snapshots (#7154)
• feat: endpoint-attributed MCP observations are carried in the standard logging pipeline, so inspected MCP tool calls are logged with bounded identity (device, app key, server label, tool name, decision) sourced from the gateway rather than from payload-supplied headers. SetMCPObservation attaches the attribution to a BifrostContext and applyMCPObservation writes it onto the MCPToolLog from both PreMCPHook and PostMCPHook, snapshotting the observation so it cannot alias across async log entries (#6959)
• feat: the error_type classification vocabulary is threaded through the framework so bifrost_error_requests_total can carry a normalized fault-domain label alongside status_code (#7141)
• feat: model pricing supports time-of-day peak and off-peak rates. TableModelPricing gains off_peak_cost_multiplier (a nullable float) and peak_hours (a JSON-serialized PeakHoursSchedule of recurring weekly windows using IANA timezone names, weekday numbers and half-open HH:MM intervals that may wrap past midnight), added by migration add_time_of_day_pricing_columns and aliased into the datasheet package so the JSON shape stays self-contained. The cost engine evaluates the schedule against the request start time and scales usage-based charges by the multiplier when the request falls outside every peak window, applied once in computeCostFromInput so every modality is covered; flat CostPerRequest and SearchQueriesCost fees are excluded, as is AdditionalCost, which is discounted independently through its own pricing rows. Both fields are exposed on PricingPatch in the OpenAPI and governance schemas, with off_peak_cost_multiplier bounded to (0, 1] (#6574, #6575, #6576)
• feat: SecretVar.RedactedIfSecret() is used for non-credential fields in ProviderConfig.Redacted(), Config.GetAllKeys() and Config.RedactMCPClientConfig(), so regions, endpoints, service URLs and MCP connection strings surface as plaintext while anything env-var or vault-backed stays masked (#7085)
• feat: GA realtime transcription sessions are routed, governed, logged and priced through the normal framework pipeline, resolving the routing model from the nested audio.input.transcription.model that arrives in session.update rather than from the connect URL (#7089)
• feat: a use_openai_endpoints column on the provider keys table, added by migration add_use_openai_endpoints_column, opts a Bedrock key or alias into Bedrock’s OpenAI-compatible endpoints instead of Converse (#7073)
• feat: the reserved tool-namespace list a provider keeps for its own server tools is read from the datasheet row reserved_tool_namespaces, so a namespace collision check no longer requires a code change (#7084)
• fix: exchangeRefreshToken includes client_secret only when the secret is non-empty, matching exchangeCodeForTokensWithPKCE. Public OAuth2 clients registered against servers that support only token_endpoint_auth_method: none have no secret, and unconditionally setting client_secret= sent an empty client_secret_post attempt that strict authorization servers answered with invalid_client, flipping the token row to needs_reauth even though the refresh token was valid (#7042)
• fix: the virtual key allowed_models: ["*"] handling for governance added in #6767 is reverted, returning the wildcard-with-empty-synced-catalog case to its previous behaviour (#7053)

• chore: upgraded core to v1.9.1 and framework to v1.7.1
• fix: removed the namespace-tool flattening that ran under should_convert_params; it spliced nested functions into the top-level list without a namespace prefix, so two namespaces sharing a function name produced duplicate tool names and an upstream 400. Flattening now lives in Bifrost core for every provider whose wire lacks the namespace type, with unique <namespace>__<function> names and response-side mapping back. should_convert_params is still accepted so existing configs load, but it no longer changes any request (#7048)

• fix: the billing-idempotency key mixes in the internally minted BillingNonce, so caller-forged x-request-id duplicates can no longer suppress charges (#7303)
• fix: virtual key credentials are validated before realtime session admission (#7317)
• chore: upgraded core to v1.9.1 and framework to v1.7.1
• feat: MCP tool logs record governance entity names beside their IDs, so the plugin stamps user_name, team_name, customer_name, business_unit_name and the multi-valued team, customer and business-unit sets onto the log entry at ingestion instead of leaving the dashboard to render raw UUIDs (#7154)
• feat: endpoint-attributed MCP inspections carry bounded identity (device, app key, server label, tool name, decision) from the gateway into the MCP authorization path, so an inspected tool call is attributed without trusting payload-supplied headers (#6959)
• feat: governance errors are classified into the normalized error_type vocabulary, so a 429 raised by a governance rate limit is distinguishable from an upstream 429 on bifrost_error_requests_total (#7141)
• feat: model allow and block lists accept regex: entries. The *_patterns fields added earlier in this release window were withdrawn in favour of the prefix form, which needs no new schema fields (#6988, #7133, #7134)
• fix: UsageTracker.Cleanup() cancels and waits for the periodic reset worker before taking its final budget and rate-limit snapshots. It previously dumped first, so trackerCancel() could cancel an in-flight rate-limit dump and fail with failed to dump rate limits to database: context canceled, and the final dump was not guaranteed to be the tracker’s last database writer. A queued ticker event can no longer start another reset cycle during shutdown, the current cycle stops when the tracker context is cancelled, and context.Canceled is treated as an expected result only when that context was actually cancelled (#7099) @Constantine3

• chore: upgraded core to v1.9.1 and framework to v1.7.1

• chore: upgraded core to v1.9.1 and framework to v1.7.1
• feat: MCP tool log rows carry governance entity names alongside their IDs, written from the request context at ingestion, so nothing is resolved on read (#7154)
• feat: endpoint-attributed MCP observations are written onto both the pending and the final MCP tool log entry, with the observation snapshotted so it cannot alias across async entries (#6959)
• feat: cost recalculation honours time-of-day peak and off-peak pricing, scaling usage-based charges by off_peak_cost_multiplier when a request falls outside every declared peak window (#6575)
• feat: GA realtime transcription sessions are logged and priced through the standard pipeline with transcription-aware pricing (#7089)
• fix: a request whose caller disconnected before the upstream finished is logged and finalized deterministically rather than roughly half the time, so abandoned requests no longer leave a log row stuck in its pending state (#6972)
• docs: clarified that CountRecalcTargets reads a materialized view that can lag, so its Total is an approximation rather than an exact count (#7078)

• chore: upgraded core to v1.9.1 and framework to v1.7.1

• chore: upgraded core to v1.9.1 and framework to v1.7.1

• chore: upgraded core to v1.9.1 and framework to v1.7.1

• chore: upgraded core to v1.9.1 and framework to v1.7.1

• chore: upgraded core to v1.9.1 and framework to v1.7.1

• chore: upgraded core to v1.9.1 and framework to v1.7.1
• fix: a continuation turn, such as a tool result following a prior user message, is classified instead of skipped when no active session state is found. BuildInputWithDisposition now returns the populated ComplexityInput including LastUserText for trailing-continuation turns in both the chat and responses paths, and computeComplexity falls back to classifying that recovered text, so new or recovered sessions get a tier assignment; the skip path applies only when LastUserText is also empty (#7122)

• chore: upgraded core to v1.9.1 and framework to v1.7.1

• feat: metrics export includes user id and user name labels, so per-user attribution is available in Prometheus (#7267)
• chore: upgraded core to v1.9.1 and framework to v1.7.1
• feat: bifrost_error_requests_total carries a normalized error_type label alongside status_code, drawn from a closed prefix-structured vocabulary (caller_*, policy_*, provider_*, bifrost_*, _OTHER), so alarm expressions can separate fault domains with a single regex instead of enumerating status codes (#7141)