Skip to main content
​
v2.3.1

​
Changelog

v2.3.1 moves the enterprise gateway onto OSS transports v2.3.1. Cluster monitoring adds per-node CPU, memory, request and gossip charts, a Cluster > Storage page with database health, and cluster incidents for memory, dropped messages and database anomalies. Guardrails add a monitor output mode that checks responses in the background, and an Incidents tab for blocked and flagged requests. The vault plugin adds Azure Key Vault and HashiCorp Kubernetes auth. The OSS release adds OpenAPI MCP servers, gateway tool search, passthrough logs as conversations, and virtual key metadata.

​
✨ Features

  • Cluster Monitoring - Every node samples process CPU, resident memory against its detected limit (cgroup v2, cgroup v1, GOMEMLIMIT or host), requests per second, and gossiped messages per minute once a minute. Samples are kept per minute for a day and as hourly rollups after that, with their own retention. New pages under Cluster: Nodes (per-node charts and a nodes table with connectivity), Incidents, and Settings. APIs are GET /api/cluster/metrics/current, GET /api/cluster/metrics/series, GET /api/cluster/incidents and GET/PUT /api/cluster/monitor/config. Cluster health conditions also appear in the main notification tray. Docs
  • Cluster Incidents - A node opens a memory_high incident when memory crosses a configurable share of its limit, and a drop-window incident when gossip messages to peers fail (one row per window, with per-peer delivered and failed counts). Database anomalies open database incidents: per node for a saturated pool or query p95 at or above 500 ms, and database-wide for high connection use, lock waits, long-running queries, replica lag, and low disk or too many parts on ClickHouse. The database_incidents setting turns database incidents off.
  • Cluster > Storage Page - Shows each store’s engine, location and health: Postgres connections, throughput, largest tables, long-running and blocked sessions; ClickHouse sizes, parts, merges and disks; SQLite pragmas. A shared Postgres database is inspected once. The report is re-read every 5 minutes, and the page also charts pool use, waits and query p95 per store and lists slow queries over 500 ms with their literals removed (/api/cluster/storage, /api/cluster/storage/series, /api/cluster/storage/slow-queries).
  • Guardrail Monitor Output Mode - Output rules accept output_mode: enforce (default) or monitor. A monitor rule evaluates the complete response in the background, after it is sent, and logs the matches without blocking or changing the response. Every monitored response is evaluated, with no concurrency cap. Docs
  • Guardrail Incidents Tab - Guardrails > Incidents lists requests blocked or flagged by a guardrail, with outcome, virtual key and time range filters. Blocked log rows now record the rule and guardrail that fired for input, output, MCP input and MCP output checks. Lakera guardrail logs include the provider request ID, and the dashboard has a guardrails chart.
  • Azure Key Vault - The vault plugin accepts type: "azure-key-vault" with an azure config block, next to AWS Secrets Manager, GCP Secret Manager and HashiCorp Vault. Docs
  • HashiCorp Vault Kubernetes Auth - HashiCorp Vault logs in with the pod’s service account token when kubernetes_role is set. kubernetes_mount_path (default kubernetes) and kubernetes_token_path (default the standard service account token path) are optional. Tokens are renewed in the background and re-issued by logging in again.
  • Datadog LLM Observability Export - The Datadog connector converts only model-call spans to LLM Observability spans, so plugin and HTTP spans no longer appear as extra workflow spans, and it sends the token and cost metric names Datadog recognises. New settings otel_llmobs_conversion, llmobs_gateway_mode and cost tags are in the connector form. Spans also carry the prompt, tool definitions, tool results, and the caller’s LLM Observability application from tracestate.
  • Prompt and A2A Fields in Connectors - BigQuery, Splunk and Datadog export the stored prompt a request ran (prompt_id, prompt_name, prompt_version). BigQuery, Datadog, Kafka and Pub/Sub export A2A agent operations, and the Kafka connector form shows the A2A option.
  • Antigravity in Edge App Controls - Antigravity CLI and Antigravity Desktop can be configured in Edge app interception settings, with their own domains and inspection rules.

​
🌎 Open Source Features

  • OpenAPI MCP Servers - New openapi MCP connection type. Bifrost turns an OpenAPI or Swagger document (inline spec, spec_file in config.json, or spec_url, up to 5 MiB) into an in-process MCP server, with one tool per operation, security_credentials per security scheme, and an optional base_url. It can be set in config.json, the MCP API, or the New MCP Server sheet. Docs
  • Gateway Tool Search - Requests that defer tools with defer_loading on a model that has no tool search of its own (Gemini, older GPT, Bedrock non-Claude) get a bifrost_tool_search function that the gateway resolves (select: or BM25, at most 3 model calls per request), on streaming and non-streaming Responses requests.
  • Passthrough Logs as Conversations - With content logging on, passthrough calls (/openai_passthrough, /anthropic_passthrough, /genai_passthrough and others) are parsed into the same fields as typed requests. Streamed responses are rebuilt from the recorded SSE. Rows are stored under their inferred request type with an is_passthrough flag and keep the raw bodies.
  • Extra Fields as Response Headers - New client.extra_fields_placement setting: body (default) or headers. With headers, inference responses send extra_fields as x-bifrost-* response headers instead of in the body, for unary and streaming calls.
  • Virtual Key Metadata - Virtual keys carry string key/value metadata (up to 50 entries) set from config.json, Helm, the API or the virtual key sheet. The list API filters by metadata_<key>=<value>, each log keeps a snapshot of its key’s metadata for log and spend filters, and the key’s values override caller-sent logging headers and dimensions. Docs
  • News Feeds - The notification tray has a News tab with product and provider announcements from RSS and Atom feeds (Bifrost, OpenAI, Anthropic, Gemini and Cursor built in). Admins can turn built-in feeds off and add custom feeds with their own logo.
  • Code Mode Scripting - The Starlark sandbox adds the json and math modules. A failed nested tool call reaches the script as an error value (r.error), the executeToolCode reply carries only what the script prints and returns, and runtime errors include a traceback. Docs
  • Native Decisions on Azure and Databricks - /v1/decisions reaches Microsoft-Decision-1 on Azure (API key, Entra ID or managed identity auth) and Databricks ai_decide. Typesafe accepts images by reference and answers box questions. Docs
  • Vertex Session Routing - Vertex keys accept use_session_routing: true, which sends the request’s prompt cache key as sessionId on Gemini generateContent calls so requests that share a prefix reach the same implicit cache. Off by default.
  • Secret References in Extra Headers - network_config.extra_headers values accept env. and vault. references. Values are masked in API responses and restored on save.
  • Prompt Caching on Tool Calls - cache_control on an assistant message’s tool_calls entry on /v1/chat/completions now reaches the Anthropic and Bedrock tool_use block.
  • Mistral Thinking Blocks - Mistral streams with thinking blocks map them to reasoning, and assistant reasoning is removed before a request is sent, so replayed conversations no longer get Mistral’s 422 extra_forbidden.
  • Observability Attributes - Spans and Prometheus metrics carry the stored prompt (bifrost.prompt.id, name, version). MCP spans carry the client name and bifrost.mcp.tool.is_error. A2A operations export bodies, stream updates and the a2a.client.operation.duration metric.
  • Calendar Date Presets - The date filter adds Today, This week and This month, resolved in the caller’s time zone.
  • MCP OAuth Redirect URI Wildcard - oauth2_server_config.allowed_redirect_uris syncs from config.json and accepts "*" to approve any remote callback. A startup warning and a dashboard alert flag the wildcard.
  • Antigravity Client Detection - Requests from Antigravity CLI and Antigravity Desktop are logged as their own apps, including when they arrive through Edge.

​
🐞 Fixed

  • Required Headers and Key Ownership in Governance - The enterprise governance plugin now enforces required_headers and virtual key object ownership, which it did not receive before.
  • Upgrade Migrations Stalling - Enterprise migrations no longer stall behind long transactions on old replicas. Audit log indexes are built in the background after startup, ALTER steps use a 5s lock timeout and retry, and migration_lock_retry_budget is honoured.
  • Concurrent Index Builds on Rolling Upgrades - A new pod waits for a live CREATE INDEX CONCURRENTLY from a killed pod instead of dropping it, which caused tuple concurrently updated on the v2.3.0 upgrade.
  • Index Migration Lock Wait - The enterprise migration lock waits up to 20 minutes while its holder is visibly running statements, instead of failing the boot.
  • Self-Peer in Cluster gRPC Pool - A node no longer adds itself as a gRPC peer, which made a single-node cluster report message drops against itself.
  • Request Body Memory on the Enterprise Server - The enterprise server applies the body buffer pool limit, so pooled connections no longer keep their largest request body.
  • Oversized Request Bodies - A body above the configured limit returns 413 naming the limit instead of a generic 400.
  • OpenAPI MCP Servers on Enterprise - The enterprise server registers the OpenAPI MCP factory, so openapi MCP clients start.
  • Home Budget Burn Lines - Burn lines are drawn per budget cycle, so hourly budgets no longer spike to hundreds of percent and daily budgets no longer show spend before their reset. A line that would need too many resets is hidden with a note.
  • Home MCP Access - My Access lists MCP servers granted through Virtual MCPs in their own table with each Virtual MCP’s /mcp/<slug> endpoint, and says when a Virtual MCP’s details could not be loaded.
  • Home Spend Rank Removed - The Spend rank card, the rank jobs, GET /api/governance/users/me/rank and the weekly badge are removed. Budget requests show when they were filed and the granted amount when it differs from the ask.
  • Native MCP Tool Logs from Edge - An approved native tool call and its later result share one log row, and device lookup for MCP logs uses the registered device.
  • Edge Signing After the Legacy CA Cutoff - Edge devices past the legacy CA key cutoff are always sent to remote signing.
  • Calendar Alignment in Governance Forms - The enterprise budget and rate limit forms describe calendar alignment correctly.
  • Identity Sync Mapping Names - Attribute mappings store the mapped entity’s name next to its ID.
  • Circuit Breaker Policy Form - The policy sheet validates with a schema, keeps the provider selector from remounting on each keystroke, and blocks double submits.
  • Go 1.27.2 - The enterprise build moves to Go 1.27.2. dd-trace-go moves from 2.8.2 to 2.11.1.
  • OSS: Go Module Path - The transports module is published as github.com/maximhq/bifrost/transports/v2, so go get resolves the v2.x tags.
    Go code that imports github.com/maximhq/bifrost/transports/bifrost-http/... must switch to github.com/maximhq/bifrost/transports/v2/bifrost-http/....
  • OSS: Explicit Public URLs for Agent Gateway and Skills - Agent-card endpoints and skills marketplace clone URLs come only from the configured public URL, never the request Host header. An existing public mcp_external_client_url is copied to the new skills_external_client_url on upgrade, and the MCP settings page requires an Issuer URL in OAuth and Both modes.
    If your Agent Gateway or skills marketplace relied on the request host, set the public URL explicitly before upgrading, or clients receive no usable agent-card endpoint.
  • OSS: Agent Gateway Destination Permissions - Agent connections reach only approved destinations and forward credentials only to permitted origins (credential_origins, require_public_target). The unused tenant and extension_uris options are removed.
  • OSS: A2A Goroutine Leak - The A2A push relay no longer leaks one goroutine every 3 seconds (about 1,200 per hour on v2.3.0).
  • OSS: ClickHouse agent_logs Mutations - ClickHouse no longer runs a heavy agent_logs mutation every minute, and agent-log correlation no longer loads large payload columns.
  • OSS: Calendar-Aligned Reset Windows - Calendar-aligned budgets and rate limits honour the duration multiplier, so a 7d budget no longer resets daily.
  • OSS: Agent OAuth Cache - Agent Gateway OAuth token and fingerprint caches are bounded at 5,000 entries.
  • OSS: Request Body Memory Retention - Buffers above server.body_buffer_pool_limit_bytes (default 4 MiB) are released after each request, and async jobs copy the raw body.
  • OSS: Rolling Upgrade Migrations - Logstore migrations no longer stall behind long queries on old replicas, and the Helm chart adds a configurable startupProbe.
  • OSS: xhigh Reasoning Effort - reasoning_effort: "xhigh" is forwarded to vLLM, Ollama, SGL and custom OpenAI-compatible providers.
  • OSS: Tool Search Round-Trips - Deferred tools on OpenAI gpt-5.4+ get the required tool_search, and tool search blocks replay unchanged across Anthropic, OpenAI and Vertex routes.
  • OSS: Client web_search and web_fetch Tools - A client tool_use named web_search or web_fetch stays a function call, so Bedrock no longer rejects the replay.
  • OSS: Bedrock Prompt Caching - Top-level cache_control creates a Bedrock cache point, and cache_control on a tool message emits one cachePoint.
  • OSS: OpenRouter Claude Cache Markers - cache_control markers to Claude on OpenRouter are clamped to four.
  • OSS: Parallel Tool Calls on Claude - parallel_tool_calls: false reaches Claude on Anthropic Responses, Bedrock and Vertex.
  • OSS: Responses Instructions - Top-level instructions are kept when the input also has a system or developer message.
  • OSS: Chat Tool Results on Responses Models - Array tool content and empty tool results convert correctly, so OpenAI and Azure no longer reject them.
  • OSS: Responses Streaming Wire Format - Lifecycle events no longer send output: null or invalid item objects.
  • OSS: Responses Input Items - configuration_update items keep their reasoning effort, and replayed assistant messages keep their status.
  • OSS: Converted Chat Stream Identity - Chat streams served through the Responses conversion carry the upstream id, model and created on every chunk.
  • OSS: Gemini Prompt Blocks - A prompt that Gemini or Vertex blocks returns an error instead of an empty response.
  • OSS: Reasoning Token Usage - Gemini and Vertex batch results include thinking tokens, and SGLang reasoning tokens are mapped.
  • OSS: Bedrock Converse Round-Trip - Converse responses keep stopReason and additionalModelResponseFields.
  • OSS: Prompt Cache Mode on Bedrock Converse - prompt_cache_options on /bedrock Converse reaches OpenAI, Azure and Bedrock Mantle gpt-5.6 models.
  • OSS: Azure Deployments Route - Unknown chat fields are forwarded on /openai/deployments/....
  • OSS: Decisions - Custom OpenAI-base providers are gated on /v1/decisions by their allowed requests, and string-encoded answers are recovered.
  • OSS: Zero-Key Custom Providers in Routing - A custom provider with no keys no longer claims every model, and no_key_supports_model no longer stops the fallback chain.
  • OSS: Keyless Provider Discovery - A provider switched to keyless runs model discovery despite leftover disabled keys, and those keys can be deleted.
  • OSS: Provider Key enabled in config.json - enabled on a provider key in config.json or Helm persists across restarts.
  • OSS: MCP Endpoint Slugs - A config.json MCP client without endpoint_slug keeps its derived slug, so /mcp/<slug> no longer returns 403 after a reboot.
  • OSS: MCP Library Installed Badge - “Installed” is computed over every MCP client, not only the first 100.
  • OSS: Code Mode Tool Results - Restored tools keep their parameters, and nested results arrive as structured values.
  • OSS: OAuth Refresh Failures - Slack and Atlassian refresh failures prompt re-auth.
  • OSS: Throughput Metric - Dashboard tokens/sec counts only inference requests.
  • OSS: Passthrough Logs - Passthrough log keys no longer show “(deleted)”, and streams without a Content-Type are no longer labelled text/plain.
  • OSS: SQLite Log Writes - SQLITE_BUSY and SQLITE_LOCKED are transient, and a failed batch is retried whole.
  • OSS: Model Catalog Prices - Per-1M token prices show up to 4 decimals.

​
🗄️ Database Migrations

This release includes logstore migrations on the logs table (logs_add_matview_refresh_state_table, logs_add_is_passthrough_column). Adding a column to logs needs a short exclusive lock, which has to wait for in-flight log writes and long-running queries. Deploy during a low-traffic window.

​
Enterprise

  • ent_add_guardrail_rule_output_mode_column - Adds output_mode (NOT NULL DEFAULT 'enforce') to enterprise_guardrail_rules. Reversible: rollback drops the column.
  • ent_add_cluster_incidents_table - Creates enterprise_cluster_incidents in the config store for memory and dropped-message incidents. Non-reversible: dropping it would delete every recorded incident. Additive.
  • ent_add_cluster_incident_database_columns - Adds the anomaly, store, peak, threshold, last and detail columns to enterprise_cluster_incidents. Non-reversible: dropping them would delete the recorded database anomaly details. Additive.
  • ent_drop_user_rank_tables - Drops enterprise_user_rank_periods and enterprise_user_rank_snapshots. Leftover rank jobs stay in the job queue. Rollback is a no-op and does not restore the tables. governance_users.badges stays in place, unused.
    This migration drops tables. An older binary running alongside it during a rolling upgrade can no longer read spend ranks, and rolling back does not restore them.
  • ent_logs_add_cluster_metrics_tables - Creates enterprise_cluster_node_samples and enterprise_cluster_node_rollups in the log store. ClickHouse creates them on boot with a TTL from the cluster monitoring retention. Reversible: rollback drops both tables.
  • ent_logs_add_cluster_store_stats - Adds the nullable config_* and logs_* pool and query columns to the sample and rollup tables, and creates enterprise_cluster_slow_queries. ClickHouse adds the columns on boot. Reversible, but rollback deletes every recorded slow query and per-store figure.
  • ent_logs_add_cluster_message_counts - Adds messages_sent and messages_failed to the sample and rollup tables. ClickHouse adds them on boot. Reversible: rollback drops the columns.

​
Open Source

  • add_mcp_client_openapi_spec_column - Adds openapi_spec to config_mcp_clients. Non-reversible: dropping it would delete the only stored copy of every OpenAPI client’s document. Additive; older binaries ignore it.
  • add_mcp_client_openapi_config_json_column - Adds openapi_config_json to config_mcp_clients. Non-reversible: dropping it would delete every OpenAPI client’s base URL and stored credentials. Additive.
  • add_client_config_extra_fields_placement_column - Adds extra_fields_placement (NOT NULL DEFAULT 'body') to config_client. Reversible: rollback drops the column.
  • add_skills_external_client_url_column - Adds skills_external_client_url to config_client. Non-reversible: dropping it would discard marketplace configuration. Additive.
  • backfill_skills_external_client_url - Copies mcp_external_client_url into skills_external_client_url where it is empty and the value is a public URL. Non-reversible: migrated marketplace configuration is preserved.
  • add_virtual_key_metadata_column - Adds the nullable metadata column to governance_virtual_keys. Non-reversible: dropping it would delete every key’s metadata. Additive.
  • add_a2a_private_allowlist_column - Adds a2a_private_allowlist_json to config_client. Reversible: rollback drops the column.
  • add_agent_destination_policy_columns - Adds credential_origins and require_public_target to config_agent_registrations. Non-reversible: removing them would discard security policy. Additive.
  • drop_agent_registration_options - Drops tenant and extension_uris from config_agent_registrations. Non-reversible: the removed values cannot be recovered.
    This migration drops columns. An older binary running alongside it during a rolling upgrade loses access to those values, and rolling back does not restore them.
  • add_vertex_use_session_routing_column - Adds vertex_use_session_routing to the provider keys table. Reversible: rollback drops the column.
  • add_rss_feeds_table - Creates rss_feeds and seeds the built-in feeds. Non-reversible: dropping it would delete custom feeds, their logos and every built-in enabled choice. Additive.
  • add_rss_feeds_icon_url_column - Adds icon_url to rss_feeds tables created before it existed. Non-reversible: dropping it would delete every custom feed logo. Additive.
  • logs_add_matview_refresh_state_table - Creates matview_refresh_state to record when each materialized view last refreshed across the cluster. Reversible: rollback drops the table.
  • logs_add_is_passthrough_column - Adds is_passthrough to logs. Rollback is a no-op and leaves the column in place. Additive.

​
🐙 Closed OSS Issues

  • #3440 - Add nodeSelector support for all workloads managed by Helm Chart
  • #6193 - Cannot pass xhigh reasoning effort to vLLM provider
  • #6499 - Add “today” “this week” “this month” to date filter
  • #7763 - All /mcp/<slug> endpoints return 403 access_denied on v2.1.1-enterprise
  • #7933 - Custom metadata on virtual keys (API, UI, and spend filtering)
  • #8078 - Anthropic endpoint converts a client tool_use named web_search/web_fetch into a server tool call (Bedrock 400)
  • #8324 - Model catalog rounds per-1M token prices to 2 decimals
  • #8359 - ClickHouse store still issues a heavyweight mutation on agent_logs every minute
  • #8361 - /anthropic/v1/messages to OpenAI gpt-5.4+ fails with 400 when tools have defer_loading but no tool_search tool
  • #8383 - calendar_aligned budgets and rate limits ignore the numeric part of reset_duration
  • #8500 - Keyless provider with residual disabled keys never discovers models
  • #8541 - Bedrock: top-level cache_control is dropped, automatic caching never creates a cache point
  • #8548 - MCP Library “Installed” badge only checks the first 100 clients
  • #8554 - Bedrock Converse cache points no longer limit gpt-5.6 caching to the system prompt in v2.3.0
  • #8568 - Empty tool message breaks requests to Responses API models
  • #8590 - goroutine count climbs ~1,200/hour on v2.3.0 until OOM
  • #8616 - Responses API configuration_update input items drop the reasoning field
  • #8684 - Gemini and Vertex batch results ignore thinking tokens
  • #8709 - A2A push relay leaks one goroutine every 3s (v2.3.0)

​
📀 Base OSS version

transports/v2.3.1 (github.com/maximhq/bifrost/transports/v2 v2.3.1), with core v1.13.0, framework v1.10.0, governance v1.10.0, and logging v1.10.0.

​
🔌 If you are compiling plugin against this release - use following deps