v2.0.0-prerelease3
Changelog
Third prerelease on the v2.0.0 line, built on OSStransports/v2.0.0-prerelease3. The enterprise side adds Prompt Guardrails (a natural-language rule classifier that runs as a guardrail provider), MCP guardrails with redaction and transformations, a canonical /api/governance route namespace with legacy aliases kept alive, custom logo and icon branding, and server-side signing for Edge device trust so signing key material is never distributed to devices. On the OSS side this release folds in everything from transports/v1.6.5 through v1.6.10 and the prerelease3 line: per-user MCP OAuth, per-model budgets and quarterly budget windows, virtual key budget overrides, async webhooks, and a large batch of provider and streaming fixes.✨ Features
- Prompt Guardrails - A new guardrail provider that classifies request and response content against a natural-language rule you write, with configurable model, output token ceiling, and timeout. It fails open on uncertainty by design, so only clear rule violations block. Docs
- Guardrail Debug for Prompt Guardrails - Prompt guardrail evaluations report their own token cost and debug output through
guardrail_debug, so the cost of running a classifier on traffic is visible per request. - MCP Guardrails - Guardrail rules can now target MCP tool traffic, not just model traffic, including redaction and transformation actions on MCP tool inputs and results, with backend config and a rules UI.
- Canonical
/api/governanceNamespace - RBAC, user, team, virtual key, access profile, business unit, SCIM and audit log routes now live under/api/governance. Legacy paths keep working through registered aliases, RBAC resource mapping follows the canonical paths, and the enterprise UI calls the new ones. - Custom Branding - Logo and icon overrides are stored in a new enterprise branding table and served through
GET/PUT/DELETE /api/brandingplus an asset route, so the dashboard shell renders your brand instead of the default one. - Server-Side Signing for Edge Device Trust - Trust material for enrolled devices is now issued and signed by the server. Devices no longer receive long-lived signing key material, the signing endpoint is rate limited, and each device carries a
remote_signing_capableflag so a fleet can be migrated in place. - Signed Agent Responses - Trust-relevant agent-facing responses are signed with an Ed25519 key that is independent of the interception key material, so an agent can detect a forged response even if the transport or a bearer credential is compromised.
- Encrypted Key Material at Rest - A migration re-encrypts any legacy plaintext private key found in the stored agent config, so key material saved by older releases is protected at rest.
- SCIM Attribute to Access Profile Mappings - IdP attribute values can be mapped directly to access profiles, with schema support, validation and normalization, auto-assignment during import, role sync and recompute paths, and a mappings editor in the SCIM wizard.
- Okta
SyncAllUsersToggle - The Okta SCIM provider can sync non-active users as well, excluding suspended and deprovisioned ones, for organizations that stage users before activation. - Token Exchange with SSO Application Credentials - MCP clients using
use_idp_credentialsreuse the SSO login application’s client id and secret, and those credentials are now resolved unconditionally onto the token exchange IdP so Microsoft Entra ID style flows work without duplicate configuration. - Delegated MCP Token Exchange - Validated IdP tokens and OIDC sessions stamp an inbound bearer on the request context, and a SCIM-backed resolver wires delegated MCP token exchange to whichever SCIM provider is enabled.
- Cluster-Wide MCP Credential Cache Eviction - MCP OAuth token and per-user header credential cache evictions are broadcast cluster-wide, and credential grants are reconciled on user delete so a removed user loses access on every node.
- Cross-Instance MCP Connection State - A new node state store and heartbeat publish each instance’s per-client MCP connection state into the shared KV store, and an aggregate view compares them, so a client that is healthy on one node and unstable on another is visible instead of averaged away.
- MCP OAuth Refresh Worker - A cluster-gossiped refresh worker renews MCP OAuth tokens and triggers a reconnect hook, plus a
needs reauthgossip action that closes sessions requiring re-authorization. - Audit Log Severity - Audit log entries carry a severity level, set through the audit middleware, so high-impact administrative actions can be filtered apart from routine ones.
- First-Time Admin Bootstrap Token - A one-time token flow creates the first admin user, replacing the previous manual bootstrap step.
- Security Headers and
robots.txt- Enterprise bootstrap adds a security headers middleware and arobots.txtroute, and a skills orphan cleanup worker removes dangling skill records. - Device Page User Filters and Additive Sync - The devices page can be filtered by user, and device inventory sync is additive instead of replacing the stored set, so a partial sync no longer drops known devices.
- Separate Allowed Domains Configuration - Allowed domains are configured independently of the rest of the interception policy, so domain scope can be changed without touching other settings.
- Access Profile Aware Virtual Key Resolution -
ensureUserVirtualKeyskips virtual key resolution when the user already has an access profile, removing an unnecessary lookup from the login path. - Enterprise Context Middleware - Every per-request fasthttp context is stamped with the enterprise marker through a dedicated middleware, so downstream plugins can rely on it being present.
- Enterprise Management Postman Collection - A generated Postman collection covers the enterprise management APIs, with dynamic discovery of workspace test commands.
- License Public Key Injection - Dev cluster, broker, harness, connector and pulse build targets inject the license public key through ldflags from the environment, so locally built binaries validate licenses the same way releases do.
- Okta Token CLI - A small
oktatokenCLI acquires Okta authorization code plus PKCE tokens locally for on-behalf-of testing. - Inline User Search in Filter Sidebars - Filter sidebars search users inline instead of loading the full user list.
🌎 Open Source Features
- MCP Per-User OAuth - MCP clients can hold per-user OAuth credentials and per-user headers, configurable from
config.jsonand the UI, with a documented shared versus per-identity token lookup contract and virtual key and user filters on the OAuth grants and MCP auth session sidebars. - Bedrock VPC Endpoints - AWS Bedrock keys can target VPC endpoints, keeping Bedrock traffic on private networking. Docs
- Per-Request Flat-Fee Pricing - A new
cost_per_requestfield flows through datasheet sync, the cost engine, custom overrides and the pricing override form, for models billed per call rather than per token. - Pricing Overrides in the Model Catalog -
/api/models/detailsexposes resolved pricing overrides and catalog rows resolve overrides server-side, so the catalog shows the price actually charged. - Virtual Key Budget Overrides - Temporary budget overrides add
override_amounton top ofmax_limitand run either for a fixed number of reset cycles or until removed, configured throughoverride_mode,override_cycles_totalandoverride_anchor_resetacross the database, governance store, admin APIs and UI. - Per-Model Budgets and Rate Limits - Virtual key provider configs accept budgets and rate limits scoped to individual models, surfaced through a unified budget override manager that groups provider and model budgets together.
- Quarterly Budget Windows - Budgets support a quarterly reset period with a configurable fiscal start month, so a fiscal year that does not begin in January windows correctly.
- Budget Usage Reset Coverage - The reset budget usage flow now covers teams, customers, model limits and provider governance, not only virtual keys.
- User Scope for Routing and Pricing - Routing rules and pricing overrides can be scoped to individual users, with a
user_idCEL variable in routing rules and a user picker in the pricing overrides UI. - Async Webhooks - Webhook delivery for async jobs, with endpoints configurable through
config.json, the admin API and the UI, an SSRF-safe dispatcher with retries, paginated delivery history, and inferencerequest_idpropagation through jobs and payloads. - Background Model Catalog Refresh - Each provider’s list-models response is re-fetched on a
live_models_sync_interval(default one hour,0disables), so models an upstream starts serving after boot appear without a restart. - Stream Truncation Detection - A new SSE truncation interface and EOF handling across providers surface upstream stream death as an error instead of a clean
[DONE]. - MCP Tool Discovery Persistence - Discovered MCP tools persist and resync uniformly across all client types through a hash-gated core callback, surviving restarts and propagating across a cluster.
- Wafer AI Provider - Wafer AI is supported as a provider.
- Lakera and Repello Argus Guardrails - Lakera and Repello Argus are available as guardrail integrations with configuration docs and UI branding.
- Bedrock HTTP/2 PING Keepalives - The Bedrock provider can send HTTP/2 PING frames on idle connections through
http2_ping_interval_in_seconds(0 disables), so quiet streams survive intermediaries that cut idle connections. Docs - Bedrock Batch Role ARN - A
batch_role_arnon Bedrock key config passes a service role to Bedrock batch jobs for S3 access, taking priority over anyrole_arnin the request. - Anthropic Default Fallback Routing - Anthropic’s
fallbacks: "default"preset is preserved through the Bifrost round trip, with the server-side fallback beta header injected for default-routing requests. - Mid-Conversation Tool Changes - The mid-conversation tool changes beta header is supported for Anthropic and Bedrock Mantle.
- Reasoning Token Tracking - Anthropic extended-thinking tokens are tracked as reasoning tokens across chat, responses and passthrough.
- Adaptive Thinking on Raw Passthrough - For adaptive-only Anthropic models, a legacy
thinking.type: "enabled"block is rewritten to the adaptive form on the raw passthrough body as well as the typed request path. - Expanded OTEL Metric Attributes - Metrics carry a service instance id plus team, customer and business unit ids and names, so exported series can be sliced per tenant without post-processing.
- Separate OTEL Metrics Pipeline - The OTEL collector supports a metrics tab independent of traces, with separate headers for traces and metrics.
- OTEL Export Timeout - A new
export_timeoutsetting (default 5 seconds) bounds how long a slow or unreachable collector can hold an export goroutine. - MCP Metrics - MCP metrics are exported through OTEL and the Prometheus telemetry plugin, plus a
resourceparameter on the MCP OAuth handshake. - Throughput Metrics - Tokens per second histogram endpoints, dashboard metrics, and throughput in model rankings and trend data.
- W3C Trace ID Propagation - Requests carry a W3C trace id on the context, so gateway logs join cleanly with upstream traces.
- Roots-Only Log Filter - A
roots_onlyfilter collapses fallback chains into their root entry with child aggregates. - User Agent and App Attribution in Logs - Logs and MCP tool logs record user agent, app, source, decision, app key and device id.
- MCP Log Redaction and Plugin Logs - MCP tool logs carry redaction mappings and plugin logs.
- S3 Log Export Metadata - Additional metadata is written alongside S3 log exports.
- Matview Maintenance Off Switch -
matview_refresh_intervalaccepts"off"to disable log store materialized view maintenance entirely. - Database Connection Controls - New
conn_max_idle_time(default 5 minutes) on both config and logs stores, acache_ttl(default 60 seconds) for password-command credential resolution, and amatview_refresh_timeoutbounding a single refresh pass. - Object Storage Archival Settings - New
archiveInterval,archiveGracePeriodandarchiveMaxObjectBytessettings, plus a toggle to always retain request and response content regardless of retention cleanup. - Cancellable Log Cost Recalculation - Log cost recalculation tasks can be cancelled from the backend.
- Routing Rule Validation - Routing CEL expressions and
scope_idreferences are validated at write time in create and update handlers. - Routing Info Headers - Routing info headers are emitted for streaming responses, inference and integration APIs, and error and passthrough paths.
- Access Profile Config Schema -
config.schema.jsonacceptsblacklisted_models(a denylist that wins overallowed_models), aweightseed for weighted routing, andmodel_budgetson access profile provider configs. - SSO Additional Scopes -
config.schema.jsonacceptsadditionalScopes, requesting extra OAuth scopes on top of the base set for authorization servers that gate claims such asgroups. - WebSocket Proxy Support - Realtime and Responses WebSocket connections route through the configured provider-level proxy (HTTP, SOCKS5, environment based) instead of always dialing direct.
- Configurable SCIM Buffer Sizes - A buffer size option on the HTTP client factory lets IdP token endpoints return headers larger than the 4KB default without failing SCIM and OAuth clients.
- Count Tokens Coverage - Count tokens support added for Bedrock Mantle, DeepSeek and SGLang, plus a retrieve-stream method on the Responses API.
- Model Reasoning Metadata - A
ModelReasoningschema field and provider-qualified model id resolution for model parameter lookups, with a requiredmodelquery param and a 404 response ongetModelParameters. - Partitioned Sidekiq Claiming - Background job claiming is partitioned with FIFO ordering per key.
- Dashboard Export and Ranking Controls - A
RankingLimitfilter withallandlimitquery params, uncapped snapshots for PDF and CSV exports, per-tab export scope, and acache_hit_typesdashboard filter. - Async Entity Selectors - Teams, customers and virtual keys load through async selector components instead of preloading full lists, and the customer list returns a server-computed virtual key count.
- Connector Latency and User Email Export - Connectors receive Bifrost latency and overhead duration, and can export user emails.
- Runware Passthrough - A
runware_passthroughpath handles passthrough mode for the Runware provider. - Shell Rewriter Hook - The UI handler exposes a
ShellRewriterhook for pre-hydration HTML rewriting. - Auth Skip Path - A context path lets trusted internal callers bypass auth resolution.
🐞 Fixed
- Migrations Before License Check -
LoadConfigruns migrations before the license check, so a fresh database no longer fails startup on a missing license table. - Guardrail Redaction Tool Results - Tool result text references are aligned for redaction, so redacted spans map back to the right content.
- Prompt Guardrail Errors - Prompt guardrail failures return specific error messages instead of a generic intervention message.
- GraySwan Canonical Content - The GraySwan integration handles raw canonical chat content arrays, sends the correct trace id, and marks policy id as required for the Cygnal API.
- List Models Governance Checks - Budget and rate limit checks are skipped for list-models calls, which do not consume model tokens.
- Device Inspect - Inspect no longer runs provider checks that could block it, and Responses instructions are handled correctly on the inspect path.
- SCIM Wizard Validation - Save-time validation errors are routed to the step that owns them, with field-level descriptions.
- Datadog Plugin Environment Variables - Environment variable support added for fields that previously had to be set literally.
- Code Scanning Fixes - Fixes across the SCIM discovery proxy, virtual key resolver and proxy paths flagged by code scanning.
- Path Normalization Auth Bypass (OSS) - Fixed a path normalization flaw that allowed auth to be bypassed.
- Empty Stream Nil Channel (OSS) - Stream requests return a closed non-nil channel for empty streams instead of
(nil, nil), which previously hung consumers on a nil-channel receive. - Proactive SSE Disconnect Detection (OSS) - Client disconnects during streaming are detected proactively instead of only when a producer loop attempts a write, fixing false-success logging on fast upstreams.
- SSE Heartbeat Corruption and Compatibility (OSS) - The stream reader will not emit a heartbeat mid-line, and the heartbeat frame no longer carries a trailing blank line that made some SSE decoders abort mid-stream.
- Closed Channel Panic on Stream Shutdown (OSS) - Fixed a race where a heartbeat goroutine mid-send at shutdown could panic with “send on closed channel”.
- Stream Termination Edge Cases (OSS) - A nil delta paired with a non-nil finish reason no longer aborts the stream, and GPT-5-series detection tolerates prefixed model names.
- Null Tool-Call Function Name on Streaming (OSS) - Streaming continuation deltas no longer materialize an absent tool-call function name as
null. - Minimal Reasoning Effort on GPT-5 Models (OSS) -
reasoning_effort: "minimal"is preserved for GPT-5-family models instead of being downgraded tolow. - Fallback Model Names (OSS) - Model refinement is idempotent, so fallback routing no longer truncates model names for prefixed providers.
- Anthropic Fallbacks and Billing (OSS) - Fallback handling and refusal responses on the Anthropic surface are fixed, and billing attributes usage to the fallback model actually served.
- Bedrock Reasoning and Cache Control (OSS) - Double emission of reasoning content on Bedrock streams is fixed,
cache_controlmarkers translate through invoke and Converse paths, tool ordering intoolConfigis deterministic for prompt cache hits, and reasoning blocks with an absent text key are no longer sent. - Bedrock Streaming Correctness (OSS) -
ConverseStreamreportsstopReason: tool_usefor tool-use turns, andmessage_startcarries an all-zero usage object when figures are unknown so strict clients accept the frame. - Bedrock Content Retention (OSS) - InvokeModel decodes Anthropic type-discriminated image, tool use and tool result blocks instead of dropping them, document-only messages are accepted, and office and PDF documents sent as OpenAI
type: "file"work. - Bedrock Header Signing Isolation (OSS) - Caller headers stored for Anthropic OAuth passthrough are no longer forwarded to other providers, preventing SigV4 signature mismatches.
- Encrypted Reasoning Handling (OSS) - Replayed encrypted reasoning no longer mints a mismatched item id, an upstream 400 on unverifiable content strips the reasoning and retries once, and Cohere emits encrypted reasoning alongside the summary rather than instead of it.
- Gemini and Vertex Fidelity (OSS) -
generateContentkeepscandidates[0].safetyRatingsandavgLogprobs, truncated responses reportMAX_TOKENS, valid integer constraints in tool schemas are accepted, and Vertex cached-content methods honour API key or context header auth. - DeepSeek Thinking on Multi-Turn (OSS) - Thinking is no longer silently disabled for ordinary multi-turn conversations through the OpenAI-compatible surface.
- vLLM Responses Streaming (OSS) - vLLM responses-stream chunks and completion events are forwarded instead of silently discarded, and truncation is handled.
- MCP Reconnect and Lock Ordering (OSS) - A lock-order inversion in the connection checker is broken, ephemeral clients are rebuilt across the whole connect and init retry, last-known tool maps survive close-first reconnects, and background reconnects are deduped.
- MCP OAuth Session Correctness (OSS) - Reauthorize is restricted to shared OAuth clients, inactive tokens are rejected on validation, the OAuth flow claim is atomic against concurrent reauth, stored scopes survive decode failures, and a verify-headers double-submit race is closed.
- MCP Tool Errors Replayed as Success (OSS) - Failed MCP tool executions are marked as errors instead of being replayed to the model as successful results.
- Session Stickiness Reconciliation (OSS) -
needs_session_stickinessis pinned acrossconfig.jsonreconciliation, so an unrelated file edit cannot revert a client to per-call. - Credential Cache Cancellation (OSS) - Credential and user token cache fills propagate context, so a cancelled request unblocks instead of waiting on an unrelated leader, and versioned LRU entries prevent a stale read from evicting a fresh value.
- Budget Counters Reset on Force-Sync (OSS) -
config.jsonforce-sync no longer overwrites live usage, last reset, and rate limit counters with file values. - Calendar Alignment Semantics (OSS) - Enabling calendar alignment preserves the currently open window and applies from the next period instead of truncating in flight.
- Governance List-Models Call (OSS) - Budgets and rate limits no longer trigger a list-models call.
- Multinode Override Counts (OSS) - Override counts are corrected for multinode setups, resolving high CPU in governance rate limit reset.
- Log Count Accuracy and Matview Scope (OSS) - The hybrid matview count no longer over-counts boundary buckets, and customer and business unit columns are added to the matview scope projection so team-data scope resolves without column errors.
- Lost Log Rows on Shared Trace IDs (OSS) - Concurrent requests inheriting the same W3C trace id no longer overwrite each other’s pending log entry.
- Live Reload Model List (OSS) - Provider reload no longer wipes the live model catalog before refetching, so a transient list-models failure cannot empty it.
- Transcription Filename Dropped (OSS) - The client’s multipart filename is carried through transcription ingress, so non-WAV containers are no longer relabelled and rejected upstream.
- Anthropic Mid-Conversation System Messages (OSS) - A system turn that cannot be forwarded natively is inlined as a user turn instead of being dropped.
- Server-Side Tool Search (OSS) - Tool search types are normalized on the Responses path and the Responses wire shape is preserved, and server-side tool invocation opt-in reaches the Gemini declaration-drop gate.
- HuggingFace Model IDs (OSS) - Backfilled HuggingFace model ids no longer duplicate the inference-provider segment.
- Together and xAI Costing (OSS) - The Together pricing provider lookup resolves model costs correctly, and USD cost ticks for xAI usage are fixed.
- HTTP Server Timeouts (OSS) - Bounded server timeouts and a request body limit are configured.
- Entra OBO Scope (OSS) -
offline_accessis combined with the audience default scope for Entra on-behalf-of instead of replacing it. - Budget Pruning Crash (OSS) - Pruning tolerates missing records for cascade-deleted budgets and configs, fixing a startup crash for API-created model configs absent from
config.json. - Virtual Key Provider Bulk Replace (OSS) - Provider config replacement is a single bulk operation instead of per-provider round trips, removing a hot-path slowdown at scale.
- pprof Content-Type (OSS) - pprof endpoints set
application/octet-streamfor scraper compatibility.
🗄️ Database Migrations
Enterprise (config store):- ent_add_guardrail_rule_target_column - Adds
targettoenterprise_guardrail_rulesso a rule can target MCP traffic. Reversible: drops the column. - ent_add_device_remote_signing_capable_column - Adds the remote-signing capability flag to the devices table. Reversible: drops the column.
- ent_add_audit_log_severity_column - Adds
severityto audit logs. Reversible: drops the column. - ent_migrate_legacy_plaintext_agent_ca_key - Re-encrypts a legacy plaintext private key found in the stored agent config. Forward only: the plaintext value is deliberately not restored on rollback.
- ent_split_oidc_session_auth_token_column - Splits the stored OIDC session token into separate id token and access token columns, classifying existing rows by audience with the provider client id. Forward only.
- ent_add_branding_table - Creates the enterprise branding table holding logo and icon overrides. Reversible: drops the table.
- ent_add_license_table - Now also stages a nullable
raw_licensecolumn on an existingenterprise_licensetable.
transports/v2.0.0-prerelease3 and v1.6.10 release notes. Two points matter for planning the upgrade:- The log store migrations alter
logsandmcp_tool_logs, the two highest-insert tables, and several build indexes on them. Run the upgrade during a low-activity window or expect elevated log-write latency while they run. merge_oauth_token_tables,drop_oauth_config_pkce_columns,drop_oauth_config_token_id_columnandadd_budget_reset_config_columncannot be rolled back. Take a database backup before upgrading.
🐙 Closed OSS Issues
- #123 - Files API support
- #4215 - HuggingFace models show provider ID twice in
/v1/models, which breaks requests - #5010 - Server-side SSE keepalive to keep long-idle streams alive through intermediaries
- #5074 - Fallback routing model selection is truncating model names
- #5186 - Anthropic-surface replay of OpenAI encrypted reasoning mints a fresh item id and OpenAI returns 400
- #5206 - Bedrock ConverseStream reports stopReason=end_turn for tool-use turns
- #5211 - Bedrock streaming can drop with “unexpected EOF” when an intermediary severs a quiet stream
- #5256 - Concurrent HTTP requests sharing a W3C trace ID lose LLM log rows
- #5279 - OpenAI /v1/responses to Anthropic drops the tool_search_tool_regex type
- #5308 - Responses API image blocks missing required “detail” field when converted from non-OpenAI providers
- #5329 -
/api/logsreturns an incorrecttotal_countfor time ranges of 24 hours or longer - #5433 -
/genaiendpoint rejects validminLength/maxLengthin tool schemas - #5472 - Bedrock rejects office and PDF document uploads via OpenAI
type:"file" - #5504 - vLLM streaming Responses API hangs forever and chunks are silently discarded
- #5546 - Upstream SSE stream death swallowed into a clean
[DONE] - #5551 -
transports/bifrost-http/libtest package does not compile on dev - #5552 - Refresh the live model catalog in the background
- #5554 - Provider reload wipes the live model catalog before refetching
- #5555 -
*StreamRequestreturns(nil, nil)for empty streams, so consumers hang forever - #5670 - Transcription drops the client’s multipart filename
- #5679 - Anthropic Messages does not propagate Gemini mixed server and client tool opt-in
- #5843 - generateContent drops
candidates[0].safetyRatingsandavgLogprobson Vertex AI responses - #5874 - SSE heartbeat frame aborts streams for openai-go ssestream consumers
- #5885 - v1.6.8 omits message_start.message.usage on Bedrock-backed providers
- #5887 - DeepSeek thinking silently lost on all multi-turn requests via OpenAI-compat inbound
- #5890 - Chat completions surface drops tool_result
is_error - #5900 - Streaming continuation chunks materialize omitted tool-call metadata as null
- #5902 - service_tier silently dropped for gpt-5.4 family
- #5905 - v1.6.8 raw passthrough heartbeat can split SSE data lines and corrupt JSON
- #5925 - config.json force-sync overwrites budget current_usage and last_reset on startup
- #5978 - Gemini reports truncated responses as FinishReason OTHER
- #6044 - normalizeOpenAIReasoningEffort maps ‘minimal’ to ‘low’ for all OpenAI models
📀 Base OSS version
transports/v2.0.0-prerelease3 (pinned as github.com/maximhq/bifrost/transports v1.6.11-0.20260813183832-666f97b09b93)🔌 If you are compiling plugin against this release - use following deps
The enterprise repo is a multi-module workspace; thegithub.com/maximhq/bifrost-enterprise/* modules at v0.0.0 resolve via the replace directives to the release source checkout.
