Skip to main content
​
v1.5.7

​
Changelog

This release adds temporary per-user budget overrides with cluster-safe propagation, automatic virtual key attachment for identity-authenticated requests without an explicit key, and a redesigned Access Profile editor with per-model rate limits. It also fixes a class of multi-node budget/rate-limit accounting bugs (phantom usage on override edits, drifting reset times, cloning double-counts) and closes several identity/SCIM correctness gaps.

​
✨ Features

  • Per-User Budget Overrides - Admins can temporarily raise or lower a user’s effective access-profile budget from the User Detail Sheet, in “cycles” or “forever” mode, gated behind the AccessProfiles.Update permission. Overrides propagate cluster-wide, survive access-profile cloning and propagation, and are correctly reconciled when a node joins or rejoins the cluster mid-override.
  • Auto-Attach Virtual Key for Identity-Authenticated Requests - Inference requests authenticated via an identity provider but arriving without an explicit virtual key now automatically resolve and attach the user’s first active, non-expired access-profile-managed VK, so governance and billing still attribute usage correctly.
  • Access Profile Editor Redesign - The Access Profile editor and detail view now use a collapsible outline/tree layout instead of an accordion, with compact meta chips, an “All models” toggle, MCP tool access details, and read-only workspace-level budget context shown inline on each provider row.
  • Per-Model Rate Limits on Access Profiles - Access profile provider model budget groups now support per-model rate limits (tokens and requests) alongside existing per-model budgets, with usage hydrated live from the generated model config and surfaced in the User Detail Sheet’s three-level Rate Limits hierarchy.
  • Blocked Models and Weight on Access Profiles - Access Profile per-provider configs gained a blacklisted_models denylist and a weight load-balancer seed, matching the fields already available on standalone Virtual Keys.
  • Async Role Attach/Detach - Attaching or detaching a role from more than 100 users is now offloaded to a background job with live WebSocket progress in the Role Sheet, instead of blocking the request.
  • SCIM Provider Lifecycle Endpoints - Added dedicated POST (create) and DELETE endpoints for SCIM providers, so the setup wizard no longer silently creates providers via PUT; PUT on a non-existent provider now returns 404, and DELETE on an active provider returns 409 until it is disabled.
  • SCIM Reliability Improvements - SCIM group-membership reconciliation is now batched with async offload for large groups, group deletes offload to a background job, PATCH requests apply mixed replace/add/remove operations correctly and no longer clear unrelated attributes, and SCIM-provisioned team-to-business-unit edges carry provenance metadata so OIDC claim syncs cannot clear them.
  • Identity Resolution Fallback and Caching - Local user resolution now falls back to email when an IdP sub does not match, adds a cluster-evicted in-memory identity cache to avoid a DB round-trip per request, and fails closed on unresolvable identities under enforcement.
  • Cluster-Wide Model Refresh - Pressing “Refresh models” on one node now propagates the refresh to every peer via a new gossip entity type, instead of only updating the local node’s catalog.
  • Guardrail Providers - Added Lakera Guard v2 and Repello Argus as guardrail integrations.

​
🌎 Open Source Features

  • Virtual Key Budget Overrides - The database schema, governance store, admin APIs, and UI now support temporary budget overrides for virtual keys, configured via override_mode, override_cycles_total, and override_anchor_reset.
  • User Scope for Routing and Pricing - Routing rules and pricing overrides can be scoped to individual users, with a user_id CEL variable and an enterprise user picker.
  • Access Profile Config Schema - config.schema.json now accepts blacklisted_models, weight, and model_budgets on access profile provider configs, which previously failed schema validation.
  • Background Model Catalog Refresh - Each provider’s list-models response is re-fetched in the background on a configurable live_models_sync_interval (default 1 hour, 0 disables), so models an upstream adds after boot no longer stay invisible until restart.
  • Stream Truncation Detection - Added stream-death detection across all providers so an upstream connection drop surfaces as an error instead of a clean [DONE].
  • Database Connection Controls - Added conn_max_idle_time, cache_ttl for password-command credential resolution, and matview_refresh_timeout bounding materialized view refresh time.
  • OTel Export Timeout - Added export_timeout (default 5s) so a slow or unreachable collector cannot hold an export goroutine indefinitely.
  • Lakera and Repello Argus Guardrails - Added as OSS guardrail integrations with configuration docs and UI branding.
  • SSO Additional Scopes - config.schema.json now accepts additionalScopes for requesting extra OAuth scopes beyond the base set.

​
🐞 Fixed

  • Multi-Node Budget Override Phantom Usage - Override edits no longer inflate budget, token, and request usage across cluster nodes; override mutations now propagate as configuration-only reloads instead of full usage rebases.
  • Multi-Node Budget Reset Drift - Reset times for sub-day rolling budgets (1h, 30m) are now computed from a fixed grid anchored to creation time instead of each node’s own clock, and override cycle counts are derived from that anchor instead of stored and re-decremented.
  • Budget/Rate-Limit Cloning - Cloning no longer copies row IDs (which caused double-counted usage) but does preserve ID and CreatedAt where needed for reset-window continuity, and now preserves OverrideCyclesTotal/OverrideAnchorReset.
  • Cluster Join Override Sync - A node that joins or rejoins gossip while a budget override is active now reconciles override state from the database instead of missing it entirely.
  • Case-Insensitive Email and User Lookups - GetUserByEmail, GetSoftDeletedUserByEmail, and /api/users search are now case-insensitive and trim whitespace.
  • Guardrail Span Names - Guardrail child spans now follow the plugin.<name>.<stage> naming convention expected by observability connectors.
  • Guardrail Tool Extraction - Guardrail providers now receive tool definitions for Responses and Count Tokens requests, not just Chat requests, and Gray Swan metadata headers are consistently sourced from trusted context.
  • Identity Cache Race - A sharded per-subject lock now guards identity cache resolve/evict so a concurrent user delete or rename cannot be lost to an in-flight lookup.
  • Auth Enforcement Bypass - Identity-provider JWTs and opaque browser session cookies could previously bypass identity resolution when enforceAuthOnInference was disabled; both now always attempt resolution.
  • SCIM RFC 7644 Conformance - SCIM filter eq comparisons are now case-insensitive and plus-addressed emails round-trip correctly.
  • SCIM PATCH Semantics - Partial PATCH updates no longer clear mapped attributes omitted from the payload, and combined replace+add/remove operations on group members now apply in order.
  • SCIM Group Claim Detection - Claims that look like group identifiers (e.g. groups) now default to group-type SCIM matching automatically.
  • Cross-IdP Deprovisioning - SCIM reconciliation now filters by (source_name, source_id) so switching identity providers no longer deprovisions users imported by a different provider.
  • Bedrock Guardrail Error Message - The ARN parse error now names the expected field and shape instead of surfacing the SDK’s opaque error.
  • Reduced Log Verbosity - DAC, access-profile, and identity-aware-proxy log lines that fire on every request were downgraded from Info to Debug.
  • Connector Whitespace Trimming - BigQuery, Datadog, Kafka, and Pub/Sub connector config fields now trim leading/trailing whitespace on save.
  • Datadog Error Metrics Parity - The bifrost.errors.total Datadog metric now carries a normalized status_code tag, matching Prometheus and OTEL.

​
🐙 Closed OSS Issues

  • #4215 - HuggingFace models show provider ID twice in /v1/models, which breaks requests
  • #4851 - v1.6.2 and v1.6.3 governance rate-limit reset causes high CPU in BumpRateLimitUsage/updateRateLimitReferences
  • #5329 - /api/logs returns an incorrect total_count for time ranges of 24 hours or longer
  • #5433 - /genai endpoint rejects valid minLength/maxLength in tool schemas (400 INVALID_ARGUMENT)
  • #5504 - vLLM streaming Responses API hangs forever, chunks are silently discarded and the request never completes
  • #5546 - Upstream SSE stream death swallowed into a clean [DONE], so dead streams appear successful
  • #5551 - transports/bifrost-http/lib test package does not compile on dev (MockConfigStore missing UpdateBudgetOverride)
  • #5552 - Refresh the live model catalog in the background (models added after boot stay invisible until restart)
  • #5554 - Provider reload wipes the live model catalog before refetching, so a transient list-models failure empties it
  • #5555 - *StreamRequest returns (nil, nil) for empty streams, so consumers hang forever on a nil-channel receive

​
📀 Base OSS version

transports/v1.6.7

​
🔌 If you are compiling plugin against this release - use following deps