Skip to main content
v2.0.0-prerelease3

Changelog

Third prerelease on the v2.0.0 line, built on OSS transports/v2.0.0-prerelease3. The enterprise side adds Prompt Guardrails (a natural-language rule classifier that runs as a guardrail provider), MCP guardrails with redaction and transformations, a canonical /api/governance route namespace with legacy aliases kept alive, custom logo and icon branding, and server-side signing for Edge device trust so signing key material is never distributed to devices. On the OSS side this release folds in everything from transports/v1.6.5 through v1.6.10 and the prerelease3 line: per-user MCP OAuth, per-model budgets and quarterly budget windows, virtual key budget overrides, async webhooks, and a large batch of provider and streaming fixes.

✨ Features

  • Prompt Guardrails - A new guardrail provider that classifies request and response content against a natural-language rule you write, with configurable model, output token ceiling, and timeout. It fails open on uncertainty by design, so only clear rule violations block. Docs
  • Guardrail Debug for Prompt Guardrails - Prompt guardrail evaluations report their own token cost and debug output through guardrail_debug, so the cost of running a classifier on traffic is visible per request.
  • MCP Guardrails - Guardrail rules can now target MCP tool traffic, not just model traffic, including redaction and transformation actions on MCP tool inputs and results, with backend config and a rules UI.
  • Canonical /api/governance Namespace - RBAC, user, team, virtual key, access profile, business unit, SCIM and audit log routes now live under /api/governance. Legacy paths keep working through registered aliases, RBAC resource mapping follows the canonical paths, and the enterprise UI calls the new ones.
  • Custom Branding - Logo and icon overrides are stored in a new enterprise branding table and served through GET/PUT/DELETE /api/branding plus an asset route, so the dashboard shell renders your brand instead of the default one.
  • Server-Side Signing for Edge Device Trust - Trust material for enrolled devices is now issued and signed by the server. Devices no longer receive long-lived signing key material, the signing endpoint is rate limited, and each device carries a remote_signing_capable flag so a fleet can be migrated in place.
  • Signed Agent Responses - Trust-relevant agent-facing responses are signed with an Ed25519 key that is independent of the interception key material, so an agent can detect a forged response even if the transport or a bearer credential is compromised.
  • Encrypted Key Material at Rest - A migration re-encrypts any legacy plaintext private key found in the stored agent config, so key material saved by older releases is protected at rest.
  • SCIM Attribute to Access Profile Mappings - IdP attribute values can be mapped directly to access profiles, with schema support, validation and normalization, auto-assignment during import, role sync and recompute paths, and a mappings editor in the SCIM wizard.
  • Okta SyncAllUsers Toggle - The Okta SCIM provider can sync non-active users as well, excluding suspended and deprovisioned ones, for organizations that stage users before activation.
  • Token Exchange with SSO Application Credentials - MCP clients using use_idp_credentials reuse the SSO login application’s client id and secret, and those credentials are now resolved unconditionally onto the token exchange IdP so Microsoft Entra ID style flows work without duplicate configuration.
  • Delegated MCP Token Exchange - Validated IdP tokens and OIDC sessions stamp an inbound bearer on the request context, and a SCIM-backed resolver wires delegated MCP token exchange to whichever SCIM provider is enabled.
  • Cluster-Wide MCP Credential Cache Eviction - MCP OAuth token and per-user header credential cache evictions are broadcast cluster-wide, and credential grants are reconciled on user delete so a removed user loses access on every node.
  • Cross-Instance MCP Connection State - A new node state store and heartbeat publish each instance’s per-client MCP connection state into the shared KV store, and an aggregate view compares them, so a client that is healthy on one node and unstable on another is visible instead of averaged away.
  • MCP OAuth Refresh Worker - A cluster-gossiped refresh worker renews MCP OAuth tokens and triggers a reconnect hook, plus a needs reauth gossip action that closes sessions requiring re-authorization.
  • Audit Log Severity - Audit log entries carry a severity level, set through the audit middleware, so high-impact administrative actions can be filtered apart from routine ones.
  • First-Time Admin Bootstrap Token - A one-time token flow creates the first admin user, replacing the previous manual bootstrap step.
  • Security Headers and robots.txt - Enterprise bootstrap adds a security headers middleware and a robots.txt route, and a skills orphan cleanup worker removes dangling skill records.
  • Device Page User Filters and Additive Sync - The devices page can be filtered by user, and device inventory sync is additive instead of replacing the stored set, so a partial sync no longer drops known devices.
  • Separate Allowed Domains Configuration - Allowed domains are configured independently of the rest of the interception policy, so domain scope can be changed without touching other settings.
  • Access Profile Aware Virtual Key Resolution - ensureUserVirtualKey skips virtual key resolution when the user already has an access profile, removing an unnecessary lookup from the login path.
  • Enterprise Context Middleware - Every per-request fasthttp context is stamped with the enterprise marker through a dedicated middleware, so downstream plugins can rely on it being present.
  • Enterprise Management Postman Collection - A generated Postman collection covers the enterprise management APIs, with dynamic discovery of workspace test commands.
  • License Public Key Injection - Dev cluster, broker, harness, connector and pulse build targets inject the license public key through ldflags from the environment, so locally built binaries validate licenses the same way releases do.
  • Okta Token CLI - A small oktatoken CLI acquires Okta authorization code plus PKCE tokens locally for on-behalf-of testing.
  • Inline User Search in Filter Sidebars - Filter sidebars search users inline instead of loading the full user list.

🌎 Open Source Features

  • MCP Per-User OAuth - MCP clients can hold per-user OAuth credentials and per-user headers, configurable from config.json and the UI, with a documented shared versus per-identity token lookup contract and virtual key and user filters on the OAuth grants and MCP auth session sidebars.
  • Bedrock VPC Endpoints - AWS Bedrock keys can target VPC endpoints, keeping Bedrock traffic on private networking. Docs
  • Per-Request Flat-Fee Pricing - A new cost_per_request field flows through datasheet sync, the cost engine, custom overrides and the pricing override form, for models billed per call rather than per token.
  • Pricing Overrides in the Model Catalog - /api/models/details exposes resolved pricing overrides and catalog rows resolve overrides server-side, so the catalog shows the price actually charged.
  • Virtual Key Budget Overrides - Temporary budget overrides add override_amount on top of max_limit and run either for a fixed number of reset cycles or until removed, configured through override_mode, override_cycles_total and override_anchor_reset across the database, governance store, admin APIs and UI.
  • Per-Model Budgets and Rate Limits - Virtual key provider configs accept budgets and rate limits scoped to individual models, surfaced through a unified budget override manager that groups provider and model budgets together.
  • Quarterly Budget Windows - Budgets support a quarterly reset period with a configurable fiscal start month, so a fiscal year that does not begin in January windows correctly.
  • Budget Usage Reset Coverage - The reset budget usage flow now covers teams, customers, model limits and provider governance, not only virtual keys.
  • User Scope for Routing and Pricing - Routing rules and pricing overrides can be scoped to individual users, with a user_id CEL variable in routing rules and a user picker in the pricing overrides UI.
  • Async Webhooks - Webhook delivery for async jobs, with endpoints configurable through config.json, the admin API and the UI, an SSRF-safe dispatcher with retries, paginated delivery history, and inference request_id propagation through jobs and payloads.
  • Background Model Catalog Refresh - Each provider’s list-models response is re-fetched on a live_models_sync_interval (default one hour, 0 disables), so models an upstream starts serving after boot appear without a restart.
  • Stream Truncation Detection - A new SSE truncation interface and EOF handling across providers surface upstream stream death as an error instead of a clean [DONE].
  • MCP Tool Discovery Persistence - Discovered MCP tools persist and resync uniformly across all client types through a hash-gated core callback, surviving restarts and propagating across a cluster.
  • Wafer AI Provider - Wafer AI is supported as a provider.
  • Lakera and Repello Argus Guardrails - Lakera and Repello Argus are available as guardrail integrations with configuration docs and UI branding.
  • Bedrock HTTP/2 PING Keepalives - The Bedrock provider can send HTTP/2 PING frames on idle connections through http2_ping_interval_in_seconds (0 disables), so quiet streams survive intermediaries that cut idle connections. Docs
  • Bedrock Batch Role ARN - A batch_role_arn on Bedrock key config passes a service role to Bedrock batch jobs for S3 access, taking priority over any role_arn in the request.
  • Anthropic Default Fallback Routing - Anthropic’s fallbacks: "default" preset is preserved through the Bifrost round trip, with the server-side fallback beta header injected for default-routing requests.
  • Mid-Conversation Tool Changes - The mid-conversation tool changes beta header is supported for Anthropic and Bedrock Mantle.
  • Reasoning Token Tracking - Anthropic extended-thinking tokens are tracked as reasoning tokens across chat, responses and passthrough.
  • Adaptive Thinking on Raw Passthrough - For adaptive-only Anthropic models, a legacy thinking.type: "enabled" block is rewritten to the adaptive form on the raw passthrough body as well as the typed request path.
  • Expanded OTEL Metric Attributes - Metrics carry a service instance id plus team, customer and business unit ids and names, so exported series can be sliced per tenant without post-processing.
  • Separate OTEL Metrics Pipeline - The OTEL collector supports a metrics tab independent of traces, with separate headers for traces and metrics.
  • OTEL Export Timeout - A new export_timeout setting (default 5 seconds) bounds how long a slow or unreachable collector can hold an export goroutine.
  • MCP Metrics - MCP metrics are exported through OTEL and the Prometheus telemetry plugin, plus a resource parameter on the MCP OAuth handshake.
  • Throughput Metrics - Tokens per second histogram endpoints, dashboard metrics, and throughput in model rankings and trend data.
  • W3C Trace ID Propagation - Requests carry a W3C trace id on the context, so gateway logs join cleanly with upstream traces.
  • Roots-Only Log Filter - A roots_only filter collapses fallback chains into their root entry with child aggregates.
  • User Agent and App Attribution in Logs - Logs and MCP tool logs record user agent, app, source, decision, app key and device id.
  • MCP Log Redaction and Plugin Logs - MCP tool logs carry redaction mappings and plugin logs.
  • S3 Log Export Metadata - Additional metadata is written alongside S3 log exports.
  • Matview Maintenance Off Switch - matview_refresh_interval accepts "off" to disable log store materialized view maintenance entirely.
  • Database Connection Controls - New conn_max_idle_time (default 5 minutes) on both config and logs stores, a cache_ttl (default 60 seconds) for password-command credential resolution, and a matview_refresh_timeout bounding a single refresh pass.
  • Object Storage Archival Settings - New archiveInterval, archiveGracePeriod and archiveMaxObjectBytes settings, plus a toggle to always retain request and response content regardless of retention cleanup.
  • Cancellable Log Cost Recalculation - Log cost recalculation tasks can be cancelled from the backend.
  • Routing Rule Validation - Routing CEL expressions and scope_id references are validated at write time in create and update handlers.
  • Routing Info Headers - Routing info headers are emitted for streaming responses, inference and integration APIs, and error and passthrough paths.
  • Access Profile Config Schema - config.schema.json accepts blacklisted_models (a denylist that wins over allowed_models), a weight seed for weighted routing, and model_budgets on access profile provider configs.
  • SSO Additional Scopes - config.schema.json accepts additionalScopes, requesting extra OAuth scopes on top of the base set for authorization servers that gate claims such as groups.
  • WebSocket Proxy Support - Realtime and Responses WebSocket connections route through the configured provider-level proxy (HTTP, SOCKS5, environment based) instead of always dialing direct.
  • Configurable SCIM Buffer Sizes - A buffer size option on the HTTP client factory lets IdP token endpoints return headers larger than the 4KB default without failing SCIM and OAuth clients.
  • Count Tokens Coverage - Count tokens support added for Bedrock Mantle, DeepSeek and SGLang, plus a retrieve-stream method on the Responses API.
  • Model Reasoning Metadata - A ModelReasoning schema field and provider-qualified model id resolution for model parameter lookups, with a required model query param and a 404 response on getModelParameters.
  • Partitioned Sidekiq Claiming - Background job claiming is partitioned with FIFO ordering per key.
  • Dashboard Export and Ranking Controls - A RankingLimit filter with all and limit query params, uncapped snapshots for PDF and CSV exports, per-tab export scope, and a cache_hit_types dashboard filter.
  • Async Entity Selectors - Teams, customers and virtual keys load through async selector components instead of preloading full lists, and the customer list returns a server-computed virtual key count.
  • Connector Latency and User Email Export - Connectors receive Bifrost latency and overhead duration, and can export user emails.
  • Runware Passthrough - A runware_passthrough path handles passthrough mode for the Runware provider.
  • Shell Rewriter Hook - The UI handler exposes a ShellRewriter hook for pre-hydration HTML rewriting.
  • Auth Skip Path - A context path lets trusted internal callers bypass auth resolution.

🐞 Fixed

  • Migrations Before License Check - LoadConfig runs migrations before the license check, so a fresh database no longer fails startup on a missing license table.
  • Guardrail Redaction Tool Results - Tool result text references are aligned for redaction, so redacted spans map back to the right content.
  • Prompt Guardrail Errors - Prompt guardrail failures return specific error messages instead of a generic intervention message.
  • GraySwan Canonical Content - The GraySwan integration handles raw canonical chat content arrays, sends the correct trace id, and marks policy id as required for the Cygnal API.
  • List Models Governance Checks - Budget and rate limit checks are skipped for list-models calls, which do not consume model tokens.
  • Device Inspect - Inspect no longer runs provider checks that could block it, and Responses instructions are handled correctly on the inspect path.
  • SCIM Wizard Validation - Save-time validation errors are routed to the step that owns them, with field-level descriptions.
  • Datadog Plugin Environment Variables - Environment variable support added for fields that previously had to be set literally.
  • Code Scanning Fixes - Fixes across the SCIM discovery proxy, virtual key resolver and proxy paths flagged by code scanning.
  • Path Normalization Auth Bypass (OSS) - Fixed a path normalization flaw that allowed auth to be bypassed.
  • Empty Stream Nil Channel (OSS) - Stream requests return a closed non-nil channel for empty streams instead of (nil, nil), which previously hung consumers on a nil-channel receive.
  • Proactive SSE Disconnect Detection (OSS) - Client disconnects during streaming are detected proactively instead of only when a producer loop attempts a write, fixing false-success logging on fast upstreams.
  • SSE Heartbeat Corruption and Compatibility (OSS) - The stream reader will not emit a heartbeat mid-line, and the heartbeat frame no longer carries a trailing blank line that made some SSE decoders abort mid-stream.
  • Closed Channel Panic on Stream Shutdown (OSS) - Fixed a race where a heartbeat goroutine mid-send at shutdown could panic with “send on closed channel”.
  • Stream Termination Edge Cases (OSS) - A nil delta paired with a non-nil finish reason no longer aborts the stream, and GPT-5-series detection tolerates prefixed model names.
  • Null Tool-Call Function Name on Streaming (OSS) - Streaming continuation deltas no longer materialize an absent tool-call function name as null.
  • Minimal Reasoning Effort on GPT-5 Models (OSS) - reasoning_effort: "minimal" is preserved for GPT-5-family models instead of being downgraded to low.
  • Fallback Model Names (OSS) - Model refinement is idempotent, so fallback routing no longer truncates model names for prefixed providers.
  • Anthropic Fallbacks and Billing (OSS) - Fallback handling and refusal responses on the Anthropic surface are fixed, and billing attributes usage to the fallback model actually served.
  • Bedrock Reasoning and Cache Control (OSS) - Double emission of reasoning content on Bedrock streams is fixed, cache_control markers translate through invoke and Converse paths, tool ordering in toolConfig is deterministic for prompt cache hits, and reasoning blocks with an absent text key are no longer sent.
  • Bedrock Streaming Correctness (OSS) - ConverseStream reports stopReason: tool_use for tool-use turns, and message_start carries an all-zero usage object when figures are unknown so strict clients accept the frame.
  • Bedrock Content Retention (OSS) - InvokeModel decodes Anthropic type-discriminated image, tool use and tool result blocks instead of dropping them, document-only messages are accepted, and office and PDF documents sent as OpenAI type: "file" work.
  • Bedrock Header Signing Isolation (OSS) - Caller headers stored for Anthropic OAuth passthrough are no longer forwarded to other providers, preventing SigV4 signature mismatches.
  • Encrypted Reasoning Handling (OSS) - Replayed encrypted reasoning no longer mints a mismatched item id, an upstream 400 on unverifiable content strips the reasoning and retries once, and Cohere emits encrypted reasoning alongside the summary rather than instead of it.
  • Gemini and Vertex Fidelity (OSS) - generateContent keeps candidates[0].safetyRatings and avgLogprobs, truncated responses report MAX_TOKENS, valid integer constraints in tool schemas are accepted, and Vertex cached-content methods honour API key or context header auth.
  • DeepSeek Thinking on Multi-Turn (OSS) - Thinking is no longer silently disabled for ordinary multi-turn conversations through the OpenAI-compatible surface.
  • vLLM Responses Streaming (OSS) - vLLM responses-stream chunks and completion events are forwarded instead of silently discarded, and truncation is handled.
  • MCP Reconnect and Lock Ordering (OSS) - A lock-order inversion in the connection checker is broken, ephemeral clients are rebuilt across the whole connect and init retry, last-known tool maps survive close-first reconnects, and background reconnects are deduped.
  • MCP OAuth Session Correctness (OSS) - Reauthorize is restricted to shared OAuth clients, inactive tokens are rejected on validation, the OAuth flow claim is atomic against concurrent reauth, stored scopes survive decode failures, and a verify-headers double-submit race is closed.
  • MCP Tool Errors Replayed as Success (OSS) - Failed MCP tool executions are marked as errors instead of being replayed to the model as successful results.
  • Session Stickiness Reconciliation (OSS) - needs_session_stickiness is pinned across config.json reconciliation, so an unrelated file edit cannot revert a client to per-call.
  • Credential Cache Cancellation (OSS) - Credential and user token cache fills propagate context, so a cancelled request unblocks instead of waiting on an unrelated leader, and versioned LRU entries prevent a stale read from evicting a fresh value.
  • Budget Counters Reset on Force-Sync (OSS) - config.json force-sync no longer overwrites live usage, last reset, and rate limit counters with file values.
  • Calendar Alignment Semantics (OSS) - Enabling calendar alignment preserves the currently open window and applies from the next period instead of truncating in flight.
  • Governance List-Models Call (OSS) - Budgets and rate limits no longer trigger a list-models call.
  • Multinode Override Counts (OSS) - Override counts are corrected for multinode setups, resolving high CPU in governance rate limit reset.
  • Log Count Accuracy and Matview Scope (OSS) - The hybrid matview count no longer over-counts boundary buckets, and customer and business unit columns are added to the matview scope projection so team-data scope resolves without column errors.
  • Lost Log Rows on Shared Trace IDs (OSS) - Concurrent requests inheriting the same W3C trace id no longer overwrite each other’s pending log entry.
  • Live Reload Model List (OSS) - Provider reload no longer wipes the live model catalog before refetching, so a transient list-models failure cannot empty it.
  • Transcription Filename Dropped (OSS) - The client’s multipart filename is carried through transcription ingress, so non-WAV containers are no longer relabelled and rejected upstream.
  • Anthropic Mid-Conversation System Messages (OSS) - A system turn that cannot be forwarded natively is inlined as a user turn instead of being dropped.
  • Server-Side Tool Search (OSS) - Tool search types are normalized on the Responses path and the Responses wire shape is preserved, and server-side tool invocation opt-in reaches the Gemini declaration-drop gate.
  • HuggingFace Model IDs (OSS) - Backfilled HuggingFace model ids no longer duplicate the inference-provider segment.
  • Together and xAI Costing (OSS) - The Together pricing provider lookup resolves model costs correctly, and USD cost ticks for xAI usage are fixed.
  • HTTP Server Timeouts (OSS) - Bounded server timeouts and a request body limit are configured.
  • Entra OBO Scope (OSS) - offline_access is combined with the audience default scope for Entra on-behalf-of instead of replacing it.
  • Budget Pruning Crash (OSS) - Pruning tolerates missing records for cascade-deleted budgets and configs, fixing a startup crash for API-created model configs absent from config.json.
  • Virtual Key Provider Bulk Replace (OSS) - Provider config replacement is a single bulk operation instead of per-provider round trips, removing a hot-path slowdown at scale.
  • pprof Content-Type (OSS) - pprof endpoints set application/octet-stream for scraper compatibility.

🗄️ Database Migrations

Enterprise (config store):
  • ent_add_guardrail_rule_target_column - Adds target to enterprise_guardrail_rules so a rule can target MCP traffic. Reversible: drops the column.
  • ent_add_device_remote_signing_capable_column - Adds the remote-signing capability flag to the devices table. Reversible: drops the column.
  • ent_add_audit_log_severity_column - Adds severity to audit logs. Reversible: drops the column.
  • ent_migrate_legacy_plaintext_agent_ca_key - Re-encrypts a legacy plaintext private key found in the stored agent config. Forward only: the plaintext value is deliberately not restored on rollback.
  • ent_split_oidc_session_auth_token_column - Splits the stored OIDC session token into separate id token and access token columns, classifying existing rows by audience with the provider client id. Forward only.
  • ent_add_branding_table - Creates the enterprise branding table holding logo and icon overrides. Reversible: drops the table.
  • ent_add_license_table - Now also stages a nullable raw_license column on an existing enterprise_license table.
Open source migrations shipped in this base are listed in the transports/v2.0.0-prerelease3 and v1.6.10 release notes. Two points matter for planning the upgrade:
  • The log store migrations alter logs and mcp_tool_logs, the two highest-insert tables, and several build indexes on them. Run the upgrade during a low-activity window or expect elevated log-write latency while they run.
  • merge_oauth_token_tables, drop_oauth_config_pkce_columns, drop_oauth_config_token_id_column and add_budget_reset_config_column cannot be rolled back. Take a database backup before upgrading.

🐙 Closed OSS Issues

  • #123 - Files API support
  • #4215 - HuggingFace models show provider ID twice in /v1/models, which breaks requests
  • #5010 - Server-side SSE keepalive to keep long-idle streams alive through intermediaries
  • #5074 - Fallback routing model selection is truncating model names
  • #5186 - Anthropic-surface replay of OpenAI encrypted reasoning mints a fresh item id and OpenAI returns 400
  • #5206 - Bedrock ConverseStream reports stopReason=end_turn for tool-use turns
  • #5211 - Bedrock streaming can drop with “unexpected EOF” when an intermediary severs a quiet stream
  • #5256 - Concurrent HTTP requests sharing a W3C trace ID lose LLM log rows
  • #5279 - OpenAI /v1/responses to Anthropic drops the tool_search_tool_regex type
  • #5308 - Responses API image blocks missing required “detail” field when converted from non-OpenAI providers
  • #5329 - /api/logs returns an incorrect total_count for time ranges of 24 hours or longer
  • #5433 - /genai endpoint rejects valid minLength/maxLength in tool schemas
  • #5472 - Bedrock rejects office and PDF document uploads via OpenAI type:"file"
  • #5504 - vLLM streaming Responses API hangs forever and chunks are silently discarded
  • #5546 - Upstream SSE stream death swallowed into a clean [DONE]
  • #5551 - transports/bifrost-http/lib test package does not compile on dev
  • #5552 - Refresh the live model catalog in the background
  • #5554 - Provider reload wipes the live model catalog before refetching
  • #5555 - *StreamRequest returns (nil, nil) for empty streams, so consumers hang forever
  • #5670 - Transcription drops the client’s multipart filename
  • #5679 - Anthropic Messages does not propagate Gemini mixed server and client tool opt-in
  • #5843 - generateContent drops candidates[0].safetyRatings and avgLogprobs on Vertex AI responses
  • #5874 - SSE heartbeat frame aborts streams for openai-go ssestream consumers
  • #5885 - v1.6.8 omits message_start.message.usage on Bedrock-backed providers
  • #5887 - DeepSeek thinking silently lost on all multi-turn requests via OpenAI-compat inbound
  • #5890 - Chat completions surface drops tool_result is_error
  • #5900 - Streaming continuation chunks materialize omitted tool-call metadata as null
  • #5902 - service_tier silently dropped for gpt-5.4 family
  • #5905 - v1.6.8 raw passthrough heartbeat can split SSE data lines and corrupt JSON
  • #5925 - config.json force-sync overwrites budget current_usage and last_reset on startup
  • #5978 - Gemini reports truncated responses as FinishReason OTHER
  • #6044 - normalizeOpenAIReasoningEffort maps ‘minimal’ to ‘low’ for all OpenAI models

📀 Base OSS version

transports/v2.0.0-prerelease3 (pinned as github.com/maximhq/bifrost/transports v1.6.11-0.20260813183832-666f97b09b93)

🔌 If you are compiling plugin against this release - use following deps

The enterprise repo is a multi-module workspace; the github.com/maximhq/bifrost-enterprise/* modules at v0.0.0 resolve via the replace directives to the release source checkout.