Skip to main content

Overview

The /elevenlabs integration serves ElevenLabs’ own API. An ElevenLabs SDK, or any HTTP or WebSocket client written for ElevenLabs, keeps its request and response shapes: Bifrost forwards each call to ElevenLabs as sent and returns what ElevenLabs answers, and adds virtual key authentication, governance, request logs and cost on the way. Endpoint: /elevenlabs

Supported routes

The rest of the ElevenLabs API, such as voices, history, the speech-to-text REST endpoints and the text-to-dialogue REST endpoints, is not served on /elevenlabs. It is on /elevenlabs_passthrough, which forwards the call as it is under the key’s credential.

Setup

Authentication

  • The client sends a Bifrost virtual key in the xi-api-key header. It is the only credential header the ElevenLabs SDKs send; they send no Authorization header.
  • A value in xi-api-key is recognised as a virtual key only when it starts with sk-bf-.
  • The client’s key is never forwarded to ElevenLabs. The gateway sends the ElevenLabs provider key it selects, as xi-api-key.
  • With client.enforce_auth_on_inference on, a request without a credential is refused with 401. On a WebSocket the refusal is a 401 at the handshake.

Text to speech


Scribe realtime

Scribe realtime is wss://<gateway>/elevenlabs/v1/speech-to-text/realtime. The ElevenLabs SDKs open it with speech_to_text.realtime.connect (speechToText.realtime.connect in JavaScript).
What reaches Scribe. model_id is required: a handshake without it is refused with 400. The whole query reaches Scribe as sent, repeated keys such as keyterms included, except token and xi-api-key. One request per segment. A segment is the audio from one committed transcript to the next. Each segment is one request against the virtual key’s limits and one logged realtime.turn row. Scribe reports no usage, so the segment is priced by the audio seconds the gateway counts from the input_audio_chunk frames: pcm_N is 16-bit mono, ulaw and alaw are 1 byte per sample, and pcm_16000 applies when the query has no audio_format. Refusals. A refusal (a model the key may not use, a spent request limit or budget, an unknown virtual key) arrives as a Scribe error frame followed by a close, before the refused segment’s audio goes upstream. Scribe’s own error frames and closes reach the client as they are.

Text to dialogue WebSocket

The frames are ElevenLabs’ own protocol and pass through unchanged. The gateway reads them only to count the text.
  • One request per session. The session is admitted when it opens: the model allowlist, budget and request limit are checked once. It is billed on close by the characters of text the client sent in inputs[].text.
  • Refusals. A refusal is a {"error": "...", "code": ...} frame followed by a close, as the socket itself reports errors, with the close code as code. A handshake without model_id is refused with 400, and one without a credential with 401 while authentication on inference is enforced.
  • Frame size. A frame larger than 1 MiB from the client is refused by the connection limit.

Next Steps