WebSocket protocol

Open a WebSocket to wss://api.ttscacherouter.dev/v1/stream with the header Authorization: Bearer <API key>. A missing, wrong or deleted key is refused at the handshake with HTTP 403. A key deleted while a connection is open is refused at its next start with unauthorized.

A complete client in Python, with pip install websockets. It speaks one sentence and saves it as out.wav; run it twice and the second run plays from the cache.

import asyncioimport jsonimport osimport uuidimport wavefrom websockets.asyncio.client import connectROUTER_URL = "wss://api.ttscacherouter.dev/v1/stream"SAMPLE_RATE = 24000async def main() -> None:    headers = {"Authorization": f"Bearer {os.environ['TTS_CACHE_ROUTER_API_KEY']}"}    segment_id = uuid.uuid4().hex    async with connect(ROUTER_URL, additional_headers=headers) as ws:        await ws.send(json.dumps({            "type": "start",            "segment_id": segment_id,            "provider": "sarvam",            "model": "bulbul:v3",            "voice": "shubh",            "language": "hi-IN",            "speed": 1.0,            "sample_rate": SAMPLE_RATE,        }))        await ws.send(json.dumps({            "type": "text", "segment_id": segment_id, "text": "नमस्ते, मैं प्रिया बोल रही हूँ।",        }))        await ws.send(json.dumps({"type": "flush", "segment_id": segment_id}))        pcm = bytearray()        async for message in ws:            if isinstance(message, bytes):                pcm += message  # 16-bit little-endian mono PCM                continue            event = json.loads(message)            if event["type"] == "sentence":                print(f"sentence {event['index']} from {event['source']}")            elif event["type"] == "done":                print(f"done: {event['billed_characters']} characters billed")                break            elif event["type"] == "error":                raise RuntimeError(f"{event['code']}: {event['message']}")    with wave.open("out.wav", "wb") as wav:        wav.setnchannels(1)        wav.setsampwidth(2)        wav.setframerate(SAMPLE_RATE)        wav.writeframes(pcm)asyncio.run(main())

A connection carries segments, one after another. A segment is one start, any number of text messages and one flush, each carrying the segment’s segment_id. The router answers each sentence with a sentence message followed by its audio as binary frames, and ends the segment with done. Send the next start on the same connection after done (or cancelled).

Sentences are spoken as soon as they’re complete, so audio usually starts before you send flush:

Messages out of order (text before start, a second start, anything after flush before done, a segment_id that isn’t the open segment’s) end the segment with a bad_request error.

Opens a segment and sets the provider, voice and settings for all of its sentences. The router calls the provider with the key stored for your org.

{  "type": "start",  "segment_id": "c0ffee",  "provider": "sarvam",  "model": "bulbul:v3",  "voice": "shubh",  "language": "hi-IN",  "speed": 1.0,  "sample_rate": 24000,  "format": "pcm",  "options": {"temperature": 0.6, "dict_id": null}}

type"start"Required

Identifies the message.

segment_idstringRequired

Your id for the segment, 1–128 characters. Sent with every text, flush and cancel, and echoed in done, cancelled and error.

provider"sarvam" | "raya" | "elevenlabs"Required

Which provider synthesizes, and which values below apply.

modelstringRequired

Sarvam: bulbul:v3 or bulbul:v4-flash. Raya: standard or m1. ElevenLabs: see Models and options.

voicestringRequired

The provider’s voice id, passed through as given: a Sarvam speaker, a Raya voice_id or an ElevenLabs voice id. It must belong to model; the provider checks it.

languagestringRequired; optional for ElevenLabs

The provider’s own language code, passed through as given: hi-IN for Sarvam, hi for Raya, en for ElevenLabs (which detects the language when it is left out).

speednumberRequired; not for ElevenLabs

Sarvam 0.5–2.0, Raya 0.5–1.5. 1.0 is normal speed. ElevenLabs takes speed in options.voice_settings.speed.

sample_rateintegerRequired

Of the audio you receive: 8000, 16000, 22050 or 24000; ElevenLabs also 32000, 44100 and 48000.

formatstringDefault "pcm"

The audio format, produced by the provider itself. Sarvam: pcm (16-bit little-endian mono), mp3, aac, mulaw or alaw. Raya: pcm or mulaw. ElevenLabs: pcm, mp3 (22050, 24000 or 44100 Hz), mulaw or alaw (both 8000 Hz).

optionsobjectDefault {}

Sarvam: temperature (0.01–1.0, default 0.6), dict_id (a pronunciation dictionary id or null) and bitrate (for mp3 and aac: 32k, 64k, 96k, 128k or 192k, default 128k); for bulbul:v4-flash only, pitch (-0.5–0.5, default 0), loudness (0.1–2.5, default 1.0) and enable_preprocessing (default false); min_buffer_size (30–200) and max_chunk_length (50–500), left out by default so each sentence is synthesized whole. Raya takes none: send {} or leave it out.

ElevenLabs: voice_settings (stability and similarity_boost 0–1, both required, and optionally style 0–1, speed 0.7–1.2 and use_speaker_boost; null for the voice’s stored settings), apply_text_normalization (auto, on or off), apply_language_text_normalization, enable_ssml_parsing (default false), enable_logging (default true), pronunciation_dictionary_locators (up to 3 {pronunciation_dictionary_id, version_id}), bitrate (for mp3: 32k at 22050 Hz, 48k at 24000 Hz, 32k–192k at 44100 Hz, default 128k), sync_alignment (send words, default true) and preferred_alignment (normalized or original).

Omitted options and their defaults share cache entries.

{  "type": "start",  "segment_id": "c0ffee",  "provider": "elevenlabs",  "model": "eleven_flash_v2_5",  "voice": "JBFqnCBsd6RMkjVDRZzb",  "sample_rate": 24000,  "format": "pcm",  "options": {"voice_settings": {"stability": 0.5, "similarity_boost": 0.75, "speed": 1.0}}}

Any field not listed here is a bad_request. Every value is listed on Models and options.

Send text as it arrives, chunked any way you like, such as an LLM’s tokens; the router finds the sentence boundaries. flush ends the segment’s input and speaks whatever text remains. A flush with no text gets a done with zeros.

{"type": "text", "segment_id": "c0ffee", "text": "नमस्ते, मैं सिटी हॉस्पिटल से "}{"type": "flush", "segment_id": "c0ffee"}

sentence comes before each sentence’s audio. index counts from 0 within the segment, text is the sentence as it was spoken (use it to align transcripts), and source is cache or provider.

{"type": "sentence", "index": 0, "text": "नमस्ते, मैं सिटी हॉस्पिटल से प्रिया बोल रही हूँ।", "source": "cache"}

The audio follows as one or more binary frames in the segment’s format, with no header. Concatenate the frames in order.

words (ElevenLabs only) gives the timings of a sentence’s words. It follows the sentence’s sentence message, interleaved with its audio, each batch before the audio it covers. index is the sentence’s; start and end are seconds from the start of that sentence’s audio. Each word’s text runs up to the next word, trailing whitespace included, so a sentence’s words concatenate to its spoken text. Turn them off with options.sync_alignment: false.

{"type": "words", "index": 0, "words": [{"text": "Hello, ", "start": 0.02, "end": 0.35}, {"text": "this ", "start": 0.35, "end": 0.52}]}

done ends the segment. sentences counts every sentence, cached_sentences the ones from the cache, and billed_characters the characters of the provider sentences: what the provider charges you for.

{"type": "done", "segment_id": "c0ffee", "sentences": 2, "cached_sentences": 1, "billed_characters": 30}

To stop speaking, for example when the user barges in, send cancel with the segment’s id. The router stops sending its audio and answers cancelled after the last frame it sends for the segment. Binary frames carry no segment id, so discard frames until cancelled arrives. The connection is then ready for the next start, already warm.

{"type": "cancel", "segment_id": "c0ffee"}{"type": "cancelled", "segment_id": "c0ffee"}

A cancel for a segment that already finished is answered with cancelled too. Sentences already sent to the provider still finish and are cached. Closing the connection also stops the segment.

Segment length, message size and idle timeouts are listed in Limits.

{  "type": "error",  "segment_id": "c0ffee",  "code": "provider_rejected",  "message": "…",  "retryable": false,  "provider_status": 403}

segment_id is null if the error came before a valid start. provider_status is the provider’s HTTP status when there is one, and null otherwise. retryable says whether the same request can succeed if you try again.

bad_requestnot retryable

The request failed validation or a message arrived out of order. The message says which field or rule.

unauthorizednot retryable

The API key is missing, wrong or deleted, or its org is disabled.

insufficient_creditsnot retryable

Your org is out of credit, so the router refuses new connections and requests until you buy more. Connections already open carry on.

provider_key_missingnot retryable

Your org has no key stored for the provider. Add one under Provider keys.

provider_rejectednot retryable

The provider refused the request, e.g. a wrong stored provider key or an unknown voice. The provider’s message is passed through.

provider_rate_limitedretryable

Your own quota at the provider is exhausted.

provider_unavailableretryable

The provider returned a server error, or the connection to it failed or closed unexpectedly.

provider_timeoutretryable

The provider took too long to send audio.

internalretryable

The router failed.