tts-cache-router docs

LiveKit Agents

The LiveKit plugin

livekit-plugins-tts-cache-router: install it, construct TTS, change settings mid-session, and what it reports.

Install#

livekit-plugins-tts-cache-router is a LiveKit Agents TTS that speaks through the router. It streams, keeps transcripts in step with speech and keeps its connections to the router warm across turns.

pip install livekit-plugins-tts-cache-router
agent.py
from livekit.plugins import tts_cache_router# The API key comes from TTS_CACHE_ROUTER_API_KEY.tts = tts_cache_router.TTS(    provider=tts_cache_router.Sarvam(        model="bulbul:v3",        speaker="shubh",        language="hi-IN",    ),)

It needs Python 3.10 or later and livekit-agents 1.8.5 or later. Set TTS_CACHE_ROUTER_API_KEY to your router API key, and store your provider key under Provider keys: the plugin never sends one.

TTS(...)#

The plugin's one class. Pass it to AgentSession(tts=...) like any other TTS.

providerSarvam | Raya | ElevenLabsRequired
The provider, model, voice and settings, in that provider’s own parameter names. See Sarvam, Raya and ElevenLabs.
api_keystr | NoneDefault $TTS_CACHE_ROUTER_API_KEY
Your router API key. Without one, the constructor raises ValueError.
base_urlstrDefault wss://api.ttscacherouter.dev
The router to connect to. Change it only for a router you run yourself.
sample_rateintDefault 24000
Of the PCM audio your agent receives: 8000, 16000, 22050 or 24000; with ElevenLabs also 32000, 44100 or 48000.
http_sessionaiohttp.ClientSession | NoneDefault None
The aiohttp session to connect with. Inside an agent job, leave it out: the job's is used.

Settings are checked when you construct TTS and the provider: an out-of-range value, an unknown model or a sample rate the provider doesn’t stream raises ValueError as your agent starts. Voices are checked by the provider on first use. tts.provider and tts.model name the underlying provider and model (Sarvam, bulbul:v3), so cost dashboards attribute them correctly.

Changing settings#

update_options changes the voice or settings mid-session, as the providers’ own plugins do. Replies that start afterwards use them; the reply being spoken finishes as it began.

Python
# The current provider's settings, in its own names:tts.update_options(speaker="priya", pace=1.1)# Or another provider altogether:tts.update_options(    provider=tts_cache_router.ElevenLabs(voice_id="JBFqnCBsd6RMkjVDRZzb", model="eleven_flash_v2_5"),)

Keyword settings are the current provider’s, by name, and are validated as in its constructor. Different settings are a different cache entry: a sentence spoken in a new voice is synthesized once, then cached for that voice too.

Interruptions#

When the user barges in, LiveKit cancels the reply. The plugin tells the router to stop and keeps the connection.

The router stops sending the reply’s audio at once and drops its sentences not yet sent to the provider. Sentences the provider was already synthesizing finish in the background and are cached, so the next time they are needed they come from the cache. The connection goes back to the plugin’s pool, warm, for the next reply.

Transcripts#

The plugin supports LiveKit's aligned transcripts, so captions follow the speech.

With Sarvam and Raya, each sentence’s text is timed to the moment its audio starts. With ElevenLabs, transcripts are word by word, from ElevenLabs’ own character timings, as ElevenLabs’ LiveKit plugin aligns them (turn this off with sync_alignment=False). Word timings are cached with the audio, so a reply from the cache is aligned too.

Metrics#

The plugin emits TTSMetrics once per reply, like any LiveKit TTS, with one difference that keeps cost reporting right.

characters_count counts only the characters the provider synthesized and billed. Sentences from the cache count zero, so a fully cached reply reports 0. LiveKit’s own count would bill every character you sent. ttfb, audio_duration and the connection fields are as usual, and the metadata names the underlying provider and model.

Python
from livekit.agents.metrics import TTSMetricsdef on_tts_metrics(metrics: TTSMetrics) -> None:    # What the provider billed for this reply: 0 when it all came from the cache.    print(metrics.characters_count, metrics.ttfb)tts.on("metrics_collected", on_tts_metrics)

Errors and retries#

Router errors become LiveKit's API errors, so LiveKit's retries and FallbackAdapter work as with any TTS.

  • A router error raises APIStatusError with the router’s retryable flag, so LiveKit retries only what can succeed on retry (per your conn_options). Its status_code is the provider’s HTTP status when there is one.
  • A refused router API key raises APIStatusError with status 403, not retryable.
  • A failed connection raises APIConnectionError, and a router that stops answering APITimeoutError; LiveKit retries both on a fresh connection.