# The LiveKit plugin

> livekit-plugins-tts-cache-router: install it, construct TTS, change settings mid-session, and what it reports.

## Install

`livekit-plugins-tts-cache-router` is a LiveKit Agents TTS that speaks through the router, for every provider. It needs Python 3.10 or later and `livekit-agents` 1.8.5 or later. This page describes version 0.4.1.

```shell
pip install livekit-plugins-tts-cache-router
```

**agent.py**

```python
from livekit.plugins import tts_cache_router as tcr

# The API key comes from TTS_CACHE_ROUTER_API_KEY.
tts = tcr.TTS(
    model="sarvam/bulbul:v3",
    voice="shubh",
    language="hi-IN",
)
```

Pass `tts` to `AgentSession(tts=tts)`. Set `TTS_CACHE_ROUTER_API_KEY` to your router API key, and store your provider key under [Provider keys](https://ttscacherouter.dev/dashboard/provider-keys): the plugin never sends one. It streams, keeps transcripts in step with speech, connects to the provider when the session starts (`prewarm()`), and keeps its connections warm across turns.

## TTS(...)

The plugin’s one class. Its settings are in the router’s own names, those of [/v1/audio/speech](https://docs.ttscacherouter.dev/clients/speech.md#body) and `GET /v1/models`, the same for every provider.

**ElevenLabs**

```python
from livekit.plugins import tts_cache_router as tcr

tts = tcr.TTS(
    model="elevenlabs/eleven_flash_v2_5",
    voice="JBFqnCBsd6RMkjVDRZzb",
    language="en",
    options={"voice_settings": {"stability": 0.5, "similarity_boost": 0.75}},
)
```

**From stored settings**

```python
from livekit.plugins import tts_cache_router as tcr

# Settings kept as data, in the names of /v1/audio/speech and GET /v1/models:
config = {
    "model": "sarvam/bulbul:v3",
    "voice": "shubh",
    "language": "hi-IN",
    "speed": 1.1,
    "options": {"temperature": 0.5},
}
tts = tcr.TTS(**config)
```

- `model` (`str`, Required): Any model `GET /v1/models` lists, with its provider: `sarvam/bulbul:v3`, `elevenlabs/eleven_flash_v2_5`, `openai/tts-1`, `deepgram/aura-2`, `xai/xai-tts`, … ([Models and options](https://docs.ttscacherouter.dev/reference/models.md)).
- `voice` (`str`, Required): The provider’s voice id, as [GET /v1/voices](https://docs.ttscacherouter.dev/reference/voices.md) lists it.
- `language` (`str | None`, Default `None`): The provider’s language code, sent as written: `hi-IN` for Sarvam, `hi` for Raya, `en` for the others. Required for Sarvam, Raya and Soniox. Optional for ElevenLabs and xAI: left out, the provider detects it. Optional for Smallest AI and Cartesia: left out, `en`. Not taken by OpenAI or Deepgram.
- `speed` (`float | None`, Default `None`): The speaking rate, where the provider takes one. None: the router's default.
- `options` (`dict | None`, Default `None`): The provider’s other settings, by their names in [Models and options](https://docs.ttscacherouter.dev/reference/models.md): `{"temperature": 0.6}` for Sarvam, `{"voice_settings": {...}}` for ElevenLabs, `{"instructions": "..."}` for OpenAI. Each provider’s page under LiveKit Agents lists its own.
- `api_key` (`str | None`, Default `$TTS_CACHE_ROUTER_API_KEY`): Your router API key. Without one, the constructor raises `ValueError`.
- `base_url` (`str`, Default `wss://api.ttscacherouter.dev`): The router. Change it only for a router you run yourself.
- `sample_rate` (`int`, Default `24000`): Of the PCM audio your agent receives, one the provider streams: 24000 works with every provider; each provider’s list is in [Models and options](https://docs.ttscacherouter.dev/reference/models.md).
- `http_session` (`aiohttp.ClientSession | None`, Default `None`): The aiohttp session to connect with. Inside an agent job, leave it out: the job's is used. Outside one, pass your own.

The router is the one that checks the settings. The constructor raises `ValueError` only for a `model` that isn’t `<provider>/<model>` or a missing `voice`. When the session starts, the plugin connects and sends the settings, and a setting the router refuses is logged as an error then; a reply with it fails with `APIStatusError` (`bad_request`, not retryable). `tts.provider` and `tts.model` name the underlying provider and model (`Sarvam`, `bulbul:v3`) for cost dashboards.

## Changing settings

`update_options` changes the voice or settings mid-session. Replies that start afterwards use them; the reply being spoken finishes as it began.

```python
# Another voice and speed, for the same provider:
tts.update_options(voice="priya", speed=1.1)

# Or another provider altogether; its language, speed and options start from none:
tts.update_options(model="elevenlabs/eleven_flash_v2_5", voice="JBFqnCBsd6RMkjVDRZzb")
```

`options` replaces the options as a whole. A sentence in a new voice or with new settings is a new cache entry: synthesized once, then cached.

## Interruptions

When the user barges in, LiveKit cancels the reply; the plugin tells the router to stop and keeps the connection.

The router stops sending the reply’s audio at once, and the provider stops the sentence it was synthesizing, so nothing cut off is cached (ElevenLabs and Cartesia finish their sentence, and cache it). The connection returns to the plugin’s pool, ready for the next reply.

## Transcripts

The plugin supports LiveKit's aligned transcripts, so captions follow the speech.

Each sentence’s text is timed to the moment its audio starts. With ElevenLabs, transcripts are word by word, from ElevenLabs’ character timings, and with Cartesia from its own (turn them off with `options={"sync_alignment": False}` or `options={"word_timestamps": False}`). Word timings are cached with the audio, so a reply from the cache is aligned too.

## Metrics

TTSMetrics, once per reply, count only the characters the provider billed.

Sentences from the cache count zero, so a fully cached reply reports `characters_count` `0`. `ttfb`, `audio_duration` and the connection fields are as usual, and the metadata names the underlying provider and model.

```python
from livekit.agents.metrics import TTSMetrics


def on_tts_metrics(metrics: TTSMetrics) -> None:
    # What the provider billed for this reply: 0 when it all came from the cache.
    print(metrics.characters_count, metrics.ttfb)


tts.on("metrics_collected", on_tts_metrics)
```

## Cache stats

Once a reply is all sent, the plugin emits `cache_stats`: how much of it came from your org’s cache. This script speaks a sentence twice, outside an agent, and prints both:

**speak_twice.py**

```python
import asyncio

import aiohttp
from livekit import rtc
from livekit.plugins import tts_cache_router as tcr


def on_cache_stats(stats: tcr.CacheStats) -> None:
    print(f"{stats.cached_sentences}/{stats.sentences} sentences from the cache, "
          f"{stats.billed_characters}/{stats.characters} characters billed by the provider")


async def main() -> None:
    async with aiohttp.ClientSession() as http:
        tts = tcr.TTS(model="openai/tts-1", voice="alloy", http_session=http)
        tts.on("cache_stats", on_cache_stats)
        for _ in range(2):  # the second time, from the cache
            frames = [event.frame async for event in tts.synthesize("Your order has shipped.")]
        with open("speech.wav", "wb") as f:
            f.write(rtc.combine_audio_frames(frames).to_wav_bytes())
        await tts.aclose()


asyncio.run(main())
```

```text
0/1 sentences from the cache, 23/23 characters billed by the provider
1/1 sentences from the cache, 0/23 characters billed by the provider
```

- `request_id` (`str`): The reply’s request id, as in its `TTSMetrics`.
- `segment_id` (`str | None`): The reply’s segment id when streamed, as in its `TTSMetrics`; `None` for `synthesize()`.
- `sentences` (`int`): The sentences the reply was split into.
- `cached_sentences` (`int`): Those served from the cache.
- `characters` (`int`): The characters of all the reply's sentences.
- `billed_characters` (`int`): The characters the provider synthesized and billed, as `TTSMetrics.characters_count`. `cached_characters` is the rest.

An interrupted reply emits both too, with `TTSMetrics.cancelled` set: they count the sentences the router sent in full before the interruption, so `characters_count` is what the provider billed for the audio your agent received.

## Errors and retries

Router errors become LiveKit's API errors, so LiveKit's retries and FallbackAdapter work as with any TTS.

- A router [error](https://docs.ttscacherouter.dev/reference/errors.md) raises `APIStatusError` with the error’s `retryable` flag, so LiveKit retries only what can succeed (per your `conn_options`). Its `status_code` is the provider’s HTTP status when there is one.
- A refused router API key raises `APIStatusError` with status 403, not retryable.
- A failed connection raises `APIConnectionError`, and a router that stops answering `APITimeoutError`; LiveKit retries both on a fresh connection.

> Waiting for the LLM is not a timeout: the plugin times out only after it has sent all of a reply’s text and the router goes quiet.

- [Sarvam](https://docs.ttscacherouter.dev/livekit/sarvam.md): Bulbul v3 and v4-flash.
- [Raya](https://docs.ttscacherouter.dev/livekit/raya.md): Bakbak standard and m1.
- [ElevenLabs](https://docs.ttscacherouter.dev/livekit/elevenlabs.md): Flash, Turbo, Multilingual, v3 and v4.
- [OpenAI](https://docs.ttscacherouter.dev/livekit/openai.md): gpt-4o-mini-tts, tts-1 and tts-1-hd.
- [Deepgram](https://docs.ttscacherouter.dev/livekit/deepgram.md): Aura-2, Aura-1 and Flux.
- [xAI](https://docs.ttscacherouter.dev/livekit/xai.md): xAI's voices in twenty languages.
- [Soniox](https://docs.ttscacherouter.dev/livekit/soniox.md): tts-rt-v2's voices in sixty-three languages.
- [Smallest AI](https://docs.ttscacherouter.dev/livekit/smallestai.md): Lightning v3.1 and v3.1 Pro.
- [Cartesia](https://docs.ttscacherouter.dev/livekit/cartesia.md): Sonic 3, 3.5 and 3.6.
- [WebSocket protocol](https://docs.ttscacherouter.dev/clients/stream.md): What the plugin sends and receives.