tts-cache-router docs

Pipecat

The Pipecat service

pipecat-tts-cache-router: install it, construct TTSCacheRouterTTSService, change settings mid-session, and what it reports.

View as Markdown

Install#

pipecat-tts-cache-router is a Pipecat TTS service that speaks through the router, for every provider. It needs Python 3.11 or later and pipecat-ai 1.12 or later. This page describes version 0.1.1.

pip install pipecat-tts-cache-router
bot.py
import osfrom pipecat_tts_cache_router import TTSCacheRouterTTSServicetts = TTSCacheRouterTTSService(    api_key=os.environ["TTS_CACHE_ROUTER_API_KEY"],    settings=TTSCacheRouterTTSService.Settings(        model="sarvam/bulbul:v3", voice="shubh", language="hi-IN"    ),)# then, in your bot:# pipeline = Pipeline([transport.input(), stt, user_aggregator, llm, tts,#                      transport.output(), assistant_aggregator])

Put tts in your pipeline where a TTS service goes, after the LLM. Set TTS_CACHE_ROUTER_API_KEY to your router API key, and store your provider key under Provider keys: the service never sends one. It connects when the pipeline starts, and the router connects to your provider at once, so the first reply doesn’t wait for either.

TTSCacheRouterTTSService(...)#

The service's one class. Give it your API key and the provider's settings.

api_keystrRequired
Your router API key.
settingsTTSCacheRouterTTSService.SettingsRequired
The provider, model, voice and settings: see Settings.
base_urlstrDefault wss://api.ttscacherouter.dev
The router. Change it only for a router you run yourself.
sample_rateint | NoneDefault None
Of the PCM audio the service produces, one the provider streams. None: the pipeline’s output sample rate (PipelineParams.audio_out_sample_rate, 24000 by default, which every provider streams). Each provider’s rates are in Models and options.

Other keyword arguments go to Pipecat’s WebsocketTTSService, except text_aggregation_mode: the router splits the text into sentences itself, so the service streams the LLM’s text to it as it arrives and each sentence starts as soon as it is complete.

Settings#

The settings are in the router’s own names, those of /v1/audio/speech and GET /v1/models, the same for every provider.

settings=TTSCacheRouterTTSService.Settings(    model="sarvam/bulbul:v3",    voice="shubh",    language="hi-IN",    options={"temperature": 0.6},)
modelstrRequired
Any model GET /v1/models lists, with its provider: sarvam/bulbul:v3, elevenlabs/eleven_flash_v2_5, openai/tts-1, deepgram/aura-2, xai/xai-tts, … (Models and options).
voicestrRequired
The provider’s voice id, as GET /v1/voices lists it.
languagestr | Language | NoneDefault None
The provider’s language code, sent as written: hi-IN for Sarvam, hi for Raya, en for ElevenLabs, Soniox, Smallest AI and Cartesia. Required for Sarvam, Raya and Soniox. Optional for ElevenLabs and xAI: left out, the provider detects it. Optional for Smallest AI and Cartesia: left out, en. Not taken by OpenAI or Deepgram.
speedfloat | NoneDefault None
The speaking rate, where the provider takes one. None: the router's default.
optionsdictDefault {}
The provider’s other settings, by their names in Models and options: {"temperature": 0.6} for Sarvam, {"voice_settings": {...}} for ElevenLabs, {"instructions": "..."} for OpenAI.

The service checks model and voice when you construct it, and raises ValueError for a malformed one. The router checks the rest as the pipeline starts, when the service connects: a wrong setting is reported then as an error (bad_request), not at the first reply.

Changing settings#

A TTSUpdateSettingsFrame changes the voice or settings mid-session. Replies that start afterwards use them; the reply being spoken finishes as it began.

Python
from pipecat.frames.frames import TTSUpdateSettingsFrame# Another voice:await worker.queue_frame(    TTSUpdateSettingsFrame(delta=TTSCacheRouterTTSService.Settings(voice="echo")))# Or another provider; its language, speed and options start from none:await worker.queue_frame(    TTSUpdateSettingsFrame(        delta=TTSCacheRouterTTSService.Settings(            model="sarvam/bulbul:v3", voice="shubh", language="hi-IN"        )    ))

A model of another provider starts from no language, speed or options unless the update gives them, and options are replaced as a whole. Between replies, the router connects to the provider for the new settings at once. A sentence in a new voice or with new settings is a new cache entry: synthesized once, then cached.

Interruptions#

When the user interrupts, the service tells the router to stop the reply and keeps the connection.

The router stops sending the reply’s audio at once, and the provider stops the sentence it was synthesizing, so nothing cut off is cached (ElevenLabs and Cartesia finish their sentence, and cache it). The next reply starts on the same connection.

Transcripts#

The service's TTSTextFrames follow the audio, so the conversation context holds what was spoken.

Each sentence’s words are timed to the moment its audio starts. With ElevenLabs and Cartesia, transcripts are word by word, from the provider’s own timings (turn them off with options={"sync_alignment": False} or options={"word_timestamps": False}). Timings are cached with the audio, so a reply from the cache is aligned too.

Metrics#

The TTS usage metric counts only the characters the provider billed.

With usage metrics on, each reply reports one TTSUsageMetricsData: the characters of its sentences the provider synthesized. Sentences from the cache count zero, so a fully cached reply reports 0. Its model is the settings’ <provider>/<model>, so usage dashboards can attribute cost to the provider. TTFB is Pipecat’s, as for any TTS service.

Python
from pipecat.pipeline.worker import PipelineParams, PipelineWorkerworker = PipelineWorker(    pipeline,    params=PipelineParams(enable_metrics=True, enable_usage_metrics=True),)

Cache stats#

Once a reply is all sent, the service calls its on_cache_stats handlers: how much of the reply came from your org’s cache. This script speaks a sentence twice, without a transport, and prints both:

speak_twice.py
import asyncioimport osfrom pipecat.frames.frames import EndFrame, TTSSpeakFramefrom pipecat.pipeline.pipeline import Pipelinefrom pipecat.pipeline.worker import PipelineWorkerfrom pipecat.workers.runner import WorkerRunnerfrom pipecat_tts_cache_router import TTSCacheRouterTTSServiceasync def main() -> None:    tts = TTSCacheRouterTTSService(        api_key=os.environ["TTS_CACHE_ROUTER_API_KEY"],        settings=TTSCacheRouterTTSService.Settings(model="openai/tts-1", voice="alloy"),    )    @tts.event_handler("on_cache_stats")    async def on_cache_stats(tts, stats):        print(f"{stats.cached_sentences}/{stats.sentences} sentences from the cache, "              f"{stats.billed_characters}/{stats.characters} characters billed by the provider")    worker = PipelineWorker(Pipeline([tts]))    runner = WorkerRunner(handle_sigint=False)    await runner.add_workers(worker)    # The second time, from the cache.    await worker.queue_frames(        [TTSSpeakFrame("Your order has shipped."), TTSSpeakFrame("Your order has shipped."), EndFrame()]    )    await runner.run()asyncio.run(main())
0/1 sentences from the cache, 23/23 characters billed by the provider1/1 sentences from the cache, 0/23 characters billed by the provider
context_idstr
The reply's TTS context id.
sentencesint
The sentences the reply was split into.
cached_sentencesint
Those served from the cache.
charactersint
The characters of all the reply's sentences.
billed_charactersint
The characters the provider synthesized and billed, as the usage metric. cached_characters is the rest.

An interrupted reply reports both too: they count the sentences the router sent in full before the interruption, so the usage is what the provider billed for the audio your bot received.

Errors#

Router errors become Pipecat ErrorFrames, with a category, so your pipeline's error handling and ServiceSwitcher work as with any TTS.

What happenedErrorFrame category
The router refused the API keyAUTHENTICATION
Your org is out of creditQUOTA
A setting the router refused (bad_request)INVALID_REQUEST
No key stored for the provider (provider_key_missing)AUTHENTICATION
The provider refused the request (provider_rejected)From the provider’s HTTP status, as Pipecat classifies it (AUTHENTICATION for 401, AUTHORIZATION for 403, QUOTA for 402); INVALID_REQUEST without one
Your provider quota (provider_rate_limited)RATE_LIMIT
The provider failed or was late, or the router failedSERVER
The connection failed or droppedCONNECTIVITY

A reply that fails ends without the rest of its audio; the next reply starts on a new connection. An AUTHENTICATION, AUTHORIZATION or INVALID_REQUEST error marks the service unusable, as Pipecat does for any service, until its settings change. When the router restarts, the service reconnects at once.