This guide describes how plain text is interpreted when your project uses the Sherpa ONNX Piper (or VITS) voices that VoiceThere runs for text-to-speech. It applies to strings you pass through speak, speakAndChat, and any other path that sends agent text to the parent TTS pipeline unchanged.

What it applies to

Use these rules when the TTS vendor is VoiceThere (voicethere) with a Piper or VITS voice selected via tts_model_id in project voice settings. See VoiceThere voice for setup. The runner passes your string through to Sherpa; there is no SSML or markup layer in between.

No SSML

Piper on Sherpa does not parse SSML. Tags such as <break>, <speak>, <phoneme>, and <prosody> are spoken as literal text or ignored unpredictably — do not rely on them for pauses or pronunciation.

Punctuation pauses

Sherpa Piper uses punctuation in the input string to insert short silences and to split streaming synthesis chunks:

, — short pause
; — medium pause
., ?, ! — sentence-end pause; also boundary for streaming TTS chunking

Colons are unreliable. For example, text like echo: may not be spoken as you expect. Prefer rephrasing or use a comma or period instead of a colon when you need an audible break.

eSpeak phoneme overrides

Piper voices on Sherpa support eSpeak-style overrides with double brackets. Use ASCII eSpeak mnemonics, not IPA symbols inside the brackets.

Example word tweak: Hello [[w3:ld]] (wild, not “world” spelled normally)
Example phoneme string: [[h@l'oU]] (stress marked with ' inside the brackets)
Inside [[ ... ]]: ' marks primary stress; _ and _: insert pauses
Put sentence punctuation (,, ., ?) outside the brackets when you want a pause after the override
Sequences like _^_en are eSpeak language switches inside the phoneme engine — not a VoiceThere API

Do not use IPA inside brackets (for example [[ h ə l oʊ ]]) — that form does not work with Piper on Sherpa.

What does not work

SSML tags (see above)
IPA inside Piper [[ ... ]] brackets
Non-speech tags such as laughter or sigh markup
Custom per-project lexicons or uploaded pronunciation dictionaries

Speed, speaker, and trailing silence

Speaking rate: project setting tts.speed defaults to 0.9 and is clamped to 0.2–2.0. Change it in the dashboard under Advanced voice pipeline → TTS speaking rate, or with the CLI:

voicethere projects voice-advanced set tts.speed 0.9

See Advanced voice settings for the full key list.

Trailing silence: tts.postUtteranceSilenceMs defaults to 2550 ms of silence after each TTS utterance completes (also configurable in Advanced voice pipeline).

Voice choice: pick the Piper voice with tts_model_id in voice settings or the dashboard Voice picker — there is no separate speaker-id API on VoiceThere.

Agent examples

From @voicethere/agent, punctuation in the string controls pauses:

import { defineAgent, speak } from "@voicethere/agent";

export default defineAgent({
  onUserSpeechFinal({ sessionId, text }) {
    speak(sessionId, "Hello. This is a pause.");
  },
});

eSpeak override for one word:

import { defineAgent, speak } from "@voicethere/agent";

export default defineAgent({
  onUserSpeechFinal({ sessionId }) {
    speak(sessionId, "That sounds [[w3:ld]].");
  },
});

More agent APIs: @voicethere/agent.

Piper TTS pauses and pronunciation