VoiceThere
Sign up

VoiceThere voice

VoiceThere's own low-latency speech-to-text and text-to-speech in the VoiceThere cloud, with no API keys. Usage is metered in VoiceThere credits. API id voicethere.

Overview of all vendors: STT & TTS vendors.

What it is

Sherpa-onnx speech models run inside the VoiceThere cloud, in the same cluster as your agent, so audio never leaves for a third-party API and latency stays low. Zipformer models handle streaming speech-to-text; Piper (and Melo for Chinese) handle text-to-speech. There is no vendor API key. API id voicethere (the older ids local-sherpa and cluster-sherpa are deprecated aliases and still accepted).

Home: sherpa-onnx

Set up on VoiceThere

  • Dashboard: Project Settings → Voice (STT / TTS). Pick voicethere on the leg this vendor supports. Add project secrets from the list below, then Deploy to cloud.
  • Secrets: Store keys as AGENT_* project secrets (see Agent environment & secrets).
  • API: Patch stt_provider / tts_provider on the project before deploy. See Control plane API.

Project secrets for this vendor

No API keys required for this vendor.

CLI (after secrets exist):

voicethere projects voice set --project YOUR_PROJECT_ID \
  --stt-provider voicethere --tts-provider voicethere \
  --stt-model-id YOUR_STT_MODEL --tts-model-id YOUR_TTS_MODEL

Models and voices you can pick

Set stt_model_id and tts_model_id in project voice settings. The defaults are en for both: a streaming Zipformer English model for STT and the Piper en_US-amy-medium voice for TTS. Other examples: STT de, es, ja; TTS en-lessac-high, en-gb-jenny, de-thorsten-high.

These are the project's default models. Each voice session starts with them, and your agent can change them while the call is running without dropping the connection, per session. See Switching language mid-conversation.

The full VoiceThere model catalog (API id voicethere) is returned by GET /api/v1/voice/sherpa-models (see Control plane API) and shown when you configure Voice in the dashboard.

Switching to it from your agent

In @voicethere/agent 0.10.0 and later, pass provider: "voicethere" in sttVendor or ttsVendor when you call setVoiceLanguage or prepareVoiceLanguage:

await setVoiceLanguage(sessionId, {
  language: "de",
  sttVendor: { provider: "voicethere" },
  ttsVendor: { provider: "voicethere" },
});

The request has the same shape for every provider. See Request shape for the fields and Failures and messages for the rejection reasons.

Older ids and CLI versions

local-sherpa and cluster-sherpa are still accepted when you write settings through the API, and the API always reports voicethere. @voicethere/cli 0.16.9 and later accepts all three spellings for --stt-provider and --tts-provider. CLI 0.16.8 and earlier rejects local-sherpa locally because the provider list now names voicethere, so pass voicethere or update the CLI.

Speech usage is metered in credits. Rates are on Speech pricing.

What the implementation does

Audio from the session is decoded and fed to VoiceThere streaming STT in the same cloud. TTS audio is synthesized there too and sent back on the agent audio track. Partial transcripts are available while the caller is speaking.

Limitations

  • Usage is metered in VoiceThere usage credits, not billed to an external speech vendor.
  • You pick ids from the VoiceThere catalog. Custom model weights can't be uploaded. The ids in project settings are only the defaults for new sessions; a running session can switch to other catalog models (see Switching from your agent).
  • Piper TTS does not support SSML. Use punctuation pauses and eSpeak [[ ... ]] overrides — see Piper TTS pauses and pronunciation.

Official docs

← All documentation