VoiceThere
Sign up

OpenAI speech on VoiceThere

Whisper and GPT transcribe models, Realtime live transcription, and OpenAI TTS on VoiceThere cloud runners with AGENT_OPENAI_API_KEY.

Overview of all vendors: STT & TTS vendors.

What it is

OpenAI provides cloud STT (Whisper and GPT-family transcribers) and TTS voices. VoiceThere sends audio from your session to OpenAI over HTTPS or WebSocket; there is no second WebRTC hop to OpenAI.

Home: OpenAI speech-to-text, OpenAI text-to-speech

Set up on VoiceThere

  • Dashboard: Project Settings → Voice (STT / TTS). Pick openai on the leg this vendor supports. Add project secrets from the list below, then Deploy to cloud.
  • Secrets: Store keys as AGENT_* project secrets (see Agent environment & secrets).
  • API: Patch stt_provider / tts_provider on the project before deploy. See Control plane API.

Project secrets for this vendor

  • AGENT_OPENAI_API_KEY: OpenAI API key. Used for OpenAI STT and/or TTS on the runner.

CLI (after secrets exist):

voicethere projects voice set --project YOUR_PROJECT_ID \
  --stt-provider openai --tts-provider openai \
  --stt-model gpt-live-transcribe --tts-model gpt-4o-mini-tts --tts-voice alloy

Models and voices you can pick

Project Voice settings in the dashboard lists every model and voice id VoiceThere accepts for this vendor. The same allowlists ship in the npm package @node-webrtc-rust/voice-catalog (pinned on your platform release).

STT examples: whisper-1, gpt-4o-mini-transcribe

TTS model examples: tts-1, tts-1-hd

TTS voice examples: alloy, ash

What the implementation does

  • whisper-1: file upload JSON transcription, no live partials.
  • gpt-4o-*-transcribe / gpt-transcribe: file upload with SSE deltas of the finished utterance.
  • gpt-live-transcribe: OpenAI Realtime WebSocket transcription session with partials while the caller speaks.
  • TTS tts-1 / tts-1-hd: HTTP audio, no SSE. gpt-4o-mini-tts may stream over SSE.

Limitations

  • Cloud usage is billed by OpenAI, not VoiceThere speech credits.
  • Pick a model that matches how much live feedback you need; file-based STT models do not emit partial transcripts during speech.

Official docs

← All documentation