Speech latency benchmarks

These are measured numbers, not vendor claims. Each table says when it was measured, from where, how many runs it has, and what the clock includes. All times are milliseconds and shown as median / p90, rounded to whole milliseconds. The raw files and the code that produced them are in the node-webrtc-rust benchmarks folder, and every definition is in the methodology.

What we measure and why

A voice agent feels slow when the gap between the end of your sentence and the start of the reply is long. That gap has three parts: the speech-to-text result (STT), your agent logic, and the first audio of the reply from text-to-speech (TTS). We measure the two speech parts, with your agent doing nothing but echoing the text back.

We measure them in two ways. End to end means a client speaks to a deployed agent and times the events it receives. These numbers include the client's network path to our EU cluster and the whole platform. Librarymeans the vendor's API called directly from a server in a Hetzner data centre in the EU. These numbers leave out the VoiceThere platform, WebRTC and the Opus encoder, so they show the vendor alone.

Everything was measured with our own paid accounts, on 2026-10-19 and 2026-10-20, in the EU region. The runs used node-webrtc-rust 0.12.0. A cold sample is the first one on a new session or provider, including connection setup. A warm sample is a later one that reuses the connection. Some vendors are not in these tables because their terms do not allow publishing benchmarks (see vendors in the published results). Google, Azure and AWS are not measured yet.

A note on speech end to final

speech_end_to_final includes the end-of-speech wait of the VAD settings we used, which are the ones a VoiceThere agent runner uses: 1100 ms of silence plus a 400 ms hold, about 1.5 seconds before the audio is even finalized. Read it as a floor of roughly 1,500 ms. The numbers to compare between vendors are the ones next to it: final_to_reply_start and tts_first_audio.

VoiceThere speech, end to end

A counting-echo agent on VoiceThere speech (English speech-to-text and text-to-speech from our own speech service). Measured 2026-10-19 and 2026-10-20 by a load client on one laptop, against our EU cluster, with node-webrtc-rust 0.12.0 in the agent runners. Nothing else was running, speech capacity scaled as it does for a customer, and the reply phrase cache was off, so every reply was synthesised. The first turn of each conversation is shown separately because it includes connection setup.

tts_first_audiois measured by the agent runner, so it does not include the client's network. The other three do. The first turn of the 1-conversation row is a single sample, so its median and p90 are the same number.

Cloud speech through VoiceThere

The same counting-echo agent with a cloud vendor for speech, one conversation of 10 turns per row, measured 2026-10-20 with the same client and cluster. The table shows the 9 warm turns; the first turn of each run is left out because a single sample has no spread. Cartesia has no speech-to-text, so that row uses VoiceThere speech-to-text with Cartesia text-to-speech. In this run ElevenLabs text-to-speech used HTTP and Cartesia text-to-speech used WebSocket, which are the defaults before node-webrtc-rust 0.12.1. The OpenAI row ran on node-webrtc-rust 0.12.2, where the agent buffers 250 ms of OpenAI audio before it plays a reply, so its TTS first audio is when playback starts.

9 warm turns per row. Ten turns is a small sample, so treat a p90 as a rough upper bound and differences of a few tens of milliseconds as noise.

Speech APIs measured directly (library)

These runs call each vendor's API from a server in a Hetzner data centre in the EU, in the same data-centre network as our agent runners, through the vendor crates of node-webrtc-rust 0.12.0. They ran on 2026-10-19, 10 runs per row, vendors one after another so they do not compete for CPU. The VoiceThere platform is not in these numbers.

Text-to-speech: time to first audio

Time from the request until the first audio chunk is received, for a short text ("Sure, one moment.", 3 words) and a sentence (15 words). Median / p90, 10 runs per row.

Groq and OpenAI text-to-speech have one transport (HTTP). OpenAI rows were measured on node-webrtc-rust 0.12.2 on 2026-10-20 with the same setup; they show when playback starts, because the agent buffers 250 ms of OpenAI audio before it plays a reply.

Speech-to-text: speech end to final

The clip is the phrase "The quick brown fox jumps over the lazy dog." (9 words) as synthetic speech, streamed in real time. The first column includes the VAD wait described above. The second column is the part after the VAD declared the end of speech, which is the part the vendor owns. Median / p90, 10 runs per row. Where a row has no warm value, that vendor was measured cold only.

Streaming and batch vendors are different kinds of numbers. Groq and OpenAI whisper-1 upload the finished utterance and recognise the whole file, while ElevenLabs and Fish Audio stream audio over a socket.

Default transports change in node-webrtc-rust 0.12.1

When a vendor offers HTTP and WebSocket, the library default is the transport with the lower warm median time to first audio, among those without audible stalls. If two warm medians are within 50 ms or 10 %, the lower cold median wins, and a remaining tie goes to HTTP. The exact rule is in the methodology. Applied to the 2026-10-19 run, the median time to first audio over both texts (warm / cold) is:

In node-webrtc-rust 0.12.1, ElevenLabs text-to-speech defaults to WebSocket and Cartesia text-to-speech defaults to HTTP. Fish Audio already defaulted to HTTP.

Reproduce it

Library numbers. From a node-webrtc-rust checkout, with a Rust toolchain, Node 18 or newer and your own vendor keys:

npm run bench:vendors -- --vendors elevenlabs,openai --runs 10 --region <label> --env-file <path>

The result folder holds raw.jsonl, summary.md and a record of the host and its network times to each vendor.

End to end against your own agent. The latency-bench example in @voicethere/clientstarts a session on your project, speaks a counting phrase from a WAV file you provide, and records the same four timings as the tables above. Your agent should only echo the final text, so the numbers show the platform and not your own logic. See the example's README for the setup and the full command.

Compare two runs. In a node-webrtc-rust checkout:

npm run bench:compare -- benchmarks/results/<ours> benchmarks/results/<yours>

What these numbers do not tell you

• One region and one point in time. Vendor load changes with the time of day and the day of the week, so a gap of a few tens of milliseconds between two single runs is noise.
• Network and vendor processing are not separated. A caller far from the EU region adds their own network time to every end-to-end number.
• The end-to-end client was one laptop. Its network path is part of every end-to-end number except TTS first audio.
• The speech is a short synthetic English phrase. Natural, noisy or long speech can behave differently, and so can other languages.
• Ten runs per library row and 9 turns per cloud-vendor row are enough to show size, not to rank vendors that sit close together.

More on pricing and voices: Speech pricing and STT & TTS vendors.