What does "inference is slower than realtime" mean?

In your LiveKit voice agent logs, you might see any of the following warnings along with an associated delay:

VAD inference is slower than realtime
inference is slower than realtime
VAD slower than realtime

The exact text depends on which version of LiveKit Agents you are running and whether your agent is Python or Node.js, but they all mean the same thing.

These warnings have nothing to do with the STT, TTS, and LLM models you use through LiveKit Inference. They mean the voice activity detection (VAD) model running inside your agent process took too long to get a small window of audio through.

What does the VAD do? The VAD tells your voice agent when the person speaking started and stopped talking. Along with turn detection, that determines when the agent replies.

How does the VAD work? Your agent connects to a LiveKit room as a participant and receives audio just like everyone else in the room. The VAD runs inside your agent's job process, where it analyzes that decoded PCM audio to detect voice activity. Audio arrives in realtime, and your agent's process has plenty to do besides VAD, so keeping up means every 32ms window of audio has to be processed in under 32ms.

What goes wrong? If the VAD cannot get through its window of audio in time, it picks up where it left off the next time it runs. Do that repeatedly and the VAD falls further and further behind. Being 50ms behind is not noticeable, but if the situation does not resolve itself, the audio the VAD is working on ends up noticeably behind the audio arriving at your agent.

A warning fires once the VAD is behind the realtime audio pipeline. The exact trigger differs slightly between Python and Node.js agents, but in both cases you will see the warning logged, telling you the VAD is no longer keeping pace with the audio coming in. To clarify: the delay reported with the warning is how far behind the VAD has fallen, so a delay of 1000ms means the VAD is processing audio 1s behind live audio. It does not mean the VAD was running for 1s.

Also note that Python reports delay in seconds, while Node.js reports it in milliseconds.

Symptoms#

A VAD that cannot keep up surfaces as a wide variety of seemingly unrelated agent symptoms. Some follow from END_OF_SPEECH firing late, since that is a key input to turn detection. Others come from whatever is starving the VAD in the first place:

The agent is slow to notice that you finished speaking.
The agent talks over people, and misses interruptions.
The greeting is clipped, or spurious interruptions occur early in the call.
Audio is choppy in egress recordings, but fine in LiveKit Agent Insights.
Memory climbs through a long session, and in extreme cases leads to job memory warnings.

How to fix it#

First, a short burst of warnings in the first few seconds of a session can be ignored, since it has no impact on VAD accuracy. This is most likely if you are still using the Silero VAD plugin.

Second, a number of issues that caused spurious reports of this warning have been fixed over time, so make sure you are on the latest version of LiveKit Agents and the LiveKit turn detector. It is also worth using the default, bundled inference VAD that AgentSession provisions for you, rather than the separate plugin.

Two notes on the above:

This is generic advice. As a framework we support multiple turn detection models and VADs, and in general the advice will always be to use the latest.
When moving from the Silero plugin to the inference VAD, be aware that the default min_silence_duration also changed (plugin, inference).

Then, most importantly, rule out a blocked event loop. Anything synchronous in your code, such as a blocking HTTP call, an in-call database write, or heavy CPU-bound computation, can lead to this warning, because time spent waiting on blocking code is counted inside the VAD inference timer. I wrote a separate blog about Diagnosing blocked event loops in LiveKit Agents, and the easiest fix for most customers has been to hand that blog to their coding agent and prompt it to look for problems.

And finally, check that the CPU load on your machine is not approaching its limit. It is still possible to block the event loop and have your CPU load still be low, so don't assume low CPU utilization means you are not blocking the loop. If you self-host, follow all the advice in our self-host agent docs, particularly the memory and CPU requirements, and check that your load_fnc and load_threshold are set correctly.

The time VAD inference actually took is reported through the VAD's metrics_collected event, which fires about once a second carrying inference_duration_total and inference_count. Divide one by the other for the average cost per window, as follows:

1from livekit.agents.metrics import VADMetrics
2
3# Within your AgentSession
4def on_vad_metrics(metrics: VADMetrics) -> None:
5    total = metrics.inference_duration_total
6    average_ms = 1000 * total / metrics.inference_count
7    logger.info("vad inference", extra={"average_ms": average_ms})
8
9# Assuming you are using the provisioned VAD from AgentSession
10if session.vad:
11    session.vad.on("metrics_collected", on_vad_metrics)

An average that is steadily high points at CPU starvation, whereas one that is normally near zero but spikes points at your event loop. For more information and samples of VAD metrics, see the VAD metrics recipe.

Other steps you can take include:

Use the full version of the audio turn-detection model, which is available to all LiveKit Cloud agents. It runs in the cloud, whereas the lightweight model runs locally and shares your CPU.
If you self-host, check that the number of pre-warmed processes matches what you expect, since a mismatch can mean the worker has the wrong idea of how many CPUs it has.

What should you not do#

Don't be tempted to give the VAD more threads, raise max_buffered_speech, or increase num_idle_processes. None of these help the VAD keep pace with realtime speech, and they can make performance worse.

If the warning is frequent and persistent, don't ignore it: you have a genuine issue that is degrading your user experience. The effect may be small or occasional, but it is worth fixing.

Don't assume this warning is related to a nearby crash in your logs. Where the two do correlate, it is usually because they share a root cause, most often CPU pressure on the process.