Configuring Turn Detection and Interruptions in LiveKit Agents

A LiveKit voice agent gives you many related but distinct turn-taking controls: turn detection modes, endpointing delays, interruption modes, VAD thresholds, and your choice of models. They all shape how natural a conversation feels, and this guide helps you choose the right configuration for your use case.

This is a configuration guide, not a primer. It assumes you already know what turn detection, VAD, endpointing, and barge-in are. If any of those terms are new, this earlier post and the LiveKit turn detection docs are both good introductions. The focus here is how to configure these features, including new features such as LiveKit's audio-based turn detector, adaptive interruption handling, and STT-native endpointing models.

Table of contents#

• Pipeline architecture

• VAD configuration

• Turn detection configuration

  • Choosing a mode
  • LiveKit turn detector
  • STT endpointing (turn_detection="stt")
  • Realtime LLM (turn_detection="realtime_llm")
  • VAD only (turn_detection="vad")
  • Manual (turn_detection="manual")

• Interruption configuration

  • Enabling interruption handling
  • Configuring interruption handling
  • User turn limits

• Other related configuration

• Common issues and resolutions

Pipeline architecture#

These features are stages in a single pipeline that user audio flows through, and each is configurable. The pipeline exists to answer three questions:

1. Is there speech right now? This is what VAD determines.
2. Has the user finished speaking? If so, the agent can safely respond. Turn detection makes this call, and endpointing configures the time to wait before committing the turn.
3. Is the user interrupting the agent? When speech arrives while the agent is talking, the agent has to decide whether to stop and yield the floor. This is interruption handling.

VAD, where present, runs first on every frame. Which of the next two questions the pipeline asks depends on whether the agent is already speaking, as illustrated below:

The remaining pieces (noise cancellation, preemptive generation) exist to make those two decisions better, faster, or cleaner. Refer to the Turns overview and Turn-taking tuning documentation for more information.

VAD configuration#

VAD is the first stage in the pipeline. The framework auto-provisions a VAD for you through the AgentSession, so you don't normally specify one. For advanced cases you can pass a vad argument yourself, but be aware that the new LiveKit turn detector imposes its own timing requirements when active (such as a minimum silence window), which can override the VAD settings you provide.

As you browse the docs and examples, you'll see VAD specified in different ways:

Bundled inference VAD (default, recommended)

AgentSession auto-provisions inference.VAD(model="silero") for you, a Silero model bundled with the Agents SDK. You can override it by passing a vad argument, which accepts these options, though most deployments should leave them unchanged.

Silero VAD plugin

The standalone Silero VAD plugin is the older, separately installed option, and is marked deprecated in the source code. You may see this listed in older examples and it still works, with fully documented configuration, but it should not be used when creating new agents.

ai-coustics VAD adapter

If you already run ai-coustics noise cancellation, its plugin ships a built-in VAD adapter. The ai-coustics VAD adapter is not required in order to use ai-coustics noise cancellation. Note that the LiveKit turn detector is not designed to run with this VAD, though it's fine for other configurations (for example, VAD-only or STT endpointing).

Alternative VADs

You can also bring your own: any object that implements the SDK's VAD interface can be passed as vad, so you can wrap a different VAD model if you have one. To turn VAD off entirely (for example, with a realtime model that does its own detection), pass vad=None in Python or vad: null in Node.js.

Turn detection configuration#

Turn detection decides when the user has finished their turn so the agent knows when to respond. This section covers the modes you can choose and how to configure each one; for the concepts behind it, see the Turns overview and the earlier post.

Choosing a mode#

The turn_detection option accepts either a turn-detector model object or one of several string modes. Decide this first, since it determines which other settings apply.

The rest of this section covers each mode in turn.

LiveKit turn detector#

This is the default turn detector, and is recommended for most developers.

The TurnDetector is an audio model that reasons over the user's speech directly, combining the words with acoustic cues like intonation and rhythm. Because it doesn't wait for a transcript, it decides faster and avoids the mid-turn-pause mistakes of transcript-only approaches.

For how the model works and how it was evaluated, see the turn detector docs and the deep-dive blog post. (An older text-based detector also exists but is deprecated and slated for removal in Agents SDK 2.0, so use the audio model for anything new.)

You configure it through two surfaces: the detector model itself, and the endpointing timing layer that controls how long to wait after detection before committing the turn. Most agents leave these at their defaults. For more detail, see the EndpointingOptions reference.

A typical configuration looks like this:

Python

1
from livekit.agents import AgentSession, TurnHandlingOptions, inference
2

3
session = AgentSession(
4
    turn_handling=TurnHandlingOptions(
5
        turn_detection=inference.TurnDetector(),  # the default if omitted
6
        endpointing={"mode": "dynamic", "min_delay": 0.3, "max_delay": 2.5}, # secs
7
    ),
8
    # ... stt, tts, llm, etc.
9
)

Node.js

1
import { inference, voice } from '@livekit/agents';
2

3
const session = new voice.AgentSession({
4
  turnHandling: {
5
    turnDetection: new inference.TurnDetector(), // the default if omitted
6
    endpointing: { mode: 'dynamic', minDelay: 300, maxDelay: 2500 }, // ms
7
  },
8
  // ... stt, tts, llm, etc.
9
});

STT endpointing (turn_detection="stt")#

Setting turn_detection="stt" hands the end-of-turn decision to your STT provider's own endpointing model instead of running the LiveKit turn detector. Use it when your STT has strong built-in turn detection, such as Deepgram Flux or AssemblyAI Universal-3 Pro. Even in this mode you still need a VAD to handle interruptions; if you don't provide one, the default is auto-provisioned.

It's worth understanding how LiveKit's endpointing options interact with STT mode. endpointing.min_delay acts as a floor measured from when the user last stopped speaking, so it adds latency on top of the provider's own decision, but endpointing.max_delay has no effect. In practice, set min_delay to 0, leave endpointing in its default fixed mode (avoid dynamic, which can let the floor drift above 0 over a session), and let the provider's own settings control the timing, such as AssemblyAI's min_turn_silence or Deepgram Flux's eot_timeout_ms.

The configuration options for STT mode:

Python

1
from livekit.agents import AgentSession, TurnHandlingOptions, inference
2

3
session = AgentSession(
4
    turn_handling=TurnHandlingOptions(
5
        turn_detection="stt",
6
        endpointing={"min_delay": 0},
7
    ),
8
    stt=inference.STT(
9
        model="assemblyai/u3-rt-pro",
10
        extra_kwargs={"min_turn_silence": 100, "max_turn_silence": 1000},
11
    ),
12
    # ... vad (auto-provisioned), llm, tts, etc.
13
)

Node.js

1
import { inference, voice } from '@livekit/agents';
2

3
const session = new voice.AgentSession({
4
  turnHandling: {
5
    turnDetection: 'stt',
6
    endpointing: { minDelay: 0 },
7
  },
8
  stt: new inference.STT({
9
    model: 'assemblyai/u3-rt-pro',
10
    modelOptions: { min_turn_silence: 100, max_turn_silence: 1000 },
11
  }),
12
  // ... vad (auto-provisioned), llm, tts, etc.
13
});

Realtime LLM (turn_detection="realtime_llm")#

Realtime models consume and produce speech directly, and most of them do their own server-side turn detection as part of that. You can choose to either use that model's built-in detection, or hand turn detection to LiveKit's turn detector instead. LiveKit recommends using the model's built-in detection where possible.

Use the model's built-in detection

Turn detection settings should be applied to the model itself, not on the AgentSession. There's no standard set of realtime configuration parameters, and capabilities vary from provider to provider, so consult your provider's documentation for the specifics (for example, Gemini Live or OpenAI Realtime).

Note: when the model handles detection, most of LiveKit's interruption options are ignored, so they should not be specified.

The example below configures the OpenAI Realtime API's server VAD:

Python

1
from livekit.agents import AgentSession, TurnHandlingOptions
2
from livekit.plugins.openai import realtime
3
from openai.types.beta.realtime.session import TurnDetection
4

5
session = AgentSession(
6
    turn_handling=TurnHandlingOptions(
7
        # the default for realtime models; shown for clarity
8
        turn_detection="realtime_llm",
9
    ),
10
    llm=realtime.RealtimeModel(
11
        turn_detection=TurnDetection(
12
            type="server_vad",
13
            threshold=0.7,
14
            prefix_padding_ms=300,
15
            silence_duration_ms=400,
16
        ),
17
    ),
18
    # ... tts, etc.
19
)

Node.js

1
import { voice } from '@livekit/agents';
2
import * as openai from '@livekit/agents-plugin-openai';
3

4
const session = new voice.AgentSession({
5
  turnHandling: {
6
    // the default for realtime models; shown for clarity
7
    turnDetection: 'realtime_llm',
8
  },
9
  llm: new openai.realtime.RealtimeModel({
10
    turnDetection: {
11
      type: 'server_vad',
12
      threshold: 0.7,
13
      prefix_padding_ms: 300,
14
      silence_duration_ms: 400,
15
    },
16
  }),
17
  // ... tts, etc.
18
});

Use the LiveKit turn detector

If your realtime model lets you disable its internal turn detection, you can use LiveKit's audio-based turn detector instead (described earlier in this guide). As noted in the turn detector docs, it works on audio directly, so it does not need a separate STT, unlike the older text-based detector. The mechanism for disabling the model's own detection is provider-specific; for the OpenAI Realtime API, for example, you set turn_detection=None.

Python

1
from livekit.agents import AgentSession, TurnHandlingOptions, inference
2
from livekit.plugins import openai
3

4
session = AgentSession(
5
    turn_handling=TurnHandlingOptions(
6
        turn_detection=inference.TurnDetector(),
7
    ),
8
    llm=openai.realtime.RealtimeModel(
9
        voice="alloy",
10
        turn_detection=None,  # hand turn detection to the LiveKit audio model
11
    ),
12
)

Node.js

1
import { inference, voice } from '@livekit/agents';
2
import * as openai from '@livekit/agents-plugin-openai';
3

4
const session = new voice.AgentSession({
5
  turnHandling: {
6
    turnDetection: new inference.TurnDetector(),
7
  },
8
  llm: new openai.realtime.RealtimeModel({
9
    voice: 'alloy',
10
    turnDetection: null, // hand turn detection to the LiveKit audio model
11
  }),
12
});

VAD only (turn_detection="vad")#

Use this mode when your language isn't among the LiveKit turn detector's supported languages, or for simple, command-style interactions where short, predictable utterances do not need semantic end-of-turn detection. Outside those cases it's usually not the right choice: VAD-only detection relies on silence alone to decide the user has finished.

To understand more about which VAD is used, see the earlier section on VAD configuration.

Python

1
from livekit.agents import AgentSession, TurnHandlingOptions
2

3
session = AgentSession(
4
    turn_handling=TurnHandlingOptions(turn_detection="vad"),
5
    # VAD is auto-provisioned; ... stt, tts, llm, etc.
6
)

Node.js

1
import { voice } from '@livekit/agents';
2

3
const session = new voice.AgentSession({
4
  turnHandling: { turnDetection: 'vad' },
5
  // VAD is auto-provisioned; ... stt, tts, llm, etc.
6
});

VAD configuration options

Specifying endpointing.min_delay acts as a floor on top of the VAD's own silence window, so the effective wait after the user stops is max(VAD silence, min_delay). The other endpointing options have no effect in this mode.

The VAD-specific configuration for the bundled inference VAD is provided in the API reference.

Manual (turn_detection="manual")#

Use manual turn detection only when you need full control over when turns start and end, for example a push-to-talk workflow. You're responsible for committing and clearing turns and for interrupting the agent, so it requires extra code on your side. The manual turn control docs include a complete example to build from.

Interruption configuration#

Interruption handling decides what the agent does when the user starts speaking over it. Handling this well makes an agent feel natural, so it's worth configuring carefully.

Enabling interruption handling#

By default, the agent can be interrupted when the user speaks, but you can disable interruptions entirely if needed. When enabled, there are two modes: "adaptive" and "vad".

Configuring interruption handling#

Interruptible agents have a few more options to fine-tune behavior. A false interruption is when VAD hears speech and stops the agent but no transcript materializes (a cough, or background noise); by default the agent waits out a silence window and then resumes where it left off.

For further detail on these configuration options, see the documentation:

A typical adaptive configuration looks like this:

Python

1
from livekit.agents import AgentSession, TurnHandlingOptions, inference
2

3
session = AgentSession(
4
    turn_handling=TurnHandlingOptions(
5
        turn_detection=inference.TurnDetector(),
6
        interruption={
7
            "enabled": True,  # the default; can be omitted
8
            "mode": "adaptive",
9
        },
10
    ),
11
    # ... stt, llm, tts, vad
12
)

Node.js

1
import { inference, voice } from '@livekit/agents';
2

3
const session = new voice.AgentSession({
4
  turnHandling: {
5
    turnDetection: new inference.TurnDetector(),
6
    interruption: {
7
      enabled: true, // the default; can be omitted
8
      mode: 'adaptive',
9
    },
10
  },
11
  // ... stt, llm, tts, vad
12
});

User turn limits#

Interruptions are user-initiated; user turn limits are the mirror image, letting the agent cut in on the user. User turn limits are disabled by default, but you can configure them to stop a user from monologuing and monopolizing the conversation.

Python

1
session = AgentSession(
2
    turn_handling={"user_turn_limit": {"max_words": 100, "max_duration": 30.0}},
3
)

Node.js

1
const session = new voice.AgentSession({
2
  turnHandling: { userTurnLimit: { maxWords: 100, maxDuration: 30_000 } },
3
});

Other related configuration#

The following settings are related to turn detection and interruption handling but don't control them directly. For more on any of these, follow the associated docs link.

Common issues and resolutions#

If your issue isn't covered by the Turn-taking tuning troubleshooting table in our docs, some additional issues and resolutions are given below.