The Talker-Reasoner Pattern for Voice Agents

The Talker-Reasoner pattern splits your voice agent into two minds. A fast conversational agent that keeps talking and a slow reasoning agent that thinks in the background. No more dead air while your agent figures things out.

Your voice agent needs to do something complex. Maybe it's researching flight options across three airlines, running a multi-step calculation, or querying a slow enterprise API. In a standard setup, the user hears nothing. Dead air. Two seconds. Three seconds. The silence feels like the call dropped.

The Talker-Reasoner pattern eliminates this problem by splitting the agent into two parallel processes. A fast "Talker" keeps the conversation flowing naturally ("Let me look into that for you, and while I do, can I ask about your seating preferences?") while a slower "Reasoner" works in the background on the complex task. When the Reasoner finishes, the Talker incorporates the results seamlessly.

This pattern is directly inspired by Daniel Kahneman's dual-process theory from Thinking, Fast and Slow. System 1 (fast, intuitive, always on) maps to the Talker. System 2 (slow, deliberate, logical) maps to the Reasoner. It was formalized for AI agents by Christakopoulou et al. (2024), and it's quickly becoming one of the most requested patterns in production voice AI.

What Is the Talker-Reasoner Pattern?#

The Talker-Reasoner pattern is a dual-agent architecture where two LLMs run in parallel with distinct roles. A fast, conversational model handles real-time interaction with the user. A slower, more capable model performs complex reasoning, planning, and tool calling in the background. The two communicate through shared state.

The core insight for voice is straightforward. The Talker never stops talking. In a standard voice agent, the LLM is the bottleneck. If it needs to do complex reasoning or call slow tools, the user hears dead air. The Talker-Reasoner pattern eliminates this by letting the Talker fill conversational space while the Reasoner works in the background. When the Reasoner finishes, the Talker weaves the results into its next response.

How It Works#

Here's a concrete example. A user calls a travel planning voice agent and says: "I'm thinking about a trip to Kyoto next month. What should I know?"

In a standard single-agent setup, the LLM would need to research Kyoto (weather, events, travel advisories, hotel availability, flight options), reason about what's most relevant, and then respond. That might take 3-5 seconds of dead air.

With the Talker-Reasoner pattern, the flow looks like this.

The user never experienced dead air. The conversation felt natural and responsive, even though complex research was happening in the background.

Two Operating Modes#

The paper identifies two distinct modes that emerge from this architecture.

Intuitive Talker Mode#

The Talker responds without waiting for the Reasoner. This works well for conversational, empathetic, and exploratory parts of a conversation. The Talker's language model is strong enough to generate coherent, helpful responses even with slightly outdated or incomplete information from the Reasoner.

Best for greetings, clarifying questions, acknowledgments, emotional support, gathering requirements, and any exchange where being conversationally present matters more than having perfect data.

System 2 Override Mode#

The Talker explicitly waits for the Reasoner to finish before responding. This is necessary for tasks where snap judgments would produce poor or dangerous results.

Best for delivering final recommendations, confirming bookings, quoting prices, providing medical or financial information, or any scenario where accuracy outweighs conversational speed.

A simple heuristic works well. If the user is exploring or sharing context, let the Talker run freely. If the user needs a specific answer or is about to make a decision, wait for the Reasoner.

What Makes It Architecturally Distinct#

The Talker-Reasoner pattern is fundamentally different from the other agent design patterns because both agents are active simultaneously on the same conversation.

The Talker-Reasoner pattern also composes with other patterns. The Reasoner might use ReAct internally (think→act→observe loops with tools). The Talker might use Handoff if it needs to transfer to a specialist. The overall system might sit inside a Supervisor. This is an orthogonal pattern that layers on top of existing architectures rather than replacing them.

When to Use the Talker-Reasoner Pattern#

When to Avoid It#

The Orchestration Challenge#

The hardest design question in the Talker-Reasoner pattern is deceptively simple. When does the Talker know to use output from the Reasoner?

It's the central design challenge when integrating reasoning models into real-time voice applications without disrupting conversational flow.

There are several approaches, each with different tradeoffs.

Polling Shared State#

The Talker checks for new Reasoner output on each turn. Simple to implement, but introduces slight delays if the Reasoner finishes between turns.

Event-Driven Notification#

The Reasoner signals the Talker when new results are available, and the Talker works them into its next natural response. More responsive but requires careful handling to avoid interrupting the Talker mid-sentence.

Explicit Injection#

The system injects Reasoner results directly into the Talker's context before its next generation. This guarantees the Talker sees the results but requires framework-level coordination.

Model Selection#

The Talker should optimize for speed (low time-to-first-token, streaming). The Reasoner should optimize for accuracy (stronger reasoning, more tools). This naturally maps to using different models or different configurations of the same model.

Building a Talker-Reasoner Voice Agent with LiveKit#

LiveKit's Agents Framework supports the Talker-Reasoner pattern through its Agent class, parallel llm.chat() API, lifecycle hooks like on_user_turn_completed(), and session.userdata for shared app state. While there's no dedicated Talker-Reasoner abstraction yet (this represents a framework gap and an active area of development), the building blocks are all available today.

The Architecture#

The Talker is the normal Agent running in the AgentSession voice loop. The Reasoner runs as a parallel async task using llm.chat() with its own tools and context. Communication happens through session.userdata. In this example, the app injects finished research into the Talker's next LLM turn from on_user_turn_completed(), which keeps the handoff natural.

Code Example#

Here's a travel planning agent that demonstrates the pattern. The Talker handles the conversation while the Reasoner researches destinations in the background.

First, the agent server and session setup.

1
import asyncio
2
from dataclasses import dataclass, field
3

4
from livekit.agents import (
5
    Agent,
6
    AgentServer,
7
    AgentSession,
8
    JobContext,
9
    JobProcess,
10
    cli,
11
    inference,
12
)
13
from livekit.agents.llm import ChatContext, ChatMessage, function_tool
14
from livekit.plugins import silero
15
from livekit.plugins.turn_detector.multilingual import MultilingualModel
16

17

18
@dataclass
19
class TravelResearchState:
20
    research: dict[str, str] = field(default_factory=dict)
21
    pending_destinations: set[str] = field(default_factory=set)
22

23
server = AgentServer()
24

25
def prewarm(proc: JobProcess) -> None:
26
    proc.userdata["vad"] = silero.VAD.load()
27

28
server.setup_fnc = prewarm
29

30
@server.rtc_session(agent_name="travel-planner")
31
async def entrypoint(ctx: JobContext) -> None:
32
    session = AgentSession[TravelResearchState](
33
        userdata=TravelResearchState(),
34
        stt=inference.STT("deepgram/nova-3", language="multi"),
35
        llm=inference.LLM(
36
            "openai/gpt-5-mini",
37
            provider="openai",
38
            extra_kwargs={"reasoning_effort": "low"},
39
        ),
40
        tts=inference.TTS(
41
            "cartesia/sonic-3",
42
            voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc",
43
        ),
44
        vad=ctx.proc.userdata["vad"],
45
        turn_detection=MultilingualModel(),
46
    )
47

48
    await session.start(
49
        agent=TravelAgent(),
50
        room=ctx.room,
51
    )
52

53
if __name__ == "__main__":
54
    cli.run_app(server)

Next, the agent definitions showing the Talker and Reasoner working together.

1
class TravelAgent(Agent):
2
    """The Talker (Fast Brain) - handles real-time conversation."""
3

4
    def __init__(self) -> None:
5
        super().__init__(
6
            instructions="""You are a friendly travel planning assistant.
7
            Keep the conversation flowing naturally. Ask clarifying questions
8
            about preferences, budget, and dates.
9

10
            If the app injects background research into the chat context,
11
            incorporate it naturally into your response. If research is still
12
            in progress, acknowledge it and continue the conversation.""",
13
            llm=inference.LLM(
14
                "openai/gpt-5-mini",
15
                provider="openai",
16
                extra_kwargs={"reasoning_effort": "low"},
17
            ),
18
        )
19
        self._reasoner_tasks: set[asyncio.Task[None]] = set()
20

21
    async def on_enter(self) -> None:
22
        self.session.generate_reply(
23
            instructions="Greet the user and ask where they're thinking of traveling.",
24
            tool_choice="none",
25
        )
26

27
    async def on_exit(self) -> None:
28
        for task in self._reasoner_tasks:
29
            task.cancel()
30

31
    async def on_user_turn_completed(
32
        self,
33
        turn_ctx: ChatContext,
34
        _new_message: ChatMessage,
35
    ) -> None:
36
        pending_destinations = list(self.session.userdata.pending_destinations)
37
        if not pending_destinations:
38
            return
39

40
        for destination in pending_destinations:
41
            result_text = self.session.userdata.research[destination]
42
            turn_ctx.add_message(
43
                role="system",
44
                content=(
45
                    f"Background research for {destination} is ready. "
46
                    f"Use these notes if they are relevant to the user's "
47
                    f"latest request: {result_text}"
48
                ),
49
            )
50

51
        self.session.userdata.pending_destinations.clear()
52

53
    @function_tool
54
    async def research_destination(
55
        self,
56
        destination: str,
57
        interests: str = "",
58
    ) -> str:
59
        """Start researching a travel destination in the background.
60
        Use when the user mentions a place they want to visit.
61

62
        Args:
63
            destination: The city or region to research.
64
            interests: Optional comma-separated list of user interests.
65
        """
66
        task = asyncio.create_task(self._run_reasoner(destination, interests))
67
        self._reasoner_tasks.add(task)
68
        task.add_done_callback(self._reasoner_tasks.discard)
69
        return f"I'm researching {destination} now. Keep talking while I gather details."
70

71
    async def _run_reasoner(self, destination: str, interests: str) -> None:
72
        """The Reasoner (Slow Brain) - runs deep research in the background."""
73
        reasoner_llm = inference.LLM("openai/gpt-5.4")
74

75
        chat_ctx = ChatContext()
76
        chat_ctx.add_message(
77
            role="system",
78
            content=(
79
                "You are a travel research assistant. Return a concise "
80
                "structured brief for another voice agent."
81
            ),
82
        )
83
        chat_ctx.add_message(role="user", content=(
84
            f"Research {destination} for a traveler interested in: "
85
            f"{interests or 'general tourism'}. "
86
            f"Find weather forecasts, top-rated restaurants, must-see "
87
            f"attractions, approximate hotel price ranges, and any "
88
            f"travel advisories. Return a structured summary."
89
        ))
90

91
        try:
92
            response = await reasoner_llm.chat(chat_ctx=chat_ctx).collect()
93
        finally:
94
            await reasoner_llm.aclose()
95

96
        result_text = response.text or "No structured summary returned."
97
        self.session.userdata.research[destination] = result_text
98
        self.session.userdata.pending_destinations.add(destination)

A few things to notice.

• The Talker uses a fast model (gpt-5-mini with low reasoning_effort) for low-latency conversation. The Reasoner uses a stronger model (gpt-5.4) for deeper analysis.
• asyncio.create_task() launches the Reasoner without blocking the Talker. The conversation continues immediately.
• session.userdata is the shared state bridge. The Reasoner writes results there, and pending_destinations marks which updates still need to be surfaced to the Talker.
• on_user_turn_completed() is a clean place to inject fresh research into the next LLM turn. This keeps the response grounded in shared state without forcing an abrupt mid-conversation update.
• The Reasoner has its own ChatContext. It doesn't need the full conversation history. It receives a focused prompt with just the information it needs, keeping token usage low.
• Track your background tasks. Keeping a set of pending tasks and canceling them in on_exit() prevents orphaned work after the session ends.
• LiveKit Inference helpers like inference.STT(...), inference.LLM(...), and inference.TTS(...) let you mix and match models for each stage of the pipeline. For OpenAI models, current docs show gpt-5-mini as a good fast option and gpt-5.4 as a stronger reasoning choice.

Real-World Production Evidence#

SAP#

SAP has publicly explored dual-process thinking for agentic AI, describing how System 1 (fast, intuitive) and System 2 (slow, deliberate) map to different agent responsibilities. In their engineering blog, they outline how lightweight orchestration handles routine tasks while full agentic reasoning handles complex, non-deterministic ones. This mirrors the Talker-Reasoner split.

Deep Research Agents#

LiveKit's exa-deep-researcher example demonstrates an extreme version of this pattern where the Reasoner performs long-running web research while the Talker maintains the conversation. It's a "fast brain, very very slow brain" architecture where research tasks can take tens of seconds, making the Talker's role in filling conversational space essential.

The Foundational Paper#

The paper "Agents Thinking Fast and Slow" (Christakopoulou et al., 2024) validated the pattern with a sleep coaching agent. The Talker maintained empathetic conversation during the "understanding" phase while the Reasoner built a structured belief model about the user's sleep patterns, barriers, and goals. During the "planning" phase, the Talker waited for the Reasoner to produce a complete coaching plan before delivering recommendations.

Common Pitfalls and How to Avoid Them#

1. Stale State Confusion#

Symptom. The Talker delivers outdated information because the Reasoner hasn't finished updating shared state.

Fix. Design the Talker's instructions to explicitly handle uncertainty. Instead of asserting stale data as fact, the Talker should hedge ("Based on what I'm seeing so far...") or ask a follow-up question to fill time. Add timestamps to shared state entries so the Talker knows how fresh the data is.

2. The "Nothing to Say" Problem#

Symptom. The Talker can't generate meaningful conversation while waiting for the Reasoner, leading to awkward filler like "Hmm, let me think... still thinking..."

Fix. Design agents for domains where the Talker can productively gather more context while the Reasoner works. Ask about preferences, confirm details, or provide background information the user might find helpful. If there's genuinely nothing to say, the pattern may not be the right fit.

3. Reasoner Results Arriving at Awkward Times#

Symptom. The Reasoner finishes mid-sentence, and the Talker abruptly changes topic to deliver the results.

Fix. Buffer Reasoner results and let the Talker incorporate them at natural conversation boundaries. Don't force immediate delivery. The results can wait for the next turn.

4. Over-Engineering Simple Interactions#

Symptom. You've built a full Talker-Reasoner system for an agent that mostly handles simple lookups.

Fix. Not every voice agent needs two brains. If a single model with good tool schemas can respond within your latency budget, the standard ReAct pattern is simpler and more maintainable. Reserve Talker-Reasoner for genuinely complex or slow reasoning tasks.

5. Token Cost Explosion#

Symptom. Running two LLMs simultaneously doubles (or more) your token consumption.

Fix. Scope the Reasoner's context aggressively. It doesn't need the full conversation history. Pass only the specific question and relevant data. Use the cheapest model that can handle the Talker role, and reserve expensive reasoning models for the Reasoner.

Key Takeaways#

• The Talker-Reasoner pattern splits a voice agent into two parallel processes. A fast conversational model that keeps talking and a slow reasoning model that thinks in the background.
• It eliminates dead air by letting the Talker fill conversational space while the Reasoner handles complex tasks
• Inspired by Kahneman's Thinking, Fast and Slow and formalized by Christakopoulou et al. in 2024
• Shared state (session.userdata in LiveKit) is the communication bridge between the two agents
• The pattern composes with other patterns. The Reasoner can use ReAct internally, the Talker can use Handoff, and the whole system can sit inside a Supervisor
• Orchestration is the hard part. Deciding when the Talker should wait for the Reasoner versus proceeding with stale data is the key design decision
• Already being applied in production, with SAP publicly exploring the pattern for enterprise AI

Getting Started#

The Talker-Reasoner pattern requires a solid foundation in LiveKit's agent framework. Here's the recommended path.

1. Start with a single-agent voice app. Follow the Agents quickstart to get a basic voice agent running.
2. Add tool calling. Get comfortable with @function_tool and the ReAct pattern so you understand how agents reason about tools.
3. Add a background task. Use asyncio.create_task() to run a parallel llm.chat() call alongside your main agent loop. Write results to session.userdata.
4. Wire up the notification. Use session.generate_reply() to prompt the Talker when the Reasoner has new results.
5. Test with real latency. Use the Agent Console to test with actual API latencies. The pattern's value becomes obvious when you introduce a tool call that takes 3+ seconds. If you'd prefer to explore without code first, Agent Builder lets you prototype a voice pipeline in your browser. When you're ready to go live, deploy to LiveKit Cloud with one click.

For reference implementations, check out the fast-preresponse example (running multiple models in the LLM stage) and the exa-deep-researcher example (a full Talker-Reasoner with long-running research tasks).

Give it a try and let us know what you're building.