Async Tools for Voice Agents: Keep Talking While Work Runs

A customer asks your voice agent for a refund. The agent calls a tool, the tool hits your order system, and the order system takes about fifteen seconds to respond. For those fifteen seconds the agent says nothing. No "give me a sec," no progress, just silence. The customer assumes the call dropped and hangs up before the refund ever finishes.

That dead air is the bug. The slow backend is not the problem…every real backend is slow sometimes. The problem is that a normal function tool blocks the conversation while it runs. This guide shows how LiveKit async tools fix it. The agent acknowledges immediately, narrates progress while the work runs, lets the customer cancel mid-task, and refuses to refund the same order twice. The examples are built against a deterministic mock backend, so the only credentials involved are your LiveKit keys and there are no external services in the way.

Async tools landed in livekit-agents 1.6.0, released 2026-06-11. The examples here use livekit-agents>=1.6 and LiveKit Inference for STT, LLM, and TTS, so the only credentials you need are LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET.

Why a normal tool blocks the conversation#

When the LLM decides to call a function tool, the runtime awaits that tool and only lets the agent speak once it returns. For fast tools that's invisible. For a slow backend it's the dead air.

Here's the naive refund tool, the one the call dies on.

1
# blocking_agent.py. The BEFORE version. No ctx.update(), so it stays synchronous.
2
@function_tool()
3
async def process_refund(self, ctx: RunContext, order_id: str) -> str:
4
    """Process a refund for a customer's order.
5

6
    Args:
7
        order_id: The order number to refund.
8
    """
9
    amount = await verify_order(order_id)
10
    result = await issue_refund(order_id, amount)
11
    return (
12
        f"Refund processed for order {result.order_id}: "
13
        f"{result.amount} dollars, confirmation {result.confirmation}."
14
    )

verify_order and issue_refund are two asyncio.sleep calls against a mock backend, about six seconds then eight. The tool awaits both and returns a single string at the end. While those awaits run, the next user turn is blocked. Nothing speaks unless you manually call session.say or session.generate_reply, there's no structured progress channel, and the call can't be cancelled. The running call also never enters the chat context, so the LLM can't see that it's working and tends to call it again. Tools that never report back behave like ordinary synchronous tools, which is exactly the trap here (async tools overview).

The key design fact is that async is not a special tool type. Any ordinary @function_tool is synchronous until the moment it calls ctx.update(). The tool above never does, so it behaves exactly like a classic blocking tool.

Progress updates with ctx.update()#

This is the fix, and it's one line. ctx.update() is new with async tools, a method on RunContext that did not exist before livekit-agents 1.6. Inside the tool, call await ctx.update(message). The first call hands control back to the LLM immediately, using message as the tool's synthetic return value, and marks the call non-blocking. Before async tools a slow tool had no way to do this, so the turn just blocked until it returned. The agent voices that message right away while the tool keeps running in the background (async tools, "Updating the user"). Later updates are scheduled to surface only when both the agent and the customer are idle, so the customer can let the tool keep running in the background and talk to the agent normally in the meantime.

Here's the fixed refund tool, the version that runs in the demo.

1
# agent.py. The fix. The first ctx.update() releases control so the agent speaks at once.
2
@function_tool(flags=ToolFlag.CANCELLABLE, on_duplicate="reject")
3
async def process_refund(self, ctx: RunContext, order_id: str) -> str:
4
    """Process a refund for a customer's order.
5

6
    Args:
7
        order_id: The order number to refund.
8
    """
9
    await ctx.update(f"Starting the refund for order {order_id}. This takes a moment.")
10

11
    async with ctx.with_filler("Still pulling up your order, hang on.", delay=3):
12
        amount = await verify_order(order_id)
13

14
    await ctx.update(
15
        f"Order {order_id} checks out for {amount} dollars. I'm putting it "
16
        "through the payment processor now, almost there."
17
    )
18

19
    followups = [
20
        "Still working on it, the payment processor is finishing up.",
21
        "Almost done, finalizing your refund now.",
22
    ]
23
    async with ctx.with_filler(
24
        lambda step: followups[step], delay=3, interval=5, max_steps=len(followups)
25
    ):
26
        result = await issue_refund(order_id, amount)
27

28
    return (
29
        f"All set. Order {result.order_id} is refunded for {result.amount} dollars. "
30
        f"The confirmation number is {result.confirmation}."
31
    )

The import is from livekit.agents.llm import ToolFlag. The signature is ctx.update(message, *, template=None), where template is an optional per-call override for the instructions sent around the update.

Walk through what the customer hears. The first ctx.update() fires before any slow work, so the agent says something like "Starting the refund for order ten-oh-one, this takes a moment" within a beat of the request. That first update is the seam between a blocking tool and an async one. While verify_order runs, the ctx.with_filler block plays a short spoken line during any quiet pause so the wait never turns back into silence. When verification finishes, the second ctx.update() adds a line to the chat context and the agent voices "Order ten-oh-one checks out for forty two dollars, I'm putting it through now, almost there". A second set of rotating fillers covers the longer write while issue_refund moves the money, and the return value is spoken as the closing line. Silence became a running narration of exactly what the agent is doing.

One wording trap is worth heading off. Because the LLM phrases each update, a mid-refund status can come out sounding like the refund already finished. The session takes an update_template inside the async_options block of tool_handling that frames every update as still in progress, so a phase change never gets voiced as "done". The final return is delivered on its own, so it stays the only line that announces completion.

Because the LLM voices each update, the customer hears real status instead of a canned "please hold" phrase. That's the difference between this and the hard-coded thinking phrases most agents fall back on.

Those ctx.with_filler(...) blocks are the second progress channel. Where ctx.update() adds a status to the chat context for the LLM to voice, with_filler plays audio directly through session.say() during quiet gaps, bypassing the LLM. It takes a fixed line or a callable that rotates through several, with delay, interval, and max_steps setting when and how often it speaks. Reach for ctx.update() first for the events that matter, then add fillers to cover the waits in between.

The before and after#

Same backend, same delays, same customer request. The async version adds a few ctx.update() calls and a couple of ctx.with_filler blocks. The blocking version gives roughly fourteen seconds of silence ending in one final sentence. The async version acknowledges in under a second and narrates the work the whole way through. That contrast is the whole point, and it's why the project ships both versions of the tool side by side.

Want to feel this in your own terminal? Scaffold a working voice agent in about thirty seconds, then drop a ctx.update() into your slowest tool.

1
lk agent init my-agent --template agent-starter-python

Letting the user cancel with CANCELLABLE#

Once a tool can run for fifteen seconds, the customer will eventually want to stop it. "Actually, cancel that." Opt a tool into cancellation with the CANCELLABLE flag.

1
@function_tool(flags=ToolFlag.CANCELLABLE, on_duplicate="reject")
2
async def process_refund(self, ctx: RunContext, order_id: str) -> str:
3
    ...

When any registered tool is cancellable, the framework auto-exposes two companion tools to the LLM. get_running_tasks() lists the cancellable calls currently running, and cancel_task(call_id) cancels one by ID. You do not define these yourself. When the customer says "stop," the LLM calls cancel_task, which raises asyncio.CancelledError inside the running tool (async tools, "Cancellation").

Cancellation is opt-in for a reason. Most writes are not safe to stop partway through, and a refund is a good example. The mock backend splits the work into two phases on purpose.

1
async def verify_order(order_id: str, *, delay: float = DEFAULT_VERIFY_SECONDS) -> str:
2
    """Look up an order and confirm it is refundable.
3

4
    Read-only phase. Safe to cancel. Returns the refund amount as a string.
5
    """
6
    await asyncio.sleep(delay)
7
    return _ORDER_AMOUNTS.get(order_id, _DEFAULT_AMOUNT)
8

9

10
async def issue_refund(
11
    order_id: str, amount: str, *, delay: float = DEFAULT_REFUND_SECONDS
12
) -> RefundResult:
13
    """Move the money for a verified order.
14

15
    Write phase. Once this starts it should run to completion. Returns the
16
    confirmation record.
17
    """
18
    await asyncio.sleep(delay)
19
    # Deterministic confirmation derived from the order id, so no randomness.
20
    confirmation = f"RF-{order_id}"
21
    return RefundResult(order_id=order_id, amount=amount, confirmation=confirmation)

verify_order is a read, so a cancel during that phase is fine, the refund never happens. issue_refund moves money, so you don't want a cancel landing mid-write. In the agent, the first ctx.update() and verify_order run before any money moves, which is the genuinely safe window to cancel. By the time issue_refund starts, the agent is past that point. If you need to harden the write further, a tool can call ctx.disallow_interruptions() around it, and cancel_task against a tool that has done so raises a ToolError instead of cancelling.

One distinction that trips people up. Interrupting is not cancelling. If the customer talks over the agent, the tool's result gets discarded but the tool's code keeps running in the background. To actually stop the work, you need CANCELLABLE plus cancel_task (function tools, "Interruptions").

Handling duplicate calls with on_duplicate#

LLMs sometimes call the same tool twice before the first call returns, especially when a long-running tool hasn't reported back yet. For a refund, a duplicate could mean a double charge. The on_duplicate argument on @function_tool decides what happens. Duplicates are detected by tool name only, not by arguments. There are four modes (async tools, "Duplicate-call handling").

• "allow" (the default) runs the duplicate with no restriction.
• "reject" rejects the duplicate and tells the LLM to cancel the running call with cancel_task instead of starting a parallel one.
• "replace" cancels the running call and starts a new one. This requires the running tool to be CANCELLABLE, otherwise it raises a ToolError.
• "confirm" sends the running call's name and arguments back to the LLM and asks it to re-call with explicit confirmation.

For money, the choice is clear. The refund tool uses on_duplicate="reject".

1
@function_tool(flags=ToolFlag.CANCELLABLE, on_duplicate="reject")
2
async def process_refund(self, ctx: RunContext, order_id: str) -> str:
3
    ...

reject is the safest default for a write that must not double-fire. allow would let two refunds run at once. replace would cancel the in-flight call and start over, which is risky once money is moving. confirm is a reasonable alternative for operations where you genuinely want the model to ask again, but for a refund, refusing the duplicate outright is the conservative call. Match the mode to the cost of getting it wrong.

Going further: across handoffs#

By default an async tool belongs to its agent, and any pending updates are dropped when the session hands off to a different agent. If you need a running tool and its updates to survive a handoff, wrap it in an AsyncToolset and pass that to the AgentSession. We don't build that here, the refund demo is a single agent, but it's the next step if your refund needs to continue while control moves to, say, a billing specialist agent (async tools, "Agent handoffs").

Use it in your own agent#

The pattern is small. Take a normal @function_tool, make ctx.update() its first line so the agent acknowledges right away, send another update at each phase change, and let the return value close things out. Add ctx.with_filler() to cover the quiet stretches, ToolFlag.CANCELLABLE when the work is safe to stop, and on_duplicate to control double calls. Match each piece to what your tool actually does, and the slow call stops being dead air.

What's next#

Async tools turn the slowest part of a voice call from a liability into a conversation. The agent acknowledges right away, narrates the work, stops when asked, and won't refund twice. All it takes is the first ctx.update().

• Docs: Read the full async tools reference for ctx.with_filler, prompt templates, and AsyncToolset.
• Related: See voice agent architecture for how the voice pipeline, latency, and turn-taking fit the larger picture of designing a voice agent.
• Deploy this on LiveKit Cloud: Ship it and scale without managing infrastructure at cloud.livekit.io.

Give it a try and let us know what you're building.