Skip to main content
Product

Agent Simulations

Run your real agent against LLM-driven simulated users in text and audio. Test full conversations before deployment, see exactly where they break, and ship with more confidence.

Start building
Documentation
Illustration of isometric grid with Agent Simulations elements
Ship with confidence
Test faster at scale
Test real-world scenarios

Test every part of your agent before you deploy

Run realistic simulations across text and audio, evaluate complete conversations, and bring real production failures back into your test suite.

Automated scenarios

Generate scenarios from your agent code and run them automatically, in parallel.

Audio simulations

Run scenarios through your full voice pipeline. LiveKit adds real-world conditions like background noise, and interruptions, then measures latency, transcription accuracy, and call quality.

Text simulations

Cheaper to run, for extra coverage of more cases. Text simulations focus on your agent's logic, testing how it understands requests, uses tools, and moves through a conversation.

Conversation evaluation

Evaluate the full conversation and confirm that the agent completed the expected task and reached the right outcome.

Production to test

Turn real production failures into reusable test scenarios that keep checking for the same issue as your agent changes.

CI Integration

Add simulations to your main branch pipeline and every commit is tested before it reaches production.

The LiveKit Platform

Build, run, and observe
agents with LiveKit Cloud

Our end-to-end platform powers enterprise-grade voice AI
for customer support at global scale.

Open AI
Salesforce
Deutsche Telekom
Zocdoc
Coursera
SpaceXAI
Headspace
Oracle
Assort Health
Spotify
Explore the full platform

Resources to help you learn and ship

Read the docs
Simulation results showing passed and failed scenarios.

Testing voice agents with Agent Simulations

product
LiveKit and Serve logos.

How Serve AI shipped a better voice agent with Agent Simulations

customers
Presenter pointing to LiveKit Agent Simulations results.

Watch a step-by-step tutorial

video

FAQs

How do I get started with Simulations?
Sign up for a LiveKit Cloud account, build an agent with the LiveKit Agents SDKs, then install the LiveKit CLI and run lk agent simulate from your agent's directory. With no scenario file, the CLI generates scenarios from your agent's source. To learn more, visit our docs.
What's the difference between text and audio simulations?
Text simulations exchange text with your agent without using STT or TTS, so they focus on your agent's logic and are cheap enough to run on every change. Audio simulations run the same scenarios through your full speech pipeline and measure latency, transcription accuracy, and turn-taking, including under conditions like background noise and packet loss. Audio turns are metered at a higher rate than text. To learn more, visit our docs.
Do I need to deploy my agent before running a simulation?
No. The CLI starts your agent as a local worker from your working tree and sends simulated calls to it, so you can test a change before it becomes a build. You can also run against an already-running agent with the --agent-name flag. To learn more, visit our docs.
What can Simulations evaluate?
Every simulation returns a pass or fail verdict with a reason, graded against the expectations you write. Audio runs also measure call quality, including the end-to-end latency the caller heard broken down by stage, transcription accuracy in both directions, turn-taking, and overall conversation quality. To learn more, visit our docs.
Can I run Simulations in CI/CD?
Yes. The CLI exits non-zero when a scenario fails, so a failed run breaks your build with no extra configuration. Text-mode requests are scheduled at low priority and paced into spare capacity in your inference quota, so they don't compete with live traffic. To learn more, visit our docs.
Can I turn production conversations into test scenarios?
Yes. Export a session from Agent Observability as a scenario and describe the outcome that should have happened, and it runs with the rest of your suite from then on. Because scenarios are checked into your repo, the scenario and the fix can ship in the same pull request. To learn more, visit our docs.

Ready to build?

Start building a voice AI agent with a free account. Reach out to us if you're interested in custom pricing.

Start building
Contact sales

No credit card required • 1,000 free agent session minutes monthly