The best open source frameworks for building realtime voice and video AI agents

Building a production voice or video AI agent means picking an open source framework to build on. This guide compares the two frameworks that matter most, LiveKit Agents and Pipecat, on architecture, adoption, developer experience, and production readiness, and covers where names like TEN Framework, Dograh, Vision Agents, and Vocode fit in.

Why choose an open source framework for your voice and video apps#

The vast majority of businesses that attempt to scale a voice or video agent eventually find that they need to:

• Give their agents deeply custom tools and/or access to their internal (often private) systems.
• Systematically tackle and minimize every millisecond of latency they can find to improve their agent's conversational quality and customer experience.
• Scale to thousands, millions, or even billions of minutes while keeping the system affordable.
• Comply with a myriad of security, privacy, and compliance requirements from a number of vectors.

As a result, most businesses that scale voice and video agents in production either start with or eventually migrate to an open source voice and video framework that can help them continue to build and iterate on their agent quickly, without sacrificing their ability to deliver high-quality customer experiences with low latency, burning down their entire cloud bill in one quarter, or accidentally introducing vulnerabilities, downtime, or privacy concerns.

Comparing the top open source frameworks for voice and video#

For developers and engineering teams building production agents, it generally comes down to two open source frameworks: LiveKit Agents and Pipecat. If you search around, several other names may appear like TEN Framework, Dograh, Vision Agents, and Vocode, and while it's worth knowing why each comes up, it's also worth understanding why none of them ends up on a production shortlist.

The two most popular options: LiveKit Agents and Pipecat#

LiveKit Agents and Pipecat both give developers (and their coding agents) tools and primitives that make it easier to build and iterate voice and video agents in production. Nearly every voice or video agent runs the same core pipeline: audio in → speech-to-text → language model → text-to-speech → audio out, with turn detection and interruption handling wrapped around it. Both frameworks support this pipeline as well as a single speech-to-speech model in place of the middle three.

Two design decisions separate LiveKit Agents from Pipecat. LiveKit Agents knows what it runs on (LiveKit's open source WebRTC and SIP server) and is built as a slightly higher abstraction over the agent. Pipecat won't assume a transport and is built as a slightly lower abstraction over the pipeline.

Here's what that looks like in practice:

LiveKit Agents runs on LiveKit's open source WebRTC and SIP infrastructure, wires that pipeline for you, and asks you to write the agent. You can swap or wrap any stage with a different speech-to-text provider, a redaction step on the transcript before it reaches the model, any LLM you choose, or a speech-to-speech model, but you work within that shape rather than adding in new stages. You give up transport choice and the ability to add, reorder, or branch stages; but you get a system that's already correct at every stage you didn't touch, with dispatch, scaling, and multi-party sessions handled underneath. It's a higher-level abstraction that aims to solve the undifferentiated problems, leaving builders to focus on programming and controlling the agent.

Pipecat asks you to connect in your own transport and assemble that same pipeline yourself, stage-by-stage. And because the pipeline is a list you write rather than a session you configure, it doesn't have to be a conversation. Nothing in it assumes a user speaks and then an agent replies, so continuous translation, an agent that only ever listens, or a stage that reacts to audio without generating any are all ordinary pipelines here, where in an agent-shaped framework you'd be working against the session model. But the defaults stop at the pipeline's edges: turn detection ships tuned, yet every stage you add must correctly pass along the frames it doesn't handle, and the things a media layer would normally handle (scaling, dispatch, multi-party sessions) are yours to build and manage. Ultimately it's a lower-level abstraction that exposes more of the underlying complexity of voice and video agents to builders.

An example of the LiveKit framework vs. Pipecat framework in code#

The quickest way to feel the difference is to build the same agent twice, then add one real-world requirement to each. The examples below are trimmed from each project's current quickstart (September 2026); the shape is what matters, not necessarily the line count.

Step 1: a working voice agent

1
from livekit import agents
2
from livekit.agents import Agent, AgentServer, AgentSession, inference
3

4
class Assistant(Agent):
5
    def __init__(self):
6
        super().__init__(instructions="You are a helpful voice AI assistant.")
7

8
server = AgentServer()
9

10
@server.rtc_session(agent_name="my-agent")
11
async def my_agent(ctx: agents.JobContext):
12
    session = AgentSession(
13
        stt=inference.STT(model="deepgram/nova-3", language="multi"),
14
        llm=inference.LLM(model="google/gemma-4-31b-it"),
15
        tts=inference.TTS(model="inworld/inworld-tts-2", voice="Ashley"),
16
    )
17
    await session.start(room=ctx.room, agent=Assistant())
18
    await session.generate_reply(instructions="Greet the user and offer your assistance.")
19

20
if __name__ == "__main__":
21
    agents.cli.run_app(server)

Pipecat, Python:

1
import os
2
from pipecat.audio.vad.silero import SileroVADAnalyzer
3
from pipecat.frames.frames import LLMRunFrame
4
from pipecat.pipeline.pipeline import Pipeline
5
from pipecat.pipeline.runner import PipelineRunner
6
from pipecat.pipeline.task import PipelineParams, PipelineTask
7
from pipecat.processors.aggregators.llm_context import LLMContext
8
from pipecat.processors.aggregators.llm_response_universal import (
9
    LLMContextAggregatorPair, LLMUserAggregatorParams,
10
)
11
from pipecat.runner.types import RunnerArguments
12
from pipecat.runner.utils import create_transport
13
from pipecat.services.cartesia.tts import CartesiaTTSService
14
from pipecat.services.deepgram.stt import DeepgramSTTService
15
from pipecat.services.openai.llm import OpenAILLMService
16
from pipecat.transports.base_transport import BaseTransport, TransportParams
17
from pipecat.transports.daily.transport import DailyParams
18

19
async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
20
    stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
21
    llm = OpenAILLMService(
22
        api_key=os.getenv("OPENAI_API_KEY"),
23
        settings=OpenAILLMService.Settings(system_instruction="You are a friendly AI assistant."),
24
    )
25
    tts = CartesiaTTSService(
26
        api_key=os.getenv("CARTESIA_API_KEY"),
27
        settings=CartesiaTTSService.Settings(voice="71a7ad14-091c-4e8e-a314-022ece01c121"),
28
    )
29

30
    context = LLMContext()
31
    user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
32
        context, user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
33
    )
34

35
    pipeline = Pipeline([
36
        transport.input(),
37
        stt,
38
        user_aggregator,
39
        llm,
40
        tts,
41
        transport.output(),
42
        assistant_aggregator,
43
    ])
44
    task = PipelineTask(pipeline, params=PipelineParams(enable_metrics=True))
45

46
    @transport.event_handler("on_client_connected")
47
    async def on_client_connected(transport, client):
48
        context.add_message({"role": "developer", "content": "Say hello and briefly introduce yourself."})
49
        await task.queue_frames([LLMRunFrame()])
50

51
    @transport.event_handler("on_client_disconnected")
52
    async def on_client_disconnected(transport, client):
53
        await task.cancel()
54

55
    await PipelineRunner(handle_sigint=runner_args.handle_sigint).run(task)
56

57
async def bot(runner_args: RunnerArguments):
58
    transport = await create_transport(runner_args, {
59
        "daily": lambda: DailyParams(audio_in_enabled=True, audio_out_enabled=True),
60
        "webrtc": lambda: TransportParams(audio_in_enabled=True, audio_out_enabled=True),
61
    })
62
    await run_bot(transport, runner_args)
63

64
if __name__ == "__main__":
65
    from pipecat.runner.run import main
66
    main()

Both work. The differences are already visible, though: LiveKit's version never mentions a transport (the session knows it's a LiveKit room), never wires the pipeline (the session does), and uses one credential for three model providers (LiveKit Inference). Pipecat's version picks a transport per run, lists seven processors in order (including two halves of the context aggregator that have to sit on opposite sides of the LLM), and carries three provider keys. That's not shown as a criticism; it's shown to display the design and tradeoffs. Pipecat is showing you the pipeline because it wants you to be able to change it.

Step 2: a real requirement

Now the compliance team asks for one thing: card numbers a caller reads aloud must never reach the language model. Same requirement, both frameworks.

LiveKit Agents has a hook that fires when the user's turn ends and before the reply is generated. You edit the message there; the pipeline is untouched.

1
import re
2
from livekit.agents import Agent, ChatContext, ChatMessage
3

4
CARD = re.compile(r"\b(?:\d[ -]?){13,19}\b")
5

6
class Assistant(Agent):
7
    def __init__(self):
8
        super().__init__(instructions="You are a helpful voice AI assistant.")
9

10
    async def on_user_turn_completed(self, turn_ctx: ChatContext, new_message: ChatMessage) -> None:
11
        new_message.content = [CARD.sub("[card number redacted]", new_message.text_content or "")]

Pipecat, Python. There's no hook above the pipeline, so you write a processor and insert it between speech-to-text and the context aggregator. It has to forward every frame it doesn't care about, or the pipeline stalls.

1
import re
2
from pipecat.frames.frames import Frame, TranscriptionFrame
3
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
4

5
CARD = re.compile(r"\b(?:\d[ -]?){13,19}\b")
6

7
class CardNumberRedactor(FrameProcessor):
8
    async def process_frame(self, frame: Frame, direction: FrameDirection):
9
        await super().process_frame(frame, direction)
10
        if isinstance(frame, TranscriptionFrame):
11
            frame.text = CARD.sub("[card number redacted]", frame.text)
12
        await self.push_frame(frame, direction)
13

14
pipeline = Pipeline([
15
    transport.input(),
16
    stt,
17
    CardNumberRedactor(),   # new stage, placed by you
18
    user_aggregator,
19
    llm,
20
    tts,
21
    transport.output(),
22
    assistant_aggregator,
23
])

Both are short. The difference is where the change lives and what it can break. On LiveKit you changed the agent; the session's pipeline, turn-taking, and interruption handling are exactly what they were. On Pipecat you changed the pipeline; the new stage is now part of the path every frame takes, and its correctness (did you forward interim transcriptions? system frames? end-of-turn signals?) is yours.

Step 3: a requirement only one of them can express in the pipeline

Now product wants a second model scoring every call for compliance, in parallel, without adding latency to the conversation. This is where Pipecat's design pays off. ParallelPipeline forks the frame stream into branches that run concurrently; the conversation branch is untouched and the scoring branch sees the same transcripts.

1
from pipecat.pipeline.parallel_pipeline import ParallelPipeline
2

3
pipeline = Pipeline([
4
    transport.input(),
5
    stt,
6
    ParallelPipeline(
7
        # Branch 1: the conversation, exactly as before
8
        [user_aggregator, llm, tts, transport.output(), assistant_aggregator],
9
        # Branch 2: a scorer that reads the same transcripts and never speaks
10
        [compliance_scorer],
11
    ),
12
])

LiveKit Agents has no equivalent declaration, because the agent isn't a pipeline you can fork. You'd get the same result a different way: the scorer joins the room as a second participant and subscribes to the same audio track, or you fan out inside an llm_node override. Both work in production; neither is a single line in the agent definition. If your architecture is built around many parallel branches, Pipecat's approach is the more natural fit, and this guide won't pretend otherwise.

Names you'll also see, and why they come up#

TEN Framework. Backed by Agora. Comes up because it's active, multimodal, and has around 11k stars. Two reasons it isn't a developer option in the same sense: its license is the Apache 2.0 text plus a clause that you may not deploy it "in a way that competes with Agora's offerings and/or that allows others to compete," and may not host it on end-user devices, which makes it source-available rather than open source; and its only shipped WebRTC transport is Agora's proprietary, metered service (a community request for a LiveKit transport has been open since 2024). If you're already committed to Agora and the license fits, it works there.

Dograh. BSD-2-Clause. Comes up as "the open source alternative to Vapi and Retell": a self-hosted visual workflow builder with telephony, BYOK across LLM/STT/TTS, and an MCP server. It's a platform, not a framework, which puts it in a different category from everything else on this list. Dograh vendors a fork of Pipecat as a git submodule and builds a product on top of it, so its framework question is already answered underneath: Pipecat. If you can't use SaaS and the people configuring agents aren't engineers, look at it. If you're deciding what to build on rather than what to deploy, you're still choosing between the two frameworks above.

Vision Agents. Apache 2.0, from Stream. Comes up on video-agent searches; around 8k stars and active. Combines computer vision models with LLMs for agents that watch live video. Same shape as TEN without the license problem: an open framework whose transport is one vendor's proprietary edge network. Worth a look if you're already on Stream.

Vocode. MIT. A first-generation framework that shaped how the category thought about streaming. Development stalled in late 2024. Not recommended for new work.

How teams choose between LiveKit Agents and Pipecat#

When teams actually sit down to pick one for their production application, it's usually a handful of practical questions that decide it, more than any nuance in architecture.

1. Do I need TypeScript or Python?#

If you code in TypeScript, LiveKit Agents is your choice; its Node.js SDK tracks the Python one closely, though a handful of options (like automatic gain control) remain Python-only. If you're a Python shop, both frameworks are still on the table. LiveKit Agents ships in Python and Node.js, whereas Pipecat is exclusively Python.

2. Is the framework well-maintained and here for the long-haul?#

If you've ever been tasked with choosing a framework, there's inevitably an engineering leader on the other end (or maybe it's you) who has seen dozens of frameworks come and go across their career and wants to know that the framework you choose will stand the test of time. The point of this analysis isn't to blindly pick the more popular option; it's to make sure you're not adopting something that will be unmaintained by the time you're in production. And it's a fair concern. One of the category's first popular open source voice-agent frameworks in the LLM era, Vocode, amassed nearly 4,000 GitHub stars before it stopped shipping updates in late 2024.

Both LiveKit Agents and Pipecat pass. Here are some signals worth considering, and where to check them yourself. The figures below are refreshed regularly, and the links let you check them yourself at any time.

Stats alone don't always tell the perfect story, so here's some additional context to the table:

• Downloads are one of the strongest signals. Anyone who's looked at PyPI or npm stats knows the raw numbers are inflated by CI pipelines and dependency installs, but the ratio survives that: a rough 3x gap on PyPI is a meaningful difference in the number of teams likely adopting the framework.
• Stars are roughly even and growing at similar rates. Between January 1 and September 15, 2026, both grew about 63–64%. Stars don't mean teams are adopting the framework, but they're a decent signal of whether one is running away as the default or one is at risk of being abandoned. Both have cleared 10k, both are the two most popular frameworks in the space, and neither has run away by this metric alone.
• The open source server passes the longevity test. LiveKit Agents runs on an open source media server with its own large community and history. If the framework's company disappeared tomorrow, the transport it depends on would still exist and still be maintainable. Pipecat's transports are other companies' services, Daily's foremost, so its longevity question is really Daily's. It also settles a practical question: Daily doesn't publish an open source WebRTC server, so a team that wants Pipecat, WebRTC, and a fully open source stack may end up running LiveKit's server underneath anyway. Pipecat even ships a LiveKit transport.
• Each is updated regularly. Both frameworks ship weekly; the day one stops is the day to re-run this test.
• There's funding to indicate runway for both. One of the biggest risks to an open source framework is that the maintainers behind it lose the financial incentive to keep investing. Both companies seem to have found a way to make money from the framework they maintain, which typically means solid runway. LiveKit's most recent round alone ($100M, January 2026) is larger than any round Daily has disclosed, which is a fair proxy for how much each company can invest in its framework.
• Both have legit companies deploying them in production. The lists differ in what kind of user they name. LiveKit's are companies running voice agents with their own customers: Salesforce, Zocdoc, SAP, telli. Pipecat's lean toward companies that build tooling or reference architectures for other developers: NVIDIA's Blueprint, AWS, Anthropic.

3. Can I get a working agent going today?#

When evaluating each framework it's important that you can easily build (and reason about the code for) a voice or video agent. Given AI-augmented development, a developer should be able to get an agent working in a day with either. Independent write-ups comparing the two consistently land on the same description: LiveKit Agents is "less code," Pipecat is "more boilerplate."

• LiveKit Agents: An AgentSession wires speech-to-text, the model, and text-to-speech for you. Turn-taking ships as a set of LiveKit-trained models: a turn detector on by default, and adaptive interruption handling that tells a real barge-in from a "yeah, go on" — the default mode for agents deployed to LiveKit Cloud when used with most STT providers. Both are config. The quickstart is a single file. Python and Node.js.
• Pipecat: You declare the pipeline explicitly, Pipeline([transport.input(), stt, context_aggregator.user(), llm, tts, transport.output(), context_aggregator.assistant()]), and every processor you add must correctly forward the frame types it doesn't handle. The foundational examples are numbered for a reason: there's a sequence to learn. Turn detection ships tuned here too (Smart Turn v3 is on by default), so the extra work is in the wiring, not the defaults. Python only on the server side.

LiveKit also ships Agent Builder: prototype a voice agent in the browser, test it, and export best-practice Python you keep editing in code. Pipecat has no equivalent; its fastest path is the numbered examples plus Claude Code skills.

Both frameworks now ship support for coding agents. Pipecat's README lists Claude Code skills for scaffolding and deploying; LiveKit publishes an MCP docs server and agent skills. That narrows the gap for teams that let an AI write the first draft. But the shape of the first hour is the same: one framework hands you a running pipeline to modify, the other hands you the parts.

4. Will it hold up in production?#

Choosing an open source framework that's easy to build with and going to be around for a while is one important side of the coin, but the other is: will it hold up in production? In reality, that's often more due to the platform, infrastructure, community, and support around the framework than the framework itself.

Picture the moment this question gets real. The agent works. Now it needs to take phone calls and browser sessions from three continents, pass a security review, and tell you why Tuesday's calls performed worse than Monday's. The framework is the same size in both projects at that point; what differs is how much of everything else already exists, and who on your team has to own the parts that don't.

Can I scale it globally without an entire team of DevOps?

The difference is how many things you're operating. On LiveKit, the media layer, agent hosting, telephony, and model access are one platform: LiveKit Cloud runs the media layer as a global mesh (nearest-edge connections, no participant cap per room), hosts agents with autoscaling, instant rollback, and burst capacity in three regions, carries phone calls through its own SIP with native numbers, and fronts model providers through LiveKit Inference with failover between them, against a 99.99% uptime target. A browser user, a phone caller, and a device all land in the same room, so adding a channel doesn't touch agent code. If you prefer to own it, the open source stack self-hosts with full parity; only the Cloud layer on top (Inference, hosting, observability, Phone Numbers, Connectors) doesn't.

For a concrete example, telli moved from another open source voice framework to LiveKit because it had become "harder to ship continuous improvement without breaking something else," and migrated 100% of call volume in three weeks: 30,000+ daily calls across 400+ SIP trunks and 30+ languages, with EU data residency and direct Deutsche Telekom trunks.

On Pipecat, agent hosting is solved and solved well: Pipecat Cloud scales from zero in four regions, starting at 50 active sessions per deployment with more on request. Everything beside it is a decision you make and then run: the media layer is Daily, a peer-to-peer connection into your process (you supply TURN), a WebSocket, or LiveKit; telephony is Daily's PSTN/SIP or a carrier stream that Pipecat's serializers terminate inside your process; models are your own keys and contracts. That's the flexibility from the architecture section, and it's real. It also means someone on your team owns the media layer, the TURN servers, and the carrier relationships. Global scale is a project you run rather than a plan you pick.

Can it meet strict privacy and compliance regulations?

Both clear the bar a procurement team will hold up: SOC 2 Type II and a HIPAA BAA, on LiveKit Cloud's Scale and Enterprise plans and through Daily's HIPAA package for Pipecat Cloud. What differs is how far one agreement reaches and how many times you have to sign.

LiveKit's BAA covers the inference path too, because that path is LiveKit's: Inference is zero data retention by default on every plan, so neither LiveKit nor the model providers log or train on your audio and prompts. PII redaction strips 41 entity types from transcripts and recordings before anything is stored. End-to-end encryption works on Cloud and self-hosted. For GDPR, EU residency controls exist at each layer: EU data region, EU agent hosting, EU-only inference routing, regional SIP endpoints. On Pipecat, the framework stores nothing and Daily states it doesn't store your data, but every STT, LLM, and TTS provider is your own contract, so retention terms, and BAAs if you're in healthcare, are negotiated one provider at a time. More paperwork; also more direct control. On either platform, residency settings are configuration, not a compliance guarantee, and GDPR obligations still sit with your application and your data path.

Can you easily observe it?

The question in production is rarely "is it up." It's "was that bad call a network problem, a model problem, or something else," which needs transcripts, model timing, participant events, and media telemetry in one place. LiveKit Cloud ships that on every plan: turn-by-turn transcripts, per-stage traces with token counts, agent logs, and the actual audio on one session timeline, 30-day retention, plus log drains for server-level events. It requires LiveKit Cloud media servers; a fully self-hosted deployment uses OpenTelemetry hooks and your own backend instead.

Pipecat's OpenTelemetry tracing is well designed (conversation → turn → service spans with token and character counts) and exports to any OTLP backend, and Pipecat Cloud adds logs with Datadog integration and stored transcripts. What isn't included is the assembled view: the backend, retention, audio storage, and the correlation between media events and model events are yours to build, which usually means one more dashboard someone maintains.

Who helps when you need it?

The tiers look alike until the top. Both run community → email → enterprise: LiveKit's forum and email support on Ship and above; Pipecat's Discord and email support for Pipecat Cloud customers. LiveKit's Enterprise tier adds a shared Slack channel, a designated solutions engineer, and a support SLA, and LiveKit staffs a forward-deployed engineering team that works inside customer engineering teams, remotely or on-site, from architecture through production debugging. Daily's Enterprise tier lists 24/7 priority support with dedicated account management; nothing comparable to forward-deployed engineers is published.

The largest public proof point for all four answers is OpenAI building ChatGPT's Advanced Voice on LiveKit. Read it as a case study rather than a benchmark; end-to-end results in any deployment still depend on user location, network quality, the models you pick, and how your tools behave.

Put the four answers together and the pattern is obvious. Pipecat gives you a framework and lets you choose the platform around it. LiveKit gives you a framework that already has one. That might not be a huge deal for a demo, but at ten thousand concurrent calls across three regions with a compliance team asking questions, it's often a major contributor to the decision for teams planning to operate at scale.

Which one should you choose?#

Choose LiveKit Agents if you're building an agent that needs to reach production with multi-platform clients, telephony, video, or enterprise scale, and you'd rather have the undifferentiated parts of the infrastructure layer solved for you.

Choose Pipecat when the pipeline itself is the product: a research stack or a novel architecture that isn't speech-to-text → model → text-to-speech (or speech-to-speech).

Frequently asked questions#

Is LiveKit open source?

Yes. The LiveKit media server, SIP service, egress, ingress, agents framework, and all client SDKs are Apache 2.0 and self-hostable with full feature parity. LiveKit Cloud is a managed deployment of the same server plus a platform layer around it.

Is Pipecat open source?

Yes. Pipecat is BSD-2-Clause and self-hostable. Pipecat Cloud is a separate managed hosting product.

Is TEN Framework open source?

Not in the Open Source Definition sense. Its license is the Apache 2.0 text plus restrictions: you may not deploy it in a way that competes with Agora's offerings, and you may not host it on end-user devices. The accurate term is source-available.

Can I use LiveKit Agents without LiveKit Cloud?

Yes. The framework runs against a self-hosted LiveKit server with the same APIs. What Cloud adds is the global mesh, managed agent hosting, built-in inference, observability, phone numbers, the Twilio and WhatsApp Connectors, and Cloud-hosted conversational models such as adaptive interruption handling.

Can I use Pipecat with LiveKit as the transport?

Yes. Pipecat ships a LiveKit transport. Some teams run Pipecat pipelines on LiveKit's media server to get its transport, SIP, and scaling while keeping Pipecat's pipeline model.

Is LiveKit Agents only for simple bots and prototyping?

No; that framing gets the trade-off backwards. The agent model is faster to start with, but its value shows up later: session logic and agent logic are separate, the framework owns the pipeline stages you didn't touch, and a change to one agent can't silently break turn-taking or interruption handling elsewhere. Those properties matter most when you have many agents, many customers, and a compliance requirement, not when you have a demo.

Is LiveKit Agents provider-agnostic?

Yes. Speech-to-text, language model, and text-to-speech providers are pluggable, and swapping one is a configuration change. LiveKit Inference on Cloud goes further by routing multiple providers through one account with mid-session failover. The one thing LiveKit Agents is not agnostic about is transport: it runs on the LiveKit media server, which is Apache 2.0 and self-hostable.

Does LiveKit Agents support parallel processing on the same audio?

Yes. There are two ways to do it, and you pick one. You can override stt_node / sttNode and split the audio stream, so a second consumer runs alongside STT inside the same agent. Or, since every agent is a participant, a second agent or process can subscribe to the same audio track and run its own pipeline independently.

Which frameworks support video?

Both. LiveKit Agents and Pipecat support video input to models and virtual avatar output. LiveKit treats video as a first-class track on the same media server the agent runs on.

Which frameworks support telephony?

Both. LiveKit Agents has a native, self-hostable SIP service, LiveKit Phone Numbers, and Cloud Connectors for Twilio Media Streams and WhatsApp. Pipecat has serializers for Twilio, Telnyx, Plivo, Vonage, Exotel, and Genesys AudioHook.

Is there an open source alternative to Vapi or Retell?

If you want a hosted-platform experience (visual builder, non-technical users) that you run yourself, Dograh is the closest thing, and it's built on Pipecat. If you're a developer, build your agent on LiveKit Agents or Pipecat directly; both give you more than a visual builder can.

Bottom line#

The open source options for realtime voice and video agents are stronger than they've ever been, and the two that matter are both well-built and free to run. The decision comes down more to whether you already have a global realtime infrastructure layer or need one, and how many knobs you want your developers turning versus getting best practices by default with the ability to override.

If assembling the pipeline is your product, or your deployment has to terminate carrier audio directly with no media server, Pipecat may be the better tool. For most teams building an agent that has to reach production, with real users on real devices, phones on real carriers, video, and a compliance team with questions, the framework that comes with its infrastructure and knows what it's running on will get you there faster and keep you there longer.