Skip to main content
Blog/Product

Introducing the C++ SDK

Since we launched LiveKit, developers have built realtime experiences on top of our infrastructure across telephony, web, mobile apps, and embedded devices like the ESP32 and Raspberry Pi. As robotics, autonomy, and physical AI move from research into production, developers need a solid primitive that meets them where they actually work.

Today we're shipping version 1.0.0 of the LiveKit C++ SDK. It gives C++ applications a lightweight, ergonomic way to publish and subscribe to audio, video, and data tracks on LiveKit's realtime infrastructure, with the same low latency and bandwidth efficiency you get from our other clients.

Why C++?

Modern robots are built with C++, including platforms like ROS2, PX4, Autoware and NVIDIA's Isaac ROS. Humanoid robots are moving from research labs into factories and warehouses, robotaxis are carrying paying passengers in a growing list of cities, and drones, surgical robots, and industrial systems are getting much more capable every day.

These systems share a problem. The intelligence they depend on lives in the cloud, and the machine has to talk to it in realtime: a stream of audio, video, and data flowing between a machine and a model in both directions, with very little latency budget to spare. Wrapping a C++ autonomy stack in another language's runtime to get it onto the network has always been a workaround. The LiveKit C++ SDK gives those teams a native path instead.

What you can build

The LiveKit C++ SDK opens up a wide range of new use cases for developers, especially when paired with recent capabilities we’ve shipped, like data tracks for structured payloads alongside audio and video, and wake word activation so embedded devices can sit idle until called.

Teams across all areas of robotics such as autonomous vehicles, humanoid robots, and simulation have already been using earlier builds across a range of workloads, including:

  • Realtime robotics. Stream sensor and camera data off a robot into a cloud agent, or send control signals back the other way. Our teleop demo drives a pan/tilt rig over LiveKit, and the RealSense demo streams depth and color data while logging it to MCAP for replay.
  • Low-latency AI inference. Bring native models into a LiveKit room as another participant, with raw frame access on both the publish and subscribe sides.
  • High-performance media applications. Embed LiveKit transport into existing C++ media pipelines (FFmpeg, GStreamer, broadcast tools, AR/VR engines) without paying the cost of a programming language hop.
  • High-performance video pipelines. LiveKit supports dedicated hardware encoders on discrete NVIDIA GPUs, and AMD/Intel CPUs. We will soon support dedicated encoders on NVIDIA Jetson, Rockchip MPP, and Raspberry Pi!

Supported platforms

The LiveKit C++ SDK runs on Linux, macOS, and Windows, on both x86-64 & ARM devices such as NVIDIA Jetson, Raspberry Pi, and Rockchip SBCs. Integration is CMake-first. The LiveKitSDK.cmake example shows how to pull the SDK into an existing CMake project in a few lines. If you're working in an even tighter footprint, LiveKit also has a native ESP32 SDK.

Hello, LiveKit

Here is a minimal example of the main function for sending and receiving video and data track frames. The sender plays the role of a robot or camera, publishing video and data track frames every 100 ms; the receiver stands in for the cloud service or operator UI, logging every frame it sees. In a production system the synthetic video would be a robot's perception output and the data track would carry sensor readings or operator commands, but the connection and publishing pattern is the same. Full source for both processes lives in the cpp-example-collection repo.

The sender creates tracks and publishes data.

Initialize LiveKit and connect to the room

1
#include "livekit/livekit.h"
2


3
// Get your url and token from env vars, args, etc.
4
const std::string url = "wss://hello.livekit.cloud";
5
const std::string token = "sender_token";
6


7
// Start the LiveKit SDK before creating rooms or tracks.
8
livekit::initialize(livekit::LogLevel::Info);
9


10
// Set your room options, here we will use defaults.
11
livekit::RoomOptions options;
12


13
// Create the room & connect to the room using a server URL and participant token.
14
auto room = std::make_unique<livekit::Room>();
15
if (!room->connect(url, token, options)) {
16
  std::cerr << "Failed to connect to LiveKit\n";
17
  return 1;
18
}

Create the VideoSource, which provides frames to the VideoTrack, then create and publish the VideoTrack

1
// Get the local participant to create tracks.
2
auto participant = room->localParticipant().lock();
3
if (!participant)
4
{
5
  std::cerr << "Unable to get the local participant!" << std::endl;
6
  return 1;
7
}
8


9
// Publish a synthetic camera track named "camera0" backed by a VideoSource.
10
auto video_source = std::make_shared<livekit::VideoSource>(640, 480);
11
auto video_track = participant->publishVideoTrack("camera0", video_source, livekit::TrackSource::SOURCE_CAMERA);
12
if (!video_track) {
13
  std::cerr << "Failed to publish video track\n";
14
  return 1;
15
}

Create and publish the DataTrack

1
// Publish a data track named "app-data" for app messages.
2
auto data_track_result = participant->publishDataTrack("app-data");
3
if (!data_track_result) {
4
  std::cerr << "Failed to publish data track\n";
5
  return 1;
6
}
7
auto data_track = data_track_result.value();
8


9
// Release the participant to reduce scope.
10
participant.reset();

Publish video and data track frames every 100ms

1
int count = 0;
2


3
while (true)
4
{
5
  // Create a 640x480 RGBA video frame.
6
  auto vf = livekit::VideoFrame::create(640, 480, livekit::VideoBufferType::RGBA);
7


8
  // Capture the frame. This publishes the frame on VideoTrack camera0.
9
  video_source->captureFrame(vf);
10


11
  const std::string message = "hello #" + std::to_string(count);
12
  const auto now_microsec =
13
        std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
14


15
  // Create and push stamped DataTrackFrame.
16
  const livekit::DataTrackFrame frame{
17
        std::vector<std::uint8_t>(message.begin(), message.end()),
18
        now_microsec,
19
        };
20


21
  // Optionally, capture the result.
22
  auto push_result = data_track->tryPush(frame);
23
  if (!push_result) {
24
    const auto& error = push_result.error();
25
    std::cerr << "[warn] Failed to push data frame: code=" << static_cast<std::uint32_t>(error.code)
26
              << " message=" << error.message << "\n";
27
  }
28


29
  ++count;
30
  std::this_thread::sleep_for(std::chrono::milliseconds(100));
31
}

The receiver side setup is the same, except now we set callbacks to the relevant tracks.

Initialize LiveKit and Connect to the room

1
#include "livekit/livekit.h"
2


3
// Get your url and token from env vars, args, etc.
4
const std::string url = "wss://hello.livekit.cloud";
5
const std::string token = "receiver_token";
6


7
// Start the LiveKit SDK before creating rooms or tracks.
8
livekit::initialize(livekit::LogLevel::Info);
9


10
// Set your room options, here we use the defaults.
11
livekit::RoomOptions options;
12


13
// Create the room & connect to the room using a server URL and participant token.
14
auto room = std::make_unique<livekit::Room>();
15
if (!room->connect(url, token, options)) {
16
  std::cerr << "Failed to connect to LiveKit\n";
17
  return 1;
18
}

Set callbacks for new video and data frames

1
// The identity of the participant sending video and data frames.
2
const std::string sender_identity = "sender_identity";
3


4
// Set the callback for new video frames from the sender's "camera0" VideoTrack.
5
room->setOnVideoFrameCallback(sender_identity, "camera0", [](const livekit::VideoFrame& frame, std::int64_t) {
6
  std::cout << "video frame: " << frame.width() << "x" << frame.height() << "\n";
7
});
8


9
// Set the callback for new frames on the sender's "app-data" DataTrack.
10
room->addOnDataFrameCallback(sender_identity, "app-data",
11
                             [](const std::vector<std::uint8_t>& payload, std::optional<std::uint64_t>) {
12
                               const std::string message(payload.begin(), payload.end());
13
                               std::cout << "data message: " << message << "\n";
14
                             });

For end-to-end samples and a fuller set of demos, see the cpp-example-collection repo.

What’s next

The C++ SDK is part of our investment in robotics and embedded systems. Our near-term roadmap includes:

  • Recording and replay, so robotics and media pipelines can capture sessions for debugging, evaluation, and training data.
  • A ROS2 bridge, so a LiveKit room can show up directly as ROS2 topics on either side of the connection.

Voice and multimodal AI agents are moving out of browsers and phone apps and into the physical world: robots, cameras, sensors, vehicles, drones, factory floors. The transport layer for that world has to reach both ends of a session natively. Our web and mobile SDKs have covered the human end; the C++ and ESP32 SDKs are how we cover the device end, and we plan to keep pushing on both.

Getting started

You can use the LiveKit C++ SDK now:

Issues, pull requests, and feedback are welcome on GitHub and in the Robotics category in our community forums.