Skip to main content
close
Blog/Tutorial

Streaming video from a canvas with WebRTC and React

With WebRTC you can live stream video from a canvas. This post is a step-by-step guide which shows you how. We use LiveKit’s WebRTC stack to build a real-time application for sending canvas video. Check out the full code.

VTuber Sample App Gif VTuber Sample App Gif

A lot of people know WebRTC as the technology that powers video chats on web. This is great for things like Google Meet, but it can be used to build much more than simple conference calls.

In fact, anything you can put on a canvas (that’s a lot of things!) can be sent as video into a WebRTC session.

Before we go into the details, note that the techniques here should apply generally to any stack. We use LiveKit to simplify the publishing flow. Follow this mini tutorial to learn how to set up your own LiveKit project.

Capturing a Canvas Using captureStream()

WebRTC works on the concept of MediaStreams which are…uhhh…streams of media. In order to send a canvas into a session, we need to get a MediaStream from the canvas. Luckily, this is super easy using the HTMLCanvasElement.captureStream() api.

Let’s create a simple canvas that draws a circle and then create a MediaStream from it:

1
export const CircleCanvasPublisher() {
2
  const canvasRef = useRef<HTMLCanvasElement | null>(null);
3
  const mediaStreamRef = useRef<MediaStream | null>(null);
4


5
  // Draw a circle
6
  useEffect(() => {
7
    if(!canvasRef.current) return;
8
    const ctx = canvasRef.current.getContext("2d");
9
    ctx.beginPath();
10
    ctx.arc(canvasRef.current.width / 2, canvasRef.current.height / 2, 50, 0, 2 * Math.PI);
11
    ctx.fillStyle = "blue";
12
    ctx.fill();
13
  }, []);
14


15
  // Create MediaStream
16
  useEffect(() => {
17
    if(mediaStreamRef.current || !canvasRef.current) return;
18


19
    // Create 30fps MediaStream
20
    mediaStreamRef.current = canvasRef.current.captureStream(30);
21
  }, [])
22


23
  return (
24
    <canvas width={300} height={300} ref={canvasRef} />
25
  )
26
}

In the above snippet we:

  • Created a canvas and stored a reference to it
  • Drew a small circle to the canvas (because why not?)
  • Created a MediaStream from the canvas at 30 frames per second and stored a reference to it

Now that we’ve got the MediaStream. All that’s left is to publish it to your session. With LiveKit this is is just a couple lines of code:

1
// At the top of the component
2
const { localParticipant } = useLocalParticipant();
3


4
// In the useEffect that creates the MediaStream
5
const publishedTrack = mediaStreamRef.current.getVideoTracks()[0];
6
localParticipant.publishTrack(publishedTrack, {
7
  name: 'video_from_canvas',
8
  source: Track.Source.Unknown,
9
});

Putting it All Together

And, voila! That’s how simple it is to send a canvas as video. We used the same technique on a WebGL canvas for our VTuber streaming app:

Here’s the full code including some additional code to create a session using LiveKit:

1
import { useEffect, useRef, useCallback } from "react";
2
import { Track } from "livekit-client";
3
import { useLocalParticipant } from "@livekit/components-react";
4


5
export const CircleCanvasPublisher() {
6
  const canvasRef = useRef<HTMLCanvasElement | null>(null);
7
  const mediaStreamRef = useRef<MediaStream | null>(null);
8
  const { localParticipant } = useLocalParticipant()
9


10
  // Draw a circle
11
  useEffect(() => {
12
    if(!canvasRef.current) return;
13
    const ctx = canvasRef.current.getContext("2d");
14
    ctx.beginPath();
15
    ctx.arc(canvasRef.current.width / 2, canvasRef.current.height / 2, 50, 0, 2 * Math.PI);
16
    ctx.fillStyle = "blue";
17
    ctx.fill();
18
  }, []);
19


20
  // Create MediaStream
21
  useEffect(() => {
22
    if(mediaStreamRef.current || !canvasRef.current) return;
23


24
    // Create 30fps MediaStream
25
    mediaStreamRef.current = canvasRef.current.captureStream(30);
26


27
    // Publish the canvas video
28
    const publishedTrack = mediaStreamRef.current.getVideoTracks()[0]
29
    localParticipant.publishTrack(publishedTrack, {
30
      name: "video_from_canvas",
31
      source: Track.Source.Camera
32
    })
33
  }, [])
34


35
  return (
36
    <canvas width={300} height={300} ref={canvasRef} />
37
  )
38
}
1
import { LiveKitRoom } from "@livekit/components-react";
2
import { CircleCanvasPublisher } from "@/components/CircleCanvasPublisher";
3


4
type RoomProps {
5
  token: string
6
  wsUrl: string
7
}
8


9
export const Room = ({token, wsUrl}: RoomProps) => {
10
  return (
11
    <LiveKitRoom token={token} serverUrl={wsUrl} connect={true}>
12
      <CircleCanvasPublisher />
13
    </LiveKitRoom>
14
  )
15
}

Join Us

You should now have everything you need to publish canvas media in a WebRTC session. If you have any feedback on this post or end up building something cool using or inspired by it, pop into the LiveKit Slack community and tell us. We love to hear from folks!