Use Virtual Backgrounds in your Video Chat Website with the Agora Web UIKit

March 21, 2023 By Ekaansh Arora In Developer

The Agora SDK introduced the virtual background extension that lets you apply a color, image or blur to the background of your video in a video call. In this blog post, we’ll take a look at how we can use this extension with the Agora Web UIKit.

You can find a code snippet at the end of this post to use the extension without the UIKit.

Prerequisites

An Agora developer account (It’s free, sign up here!)
Node.js LTS
A high-level understanding of React

Setup

You can get the code for the example on GitHub, or you can create your own React project. Open a terminal and execute:

npx create-react-app demo --template typescript
cd demo

Install the Agora React UIKit & the Virtual Background Extension:

npm i agora-react-uikit agora-extension-virtual-background

Note: At the time of writing, the current agora-react-uikit release is v1.1.0.

That’s the setup. You can now execute npm start to start the web server on localhost.

Building the Live Stream

The UIKit gives you access to a high-level component called <AgoraUIKit> that can be used to render a full video call or live stream, read more. We’ll use the modular components to compose the video call and add the virtual background feature.

Clear out the App.tsx file to start fresh:

We’ll create a state variable called videocall. When it’s true we’ll render the videocall, and when it’s false we’ll render a button to toggle the application state and start the stream.

To build the videocall, we’ll wrap everything with the PropsContext component to pass in the user props to the UIKit. We’ll add the props to the rtcProps — Agora App ID, channel name and token. When the user clicks the end call button, we’ll toggle to set the videocall state to false.

Note: Token authentication is recommended for all RTE apps running in production environments. For more information about token-based authentication in the Agora platform, see this guide: https://docs.agora.io/en/Video/token?platform=All%20Platforms.

We’ll wrap our application with in-built components from the UIKit, these components give us access to UIKit data and handle a lot of the video call logic for us. We’ll use TrackConfigure: gives us access to the local tracks, RtcConfigure: handles the call logic, LocalUserContext: handles the local user state and RtmConfigure: handles the signalling logic.

We’ll render these components from the UIKit — RemoteMutePopup: display the mute request popup, GridVideo: renders the user videos in a grid and LocalControls: renders the controls to mute/unmute the local tracks. We’ll add a new VirtualBackground component that we’ll define now!

Using the Agora virtual background extension

To make the virtual background performant, Agora uses a WebAssembly (WASM) module. We’ll need to host the WASM file — you can host it separately via a CDN or bundle it with your website. Let’s create the VirtualBackground component:

We’ll create a state variable extensionActive to enable/disable the virtual background. We can get access to the localVideoTrack from the TracksContext. We’ll create an instance of the VirtualBackgroundExtension called ext and we’ll create a ref to store the background processor.

Inside the useEffect we’ll initialize the extension by calling the registerExtensions method with our extension instance. We’ll call the createProcessor method on the extension and assign it to the processor ref. We’ll then call the init method on the extension with the path to the WASM file, this can be an absolute or a relative URL.

We’ll create the enableBackground where we pipe the localVideoTrack to the processor which is then piped to track’s processorDestination. We can use the setOptions method to define the virtual background, for this example I’m using a pink color, but you can also use a blur effect or an image. We’ll enable the processor and update our state.

For the disableBackground function, we’ll just unpipe the local track and disable the processor before updating the state.

For the UI we’ll render a div element that calls the correct method to toggle the effect based on our state.

If you’re not using the UIKit you can use the following snippet:

Conclusion

That’s how you can use a WebAssembly based Agora Extension for Virtual Backgrounds with the Agora Web UIKit. You can read more about how you can customize the design and functionality of the UIKit (for example, you can use it for live-streaming) in this blog post.

If you have questions while using this project or the Agora Web UIKit, I invite you to join the Agora Developer Slack community, where you can ask them in the #web-help-me channel. Feel free to open issues for feature requests or to report bugs on the UIKit Repo or the project repo.