Add Custom Backgrounds to your Live Video Calling application using the Agora Flutter UIKit

Creating a video streaming app is amazingly easy with the Agora SDK and the Agora Flutter UIKit. Moreover, Agora has many features that enhance video call quality and convenience. For example, virtual backgrounds bring life to video calls.

We can customize the background of a video call in three ways:

• Using an image as the background
• Applying a solid color to the background
• Applying blur effects to the existing background

In this tutorial, you will learn how to add virtual backgrounds in Flutter using the Agora Flutter SDK and the Agora Flutter UIKit.

Prerequisites and requirements

• An Agora developer account — Sign up here
• Knowledge of how to create a live-streaming Flutter application using Agora’s Flutter UIKit. Check out the tutorial here.
• Basic knowledge of Flutter development.
• An editor like Android Studio or VS Code, which supports Flutter development
• An Android/iOS device.

Setup

Step 1: If you are not integrating this into an existing project, create a new Flutter application using your editor.

Step 2: Open your pubspec.yaml file and add the following dependencies

image_picker: ^0.8.4+9
  agora_uikit:
    git:
      url: https://github.com/AgoraIO-Community/Flutter-UIKit

Step 3: Make sure your project-level build.gradle under Android directory has the following:

allprojects {
    repositories {
        google()
        mavenCentral()
        maven { url 'https://www.jitpack.io' }
    }
}

Step 4: Lastly, we need to add the following permissions to our AndroidManifest.xml file under android/app/src/main directory for necessary user permissions:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.BLUETOOTH" />

And, voila! We’re done setting up, Now, let’s dive in to some code!

Video Call UI

For our application, we won’t have to work on building the UI too much, because all that is already taken care of by the Agora Flutter UIKit.

Implementing Virtual Background

Here, we have used main.dart as the main activity used for launching our video call app

Importing necessary dependencies

import 'package:agora_uikit/agora_uikit.dart';
import 'package:custom_background_flutter/constants.dart';
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';
import 'package:agora_rtc_engine/rtc_engine.dart';
import 'package:image_picker/image_picker.dart';

Declaring some variables

We will be needing the following three variables in our entire implementation.

• _client — This is a variable of type AgoraClient used to calling the Flutter UIKIT without which the video call would not be possible
• _engine — This is a variable of type RTCEngine is used to utilize all the methods of Agora’s Flutter SDK
• _picker — This variable of type ImagePicker is used fetch images either from the phone’s gallery or camera
• _virtualBackgroundToggle — This bool variable is used maintain whether the Virtual Background Effect is enabled or disabled

We can add these in the Stateful/Stateless widget which we will be passing to the home property of our Material Class, in the following manner:

late AgoraClient _client;
late RtcEngine _engine;
final ImagePicker _picker = ImagePicker();
late bool _virtualBackgroundToggle;

Initializing the Flutter UIKIT

We need to initialize the Flutter UIKIT in the initState() before the build method is called

VirtualBackground Effect Button

We will use the extraButtons property in AgoraVideoButtons class to make a button to enable or disable the Virtual Background effect. Over here, we will choose which effect to apply.

Virtual Backgrounds automatically detect any subjects in the video call and doesn’t overlap with any of them. These come in 3 different ways in Agora’s Flutter SDK

• Virtual Background Image — We can fetch any image from the phone’s gallery or camera using any available package and add that as a background to our video call.
• Virtual Background Color — We can add any color in it’s hex form as our background.
• Virtual Background Blur — We can add an additional blur effect our existing background in three levels: Low, Medium and High.

It’s crucial to remember to enable only one of the above Virtual Backgrounds at a time. In the above gist, I have used Virtual Background Image, as an example. We can enable any of the other two while the commenting out the rest.

Applying Virtual Background Image

We will apply Virtual Background Image in the following steps:

• Fetch image from Gallery/Camera using a third-party package by launching appropriate intents
• Enable the Virtual Background Image when the Activity returns a result
• The image chosen must match these specifications

Applying Virtual Background Color

We can use any hex color for Virtual Background which matches these specifications.

Applying Virtual Background Blur

We can apply three types of Blur Levels: Low, Medium and High

Enabling Virtual Background

Finally, we need to enable the Virtual Background using Agora’s Flutter UIKIT and Flutter SDK. Here, we will implement the enableVirtualBackground() method which you might have found above.

Disabling Virtual Background

This is where we will simply disable the Virtual Background in just one line of code! All thanks for Agora’s amazing Flutter UIKIT and Flutter SDK

Conclusion

We now have a video call app with cool Virtual Backgrounds! We should be able to run the app successfully in our Android application.

There are a ton of other features which are all very simple to implement. You can check them out here.

Testing

You can try out this app by following the GitHub link. After cloning the repo, just run the app on a physical Android device to test the application.

Other Resources

To learn more about the Agora Android SDK and other use cases, see the developer guide here.

You can also have a look at the complete documentation for the functions discussed above and many more here.

And I invite you to join the Agora.io Developer Slack community.