Skip to content featured

Extensions Marketplace: How to Add Conversation Intelligence to Your Android Application Using Agora and

By Author: Rishav Naskar In Developer

Building a video call app with Agora is child’s play, but what about adding cool features such as AR filters, voice changers, speech intelligence, and monetization? That’s where Agora’s Extensions Marketplace comes in.

Launched just last month, the Extensions Marketplace already has several amazing extensions available, with more to come.

In this blog post, we’ll show you how to easily add conversation intelligence (speech-to-text) to your Agora voice calling or video live streaming application using the Conversation Intelligence extension.



Step 1: We need to enable the extension from our Agora console. To do that, go to your Agora Console and navigate to the Extensions Marketplace section.

how to add conversation intelligence to your android application using agora and symbl 1

Step 2: Select extension from the list and activate the same on the console. Copy or store the credentials (Symbl App ID and Symbl App Secret) of your project somewhere safe and private.

how to add conversation intelligence to your android application using agora and symbl 2

Step 3: Head over to Android Studio to add the extension to our application. For this, clone the Agora UIKit GitHub repo and switch to the extension-update branch. Now clone this repository locally and open the project in File Explorer.

You will find the agorauikit_android directory inside this cloned UIKit project. Copy and paste this directory into your application, at the parent level. This directory essentially contains the Agora Android UIKit and is crucial to making our extension work.

Step 4: Let’s import the SDK files, the UIKit project, and a few other dependencies in our app-level level build.gradle:

  implementation 'ai.symbl:android.extension:1.0.0'implementation ''
  implementation 'ai.symbl:android.extension:1.0.0'
  implementation 'com.squareup.okhttp3:okhttp:3.10.0'
  implementation ''

Step 5: Head over to the project-level build.gradle.kts of the UIKit and do the following:

  1. Remove the plug-in maven if present, as it is deprecated.
  2. Remove the version code and the version name if present.

And with that, we are done setting up our project!

Video Call UI

For our application, we won’t have to work on building the UI too much, because all that is already taken care of by the Agora Android UIKit.

Basic Video Call using Agora Android UIKit

Integrating Conversation Intelligence SDK

Here, we have used CallActivity.kt as the main activity used for launching our video call app.

To begin with, let’s first import the following files into CallActivity.kt

import io.agora.agorauikit_android.AgoraConnectionData
import io.agora.agorauikit_android.AgoraVideoViewer
import io.agora.agorauikit_android.requestPermissions
import io.agora.rtc2.Constants
import io.agora.rtc2.IMediaExtensionObserver
import io.agora.rtc2.IRtcEngineEventHandler
import io.agora.rtc2.UserInfo

Adding the extension

We will update the AgoraVideoViewer class with the list of extension names that you want. In this case, we will be using the Conversation Intelligence extension.

Sending the extension name via AgoraVideoViewer class

Preparing the Extension

Here we will add the necessary objects and variables required to run the APIs required by the extension.

First, we need to implement the IMediaExtensionObserver interface along with our CallActivity.kt. To do this, do the following:

class CallActivity : AppCompatActivity(), IMediaExtensionObserver {

Now, add the following private variables in CallActivity.kt

  • meetingId refers to the channel name used in the Agora Video Call
  • symblUserId refers to the local user ID used by to identify the same
  • TAG refers to the tag used while logging data/errors
  • infoTextView refers to the TextView we created to log the result of the extension which will basically show the successful conversion of speech to text
private val meetingId = "test"
private val symblUserId = ""
private val TAG = "Agora_SymblTag java :"
private lateinit var infoTextView: TextView

Creating the response data class

Now that we have taken care of these, we need to understand how the extension works. The extension listens to speech using the smartphone’s microphone, converts it into text and returns the response to us in JSON format.

To log the response properly, let’s quickly define a data class to decode the JSON response which we will receive from the extension.

Response data class

Initializing the extension

To initialize the extension, we have to add two methods — enableExtension and setExtensionProperty.

Here we will create a few methods to properly initialize the extension:

  • initAgoraEngine is where we we change video configuration, enable the extension, set user roles, join a meeting and other RTCEngine related changes.
initAgoraEngine method
  • setSymblPluginConfigs is where we configure the extension and it’s APIs to work properly. It is here that we require our previously stored Symbl App ID and Symbl App Secret.
setSymblPluginConfigs method
  • enableEffect or disableEffect is where we enable or disable the extension using the setExtensionProperty method of the RTCEngine.
enableEffect/disableEffect methods
  • onEvent is a method of the IMediaExtensionObserver super class which we will call in CallActivity.kt. This method will be fired whenever there is a successful conversion of speech to text using the extension. So this method is extremely crucial for our extension to display/log the results properly.
onEvent method

Modifying onCreate

We need to one last change in the onCreate method of the CallActivity to start our extension during video calls:

  • Add some bottom margin to the Frame Layout on which the Video Call will run and add it to the current ContentView.
val frameLayout: FrameLayout.LayoutParams = FrameLayout.LayoutParams(
).also { it.setMargins(0, 0, 0, 250) }

this.addContentView(agView, frameLayout)

Layout for the Video Call

I have shown the layout I used for this video call along with displaying the results of the extension here. However, you may choose to use a layout of your own and display the results of the conversion as you like.


You now have a video call app complete with the new extension from! You should be able to run the extension successfully in your Android application with the help of the Agora Android UIKit.

There are other extensions I’d encourage you to try out, and they can all be found here.

The same settings can be applied to an application made directly with the Agora SDK, but you will need to make sure you’re using the 4.x version of the SDK. Details can be found here.


You can try out this app by following the GitHub link. After cloning the repo, just run the app on a physical Android device to test the application.

Other Resources

To learn more about the Agora Android SDK and other use cases, see the developer guide here.

You can also have a look at the complete documentation for the functions discussed above and many more here.

And I invite you to join the Agora Developer Slack community.