Skip to content
Testing Agora vs Zoom for Multi-Party Web Video Calls: A Comparative Analysis of Video SDKs featured

Testing Agora vs Zoom for Multi-Party Web Video Calls: A Comparative Analysis of Video SDKs

By Author: Team Agora In Business, Developer

Part 4: Multi-Party Web Video Calls

Summary: In Part 1, Part 2, and Part 3 of this blog series, Agora reviewed results from a recent Zoom study that suggested its Video SDK outperformed Agora’s Video SDK on several fronts and then detailed the results of our own comprehensive testing. Agora topped Zoom’s Video SDK throughout our testing in numerous vital scenarios.

This testing series highlights important considerations when assessing Video SDKs, from analyzing performance under constrained network conditions to maximizing throughput given available bandwidth and keeping central processing unit (CPU) and random-access memory (RAM) usage low, regardless of device type.

Each blog in the series focused on testing in predefined real-world scenarios applicable to Agora and Zoom.

A quick look back: in our testing and analysis, Agora demonstrated the performance advantages of its Video SDK in 1:1 PC calling scenarios, multi-host video calls (Desktop PCs), and  during multi-party mobile video calls with up to 32 participants.

In each of those three scenarios, Agora’s Video SDK outperformed Zoom’s under varying real-world conditions.

This fourth and final installment of the series focuses on Agora and Zoom Video SDKs, examining how each performed during multi-party web video calls with 32 participants.

Read on to see the results!


Test Configuration and Scenarios

Before we take a deep dive into the results, let’s review the test setup, configuration and scenarios covered in the testing.

Table 1 – Test Setup

ProductAgora: https://videocall.agora.io/ based on Web SDK version 4.17.0
Zoom: https://zoom.us/wc
ScenarioVideo 32v32

Sender: MacBook Pro
Processor: 2.6 GHz 6-Core Intel Core i7
Memory: 32 GB 2400 MHz DDR4

Receiver: HP ProBook 630 G8 Notebook PC
Processor: Intel Core i7-1185G7 @ 3.00 GHz
Product Configuration Agora: 1280×720 @ 24fps
Zoom: 1280×720 @ 24fps
Network Configuration Sender: WIFI, ATTWrt5g
Receiver: WIFI, ATTWrt5g 

Table 2 – Test Case Scenarios

IDDescription
UnlimitedNormal network
Limited1MThe test starts with no bandwidth limitation, then a low bandwidth 1 Mbps is applied, first on the send side, then applied on the receiver side.
UL25%PLUplink network has 25% packet loss
DL25%PLDownlink network has 25% packet loss
Limited1M25%PLThe test starts with no bandwidth limitation, then a low bandwidth 1 Mbps with 25% packet loss is applied, first on the send side, then applied on the receiver side.
UL600msJitterUplink network has 600ms jitter
DL600msJitter Downlink network has 600ms jitter

For Agora, we tested using our Web SDK version: 4.17.0. For Zoom, we tested using https://pwa.zoom.us/wc/  [Version: 1.4.8.1250 (0322) to ensure Zoom’s best practices were properly implemented for comparison.

After nearly a decade of providing best-in-class video SDKs to thousands of customers and hundreds of use cases, the Agora team was not surprised to come out on top in every test case scenario against a relative newcomer to the Video SDK game. 

The results are in – an overview:

  • Under normal network conditions: The frame rate (FPS) for each was steady, 23 FPS for Agora and 21 FPS for Zoom. 
  • Packet loss and jitter: Agora consistently achieved a better frame rate than Zoom under numerous conditions, including when the network had uplink/downlink packet loss of 25%, as well as in cases where the uplink/downlink experienced jitter of 600ms.
  • Limited bandwidth: When throttling to 1Mbps on the send side and then on the receive side, Agora performed significantly better than Zoom, rapidly recovering to 23 FPS once the throttle was removed. It took Zoom 30 seconds to recover to 13 FPS.
  • 25% packet loss with 1Mbps network bandwidth limitation: Agora rapidly recovered to a frame rate near 23 FPS, while Zoom struggled to recover to a much lower frame rate near 15 FPS. Zoom had a frozen video for 18 seconds while the limitations were present.
  • CPU and RAM: Agora consistently consumed fewer CPU resources vs. Zoom and had lower RAM usage than Zoom.

The Agora Video SDK test results were robust even in poor network scenarios. A reliable Video SDK that can function while others struggle to deliver a high-quality user experience in adverse conditions can mean the difference between keeping customers or losing existing business.

Frame Rate Testing

In these tests, we again focused primarily on the effect of packet loss and limiting network conditions on the send and receive frame rates.

Normal Network Conditions

As you can see in the graph below, with the network functioning under normal conditions, the frame rate is steady at 23 FPS for Agora and 21 FPS for Zoom, for calls with 32 participants.

Figure 1: FPS comparison for Agora and Zoom under a normal network condition.
Figure 1: FPS comparison for Agora and Zoom under a normal network condition.

When introducing 25% uplink packet loss, Agora remains steady at 21 FPS while Zoom begins lagging at 17 FPS.

Figure 2: FPS comparison for Agora and Zoom with network having uplink packet loss of 25%.
Figure 2: FPS comparison for Agora and Zoom with network having uplink packet loss of 25%.

Much the same with 25% downlink packet loss, Agora is an even 20 FPS while Zoom shows 15 FPS.

Figure 3: FPS comparison for Agora and Zoom with network having downlink packet loss of 25%.
Figure 3: FPS comparison for Agora and Zoom with network having downlink packet loss of 25%.

A significant variation in latency can occur when sending traffic over an IP network. When simulating conditions of a highly congested network having uplink jitter of 600ms, Agora outperforms Zoom, holding steady at 18 FPS. In comparison, Zoom drops to 7 FPS, with a noticeable difference in perceived video quality by end users.

Figure 4: FPS comparison for Agora and Zoom with network having uplink 600ms jitter.
Figure 4: FPS comparison for Agora and Zoom with network having uplink 600ms jitter.

Nearly identical results occurred when conditions were simulated for downlink jitter of 600ms, with Agora holding at 12 FPS and Zoom bottoming out at 3 FPS. Under these conditions, participants in the Zoom call experienced choppy video.

Figure 5: FPS comparison for Agora and Zoom with network having downlink 600ms jitter.
Figure 5: FPS comparison for Agora and Zoom with network having downlink 600ms jitter.

Frame Rate Recovery, 1Mbps Limit

This test starts without any bandwidth limitations. After 60 seconds, the network is throttled to 1Mbps in the upstream direction and then in the downstream direction for a total of 30 seconds. Under these challenging conditions, Agora performs significantly better than Zoom, rapidly recovering to 23 FPS once the limitations are removed. It takes Zoom 30 seconds to recover to 13 FPS.

Figure 6: Frame rate recovery in limited1M test case.
Figure 6: Frame rate recovery in limited1M test case.

Frame Rate Recovery, 1Mbps Limit and 25% Packet Loss

This test starts without any bandwidth limitations or simulated packet loss. After 60 seconds, the network is throttled to 1Mbps in the upstream direction along with 25% simulated packet loss. These conditions are then applied in the downstream direction for a total of 30 seconds.

Once the bandwidth limitation and simulated packet loss conditions are removed, Agora rapidly recovers to a frame rate near 23 FPS, while Zoom struggles to recover to a much lower frame rate near 15 FPS. End users on the Zoom call experience frozen video for approximately 18 seconds while the limitations are present.

Figure 7: Frame rate recovery in limited1M25%PL test case.
Figure 7: Frame rate recovery in limited1M25%PL test case.

As we discussed in previous blogs, Agora has several ways to combat network disruptions and costly packet loss. Most notably, these include leveraging our SD-RTN™ network as an overlay to the public internet, implementing technologies that optimize performance over last-mile connections, and device optimization.

Our SD-RTN™ routes traffic around impairments on the internet using AI algorithms and optimally shapes real-time traffic for the best performance.

Agora also implements technologies to smooth out the effects of packet loss to optimize the end-user experience.

CPU Utilization

Table 3 – CPU Comparison

Processor: Intel Core i7-1185G7 @ 3.00 GHz

Test Case ID AgoraZoom
Unlimited8.25%10.32%
Limited1M8.23%9.45%
UL25%PL7.78%9.39%
DL25%PL7.69%9.31%
Limited1M25%PL8.16%9.35%
UL600msJitter6.18%7.87%
DL600msJitter6.09%7.72%

Agora consumed fewer CPU resources vs. Zoom in all scenarios, with an average of 7.48% utilization vs. Zoom at an average of 9.05%.

RAM Utilization

Table 4 – RAM Comparison 

Test Case ID AgoraZoom
Unlimited225 KB606 KB
Limited1M229 KB689 KB
UL25%PL239 KB617 KB
DL25%PL255 KB627 KB
Limited1M25%PL258 KB708 KB
UL600msJitter242 KB639 KB
DL600msJitter256 KB652 KB

Throughout testing, Agora consumed less RAM than Zoom in every scenario, with an average of 243 KB to Zoom’s 648 KB.

Conclusion

Each of these four comprehensive blogs comparing Agora and Zoom’s video SDKs has taken you inside varying scenarios and conditions experienced when implementing. When it comes to providing real-time communication tools, delivering a high-quality and consistent end-user experience is critical.

  • The first test compared Agora and Zoom for 1:1 calling scenarios between PCs. Detailed testing demonstrated the performance advantages of Agora’s SDK in 1:1 PC calling in every scenario.
  • In our second test, we stacked up the two video SDKs against one another in multi-host video calls on PCs with 8, 16, and 32 participants under varying real-world network conditions, Agora outperformed Zoom in all five scenarios.
  • In our third test, a side-by-side during Multi-Party Mobile Video Calls with up to 32 participants, Agora again beat Zoom in all five key categories.
  • And lastly here, with multi-party Web video calls, the results were much the same.

At Agora, our customers—ranging from some of the largest social media companies in the world to bootstrapped startups—continue to leverage our SDKs to easily integrate video, voice, and live interactive streaming experiences into any app. We empower developers to deliver best-in-class real-time in-app experiences, with global delivery and scale.

We can help you too. Talk to us for a free consultation.