Agora vs Zoom: Look at the Big Picture When Evaluating Real-time Engagement Solutions
Multi-Host Video Calls (Desktop PCs) – Video SDK Comparison
This blog is part of a series comparing Zoom and Agora for different real-time video scenarios. Agora’s detailed testing demonstrated the performance advantages of its SDK in 1:1 PC calling scenarios. For a summary of all testing scenarios, check out this blog: Agora vs. Zoom: A Comprehensive Comparison of Video SDKs
As we continue dissecting the benchmark tests commissioned by Zoom, it’s important to remember that while adding real-time video to an app increases engagement and enhances business opportunities, not all SDKs are created equally.
When shopping for a video SDK, it is crucial to zoom out and look at the big picture, and that’s why we decided to look further into Zoom’s results and further expand on what is relevant for customers to consider when evaluating real-time engagement (RTE) solutions.
In this blog, we compare how the two video SDKs stack up against one another during multi-host video calls on PCs with 8, 16, and 32 participants under varying real-world network conditions.
To see the detailed results for each test case, check out blogs covering the other scenarios here:
Test Configuration and Scenarios
| Agora | Zoom | |
|---|---|---|
| Network Connectivity | Wi-Fi | Wi-Fi |
| Participants in Sessions | 8, 16, 32 | 8, 16, 32 |
| Versions Tested | Native SDK 4.1.0 | Zoom Application version 5.12.3 |
| Receive Test PC Specification | HP ProBook 630 G8 Notebook PC with an Intel Core i7-1185G7 @ 3.00 GHz processor | HP ProBook 630 G8 Notebook PC with an Intel Core i7-1185G7 @ 3.00 GHz processor |
| Video Configuration for Receive Test PC | Receive 1280×720 @ 30 FPS | Receive 1280×720 @ 30 FPS |
| Participant Devices | Mac or Windows PCs | Mac or Windows PCs |
| Scenario | Description |
|---|---|
| Normal Network Conditions | No added packet loss or bandwidth limitations |
| Uplink 25% Packet Loss | 25% packet loss is applied in the uplink (sending) direction from one test PC |
| Downlink 25% Packet Loss | 25% packet loss is applied in the downlink (receiving) direction to the Receive Test PC |
| Limited 1 Mbps Throughput | The test started with no bandwidth limitation, then the network was throttled to 1Mbps in the upstream direction and then in the downstream direction for a total of 30 seconds. |
| Limited 1Mbps Throughput and 25% Packet Loss | The test started with no bandwidth limitation, then the network was throttled to 1Mbps in the upstream direction with 25% packet loss applied. Next the network was throttled to 1Mbps in the downstream direction with 25% packet loss applied. The total duration of these impairments was 30 seconds. |
For Agora, we tested using our Native SDK version 4.1.0, and for Zoom, we tested the recent Zoom Application version 5.12.3, to ensure Zoom’s best practices were properly implemented for comparison.
All results were observed and measured from the Receive Test PC.
For the TLWR (Too Long, Won’t Read) Crowd
A snapshot of the results:
- Under normal network conditions, both Agora and Zoom performed well with fluent video and audio and with an average received frame rate between 27 and 28 Frames Per Second (FPS) in calls (8, 16, 32).
- With 25% uplink packet loss Agora holds a slight 2 to 3 FPS advantage against Zoom on the received frame rate in calls (8, 16 and 32).
- With 25% downlink packet loss Agora performs better than Zoom with a 4 to 6 FPS advantage against Zoom on the received frame rate in calls (8, 16, 32).
- When the 1Mbps limit was applied, the Receive Test PC measured a frame rate drop both for Zoom and Agora. Zoom dropped to 0 FPS for a few seconds and frozen video was experienced during that time. With Agora, no choppy or frozen video was observed on the Receive Test PC. After the 1Mbps limit was removed, Agora consistently recovered to achieve an average frame rate close to 25 FPS, while Zoom struggled to recover with significant fluctuations in FPS.
- When the impairments (1 Mbps limit and 25% packet loss) were applied, the Receive Test PC measured a frame rate drop both for Zoom and Agora. Zoom dropped to 0 FPS for over 15 seconds total and frozen video was experienced during that time. After the impairments were removed, Agora recovered to achieve a higher average received frame rate (FPS) than Zoom.
- Bitrate is vital when communicating with others via a video call. To deliver the best possible experience and quality for end users, maintaining the highest possible video throughput for the given network conditions is essential. Agora achieved a similar or higher sent and received bitrate vs. Zoom.
Frame Rate Test
In this test, we focused primarily on the effect of packet loss and limiting network conditions on the received frame rate.
Agora minimizes packet loss and other poor network conditions by leveraging our SD-RTN™ network as an overlay to the public internet, implementing technologies that optimize performance over last-mile connections, and device optimizations. Our SD-RTN™ routes traffic around impairments on the internet using AI algorithms and optimally shapes real-time traffic for the best performance. Agora also implements technologies to smooth out the effects of packet loss to optimize the end-user experience.
Normal Network
Under normal network conditions, we see the average received frame rate was between 27 and 28 Frames Per Second (FPS) for both Zoom and Agora in calls with 8, 16 and 32 participants.
25% Uplink Packet Loss
With 25% uplink packet loss Agora maintained a slight 2 to 3 FPS advantage against Zoom on the received frame rate in calls with 8, 16 and 32 participants.
25% Downlink Packet Loss
With 25% downlink packet loss Agora performed better than Zoom with a 6 to 7 FPS advantage against Zoom on the received frame rate in calls with 8, 16 and 32 participants
Frame Rate Recovery with 1Mbps Limit
The test started with no bandwidth limitation, then the network was throttled to 1Mbps in the upstream direction and then in the downstream direction for a total of 30 seconds. This simulated a real-world scenario commonly seen on wireless networks.
8 Video Participants
When the 1Mbps limit was applied, the Receive Test PC measured a frame rate drop both for Zoom and Agora. Zoom dropped to 0 FPS for a few seconds and frozen video was experienced during that time. After the 1Mbps limit was removed, Agora rapidly recovered to achieve an average frame rate close to 25 FPS, while Zoom struggled to recover with significant fluctuations in FPS.
16 Video Participants
With 16 video participants, the results were very similar to what was observed in the case with 8 video participants above.
32 Video Participants
With 32 video participants, the results were very similar to what was observed in the cases with 8 and 16 video participants above. In this case, Agora took about 20 seconds to fully recover, but did so in a smooth and linear fashion. Again, Zoom struggled to recover with significant fluctuations in FPS.
Frame Rate Recovery, 1Mbps Limit and 25% Packet Loss
The test started with no bandwidth limitation, then the network was throttled to 1Mbps in the upstream direction with 25% packet loss applied. Next the network was throttled to 1Mbps in the downstream direction with 25% packet loss applied. The total duration of these impairments was 30 seconds. This simulated a more challenging real-world scenario commonly seen on wireless networks.
8 Video Participants
When the impairments were applied, the Receive Test PC measured a frame rate drop both for Zoom and Agora. Zoom dropped to 0 FPS for over 15 seconds total and frozen video was experienced during that time. After the impairments were removed, Agora rapidly recovered to achieve an average frame rate close to 25 FPS, while Zoom struggled to recover with frozen video for more than 5 seconds, and then had significant fluctuations in FPS.
16 Video Participants
With 16 video participants, the results were very similar to what was observed in the case with 8 video participants above. In this case, after the impairments were removed, the received frame rate on Agora fluctuated more than in the 8-video participant case, however the average FPS remained higher than Zoom.
32 Video Participants
With 32 video participants, the results were similar to what was observed in the cases with 8 and 16 video participants above. In this case, Agora took about 20 seconds to fully recover, but then maintained a stable average FPS of around 27. Zoom initially recovered more quickly, however, a significant fluctuation in FPS was observed from 150-180 seconds.
Bitrate Test
Bitrate is also vital when communicating with others via a video call. To deliver the best possible experience and quality for end users, maintaining the highest possible video throughput for the given network conditions is essential.
The data below focused on the case with 32 video participants for simplicity. The observations for the 8 and 16 video participant cases were similar.
Normal Network
Under normal network conditions, Agora sent and received an average of 3000 Kbps vs. Zoom which sent at 2500 Kbps and received at 2300 Kbps.
25% Uplink Packet Loss
With an uplink packet loss of 25%, Agora’s sent and received bitrate remained higher than Zoom.
25% Downlink Packet Loss
With 25% downlink packet loss, Agora and Zoom had similar sent and received bitrate performance.
CPU Utilization
During the test scenarios, we monitored the CPU utilization on the Receive Test PC (HP ProBook 630 G8 Notebook PC with an Intel Core i7-1185G7 @ 3.00 GHz processor).
For simplicity, we summarized the results for the 32-participant video scenario, which is the most resource intensive.
CPU Utilization
| Scenario | Agora | Zoom |
|---|---|---|
| Unlimited bandwidth | 1.2% | 1.3% |
| Limited 1 Mbps Throughput | 1.1% | 1.4% |
| Uplink 25% Packet Loss | 1.3% | 1.5% |
| Downlink 25% Packet Loss | 1.3% | 1.5% |
| Limited 1Mbps Throughput and 25% Packet Loss | 1.0% | 1.4% |
As you can see, Agora consumed fewer CPU resources vs. Zoom in all scenarios, with an average of 1.2% utilization vs. Zoom at an average of 1.4%.
RAM Utilization
During the test scenarios, we monitored the RAM utilization on the Receive Test PC (HP ProBook 630 G8 Notebook PC with an Intel Core i7-1185G7 @ 3.00 GHz processor).
For simplicity, we summarized the results for the 32-participant video scenario, which is the most resource intensive.
RAM Utilization (MB)
| Network Condition | Agora | Zoom |
|---|---|---|
| Unlimited bandwidth | 704 | 436 |
| Limited 1 Mbps Throughput | 708 | 446 |
| Uplink 25% Packet Loss | 705 | 445 |
| Downlink 25% Packet Loss | 712 | 465 |
| Limited 1Mbps Throughput and 25% Packet Loss | 706 | 452 |
Here Agora consumed more RAM than Zoom, with an average of 707MB vs. Zoom at an average of 449MB. For the PC use case, we would not expect a user experience impact for either Zoom or Agora, even with multiple applications running. A low-end PC typically comes standard with 8-16GB of RAM.
Conclusion
In this blog, we compared how Agora and Zoom stack up against one another’s video calls on PCs with 8, 16, and 32 participants under varying real-world network conditions.
The Agora SDK demonstrated considerable advantages under all network scenarios and was able to adapt efficiently to changing network conditions. Much like in Part 1 of this blog series, which featured Agora and Zoom testing for 1:1 calling scenarios between PCs, the Agora Video SDK performed more reliably and efficiently overall, especially under more challenging conditions where Zoom froze or had choppy video – Agora delivered fluent, natural audio and video.
Whether it’s online classrooms, virtual events, enterprise collaboration spaces, virtual meetings or use cases in the metaverse, stability and reliability can mean the difference between success and failure.
Next Up
For a summary of all testing scenarios, check out this blog:
To see the detailed results for each test case, check out blogs covering the other scenarios here:
Ready to test Agora’s performance for yourself? Sign up for free