Amazon IVS Real-Time Streaming vs. Agora: Comparison of Real-Time Engagement Performance
How does Agora compare to Amazon’s recently launched IVS Real-Time Streaming? In this blog we’ll compare Agora vs. Amazon IVS Real-Time Streaming, looking at the key elements of real-time engagement (RTE) and a detailed comparison of performance test results.
Jump to a section:
Real-time engagement comparison
Successful apps focus on experiences which delight users and increase engagement and retention. Real-time engagement technology enables shared and immersive in-app experiences and is an essential component of any modern application with the goal of enabling live interaction between end-users.
Real-Time Engagement (RTE) is defined by 3 key elements:
- Shared context – an immersive in-app experience where people can interact with each other in a shared digital environment in real-time.
- Interactivity at scale – Whether the interaction is one-on-one, one-to-many people, or many-to-many (up to an audience of millions) the interaction must be natural and fluent.
- Ubiquity – Users require consistent experiences regardless of their location, network conditions, or their device of choice.
Shared context
To provide an immersive in-app experience where people can interact with each other in real-time in a shared digital environment, the different modes of interaction that users require must be considered. In addition to real-time audio and video, users often want to share ideas or media via chat, ideate together on an interactive whiteboard or interact with objects in a shared digital environment.
Amazon’s IVS Low-Latency Streaming is based on the technology which powers Twitch, originally developed and optimized for streaming to larger audiences with one-way delivery from host to viewers with 3-5 second latency. Amazon IVS Real-Time Streaming is a recent addition which is enabled through the same IVS Broadcast SDK. The solution also supports basic chat capabilities. Overall, while Amazon provides only a subset of the capabilities required to enable shared context in an app.
In contrast, Agora designed and built our network and SDKs to enable shared context by creating a global overlay to the internet leveraging proprietary technologies that enable the internet to work in real-time and deliver a natural and fluent experience—even under challenging network conditions. In addition to real-time audio and video, Agora provides feature-rich in-app Chat, Interactive Whiteboard. and real-time Signaling, which enables data synchronization like real-time location, online presence, and natural interaction with objects in shared virtual environments.
When end-users interact in a shared digital environment in real-time, it is also important for them to be able to express themselves and have fun within a safe environment free from unwanted content or other online harms. The Agora Extensions Marketplace makes it easy to add capabilities like face filters and content moderation to any app with just a few lines of code. Agora has taken a comprehensive approach to making it easy for customers to provide shared context within their app. Amazon IVS does not offer a similar solution, so the complexity in adding new features and capabilities is high.
Interactivity at scale
Amazon’s IVS Real-Time Streaming claims support for up to a maximum of 12 hosts on a stage and an audience of up to 10,000 viewers with latency that can be under 300 milliseconds from host to viewer.
Agora’s Software Defined Real-Time Network (SD-RTNTM) is a global network optimized for delivering real-time audio, video and streaming in extremely high scale, with hundreds of nodes and over 50B minutes of traffic per month.
There are many use cases which require a much larger number of hosts as well as the ability to interact with audiences larger than 10,000 viewers. Consider for example a live virtual concert or sporting event, where audiences can be in the 10s of thousands or even larger. Agora supports up to 128 hosts where audience members can subscribe to 50 simultaneous streams of their choice, interactivity with more than a thousand active participants, and broadcasting to an audience of 1M or more.
In the IVS Real-Time Streaming blog, the coding example shows that a stage must be configured specifying an Amazon AWS region (e.g., us-east-1). Once the stage is configured, all hosts on that stage must connect to that stage in that specified region, regardless of where those hosts are geographically located. This approach can lead to increased audio and video latency and impact interactivity when hosts are geographically distributed. In contrast, with Agora, hosts (and audience members) connect to the closest SD-RTNTM node where audio and video is processed in an optimal distributed approach to provide the best performance with the lowest latency. Agora’s approach makes it simple to scale while delivering an optimal experience.
Ubiquity
End-users today demand the ability to interact with anyone, anywhere, at any time – using their device of choice. The technology must be transparent and flexible enough to run on high-end to low-end mobile devices, PCs, and VR/AR glasses, regardless of operating system or available memory. Further, the technology needs to adapt automatically to provide the best possible experience – from near-perfect networks to the most challenging of last-mile environments with high packet loss and poor coverage.
Amazon IVS claims support for web, iOS, and Android and support for layered encoding (simulcast) to adapt to varying available bandwidth conditions.
Agora supports Android, iOS, Web, MacOS, Windows, React Native, Flutter, Unity, Unreal, Electron and simulcast (SVC)–providing developers with flexibility and choice. We designed our proprietary solution end-to-end with ubiquity in mind for end-users considering the device, the network (SD-RTNTM), and the challenges of the last mile.
Now, let’s look at the performance test results to see how Agora’s approach stacks up vs. Amazon IVS Real-Time Streaming.
Performance testing
This benchmark test of Agora and Amazon IVS Real-Time for Android focuses on performance evaluated based on two metrics: fluency and latency.
Fluency assesses the smoothness and uninterrupted playback of the live stream during the viewing experience. Fluency is used to determine how frequently moving objects freeze within the frame during the live stream. This assessment is achieved by measuring the ratio of frozen frames within specified thresholds, such as 200ms, 300ms, 500ms, and 600ms.
Latency is the measurement of the delay from when a camera captures a live stream at the near end to when the stream becomes visible on a viewer’s screen at the far end.
Test setup
The components in the test setup are listed in the table below.
Client B captures a moving globe and sends the packetized data to Agora’s SD-RTN/AWS cloud. Subsequently, Client A retrieves the packetized data from Agora’s SD-RTN/AWS cloud. The video displayed on Client A is recorded to facilitate fluency calculations. Notably, uplink network impairment is applied to Client B, while downlink network impairment is applied to Client A.
| Product | Agora Interactive Live Steaming – Android SDK v4.1.1.3 Amazon IVS Real-Time: https://github.com/aws-samples/amazon-ivs-real-time-for-android-demo/ |
| Scenario | One-to-one video call Sender – Galaxy S20 • Processor: Snapdragon 865/Exynos 990 chipsets. RAM: 12GB Receiver – OPPO RX17 Pro • Processor: Snapdragon 710. RAM: 6GB |
| Network Configuration | Sender – WIFI, ATTWrt5g Receiver – WIFI, ATTWrt5g |
| Network Impairment | Linux Traffic Control (tc) with Network Emulation (netem) used to impair a network on a one-to-one video calling for the benchmark test. |
Why do packet loss and jitter matter?
In any scenario where a user or host is streaming from mobile, uplink packet loss and jitter is common. Downlink packet loss and jitter can disrupt the live experience for any end user participating from mobile, a low bandwidth connection, or anywhere with network congestion. The disruption typically comes in the form of drops in frame rate (FPS) and frozen video.
For example, in live shopping scenarios the host is often streaming from mobile and commonly experiences anywhere from 10% to 20% packet loss. When that packet loss translates to drops in frame rate, it makes the video choppy and laggy for the audience, creating a subpar experience. This also matters for scenarios like telehealth, where patients may join a virtual consultation from a rural area with low bandwidth or urban area with high network congestion.
On the downlink side, jitter and packet loss can disrupt any stream or broadcast from live sports to live shopping where viewers or virtual attendees have limited bandwidth or network issues. Maintaining frame rate and avoiding freezes is essential for any viewing experience while low latency and interactivity are required for any real-time experience to be engaging for the user.
Test results
Testing fluency for Agora and Amazon IVS Real-Time involves executing 19 test cases under diverse network conditions including uplink packet loss, uplink jitter, downlink packet loss, and downlink jitter. Testing also included latency performance results. View the full table of test scenarios and results here: Amazon IVS Real-Time Streaming vs. Agora – Performance test results.
When testing for the described fluency scenarios, we recorded the video performance of Agora and Amazon IVS and compared them side by side in this video:
The side-by-side comparison shows Agora’s video to be smoother, with better fluency and lower latency. See the detailed test results for both fluency and latency below.
Fluency test results – how packet loss impacted frames per second (FPS)
Downlink packet loss 20%
Downlink packet loss 30%
Uplink packet loss 10%
Uplink packet loss 50%
Fluency test results – How jitter impacted frames per second (FPS)
Downlink jitter 300ms
Uplink jitter 300ms
Latency test results
Latency
Summary
Based on the testing, Agora experiences fewer frame freezes and maintains a higher frame rate compared to Amazon IVS Real-Time Streaming, all with lower latency.
Agora’s advantage is particularly evident when the uplink/downlink network exhibits packet loss ranging from 10% to 50%, or when there’s jitter in the range of 100ms to 600ms in the uplink/downlink network, or in cases where the uplink/downlink network is constrained to 1Mbps with a 10% packet drop rate. These are all very common situations for any users or streamers on mobile or in areas with limited internet speed and bandwidth. It’s also clear that Agora demonstrates lower latency than Amazon IVS Real-Time Streaming. Marinating low latency is essential in any scenario involving real-time communication or in live scenarios like auctions, multiplayer gaming, or sports betting.
When combining the comparison of real-time engagement functionality and fluency/latency test results, Agora comes out ahead of Amazon IVS as a better choice for interactive and low-latency streaming experiences. Try Agora for free here: Get started with Agora.
