Airtel XStream Play uses QuestDB for real-time engagement and device insights

Airtel XStream Play is an "Over the Top" (OTT) media provider, offering high quality media streaming services. Millions of subscribers connect to XStream Play every month to watch their favourite movies, TV shows and sports including cricket and football. As a subsidiary of Bharti Airtel, they are part of a leading global telecommunications company with operations in 18 countries across Asia and Africa.

XStream Play is the fastest growing aggregator platform in India, with over 100 million users. Their library features a blend of Indian and Western content from partners such as Sony LIV, Lionsgate Play, Hoichoi, ManoramaMax, Ultra and more.

Unpacking the XStream Play analytics opportunity

The Airtel XStream Play engineering team wanted to expand their user engagement analytics. As an OTT platform offering rich video media, everything happens by time. What time something was watched, how long it was watched, which time segments, and from which device, are questions that matter. These data points, pulled from a regionally distributed content delivery network, inform capacity planning and spend, and also creative decisions like which content engages and where it drops off.

Why Airtel XStream Play chose QuestDB for monitoring user engagement

To try and capture their rapidly growing bursts of time series data, the team began with a different analytical database. For lower traffic it worked. Once the service exceeded 100 million users, the database failed to properly aggregate data and latency increased as it struggled with ingestion throughput.

XStream Play is growing fast. Current ingestion peak is over 1 billion records per day, roughly 16,000 per second. For the analytics engine to meet current and future needs, the database had to handle significantly higher throughput. That brought them to QuestDB.

Ajay Pilaniya, Senior Engineer at XStream Play responsible for the implementation, explains the choice: "We initially ran benchmarks on TimescaleDB as a replacement, but there were major latency issues. During that time we found a benchmark comparison between TimescaleDB and QuestDB via a Google search and from there we started exploring QuestDB. QuestDB was able to ingest our records much faster than any other database."

Raw performance comparisons are vital when choosing a time series database for high volume workloads. During implementation, responsive support is also essential.

How Airtel XStream Play uses QuestDB

Airtel XStream Play takes advantage of both the high throughput ingest and the high performance querying of QuestDB. Searching a very large corpus requires only a small set of precise queries.

Devices can be smartphones, smart TVs, or personal computers. Understanding device capabilities helps the team infer bandwidth and screen resolution so that content can be encoded and delivered efficiently.

SELECT
  UPPER(userdevice) AS UserDevice,
  UPPER(userdevicebrand) AS UserDeviceBrand,
  UPPER(useros) AS UserOS,
  SUM(totalrequestids) AS RequestId,
  SUM(totaluserids) AS UserId,
  SUM(segmentcount) AS SegmentCount,
  SUM(totalfilesize) AS TotalFileSize
FROM
  'device_analytics_';

Note that the example contains no WHERE clause. It scans the entire table and returns all rows. This demonstrates QuestDB performance on large workloads. The end result is a fast, flexible and functional internal dashboard.

Content analytics provides another strong example. The following SQL creates a table for content analytics data with content properties, request counts and unique user counts.

CREATE TABLE IF NOT EXISTS content_analytics_ (
  ts TIMESTAMP,
  ua TIMESTAMP,
  ca TIMESTAMP,
  ContentId SYMBOL CAPACITY 50000 CACHE,
  StreamingPartnerId SYMBOL CAPACITY 500 CACHE,
  TotalRequestIds long,
  TotalUserIds long,
  SegmentCount long,
  TotalFileSize long
) timestamp(ts) PARTITION BY DAY;

The query provides fast delivery of the rich data needed to visualize and better predict network volume. Consider the value of the following chart when planning capacity over specific windows like cultural or seasonal events.

XStream Play is an at scale media streaming service generating sustained and high throughput event data. To remain competitive, it is essential to act on insights during growth. With QuestDB, the team gets what they need without worrying that the storage engine will break as they grow. It is rapid ingest and simple, fast queries.

Ajay and the team have been running QuestDB in production for over a year. Initially QuestDB met most requirements, but one specific need was missing. "Most of the requirements were fulfilled by existing features and one missing requirement which I raised is also live now," says Ajay.

The missing piece was data deduplication, which provides exactly once semantics. With deduplication, users have the guarantee data will never be duplicated, even if the same data is sent multiple times, which often happens in distributed systems.

See also

• QuestDB vs TimescaleDB benchmark
• QuestDB vs InfluxDB benchmark