Apache Pulsar Outperforms Apache Kafka by 2.5x on OpenMessaging Benchmark

Pulsar sets the performance pace, delivering 150% better throughput with up to 40% lower latency

SAN JOSE, Calif.--()--Streamlio, the intelligent platform for fast data, today announced leading benchmark performance results in tests performed by industry analyst firm Gigaom using the newly-announced OpenMessaging performance benchmark. This benchmark, designed to evaluate performance of messaging and queuing solutions in real-world scenarios, showed Apache Pulsar delivering consistently superior throughput and latency even at increasing scale.

Messaging solutions for streaming data are a critical component of modern software infrastructure, providing the glue that connects diverse data with users and applications. The increasing demands on messaging solutions to support data-driven organizations has made performance and scalability of these solutions ever more important. Without that performance and scalability, users face headaches and complex patchworks of attempts to address those limitations.

OpenMessaging is a collaborative Linux Foundation effort supported by numerous companies including Alibaba, DiDi, Streamlio and Yahoo! who are invested in cloud, big data, and standard APIs. Its mission is to create a globally adopted, vendor-neutral, and open standard for distributed messaging that can be deployed in cloud, on-premises, and hybrid use cases.

The OpenMessaging benchmark is a multi-platform, open benchmark created within the Linux Foundation to provide users and architects with the tools they need to evaluate and compare performance of messaging solutions. The benchmark, provided as a kit that can be run, modified, and extended by users, consists of a set of tests based on real-world usage scenarios, including scenarios requiring guaranteed data durability. It tests producer and consumer latency, maximum end-to-end throughput, and performance scalability.

Industry analyst firm Gigaom performed an evaluation of Apache Pulsar and Apache Kafka using the OpenMessaging benchmark. Key results from their testing include:

  • Up to 150% higher maximum throughput with Apache Pulsar
  • Up to 40% lower message latency and greater consistency in latency
  • Better scalability that delivered consistent results across a range of message sizes and partition counts

“Gigaom believes performance is a key selection criterion for those considering a streaming or message queuing solution. However, if a platform cannot perform at the scale your enterprise needs, it really does not matter if it meets all your other checkbox criteria,” said William McKnight of Gigaom. “Apache Pulsar outpaced Kafka across all the workloads tested in our evaluation using the OpenMessaging benchmark, making a strong case for the platform among enterprises needing performance and scalability today and in the near future.”

The results of this benchmark further validate the ability of Pulsar to deliver the performance needed for today’s data applications. Proven to scale to billions of messages per day on millions of topics across multiple datacenters at Yahoo, Apache Pulsar ensures that users at any scale can meet demanding SLAs at a fraction of the cost and without the complexity of alternatives.

“Apache Pulsar was designed from the start to deliver leading performance and scalability,” said Matteo Merli, co-founder of Streamlio. “This benchmark result demonstrates the impact of those design decisions, providing performance and scalability that organizations can rely on to handle their ever-growing needs for fast action on streaming data.”

Gigaom and Streamlio will be hosting a webinar on March 22, 2018 at 10am PST to discuss best practices for benchmarking messaging solutions and to dive into Gigaom’s benchmark analysis of Apache Pulsar. Register at https://gigaom.com/event/integrating-the-enterprise-with-a-streaming-data-approach/.


Streamlio: streaml.io

Twitter: twitter.com/streamlio

LinkedIn: linkedin.com/company/streamlio

OpenMessaging: openmessaging.cloud

OpenMessaging Benchmark: github.com/openmessaging/openmessaging-benchmark

About Streamlio

Streamlio delivers the first intelligent platform for fast data. Streamlio’s solution is built on leading open source technologies for messaging, processing, and storage of streaming data that have been proven at scale in companies including Twitter and Yahoo. Founded by industry veterans and funded by Lightspeed Venture Partners, Streamlio helps organizations build the data-driven applications they need to react to data in real-time. Learn more online at streaml.io.


Jon Bock

Release Summary

New performance benchmark test shows Apache Pulsar delivering 2.5x better performance on OpenMessaging Benchmark


Jon Bock