Alibaba cloud announced today that it will make the real-time computing platform Blink open source.
Alibaba cloud (阿里云) announced today that it makes the real-time computing platform Blink (Flink) open source. Blink is a forked version of Apache Flink and has been tailored to meet some of the unique requirements that Alibaba has.
Stephan Ewen, CTO at Data Artisans, says “Alibaba is the main contributor to Flink. It is happy to see that Alibaba provides an open source of its internal-use and improved version of Flink to the community, which will let developers access the advanced computing power.”
Zhou Jingren (周靖人), vice president at Alibaba Group, says “Alibaba has been benefited from Flink for the past few years, and now it is time to give back to the community through the opening source.”
Flink is an open source stream processing technology, which could process both stream and batch data. Unlike Storm, Samza and Kafka Stream are for processing stream data. While, Spark, Hive and Pig are for processing batch data respectively.
According to its website, Flink features low latency, exactly-once state consistency, and high throughput. Low latency and exactly-once state consistency could be achieved through its asynchronous and incremental checkpointing algorithm. Low latency and high throughput could be achieved through the minimization of network communications among networks by only allowing network communication for stream processing happens along the logical edges in the job’s operator graph (vertically) instead of processing between the parallel instances of an operator (horizontally).
Earlier in January 2019, Alibaba acquired Data Artisans, the German big data startup leading the development of Flink for EUR 90 million (USD 103 million). Data Artisans has also serviced enterprise customers such as Netflix, ING, Uber, and Alibaba etc.
Back to 2016, Alibaba firstly started to apply Flink in its searching function. In 2017, it has used Flink in real-time computing services for Alibaba backed companies. Later, Alibaba developed its own extensions to deal with their requirements, known as Blink. Now all Alibaba’s core businesses are powered by Blink. Alibaba significantly increases its computing power and reduces latency into millisecond-level latency. In Single’s Day 2018, Blink handled 1.7 billion requests at the peak time and trillion-level requests in one day.