---
title: Flink
---
## Flink
Apache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities.
The core of Apache Flink is a distributed streaming dataflow engine written in Java and Scala. Flink executes arbitrary dataflow programs in a data-parallel and pipelined manner.
Flink's pipelined runtime system enables the execution of bulk/batch and stream processing programs. Furthermore, Flink's runtime supports the execution of iterative algorithms natively.
Flink provides a high-throughput, low-latency streaming engine as well as support for event-time processing and state management. Flink applications are fault-tolerant in the event of machine failure and support exactly-once semantics.
Programs can be written in Java, Scala, Python, and SQL and are automatically compiled and optimized into dataflow programs that are executed in a cluster or cloud environment.
Flink does not provide its own data storage system and provides data source and sink connectors to systems such as Amazon Kinesis, Apache Kafka, HDFS, Apache Cassandra, and ElasticSearch.
**What Is New in Apache Flink?**
* Flink implements actual streaming processing and not imitates it with micro-batch processing. In Spark streaming is a special case of batching, while in Flink batching is a special case of streaming (stream of a finite size)
* Flink has better support for cyclical and iterative processing
* Flink has lower latency and higher throughput
* Flink has more powerful windows operators
* Flink implements lightweight distributed snapshots that has low overhead and only-once processing guarantees in stream processing, without using micro batching as Spark does
* Flink supports mutable state in stream processing
### Features
* A streaming-first runtime that supports both batch processing and data streaming programs
* Elegant and fluent APIs in Java and Scala
* A runtime that supports very high throughput and low event latency at the same time
* Support for *event time* and *out-of-order* processing in the DataStream API, based on the *Dataflow Model*
* Flexible windowing (time, count, sessions, custom triggers) accross different time semantics (event time, processing time)
* Fault-tolerance with *exactly-once* processing guarantees
* Natural back-pressure in streaming programs
* Libraries for Graph processing (batch), Machine Learning (batch), and Complex Event Processing (streaming)
* Built-in support for iterative programs (BSP) in the DataSet (batch) API
* Custom memory management for efficient and robust switching between in-memory and out-of-core data processing algorithms
* Compatibility layers for Apache Hadoop MapReduce and Apache Storm
* Integration with YARN, HDFS, HBase, and other components of the Apache Hadoop ecosystem
### Flink Usage
Prerequisites for building Flink:
* Unix-like environment (We use Linux, Mac OS X, Cygwin)
* git
* Maven (we recommend version 3.0.4)
* Java 7 or 8
```
git clone https://github.com/apache/flink.git
cd flink
mvn clean package -DskipTests # this will take up to 10 minutes
```
## Developing Flink
The Flink committers use IntelliJ IDEA to develop the Flink codebase.
We recommend IntelliJ IDEA for developing projects that involve Scala code.
Minimal requirements for an IDE are:
* Support for Java and Scala (also mixed projects)
* Support for Maven with Java and Scala
#### More Information:
* Flink website: Apache Flink
* Flink documentation: flinkdocs
* Quick flink tutorial: quick start
* How to guide: howto
* Flink vs Spark: comparison