.circleci | ||
.github/workflows | ||
benchmark | ||
bin | ||
core | ||
docs | ||
examples | ||
src | ||
targets | ||
test | ||
.ghci | ||
.gitignore | ||
.hlint.ignore | ||
.hlint.yaml | ||
.packcheck.ignore | ||
appveyor.yml | ||
cabal.project | ||
cabal.project.coverage | ||
cabal.project.doctest | ||
cabal.project.ghc-head | ||
cabal.project.hpc-coveralls | ||
cabal.project.O0 | ||
cabal.project.packdiff | ||
cabal.project.report | ||
cabal.project.streamly | ||
cabal.project.Werror | ||
cabal.project.Werror-nocode | ||
CHANGELOG.md | ||
configure | ||
configure.ac | ||
CONTRIBUTING.md | ||
default.nix | ||
hie.yaml | ||
LICENSE | ||
README.md | ||
Setup.hs | ||
stack.yaml | ||
streamly.cabal |
Streamly: Idiomatic Haskell with the Performance of C
Streamly is a Haskell library that provides building blocks to build safe, scalable, modular and high performance software. Streamly offers:
- The type safety of Haskell.
- The performance of C programs.
- Powerful building blocks for modular code.
- Idiomatic functional programming.
- Declarative, fearless concurrency.
- Ecossytem libraries for quick development.
Please read the Streamly Setup and Usage Guide and Streamly Quick Overview to get a taste of the library. Streamly comes with comprehensive documentation, please visit the Haskell Streamly website for documentation.
Performance with Modularity
Usually, you have to pick one of the two, performance or
modularity. Using Haskell Streamly you can write highly
modular code and still achieve performance close to an equivalent
(imperative) C program. Streamly exploits GHC's stream fusion
optimizations (case-of-case
and spec-constr
) aggressively to achieve
both modularity and performance at the same time.
Streamly offers excellent performance even for byte-at-a-time stream
operations using efficient abstractions like Unfold
s and
Fold
s. Byte-at-a-time stream operations can simplify programming
because the developer does not have to deal explicitly with chunking
and re-combining data.
GHC Plugin for Stream Fusion
Streamly usually performs very well without any compiler plugins. However, we have fixed some deficiencies in GHC's optimizer using a compiler plugin. We hope to fold these optimizations into GHC in the future; until then we recommend that you use this plugin for applications that are performance sensitive.
Performance Benchmarks
We measured several Haskell streaming implementations using various micro-benchmarks. Please see the streaming benchmarks page for a detailed comparison of Streamly against other streaming libraries.
Our results show that Streamly is the fastest effectful streaming implementation on almost all the measured microbenchmarks. In many cases it runs up to 100x faster, and in some cases even 1000x faster than some of the tested alternatives. In some composite operation benchmarks Streamly turns out to be significantly faster than Haskell's list implementation.
Note: If you can write a program in some other way or with some other language that runs significantly faster than what Streamly offers, please let us know and we will improve.
Applications
Streamly comes equipped with a very powerful set of abstractions to
accomplish many kinds of programming tasks: it provides support for
programming with streams and arrays, for reading and writing from the
file system and from the network, for time domain programming (reactive
programming), and for reacting to file system events using fsnotify
.
Please view Streamly's documentation for more information about Streamly's features.
Design
Design Goals
Our goals for Streamly from the very beginning have been:
- To achieve simplicity by unifying abstractions.
- To offer high performance.
These goals are hard to achieve simultaneously because they are usually inversely related. We have spent many years trying to get the abstractions right without compromising performance.
Unfold
is an example of an abstraction that we have created to achieve
high performance when mapping streams on streams. Unfold
allows stream
generation to be optimized well by the compiler through stream fusion.
A Fold
with termination capability is another example which modularizes
stream elimination operations through stream fusion. Terminating folds
can perform many simple parsing tasks that do not require backtracking.
In Streamly, Parser
s are a natural extension to terminating Fold
s;
Parser
s add the ability to backtrack to Fold
s. Unification leads
to simpler abstractions and lower cognitive overheads while also not
compromising performance.
Concurrency Design
Streamly uses lock-free synchronization for achieving concurrent operation with low overheads. The number of tasks performed concurrently are determined automatically based on the rate at which a consumer is consuming the results. In other words, you do not need to manage thread pools or decide how many threads to use for a particular task. For CPU-bound tasks Streamly will try to keep the number of threads close to the number of CPUs available; for IO-bound tasks it will utilize more threads.
The parallelism available during program execution can be utilized with
very little overhead even where the task size is very small, because
Streamly will automatically switch between serial or batched execution
of tasks on the same CPU depending on whichever is more efficient.
Please see our concurrency benchmarks for more
detailed performance measurements, and for a comparison with the async
package.
Credits
The following authors/libraries have influenced or inspired this library in a significant way:
Please see the credits
directory for a full
list of contributors, credits and licenses.
Licensing
Streamly is an open source project available under a liberal BSD-3-Clause license
Contributing to Streamly
As an open project we welcome contributions:
Getting Support
Professional support is available for Streamly: please contact support@composewell.com.
You can also join our community chat channel on Gitter.