1# [Streamly][]: Idiomatic Haskell with the Performance of C
3[](https://gitter.im/composewell/streamly)
4[](https://hackage.haskell.org/package/streamly)
6Streamly is a Haskell library that provides building blocks to build
7safe, scalable, modular and high performance software. Streamly offers:
9* The type safety of Haskell.
10* The performance of C programs.
11* Powerful building blocks for modular code.
12* Idiomatic functional programming.
13* Declarative, fearless concurrency.
14* Ecossytem libraries for quick development.
16Please read the [Streamly Setup and Usage
17Guide](/docs/User/Tutorials/setup-and-usage.md) and [Streamly Quick
18Overview](/docs/User/Tutorials/quick-overview.md) to get a taste of the
19library. Streamly comes with comprehensive documentation, please visit
20the [Haskell Streamly website][Streamly] for documentation.
22## Performance with Modularity
24Usually, you have to pick one of the two, performance or
25modularity. Using [Haskell Streamly][Streamly] you can write highly
26modular code and still achieve performance close to an equivalent
27(imperative) C program. Streamly exploits GHC's stream fusion
28optimizations (`case-of-case` and `spec-constr`) aggressively to achieve
29both modularity and performance at the same time.
31Streamly offers excellent performance even for byte-at-a-time stream
32operations using efficient abstractions like `Unfold`s and
33`Fold`s. Byte-at-a-time stream operations can simplify programming
34because the developer does not have to deal explicitly with chunking
37### GHC Plugin for Stream Fusion
39[Streamly][] usually performs very well without any compiler plugins.
40However, we have fixed some deficiencies in GHC's optimizer using a
41[compiler plugin](https://github.com/composewell/fusion-plugin). We
42hope to fold these optimizations into GHC in the future; until then we
43recommend that you use this plugin for applications that are performance
46### Performance Benchmarks
48We measured several Haskell streaming implementations
49using various micro-benchmarks. Please see the [streaming
50benchmarks][streaming-benchmarks] page for a detailed comparison of
51Streamly against other streaming libraries.
53Our results show that [Streamly][] is the fastest effectful streaming
54implementation on almost all the measured microbenchmarks. In many cases
55it runs up to 100x faster, and in some cases even 1000x faster than
56some of the tested alternatives. In some composite operation benchmarks
57[Streamly][] turns out to be significantly faster than Haskell's list
60*Note*: If you can write a program in some other way or with some other
61language that runs significantly faster than what [Streamly][] offers,
62please let us know and we will improve.
66Streamly comes equipped with a very powerful set of abstractions to
67accomplish many kinds of programming tasks: it provides support for
68programming with streams and arrays, for reading and writing from the
69file system and from the network, for time domain programming (reactive
70programming), and for reacting to file system events using `fsnotify`.
72Please view [Streamly's documentation][Streamly] for more information
73about Streamly's features.
79Our goals for [Streamly][] from the very beginning have been:
811. To achieve simplicity by unifying abstractions.
822. To offer high performance.
84These goals are hard to achieve simultaneously because they are usually
85inversely related. We have spent many years trying to get the abstractions
86right without compromising performance.
88`Unfold` is an example of an abstraction that we have created to achieve
89high performance when mapping streams on streams. `Unfold` allows stream
90generation to be optimized well by the compiler through stream fusion.
91A `Fold` with termination capability is another example which modularizes
92stream elimination operations through stream fusion. Terminating folds
93can perform many simple parsing tasks that do not require backtracking.
94In Streamly, `Parser`s are a natural extension to terminating `Fold`s;
95`Parser`s add the ability to backtrack to `Fold`s. Unification leads
96to simpler abstractions and lower cognitive overheads while also not
97compromising performance.
101Streamly uses lock-free synchronization for achieving concurrent
102operation with low overheads. The number of tasks performed concurrently
103are determined automatically based on the rate at which a consumer
104is consuming the results. In other words, you do not need to manage
105thread pools or decide how many threads to use for a particular task.
106For CPU-bound tasks Streamly will try to keep the number of threads
107close to the number of CPUs available; for IO-bound tasks it will utilize
110The parallelism available during program execution can be utilized with
111very little overhead even where the task size is very small, because
112Streamly will automatically switch between serial or batched execution
113of tasks on the same CPU depending on whichever is more efficient.
114Please see our [concurrency benchmarks][concurrency-benchmarks] for more
115detailed performance measurements, and for a comparison with the `async`
120The following authors/libraries have influenced or inspired this library in a
123 * Roman Leshchinskiy ([vector](http://hackage.haskell.org/package/vector))
124 * Gabriella Gonzalez ([foldl](https://hackage.haskell.org/package/foldl))
125 * Alberto G. Corona ([transient](https://hackage.haskell.org/package/transient))
127Please see the [`credits`](/docs/User/ProjectRelated/Credits.md) directory for a full
128list of contributors, credits and licenses.
132Streamly is an [open source](https://github.com/composewell/streamly)
133project available under a liberal [BSD-3-Clause license][LICENSE]
135## Contributing to Streamly
137As an open project we welcome contributions:
139* [Streamly Contributor's Guide][CONTRIBUTING.md]
140* [Contact the streamly development team](mailto:streamly@composewell.com)
144Professional support is available for [Streamly][]: please contact
145[support@composewell.com](mailto:support@composewell.com).
147You can also join our [community chat
148channel](https://gitter.im/composewell/streamly) on Gitter.
154[Streamly]: https://streamly.composewell.com/
155[streaming-benchmarks]: https://github.com/composewell/streaming-benchmarks
156[concurrency-benchmarks]: https://github.com/composewell/concurrency-benchmarks
159Keep all the unstable links here so that they can be updated to stable
160links (for online docs) before we release.
165[CONTRIBUTING.md]: /CONTRIBUTING.md
keysesc to close
- j k
- move down, up
- enter
- open
- esc
- back, or up a level
- 1 2 3 4
- readme, files, commits, refs
- /
- filter
- y
- copy the clone command
- c
- compare the chosen commit with another (or shift-click both)
- ?
- show or hide this list