@@ -1,119 +1,83 @@
1−# [Streamly][]: Idiomatic Haskell with the Performance of C
1+# [Streamly][]: Idiomatic Haskell with C-Like Performance
3[](https://gitter.im/composewell/streamly)
3[](https://gitter.im/composewell/streamly)
4[](https://hackage.haskell.org/package/streamly)
4[](https://hackage.haskell.org/package/streamly)
6−Streamly is a Haskell library that provides building blocks to build
6+Streamly is a powerful Haskell library that provides developers with
7+the essential building blocks to create safe, scalable, modular, and
7−safe, scalable, modular and high performance software. Streamly offers:
8+high-performance software. With Streamly, developers can enjoy the
9+benefits of Haskell's type safety while leveraging C-like program
10+performance. Streamly offers a comprehensive range of features,
9−* The type safety of Haskell.
13+* Haskell's strong type safety.
10−* The performance of C programs.
14+* C-program-like performance capabilities.
11−* Powerful building blocks for modular code.
15+* Flexible, modular building blocks.
12* Idiomatic functional programming.
16* Idiomatic functional programming.
13−* Declarative, fearless concurrency.
17+* Fearless, declarative concurrency for seamless parallel execution.
14−* Ecossytem libraries for quick development.
18+* A collection of ecosystem libraries for fast and efficient development.
16−Please read the [Streamly Setup and Usage
20+Check out the [Streamly Setup and Usage
17−Guide](/docs/User/Tutorials/setup-and-usage.md) and [Streamly Quick
21+Guide](/docs/User/Tutorials/setup-and-usage.md) and [Quick
18−Overview](/docs/User/Tutorials/quick-overview.md) to get a taste of the
22+Overview](/docs/User/Tutorials/quick-overview.md) for an introduction
19−library. Streamly comes with comprehensive documentation, please visit
23+to the library. For more detailed documentation, visit the [Haskell
20−the [Haskell Streamly website][Streamly] for documentation.
24+Streamly website][Streamly].
22−## Performance with Modularity
26+## Blazing Fast
24−Usually, you have to pick one of the two, performance or
28+Streamly delivers C-like speed in Haskell by fusing stream pipelines
25−modularity. Using [Haskell Streamly][Streamly] you can write highly
29+using the stream-fusion technique, resulting in compiled code that is
26−modular code and still achieve performance close to an equivalent
30+equivalent to handwritten C code, eliminating intermediate allocations
27−(imperative) C program. Streamly exploits GHC's stream fusion
31+and function calls.
28−optimizations (`case-of-case` and `spec-constr`) aggressively to achieve
32+
29−both modularity and performance at the same time.
33+For a comprehensive comparison of Streamly to other Haskell streaming
31−Streamly offers excellent performance even for byte-at-a-time stream
32−operations using efficient abstractions like `Unfold`s and
33−`Fold`s. Byte-at-a-time stream operations can simplify programming
34−because the developer does not have to deal explicitly with chunking
35−and re-combining data.
37−### GHC Plugin for Stream Fusion
39−[Streamly][] usually performs very well without any compiler plugins.
40−However, we have fixed some deficiencies in GHC's optimizer using a
41−[compiler plugin](https://github.com/composewell/fusion-plugin). We
42−hope to fold these optimizations into GHC in the future; until then we
43−recommend that you use this plugin for applications that are performance
46−### Performance Benchmarks
48−We measured several Haskell streaming implementations
49−using various micro-benchmarks. Please see the [streaming
50−benchmarks][streaming-benchmarks] page for a detailed comparison of
34+libraries, check out our [streaming benchmarks][streaming-benchmarks]
51−Streamly against other streaming libraries.
35+page. In fact, Streamly's fused loops can be up to 100 times faster than
52−
36+those of libraries without stream fusion.
53−Our results show that [Streamly][] is the fastest effectful streaming
37+
54−implementation on almost all the measured microbenchmarks. In many cases
38+## Declarative Concurrency
55−it runs up to 100x faster, and in some cases even 1000x faster than
39+
56−some of the tested alternatives. In some composite operation benchmarks
40+Streamly introduces declarative concurrency to standard functional
57−[Streamly][] turns out to be significantly faster than Haskell's list
41+streaming abstractions. Declarative concurrency abstracts away the
58−implementation.
42+low-level details of concurrency management, such as locks and threads,
59−
43+and allows for easier and safer parallelization of code. For example,
60−*Note*: If you can write a program in some other way or with some other
44+with Streamly you can do things like repeat actions concurrently to
61−language that runs significantly faster than what [Streamly][] offers,
45+generate a stream of results, map functions concurrently on a stream,
62−please let us know and we will improve.
46+and combine multiple streams concurrently to create a single output
66−Streamly comes equipped with a very powerful set of abstractions to
50+
67−accomplish many kinds of programming tasks: it provides support for
51+Streamly provides a comprehensive and unified API for basic programming
68−programming with streams and arrays, for reading and writing from the
52+needs, covering a wide range of areas including streaming, concurrency,
69−file system and from the network, for time domain programming (reactive
53+logic programming, reactive programming, pinned and unpinned arrays,
70−programming), and for reacting to file system events using `fsnotify`.
54+serialization, builders, parsers, unicode processing, file-io, file
71−
55+system events, and network-io. By unifying functionality from disparate
56+Haskell libraries, Streamly simplifies development while delivering
57+equivalent or improved performance. Additionally, the complexity
58+of handling combinations of lazy, strict, bytestring, and text is
59+eliminated by using streams for lazy evaluation, and by generalizing
60+bytestring and text to arrays.
72−Please view [Streamly's documentation][Streamly] for more information
62+Check out [Streamly's documentation][Streamly] for more information
73about Streamly's features.
63about Streamly's features.
75−## Design
65+## Batteries Included
77−### Design Goals
67+In addition to the fundamental programming constructs, Streamly also
78−
68+provides higher-level functionality through supporting packages such as
79−Our goals for [Streamly][] from the very beginning have been:
69+[streamly-process][], [streamly-shell][], and [streamly-coreutils][]
80−
70+that are essential for general programming tasks. Check out the
81−1. To achieve simplicity by unifying abstractions.
71+[streamly-examples][] repository for some program snippets.
82−2. To offer high performance.
72+
84−These goals are hard to achieve simultaneously because they are usually
74+
85−inversely related. We have spent many years trying to get the abstractions
75+Traditionally, you must choose between modularity and performance when
86−right without compromising performance.
76+writing code. However, with [Haskell Streamly][Streamly], you can have
87−
77+the best of both worlds. By taking advantage of GHC's stream fusion
88−`Unfold` is an example of an abstraction that we have created to achieve
78+optimizations (such as `case-of-case` and `spec-constr`), Streamly achieves
89−high performance when mapping streams on streams. `Unfold` allows stream
79+performance comparable to an equivalent C program while still allowing
90−generation to be optimized well by the compiler through stream fusion.
80+for highly modular code.
91−A `Fold` with termination capability is another example which modularizes
92−stream elimination operations through stream fusion. Terminating folds
93−can perform many simple parsing tasks that do not require backtracking.
94−In Streamly, `Parser`s are a natural extension to terminating `Fold`s;
95−`Parser`s add the ability to backtrack to `Fold`s. Unification leads
96−to simpler abstractions and lower cognitive overheads while also not
97−compromising performance.
99−### Concurrency Design
101−Streamly uses lock-free synchronization for achieving concurrent
102−operation with low overheads. The number of tasks performed concurrently
103−are determined automatically based on the rate at which a consumer
104−is consuming the results. In other words, you do not need to manage
105−thread pools or decide how many threads to use for a particular task.
106−For CPU-bound tasks Streamly will try to keep the number of threads
107−close to the number of CPUs available; for IO-bound tasks it will utilize
110−The parallelism available during program execution can be utilized with
111−very little overhead even where the task size is very small, because
112−Streamly will automatically switch between serial or batched execution
113−of tasks on the same CPU depending on whichever is more efficient.
114−Please see our [concurrency benchmarks][concurrency-benchmarks] for more
115−detailed performance measurements, and for a comparison with the `async`
118## Credits
82## Credits
@@ -154,6 +118,10 @@ Link References.
154[Streamly]: https://streamly.composewell.com/
118[Streamly]: https://streamly.composewell.com/
155[streaming-benchmarks]: https://github.com/composewell/streaming-benchmarks
119[streaming-benchmarks]: https://github.com/composewell/streaming-benchmarks
156[concurrency-benchmarks]: https://github.com/composewell/concurrency-benchmarks
120[concurrency-benchmarks]: https://github.com/composewell/concurrency-benchmarks
121+[streamly-examples]: https://github.com/composewell/streamly-examples
122+[streamly-process]: https://github.com/composewell/streamly-process
123+[streamly-shell]: https://github.com/composewell/streamly-shell
124+[streamly-coreutils]: https://github.com/composewell/streamly-coreutils
159Keep all the unstable links here so that they can be updated to stable
127Keep all the unstable links here so that they can be updated to stable
@@ -162,5 +130,5 @@ links (for online docs) before we release.
163<!-- local files -->
131<!-- local files -->
164[LICENSE]: /LICENSE
132[LICENSE]: /LICENSE
165−[CONTRIBUTING.md]: /CONTRIBUTING.md
133+[CONTRIBUTING.md]: /docs/Developer/Contributing.md
166[docs]: docs/
134[docs]: docs/