@@ -1,119 +1,83 @@
1−# [Streamly][]: Idiomatic Haskell with the Performance of C
1+# [Streamly][]: Idiomatic Haskell with C-Like Performance
22
33[](https://gitter.im/composewell/streamly)
44[](https://hackage.haskell.org/package/streamly)
55
6−Streamly is a Haskell library that provides building blocks to build
7−safe, scalable, modular and high performance software. Streamly offers:
6+Streamly is a powerful Haskell library that provides developers with
7+the essential building blocks to create safe, scalable, modular, and
8+high-performance software. With Streamly, developers can enjoy the
9+benefits of Haskell's type safety while leveraging C-like program
10+performance. Streamly offers a comprehensive range of features,
11+comprising:
812
9−* The type safety of Haskell.
10−* The performance of C programs.
11−* Powerful building blocks for modular code.
13+* Haskell's strong type safety.
14+* C-program-like performance capabilities.
15+* Flexible, modular building blocks.
1216* Idiomatic functional programming.
13−* Declarative, fearless concurrency.
14−* Ecossytem libraries for quick development.
15−
16−Please read the [Streamly Setup and Usage
17−Guide](/docs/User/Tutorials/setup-and-usage.md) and [Streamly Quick
18−Overview](/docs/User/Tutorials/quick-overview.md) to get a taste of the
19−library. Streamly comes with comprehensive documentation, please visit
20−the [Haskell Streamly website][Streamly] for documentation.
21−
22−## Performance with Modularity
23−
24−Usually, you have to pick one of the two, performance or
25−modularity. Using [Haskell Streamly][Streamly] you can write highly
26−modular code and still achieve performance close to an equivalent
27−(imperative) C program. Streamly exploits GHC's stream fusion
28−optimizations (`case-of-case` and `spec-constr`) aggressively to achieve
29−both modularity and performance at the same time.
30−
31−Streamly offers excellent performance even for byte-at-a-time stream
32−operations using efficient abstractions like `Unfold`s and
33−`Fold`s. Byte-at-a-time stream operations can simplify programming
34−because the developer does not have to deal explicitly with chunking
35−and re-combining data.
36−
37−### GHC Plugin for Stream Fusion
38−
39−[Streamly][] usually performs very well without any compiler plugins.
40−However, we have fixed some deficiencies in GHC's optimizer using a
41−[compiler plugin](https://github.com/composewell/fusion-plugin). We
42−hope to fold these optimizations into GHC in the future; until then we
43−recommend that you use this plugin for applications that are performance
44−sensitive.
45−
46−### Performance Benchmarks
47−
48−We measured several Haskell streaming implementations
49−using various micro-benchmarks. Please see the [streaming
50−benchmarks][streaming-benchmarks] page for a detailed comparison of
51−Streamly against other streaming libraries.
52−
53−Our results show that [Streamly][] is the fastest effectful streaming
54−implementation on almost all the measured microbenchmarks. In many cases
55−it runs up to 100x faster, and in some cases even 1000x faster than
56−some of the tested alternatives. In some composite operation benchmarks
57−[Streamly][] turns out to be significantly faster than Haskell's list
58−implementation.
59−
60−*Note*: If you can write a program in some other way or with some other
61−language that runs significantly faster than what [Streamly][] offers,
62−please let us know and we will improve.
63−
64−## Applications
65−
66−Streamly comes equipped with a very powerful set of abstractions to
67−accomplish many kinds of programming tasks: it provides support for
68−programming with streams and arrays, for reading and writing from the
69−file system and from the network, for time domain programming (reactive
70−programming), and for reacting to file system events using `fsnotify`.
71−
72−Please view [Streamly's documentation][Streamly] for more information
17+* Fearless, declarative concurrency for seamless parallel execution.
18+* A collection of ecosystem libraries for fast and efficient development.
19+
20+Check out the [Streamly Setup and Usage
21+Guide](/docs/User/Tutorials/setup-and-usage.md) and [Quick
22+Overview](/docs/User/Tutorials/quick-overview.md) for an introduction
23+to the library. For more detailed documentation, visit the [Haskell
24+Streamly website][Streamly].
25+
26+## Blazing Fast
27+
28+Streamly delivers C-like speed in Haskell by fusing stream pipelines
29+using the stream-fusion technique, resulting in compiled code that is
30+equivalent to handwritten C code, eliminating intermediate allocations
31+and function calls.
32+
33+For a comprehensive comparison of Streamly to other Haskell streaming
34+libraries, check out our [streaming benchmarks][streaming-benchmarks]
35+page. In fact, Streamly's fused loops can be up to 100 times faster than
36+those of libraries without stream fusion.
37+
38+## Declarative Concurrency
39+
40+Streamly introduces declarative concurrency to standard functional
41+streaming abstractions. Declarative concurrency abstracts away the
42+low-level details of concurrency management, such as locks and threads,
43+and allows for easier and safer parallelization of code. For example,
44+with Streamly you can do things like repeat actions concurrently to
45+generate a stream of results, map functions concurrently on a stream,
46+and combine multiple streams concurrently to create a single output
47+stream.
48+
49+## Unified API
50+
51+Streamly provides a comprehensive and unified API for basic programming
52+needs, covering a wide range of areas including streaming, concurrency,
53+logic programming, reactive programming, pinned and unpinned arrays,
54+serialization, builders, parsers, unicode processing, file-io, file
55+system events, and network-io. By unifying functionality from disparate
56+Haskell libraries, Streamly simplifies development while delivering
57+equivalent or improved performance. Additionally, the complexity
58+of handling combinations of lazy, strict, bytestring, and text is
59+eliminated by using streams for lazy evaluation, and by generalizing
60+bytestring and text to arrays.
61+
62+Check out [Streamly's documentation][Streamly] for more information
7363about Streamly's features.
7464
75−## Design
76−
77−### Design Goals
78−
79−Our goals for [Streamly][] from the very beginning have been:
80−
81−1. To achieve simplicity by unifying abstractions.
82−2. To offer high performance.
83−
84−These goals are hard to achieve simultaneously because they are usually
85−inversely related. We have spent many years trying to get the abstractions
86−right without compromising performance.
87−
88−`Unfold` is an example of an abstraction that we have created to achieve
89−high performance when mapping streams on streams. `Unfold` allows stream
90−generation to be optimized well by the compiler through stream fusion.
91−A `Fold` with termination capability is another example which modularizes
92−stream elimination operations through stream fusion. Terminating folds
93−can perform many simple parsing tasks that do not require backtracking.
94−In Streamly, `Parser`s are a natural extension to terminating `Fold`s;
95−`Parser`s add the ability to backtrack to `Fold`s. Unification leads
96−to simpler abstractions and lower cognitive overheads while also not
97−compromising performance.
98−
99−### Concurrency Design
100−
101−Streamly uses lock-free synchronization for achieving concurrent
102−operation with low overheads. The number of tasks performed concurrently
103−are determined automatically based on the rate at which a consumer
104−is consuming the results. In other words, you do not need to manage
105−thread pools or decide how many threads to use for a particular task.
106−For CPU-bound tasks Streamly will try to keep the number of threads
107−close to the number of CPUs available; for IO-bound tasks it will utilize
108−more threads.
109−
110−The parallelism available during program execution can be utilized with
111−very little overhead even where the task size is very small, because
112−Streamly will automatically switch between serial or batched execution
113−of tasks on the same CPU depending on whichever is more efficient.
114−Please see our [concurrency benchmarks][concurrency-benchmarks] for more
115−detailed performance measurements, and for a comparison with the `async`
116−package.
65+## Batteries Included
66+
67+In addition to the fundamental programming constructs, Streamly also
68+provides higher-level functionality through supporting packages such as
69+[streamly-process][], [streamly-shell][], and [streamly-coreutils][]
70+that are essential for general programming tasks. Check out the
71+[streamly-examples][] repository for some program snippets.
72+
73+## Highly Modular
74+
75+Traditionally, you must choose between modularity and performance when
76+writing code. However, with [Haskell Streamly][Streamly], you can have
77+the best of both worlds. By taking advantage of GHC's stream fusion
78+optimizations (such as `case-of-case` and `spec-constr`), Streamly achieves
79+performance comparable to an equivalent C program while still allowing
80+for highly modular code.
11781
11882## Credits
11983
@@ -154,6 +118,10 @@ Link References.
154118[Streamly]: https://streamly.composewell.com/
155119[streaming-benchmarks]: https://github.com/composewell/streaming-benchmarks
156120[concurrency-benchmarks]: https://github.com/composewell/concurrency-benchmarks
121+[streamly-examples]: https://github.com/composewell/streamly-examples
122+[streamly-process]: https://github.com/composewell/streamly-process
123+[streamly-shell]: https://github.com/composewell/streamly-shell
124+[streamly-coreutils]: https://github.com/composewell/streamly-coreutils
157125
158126<!--
159127Keep all the unstable links here so that they can be updated to stable
@@ -162,5 +130,5 @@ links (for online docs) before we release.
162130
163131<!-- local files -->
164132[LICENSE]: /LICENSE
165−[CONTRIBUTING.md]: /CONTRIBUTING.md
133+[CONTRIBUTING.md]: /docs/Developer/Contributing.md
166134[docs]: docs/