@@ -11,46 +11,57 @@
1111## Streaming Concurrently
1212
1313Haskell lists express pure computations using composable stream operations like
14−`:`, `unfold`, `map`, `filter`, `zip` and `fold`. Streamly extends this data
15−flow programming model of pure lists to lists of concurrent monadic
16−computations (streams) using the same primitives.
17−
18−Streamly expresses concurrency using the list primitives, and standard, well
19−known abstractions, without having to know any low level notions of concurrency
20−like threads, locking or synchronization. Concurrency is automatically scaled
21−up or down based on the need of the application, so that we can say goodbye to
22−managing thread pools and associated sizing issues as well. This is true,
23−fearless and declarative concurrency. Streamly can be thought of as concurrent
24−monadic lists, if you know Haskell lists then you already know how to use
25−streamly.
14+`:`, `unfold`, `map`, `filter`, `zip` and `fold`. Streamly is exactly like
15+lists except that it can express sequences of pure as well as monadic
16+computations aka streams. More importantly, it can express monadic sequences
17+with concurrent execution semantics without introducing any additional APIs.
18+
19+Streamly expresses concurrency using standard, well known abstractions.
20+Concurrency semantics are defined for list operations, semigroup, applicative
21+and monadic compositions. Programmer does not need to know any low level
22+notions of concurrency like threads, locking or synchronization. Concurrent
23+and non-concurrent programs are fundamentally the same. A chosen segment of
24+the program can be made concurrent by annotating it with an appropriate
25+combinator. We can choose a combinator for lookahead style or asynchronous
26+concurrency. Concurrency is automatically scaled up or down based on the
27+demand from the consumer application, we can finally say goodbye to managing
28+thread pools and associated sizing issues. The result is truly fearless
29+and declarative monadic concurrency.
2630
2731## Where to use streamly?
2832
29−Everywhere. The answer to this question would be similar to the answer to -
30−"Where do I use Haskell lists?". Streamly generalizes lists to monadic
31−streams, and IO monad to non-deterministic stream composition with concurrency. The
32−`IO` monad becomes a special case of streamly, if we use single element streams
33−the behavior of streamly is identical to the IO monad. It can be replaced with
34−streamly by just prefixing IO actions with `liftIO`, without any loss of
35−performance. Pure lists too become a special case of streamly; if we use the
36−`Identity` monad, streams turn into pure lists. Non-concurrent programs become
37−a special case of concurrent ones, by just adding a combinator, a non-concurrent
38−program becomes concurrent.
39−
40−We can say that streamly is a superset of lists and IO, with builtin
41−concurrency. If you want to write a program that involves IO, concurrent or
42−not, then you can just use streamly as the base monad, heck, you could even use
43−streamly for pure computations, as streamly performs at par with pure lists or
44−`vector`. In fact, streamly is better than lists because it appends much
45−faster than lists, you do not need difference lists for that.
46−
47−If you need convincing for using streaming or data flow programming paradigm
48−itself then try to answer this question - why do we use lists? It boils down to
49−why we use functional programming in the first place, and Haskell is successful
50−in enforcing this for pure computations, but not for monadic computations. In
51−the absence of a standard, easy to use or enforced data flow programming
52−library for monadic computations, and the IO monad providing an escape hatch to
53−an imperative model, we just love to fall into the imperative trap.
33+Streamly is a general purpose programming framwework. It can be used equally
34+efficiently from a simple `Hello World!` program to a massively concurrent
35+application. The answer to the question, "where to use streamly?" - would be
36+similar to the answer to - "Where to use Haskell lists or the IO monad?".
37+Streamly generalizes lists to monadic streams, and the `IO` monad to
38+non-deterministic and concurrent stream composition. The `IO` monad is a
39+special case of streamly; if we use single element streams the behavior of
40+streamly becomes identical to the IO monad. The IO monad code can be replaced
41+with streamly by just prefixing the IO actions with `liftIO`, without any other
42+changes, and without any loss of performance. Pure lists too are a special
43+case of streamly; if we use `Identity` as the underlying monad, streamly
44+streams turn into pure lists. Non-concurrent programs are just a special case
45+of concurrent ones, simply adding a combinator turns a non-concurrent program
46+into a concurrent one.
47+
48+In other words, streamly combines the functionality of lists and IO, with
49+builtin concurrency. If you want to write a program that involves IO,
50+concurrent or not, then you can just use streamly as the base monad, in fact,
51+you could even use streamly for pure computations, as streamly performs at par
52+with pure lists or `vector`.
53+
54+## Why data flow programming?
55+
56+If you need some convincing for using streaming or data flow programming
57+paradigm itself then try to answer this question - why do we use lists in
58+Haskell? It boils down to why we use functional programming in the first place.
59+Haskell is successful in enforcing the functional data flow paradigm for pure
60+computations using lists, but not for monadic computations. In the absence of a
61+standard and easy to use data flow programming paradigm for monadic
62+computations, and the IO monad providing an escape hatch to an imperative
63+model, we just love to fall into the imperative trap, and start asking the same
64+fundamental question again - why do we have to use the streaming data model?
5465
5566## Show me an example
5667
@@ -89,9 +100,9 @@ main = runStream $ aheadly $ listDirRecursive
89100```
90101
91102Isn't that magical? What's going on here? Streamly does not introduce any new
92−abstractions, it just uses the standard abstractions like `Semigroup` or
103+abstractions, it just uses standard abstractions like `Semigroup` or
93104`Monoid` to combine monadic streams concurrently, the way lists combine a
94−sequence of pure values non-concurrently. Therefore, the `foldMap` in the code
105+sequence of pure values non-concurrently. The `foldMap` in the code
95106above turns into a concurrent monoidal composition of a stream of `readdir`
96107computations.
97108
@@ -348,6 +359,22 @@ rate. Rate control works precisely even at throughputs as high as millions of
348359yields per second. For more sophisticated rate control see the haddock
349360documentation.
350361
362+## Exceptions
363+
364+From a library user point of view, there is nothing much to learn or talk about
365+exceptions. Synchronous exceptions work just the way they are supposed to work
366+in any standard non-concurrent code. When concurrent streams are combined
367+together, exceptions from the constituent streams are propagated to the
368+consumer stream. When an exception occurs in any of the constituent streams
369+other concurrent streams are promptly terminated. Exceptions can be thrown
370+using the `MonadThrow` instance.
371+
372+There is no notion of explicit threads in streamly, therefore, no
373+asynchronous exceptions to deal with. You can just ignore the zillions of
374+blogs, talks, caveats about async exceptions. Async exceptions just don't
375+exist. Please don't use things like `myThreadId` and `throwTo` just for fun!
376+
377+
351378## Reactive Programming (FRP)
352379
353380Streamly is a foundation for first class reactive programming as well by virtue
@@ -419,6 +446,12 @@ For more information, see:
419446 * [Quick tutorial comparing streamly with the async package](https://github.com/composewell/streamly/blob/master/docs/Async.md)
420447 * [Concurrency benchmarks comparing streamly with async](https://github.com/composewell/concurrency-benchmarks)
421448
449+## Support
450+
451+If you require professional support, consulting, training or timely
452+enhancements to the library please contact
453+[support@composewell.com](mailto:support@composewell.com).
454+
422455## Contributing
423456
424457The code is available under BSD-3 license