@@ -0,0 +1,200 @@
1+## `async/wait` Model vs Concurrent Streams
2+
3+Streamly is a library to make concurrent programming a joy. The venerable
4+`async` package is the go to package for concurrent programming for most
5+Haskellers. Streamly is a higher level library than `async` and provides a lot
6+more power and functionality, using a simpler and concise expression of
7+concurrency. You should be able to express everything with streamly that you
8+can with `async`, if you can't please raise an issue. If you are familiar with
9+`async`, in this document we highlight how streamly can be used where you would
10+use `async`.
11+
12+Unlike `async`, streamly does not use a spawn and `wait` model. Streamly uses
13+a more high level approach to concurrency and has no explicit notion of
14+threads. In streamly, we compose multiple actions as a stream and then express
15+whether you want to run the actions in the stream `serially` or `parallely`.
16+There are many different ways in which you can run streams concurrently, see
17+the reference documentation for details.
18+
19+Since there is no explicit notion of threads in streamly, there are no
20+equivalents of `async`, `wait`, `cancel`, `poll` or `link` combinators from the
21+`async` package.
22+
23+Since streamly is a monad transformer it can work with all monads and not just
24+IO, you won't need adaptations like `lifted-async` to use it for a generic
25+monad.
26+
27+## Using Streamly for Concurrency
28+
29+You can write all of your program in a streamly monad and use the full power of
30+the library. Streamly can be used as a direct replacement of the IO monad with
31+no loss of performance, and no change in code except using `liftIO` or `yieldM`
32+to run any IO actions. Streamly IO monads (e.g. `SerialT IO`) are just a
33+generalization of the IO monad with non-deterministic composition of streams
34+added on top.
35+
36+However, if you would like to just run only some concurrent portions of your
37+program using streamly, you can do that too. Just use `runStream` if you want
38+to run the stream without collecting the outputs of the concurrent actions or
39+use `toList` if you want to convert the output stream into a list. Other
40+stream folding operations can also be used, see the docs for more details.
41+
42+## Features as Compared with `async`
43+
44+Use the following imports to run the snippets shown below:
45+
46+```haskell
47+import Streamly
48+import Streamly.Prelude ((|:))
49+import qualified Streamly.Prelude as S
50+import qualified Data.Text as Text
51+import Control.Concurrent (threadDelay)
52+```
53+
54+Let us simulate a URL fetch with a delay of `n` seconds using the following
55+functions:
56+
57+```haskell
58+getURL :: Int -> IO String
59+getURL n = threadDelay (n * 1000000) >> return (show n)
60+getURLString = getURL
61+getURLText n = getURL n >>= return . Text.pack
62+```
63+
64+### concurrently
65+
66+You can run any number of actions concurrently. For example, to fetch two URLs
67+concurrently:
68+
69+```haskell
70+ urls <- S.toList $ parallely $ getURL 2 |: getURL 1 |: S.nil
71+```
72+
73+This would return the results in their arrival order i.e. first 1 and then 2.
74+If you want to preserve the order of the results, use the lookahead style
75+stream `aheadly` instead. In the following example both URLs are fetched
76+concurrently, and even though URL 1 arrives before URL 2 the results will
77+return 2 first and then 1.
78+
79+```haskell
80+ urls <- S.toList $ aheadly $ getURL 2 |: getURL 1 |: S.nil
81+```
82+
83+### concurrently_
84+
85+Use `runStream` instead of `toList` to run the actions but ignore the results:
86+
87+```haskell
88+ runStream $ parallely $ getURL 1 |: getURL 2 |: S.nil
89+```
90+
91+### Concurrent Applicative
92+
93+If the actions that you are executing result in different output types you can
94+use applicative zip to collect the results or to directly apply them to a
95+function:
96+
97+```haskell
98+ tuples <- S.toList $ zipAsyncly $
99+ (,) <$> S.yieldM (getURLString 1) <*> S.yieldM (getURLText 2)
100+```
101+
102+### race
103+
104+To run multiple actions and use the first result:
105+
106+```haskell
107+ urls <- S.toList $ S.take 1 $ parallely $ getURL 1 |: getURL 2 |: S.nil
108+```
109+
110+After the first result arrives, the rest of the actions are canceled
111+automatically. You can take first `n` results as they arrive:
112+
113+```haskell
114+ urls <- S.toList $ S.take 2 $ parallely $ getURL 1 |: getURL 2 |: S.nil
115+```
116+
117+### mapConcurrently
118+
119+There are many ways to map concurrently on a container and collect the results:
120+
121+You can create a concurrent stream from a `Foldable` container of monadic
122+actions:
123+
124+```haskell
125+ urls <- S.toList $ aheadly $ S.fromFoldableM $ fmap getURL [1..3]
126+```
127+
128+You can first convert a `Foldable` into a stream and then map an action on the
129+stream concurrently:
130+
131+```haskell
132+ urls <- S.toList $ aheadly $ S.mapM getURL $ foldMap return [1..3]
133+```
134+
135+You can map a monadic action to a `Foldable` container to convert it into a
136+stream and at the same time fold it:
137+
138+```haskell
139+ urls <- S.toList $ aheadly $ foldMap (S.yieldM . getURL) [1..3]
140+```
141+
142+### replicateConcurrently
143+
144+Streamly has not just the equivalent of `replicateConcurrently` which is
145+`replicateM` but many more ways to generate concurrent streams, for example,
146+`|:`, `unfoldrM`, `repeatM`, `iterateM`, `fromFoldableM` etc. See the
147+[Streamly.Prelude](https://hackage.haskell.org/package/streamly/docs/Streamly-Prelude.html)
148+module documentation for more details.
149+
150+```haskell
151+ xs <- S.toList $ parallely $ S.replicateM 2 $ getURL 1
152+```
153+
154+### Functor
155+
156+The stream resulting from concurrent actions can be mapped serially or
157+concurrently.
158+
159+To map serially just use `fmap`:
160+
161+```haskell
162+ xs <- S.toList $ parallely $ fmap (+1) $ return 1 |: return 2 |: S.nil
163+```
164+
165+To map a monadic action concurrently on all elements of the stream use `mapM`:
166+
167+```haskell
168+ xs <- S.toList $ parallely $ S.mapM (\x -> return (x + 1))
169+ $ return 1 |: return 2 |: S.nil
170+```
171+
172+### Semigroup
173+
174+The `Semigroup` instances of streamly merge multiple streams serially or
175+concurrently.
176+
177+### Monad
178+
179+The `Monad` instances of streamly nest loops concurrently (concurrent
180+non-determinism).
181+
182+### Performance
183+
184+Streamly has very little concurrency overhead (ranging from a few 100
185+nanoseconds to a few microseconds on a 2.2 GHz Intel Core i7), you can even run
186+very lightweight actions in parallel without worrying about the overhead of
187+concurrency. See the performance benchmarks [comparing streamly with the `async`
188+package in this repo](https://github.com/composewell/concurrency-benchmarks).
189+
190+## Further Reading
191+
192+There is much more that you can do with streamly. For example, you can use the
193+`maxThreads` combinator to restrict the total number of concurrent threads or
194+use the `maxBuffer` combinator to restrict the total number of bufferred
195+results or you can use the `avgRate` combinator to control the rate at which
196+the concurrent actions are executed.
197+
198+See the [haddock documentation on
199+hackage](https://hackage.haskell.org/package/streamly) and [a comprehensive tutorial
200+here](https://hackage.haskell.org/package/streamly/docs/Streamly-Tutorial.html).