@@ -0,0 +1,200 @@
1+## `async/wait` Model vs Concurrent Streams
3+Streamly is a library to make concurrent programming a joy. The venerable
4+`async` package is the go to package for concurrent programming for most
5+Haskellers. Streamly is a higher level library than `async` and provides a lot
6+more power and functionality, using a simpler and concise expression of
7+concurrency. You should be able to express everything with streamly that you
8+can with `async`, if you can't please raise an issue. If you are familiar with
9+`async`, in this document we highlight how streamly can be used where you would
12+Unlike `async`, streamly does not use a spawn and `wait` model. Streamly uses
13+a more high level approach to concurrency and has no explicit notion of
14+threads. In streamly, we compose multiple actions as a stream and then express
15+whether you want to run the actions in the stream `serially` or `parallely`.
16+There are many different ways in which you can run streams concurrently, see
17+the reference documentation for details.
19+Since there is no explicit notion of threads in streamly, there are no
20+equivalents of `async`, `wait`, `cancel`, `poll` or `link` combinators from the
23+Since streamly is a monad transformer it can work with all monads and not just
24+IO, you won't need adaptations like `lifted-async` to use it for a generic
27+## Using Streamly for Concurrency
29+You can write all of your program in a streamly monad and use the full power of
30+the library. Streamly can be used as a direct replacement of the IO monad with
31+no loss of performance, and no change in code except using `liftIO` or `yieldM`
32+to run any IO actions. Streamly IO monads (e.g. `SerialT IO`) are just a
33+generalization of the IO monad with non-deterministic composition of streams
36+However, if you would like to just run only some concurrent portions of your
37+program using streamly, you can do that too. Just use `runStream` if you want
38+to run the stream without collecting the outputs of the concurrent actions or
39+use `toList` if you want to convert the output stream into a list. Other
40+stream folding operations can also be used, see the docs for more details.
42+## Features as Compared with `async`
44+Use the following imports to run the snippets shown below:
48+import Streamly.Prelude ((|:))
49+import qualified Streamly.Prelude as S
50+import qualified Data.Text as Text
51+import Control.Concurrent (threadDelay)
54+Let us simulate a URL fetch with a delay of `n` seconds using the following
58+getURL :: Int -> IO String
59+getURL n = threadDelay (n * 1000000) >> return (show n)
61+getURLText n = getURL n >>= return . Text.pack
66+You can run any number of actions concurrently. For example, to fetch two URLs
70+ urls <- S.toList $ parallely $ getURL 2 |: getURL 1 |: S.nil
73+This would return the results in their arrival order i.e. first 1 and then 2.
74+If you want to preserve the order of the results, use the lookahead style
75+stream `aheadly` instead. In the following example both URLs are fetched
76+concurrently, and even though URL 1 arrives before URL 2 the results will
77+return 2 first and then 1.
80+ urls <- S.toList $ aheadly $ getURL 2 |: getURL 1 |: S.nil
85+Use `runStream` instead of `toList` to run the actions but ignore the results:
88+ runStream $ parallely $ getURL 1 |: getURL 2 |: S.nil
91+### Concurrent Applicative
93+If the actions that you are executing result in different output types you can
94+use applicative zip to collect the results or to directly apply them to a
98+ tuples <- S.toList $ zipAsyncly $
99+ (,) <$> S.yieldM (getURLString 1) <*> S.yieldM (getURLText 2)
104+To run multiple actions and use the first result:
107+ urls <- S.toList $ S.take 1 $ parallely $ getURL 1 |: getURL 2 |: S.nil
110+After the first result arrives, the rest of the actions are canceled
111+automatically. You can take first `n` results as they arrive:
114+ urls <- S.toList $ S.take 2 $ parallely $ getURL 1 |: getURL 2 |: S.nil
119+There are many ways to map concurrently on a container and collect the results:
121+You can create a concurrent stream from a `Foldable` container of monadic
125+ urls <- S.toList $ aheadly $ S.fromFoldableM $ fmap getURL [1..3]
128+You can first convert a `Foldable` into a stream and then map an action on the
132+ urls <- S.toList $ aheadly $ S.mapM getURL $ foldMap return [1..3]
135+You can map a monadic action to a `Foldable` container to convert it into a
136+stream and at the same time fold it:
139+ urls <- S.toList $ aheadly $ foldMap (S.yieldM . getURL) [1..3]
142+### replicateConcurrently
144+Streamly has not just the equivalent of `replicateConcurrently` which is
145+`replicateM` but many more ways to generate concurrent streams, for example,
146+`|:`, `unfoldrM`, `repeatM`, `iterateM`, `fromFoldableM` etc. See the
147+[Streamly.Prelude](https://hackage.haskell.org/package/streamly/docs/Streamly-Prelude.html)
148+module documentation for more details.
151+ xs <- S.toList $ parallely $ S.replicateM 2 $ getURL 1
156+The stream resulting from concurrent actions can be mapped serially or
159+To map serially just use `fmap`:
162+ xs <- S.toList $ parallely $ fmap (+1) $ return 1 |: return 2 |: S.nil
165+To map a monadic action concurrently on all elements of the stream use `mapM`:
168+ xs <- S.toList $ parallely $ S.mapM (\x -> return (x + 1))
169+ $ return 1 |: return 2 |: S.nil
174+The `Semigroup` instances of streamly merge multiple streams serially or
179+The `Monad` instances of streamly nest loops concurrently (concurrent
184+Streamly has very little concurrency overhead (ranging from a few 100
185+nanoseconds to a few microseconds on a 2.2 GHz Intel Core i7), you can even run
186+very lightweight actions in parallel without worrying about the overhead of
187+concurrency. See the performance benchmarks [comparing streamly with the `async`
188+package in this repo](https://github.com/composewell/concurrency-benchmarks).
192+There is much more that you can do with streamly. For example, you can use the
193+`maxThreads` combinator to restrict the total number of concurrent threads or
194+use the `maxBuffer` combinator to restrict the total number of bufferred
195+results or you can use the `avgRate` combinator to control the rate at which
196+the concurrent actions are executed.
198+See the [haddock documentation on
199+hackage](https://hackage.haskell.org/package/streamly) and [a comprehensive tutorial
200+here](https://hackage.haskell.org/package/streamly/docs/Streamly-Tutorial.html).