@@ -8,7 +8,7 @@
88--
99-- Streamly, short for stream concurrently, combines the essence of
1010-- non-determinism, streaming and concurrency in functional programming.
11−-- Concurrent and non-concurrent applications are almost indistinguisable,
11+-- Concurrent and non-concurrent applications are almost indistinguishable,
1212-- concurrency capability does not at all impact the performance of
1313-- non-concurrent case.
1414-- Streaming enables writing modular, composable and scalable applications with
@@ -42,23 +42,26 @@ module Streamly.Tutorial
4242 -- ** Eliminating Streams
4343 -- $eliminating
4444
45− -- * Combining Streams
45+ -- * Transforming Streams
46+ -- $transformation
47+
48+ -- * Merging Streams
4649 -- $combining
4750
4851 -- ** Semigroup Style
4952 -- $semigroup
5053
51− -- *** Serial composition ('<>')
54+ -- *** Serial Composition ('streamly')
5255 -- $serial
5356
54− -- *** Async composition ('<|')
55− -- $parallel
56−
57− -- *** Interleaved composition ('<=>')
57+ -- *** Serial Interleaved Composition ('costreamly')
5858 -- $interleaved
5959
60− -- *** Fair Concurrent composition ('<|>')
61− -- $fairParallel
60+ -- *** Left Biased Concurrent Composition ('coparallely')
61+ -- $coparallel
62+
63+ -- *** Round Robin Concurrent Composition ('parallely')
64+ -- $parallel
6265
6366 -- *** Custom composition
6467 -- $custom
@@ -66,8 +69,8 @@ module Streamly.Tutorial
6669 -- ** Monoid Style
6770 -- $monoid
6871
69− -- * Transforming Streams
70− -- $transforming
72+ -- * Nesting Streams
73+ -- $nesting
7174
7275 -- ** Monad
7376 -- $monad
@@ -133,73 +136,120 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
133136-- $streams
134137--
135138-- Streamly provides many different stream types depending on the desired
136−-- composition style. The simplest type is 'StreamT'. 'StreamT' is a monad
137−-- transformer, the type @StreamT m a@ represents a stream of values of type
138−-- 'a' in some underlying monad 'm'. For example, @StreamT IO Int@ is a stream
139−-- of 'Int' in 'IO' monad.
139+-- composition style. The simplest type is 'Stream', @Stream a@ represents a
140+-- stream of elements of type @a@ in IO monad. To represent streams in an
141+-- arbitrary monad use the more general monad transformer type 'StreamT'.
142+-- @StreamT m a@ represents a stream of values of type 'a' in some underlying
143+-- monad 'm'. For example, @StreamT IO Int@ is a stream of 'Int' in 'IO' monad.
144+-- In fact, the type 'Stream' is a synonym for @StreamT IO@.
140145
141146-- $generating
142147--
143−-- Pure values can be placed into the stream type using 'return' or 'pure'.
144−-- Effects in the IO monad can be lifted to the stream type using the 'liftIO'
145−-- combinator. In a transformer stack we can lift actions from the lower monad
146−-- using the 'lift' combinator. Some examples of streams with a single element:
148+-- 'nil' represents an empty stream and 'consM' or its operator form '|:' adds
149+-- a monadic action at the head of the stream.
147150--
148151-- @
149−-- return 1 :: 'StreamT' IO Int
152+-- > toList nil
153+-- []
154+-- > toList $ getLine |: getLine |: nil
155+-- hello
156+-- world
157+-- ["hello","world"]
150158-- @
159+--
160+-- To create a singleton stream from a pure value use 'pure' and to create a
161+-- singleton stream from a monadic action use 'once'.
162+--
151163-- @
152−-- liftIO $ putStrLn "Hello world!" :: 'StreamT' IO ()
164+-- > toList $ pure 1
165+-- [1]
166+-- > toList $ once getLine
167+-- hello
168+-- ["hello"]
153169-- @
154170--
155−-- We can combine streams using '<>' to create streams of many elements:
171+-- To lift a value from an underlying monad in a monad transformer stack into a
172+-- singleton stream use 'lift' and to lift from an IO action use 'liftIO'.
156173--
157174-- @
158−-- return 1 <> return 2 <> return 3 :: 'StreamT' IO Int
175+-- > runStream $ liftIO $ putStrLn "Hello world!"
176+-- Hello world!
177+-- > runStream $ lift $ putStrLn "Hello world!"
178+-- Hello world!
159179-- @
160180--
161−-- For more ways to construct or generate a stream see the module
162−-- "Streamly.Prelude".
163−
164−-- $eliminating
165−--
166−-- 'runStreamT' runs a composed 'StreamT' computation, lowering the type into
167−-- the underlying monad and discarding the result stream:
181+-- To create a stream of pure values from a 'Foldable' container use
182+-- 'fromFoldable'.
168183--
169184-- @
170−-- import "Streamly"
171−--
172−-- main = 'runStreamT' $ liftIO $ putStrLn "Hello world!"
185+-- > toList $ fromFoldable [1..3]
186+-- [1,2,3]
173187-- @
174188--
175−-- 'toList' runs a stream computation and collects the result stream in a list
176−-- in the underlying monad. 'toList' is a polymorphic function that works on
177−-- multiple stream types belonging to the class 'IsStream'. Therefore, before
178−-- you run a stream you need to tell how you want to interpret the stream by
179−-- using one of the stream type combinators ('serially', 'asyncly', 'parallely'
180−-- etc.). The combinator 'serially' is equivalent to annotating the type as @::
181−-- StreamT@.
189+-- To create a stream from monadic actions in a 'Foldable' container just use a
190+-- right fold:
182191--
183192-- @
184−-- import "Streamly"
185−-- import "Streamly.Prelude"
186−--
187−-- main = do
188−-- xs \<- 'toList' $ 'serially' $ return 1 <> return 2
189−-- print xs
193+-- > runStream $ Prelude.foldr (|:) nil [putStr "Hello ", putStrLn "world!"]
194+-- Hello world!
190195-- @
191196--
192−-- For other ways to eliminate or fold a stream see the module
193−-- "Streamly.Prelude".
197+-- For more ways to construct a stream see the module "Streamly.Prelude".
198+
199+-- $eliminating
200+--
201+-- We have already seen 'runStream' and 'toList' to eliminate a stream in the
202+-- examples above. 'runStream' runs a stream discarding the results i.e. only
203+-- for effects. 'toList' runs the stream and collects the results in a list.
204+--
205+-- For other ways to eliminate a stream see the @Folding@ section in
206+-- "Streamly.Prelude" module.
207+
208+-- $transformation
209+--
210+-- Transformation over a stream is the equivalent of a @for@ loop construct in
211+-- imperative paradigm. We iterate over every element in the stream and perform
212+-- certain transformations for each element. Transformations may involve
213+-- mapping functions over the elements, filtering elements from the stream or
214+-- folding all the elements in the stream into a single value. Streamly streams
215+-- are exactly like lists and you can perform all the transformations in the
216+-- same way as you would on lists. For example the following snippet reads
217+-- lines from standard input, filters blank lines, drops the first non-blank
218+-- line, takes the next two, up cases them, numbers them and prints them:
219+--
220+-- @
221+-- import Streamly
222+-- import qualified Streamly.Prelude as S
223+-- import Data.Char (toUpper)
224+-- import Data.Function ((&))
225+-- import Data.Semigroup (cycle1)
226+--
227+-- main = runStream $ cycle1 (S.once getLine)
228+-- & S.filter (not . null)
229+-- & S.drop 1
230+-- & S.take 2
231+-- & fmap (map toUpper)
232+-- & S.zipWith (\\n s -> show n ++ " " ++ s) (S.fromFoldable [1..])
233+-- & S.mapM putStrLn
234+-- @
194235
195236-- $semigroup
196−-- Streams of the same type can be combined into a composite stream in many
197−-- different ways using one of the semigroup style binary composition operators
198−-- i.e. '<>', '<=>', '<|', '<|>', 'mplus'. These operators work on all stream
199−-- types ('StreamT', 'AsyncT' etc.) uniformly.
200237--
201−-- To illustrate the concurrent aspects, we will use the following @delay@
202−-- function to introduce a delay specified in seconds.
238+-- We can combine two streams into a single stream using semigroup composition
239+-- operation '<>'. Streams can be combined in many different ways as described
240+-- in the following sections, the '<>' operation behaves differently depending
241+-- on the stream type in effect. Streamly streams can be of many
242+-- different types i.e. 'Stream', 'Costream', 'Coparallel', 'Parallel',
243+-- 'ZipStream', and 'ZipParallel'. The first four are monads whereas the last
244+-- two are applicatives. These types can be freely converted from one to
245+-- another to acheive the desired composition style. To force a particular
246+-- type of composition we coerce the stream type using the corresponding type
247+-- adapting combinator from 'streamly', 'costreamly', 'coparallely',
248+-- 'parallely', 'zipStreamly' or 'zipParallely'. The default stream type is
249+-- 'Stream' unless you change it by using one of the combinators.
250+--
251+-- To illustrate the concurrent vs serial composition aspects, we will use the
252+-- following @delay@ function to introduce a delay specified in seconds.
203253--
204254-- @
205255-- import "Streamly"
@@ -213,48 +263,157 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
213263
214264-- $serial
215265--
216−-- We have already seen, the '<>' operator. It composes two streams in series
217−-- i.e. the first stream is completely exhausted and then the second stream is
218−-- processed. The following example prints the sequence 3, 2, 1 and takes a
219−-- total of 6 seconds because everything is serial:
266+-- The 'Semigroup' operation '<>' of the 'Stream' type combines the two streams
267+-- in a serial manner. We use the 'streamly' type combinator to effect 'Stream'
268+-- style of composition. We can also use the 'Stream' type annotation for the
269+-- stream type to acheive the same effect. However, since 'Stream' is the
270+-- default type unless explicitly specified by using a combinator, we can
271+-- omit using an explicit combinator or type annotation for this style of
272+-- composition.
273+--
274+-- When two streams with multiple elements are combined in this manner, the
275+-- monadic actions in the two streams are performed sequentially i.e. first all
276+-- actions in the first stream are performed sequentially and then all actions
277+-- in the second stream are performed sequentially. The following example
278+-- prints the sequence 1, 2, 3, 4:
279+--
280+-- @
281+-- main = 'runStream' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
282+-- @
283+-- @
284+-- 1
285+-- 2
286+-- 3
287+-- 4
288+-- @
289+--
290+-- All actions in both the streams are performed serially in the same thread.
291+-- In the following example we can see that all actions are performed in the
292+-- same thread and take a combined total of 6 seconds:
220293--
221294-- @
222−-- main = 'runStreamT' $ delay 3 <> delay 2 <> delay 1
295+-- main = 'runStream' $ delay 3 <> delay 2 <> delay 1
223296-- @
224297-- @
225298-- ThreadId 36: Delay 3
226299-- ThreadId 36: Delay 2
227300-- ThreadId 36: Delay 1
228301-- @
302+--
303+-- The polymorphic version of the 'Stream' binary operation '<>' is called
304+-- 'splice'. We can use 'splice' to join streams in a sequential manner
305+-- irrespective of the type of stream:
306+--
307+-- @
308+-- main = 'runStream' $ (print 1 |: print 2 |: nil) \`splice` (print 3 |: print 4 |: nil)
309+-- @
229310
230311-- $interleaved
231−-- The '<=>' operator is serial like '<>' but it interleaves the two streams
232−-- i.e. it yields one element from the first stream and then one element from
233−-- the second stream, and so on. The following example prints the sequence 1,
234−-- 3, 2, 4 and takes a total of 10 seconds because everything is serial:
312+--
313+-- The 'Semigroup' operation '<>' of the 'Costream' type combines the two
314+-- streams in a serial interleaved manner. We use the 'costreamly' type
315+-- combinator to effect 'Costream' style of composition. We can also use the
316+-- 'Costream' type annotation for the stream type to acheive the same effect.
317+--
318+-- When two streams with multiple elements are combined in this manner, the
319+-- monadic actions in the two streams are performed in an interleaved manner
320+-- i.e. one action from the first stream is performed and its output yielded
321+-- and then one action from the second stream is performed and its output
322+-- yielded, and so on. The following example prints the sequence 1, 3, 2, 4
235323--
236324-- @
237−-- main = 'runStreamT' $ (delay 1 <> delay 2) '<=>' (delay 3 <> delay 4)
325+-- main = 'runStream' . 'costreamly' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
326+-- @
327+-- @
328+-- 1
329+-- 3
330+-- 2
331+-- 4
332+-- @
333+--
334+-- Even though the monadic actions of the two streams are performed in an
335+-- interleaved manner they are all performed serially in the same thread. In
336+-- the following example we can see that all actions are performed in the same
337+-- thread and take a combined total of 6 seconds:
338+--
339+-- @
340+-- main = 'runStream' . 'costreamly' $ delay 3 <> delay 2 <> delay 1
238341-- @
239342-- @
240−-- ThreadId 36: Delay 1
241343-- ThreadId 36: Delay 3
242344-- ThreadId 36: Delay 2
243−-- ThreadId 36: Delay 4
345+-- ThreadId 36: Delay 1
346+-- @
347+--
348+-- The polymorphic version of the 'Costream' binary operation '<>' is called
349+-- 'cosplice'. We can use 'cosplice' to join streams in an interleaved manner
350+-- irrespective of the type, notice that we have not used the 'costreamly'
351+-- combinator in the following example:
352+--
353+-- @
354+-- main = 'runStream' $ (print 1 |: print 2 |: nil) \`cosplice` (print 3 |: print 4 |: nil)
355+-- @
356+-- @
357+-- 1
358+-- 3
359+-- 2
360+-- 4
244361-- @
245362--
246−-- Note that this operator cannot be used to fold infinite containers since it
247−-- requires preserving the state until a stream is finished. To be clear, it
248−-- can combine infinite streams but not infinite number of streams.
363+-- Note that this composition cannot be used to fold infinite number of streams
364+-- since it requires preserving the state until a stream is finished.
249365
250−-- $parallel
366+-- $coparallel
367+--
368+-- The 'Semigroup' operation '<>' of the 'Coparallel' type combines the two
369+-- streams in a left biased concurrent manner. We use the 'coparallel' type
370+-- combinator to effect 'Coparallel' style of composition. We can also use the
371+-- 'Coparallel' type annotation for the stream type to acheive the same effect.
251372--
252−-- The '<|' operator can run both computations concurrently, /when needed/.
253−-- In the following example since the first computation blocks we start the
254−-- next one in a separate thread and so on:
373+-- When two streams with multiple elements are combined in this manner, the
374+-- monadic actions in the two streams can be performed concurrently /when
375+-- needed/ with a bias towards the left stream i.e. actions from the left
376+-- stream are performed sequentially, if the left stream blocks or cannot
377+-- generate enough output to keep the consumer of the stream busy then it can
378+-- execute actions from the right stream concurrently. The outputs of the
379+-- concurrent actions are yielded by the composite stream in the order in which
380+-- they complete. This can be thought of as the concurrent analogue of the
381+-- 'Stream' style serial composition. If concurrent execution is not trigerred
382+-- by consumer demand then it would execute actions exactly in the same order
383+-- as 'Stream'. In the following example the first stream does not block,
384+-- therefore the first stream is completely exhausted before the second.
255385--
256386-- @
257−-- main = 'runStreamT' $ delay 3 '<|' delay 2 '<|' delay 1
387+-- main = 'runStream' . 'coparallely' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
388+-- @
389+-- @
390+-- 1
391+-- 2
392+-- 3
393+-- 4
394+-- @
395+--
396+-- If the first stream blocks, we can yield from the second. In the example
397+-- below each yield in the stream has a constant delay of 1 second therefore 1
398+-- and 3 would be yielded first and then 2 and 4 would be yielded.
399+--
400+-- @
401+-- main = 'runStream' . 'coparallely' $ (p 1 |: p 2 |: nil) <> (p 3 |: p 4 |: nil)
402+-- where p n = threadDelay 1000000 >> print n
403+-- @
404+-- @
405+-- 1
406+-- 3
407+-- 2
408+-- 4
409+-- @
410+--
411+-- In the following example we can see that new threads are started when a
412+-- computation blocks. Notice that the output from the stream with the
413+-- shortest delay is printed first:
414+--
415+-- @
416+-- main = 'runStream' . 'coparallely' $ delay 3 '<>' delay 2 '<>' delay 1
258417-- @
259418-- @
260419-- ThreadId 42: Delay 1
@@ -262,14 +421,18 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
262421-- ThreadId 40: Delay 3
263422-- @
264423--
265−-- This is the concurrent version of the '<>' operator. The computations are
266−-- triggered in the same order as '<>' except that they are concurrent. When
267−-- we have a tree of computations composed using this operator, the tree is
268−-- traversed in DFS style just like '<>'.
424+-- When we have a tree of computations composed using this style, the tree is
425+-- traversed in DFS style just like the serial style, the only difference is
426+-- that here we can move on to executing the next stream if a stream blocks.
427+-- However, we will not start new threads if we have sufficient output to
428+-- saturate the consumer. This is why we call it left-biased demand driven or
429+-- adaptive concurrency style, the concurrency tends to stay on the left side
430+-- of the composition as long as possible. More threads are started based on
431+-- the pull rate of the consumer. The following example prints an output every
432+-- second as all of the actions are concurrent.
269433--
270434-- @
271−-- main = 'runStreamT' $ (p 1 '<|' p 2) '<|' (p 3 '<|' p 4)
272−-- where p = liftIO . print
435+-- main = 'runStream' . 'coparallely' $ (delay 1 <> delay 2) <> (delay 3 <> delay 4)
273436-- @
274437-- @
275438-- 1
@@ -278,21 +441,16 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
278441-- 4
279442-- @
280443--
281−-- Concurrency provided by this operator is demand driven. The second
282−-- computation is run concurrently with the first only if the first computation
283−-- is not producing enough output to keep the stream consumer busy otherwise
284−-- the second computation is run serially after the previous one. The number of
285−-- concurrent threads is adapted dynamically based on the pull rate of the
286−-- consumer of the stream.
287−-- As you can see, in the following example the computations are run in a
288−-- single thread one after another, because none of them blocks. However, if
289−-- the thread consuming the stream were faster than the producer then it would
290−-- have started parallel threads for each computation to keep up even if none
291−-- of them blocks:
444+-- All the computations may even run in a single thread when more threads are
445+-- not needed. As you can see, in the following example the computations are
446+-- run in a single thread one after another, because none of them blocks.
447+-- However, if the thread consuming the stream were faster than the producer
448+-- then it would have started parallel threads for each computation to keep up
449+-- even if none of them blocks:
292450--
293451-- @
294−-- main = 'runStreamT' $ traced (sqrt 9) '<|' traced (sqrt 16) '<|' traced (sqrt 25)
295−-- where traced m = liftIO (myThreadId >>= print) >> m
452+-- main = 'runStream' . 'coparallely' $ traced (sqrt 9) '<>' traced (sqrt 16) '<>' traced (sqrt 25)
453+-- where traced m = once (myThreadId >>= print) >> return m
296454-- @
297455-- @
298456-- ThreadId 40
@@ -300,46 +458,101 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
300458-- ThreadId 40
301459-- @
302460--
461+-- Note that the order of printing in the above examples may change due to
462+-- variations in scheduling latencies for concurrent threads.
463+--
464+-- The polymorphic version of the 'Coparallel' binary operation '<>' is called
465+-- 'coparallel'. We can use 'coparallel' to join streams in a left biased
466+-- adaptively concurrent manner irrespective of the type, notice that we have
467+-- not used the 'coparallely' combinator in the following example:
468+--
469+-- @
470+-- main = 'runStream' $ delay 3 \`coparallel` delay 2 \`coparallel` delay 1
471+-- @
472+-- @
473+-- ThreadId 42: Delay 1
474+-- ThreadId 41: Delay 2
475+-- ThreadId 40: Delay 3
476+-- @
477+--
303478-- Since the concurrency provided by this operator is demand driven it cannot
304−-- be used when the composed computations have timers that are relative to each
305−-- other because all computations may not be started at the same time and
479+-- be used when the composed computations start timers that are relative to
480+-- each other because all computations may not be started at the same time and
306481-- therefore timers in all of them may not start at the same time. When
307482-- relative timing among all computations is important or when we need to start
308−-- all computations at once for some reason '<|>' must be used instead.
309−-- However, '<|' is useful in situations when we want to optimally utilize the
310−-- resources and we know that the computations can run in parallel but we do
311−-- not care if they actually run in parallel or not, that decision is left to
312−-- the scheduler. Also, note that this operator can be used to fold infinite
313−-- containers in contrast to '<|>', because it does not require us to run all
314−-- of them at the same time.
315−--
316−-- The left bias (or the DFS style) of the operator '<|' is suggested by its
317−-- shape. You can also think of this as an unbalanced version of the fairly
318−-- parallel operator '<|>'.
483+-- all computations at once for any reason 'Parallel' style must be used
484+-- instead.
485+--
486+-- 'Coparallel' style should be preferred over 'Parallel' unless you really
487+-- need 'Parallel'. It utilizes the resources optimally. It should be used when
488+-- we know that the computations can run in parallel but we do not care if they
489+-- actually run in parallel or not, that decision can be left to the scheduler
490+-- based on demand. Also, note that this operator can be used to fold infinite
491+-- number of streams in contrast to the 'Parallel' style, because it does not
492+-- require us to run all of them at the same time in a fair manner.
319493
320−-- $fairParallel
494+-- $parallel
321495--
322−-- The 'Alternative' composition operator '<|>', like '<|', runs the composed
323−-- computations concurrently. However, unlike '<|' it runs all of the
324−-- computations in fairly parallel manner using a round robin scheduling
325−-- mechanism. This can be considered as the concurrent version of the fairly
326−-- interleaved serial operation '<=>'. Note that this cannot be used on
327−-- infinite containers, as it will lead to an infinite sized scheduling queue.
496+-- The 'Semigroup' operation '<>' of the 'Parallel' type combines the two
497+-- streams in a fairly concurrent manner with round robin scheduling. We use
498+-- the 'parallel' type combinator to effect 'Parallel' style of composition. We
499+-- can also use the 'Parallel' type annotation for the stream type to acheive
500+-- the same effect.
328501--
329−-- The following example sends a query to three search engines in parallel and
330−-- prints the name of the search engine as a response arrives:
502+-- When two streams with multiple elements are combined in this manner, the
503+-- monadic actions in both the streams are performed concurrently with a fair
504+-- round robin scheduling. The outputs are yielded in the order in which the
505+-- actions complete. This can be thought of as the concurrent analogue of the
506+-- 'Costream' style serial composition.
507+--
508+-- The following example sends a query to all the three search engines in
509+-- parallel and prints the name of the search engines in the order in which the
510+-- responses arrive:
331511--
332512-- @
333513-- import "Streamly"
514+-- import Streamly.Prelude (once)
334515-- import Network.HTTP.Simple
335516--
336−-- main = 'runStreamT' $ google \<|> bing \<|> duckduckgo
517+-- main = 'runStream' . 'parallely' $ google \<> bing \<> duckduckgo
337518-- where
338519-- google = get "https://www.google.com/search?q=haskell"
339520-- bing = get "https://www.bing.com/search?q=haskell"
340521-- duckduckgo = get "https://www.duckduckgo.com/?q=haskell"
341−-- get s = liftIO (httpNoBody (parseRequest_ s) >> putStrLn (show s))
522+-- get s = once (httpNoBody (parseRequest_ s) >> putStrLn (show s))
523+-- @
524+--
525+-- In the following example we can see that the fairly parallel composition
526+-- yields outputs in the same order as the 'Costream' composition, though this
527+-- is not always guaranteed because of thread scheduling variances.
528+--
342529-- @
530+-- main = 'runStream' . 'parallely' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
531+-- @
532+-- @
533+-- 1
534+-- 3
535+-- 2
536+-- 4
537+-- @
538+--
539+-- The polymorphic version of the 'Parallel' binary operation '<>' is called
540+-- 'parallel'. We can use 'parallel' to join streams in a fairly
541+-- concurrent manner irrespective of the type, notice that we have
542+-- not used the 'parallely' combinator in the following example:
543+--
544+-- @
545+-- main = 'runStream' $ delay 3 \`parallel` delay 2 \`parallel` delay 1
546+-- @
547+-- @
548+-- ThreadId 42: Delay 1
549+-- ThreadId 41: Delay 2
550+-- ThreadId 40: Delay 3
551+-- @
552+--
553+-- Note that this style of composition cannot be used to combine infinite
554+-- number of streams, as it will lead to an infinite sized scheduling queue.
555+--
343556
344557-- $custom
345558--
@@ -354,31 +567,35 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
354567
355568-- $monoid
356569--
357−-- Each of the semigroup compositions described has an identity that can be
358−-- used to fold a possibly empty container. An empty stream is represented by
359−-- 'nil' which can be represented in various standard forms as 'mempty',
360−-- 'empty' or 'mzero'.
361−-- Some fold utilities are also provided by the library for convenience:
570+-- We can use 'Monoid' instances to fold a container of streams in the desired
571+-- style using 'fold' or 'foldMap'. We have also provided some fold utilities
572+-- to fold streams using the polymorphic combine operations:
362573--
363−-- * 'foldWith' folds a 'Foldable' container of stream computations using the
364−-- given composition operator.
365−-- * 'foldMapWith' folds like foldWith but also maps a function before folding.
366−-- * 'forEachWith' is like foldMapwith but the container argument comes before
574+-- * 'foldWith' is like 'fold', it folds a 'Foldable' container of streams
575+-- using the given composition operator.
576+-- * 'foldMapWith' is like 'foldMap', it folds like @foldWith@ but also maps a
577+-- function before folding.
578+-- * 'forEachWith' is like @foldMapwith@ but the container argument comes before
367579-- the function argument.
368580--
369−-- All of the following are equivalent:
581+-- All of the following are equivalent and start ten concurrent tasks each with
582+-- a delay from 1 to 10 seconds, resulting in the printing of each number every
583+-- second:
370584--
371585-- @
372586-- import "Streamly"
373587-- import "Streamly.Prelude"
588+-- import Control.Concurrent
374589--
375590-- main = do
376−-- 'toList' . 'serially' $ 'foldWith' (<>) (map return [1..10]) >>= print
377−-- 'toList' . 'serially' $ 'foldMapWith' (<>) return [1..10] >>= print
378−-- 'toList' . 'serially' $ 'forEachWith' (<>) [1..10] return >>= print
591+-- runStream $ parallely $ foldMap delay [1..10]
592+-- runStream $ 'foldWith' parallel (map delay [1..10])
593+-- runStream $ 'foldMapWith' parallel delay [1..10]
594+-- runStream $ 'forEachWith' parallel [1..10] delay
595+-- where delay n = once $ threadDelay (n * 1000000) >> print n
379596-- @
380597
381−-- $transforming
598+-- $nesting
382599--
383600-- The previous section discussed ways to merge the elements of two streams
384601-- without doing any transformation on them. In this section we will explore