@@ -75,16 +75,16 @@ module Streamly.Tutorial
7575 -- ** Monad
7676 -- $monad
7777
78− -- *** Serial Composition ('StreamT')
78+ -- *** Serial Nesting ('streamly')
7979 -- $regularSerial
8080
81− -- *** Async Composition ('AsyncT')
82− -- $concurrentNesting
83−
84− -- *** Interleaved Composition ('InterleavedT')
81+ -- *** Serial Interleaved Nesting ('costreamly')
8582 -- $interleavedNesting
8683
87− -- *** Fair Concurrent Composition ('ParallelT')
84+ -- *** Left Biased Concurrent Nesting ('coparallely')
85+ -- $concurrentNesting
86+
87+ -- *** Round Robin Concurrent Nesting ('parallely')
8888 -- $fairlyConcurrentNesting
8989
9090 -- *** Exercise
@@ -108,6 +108,9 @@ module Streamly.Tutorial
108108 -- * Summary of Compositions
109109 -- $compositionSummary
110110
111+ -- * Monad transformers
112+ -- $monadtransformers
113+
111114 -- * Concurrent Programming
112115 -- $concurrent
113116
@@ -137,11 +140,33 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
137140--
138141-- Streamly provides many different stream types depending on the desired
139142-- composition style. The simplest type is 'Stream', @Stream a@ represents a
140−-- stream of elements of type @a@ in IO monad. To represent streams in an
141−-- arbitrary monad use the more general monad transformer type 'StreamT'.
142−-- @StreamT m a@ represents a stream of values of type 'a' in some underlying
143−-- monad 'm'. For example, @StreamT IO Int@ is a stream of 'Int' in 'IO' monad.
144−-- In fact, the type 'Stream' is a synonym for @StreamT IO@.
143+-- stream of elements of type @a@ in IO monad. Streams are very much like lists
144+-- the only difference is that they are lists of monadic actions instead of
145+-- pure values. We can call them monadic lists. You can apply all the
146+-- constructs related to lists to monadic streams as well.
147+
148+-- $monadtransformers
149+--
150+-- To represent streams in an arbitrary monad use the more general monad
151+-- transformer types for example the monad transformer type corresponding to
152+-- the 'Stream' type is 'StreamT'. @StreamT m a@ represents a stream of values
153+-- of type 'a' in some underlying monad 'm'. For example, @StreamT IO Int@ is a
154+-- stream of 'Int' in 'IO' monad. In fact, the type 'Stream' is a synonym for
155+-- @StreamT IO@.
156+--
157+-- Similarly we have monad transformer types for other stream types as well viz.
158+-- 'CostreamT', 'CoparallelT' and 'ParallelT'.
159+--
160+-- To lift a value from an underlying monad in a monad transformer stack into a
161+-- singleton stream use 'lift' and to lift from an IO action use 'liftIO'.
162+--
163+-- @
164+-- > runStream $ liftIO $ putStrLn "Hello world!"
165+-- Hello world!
166+-- > runStream $ lift $ putStrLn "Hello world!"
167+-- Hello world!
168+-- @
169+--
145170
146171-- $generating
147172--
@@ -168,26 +193,18 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
168193-- ["hello"]
169194-- @
170195--
171−-- To lift a value from an underlying monad in a monad transformer stack into a
172−-- singleton stream use 'lift' and to lift from an IO action use 'liftIO'.
173−--
174−-- @
175−-- > runStream $ liftIO $ putStrLn "Hello world!"
176−-- Hello world!
177−-- > runStream $ lift $ putStrLn "Hello world!"
178−-- Hello world!
179−-- @
180−--
181−-- To create a stream of pure values from a 'Foldable' container use
182−-- 'fromFoldable'.
196+-- To create a stream from pure values in a 'Foldable' container use
197+-- 'fromFoldable' which is equivalent to a fold using 'cons' and 'nil':
183198--
184199-- @
185200-- > toList $ fromFoldable [1..3]
186201-- [1,2,3]
202+-- > toList $ Prelude.foldr cons nil [1..3]
203+-- [1,2,3]
187204-- @
188205--
189206-- To create a stream from monadic actions in a 'Foldable' container just use a
190−-- right fold:
207+-- right fold using 'consM' and 'nil':
191208--
192209-- @
193210-- > runStream $ Prelude.foldr (|:) nil [putStr "Hello ", putStrLn "world!"]
@@ -213,9 +230,18 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
213230-- mapping functions over the elements, filtering elements from the stream or
214231-- folding all the elements in the stream into a single value. Streamly streams
215232-- are exactly like lists and you can perform all the transformations in the
216−-- same way as you would on lists. For example the following snippet reads
217−-- lines from standard input, filters blank lines, drops the first non-blank
218−-- line, takes the next two, up cases them, numbers them and prints them:
233+-- same way as you would on lists.
234+--
235+-- Here is a simple console echo program that just echoes every input line,
236+-- forever:
237+--
238+-- @
239+-- > runStream $ cycle1 (S.once getLine) & S.mapM putStrLn
240+-- @
241+--
242+-- The following code snippet reads lines from standard input, filters blank
243+-- lines, drops the first non-blank line, takes the next two, up cases them,
244+-- numbers them and prints them:
219245--
220246-- @
221247-- import Streamly
@@ -253,9 +279,10 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
253279--
254280-- @
255281-- import "Streamly"
282+-- import "Streamly.Prelude"(once)
256283-- import Control.Concurrent
257284--
258−-- delay n = liftIO $ do
285+-- delay n = once $ do
259286-- threadDelay (n * 1000000)
260287-- tid \<- myThreadId
261288-- putStrLn (show tid ++ ": Delay " ++ show n)
@@ -371,8 +398,8 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
371398-- 'Coparallel' type annotation for the stream type to acheive the same effect.
372399--
373400-- When two streams with multiple elements are combined in this manner, the
374−-- monadic actions in the two streams can be performed concurrently /when
375−-- needed/ with a bias towards the left stream i.e. actions from the left
401+-- monadic actions in the two streams can be performed concurrently
402+-- /when needed/ with a bias towards the left stream i.e. actions from the left
376403-- stream are performed sequentially, if the left stream blocks or cannot
377404-- generate enough output to keep the consumer of the stream busy then it can
378405-- execute actions from the right stream concurrently. The outputs of the
@@ -597,39 +624,50 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
597624
598625-- $nesting
599626--
600−-- The previous section discussed ways to merge the elements of two streams
601−-- without doing any transformation on them. In this section we will explore
602−-- how to transform streams using 'Functor', 'Applicative' or 'Monad' style
603−-- compositions. The applicative and monad compositions of all 'IsStream' types
604−-- behave exactly the same way as a list transformer. For simplicity of
605−-- illustration we are using streams of pure values in the following examples.
606−-- However, the real application of streams arises when these streams are
607−-- generated using monadic actions.
608−
627+-- Till now we discussed ways to apply transformations on a stream or to merge
628+-- streams together to create another stream. We mentioned earlier that
629+-- transforming a stream is similar to a @for@ loop in the imperative paradigm.
630+-- We will now discuss the concept of a nested composition of streams which is
631+-- analogous to nested @for@ loops in the imperative paradigm. Functional
632+-- programmers call this style of composition a list transformer or @ListT@.
633+-- Logic programmers call it a logic monad or non-deterministic composition,
634+-- but for ordinary imperative minded people like me it is easier to think in
635+-- terms of good old nested @for@ loops.
636+--
609637-- $monad
610638--
611−-- In functional programmer's parlance the 'Monad' instance of 'IsStream'
612−-- types implement non-determinism, exploring all possible combination of
613−-- choices from both the streams. From an imperative programmer's point of view
614−-- it behaves like nested loops i.e. for each element in the first stream and
615−-- for each element in the second stream apply the body of the loop. If you are
616−-- familiar with list transformer this behavior is exactly the same as that of
617−-- a list transformer.
618−--
619−-- Just like we saw in sum style compositions earlier, monadic composition also
620−-- has multiple variants each of which exactly corresponds to one of the sum
621−-- style composition variant.
639+-- In functional programmer's parlance the 'Monad' instances of different
640+-- 'IsStream' types implement non-determinism, exploring all possible
641+-- combination of choices from both the streams. From an imperative
642+-- programmer's point of view it behaves like nested loops i.e. for each
643+-- element in the first stream and for each element in the second stream
644+-- execute the body of the loop.
645+--
646+-- The 'Monad' instances of 'Stream', 'Costream', 'Coparallel' and 'Parallel'
647+-- stream types support different flavors of nested looping. In other words,
648+-- they are all variants of list transformer. The nesting behavior of these
649+-- types exactly correspond to the way they merge streams as we discussed in
650+-- the previous section.
651+--
622652
623653-- $regularSerial
624654--
625−-- When we interpret the monadic composition as 'StreamT' we get a standard
626−-- list transformer like serial composition.
655+-- The 'Monad' composition of 'Stream' type behaves like a standard list
656+-- transformer. This is the default when we do not use an explicit type
657+-- combinator. However, the 'streamly' type combinator can be used to switch to
658+-- this style of composition. We will see how this style of composition works
659+-- in the following examples.
660+--
661+-- Let's start with an example with a simple @for@ loop without any nesting.
662+-- For simplicity of illustration we are using streams of pure values in all
663+-- the examples. However, the streams could also be made of monadic actions
664+-- instead.
627665--
628666-- @
629667-- import "Streamly"
630668-- import "Streamly.Prelude"
631669--
632−-- main = 'runStreamT' $ do
670+-- main = 'runStream' $ do
633671-- x <- 'fromFoldable' [3,2,1]
634672-- delay x
635673-- @
@@ -639,34 +677,33 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
639677-- ThreadId 30: Delay 1
640678-- @
641679--
642−-- As you can see the code after the @fromFoldable@ statement is run three
643−-- times, once for each value of @x@. All the three iterations are serial and
644−-- run in the same thread one after another. When compared to imperative
645−-- programming, this can also be viewed as a @for@ loop with three iterations.
680+-- As you can see, the code after the @fromFoldable@ statement is run three
681+-- times, once for each value of @x@ drawn from the stream. All the three
682+-- iterations are serial and run in the same thread one after another. In
683+-- imperative terms this is equivalent to a @for@ loop with three iterations.
646684--
647685-- A console echo loop copying standard input to standard output can simply be
648686-- written like this:
649687--
650688-- @
651689-- import "Streamly"
652−-- import Data.Semigroup (cycle1)
653690--
654−-- main = 'runStreamT' $ cycle1 (liftIO getLine) >>= liftIO . putStrLn
691+-- main = 'runStream' $ forever $ once getLine >>= once . putStrLn
655692-- @
656693--
657694-- When multiple streams are composed using this style they nest in a DFS
658−-- manner i.e. nested iterations of an iteration are executed before we proceed
659−-- to the next iteration at higher level. This behaves just like nested @for@
695+-- manner i.e. nested iterations of a loop are executed before we proceed to
696+-- the next iteration of the parent loop. This behaves just like nested @for@
660697-- loops in imperative programming.
661698--
662699-- @
663700-- import "Streamly"
664701-- import "Streamly.Prelude"
665702--
666−-- main = 'runStreamT' $ do
703+-- main = 'runStream' $ do
667704-- x <- 'fromFoldable' [1,2]
668705-- y <- 'fromFoldable' [3,4]
669−-- liftIO $ putStrLn $ show (x, y)
706+-- once $ putStrLn $ show (x, y)
670707-- @
671708-- @
672709-- (1,3)
@@ -675,23 +712,27 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
675712-- (2,4)
676713-- @
677714--
678−-- You will also notice that this is the monadic equivalent of the sum style
679−-- composition using '<>'.
715+-- Notice that this is analogous to merging streams of type 'Stream' or merging
716+-- streams using 'splice'.
680717
681718-- $concurrentNesting
682719--
683−-- When we interpret the monadic composition as 'AsyncT' we get a /concurrent/
684−-- list-transformer like composition. Multiple monadic continuations (or loop
685−-- iterations) may be started concurrently. Concurrency is demand driven
686−-- i.e. more concurrent iterations are started only if the previous iterations
687−-- are not able to produce enough output for the consumer of the output stream.
688−-- This is the concurrent version of 'StreamT'.
720+-- The 'Monad' composition of 'Coparallel' type can perform the iterations of a
721+-- loop concurrently. Concurrency is demand driven i.e. more concurrent
722+-- iterations are started only if the previous iterations are not able to
723+-- produce enough output for the consumer of the output stream. This works
724+-- exactly the same way as the merging of two streams 'coparallely' works.
725+-- This is the concurrent analogue of 'Stream' style monadic composition.
726+--
727+-- The 'coparallely' type combinator can be used to switch to this
728+-- style of composition. Alternatively, a type annotation can be used to
729+-- specify the type of the stream as 'Coparallel'.
689730--
690731-- @
691732-- import "Streamly"
692733-- import "Streamly.Prelude"
693734--
694−-- main = 'runAsyncT' $ do
735+-- main = 'runStream' . 'coparallely' $ do
695736-- x <- 'fromFoldable' [3,2,1]
696737-- delay x
697738-- @
@@ -701,27 +742,27 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
701742-- ThreadId 38: Delay 3
702743-- @
703744--
704−-- As you can see the code after the @fromFoldable'@ statement is run three
745+-- As you can see the code after the @fromFoldable@ statement is run three
705746-- times, once for each value of @x@. All the three iterations are concurrent
706747-- and run in different threads. The iteration with least delay finishes first.
707748-- When compared to imperative programming, this can be viewed as a @for@ loop
708749-- with three concurrent iterations.
709750--
710−-- Concurrency is demand driven just as in the case of '<|'. When multiple
711−-- streams are composed using this style the iterations are triggered in a DFS
712−-- manner just like 'StreamT' i.e. nested iterations are executed before we
713−-- proceed to the next iteration at higher level. However, unlike 'StreamT'
714−-- more than one iterations may be started concurrently, and based on the
715−-- demand from the consumer.
751+-- Concurrency is demand driven just as in the case of 'coparallel' merging.
752+-- When multiple streams are composed using this style, the iterations are
753+-- triggered in a DFS manner just like 'Stream' i.e. nested iterations are
754+-- executed before we proceed to the next iteration at higher level. However,
755+-- unlike 'Stream' more than one iterations may be started concurrently based
756+-- on the demand from the consumer.
716757--
717758-- @
718759-- import "Streamly"
719760-- import "Streamly.Prelude"
720761--
721−-- main = 'runAsyncT' $ do
762+-- main = 'runStream' . 'coparallely' $ do
722763-- x <- 'fromFoldable' [1,2]
723764-- y <- 'fromFoldable' [3,4]
724−-- liftIO $ putStrLn $ show (x, y)
765+-- once $ putStrLn $ show (x, y)
725766-- @
726767-- @
727768-- (1,3)
@@ -729,25 +770,24 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
729770-- (2,3)
730771-- (2,4)
731772-- @
732−--
733−-- You will notice that this is the monadic equivalent of the '<|' style
734−-- sum composition. The same caveats apply to this as the '<|' operation.
735773
736774-- $interleavedNesting
737775--
738−-- When we interpret the monadic composition as 'InterleavedT' we get a serial
739−-- but fairly interleaved list-transformer like composition. The monadic
740−-- continuations or iterations of the outer loop are fairly interleaved with
741−-- the continuations or iterations of the inner loop.
776+-- The 'Monad' composition of 'Costream' type interleaves the iterations of
777+-- outer and inner loops in a nested loop composition. This works exactly the
778+-- same way as the merging of two streams in 'costreamly' fashion works. The
779+-- 'costreamly' type combinator can be used to switch to this style of
780+-- composition. Alternatively, a type annotation can be used to specify the
781+-- type of the stream as 'Costream'.
742782--
743783-- @
744784-- import "Streamly"
745785-- import "Streamly.Prelude"
746786--
747−-- main = 'runInterleavedT' $ do
787+-- main = 'runStream' . 'costreamly' $ do
748788-- x <- 'fromFoldable' [1,2]
749789-- y <- 'fromFoldable' [3,4]
750−-- liftIO $ putStrLn $ show (x, y)
790+-- once $ putStrLn $ show (x, y)
751791-- @
752792-- @
753793-- (1,3)
@@ -756,21 +796,21 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
756796-- (2,4)
757797-- @
758798--
759−-- You will notice that this is the monadic equivalent of the '<=>' style
760−-- sum composition. The same caveats apply to this as the '<=>' operation.
761799
762800-- $fairlyConcurrentNesting
763801--
764−-- When we interpret the monadic composition as 'ParallelT' we get a
765−-- /concurrent/ list-transformer like composition just like 'AsyncT'. The
766−-- difference is that this is fully parallel with all iterations starting
767−-- concurrently instead of the demand driven concurrency of 'AsyncT'.
802+-- The 'Monad' composition of 'Parallel' type starts all the iterations of a
803+-- loop concurrently instead of demand driven as in the case of 'Coparallel'.
804+-- This works exactly the same way as the merging streams 'parallely' works.
805+-- The 'parallely' type combinator can be used to switch to this style of
806+-- composition. Alternatively, a type annotation can be used to specify the
807+-- type of the stream as 'Parallel'.
768808--
769809-- @
770810-- import "Streamly"
771811-- import "Streamly.Prelude"
772812--
773−-- main = 'runParallelT' $ do
813+-- main = 'runStream' . 'parallely' $ do
774814-- x <- 'fromFoldable' [3,2,1]
775815-- delay x
776816-- @
@@ -779,28 +819,24 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
779819-- ThreadId 39: Delay 2
780820-- ThreadId 38: Delay 3
781821-- @
782−--
783−-- You will notice that this is the monadic equivalent of the '<|>' style
784−-- sum composition. The same caveats apply to this as the '<|>' operation.
785822
786823-- $monadExercise
787824--
788825-- The streamly code is usually written in a way that is agnostic of the
789826-- specific monadic composition type. We use a polymorphic type with a
790827-- 'IsStream' type class constraint. When running the stream we can choose the
791−-- specific mode of composition. For example look at the following code.
828+-- specific mode of composition. For example take a look at the following code.
792829--
793830-- @
794831-- import "Streamly"
795832-- import "Streamly.Prelude"
796833--
797−--
798834-- composed :: 'IsStream' t => t m a
799835-- composed = do
800836-- sz <- sizes
801837-- cl <- colors
802838-- sh <- shapes
803−-- liftIO $ putStrLn $ show (sz, cl, sh)
839+-- once $ putStrLn $ show (sz, cl, sh)
804840--
805841-- where
806842--
@@ -812,20 +848,10 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
812848-- Now we can interpret this in whatever way we want:
813849--
814850-- @
815−-- main = 'runStreamT' composed
816−-- main = 'runAsyncT' composed
817−-- main = 'runInterleavedT' composed
818−-- main = 'runParallelT' composed
819−-- @
820−--
821−-- Equivalently, we can also write it using the type adapter combinators, like
822−-- this:
823−--
824−-- @
825−-- main = 'runStream' $ 'serially' $ composed
826−-- main = 'runStream' $ 'asyncly' $ composed
827−-- main = 'runStream' $ 'interleaving' $ composed
828−-- main = 'runStream' $ 'parallely' $ composed
851+-- main = 'runStream' . 'streamly' $ composed
852+-- main = 'runStream' . 'costreamly' $ composed
853+-- main = 'runStream' . 'coparallely' $ composed
854+-- main = 'runStream' . 'parallely' $ composed
829855-- @
830856--
831857-- As an exercise try to figure out the output of this code for each mode of
@@ -834,15 +860,13 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
834860-- $functor
835861--
836862-- 'fmap' transforms a stream by mapping a function on all elements of the
837−-- stream. The functor instance of each stream type defines 'fmap' to be
838−-- precisely the same as 'liftM', and therefore 'fmap' is always serial
839−-- irrespective of the type. For concurrent mapping, alternative versions of
840−-- 'fmap', namely, 'asyncMap' and 'parMap' are provided.
863+-- stream. 'fmap' behaves in the same way for all stream types, it is always
864+-- serial.
841865--
842866-- @
843867-- import "Streamly"
844868--
845−-- main = ('toList' $ 'serially' $ fmap show $ 'fromFoldable' [1..10]) >>= print
869+-- main = ('toList' $ fmap show $ 'fromFoldable' [1..10]) >>= print
846870-- @
847871--
848872-- Also see the 'mapM' and 'sequence' functions for mapping actions, in the
@@ -851,11 +875,11 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
851875-- $applicative
852876--
853877-- Applicative is precisely the same as the 'ap' operation of 'Monad'. For
854−-- zipping and parallel applicatives separate types 'ZipSerial' and 'ZipAsync'
855−-- are provided.
878+-- zipping applicatives separate types 'ZipStream' and 'ZipParallel' are
879+-- provided.
856880--
857−-- The following example runs all iterations serially and takes a total 17
858−-- seconds (1 + 3 + 4 + 2 + 3 + 4):
881+-- The following example uses the 'Stream' applicative, it runs all iterations
882+-- serially and takes a total 17 seconds (1 + 3 + 4 + 2 + 3 + 4):
859883--
860884-- @
861885-- import "Streamly"
@@ -866,7 +890,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
866890-- s2 = d 3 <> d 4
867891-- d n = delay n >> return n
868892--
869−-- main = ('toList' . 'serially' $ (,) \<$> s1 \<*> s2) >>= print
893+-- main = ('toList' . 'streamly' $ (,) \<$> s1 \<*> s2) >>= print
870894-- @
871895-- @
872896-- ThreadId 36: Delay 1
@@ -878,11 +902,11 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
878902-- [(1,3),(1,4),(2,3),(2,4)]
879903-- @
880904--
881−-- Similalrly interleaving runs the iterations in an interleaved order but
882−-- since it is serial it takes a total of 17 seconds:
905+-- Similalrly 'Costream' applicative runs the iterations in an interleaved
906+-- order but since it is serial it takes a total of 17 seconds:
883907--
884908-- @
885−-- main = ('toList' . 'interleaving' $ (,) \<$> s1 \<*> s2) >>= print
909+-- main = ('toList' . 'costreamly' $ (,) \<$> s1 \<*> s2) >>= print
886910-- @
887911-- @
888912-- ThreadId 36: Delay 1
@@ -894,11 +918,11 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
894918-- [(1,3),(2,3),(1,4),(2,4)]
895919-- @
896920--
897−-- 'AsyncT' can run the iterations concurrently and therefore takes a total
921+-- 'Coparallel' can run the iterations concurrently and therefore takes a total
898922-- of 10 seconds (1 + 2 + 3 + 4):
899923--
900924-- @
901−-- main = ('toList' . 'asyncly' $ (,) \<$> s1 \<*> s2) >>= print
925+-- main = ('toList' . 'coparallely' $ (,) \<$> s1 \<*> s2) >>= print
902926-- @
903927-- @
904928-- ThreadId 34: Delay 1
@@ -910,7 +934,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
910934-- [(1,3),(2,3),(1,4),(2,4)]
911935-- @
912936--
913−-- Similalrly 'ParallelT' as well can run the iterations concurrently and
937+-- Similalrly 'Parallel' as well can run the iterations concurrently and
914938-- therefore takes a total of 10 seconds (1 + 2 + 3 + 4):
915939--
916940-- @
@@ -933,17 +957,17 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
933957-- streamly.
934958--
935959-- @
936−-- +-----+--------------+------------+
937−-- | | Serial | Concurrent |
938−-- +=====+==============+============+
939−-- | DFS | 'StreamT' | 'AsyncT' |
940−-- | +--------------+------------+
941−-- | | '<>' | '<|' |
942−-- +-----+--------------+------------+
943−-- | BFS | 'InterleavedT' | 'ParallelT' |
944−-- | +--------------+------------+
945−-- | | '<=>' | '<|>' |
946−-- +-----+--------------+------------+
960+-- +-----+--------------+--------------+
961+-- | | Serial | Concurrent |
962+-- +=====+==============+==============+
963+-- | DFS | 'Stream' | 'Coparallel' |
964+-- | +--------------+--------------+
965+-- | | 'splice' | 'coparallel' |
966+-- +-----+--------------+--------------+
967+-- | BFS | 'Costream' | 'Parallel' |
968+-- | +--------------+--------------+
969+-- | | 'cosplice' | 'parallel' |
970+-- +-----+--------------+--------------+
947971-- @
948972
949973-- $zipping
@@ -951,12 +975,15 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
951975-- Zipping is a special transformation where the corresponding elements of two
952976-- streams are combined together using a zip function producing a new stream of
953977-- outputs. Two different types are provided for serial and concurrent zipping.
954−-- These types provide an applicative instance that zips the argument streams.
955−-- Also see the zipping function in the "Streamly.Prelude" module.
978+-- These types provide an applicative instance that can be used to lift
979+-- functions to zip the argument streams.
980+-- Also see the zipping functions in the "Streamly.Prelude" module.
956981
957982-- $serialzip
958983--
959−-- 'ZipSerial' zips streams serially:
984+-- The applicative instance of 'ZipStream' type zips streams serially.
985+-- 'zipStreamly' type combinator can be used to switch to serial applicative
986+-- zip composition:
960987--
961988-- @
962989-- import "Streamly"
@@ -964,10 +991,10 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
964991-- import Control.Concurrent
965992--
966993-- d n = delay n >> return n
967−-- s1 = 'adapt' . 'serially' $ d 1 <> d 2
968−-- s2 = 'adapt' . 'serially' $ d 3 <> d 4
994+-- s1 = 'streamly' $ d 1 <> d 2
995+-- s2 = 'streamly' $ d 3 <> d 4
969996--
970−-- main = ('toList' . 'zipping' $ (,) \<$> s1 \<*> s2) >>= print
997+-- main = ('toList' . 'zipStreamly' $ (,) \<$> s1 \<*> s2) >>= print
971998-- @
972999--
9731000-- This takes total 10 seconds to zip, which is (1 + 2 + 3 + 4) since
@@ -983,7 +1010,10 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
9831010
9841011-- $parallelzip
9851012--
986−-- 'ZipAsync' zips streams concurrently:
1013+-- The applicative instance of 'ZipParallel' type zips streams concurrently.
1014+-- 'zipParallely' type combinator can be used to switch to parallel applicative
1015+-- zip composition:
1016+--
9871017--
9881018-- @
9891019-- import "Streamly"
@@ -992,12 +1022,12 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
9921022-- import System.IO (stdout, hSetBuffering, BufferMode(LineBuffering))
9931023--
9941024-- d n = delay n >> return n
995−-- s1 = 'adapt' . 'serially' $ d 1 <> d 2
996−-- s2 = 'adapt' . 'serially' $ d 3 <> d 4
1025+-- s1 = 'streamly' $ d 1 <> d 2
1026+-- s2 = 'streamly' $ d 3 <> d 4
9971027--
9981028-- main = do
999−-- liftIO $ hSetBuffering stdout LineBuffering
1000−-- ('toList' . 'zippingAsync' $ (,) \<$> s1 \<*> s2) >>= print
1029+-- hSetBuffering stdout LineBuffering
1030+-- ('toList' . 'zipParallely' $ (,) \<$> s1 \<*> s2) >>= print
10011031-- @
10021032--
10031033-- This takes 7 seconds to zip, which is max (1,3) + max (2,4) because 1 and 3
@@ -1014,16 +1044,18 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
10141044-- $concurrent
10151045--
10161046-- When writing concurrent programs there are two distinct places where the
1017−-- programmer chooses the type of concurrency. First, when /generating/ a
1018−-- stream by combining other streams we can use one of the sum style operators
1019−-- to combine them concurrently or serially. Second, when /processing/ a stream
1020−-- in a monadic composition we can choose one of the monad composition types to
1047+-- programmer can control the concurrency. First, when /composing/ a stream by
1048+-- merging multiple streams we can choose an appropriate sum style operators to
1049+-- combine them concurrently or serially. Second, when /processing/ a stream in
1050+-- a monadic composition we can choose one of the monad composition types to
10211051-- choose the desired type of concurrency.
10221052--
10231053-- In the following example the squares of @x@ and @y@ are computed
1024−-- concurrently using the '<|' operator and the square roots of their sum are
1025−-- also computed concurrently by using the 'asyncly' combinator. We can choose
1026−-- different combinators e.g. '<>' and 'serially', to control the concurrency.
1054+-- concurrently using the 'coparallel' operation and the square roots of their
1055+-- sum are computed serially because of the 'streamly' combinator. We can
1056+-- choose different combinators for the monadic processing and the stream
1057+-- generation, to control the concurrency. We can also use the 'coparallely'
1058+-- combinator instead of explicitly folding with 'coparallel'.
10271059--
10281060-- @
10291061-- import "Streamly"
@@ -1032,11 +1064,11 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
10321064--
10331065-- main = do
10341066-- z \<- 'toList'
1035−-- $ 'asyncly' -- Concurrent monadic processing (sqrt below)
1067+-- $ 'streamly' -- Serial monadic processing (sqrt below)
10361068-- $ do
1037−-- x2 \<- 'forEachWith' ('<|') [1..100] $ -- Concurrent @"for"@ loop
1069+-- x2 \<- 'forEachWith' 'coparallel' [1..100] $ -- Concurrent @"for"@ loop
10381070-- \\x -> return $ x * x -- body of the loop
1039−-- y2 \<- 'forEachWith' ('<|') [1..100] $
1071+-- y2 \<- 'forEachWith' 'coparallel' [1..100] $
10401072-- \\y -> return $ y * y
10411073-- return $ sqrt (x2 + y2)
10421074-- print $ sum z
@@ -1061,12 +1093,11 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
10611093-- (active hyperlinks are the streamly APIs) and yet it is a reactive
10621094-- application.
10631095--
1064−--
10651096-- This application has two independent and concurrent sources of event
10661097-- streams, @acidRain@ and @userAction@. @acidRain@ continuously generates
1067−-- events that deteriorate the health of the game character. @userAction@ can
1068−-- be "potion" or "quit". When the user types "potion" the health improves and
1069−-- the game continues.
1098+-- events that deteriorate the health of the character in the game.
1099+-- @userAction@ can be "potion" or "quit". When the user types "potion" the
1100+-- health improves and the game continues.
10701101--
10711102-- @
10721103-- {-\# LANGUAGE FlexibleContexts #-}
@@ -1074,7 +1105,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
10741105-- import "Streamly"
10751106-- import Control.Concurrent (threadDelay)
10761107-- import Control.Monad (when)
1077−-- import Control.Monad.State
1108+-- import Control.Monad.State (MonadState, get, modify, runStateT)
10781109-- import Data.Semigroup (cycle1)
10791110--
10801111-- data Event = Harm Int | Heal Int | Quit deriving (Show)
@@ -1092,9 +1123,9 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
10921123-- acidRain :: MonadIO m => 'StreamT' m Event
10931124-- acidRain = cycle1 $ liftIO (threadDelay 1000000) >> return (Harm 1)
10941125--
1095−-- game :: ('MonadAsync' m, MonadState Int m) => 'StreamT' m ()
1126+-- game :: ('MonadParallel' m, MonadState Int m) => 'StreamT' m ()
10961127-- game = do
1097−-- event \<- userAction \<|> acidRain
1128+-- event \<- userAction \`parallel` acidRain
10981129-- case event of
10991130-- Harm n -> modify $ \\h -> h - n
11001131-- Heal n -> modify $ \\h -> h + n
@@ -1107,7 +1138,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
11071138-- main = do
11081139-- putStrLn "Your health is deteriorating due to acid rain,\\
11091140-- \\ type \\"potion\\" or \\"quit\\""
1110−-- _ <- runStateT ('runStreamT' game) 60
1141+-- _ <- runStateT ('runStream' game) 60
11111142-- return ()
11121143-- @
11131144--
@@ -1127,65 +1158,96 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
11271158-- package which is specifically and carefully designed to measure the
11281159-- performance of Haskell streaming libraries fairly and squarely in the right
11291160-- way. Streamly performs at par or even better than most streaming libraries
1130−-- for common operations even though it needs to deal with the concurrency
1161+-- for serial operations even though it needs to deal with the concurrency
11311162-- capability.
11321163
11331164-- $interop
11341165--
11351166-- We can use @unfoldr@ and @uncons@ to convert one streaming type to another.
1136−-- We will assume the following common code to be available in the examples
1137−-- demonstrated below.
1167+--
1168+-- Interop with @vector@:
11381169--
11391170-- @
1140−-- import "Streamly"
1141−-- import "Streamly.Prelude"
1142−-- import System.IO (stdin)
1171+-- import Streamly
1172+-- import qualified Streamly.Prelude as S
1173+-- import qualified Data.Vector.Fusion.Stream.Monadic as V
1174+--
1175+-- main = do
1176+-- -- streamly to vector
1177+-- V.toList (V.unfoldrM S.uncons (S.fromFoldable [1..3])) >>= print
1178+--
1179+-- -- vector to streamly
1180+-- S.toList (S.unfoldrM unconsV (V.fromList [1..3])) >>= print
11431181--
1144−-- -- Adapt uncons to return an Either instead of Maybe
1145−-- unconsE s = 'uncons' s >>= maybe (return $ Left ()) (return . Right)
1146−-- stdinLn = 'serially' $ 'fromHandle' stdin
1182+-- where
1183+-- unconsV v = do
1184+-- r <- V.null v
1185+-- if r
1186+-- then return Nothing
1187+-- else do
1188+-- h <- V.head v
1189+-- return $ Just (h, V.tail v)
11471190-- @
11481191--
11491192-- Interop with @pipes@:
11501193--
11511194-- @
1195+-- import "Streamly"
1196+-- import qualified "Streamly.Prelude" as S
11521197-- import qualified Pipes as P
11531198-- import qualified Pipes.Prelude as P
11541199--
11551200-- main = do
11561201-- -- streamly to pipe
1157−-- P.runEffect $ P.for (P.unfoldr unconsE stdinLn) (lift . putStrLn)
1202+-- P.toListM (P.unfoldr unconsS (S.'fromFoldable' [1..3])) >>= print
11581203--
11591204-- -- pipe to streamly
1205+-- S.'toList' (S.'unfoldrM' unconsP (P.each [1..3])) >>= print
1206+--
1207+-- where
11601208-- -- Adapt P.next to return a Maybe instead of Either
1161−-- let nextM p = P.next p >>= either (\\_ -> return Nothing) (return . Just)
1162−-- 'runStreamT' $ 'unfoldrM' nextM P.stdinLn >>= lift . putStrLn
1209+-- unconsP p = P.next p >>= either (\\_ -> return Nothing) (return . Just)
1210+--
1211+-- -- Adapt S.uncons to return an Either instead of Maybe
1212+-- unconsS s = S.'uncons' s >>= maybe (return $ Left ()) (return . Right)
11631213-- @
11641214--
11651215-- Interop with @streaming@:
11661216--
11671217-- @
1168−-- import qualified Streaming as S
1169−-- import qualified Streaming.Prelude as S
1218+-- import "Streamly"
1219+-- import qualified "Streamly.Prelude" as S
1220+-- import qualified Streaming as SG
1221+-- import qualified Streaming.Prelude as SG
11701222--
11711223-- main = do
11721224-- -- streamly to streaming
1173−-- S.stdoutLn $ S.unfoldr unconsE stdinLn
1225+-- SG.toList (SG.unfoldr unconsS (S.'fromFoldable' [1..3])) >>= print
11741226--
11751227-- -- streaming to streamly
1176−-- 'runStreamT' $ unfoldrM S.uncons S.stdinLn >>= lift . putStrLn
1228+-- S.'toList' (S.unfoldrM SG.uncons (SG.each [1..3])) >>= print
1229+--
1230+-- where
11771231--
1232+-- -- Adapt S.uncons to return an Either instead of Maybe
1233+-- unconsS s = S.'uncons' s >>= maybe (return $ Left ()) (return . Right)
11781234-- @
11791235--
11801236-- Interop with @conduit@:
11811237--
11821238-- @
1239+-- import "Streamly"
1240+-- import qualified "Streamly.Prelude" as S
11831241-- import qualified Data.Conduit as C
11841242-- import qualified Data.Conduit.List as C
11851243-- import qualified Data.Conduit.Combinators as C
11861244--
1187−-- -- streamly to conduit
1188−-- main = (C.unfoldM 'uncons' stdinLn) C.$$ C.print
1245+-- main = do
1246+-- -- streamly to conduit
1247+-- C.runConduit (C.unfoldM S.'uncons' (S.'fromFoldable' [1..3]) C..| C.sinkList) >>= print
1248+--
1249+-- -- It seems there is no way out of a conduit as it does not provide an
1250+-- -- uncons or a tail function.
11891251-- @
11901252
11911253-- $comparison
@@ -1197,17 +1259,19 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
11971259--
11981260-- @
11991261-- +-----------------+----------------+
1200−-- | Non-determinism | <https://hackage.haskell.org/package/list-t list-t> |
1262+-- | Non-determinism | <https://hackage.haskell.org/package/pipes pipes> |
1263+-- | +----------------+
1264+-- | | <https://hackage.haskell.org/package/list-t list-t> |
12011265-- | +----------------+
12021266-- | | <https://hackage.haskell.org/package/logict logict> |
12031267-- +-----------------+----------------+
1204−-- | Streaming | <https://hackage.haskell.org/package/streaming streaming> |
1268+-- | Streaming | <https://hackage.haskell.org/package/vector vector> |
12051269-- | +----------------+
1206−-- | | <https://hackage.haskell.org/package/conduit conduit> |
1270+-- | | <https://hackage.haskell.org/package/streaming streaming> |
12071271-- | +----------------+
12081272-- | | <https://hackage.haskell.org/package/pipes pipes> |
12091273-- | +----------------+
1210−-- | | <https://hackage.haskell.org/package/simple-conduit simple-conduit> |
1274+-- | | <https://hackage.haskell.org/package/conduit conduit> |
12111275-- +-----------------+----------------+
12121276-- | Concurrency | <https://hackage.haskell.org/package/async async> |
12131277-- | +----------------+
@@ -1221,31 +1285,30 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
12211285-- +-----------------+----------------+
12221286-- @
12231287--
1224−-- Streamly covers all the functionality provided by both the non-determinism
1225−-- packages listed above and provides better performance in comparison to
1226−-- those. In fact, at the core streamly is a list transformer but it naturally
1227−-- integrates the concurrency dimension to the basic list transformer
1228−-- functionality.
1288+-- Streamly is a list-transformer. It provides all the functionality provided
1289+-- by any of the list transformer and logic programming packages listed above.
1290+-- In addition, Streamly naturally integrates the concurrency dimension to the
1291+-- basic list transformer functionality.
12291292--
1230−-- When it comes to streaming, in terms of core concepts, @simple-conduit@ is
1231−-- the package that is closest to streamly if we set aside the concurrency
1232−-- dimension, both are streaming packages with list transformer like monad
1233−-- composition. However, in terms of API @streamly@ is more like the @streaming@
1234−-- package. Streamly can be used to achieve more or less the functionality
1235−-- provided by any of the streaming packages listed above. The types and API of
1236−-- streamly are much simpler in comparison to conduit and pipes. It is more or
1237−-- less like the standard Haskell list APIs.
1293+-- When it comes to streaming, in terms of the streaming API streamly is almost
1294+-- identical to the vector package. Streamly, vector and streaming packages all
1295+-- represent a stream as data and are therefore similar in the fundamental
1296+-- approach to streaming. The fundamental difference is that streamly adds
1297+-- concurrency support and the monad instance provides concurrent looping.
1298+-- Other streaming libraries like pipes, conduit and machines represent and
1299+-- compose stream processors rather than the stream data and therefore fall in
1300+-- another class of streaming libraries and have comparatively more complicated
1301+-- types.
12381302--
12391303-- When it comes to concurrency, streamly can do everything that the @async@
12401304-- package can do and more. async provides applicative concurrency whereas
1241−-- streamly provides both applicative and monadic concurrency. The 'ZipAsync'
1242−-- type behaves like the applicative instance of async. This work was
1243−-- originally inspired by the concurrency implementation in @transient@ though
1244−-- it has no resemblence with that. Streamly provides concurrency as transient
1245−-- does but in a sort of dual manner, it can lazily stream the output. In
1305+-- streamly provides both applicative and monadic concurrency. The
1306+-- 'ZipParallel' type behaves like the applicative instance of async. In
12461307-- comparison to transient streamly has a first class streaming interface and
12471308-- is a monad transformer that can be used universally in any Haskell monad
1248−-- transformer stack.
1309+-- transformer stack. Streamly was in fact originally inspired by the
1310+-- concurrency implementation in @transient@ though it has no resemblence with
1311+-- that and takes a lazy pull approach versus transient's strict push approach.
12491312--
12501313-- The non-determinism, concurrency and streaming combination make streamly a
12511314-- strong FRP capable library as well. FRP is fundamentally stream of events