1{-# OPTIONS_GHC -fno-warn-unused-imports #-}
3-- Module : Streamly.Tutorial
4-- Copyright : (c) 2017 Harendra Kumar
7-- Maintainer : harendra.kumar@gmail.com
9-- Streamly is a general computing framework based on streaming IO. The IO
10-- monad and pure lists are a special case of streamly. On one hand, streamly
11-- extends the lists of pure values to lists of monadic actions, on the other
12-- hand it extends the IO monad with concurrrent non-determinism. In simple
13-- imperative terms we can say that streamly extends the IO monad with @for@
14-- loops and nested @for@ loops with concurrency support. You can understand
15-- this analogy better once you can go through this tutorial.
17-- Streaming in general enables writing modular, composable and scalable
18-- applications with ease, and concurrency allows you to make them scale and
19-- perform well. Streamly enables writing scalable concurrent applications
20-- without being aware of threads or synchronization. No explicit thread
21-- control is needed, where applicable the concurrency rate is automatically
22-- controlled based on the demand by the consumer. However, combinators can be
23-- used to fine tune the concurrency control.
25-- Streaming and concurrency together enable expressing reactive applications
26-- conveniently. See the @CirclingSquare@ example in the examples directory for
27-- a simple SDL based FRP example. To summarize, streamly provides a unified
28-- computing framework for streaming, non-determinism and functional reactive
29-- programming in an elegant and simple API that is a natural extension of pure
30-- lists to monadic streams.
32-- In this tutorial we will go over the basic concepts and how to
33-- use the library. See the last section for further reading resources.
35module Streamly.Tutorial
40 -- * Concurrent Streams
43 -- * Combining Streams
46 -- * Imports and Supporting Code
49 -- * Generating Streams
52 -- * Generating Streams Concurrently
53 -- $generatingConcurrently
55 -- * Eliminating Streams
58 -- * Concurrent Pipeline Stages
59 -- $concurrentApplication
61 -- * Transforming Streams
64 -- * Mapping Concurrently
65 -- $concurrentTransformation
72 -- *** Deep Serial Composition ('Serial')
75 -- *** Wide Serial Composition ('WSerial')
78 -- *** Deep Ahead Composition ('Ahead')
81 -- *** Deep Asynchronous Composition ('Async')
84 -- *** Wide Asynchronous Composition ('WAsync')
87 -- *** Parallel Asynchronous Composition ('Parallel')
90 -- *** Custom composition
102 -- *** Deep Serial Nesting ('Serial')
105 -- *** Wide Serial Nesting ('WSerial')
106 -- $interleavedNesting
108 -- *** Deep Ahead Nesting ('Ahead')
111 -- *** Deep Asynchronous Nesting ('Async')
112 -- $concurrentNesting
114 -- *** Wide Asynchronous Nesting ('WAsync')
117 -- *** Parallel Asynchronous Nesting ('Parallel')
135 -- ** Parallel Zipping
138 -- * Monad transformers
139 -- $monadtransformers
141 -- * Concurrent Programming
144 -- * Reactive Programming
147 -- * Writing Concurrent Programs
153 -- * Interoperation with Streaming Libraries
156 -- * Comparison with Existing Packages
159 -- * Where to go next?
165import Streamly.Prelude
167import Control.Applicative
169import Control.Monad.IO.Class (MonadIO(..))
170import Control.Monad.Trans.Class (MonadTrans (lift))
174-- The way a list represents a sequence of pure values, a stream represents a
175-- sequence of monadic actions. The monadic stream API offered by Streamly is
176-- very close to the Haskell "Prelude" pure lists' API, it can be considered as a
177-- natural extension of lists to monadic actions. Streamly streams provide
178-- concurrent composition and merging of streams. It can be considered as a
179-- concurrent list transformer. In contrast to the "Prelude" lists, merging or
180-- appending streams of arbitrary length is scalable and inexpensive.
182-- The basic stream type is 'Serial', it represents a sequence of IO actions,
183-- and is a 'Monad'. The 'Serial' monad is almost a drop in replacement for
184-- the 'IO' monad, IO monad is a special case of the 'Serial' monad; IO monad
185-- represents a single IO action whereas the 'Serial' monad represents a series
186-- of IO actions. The only change you need to make to go from 'IO' to 'Serial'
187-- is to use 'runStream' to run the monad and to prefix the IO actions with
188-- either 'once' or 'liftIO'. If you use liftIO you can switch from 'Serial'
189-- to IO monad by simply removing the 'runStream' function; no other changes
190-- are needed unless you have used some stream specific composition or
193-- Similarly, the 'Serial' type is almost a drop in replacement for pure lists,
194-- pure lists are a special case of monadic streams. If you use 'nil' in place
195-- of '[]' and '|:' in place ':' you can replace a list with a 'Serial' stream.
196-- The only difference is that the elements must be monadic type and to operate
197-- on the streams we must use the corresponding functions from
198-- "Streamly.Prelude" instead of using the base "Prelude".
202-- Streams can be generated concurrently, even infnite streams can be generated
203-- concurrently using controlled concurrency, streams can be merged
204-- concurrently, multiple stages in a streaming pipeline can run concurrently,
205-- streams can be mapped concurrently, they can be zipped concurrently, and in
206-- a monadic composition they combine like a list transformer providing
207-- concurrent non-determinism.
209-- There are three basic concurrent stream styles, 'Ahead', 'Async', and
210-- 'Parallel'. The 'Ahead' style streams are similar to 'Serial' except that
211-- they can speculatively execute multiple stream actions concurrently in
212-- advance. 'Ahead' would return exactly the same stream as 'Serial' except
213-- that it may execute the actions concurrently. The 'Async' style streams,
214-- like 'Ahead', speculatively execute multiple stream actions in advance but
215-- return the results in their finishing order rather than in the stream
216-- traversal order. 'Parallel' is like 'Async' except that it provides
217-- unbounded parallelism instead of controlled parallelism.
219-- For easy reference, we can classify the stream types based on /execution order/,
220-- /consumption order/, and /bounded or unbounded/ concurrency.
221-- Execution could be serial (i.e. synchronous) or asynchronous. In serial
222-- execution we execute the next action in the stream only after the previous
223-- one has finished executing. In asynchronous execution multiple actions in
224-- the stream can be executed asynchronously i.e. the next action can start
225-- executing even before the first one has finished. Consumption order
226-- determines the order in which the outputs generated by the composition are
227-- consumed. Consumption could be serial or asynchronous. In serial
228-- consumption, the outputs are consumed in the traversal order, in
229-- asynchronous consumption the outputs are consumed as they arrive i.e. first
230-- come first serve order.
233-- +------------+--------------+--------------+--------------+
234-- | Type | Execution | Consumption | Concurrency |
235-- +============+==============+==============+==============+
236-- | 'Serial' | Serial | Serial | None |
237-- +------------+--------------+--------------+--------------+
238-- | 'Ahead' | Asynchronous | Serial | bounded |
239-- +------------+--------------+--------------+--------------+
240-- | 'Async' | Asynchronous | Asynchronous | bounded |
241-- +------------+--------------+--------------+--------------+
242-- | 'Parallel' | Asynchronous | Asynchronous | unbounded |
243-- +------------+--------------+--------------+--------------+
246-- All these types can be freely inter-converted using type conversion
247-- combinators or type annotations without any cost, to acheive the desired
248-- composition style. To force a particular type of composition we coerce the
249-- stream type using the corresponding type adapting combinator from
250-- 'serially', 'aheadly', 'asyncly', or 'parallely'. The default stream type
251-- is inferred as 'Serial' unless you change it by using one of the combinators
252-- or using a type annotation.
256-- Streams can be combined using semigroup or monoid composition to form a
257-- composite stream. Traversal of a composition of streams could be @deep@ or
258-- @wide@. Deep goes depth first i.e. each stream is traversed fully before
259-- we traverse the next stream. Wide goes breadth first i.e. one element from
260-- each stream is traversed before coming back to the first stream again.
262-- Each stream type has a wide traversal variant prefixed by 'W'. The wide
263-- variant differs only in the Semigroup\/Monoid, Applicative\/Monad
264-- compositions of the streams.
265-- The following table summarizes the basic types and the corresponding wide
269-- +------------+-----------+
271-- +============+===========+
272-- | 'Serial' | 'WSerial' |
273-- +------------+-----------+
274-- | 'Ahead' | 'WAhead' |
275-- +------------+-----------+
276-- | 'Async' | 'WAsync' |
277-- +------------+-----------+
280-- Other than these types there are also 'ZipSerial' and 'ZipAsync' types that
281-- zip streams serially or concurrently using 'Applicative' operation. These
282-- types are not monads they are only applicatives and they do not differ in
283-- 'Semigroup' composition.
288-- When writing concurrent programs it is advised to not use the concurrent
289-- style stream combinators blindly at the top level. That might create too
290-- much concurrency where it is not even required, and can even degrade
291-- performance in some cases. In some cases it can also lead to surprising
292-- behavior because of some code that is supposed to be serial becoming
293-- concurrent. Please be aware that all concurrency capable APIs that you may
294-- have used under the scope of a concurrent stream combinator will become
295-- concurrent. For example if you have a 'repeatM' somewhere in your program
296-- and you use 'parallely' on top, the 'repeatM' becomes fully parallel,
297-- resulting into an infinite parallel execution . Instead, use the
298-- /Keep It Serial and Stupid/ principle, start with the default serial
299-- composition and enable concurrent combinators only when and where necessary.
300-- When you use a concurrent combinator you can use an explicit 'serially'
301-- combinator to suppress any unnecessary concurrency under the scope of that
306-- To represent streams in an arbitrary monad use the more general monad
307-- transformer types for example the monad transformer type corresponding to
308-- the 'Serial' type is 'SerialT'. @SerialT m a@ represents a stream of values
309-- of type 'a' in some underlying monad 'm'. For example, @SerialT IO Int@ is a
310-- stream of 'Int' in 'IO' monad. In fact, the type 'Serial' is a synonym for
313-- Similarly we have monad transformer types for other stream types as well viz.
314-- 'WSerialT', 'AsyncT', 'WAsyncT' and 'ParallelT'.
316-- To lift a value from an underlying monad in a monad transformer stack into a
317-- singleton stream use 'lift' and to lift from an IO action use 'liftIO'.
320-- > 'runStream' $ liftIO $ putStrLn "Hello world!"
322-- > 'runStream' $ lift $ putStrLn "Hello world!"
329-- We will assume the following imports in this tutorial. Go ahead, fire up a
330-- GHCi session and import these lines to start playing.
333-- > import "Streamly"
334-- > import "Streamly.Prelude" ((|:))
335-- > import qualified "Streamly.Prelude" as S
338-- 'nil' represents an empty stream and 'consM' or its operator form '|:' adds
339-- a monadic action at the head of the stream.
342-- > S.'toList' S.'nil'
344-- > S.'toList' $ 'getLine' |: 'getLine' |: S.'nil'
350-- To create a singleton stream from a pure value use 'pure' and to create a
351-- singleton stream from a monadic action use 'once'.
354-- > S.'toList' $ 'pure' 1
356-- > S.'toList' $ S.'once' 'getLine'
361-- To create a stream from pure values in a 'Foldable' container use
362-- 'fromFoldable' which is equivalent to a fold using 'cons' and 'nil':
365-- > S.'toList' $ S.'fromFoldable' [1..3]
367-- > S.'toList' $ 'Prelude.foldr' S.'cons' S.'nil' [1..3]
371-- To create a stream from monadic actions in a 'Foldable' container just use a
372-- right fold using 'consM' and 'nil':
375-- > 'runStream' $ 'Prelude.foldr' ('|:') S.'nil' ['putStr' "Hello ", 'putStrLn' "world!"]
379-- For more ways to construct a stream see the module "Streamly.Prelude".
381-- $generatingConcurrently
383-- Monadic construction and generation functions e.g. 'consM', 'unfoldrM',
384-- 'replicateM', 'repeatM', 'iterateM' and 'fromFoldableM' etc work
385-- concurrently when used with appropriate stream type combinator. The pure
386-- versions of these APIs are not concurrent, however you can use the monadic
387-- versions even for pure computations by wrapping the pure value in a monad to
388-- get the concurrent generation capability where required.
390-- The following code finishes in 3 seconds (6 seconds when serial):
393-- > let p n = threadDelay (n * 1000000) >> return n
394-- > S.'toList' $ 'parallely' $ p 3 |: p 2 |: p 1 |: S.'nil'
396-- > S.'toList' $ 'aheadly' $ p 3 |: p 2 |: p 1 |: S.'nil'
399-- The following finishes in 10 seconds (100 seconds when serial):
402-- > 'runStream' $ 'asyncly' $ S.'replicateM' 10 $ p 10
408-- We have already seen 'runStream' and 'toList' to eliminate a stream in the
409-- examples above. 'runStream' runs a stream discarding the results i.e. only
410-- for effects. 'toList' runs the stream and collects the results in a list.
412-- For other ways to eliminate a stream see the @Folding@ section in
413-- "Streamly.Prelude" module.
417-- Transformation over a stream is the equivalent of a @for@ loop construct in
418-- imperative paradigm. We iterate over every element in the stream and perform
419-- certain transformations for each element. Transformations may involve
420-- mapping functions over the elements, filtering elements from the stream or
421-- folding all the elements in the stream into a single value. Streamly streams
422-- are exactly like lists and you can perform all the transformations in the
423-- same way as you would on lists.
425-- Here is a simple console echo program that just echoes every input line,
429-- > 'runStream' $ S.'repeatM' getLine & S.'mapM' putStrLn
432-- The following code snippet reads lines from standard input, filters blank
433-- lines, drops the first non-blank line, takes the next two, up cases them,
434-- numbers them and prints them:
438-- import qualified "Streamly.Prelude" as S
439-- import Data.Char (toUpper)
440-- import Data.Function ((&))
442-- main = 'runStream' $
443-- S.'repeatM' getLine
444-- & S.'filter' (not . null)
447-- & fmap (map toUpper)
448-- & S.'zipWith' (\\n s -> show n ++ " " ++ s) (S.'fromFoldable' [1..])
449-- & S.'mapM' putStrLn
452-- $concurrentTransformation
454-- Monadic transformation functions 'mapM' and 'sequence' work concurrently
455-- when used with appropriate stream type combinators. The pure versions do not
456-- work concurrently, however you can use the monadic versions even for pure
457-- computations to get the concurrent transformation capability where required.
459-- This would print a value every second (2 seconds when serial):
462-- > let p n = threadDelay (n * 1000000) >> return n
463-- > runStream $ aheadly $ S.'mapM' (\\x -> p 1 >> print x) (serially $ repeatM (p 1))
467-- $concurrentApplication
469-- The concurrent function application operators '|$' and '|&' apply a stream
470-- argument to a stream function concurrently to compose a concurrent pipeline
471-- of stream processing functions:
473-- Because both the stages run concurrently, we would see a delay of only 1
474-- second instead of 2 seconds in the following:
477-- > let p n = threadDelay (n * 1000000) >> return n
478-- > runStream $ S.'repeatM' (p 1) '|&' S.'mapM' (\\x -> p 1 >> print x)
483-- We can combine two streams into a single stream using semigroup composition
484-- operation '<>'. Streams can be combined in many different ways as described
485-- in the following sections, the '<>' operation behaves differently depending
486-- on the stream type in effect. The stream type and therefore the composition
487-- style can be changed at any point using one of the type combinators as
492-- In most of example snippets we do not repeat the imports. Where imports are
493-- not explicitly specified use the imports shown below.
497-- import "Streamly.Prelude" ((|:), nil)
498-- import qualified "Streamly.Prelude" as S
500-- import Control.Concurrent
501-- import Control.Monad (forever)
504-- To illustrate concurrent vs serial composition aspects, we will use the
505-- following @delay@ function to introduce a sleep or delay specified in
506-- seconds. After the delay it prints the number of seconds it slept.
509-- delay n = S.'once' $ do
510-- threadDelay (n * 1000000)
512-- putStrLn (show tid ++ ": Delay " ++ show n)
517-- The 'Semigroup' operation '<>' of the 'Serial' type combines the two streams
518-- in a /serial depth first/ manner. We use the 'serially' type combinator to
519-- effect 'Serial' style of composition. We can also use an explicit 'Serial'
520-- type annotation for the stream to acheive the same effect. However, since
521-- 'Serial' is the default type unless explicitly specified by using a
522-- combinator, we can omit using an explicit combinator or type annotation for
523-- this style of composition.
525-- When two streams with multiple elements are combined in this manner, the
526-- monadic actions in the two streams are performed sequentially i.e. first all
527-- actions in the first stream are performed sequentially and then all actions
528-- in the second stream are performed sequentially. We call it
529-- /serial depth first/ as the full depth of one stream is fully traversed
530-- before we move to the next. The following example prints the sequence 1, 2,
534-- main = 'runStream' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
543-- All actions in both the streams are performed serially in the same thread.
544-- In the following example we can see that all actions are performed in the
545-- same thread and take a combined total of @3 + 2 + 1 = 6@ seconds:
548-- main = 'runStream' $ delay 3 <> delay 2 <> delay 1
551-- ThreadId 36: Delay 3
552-- ThreadId 36: Delay 2
553-- ThreadId 36: Delay 1
556-- The polymorphic version of the binary operation '<>' of the 'Serial' type is
557-- 'serial'. We can use 'serial' to join streams in a sequential manner
558-- irrespective of the type of stream:
561-- main = 'runStream' $ (print 1 |: print 2 |: nil) \`serial` (print 3 |: print 4 |: nil)
566-- The 'Semigroup' operation '<>' of the 'WSerial' type combines the two
567-- streams in a /serial breadth first/ manner. We use the 'wSerially' type
568-- combinator to effect 'WSerial' style of composition. We can also use the
569-- 'WSerial' type annotation for the stream to acheive the same effect.
571-- When two streams with multiple elements are combined in this manner, we
572-- traverse all the streams in a breadth first manner i.e. one action from each
573-- stream is peformed and yielded to the resulting stream before we come back
574-- to the first stream again and so on.
575-- The following example prints the sequence 1, 3, 2, 4
578-- main = 'runStream' . 'wSerially' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
587-- Even though the monadic actions of the two streams are performed in an
588-- interleaved manner they are all performed serially in the same thread. In
589-- the following example we can see that all actions are performed in the same
590-- thread and take a combined total of @3 + 2 + 1 = 6@ seconds:
593-- main = 'runStream' . 'wSerially' $ delay 3 <> delay 2 <> delay 1
596-- ThreadId 36: Delay 3
597-- ThreadId 36: Delay 2
598-- ThreadId 36: Delay 1
601-- The polymorphic version of the 'WSerial' binary operation '<>' is called
602-- 'wSerial'. We can use 'wSerial' to join streams in an interleaved manner
603-- irrespective of the type, notice that we have not used the 'wSerially'
604-- combinator in the following example:
607-- main = 'runStream' $ (print 1 |: print 2 |: nil) \`wSerial` (print 3 |: print 4 |: nil)
616-- Note that this composition cannot be used to fold infinite number of streams
617-- since it requires preserving the state until a stream is finished.
621-- The 'Semigroup' operation '<>' of the 'Ahead' type combines two streams in a
622-- /serial depth first/ manner with concurrent lookahead. We use the 'aheadly'
623-- type combinator to effect 'Ahead' style of composition. We can also use an
624-- explicit 'Ahead' type annotation for the stream to acheive the same effect.
626-- When two streams are combined in this manner, the streams are traversed in
627-- depth first manner just like 'Serial', however it can execute the next
628-- stream concurrently and keep the results ready when its turn arrives.
629-- Concurrent execution of the next stream(s) is performed if the first stream
630-- blocks or if it cannot produce output at the rate that is enough to meet the
631-- consumer demand. Multiple streams can be executed concurrently to meet the
632-- demand. The following example would print the result in a second even
633-- though each action in each stream takes one second:
637-- xs \<- 'toList' . 'aheadly' $ (p 1 |: p 2 |: nil) <> (p 3 |: p 4 |: nil)
639-- where p n = threadDelay 1000000 >> return n
645-- Each stream is constructed 'aheadly' and then both the streams are merged
646-- 'aheadly', therefore, all the actions can run concurrently but the result is
647-- presented in serial order.
649-- You can also use the polymorphic combinator 'ahead' in place of '<>' to
650-- compose any type of streams in this manner.
654-- The 'Semigroup' operation '<>' of the 'Async' type combines the two
655-- streams in a depth first manner with parallel look ahead. We use the
656-- 'asyncly' type combinator to effect 'Async' style of composition. We
657-- can also use the 'Async' type annotation for the stream type to acheive
660-- When two streams with multiple elements are combined in this manner, the
661-- streams are traversed in depth first manner just like 'Serial', however it
662-- can execute the next stream concurrently and return the results from it
663-- as they arrive i.e. the results from the next stream may be yielded even
664-- before the results from the first stream. Concurrent execution of the next
665-- stream(s) is performed if the first stream blocks or if it cannot produce
666-- output at the rate that is enough to meet the consumer demand. Multiple
667-- streams can be executed concurrently to meet the demand.
668-- In the example below each element in the stream introduces a constant delay
669-- of 1 second, however, it takes just one second to produce all the results.
670-- The results are not guaranteed to be in any particular order:
674-- xs \<- 'runStream' . 'asyncly' $ (p 1 |: p 2 |: nil) <> (p 3 |: p 4 |: nil)
676-- where p n = threadDelay 1000000 >> return n
682-- The constituent streams are also composed in 'Async' manner and the
683-- composition of streams too. We can compose the constituent streams to run
684-- serially, in that case it would take 2 seconds to produce all the results.
685-- The elements in the serial streams would be in serial order in the results:
689-- xs \<- 'runStream' . 'asyncly' $ (serially $ p 1 |: p 2 |: nil) <> (serially $ p 3 |: p 4 |: nil)
691-- where p n = threadDelay 1000000 >> return n
697-- In the following example we can see that new threads are started when a
698-- computation blocks. Notice that the output from the stream with the
699-- shortest delay is printed first. The whole computation takes @maximum of
700-- (3, 2, 1) = 3@ seconds:
703-- main = 'runStream' . 'asyncly' $ delay 3 '<>' delay 2 '<>' delay 1
706-- ThreadId 42: Delay 1
707-- ThreadId 41: Delay 2
708-- ThreadId 40: Delay 3
711-- When we have a tree of computations composed using this style, the tree is
712-- traversed in DFS style just like the 'Serial' style, the only difference is
713-- that here we can move on to executing the next stream if a stream blocks.
714-- However, we will not start new threads if we have sufficient output to
715-- saturate the consumer. This is why we call it left-biased demand driven or
716-- adaptive concurrency style, the concurrency tends to stay on the left side
717-- of the composition as long as possible. More threads are started based on
718-- the pull rate of the consumer. The following example prints an output every
719-- second as all of the actions are concurrent.
722-- main = 'runStream' . 'asyncly' $ (delay 1 <> delay 2) <> (delay 3 <> delay 4)
731-- All the computations may even run in a single thread when more threads are
732-- not needed. As you can see, in the following example the computations are
733-- run in a single thread one after another, because none of them blocks.
734-- However, if the thread consuming the stream were faster than the producer
735-- then it would have started parallel threads for each computation to keep up
736-- even if none of them blocks:
739-- main = 'runStream' . 'asyncly' $ traced (sqrt 9) '<>' traced (sqrt 16) '<>' traced (sqrt 25)
740-- where traced m = S.'once' (myThreadId >>= print) >> return m
748-- Note that the order of printing in the above examples may change due to
749-- variations in scheduling latencies for concurrent threads.
751-- The polymorphic version of the 'Async' binary operation '<>' is called
752-- 'async'. We can use 'async' to join streams in a left biased
753-- adaptively concurrent manner irrespective of the type, notice that we have
754-- not used the 'asyncly' combinator in the following example:
757-- main = 'runStream' $ delay 3 \`async` delay 2 \`async` delay 1
760-- ThreadId 42: Delay 1
761-- ThreadId 41: Delay 2
762-- ThreadId 40: Delay 3
765-- Since the concurrency provided by this operator is demand driven it cannot
766-- be used when the composed computations start timers that are relative to
767-- each other because all computations may not be started at the same time and
768-- therefore timers in all of them may not start at the same time. When
769-- relative timing among all computations is important or when we need to start
770-- all computations at once for any reason 'Parallel' style must be used
773-- 'Async' style should be preferred over 'Parallel' or 'WAsync' unless you
774-- really need those. It utilizes the resources optimally. It should be used
775-- when we know that the computations can run in parallel but we do not care if
776-- they actually run in parallel or not, that decision can be left to the
777-- scheduler based on demand. Also, note that this operator can be used to fold
778-- infinite number of streams in contrast to the 'Parallel' or 'WAsync' styles,
779-- because it does not require us to run all of them at the same time in a fair
784-- The 'Semigroup' operation '<>' of the 'WAsync' type combines two streams in
785-- a concurrent manner using /breadth first traversal/. We use the 'wAsyncly'
786-- type combinator to effect 'WAsync' style of composition. We can also use the
787-- 'WAsync' type annotation for the stream to achieve the same effect.
789-- When streams with multiple elements are combined in this manner, we traverse
790-- all the streams concurrently in a breadth first manner i.e. one action from
791-- each stream is performed and yielded to the resulting stream before we come
792-- back to the first stream again and so on. Even though we execute the actions
793-- in a breadth first order the outputs are consumed on a first come first
796-- In the following example we can see that outputs are produced in the breadth
797-- first traversal order but this is not guaranteed.
800-- main = 'runStream' . 'wAsyncly' $ (serially $ print 1 |: print 2 |: nil) <> (serially $ print 3 |: print 4 |: nil)
809-- The polymorphic version of the binary operation '<>' of the 'WAsync' type is
810-- 'wAsync'. We can use 'wAsync' to join streams using a breadth first
811-- concurrent traversal irrespective of the type, notice that we have not used
812-- the 'wAsyncly' combinator in the following example:
815-- main = 'runStream' $ delay 3 \`wAsync` delay 2 \`wAsync` delay 1
818-- ThreadId 42: Delay 1
819-- ThreadId 41: Delay 2
820-- ThreadId 40: Delay 3
823-- Since the concurrency provided by this style is demand driven it may not
824-- be used when the composed computations start timers that are relative to
825-- each other because all computations may not be started at the same time and
826-- therefore timers in all of them may not start at the same time. When
827-- relative timing among all computations is important or when we need to start
828-- all computations at once for any reason 'Parallel' style must be used
834-- The 'Semigroup' operation '<>' of the 'Parallel' type combines the two
835-- streams in a fairly concurrent manner with round robin scheduling. We use
836-- the 'parallely' type combinator to effect 'Parallel' style of composition.
837-- We can also use the 'Parallel' type annotation for the stream type to
838-- acheive the same effect.
840-- When two streams with multiple elements are combined in this manner, the
841-- monadic actions in both the streams are performed concurrently with a fair
842-- round robin scheduling. The outputs are yielded in the order in which the
843-- actions complete. This is pretty similar to the 'WAsync' type, the
844-- difference is that 'WAsync' is adaptive to the consumer demand and may or
845-- may not execute all actions in parallel depending on the demand, whereas
846-- 'Parallel' runs all the streams in parallel irrespective of the demand.
848-- The following example sends a query to all the three search engines in
849-- parallel and prints the name of the search engines in the order in which the
854-- import qualified Streamly.Prelude as S
855-- import Network.HTTP.Simple
857-- main = 'runStream' . 'parallely' $ google \<> bing \<> duckduckgo
859-- google = get "https://www.google.com/search?q=haskell"
860-- bing = get "https://www.bing.com/search?q=haskell"
861-- duckduckgo = get "https://www.duckduckgo.com/?q=haskell"
862-- get s = S.'once' (httpNoBody (parseRequest_ s) >> putStrLn (show s))
865-- The polymorphic version of the binary operation '<>' of the 'Parallel' type
866-- is 'parallel'. We can use 'parallel' to join streams in a fairly concurrent
867-- manner irrespective of the type, notice that we have not used the
868-- 'parallely' combinator in the following example:
871-- main = 'runStream' $ delay 3 \`parallel` delay 2 \`wAsync` delay 1
874-- ThreadId 42: Delay 1
875-- ThreadId 41: Delay 2
876-- ThreadId 40: Delay 3
879-- Note that this style of composition cannot be used to combine infinite
880-- number of streams, as it will lead to an infinite sized scheduling queue.
885-- The 'mkAsync' API can be used to create references to asynchronously running
886-- stream computations. We can then use 'uncons' to explore the streams
887-- arbitrarily and then recompose individual elements to create a new stream.
888-- This way we can dynamically decide which stream to explore at any given
889-- time. Take an example of a merge sort of two sorted streams. We need to
890-- keep consuming items from the stream which has the lowest item in the sort
891-- order. This can be achieved using async references to streams. See
892-- "MergeSort.hs" in the examples directory.
896-- We can use 'Monoid' instances to fold a container of streams in the desired
897-- style using 'fold' or 'foldMap'. We have also provided some fold utilities
898-- to fold streams using the polymorphic combine operations:
900-- * 'foldWith' is like 'fold', it folds a 'Foldable' container of streams
901-- using the given composition operator.
902-- * 'foldMapWith' is like 'foldMap', it folds like @foldWith@ but also maps a
903-- function before folding.
904-- * 'forEachWith' is like @foldMapwith@ but the container argument comes before
905-- the function argument.
907-- All of the following are equivalent and start ten concurrent tasks each with
908-- a delay from 1 to 10 seconds, resulting in the printing of each number every
913-- import qualified "Streamly.Prelude" as S
914-- import Control.Concurrent
917-- 'runStream' $ 'asyncly' $ foldMap delay [1..10]
918-- 'runStream' $ 'foldWith' 'async' (map delay [1..10])
919-- 'runStream' $ 'foldMapWith' 'async' delay [1..10]
920-- 'runStream' $ 'forEachWith' 'async' [1..10] delay
921-- where delay n = S.'once' $ threadDelay (n * 1000000) >> print n
926-- Till now we discussed ways to apply transformations on a stream or to merge
927-- streams together to create another stream. We mentioned earlier that
928-- transforming a stream is similar to a @for@ loop in the imperative paradigm.
929-- We will now discuss the concept of a nested composition of streams which is
930-- analogous to nested @for@ loops in the imperative paradigm. Functional
931-- programmers call this style of composition a list transformer or @ListT@.
932-- Logic programmers call it a logic monad or non-deterministic composition,
933-- but for ordinary imperative minded people like me it is easier to think in
934-- terms of good old nested @for@ loops.
938-- In functional programmer's parlance the 'Monad' instances of different
939-- 'IsStream' types implement non-determinism, exploring all possible
940-- combination of choices from both the streams. From an imperative
941-- programmer's point of view it behaves like nested loops i.e. for each
942-- element in the first stream and for each element in the second stream
943-- execute the body of the loop.
945-- The 'Monad' instances of 'Serial', 'WSerial', 'Async' and 'WAsync'
946-- stream types support different flavors of nested looping. In other words,
947-- they are all variants of list transformer. The nesting behavior of these
948-- types correspond exactly to the way they merge streams as we discussed in
949-- the previous section.
954-- The 'Monad' composition of the 'Serial' type behaves like a standard list
955-- transformer. This is the default when we do not use an explicit type
956-- combinator. However, the 'serially' type combinator can be used to switch to
957-- this style of composition. We will see how this style of composition works
958-- in the following examples.
960-- Let's start with an example with a simple @for@ loop without any nesting.
961-- For simplicity of illustration we are using streams of pure values in all
962-- the examples. However, the streams could also be made of monadic actions
967-- import qualified "Streamly.Prelude" as S
969-- main = 'runStream' $ do
970-- x <- S.'fromFoldable' [3,2,1]
974-- ThreadId 30: Delay 3
975-- ThreadId 30: Delay 2
976-- ThreadId 30: Delay 1
979-- As we can see, the code after the @fromFoldable@ statement is run three
980-- times, once for each value of @x@ drawn from the stream. All the three
981-- iterations are serial and run in the same thread one after another. In
982-- imperative terms this is equivalent to a @for@ loop with three iterations.
984-- A console echo loop copying standard input to standard output can simply be
989-- import qualified "Streamly.Prelude" as S
991-- main = 'runStream' $ forever $ S.once getLine >>= S.once . putStrLn
994-- When multiple streams are composed using this style they nest in a DFS
995-- manner i.e. nested iterations of a loop are executed before we proceed to
996-- the next iteration of the parent loop. This behaves just like nested @for@
997-- loops in imperative programming.
1001-- import qualified "Streamly.Prelude" as S
1003-- main = 'runStream' $ do
1004-- x <- S.'fromFoldable' [1,2]
1005-- y <- S.'fromFoldable' [3,4]
1006-- S.'once' $ putStrLn $ show (x, y)
1015-- Notice that this is analogous to merging streams of type 'Serial' or merging
1016-- streams using 'serial'.
1020-- The 'Monad' composition of 'Ahead' type behaves just like 'Serial' except
1021-- that it can speculatively perform a bounded number of next iterations of a
1022-- loop concurrently.
1024-- The 'aheadly' type combinator can be used to switch to this style of
1025-- composition. Alternatively, a type annotation can be used to specify the
1026-- type of the stream as 'Ahead'.
1030-- import "Streamly.Prelude" as S
1032-- comp = 'toList' . 'aheadly' $ do
1033-- x <- S.'fromFoldable' [3,2,1]
1034-- delay x >> return x
1036-- main = comp >> print
1039-- ThreadId 40: Delay 1
1040-- ThreadId 39: Delay 2
1041-- ThreadId 38: Delay 3
1045-- This code finishes in 3 seconds, 'Serial' would take 6 seconds. As we can
1046-- see all the three iterations are concurrent and run in different threads,
1047-- however, the results are returned in the serial order.
1049-- Concurrency is demand driven, when multiple streams are composed using this
1050-- style, the iterations are executed in a depth first manner just like
1051-- 'Serial' i.e. nested iterations are executed before we proceed to the next
1052-- outer iteration. The only difference is that we may execute multiple future
1053-- iterations concurrently and keep the results ready.
1056-- $concurrentNesting
1058-- The 'Monad' composition of 'Async' type can perform the iterations of a
1059-- loop concurrently. Concurrency is demand driven i.e. more concurrent
1060-- iterations are started only if the previous iterations are not able to
1061-- produce enough output for the consumer of the output stream. This works
1062-- exactly the same way as the merging of two streams 'asyncly' works.
1063-- This is the concurrent analogue of 'Serial' style monadic composition.
1065-- The 'asyncly' type combinator can be used to switch to this style of
1066-- composition. Alternatively, a type annotation can be used to specify the
1067-- type of the stream as 'Async'.
1071-- import "Streamly.Prelude"
1073-- main = 'runStream' . 'asyncly' $ do
1074-- x <- S.'fromFoldable' [3,2,1]
1078-- ThreadId 40: Delay 1
1079-- ThreadId 39: Delay 2
1080-- ThreadId 38: Delay 3
1083-- As we can see the code after the @fromFoldable@ statement is run three
1084-- times, once for each value of @x@. All the three iterations are concurrent
1085-- and run in different threads. The iteration with least delay finishes first.
1086-- When compared to imperative programming, this can be viewed as a @for@ loop
1087-- with three concurrent iterations.
1089-- Concurrency is demand driven just as in the case of 'async' merging.
1090-- When multiple streams are composed using this style, the iterations are
1091-- triggered in a depth first manner just like 'Serial' i.e. nested iterations are
1092-- executed before we proceed to the next iteration at higher level. However,
1093-- unlike 'Serial' more than one iterations may be started concurrently based
1094-- on the demand from the consumer of the stream.
1098-- import qualified "Streamly.Prelude" as S
1100-- main = 'runStream' . 'asyncly' $ do
1101-- x <- S.'fromFoldable' [1,2]
1102-- y <- S.'fromFoldable' [3,4]
1103-- S.'once' $ putStrLn $ show (x, y)
1112-- $interleavedNesting
1114-- The 'Monad' composition of 'WSerial' type interleaves the iterations of
1115-- outer and inner loops in a nested loop composition. This works exactly the
1116-- same way as the merging of two streams in 'wSerially' fashion works. The
1117-- 'wSerially' type combinator can be used to switch to this style of
1118-- composition. Alternatively, a type annotation can be used to specify the
1119-- type of the stream as 'WSerial'.
1123-- import qualified "Streamly.Prelude" as S
1125-- main = 'runStream' . 'wSerially' $ do
1126-- x <- S.'fromFoldable' [1,2]
1127-- y <- S.'fromFoldable' [3,4]
1128-- S.once $ putStrLn $ show (x, y)
1140-- Just like 'Async' the 'Monad' composition of 'WAsync' runs the iterations of
1141-- a loop concurrently. The difference is in the nested loop behavior. The
1142-- nested loops in this type are traversed and executed in a breadth first
1143-- manner rather than the depth first manner of 'Async' style.
1144-- The loop nesting works exactly the same way as the merging of streams
1145-- 'wAsyncly' works. The 'wAsyncly' type combinator can be used to switch to
1146-- this style of composition. Alternatively, a type annotation can be used to
1147-- specify the type of the stream as 'WAsync'.
1151-- import qualified "Streamly.Prelude" as S
1153-- main = 'runStream' . 'wAsyncly' $ do
1154-- x <- S.'fromFoldable' [1,2]
1155-- y <- S.'fromFoldable' [3,4]
1156-- S.'once' $ putStrLn $ show (x, y)
1167-- Just like 'Async' or 'WAsync' the 'Monad' composition of 'Parallel' runs the
1168-- iterations of a loop concurrently. The difference is in the nested loop
1169-- behavior. The streams at each nest level is run fully concurrently
1170-- irrespective of the demand. The loop nesting works exactly the same way as
1171-- the merging of streams 'parallely' works. The 'parallely' type combinator
1172-- can be used to switch to this style of composition. Alternatively, a type
1173-- annotation can be used to specify the type of the stream as 'Parallel'.
1177-- import qualified "Streamly.Prelude" as S
1179-- main = 'runStream' . 'parallely' $ do
1180-- x <- S.'fromFoldable' [3,2,1]
1184-- ThreadId 40: Delay 1
1185-- ThreadId 39: Delay 2
1186-- ThreadId 38: Delay 3
1191-- Streamly code is usually written in a way that is agnostic of the
1192-- specific monadic composition type. We use a polymorphic type with a
1193-- 'IsStream' type class constraint. When running the stream we can choose the
1194-- specific mode of composition. For example take a look at the following code.
1198-- import qualified "Streamly.Prelude" as S
1200-- composed :: (IsStream t, Monad (t IO)) => t IO ()
1205-- S.'once' $ putStrLn $ show (sz, cl, sh)
1209-- sizes = S.'fromFoldable' [1, 2, 3]
1210-- colors = S.'fromFoldable' ["red", "green", "blue"]
1211-- shapes = S.'fromFoldable' ["triangle", "square", "circle"]
1214-- Now we can interpret this in whatever way we want:
1217-- main = 'runStream' . 'serially' $ composed
1218-- main = 'runStream' . 'wSerially' $ composed
1219-- main = 'runStream' . 'asyncly' $ composed
1220-- main = 'runStream' . 'wAsyncly' $ composed
1221-- main = 'runStream' . 'parallely' $ composed
1224-- As an exercise try to figure out the output of this code for each mode of
1229-- 'fmap' transforms a stream by mapping a function on all elements of the
1230-- stream. 'fmap' behaves in the same way for all stream types, it is always
1235-- import qualified "Streamly.Prelude" as S
1237-- main = (S.'toList' $ fmap show $ S.'fromFoldable' [1..10]) >>= print
1240-- Also see the 'mapM' and 'sequence' functions for mapping actions, in the
1241-- "Streamly.Prelude" module.
1245-- Applicative is precisely the same as the 'ap' operation of 'Monad'. For
1246-- zipping applicatives separate types 'ZipSerial' and 'ZipAsync' are
1249-- The following example uses the 'Serial' applicative, it runs all iterations
1250-- serially and takes a total 17 seconds (1 + 3 + 4 + 2 + 3 + 4):
1254-- import qualified "Streamly.Prelude" as S
1255-- import Control.Concurrent
1259-- d n = delay n >> return n
1261-- main = (S.'toList' . 'serially' $ (,) \<$> s1 \<*> s2) >>= print
1264-- ThreadId 36: Delay 1
1265-- ThreadId 36: Delay 3
1266-- ThreadId 36: Delay 4
1267-- ThreadId 36: Delay 2
1268-- ThreadId 36: Delay 3
1269-- ThreadId 36: Delay 4
1270-- [(1,3),(1,4),(2,3),(2,4)]
1273-- Similalrly 'WSerial' applicative runs the iterations in an interleaved
1274-- order but since it is serial it takes a total of 17 seconds:
1277-- main = (S.'toList' . 'wSerially' $ (,) \<$> s1 \<*> s2) >>= print
1280-- ThreadId 36: Delay 1
1281-- ThreadId 36: Delay 3
1282-- ThreadId 36: Delay 2
1283-- ThreadId 36: Delay 3
1284-- ThreadId 36: Delay 4
1285-- ThreadId 36: Delay 4
1286-- [(1,3),(2,3),(1,4),(2,4)]
1289-- 'Async' can run the iterations concurrently and therefore takes a total
1290-- of 10 seconds (1 + 2 + 3 + 4):
1293-- main = (S.'toList' . 'asyncly' $ (,) \<$> s1 \<*> s2) >>= print
1296-- ThreadId 34: Delay 1
1297-- ThreadId 36: Delay 2
1298-- ThreadId 35: Delay 3
1299-- ThreadId 36: Delay 3
1300-- ThreadId 35: Delay 4
1301-- ThreadId 36: Delay 4
1302-- [(1,3),(2,3),(1,4),(2,4)]
1305-- Similalrly 'WAsync' as well can run the iterations concurrently and
1306-- therefore takes a total of 10 seconds (1 + 2 + 3 + 4):
1309-- main = (S.'toList' . 'wAsyncly' $ (,) \<$> s1 \<*> s2) >>= print
1312-- ThreadId 34: Delay 1
1313-- ThreadId 36: Delay 2
1314-- ThreadId 35: Delay 3
1315-- ThreadId 36: Delay 3
1316-- ThreadId 35: Delay 4
1317-- ThreadId 36: Delay 4
1318-- [(1,3),(2,3),(1,4),(2,4)]
1323-- Zipping is a special transformation where the corresponding elements of two
1324-- streams are combined together using a zip function producing a new stream of
1325-- outputs. Two different types are provided for serial and concurrent zipping.
1326-- These types provide an applicative instance that can be used to lift
1327-- functions to zip the argument streams.
1328-- Also see the zipping functions in the "Streamly.Prelude" module.
1332-- The applicative instance of 'ZipSerial' type zips streams serially.
1333-- 'zipSerially' type combinator can be used to switch to serial applicative
1338-- import qualified "Streamly.Prelude" as S
1339-- import Control.Concurrent
1341-- d n = delay n >> return n
1342-- s1 = 'serially' $ d 1 <> d 2
1343-- s2 = 'serially' $ d 3 <> d 4
1345-- main = (S.'toList' . 'zipSerially' $ (,) \<$> s1 \<*> s2) >>= print
1348-- This takes total 10 seconds to zip, which is (1 + 2 + 3 + 4) since
1349-- everything runs serially:
1352-- ThreadId 29: Delay 1
1353-- ThreadId 29: Delay 3
1354-- ThreadId 29: Delay 2
1355-- ThreadId 29: Delay 4
1361-- The applicative instance of 'ZipAsync' type zips streams concurrently.
1362-- 'zipAsyncly' type combinator can be used to switch to parallel applicative
1368-- import qualified "Streamly.Prelude" as S
1369-- import Control.Concurrent
1370-- import System.IO (stdout, hSetBuffering, BufferMode(LineBuffering))
1372-- d n = delay n >> return n
1373-- s1 = 'serially' $ d 1 <> d 2
1374-- s2 = 'serially' $ d 3 <> d 4
1377-- hSetBuffering stdout LineBuffering
1378-- (S.'toList' . 'zipAsyncly' $ (,) \<$> s1 \<*> s2) >>= print
1381-- This takes 7 seconds to zip, which is max (1,3) + max (2,4) because 1 and 3
1382-- are produced concurrently, and 2 and 4 are produced concurrently:
1385-- ThreadId 32: Delay 1
1386-- ThreadId 32: Delay 2
1387-- ThreadId 33: Delay 3
1388-- ThreadId 33: Delay 4
1394-- When writing concurrent programs there are two distinct places where the
1395-- programmer can control the concurrency. First, when /composing/ a stream by
1396-- merging multiple streams we can choose an appropriate sum style operators to
1397-- combine them concurrently or serially. Second, when /processing/ a stream in
1398-- a monadic composition we can choose one of the monad composition types to
1399-- choose the desired type of concurrency.
1401-- In the following example the squares of @x@ and @y@ are computed
1402-- concurrently using the 'async' operation and the square roots of their
1403-- sum are computed serially because of the 'streamly' combinator. We can
1404-- choose different combinators for the monadic processing and the stream
1405-- generation, to control the concurrency. We can also use the 'asyncly'
1406-- combinator instead of explicitly folding with 'async'.
1410-- import qualified "Streamly.Prelude" as S
1411-- import Data.List (sum)
1415-- $ 'serially' -- Serial monadic processing (sqrt below)
1417-- x2 \<- 'forEachWith' 'async' [1..100] $ -- Concurrent @"for"@ loop
1418-- \\x -> return $ x * x -- body of the loop
1419-- y2 \<- 'forEachWith' 'async' [1..100] $
1420-- \\y -> return $ y * y
1421-- return $ sqrt (x2 + y2)
1425-- We can see how this directly maps to the imperative style
1426-- <https://en.wikipedia.org/wiki/OpenMP OpenMP> model, we use combinators
1427-- and operators instead of the ugly pragmas.
1429-- For more concurrent programming examples see,
1430-- "ListDir.hs", "MergeSort.hs" and "SearchQuery.hs" in the examples directory.
1434-- Reactive programming is nothing but concurrent streaming which is what
1435-- streamly is all about. With streamly we can generate streams of events,
1436-- merge streams that are generated concurrently and process events
1437-- concurrently. We can do all this without any knowledge about the specifics
1438-- of the implementation of concurrency. In the following example you will see
1439-- that the code is just regular Haskell code without much streamly APIs used
1440-- (active hyperlinks are the streamly APIs) and yet it is a reactive
1443-- This application has two independent and concurrent sources of event
1444-- streams, @acidRain@ and @userAction@. @acidRain@ continuously generates
1445-- events that deteriorate the health of the character in the game.
1446-- @userAction@ can be "potion" or "quit". When the user types "potion" the
1447-- health improves and the game continues.
1450-- {-\# LANGUAGE FlexibleContexts #-}
1453-- import Control.Concurrent (threadDelay)
1454-- import Control.Monad (when)
1455-- import Control.Monad.IO.Class (MonadIO(..))
1456-- import Control.Monad.State (MonadState, get, modify, runStateT)
1457-- import Data.Semigroup (cycle1)
1459-- data Event = Harm Int | Heal Int | Quit deriving (Show)
1461-- userAction :: MonadIO m => 'SerialT' m Event
1462-- userAction = cycle1 $ liftIO askUser
1465-- command <- getLine
1467-- "potion" -> return (Heal 10)
1468-- "quit" -> return Quit
1469-- _ -> putStrLn "What?" >> askUser
1471-- acidRain :: MonadIO m => 'SerialT' m Event
1472-- acidRain = cycle1 $ liftIO (threadDelay 1000000) >> return (Harm 1)
1474-- game :: ('MonadAsync' m, MonadState Int m) => 'SerialT' m ()
1476-- event \<- userAction \`parallel` acidRain
1478-- Harm n -> modify $ \\h -> h - n
1479-- Heal n -> modify $ \\h -> h + n
1480-- Quit -> fail "quit"
1483-- when (h <= 0) $ fail "You die!"
1484-- liftIO $ putStrLn $ "Health = " ++ show h
1487-- putStrLn "Your health is deteriorating due to acid rain,\\
1488-- \\ type \\"potion\\" or \\"quit\\""
1489-- _ <- runStateT ('runStream' game) 60
1493-- You can also find the source of this example in the examples directory as
1494-- "AcidRain.hs". It has been adapted from Gabriel's
1495-- <https://hackage.haskell.org/package/pipes-concurrency-2.0.8/docs/Pipes-Concurrent-Tutorial.html pipes-concurrency>
1497-- This is much simpler compared to the pipes version because of the builtin
1498-- concurrency in streamly. You can also find a SDL based reactive programming
1499-- example adapted from Yampa in "Streamly.Examples.CirclingSquare".
1503-- Streamly is highly optimized for performance, it is designed for serious
1504-- high performing, concurrent and scalable applications. We have created the
1505-- <https://hackage.haskell.org/package/streaming-benchmarks streaming-benchmarks>
1506-- package which is specifically and carefully designed to measure the
1507-- performance of Haskell streaming libraries fairly and squarely in the right
1508-- way. Streamly performs at par or even better than most streaming libraries
1509-- for serial operations even though it needs to deal with the concurrency
1514-- We can use @unfoldr@ and @uncons@ to convert one streaming type to another.
1516-- Interop with @vector@:
1520-- import qualified Streamly.Prelude as S
1521-- import qualified Data.Vector.Fusion.Stream.Monadic as V
1523-- -- | vector to streamly
1524-- fromVector :: (IsStream t, Monad m) => V.Stream m a -> t m a
1525-- fromVector = S.unfoldrM unconsV
1530-- then return Nothing
1533-- return $ Just (h, V.tail v)
1535-- -- | streamly to vector
1536-- toVector :: Monad m => SerialT m a -> V.Stream m a
1537-- toVector = V.unfoldrM (S.uncons . adapt)
1540-- S.toList (fromVector (V.fromList [1..3])) >>= print
1541-- V.toList (toVector (S.fromFoldable [1..3])) >>= print
1544-- Interop with @pipes@:
1548-- import qualified "Streamly.Prelude" as S
1549-- import qualified Pipes as P
1550-- import qualified Pipes.Prelude as P
1552-- -- | pipes to streamly
1553-- fromPipes :: (IsStream t, Monad m) => P.Producer a m r -> t m a
1554-- fromPipes = S.'unfoldrM' unconsP
1556-- -- Adapt P.next to return a Maybe instead of Either
1557-- unconsP p = P.next p >>= either (\\_ -> return Nothing) (return . Just)
1559-- -- | streamly to pipes
1560-- toPipes :: Monad m => SerialT m a -> P.Producer a m ()
1561-- toPipes = P.unfoldr unconsS
1563-- -- Adapt S.uncons to return an Either instead of Maybe
1564-- unconsS s = S.'uncons' s >>= maybe (return $ Left ()) (return . Right)
1567-- S.'toList' (fromPipes (P.each [1..3])) >>= print
1568-- P.toListM (toPipes (S.'fromFoldable' [1..3])) >>= print
1571-- Interop with @streaming@:
1575-- import qualified "Streamly.Prelude" as S
1576-- import qualified Streaming as SG
1577-- import qualified Streaming.Prelude as SG
1579-- -- | streaming to streamly
1580-- fromStreaming :: (IsStream t, Monad m) => SG.Stream (SG.Of a) m r -> t m a
1581-- fromStreaming = S.unfoldrM SG.uncons
1583-- -- | streamly to streaming
1584-- toStreaming :: Monad m => SerialT m a -> SG.Stream (SG.Of a) m ()
1585-- toStreaming = SG.unfoldr unconsS
1587-- -- Adapt S.uncons to return an Either instead of Maybe
1588-- unconsS s = S.'uncons' s >>= maybe (return $ Left ()) (return . Right)
1591-- S.toList (fromStreaming (SG.each [1..3])) >>= print
1592-- SG.toList (toStreaming (S.fromFoldable [1..3])) >>= print
1595-- Interop with @conduit@:
1599-- import qualified "Streamly.Prelude" as S
1600-- import qualified Data.Conduit as C
1601-- import qualified Data.Conduit.List as C
1602-- import qualified Data.Conduit.Combinators as C
1604-- -- It seems there is no way out of a conduit as it does not provide an
1605-- -- uncons or a tail function. We can convert streamly to conduit though.
1607-- -- | streamly to conduit
1608-- toConduit :: Monad m => SerialT m a -> C.ConduitT i a m ()
1609-- toConduit s = C.unfoldM S.'uncons' s
1612-- C.runConduit (toConduit (S.'fromFoldable' [1..3]) C..| C.sinkList) >>= print
1617-- List transformers and logic programming monads also provide a product style
1618-- composition similar to streamly, however streamly generalizes it with the
1619-- time dimension; allowing streams to be composed in an asynchronous and
1620-- concurrent fashion in many different ways. It also provides multiple
1621-- alternative ways of composing streams e.g. serial, interleaved or
1624-- This seemingly simple addition of asynchronicity and concurrency to product
1625-- style streaming composition unifies a number of disparate abstractions into
1626-- one powerful, concise and elegant abstraction. A wide variety of
1627-- programming problems can be solved elegantly with this abstraction. In
1628-- particular, it unifies three major programming domains namely
1629-- non-deterministic (logic) programming, concurrent programming and functional
1630-- reactive programming. In other words, you can do everything with this one
1631-- abstraction that you could do with the popular libraries listed under these
1632-- categories in the list below.
1635-- +-----------------+----------------+
1636-- | Non-determinism | <https://hackage.haskell.org/package/pipes pipes> |
1637-- | +----------------+
1638-- | | <https://hackage.haskell.org/package/list-t list-t> |
1639-- | +----------------+
1640-- | | <https://hackage.haskell.org/package/logict logict> |
1641-- +-----------------+----------------+
1642-- | Streaming | <https://hackage.haskell.org/package/vector vector> |
1643-- | +----------------+
1644-- | | <https://hackage.haskell.org/package/streaming streaming> |
1645-- | +----------------+
1646-- | | <https://hackage.haskell.org/package/pipes pipes> |
1647-- | +----------------+
1648-- | | <https://hackage.haskell.org/package/conduit conduit> |
1649-- +-----------------+----------------+
1650-- | Concurrency | <https://hackage.haskell.org/package/async async> |
1651-- | +----------------+
1652-- | | <https://hackage.haskell.org/package/transient transient> |
1653-- +-----------------+----------------+
1654-- | FRP | <https://hackage.haskell.org/package/Yampa Yampa> |
1655-- | +----------------+
1656-- | | <https://hackage.haskell.org/package/dunai dunai> |
1657-- | +----------------+
1658-- | | <https://hackage.haskell.org/package/reflex reflex> |
1659-- +-----------------+----------------+
1662-- Streamly is a list-transformer. It provides all the functionality provided
1663-- by any of the list transformer and logic programming packages listed above.
1664-- In addition, Streamly naturally integrates the concurrency dimension to the
1665-- basic list transformer functionality.
1667-- When it comes to streaming, in terms of the streaming API streamly is almost
1668-- identical to the vector package. Streamly, vector and streaming packages all
1669-- represent a stream as data and are therefore similar in the fundamental
1670-- approach to streaming. The fundamental difference is that streamly adds
1671-- concurrency support and the monad instance provides concurrent looping.
1672-- Other streaming libraries like pipes, conduit and machines represent and
1673-- compose stream processors rather than the stream data and therefore fall in
1674-- another class of streaming libraries and have comparatively more complicated
1677-- When it comes to concurrency, streamly can do everything that the @async@
1678-- package can do and more. async provides applicative concurrency whereas
1679-- streamly provides both applicative and monadic concurrency. The
1680-- 'ZipAsync' type behaves like the applicative instance of async. In
1681-- comparison to transient streamly has a first class streaming interface and
1682-- is a monad transformer that can be used universally in any Haskell monad
1683-- transformer stack. Streamly was in fact originally inspired by the
1684-- concurrency implementation in @transient@ though it has no resemblence with
1685-- that and takes a lazy pull approach versus transient's strict push approach.
1687-- The non-determinism, concurrency and streaming combination make streamly a
1688-- strong FRP capable library as well. FRP is fundamentally stream of events
1689-- that can be processed concurrently. The example in this tutorial as well as
1690-- the "Streamly.Examples.CirclingSquare" example from Yampa demonstrate the
1691-- basic FRP capability of streamly. In core concepts streamly is strikingly
1692-- similar to @dunai@. dunai was designed from a FRP perspective and streamly
1693-- was originally designed from a concurrency perspective. However, both have
1694-- similarity at the core.
1698-- * Read the documentation of "Streamly" module
1699-- * Read the documentation of "Streamly.Prelude" module
1700-- * See the examples in the "examples" directory of the package
1701-- * See the tests in the "test" directory of the package
keysesc to close
- j k
- move down, up
- enter
- open
- esc
- back, or up a level
- 1 2 3 4
- readme, files, commits, refs
- /
- filter
- y
- copy the clone command
- c
- compare the chosen commit with another (or shift-click both)
- ?
- show or hide this list