{-# OPTIONS_GHC -fno-warn-unused-imports #-}
-- |
-- Module      : Streamly.Tutorial
-- Copyright   : (c) 2017 Harendra Kumar
--
-- License     : BSD3
-- Maintainer  : harendra.kumar@gmail.com
--
-- Streamly, short for stream concurrently, combines the essence of
-- non-determinism, streaming and concurrency in functional programming.
-- Concurrent and non-concurrent applications are almost indistinguishable,
-- concurrency capability does not at all impact the performance of
-- non-concurrent case.
-- Streaming enables writing modular, composable and scalable applications with
-- ease and concurrency allows you to make them scale and perform well.
-- Streamly enables writing concurrent applications without being aware of
-- threads or synchronization. No explicit thread control is needed, where
-- applicable the concurrency rate is automatically controlled based on the
-- demand by the consumer. However, combinators are provided to fine tune the
-- concurrency control.
-- Streaming and concurrency together enable expressing reactive applications
-- conveniently. See "Streamly.Examples" for a simple SDL based FRP example.
--
-- Streamly streams are very much like the Haskell lists and most of the
-- functions that work on lists have a counterpart that works on streams.
-- However, streamly streams can be generated, consumed or combined
-- concurrently. In this tutorial we will go over the basic concepts and how to
-- use the library.  The documentation of @Streamly@ module has more details on
-- core APIs.  For more APIs for constructing, folding, filtering, mapping and
-- zipping etc. see the documentation of "Streamly.Prelude" module. For
-- examples and other ways to use the library see the module
-- "Streamly.Examples" as well.

module Streamly.Tutorial
    (
    -- * Streams
    -- $streams

    -- ** Generating Streams
    -- $generating

    -- ** Eliminating Streams
    -- $eliminating

    -- * Transforming Streams
    -- $transformation

    -- * Merging Streams
    -- $combining

    -- ** Semigroup Style
    -- $semigroup

    -- *** Serial Composition ('streamly')
    -- $serial

    -- *** Serial Interleaved Composition ('costreamly')
    -- $interleaved

    -- *** Left Biased Concurrent Composition ('coparallely')
    -- $coparallel

    -- *** Round Robin Concurrent Composition ('parallely')
    -- $parallel

    -- *** Custom composition
    -- $custom

    -- ** Monoid Style
    -- $monoid

    -- * Nesting Streams
    -- $nesting

    -- ** Monad
    -- $monad

    -- *** Serial Nesting ('streamly')
    -- $regularSerial

    -- *** Serial Interleaved Nesting ('costreamly')
    -- $interleavedNesting

    -- *** Left Biased Concurrent Nesting ('coparallely')
    -- $concurrentNesting

    -- *** Round Robin Concurrent Nesting ('parallely')
    -- $fairlyConcurrentNesting

    -- *** Exercise
    -- $monadExercise

    -- ** Applicative
    -- $applicative

    -- ** Functor
    -- $functor

    -- * Zipping Streams
    -- $zipping

    -- ** Serial Zipping
    -- $serialzip

    -- ** Parallel Zipping
    -- $parallelzip

    -- * Summary of Compositions
    -- $compositionSummary

    -- * Monad transformers
    -- $monadtransformers

    -- * Concurrent Programming
    -- $concurrent

    -- * Reactive Programming
    -- $reactive

    -- * Performance
    -- $performance

    -- * Interoperation with Streaming Libraries
    -- $interop

    -- * Comparison with Existing Packages
    -- $comparison
    )
where

import Streamly
import Streamly.Prelude
import Data.Semigroup
import Control.Applicative
import Control.Monad
import Control.Monad.IO.Class      (MonadIO(..))
import Control.Monad.Trans.Class   (MonadTrans (lift))

-- $streams
--
-- Streamly provides many different stream types depending on the desired
-- composition style. The simplest type is 'Stream', @Stream a@ represents a
-- stream of elements of type @a@ in IO monad. Streams are very much like lists
-- the only difference is that they are lists of monadic actions instead of
-- pure values. We can call them monadic lists. You can apply all the
-- constructs related to lists to monadic streams as well.

-- $monadtransformers
--
-- To represent streams in an arbitrary monad use the more general monad
-- transformer types for example the monad transformer type corresponding to
-- the 'Stream' type is 'StreamT'.  @StreamT m a@ represents a stream of values
-- of type 'a' in some underlying monad 'm'. For example, @StreamT IO Int@ is a
-- stream of 'Int' in 'IO' monad.  In fact, the type 'Stream' is a synonym for
-- @StreamT IO@.
--
-- Similarly we have monad transformer types for other stream types as well viz.
-- 'CostreamT', 'CoparallelT' and 'ParallelT'.
--
-- To lift a value from an underlying monad in a monad transformer stack into a
-- singleton stream use 'lift' and to lift from an IO action use 'liftIO'.
--
-- @
-- > runStream $ liftIO $ putStrLn "Hello world!"
-- Hello world!
-- > runStream $ lift $ putStrLn "Hello world!"
-- Hello world!
-- @
--

-- $generating
--
-- 'nil' represents an empty stream and 'consM' or its operator form '|:' adds
-- a monadic action at the head of the stream.
--
-- @
-- > toList nil
-- []
-- > toList $ getLine |: getLine |: nil
-- hello
-- world
-- ["hello","world"]
-- @
--
-- To create a singleton stream from a pure value use 'pure' and to create a
-- singleton stream from a monadic action use 'once'.
--
-- @
-- > toList $ pure 1
-- [1]
-- > toList $ once getLine
-- hello
-- ["hello"]
-- @
--
-- To create a stream from pure values in a 'Foldable' container use
-- 'fromFoldable' which is equivalent to a fold using 'cons' and 'nil':
--
-- @
-- > toList $ fromFoldable [1..3]
-- [1,2,3]
-- > toList $ Prelude.foldr cons nil [1..3]
-- [1,2,3]
-- @
--
-- To create a stream from monadic actions in a 'Foldable' container just use a
-- right fold using 'consM' and 'nil':
--
-- @
-- > runStream $ Prelude.foldr (|:) nil [putStr "Hello ", putStrLn "world!"]
-- Hello world!
-- @
--
-- For more ways to construct a stream see the module "Streamly.Prelude".

-- $eliminating
--
-- We have already seen 'runStream' and 'toList' to eliminate a stream in the
-- examples above.  'runStream' runs a stream discarding the results i.e. only
-- for effects.  'toList' runs the stream and collects the results in a list.
--
-- For other ways to eliminate a stream see the @Folding@ section in
-- "Streamly.Prelude" module.

-- $transformation
--
-- Transformation over a stream is the equivalent of a @for@ loop construct in
-- imperative paradigm. We iterate over every element in the stream and perform
-- certain transformations for each element.  Transformations may involve
-- mapping functions over the elements, filtering elements from the stream or
-- folding all the elements in the stream into a single value. Streamly streams
-- are exactly like lists and you can perform all the transformations in the
-- same way as you would on lists.
--
-- Here is a simple console echo program that just echoes every input line,
-- forever:
--
-- @
-- > runStream $ S.repeatM getLine & S.mapM putStrLn
-- @
--
-- The following code snippet reads lines from standard input, filters blank
-- lines, drops the first non-blank line, takes the next two, up cases them,
-- numbers them and prints them:
--
-- @
-- import Streamly
-- import qualified Streamly.Prelude as S
-- import Data.Char (toUpper)
-- import Data.Function ((&))
--
-- main = runStream $
--        S.repeatM getLine
--      & S.filter (not . null)
--      & S.drop 1
--      & S.take 2
--      & fmap (map toUpper)
--      & S.zipWith (\\n s -> show n ++ " " ++ s) (S.fromFoldable [1..])
--      & S.mapM putStrLn
-- @

-- $semigroup
--
-- We can combine two streams into a single stream using semigroup composition
-- operation '<>'.  Streams can be combined in many different ways as described
-- in the following sections, the '<>' operation behaves differently depending
-- on the stream type in effect.  Streamly streams can be of many
-- different types i.e. 'Stream', 'Costream', 'Coparallel', 'Parallel',
-- 'ZipStream', and 'ZipParallel'. The first four are monads whereas the last
-- two are applicatives. These types can be freely converted from one to
-- another to acheive the desired composition style.  To force a particular
-- type of composition we coerce the stream type using the corresponding type
-- adapting combinator from 'streamly', 'costreamly', 'coparallely',
-- 'parallely', 'zipStreamly' or 'zipParallely'. The default stream type is
-- 'Stream' unless you change it by using one of the combinators.
--
-- To illustrate the concurrent vs serial composition aspects, we will use the
-- following @delay@ function to introduce a delay specified in seconds.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"(once)
-- import Control.Concurrent
--
-- delay n = once $ do
--  threadDelay (n * 1000000)
--  tid \<- myThreadId
--  putStrLn (show tid ++ ": Delay " ++ show n)
-- @

-- $serial
--
-- The 'Semigroup' operation '<>' of the 'Stream' type combines the two streams
-- in a serial manner. We use the 'streamly' type combinator to effect 'Stream'
-- style of composition. We can also use the 'Stream' type annotation for the
-- stream type to acheive the same effect.  However, since 'Stream' is the
-- default type unless explicitly specified by using a combinator, we can
-- omit using an explicit combinator or type annotation for this style of
-- composition.
--
-- When two streams with multiple elements are combined in this manner, the
-- monadic actions in the two streams are performed sequentially i.e. first all
-- actions in the first stream are performed sequentially and then all actions
-- in the second stream are performed sequentially.  The following example
-- prints the sequence 1, 2, 3, 4:
--
-- @
-- main = 'runStream' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
-- @
-- @
-- 1
-- 2
-- 3
-- 4
-- @
--
-- All actions in both the streams are performed serially in the same thread.
-- In the following example we can see that all actions are performed in the
-- same thread and take a combined total of 6 seconds:
--
-- @
-- main = 'runStream' $ delay 3 <> delay 2 <> delay 1
-- @
-- @
-- ThreadId 36: Delay 3
-- ThreadId 36: Delay 2
-- ThreadId 36: Delay 1
-- @
--
-- The polymorphic version of the 'Stream' binary operation '<>' is called
-- 'splice'. We can use 'splice' to join streams in a sequential manner
-- irrespective of the type of stream:
--
-- @
-- main = 'runStream' $ (print 1 |: print 2 |: nil) \`splice` (print 3 |: print 4 |: nil)
-- @

-- $interleaved
--
-- The 'Semigroup' operation '<>' of the 'Costream' type combines the two
-- streams in a serial interleaved manner. We use the 'costreamly' type
-- combinator to effect 'Costream' style of composition. We can also use the
-- 'Costream' type annotation for the stream type to acheive the same effect.
--
-- When two streams with multiple elements are combined in this manner, the
-- monadic actions in the two streams are performed in an interleaved manner
-- i.e. one action from the first stream is performed and its output yielded
-- and then one action from the second stream is performed and its output
-- yielded, and so on.  The following example prints the sequence 1, 3, 2, 4
--
-- @
-- main = 'runStream' . 'costreamly' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
-- @
-- @
-- 1
-- 3
-- 2
-- 4
-- @
--
-- Even though the monadic actions of the two streams are performed in an
-- interleaved manner they are all performed serially in the same thread. In
-- the following example we can see that all actions are performed in the same
-- thread and take a combined total of 6 seconds:
--
-- @
-- main = 'runStream' . 'costreamly' $ delay 3 <> delay 2 <> delay 1
-- @
-- @
-- ThreadId 36: Delay 3
-- ThreadId 36: Delay 2
-- ThreadId 36: Delay 1
-- @
--
-- The polymorphic version of the 'Costream' binary operation '<>' is called
-- 'cosplice'. We can use 'cosplice' to join streams in an interleaved manner
-- irrespective of the type, notice that we have not used the 'costreamly'
-- combinator in the following example:
--
-- @
-- main = 'runStream' $ (print 1 |: print 2 |: nil) \`cosplice` (print 3 |: print 4 |: nil)
-- @
-- @
-- 1
-- 3
-- 2
-- 4
-- @
--
-- Note that this composition cannot be used to fold infinite number of streams
-- since it requires preserving the state until a stream is finished.

-- $coparallel
--
-- The 'Semigroup' operation '<>' of the 'Coparallel' type combines the two
-- streams in a left biased concurrent manner. We use the 'coparallel' type
-- combinator to effect 'Coparallel' style of composition. We can also use the
-- 'Coparallel' type annotation for the stream type to acheive the same effect.
--
-- When two streams with multiple elements are combined in this manner, the
-- monadic actions in the two streams can be performed concurrently
-- /when needed/ with a bias towards the left stream i.e. actions from the left
-- stream are performed sequentially, if the left stream blocks or cannot
-- generate enough output to keep the consumer of the stream busy then it can
-- execute actions from the right stream concurrently. The outputs of the
-- concurrent actions are yielded by the composite stream in the order in which
-- they complete.  This can be thought of as the concurrent analogue of the
-- 'Stream' style serial composition. If concurrent execution is not trigerred
-- by consumer demand then it would execute actions exactly in the same order
-- as 'Stream'.  In the following example the first stream does not block,
-- therefore the first stream is completely exhausted before the second.
--
-- @
-- main = 'runStream' . 'coparallely' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
-- @
-- @
-- 1
-- 2
-- 3
-- 4
-- @
--
-- If the first stream blocks, we can yield from the second. In the example
-- below each yield in the stream has a constant delay of 1 second therefore 1
-- and 3 would be yielded first and then 2 and 4 would be yielded.
--
-- @
-- main = 'runStream' . 'coparallely' $ (p 1 |: p 2 |: nil) <> (p 3 |: p 4 |: nil)
--  where p n = threadDelay 1000000 >> print n
-- @
-- @
-- 1
-- 3
-- 2
-- 4
-- @
--
-- In the following example we can see that new threads are started when a
-- computation blocks.  Notice that the output from the stream with the
-- shortest delay is printed first:
--
-- @
-- main = 'runStream' . 'coparallely' $ delay 3 '<>' delay 2 '<>' delay 1
-- @
-- @
-- ThreadId 42: Delay 1
-- ThreadId 41: Delay 2
-- ThreadId 40: Delay 3
-- @
--
-- When we have a tree of computations composed using this style, the tree is
-- traversed in DFS style just like the serial style, the only difference is
-- that here we can move on to executing the next stream if a stream blocks.
-- However, we will not start new threads if we have sufficient output to
-- saturate the consumer.  This is why we call it left-biased demand driven or
-- adaptive concurrency style, the concurrency tends to stay on the left side
-- of the composition as long as possible. More threads are started based on
-- the pull rate of the consumer. The following example prints an output every
-- second as all of the actions are concurrent.
--
-- @
-- main = 'runStream' . 'coparallely' $ (delay 1 <> delay 2) <> (delay 3 <> delay 4)
-- @
-- @
-- 1
-- 2
-- 3
-- 4
-- @
--
-- All the computations may even run in a single thread when more threads are
-- not needed. As you can see, in the following example the computations are
-- run in a single thread one after another, because none of them blocks.
-- However, if the thread consuming the stream were faster than the producer
-- then it would have started parallel threads for each computation to keep up
-- even if none of them blocks:
--
-- @
-- main = 'runStream' . 'coparallely' $ traced (sqrt 9) '<>' traced (sqrt 16) '<>' traced (sqrt 25)
--  where traced m = once (myThreadId >>= print) >> return m
-- @
-- @
-- ThreadId 40
-- ThreadId 40
-- ThreadId 40
-- @
--
-- Note that the order of printing in the above examples may change due to
-- variations in scheduling latencies for concurrent threads.
--
-- The polymorphic version of the 'Coparallel' binary operation '<>' is called
-- 'coparallel'. We can use 'coparallel' to join streams in a left biased
-- adaptively concurrent manner irrespective of the type, notice that we have
-- not used the 'coparallely' combinator in the following example:
--
-- @
-- main = 'runStream' $ delay 3 \`coparallel` delay 2 \`coparallel` delay 1
-- @
-- @
-- ThreadId 42: Delay 1
-- ThreadId 41: Delay 2
-- ThreadId 40: Delay 3
-- @
--
-- Since the concurrency provided by this operator is demand driven it cannot
-- be used when the composed computations start timers that are relative to
-- each other because all computations may not be started at the same time and
-- therefore timers in all of them may not start at the same time.  When
-- relative timing among all computations is important or when we need to start
-- all computations at once for any reason 'Parallel' style must be used
-- instead.
--
-- 'Coparallel' style should be preferred over 'Parallel' unless you really
-- need 'Parallel'. It utilizes the resources optimally. It should be used when
-- we know that the computations can run in parallel but we do not care if they
-- actually run in parallel or not, that decision can be left to the scheduler
-- based on demand. Also, note that this operator can be used to fold infinite
-- number of streams in contrast to the 'Parallel' style, because it does not
-- require us to run all of them at the same time in a fair manner.

-- $parallel
--
-- The 'Semigroup' operation '<>' of the 'Parallel' type combines the two
-- streams in a fairly concurrent manner with round robin scheduling. We use
-- the 'parallel' type combinator to effect 'Parallel' style of composition. We
-- can also use the 'Parallel' type annotation for the stream type to acheive
-- the same effect.
--
-- When two streams with multiple elements are combined in this manner, the
-- monadic actions in both the streams are performed concurrently with a fair
-- round robin scheduling.  The outputs are yielded in the order in which the
-- actions complete.  This can be thought of as the concurrent analogue of the
-- 'Costream' style serial composition.
--
-- The following example sends a query to all the three search engines in
-- parallel and prints the name of the search engines in the order in which the
-- responses arrive:
--
-- @
-- import "Streamly"
-- import Streamly.Prelude (once)
-- import Network.HTTP.Simple
--
-- main = 'runStream' . 'parallely' $ google \<> bing \<> duckduckgo
--     where
--         google     = get "https://www.google.com/search?q=haskell"
--         bing       = get "https://www.bing.com/search?q=haskell"
--         duckduckgo = get "https://www.duckduckgo.com/?q=haskell"
--         get s = once (httpNoBody (parseRequest_ s) >> putStrLn (show s))
-- @
--
-- In the following example we can see that the fairly parallel composition
-- yields outputs in the same order as the 'Costream' composition, though this
-- is not always guaranteed because of thread scheduling variances.
--
-- @
-- main = 'runStream' . 'parallely' $ (print 1 |: print 2 |: nil) <> (print 3 |: print 4 |: nil)
-- @
-- @
-- 1
-- 3
-- 2
-- 4
-- @
--
-- The polymorphic version of the 'Parallel' binary operation '<>' is called
-- 'parallel'. We can use 'parallel' to join streams in a fairly
-- concurrent manner irrespective of the type, notice that we have
-- not used the 'parallely' combinator in the following example:
--
-- @
-- main = 'runStream' $ delay 3 \`parallel` delay 2 \`parallel` delay 1
-- @
-- @
-- ThreadId 42: Delay 1
-- ThreadId 41: Delay 2
-- ThreadId 40: Delay 3
-- @
--
-- Note that this style of composition cannot be used to combine infinite
-- number of streams, as it will lead to an infinite sized scheduling queue.
--

-- $custom
--
-- The 'async' API can be used to create references to asynchronously running
-- stream computations. We can then use 'uncons' to explore the streams
-- arbitrarily and then recompose individual elements to create a new stream.
-- This way we can dynamically decide which stream to explore at any given
-- time.  Take an example of a merge sort of two sorted streams. We need to
-- keep consuming items from the stream which has the lowest item in the sort
-- order.  This can be achieved using async references to streams. See
-- "Streamly.Examples.MergeSortedStreams".

-- $monoid
--
-- We can use 'Monoid' instances to fold a container of streams in the desired
-- style using 'fold' or 'foldMap'.  We have also provided some fold utilities
-- to fold streams using the polymorphic combine operations:
--
-- * 'foldWith' is like 'fold', it folds a 'Foldable' container of streams
-- using the given composition operator.
-- * 'foldMapWith' is like 'foldMap', it folds like @foldWith@ but also maps a
-- function before folding.
-- * 'forEachWith' is like @foldMapwith@ but the container argument comes before
-- the function argument.
--
-- All of the following are equivalent and start ten concurrent tasks each with
-- a delay from 1 to 10 seconds, resulting in the printing of each number every
-- second:
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
-- import Control.Concurrent
--
-- main = do
--  runStream $ parallely $ foldMap delay [1..10]
--  runStream $ 'foldWith'    parallel (map delay [1..10])
--  runStream $ 'foldMapWith' parallel delay [1..10]
--  runStream $ 'forEachWith' parallel [1..10] delay
--  where delay n = once $ threadDelay (n * 1000000) >> print n
-- @

-- $nesting
--
-- Till now we discussed ways to apply transformations on a stream or to merge
-- streams together to create another stream. We mentioned earlier that
-- transforming a stream is similar to a @for@ loop in the imperative paradigm.
-- We will now discuss the concept of a nested composition of streams which is
-- analogous to nested @for@ loops in the imperative paradigm. Functional
-- programmers call this style of composition a list transformer or @ListT@.
-- Logic programmers call it a logic monad or non-deterministic composition,
-- but for ordinary imperative minded people like me it is easier to think in
-- terms of good old nested @for@ loops.
--
-- $monad
--
-- In functional programmer's parlance the 'Monad' instances of different
-- 'IsStream' types implement non-determinism, exploring all possible
-- combination of choices from both the streams. From an imperative
-- programmer's point of view it behaves like nested loops i.e.  for each
-- element in the first stream and for each element in the second stream
-- execute the body of the loop.
--
-- The 'Monad' instances of 'Stream', 'Costream', 'Coparallel' and 'Parallel'
-- stream types support different flavors of nested looping.  In other words,
-- they are all variants of list transformer.  The nesting behavior of these
-- types exactly correspond to the way they merge streams as we discussed in
-- the previous section.
--

-- $regularSerial
--
-- The 'Monad' composition of 'Stream' type behaves like a standard list
-- transformer. This is the default when we do not use an explicit type
-- combinator. However, the 'streamly' type combinator can be used to switch to
-- this style of composition. We will see how this style of composition works
-- in the following examples.
--
-- Let's start with an example with a simple @for@ loop without any nesting.
-- For simplicity of illustration we are using streams of pure values in all
-- the examples.  However, the streams could also be made of monadic actions
-- instead.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
--
-- main = 'runStream' $ do
--     x <- 'fromFoldable' [3,2,1]
--     delay x
-- @
-- @
-- ThreadId 30: Delay 3
-- ThreadId 30: Delay 2
-- ThreadId 30: Delay 1
-- @
--
-- As you can see, the code after the @fromFoldable@ statement is run three
-- times, once for each value of @x@ drawn from the stream. All the three
-- iterations are serial and run in the same thread one after another. In
-- imperative terms this is equivalent to a @for@ loop with three iterations.
--
-- A console echo loop copying standard input to standard output can simply be
-- written like this:
--
-- @
-- import "Streamly"
--
-- main = 'runStream' $ forever $ once getLine >>= once . putStrLn
-- @
--
-- When multiple streams are composed using this style they nest in a DFS
-- manner i.e. nested iterations of a loop are executed before we proceed to
-- the next iteration of the parent loop. This behaves just like nested @for@
-- loops in imperative programming.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
--
-- main = 'runStream' $ do
--     x <- 'fromFoldable' [1,2]
--     y <- 'fromFoldable' [3,4]
--     once $ putStrLn $ show (x, y)
-- @
-- @
-- (1,3)
-- (1,4)
-- (2,3)
-- (2,4)
-- @
--
-- Notice that this is analogous to merging streams of type 'Stream' or merging
-- streams using 'splice'.

-- $concurrentNesting
--
-- The 'Monad' composition of 'Coparallel' type can perform the iterations of a
-- loop concurrently.  Concurrency is demand driven i.e. more concurrent
-- iterations are started only if the previous iterations are not able to
-- produce enough output for the consumer of the output stream.  This works
-- exactly the same way as the merging of two streams 'coparallely' works.
-- This is the concurrent analogue of 'Stream' style monadic composition.
--
-- The 'coparallely' type combinator can be used to switch to this
-- style of composition. Alternatively, a type annotation can be used to
-- specify the type of the stream as 'Coparallel'.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
--
-- main = 'runStream' . 'coparallely' $ do
--     x <- 'fromFoldable' [3,2,1]
--     delay x
-- @
-- @
-- ThreadId 40: Delay 1
-- ThreadId 39: Delay 2
-- ThreadId 38: Delay 3
-- @
--
-- As you can see the code after the @fromFoldable@ statement is run three
-- times, once for each value of @x@. All the three iterations are concurrent
-- and run in different threads. The iteration with least delay finishes first.
-- When compared to imperative programming, this can be viewed as a @for@ loop
-- with three concurrent iterations.
--
-- Concurrency is demand driven just as in the case of 'coparallel' merging.
-- When multiple streams are composed using this style, the iterations are
-- triggered in a DFS manner just like 'Stream' i.e. nested iterations are
-- executed before we proceed to the next iteration at higher level. However,
-- unlike 'Stream' more than one iterations may be started concurrently based
-- on the demand from the consumer.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
--
-- main = 'runStream' . 'coparallely' $ do
--     x <- 'fromFoldable' [1,2]
--     y <- 'fromFoldable' [3,4]
--     once $ putStrLn $ show (x, y)
-- @
-- @
-- (1,3)
-- (1,4)
-- (2,3)
-- (2,4)
-- @

-- $interleavedNesting
--
-- The 'Monad' composition of 'Costream' type interleaves the iterations of
-- outer and inner loops in a nested loop composition. This works exactly the
-- same way as the merging of two streams in 'costreamly' fashion works.  The
-- 'costreamly' type combinator can be used to switch to this style of
-- composition. Alternatively, a type annotation can be used to specify the
-- type of the stream as 'Costream'.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
--
-- main = 'runStream' . 'costreamly' $ do
--     x <- 'fromFoldable' [1,2]
--     y <- 'fromFoldable' [3,4]
--     once $ putStrLn $ show (x, y)
-- @
-- @
-- (1,3)
-- (2,3)
-- (1,4)
-- (2,4)
-- @
--

-- $fairlyConcurrentNesting
--
-- The 'Monad' composition of 'Parallel' type starts all the iterations of a
-- loop concurrently instead of demand driven as in the case of 'Coparallel'.
-- This works exactly the same way as the merging streams 'parallely' works.
-- The 'parallely' type combinator can be used to switch to this style of
-- composition. Alternatively, a type annotation can be used to specify the
-- type of the stream as 'Parallel'.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
--
-- main = 'runStream' . 'parallely' $ do
--     x <- 'fromFoldable' [3,2,1]
--     delay x
-- @
-- @
-- ThreadId 40: Delay 1
-- ThreadId 39: Delay 2
-- ThreadId 38: Delay 3
-- @

-- $monadExercise
--
-- The streamly code is usually written in a way that is agnostic of the
-- specific monadic composition type. We use a polymorphic type with a
-- 'IsStream' type class constraint. When running the stream we can choose the
-- specific mode of composition. For example take a look at the following code.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
--
-- composed :: 'IsStream' t => t m a
-- composed = do
--     sz <- sizes
--     cl <- colors
--     sh <- shapes
--     once $ putStrLn $ show (sz, cl, sh)
--
--     where
--
--     sizes  = 'fromFoldable' [1, 2, 3]
--     colors = 'fromFoldable' ["red", "green", "blue"]
--     shapes = 'fromFoldable' ["triangle", "square", "circle"]
-- @
--
-- Now we can interpret this in whatever way we want:
--
-- @
-- main = 'runStream' . 'streamly'    $ composed
-- main = 'runStream' . 'costreamly'  $ composed
-- main = 'runStream' . 'coparallely' $ composed
-- main = 'runStream' . 'parallely'   $ composed
-- @
--
--  As an exercise try to figure out the output of this code for each mode of
--  composition.

-- $functor
--
-- 'fmap' transforms a stream by mapping a function on all elements of the
-- stream. 'fmap' behaves in the same way for all stream types, it is always
-- serial.
--
-- @
-- import "Streamly"
--
-- main = ('toList' $ fmap show $ 'fromFoldable' [1..10]) >>= print
-- @
--
-- Also see the 'mapM' and 'sequence' functions for mapping actions, in the
-- "Streamly.Prelude" module.

-- $applicative
--
-- Applicative is precisely the same as the 'ap' operation of 'Monad'. For
-- zipping applicatives separate types 'ZipStream' and 'ZipParallel' are
-- provided.
--
-- The following example uses the 'Stream' applicative, it runs all iterations
-- serially and takes a total 17 seconds (1 + 3 + 4 + 2 + 3 + 4):
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
-- import Control.Concurrent
--
-- s1 = d 1 <> d 2
-- s2 = d 3 <> d 4
-- d n = delay n >> return n
--
-- main = ('toList' . 'streamly' $ (,) \<$> s1 \<*> s2) >>= print
-- @
-- @
-- ThreadId 36: Delay 1
-- ThreadId 36: Delay 3
-- ThreadId 36: Delay 4
-- ThreadId 36: Delay 2
-- ThreadId 36: Delay 3
-- ThreadId 36: Delay 4
-- [(1,3),(1,4),(2,3),(2,4)]
-- @
--
-- Similalrly 'Costream' applicative runs the iterations in an interleaved
-- order but since it is serial it takes a total of 17 seconds:
--
-- @
-- main = ('toList' . 'costreamly' $ (,) \<$> s1 \<*> s2) >>= print
-- @
-- @
-- ThreadId 36: Delay 1
-- ThreadId 36: Delay 3
-- ThreadId 36: Delay 2
-- ThreadId 36: Delay 3
-- ThreadId 36: Delay 4
-- ThreadId 36: Delay 4
-- [(1,3),(2,3),(1,4),(2,4)]
-- @
--
-- 'Coparallel' can run the iterations concurrently and therefore takes a total
-- of 10 seconds (1 + 2 + 3 + 4):
--
-- @
-- main = ('toList' . 'coparallely' $ (,) \<$> s1 \<*> s2) >>= print
-- @
-- @
-- ThreadId 34: Delay 1
-- ThreadId 36: Delay 2
-- ThreadId 35: Delay 3
-- ThreadId 36: Delay 3
-- ThreadId 35: Delay 4
-- ThreadId 36: Delay 4
-- [(1,3),(2,3),(1,4),(2,4)]
-- @
--
-- Similalrly 'Parallel' as well can run the iterations concurrently and
-- therefore takes a total of 10 seconds (1 + 2 + 3 + 4):
--
-- @
-- main = ('toList' . 'parallely' $ (,) \<$> s1 \<*> s2) >>= print
-- @
-- @
-- ThreadId 34: Delay 1
-- ThreadId 36: Delay 2
-- ThreadId 35: Delay 3
-- ThreadId 36: Delay 3
-- ThreadId 35: Delay 4
-- ThreadId 36: Delay 4
-- [(1,3),(2,3),(1,4),(2,4)]
-- @

-- $compositionSummary
--
-- The following table summarizes the types for monadic compositions and the
-- operators for sum style compositions. This table captures the essence of
-- streamly.
--
-- @
-- +-----+--------------+--------------+
-- |     | Serial       | Concurrent   |
-- +=====+==============+==============+
-- | DFS | 'Stream'       | 'Coparallel'   |
-- |     +--------------+--------------+
-- |     | 'splice'       | 'coparallel'   |
-- +-----+--------------+--------------+
-- | BFS | 'Costream'     | 'Parallel'     |
-- |     +--------------+--------------+
-- |     | 'cosplice'     | 'parallel'     |
-- +-----+--------------+--------------+
-- @

-- $zipping
--
-- Zipping is a special transformation where the corresponding elements of two
-- streams are combined together using a zip function producing a new stream of
-- outputs. Two different types are provided for serial and concurrent zipping.
-- These types provide an applicative instance that can be used to lift
-- functions to zip the argument streams.
-- Also see the zipping functions in the "Streamly.Prelude" module.

-- $serialzip
--
-- The applicative instance of 'ZipStream' type zips streams serially.
-- 'zipStreamly' type combinator can be used to switch to serial applicative
-- zip composition:
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
-- import Control.Concurrent
--
-- d n = delay n >> return n
-- s1 = 'streamly' $ d 1 <> d 2
-- s2 = 'streamly' $ d 3 <> d 4
--
-- main = ('toList' . 'zipStreamly' $ (,) \<$> s1 \<*> s2) >>= print
-- @
--
-- This takes total 10 seconds to zip, which is (1 + 2 + 3 + 4) since
-- everything runs serially:
--
-- @
-- ThreadId 29: Delay 1
-- ThreadId 29: Delay 3
-- ThreadId 29: Delay 2
-- ThreadId 29: Delay 4
-- [(1,3),(2,4)]
-- @

-- $parallelzip
--
-- The applicative instance of 'ZipParallel' type zips streams concurrently.
-- 'zipParallely' type combinator can be used to switch to parallel applicative
-- zip composition:
--
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude"
-- import Control.Concurrent
-- import System.IO (stdout, hSetBuffering, BufferMode(LineBuffering))
--
-- d n = delay n >> return n
-- s1 = 'streamly' $ d 1 <> d 2
-- s2 = 'streamly' $ d 3 <> d 4
--
-- main = do
--     hSetBuffering stdout LineBuffering
--     ('toList' . 'zipParallely' $ (,) \<$> s1 \<*> s2) >>= print
-- @
--
-- This takes 7 seconds to zip, which is max (1,3) + max (2,4) because 1 and 3
-- are produced concurrently, and 2 and 4 are produced concurrently:
--
-- @
-- ThreadId 32: Delay 1
-- ThreadId 32: Delay 2
-- ThreadId 33: Delay 3
-- ThreadId 33: Delay 4
-- [(1,3),(2,4)]
-- @

-- $concurrent
--
-- When writing concurrent programs there are two distinct places where the
-- programmer can control the concurrency. First, when /composing/ a stream by
-- merging multiple streams we can choose an appropriate sum style operators to
-- combine them concurrently or serially. Second, when /processing/ a stream in
-- a monadic composition we can choose one of the monad composition types to
-- choose the desired type of concurrency.
--
-- In the following example the squares of @x@ and @y@ are computed
-- concurrently using the 'coparallel' operation and the square roots of their
-- sum are computed serially because of the 'streamly' combinator. We can
-- choose different combinators for the monadic processing and the stream
-- generation, to control the concurrency.  We can also use the 'coparallely'
-- combinator instead of explicitly folding with 'coparallel'.
--
-- @
-- import "Streamly"
-- import "Streamly.Prelude" (toList)
-- import Data.List (sum)
--
-- main = do
--     z \<-   'toList'
--          $ 'streamly'     -- Serial monadic processing (sqrt below)
--          $ do
--              x2 \<- 'forEachWith' 'coparallel' [1..100] $ -- Concurrent @"for"@ loop
--                          \\x -> return $ x * x  -- body of the loop
--              y2 \<- 'forEachWith' 'coparallel' [1..100] $
--                          \\y -> return $ y * y
--              return $ sqrt (x2 + y2)
--     print $ sum z
-- @
--
-- You can see how this directly maps to the imperative style
-- <https://en.wikipedia.org/wiki/OpenMP OpenMP> model, we use combinators
-- and operators instead of the ugly pragmas.
--
-- For more concurrent programming examples see,
-- "Streamly.Examples.ListDirRecursive", "Streamly.Examples.MergeSortedStreams"
-- and "Streamly.Examples.SearchEngineQuery".

-- $reactive
--
-- Reactive programming is nothing but concurrent streaming which is what
-- streamly is all about. With streamly we can generate streams of events,
-- merge streams that are generated concurrently and process events
-- concurrently. We can do all this without any knowledge about the specifics
-- of the implementation of concurrency. In the following example you will see
-- that the code is just regular Haskell code without much streamly APIs used
-- (active hyperlinks are the streamly APIs) and yet it is a reactive
-- application.
--
-- This application has two independent and concurrent sources of event
-- streams, @acidRain@ and @userAction@. @acidRain@ continuously generates
-- events that deteriorate the health of the character in the game.
-- @userAction@ can be "potion" or "quit". When the user types "potion" the
-- health improves and the game continues.
--
-- @
-- {-\# LANGUAGE FlexibleContexts #-}
--
-- import "Streamly"
-- import Control.Concurrent (threadDelay)
-- import Control.Monad (when)
-- import Control.Monad.State (MonadState, get, modify, runStateT)
-- import Data.Semigroup (cycle1)
--
-- data Event = Harm Int | Heal Int | Quit deriving (Show)
--
-- userAction :: MonadIO m => 'StreamT' m Event
-- userAction = cycle1 $ liftIO askUser
--     where
--     askUser = do
--         command <- getLine
--         case command of
--             "potion" -> return (Heal 10)
--             "quit"   -> return  Quit
--             _        -> putStrLn "What?" >> askUser
--
-- acidRain :: MonadIO m => 'StreamT' m Event
-- acidRain = cycle1 $ liftIO (threadDelay 1000000) >> return (Harm 1)
--
-- game :: ('MonadParallel' m, MonadState Int m) => 'StreamT' m ()
-- game = do
--     event \<- userAction \`parallel` acidRain
--     case event of
--         Harm n -> modify $ \\h -> h - n
--         Heal n -> modify $ \\h -> h + n
--         Quit   -> fail "quit"
--
--     h <- get
--     when (h <= 0) $ fail "You die!"
--     liftIO $ putStrLn $ "Health = " ++ show h
--
-- main = do
--     putStrLn "Your health is deteriorating due to acid rain,\\
--              \\ type \\"potion\\" or \\"quit\\""
--     _ <- runStateT ('runStream' game) 60
--     return ()
-- @
--
-- You can also find the source of this example in
-- "Streamly.Examples.AcidRainGame". It has been adapted from Gabriel's
-- <https://hackage.haskell.org/package/pipes-concurrency-2.0.8/docs/Pipes-Concurrent-Tutorial.html pipes-concurrency>
-- package.
-- This is much simpler compared to the pipes version because of the builtin
-- concurrency in streamly. You can also find a SDL based reactive programming
-- example adapted from Yampa in "Streamly.Examples.CirclingSquare".

-- $performance
--
-- Streamly is highly optimized for performance, it is designed for serious
-- high performing, concurrent and scalable applications. We have created the
-- <https://hackage.haskell.org/package/streaming-benchmarks streaming-benchmarks>
-- package which is specifically and carefully designed to measure the
-- performance of Haskell streaming libraries fairly and squarely in the right
-- way. Streamly performs at par or even better than most streaming libraries
-- for serial operations even though it needs to deal with the concurrency
-- capability.

-- $interop
--
-- We can use @unfoldr@ and @uncons@ to convert one streaming type to another.
--
--  Interop with @vector@:
--
-- @
-- import Streamly
-- import qualified Streamly.Prelude as S
-- import qualified Data.Vector.Fusion.Stream.Monadic as V
--
-- main = do
--     -- streamly to vector
--     V.toList (V.unfoldrM S.uncons (S.fromFoldable [1..3])) >>= print
--
--     -- vector to streamly
--     S.toList (S.unfoldrM unconsV (V.fromList [1..3])) >>= print
--
--     where
--     unconsV v = do
--         r <- V.null v
--         if r
--         then return Nothing
--         else do
--             h <- V.head v
--             return $ Just (h, V.tail v)
-- @
--
--  Interop with @pipes@:
--
-- @
-- import "Streamly"
-- import qualified "Streamly.Prelude" as S
-- import qualified Pipes as P
-- import qualified Pipes.Prelude as P
--
-- main = do
--     -- streamly to pipe
--     P.toListM (P.unfoldr unconsS (S.'fromFoldable' [1..3])) >>= print
--
--     -- pipe to streamly
--     S.'toList' (S.'unfoldrM' unconsP (P.each [1..3])) >>= print
--
--     where
--     -- Adapt P.next to return a Maybe instead of Either
--     unconsP p = P.next p >>= either (\\_ -> return Nothing) (return . Just)
--
--     -- Adapt S.uncons to return an Either instead of Maybe
--     unconsS s = S.'uncons' s >>= maybe (return $ Left ()) (return . Right)
-- @
--
-- Interop with @streaming@:
--
-- @
-- import "Streamly"
-- import qualified "Streamly.Prelude" as S
-- import qualified Streaming as SG
-- import qualified Streaming.Prelude as SG
--
-- main = do
--     -- streamly to streaming
--     SG.toList (SG.unfoldr unconsS (S.'fromFoldable' [1..3])) >>= print
--
--     -- streaming to streamly
--     S.'toList' (S.unfoldrM SG.uncons (SG.each [1..3])) >>= print
--
--     where
--
--     -- Adapt S.uncons to return an Either instead of Maybe
--     unconsS s = S.'uncons' s >>= maybe (return $ Left ()) (return . Right)
-- @
--
-- Interop with @conduit@:
--
-- @
-- import "Streamly"
-- import qualified "Streamly.Prelude" as S
-- import qualified Data.Conduit as C
-- import qualified Data.Conduit.List as C
-- import qualified Data.Conduit.Combinators as C
--
-- main = do
--  -- streamly to conduit
--  C.runConduit (C.unfoldM S.'uncons' (S.'fromFoldable' [1..3]) C..| C.sinkList) >>= print
--
--  -- It seems there is no way out of a conduit as it does not provide an
--  -- uncons or a tail function.
-- @

-- $comparison
--
-- Streamly unifies non-determinism, streaming, concurrency and FRP
-- functionality that is otherwise covered by several disparate packages, and
-- it does that with a surprisingly concise API.  Here is a list of popular and
-- well-known packages in all these areas:
--
-- @
-- +-----------------+----------------+
-- | Non-determinism | <https://hackage.haskell.org/package/pipes pipes>          |
-- |                 +----------------+
-- |                 | <https://hackage.haskell.org/package/list-t list-t>         |
-- |                 +----------------+
-- |                 | <https://hackage.haskell.org/package/logict logict>         |
-- +-----------------+----------------+
-- | Streaming       | <https://hackage.haskell.org/package/vector vector>         |
-- |                 +----------------+
-- |                 | <https://hackage.haskell.org/package/streaming streaming>      |
-- |                 +----------------+
-- |                 | <https://hackage.haskell.org/package/pipes pipes>          |
-- |                 +----------------+
-- |                 | <https://hackage.haskell.org/package/conduit conduit>        |
-- +-----------------+----------------+
-- | Concurrency     | <https://hackage.haskell.org/package/async async>          |
-- |                 +----------------+
-- |                 | <https://hackage.haskell.org/package/transient transient>      |
-- +-----------------+----------------+
-- | FRP             | <https://hackage.haskell.org/package/Yampa Yampa>          |
-- |                 +----------------+
-- |                 | <https://hackage.haskell.org/package/dunai dunai>          |
-- |                 +----------------+
-- |                 | <https://hackage.haskell.org/package/reflex reflex>         |
-- +-----------------+----------------+
-- @
--
-- Streamly is a list-transformer. It provides all the functionality provided
-- by any of the list transformer and logic programming packages listed above.
-- In addition, Streamly naturally integrates the concurrency dimension to the
-- basic list transformer functionality.
--
-- When it comes to streaming, in terms of the streaming API streamly is almost
-- identical to the vector package. Streamly, vector and streaming packages all
-- represent a stream as data and are therefore similar in the fundamental
-- approach to streaming. The fundamental difference is that streamly adds
-- concurrency support and the monad instance provides concurrent looping.
-- Other streaming libraries like pipes, conduit and machines represent and
-- compose stream processors rather than the stream data and therefore fall in
-- another class of streaming libraries and have comparatively more complicated
-- types.
--
-- When it comes to concurrency, streamly can do everything that the @async@
-- package can do and more. async provides applicative concurrency whereas
-- streamly provides both applicative and monadic concurrency. The
-- 'ZipParallel' type behaves like the applicative instance of async.  In
-- comparison to transient streamly has a first class streaming interface and
-- is a monad transformer that can be used universally in any Haskell monad
-- transformer stack.  Streamly was in fact originally inspired by the
-- concurrency implementation in @transient@ though it has no resemblence with
-- that and takes a lazy pull approach versus transient's strict push approach.
--
-- The non-determinism, concurrency and streaming combination make streamly a
-- strong FRP capable library as well. FRP is fundamentally stream of events
-- that can be processed concurrently. The example in this tutorial as well as
-- the "Streamly.Examples.CirclingSquare" example from Yampa demonstrate the
-- basic FRP capability of streamly. In core concepts streamly is strikingly
-- similar to @dunai@. dunai was designed from a FRP perspective and streamly
-- was originally designed from a concurrency perspective. However, both have
-- similarity at the core.