@@ -1,3 +1,4 @@
2-- Module : Streamly.Internal.Data.Unfold.Type
3-- Module : Streamly.Internal.Data.Unfold.Type
3-- Copyright : (c) 2019 Composewell Technologies
4-- Copyright : (c) 2019 Composewell Technologies
@@ -6,14 +7,49 @@
6-- Stability : experimental
7-- Stability : experimental
7-- Portability : GHC
8-- Portability : GHC
9−-- To run the examples in this module:
10+-- An unfold is akin to a reader. It is the streaming equivalent of a reader.
10−--
11+-- The argument @a@ is the environment of the reader. That's the reason the
11−-- >>> import qualified Streamly.Data.Stream as Stream
12+-- default unfolds in various modules are named "reader".
12−-- >>> import qualified Streamly.Data.Fold as Fold
13+
13−-- >>> import qualified Streamly.Internal.Data.Unfold as Unfold
14+-- = Performance Notes
16+-- 'Unfold' representation is more efficient than using streams when combining
17+-- streams. 'Unfold' type allows multiple unfold actions to be composed into a
18+-- single unfold function in an efficient manner by enabling the compiler to
19+-- perform stream fusion optimization.
20+-- @Unfold m a b@ can be considered roughly equivalent to an action @a -> t m
21+-- b@ (where @t@ is a stream type). Instead of using an 'Unfold' one could just
22+-- use a function of the shape @a -> t m b@. However, working with stream types
23+-- like t'Streamly.SerialT' does not allow the compiler to perform stream fusion
24+-- optimization when merging, appending or concatenating multiple streams.
25+-- Even though stream based combinator have excellent performance, they are
26+-- much less efficient when compared to combinators using 'Unfold'. For
27+-- example, the 'Streamly.Data.Stream.concatMap' combinator which uses @a -> t m b@
28+-- (where @t@ is a stream type) to generate streams is much less efficient
29+-- compared to 'Streamly.Data.Stream.unfoldMany'.
31+-- On the other hand, transformation operations on stream types are as
32+-- efficient as transformations on 'Unfold'.
34+-- We should note that in some cases working with stream types may be more
35+-- convenient compared to working with the 'Unfold' type. However, if extra
36+-- performance boost is important then 'Unfold' based composition should be
37+-- preferred compared to stream based composition when merging or concatenating
15module Streamly.Internal.Data.Unfold.Type
40module Streamly.Internal.Data.Unfold.Type
43+ -- | To execute the code examples provided in this module in ghci, please
44+ -- run the following commands first.
16− ( Unfold (..)
52+ Unfold (..)
18 -- * Basic Constructors
54 -- * Basic Constructors
19 , mkUnfoldM
55 , mkUnfoldM
@@ -84,22 +120,96 @@ import Streamly.Internal.Data.Stream.StreamD.Step (Step(..))
85import Prelude hiding (map, mapM, concatMap, zipWith, takeWhile)
121import Prelude hiding (map, mapM, concatMap, zipWith, takeWhile)
87−-- $setup
123+#include "DocTestDataUnfold.hs"
88−-- >>> import qualified Streamly.Data.Stream as Stream
124+
89−-- >>> import qualified Streamly.Data.Fold as Fold
125+-- $notes
90−-- >>> import qualified Streamly.Internal.Data.Unfold as Unfold
126+--
127+-- What makes streams less efficient is also what makes them more convenient to
128+-- use and powerful. The stream data type (Stream m a) bundles the state along
129+-- with the stream generator function making it opaque, whereas an unfold
130+-- exposes the state (Unfold m s a) to the user. This allows the Unfold to be
131+-- unfolded (inlined) inside a nested loop without having to bundle the state
132+-- and the generator together, the stream state can be saved and passed
133+-- independent of the generator function. On the other hand in a stream type we
134+-- have to bundle the stream state and the generator function together to save
135+-- the stream. This makes it inefficient because it requires boxing and
136+-- constructor allocation. However, this makes streams more convenient as we do
137+-- not need to pass around the state/seed separately.
141+-- The order of arguments allows 'Category' and 'Arrow' instances but precludes
142+-- contravariant and contra-applicative.
144+-- = Unfolds and Streams
146+-- An 'Unfold' type is the same as the direct style 'Stream' type except that
147+-- it uses an inject function to determine the initial state of the stream
148+-- based on an input. A stream is a special case of Unfold when the static
149+-- input is unit or Void.
151+-- This allows an important optimization to occur in several cases, making the
152+-- 'Unfold' a more efficient abstraction. Consider the 'concatMap' and
153+-- 'unfoldMany' operations, the latter is more efficient. 'concatMap'
154+-- generates a new stream object from each element in the stream by applying
155+-- the supplied function to the element, the stream object includes the "step"
156+-- function as well as the initial "state" of the stream. Since the stream is
157+-- generated dynamically the compiler does not know the step function or the
158+-- state type statically at compile time, therefore, it cannot inline it. On
159+-- the other hand in case of 'unfoldMany' the compiler has visibility into
160+-- the unfold's state generation function, therefore, the compiler knows all
161+-- the types statically and it can inline the inject as well as the step
162+-- functions, generating efficient code. Essentially, the stream is not opaque
163+-- to the consumer in case of unfolds, the consumer knows how to generate the
164+-- stream from a seed using a known "inject" and "step" functions.
166+-- A Stream is like a data object whereas unfold is like a function. Being
167+-- function like, an Unfold is an instance of 'Category' and 'Arrow' type
170+-- = Unfolds and Folds
172+-- Streams forcing a closed control flow loop can be categorized under
173+-- two types, unfolds and folds, both of these are duals of each other.
175+-- Unfold streams are really generators of a sequence of elements, we can also
176+-- call them pull style streams. These are lazy producers of streams. On each
177+-- evaluation the producer generates the next element. A consumer can
178+-- therefore pull elements from the stream whenever it wants to. A stream
179+-- consumer can multiplex pull streams by pulling elements from the chosen
180+-- streams, therefore, pull streams allow merging or multiplexing. On the
181+-- other hand, with this representation we cannot split or demultiplex a
182+-- stream. So really these are stream sources that can be generated from a
183+-- seed and can be merged or zipped into a single stream.
185+-- The dual of Unfolds are Folds. Folds can also be called as push style
186+-- streams or reducers. These are strict consumers of streams. We keep pushing
187+-- elements to a fold and we can extract the result at any point. A driver can
188+-- choose which fold to push to and can also push the same element to multiple
189+-- folds. Therefore, folds allow splitting or demultiplexing a stream. On the
190+-- other hand, we cannot merge streams using this representation. So really
191+-- these are stream consumers that reduce the stream to a single value, these
192+-- consumers can be composed such that a stream can be split over multiple
197+-- Composing a tree or graph of computations with unfolds can be much more
198+-- efficient compared to composing with the Monad instance. The reason is that
199+-- unfolds allow the compiler to statically know the state and optimize it
200+-- using stream fusion whereas it is not possible with the monad bind because
201+-- the state is determined dynamically.
205+-- An unfold acts as a reader (see 'Reader' monad). The input to an unfold acts
206+-- as the read-only environment. The environment can be extracted using the
207+-- 'identity' unfold (equivalent to 'ask') and transformed using 'lmap'.
92------------------------------------------------------------------------------
209------------------------------------------------------------------------------
93-- Monadic Unfolds
210-- Monadic Unfolds
94------------------------------------------------------------------------------
211------------------------------------------------------------------------------
96−-- The order of arguments allows 'Category' and 'Arrow' instances but precludes
97−-- contravariant and contra-applicative.
99−-- An unfold is akin to a reader. It is the streaming equivalent of a reader.
100−-- The argument @a@ is the environment of the reader. That's the reason the
101−-- default unfolds in various modules are called "read".
103-- | An @Unfold m a b@ is a generator of a stream of values of type @b@ from a
213-- | An @Unfold m a b@ is a generator of a stream of values of type @b@ from a
104-- seed of type 'a' in 'Monad' @m@.
214-- seed of type 'a' in 'Monad' @m@.
@@ -130,7 +240,7 @@ mkUnfoldrM step = Unfold step pure
130-- The type 'Step' is isomorphic to 'Maybe'. Ideally unfoldrM should be the
240-- The type 'Step' is isomorphic to 'Maybe'. Ideally unfoldrM should be the
131-- same as mkUnfoldrM, this is for compatibility with traditional Maybe based
241-- same as mkUnfoldrM, this is for compatibility with traditional Maybe based
132-- unfold step functions.
242-- unfold step functions.
134-- | Build a stream by unfolding a /monadic/ step function starting from a seed.
244-- | Build a stream by unfolding a /monadic/ step function starting from a seed.
135-- The step function returns the next element in the stream and the next seed
245-- The step function returns the next element in the stream and the next seed
136-- value. When it is done it returns 'Nothing' and the stream ends.
246-- value. When it is done it returns 'Nothing' and the stream ends.
@@ -369,6 +479,7 @@ instance Functor m => Functor (Unfold m a) where
369------------------------------------------------------------------------------
479------------------------------------------------------------------------------
371-- XXX Shouldn't this be Unfold m (m a) a ?
481-- XXX Shouldn't this be Unfold m (m a) a ?
372-- | The unfold discards its input and generates a function stream using the
483-- | The unfold discards its input and generates a function stream using the
373-- supplied monadic action.
484-- supplied monadic action.
@@ -386,7 +497,7 @@ fromEffect m = Unfold step inject
387-- XXX Shouldn't this be Unfold m a a ? Which is identity. Should this function
498-- XXX Shouldn't this be Unfold m a a ? Which is identity. Should this function
388-- even exist for Unfolds. Should we have applicative/Monad for unfolds?
499-- even exist for Unfolds. Should we have applicative/Monad for unfolds?
390-- | Discards the unfold input and always returns the argument of 'fromPure'.
501-- | Discards the unfold input and always returns the argument of 'fromPure'.
392-- > fromPure = fromEffect . pure
503-- > fromPure = fromEffect . pure