@@ -1,3 +1,4 @@
1+{-# LANGUAGE CPP #-}
12-- |
23-- Module : Streamly.Internal.Data.Unfold.Type
34-- Copyright : (c) 2019 Composewell Technologies
@@ -6,14 +7,49 @@
67-- Stability : experimental
78-- Portability : GHC
89--
9−-- To run the examples in this module:
10−--
11−-- >>> import qualified Streamly.Data.Stream as Stream
12−-- >>> import qualified Streamly.Data.Fold as Fold
13−-- >>> import qualified Streamly.Internal.Data.Unfold as Unfold
10+-- An unfold is akin to a reader. It is the streaming equivalent of a reader.
11+-- The argument @a@ is the environment of the reader. That's the reason the
12+-- default unfolds in various modules are named "reader".
13+
14+-- = Performance Notes
15+--
16+-- 'Unfold' representation is more efficient than using streams when combining
17+-- streams. 'Unfold' type allows multiple unfold actions to be composed into a
18+-- single unfold function in an efficient manner by enabling the compiler to
19+-- perform stream fusion optimization.
20+-- @Unfold m a b@ can be considered roughly equivalent to an action @a -> t m
21+-- b@ (where @t@ is a stream type). Instead of using an 'Unfold' one could just
22+-- use a function of the shape @a -> t m b@. However, working with stream types
23+-- like t'Streamly.SerialT' does not allow the compiler to perform stream fusion
24+-- optimization when merging, appending or concatenating multiple streams.
25+-- Even though stream based combinator have excellent performance, they are
26+-- much less efficient when compared to combinators using 'Unfold'. For
27+-- example, the 'Streamly.Data.Stream.concatMap' combinator which uses @a -> t m b@
28+-- (where @t@ is a stream type) to generate streams is much less efficient
29+-- compared to 'Streamly.Data.Stream.unfoldMany'.
30+--
31+-- On the other hand, transformation operations on stream types are as
32+-- efficient as transformations on 'Unfold'.
33+--
34+-- We should note that in some cases working with stream types may be more
35+-- convenient compared to working with the 'Unfold' type. However, if extra
36+-- performance boost is important then 'Unfold' based composition should be
37+-- preferred compared to stream based composition when merging or concatenating
38+-- streams.
1439
1540module Streamly.Internal.Data.Unfold.Type
16− ( Unfold (..)
41+ (
42+ -- * Setup
43+ -- | To execute the code examples provided in this module in ghci, please
44+ -- run the following commands first.
45+ --
46+ -- $setup
47+
48+ -- * General Notes
49+ -- $notes
50+
51+ -- * Type
52+ Unfold (..)
1753
1854 -- * Basic Constructors
1955 , mkUnfoldM
@@ -84,22 +120,96 @@ import Streamly.Internal.Data.Stream.StreamD.Step (Step(..))
84120
85121import Prelude hiding (map, mapM, concatMap, zipWith, takeWhile)
86122
87−-- $setup
88−-- >>> import qualified Streamly.Data.Stream as Stream
89−-- >>> import qualified Streamly.Data.Fold as Fold
90−-- >>> import qualified Streamly.Internal.Data.Unfold as Unfold
123+#include "DocTestDataUnfold.hs"
124+
125+-- $notes
126+--
127+-- What makes streams less efficient is also what makes them more convenient to
128+-- use and powerful. The stream data type (Stream m a) bundles the state along
129+-- with the stream generator function making it opaque, whereas an unfold
130+-- exposes the state (Unfold m s a) to the user. This allows the Unfold to be
131+-- unfolded (inlined) inside a nested loop without having to bundle the state
132+-- and the generator together, the stream state can be saved and passed
133+-- independent of the generator function. On the other hand in a stream type we
134+-- have to bundle the stream state and the generator function together to save
135+-- the stream. This makes it inefficient because it requires boxing and
136+-- constructor allocation. However, this makes streams more convenient as we do
137+-- not need to pass around the state/seed separately.
138+--
139+-- Unfold Type:
140+--
141+-- The order of arguments allows 'Category' and 'Arrow' instances but precludes
142+-- contravariant and contra-applicative.
143+--
144+-- = Unfolds and Streams
145+--
146+-- An 'Unfold' type is the same as the direct style 'Stream' type except that
147+-- it uses an inject function to determine the initial state of the stream
148+-- based on an input. A stream is a special case of Unfold when the static
149+-- input is unit or Void.
150+--
151+-- This allows an important optimization to occur in several cases, making the
152+-- 'Unfold' a more efficient abstraction. Consider the 'concatMap' and
153+-- 'unfoldMany' operations, the latter is more efficient. 'concatMap'
154+-- generates a new stream object from each element in the stream by applying
155+-- the supplied function to the element, the stream object includes the "step"
156+-- function as well as the initial "state" of the stream. Since the stream is
157+-- generated dynamically the compiler does not know the step function or the
158+-- state type statically at compile time, therefore, it cannot inline it. On
159+-- the other hand in case of 'unfoldMany' the compiler has visibility into
160+-- the unfold's state generation function, therefore, the compiler knows all
161+-- the types statically and it can inline the inject as well as the step
162+-- functions, generating efficient code. Essentially, the stream is not opaque
163+-- to the consumer in case of unfolds, the consumer knows how to generate the
164+-- stream from a seed using a known "inject" and "step" functions.
165+--
166+-- A Stream is like a data object whereas unfold is like a function. Being
167+-- function like, an Unfold is an instance of 'Category' and 'Arrow' type
168+-- classes.
169+--
170+-- = Unfolds and Folds
171+--
172+-- Streams forcing a closed control flow loop can be categorized under
173+-- two types, unfolds and folds, both of these are duals of each other.
174+--
175+-- Unfold streams are really generators of a sequence of elements, we can also
176+-- call them pull style streams. These are lazy producers of streams. On each
177+-- evaluation the producer generates the next element. A consumer can
178+-- therefore pull elements from the stream whenever it wants to. A stream
179+-- consumer can multiplex pull streams by pulling elements from the chosen
180+-- streams, therefore, pull streams allow merging or multiplexing. On the
181+-- other hand, with this representation we cannot split or demultiplex a
182+-- stream. So really these are stream sources that can be generated from a
183+-- seed and can be merged or zipped into a single stream.
184+--
185+-- The dual of Unfolds are Folds. Folds can also be called as push style
186+-- streams or reducers. These are strict consumers of streams. We keep pushing
187+-- elements to a fold and we can extract the result at any point. A driver can
188+-- choose which fold to push to and can also push the same element to multiple
189+-- folds. Therefore, folds allow splitting or demultiplexing a stream. On the
190+-- other hand, we cannot merge streams using this representation. So really
191+-- these are stream consumers that reduce the stream to a single value, these
192+-- consumers can be composed such that a stream can be split over multiple
193+-- consumers.
194+--
195+-- Performance:
196+--
197+-- Composing a tree or graph of computations with unfolds can be much more
198+-- efficient compared to composing with the Monad instance. The reason is that
199+-- unfolds allow the compiler to statically know the state and optimize it
200+-- using stream fusion whereas it is not possible with the monad bind because
201+-- the state is determined dynamically.
202+--
203+-- Reader:
204+--
205+-- An unfold acts as a reader (see 'Reader' monad). The input to an unfold acts
206+-- as the read-only environment. The environment can be extracted using the
207+-- 'identity' unfold (equivalent to 'ask') and transformed using 'lmap'.
91208
92209------------------------------------------------------------------------------
93210-- Monadic Unfolds
94211------------------------------------------------------------------------------
95212
96−-- The order of arguments allows 'Category' and 'Arrow' instances but precludes
97−-- contravariant and contra-applicative.
98−--
99−-- An unfold is akin to a reader. It is the streaming equivalent of a reader.
100−-- The argument @a@ is the environment of the reader. That's the reason the
101−-- default unfolds in various modules are called "read".
102−--
103213-- | An @Unfold m a b@ is a generator of a stream of values of type @b@ from a
104214-- seed of type 'a' in 'Monad' @m@.
105215--
@@ -130,7 +240,7 @@ mkUnfoldrM step = Unfold step pure
130240-- The type 'Step' is isomorphic to 'Maybe'. Ideally unfoldrM should be the
131241-- same as mkUnfoldrM, this is for compatibility with traditional Maybe based
132242-- unfold step functions.
133−--
243+
134244-- | Build a stream by unfolding a /monadic/ step function starting from a seed.
135245-- The step function returns the next element in the stream and the next seed
136246-- value. When it is done it returns 'Nothing' and the stream ends.
@@ -369,6 +479,7 @@ instance Functor m => Functor (Unfold m a) where
369479------------------------------------------------------------------------------
370480
371481-- XXX Shouldn't this be Unfold m (m a) a ?
482+
372483-- | The unfold discards its input and generates a function stream using the
373484-- supplied monadic action.
374485--
@@ -386,7 +497,7 @@ fromEffect m = Unfold step inject
386497
387498-- XXX Shouldn't this be Unfold m a a ? Which is identity. Should this function
388499-- even exist for Unfolds. Should we have applicative/Monad for unfolds?
389−--
500+
390501-- | Discards the unfold input and always returns the argument of 'fromPure'.
391502--
392503-- > fromPure = fromEffect . pure