on master help sign in
Add overall docs for unfold module

Also remove some exports which may not be required for using unfolds with streams.

commit7a1463d0a217d235fc05a1d560f1359bb5020bbd authorHarendra Kumar <harendra@composewell.com> dateThu 25 Mar 2021, 02:02 (6 years ago) parentea6f2fb
4 files changed+154−65
Changelog.md+1 src/Streamly/Data/Unfold.hs+81−53 src/Streamly/Internal/Data/Unfold.hs+47−5 src/Streamly/Internal/Data/Unfold/Type.hs+25−7
Changelog.md+1
@@ -36,6 +36,7 @@
3636* New encoding/decoding routines, `encodeUtf8'`, `encodeLatin1'`, `decodeUtf8'`,
3737 are added, these routines fail when they encounter any invalid characters.
3838* Several new functions added to `Streamly.Data.Fold`
39+* Several Unfold routines and combinators added to `Streamly.Data.Unfold`.
3940
4041### Bug Fixes
4142
src/Streamly/Data/Unfold.hs+81−53
@@ -8,57 +8,93 @@
88-- Stability : experimental
99-- Portability : GHC
1010--
11−-- 'Unfold' type represents an effectful action that generates a stream of
12−-- values from a single starting value often called a seed value. Values can be
13−-- generated and /pulled/ from the 'Unfold' one at a time. It can also be
14−-- called a producer or a source of stream. It is a data representation of the
15−-- standard 'Streamly.Prelude.unfoldr' function. An 'Unfold' can be converted
16−-- into a stream type using 'Streamly.Prelude.unfold' by supplying the seed.
17−--
18−-- = Performance Notes
19−--
20−-- 'Unfold' representation is more efficient than using streams when combining
21−-- streams. 'Unfold' type allows multiple unfold actions to be composed into a
22−-- single unfold function in an efficient manner by enabling the compiler to
23−-- perform stream fusion optimization.
24−-- @Unfold m a b@ can be considered roughly equivalent to an action @a -> t m
25−-- b@ (where @t@ is a stream type). Instead of using an 'Unfold' one could just
26−-- use a function of the shape @a -> t m b@. However, working with stream types
27−-- like t'Streamly.SerialT' does not allow the compiler to perform stream fusion
28−-- optimization when merging, appending or concatenating multiple streams.
29−-- Even though stream based combinator have excellent performance, they are
30−-- much less efficient when compared to combinators using 'Unfold'. For
31−-- example, the 'Streamly.Prelude.concatMap' combinator which uses @a -> t m b@
32−-- (where @t@ is a stream type) to generate streams is much less efficient
33−-- compared to 'Streamly.Prelude.unfoldMany'.
34−--
35−-- On the other hand, transformation operations on stream types are as
36−-- efficient as transformations on 'Unfold'.
37−--
38−-- We should note that in some cases working with stream types may be more
39−-- convenient compared to working with the 'Unfold' type. However, if extra
40−-- performance boost is important then 'Unfold' based composition should be
41−-- preferred compared to stream based composition when merging or concatenating
42−-- streams.
43−--
44−-- = Programmer Notes
45−--
46−-- > import qualified Streamly.Data.Unfold as UF
47−--
48−-- More, not yet exposed, unfold combinators can be found in
49−-- "Streamly.Internal.Data.Unfold".
50−
51−-- The stream types (e.g. t'Streamly.SerialT') can be considered as a special
52−-- case of 'Unfold' with no starting seed.
11+-- An 'Unfold' is a source or a producer of a stream of values. It takes a
12+-- seed value as an input and unfolds it into a sequence of values.
13+--
14+-- >>> import qualified Streamly.Data.Fold as Fold
15+-- >>> import qualified Streamly.Data.Unfold as Unfold
16+-- >>> import qualified Streamly.Prelude as Stream
17+--
18+-- For example, the 'fromList' Unfold generates a stream of values from a
19+-- supplied list. Unfolds can be converted to 'Streamly.Prelude.SerialT'
20+-- stream using the Stream.unfold operation.
21+--
22+-- >>> stream = Stream.unfold Unfold.fromList [1..100]
23+-- >>> Stream.sum stream
24+-- 5050
25+--
26+-- All the serial stream generation operations in "Streamly.Prelude"
27+-- can be expressed using unfolds:
28+--
29+-- > Stream.fromList = Stream.unfold Unfold.fromList [1..100]
30+--
31+-- Conceptually, an 'Unfold' is just like "Data.List.unfoldr". Let us write a
32+-- step function to unfold a list using "Data.List.unfoldr":
33+--
34+-- >>> :{
35+-- f [] = Nothing
36+-- f (x:xs) = Just (x, xs)
37+-- :}
38+--
39+-- >>> Data.List.unfoldr f [1,2,3]
40+-- [1,2,3]
41+--
42+-- Unfold.unfoldr is just the same, it uses the same step function:
43+--
44+-- >>> Stream.toList $ Stream.unfold (Unfold.unfoldr f) [1,2,3]
45+-- [1,2,3]
46+--
47+-- The input of an unfold can be transformed using 'lmap':
48+--
49+-- >>> u = Unfold.lmap (fmap (+1)) Unfold.fromList
50+-- >>> Stream.toList $ Stream.unfold u [1..5]
51+-- [2,3,4,5,6]
5352--
53+-- 'Unfold' streams can be transformed using transformation combinators. For
54+-- example, to retain only the first two elements of an unfold:
55+--
56+-- >>> u = Unfold.take 2 Unfold.fromList
57+-- >>> Stream.toList $ Stream.unfold u [1..100]
58+-- [1,2]
59+--
60+-- Multiple unfolds can be combined in several interesting ways. For example,
61+-- to generate nested looping as in imperative languages (also known as cross
62+-- product of the two streams):
63+--
64+-- >>> u1 = Unfold.lmap fst Unfold.fromList
65+-- >>> u2 = Unfold.lmap snd Unfold.fromList
66+-- >>> u = Unfold.crossWith (,) u1 u2
67+-- >>> Stream.toList $ Stream.unfold u ([1,2,3], [4,5,6])
68+-- [(1,4),(1,5),(1,6),(2,4),(2,5),(2,6),(3,4),(3,5),(3,6)]
69+--
70+-- Nested loops using unfolds provide C like performance due to complete stream
71+-- fusion.
72+--
73+-- Please see "Streamly.Internal.Data.Unfold" for additional @Pre-release@
74+-- functions.
75+--
76+-- = Unfolds vs. Streams
77+--
78+-- Unfolds' raison d'etre is their efficiency in nested stream operations due
79+-- to complete stream fusion. 'Streamly.Prelude.concatMap' or the 'Monad'
80+-- instance of streams use stream generation operations of the shape @a -> t m
81+-- b@ and then flatten the resulting stream. This implementation is more
82+-- powerful but does not allow for complete stream fusion. Unfolds provide
83+-- less powerful but more efficient 'Streamly.Prelude.unfoldMany', 'many' and
84+-- 'crossWith' operations as an alternative to a subset of use cases of
85+-- 'concatMap' and 'Applicative' stream operations.
86+--
87+-- "Streamly.Prelude" exports polymorphic stream generation operations that
88+-- provide the same functionality as unfolds in this module. Since unfolds can
89+-- be easily converted to streams, several modules in streamly provide only
90+-- unfolds for serial stream generation. We cannot use unfolds exclusively for
91+-- stream generation as they do not support concurrency.
92+
5493module Streamly.Data.Unfold
5594 (
5695 -- * Unfold Type
5796 Unfold
5897
59− -- * Folding
60− , fold
61−
6298 -- * Unfolds
6399 -- One to one correspondence with
64100 -- "Streamly.Internal.Data.Stream.IsStream.Generate"
@@ -106,14 +142,6 @@ module Streamly.Data.Unfold
106142
107143 -- ** Nesting
108144 , many
109−
110− -- ** Resource Management
111− , finally
112− , bracket
113−
114− -- ** Exceptions
115− , onException
116− , handle
117145 )
118146where
119147
src/Streamly/Internal/Data/Unfold.hs+47−5
@@ -8,6 +8,11 @@
88-- Stability : experimental
99-- Portability : GHC
1010--
11+-- To run the examples in this module:
12+--
13+-- >>> import qualified Streamly.Data.Fold as Fold
14+-- >>> import qualified Streamly.Internal.Data.Unfold as Unfold
15+--
1116-- = Unfolds and Streams
1217--
1318-- An 'Unfold' type is the same as the direct style 'Stream' type except that
@@ -72,6 +77,33 @@
7277-- representation. We can use that for composable folds in StreamK
7378-- representation.
7479--
80+
81+-- = Performance Notes
82+--
83+-- 'Unfold' representation is more efficient than using streams when combining
84+-- streams. 'Unfold' type allows multiple unfold actions to be composed into a
85+-- single unfold function in an efficient manner by enabling the compiler to
86+-- perform stream fusion optimization.
87+-- @Unfold m a b@ can be considered roughly equivalent to an action @a -> t m
88+-- b@ (where @t@ is a stream type). Instead of using an 'Unfold' one could just
89+-- use a function of the shape @a -> t m b@. However, working with stream types
90+-- like t'Streamly.SerialT' does not allow the compiler to perform stream fusion
91+-- optimization when merging, appending or concatenating multiple streams.
92+-- Even though stream based combinator have excellent performance, they are
93+-- much less efficient when compared to combinators using 'Unfold'. For
94+-- example, the 'Streamly.Prelude.concatMap' combinator which uses @a -> t m b@
95+-- (where @t@ is a stream type) to generate streams is much less efficient
96+-- compared to 'Streamly.Prelude.unfoldMany'.
97+--
98+-- On the other hand, transformation operations on stream types are as
99+-- efficient as transformations on 'Unfold'.
100+--
101+-- We should note that in some cases working with stream types may be more
102+-- convenient compared to working with the 'Unfold' type. However, if extra
103+-- performance boost is important then 'Unfold' based composition should be
104+-- preferred compared to stream based composition when merging or concatenating
105+-- streams.
106+
75107module Streamly.Internal.Data.Unfold
76108 (
77109 -- * Unfold Type
@@ -177,17 +209,19 @@ module Streamly.Internal.Data.Unfold
177209 , concatMapM
178210 , bind
179211
180− -- ** Exceptions
212+ -- ** Resource Management
181213 , gbracket_
182214 , gbracket
183215 , before
184− , after_
185216 , after
186− , onException
187− , finally_
217+ , after_
188218 , finally
189− , bracket_
219+ , finally_
190220 , bracket
221+ , bracket_
222+
223+ -- ** Exceptions
224+ , onException
191225 , handle
192226 )
193227where
@@ -326,6 +360,9 @@ swap = lmap Tuple.swap
326360-- @Fold m b c@, returns a monadic action @a -> m c@ representing the
327361-- application of the fold on the unfolded stream.
328362--
363+-- >>> Unfold.fold Fold.sum Unfold.fromList [1..100]
364+-- 5050
365+--
329366-- /Pre-release/
330367--
331368{-# INLINE_NORMAL fold #-}
@@ -544,6 +581,11 @@ fromIndicesM gen = Unfold step return
544581-- Filtering
545582-------------------------------------------------------------------------------
546583
584+-- |
585+-- >>> u = Unfold.take 2 Unfold.fromList
586+-- >>> Unfold.fold Fold.toList u [1..100]
587+-- [1,2]
588+--
547589{-# INLINE_NORMAL take #-}
548590take :: Monad m => Int -> Unfold m a b -> Unfold m a b
549591take n (Unfold step1 inject1) = Unfold step inject
src/Streamly/Internal/Data/Unfold/Type.hs+25−7
@@ -5,6 +5,12 @@
55-- Maintainer : streamly@composewell.com
66-- Stability : experimental
77-- Portability : GHC
8+--
9+-- To run the examples in this module:
10+--
11+-- >>> import qualified Streamly.Prelude as Stream
12+-- >>> import qualified Streamly.Data.Fold as Fold
13+-- >>> import qualified Streamly.Internal.Data.Unfold as Unfold
814
915module Streamly.Internal.Data.Unfold.Type
1016 ( Unfold (..)
@@ -57,12 +63,6 @@ import Streamly.Internal.Data.Stream.StreamD.Step (Step(..))
5763
5864import Prelude hiding (const, map, concatMap, zipWith)
5965
60−-- $setup
61−-- >>> :m
62−-- >>> import qualified Streamly.Prelude as Stream
63−-- >>> import qualified Streamly.Data.Fold as Fold
64−-- >>> import qualified Streamly.Data.Unfold as Unfold
65−
6666------------------------------------------------------------------------------
6767-- Monadic Unfolds
6868------------------------------------------------------------------------------
@@ -121,6 +121,14 @@ unfoldrM next = Unfold step pure
121121
122122-- | Like 'unfoldrM' but uses a pure step function.
123123--
124+-- >>> :{
125+-- f [] = Nothing
126+-- f (x:xs) = Just (x, xs)
127+-- :}
128+--
129+-- >>> Unfold.fold Fold.toList (Unfold.unfoldr f) [1,2,3]
130+-- [1,2,3]
131+--
124132-- /Pre-release/
125133--
126134{-# INLINE unfoldr #-}
@@ -133,6 +141,10 @@ unfoldr step = unfoldrM (pure . step)
133141
134142-- | Map a function on the input argument of the 'Unfold'.
135143--
144+-- >>> u = Unfold.lmap (fmap (+1)) Unfold.fromList
145+-- >>> Unfold.fold Fold.toList u [1..5]
146+-- [2,3,4,5,6]
147+--
136148-- @
137149-- lmap f = Unfold.many (Unfold.function f)
138150-- @
@@ -246,6 +258,12 @@ crossWithM f (Unfold step1 inject1) (Unfold step2 inject2) = Unfold step inject
246258--
247259-- > crossWith f = crossWithM (\b c -> return $ f b c)
248260--
261+-- >>> u1 = Unfold.lmap fst Unfold.fromList
262+-- >>> u2 = Unfold.lmap snd Unfold.fromList
263+-- >>> u = Unfold.crossWith (,) u1 u2
264+-- >>> Unfold.fold Fold.toList u ([1,2,3], [4,5,6])
265+-- [(1,4),(1,5),(1,6),(2,4),(2,5),(2,6),(3,4),(3,5),(3,6)]
266+--
249267-- /Pre-release/
250268{-# INLINE crossWith #-}
251269crossWith :: Monad m =>
@@ -507,7 +525,7 @@ zipWithM f (Unfold step1 inject1) (Unfold step2 inject2) = Unfold step inject
507525-- >>> square = fmap (\x -> x * x) Unfold.fromList
508526-- >>> cube = fmap (\x -> x * x * x) Unfold.fromList
509527-- >>> u = Unfold.zipWith (,) square cube
510−-- >>> Unfold.fold u Fold.toList [1..5]
528+-- >>> Unfold.fold Fold.toList u [1..5]
511529-- [(1,1),(4,8),(9,27),(16,64),(25,125)]
512530--
513531-- > zipWith f = zipWithM (\a b -> return $ f a b)