@@ -13,83 +13,146 @@
1313--
1414-- >>> import qualified Streamly.Data.Fold as Fold
1515-- >>> import qualified Streamly.Data.Unfold as Unfold
16−-- >>> import qualified Streamly.Prelude as Stream
16+-- >>> import qualified Streamly.Internal.Data.Stream as Stream
1717--
1818-- For example, the 'fromList' Unfold generates a stream of values from a
19−-- supplied list. Unfolds can be converted to 'Streamly.Prelude.SerialT'
20−-- stream using the Stream.unfold operation.
19+-- supplied list. Unfolds can be converted to 'Streamly.Internal.Data.Stream'
20+-- using the 'Stream.unfold' operation.
2121--
2222-- >>> stream = Stream.unfold Unfold.fromList [1..100]
23−-- >>> Stream.sum stream
23+-- >>> Stream.fold Fold.sum stream
2424-- 5050
2525--
26−-- All the serial stream generation operations in "Streamly.Prelude"
27−-- can be expressed using unfolds:
28−--
29−-- > Stream.fromList = Stream.unfold Unfold.fromList [1..100]
30−--
31−-- Conceptually, an 'Unfold' is just like "Data.List.unfoldr". Let us write a
32−-- step function to unfold a list using "Data.List.unfoldr":
33−--
34−-- >>> :{
35−-- f [] = Nothing
36−-- f (x:xs) = Just (x, xs)
37−-- :}
38−--
39−-- >>> Data.List.unfoldr f [1,2,3]
40−-- [1,2,3]
41−--
42−-- Unfold.unfoldr is just the same, it uses the same step function:
43−--
44−-- >>> Stream.toList $ Stream.unfold (Unfold.unfoldr f) [1,2,3]
45−-- [1,2,3]
46−--
47−-- The input of an unfold can be transformed using 'lmap':
26+-- The input seed of an unfold can be transformed using 'lmap':
4827--
4928-- >>> u = Unfold.lmap (fmap (+1)) Unfold.fromList
50−-- >>> Stream.toList $ Stream.unfold u [1..5]
29+-- >>> Stream.fold Fold.toList $ Stream.unfold u [1..5]
5130-- [2,3,4,5,6]
5231--
53−-- 'Unfold' streams can be transformed using transformation combinators. For
54−-- example, to retain only the first two elements of an unfold:
32+-- Output stream of an 'Unfold' can be transformed using transformation
33+-- combinators. For example, to retain only the first two elements of an
34+-- unfold:
5535--
5636-- >>> u = Unfold.take 2 Unfold.fromList
57−-- >>> Stream.toList $ Stream.unfold u [1..100]
37+-- >>> Stream.fold Fold.toList $ Stream.unfold u [1..100]
5838-- [1,2]
5939--
60−-- Multiple unfolds can be combined in several interesting ways. For example,
61−-- to generate nested looping as in imperative languages (also known as cross
62−-- product of the two streams):
40+-- Unfolds can be nested efficiently. For example, to implement nested looping:
6341--
6442-- >>> u1 = Unfold.lmap fst Unfold.fromList
6543-- >>> u2 = Unfold.lmap snd Unfold.fromList
6644-- >>> u = Unfold.crossWith (,) u1 u2
67−-- >>> Stream.toList $ Stream.unfold u ([1,2,3], [4,5,6])
45+-- >>> Stream.fold Fold.toList $ Stream.unfold u ([1,2,3], [4,5,6])
6846-- [(1,4),(1,5),(1,6),(2,4),(2,5),(2,6),(3,4),(3,5),(3,6)]
6947--
70−-- Nested loops using unfolds provide C like performance due to complete stream
71−-- fusion.
48+-- 'Unfold' @u1@ generates a stream from the first list in the input tuple,
49+-- @u2@ generates another stream from the second list. The combines 'Unfold'
50+-- @u@ nests the two streams i.e. for each element in first stream, for each
51+-- element in second stream apply the supplied function (i.e. @(,)@) to the
52+-- pair of elements.
53+--
54+-- This is the equivalent of the nested looping construct from imperative
55+-- languages, also known as the cross product of two streams in functional
56+-- parlance.
7257--
7358-- Please see "Streamly.Internal.Data.Unfold" for additional @Pre-release@
7459-- functions.
7560--
61+-- = Creating New Unfolds
62+--
63+-- There are many commonly used unfolds provided in this module. However, you
64+-- can always create your own as well. An 'Unfold' is just a data
65+-- representation of a stream generator function. It consists of an @inject@
66+-- function which covnerts the supplied seed into an internal state of the
67+-- unfold, and a @step@ function which takes the state and generates the next
68+-- output in the stream. For those familiar with the list "Data.List.unfoldr"
69+-- function, this is a data representation of the same.
70+--
71+-- Smart constructor functions are provided in this module for constructing new
72+-- 'Unfolds'. For example, you can use the 'Unfold.unfoldr' constructor to
73+-- create an 'Unfold' from a pure step function, unfoldr uses 'id' as the
74+-- @inject@ function.
75+--
76+-- Let's define a simple pure step function:
77+--
78+-- >>> :{
79+-- f [] = Nothing
80+-- f (x:xs) = Just (x, xs)
81+-- :}
82+--
83+-- Create an 'Unfold' from the step function:
84+--
85+-- >>> u = Unfold.unfoldr f
86+--
87+-- Run the 'Unfold':
88+--
89+-- >>> Stream.fold Fold.toList $ Stream.unfold u [1,2,3]
90+-- [1,2,3]
91+--
92+-- The 'Unfold.unfoldr' smart constructor is essentially the same as the list
93+-- "Data.List.unfoldr" function. We can use the same step function in both::
94+--
95+-- >>> Data.List.unfoldr f [1,2,3]
96+-- [1,2,3]
97+--
7698-- = Unfolds vs. Streams
7799--
78−-- Unfolds' raison d'etre is their efficiency in nested stream operations due
79−-- to complete stream fusion. 'Streamly.Prelude.concatMap' or the 'Monad'
80−-- instance of streams use stream generation operations of the shape @a -> t m
81−-- b@ and then flatten the resulting stream. This implementation is more
82−-- powerful but does not allow for complete stream fusion. Unfolds provide
83−-- less powerful but more efficient 'Streamly.Prelude.unfoldMany', 'many' and
84−-- 'crossWith' operations as an alternative to a subset of use cases of
85−-- 'concatMap' and 'Applicative' stream operations.
86−--
87−-- "Streamly.Prelude" exports polymorphic stream generation operations that
88−-- provide the same functionality as unfolds in this module. Since unfolds can
89−-- be easily converted to streams, several modules in streamly provide only
90−-- unfolds for serial stream generation. We cannot use unfolds exclusively for
91−-- stream generation as they do not support concurrency.
92−
100+-- The 'Unfold' abstraction for representing streams was introduced in Streamly
101+-- to provide C like performance for nested looping of streams. 'Unfold' and
102+-- 'Stream' abstractions are similar with the following differences:
103+--
104+-- * 'Stream' is less efficient than 'Unfold' for nesting.
105+-- * 'Stream' is more powerful than 'Unfold'.
106+-- * 'Stream' API is more convenient for programming
107+--
108+-- Unfolds can be easily converted to streams using 'Stream.unfold', however,
109+-- vice versa is not possible. To provide a familiar analogy, 'Unfold' is to
110+-- 'Stream' as 'Applicative' is to 'Monad'.
111+--
112+-- To demonstrate the efficiency of unfolds, the nested loop example in the
113+-- previous section can be implemented with concatMap or Monad instance of
114+-- streams as follows:
115+--
116+-- @
117+-- do
118+-- x <- Stream.unfold Unfold.fromList [1,2,3]
119+-- y <- Stream.unfold Unfold.fromList [4,5,6]
120+-- return (x, y)
121+-- @
122+--
123+-- As you can see, this is more convenient to write than using the 'crossWith'
124+-- unfold combinator. However, this turns out to be many times slower than the
125+-- unfold implementation. The Unfold version is equivalent in performance to
126+-- the C implementation of the same nested loop. Similarly, unfolds can be
127+-- nested with streams using the 'unfoldMany' combinator which is a much more
128+-- efficient alternative to the 'concatMap' operation.
129+--
130+-- Streams use a hybrid implementation approach using direct style as well as
131+-- CPS. Unfolds do not use CPS, therefore, lack the power that is afforded to
132+-- streams by CPS. The CPS implementation allows infinitely scalable @cons@ and
133+-- @append@ operations in streams. It is also used to implement concurrency in
134+-- streams.
135+--
136+-- To summarize, unfolds are a high performance solution to the nesting
137+-- problem. Since streams provide a more palatable API for programming, work
138+-- with streams unless you need unfolds for better performance in nesting
139+-- situations. There is little difference in the way in which unfolds and
140+-- streams are written, it is easy to adapt a stream to an unfold. If you are
141+-- writing an unfold you can convert it to stream for free using
142+-- 'Stream.unfold'.
143+
144+-- What makes streams less efficient is also what makes them more conveient to
145+-- use and powerful. The stream data type (Stream m a) bundles the state along
146+-- with the stream generator function making it opaque, whereas an unfold
147+-- exposes the state (Unfold m s a) to the user. This allows the Unfold to be
148+-- unfolded (inlined) inside a nested loop without having to bundle the state
149+-- and the generator together, the stream state can be saved and passed
150+-- independent of the generator function. On the other hand in a stream type we
151+-- have to bundle the stream state and the generator function together to save
152+-- the stream. This makes it inefficient because it requires boxing and
153+-- constructor allocation. However, this makes streams more convenient as we do
154+-- not need to pass around the state/seed separately.
155+--
93156module Streamly.Data.Unfold
94157 (
95158 -- * Unfold Type