@@ -13,83 +13,146 @@
14-- >>> import qualified Streamly.Data.Fold as Fold
14-- >>> import qualified Streamly.Data.Fold as Fold
15-- >>> import qualified Streamly.Data.Unfold as Unfold
15-- >>> import qualified Streamly.Data.Unfold as Unfold
16−-- >>> import qualified Streamly.Prelude as Stream
16+-- >>> import qualified Streamly.Internal.Data.Stream as Stream
18-- For example, the 'fromList' Unfold generates a stream of values from a
18-- For example, the 'fromList' Unfold generates a stream of values from a
19−-- supplied list. Unfolds can be converted to 'Streamly.Prelude.SerialT'
19+-- supplied list. Unfolds can be converted to 'Streamly.Internal.Data.Stream'
20−-- stream using the Stream.unfold operation.
20+-- using the 'Stream.unfold' operation.
22-- >>> stream = Stream.unfold Unfold.fromList [1..100]
22-- >>> stream = Stream.unfold Unfold.fromList [1..100]
23−-- >>> Stream.sum stream
23+-- >>> Stream.fold Fold.sum stream
26−-- All the serial stream generation operations in "Streamly.Prelude"
27−-- can be expressed using unfolds:
29−-- > Stream.fromList = Stream.unfold Unfold.fromList [1..100]
31−-- Conceptually, an 'Unfold' is just like "Data.List.unfoldr". Let us write a
32−-- step function to unfold a list using "Data.List.unfoldr":
36−-- f (x:xs) = Just (x, xs)
39−-- >>> Data.List.unfoldr f [1,2,3]
42−-- Unfold.unfoldr is just the same, it uses the same step function:
44−-- >>> Stream.toList $ Stream.unfold (Unfold.unfoldr f) [1,2,3]
47−-- The input of an unfold can be transformed using 'lmap':
26+-- The input seed of an unfold can be transformed using 'lmap':
49-- >>> u = Unfold.lmap (fmap (+1)) Unfold.fromList
28-- >>> u = Unfold.lmap (fmap (+1)) Unfold.fromList
50−-- >>> Stream.toList $ Stream.unfold u [1..5]
29+-- >>> Stream.fold Fold.toList $ Stream.unfold u [1..5]
51-- [2,3,4,5,6]
30-- [2,3,4,5,6]
53−-- 'Unfold' streams can be transformed using transformation combinators. For
32+-- Output stream of an 'Unfold' can be transformed using transformation
54−-- example, to retain only the first two elements of an unfold:
33+-- combinators. For example, to retain only the first two elements of an
56-- >>> u = Unfold.take 2 Unfold.fromList
36-- >>> u = Unfold.take 2 Unfold.fromList
57−-- >>> Stream.toList $ Stream.unfold u [1..100]
37+-- >>> Stream.fold Fold.toList $ Stream.unfold u [1..100]
60−-- Multiple unfolds can be combined in several interesting ways. For example,
40+-- Unfolds can be nested efficiently. For example, to implement nested looping:
61−-- to generate nested looping as in imperative languages (also known as cross
62−-- product of the two streams):
64-- >>> u1 = Unfold.lmap fst Unfold.fromList
42-- >>> u1 = Unfold.lmap fst Unfold.fromList
65-- >>> u2 = Unfold.lmap snd Unfold.fromList
43-- >>> u2 = Unfold.lmap snd Unfold.fromList
66-- >>> u = Unfold.crossWith (,) u1 u2
44-- >>> u = Unfold.crossWith (,) u1 u2
67−-- >>> Stream.toList $ Stream.unfold u ([1,2,3], [4,5,6])
45+-- >>> Stream.fold Fold.toList $ Stream.unfold u ([1,2,3], [4,5,6])
68-- [(1,4),(1,5),(1,6),(2,4),(2,5),(2,6),(3,4),(3,5),(3,6)]
46-- [(1,4),(1,5),(1,6),(2,4),(2,5),(2,6),(3,4),(3,5),(3,6)]
70−-- Nested loops using unfolds provide C like performance due to complete stream
48+-- 'Unfold' @u1@ generates a stream from the first list in the input tuple,
71−-- fusion.
49+-- @u2@ generates another stream from the second list. The combines 'Unfold'
50+-- @u@ nests the two streams i.e. for each element in first stream, for each
51+-- element in second stream apply the supplied function (i.e. @(,)@) to the
54+-- This is the equivalent of the nested looping construct from imperative
55+-- languages, also known as the cross product of two streams in functional
73-- Please see "Streamly.Internal.Data.Unfold" for additional @Pre-release@
58-- Please see "Streamly.Internal.Data.Unfold" for additional @Pre-release@
74-- functions.
59-- functions.
61+-- = Creating New Unfolds
63+-- There are many commonly used unfolds provided in this module. However, you
64+-- can always create your own as well. An 'Unfold' is just a data
65+-- representation of a stream generator function. It consists of an @inject@
66+-- function which covnerts the supplied seed into an internal state of the
67+-- unfold, and a @step@ function which takes the state and generates the next
68+-- output in the stream. For those familiar with the list "Data.List.unfoldr"
69+-- function, this is a data representation of the same.
71+-- Smart constructor functions are provided in this module for constructing new
72+-- 'Unfolds'. For example, you can use the 'Unfold.unfoldr' constructor to
73+-- create an 'Unfold' from a pure step function, unfoldr uses 'id' as the
76+-- Let's define a simple pure step function:
80+-- f (x:xs) = Just (x, xs)
83+-- Create an 'Unfold' from the step function:
85+-- >>> u = Unfold.unfoldr f
89+-- >>> Stream.fold Fold.toList $ Stream.unfold u [1,2,3]
92+-- The 'Unfold.unfoldr' smart constructor is essentially the same as the list
93+-- "Data.List.unfoldr" function. We can use the same step function in both::
95+-- >>> Data.List.unfoldr f [1,2,3]
76-- = Unfolds vs. Streams
98-- = Unfolds vs. Streams
78−-- Unfolds' raison d'etre is their efficiency in nested stream operations due
100+-- The 'Unfold' abstraction for representing streams was introduced in Streamly
79−-- to complete stream fusion. 'Streamly.Prelude.concatMap' or the 'Monad'
101+-- to provide C like performance for nested looping of streams. 'Unfold' and
80−-- instance of streams use stream generation operations of the shape @a -> t m
102+-- 'Stream' abstractions are similar with the following differences:
81−-- b@ and then flatten the resulting stream. This implementation is more
103+--
82−-- powerful but does not allow for complete stream fusion. Unfolds provide
104+-- * 'Stream' is less efficient than 'Unfold' for nesting.
83−-- less powerful but more efficient 'Streamly.Prelude.unfoldMany', 'many' and
105+-- * 'Stream' is more powerful than 'Unfold'.
84−-- 'crossWith' operations as an alternative to a subset of use cases of
106+-- * 'Stream' API is more convenient for programming
85−-- 'concatMap' and 'Applicative' stream operations.
107+--
86−--
108+-- Unfolds can be easily converted to streams using 'Stream.unfold', however,
87−-- "Streamly.Prelude" exports polymorphic stream generation operations that
109+-- vice versa is not possible. To provide a familiar analogy, 'Unfold' is to
88−-- provide the same functionality as unfolds in this module. Since unfolds can
110+-- 'Stream' as 'Applicative' is to 'Monad'.
89−-- be easily converted to streams, several modules in streamly provide only
111+--
90−-- unfolds for serial stream generation. We cannot use unfolds exclusively for
112+-- To demonstrate the efficiency of unfolds, the nested loop example in the
91−-- stream generation as they do not support concurrency.
113+-- previous section can be implemented with concatMap or Monad instance of
92−
114+-- streams as follows:
118+-- x <- Stream.unfold Unfold.fromList [1,2,3]
119+-- y <- Stream.unfold Unfold.fromList [4,5,6]
123+-- As you can see, this is more convenient to write than using the 'crossWith'
124+-- unfold combinator. However, this turns out to be many times slower than the
125+-- unfold implementation. The Unfold version is equivalent in performance to
126+-- the C implementation of the same nested loop. Similarly, unfolds can be
127+-- nested with streams using the 'unfoldMany' combinator which is a much more
128+-- efficient alternative to the 'concatMap' operation.
130+-- Streams use a hybrid implementation approach using direct style as well as
131+-- CPS. Unfolds do not use CPS, therefore, lack the power that is afforded to
132+-- streams by CPS. The CPS implementation allows infinitely scalable @cons@ and
133+-- @append@ operations in streams. It is also used to implement concurrency in
136+-- To summarize, unfolds are a high performance solution to the nesting
137+-- problem. Since streams provide a more palatable API for programming, work
138+-- with streams unless you need unfolds for better performance in nesting
139+-- situations. There is little difference in the way in which unfolds and
140+-- streams are written, it is easy to adapt a stream to an unfold. If you are
141+-- writing an unfold you can convert it to stream for free using
144+-- What makes streams less efficient is also what makes them more conveient to
145+-- use and powerful. The stream data type (Stream m a) bundles the state along
146+-- with the stream generator function making it opaque, whereas an unfold
147+-- exposes the state (Unfold m s a) to the user. This allows the Unfold to be
148+-- unfolded (inlined) inside a nested loop without having to bundle the state
149+-- and the generator together, the stream state can be saved and passed
150+-- independent of the generator function. On the other hand in a stream type we
151+-- have to bundle the stream state and the generator function together to save
152+-- the stream. This makes it inefficient because it requires boxing and
153+-- constructor allocation. However, this makes streams more convenient as we do
154+-- not need to pass around the state/seed separately.
93module Streamly.Data.Unfold
156module Streamly.Data.Unfold
95 -- * Unfold Type
158 -- * Unfold Type