@@ -0,0 +1,215 @@
2+(c) 2022, Composewell Technologies.
3+SPDX-License-Identifer: BSD-3-Clause
6+# Haskell Lazy Evaluation
8+Haskell is fundamentally different from other languages. In most
9+languages, a variable name refers to some concrete data in memory; in
10+Haskell, a variable may refer to concrete data or it could refer to
11+a computation or expression which when evaluated would produce the
12+concrete data that the variable is referring to. When the value of the
13+variable is needed, the computation is evaluated and replaced by
16+When a Haskell program starts, the entire program is an unevaluated
17+expression which is evaluated on-demand as needed. The top level
18+expression is forced to produce data when a result is demanded from it
19+for IO. Parts of the expression are reduced as needed, the reduction
20+process can be controlled by the programmer.
22+## Unevaluated Expressions
24+When we assign a value to a name, the name refers to the unevaluated
25+expression and not what it would produce when evaluated. For example:
28+>>> infiniteList = [1..]
31+The name `infiniteList` refers to an expression which when evaluated
32+would generate the list. `infiniteList` is not an actual physical list
33+but just an expression to generate the list `when needed`. Because we
34+are not storing the actual list in memory, we can represent an infinite
35+list without requiring infinite amount of memory.
37+Let's try to get the length of this list:
40+>>> len = length infiniteList
43+Like before, the name `len` refers to an expression that when evaluated
44+would give us the length of the list. Till now Haskell has been saying
45+that I duly noted how to do what you want to do but I will do it only
46+when you need it. Let's ask it to do that now:
53+Now, Haskell has no choice but to start evaluating the expression but we
54+know it can never finish because the list is infinite.
56+All expressions in Haskell are unevaluated by default. Haskell
57+evaluates the expressions only when it is absolutely required e.g. when
58+we have to print something, or write it to a file or to network.
60+## On-demand Expression Evaluation
62+Let us illustrate expression evaluation in Haskell with an example. Write
63+this little program in a file `example.hs`:
67+ let largeList = [1..1000000::Int]
72+The expression `largeList` is never evaluated to generate the entire
73+list at once. It is evaluated on-demand one element at a time by the
74+`length` function. The `length` function is in turn evaluated by the
75+`print` function when it evaluates its argument `n`. The `print`
76+function is evaluated by the top level program because it wants to print
79+We can verify that the whole list is not stored in memory by looking at
80+the memory usage of this program. Let's compile and run it:
86+ 51,480 bytes allocated in the heap
87+ 24 bytes copied during GC
88+ 44,328 bytes maximum residency (1 sample(s))
91+`largeList` is an expression representing a million element list, but
92+the program takes only 44K maximum memory. The `length` function
93+evaluates `largeList` one element at a time to count them, as soon as
94+an element is counted it can be garbage collected as there are no more
95+references to it in the entire program.
97+## Expressions Holding up Heap Memory
99+Let's try this program instead:
103+ let largeList = [1..1000000::Int]
113+ 64,051,552 bytes allocated in the heap
114+ 400 bytes copied during GC
115+ 26,233,360 bytes maximum residency (4 sample(s))
118+This program is showing 26MB maximum residency instead of 44K
119+earlier. Why is it so? To evaluate the first element of the tuple, `x1`,
120+we need to evaluate the entire list to get to the last element. However,
121+the head of the list is referenced by the element `x2` as well which
122+is yet to be evaluated and used. Therefore, we need to retain the head
123+until `x2` is printed and freed. But head of the list is referencing the
124+next element in the list which in turn is referring to the next element
125+and so on, therefore, the entire list is retained in memory between the
126+evaluation of `x1` and `x2`.
128+If we change the print statement to `(x2, x1)` then we do not need to
129+retain the entire list between the two evaluations.
131+When memory is unexpectedly retained in a program it is known as a
132+"space-leak" in the Haskell parlance.
134+## Forcing Expression Evaluation
136+Even if there is no explicit dependency in the program forcing
137+evaluation of an expression, the programmer can force the evaluation of
138+an expression using `seq`:
142+ let largeList = [1..1000000::Int]
145+ in print (x1 `seq` x2)
148+Similar to the example in previous section, this example also retains
149+the entire list in memory because `seq` forces `x1` to be evaluated
150+before `x2` and then returns `x2`, the evaluated list is held until `x2` is
153+## Avoiding Unnecessary Memory Retention
155+Memory retention can be avoided by changing the way we construct the
156+expression to process the data. We can use Streamly to avoid memory
157+retention in the above program.
159+Let's first write it using streamly and reproduce the same problem:
162+import Data.Functor.Identity (runIdentity)
163+import qualified Streamly.Data.Stream as Stream
164+import qualified Streamly.Data.Fold as Fold
167+ let largeList = Stream.fromList [1..1000000::Int]
168+ x1 = runIdentity $ Stream.fold Fold.latest largeList
169+ x2 = runIdentity $ Stream.fold Fold.one largeList
173+This program has the same behavior as the list based program, and for
179+(Just 1000000,Just 1)
180+ 64,052,520 bytes allocated in the heap
181+ 400 bytes copied during GC
182+ 26,233,360 bytes maximum residency (4 sample(s))
185+However, we can compose the two folds using the `teeWith` combinator
186+such that we distribute a each element of the list to both the folds
187+simultaneously, thus avoiding the retainment of the entire list:
190+import Data.Functor.Identity (runIdentity)
191+import qualified Streamly.Data.Stream as Stream
192+import qualified Streamly.Data.Fold as Fold
195+ let largeList = Stream.fromList [1..1000000::Int]
196+ x = Fold.teeWith (,) Fold.latest Fold.one
197+ in print $ runIdentity $ Stream.fold x largeList
203+(Just 1000000,Just 1)
204+ 64,052,736 bytes allocated in the heap
205+ 280 bytes copied during GC
206+ 44,328 bytes maximum residency (2 sample(s))
209+This program uses constant small memory footprint (44K) irrespective of
210+the order of the elements in the tuple. When an element is generated by
211+the list it is supplied to both the folds before generating the next
212+element, therefore, it can be immediately freed.
214+Streamly covers all the functionality of standard Haskell lists but in addition
215+provides powerful composition tools for better efficiency.