@@ -0,0 +1,215 @@
1+<!--
2+(c) 2022, Composewell Technologies.
3+SPDX-License-Identifer: BSD-3-Clause
4+-->
5+
6+# Haskell Lazy Evaluation
7+
8+Haskell is fundamentally different from other languages. In most
9+languages, a variable name refers to some concrete data in memory; in
10+Haskell, a variable may refer to concrete data or it could refer to
11+a computation or expression which when evaluated would produce the
12+concrete data that the variable is referring to. When the value of the
13+variable is needed, the computation is evaluated and replaced by
14+the result.
15+
16+When a Haskell program starts, the entire program is an unevaluated
17+expression which is evaluated on-demand as needed. The top level
18+expression is forced to produce data when a result is demanded from it
19+for IO. Parts of the expression are reduced as needed, the reduction
20+process can be controlled by the programmer.
21+
22+## Unevaluated Expressions
23+
24+When we assign a value to a name, the name refers to the unevaluated
25+expression and not what it would produce when evaluated. For example:
26+
27+```
28+>>> infiniteList = [1..]
29+```
30+
31+The name `infiniteList` refers to an expression which when evaluated
32+would generate the list. `infiniteList` is not an actual physical list
33+but just an expression to generate the list `when needed`. Because we
34+are not storing the actual list in memory, we can represent an infinite
35+list without requiring infinite amount of memory.
36+
37+Let's try to get the length of this list:
38+
39+```
40+>>> len = length infiniteList
41+```
42+
43+Like before, the name `len` refers to an expression that when evaluated
44+would give us the length of the list. Till now Haskell has been saying
45+that I duly noted how to do what you want to do but I will do it only
46+when you need it. Let's ask it to do that now:
47+
48+```
49+>>> len
50+...hangs forever
51+```
52+
53+Now, Haskell has no choice but to start evaluating the expression but we
54+know it can never finish because the list is infinite.
55+
56+All expressions in Haskell are unevaluated by default. Haskell
57+evaluates the expressions only when it is absolutely required e.g. when
58+we have to print something, or write it to a file or to network.
59+
60+## On-demand Expression Evaluation
61+
62+Let us illustrate expression evaluation in Haskell with an example. Write
63+this little program in a file `example.hs`:
64+
65+```haskell
66+main =
67+ let largeList = [1..1000000::Int]
68+ n = length largeList
69+ in print n
70+```
71+
72+The expression `largeList` is never evaluated to generate the entire
73+list at once. It is evaluated on-demand one element at a time by the
74+`length` function. The `length` function is in turn evaluated by the
75+`print` function when it evaluates its argument `n`. The `print`
76+function is evaluated by the top level program because it wants to print
77+`n` to the console.
78+
79+We can verify that the whole list is not stored in memory by looking at
80+the memory usage of this program. Let's compile and run it:
81+
82+```
83+$ ghc -O example.hs
84+$ ./example +RTS -s
85+1000000
86+ 51,480 bytes allocated in the heap
87+ 24 bytes copied during GC
88+ 44,328 bytes maximum residency (1 sample(s))
89+```
90+
91+`largeList` is an expression representing a million element list, but
92+the program takes only 44K maximum memory. The `length` function
93+evaluates `largeList` one element at a time to count them, as soon as
94+an element is counted it can be garbage collected as there are no more
95+references to it in the entire program.
96+
97+## Expressions Holding up Heap Memory
98+
99+Let's try this program instead:
100+
101+```haskell
102+main = do
103+ let largeList = [1..1000000::Int]
104+ x1 = last largeList
105+ x2 = head largeList
106+ in print (x1, x2)
107+```
108+
109+```
110+$ ghc -O example.hs
111+$ ./example +RTS -s
112+(1000000,1)
113+ 64,051,552 bytes allocated in the heap
114+ 400 bytes copied during GC
115+ 26,233,360 bytes maximum residency (4 sample(s))
116+```
117+
118+This program is showing 26MB maximum residency instead of 44K
119+earlier. Why is it so? To evaluate the first element of the tuple, `x1`,
120+we need to evaluate the entire list to get to the last element. However,
121+the head of the list is referenced by the element `x2` as well which
122+is yet to be evaluated and used. Therefore, we need to retain the head
123+until `x2` is printed and freed. But head of the list is referencing the
124+next element in the list which in turn is referring to the next element
125+and so on, therefore, the entire list is retained in memory between the
126+evaluation of `x1` and `x2`.
127+
128+If we change the print statement to `(x2, x1)` then we do not need to
129+retain the entire list between the two evaluations.
130+
131+When memory is unexpectedly retained in a program it is known as a
132+"space-leak" in the Haskell parlance.
133+
134+## Forcing Expression Evaluation
135+
136+Even if there is no explicit dependency in the program forcing
137+evaluation of an expression, the programmer can force the evaluation of
138+an expression using `seq`:
139+
140+```haskell
141+main = do
142+ let largeList = [1..1000000::Int]
143+ x1 = last largeList
144+ x2 = head largeList
145+ in print (x1 `seq` x2)
146+```
147+
148+Similar to the example in previous section, this example also retains
149+the entire list in memory because `seq` forces `x1` to be evaluated
150+before `x2` and then returns `x2`, the evaluated list is held until `x2` is
151+returned.
152+
153+## Avoiding Unnecessary Memory Retention
154+
155+Memory retention can be avoided by changing the way we construct the
156+expression to process the data. We can use Streamly to avoid memory
157+retention in the above program.
158+
159+Let's first write it using streamly and reproduce the same problem:
160+
161+```haskell
162+import Data.Functor.Identity (runIdentity)
163+import qualified Streamly.Data.Stream as Stream
164+import qualified Streamly.Data.Fold as Fold
165+
166+main = do
167+ let largeList = Stream.fromList [1..1000000::Int]
168+ x1 = runIdentity $ Stream.fold Fold.latest largeList
169+ x2 = runIdentity $ Stream.fold Fold.one largeList
170+ in print (x1, x2)
171+```
172+
173+This program has the same behavior as the list based program, and for
174+the same reasons.
175+
176+```
177+$ ghc -O2 example.hs
178+$ ./example +RTS -s
179+(Just 1000000,Just 1)
180+ 64,052,520 bytes allocated in the heap
181+ 400 bytes copied during GC
182+ 26,233,360 bytes maximum residency (4 sample(s))
183+```
184+
185+However, we can compose the two folds using the `teeWith` combinator
186+such that we distribute a each element of the list to both the folds
187+simultaneously, thus avoiding the retainment of the entire list:
188+
189+```haskell
190+import Data.Functor.Identity (runIdentity)
191+import qualified Streamly.Data.Stream as Stream
192+import qualified Streamly.Data.Fold as Fold
193+
194+main = do
195+ let largeList = Stream.fromList [1..1000000::Int]
196+ x = Fold.teeWith (,) Fold.latest Fold.one
197+ in print $ runIdentity $ Stream.fold x largeList
198+```
199+
200+```
201+$ ghc -O2 example.hs
202+$ ./example +RTS -s
203+(Just 1000000,Just 1)
204+ 64,052,736 bytes allocated in the heap
205+ 280 bytes copied during GC
206+ 44,328 bytes maximum residency (2 sample(s))
207+```
208+
209+This program uses constant small memory footprint (44K) irrespective of
210+the order of the elements in the tuple. When an element is generated by
211+the list it is supplied to both the folds before generating the next
212+element, therefore, it can be immediately freed.
213+
214+Streamly covers all the functionality of standard Haskell lists but in addition
215+provides powerful composition tools for better efficiency.