@@ -0,0 +1,225 @@
1+# Guidelines for writing well optimized code
2+
3+Stream operations are always part of a loop. Usually the loop consiste of a
4+stream generation or an unfold operation followed by stream transformation
5+functions (e.g. map) and then a stream elimination operation or a fold
6+operation.
7+
8+The default or most common stream representation used in streamly is `StreamD`
9+which is a direct style stream reprensentation. All the direct style
10+operations in a loop "fuse" together to form a tight machine loop eliminating
11+any intermediate constructors, therefore, reducing allocations and cpu cost.
12+This elimination of intermediate constructors in a loop is known as stream
13+fusion.
14+
15+## General
16+
17+To enable stream fusion in direct stream all the operations that are part of a
18+loop must be inlined:
19+
20+* Use an explicit INLINE pragma on any combinator that could be part of a loop
21+ to ensure they will be inlined.
22+* When a higher order function consumes another function then ensure that the
23+ higher order function is inlined in the same phase or before the function it
24+ is consuming. Use appropriate inline phases to ensure proper inline ordering
25+ when required.
26+
27+## Streams and Unfolds
28+
29+Stream and Unfold State. Direct style stream and unfold combinators use a
30+state data type to represent the internal state of the stream/unfold generator.
31+
32+* In some cases we may have to add a `FUSE` annotation on the state type to
33+ ensure that all the internal join points created by GHC are inlined with the
34+ help of the `fusion-plugin`.
35+* In rare cases we may need to use a strictness annotation on the state.
36+ `parseMany` is one such example.
37+
38+Step function of a stream or unfold:
39+
40+* Never make the step function recursive. Always use the `Skip` constructor to
41+ remove recursion. Recursive step function can introduce optimization barriers
42+ that are harder to remove by the GHC simplifier.
43+* Try to make sure that there is a single yield point in the whole state
44+ machine. A single yield point is usually desirable, however not always
45+ necessary.
46+* The step function must be annotated such that it gets inlined after the main
47+ combinator (`INLINE_LATE`).
48+
49+## Fold and Parser drivers
50+
51+Recursive loop closing operations:
52+
53+* When writing recursive looping combinators using StreamD (e.g. foldlM' in
54+ StreamD) use a strict SPEC argument in the recursive loops to ensure that
55+ "Spec Constructor" optimization removes boxed arguments and reduces
56+ allocations.
57+
58+## Fold and Parser combinators
59+
60+State of a Fold or Parser:
61+
62+* The accumulator of a fold must be a strict data structure. Use the strict
63+ data structures provided in `Streamly.Internal.Data` for this purpose or use
64+ explicit bang annotation to make the data strict.
65+* In some cases you may need a `FUSE` annotation on the state type to ensure
66+ that any internal join points created by GHC are always inlined with the help
67+ of the fusion-plugin.
68+
69+The step function of a Fold or Parser:
70+
71+* Never make the fold step recursive, recursion creates an optimization barrier
72+ for the GHC simplifier. Use `Partial` or `Continue` constructors to avoid
73+ recursion.
74+
75+* Sometimes you may need an explicit INLINE on the step function.
76+
77+## StreamK operations
78+
79+StreamK uses foldr/build fusion to a very limited degree. StreamK is not the
80+primary representation in streamly but is used for several operations that
81+cannot scale in StreamD represenation. StreamK is relatively immune to compiler
82+optimizations. In some cases you may need an INLINE pragma to improve the
83+performance.
84+
85+# GHC Optimizations
86+
87+There are three important levels of optimizations used in streamly:
88+
89+* CPS style stream representation (StreamK) to direct style (StreamD) and
90+ vice-versa using rewrite rules
91+* Stream fusion using case-of-case optimization in StreamD
92+* foldr/build fusion using rewrite rules in StreamK
93+
94+## INLINE Phases
95+
96+Inlining of functions at the right time is crucial for all these optimization
97+to work. A missing inline or inline in an incorrect GHC simplifier phase can
98+adversely impact performance. We use three builtin phases of GHC simplifier
99+for inlining i.e. phase 0, 1 and 2. We have defined them as follows in
100+ `src/inline.hs`:
101+
102+```
103+#define INLINE_EARLY INLINE [2]
104+#define INLINE_NORMAL INLINE [1]
105+#define INLINE_LATE INLINE [0]
106+```
107+
108+We also use INLINE [3] in some cases.
109+
110+## `fromStreamD/toStreamD` Fusion
111+
112+The combinators in `Streamly.Prelude` are defined in terms of combinators in
113+`Streamly.Internal.Data.Stream.StreamD` (Direct style streams) or
114+`Streamly.Internal.Data.Stream.StreamK` (CPS style streams). We convert the
115+stream from `StreamD` to `StreamK` representation or vice versa in some cases.
116+
117+Most operations use the StreamD representation, however, stream append
118+operations and monad instances use StreamK representation because StreamD would
119+not perform well in these cases. We use rewrite rules to convert from one
120+representation to another when required. For this reason the combinators in
121+Streamly.Prelude are written using fromStreamD/fromStreamK etc.
122+
123+In the first inlining phase (INLINE_EARLY or INLINE) we expand the combinators
124+in `Streamly.Prelude` into fromStreamD/fromStreamK/toStreamD/toStreamK and
125+combinators defined in StreamD or StreamK modules. Once we do that
126+fromStreamD/toStreamD get exposed and we can apply rewrite rules to rewrite
127+transformations like `fromStreamK . toStreamK` to `id`. A plain `INLINE`
128+pragma is usually enough on combinators in `Streamly.Prelude`.
129+
130+```
131+{-# RULES "fromStreamK/toStreamK fusion"
132+ forall s. toStreamK (fromStreamK s) = s #-}
133+```
134+
135+Also, we have to prevent fromStreamK and toStreamK themselves from inlining in
136+this phase so that rewrite rules can be applied on them, therefore, we annotate
137+these functions with `INLINE_LATE`.
138+
139+## Fallback Rules
140+
141+In some cases, if the operation could not fuse we want to use a fallback
142+rewrite rule in the next phase. For such cases we use the INLINE_EARLY phase
143+for the first rewrite and the INLINE_NORMAL phase for the fallback rules.
144+
145+The fallback rules make sure that if we could not fuse the direct style
146+operations then better use the CPS style operation, because unfused direct
147+style would have worse performance than the CPS style ops.
148+
149+```
150+{-# INLINE_EARLY unfoldr #-}
151+unfoldr :: (Monad m, IsStream t) => (b -> Maybe (a, b)) -> b -> t m a
152+unfoldr step seed = fromStreamS (S.unfoldr step seed)
153+{-# RULES "unfoldr fallback to StreamK" [1]
154+ forall a b. S.toStreamK (S.unfoldr a b) = K.unfoldr a b #-}
155+```
156+
157+## High Level Operation Fusion
158+
159+Since each high level combinator in `Streamly.Prelude` is wrapped in
160+`fromStreamD/toStreamD` etc. the combinator fusion cannot work unless we have
161+removed those and exposed consecutive operations e.g. a `map` followed by
162+another `map`. Assuming that redundant `fromStreamK/toStreamK` have been
163+removed in the `INLINE_EARLY` phase, we can then apply the combinator fusion
164+rules in the `INLINE_NORMAL` phase. For example, we can fuse two `map`
165+operations into a single `map` operation. Note that now we have exposed the
166+`StreamD/StreamK` implementations of combinators and the rules would apply on
167+those.
168+
169+## Inlining Higher Order Functions
170+
171+Note that partially applied functions cannot be inlined. So if we have a code
172+like this:
173+
174+```
175+concatMap1 src = runStream $ S.concatMap (S.replicate 3) src
176+```
177+
178+We want to ensure that `concatMap` gets inlined before `replicate` so that
179+`replicate` becomes fully applied before it gets inlined. Currently ensuring
180+that both of them are inlined in the same phase (`INLINE_NORMAL`) seems to be
181+enough to achieve that. In general, we should try to ensure that higher order
182+functions are inlined before or in the same phase as the functions they can
183+consume as arguments. This means `StreamD` combinators should not be marked
184+as `INLINE` or `INLINE_EARLY`, instead they should all be marked as
185+`INLINE_NORMAL` because higher order funcitons like `concatMap`/`map`/`mapM`
186+etc are marked as `INLINE_NORMAL`. `StreamD` functions in other modules like
187+`Streamly.Data.Array.Storable.Foreign` should also follow the same rules.
188+
189+## Stream Fusion
190+
191+In StreamD combinators, inlining the inner step or loop functions too early
192+i.e. in the same pahse or before the outer function is inlined may block stream
193+fusion opportunities. Therefore, the inner step functions and folding loops are
194+marked as INLINE_LATE.
195+
196+## Specialization
197+
198+In some cases, the `step` function in `StreamD` does not get specialized when
199+inlined unless it is provided with an explicit signature or made a lambda, for
200+example, in the `replicate/replicateM` combinator we need the type annotation
201+on `i` to get it specialized:
202+
203+```
204+ {-# INLINE_LATE step #-}
205+ step _ (i :: Int) =
206+ if i <= 0
207+ then return Stop
208+ else do
209+ x <- action
210+ return $ Yield x (i - 1)
211+```
212+
213+`-flate-specialise` also helps in this case.
214+
215+## Stream and Fold State Data Structures
216+
217+Since state is an internal data structure threaded around in the loop, it is a
218+good practice to use strict unboxed fields for state data structures where
219+possible. In most cases it is not necessary, but in some cases it may affect
220+fusion and make a difference of 10x performance or more. For example, using
221+non-strict fields can increase the code size for internal join points and
222+functions created during transformations, which can affect the inlining of
223+these code blocks which in turn can affect stream fusion.
224+
225+See https://gitlab.haskell.org/ghc/ghc/issues/17075 .