@@ -5,17 +5,76 @@ stream generation or an unfold operation followed by stream transformation
5functions (e.g. map) and then a stream elimination operation or a fold
5functions (e.g. map) and then a stream elimination operation or a fold
8−The default or most common stream representation used in streamly is `StreamD`
8+The default or most common stream representation used in streamly is
9+`StreamD` which is a direct style (compare with CPS representation
9−which is a direct style stream representation. All the direct style
10+`StreamK`) stream representation. All the direct style operations in
10−operations in a loop "fuse" together to form a tight machine loop eliminating
11+a loop "fuse" together to form a tight machine loop eliminating any
11−any intermediate constructors, therefore, reducing allocations and cpu cost.
12+intermediate constructors, therefore, reducing allocations and cpu cost.
12−This elimination of intermediate constructors in a loop is known as stream
13+This elimination of intermediate constructors in a stream loop is known
13−fusion.
14+as stream fusion.
15−## General
16+## Writing high performance code using streamly
18+This section outlines guidelines for writing code using the available
19+combinators. The guidelines for writing new combinators are provided in
20+the following section.
22+Note that there is no absolute benchmark for performance, most of
23+the time even without following the guidelines you may get excellent
24+performance for the task at hand. Two important points to keep in mind
27+* See if you actually need more performance
28+* See if the code you are optimizing is in fast path
32+When your performance requirements are stringent you may want to care
33+about not breaking stream fusion unnecessarily. Certain operations do
34+not fuse and act as a barrier to stream fusion. If these operations are
35+part of a loop, the loop won't fuse completely. In some cases these may
36+be necessary but in others it may be possible to replace these with
37+better fusing operations. These operations include:
39+* Avoid unnecessary use of stream append operations in code that should
40+ fuse. Append operations in general force CPS style breaking stream
41+ fusion. Some direct style append operations can fuse but they won't
42+ scale for more than a few appends.
43+* Avoid unnecessary use of the stream monad if fusion is important. You
44+ may use functor or applicative though.
45+* Concurrent stream operations cannot fuse
46+* Operations involving exception primitives like catch/throw/mask
47+ on elements of the stream cannot fuse.
48+* Stream loops involving FFI calls on the elements of the stream
50+The haddock documentation includes a note when an operation cannot fuse.
54+* Use unfolds especially when higher order operations are involved. For
55+ example, `concatUnfold` can fuse completely whereas `concatMap` would
57+* Use `outerProduct` in `Unfold` module instead of using the monad instance of
58+ streams to fuse nested loops where performance matters. See the unfold
59+ benchmarks for an example. `outerProduct` can give you C like nested loop
64+To make sure fusion can occur INLINE any operations that are part of a
65+stream loop but factored out as separate functions.
69+Keep the fold and scan accumulators strict. You could also consider
70+using mutable state in a fold accumulator if the state is large. A
71+large immutable structure as an accumulator may cause optimization
72+issues. See the `WordCount` example for an example of this case.
74+## Writing streamly combinators
17−To enable stream fusion in direct stream all the operations that are part of a
76+To enable stream fusion in a direct style stream all the operations that
18−loop must be inlined:
77+are part of a loop must be inlined:
20* Use an explicit INLINE pragma on any combinator that could be part of a loop
79* Use an explicit INLINE pragma on any combinator that could be part of a loop
21 to ensure they will be inlined.
80 to ensure they will be inlined.
@@ -24,7 +83,7 @@ loop must be inlined:
24 is consuming. Use appropriate inline phases to ensure proper inline ordering
83 is consuming. Use appropriate inline phases to ensure proper inline ordering
25 when required.
84 when required.
27−## Streams and Unfolds
86+### Streams and Unfolds
29Stream and Unfold State. Direct style stream and unfold combinators use a
88Stream and Unfold State. Direct style stream and unfold combinators use a
30state data type to represent the internal state of the stream/unfold generator.
89state data type to represent the internal state of the stream/unfold generator.
@@ -32,21 +91,52 @@ state data type to represent the internal state of the stream/unfold generator.
32* In some cases we may have to add a `FUSE` annotation on the state type to
91* In some cases we may have to add a `FUSE` annotation on the state type to
33 ensure that all the internal join points created by GHC are inlined with the
92 ensure that all the internal join points created by GHC are inlined with the
34 help of the `fusion-plugin`.
93 help of the `fusion-plugin`.
35−* In rare cases we may need to use a strictness annotation on the state.
94+* In rare cases we may need to use a strictness annotation on the state
36− `parseMany` is one such example.
95+ to allow fusion. `parseMany` and `splitOnSeq` are such examples.
38Step function of a stream or unfold:
97Step function of a stream or unfold:
40−* Never make the step function recursive. Always use the `Skip` constructor to
99+* Do not introduce unnecessary states. If there is only one entry point of a
41− remove recursion. Recursive step function can introduce optimization barriers
100+ state then perhaps you want to collapse the state into the state from which
42− that are harder to remove by the GHC simplifier.
101+ it is called. More states means more jumps and may affect code locality, and
43−* Try to make sure that there is a single yield point in the whole state
102+ efficiency of low level code (e.g. register allocation) because of
44− machine. A single yield point is usually desirable, however not always
103+ independent placement of the code blocks in different states.
45− necessary.
104+* Keep minimum possible data in state. More variables in state may lead
105+ to poor performance because the state may not fit into registers and the
106+ spill may cause allocations on each iteration of the loop. Mutable state
107+ may help in such cases.
46* The step function must be annotated such that it gets inlined after the main
108* The step function must be annotated such that it gets inlined after the main
47 combinator (`INLINE_LATE`).
109 combinator (`INLINE_LATE`).
111+Multiple yield points or single?:
113+* A single yield point is usually desirable, however, not always necessary.
114+ In some cases multiple yield points may in fact be needed for fusion,
115+ see `splitOnSeq` for an example. Or maybe its fusing because of a
116+ direct yield instead of going through an indirect common yielding
119+Recursion in step function:
121+* In general, we avoid making the step function recursive. Use the
122+ `Skip` constructor to remove recursion. Recursive step function can
123+ introduce optimization barriers that are harder to remove by the GHC
126+ However, in some cases it may be better to have a local recursive
127+ loop. A recursive loop can help us avoid threading around some large
128+ state values every time. Values that do not change across a loop can
129+ be factored in the scope outside the loop (static argument transform),
130+ this way we can create a local loop which is more efficient than
131+ threading around the state in a larger loop. See the `splitOnSeq`
132+ combinator as an example where we use a local recursive loop, it fuses
133+ and is significantly efficient compared to using `Skip`.
135+ In a local recursive loop use SPEC and annotate even the rest of the
136+ loop arguments as strict where needed. We have observed that when the
137+ arguments were not strict the loop does not fuse (splitOnSeq).
49−## Fold and Parser drivers
139+### Fold and Parser drivers
51Recursive loop closing operations:
141Recursive loop closing operations:
@@ -55,7 +145,7 @@ Recursive loop closing operations:
55 "Spec Constructor" optimization removes boxed arguments and reduces
145 "Spec Constructor" optimization removes boxed arguments and reduces
56 allocations.
146 allocations.
58−## Fold and Parser combinators
148+### Fold and Parser combinators
60State of a Fold or Parser:
150State of a Fold or Parser:
@@ -70,16 +160,16 @@ The step function of a Fold or Parser:
71* Never make the fold step recursive, recursion creates an optimization barrier
161* Never make the fold step recursive, recursion creates an optimization barrier
72 for the GHC simplifier. Use `Partial` or `Continue` constructors to avoid
162 for the GHC simplifier. Use `Partial` or `Continue` constructors to avoid
73− recursion.
163+ recursion.
75* Sometimes you may need an explicit INLINE on the step function.
165* Sometimes you may need an explicit INLINE on the step function.
77−## NOINLINE, isolating the closed loop
167+### NOINLINE, isolating the closed loop
79We want the loop iterations to be optimized and the loop stages to be fused to
169We want the loop iterations to be optimized and the loop stages to be fused to
80generate a tighter loop. However, it is not necessarily optimal to inline the
170generate a tighter loop. However, it is not necessarily optimal to inline the
81whole loop itself into a parent function. For example, consider the following
171whole loop itself into a parent function. For example, consider the following
82−function:
172+function in the `FileSystem.Handle` benchmarks:
85{-# NOINLINE readWriteAfter_Stream #-}
175{-# NOINLINE readWriteAfter_Stream #-}
@@ -91,13 +181,20 @@ readWriteAfter_Stream inh devNull =
92If this is inlined into a parent benchmark group list, this leads to
182If this is inlined into a parent benchmark group list, this leads to
93many times performance degradation. That's because inlining the loop
183many times performance degradation. That's because inlining the loop
94−into a bigger structure may interfere with the optimization of the loop
184+into a bigger structure interferes with the optimization of the loop
95−itself and it may not fuse. Whereas it is desirable to INLINE all
185+body itself and it may not fuse. Whereas it is desirable to INLINE all
96the stages of a loop, it is often not useful to inline the whole loop
186the stages of a loop, it is often not useful to inline the whole loop
97itself, in fact we may have to occasionally use a `NOINLINE` so that the
187itself, in fact we may have to occasionally use a `NOINLINE` so that the
98compiler does not inline it.
188compiler does not inline it.
192+It is not always useful to specialize function calls for all constructors, some
193+constructors may just add to code bloat or add overhead of passing unboxed
194+values in a loop. In such cases the `NoSpecConstr` annotation can be useful.
195+See the `parselMx'` function in `StreamD` module.
100−## StreamK operations
197+### StreamK operations
102StreamK uses foldr/build fusion to a very limited degree. StreamK is not the
199StreamK uses foldr/build fusion to a very limited degree. StreamK is not the
103primary representation in streamly but is used for several operations that
200primary representation in streamly but is used for several operations that
@@ -105,12 +202,21 @@ cannot scale in StreamD representation. StreamK is relatively immune to compiler
105optimizations. In some cases you may need an INLINE pragma to improve the
202optimizations. In some cases you may need an INLINE pragma to improve the
106performance.
203performance.
205+### How to debug non-fusing code?
207+Strip down the code to a minimal version until it starts fusing and then
208+start building it up from there adding more things incrementally. At
209+each stage keep checking if the code is fusing. At some point it
210+won't fuse. See what we added that made the code not fuse. Go through
211+the guidelines above to check if we did something that that is not
212+recommended. Or raise an issue for GHC to be fixed if possible.
108−# GHC Optimizations
214+# GHC Optimizations In Streamly
110There are three important levels of optimizations used in streamly:
216There are three important levels of optimizations used in streamly:
112−* CPS style stream representation (StreamK) to direct style (StreamD) and
218+* CPS style stream representation (StreamK) to direct style (StreamD)
113− vice-versa using rewrite rules
219+ conversion and vice-versa using rewrite rules
114* Stream fusion using case-of-case optimization in StreamD
220* Stream fusion using case-of-case optimization in StreamD
115* foldr/build fusion using rewrite rules in StreamK
221* foldr/build fusion using rewrite rules in StreamK