@@ -0,0 +1,275 @@
1+# Upgrading to streamly 0.9.0
2+
3+`Streamly.Prelude` module has been deprecated, equivalent
4+functionality is covered by the `Streamly.Data.Stream`,
5+`Streamly.Data.Stream.Prelude`, and `Streamly.Data.Fold` modules. The
6+new modules use a monomorphic `Stream` type instead of the polymorphic
7+`IsStream t` type.
8+
9+`Streamly.Data.Stream` module and the `Stream` type are designed
10+for writing high-performance fused pipelines not involving explicit
11+recursion. For writing code that may require recursive function calls,
12+`Streamly.Data.Stream.StreamK` module and the `StreamK` type have been
13+introduced which provide a CPS based stream implementation. `Stream` and
14+`StreamK` types can be easily interconverted. These changes have been made to
15+make performance robust and not rely on GHC rewrite rules which could be
16+fragile. For example, GHC 9.0.x had broken the rewrite rule which was
17+not fixed until GHC-9.2.2. This split also gives more power and
18+transparency of the performance behavior to the programmer.
19+
20+Instead of using separate stream types for concurrent code, now
21+you have to use explicit concurrent combinators with appropriate
22+concurrency parameters. These combinators are available in the
23+`Streamly.Data.Stream.Prelude` module. This change has been made to allow
24+programmers to control concurrent behavior in a more robust way and
25+reduce pitfalls.
26+
27+The old code can be adapted to use the new modules with some
28+changes. More details about this is supplied in the following
29+sections. Assume the following imports in the code snippets below:
30+
31+```haskell docspec
32+>>> import qualified Streamly.Data.Stream as Stream
33+>>> import qualified Streamly.Data.StreamK as StreamK
34+>>> import qualified Streamly.Data.Stream.Prelude as Stream
35+>>> import qualified Streamly.Data.Fold as Fold
36+>>> import qualified Streamly.Data.Parser as Parser
37+```
38+
39+## The `Stream` and `StreamK` types
40+
41+The following types are removed:
42+
43+* `IsStream`
44+* `ZipSerialM`
45+* `ZipAsyncM`
46+* `WSerialT`
47+* `WAsyncT`
48+* `SerialT`
49+* `ParallelT`
50+* `AsyncT`
51+* `AheadT`
52+
53+In the new release, the `Stream` type is the primary stream type that
54+you will use most of the time. You can think of it as a replacement for
55+the `SerialT` type. However, it does not provide an Applicative or Monad
56+instance.
57+
58+The `CrossStream` type in `Streamly.Internal.Data.Stream.StreamD` is a
59+wrapper over `Stream` type supplying the Monad instance. However, see
60+the "Performance Notes" section in the `Streamly.Data.Stream` module for
61+limitations of the `Stream` type. The `StreamK` type and `CrossStreamK`
62+(in Streamly.Internal.Data.Stream.StreamK) could be used to overcome the
63+limitations of `Stream` type.
64+
65+If required, you can use the template-haskell functions in
66+`Streamly.Data.Stream.MkType` to create stream type wrappers (like
67+ZipSerialM or WSerialT) with custom Applicative or Monadic properties.
68+But in general, try to avoid specific types and use explicit functions
69+from the stream module.
70+
71+`adapt` is not needed anymore.
72+
73+```haskell docspec
74+>>> (.:) = StreamK.cons
75+>>> cons = StreamK.cons
76+>>> wSerial = StreamK.interleave
77+>>> serial = StreamK.append
78+>>> fromIndicesM f = Stream.mapM f $ Stream.enumerateFrom 0
79+>>> fromIndices f = fmap f $ Stream.enumerateFrom 0
80+>>> fromListM = Stream.sequence . Stream.fromList
81+>>> fromFoldable = StreamK.toStream . StreamK.fromFoldable
82+>>> fromFoldableM = Stream.sequence . fromFoldable
83+```
84+
85+## Stream folding functions
86+
87+Explicit stream fold functions have been omitted from the new stream
88+module. You can use the following equivalent definitions:
89+
90+```haskell docspec
91+>>> toList = Stream.fold Fold.toList
92+>>> the = Stream.fold Fold.the
93+>>> sum = Stream.fold Fold.sum
94+>>> product = Stream.fold Fold.product
95+>>> or = Stream.fold Fold.or
96+>>> null = Stream.fold Fold.null
97+>>> elemIndex a = Stream.fold (Fold.elemIndex a)
98+>>> elem a = Stream.fold (Fold.elem a)
99+>>> notElem a = Stream.fold (Fold.notElem a)
100+>>> minimumBy ordering = Stream.fold (Fold.minimumBy ordering)
101+>>> minimum = Stream.fold Fold.minimum
102+>>> maximumBy ordering = Stream.fold (Fold.maximumBy ordering)
103+>>> maximum = Stream.fold Fold.maximum
104+>>> mapM_ f = Stream.fold (Fold.drainMapM f)
105+>>> lookup a = Stream.fold (Fold.lookup a)
106+>>> length = Stream.fold Fold.length
107+>>> last = Stream.fold Fold.latest
108+>>> head = Stream.fold Fold.one
109+>>> foldr f a = Stream.fold (Fold.foldr' f a)
110+>>> foldlM' f a = Stream.fold (Fold.foldlM' f a)
111+>>> foldl1 f = Stream.fold (Fold.foldl1' f)
112+>>> foldl' f a = Stream.fold (Fold.foldl' f a)
113+>>> findIndex eq = Stream.fold (Fold.findIndex eq)
114+>>> find eq = Stream.fold (Fold.find eq)
115+>>> findM eq = Stream.fold (Fold.findM eq)
116+>>> drainWhile predicate = Stream.fold Fold.drain . Stream.takeWhile predicate
117+>>> drainN i = Stream.fold Fold.drain . Stream.take i
118+>>> drain = Stream.fold Fold.drain
119+>>> any predicate = Stream.fold (Fold.any predicate)
120+>>> and = Stream.fold Fold.and
121+>>> all predicate = Stream.fold (Fold.all predicate)
122+>>> (!!) i = Stream.fold (Fold.index i)
123+>>> tail = Streamly.Internal.Data.Stream.StreamK.tail
124+>>> init = Streamly.Internal.Data.Stream.StreamK.init
125+>>> foldrM = Streamly.Internal.Data.Stream.StreamD.foldrM
126+```
127+
128+Mapping functions:
129+
130+```haskell docspec
131+>>> map = fmap
132+```
133+
134+Similarly for scanning use `Stream.scan` or `Stream.postscan`
135+with an appropriate fold.
136+
137+```haskell docspec
138+>>> scanl' f z = Stream.scan (Fold.foldl' f z)
139+>>> scanlM' f z = Stream.scan (Fold.foldlM' f z)
140+>>> postscanl' f z = Stream.postscan (Fold.foldl' f z)
141+>>> postscanlM' f z = Stream.postscan (Fold.foldlM' f z)
142+>>> scanl1' f = Stream.catMaybes . Stream.scan (Fold.foldl1' f)
143+>>> scanl1M' f = Stream.catMaybes . Stream.scan (Fold.foldlM1' f)
144+>>> concatMapWith = StreamK.concatMapWith
145+>>> concatFoldableWith f = Prelude.foldr f StreamK.nil
146+>>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
147+>>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
148+```
149+
150+Filters:
151+
152+```haskell docspec
153+>>> deleteBy cmp x = Stream.scanMaybe (Fold.deleteBy cmp x)
154+>>> findIndices p = Stream.scanMaybe (Fold.findIndices p)
155+>>> elemIndices a = findIndices (== a)
156+```
157+
158+Direct implementations of most of these folds, scans and filters are also
159+available in the `Streamly.Internal.Data.Stream.StreamD` module. Those may in
160+fact be better fusible in some situations.
161+
162+## Stream splitting and grouping functions
163+
164+Stream splitting and grouping functions like `splitOn`, `wordsBy`, and
165+`groupsBy` have been omitted from the new stream module as these can
166+now be implemented using `foldMany` and an appropriate fold from the
167+`Streamly.Data.Fold` module or using `parseMany` and an appropriate
168+parser from the `Streamly.Data.Parser` module.
169+
170+```haskell docspec
171+>>> uniq = Stream.scanMaybe (Fold.uniqBy (==))
172+>>> splitWithSuffix predicate f = Stream.foldMany (Fold.takeEndBy predicate f)
173+>>> splitOn = Streamly.Internal.Data.Stream.StreamD.splitOn
174+>>> splitOnSuffix predicate f = Stream.foldMany (Fold.takeEndBy_ predicate f)
175+>>> indexedR = Streamly.Internal.Data.Stream.StreamD.indexedR
176+>>> groupsBy eq fld = Stream.parseMany (Parser.groupBy eq fld)
177+>>> groups = groupsBy (==)
178+>>> groupsByRolling = Streamly.Internal.Data.Stream.StreamD.groupsRollingBy
179+>>> wordsBy p f = Stream.parseMany (Parser.wordBy p f)
180+>>> chunksOf n f = Stream.foldMany (Fold.take n f)
181+```
182+
183+Direct implementation of these are also available in
184+`Streamly.Internal.Data.Stream.StreamD`.
185+
186+## Concurrency
187+
188+Earlier, concurrent and non-concurrent code used the same
189+combinators. The code was made concurrent by using different concurrent
190+stream types such as `AsyncT`, `ParallelT` etc. Now you use the same
191+stream type everywhere, you have to choose a concurrent combinator
192+for concurrent behavior. For example, use `mapM` for serial behavior
193+and `parMapM` for concurrent behavior. Concurrent combinators can
194+be imported from `Streamly.Data.Stream.Prelude` module. Concurrent
195+combinators are prefixed with `par`.
196+
197+Parallel combinators take a concurrency config argument to specify the
198+concurrency control parameters. The following combinators have the same
199+meaning as before except that they are used to set the config parameters
200+instead of being applied on the stream.
201+* `rate`
202+* `maxRate`
203+* `constRate`
204+* `avgRate`
205+* `minRate`
206+* `maxThreads`
207+* `maxBuffer`
208+
209+A stream is evaluated asynchronously using `parEval`:
210+
211+```haskell docspec
212+>>> :set -XFlexibleContexts
213+>>> mkAsync = Stream.parEval id
214+```
215+
216+Earlier `consM` was used to create an implicitly concurrent stream of
217+actions. In the new release, an equivalent effect is achieved by using
218+a serial `consM` to create a stream of actions and then explicitly using
219+`Stream.parEval` on it to evaluate it concurrently.
220+
221+```haskell docspec
222+>>> consM = StreamK.consM
223+>>> (|:) = consM
224+```
225+
226+Note that you will have to use `StreamK.toStream` to covert it to `Stream`
227+before using `parEval` on it.
228+
229+Existing generation combinators that can be implemented using new primitives:
230+
231+```haskell docspec
232+>>> repeatM = Stream.parRepeatM
233+>>> replicateM = Stream.parReplicateM
234+>>> unfoldrM step = Stream.parEval id . Stream.unfoldrM step
235+>>> iterateM step = Stream.parEval id . Stream.iterateM step
236+>>> fromIndicesM f = Stream.parEval id . fromIndicesM f
237+>>> fromListM = Stream.parSequence id . Stream.fromList
238+>>> fromFoldableM = Stream.parSequence id . StreamK.toStream . StreamK.fromFoldable
239+```
240+
241+Existing transformation combinators that can be implemented using `parEval`:
242+
243+```haskell docspec
244+>>> (|$.) f = f . Stream.parEval id
245+>>> (|&.) = flip (|$.)
246+>>> (|$) f = f . Stream.parEval id
247+>>> (|&) = flip (|$)
248+>>> sequence = Stream.parSequence
249+>>> mapM = Stream.parMapM
250+```
251+
252+`parList` is used to evaluate multiple streams concurrently and combine the
253+outputs. Existing combinators that can be implemented using `parList`:
254+
255+```haskell docspec
256+>>> async x y = Stream.parList id [x, y]
257+>>> wAsync x y = Stream.parList (Stream.interleaved True) [x, y]
258+>>> parallel x y = Stream.parList (Stream.eager True) [x, y]
259+>>> ahead x y = Stream.parList (Stream.ordered True) [x, y]
260+```
261+
262+Concurrent zipping and merging combinators:
263+
264+```haskell docspec
265+>>> zipAsyncWithM = Stream.parZipWith id
266+>>> zipAsyncWith = Stream.parZipWith id
267+>>> mergeAsyncByM = Stream.parMergeByM id
268+>>> mergeAsyncBy = Stream.parMergeBy id
269+```
270+
271+The equivalent of `concatMapWith` using a concurrent combining operation in the
272+new release is `parConcatMap`. The config argument in `parConcatMap` can
273+specify an equivalent of the combining operation. Similarly, concurrent
274+`concatFoldableWith`, `concatMapFoldableWith`, `concatForFoldableWith` can also
275+be expressed using `parConcatMap`.