git.iko.soy/streamly
on e4934eb help sign in
streamly/docs/User/Project/Upgrading.md
306 lines 11 kB history raw
1# Upgrading to streamly 0.9.0
3Also see:
5* [0.9.0 API Changelog](/docs/User/ProjectRelated/ApiChangelogs/0.8.3-0.9.0.txt) or
6 https://hackage.haskell.org/package/streamly-0.9.0/docs/docs/User/ProjectRelated/ApiChangelogs/0.8.3-0.9.0.txt
8`Streamly.Prelude` module has been deprecated, equivalent
9functionality is covered by the `Streamly.Data.Stream`,
10`Streamly.Data.Stream.Prelude`, and `Streamly.Data.Fold` modules. The
11new modules use a monomorphic `Stream` type instead of the polymorphic
12`IsStream t` type.
14`Streamly.Data.Stream` module and the `Stream` type are designed
15for writing high-performance fused pipelines not involving explicit
16recursion. For writing code that may require recursive function calls,
17`Streamly.Data.Stream.StreamK` module and the `StreamK` type have been
18introduced which provide a CPS based stream implementation. `Stream` and
19`StreamK` types can be easily interconverted. These changes have been made to
20make performance robust and not rely on GHC rewrite rules which could be
21fragile. For example, GHC 9.0.x had broken the rewrite rule which was
22not fixed until GHC-9.2.2. This split also gives more power and
23transparency of the performance behavior to the programmer.
25Instead of using separate stream types for concurrent code, now
26you have to use explicit concurrent combinators with appropriate
27concurrency parameters. These combinators are available in the
28`Streamly.Data.Stream.Prelude` module. This change has been made to allow
29programmers to control concurrent behavior in a more robust way and
30reduce pitfalls.
32The old code can be adapted to use the new modules with some
33changes. More details about this is supplied in the following sections.
34Please open an issue if you cannot find a way to adapt the code to the new
35release.
37Assume the following imports in the code snippets below:
39```haskell docspec
40>>> import qualified Streamly.Data.Stream as Stream
41>>> import qualified Streamly.Data.StreamK as StreamK
42>>> import qualified Streamly.Data.Stream.Prelude as Stream
43>>> import qualified Streamly.Data.Fold as Fold
44>>> import qualified Streamly.Data.Parser as Parser
45```
47## Dependencies in cabal file
49If you allow streamly < 0.9 in version bounds then you may not be able
50to build if you:
52* also depend on the new `streamly-core` package (module name conflicts)
53* depend on `streamly-bytestring` >= 0.2 (type conflict)
55You can use a build flag to allow older streamly versions:
57```
58flag old-streamly
59 description: Allow streamly versions lower than 0.9
60 manual: True
61 default: False
62```
64```
65 if flag(old-streamly)
66 build-depends:
67 streamly >=0.8.3 && < 0.9
68 else
69 build-depends:
70 streamly-core >= 0.1.0 && < 0.2,
71 streamly >=0.9.0 && < 0.10
72```
74## The `Stream` and `StreamK` types
76The following types are removed:
78* `IsStream`
79* `ZipSerialM`
80* `ZipAsyncM`
81* `WSerialT`
82* `WAsyncT`
83* `SerialT`
84* `ParallelT`
85* `AsyncT`
86* `AheadT`
88In the new release, the `Stream` type is the primary stream type that
89you will use most of the time. You can think of it as a replacement for
90the `SerialT` type. However, it does not provide an Applicative or Monad
91instance.
93The `CrossStream` type in `Streamly.Internal.Data.Stream` is a
94wrapper over `Stream` type supplying the Monad instance. However, see
95the "Stream Fusion" section in the `Streamly.Data.Stream` module for
96limitations of the `Stream` type. The `StreamK` type and `CrossStreamK`
97(in Streamly.Internal.Data.Stream.StreamK) could be used to overcome the
98limitations of `Stream` type.
100If required, you can use the template-haskell functions in
101`Streamly.Data.Stream.MkType` to create stream type wrappers (like
102ZipSerialM or WSerialT) with custom Applicative or Monadic properties.
103But in general, try to avoid specific types and use explicit functions
104from the stream module.
106`adapt` is not needed anymore.
108```haskell docspec
109>>> (.:) = StreamK.cons
110>>> cons = StreamK.cons
111>>> wSerial = StreamK.interleave
112>>> serial = StreamK.append
113>>> fromIndicesM f = Stream.mapM f $ Stream.enumerateFrom 0
114>>> fromIndices f = fmap f $ Stream.enumerateFrom 0
115>>> fromListM = Stream.sequence . Stream.fromList
116>>> fromFoldable = StreamK.toStream . StreamK.fromFoldable
117>>> fromFoldableM = Stream.sequence . fromFoldable
118```
120## Stream folding functions
122Explicit stream fold functions have been omitted from the new stream
123module. You can use the following equivalent definitions:
125```haskell docspec
126>>> the = Stream.fold Fold.the
127>>> sum = Stream.fold Fold.sum
128>>> product = Stream.fold Fold.product
129>>> or = Stream.fold Fold.or
130>>> null = Stream.fold Fold.null
131>>> elemIndex a = Stream.fold (Fold.elemIndex a)
132>>> elem a = Stream.fold (Fold.elem a)
133>>> notElem a = Stream.fold (Fold.notElem a)
134>>> minimumBy ordering = Stream.fold (Fold.minimumBy ordering)
135>>> minimum = Stream.fold Fold.minimum
136>>> maximumBy ordering = Stream.fold (Fold.maximumBy ordering)
137>>> maximum = Stream.fold Fold.maximum
138>>> mapM_ f = Stream.fold (Fold.drainMapM f)
139>>> lookup a = Stream.fold (Fold.lookup a)
140>>> length = Stream.fold Fold.length
141>>> last = Stream.fold Fold.latest
142>>> head = Stream.fold Fold.one
143>>> foldlM' f a = Stream.fold (Fold.foldlM' f a)
144>>> foldl1 f = Stream.fold (Fold.foldl1' f)
145>>> foldl' f a = Stream.fold (Fold.foldl' f a)
146>>> findIndex eq = Stream.fold (Fold.findIndex eq)
147>>> find eq = Stream.fold (Fold.find eq)
148>>> findM eq = Stream.fold (Fold.findM eq)
149>>> drainWhile p = Stream.fold Fold.drain . Stream.takeWhile p
150>>> drainN i = Stream.fold Fold.drain . Stream.take i
151>>> drain = Stream.fold Fold.drain
152>>> any p = Stream.fold (Fold.any p)
153>>> and = Stream.fold Fold.and
154>>> all p = Stream.fold (Fold.all p)
155>>> (!!) i = Stream.fold (Fold.index i)
156>>> tail = Streamly.Internal.Data.StreamK.tail
157>>> init = Streamly.Internal.Data.StreamK.init
158```
160Mapping functions:
162```haskell docspec
163>>> map = fmap
164```
166Similarly for scanning use `Stream.scan` or `Stream.postscan`
167with an appropriate fold.
169```haskell docspec
170>>> scanl' f z = Stream.scan (Fold.foldl' f z)
171>>> scanlM' f z = Stream.scan (Fold.foldlM' f z)
172>>> postscanl' f z = Stream.postscan (Fold.foldl' f z)
173>>> postscanlM' f z = Stream.postscan (Fold.foldlM' f z)
174>>> scanl1' f = Stream.catMaybes . Stream.scan (Fold.foldl1' f)
175>>> scanl1M' f = Stream.catMaybes . Stream.scan (Fold.foldlM1' f)
176>>> concatMapWith = StreamK.concatMapWith
177>>> concatFoldableWith f = Prelude.foldr f StreamK.nil
178>>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
179>>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
180```
182Filters:
184```haskell docspec
185>>> deleteBy cmp x = Stream.scanMaybe (Fold.deleteBy cmp x)
186>>> findIndices p = Stream.scanMaybe (Fold.findIndices p)
187>>> elemIndices a = findIndices (== a)
188```
190Custom implementations of most of these folds, scans and filters are also
191available in the `Streamly.Internal.Data.Stream` module.
193## Stream splitting and grouping functions
195Stream splitting and grouping functions like `splitOn`, `wordsBy`, and
196`groupsBy` have been omitted from the new stream module as these can
197now be implemented using `foldMany` and an appropriate fold from the
198`Streamly.Data.Fold` module, or using `parseMany` and an appropriate
199parser from the `Streamly.Data.Parser` module.
201```haskell docspec
202>>> uniq = Stream.scanMaybe (Fold.uniqBy (==))
203>>> splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
204>>> splitOn = Streamly.Internal.Data.Stream.splitOn
205>>> splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
206>>> indexedR = Streamly.Internal.Data.Stream.indexedR
207>>> groupsBy eq fld = Stream.parseMany (Parser.groupBy eq fld)
208>>> groups = groupsBy (==)
209>>> groupsByRolling = Streamly.Internal.Data.Stream.groupsRollingBy
210>>> wordsBy p f = Stream.parseMany (Parser.wordBy p f)
211>>> chunksOf n f = Stream.foldMany (Fold.take n f)
212```
214Direct implementation of these are also available in
215`Streamly.Internal.Data.Stream`.
217## Concurrency
219Earlier, concurrent and non-concurrent code used the same
220combinators. The code was made concurrent by using different concurrent
221stream types such as `AsyncT`, `ParallelT` etc. Now you use the same
222stream type everywhere, you have to choose a concurrent combinator
223for concurrent behavior. For example, use `mapM` for serial behavior
224and `parMapM` for concurrent behavior. Concurrent combinators can
225be imported from `Streamly.Data.Stream.Prelude` module. Concurrent
226combinators are prefixed with `par`.
228Parallel combinators take a concurrency config argument to specify the
229concurrency control parameters. The following combinators have the same
230meaning as before except that they are used to set the config parameters
231instead of being applied on the stream.
232* `rate`
233* `maxRate`
234* `constRate`
235* `avgRate`
236* `minRate`
237* `maxThreads`
238* `maxBuffer`
240A stream is evaluated asynchronously using `parEval`:
242```haskell docspec
243>>> :set -XFlexibleContexts
244>>> mkAsync = Stream.parEval id
245```
247Earlier `consM` was used to create an implicitly concurrent stream of
248actions. In the new release, an equivalent effect is achieved by using
249a serial `consM` to create a stream of actions and then explicitly using
250`Stream.parEval` on it to evaluate it concurrently.
252```haskell docspec
253>>> consM = StreamK.consM
254>>> (|:) = consM
255```
257Note that you will have to use `StreamK.toStream` to covert it to `Stream`
258before using `parEval` on it.
260Existing generation combinators that can be implemented using new primitives:
262```haskell docspec
263>>> repeatM = Stream.parRepeatM
264>>> replicateM = Stream.parReplicateM
265>>> unfoldrM step = Stream.parEval id . Stream.unfoldrM step
266>>> iterateM step = Stream.parEval id . Stream.iterateM step
267>>> fromIndicesM f = Stream.parEval id . fromIndicesM f
268>>> fromListM = Stream.parSequence id . Stream.fromList
269>>> fromFoldableM = Stream.parSequence id . StreamK.toStream . StreamK.fromFoldable
270```
272Existing transformation combinators that can be implemented using `parEval`:
274```haskell docspec
275>>> (|$.) f = f . Stream.parEval id
276>>> (|&.) = flip (|$.)
277>>> (|$) f = f . Stream.parEval id
278>>> (|&) = flip (|$)
279>>> sequence = Stream.parSequence
280>>> mapM = Stream.parMapM
281```
283`parList` is used to evaluate multiple streams concurrently and combine the
284outputs. Existing combinators that can be implemented using `parList`:
286```haskell docspec
287>>> async x y = Stream.parList id [x, y]
288>>> wAsync x y = Stream.parList (Stream.interleaved True) [x, y]
289>>> parallel x y = Stream.parList (Stream.eager True) [x, y]
290>>> ahead x y = Stream.parList (Stream.ordered True) [x, y]
291```
293Concurrent zipping and merging combinators:
295```haskell docspec
296>>> zipAsyncWithM = Stream.parZipWith id
297>>> zipAsyncWith = Stream.parZipWith id
298>>> mergeAsyncByM = Stream.parMergeByM id
299>>> mergeAsyncBy = Stream.parMergeBy id
300```
302The equivalent of `concatMapWith` using a concurrent combining operation in the
303new release is `parConcatMap`. The config argument in `parConcatMap` can
304specify an equivalent of the combining operation. Similarly, concurrent
305`concatFoldableWith`, `concatMapFoldableWith`, `concatForFoldableWith` can also
306be expressed using `parConcatMap`.