git.iko.soy/streamly
on master help sign in
Fix documentation for movement of APIs to prelude

hide ltake and ltakeWhile as well.

commit25066363883ae73392544fe3b3d690dcc78965de authorHarendra Kumar <harendra.kumar@gmail.com> dateMon 5 Aug 2019, 07:19 (7 years ago) parente621237
4 files changed+222−155
compare with master
unifiedsplit
src/Streamly.hs+135−72 src/Streamly/Fold.hs+43−44 src/Streamly/Fold/Types.hs+4−4 src/Streamly/Prelude.hs+40−35
src/Streamly.hs+135−72
@@ -7,22 +7,49 @@
77-- Stability : experimental
88-- Portability : GHC
99--
10−-- Haskell lists provide a simple and powerful API but they are limited to
11−-- holding only pure values. Streamly streams are a logical extension of
12−-- Haskell lists to support monadic sequences and powerful declarative
13−-- concurrency. They provide almost the same API as lists (See
14−-- "Streamly.Prelude"). In fact, Haskell lists can be expressed as pure
15−-- streams, a special case of monadic streams, with similar or better
16−-- performance and almost drop-in replacement.
17−--
18−-- Streams are designed for high performance applications and do not exhibit
19−-- issues that are usually associated with lists. For example, streams
20−-- express strings as streams of 'Char' and bytestrings as streams of 'Word8'
21−-- without any issues, obviating the need for special purpose libraries like
22−-- bytestring and text, and various lazy and strict falvors of those. Streamly
23−-- arrays (See "Streamly.Array") complement streams for storing or buffering
24−-- data efficiently and facilitating efficient interfacing of streams with IO
25−-- systems.
10+-- Streamly is a general purpose programming framework using cocnurrent data
11+-- flow programming paradigm. It can be considered as a generalization of
12+-- Haskell lists to monadic streaming with concurrent composition capability.
13+-- The serial stream type in streamly @SerialT m a@ is like the list type @[a]@
14+-- parameterized by the monad @m@. For example, @SerialT IO a@ is a moral
15+-- equivalent of @[a]@ in the IO monad. Streams are constructed very much like
16+-- lists, except that they use 'nil' and 'cons' instead of '[]' and ':'.
17+--
18+-- @
19+-- > import "Streamly"
20+-- > import "Streamly.Prelude" (cons, consM)
21+-- > import qualified "Streamly.Prelude" as S
22+-- >
23+-- > S.toList $ 1 \`cons` 2 \`cons` 3 \`cons` nil
24+-- [1,2,3]
25+-- @
26+--
27+-- Unlike lists, streams can be constructed from monadic effects:
28+--
29+-- @
30+-- > S.'toList' $ 'getLine' \`consM` 'getLine' \`consM` S.'nil'
31+-- hello
32+-- world
33+-- ["hello","world"]
34+-- @
35+--
36+-- Streams are processed just like lists, with list like combinators, except
37+-- that they are monadic and work in a streaming fashion. Here is a simple
38+-- console echo program example:
39+--
40+-- @
41+-- > S.drain $ S.repeatM getLine & S.mapM putStrLn
42+-- @
43+--
44+-- @SerialT Identity a@ is a moral equivalent of pure lists. Streamly utilizes
45+-- fusion for high performance, therefore, we can represent and process strings
46+-- as streams of 'Char', encode and decode the streams to/from UTF8 and
47+-- serialize them to @Array Word8@ obviating the need for special purpose
48+-- libraries like @bytestring@ and @text@.
49+--
50+-- For more details
51+-- please see the "Streamly.Tutorial" module and the examples directory in this
52+-- package.
2653
2754{-# LANGUAGE CPP #-}
2855{-# LANGUAGE MultiParamTypeClasses #-}
@@ -35,12 +62,19 @@
3562
3663module Streamly
3764 (
38− -- * Streams and Folds
39− -- $overview
40−
41− -- * Streams Overview
65+ -- * Concepts Overview
66+ -- ** Streams
4267 -- $streams
4368
69+ -- ** Folds
70+ -- $folds
71+
72+ -- ** Arrays
73+ -- $arrays
74+
75+ -- * Module Overview
76+ -- $streamtypes
77+
4478 -- * Type Synonyms
4579 MonadAsync
4680
@@ -165,43 +199,52 @@ import qualified Streamly.Streams.StreamK as K
165199-- XXX provide good succinct examples of pipelining, merging, splitting ect.
166200-- below.
167201--
168−-- $overview
169−-- Streamly is a general purpose concurrent data flow programming engine
170−-- expressing a program as a network of streams and folds. Stream types (e.g.
171−-- 'SerialT') exported from this module represent streams or more specifically
172−-- stream producers while the fold type exported from "Streamly.Fold"
173−-- represents stream consumers. Both producers and consumers can be
174−-- transformed in many ways e.g. using map, scan, filter etc. Transformations
175−-- can be chained into a pipeline, multiple streams can be merged together or a
176−-- stream can be split into multiple independent data flows. All these
177−-- compositions can be combined together to form an arbitrary data flow network
178−-- expressing any kind of general purpose computing in a declarative manner.
179−--
180−-- The simplest way to compose a data processing pipeline is to generate a
181−-- stream of data (e.g. from a file or network), chain pipelines of
182−-- transformations on the stream and finally pass it to a fold. Folds represent
183−-- specific ways to consume a stream (e.g. producing a sum of numbers).
184−-- Transformations can also be applied on folds before they are attached with a
185−-- stream.
202+-- $streams
186203--
187−-- @
204+-- A program is expressed as a network of streams and folds. A stream is a
205+-- source or generator of data elements and a fold is a consumer of data elements that
206+-- reduces multiple input elements to a single value.
188207--
189−-- ---Stream m a-->stream transforms-->fold transforms-->Fold m a b
190−-- @
208+-- In the following example, a 'Word8' stream is generated by using
209+-- 'Streamly.FileSystem.Handle.read' on a file handle, then the
210+-- 'Streamly.Prelude.splitBySuffix' transformation splits the stream on
211+-- newlines (ascii value 10); it uses the 'Streamly.Fold.drain' fold to reduce
212+-- the resulting lines to unit values (@()@), 'Streamly.Prelude.length' fold
213+-- then counts the unit elements in the resulting stream which gives us the
214+-- number of lines in the file:
191215--
192−-- Stream producers can be appended, merged, zipped or nested to build an
193−-- arbitrarily complex composed stream producer. Transformations can be applied
194−-- at any point in the composition tree. See "Streamly.Prelude" module for more
195−-- details on the combinators.
216+-- > S.length $ S.splitOnSuffix FL.drain 10 $ FH.read fh
196217--
197−-- @
218+-- The following example folds the lines to arrays of 'Word8' using the
219+-- 'Streamly.Mem.Array.writeF' fold and then wraps the lines in square
220+-- brackets before writing them to standard output using
221+-- 'Streamly.FileSystem.Handle.write':
198222--
199−-- -------Stream m a-->transform-->|
200−-- |
201−-- -------Stream m a-->transform-->|=>---transform-->Stream m a--->
202−-- |
203−-- -------Stream m a-->transform-->|
204−-- @
223+-- > wrapLine ln = S.fromList "[" <> A.read ln <> S.fromList "]\n"
224+-- > readLines = S.splitOnSuffix A.writeF 10
225+-- > FH.write stdout $ S.concatMap wrapLine $ readLines fh1
226+--
227+-- One stream can be appended after another:
228+--
229+-- > FH.write stdout $ S.concatMap wrapLine $ readLines fh1 <> readLines fh2
230+--
231+-- The following example reads two files concurrently, merges the lines from
232+-- the two streams and writes the resulting stream to another file:
233+--
234+-- > FH.write stdout $ S.concatMap wrapLine $ readLines fh1 `parallel` readLines fh2
235+--
236+-- There are many ways to generate, merge, zip, transform and fold data
237+-- streams. Many transformations can be chained in a stream pipeline. See
238+-- "Streamly.Prelude" module for combinators to manipulate streams.
239+
240+-- $folds
241+--
242+-- The way stream types in this module like 'SerialT' represent data sources,
243+-- the same way the 'Fold' type from "Streamly.Fold" represents data sinks or
244+-- reducers of streams. Reducers can be combined to consume a stream source in
245+-- many ways. The simplest is to reduce a stream source using a fold e.g.:
246+--
247+-- > S.runFold FL.length $ S.enumerateTo 100
205248--
206249-- Folds are consumers of streams and can be used to split a stream into
207250-- multiple independent flows. Grouping transforms a stream by applying a fold
@@ -222,7 +265,37 @@ import qualified Streamly.Streams.StreamK as K
222265-- @
223266--
224267
225−-- $streams
268+-- $arrays
269+--
270+-- Streamly arrays (See "Streamly.Mem.Array") complement streams to provide an
271+-- efficient computing paradigm. Streams are suitable for immutable
272+-- transformations of /potentially infinite/ data using /sequential access/ and
273+-- pipelined transformations whereas arrays are suitable for in-place
274+-- transformations of /necessarily finite/ data using /random access/. Streams
275+-- are synonymous with /sequential pipelined processing/ whereas arrays are
276+-- synonymous with /efficient buffering and random access/.
277+--
278+-- In general, a data processing pipeline reads data from some IO device, does
279+-- some processing on it and finally writes the output to another IO device.
280+-- Streams provide the overall framework of sequential processing pipeline in
281+-- which arrays are used as buffering elements in the middle. In addition to
282+-- buffering in the middle, arrays can also be used at the boundaries of the
283+-- pipeline to efficiently interface with external storage systems like memory,
284+-- files and network. If streams are the pipes in a water pipeline network
285+-- then arrays are like the storage tanks in the middle. On the input side,
286+-- think of arrays as buckets to fetch water to feed the pipeline and on the
287+-- output side buckets to remove the processed water.
288+--
289+-- 'ByteString' data type from the 'bytestring' package and the 'Text' data
290+-- type from the 'text' package are special cases of arrays. 'ByteString' is
291+-- like @Array Word8@ and 'Text' is like @utf16@ encoded @Array Word8@.
292+-- Streamly arrays can be transformed as efficiently as @bytestring@ or @text@
293+-- by using stream operations on them.
294+
295+-- Streams and arrays are equally important in computing. They are computing
296+-- duals of each other.
297+
298+-- $streamtypes
226299-- The basic stream type is 'Serial', it represents a sequence of IO actions,
227300-- and is a 'Monad'. The type 'SerialT' is a monad transformer that can
228301-- represent a sequence of actions in an arbitrary monad. The type 'Serial' is
@@ -236,32 +309,22 @@ import qualified Streamly.Streams.StreamK as K
236309-- explicit stream type combinators are used, the default stream type is
237310-- inferred as 'Serial'.
238311--
239−-- Here is a simple console echo program example:
240−--
241−-- @
242−-- > drain $ S.repeatM getLine & S.mapM putStrLn
243−-- @
244−--
245−-- For more details please see the "Streamly.Tutorial" module and the examples
246−-- directory in this package.
312+-- This module exports stream types, instances and combinators for:
247313--
248−-- This module exports stream types, instances and some basic operations.
249−-- Functionality exported by this module include:
250−--
251−-- * Semigroup append ('<>') instances as well as explicit operations for merging streams
252−-- * Monad and Applicative instances for looping over streams
253−-- * Zip Applicatives for zipping streams
254−-- * Stream type combinators to convert between different composition styles
255−-- * Some basic utilities to run and fold streams
256−--
257−-- See the "Streamly.Prelude" module for comprehensive APIs for construction,
258−-- generation, elimination and transformation of streams.
314+-- * converting between different stream types
315+-- * appending and concurrently merging streams
316+-- * Concurrency control
317+-- * Concurrent function application
318+-- * Stream rate control
259319--
260320-- This module is designed to be imported unqualified:
261321--
262322-- @
263323-- import Streamly
264324-- @
325+--
326+-- See the "Streamly.Prelude" module for APIs for construction,
327+-- generation, elimination and transformation of streams.
265328
266329------------------------------------------------------------------------------
267330-- Eliminating a stream
src/Streamly/Fold.hs+43−44
@@ -15,14 +15,15 @@
1515-- Stability : experimental
1616-- Portability : GHC
1717--
18−-- A left fold consumes a stream and reduces it to a single value. Using the
19−-- fold combinators in "Streamly.Prelude" module only one fold operation can be
20−-- applied to a stream. This module provides a 'Fold' type that represents a
21−-- left fold. Multiple such folds can be combined in different ways; a stream
22−-- can then be supplied to the combined fold.
23−-- For example, a distributive applicative composition distributes the input to
18+-- A left fold consumes a stream and reduces it to a single value. The fold
19+-- operations in "Streamly.Prelude" cannot be combined such that multiple of
20+-- them can run on the same stream. This module provides a 'Fold' type that
21+-- represents a left fold. Multiple such folds can be combined using
22+-- combinators; a stream can then be supplied to the combined fold and it would
23+-- distribute the input to constituent folds according to the composition.
24+-- For example, an applicative composition distributes the same input to
2425-- the constituent folds and then combines the fold outputs. Similarly, a
25−-- partitioning composition can partition the input among constituent folds.
26+-- partitioning combinator can divide the input among constituent folds.
2627-- All the combinators in this module are of true streaming nature, stream
2728-- elements are not unnecessarily buffered in memory, guaranteeing a constant
2829-- memory consumption.
@@ -32,18 +33,9 @@
3233-- sources in interesting ways whereas this module provides combinators that
3334-- combine stream consumers in interesting ways. In other words,
3435-- "Streamly.Prelude" provides stream merging capabilities while
35−-- "Streamly.Fold" provides stream splitting capabilities. Both the modules
36−-- are organized in the same way so that you can easily find the corresponding
37−-- operations.
36+-- "Streamly.Fold" provides stream splitting capabilities.
3837--
3938-- > import qualified Streamly.Fold as FL
40−--
41−-- A left fold is represented by the type 'Fold'. @Fold m a b@ folds an
42−-- input stream consisting of values of type @a@ to a structure of type
43−-- @b@. The fold can be run using the 'foldl'' combinator and an input stream.
44−--
45−-- >>> FL.foldl' FL.sum (S.enumerateFromTo 1 100)
46−-- 5050
4739
4840-- Also see the "Streamly.Sink" module that provides specialized left folds
4941-- that discard the outputs.
@@ -53,6 +45,13 @@
5345module Streamly.Fold
5446 (
5547 -- * Fold Type
48+ -- |
49+ -- A 'Fold' can be run over a stream using the 'runFold' combinator in
50+ -- "Streamly.Prelude":
51+ --
52+ -- >>> S.runFold FL.sum (S.enumerateFromTo 1 100)
53+ -- 5050
54+
5655 Fold -- (..)
5756
5857 -- , tail
@@ -118,7 +117,7 @@ module Streamly.Fold
118117 -- Transformations can be applied either on the input side or on the output
119118 -- side. The 'Functor' instance of a fold maps on the output of the fold:
120119 --
121− -- >>> FL.foldl' (fmap show FL.sum) (S.enumerateFromTo 1 100)
120+ -- >>> S.runFold (fmap show FL.sum) (S.enumerateFromTo 1 100)
122121 -- "5050"
123122 --
124123 -- However, the input side or contravariant transformations are more
@@ -164,12 +163,12 @@ module Streamly.Fold
164163 , lreverse
165164 -}
166165
166+ {-
167167 -- * Parsing
168168 -- ** Trimming
169169 , ltake
170170 -- , lrunFor -- time
171171 , ltakeWhile
172− {-
173172 , ltakeWhileM
174173 , ldrop
175174 , ldropWhile
@@ -194,7 +193,7 @@ module Streamly.Fold
194193 -- stream twice:
195194 --
196195 -- >>> let avg = (/) <$> FL.sum <*> fmap fromIntegral FL.length
197− -- >>> FL.foldl' avg (S.enumerateFromTo 1.0 100.0)
196+ -- >>> S.runFold avg (S.enumerateFromTo 1.0 100.0)
198197 -- 50.5
199198 --
200199 -- The 'Semigroup' and 'Monoid' instances of a distributing fold distribute
@@ -202,7 +201,7 @@ module Streamly.Fold
202201 -- Semigroup instances of the output types:
203202 --
204203 -- >>> import Data.Monoid (Sum)
205− -- >>> FL.foldl' (FL.head <> FL.last) (fmap Sum $ S.enumerateFromTo 1.0 100.0)
204+ -- >>> S.runFold (FL.head <> FL.last) (fmap Sum $ S.enumerateFromTo 1.0 100.0)
206205 -- Just (Sum {getSum = 101.0})
207206 --
208207 -- The 'Num', 'Floating', and 'Fractional' instances work in the same way.
@@ -339,7 +338,7 @@ _transform (Pipe pstep1 pstep2 pinitial) (Fold fstep finitial fextract) =
339338
340339-- | @(lmap f fold)@ maps the function @f@ on the input of the fold.
341340--
342−-- >>> FL.foldl' (FL.lmap (\x -> x * x) FL.sum) (S.enumerateFromTo 1 100)
341+-- >>> S.runFold (FL.lmap (\x -> x * x) FL.sum) (S.enumerateFromTo 1 100)
343342-- 338350
344343--
345344-- @since 0.7.0
@@ -547,7 +546,7 @@ stdDev = sqrt variance
547546-- | Fold an input stream consisting of monoidal elements using 'mappend'
548547-- and 'mempty'.
549548--
550−-- > FL.foldl FL.mconcat (S.map Sum $ S.enumerateFromTo 1 10)
549+-- > S.runFold FL.mconcat (S.map Sum $ S.enumerateFromTo 1 10)
551550--
552551-- @since 0.7.0
553552{-# INLINABLE mconcat #-}
@@ -560,7 +559,7 @@ mconcat = Fold (\x a -> return $ mappend x a) (return mempty) return
560559-- Make a fold from a pure function that folds the output of the function
561560-- using 'mappend' and 'mempty'.
562561--
563−-- > FL.foldl (FL.foldMap Sum) $ S.enumerateFromTo 1 10
562+-- > S.runFold (FL.foldMap Sum) $ S.enumerateFromTo 1 10
564563--
565564-- @since 0.7.0
566565{-# INLINABLE foldMap #-}
@@ -573,7 +572,7 @@ foldMap f = lmap f mconcat
573572-- Make a fold from a monadic function that folds the output of the function
574573-- using 'mappend' and 'mempty'.
575574--
576−-- > FL.foldM (FL.foldMapM (return . Sum)) $ S.enumerateFromTo 1 10
575+-- > S.runFold (FL.foldMapM (return . Sum)) $ S.enumerateFromTo 1 10
577576--
578577-- @since 0.7.0
579578{-# INLINABLE foldMapM #-}
@@ -781,7 +780,7 @@ or = Fold (\x a -> return $ x || a) (return False) return
781780
782781-- | Include only those elements that pass a predicate.
783782--
784−-- >>> FL.foldl (lfilter (> 5) FL.sum) [1..10]
783+-- >>> S.runFold (lfilter (> 5) FL.sum) [1..10]
785784-- 40
786785--
787786-- @since 0.7.0
@@ -805,9 +804,9 @@ _lfilterM f (Fold step begin done) = Fold step' begin done
805804-- | Take first 'n' elements from the stream and discard the rest.
806805--
807806-- @since 0.7.0
808−{-# INLINABLE ltake #-}
809−ltake :: Monad m => Int -> Fold m a b -> Fold m a b
810−ltake n (Fold step initial done) = Fold step' initial' done'
807+{-# INLINABLE _ltake #-}
808+_ltake :: Monad m => Int -> Fold m a b -> Fold m a b
809+_ltake n (Fold step initial done) = Fold step' initial' done'
811810 where
812811 initial' = fmap (Tuple' 0) initial
813812 step' (Tuple' i r) a = do
@@ -821,9 +820,9 @@ ltake n (Fold step initial done) = Fold step' initial' done'
821820-- | Takes elements from the input as long as the predicate succeeds.
822821--
823822-- @since 0.7.0
824−{-# INLINABLE ltakeWhile #-}
825−ltakeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
826−ltakeWhile predicate (Fold step initial done) = Fold step' initial' done'
823+{-# INLINABLE _ltakeWhile #-}
824+_ltakeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
825+_ltakeWhile predicate (Fold step initial done) = Fold step' initial' done'
827826 where
828827 initial' = fmap Left' initial
829828 step' (Left' r) a = do
@@ -845,7 +844,7 @@ ltakeWhile predicate (Fold step initial done) = Fold step' initial' done'
845844-- ---stream m a---| |---m (b,c)
846845-- |-------Fold m a c--------|
847846-- @
848−-- >>> FL.foldl' (FL.tee FL.sum FL.length) (S.enumerateFromTo 1.0 100.0)
847+-- >>> S.runFold (FL.tee FL.sum FL.length) (S.enumerateFromTo 1.0 100.0)
849848-- (5050.0,100)
850849--
851850-- @since 0.7.0
@@ -887,7 +886,7 @@ foldCons (Fold stepL beginL doneL) (Fold stepR beginR doneR) =
887886-- ...
888887-- @
889888--
890−-- >>> FL.foldl' (FL.distribute [FL.sum, FL.length]) (S.enumerateFromTo 1 5)
889+-- >>> S.runFold (FL.distribute [FL.sum, FL.length]) (S.enumerateFromTo 1 5)
891890-- [15,5]
892891--
893892-- This is the consumer side dual of the producer side 'sequence' operation.
@@ -916,7 +915,7 @@ distribute (x:xs) = foldCons x (distribute xs)
916915--
917916-- >>> import System.Random (randomIO)
918917-- >>> randomly a = randomIO >>= \x -> return $ if x then Left a else Right a
919−-- >>> FL.foldl' (FL.partitionByM randomly FL.length FL.length) (S.enumerateFromTo 1 100)
918+-- >>> S.runFold (FL.partitionByM randomly FL.length FL.length) (S.enumerateFromTo 1 100)
920919-- (59,41)
921920--
922921-- Send input to the two folds in a proportion of 2:1:
@@ -932,7 +931,7 @@ distribute (x:xs) = foldCons x (distribute xs)
932931--
933932-- main = do
934933-- f <- proportionately 2 1
935−-- r <- FL.foldl' (FL.partitionByM f FL.length FL.length) (S.enumerateFromTo (1 :: Int) 100)
934+-- r <- S.runFold (FL.partitionByM f FL.length FL.length) (S.enumerateFromTo (1 :: Int) 100)
936935-- print r
937936-- @
938937-- @
@@ -971,7 +970,7 @@ partitionByM f (Fold stepL beginL doneL) (Fold stepR beginR doneR) =
971970-- >>> let f = FL.partitionBy (\\n -> if even n then Left n else Right n)
972971-- (fmap (("Even " ++) . show) FL.length)
973972-- (fmap (("Odd " ++) . show) FL.length)
974−-- in FL.foldl' f (S.enumerateFromTo 1 100)
973+-- in S.runFold f (S.enumerateFromTo 1 100)
975974-- ("Even 50","Odd 50")
976975-- @
977976--
@@ -1058,7 +1057,7 @@ demuxWith f kv = Fold step initial extract
10581057-- @
10591058-- > let table = Data.Map.fromList [(\"SUM", FL.sum), (\"PRODUCT", FL.product)]
10601059-- input = S.fromList [(\"SUM",1),(\"PRODUCT",2),(\"SUM",3),(\"PRODUCT",4)]
1061−-- in FL.foldl' (FL.demux table) input
1060+-- in S.runFold (FL.demux table) input
10621061-- One 1
10631062-- Two 2
10641063-- @
@@ -1116,7 +1115,7 @@ demuxWith_ f kv = Fold step initial extract
11161115-- > let prn = FL.drainBy print
11171116-- > let table = Data.Map.fromList [(\"ONE", prn), (\"TWO", prn)]
11181117-- input = S.fromList [(\"ONE",1),(\"TWO",2)]
1119−-- in FL.foldl' (FL.demux_ table) input
1118+-- in S.runFold (FL.demux_ table) input
11201119-- One 1
11211120-- Two 2
11221121-- @
@@ -1136,7 +1135,7 @@ demux_ fs = demuxWith_ fst (Map.map (lmap snd) fs)
11361135--
11371136-- @
11381137-- > let input = S.fromList [(\"ONE",1),(\"ONE",1.1),(\"TWO",2), (\"TWO",2.2)]
1139−-- in FL.foldl' (FL.classify FL.toListRev) input
1138+-- in S.runFold (FL.classify FL.toListRev) input
11401139-- fromList [(\"ONE",[1.1,1.0]),(\"TWO",[2.2,2.0])]
11411140-- @
11421141--
@@ -1166,7 +1165,7 @@ classifyWith f (Fold step initial extract) = Fold step' initial' extract'
11661165--
11671166-- @
11681167-- > let input = S.fromList [(\"ONE",1),(\"ONE",1.1),(\"TWO",2), (\"TWO",2.2)]
1169−-- in FL.foldl' (FL.classify FL.toListRev) input
1168+-- in S.runFold (FL.classify FL.toListRev) input
11701169-- fromList [(\"ONE",[1.1,1.0]),(\"TWO",[2.2,2.0])]
11711170-- @
11721171--
@@ -1238,9 +1237,9 @@ unzip = unzipWith id
12381237-- accumulator. Thus we can resume the fold later and feed it more input.
12391238--
12401239-- >> do
1241−-- > more <- FL.foldl (FL.duplicate FL.sum) (S.enumerateFromTo 1 10)
1242−-- > evenMore <- FL.foldl (FL.duplicate more) (S.enumerateFromTo 11 20)
1243−-- > FL.foldl evenMore (S.enumerateFromTo 21 30)
1240+-- > more <- S.runFold (FL.duplicate FL.sum) (S.enumerateFromTo 1 10)
1241+-- > evenMore <- S.runFold (FL.duplicate more) (S.enumerateFromTo 11 20)
1242+-- > S.runFold evenMore (S.enumerateFromTo 21 30)
12441243-- > 465
12451244--
12461245-- @since 0.7.0
src/Streamly/Fold/Types.hs+4−4
@@ -25,13 +25,14 @@ import Streamly.Strict (Tuple'(..))
2525
2626-- | Represents a left fold over an input stream of values of type @a@ to a
2727-- single value of type @b@ in 'Monad' @m@.
28+--
29+-- @since 0.7.0
2830
2931-- The fold uses an intermediate type @x@ as accumulator. The fold accumulator
3032-- is initialized by calling the @init@ function and is then driven by calling
3133-- the step function repeatedly. When the fold is done the @extract@ function
3234-- is used to map the intermediate type @x@ to the final type @b@. This allows
3335-- the state of the fold to be embedded in an arbitrary type @x@.
34−--
3536data Fold m a b =
3637 -- | @Fold @ @ step @ @ initial @ @ extract@
3738 forall x. Fold (x -> a -> m x) (m x) (x -> m b)
@@ -46,9 +47,8 @@ instance Applicative m => Functor (Fold m a) where
4647 {-# INLINE (<$) #-}
4748 (<$) b = \_ -> pure b
4849
49−-- | The input (type @a@) of the composed fold is distributed to both the
50−-- constituent folds. The outputs of the constituent folds (type @b@) are
51−-- applied to a function.
50+-- | The fold resulting from '<*>' distributes its input to both the argument
51+-- folds and combines their output using the supplied function.
5252instance Applicative m => Applicative (Fold m a) where
5353 {-# INLINE pure #-}
5454 pure b = Fold (\() _ -> pure ()) (pure ()) (\() -> pure b)
src/Streamly/Prelude.hs+40−35
@@ -170,16 +170,18 @@ module Streamly.Prelude
170170-- the previous step. However, it is possible to fold parts of the stream in
171171-- parallel and then combine the results using a monoid.
172172
173+ -- ** Right Folds
173174 , foldrM
174175 , foldrS
175176 , foldrT
176177 , foldr
177178
179+ -- ** Left Folds
178180 , foldl'
179181 , foldl1'
180182 , foldlM'
181183
182− -- * Running Folds
184+ -- ** Composable Left Folds
183185 -- $runningfolds
184186
185187 , runFold
@@ -316,6 +318,7 @@ module Streamly.Prelude
316318 -- > foldl f z xs = last $ scanl f z xs
317319 -- > foldr f z xs = head $ scanr f z xs
318320
321+ -- ** Left scans
319322 , scanl'
320323 , scanlM'
321324 , postscanl'
@@ -325,7 +328,7 @@ module Streamly.Prelude
325328 , scanl1'
326329 , scanl1M'
327330
328− -- ** Running Scans
331+ -- ** Scan Using Fold
329332 , runScan
330333 , runPostscan
331334
@@ -339,6 +342,7 @@ module Streamly.Prelude
339342 -- , lprescanl'
340343 -- , lprescanlM'
341344
345+ -- ** Indexing
342346 , indexed
343347 , indexedR
344348 -- , timestamped
@@ -460,7 +464,7 @@ module Streamly.Prelude
460464 -- , interposeBy
461465 -- , intercalate
462466
463− -- ** Breaking
467+ -- -- ** Breaking
464468
465469 -- By chunks
466470 -- , splitAt -- spanN
@@ -490,14 +494,14 @@ module Streamly.Prelude
490494
491495 -- , groupScan
492496
493− -- *** Chunks
497+ -- -- *** Chunks
494498 , chunksOf
495499 , sessionsOf
496500
497501 -- , lchunksOf
498502 -- , lsessionsOf
499503
500− -- *** Using Element Separators
504+ -- -- *** Using Element Separators
501505 -- On == Dropping the separator
502506 , splitOn
503507 , splitOnSuffix
@@ -1346,19 +1350,20 @@ foldlM' step begin m = S.foldlM' step begin $ toStreamS m
13461350
13471351-- $runningfolds
13481352--
1349−-- We can use the left folds in this module instead of the folds in
1350−-- "Streamly.Prelude". For example the following two ways of folding are
1351−-- equivalent in functionality and performance,
1353+-- "Streamly.Fold" module defines composable left folds which can be combined
1354+-- together in many interesting ways. Those folds can be run using 'runFold'.
1355+-- The following two ways of folding are equivalent in functionality and
1356+-- performance,
13521357--
1353−-- >>> FL.foldl' FL.sum (S.enumerateFromTo 1 100)
1358+-- >>> S.runFold FL.sum (S.enumerateFromTo 1 100)
13541359-- 5050
13551360-- >>> S.sum (S.enumerateFromTo 1 100)
13561361-- 5050
13571362--
1358−-- However, left folds are push type folds. That means we push the entire input
1359−-- to a fold before we can get the output. Therefore, the performance is
1363+-- However, left folds cannot terminate early even if it does not need to
1364+-- consume more input to determine the result. Therefore, the performance is
13601365-- equivalent only for full folds like 'sum' and 'length'. For partial folds
1361−-- like 'head' or 'any' the folds in "Streamly.Prelude" may be much more
1366+-- like 'head' or 'any' the the folds defined in this module may be much more
13621367-- efficient because they are implemented as right folds that terminate as soon
13631368-- as we get the result. Note that when a full fold is composed with a partial
13641369-- fold in parallel the performance is not impacted as we anyway have to
@@ -1366,13 +1371,13 @@ foldlM' step begin m = S.foldlM' step begin $ toStreamS m
13661371--
13671372-- >>> S.head (1 `S.cons` undefined)
13681373-- Just 1
1369−-- >>> FL.foldl' FL.head (1 `S.cons` undefined)
1374+-- >>> S.runFold FL.head (1 `S.cons` undefined)
13701375-- *** Exception: Prelude.undefined
13711376--
13721377-- However, we can wrap the fold in a scan to convert it into a lazy stream of
13731378-- fold steps. We can then terminate the stream whenever we want. For example,
13741379--
1375−-- >>> S.toList $ S.take 1 $ FL.scanl' FL.head (1 `S.cons` undefined)
1380+-- >>> S.toList $ S.take 1 $ S.runScan FL.head (1 `S.cons` undefined)
13761381-- [Nothing]
13771382--
13781383-- The following example extracts the input stream up to a point where the
@@ -1382,15 +1387,15 @@ foldlM' step begin m = S.foldlM' step begin $ toStreamS m
13821387-- > S.toList
13831388-- $ S.map (fromJust . fst)
13841389-- $ S.takeWhile (\\(_,x) -> x <= 10)
1385−-- $ FL.postscanl' ((,) \<$> FL.last \<*> avg) (S.enumerateFromTo 1.0 100.0)
1390+-- $ S.runPostscan ((,) \<$> FL.last \<*> avg) (S.enumerateFromTo 1.0 100.0)
13861391-- @
13871392-- @
13881393-- [1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0]
13891394-- @
13901395
1391−-- | Fold a stream using the supplied monadic fold.
1396+-- | Fold a stream using the supplied left fold.
13921397--
1393−-- >>> S.foldlStream FL.sum (S.enumerateFromTo 1 100)
1398+-- >>> S.runFold FL.sum (S.enumerateFromTo 1 100)
13941399-- 5050
13951400--
13961401-- @since 0.7.0
@@ -2181,15 +2186,15 @@ mapMaybeMSerial f m = fromStreamD $ D.mapMaybeM f $ toStreamD m
21812186-- XXX Use a compact region list to temporarily store the list, in both reverse
21822187-- as well as in reverse'.
21832188--
2189+-- /Note:/ 'reverse'' is much faster than this, use that when performance
2190+-- matters.
2191+--
21842192-- |
21852193-- > reverse = S.foldlT (flip S.cons) S.nil
21862194--
21872195-- Returns the elements of the stream in reverse order. The stream must be
21882196-- finite. Note that this necessarily buffers the entire stream in memory.
21892197--
2190−-- /Note:/ 'reverse'' is much faster than this, use that when performance
2191−-- matters.
2192−--
21932198-- /Since 0.7.0 (Monad m constraint)/
21942199--
21952200-- /Since: 0.1.1/
@@ -2567,7 +2572,7 @@ intercalate :: (IsStream t, MonadAsync m) => t m a -> t m a -> t m a
25672572-- elements of its input are consumed by fold @f1@ and the rest of the stream
25682573-- is consumed by fold @f2@.
25692574--
2570−-- > let splitAt_ n xs = FL.foldl' (FL.splitAt n FL.toList FL.toList) $ S.fromList xs
2575+-- > let splitAt_ n xs = S.runFold (FL.splitAt n FL.toList FL.toList) $ S.fromList xs
25712576--
25722577-- >>> splitAt_ 6 "Hello World!"
25732578-- > ("Hello ","World!")
@@ -2656,7 +2661,7 @@ groupScan split fold m = undefined
26562661-- | Group the input stream into groups of @n@ elements each and then fold each
26572662-- group using the provided fold function.
26582663--
2659−-- >> S.toList $ FL.chunksOf 2 FL.sum (S.enumerateFromTo 1 10)
2664+-- >> S.toList $ S.chunksOf 2 FL.sum (S.enumerateFromTo 1 10)
26602665-- > [3,7,11,15,19]
26612666--
26622667-- This can be considered as an n-fold version of 'ltake' where we apply
@@ -2736,7 +2741,7 @@ _spanBy cmp (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
27362741-- input as long as the predicate @p@ is 'True'. @f2@ consumes the rest of the
27372742-- input.
27382743--
2739−-- > let span_ p xs = FL.foldl' (FL.span p FL.toList FL.toList) $ S.fromList xs
2744+-- > let span_ p xs = S.runFold (S.span p FL.toList FL.toList) $ S.fromList xs
27402745--
27412746-- >>> span_ (< 1) [1,2,3]
27422747-- > ([],[1,2,3])
@@ -2781,7 +2786,7 @@ span p (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
27812786--
27822787-- This is the binary version of 'splitBy'.
27832788--
2784−-- > let break_ p xs = FL.foldl' (FL.break p FL.toList FL.toList) $ S.fromList xs
2789+-- > let break_ p xs = S.runFold (S.break p FL.toList FL.toList) $ S.fromList xs
27852790--
27862791-- >>> break_ (< 1) [3,2,1]
27872792-- > ([3,2,1],[])
@@ -2838,7 +2843,7 @@ _spanRollingBy cmp (Fold stepL initialL extractL) (Fold stepR initialR extractR)
28382843-- group is folded using the fold @f@ and the result of the fold is emitted in
28392844-- the output stream.
28402845--
2841−-- >>> S.toList $ FL.groupsBy (>) FL.toList $ S.fromList [1,3,7,0,2,5]
2846+-- >>> S.toList $ S.groupsBy (>) FL.toList $ S.fromList [1,3,7,0,2,5]
28422847-- > [[1,3,7],[0,2,5]]
28432848--
28442849-- @since 0.7.0
@@ -2859,7 +2864,7 @@ groupsBy cmp f m = D.fromStreamD $ D.groupsBy cmp f (D.toStreamD m)
28592864-- comparison fails a new group is started. Each group is folded using the fold
28602865-- @f@.
28612866--
2862−-- >>> S.toList $ FL.groupsByRolling (\a b -> a + 1 == b) FL.toList $ S.fromList [1,2,3,7,8,9]
2867+-- >>> S.toList $ S.groupsByRolling (\a b -> a + 1 == b) FL.toList $ S.fromList [1,2,3,7,8,9]
28632868-- > [[1,2,3],[7,8,9]]
28642869--
28652870-- @since 0.7.0
@@ -2878,7 +2883,7 @@ groupsByRolling cmp f m = D.fromStreamD $ D.groupsRollingBy cmp f (D.toStreamD
28782883--
28792884-- Groups contiguous spans of equal elements together in individual groups.
28802885--
2881−-- >>> S.toList $ FL.groups FL.toList $ S.fromList [1,1,2,2]
2886+-- >>> S.toList $ S.groups FL.toList $ S.fromList [1,1,2,2]
28822887-- > [[1,1],[2,2]]
28832888--
28842889-- @since 0.7.0
@@ -2895,7 +2900,7 @@ groups = groupsBy (==)
28952900-- stream before the sequence and the second part consisting of the sequence
28962901-- and the rest of the stream.
28972902--
2898−-- > let breakOn_ pat xs = FL.foldl' (FL.breakOn pat FL.toList FL.toList) $ S.fromList xs
2903+-- > let breakOn_ pat xs = S.runFold (S.breakOn pat FL.toList FL.toList) $ S.fromList xs
28992904--
29002905-- >>> breakOn_ "dear" "Hello dear world!"
29012906-- > ("Hello ","dear world!")
@@ -2931,7 +2936,7 @@ breakOn pat f m = undefined
29312936--
29322937-- Let's use the following definition for illustration:
29332938--
2934−-- > splitOn' p xs = S.toList $ FL.splitOn p (FL.toList) (S.fromList xs)
2939+-- > splitOn' p xs = S.toList $ S.splitOn p (FL.toList) (S.fromList xs)
29352940--
29362941-- >>> splitOn' (== '.') ""
29372942-- [""]
@@ -2975,7 +2980,7 @@ splitOn predicate f m =
29752980-- ".--." => "" "--"
29762981-- @
29772982--
2978−-- > splitOnSuffix' p xs = S.toList $ FL.splitSuffixBy p (FL.toList) (S.fromList xs)
2983+-- > splitOnSuffix' p xs = S.toList $ S.splitSuffixBy p (FL.toList) (S.fromList xs)
29792984--
29802985-- >>> splitOnSuffix' (== '.') ""
29812986-- []
@@ -3021,7 +3026,7 @@ splitOnSuffix predicate f m =
30213026-- @["a","b"]@. In other words, it treats the input like words separated by
30223027-- whitespace elements determined by the predicate.
30233028--
3024−-- > wordsBy' p xs = S.toList $ FL.wordsBy p (FL.toList) (S.fromList xs)
3029+-- > wordsBy' p xs = S.toList $ S.wordsBy p (FL.toList) (S.fromList xs)
30253030--
30263031-- >>> wordsBy' (== ',') ""
30273032-- > []
@@ -3045,7 +3050,7 @@ wordsBy predicate f m =
30453050-- | Like 'splitOnSuffix' but keeps the suffix attached to the resulting
30463051-- splits.
30473052--
3048−-- > splitBySuffix' p xs = S.toList $ FL.splitBySuffix p (FL.toList) (S.fromList xs)
3053+-- > splitBySuffix' p xs = S.toList $ S.splitBySuffix p (FL.toList) (S.fromList xs)
30493054--
30503055-- >>> splitBySuffix' (== '.') ""
30513056-- []
@@ -3119,7 +3124,7 @@ splitBySuffix predicate f m =
31193124-- For illustration, let's define a function that operates on pure lists:
31203125--
31213126-- @
3122−-- splitOnSeq' pat xs = S.toList $ FL.splitOnSeq (A.fromList pat) (FL.toList) (S.fromList xs)
3127+-- splitOnSeq' pat xs = S.toList $ S.splitOnSeq (A.fromList pat) (FL.toList) (S.fromList xs)
31233128-- @
31243129--
31253130-- >>> splitOnSeq' "" "hello"
@@ -3184,7 +3189,7 @@ splitOnAny subseq f m = undefined -- D.fromStreamD $ D.splitOnAny f subseq (D.to
31843189-- | Like 'splitSuffixBy' but the separator is a sequence of elements, instead
31853190-- of a predicate for a single element.
31863191--
3187−-- > splitSuffixOn_ pat xs = S.toList $ FL.splitSuffixOn (A.fromList pat) (FL.toList) (S.fromList xs)
3192+-- > splitSuffixOn_ pat xs = S.toList $ S.splitSuffixOn (A.fromList pat) (FL.toList) (S.fromList xs)
31883193--
31893194-- >>> splitSuffixOn_ "." ""
31903195-- [""]
@@ -3234,7 +3239,7 @@ wordsOn subseq f m = undefined -- D.fromStreamD $ D.wordsOn f subseq (D.toStream
32343239--
32353240-- | Like 'splitOnSeq' but splits the separator as well, as an infix token.
32363241--
3237−-- > splitOn'_ pat xs = S.toList $ FL.splitOn' (A.fromList pat) (FL.toList) (S.fromList xs)
3242+-- > splitOn'_ pat xs = S.toList $ S.splitOn' (A.fromList pat) (FL.toList) (S.fromList xs)
32383243--
32393244-- >>> splitOn'_ "" "hello"
32403245-- > ["h","","e","","l","","l","","o"]