on master help sign in
Update documentation of serial streams.
commit5616d0eab0a73af2cf0e8c3ccf8a973e9fc0cf47 authorHarendra Kumar <harendra@composewell.com> dateMon 29 Mar 2021, 19:11 (6 years ago) parent7ab0093
2 files changed+89−96
src/Streamly/Internal/Data/Stream/Serial.hs+57−68 src/Streamly/Prelude.hs+32−28
src/Streamly/Internal/Data/Stream/Serial.hs+57−68
@@ -9,6 +9,9 @@
99-- Stability : experimental
1010-- Portability : GHC
1111--
12+-- To run examples in this module:
13+--
14+-- >>> import qualified Streamly.Prelude as Stream
1215--
1316module Streamly.Internal.Data.Stream.Serial
1417 (
@@ -84,61 +87,41 @@ import Prelude hiding (map, mapM, errorWithoutStackTrace)
8487-- SerialT
8588------------------------------------------------------------------------------
8689
87−-- | The 'Semigroup' operation for 'SerialT' behaves like a regular append
88−-- operation. Therefore, when @a <> b@ is evaluated, stream @a@ is evaluated
89−-- first until it exhausts and then stream @b@ is evaluated. In other words,
90−-- the elements of stream @b@ are appended to the elements of stream @a@. This
91−-- operation can be used to fold an infinite lazy container of streams.
92−--
93−-- @
94−-- import Streamly
95−-- import qualified "Streamly.Prelude" as S
90+-- | The 'Semigroup' operation @a <> b@ (same as 'Streamly.Prelude.serial')
91+-- appends stream @b@ to stream @a@:
9692--
97−-- main = (S.toList . 'serially' $ (S.fromList [1,2]) \<\> (S.fromList [3,4])) >>= print
98−-- @
99−-- @
93+-- >>> stream1 = Stream.fromList [1,2]
94+-- >>> stream2 = Stream.fromList [3,4]
95+-- >>> Stream.toList $ stream1 <> stream2
10096-- [1,2,3,4]
101−-- @
10297--
103−-- The 'Monad' instance runs the /monadic continuation/ for each
104−-- element of the stream, serially.
98+-- This operation can be used to fold an infinite lazy container of streams.
99+-- Since the ordering of elements is important in serial streams, this
100+-- operation is not commutative i.e. a <> b and b <> a are not the same.
105101--
106−-- @
107−-- main = S.drain . 'serially' $ do
108−-- x <- return 1 \<\> return 2
109−-- S.yieldM $ print x
110−-- @
111−-- @
102+-- Monad bind is same as @flip . concatMapWith serial@. A single 'Monad' bind
103+-- behaves like a @for@ loop:
104+--
105+-- >>> :{
106+-- Stream.drain $ do
107+-- x <- Stream.fromList [1,2] -- foreach x in stream
108+-- Stream.yieldM $ print x
109+-- :}
112110-- 1
113111-- 2
114−-- @
115112--
116−-- 'SerialT' nests streams serially in a depth first manner.
113+-- Nested monad binds behave like nested @for@ loops:
117114--
118−-- @
119−-- main = S.drain . 'serially' $ do
120−-- x <- return 1 \<\> return 2
121−-- y <- return 3 \<\> return 4
122−-- S.yieldM $ print (x, y)
123−-- @
124−-- @
115+-- >>> :{
116+-- Stream.drain $ do
117+-- x <- Stream.fromList [1,2] -- foreach x in stream
118+-- y <- Stream.fromList [3,4] -- foreach y in stream
119+-- Stream.yieldM $ print (x, y)
120+-- :}
125121-- (1,3)
126122-- (1,4)
127123-- (2,3)
128124-- (2,4)
129−-- @
130−--
131−-- We call the monadic code being run for each element of the stream a monadic
132−-- continuation. In imperative paradigm we can think of this composition as
133−-- nested @for@ loops and the monadic continuation is the body of the loop. The
134−-- loop iterates for all elements of the stream.
135−--
136−-- Note that the behavior and semantics of 'SerialT', including 'Semigroup'
137−-- and 'Monad' instances are exactly like Haskell lists except that 'SerialT'
138−-- can contain effectful actions while lists are pure.
139−--
140−-- In the code above, the 'serially' combinator can be omitted as the default
141−-- stream type is 'SerialT'.
142125--
143126-- /Since: 0.2.0 ("Streamly")/
144127--
@@ -275,12 +258,13 @@ TRAVERSABLE_INSTANCE(SerialT)
275258-- WSerialT
276259------------------------------------------------------------------------------
277260
278−-- | The 'Semigroup' operation for 'WSerialT' interleaves the elements from the
279−-- two streams. Therefore, when @a <> b@ is evaluated, stream @a@ is evaluated
280−-- first to produce the first element of the combined stream and then stream
281−-- @b@ is evaluated to produce the next element of the combined stream, and
282−-- then we go back to evaluating stream @a@ and so on. In other words, the
283−-- elements of stream @a@ are interleaved with the elements of stream @b@.
261+-- | The 'Semigroup' operation @a <> b@ (same as 'Streamly.Prelude.wSerial')
262+-- interleaves stream @a@ and stream @b@:
263+--
264+-- >>> stream1 = Stream.fromList [1,2]
265+-- >>> stream2 = Stream.fromList [3,4]
266+-- >>> Stream.toList $ Stream.wSerially $ stream1 <> stream2
267+-- [1,3,2,4]
284268--
285269-- Note that evaluation of @a <> b <> c@ does not schedule @a@, @b@ and @c@
286270-- with equal priority. This expression is equivalent to @a <> (b <> c)@,
@@ -290,6 +274,9 @@ TRAVERSABLE_INSTANCE(SerialT)
290274-- leftmost stream gets the same scheduling priority as the rest of the
291275-- streams taken together. The same is true for each subexpression on the right.
292276--
277+-- This also means that this operation is associative only if we disregard the
278+-- ordering of elements in the resulting stream.
279+--
293280-- Note that this operation cannot be used to fold a container of infinite
294281-- streams as the state that it needs to maintain is proportional to the number
295282-- of streams.
@@ -297,32 +284,34 @@ TRAVERSABLE_INSTANCE(SerialT)
297284-- The @W@ in the name stands for @wide@ or breadth wise scheduling in
298285-- contrast to the depth wise scheduling behavior of 'SerialT'.
299286--
300−-- @
301−-- import Streamly
302−-- import qualified "Streamly.Prelude" as S
303−--
304−-- main = (S.toList . 'wSerially' $ (S.fromList [1,2]) \<\> (S.fromList [3,4])) >>= print
305−-- @
306−-- @
307−-- [1,3,2,4]
308−-- @
287+-- Monad bind is same as @flip . concatMapWith wSerial@. A single 'Monad' bind
288+-- behaves like a @for@ loop:
309289--
310−-- Similarly, the 'Monad' instance interleaves the iterations of the
311−-- inner and the outer loop, nesting loops in a breadth first manner.
290+-- >>> :{
291+-- Stream.drain $ Stream.wSerially $ do
292+-- x <- Stream.fromList [1,2] -- foreach x in stream
293+-- Stream.yieldM $ print x
294+-- :}
295+-- 1
296+-- 2
312297--
298+-- Nested monad binds behave like interleaved nested @for@ loops:
313299--
314−-- @
315−-- main = S.drain . 'wSerially' $ do
316−-- x <- return 1 \<\> return 2
317−-- y <- return 3 \<\> return 4
318−-- S.yieldM $ print (x, y)
319−-- @
320−-- @
300+-- >>> :{
301+-- Stream.drain $ Stream.wSerially $ do
302+-- x <- Stream.fromList [1,2] -- foreach x in stream
303+-- y <- Stream.fromList [3,4] -- foreach y in stream
304+-- Stream.yieldM $ print (x, y)
305+-- :}
321306-- (1,3)
322307-- (2,3)
323308-- (1,4)
324309-- (2,4)
325−-- @
310+--
311+-- It is a result of interleaving all the nested iterations of @1@ and all the
312+-- nested iterations of @2@:
313+--
314+-- > Stream.fromList [(1,3),(1,4)] `wSerial` Stream.fromList [(2,3),(2,4)]
326315--
327316-- /Since: 0.2.0 ("Streamly")/
328317--
src/Streamly/Prelude.hs+32−28
@@ -114,16 +114,24 @@
114114--
115115-- Like 'consM', there are several other stream generation operations whose
116116-- execution behavior depends on the stream type, they all follow behavior
117−-- similar to 'consM'. For example, 'replicateM' in the following example
118−-- executes the replicated actions concurrently, just like 'consM':
117+-- similar to 'consM'.
118+--
119+-- By default, folds like 'drain' force the stream type to be 'SerialT', so
120+-- 'replicateM' in the following code runs serially, and takes 10 seconds:
121+--
122+-- >>> Stream.drain $ Stream.replicateM 10 $ delay 1
123+-- ...
124+--
125+-- We can use the 'asyncly' combinator to force the argument stream to be of
126+-- 'AsyncT' type, 'replicateM' in the following example executes the replicated
127+-- actions concurrently, thus taking only 1 second:
119128--
120129-- >>> Stream.drain $ Stream.asyncly $ Stream.replicateM 10 $ delay 1
121130-- ...
122131--
123−-- Above, the 'asyncly' combinator forces the argument stream to be of type
124−-- 'AsyncT'. By default, there is a limit on how many concurrent actions may
125−-- be executed at a time, the "Concurrency Control" section describes
126−-- combinators that can be used to control the concurrency.
132+-- By default, there is a limit on how many concurrent actions may be executed
133+-- at a time, the "Concurrency Control" section describes combinators that can
134+-- be used to control the concurrency.
127135--
128136-- We can use 'mapM' to map an action concurrently:
129137--
@@ -156,7 +164,7 @@
156164--
157165-- >>> stream1 = Stream.fromListM [delay 1, delay 2]
158166-- >>> stream2 = Stream.fromListM [delay 3, delay 4]
159−-- >>> Stream.drain $ Stream.serially $ stream1 <> stream2
167+-- >>> Stream.drain $ stream1 <> stream2
160168-- 1 sec
161169-- 2 sec
162170-- 3 sec
@@ -188,7 +196,7 @@
188196--
189197-- >>> stream1 = Stream.fromListM [delay 1, delay 2]
190198-- >>> stream2 = Stream.fromListM [delay 3, delay 4]
191−-- >>> Stream.drain $ Stream.asyncly $ stream1 `parallel` stream2
199+-- >>> Stream.drain $ stream1 `parallel` stream2
192200-- 1 sec
193201-- 2 sec
194202-- 3 sec
@@ -217,6 +225,18 @@
217225-- behavior of the stream as well as the behavior of 'Semigroup' and 'Monad'
218226-- instances.
219227--
228+-- == Caveats
229+--
230+-- When we use combinators like 'asyncly' on a piece of code, all combinators
231+-- inside the argument of asyncly become concurrent which is often counter
232+-- productive. Therefore, we recommend that in a pipeline, you identify the
233+-- combinators that you really want to be concurrent and add a 'serially' after
234+-- those combinators so that the code following the combinator remains serial:
235+--
236+-- @
237+-- Stream.asyncly $ ... concurrent combinator here ... $ Stream.serially $ ...
238+-- @
239+--
220240-- == Polymorphic Combinators
221241--
222242-- The combinators in this module are polymorphic in stream type. The stream is
@@ -869,27 +889,11 @@ import Streamly.Internal.Data.Stream.IsStream
869889
870890-- $serial
871891--
872−-- When a stream consumer demands an element from a serial stream constructed
873−-- as @a \`consM` b \`consM` ... nil@, the action @a@ at the head of the stream
874−-- sequence is executed and the result is supplied to the consumer. When the
875−-- next element is demanded, the action @b@ is executed and its result is
876−-- supplied. Thus, the effects are performed and results are consumed strictly
877−-- in a serial order. Serial streams can be considered as /spatially ordered/
878−-- streams as the order of execution and consumption is the same as the spatial
879−-- order in which the actions are composed by the programmer.
880−--
881−-- Serial streams enforce the side effects as well as the results of the
882−-- actions to be in the same order in which the actions are added to the
883−-- stream. Therefore, the semigroup operation for serial streams is not
884−-- commutative:
885−--
886−-- @
887−-- a <> b is not the same as b <> a
888−-- @
892+-- Serial streams are /spatially ordered/, they execute the actions in the
893+-- stream strictly one after the other.
889894--
890−-- There are two serial stream types 'SerialT' and 'WSerialT'. The stream
891−-- evaluation of both the variants works in the same way as described above,
892−-- they differ only in the 'Semigroup' and 'Monad' implementaitons.
895+-- There are two serial stream types 'SerialT' and 'WSerialT'. They differ
896+-- only in the 'Semigroup' and 'Monad' instance behavior.
893897
894898-- $ahead
895899--