5616d0e · streamly · git.iko.soy

git.iko.soy/streamly

on master help sign in

Update documentation of serial streams.

5616d0eHarendra Kumar6 years ago

src/Streamly/Internal/Data/Stream/Serial.hs+57−68

compare with master

unifiedsplit

@@ -9,6 +9,9 @@

9-- Stability : experimental

10-- Portability : GHC

11--

12+-- To run examples in this module:

13+--

14+-- >>> import qualified Streamly.Prelude as Stream

12--

15--

13module Streamly.Internal.Data.Stream.Serial

16module Streamly.Internal.Data.Stream.Serial

14 (

17 (

@@ -84,61 +87,41 @@ import Prelude hiding (map, mapM, errorWithoutStackTrace)

84-- SerialT

87-- SerialT

85------------------------------------------------------------------------------

88------------------------------------------------------------------------------

87−-- | The 'Semigroup' operation for 'SerialT' behaves like a regular append

90+-- | The 'Semigroup' operation @a <> b@ (same as 'Streamly.Prelude.serial')

88−-- operation. Therefore, when @a <> b@ is evaluated, stream @a@ is evaluated

91+-- appends stream @b@ to stream @a@:

89−-- first until it exhausts and then stream @b@ is evaluated. In other words,

90−-- the elements of stream @b@ are appended to the elements of stream @a@. This

91−-- operation can be used to fold an infinite lazy container of streams.

92−--

93−-- @

94−-- import Streamly

95−-- import qualified "Streamly.Prelude" as S

96--

92--

97−-- main = (S.toList . 'serially' $ (S.fromList [1,2]) \<\> (S.fromList [3,4])) >>= print

93+-- >>> stream1 = Stream.fromList [1,2]

98−-- @

94+-- >>> stream2 = Stream.fromList [3,4]

99−-- @

95+-- >>> Stream.toList $ stream1 <> stream2

100-- [1,2,3,4]

96-- [1,2,3,4]

101−-- @

102--

97--

103−-- The 'Monad' instance runs the /monadic continuation/ for each

98+-- This operation can be used to fold an infinite lazy container of streams.

104−-- element of the stream, serially.

99+-- Since the ordering of elements is important in serial streams, this

100+-- operation is not commutative i.e. a <> b and b <> a are not the same.

105--

101--

106−-- @

102+-- Monad bind is same as @flip . concatMapWith serial@. A single 'Monad' bind

103+-- behaves like a @for@ loop:

104+--

105+-- >>> :{

107−-- main = S.drain . 'serially' $ do

106+-- Stream.drain $ do

108−-- x <- return 1 \<\> return 2

107+-- x <- Stream.fromList [1,2] -- foreach x in stream

109−-- S.yieldM $ print x

108+-- Stream.yieldM $ print x

110−-- @

109+-- :}

111−-- @

112-- 1

110-- 1

113-- 2

111-- 2

114−-- @

115--

112--

116−-- 'SerialT' nests streams serially in a depth first manner.

113+-- Nested monad binds behave like nested @for@ loops:

117--

114--

118−-- @

115+-- >>> :{

119−-- main = S.drain . 'serially' $ do

116+-- Stream.drain $ do

120−-- x <- return 1 \<\> return 2

117+-- x <- Stream.fromList [1,2] -- foreach x in stream

121−-- y <- return 3 \<\> return 4

118+-- y <- Stream.fromList [3,4] -- foreach y in stream

122−-- S.yieldM $ print (x, y)

119+-- Stream.yieldM $ print (x, y)

123−-- @

120+-- :}

124−-- @

125-- (1,3)

121-- (1,3)

126-- (1,4)

122-- (1,4)

127-- (2,3)

123-- (2,3)

128-- (2,4)

124-- (2,4)

129−-- @

130−--

131−-- We call the monadic code being run for each element of the stream a monadic

132−-- continuation. In imperative paradigm we can think of this composition as

133−-- nested @for@ loops and the monadic continuation is the body of the loop. The

134−-- loop iterates for all elements of the stream.

135−--

136−-- Note that the behavior and semantics of 'SerialT', including 'Semigroup'

137−-- and 'Monad' instances are exactly like Haskell lists except that 'SerialT'

138−-- can contain effectful actions while lists are pure.

139−--

140−-- In the code above, the 'serially' combinator can be omitted as the default

141−-- stream type is 'SerialT'.

142--

125--

143-- /Since: 0.2.0 ("Streamly")/

126-- /Since: 0.2.0 ("Streamly")/

144--

127--

@@ -275,12 +258,13 @@ TRAVERSABLE_INSTANCE(SerialT)

275-- WSerialT

258-- WSerialT

276------------------------------------------------------------------------------

259------------------------------------------------------------------------------

277

260

278−-- | The 'Semigroup' operation for 'WSerialT' interleaves the elements from the

261+-- | The 'Semigroup' operation @a <> b@ (same as 'Streamly.Prelude.wSerial')

279−-- two streams. Therefore, when @a <> b@ is evaluated, stream @a@ is evaluated

262+-- interleaves stream @a@ and stream @b@:

280−-- first to produce the first element of the combined stream and then stream

263+--

281−-- @b@ is evaluated to produce the next element of the combined stream, and

264+-- >>> stream1 = Stream.fromList [1,2]

282−-- then we go back to evaluating stream @a@ and so on. In other words, the

265+-- >>> stream2 = Stream.fromList [3,4]

283−-- elements of stream @a@ are interleaved with the elements of stream @b@.

266+-- >>> Stream.toList $ Stream.wSerially $ stream1 <> stream2

267+-- [1,3,2,4]

284--

268--

285-- Note that evaluation of @a <> b <> c@ does not schedule @a@, @b@ and @c@

269-- Note that evaluation of @a <> b <> c@ does not schedule @a@, @b@ and @c@

286-- with equal priority. This expression is equivalent to @a <> (b <> c)@,

270-- with equal priority. This expression is equivalent to @a <> (b <> c)@,

@@ -290,6 +274,9 @@ TRAVERSABLE_INSTANCE(SerialT)

290-- leftmost stream gets the same scheduling priority as the rest of the

274-- leftmost stream gets the same scheduling priority as the rest of the

291-- streams taken together. The same is true for each subexpression on the right.

275-- streams taken together. The same is true for each subexpression on the right.

292--

276--

277+-- This also means that this operation is associative only if we disregard the

278+-- ordering of elements in the resulting stream.

279+--

293-- Note that this operation cannot be used to fold a container of infinite

280-- Note that this operation cannot be used to fold a container of infinite

294-- streams as the state that it needs to maintain is proportional to the number

281-- streams as the state that it needs to maintain is proportional to the number

295-- of streams.

282-- of streams.

@@ -297,32 +284,34 @@ TRAVERSABLE_INSTANCE(SerialT)

297-- The @W@ in the name stands for @wide@ or breadth wise scheduling in

284-- The @W@ in the name stands for @wide@ or breadth wise scheduling in

298-- contrast to the depth wise scheduling behavior of 'SerialT'.

285-- contrast to the depth wise scheduling behavior of 'SerialT'.

299--

286--

300−-- @

287+-- Monad bind is same as @flip . concatMapWith wSerial@. A single 'Monad' bind

301−-- import Streamly

288+-- behaves like a @for@ loop:

302−-- import qualified "Streamly.Prelude" as S

303−--

304−-- main = (S.toList . 'wSerially' $ (S.fromList [1,2]) \<\> (S.fromList [3,4])) >>= print

305−-- @

306−-- @

307−-- [1,3,2,4]

308−-- @

309--

289--

310−-- Similarly, the 'Monad' instance interleaves the iterations of the

290+-- >>> :{

311−-- inner and the outer loop, nesting loops in a breadth first manner.

291+-- Stream.drain $ Stream.wSerially $ do

292+-- x <- Stream.fromList [1,2] -- foreach x in stream

293+-- Stream.yieldM $ print x

294+-- :}

295+-- 1

296+-- 2

312--

297--

298+-- Nested monad binds behave like interleaved nested @for@ loops:

313--

299--

314−-- @

300+-- >>> :{

315−-- main = S.drain . 'wSerially' $ do

301+-- Stream.drain $ Stream.wSerially $ do

316−-- x <- return 1 \<\> return 2

302+-- x <- Stream.fromList [1,2] -- foreach x in stream

317−-- y <- return 3 \<\> return 4

303+-- y <- Stream.fromList [3,4] -- foreach y in stream

318−-- S.yieldM $ print (x, y)

304+-- Stream.yieldM $ print (x, y)

319−-- @

305+-- :}

320−-- @

321-- (1,3)

306-- (1,3)

322-- (2,3)

307-- (2,3)

323-- (1,4)

308-- (1,4)

324-- (2,4)

309-- (2,4)

325−-- @

310+--

311+-- It is a result of interleaving all the nested iterations of @1@ and all the

312+-- nested iterations of @2@:

313+--

314+-- > Stream.fromList [(1,3),(1,4)] `wSerial` Stream.fromList [(2,3),(2,4)]

326--

315--

327-- /Since: 0.2.0 ("Streamly")/

316-- /Since: 0.2.0 ("Streamly")/

328--

317--

src/Streamly/Prelude.hs+32−28

@@ -114,16 +114,24 @@

114--

115-- Like 'consM', there are several other stream generation operations whose

116-- execution behavior depends on the stream type, they all follow behavior

117+-- similar to 'consM'.

118+--

119+-- By default, folds like 'drain' force the stream type to be 'SerialT', so

117−-- similar to 'consM'. For example, 'replicateM' in the following example

120+-- 'replicateM' in the following code runs serially, and takes 10 seconds:

121+--

122+-- >>> Stream.drain $ Stream.replicateM 10 $ delay 1

123+-- ...

124+--

125+-- We can use the 'asyncly' combinator to force the argument stream to be of

126+-- 'AsyncT' type, 'replicateM' in the following example executes the replicated

118−-- executes the replicated actions concurrently, just like 'consM':

127+-- actions concurrently, thus taking only 1 second:

119--

128--

120-- >>> Stream.drain $ Stream.asyncly $ Stream.replicateM 10 $ delay 1

129-- >>> Stream.drain $ Stream.asyncly $ Stream.replicateM 10 $ delay 1

121-- ...

130-- ...

122--

131--

123−-- Above, the 'asyncly' combinator forces the argument stream to be of type

124−-- 'AsyncT'. By default, there is a limit on how many concurrent actions may

132+-- By default, there is a limit on how many concurrent actions may be executed

125−-- be executed at a time, the "Concurrency Control" section describes

133+-- at a time, the "Concurrency Control" section describes combinators that can

126−-- combinators that can be used to control the concurrency.

134+-- be used to control the concurrency.

127--

135--

128-- We can use 'mapM' to map an action concurrently:

136-- We can use 'mapM' to map an action concurrently:

129--

137--

@@ -156,7 +164,7 @@

156--

164--

157-- >>> stream1 = Stream.fromListM [delay 1, delay 2]

165-- >>> stream1 = Stream.fromListM [delay 1, delay 2]

158-- >>> stream2 = Stream.fromListM [delay 3, delay 4]

166-- >>> stream2 = Stream.fromListM [delay 3, delay 4]

159−-- >>> Stream.drain $ Stream.serially $ stream1 <> stream2

167+-- >>> Stream.drain $ stream1 <> stream2

160-- 1 sec

168-- 1 sec

161-- 2 sec

169-- 2 sec

162-- 3 sec

170-- 3 sec

@@ -188,7 +196,7 @@

188--

196--

189-- >>> stream1 = Stream.fromListM [delay 1, delay 2]

197-- >>> stream1 = Stream.fromListM [delay 1, delay 2]

190-- >>> stream2 = Stream.fromListM [delay 3, delay 4]

198-- >>> stream2 = Stream.fromListM [delay 3, delay 4]

191−-- >>> Stream.drain $ Stream.asyncly $ stream1 `parallel` stream2

199+-- >>> Stream.drain $ stream1 `parallel` stream2

192-- 1 sec

200-- 1 sec

193-- 2 sec

201-- 2 sec

194-- 3 sec

202-- 3 sec

@@ -217,6 +225,18 @@

217-- behavior of the stream as well as the behavior of 'Semigroup' and 'Monad'

225-- behavior of the stream as well as the behavior of 'Semigroup' and 'Monad'

218-- instances.

226-- instances.

219--

227--

228+-- == Caveats

229+--

230+-- When we use combinators like 'asyncly' on a piece of code, all combinators

231+-- inside the argument of asyncly become concurrent which is often counter

232+-- productive. Therefore, we recommend that in a pipeline, you identify the

233+-- combinators that you really want to be concurrent and add a 'serially' after

234+-- those combinators so that the code following the combinator remains serial:

235+--

236+-- @

237+-- Stream.asyncly $ ... concurrent combinator here ... $ Stream.serially $ ...

238+-- @

239+--

220-- == Polymorphic Combinators

240-- == Polymorphic Combinators

221--

241--

222-- The combinators in this module are polymorphic in stream type. The stream is

242-- The combinators in this module are polymorphic in stream type. The stream is

@@ -869,27 +889,11 @@ import Streamly.Internal.Data.Stream.IsStream

869

889

870-- $serial

890-- $serial

871--

891--

872−-- When a stream consumer demands an element from a serial stream constructed

873−-- as @a \`consM` b \`consM` ... nil@, the action @a@ at the head of the stream

874−-- sequence is executed and the result is supplied to the consumer. When the

875−-- next element is demanded, the action @b@ is executed and its result is

876−-- supplied. Thus, the effects are performed and results are consumed strictly

877−-- in a serial order. Serial streams can be considered as /spatially ordered/

892+-- Serial streams are /spatially ordered/, they execute the actions in the

878−-- streams as the order of execution and consumption is the same as the spatial

893+-- stream strictly one after the other.

879−-- order in which the actions are composed by the programmer.

880−--

881−-- Serial streams enforce the side effects as well as the results of the

882−-- actions to be in the same order in which the actions are added to the

883−-- stream. Therefore, the semigroup operation for serial streams is not

884−-- commutative:

885−--

886−-- @

887−-- a <> b is not the same as b <> a

888−-- @

889--

894--

890−-- There are two serial stream types 'SerialT' and 'WSerialT'. The stream

895+-- There are two serial stream types 'SerialT' and 'WSerialT'. They differ

891−-- evaluation of both the variants works in the same way as described above,

892−-- they differ only in the 'Semigroup' and 'Monad' implementaitons.

896+-- only in the 'Semigroup' and 'Monad' instance behavior.

893

897

894-- $ahead

898-- $ahead

895--

899--