@@ -11,12 +11,230 @@
1111-- Stability : experimental
1212-- Portability : GHC
1313--
14−-- This module is designed to be imported qualified:
14+-- To run examples in this module:
15+--
16+-- >>> import qualified Streamly.Data.Fold as Fold
17+-- >>> import qualified Streamly.Prelude as Stream
18+--
19+-- We will add some more imports in the examples as needed.
20+--
21+-- = Overview
22+--
23+-- Streamly is a framework for modular data flow based programming and
24+-- declarative concurrency. Streamly API is similar to Haskell lists.
25+--
26+-- The type @SerialT IO a@ is an effectful equivalent of a list @[a]@ using the
27+-- IO monad. Streams can be constructed like lists, except that they use 'nil'
28+-- instead of '[]' and 'cons' instead of ':'. `cons` constructs a pure stream
29+-- which is more or less the same as a list:
30+--
31+-- >>> import Streamly.Prelude (SerialT, cons, consM, nil)
32+-- >>> stream :: SerialT IO Int = 1 `cons` 2 `cons` nil
33+-- >>> Stream.toList stream -- IO [Int]
34+-- [1,2]
35+--
36+-- 'consM' constructs an effectful stream:
1537--
1638-- @
17−-- import qualified Streamly.Prelude as S
39+-- > stream = 'getLine' \`consM` 'getLine' \`consM` 'nil'
40+-- > Stream.'toList' stream
41+-- hello
42+-- world
43+-- ["hello","world"]
1844-- @
1945--
46+-- In the following example, 'repeatM' generates an infinite stream of 'String'
47+-- by repeatedly performing the 'getLine' IO action. 'mapM' then applies
48+-- 'putStrLn' on each element in the stream converting it to stream of '()'.
49+-- Finally, 'drain' folds the stream to IO discarding the () values, thus
50+-- producing only effects.
51+--
52+-- >>> :{
53+-- Stream.repeatM getLine -- SerialT IO String
54+-- & Stream.mapM putStrLn -- SerialT IO ()
55+-- & Stream.drain -- IO ()
56+-- :}
57+--
58+-- This is a console echo program. It is an example of a declarative loop.
59+-- Compare it with an imperative @while@ loop.
60+--
61+-- Hopefully, this gives you an idea how we can program declaratively by
62+-- represneting loops using streams. For more details please see the
63+-- "Streamly.Tutorial" module.
64+--
65+-- == Stream Types
66+--
67+-- One way in which stream types are distinguished is how the sequence of
68+-- actions forming the stream are executed. Below we will see how 'SerialT',
69+-- 'AsyncT' and 'AheadT' stream types differ from each other.
70+--
71+-- To demonstrate the serial vs. concurrent execution behavior we will use the
72+-- following blocking action that adds a delay of @n@ seconds:
73+--
74+-- >>> import Control.Concurrent (threadDelay)
75+-- >>> :{
76+-- delay n = do
77+-- threadDelay (n * 1000000) -- sleep for n seconds
78+-- putStrLn (show n ++ " sec") -- print "n sec"
79+-- return n -- IO Int
80+-- :}
81+--
82+-- 'SerialT' executes actions serially, so the total delay in the following
83+-- example is @2 + 1 = 3@ seconds:
84+--
85+-- >>> stream :: SerialT IO Int = delay 2 `consM` delay 1 `consM` nil
86+-- >>> Stream.toList stream -- IO [Int]
87+-- 2 sec
88+-- 1 sec
89+-- [2,1]
90+--
91+-- 'AsyncT' executes the actions concurrently, so the total delay is @max 2 1 =
92+-- 2@ seconds:
93+--
94+-- >>> stream :: AsyncT IO Int = delay 2 `consM` delay 1 `consM` nil
95+-- >>> Stream.toList stream -- IO [Int]
96+-- 1 sec
97+-- 2 sec
98+-- [1,2]
99+--
100+-- 'AsyncT' produces the results in the order in which execution finishes.
101+-- Notice the order of elements in the list above, it is not the same as the
102+-- order of actions in the stream.
103+--
104+-- 'AheadT' is similar to 'AsyncT' but the order of results is the same as the
105+-- order of actions, even though they execute concurrently:
106+--
107+-- >>> stream :: AheadT IO Int = delay 2 `consM` delay 1 `consM` nil
108+-- >>> Stream.toList stream -- IO [Int]
109+-- 1 sec
110+-- 2 sec
111+-- [2,1]
112+--
113+-- == Concurrent Combinators
114+--
115+-- Like 'consM', there are several other stream generation operations whose
116+-- execution behavior depends on the stream type, they all follow behavior
117+-- similar to 'consM'. For example, 'replicateM' in the following example
118+-- executes the replicated actions concurrently, just like 'consM':
119+--
120+-- >>> Stream.drain $ Stream.asyncly $ Stream.replicateM 10 $ delay 1
121+-- ...
122+--
123+-- Above, the 'asyncly' combinator forces the argument stream to be of type
124+-- 'AsyncT'. By default, there is a limit on how many concurrent actions may
125+-- be executed at a time, the "Concurrency Control" section describes
126+-- combinators that can be used to control the concurrency.
127+--
128+-- We can use 'mapM' to map an action concurrently:
129+--
130+-- >>> f x = delay 1 >> return (x + 1)
131+-- >>> Stream.toList $ Stream.aheadly $ Stream.mapM f $ Stream.fromList [1..3]
132+-- ...
133+-- [2,3,4]
134+--
135+-- 'aheadly' forces mapM to happen in 'AheadT' style, thus all three actions
136+-- take only one second even though each individual action blocks for a second.
137+--
138+-- See the documentation of individual combinators to check if it is concurrent
139+-- or not. The concurrent combinators necessarily have a @MonadAsync m@
140+-- constraint. However, a @MonadAsync m@ constraint does not necessarily mean
141+-- that the combinator is concurrent.
142+--
143+-- == Combining Two streams
144+--
145+-- Earlier we distinguished stream types based on the execution behavior of
146+-- actions within a stream. Stream types are also distinguished based on how
147+-- actions from different streams are scheduled for execution when two streams
148+-- are combined together.
149+--
150+-- For example, both 'SerialT' and 'WSerialT' execute actions within the stream
151+-- serially, however, they differ in how actions from individual streams are
152+-- executed when two streams are combined with '<>' (the 'Semigroup' instance).
153+--
154+-- For 'SerialT', '<>' has an appending behavior i.e. it executes the actions
155+-- from the second stream after executing actions from the first stream:
156+--
157+-- >>> stream1 = Stream.fromListM [delay 1, delay 2]
158+-- >>> stream2 = Stream.fromListM [delay 3, delay 4]
159+-- >>> Stream.drain $ Stream.serially $ stream1 <> stream2
160+-- 1 sec
161+-- 2 sec
162+-- 3 sec
163+-- 4 sec
164+--
165+-- For 'WSerialT', '<>' has an interleaving behavior i.e. it executes one
166+-- action from the first stream and then one action from the second stream and
167+-- so on:
168+--
169+-- >>> stream1 = Stream.fromListM [delay 1, delay 2]
170+-- >>> stream2 = Stream.fromListM [delay 3, delay 4]
171+-- >>> Stream.drain $ Stream.wSerially $ stream1 <> stream2
172+-- 1 sec
173+-- 3 sec
174+-- 2 sec
175+-- 4 sec
176+--
177+-- The '<>' operation of 'SerialT' and 'WSerialT' is the same as 'serial' and
178+-- 'wSerial' respectively. The 'serial' combinator combines two streams of any
179+-- type in the same way as a serial stream combines.
180+--
181+-- == Combining N streams
182+--
183+-- The 'concatMapWith' combinator can be used to generalize the two stream
184+-- combining combinators to @n@ streams.
185+--
186+-- Let's take an example of the `parallel` combinator ('<>' operation of
187+-- 'ParallelT'), it schedules both the streams concurrently.
188+--
189+-- >>> stream1 = Stream.fromListM [delay 1, delay 2]
190+-- >>> stream2 = Stream.fromListM [delay 3, delay 4]
191+-- >>> Stream.drain $ Stream.asyncly $ stream1 `parallel` stream2
192+-- 1 sec
193+-- 2 sec
194+-- 3 sec
195+-- 4 sec
196+--
197+-- We can use @concatMapWith parallel@ to read concurrently from all incoming
198+-- network connections and combine the input streams into a single output
199+-- stream:
200+--
201+-- @
202+-- import qualified Streamly.Network.Inet.TCP as TCP
203+-- import qualified Streamly.Network.Socket as Socket
204+--
205+-- Stream.unfold TCP.acceptOnPort 8090
206+-- & Stream.concatMapWith Stream.parallel (Stream.unfold Socket.read)
207+-- @
208+--
209+-- See the @streamly-examples@ repository for a full working example.
210+--
211+-- The 'Monad' instance of 'ParallelT' uses @concatMapWith parallel@ as its
212+-- bind operation. Therefore, the monad composition behaves in the same way.
213+-- The monad instance of stream types is essentially nested looping, it means
214+-- each iteration of the loop for 'ParallelT' stream can run concurrently.
215+--
216+-- See the documentation for individual stream types for the specific execution
217+-- behavior of the stream as well as the behavior of 'Semigroup' and 'Monad'
218+-- instances.
219+--
220+-- == Polymorphic Combinators
221+--
222+-- The combinators in this module are polymorphic in stream type. The stream is
223+-- usually specified as @t m a@ where the stream type @t@ is a member of the
224+-- 'IsStream' type class (e.g. 'SerialT' or 'AsyncT'), @m@ is the underlying
225+-- 'Monad' of the stream (e.g. IO) and @a@ is the type of elements in the
226+-- stream (e.g. Int).
227+--
228+-- Deconstruction and folding combinators accept a 'SerialT' type instead of a
229+-- polymorphic type to ensure that streams always have a concrete monomorphic
230+-- type by default, reducing type errors.
231+--
232+-- In case you want to adapt a stream type to another stream type you can use
233+-- one of the type adapting combinators provided in the section "Stream Type
234+-- Adaptors".
235+--
236+-- == Conventions
237+--
20238-- Functions with the suffix @M@ are general functions that work on monadic
21239-- arguments. The corresponding functions without the suffix @M@ work on pure
22240-- arguments and can in general be derived from their monadic versions but are
@@ -26,21 +244,12 @@
26244-- In many cases, short definitions of the combinators are provided in the
27245-- documentation for illustration. The actual implementation may differ for
28246-- performance reasons.
29−--
30−-- Functions having a 'MonadAsync' constraint work concurrently when used with
31−-- appropriate stream type combinator. Please be careful to not use 'parallely'
32−-- with infinite streams.
33−--
34−-- Deconstruction and folds accept a 'SerialT' type instead of a polymorphic
35−-- type to ensure that streams always have a concrete monomorphic type by
36−-- default, reducing type errors. In case you want to use any other type of
37−-- stream you can use one of the type combinators provided to convert the
38−-- stream type.
39247
40248module Streamly.Prelude
41249 (
42250 -- * Stream Types
43− -- $streamtypes
251+ -- | Stream types that end with a @T@ (e.g. 'SerialT') are monad
252+ -- transformers.
44253
45254 -- ** Serial Streams
46255 -- $serial
@@ -658,25 +867,6 @@ import Prelude
658867
659868import Streamly.Internal.Data.Stream.IsStream
660869
661−-- $setup
662−-- >>> :m
663−-- >>> import qualified Streamly.Prelude as Stream
664−-- >>> import qualified Streamly.Data.Fold as Fold
665−
666−-- $streamtypes
667−-- The basic stream type is 'Serial', it represents a sequence of IO actions,
668−-- and is a 'Monad'. The type 'SerialT' is a monad transformer that can
669−-- represent a sequence of actions in an arbitrary monad. The type 'Serial' is
670−-- in fact a synonym for @SerialT IO@. There are a few more types similar to
671−-- 'SerialT', all of them represent a stream and differ only in the
672−-- 'Semigroup', 'Applicative' and 'Monad' compositions of the stream. 'Serial'
673−-- and 'WSerial' types compose serially whereas 'Async' and 'WAsync'
674−-- types compose concurrently. All these types can be freely inter-converted
675−-- using type combinators without any cost. You can freely switch to any type
676−-- of composition at any point in the program. When no type annotation or
677−-- explicit stream type combinators are used, the default stream type is
678−-- inferred as 'Serial'.
679−
680870-- $serial
681871--
682872-- When a stream consumer demands an element from a serial stream constructed