on 09c955e help sign in
streamly/src/Streamly.hs
251 lines 8.1 kB history raw
1-- |
2-- Module : Streamly
3-- Copyright : (c) 2017 Harendra Kumar
4--
5-- License : BSD3
6-- Maintainer : harendra.kumar@gmail.com
7-- Stability : experimental
8-- Portability : GHC
9--
10-- The way a list represents a sequence of pure values, a stream represents a
11-- sequence of monadic actions. The monadic stream API offered by Streamly is
12-- very close to the Haskell "Prelude" pure lists' API, it can be considered as
13-- a natural extension of lists to monadic actions. Streamly streams provide
14-- concurrent composition and merging of streams. It can be considered as a
15-- concurrent list transformer. In contrast to the "Prelude" lists, merging or
16-- appending streams of arbitrary length is scalable and inexpensive.
17--
18-- The basic stream type is 'Serial', it represents a sequence of IO actions,
19-- and is a 'Monad'. The type 'SerialT' is a monad transformer that can
20-- represent a sequence of actions in an arbitrary monad. The type 'Serial' is
21-- in fact a synonym for @SerialT IO@. There are a few more types similar to
22-- 'SerialT', all of them represent a stream and differ only in the
23-- 'Semigroup', 'Applicative' and 'Monad' compositions of the stream. 'Serial'
24-- and 'WSerial' types compose serially whereas 'Async' and 'WAsync'
25-- types compose concurrently. All these types can be freely inter-converted
26-- using type combinators without any cost. You can freely switch to any type
27-- of composition at any point in the program. When no type annotation or
28-- explicit stream type combinators are used, the default stream type is
29-- inferred as 'Serial'.
30--
31-- Here is a simple console echo program example:
32--
33-- @
34-- > runStream $ S.repeatM getLine & S.mapM putStrLn
35-- @
36--
37-- For more details please see the "Streamly.Tutorial" module and the examples
38-- directory in this package.
39--
40-- This module exports stream types, instances and some basic operations.
41-- Functionality exported by this module include:
42--
43-- * Semigroup append ('<>') instances as well as explicit operations for merging streams
44-- * Monad and Applicative instances for looping over streams
45-- * Zip Applicatives for zipping streams
46-- * Stream type combinators to convert between different composition styles
47-- * Some basic utilities to run and fold streams
48--
49-- See the "Streamly.Prelude" module for comprehensive APIs for construction,
50-- generation, elimination and transformation of streams.
51--
52-- This module is designed to be imported unqualified:
53--
54-- @
55-- import Streamly
56-- @
58module Streamly
59 (
60 MonadAsync
62 -- * Stream transformers
63 -- ** Serial Streams
64 -- $serial
65 , SerialT
66 , WSerialT
68 -- ** Concurrent Lookahead Streams
69 -- $lookahead
70 , AheadT
72 -- ** Concurrent Asynchronous Streams
73 -- $async
74 , AsyncT
75 , WAsyncT
76 , ParallelT
78 -- ** Zipping Streams
79 -- $zipping
80 , ZipSerialM
81 , ZipAsyncM
83 -- * Running Streams
84 , runStream
86 -- * Parallel Function Application
87 -- $application
88 , (|$)
89 , (|&)
90 , (|$.)
91 , (|&.)
92 , mkAsync
94 -- * Merging Streams
95 -- $sum
96 , serial
97 , wSerial
98 , ahead
99 , async
100 , wAsync
101 , parallel
103 -- * Concurrency Control
104 -- $concurrency
105 , maxThreads
106 , maxBuffer
108 -- * Folding Containers of Streams
109 -- $foldutils
110 , foldWith
111 , foldMapWith
112 , forEachWith
114 -- * Stream Type Adapters
115 -- $adapters
116 , IsStream ()
118 , serially
119 , wSerially
120 , asyncly
121 , aheadly
122 , wAsyncly
123 , parallely
124 , zipSerially
125 , zipAsyncly
126 , adapt
128 -- * IO Streams
129 , Serial
130 , WSerial
131 , Ahead
132 , Async
133 , WAsync
134 , Parallel
135 , ZipSerial
136 , ZipAsync
138 -- * Re-exports
139 , Semigroup (..)
140 -- * Deprecated
141 , Streaming
142 , runStreaming
143 , runStreamT
144 , runInterleavedT
145 , runAsyncT
146 , runParallelT
147 , runZipStream
148 , runZipAsync
149 , StreamT
150 , InterleavedT
151 , ZipStream
152 , interleaving
153 , zipping
154 , zippingAsync
155 , (<=>)
156 , (<|)
157 )
158where
160import Streamly.Streams.StreamK hiding (runStream, serial)
161import Streamly.Streams.Serial
162import Streamly.Streams.Async
163import Streamly.Streams.Ahead
164import Streamly.Streams.Parallel
165import Streamly.Streams.Zip
166import Streamly.Streams.Prelude
167import Streamly.Streams.SVar (maxThreads, maxBuffer)
168import Streamly.SVar (MonadAsync)
169import Data.Semigroup (Semigroup(..))
171-- $serial
172--
173-- Serial streams compose serially or non-concurrently. In a composed stream,
174-- each action is executed only after the previous action has finished. The two
175-- serial stream types 'SerialT' and 'WSerialT' differ in how they traverse the
176-- streams in a 'Semigroup' or 'Monad' composition.
178-- $async
179--
180-- The async style streams execute actions asynchronously and consume the
181-- outputs as well asynchronously. In a composed stream, at any point of time
182-- more than one stream can run concurrently and yield elements. The elements
183-- are yielded by the composed stream as they are generated by the constituent
184-- streams on a first come first serve basis. Therefore, on each run the
185-- stream may yield elements in a different sequence depending on the delays
186-- introduced by scheduling. The two async types 'AsyncT' and 'WAsyncT' differ
187-- in how they traverse streams in 'Semigroup' or 'Monad' compositions.
189-- $zipping
190--
191-- 'ZipSerialM' and 'ZipAsyncM', provide 'Applicative' instances for zipping the
192-- corresponding elements of two streams together. Note that these types are
193-- not monads.
195-- $application
196--
197-- Stream processing functions can be composed in a chain using function
198-- application with or without the '$' operator, or with reverse function
199-- application operator '&'. Streamly provides concurrent versions of these
200-- operators applying stream processing functions such that each stage of the
201-- stream can run in parallel. The operators start with a @|@; we can read '|$'
202-- as "@parallel dollar@" to remember that @|@ comes before '$'.
203--
204-- Imports for the code snippets below:
205--
206-- @
207-- import Streamly
208-- import qualified Streamly.Prelude as S
209-- import Control.Concurrent
210-- @
212-- $sum
213-- The 'Semigroup' operation '<>' of each stream type combines two streams in a
214-- type specific manner. This section provides polymorphic versions of '<>'
215-- which can be used to combine two streams in a predetermined way irrespective
216-- of the type.
218-- $concurrency
219--
220-- These combinators can be used at any point in a stream composition to
221-- control the concurrency of the enclosed stream. When the combinators are
222-- used in a nested manner, the nearest enclosing combinator overrides the
223-- outer ones. These combinators have no effect on 'Parallel' streams,
224-- concurrency for 'Parallel' streams is always unbounded.
225-- Note that the use of these combinators does not enable concurrency, to
226-- enable concurrency you have to use one of the concurrent stream type
227-- combinators.
229-- $adapters
230--
231-- You may want to use different stream composition styles at different points
232-- in your program. Stream types can be freely converted or adapted from one
233-- type to another. The 'IsStream' type class facilitates type conversion of
234-- one stream type to another. It is not used directly, instead the type
235-- combinators provided below are used for conversions.
236--
237-- To adapt from one monomorphic type (e.g. 'AsyncT') to another monomorphic
238-- type (e.g. 'SerialT') use the 'adapt' combinator. To give a polymorphic code
239-- a specific interpretation or to adapt a specific type to a polymorphic type
240-- use the type specific combinators e.g. 'asyncly' or 'wSerially'. You
241-- cannot adapt polymorphic code to polymorphic code, as the compiler would not know
242-- which specific type you are converting from or to. If you see a an
243-- @ambiguous type variable@ error then most likely you are using 'adapt'
244-- unnecessarily on polymorphic code.
245--
247-- $foldutils
248--
249-- These are variants of standard 'Foldable' fold functions that use a
250-- polymorphic stream sum operation (e.g. 'async' or 'wSerial') to fold a
251-- container of streams.