git.iko.soy/streamly
on master help sign in
update docs, readme, cabal description and tutorial
ca0e9a9Harendra Kumar
Changelog.md+3
compare with master
unifiedsplit
@@ -64,6 +64,9 @@
64
64
65### Enhancements
65### Enhancements
66* Add the following functions:
66* Add the following functions:
67+ * `consM` and `|:` operator to construct streams from monadic actions
68+ * `once` to create a singleton stream from a monadic action
69+ * `repeatM` to construct a stream by repeating a monadic action
67 * `scanl'` strict left scan
70 * `scanl'` strict left scan
68 * `foldl'` strict left fold
71 * `foldl'` strict left fold
69 * `foldlM'` strict left fold with a monadic fold function
72 * `foldlM'` strict left fold with a monadic fold function
README.md+120−74
@@ -8,23 +8,118 @@
8
8
9## Stream`ing` `Concurrent`ly
9## Stream`ing` `Concurrent`ly
10
10
11−Streamly is a monad transformer unifying non-determinism
11+Streamly, short for stream concurrently, is a simple yet powerful streaming
12+library with concurrent merging and concurrent nested looping support. A stream
13+is just like a list except that it is a list of monadic actions rather than
14+pure values. Streamly streams can be generated, consumed, combined, or
15+transformed serially or concurrently. We can loop over a stream serially or
16+concurrently. We can also have serial or concurrent nesting of loops. For
17+those familiar with the list transformer concept this is a concurrent list
18+transformer. Streamly uses standard composition abstractions. Concurrent
19+composition is just the same as serial composition except that we use a simple
20+combinator to request a concurrent composition instead of serial. The
21+programmer does not have to be aware of threads, locking or synchronization to
22+write scalable concurrent programs.
23+
24+Streamly provides functionality that is equivalent to streaming libraries
12−([list-t](https://hackage.haskell.org/package/list-t)/[logict](https://hackage.haskell.org/package/logict)),
25+like [pipes](https://hackage.haskell.org/package/pipes) and
26+[conduit](https://hackage.haskell.org/package/conduit) but with a list like
27+API. The streaming API of streamly is close to the monadic streams API of the
13−concurrency ([async](https://hackage.haskell.org/package/async)),
28+[vector](https://hackage.haskell.org/package/vector) package and similar in
29+concept to the [streaming](https://hackage.haskell.org/package/streaming)
30+package. In addition to streaming, streamly subsumes the functionality of list
31+transformer libraries like `pipes` or
14−streaming ([conduit](https://hackage.haskell.org/package/conduit)\/[pipes](https://hackage.haskell.org/package/pipes)),
32+[list-t](https://hackage.haskell.org/package/list-t) and also the logic
15−and FRP ([Yampa](https://hackage.haskell.org/package/Yampa)\/[reflex](https://hackage.haskell.org/package/reflex))
33+programming library [logict](https://hackage.haskell.org/package/logict). On
16−functionality in a concise and intuitive API.
34+the concurrency side, it subsumes the functionality of the
17−High level concurrency makes concurrent applications almost indistinguishable
35+[async](https://hackage.haskell.org/package/async) package. Because it streams
18−from non-concurrent ones. By changing a single combinator you can control
36+and supports concurrency we can write FRP applications similar in concept to
19−whether the code runs serially or concurrently. It naturally integrates
37+[Yampa](https://hackage.haskell.org/package/Yampa) or
20−concurrency with streaming rather than adding it as an afterthought.
38+[reflex](https://hackage.haskell.org/package/reflex).
39+
40+Streamly has excellent performance, see
41+[streaming-benchmarks](https://github.com/composewell/streaming-benchmarks)
21−Moreover, it interworks with the popular streaming libraries.
42+for a comparison of popular streaming libraries on micro-benchmarks. For
22−
43+more information:
23−See the haddock documentation for full reference. It is recommended that you
44+
24−read `Streamly.Tutorial` first. Also see `Streamly.Examples` for some working
45+ * [Streamly.Tutorial](https://hackage.haskell.org/package/streamly-0.1.2/docs/Streamly-Tutorial.html) module in the haddock documentation for a detailed introduction
25−examples.
46+ * [examples](https://github.com/composewell/streamly/tree/master/examples) directory in the package for some simple practical examples
26−
47+
27−## Non-determinism
48+## Streaming Pipelines
49+
50+Unlike `pipes` or `conduit` and like `vector` and `streaming` `streamly`
51+composes stream data instead of stream processors (functions). A stream is
52+just like a list and is explicitly passed around to functions that process the
53+stream. Therefore, no special operator is needed to join stages in a streaming
54+pipeline, just a forward (`$`) or reverse (`&`) function application operator
55+is enough. Combinators are provided in `Streamly.Prelude` to transform or fold
56+streams.
57+
58+```haskell
59+import Streamly
60+import qualified Streamly.Prelude as S
61+import Data.Function ((&))
62+
63+main = runStream $
64+ S.repeatM getLine
65+ & fmap read
66+ & S.filter even
67+ & S.takeWhile (<= 9)
68+ & fmap (\x -> x * x)
69+ & S.mapM print
70+```
71+
72+## Serial and Concurrent Merging
73+
74+Semigroup and Monoid instances can be used to fold streams serially or
75+concurrently. In the following example we are composing ten actions in the
76+stream each with a delay of 1 to 10 seconds. Since all the actions are
77+concurrent we see one output printed every second:
78+
79+``` haskell
80+import Streamly
81+import qualified Streamly.Prelude as S
82+import Control.Concurrent (threadDelay)
83+
84+main = S.toList $ parallely $ foldMap delay [1..10]
85+ where delay n = S.once $ threadDelay (n * 1000000) >> print n
86+```
87+
88+Streams can be combined together in many ways (see the tutorial for more ways):
89+
90+``` haskell
91+import Streamly
92+import qualified Streamly.Prelude as S
93+import Control.Concurrent
94+
95+delay n = S.once $ do
96+ threadDelay (n * 1000000)
97+ tid <- myThreadId
98+ putStrLn (show tid ++ ": Delay " ++ show n)
99+```
100+### Serial
101+
102+```haskell
103+main = runStream $ delay 3 <> delay 2 <> delay 1
104+```
105+```
106+ThreadId 36: Delay 3
107+ThreadId 36: Delay 2
108+ThreadId 36: Delay 1
109+```
110+
111+### Parallel
112+
113+```haskell
114+main = runStream . parallely $ delay 3 <> delay 2 <> delay 1
115+```
116+```
117+ThreadId 42: Delay 1
118+ThreadId 41: Delay 2
119+ThreadId 40: Delay 3
120+```
121+
122+## Nested Loops (aka List Transformer)
28
123
29The monad instance composes like a list monad.
124The monad instance composes like a list monad.
30
125
@@ -46,7 +141,7 @@ main = runStream loops
46(2,4)
141(2,4)
47```
142```
48
143
49−## Magical Concurrency
144+## Concurrent Nested Loops
50
145
51To run the above code with demand-driven concurrency i.e. each iteration in the
146To run the above code with demand-driven concurrency i.e. each iteration in the
52loops can run concurrently depending on the consumer rate:
147loops can run concurrently depending on the consumer rate:
@@ -67,6 +162,8 @@ To run it serially but interleaving the outer and inner loop iterations:
67main = runStream $ costreamly $ loops
162main = runStream $ costreamly $ loops
68```
163```
69
164
165+## Magical Concurrency
166+
70Streams can perform semigroup (<>) and monadic bind (>>=) operations
167Streams can perform semigroup (<>) and monadic bind (>>=) operations
71concurrently using combinators like `coparallelly`, `parallelly`. For example,
168concurrently using combinators like `coparallelly`, `parallelly`. For example,
72to concurrently generate squares of a stream of numbers and then concurrently
169to concurrently generate squares of a stream of numbers and then concurrently
@@ -112,65 +209,14 @@ The concurrency facilities provided by streamly can be compared with
112[Cilk](https://en.wikipedia.org/wiki/Cilk) but with a more declarative
209[Cilk](https://en.wikipedia.org/wiki/Cilk) but with a more declarative
113expression.
210expression.
114
211
115−## Streaming
116−
117−Streaming is effortless, simple and straightforward. Streamly data type behaves
118−just like a list and combinators are provided in `Streamly.Prelude` to
119−transform or fold streamly streams. Unlike other libraries and like `streaming`
120−library the combinators explicitly consume a stream and produce a stream,
121−therefore, no special operator is needed to join stream stages, just a forward
122−(`$`) or reverse (`&`) function application operator is enough.
123−
124−```haskell
125−import Streamly
126−import qualified Streamly.Prelude as S
127−import Data.Function ((&))
128−
129−main = S.fromFoldable [1..10]
130− & fmap (+ 1)
131− & S.drop 2
132− & S.filter even
133− & fmap (* 3)
134− & S.takeWhile (< 25)
135− & S.mapM (\x -> putStrLn ("saw " ++ show x) >> return x)
136− & S.toList
137− >>= print
138−```
139−
140−Fold style combinators can be used to fold purely or monadically. You can also
141−use the beautiful `foldl` library for folding.
142−
143−```haskell
144−main = S.fromFoldable [1..10]
145− & S.foldl (+) 0 id
146− >>= print
147−```
148−
149−Streams can be combined together in multiple ways:
150−
151−```haskell
152−main = do
153− let p s = toList s >>= print
154− -- serial, this is the default even if you omit the `streamly` combinator
155− p $ streamly $ S.fromFoldable [1..10] <> S.fromFoldable [11..20]
156− -- serial but interleaved
157− p $ costreamly $ S.fromFoldable [1..10] <> S.fromFoldable [11..20]
158− -- left-biased, demand driven parallel
159− p $ coparallely $ S.fromFoldable [1..10] <> S.fromFoldable [11..20]
160− -- round-robin parallel
161− p $ parallely $ S.fromFoldable [1..10] <> S.fromFoldable [11..20]
162−```
163−
164−As we have already seen streams can be combined using monadic composition in a
165−non-deterministic manner. This allows arbitrary manipulation and combining of
166−streams. See `Streamly.Examples.MergeSortedStreams` for a more complicated
167−example.
168−
169## Reactive Programming (FRP)
212## Reactive Programming (FRP)
170
213
171Streamly is a foundation for first class reactive programming as well by virtue
214Streamly is a foundation for first class reactive programming as well by virtue
172−of integrating concurrency and streaming. See `Streamly.Examples.AcidRainGame`
215+of integrating concurrency and streaming. See
216+[AcidRain.hs](https://github.com/composewell/streamly/tree/master/examples/AcidRain.hs)
217+for a console based FRP game example and
218+[CirclingSquare.hs](https://github.com/composewell/streamly/tree/master/examples/CirclingSquare.hs)
173−and `Streamly.Examples.CirclingSquare` for an SDL based animation example.
219+for an SDL based animation example.
174
220
175## Performance
221## Performance
176
222
src/Streamly.hs+70−96
@@ -6,28 +6,60 @@
6-- Maintainer : harendra.kumar@gmail.com
6-- Maintainer : harendra.kumar@gmail.com
7-- Stability : experimental
7-- Stability : experimental
8-- Portability : GHC
8-- Portability : GHC
9+--
10+-- The way a list represents a sequence of pure values, a stream represents a
11+-- sequence of monadic actions. The monadic stream API offered by Streamly is
12+-- very close to the Haskell "Prelude" pure lists' API, it can be considered as
13+-- a natural extension of lists to monadic actions. Streamly streams provide
14+-- concurrent composition and merging of streams. It can be considered as a
15+-- concurrent list transformer. In contrast to the "Prelude" lists, merging or
16+-- appending streams of arbitrary length is scalable and inexpensive.
17+--
18+-- The basic stream type is 'Stream', it represents a sequence of IO actions,
19+-- and is a 'Monad'. The type 'StreamT' is a monad transformer that can
20+-- represent a sequence of actions in an arbitrary monad. The type 'Stream' is
21+-- in fact a synonym for @StreamT IO@. There are a few more types similar to
22+-- 'StreamT', all of them represent a stream and differ only in the
23+-- 'Semigroup', 'Applicative' and 'Monad' compositions of the stream. 'Stream'
24+-- and 'Costream' types compose serially whereas 'Coparallel' and 'Parallel'
25+-- types compose concurrently. All these types can be freely inter-converted
26+-- using type combinators without any cost. You can freely switch to any type
27+-- of composition at any point in the program. When no type annotation or
28+-- explicit stream type combinators are used, the default stream type is
29+-- inferred as 'Stream'.
30+--
31+-- Here is a simple console echo program example:
32+--
33+-- @
34+-- > runStream $ S.repeatM getLine & S.mapM putStrLn
35+-- @
36+--
37+-- For more details please see the "Streamly.Tutorial" module and the examples
38+-- directory in this package.
39+--
40+-- This module exports stream types, instances and some basic operations.
41+-- Functionality exported by this module include:
42+--
43+-- * Semigroup append ('<>') instances as well as explicit operations for merging streams
44+-- * Monad and Applicative instances for looping over streams
45+-- * Zip Applicatives for zipping streams
46+-- * Stream type combinators to convert between different composition styles
47+-- * Some basic utilities to run and fold streams
48+--
49+-- See the "Streamly.Prelude" module for comprehensive APIs for construction,
50+-- generation, elimination and transformation of streams.
51+--
52+-- This module is designed to be imported unqualified:
53+--
54+-- @
55+-- import Streamly
56+-- @
9
57
10module Streamly
58module Streamly
11 (
59 (
12− -- * Background
13− -- $background
14−
15− -- * Overview
16− -- $overview
17−
18 MonadParallel
60 MonadParallel
19
61
20− -- * IO Streams
21− , Stream
22− , Costream
23− , Coparallel
24− , Parallel
25− , ZipStream
26− , ZipParallel
27−
28 -- * Stream transformers
62 -- * Stream transformers
29− , IsStream
30−
31 -- ** Serial Streams
63 -- ** Serial Streams
32 -- $serial
64 -- $serial
33 , StreamT
65 , StreamT
@@ -43,8 +75,17 @@ module Streamly
43 , ZipStreamM
75 , ZipStreamM
44 , ZipParallelM
76 , ZipParallelM
45
77
78+ -- * Polymorphic Sum Operations
79+ -- $sum
80+ , splice
81+ , cosplice
82+ , coparallel
83+ , parallel
84+
46 -- * Stream Type Adapters
85 -- * Stream Type Adapters
47 -- $adapters
86 -- $adapters
87+ , IsStream
88+
48 , streamly
89 , streamly
49 , costreamly
90 , costreamly
50 , coparallely
91 , coparallely
@@ -53,19 +94,20 @@ module Streamly
53 , zipParallely
94 , zipParallely
54 , adapt
95 , adapt
55
96
97+ -- * IO Streams
98+ , Stream
99+ , Costream
100+ , Coparallel
101+ , Parallel
102+ , ZipStream
103+ , ZipParallel
104+
56 -- * Running Streams
105 -- * Running Streams
57 , runStream
106 , runStream
58
107
59 -- * Transformation
108 -- * Transformation
60 , async
109 , async
61
110
62− -- * Polymorphic Sum Operations
63− -- $sum
64− , splice
65− , cosplice
66− , coparallel
67− , parallel
68−
69 -- * Polymorphic Fold Utilities
111 -- * Polymorphic Fold Utilities
70 -- $foldutils
112 -- $foldutils
71 , foldWith
113 , foldWith
@@ -99,76 +141,6 @@ where
99import Streamly.Streams
141import Streamly.Streams
100import Data.Semigroup (Semigroup(..))
142import Data.Semigroup (Semigroup(..))
101
143
102−-- $background
103−--
104−-- Streamly provides a monad transformer that extends the product style
105−-- composition of monads to streams of many elements of the same type; it is a
106−-- functional programming equivalent of nested loops from imperative
107−-- programming. Composing each element in one stream with each element in the
108−-- other stream generalizes the monadic product of single elements. You can
109−-- think of the IO monad as a special case of the more general @StreamT IO@
110−-- monad; with single element streams. List transformers and logic programming
111−-- monads also provide a similar product style composition of streams, however
112−-- streamly generalizes it with the time dimension; allowing streams to be
113−-- composed in an asynchronous and concurrent fashion in many different ways.
114−-- It also provides multiple alternative ways of composing streams e.g.
115−-- serial, interleaved or concurrent.
116−--
117−-- The seemingly simple addition of asynchronicity and concurrency to product
118−-- style streaming composition unifies a number of disparate abstractions into
119−-- one powerful and elegant abstraction. A wide variety of programming
120−-- problems can be solved elegantly with this abstraction. In particular, it
121−-- unifies three major programming domains namely non-deterministic (logic)
122−-- programming, concurrent programming and functional reactive programming. In
123−-- other words, you can do everything with this one abstraction that you could
124−-- with list transformers (e.g.
125−-- <https://hackage.haskell.org/package/list-t list-t>), logic programming
126−-- monads (e.g. <https://hackage.haskell.org/package/logict logict>),
127−-- streaming libraries (a lot of what
128−-- <https://hackage.haskell.org/package/conduit conduit> or
129−-- <https://hackage.haskell.org/package/pipes pipes> can do), concurrency
130−-- libraries (e.g. <https://hackage.haskell.org/package/async async>) and FRP
131−-- libraries (e.g. <https://hackage.haskell.org/package/Yampa Yampa> or
132−-- <https://hackage.haskell.org/package/reflex reflex>).
133−
134−-- $overview
135−--
136−-- Streamly provides six distinct stream types i.e. 'StreamT', 'CostreamT',
137−-- 'CoparallelT' and 'ParallelT', 'ZipStreamM' and 'ZipParallelM', each representing a
138−-- stream of elements. All these types have the same underlying representation
139−-- and can be adapted from one to another using type adaptor combinators
140−-- described later. Each of these types belongs to the 'IsStream' type class
141−-- which helps converting the specific type to and from the underlying generic
142−-- stream type.
143−--
144−-- The types 'StreamT', 'CostreamT', 'CoparallelT' and 'ParallelT' are 'Monad'
145−-- transformers with the monadic bind operation combining streams in a product
146−-- style in much the same way as a list monad or a list transformer i.e. each
147−-- element from one stream is combined with every element of the other stream.
148−-- However, the applicative and monadic composition of these types differ in
149−-- terms of the ordering and time sequence in which the elements from two
150−-- streams are combined. 'StreamT' and 'CostreamT' compose streams streamly
151−-- whereas 'CoparallelT' and 'ParallelT' are their concurrent counterparts. See the
152−-- documentation of the respective types for more details.
153−--
154−-- The types 'ZipStreamM' and 'ZipParallelM' provide 'Applicative' instances to zip
155−-- two streams together i.e. each element in one stream is combined with the
156−-- corresponding element in the other stream. 'ZipStreamM' generates the streams
157−-- being zipped serially whereas 'ZipParallelM' produces both the elements being
158−-- zipped concurrently.
159−--
160−-- Two streams of the same type can be merged using a sum style composition to
161−-- generate a stream of the same type where the output stream would contain all
162−-- elements of both the streams. However, the sequence or the manner
163−-- (concurrent or serial) in which the elements in the resulting stream are
164−-- produced depends on the type of the stream.
165−--
166−-- For more details please see the "Streamly.Tutorial" and "Streamly.Examples"
167−-- (the latter is available only when built with the 'examples' build flag).
168−
169−-- A simple inline example here illustrating applicative, monad and alternative
170−-- compositions.
171−
172-- $serial
144-- $serial
173--
145--
174-- Serial streams compose serially or non-concurrently. In a composed stream,
146-- Serial streams compose serially or non-concurrently. In a composed stream,
@@ -205,9 +177,11 @@ import Data.Semigroup (Semigroup(..))
205
177
206-- $adapters
178-- $adapters
207--
179--
208−-- You may want to use different stream types at different points in your
180+-- You may want to use different stream composition styles at different points
209−-- program. Stream types can be converted or adapted from one type to another
181+-- in your program. Stream types can be freely converted or adapted from one
210−-- to make them interwork with each other.
182+-- type to another. The 'IsStream' type class facilitates type conversion of
183+-- one stream type to another. It is not used directly, instead the type
184+-- combinators provided below are used for conversions.
211--
185--
212-- To adapt from one monomorphic type (e.g. 'ParallelT') to another monomorphic
186-- To adapt from one monomorphic type (e.g. 'ParallelT') to another monomorphic
213-- type (e.g. 'StreamT') use the 'adapt' combinator. To give a polymorphic code
187-- type (e.g. 'StreamT') use the 'adapt' combinator. To give a polymorphic code
src/Streamly/Prelude.hs+6
@@ -14,6 +14,12 @@
14-- Stability : experimental
14-- Stability : experimental
15-- Portability : GHC
15-- Portability : GHC
16--
16--
17+-- This module is designed to be imported qualified:
18+--
19+-- @
20+-- import qualified Streamly.Prelude as S
21+-- @
22+--
17-- Functions with the suffix @M@ are general functions that work on monadic
23-- Functions with the suffix @M@ are general functions that work on monadic
18-- arguments. The corresponding functions without the suffix @M@ work on pure
24-- arguments. The corresponding functions without the suffix @M@ work on pure
19-- arguments and can in general be derived from their monadic versions but are
25-- arguments and can in general be derived from their monadic versions but are
src/Streamly/Tutorial.hs+165−99
@@ -6,40 +6,41 @@
6-- License : BSD3
6-- License : BSD3
7-- Maintainer : harendra.kumar@gmail.com
7-- Maintainer : harendra.kumar@gmail.com
8--
8--
9−-- Streamly, short for stream concurrently, combines the essence of
9+-- Streamly is a general computing framework based on streaming IO. The IO
10−-- non-determinism, streaming and concurrency in functional programming.
10+-- monad and pure lists are a special case of streamly. On one hand, streamly
11−-- Concurrent and non-concurrent applications are almost indistinguishable,
11+-- extends the lists of pure values to lists of monadic actions, on the other
12−-- concurrency capability does not at all impact the performance of
12+-- hand it extends the IO monad with concurrrent non-determinism. In simple
13−-- non-concurrent case.
13+-- imperative terms we can say that streamly extends the IO monad with @for@
14+-- loops and nested @for@ loops with concurrency support. You can understand
15+-- this analogy better once you can go through this tutorial.
16+--
14−-- Streaming enables writing modular, composable and scalable applications with
17+-- Streaming in general enables writing modular, composable and scalable
15−-- ease and concurrency allows you to make them scale and perform well.
18+-- applications with ease, and concurrency allows you to make them scale and
16−-- Streamly enables writing concurrent applications without being aware of
19+-- perform well. Streamly enables writing scalable concurrent applications
17−-- threads or synchronization. No explicit thread control is needed, where
20+-- without being aware of threads or synchronization. No explicit thread
18−-- applicable the concurrency rate is automatically controlled based on the
21+-- control is needed, where applicable the concurrency rate is automatically
19−-- demand by the consumer. However, combinators are provided to fine tune the
22+-- controlled based on the demand by the consumer. However, combinators can be
20−-- concurrency control.
23+-- used to fine tune the concurrency control.
24+--
21-- Streaming and concurrency together enable expressing reactive applications
25-- Streaming and concurrency together enable expressing reactive applications
26+-- conveniently. See the @CirclingSquare@ example in the examples directory for
22−-- conveniently. See "Streamly.Examples" for a simple SDL based FRP example.
27+-- a simple SDL based FRP example. To summarize, streamly provides a unified
23−--
28+-- computing framework for streaming, non-determinism and functional reactive
24−-- Streamly streams are very much like the Haskell lists and most of the
29+-- programming in an elegant and simple API that is a natural extension of pure
25−-- functions that work on lists have a counterpart that works on streams.
30+-- lists to monadic streams.
26−-- However, streamly streams can be generated, consumed or combined
31+--
27−-- concurrently. In this tutorial we will go over the basic concepts and how to
32+-- In this tutorial we will go over the basic concepts and how to
28−-- use the library. The documentation of @Streamly@ module has more details on
33+-- use the library. See the last section for further reading resources.
29−-- core APIs. For more APIs for constructing, folding, filtering, mapping and
30−-- zipping etc. see the documentation of "Streamly.Prelude" module. For
31−-- examples and other ways to use the library see the module
32−-- "Streamly.Examples" as well.
33
34
34module Streamly.Tutorial
35module Streamly.Tutorial
35 (
36 (
36 -- * Streams
37 -- * Streams
37 -- $streams
38 -- $streams
38
39
39− -- ** Generating Streams
40+ -- * Generating Streams
40 -- $generating
41 -- $generating
41
42
42− -- ** Eliminating Streams
43+ -- * Eliminating Streams
43 -- $eliminating
44 -- $eliminating
44
45
45 -- * Transforming Streams
46 -- * Transforming Streams
@@ -125,6 +126,9 @@ module Streamly.Tutorial
125
126
126 -- * Comparison with Existing Packages
127 -- * Comparison with Existing Packages
127 -- $comparison
128 -- $comparison
129+
130+ -- * Where to go next?
131+ -- $furtherReading
128 )
132 )
129where
133where
130
134
@@ -138,12 +142,43 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
138
142
139-- $streams
143-- $streams
140--
144--
141−-- Streamly provides many different stream types depending on the desired
145+-- The way a list represents a sequence of pure values, a stream represents a
142−-- composition style. The simplest type is 'Stream', @Stream a@ represents a
146+-- sequence of monadic actions. The monadic stream API offered by Streamly is
143−-- stream of elements of type @a@ in IO monad. Streams are very much like lists
147+-- very close to the Haskell "Prelude" pure lists' API, it can be considered as a
148+-- natural extension of lists to monadic actions. Streamly streams provide
149+-- concurrent composition and merging of streams. It can be considered as a
150+-- concurrent list transformer. In contrast to the "Prelude" lists, merging or
151+-- appending streams of arbitrary length is scalable and inexpensive.
152+--
153+-- The basic stream type is 'Stream', it represents a sequence of IO actions,
154+-- and is a 'Monad'. The type 'StreamT' is a monad transformer that can
155+-- represent a sequence of actions in an arbitrary monad. The type 'Stream' is
156+-- in fact a synonym for @StreamT IO@. There are a few more types similar to
157+-- 'StreamT' that all represent streams and differ only in the 'Semigroup',
158+-- 'Applicative' and 'Monad' compositions of the streams. 'Stream' and
159+-- 'Costream' types compose serially whereas 'Coparallel' and 'Parallel' types
160+-- compose concurrently. All these types can be freely inter-converted using
161+-- type combinators without any cost. You can freely switch to any type of
162+-- composition at any point in the program. When no type annotation or
163+-- explicit stream type combinators are used, the default stream type is
164+-- inferred as 'Stream'.
165+--
166+-- The 'Stream' monad is almost a drop in replacement for the 'IO' monad, IO
167+-- monad is a special case of the 'Stream' monad; IO monad represents a single
168+-- IO action whereas the 'Stream' monad represents a sequence of IO actions.
169+-- The only change you need to make to go from 'IO' to 'Stream' is to use
170+-- 'runStream' to run the monad and to prefix the IO actions with 'once'. You
171+-- can also use 'liftIO' instead of 'once'. In fact, if you use liftIO you can
172+-- switch from 'Stream' to IO monad by simply removing the 'runStream'
173+-- function; no other changes are needed unless you have used some stream
174+-- specific composition or combinators.
175+--
176+-- Similarly, the 'Stream' type is almost a drop in replacement for pure lists,
177+-- pure lists are a special case of monadic streams. If you use 'nil' in place
178+-- of '[]' and '|:' in place ':' you can replace a list with a 'Stream'. The
144−-- the only difference is that they are lists of monadic actions instead of
179+-- only difference is that the elements must be monadic type and to operate on
145−-- pure values. We can call them monadic lists. You can apply all the
180+-- the streams we must use the corresponding functions from "Streamly.Prelude"
146−-- constructs related to lists to monadic streams as well.
181+-- instead of using the base "Prelude".
147
182
148-- $monadtransformers
183-- $monadtransformers
149--
184--
@@ -161,22 +196,31 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
161-- singleton stream use 'lift' and to lift from an IO action use 'liftIO'.
196-- singleton stream use 'lift' and to lift from an IO action use 'liftIO'.
162--
197--
163-- @
198-- @
164−-- > runStream $ liftIO $ putStrLn "Hello world!"
199+-- > 'runStream' $ liftIO $ putStrLn "Hello world!"
165-- Hello world!
200-- Hello world!
166−-- > runStream $ lift $ putStrLn "Hello world!"
201+-- > 'runStream' $ lift $ putStrLn "Hello world!"
167-- Hello world!
202-- Hello world!
168-- @
203-- @
169--
204--
170
205
171-- $generating
206-- $generating
172--
207--
208+-- We will assume the following imports in this tutorial. Go ahead, fire up a
209+-- GHCi session and import these lines to start playing.
210+--
211+-- @
212+-- > import "Streamly"
213+-- > import "Streamly.Prelude" (|:)
214+-- > import qualified "Streamly.Prelude" as S
215+-- @
216+--
173-- 'nil' represents an empty stream and 'consM' or its operator form '|:' adds
217-- 'nil' represents an empty stream and 'consM' or its operator form '|:' adds
174-- a monadic action at the head of the stream.
218-- a monadic action at the head of the stream.
175--
219--
176-- @
220-- @
177−-- > toList nil
221+-- > S.'toList' S.'nil'
178-- []
222-- []
179−-- > toList $ getLine |: getLine |: nil
223+-- > S.'toList' $ 'getLine' |: 'getLine' |: S.'nil'
180-- hello
224-- hello
181-- world
225-- world
182-- ["hello","world"]
226-- ["hello","world"]
@@ -186,9 +230,9 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
186-- singleton stream from a monadic action use 'once'.
230-- singleton stream from a monadic action use 'once'.
187--
231--
188-- @
232-- @
189−-- > toList $ pure 1
233+-- > S.'toList' $ 'pure' 1
190-- [1]
234-- [1]
191−-- > toList $ once getLine
235+-- > S.'toList' $ S.'once' 'getLine'
192-- hello
236-- hello
193-- ["hello"]
237-- ["hello"]
194-- @
238-- @
@@ -197,9 +241,9 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
197-- 'fromFoldable' which is equivalent to a fold using 'cons' and 'nil':
241-- 'fromFoldable' which is equivalent to a fold using 'cons' and 'nil':
198--
242--
199-- @
243-- @
200−-- > toList $ fromFoldable [1..3]
244+-- > S.'toList' $ S.'fromFoldable' [1..3]
201-- [1,2,3]
245-- [1,2,3]
202−-- > toList $ Prelude.foldr cons nil [1..3]
246+-- > S.'toList' $ 'Prelude.foldr' S.'cons' S.'nil' [1..3]
203-- [1,2,3]
247-- [1,2,3]
204-- @
248-- @
205--
249--
@@ -207,7 +251,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
207-- right fold using 'consM' and 'nil':
251-- right fold using 'consM' and 'nil':
208--
252--
209-- @
253-- @
210−-- > runStream $ Prelude.foldr (|:) nil [putStr "Hello ", putStrLn "world!"]
254+-- > 'runStream' $ 'Prelude.foldr' ('|:') S.'nil' ['putStr' "Hello ", 'putStrLn' "world!"]
211-- Hello world!
255-- Hello world!
212-- @
256-- @
213--
257--
@@ -236,7 +280,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
236-- forever:
280-- forever:
237--
281--
238-- @
282-- @
239−-- > runStream $ S.repeatM getLine & S.mapM putStrLn
283+-- > 'runStream' $ S.'repeatM' getLine & S.'mapM' putStrLn
240-- @
284-- @
241--
285--
242-- The following code snippet reads lines from standard input, filters blank
286-- The following code snippet reads lines from standard input, filters blank
@@ -244,19 +288,19 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
244-- numbers them and prints them:
288-- numbers them and prints them:
245--
289--
246-- @
290-- @
247−-- import Streamly
291+-- import "Streamly"
248−-- import qualified Streamly.Prelude as S
292+-- import qualified "Streamly.Prelude" as S
249-- import Data.Char (toUpper)
293-- import Data.Char (toUpper)
250-- import Data.Function ((&))
294-- import Data.Function ((&))
251--
295--
252−-- main = runStream $
296+-- main = 'runStream' $
253−-- S.repeatM getLine
297+-- S.'repeatM' getLine
254−-- & S.filter (not . null)
298+-- & S.'filter' (not . null)
255−-- & S.drop 1
299+-- & S.'drop' 1
256−-- & S.take 2
300+-- & S.'take' 2
257-- & fmap (map toUpper)
301-- & fmap (map toUpper)
258−-- & S.zipWith (\\n s -> show n ++ " " ++ s) (S.fromFoldable [1..])
302+-- & S.'zipWith' (\\n s -> show n ++ " " ++ s) (S.'fromFoldable' [1..])
259−-- & S.mapM putStrLn
303+-- & S.'mapM' putStrLn
260-- @
304-- @
261
305
262-- $semigroup
306-- $semigroup
@@ -279,10 +323,11 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
279--
323--
280-- @
324-- @
281-- import "Streamly"
325-- import "Streamly"
282−-- import "Streamly.Prelude"(once)
326+-- import "Streamly.Prelude" (nil)
327+-- import qualified "Streamly.Prelude" as S
283-- import Control.Concurrent
328-- import Control.Concurrent
284--
329--
285−-- delay n = once $ do
330+-- delay n = S.'once' $ do
286-- threadDelay (n * 1000000)
331-- threadDelay (n * 1000000)
287-- tid \<- myThreadId
332-- tid \<- myThreadId
288-- putStrLn (show tid ++ ": Delay " ++ show n)
333-- putStrLn (show tid ++ ": Delay " ++ show n)
@@ -477,7 +522,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
477--
522--
478-- @
523-- @
479-- main = 'runStream' . 'coparallely' $ traced (sqrt 9) '<>' traced (sqrt 16) '<>' traced (sqrt 25)
524-- main = 'runStream' . 'coparallely' $ traced (sqrt 9) '<>' traced (sqrt 16) '<>' traced (sqrt 25)
480−-- where traced m = once (myThreadId >>= print) >> return m
525+-- where traced m = S.'once' (myThreadId >>= print) >> return m
481-- @
526-- @
482-- @
527-- @
483-- ThreadId 40
528-- ThreadId 40
@@ -538,7 +583,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
538--
583--
539-- @
584-- @
540-- import "Streamly"
585-- import "Streamly"
541−-- import Streamly.Prelude (once)
586+-- import qualified Streamly.Prelude as S
542-- import Network.HTTP.Simple
587-- import Network.HTTP.Simple
543--
588--
544-- main = 'runStream' . 'parallely' $ google \<> bing \<> duckduckgo
589-- main = 'runStream' . 'parallely' $ google \<> bing \<> duckduckgo
@@ -546,7 +591,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
546-- google = get "https://www.google.com/search?q=haskell"
591-- google = get "https://www.google.com/search?q=haskell"
547-- bing = get "https://www.bing.com/search?q=haskell"
592-- bing = get "https://www.bing.com/search?q=haskell"
548-- duckduckgo = get "https://www.duckduckgo.com/?q=haskell"
593-- duckduckgo = get "https://www.duckduckgo.com/?q=haskell"
549−-- get s = once (httpNoBody (parseRequest_ s) >> putStrLn (show s))
594+-- get s = S.'once' (httpNoBody (parseRequest_ s) >> putStrLn (show s))
550-- @
595-- @
551--
596--
552-- In the following example we can see that the fairly parallel composition
597-- In the following example we can see that the fairly parallel composition
@@ -611,15 +656,15 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
611--
656--
612-- @
657-- @
613-- import "Streamly"
658-- import "Streamly"
614−-- import "Streamly.Prelude"
659+-- import qualified "Streamly.Prelude" as S
615-- import Control.Concurrent
660-- import Control.Concurrent
616--
661--
617-- main = do
662-- main = do
618−-- runStream $ parallely $ foldMap delay [1..10]
663+-- 'runStream' $ 'parallely' $ foldMap delay [1..10]
619−-- runStream $ 'foldWith' parallel (map delay [1..10])
664+-- 'runStream' $ 'foldWith' 'parallel' (map delay [1..10])
620−-- runStream $ 'foldMapWith' parallel delay [1..10]
665+-- 'runStream' $ 'foldMapWith' 'parallel' delay [1..10]
621−-- runStream $ 'forEachWith' parallel [1..10] delay
666+-- 'runStream' $ 'forEachWith' 'parallel' [1..10] delay
622−-- where delay n = once $ threadDelay (n * 1000000) >> print n
667+-- where delay n = S.'once' $ threadDelay (n * 1000000) >> print n
623-- @
668-- @
624
669
625-- $nesting
670-- $nesting
@@ -665,10 +710,10 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
665--
710--
666-- @
711-- @
667-- import "Streamly"
712-- import "Streamly"
668−-- import "Streamly.Prelude"
713+-- import qualified "Streamly.Prelude" as S
669--
714--
670-- main = 'runStream' $ do
715-- main = 'runStream' $ do
671−-- x <- 'fromFoldable' [3,2,1]
716+-- x <- S.'fromFoldable' [3,2,1]
672-- delay x
717-- delay x
673-- @
718-- @
674-- @
719-- @
@@ -687,8 +732,9 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
687--
732--
688-- @
733-- @
689-- import "Streamly"
734-- import "Streamly"
735+-- import qualified "Streamly.Prelude" as S
690--
736--
691−-- main = 'runStream' $ forever $ once getLine >>= once . putStrLn
737+-- main = 'runStream' $ forever $ S.once getLine >>= S.once . putStrLn
692-- @
738-- @
693--
739--
694-- When multiple streams are composed using this style they nest in a DFS
740-- When multiple streams are composed using this style they nest in a DFS
@@ -698,12 +744,12 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
698--
744--
699-- @
745-- @
700-- import "Streamly"
746-- import "Streamly"
701−-- import "Streamly.Prelude"
747+-- import qualified "Streamly.Prelude" as S
702--
748--
703-- main = 'runStream' $ do
749-- main = 'runStream' $ do
704−-- x <- 'fromFoldable' [1,2]
750+-- x <- S.'fromFoldable' [1,2]
705−-- y <- 'fromFoldable' [3,4]
751+-- y <- S.'fromFoldable' [3,4]
706−-- once $ putStrLn $ show (x, y)
752+-- S.'once' $ putStrLn $ show (x, y)
707-- @
753-- @
708-- @
754-- @
709-- (1,3)
755-- (1,3)
@@ -757,12 +803,12 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
757--
803--
758-- @
804-- @
759-- import "Streamly"
805-- import "Streamly"
760−-- import "Streamly.Prelude"
806+-- import qualified "Streamly.Prelude" as S
761--
807--
762-- main = 'runStream' . 'coparallely' $ do
808-- main = 'runStream' . 'coparallely' $ do
763−-- x <- 'fromFoldable' [1,2]
809+-- x <- S.'fromFoldable' [1,2]
764−-- y <- 'fromFoldable' [3,4]
810+-- y <- S.'fromFoldable' [3,4]
765−-- once $ putStrLn $ show (x, y)
811+-- S.'once' $ putStrLn $ show (x, y)
766-- @
812-- @
767-- @
813-- @
768-- (1,3)
814-- (1,3)
@@ -782,12 +828,12 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
782--
828--
783-- @
829-- @
784-- import "Streamly"
830-- import "Streamly"
785−-- import "Streamly.Prelude"
831+-- import qualified "Streamly.Prelude" as S
786--
832--
787-- main = 'runStream' . 'costreamly' $ do
833-- main = 'runStream' . 'costreamly' $ do
788−-- x <- 'fromFoldable' [1,2]
834+-- x <- S.'fromFoldable' [1,2]
789−-- y <- 'fromFoldable' [3,4]
835+-- y <- S.'fromFoldable' [3,4]
790−-- once $ putStrLn $ show (x, y)
836+-- S.once $ putStrLn $ show (x, y)
791-- @
837-- @
792-- @
838-- @
793-- (1,3)
839-- (1,3)
@@ -808,10 +854,10 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
808--
854--
809-- @
855-- @
810-- import "Streamly"
856-- import "Streamly"
811−-- import "Streamly.Prelude"
857+-- import qualified "Streamly.Prelude" as S
812--
858--
813-- main = 'runStream' . 'parallely' $ do
859-- main = 'runStream' . 'parallely' $ do
814−-- x <- 'fromFoldable' [3,2,1]
860+-- x <- S.'fromFoldable' [3,2,1]
815-- delay x
861-- delay x
816-- @
862-- @
817-- @
863-- @
@@ -829,20 +875,20 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
829--
875--
830-- @
876-- @
831-- import "Streamly"
877-- import "Streamly"
832−-- import "Streamly.Prelude"
878+-- import qualified "Streamly.Prelude" as S
833--
879--
834-- composed :: 'IsStream' t => t m a
880-- composed :: 'IsStream' t => t m a
835-- composed = do
881-- composed = do
836-- sz <- sizes
882-- sz <- sizes
837-- cl <- colors
883-- cl <- colors
838-- sh <- shapes
884-- sh <- shapes
839−-- once $ putStrLn $ show (sz, cl, sh)
885+-- S.'once' $ putStrLn $ show (sz, cl, sh)
840--
886--
841-- where
887-- where
842--
888--
843−-- sizes = 'fromFoldable' [1, 2, 3]
889+-- sizes = S.'fromFoldable' [1, 2, 3]
844−-- colors = 'fromFoldable' ["red", "green", "blue"]
890+-- colors = S.'fromFoldable' ["red", "green", "blue"]
845−-- shapes = 'fromFoldable' ["triangle", "square", "circle"]
891+-- shapes = S.'fromFoldable' ["triangle", "square", "circle"]
846-- @
892-- @
847--
893--
848-- Now we can interpret this in whatever way we want:
894-- Now we can interpret this in whatever way we want:
@@ -865,8 +911,9 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
865--
911--
866-- @
912-- @
867-- import "Streamly"
913-- import "Streamly"
914+-- import qualified "Streamly.Prelude" as S
868--
915--
869−-- main = ('toList' $ fmap show $ 'fromFoldable' [1..10]) >>= print
916+-- main = (S.'toList' $ fmap show $ S.'fromFoldable' [1..10]) >>= print
870-- @
917-- @
871--
918--
872-- Also see the 'mapM' and 'sequence' functions for mapping actions, in the
919-- Also see the 'mapM' and 'sequence' functions for mapping actions, in the
@@ -883,14 +930,14 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
883--
930--
884-- @
931-- @
885-- import "Streamly"
932-- import "Streamly"
886−-- import "Streamly.Prelude"
933+-- import qualified "Streamly.Prelude" as S
887-- import Control.Concurrent
934-- import Control.Concurrent
888--
935--
889-- s1 = d 1 <> d 2
936-- s1 = d 1 <> d 2
890-- s2 = d 3 <> d 4
937-- s2 = d 3 <> d 4
891-- d n = delay n >> return n
938-- d n = delay n >> return n
892--
939--
893−-- main = ('toList' . 'streamly' $ (,) \<$> s1 \<*> s2) >>= print
940+-- main = (S.'toList' . 'streamly' $ (,) \<$> s1 \<*> s2) >>= print
894-- @
941-- @
895-- @
942-- @
896-- ThreadId 36: Delay 1
943-- ThreadId 36: Delay 1
@@ -906,7 +953,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
906-- order but since it is serial it takes a total of 17 seconds:
953-- order but since it is serial it takes a total of 17 seconds:
907--
954--
908-- @
955-- @
909−-- main = ('toList' . 'costreamly' $ (,) \<$> s1 \<*> s2) >>= print
956+-- main = (S.'toList' . 'costreamly' $ (,) \<$> s1 \<*> s2) >>= print
910-- @
957-- @
911-- @
958-- @
912-- ThreadId 36: Delay 1
959-- ThreadId 36: Delay 1
@@ -922,7 +969,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
922-- of 10 seconds (1 + 2 + 3 + 4):
969-- of 10 seconds (1 + 2 + 3 + 4):
923--
970--
924-- @
971-- @
925−-- main = ('toList' . 'coparallely' $ (,) \<$> s1 \<*> s2) >>= print
972+-- main = (S.'toList' . 'coparallely' $ (,) \<$> s1 \<*> s2) >>= print
926-- @
973-- @
927-- @
974-- @
928-- ThreadId 34: Delay 1
975-- ThreadId 34: Delay 1
@@ -938,7 +985,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
938-- therefore takes a total of 10 seconds (1 + 2 + 3 + 4):
985-- therefore takes a total of 10 seconds (1 + 2 + 3 + 4):
939--
986--
940-- @
987-- @
941−-- main = ('toList' . 'parallely' $ (,) \<$> s1 \<*> s2) >>= print
988+-- main = (S.'toList' . 'parallely' $ (,) \<$> s1 \<*> s2) >>= print
942-- @
989-- @
943-- @
990-- @
944-- ThreadId 34: Delay 1
991-- ThreadId 34: Delay 1
@@ -987,14 +1034,14 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
987--
1034--
988-- @
1035-- @
989-- import "Streamly"
1036-- import "Streamly"
990−-- import "Streamly.Prelude"
1037+-- import qualified "Streamly.Prelude" as S
991-- import Control.Concurrent
1038-- import Control.Concurrent
992--
1039--
993-- d n = delay n >> return n
1040-- d n = delay n >> return n
994-- s1 = 'streamly' $ d 1 <> d 2
1041-- s1 = 'streamly' $ d 1 <> d 2
995-- s2 = 'streamly' $ d 3 <> d 4
1042-- s2 = 'streamly' $ d 3 <> d 4
996--
1043--
997−-- main = ('toList' . 'zipStreamly' $ (,) \<$> s1 \<*> s2) >>= print
1044+-- main = (S.'toList' . 'zipStreamly' $ (,) \<$> s1 \<*> s2) >>= print
998-- @
1045-- @
999--
1046--
1000-- This takes total 10 seconds to zip, which is (1 + 2 + 3 + 4) since
1047-- This takes total 10 seconds to zip, which is (1 + 2 + 3 + 4) since
@@ -1017,7 +1064,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
1017--
1064--
1018-- @
1065-- @
1019-- import "Streamly"
1066-- import "Streamly"
1020−-- import "Streamly.Prelude"
1067+-- import qualified "Streamly.Prelude" as S
1021-- import Control.Concurrent
1068-- import Control.Concurrent
1022-- import System.IO (stdout, hSetBuffering, BufferMode(LineBuffering))
1069-- import System.IO (stdout, hSetBuffering, BufferMode(LineBuffering))
1023--
1070--
@@ -1027,7 +1074,7 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
1027--
1074--
1028-- main = do
1075-- main = do
1029-- hSetBuffering stdout LineBuffering
1076-- hSetBuffering stdout LineBuffering
1030−-- ('toList' . 'zipParallely' $ (,) \<$> s1 \<*> s2) >>= print
1077+-- (S.'toList' . 'zipParallely' $ (,) \<$> s1 \<*> s2) >>= print
1031-- @
1078-- @
1032--
1079--
1033-- This takes 7 seconds to zip, which is max (1,3) + max (2,4) because 1 and 3
1080-- This takes 7 seconds to zip, which is max (1,3) + max (2,4) because 1 and 3
@@ -1059,11 +1106,11 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
1059--
1106--
1060-- @
1107-- @
1061-- import "Streamly"
1108-- import "Streamly"
1062−-- import "Streamly.Prelude" (toList)
1109+-- import qualified "Streamly.Prelude" as S
1063-- import Data.List (sum)
1110-- import Data.List (sum)
1064--
1111--
1065-- main = do
1112-- main = do
1066−-- z \<- 'toList'
1113+-- z \<- S.'toList'
1067-- $ 'streamly' -- Serial monadic processing (sqrt below)
1114-- $ 'streamly' -- Serial monadic processing (sqrt below)
1068-- $ do
1115-- $ do
1069-- x2 \<- 'forEachWith' 'coparallel' [1..100] $ -- Concurrent @"for"@ loop
1116-- x2 \<- 'forEachWith' 'coparallel' [1..100] $ -- Concurrent @"for"@ loop
@@ -1252,10 +1299,22 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
1252
1299
1253-- $comparison
1300-- $comparison
1254--
1301--
1255−-- Streamly unifies non-determinism, streaming, concurrency and FRP
1302+-- List transformers and logic programming monads also provide a product style
1256−-- functionality that is otherwise covered by several disparate packages, and
1303+-- composition similar to streamly, however streamly generalizes it with the
1257−-- it does that with a surprisingly concise API. Here is a list of popular and
1304+-- time dimension; allowing streams to be composed in an asynchronous and
1258−-- well-known packages in all these areas:
1305+-- concurrent fashion in many different ways. It also provides multiple
1306+-- alternative ways of composing streams e.g. serial, interleaved or
1307+-- concurrent.
1308+--
1309+-- This seemingly simple addition of asynchronicity and concurrency to product
1310+-- style streaming composition unifies a number of disparate abstractions into
1311+-- one powerful, concise and elegant abstraction. A wide variety of
1312+-- programming problems can be solved elegantly with this abstraction. In
1313+-- particular, it unifies three major programming domains namely
1314+-- non-deterministic (logic) programming, concurrent programming and functional
1315+-- reactive programming. In other words, you can do everything with this one
1316+-- abstraction that you could do with the popular libraries listed under these
1317+-- categories in the list below.
1259--
1318--
1260-- @
1319-- @
1261-- +-----------------+----------------+
1320-- +-----------------+----------------+
@@ -1318,3 +1377,10 @@ import Control.Monad.Trans.Class (MonadTrans (lift))
1318-- similar to @dunai@. dunai was designed from a FRP perspective and streamly
1377-- similar to @dunai@. dunai was designed from a FRP perspective and streamly
1319-- was originally designed from a concurrency perspective. However, both have
1378-- was originally designed from a concurrency perspective. However, both have
1320-- similarity at the core.
1379-- similarity at the core.
1380+
1381+-- $furtherReading
1382+--
1383+-- * Read the documentation of "Streamly" module
1384+-- * Read the documentation of "Streamly.Prelude" module
1385+-- * See the examples in the "examples" directory of the package
1386+-- * See the tests in the "test" directory of the package
streamly.cabal+42−15
@@ -2,22 +2,49 @@ name: streamly
2version: 0.1.2
2version: 0.1.2
3synopsis: Beautiful Streaming, Concurrent and Reactive Composition
3synopsis: Beautiful Streaming, Concurrent and Reactive Composition
4description:
4description:
5− Streamly is a monad transformer unifying non-determinism
5+ Streamly, short for streaming concurrently, is a simple yet powerful
6− (<https://hackage.haskell.org/package/list-t list-t>\/<https://hackage.haskell.org/package/logict logict>),
6+ streaming library with concurrent merging and concurrent nested looping
7− concurrency (<https://hackage.haskell.org/package/async async>),
7+ support. A stream is just like a list except that it is a list of monadic
8− streaming (<https://hackage.haskell.org/package/conduit conduit>\/<https://hackage.haskell.org/package/pipes pipes>),
8+ actions rather than pure values. Streamly streams can be generated,
9− and FRP (<https://hackage.haskell.org/package/Yampa Yampa>\/<https://hackage.haskell.org/package/reflex reflex>)
9+ consumed, combined, or transformed serially or concurrently. We can loop over
10− functionality in a concise and intuitive API.
10+ a stream serially or concurrently. We can also have serial or concurrent
11− High level concurrency makes concurrent applications almost indistinguishable
11+ nesting of loops. For those familiar with list transformer concept streamly
12− from non-concurrent ones. By changing a single combinator you can control
12+ is a concurrent list transformer. Streamly uses standard composition
13− whether the code runs serially or concurrently. It naturally integrates
13+ abstractions. Concurrent composition is just the same as serial composition
14− concurrency with streaming rather than adding it as an afterthought.
14+ except that we use a simple combinator to request a concurrent composition
15− Moreover, it interworks with the popular streaming libraries.
15+ instead of serial. The programmer does not have to be aware of threads,
16+ locking or synchronization to write scalable concurrent programs.
16 .
17 .
18+ Streamly provides functionality that is equivalent to streaming libraries
19+ like <https://hackage.haskell.org/package/pipes pipes> and
20+ <https://hackage.haskell.org/package/conduit conduit> but with a simple list
21+ like API. The streaming API of streamly is close to the monadic streams API
22+ of the <https://hackage.haskell.org/package/vector vector> package and
23+ similar in concept to the
24+ <https://hackage.haskell.org/package/streaming streaming> package. In
25+ addition to the streaming functionality, streamly subsumes the functionality
26+ of list transformer libraries like @pipes@ or
27+ <https://hackage.haskell.org/package/list-t list-t> and also the logic
28+ programming library <https://hackage.haskell.org/package/logict logict>. On
29+ the concurrency side, it subsumes the functionality of the
30+ <https://hackage.haskell.org/package/async async> package. Because it
31+ supports streaming with concurrency we can write FRP applications similar in
32+ concept to <https://hackage.haskell.org/package/Yampa Yampa> or
33+ <https://hackage.haskell.org/package/reflex reflex>.
34+ .
35+ Streamly has excellent performance, see
36+ <https://github.com/composewell/streaming-benchmarks streaming-benchmarks>
37+ for a comparison of popular streaming libraries on micro-benchmarks. For
38+ file IO, currently the library provides only one API to stream the lines in
39+ the file as Strings. Future versions will provide better streaming file IO
40+ options. Streamly interworks with the popular streaming libraries, see the
41+ interworking section in "Streamly.Tutorial".
42+ .
43+ Where to find more information:
44+ .
45+ * @README@ shipped with the package for a quick overview
17− See the README for an overview and the haddock documentation for full
46+ * "Streamly.Tutorial" module in the haddock documentation for a detailed introduction
18− reference. It is recommended to read the comprehensive tutorial module
47+ * @examples@ directory in the package for some simple practical examples
19− "Streamly.Tutorial" first. Also see "Streamly.Examples" for some working
20− examples.
21
48
22homepage: https://github.com/composewell/streamly
49homepage: https://github.com/composewell/streamly
23bug-reports: https://github.com/composewell/streamly/issues
50bug-reports: https://github.com/composewell/streamly/issues