@@ -361,42 +361,49 @@ documentation.
364−The `Streamly.Memory.Array` module provides mutable arrays. Arrays are the
364+The `Streamly.Memory.Array` module provides immutable arrays. Arrays are the
365−computing duals of streams. Streams are good at sequential access, immutable
365+computing duals of streams. Streams are good at sequential access and immutable
366−transformations of in-transit data whereas arrays are good at random access,
366+transformations of in-transit data whereas arrays are good at random access and
367in-place transformations of buffered data. Unlike streams which are potentially
367in-place transformations of buffered data. Unlike streams which are potentially
368infinite, arrays are necessarily finite. Arrays can be used as an efficient
368infinite, arrays are necessarily finite. Arrays can be used as an efficient
369interface between streams and external storage systems like memory, files and
369interface between streams and external storage systems like memory, files and
370network. Streams and arrays complete each other to provide a general purpose
370network. Streams and arrays complete each other to provide a general purpose
371computing system. The design of streamly as a general purpose computing
371computing system. The design of streamly as a general purpose computing
372framework is centered around these two fundamental aspects of computing and
372framework is centered around these two fundamental aspects of computing and
373−storage.
373+storage.
375−Arrays in streamly use pinned memory outside GC and therefore avoid any GC
375+`Streamly.Memory.Array` uses pinned memory outside GC and therefore avoid any
376−overhead for the storage in arrays. `ByteString` data type from the
376+GC overhead for the storage in arrays. Streamly allows efficient
377−`bytestring` package and the `Text` data type from the `text` package are just
377+transformations over arrays using streams. It uses arrays to transfer data to
378−special cases of arrays. `ByteString` is equivalent to `Array Word8` and `Text`
378+and from the operating system and to store data in memory.
379−is equivalent to a `utf16` encoded `Array Word8`. All the bytestring and text
380−operations can be performed on arrays with equivalent or better performance by
381−converting them to and from streams.
385−`Streamly.Data.Fold` module provides composable left folds. A `Fold` is a consumer
382+Folds are consumers of streams. `Streamly.Data.Fold` module provides a `Fold`
386−of a stream. Folds can be used to fold a stream. Folds can be composed in many
383+type that represents a `foldl'`. Such folds can be efficiently composed
384+allowing the compiler to perform stream fusion and therefore implement high
385+performance combinators for consuming streams. A stream can be distributed to
387−ways, a stream can be distributed to multiple folds, or it can be partitioned
386+multiple folds, or it can be partitioned across multiple folds, or
388−across multiple folds, or demultiplexed over multiple folds, or unzipped to two
387+demultiplexed over multiple folds, or unzipped to two folds. We can also use
389−folds. We can also use folds to fold segments of stream generating a stream of
388+folds to fold segments of stream generating a stream of the folded results.
392If you are familiar with the `foldl` library, these are the same composable
390If you are familiar with the `foldl` library, these are the same composable
393left folds but simpler and better integrated with streamly, and with many more
391left folds but simpler and better integrated with streamly, and with many more
394powerful ways of composing and applying them.
392powerful ways of composing and applying them.
396+Unfolds are duals of folds. Folds help us compose consumers of streams
397+efficiently and unfolds help us compose producers of streams efficiently.
398+`Streamly.Data.Unfold` provides an `Unfold` type that represents an `unfoldr`
399+or a stream generator. Such generators can be combined together efficiently
400+allowing the compiler to perform stream fusion and implement high performance
401+stream merging combinators.
396## File IO
403## File IO
398The following code snippets implement some common Unix command line utilities
405The following code snippets implement some common Unix command line utilities
399−using streamly. You can compile these with `ghc -O2 -fspec-constr-recursive=10
406+using streamly. You can compile these with `ghc -O2 -fspec-constr-recursive=16
400-fmax-worker-args=16` and compare the performance with regular GNU coreutils
407-fmax-worker-args=16` and compare the performance with regular GNU coreutils
401available on your system. Though many of these are not most optimal solutions
408available on your system. Though many of these are not most optimal solutions
402to keep them short and elegant. Source file
409to keep them short and elegant. Source file
@@ -431,14 +438,14 @@ withArg2 f = do
433``` haskell
440``` haskell
434−cat = S.fold (FH.writeArrays stdout) . S.unfold FH.readArrays
441+cat = S.fold (FH.writeChunks stdout) . S.unfold FH.readChunks
435main = withArg cat
442main = withArg cat
440``` haskell
447``` haskell
441−cp src dst = S.fold (FH.writeArrays dst) $ S.unfold FH.readArrays src
448+cp src dst = S.fold (FH.writeChunks dst) $ S.unfold FH.readChunks src
442main = withArg2 cp
449main = withArg2 cp