on master help sign in
Add intro to arrays and folds
commit3fcfe4afce04280e5695013977b645686650e531 authorHarendra Kumar <harendra.kumar@gmail.com> dateTue 30 Jul 2019, 03:45 (7 years ago) parente7bc7ea
1 file changed+34
README.md+34
README.md+34
@@ -361,6 +361,40 @@ rate. Rate control works precisely even at throughputs as high as millions of
361361yields per second. For more sophisticated rate control see the haddock
362362documentation.
363363
364+## Arrays
365+
366+The `Streamly.Mem.Array` module provides mutable arrays. Arrays are the
367+computing duals of streams. Streams are good at sequential access, immutable
368+transformations of in-transit data whereas arrays are good at random access,
369+in-place transformations of buffered data. Unlike streams which are potentially
370+infinite, arrays are necessarily finite. Arrays can be used as an efficient
371+interface between streams and external storage systems like memory, files and
372+network. Streams and arrays complete each other to provide a general purpose
373+computing system. The design of streamly as a general purpose computing
374+framework is centered around these two fundamental aspects of computing and
375+storage.
376+
377+Arrays in streamly use pinned memory outside GC and therefore avoid any GC
378+overhead for the storage in arrays. `ByteString` data type from the
379+`bytestring` package and the `Text` data type from the `text` package are just
380+special cases of arrays. `ByteString` is equivalent to `Array Word8` and `Text`
381+is equivalent to a `utf16` encoded `Array Word8`. All the bytestring and text
382+operations can be performed on arrays with equivalent or better performance by
383+converting them to and from streams.
384+
385+## Folds
386+
387+`Streamly.Fold` module provides composable left folds. A `Fold` is a consumer
388+of a stream. Folds can be used to fold a stream. Folds can be composed in many
389+ways, a stream can be distributed to multiple folds, or it can be partitioned
390+across multiple folds, or demultiplexed over multiple folds, or unzipped to two
391+folds. We can also use folds to fold segments of stream generating a stream of
392+the folded results.
393+
394+If you are familiar with the `foldl` library, these are the same composable
395+left folds but simpler and better integrated with streamly, and with many more
396+powerful ways of composing and applying them.
397+
364398## File IO
365399
366400The following code snippet implements some common Unix command line utilities