This is the initial version of stream parsing. It implements a "Parse"
type, some parsers based on that type, applicative composition, operations to
run parsers (parse) and to run parsers over chunks of a stream (parseChunks).
Parsers are just an extension of Folds in Streamly.Data.Fold. Parsers just add
backtracking and failure capabilities to folds.
Operations like splitOn to split a stream on a predicate can now be expressed
using a parser applied repeatedly on a stream. For example, line splitting can
be done using parsers. Parsers are as fast as fastest custom code for
line splitting.
* Remove lib/B/Group.hs
* The idea: Have all the groups in Prelude and
export only the groups, that way, all the functions
aren't compiled multiple times (without and with
inlining).
* move common modules to lib directory. This is required because otherwise
cabal/ghc would try to use them directly and emit warnings about
missing-modules.
* move the prelude benchmark executables to Streamly/Benchmark/Prelude
to be consistent with the organization of other modules.
So that we can limit the heap to a reasonable value in tests.
Also move the chunksOf benchmarks out of the DEVBUILD flag as they do not
depend on a well formed text file.
The predicate provides more power to the combinator by allowing you to count
only certain elements in the stream and filter out others. This is efficient
compared to an async tap followed by a filter and fold as here filtering is
done on the source.
Do not use "tail" in zip benchmarks otherwise the tail perf can impact these
benchmarks. Also it makes them consistent with other such benchmarks.
Use half the stream size for each side of zip so that we effectively
create a stream of size similar to other linear benchmarks.
The "after", "finally" and "bracket" combinators did not run the "cleanup"
handler in case the stream is lazily partially evaluated e.g. using the lazy
right fold (foldrM), "Streamly.Prelude.head" is an example of such a fold.
Since we run the cleanup action when the stream Stops, the action won't be run
if the stream is not fully drained.
In the new implementation, we use a GC hook to run the cleanup action in case
the stream got garbage collected even before finishing. This will take care of
the lazy right fold cases mentioned above.
* We are testing two extreme cases, add a middle case as well where the outer
and inner streams are of equal size.
* Enable some pure benchmarks as well
* Separate the zip benchmarks in a separate group as they are scalable
(memory consumption does not increase with stream size) and parallel
benchmarks are not scalable.
* Document the precise behavior, some changes were made to the earlier behavior
* Make some changes to implementation according to (newly) documented behavior
* TakeByTime: perform the time check before generating the element so that we
do not drop an element after generation.
* TakeByTime now yields at least one element if the duration is non-zero
* dropByTime does not check the time after the drop duration is over
* Add inspection tests
* make the tests for shorter duration, earlier tests took too long
* pollCounts to poll the element count in another thread
* delayPost to introduce a delay in polling
* rollingMap to compute diff of successive elements
These combinators can be used to compute and report the element
processing rate in a stream.
* Instead of using hard coded numbers scale them based on the stream size.
* Add concatMapWith benchmarks for concurrent streams
* Add a linear-async-long benchmark that works on streams of 10 million
elements
We need large streams to detect memory leak issues. Specifically, we
could not figure out the concatMapWith memory leak issue without a
stream of at least 10s of millions of elements.
For long benchmarks we use gauge defaults so that we do not run many
iterations/samples.
Keep related benchmark code together and try to keep the order in sync with
the Streamly.Prelude module.
This is purely a reorganization of code by moving related code together
and adding comments for subsections of code. There is no functionality
change at all.
The change of StreamK/Type.serial impl makes this test fail because of IsStream
dictionary still being around. However, there is not much difference in the
actual benchmark results. And, serial impl change has other much better results
in other benchmarks.
