git.iko.soy/streamly
on d29030f help sign in
30c5a56
swap
0 ahead, 373 behind
299 files changed+23459−25638
since splitsnapshots
unifiedsplit
.circleci/config.yml+277−73 .github/workflows/haskell.yml+91−84 .github/workflows/regression-check.yml+3−3 .hlint.ignore+2−7 benchmark/NanoBenchmarks.hs+1−1 benchmark/Streamly/Benchmark/Data/Array/CommonImports.hs+1−1 benchmark/Streamly/Benchmark/Data/MutArray.hs → benchmark/Streamly/Benchmark/Data/Array/Mut.hs+4−4 benchmark/Streamly/Benchmark/Data/Array/Stream.hs+4−4 benchmark/Streamly/Benchmark/Data/Fold.hs+6−29 benchmark/Streamly/Benchmark/Data/Fold/Window.hs+40−40 benchmark/Streamly/Benchmark/Data/Parser.hs+10−10 benchmark/Streamly/Benchmark/Data/ParserK.hs+37−90 benchmark/Streamly/Benchmark/Data/Ring/Unboxed.hs+4−3 benchmark/Streamly/Benchmark/Data/Serialize.hs−591 benchmark/Streamly/Benchmark/Data/Serialize/RecCompatible.hs−24 benchmark/Streamly/Benchmark/Data/Serialize/RecNonCompatible.hs−21 benchmark/Streamly/Benchmark/Data/Serialize/TH.hs−76 benchmark/Streamly/Benchmark/Data/Stream/Common.hs+1−3 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentCommon.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentEager.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentInterleaved.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentOrdered.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Eliminate.hs+1−3 benchmark/Streamly/Benchmark/Data/Stream/Exceptions.hs+4−14 benchmark/Streamly/Benchmark/Data/Stream/Expand.hs+1−9 benchmark/Streamly/Benchmark/Data/Stream/Generate.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Lift.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Reduce.hs+2−2 benchmark/Streamly/Benchmark/Data/Stream/Split.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/StreamD.hs+2−4 benchmark/Streamly/Benchmark/Data/Stream/StreamKAlt.hs → benchmark/Streamly/Benchmark/Data/Stream/StreamDK.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs+3−4 benchmark/Streamly/Benchmark/Data/Stream/ToStreamK.hs+4−3 benchmark/Streamly/Benchmark/Data/Stream/Transform.hs+2−2 benchmark/Streamly/Benchmark/Data/Unfold.hs+2−11 benchmark/Streamly/Benchmark/FileSystem/Handle.hs+1−1 benchmark/Streamly/Benchmark/FileSystem/Handle/Read.hs+10−9 benchmark/Streamly/Benchmark/FileSystem/Handle/ReadWrite.hs+12−11 benchmark/Streamly/Benchmark/Prelude/Adaptive.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Ahead.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Async.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Concurrent.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Merge.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Parallel.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Rate.hs+1−1 benchmark/Streamly/Benchmark/Prelude/WAsync.hs+1−1 benchmark/Streamly/Benchmark/Prelude/WSerial.hs+1−1 benchmark/Streamly/Benchmark/Prelude/ZipAsync.hs+1−1 benchmark/Streamly/Benchmark/Prelude/ZipSerial.hs+1−1 benchmark/Streamly/Benchmark/Unicode/Char.hs+1−1 benchmark/Streamly/Benchmark/Unicode/Parser.hs+20−37 benchmark/Streamly/Benchmark/Unicode/Stream.hs+8−26 benchmark/Streamly/Benchmark/Unicode/Utf8.hs+1−1 benchmark/bench-runner/Main.hs+6−34 benchmark/lib/Streamly/Benchmark/Common.hs+1−1 benchmark/lib/Streamly/Benchmark/Common/Handle.hs+1−3 benchmark/lib/Streamly/Benchmark/Prelude.hs+1−1 benchmark/streamly-benchmarks.cabal+42−71 configure+9−9 configure.ac+1−1 core/LICENSE−41 core/configure+9−9 core/configure.ac+1−1 core/docs/ApiChangelogs/0.1.0-0.2.0.txt−1471 core/docs/ApiChangelogs/0.2.0-0.2.2.txt−306 core/docs/Changelog.md+3−64 core/src/DocTestDataArray.hs−8 core/src/DocTestDataFold.hs+1 core/src/DocTestDataMutArray.hs+1−1 core/src/DocTestDataMutArrayGeneric.hs+1−1 core/src/DocTestDataParserK.hs−15 core/src/DocTestDataStream.hs+1−2 core/src/DocTestFileSystemHandle.hs−15 core/src/DocTestUnicodeParser.hs−9 core/src/DocTestUnicodeStream.hs−12 core/src/DocTestUnicodeString.hs−5 core/src/Streamly/Console/Stdio.hs+27−1 core/src/Streamly/Data/Array.hs+61−58 core/src/Streamly/Data/Array/Generic.hs+13−16 core/src/Streamly/Data/Fold.hs+119−150 core/src/Streamly/Data/MutArray.hs+12−28 core/src/Streamly/Data/MutArray/Generic.hs+4−9 core/src/Streamly/Data/MutByteArray.hs−106 core/src/Streamly/Data/Parser.hs+60−130 core/src/Streamly/Data/ParserK.hs+10−105 core/src/Streamly/Data/Stream.hs+109−221 core/src/Streamly/Data/Stream/StreamDK.hs+505 core/src/Streamly/Data/Stream/Zip.hs+16 core/src/Streamly/Data/StreamK.hs+19−49 core/src/Streamly/Data/Unfold.hs−2 core/src/Streamly/FileSystem/Handle.hs+29−15 core/src/Streamly/Internal/Data/Array.hs+164−611 core/src/Streamly/Internal/Data/Array/Generic.hs+12−50 core/src/Streamly/Internal/Data/MutArray/Generic.hs → core/src/Streamly/Internal/Data/Array/Generic/Mut/Type.hs+54−177 core/src/Streamly/Internal/Data/Array/Mut.hs+86 core/src/Streamly/Internal/Data/Array/Mut/Stream.hs+323 core/src/Streamly/Internal/Data/MutArray/Type.hs → core/src/Streamly/Internal/Data/Array/Mut/Type.hs+482−1394 core/src/Streamly/Internal/Data/Array/Type.hs+122−570 core/src/Streamly/Internal/Data/Builder.hs+9−11 core/src/Streamly/Internal/Data/Fold.hs+2574−9 core/src/Streamly/Internal/Data/Fold/Chunked.hs+18−20 core/src/Streamly/Internal/Data/Fold/Combinators.hs−2452 core/src/Streamly/Internal/Data/Fold/Container.hs+32−75 core/src/Streamly/Internal/Data/Fold/Step.hs+2−2 core/src/Streamly/Internal/Data/Fold/Tee.hs−2 core/src/Streamly/Internal/Data/Fold/Type.hs+120−344 core/src/Streamly/Internal/Data/Fold/Window.hs+53−59 core/src/Streamly/Internal/Data/IOFinalizer.hs+2−12 core/src/Streamly/Internal/Data/IORef/Unboxed.hs+20−23 core/src/Streamly/Internal/Data/IsMap.hs−8 core/src/Streamly/Internal/Data/List.hs+15−81 core/src/Streamly/Internal/Data/MutArray.hs−207 core/src/Streamly/Internal/Data/MutArray/Stream.hs−145 core/src/Streamly/Internal/Data/MutByteArray.hs−251 core/src/Streamly/Internal/Data/MutByteArray/Type.hs−321 core/src/Streamly/Internal/Data/Parser.hs+143−135 core/src/Streamly/Internal/Data/Parser/Tee.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Tee.hs+1−1 core/src/Streamly/Internal/Data/Parser/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Type.hs+71−94 core/src/Streamly/Internal/Data/Parser/ParserDK.hs+1544 core/src/Streamly/Internal/Data/ParserK/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserK/Type.hs+84−291 core/src/Streamly/Internal/Data/ParserK.hs−16 core/src/Streamly/Internal/Data/Pipe.hs+8−5 core/src/Streamly/Internal/Data/Producer.hs+13−6 core/src/Streamly/Internal/Data/Producer/Source.hs+1−1 core/src/Streamly/Internal/Data/Producer/Type.hs+1−1 core/src/Streamly/Internal/Data/Refold/Type.hs+1−1 core/src/Streamly/Internal/Data/Ring.hs+134−589 core/src/Streamly/Internal/Data/Ring/Generic.hs−185 core/src/Streamly/Internal/Data/Ring/Unboxed.hs+616 core/src/Streamly/Internal/Data/Serialize.hs+157 core/src/Streamly/Internal/Data/Serialize/TH.hs−530 core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hs−475 core/src/Streamly/Internal/Data/Serialize/TH/Common.hs−70 core/src/Streamly/Internal/Data/Serialize/TH/RecHeader.hs−411 core/src/Streamly/Internal/Data/Serialize/Type.hs−317 core/src/Streamly/Internal/Data/Stream.hs+20−20 core/src/Streamly/Internal/Data/Stream/Bottom.hs+670 core/src/Streamly/Internal/Data/Array/Stream.hs → core/src/Streamly/Internal/Data/Stream/Chunked.hs+577−74 src/Streamly/Internal/Data/Stream/Common.hs → core/src/Streamly/Internal/Data/Stream/Common.hs+3−3 core/src/Streamly/Internal/Data/Stream/Cross.hs+143 core/src/Streamly/Internal/Data/Stream/Eliminate.hs+227−699 core/src/Streamly/Internal/Data/Stream/Enumerate.hs+560 core/src/Streamly/Internal/Data/Stream/Exception.hs+90−369 core/src/Streamly/Internal/Data/Stream/Expand.hs+893 core/src/Streamly/Internal/Data/Stream/Generate.hs+217−948 core/src/Streamly/Internal/Data/Stream/Lift.hs+36−70 core/src/Streamly/Internal/Data/Stream/Reduce.hs+444 core/src/Streamly/Internal/Data/Stream/Container.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Container.hs+58−66 core/src/Streamly/Internal/Data/Stream/StreamD/Eliminate.hs+843 core/src/Streamly/Internal/Data/Stream/StreamD/Exception.hs+479 core/src/Streamly/Internal/Data/Stream/StreamD/Generate.hs+1190 core/src/Streamly/Internal/Data/Stream/StreamD/Lift.hs+129 core/src/Streamly/Internal/Data/Stream/Nesting.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Nesting.hs+228−115 core/src/Streamly/Internal/Data/Stream/Step.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Step.hs+2−2 core/src/Streamly/Internal/Data/Stream/StreamD/Top.hs+353 core/src/Streamly/Internal/Data/Stream/StreamD/Transform.hs+1945 core/src/Streamly/Internal/Data/Stream/StreamD/Transformer.hs+182 core/src/Streamly/Internal/Data/Stream/StreamD/Type.hs+2085 core/src/Streamly/Internal/Data/Stream/StreamDK.hs+52 core/src/Streamly/Internal/Data/StreamK/Alt.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Alt.hs+1−1 core/src/Streamly/Internal/Data/StreamK/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Transformer.hs+3−3 core/src/Streamly/Internal/Data/StreamK/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Type.hs+41−95 core/src/Streamly/Internal/Data/Stream/Top.hs−480 core/src/Streamly/Internal/Data/Stream/Transform.hs+282−1197 core/src/Streamly/Internal/Data/Stream/Transformer.hs+43−91 core/src/Streamly/Internal/Data/Stream/Type.hs+229−1958 core/src/Streamly/Internal/Data/Stream/Zip.hs+91 core/src/Streamly/Internal/Data/StreamK.hs+127−435 core/src/Streamly/Internal/Data/Time/Clock.hs+3−3 core/src/Streamly/Internal/Data/Time/Units.hs+3 core/src/Streamly/Internal/Data/Unbox.hs+258−313 core/src/Streamly/Internal/Data/Unbox/TH.hs−498 core/src/Streamly/Internal/Data/Unfold.hs+57−19 core/src/Streamly/Internal/Data/Unfold/Enumeration.hs+1 core/src/Streamly/Internal/Data/Unfold/Type.hs+8−4 core/src/Streamly/Internal/FileSystem/Dir.hs+2−2 core/src/Streamly/Internal/FileSystem/File.hs+16−14 core/src/Streamly/Internal/FileSystem/Handle.hs+48−39 core/src/Streamly/Internal/FileSystem/Path.hs−515 core/src/Streamly/Internal/Data/Binary/Parser.hs → core/src/Streamly/Internal/Serialize/FromBytes.hs+4−10 core/src/Streamly/Internal/Data/Binary/Stream.hs → core/src/Streamly/Internal/Serialize/ToBytes.hs+2−12 core/src/Streamly/Internal/Unicode/Array.hs−6 core/src/Streamly/Internal/Unicode/Parser.hs+9−160 core/src/Streamly/Internal/Unicode/Stream.hs+25−53 core/src/Streamly/Internal/Unicode/String.hs+12−17 core/src/Streamly/Unicode/Parser.hs+2−11 core/src/Streamly/Unicode/Stream.hs−9 core/src/Streamly/Unicode/String.hs+1−11 core/streamly-core.cabal+94−150 docs/Developer/module-organization.md+1−1 docs/Developer/optimization-guidelines.md−12 docs/Developer/paths.rst+2−25 docs/User/HowTo/Compiling.md+3−5 docs/User/Project/ApiChangelogs/0.9.0-0.10.0.txt−424 docs/User/Project/CONTRIBUTORS.md−8 docs/User/Project/Changelog.md+9−26 docs/User/Project/README.md+2−9 docs/User/Project/Upgrading-0.8-to-0.9.md → docs/User/Project/Upgrading.md+9−87 docs/User/Tutorials/quick-overview.md+14−10 docs/streamly-docs.cabal+1−1 hie.yaml+6−6 src/Streamly.hs+624 src/Streamly/Data/Fold/Prelude.hs+14−32 src/Streamly/Data/Fold/Tee.hs+1−1 src/Streamly/Data/Stream/Concurrent.hs+194 src/Streamly/Data/Stream/Exception.hs+24 src/Streamly/Data/Stream/MkType.hs+10−41 src/Streamly/Data/Stream/Prelude.hs+15−217 src/Streamly/Data/Stream/Time.hs+37 src/Streamly/Data/Unicode/Stream.hs+99 src/Streamly/Internal/Data/Array/ArrayMacros.h+40 src/Streamly/Internal/Data/Channel.hs−19 src/Streamly/Internal/Data/Fold/Async.hs+3−6 src/Streamly/Internal/Data/Fold/Concurrent.hs+44−1 src/Streamly/Internal/Data/Fold/Concurrent/Channel.hs+10−29 src/Streamly/Internal/Data/Fold/Concurrent/Channel/Type.hs+4−4 src/Streamly/Internal/Data/Fold/Prelude.hs−19 src/Streamly/Internal/Data/Fold/SVar.hs+7−8 src/Streamly/Internal/Data/IsMap/HashMap.hs−6 src/Streamly/Internal/Data/SVar.hs−1 src/Streamly/Internal/Data/SmallArray.hs+3−3 src/Streamly/Internal/Data/Stream/Ahead.hs+3−3 src/Streamly/Internal/Data/Stream/Async.hs+3−3 src/Streamly/Internal/Data/Channel/Dispatcher.hs → src/Streamly/Internal/Data/Stream/Channel/Dispatcher.hs+3−3 src/Streamly/Internal/Data/Channel/Types.hs → src/Streamly/Internal/Data/Stream/Channel/Types.hs+37−12 src/Streamly/Internal/Data/Channel/Worker.hs → src/Streamly/Internal/Data/Stream/Channel/Worker.hs+3−3 src/Streamly/Internal/Data/Stream/Concurrent.hs+42−51 src/Streamly/Internal/Data/Stream/Concurrent/Channel.hs+14−17 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Append.hs+11−11 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Consumer.hs+2−2 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Dispatcher.hs+3−3 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Interleave.hs+9−9 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Operations.hs+4−4 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Type.hs+4−4 src/Streamly/Internal/Data/Stream/Lifted.hs → src/Streamly/Internal/Data/Stream/Exception/Lifted.hs+10−14 src/Streamly/Internal/Data/Stream/IsStream.hs−6 src/Streamly/Internal/Data/Stream/IsStream/Combinators.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Common.hs+6−6 src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs+2−2 src/Streamly/Internal/Data/Stream/IsStream/Exception.hs+4−4 src/Streamly/Internal/Data/Stream/IsStream/Expand.hs+3−7 src/Streamly/Internal/Data/Stream/IsStream/Generate.hs+6−6 src/Streamly/Internal/Data/Stream/IsStream/Reduce.hs+7−7 src/Streamly/Internal/Data/Stream/IsStream/Top.hs+3−2 src/Streamly/Internal/Data/Stream/IsStream/Transform.hs+3−3 src/Streamly/Internal/Data/Stream/IsStream/Type.hs+3−3 src/Streamly/Internal/Data/Stream/MkType.hs+34−33 src/Streamly/Internal/Data/Stream/Parallel.hs+3−3 src/Streamly/Internal/Data/Stream/Prelude.hs−19 src/Streamly/Internal/Data/Stream/SVar.hs−18 src/Streamly/Internal/Data/Stream/SVar/Eliminate.hs+3−3 src/Streamly/Internal/Data/Stream/SVar/Generate.hs+2−4 src/Streamly/Internal/Data/Stream/Serial.hs+5−8 src/Streamly/Internal/Data/Stream/Time.hs+4−4 src/Streamly/Internal/Data/Stream/Zip.hs+2−41 src/Streamly/Internal/Data/Stream/Zip/Concurrent.hs+50 src/Streamly/Internal/Data/Stream/ZipAsync.hs+2−3 src/Streamly/Internal/Data/Unfold/Exception.hs+3−1 src/Streamly/Internal/Data/Unfold/Prelude.hs−17 src/Streamly/Internal/Data/Unfold/SVar.hs+2−3 src/Streamly/Internal/FileSystem/Event.hs+2−5 src/Streamly/Internal/FileSystem/Event/Darwin.hs+5−5 src/Streamly/Internal/FileSystem/Event/Linux.hs+9−15 src/Streamly/Internal/FileSystem/Event/Windows.hs+8−11 src/Streamly/Internal/FileSystem/FD.hs+13−14 src/Streamly/Internal/Network/Inet/TCP.hs+48−83 src/Streamly/Internal/Network/Socket.hs+33−34 src/Streamly/Internal/System/IOVec.hs+6−6 src/Streamly/Internal/Unicode/Char.hs+7−9 src/Streamly/Memory/Array.hs+74 src/Streamly/Network/Inet/TCP.hs+30−41 src/Streamly/Network/Socket.hs+30−16 streamly.cabal+78−104 targets/Targets.hs+4−10 test/Streamly/Test/Data/Array.hs+10−9 test/Streamly/Test/Data/MutArray.hs → test/Streamly/Test/Data/Array/Mut.hs+5−5 test/Streamly/Test/Data/Array/Stream.hs+1−1 test/Streamly/Test/Data/Fold.hs+8−8 test/Streamly/Test/Data/Fold/Window.hs+9−8 test/Streamly/Test/Data/Parser.hs+114−263 test/Streamly/Test/Data/ParserK.hs → test/Streamly/Test/Data/Parser/ParserD.hs+10−14 test/Streamly/Test/Data/Ring/Unboxed.hs+2−2 test/Streamly/Test/Data/Serialize.hs−306 test/Streamly/Test/Data/Serialize/CompatV0.hs−46 test/Streamly/Test/Data/Serialize/CompatV1.hs−48 test/Streamly/Test/Data/Serialize/TH.hs−82 test/Streamly/Test/Data/Stream/Concurrent.hs+1−1 test/Streamly/Test/Data/Unbox.hs+66−164 test/Streamly/Test/Data/Unbox/TH.hs−146 test/Streamly/Test/FileSystem/Event/Common.hs+1−1 test/Streamly/Test/FileSystem/Handle.hs+2−2 test/Streamly/Test/Prelude/Serial.hs+3−7 test/Streamly/Test/Prelude/Top.hs+1−1 test/Streamly/Test/Serialize/Serializable.hs+2−2 test/Streamly/Test/Unicode/Char.hs+1−1 test/Streamly/Test/Unicode/Parser.hs+93−130 test/Streamly/Test/Unicode/Stream.hs+1−1 test/lib/Streamly/Test/Prelude/Common.hs+2−9 test/streamly-tests.cabal+16−63
.circleci/config.yml+277−73
@@ -1,7 +1,7 @@
11version: 2.1
22
33#-----------------------------------------------------------------------------
4−# packcheck-0.7.0
4+# packcheck-0.4.2
55# Packcheck global environment variables
66#-----------------------------------------------------------------------------
77
@@ -11,8 +11,6 @@ env: &env
1111 # Common options
1212 # ------------------------------------------------------------------------
1313 # GHC_OPTIONS: "-Werror"
14− # For updating see: https://downloads.haskell.org/~ghcup/
15− GHCUP_VERSION: 0.1.20.0
1614 CABAL_REINIT_CONFIG: "y"
1715 LC_ALL: "C.UTF-8"
1816
@@ -24,6 +22,7 @@ env: &env
2422 # DISABLE_DOCS: "y"
2523 # DISABLE_SDIST_BUILD: "y"
2624 # DISABLE_DIST_CHECKS: "y"
25+ ENABLE_INSTALL: "y"
2726
2827 # ------------------------------------------------------------------------
2928 # stack options
@@ -39,12 +38,15 @@ env: &env
3938 # cabal options
4039 # ------------------------------------------------------------------------
4140 CABAL_CHECK_RELAX: "y"
41+ CABAL_NO_SANDBOX: "y"
42+ CABAL_HACKAGE_MIRROR: "hackage.haskell.org:http://hackage.fpcomplete.com"
4243 CABAL_BUILD_OPTIONS: "-j1 --flag limit-build-mem"
4344
4445 # ------------------------------------------------------------------------
4546 # Where to find the required tools
4647 # ------------------------------------------------------------------------
47− PATH: /sbin:/usr/sbin:/bin:/usr/bin
48+ PATH: /opt/ghc/bin:/opt/ghcjs/bin:/sbin:/usr/sbin:/bin:/usr/bin
49+ TOOLS_DIR: /opt
4850
4951 # ------------------------------------------------------------------------
5052 # Location of packcheck.sh (the shell script invoked to perform CI tests ).
@@ -57,15 +59,15 @@ env: &env
5759 # If you have not committed packcheck.sh in your repo at PACKCHECK
5860 # then it is automatically pulled from this URL.
5961 PACKCHECK_GITHUB_URL: "https://raw.githubusercontent.com/composewell/packcheck"
60− PACKCHECK_GITHUB_COMMIT: "f6cfd7b7a33b4bf6fd63cce6a3fb3d26f5031403"
62+ PACKCHECK_GITHUB_COMMIT: "a302772c8f3791c477edd1764b82f8d1f5b22fdf"
6163
6264executors:
6365 amd64-executor:
6466 docker:
65− - image: ubuntu:latest
67+ - image: ubuntu:xenial
6668 x86-executor:
6769 docker:
68− - image: i386/ubuntu:focal
70+ - image: i386/ubuntu:eoan
6971
7072#-----------------------------------------------------------------------------
7173# Common utility stuff, not to be modified usually
@@ -77,50 +79,38 @@ preinstall: &preinstall
7779 # required for https/cache save and restore
7880 apt-get install -y ca-certificates
7981
80− # Alternative way of installing ghc and cabal, directly from
81− # haskell.org instead of using ghcup. NOTE: this is for Debian
82− # only and is debian release specific.
83− # gnupg is required for apt-key to work
84− #apt-get install -y gnupg
85− #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574
86− #echo "deb http://downloads.haskell.org/debian buster main" >> /etc/apt/sources.list
87− #apt-get update
88−
89− # required for outbound https for stack and for stack setup
90− apt-get install -y netbase xz-utils make
91− apt-get install -y zlib1g-dev
82+ # required to (re)generate the configure script
83+ apt-get install -y autoconf
9284
93− # For ghcup to install ghc
94− if test -n "$GHCUP_VERSION"
95− then
96− apt-get install -y gcc
97− apt-get install -y g++
98− fi
85+ # For ghc and cabal-install packages from hvr's ppa
86+ # gnupg is required for apt-key to work
87+ apt-get install -y gnupg
88+ apt-get install -y apt-transport-https
89+ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574
90+ apt-get install -y software-properties-common
91+ add-apt-repository -y ppa:hvr/ghc
92+ # echo "deb http://downloads.haskell.org/debian stretch main" >> /etc/apt/sources.list
93+ echo "deb-src http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main" >> /etc/apt/sources.list
94+ apt-get update
9995
100− # libgmp required by ghc for linking
101− apt-get install -y libgmp-dev
102− apt-get install -y libtinfo-dev
96+ # required for packcheck
97+ apt-get install -y curl
10398
104− # Required by cabal when git URL is specified in project file
99+ # required for circleci.
100+ apt-get install -y libcurl4-gnutls-dev
105101 apt-get install -y git
102+ apt-get install -y libtinfo-dev
106103
107− # Required for and by packcheck
108− apt-get install -y curl
104+ # required for outbound https for stack and for stack setup
105+ apt-get install -y netbase xz-utils make
109106
107+project-preinstall: &project-preinstall
108+ run: |
110109 # Get packcheck if needed
111− if test ! -e "$PACKCHECK"
112− then
113− if test -z "$PACKCHECK_GITHUB_COMMIT"
114− then
115− die "PACKCHECK_GITHUB_COMMIT is not specified."
116− fi
117− PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
118− curl --fail -sL -o "$PACKCHECK" $PACKCHECK_URL || exit 1
119− chmod +x $PACKCHECK
120− elif test ! -x "$PACKCHECK"
121− then
122− chmod +x $PACKCHECK
123− fi
110+ CURL=$(which curl)
111+ PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
112+ if test ! -e "$PACKCHECK"; then $CURL -sL -o "$PACKCHECK" $PACKCHECK_URL; fi;
113+ chmod +x $PACKCHECK
124114
125115restore: &restore
126116 # Needs to happen after installing ca-certificates
@@ -136,71 +126,271 @@ save: &save
136126 save_cache:
137127 key: v1-{{ .Environment.CIRCLE_JOB }}-{{ .Branch }}
138128 paths:
139− - ~/.local
140129 - ~/.cabal
130+ - ~/.ghc
131+ - ~/.local
141132 - ~/.stack
142− - ~/.ghcup
143133
144134#-----------------------------------------------------------------------------
145135# Build matrix
146136#-----------------------------------------------------------------------------
147137
148138jobs:
149− cabal-ghc-8_6_5:
139+ cabal-ghc-8_10_4:
150140 <<: *env
151141 executor: amd64-executor
152142 steps:
143+ - *preinstall
153144 - checkout
145+ - *project-preinstall
154146 - *restore
155− - *preinstall
156147 - run:
157148 environment:
158− GHCVER: "8.6.5"
159− CABAL_VERSION: "3.6.2.0"
160149 CABAL_PROJECT: "cabal.project"
150+ DISABLE_TEST: "yes"
151+ DISABLE_BENCH: "yes"
161152 DISABLE_SDIST_BUILD: "yes"
162− CABAL_BUILD_OPTIONS: "--flag debug --flag -opt"
163153 command: |
154+ apt-get install -y ghc-8.10.4
155+ apt-get install -y cabal-install-3.4
164156 bash -c "$PACKCHECK cabal"
165157 - *save
166− cabal-ghc-9_8_1-docspec:
158+ stack-ghc-8_8:
167159 <<: *env
168160 executor: amd64-executor
169161 steps:
162+ - *preinstall
170163 - checkout
164+ - *project-preinstall
171165 - *restore
166+ - run:
167+ name: GHC 8.8 + stack lts-16.12
168+ environment:
169+ BUILD: "stack"
170+ RESOLVER: "lts-16.12"
171+ SDIST_OPTIONS: "--ignore-check"
172+ STACK_BUILD_OPTIONS: "--flag streamly-benchmarks:-opt"
173+ command: |
174+ apt-get build-dep -y ghc-8.8.4
175+ apt-get install -y cabal-install-3.2
176+ bash -c "$PACKCHECK $BUILD"
177+ - *save
178+ cabal-ghc-8_6_5:
179+ <<: *env
180+ executor: amd64-executor
181+ steps:
172182 - *preinstall
183+ - checkout
184+ - *project-preinstall
185+ - *restore
186+ - run: |
187+ apt-get install -y ghc-8.6.3
188+ apt-get install -y cabal-install-2.4
189+ bash -c "$PACKCHECK cabal-v2"
190+ - *save
191+ cabal-ghc-8_4_4:
192+ <<: *env
193+ executor: x86-executor
194+ steps:
195+ - *preinstall
196+ - checkout
197+ - *project-preinstall
198+ - *restore
199+ - run:
200+ name: GHC 8.4.4 + x86 + debug
201+ environment:
202+ BUILD: "cabal-v2"
203+ GHCVER: "8.4.4"
204+ CABAL_BUILD_OPTIONS: "--flag debug --flag limit-build-mem"
205+ command: |
206+ apt-get install -y ghc-8.4.4
207+ apt-get install -y cabal-install-3.2
208+ bash -c "$PACKCHECK $BUILD"
209+ - *save
210+ cabal-ghc-8_2_2:
211+ <<: *env
212+ executor: amd64-executor
213+ steps:
214+ - *preinstall
215+ - checkout
216+ - *project-preinstall
217+ - *restore
173218 - run:
174219 environment:
175− GHCVER: "9.8.1"
176− CABAL_VERSION: "3.10.1.0"
177− CABAL_PROJECT: "cabal.project.doctest"
178− DISABLE_SDIST_BUILD: "y"
179− DISABLE_TEST: "y"
180− DISABLE_BENCH: "y"
181− DISABLE_DOCS: "y"
182− ENABLE_DOCSPEC: "y"
183− DOCSPEC_URL: https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz
184− DOCSPEC_OPTIONS: "--timeout 60 --check-properties --property-variables xs"
220+ BUILD: "cabal-v2"
221+ DISABLE_TEST: "yes"
222+ DISABLE_BENCH: "yes"
223+ DISABLE_DOCS: "yes"
224+ DISABLE_SDIST_BUILD: "yes"
225+ DISABLE_DIST_CHECKS: "yes"
185226 command: |
186− bash -c "$PACKCHECK cabal"
227+ apt-get install -y ghc-8.2.2
228+ apt-get install -y cabal-install-3.2
229+ bash -c "$PACKCHECK $BUILD"
187230 - *save
188− hlint-trailing-spaces:
231+ cabal-ghc-8_0_2:
189232 <<: *env
190233 executor: amd64-executor
191234 steps:
235+ - *preinstall
192236 - checkout
237+ - *project-preinstall
193238 - *restore
239+ - run:
240+ environment:
241+ BUILD: "cabal-v2"
242+ DISABLE_TEST: "yes"
243+ DISABLE_BENCH: "yes"
244+ DISABLE_DOCS: "yes"
245+ DISABLE_SDIST_BUILD: "yes"
246+ DISABLE_DIST_CHECKS: "yes"
247+ command: |
248+ apt-get install -y ghc-8.0.2
249+ apt-get install -y cabal-install-3.2
250+ bash -c "$PACKCHECK $BUILD"
251+ - *save
252+ cabal-ghc-7_10_3:
253+ <<: *env
254+ executor: amd64-executor
255+ steps:
194256 - *preinstall
257+ - checkout
258+ - *project-preinstall
259+ - *restore
260+ - run: |
261+ apt-get install -y ghc-7.10.3
262+ apt-get install -y cabal-install-2.4
263+ bash -c "$PACKCHECK cabal-v2"
264+ - *save
265+ stack-ghc-8_4:
266+ <<: *env
267+ executor: amd64-executor
268+ steps:
269+ - *preinstall
270+ - checkout
271+ - *project-preinstall
272+ - *restore
273+ - run: |
274+ bash -c "$PACKCHECK stack RESOLVER=lts-12"
275+ - *save
276+ stack-ghc-8_2:
277+ <<: *env
278+ executor: amd64-executor
279+ steps:
280+ - *preinstall
281+ - checkout
282+ - *project-preinstall
283+ - *restore
284+ - run: |
285+ bash -c "$PACKCHECK stack RESOLVER=lts-11"
286+ - *save
287+ cabal-ghcjs-8_4:
288+ <<: *env
289+ executor: amd64-executor
290+ steps:
291+ - *preinstall
292+ - checkout
293+ - *project-preinstall
294+ - *restore
195295 - run:
196− name: hlint and trailing spaces
296+ name: GHCJS 8.4+no-test+no-docs
197297 environment:
298+ BUILD: cabal-v2
299+ ENABLE_GHCJS: "yes"
300+ DISABLE_TEST: "yes"
301+ ENABLE_INSTALL: ""
302+ DISABLE_DOCS: "yes"
198303 DISABLE_SDIST_BUILD: "yes"
199− HLINT_VERSION: 3.6.1
304+ command: |
305+ add-apt-repository -y ppa:hvr/ghcjs
306+ add-apt-repository -y 'deb https://deb.nodesource.com/node_11.x xenial main'
307+ curl -sSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -
308+ apt-get update
309+ apt-get install -y cabal-install-3.2
310+ apt-get install -y nodejs
311+ apt-get install -y ghcjs-8.4
312+ cabal user-config update -a "jobs: 1"
313+ bash -c "$PACKCHECK $BUILD"
314+ - *save
315+ coveralls-ghc-8_2_2:
316+ <<: *env
317+ executor: amd64-executor
318+ steps:
319+ - *preinstall
320+ - checkout
321+ - *project-preinstall
322+ - *restore
323+ - run:
324+ environment:
325+ BUILD: cabal
326+ COVERALLS_OPTIONS: "--repo-token=\"$REPO_TOKEN\" --coverage-mode=StrictlyFullLines --exclude-dir=test test"
327+ GHC_OPTIONS: "-DCOVERAGE_BUILD"
328+ command: |
329+ apt-get install -y ghc-8.2.2
330+ apt-get install -y cabal-install-2.4
331+ bash -c "$PACKCHECK cabal-v1"
332+ - *save
333+ coveralls-ghc-8_8_3:
334+ <<: *env
335+ executor: amd64-executor
336+ steps:
337+ - *preinstall
338+ - checkout
339+ - *project-preinstall
340+ - *restore
341+ - run:
342+ environment:
343+ BUILD: cabal-v2
344+ COVERALLS_OPTIONS: "--repo-token=\"$REPO_TOKEN\" --coverage-mode=StrictlyFullLines --exclude-dir=test"
345+ CABAL_PROJECT: "cabal.project.coverage"
346+ CABAL_BUILD_TARGETS: ""
347+ command: |
348+ apt-get install -y ghc-8.8.3
349+ apt-get install -y cabal-install-3.0
350+ bash -c "$PACKCHECK cabal-v2"
351+ no_output_timeout: 25m
352+ - *save
353+ coveralls-ghc-8_10_2:
354+ <<: *env
355+ executor: amd64-executor
356+ steps:
357+ - *preinstall
358+ - checkout
359+ - *project-preinstall
360+ - *restore
361+ - run:
362+ name: Build and test with inspection, coverage, Werror
363+ command: |
364+ apt-get install -y ghc-8.10.2
365+ apt-get install -y cabal-install-3.2
366+ # Run tests with coverage
367+ cabal update
368+ cabal user-config update -a "jobs: 1"
369+ bin/test.sh --coverage --raw
370+ # Upload results to coveralls.io
371+ PATH=$HOME/.cabal/bin:$PATH
372+ export PATH
373+ which hpc-coveralls 2>/dev/null || cabal install --project-file cabal.project.hpc-coveralls hpc-coveralls
374+ hpc-coveralls --repo-token="$REPO_TOKEN" --coverage-mode=StrictlyFullLines
375+ no_output_timeout: 25m
376+ - *save
377+ hlint-src:
378+ <<: *env
379+ executor: amd64-executor
380+ steps:
381+ - *preinstall
382+ - checkout
383+ - *project-preinstall
384+ - *restore
385+ - run:
386+ name: Hlint src
387+ environment:
388+ BUILD: hlint
200389 HLINT_OPTIONS: lint
390+ DISABLE_SDIST_BUILD: "yes"
201391 HLINT_TARGETS: core/src src test benchmark
202392 command: |
203− bash -c "$PACKCHECK hlint" || exit 1
393+ bash -c "$PACKCHECK $BUILD" || exit 1
204394 echo "Checking trailing spaces..."
205395 count=$(find . -name "*.hs" -exec grep -H '\ $' {} \; | tee /dev/tty | wc -l)
206396 exit $count
@@ -210,9 +400,23 @@ workflows:
210400 version: 2
211401 build:
212402 jobs:
213− #- cabal-ghc-8_6_5:
214− # name: 8.6.5-debug-unoptimized
215− - cabal-ghc-9_8_1-docspec:
216− name: ghc-9.8.1-docspec
217− - hlint-trailing-spaces:
218− name: hlint and trailing spaces
403+ - cabal-ghc-8_10_4:
404+ name: GHC 8.10.4 + sdist + no-test + no-bench
405+ #- cabal-ghc-8.6.5
406+ #- cabal-ghc-8_4_4:
407+ # name: GHC 8.4.4 + x86 + debug
408+ #- cabal-ghc-8_2_2:
409+ # name: GHC 8.2.2 + no-test + no-bench + no-docs
410+ #- cabal-ghc-8_0_2:
411+ # name: GHC 8.0.2 + no-test + no-bench + no-docs
412+ #- cabal-ghc-7.10.3
413+ # - cabal-ghcjs-8_4:
414+ # name: GHCJS 8.4 + no-test + no-docs
415+ #- stack-ghc-8.4
416+ #- stack-ghc-8.2
417+ #- coveralls-ghc-8.2.2
418+ #- coveralls-ghc-8.8.3
419+ #- coveralls-ghc-8_10_2:
420+ # name: GHC 8.10.2 + inspection + coverage + Werror
421+ - hlint-src:
422+ name: Hlint src + Trailing Spaces
.github/workflows/haskell.yml+91−84
@@ -13,15 +13,9 @@ jobs:
1313 # packcheck environment variables
1414 LC_ALL: C.UTF-8
1515 BUILD: ${{ matrix.build }}
16− # For updating see: https://downloads.haskell.org/~ghcup/
17− GHCUP_VERSION: 0.1.20.0
18− DOCSPEC_URL: https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz
19− DOCSPEC_OPTIONS: "--timeout 60 --check-properties --property-variables xs"
20− GHCUP_GHC_OPTIONS: ${{ matrix.ghcup_ghc_options }}
2116 GHCVER: ${{ matrix.ghc_version }}
2217 CABALVER: ${{ matrix.cabal_version }}
2318 DISABLE_DOCS: ${{ matrix.disable_docs }}
24− ENABLE_DOCSPEC: ${{ matrix.enable_docspec }}
2519 DISABLE_TEST: ${{ matrix.disable_test }}
2620 DISABLE_BENCH: ${{ matrix.disable_bench }}
2721 DISABLE_DIST_CHECKS: ${{ matrix.disable_dist_checks }}
@@ -45,18 +39,14 @@ jobs:
4539 STACK_BUILD_OPTIONS: ${{ matrix.stack_build_options }}
4640
4741 # packcheck location and revision
48− PACKCHECK: "./packcheck.sh"
42+ PACKCHECK_LOCAL_PATH: "./packcheck.sh"
4943 PACKCHECK_GITHUB_URL: "https://raw.githubusercontent.com/composewell/packcheck"
50− PACKCHECK_GITHUB_COMMIT: "v0.7.0"
44+ PACKCHECK_GITHUB_COMMIT: "a302772c8f3791c477edd1764b82f8d1f5b22fdf"
5145
5246 # Pull token from "secrets" setting of the github repo
5347 COVERALLS_TOKEN: ${{ secrets.COVERALLS_TOKEN }}
5448 COVERAGE: ${{ matrix.coverage }}
55−
56− # hlint
57− HLINT_VERSION: 3.6.1
58− HLINT_OPTIONS: "lint"
59− HLINT_TARGETS: "core/src src test benchmark"
49+ DOCTEST: ${{ matrix.doctest }}
6050
6151 # Subdir
6252 SUBDIR: ${{ matrix.subdir }}
@@ -68,8 +58,8 @@ jobs:
6858 matrix:
6959 # The order is important to optimize fail-fast.
7060 name:
71− - 9.8.1-Werror
72− # - 9.8.1-docspec
61+ - 9.4.4-Werror
62+ - 9.2.7-docspec
7363 # - 8.10.7-coverage
7464
7565 # Note: if cabal.project is not specified benchmarks and tests won't
@@ -78,76 +68,53 @@ jobs:
7868 include:
7969 - name: head
8070 ghc_version: head
81− # The URL may change, to find a working URL go to https://gitlab.haskell.org/ghc/ghc/-/jobs/
82− # Find a debian10/11/12 job, click on a passed/failed status, at the
83− # end of the output you will find the tar.xz name, put that tar
84− # name after "raw/", and put the job name after "job=".
85− # Also see https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix
86− #
87− # May also use ghcup for installing ghc head version, use the
88− # version "LatestNightly", and the following config:
89− # ghcup config add-release-channel https://ghc.gitlab.haskell.org/ghcup-metadata/ghcup-nightlies-0.0.7.yaml
90− ghcup_ghc_options: "-u https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-deb10-int_native-validate.tar.xz?job=x86_64-linux-deb10-int_native-validate"
9171 runner: ubuntu-latest
9272 build: cabal
93− cabal_version: 3.10.2.0
73+ cabal_version: 3.8.1.0
9474 cabal_project: cabal.project.ghc-head
9575 disable_sdist_build: "y"
9676 ignore_error: true
97− # Note: use linux for warning build for convenient dev testing
98− - name: 9.8.1-Werror
99− ghc_version: 9.8.1
100− runner: ubuntu-latest
101− build: cabal
102− cabal_version: 3.10.1.0
103− disable_sdist_build: "y"
104− cabal_project: cabal.project.Werror
105− ignore_error: false
106− #- name: 9.8.1-docspec
107− # ghc_version: 9.8.1
108− # runner: ubuntu-latest
109− # build: cabal
110− # cabal_version: 3.10.1.0
111− # cabal_project: cabal.project.doctest
112− # disable_test: "y"
113− # disable_bench: "y"
114− # disable_docs: "y"
115− # enable_docspec: "y"
116− # disable_sdist_build: "y"
117− # ignore_error: false
118− - name: 9.8.1-fusion-inspection
119− ghc_version: 9.8.1
77+ - name: 9.6.1
78+ ghc_version: 9.6.1
12079 runner: ubuntu-latest
12180 build: cabal
12281 cabal_version: 3.10.1.0
12382 disable_sdist_build: "y"
12483 cabal_project: cabal.project
125− cabal_build_options: "--flag fusion-plugin --flag inspection"
126− ignore_error: false
127− - name: 9.6.3-macos
128− ghc_version: 9.6.3
129− runner: macos-latest
84+ ignore_error: true
85+ - name: 9.4.4-Werror
86+ ghc_version: 9.4.4
87+ runner: ubuntu-latest
13088 build: cabal
131− cabal_version: 3.10.1.0
89+ cabal_version: 3.8.1.0
13290 disable_sdist_build: "y"
133− cabal_project: cabal.project
91+ cabal_project: cabal.project.Werror
13492 ignore_error: false
135− - name: 9.4.7
136− ghc_version: 9.4.7
93+ - name: 9.4.4-fusion-inspection
94+ ghc_version: 9.4.4
13795 runner: ubuntu-latest
13896 build: cabal
13997 cabal_version: 3.8.1.0
14098 disable_sdist_build: "y"
14199 cabal_project: cabal.project
100+ cabal_build_options: "--flag fusion-plugin --flag inspection"
142101 ignore_error: false
143− - name: 9.2.8
144− ghc_version: 9.2.8
102+ - name: 9.2.7
103+ ghc_version: 9.2.7
145104 runner: ubuntu-latest
146105 build: cabal
147106 cabal_project: cabal.project
148107 cabal_version: 3.6.2.0
149108 disable_sdist_build: "y"
150109 ignore_error: false
110+ - name: 9.2.7-docspec
111+ ghc_version: 9.2.7
112+ runner: ubuntu-latest
113+ doctest: "y"
114+ cabal_version: 3.6.2.0
115+ cabal_project: cabal.project
116+ disable_sdist_build: "y"
117+ ignore_error: false
151118 - name: 9.0.2-streamly-sdist
152119 ghc_version: 9.0.2
153120 runner: ubuntu-latest
@@ -171,6 +138,14 @@ jobs:
171138 stack_build_options: "--flag streamly-benchmarks:-opt"
172139 cabal_version: 3.6.2.0
173140 ignore_error: false
141+ - name: 8.10.7-macos
142+ ghc_version: 8.10.7
143+ runner: macos-latest
144+ build: cabal
145+ cabal_version: 3.6.2.0
146+ cabal_project: cabal.project
147+ disable_sdist_build: "y"
148+ ignore_error: false
174149 # - name: 8.10.7-coverage
175150 # ghc_version: 8.10.7
176151 # runner: ubuntu-latest
@@ -185,6 +160,15 @@ jobs:
185160 cabal_project: cabal.project
186161 disable_sdist_build: "y"
187162 ignore_error: false
163+ - name: 8.6.5-no-fusion-unoptimized
164+ ghc_version: 8.6.5
165+ runner: ubuntu-latest
166+ build: cabal
167+ cabal_version: 3.6.2.0
168+ cabal_project: cabal.project
169+ cabal_build_options: "--flag no-fusion --flag -opt"
170+ disable_sdist_build: "y"
171+ ignore_error: false
188172 - name: 8.6.5-debug-unoptimized
189173 ghc_version: 8.6.5
190174 runner: ubuntu-latest
@@ -194,44 +178,66 @@ jobs:
194178 cabal_build_options: "--flag debug --flag -opt"
195179 disable_sdist_build: "y"
196180 ignore_error: false
197− - name: hlint
198− build: hlint
199− runner: ubuntu-latest
200− ignore_error: true
201181
202182 steps:
203183 - uses: actions/checkout@v2
204184
205185 - uses: actions/cache@v1
206− name: Cache common directories
186+ name: Cache ~/.cabal
207187 with:
208− path: |
209− ~/.local
210− ~/.cabal
211− ~/.stack
212− ~/.ghcup
188+ path: ~/.cabal
213189 # Bump the key version to clear the cache
214190 key: ${{ runner.os }}-${{ matrix.ghc_version }}-cabal-v2
215191
216192 - name: Download packcheck
217193 run: |
218− if test ! -e "$PACKCHECK"
219− then
220− if test -z "$PACKCHECK_GITHUB_COMMIT"
221− then
222− die "PACKCHECK_GITHUB_COMMIT is not specified."
223− fi
224− PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
225− curl --fail -sL -o "$PACKCHECK" $PACKCHECK_URL || exit 1
226− chmod +x $PACKCHECK
227− elif test ! -x "$PACKCHECK"
194+ # Get packcheck if needed
195+ CURL=$(which curl)
196+ PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
197+ if test ! -e "$PACKCHECK_LOCAL_PATH"; then $CURL -sL -o "$PACKCHECK_LOCAL_PATH" $PACKCHECK_URL; fi;
198+ chmod +x $PACKCHECK_LOCAL_PATH
199+
200+ os=$(uname -s -m)
201+ case "$os" in
202+ "Linux x86_64") GHCUP_ARCH="x86_64-linux" ;;
203+ "Darwin x86_64") GHCUP_ARCH="x86_64-apple-darwin" ;;
204+ *) echo "Unknown OS/Arch: $os"; exit 1;;
205+ esac
206+
207+ # Check available versions here: https://downloads.haskell.org/~ghcup/
208+ GHCUP_VER=0.1.18.0
209+ $CURL -sL -o ./ghcup https://downloads.haskell.org/~ghcup/$GHCUP_VER/${GHCUP_ARCH}-ghcup-$GHCUP_VER
210+ chmod +x ./ghcup
211+ if test "$GHCVER" = "head"
228212 then
229− chmod +x $PACKCHECK
213+ # The URL may change, to find a working URL go to https://gitlab.haskell.org/ghc/ghc/-/jobs/
214+ # Find a debian10 job, click on a passed/failed job, at the
215+ # end of the output you will find the tar.xz name, put that after
216+ # "raw/", and put the job name after "job=".
217+ # Also see https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix
218+ ./ghcup install ghc -u https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-deb10-int_native-validate.tar.xz?job=x86_64-linux-deb10-int_native-validate head
219+ else
220+ ./ghcup install ghc $GHCVER
230221 fi
222+ ./ghcup install cabal $CABALVER
223+ ./ghcup set ghc $GHCVER
224+ ./ghcup set cabal $CABALVER
231225
232226 - name: Run tests
233227 run: |
234− if test -n "$COVERAGE"
228+ if test -n "$DOCTEST"
229+ then
230+ mkdir -p $HOME/.cabal/bin
231+ curl -sL https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz > cabal-docspec.xz
232+ echo '0829bd034fba901cbcfe491d98ed8b28fd54f9cb5c91fa8e1ac62dc4413c9562 cabal-docspec.xz' | sha256sum -c -
233+ xz -d < cabal-docspec.xz > $HOME/.cabal/bin/cabal-docspec
234+ rm -f cabal-docspec.xz
235+ chmod a+x $HOME/.cabal/bin/cabal-docspec
236+ cabal update
237+ cabal build --project-file cabal.project.doctest all
238+ $HOME/.cabal/bin/cabal-docspec --version
239+ $HOME/.cabal/bin/cabal-docspec --timeout 60 --check-properties --property-variables xs
240+ elif test -n "$COVERAGE"
235241 then
236242 # Run tests with coverage
237243 cabal update
@@ -244,15 +250,16 @@ jobs:
244250 # XXX Uncomment this and fix it properly later
245251 # hpc-coveralls --repo-token="$COVERALLS_TOKEN" --coverage-mode=StrictlyFullLines
246252 else
253+ export TOOLS_DIR=/opt
247254 # /usr/local/opt/curl/bin for macOS
248255 export PATH=$HOME/.local/bin:$HOME/.ghcup/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/opt/curl/bin
249256 if test -n "$SUBDIR"
250257 then
251− mv "$PACKCHECK" "$SUBDIR/packcheck.sh"
258+ mv "$PACKCHECK_LOCAL_PATH" "$SUBDIR/packcheck.sh"
252259 # This is required as dist-newstyle will be created on the top level
253260 # directory as it is considered the working dir.
254261 rm cabal.project
255262 cd "$SUBDIR"
256263 fi
257− bash -c "$PACKCHECK $BUILD"
264+ bash -c "$PACKCHECK_LOCAL_PATH $BUILD"
258265 fi
.github/workflows/regression-check.yml+3−3
@@ -19,15 +19,15 @@ jobs:
1919 env:
2020 CI_BENCHMARKS_WITH_CUTOFF: >-
2121 Data.Array
22− Data.MutArray
22+ Data.Array.Mut
2323 Data.Array.Generic
2424 Data.Ring.Unboxed
2525 Data.Fold
2626 Data.Fold.Window
2727 Data.Parser
2828 Data.ParserK
29− Data.StreamD
30− Data.StreamK:6
29+ Data.Stream.StreamD
30+ Data.Stream.StreamK:6
3131 Data.Unfold
3232 Data.Stream
3333 Data.Stream.Concurrent
.hlint.ignore+2−7
@@ -2,18 +2,14 @@ core/src/Streamly/Internal/Data/Stream/StreamK/Type.hs
22core/src/Streamly/Internal/Data/Pipe/Type.hs
33core/src/Streamly/Internal/Unicode/Stream.hs
44src/Streamly/Internal/Data/SmallArray/Type.hs
5−src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs
6−src/Streamly/Internal/Data/Stream/Serial.hs
7−src/Streamly/Internal/Data/Stream/Zip.hs
85test/Streamly/Test/Data/Array.hs
96test/Streamly/Test/Data/Parser.hs
10−test/Streamly/Test/Data/ParserK.hs
7+test/Streamly/Test/Data/Parser/ParserD.hs
118test/Streamly/Test/Data/SmallArray.hs
129test/Streamly/Test/Data/Unfold.hs
1310test/Streamly/Test/FileSystem/Event.hs
1411test/Streamly/Test/Prelude/Concurrent.hs
1512test/Streamly/Test/Data/Stream/Concurrent.hs
16−test/lib/Streamly/Test/Prelude/Common.hs
1713test/Streamly/Test/Prelude/Fold.hs
1814test/Streamly/Test/Prelude/Rate.hs
1915test/Streamly/Test/Prelude/Serial.hs
@@ -25,13 +21,12 @@ benchmark/lib/Streamly/Benchmark/Prelude.hs
2521benchmark/NanoBenchmarks.hs
2622benchmark/Streamly/Benchmark/Data/Array.hs
2723benchmark/Streamly/Benchmark/Data/Parser.hs
28−benchmark/Streamly/Benchmark/Data/ParserK.hs
24+benchmark/Streamly/Benchmark/Data/Parser/ParserD.hs
2925benchmark/Streamly/Benchmark/Data/Stream/StreamDK.hs
3026benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs
3127benchmark/Streamly/Benchmark/Data/Unfold.hs
3228benchmark/Streamly/Benchmark/FileSystem/Handle.hs
3329benchmark/Streamly/Benchmark/Prelude/Async.hs
34−benchmark/Streamly/Benchmark/Prelude/Concurrent.hs
3530benchmark/Streamly/Benchmark/Prelude/Merge.hs
3631benchmark/Streamly/Benchmark/Prelude/Parallel.hs
3732benchmark/Streamly/Benchmark/Prelude/Rate.hs
benchmark/NanoBenchmarks.hs+1−1
@@ -18,7 +18,7 @@ import qualified Streamly.Data.Fold as FL
1818import qualified Streamly.Internal.Data.Stream as S
1919import qualified Streamly.Internal.Data.Stream.IsStream as Internal
2020
21−import Test.Tasty.Bench
21+import Gauge
2222import System.Random
2323
2424maxValue :: Int
benchmark/Streamly/Benchmark/Data/Array/CommonImports.hs+1−1
@@ -9,7 +9,7 @@ import qualified Streamly.Data.Fold as Fold
99import qualified Streamly.Data.Stream as S
1010import qualified Streamly.Internal.Data.Stream as Stream
1111
12−import Test.Tasty.Bench
12+import Gauge
1313import Streamly.Benchmark.Common hiding (benchPureSrc)
1414import qualified Stream.Common as P
1515
benchmark/Streamly/Benchmark/Data/MutArray.hs → benchmark/Streamly/Benchmark/Data/Array/Mut.hsrenamed+4−4
@@ -42,14 +42,14 @@ import Prelude
4242 , id
4343 , undefined
4444 )
45−import Streamly.Internal.Data.MutArray (MutArray)
45+import Streamly.Internal.Data.Array.Mut (MutArray)
4646
4747import qualified Streamly.Internal.Data.Array as Array
48−import qualified Streamly.Internal.Data.MutArray as MArray
48+import qualified Streamly.Internal.Data.Array.Mut as MArray
4949import qualified Streamly.Internal.Data.Fold as Fold
5050import qualified Streamly.Internal.Data.Stream as Stream
5151
52−import Test.Tasty.Bench
52+import Gauge
5353import Streamly.Benchmark.Common hiding (benchPureSrc)
5454
5555type Stream = MutArray
@@ -274,7 +274,7 @@ o_1_space_serial_marray value ~(array, indices) =
274274-------------------------------------------------------------------------------
275275
276276moduleName :: String
277−moduleName = "Data.MutArray"
277+moduleName = "Data.Array.Mut"
278278
279279main :: IO ()
280280main = do
benchmark/Streamly/Benchmark/Data/Array/Stream.hs+4−4
@@ -39,7 +39,7 @@ import Prelude hiding ()
3939
4040import qualified Streamly.Data.Stream as Stream
4141import qualified Streamly.Internal.Data.Array as Array
42−import qualified Streamly.Internal.Data.Array.Stream as ArrayStream
42+import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
4343import qualified Streamly.Internal.Data.Fold as Fold
4444import qualified Streamly.Internal.Data.Parser as Parser
4545import qualified Streamly.Internal.Data.Stream as Stream
@@ -47,14 +47,14 @@ import qualified Streamly.Internal.Data.StreamK as StreamK
4747import qualified Streamly.Internal.FileSystem.Handle as Handle
4848import qualified Streamly.Internal.Unicode.Stream as Unicode
4949
50−import Test.Tasty.Bench hiding (env)
50+import Gauge hiding (env)
5151import Streamly.Benchmark.Common
5252import Streamly.Benchmark.Common.Handle
5353import Control.Monad.IO.Class (MonadIO)
5454
5555#ifdef INSPECTION
56−import Streamly.Internal.Data.MutByteArray (Unbox)
57−import Streamly.Internal.Data.Stream (Step(..))
56+import Streamly.Internal.Data.Unbox (Unbox)
57+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..))
5858import Test.Inspection
5959#endif
6060
benchmark/Streamly/Benchmark/Data/Fold.hs+6−29
@@ -12,11 +12,6 @@
1212{-# OPTIONS_GHC -fno-warn-warnings-deprecations #-}
1313{-# OPTIONS_GHC -Wno-orphans #-}
1414
15−#undef FUSION_CHECK
16−#ifdef FUSION_CHECK
17−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
18−#endif
19−
2015module Main (main) where
2116
2217import Control.DeepSeq (NFData(..))
@@ -33,15 +28,16 @@ import System.Random (randomRIO)
3328import Streamly.Internal.Data.Stream (Stream)
3429import Streamly.Internal.Data.Fold (Fold(..))
3530import Streamly.Internal.Data.IsMap.HashMap ()
36−import Streamly.Internal.Data.MutArray (MutArray)
31+import Streamly.Internal.Data.Array.Mut (MutArray)
3732
3833import qualified Streamly.Internal.Data.Fold as FL
3934import qualified Streamly.Data.Fold.Prelude as Fold
35+import qualified Streamly.Internal.Data.Fold.Container as FL
4036import qualified Streamly.Internal.Data.Unfold as Unfold
4137import qualified Streamly.Internal.Data.Pipe as Pipe
4238import qualified Streamly.Internal.Data.Stream as Stream
4339
44−import Test.Tasty.Bench
40+import Gauge
4541import Streamly.Benchmark.Common
4642import Prelude hiding (all, any, take, unzip, sequence_, filter)
4743
@@ -269,13 +265,13 @@ unzip = Stream.fold $ FL.lmap (\a -> (a, a)) (FL.unzip FL.sum FL.length)
269265{-# INLINE unzipWithFstM #-}
270266unzipWithFstM :: Monad m => Stream m Int -> m (Int, Int)
271267unzipWithFstM = do
272− let f a = return (a + 1, a)
268+ let f = \a -> return (a + 1, a)
273269 Stream.fold (FL.unzipWithFstM f FL.sum FL.length)
274270
275271{-# INLINE unzipWithMinM #-}
276272unzipWithMinM :: Monad m => Stream m Int -> m (Int, Int)
277273unzipWithMinM = do
278− let f a = return (a + 1, a)
274+ let f = \a -> return (a + 1, a)
279275 Stream.fold (FL.unzipWithMinM f FL.sum FL.length)
280276
281277-------------------------------------------------------------------------------
@@ -507,9 +503,7 @@ o_n_heap_serial value =
507503-------------------------------------------------------------------------------
508504
509505main :: IO ()
510−main = do
511−#ifndef FUSION_CHECK
512− runWithCLIOpts defaultStreamSize allBenchmarks
506+main = runWithCLIOpts defaultStreamSize allBenchmarks
513507
514508 where
515509
@@ -522,20 +516,3 @@ main = do
522516 , bgroup (o_n_space_prefix moduleName) (o_n_space_serial value)
523517 , bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
524518 ]
525−#else
526− -- Enable FUSION_CHECK macro at the beginning of the file
527− -- Enable one benchmark below, and run the benchmark
528− -- Check the .dump-simpl output
529− let value = 100000
530− let input = source value 1
531− let getKey buckets = (`mod` buckets)
532− let getFold k =
533− return $ case k of
534− 0 -> FL.sum
535− 1 -> FL.length
536− _ -> FL.length
537−
538− -- demuxToMap (getKey 64) (getFold . getKey 64) input
539− toIntMapIO (getKey 64) input
540− return ()
541−#endif
benchmark/Streamly/Benchmark/Data/Fold/Window.hs+40−40
@@ -8,12 +8,12 @@ import Streamly.Internal.Data.Stream (Stream)
88import System.Random (randomRIO)
99
1010import qualified Streamly.Data.Fold as Fold
11−import qualified Streamly.Internal.Data.Fold as Window
12−import qualified Streamly.Internal.Data.Ring as Ring
11+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
12+import qualified Streamly.Internal.Data.Fold.Window as Window
1313import qualified Streamly.Internal.Data.Stream as Stream
1414
1515import Streamly.Benchmark.Common
16−import Test.Tasty.Bench
16+import Gauge
1717
1818{-# INLINE source #-}
1919source :: (Monad m, Num a, Stream.Enumerable a) =>
@@ -70,55 +70,55 @@ o_1_space_folds :: Int -> [Benchmark]
7070o_1_space_folds numElements =
7171 [ bgroup "fold"
7272 [ benchWithFold numElements "minimum (window size 100)"
73− (Window.windowMinimum 100)
73+ (Window.minimum 100)
7474 , benchWithFold numElements "minimum (window size 1000)"
75− (Window.windowMinimum 1000)
75+ (Window.minimum 1000)
7676 , benchWith sourceDescendingInt numElements
7777 "minimum descending (window size 1000)"
78− (Window.windowMinimum 1000)
78+ (Window.minimum 1000)
7979
8080 , benchWithFold numElements "maximum (window size 100)"
81− (Window.windowMaximum 100)
81+ (Window.maximum 100)
8282 , benchWithFold numElements "maximum (window size 1000)"
83− (Window.windowMaximum 1000)
83+ (Window.maximum 1000)
8484 , benchWith sourceDescendingInt numElements
8585 "maximum descending (window size 1000)"
86− (Window.windowMaximum 1000)
86+ (Window.maximum 1000)
8787
8888 , benchWithFold numElements "range (window size 100)"
89− (Window.windowRange 100)
89+ (Window.range 100)
9090 , benchWithFold numElements "range (window size 1000)"
91− (Window.windowRange 1000)
91+ (Window.range 1000)
9292 , benchWith sourceDescendingInt numElements
9393 "range descending (window size 1000)"
94− (Window.windowRange 1000)
94+ (Window.range 1000)
9595
9696 , benchWithFoldInt numElements "sumInt (window size 100)"
97− (Ring.slidingWindow 100 Window.windowSumInt)
97+ (Ring.slidingWindow 100 Window.sumInt)
9898 , benchWithFoldInt numElements "sum for Int (window size 100)"
99− (Ring.slidingWindow 100 Window.windowSum)
99+ (Ring.slidingWindow 100 Window.sum)
100100 , benchWithFold numElements "sum (window size 100)"
101− (Ring.slidingWindow 100 Window.windowSum)
101+ (Ring.slidingWindow 100 Window.sum)
102102 , benchWithFold numElements "sum (window size 1000)"
103− (Ring.slidingWindow 1000 Window.windowSum)
103+ (Ring.slidingWindow 1000 Window.sum)
104104 , benchWithFold numElements "sum (entire stream)"
105− (Window.cumulative Window.windowSum)
105+ (Window.cumulative Window.sum)
106106 , benchWithFold numElements "sum (Data.Fold)"
107107 Fold.sum
108108
109109 , benchWithFold numElements "mean (window size 100)"
110− (Ring.slidingWindow 100 Window.windowMean)
110+ (Ring.slidingWindow 100 Window.mean)
111111 , benchWithFold numElements "mean (window size 1000)"
112− (Ring.slidingWindow 1000 Window.windowMean)
112+ (Ring.slidingWindow 1000 Window.mean)
113113 , benchWithFold numElements "mean (entire stream)"
114− (Window.cumulative Window.windowMean)
114+ (Window.cumulative Window.mean)
115115 , benchWithFold numElements "mean (Data.Fold)"
116116 Fold.mean
117117
118118 , benchWithFold numElements "powerSum 2 (window size 100)"
119− (Ring.slidingWindow 100 (Window.windowPowerSum 2))
119+ (Ring.slidingWindow 100 (Window.powerSum 2))
120120 , benchWithFold numElements "powerSum 2 (entire stream)"
121− (Window.cumulative (Window.windowPowerSum 2))
121+ (Window.cumulative (Window.powerSum 2))
122122
123123 ]
124124 ]
@@ -127,51 +127,51 @@ o_1_space_scans :: Int -> [Benchmark]
127127o_1_space_scans numElements =
128128 [ bgroup "scan"
129129 [ benchWithPostscan numElements "minimum (window size 10)"
130− (Window.windowMinimum 10)
130+ (Window.minimum 10)
131131 -- Below window size 30 the linear search based impl performs better
132132 -- than the dequeue based implementation.
133133 , benchWithPostscan numElements "minimum (window size 30)"
134− (Window.windowMinimum 30)
134+ (Window.minimum 30)
135135 , benchWithPostscan numElements "minimum (window size 1000)"
136− (Window.windowMinimum 1000)
136+ (Window.minimum 1000)
137137 , benchScanWith sourceDescendingInt numElements
138138 "minimum descending (window size 1000)"
139− (Window.windowMinimum 1000)
139+ (Window.minimum 1000)
140140
141141 , benchWithPostscan numElements "maximum (window size 10)"
142− (Window.windowMaximum 10)
142+ (Window.maximum 10)
143143 , benchWithPostscan numElements "maximum (window size 30)"
144− (Window.windowMaximum 30)
144+ (Window.maximum 30)
145145 , benchWithPostscan numElements "maximum (window size 1000)"
146− (Window.windowMaximum 1000)
146+ (Window.maximum 1000)
147147 , benchScanWith sourceDescendingInt numElements
148148 "maximum descending (window size 1000)"
149− (Window.windowMaximum 1000)
149+ (Window.maximum 1000)
150150
151151 , benchWithPostscan numElements "range (window size 10)"
152− (Window.windowRange 10)
152+ (Window.range 10)
153153 , benchWithPostscan numElements "range (window size 30)"
154− (Window.windowRange 30)
154+ (Window.range 30)
155155 , benchWithPostscan numElements "range (window size 1000)"
156− (Window.windowRange 1000)
156+ (Window.range 1000)
157157 , benchScanWith sourceDescendingInt numElements
158158 "range descending (window size 1000)"
159− (Window.windowRange 1000)
159+ (Window.range 1000)
160160
161161 , benchWithPostscan numElements "sum (window size 100)"
162− (Ring.slidingWindow 100 Window.windowSum)
162+ (Ring.slidingWindow 100 Window.sum)
163163 , benchWithPostscan numElements "sum (window size 1000)"
164− (Ring.slidingWindow 1000 Window.windowSum)
164+ (Ring.slidingWindow 1000 Window.sum)
165165
166166 , benchWithPostscan numElements "mean (window size 100)"
167− (Ring.slidingWindow 100 Window.windowMean)
167+ (Ring.slidingWindow 100 Window.mean)
168168 , benchWithPostscan numElements "mean (window size 1000)"
169− (Ring.slidingWindow 1000 Window.windowMean)
169+ (Ring.slidingWindow 1000 Window.mean)
170170
171171 , benchWithPostscan numElements "powerSum 2 (window size 100)"
172− (Ring.slidingWindow 100 (Window.windowPowerSum 2))
172+ (Ring.slidingWindow 100 (Window.powerSum 2))
173173 , benchWithPostscan numElements "powerSum 2 (window size 1000)"
174− (Ring.slidingWindow 1000 (Window.windowPowerSum 2))
174+ (Ring.slidingWindow 1000 (Window.powerSum 2))
175175 ]
176176 ]
177177
benchmark/Streamly/Benchmark/Data/Parser.hs+10−10
@@ -47,9 +47,10 @@ import qualified Streamly.Internal.Data.Fold as Fold
4747import qualified Streamly.Internal.Data.Parser as PR
4848import qualified Streamly.Data.Stream as Stream
4949import qualified Streamly.Internal.Data.Producer as Producer
50+import qualified Streamly.Internal.Data.Producer.Source as Source
5051import qualified Streamly.Internal.Data.Stream as Stream
5152
52−import Test.Tasty.Bench hiding (env)
53+import Gauge hiding (env)
5354import Streamly.Benchmark.Common
5455import Streamly.Benchmark.Common.Handle
5556
@@ -348,7 +349,7 @@ split_ value =
348349-- PR.dropWhile (<= (value * 1 `div` 4)) *> PR.die "alt"
349350{-# INLINE takeWhileFail #-}
350351takeWhileFail :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
351−takeWhileFail predicate (Fold fstep finitial _ ffinal) =
352+takeWhileFail predicate (Fold fstep finitial fextract) =
352353 Parser step initial extract
353354
354355 where
@@ -369,7 +370,7 @@ takeWhileFail predicate (Fold fstep finitial _ ffinal) =
369370 Fold.Done b -> Done 0 b
370371 else return $ Error "fail"
371372
372− extract s = fmap (Done 0) (ffinal s)
373+ extract s = fmap (Done 0) (fextract s)
373374
374375{-# INLINE alt2 #-}
375376alt2 :: Monad m
@@ -634,13 +635,13 @@ choice value =
634635{-# INLINE parseManyUnfoldArrays #-}
635636parseManyUnfoldArrays :: Int -> [Array.Array Int] -> IO ()
636637parseManyUnfoldArrays count arrays = do
637− let src = Producer.source (Just (Producer.OuterLoop arrays))
638+ let src = Source.source (Just (Producer.OuterLoop arrays))
638639 let parser = PR.fromFold (Fold.take count Fold.drain)
639640 let readSrc =
640− Producer.producer
641+ Source.producer
641642 $ Producer.concat Producer.fromList Array.producer
642643 let streamParser =
643− Producer.simplify (Producer.parseMany parser readSrc)
644+ Producer.simplify (Source.parseMany parser readSrc)
644645 Stream.fold Fold.drain $ Stream.unfold streamParser src
645646
646647-------------------------------------------------------------------------------
@@ -714,6 +715,8 @@ instance NFData ParseError where
714715o_1_space_serial :: Int -> [Benchmark]
715716o_1_space_serial value =
716717 [ benchIOSink value "takeBetween" $ takeBetween value
718+ , benchIOSink value "takeEQ" $ takeEQ value
719+ , benchIOSink value "takeGE" $ takeGE value
717720 , benchIOSink value "takeWhile" $ takeWhile value
718721 , benchIOSink value "takeWhileP" $ takeWhileP value
719722 , benchIOSink value "takeP" $ takeP value
@@ -810,11 +813,8 @@ o_1_space_serial_unfold bound arrays =
810813o_n_heap_serial :: Int -> [Benchmark]
811814o_n_heap_serial value =
812815 [
813− benchIOSink value "takeEQ" $ takeEQ value
814− , benchIOSink value "takeGE" $ takeGE value
815−
816816 -- lookahead benchmark holds the entire input till end
817− , benchIOSink value "lookAhead" $ lookAhead value
817+ benchIOSink value "lookAhead" $ lookAhead value
818818
819819 -- o-n-heap because of backtracking
820820 , benchIOSrc sourceEscapedFrames value "takeFramedByEsc_"
benchmark/Streamly/Benchmark/Data/ParserK.hs+37−90
@@ -5,10 +5,6 @@
55-- License : BSD-3-Clause
66-- Maintainer : streamly@composewell.com
77
8−-- BENCH_CHUNKED -> adaptC
9−-- BENCH_CHUNKED_GENERIC -> adaptCG
10−-- BENCH_SINGULAR -> adapt
11−
128{-# LANGUAGE CPP #-}
139{-# LANGUAGE FlexibleContexts #-}
1410{-# OPTIONS_GHC -Wno-orphans #-}
@@ -22,14 +18,8 @@ import Control.Applicative ((<|>))
2218import Control.DeepSeq (NFData(..))
2319import Control.Monad.IO.Class (MonadIO)
2420import Data.Foldable (asum)
25−#ifdef BENCH_CHUNKED
2621import Streamly.Data.Array (Array, Unbox)
27−#endif
28−#ifdef BENCH_CHUNKED_GENERIC
29−import Streamly.Data.Array.Generic (Array)
30−import qualified Streamly.Internal.Data.Array.Generic as GenArr (chunksOf)
31−#endif
32−import Streamly.Internal.Data.Fold (Fold(..))
22+import Streamly.Internal.Data.Fold.Type (Fold(..))
3323import Streamly.Data.StreamK (StreamK)
3424import Streamly.Internal.Data.Parser
3525 (ParseError(..), Parser(..), Initial(..), Step(..))
@@ -45,52 +35,12 @@ import qualified Streamly.Data.Stream as Stream
4535import qualified Streamly.Data.Fold as FL
4636import qualified Streamly.Internal.Data.Fold as Fold
4737import qualified Streamly.Data.Parser as PRD
48−import qualified Streamly.Internal.Data.ParserK as PR
38+import qualified Streamly.Data.ParserK as PR
4939import qualified Streamly.Internal.Data.StreamK as StreamK
5040
51−import Test.Tasty.Bench
41+import Gauge
5242import Streamly.Benchmark.Common
5343
54−-------------------------------------------------------------------------------
55−-- CPP Helpers
56−-------------------------------------------------------------------------------
57−
58−#ifdef BENCH_CHUNKED
59−
60−#define PARSE_OP StreamK.parseChunks
61−#define FROM_PARSER adaptC
62−#define INPUT (Array a)
63−#define PARSE_ELEM (Array Int)
64−#define CONSTRAINT_IO (MonadIO m, Unbox a)
65−#define CONSTRAINT (Monad m, Unbox a)
66−#define MODULE_NAME "Data.ParserK.Chunked"
67−
68−#endif
69−
70−#ifdef BENCH_CHUNKED_GENERIC
71−
72−#define PARSE_OP StreamK.parseChunksGeneric
73−#define FROM_PARSER adaptCG
74−#define INPUT (Array a)
75−#define PARSE_ELEM (Array Int)
76−#define CONSTRAINT_IO (MonadIO m)
77−#define CONSTRAINT (Monad m)
78−#define MODULE_NAME "Data.ParserK.Chunked.Generic"
79−
80−#endif
81−
82−#ifdef BENCH_SINGULAR
83−
84−#define PARSE_OP StreamK.parse
85−#define FROM_PARSER adapt
86−#define INPUT a
87−#define PARSE_ELEM Int
88−#define CONSTRAINT_IO (MonadIO m)
89−#define CONSTRAINT (Monad m)
90−#define MODULE_NAME "Data.ParserK"
91−
92−#endif
93−
9444-------------------------------------------------------------------------------
9545-- Utilities
9646-------------------------------------------------------------------------------
@@ -110,52 +60,49 @@ sourceUnfoldrM value n = Stream.unfoldrM step n
11060{-# INLINE benchIOSink #-}
11161benchIOSink
11262 :: NFData b
113− => Int -> String -> (StreamK IO PARSE_ELEM -> IO b) -> Benchmark
63+ => Int -> String -> (StreamK IO (Array Int) -> IO b) -> Benchmark
11464benchIOSink value name f =
11565 bench name $ nfIO $ randomRIO (1,1)
11666 >>= f
11767 . StreamK.fromStream
118−#ifdef BENCH_CHUNKED
11968 . Stream.chunksOf 4000
120−#endif
121−#ifdef BENCH_CHUNKED_GENERIC
122− . GenArr.chunksOf 4000
123−#endif
12469 . sourceUnfoldrM value
12570
12671-------------------------------------------------------------------------------
12772-- Parsers
12873-------------------------------------------------------------------------------
12974
75+#define PARSE_OP StreamK.parseChunks
76+
13077{-# INLINE one #-}
13178one :: MonadIO m =>
132− Int -> StreamK m PARSE_ELEM -> m (Either ParseError (Maybe Int))
133−one value = PARSE_OP p
79+ Int -> StreamK m (Array Int) -> m (Either ParseError (Maybe Int))
80+one value = StreamK.parseChunks p
13481
13582 where
13683
13784 p = do
138− m <- PR.FROM_PARSER (PRD.fromFold FL.one)
85+ m <- PR.fromFold FL.one
13986 case m of
14087 Just i -> if i >= value then pure m else p
14188 Nothing -> pure Nothing
14289
14390{-# INLINE satisfy #-}
144−satisfy :: CONSTRAINT_IO => (a -> Bool) -> PR.ParserK INPUT m a
145−satisfy = PR.FROM_PARSER . PRD.satisfy
91+satisfy :: (MonadIO m, Unbox a) => (a -> Bool) -> PR.ParserK a m a
92+satisfy = PR.fromParser . PRD.satisfy
14693
14794{-# INLINE takeWhile #-}
148−takeWhile :: CONSTRAINT_IO => (a -> Bool) -> PR.ParserK INPUT m ()
149−takeWhile p = PR.FROM_PARSER $ PRD.takeWhile p FL.drain
95+takeWhile :: (MonadIO m, Unbox a) => (a -> Bool) -> PR.ParserK a m ()
96+takeWhile p = PR.fromParser $ PRD.takeWhile p FL.drain
15097
15198{-# INLINE takeWhileK #-}
15299takeWhileK :: MonadIO m =>
153− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
100+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
154101takeWhileK value = PARSE_OP (takeWhile (<= value))
155102
156103{-# INLINE splitAp2 #-}
157104splitAp2 :: MonadIO m
158− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ((), ()))
105+ => Int -> StreamK m (Array Int) -> m (Either ParseError ((), ()))
159106splitAp2 value =
160107 PARSE_OP
161108 ((,)
@@ -165,7 +112,7 @@ splitAp2 value =
165112
166113{-# INLINE splitAp8 #-}
167114splitAp8 :: MonadIO m
168− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
115+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
169116splitAp8 value =
170117 PARSE_OP
171118 ( (\() () () () () () () () -> ())
@@ -180,7 +127,7 @@ splitAp8 value =
180127 )
181128
182129{-# INLINE sequenceA #-}
183−sequenceA :: MonadIO m => Int -> StreamK m PARSE_ELEM -> m Int
130+sequenceA :: MonadIO m => Int -> StreamK m (Array Int) -> m Int
184131sequenceA value xs = do
185132 let parser = satisfy (> 0)
186133 list = Prelude.replicate value parser
@@ -189,14 +136,14 @@ sequenceA value xs = do
189136
190137{-# INLINE sequenceA_ #-}
191138sequenceA_ :: MonadIO m =>
192− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
139+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
193140sequenceA_ value xs = do
194141 let parser = satisfy (> 0)
195142 list = Prelude.replicate value parser
196143 PARSE_OP (F.sequenceA_ list) xs
197144
198145{-# INLINE sequence #-}
199−sequence :: MonadIO m => Int -> StreamK m PARSE_ELEM -> m Int
146+sequence :: MonadIO m => Int -> StreamK m (Array Int) -> m Int
200147sequence value xs = do
201148 let parser = satisfy (> 0)
202149 list = Prelude.replicate value parser
@@ -205,7 +152,7 @@ sequence value xs = do
205152
206153{-# INLINE sequence_ #-}
207154sequence_ :: MonadIO m =>
208− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
155+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
209156sequence_ value =
210157 let parser = satisfy (> 0)
211158 list = Prelude.replicate value parser
@@ -213,7 +160,7 @@ sequence_ value =
213160
214161{-# INLINE takeWhileFailD #-}
215162takeWhileFailD :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
216−takeWhileFailD predicate (Fold fstep finitial _ ffinal) =
163+takeWhileFailD predicate (Fold fstep finitial fextract) =
217164 Parser step initial extract
218165
219166 where
@@ -234,16 +181,16 @@ takeWhileFailD predicate (Fold fstep finitial _ ffinal) =
234181 Fold.Done b -> Done 0 b
235182 else return $ Error "fail"
236183
237− extract s = fmap (Done 0) (ffinal s)
184+ extract s = fmap (Done 0) (fextract s)
238185
239186{-# INLINE takeWhileFail #-}
240−takeWhileFail :: CONSTRAINT =>
241− (a -> Bool) -> Fold m a b -> PR.ParserK INPUT m b
242−takeWhileFail p f = PR.FROM_PARSER (takeWhileFailD p f)
187+takeWhileFail :: (Monad m, Unbox a) =>
188+ (a -> Bool) -> Fold m a b -> PR.ParserK a m b
189+takeWhileFail p f = PR.fromParser (takeWhileFailD p f)
243190
244191{-# INLINE alt2 #-}
245192alt2 :: MonadIO m
246− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
193+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
247194alt2 value =
248195 PARSE_OP
249196 ( takeWhileFail (<= (value `div` 2)) Fold.drain
@@ -252,7 +199,7 @@ alt2 value =
252199
253200{-# INLINE alt8 #-}
254201alt8 :: MonadIO m
255− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
202+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
256203alt8 value =
257204 PARSE_OP
258205 ( takeWhileFail (<= ( value `div` 8)) Fold.drain
@@ -267,7 +214,7 @@ alt8 value =
267214
268215{-# INLINE alt16 #-}
269216alt16 :: MonadIO m
270− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
217+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
271218alt16 value =
272219 PARSE_OP
273220 ( takeWhileFail (<= ( value `div` 16)) Fold.drain
@@ -289,27 +236,27 @@ alt16 value =
289236 )
290237
291238{-# INLINE manyAlt #-}
292−manyAlt :: MonadIO m => StreamK m PARSE_ELEM -> m Int
239+manyAlt :: MonadIO m => StreamK m (Array Int) -> m Int
293240manyAlt xs = do
294241 x <- PARSE_OP (AP.many (satisfy (> 0))) xs
295242 return $ Prelude.length x
296243
297244{-# INLINE someAlt #-}
298−someAlt :: MonadIO m => StreamK m PARSE_ELEM -> m Int
245+someAlt :: MonadIO m => StreamK m (Array Int) -> m Int
299246someAlt xs = do
300247 x <- PARSE_OP (AP.some (satisfy (> 0))) xs
301248 return $ Prelude.length x
302249
303250{-# INLINE choice #-}
304251choice :: MonadIO m =>
305− Int -> StreamK m PARSE_ELEM -> m (Either ParseError Int)
252+ Int -> StreamK m (Array Int) -> m (Either ParseError Int)
306253choice value =
307254 PARSE_OP (asum (replicate value (satisfy (< 0)))
308255 AP.<|> satisfy (> 0))
309256
310257{-# INLINE monad2 #-}
311258monad2 :: MonadIO m
312− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
259+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
313260monad2 value =
314261 PARSE_OP $ do
315262 takeWhile (<= (value `div` 2))
@@ -317,7 +264,7 @@ monad2 value =
317264
318265{-# INLINE monad4 #-}
319266monad4 :: MonadIO m
320− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
267+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
321268monad4 value =
322269 PARSE_OP $ do
323270 takeWhile (<= ( value `div` 4))
@@ -327,7 +274,7 @@ monad4 value =
327274
328275{-# INLINE monad8 #-}
329276monad8 :: MonadIO m
330− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
277+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
331278monad8 value =
332279 PARSE_OP $ do
333280 takeWhile (<= ( value `div` 8))
@@ -341,7 +288,7 @@ monad8 value =
341288
342289{-# INLINE monad16 #-}
343290monad16 :: MonadIO m
344− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
291+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
345292monad16 value =
346293 PARSE_OP $ do
347294 takeWhile (<= ( value `div` 16))
@@ -366,7 +313,7 @@ monad16 value =
366313-------------------------------------------------------------------------------
367314
368315moduleName :: String
369−moduleName = MODULE_NAME
316+moduleName = "Data.ParserK"
370317
371318instance NFData ParseError where
372319 {-# INLINE rnf #-}
@@ -388,7 +335,7 @@ o_1_space_serial value =
388335 ]
389336
390337{-# INLINE sepBy1 #-}
391−sepBy1 :: MonadIO m => StreamK m PARSE_ELEM -> m Int
338+sepBy1 :: MonadIO m => StreamK m (Array Int) -> m Int
392339sepBy1 xs = do
393340 x <- PARSE_OP (parser (satisfy odd) (satisfy even)) xs
394341 return $ Prelude.length x
benchmark/Streamly/Benchmark/Data/Ring/Unboxed.hs+4−3
@@ -12,11 +12,12 @@ module Main (main) where
1212
1313import Control.Monad (void)
1414import GHC.Ptr (Ptr(..))
15−import qualified Streamly.Internal.Data.Array as Array
16−import qualified Streamly.Internal.Data.Ring as Ring
15+
16+import qualified Streamly.Internal.Data.Array.Type as Array
17+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
1718import qualified Data.Foldable as P
1819
19−import Test.Tasty.Bench
20+import Gauge
2021import Streamly.Benchmark.Common
2122
2223import Prelude as P
benchmark/Streamly/Benchmark/Data/Serialize.hsdeleted−591
@@ -1,591 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−{- HLINT ignore -}
3−
4−#undef FUSION_CHECK
5−#ifdef FUSION_CHECK
6−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
7−#endif
8−
9−-- |
10−-- Module : Streamly.Benchmark.Data.Serialize
11−-- Copyright : (c) 2023 Composewell
12−-- License : BSD-3-Clause
13−-- Maintainer : streamly@composewell.com
14−
15−module Main (main) where
16−
17−-------------------------------------------------------------------------------
18−-- Imports
19−-------------------------------------------------------------------------------
20−
21−import Control.DeepSeq (NFData(..), deepseq)
22−import Control.Monad (when, void)
23−import GHC.Generics (Generic)
24−import System.Random (randomRIO)
25−#ifndef USE_UNBOX
26−import Test.QuickCheck (Arbitrary, arbitrary)
27−#endif
28−
29−import Streamly.Internal.Data.MutByteArray
30−import qualified Streamly.Internal.Data.MutByteArray as MBA
31−#ifdef USE_UNBOX
32−import Data.Proxy (Proxy(..))
33−#else
34−import Control.DeepSeq (force)
35−import Test.QuickCheck (oneof, generate)
36−#endif
37−
38−import Test.Tasty.Bench
39−import Streamly.Benchmark.Common
40−
41−#ifndef USE_UNBOX
42−import Streamly.Benchmark.Data.Serialize.RecCompatible
43−import Streamly.Benchmark.Data.Serialize.RecNonCompatible
44−#endif
45−
46−#ifdef USE_UNBOX
47−#define SERIALIZE_CLASS Unbox
48−#define DERIVE_CLASS(typ) $(deriveUnbox [d|instance Unbox typ|])
49−#define SERIALIZE_OP pokeAt
50−#define DESERIALIZE_OP peekAt
51−#else
52−#define SERIALIZE_CLASS Serialize
53−#define DERIVE_CLASS(typ) $(deriveSerialize [d|instance Serialize typ|])
54−#define SERIALIZE_OP serializeAt
55−#define DESERIALIZE_OP deserializeAt
56−#endif
57−
58−-------------------------------------------------------------------------------
59−-- Types
60−-------------------------------------------------------------------------------
61−
62−data Unit = Unit
63− deriving (Generic, Show, Eq)
64−
65−#ifndef USE_TH
66−instance SERIALIZE_CLASS Unit
67−#else
68−DERIVE_CLASS(Unit)
69−#endif
70−
71−instance NFData Unit where
72− {-# INLINE rnf #-}
73− rnf _ = ()
74−
75−data Sum2
76− = Sum21
77− | Sum22
78− deriving (Generic, Show, Eq)
79−
80−#ifndef USE_TH
81−instance SERIALIZE_CLASS Sum2
82−#else
83−DERIVE_CLASS(Sum2)
84−#endif
85−
86−instance NFData Sum2 where
87− {-# INLINE rnf #-}
88− rnf _ = ()
89−
90−data Sum25
91− = Sum251
92− | Sum252
93− | Sum253
94− | Sum254
95− | Sum255
96− | Sum256
97− | Sum257
98− | Sum258
99− | Sum259
100− | Sum2510
101− | Sum2511
102− | Sum2512
103− | Sum2513
104− | Sum2514
105− | Sum2515
106− | Sum2516
107− | Sum2517
108− | Sum2518
109− | Sum2519
110− | Sum2520
111− | Sum2521
112− | Sum2522
113− | Sum2523
114− | Sum2524
115− | Sum2525
116− deriving (Generic, Show, Eq)
117−
118−#ifndef USE_TH
119−instance SERIALIZE_CLASS Sum25
120−#else
121−DERIVE_CLASS(Sum25)
122−#endif
123−
124−instance NFData Sum25 where
125− {-# INLINE rnf #-}
126− rnf _ = ()
127−
128−data Product25
129− = Product25
130− Int
131− Int
132− Int
133− Int
134− Int
135− Int
136− Int
137− Int
138− Int
139− Int
140− Int
141− Int
142− Int
143− Int
144− Int
145− Int
146− Int
147− Int
148− Int
149− Int
150− Int
151− Int
152− Int
153− Int
154− Int
155− deriving (Generic, Show)
156−
157−#ifndef USE_TH
158−instance SERIALIZE_CLASS Product25
159−#else
160−DERIVE_CLASS(Product25)
161−#endif
162−
163−instance NFData Product25 where
164− {-# INLINE rnf #-}
165− rnf
166− (Product25 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20 a21 a22 a23 a24 a25)
167− = a1
168− `seq` a2
169− `seq` a3
170− `seq` a4
171− `seq` a5
172− `seq` a6
173− `seq` a7
174− `seq` a8
175− `seq` a9
176− `seq` a10
177− `seq` a11
178− `seq` a12
179− `seq` a13
180− `seq` a14
181− `seq` a15
182− `seq` a16
183− `seq` a17
184− `seq` a18
185− `seq` a19
186− `seq` a20
187− `seq` a21
188− `seq` a22
189− `seq` a23
190− `seq` a24
191− `seq` a25
192− `seq` ()
193−
194−-- XXX derived Eq instance is not inlined
195−instance Eq Product25 where
196− {-# INLINE (==) #-}
197− (==)
198− (Product25 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20 a21 a22 a23 a24 a25)
199− (Product25 b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 b17 b18 b19 b20 b21 b22 b23 b24 b25)
200− = a1 == b1
201− && a2 == b2
202− && a3 == b3
203− && a4 == b4
204− && a5 == b5
205− && a6 == b6
206− && a7 == b7
207− && a8 == b8
208− && a9 == b9
209− && a10 == b10
210− && a11 == b11
211− && a12 == b12
212− && a13 == b13
213− && a14 == b14
214− && a15 == b15
215− && a16 == b16
216− && a17 == b17
217− && a18 == b18
218− && a19 == b19
219− && a20 == b20
220− && a21 == b21
221− && a22 == b22
222− && a23 == b23
223− && a24 == b24
224− && a25 == b25
225−
226−-------------------------------------------------------------------------------
227−-- Simple non-recursive ADT
228−-------------------------------------------------------------------------------
229−
230−{-
231−data CustomDT1
232− = CDT1C1
233− | CDT1C2 Int
234− | CDT1C3 Int Int
235− deriving (Generic, Show, Eq)
236−
237−#ifndef USE_TH
238−instance SERIALIZE_CLASS CustomDT1
239−#else
240−DERIVE_CLASS(CustomDT1)
241−#endif
242−
243−instance NFData CustomDT1 where
244− {-# INLINE rnf #-}
245− rnf CDT1C1 = ()
246− rnf (CDT1C2 i) = i `seq` ()
247− rnf (CDT1C3 i1 i2) = i1 `seq` i2 `seq` ()
248−-}
249−
250−-------------------------------------------------------------------------------
251−-- Recursive ADT
252−-------------------------------------------------------------------------------
253−
254−#ifndef USE_UNBOX
255−data BinTree a
256− = Tree (BinTree a) (BinTree a)
257− | Leaf a
258− deriving (Show, Read, Eq, Generic)
259−
260−#ifndef USE_TH
261−instance Serialize (BinTree a)
262−#else
263−$(deriveSerialize [d|instance Serialize a => Serialize (BinTree a)|])
264−#endif
265−
266−instance NFData a => NFData (BinTree a) where
267− {-# INLINE rnf #-}
268− rnf (Leaf a) = rnf a `seq` ()
269− rnf (Tree l r) = rnf l `seq` rnf r
270−
271−-- XXX This may not terminate, or could become really large.
272−instance Arbitrary a => Arbitrary (BinTree a) where
273− arbitrary = oneof [Leaf <$> arbitrary, Tree <$> arbitrary <*> arbitrary]
274−
275−mkBinTree :: (Arbitrary a) => Int -> IO (BinTree a)
276−mkBinTree = go (generate arbitrary)
277−
278− where
279−
280− go r 0 = Leaf <$> r
281− go r n = Tree <$> go r (n - 1) <*> go r (n - 1)
282−
283−#endif
284−
285−-------------------------------------------------------------------------------
286−-- Size helpers
287−-------------------------------------------------------------------------------
288−
289−{-# INLINE getSize #-}
290−getSize :: forall a. SERIALIZE_CLASS a => a -> Int
291−#ifdef USE_UNBOX
292−getSize _ = sizeOf (Proxy :: Proxy a)
293−#else
294−getSize = addSizeTo 0
295−#endif
296−
297−-------------------------------------------------------------------------------
298−-- Common helpers
299−-------------------------------------------------------------------------------
300−
301−{- HLINT ignore "Eta reduce" -}
302−-- Parts of "f" that are dependent on val will not be optimized out.
303−{-# INLINE loop #-}
304−loop :: Int -> (a -> IO b) -> a -> IO ()
305−loop count f val = go count val
306− where
307−
308− go n x = when (n > 0) $ f x >> go (n-1) x
309−
310−-- The first arg of "f" is the environment which is not threaded around in the
311−-- loop.
312−{-# INLINE loopWith #-}
313−loopWith :: Int -> (e -> a -> IO b) -> e -> a -> IO ()
314−loopWith count f e val = go count val
315− where
316−
317− go n x = when (n > 0) $ f e x >> go (n-1) x
318−
319−benchSink :: NFData b => String -> Int -> (Int -> IO b) -> Benchmark
320−benchSink name times f = bench name (nfIO (randomRIO (times, times) >>= f))
321−
322−-------------------------------------------------------------------------------
323−-- Serialization Helpers
324−-------------------------------------------------------------------------------
325−
326−{-
327−{-# INLINE pokeWithSize #-}
328−pokeWithSize :: SERIALIZE_CLASS a => MutByteArray -> a -> IO ()
329−pokeWithSize arr val = do
330− let n = getSize val
331− n `seq` SERIALIZE_OP 0 arr val >> return ()
332−
333−{-# INLINE pokeTimesWithSize #-}
334−pokeTimesWithSize :: SERIALIZE_CLASS a => a -> Int -> IO ()
335−pokeTimesWithSize val times = do
336− let n = getSize val
337− arr <- MBA.new n
338− loopWith times pokeWithSize arr val
339−-}
340−
341−{-# INLINE poke #-}
342−poke :: SERIALIZE_CLASS a => MutByteArray -> a -> IO ()
343−poke arr val = void (SERIALIZE_OP 0 arr val)
344−
345−{-# INLINE pokeTimes #-}
346−pokeTimes :: SERIALIZE_CLASS a => a -> Int -> IO ()
347−pokeTimes val times = do
348− let n = getSize val
349− arr <- MBA.new n
350− loopWith times poke arr val
351−
352−{-# INLINE encode #-}
353−encode :: SERIALIZE_CLASS a => a -> IO ()
354−encode val = do
355− let n = getSize val
356− arr <- MBA.new n
357− void (SERIALIZE_OP 0 arr val)
358−
359−{-# INLINE encodeTimes #-}
360−encodeTimes :: SERIALIZE_CLASS a => a -> Int -> IO ()
361−encodeTimes val times = loop times encode val
362−
363−{-# INLINE peek #-}
364−peek :: forall a. (NFData a, SERIALIZE_CLASS a) =>
365− (a, Int) -> MutByteArray -> IO ()
366−#ifdef USE_UNBOX
367−peek (_val, _) arr = do
368− (val1 :: a) <- DESERIALIZE_OP 0 arr
369−#else
370−peek (_val, n) arr = do
371− (_, val1 :: a) <- DESERIALIZE_OP 0 arr n
372−#endif
373− -- If the datatype is not deeply strict or deepseq is not used then use
374− -- Equality.
375− -- Ensure that we are actually constructing the type and using it. This
376− -- is important, otherwise the structure is created and discarded, the
377− -- cost of creation of the structure is not accounted. Otherwise we may
378− -- just read the values and discard them. The comparison adds to the
379− -- cost though. We could use deepseq but then we need to write
380− -- instances of NFData and ensure that they are correct and perform
381− -- well. Equality check also ensures correctness.
382− {-
383− if (val1 /= val)
384− then error "peek: no match"
385− else return ()
386− -}
387− val1 `deepseq` return ()
388−
389−{-# INLINE peekTimes #-}
390−peekTimes :: (NFData a, SERIALIZE_CLASS a) => Int -> a -> Int -> IO ()
391−peekTimes n val times = do
392− arr <- MBA.new n
393− _ <- SERIALIZE_OP 0 arr val
394− loopWith times peek (val, n) arr
395−
396−{-# INLINE trip #-}
397−trip :: forall a. (NFData a, SERIALIZE_CLASS a) => a -> IO ()
398−trip val = do
399− let n = getSize val
400− arr <- MBA.new n
401− _ <- SERIALIZE_OP 0 arr val
402−#ifdef USE_UNBOX
403− (val1 :: a) <- DESERIALIZE_OP 0 arr
404−#else
405− (_, val1 :: a) <- DESERIALIZE_OP 0 arr n
406−#endif
407− -- Do not remove this or use deepseq, see the comments in peek.
408− {-
409− if (val1 /= val)
410− then error "roundtrip: no match"
411− else return ()
412− -}
413− -- Note: deepseq is not needed if the datatype is strict
414− val1 `deepseq` return ()
415−
416−{-# INLINE roundtrip #-}
417−roundtrip :: (NFData a, SERIALIZE_CLASS a) => a -> Int -> IO ()
418−roundtrip val times = loop times trip val
419−
420−-------------------------------------------------------------------------------
421−-- Benchmarks
422−-------------------------------------------------------------------------------
423−
424−{-# INLINE benchConst #-}
425−benchConst ::
426− String
427− -> (forall a. (NFData a, SERIALIZE_CLASS a) =>
428− Int -> a -> Int -> IO ())
429− -> Int
430− -> Benchmark
431−benchConst gname f times =
432− bgroup gname
433− [ let !n = getSize Unit
434− in benchSink "Unit" times (f n Unit)
435− {-
436− , let !n = getSize CDT1C1
437− in benchSink "C1" times (f n CDT1C1)
438− , let val = CDT1C2 5
439− !n = getSize val
440− in benchSink "C2" (times `div` 2) (f n val)
441− , let val = CDT1C3 5 2
442− !n = getSize val
443− in benchSink "C3" (times `div` 3) (f n val)
444− -}
445− , let !n = getSize Sum21
446− in benchSink "Sum2" times (f n Sum21)
447− , let !n = getSize Sum2525
448− in benchSink "Sum25" times (f n Sum2525)
449− , let val = Product25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
450− !n = getSize val
451− in benchSink "Product25" (times `div` 26) (f n val)
452− ]
453−
454−#ifndef USE_UNBOX
455−{-# INLINE benchVar #-}
456−benchVar ::
457− String
458− -> (forall a. (NFData a, SERIALIZE_CLASS a) =>
459− Int -> a -> Int -> IO ())
460− -> BinTree Int
461− -> [Int]
462− -> Int
463− -> Benchmark
464−benchVar gname f tInt lInt times =
465− bgroup gname
466− [ let !n = getSize tInt
467− in benchSink "bintree-int" times (f n tInt)
468− , let !n = getSize lInt
469− in benchSink "list-int" times (f n lInt)
470− ]
471−
472−{-
473−{-# INLINE benchGeneric #-}
474−benchGeneric
475− :: (NFData b, Serialize b)
476− => String
477− -> String
478− -> (forall a. (NFData a, Serialize a) =>
479− Int -> a -> Int -> IO ())
480− -> b
481− -> Int
482− -> Benchmark
483−benchGeneric gname valId f val times =
484− bgroup gname
485− [ let !n = getSize val
486− in benchSink valId times (f n val)
487− ]
488−-}
489−
490−{-# INLINE benchLargeRec #-}
491−benchLargeRec ::
492− String
493− -> (forall a. (NFData a, Serialize a) =>
494− Int -> a -> Int -> IO ())
495− -> RecNonCompatible
496− -> RecCompatible
497− -> Int
498− -> Benchmark
499−benchLargeRec gname f recL recR times =
500− bgroup gname
501− [ let !n = getSize recL
502− in benchSink "RecNonCompatible" times (f n recL)
503− , let !n = getSize recR
504− in benchSink "RecCompatible" times (f n recR)
505− ]
506−#endif
507−
508−-- Times is scaled by the number of constructors to normalize
509−#ifdef USE_UNBOX
510−allBenchmarks :: Int -> [Benchmark]
511−allBenchmarks times =
512−#else
513−allBenchmarks ::
514− BinTree Int -> [Int] -> RecNonCompatible -> RecCompatible -> Int -> [Benchmark]
515−allBenchmarks tInt lInt recL recR times =
516−#endif
517− [ bgroup "sizeOf"
518− [
519−#ifndef USE_UNBOX
520− bench "bintree-int" $ nf getSize tInt
521− , bench "list-int" $ nf getSize lInt
522−#endif
523− ]
524− , benchConst "poke" (const pokeTimes) times
525− -- , benchConst "pokeWithSize" (const pokeTimesWithSize) times
526− , benchConst "encode" (const encodeTimes) times
527− , benchConst "peek" peekTimes times
528− , benchConst "roundtrip" (const roundtrip) times
529−#ifndef USE_UNBOX
530− , benchVar "poke" (const pokeTimes) tInt lInt 1
531− -- , benchVar "pokeWithSize" (const pokeTimesWithSize) tInt lInt 1
532− , benchVar "encode" (const encodeTimes) tInt lInt 1
533− , benchVar "peek" peekTimes tInt lInt 1
534− , benchVar "roundtrip" (const roundtrip) tInt lInt 1
535− , benchLargeRec "poke" (const pokeTimes) recL recR times
536− , benchLargeRec "encode" (const encodeTimes) recL recR times
537− , benchLargeRec "peek" peekTimes recL recR times
538− , benchLargeRec "roundtrip" (const roundtrip) recL recR times
539−#endif
540− ]
541−
542−-------------------------------------------------------------------------------
543−-- Driver
544−-------------------------------------------------------------------------------
545−
546−main :: IO ()
547−main = do
548−#ifndef USE_UNBOX
549− -- Approximately 100000 constructors
550− -- Assuming Leaf nodes constitute two constructors (Leaf, Int) and internal
551− -- nodes 1 level = log_2 (100001/3) + 1 = 16
552− !(tInt :: BinTree Int) <- force <$> mkBinTree 16
553−
554− let !recL = valRecNonCompatible
555− let !recR = valRecCompatible
556−
557− -- Approximately 100000 constructors, assuming two constructors (Cons, Int)
558− -- per element.
559− let lInt = [1..50000 :: Int]
560− let !len = length lInt -- evaluate the list
561−#endif
562−#ifndef FUSION_CHECK
563−#ifdef USE_UNBOX
564− runWithCLIOpts defaultStreamSize allBenchmarks
565−#else
566− len
567− `seq` runWithCLIOpts
568− defaultStreamSize
569− (allBenchmarks tInt lInt recL recR)
570−#endif
571−#else
572− -- Enable FUSION_CHECK macro at the beginning of the file
573− -- Enable one benchmark below, and run the benchmark
574− -- Check the .dump-simpl output
575− let value = 100000
576− -- print $ getSize (CDT1C3 4 2)
577− -- print $ getSize tInt
578− -- print $ getSize lInt
579−
580− -- pokeTimes tInt 1
581−
582− -- peekTimes ((CDT1C2 (5 :: Int)) :: CustomDT1) value
583− -- peekTimes (Sum2525) value
584− -- peekTimes (Product25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25) value
585− -- peekTimes tInt 1
586− let !n = getSize lInt
587− peekTimes n lInt 1
588−
589− -- roundtrip ((CDT1C2 (5 :: Int)) :: CustomDT1) value
590− return ()
591−#endif
benchmark/Streamly/Benchmark/Data/Serialize/RecCompatible.hsdeleted−24
@@ -1,24 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.RecCompatible
4− ( RecCompatible(..)
5− , valRecCompatible
6− ) where
7−
8−--------------------------------------------------------------------------------
9−-- Imports
10−--------------------------------------------------------------------------------
11−
12−import Streamly.Internal.Data.MutByteArray (Serialize)
13−import Streamly.Benchmark.Data.Serialize.TH (genLargeRecord)
14−
15−import qualified Streamly.Internal.Data.MutByteArray as Serialize
16−
17−--------------------------------------------------------------------------------
18−-- Code
19−--------------------------------------------------------------------------------
20−
21−$(genLargeRecord "RecCompatible" 50)
22−$(Serialize.deriveSerializeWith
23− (Serialize.encodeRecordFields True)
24− [d|instance Serialize RecCompatible|])
benchmark/Streamly/Benchmark/Data/Serialize/RecNonCompatible.hsdeleted−21
@@ -1,21 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.RecNonCompatible
4− ( RecNonCompatible(..)
5− , valRecNonCompatible
6− ) where
7−
8−--------------------------------------------------------------------------------
9−-- Imports
10−--------------------------------------------------------------------------------
11−
12−import Streamly.Benchmark.Data.Serialize.TH (genLargeRecord)
13−
14−import qualified Streamly.Internal.Data.MutByteArray as Serialize
15−
16−--------------------------------------------------------------------------------
17−-- Code
18−--------------------------------------------------------------------------------
19−
20−$(genLargeRecord "RecNonCompatible" 50)
21−$(Serialize.deriveSerialize [d|instance Serialize.Serialize RecNonCompatible|])
benchmark/Streamly/Benchmark/Data/Serialize/TH.hsdeleted−76
@@ -1,76 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.TH (genLargeRecord) where
4−
5−--------------------------------------------------------------------------------
6−-- Imports
7−--------------------------------------------------------------------------------
8−
9−import Language.Haskell.TH
10−
11−import Streamly.Internal.Data.MutByteArray (makeI)
12−import Control.DeepSeq (NFData(..))
13−
14−--------------------------------------------------------------------------------
15−-- Large Record
16−--------------------------------------------------------------------------------
17−
18−genLargeRecord :: String -> Int -> Q [Dec]
19−genLargeRecord tyName numFields =
20− sequence
21− [ dataD
22− (pure [])
23− (mkName tyName)
24− []
25− Nothing
26− [mkCon tyName]
27− [derivClause Nothing [conT ''Eq, conT ''Show]]
28− , mkValueSigDec
29− , mkValueDec
30− , nfDataInstance tyName
31− ]
32−
33− where
34−
35− fieldTypeChoices = [conT ''()]
36− chooseCycle i xs = xs !! (i `mod` length xs)
37− nfDataInstance nm =
38− instanceD
39− (pure [])
40− (appT (conT ''NFData) (conT (mkName nm)))
41− [ funD
42− 'rnf
43− [ clause
44− [ conP
45− (mkName nm)
46− (varP . makeI <$> [0 .. (numFields - 1)])
47− ]
48− (normalB
49− (foldl
50− (\b a -> [|rnf $(b) `seq` rnf $(a)|])
51− [|()|]
52− (varE . makeI <$> [0 .. (numFields - 1)])))
53− []
54− ]
55− ]
56− valueName = mkName $ "val" ++ tyName
57− mkValueSigDec = sigD valueName [t|$(conT (mkName tyName))|]
58− mkValueDec =
59− funD
60− valueName
61− [ clause
62− []
63− (normalB
64− (foldl
65− (\b a -> [|$(b) $(a)|])
66− (conE (mkName tyName))
67− (conE '() <$ [0 .. (numFields - 1)])))
68− []
69− ]
70− mkCon nm = recC (mkName nm) (mkField <$> [0 .. (numFields - 1)])
71− mkField i =
72− varBangType
73− (mkName ("field" ++ show i))
74− (bangType
75− (bang noSourceUnpackedness noSourceStrictness)
76− (chooseCycle i fieldTypeChoices))
benchmark/Streamly/Benchmark/Data/Stream/Common.hs+1−3
@@ -71,9 +71,7 @@ module Stream.Common
7171 )
7272where
7373
74−#if !MIN_VERSION_base(4,18,0)
7574import Control.Applicative (liftA2)
76−#endif
7775import Control.DeepSeq (NFData)
7876import Control.Exception (try)
7977import GHC.Exception (ErrorCall)
@@ -104,7 +102,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
104102#endif
105103#endif
106104
107−import Test.Tasty.Bench
105+import Gauge
108106import Prelude hiding (Foldable(..), mapM, replicate)
109107
110108#ifdef USE_STREAMK
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentCommon.hs+3−3
@@ -14,14 +14,14 @@ where
1414import Stream.Common
1515 (composeN, benchIO, benchIOSink, benchIOSrc, sourceUnfoldrM)
1616import Streamly.Data.Stream (Stream)
17−import Streamly.Internal.Data.Stream.Prelude (MonadAsync, Config)
17+import Streamly.Internal.Data.Stream.Concurrent (MonadAsync, Config)
1818
1919import qualified Data.List as List
2020import qualified Streamly.Data.Fold as Fold
2121import qualified Streamly.Data.Stream as Stream
22−import qualified Streamly.Internal.Data.Stream.Prelude as Async
22+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
2323
24−import Test.Tasty.Bench
24+import Gauge
2525import Prelude hiding (mapM)
2626import Streamly.Benchmark.Common
2727
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentEager.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentEager"
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentInterleaved.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentInterleaved"
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentOrdered.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentOrdered"
benchmark/Streamly/Benchmark/Data/Stream/Eliminate.hs+1−3
@@ -57,7 +57,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
5757#endif
5858#endif
5959
60−import Test.Tasty.Bench
60+import Gauge
6161import Streamly.Benchmark.Common
6262import Prelude hiding (length, sum, or, and, any, all, notElem, elem, (!!),
6363 lookup, repeat, minimum, maximum, product, last, mapM_, init)
@@ -132,14 +132,12 @@ foldableAny :: Int -> Int -> Bool
132132foldableAny value n =
133133 Prelude.any (> (value + 1)) (sourceUnfoldr value n :: Stream Identity Int)
134134
135−{- HLINT ignore "Use all"-}
136135{-# INLINE foldableAnd #-}
137136foldableAnd :: Int -> Int -> Bool
138137foldableAnd value n =
139138 Prelude.and $ fmap
140139 (<= (value + 1)) (sourceUnfoldr value n :: Stream Identity Int)
141140
142−{- HLINT ignore "Use any"-}
143141{-# INLINE foldableOr #-}
144142foldableOr :: Int -> Int -> Bool
145143foldableOr value n =
benchmark/Streamly/Benchmark/Data/Stream/Exceptions.hs+4−14
@@ -36,22 +36,21 @@ import System.IO (Handle, hClose, hPutChar)
3636import qualified Streamly.FileSystem.Handle as FH
3737import qualified Streamly.Internal.FileSystem.Handle as IFH
3838import qualified Streamly.Internal.Data.Unfold as IUF
39−import qualified Streamly.Internal.Data.Unfold.Prelude as IUF
39+import qualified Streamly.Internal.Data.Unfold.Exception as IUF
4040
4141#ifdef USE_PRELUDE
4242import qualified Streamly.Internal.Data.Stream.IsStream as Stream
4343#else
4444import qualified Streamly.Internal.Data.Stream as Stream
45−import qualified Streamly.Internal.Data.Stream.Prelude as Stream
45+import qualified Streamly.Internal.Data.Stream.Exception.Lifted as Stream
4646#endif
4747
48−import Test.Tasty.Bench hiding (env)
48+import Gauge hiding (env)
4949import Prelude hiding (last, length)
5050import Streamly.Benchmark.Common
5151import Streamly.Benchmark.Common.Handle
5252
5353#ifdef INSPECTION
54−import Control.Monad.Catch (MonadCatch)
5554import Test.Inspection
5655
5756import qualified Streamly.Internal.Data.Stream as D
@@ -178,8 +177,7 @@ inspect $ hasNoTypeClasses 'readWriteOnExceptionStream
178177-- | Send the file contents to /dev/null with exception handling
179178readWriteHandleExceptionStream :: Handle -> Handle -> IO ()
180179readWriteHandleExceptionStream inh devNull =
181− let handler (_e :: SomeException) =
182− return $ Stream.fromEffect (hClose inh >> return 10)
180+ let handler (_e :: SomeException) = Stream.fromEffect (hClose inh >> return 10)
183181 readEx = Stream.handle handler (Stream.unfold FH.reader inh)
184182 in Stream.fold (FH.write devNull) readEx
185183
@@ -287,12 +285,8 @@ readChunksOnException inh devNull =
287285 in IUF.fold (IFH.writeChunks devNull) readEx inh
288286
289287#ifdef INSPECTION
290−#if __GLASGOW_HASKELL__ >= 906
291−inspect $ hasNoTypeClassesExcept 'readChunksOnException [''MonadCatch]
292−#else
293288inspect $ hasNoTypeClasses 'readChunksOnException
294289#endif
295−#endif
296290
297291-- | Send the file contents to /dev/null with exception handling
298292readChunksBracket_ :: Handle -> Handle -> IO ()
@@ -301,12 +295,8 @@ readChunksBracket_ inh devNull =
301295 in IUF.fold (IFH.writeChunks devNull) readEx inh
302296
303297#ifdef INSPECTION
304−#if __GLASGOW_HASKELL__ >= 906
305−inspect $ hasNoTypeClassesExcept 'readChunksBracket_ [''MonadCatch]
306−#else
307298inspect $ hasNoTypeClasses 'readChunksBracket_
308299#endif
309−#endif
310300
311301readChunksBracket :: Handle -> Handle -> IO ()
312302readChunksBracket inh devNull =
benchmark/Streamly/Benchmark/Data/Stream/Expand.hs+1−9
@@ -54,7 +54,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
5454#endif
5555#endif
5656
57−import Test.Tasty.Bench
57+import Gauge
5858import Stream.Common
5959import Streamly.Benchmark.Common
6060import Prelude hiding (concatMap)
@@ -198,11 +198,7 @@ concatMapPure outer inner n =
198198 (fromStream $ sourceUnfoldr outer n)
199199
200200#ifdef INSPECTION
201−#if __GLASGOW_HASKELL__ >= 906
202−inspect $ hasNoTypeClassesExcept 'concatMapPure [''Applicative]
203−#else
204201inspect $ hasNoTypeClasses 'concatMapPure
205−#endif
206202inspect $ 'concatMapPure `hasNoType` ''SPEC
207203#endif
208204
@@ -215,11 +211,7 @@ concatMapRepl outer inner n =
215211 (fromStream . S.replicate inner) (fromStream $ sourceUnfoldrM outer n)
216212
217213#ifdef INSPECTION
218−#if __GLASGOW_HASKELL__ >= 906
219−inspect $ hasNoTypeClassesExcept 'concatMapRepl [''Applicative]
220−#else
221214inspect $ hasNoTypeClasses 'concatMapRepl
222−#endif
223215inspect $ 'concatMapRepl `hasNoType` ''SPEC
224216#endif
225217
benchmark/Streamly/Benchmark/Data/Stream/Generate.hs+1−1
@@ -40,7 +40,7 @@ import qualified Stream.Common as Common
4040#endif
4141#endif
4242
43−import Test.Tasty.Bench
43+import Gauge
4444import Streamly.Benchmark.Common
4545import qualified Prelude
4646
benchmark/Streamly/Benchmark/Data/Stream/Lift.hs+1−1
@@ -37,7 +37,7 @@ import qualified Control.Monad.State.Strict as State
3737#endif
3838#endif
3939
40−import Test.Tasty.Bench
40+import Gauge
4141import Streamly.Benchmark.Common
4242import Prelude hiding (reverse, tail)
4343
benchmark/Streamly/Benchmark/Data/Stream/Reduce.hs+2−2
@@ -48,7 +48,7 @@ import Streamly.Internal.Data.Stream (Stream)
4848import qualified Streamly.Internal.Data.Stream as S
4949#ifndef USE_STREAMLY_CORE
5050import qualified Streamly.Data.Stream.Prelude as S
51−import qualified Streamly.Internal.Data.Stream.Prelude as S
51+import qualified Streamly.Internal.Data.Stream.Time as S
5252#endif
5353
5454#ifdef USE_STREAMK
@@ -61,7 +61,7 @@ import qualified Streamly.Internal.Data.Stream as K
6161
6262#endif
6363
64−import Test.Tasty.Bench
64+import Gauge
6565import Streamly.Benchmark.Common
6666import Stream.Common
6767import Prelude hiding (reverse, tail)
benchmark/Streamly/Benchmark/Data/Stream/Split.hs+3−3
@@ -35,15 +35,15 @@ import qualified Streamly.Internal.FileSystem.Handle as IFH
3535import qualified Streamly.Internal.Unicode.Stream as IUS
3636import qualified Streamly.Prelude as S
3737
38−import Test.Tasty.Bench hiding (env)
38+import Gauge hiding (env)
3939import Prelude hiding (last, length)
4040import Streamly.Benchmark.Common
4141import Streamly.Benchmark.Common.Handle
4242
4343#ifdef INSPECTION
44−import Streamly.Internal.Data.Stream (Step(..))
44+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..))
4545
46−import qualified Streamly.Internal.Data.MutArray as MA
46+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
4747import qualified Streamly.Internal.Data.Unfold as IUF
4848
4949import Test.Inspection
benchmark/Streamly/Benchmark/Data/Stream/StreamD.hs+2−4
@@ -22,12 +22,10 @@
2222
2323module Main (main) where
2424
25−#if !MIN_VERSION_base(4,18,0)
2625import Control.Applicative (liftA2)
27−#endif
2826import Control.Monad (when)
2927import Data.Maybe (isJust)
30−import Test.Tasty.Bench (bench, nfIO, bgroup, Benchmark, defaultMain)
28+import Gauge (bench, nfIO, bgroup, Benchmark, defaultMain)
3129import System.Random (randomRIO)
3230import Prelude hiding (tail, mapM_, foldl, last, map, mapM, concatMap, zip)
3331
@@ -463,7 +461,7 @@ inspect $ 'unfoldManyRepl `hasNoType` ''SPEC
463461-------------------------------------------------------------------------------
464462
465463moduleName :: String
466−moduleName = "Data.StreamD"
464+moduleName = "Data.Stream.StreamD"
467465
468466o_1_space :: [Benchmark]
469467o_1_space =
benchmark/Streamly/Benchmark/Data/Stream/StreamKAlt.hs → benchmark/Streamly/Benchmark/Data/Stream/StreamDK.hsrenamed+3−3
@@ -18,12 +18,12 @@ import Prelude hiding ()
1818-- import qualified Prelude as P
1919-- import qualified Data.List as List
2020
21−import qualified Streamly.Internal.Data.StreamK.Alt as S
21+import qualified Streamly.Internal.Data.Stream.StreamDK as S
2222-- import qualified Streamly.Internal.Data.Stream.Common as SP
2323-- import qualified Streamly.Internal.Data.SVar as S
2424
2525import Streamly.Benchmark.Common
26−import Test.Tasty.Bench (bgroup, Benchmark, defaultMain)
26+import Gauge (bgroup, Benchmark, defaultMain)
2727
2828value :: Int
2929value = 100000
@@ -452,4 +452,4 @@ o_1_space =
452452 ]
453453
454454main :: IO ()
455−main = defaultMain [o_1_space]
455+main = defaultMain $ concat [o_1_space]
benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs+3−4
@@ -21,9 +21,7 @@
2121
2222module Main (main) where
2323
24−#if !MIN_VERSION_base(4,18,0)
2524import Control.Applicative (liftA2)
26−#endif
2725import Control.Monad (when)
2826import Data.Maybe (isJust)
2927import System.Random (randomRIO)
@@ -35,9 +33,10 @@ import Prelude hiding
3533import qualified Prelude as P
3634import qualified Data.List as List
3735
36+import qualified Streamly.Internal.Data.Stream.StreamK.Type as S
3837import qualified Streamly.Internal.Data.StreamK as S
3938
40−import Test.Tasty.Bench (bench, nfIO, bgroup, Benchmark, defaultMain)
39+import Gauge (bench, nfIO, bgroup, Benchmark, defaultMain)
4140
4241import Streamly.Benchmark.Common
4342
@@ -634,7 +633,7 @@ filterAllInNestedList str = do
634633-------------------------------------------------------------------------------
635634
636635moduleName :: String
637−moduleName = "Data.StreamK"
636+moduleName = "Data.Stream.StreamK"
638637
639638o_1_space_generation :: Int -> Benchmark
640639o_1_space_generation streamLen =
benchmark/Streamly/Benchmark/Data/Stream/ToStreamK.hs+4−3
@@ -1,5 +1,5 @@
11-- |
2−-- Module : Streamly.Benchmark.Data.StreamK
2+-- Module : Streamly.Benchmark.Data.Stream.StreamK
33-- Copyright : (c) 2018 Composewell Technologies
44--
55-- License : BSD3
@@ -33,11 +33,12 @@ import Prelude hiding
3333import qualified Prelude as P
3434-- import qualified Data.List as List
3535
36+import qualified Streamly.Internal.Data.Stream.StreamK.Type as S
3637import qualified Streamly.Internal.Data.StreamK as S
3738import qualified Streamly.Internal.Data.Stream as D
3839import qualified Streamly.Internal.Data.Fold as Fold
3940
40−import Test.Tasty.Bench (bgroup, Benchmark, defaultMain)
41+import Gauge (bgroup, Benchmark, defaultMain)
4142
4243import Streamly.Benchmark.Common
4344
@@ -120,7 +121,7 @@ fromFoldable streamLen n = S.fromFoldable [n..n+streamLen]
120121{-# INLINE fromFoldableM #-}
121122fromFoldableM :: Monad m => Int -> Int -> Stream m Int
122123fromFoldableM streamLen n =
123− Prelude.foldr (S.consM . return) S.nil [n .. n + streamLen]
124+ Prelude.foldr S.consM S.nil (Prelude.fmap return [n..n+streamLen])
124125
125126{-
126127{-# INLINABLE concatMapFoldableWith #-}
benchmark/Streamly/Benchmark/Data/Stream/Transform.hs+2−2
@@ -46,7 +46,7 @@ import Streamly.Internal.Data.Time.Units
4646import Streamly.Internal.Data.Stream (Stream)
4747import qualified Streamly.Internal.Data.Stream as Stream
4848#ifndef USE_STREAMLY_CORE
49−import qualified Streamly.Internal.Data.Stream.Prelude as Stream
49+import qualified Streamly.Internal.Data.Stream.Time as Stream
5050#endif
5151#ifdef USE_STREAMK
5252import Control.DeepSeq (NFData(..))
@@ -58,7 +58,7 @@ import qualified Streamly.Internal.Data.StreamK as StreamK
5858#endif
5959#endif
6060
61−import Test.Tasty.Bench
61+import Gauge
6262import Stream.Common hiding (scanl')
6363import Streamly.Benchmark.Common
6464import Prelude hiding (sequence, mapM)
benchmark/Streamly/Benchmark/Data/Unfold.hs+2−11
@@ -31,18 +31,17 @@ import qualified Prelude
3131import qualified Streamly.FileSystem.Handle as FH
3232import qualified Streamly.Internal.Data.Fold as FL
3333import qualified Streamly.Internal.Data.Unfold as UF
34−import qualified Streamly.Internal.Data.Unfold.Prelude as UF
34+import qualified Streamly.Internal.Data.Unfold.Exception as UF
3535import qualified Streamly.Internal.Data.Stream as S
3636import qualified Streamly.Internal.Data.Stream as D
3737import qualified Streamly.Internal.Data.StreamK as K
3838
39−import Test.Tasty.Bench hiding (env)
39+import Gauge hiding (env)
4040import Prelude hiding (take, filter, zipWith, map, mapM, takeWhile)
4141import Streamly.Benchmark.Common
4242import Streamly.Benchmark.Common.Handle
4343
4444#ifdef INSPECTION
45−import Control.Monad.Catch (MonadCatch)
4645import Test.Inspection
4746#endif
4847
@@ -777,11 +776,7 @@ readWriteFinally_Unfold inh devNull =
777776 in S.fold (FH.write devNull) $ S.unfold readEx inh
778777
779778#ifdef INSPECTION
780−#if __GLASGOW_HASKELL__ >= 906
781−inspect $ hasNoTypeClassesExcept 'readWriteFinally_Unfold [''MonadCatch]
782−#else
783779inspect $ hasNoTypeClasses 'readWriteFinally_Unfold
784−#endif
785780-- inspect $ 'readWriteFinallyUnfold `hasNoType` ''Step
786781#endif
787782
@@ -797,11 +792,7 @@ readWriteBracket_Unfold inh devNull =
797792 in S.fold (FH.write devNull) $ S.unfold readEx inh
798793
799794#ifdef INSPECTION
800−#if __GLASGOW_HASKELL__ >= 906
801−inspect $ hasNoTypeClassesExcept 'readWriteBracket_Unfold [''MonadCatch]
802−#else
803795inspect $ hasNoTypeClasses 'readWriteBracket_Unfold
804−#endif
805796-- inspect $ 'readWriteBracketUnfold `hasNoType` ''Step
806797#endif
807798
benchmark/Streamly/Benchmark/FileSystem/Handle.hs+1−1
@@ -22,7 +22,7 @@ import Streamly.Benchmark.Common.Handle (mkHandleBenchEnv)
2222import qualified Handle.ReadWrite as RW
2323import qualified Handle.Read as RO
2424
25−import Test.Tasty.Bench hiding (env)
25+import Gauge hiding (env)
2626import Prelude hiding (last, length)
2727import Streamly.Benchmark.Common
2828
benchmark/Streamly/Benchmark/FileSystem/Handle/Read.hs+10−9
@@ -33,6 +33,7 @@ import qualified Streamly.Data.Stream as Stream
3333import qualified Streamly.Data.Fold as Fold
3434import qualified Streamly.FileSystem.Handle as FH
3535import qualified Streamly.Internal.Data.Array as A
36+import qualified Streamly.Internal.Data.Array.Type as AT
3637import qualified Streamly.Internal.Data.Fold as FL
3738import qualified Streamly.Internal.Data.Stream as IP
3839import qualified Streamly.Internal.FileSystem.Handle as IFH
@@ -40,14 +41,14 @@ import qualified Streamly.Internal.Unicode.Stream as IUS
4041import qualified Streamly.Data.Stream.Prelude as S
4142import qualified Streamly.Unicode.Stream as SS
4243
43−import Test.Tasty.Bench hiding (env)
44+import Gauge hiding (env)
4445import Prelude hiding (last, length)
4546import Streamly.Benchmark.Common.Handle
4647
4748#ifdef INSPECTION
48−import Streamly.Internal.Data.Stream (Step(..), FoldMany)
49+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..), FoldMany)
4950
50−import qualified Streamly.Internal.Data.MutArray as MutArray
51+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
5152import qualified Streamly.Internal.Data.Stream as D
5253import qualified Streamly.Internal.Data.Unfold as IUF
5354
@@ -68,7 +69,7 @@ readLast = S.fold Fold.last . S.unfold FH.reader
6869inspect $ hasNoTypeClasses 'readLast
6970inspect $ 'readLast `hasNoType` ''Step -- S.unfold
7071inspect $ 'readLast `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
71−inspect $ 'readLast `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
72+inspect $ 'readLast `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
7273#endif
7374
7475-- assert that flattenArrays constructors are not present
@@ -80,7 +81,7 @@ readCountBytes = S.fold Fold.length . S.unfold FH.reader
8081inspect $ hasNoTypeClasses 'readCountBytes
8182inspect $ 'readCountBytes `hasNoType` ''Step -- S.unfold
8283inspect $ 'readCountBytes `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
83−inspect $ 'readCountBytes `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
84+inspect $ 'readCountBytes `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
8485#endif
8586
8687-- | Count the number of lines in a file.
@@ -95,7 +96,7 @@ readCountLines =
9596inspect $ hasNoTypeClasses 'readCountLines
9697inspect $ 'readCountLines `hasNoType` ''Step
9798inspect $ 'readCountLines `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
98−inspect $ 'readCountLines `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
99+inspect $ 'readCountLines `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
99100#endif
100101
101102-- | Count the number of words in a file.
@@ -119,7 +120,7 @@ readSumBytes = S.fold Fold.sum . S.unfold FH.reader
119120inspect $ hasNoTypeClasses 'readSumBytes
120121inspect $ 'readSumBytes `hasNoType` ''Step
121122inspect $ 'readSumBytes `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
122−inspect $ 'readSumBytes `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
123+inspect $ 'readSumBytes `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
123124#endif
124125
125126-- XXX When we mark this with INLINE and we have two benchmarks using S.drain
@@ -235,13 +236,13 @@ groupsOf :: Int -> Handle -> IO Int
235236groupsOf n inh =
236237 -- writeNUnsafe gives 2.5x boost here over writeN.
237238 S.fold Fold.length
238− $ IP.groupsOf n (A.writeNUnsafe n) (S.unfold FH.reader inh)
239+ $ IP.groupsOf n (AT.writeNUnsafe n) (S.unfold FH.reader inh)
239240
240241#ifdef INSPECTION
241242inspect $ hasNoTypeClasses 'groupsOf
242243inspect $ 'groupsOf `hasNoType` ''Step
243244inspect $ 'groupsOf `hasNoType` ''FoldMany
244−inspect $ 'groupsOf `hasNoType` ''MutArray.ArrayUnsafe -- AT.writeNUnsafe
245+inspect $ 'groupsOf `hasNoType` ''AT.ArrayUnsafe -- AT.writeNUnsafe
245246 -- FH.read/A.read
246247inspect $ 'groupsOf `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
247248#endif
benchmark/Streamly/Benchmark/FileSystem/Handle/ReadWrite.hs+12−11
@@ -34,15 +34,16 @@ import qualified Streamly.Internal.FileSystem.Handle as IFH
3434import qualified Streamly.Data.Array as A
3535import qualified Streamly.Data.Stream.Prelude as S
3636
37−import Test.Tasty.Bench hiding (env)
37+import Gauge hiding (env)
3838import Streamly.Benchmark.Common.Handle
3939
4040#ifdef INSPECTION
41−import Streamly.Internal.Data.Stream (Step(..))
41+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..))
4242
43−import qualified Streamly.Internal.Data.Stream as D
43+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
4444import qualified Streamly.Internal.Data.Tuple.Strict as Strict
45−import qualified Streamly.Internal.Data.MutArray as MutArray
45+import qualified Streamly.Internal.Data.Array.Mut.Stream as MAS
46+import qualified Streamly.Internal.Data.Array.Type as AT
4647
4748import Test.Inspection
4849#endif
@@ -82,7 +83,7 @@ copyStream inh outh = S.fold (FH.write outh) (S.unfold FH.reader inh)
8283inspect $ hasNoTypeClasses 'copyStream
8384inspect $ 'copyStream `hasNoType` ''Step -- S.unfold
8485inspect $ 'copyStream `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
85−inspect $ 'copyStream `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
86+inspect $ 'copyStream `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
8687 -- FH.read/A.read
8788inspect $ 'copyStream `hasNoType` ''Strict.Tuple3' -- FH.write/chunksOf
8889#endif
@@ -108,8 +109,8 @@ readFromBytesNull inh devNull = IFH.putBytes devNull $ S.unfold FH.reader inh
108109#ifdef INSPECTION
109110inspect $ hasNoTypeClasses 'readFromBytesNull
110111inspect $ 'readFromBytesNull `hasNoType` ''Step
111−inspect $ 'readFromBytesNull `hasNoType` ''MutArray.SpliceState
112−inspect $ 'readFromBytesNull `hasNoType` ''MutArray.ArrayUnsafe -- FH.fromBytes/S.chunksOf
112+inspect $ 'readFromBytesNull `hasNoType` ''MAS.SpliceState
113+inspect $ 'readFromBytesNull `hasNoType` ''AT.ArrayUnsafe -- FH.fromBytes/S.chunksOf
113114inspect $ 'readFromBytesNull `hasNoType` ''D.FoldMany
114115#endif
115116
@@ -122,8 +123,8 @@ readWithFromBytesNull inh devNull =
122123#ifdef INSPECTION
123124inspect $ hasNoTypeClasses 'readWithFromBytesNull
124125inspect $ 'readWithFromBytesNull `hasNoType` ''Step
125−inspect $ 'readWithFromBytesNull `hasNoType` ''MutArray.SpliceState
126−inspect $ 'readWithFromBytesNull `hasNoType` ''MutArray.ArrayUnsafe -- FH.fromBytes/S.chunksOf
126+inspect $ 'readWithFromBytesNull `hasNoType` ''MAS.SpliceState
127+inspect $ 'readWithFromBytesNull `hasNoType` ''AT.ArrayUnsafe -- FH.fromBytes/S.chunksOf
127128inspect $ 'readWithFromBytesNull `hasNoType` ''D.FoldMany
128129#endif
129130
@@ -170,7 +171,7 @@ writeReadWith inh devNull = IUF.fold fld unf (defaultChunkSize, inh)
170171inspect $ hasNoTypeClasses 'writeReadWith
171172inspect $ 'writeReadWith `hasNoType` ''Step
172173inspect $ 'writeReadWith `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
173−inspect $ 'writeReadWith `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
174+inspect $ 'writeReadWith `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
174175 -- FH.read/A.read
175176#endif
176177
@@ -187,7 +188,7 @@ writeRead inh devNull = IUF.fold fld unf inh
187188inspect $ hasNoTypeClasses 'writeRead
188189inspect $ 'writeRead `hasNoType` ''Step
189190inspect $ 'writeRead `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
190−inspect $ 'writeRead `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
191+inspect $ 'writeRead `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
191192 -- FH.read/A.read
192193#endif
193194
benchmark/Streamly/Benchmark/Prelude/Adaptive.hs+1−1
@@ -10,7 +10,7 @@
1010import Control.Concurrent (threadDelay)
1111import Control.Monad (when)
1212import Control.Monad.IO.Class (liftIO)
13−import Test.Tasty.Bench
13+import Gauge
1414import Streamly.Prelude as S
1515import System.Random (randomRIO)
1616
benchmark/Streamly/Benchmark/Prelude/Ahead.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.Ahead"
benchmark/Streamly/Benchmark/Prelude/Async.hs+1−1
@@ -16,7 +16,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream.Transform as Transform
1616import Streamly.Benchmark.Common
1717import Streamly.Benchmark.Prelude
1818
19−import Test.Tasty.Bench
19+import Gauge
2020
2121moduleName :: String
2222moduleName = "Prelude.Async"
benchmark/Streamly/Benchmark/Prelude/Concurrent.hs+1−1
@@ -15,7 +15,7 @@ import Streamly.Prelude
1515 , fromWAsync, parallel, fromParallel
1616 )
1717
18−import Test.Tasty.Bench
18+import Gauge
1919import qualified Streamly.Prelude as S
2020
2121-------------------------------------------------------------------------------
benchmark/Streamly/Benchmark/Prelude/Merge.hs+1−1
@@ -30,7 +30,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream as Stream
3030import Streamly.Benchmark.Common
3131import Streamly.Benchmark.Prelude
3232
33−import Test.Tasty.Bench
33+import Gauge
3434
3535#ifdef INSPECTION
3636import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Prelude/Parallel.hs+1−1
@@ -22,7 +22,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream as Internal
2222import Streamly.Benchmark.Common
2323import Streamly.Benchmark.Prelude
2424
25−import Test.Tasty.Bench
25+import Gauge
2626
2727moduleName :: String
2828moduleName = "Prelude.Parallel"
benchmark/Streamly/Benchmark/Prelude/Rate.hs+1−1
@@ -14,7 +14,7 @@ import qualified Streamly.Prelude as S
1414import Streamly.Benchmark.Common
1515import Streamly.Benchmark.Prelude
1616
17−import Test.Tasty.Bench
17+import Gauge
1818
1919moduleName :: String
2020moduleName = "Prelude.Rate"
benchmark/Streamly/Benchmark/Prelude/WAsync.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.WAsync"
benchmark/Streamly/Benchmark/Prelude/WSerial.hs+1−1
@@ -29,7 +29,7 @@ import qualified Streamly.Internal.Data.Unfold as UF
2929import Streamly.Benchmark.Common
3030import Streamly.Benchmark.Prelude
3131
32−import Test.Tasty.Bench
32+import Gauge
3333
3434#ifdef INSPECTION
3535import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Prelude/ZipAsync.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.ZipAsync"
benchmark/Streamly/Benchmark/Prelude/ZipSerial.hs+1−1
@@ -29,7 +29,7 @@ import qualified Streamly.Prelude as S
2929import Streamly.Benchmark.Common
3030import Streamly.Benchmark.Prelude hiding (sourceUnfoldrM)
3131
32−import Test.Tasty.Bench
32+import Gauge
3333
3434#ifdef INSPECTION
3535import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Unicode/Char.hs+1−1
@@ -16,7 +16,7 @@ import Control.DeepSeq (NFData(..))
1616import Streamly.Internal.Data.Array (Array)
1717import System.FilePath (dropExtensions, takeFileName)
1818import System.FilePath.Posix ((</>))
19−import Test.Tasty.Bench (Benchmark, bench, bgroup, defaultMain, env, nfIO)
19+import Gauge.Main (Benchmark, bench, bgroup, defaultMain, env, nfIO)
2020import Streamly.Internal.Unicode.Char
2121 ( NormalizationMode(NFC, NFD, NFKC, NFKD)
2222 , normalize
benchmark/Streamly/Benchmark/Unicode/Parser.hs+20−37
@@ -10,11 +10,6 @@
1010{-# LANGUAGE ScopedTypeVariables #-}
1111{-# OPTIONS_GHC -Wno-orphans #-}
1212
13−#undef FUSION_CHECK
14−#ifdef FUSION_CHECK
15−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
16−#endif
17−
1813module Main
1914 (
2015 main
@@ -27,18 +22,28 @@ import Streamly.Internal.Data.Stream (Stream)
2722import Prelude hiding
2823 (any, all, take, sequence, sequence_, sequenceA, takeWhile, dropWhile)
2924
30−import qualified Streamly.Data.Array as Array
25+import qualified Streamly.Internal.Data.Fold as Fold
3126import qualified Streamly.Data.Stream as Stream
27+import qualified Streamly.Data.Unfold as Unfold
3228import qualified Streamly.Internal.Unicode.Parser as PRU
3329
34−import Test.Tasty.Bench hiding (env)
30+import Gauge hiding (env)
3531import Streamly.Benchmark.Common
32+import Streamly.Benchmark.Common.Handle
33+
34+{-# INLINE sourceUnfoldrM #-}
35+sourceUnfoldrM :: Monad m => Int -> Int -> Stream m Int
36+sourceUnfoldrM value n = Stream.unfoldrM step n
37+ where
38+ step cnt =
39+ if cnt > n + value
40+ then return Nothing
41+ else return (Just (cnt, cnt + 1))
3642
3743runParser :: Int -> (Stream IO Char -> IO a) -> IO ()
3844runParser count p = do
3945 let v = "+123456789.123456789e-123"
40− let !(arr :: Array.Array Char) = Array.fromListN (length v) v
41− let s = Stream.unfold Array.reader arr
46+ let s = Stream.unfold Unfold.fromList v
4247 replicateM_ count (p s)
4348
4449-- | Takes a fold method, and uses it with a default source.
@@ -46,23 +51,10 @@ runParser count p = do
4651benchIOSink :: Int -> String -> (Stream IO Char -> IO b) -> Benchmark
4752benchIOSink value name f = bench name $ nfIO $ runParser value f
4853
49−{-# INLINE doubleParser #-}
50−doubleParser :: Monad m => Stream m Char -> m (Either ParseError (Int, Int))
51−doubleParser = Stream.parse PRU.doubleParser
52−
53−{-# INLINE number #-}
54−number :: Monad m => Stream m Char -> m (Either ParseError (Integer, Int))
55−number = Stream.parse PRU.number
56−
5754{-# INLINE double #-}
5855double :: Monad m => Stream m Char -> m (Either ParseError Double)
5956double = Stream.parse PRU.double
6057
61−{-# INLINE numberDouble #-}
62−numberDouble :: Monad m => Stream m Char -> m (Either ParseError Double)
63−numberDouble = Stream.parse p
64− where p = uncurry PRU.mkDouble <$> PRU.number
65−
6658-------------------------------------------------------------------------------
6759-- Benchmarks
6860-------------------------------------------------------------------------------
@@ -77,10 +69,7 @@ instance NFData ParseError where
7769o_n_heap_serial :: Int -> [Benchmark]
7870o_n_heap_serial value =
7971 [
80− benchIOSink value "doubleParser" doubleParser
81− , benchIOSink value "number" number
82− , benchIOSink value "double (doubleParser)" double
83− , benchIOSink value "double (number)" numberDouble
72+ benchIOSink value "double" double
8473 ]
8574
8675-------------------------------------------------------------------------------
@@ -89,19 +78,13 @@ o_n_heap_serial value =
8978
9079main :: IO ()
9180main = do
92−#ifndef FUSION_CHECK
93− runWithCLIOpts defaultStreamSize allBenchmarks
81+ env <- mkHandleBenchEnv
82+ runWithCLIOptsEnv defaultStreamSize alloc (allBenchmarks env)
9483
9584 where
9685
97− allBenchmarks value =
86+ alloc value = Stream.fold Fold.toList $ Stream.chunksOf 100 $ sourceUnfoldrM value 0
87+
88+ allBenchmarks _ _ value =
9889 [ bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
9990 ]
100−#else
101− -- Enable FUSION_CHECK macro at the beginning of the file
102− -- Enable one benchmark below, and run the benchmark
103− -- Check the .dump-simpl output
104− let value = 100000
105− runParser value double
106− return ()
107−#endif
benchmark/Streamly/Benchmark/Unicode/Stream.hs+8−26
@@ -11,11 +11,6 @@
1111{-# LANGUAGE CPP #-}
1212{-# LANGUAGE ScopedTypeVariables #-}
1313
14−#undef FUSION_CHECK
15−#ifdef FUSION_CHECK
16−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
17−#endif
18−
1914#ifdef __HADDOCK_VERSION__
2015#undef INSPECTION
2116#endif
@@ -38,17 +33,16 @@ import qualified Streamly.Internal.FileSystem.Handle as Handle
3833import qualified Streamly.Internal.Unicode.Array as UnicodeArr
3934import qualified Streamly.Internal.Unicode.Stream as Unicode
4035
41−import Test.Tasty.Bench hiding (env)
36+import Gauge hiding (env)
4237import Streamly.Benchmark.Common
4338import Streamly.Benchmark.Common.Handle
4439
4540#ifdef INSPECTION
46−import Streamly.Internal.Data.MutByteArray (Unbox)
47−import Streamly.Internal.Data.Stream (Step(..))
48−import qualified Streamly.Internal.Data.Array as Array
49−import qualified Streamly.Internal.Data.MutArray as MutArray
50−import qualified Streamly.Internal.Data.Fold as Fold
41+import Streamly.Internal.Data.Unbox (Unbox)
42+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..))
43+import qualified Streamly.Internal.Data.Fold.Type as Fold
5144import qualified Streamly.Internal.Data.Tuple.Strict as Strict
45+import qualified Streamly.Internal.Data.Array.Type as Array
5246
5347import Test.Inspection
5448#endif
@@ -224,7 +218,7 @@ inspect $ 'copyStreamLatin1' `hasNoType` ''Step
224218inspect $ 'copyStreamLatin1' `hasNoType` ''Unfold.ConcatState -- Handle.read/UF.many
225219
226220inspect $ 'copyStreamLatin1' `hasNoType` ''Fold.Step
227−inspect $ 'copyStreamLatin1' `hasNoType` ''MutArray.ArrayUnsafe -- Handle.write/writeNUnsafe
221+inspect $ 'copyStreamLatin1' `hasNoType` ''Array.ArrayUnsafe -- Handle.write/writeNUnsafe
228222 -- Handle.read/Array.read
229223inspect $ 'copyStreamLatin1' `hasNoType` ''Strict.Tuple3' -- Handle.write/chunksOf
230224#endif
@@ -245,7 +239,7 @@ inspect $ 'copyStreamLatin1 `hasNoType` ''Unfold.ConcatState -- Handle.read/UF.m
245239
246240inspect $ 'copyStreamLatin1 `hasNoType` ''Fold.ManyState
247241inspect $ 'copyStreamLatin1 `hasNoType` ''Fold.Step
248−inspect $ 'copyStreamLatin1 `hasNoType` ''MutArray.ArrayUnsafe -- Handle.write/writeNUnsafe
242+inspect $ 'copyStreamLatin1 `hasNoType` ''Array.ArrayUnsafe -- Handle.write/writeNUnsafe
249243 -- Handle.read/Array.read
250244inspect $ 'copyStreamLatin1 `hasNoType` ''Strict.Tuple3' -- Handle.write/chunksOf
251245#endif
@@ -286,8 +280,7 @@ _copyStreamUtf8'Fold :: Handle -> Handle -> IO ()
286280_copyStreamUtf8'Fold inh outh =
287281 Stream.fold (Handle.write outh)
288282 $ Unicode.encodeUtf8
289− $ Stream.catRights
290− $ Stream.parseMany Unicode.writeCharUtf8'
283+ $ Stream.foldMany Unicode.writeCharUtf8'
291284 $ Stream.unfold Handle.reader inh
292285
293286{-# NOINLINE _copyStreamUtf8Parser #-}
@@ -324,7 +317,6 @@ o_1_space_decode_encode_read env =
324317
325318main :: IO ()
326319main = do
327−#ifndef FUSION_CHECK
328320 env <- mkHandleBenchEnv
329321 defaultMain (allBenchmarks env)
330322
@@ -337,13 +329,3 @@ main = do
337329 , o_1_space_decode_encode_read env
338330 ]
339331 ]
340−#else
341− -- Enable FUSION_CHECK macro at the beginning of the file
342− -- Enable one benchmark below, and run the benchmark
343− -- Check the .dump-simpl output
344− env <- mkHandleBenchEnv
345− let mkHandles (RefHandles {bigInH = inh, outputH = outh}) = Handles inh outh
346− (Handles inh outh) <- getHandles env mkHandles
347− copyStreamLatin1' inh outh
348− return ()
349−#endif
benchmark/Streamly/Benchmark/Unicode/Utf8.hs+1−1
@@ -13,7 +13,7 @@
1313import Data.Char (chr)
1414import Streamly.Internal.Unicode.Utf8 (Utf8)
1515
16−import Test.Tasty.Bench hiding (env)
16+import Gauge hiding (env)
1717import Streamly.Benchmark.Common
1818
1919import qualified Streamly.Internal.Unicode.Utf8 as Utf8
benchmark/bench-runner/Main.hs+6−34
@@ -16,11 +16,9 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
1616 benchName = drop 4 benchName0
1717
1818 general
19− -- GHC-9.6 requires 32M heap vs 16M for GHC-9.4
20− | "o-1-sp" `isInfixOf` benchName = "-K36K -M32M"
19+ | "o-1-sp" `isInfixOf` benchName = "-K36K -M16M"
2120 | "o-n-h" `isInfixOf` benchName = "-K36K -M32M"
22− -- GHC-9.6 requires 32M heap vs 16M for GHC-9.4
23− | "o-n-st" `isInfixOf` benchName = "-K1M -M32M"
21+ | "o-n-st" `isInfixOf` benchName = "-K1M -M16M"
2422 | "o-n-sp" `isInfixOf` benchName = "-K1M -M32M"
2523 | otherwise = ""
2624
@@ -29,17 +27,6 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
2927 | otherwise = ""
3028
3129 benchSpecific
32− -- GHC-9.6 requires 64M, earlier it was 16M
33− | "Data.Fold/o-n-heap.key-value.toHashMapIO (max buckets) sum" == benchName =
34− "-M64M"
35−
36− -- This is required only for the --long case because we allocate all
37− -- the arrays upfront. depends on the size of the stream.
38− | "Data.Parser/o-1-space" `isPrefixOf` benchName =
39− "-M128M"
40−
41− -----------------------------------------------------------------------
42−
4330 | "Prelude.Parallel/o-n-heap.mapping.mapM" == benchName = "-M256M"
4431 | "Prelude.Parallel/o-n-heap.monad-outer-product."
4532 `isPrefixOf` benchName = "-M256M"
@@ -70,18 +57,12 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
7057 -- GHC 9.4.4 requires 4M
7158 | "Data.Stream.StreamDK/o-n-space.traversable."
7259 `isPrefixOf` benchName = "-K4M"
73− -- GHC-9.6 requires 64M, earlier it was 32M
74− | "Data.Stream.StreamDK/o-n-heap.buffered.readsPrec pure streams" == benchName =
75− "-M64M"
76− -- GHC-9.6 requires 64M, earlier it was 32M
77− | "Data.Stream.StreamDK/o-n-heap.buffered.showPrec Haskell lists" == benchName =
78− "-M64M"
7960
8061 -----------------------------------------------------------------------
8162
82− | "Data.StreamD/o-n-space.elimination.toList" == benchName =
63+ | "Data.Stream.StreamD/o-n-space.elimination.toList" == benchName =
8364 "-K2M"
84− | "Data.StreamK/o-n-space.elimination.toList" == benchName =
65+ | "Data.Stream.StreamK/o-n-space.elimination.toList" == benchName =
8566 "-K2M"
8667
8768 -----------------------------------------------------------------------
@@ -93,13 +74,6 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
9374 | "Data.Stream/o-n-space.iterated."
9475 `isPrefixOf` benchName = "-K4M"
9576
96− -- GHC-9.6 requires 64M, earlier it was 32M
97− | "Data.Stream/o-n-heap.buffered.showPrec Haskell lists" == benchName =
98− "-M64M"
99− -- GHC-9.6 requires 64M, earlier it was 32M
100− | "Data.Stream/o-n-heap.buffered.readsPrec pure streams" == benchName =
101− "-M64M"
102−
10377 | "Data.Stream.ConcurrentEager/o-n-heap.monad-outer-product.toNullAp"
10478 `isPrefixOf` benchName = "-M1024M"
10579 | "Data.Stream.ConcurrentEager/o-1-space."
@@ -112,12 +86,10 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
11286
11387 | "Data.Array" `isPrefixOf` benchName
11488 && "/o-1-space.generation.read" `isSuffixOf` benchName = "-M32M"
115− -- XXX GHC 9.6 onwards needs 64M, earlier it was 32M
11689 | "Data.Array" `isPrefixOf` benchName
117− && "/o-1-space.generation.show" `isSuffixOf` benchName = "-M64M"
118− -- XXX GHC 9.6 onwards needs 64M, earlier it was 32M
90+ && "/o-1-space.generation.show" `isSuffixOf` benchName = "-M32M"
11991 | "Data.Array.Generic/o-1-space.transformationX4.map"
120− `isPrefixOf` benchName = "-M64M"
92+ `isPrefixOf` benchName = "-M32M"
12193
12294 -- XXX For --long option, need to check why so much heap is required.
12395 -- Note, if we remove the chunked stream module we need to keep the
benchmark/lib/Streamly/Benchmark/Common.hs+1−1
@@ -56,7 +56,7 @@ import System.Random (randomRIO)
5656import qualified Streamly.Internal.Data.Fold as Fold
5757import qualified Streamly.Internal.Data.Stream as S
5858
59−import Test.Tasty.Bench
59+import Gauge
6060
6161-------------------------------------------------------------------------------
6262-- Benchmark Prefixes
benchmark/lib/Streamly/Benchmark/Common/Handle.hs+1−3
@@ -30,8 +30,6 @@ module Streamly.Benchmark.Common.Handle
3030 , isSpace
3131 , isSp
3232 , mkHandleBenchEnv
33− , Handles(..)
34− , getHandles
3533 )
3634where
3735
@@ -48,7 +46,7 @@ import System.Process (callCommand)
4846
4947import Data.IORef
5048import Prelude hiding (last, length)
51−import Test.Tasty.Bench hiding (env)
49+import Gauge hiding (env)
5250
5351
5452scratchDir :: String
benchmark/lib/Streamly/Benchmark/Prelude.hs+1−1
@@ -82,7 +82,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream.Type as IsStream
8282import qualified Streamly.Internal.Data.Pipe as Pipe
8383import qualified Streamly.Internal.Data.Stream.Serial as Serial
8484
85−import Test.Tasty.Bench
85+import Gauge
8686import Streamly.Internal.Data.Time.Units
8787
8888-- Common polymorphic stream APIs used across multiple stream modules
benchmark/streamly-benchmarks.cabal+42−71
@@ -54,6 +54,11 @@ flag opt
5454 manual: True
5555 default: True
5656
57+flag use-gauge
58+ description: Use gauge instead of tasty-bench for benchmarking
59+ manual: True
60+ default: False
61+
5762flag use-streamly-core
5863 description: Benchmark only core package
5964 manual: True
@@ -140,10 +145,6 @@ common compile-options
140145 -Wno-missing-kind-signatures
141146 -Wno-operator-whitespace
142147
143− if impl(ghc >= 9.8)
144− ghc-options:
145− -Wno-missing-role-annotations
146−
147148 if flag(has-llvm)
148149 ghc-options: -fllvm
149150
@@ -151,17 +152,15 @@ common compile-options
151152 ghc-options: -Wmissed-specialisations
152153 -Wall-missed-specialisations
153154
155+ if flag(dev) || flag(debug)
156+ ghc-options: -fno-ignore-asserts
157+
154158common optimization-options
155159 if flag(opt)
156160 ghc-options: -O2
157161 -fdicts-strict
158162 -fspec-constr-recursive=16
159163 -fmax-worker-args=16
160−
161− -- For this to be effective it must come after the -O2 option
162− if flag(dev) || flag(debug)
163− ghc-options: -fno-ignore-asserts
164−
165164 if flag(fusion-plugin) && !impl(ghcjs) && !impl(ghc < 8.6)
166165 ghc-options: -fplugin Fusion.Plugin
167166 else
@@ -175,34 +174,43 @@ common bench-depends
175174 -- Core libraries shipped with ghc, the min and max
176175 -- constraints of these libraries should match with
177176 -- the GHC versions we support
178− base >= 4.9 && < 4.20
179− , deepseq >= 1.4.1 && < 1.6
177+ base >= 4.9 && < 4.19
178+ , deepseq >= 1.4.1 && < 1.5
180179 , mtl >= 2.2 && < 2.3.2
181180
182181 -- other libraries
183− , exceptions >= 0.8 && < 0.11
184182 , random >= 1.0 && < 2.0
185183 , transformers >= 0.4 && < 0.7
186− , containers >= 0.5 && < 0.8
184+ , containers >= 0.5 && < 0.7
187185 , hashable >= 1.3 && < 1.5
188186 , unordered-containers >= 0.2 && < 0.3
189187 , process >= 1.4 && < 1.7
190188 , directory >= 1.2.2 && < 1.4
191− , filepath >= 1.4.1 && < 1.6
192− , ghc-prim >= 0.4 && < 0.12
193− , tasty-bench >= 0.3 && < 0.4
194− , tasty >= 1.4.1 && < 1.6
195− , streamly-core
189+ , filepath >= 1.4.1 && < 1.5
190+ , ghc-prim >= 0.4 && < 0.11
196191
197− if !flag(use-streamly-core)
192+ if flag(use-streamly-core)
193+ build-depends: streamly-core
194+ else
198195 build-depends: streamly
196+ , streamly-core
197+
198+ if flag(use-gauge)
199+ build-depends: gauge >= 0.2.4 && < 0.3
200+ else
201+ build-depends: tasty-bench >= 0.3 && < 0.4
202+ , tasty >= 1.4.1 && < 1.5
203+ mixins: tasty-bench
204+ (Test.Tasty.Bench as Gauge
205+ , Test.Tasty.Bench as Gauge.Main
206+ )
199207
200208 if flag(fusion-plugin) && !impl(ghcjs) && !impl(ghc < 8.6)
201209 build-depends:
202210 fusion-plugin >= 0.2 && < 0.3
203211 if flag(inspection)
204− build-depends: template-haskell >= 2.14 && < 2.22
205− , inspection-testing >= 0.4 && < 0.6
212+ build-depends: template-haskell >= 2.14 && < 2.20
213+ , inspection-testing >= 0.4 && < 0.5
206214 -- Array uses a Storable constraint in dev build making several inspection
207215 -- tests fail
208216 if flag(dev) && flag(inspection)
@@ -348,34 +356,6 @@ benchmark Data.Stream.ConcurrentOrdered
348356 else
349357 buildable: True
350358
351−benchmark Data.Unbox
352− import: bench-options
353− type: exitcode-stdio-1.0
354− hs-source-dirs: .
355− cpp-options: -DUSE_UNBOX
356− main-is: Streamly/Benchmark/Data/Serialize.hs
357−
358−benchmark Data.Unbox.Derive.TH
359− import: bench-options
360− type: exitcode-stdio-1.0
361− hs-source-dirs: .
362− cpp-options: -DUSE_UNBOX
363− cpp-options: -DUSE_TH
364− main-is: Streamly/Benchmark/Data/Serialize.hs
365−
366−benchmark Data.Serialize
367− import: bench-options
368− type: exitcode-stdio-1.0
369− hs-source-dirs: .
370− cpp-options: -DUSE_TH
371− main-is: Streamly/Benchmark/Data/Serialize.hs
372− build-depends: QuickCheck, template-haskell
373− other-modules:
374− Streamly.Benchmark.Data.Serialize.TH
375− Streamly.Benchmark.Data.Serialize.RecCompatible
376− Streamly.Benchmark.Data.Serialize.RecNonCompatible
377− if flag(limit-build-mem)
378− ghc-options: +RTS -M1000M -RTS
379359
380360benchmark Data.Unfold
381361 import: bench-options
@@ -427,19 +407,6 @@ benchmark Data.ParserK
427407 import: bench-options
428408 type: exitcode-stdio-1.0
429409 hs-source-dirs: Streamly/Benchmark/Data
430− cpp-options: -DBENCH_SINGULAR
431− main-is: ParserK.hs
432− if impl(ghcjs)
433− buildable: False
434− else
435− buildable: True
436− build-depends: exceptions >= 0.8 && < 0.11
437−
438−benchmark Data.ParserK.Chunked
439− import: bench-options
440− type: exitcode-stdio-1.0
441− hs-source-dirs: Streamly/Benchmark/Data
442− cpp-options: -DBENCH_CHUNKED
443410 main-is: ParserK.hs
444411 if impl(ghcjs)
445412 buildable: False
@@ -447,13 +414,17 @@ benchmark Data.ParserK.Chunked
447414 buildable: True
448415 build-depends: exceptions >= 0.8 && < 0.11
449416
450−benchmark Data.ParserK.Chunked.Generic
417+-- Note: to use this we have to set DISABLE_FUSION in ParserD.hs so that
418+-- the rewrite rules are disabled. We can also use the "no-fusion" build
419+-- flag but we need to keep in mind that it disables fusion for streams
420+-- as well.
421+benchmark Data.FromParserK
451422 import: bench-options
452423 type: exitcode-stdio-1.0
453424 hs-source-dirs: Streamly/Benchmark/Data
454− cpp-options: -DBENCH_CHUNKED_GENERIC
425+ cpp-options: -DFROM_PARSERK
455426 main-is: ParserK.hs
456− if impl(ghcjs)
427+ if flag(use-streamly-core) || impl(ghcjs)
457428 buildable: False
458429 else
459430 buildable: True
@@ -463,7 +434,7 @@ benchmark Data.ParserK.Chunked.Generic
463434-- Raw Streams
464435-------------------------------------------------------------------------------
465436
466−benchmark Data.StreamD
437+benchmark Data.Stream.StreamD
467438 import: bench-options
468439 type: exitcode-stdio-1.0
469440 hs-source-dirs: Streamly/Benchmark/Data/Stream
@@ -471,7 +442,7 @@ benchmark Data.StreamD
471442 if impl(ghcjs)
472443 buildable: False
473444
474−benchmark Data.StreamK
445+benchmark Data.Stream.StreamK
475446 import: bench-options
476447 type: exitcode-stdio-1.0
477448 hs-source-dirs: Streamly/Benchmark/Data/Stream
@@ -487,11 +458,11 @@ benchmark Data.Stream.ToStreamK
487458 if !flag(dev) || impl(ghcjs)
488459 buildable: False
489460
490−benchmark Data.StreamK.Alt
461+benchmark Data.Stream.StreamK.Alt
491462 import: bench-options
492463 type: exitcode-stdio-1.0
493464 hs-source-dirs: Streamly/Benchmark/Data/Stream
494− main-is: StreamKAlt.hs
465+ main-is: StreamDK.hs
495466 if !flag(dev) || impl(ghcjs)
496467 buildable: False
497468
@@ -680,10 +651,10 @@ benchmark Data.Array
680651 if flag(limit-build-mem)
681652 ghc-options: +RTS -M1500M -RTS
682653
683−benchmark Data.MutArray
654+benchmark Data.Array.Mut
684655 import: bench-options
685656 type: exitcode-stdio-1.0
686− main-is: Streamly/Benchmark/Data/MutArray.hs
657+ main-is: Streamly/Benchmark/Data/Array/Mut.hs
687658 if flag(limit-build-mem)
688659 ghc-options: +RTS -M1500M -RTS
689660 if flag(use-streamly-core)
configure+9−9
@@ -1,6 +1,6 @@
11#! /bin/sh
22# Guess values for system-dependent variables and create Makefiles.
3−# Generated by GNU Autoconf 2.71 for streamly 0.10.1.
3+# Generated by GNU Autoconf 2.71 for streamly 0.10.0.
44#
55# Report bugs to <streamly@composewell.com>.
66#
@@ -610,8 +610,8 @@ MAKEFLAGS=
610610# Identity of this package.
611611PACKAGE_NAME='streamly'
612612PACKAGE_TARNAME='streamly'
613−PACKAGE_VERSION='0.10.1'
614−PACKAGE_STRING='streamly 0.10.1'
613+PACKAGE_VERSION='0.10.0'
614+PACKAGE_STRING='streamly 0.10.0'
615615PACKAGE_BUGREPORT='streamly@composewell.com'
616616PACKAGE_URL='https://streamly.composewell.com'
617617
@@ -1224,7 +1224,7 @@ if test "$ac_init_help" = "long"; then
12241224 # Omit some internal or obsolete options to make the list less imposing.
12251225 # This message is too long to be a string in the A/UX 3.1 sh.
12261226 cat <<_ACEOF
1227−\`configure' configures streamly 0.10.1 to adapt to many kinds of systems.
1227+\`configure' configures streamly 0.10.0 to adapt to many kinds of systems.
12281228
12291229Usage: $0 [OPTION]... [VAR=VALUE]...
12301230
@@ -1286,7 +1286,7 @@ fi
12861286
12871287if test -n "$ac_init_help"; then
12881288 case $ac_init_help in
1289− short | recursive ) echo "Configuration of streamly 0.10.1:";;
1289+ short | recursive ) echo "Configuration of streamly 0.10.0:";;
12901290 esac
12911291 cat <<\_ACEOF
12921292
@@ -1372,7 +1372,7 @@ fi
13721372test -n "$ac_init_help" && exit $ac_status
13731373if $ac_init_version; then
13741374 cat <<\_ACEOF
1375−streamly configure 0.10.1
1375+streamly configure 0.10.0
13761376generated by GNU Autoconf 2.71
13771377
13781378Copyright (C) 2021 Free Software Foundation, Inc.
@@ -1500,7 +1500,7 @@ cat >config.log <<_ACEOF
15001500This file contains any messages produced by compilers while
15011501running configure, to aid debugging if configure makes a mistake.
15021502
1503−It was created by streamly $as_me 0.10.1, which was
1503+It was created by streamly $as_me 0.10.0, which was
15041504generated by GNU Autoconf 2.71. Invocation command line was
15051505
15061506 $ $0$ac_configure_args_raw
@@ -3800,7 +3800,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
38003800# report actual input values of CONFIG_FILES etc. instead of their
38013801# values after options handling.
38023802ac_log="
3803−This file was extended by streamly $as_me 0.10.1, which was
3803+This file was extended by streamly $as_me 0.10.0, which was
38043804generated by GNU Autoconf 2.71. Invocation command line was
38053805
38063806 CONFIG_FILES = $CONFIG_FILES
@@ -3856,7 +3856,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\
38563856cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
38573857ac_cs_config='$ac_cs_config_escaped'
38583858ac_cs_version="\\
3859−streamly config.status 0.10.1
3859+streamly config.status 0.10.0
38603860configured by $0, generated by GNU Autoconf 2.71,
38613861 with options \\"\$ac_cs_config\\"
38623862
configure.ac+1−1
@@ -3,7 +3,7 @@
33# See https://www.gnu.org/software/autoconf/manual/autoconf.html for help on
44# the macros used in this file.
55
6−AC_INIT([streamly], [0.10.1], [streamly@composewell.com], [streamly], [https://streamly.composewell.com])
6+AC_INIT([streamly], [0.10.0], [streamly@composewell.com], [streamly], [https://streamly.composewell.com])
77
88# To suppress "WARNING: unrecognized options: --with-compiler"
99AC_ARG_WITH([compiler], [GHC])
core/LICENSE−41
@@ -279,44 +279,3 @@ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
279279LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
280280OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
281281DAMAGE.
282−
283−-------------------------------------------------------------------------------
284−th-utilities-0.2.5.0 (https://hackage.haskell.org/package/th-utilities)
285−-------------------------------------------------------------------------------
286−Copyright (c) 2016 FP Complete Corporation.
287−
288−Permission is hereby granted, free of charge, to any person obtaining
289−a copy of this software and associated documentation files (the
290−"Software"), to deal in the Software without restriction, including
291−without limitation the rights to use, copy, modify, merge, publish,
292−distribute, sublicense, and/or sell copies of the Software, and to
293−permit persons to whom the Software is furnished to do so, subject to
294−the following conditions:
295−
296−The above copyright notice and this permission notice shall be
297−included in all copies or substantial portions of the Software.
298−
299−THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
300−EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
301−MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
302−NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
303−LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
304−OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
305−WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
306−
307−-------------------------------------------------------------------------------
308−th-abstraction-0.5.0.0 (https://hackage.haskell.org/package/th-abstraction)
309−-------------------------------------------------------------------------------
310−Copyright (c) 2017-2020 Eric Mertens
311−
312−Permission to use, copy, modify, and/or distribute this software for any purpose
313−with or without fee is hereby granted, provided that the above copyright notice
314−and this permission notice appear in all copies.
315−
316−THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
317−REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
318−FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
319−INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
320−OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
321−TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
322−THIS SOFTWARE.
core/configure+9−9
@@ -1,6 +1,6 @@
11#! /bin/sh
22# Guess values for system-dependent variables and create Makefiles.
3−# Generated by GNU Autoconf 2.71 for streamly-core 0.2.2.
3+# Generated by GNU Autoconf 2.71 for streamly-core 0.2.0.
44#
55# Report bugs to <streamly@composewell.com>.
66#
@@ -610,8 +610,8 @@ MAKEFLAGS=
610610# Identity of this package.
611611PACKAGE_NAME='streamly-core'
612612PACKAGE_TARNAME='streamly-core'
613−PACKAGE_VERSION='0.2.2'
614−PACKAGE_STRING='streamly-core 0.2.2'
613+PACKAGE_VERSION='0.2.0'
614+PACKAGE_STRING='streamly-core 0.2.0'
615615PACKAGE_BUGREPORT='streamly@composewell.com'
616616PACKAGE_URL='https://streamly.composewell.com'
617617
@@ -1256,7 +1256,7 @@ if test "$ac_init_help" = "long"; then
12561256 # Omit some internal or obsolete options to make the list less imposing.
12571257 # This message is too long to be a string in the A/UX 3.1 sh.
12581258 cat <<_ACEOF
1259−\`configure' configures streamly-core 0.2.2 to adapt to many kinds of systems.
1259+\`configure' configures streamly-core 0.2.0 to adapt to many kinds of systems.
12601260
12611261Usage: $0 [OPTION]... [VAR=VALUE]...
12621262
@@ -1318,7 +1318,7 @@ fi
13181318
13191319if test -n "$ac_init_help"; then
13201320 case $ac_init_help in
1321− short | recursive ) echo "Configuration of streamly-core 0.2.2:";;
1321+ short | recursive ) echo "Configuration of streamly-core 0.2.0:";;
13221322 esac
13231323 cat <<\_ACEOF
13241324
@@ -1404,7 +1404,7 @@ fi
14041404test -n "$ac_init_help" && exit $ac_status
14051405if $ac_init_version; then
14061406 cat <<\_ACEOF
1407−streamly-core configure 0.2.2
1407+streamly-core configure 0.2.0
14081408generated by GNU Autoconf 2.71
14091409
14101410Copyright (C) 2021 Free Software Foundation, Inc.
@@ -1622,7 +1622,7 @@ cat >config.log <<_ACEOF
16221622This file contains any messages produced by compilers while
16231623running configure, to aid debugging if configure makes a mistake.
16241624
1625−It was created by streamly-core $as_me 0.2.2, which was
1625+It was created by streamly-core $as_me 0.2.0, which was
16261626generated by GNU Autoconf 2.71. Invocation command line was
16271627
16281628 $ $0$ac_configure_args_raw
@@ -3832,7 +3832,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
38323832# report actual input values of CONFIG_FILES etc. instead of their
38333833# values after options handling.
38343834ac_log="
3835−This file was extended by streamly-core $as_me 0.2.2, which was
3835+This file was extended by streamly-core $as_me 0.2.0, which was
38363836generated by GNU Autoconf 2.71. Invocation command line was
38373837
38383838 CONFIG_FILES = $CONFIG_FILES
@@ -3888,7 +3888,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\
38883888cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
38893889ac_cs_config='$ac_cs_config_escaped'
38903890ac_cs_version="\\
3891−streamly-core config.status 0.2.2
3891+streamly-core config.status 0.2.0
38923892configured by $0, generated by GNU Autoconf 2.71,
38933893 with options \\"\$ac_cs_config\\"
38943894
core/configure.ac+1−1
@@ -3,7 +3,7 @@
33# See https://www.gnu.org/software/autoconf/manual/autoconf.html for help on
44# the macros used in this file.
55
6−AC_INIT([streamly-core], [0.2.2], [streamly@composewell.com], [streamly-core], [https://streamly.composewell.com])
6+AC_INIT([streamly-core], [0.2.0], [streamly@composewell.com], [streamly-core], [https://streamly.composewell.com])
77
88# To suppress "WARNING: unrecognized options: --with-compiler"
99AC_ARG_WITH([compiler], [GHC])
core/docs/ApiChangelogs/0.1.0-0.2.0.txtdeleted−1471
@@ -1,1471 +0,0 @@
1−---------------------------------
2−API Annotations
3−---------------------------------
4−
5−[A] : Added
6−[R] : Removed
7−[C] : Changed
8−[O] : Old definition
9−[N] : New definition
10−[D] : Deprecated
11−
12−---------------------------------
13−API diff
14−---------------------------------
15−
16−[C] Streamly.Unicode.Parser
17− [A] double :: Monad m => Parser Char m Double
18−[C] Streamly.FileSystem.Handle
19− [A] readWith :: MonadIO m => Int -> Handle -> Stream m Word8
20− [A] readChunksWith :: MonadIO m => Int -> Handle -> Stream m (Array Word8)
21−[C] Streamly.FileSystem.Dir
22− [C] readEither
23− [O] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
24− [N] readEither :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
25− [C] read
26− [O] read :: MonadIO m => FilePath -> Stream m FilePath
27− [N] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
28−[C] Streamly.Data.Unfold
29− [A] class Enum a => Enumerable a
30− [A] second :: b -> Unfold m (a, b) c -> Unfold m a c
31− [A] first :: a -> Unfold m (a, b) c -> Unfold m b c
32− [A] enumerateFromTo :: (Enumerable a, Monad m) => Unfold m (a, a) a
33− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => Unfold m (a, a, a) a
34− [A] enumerateFromThen :: (Enumerable a, Monad m) => Unfold m (a, a) a
35− [A] enumerateFrom :: (Enumerable a, Monad m) => Unfold m a a
36−[C] Streamly.Data.StreamK
37− [C] parseChunks
38− [O] parseChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b)
39− [N] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
40− [C] parseBreakChunks
41− [O] parseBreakChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
42− [N] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
43− [A] parseBreak :: forall m a b. Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
44− [A] parse :: Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b)
45− [A] handle :: (MonadCatch m, Exception e) => (e -> m (StreamK m a)) -> StreamK m a -> StreamK m a
46− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> StreamK m a) -> StreamK m a
47−[C] Streamly.Data.Stream
48− [A] wordsBy :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
49− [A] splitOn :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
50− [C] handle
51− [O] handle :: (MonadCatch m, Exception e) => (e -> Stream m a) -> Stream m a -> Stream m a
52− [N] handle :: (MonadCatch m, Exception e) => (e -> m (Stream m a)) -> Stream m a -> Stream m a
53− [A] groupsOf :: Monad m => Int -> Fold m a b -> Stream m a -> Stream m b
54−[C] Streamly.Data.ParserK
55− [D] fromParser :: (MonadIO m, Unbox a) => Parser a m b -> ParserK (Array a) m b
56− [D] fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK (Array a) m b
57− [A] adaptCG :: Monad m => Parser a m b -> ParserK (Array a) m b
58− [A] adaptC :: (Monad m, Unbox a) => Parser a m b -> ParserK (Array a) m b
59− [A] adapt :: Monad m => Parser a m b -> ParserK a m b
60−[C] Streamly.Data.Parser
61− [A] groupByRollingEither :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
62− [A] groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
63−[A] Streamly.Data.MutByteArray
64− [A] class Unbox a
65− [A] class Serialize a
66− [A] SerializeConfig
67− [A] MutByteArray
68− [A] unpin :: MutByteArray -> IO MutByteArray
69− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
70− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
71− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
72− [A] pinnedNew :: Int -> IO MutByteArray
73− [A] pin :: MutByteArray -> IO MutByteArray
74− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
75− [A] new :: Int -> IO MutByteArray
76− [A] isPinned :: MutByteArray -> Bool
77− [A] inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
78− [A] inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
79− [A] inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig
80− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
81− [A] deriveUnbox :: Q [Dec] -> Q [Dec]
82− [A] deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]
83− [A] deriveSerialize :: Q [Dec] -> Q [Dec]
84− [A] addSizeTo :: Serialize a => Int -> a -> Int
85−[C] Streamly.Data.MutArray.Generic
86− [A] write :: MonadIO m => Fold m a (MutArray a)
87− [A] readRev :: MonadIO m => MutArray a -> Stream m a
88− [A] read :: MonadIO m => MutArray a -> Stream m a
89− [A] putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
90− [A] modifyIndexUnsafe :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
91− [A] length :: MutArray a -> Int
92− [A] getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
93− [C] getIndex
94− [O] getIndex :: MonadIO m => Int -> MutArray a -> m a
95− [N] getIndex :: MonadIO m => Int -> MutArray a -> m (Maybe a)
96− [A] fromListN :: MonadIO m => Int -> [a] -> m (MutArray a)
97− [A] fromList :: MonadIO m => [a] -> m (MutArray a)
98−[C] Streamly.Data.MutArray
99− [A] unpin :: MutArray a -> IO (MutArray a)
100− [A] readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
101− [A] read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
102− [A] putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
103− [D] pokeByteIndex :: Unbox a => Int -> MutByteArray -> a -> IO ()
104− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
105− [A] pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
106− [A] pin :: MutArray a -> IO (MutArray a)
107− [D] peekByteIndex :: Unbox a => Int -> MutByteArray -> IO a
108− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
109− [D] newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
110− [A] modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
111− [A] modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
112− [A] modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()
113− [A] isPinned :: MutArray a -> Bool
114− [A] getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
115− [C] getIndex
116− [O] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
117− [N] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
118−[C] Streamly.Data.Array.Generic
119− [A] toList :: Array a -> [a]
120− [A] getIndex :: Int -> Array a -> Maybe a
121−[C] Streamly.Data.Array
122− [A] unpin :: Array a -> IO (Array a)
123− [D] pokeByteIndex :: Unbox a => Int -> MutByteArray -> a -> IO ()
124− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
125− [A] pin :: Array a -> IO (Array a)
126− [D] peekByteIndex :: Unbox a => Int -> MutByteArray -> IO a
127− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
128− [A] isPinned :: Array a -> Bool
129−[C] Streamly.Console.Stdio
130−
131−---------------------------------
132−Internal API diff
133−---------------------------------
134−
135−[C] Streamly.Internal.Unicode.Stream
136− [C] writeCharUtf8'
137− [O] writeCharUtf8' :: Monad m => Fold m Word8 Char
138− [N] writeCharUtf8' :: Monad m => Parser Word8 m Char
139−[C] Streamly.Internal.Unicode.Parser
140− [A] number :: Monad m => Parser Char m (Integer, Int)
141− [A] mkDouble :: Integer -> Int -> Double
142− [A] doubleParser :: Monad m => Parser Char m (Int, Int)
143− [C] double
144− [O] double :: Parser Char m Double
145− [N] double :: Monad m => Parser Char m Double
146−[R] Streamly.Internal.Serialize.ToBytes
147−[R] Streamly.Internal.Serialize.FromBytes
148−[C] Streamly.Internal.FileSystem.File
149− [A] writeAppendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()
150− [A] writeAppendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()
151− [A] writeAppendArray :: FilePath -> Array a -> IO ()
152− [A] writeAppend :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()
153− [R] appendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()
154− [R] appendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()
155− [R] appendArray :: FilePath -> Array a -> IO ()
156− [R] append :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()
157−[C] Streamly.Internal.FileSystem.Dir
158− [C] reader
159− [O] reader :: MonadIO m => Unfold m FilePath FilePath
160− [N] reader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
161− [C] readFiles
162− [O] readFiles :: MonadIO m => FilePath -> Stream m FilePath
163− [N] readFiles :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
164− [C] readEitherPaths
165− [O] readEitherPaths :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
166− [N] readEitherPaths :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
167− [C] readEither
168− [O] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
169− [N] readEither :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
170− [C] readDirs
171− [O] readDirs :: MonadIO m => FilePath -> Stream m FilePath
172− [N] readDirs :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
173− [C] read
174− [O] read :: MonadIO m => FilePath -> Stream m FilePath
175− [N] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
176− [C] fileReader
177− [O] fileReader :: MonadIO m => Unfold m FilePath FilePath
178− [N] fileReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
179− [C] eitherReaderPaths
180− [O] eitherReaderPaths :: MonadIO m => Unfold m FilePath (Either FilePath FilePath)
181− [N] eitherReaderPaths :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Either FilePath FilePath)
182− [C] eitherReader
183− [O] eitherReader :: MonadIO m => Unfold m FilePath (Either FilePath FilePath)
184− [N] eitherReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Either FilePath FilePath)
185− [C] dirReader
186− [O] dirReader :: MonadIO m => Unfold m FilePath FilePath
187− [N] dirReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
188−[R] Streamly.Internal.Data.Unfold.Type
189−[R] Streamly.Internal.Data.Unfold.Enumeration
190−[C] Streamly.Internal.Data.Unfold
191− [C] Unfold
192− [A] Unfold :: (s -> m (Step s b)) -> (a -> m s) -> Unfold m a b
193− [A] takeWhileMWithInput :: Monad m => (a -> b -> m Bool) -> Unfold m a b -> Unfold m a b
194− [A] manyInterleave :: Monad m => Unfold m a b -> Unfold m c a -> Unfold m c b
195− [A] enumerateFromStepIntegral :: (Monad m, Integral a) => Unfold m (a, a) a
196− [A] crossApplySnd :: Unfold m a b -> Unfold m a c -> Unfold m a c
197− [A] crossApplyFst :: Unfold m a b -> Unfold m a c -> Unfold m a b
198− [A] concatMap :: Monad m => (b -> Unfold m a c) -> Unfold m a b -> Unfold m a c
199−[R] Streamly.Internal.Data.Unboxed
200−[C] Streamly.Internal.Data.Time.Units
201− [R] Streamly.Internal.Data.Unboxed.Unbox
202− [A] Streamly.Internal.Data.Unbox.Unbox
203− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.NanoSecond64
204− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.MilliSecond64
205− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.MicroSecond64
206−[R] Streamly.Internal.Data.Time.Clock.Type
207−[A] Streamly.Internal.Data.StreamK
208− [A] CrossStreamK
209− [A] (FixityR,6)
210− [A] (FixityR,6)
211− [A] (FixityR,6)
212− [A] (FixityR,5)
213− [A] (FixityR,5)
214− [A] (FixityR,6)
215− [A] (FixityR,5)
216− [A] StreamK
217− [A] MkStream :: (forall r. State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
218− [A] zipWithM :: Monad m => (a -> b -> m c) -> StreamK m a -> StreamK m b -> StreamK m c
219− [A] zipWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
220− [A] unfoldrMWith :: Monad m => (m a -> StreamK m a -> StreamK m a) -> (b -> m (Maybe (a, b))) -> b -> StreamK m a
221− [A] unfoldrM :: Monad m => (b -> m (Maybe (a, b))) -> b -> StreamK m a
222− [A] unfoldr :: (b -> Maybe (a, b)) -> b -> StreamK m a
223− [A] uncons :: Applicative m => StreamK m a -> m (Maybe (a, StreamK m a))
224− [A] unShare :: StreamK m a -> StreamK m a
225− [A] unCross :: CrossStreamK m a -> StreamK m a
226− [A] toStream :: Applicative m => StreamK m a -> Stream m a
227− [A] toList :: Monad m => StreamK m a -> m [a]
228− [A] the :: (Eq a, Monad m) => StreamK m a -> m (Maybe a)
229− [A] takeWhile :: (a -> Bool) -> StreamK m a -> StreamK m a
230− [A] take :: Int -> StreamK m a -> StreamK m a
231− [A] tail :: Applicative m => StreamK m a -> m (Maybe (StreamK m a))
232− [A] sortBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> StreamK m a
233− [A] sequence :: Monad m => StreamK m (m a) -> StreamK m a
234− [A] scanlx' :: (x -> a -> x) -> x -> (x -> b) -> StreamK m a -> StreamK m b
235− [A] scanl' :: (b -> a -> b) -> b -> StreamK m a -> StreamK m b
236− [A] reverse :: StreamK m a -> StreamK m a
237− [A] replicateMWith :: (m a -> StreamK m a -> StreamK m a) -> Int -> m a -> StreamK m a
238− [A] replicateM :: Monad m => Int -> m a -> StreamK m a
239− [A] replicate :: Int -> a -> StreamK m a
240− [A] repeatMWith :: (m a -> t m a -> t m a) -> m a -> t m a
241− [A] repeatM :: Monad m => m a -> StreamK m a
242− [A] repeat :: a -> StreamK m a
243− [A] parseDBreak :: Monad m => Parser a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
244− [A] parseD :: Monad m => Parser a m b -> StreamK m a -> m (Either ParseError b)
245− [A] parseChunksGeneric :: Monad m => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
246− [A] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
247− [A] parseBreakChunksGeneric :: forall m a b. Monad m => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
248− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
249− [A] parseBreak :: forall m a b. Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
250− [A] parse :: Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b)
251− [A] null :: Monad m => StreamK m a -> m Bool
252− [A] notElem :: (Monad m, Eq a) => a -> StreamK m a -> m Bool
253− [A] nilM :: Applicative m => m b -> StreamK m a
254− [A] nil :: StreamK m a
255− [A] mkStream :: (forall r. State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
256− [A] mkCross :: StreamK m a -> CrossStreamK m a
257− [A] minimumBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> m (Maybe a)
258− [A] minimum :: (Monad m, Ord a) => StreamK m a -> m (Maybe a)
259− [A] mfix :: Monad m => (m a -> StreamK m a) -> StreamK m a
260− [A] mergeMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
261− [A] mergeIterateWith :: (StreamK m a -> StreamK m a -> StreamK m a) -> (a -> StreamK m a) -> StreamK m a -> StreamK m a
262− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
263− [A] mergeBy :: (a -> a -> Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
264− [A] maximumBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> m (Maybe a)
265− [A] maximum :: (Monad m, Ord a) => StreamK m a -> m (Maybe a)
266− [A] mapMaybe :: (a -> Maybe b) -> StreamK m a -> StreamK m b
267− [A] mapM_ :: Monad m => (a -> m b) -> StreamK m a -> m ()
268− [A] mapMWith :: (m b -> StreamK m b -> StreamK m b) -> (a -> m b) -> StreamK m a -> StreamK m b
269− [A] mapMSerial :: Monad m => (a -> m b) -> StreamK m a -> StreamK m b
270− [A] mapM :: Monad m => (a -> m b) -> StreamK m a -> StreamK m b
271− [A] map :: (a -> b) -> StreamK m a -> StreamK m b
272− [A] lookup :: (Monad m, Eq a) => a -> StreamK m (a, b) -> m (Maybe b)
273− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => StreamK m a -> StreamK (t m) a
274− [A] last :: Monad m => StreamK m a -> m (Maybe a)
275− [A] iterateMWith :: Monad m => (m a -> StreamK m a -> StreamK m a) -> (a -> m a) -> m a -> StreamK m a
276− [A] iterateM :: Monad m => (a -> m a) -> m a -> StreamK m a
277− [A] iterate :: (a -> a) -> a -> StreamK m a
278− [A] intersperseM :: Monad m => m a -> StreamK m a -> StreamK m a
279− [A] intersperse :: Monad m => a -> StreamK m a -> StreamK m a
280− [A] interleaveMin :: StreamK m a -> StreamK m a -> StreamK m a
281− [A] interleaveFst :: StreamK m a -> StreamK m a -> StreamK m a
282− [A] interleave :: StreamK m a -> StreamK m a -> StreamK m a
283− [A] insertBy :: (a -> a -> Ordering) -> a -> StreamK m a -> StreamK m a
284− [A] init :: Applicative m => StreamK m a -> m (Maybe (StreamK m a))
285− [A] hoist :: (Monad m, Monad n) => (forall x. m x -> n x) -> StreamK m a -> StreamK n a
286− [A] head :: Monad m => StreamK m a -> m (Maybe a)
287− [A] handle :: (MonadCatch m, Exception e) => (e -> m (StreamK m a)) -> StreamK m a -> StreamK m a
288− [A] fromYieldK :: YieldK m a -> StreamK m a
289− [A] fromStream :: Monad m => Stream m a -> StreamK m a
290− [A] fromStopK :: StopK m -> StreamK m a
291− [A] fromPure :: a -> StreamK m a
292− [A] fromList :: [a] -> StreamK m a
293− [A] fromIndicesMWith :: (m a -> StreamK m a -> StreamK m a) -> (Int -> m a) -> StreamK m a
294− [A] fromIndicesM :: Monad m => (Int -> m a) -> StreamK m a
295− [A] fromIndices :: (Int -> a) -> StreamK m a
296− [A] fromFoldableM :: (Foldable f, Monad m) => f (m a) -> StreamK m a
297− [A] fromFoldable :: Foldable f => f a -> StreamK m a
298− [A] fromEffect :: Monad m => m a -> StreamK m a
299− [A] foldrT :: (Monad m, Monad (s m), MonadTrans s) => (a -> s m b -> s m b) -> s m b -> StreamK m a -> s m b
300− [A] foldrSShared :: (a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
301− [A] foldrSM :: Monad m => (m a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
302− [A] foldrS :: (a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
303− [A] foldrM :: (a -> m b -> m b) -> m b -> StreamK m a -> m b
304− [A] foldr1 :: Monad m => (a -> a -> a) -> StreamK m a -> m (Maybe a)
305− [A] foldr :: Monad m => (a -> b -> b) -> b -> StreamK m a -> m b
306− [A] foldlx' :: forall m a b x. Monad m => (x -> a -> x) -> x -> (x -> b) -> StreamK m a -> m b
307− [A] foldlT :: (Monad m, Monad (s m), MonadTrans s) => (s m b -> a -> s m b) -> s m b -> StreamK m a -> s m b
308− [A] foldlS :: (StreamK m b -> a -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
309− [A] foldlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> StreamK m a -> m b
310− [A] foldlM' :: Monad m => (b -> a -> m b) -> m b -> StreamK m a -> m b
311− [A] foldl' :: Monad m => (b -> a -> b) -> b -> StreamK m a -> m b
312− [A] foldStreamShared :: State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> StreamK m a -> m r
313− [A] foldStream :: State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> StreamK m a -> m r
314− [A] foldEither :: Monad m => Fold m a b -> StreamK m a -> m (Either (Fold m a b) (b, StreamK m a))
315− [A] foldConcat :: Monad m => Producer m a b -> Fold m b c -> StreamK m a -> m (c, StreamK m a)
316− [A] foldBreak :: Monad m => Fold m a b -> StreamK m a -> m (b, StreamK m a)
317− [A] fold :: Monad m => Fold m a b -> StreamK m a -> m b
318− [A] findM :: Monad m => (a -> m Bool) -> StreamK m a -> m (Maybe a)
319− [A] findIndices :: (a -> Bool) -> StreamK m a -> StreamK m Int
320− [A] find :: Monad m => (a -> Bool) -> StreamK m a -> m (Maybe a)
321− [A] filter :: (a -> Bool) -> StreamK m a -> StreamK m a
322− [A] evalStateT :: Monad m => m s -> StreamK (StateT s m) a -> StreamK m a
323− [A] elem :: (Monad m, Eq a) => a -> StreamK m a -> m Bool
324− [A] dropWhile :: (a -> Bool) -> StreamK m a -> StreamK m a
325− [A] drop :: Int -> StreamK m a -> StreamK m a
326− [A] drain :: Monad m => StreamK m a -> m ()
327− [A] deleteBy :: (a -> a -> Bool) -> a -> StreamK m a -> StreamK m a
328− [A] crossWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
329− [A] crossApplyWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> StreamK m (a -> b) -> StreamK m a -> StreamK m b
330− [A] crossApplySnd :: StreamK m a -> StreamK m b -> StreamK m b
331− [A] crossApplyFst :: StreamK m a -> StreamK m b -> StreamK m a
332− [A] crossApply :: StreamK m (a -> b) -> StreamK m a -> StreamK m b
333− [A] cross :: Monad m => StreamK m a -> StreamK m b -> StreamK m (a, b)
334− [A] consMBy :: Monad m => (StreamK m a -> StreamK m a -> StreamK m a) -> m a -> StreamK m a -> StreamK m a
335− [A] consM :: Monad m => m a -> StreamK m a -> StreamK m a
336− [A] consK :: YieldK m a -> StreamK m a -> StreamK m a
337− [A] cons :: a -> StreamK m a -> StreamK m a
338− [A] conjoin :: Monad m => StreamK m a -> StreamK m a -> StreamK m a
339− [A] concatMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
340− [A] concatMapEffect :: Monad m => (b -> StreamK m a) -> m b -> StreamK m a
341− [A] concatMap :: (a -> StreamK m b) -> StreamK m a -> StreamK m b
342− [A] concatIterateWith :: (StreamK m a -> StreamK m a -> StreamK m a) -> (a -> StreamK m a) -> StreamK m a -> StreamK m a
343− [A] concatIterateScanWith :: Monad m => (StreamK m a -> StreamK m a -> StreamK m a) -> (b -> a -> m (b, StreamK m a)) -> m b -> StreamK m a -> StreamK m a
344− [A] concatIterateLeftsWith :: b ~ Either a c => (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m b -> StreamK m b
345− [A] concatEffect :: Monad m => m (StreamK m a) -> StreamK m a
346− [A] buildSM :: Monad m => ((m a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a
347− [A] buildS :: ((a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a
348− [A] buildM :: Monad m => (forall r. (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
349− [A] build :: forall m a. (forall b. (a -> b -> b) -> b -> b) -> StreamK m a
350− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> StreamK m a) -> StreamK m a
351− [A] bindWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> StreamK m a -> (a -> StreamK m b) -> StreamK m b
352− [A] before :: Monad m => m b -> StreamK m a -> StreamK m a
353− [A] augmentSM :: Monad m => ((m a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a
354− [A] augmentS :: ((a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a
355− [A] append :: StreamK m a -> StreamK m a -> StreamK m a
356− [A] any :: Monad m => (a -> Bool) -> StreamK m a -> m Bool
357− [A] all :: Monad m => (a -> Bool) -> StreamK m a -> m Bool
358− [A] (.:) :: a -> StreamK m a -> StreamK m a
359− [A] (!!) :: Monad m => StreamK m a -> Int -> m (Maybe a)
360−[R] Streamly.Internal.Data.Stream.StreamK.Type
361−[R] Streamly.Internal.Data.Stream.StreamK.Transformer
362−[R] Streamly.Internal.Data.Stream.StreamK
363−[R] Streamly.Internal.Data.Stream.StreamD.Type
364−[R] Streamly.Internal.Data.Stream.StreamD.Transformer
365−[R] Streamly.Internal.Data.Stream.StreamD.Transform
366−[R] Streamly.Internal.Data.Stream.StreamD.Top
367−[R] Streamly.Internal.Data.Stream.StreamD.Step
368−[R] Streamly.Internal.Data.Stream.StreamD.Nesting
369−[R] Streamly.Internal.Data.Stream.StreamD.Lift
370−[R] Streamly.Internal.Data.Stream.StreamD.Generate
371−[R] Streamly.Internal.Data.Stream.StreamD.Exception
372−[R] Streamly.Internal.Data.Stream.StreamD.Eliminate
373−[R] Streamly.Internal.Data.Stream.StreamD.Container
374−[D] Streamly.Internal.Data.Stream.StreamD
375−[R] Streamly.Internal.Data.Stream.Common
376−[R] Streamly.Internal.Data.Stream.Chunked
377−[C] Streamly.Internal.Data.Stream
378− [A] class Enum a => Enumerable a
379− [A] Stream
380− [A] UnStream :: (State StreamK m a -> s -> m (Step s a)) -> s -> Stream m a
381− [A] Step
382− [A] Yield :: a -> s -> Step s a
383− [A] Stop :: Step s a
384− [A] Skip :: s -> Step s a
385− [A] InterleaveState
386− [A] InterleaveSecondOnly :: s2 -> InterleaveState s1 s2
387− [A] InterleaveSecond :: s1 -> s2 -> InterleaveState s1 s2
388− [A] InterleaveFirstOnly :: s1 -> InterleaveState s1 s2
389− [A] InterleaveFirst :: s1 -> s2 -> InterleaveState s1 s2
390− [A] FoldManyPost
391− [A] FoldManyPostYield :: b -> FoldManyPost s fs b a -> FoldManyPost s fs b a
392− [A] FoldManyPostStart :: s -> FoldManyPost s fs b a
393− [A] FoldManyPostLoop :: s -> fs -> FoldManyPost s fs b a
394− [A] FoldManyPostDone :: FoldManyPost s fs b a
395− [A] FoldMany
396− [A] FoldManyYield :: b -> FoldMany s fs b a -> FoldMany s fs b a
397− [A] FoldManyStart :: s -> FoldMany s fs b a
398− [A] FoldManyLoop :: s -> fs -> FoldMany s fs b a
399− [A] FoldManyFirst :: fs -> s -> FoldMany s fs b a
400− [A] FoldManyDone :: FoldMany s fs b a
401− [A] CrossStream
402− [A] ConcatUnfoldInterleaveState
403− [A] ConcatUnfoldInterleaveOuter :: o -> [i] -> ConcatUnfoldInterleaveState o i
404− [A] ConcatUnfoldInterleaveInnerR :: [i] -> [i] -> ConcatUnfoldInterleaveState o i
405− [A] ConcatUnfoldInterleaveInnerL :: [i] -> [i] -> ConcatUnfoldInterleaveState o i
406− [A] ConcatUnfoldInterleaveInner :: o -> [i] -> ConcatUnfoldInterleaveState o i
407− [A] ConcatMapUState
408− [A] ConcatMapUOuter :: o -> ConcatMapUState o i
409− [A] ConcatMapUInner :: o -> i -> ConcatMapUState o i
410− [A] AppendState
411− [A] AppendSecond :: s2 -> AppendState s1 s2
412− [A] AppendFirst :: s1 -> AppendState s1 s2
413− [A] pattern Stream :: (State StreamK m a -> s -> m (Step s a)) -> s -> Stream m a
414− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Stream m b -> Stream m c
415− [A] zipWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
416− [A] wordsBy :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
417− [A] with :: Monad m => (Stream m a -> Stream m (s, a)) -> (((s, a) -> b) -> Stream m (s, a) -> Stream m (s, a)) -> ((s, a) -> b) -> Stream m a -> Stream m a
418− [A] usingStateT :: Monad m => m s -> (Stream (StateT s m) a -> Stream (StateT s m) a) -> Stream m a -> Stream m a
419− [A] usingReaderT :: Monad m => m r -> (Stream (ReaderT r m) a -> Stream (ReaderT r m) a) -> Stream m a -> Stream m a
420− [A] uniqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a
421− [A] uniq :: (Eq a, Monad m) => Stream m a -> Stream m a
422− [A] unionWithStreamGenericBy :: MonadIO m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
423− [A] unionWithStreamAscBy :: (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
424− [A] unfoldrM :: Monad m => (s -> m (Maybe (a, s))) -> s -> Stream m a
425− [A] unfoldr :: Monad m => (s -> Maybe (a, s)) -> s -> Stream m a
426− [A] unfoldRoundRobin :: Monad m => Unfold m a b -> Stream m a -> Stream m b
427− [A] unfoldMany :: Monad m => Unfold m a b -> Stream m a -> Stream m b
428− [A] unfoldIterateDfs :: Monad m => Unfold m a a -> Stream m a -> Stream m a
429− [A] unfoldIterateBfsRev :: Monad m => Unfold m a a -> Stream m a -> Stream m a
430− [A] unfoldIterateBfs :: Monad m => Unfold m a a -> Stream m a -> Stream m a
431− [A] unfoldInterleave :: Monad m => Unfold m a b -> Stream m a -> Stream m b
432− [A] unfold :: Applicative m => Unfold m a b -> a -> Stream m b
433− [A] uncons :: Monad m => Stream m a -> m (Maybe (a, Stream m a))
434− [A] unCross :: CrossStream m a -> Stream m a
435− [A] transform :: Monad m => Pipe m a b -> Stream m a -> Stream m b
436− [A] trace_ :: Monad m => m b -> Stream m a -> Stream m a
437− [A] trace :: Monad m => (a -> m b) -> Stream m a -> Stream m a
438− [A] toStreamK :: Monad m => Stream m a -> StreamK m a
439− [A] toListRev :: Monad m => Stream m a -> m [a]
440− [A] toList :: Monad m => Stream m a -> m [a]
441− [A] timestamped :: MonadIO m => Stream m a -> Stream m (AbsTime, a)
442− [A] timestampWith :: MonadIO m => Double -> Stream m a -> Stream m (AbsTime, a)
443− [A] timesWith :: MonadIO m => Double -> Stream m (AbsTime, RelTime64)
444− [A] times :: MonadIO m => Stream m (AbsTime, RelTime64)
445− [A] timeout :: AbsTime -> t m ()
446− [A] timeIndexed :: MonadIO m => Stream m a -> Stream m (RelTime64, a)
447− [A] timeIndexWith :: MonadIO m => Double -> Stream m a -> Stream m (RelTime64, a)
448− [A] the :: (Eq a, Monad m) => Stream m a -> m (Maybe a)
449− [A] tapOffsetEvery :: Monad m => Int -> Int -> Fold m a b -> Stream m a -> Stream m a
450− [A] tap :: Monad m => Fold m a b -> Stream m a -> Stream m a
451− [A] takeWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
452− [A] takeWhileLast :: (a -> Bool) -> Stream m a -> Stream m a
453− [A] takeWhileAround :: (a -> Bool) -> Stream m a -> Stream m a
454− [A] takeWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
455− [A] takeEndByM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
456− [A] takeEndBy :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
457− [A] take :: Applicative m => Int -> Stream m a -> Stream m a
458− [A] tail :: Monad m => Stream m a -> m (Maybe (Stream m a))
459− [A] stripSuffixUnbox :: (MonadIO m, Eq a, Unbox a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
460− [A] stripSuffix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
461− [A] stripPrefix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
462− [A] strideFromThen :: Monad m => Int -> Int -> Stream m a -> Stream m a
463− [A] splitOnSuffixSeqAny :: [Array a] -> Fold m a b -> Stream m a -> Stream m b
464− [A] splitOnSuffixSeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) => Bool -> Array a -> Fold m a b -> Stream m a -> Stream m b
465− [A] splitOnSeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) => Array a -> Fold m a b -> Stream m a -> Stream m b
466− [A] splitOnPrefix :: (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
467− [A] splitOnAny :: [Array a] -> Fold m a b -> Stream m a -> Stream m b
468− [A] splitOn :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
469− [A] splitInnerBySuffix :: (Monad m, Eq (f a), Monoid (f a)) => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
470− [A] splitInnerBy :: Monad m => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
471− [A] slicesBy :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
472− [A] sliceOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
473− [A] sequence :: Monad m => Stream m (m a) -> Stream m a
474− [A] scanlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> Stream m b
475− [A] scanlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> Stream m b
476− [A] scanlMAfter' :: Monad m => (b -> a -> m b) -> m b -> (b -> m b) -> Stream m a -> Stream m b
477− [A] scanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
478− [A] scanlM :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
479− [A] scanl1M' :: Monad m => (a -> a -> m a) -> Stream m a -> Stream m a
480− [A] scanl1M :: Monad m => (a -> a -> m a) -> Stream m a -> Stream m a
481− [A] scanl1' :: Monad m => (a -> a -> a) -> Stream m a -> Stream m a
482− [A] scanl1 :: Monad m => (a -> a -> a) -> Stream m a -> Stream m a
483− [A] scanl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
484− [A] scanl :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
485− [A] scanMaybe :: Monad m => Fold m a (Maybe b) -> Stream m a -> Stream m b
486− [A] scanMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
487− [A] scan :: Monad m => Fold m a b -> Stream m a -> Stream m b
488− [A] runStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m (s, a)
489− [A] runReaderT :: Monad m => m s -> Stream (ReaderT s m) a -> Stream m a
490− [A] runInnerWithState :: Monad m => (forall b. s -> t m b -> m (b, s)) -> m s -> Stream (t m) a -> Stream m (s, a)
491− [A] runInnerWith :: Monad m => (forall b. t m b -> m b) -> Stream (t m) a -> Stream m a
492− [A] roundRobin :: Monad m => Stream m a -> Stream m a -> Stream m a
493− [A] rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Stream m a -> Stream m b
494− [A] rollingMap2 :: Monad m => (a -> a -> b) -> Stream m a -> Stream m b
495− [A] rollingMap :: Monad m => (Maybe a -> a -> b) -> Stream m a -> Stream m b
496− [A] reverseUnbox :: (MonadIO m, Unbox a) => Stream m a -> Stream m a
497− [A] reverse :: Monad m => Stream m a -> Stream m a
498− [A] replicateM :: Monad m => Int -> m a -> Stream m a
499− [A] replicate :: Monad m => Int -> a -> Stream m a
500− [A] repeated :: Stream m a -> Stream m a
501− [A] repeatM :: Monad m => m a -> Stream m a
502− [A] repeat :: Monad m => a -> Stream m a
503− [A] relTimesWith :: MonadIO m => Double -> Stream m RelTime64
504− [A] relTimes :: MonadIO m => Stream m RelTime64
505− [A] refoldMany :: Monad m => Refold m x a b -> m x -> Stream m a -> Stream m b
506− [A] refoldIterateM :: Monad m => Refold m b a b -> m b -> Stream m a -> Stream m b
507− [A] reduceIterateBfs :: Monad m => (a -> a -> m a) -> Stream m a -> m (Maybe a)
508− [A] reassembleBy :: Fold m a b -> (a -> a -> Int) -> Stream m a -> Stream m b
509− [A] prune :: (a -> Bool) -> Stream m a -> Stream m a
510− [A] prescanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
511− [A] prescanl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
512− [A] postscanlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> Stream m b
513− [A] postscanlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> Stream m b
514− [A] postscanlMAfter' :: Monad m => (b -> a -> m b) -> m b -> (b -> m b) -> Stream m a -> Stream m b
515− [A] postscanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
516− [A] postscanlM :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
517− [A] postscanl' :: Monad m => (a -> b -> a) -> a -> Stream m b -> Stream m a
518− [A] postscanl :: Monad m => (a -> b -> a) -> a -> Stream m b -> Stream m a
519− [A] postscan :: Monad m => Fold m a b -> Stream m a -> Stream m b
520− [A] parseSequence :: Stream m (Parser a m b) -> Stream m a -> Stream m b
521− [A] parseManyTill :: Parser a m b -> Parser a m x -> Stream m a -> Stream m b
522− [A] parseManyD :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
523− [A] parseMany :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
524− [A] parseIterateD :: Monad m => (b -> Parser a m b) -> b -> Stream m a -> Stream m (Either ParseError b)
525− [A] parseIterate :: Monad m => (b -> Parser a m b) -> b -> Stream m a -> Stream m (Either ParseError b)
526− [A] parseD :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
527− [A] parseBreakD :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b, Stream m a)
528− [A] parseBreak :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b, Stream m a)
529− [A] parse :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
530− [A] onException :: MonadCatch m => m b -> Stream m a -> Stream m a
531− [A] null :: Monad m => Stream m a -> m Bool
532− [A] nub :: (Monad m, Ord a) => Stream m a -> Stream m a
533− [A] notElem :: (Monad m, Eq a) => a -> Stream m a -> m Bool
534− [A] nilM :: Applicative m => m b -> Stream m a
535− [A] nil :: Applicative m => Stream m a
536− [A] morphInner :: Monad n => (forall x. m x -> n x) -> Stream m a -> Stream n a
537− [A] mkCross :: Stream m a -> CrossStream m a
538− [A] minimumBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> m (Maybe a)
539− [A] minimum :: (Monad m, Ord a) => Stream m a -> m (Maybe a)
540− [A] mergeMinBy :: (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
541− [A] mergeFstBy :: (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
542− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
543− [A] mergeBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
544− [A] maximumBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> m (Maybe a)
545− [A] maximum :: (Monad m, Ord a) => Stream m a -> m (Maybe a)
546− [A] mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Stream m a -> Stream m b
547− [A] mapMaybe :: Monad m => (a -> Maybe b) -> Stream m a -> Stream m b
548− [A] mapM_ :: Monad m => (a -> m b) -> Stream m a -> m ()
549− [A] mapM :: Monad m => (a -> m b) -> Stream m a -> Stream m b
550− [A] map :: Monad m => (a -> b) -> Stream m a -> Stream m b
551− [A] lookup :: (Monad m, Eq a) => a -> Stream m (a, b) -> m (Maybe b)
552− [A] liftInnerWith :: Monad (t m) => (forall b. m b -> t m b) -> Stream m a -> Stream (t m) a
553− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => Stream m a -> Stream (t m) a
554− [A] last :: Monad m => Stream m a -> m (Maybe a)
555− [A] joinOuterGeneric :: MonadIO m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (Maybe a, Maybe b)
556− [A] joinOuterAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (Maybe a, Maybe b)
557− [A] joinOuter :: (Ord k, MonadIO m) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, Maybe a, Maybe b)
558− [A] joinLeftGeneric :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (a, Maybe b)
559− [A] joinLeftAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (a, Maybe b)
560− [A] joinLeft :: (Ord k, Monad m) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, a, Maybe b)
561− [A] joinInnerGeneric :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (a, b)
562− [A] joinInnerAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (a, b)
563− [A] joinInner :: (Monad m, Ord k) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, a, b)
564− [A] iterateM :: Monad m => (a -> m a) -> m a -> Stream m a
565− [A] iterate :: Monad m => (a -> a) -> a -> Stream m a
566− [A] isSuffixOfUnbox :: (MonadIO m, Eq a, Unbox a) => Stream m a -> Stream m a -> m Bool
567− [A] isSuffixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
568− [A] isSubsequenceOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
569− [A] isPrefixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
570− [A] isInfixOf :: (MonadIO m, Eq a, Enum a, Storable a, Unbox a) => Stream m a -> Stream m a -> m Bool
571− [A] intersperseM_ :: Monad m => m b -> Stream m a -> Stream m a
572− [A] intersperseMWith :: Int -> m a -> Stream m a -> Stream m a
573− [A] intersperseMSuffix_ :: Monad m => m b -> Stream m a -> Stream m a
574− [A] intersperseMSuffixWith :: forall m a. Monad m => Int -> m a -> Stream m a -> Stream m a
575− [A] intersperseMSuffix :: forall m a. Monad m => m a -> Stream m a -> Stream m a
576− [A] intersperseMPrefix_ :: Monad m => m b -> Stream m a -> Stream m a
577− [A] intersperseM :: Monad m => m a -> Stream m a -> Stream m a
578− [A] intersperse :: Monad m => a -> Stream m a -> Stream m a
579− [A] intersectBySorted :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
580− [A] interposeSuffixM :: Monad m => m c -> Unfold m b c -> Stream m b -> Stream m c
581− [A] interposeSuffix :: Monad m => c -> Unfold m b c -> Stream m b -> Stream m c
582− [A] interposeM :: Monad m => m c -> Unfold m b c -> Stream m b -> Stream m c
583− [A] interpose :: Monad m => c -> Unfold m b c -> Stream m b -> Stream m c
584− [A] interleaveMin :: Monad m => Stream m a -> Stream m a -> Stream m a
585− [A] interleaveFstSuffix :: Monad m => Stream m a -> Stream m a -> Stream m a
586− [A] interleaveFst :: Monad m => Stream m a -> Stream m a -> Stream m a
587− [A] interleave :: Monad m => Stream m a -> Stream m a -> Stream m a
588− [A] intercalateSuffix :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
589− [A] intercalate :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
590− [A] insertBy :: Monad m => (a -> a -> Ordering) -> a -> Stream m a -> Stream m a
591− [A] indexedR :: Monad m => Int -> Stream m a -> Stream m (Int, a)
592− [A] indexed :: Monad m => Stream m a -> Stream m (Int, a)
593− [A] headElse :: Monad m => a -> Stream m a -> m a
594− [A] head :: Monad m => Stream m a -> m (Maybe a)
595− [A] handle :: (MonadCatch m, Exception e) => (e -> m (Stream m a)) -> Stream m a -> Stream m a
596− [A] groupsWhile :: Monad m => (a -> a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
597− [A] groupsRollingBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
598− [A] groupsOf :: Monad m => Int -> Fold m a b -> Stream m a -> Stream m b
599− [A] gintercalateSuffix :: Monad m => Unfold m a c -> Stream m a -> Unfold m b c -> Stream m b -> Stream m c
600− [A] gintercalate :: Monad m => Unfold m a c -> Stream m a -> Unfold m b c -> Stream m b -> Stream m c
601− [A] ghandle :: (MonadCatch m, Exception e) => (e -> Stream m a -> m (Stream m a)) -> Stream m a -> Stream m a
602− [A] generateM :: Monad m => Int -> (Int -> m a) -> Stream m a
603− [A] generate :: Monad m => Int -> (Int -> a) -> Stream m a
604− [A] generalizeInner :: Monad m => Stream Identity a -> Stream m a
605− [A] gbracket_ :: Monad m => m c -> (c -> m d) -> (c -> e -> Stream m b -> m (Stream m b)) -> (forall s. m s -> m (Either e s)) -> (c -> Stream m b) -> Stream m b
606− [A] gbracket :: MonadIO m => IO c -> (c -> IO d1) -> (c -> e -> Stream m b -> IO (Stream m b)) -> (c -> IO d2) -> (forall s. m s -> m (Either e s)) -> (c -> Stream m b) -> Stream m b
607− [A] fromStreamK :: Applicative m => StreamK m a -> Stream m a
608− [A] fromPure :: Applicative m => a -> Stream m a
609− [A] fromPtrN :: (Monad m, Storable a) => Int -> Ptr a -> Stream m a
610− [A] fromPtr :: forall m a. (Monad m, Storable a) => Ptr a -> Stream m a
611− [A] fromListM :: Monad m => [m a] -> Stream m a
612− [A] fromList :: Applicative m => [a] -> Stream m a
613− [A] fromIndicesM :: Monad m => (Int -> m a) -> Stream m a
614− [A] fromIndices :: Monad m => (Int -> a) -> Stream m a
615− [A] fromFoldableM :: (Monad m, Foldable f) => f (m a) -> Stream m a
616− [A] fromFoldable :: (Monad m, Foldable f) => f a -> Stream m a
617− [A] fromEffect :: Applicative m => m a -> Stream m a
618− [A] fromByteStr# :: Monad m => Addr# -> Stream m Word8
619− [A] foldrT :: (Monad m, Monad (t m), MonadTrans t) => (a -> t m b -> t m b) -> t m b -> Stream m a -> t m b
620− [A] foldrS :: Monad m => (a -> Stream m b -> Stream m b) -> Stream m b -> Stream m a -> Stream m b
621− [A] foldrMx :: Monad m => (a -> m x -> m x) -> m x -> (m x -> m b) -> Stream m a -> m b
622− [A] foldrM :: Monad m => (a -> m b -> m b) -> m b -> Stream m a -> m b
623− [A] foldr1 :: Monad m => (a -> a -> a) -> Stream m a -> m (Maybe a)
624− [A] foldr :: Monad m => (a -> b -> b) -> b -> Stream m a -> m b
625− [A] foldlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> m b
626− [A] foldlT :: (Monad m, Monad (s m), MonadTrans s) => (s m b -> a -> s m b) -> s m b -> Stream m a -> s m b
627− [A] foldlS :: Monad m => (Stream m b -> a -> Stream m b) -> Stream m b -> Stream m a -> Stream m b
628− [A] foldlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> m b
629− [A] foldlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> m b
630− [A] foldl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> m b
631− [A] foldSequence :: Stream m (Fold m a b) -> Stream m a -> Stream m b
632− [A] foldManyPost :: Monad m => Fold m a b -> Stream m a -> Stream m b
633− [A] foldMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
634− [A] foldIterateM :: Monad m => (b -> m (Fold m a b)) -> m b -> Stream m a -> Stream m b
635− [A] foldIterateBfs :: Fold m a (Either a a) -> Stream m a -> m (Maybe a)
636− [A] foldEither :: Monad m => Fold m a b -> Stream m a -> m (Either (Fold m a b) (b, Stream m a))
637− [A] foldBreak :: Monad m => Fold m a b -> Stream m a -> m (b, Stream m a)
638− [A] foldAddLazy :: Monad m => Fold m a b -> Stream m a -> Fold m a b
639− [A] foldAdd :: Monad m => Fold m a b -> Stream m a -> m (Fold m a b)
640− [A] fold :: Monad m => Fold m a b -> Stream m a -> m b
641− [A] findM :: Monad m => (a -> m Bool) -> Stream m a -> m (Maybe a)
642− [A] findIndices :: Monad m => (a -> Bool) -> Stream m a -> Stream m Int
643− [A] find :: Monad m => (a -> Bool) -> Stream m a -> m (Maybe a)
644− [A] finallyUnsafe :: MonadCatch m => m b -> Stream m a -> Stream m a
645− [A] finallyIO :: (MonadIO m, MonadCatch m) => IO b -> Stream m a -> Stream m a
646− [A] filterM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
647− [A] filterInStreamGenericBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
648− [A] filterInStreamAscBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
649− [A] filter :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
650− [A] evalStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m a
651− [A] eqBy :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> m Bool
652− [A] enumerateTo :: (Monad m, Bounded a, Enumerable a) => a -> Stream m a
653− [A] enumerateFromToSmall :: (Monad m, Enum a) => a -> a -> Stream m a
654− [A] enumerateFromToIntegral :: (Monad m, Integral a) => a -> a -> Stream m a
655− [A] enumerateFromToFractional :: (Monad m, Fractional a, Ord a) => a -> a -> Stream m a
656− [A] enumerateFromTo :: (Enumerable a, Monad m) => a -> a -> Stream m a
657− [A] enumerateFromThenToSmall :: (Monad m, Enum a) => a -> a -> a -> Stream m a
658− [A] enumerateFromThenToIntegral :: (Monad m, Integral a) => a -> a -> a -> Stream m a
659− [A] enumerateFromThenToFractional :: (Monad m, Fractional a, Ord a) => a -> a -> a -> Stream m a
660− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => a -> a -> a -> Stream m a
661− [A] enumerateFromThenSmallBounded :: (Monad m, Enumerable a, Bounded a) => a -> a -> Stream m a
662− [A] enumerateFromThenNum :: (Monad m, Num a) => a -> a -> Stream m a
663− [A] enumerateFromThenIntegral :: (Monad m, Integral a, Bounded a) => a -> a -> Stream m a
664− [A] enumerateFromThenFractional :: (Monad m, Fractional a) => a -> a -> Stream m a
665− [A] enumerateFromThen :: (Enumerable a, Monad m) => a -> a -> Stream m a
666− [A] enumerateFromStepNum :: (Monad m, Num a) => a -> a -> Stream m a
667− [A] enumerateFromStepIntegral :: (Integral a, Monad m) => a -> a -> Stream m a
668− [A] enumerateFromNum :: (Monad m, Num a) => a -> Stream m a
669− [A] enumerateFromIntegral :: (Monad m, Integral a, Bounded a) => a -> Stream m a
670− [A] enumerateFromFractional :: (Monad m, Fractional a) => a -> Stream m a
671− [A] enumerateFromBounded :: (Monad m, Enumerable a, Bounded a) => a -> Stream m a
672− [A] enumerateFrom :: (Enumerable a, Monad m) => a -> Stream m a
673− [A] enumerate :: (Monad m, Bounded a, Enumerable a) => Stream m a
674− [A] elemIndices :: (Monad m, Eq a) => a -> Stream m a -> Stream m Int
675− [A] elem :: (Monad m, Eq a) => a -> Stream m a -> m Bool
676− [A] durations :: Double -> t m RelTime64
677− [A] dropWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
678− [A] dropWhileLast :: (a -> Bool) -> Stream m a -> Stream m a
679− [A] dropWhileAround :: (a -> Bool) -> Stream m a -> Stream m a
680− [A] dropWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
681− [A] dropSuffix :: Stream m a -> Stream m a -> Stream m a
682− [A] dropPrefix :: Stream m a -> Stream m a -> Stream m a
683− [A] dropLast :: Int -> Stream m a -> Stream m a
684− [A] dropInfix :: Stream m a -> Stream m a -> Stream m a
685− [A] drop :: Monad m => Int -> Stream m a -> Stream m a
686− [A] drain :: Monad m => Stream m a -> m ()
687− [A] deleteInStreamGenericBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
688− [A] deleteInStreamAscBy :: (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
689− [A] deleteBy :: Monad m => (a -> a -> Bool) -> a -> Stream m a -> Stream m a
690− [A] delayPre :: MonadIO m => Double -> Stream m a -> Stream m a
691− [A] delayPost :: MonadIO m => Double -> Stream m a -> Stream m a
692− [A] delay :: MonadIO m => Double -> Stream m a -> Stream m a
693− [A] crossWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
694− [A] crossApplySnd :: Functor f => Stream f a -> Stream f b -> Stream f b
695− [A] crossApplyFst :: Functor f => Stream f a -> Stream f b -> Stream f a
696− [A] crossApply :: Functor f => Stream f (a -> b) -> Stream f a -> Stream f b
697− [A] cross :: Monad m => Stream m a -> Stream m b -> Stream m (a, b)
698− [A] consM :: Applicative m => m a -> Stream m a -> Stream m a
699− [A] cons :: Applicative m => a -> Stream m a -> Stream m a
700− [A] concatMapM :: Monad m => (a -> m (Stream m b)) -> Stream m a -> Stream m b
701− [A] concatMap :: Monad m => (a -> Stream m b) -> Stream m a -> Stream m b
702− [A] concatIterateScan :: Monad m => (b -> a -> m b) -> (b -> m (Maybe (b, Stream m a))) -> b -> Stream m a
703− [A] concatIterateDfs :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
704− [A] concatIterateBfsRev :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
705− [A] concatIterateBfs :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
706− [A] concatEffect :: Monad m => m (Stream m a) -> Stream m a
707− [A] concat :: Monad m => Stream m (Stream m a) -> Stream m a
708− [A] cmpBy :: Monad m => (a -> b -> Ordering) -> Stream m a -> Stream m b -> m Ordering
709− [A] catRights :: Monad m => Stream m (Either a b) -> Stream m b
710− [A] catMaybes :: Monad m => Stream m (Maybe a) -> Stream m a
711− [A] catLefts :: Monad m => Stream m (Either a b) -> Stream m a
712− [A] catEithers :: Monad m => Stream m (Either a a) -> Stream m a
713− [A] bracketUnsafe :: MonadCatch m => m b -> (b -> m c) -> (b -> Stream m a) -> Stream m a
714− [A] bracketIO3 :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> IO d) -> (b -> IO e) -> (b -> Stream m a) -> Stream m a
715− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> Stream m a) -> Stream m a
716− [A] before :: Monad m => m b -> Stream m a -> Stream m a
717− [A] append :: Monad m => Stream m a -> Stream m a -> Stream m a
718− [A] any :: Monad m => (a -> Bool) -> Stream m a -> m Bool
719− [A] all :: Monad m => (a -> Bool) -> Stream m a -> m Bool
720− [A] afterUnsafe :: Monad m => m b -> Stream m a -> Stream m a
721− [A] afterIO :: MonadIO m => IO b -> Stream m a -> Stream m a
722− [A] absTimesWith :: MonadIO m => Double -> Stream m AbsTime
723− [A] absTimes :: MonadIO m => Stream m AbsTime
724− [A] (!!) :: Monad m => Stream m a -> Int -> m (Maybe a)
725−[R] Streamly.Internal.Data.Ring.Unboxed
726−[A] Streamly.Internal.Data.Ring.Generic
727− [A] Ring
728− [A] [ringMax] :: Ring a -> !Int
729− [A] [ringHead] :: Ring a -> !Int
730− [A] [ringArr] :: Ring a -> MutArray a
731− [A] Ring :: MutArray a -> !Int -> !Int -> Ring a
732− [A] writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
733− [A] unsafeInsertRingWith :: Ring a -> a -> IO Int
734− [A] toStreamWith :: Int -> Ring a -> Stream m a
735− [A] toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
736− [A] seek :: MonadIO m => Int -> Ring a -> m (Ring a)
737− [A] createRing :: MonadIO m => Int -> m (Ring a)
738− [A] copyToMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
739−[C] Streamly.Internal.Data.Ring
740− [C] Ring
741− [A] [ringStart] :: Ring a -> {-# UNPACK #-} !ForeignPtr a
742− [R] [ringMax] :: Ring a -> !Int
743− [R] [ringHead] :: Ring a -> !Int
744− [A] [ringBound] :: Ring a -> {-# UNPACK #-} !Ptr a
745− [R] [ringArr] :: Ring a -> MutArray a
746− [C] Ring
747− [O] Ring :: MutArray a -> !Int -> !Int -> Ring a
748− [N] Ring :: {-# UNPACK #-} !ForeignPtr a -> {-# UNPACK #-} !Ptr a -> Ring a
749− [A] GHC.Show.Show
750− [A] instance (GHC.Show.Show a, GHC.Show.Show b, GHC.Show.Show c, GHC.Show.Show d) => GHC.Show.Show (Streamly.Internal.Data.Ring.Tuple4' a b c d)
751− [A] writeN :: Int -> Fold m a (Ring a)
752− [R] writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
753− [R] unsafeInsertRingWith :: Ring a -> a -> IO Int
754− [A] unsafeInsert :: Storable a => Ring a -> Ptr a -> a -> IO (Ptr a)
755− [A] unsafeFoldRingNM :: forall m a b. (MonadIO m, Storable a) => Int -> Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
756− [A] unsafeFoldRingM :: forall m a b. (MonadIO m, Storable a) => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
757− [A] unsafeFoldRingFullM :: forall m a b. (MonadIO m, Storable a) => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
758− [A] unsafeFoldRing :: forall a b. Storable a => Ptr a -> (b -> a -> b) -> b -> Ring a -> b
759− [A] unsafeEqArrayN :: Ring a -> Ptr a -> Array a -> Int -> Bool
760− [A] unsafeEqArray :: Ring a -> Ptr a -> Array a -> Bool
761− [R] toStreamWith :: Int -> Ring a -> Stream m a
762− [R] toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
763− [A] startOf :: Ring a -> Ptr a
764− [A] slidingWindowWith :: forall m a b. (MonadIO m, Storable a, Unbox a) => Int -> Fold m ((a, Maybe a), m (MutArray a)) b -> Fold m a b
765− [A] slidingWindow :: forall m a b. (MonadIO m, Storable a, Unbox a) => Int -> Fold m (a, Maybe a) b -> Fold m a b
766− [A] slide :: Ring a -> a -> m (Ring a)
767− [R] seek :: MonadIO m => Int -> Ring a -> m (Ring a)
768− [A] ringsOf :: Int -> Stream m a -> Stream m (MutArray a)
769− [A] readRev :: Unfold m (MutArray a) a
770− [A] read :: forall m a. (MonadIO m, Storable a) => Unfold m (Ring a, Ptr a, Int) a
771− [A] putIndex :: Ring a -> Int -> a -> m ()
772− [A] newRing :: Int -> m (Ring a)
773− [A] new :: forall a. Storable a => Int -> IO (Ring a, Ptr a)
774− [A] moveBy :: forall a. Storable a => Int -> Ring a -> Ptr a -> Ptr a
775− [A] modifyIndex :: Ring a -> Int -> (a -> (a, b)) -> m b
776− [A] length :: Ring a -> Int
777− [A] getIndexUnsafe :: Ring a -> Int -> m a
778− [A] getIndexRev :: Ring a -> Int -> m a
779− [A] getIndex :: Ring a -> Int -> m a
780− [A] fromArray :: MutArray a -> Ring a
781− [R] createRing :: MonadIO m => Int -> m (Ring a)
782− [A] castUnsafe :: Ring a -> Ring b
783− [A] cast :: forall a b. Storable b => Ring a -> Maybe (Ring b)
784− [A] bytesFree :: Ring a -> Int
785− [A] byteLength :: Ring a -> Int
786− [A] byteCapacity :: Ring a -> Int
787− [A] asBytes :: Ring a -> Ring Word8
788− [A] advance :: forall a. Storable a => Ring a -> Ptr a -> Ptr a
789−[R] Streamly.Internal.Data.Producer.Type
790−[R] Streamly.Internal.Data.Producer.Source
791−[C] Streamly.Internal.Data.Producer
792− [A] Source
793− [A] unread :: [b] -> Source a b -> Source a b
794− [A] translate :: Functor m => (a -> c) -> (c -> a) -> Producer m c b -> Producer m a b
795− [A] source :: Maybe a -> Source a b
796− [A] producer :: Monad m => Producer m a b -> Producer m (Source a b) b
797− [A] parseManyD :: Monad m => Parser a m b -> Producer m (Source x a) a -> Producer m (Source x a) (Either ParseError b)
798− [A] parseMany :: Monad m => Parser a m b -> Producer m (Source x a) a -> Producer m (Source x a) (Either ParseError b)
799− [A] parse :: Monad m => Parser a m b -> Producer m (Source s a) a -> Source s a -> m (Either ParseError b, Source s a)
800− [A] lmap :: (a -> a) -> Producer m a b -> Producer m a b
801− [A] isEmpty :: Source a b -> Bool
802−[R] Streamly.Internal.Data.Pipe.Type
803−[C] Streamly.Internal.Data.Pipe
804− [A] Step
805− [A] Yield :: a -> s -> Step s a
806− [A] Continue :: s -> Step s a
807− [A] PipeState
808− [A] Produce :: s2 -> PipeState s1 s2
809− [A] Consume :: s1 -> PipeState s1 s2
810− [C] Pipe
811− [A] Pipe :: (s1 -> a -> m (Step (PipeState s1 s2) b)) -> (s2 -> m (Step (PipeState s1 s2) b)) -> s1 -> Pipe m a b
812−[A] Streamly.Internal.Data.ParserK
813− [A] Step
814− [A] Partial :: !Int -> (Input a -> m (Step a m r)) -> Step a m r
815− [A] Error :: !Int -> String -> Step a m r
816− [A] Done :: !Int -> r -> Step a m r
817− [A] Continue :: !Int -> (Input a -> m (Step a m r)) -> Step a m r
818− [A] ParseResult
819− [A] Success :: !Int -> !b -> ParseResult b
820− [A] Failure :: !Int -> !String -> ParseResult b
821− [A] Input
822− [A] None :: Input a
823− [A] Chunk :: a -> Input a
824− [A] ParserK
825− [A] [runParser] :: ParserK a m b -> forall r. (ParseResult b -> Int -> Input a -> m (Step a m r)) -> Int -> Int -> Input a -> m (Step a m r)
826− [A] MkParser :: (forall r. (ParseResult b -> Int -> Input a -> m (Step a m r)) -> Int -> Int -> Input a -> m (Step a m r)) -> ParserK a m b
827− [A] fromPure :: b -> ParserK a m b
828− [A] fromEffect :: Monad m => m b -> ParserK a m b
829− [A] die :: String -> ParserK a m b
830− [A] adaptCG :: Monad m => Parser a m b -> ParserK (Array a) m b
831− [A] adaptC :: (Monad m, Unbox a) => Parser a m b -> ParserK (Array a) m b
832− [A] adapt :: Monad m => Parser a m b -> ParserK a m b
833−[R] Streamly.Internal.Data.Parser.ParserK.Type
834−[R] Streamly.Internal.Data.Parser.ParserD.Type
835−[R] Streamly.Internal.Data.Parser.ParserD.Tee
836−[R] Streamly.Internal.Data.Parser.ParserD
837−[C] Streamly.Internal.Data.Parser
838− [A] Step
839− [A] Partial :: !Int -> !s -> Step s b
840− [A] Error :: !String -> Step s b
841− [A] Done :: !Int -> !b -> Step s b
842− [A] Continue :: !Int -> !s -> Step s b
843− [A] Parser
844− [A] Parser :: (s -> a -> m (Step s b)) -> m (Initial s b) -> (s -> m (Step s b)) -> Parser a m b
845− [A] Initial
846− [A] IPartial :: !s -> Initial s b
847− [A] IError :: !String -> Initial s b
848− [A] IDone :: !b -> Initial s b
849− [A] GHC.Show.Show
850− [A] instance (GHC.Show.Show a, GHC.Show.Show b) => GHC.Show.Show (Streamly.Internal.Data.Parser.Tuple'Fused a b)
851− [A] ParseError
852− [A] ParseError :: String -> ParseError
853− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Fold m c x -> Parser b m x
854− [A] zip :: Monad m => Stream m a -> Fold m (a, b) x -> Parser b m x
855− [A] wordWithQuotes :: (Monad m, Eq a) => Bool -> (a -> a -> Maybe a) -> a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
856− [A] wordProcessQuotes :: (Monad m, Eq a) => a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
857− [A] wordKeepQuotes :: (Monad m, Eq a) => a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
858− [A] wordFramedBy :: Monad m => (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
859− [A] wordBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
860− [A] toFold :: Monad m => Parser a m b -> Fold m a b
861− [A] takeWhileP :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
862− [A] takeWhile1 :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
863− [A] takeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
864− [A] takeStartBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
865− [A] takeStartBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
866− [A] takeP :: Monad m => Int -> Parser a m b -> Parser a m b
867− [A] takeGE :: Monad m => Int -> Fold m a b -> Parser a m b
868− [A] takeFramedBy_ :: Monad m => (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
869− [A] takeFramedByGeneric :: Monad m => Maybe (a -> Bool) -> Maybe (a -> Bool) -> Maybe (a -> Bool) -> Fold m a b -> Parser a m b
870− [A] takeFramedByEsc_ :: Monad m => (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
871− [A] takeEndBy_ :: (a -> Bool) -> Parser a m b -> Parser a m b
872− [A] takeEndByEsc :: Monad m => (a -> Bool) -> (a -> Bool) -> Parser a m b -> Parser a m b
873− [A] takeEndBy :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
874− [A] takeEitherSepBy :: (a -> Bool) -> Fold m (Either a b) c -> Parser a m c
875− [A] takeEQ :: Monad m => Int -> Fold m a b -> Parser a m b
876− [A] takeBetween :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
877− [A] subsequenceBy :: (a -> a -> Bool) -> Stream m a -> Parser a m ()
878− [A] streamEqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Parser a m ()
879− [A] split_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
880− [A] splitWith :: Monad m => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
881− [A] splitSome :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
882− [A] splitManyPost :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
883− [A] splitMany :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
884− [A] spanByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
885− [A] spanBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
886− [A] span :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
887− [A] some :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
888− [A] sequence :: Monad m => Stream m (Parser a m b) -> Fold m b c -> Parser a m c
889− [A] sepByAll :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
890− [A] sepBy1 :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
891− [A] sepBy :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
892− [A] satisfy :: Monad m => (a -> Bool) -> Parser a m a
893− [A] sampleFromthen :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
894− [A] roundRobin :: t (Parser a m b) -> Fold m b c -> Parser a m c
895− [A] rmapM :: Monad m => (b -> m c) -> Parser a m b -> Parser a m c
896− [A] retryMaxTotal :: Int -> Parser a m b -> Fold m b c -> Parser a m c
897− [A] retryMaxSuccessive :: Int -> Parser a m b -> Fold m b c -> Parser a m c
898− [A] retry :: Parser a m b -> Parser a m b
899− [A] postscan :: Fold m a b -> Parser b m c -> Parser a m c
900− [A] peek :: Monad m => Parser a m a
901− [A] oneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
902− [A] oneNotEq :: (Monad m, Eq a) => a -> Parser a m a
903− [A] oneEq :: (Monad m, Eq a) => a -> Parser a m a
904− [A] one :: Monad m => Parser a m a
905− [A] noneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
906− [A] noErrorUnsafeSplit_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
907− [A] noErrorUnsafeSplitWith :: Monad m => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
908− [A] noErrorUnsafeConcatMap :: Monad m => (b -> Parser a m c) -> Parser a m b -> Parser a m c
909− [A] maybe :: Monad m => (a -> Maybe b) -> Parser a m b
910− [A] manyTillP :: Parser a m b -> Parser a m x -> Parser b m c -> Parser a m c
911− [A] manyTill :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
912− [A] manyThen :: Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
913− [A] manyP :: Parser a m b -> Parser b m c -> Parser a m c
914− [A] many :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
915− [A] makeIndexFilter :: (Fold m (s, a) b -> Parser a m b) -> (((s, a) -> Bool) -> Fold m (s, a) b -> Fold m (s, a) b) -> ((s, a) -> Bool) -> Fold m a b -> Parser a m b
916− [A] lookAhead :: Monad m => Parser a m b -> Parser a m b
917− [A] lmapM :: Monad m => (a -> m b) -> Parser b m r -> Parser a m r
918− [A] lmap :: (a -> b) -> Parser b m r -> Parser a m r
919− [A] listEqBy :: Monad m => (a -> a -> Bool) -> [a] -> Parser a m [a]
920− [A] listEq :: (Monad m, Eq a) => [a] -> Parser a m [a]
921− [A] indexed :: forall m a b. Monad m => Fold m (Int, a) b -> Parser a m b
922− [A] groupByRollingEither :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
923− [A] groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
924− [A] groupBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
925− [A] fromPure :: Monad m => b -> Parser a m b
926− [A] fromFoldMaybe :: Monad m => String -> Fold m a (Maybe b) -> Parser a m b
927− [A] fromFold :: Monad m => Fold m a b -> Parser a m b
928− [A] fromEffect :: Monad m => m b -> Parser a m b
929− [A] filter :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
930− [A] extractStep :: Monad m => (s -> m (Step s1 b)) -> Step s b -> m (Step s1 b)
931− [A] eof :: Monad m => Parser a m ()
932− [A] either :: Monad m => (a -> Either String b) -> Parser a m b
933− [A] dropWhile :: Monad m => (a -> Bool) -> Parser a m ()
934− [A] dieM :: Monad m => m String -> Parser a m b
935− [A] die :: Monad m => String -> Parser a m b
936− [A] deintercalateAll :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
937− [A] deintercalate1 :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
938− [A] deintercalate :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
939− [A] countBetween :: Int -> Int -> Parser a m b -> Fold m b c -> Parser a m c
940− [A] count :: Int -> Parser a m b -> Fold m b c -> Parser a m c
941− [A] concatMap :: Monad m => (b -> Parser a m c) -> Parser a m b -> Parser a m c
942− [A] blockWithQuotes :: (Monad m, Eq a) => (a -> Bool) -> (a -> Bool) -> a -> a -> Fold m a b -> Parser a m b
943− [A] bimapOverrideCount :: Int -> (s -> s1) -> (b -> b1) -> Step s b -> Step s1 b1
944− [A] alt :: Monad m => Parser x m a -> Parser x m a -> Parser x m a
945−[A] Streamly.Internal.Data.MutByteArray
946− [A] class Unbox a
947− [A] class SizeOfRep (f :: Type -> Type)
948− [A] class Serialize a
949− [A] TypeOfType
950− [A] UnitType :: Name -> TypeOfType
951− [A] TheType :: SimpleDataCon -> TypeOfType
952− [A] MultiType :: [SimpleDataCon] -> TypeOfType
953− [A] SimpleDataCon
954− [A] SimpleDataCon :: Name -> [Field] -> SimpleDataCon
955− [A] SerializeConfig
956− [A] [cfgRecordSyntaxWithHeader] :: SerializeConfig -> Bool
957− [A] [cfgInlineSize] :: SerializeConfig -> Maybe Inline
958− [A] [cfgInlineSerialize] :: SerializeConfig -> Maybe Inline
959− [A] [cfgInlineDeserialize] :: SerializeConfig -> Maybe Inline
960− [A] [cfgConstructorTagAsString] :: SerializeConfig -> Bool
961− [A] SerializeConfig :: Maybe Inline -> Maybe Inline -> Maybe Inline -> Bool -> Bool -> SerializeConfig
962− [A] PinnedState
963− [A] Unpinned :: PinnedState
964− [A] Pinned :: PinnedState
965− [A] MutByteArray
966− [A] MutByteArray :: MutableByteArray# RealWorld -> MutByteArray
967− [A] DataType
968− [A] [dtTvs] :: DataType -> [Name]
969− [A] [dtName] :: DataType -> Name
970− [A] [dtCxt] :: DataType -> Cxt
971− [A] [dtCons] :: DataType -> [DataCon]
972− [A] DataType :: Name -> [Name] -> Cxt -> [DataCon] -> DataType
973− [A] DataCon
974− [A] [dcTvs] :: DataCon -> [Name]
975− [A] [dcName] :: DataCon -> Name
976− [A] [dcFields] :: DataCon -> [(Maybe Name, Type)]
977− [A] [dcCxt] :: DataCon -> Cxt
978− [A] DataCon :: Name -> [Name] -> Cxt -> [(Maybe Name, Type)] -> DataCon
979− [A] BoundedPtr
980− [A] BoundedPtr :: MutByteArray -> Int -> Int -> BoundedPtr
981− [A] Streamly.Internal.Data.Serialize.Type.Serialize
982− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize Streamly.Internal.Data.MutByteArray.LiftedInteger
983− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize GHC.Num.Integer.Integer
984− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize a => Streamly.Internal.Data.Serialize.Type.Serialize (GHC.Maybe.Maybe a)
985− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize (Data.Proxy.Proxy a)
986− [A] instance (Streamly.Internal.Data.Serialize.Type.Serialize a, Streamly.Internal.Data.Serialize.Type.Serialize b) => Streamly.Internal.Data.Serialize.Type.Serialize (Data.Either.Either a b)
987− [A] Peeker
988− [A] Peeker :: Builder BoundedPtr IO a -> Peeker a
989− [A] type Field = (Maybe Name, Type)
990− [A] xorCmp :: [Word8] -> Name -> Name -> Q Exp
991− [A] wListToString :: [Word8] -> String
992− [A] w8_int :: Word8 -> Int
993− [A] w32_int :: Word32 -> Int
994− [A] unpin :: MutByteArray -> IO MutByteArray
995− [A] typeOfType :: Type -> [DataCon] -> TypeOfType
996− [A] skipByte :: Peeker ()
997− [A] sizeOfRep :: SizeOfRep f => f x -> Int
998− [A] sizeOfMutableByteArray :: MutByteArray -> IO Int
999− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
1000− [A] simplifyDataCon :: DataCon -> SimpleDataCon
1001− [A] serializeW8List :: Name -> Name -> [Word8] -> Q Exp
1002− [A] serializeConfig :: SerializeConfig
1003− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
1004− [A] runPeeker :: Peeker a -> BoundedPtr -> IO a
1005− [A] reifyDataType :: Name -> Q DataType
1006− [A] readUnsafe :: Unbox a => Peeker a
1007− [A] read :: Unbox a => Peeker a
1008− [A] putSliceUnsafe :: MonadIO m => MutByteArray -> Int -> MutByteArray -> Int -> Int -> m ()
1009− [A] pokeRep :: PokeRep f => f a -> BoundedPtr -> IO BoundedPtr
1010− [A] pokeBoundedPtrUnsafe :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr
1011− [A] pokeBoundedPtr :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr
1012− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
1013− [A] pinnedNewAlignedBytes :: Int -> Int -> IO MutByteArray
1014− [A] pinnedNew :: Int -> IO MutByteArray
1015− [A] pinnedCloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
1016− [A] pin :: MutByteArray -> IO MutByteArray
1017− [A] peekRep :: PeekRep f => Peeker (f x)
1018− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
1019− [A] openConstructor :: Name -> Int -> Q Pat
1020− [A] nil :: MutByteArray
1021− [A] newBytesAs :: PinnedState -> Int -> IO MutByteArray
1022− [A] new :: Int -> IO MutByteArray
1023− [A] mkSerializeExprFields :: Name -> [Field] -> Q Exp
1024− [A] mkRecSizeOfExpr :: SimpleDataCon -> Q Exp
1025− [A] mkRecSerializeExpr :: Name -> SimpleDataCon -> Q Exp
1026− [A] mkRecDeserializeExpr :: Name -> Name -> Name -> SimpleDataCon -> Q Exp
1027− [A] mkFieldName :: Int -> Name
1028− [A] mkDeserializeKeysDec :: Name -> Name -> SimpleDataCon -> Q [Dec]
1029− [A] mkDeserializeExprOne :: Name -> SimpleDataCon -> Q Exp
1030− [A] matchConstructor :: Name -> Int -> Q Exp -> Q Match
1031− [A] makeN :: Int -> Name
1032− [A] makeI :: Int -> Name
1033− [A] makeA :: Int -> Name
1034− [A] litProxy :: Unbox a => Proxy a -> Q Exp
1035− [A] litIntegral :: Integral a => a -> Q Exp
1036− [A] isUnitType :: [DataCon] -> Bool
1037− [A] isRecordSyntax :: SimpleDataCon -> Bool
1038− [A] isPinned :: MutByteArray -> Bool
1039− [A] int_w8 :: Int -> Word8
1040− [A] int_w32 :: Int -> Word32
1041− [A] inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
1042− [A] inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
1043− [A] inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig
1044− [A] getMutableByteArray# :: MutByteArray -> MutableByteArray# RealWorld
1045− [A] genericSizeOf :: forall a. SizeOfRep (Rep a) => Proxy a -> Int
1046− [A] genericPokeByteIndex :: (Generic a, PokeRep (Rep a)) => MutByteArray -> Int -> a -> IO ()
1047− [A] genericPeekByteIndex :: (Generic a, PeekRep (Rep a)) => MutByteArray -> Int -> IO a
1048− [A] errorUnsupported :: String -> a
1049− [A] errorUnimplemented :: a
1050− [A] encodeRecordFields :: Bool -> SerializeConfig -> SerializeConfig
1051− [A] encodeConstrNames :: Bool -> SerializeConfig -> SerializeConfig
1052− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
1053− [A] deriveUnbox :: Q [Dec] -> Q [Dec]
1054− [A] deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]
1055− [A] deriveSerialize :: Q [Dec] -> Q [Dec]
1056− [A] conUpdateFuncDec :: Name -> [Field] -> Q [Dec]
1057− [A] cloneSliceUnsafeAs :: MonadIO m => PinnedState -> Int -> Int -> MutByteArray -> m MutByteArray
1058− [A] cloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
1059− [A] c2w :: Char -> Word8
1060− [A] asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
1061− [A] addSizeTo :: Serialize a => Int -> a -> Int
1062− [A] _x :: Name
1063− [A] _val :: Name
1064− [A] _tag :: Name
1065− [A] _initialOffset :: Name
1066− [A] _endOffset :: Name
1067− [A] _arr :: Name
1068− [A] _acc :: Name
1069−[A] Streamly.Internal.Data.MutArray.Stream
1070− [A] SpliceState
1071− [A] SpliceYielding :: arr -> SpliceState s arr -> SpliceState s arr
1072− [A] SpliceInitial :: s -> SpliceState s arr
1073− [A] SpliceFinish :: SpliceState s arr
1074− [A] SpliceBuffering :: s -> arr -> SpliceState s arr
1075− [A] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
1076− [A] splitOn :: (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> Stream m (MutArray a)
1077− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1078− [A] packArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1079− [A] lpackArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
1080− [A] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
1081− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1082− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1083− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (Either ParseError (MutArray a))
1084− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1085− [A] compactEQ :: Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1086− [A] compact :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1087− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1088−[A] Streamly.Internal.Data.MutArray.Generic
1089− [A] MutArray
1090− [A] [arrTrueLen] :: MutArray a -> {-# UNPACK #-} !Int
1091− [A] [arrStart] :: MutArray a -> {-# UNPACK #-} !Int
1092− [A] [arrLen] :: MutArray a -> {-# UNPACK #-} !Int
1093− [A] [arrContents#] :: MutArray a -> MutableArray# RealWorld a
1094− [A] MutArray :: MutableArray# RealWorld a -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> MutArray a
1095− [A] writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
1096− [A] writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
1097− [A] writeN :: MonadIO m => Int -> Fold m a (MutArray a)
1098− [A] write :: MonadIO m => Fold m a (MutArray a)
1099− [A] uninit :: MonadIO m => MutArray a -> Int -> m (MutArray a)
1100− [A] toStreamK :: MonadIO m => MutArray a -> StreamK m a
1101− [A] toList :: MonadIO m => MutArray a -> m [a]
1102− [A] strip :: MonadIO m => (a -> Bool) -> MutArray a -> m (MutArray a)
1103− [A] snocWith :: MonadIO m => (Int -> Int) -> MutArray a -> a -> m (MutArray a)
1104− [A] snocUnsafe :: MonadIO m => MutArray a -> a -> m (MutArray a)
1105− [A] snoc :: MonadIO m => MutArray a -> a -> m (MutArray a)
1106− [A] realloc :: MonadIO m => Int -> MutArray a -> m (MutArray a)
1107− [A] reader :: MonadIO m => Unfold m (MutArray a) a
1108− [A] readRev :: MonadIO m => MutArray a -> Stream m a
1109− [A] read :: MonadIO m => MutArray a -> Stream m a
1110− [A] putSliceUnsafe :: MonadIO m => MutArray a -> Int -> MutArray a -> Int -> Int -> m ()
1111− [A] putIndices :: MonadIO m => MutArray a -> Fold m (Int, a) ()
1112− [A] putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
1113− [A] putIndex :: MonadIO m => Int -> MutArray a -> a -> m ()
1114− [A] producerWith :: Monad m => (forall b. IO b -> m b) -> Producer m (MutArray a) a
1115− [A] producer :: MonadIO m => Producer m (MutArray a) a
1116− [A] nil :: MonadIO m => m (MutArray a)
1117− [A] new :: MonadIO m => Int -> m (MutArray a)
1118− [A] modifyIndexUnsafe :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
1119− [A] modifyIndex :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
1120− [A] length :: MutArray a -> Int
1121− [A] getSliceUnsafe :: Int -> Int -> MutArray a -> MutArray a
1122− [A] getSlice :: Int -> Int -> MutArray a -> MutArray a
1123− [A] getIndexUnsafeWith :: MonadIO m => MutableArray# RealWorld a -> Int -> m a
1124− [A] getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
1125− [A] getIndex :: MonadIO m => Int -> MutArray a -> m (Maybe a)
1126− [A] fromStreamN :: MonadIO m => Int -> Stream m a -> m (MutArray a)
1127− [A] fromStream :: MonadIO m => Stream m a -> m (MutArray a)
1128− [A] fromPureStream :: MonadIO m => Stream Identity a -> m (MutArray a)
1129− [A] fromListN :: MonadIO m => Int -> [a] -> m (MutArray a)
1130− [A] fromList :: MonadIO m => [a] -> m (MutArray a)
1131− [A] eq :: (MonadIO m, Eq a) => MutArray a -> MutArray a -> m Bool
1132− [A] cmp :: (MonadIO m, Ord a) => MutArray a -> MutArray a -> m Ordering
1133− [A] clone :: MonadIO m => MutArray a -> m (MutArray a)
1134− [A] chunksOf :: forall m a. MonadIO m => Int -> Stream m a -> Stream m (MutArray a)
1135−[A] Streamly.Internal.Data.MutArray
1136− [A] MutByteArray
1137− [A] MutArray
1138− [A] [arrStart] :: MutArray a -> {-# UNPACK #-} !Int
1139− [A] [arrEnd] :: MutArray a -> {-# UNPACK #-} !Int
1140− [A] [arrContents] :: MutArray a -> {-# UNPACK #-} !MutByteArray
1141− [A] [arrBound] :: MutArray a -> {-# UNPACK #-} !Int
1142− [A] MutArray :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> MutArray a
1143− [A] IORef
1144− [A] ArrayUnsafe
1145− [A] ArrayUnsafe :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> ArrayUnsafe a
1146− [A] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1147− [A] writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1148− [A] writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
1149− [A] writeNWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
1150− [A] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1151− [A] writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1152− [A] writeIORef :: Unbox a => IORef a -> a -> IO ()
1153− [A] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
1154− [A] writeAppendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
1155− [A] writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
1156− [A] writeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
1157− [A] writeAppend :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
1158− [A] write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
1159− [A] unsafeSwapIndices :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> MutArray a -> m ()
1160− [A] unpin :: MutArray a -> IO (MutArray a)
1161− [A] toStreamKWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> StreamK m a
1162− [A] toStreamKRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> StreamK m a
1163− [A] toStreamKRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
1164− [A] toStreamK :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
1165− [A] toStreamDWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
1166− [A] toStreamDRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
1167− [A] toList :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m [a]
1168− [A] swapIndices :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> MutArray a -> m ()
1169− [A] strip :: forall a m. (Unbox a, MonadIO m) => (a -> Bool) -> MutArray a -> m (MutArray a)
1170− [A] splitOn :: (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> Stream m (MutArray a)
1171− [A] splitAt :: forall a. Unbox a => Int -> MutArray a -> (MutArray a, MutArray a)
1172− [A] spliceWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int -> Int) -> MutArray a -> MutArray a -> m (MutArray a)
1173− [A] spliceUnsafe :: MonadIO m => MutArray a -> MutArray a -> m (MutArray a)
1174− [A] spliceExp :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
1175− [A] spliceCopy :: forall m a. MonadIO m => MutArray a -> MutArray a -> m (MutArray a)
1176− [A] splice :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
1177− [A] snocWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> MutArray a -> a -> m (MutArray a)
1178− [A] snocUnsafe :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1179− [A] snocMay :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (Maybe (MutArray a))
1180− [A] snocLinear :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1181− [A] snoc :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1182− [A] shuffleBy :: (a -> a -> m Bool) -> MutArray a -> MutArray a -> m ()
1183− [A] roundUpToPower2 :: Int -> Int
1184− [A] rightSize :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m (MutArray a)
1185− [A] reverse :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m ()
1186− [A] resizeExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1187− [A] resize :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1188− [A] realloc :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1189− [A] readerRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Unfold m (MutArray a) a
1190− [A] readerRev :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
1191− [A] reader :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
1192− [A] readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
1193− [A] readIORef :: Unbox a => IORef a -> IO a
1194− [A] read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
1195− [A] putIndices :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Fold m (Int, a) ()
1196− [A] putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
1197− [A] putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
1198− [A] producerWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Producer m (MutArray a) a
1199− [A] producer :: forall m a. (MonadIO m, Unbox a) => Producer m (MutArray a) a
1200− [A] pollIntIORef :: (MonadIO m, Unbox a) => IORef a -> Stream m a
1201− [A] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1202− [A] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (MutArray a)
1203− [A] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1204− [A] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
1205− [A] pinnedNewBytes :: MonadIO m => Int -> m (MutArray a)
1206− [A] pinnedNewAligned :: (MonadIO m, Unbox a) => Int -> Int -> m (MutArray a)
1207− [A] pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
1208− [A] pinnedFromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1209− [A] pinnedFromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1210− [A] pinnedClone :: MonadIO m => MutArray a -> m (MutArray a)
1211− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1212− [A] pin :: MutArray a -> IO (MutArray a)
1213− [A] permute :: MutArray a -> m Bool
1214− [A] partitionBy :: forall m a. (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> m (MutArray a, MutArray a)
1215− [A] nil :: MutArray a
1216− [A] newIORef :: forall a. Unbox a => a -> IO (IORef a)
1217− [A] newArrayWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int -> m MutByteArray) -> Int -> Int -> m (MutArray a)
1218− [A] new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
1219− [A] modifyIndices :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (Int -> a -> a) -> Fold m Int ()
1220− [A] modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
1221− [A] modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
1222− [A] modifyIORef' :: Unbox a => IORef a -> (a -> a) -> IO ()
1223− [A] modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()
1224− [A] mergeBy :: Int -> (MutArray a -> MutArray a -> m ()) -> MutArray a -> m ()
1225− [A] memcpy :: Ptr Word8 -> Ptr Word8 -> Int -> IO ()
1226− [A] memcmp :: Ptr Word8 -> Ptr Word8 -> Int -> IO Bool
1227− [A] length :: forall a. Unbox a => MutArray a -> Int
1228− [A] isPinned :: MutArray a -> Bool
1229− [A] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
1230− [A] getSliceUnsafe :: forall a. Unbox a => Int -> Int -> MutArray a -> MutArray a
1231− [A] getSlice :: forall a. Unbox a => Int -> Int -> MutArray a -> MutArray a
1232− [A] getIndicesD :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
1233− [A] getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1234− [A] getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1235− [A] getIndexRev :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1236− [A] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
1237− [A] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
1238− [A] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
1239− [A] fromStreamD :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
1240− [A] fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
1241− [A] fromPureStream :: (MonadIO m, Unbox a) => Stream Identity a -> m (MutArray a)
1242− [A] fromListRevN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1243− [A] fromListRev :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1244− [A] fromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1245− [A] fromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1246− [A] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
1247− [A] foldr :: (MonadIO m, Unbox a) => (a -> b -> b) -> b -> MutArray a -> m b
1248− [A] foldl' :: (MonadIO m, Unbox a) => (b -> a -> b) -> b -> MutArray a -> m b
1249− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1250− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1251− [A] divideBy :: Int -> (MutArray a -> m (MutArray a, MutArray a)) -> MutArray a -> m ()
1252− [A] cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
1253− [A] clone :: MonadIO m => MutArray a -> m (MutArray a)
1254− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1255− [A] castUnsafe :: MutArray a -> MutArray b
1256− [A] cast :: forall a b. Unbox b => MutArray a -> Maybe (MutArray b)
1257− [A] c_memchr :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)
1258− [A] bytesFree :: MutArray a -> Int
1259− [A] byteLength :: MutArray a -> Int
1260− [A] byteCapacity :: MutArray a -> Int
1261− [A] bubble :: (MonadIO m, Unbox a) => (a -> a -> Ordering) -> MutArray a -> m ()
1262− [A] breakOn :: MonadIO m => Word8 -> MutArray Word8 -> m (MutArray Word8, Maybe (MutArray Word8))
1263− [A] blockSize :: Int
1264− [A] asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
1265− [A] asBytes :: MutArray a -> MutArray Word8
1266− [A] arrayChunkBytes :: Int
1267− [A] allocBytesToElemCount :: Unbox a => a -> Int -> Int
1268−[C] Streamly.Internal.Data.IsMap
1269− [A] mapTraverseWithKey :: (IsMap f, Applicative t) => (Key f -> a -> t b) -> f a -> t (f b)
1270−[R] Streamly.Internal.Data.IORef.Unboxed
1271−[R] Streamly.Internal.Data.Fold.Window
1272−[R] Streamly.Internal.Data.Fold.Type
1273−[R] Streamly.Internal.Data.Fold.Tee
1274−[R] Streamly.Internal.Data.Fold.Step
1275−[R] Streamly.Internal.Data.Fold.Container
1276−[D] Streamly.Internal.Data.Fold.Chunked
1277−[C] Streamly.Internal.Data.Fold
1278− [A] ManyState
1279− [C] Fold
1280− [C] Fold
1281− [O] Fold :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> Fold m a b
1282− [N] Fold :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> (s -> m b) -> Fold m a b
1283− [A] windowSumInt :: forall m a. (Monad m, Integral a) => Fold m (a, Maybe a) a
1284− [A] windowSum :: forall m a. (Monad m, Num a) => Fold m (a, Maybe a) a
1285− [A] windowRollingMapM :: Monad m => (Maybe a -> a -> m (Maybe b)) -> Fold m (a, Maybe a) (Maybe b)
1286− [A] windowRollingMap :: Monad m => (Maybe a -> a -> Maybe b) -> Fold m (a, Maybe a) (Maybe b)
1287− [A] windowRange :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe (a, a))
1288− [A] windowPowerSumFrac :: (Monad m, Floating a) => a -> Fold m (a, Maybe a) a
1289− [A] windowPowerSum :: (Monad m, Num a) => Int -> Fold m (a, Maybe a) a
1290− [A] windowMinimum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
1291− [A] windowMean :: forall m a. (Monad m, Fractional a) => Fold m (a, Maybe a) a
1292− [A] windowMaximum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
1293− [A] windowLmap :: (c -> a) -> Fold m (a, Maybe a) b -> Fold m (c, Maybe c) b
1294− [A] windowLength :: (Monad m, Num b) => Fold m (a, Maybe a) b
1295− [A] toSet :: (Monad m, Ord a) => Fold m a (Set a)
1296− [A] toMapIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
1297− [A] toMap :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
1298− [A] toIntSet :: Monad m => Fold m Int IntSet
1299− [A] toContainerIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (f b)
1300− [A] toContainer :: (Monad m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (f b)
1301− [A] nubInt :: Monad m => Fold m Int (Maybe Int)
1302− [A] nub :: (Monad m, Ord a) => Fold m a (Maybe a)
1303− [A] mapMStep :: Applicative m => (a -> m b) -> Step s a -> m (Step s b)
1304− [A] kvToMap :: (Monad m, Ord k) => Fold m a b -> Fold m (k, a) (Map k b)
1305− [A] frequency :: (Monad m, Ord a) => Fold m a (Map a Int)
1306− [A] demuxToMapIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
1307− [A] demuxToMap :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
1308− [A] demuxToContainerIO :: (MonadIO m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
1309− [A] demuxToContainer :: (Monad m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
1310− [A] demuxKvToMap :: (Monad m, Ord k) => (k -> m (Fold m a b)) -> Fold m (k, a) (Map k b)
1311− [A] demuxKvToContainer :: (Monad m, IsMap f, Traversable f) => (Key f -> m (Fold m a b)) -> Fold m (Key f, a) (f b)
1312− [A] demuxIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
1313− [A] demuxGenericIO :: (MonadIO m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (m (f b), Maybe (Key f, b))
1314− [A] demuxGeneric :: (Monad m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (m (f b), Maybe (Key f, b))
1315− [A] demux :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
1316− [A] cumulative :: Fold m (a, Maybe a) b -> Fold m a b
1317− [A] countDistinctInt :: Monad m => Fold m Int Int
1318− [A] countDistinct :: (Monad m, Ord a) => Fold m a Int
1319− [A] classifyIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
1320− [A] classifyGenericIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
1321− [A] classifyGeneric :: (Monad m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
1322− [A] classify :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
1323− [A] chainStepM :: Applicative m => (s1 -> m s2) -> (a -> m (Step s2 b)) -> Step s1 a -> m (Step s2 b)
1324−[C] Streamly.Internal.Data.Builder
1325− [C] Builder
1326− [C] Builder
1327− [O] Builder :: (s -> m (s, a)) -> Builder s m a
1328− [N] Builder :: (s -> m (a, s)) -> Builder s m a
1329−[A] Streamly.Internal.Data.Binary.Stream
1330− [A] class ToBytes a
1331− [A] word8 :: Applicative m => Word8 -> Stream m Word8
1332− [A] word64le :: Monad m => Word64 -> Stream m Word8
1333− [A] word64host :: Monad m => Word64 -> Stream m Word8
1334− [A] word64be :: Monad m => Word64 -> Stream m Word8
1335− [A] word32le :: Monad m => Word32 -> Stream m Word8
1336− [A] word32be :: Monad m => Word32 -> Stream m Word8
1337− [A] word16le :: Monad m => Word16 -> Stream m Word8
1338− [A] word16be :: Monad m => Word16 -> Stream m Word8
1339− [A] unit :: Applicative m => Stream m Word8
1340− [A] toBytes :: ToBytes a => a -> Stream m Word8
1341− [A] ordering :: Applicative m => Ordering -> Stream m Word8
1342− [A] int8 :: Applicative m => Int8 -> Stream m Word8
1343− [A] int64le :: Monad m => Int64 -> Stream m Word8
1344− [A] int64be :: Monad m => Int64 -> Stream m Word8
1345− [A] int32le :: Monad m => Int32 -> Stream m Word8
1346− [A] int32be :: Monad m => Int32 -> Stream m Word8
1347− [A] int16le :: Monad m => Int16 -> Stream m Word8
1348− [A] int16be :: Monad m => Int16 -> Stream m Word8
1349− [A] float32le :: Monad m => Float -> Stream m Word8
1350− [A] float32be :: Monad m => Float -> Stream m Word8
1351− [A] double64le :: Monad m => Double -> Stream m Word8
1352− [A] double64be :: Monad m => Double -> Stream m Word8
1353− [A] charUtf8 :: Monad m => Char -> Stream m Word8
1354− [A] charLatin1 :: Applicative m => Char -> Stream m Word8
1355− [A] bool :: Applicative m => Bool -> Stream m Word8
1356−[A] Streamly.Internal.Data.Binary.Parser
1357− [A] class FromBytes a
1358− [A] word8 :: Monad m => Parser Word8 m Word8
1359− [A] word64le :: Monad m => Parser Word8 m Word64
1360− [A] word64host :: MonadIO m => Parser Word8 m Word64
1361− [A] word64be :: Monad m => Parser Word8 m Word64
1362− [A] word32le :: Monad m => Parser Word8 m Word32
1363− [A] word32be :: Monad m => Parser Word8 m Word32
1364− [A] word16le :: Monad m => Parser Word8 m Word16
1365− [A] word16be :: Monad m => Parser Word8 m Word16
1366− [A] unit :: Monad m => Parser Word8 m ()
1367− [A] ordering :: Monad m => Parser Word8 m Ordering
1368− [A] int8 :: Monad m => Parser Word8 m Int8
1369− [A] int64le :: Monad m => Parser Word8 m Int64
1370− [A] int64be :: Monad m => Parser Word8 m Int64
1371− [A] int32le :: Monad m => Parser Word8 m Int32
1372− [A] int32be :: Monad m => Parser Word8 m Int32
1373− [A] int16le :: Monad m => Parser Word8 m Int16
1374− [A] int16be :: Monad m => Parser Word8 m Int16
1375− [A] fromBytes :: FromBytes a => Parser Word8 m a
1376− [A] float32le :: MonadIO m => Parser Word8 m Float
1377− [A] float32be :: MonadIO m => Parser Word8 m Float
1378− [A] eqWord8 :: Monad m => Word8 -> Parser Word8 m Word8
1379− [A] double64le :: MonadIO m => Parser Word8 m Double
1380− [A] double64be :: MonadIO m => Parser Word8 m Double
1381− [A] charLatin1 :: Monad m => Parser Word8 m Char
1382− [A] bool :: Monad m => Parser Word8 m Bool
1383−[R] Streamly.Internal.Data.Array.Type
1384−[A] Streamly.Internal.Data.Array.Stream
1385− [A] unlines :: forall m a. (MonadIO m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1386− [A] toArray :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
1387− [A] splitOnSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
1388− [A] splitOn :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
1389− [A] runArrayParserDBreak :: forall m a b. (MonadIO m, Unbox a) => Parser (Array a) m b -> Stream m (Array a) -> m (Either ParseError b, Stream m (Array a))
1390− [A] runArrayFoldMany :: (Monad m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> StreamK m (Either ParseError b)
1391− [A] runArrayFoldBreak :: (MonadIO m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1392− [A] runArrayFold :: (MonadIO m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> m (Either ParseError b)
1393− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1394− [A] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
1395− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1396− [A] parseBreak :: (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1397− [A] lpackArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
1398− [A] interposeSuffix :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1399− [A] interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1400− [A] intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a
1401− [A] foldBreakD :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
1402− [A] foldBreak :: (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
1403− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1404− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1405− [A] concatRev :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
1406− [A] concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
1407− [A] compact :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
1408− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1409− [A] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
1410−[R] Streamly.Internal.Data.Array.Mut.Type
1411−[R] Streamly.Internal.Data.Array.Mut.Stream
1412−[R] Streamly.Internal.Data.Array.Mut
1413−[R] Streamly.Internal.Data.Array.Generic.Mut.Type
1414−[C] Streamly.Internal.Data.Array.Generic
1415− [A] getIndex :: Int -> Array a -> Maybe a
1416− [A] fromPureStream :: Stream Identity a -> Array a
1417− [A] fromByteStr# :: Addr# -> Array Word8
1418− [A] chunksOf :: forall m a. MonadIO m => Int -> Stream m a -> Stream m (Array a)
1419−[C] Streamly.Internal.Data.Array
1420− [A] ArrayUnsafe
1421− [A] ArrayUnsafe :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> ArrayUnsafe a
1422− [C] Array
1423− [A] [arrStart] :: Array a -> {-# UNPACK #-} !Int
1424− [A] [arrEnd] :: Array a -> {-# UNPACK #-} !Int
1425− [A] [arrContents] :: Array a -> {-# UNPACK #-} !MutByteArray
1426− [A] Array :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> Array a
1427− [A] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1428− [A] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1429− [R] writeNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
1430− [A] unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b
1431− [A] unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a
1432− [D] unsafeIndex :: forall a. Unbox a => Int -> Array a -> a
1433− [A] unsafeFreezeWithShrink :: Unbox a => MutArray a -> Array a
1434− [A] unpin :: Array a -> IO (Array a)
1435− [A] toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a
1436− [A] toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a
1437− [A] toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
1438− [A] toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
1439− [A] splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)
1440− [A] splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
1441− [A] serialize :: Serialize a => a -> Array Word8
1442− [A] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1443− [A] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
1444− [A] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1445− [A] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
1446− [A] pinnedSerialize :: Serialize a => a -> Array Word8
1447− [A] pinnedFromListN :: Unbox a => Int -> [a] -> Array a
1448− [A] pinnedFromList :: Unbox a => [a] -> Array a
1449− [A] pinnedClone :: MonadIO m => Array a -> m (Array a)
1450− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1451− [A] pin :: Array a -> IO (Array a)
1452− [A] nil :: Array a
1453− [A] isPinned :: Array a -> Bool
1454− [A] getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a
1455− [A] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
1456− [A] fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)
1457− [A] fromPureStream :: Unbox a => Stream Identity a -> Array a
1458− [A] fromListRevN :: Unbox a => Int -> [a] -> Array a
1459− [A] fromListRev :: Unbox a => [a] -> Array a
1460− [A] fromByteStr# :: Addr# -> Array Word8
1461− [A] foldr :: Unbox a => (a -> b -> b) -> b -> Array a -> b
1462− [A] foldl' :: forall a b. Unbox a => (b -> a -> b) -> b -> Array a -> b
1463− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1464− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1465− [A] encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8
1466− [A] deserialize :: Serialize a => Array Word8 -> a
1467− [A] clone :: MonadIO m => Array a -> m (Array a)
1468− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1469− [A] byteLength :: Array a -> Int
1470− [A] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
1471− [A] breakOn :: MonadIO m => Word8 -> Array Word8 -> m (Array Word8, Maybe (Array Word8))
core/docs/ApiChangelogs/0.2.0-0.2.2.txtdeleted−306
@@ -1,306 +0,0 @@
1−---------------------------------
2−API Annotations
3−---------------------------------
4−
5−[A] : Added
6−[R] : Removed
7−[C] : Changed
8−[O] : Old definition
9−[N] : New definition
10−[D] : Deprecated
11−
12−---------------------------------
13−API diff
14−---------------------------------
15−
16−[C] Streamly.Data.Stream
17− [A] (FixityR,5)
18− [A] (FixityR,5)
19−[C] Streamly.Data.MutArray.Generic
20− [A] emptyOf :: MonadIO m => Int -> m (MutArray a)
21− [A] createOf :: MonadIO m => Int -> Fold m a (MutArray a)
22− [A] create :: MonadIO m => Fold m a (MutArray a)
23−[C] Streamly.Data.MutArray
24− [A] pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
25− [A] emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
26− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
27− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
28− [A] appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
29− [A] append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
30−[C] Streamly.Data.Array.Generic
31− [A] createOf :: MonadIO m => Int -> Fold m a (Array a)
32− [A] create :: MonadIO m => Fold m a (Array a)
33−[C] Streamly.Data.Array
34− [A] class Serialize a
35− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
36− [A] pinnedSerialize :: Serialize a => a -> Array Word8
37− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
38− [A] deserialize :: Serialize a => Array Word8 -> a
39− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
40− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
41− [A] addSizeTo :: Serialize a => Int -> a -> Int
42−
43−---------------------------------
44−Internal API diff
45−---------------------------------
46−
47−[C] Streamly.Internal.Unicode.Stream
48− [A] encodeUtf16le' :: Stream m Char -> Stream m Word16
49− [A] decodeUtf16le' :: Stream m Word16 -> Stream m Char
50−[A] Streamly.Internal.FileSystem.Path
51− [A] class IsPath a
52− [A] Rel
53− [A] File
54− [A] Dir
55− [A] Abs
56− [A] Streamly.Internal.FileSystem.Path.IsPath
57− [A] instance Streamly.Internal.FileSystem.Path.IsPath Streamly.Internal.FileSystem.Path.Path
58− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel Streamly.Internal.FileSystem.Path.Path)
59− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path))
60− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path))
61− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path)
62− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path)
63− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs Streamly.Internal.FileSystem.Path.Path)
64− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path))
65− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path))
66− [A] GHC.Show.Show
67− [A] instance GHC.Show.Show Streamly.Internal.FileSystem.Path.PathException
68− [A] GHC.Exception.Type.Exception
69− [A] instance GHC.Exception.Type.Exception Streamly.Internal.FileSystem.Path.PathException
70− [A] GHC.Classes.Eq
71− [A] instance GHC.Classes.Eq Streamly.Internal.FileSystem.Path.PathException
72− [A] Path
73− [A] Path :: Array Word8 -> Path
74− [A] toString :: Path -> [Char]
75− [A] toPath :: IsPath a => a -> Path
76− [A] toChunk :: Path -> Array Word8
77− [A] toChars :: Monad m => Path -> Stream m Char
78− [A] relfile :: QuasiQuoter
79− [A] reldir :: QuasiQuoter
80− [A] rel :: QuasiQuoter
81− [A] primarySeparator :: Char
82− [A] path :: QuasiQuoter
83− [A] mkRelFile :: String -> Q Exp
84− [A] mkRelDir :: String -> Q Exp
85− [A] mkRel :: String -> Q Exp
86− [A] mkPath :: String -> Q Exp
87− [A] mkFile :: String -> Q Exp
88− [A] mkDir :: String -> Q Exp
89− [A] mkAbsFile :: String -> Q Exp
90− [A] mkAbsDir :: String -> Q Exp
91− [A] mkAbs :: String -> Q Exp
92− [A] isSeparator :: Char -> Bool
93− [A] fromString :: MonadThrow m => [Char] -> m Path
94− [A] fromPathUnsafe :: IsPath a => Path -> a
95− [A] fromPath :: (IsPath a, MonadThrow m) => Path -> m a
96− [A] fromChunkUnsafe :: Array Word8 -> Path
97− [A] fromChunk :: MonadThrow m => Array Word8 -> m Path
98− [A] fromChars :: MonadThrow m => Stream Identity Char -> m Path
99− [A] file :: QuasiQuoter
100− [A] extendPath :: Path -> Path -> Path
101− [A] extendDir :: (IsPath (a (Dir Path)), IsPath b, IsPath (a b)) => a (Dir Path) -> Rel b -> a b
102− [A] dir :: QuasiQuoter
103− [A] adaptPath :: (MonadThrow m, IsPath a, IsPath b) => a -> m b
104− [A] absfile :: QuasiQuoter
105− [A] absdir :: QuasiQuoter
106− [A] abs :: QuasiQuoter
107−[C] Streamly.Internal.Data.Stream
108− [A] (FixityR,5)
109− [A] (FixityR,5)
110− [C] splitInnerBySuffix
111− [O] splitInnerBySuffix :: (Monad m, Eq (f a), Monoid (f a)) => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
112− [N] splitInnerBySuffix :: Monad m => (f a -> Bool) -> (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
113− [D] sliceOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
114− [A] indexOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
115−[C] Streamly.Internal.Data.MutByteArray
116− [A] unsafePinnedAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
117− [A] unsafeAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
118− [D] nil :: MutByteArray
119− [A] empty :: MutByteArray
120− [D] asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
121−[D] Streamly.Internal.Data.MutArray.Stream
122− [D] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
123− [D] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
124− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
125− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
126−[C] Streamly.Internal.Data.MutArray.Generic
127− [D] writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
128− [D] writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
129− [A] unsafeCreateOf :: MonadIO m => Int -> Fold m a (MutArray a)
130− [A] emptyOf :: MonadIO m => Int -> m (MutArray a)
131− [A] createWith :: MonadIO m => Int -> Fold m a (MutArray a)
132− [A] createOf :: MonadIO m => Int -> Fold m a (MutArray a)
133− [A] create :: MonadIO m => Fold m a (MutArray a)
134−[C] Streamly.Internal.Data.MutArray
135− [A] SpliceState
136− [A] SpliceYielding :: arr -> SpliceState s arr -> SpliceState s arr
137− [A] SpliceInitial :: s -> SpliceState s arr
138− [A] SpliceFinish :: SpliceState s arr
139− [A] SpliceBuffering :: s -> arr -> SpliceState s arr
140− [R] MutByteArray
141− [D] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
142− [D] writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
143− [D] writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
144− [D] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
145− [D] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
146− [D] writeAppendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
147− [D] writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
148− [A] unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
149− [A] unsafePinnedAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
150− [A] unsafeCreateOfWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
151− [A] unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
152− [A] unsafeAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
153− [A] unsafeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
154− [A] toStreamWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
155− [A] toStreamRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
156− [R] toStreamDWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
157− [R] toStreamDRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
158− [A] slicerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
159− [A] sliceIndexerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
160− [A] revCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
161− [D] resizeExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
162− [D] resize :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
163− [A] pokeSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
164− [A] pokeAppendMay :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> a -> m (Maybe (MutArray Word8))
165− [A] pokeAppend :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> a -> m (MutArray Word8)
166− [D] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
167− [D] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
168− [D] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
169− [D] pinnedNewBytes :: MonadIO m => Int -> m (MutArray a)
170− [A] pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
171− [A] pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
172− [A] pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
173− [A] pinnedCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
174− [A] peekUnconsUnsafe :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> m (a, MutArray Word8)
175− [A] peekUncons :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> m (Maybe a, MutArray Word8)
176− [A] peekSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
177− [A] pPinnedCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Parser (MutArray a) m (MutArray a)
178− [A] pCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Parser (MutArray a) m (MutArray a)
179− [D] nil :: MutArray a
180− [A] lPinnedCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
181− [A] lCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
182− [A] indexReaderWith :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
183− [A] indexReader :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
184− [A] growExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
185− [A] grow :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
186− [D] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
187− [R] getIndicesD :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
188− [D] getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
189− [D] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
190− [A] fromStreamN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
191− [D] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
192− [D] fromStreamD :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
193− [A] fromPureStreamN :: (MonadIO m, Unbox a) => Int -> Stream Identity a -> m (MutArray a)
194− [A] fromPtrN :: MonadIO m => Int -> Ptr Word8 -> m (MutArray Word8)
195− [A] fromChunksRealloced :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> m (MutArray a)
196− [A] fromChunksK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
197− [A] fromByteStr# :: MonadIO m => Addr# -> m (MutArray Word8)
198− [D] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
199− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
200− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
201− [A] fPinnedCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) (MutArray a)
202− [A] fCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) (MutArray a)
203− [A] emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
204− [A] empty :: MutArray a
205− [A] createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
206− [A] createOfWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
207− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
208− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
209− [A] concatWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m (MutArray a) -> Stream m a
210− [A] concatRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m (MutArray a) -> Stream m a
211− [A] concatRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
212− [A] concat :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
213− [A] compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (MutArray Word8) -> Stream m (MutArray Word8)
214− [A] compactOnByte :: MonadIO m => Word8 -> Stream m (MutArray Word8) -> Stream m (MutArray Word8)
215− [A] compactLeAs :: forall m a. (MonadIO m, Unbox a) => PinnedState -> Int -> Stream m (MutArray a) -> Stream m (MutArray a)
216− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
217− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
218− [A] compactEQ :: Int -> Stream m (MutArray a) -> Stream m (MutArray a)
219− [D] cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
220− [A] byteEq :: MonadIO m => MutArray a -> MutArray a -> m Bool
221− [A] byteCmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
222− [A] buildChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
223− [D] asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
224− [A] appendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
225− [A] appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
226− [A] append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
227−[D] Streamly.Internal.Data.Array.Stream
228− [C] interposeSuffix
229− [O] interposeSuffix :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
230− [N] interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
231− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
232− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
233− [C] concatRev
234− [O] concatRev :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
235− [N] concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
236− [D] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
237−[C] Streamly.Internal.Data.Array.Generic
238− [A] createOf :: MonadIO m => Int -> Fold m a (Array a)
239− [A] create :: MonadIO m => Fold m a (Array a)
240−[C] Streamly.Internal.Data.Array
241− [R] ArrayUnsafe
242− [D] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
243− [D] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
244− [A] unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
245− [A] unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
246− [A] unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
247− [D] toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
248− [D] toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
249− [C] splice
250− [O] splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
251− [N] splice :: MonadIO m => Array a -> Array a -> m (Array a)
252− [A] slicerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Array a)
253− [A] sliceIndexerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Int, Int)
254− [D] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
255− [D] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
256− [D] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
257− [D] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
258− [A] pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
259− [A] pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
260− [A] pinnedCompactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
261− [A] parseBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
262− [D] nil :: Array a
263− [A] lPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
264− [A] lCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
265− [A] interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
266− [A] interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
267− [A] intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a
268− [A] indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a
269− [A] indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
270− [A] indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int
271− [D] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Array a)
272− [R] getIndicesFromThenTo :: Unfold m (Int, Int, Int, Array a) a
273− [D] getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
274− [D] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Int, Int)
275− [D] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
276− [D] fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)
277− [A] fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a
278− [A] fromPtrN :: Int -> Ptr Word8 -> Array Word8
279− [A] fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)
280− [A] fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
281− [A] fromByteStr :: Ptr Word8 -> Array Word8
282− [A] foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b
283− [A] foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
284− [A] foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
285− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
286− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
287− [C] findIndicesOf
288− [O] findIndicesOf :: (a -> Bool) -> Unfold Identity (Array a) Int
289− [N] findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int
290− [A] fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
291− [A] fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
292− [A] empty :: Array a
293− [A] createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
294− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
295− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
296− [A] concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
297− [A] concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
298− [A] compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
299− [A] compactOnByte :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
300− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
301− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
302− [A] byteEq :: Array a -> Array a -> Bool
303− [A] byteCmp :: Array a -> Array a -> Ordering
304− [A] buildChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
305− [D] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
306− [D] asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
core/docs/Changelog.md+3−64
@@ -1,73 +1,12 @@
11# Changelog
22
3−## 0.2.2 (Jan 2024)
3+## Unreleased
44
5−* Add fixities `infixr 5` for `cons` and `consM` functions.
6−* Fix a bug in Array `Eq` instance when the type is a sum type with
7− differently sized constructors.
8−* lpackArraysChunksOf, compact, writeChunksWith, putChunksWith now take the
9− buffer size in number of array elements instead of bytes.
10−
11−## 0.2.1 (Dec 2023)
12−
13−* Make the serialization of the unit constructor deterministic.
14−* Expose `pinnedSerialize` & `deserialize` via `Streamly.Data.Array`.
15−
16−## 0.2.0 (Nov 2023)
17−
18−See [0.1.0-0.2.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0-0.2.0.txt)
19−for a full list of API changes in this release. Only a few significant
20−changes are mentioned here.
21−
22−### Breaking Changes
23−
24−* `ParserK` in `Streamly.Data.ParserK` is not implicitly specialized
25− to arrays anymore. To adapt to the new code, change `ParserK a m
26− b` to `ParserK (Array a) m b` where the `Array` type comes from
27− `Streamly.Data.Array`. This change also affected the signatures of
28− `parseChunks` and `parseBreakChunks`.
29−* Changed the signature of 'Streamly.Data.Stream.handle' to make the
30− exception handler monadic.
31−* Behavior change: Exceptions are now rethrown promptly in `bracketIO`.
32−
33−### Enhancements
34−
35−* __Serialization__: Added a `Streamly.Data.MutByteArray` module with a
36− `Serialize` type class for fast binary serialization. The Data.Array
37− module supplies the `serialize` and `deserialize` operations for arrays.
38−* __Unpinned Arrays__: Unboxed arrays are now created unpinned by default,
39− they were created pinned earlier. During IO operations, unpinned arrays
40− are automatically copied to pinned memory. When arrays are directly
41− passed to IO operations programmers can choose to create them pinned to
42− avoid a copy. To create pinned arrays, use the internal APIs with the
43− `pinned*` prefix.
44−* StreamK now supports native exception handling routines (handle, bracketIO).
45− Earlier we had to convert it to the `Stream` type for exception handling.
46−
47−### Deprecations
48−
49−See [0.1.0-0.2.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0-0.2.0.txt)
50−for a full list of deprecations.
51−
52−### Internal API Changes
53−
54−* Fold constructor has changed, added a `final` field to support
55− finalization and cleanup of a chain of folds. The `extract` field is
56− now used only for mapping the fold internal state to fold result for
57− scanning purposes. If your fold does not require cleanup you can just use
58− your existing `extract` function as `final` as well to adapt to this change.
59−* Many low level internal modules have been removed, they are entirely
60− exported from higher level internal modules. If you were importing any
61− of the missing low level modules then import the higher level modules instead.
62−* Internal module changes:
63− * Streamly.Internal.Serialize.FromBytes -> Streamly.Internal.Data.Binary.Parser
64− * Streamly.Internal.Serialize.ToBytes -> Streamly.Internal.Data.Binary.Stream
65− * Streamly.Internal.Data.Unbox is now exported via Streamly.Internal.Data.Serialize
66− * Streamly.Internal.Data.IORef.Unboxed is now exported via Streamly.Internal.Data.Serialize
5+* Arrays can now be pinned or unpinned. They were always pinned earlier.
676
687## 0.1.0 (March 2023)
698
70−Also see [streamly-core-0.1.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0.txt) or
9+Also see [streamly-core-0.1.0 API Changelog](/core/docs/ApiChangelogs/0.1.0.txt) or
7110https://hackage.haskell.org/package/streamly-core-0.1.0/docs/docs/ApiChangelogs/0.1.0.txt
7211
7312`streamly` package is split into two packages, (1) `streamly-core` that
core/src/DocTestDataArray.hs−8
@@ -1,7 +1,6 @@
11{- $setup
22>>> :m
33>>> :set -XFlexibleContexts
4−>>> :set -XMagicHash
54>>> import Data.Function ((&))
65>>> import Data.Functor.Identity (Identity(..))
76>>> import System.IO.Unsafe (unsafePerformIO)
@@ -11,12 +10,5 @@
1110
1211>>> import qualified Streamly.Data.Array as Array
1312>>> import qualified Streamly.Data.Fold as Fold
14−>>> import qualified Streamly.Data.ParserK as ParserK
1513>>> import qualified Streamly.Data.Stream as Stream
16−>>> import qualified Streamly.Data.StreamK as StreamK
17−
18−For APIs that have not been released yet.
19−
20−>>> import qualified Streamly.Internal.Data.Array as Array
21−>>> import qualified Streamly.Internal.Data.Stream as Stream
2214-}
core/src/DocTestDataFold.hs+1
@@ -24,4 +24,5 @@
2424For APIs that have not been released yet.
2525
2626>>> import qualified Streamly.Internal.Data.Fold as Fold
27+>>> import qualified Streamly.Internal.Data.Fold.Window as FoldW
2728-}
core/src/DocTestDataMutArray.hs+1−1
@@ -6,5 +6,5 @@
66
77For APIs that have not been released yet.
88
9−>>> import Streamly.Internal.Data.MutArray as MutArray
9+>>> import Streamly.Internal.Data.Array.Mut as MutArray
1010-}
core/src/DocTestDataMutArrayGeneric.hs+1−1
@@ -6,5 +6,5 @@
66
77For APIs that have not been released yet.
88
9−>>> import Streamly.Internal.Data.MutArray.Generic as MutArray
9+>>> import Streamly.Internal.Data.Array.Generic.Mut.Type as MutArray
1010-}
core/src/DocTestDataParserK.hsdeleted−15
@@ -1,15 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import Control.Applicative ((<|>))
4−>>> import Data.Char (isDigit, isAlpha)
5−
6−>>> import Streamly.Data.Parser (Parser)
7−>>> import Streamly.Data.ParserK (ParserK)
8−
9−>>> import qualified Streamly.Data.Parser as Parser
10−>>> import qualified Streamly.Data.ParserK as ParserK
11−
12−For APIs that have not been released yet.
13−
14−>>> import qualified Streamly.Internal.Data.ParserK as ParserK
15−-}
core/src/DocTestDataStream.hs+1−2
@@ -6,7 +6,6 @@
66>>> import Control.Monad.IO.Class (MonadIO (liftIO))
77>>> import Control.Monad.Trans.Class (lift)
88>>> import Control.Monad.Trans.Identity (runIdentityT)
9−>>> import Data.Char (isSpace)
109>>> import Data.Either (fromLeft, fromRight, isLeft, isRight, either)
1110>>> import Data.Maybe (fromJust, isJust)
1211>>> import Data.Function (fix, (&))
@@ -31,9 +30,9 @@
3130For APIs that have not been released yet.
3231
3332>>> import qualified Streamly.Internal.Data.Fold as Fold
33+>>> import qualified Streamly.Internal.Data.Fold.Window as Window
3434>>> import qualified Streamly.Internal.Data.Parser as Parser
3535>>> import qualified Streamly.Internal.Data.Stream as Stream
36−>>> import qualified Streamly.Internal.Data.StreamK as StreamK
3736>>> import qualified Streamly.Internal.Data.Unfold as Unfold
3837>>> import qualified Streamly.Internal.FileSystem.Dir as Dir
3938-}
core/src/DocTestFileSystemHandle.hsdeleted−15
@@ -1,15 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import qualified Streamly.Data.Array as Array
4−>>> import qualified Streamly.FileSystem.Handle as Handle hiding (readChunks)
5−>>> import qualified Streamly.Data.Fold as Fold
6−>>> import qualified Streamly.Data.Stream as Stream
7−>>> import qualified Streamly.Data.Unfold as Unfold
8−
9−For APIs that have not been released yet.
10−
11−>>> import qualified Streamly.Internal.Data.Array as Array (unsafeCreateOf)
12−>>> import qualified Streamly.Internal.Data.Unfold as Unfold (first)
13−>>> import qualified Streamly.Internal.FileSystem.Handle as Handle
14−>>> import qualified Streamly.Internal.System.IO as IO (defaultChunkSize)
15−-}
core/src/DocTestUnicodeParser.hsdeleted−9
@@ -1,9 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import qualified Streamly.Data.Stream as Stream
4−>>> import qualified Streamly.Unicode.Parser as Unicode
5−
6−For APIs that have not been released yet.
7−
8−>>> import qualified Streamly.Internal.Unicode.Parser as Unicode (number, mkDouble)
9−-}
core/src/DocTestUnicodeStream.hsdeleted−12
@@ -1,12 +0,0 @@
1−{- $setup
2−>>> :m
3−
4−>>> import qualified Streamly.Data.Fold as Fold
5−>>> import qualified Streamly.Data.Stream as Stream
6−>>> import qualified Streamly.Unicode.Stream as Unicode
7−
8−For APIs that have not been released yet.
9−
10−>>> :set -XMagicHash
11−>>> import qualified Streamly.Internal.Unicode.Stream as Unicode
12−-}
core/src/DocTestUnicodeString.hsdeleted−5
@@ -1,5 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> :set -XQuasiQuotes
4−>>> import Streamly.Internal.Unicode.String
5−-}
core/src/Streamly/Console/Stdio.hs+27−1
@@ -24,7 +24,33 @@ module Streamly.Console.Stdio
2424 -- * Write (stderr)
2525 , writeErr
2626 , writeErrChunks
27+
28+ -- * Deprecated
29+ , read
30+ , readChunks
2731 )
2832where
2933
30−import Streamly.Internal.Console.Stdio
34+import Control.Monad.IO.Class (MonadIO(..))
35+import Data.Word (Word8)
36+import Streamly.Internal.Data.Array.Type (Array)
37+import Streamly.Internal.Data.Unfold (Unfold)
38+
39+import Streamly.Internal.Console.Stdio hiding (read, readChunks)
40+import Prelude hiding (read)
41+
42+-- Same as 'reader'
43+--
44+-- @since 0.8.0
45+{-# DEPRECATED read "Please use 'reader' instead" #-}
46+{-# INLINE read #-}
47+read :: MonadIO m => Unfold m () Word8
48+read = reader
49+
50+-- Same as 'chunkReader'
51+--
52+-- @since 0.8.0
53+{-# DEPRECATED readChunks "Please use 'chunkReader' instead" #-}
54+{-# INLINE readChunks #-}
55+readChunks :: MonadIO m => Unfold m () (Array Word8)
56+readChunks = chunkReader
core/src/Streamly/Data/Array.hs+61−58
@@ -29,80 +29,64 @@ module Streamly.Data.Array
2929 -- $overview
3030
3131 -- * The Array Type
32− Array
33−
34− -- * Pinning & Unpinning
35− -- | Arrays are created unpinned by default unless pinned versions of
36− -- creation APIs are used. Look for APIs with @pinned@ prefix in
37− -- "Streamly.Internal.Data.Array" for some unreleased pinned creation APIs.
38− -- If an array is to be sent to the OS without any further modification
39− -- then it should be created pinned in the first place instead of pinning
40− -- it later. Pinning an unpinned array has a copy overhead. OS interfacing
41− -- APIs create a pinned array directly or convert an unpinned array to
42− -- pinned array before sending it to the OS.
43− , pin
44− , unpin
45− , isPinned
32+ A.Array
4633
4734 -- * Construction
4835 -- | When performance matters, the fastest way to generate an array is
49− -- 'createOf'. 'IsList' and 'IsString' instances can be
36+ -- 'writeN'. 'IsList' and 'IsString' instances can be
5037 -- used to conveniently construct arrays from literal values.
5138 -- 'OverloadedLists' extension or 'fromList' can be used to construct an
5239 -- array from a list literal. Similarly, 'OverloadedStrings' extension or
5340 -- 'fromList' can be used to construct an array from a string literal.
5441
55− -- ** From Stream
56− , createOf
57− , create
58− , writeLastN -- drop old (ring buffer)
42+ -- Pure List APIs
43+ , A.fromListN
44+ , A.fromList
5945
60− -- ** From List
61− , fromListN
62− , fromList
46+ -- Monadic APIs
47+ , A.writeN -- drop new
48+ , A.write -- full buffer
49+ , writeLastN -- drop old (ring buffer)
6350
64− -- * To List
51+ -- * Conversion
6552 -- 'GHC.Exts.toList' from "GHC.Exts" can be used to convert an array to a
6653 -- list.
67− , toList
68−
69− -- * To Stream
70− , read
71− , readRev
54+ , A.toList
7255
7356 -- * Unfolds
74− , reader
75− , readerRev
57+ , A.reader
58+ , A.readerRev
7659
7760 -- * Casting
7861 , cast
7962 , asBytes
8063
8164 -- * Random Access
82− , length
65+ , A.length
8366 -- , (!!)
84− , getIndex
67+ , A.getIndex
8568
86− -- * Serialization
87− , pinnedSerialize
88− , deserialize
89−
90− -- * Re-exports
69+ -- * Unbox Type Class
9170 , Unbox (..)
92− , Serialize(..)
71+
72+ -- * Pinning & Unpinning
73+ , pin
74+ , unpin
9375
9476 -- * Deprecated
95− , writeN -- drop new
96− , write -- full buffer
77+ , read
78+ , readRev
9779 )
9880where
9981
10082#include "inline.hs"
10183
102−import Streamly.Internal.Data.Array
103−import Streamly.Internal.Data.MutByteArray (Unbox(..), Serialize(..))
84+import Streamly.Internal.Data.Unfold (Unfold)
85+import Streamly.Internal.Data.Array as A hiding (read, readRev)
86+import Streamly.Internal.Data.Array.Type (pin, unpin)
10487
105−import Prelude hiding (read, length)
88+import Streamly.Internal.Data.Unbox (Unbox (..))
89+import Prelude hiding (read)
10690
10791#include "DocTestDataArray.hs"
10892
@@ -118,7 +102,7 @@ import Prelude hiding (read, length)
118102--
119103-- Convert array to stream, and fold the stream:
120104--
121−-- >>> fold f arr = Array.read arr & Stream.fold f
105+-- >>> fold f arr = Stream.unfold Array.reader arr & Stream.fold f
122106-- >>> fold Fold.sum (Array.fromList [1,2,3::Int])
123107-- 6
124108--
@@ -126,17 +110,15 @@ import Prelude hiding (read, length)
126110--
127111-- Convert array to stream, transform, and fold back to array:
128112--
129−-- >>> amap f arr = Array.read arr & fmap f & Stream.fold (Array.createOf (Array.length arr))
113+-- >>> amap f arr = Stream.unfold Array.reader arr & fmap f & Stream.fold Array.write
130114-- >>> amap (+1) (Array.fromList [1,2,3::Int])
131115-- fromList [2,3,4]
132116--
133117-- == Pinned and Unpinned Arrays
134118--
135−-- The array type can use both pinned and unpinned memory under the hood. The
136−-- default array creation operations create unpinned arrays. IO operations
137−-- automatically copy an array to pinned memory if the array passed to it is
138−-- unpinned. Programmers can use appropriate pinned array generation APIs to
139−-- reduce the copying if it happens.
119+-- The array type can use both pinned and unpinned memory under the hood. The
120+-- arrays are unpinned by default and are only pinned while doing any I/O
121+-- operations.
140122--
141123-- Unpinned arrays have the advantage of allowing automatic defragmentation of
142124-- the memory by GC. Whereas pinned arrays have the advantage of not requiring
@@ -147,17 +129,24 @@ import Prelude hiding (read, length)
147129--
148130-- == Creating Arrays from Non-IO Streams
149131--
150−-- Array creation folds require 'MonadIO' otherwise the compiler may
151−-- incorrectly share the array memory thinking it is pure.
132+-- Array creation folds require 'MonadIO' because they need to sequence effects
133+-- in IO streams. To operate on streams in pure Monads like 'Identity' you can
134+-- morph it to IO monad as follows:
135+--
136+-- The 'MonadIO' based folds can be morphed to 'Identity' stream folds:
152137--
153−-- See the 'fromPureStream' unreleased API to generate an array from an
154−-- Identity stream safely without using MonadIO constraint.
138+-- >>> purely = Fold.morphInner (Identity . unsafePerformIO)
139+-- >>> Stream.fold (purely Array.write) $ Stream.fromList [1,2,3::Int]
140+-- Identity fromList [1,2,3]
155141--
156−-- Note that 'Identity' streams can be generalized to IO streams:
142+-- Since it is a pure stream we can use 'unsafePerformIO' to extract the result
143+-- of fold from IO.
157144--
158−-- >>> stream = Stream.fromList [1,2,3] :: Stream Identity Int
159−-- >>> pure2IO s = Stream.morphInner (return . runIdentity) s :: Stream IO Int
160−-- >>> Stream.fold Array.create (pure2IO stream)
145+-- Alternatively, 'Identity' streams can be generalized to IO streams:
146+--
147+-- >>> pure = Stream.fromList [1,2,3] :: Stream Identity Int
148+-- >>> generally = Stream.morphInner (return . runIdentity)
149+-- >>> Stream.fold Array.write (generally pure :: Stream IO Int)
161150-- fromList [1,2,3]
162151--
163152-- == Programming Tips
@@ -165,3 +154,17 @@ import Prelude hiding (read, length)
165154-- This module is designed to be imported qualified:
166155--
167156-- >>> import qualified Streamly.Data.Array as Array
157+
158+-- | Same as 'reader'
159+--
160+{-# DEPRECATED read "Please use 'reader' instead" #-}
161+{-# INLINE_NORMAL read #-}
162+read :: (Monad m, Unbox a) => Unfold m (Array a) a
163+read = reader
164+
165+-- | Same as 'readerRev'
166+--
167+{-# DEPRECATED readRev "Please use 'readerRev' instead" #-}
168+{-# INLINE_NORMAL readRev #-}
169+readRev :: (Monad m, Unbox a) => Unfold m (Array a) a
170+readRev = readerRev
core/src/Streamly/Data/Array/Generic.hs+13−16
@@ -15,37 +15,34 @@ module Streamly.Data.Array.Generic
1515 ( Array
1616
1717 -- * Construction
18− , fromListN
19− , fromList
18+ , A.fromListN
19+ , A.fromList
2020
2121 -- MonadicAPIs
22− , createOf
23− , create
22+ , A.writeN
23+ , A.write
2424
2525 -- * Conversion
26− , toList
26+ , A.toList
2727
2828 -- * Streams
29− , read
30− , readRev
29+ , A.read
30+ , A.readRev
3131
3232 -- * Unfolds
33− , reader
33+ , A.reader
3434 -- , A.readerRev
3535
3636 -- * Random Access
37− , length
38− , getIndex
37+ , A.length
38+ , A.getIndex
3939
4040 -- -- * Folding Arrays
4141 -- , A.streamFold
4242 -- , A.fold
43−
44− -- * Deprecated
45− , writeN
46− , write
4743 )
4844where
4945
50−import Streamly.Internal.Data.Array.Generic
51−import Prelude hiding (length, read)
46+import Streamly.Internal.Data.Array.Generic (Array)
47+
48+import qualified Streamly.Internal.Data.Array.Generic as A
core/src/Streamly/Data/Fold.hs+119−150
@@ -10,156 +10,6 @@
1010-- Fast, composable stream consumers with ability to terminate, supporting
1111-- stream fusion.
1212--
13−-- == Using Folds
14−--
15−-- This module provides elementary folds and fold combinators that can be used
16−-- to consume a stream of data and reduce it to a final value, or transform it
17−-- in a stateful manner using scans. A data stream can be reduced into a stream
18−-- of folded data elements by folding segments of the stream. Fold combinators
19−-- can be used to compose multiple folds in parallel or to create a pipeline of
20−-- folds such that the next fold consumes the result of the previous fold. To
21−-- run these folds on a stream see 'Streamly.Data.Stream.fold',
22−-- 'Streamly.Data.Stream.scan', 'Streamly.Data.Stream.postscan',
23−-- 'Streamly.Data.Stream.scanMaybe', 'Streamly.Data.Stream.foldMany' and other
24−-- operations accepting 'Fold' type as argument "Streamly.Data.Stream".
25−--
26−-- == Reducing a Stream
27−--
28−-- A 'Fold' is a consumer of a stream of values. A fold driver (such as
29−-- 'Streamly.Data.Stream.fold') initializes the fold @accumulator@, runs the
30−-- fold @step@ function in a loop, processing the input stream one element at a
31−-- time and accumulating the result. The loop continues until the fold
32−-- terminates, at which point the accumulated result is returned.
33−--
34−-- For example, a 'sum' Fold represents a stream consumer that adds the values
35−-- in the input stream:
36−--
37−-- >>> Stream.fold Fold.sum $ Stream.fromList [1..100]
38−-- 5050
39−--
40−-- Conceptually, a 'Fold' is a data type that mimics a strict left fold
41−-- ('Data.List.foldl'). The above example is similar to a left fold using
42−-- @(+)@ as the step and @0@ as the initial value of the accumulator:
43−--
44−-- >>> Data.List.foldl' (+) 0 [1..100]
45−-- 5050
46−--
47−-- 'Fold's have an early termination capability e.g. the 'one' fold terminates
48−-- after consuming one element:
49−--
50−-- >>> Stream.fold Fold.one $ Stream.fromList [1..]
51−-- Just 1
52−--
53−-- The above example is similar to the following right fold:
54−--
55−-- >>> Prelude.foldr (\x _ -> Just x) Nothing [1..]
56−-- Just 1
57−--
58−-- 'Fold's can be combined together using combinators. For example, to create a
59−-- fold that sums first two elements in a stream:
60−--
61−-- >>> sumTwo = Fold.take 2 Fold.sum
62−-- >>> Stream.fold sumTwo $ Stream.fromList [1..100]
63−-- 3
64−--
65−-- == Parallel Composition
66−--
67−-- Folds can be combined to run in parallel on the same input. For example, to
68−-- compute the average of numbers in a stream without going through the stream
69−-- twice:
70−--
71−-- >>> avg = Fold.teeWith (/) Fold.sum (fmap fromIntegral Fold.length)
72−-- >>> Stream.fold avg $ Stream.fromList [1.0..100.0]
73−-- 50.5
74−--
75−-- Folds can be combined so as to partition the input stream over multiple
76−-- folds. For example, to count even and odd numbers in a stream:
77−--
78−-- >>> split n = if even n then Left n else Right n
79−-- >>> stream = fmap split $ Stream.fromList [1..100]
80−-- >>> countEven = fmap (("Even " ++) . show) Fold.length
81−-- >>> countOdd = fmap (("Odd " ++) . show) Fold.length
82−-- >>> f = Fold.partition countEven countOdd
83−-- >>> Stream.fold f stream
84−-- ("Even 50","Odd 50")
85−--
86−-- == Sequential Composition
87−--
88−-- Terminating folds can be combined to parse the stream serially such that the
89−-- first fold consumes the input until it terminates and the second fold
90−-- consumes the rest of the input until it terminates:
91−--
92−-- >>> f = Fold.splitWith (,) (Fold.take 8 Fold.toList) (Fold.takeEndBy (== '\n') Fold.toList)
93−-- >>> Stream.fold f $ Stream.fromList "header: hello\n"
94−-- ("header: ","hello\n")
95−--
96−-- == Splitting a Stream
97−--
98−-- A 'Fold' can be applied repeatedly on a stream to transform it to a stream
99−-- of fold results. To split a stream on newlines:
100−--
101−-- >>> f = Fold.takeEndBy (== '\n') Fold.toList
102−-- >>> Stream.fold Fold.toList $ Stream.foldMany f $ Stream.fromList "Hello there!\nHow are you\n"
103−-- ["Hello there!\n","How are you\n"]
104−--
105−-- Similarly, we can split the input of a fold too:
106−--
107−-- >>> Stream.fold (Fold.many f Fold.toList) $ Stream.fromList "Hello there!\nHow are you\n"
108−-- ["Hello there!\n","How are you\n"]
109−--
110−-- == Folds vs. Streams
111−--
112−-- We can often use streams or folds to achieve the same goal. However, streams
113−-- are more efficient in composition of producers (e.g.
114−-- 'Data.Stream.append' or 'Data.Stream.mergeBy') whereas folds are
115−-- more efficient in composition of consumers (e.g. 'splitWith', 'partition'
116−-- or 'teeWith').
117−--
118−-- Streams are producers, transformations on streams happen on the output side:
119−--
120−-- >>> :{
121−-- f stream =
122−-- Stream.filter odd stream
123−-- & fmap (+1)
124−-- & Stream.fold Fold.sum
125−-- :}
126−--
127−-- >>> f $ Stream.fromList [1..100 :: Int]
128−-- 2550
129−--
130−-- Folds are stream consumers with an input stream and an output value, stream
131−-- transformations on folds happen on the input side:
132−--
133−-- >>> :{
134−-- f =
135−-- Fold.filter odd
136−-- $ Fold.lmap (+1)
137−-- $ Fold.sum
138−-- :}
139−--
140−-- >>> Stream.fold f $ Stream.fromList [1..100 :: Int]
141−-- 2550
142−--
143−-- Notice the similiarity in the definition of @f@ in both cases, the only
144−-- difference is the composition by @&@ vs @$@ and the use @lmap@ vs @map@, the
145−-- difference is due to output vs input side transformations.
146−--
147−-- == Fusion Limitations
148−--
149−-- Folds support stream fusion for generating loops comparable to the speed of
150−-- C. However, it has some limitations. For fusion to work, the folds must be
151−-- inlined, folds must be statically known and not generated dynamically, folds
152−-- should not be passed recursively.
153−--
154−-- Another limitation is due to the quadratic complexity causing slowdown when
155−-- too many nested compositions are used. Especially, the performance of the
156−-- Applicative instance and splitting operations (e.g. 'splitWith') degrades
157−-- quadratically (O(n^2)) when combined @n@ times, roughly 8 or less sequenced
158−-- operations are fine. For these cases folds can be converted to parsers and
159−-- then used as ParserK.
160−--
161−-- == Experimental APIs
162−--
16313-- Please refer to "Streamly.Internal.Data.Fold" for more functions that have
16414-- not yet been released.
16515
@@ -171,6 +21,9 @@ module Streamly.Data.Fold
17121 --
17222 -- $setup
17323
24+ -- * Overview
25+ -- $overview
26+
17427 -- * Running A Fold
17528 drive
17629 -- XXX Should we have a stream returning function in fold module?
@@ -402,9 +255,125 @@ import Prelude
402255 span, splitAt, break, mapM, maybe)
403256
404257import Streamly.Internal.Data.Fold
258+import Streamly.Internal.Data.Fold.Container
405259
406260#include "DocTestDataFold.hs"
407261
262+-- $overview
263+--
264+-- A 'Fold' is a consumer of a stream of values. A fold driver (such as
265+-- 'Streamly.Data.Stream.fold') initializes the fold @accumulator@, runs the
266+-- fold @step@ function in a loop, processing the input stream one element at a
267+-- time and accumulating the result. The loop continues until the fold
268+-- terminates, at which point the accumulated result is returned.
269+--
270+-- For example, a 'sum' Fold represents a stream consumer that adds the values
271+-- in the input stream:
272+--
273+-- >>> Stream.fold Fold.sum $ Stream.fromList [1..100]
274+-- 5050
275+--
276+-- Conceptually, a 'Fold' is a data type that mimics a strict left fold
277+-- ('Data.List.foldl'). The above example is similar to a left fold using
278+-- @(+)@ as the step and @0@ as the initial value of the accumulator:
279+--
280+-- >>> Data.List.foldl' (+) 0 [1..100]
281+-- 5050
282+--
283+-- 'Fold's have an early termination capability e.g. the 'one' fold terminates
284+-- after consuming one element:
285+--
286+-- >>> Stream.fold Fold.one $ Stream.fromList [1..]
287+-- Just 1
288+--
289+-- The above example is similar to the following right fold:
290+--
291+-- >>> Prelude.foldr (\x _ -> Just x) Nothing [1..]
292+-- Just 1
293+--
294+-- 'Fold's can be combined together using combinators. For example, to create a
295+-- fold that sums first two elements in a stream:
296+--
297+-- >>> sumTwo = Fold.take 2 Fold.sum
298+-- >>> Stream.fold sumTwo $ Stream.fromList [1..100]
299+-- 3
300+--
301+-- Folds can be combined to run in parallel on the same input. For example, to
302+-- compute the average of numbers in a stream without going through the stream
303+-- twice:
304+--
305+-- >>> avg = Fold.teeWith (/) Fold.sum (fmap fromIntegral Fold.length)
306+-- >>> Stream.fold avg $ Stream.fromList [1.0..100.0]
307+-- 50.5
308+--
309+-- Folds can be combined so as to partition the input stream over multiple
310+-- folds. For example, to count even and odd numbers in a stream:
311+--
312+-- >>> split n = if even n then Left n else Right n
313+-- >>> stream = fmap split $ Stream.fromList [1..100]
314+-- >>> countEven = fmap (("Even " ++) . show) Fold.length
315+-- >>> countOdd = fmap (("Odd " ++) . show) Fold.length
316+-- >>> f = Fold.partition countEven countOdd
317+-- >>> Stream.fold f stream
318+-- ("Even 50","Odd 50")
319+--
320+-- Terminating folds can be combined to parse the stream serially such that the
321+-- first fold consumes the input until it terminates and the second fold
322+-- consumes the rest of the input until it terminates:
323+--
324+-- >>> f = Fold.splitWith (,) (Fold.take 8 Fold.toList) (Fold.takeEndBy (== '\n') Fold.toList)
325+-- >>> Stream.fold f $ Stream.fromList "header: hello\n"
326+-- ("header: ","hello\n")
327+--
328+-- A 'Fold' can be applied repeatedly on a stream to transform it to a stream
329+-- of fold results. To split a stream on newlines:
330+--
331+-- >>> f = Fold.takeEndBy (== '\n') Fold.toList
332+-- >>> Stream.fold Fold.toList $ Stream.foldMany f $ Stream.fromList "Hello there!\nHow are you\n"
333+-- ["Hello there!\n","How are you\n"]
334+--
335+-- Similarly, we can split the input of a fold too:
336+--
337+-- >>> Stream.fold (Fold.many f Fold.toList) $ Stream.fromList "Hello there!\nHow are you\n"
338+-- ["Hello there!\n","How are you\n"]
339+--
340+-- = Folds vs. Streams
341+--
342+-- We can often use streams or folds to achieve the same goal. However, streams
343+-- are more efficient in composition of producers (e.g.
344+-- 'Data.Stream.append' or 'Data.Stream.mergeBy') whereas folds are
345+-- more efficient in composition of consumers (e.g. 'splitWith', 'partition'
346+-- or 'teeWith').
347+--
348+-- Streams are producers, transformations on streams happen on the output side:
349+--
350+-- >>> :{
351+-- f stream =
352+-- Stream.filter odd stream
353+-- & fmap (+1)
354+-- & Stream.fold Fold.sum
355+-- :}
356+--
357+-- >>> f $ Stream.fromList [1..100 :: Int]
358+-- 2550
359+--
360+-- Folds are stream consumers with an input stream and an output value, stream
361+-- transformations on folds happen on the input side:
362+--
363+-- >>> :{
364+-- f =
365+-- Fold.filter odd
366+-- $ Fold.lmap (+1)
367+-- $ Fold.sum
368+-- :}
369+--
370+-- >>> Stream.fold f $ Stream.fromList [1..100 :: Int]
371+-- 2550
372+--
373+-- Notice the similiarity in the definition of @f@ in both cases, the only
374+-- difference is the composition by @&@ vs @$@ and the use @lmap@ vs @map@, the
375+-- difference is due to output vs input side transformations.
376+
408377--------------------------------------------------------------------------------
409378-- Deprecated
410379--------------------------------------------------------------------------------
core/src/Streamly/Data/MutArray.hs+12−28
@@ -12,7 +12,7 @@
1212-- contents of a mutable array can be modified in-place. For general
1313-- documentation, please refer to the original module.
1414--
15−-- Please refer to "Streamly.Internal.Data.MutArray" for functions that have
15+-- Please refer to "Streamly.Internal.Data.Array.Mut" for functions that have
1616-- not yet been released.
1717--
1818-- For mutable arrays that work on boxed types, not requiring the 'Unbox'
@@ -32,27 +32,22 @@ module Streamly.Data.MutArray
3232 -- * Construction
3333
3434 -- Uninitialized Arrays
35− , emptyOf
36− , pinnedEmptyOf
35+ , new
36+ , newPinned
3737
3838 -- From containers
3939 , fromListN
4040 , fromList
41− , createOf
42− , create
41+ , writeN -- drop new
42+ , write -- full buffer
4343 -- writeLastN
4444
45− -- * Pinning & Unpinning
46− , pin
47− , unpin
48− , isPinned
49−
5045 -- * Appending elements
5146 , snoc
5247
5348 -- * Appending streams
54− , appendN
55− , append
49+ , writeAppendN -- XXX Rename to writeSnocN?
50+ , writeAppend -- XXX Rename to writeSnoc?
5651
5752 -- * Inplace mutation
5853 , putIndex
@@ -83,28 +78,17 @@ module Streamly.Data.MutArray
8378 -- * Size
8479 , length
8580
86− -- * Re-exports
81+ -- * Unbox Type Class
8782 , Unbox (..)
8883
89− -- * Deprecated
90− , newPinned
91− , new
92− , pinnedNew
93− , writeN
94− , write
95− , writeAppendN
96− , writeAppend
84+ -- * Pinning & Unpinning
85+ , pin
86+ , unpin
9787 )
9888where
9989
10090import Prelude hiding (length, read)
101−import Streamly.Internal.Data.MutArray
91+import Streamly.Internal.Data.Array.Mut
10292import Streamly.Internal.Data.Unbox (Unbox (..))
103−import Control.Monad.IO.Class (MonadIO)
10493
10594#include "DocTestDataMutArray.hs"
106−
107−{-# DEPRECATED newPinned "Please use pinnedNew instead." #-}
108−{-# INLINE newPinned #-}
109−newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
110−newPinned = pinnedNew
core/src/Streamly/Data/MutArray/Generic.hs+4−9
@@ -23,11 +23,11 @@ module Streamly.Data.MutArray.Generic
2323 MutArray
2424
2525 -- * Construction
26− , emptyOf
26+ , new
2727 , fromListN
2828 , fromList
29− , createOf
30− , create
29+ , writeN
30+ , write
3131
3232 -- * Appending elements
3333 , snoc
@@ -56,15 +56,10 @@ module Streamly.Data.MutArray.Generic
5656
5757 -- * Size
5858 , length
59−
60− -- * Deprecated
61− , new
62− , writeN
63− , write
6459 )
6560where
6661
67−import Streamly.Internal.Data.MutArray.Generic
62+import Streamly.Internal.Data.Array.Generic.Mut.Type
6863import Prelude hiding (read, length)
6964
7065#include "DocTestDataMutArrayGeneric.hs"
core/src/Streamly/Data/MutByteArray.hsdeleted−106
@@ -1,106 +0,0 @@
1−-- |
2−-- Module : Streamly.Data.MutByteArray
3−-- Copyright : (c) 2023 Composewell Technologies
4−-- License : BSD3-3-Clause
5−-- Maintainer : streamly@composewell.com
6−-- Portability : GHC
7−--
8−-- A low level byte Array type 'MutByteArray', along with type classes 'Unbox'
9−-- and 'Serialize' for fast binary serialization and deserialization of Haskell
10−-- values. Serialization, deserialization performance is similar to, and in
11−-- some cases many times better than the store package. Conceptually, the
12−-- 'Serialize' type class works in the same way as store.
13−--
14−-- Serialize instances are configurable to use constructor names (see
15−-- 'Streamly.Internal.Data.MutByteArray.encodeConstrNames'), record field names (see
16−-- 'Streamly.Internal.Data.MutByteArray.encodeRecordFields') instead of binary
17−-- encoded values. This is an experimental feature which allows JSON like
18−-- properties with faster speed. For example, you can change the order of
19−-- constructors or record fields without affecting serialized value.
20−--
21−-- Higher level unboxed array modules "Streamly.Data.Array" and
22−-- "Streamly.Data.MutArray" are built on top of this module. Unboxed arrays are
23−-- essentially serialized Haskell values. Array modules provide higher level
24−-- serialization routines like 'Streamly.Internal.Data.Array.pinnedSerialize'
25−-- and 'Streamly.Internal.Data.Array.deserialize' from the
26−-- "Streamly.Internal.Data.Array" module.
27−--
28−-- == Mutable Byte Array
29−--
30−-- 'MutByteArray' is a primitive mutable array in the IO monad. 'Unbox' and
31−-- 'Serialize' type classes use this primitive array to serialize data to and
32−-- deserialize it from. This array is used to build higher level unboxed
33−-- array types 'Streamly.Data.MutArray.MutArray' and 'Streamly.Data.Array.Array'.
34−--
35−-- == Using Unbox
36−--
37−-- The 'Unbox' type class is simple and used to serialize non-recursive fixed
38−-- size data types. This type class is primarily used to implement unboxed
39−-- arrays. Unboxed arrays are just a sequence of serialized fixed length
40−-- Haskell data types. Instances of this type class can be derived using
41−-- 'Generic' or template haskell based deriving functions provided in this
42−-- module.
43−--
44−-- Writing a data type to an array using the array creation routines in
45−-- "Streamly.Data.Array" or "Streamly.Data.MutArray" (e.g. @writeN@ or
46−-- @fromListN@), serializes the type to the array. Similarly, reading the data
47−-- type from the array deserializes it. You can also serialize and deserialize
48−-- directly to and from a 'MutByteArray', using the type class methods.
49−--
50−-- == Using Serialize
51−--
52−-- The 'Serialize' type class is a superset of the 'Unbox' type class, it can
53−-- serialize variable length data types as well e.g. Haskell lists. Use
54−-- 'deriveSerialize' to derive the instances of the type class automatically
55−-- and then use the type class methods to serialize and deserialize to and from
56−-- a 'MutByteArray'.
57−--
58−-- See 'Streamly.Internal.Data.Array.pinnedSerialize' and
59−-- 'Streamly.Internal.Data.Array.deserialize' for 'Array' type based
60−-- serialization.
61−--
62−-- == Comparing serialized values
63−--
64−-- When using the `Unbox` type class the same value may result in differing
65−-- serialized bytes because of unused uninitialized data in case of sum types.
66−-- Therefore, byte comparison of serialized values is not reliable.
67−--
68−-- However, the 'Serialize' type class guarantees that the serialized values
69−-- are always exactly the same and byte comparison of serialized is reliable.
70−--
71−module Streamly.Data.MutByteArray
72− (
73−
74− -- * Mutable Byte Array
75− -- | The standard way to read from or write to a 'MutByteArray' is by using
76− -- the 'Unbox' or 'Serialize' type class methods.
77− MutByteArray
78− , isPinned
79− , pin
80− , unpin
81− , new
82− , pinnedNew
83−
84− -- * Unbox
85− , Unbox(..)
86− , deriveUnbox
87−
88− -- * Serialize
89− , Serialize(..)
90−
91− -- ** Instance Config
92− , SerializeConfig
93− , inlineAddSizeTo
94− , inlineSerializeAt
95− , inlineDeserializeAt
96−
97− -- ** Instance Deriving
98− , deriveSerialize
99− , deriveSerializeWith
100− ) where
101−
102−--------------------------------------------------------------------------------
103−-- Imports
104−--------------------------------------------------------------------------------
105−
106−import Streamly.Internal.Data.MutByteArray
core/src/Streamly/Data/Parser.hs+60−130
@@ -7,137 +7,12 @@
77-- Stability : pre-release
88-- Portability : GHC
99--
10−-- Parsers are stream consumers like folds with the following differences:
10+-- Fast, composable stream consumers with ability to terminate, backtrack and
11+-- fail, supporting stream fusion. Parsers are a natural extension of
12+-- "Streamly.Data.Fold". Parsers and folds can be interconverted.
1113--
12−-- * folds cannot fail but parsers can fail and backtrack.
13−-- * folds can be composed as a Tee but parsers cannot.
14−-- * folds can be used for scanning but parsers cannot.
15−-- * folds can be converted to parsers.
16−--
17−-- This module implements parsers with stream fusion which compile to efficient
18−-- loops comparable to the speed of C.
19−--
20−-- == Using Parsers
21−--
22−-- This module provides elementary parsers and parser combinators that can be
23−-- used to parse a stream of data. Additionally, all the folds from the
24−-- "Streamly.Data.Fold" module can be converted to parsers using 'fromFold'.
25−-- All the parsing functionality provided by popular parsing libraries, and
26−-- more is available. Also see "Streamly.Unicode.Parser" module for Char stream
27−-- parsers.
28−--
29−-- A data stream can be transformed to a stream of parsed data elements. Parser
30−-- combinators can be used to create a pipeline of folds or parsers such that
31−-- the next fold or parser consumes the result of the previous parser. See
32−-- 'Streamly.Data.Stream.parse' and 'Streamly.Data.Stream.parseMany' to run
33−-- these parsers on a stream.
34−--
35−-- == Parser vs ParserK
36−--
37−-- There are two functionally equivalent parsing modules,
38−-- "Streamly.Data.Parser" (this module) and "Streamly.Data.ParserK". The latter
39−-- is a CPS based wrapper over the former, and can be used for parsing in
40−-- general. "Streamly.Data.Parser" enables stream fusion and should be
41−-- preferred over "Streamly.Data.ParserK" for high performance stream parsing
42−-- use cases. However, there are a few cases where this module is not
43−-- suitable and ParserK should be used instead.
44−--
45−-- For static fusion, parser combinators have to use strict pattern matching on
46−-- arguments of type Parser. This leads to infinte loop when a parser is
47−-- defined recursively, due to strict evaluation of the recursive call. For
48−-- example, the following implementation loops infinitely because of the
49−-- recursive use of parser @p@ in the @*>@ combinator:
50−--
51−-- >>> import Streamly.Data.Parser (Parser)
52−-- >>> import qualified Streamly.Data.Fold as Fold
53−-- >>> import qualified Streamly.Data.Parser as Parser
54−-- >>> import qualified Streamly.Data.Stream as Stream
55−-- >>> import Control.Applicative ((<|>))
56−--
57−-- >>> :{
58−-- >>> p :: Monad m => Parser Char m String
59−-- >>> p = Parser.satisfy (== '(') *> p <|> Parser.fromFold Fold.toList
60−-- >>> :}
61−--
62−-- Use ParserK when recursive use is required:
63−--
64−-- >>> import Streamly.Data.ParserK (ParserK)
65−-- >>> import qualified Streamly.Data.StreamK as StreamK
66−-- >>> import qualified Streamly.Internal.Data.StreamK as StreamK (parse)
67−-- >>> import qualified Streamly.Internal.Data.ParserK as ParserK (adapt)
68−--
69−-- >>> :{
70−-- >>> p :: Monad m => ParserK Char m String
71−-- >>> p = ParserK.adapt (Parser.satisfy (== '(')) *> p <|> ParserK.adapt (Parser.fromFold Fold.toList)
72−-- >>> :}
73−--
74−-- >>> StreamK.parse p $ StreamK.fromStream $ Stream.fromList "hello"
75−-- Right "hello"
76−--
77−-- For this reason Applicative, Alternative or Monad compositions with
78−-- recursion cannot be used with the 'Parser' type. Alternative type class based
79−-- operations like 'asum' and Alternative based generic parser combinators use
80−-- recursion. Similarly, Applicative type class based operations like
81−-- 'Prelude.sequence' use recursion. Custom implementations of many such
82−-- operations are provided in this module (e.g. 'some', 'many'), and those
83−-- should be used instead.
84−--
85−-- Another limitation of Parser type is due to the quadratic complexity causing
86−-- slowdown when too many nested compositions are used. Especially Applicative,
87−-- Monad, Alternative instances, and sequenced parsing operations (e.g. nested
88−-- 'one', and 'splitWith') degrade the performance quadratically (O(n^2)) when
89−-- combined @n@ times, roughly 8 or less sequenced parsers are fine. READ THE
90−-- DOCS OF APPLICATIVE, MONAD AND ALTERNATIVE INSTANCES.
91−--
92−-- == Streaming Parsers
93−--
94−-- With 'Streamly.Data.ParserK.ParserK' you can use the generic Alternative
95−-- type class based parsers from the
96−-- <https://hackage.haskell.org/package/parser-combinators parser-combinators>
97−-- library or similar. However, we recommend that you use the equivalent
98−-- functionality from this module for better performance and for streaming
99−-- behavior.
100−--
101−-- Firstly, the combinators in this module are faster due to stream fusion.
102−-- Secondly, these are streaming in nature as the results can be passed
103−-- directly to other stream consumers (folds or parsers). The Alternative type
104−-- class based parsers would end up buffering all the results in lists before
105−-- they can be consumed.
106−--
107−-- When recursion or heavy nesting is needed use ParserK.
108−--
109−-- == Error Reporting
110−--
111−-- These parsers do not report the error context (e.g. line number or column).
112−-- This may be supported in future.
113−--
114−-- == Monad Transformer Stack
115−--
116−-- 'MonadTrans' instance is not provided. If the 'Parser' type is the top most
117−-- layer (which should be the case almost always) you can just use 'fromEffect'
118−-- to execute the lower layer monad effects.
119−--
120−-- == Parser vs ParserK Implementation
121−--
122−-- The 'Parser' type represents a stream consumer by composing state as data
123−-- which enables stream fusion. Stream fusion generates a tight loop without
124−-- any constructor allocations between the stages, providing C like performance
125−-- for the loop. Stream fusion works when multiple functions are combined in a
126−-- pipeline statically. Therefore, the operations in this module must be
127−-- inlined and must not be used recursively to allow for stream fusion.
128−--
129−-- The 'ParserK' type represents a stream consumer by composing function calls,
130−-- therefore, a function call overhead is incurred at each composition. It is
131−-- quite fast in general but may be a few times slower than a fused parser.
132−-- However, it allows for scalable dynamic composition especially parsers can
133−-- be used in recursive calls. Using the 'ParserK' type operations like
134−-- 'splitWith' provide linear (O(n)) performance with respect to the number of
135−-- compositions.
136−--
137−-- == Experimental APIs
138−--
139−-- Please refer to "Streamly.Internal.Data.Parser" for functions that have not
140−-- yet been released.
14+-- Please refer to "Streamly.Internal.Data.Parser" for functions that have
15+-- not yet been released.
14116--
14217module Streamly.Data.Parser
14318 (
@@ -147,6 +22,9 @@ module Streamly.Data.Parser
14722 --
14823 -- $setup
14924
25+ -- * Overview
26+ -- $overview
27+
15028 -- * Parser Type
15129 Parser
15230
@@ -245,3 +123,55 @@ import Streamly.Internal.Data.Parser
245123import Prelude hiding (dropWhile, takeWhile, filter)
246124
247125#include "DocTestDataParser.hs"
126+
127+-- $overview
128+--
129+-- Several combinators in this module can be many times faster than CPS based
130+-- parsers because of stream fusion. For example,
131+-- 'Streamly.Internal.Data.Parser.many' combinator in this module is much
132+-- faster than the 'Control.Applicative.many' combinator of
133+-- 'Control.Applicative.Alternative' type class used by CPS based parsers.
134+--
135+-- The use of 'Alternative' type class, in parsers has another drawback.
136+-- Alternative based parsers use plain Haskell lists to collect the results. In
137+-- a strict Monad like IO, the results are necessarily buffered before they can
138+-- be consumed. This may not perform optimally in streaming applications
139+-- processing large amounts of data. Equivalent combinators in this module can
140+-- consume the results of parsing using a 'Fold' or another parser, thus
141+-- providing a scalable and composable consumer.
142+--
143+-- Note that these parsers do not report the error context (e.g. line number or
144+-- column). This may be supported in future.
145+--
146+-- mtl instances are not provided. If the 'Parser' type is the top most layer
147+-- (which should be the case almost always) you can just use 'fromEffect' to
148+-- execute the lower layer monad effects.
149+--
150+-- == Performance Notes
151+--
152+-- The 'Parser' type represents a stream consumer by composing state as data
153+-- which enables stream fusion. Stream fusion generates a tight loop without
154+-- any constructor allocations between the stages, providing C like performance
155+-- for the loop. Stream fusion works when multiple functions are combined in a
156+-- pipeline statically. Therefore, the operations in this module must be
157+-- inlined and must not be used recursively to allow for stream fusion. Note
158+-- that operations like 'sequence', and 'asum' that compose pasrers using
159+-- recursion should be avoided with these parsers. You can use these with the
160+-- 'ParserK' module instead.
161+--
162+-- Using the 'Parser' type, parsing operations like 'one', 'splitWith' etc.
163+-- degrade quadratically (O(n^2)) when combined many times. If you need to
164+-- combine these operations, say more than 8 times in a single loop, then you
165+-- should consider using the continuation style parser type 'ParserK' instead.
166+-- Also, if you need to use these operations in a recursive loop you should use
167+-- 'ParserK' instead.
168+--
169+-- The 'ParserK' type represents a stream consumer by composing function calls,
170+-- therefore, a function call overhead is incurred at each composition. It is
171+-- quite fast in general but may be a few times slower than a fused parser.
172+-- However, it allows for scalable dynamic composition especially parsers can
173+-- be used in recursive calls. Using the 'ParserK' type operations like
174+-- 'splitWith' provide linear (O(n)) performance with respect to the number of
175+-- compositions..
176+--
177+-- 'Parser' and 'ParserK' types can be interconverted.
core/src/Streamly/Data/ParserK.hs+10−105
@@ -1,4 +1,3 @@
1−{-# LANGUAGE CPP #-}
21-- |
32-- Module : Streamly.Data.ParserK
43-- Copyright : (c) 2023 Composewell Technologies
@@ -7,112 +6,26 @@
76-- Stability : pre-release
87-- Portability : GHC
98--
10−-- See the general notes about parsing in the "Streamly.Data.Parser" module.
11−-- This module implements a using Continuation Passing Style (CPS) wrapper over
12−-- the "Streamly.Data.Parser" module. It is as fast or faster than attoparsec.
9+-- Parsers using Continuation Passing Style (CPS). See notes in
10+-- "Streamly.Data.Parser" module to know when to use this module.
1311--
14−-- == Parser vs ParserK
15−--
16−-- 'ParserK' is preferred over 'Streamly.Data.Parser.Parser' when extensive
17−-- applicative, alternative and monadic composition is required, or when
18−-- recursive or dynamic composition of parsers is required. The
19−-- 'Streamly.Data.Parser.Parser' type fuses statically and creates efficient
20−-- loops whereas 'ParserK' uses function call based composition and has
21−-- comparatively larger runtime overhead but it is better suited to the
22−-- specific use cases mentioned above. 'ParserK' also allows to efficient parse
23−-- a stream of arrays, it can also break the input stream into a parse result
24−-- and remaining stream so that the stream can be parsed independently in
25−-- segments.
26−--
27−-- == Using ParserK
28−--
29−-- All the parsers from the "Streamly.Data.Parser" module can be adapted to
30−-- ParserK using the 'Streamly.Data.ParserK.adaptC',
31−-- 'Streamly.Internal.Data.ParserK.adapt', and
32−-- 'Streamly.Internal.Data.ParserK.adaptCG' combinators.
33−--
34−-- 'Streamly.Data.StreamK.parseChunks' runs a parser on a stream of unboxed
35−-- arrays, this is the preferred and most efficient way to parse chunked input.
36−-- The more general 'Streamly.Data.StreamK.parseBreakChunks' function returns
37−-- the remaining stream as well along with the parse result. There are
38−-- 'Streamly.Internal.Data.StreamK.parseChunksGeneric',
39−-- 'Streamly.Internal.Data.StreamK.parseBreakChunksGeneric' as well to run
40−-- parsers on boxed arrays. 'Streamly.Internal.Data.StreamK.parse',
41−-- 'Streamly.Internal.Data.StreamK.parseBreak' run parsers on a stream of
42−-- individual elements instead of stream of arrays.
43−--
44−-- == Monadic Composition
45−--
46−-- Monad composition can be used for lookbehind parsers, we can dynamically
47−-- compose new parsers based on the results of the previously parsed values.
48−--
49−-- If we have to parse "a9" or "9a" but not "99" or "aa" we can use the
50−-- following non-monadic, backtracking parser:
51−--
52−-- >>> digits p1 p2 = ((:) <$> p1 <*> ((:) <$> p2 <*> pure []))
53−-- >>> :{
54−-- backtracking :: Monad m => ParserK Char m String
55−-- backtracking = ParserK.adapt $
56−-- digits (Parser.satisfy isDigit) (Parser.satisfy isAlpha)
57−-- <|>
58−-- digits (Parser.satisfy isAlpha) (Parser.satisfy isDigit)
59−-- :}
60−--
61−-- We know that if the first parse resulted in a digit at the first place then
62−-- the second parse is going to fail. However, we waste that information and
63−-- parse the first character again in the second parse only to know that it is
64−-- not an alphabetic char. By using lookbehind in a 'Monad' composition we can
65−-- avoid redundant work:
66−--
67−-- >>> data DigitOrAlpha = Digit Char | Alpha Char
68−--
69−-- >>> :{
70−-- lookbehind :: Monad m => ParserK Char m String
71−-- lookbehind = do
72−-- x1 <- ParserK.adapt $
73−-- Digit <$> Parser.satisfy isDigit
74−-- <|> Alpha <$> Parser.satisfy isAlpha
75−-- -- Note: the parse depends on what we parsed already
76−-- x2 <- ParserK.adapt $
77−-- case x1 of
78−-- Digit _ -> Parser.satisfy isAlpha
79−-- Alpha _ -> Parser.satisfy isDigit
80−-- return $ case x1 of
81−-- Digit x -> [x,x2]
82−-- Alpha x -> [x,x2]
83−-- :}
84−--
85−-- == Experimental APIs
86−--
87−-- Please refer to "Streamly.Internal.Data.ParserK" for functions that have
88−-- not yet been released.
12+-- To run a 'ParserK' use 'Streamly.Data.StreamK.parseChunks'.
8913--
9014module Streamly.Data.ParserK
9115 (
92− -- * Setup
93− -- | To execute the code examples provided in this module in ghci, please
94− -- run the following commands first.
95− --
96− -- $setup
97−
9816 -- * Parser Type
9917 ParserK
10018
10119 -- * Parsers
10220 -- ** Conversions
103− , adapt
104− , adaptC
105− , adaptCG
21+ , fromFold
22+ , fromParser
10623 -- , toParser
10724
10825 -- ** Without Input
10926 , fromPure
11027 , fromEffect
11128 , die
112−
113− -- * Deprecated
114− , fromFold
115− , fromParser
11629 )
11730
11831where
@@ -120,20 +33,12 @@ where
12033import Control.Monad.IO.Class (MonadIO)
12134import Streamly.Internal.Data.Fold (Fold)
12235import Streamly.Internal.Data.Unbox (Unbox)
123−import Streamly.Internal.Data.Array (Array)
12436import qualified Streamly.Internal.Data.Parser as ParserD
12537
126−import Streamly.Internal.Data.ParserK.Type
127−
128−#include "DocTestDataParserK.hs"
38+import Streamly.Internal.Data.Parser.ParserK.Type
12939
130−{-# DEPRECATED fromFold "Please use \"ParserK.adaptC . Parser.fromFold\" instead." #-}
40+-- | Convert a 'Fold' to a 'ParserK'.
41+--
13142{-# INLINE fromFold #-}
132−fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK (Array a) m b
133−fromFold = adaptC . ParserD.fromFold
134−
135−{-# DEPRECATED fromParser "Please use \"adaptC\" instead." #-}
136−{-# INLINE fromParser #-}
137−fromParser ::
138− (MonadIO m, Unbox a) => ParserD.Parser a m b -> ParserK (Array a) m b
139−fromParser = adaptC
43+fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK a m b
44+fromFold = fromParser . ParserD.fromFold
core/src/Streamly/Data/Stream.hs+109−221
@@ -8,20 +8,15 @@
88-- Stability : released
99-- Portability : GHC
1010--
11−-- Streams represented as state machines, that fuse together when composed
12−-- statically, eliminating function calls or intermediate constructor
13−-- allocations - generating tight, efficient loops. Suitable for high
14−-- performance looping operations.
15−--
16−-- If you need to call these operations recursively in a loop (i.e. composed
17−-- dynamically) then it is recommended to use the continuation passing style
18−-- (CPS) stream operations from the "Streamly.Data.StreamK" module. 'Stream'
19−-- and 'StreamK' types are interconvertible. See more details in the
20−-- documentation below regarding 'Stream' vs 'StreamK'.
11+-- Fast, composable stream producers with ability to terminate, supporting
12+-- stream fusion.
2113--
2214-- Please refer to "Streamly.Internal.Data.Stream" for more functions that have
2315-- not yet been released.
2416--
17+-- For continuation passing style (CPS) stream type, please refer to
18+-- the "Streamly.Data.StreamK" module.
19+--
2520-- Checkout the <https://github.com/composewell/streamly-examples>
2621-- repository for many more real world examples of stream programming.
2722
@@ -42,6 +37,9 @@ module Streamly.Data.Stream
4237 -- * Construction
4338 -- | Functions ending in the general shape @b -> Stream m a@.
4439 --
40+ -- See also: "Streamly.Internal.Data.Stream.Generate" for
41+ -- @Pre-release@ functions.
42+ --
4543 -- Useful Idioms:
4644 --
4745 -- >>> fromIndices f = fmap f $ Stream.enumerateFrom 0
@@ -51,12 +49,11 @@ module Streamly.Data.Stream
5149 -- >>> fromFoldableM = Stream.sequence . fromFoldable
5250
5351 -- ** Primitives
54− -- | A fused 'Stream' is never constructed using these primitives, they are
55− -- typically generated by converting containers like list into streams, or
56− -- generated using custom functions provided in this module. The 'cons'
57− -- primitive in this module has a rare use in fusing a small number of
58− -- elements. On the other hand, it is common to construct 'StreamK' stream
59− -- using the StreamK.'StreamK.cons' primitive.
52+ -- | A fused 'Stream' is never constructed using 'cons', we always convert
53+ -- other type of containers like list into streams, or generate it using
54+ -- custom functions provided in this module. The 'cons' primitive has a
55+ -- rare use in fusing a small number of elements. On the other hand, it is
56+ -- common to construct 'StreamK' stream using the 'StreamK.cons' primitive.
6057 , nil
6158 , nilM
6259 , cons
@@ -70,15 +67,6 @@ module Streamly.Data.Stream
7067
7168 -- ** From Values
7269 -- | Generate a monadic stream from a seed value or values.
73− --
74− -- Non-empty pure streams can be cycled using repeat and unfold:
75− --
76− -- >>> cycle = Stream.unfoldMany Unfold.fromList . Stream.repeat
77− --
78− -- Non-empty StreamK can be cycled using append:
79− --
80− -- >>> cycle xs = let ys = xs `StreamK.append` ys in ys
81− --
8270 , fromPure
8371 , fromEffect
8472 , repeat
@@ -123,6 +111,8 @@ module Streamly.Data.Stream
123111 -- | Functions ending in the general shape @Stream m a -> m b@ or @Stream m
124112 -- a -> m (b, Stream m a)@
125113 --
114+ -- See also: "Streamly.Internal.Data.Stream.Eliminate" for @Pre-release@
115+ -- functions.
126116
127117-- EXPLANATION: In imperative terms a fold can be considered as a loop over the stream
128118-- that reduces the stream to a single value.
@@ -221,57 +211,21 @@ module Streamly.Data.Stream
221211 -- operations like mapping a function over the stream.
222212 , foldrM
223213 , foldr
224− -- foldr1
225214
226215 -- ** Specific Folds
227− -- | Streams are folded using folds in "Streamly.Data.Fold". Here are some
228− -- idioms and equivalents of Data.List APIs using folds:
216+ -- | Usually you can use the folds in "Streamly.Data.Fold". However, some
217+ -- folds that may be commonly used or may have an edge in performance in
218+ -- some cases are provided here.
219+ --
220+ -- Useful idioms:
229221 --
230222 -- >>> foldlM' f a = Stream.fold (Fold.foldlM' f a)
231− -- >>> foldl1' f = Stream.fold (Fold.foldl1' f)
223+ -- >>> foldl1 f = Stream.fold (Fold.foldl1' f)
232224 -- >>> foldl' f a = Stream.fold (Fold.foldl' f a)
233225 -- >>> drain = Stream.fold Fold.drain
234226 -- >>> mapM_ f = Stream.fold (Fold.drainMapM f)
235227 -- >>> length = Stream.fold Fold.length
236− -- >>> genericLength = Stream.fold Fold.lengthGeneric
237228 -- >>> head = Stream.fold Fold.one
238− -- >>> last = Stream.fold Fold.latest
239− -- >>> null = Stream.fold Fold.null
240− -- >>> and = Stream.fold Fold.and
241− -- >>> or = Stream.fold Fold.or
242− -- >>> any p = Stream.fold (Fold.any p)
243− -- >>> all p = Stream.fold (Fold.all p)
244− -- >>> sum = Stream.fold Fold.sum
245− -- >>> product = Stream.fold Fold.product
246− -- >>> maximum = Stream.fold Fold.maximum
247− -- >>> maximumBy cmp = Stream.fold (Fold.maximumBy cmp)
248− -- >>> minimum = Stream.fold Fold.minimum
249− -- >>> minimumBy cmp = Stream.fold (Fold.minimumBy cmp)
250− -- >>> elem x = Stream.fold (Fold.elem x)
251− -- >>> notElem x = Stream.fold (Fold.notElem x)
252− -- >>> lookup x = Stream.fold (Fold.lookup x)
253− -- >>> find p = Stream.fold (Fold.find p)
254− -- >>> (!?) i = Stream.fold (Fold.index i)
255− -- >>> genericIndex i = Stream.fold (Fold.indexGeneric i)
256− -- >>> elemIndex x = Stream.fold (Fold.elemIndex x)
257− -- >>> findIndex p = Stream.fold (Fold.findIndex p)
258− --
259− -- Some equivalents of Data.List APIs from the Stream module:
260− --
261− -- >>> head = fmap (fmap fst) . Stream.uncons
262− -- >>> tail = fmap (fmap snd) . Stream.uncons
263− -- >>> tail = Stream.tail -- unreleased API
264− -- >>> init = Stream.init -- unreleased API
265− --
266− -- A Stream based toList fold implementation is provided below because it
267− -- has a better performance compared to the fold.
268−
269− -- Functions in Data.List, missing here:
270− -- unsnoc = Stream.parseBreak (Parser.init Fold.toList)
271− -- genericTake
272− -- genericDrop
273− -- genericSplitAt
274− -- genericReplicate
275229 , toList
276230
277231 -- * Mapping
@@ -299,6 +253,8 @@ module Streamly.Data.Stream
299253 -- * Scanning
300254 -- | Stateful one-to-one transformations.
301255 --
256+ -- See also: "Streamly.Internal.Data.Stream.Transform" for
257+ -- @Pre-release@ functions.
302258
303259 {-
304260 -- ** Left scans
@@ -325,15 +281,6 @@ module Streamly.Data.Stream
325281 , scan
326282 , postscan
327283 -- XXX postscan1 can be implemented using Monoids or Refolds.
328− -- The following scans from Data.List are not provided.
329− -- XXX scanl
330− -- XXX scanl1
331− -- XXX scanr
332− -- XXX scanr1
333− -- XXX mapAccumL
334− -- XXX mapAccumR
335− -- XXX inits
336− -- XXX tails
337284
338285 -- ** Specific scans
339286 -- Indexing can be considered as a special type of zipping where we zip a
@@ -342,8 +289,6 @@ module Streamly.Data.Stream
342289
343290 -- * Insertion
344291 -- | Add elements to the stream.
345− --
346− -- >>> insert = Stream.insertBy compare
347292
348293 -- Inserting elements is a special case of interleaving/merging streams.
349294 , insertBy
@@ -379,15 +324,12 @@ module Streamly.Data.Stream
379324 -- "Streamly.Internal.Data.Fold" can be used along with 'scanMaybe' to
380325 -- perform stateful filtering operations in general.
381326 --
382− -- Idioms and equivalents of Data.List APIs:
327+ -- Useful idioms:
383328 --
384329 -- >>> deleteBy cmp x = Stream.scanMaybe (Fold.deleteBy cmp x)
385− -- >>> deleteBy = Stream.deleteBy -- unreleased API
386− -- >>> delete = deleteBy (==)
387330 -- >>> findIndices p = Stream.scanMaybe (Fold.findIndices p)
388331 -- >>> elemIndices a = findIndices (== a)
389332 -- >>> uniq = Stream.scanMaybe (Fold.uniqBy (==))
390− -- >>> partition p = Stream.fold (Fold.partition Fold.toList Fold.toList) . fmap (if p then Left else Right)
391333 , scanMaybe
392334 , take
393335 , takeWhile
@@ -395,8 +337,6 @@ module Streamly.Data.Stream
395337 , drop
396338 , dropWhile
397339 , dropWhileM
398− -- XXX write to an array in reverse and then read in reverse
399− -- > dropWhileEnd = reverse . dropWhile p . reverse
400340
401341 -- XXX These are available as scans in folds. We need to check the
402342 -- performance though. If these are common and we need convenient stream
@@ -418,42 +358,34 @@ module Streamly.Data.Stream
418358 -- , elemIndices
419359
420360 -- * Combining Two Streams
421− -- | Note that these operations are suitable for statically fusing a few
422− -- streams, they have a quadratic O(n^2) time complexity wrt to the number
423− -- of streams. If you want to compose many streams dynamically using binary
424− -- combining operations see the corresponding operations in
425− -- "Streamly.Data.StreamK".
426− --
427− -- When fusing more than two streams it is more efficient if the binary
428− -- operations are composed as a balanced tree rather than a right
429− -- associative or left associative one e.g.:
430− --
431− -- >>> s1 = Stream.fromList [1,2] `Stream.append` Stream.fromList [3,4]
432− -- >>> s2 = Stream.fromList [4,5] `Stream.append` Stream.fromList [6,7]
433− -- >>> s = s1 `Stream.append` s2
434−
435361 -- ** Appending
436− -- | Equivalent of Data.List append:
437− --
438− -- >>> (++) = Stream.append
439362 , append
440363
441364 -- ** Interleaving
365+ -- | When interleaving more than two streams you may want to interleave
366+ -- them pairwise creating a balanced binary merge tree.
442367 , interleave
443368
444369 -- ** Merging
370+ -- | When merging more than two streams you may want to merging them
371+ -- pairwise creating a balanced binary merge tree.
372+ --
373+ -- Merging of @n@ streams can be performed by combining the streams pair
374+ -- wise using 'mergeMapWith' to give O(n * log n) time complexity. If used
375+ -- with 'concatMapWith' it will have O(n^2) performance.
376+
445377 , mergeBy
446378 , mergeByM
447379
448380 -- ** Zipping
449− -- | Idioms and equivalents of Data.List APIs:
381+ -- | When zipping more than two streams you may want to zip them
382+ -- pairwise creating a balanced binary tree.
450383 --
451− -- >>> zip = Stream.zipWith (,)
452− -- >>> unzip = Stream.fold (Fold.unzip Fold.toList Fold.toList)
384+ -- Zipping of @n@ streams can be performed by combining the streams pair
385+ -- wise using 'mergeMapWith' with O(n * log n) time complexity. If used
386+ -- with 'concatMapWith' it will have O(n^2) performance.
453387 , zipWith
454388 , zipWithM
455− -- XXX zipWith3,4,5,6,7
456− -- XXX unzip3,4,5,6,7
457389 -- , ZipStream (..)
458390
459391 -- ** Cross Product
@@ -486,109 +418,62 @@ module Streamly.Data.Stream
486418 -- One dimension loops are just a special case of nested loops. For
487419 -- example, 'concatMap' can degenerate to a simple map operation:
488420 --
489− -- >>> map f m = Stream.concatMap (\x -> Stream.fromPure (f x)) m
421+ -- > map f m = S.concatMap (\x -> S.fromPure (f x)) m
490422 --
491423 -- Similarly, 'concatMap' can perform filtering by mapping an element to a
492424 -- 'nil' stream:
493425 --
494− -- >>> filter p m = Stream.concatMap (\x -> if p x then Stream.fromPure x else Stream.nil) m
495− --
496− -- Idioms and equivalents of Data.List APIs:
426+ -- > filter p m = S.concatMap (\x -> if p x then S.fromPure x else S.nil) m
497427 --
498− -- >>> concat = Stream.concatMap id
499− -- >>> unlines = Stream.interposeSuffix '\n'
500− -- >>> unlines = Stream.intercalateSuffix Unfold.fromList "\n"
501− -- >>> unwords = Stream.interpose ' '
502− -- >>> unwords = Stream.intercalate Unfold.fromList " "
503428
504429 , concatEffect
505430 , concatMap
506431 , concatMapM
507432
508433 -- * Repeated Fold
509− -- | Idioms and equivalents of Data.List APIs:
434+ -- | Useful idioms:
510435 --
511− -- >>> groupsOf n = Stream.foldMany (Fold.take n Fold.toList)
512− -- >>> groupBy eq = Stream.groupsWhile eq Fold.toList
513− -- >>> groupBy eq = Stream.parseMany (Parser.groupBy eq Fold.toList)
514− -- >>> groupsByRolling eq = Stream.parseMany (Parser.groupByRolling eq Fold.toList)
515− -- >>> groups = groupBy (==)
436+ -- >>> splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
437+ -- >>> splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
438+ -- >>> groupsBy eq f = Stream.parseMany (Parser.groupBy eq f)
439+ -- >>> groupsByRolling eq f = Stream.parseMany (Parser.groupByRolling eq f)
440+ -- >>> groupsOf n f = Stream.foldMany (Fold.take n f)
516441 , foldMany -- XXX Rename to foldRepeat
517442 , groupsOf
518443 , parseMany
519444
520445 -- * Splitting
521− -- | Idioms and equivalents of Data.List APIs:
522− --
523− -- >>> splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
524− -- >>> splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
525− -- >>> lines = splitOnSuffix (== '\n')
526− -- >>> words = Stream.wordsBy isSpace
527− -- >>> splitAt n = Stream.fold (Fold.splitAt n Fold.toList Fold.toList)
528− -- >>> span p = Parser.splitWith (,) (Parser.takeWhile p Fold.toList) (Parser.fromFold Fold.toList)
529− -- >>> break p = span (not . p)
530446 , splitOn
531447 , wordsBy
532448
533449 -- * Buffered Operations
534450 -- | Operations that require buffering of the stream.
535451 -- Reverse is essentially a left fold followed by an unfold.
536− --
537− -- Idioms and equivalents of Data.List APIs:
538− --
539− -- >>> nub = Stream.fold Fold.toList . Stream.scanMaybe Fold.nub
540− -- >>> nub = Stream.ordNub -- unreleased API
541− -- >>> sortBy = StreamK.sortBy
542− -- >>> sortOn f = StreamK.sortOn -- unreleased API
543− -- >>> deleteFirstsBy = Stream.deleteFirstsBy -- unreleased
544− -- >>> (\\) = Stream.deleteFirstsBy (==) -- unreleased
545− -- >>> intersectBy = Stream.intersectBy -- unreleased
546− -- >>> intersect = Stream.intersectBy (==) -- unreleased
547− -- >>> unionBy = Stream.unionBy -- unreleased
548− -- >>> union = Stream.unionBy (==) -- unreleased
549− --
550452 , reverse
551− -- XXX transpose: write the streams to arrays and then stream transposed.
552− -- XXX subsequences
553− -- XXX permutations
554− -- , nub
555− -- , ordNub
556− -- , nubBy
557453
558454 -- * Multi-Stream folds
559455 -- | Operations that consume multiple streams at the same time.
560456 , eqBy
561457 , cmpBy
562458 , isPrefixOf
563− , isInfixOf
564− -- , isSuffixOf
565− -- , isSuffixOfUnbox
566459 , isSubsequenceOf
567460
568461 -- trimming sequences
569− -- , stripPrefix
570− -- , stripSuffix
571− -- , stripSuffixUnbox
462+ , stripPrefix
572463
573464 -- Exceptions and resource management depend on the "exceptions" package
574465 -- XXX We can have IO Stream operations not depending on "exceptions"
575466 -- in Exception.Base
576467
577468 -- * Exceptions
578− -- | Note that the stream exception handling routines catch and handle
579− -- exceptions only in the stream generation steps and not in the consumer
580− -- of the stream. For example, if we are folding or parsing a stream - any
581− -- exceptions in the fold or parse steps won't be observed by the stream
582− -- exception handlers. Exceptions in the fold or parse steps can be handled
583− -- using the fold or parse exception handling routines. You can wrap the
584− -- stream elimination function in the monad exception handler to observe
585− -- exceptions in the stream as well as the consumer.
586− --
587− -- Most of these combinators inhibit stream fusion, therefore, when
469+ -- | Most of these combinators inhibit stream fusion, therefore, when
588470 -- possible, they should be called in an outer loop to mitigate the cost.
589471 -- For example, instead of calling them on a stream of chars call them on a
590472 -- stream of arrays before flattening it to a stream of chars.
591473 --
474+ -- See also: "Streamly.Internal.Data.Stream.Exception" for
475+ -- @Pre-release@ functions.
476+
592477 , onException
593478 , handle
594479
@@ -596,19 +481,8 @@ module Streamly.Data.Stream
596481 -- | 'bracket' is the most general resource management operation, all other
597482 -- operations can be expressed using it. These functions have IO suffix
598483 -- because the allocation and cleanup functions are IO actions. For
599− -- generalized allocation and cleanup functions, see the functions without
484+ -- generalized allocation and cleanup functions see the functions without
600485 -- the IO suffix in the "streamly" package.
601− --
602− -- Note that these operations bracket the stream generation only, they do
603− -- not cover the stream consumer. This means if an exception occurs in
604− -- the consumer of the stream (e.g. in a fold or parse step) then the
605− -- exception won't be observed by the stream resource handlers, in that
606− -- case the resource cleanup handler runs when the stream is garbage
607− -- collected.
608− --
609− -- Monad level resource management can always be used around the stream
610− -- elimination functions, such a function can observe exceptions in both
611− -- the stream and its consumer.
612486 , before
613487 , afterIO
614488 , finallyIO
@@ -622,20 +496,25 @@ module Streamly.Data.Stream
622496 , runReaderT
623497 , runStateT
624498
625− -- XXX Arrays could be different types, therefore, this should be in
626− -- specific array module. Or maybe we should abstract over array types.
627499 -- * Stream of Arrays
628500 , Array.chunksOf
501+ , chunksSplitOn
629502 )
630503where
631504
505+import Control.Monad.IO.Class (MonadIO)
506+import Data.Word (Word8)
507+import Streamly.Data.Array (Array)
632508import Streamly.Internal.Data.Stream
633509import Prelude
634510 hiding (filter, drop, dropWhile, take, takeWhile, zipWith, foldr,
635− mapM, sequence, reverse, iterate, foldr1, repeat, replicate,
636− concatMap)
511+ foldl, map, mapM, mapM_, sequence, all, any, sum, product, elem,
512+ notElem, maximum, minimum, head, last, tail, length, null,
513+ reverse, iterate, init, and, or, lookup, foldr1, (!!),
514+ scanl, scanl1, repeat, replicate, concatMap, span)
637515
638516import qualified Streamly.Internal.Data.Array.Type as Array
517+import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
639518
640519#include "DocTestDataStream.hs"
641520
@@ -674,11 +553,11 @@ import qualified Streamly.Internal.Data.Array.Type as Array
674553--
675554-- == Stream Fusion
676555--
677−-- The fused 'Stream' type in this module employs stream fusion for C-like
678−-- performance when looping over data. It represents the stream as a state
679−-- machine using an explicit state, and a step function working on the state. A
680−-- typical stream operation consumes elements from the previous state machine
681−-- in a stream pipeline, transforms the elements and yields new values for the
556+-- The fused 'Stream' type employs stream fusion for C-like performance when
557+-- looping over data. It represents a stream source or transformation by
558+-- defining a state machine with explicit state, and a step function working on
559+-- the state. A typical stream operation consumes elements from the previous
560+-- state machine in the pipeline, transforms them and yields new values for the
682561-- next stage to consume. The stream operations are modular and represent a
683562-- single task, they have no knowledge of previous or next operation on the
684563-- elements.
@@ -687,7 +566,7 @@ import qualified Streamly.Internal.Data.Array.Type as Array
687566-- transformation operations and a stream consumer. All these operations taken
688567-- together form a closed loop processing the stream elements. Elements are
689568-- transferred between stages using a boxed data constructor. However, all the
690−-- stages of the pipeline are fused together by GHC, eliminating the boxing of
569+-- stages of the pipeline are fused together by GHC, eliminating the
691570-- intermediate constructors, and thus forming a tight C like loop without any
692571-- boxed data being used in the loop.
693572--
@@ -702,45 +581,54 @@ import qualified Streamly.Internal.Data.Array.Type as Array
702581--
703582-- == Stream vs StreamK
704583--
705−-- The fused stream model avoids constructor allocations and function call
706−-- overheads. However, the stream is represented as a state machine, and to
707−-- generate stream elements it has to navigate the decision tree of the state
708−-- machine. Moreover, the state machine is cranked for each element in the
709−-- stream. This performs extremely well when the number of states are limited.
710−-- The state machine starts getting expensive as the number of states increase.
711−-- For example, generating a stream from a list requires a single state and is
712−-- very efficient, even if it has millions of elements. However, using 'cons'
713−-- to construct a million element stream would be a disaster.
584+-- The fused stream model avoids constructor allocations or function call
585+-- overheads. However, the stream is represented as a state machine and to
586+-- generate elements it has to navigate the decision tree of the state machine.
587+-- Moreover, the state machine is cranked for each element in the stream. This
588+-- performs extremely well when the number of states are limited. The state
589+-- machine starts getting expensive as the number of states increase. For
590+-- example, generating a million element stream from a list requires a single
591+-- state and is very efficient. However, using fused 'cons' to generate a
592+-- million element stream would be a disaster.
714593--
715594-- A typical worst case scenario for fused stream model is a large number of
716595-- `cons` or `append` operations. A few static `cons` or `append` operations
717−-- are very fast and much faster than a CPS style stream because CPS involves a
718−-- function call for each element whereas fused stream involves a few
719−-- conditional branches in the state machine. However, constructing a large
720−-- stream using `cons` introduces as many states in the state machine as the
721−-- number of elements. If we compose `cons` as a balanced binary tree it will
722−-- take @n * log n@ time to navigate the tree, and @n * n@ if it is a right
723−-- associative composition.
724−--
596+-- are very fast and much faster than a CPS style stream. However, if we
597+-- construct a large stream using `cons` it introduces as many states in the
598+-- state machine as the number of elements. If we compose the `cons` as a
599+-- binary tree it will take @n * log n@ time to navigate the tree, and @n * n@
600+-- if it is a right associative composition.
601+--
602+-- For quadratic cases of fused stream, after a certain threshold the CPS
603+-- stream would perform much better and exhibit linear performance behavior.
725604-- Operations like 'cons' or 'append'; are typically recursively called to
726605-- construct a lazy infinite stream. For such use cases the CPS style 'StreamK'
727−-- should be used. CPS streams do not have a state machine that needs to be
606+-- type is provided. CPS streams do not have a state machine that needs to be
728607-- cranked for each element, past state has no effect on the future element
729−-- processing. However, CPS incurs a function call overhead for each element
730−-- processed, the overhead could be large compared to a fused state machine
731−-- even if it has many states. However, because of its linear performance
732−-- characterstics, after a certain threshold of stream compositions the CPS
733−-- stream would perform much better than the quadratic fused stream operations.
734−--
735−-- As a general guideline, you need to use 'StreamK' when you have to use
736−-- 'cons', 'append' or other operations having quadratic complexity at a large
737−-- scale. Typically, in such cases you need to compose the stream recursively,
738−-- by calling an operation in a loop. The decision to compose the stream is
739−-- taken at run time rather than statically at compile time.
740−--
741−-- Typically you would compose a 'StreamK' of chunks of data so that the
742−-- StreamK overhead is not high, and then process the chunks using 'Stream' by
743−-- using statically fused stream pipeline operations on the chunks.
608+-- processing. However, it incurs a function call overhead for each operation
609+-- for each element, which could be very large overhead compared to fused state
610+-- machines even if it has many states and cranks it for each element. But in
611+-- some cases scales tip in favor of the CPS stream. In those cases even though
612+-- CPS has a large constant overhead, it has a linear performance rather than
613+-- quadratic.
614+--
615+-- As a general guideline, if you have to use 'cons' or 'append' or operations
616+-- of similar nature, at a large scale, then 'StreamK' should be used. When you
617+-- need to compose the stream dynamically or recursively, then 'StreamK' should
618+-- be used. Typically you would use a dynamically generated 'StreamK' with
619+-- chunks of data which can then be processed by statically fused stream
620+-- pipeline operations.
744621--
745622-- 'Stream' and 'StreamK' types can be interconverted. See
746623-- "Streamly.Data.StreamK" module for conversion operations.
624+
625+-- | Split a stream of arrays on a given separator byte, dropping the separator
626+-- and coalescing all the arrays between two separators into a single array.
627+--
628+{-# INLINE chunksSplitOn #-}
629+chunksSplitOn
630+ :: (MonadIO m)
631+ => Word8
632+ -> Stream m (Array Word8)
633+ -> Stream m (Array Word8)
634+chunksSplitOn = ArrayStream.splitOn
core/src/Streamly/Data/Stream/StreamDK.hsnew file+505
@@ -0,0 +1,505 @@
1+-- |
2+-- Module : Streamly.Data.Stream.StreamDK
3+-- Copyright : (c) 2017 Composewell Technologies
4+--
5+-- License : BSD3
6+-- Maintainer : streamly@composewell.com
7+-- Stability : released
8+-- Portability : GHC
9+--
10+-- To run examples in this module:
11+--
12+-- >>> import qualified Streamly.Data.Fold as Fold
13+-- >>> import qualified Streamly.Data.Stream.StreamDK as Stream
14+--
15+-- We will add some more imports in the examples as needed.
16+--
17+-- For effectful streams we will use the following IO action:
18+--
19+-- >>> effect n = print n >> return n
20+--
21+-- = Overview
22+--
23+-- Streamly is a framework for modular data flow based programming and
24+-- declarative concurrency. Powerful stream fusion framework in streamly
25+-- allows high performance combinatorial programming even when using byte level
26+-- streams. Streamly API is similar to Haskell lists.
27+--
28+-- Streams can be constructed like lists, except that they use 'nil' instead of
29+-- '[]' and 'cons' instead of ':'.
30+--
31+-- `cons` adds a pure value at the head of the stream:
32+--
33+-- >>> import Streamly.Data.Stream.StreamDK (Stream, cons, consM, nil)
34+-- >>> stream = 1 `cons` 2 `cons` nil :: Stream IO Int
35+-- >>> Stream.fold Fold.toList stream -- IO [Int]
36+-- [1,2]
37+--
38+-- `consM` adds an effect at the head of the stream:
39+--
40+-- >>> stream = effect 1 `consM` effect 2 `consM` nil
41+-- >>> Stream.fold Fold.toList stream
42+-- 1
43+-- 2
44+-- [1,2]
45+--
46+-- == Console Echo Example
47+--
48+-- In the following example, 'repeatM' generates an infinite stream of 'String'
49+-- by repeatedly performing the 'getLine' IO action. 'mapM' then applies
50+-- 'putStrLn' on each element in the stream converting it to stream of '()'.
51+-- Finally, 'drain' folds the stream to IO discarding the () values, thus
52+-- producing only effects.
53+--
54+-- >>> import Data.Function ((&))
55+--
56+-- >>> :{
57+-- echo =
58+-- Stream.repeatM getLine -- Stream IO String
59+-- & Stream.mapM putStrLn -- Stream IO ()
60+-- & Stream.fold Fold.drain -- IO ()
61+-- :}
62+--
63+-- This is a console echo program. It is an example of a declarative loop
64+-- written using streaming combinators. Compare it with an imperative @while@
65+-- loop.
66+--
67+-- Hopefully, this gives you an idea how we can program declaratively by
68+-- representing loops using streams. In this module, you can find all
69+-- "Data.List" like functions and many more powerful combinators to perform
70+-- common programming tasks.
71+--
72+-- == Useful Idioms
73+--
74+-- >>> fromListM = Stream.sequence . Stream.fromList
75+-- >>> fromFoldableM = Stream.sequence . Stream.fromFoldable
76+-- >>> fromIndices f = fmap f $ Stream.enumerateFrom 0
77+--
78+-- Also see "Streamly.Internal.Data.Stream.StreamDK" module for many more @Pre-release@
79+-- combinators. See the <https://github.com/composewell/streamly-examples>
80+-- repository for many more real world examples of stream programming.
81+--
82+-- == Performance Notes
83+--
84+-- Operations annotated as /CPS/ force the use of continuation passing style
85+-- streams. Therefore, such operations are not subjected to stream fusion.
86+-- However, for some of these operations you can find fusible alternatives in
87+-- the internal modules, which are perfectly fine to use but you need to
88+-- understand the implications especially the O(n^2) nature of those
89+-- operations..
90+--
91+module Streamly.Data.Stream.StreamDK
92+ (
93+ Stream
94+ -- * Construction
95+ -- | Functions ending in the general shape @b -> Stream m a@.
96+ --
97+ -- See also: "Streamly.Internal.Data.Stream.Generate" for
98+ -- @Pre-release@ functions.
99+
100+ -- ** Primitives
101+ -- | Primitives to construct a stream from pure values or monadic actions.
102+ -- All other stream construction and generation combinators described later
103+ -- can be expressed in terms of these primitives. However, the special
104+ -- versions provided in this module can be much more efficient in most
105+ -- cases. Users can create custom combinators using these primitives.
106+ , nil
107+ , nilM
108+ , cons
109+ , consM
110+ -- , cons2 -- fused version
111+ -- , consM2 -- fused version
112+
113+ -- ** Unfolding
114+ -- | 'unfoldrM' is the most general way of generating a stream efficiently.
115+ -- All other generation operations can be expressed using it.
116+ , unfoldr
117+ , unfoldrM
118+
119+ -- ** From Values
120+ -- | Generate a monadic stream from a seed value or values.
121+ , fromPure
122+ , fromEffect
123+ , repeat
124+ , repeatM
125+ , replicate
126+ , replicateM
127+
128+ -- Note: Using enumeration functions e.g. 'Prelude.enumFromThen' turns out
129+ -- to be slightly faster than the idioms like @[from, then..]@.
130+ --
131+ -- ** Enumeration
132+ -- | We can use the 'Enum' type class to enumerate a type producing a list
133+ -- and then convert it to a stream:
134+ --
135+ -- @
136+ -- 'fromList' $ 'Prelude.enumFromThen' from then
137+ -- @
138+ --
139+ -- However, this is not particularly efficient.
140+ -- The 'Enumerable' type class provides corresponding functions that
141+ -- generate a stream instead of a list, efficiently.
142+
143+ , Enumerable (..)
144+ , enumerate
145+ , enumerateTo
146+
147+ -- ** Iteration
148+ , iterate
149+ , iterateM
150+
151+ -- ** From Containers
152+ -- | Convert an input structure, container or source into a stream. All of
153+ -- these can be expressed in terms of primitives.
154+ , fromList
155+ , fromFoldable
156+
157+ -- ** From Unfolds
158+ -- | Most of the above stream generation operations can also be expressed
159+ -- using the corresponding unfolds in the "Streamly.Data.Unfold" module.
160+ , unfold -- XXX rename to fromUnfold?
161+
162+ -- * Elimination
163+ -- | Functions ending in the general shape @Stream m a -> m b@ or @Stream m
164+ -- a -> m (b, Stream m a)@
165+ --
166+ -- See also: "Streamly.Internal.Data.Stream.Eliminate" for @Pre-release@
167+ -- functions.
168+
169+-- EXPLANATION: In imperative terms a fold can be considered as a loop over the stream
170+-- that reduces the stream to a single value.
171+-- Left and right folds both use a fold function @f@ and an identity element
172+-- @z@ (@zero@) to deconstruct a recursive data structure and reconstruct a
173+-- new data structure. The new structure may be a recursive construction (a
174+-- container) or a non-recursive single value reduction of the original
175+-- structure.
176+--
177+-- Both right and left folds are mathematical duals of each other, they are
178+-- functionally equivalent. Operationally, a left fold on a left associated
179+-- structure behaves exactly in the same way as a right fold on a right
180+-- associated structure. Similarly, a left fold on a right associated structure
181+-- behaves in the same way as a right fold on a left associated structure.
182+-- However, the behavior of a right fold on a right associated structure is
183+-- operationally different (even though functionally equivalent) than a left
184+-- fold on the same structure.
185+--
186+-- On right associated structures like Haskell @cons@ lists or Streamly
187+-- streams, a lazy right fold is naturally suitable for lazy recursive
188+-- reconstruction of a new structure, while a strict left fold is naturally
189+-- suitable for efficient reduction. In right folds control is in the hand of
190+-- the @puller@ whereas in left folds the control is in the hand of the
191+-- @pusher@.
192+--
193+-- The behavior of right and left folds are described in detail in the
194+-- individual fold's documentation. To illustrate the two folds for right
195+-- associated @cons@ lists:
196+--
197+-- > foldr :: (a -> b -> b) -> b -> [a] -> b
198+-- > foldr f z [] = z
199+-- > foldr f z (x:xs) = x `f` foldr f z xs
200+-- >
201+-- > foldl :: (b -> a -> b) -> b -> [a] -> b
202+-- > foldl f z [] = z
203+-- > foldl f z (x:xs) = foldl f (z `f` x) xs
204+--
205+-- @foldr@ is conceptually equivalent to:
206+--
207+-- > foldr f z [] = z
208+-- > foldr f z [x] = f x z
209+-- > foldr f z xs = foldr f (foldr f z (tail xs)) [head xs]
210+--
211+-- @foldl@ is conceptually equivalent to:
212+--
213+-- > foldl f z [] = z
214+-- > foldl f z [x] = f z x
215+-- > foldl f z xs = foldl f (foldl f z (init xs)) [last xs]
216+--
217+-- Left and right folds are duals of each other.
218+--
219+-- @
220+-- foldr f z xs = foldl (flip f) z (reverse xs)
221+-- foldl f z xs = foldr (flip f) z (reverse xs)
222+-- @
223+--
224+-- More generally:
225+--
226+-- @
227+-- foldr f z xs = foldl g id xs z where g k x = k . f x
228+-- foldl f z xs = foldr g id xs z where g x k = k . flip f x
229+-- @
230+--
231+
232+-- NOTE: Folds are inherently serial as each step needs to use the result of
233+-- the previous step. However, it is possible to fold parts of the stream in
234+-- parallel and then combine the results using a monoid.
235+
236+ -- ** Primitives
237+ -- Consuming a part of the stream and returning the rest. Functions
238+ -- ending in the general shape @Stream m a -> m (b, Stream m a)@
239+ , uncons
240+
241+ -- ** Folding
242+ -- XXX Need to have a general parse operation here which can be used to
243+ -- express all others.
244+ , fold -- XXX rename to run? We can have a Stream.run and Fold.run.
245+ -- XXX fold1 can be achieved using Monoids or Refolds.
246+ -- XXX We can call this just "break" and parseBreak as "munch"
247+ , foldBreak
248+
249+ -- XXX should we have a Fold returning function in stream module?
250+ -- , foldAdd
251+ -- , buildl
252+
253+ -- ** Parsing
254+ , parse
255+ , parseBreak
256+
257+ -- -- ** Lazy Right Folds
258+ -- Consuming a stream to build a right associated expression, suitable
259+ -- for lazy evaluation. Evaluation of the input happens when the output of
260+ -- the fold is evaluated, the fold output is a lazy thunk.
261+ --
262+ -- This is suitable for stream transformation operations, for example,
263+ -- operations like mapping a function over the stream.
264+ -- , foldrM
265+ -- , foldr
266+
267+ -- * Mapping
268+ -- | Stateless one-to-one transformations. Use 'fmap' for mapping a pure
269+ -- function on a stream.
270+
271+ -- EXPLANATION:
272+ -- In imperative terms a map operation can be considered as a loop over
273+ -- the stream that transforms the stream into another stream by performing
274+ -- an operation on each element of the stream.
275+ --
276+ -- 'map' is the least powerful transformation operation with strictest
277+ -- guarantees. A map, (1) is a stateless loop which means that no state is
278+ -- allowed to be carried from one iteration to another, therefore,
279+ -- operations on different elements are guaranteed to not affect each
280+ -- other, (2) is a strictly one-to-one transformation of stream elements
281+ -- which means it guarantees that no elements can be added or removed from
282+ -- the stream, it can merely transform them.
283+ , sequence
284+ , mapM
285+ -- , trace -- XXX Use "tracing" map instead?
286+ , tap
287+ , delay
288+
289+ -- * Scanning
290+ -- | Stateful one-to-one transformations.
291+ --
292+ -- See also: "Streamly.Internal.Data.Stream.Transform" for
293+ -- @Pre-release@ functions.
294+ , scan
295+ , postscan
296+ -- XXX postscan1 can be implemented using Monoids or Refolds.
297+ -- Indexing can be considered as a special type of zipping where we zip a
298+ -- stream with an index stream.
299+ , indexed
300+
301+ -- * Insertion
302+ -- | Add elements to the stream.
303+
304+ -- Inserting elements is a special case of interleaving/merging streams.
305+ , insertBy
306+ , intersperseM
307+ , intersperseM_
308+ , intersperse
309+
310+ -- * Filtering
311+ -- | Remove elements from the stream.
312+
313+ -- ** Stateless Filters
314+ -- | 'mapMaybeM' is the most general stateless filtering operation. All
315+ -- other filtering operations can be expressed using it.
316+
317+ -- EXPLANATION:
318+ -- In imperative terms a filter over a stream corresponds to a loop with a
319+ -- @continue@ clause for the cases when the predicate fails.
320+
321+ , mapMaybe
322+ , mapMaybeM
323+ , filter
324+ , filterM
325+
326+ -- Filter and concat
327+ , catMaybes
328+ , catLefts
329+ , catRights
330+ , catEithers
331+
332+ -- ** Stateful Filters
333+ -- | 'scanMaybe' is the most general stateful filtering operation. The
334+ -- filtering folds (folds returning a 'Maybe' type) in
335+ -- "Streamly.Internal.Data.Fold" can be used along with 'scanMaybe' to
336+ -- perform stateful filtering operations in general.
337+ , scanMaybe
338+ , take
339+ , takeWhile
340+ , takeWhileM
341+ , drop
342+ , dropWhile
343+ , dropWhileM
344+
345+ -- XXX These are available as scans in folds. We need to check the
346+ -- performance though. If these are common and we need convenient stream
347+ -- ops then we can expose these.
348+
349+ -- , deleteBy
350+ -- , uniq
351+ -- , uniqBy
352+
353+ -- -- ** Sampling
354+ -- , strideFromThen
355+
356+ -- -- ** Searching
357+ -- Finding the presence or location of an element, a sequence of elements
358+ -- or another stream within a stream.
359+
360+ -- -- ** Searching Elements
361+ -- , findIndices
362+ -- , elemIndices
363+
364+ -- * Combining Two Streams
365+ -- ** Appending
366+ , append
367+
368+ -- ** Interleaving
369+ , interleave
370+ -- , interleave2
371+
372+ -- ** Merging
373+ -- | Merging of @n@ streams can be performed by combining the streams pair
374+ -- wise using 'mergeMapWith' to give O(n * log n) time complexity. If used
375+ -- with 'concatMapWith' it will have O(n^2) performance.
376+
377+ , mergeBy
378+ , mergeByM
379+ -- , mergeBy2
380+ -- , mergeByM2
381+
382+ -- ** Zipping
383+ -- | Zipping of @n@ streams can be performed by combining the streams pair
384+ -- wise using 'mergeMapWith' with O(n * log n) time complexity. If used
385+ -- with 'concatMapWith' it will have O(n^2) performance.
386+ , zipWith
387+ , zipWithM
388+ -- , zipWith2
389+ -- , zipWithM2
390+ , ZipStream (..)
391+
392+ -- ** Cross Product
393+ -- XXX The argument order in this operation is such that it seems we are
394+ -- transforming the first stream using the second stream because the second
395+ -- stream is evaluated many times or buffered and better be finite, first
396+ -- stream could potentially be infinite. In the tradition of using the
397+ -- transformed stream at the end we can have a flipped version called
398+ -- "crossMap" or "nestWith".
399+ , crossWith
400+ -- , cross
401+ -- , joinInner
402+ , CrossStream (..)
403+
404+ -- * Unfold Each
405+ , unfoldMany
406+ , intercalate
407+ , intercalateSuffix
408+
409+ -- * Stream of streams
410+ -- | Stream operations like map and filter represent loop processing in
411+ -- imperative programming terms. Similarly, the imperative concept of
412+ -- nested loops are represented by streams of streams. The 'concatMap'
413+ -- operation represents nested looping.
414+ -- A 'concatMap' operation loops over the input stream and then for each
415+ -- element of the input stream generates another stream and then loops over
416+ -- that inner stream as well producing effects and generating a single
417+ -- output stream.
418+ --
419+ -- One dimension loops are just a special case of nested loops. For
420+ -- example, 'concatMap' can degenerate to a simple map operation:
421+ --
422+ -- > map f m = S.concatMap (\x -> S.fromPure (f x)) m
423+ --
424+ -- Similarly, 'concatMap' can perform filtering by mapping an element to a
425+ -- 'nil' stream:
426+ --
427+ -- > filter p m = S.concatMap (\x -> if p x then S.fromPure x else S.nil) m
428+ --
429+
430+ , concatEffect
431+ , concatMapWith
432+ , concatMap
433+ , concatMapM
434+ , mergeMapWith
435+
436+ -- * Repeated Fold
437+ , foldMany -- XXX Rename to foldRepeat
438+ , parseMany
439+ , chunksOf
440+
441+ -- * Buffered Operations
442+ -- | Operations that require buffering of the stream.
443+ -- Reverse is essentially a left fold followed by an unfold.
444+ , reverse
445+ , sortBy
446+
447+ -- * Multi-Stream folds
448+ -- | Operations that consume multiple streams at the same time.
449+ , eqBy
450+ , cmpBy
451+ , isPrefixOf
452+ , isSubsequenceOf
453+
454+ -- trimming sequences
455+ , stripPrefix
456+
457+ -- Exceptions and resource management depend on the "exceptions" package
458+ -- XXX We can have IO Stream operations not depending on "exceptions"
459+ -- in Exception.Base
460+
461+ -- * Exceptions
462+ -- | Most of these combinators inhibit stream fusion, therefore, when
463+ -- possible, they should be called in an outer loop to mitigate the cost.
464+ -- For example, instead of calling them on a stream of chars call them on a
465+ -- stream of arrays before flattening it to a stream of chars.
466+ --
467+ -- See also: "Streamly.Internal.Data.Stream.Exception" for
468+ -- @Pre-release@ functions.
469+
470+ , onException
471+ , handle
472+
473+ -- * Resource Management
474+ -- | 'bracket' is the most general resource management operation, all other
475+ -- operations can be expressed using it. These functions have IO suffix
476+ -- because the allocation and cleanup functions are IO actions. For
477+ -- generalized allocation and cleanup functions see the functions without
478+ -- the IO suffix in the "streamly" package.
479+ , before
480+ , afterIO
481+ , finallyIO
482+ , bracketIO
483+ , bracketIO3
484+
485+ -- * Transforming Inner Monad
486+
487+ , morphInner
488+ , liftInner
489+ , runReaderT
490+ , runStateT
491+
492+ -- -- * Stream Types
493+ -- $serial
494+ -- , Interleave
495+ -- , Zip
496+ )
497+where
498+
499+import Streamly.Internal.Data.Stream.StreamDK
500+import Prelude
501+ hiding (filter, drop, dropWhile, take, takeWhile, zipWith, foldr,
502+ foldl, map, mapM, mapM_, sequence, all, any, sum, product, elem,
503+ notElem, maximum, minimum, head, last, tail, length, null,
504+ reverse, iterate, init, and, or, lookup, foldr1, (!!),
505+ scanl, scanl1, repeat, replicate, concatMap, span)
core/src/Streamly/Data/Stream/Zip.hsnew file+16
@@ -0,0 +1,16 @@
1+-- |
2+-- Module : Streamly.Data.Stream.Zip
3+-- Copyright : (c) 2017 Composewell Technologies
4+--
5+-- License : BSD3
6+-- Maintainer : streamly@composewell.com
7+-- Stability : released
8+-- Portability : GHC
9+--
10+module Streamly.Data.Stream.Zip
11+ (
12+ ZipStream (..)
13+ )
14+where
15+
16+import Streamly.Internal.Data.Stream.Zip
core/src/Streamly/Data/StreamK.hs+19−49
@@ -8,30 +8,18 @@
88-- Stability : released
99-- Portability : GHC
1010--
11−-- Streams represented as chains of functions calls using Continuation Passing
12−-- Style (CPS), suitable for dynamically composing potentially large number of
13−-- streams.
14−--
15−-- Unlike the statically fused operations in "Streamly.Data.Stream", StreamK
16−-- operations are less efficient, involving a function call overhead for each
17−-- element, but they exhibit linear O(n) time complexity wrt to the number of
18−-- stream compositions. Therefore, they are suitable for dynamically composing
19−-- streams e.g. appending potentially infinite streams in recursive loops.
20−-- While fused streams can be used to efficiently process elements as small as
21−-- a single byte, CPS streams are typically used on bigger chunks of data to
22−-- avoid the larger overhead per element. For more details See the @Stream vs
23−-- StreamK@ section in the "Streamly.Data.Stream" module.
24−--
25−-- In addition to the combinators in this module, you can use operations from
26−-- "Streamly.Data.Stream" for StreamK as well by converting StreamK to Stream
27−-- ('toStream'), and vice-versa ('fromStream'). Please refer to
28−-- "Streamly.Internal.Data.StreamK" for more functions that have not yet been
29−-- released.
30−--
31−-- For documentation see the corresponding combinators in
32−-- "Streamly.Data.Stream". Documentation has been omitted in this module unless
33−-- there is a difference worth mentioning or if the combinator does not exist
34−-- in "Streamly.Data.Stream".
11+-- Streams using Continuation Passing Style (CPS). See the @Stream vs StreamK@
12+-- section in the "Streamly.Data.Stream" module to know when to use this
13+-- module.
14+--
15+-- Please refer to "Streamly.Internal.Data.StreamK" for more functions
16+-- that have not yet been released.
17+--
18+-- Some useful idioms:
19+--
20+-- >>> concatFoldableWith f = Prelude.foldr f StreamK.nil
21+-- >>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
22+-- >>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
3523
3624-- Notes:
3725--
@@ -77,8 +65,6 @@ module Streamly.Data.StreamK
7765 , fromEffect
7866
7967 -- ** From Stream
80− -- | Please note that 'Stream' type does not observe any exceptions from
81− -- the consumer of the stream whereas 'StreamK' does.
8268 , fromStream
8369 , toStream
8470
@@ -95,8 +81,7 @@ module Streamly.Data.StreamK
9581 -- , foldBreak
9682
9783 -- ** Parsing
98− , parse
99− , parseBreak
84+ -- , parseBreak
10085 , parseBreakChunks
10186 , parseChunks
10287
@@ -106,11 +91,6 @@ module Streamly.Data.StreamK
10691 , take
10792
10893 -- * Combining Two Streams
109− -- | Unlike the operations in "Streamly.Data.Stream", these operations can
110− -- be used to dynamically compose large number of streams e.g. using the
111− -- 'concatMapWith' and 'mergeMapWith' operations. They have a linear O(n)
112− -- time complexity wrt to the number of streams being composed.
113−
11494 -- ** Appending
11595 , append
11696
@@ -134,12 +114,6 @@ module Streamly.Data.StreamK
134114 -- , CrossStreamK (..)
135115
136116 -- * Stream of streams
137− -- | Some useful idioms:
138− --
139− -- >>> concatFoldableWith f = Prelude.foldr f StreamK.nil
140− -- >>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
141− -- >>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
142− --
143117 , concatEffect
144118 -- , concatMap
145119 , concatMapWith
@@ -148,16 +122,6 @@ module Streamly.Data.StreamK
148122 -- * Buffered Operations
149123 , reverse
150124 , sortBy
151−
152− -- * Exceptions
153− -- | Please note that 'Stream' type does not observe any exceptions from
154− -- the consumer of the stream whereas 'StreamK' does.
155− , handle
156−
157− -- * Resource Management
158− -- | Please note that 'Stream' type does not observe any exceptions from
159− -- the consumer of the stream whereas 'StreamK' does.
160− , bracketIO
161125 )
162126where
163127
@@ -192,3 +156,9 @@ import Prelude hiding (reverse, zipWith, mapM, dropWhile, take)
192156-- 1
193157-- 2
194158-- [1,2]
159+--
160+-- == Exception Handling
161+--
162+-- There are no native exception handling operations in the StreamK module,
163+-- please convert to 'Stream' type and use exception handling operations from
164+-- "Streamly.Data.Stream".
core/src/Streamly/Data/Unfold.hs−2
@@ -65,8 +65,6 @@ module Streamly.Data.Unfold
6565 -- ** Mapping on Input
6666 , lmap
6767 , lmapM
68− , first
69− , second
7068
7169 -- ** Mapping on Output
7270 , mapM
core/src/Streamly/FileSystem/Handle.hs+29−15
@@ -1,4 +1,4 @@
1−{-# LANGUAGE CPP #-}
1+#include "inline.hs"
22
33-- |
44-- Module : Streamly.FileSystem.Handle
@@ -40,12 +40,6 @@
4040--
4141module Streamly.FileSystem.Handle
4242 (
43− -- * Setup
44− -- | To execute the code examples provided in this module in ghci, please
45− -- run the following commands first.
46− --
47− -- $setup
48−
4943 -- * Singleton IO
5044 -- | Read or write a single buffer.
5145 getChunk
@@ -81,13 +75,13 @@ module Streamly.FileSystem.Handle
8175 -- position of the file handle. The stream ends as soon as EOF is
8276 -- encountered.
8377
84− -- *** Streams
85− , read
86− , readWith
87− , readChunks
88− , readChunksWith
78+ -- -- *** Streams
79+ -- , read
80+ -- , readWith
81+ -- , readChunks
82+ -- , readChunksWith
8983
90− -- *** Unfolds
84+ -- -- *** Unfolds
9185 , reader
9286 , readerWith
9387 , chunkReader
@@ -104,14 +98,34 @@ module Streamly.FileSystem.Handle
10498 , writeChunks
10599
106100 -- * Deprecated
101+ , read
107102 , readWithBufferOf
103+ , readChunks
108104 , readChunksWithBufferOf
109105 , writeChunksWithBufferOf
110106 , writeWithBufferOf
111107 )
112108where
113109
114−import Streamly.Internal.FileSystem.Handle
110+import Control.Monad.IO.Class (MonadIO(..))
111+import Data.Word (Word8)
112+import Streamly.Internal.Data.Array.Type (Array)
113+import Streamly.Internal.Data.Unfold.Type (Unfold)
114+import System.IO (Handle)
115+
116+import Streamly.Internal.FileSystem.Handle hiding (read, readChunks)
115117import Prelude hiding (read)
116118
117−#include "DocTestFileSystemHandle.hs"
119+-- | Same as 'reader'
120+--
121+{-# DEPRECATED read "Please use 'reader' instead" #-}
122+{-# INLINE read #-}
123+read :: MonadIO m => Unfold m Handle Word8
124+read = reader
125+
126+-- | Same as 'chunkReader'
127+--
128+{-# DEPRECATED readChunks "Please use 'chunkReader' instead" #-}
129+{-# INLINE readChunks #-}
130+readChunks :: MonadIO m => Unfold m Handle (Array Word8)
131+readChunks = chunkReader
core/src/Streamly/Internal/Data/Array.hs+164−611
@@ -2,6 +2,7 @@
22-- |
33-- Module : Streamly.Internal.Data.Array
44-- Copyright : (c) 2019 Composewell Technologies
5+--
56-- License : BSD3
67-- Maintainer : streamly@composewell.com
78-- Stability : experimental
@@ -16,33 +17,59 @@ module Streamly.Internal.Data.Array
1617 -- $design
1718
1819 -- * The Array Type
19− module Streamly.Internal.Data.Array.Type
20+ Array
2021
2122 -- * Construction
23+
24+ -- Pure List APIs
25+ , A.fromListN
26+ , A.fromList
27+
28+ -- Stream Folds
29+ , fromStreamN
30+ , fromStream
31+
2232 -- Monadic Folds
33+ , A.writeN -- drop new
34+ , A.pinnedWriteNAligned
35+ , A.write -- full buffer
2336 , writeLastN
2437
38+ -- * Elimination
39+ -- ** Conversion
40+ , A.toList
41+
42+ -- ** Streams
43+ , A.read
44+ , A.readRev
45+
46+ -- ** Unfolds
47+ , reader
48+ , readerUnsafe
49+ , A.readerRev
50+ , producer -- experimental
51+
2552 -- * Random Access
2653 -- , (!!)
2754 , getIndex
55+ , A.unsafeIndex -- XXX Rename to getIndexUnsafe??
2856 , getIndexRev
29− , last -- XXX getLastIndex?
57+ , last -- XXX getIndexLast?
58+ , getIndices
59+ , getIndicesFromThenTo
3060 -- , getIndicesFrom -- read from a given position to the end of file
3161 -- , getIndicesUpto -- read from beginning up to the given position
3262 -- , getIndicesFromTo
3363 -- , getIndicesFromRev -- read from a given position to the beginning of file
3464 -- , getIndicesUptoRev -- read from end to the given position in file
35− , indexReader
36− , indexReaderFromThenTo
3765
3866 -- * Size
67+ , length
3968 , null
4069
4170 -- * Search
4271 , binarySearch
4372 , findIndicesOf
44− -- getIndicesOf
45− , indexFinder -- see splitOn
4673 -- , findIndexOf
4774 -- , find
4875
@@ -50,96 +77,66 @@ module Streamly.Internal.Data.Array
5077 , cast
5178 , asBytes
5279 , castUnsafe
80+ , asPtrUnsafe
5381 , asCStringUnsafe
82+ , A.unsafeFreeze -- asImmutableUnsafe?
83+ , A.unsafeThaw -- asMutableUnsafe?
5484
5585 -- * Subarrays
5686 , getSliceUnsafe
5787 -- , getSlice
58− , sliceIndexerFromLen
59− , slicerFromLen
88+ , genSlicesFromLen
89+ , getSlicesFromLen
6090 , splitOn
6191
6292 -- * Streaming Operations
6393 , streamTransform
6494
65− -- * Folding
95+ -- ** Folding
6696 , streamFold
6797 , fold
6898
69− -- * Stream of Arrays
70−
71− -- XXX these are probably not very useful to have in this module as we can
72− -- express these idiomatically using streams.
73− , interpose
74− , interposeSuffix
75− , intercalateSuffix
76−
77− , compactLE
78− , pinnedCompactLE
79− , compactOnByte
80− , compactOnByteSuffix
81−
82− , foldBreakChunks
83− , foldChunks
84− , foldBreakChunksK
85− , parseBreakChunksK
86−
87− -- * Serialization
88− , encodeAs
89− , serialize
90− , pinnedSerialize
91− , deserialize
92−
9399 -- * Deprecated
94− , genSlicesFromLen
95− , getSlicesFromLen
96− , getIndices
100+ , A.toStream
101+ , A.toStreamRev
97102 )
98103where
99104
100−#include "assert.hs"
101105#include "inline.hs"
102106#include "ArrayMacros.h"
103107
108+import Control.Exception (assert)
109+import Control.Monad (when)
104110import Control.Monad.IO.Class (MonadIO(..))
105−-- import Data.Bifunctor (first)
106−-- import Data.Either (fromRight)
107111import Data.Functor.Identity (Identity)
108112import Data.Proxy (Proxy(..))
109113import Data.Word (Word8)
110114import Foreign.C.String (CString)
111115import Foreign.Ptr (castPtr)
112116import Foreign.Storable (Storable)
113−import GHC.Types (SPEC(..))
114117import Streamly.Internal.Data.Unbox (Unbox(..))
115118import Prelude hiding (length, null, last, map, (!!), read, concat)
116119
117−import Streamly.Internal.Data.MutByteArray.Type (PinnedState(..), MutByteArray)
118−import Streamly.Internal.Data.Serialize.Type (Serialize)
120+import Streamly.Internal.Data.Array.Mut.Type (ArrayUnsafe(..))
121+import Streamly.Internal.Data.Array.Type
122+ (Array(..), length, asPtrUnsafe)
119123import Streamly.Internal.Data.Fold.Type (Fold(..))
120−import Streamly.Internal.Data.Parser (Parser(..), Initial(..), ParseError(..))
121−import Streamly.Internal.Data.Stream (Stream(..))
122−import Streamly.Internal.Data.StreamK (StreamK)
123−import Streamly.Internal.Data.SVar.Type (adaptState, defState)
124−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3Fused'(..))
124+import Streamly.Internal.Data.Producer.Type (Producer(..))
125+import Streamly.Internal.Data.Stream (Stream)
126+import Streamly.Internal.Data.Tuple.Strict (Tuple3Fused'(..))
125127import Streamly.Internal.Data.Unfold.Type (Unfold(..))
126128import Streamly.Internal.System.IO (unsafeInlineIO)
127129
128−import qualified Streamly.Internal.Data.Fold.Type as Fold
129−import qualified Streamly.Internal.Data.Serialize.Type as Serialize
130−import qualified Streamly.Internal.Data.MutByteArray.Type as MBA
131−import qualified Streamly.Internal.Data.MutArray as MA
130+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
131+import qualified Streamly.Internal.Data.Array.Mut as MA
132+import qualified Streamly.Internal.Data.Array.Type as A
132133import qualified Streamly.Internal.Data.Fold as FL
133−import qualified Streamly.Internal.Data.Ring as RB
134−import qualified Streamly.Internal.Data.Parser as Parser
135−-- import qualified Streamly.Internal.Data.ParserK as ParserK
134+import qualified Streamly.Internal.Data.Producer.Type as Producer
135+import qualified Streamly.Internal.Data.Producer as Producer
136+import qualified Streamly.Internal.Data.Ring.Unboxed as RB
136137import qualified Streamly.Internal.Data.Stream as D
137138import qualified Streamly.Internal.Data.Stream as Stream
138−import qualified Streamly.Internal.Data.StreamK as StreamK
139139import qualified Streamly.Internal.Data.Unfold as Unfold
140−import qualified Prelude
141−
142−import Streamly.Internal.Data.Array.Type
143140
144141#include "DocTestDataArray.hs"
145142
@@ -166,18 +163,91 @@ import Streamly.Internal.Data.Array.Type
166163-- if we use one array for each chunk. This is still significant to add
167164-- pressure to GC.
168165
166+-------------------------------------------------------------------------------
167+-- Construction
168+-------------------------------------------------------------------------------
169+
170+-- | Create an 'Array' from the first N elements of a stream. The array is
171+-- allocated to size N, if the stream terminates before N elements then the
172+-- array may hold less than N elements.
173+--
174+-- /Pre-release/
175+{-# INLINE fromStreamN #-}
176+fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
177+fromStreamN n m = do
178+ when (n < 0) $ error "writeN: negative write count specified"
179+ A.fromStreamDN n m
180+
181+-- | Create an 'Array' from a stream. This is useful when we want to create a
182+-- single array from a stream of unknown size. 'writeN' is at least twice
183+-- as efficient when the size is already known.
184+--
185+-- Note that if the input stream is too large memory allocation for the array
186+-- may fail. When the stream size is not known, `chunksOf` followed by
187+-- processing of indvidual arrays in the resulting stream should be preferred.
188+--
189+-- /Pre-release/
190+{-# INLINE fromStream #-}
191+fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
192+fromStream = Stream.fold A.write
193+-- write m = A.fromStreamD $ D.fromStreamK m
194+
169195-------------------------------------------------------------------------------
170196-- Elimination
171197-------------------------------------------------------------------------------
172198
199+{-# INLINE_NORMAL producer #-}
200+producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a
201+producer =
202+ Producer.translate A.unsafeThaw A.unsafeFreeze
203+ $ MA.producerWith (return . unsafeInlineIO)
204+
205+-- | Unfold an array into a stream.
206+--
207+{-# INLINE_NORMAL reader #-}
208+reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
209+reader = Producer.simplify producer
210+
211+-- | Unfold an array into a stream, does not check the end of the array, the
212+-- user is responsible for terminating the stream within the array bounds. For
213+-- high performance application where the end condition can be determined by
214+-- a terminating fold.
215+--
216+-- Written in the hope that it may be faster than "read", however, in the case
217+-- for which this was written, "read" proves to be faster even though the core
218+-- generated with unsafeRead looks simpler.
219+--
220+-- /Pre-release/
221+--
222+{-# INLINE_NORMAL readerUnsafe #-}
223+readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
224+readerUnsafe = Unfold step inject
225+ where
226+
227+ inject (Array contents start end) =
228+ return (ArrayUnsafe contents end start)
229+
230+ {-# INLINE_LATE step #-}
231+ step (ArrayUnsafe contents end p) = do
232+ -- unsafeInlineIO allows us to run this in Identity monad for pure
233+ -- toList/foldr case which makes them much faster due to not
234+ -- accumulating the list and fusing better with the pure consumers.
235+ --
236+ -- This should be safe as the array contents are guaranteed to be
237+ -- evaluated/written to before we peek at them.
238+ let !x = unsafeInlineIO $ peekByteIndex p contents
239+ let !p1 = INDEX_NEXT(p,a)
240+ return $ D.Yield x (ArrayUnsafe contents end p1)
241+
173242-- |
174243--
244+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
175245-- >>> null arr = Array.byteLength arr == 0
176246--
177247-- /Pre-release/
178248{-# INLINE null #-}
179249null :: Array a -> Bool
180−null arr = byteLength arr == 0
250+null arr = A.byteLength arr == 0
181251
182252-- | Like 'getIndex' but indexes the array in reverse from the end.
183253--
@@ -189,11 +259,12 @@ getIndexRev i arr =
189259 $ do
190260 let elemPtr = RINDEX_OF(arrEnd arr, i, a)
191261 if i >= 0 && elemPtr >= arrStart arr
192− then Just <$> peekAt elemPtr (arrContents arr)
262+ then Just <$> peekByteIndex elemPtr (arrContents arr)
193263 else return Nothing
194264
195265-- |
196266--
267+-- >>> import qualified Streamly.Internal.Data.Array as Array
197268-- >>> last arr = Array.getIndexRev arr 0
198269--
199270-- /Pre-release/
@@ -205,8 +276,6 @@ last = getIndexRev 0
205276-- Folds with Array as the container
206277-------------------------------------------------------------------------------
207278
208−-- XXX We should generate this from Ring.
209−
210279-- | @writeLastN n@ folds a maximum of @n@ elements from the end of the input
211280-- stream to an 'Array'.
212281--
@@ -215,7 +284,7 @@ writeLastN ::
215284 (Storable a, Unbox a, MonadIO m) => Int -> Fold m a (Array a)
216285writeLastN n
217286 | n <= 0 = fmap (const mempty) FL.drain
218− | otherwise = unsafeFreeze <$> Fold step initial done done
287+ | otherwise = A.unsafeFreeze <$> Fold step initial done
219288
220289 where
221290
@@ -228,9 +297,11 @@ writeLastN n
228297 in fmap f $ liftIO $ RB.new n
229298
230299 done (Tuple3Fused' rb rh i) = do
231− arr <- MA.new n
232− -- XXX We should write a read unfold for ring.
233− foldFunc i rh MA.snocUnsafe arr rb
300+ arr <- liftIO $ MA.new n
301+ foldFunc i rh snoc' arr rb
302+
303+ -- XXX We should write a read unfold for ring.
304+ snoc' b a = liftIO $ MA.snocUnsafe b a
234305
235306 foldFunc i
236307 | i < n = RB.unsafeFoldRingM
@@ -256,26 +327,19 @@ binarySearch = undefined
256327-- compared to streams by using SIMD instructions.
257328-- We can also return a bit array instead.
258329
259−-- Can use SIMD.
260−
261330-- | Perform a linear search to find all the indices where a given element is
262331-- present in an array.
263332--
264333-- /Unimplemented/
265−indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int
266−indexFinder = undefined
267−
268−-- |
269−-- /Unimplemented/
270−findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int
271−findIndicesOf p = Stream.unfold (indexFinder p)
334+findIndicesOf :: (a -> Bool) -> Unfold Identity (Array a) Int
335+findIndicesOf = undefined
272336
273337{-
274338findIndexOf :: (a -> Bool) -> Array a -> Maybe Int
275−findIndexOf p = Unfold.fold Fold.one . Stream.unfold (indexFinder p)
339+findIndexOf p = Unfold.fold Fold.one . Stream.unfold (findIndicesOf p)
276340
277341find :: (a -> Bool) -> Array a -> Bool
278−find = Unfold.fold Fold.null . Stream.unfold (indexFinder p)
342+find = Unfold.fold Fold.null . Stream.unfold (findIndicesOf p)
279343-}
280344
281345-------------------------------------------------------------------------------
@@ -317,45 +381,29 @@ splitOn :: (Monad m, Unbox a) =>
317381 (a -> Bool) -> Array a -> Stream m (Array a)
318382splitOn predicate arr =
319383 fmap (\(i, len) -> getSliceUnsafe i len arr)
320− $ D.indexOnSuffix predicate (read arr)
321−
322−{-# INLINE sliceIndexerFromLen #-}
323−sliceIndexerFromLen :: forall m a. (Monad m, Unbox a)
324− => Int -- ^ from index
325− -> Int -- ^ length of the slice
326− -> Unfold m (Array a) (Int, Int)
327−sliceIndexerFromLen from len =
328− Unfold.lmap unsafeThaw (MA.sliceIndexerFromLen from len)
384+ $ D.sliceOnSuffix predicate (A.toStreamD arr)
329385
330−{-# DEPRECATED genSlicesFromLen "Please use sliceIndexerFromLen instead." #-}
331386{-# INLINE genSlicesFromLen #-}
332387genSlicesFromLen :: forall m a. (Monad m, Unbox a)
333388 => Int -- ^ from index
334389 -> Int -- ^ length of the slice
335390 -> Unfold m (Array a) (Int, Int)
336−genSlicesFromLen = sliceIndexerFromLen
391+genSlicesFromLen from len =
392+ Unfold.lmap A.unsafeThaw (MA.genSlicesFromLen from len)
337393
338394-- | Generate a stream of slices of specified length from an array, starting
339395-- from the supplied array index. The last slice may be shorter than the
340396-- requested length.
341397--
342398-- /Pre-release//
343−{-# INLINE slicerFromLen #-}
344−slicerFromLen :: forall m a. (Monad m, Unbox a)
345− => Int -- ^ from index
346− -> Int -- ^ length of the slice
347− -> Unfold m (Array a) (Array a)
348−slicerFromLen from len =
349− fmap unsafeFreeze
350− $ Unfold.lmap unsafeThaw (MA.slicerFromLen from len)
351−
352−{-# DEPRECATED getSlicesFromLen "Please use slicerFromLen instead." #-}
353399{-# INLINE getSlicesFromLen #-}
354400getSlicesFromLen :: forall m a. (Monad m, Unbox a)
355401 => Int -- ^ from index
356402 -> Int -- ^ length of the slice
357403 -> Unfold m (Array a) (Array a)
358−getSlicesFromLen = slicerFromLen
404+getSlicesFromLen from len =
405+ fmap A.unsafeFreeze
406+ $ Unfold.lmap A.unsafeThaw (MA.getSlicesFromLen from len)
359407
360408-------------------------------------------------------------------------------
361409-- Random reads and writes
@@ -373,7 +421,7 @@ getIndex i arr =
373421 $ do
374422 let elemPtr = INDEX_OF(arrStart arr, i, a)
375423 if i >= 0 && INDEX_VALID(elemPtr, arrEnd arr, a)
376− then Just <$> peekAt elemPtr (arrContents arr)
424+ then Just <$> peekByteIndex elemPtr (arrContents arr)
377425 else return Nothing
378426
379427-- | Given a stream of array indices, read the elements on those indices from
@@ -385,25 +433,19 @@ getIndex i arr =
385433-- @
386434-- read =
387435-- let u = lmap (\arr -> (0, length arr - 1)) Unfold.enumerateFromTo
388−-- in Unfold.lmap f (indexReader arr)
436+-- in Unfold.lmap f (getIndices arr)
389437--
390438-- readRev =
391439-- let i = length arr - 1
392−-- in Unfold.lmap f (indexReaderFromThenTo i (i - 1) 0)
440+-- in Unfold.lmap f (getIndicesFromThenTo i (i - 1) 0)
393441-- @
394442--
395443-- /Pre-release/
396−{-# INLINE indexReader #-}
397−indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
398−indexReader m =
399− let unf = MA.indexReaderWith (return . unsafeInlineIO) m
400− in Unfold.lmap unsafeThaw unf
401−
402−-- XXX DO NOT REMOVE, change the signature to use Stream instead of unfold
403−{-# DEPRECATED getIndices "Please use getIndices instead." #-}
404444{-# INLINE getIndices #-}
405445getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
406−getIndices = indexReader
446+getIndices m =
447+ let unf = MA.getIndicesD (return . unsafeInlineIO) m
448+ in Unfold.lmap A.unsafeThaw unf
407449
408450-- | Unfolds @(from, then, to, array)@ generating a finite stream whose first
409451-- element is the array value from the index @from@ and the successive elements
@@ -419,9 +461,9 @@ getIndices = indexReader
419461-- @
420462--
421463-- /Unimplemented/
422−{-# INLINE indexReaderFromThenTo #-}
423−indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a
424−indexReaderFromThenTo = undefined
464+{-# INLINE getIndicesFromThenTo #-}
465+getIndicesFromThenTo :: Unfold m (Int, Int, Int, Array a) a
466+getIndicesFromThenTo = undefined
425467
426468-------------------------------------------------------------------------------
427469-- Transform via stream operations
@@ -438,7 +480,7 @@ indexReaderFromThenTo = undefined
438480{-# INLINE runPipe #-}
439481runPipe :: (MonadIO m, Unbox a, Unbox b)
440482 => Pipe m a b -> Array a -> m (Array b)
441−runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (read arr)
483+runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (A.read arr)
442484-}
443485
444486-- XXX For transformations that cannot change the number of elements e.g. "map"
@@ -452,7 +494,7 @@ runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (read arr)
452494streamTransform :: forall m a b. (MonadIO m, Unbox a, Unbox b)
453495 => (Stream m a -> Stream m b) -> Array a -> m (Array b)
454496streamTransform f arr =
455− Stream.fold (createWith (length arr)) $ f (read arr)
497+ Stream.fold (A.writeWith (length arr)) $ f (A.read arr)
456498
457499-------------------------------------------------------------------------------
458500-- Casts
@@ -486,7 +528,7 @@ asBytes = castUnsafe
486528--
487529cast :: forall a b. (Unbox b) => Array a -> Maybe (Array b)
488530cast arr =
489− let len = byteLength arr
531+ let len = A.byteLength arr
490532 r = len `mod` SIZE_OF(b)
491533 in if r /= 0
492534 then Nothing
@@ -504,9 +546,9 @@ cast arr =
504546--
505547asCStringUnsafe :: Array a -> (CString -> IO b) -> IO b
506548asCStringUnsafe arr act = do
507− let arr1 = asBytes arr <> fromList [0]
508− -- unsafePinnedAsPtr makes sure the array is pinned
509− unsafePinnedAsPtr arr1 $ \ptr -> act (castPtr ptr)
549+ let arr1 = asBytes arr <> A.fromList [0]
550+ -- asPtrUnsafe makes sure the array is pinned
551+ asPtrUnsafe arr1 $ \ptr -> act (castPtr ptr)
510552
511553-------------------------------------------------------------------------------
512554-- Folds
@@ -519,500 +561,11 @@ asCStringUnsafe arr act = do
519561-- /Pre-release/
520562{-# INLINE fold #-}
521563fold :: forall m a b. (Monad m, Unbox a) => Fold m a b -> Array a -> m b
522−fold f arr = Stream.fold f (read arr)
564+fold f arr = Stream.fold f (A.read arr)
523565
524566-- | Fold an array using a stream fold operation.
525567--
526568-- /Pre-release/
527569{-# INLINE streamFold #-}
528570streamFold :: (Monad m, Unbox a) => (Stream m a -> m b) -> Array a -> m b
529−streamFold f arr = f (read arr)
530−
531−--------------------------------------------------------------------------------
532−-- Serialization
533−--------------------------------------------------------------------------------
534−
535−{-# INLINE encodeAs #-}
536−encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8
537−encodeAs ps a =
538− unsafeInlineIO $ do
539− let len = Serialize.addSizeTo 0 a
540− mbarr <- MBA.newBytesAs ps len
541− off <- Serialize.serializeAt 0 mbarr a
542− assertM(len == off)
543− pure $ Array mbarr 0 off
544−
545−-- |
546−-- Properties:
547−-- 1. Identity: @deserialize . serialize == id@
548−-- 2. Encoded equivalence: @serialize a == serialize a@
549−{-# INLINE serialize #-}
550−serialize :: Serialize a => a -> Array Word8
551−serialize = encodeAs Unpinned
552−
553−-- | Serialize a Haskell type to a pinned byte array. The array is allocated
554−-- using pinned memory so that it can be used directly in OS APIs for writing
555−-- to file or sending over the network.
556−--
557−-- Properties:
558−-- 1. Identity: @deserialize . pinnedSerialize == id@
559−-- 2. Encoded equivalence: @pinnedSerialize a == pinnedSerialize a@
560−{-# INLINE pinnedSerialize #-}
561−pinnedSerialize :: Serialize a => a -> Array Word8
562−pinnedSerialize = encodeAs Pinned
563−
564−-- | Decode a Haskell type from a byte array containing its serialized
565−-- representation.
566−{-# INLINE deserialize #-}
567−deserialize :: Serialize a => Array Word8 -> a
568−deserialize arr@(Array {..}) = unsafeInlineIO $ do
569− let lenArr = length arr
570− (off, val) <-
571− Serialize.deserializeAt arrStart arrContents (arrStart + lenArr)
572− assertM(off == arrStart + lenArr)
573− pure val
574−
575−-------------------------------------------------------------------------------
576−-- Streams of Arrays
577−-------------------------------------------------------------------------------
578−
579−-- TODO: efficiently compare two streams of arrays. Two streams can have chunks
580−-- of different sizes, we can handle that in the stream comparison abstraction.
581−-- This could be useful e.g. to fast compare whether two files differ.
582−
583−-- | Insert the given element between arrays and flatten.
584−--
585−-- >>> interpose x = Stream.interpose x Array.reader
586−--
587−{-# INLINE interpose #-}
588−interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
589−interpose x = D.interpose x reader
590−
591−data FlattenState s =
592− OuterLoop s
593− | InnerLoop s !MutByteArray !Int !Int
594−
595−-- | Insert the given element after each array and flatten. This is similar to
596−-- unlines.
597−--
598−-- >>> interposeSuffix x = Stream.interposeSuffix x Array.reader
599−--
600−{-# INLINE_NORMAL interposeSuffix #-}
601−interposeSuffix :: forall m a. (Monad m, Unbox a)
602− => a -> Stream m (Array a) -> Stream m a
603−-- This does not require MonadIO constraint.
604−-- interposeSuffix x = D.interposeSuffix x reader
605−interposeSuffix sep (D.Stream step state) = D.Stream step' (OuterLoop state)
606−
607− where
608−
609− {-# INLINE_LATE step' #-}
610− step' gst (OuterLoop st) = do
611− r <- step (adaptState gst) st
612− return $ case r of
613− D.Yield Array{..} s ->
614− D.Skip (InnerLoop s arrContents arrStart arrEnd)
615− D.Skip s -> D.Skip (OuterLoop s)
616− D.Stop -> D.Stop
617−
618− step' _ (InnerLoop st _ p end) | p == end =
619− return $ D.Yield sep $ OuterLoop st
620−
621− step' _ (InnerLoop st contents p end) = do
622− x <- pure $ unsafeInlineIO $ peekAt p contents
623− return $ D.Yield x (InnerLoop st contents (INDEX_NEXT(p,a)) end)
624−
625−-- | Insert the given array after each array and flatten.
626−--
627−-- >>> intercalateSuffix = Stream.intercalateSuffix Array.reader
628−--
629−{-# INLINE intercalateSuffix #-}
630−intercalateSuffix :: (Monad m, Unbox a)
631− => Array a -> Stream m (Array a) -> Stream m a
632−intercalateSuffix = D.intercalateSuffix reader
633−
634−-- | @compactLE n@ coalesces adjacent arrays in the input stream
635−-- only if the combined size would be less than or equal to n.
636−--
637−-- Generates unpinned arrays irrespective of the pinning status of input
638−-- arrays.
639−{-# INLINE_NORMAL compactLE #-}
640−compactLE :: (MonadIO m, Unbox a)
641− => Int -> Stream m (Array a) -> Stream m (Array a)
642−compactLE n stream =
643− D.map unsafeFreeze $ MA.compactLE n $ D.map unsafeThaw stream
644−
645−-- | Pinned version of 'compactLE'.
646−{-# INLINE_NORMAL pinnedCompactLE #-}
647−pinnedCompactLE :: (MonadIO m, Unbox a)
648− => Int -> Stream m (Array a) -> Stream m (Array a)
649−pinnedCompactLE n stream =
650− D.map unsafeFreeze $ MA.pinnedCompactLE n $ D.map unsafeThaw stream
651−
652−-- | Split a stream of arrays on a given separator byte, dropping the separator
653−-- and coalescing all the arrays between two separators into a single array.
654−--
655−{-# INLINE compactOnByte #-}
656−compactOnByte
657− :: (MonadIO m)
658− => Word8
659− -> Stream m (Array Word8)
660− -> Stream m (Array Word8)
661−compactOnByte byte =
662− fmap unsafeFreeze . MA.compactOnByte byte . fmap unsafeThaw
663−
664−-- | Like 'compactOnByte' considers the separator in suffix position instead of
665−-- infix position.
666−{-# INLINE compactOnByteSuffix #-}
667−compactOnByteSuffix
668− :: (MonadIO m)
669− => Word8
670− -> Stream m (Array Word8)
671− -> Stream m (Array Word8)
672−compactOnByteSuffix byte =
673− fmap unsafeFreeze . MA.compactOnByteSuffix byte . fmap unsafeThaw
674−
675−-------------------------------------------------------------------------------
676−-- Folding Streams of Arrays
677−-------------------------------------------------------------------------------
678−
679−-- XXX This should not be used for breaking a stream as the D.cons used in
680−-- reconstructing the stream could be very bad for performance. This can only
681−-- be useful in folding without breaking.
682−{-# INLINE_NORMAL foldBreakChunks #-}
683−foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) =>
684− Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
685−foldBreakChunks (Fold fstep initial _ final) stream@(Stream step state) = do
686− res <- initial
687− case res of
688− Fold.Partial fs -> go SPEC state fs
689− Fold.Done fb -> return $! (fb, stream)
690−
691− where
692−
693− {-# INLINE go #-}
694− go !_ st !fs = do
695− r <- step defState st
696− case r of
697− Stream.Yield (Array contents start end) s ->
698− let fp = Tuple' end contents
699− in goArray SPEC s fp start fs
700− Stream.Skip s -> go SPEC s fs
701− Stream.Stop -> do
702− b <- final fs
703− return (b, D.nil)
704−
705− goArray !_ s (Tuple' end _) !cur !fs
706− | cur == end = do
707− go SPEC s fs
708− goArray !_ st fp@(Tuple' end contents) !cur !fs = do
709− x <- liftIO $ peekAt cur contents
710− res <- fstep fs x
711− let next = INDEX_NEXT(cur,a)
712− case res of
713− Fold.Done b -> do
714− let arr = Array contents next end
715− return $! (b, D.cons arr (D.Stream step st))
716− Fold.Partial fs1 -> goArray SPEC st fp next fs1
717−
718−-- This may be more robust wrt fusion compared to unfoldMany?
719−
720−-- | Fold a stream of arrays using a 'Fold'. This is equivalent to the
721−-- following:
722−--
723−-- >>> foldChunks f = Stream.fold f . Stream.unfoldMany Array.reader
724−--
725−foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b
726−foldChunks f s = fmap fst (foldBreakChunks f s)
727−-- foldStream f = Stream.fold f . Stream.unfoldMany reader
728−
729−-- | Fold a stream of arrays using a 'Fold' and return the remaining stream.
730−--
731−-- The following alternative to this function allows composing the fold using
732−-- the parser Monad:
733−--
734−-- @
735−-- foldBreakStreamK f s =
736−-- fmap (first (fromRight undefined))
737−-- $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s
738−-- @
739−--
740−-- We can compare perf and remove this one or define it in terms of that.
741−--
742−foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) =>
743− Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
744−{-
745−foldBreakChunksK f s =
746− fmap (first (fromRight undefined))
747− $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s
748−-}
749−foldBreakChunksK (Fold fstep initial _ final) stream = do
750− res <- initial
751− case res of
752− Fold.Partial fs -> go fs stream
753− Fold.Done fb -> return (fb, stream)
754−
755− where
756−
757− {-# INLINE go #-}
758− go !fs st = do
759− let stop = (, StreamK.nil) <$> final fs
760− single a = yieldk a StreamK.nil
761− yieldk (Array contents start end) r =
762− let fp = Tuple' end contents
763− in goArray fs r fp start
764− in StreamK.foldStream defState yieldk single stop st
765−
766− goArray !fs st (Tuple' end _) !cur
767− | cur == end = do
768− go fs st
769− goArray !fs st fp@(Tuple' end contents) !cur = do
770− x <- liftIO $ peekAt cur contents
771− res <- fstep fs x
772− let next = INDEX_NEXT(cur,a)
773− case res of
774− Fold.Done b -> do
775− let arr = Array contents next end
776− return $! (b, StreamK.cons arr st)
777− Fold.Partial fs1 -> goArray fs1 st fp next
778−
779−{-
780−-- This can be generalized to any type provided it can be unfolded to a stream
781−-- and it can be combined using a semigroup operation.
782−--
783−{-# INLINE_NORMAL parseBreakD #-}
784−parseBreakD ::
785− forall m a b. (MonadIO m, MonadThrow m, Unbox a)
786− => PRD.Parser a m b
787− -> D.Stream m (Array.Array a)
788− -> m (b, D.Stream m (Array.Array a))
789−parseBreakD
790− (PRD.Parser pstep initial extract) stream@(D.Stream step state) = do
791−
792− res <- initial
793− case res of
794− PRD.IPartial s -> go SPEC state (List []) s
795− PRD.IDone b -> return (b, stream)
796− PRD.IError err -> throwM $ ParseError err
797−
798− where
799−
800− -- "backBuf" contains last few items in the stream that we may have to
801− -- backtrack to.
802− --
803− -- XXX currently we are using a dumb list based approach for backtracking
804− -- buffer. This can be replaced by a sliding/ring buffer using Data.Array.
805− -- That will allow us more efficient random back and forth movement.
806− go !_ st backBuf !pst = do
807− r <- step defState st
808− case r of
809− D.Yield (Array contents start end) s ->
810− gobuf SPEC s backBuf
811− (Tuple' end contents) start pst
812− D.Skip s -> go SPEC s backBuf pst
813− D.Stop -> do
814− b <- extract pst
815− return (b, D.nil)
816−
817− -- Use strictness on "cur" to keep it unboxed
818− gobuf !_ s backBuf (Tuple' end _) !cur !pst
819− | cur == end = do
820− go SPEC s backBuf pst
821− gobuf !_ s backBuf fp@(Tuple' end contents) !cur !pst = do
822− x <- liftIO $ peekByteIndex contents cur
823− pRes <- pstep pst x
824− let next = INDEX_NEXT(cur,a)
825− case pRes of
826− PR.Partial 0 pst1 ->
827− gobuf SPEC s (List []) fp next pst1
828− PR.Partial n pst1 -> do
829− assert (n <= Prelude.length (x:getList backBuf)) (return ())
830− let src0 = Prelude.take n (x:getList backBuf)
831− arr0 = A.fromListN n (Prelude.reverse src0)
832− arr1 = Array contents next end
833− src = arr0 <> arr1
834− let !(Array cont1 start end1) = src
835− fp1 = Tuple' end1 cont1
836− gobuf SPEC s (List []) fp1 start pst1
837− PR.Continue 0 pst1 ->
838− gobuf SPEC s (List (x:getList backBuf)) fp next pst1
839− PR.Continue n pst1 -> do
840− assert (n <= Prelude.length (x:getList backBuf)) (return ())
841− let (src0, buf1) = splitAt n (x:getList backBuf)
842− arr0 = A.fromListN n (Prelude.reverse src0)
843− arr1 = Array contents next end
844− src = arr0 <> arr1
845− let !(Array cont1 start end1) = src
846− fp1 = Tuple' end1 cont1
847− gobuf SPEC s (List buf1) fp1 start pst1
848− PR.Done 0 b -> do
849− let arr = Array contents next end
850− return (b, D.cons arr (D.Stream step s))
851− PR.Done n b -> do
852− assert (n <= Prelude.length (x:getList backBuf)) (return ())
853− let src0 = Prelude.take n (x:getList backBuf)
854− -- XXX create the array in reverse instead
855− arr0 = A.fromListN n (Prelude.reverse src0)
856− arr1 = Array contents next end
857− -- XXX Use StreamK to avoid adding arbitrary layers of
858− -- constructors every time.
859− str = D.cons arr0 (D.cons arr1 (D.Stream step s))
860− return (b, str)
861− PR.Error err -> throwM $ ParseError err
862−-}
863−
864−-- | Parse an array stream using the supplied 'Parser'. Returns the parse
865−-- result and the unconsumed stream. Throws 'ParseError' if the parse fails.
866−--
867−-- The following alternative to this function allows composing the parser using
868−-- the parser Monad:
869−--
870−-- >>> parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)
871−--
872−-- We can compare perf and remove this one or define it in terms of that.
873−--
874−-- /Internal/
875−--
876−{-# INLINE_NORMAL parseBreakChunksK #-}
877−parseBreakChunksK ::
878− forall m a b. (MonadIO m, Unbox a)
879− => Parser a m b
880− -> StreamK m (Array a)
881− -> m (Either ParseError b, StreamK m (Array a))
882−-- parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)
883−parseBreakChunksK (Parser pstep initial extract) stream = do
884− res <- initial
885− case res of
886− IPartial s -> go s stream []
887− IDone b -> return (Right b, stream)
888− IError err -> return (Left (ParseError err), stream)
889−
890− where
891−
892− -- "backBuf" contains last few items in the stream that we may have to
893− -- backtrack to.
894− --
895− -- XXX currently we are using a dumb list based approach for backtracking
896− -- buffer. This can be replaced by a sliding/ring buffer using Data.Array.
897− -- That will allow us more efficient random back and forth movement.
898− go !pst st backBuf = do
899− let stop = goStop pst backBuf -- (, K.nil) <$> extract pst
900− single a = yieldk a StreamK.nil
901− yieldk arr r = goArray pst backBuf r arr
902− in StreamK.foldStream defState yieldk single stop st
903−
904− -- Use strictness on "cur" to keep it unboxed
905− goArray !pst backBuf st (Array _ cur end) | cur == end = go pst st backBuf
906− goArray !pst backBuf st (Array contents cur end) = do
907− x <- liftIO $ peekAt cur contents
908− pRes <- pstep pst x
909− let next = INDEX_NEXT(cur,a)
910− case pRes of
911− Parser.Partial 0 s ->
912− goArray s [] st (Array contents next end)
913− Parser.Partial n s -> do
914− assert (n <= Prelude.length (x:backBuf)) (return ())
915− let src0 = Prelude.take n (x:backBuf)
916− arr0 = fromListN n (Prelude.reverse src0)
917− arr1 = Array contents next end
918− src = arr0 <> arr1
919− goArray s [] st src
920− Parser.Continue 0 s ->
921− goArray s (x:backBuf) st (Array contents next end)
922− Parser.Continue n s -> do
923− assert (n <= Prelude.length (x:backBuf)) (return ())
924− let (src0, buf1) = Prelude.splitAt n (x:backBuf)
925− arr0 = fromListN n (Prelude.reverse src0)
926− arr1 = Array contents next end
927− src = arr0 <> arr1
928− goArray s buf1 st src
929− Parser.Done 0 b -> do
930− let arr = Array contents next end
931− return (Right b, StreamK.cons arr st)
932− Parser.Done n b -> do
933− assert (n <= Prelude.length (x:backBuf)) (return ())
934− let src0 = Prelude.take n (x:backBuf)
935− -- XXX Use fromListRevN once implemented
936− -- arr0 = A.fromListRevN n src0
937− arr0 = fromListN n (Prelude.reverse src0)
938− arr1 = Array contents next end
939− str = StreamK.cons arr0 (StreamK.cons arr1 st)
940− return (Right b, str)
941− Parser.Error err -> do
942− let n = Prelude.length backBuf
943− arr0 = fromListN n (Prelude.reverse backBuf)
944− arr1 = Array contents cur end
945− str = StreamK.cons arr0 (StreamK.cons arr1 stream)
946− return (Left (ParseError err), str)
947−
948− -- This is a simplified goArray
949− goExtract !pst backBuf (Array _ cur end)
950− | cur == end = goStop pst backBuf
951− goExtract !pst backBuf (Array contents cur end) = do
952− x <- liftIO $ peekAt cur contents
953− pRes <- pstep pst x
954− let next = INDEX_NEXT(cur,a)
955− case pRes of
956− Parser.Partial 0 s ->
957− goExtract s [] (Array contents next end)
958− Parser.Partial n s -> do
959− assert (n <= Prelude.length (x:backBuf)) (return ())
960− let src0 = Prelude.take n (x:backBuf)
961− arr0 = fromListN n (Prelude.reverse src0)
962− arr1 = Array contents next end
963− src = arr0 <> arr1
964− goExtract s [] src
965− Parser.Continue 0 s ->
966− goExtract s backBuf (Array contents next end)
967− Parser.Continue n s -> do
968− assert (n <= Prelude.length (x:backBuf)) (return ())
969− let (src0, buf1) = Prelude.splitAt n (x:backBuf)
970− arr0 = fromListN n (Prelude.reverse src0)
971− arr1 = Array contents next end
972− src = arr0 <> arr1
973− goExtract s buf1 src
974− Parser.Done 0 b -> do
975− let arr = Array contents next end
976− return (Right b, StreamK.fromPure arr)
977− Parser.Done n b -> do
978− assert (n <= Prelude.length backBuf) (return ())
979− let src0 = Prelude.take n backBuf
980− -- XXX Use fromListRevN once implemented
981− -- arr0 = A.fromListRevN n src0
982− arr0 = fromListN n (Prelude.reverse src0)
983− arr1 = Array contents next end
984− str = StreamK.cons arr0 (StreamK.fromPure arr1)
985− return (Right b, str)
986− Parser.Error err -> do
987− let n = Prelude.length backBuf
988− arr0 = fromListN n (Prelude.reverse backBuf)
989− arr1 = Array contents cur end
990− str = StreamK.cons arr0 (StreamK.cons arr1 stream)
991− return (Left (ParseError err), str)
992−
993− -- This is a simplified goExtract
994− {-# INLINE goStop #-}
995− goStop !pst backBuf = do
996− pRes <- extract pst
997− case pRes of
998− Parser.Partial _ _ -> error "Bug: parseBreak: Partial in extract"
999− Parser.Continue 0 s ->
1000− goStop s backBuf
1001− Parser.Continue n s -> do
1002− assert (n <= Prelude.length backBuf) (return ())
1003− let (src0, buf1) = Prelude.splitAt n backBuf
1004− arr = fromListN n (Prelude.reverse src0)
1005− goExtract s buf1 arr
1006− Parser.Done 0 b ->
1007− return (Right b, StreamK.nil)
1008− Parser.Done n b -> do
1009− assert (n <= Prelude.length backBuf) (return ())
1010− let src0 = Prelude.take n backBuf
1011− -- XXX Use fromListRevN once implemented
1012− -- arr0 = A.fromListRevN n src0
1013− arr0 = fromListN n (Prelude.reverse src0)
1014− return (Right b, StreamK.fromPure arr0)
1015− Parser.Error err -> do
1016− let n = Prelude.length backBuf
1017− arr0 = fromListN n (Prelude.reverse backBuf)
1018− return (Left (ParseError err), StreamK.fromPure arr0)
571+streamFold f arr = f (A.read arr)
core/src/Streamly/Internal/Data/Array/Generic.hs+12−50
@@ -11,8 +11,8 @@ module Streamly.Internal.Data.Array.Generic
1111
1212 -- * Construction
1313 , nil
14− , createOf
15− , create
14+ , writeN
15+ , write
1616 , writeWith
1717 , writeLastN
1818
@@ -24,8 +24,6 @@ module Streamly.Internal.Data.Array.Generic
2424 , fromListN
2525 , fromList
2626
27− , chunksOf
28−
2927 -- * Elimination
3028 , length
3129 , reader
@@ -44,10 +42,6 @@ module Streamly.Internal.Data.Array.Generic
4442 , getIndex
4543 , getSliceUnsafe
4644 , strip
47−
48− -- * Deprecated
49− , writeN
50− , write
5145 )
5246where
5347
@@ -63,17 +57,17 @@ import GHC.IO (unsafePerformIO)
6357import Text.Read (readPrec)
6458
6559import Streamly.Internal.Data.Fold.Type (Fold(..))
66−import Streamly.Internal.Data.Stream.Type (Stream)
60+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
6761import Streamly.Internal.Data.Unfold.Type (Unfold(..))
6862import Streamly.Internal.System.IO (unsafeInlineIO)
6963
70−import qualified Streamly.Internal.Data.MutArray.Generic as MArray
64+import qualified Streamly.Internal.Data.Array.Generic.Mut.Type as MArray
7165import qualified Streamly.Internal.Data.Fold.Type as FL
7266import qualified Streamly.Internal.Data.Producer.Type as Producer
7367import qualified Streamly.Internal.Data.Producer as Producer
74−import qualified Streamly.Internal.Data.Ring.Generic as RB
75−import qualified Streamly.Internal.Data.Stream.Type as D
76−import qualified Streamly.Internal.Data.Stream.Generate as D
68+import qualified Streamly.Internal.Data.Ring as RB
69+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
70+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
7771import qualified Text.ParserCombinators.ReadPrec as ReadPrec
7872
7973import Prelude hiding (Foldable(..), read)
@@ -108,33 +102,21 @@ nil = unsafePerformIO $ unsafeFreeze <$> MArray.nil
108102-- Construction - Folds
109103-------------------------------------------------------------------------------
110104
111−{-# INLINE_NORMAL createOf #-}
112−createOf :: MonadIO m => Int -> Fold m a (Array a)
113−createOf = fmap unsafeFreeze <$> MArray.createOf
114−
115−-- XXX Deprecate in major
116−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
117−{-# INLINE writeN #-}
105+{-# INLINE_NORMAL writeN #-}
118106writeN :: MonadIO m => Int -> Fold m a (Array a)
119−writeN = createOf
107+writeN = fmap unsafeFreeze <$> MArray.writeN
120108
121109{-# INLINE_NORMAL writeWith #-}
122110writeWith :: MonadIO m => Int -> Fold m a (Array a)
123−writeWith elemCount = unsafeFreeze <$> MArray.createWith elemCount
111+writeWith elemCount = unsafeFreeze <$> MArray.writeWith elemCount
124112
125113-- | Fold the whole input to a single array.
126114--
127115-- /Caution! Do not use this on infinite streams./
128116--
129−{-# INLINE create #-}
130−create :: MonadIO m => Fold m a (Array a)
131−create = fmap unsafeFreeze MArray.create
132−
133−-- XXX Deprecate in major
134−-- {-# DEPRECATED write "Please use create instead." #-}
135117{-# INLINE write #-}
136118write :: MonadIO m => Fold m a (Array a)
137−write = create
119+write = fmap unsafeFreeze MArray.write
138120
139121fromPureStream :: Stream Identity a -> Array a
140122fromPureStream x =
@@ -145,15 +127,6 @@ fromPureStream x =
145127fromByteStr# :: Addr# -> Array Word8
146128fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
147129
148−-------------------------------------------------------------------------------
149−-- Stream Ops
150−-------------------------------------------------------------------------------
151−
152−{-# INLINE_NORMAL chunksOf #-}
153−chunksOf :: forall m a. MonadIO m
154− => Int -> Stream m a -> Stream m (Array a)
155−chunksOf n strm = fmap unsafeFreeze $ MArray.chunksOf n strm
156−
157130-------------------------------------------------------------------------------
158131-- Construction - from streams
159132-------------------------------------------------------------------------------
@@ -258,17 +231,6 @@ getIndex i arr@Array {..} =
258231 then Just $ getIndexUnsafe i arr
259232 else Nothing
260233
261−-- >>> import qualified Streamly.Data.Stream as Stream
262−-- >>> import qualified Streamly.Data.Fold as Fold
263−-- >>> import qualified Streamly.Internal.Data.Array.Generic as Array
264−-- >>> import Data.Function ((&))
265−-- >>> :{
266−-- Stream.fromList [1,2,3,4,5::Int]
267−-- & Stream.scan (Array.writeLastN 2)
268−-- & Stream.fold Fold.toList
269−-- :}
270−-- [fromList [],fromList [1],fromList [1,2],fromList [2,3],fromList [3,4],fromList [4,5]]
271−--
272234{-# INLINE writeLastN #-}
273235writeLastN :: MonadIO m => Int -> Fold m a (Array a)
274236writeLastN n = FL.rmapM f (RB.writeLastN n)
@@ -276,7 +238,7 @@ writeLastN n = FL.rmapM f (RB.writeLastN n)
276238 where
277239
278240 f rb = do
279− arr <- RB.copyToMutArray 0 n rb
241+ arr <- RB.toMutArray 0 n rb
280242 return $ unsafeFreeze arr
281243
282244{-# INLINE getSliceUnsafe #-}
core/src/Streamly/Internal/Data/MutArray/Generic.hs → core/src/Streamly/Internal/Data/Array/Generic/Mut/Type.hsrenamed+54−177
@@ -1,14 +1,14 @@
11{-# LANGUAGE CPP #-}
22{-# LANGUAGE UnboxedTuples #-}
33-- |
4−-- Module : Streamly.Internal.Data.MutArray.Generic
4+-- Module : Streamly.Internal.Data.Array.Generic.Mut.Type
55-- Copyright : (c) 2020 Composewell Technologies
66-- License : BSD3-3-Clause
77-- Maintainer : streamly@composewell.com
88-- Stability : experimental
99-- Portability : GHC
1010--
11−module Streamly.Internal.Data.MutArray.Generic
11+module Streamly.Internal.Data.Array.Generic.Mut.Type
1212(
1313 -- * Type
1414 -- $arrayNotes
@@ -19,14 +19,14 @@ module Streamly.Internal.Data.MutArray.Generic
1919 , nil
2020
2121 -- *** Uninitialized Arrays
22− , emptyOf
22+ , new
2323 -- , newArrayWith
2424
2525 -- *** From streams
26− , unsafeCreateOf
27− , createOf
28− , createWith -- createOfMin/createMin/createGE?
29− , create
26+ , writeNUnsafe
27+ , writeN
28+ , writeWith
29+ , write
3030 , fromStreamN
3131 , fromStream
3232 , fromPureStream
@@ -101,7 +101,6 @@ module Streamly.Internal.Data.MutArray.Generic
101101 -- ** Random reads
102102 , getIndex
103103 , getIndexUnsafe
104− , getIndexUnsafeWith
105104 -- , getIndices
106105 -- , getFromThenTo
107106 -- , getIndexRev
@@ -133,7 +132,7 @@ module Streamly.Internal.Data.MutArray.Generic
133132 -- multidimensional array representations.
134133
135134 -- ** Construct from streams
136− , chunksOf
135+ -- , chunksOf
137136 -- , arrayStreamKFromStreamD
138137 -- , writeChunks
139138
@@ -160,13 +159,6 @@ module Streamly.Internal.Data.MutArray.Generic
160159 -- , appendSliceFrom
161160
162161 , clone
163−
164− -- * Deprecated
165− , new
166− , writeNUnsafe
167− , writeN
168− , writeWith
169− , write
170162 )
171163where
172164
@@ -189,15 +181,14 @@ import GHC.Int (Int(..))
189181import Streamly.Internal.Data.Fold.Type (Fold(..))
190182import Streamly.Internal.Data.Producer.Type (Producer (..))
191183import Streamly.Internal.Data.Unfold.Type (Unfold(..))
192−import Streamly.Internal.Data.Stream.Type (Stream)
193−import Streamly.Internal.Data.SVar.Type (adaptState)
184+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
194185
195186import qualified Streamly.Internal.Data.Fold.Type as FL
196187import qualified Streamly.Internal.Data.Producer as Producer
197−import qualified Streamly.Internal.Data.Stream.Type as D
198−import qualified Streamly.Internal.Data.Stream.Generate as D
199−import qualified Streamly.Internal.Data.Stream.Lift as D
200−import qualified Streamly.Internal.Data.StreamK.Type as K
188+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
189+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
190+import qualified Streamly.Internal.Data.Stream.StreamD.Lift as D
191+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
201192
202193import Prelude hiding (read, length)
203194
@@ -238,18 +229,18 @@ bottomElement =
238229
239230 where
240231
241− funcName = "Streamly.Internal.Data.MutArray.Generic.bottomElement:"
232+ funcName = "Streamly.Internal.Data.Array.Generic.Mut.Type.bottomElement:"
242233
243234-- XXX Would be nice if GHC can provide something like newUninitializedArray# so
244235-- that we do not have to write undefined or error in the whole array.
245236
246−-- | @emptyOf count@ allocates a zero length array that can be extended to hold
237+-- | @new count@ allocates a zero length array that can be extended to hold
247238-- up to 'count' items without reallocating.
248239--
249240-- /Pre-release/
250−{-# INLINE emptyOf #-}
251−emptyOf :: MonadIO m => Int -> m (MutArray a)
252−emptyOf n@(I# n#) =
241+{-# INLINE new #-}
242+new :: MonadIO m => Int -> m (MutArray a)
243+new n@(I# n#) =
253244 liftIO
254245 $ IO
255246 $ \s# ->
@@ -258,12 +249,6 @@ emptyOf n@(I# n#) =
258249 let ma = MutArray arr# 0 0 n
259250 in (# s1#, ma #)
260251
261−-- XXX Deprecate in major
262−-- {-# DEPRECATED new "Please use emptyOf instead." #-}
263−{-# INLINE new #-}
264−new :: MonadIO m => Int -> m (MutArray a)
265−new = emptyOf
266−
267252-- XXX This could be pure?
268253
269254-- |
@@ -278,21 +263,6 @@ nil = new 0
278263-- Random writes
279264-------------------------------------------------------------------------------
280265
281−-- | Write the given element to the given index of the 'MutableArray#'. Does not
282−-- check if the index is out of bounds of the array.
283−--
284−-- /Pre-release/
285−{-# INLINE putIndexUnsafeWith #-}
286−putIndexUnsafeWith :: MonadIO m => Int -> MutableArray# RealWorld a -> a -> m ()
287−putIndexUnsafeWith n _arrContents# x =
288− liftIO
289− $ IO
290− $ \s# ->
291− case n of
292− I# n# ->
293− let s1# = writeArray# _arrContents# n# x s#
294− in (# s1#, () #)
295−
296266-- | Write the given element to the given index of the array. Does not check if
297267-- the index is out of bounds of the array.
298268--
@@ -301,7 +271,13 @@ putIndexUnsafeWith n _arrContents# x =
301271putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
302272putIndexUnsafe i MutArray {..} x =
303273 assert (i >= 0 && i < arrLen)
304− putIndexUnsafeWith (i + arrStart) arrContents# x
274+ (liftIO
275+ $ IO
276+ $ \s# ->
277+ case i + arrStart of
278+ I# n# ->
279+ let s1# = writeArray# arrContents# n# x s#
280+ in (# s1#, () #))
305281
306282invalidIndex :: String -> Int -> a
307283invalidIndex label i =
@@ -331,7 +307,7 @@ putIndices arr = FL.foldlM' step (return ())
331307
332308 where
333309
334− step () (i, x) = putIndex i arr x
310+ step () (i, x) = liftIO (putIndex i arr x)
335311
336312-- | Modify a given index of an array using a modifier function without checking
337313-- the bounds.
@@ -480,25 +456,17 @@ uninit arr@MutArray{..} len =
480456-- Random reads
481457-------------------------------------------------------------------------------
482458
483−-- | Return the element at the specified index without checking the bounds from
484−-- a @MutableArray# RealWorld@.
485−--
486−-- Unsafe because it does not check the bounds of the array.
487−{-# INLINE getIndexUnsafeWith #-}
488−getIndexUnsafeWith :: MonadIO m => MutableArray# RealWorld a -> Int -> m a
489−getIndexUnsafeWith _arrContents# n =
490− liftIO
491− $ IO
492− $ \s# ->
493− let !(I# i#) = n
494− in readArray# _arrContents# i# s#
495−
496459-- | Return the element at the specified index without checking the bounds.
497460--
498461-- Unsafe because it does not check the bounds of the array.
499462{-# INLINE_NORMAL getIndexUnsafe #-}
500463getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
501−getIndexUnsafe n MutArray {..} = getIndexUnsafeWith arrContents# (n + arrStart)
464+getIndexUnsafe n MutArray {..} =
465+ liftIO
466+ $ IO
467+ $ \s# ->
468+ let !(I# i#) = arrStart + n
469+ in readArray# arrContents# i# s#
502470
503471-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
504472--
@@ -602,16 +570,16 @@ readRev arr@MutArray{..} =
602570arrayChunkSize :: Int
603571arrayChunkSize = 1024
604572
605−-- | Like 'createOf' but does not check the array bounds when writing. The fold
573+-- | Like 'writeN' but does not check the array bounds when writing. The fold
606574-- driver must not call the step function more than 'n' times otherwise it will
607575-- corrupt the memory and crash. This function exists mainly because any
608576-- conditional in the step function blocks fusion causing 10x performance
609577-- slowdown.
610578--
611579-- /Pre-release/
612−{-# INLINE_NORMAL unsafeCreateOf #-}
613−unsafeCreateOf :: MonadIO m => Int -> Fold m a (MutArray a)
614−unsafeCreateOf n = Fold step initial return return
580+{-# INLINE_NORMAL writeNUnsafe #-}
581+writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
582+writeNUnsafe n = Fold step initial return
615583
616584 where
617585
@@ -619,81 +587,59 @@ unsafeCreateOf n = Fold step initial return return
619587
620588 step arr x = FL.Partial <$> snocUnsafe arr x
621589
622−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
623−{-# INLINE writeNUnsafe #-}
624−writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
625−writeNUnsafe = unsafeCreateOf
626−
627−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
590+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
628591-- 'Array'.
629592--
630−-- >>> createOf n = Fold.take n (MutArray.unsafeCreateOf n)
593+-- >>> writeN n = Fold.take n (MutArray.writeNUnsafe n)
631594--
632595-- /Pre-release/
633−{-# INLINE_NORMAL createOf #-}
634−createOf :: MonadIO m => Int -> Fold m a (MutArray a)
635−createOf n = FL.take n $ unsafeCreateOf n
636−
637−-- XXX Deprecate in major
638−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
639−{-# INLINE writeN #-}
596+{-# INLINE_NORMAL writeN #-}
640597writeN :: MonadIO m => Int -> Fold m a (MutArray a)
641−writeN = createOf
598+writeN n = FL.take n $ writeNUnsafe n
642599
643−-- >>> f n = MutArray.writeAppendWith (* 2) (MutArray.pinnedNew n)
600+-- >>> f n = MutArray.writeAppendWith (* 2) (MutArray.newPinned n)
644601-- >>> writeWith n = Fold.rmapM MutArray.rightSize (f n)
645602-- >>> writeWith n = Fold.rmapM MutArray.fromArrayStreamK (MutArray.writeChunks n)
646603
647−-- | @createWith minCount@ folds the whole input to a single array. The array
604+-- | @writeWith minCount@ folds the whole input to a single array. The array
648605-- starts at a size big enough to hold minCount elements, the size is doubled
649606-- every time the array needs to be grown.
650607--
651608-- /Caution! Do not use this on infinite streams./
652609--
653610-- /Pre-release/
654−{-# INLINE_NORMAL createWith #-}
655−createWith :: MonadIO m => Int -> Fold m a (MutArray a)
656−-- writeWith n = FL.rmapM rightSize $ writeAppendWith (* 2) (pinnedNew n)
657−createWith elemCount = FL.rmapM extract $ FL.foldlM' step initial
611+{-# INLINE_NORMAL writeWith #-}
612+writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
613+-- writeWith n = FL.rmapM rightSize $ writeAppendWith (* 2) (newPinned n)
614+writeWith elemCount = FL.rmapM extract $ FL.foldlM' step initial
658615
659616 where
660617
661618 initial = do
662− when (elemCount < 0) $ error "createWith: elemCount is negative"
663− new elemCount
619+ when (elemCount < 0) $ error "writeWith: elemCount is negative"
620+ liftIO $ new elemCount
664621
665622 step arr@(MutArray _ start end bound) x
666623 | end == bound = do
667624 let oldSize = end - start
668625 newSize = max (oldSize * 2) 1
669− arr1 <- realloc newSize arr
626+ arr1 <- liftIO $ realloc newSize arr
670627 snocUnsafe arr1 x
671628 step arr x = snocUnsafe arr x
672629
673− -- extract = rightSize
630+ -- extract = liftIO . rightSize
674631 extract = return
675632
676−{-# DEPRECATED writeWith "Please use createWith instead." #-}
677−{-# INLINE writeWith #-}
678−writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
679−writeWith = createWith
680−
681633-- | Fold the whole input to a single array.
682634--
683−-- Same as 'createWith' using an initial array size of 'arrayChunkSize' bytes
635+-- Same as 'writeWith' using an initial array size of 'arrayChunkSize' bytes
684636-- rounded up to the element size.
685637--
686638-- /Caution! Do not use this on infinite streams./
687639--
688−{-# INLINE create #-}
689−create :: MonadIO m => Fold m a (MutArray a)
690−create = writeWith arrayChunkSize
691−
692−-- XXX Deprecate in major
693−-- {-# DEPRECATED write "Please use create instead." #-}
694640{-# INLINE write #-}
695641write :: MonadIO m => Fold m a (MutArray a)
696−write = create
642+write = writeWith arrayChunkSize
697643
698644-- | Create a 'MutArray' from the first @n@ elements of a stream. The
699645-- array is allocated to size @n@, if the stream terminates before @n@
@@ -718,76 +664,7 @@ fromList xs = fromStream $ D.fromList xs
718664{-# INLINABLE fromPureStream #-}
719665fromPureStream :: MonadIO m => Stream Identity a -> m (MutArray a)
720666fromPureStream xs =
721− D.fold write $ D.morphInner (return . runIdentity) xs
722−
723−-------------------------------------------------------------------------------
724−-- Chunking
725−-------------------------------------------------------------------------------
726−
727−data GroupState s a start end bound
728− = GroupStart s
729− | GroupBuffer s (MutableArray# RealWorld a) start end bound
730− | GroupYield
731− (MutableArray# RealWorld a)
732− start
733− end
734− bound
735− (GroupState s a start end bound)
736− | GroupFinish
737−
738−-- | @chunksOf n stream@ groups the input stream into a stream of
739−-- arrays of size n.
740−--
741−-- @chunksOf n = foldMany (MutArray.writeN n)@
742−--
743−-- /Pre-release/
744−{-# INLINE_NORMAL chunksOf #-}
745−chunksOf :: forall m a. MonadIO m
746− => Int -> D.Stream m a -> D.Stream m (MutArray a)
747−-- XXX the idiomatic implementation leads to large regression in the D.reverse'
748−-- benchmark. It seems it has difficulty producing optimized code when
749−-- converting to StreamK. Investigate GHC optimizations.
750−-- chunksOf n = D.foldMany (writeN n)
751−chunksOf n (D.Stream step state) =
752− D.Stream step' (GroupStart state)
753−
754− where
755−
756− -- start is always 0
757− -- end and len are always equal
758−
759− {-# INLINE_LATE step' #-}
760− step' _ (GroupStart st) = do
761− when (n <= 0) $
762− -- XXX we can pass the module string from the higher level API
763− error $ "Streamly.Internal.Data.Array.Generic.Mut.Type.chunksOf: "
764− ++ "the size of arrays [" ++ show n
765− ++ "] must be a natural number"
766− (MutArray contents start end bound :: MutArray a) <- new n
767− return $ D.Skip (GroupBuffer st contents start end bound)
768−
769− step' gst (GroupBuffer st contents start end bound) = do
770− r <- step (adaptState gst) st
771− case r of
772− D.Yield x s -> do
773− putIndexUnsafeWith end contents x
774− let end1 = end + 1
775− return $
776− if end1 >= bound
777− then D.Skip
778− (GroupYield
779− contents start end1 bound (GroupStart s))
780− else D.Skip (GroupBuffer s contents start end1 bound)
781− D.Skip s ->
782− return $ D.Skip (GroupBuffer s contents start end bound)
783− D.Stop ->
784− return
785− $ D.Skip (GroupYield contents start end bound GroupFinish)
786−
787− step' _ (GroupYield contents start end bound next) =
788− return $ D.Yield (MutArray contents start end bound) next
789−
790− step' _ GroupFinish = return D.Stop
667+ liftIO $ D.fold write $ D.morphInner (return . runIdentity) xs
791668
792669-------------------------------------------------------------------------------
793670-- Unfolds
@@ -850,7 +727,7 @@ putSliceUnsafe src srcStart dst dstStart len = liftIO $ do
850727
851728{-# INLINE clone #-}
852729clone :: MonadIO m => MutArray a -> m (MutArray a)
853−clone src = do
730+clone src = liftIO $ do
854731 let len = arrLen src
855732 dst <- new len
856733 putSliceUnsafe src 0 dst 0 len
core/src/Streamly/Internal/Data/Array/Mut.hsnew file+86
@@ -0,0 +1,86 @@
1+-- |
2+-- Module : Streamly.Internal.Data.Array.Mut
3+-- Copyright : (c) 2020 Composewell Technologies
4+-- License : BSD-3-Clause
5+-- Maintainer : streamly@composewell.com
6+-- Stability : experimental
7+-- Portability : GHC
8+--
9+module Streamly.Internal.Data.Array.Mut
10+ (
11+ module Streamly.Internal.Data.Array.Mut.Type
12+ , splitOn
13+ , genSlicesFromLen
14+ , getSlicesFromLen
15+ , fromStream
16+ )
17+where
18+
19+#include "inline.hs"
20+
21+import Control.Monad.IO.Class (MonadIO(..))
22+import Streamly.Internal.Data.Unbox (Unbox)
23+import Streamly.Internal.Data.Stream (Stream)
24+import Streamly.Internal.Data.Unfold.Type (Unfold(..))
25+
26+import qualified Streamly.Internal.Data.Stream as D
27+import qualified Streamly.Internal.Data.Unfold as Unfold
28+
29+import Prelude hiding (foldr, length, read, splitAt)
30+import Streamly.Internal.Data.Array.Mut.Type
31+
32+-- | Split the array into a stream of slices using a predicate. The element
33+-- matching the predicate is dropped.
34+--
35+-- /Pre-release/
36+{-# INLINE splitOn #-}
37+splitOn :: (MonadIO m, Unbox a) =>
38+ (a -> Bool) -> MutArray a -> Stream m (MutArray a)
39+splitOn predicate arr =
40+ fmap (\(i, len) -> getSliceUnsafe i len arr)
41+ $ D.sliceOnSuffix predicate (read arr)
42+
43+-- | Generate a stream of array slice descriptors ((index, len)) of specified
44+-- length from an array, starting from the supplied array index. The last slice
45+-- may be shorter than the requested length depending on the array length.
46+--
47+-- /Pre-release/
48+{-# INLINE genSlicesFromLen #-}
49+genSlicesFromLen :: forall m a. (Monad m, Unbox a)
50+ => Int -- ^ from index
51+ -> Int -- ^ length of the slice
52+ -> Unfold m (MutArray a) (Int, Int)
53+genSlicesFromLen from len =
54+ let fromThenTo n = (from, from + len, n - 1)
55+ mkSlice n i = return (i, min len (n - i))
56+ in Unfold.lmap length
57+ $ Unfold.mapM2 mkSlice
58+ $ Unfold.lmap fromThenTo Unfold.enumerateFromThenTo
59+
60+-- | Generate a stream of slices of specified length from an array, starting
61+-- from the supplied array index. The last slice may be shorter than the
62+-- requested length depending on the array length.
63+--
64+-- /Pre-release/
65+{-# INLINE getSlicesFromLen #-}
66+getSlicesFromLen :: forall m a. (Monad m, Unbox a)
67+ => Int -- ^ from index
68+ -> Int -- ^ length of the slice
69+ -> Unfold m (MutArray a) (MutArray a)
70+getSlicesFromLen from len =
71+ let mkSlice arr (i, n) = return $ getSliceUnsafe i n arr
72+ in Unfold.mapM2 mkSlice (genSlicesFromLen from len)
73+
74+-- | Create an 'Array' from a stream. This is useful when we want to create a
75+-- single array from a stream of unknown size. 'writeN' is at least twice
76+-- as efficient when the size is already known.
77+--
78+-- Note that if the input stream is too large memory allocation for the array
79+-- may fail. When the stream size is not known, `chunksOf` followed by
80+-- processing of indvidual arrays in the resulting stream should be preferred.
81+--
82+-- /Pre-release/
83+{-# INLINE fromStream #-}
84+fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
85+fromStream = fromStreamD
86+-- fromStream (Stream m) = P.fold write m
core/src/Streamly/Internal/Data/Array/Mut/Stream.hsnew file+323
@@ -0,0 +1,323 @@
1+-- |
2+-- Module : Streamly.Internal.Data.Array.Mut.Stream
3+-- Copyright : (c) 2019 Composewell Technologies
4+-- License : BSD3-3-Clause
5+-- Maintainer : streamly@composewell.com
6+-- Stability : experimental
7+-- Portability : GHC
8+--
9+-- Combinators to efficiently manipulate streams of mutable arrays.
10+--
11+module Streamly.Internal.Data.Array.Mut.Stream
12+ (
13+ -- * Generation
14+ chunksOf
15+
16+ -- * Compaction
17+ , packArraysChunksOf
18+ , SpliceState (..)
19+ , lpackArraysChunksOf
20+ , compact
21+ , compactLE
22+ , compactEQ
23+ , compactGE
24+ )
25+where
26+
27+#include "inline.hs"
28+#include "ArrayMacros.h"
29+
30+import Control.Monad.IO.Class (MonadIO(..))
31+import Control.Monad (when)
32+import Data.Bifunctor (first)
33+import Data.Proxy (Proxy(..))
34+import Streamly.Internal.Data.Unbox (Unbox, sizeOf)
35+import Streamly.Internal.Data.Array.Mut.Type (MutArray(..))
36+import Streamly.Internal.Data.Fold.Type (Fold(..))
37+import Streamly.Internal.Data.Parser (ParseError)
38+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
39+import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
40+
41+import qualified Streamly.Internal.Data.Array.Mut.Type as MArray
42+import qualified Streamly.Internal.Data.Fold.Type as FL
43+import qualified Streamly.Internal.Data.Parser as ParserD
44+import qualified Streamly.Internal.Data.Stream as D
45+
46+-- | @chunksOf n stream@ groups the elements in the input stream into arrays of
47+-- @n@ elements each.
48+--
49+-- Same as the following but may be more efficient:
50+--
51+-- > chunksOf n = Stream.foldMany (MArray.writeN n)
52+--
53+-- /Pre-release/
54+{-# INLINE chunksOf #-}
55+chunksOf :: (MonadIO m, Unbox a)
56+ => Int -> Stream m a -> Stream m (MutArray a)
57+chunksOf = MArray.chunksOf
58+
59+-------------------------------------------------------------------------------
60+-- Compact
61+-------------------------------------------------------------------------------
62+
63+data SpliceState s arr
64+ = SpliceInitial s
65+ | SpliceBuffering s arr
66+ | SpliceYielding arr (SpliceState s arr)
67+ | SpliceFinish
68+
69+-- XXX This can be removed once compactLEFold/compactLE are implemented.
70+--
71+-- | This mutates the first array (if it has space) to append values from the
72+-- second one. This would work for immutable arrays as well because an
73+-- immutable array never has space so a new array is allocated instead of
74+-- mutating it.
75+--
76+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
77+-- maximum specified size. Note that if a single array is bigger than the
78+-- specified size we do not split it to fit. When we coalesce multiple arrays
79+-- if the size would exceed the specified size we do not coalesce therefore the
80+-- actual array size may be less than the specified chunk size.
81+--
82+-- @since 0.7.0
83+{-# INLINE_NORMAL packArraysChunksOf #-}
84+packArraysChunksOf :: (MonadIO m, Unbox a)
85+ => Int -> D.Stream m (MutArray a) -> D.Stream m (MutArray a)
86+packArraysChunksOf n (D.Stream step state) =
87+ D.Stream step' (SpliceInitial state)
88+
89+ where
90+
91+ {-# INLINE_LATE step' #-}
92+ step' gst (SpliceInitial st) = do
93+ when (n <= 0) $
94+ -- XXX we can pass the module string from the higher level API
95+ error $ "Streamly.Internal.Data.Array.Mut.Type.packArraysChunksOf: the size of "
96+ ++ "arrays [" ++ show n ++ "] must be a natural number"
97+ r <- step gst st
98+ case r of
99+ D.Yield arr s -> return $
100+ let len = MArray.byteLength arr
101+ in if len >= n
102+ then D.Skip (SpliceYielding arr (SpliceInitial s))
103+ else D.Skip (SpliceBuffering s arr)
104+ D.Skip s -> return $ D.Skip (SpliceInitial s)
105+ D.Stop -> return D.Stop
106+
107+ step' gst (SpliceBuffering st buf) = do
108+ r <- step gst st
109+ case r of
110+ D.Yield arr s -> do
111+ let len = MArray.byteLength buf + MArray.byteLength arr
112+ if len > n
113+ then return $
114+ D.Skip (SpliceYielding buf (SpliceBuffering s arr))
115+ else do
116+ buf' <- if MArray.byteCapacity buf < n
117+ then liftIO $ MArray.realloc n buf
118+ else return buf
119+ buf'' <- MArray.splice buf' arr
120+ return $ D.Skip (SpliceBuffering s buf'')
121+ D.Skip s -> return $ D.Skip (SpliceBuffering s buf)
122+ D.Stop -> return $ D.Skip (SpliceYielding buf SpliceFinish)
123+
124+ step' _ SpliceFinish = return D.Stop
125+
126+ step' _ (SpliceYielding arr next) = return $ D.Yield arr next
127+
128+-- XXX Remove this once compactLEFold is implemented
129+-- lpackArraysChunksOf = Fold.many compactLEFold
130+--
131+{-# INLINE_NORMAL lpackArraysChunksOf #-}
132+lpackArraysChunksOf :: (MonadIO m, Unbox a)
133+ => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
134+lpackArraysChunksOf n (Fold step1 initial1 extract1) =
135+ Fold step initial extract
136+
137+ where
138+
139+ initial = do
140+ when (n <= 0) $
141+ -- XXX we can pass the module string from the higher level API
142+ error $ "Streamly.Internal.Data.Array.Mut.Type.packArraysChunksOf: the size of "
143+ ++ "arrays [" ++ show n ++ "] must be a natural number"
144+
145+ r <- initial1
146+ return $ first (Tuple' Nothing) r
147+
148+ extract (Tuple' Nothing r1) = extract1 r1
149+ extract (Tuple' (Just buf) r1) = do
150+ r <- step1 r1 buf
151+ case r of
152+ FL.Partial rr -> extract1 rr
153+ FL.Done _ -> return ()
154+
155+ step (Tuple' Nothing r1) arr =
156+ let len = MArray.byteLength arr
157+ in if len >= n
158+ then do
159+ r <- step1 r1 arr
160+ case r of
161+ FL.Done _ -> return $ FL.Done ()
162+ FL.Partial s -> do
163+ extract1 s
164+ res <- initial1
165+ return $ first (Tuple' Nothing) res
166+ else return $ FL.Partial $ Tuple' (Just arr) r1
167+
168+ step (Tuple' (Just buf) r1) arr = do
169+ let len = MArray.byteLength buf + MArray.byteLength arr
170+ buf' <- if MArray.byteCapacity buf < len
171+ then liftIO $ MArray.realloc (max n len) buf
172+ else return buf
173+ buf'' <- MArray.splice buf' arr
174+
175+ -- XXX this is common in both the equations of step
176+ if len >= n
177+ then do
178+ r <- step1 r1 buf''
179+ case r of
180+ FL.Done _ -> return $ FL.Done ()
181+ FL.Partial s -> do
182+ extract1 s
183+ res <- initial1
184+ return $ first (Tuple' Nothing) res
185+ else return $ FL.Partial $ Tuple' (Just buf'') r1
186+
187+-- XXX Same as compactLE, to be removed once that is implemented.
188+--
189+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
190+-- maximum specified size in bytes.
191+--
192+-- /Internal/
193+{-# INLINE compact #-}
194+compact :: (MonadIO m, Unbox a)
195+ => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
196+compact = packArraysChunksOf
197+
198+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
199+-- maximum specified size. Note that if a single array is bigger than the
200+-- specified size we do not split it to fit. When we coalesce multiple arrays
201+-- if the size would exceed the specified size we do not coalesce therefore the
202+-- actual array size may be less than the specified chunk size.
203+--
204+-- /Internal/
205+{-# INLINE_NORMAL compactLEParserD #-}
206+compactLEParserD ::
207+ forall m a. (MonadIO m, Unbox a)
208+ => Int -> ParserD.Parser (MutArray a) m (MutArray a)
209+compactLEParserD n = ParserD.Parser step initial extract
210+
211+ where
212+
213+ nBytes = n * SIZE_OF(a)
214+
215+ initial =
216+ return
217+ $ if n <= 0
218+ then error
219+ $ functionPath
220+ ++ ": the size of arrays ["
221+ ++ show n ++ "] must be a natural number"
222+ else ParserD.IPartial Nothing
223+
224+ step Nothing arr =
225+ return
226+ $ let len = MArray.byteLength arr
227+ in if len >= nBytes
228+ then ParserD.Done 0 arr
229+ else ParserD.Partial 0 (Just arr)
230+ step (Just buf) arr =
231+ let len = MArray.byteLength buf + MArray.byteLength arr
232+ in if len > nBytes
233+ then return $ ParserD.Done 1 buf
234+ else do
235+ buf1 <-
236+ if MArray.byteCapacity buf < nBytes
237+ then liftIO $ MArray.realloc nBytes buf
238+ else return buf
239+ buf2 <- MArray.splice buf1 arr
240+ return $ ParserD.Partial 0 (Just buf2)
241+
242+ extract Nothing = return $ ParserD.Done 0 MArray.nil
243+ extract (Just buf) = return $ ParserD.Done 0 buf
244+
245+ functionPath =
246+ "Streamly.Internal.Data.Array.Mut.Stream.compactLEParserD"
247+
248+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
249+-- minimum specified size. Note that if all the arrays in the stream together
250+-- are smaller than the specified size the resulting array will be smaller than
251+-- the specified size. When we coalesce multiple arrays if the size would exceed
252+-- the specified size we stop coalescing further.
253+--
254+-- /Internal/
255+{-# INLINE_NORMAL compactGEFold #-}
256+compactGEFold ::
257+ forall m a. (MonadIO m, Unbox a)
258+ => Int -> FL.Fold m (MutArray a) (MutArray a)
259+compactGEFold n = Fold step initial extract
260+
261+ where
262+
263+ nBytes = n * SIZE_OF(a)
264+
265+ initial =
266+ return
267+ $ if n < 0
268+ then error
269+ $ functionPath
270+ ++ ": the size of arrays ["
271+ ++ show n ++ "] must be a natural number"
272+ else FL.Partial Nothing
273+
274+ step Nothing arr =
275+ return
276+ $ let len = MArray.byteLength arr
277+ in if len >= nBytes
278+ then FL.Done arr
279+ else FL.Partial (Just arr)
280+ step (Just buf) arr = do
281+ let len = MArray.byteLength buf + MArray.byteLength arr
282+ buf1 <-
283+ if MArray.byteCapacity buf < len
284+ then liftIO $ MArray.realloc (max len nBytes) buf
285+ else return buf
286+ buf2 <- MArray.splice buf1 arr
287+ if len >= n
288+ then return $ FL.Done buf2
289+ else return $ FL.Partial (Just buf2)
290+
291+ extract Nothing = return MArray.nil
292+ extract (Just buf) = return buf
293+
294+ functionPath =
295+ "Streamly.Internal.Data.Array.Mut.Stream.compactGEFold"
296+
297+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
298+-- maximum specified size in bytes.
299+--
300+-- /Internal/
301+compactLE :: (MonadIO m, Unbox a) =>
302+ Int -> Stream m (MutArray a) -> Stream m (Either ParseError (MutArray a))
303+compactLE n = D.parseManyD (compactLEParserD n)
304+
305+-- | Like 'compactLE' but generates arrays of exactly equal to the size
306+-- specified except for the last array in the stream which could be shorter.
307+--
308+-- /Unimplemented/
309+{-# INLINE compactEQ #-}
310+compactEQ :: -- (MonadIO m, Unbox a) =>
311+ Int -> Stream m (MutArray a) -> Stream m (MutArray a)
312+compactEQ _n _xs = undefined
313+ -- IsStream.fromStreamD $ D.foldMany (compactEQFold n) (IsStream.toStreamD xs)
314+
315+-- | Like 'compactLE' but generates arrays of size greater than or equal to the
316+-- specified except for the last array in the stream which could be shorter.
317+--
318+-- /Internal/
319+{-# INLINE compactGE #-}
320+compactGE ::
321+ (MonadIO m, Unbox a)
322+ => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
323+compactGE n = D.foldMany (compactGEFold n)
core/src/Streamly/Internal/Data/MutArray/Type.hs → core/src/Streamly/Internal/Data/Array/Mut/Type.hsrenamed+482−1394
@@ -1,7 +1,7 @@
11{-# LANGUAGE CPP #-}
22{-# LANGUAGE UnboxedTuples #-}
33-- |
4−-- Module : Streamly.Internal.Data.MutArray.Type
4+-- Module : Streamly.Internal.Data.Array.Mut.Type
55-- Copyright : (c) 2020 Composewell Technologies
66-- License : BSD3-3-Clause
77-- Maintainer : streamly@composewell.com
@@ -26,82 +26,59 @@
2626-- strive to keep the API consistent for both. Ideally, you should be able to
2727-- replace one with another with little changes to the code.
2828
29−module Streamly.Internal.Data.MutArray.Type
29+module Streamly.Internal.Data.Array.Mut.Type
3030 (
31− -- ** Type
31+ -- * Type
3232 -- $arrayNotes
3333 MutArray (..)
34−
35− -- ** Conversion
36− -- *** Pinned and Unpinned
34+ , MutableByteArray
35+ , touch
3736 , pin
3837 , unpin
3938 , isPinned
4039
41− -- ** Casting
42− , cast
43− , castUnsafe -- XXX unsafeCast
44− , asBytes
45− , unsafePinnedAsPtr
46− , unsafeAsPtr
47−
40+ -- * Constructing and Writing
4841 -- ** Construction
49− , empty
50−
51− -- *** New
52− -- | New arrays are always empty arrays with some reserve capacity to
53− -- extend the length without reallocating.
54− , emptyOf
55− , newArrayWith -- emptyAlignedWith
56− , pinnedEmptyOf
57− , pinnedNewAligned -- XXX not required
58− -- , new -- uninitialized array of specified length
59−
60− -- *** Cloning
61− , clone
62− , pinnedClone
63−
64− -- *** Slicing
65− -- | Get a subarray without copying
66− , getSliceUnsafe -- XXX unsafeGetSlice
67− , getSlice
68− , splitAt -- XXX should be able to express using getSlice
69− , breakOn
42+ , nil
43+
44+ -- *** Uninitialized Arrays
45+ , newPinned
46+ , newPinnedBytes
47+ , newAlignedPinned
48+ , new
49+ , newArrayWith
50+
51+ -- *** Initialized Arrays
52+ , withNewArrayUnsafePinned
7053
71− -- *** Stream Folds
54+ -- *** From streams
7255 , ArrayUnsafe (..)
73− , unsafeCreateOfWith
74− , unsafeCreateOf
75− , unsafePinnedCreateOf
76− , pinnedCreateOf
77− , createOfWith
78− , createOf
79− , revCreateOf
80−
81− , pinnedCreate
82− , createWith
83− , create
84− -- , revCreate
85−
86− -- *** From containers
56+ , writeNWithUnsafe
57+ , writeNWith
58+ , writeNUnsafe
59+ , writeNUnsafeAs
60+ , writeN
61+ , writeNAs
62+ , pinnedWriteNAligned
63+
64+ , writeWith
65+ , write
66+
67+ , writeRevN
68+ -- , writeRev
69+
70+ -- ** From containers
8771 , fromListN
88− , pinnedFromListN
8972 , fromList
90− , pinnedFromList
9173 , fromListRevN
9274 , fromListRev
93− , fromStreamN
94− , fromStream
95− , fromPureStreamN
75+ , fromStreamDN
76+ , fromStreamD
9677 , fromPureStream
97− , fromByteStr#
98− , fromPtrN
99− , fromChunksK
100− , fromChunksRealloced -- fromSmallChunks
10178
102− -- ** Random writes
79+ -- * Random writes
10380 , putIndex
104− , putIndexUnsafe -- XXX unsafePutIndex
81+ , putIndexUnsafe
10582 , putIndices
10683 -- , putFromThenTo
10784 -- , putFrom -- start writing at the given position
@@ -109,71 +86,78 @@ module Streamly.Internal.Data.MutArray.Type
10986 -- , putFromTo
11087 -- , putFromRev
11188 -- , putUptoRev
112− , modifyIndexUnsafe -- XXX unsafeModifyIndex
89+ , modifyIndexUnsafe
11390 , modifyIndex
11491 , modifyIndices
11592 , modify
11693 , swapIndices
11794 , unsafeSwapIndices
11895
119− -- ** Reading
96+ -- * Growing and Shrinking
97+ -- Arrays grow only at the end, though it is possible to grow on both sides
98+ -- and therefore have a cons as well as snoc. But that will require two
99+ -- bounds in the array representation.
120100
121− -- *** Indexing
122− , getIndex
123− , getIndexUnsafe -- XXX unsafeGetIndex
124− -- , getFromThenTo
125− , getIndexRev -- getRevIndex?
126− , indexReader
127− , indexReaderWith
101+ -- ** Appending elements
102+ , snocWith
103+ , snoc
104+ , snocLinear
105+ , snocMay
106+ , snocUnsafe
107+
108+ -- ** Appending streams
109+ , writeAppendNUnsafe
110+ , writeAppendN
111+ , writeAppendWith
112+ , writeAppend
113+
114+ -- * Eliminating and Reading
128115
129− -- *** To Streams
116+ -- ** To streams
117+ , reader
118+ , readerRevWith
119+ , readerRev
120+
121+ -- ** To containers
122+ , toStreamDWith
123+ , toStreamDRevWith
124+ , toStreamKWith
125+ , toStreamKRevWith
130126 , read
131127 , readRev
132− , toStreamWith
133− , toStreamRevWith
134128 , toStreamK
135− , toStreamKWith
136129 , toStreamKRev
137− , toStreamKRevWith
138−
139− -- *** To Containers
140130 , toList
141131
142− -- *** Unfolds
143132 -- experimental
144133 , producerWith
145134 , producer
146135
147− , reader
148− , readerRevWith
149− , readerRev
150−
151− -- ** Size and Capacity
152− -- *** Size
153− , length
154− , byteLength
155−
156− -- *** Capacity
157− -- , capacity
158− , byteCapacity
159− , bytesFree
136+ -- ** Random reads
137+ , getIndex
138+ , getIndexUnsafe
139+ , getIndices
140+ , getIndicesD
141+ -- , getFromThenTo
142+ , getIndexRev
160143
161− -- *** Capacity Management
144+ -- * Memory Management
162145 , blockSize
163146 , arrayChunkBytes
164147 , allocBytesToElemCount
165− , realloc -- this is "resize"
166− , grow
167− , growExp
148+ , realloc
149+ , resize
150+ , resizeExp
168151 , rightSize
169152
170− -- ** Folding
171− , foldl'
172− , foldr
173− , byteCmp
174− , byteEq
153+ -- * Size
154+ , length
155+ , byteLength
156+ -- , capacity
157+ , byteCapacity
158+ , bytesFree
175159
176− -- ** In-place Mutation Algorithms
160+ -- * In-place Mutation Algorithms
177161 , strip
178162 , reverse
179163 , permute
@@ -183,121 +167,60 @@ module Streamly.Internal.Data.MutArray.Type
183167 , mergeBy
184168 , bubble
185169
186− -- ** Growing and Shrinking
187− -- | Arrays grow only at the end, though it is possible to grow on both
188− -- sides and therefore have a cons as well as snoc. But that will require
189− -- both lower and upper bound in the array representation.
170+ -- * Casting
171+ , cast
172+ , castUnsafe
173+ , asBytes
174+ , asPtrUnsafe
190175
191− -- *** Appending elements
192− , snocWith
193− , snoc
194− , snocLinear
195− , snocMay
196− , snocUnsafe -- XXX unsafeSnoc
176+ -- * Folding
177+ , foldl'
178+ , foldr
179+ , cmp
180+
181+ -- * Arrays of arrays
182+ -- We can add dimensionality parameter to the array type to get
183+ -- multidimensional arrays. Multidimensional arrays would just be a
184+ -- convenience wrapper on top of single dimensional arrays.
197185
198− -- *** Appending streams
199− , unsafeAppendN
200− , appendN
201− , appendWith
202− , append
186+ -- | Operations dealing with multiple arrays, streams of arrays or
187+ -- multidimensional array representations.
203188
204− -- *** Appending arrays
189+ -- ** Construct from streams
190+ , chunksOf
191+ , chunksOfAs
192+ , arrayStreamKFromStreamD
193+ , writeChunks
194+
195+ -- ** Eliminate to streams
196+ , flattenArrays
197+ , flattenArraysRev
198+ , fromArrayStreamK
199+
200+ -- ** Construct from arrays
201+ -- get chunks without copying
202+ , getSliceUnsafe
203+ , getSlice
204+ -- , getSlicesFromLenN
205+ , splitAt -- XXX should be able to express using getSlice
206+ , breakOn
207+
208+ -- ** Appending arrays
205209 , spliceCopy
206210 , spliceWith
207211 , splice
208212 , spliceExp
209− , spliceUnsafe -- XXX unsafeSplice
213+ , spliceUnsafe
214+ , putSliceUnsafe
210215 -- , putSlice
211216 -- , appendSlice
212217 -- , appendSliceFrom
213218
214− -- ** Serialization using Unbox
215− , pokeAppend
216− , pokeAppendMay
217− , pokeSkipUnsafe -- unsafePokeSkip
218−
219− -- ** Deserialization using Unbox
220− , peekUncons
221− , peekUnconsUnsafe -- unsafePeekUncons
222− , peekSkipUnsafe -- unsafePeekSkip
223−
224− -- Arrays of arrays
225− -- We can add dimensionality parameter to the array type to get
226− -- multidimensional arrays. Multidimensional arrays would just be a
227− -- convenience wrapper on top of single dimensional arrays.
228−
229− -- ** Streams of Arrays
230− -- *** Chunk
231− -- | Group a stream into arrays.
232− , chunksOf
233− , pinnedChunksOf
234− , buildChunks
235−
236− -- *** Split
237− -- | Split an array into slices.
238−
239− -- , getSlicesFromLenN
240− , splitOn
241− -- , slicesOf
242−
243− -- *** Concat
244− -- | Append the arrays in a stream to form a stream of elements.
245− , concatWith
246− , concatRevWith
247− , concat
248− , concatRev
249−
250− -- *** Compact
251− -- | Append the arrays in a stream to form a stream of larger arrays.
252− , SpliceState (..)
253− , pCompactLE
254− , pPinnedCompactLE
255− , compactLeAs
256− , fCompactGE
257− , fPinnedCompactGE
258− , lCompactGE
259− , lPinnedCompactGE
260− , compactGE
261− , compactEQ
262−
263− -- ** Utilities
219+ -- * Utilities
264220 , roundUpToPower2
265221 , memcpy
266222 , memcmp
267223 , c_memchr
268−
269− -- * Deprecated
270− , asPtrUnsafe
271− , writeChunks
272− , flattenArrays
273− , flattenArraysRev
274− , fromArrayStreamK
275− , fromStreamDN
276− , fromStreamD
277− , cmp
278− , getIndices
279− , getIndicesWith
280− , resize
281− , resizeExp
282− , nil
283− , new
284− , pinnedNew
285− , pinnedNewBytes
286− , writeAppendNUnsafe
287− , writeAppendN
288− , writeAppendWith
289− , writeAppend
290− , writeNWithUnsafe
291− , writeNWith
292− , writeNUnsafe
293− , pinnedWriteNUnsafe
294− , writeN
295− , pinnedWriteN
296− , pinnedWriteNAligned -- XXX not required
297− , writeWith
298− , write
299− , pinnedWrite
300− , writeRevN
301224 )
302225where
303226
@@ -308,52 +231,48 @@ where
308231
309232import Control.Monad (when, void)
310233import Control.Monad.IO.Class (MonadIO(..))
311−import Data.Bifunctor (first)
312234import Data.Bits (shiftR, (.|.), (.&.))
313235import Data.Functor.Identity (Identity(..))
314236import Data.Proxy (Proxy(..))
315237import Data.Word (Word8)
316238import Foreign.C.Types (CSize(..), CInt(..))
317239import Foreign.Ptr (plusPtr, minusPtr, nullPtr)
318−import Streamly.Internal.Data.MutByteArray.Type
319− ( MutByteArray(..)
240+import Streamly.Internal.Data.Unbox
241+ ( MutableByteArray(..)
242+ , Unbox(..)
320243 , PinnedState(..)
321244 , getMutableByteArray#
322− , putSliceUnsafe
245+ , touch
323246 )
324−import Streamly.Internal.Data.Unbox (Unbox(..))
325247import GHC.Base
326248 ( IO(..)
327249 , Int(..)
250+ , byteArrayContents#
328251 , compareByteArrays#
329252 , copyMutableByteArray#
330253 )
331254import GHC.Base (noinline)
332−import GHC.Exts (unsafeCoerce#, Addr#)
255+import GHC.Exts (unsafeCoerce#)
333256import GHC.Ptr (Ptr(..))
334257
335258import Streamly.Internal.Data.Fold.Type (Fold(..))
336259import Streamly.Internal.Data.Producer.Type (Producer (..))
337−import Streamly.Internal.Data.Stream.Type (Stream)
338−import Streamly.Internal.Data.Parser.Type (Parser (..))
339−import Streamly.Internal.Data.StreamK.Type (StreamK)
260+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
261+import Streamly.Internal.Data.Stream.StreamK.Type (StreamK)
340262import Streamly.Internal.Data.SVar.Type (adaptState, defState)
341−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
342263import Streamly.Internal.Data.Unfold.Type (Unfold(..))
343264import Streamly.Internal.System.IO (arrayPayloadSize, defaultChunkSize)
344265
345266import qualified Streamly.Internal.Data.Fold.Type as FL
346−import qualified Streamly.Internal.Data.MutByteArray.Type as Unboxed
347−import qualified Streamly.Internal.Data.Parser.Type as Parser
348−-- import qualified Streamly.Internal.Data.Fold.Type as Fold
349267import qualified Streamly.Internal.Data.Producer as Producer
350−import qualified Streamly.Internal.Data.Stream.Type as D
351−import qualified Streamly.Internal.Data.Stream.Lift as D
352−import qualified Streamly.Internal.Data.StreamK.Type as K
268+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
269+import qualified Streamly.Internal.Data.Stream.StreamD.Lift as D
270+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
271+import qualified Streamly.Internal.Data.Unbox as Unboxed
353272import qualified Prelude
354273
355274import Prelude hiding
356− (Foldable(..), concat, read, unlines, splitAt, reverse, truncate)
275+ (Foldable(..), read, unlines, splitAt, reverse, truncate)
357276
358277#include "DocTestDataMutArray.hs"
359278
@@ -361,7 +280,6 @@ import Prelude hiding
361280-- Foreign helpers
362281-------------------------------------------------------------------------------
363282
364−-- NOTE: Have to be "ccall unsafe" so that we can pass unpinned memory to these
365283foreign import ccall unsafe "string.h memcpy" c_memcpy
366284 :: Ptr Word8 -> Ptr Word8 -> CSize -> IO (Ptr Word8)
367285
@@ -371,9 +289,6 @@ foreign import ccall unsafe "string.h memchr" c_memchr
371289foreign import ccall unsafe "string.h memcmp" c_memcmp
372290 :: Ptr Word8 -> Ptr Word8 -> CSize -> IO CInt
373291
374−foreign import ccall unsafe "string.h strlen" c_strlen
375− :: Ptr Word8 -> IO CSize
376−
377292-- | Given an 'Unboxed' type (unused first arg) and a number of bytes, return
378293-- how many elements of that type will completely fit in those bytes.
379294--
@@ -428,7 +343,7 @@ data MutArray a =
428343 -- The array is a range into arrContents. arrContents may be a superset of
429344 -- the slice represented by the array. All offsets are in bytes.
430345 MutArray
431− { arrContents :: {-# UNPACK #-} !MutByteArray
346+ { arrContents :: {-# UNPACK #-} !MutableByteArray
432347 , arrStart :: {-# UNPACK #-} !Int -- ^ index into arrContents
433348 , arrEnd :: {-# UNPACK #-} !Int -- ^ index into arrContents
434349 -- Represents the first invalid index of
@@ -440,25 +355,22 @@ data MutArray a =
440355-- Pinning & Unpinning
441356-------------------------------------------------------------------------------
442357
443−-- | Return a copy of the array in pinned memory if unpinned, else return the
444−-- original array.
358+-- | Copy the 'MutArray' to pinned memory if unpinned, else do nothing. The
359+-- overhead is a copy if the input array is unpinned.
445360{-# INLINE pin #-}
446361pin :: MutArray a -> IO (MutArray a)
447−pin arr@MutArray{..} =
448− if Unboxed.isPinned arrContents
449− then pure arr
450− else pinnedClone arr
362+pin arr@MutArray{..} = do
363+ contents <- Unboxed.pin arrContents
364+ return $ arr {arrContents = contents}
451365
452−-- | Return a copy of the array in unpinned memory if pinned, else return the
453−-- original array.
366+-- | Copy the 'MutArray' to unpinned memory if pinned, else do nothing. The
367+-- overhead is a copy if the input array is pinned.
454368{-# INLINE unpin #-}
455369unpin :: MutArray a -> IO (MutArray a)
456−unpin arr@MutArray{..} =
457− if Unboxed.isPinned arrContents
458− then clone arr
459− else pure arr
370+unpin arr@MutArray{..} = do
371+ contents <- Unboxed.unpin arrContents
372+ return $ arr {arrContents = contents}
460373
461−-- | Return 'True' if the array is allocated in pinned memory.
462374{-# INLINE isPinned #-}
463375isPinned :: MutArray a -> Bool
464376isPinned MutArray{..} = Unboxed.isPinned arrContents
@@ -476,11 +388,9 @@ isPinned MutArray{..} = Unboxed.isPinned arrContents
476388-- files and arrays.
477389
478390-- GHC always guarantees word-aligned memory, alignment is important only when
479−-- we need more than that. See stg_pinnedNewAlignedByteArrayzh and
391+-- we need more than that. See stg_newAlignedPinnedByteArrayzh and
480392-- allocatePinned in GHC source.
481393
482−-- XXX Rename to emptyAlignedWith, alignSize should be first arg.
483−
484394-- | @newArrayWith allocator alignment count@ allocates a new array of zero
485395-- length and with a capacity to hold @count@ elements, using @allocator
486396-- size alignment@ as the memory allocator function.
@@ -493,7 +403,7 @@ isPinned MutArray{..} = Unboxed.isPinned arrContents
493403-- /Pre-release/
494404{-# INLINE newArrayWith #-}
495405newArrayWith :: forall m a. (MonadIO m, Unbox a)
496− => (Int -> Int -> m MutByteArray) -> Int -> Int -> m (MutArray a)
406+ => (Int -> Int -> m MutableByteArray) -> Int -> Int -> m (MutArray a)
497407newArrayWith alloc alignSize count = do
498408 let size = max (count * SIZE_OF(a)) 0
499409 contents <- alloc size alignSize
@@ -504,25 +414,12 @@ newArrayWith alloc alignSize count = do
504414 , arrBound = size
505415 }
506416
507−-- For arrays "nil" sounds a bit odd. empty is better. The only problem with
508−-- empty is that it is also used by the Alternative type class. But assuming we
509−-- will mostly import the Array module qualified this should be fine.
510−
511−-- | Create an empty array.
512−empty ::
513−#ifdef DEVBUILD
514− Unbox a =>
515−#endif
516− MutArray a
517−empty = MutArray Unboxed.empty 0 0 0
518−
519−{-# DEPRECATED nil "Please use empty instead." #-}
520417nil ::
521418#ifdef DEVBUILD
522419 Unbox a =>
523420#endif
524421 MutArray a
525−nil = empty
422+nil = MutArray Unboxed.nil 0 0 0
526423
527424{-# INLINE newBytesAs #-}
528425newBytesAs :: MonadIO m =>
@@ -531,7 +428,7 @@ newBytesAs :: MonadIO m =>
531428#endif
532429 PinnedState -> Int -> m (MutArray a)
533430newBytesAs ps bytes = do
534− contents <- liftIO $ Unboxed.newBytesAs ps bytes
431+ contents <- liftIO $ Unboxed.newBytes ps bytes
535432 return $ MutArray
536433 { arrContents = contents
537434 , arrStart = 0
@@ -539,62 +436,66 @@ newBytesAs ps bytes = do
539436 , arrBound = bytes
540437 }
541438
542−-- | Allocates a pinned empty array that with a reserved capacity of bytes.
543−-- The memory of the array is uninitialized and the allocation is aligned as
544−-- per the 'Unboxed' instance of the type.
439+-- | Allocates a pinned empty array that can hold 'count' items. The memory of
440+-- the array is uninitialized and the allocation is aligned as per the
441+-- 'Unboxed' instance of the type.
545442--
546443-- /Pre-release/
547−{-# INLINE pinnedNewBytes #-}
548−{-# DEPRECATED pinnedNewBytes "Please use pinnedEmptyOf with appropriate calculation" #-}
549−pinnedNewBytes :: MonadIO m =>
444+{-# INLINE newPinnedBytes #-}
445+newPinnedBytes :: MonadIO m =>
550446#ifdef DEVBUILD
551447 Unbox a =>
552448#endif
553449 Int -> m (MutArray a)
554−pinnedNewBytes = newBytesAs Pinned
450+newPinnedBytes = newBytesAs Pinned
555451
556452-- | Like 'newArrayWith' but using an allocator is a pinned memory allocator and
557453-- the alignment is dictated by the 'Unboxed' instance of the type.
558454--
559455-- /Internal/
560−{-# INLINE pinnedNewAligned #-}
561−pinnedNewAligned :: (MonadIO m, Unbox a) => Int -> Int -> m (MutArray a)
562−pinnedNewAligned =
563− newArrayWith (\s a -> liftIO $ Unboxed.pinnedNewAlignedBytes s a)
456+{-# INLINE newAlignedPinned #-}
457+newAlignedPinned :: (MonadIO m, Unbox a) => Int -> Int -> m (MutArray a)
458+newAlignedPinned =
459+ newArrayWith (\s a -> liftIO $ Unboxed.newAlignedPinnedBytes s a)
564460
565461{-# INLINE newAs #-}
566462newAs :: (MonadIO m, Unbox a) => PinnedState -> Int -> m (MutArray a)
567463newAs ps =
568464 newArrayWith
569− (\s _ -> liftIO $ Unboxed.newBytesAs ps s)
465+ (\s _ -> liftIO $ Unboxed.newBytes ps s)
570466 (error "new: alignment is not used in unpinned arrays.")
571467
572468-- XXX can unaligned allocation be more efficient when alignment is not needed?
573−
574−-- | Allocates a pinned array of zero length but growable to the specified
575−-- capacity without reallocation.
576−{-# INLINE pinnedEmptyOf #-}
577−pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
578−pinnedEmptyOf = newAs Pinned
579−
580−-- XXX Deprecate in major
581−-- {-# DEPRECATED pinnedNew "Please use pinnedEmptyOf instead." #-}
582−{-# INLINE pinnedNew #-}
583−pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
584−pinnedNew = pinnedEmptyOf
585−
586−-- | Allocates an unpinned array of zero length but growable to the specified
587−-- capacity without reallocation.
588469--
589−{-# INLINE emptyOf #-}
590−emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
591−emptyOf = newAs Unpinned
470+-- | Allocates an empty pinned array that can hold 'count' items. The memory of
471+-- the array is uninitialized and the allocation is aligned as per the 'Unboxed'
472+-- instance of the type.
473+--
474+{-# INLINE newPinned #-}
475+newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
476+newPinned = newAs Pinned
592477
593−-- XXX Deprecate in major
594−-- {-# DEPRECATED new "Please use emptyOf instead." #-}
478+-- | Allocates an empty unpinned array that can hold 'count' items. The memory
479+-- of the array is uninitialized.
480+--
595481{-# INLINE new #-}
596482new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
597−new = emptyOf
483+new = newAs Unpinned
484+
485+-- XXX This should create a full length uninitialzed array so that the pointer
486+-- can be used.
487+
488+-- | Allocate a pinned MutArray of the given size and run an IO action passing
489+-- the array start pointer.
490+--
491+-- /Internal/
492+{-# INLINE withNewArrayUnsafePinned #-}
493+withNewArrayUnsafePinned ::
494+ (MonadIO m, Unbox a) => Int -> (Ptr a -> m ()) -> m (MutArray a)
495+withNewArrayUnsafePinned count f = do
496+ arr <- newPinned count
497+ asPtrUnsafe arr
498+ $ \p -> f p >> return arr
598499
599500-------------------------------------------------------------------------------
600501-- Random writes
@@ -610,7 +511,7 @@ putIndexUnsafe :: forall m a. (MonadIO m, Unbox a)
610511putIndexUnsafe i MutArray{..} x = do
611512 let index = INDEX_OF(arrStart, i, a)
612513 assert (i >= 0 && INDEX_VALID(index, arrEnd, a)) (return ())
613− liftIO $ pokeAt index arrContents x
514+ liftIO $ pokeByteIndex index arrContents x
614515
615516invalidIndex :: String -> Int -> a
616517invalidIndex label i =
@@ -628,7 +529,7 @@ putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
628529putIndex i MutArray{..} x = do
629530 let index = INDEX_OF(arrStart,i,a)
630531 if i >= 0 && INDEX_VALID(index,arrEnd,a)
631− then liftIO $ pokeAt index arrContents x
532+ then liftIO $ pokeByteIndex index arrContents x
632533 else invalidIndex "putIndex" i
633534
634535-- | Write an input stream of (index, value) pairs to an array. Throws an
@@ -642,7 +543,7 @@ putIndices arr = FL.foldlM' step (return ())
642543
643544 where
644545
645− step () (i, x) = putIndex i arr x
546+ step () (i, x) = liftIO (putIndex i arr x)
646547
647548-- | Modify a given index of an array using a modifier function.
648549--
@@ -654,9 +555,9 @@ modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) =>
654555modifyIndexUnsafe i MutArray{..} f = liftIO $ do
655556 let index = INDEX_OF(arrStart,i,a)
656557 assert (i >= 0 && INDEX_NEXT(index,a) <= arrEnd) (return ())
657− r <- peekAt index arrContents
558+ r <- peekByteIndex index arrContents
658559 let (x, res) = f r
659− pokeAt index arrContents x
560+ pokeByteIndex index arrContents x
660561 return res
661562
662563-- | Modify a given index of an array using a modifier function.
@@ -668,9 +569,9 @@ modifyIndex i MutArray{..} f = do
668569 let index = INDEX_OF(arrStart,i,a)
669570 if i >= 0 && INDEX_VALID(index,arrEnd,a)
670571 then liftIO $ do
671− r <- peekAt index arrContents
572+ r <- peekByteIndex index arrContents
672573 let (x, res) = f r
673− pokeAt index arrContents x
574+ pokeByteIndex index arrContents x
674575 return res
675576 else invalidIndex "modifyIndex" i
676577
@@ -704,8 +605,8 @@ modify MutArray{..} f = liftIO $
704605
705606 go i =
706607 when (INDEX_VALID(i,arrEnd,a)) $ do
707− r <- peekAt i arrContents
708− pokeAt i arrContents (f r)
608+ r <- peekByteIndex i arrContents
609+ pokeByteIndex i arrContents (f r)
709610 go (INDEX_NEXT(i,a))
710611
711612-- XXX We could specify the number of bytes to swap instead of Proxy. Need
@@ -714,15 +615,15 @@ modify MutArray{..} f = liftIO $
714615swapArrayByteIndices ::
715616 forall a. Unbox a
716617 => Proxy a
717− -> MutByteArray
618+ -> MutableByteArray
718619 -> Int
719620 -> Int
720621 -> IO ()
721622swapArrayByteIndices _ arrContents i1 i2 = do
722− r1 <- peekAt i1 arrContents
723− r2 <- peekAt i2 arrContents
724− pokeAt i1 arrContents (r2 :: a)
725− pokeAt i2 arrContents (r1 :: a)
623+ r1 <- peekByteIndex i1 arrContents
624+ r2 <- peekByteIndex i2 arrContents
625+ pokeByteIndex i1 arrContents (r2 :: a)
626+ pokeByteIndex i2 arrContents (r1 :: a)
726627
727628-- | Swap the elements at two indices without validating the indices.
728629--
@@ -838,23 +739,26 @@ arrayChunkBytes = 1024
838739roundDownTo :: Int -> Int -> Int
839740roundDownTo elemSize size = size - (size `mod` elemSize)
840741
742+-- XXX See if resizing can be implemented by reading the old array as a stream
743+-- and then using writeN to the new array.
744+--
841745-- NOTE: we are passing elemSize explicitly to avoid an Unboxed constraint.
842746-- Since this is not inlined Unboxed consrraint leads to dictionary passing
843747-- which complicates some inspection tests.
844748--
845−{-# NOINLINE reallocExplicitAs #-}
846−reallocExplicitAs :: PinnedState -> Int -> Int -> MutArray a -> IO (MutArray a)
847−reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
749+{-# NOINLINE reallocExplicit #-}
750+reallocExplicit :: Int -> Int -> MutArray a -> IO (MutArray a)
751+reallocExplicit elemSize newCapacityInBytes MutArray{..} = do
848752 assertM(arrEnd <= arrBound)
849753
850754 -- Allocate new array
851755 let newCapMaxInBytes = roundUpLargeArray newCapacityInBytes
852756 contents <-
853− if ps == Pinned
854− then Unboxed.pinnedNew newCapMaxInBytes
855− else Unboxed.new newCapMaxInBytes
856− let !(MutByteArray mbarrFrom#) = arrContents
857− !(MutByteArray mbarrTo#) = contents
757+ if Unboxed.isPinned arrContents
758+ then Unboxed.newPinnedBytes newCapMaxInBytes
759+ else Unboxed.newUnpinnedBytes newCapMaxInBytes
760+ let !(MutableByteArray mbarrFrom#) = arrContents
761+ !(MutableByteArray mbarrTo#) = contents
858762
859763 -- Copy old data
860764 let oldStart = arrStart
@@ -875,10 +779,6 @@ reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
875779 , arrBound = newCapInBytes
876780 }
877781
878−-- XXX Should these be called reallocBytes etc?
879−-- XXX We may also need reallocAs to allocate as pinned/unpinned explicitly. In
880−-- fact clone/pinnedClone can be implemented using reallocAs.
881−
882782-- | @realloc newCapacity array@ reallocates the array to the specified
883783-- capacity in bytes.
884784--
@@ -887,15 +787,9 @@ reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
887787-- down to multiples of array size. If the new size is more than
888788-- 'largeObjectThreshold' then it is rounded up to the block size (4K).
889789--
890−-- If the original array is pinned, the newly allocated array is also pinned.
891790{-# INLINABLE realloc #-}
892791realloc :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
893−realloc bytes arr =
894− let ps =
895− if isPinned arr
896− then Pinned
897− else Unpinned
898− in liftIO $ reallocExplicitAs ps (SIZE_OF(a)) bytes arr
792+realloc bytes arr = liftIO $ reallocExplicit (SIZE_OF(a)) bytes arr
899793
900794-- | @reallocWith label capSizer minIncrBytes array@. The label is used
901795-- in error messages and the capSizer is used to determine the capacity of the
@@ -918,14 +812,14 @@ reallocWith label capSizer minIncrBytes arr = do
918812 where
919813
920814 badSize newSize =
921− Prelude.concat
815+ concat
922816 [ label
923817 , ": new array size (in bytes) is less than required size "
924818 , show newSize
925819 , ". Please check the sizing function passed."
926820 ]
927821
928−-- | @grow newCapacity array@ changes the total capacity of the array so that
822+-- | @resize newCapacity array@ changes the total capacity of the array so that
929823-- it is enough to hold the specified number of elements. Nothing is done if
930824-- the specified capacity is less than the length of the array.
931825--
@@ -933,46 +827,34 @@ reallocWith label capSizer minIncrBytes arr = do
933827-- the block size (4K).
934828--
935829-- /Pre-release/
936−{-# INLINE grow #-}
937−grow :: forall m a. (MonadIO m, Unbox a) =>
830+{-# INLINE resize #-}
831+resize :: forall m a. (MonadIO m, Unbox a) =>
938832 Int -> MutArray a -> m (MutArray a)
939−grow nElems arr@MutArray{..} = do
833+resize nElems arr@MutArray{..} = do
940834 let req = SIZE_OF(a) * nElems
941− cap = arrBound - arrStart
942− if req < cap
835+ len = arrEnd - arrStart
836+ if req < len
943837 then return arr
944838 else realloc req arr
945839
946−{-# DEPRECATED resize "Please use grow instead." #-}
947−{-# INLINE resize #-}
948−resize :: forall m a. (MonadIO m, Unbox a) =>
949− Int -> MutArray a -> m (MutArray a)
950−resize = grow
951−
952−-- | Like 'grow' but if the requested byte capacity is more than
953−-- 'largeObjectThreshold' then it is rounded up to the closest power of 2.
840+-- | Like 'resize' but if the byte capacity is more than 'largeObjectThreshold'
841+-- then it is rounded up to the closest power of 2.
954842--
955843-- /Pre-release/
956−{-# INLINE growExp #-}
957−growExp :: forall m a. (MonadIO m, Unbox a) =>
844+{-# INLINE resizeExp #-}
845+resizeExp :: forall m a. (MonadIO m, Unbox a) =>
958846 Int -> MutArray a -> m (MutArray a)
959−growExp nElems arr@MutArray{..} = do
847+resizeExp nElems arr@MutArray{..} = do
960848 let req = roundUpLargeArray (SIZE_OF(a) * nElems)
961849 req1 =
962850 if req > largeObjectThreshold
963851 then roundUpToPower2 req
964852 else req
965− cap = arrBound - arrStart
966− if req1 < cap
853+ len = arrEnd - arrStart
854+ if req1 < len
967855 then return arr
968856 else realloc req1 arr
969857
970−{-# DEPRECATED resizeExp "Please use growExp instead." #-}
971−{-# INLINE resizeExp #-}
972−resizeExp :: forall m a. (MonadIO m, Unbox a) =>
973− Int -> MutArray a -> m (MutArray a)
974−resizeExp = growExp
975−
976858-- | Resize the allocated memory to drop any reserved free space at the end of
977859-- the array and reallocate it to reduce wastage.
978860--
@@ -1017,7 +899,7 @@ rightSize arr@MutArray{..} = do
1017899snocNewEnd :: (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m (MutArray a)
1018900snocNewEnd newEnd arr@MutArray{..} x = liftIO $ do
1019901 assert (newEnd <= arrBound) (return ())
1020− pokeAt arrEnd arrContents x
902+ pokeByteIndex arrEnd arrContents x
1021903 return $ arr {arrEnd = newEnd}
1022904
1023905-- | Really really unsafe, appends the element into the first array, may
@@ -1037,7 +919,7 @@ snocUnsafe arr@MutArray{..} = snocNewEnd (INDEX_NEXT(arrEnd,a)) arr
1037919{-# INLINE snocMay #-}
1038920snocMay :: forall m a. (MonadIO m, Unbox a) =>
1039921 MutArray a -> a -> m (Maybe (MutArray a))
1040−snocMay arr@MutArray{..} x = do
922+snocMay arr@MutArray{..} x = liftIO $ do
1041923 let newEnd = INDEX_NEXT(arrEnd,a)
1042924 if newEnd <= arrBound
1043925 then Just <$> snocNewEnd newEnd arr x
@@ -1051,7 +933,7 @@ snocWithRealloc :: forall m a. (MonadIO m, Unbox a) =>
1051933 -> a
1052934 -> m (MutArray a)
1053935snocWithRealloc sizer arr x = do
1054− arr1 <- reallocWith "snocWith" sizer (SIZE_OF(a)) arr
936+ arr1 <- liftIO $ reallocWith "snocWith" sizer (SIZE_OF(a)) arr
1055937 snocUnsafe arr1 x
1056938
1057939-- | @snocWith sizer arr elem@ mutates @arr@ to append @elem@. The length of
@@ -1073,7 +955,7 @@ snocWith :: forall m a. (MonadIO m, Unbox a) =>
1073955 -> MutArray a
1074956 -> a
1075957 -> m (MutArray a)
1076−snocWith allocSize arr x = do
958+snocWith allocSize arr x = liftIO $ do
1077959 let newEnd = INDEX_NEXT(arrEnd arr,a)
1078960 if newEnd <= arrBound arr
1079961 then snocNewEnd newEnd arr x
@@ -1117,126 +999,6 @@ snoc = snocWith f
1117999 then oldSize * 2
11181000 else roundUpToPower2 oldSize * 2
11191001
1120−-------------------------------------------------------------------------------
1121−-- Serialization/Deserialization
1122−-------------------------------------------------------------------------------
1123−
1124−{-# INLINE pokeNewEnd #-}
1125−pokeNewEnd :: (MonadIO m, Unbox a) =>
1126− Int -> MutArray Word8 -> a -> m (MutArray Word8)
1127−pokeNewEnd newEnd arr@MutArray{..} x = liftIO $ do
1128− assert (newEnd <= arrBound) (return ())
1129− liftIO $ pokeAt arrEnd arrContents x
1130− return $ arr {arrEnd = newEnd}
1131−
1132−-- | Really really unsafe, unboxes a Haskell type and appends the resulting
1133−-- bytes to the byte array, may cause silent data corruption or if you are
1134−-- lucky a segfault if the array does not have enough space to append the
1135−-- element.
1136−--
1137−-- /Internal/
1138−{-# INLINE pokeAppendUnsafe #-}
1139−pokeAppendUnsafe :: forall m a. (MonadIO m, Unbox a) =>
1140− MutArray Word8 -> a -> m (MutArray Word8)
1141−pokeAppendUnsafe arr@MutArray{..} = pokeNewEnd (arrEnd + SIZE_OF(a)) arr
1142−
1143−-- | Skip the specified number of bytes in the array. The data in the skipped
1144−-- region remains uninitialzed.
1145−{-# INLINE pokeSkipUnsafe #-}
1146−pokeSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
1147−pokeSkipUnsafe n arr@MutArray{..} = do
1148− let newEnd = arrEnd + n
1149− in assert (newEnd <= arrBound) (arr {arrEnd = newEnd})
1150−
1151−-- | Like 'pokeAppend' but does not grow the array when pre-allocated array
1152−-- capacity becomes full.
1153−--
1154−-- /Internal/
1155−{-# INLINE pokeAppendMay #-}
1156−pokeAppendMay :: forall m a. (MonadIO m, Unbox a) =>
1157− MutArray Word8 -> a -> m (Maybe (MutArray Word8))
1158−pokeAppendMay arr@MutArray{..} x = liftIO $ do
1159− let newEnd = arrEnd + SIZE_OF(a)
1160− if newEnd <= arrBound
1161− then Just <$> pokeNewEnd newEnd arr x
1162− else return Nothing
1163−
1164−{-# NOINLINE pokeWithRealloc #-}
1165−pokeWithRealloc :: forall m a. (MonadIO m, Unbox a) =>
1166− (Int -> Int)
1167− -> MutArray Word8
1168− -> a
1169− -> m (MutArray Word8)
1170−pokeWithRealloc sizer arr x = do
1171− arr1 <- liftIO $ reallocWith "pokeWithRealloc" sizer (SIZE_OF(a)) arr
1172− pokeAppendUnsafe arr1 x
1173−
1174−{-# INLINE pokeAppendWith #-}
1175−pokeAppendWith :: forall m a. (MonadIO m, Unbox a) =>
1176− (Int -> Int)
1177− -> MutArray Word8
1178− -> a
1179− -> m (MutArray Word8)
1180−pokeAppendWith allocSize arr x = liftIO $ do
1181− let newEnd = arrEnd arr + SIZE_OF(a)
1182− if newEnd <= arrBound arr
1183− then pokeNewEnd newEnd arr x
1184− else pokeWithRealloc allocSize arr x
1185−
1186−-- | Unbox a Haskell type and append the resulting bytes to a mutable byte
1187−-- array. The array is grown exponentially when more space is needed.
1188−--
1189−-- Definition:
1190−--
1191−-- >>> pokeAppend arr x = MutArray.castUnsafe <$> MutArray.snoc (MutArray.castUnsafe arr) x
1192−--
1193−{-# INLINE pokeAppend #-}
1194−pokeAppend :: forall m a. (MonadIO m, Unbox a) =>
1195− MutArray Word8 -> a -> m (MutArray Word8)
1196−pokeAppend = pokeAppendWith f
1197−
1198− where
1199−
1200− f oldSize =
1201− if isPower2 oldSize
1202− then oldSize * 2
1203− else roundUpToPower2 oldSize * 2
1204−
1205−-- | Really really unsafe, create a Haskell value from an unboxed byte array,
1206−-- does not check if the array is big enough, may return garbage or if you are
1207−-- lucky may cause a segfault.
1208−--
1209−-- /Internal/
1210−{-# INLINE peekUnconsUnsafe #-}
1211−peekUnconsUnsafe :: forall m a. (MonadIO m, Unbox a) =>
1212− MutArray Word8 -> m (a, MutArray Word8)
1213−peekUnconsUnsafe MutArray{..} = do
1214− let start1 = arrStart + SIZE_OF(a)
1215− assert (start1 <= arrEnd) (return ())
1216− liftIO $ do
1217− r <- peekAt arrStart arrContents
1218− return (r, MutArray arrContents start1 arrEnd arrBound)
1219−
1220−-- | Discard the specified number of bytes in the array.
1221−{-# INLINE peekSkipUnsafe #-}
1222−peekSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
1223−peekSkipUnsafe n MutArray{..} =
1224− let start1 = arrStart + n
1225− in assert (start1 <= arrEnd) (MutArray arrContents start1 arrEnd arrBound)
1226−
1227−-- | Create a Haskell value from its unboxed representation from the head of a
1228−-- byte array, return the value and the remaining array.
1229−{-# INLINE peekUncons #-}
1230−peekUncons :: forall m a. (MonadIO m, Unbox a) =>
1231− MutArray Word8 -> m (Maybe a, MutArray Word8)
1232−peekUncons arr@MutArray{..} = do
1233− let start1 = arrStart + SIZE_OF(a)
1234− if start1 > arrEnd
1235− then return (Nothing, arr)
1236− else liftIO $ do
1237− r <- peekAt arrStart arrContents
1238− return (Just r, MutArray arrContents start1 arrEnd arrBound)
1239−
12401002-------------------------------------------------------------------------------
12411003-- Random reads
12421004-------------------------------------------------------------------------------
@@ -1251,7 +1013,7 @@ getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
12511013getIndexUnsafe i MutArray{..} = do
12521014 let index = INDEX_OF(arrStart,i,a)
12531015 assert (i >= 0 && INDEX_VALID(index,arrEnd,a)) (return ())
1254− liftIO $ peekAt index arrContents
1016+ liftIO $ peekByteIndex index arrContents
12551017
12561018-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
12571019--
@@ -1260,7 +1022,7 @@ getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
12601022getIndex i MutArray{..} = do
12611023 let index = INDEX_OF(arrStart,i,a)
12621024 if i >= 0 && INDEX_VALID(index,arrEnd,a)
1263− then liftIO $ Just <$> peekAt index arrContents
1025+ then liftIO $ Just <$> peekByteIndex index arrContents
12641026 else return Nothing
12651027
12661028-- | /O(1)/ Lookup the element at the given index from the end of the array.
@@ -1273,16 +1035,21 @@ getIndexRev :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
12731035getIndexRev i MutArray{..} = do
12741036 let index = RINDEX_OF(arrEnd,i,a)
12751037 if i >= 0 && index >= arrStart
1276− then liftIO $ peekAt index arrContents
1038+ then liftIO $ peekByteIndex index arrContents
12771039 else invalidIndex "getIndexRev" i
12781040
12791041data GetIndicesState contents start end st =
12801042 GetIndicesState contents start end st
12811043
1282−{-# INLINE indexReaderWith #-}
1283−indexReaderWith :: (Monad m, Unbox a) =>
1044+-- | Given an unfold that generates array indices, read the elements on those
1045+-- indices from the supplied MutArray. An error is thrown if an index is out of
1046+-- bounds.
1047+--
1048+-- /Pre-release/
1049+{-# INLINE getIndicesD #-}
1050+getIndicesD :: (Monad m, Unbox a) =>
12841051 (forall b. IO b -> m b) -> D.Stream m Int -> Unfold m (MutArray a) a
1285−indexReaderWith liftio (D.Stream stepi sti) = Unfold step inject
1052+getIndicesD liftio (D.Stream stepi sti) = Unfold step inject
12861053
12871054 where
12881055
@@ -1301,26 +1068,9 @@ indexReaderWith liftio (D.Stream stepi sti) = Unfold step inject
13011068 D.Skip s -> return $ D.Skip (GetIndicesState contents start end s)
13021069 D.Stop -> return D.Stop
13031070
1304−{-# DEPRECATED getIndicesWith "Please use indexReaderWith instead." #-}
1305−{-# INLINE getIndicesWith #-}
1306−getIndicesWith :: (Monad m, Unbox a) =>
1307− (forall b. IO b -> m b) -> D.Stream m Int -> Unfold m (MutArray a) a
1308−getIndicesWith = indexReaderWith
1309−
1310−-- | Given an unfold that generates array indices, read the elements on those
1311−-- indices from the supplied MutArray. An error is thrown if an index is out of
1312−-- bounds.
1313−--
1314−-- /Pre-release/
1315−{-# INLINE indexReader #-}
1316−indexReader :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1317−indexReader = indexReaderWith liftIO
1318−
1319−-- XXX DO NOT REMOVE, change the signature to use Stream instead of unfold
1320−{-# DEPRECATED getIndices "Please use indexReader instead." #-}
13211071{-# INLINE getIndices #-}
13221072getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1323−getIndices = indexReader
1073+getIndices = getIndicesD liftIO
13241074
13251075-------------------------------------------------------------------------------
13261076-- Subarrays
@@ -1425,7 +1175,7 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14251175
14261176 -- Invariant low < high on entry, and on return as well
14271177 moveHigh low high = do
1428− h <- peekAt high arrContents
1178+ h <- peekByteIndex high arrContents
14291179 if f h
14301180 then
14311181 -- Correctly classified, continue the loop
@@ -1445,7 +1195,7 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14451195 -- low <= high
14461196 -- Both low and high are valid locations within the array
14471197 go low high = do
1448− l <- peekAt low arrContents
1198+ l <- peekByteIndex low arrContents
14491199 if f l
14501200 then
14511201 -- low is wrongly classified
@@ -1456,8 +1206,8 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14561206 case r of
14571207 Nothing -> return low
14581208 Just (high1, h) -> do -- low < high1
1459− pokeAt low arrContents h
1460− pokeAt high1 arrContents l
1209+ pokeByteIndex low arrContents h
1210+ pokeByteIndex high1 arrContents l
14611211 let low1 = INDEX_NEXT(low,a)
14621212 high2 = INDEX_PREV(high1,a)
14631213 if low1 <= high2
@@ -1591,7 +1341,7 @@ chunksOfAs ps n (D.Stream step state) =
15911341 r <- step (adaptState gst) st
15921342 case r of
15931343 D.Yield x s -> do
1594− liftIO $ pokeAt end contents x
1344+ liftIO $ pokeByteIndex end contents x
15951345 let end1 = INDEX_NEXT(end,a)
15961346 return $
15971347 if end1 >= bound
@@ -1610,12 +1360,10 @@ chunksOfAs ps n (D.Stream step state) =
16101360
16111361 step' _ GroupFinish = return D.Stop
16121362
1613−-- | @chunksOf n stream@ groups the elements in the input stream into arrays of
1614−-- @n@ elements each.
1363+-- | @chunksOf n stream@ groups the input stream into a stream of
1364+-- arrays of size n.
16151365--
1616−-- Same as the following but may be more efficient:
1617−--
1618−-- >>> chunksOf n = Stream.foldMany (MutArray.createOf n)
1366+-- @chunksOf n = StreamD.foldMany (MutArray.writeN n)@
16191367--
16201368-- /Pre-release/
16211369{-# INLINE_NORMAL chunksOf #-}
@@ -1624,37 +1372,18 @@ chunksOf :: forall m a. (MonadIO m, Unbox a)
16241372-- XXX the idiomatic implementation leads to large regression in the D.reverse'
16251373-- benchmark. It seems it has difficulty producing optimized code when
16261374-- converting to StreamK. Investigate GHC optimizations.
1627−-- chunksOf n = D.foldMany (createOf n)
1375+-- chunksOf n = D.foldMany (writeN n)
16281376chunksOf = chunksOfAs Unpinned
16291377
1630−-- | Like 'chunksOf' but creates pinned arrays.
1631−{-# INLINE_NORMAL pinnedChunksOf #-}
1632−pinnedChunksOf :: forall m a. (MonadIO m, Unbox a)
1633− => Int -> D.Stream m a -> D.Stream m (MutArray a)
1634−-- pinnedChunksOf n = D.foldMany (pinnedCreateOf n)
1635−pinnedChunksOf = chunksOfAs Pinned
1636−
1637−-- | When we are buffering a stream of unknown size into an array we do not
1638−-- know how much space to pre-allocate. So we start with the min size and emit
1639−-- the array then keep on doubling the size every time. Thus we do not need to
1640−-- guess the optimum chunk size.
1641−--
1642−-- We can incorporate this in chunksOfAs if the additional size parameter does
1643−-- not impact perf.
1644−--
1645−{-# INLINE _chunksOfRange #-}
1646−_chunksOfRange :: -- (MonadIO m, Unbox a) =>
1647− PinnedState -> Int -> Int -> D.Stream m a -> D.Stream m (MutArray a)
1648−_chunksOfRange _ps _low _hi = undefined
1649−
1378+-- XXX This should take a PinnedState
16501379-- XXX buffer to a list instead?
16511380-- | Buffer the stream into arrays in memory.
1652−{-# INLINE arrayStreamKFromStreamDAs #-}
1653−arrayStreamKFromStreamDAs :: forall m a. (MonadIO m, Unbox a) =>
1654− PinnedState -> D.Stream m a -> m (StreamK m (MutArray a))
1655−arrayStreamKFromStreamDAs ps =
1381+{-# INLINE arrayStreamKFromStreamD #-}
1382+arrayStreamKFromStreamD :: forall m a. (MonadIO m, Unbox a) =>
1383+ D.Stream m a -> m (StreamK m (MutArray a))
1384+arrayStreamKFromStreamD =
16561385 let n = allocBytesToElemCount (undefined :: a) defaultChunkSize
1657− in D.foldr K.cons K.nil . chunksOfAs ps n
1386+ in D.foldr K.cons K.nil . chunksOf n
16581387
16591388-------------------------------------------------------------------------------
16601389-- Streams of arrays - Flattening
@@ -1664,10 +1393,16 @@ data FlattenState s contents a =
16641393 OuterLoop s
16651394 | InnerLoop s contents !Int !Int
16661395
1667−{-# INLINE_NORMAL concatWith #-}
1668−concatWith :: forall m a. (Monad m, Unbox a)
1669− => (forall b. IO b -> m b) -> D.Stream m (MutArray a) -> D.Stream m a
1670−concatWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
1396+-- | Use the "reader" unfold instead.
1397+--
1398+-- @flattenArrays = unfoldMany reader@
1399+--
1400+-- We can try this if there are any fusion issues in the unfold.
1401+--
1402+{-# INLINE_NORMAL flattenArrays #-}
1403+flattenArrays :: forall m a. (MonadIO m, Unbox a)
1404+ => D.Stream m (MutArray a) -> D.Stream m a
1405+flattenArrays (D.Stream step state) = D.Stream step' (OuterLoop state)
16711406
16721407 where
16731408
@@ -1684,30 +1419,19 @@ concatWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
16841419 return $ D.Skip $ OuterLoop st
16851420
16861421 step' _ (InnerLoop st contents p end) = do
1687− x <- liftio $ peekAt p contents
1422+ x <- liftIO $ peekByteIndex p contents
16881423 return $ D.Yield x (InnerLoop st contents (INDEX_NEXT(p,a)) end)
16891424
1690−-- | Use the "reader" unfold instead.
1425+-- | Use the "readerRev" unfold instead.
16911426--
1692−-- @concat = unfoldMany reader@
1427+-- @flattenArrays = unfoldMany readerRev@
16931428--
16941429-- We can try this if there are any fusion issues in the unfold.
16951430--
1696−{-# INLINE_NORMAL concat #-}
1697−concat :: forall m a. (MonadIO m, Unbox a)
1698− => D.Stream m (MutArray a) -> D.Stream m a
1699−concat = concatWith liftIO
1700−
1701−{-# DEPRECATED flattenArrays "Please use \"unfoldMany reader\" instead." #-}
1702−{-# INLINE flattenArrays #-}
1703−flattenArrays :: forall m a. (MonadIO m, Unbox a)
1431+{-# INLINE_NORMAL flattenArraysRev #-}
1432+flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
17041433 => D.Stream m (MutArray a) -> D.Stream m a
1705−flattenArrays = concat
1706−
1707−{-# INLINE_NORMAL concatRevWith #-}
1708−concatRevWith :: forall m a. (Monad m, Unbox a)
1709− => (forall b. IO b -> m b) -> D.Stream m (MutArray a) -> D.Stream m a
1710−concatRevWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
1434+flattenArraysRev (D.Stream step state) = D.Stream step' (OuterLoop state)
17111435
17121436 where
17131437
@@ -1725,33 +1449,16 @@ concatRevWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
17251449 return $ D.Skip $ OuterLoop st
17261450
17271451 step' _ (InnerLoop st contents p start) = do
1728− x <- liftio $ peekAt p contents
1452+ x <- liftIO $ peekByteIndex p contents
17291453 let cur = INDEX_PREV(p,a)
17301454 return $ D.Yield x (InnerLoop st contents cur start)
17311455
1732−-- | Use the "readerRev" unfold instead.
1733−--
1734−-- @concat = unfoldMany readerRev@
1735−--
1736−-- We can try this if there are any fusion issues in the unfold.
1737−--
1738−{-# INLINE_NORMAL concatRev #-}
1739−concatRev :: forall m a. (MonadIO m, Unbox a)
1740− => D.Stream m (MutArray a) -> D.Stream m a
1741−concatRev = concatRevWith liftIO
1742−
1743−{-# DEPRECATED flattenArraysRev "Please use \"unfoldMany readerRev\" instead." #-}
1744−{-# INLINE flattenArraysRev #-}
1745−flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
1746− => D.Stream m (MutArray a) -> D.Stream m a
1747−flattenArraysRev = concatRev
1748−
17491456-------------------------------------------------------------------------------
17501457-- Unfolds
17511458-------------------------------------------------------------------------------
17521459
17531460data ArrayUnsafe a = ArrayUnsafe
1754− {-# UNPACK #-} !MutByteArray -- contents
1461+ {-# UNPACK #-} !MutableByteArray -- contents
17551462 {-# UNPACK #-} !Int -- index 1
17561463 {-# UNPACK #-} !Int -- index 2
17571464
@@ -1781,7 +1488,7 @@ producerWith liftio = Producer step (return . toArrayUnsafe) extract
17811488 -- few actions for correctness and execution order sanity. We want
17821489 -- the peek to occur right here and not lazily at some later point
17831490 -- because we want the peek to be ordered with respect to the touch.
1784− !x <- liftio $ peekAt cur contents
1491+ !x <- liftio $ peekByteIndex cur contents
17851492 return $ D.Yield x (ArrayUnsafe contents (INDEX_NEXT(cur,a)) end)
17861493
17871494 extract = return . fromArrayUnsafe
@@ -1812,7 +1519,7 @@ readerRevWith liftio = Unfold step inject
18121519 {-# INLINE_LATE step #-}
18131520 step (ArrayUnsafe _ start p) | p < start = return D.Stop
18141521 step (ArrayUnsafe contents start p) = do
1815− !x <- liftio $ peekAt p contents
1522+ !x <- liftio $ peekByteIndex p contents
18161523 return $ D.Yield x (ArrayUnsafe contents start (INDEX_PREV(p,a)))
18171524
18181525-- | Unfold an array into a stream in reverse order.
@@ -1843,7 +1550,7 @@ toListFB c n MutArray{..} = go arrStart
18431550 -- evaluated/written to before we peek at them.
18441551 -- XXX
18451552 let !x = unsafeInlineIO $ do
1846− r <- peekAt arrContents p
1553+ r <- peekByteIndex arrContents p
18471554 return r
18481555 in c x (go (PTR_NEXT(p,a)))
18491556-}
@@ -1860,21 +1567,21 @@ toList MutArray{..} = liftIO $ go arrStart
18601567
18611568 go p | assert (p <= arrEnd) (p == arrEnd) = return []
18621569 go p = do
1863− x <- peekAt p arrContents
1570+ x <- peekByteIndex p arrContents
18641571 (:) x <$> go (INDEX_NEXT(p,a))
18651572
1866−{-# INLINE_NORMAL toStreamWith #-}
1867−toStreamWith ::
1573+{-# INLINE_NORMAL toStreamDWith #-}
1574+toStreamDWith ::
18681575 forall m a. (Monad m, Unbox a)
18691576 => (forall b. IO b -> m b) -> MutArray a -> D.Stream m a
1870−toStreamWith liftio MutArray{..} = D.Stream step arrStart
1577+toStreamDWith liftio MutArray{..} = D.Stream step arrStart
18711578
18721579 where
18731580
18741581 {-# INLINE_LATE step #-}
18751582 step _ p | assert (p <= arrEnd) (p == arrEnd) = return D.Stop
18761583 step _ p = liftio $ do
1877− r <- peekAt p arrContents
1584+ r <- peekByteIndex p arrContents
18781585 return $ D.Yield r (INDEX_NEXT(p,a))
18791586
18801587-- | Convert a 'MutArray' into a stream.
@@ -1883,7 +1590,7 @@ toStreamWith liftio MutArray{..} = D.Stream step arrStart
18831590--
18841591{-# INLINE_NORMAL read #-}
18851592read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> D.Stream m a
1886−read = toStreamWith liftIO
1593+read = toStreamDWith liftIO
18871594
18881595{-# INLINE toStreamKWith #-}
18891596toStreamKWith ::
@@ -1895,18 +1602,18 @@ toStreamKWith liftio MutArray{..} = go arrStart
18951602
18961603 go p | assert (p <= arrEnd) (p == arrEnd) = K.nil
18971604 | otherwise =
1898− let elemM = peekAt p arrContents
1605+ let elemM = peekByteIndex p arrContents
18991606 in liftio elemM `K.consM` go (INDEX_NEXT(p,a))
19001607
19011608{-# INLINE toStreamK #-}
19021609toStreamK :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
19031610toStreamK = toStreamKWith liftIO
19041611
1905−{-# INLINE_NORMAL toStreamRevWith #-}
1906−toStreamRevWith ::
1612+{-# INLINE_NORMAL toStreamDRevWith #-}
1613+toStreamDRevWith ::
19071614 forall m a. (Monad m, Unbox a)
19081615 => (forall b. IO b -> m b) -> MutArray a -> D.Stream m a
1909−toStreamRevWith liftio MutArray{..} =
1616+toStreamDRevWith liftio MutArray{..} =
19101617 let p = INDEX_PREV(arrEnd,a)
19111618 in D.Stream step p
19121619
@@ -1915,7 +1622,7 @@ toStreamRevWith liftio MutArray{..} =
19151622 {-# INLINE_LATE step #-}
19161623 step _ p | p < arrStart = return D.Stop
19171624 step _ p = liftio $ do
1918− r <- peekAt p arrContents
1625+ r <- peekByteIndex p arrContents
19191626 return $ D.Yield r (INDEX_PREV(p,a))
19201627
19211628-- | Convert a 'MutArray' into a stream in reverse order.
@@ -1924,7 +1631,7 @@ toStreamRevWith liftio MutArray{..} =
19241631--
19251632{-# INLINE_NORMAL readRev #-}
19261633readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> D.Stream m a
1927−readRev = toStreamRevWith liftIO
1634+readRev = toStreamDRevWith liftIO
19281635
19291636{-# INLINE toStreamKRevWith #-}
19301637toStreamKRevWith ::
@@ -1938,7 +1645,7 @@ toStreamKRevWith liftio MutArray {..} =
19381645
19391646 go p | p < arrStart = K.nil
19401647 | otherwise =
1941− let elemM = peekAt p arrContents
1648+ let elemM = peekByteIndex p arrContents
19421649 in liftio elemM `K.consM` go (INDEX_PREV(p,a))
19431650
19441651{-# INLINE toStreamKRev #-}
@@ -1973,7 +1680,7 @@ foldr f z arr = D.foldr f z $ read arr
19731680
19741681-- XXX Keep the bound intact to not lose any free space? Perf impact?
19751682
1976−-- | @unsafeAppendN n arr@ appends up to @n@ input items to the supplied
1683+-- | @writeAppendNUnsafe n arr@ appends up to @n@ input items to the supplied
19771684-- array.
19781685--
19791686-- Unsafe: Do not drive the fold beyond @n@ elements, it will lead to memory
@@ -1982,12 +1689,13 @@ foldr f z arr = D.foldr f z $ read arr
19821689-- Any free space left in the array after appending @n@ elements is lost.
19831690--
19841691-- /Internal/
1985−{-# INLINE_NORMAL unsafeAppendN #-}
1986−unsafeAppendN :: forall m a. (MonadIO m, Unbox a) =>
1692+{-# INLINE_NORMAL writeAppendNUnsafe #-}
1693+writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) =>
19871694 Int
19881695 -> m (MutArray a)
19891696 -> Fold m a (MutArray a)
1990−unsafeAppendN n action = fmap fromArrayUnsafe $ FL.foldlM' step initial
1697+writeAppendNUnsafe n action =
1698+ fmap fromArrayUnsafe $ FL.foldlM' step initial
19911699
19921700 where
19931701
@@ -2000,59 +1708,38 @@ unsafeAppendN n action = fmap fromArrayUnsafe $ FL.foldlM' step initial
20001708 -- otherwise we lose that space.
20011709 arr1 <-
20021710 if free < needed
2003− then noinline reallocWith "unsafeAppendN" (+ needed) needed arr
1711+ then noinline reallocWith "writeAppendNUnsafeWith" (+ needed) needed arr
20041712 else return arr
20051713 return $ toArrayUnsafe arr1
20061714
20071715 step (ArrayUnsafe contents start end) x = do
2008− liftIO $ pokeAt end contents x
1716+ liftIO $ pokeByteIndex end contents x
20091717 return $ ArrayUnsafe contents start (INDEX_NEXT(end,a))
20101718
2011−{-# DEPRECATED writeAppendNUnsafe "Please use unsafeAppendN instead." #-}
2012−{-# INLINE writeAppendNUnsafe #-}
2013−writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) =>
2014− Int
2015− -> m (MutArray a)
2016− -> Fold m a (MutArray a)
2017−writeAppendNUnsafe = unsafeAppendN
2018−
20191719-- | Append @n@ elements to an existing array. Any free space left in the array
20201720-- after appending @n@ elements is lost.
20211721--
2022−-- >>> appendN n initial = Fold.take n (MutArray.unsafeAppendN n initial)
1722+-- >>> writeAppendN n initial = Fold.take n (MutArray.writeAppendNUnsafe n initial)
20231723--
2024−{-# INLINE_NORMAL appendN #-}
2025−appendN :: forall m a. (MonadIO m, Unbox a) =>
2026− Int -> m (MutArray a) -> Fold m a (MutArray a)
2027−appendN n initial = FL.take n (unsafeAppendN n initial)
2028−
2029−-- XXX Deprecate in major
2030−-- {-# DEPRECATED writeAppendN "Please use appendN instead." #-}
2031−{-# INLINE writeAppendN #-}
1724+{-# INLINE_NORMAL writeAppendN #-}
20321725writeAppendN :: forall m a. (MonadIO m, Unbox a) =>
20331726 Int -> m (MutArray a) -> Fold m a (MutArray a)
2034−writeAppendN = appendN
1727+writeAppendN n initial = FL.take n (writeAppendNUnsafe n initial)
20351728
2036−-- | @appendWith realloc action@ mutates the array generated by @action@ to
1729+-- | @writeAppendWith realloc action@ mutates the array generated by @action@ to
20371730-- append the input stream. If there is no reserved space available in the
20381731-- array it is reallocated to a size in bytes determined by @realloc oldSize@,
20391732-- where @oldSize@ is the current size of the array in bytes.
20401733--
20411734-- Note that the returned array may be a mutated version of original array.
20421735--
2043−-- >>> appendWith sizer = Fold.foldlM' (MutArray.snocWith sizer)
1736+-- >>> writeAppendWith sizer = Fold.foldlM' (MutArray.snocWith sizer)
20441737--
20451738-- /Pre-release/
2046−{-# INLINE appendWith #-}
2047−appendWith :: forall m a. (MonadIO m, Unbox a) =>
2048− (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
2049−appendWith sizer = FL.foldlM' (snocWith sizer)
2050−
2051−{-# DEPRECATED writeAppendWith "Please use appendWith instead." #-}
20521739{-# INLINE writeAppendWith #-}
20531740writeAppendWith :: forall m a. (MonadIO m, Unbox a) =>
20541741 (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
2055−writeAppendWith = appendWith
1742+writeAppendWith sizer = FL.foldlM' (snocWith sizer)
20561743
20571744-- | @append action@ mutates the array generated by @action@ to append the
20581745-- input stream. If there is no reserved space available in the array it is
@@ -2060,101 +1747,64 @@ writeAppendWith = appendWith
20601747--
20611748-- Note that the returned array may be a mutated version of original array.
20621749--
2063−-- >>> append = MutArray.appendWith (* 2)
1750+-- >>> writeAppend = MutArray.writeAppendWith (* 2)
20641751--
2065−{-# INLINE append #-}
2066−append :: forall m a. (MonadIO m, Unbox a) =>
2067− m (MutArray a) -> Fold m a (MutArray a)
2068−append = appendWith (* 2)
2069−
2070−-- XXX Deprecate in major
2071−-- {-# DEPRECATED writeAppend "Please use append instead." #-}
20721752{-# INLINE writeAppend #-}
20731753writeAppend :: forall m a. (MonadIO m, Unbox a) =>
20741754 m (MutArray a) -> Fold m a (MutArray a)
2075−writeAppend = append
1755+writeAppend = writeAppendWith (* 2)
20761756
20771757-- XXX We can carry bound as well in the state to make sure we do not lose the
20781758-- remaining capacity. Need to check perf impact.
20791759--
2080−-- | Like 'unsafeCreateOf' but takes a new array allocator @alloc size@
2081−-- function as argument.
1760+-- | Like 'writeNUnsafe' but takes a new array allocator @alloc size@ function
1761+-- as argument.
20821762--
2083−-- >>> unsafeCreateOfWith alloc n = MutArray.unsafeAppendN (alloc n) n
1763+-- >>> writeNWithUnsafe alloc n = MutArray.writeAppendNUnsafe (alloc n) n
20841764--
20851765-- /Pre-release/
2086−{-# INLINE_NORMAL unsafeCreateOfWith #-}
2087−unsafeCreateOfWith :: forall m a. (MonadIO m, Unbox a)
1766+{-# INLINE_NORMAL writeNWithUnsafe #-}
1767+writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a)
20881768 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2089−unsafeCreateOfWith alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
1769+writeNWithUnsafe alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
20901770
20911771 where
20921772
20931773 initial = toArrayUnsafe <$> alloc (max n 0)
20941774
20951775 step (ArrayUnsafe contents start end) x = do
2096− liftIO $ pokeAt end contents x
1776+ liftIO $ pokeByteIndex end contents x
20971777 return
20981778 $ ArrayUnsafe contents start (INDEX_NEXT(end,a))
20991779
2100−{-# DEPRECATED writeNWithUnsafe "Please use unsafeCreateOfWith instead." #-}
2101−{-# INLINE writeNWithUnsafe #-}
2102−writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a)
2103− => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2104−writeNWithUnsafe = unsafeCreateOfWith
2105−
21061780{-# INLINE_NORMAL writeNUnsafeAs #-}
21071781writeNUnsafeAs :: forall m a. (MonadIO m, Unbox a)
21081782 => PinnedState -> Int -> Fold m a (MutArray a)
2109−writeNUnsafeAs ps = unsafeCreateOfWith (newAs ps)
1783+writeNUnsafeAs ps = writeNWithUnsafe (newAs ps)
21101784
2111−-- | Like 'createOf' but does not check the array bounds when writing. The fold
1785+-- | Like 'writeN' but does not check the array bounds when writing. The fold
21121786-- driver must not call the step function more than 'n' times otherwise it will
21131787-- corrupt the memory and crash. This function exists mainly because any
21141788-- conditional in the step function blocks fusion causing 10x performance
21151789-- slowdown.
21161790--
2117−-- >>> unsafeCreateOf = MutArray.unsafeCreateOfWith MutArray.emptyOf
1791+-- >>> writeNUnsafe = MutArray.writeNWithUnsafe MutArray.new
21181792--
2119−{-# INLINE_NORMAL unsafeCreateOf #-}
2120−unsafeCreateOf :: forall m a. (MonadIO m, Unbox a)
2121− => Int -> Fold m a (MutArray a)
2122−unsafeCreateOf = writeNUnsafeAs Unpinned
2123−
2124−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
2125−{-# INLINE writeNUnsafe #-}
1793+{-# INLINE_NORMAL writeNUnsafe #-}
21261794writeNUnsafe :: forall m a. (MonadIO m, Unbox a)
21271795 => Int -> Fold m a (MutArray a)
2128−writeNUnsafe = unsafeCreateOf
1796+writeNUnsafe = writeNUnsafeAs Unpinned
21291797
2130−-- | Like 'unsafeCreateOf' but creates a pinned array.
2131−{-# INLINE_NORMAL unsafePinnedCreateOf #-}
2132−unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a)
2133− => Int -> Fold m a (MutArray a)
2134−unsafePinnedCreateOf = writeNUnsafeAs Pinned
2135−
2136−{-# DEPRECATED pinnedWriteNUnsafe "Please use unsafePinnedCreateOf instead." #-}
2137−{-# INLINE pinnedWriteNUnsafe #-}
2138−pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a)
2139− => Int -> Fold m a (MutArray a)
2140−pinnedWriteNUnsafe = unsafePinnedCreateOf
2141−
2142−-- | @createOfWith alloc n@ folds a maximum of @n@ elements into an array
1798+-- | @writeNWith alloc n@ folds a maximum of @n@ elements into an array
21431799-- allocated using the @alloc@ function.
21441800--
2145−-- >>> createOfWith alloc n = Fold.take n (MutArray.unsafeCreateOfWith alloc n)
2146−-- >>> createOfWith alloc n = MutArray.appendN (alloc n) n
1801+-- >>> writeNWith alloc n = Fold.take n (MutArray.writeNWithUnsafe alloc n)
1802+-- >>> writeNWith alloc n = MutArray.writeAppendN (alloc n) n
21471803--
2148−{-# INLINE_NORMAL createOfWith #-}
2149−createOfWith :: forall m a. (MonadIO m, Unbox a)
2150− => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2151−createOfWith alloc n = FL.take n (unsafeCreateOfWith alloc n)
2152−
2153−-- {-# DEPRECATED writeNWith "Please use createOfWith instead." #-}
2154−{-# INLINE writeNWith #-}
1804+{-# INLINE_NORMAL writeNWith #-}
21551805writeNWith :: forall m a. (MonadIO m, Unbox a)
21561806 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2157−writeNWith = createOfWith
1807+writeNWith alloc n = FL.take n (writeNWithUnsafe alloc n)
21581808
21591809{-# INLINE_NORMAL writeNAs #-}
21601810writeNAs ::
@@ -2162,42 +1812,20 @@ writeNAs ::
21621812 => PinnedState
21631813 -> Int
21641814 -> Fold m a (MutArray a)
2165−writeNAs ps = createOfWith (newAs ps)
1815+writeNAs ps = writeNWith (newAs ps)
21661816
2167−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
1817+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
21681818-- 'MutArray'.
21691819--
2170−-- >>> createOf = MutArray.createOfWith MutArray.new
2171−-- >>> createOf n = Fold.take n (MutArray.unsafeCreateOf n)
2172−-- >>> createOf n = MutArray.appendN n (MutArray.emptyOf n)
1820+-- >>> writeN = MutArray.writeNWith MutArray.new
1821+-- >>> writeN n = Fold.take n (MutArray.writeNUnsafe n)
1822+-- >>> writeN n = MutArray.writeAppendN n (MutArray.new n)
21731823--
2174−{-# INLINE_NORMAL createOf #-}
2175−createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2176−createOf = writeNAs Unpinned
2177−
2178−-- XXX Deprecate in major
2179−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
2180−{-# INLINE writeN #-}
1824+{-# INLINE_NORMAL writeN #-}
21811825writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2182−writeN = createOf
1826+writeN = writeNAs Unpinned
21831827
2184−-- | Like 'createOf' but creates a pinned array.
2185−{-# INLINE_NORMAL pinnedCreateOf #-}
2186−pinnedCreateOf ::
2187− forall m a. (MonadIO m, Unbox a)
2188− => Int
2189− -> Fold m a (MutArray a)
2190−pinnedCreateOf = writeNAs Pinned
2191−
2192−{-# DEPRECATED pinnedWriteN "Please use pinnedCreateOf instead." #-}
2193−{-# INLINE pinnedWriteN #-}
2194−pinnedWriteN ::
2195− forall m a. (MonadIO m, Unbox a)
2196− => Int
2197− -> Fold m a (MutArray a)
2198−pinnedWriteN = pinnedCreateOf
2199−
2200−-- | Like unsafeCreateOfWith but writes the array in reverse order.
1828+-- | Like writeNWithUnsafe but writes the array in reverse order.
22011829--
22021830-- /Internal/
22031831{-# INLINE_NORMAL writeRevNWithUnsafe #-}
@@ -2214,11 +1842,11 @@ writeRevNWithUnsafe alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
22141842
22151843 step (ArrayUnsafe contents start end) x = do
22161844 let ptr = INDEX_PREV(start,a)
2217− liftIO $ pokeAt ptr contents x
1845+ liftIO $ pokeByteIndex ptr contents x
22181846 return
22191847 $ ArrayUnsafe contents ptr end
22201848
2221−-- | Like createOfWith but writes the array in reverse order.
1849+-- | Like writeNWith but writes the array in reverse order.
22221850--
22231851-- /Internal/
22241852{-# INLINE_NORMAL writeRevNWith #-}
@@ -2226,36 +1854,31 @@ writeRevNWith :: forall m a. (MonadIO m, Unbox a)
22261854 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
22271855writeRevNWith alloc n = FL.take n (writeRevNWithUnsafe alloc n)
22281856
2229−-- | Like 'createOf' but writes the array in reverse order.
1857+-- | Like writeN but writes the array in reverse order.
22301858--
22311859-- /Pre-release/
2232−{-# INLINE_NORMAL revCreateOf #-}
2233−revCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2234−revCreateOf = writeRevNWith new
2235−
2236−{-# DEPRECATED writeRevN "Please use revCreateOf instead." #-}
2237−{-# INLINE writeRevN #-}
1860+{-# INLINE_NORMAL writeRevN #-}
22381861writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2239−writeRevN = revCreateOf
1862+writeRevN = writeRevNWith new
22401863
22411864-- | @pinnedWriteNAligned align n@ folds a maximum of @n@ elements from the
22421865-- input stream to a 'MutArray' aligned to the given size.
22431866--
2244−-- >>> pinnedWriteNAligned align = MutArray.createOfWith (MutArray.pinnedNewAligned align)
2245−-- >>> pinnedWriteNAligned align n = MutArray.appendN n (MutArray.pinnedNewAligned align n)
1867+-- >>> pinnedWriteNAligned align = MutArray.writeNWith (MutArray.newAlignedPinned align)
1868+-- >>> pinnedWriteNAligned align n = MutArray.writeAppendN n (MutArray.newAlignedPinned align n)
22461869--
22471870-- /Pre-release/
22481871--
22491872{-# INLINE_NORMAL pinnedWriteNAligned #-}
22501873pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a)
22511874 => Int -> Int -> Fold m a (MutArray a)
2252−pinnedWriteNAligned align = createOfWith (pinnedNewAligned align)
1875+pinnedWriteNAligned align = writeNWith (newAlignedPinned align)
22531876
22541877-- XXX Buffer to a list instead?
2255−
1878+--
22561879-- | Buffer a stream into a stream of arrays.
22571880--
2258−-- >>> buildChunks n = Fold.many (MutArray.createOf n) Fold.toStreamK
1881+-- >>> writeChunks n = Fold.many (MutArray.writeN n) Fold.toStreamK
22591882--
22601883-- Breaking an array into an array stream can be useful to consume a large
22611884-- array sequentially such that memory of the array is released incrementatlly.
@@ -2264,43 +1887,14 @@ pinnedWriteNAligned align = createOfWith (pinnedNewAligned align)
22641887--
22651888-- /Unimplemented/
22661889--
2267−{-# INLINE_NORMAL buildChunks #-}
2268−buildChunks :: (MonadIO m, Unbox a) =>
2269− Int -> Fold m a (StreamK n (MutArray a))
2270−buildChunks n = FL.many (createOf n) FL.toStreamK
2271−
2272−{-# DEPRECATED writeChunks "Please use buildChunks instead." #-}
2273−{-# INLINE writeChunks #-}
1890+{-# INLINE_NORMAL writeChunks #-}
22741891writeChunks :: (MonadIO m, Unbox a) =>
22751892 Int -> Fold m a (StreamK n (MutArray a))
2276−writeChunks = buildChunks
2277−
2278−{-# INLINE_NORMAL writeWithAs #-}
2279−writeWithAs :: forall m a. (MonadIO m, Unbox a)
2280− => PinnedState -> Int -> Fold m a (MutArray a)
2281−-- writeWithAs ps n = FL.rmapM rightSize $ appendWith (* 2) (newAs ps n)
2282−writeWithAs ps elemCount =
2283− FL.rmapM extract $ FL.foldlM' step initial
2284−
2285− where
2286−
2287− initial = do
2288− when (elemCount < 0) $ error "createWith: elemCount is negative"
2289− newAs ps elemCount
2290−
2291− step arr@(MutArray _ start end bound) x
2292− | INDEX_NEXT(end,a) > bound = do
2293− let oldSize = end - start
2294− newSize = max (oldSize * 2) 1
2295− arr1 <- liftIO $ reallocExplicitAs ps (SIZE_OF(a)) newSize arr
2296− snocUnsafe arr1 x
2297− step arr x = snocUnsafe arr x
2298−
2299− extract = liftIO . rightSize
1893+writeChunks n = FL.many (writeN n) FL.toStreamK
23001894
2301−-- XXX Compare createWith with fromStreamD which uses an array of streams
2302−-- implementation. We can write this using buildChunks above if that is faster.
2303−-- If createWith is faster then we should use that to implement
1895+-- XXX Compare writeWith with fromStreamD which uses an array of streams
1896+-- implementation. We can write this using writeChunks above if that is faster.
1897+-- If writeWith is faster then we should use that to implement
23041898-- fromStreamD.
23051899--
23061900-- XXX The realloc based implementation needs to make one extra copy if we use
@@ -2315,91 +1909,74 @@ writeWithAs ps elemCount =
23151909-- XXX check if GHC's memory allocator is efficient enough. We can try the C
23161910-- malloc to compare against.
23171911
2318−-- | @createWith minCount@ folds the whole input to a single array. The array
1912+-- | @writeWith minCount@ folds the whole input to a single array. The array
23191913-- starts at a size big enough to hold minCount elements, the size is doubled
23201914-- every time the array needs to be grown.
23211915--
23221916-- /Caution! Do not use this on infinite streams./
23231917--
2324−-- >>> f n = MutArray.appendWith (* 2) (MutArray.emptyOf n)
2325−-- >>> createWith n = Fold.rmapM MutArray.rightSize (f n)
2326−-- >>> createWith n = Fold.rmapM MutArray.fromChunksK (MutArray.buildChunks n)
1918+-- >>> f n = MutArray.writeAppendWith (* 2) (MutArray.new n)
1919+-- >>> writeWith n = Fold.rmapM MutArray.rightSize (f n)
1920+-- >>> writeWith n = Fold.rmapM MutArray.fromArrayStreamK (MutArray.writeChunks n)
23271921--
23281922-- /Pre-release/
2329−{-# INLINE_NORMAL createWith #-}
2330−createWith :: forall m a. (MonadIO m, Unbox a)
2331− => Int -> Fold m a (MutArray a)
2332−-- createWith n = FL.rmapM rightSize $ appendWith (* 2) (emptyOf n)
2333−createWith = writeWithAs Unpinned
2334−
2335−{-# DEPRECATED writeWith "Please use createWith instead." #-}
2336−{-# INLINE writeWith #-}
1923+{-# INLINE_NORMAL writeWith #-}
23371924writeWith :: forall m a. (MonadIO m, Unbox a)
23381925 => Int -> Fold m a (MutArray a)
2339−writeWith = createWith
1926+-- writeWith n = FL.rmapM rightSize $ writeAppendWith (* 2) (new n)
1927+writeWith elemCount =
1928+ FL.rmapM extract $ FL.foldlM' step initial
1929+
1930+ where
1931+
1932+ initial = do
1933+ when (elemCount < 0) $ error "writeWith: elemCount is negative"
1934+ liftIO $ new elemCount
1935+
1936+ step arr@(MutArray _ start end bound) x
1937+ | INDEX_NEXT(end,a) > bound = do
1938+ let oldSize = end - start
1939+ newSize = max (oldSize * 2) 1
1940+ arr1 <- liftIO $ reallocExplicit (SIZE_OF(a)) newSize arr
1941+ snocUnsafe arr1 x
1942+ step arr x = snocUnsafe arr x
1943+
1944+ extract = liftIO . rightSize
23401945
23411946-- | Fold the whole input to a single array.
23421947--
2343−-- Same as 'createWith' using an initial array size of 'arrayChunkBytes' bytes
1948+-- Same as 'writeWith' using an initial array size of 'arrayChunkBytes' bytes
23441949-- rounded up to the element size.
23451950--
23461951-- /Caution! Do not use this on infinite streams./
23471952--
2348−{-# INLINE create #-}
2349−create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2350−create = createWith (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
2351−
2352−-- XXX Deprecate in major
2353−-- {-# DEPRECATED write "Please use create instead." #-}
23541953{-# INLINE write #-}
23551954write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2356−write = create
2357−
2358−-- | Like 'create' but creates a pinned array.
2359−{-# INLINE pinnedCreate #-}
2360−pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2361−pinnedCreate =
2362− writeWithAs Pinned (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
2363−
2364−{-# DEPRECATED pinnedWrite "Please use pinnedCreate instead." #-}
2365−{-# INLINE pinnedWrite #-}
2366−pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2367−pinnedWrite = pinnedCreate
1955+write = writeWith (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
23681956
23691957-------------------------------------------------------------------------------
23701958-- construct from streams, known size
23711959-------------------------------------------------------------------------------
23721960
2373−{-# INLINE_NORMAL fromStreamDNAs #-}
2374−fromStreamDNAs :: forall m a. (MonadIO m, Unbox a)
2375− => PinnedState -> Int -> D.Stream m a -> m (MutArray a)
2376−fromStreamDNAs ps limit str = do
2377− (arr :: MutArray a) <- newAs ps limit
1961+-- | Use the 'writeN' fold instead.
1962+--
1963+-- >>> fromStreamDN n = Stream.fold (MutArray.writeN n)
1964+--
1965+{-# INLINE_NORMAL fromStreamDN #-}
1966+fromStreamDN :: forall m a. (MonadIO m, Unbox a)
1967+ => Int -> D.Stream m a -> m (MutArray a)
1968+-- fromStreamDN n = D.fold (writeN n)
1969+fromStreamDN limit str = do
1970+ (arr :: MutArray a) <- liftIO $ new limit
23781971 end <- D.foldlM' (fwrite (arrContents arr)) (return $ arrEnd arr) $ D.take limit str
23791972 return $ arr {arrEnd = end}
23801973
23811974 where
23821975
23831976 fwrite arrContents ptr x = do
2384− liftIO $ pokeAt ptr arrContents x
1977+ liftIO $ pokeByteIndex ptr arrContents x
23851978 return $ INDEX_NEXT(ptr,a)
23861979
2387−-- | Use the 'createOf' fold instead.
2388−--
2389−-- >>> fromStreamN n = Stream.fold (MutArray.createOf n)
2390−--
2391−{-# INLINE_NORMAL fromStreamN #-}
2392−fromStreamN :: forall m a. (MonadIO m, Unbox a)
2393− => Int -> D.Stream m a -> m (MutArray a)
2394−-- fromStreamDN n = D.fold (createOf n)
2395−fromStreamN = fromStreamDNAs Unpinned
2396−
2397−{-# DEPRECATED fromStreamDN "Please use fromStreamN instead." #-}
2398−{-# INLINE fromStreamDN #-}
2399−fromStreamDN :: forall m a. (MonadIO m, Unbox a)
2400− => Int -> D.Stream m a -> m (MutArray a)
2401−fromStreamDN = fromStreamN
2402−
24031980-- | Create a 'MutArray' from the first N elements of a list. The array is
24041981-- allocated to size N, if the list terminates before N elements then the
24051982-- array may hold less than N elements.
@@ -2408,80 +1985,21 @@ fromStreamDN = fromStreamN
24081985fromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
24091986fromListN n xs = fromStreamDN n $ D.fromList xs
24101987
2411−-- | Like 'fromListN' but creates a pinned array.
2412−{-# INLINABLE pinnedFromListN #-}
2413−pinnedFromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
2414−pinnedFromListN n xs = fromStreamDNAs Pinned n $ D.fromList xs
2415−
24161988-- | Like fromListN but writes the array in reverse order.
24171989--
24181990-- /Pre-release/
24191991{-# INLINE fromListRevN #-}
24201992fromListRevN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
2421−fromListRevN n xs = D.fold (revCreateOf n) $ D.fromList xs
2422−
2423−-- | Convert a pure stream in Identity monad to a mutable array.
2424−{-# INLINABLE fromPureStreamN #-}
2425−fromPureStreamN :: (MonadIO m, Unbox a) =>
2426− Int -> Stream Identity a -> m (MutArray a)
2427−fromPureStreamN n xs =
2428− D.fold (createOf n) $ D.morphInner (return . runIdentity) xs
1993+fromListRevN n xs = D.fold (writeRevN n) $ D.fromList xs
24291994
24301995-- | Convert a pure stream in Identity monad to a mutable array.
24311996{-# INLINABLE fromPureStream #-}
24321997fromPureStream :: (MonadIO m, Unbox a) => Stream Identity a -> m (MutArray a)
24331998fromPureStream xs =
2434− D.fold create $ D.morphInner (return . runIdentity) xs
2435−
2436−{-# INLINABLE fromPtrN #-}
2437−fromPtrN :: MonadIO m => Int -> Ptr Word8 -> m (MutArray Word8)
2438−fromPtrN len addr = do
2439− -- memcpy is better than stream copy when the size is known.
2440− -- XXX We can implement a stream copy in a similar way by streaming Word64
2441− -- first and then remaining Word8.
2442− arr <- new len
2443− _ <- unsafeAsPtr arr
2444− (\ptr -> liftIO $ c_memcpy ptr addr (fromIntegral len))
2445− return (arr {arrEnd = len})
2446−
2447−{-# INLINABLE fromByteStr# #-}
2448−fromByteStr# :: MonadIO m => Addr# -> m (MutArray Word8)
2449−fromByteStr# addr = do
2450− -- It is better to count the size first and allocate exact space.
2451− -- Also, memcpy is better than stream copy when the size is known.
2452− -- C strlen compares 4 bytes at a time, so is better than the stream
2453− -- version. https://github.com/bminor/glibc/blob/master/string/strlen.c
2454− -- XXX We can possibly use a stream of Word64 to do the same.
2455− -- fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
2456− len <- liftIO $ c_strlen (Ptr addr)
2457− let lenInt = fromIntegral len
2458− arr <- new lenInt
2459− _ <- unsafeAsPtr arr (\ptr -> liftIO $ c_memcpy ptr (Ptr addr) len)
2460− return (arr {arrEnd = lenInt})
1999+ liftIO $ D.fold write $ D.morphInner (return . runIdentity) xs
24612000
24622001-------------------------------------------------------------------------------
2463−-- convert a stream of arrays to a single array by reallocating and copying
2464−-------------------------------------------------------------------------------
2465−
2466−-- XXX Both of these implementations of splicing seem to perform equally well.
2467−-- We need to perform benchmarks over a range of sizes though.
2468−
2469−-- | Also see 'fromChunksK'.
2470−{-# INLINE fromChunksRealloced #-}
2471−fromChunksRealloced :: forall m a. (MonadIO m, Unbox a)
2472− => Stream m (MutArray a) -> m (MutArray a)
2473−fromChunksRealloced s = do
2474− res <- D.uncons s
2475− case res of
2476− Just (a, strm) -> do
2477− arr <- D.foldlM' spliceExp (pure a) strm
2478− -- Reallocation is exponential so there may be 50% empty space in
2479− -- worst case. One more reallocation to reclaim the space.
2480− rightSize arr
2481− Nothing -> pure nil
2482−
2483−-------------------------------------------------------------------------------
2484−-- convert a stream of arrays to a single array by buffering arrays first
2002+-- convert stream to a single array
24852003-------------------------------------------------------------------------------
24862004
24872005{-# INLINE arrayStreamKLength #-}
@@ -2491,72 +2009,29 @@ arrayStreamKLength as = K.foldl' (+) 0 (K.map length as)
24912009-- | Convert an array stream to an array. Note that this requires peak memory
24922010-- that is double the size of the array stream.
24932011--
2494−{-# INLINE fromChunkskAs #-}
2495−fromChunkskAs :: (Unbox a, MonadIO m) =>
2496− PinnedState -> StreamK m (MutArray a) -> m (MutArray a)
2497−fromChunkskAs ps as = do
2498− len <- arrayStreamKLength as
2499− arr <- newAs ps len
2500− -- XXX is StreamK fold faster or StreamD fold?
2501− K.foldlM' spliceUnsafe (pure arr) as
2502− -- fromStreamDN len $ D.unfoldMany reader $ D.fromStreamK as
2503−
2504−-- XXX Need to compare this with fromChunks and fromChunkList and keep the
2505−-- fastest or simplest one if all are equally fast.
2506−
2507−-- | Convert an array stream to an array. Note that this requires peak memory
2508−-- that is double the size of the array stream.
2509−--
2510−-- Also see 'fromChunksRealloced'.
2511−--
2512−{-# INLINE fromChunksK #-}
2513−fromChunksK :: (Unbox a, MonadIO m) =>
2514− StreamK m (MutArray a) -> m (MutArray a)
2515−fromChunksK = fromChunkskAs Unpinned
2516−
2517−{-# DEPRECATED fromArrayStreamK "Please use fromChunksK instead." #-}
25182012{-# INLINE fromArrayStreamK #-}
25192013fromArrayStreamK :: (Unbox a, MonadIO m) =>
25202014 StreamK m (MutArray a) -> m (MutArray a)
2521−fromArrayStreamK = fromChunksK
2522−
2523−{-# INLINE fromStreamDAs #-}
2524−fromStreamDAs ::
2525− (MonadIO m, Unbox a) => PinnedState -> D.Stream m a -> m (MutArray a)
2526−fromStreamDAs ps m =
2527− arrayStreamKFromStreamDAs Unpinned m >>= fromChunkskAs ps
2528−
2529−-- | Create an 'Array' from a stream. This is useful when we want to create a
2530−-- single array from a stream of unknown size. 'createOf' is at least twice
2531−-- as efficient when the size is already known.
2532−--
2533−-- Note that if the input stream is too large memory allocation for the array
2534−-- may fail. When the stream size is not known, `chunksOf` followed by
2535−-- processing of indvidual arrays in the resulting stream should be preferred.
2536−--
2537−-- /Pre-release/
2538−{-# INLINE fromStream #-}
2539−fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
2540−fromStream = fromStreamDAs Unpinned
2015+fromArrayStreamK as = do
2016+ len <- arrayStreamKLength as
2017+ fromStreamDN len $ D.unfoldMany reader $ D.fromStreamK as
25412018
2542−-- fromStream (Stream m) = P.fold create m
25432019-- CAUTION: a very large number (millions) of arrays can degrade performance
25442020-- due to GC overhead because we need to buffer the arrays before we flatten
25452021-- all the arrays.
25462022--
2547−-- XXX Compare if this is faster or "fold create".
2023+-- XXX Compare if this is faster or "fold write".
25482024--
25492025-- | We could take the approach of doubling the memory allocation on each
25502026-- overflow. This would result in more or less the same amount of copying as in
25512027-- the chunking approach. However, if we have to shrink in the end then it may
25522028-- result in an extra copy of the entire data.
25532029--
2554−-- >>> fromStreamD = StreamD.fold MutArray.create
2030+-- >>> fromStreamD = StreamD.fold MutArray.write
25552031--
25562032{-# INLINE fromStreamD #-}
2557−{-# DEPRECATED fromStreamD "Please use fromStream instead." #-}
25582033fromStreamD :: (MonadIO m, Unbox a) => D.Stream m a -> m (MutArray a)
2559−fromStreamD = fromStream
2034+fromStreamD m = arrayStreamKFromStreamD m >>= fromArrayStreamK
25602035
25612036-- | Create a 'MutArray' from a list. The list must be of finite size.
25622037--
@@ -2564,11 +2039,6 @@ fromStreamD = fromStream
25642039fromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
25652040fromList xs = fromStreamD $ D.fromList xs
25662041
2567−-- | Like 'fromList' but creates a pinned array.
2568−{-# INLINE pinnedFromList #-}
2569−pinnedFromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
2570−pinnedFromList xs = fromStreamDAs Pinned $ D.fromList xs
2571−
25722042-- XXX We are materializing the whole list first for getting the length. Check
25732043-- if the 'fromList' like chunked implementation would fare better.
25742044
@@ -2577,75 +2047,44 @@ pinnedFromList xs = fromStreamDAs Pinned $ D.fromList xs
25772047fromListRev :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
25782048fromListRev xs = fromListRevN (Prelude.length xs) xs
25792049
2580−-------------------------------------------------------------------------------
2581−-- Cloning
2582−-------------------------------------------------------------------------------
2583−
2584−-- Arrays are aligned on 64-bit boundaries. The fastest way to copy an array is
2585−-- to unsafeCast it to Word64, read it, write it to Word64 array and unsafeCast
2586−-- it again. We can use SIMD read/write as well.
2587−
2588−{-# INLINE cloneAs #-}
2589−cloneAs ::
2590− ( MonadIO m
2591−#ifdef DEVBUILD
2592− , Unbox a
2593−#endif
2594− )
2595− => PinnedState -> MutArray a -> m (MutArray a)
2596−cloneAs ps src =
2597− do
2598− let startSrc = arrStart src
2599− srcLen = arrEnd src - startSrc
2600− newArrContents <-
2601− Unboxed.cloneSliceUnsafeAs ps startSrc srcLen (arrContents src)
2602− return $ MutArray newArrContents 0 srcLen srcLen
2603−
2604−{-# INLINE clone #-}
2605−clone ::
2606− ( MonadIO m
2607−#ifdef DEVBUILD
2608− , Unbox a
2609−#endif
2610− )
2611− => MutArray a -> m (MutArray a)
2612−clone = cloneAs Unpinned
2613−
2614−{-# INLINE pinnedClone #-}
2615−pinnedClone ::
2616− ( MonadIO m
2617−#ifdef DEVBUILD
2618− , Unbox a
2619−#endif
2620− )
2621− => MutArray a -> m (MutArray a)
2622−pinnedClone = cloneAs Pinned
2623−
26242050-------------------------------------------------------------------------------
26252051-- Combining
26262052-------------------------------------------------------------------------------
26272053
2628−-- | Copy two arrays into a newly allocated array. If the first array is pinned
2629−-- the spliced array is also pinned.
2054+-- | Put a sub range of a source array into a subrange of a destination array.
2055+-- This is not safe as it does not check the bounds.
2056+{-# INLINE putSliceUnsafe #-}
2057+putSliceUnsafe :: MonadIO m => MutArray a -> Int -> MutArray a -> Int -> Int -> m ()
2058+putSliceUnsafe src srcStartBytes dst dstStartBytes lenBytes = liftIO $ do
2059+ assertM(lenBytes <= arrBound dst - dstStartBytes)
2060+ assertM(lenBytes <= arrEnd src - srcStartBytes)
2061+ let !(I# srcStartBytes#) = srcStartBytes
2062+ !(I# dstStartBytes#) = dstStartBytes
2063+ !(I# lenBytes#) = lenBytes
2064+ let arrS# = getMutableByteArray# (arrContents src)
2065+ arrD# = getMutableByteArray# (arrContents dst)
2066+ IO $ \s# -> (# copyMutableByteArray#
2067+ arrS# srcStartBytes# arrD# dstStartBytes# lenBytes# s#
2068+ , () #)
2069+
2070+-- | Copy two arrays into a newly allocated array.
26302071{-# INLINE spliceCopy #-}
26312072spliceCopy :: forall m a. MonadIO m =>
26322073#ifdef DEVBUILD
26332074 Unbox a =>
26342075#endif
26352076 MutArray a -> MutArray a -> m (MutArray a)
2636−spliceCopy arr1 arr2 = do
2077+spliceCopy arr1 arr2 = liftIO $ do
26372078 let start1 = arrStart arr1
26382079 start2 = arrStart arr2
26392080 len1 = arrEnd arr1 - start1
26402081 len2 = arrEnd arr2 - start2
2082+ newArrContents <- liftIO $ Unboxed.newUnpinnedBytes (len1 + len2)
26412083 let len = len1 + len2
2642− newArrContents <-
2643− if Unboxed.isPinned (arrContents arr1)
2644− then liftIO $ Unboxed.pinnedNew len
2645− else liftIO $ Unboxed.new len
2646− putSliceUnsafe (arrContents arr1) start1 newArrContents 0 len1
2647− putSliceUnsafe (arrContents arr2) start2 newArrContents len1 len2
2648− return $ MutArray newArrContents 0 len len
2084+ newArr = MutArray newArrContents 0 len len
2085+ putSliceUnsafe arr1 start1 newArr 0 len1
2086+ putSliceUnsafe arr2 start2 newArr len1 len2
2087+ return newArr
26492088
26502089-- | Really really unsafe, appends the second array into the first array. If
26512090-- the first array does not have enough space it may cause silent data
@@ -2654,13 +2093,12 @@ spliceCopy arr1 arr2 = do
26542093spliceUnsafe :: MonadIO m =>
26552094 MutArray a -> MutArray a -> m (MutArray a)
26562095spliceUnsafe dst src =
2657− do
2096+ liftIO $ do
26582097 let startSrc = arrStart src
26592098 srcLen = arrEnd src - startSrc
26602099 endDst = arrEnd dst
26612100 assertM(endDst + srcLen <= arrBound dst)
2662− putSliceUnsafe
2663− (arrContents src) startSrc (arrContents dst) endDst srcLen
2101+ putSliceUnsafe src startSrc dst endDst srcLen
26642102 return $ dst {arrEnd = endDst + srcLen}
26652103
26662104-- | @spliceWith sizer dst src@ mutates @dst@ to append @src@. If there is no
@@ -2676,7 +2114,7 @@ spliceWith :: forall m a. (MonadIO m, Unbox a) =>
26762114 (Int -> Int -> Int) -> MutArray a -> MutArray a -> m (MutArray a)
26772115spliceWith sizer dst@(MutArray _ start end bound) src = do
26782116{-
2679− let f = appendWith (`sizer` byteLength src) (return dst)
2117+ let f = writeAppendWith (`sizer` byteLength src) (return dst)
26802118 in D.fold f (toStreamD src)
26812119-}
26822120 assert (end <= bound) (return ())
@@ -2692,7 +2130,7 @@ spliceWith sizer dst@(MutArray _ start end bound) src = do
26922130 $ "splice: newSize is less than the total size "
26932131 ++ "of arrays being appended. Please check the "
26942132 ++ "sizer function passed."
2695− realloc newSizeInBytes dst
2133+ liftIO $ realloc newSizeInBytes dst
26962134 else return dst
26972135 spliceUnsafe dst1 src
26982136
@@ -2704,8 +2142,6 @@ spliceWith sizer dst@(MutArray _ start end bound) src = do
27042142--
27052143-- >>> splice = MutArray.spliceWith (+)
27062144--
2707−-- If the original array is pinned the spliced array is also pinned.
2708−--
27092145-- /Pre-release/
27102146{-# INLINE splice #-}
27112147splice :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
@@ -2729,24 +2165,14 @@ spliceExp = spliceWith (\l1 l2 -> max (l1 * 2) (l1 + l2))
27292165-- Splitting
27302166-------------------------------------------------------------------------------
27312167
2732−-- | Generate a stream of array slices using a predicate. The array element
2733−-- matching the predicate is dropped.
2734−--
2735−-- /Pre-release/
2736−{-# INLINE splitOn #-}
2737−splitOn :: (MonadIO m, Unbox a) =>
2738− (a -> Bool) -> MutArray a -> Stream m (MutArray a)
2739−splitOn predicate arr =
2740− fmap (\(i, len) -> getSliceUnsafe i len arr)
2741− $ D.indexOnSuffix predicate (read arr)
2742−
27432168-- | Drops the separator byte
27442169{-# INLINE breakOn #-}
27452170breakOn :: MonadIO m
27462171 => Word8 -> MutArray Word8 -> m (MutArray Word8, Maybe (MutArray Word8))
2747−breakOn sep arr@MutArray{..} = unsafeAsPtr arr $ \p -> liftIO $ do
2172+breakOn sep arr@MutArray{..} = asPtrUnsafe arr $ \p -> liftIO $ do
2173+ -- XXX Instead of using asPtrUnsafe (pinning memory) we can pass unlifted
2174+ -- Addr# to memchr and it should be safe (from ghc 8.4).
27482175 -- XXX We do not need memchr here, we can use a Haskell equivalent.
2749− -- Need efficient stream based primitives that work on Word64.
27502176 loc <- c_memchr p sep (fromIntegral $ byteLength arr)
27512177 let sepIndex = loc `minusPtr` p
27522178 return $
@@ -2767,41 +2193,32 @@ breakOn sep arr@MutArray{..} = unsafeAsPtr arr $ \p -> liftIO $ do
27672193 }
27682194 )
27692195
2770−-- | Like 'splitAt' but does not check whether the index is valid.
2771−--
2772−{-# INLINE unsafeSplitAt #-}
2773−unsafeSplitAt :: forall a. Unbox a =>
2774− Int -> MutArray a -> (MutArray a, MutArray a)
2775−unsafeSplitAt i MutArray{..} =
2776− let off = i * SIZE_OF(a)
2777− p = arrStart + off
2778− in ( MutArray
2779− { arrContents = arrContents
2780− , arrStart = arrStart
2781− , arrEnd = p
2782− , arrBound = p
2783− }
2784− , MutArray
2785− { arrContents = arrContents
2786− , arrStart = p
2787− , arrEnd = arrEnd
2788− , arrBound = arrBound
2789− }
2790− )
2791−
27922196-- | Create two slices of an array without copying the original array. The
27932197-- specified index @i@ is the first index of the second slice.
27942198--
2795−{-# INLINE splitAt #-}
27962199splitAt :: forall a. Unbox a => Int -> MutArray a -> (MutArray a, MutArray a)
2797−splitAt i arr =
2200+splitAt i arr@MutArray{..} =
27982201 let maxIndex = length arr - 1
27992202 in if i < 0
28002203 then error "sliceAt: negative array index"
28012204 else if i > maxIndex
28022205 then error $ "sliceAt: specified array index " ++ show i
28032206 ++ " is beyond the maximum index " ++ show maxIndex
2804− else unsafeSplitAt i arr
2207+ else let off = i * SIZE_OF(a)
2208+ p = arrStart + off
2209+ in ( MutArray
2210+ { arrContents = arrContents
2211+ , arrStart = arrStart
2212+ , arrEnd = p
2213+ , arrBound = p
2214+ }
2215+ , MutArray
2216+ { arrContents = arrContents
2217+ , arrStart = p
2218+ , arrEnd = arrEnd
2219+ , arrBound = arrBound
2220+ }
2221+ )
28052222
28062223-------------------------------------------------------------------------------
28072224-- Casting
@@ -2851,16 +2268,13 @@ cast arr =
28512268-- guarantees that unsafe calls will be performed in the calling thread. Making
28522269-- it safe to pass heap-allocated objects to unsafe functions.
28532270
2854−-- XXX Should we just name it asPtr, the unsafety is implicit for any pointer
2855−-- operations. And we are safe from Haskell perspective because we will be
2856−-- pinning the memory.
2857−--
2858−-- XXX we cannot pass the length of the ptr here as in some cases it may not be
2859−-- available e.g. a null terminated C string. However, we can create another
2860−-- flavor of the API e.g. asPtrN.
2271+-- Unsafe because of direct pointer operations. The user must ensure that they
2272+-- are writing within the legal bounds of the array. Should we just name it
2273+-- asPtr, the unsafety is implicit for any pointer operations. And we are safe
2274+-- from Haskell perspective because we will be pinning the memory.
28612275
2862−-- | Use a @MutArray a@ as @Ptr a@. This is useful when we want to pass an
2863−-- array as a pointer to some operating system call or to a "safe" FFI call.
2276+-- | Use an @MutArray a@ as @Ptr a@. This is useful when we want to pass an array
2277+-- as a pointer to some operating system call or to a "safe" FFI call.
28642278--
28652279-- If the array is not pinned it is copied to pinned memory before passing it
28662280-- to the monadic action.
@@ -2870,382 +2284,56 @@ cast arr =
28702284-- opportunistically before this operation occurs, to avoid the cost of a copy
28712285-- if possible.
28722286--
2873−-- /Unsafe/ because of direct pointer operations. The user must ensure that
2874−-- they are writing within the legal bounds of the array.
2287+-- /Unsafe/
28752288--
28762289-- /Pre-release/
28772290--
2878−{-# INLINE unsafePinnedAsPtr #-}
2879−unsafePinnedAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2880−unsafePinnedAsPtr arr f =
2881− Unboxed.unsafePinnedAsPtr
2882− (arrContents arr) (\ptr -> f (ptr `plusPtr` arrStart arr))
2883−
2884−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
2885−{-# INLINE asPtrUnsafe #-}
28862291asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2887−asPtrUnsafe = unsafePinnedAsPtr
2292+asPtrUnsafe arr f = do
2293+ contents <- liftIO $ Unboxed.pin $ arrContents arr
2294+ let !ptr = Ptr (byteArrayContents#
2295+ (unsafeCoerce# (getMutableByteArray# contents)))
28882296
2889−{-# INLINE unsafeAsPtr #-}
2890−unsafeAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2891−unsafeAsPtr arr f =
2892− Unboxed.unsafeAsPtr
2893− (arrContents arr) (\ptr -> f (ptr `plusPtr` arrStart arr))
2297+ r <- f (ptr `plusPtr` arrStart arr)
2298+ liftIO $ touch contents
2299+ return r
28942300
28952301-------------------------------------------------------------------------------
28962302-- Equality
28972303-------------------------------------------------------------------------------
28982304
2899−-- | Byte compare two arrays. Compare the length of the arrays. If the length
2900−-- is equal, compare the lexicographical ordering of two underlying byte arrays
2901−-- otherwise return the result of length comparison.
2902−--
2903−-- /Unsafe/: Note that the 'Unbox' instance of sum types with constructors of
2904−-- different sizes may leave some memory uninitialized which can make byte
2905−-- comparison unreliable.
2305+-- | Compare the length of the arrays. If the length is equal, compare the
2306+-- lexicographical ordering of two underlying byte arrays otherwise return the
2307+-- result of length comparison.
29062308--
29072309-- /Pre-release/
2908−{-# INLINE byteCmp #-}
2909−byteCmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
2910−byteCmp arr1 arr2 = do
2911− let marr1 = getMutableByteArray# (arrContents arr1)
2912− marr2 = getMutableByteArray# (arrContents arr2)
2913− !(I# st1#) = arrStart arr1
2914− !(I# st2#) = arrStart arr2
2915− !(I# len#) = byteLength arr1
2916− case compare (byteLength arr1) (byteLength arr2) of
2917− EQ -> do
2918− r <- liftIO $ IO $ \s# ->
2919− let res =
2920− I#
2921− (compareByteArrays#
2922− (unsafeCoerce# marr1)
2923− st1#
2924− (unsafeCoerce# marr2)
2925− st2#
2926− len#)
2927− in (# s#, res #)
2928− return $ compare r 0
2929− x -> return x
2930−
29312310{-# INLINE cmp #-}
2932−{-# DEPRECATED cmp "Please use byteCmp instead." #-}
29332311cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
2934−cmp = byteCmp
2935−
2936−-- | Byte equality of two arrays.
2937−--
2938−-- >>> byteEq arr1 arr2 = (==) EQ $ MArray.byteCmp arr1 arr2
2939−--
2940−-- /Unsafe/: See 'byteCmp'.
2941−{-# INLINE byteEq #-}
2942−byteEq :: MonadIO m => MutArray a -> MutArray a -> m Bool
2943−byteEq arr1 arr2 = fmap (EQ ==) $ byteCmp arr1 arr2
2944−
2945−-------------------------------------------------------------------------------
2946−-- Compact
2947−-------------------------------------------------------------------------------
2948−
2949−-- Note: LE versions avoid an extra copy compared to GE. LE parser trades
2950−-- backtracking one array in lieu of avoiding a copy. However, LE and GE both
2951−-- can leave some memory unused. They can split the last array to fit it
2952−-- exactly in the space.
2953−
2954−{-# INLINE_NORMAL pCompactLeAs #-}
2955−pCompactLeAs ::
2956− forall m a. (MonadIO m, Unbox a)
2957− => PinnedState -> Int -> Parser (MutArray a) m (MutArray a)
2958−pCompactLeAs ps maxElems = Parser step initial extract
2959−
2960− where
2961−
2962− maxBytes = maxElems * SIZE_OF(a)
2963−
2964− functionName = "Streamly.Internal.Data.MutArray.pCompactLE"
2965−
2966− initial =
2967− return
2968− $ if maxElems <= 0
2969− then error
2970− $ functionName
2971− ++ ": the size of arrays ["
2972− ++ show maxElems ++ "] must be a natural number"
2973− else Parser.IPartial Nothing
2974−
2975− step Nothing arr =
2976− return
2977− $ let len = byteLength arr
2978− in if len >= maxBytes
2979− then Parser.Done 0 arr
2980− else Parser.Partial 0 (Just arr)
2981− -- XXX Split the last array to use the space more compactly.
2982− step (Just buf) arr =
2983− let len = byteLength buf + byteLength arr
2984− in if len > maxBytes
2985− then return $ Parser.Done 1 buf
2986− else do
2987− buf1 <-
2988− if byteCapacity buf < maxBytes
2989− then liftIO $ reallocExplicitAs
2990− ps (SIZE_OF(a)) maxBytes buf
2991− else return buf
2992− buf2 <- spliceUnsafe buf1 arr
2993− return $ Parser.Partial 0 (Just buf2)
2994−
2995− extract Nothing = return $ Parser.Done 0 nil
2996− extract (Just buf) = return $ Parser.Done 0 buf
2997−
2998−-- | Parser @pCompactLE maxElems@ coalesces adjacent arrays in the input stream
2999−-- only if the combined size would be less than or equal to @maxElems@
3000−-- elements. Note that it won't split an array if the original array is already
3001−-- larger than maxElems.
3002−--
3003−-- @maxElems@ must be greater than 0.
3004−--
3005−-- Generates unpinned arrays irrespective of the pinning status of input
3006−-- arrays.
3007−--
3008−-- /Internal/
3009−{-# INLINE pCompactLE #-}
3010−pCompactLE ::
3011− forall m a. (MonadIO m, Unbox a)
3012− => Int -> Parser (MutArray a) m (MutArray a)
3013−pCompactLE = pCompactLeAs Unpinned
3014−
3015−-- | Pinned version of 'pCompactLE'.
3016−{-# INLINE pPinnedCompactLE #-}
3017−pPinnedCompactLE ::
3018− forall m a. (MonadIO m, Unbox a)
3019− => Int -> Parser (MutArray a) m (MutArray a)
3020−pPinnedCompactLE = pCompactLeAs Pinned
3021−
3022−data SpliceState s arr
3023− = SpliceInitial s
3024− | SpliceBuffering s arr
3025− | SpliceYielding arr (SpliceState s arr)
3026− | SpliceFinish
3027−
3028−-- This mutates the first array (if it has space) to append values from the
3029−-- second one. This would work for immutable arrays as well because an
3030−-- immutable array never has additional space so a new array is allocated
3031−-- instead of mutating it.
3032−
3033−{-# INLINE_NORMAL compactLeAs #-}
3034−compactLeAs :: forall m a. (MonadIO m, Unbox a)
3035− => PinnedState -> Int -> D.Stream m (MutArray a) -> D.Stream m (MutArray a)
3036−compactLeAs ps maxElems (D.Stream step state) =
3037− D.Stream step' (SpliceInitial state)
3038−
3039− where
3040−
3041− maxBytes = maxElems * SIZE_OF(a)
3042−
3043− functionName = "Streamly.Internal.Data.MutArray.rCompactLE"
3044−
3045− {-# INLINE_LATE step' #-}
3046− step' gst (SpliceInitial st) = do
3047− when (maxElems <= 0) $
3048− -- XXX we can pass the module string from the higher level API
3049− error $ functionName ++ ": the size of arrays [" ++ show maxElems
3050− ++ "] must be a natural number"
3051− r <- step gst st
3052− case r of
3053− D.Yield arr s -> return $
3054− let len = byteLength arr
3055− in if len >= maxBytes
3056− then D.Skip (SpliceYielding arr (SpliceInitial s))
3057− else D.Skip (SpliceBuffering s arr)
3058− D.Skip s -> return $ D.Skip (SpliceInitial s)
3059− D.Stop -> return D.Stop
3060−
3061− -- XXX Split the last array to use the space more compactly.
3062− step' gst (SpliceBuffering st buf) = do
3063− r <- step gst st
3064− case r of
3065− D.Yield arr s -> do
3066− let len = byteLength buf + byteLength arr
3067− if len > maxBytes
3068− then return $
3069− D.Skip (SpliceYielding buf (SpliceBuffering s arr))
3070− else do
3071− buf1 <- if byteCapacity buf < maxBytes
3072− then liftIO $ reallocExplicitAs
3073− ps (SIZE_OF(a)) maxBytes buf
3074− else return buf
3075− buf2 <- spliceUnsafe buf1 arr
3076− return $ D.Skip (SpliceBuffering s buf2)
3077− D.Skip s -> return $ D.Skip (SpliceBuffering s buf)
3078− D.Stop -> return $ D.Skip (SpliceYielding buf SpliceFinish)
3079−
3080− step' _ SpliceFinish = return D.Stop
3081−
3082− step' _ (SpliceYielding arr next) = return $ D.Yield arr next
3083−
3084−
3085−{-# INLINE_NORMAL fCompactGeAs #-}
3086−fCompactGeAs ::
3087− forall m a. (MonadIO m, Unbox a)
3088− => PinnedState -> Int -> FL.Fold m (MutArray a) (MutArray a)
3089−fCompactGeAs ps minElems = Fold step initial extract extract
3090−
3091− where
3092−
3093− minBytes = minElems * SIZE_OF(a)
3094−
3095− functionName = "Streamly.Internal.Data.MutArray.fCompactGE"
3096−
3097− initial =
3098− return
3099− $ if minElems < 0
3100− then error
3101− $ functionName
3102− ++ ": the size of arrays ["
3103− ++ show minElems ++ "] must be a natural number"
3104− else FL.Partial Nothing
3105−
3106− step Nothing arr =
3107− return
3108− $ let len = byteLength arr
3109− in if len >= minBytes
3110− then FL.Done arr
3111− else FL.Partial (Just arr)
3112− -- XXX Buffer arrays as a list to avoid copy and reallocations
3113− step (Just buf) arr = do
3114− let len = byteLength buf + byteLength arr
3115− buf1 <-
3116− if byteCapacity buf < len
3117− then liftIO $ reallocExplicitAs
3118− ps (SIZE_OF(a)) (max minBytes len) buf
3119− else return buf
3120− buf2 <- spliceUnsafe buf1 arr
3121− if len >= minBytes
3122− then return $ FL.Done buf2
3123− else return $ FL.Partial (Just buf2)
3124−
3125− extract Nothing = return nil
3126− extract (Just buf) = return buf
3127−
3128−-- | Fold @fCompactGE minElems@ coalesces adjacent arrays in the input stream
3129−-- until the size becomes greater than or equal to @minElems@.
3130−--
3131−-- Generates unpinned arrays irrespective of the pinning status of input
3132−-- arrays.
3133−{-# INLINE fCompactGE #-}
3134−fCompactGE ::
3135− forall m a. (MonadIO m, Unbox a)
3136− => Int -> FL.Fold m (MutArray a) (MutArray a)
3137−fCompactGE = fCompactGeAs Unpinned
3138−
3139−-- | Pinned version of 'fCompactGE'.
3140−{-# INLINE fPinnedCompactGE #-}
3141−fPinnedCompactGE ::
3142− forall m a. (MonadIO m, Unbox a)
3143− => Int -> FL.Fold m (MutArray a) (MutArray a)
3144−fPinnedCompactGE = fCompactGeAs Pinned
3145−
3146−{-# INLINE_NORMAL lCompactGeAs #-}
3147−lCompactGeAs :: forall m a. (MonadIO m, Unbox a)
3148− => PinnedState -> Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3149−-- The fold version turns out to be a little bit slower.
3150−-- lCompactGeAs ps n = FL.many (fCompactGeAs ps n)
3151−lCompactGeAs ps minElems (Fold step1 initial1 _ final1) =
3152− Fold step initial extract final
3153−
3154− where
3155−
3156− minBytes = minElems * SIZE_OF(a)
3157−
3158− functionName = "Streamly.Internal.Data.MutArray.lCompactGE"
3159−
3160− initial = do
3161− when (minElems <= 0) $
3162− -- XXX we can pass the module string from the higher level API
3163− error $ functionName ++ ": the size of arrays ["
3164− ++ show minElems ++ "] must be a natural number"
3165−
3166− r <- initial1
3167− return $ first (Tuple' Nothing) r
3168−
3169− {-# INLINE runInner #-}
3170− runInner len acc buf =
3171− if len >= minBytes
3172− then do
3173− r <- step1 acc buf
3174− case r of
3175− FL.Done _ -> return $ FL.Done ()
3176− FL.Partial s -> do
3177− _ <- final1 s
3178− res <- initial1
3179− return $ first (Tuple' Nothing) res
3180− else return $ FL.Partial $ Tuple' (Just buf) acc
3181−
3182− step (Tuple' Nothing r1) arr =
3183− runInner (byteLength arr) r1 arr
3184−
3185− -- XXX Buffer arrays as a list to avoid copy and reallocations
3186− step (Tuple' (Just buf) r1) arr = do
3187− let len = byteLength buf + byteLength arr
3188− buf1 <- if byteCapacity buf < len
3189− then liftIO $ reallocExplicitAs
3190− ps (SIZE_OF(a)) (max minBytes len) buf
3191− else return buf
3192− buf2 <- spliceUnsafe buf1 arr
3193− runInner len r1 buf2
3194−
3195− -- XXX Several folds do extract >=> final, therefore, we need to make final
3196− -- return "m b" rather than using extract post it if we want extract to be
3197− -- partial.
3198− --
3199− -- extract forces the pending buffer to be sent to the fold which is not
3200− -- what we want.
3201− extract _ = error "lCompactGE: not designed for scanning"
3202−
3203− final (Tuple' Nothing r1) = final1 r1
3204− final (Tuple' (Just buf) r1) = do
3205− r <- step1 r1 buf
3206− case r of
3207− FL.Partial rr -> final1 rr
3208− FL.Done _ -> return ()
3209−
3210−-- | Like 'compactGE' but for transforming folds instead of stream.
3211−--
3212−-- >>> lCompactGE n = Fold.many (MutArray.fCompactGE n)
3213−--
3214−-- Generates unpinned arrays irrespective of the pinning status of input
3215−-- arrays.
3216−{-# INLINE lCompactGE #-}
3217−lCompactGE :: forall m a. (MonadIO m, Unbox a)
3218− => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3219−lCompactGE = lCompactGeAs Unpinned
3220−
3221−-- | Pinned version of 'lCompactGE'.
3222−{-# INLINE lPinnedCompactGE #-}
3223−lPinnedCompactGE :: forall m a. (MonadIO m, Unbox a)
3224− => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3225−lPinnedCompactGE = lCompactGeAs Pinned
3226−
3227−-- | @compactGE n stream@ coalesces adjacent arrays in the @stream@ until
3228−-- the size becomes greater than or equal to @n@.
3229−--
3230−-- >>> compactGE n = Stream.foldMany (MutArray.fCompactGE n)
3231−--
3232−{-# INLINE compactGE #-}
3233−compactGE ::
3234− (MonadIO m, Unbox a)
3235− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
3236−compactGE n = D.foldMany (fCompactGE n)
3237−
3238−-- | 'compactEQ n' coalesces adajacent arrays in the input stream to
3239−-- arrays of exact size @n@.
3240−--
3241−-- /Unimplemented/
3242−{-# INLINE compactEQ #-}
3243−compactEQ :: -- (MonadIO m, Unbox a) =>
3244− Int -> Stream m (MutArray a) -> Stream m (MutArray a)
3245−compactEQ _n = undefined -- D.parseManyD (pCompactEQ n)
2312+cmp arr1 arr2 =
2313+ liftIO
2314+ $ do
2315+ let marr1 = getMutableByteArray# (arrContents arr1)
2316+ marr2 = getMutableByteArray# (arrContents arr2)
2317+ !(I# st1#) = arrStart arr1
2318+ !(I# st2#) = arrStart arr2
2319+ !(I# len#) = byteLength arr1
2320+ case compare (byteLength arr1) (byteLength arr2) of
2321+ EQ -> do
2322+ r <- liftIO $ IO $ \s# ->
2323+ let res =
2324+ I#
2325+ (compareByteArrays#
2326+ (unsafeCoerce# marr1)
2327+ st1#
2328+ (unsafeCoerce# marr2)
2329+ st2#
2330+ len#)
2331+ in (# s#, res #)
2332+ return $ compare r 0
2333+ x -> return x
32462334
32472335-------------------------------------------------------------------------------
3248−-- In-place mutation algorithms
2336+-- NFData
32492337-------------------------------------------------------------------------------
32502338
32512339-- | Strip elements which match with predicate from both ends.
@@ -3274,7 +2362,7 @@ strip eq arr@MutArray{..} = liftIO $ do
32742362 getStart cur = do
32752363 if cur < arrEnd
32762364 then do
3277− r <- peekAt cur arrContents
2365+ r <- peekByteIndex cur arrContents
32782366 if eq r
32792367 then getStart (INDEX_NEXT(cur,a))
32802368 else return cur
@@ -3284,7 +2372,7 @@ strip eq arr@MutArray{..} = liftIO $ do
32842372 if cur > low
32852373 then do
32862374 let prev = INDEX_PREV(cur,a)
3287− r <- peekAt prev arrContents
2375+ r <- peekByteIndex prev arrContents
32882376 if eq r
32892377 then getLast prev low
32902378 else return cur
core/src/Streamly/Internal/Data/Array/Type.hs+122−570
@@ -1,158 +1,84 @@
11{-# LANGUAGE CPP #-}
2−{-# LANGUAGE TypeFamilies #-}
3−-- Must come after TypeFamilies, otherwise it is re-enabled.
4−-- MonoLocalBinds enabled by TypeFamilies causes perf regressions in general.
5−{-# LANGUAGE NoMonoLocalBinds #-}
62-- |
73-- Module : Streamly.Internal.Data.Array.Type
84-- Copyright : (c) 2020 Composewell Technologies
5+--
96-- License : BSD3-3-Clause
107-- Maintainer : streamly@composewell.com
118-- Stability : experimental
129-- Portability : GHC
1310--
14−-- See notes in "Streamly.Internal.Data.MutArray.Type"
11+-- See notes in "Streamly.Internal.Data.Array.Mut.Type"
1512--
16−
1713module Streamly.Internal.Data.Array.Type
1814 (
19− -- ** Type
2015 -- $arrayNotes
2116 Array (..)
17+ , asPtrUnsafe
2218
23− -- ** Conversion
24− -- *** Mutable and Immutable
19+ -- * Freezing and Thawing
2520 , unsafeFreeze
2621 , unsafeFreezeWithShrink
2722 , unsafeThaw
2823
29− -- *** Pinned and Unpinned
24+ -- * Pinning and Unpinning
3025 , pin
3126 , unpin
3227 , isPinned
3328
34− -- *** Casting
35− , unsafePinnedAsPtr
36−
37− -- ** Construction
38− , empty
39−
40− -- *** Cloning
41− -- XXX Why would we clone an immutable array?
42− , clone
43− , pinnedClone
29+ -- * Construction
30+ , splice
4431
45− -- *** Slicing
46− -- | Get a subarray without copying
47− , splitAt
48− , breakOn -- XXX requires MonadIO
49−
50− -- *** Stream Folds
51− , unsafeMakePure
52− , createOf
53− , pinnedCreateOf
54− , unsafeCreateOf
55− , unsafePinnedCreateOf
56− , create
57− , pinnedCreate
58− , createWith
59−
60− -- *** From containers
61− , fromListN
62− , pinnedFromListN
6332 , fromList
64− , pinnedFromList
65− , fromListRevN
33+ , fromListN
6634 , fromListRev
67− , fromStreamN
68− , fromStream
69− , fromPureStreamN
35+ , fromListRevN
36+ , fromStreamDN
37+ , fromStreamD
7038 , fromPureStream
7139 , fromByteStr#
72− , fromByteStr
73− , fromPtrN
74− , fromChunks
75− , fromChunksK
76−
77− -- ** Reading
7840
79− -- *** Indexing
80− , unsafeIndexIO -- XXX unsafeGetIndexIO
81− , getIndexUnsafe -- XXX unsafeGetIndex
41+ -- * Split
42+ , breakOn
8243
83− -- *** To Streams
84− , read
85− , readRev
86− , toStreamK
87− , toStreamKRev
88−
89− -- *** To Containers
90− , toList
91−
92− -- *** Unfolds
93− , producer -- experimental
94− , readerUnsafe
95− , reader
96− , readerRev
97−
98− -- *** Size
99− , length
44+ -- * Elimination
45+ , unsafeIndexIO
46+ , unsafeIndex -- getIndexUnsafe
10047 , byteLength
48+ , length
10149
102− -- ** Folding
10350 , foldl'
10451 , foldr
105− , byteCmp
106− , byteEq
107−
108− -- ** Appending
109− , splice -- XXX requires MonadIO
52+ , splitAt
11053
111− -- ** Streams of arrays
112− -- *** Chunk
113− -- | Group a stream into arrays.
114− , chunksOf
115− , pinnedChunksOf
116− , buildChunks
117−
118− -- *** Split
119− -- | Split an array into slices.
120−
121− -- *** Concat
122− -- | Append the arrays in a stream to form a stream of elements.
123− , concat
124− , concatRev
125−
126− -- *** Compact
127− -- | Append the arrays in a stream to form a stream of larger arrays.
128− , fCompactGE
129− , fPinnedCompactGE
130− , lCompactGE
131− , lPinnedCompactGE
132− , compactGE
133−
134− -- ** Deprecated
135− , asPtrUnsafe
136− , unsafeIndex
137− , bufferChunks
138− , flattenArrays
139− , flattenArraysRev
140− , fromArrayStreamK
141− , fromStreamDN
142− , fromStreamD
14354 , toStreamD
14455 , toStreamDRev
56+ , toStreamK
57+ , toStreamKRev
14558 , toStream
14659 , toStreamRev
147− , nil
60+ , read
61+ , readRev
62+ , readerRev
63+ , toList
64+
65+ -- * Folds
14866 , writeWith
14967 , writeN
150− , pinnedWriteN
68+ , writeNAs
15169 , writeNUnsafe
152− , pinnedWriteNUnsafe
70+ , writeNUnsafeAs
71+ , MA.ArrayUnsafe (..)
15372 , pinnedWriteNAligned
15473 , write
155− , pinnedWrite
74+ , unsafeMakePure
75+
76+ -- * Streams of arrays
77+ , chunksOf
78+ , chunksOfAs
79+ , bufferChunks
80+ , flattenArrays
81+ , flattenArraysRev
15682 )
15783where
15884
@@ -160,36 +86,31 @@ where
16086#include "inline.hs"
16187
16288import Control.Exception (assert)
163−import Control.Monad (replicateM, when)
89+import Control.Monad (replicateM)
16490import Control.Monad.IO.Class (MonadIO(..))
16591import Data.Functor.Identity (Identity(..))
166−import Data.Int (Int8, Int16, Int32, Int64)
16792import Data.Proxy (Proxy(..))
168−import Data.Word (Word8, Word16, Word32, Word64)
93+import Data.Word (Word8)
16994import GHC.Base (build)
17095import GHC.Exts (IsList, IsString(..), Addr#)
17196
17297import GHC.IO (unsafePerformIO)
17398import GHC.Ptr (Ptr(..))
174−import Streamly.Internal.Data.Producer.Type (Producer(..))
175−import Streamly.Internal.Data.MutArray.Type (MutArray(..))
176−import Streamly.Internal.Data.MutByteArray.Type (MutByteArray)
99+import Streamly.Internal.Data.Array.Mut.Type (MutArray(..), MutableByteArray)
177100import Streamly.Internal.Data.Fold.Type (Fold(..))
178−import Streamly.Internal.Data.Stream.Type (Stream)
179−import Streamly.Internal.Data.StreamK.Type (StreamK)
180−import Streamly.Internal.Data.Unbox (Unbox(..))
101+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
102+import Streamly.Internal.Data.Unbox (PinnedState, Unbox(..))
181103import Streamly.Internal.Data.Unfold.Type (Unfold(..))
182104import Text.Read (readPrec)
183105
184−import Prelude hiding (Foldable(..), concat, read, unlines, splitAt)
106+import Prelude hiding (Foldable(..), read, unlines, splitAt)
185107
186108import qualified GHC.Exts as Exts
187−import qualified Streamly.Internal.Data.Fold.Type as Fold
188−import qualified Streamly.Internal.Data.MutArray.Type as MA
189−import qualified Streamly.Internal.Data.Stream.Type as D
190−import qualified Streamly.Internal.Data.StreamK.Type as K
191−import qualified Streamly.Internal.Data.MutByteArray.Type as Unboxed
192−import qualified Streamly.Internal.Data.Producer as Producer
109+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
110+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
111+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
112+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
113+import qualified Streamly.Internal.Data.Unbox as Unboxed
193114import qualified Streamly.Internal.Data.Unfold.Type as Unfold
194115import qualified Text.ParserCombinators.ReadPrec as ReadPrec
195116
@@ -232,7 +153,7 @@ data Array a =
232153#endif
233154 -- All offsets are in terms of bytes from the start of arraycontents
234155 Array
235− { arrContents :: {-# UNPACK #-} !MutByteArray
156+ { arrContents :: {-# UNPACK #-} !MutableByteArray
236157 , arrStart :: {-# UNPACK #-} !Int -- offset
237158 , arrEnd :: {-# UNPACK #-} !Int -- offset + len
238159 }
@@ -243,20 +164,14 @@ data Array a =
243164
244165-- | Use an @Array a@ as @Ptr a@.
245166--
246−-- See 'MA.unsafePinnedAsPtr' in the Mutable array module for more details.
167+-- See 'MA.asPtrUnsafe' in the Mutable array module for more details.
247168--
248169-- /Unsafe/
249170--
250171-- /Pre-release/
251172--
252−{-# INLINE unsafePinnedAsPtr #-}
253−unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
254−unsafePinnedAsPtr arr = MA.unsafePinnedAsPtr (unsafeThaw arr)
255−
256−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
257−{-# INLINE asPtrUnsafe #-}
258173asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
259−asPtrUnsafe = unsafePinnedAsPtr
174+asPtrUnsafe arr = MA.asPtrUnsafe (unsafeThaw arr)
260175
261176-------------------------------------------------------------------------------
262177-- Freezing and Thawing
@@ -303,19 +218,18 @@ unsafeThaw (Array ac as ae) = MutArray ac as ae ae
303218-- Pinning & Unpinning
304219-------------------------------------------------------------------------------
305220
306−-- | Return a copy of the 'Array' in pinned memory if unpinned, else return the
307−-- original array.
221+-- | Copy the 'Array' to pinned memory if unpinned, else do nothing. The
222+-- overhead is a copy if the input array is unpinned.
308223{-# INLINE pin #-}
309224pin :: Array a -> IO (Array a)
310225pin = fmap unsafeFreeze . MA.pin . unsafeThaw
311226
312−-- | Return a copy of the 'Array' in unpinned memory if pinned, else return the
313−-- original array.
227+-- | Copy the 'Array' to unpinned memory if pinned, else do nothing. The
228+-- overhead is a copy if the input array is pinned.
314229{-# INLINE unpin #-}
315230unpin :: Array a -> IO (Array a)
316231unpin = fmap unsafeFreeze . MA.unpin . unsafeThaw
317232
318−-- | Return 'True' if the array is allocated in pinned memory.
319233{-# INLINE isPinned #-}
320234isPinned :: Array a -> Bool
321235isPinned = MA.isPinned . unsafeThaw
@@ -324,14 +238,11 @@ isPinned = MA.isPinned . unsafeThaw
324238-- Construction
325239-------------------------------------------------------------------------------
326240
327−-- | Copy two immutable arrays into a new array. If you want to splice more
328−-- than two arrays then this operation would be highly inefficient because it
329−-- would make a copy on every splice operation, instead use the
330−-- 'fromChunksK' operation to combine n immutable arrays.
241+-- Splice two immutable arrays creating a new array.
331242{-# INLINE splice #-}
332−splice :: MonadIO m => Array a -> Array a -> m (Array a)
243+splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
333244splice arr1 arr2 =
334− unsafeFreeze <$> MA.spliceCopy (unsafeThaw arr1) (unsafeThaw arr2)
245+ unsafeFreeze <$> MA.splice (unsafeThaw arr1) (unsafeThaw arr2)
335246
336247-- | Create an 'Array' from the first N elements of a list. The array is
337248-- allocated to size N, if the list terminates before N elements then the
@@ -341,12 +252,6 @@ splice arr1 arr2 =
341252fromListN :: Unbox a => Int -> [a] -> Array a
342253fromListN n xs = unsafePerformIO $ unsafeFreeze <$> MA.fromListN n xs
343254
344−-- | Like 'fromListN' but creates a pinned array.
345−{-# INLINABLE pinnedFromListN #-}
346−pinnedFromListN :: Unbox a => Int -> [a] -> Array a
347−pinnedFromListN n xs =
348− unsafePerformIO $ unsafeFreeze <$> MA.pinnedFromListN n xs
349−
350255-- | Create an 'Array' from the first N elements of a list in reverse order.
351256-- The array is allocated to size N, if the list terminates before N elements
352257-- then the array may hold less than N elements.
@@ -362,11 +267,6 @@ fromListRevN n xs = unsafePerformIO $ unsafeFreeze <$> MA.fromListRevN n xs
362267fromList :: Unbox a => [a] -> Array a
363268fromList xs = unsafePerformIO $ unsafeFreeze <$> MA.fromList xs
364269
365−-- | Like 'fromList' but creates a pinned array.
366−{-# INLINE pinnedFromList #-}
367−pinnedFromList :: Unbox a => [a] -> Array a
368−pinnedFromList xs = unsafePerformIO $ unsafeFreeze <$> MA.pinnedFromList xs
369−
370270-- | Create an 'Array' from a list in reverse order. The list must be of finite
371271-- size.
372272--
@@ -375,83 +275,29 @@ pinnedFromList xs = unsafePerformIO $ unsafeFreeze <$> MA.pinnedFromList xs
375275fromListRev :: Unbox a => [a] -> Array a
376276fromListRev xs = unsafePerformIO $ unsafeFreeze <$> MA.fromListRev xs
377277
378−-- | Create an 'Array' from the first N elements of a stream. The array is
379−-- allocated to size N, if the stream terminates before N elements then the
380−-- array may hold less than N elements.
381−--
382−-- >>> fromStreamN n = Stream.fold (Array.writeN n)
383−--
384−-- /Pre-release/
385−{-# INLINE_NORMAL fromStreamN #-}
386−fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
387−fromStreamN n m = do
388− when (n < 0) $ error "writeN: negative write count specified"
389− unsafeFreeze <$> MA.fromStreamN n m
390−-- fromStreamN n = D.fold (writeN n)
391−
392−{-# DEPRECATED fromStreamDN "Please use fromStreamN instead." #-}
278+{-# INLINE_NORMAL fromStreamDN #-}
393279fromStreamDN :: forall m a. (MonadIO m, Unbox a)
394280 => Int -> D.Stream m a -> m (Array a)
395−fromStreamDN = fromStreamN
281+fromStreamDN limit str = unsafeFreeze <$> MA.fromStreamDN limit str
396282
397−-- | Create an 'Array' from a stream. This is useful when we want to create a
398−-- single array from a stream of unknown size. 'writeN' is at least twice
399−-- as efficient when the size is already known.
400−--
401−-- >>> fromStream = Stream.fold Array.write
402−--
403−-- Note that if the input stream is too large memory allocation for the array
404−-- may fail. When the stream size is not known, `chunksOf` followed by
405−-- processing of indvidual arrays in the resulting stream should be preferred.
406−--
407−-- /Pre-release/
408283{-# INLINE_NORMAL fromStreamD #-}
409−fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
410−fromStream = D.fold write
411−-- fromStreamD str = unsafeFreeze <$> MA.fromStream str
412−
413−{-# DEPRECATED fromStreamD "Please use fromStream instead." #-}
414284fromStreamD :: forall m a. (MonadIO m, Unbox a)
415285 => D.Stream m a -> m (Array a)
416−fromStreamD = fromStream
417−
418−-------------------------------------------------------------------------------
419−-- Cloning
420−-------------------------------------------------------------------------------
421−
422−{-# INLINE clone #-}
423−clone ::
424− ( MonadIO m
425−#ifdef DEVBUILD
426− , Unbox a
427−#endif
428− )
429− => Array a -> m (Array a)
430−clone = fmap unsafeFreeze . MA.clone . unsafeThaw
431−
432−{-# INLINE pinnedClone #-}
433−pinnedClone ::
434− ( MonadIO m
435−#ifdef DEVBUILD
436− , Unbox a
437−#endif
438− )
439− => Array a -> m (Array a)
440−pinnedClone = fmap unsafeFreeze . MA.pinnedClone . unsafeThaw
286+fromStreamD str = unsafeFreeze <$> MA.fromStreamD str
441287
442288-------------------------------------------------------------------------------
443289-- Streams of arrays
444290-------------------------------------------------------------------------------
445291
446−{-# INLINE buildChunks #-}
447−buildChunks :: (MonadIO m, Unbox a) =>
448− D.Stream m a -> m (K.StreamK m (Array a))
449−buildChunks m = D.foldr K.cons K.nil $ chunksOf defaultChunkSize m
450−
451−{-# DEPRECATED bufferChunks "Please use buildChunks instead." #-}
292+{-# INLINE bufferChunks #-}
452293bufferChunks :: (MonadIO m, Unbox a) =>
453294 D.Stream m a -> m (K.StreamK m (Array a))
454−bufferChunks = buildChunks
295+bufferChunks m = D.foldr K.cons K.nil $ chunksOf defaultChunkSize m
296+
297+{-# INLINE_NORMAL chunksOfAs #-}
298+chunksOfAs :: forall m a. (MonadIO m, Unbox a)
299+ => PinnedState -> Int -> D.Stream m a -> D.Stream m (Array a)
300+chunksOfAs ps n str = D.map unsafeFreeze $ MA.chunksOfAs ps n str
455301
456302-- | @chunksOf n stream@ groups the elements in the input stream into arrays of
457303-- @n@ elements each.
@@ -466,106 +312,27 @@ chunksOf :: forall m a. (MonadIO m, Unbox a)
466312 => Int -> D.Stream m a -> D.Stream m (Array a)
467313chunksOf n str = D.map unsafeFreeze $ MA.chunksOf n str
468314
469−-- | Like 'chunksOf' but creates pinned arrays.
470−{-# INLINE_NORMAL pinnedChunksOf #-}
471−pinnedChunksOf :: forall m a. (MonadIO m, Unbox a)
472− => Int -> D.Stream m a -> D.Stream m (Array a)
473−pinnedChunksOf n str = D.map unsafeFreeze $ MA.pinnedChunksOf n str
474−
475−-- | Convert a stream of arrays into a stream of their elements.
315+-- | Use the "read" unfold instead.
316+--
317+-- @flattenArrays = unfoldMany read@
476318--
477−-- >>> concat = Stream.unfoldMany Array.reader
319+-- We can try this if there are any fusion issues in the unfold.
478320--
479−{-# INLINE_NORMAL concat #-}
480−concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
481−-- XXX this requires MonadIO whereas the unfoldMany version does not
482−concat = MA.concatWith (pure . unsafeInlineIO) . D.map unsafeThaw
483−-- concat = D.unfoldMany reader
484−
485−{-# DEPRECATED flattenArrays "Please use \"unfoldMany reader\" instead." #-}
486−{-# INLINE flattenArrays #-}
321+{-# INLINE_NORMAL flattenArrays #-}
487322flattenArrays :: forall m a. (MonadIO m, Unbox a)
488323 => D.Stream m (Array a) -> D.Stream m a
489−flattenArrays = concat
324+flattenArrays = MA.flattenArrays . D.map unsafeThaw
490325
491−-- | Convert a stream of arrays into a stream of their elements reversing the
492−-- contents of each array before flattening.
326+-- | Use the "readRev" unfold instead.
493327--
494−-- >>> concatRev = Stream.unfoldMany Array.readerRev
328+-- @flattenArrays = unfoldMany readRev@
495329--
496−{-# INLINE_NORMAL concatRev #-}
497−concatRev :: forall m a. (Monad m, Unbox a)
498− => D.Stream m (Array a) -> D.Stream m a
499−-- XXX this requires MonadIO whereas the unfoldMany version does not
500−concatRev = MA.concatRevWith (pure . unsafeInlineIO) . D.map unsafeThaw
501−-- concatRev = D.unfoldMany readerRev
502−
503−{-# DEPRECATED flattenArraysRev "Please use \"unfoldMany readerRev\" instead." #-}
504−{-# INLINE flattenArraysRev #-}
330+-- We can try this if there are any fusion issues in the unfold.
331+--
332+{-# INLINE_NORMAL flattenArraysRev #-}
505333flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
506334 => D.Stream m (Array a) -> D.Stream m a
507−flattenArraysRev = concatRev
508−
509−-------------------------------------------------------------------------------
510−-- Compact
511−-------------------------------------------------------------------------------
512−
513−-- XXX Note that this thaws immutable arrays for appending, that may be
514−-- problematic if multiple users do the same thing, however, thawed immutable
515−-- arrays would have no capacity to append, therefore, a copy will be forced
516−-- anyway.
517−
518−-- | Fold @fCompactGE n@ coalesces adjacent arrays in the input stream
519−-- until the size becomes greater than or equal to n.
520−--
521−-- Generates unpinned arrays irrespective of the pinning status of input
522−-- arrays.
523−{-# INLINE_NORMAL fCompactGE #-}
524−fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
525−fCompactGE n = fmap unsafeFreeze $ Fold.lmap unsafeThaw $ MA.fCompactGE n
526−
527−-- | PInned version of 'fCompactGE'.
528−{-# INLINE_NORMAL fPinnedCompactGE #-}
529−fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
530−fPinnedCompactGE n =
531− fmap unsafeFreeze $ Fold.lmap unsafeThaw $ MA.fPinnedCompactGE n
532−
533−-- | @compactGE n stream@ coalesces adjacent arrays in the @stream@ until
534−-- the size becomes greater than or equal to @n@.
535−--
536−-- >>> compactGE n = Stream.foldMany (Array.fCompactGE n)
537−--
538−-- Generates unpinned arrays irrespective of the pinning status of input
539−-- arrays.
540−{-# INLINE compactGE #-}
541−compactGE ::
542− (MonadIO m, Unbox a)
543− => Int -> Stream m (Array a) -> Stream m (Array a)
544−compactGE n stream =
545− D.map unsafeFreeze $ MA.compactGE n $ D.map unsafeThaw stream
546−
547−-- | Like 'compactGE' but for transforming folds instead of stream.
548−--
549−-- >>> lCompactGE n = Fold.many (Array.fCompactGE n)
550−--
551−-- Generates unpinned arrays irrespective of the pinning status of input
552−-- arrays.
553−{-# INLINE_NORMAL lCompactGE #-}
554−lCompactGE :: (MonadIO m, Unbox a)
555− => Int -> Fold m (Array a) () -> Fold m (Array a) ()
556−lCompactGE n fld =
557− Fold.lmap unsafeThaw $ MA.lCompactGE n (Fold.lmap unsafeFreeze fld)
558−
559−-- | Pinned version of 'lCompactGE'.
560−{-# INLINE_NORMAL lPinnedCompactGE #-}
561−lPinnedCompactGE :: (MonadIO m, Unbox a)
562− => Int -> Fold m (Array a) () -> Fold m (Array a) ()
563−lPinnedCompactGE n fld =
564− Fold.lmap unsafeThaw $ MA.lPinnedCompactGE n (Fold.lmap unsafeFreeze fld)
565−
566−-------------------------------------------------------------------------------
567−-- Splitting
568−-------------------------------------------------------------------------------
335+flattenArraysRev = MA.flattenArraysRev . D.map unsafeThaw
569336
570337-- Drops the separator byte
571338{-# INLINE breakOn #-}
@@ -587,14 +354,9 @@ unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a
587354unsafeIndexIO i arr = MA.getIndexUnsafe i (unsafeThaw arr)
588355
589356-- | Return element at the specified index without checking the bounds.
590−{-# INLINE_NORMAL getIndexUnsafe #-}
591−getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a
592−getIndexUnsafe i arr = let !r = unsafeInlineIO $ unsafeIndexIO i arr in r
593−
594−{-# DEPRECATED unsafeIndex "Please use 'getIndexUnsafe' instead" #-}
595357{-# INLINE_NORMAL unsafeIndex #-}
596358unsafeIndex :: forall a. Unbox a => Int -> Array a -> a
597−unsafeIndex = getIndexUnsafe
359+unsafeIndex i arr = let !r = unsafeInlineIO $ unsafeIndexIO i arr in r
598360
599361-- | /O(1)/ Get the byte length of the array.
600362--
@@ -609,68 +371,24 @@ byteLength = MA.byteLength . unsafeThaw
609371length :: Unbox a => Array a -> Int
610372length arr = MA.length (unsafeThaw arr)
611373
612−{-# INLINE_NORMAL producer #-}
613−producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a
614−producer =
615− Producer.translate unsafeThaw unsafeFreeze
616− $ MA.producerWith (return . unsafeInlineIO)
617−
618−-- | Unfold an array into a stream.
619−--
620−{-# INLINE_NORMAL reader #-}
621−reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
622−reader = Producer.simplify producer
623−
624−-- | Unfold an array into a stream, does not check the end of the array, the
625−-- user is responsible for terminating the stream within the array bounds. For
626−-- high performance application where the end condition can be determined by
627−-- a terminating fold.
628−--
629−-- Written in the hope that it may be faster than "read", however, in the case
630−-- for which this was written, "read" proves to be faster even though the core
631−-- generated with unsafeRead looks simpler.
632−--
633−-- /Pre-release/
634−--
635−{-# INLINE_NORMAL readerUnsafe #-}
636−readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
637−readerUnsafe = Unfold step inject
638− where
639−
640− inject (Array contents start end) =
641− return (MA.ArrayUnsafe contents end start)
642−
643− {-# INLINE_LATE step #-}
644− step (MA.ArrayUnsafe contents end p) = do
645− -- unsafeInlineIO allows us to run this in Identity monad for pure
646− -- toList/foldr case which makes them much faster due to not
647− -- accumulating the list and fusing better with the pure consumers.
648− --
649− -- This should be safe as the array contents are guaranteed to be
650− -- evaluated/written to before we peek at them.
651− let !x = unsafeInlineIO $ peekAt p contents
652− let !p1 = INDEX_NEXT(p,a)
653− return $ D.Yield x (MA.ArrayUnsafe contents end p1)
654−
655374-- | Unfold an array into a stream in reverse order.
656375--
657376{-# INLINE_NORMAL readerRev #-}
658377readerRev :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
659378readerRev = Unfold.lmap unsafeThaw $ MA.readerRevWith (return . unsafeInlineIO)
660379
661−{-# DEPRECATED toStreamD "Please use 'read' instead." #-}
662380{-# INLINE_NORMAL toStreamD #-}
663381toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> D.Stream m a
664−toStreamD = read
382+toStreamD arr = MA.toStreamDWith (return . unsafeInlineIO) (unsafeThaw arr)
665383
666384{-# INLINE toStreamK #-}
667385toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> K.StreamK m a
668386toStreamK arr = MA.toStreamKWith (return . unsafeInlineIO) (unsafeThaw arr)
669387
670−{-# DEPRECATED toStreamDRev "Please use 'readRev' instead." #-}
671388{-# INLINE_NORMAL toStreamDRev #-}
672389toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> D.Stream m a
673−toStreamDRev = readRev
390+toStreamDRev arr =
391+ MA.toStreamDRevWith (return . unsafeInlineIO) (unsafeThaw arr)
674392
675393{-# INLINE toStreamKRev #-}
676394toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> K.StreamK m a
@@ -682,7 +400,7 @@ toStreamKRev arr =
682400-- /Pre-release/
683401{-# INLINE_EARLY read #-}
684402read :: (Monad m, Unbox a) => Array a -> Stream m a
685−read arr = MA.toStreamWith (return . unsafeInlineIO) (unsafeThaw arr)
403+read = toStreamD
686404
687405-- | Same as 'read'
688406--
@@ -699,7 +417,7 @@ toStream = read
699417-- /Pre-release/
700418{-# INLINE_EARLY readRev #-}
701419readRev :: (Monad m, Unbox a) => Array a -> Stream m a
702−readRev arr = MA.toStreamRevWith (return . unsafeInlineIO) (unsafeThaw arr)
420+readRev = toStreamDRev
703421
704422-- | Same as 'readRev'
705423--
@@ -723,7 +441,6 @@ foldr f z arr = runIdentity $ D.foldr f z $ toStreamD arr
723441-- | Create two slices of an array without copying the original array. The
724442-- specified index @i@ is the first index of the second slice.
725443--
726−{-# INLINE splitAt #-}
727444splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)
728445splitAt i arr = (unsafeFreeze a, unsafeFreeze b)
729446 where
@@ -743,8 +460,8 @@ toListFB c n Array{..} = go arrStart
743460 -- accumulating the list and fusing better with the pure consumers.
744461 --
745462 -- This should be safe as the array contents are guaranteed to be
746− -- evaluated/written to before we peekAt at them.
747− let !x = unsafeInlineIO $ peekAt p arrContents
463+ -- evaluated/written to before we peekByteIndex at them.
464+ let !x = unsafeInlineIO $ peekByteIndex p arrContents
748465 in c x (go (INDEX_NEXT(p,a)))
749466
750467-- | Convert an 'Array' into a list.
@@ -757,28 +474,20 @@ toList s = build (\c n -> toListFB c n s)
757474-- Folds
758475-------------------------------------------------------------------------------
759476
760−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
477+{-# INLINE_NORMAL writeNAs #-}
478+writeNAs ::
479+ forall m a. (MonadIO m, Unbox a)
480+ => PinnedState
481+ -> Int
482+ -> Fold m a (Array a)
483+writeNAs ps = fmap unsafeFreeze . MA.writeNAs ps
484+
485+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
761486-- 'Array'.
762487--
763−{-# INLINE_NORMAL createOf #-}
764−createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
765−createOf = fmap unsafeFreeze . MA.createOf
766−
767−-- XXX Deprecate in major
768−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
769−{-# INLINE writeN #-}
488+{-# INLINE_NORMAL writeN #-}
770489writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
771−writeN = createOf
772−
773−-- | Like 'createOf' but creates a pinned array.
774−{-# INLINE_NORMAL pinnedCreateOf #-}
775−pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
776−pinnedCreateOf = fmap unsafeFreeze . MA.pinnedCreateOf
777−
778−{-# DEPRECATED pinnedWriteN "Please use pinnedCreateOf instead." #-}
779−{-# INLINE pinnedWriteN #-}
780−pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
781−pinnedWriteN = pinnedCreateOf
490+writeN = fmap unsafeFreeze . MA.writeN
782491
783492-- | @pinnedWriteNAligned alignment n@ folds a maximum of @n@ elements from the input
784493-- stream to an 'Array' aligned to the given size.
@@ -786,74 +495,39 @@ pinnedWriteN = pinnedCreateOf
786495-- /Pre-release/
787496--
788497{-# INLINE_NORMAL pinnedWriteNAligned #-}
789−{-# DEPRECATED pinnedWriteNAligned "To be removed." #-}
790498pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a)
791499 => Int -> Int -> Fold m a (Array a)
792500pinnedWriteNAligned alignSize = fmap unsafeFreeze . MA.pinnedWriteNAligned alignSize
793501
794−-- | Like 'createOf' but does not check the array bounds when writing. The fold
502+{-# INLINE_NORMAL writeNUnsafeAs #-}
503+writeNUnsafeAs :: forall m a. (MonadIO m, Unbox a)
504+ => PinnedState -> Int -> Fold m a (Array a)
505+writeNUnsafeAs ps n = unsafeFreeze <$> MA.writeNUnsafeAs ps n
506+
507+-- | Like 'writeN' but does not check the array bounds when writing. The fold
795508-- driver must not call the step function more than 'n' times otherwise it will
796509-- corrupt the memory and crash. This function exists mainly because any
797510-- conditional in the step function blocks fusion causing 10x performance
798511-- slowdown.
799512--
800−{-# INLINE_NORMAL unsafeCreateOf #-}
801−unsafeCreateOf :: forall m a. (MonadIO m, Unbox a)
802− => Int -> Fold m a (Array a)
803−unsafeCreateOf n = unsafeFreeze <$> MA.unsafeCreateOf n
804−
805−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
806−{-# INLINE writeNUnsafe #-}
513+{-# INLINE_NORMAL writeNUnsafe #-}
807514writeNUnsafe :: forall m a. (MonadIO m, Unbox a)
808515 => Int -> Fold m a (Array a)
809−writeNUnsafe = unsafeCreateOf
810−
811−{-# INLINE_NORMAL unsafePinnedCreateOf #-}
812−unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a)
813− => Int -> Fold m a (Array a)
814−unsafePinnedCreateOf n = unsafeFreeze <$> MA.unsafePinnedCreateOf n
815−
816−{-# DEPRECATED pinnedWriteNUnsafe "Please use unsafePinnedCreateOf instead." #-}
817−{-# INLINE pinnedWriteNUnsafe #-}
818−pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a)
819− => Int -> Fold m a (Array a)
820−pinnedWriteNUnsafe = unsafePinnedCreateOf
516+writeNUnsafe n = unsafeFreeze <$> MA.writeNUnsafe n
821517
822−{-# INLINE_NORMAL createWith #-}
823−createWith :: forall m a. (MonadIO m, Unbox a)
824− => Int -> Fold m a (Array a)
825−-- createWith n = FL.rmapM spliceArrays $ toArraysOf n
826−createWith elemCount = unsafeFreeze <$> MA.createWith elemCount
827−
828−{-# DEPRECATED writeWith "Please use createWith instead." #-}
829−{-# INLINE writeWith #-}
518+{-# INLINE_NORMAL writeWith #-}
830519writeWith :: forall m a. (MonadIO m, Unbox a)
831520 => Int -> Fold m a (Array a)
832−writeWith = createWith
521+-- writeWith n = FL.rmapM spliceArrays $ toArraysOf n
522+writeWith elemCount = unsafeFreeze <$> MA.writeWith elemCount
833523
834524-- | Fold the whole input to a single array.
835525--
836526-- /Caution! Do not use this on infinite streams./
837527--
838−{-# INLINE create #-}
839−create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
840−create = fmap unsafeFreeze MA.create
841−
842−-- XXX Deprecate in major
843−-- {-# DEPRECATED write "Please use create instead." #-}
844528{-# INLINE write #-}
845529write :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
846−write = create
847−
848−-- | Like 'create' but creates a pinned array.
849−{-# INLINE pinnedCreate #-}
850−pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
851−pinnedCreate = fmap unsafeFreeze MA.pinnedCreate
852−
853−{-# DEPRECATED pinnedWrite "Please use pinnedCreate instead." #-}
854−{-# INLINE pinnedWrite #-}
855−pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
856−pinnedWrite = pinnedCreate
530+write = fmap unsafeFreeze MA.write
857531
858532-- | Fold "step" has a dependency on "initial", and each step is dependent on
859533-- the previous invocation of step due to state passing, finally extract
@@ -875,20 +549,15 @@ pinnedWrite = pinnedCreate
875549-- could be unsafe and dangerous. This is dangerous especially when used with
876550-- foldMany like operations.
877551--
552+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
878553-- >>> unsafePureWrite = Array.unsafeMakePure Array.write
879554--
880555{-# INLINE unsafeMakePure #-}
881556unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b
882−unsafeMakePure (Fold step initial extract final) =
557+unsafeMakePure (Fold step initial extract) =
883558 Fold (\x a -> return $! unsafeInlineIO (step x a))
884559 (return $! unsafePerformIO initial)
885560 (\s -> return $! unsafeInlineIO $ extract s)
886− (\s -> return $! unsafeInlineIO $ final s)
887−
888−{-# INLINE fromPureStreamN #-}
889−fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a
890−fromPureStreamN n x =
891− unsafePerformIO $ fmap unsafeFreeze (MA.fromPureStreamN n x)
892561
893562-- | Convert a pure stream in Identity monad to an immutable array.
894563--
@@ -901,19 +570,6 @@ fromPureStream x = unsafePerformIO $ fmap (unsafeFreeze) (MA.fromPureStream x)
901570-- fromPureStream = runIdentity . D.fold (unsafeMakePure write)
902571-- fromPureStream = fromList . runIdentity . D.toList
903572
904−-- XXX This should be monadic.
905−
906−-- | Copy an immutable 'Ptr Word8' sequence into an array.
907−--
908−-- /Unsafe:/ The caller is responsible for safe addressing.
909−--
910−-- Note that this should be evaluated strictly to ensure that we do not hold
911−-- the reference to the pointer in a lazy thunk.
912−fromPtrN :: Int -> Ptr Word8 -> Array Word8
913−fromPtrN n addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromPtrN n addr)
914−
915−-- XXX This should be monadic.
916−
917573-- | Copy a null terminated immutable 'Addr#' Word8 sequence into an array.
918574--
919575-- /Unsafe:/ The caller is responsible for safe addressing.
@@ -921,49 +577,13 @@ fromPtrN n addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromPtrN n addr)
921577-- Note that this is completely safe when reading from Haskell string
922578-- literals because they are guaranteed to be NULL terminated:
923579--
580+-- >>> :set -XMagicHash
581+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
924582-- >>> Array.toList $ Array.fromByteStr# "\1\2\3\0"#
925583-- [1,2,3]
926584--
927−-- Note that this should be evaluated strictly to ensure that we do not hold
928−-- the reference to the pointer in a lazy thunk.
929585fromByteStr# :: Addr# -> Array Word8
930−fromByteStr# addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromByteStr# addr)
931−
932−-- XXX This should be monadic.
933−
934−-- | Note that this should be evaluated strictly to ensure that we do not hold
935−-- the reference to the pointer in a lazy thunk.
936−fromByteStr :: Ptr Word8 -> Array Word8
937−fromByteStr (Ptr addr#) = fromByteStr# addr#
938−
939−-- XXX implement fromChunks/fromChunkList instead?
940−
941−-- | Convert an array stream to an array. Note that this requires peak memory
942−-- that is double the size of the array stream.
943−--
944−{-# INLINE fromChunksK #-}
945−fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)
946−fromChunksK stream =
947− -- We buffer the entire stream and then allocate the target array of the
948− -- same size, thus requiring double the memory.
949− fmap unsafeFreeze $ MA.fromChunksK $ fmap unsafeThaw stream
950−
951−{-# DEPRECATED fromArrayStreamK "Please use fromChunksK instead." #-}
952−fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (Array a) -> m (Array a)
953−fromArrayStreamK = fromChunksK
954−
955−-- | Given a stream of arrays, splice them all together to generate a single
956−-- array. The stream must be /finite/.
957−--
958−{-# INLINE fromChunks #-}
959−fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
960−fromChunks s =
961− -- XXX Check which implementation is better
962− -- This may also require double the memory as we double the space every
963− -- time, when copying the last array we may have reallocated almost double
964− -- the space required before we right size it.
965− fmap unsafeFreeze $ MA.fromChunksRealloced (fmap unsafeThaw s)
966− -- fromChunkStreamK $ D.toStreamK s
586+fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
967587
968588-------------------------------------------------------------------------------
969589-- Instances
@@ -995,66 +615,12 @@ instance Unbox a => IsList (Array a) where
995615 {-# INLINE toList #-}
996616 toList = toList
997617
998−-- | Byte compare two arrays. Compare the length of the arrays. If the length
999−-- is equal, compare the lexicographical ordering of two underlying byte arrays
1000−-- otherwise return the result of length comparison.
1001−--
1002−-- /Unsafe/: Note that the 'Unbox' instance of sum types with constructors of
1003−-- different sizes may leave some memory uninitialized which can make byte
1004−-- comparison unreliable.
1005−--
1006−-- /Pre-release/
1007−{-# INLINE byteCmp #-}
1008−byteCmp :: Array a -> Array a -> Ordering
1009−byteCmp arr1 arr2 =
1010− -- unsafePerformIO?
1011− unsafeInlineIO $! unsafeThaw arr1 `MA.byteCmp` unsafeThaw arr2
1012−
1013−-- | Byte equality of two arrays.
1014−--
1015−-- >>> byteEq arr1 arr2 = (==) EQ $ Array.byteCmp arr1 arr2
1016−--
1017−-- /Unsafe/: See 'byteCmp'.
1018−{-# INLINE byteEq #-}
1019−byteEq :: Array a -> Array a -> Bool
1020−byteEq arr1 arr2 = (==) EQ $ byteCmp arr1 arr2
1021−
1022−#define MK_EQ_INSTANCE(typ) \
1023−instance {-# OVERLAPPING #-} Eq (Array typ) where { \
1024−; {-# INLINE (==) #-} \
1025−; (==) = byteEq \
1026−}
1027−
1028−MK_EQ_INSTANCE(Char)
1029−MK_EQ_INSTANCE(Word8)
1030−MK_EQ_INSTANCE(Word16)
1031−MK_EQ_INSTANCE(Word32)
1032−
1033−-- XXX The Word64 default instance should be as fast because we are comparing
1034−-- 64-bit at a time.
1035−MK_EQ_INSTANCE(Word64)
1036−MK_EQ_INSTANCE(Int)
1037−MK_EQ_INSTANCE(Int8)
1038−MK_EQ_INSTANCE(Int16)
1039−MK_EQ_INSTANCE(Int32)
1040−
1041−-- XXX The Int64 default instance should be as fast.
1042−MK_EQ_INSTANCE(Int64)
1043−
1044−-- | If the type allows a byte-by-byte comparison this instance can be
1045−-- overlapped by a more specific instance that uses 'byteCmp'. Byte comparison
1046−-- can be significantly faster.
1047−--
1048−instance {-# OVERLAPPABLE #-} (Unbox a, Eq a) => Eq (Array a) where
618+-- XXX we are assuming that Unboxed equality means element equality. This may
619+-- or may not be correct? arrcmp is 40% faster compared to stream equality.
620+instance (Unbox a, Eq a) => Eq (Array a) where
1049621 {-# INLINE (==) #-}
1050622 arr1 == arr2 =
1051− -- Does unboxed byte equality mean element equality?
1052− -- XXX This is incorrect for sum types, as we may have some
1053− -- uninitialized memory in that case. If we always initialize the
1054− -- unused memory to zero we can use this.
1055− -- Byte comparison is 40% faster compared to stream equality.
1056− -- (==) EQ $ unsafeInlineIO $! unsafeThaw arr1 `MA.cmp` unsafeThaw arr2
1057− (toStreamD arr1 :: Stream Identity a) == toStreamD arr2
623+ (==) EQ $ unsafeInlineIO $! unsafeThaw arr1 `MA.cmp` unsafeThaw arr2
1058624
1059625instance (Unbox a, Ord a) => Ord (Array a) where
1060626 {-# INLINE compare #-}
@@ -1096,7 +662,7 @@ _toStreamD_ size Array{..} = D.Stream step arrStart
1096662 {-# INLINE_LATE step #-}
1097663 step _ p | p == arrEnd = return D.Stop
1098664 step _ p = liftIO $ do
1099− x <- peekAt p arrContents
665+ x <- peekByteIndex p arrContents
1100666 return $ D.Yield x (p + size)
1101667
1102668{-
@@ -1119,29 +685,15 @@ instance Foldable Array where
1119685-- Semigroup and Monoid
1120686-------------------------------------------------------------------------------
1121687
1122−-- XXX Deprecate and remove the Semigroup and Monoid instances because of
1123−-- potential misuse chances.
1124−
1125−-- | This should not be used for combining many or N arrays as it would copy
1126−-- the two arrays everytime to a new array. For coalescing multiple arrays use
1127−-- 'fromChunksK' instead.
1128688instance Unbox a => Semigroup (Array a) where
1129689 arr1 <> arr2 = unsafePerformIO $ splice arr1 arr2
1130690
1131−empty ::
1132−#ifdef DEVBUILD
1133− Unbox a =>
1134−#endif
1135− Array a
1136−empty = Array Unboxed.empty 0 0
1137−
1138−{-# DEPRECATED nil "Please use empty instead." #-}
1139691nil ::
1140692#ifdef DEVBUILD
1141693 Unbox a =>
1142694#endif
1143695 Array a
1144−nil = empty
696+nil = Array Unboxed.nil 0 0
1145697
1146698instance Unbox a => Monoid (Array a) where
1147699 mempty = nil
core/src/Streamly/Internal/Data/Builder.hs+9−11
@@ -16,9 +16,7 @@ module Streamly.Internal.Data.Builder
1616 )
1717where
1818
19−#if !MIN_VERSION_base(4,18,0)
2019import Control.Applicative (liftA2)
21−#endif
2220
2321------------------------------------------------------------------------------
2422-- The Builder type
@@ -29,16 +27,16 @@ import Control.Applicative (liftA2)
2927-- or even a Fold. Unlike fold the step function is one-shot and not called in
3028-- a loop.
3129newtype Builder s m a =
32− Builder (s -> m (a, s))
30+ Builder (s -> m (s, a))
3331
3432-- | Maps a function on the output of the fold (the type @b@).
3533instance Functor m => Functor (Builder s m) where
3634 {-# INLINE fmap #-}
37− fmap f (Builder step1) = Builder (fmap (\ (a, s) -> (f a, s)) . step1)
35+ fmap f (Builder step1) = Builder (fmap (fmap f) . step1)
3836
3937{-# INLINE fromPure #-}
4038fromPure :: Applicative m => b -> Builder s m b
41−fromPure b = Builder (\s -> pure (b, s))
39+fromPure b = Builder (\s -> pure (s, b))
4240
4341-- | Chain the actions and zip the outputs.
4442{-# INLINE sequenceWith #-}
@@ -49,9 +47,9 @@ sequenceWith func (Builder stepL) (Builder stepR) = Builder step
4947 where
5048
5149 step s = do
52− (x, s1) <- stepL s
53− (y, s2) <- stepR s1
54− pure (func x y, s2)
50+ (s1, x) <- stepL s
51+ (s2, y) <- stepR s1
52+ pure (s2, func x y)
5553
5654instance Monad m => Applicative (Builder a m) where
5755 {-# INLINE pure #-}
@@ -76,7 +74,7 @@ instance Monad m => Monad (Builder a m) where
7674 where
7775
7876 step s = do
79− (x, s1) <- stepL s
77+ (s1, x) <- stepL s
8078 let Builder stepR = f x
81− (y, s2) <- stepR s1
82− pure (y, s2)
79+ (s2, y) <- stepR s1
80+ pure (s2, y)
core/src/Streamly/Internal/Data/Fold.hs+2574−9
@@ -16,18 +16,2583 @@ module Streamly.Internal.Data.Fold
1616 -- * Imports
1717 -- $setup
1818
19− module Streamly.Internal.Data.Fold.Type
20− , module Streamly.Internal.Data.Fold.Tee
21− , module Streamly.Internal.Data.Fold.Combinators
22− , module Streamly.Internal.Data.Fold.Container
23− , module Streamly.Internal.Data.Fold.Window
19+ -- * Fold Type
20+ Step (..)
21+ , Fold (..)
22+ , Tee (..)
23+
24+ -- * Constructors
25+ -- | Which constructor to use?
26+ --
27+ -- * @foldl*@: If the fold never terminates i.e. does not use the 'Done'
28+ -- constructor otherwise use the @foldt*@ variants.
29+ -- * @*M@: Use the @M@ suffix variants if any of the step, initial, or
30+ -- extract function is monadic, otherwise use the pure variants.
31+ --
32+ , foldl'
33+ , foldlM'
34+ , foldl1'
35+ , foldlM1'
36+ , foldt'
37+ , foldtM'
38+ , foldr'
39+ , foldrM'
40+
41+ -- * Mappers
42+ -- | Monadic functions useful with mapM/lmapM on folds or streams.
43+ , tracing
44+ , trace
45+
46+ -- * Folds
47+
48+ -- ** Accumulators
49+ -- *** Semigroups and Monoids
50+ , sconcat
51+ , mconcat
52+ , foldMap
53+ , foldMapM
54+
55+ -- *** Reducers
56+ , drain
57+ , drainMapM
58+ , the
59+ , length
60+ , lengthGeneric
61+ , mean
62+ , rollingHash
63+ , defaultSalt
64+ , rollingHashWithSalt
65+ , rollingHashFirstN
66+ -- , rollingHashLastN
67+
68+ -- *** Saturating Reducers
69+ -- | 'product' terminates if it becomes 0. Other folds can theoretically
70+ -- saturate on bounded types, and therefore terminate, however, they will
71+ -- run forever on unbounded types like Integer/Double.
72+ , sum
73+ , product
74+ , maximumBy
75+ , maximum
76+ , minimumBy
77+ , minimum
78+
79+ -- *** Collectors
80+ -- | Avoid using these folds in scalable or performance critical
81+ -- applications, they buffer all the input in GC memory which can be
82+ -- detrimental to performance if the input is large.
83+ , toList
84+ , toListRev
85+ -- $toListRev
86+ , toStream
87+ , toStreamRev
88+ , toStreamK
89+ , toStreamKRev
90+ , topBy
91+ , top
92+ , bottomBy
93+ , bottom
94+
95+ -- *** Scanners
96+ -- | Stateful transformation of the elements. Useful in combination with
97+ -- the 'scanMaybe' combinator. For scanners the result of the fold is
98+ -- usually a transformation of the current element rather than an
99+ -- aggregation of all elements till now.
100+ , latest
101+ -- , nthLast -- using Ring array
102+ , indexingWith
103+ , indexing
104+ , indexingRev
105+ , rollingMapM
106+
107+ -- *** Filters
108+ -- | Useful in combination with the 'scanMaybe' combinator.
109+ , filtering
110+ , deleteBy
111+ , uniqBy
112+ , uniq
113+ , repeated
114+ , findIndices
115+ , elemIndices
116+
117+ -- ** Terminating Folds
118+ -- *** Empty folds
119+ -- | Folds that return a result without consuming any input.
120+ , fromPure
121+ , fromEffect
122+ , fromRefold
123+
124+ -- *** Singleton folds
125+ -- | Folds that terminate after consuming exactly one input element. All
126+ -- these can be implemented in terms of the 'maybe' fold.
127+ , one
128+ , null -- XXX not very useful and could be problematic, remove it?
129+ , satisfy
130+ , maybe
131+
132+ -- *** Multi folds
133+ -- | Terminate after consuming one or more elements.
134+ , drainN
135+ -- , lastN
136+ -- , (!!)
137+ , indexGeneric
138+ , index
139+ , findM
140+ , find
141+ , lookup
142+ , findIndex
143+ , elemIndex
144+ , elem
145+ , notElem
146+ , all
147+ , any
148+ , and
149+ , or
150+
151+ -- ** Trimmers
152+ -- | Useful in combination with the 'scanMaybe' combinator.
153+ , taking
154+ , dropping
155+ , takingEndByM
156+ , takingEndBy
157+ , takingEndByM_
158+ , takingEndBy_
159+ , droppingWhileM
160+ , droppingWhile
161+ , prune
162+
163+ -- * Running A Fold
164+ , drive
165+ -- , breakStream
166+
167+ -- * Building Incrementally
168+ , extractM
169+ , reduce
170+ , close
171+ , isClosed
172+ , snoc
173+ , snocl
174+ , snocM
175+ , snoclM
176+
177+ , addOne
178+ , addStream
179+
180+ -- * Combinators
181+ -- ** Utilities
182+ , with
183+
184+ -- ** Transforming the Monad
185+ , morphInner
186+ , generalizeInner
187+
188+ -- ** Mapping on output
189+ , rmapM
190+
191+ -- ** Mapping on Input
192+ , transform
193+ , lmap
194+ --, lsequence
195+ , lmapM
196+
197+ -- ** Sliding Window
198+ , slide2
199+
200+ -- ** Scanning Input
201+ , scan
202+ , scanMany
203+ , postscan
204+ , indexed
205+
206+ -- ** Zipping Input
207+ , zipStreamWithM
208+ , zipStream
209+
210+ -- ** Filtering Input
211+ , catMaybes
212+ , mapMaybeM
213+ , mapMaybe
214+ , scanMaybe
215+ , filter
216+ , filterM
217+ , sampleFromthen
218+
219+ -- Either streams
220+ , catLefts
221+ , catRights
222+ , catEithers
223+
224+ {-
225+ -- ** Insertion
226+ -- | Insertion adds more elements to the stream.
227+
228+ , insertBy
229+ , intersperseM
230+
231+ -- ** Reordering
232+ , reverse
233+ -}
234+
235+ -- ** Trimming
236+ , take
237+
238+ -- By elements
239+ , takeEndBy
240+ , takeEndBy_
241+ , takeEndBySeq
242+ , takeEndBySeq_
243+ {-
244+ , drop
245+ , dropWhile
246+ , dropWhileM
247+ -}
248+
249+ -- ** Serial Append
250+ , splitWith
251+ , split_
252+ -- , tail
253+ -- , init
254+ , splitAt -- spanN
255+ -- , splitIn -- sessionN
256+
257+ -- ** Parallel Distribution
258+ , teeWith
259+ , tee
260+ , teeWithFst
261+ , teeWithMin
262+ , distribute
263+ -- , distributeFst
264+ -- , distributeMin
265+
266+ -- ** Unzipping
267+ , unzip
268+ -- These two can be expressed using lmap/lmapM and unzip
269+ , unzipWith
270+ , unzipWithM
271+ , unzipWithFstM
272+ , unzipWithMinM
273+
274+ -- ** Parallel Alternative
275+ , shortest
276+ , longest
277+
278+ -- ** Partitioning
279+ , partitionByM
280+ , partitionByFstM
281+ , partitionByMinM
282+ , partitionBy
283+ , partition
284+
285+ -- ** Splitting
286+ , many
287+ , manyPost
288+ , groupsOf
289+ , chunksBetween
290+ , refoldMany
291+ , refoldMany1
292+ , intersperseWithQuotes
293+
294+ -- ** Nesting
295+ , unfoldMany
296+ , concatSequence
297+ , concatMap
298+ , duplicate
299+ , refold
300+
301+ -- * Deprecated
302+ , foldr
303+ , drainBy
304+ , last
305+ , head
306+ , sequence
307+ , mapM
308+ , variance
309+ , stdDev
310+ , serialWith
24311 )
25312where
26313
27−import Streamly.Internal.Data.Fold.Combinators
28−import Streamly.Internal.Data.Fold.Container
29−import Streamly.Internal.Data.Fold.Tee
314+#include "inline.hs"
315+#include "ArrayMacros.h"
316+
317+import Control.Monad (void)
318+import Control.Monad.IO.Class (MonadIO(..))
319+import Data.Bifunctor (first)
320+import Data.Bits (shiftL, shiftR, (.|.), (.&.))
321+import Data.Either (isLeft, isRight, fromLeft, fromRight)
322+import Data.Int (Int64)
323+import Data.Proxy (Proxy(..))
324+import Data.Word (Word32)
325+import Foreign.Storable (Storable, peek)
326+import Streamly.Internal.Data.Array.Mut.Type (MutArray(..))
327+import Streamly.Internal.Data.Maybe.Strict (Maybe'(..), toMaybe)
328+import Streamly.Internal.Data.Pipe.Type (Pipe (..), PipeState(..))
329+import Streamly.Internal.Data.Unbox (Unbox, sizeOf)
330+import Streamly.Internal.Data.Unfold.Type (Unfold(..))
331+import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3'(..))
332+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
333+
334+import qualified Prelude
335+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
336+import qualified Streamly.Internal.Data.Array.Type as Array
337+import qualified Streamly.Internal.Data.Fold.Window as FoldW
338+import qualified Streamly.Internal.Data.Pipe.Type as Pipe
339+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
340+import qualified Streamly.Internal.Data.Stream.StreamD.Type as StreamD
341+
342+import Prelude hiding
343+ ( Foldable(..), filter, drop, dropWhile, take, takeWhile, zipWith
344+ , map, mapM_, sequence, all, any
345+ , notElem, head, last, tail
346+ , reverse, iterate, init, and, or, lookup, (!!)
347+ , scanl, scanl1, replicate, concatMap, mconcat, unzip
348+ , span, splitAt, break, mapM, zip, maybe)
30349import Streamly.Internal.Data.Fold.Type
31−import Streamly.Internal.Data.Fold.Window
350+import Streamly.Internal.Data.Fold.Tee
32351
33352#include "DocTestDataFold.hs"
353+
354+------------------------------------------------------------------------------
355+-- Running
356+------------------------------------------------------------------------------
357+
358+-- | Drive a fold using the supplied 'Stream', reducing the resulting
359+-- expression strictly at each step.
360+--
361+-- Definition:
362+--
363+-- >>> drive = flip Stream.fold
364+--
365+-- Example:
366+--
367+-- >>> Fold.drive (Stream.enumerateFromTo 1 100) Fold.sum
368+-- 5050
369+--
370+{-# INLINE drive #-}
371+drive :: Monad m => Stream m a -> Fold m a b -> m b
372+drive = flip StreamD.fold
373+
374+{-
375+-- | Like 'drive' but also returns the remaining stream. The resulting stream
376+-- would be 'Stream.nil' if the stream finished before the fold.
377+--
378+-- Definition:
379+--
380+-- >>> breakStream = flip Stream.foldBreak
381+--
382+-- /CPS/
383+--
384+{-# INLINE breakStreamK #-}
385+breakStreamK :: Monad m => StreamK m a -> Fold m a b -> m (b, StreamK m a)
386+breakStreamK strm fl = fmap f $ K.foldBreak fl (Stream.toStreamK strm)
387+
388+ where
389+
390+ f (b, str) = (b, Stream.fromStreamK str)
391+-}
392+
393+-- | Append a stream to a fold to build the fold accumulator incrementally. We
394+-- can repeatedly call 'addStream' on the same fold to continue building the
395+-- fold and finally use 'drive' to finish the fold and extract the result. Also
396+-- see the 'Streamly.Data.Fold.addOne' operation which is a singleton version
397+-- of 'addStream'.
398+--
399+-- Definitions:
400+--
401+-- >>> addStream stream = Fold.drive stream . Fold.duplicate
402+--
403+-- Example, build a list incrementally:
404+--
405+-- >>> :{
406+-- pure (Fold.toList :: Fold IO Int [Int])
407+-- >>= Fold.addOne 1
408+-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
409+-- >>= Fold.drive Stream.nil
410+-- >>= print
411+-- :}
412+-- [1,2,3,4]
413+--
414+-- This can be used as an O(n) list append compared to the O(n^2) @++@ when
415+-- used for incrementally building a list.
416+--
417+-- Example, build a stream incrementally:
418+--
419+-- >>> :{
420+-- pure (Fold.toStream :: Fold IO Int (Stream Identity Int))
421+-- >>= Fold.addOne 1
422+-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
423+-- >>= Fold.drive Stream.nil
424+-- >>= print
425+-- :}
426+-- fromList [1,2,3,4]
427+--
428+-- This can be used as an O(n) stream append compared to the O(n^2) @<>@ when
429+-- used for incrementally building a stream.
430+--
431+-- Example, build an array incrementally:
432+--
433+-- >>> :{
434+-- pure (Array.write :: Fold IO Int (Array Int))
435+-- >>= Fold.addOne 1
436+-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
437+-- >>= Fold.drive Stream.nil
438+-- >>= print
439+-- :}
440+-- fromList [1,2,3,4]
441+--
442+-- Example, build an array stream incrementally:
443+--
444+-- >>> :{
445+-- let f :: Fold IO Int (Stream Identity (Array Int))
446+-- f = Fold.groupsOf 2 (Array.writeN 3) Fold.toStream
447+-- in pure f
448+-- >>= Fold.addOne 1
449+-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
450+-- >>= Fold.drive Stream.nil
451+-- >>= print
452+-- :}
453+-- fromList [fromList [1,2],fromList [3,4]]
454+--
455+addStream :: Monad m => Stream m a -> Fold m a b -> m (Fold m a b)
456+addStream stream = drive stream . duplicate
457+
458+------------------------------------------------------------------------------
459+-- Transformations on fold inputs
460+------------------------------------------------------------------------------
461+
462+-- | Flatten the monadic output of a fold to pure output.
463+--
464+{-# DEPRECATED sequence "Use \"rmapM id\" instead" #-}
465+{-# INLINE sequence #-}
466+sequence :: Monad m => Fold m a (m b) -> Fold m a b
467+sequence = rmapM id
468+
469+-- | Map a monadic function on the output of a fold.
470+--
471+{-# DEPRECATED mapM "Use rmapM instead" #-}
472+{-# INLINE mapM #-}
473+mapM :: Monad m => (b -> m c) -> Fold m a b -> Fold m a c
474+mapM = rmapM
475+
476+-- |
477+-- >>> mapMaybeM f = Fold.lmapM f . Fold.catMaybes
478+--
479+{-# INLINE mapMaybeM #-}
480+mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Fold m b r -> Fold m a r
481+mapMaybeM f = lmapM f . catMaybes
482+
483+-- | @mapMaybe f fold@ maps a 'Maybe' returning function @f@ on the input of
484+-- the fold, filters out 'Nothing' elements, and return the values extracted
485+-- from 'Just'.
486+--
487+-- >>> mapMaybe f = Fold.lmap f . Fold.catMaybes
488+-- >>> mapMaybe f = Fold.mapMaybeM (return . f)
489+--
490+-- >>> f x = if even x then Just x else Nothing
491+-- >>> fld = Fold.mapMaybe f Fold.toList
492+-- >>> Stream.fold fld (Stream.enumerateFromTo 1 10)
493+-- [2,4,6,8,10]
494+--
495+{-# INLINE mapMaybe #-}
496+mapMaybe :: Monad m => (a -> Maybe b) -> Fold m b r -> Fold m a r
497+mapMaybe f = lmap f . catMaybes
498+
499+------------------------------------------------------------------------------
500+-- Transformations on fold inputs
501+------------------------------------------------------------------------------
502+
503+-- | Apply a monadic function on the input and return the input.
504+--
505+-- >>> Stream.fold (Fold.lmapM (Fold.tracing print) Fold.drain) $ (Stream.enumerateFromTo (1 :: Int) 2)
506+-- 1
507+-- 2
508+--
509+-- /Pre-release/
510+--
511+{-# INLINE tracing #-}
512+tracing :: Monad m => (a -> m b) -> (a -> m a)
513+tracing f x = void (f x) >> return x
514+
515+-- | Apply a monadic function to each element flowing through and discard the
516+-- results.
517+--
518+-- >>> Stream.fold (Fold.trace print Fold.drain) $ (Stream.enumerateFromTo (1 :: Int) 2)
519+-- 1
520+-- 2
521+--
522+-- >>> trace f = Fold.lmapM (Fold.tracing f)
523+--
524+-- /Pre-release/
525+{-# INLINE trace #-}
526+trace :: Monad m => (a -> m b) -> Fold m a r -> Fold m a r
527+trace f = lmapM (tracing f)
528+
529+-- rename to lpipe?
530+--
531+-- | Apply a transformation on a 'Fold' using a 'Pipe'.
532+--
533+-- /Pre-release/
534+{-# INLINE transform #-}
535+transform :: Monad m => Pipe m a b -> Fold m b c -> Fold m a c
536+transform (Pipe pstep1 pstep2 pinitial) (Fold fstep finitial fextract) =
537+ Fold step initial extract
538+
539+ where
540+
541+ initial = first (Tuple' pinitial) <$> finitial
542+
543+ step (Tuple' ps fs) x = do
544+ r <- pstep1 ps x
545+ go fs r
546+
547+ where
548+
549+ -- XXX use SPEC?
550+ go acc (Pipe.Yield b (Consume ps')) = do
551+ acc' <- fstep acc b
552+ return
553+ $ case acc' of
554+ Partial s -> Partial $ Tuple' ps' s
555+ Done b2 -> Done b2
556+ go acc (Pipe.Yield b (Produce ps')) = do
557+ acc' <- fstep acc b
558+ r <- pstep2 ps'
559+ case acc' of
560+ Partial s -> go s r
561+ Done b2 -> return $ Done b2
562+ go acc (Pipe.Continue (Consume ps')) =
563+ return $ Partial $ Tuple' ps' acc
564+ go acc (Pipe.Continue (Produce ps')) = do
565+ r <- pstep2 ps'
566+ go acc r
567+
568+ extract (Tuple' _ fs) = fextract fs
569+
570+{-# INLINE scanWith #-}
571+scanWith :: Monad m => Bool -> Fold m a b -> Fold m b c -> Fold m a c
572+scanWith isMany (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
573+ Fold step initial extract
574+
575+ where
576+
577+ {-# INLINE runStep #-}
578+ runStep actionL sR = do
579+ rL <- actionL
580+ case rL of
581+ Done bL -> do
582+ rR <- stepR sR bL
583+ case rR of
584+ Partial sR1 ->
585+ if isMany
586+ then runStep initialL sR1
587+ else Done <$> extractR sR1
588+ Done bR -> return $ Done bR
589+ Partial sL -> do
590+ !b <- extractL sL
591+ rR <- stepR sR b
592+ return
593+ $ case rR of
594+ Partial sR1 -> Partial (sL, sR1)
595+ Done bR -> Done bR
596+
597+ initial = do
598+ r <- initialR
599+ case r of
600+ Partial sR -> runStep initialL sR
601+ Done b -> return $ Done b
602+
603+ step (sL, sR) x = runStep (stepL sL x) sR
604+
605+ extract = extractR . snd
606+
607+-- | Scan the input of a 'Fold' to change it in a stateful manner using another
608+-- 'Fold'. The scan stops as soon as the fold terminates.
609+--
610+-- /Pre-release/
611+{-# INLINE scan #-}
612+scan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
613+scan = scanWith False
614+
615+-- XXX This does not fuse beacuse of the recursive step. Need to investigate.
616+--
617+-- | Scan the input of a 'Fold' to change it in a stateful manner using another
618+-- 'Fold'. The scan restarts with a fresh state if the fold terminates.
619+--
620+-- /Pre-release/
621+{-# INLINE scanMany #-}
622+scanMany :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
623+scanMany = scanWith True
624+
625+------------------------------------------------------------------------------
626+-- Filters
627+------------------------------------------------------------------------------
628+
629+-- | Returns the latest element omitting the first occurrence that satisfies
630+-- the given equality predicate.
631+--
632+-- Example:
633+--
634+-- >>> input = Stream.fromList [1,3,3,5]
635+-- >>> Stream.fold Fold.toList $ Stream.scanMaybe (Fold.deleteBy (==) 3) input
636+-- [1,3,5]
637+--
638+{-# INLINE_NORMAL deleteBy #-}
639+deleteBy :: Monad m => (a -> a -> Bool) -> a -> Fold m a (Maybe a)
640+deleteBy eq x0 = fmap extract $ foldl' step (Tuple' False Nothing)
641+
642+ where
643+
644+ step (Tuple' False _) x =
645+ if eq x x0
646+ then Tuple' True Nothing
647+ else Tuple' False (Just x)
648+ step (Tuple' True _) x = Tuple' True (Just x)
649+
650+ extract (Tuple' _ x) = x
651+
652+-- | Provide a sliding window of length 2 elements.
653+--
654+-- See "Streamly.Internal.Data.Fold.Window".
655+--
656+{-# INLINE slide2 #-}
657+slide2 :: Monad m => Fold m (a, Maybe a) b -> Fold m a b
658+slide2 (Fold step1 initial1 extract1) = Fold step initial extract
659+
660+ where
661+
662+ initial =
663+ first (Tuple' Nothing) <$> initial1
664+
665+ step (Tuple' prev s) cur =
666+ first (Tuple' (Just cur)) <$> step1 s (cur, prev)
667+
668+ extract (Tuple' _ s) = extract1 s
669+
670+-- | Return the latest unique element using the supplied comparison function.
671+-- Returns 'Nothing' if the current element is same as the last element
672+-- otherwise returns 'Just'.
673+--
674+-- Example, strip duplicate path separators:
675+--
676+-- >>> input = Stream.fromList "//a//b"
677+-- >>> f x y = x == '/' && y == '/'
678+-- >>> Stream.fold Fold.toList $ Stream.scanMaybe (Fold.uniqBy f) input
679+-- "/a/b"
680+--
681+-- Space: @O(1)@
682+--
683+-- /Pre-release/
684+--
685+{-# INLINE uniqBy #-}
686+uniqBy :: Monad m => (a -> a -> Bool) -> Fold m a (Maybe a)
687+uniqBy eq = rollingMap f
688+
689+ where
690+
691+ f pre curr =
692+ case pre of
693+ Nothing -> Just curr
694+ Just x -> if x `eq` curr then Nothing else Just curr
695+
696+-- | See 'uniqBy'.
697+--
698+-- Definition:
699+--
700+-- >>> uniq = Fold.uniqBy (==)
701+--
702+{-# INLINE uniq #-}
703+uniq :: (Monad m, Eq a) => Fold m a (Maybe a)
704+uniq = uniqBy (==)
705+
706+-- | Strip all leading and trailing occurrences of an element passing a
707+-- predicate and make all other consecutive occurrences uniq.
708+--
709+-- >> prune p = Stream.dropWhileAround p $ Stream.uniqBy (x y -> p x && p y)
710+--
711+-- @
712+-- > Stream.prune isSpace (Stream.fromList " hello world! ")
713+-- "hello world!"
714+--
715+-- @
716+--
717+-- Space: @O(1)@
718+--
719+-- /Unimplemented/
720+{-# INLINE prune #-}
721+prune ::
722+ -- (Monad m, Eq a) =>
723+ (a -> Bool) -> Fold m a (Maybe a)
724+prune = error "Not implemented yet!"
725+
726+-- | Emit only repeated elements, once.
727+--
728+-- /Unimplemented/
729+repeated :: -- (Monad m, Eq a) =>
730+ Fold m a (Maybe a)
731+repeated = error "Not implemented yet!"
732+
733+------------------------------------------------------------------------------
734+-- Left folds
735+------------------------------------------------------------------------------
736+
737+------------------------------------------------------------------------------
738+-- Run Effects
739+------------------------------------------------------------------------------
740+
741+-- |
742+-- Definitions:
743+--
744+-- >>> drainMapM f = Fold.lmapM f Fold.drain
745+-- >>> drainMapM f = Fold.foldMapM (void . f)
746+--
747+-- Drain all input after passing it through a monadic function. This is the
748+-- dual of mapM_ on stream producers.
749+--
750+{-# INLINE drainMapM #-}
751+drainMapM :: Monad m => (a -> m b) -> Fold m a ()
752+drainMapM f = lmapM f drain
753+
754+{-# DEPRECATED drainBy "Please use 'drainMapM' instead." #-}
755+{-# INLINE drainBy #-}
756+drainBy :: Monad m => (a -> m b) -> Fold m a ()
757+drainBy = drainMapM
758+
759+-- | Returns the latest element of the input stream, if any.
760+--
761+-- >>> latest = Fold.foldl1' (\_ x -> x)
762+-- >>> latest = fmap getLast $ Fold.foldMap (Last . Just)
763+--
764+{-# INLINE latest #-}
765+latest :: Monad m => Fold m a (Maybe a)
766+latest = foldl1' (\_ x -> x)
767+
768+{-# DEPRECATED last "Please use 'latest' instead." #-}
769+{-# INLINE last #-}
770+last :: Monad m => Fold m a (Maybe a)
771+last = latest
772+
773+-- | Terminates with 'Nothing' as soon as it finds an element different than
774+-- the previous one, returns 'the' element if the entire input consists of the
775+-- same element.
776+--
777+{-# INLINE the #-}
778+the :: (Monad m, Eq a) => Fold m a (Maybe a)
779+the = foldt' step initial id
780+
781+ where
782+
783+ initial = Partial Nothing
784+
785+ step Nothing x = Partial (Just x)
786+ step old@(Just x0) x =
787+ if x0 == x
788+ then Partial old
789+ else Done Nothing
790+
791+------------------------------------------------------------------------------
792+-- To Summary
793+------------------------------------------------------------------------------
794+
795+-- | Like 'length', except with a more general 'Num' return value
796+--
797+-- Definition:
798+--
799+-- >>> lengthGeneric = fmap getSum $ Fold.foldMap (Sum . const 1)
800+-- >>> lengthGeneric = Fold.foldl' (\n _ -> n + 1) 0
801+--
802+-- /Pre-release/
803+{-# INLINE lengthGeneric #-}
804+lengthGeneric :: (Monad m, Num b) => Fold m a b
805+lengthGeneric = foldl' (\n _ -> n + 1) 0
806+
807+-- | Determine the length of the input stream.
808+--
809+-- Definition:
810+--
811+-- >>> length = Fold.lengthGeneric
812+-- >>> length = fmap getSum $ Fold.foldMap (Sum . const 1)
813+--
814+{-# INLINE length #-}
815+length :: Monad m => Fold m a Int
816+length = lengthGeneric
817+
818+
819+-- | Determine the sum of all elements of a stream of numbers. Returns additive
820+-- identity (@0@) when the stream is empty. Note that this is not numerically
821+-- stable for floating point numbers.
822+--
823+-- >>> sum = FoldW.cumulative FoldW.sum
824+--
825+-- Same as following but numerically stable:
826+--
827+-- >>> sum = Fold.foldl' (+) 0
828+-- >>> sum = fmap Data.Monoid.getSum $ Fold.foldMap Data.Monoid.Sum
829+--
830+{-# INLINE sum #-}
831+sum :: (Monad m, Num a) => Fold m a a
832+sum = FoldW.cumulative FoldW.sum
833+
834+-- | Determine the product of all elements of a stream of numbers. Returns
835+-- multiplicative identity (@1@) when the stream is empty. The fold terminates
836+-- when it encounters (@0@) in its input.
837+--
838+-- Same as the following but terminates on multiplication by @0@:
839+--
840+-- >>> product = fmap Data.Monoid.getProduct $ Fold.foldMap Data.Monoid.Product
841+--
842+{-# INLINE product #-}
843+product :: (Monad m, Num a, Eq a) => Fold m a a
844+product = foldt' step (Partial 1) id
845+
846+ where
847+
848+ step x a =
849+ if a == 0
850+ then Done 0
851+ else Partial $ x * a
852+
853+------------------------------------------------------------------------------
854+-- To Summary (Maybe)
855+------------------------------------------------------------------------------
856+
857+-- | Determine the maximum element in a stream using the supplied comparison
858+-- function.
859+--
860+{-# INLINE maximumBy #-}
861+maximumBy :: Monad m => (a -> a -> Ordering) -> Fold m a (Maybe a)
862+maximumBy cmp = foldl1' max'
863+
864+ where
865+
866+ max' x y =
867+ case cmp x y of
868+ GT -> x
869+ _ -> y
870+
871+-- | Determine the maximum element in a stream.
872+--
873+-- Definitions:
874+--
875+-- >>> maximum = Fold.maximumBy compare
876+-- >>> maximum = Fold.foldl1' max
877+--
878+-- Same as the following but without a default maximum. The 'Max' Monoid uses
879+-- the 'minBound' as the default maximum:
880+--
881+-- >>> maximum = fmap Data.Semigroup.getMax $ Fold.foldMap Data.Semigroup.Max
882+--
883+{-# INLINE maximum #-}
884+maximum :: (Monad m, Ord a) => Fold m a (Maybe a)
885+maximum = foldl1' max
886+
887+-- | Computes the minimum element with respect to the given comparison function
888+--
889+{-# INLINE minimumBy #-}
890+minimumBy :: Monad m => (a -> a -> Ordering) -> Fold m a (Maybe a)
891+minimumBy cmp = foldl1' min'
892+
893+ where
894+
895+ min' x y =
896+ case cmp x y of
897+ GT -> y
898+ _ -> x
899+
900+-- | Determine the minimum element in a stream using the supplied comparison
901+-- function.
902+--
903+-- Definitions:
904+--
905+-- >>> minimum = Fold.minimumBy compare
906+-- >>> minimum = Fold.foldl1' min
907+--
908+-- Same as the following but without a default minimum. The 'Min' Monoid uses the
909+-- 'maxBound' as the default maximum:
910+--
911+-- >>> maximum = fmap Data.Semigroup.getMin $ Fold.foldMap Data.Semigroup.Min
912+--
913+{-# INLINE minimum #-}
914+minimum :: (Monad m, Ord a) => Fold m a (Maybe a)
915+minimum = foldl1' min
916+
917+------------------------------------------------------------------------------
918+-- To Summary (Statistical)
919+------------------------------------------------------------------------------
920+
921+-- | Compute a numerically stable arithmetic mean of all elements in the input
922+-- stream.
923+--
924+{-# INLINE mean #-}
925+mean :: (Monad m, Fractional a) => Fold m a a
926+mean = fmap done $ foldl' step begin
927+
928+ where
929+
930+ begin = Tuple' 0 0
931+
932+ step (Tuple' x n) y =
933+ let n1 = n + 1
934+ in Tuple' (x + (y - x) / n1) n1
935+
936+ done (Tuple' x _) = x
937+
938+-- | Compute a numerically stable (population) variance over all elements in
939+-- the input stream.
940+--
941+{-# DEPRECATED variance "Use the streamly-statistics package instead" #-}
942+{-# INLINE variance #-}
943+variance :: (Monad m, Fractional a) => Fold m a a
944+variance = fmap done $ foldl' step begin
945+
946+ where
947+
948+ begin = Tuple3' 0 0 0
949+
950+ step (Tuple3' n mean_ m2) x = Tuple3' n' mean' m2'
951+
952+ where
953+
954+ n' = n + 1
955+ mean' = (n * mean_ + x) / (n + 1)
956+ delta = x - mean_
957+ m2' = m2 + delta * delta * n / (n + 1)
958+
959+ done (Tuple3' n _ m2) = m2 / n
960+
961+-- | Compute a numerically stable (population) standard deviation over all
962+-- elements in the input stream.
963+--
964+{-# DEPRECATED stdDev "Use the streamly-statistics package instead" #-}
965+{-# INLINE stdDev #-}
966+stdDev :: (Monad m, Floating a) => Fold m a a
967+stdDev = sqrt <$> variance
968+
969+-- | Compute an 'Int' sized polynomial rolling hash
970+--
971+-- > H = salt * k ^ n + c1 * k ^ (n - 1) + c2 * k ^ (n - 2) + ... + cn * k ^ 0
972+--
973+-- Where @c1@, @c2@, @cn@ are the elements in the input stream and @k@ is a
974+-- constant.
975+--
976+-- This hash is often used in Rabin-Karp string search algorithm.
977+--
978+-- See https://en.wikipedia.org/wiki/Rolling_hash
979+--
980+{-# INLINE rollingHashWithSalt #-}
981+rollingHashWithSalt :: (Monad m, Enum a) => Int64 -> Fold m a Int64
982+rollingHashWithSalt = foldl' step
983+
984+ where
985+
986+ k = 2891336453 :: Int64
987+
988+ step cksum a = cksum * k + fromIntegral (fromEnum a)
989+
990+-- | A default salt used in the implementation of 'rollingHash'.
991+{-# INLINE defaultSalt #-}
992+defaultSalt :: Int64
993+defaultSalt = -2578643520546668380
994+
995+-- | Compute an 'Int' sized polynomial rolling hash of a stream.
996+--
997+-- >>> rollingHash = Fold.rollingHashWithSalt Fold.defaultSalt
998+--
999+{-# INLINE rollingHash #-}
1000+rollingHash :: (Monad m, Enum a) => Fold m a Int64
1001+rollingHash = rollingHashWithSalt defaultSalt
1002+
1003+-- | Compute an 'Int' sized polynomial rolling hash of the first n elements of
1004+-- a stream.
1005+--
1006+-- >>> rollingHashFirstN n = Fold.take n Fold.rollingHash
1007+--
1008+-- /Pre-release/
1009+{-# INLINE rollingHashFirstN #-}
1010+rollingHashFirstN :: (Monad m, Enum a) => Int -> Fold m a Int64
1011+rollingHashFirstN n = take n rollingHash
1012+
1013+-- XXX Compare this with the implementation in Fold.Window, preferrably use the
1014+-- latter if performance is good.
1015+
1016+-- | Apply a function on every two successive elements of a stream. The first
1017+-- argument of the map function is the previous element and the second argument
1018+-- is the current element. When processing the very first element in the
1019+-- stream, the previous element is 'Nothing'.
1020+--
1021+-- /Pre-release/
1022+--
1023+{-# INLINE rollingMapM #-}
1024+rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Fold m a b
1025+rollingMapM f = Fold step initial extract
1026+
1027+ where
1028+
1029+ -- XXX We need just a postscan. We do not need an initial result here.
1030+ -- Or we can supply a default initial result as an argument to rollingMapM.
1031+ initial = return $ Partial (Nothing, error "Empty stream")
1032+
1033+ step (prev, _) cur = do
1034+ x <- f prev cur
1035+ return $ Partial (Just cur, x)
1036+
1037+ extract = return . snd
1038+
1039+-- |
1040+-- >>> rollingMap f = Fold.rollingMapM (\x y -> return $ f x y)
1041+--
1042+{-# INLINE rollingMap #-}
1043+rollingMap :: Monad m => (Maybe a -> a -> b) -> Fold m a b
1044+rollingMap f = rollingMapM (\x y -> return $ f x y)
1045+
1046+------------------------------------------------------------------------------
1047+-- Monoidal left folds
1048+------------------------------------------------------------------------------
1049+
1050+-- | Semigroup concat. Append the elements of an input stream to a provided
1051+-- starting value.
1052+--
1053+-- Definition:
1054+--
1055+-- >>> sconcat = Fold.foldl' (<>)
1056+--
1057+-- >>> semigroups = fmap Data.Monoid.Sum $ Stream.enumerateFromTo 1 10
1058+-- >>> Stream.fold (Fold.sconcat 10) semigroups
1059+-- Sum {getSum = 65}
1060+--
1061+{-# INLINE sconcat #-}
1062+sconcat :: (Monad m, Semigroup a) => a -> Fold m a a
1063+sconcat = foldl' (<>)
1064+
1065+-- | Monoid concat. Fold an input stream consisting of monoidal elements using
1066+-- 'mappend' and 'mempty'.
1067+--
1068+-- Definition:
1069+--
1070+-- >>> mconcat = Fold.sconcat mempty
1071+--
1072+-- >>> monoids = fmap Data.Monoid.Sum $ Stream.enumerateFromTo 1 10
1073+-- >>> Stream.fold Fold.mconcat monoids
1074+-- Sum {getSum = 55}
1075+--
1076+{-# INLINE mconcat #-}
1077+mconcat ::
1078+ ( Monad m
1079+ , Monoid a) => Fold m a a
1080+mconcat = sconcat mempty
1081+
1082+-- |
1083+-- Definition:
1084+--
1085+-- >>> foldMap f = Fold.lmap f Fold.mconcat
1086+--
1087+-- Make a fold from a pure function that folds the output of the function
1088+-- using 'mappend' and 'mempty'.
1089+--
1090+-- >>> sum = Fold.foldMap Data.Monoid.Sum
1091+-- >>> Stream.fold sum $ Stream.enumerateFromTo 1 10
1092+-- Sum {getSum = 55}
1093+--
1094+{-# INLINE foldMap #-}
1095+foldMap :: (Monad m, Monoid b) => (a -> b) -> Fold m a b
1096+foldMap f = lmap f mconcat
1097+
1098+-- |
1099+-- Definition:
1100+--
1101+-- >>> foldMapM f = Fold.lmapM f Fold.mconcat
1102+--
1103+-- Make a fold from a monadic function that folds the output of the function
1104+-- using 'mappend' and 'mempty'.
1105+--
1106+-- >>> sum = Fold.foldMapM (return . Data.Monoid.Sum)
1107+-- >>> Stream.fold sum $ Stream.enumerateFromTo 1 10
1108+-- Sum {getSum = 55}
1109+--
1110+{-# INLINE foldMapM #-}
1111+foldMapM :: (Monad m, Monoid b) => (a -> m b) -> Fold m a b
1112+foldMapM act = foldlM' step (pure mempty)
1113+
1114+ where
1115+
1116+ step m a = do
1117+ m' <- act a
1118+ return $! mappend m m'
1119+
1120+------------------------------------------------------------------------------
1121+-- To Containers
1122+------------------------------------------------------------------------------
1123+
1124+-- $toListRev
1125+-- This is more efficient than 'Streamly.Internal.Data.Fold.toList'. toList is
1126+-- exactly the same as reversing the list after 'toListRev'.
1127+
1128+-- | Buffers the input stream to a list in the reverse order of the input.
1129+--
1130+-- Definition:
1131+--
1132+-- >>> toListRev = Fold.foldl' (flip (:)) []
1133+--
1134+-- /Warning!/ working on large lists accumulated as buffers in memory could be
1135+-- very inefficient, consider using "Streamly.Array" instead.
1136+--
1137+
1138+-- xn : ... : x2 : x1 : []
1139+{-# INLINE toListRev #-}
1140+toListRev :: Monad m => Fold m a [a]
1141+toListRev = foldl' (flip (:)) []
1142+
1143+------------------------------------------------------------------------------
1144+-- Partial Folds
1145+------------------------------------------------------------------------------
1146+
1147+-- | A fold that drains the first n elements of its input, running the effects
1148+-- and discarding the results.
1149+--
1150+-- Definition:
1151+--
1152+-- >>> drainN n = Fold.take n Fold.drain
1153+--
1154+-- /Pre-release/
1155+{-# INLINE drainN #-}
1156+drainN :: Monad m => Int -> Fold m a ()
1157+drainN n = take n drain
1158+
1159+------------------------------------------------------------------------------
1160+-- To Elements
1161+------------------------------------------------------------------------------
1162+
1163+-- | Like 'index', except with a more general 'Integral' argument
1164+--
1165+-- /Pre-release/
1166+{-# INLINE indexGeneric #-}
1167+indexGeneric :: (Integral i, Monad m) => i -> Fold m a (Maybe a)
1168+indexGeneric i = foldt' step (Partial 0) (const Nothing)
1169+
1170+ where
1171+
1172+ step j a =
1173+ if i == j
1174+ then Done $ Just a
1175+ else Partial (j + 1)
1176+
1177+-- | Return the element at the given index.
1178+--
1179+-- Definition:
1180+--
1181+-- >>> index = Fold.indexGeneric
1182+--
1183+{-# INLINE index #-}
1184+index :: Monad m => Int -> Fold m a (Maybe a)
1185+index = indexGeneric
1186+
1187+-- | Consume a single input and transform it using the supplied 'Maybe'
1188+-- returning function.
1189+--
1190+-- /Pre-release/
1191+--
1192+{-# INLINE maybe #-}
1193+maybe :: Monad m => (a -> Maybe b) -> Fold m a (Maybe b)
1194+maybe f = foldt' (const (Done . f)) (Partial Nothing) id
1195+
1196+-- | Consume a single element and return it if it passes the predicate else
1197+-- return 'Nothing'.
1198+--
1199+-- Definition:
1200+--
1201+-- >>> satisfy f = Fold.maybe (\a -> if f a then Just a else Nothing)
1202+--
1203+-- /Pre-release/
1204+{-# INLINE satisfy #-}
1205+satisfy :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1206+satisfy f = maybe (\a -> if f a then Just a else Nothing)
1207+{-
1208+satisfy f = Fold step (return $ Partial ()) (const (return Nothing))
1209+
1210+ where
1211+
1212+ step () a = return $ Done $ if f a then Just a else Nothing
1213+-}
1214+
1215+-- Naming notes:
1216+--
1217+-- "head" and "next" are two alternative names for the same API. head sounds
1218+-- apt in the context of lists but next sounds more apt in the context of
1219+-- streams where we think in terms of generating and consuming the next element
1220+-- rather than taking the head of some static/persistent structure.
1221+--
1222+-- We also want to keep the nomenclature consistent across folds and parsers,
1223+-- "head" becomes even more unintuitive for parsers because there are two
1224+-- possible variants viz. peek and next.
1225+--
1226+-- Also, the "head" fold creates confusion in situations like
1227+-- https://github.com/composewell/streamly/issues/1404 where intuitive
1228+-- expectation from head is to consume the entire stream and just give us the
1229+-- head. There we want to convey the notion that we consume one element from
1230+-- the stream and stop. The name "one" already being used in parsers for this
1231+-- purpose sounds more apt from this perspective.
1232+--
1233+-- The source of confusion is perhaps due to the fact that some folds consume
1234+-- the entire stream and others terminate early. It may have been clearer if we
1235+-- had separate abstractions for the two use cases.
1236+
1237+-- XXX We can possibly use "head" for the purposes of reducing the entire
1238+-- stream to the head element i.e. take the head and drain the rest.
1239+
1240+-- | Take one element from the stream and stop.
1241+--
1242+-- Definition:
1243+--
1244+-- >>> one = Fold.maybe Just
1245+--
1246+-- This is similar to the stream 'Stream.uncons' operation.
1247+--
1248+{-# INLINE one #-}
1249+one :: Monad m => Fold m a (Maybe a)
1250+one = maybe Just
1251+
1252+-- | Extract the first element of the stream, if any.
1253+--
1254+-- >>> head = Fold.one
1255+--
1256+{-# DEPRECATED head "Please use \"one\" instead" #-}
1257+{-# INLINE head #-}
1258+head :: Monad m => Fold m a (Maybe a)
1259+head = one
1260+
1261+-- | Returns the first element that satisfies the given predicate.
1262+--
1263+-- /Pre-release/
1264+{-# INLINE findM #-}
1265+findM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1266+findM predicate = Fold step (return $ Partial ()) (const $ return Nothing)
1267+
1268+ where
1269+
1270+ step () a =
1271+ let f r =
1272+ if r
1273+ then Done (Just a)
1274+ else Partial ()
1275+ in f <$> predicate a
1276+
1277+-- | Returns the first element that satisfies the given predicate.
1278+--
1279+{-# INLINE find #-}
1280+find :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1281+find p = findM (return . p)
1282+
1283+-- | In a stream of (key-value) pairs @(a, b)@, return the value @b@ of the
1284+-- first pair where the key equals the given value @a@.
1285+--
1286+-- Definition:
1287+--
1288+-- >>> lookup x = fmap snd <$> Fold.find ((== x) . fst)
1289+--
1290+{-# INLINE lookup #-}
1291+lookup :: (Eq a, Monad m) => a -> Fold m (a,b) (Maybe b)
1292+lookup a0 = foldt' step (Partial ()) (const Nothing)
1293+
1294+ where
1295+
1296+ step () (a, b) =
1297+ if a == a0
1298+ then Done $ Just b
1299+ else Partial ()
1300+
1301+-- | Returns the first index that satisfies the given predicate.
1302+--
1303+{-# INLINE findIndex #-}
1304+findIndex :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
1305+findIndex predicate = foldt' step (Partial 0) (const Nothing)
1306+
1307+ where
1308+
1309+ step i a =
1310+ if predicate a
1311+ then Done $ Just i
1312+ else Partial (i + 1)
1313+
1314+-- | Returns the index of the latest element if the element satisfies the given
1315+-- predicate.
1316+--
1317+{-# INLINE findIndices #-}
1318+findIndices :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
1319+findIndices predicate =
1320+ -- XXX implement by combining indexing and filtering scans
1321+ fmap (either (const Nothing) Just) $ foldl' step (Left (-1))
1322+
1323+ where
1324+
1325+ step i a =
1326+ if predicate a
1327+ then Right (either id id i + 1)
1328+ else Left (either id id i + 1)
1329+
1330+-- | Returns the index of the latest element if the element matches the given
1331+-- value.
1332+--
1333+-- Definition:
1334+--
1335+-- >>> elemIndices a = Fold.findIndices (== a)
1336+--
1337+{-# INLINE elemIndices #-}
1338+elemIndices :: (Monad m, Eq a) => a -> Fold m a (Maybe Int)
1339+elemIndices a = findIndices (== a)
1340+
1341+-- | Returns the first index where a given value is found in the stream.
1342+--
1343+-- Definition:
1344+--
1345+-- >>> elemIndex a = Fold.findIndex (== a)
1346+--
1347+{-# INLINE elemIndex #-}
1348+elemIndex :: (Eq a, Monad m) => a -> Fold m a (Maybe Int)
1349+elemIndex a = findIndex (== a)
1350+
1351+------------------------------------------------------------------------------
1352+-- To Boolean
1353+------------------------------------------------------------------------------
1354+
1355+-- Similar to 'eof' parser, but the fold consumes and discards an input element
1356+-- when not at eof. XXX Remove or Rename to "eof"?
1357+
1358+-- | Consume one element, return 'True' if successful else return 'False'. In
1359+-- other words, test if the input is empty or not.
1360+--
1361+-- WARNING! It consumes one element if the stream is not empty. If that is not
1362+-- what you want please use the eof parser instead.
1363+--
1364+-- Definition:
1365+--
1366+-- >>> null = fmap isJust Fold.one
1367+--
1368+{-# INLINE null #-}
1369+null :: Monad m => Fold m a Bool
1370+null = foldt' (\() _ -> Done False) (Partial ()) (const True)
1371+
1372+-- | Returns 'True' if any element of the input satisfies the predicate.
1373+--
1374+-- Definition:
1375+--
1376+-- >>> any p = Fold.lmap p Fold.or
1377+--
1378+-- Example:
1379+--
1380+-- >>> Stream.fold (Fold.any (== 0)) $ Stream.fromList [1,0,1]
1381+-- True
1382+--
1383+{-# INLINE any #-}
1384+any :: Monad m => (a -> Bool) -> Fold m a Bool
1385+any predicate = foldt' step initial id
1386+
1387+ where
1388+
1389+ initial = Partial False
1390+
1391+ step _ a =
1392+ if predicate a
1393+ then Done True
1394+ else Partial False
1395+
1396+-- | Return 'True' if the given element is present in the stream.
1397+--
1398+-- Definition:
1399+--
1400+-- >>> elem a = Fold.any (== a)
1401+--
1402+{-# INLINE elem #-}
1403+elem :: (Eq a, Monad m) => a -> Fold m a Bool
1404+elem a = any (== a)
1405+
1406+-- | Returns 'True' if all elements of the input satisfy the predicate.
1407+--
1408+-- Definition:
1409+--
1410+-- >>> all p = Fold.lmap p Fold.and
1411+--
1412+-- Example:
1413+--
1414+-- >>> Stream.fold (Fold.all (== 0)) $ Stream.fromList [1,0,1]
1415+-- False
1416+--
1417+{-# INLINE all #-}
1418+all :: Monad m => (a -> Bool) -> Fold m a Bool
1419+all predicate = foldt' step initial id
1420+
1421+ where
1422+
1423+ initial = Partial True
1424+
1425+ step _ a =
1426+ if predicate a
1427+ then Partial True
1428+ else Done False
1429+
1430+-- | Returns 'True' if the given element is not present in the stream.
1431+--
1432+-- Definition:
1433+--
1434+-- >>> notElem a = Fold.all (/= a)
1435+--
1436+{-# INLINE notElem #-}
1437+notElem :: (Eq a, Monad m) => a -> Fold m a Bool
1438+notElem a = all (/= a)
1439+
1440+-- | Returns 'True' if all elements are 'True', 'False' otherwise
1441+--
1442+-- Definition:
1443+--
1444+-- >>> and = Fold.all (== True)
1445+--
1446+{-# INLINE and #-}
1447+and :: Monad m => Fold m Bool Bool
1448+and = all (== True)
1449+
1450+-- | Returns 'True' if any element is 'True', 'False' otherwise
1451+--
1452+-- Definition:
1453+--
1454+-- >>> or = Fold.any (== True)
1455+--
1456+{-# INLINE or #-}
1457+or :: Monad m => Fold m Bool Bool
1458+or = any (== True)
1459+
1460+------------------------------------------------------------------------------
1461+-- Grouping/Splitting
1462+------------------------------------------------------------------------------
1463+
1464+------------------------------------------------------------------------------
1465+-- Grouping without looking at elements
1466+------------------------------------------------------------------------------
1467+
1468+------------------------------------------------------------------------------
1469+-- Binary APIs
1470+------------------------------------------------------------------------------
1471+
1472+-- | @splitAt n f1 f2@ composes folds @f1@ and @f2@ such that first @n@
1473+-- elements of its input are consumed by fold @f1@ and the rest of the stream
1474+-- is consumed by fold @f2@.
1475+--
1476+-- >>> let splitAt_ n xs = Stream.fold (Fold.splitAt n Fold.toList Fold.toList) $ Stream.fromList xs
1477+--
1478+-- >>> splitAt_ 6 "Hello World!"
1479+-- ("Hello ","World!")
1480+--
1481+-- >>> splitAt_ (-1) [1,2,3]
1482+-- ([],[1,2,3])
1483+--
1484+-- >>> splitAt_ 0 [1,2,3]
1485+-- ([],[1,2,3])
1486+--
1487+-- >>> splitAt_ 1 [1,2,3]
1488+-- ([1],[2,3])
1489+--
1490+-- >>> splitAt_ 3 [1,2,3]
1491+-- ([1,2,3],[])
1492+--
1493+-- >>> splitAt_ 4 [1,2,3]
1494+-- ([1,2,3],[])
1495+--
1496+-- > splitAt n f1 f2 = Fold.splitWith (,) (Fold.take n f1) f2
1497+--
1498+-- /Internal/
1499+
1500+{-# INLINE splitAt #-}
1501+splitAt
1502+ :: Monad m
1503+ => Int
1504+ -> Fold m a b
1505+ -> Fold m a c
1506+ -> Fold m a (b, c)
1507+splitAt n fld = splitWith (,) (take n fld)
1508+
1509+------------------------------------------------------------------------------
1510+-- Element Aware APIs
1511+------------------------------------------------------------------------------
1512+--
1513+------------------------------------------------------------------------------
1514+-- Binary APIs
1515+------------------------------------------------------------------------------
1516+
1517+{-# INLINE takingEndByM #-}
1518+takingEndByM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1519+takingEndByM p = Fold step initial (return . toMaybe)
1520+
1521+ where
1522+
1523+ initial = return $ Partial Nothing'
1524+
1525+ step _ a = do
1526+ r <- p a
1527+ return
1528+ $ if r
1529+ then Done $ Just a
1530+ else Partial $ Just' a
1531+
1532+-- |
1533+--
1534+-- >>> takingEndBy p = Fold.takingEndByM (return . p)
1535+--
1536+{-# INLINE takingEndBy #-}
1537+takingEndBy :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1538+takingEndBy p = takingEndByM (return . p)
1539+
1540+{-# INLINE takingEndByM_ #-}
1541+takingEndByM_ :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1542+takingEndByM_ p = Fold step initial (return . toMaybe)
1543+
1544+ where
1545+
1546+ initial = return $ Partial Nothing'
1547+
1548+ step _ a = do
1549+ r <- p a
1550+ return
1551+ $ if r
1552+ then Done Nothing
1553+ else Partial $ Just' a
1554+
1555+-- |
1556+--
1557+-- >>> takingEndBy_ p = Fold.takingEndByM_ (return . p)
1558+--
1559+{-# INLINE takingEndBy_ #-}
1560+takingEndBy_ :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1561+takingEndBy_ p = takingEndByM_ (return . p)
1562+
1563+{-# INLINE droppingWhileM #-}
1564+droppingWhileM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1565+droppingWhileM p = Fold step initial (return . toMaybe)
1566+
1567+ where
1568+
1569+ initial = return $ Partial Nothing'
1570+
1571+ step Nothing' a = do
1572+ r <- p a
1573+ return
1574+ $ Partial
1575+ $ if r
1576+ then Nothing'
1577+ else Just' a
1578+ step _ a = return $ Partial $ Just' a
1579+
1580+-- |
1581+-- >>> droppingWhile p = Fold.droppingWhileM (return . p)
1582+--
1583+{-# INLINE droppingWhile #-}
1584+droppingWhile :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1585+droppingWhile p = droppingWhileM (return . p)
1586+
1587+-- Note: Keep this consistent with S.splitOn. In fact we should eliminate
1588+-- S.splitOn in favor of the fold.
1589+--
1590+-- XXX Use Fold.many instead once it is fixed.
1591+-- > Stream.splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
1592+
1593+-- | Like 'takeEndBy' but drops the element on which the predicate succeeds.
1594+--
1595+-- Example:
1596+--
1597+-- >>> input = Stream.fromList "hello\nthere\n"
1598+-- >>> line = Fold.takeEndBy_ (== '\n') Fold.toList
1599+-- >>> Stream.fold line input
1600+-- "hello"
1601+--
1602+-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1603+-- ["hello","there"]
1604+--
1605+{-# INLINE takeEndBy_ #-}
1606+takeEndBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1607+-- takeEndBy_ predicate = scanMaybe (takingEndBy_ predicate)
1608+takeEndBy_ predicate (Fold fstep finitial fextract) =
1609+ Fold step finitial fextract
1610+
1611+ where
1612+
1613+ step s a =
1614+ if not (predicate a)
1615+ then fstep s a
1616+ else Done <$> fextract s
1617+
1618+-- Note:
1619+-- > Stream.splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
1620+
1621+-- | Take the input, stop when the predicate succeeds taking the succeeding
1622+-- element as well.
1623+--
1624+-- Example:
1625+--
1626+-- >>> input = Stream.fromList "hello\nthere\n"
1627+-- >>> line = Fold.takeEndBy (== '\n') Fold.toList
1628+-- >>> Stream.fold line input
1629+-- "hello\n"
1630+--
1631+-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1632+-- ["hello\n","there\n"]
1633+--
1634+{-# INLINE takeEndBy #-}
1635+takeEndBy :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1636+-- takeEndBy predicate = scanMaybe (takingEndBy predicate)
1637+takeEndBy predicate (Fold fstep finitial fextract) =
1638+ Fold step finitial fextract
1639+
1640+ where
1641+
1642+ step s a = do
1643+ res <- fstep s a
1644+ if not (predicate a)
1645+ then return res
1646+ else do
1647+ case res of
1648+ Partial s1 -> Done <$> fextract s1
1649+ Done b -> return $ Done b
1650+
1651+------------------------------------------------------------------------------
1652+-- Binary splitting on a separator
1653+------------------------------------------------------------------------------
1654+
1655+data SplitOnSeqState acc a rb rh w ck =
1656+ SplitOnSeqEmpty !acc
1657+ | SplitOnSeqSingle !acc !a
1658+ | SplitOnSeqWord !acc !Int !w
1659+ | SplitOnSeqWordLoop !acc !w
1660+ | SplitOnSeqKR !acc !Int !rb !rh
1661+ | SplitOnSeqKRLoop !acc !ck !rb !rh
1662+
1663+-- XXX Need to add tests for takeEndBySeq, we have tests for takeEndBySeq_ .
1664+
1665+-- | Continue taking the input until the input sequence matches the supplied
1666+-- sequence, taking the supplied sequence as well. If the pattern is empty this
1667+-- acts as an identity fold.
1668+--
1669+-- >>> s = Stream.fromList "hello there. How are you?"
1670+-- >>> f = Fold.takeEndBySeq (Array.fromList "re") Fold.toList
1671+-- >>> Stream.fold f s
1672+-- "hello there"
1673+--
1674+-- >>> Stream.fold Fold.toList $ Stream.foldMany f s
1675+-- ["hello there",". How are"," you?"]
1676+--
1677+-- /Pre-release/
1678+{-# INLINE takeEndBySeq #-}
1679+takeEndBySeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
1680+ Array.Array a
1681+ -> Fold m a b
1682+ -> Fold m a b
1683+takeEndBySeq patArr (Fold fstep finitial fextract) =
1684+ Fold step initial extract
1685+
1686+ where
1687+
1688+ patLen = Array.length patArr
1689+
1690+ initial = do
1691+ res <- finitial
1692+ case res of
1693+ Partial acc
1694+ | patLen == 0 ->
1695+ -- XXX Should we match nothing or everything on empty
1696+ -- pattern?
1697+ -- Done <$> fextract acc
1698+ return $ Partial $ SplitOnSeqEmpty acc
1699+ | patLen == 1 -> do
1700+ pat <- liftIO $ Array.unsafeIndexIO 0 patArr
1701+ return $ Partial $ SplitOnSeqSingle acc pat
1702+ | SIZE_OF(a) * patLen <= sizeOf (Proxy :: Proxy Word) ->
1703+ return $ Partial $ SplitOnSeqWord acc 0 0
1704+ | otherwise -> do
1705+ (rb, rhead) <- liftIO $ Ring.new patLen
1706+ return $ Partial $ SplitOnSeqKR acc 0 rb rhead
1707+ Done b -> return $ Done b
1708+
1709+ -- Word pattern related
1710+ maxIndex = patLen - 1
1711+
1712+ elemBits = SIZE_OF(a) * 8
1713+
1714+ wordMask :: Word
1715+ wordMask = (1 `shiftL` (elemBits * patLen)) - 1
1716+
1717+ wordPat :: Word
1718+ wordPat = wordMask .&. Array.foldl' addToWord 0 patArr
1719+
1720+ addToWord wd a = (wd `shiftL` elemBits) .|. fromIntegral (fromEnum a)
1721+
1722+ -- For Rabin-Karp search
1723+ k = 2891336453 :: Word32
1724+ coeff = k ^ patLen
1725+
1726+ addCksum cksum a = cksum * k + fromIntegral (fromEnum a)
1727+
1728+ deltaCksum cksum old new =
1729+ addCksum cksum new - coeff * fromIntegral (fromEnum old)
1730+
1731+ -- XXX shall we use a random starting hash or 1 instead of 0?
1732+ -- XXX Need to keep this cached across fold calls in foldmany
1733+ -- XXX We may need refold to inject the cached state instead of
1734+ -- initializing the state every time.
1735+ -- XXX Allocation of ring buffer should also be done once
1736+ patHash = Array.foldl' addCksum 0 patArr
1737+
1738+ step (SplitOnSeqEmpty s) x = do
1739+ res <- fstep s x
1740+ case res of
1741+ Partial s1 -> return $ Partial $ SplitOnSeqEmpty s1
1742+ Done b -> return $ Done b
1743+ step (SplitOnSeqSingle s pat) x = do
1744+ res <- fstep s x
1745+ case res of
1746+ Partial s1
1747+ | pat /= x -> return $ Partial $ SplitOnSeqSingle s1 pat
1748+ | otherwise -> Done <$> fextract s1
1749+ Done b -> return $ Done b
1750+ step (SplitOnSeqWord s idx wrd) x = do
1751+ res <- fstep s x
1752+ let wrd1 = addToWord wrd x
1753+ case res of
1754+ Partial s1
1755+ | idx == maxIndex -> do
1756+ if wrd1 .&. wordMask == wordPat
1757+ then Done <$> fextract s1
1758+ else return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1759+ | otherwise ->
1760+ return $ Partial $ SplitOnSeqWord s1 (idx + 1) wrd1
1761+ Done b -> return $ Done b
1762+ step (SplitOnSeqWordLoop s wrd) x = do
1763+ res <- fstep s x
1764+ let wrd1 = addToWord wrd x
1765+ case res of
1766+ Partial s1
1767+ | wrd1 .&. wordMask == wordPat ->
1768+ Done <$> fextract s1
1769+ | otherwise ->
1770+ return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1771+ Done b -> return $ Done b
1772+ step (SplitOnSeqKR s idx rb rh) x = do
1773+ res <- fstep s x
1774+ case res of
1775+ Partial s1 -> do
1776+ rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1777+ if idx == maxIndex
1778+ then do
1779+ let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
1780+ let !ringHash = fld addCksum 0 rb
1781+ if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1782+ then Done <$> fextract s1
1783+ else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1784+ else
1785+ return $ Partial $ SplitOnSeqKR s1 (idx + 1) rb rh1
1786+ Done b -> return $ Done b
1787+ step (SplitOnSeqKRLoop s cksum rb rh) x = do
1788+ res <- fstep s x
1789+ case res of
1790+ Partial s1 -> do
1791+ old <- liftIO $ peek rh
1792+ rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1793+ let ringHash = deltaCksum cksum old x
1794+ if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1795+ then Done <$> fextract s1
1796+ else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1797+ Done b -> return $ Done b
1798+
1799+ extract state =
1800+ let st =
1801+ case state of
1802+ SplitOnSeqEmpty s -> s
1803+ SplitOnSeqSingle s _ -> s
1804+ SplitOnSeqWord s _ _ -> s
1805+ SplitOnSeqWordLoop s _ -> s
1806+ SplitOnSeqKR s _ _ _ -> s
1807+ SplitOnSeqKRLoop s _ _ _ -> s
1808+ in fextract st
1809+
1810+-- | Like 'takeEndBySeq' but discards the matched sequence.
1811+--
1812+-- /Pre-release/
1813+--
1814+{-# INLINE takeEndBySeq_ #-}
1815+takeEndBySeq_ :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
1816+ Array.Array a
1817+ -> Fold m a b
1818+ -> Fold m a b
1819+takeEndBySeq_ patArr (Fold fstep finitial fextract) =
1820+ Fold step initial extract
1821+
1822+ where
1823+
1824+ patLen = Array.length patArr
1825+
1826+ initial = do
1827+ res <- finitial
1828+ case res of
1829+ Partial acc
1830+ | patLen == 0 ->
1831+ -- XXX Should we match nothing or everything on empty
1832+ -- pattern?
1833+ -- Done <$> fextract acc
1834+ return $ Partial $ SplitOnSeqEmpty acc
1835+ | patLen == 1 -> do
1836+ pat <- liftIO $ Array.unsafeIndexIO 0 patArr
1837+ return $ Partial $ SplitOnSeqSingle acc pat
1838+ -- XXX Need to add tests for this case
1839+ | SIZE_OF(a) * patLen <= sizeOf (Proxy :: Proxy Word) ->
1840+ return $ Partial $ SplitOnSeqWord acc 0 0
1841+ | otherwise -> do
1842+ (rb, rhead) <- liftIO $ Ring.new patLen
1843+ return $ Partial $ SplitOnSeqKR acc 0 rb rhead
1844+ Done b -> return $ Done b
1845+
1846+ -- Word pattern related
1847+ maxIndex = patLen - 1
1848+
1849+ elemBits = SIZE_OF(a) * 8
1850+
1851+ wordMask :: Word
1852+ wordMask = (1 `shiftL` (elemBits * patLen)) - 1
1853+
1854+ elemMask :: Word
1855+ elemMask = (1 `shiftL` elemBits) - 1
1856+
1857+ wordPat :: Word
1858+ wordPat = wordMask .&. Array.foldl' addToWord 0 patArr
1859+
1860+ addToWord wd a = (wd `shiftL` elemBits) .|. fromIntegral (fromEnum a)
1861+
1862+ -- For Rabin-Karp search
1863+ k = 2891336453 :: Word32
1864+ coeff = k ^ patLen
1865+
1866+ addCksum cksum a = cksum * k + fromIntegral (fromEnum a)
1867+
1868+ deltaCksum cksum old new =
1869+ addCksum cksum new - coeff * fromIntegral (fromEnum old)
1870+
1871+ -- XXX shall we use a random starting hash or 1 instead of 0?
1872+ -- XXX Need to keep this cached across fold calls in foldMany
1873+ -- XXX We may need refold to inject the cached state instead of
1874+ -- initializing the state every time.
1875+ -- XXX Allocation of ring buffer should also be done once
1876+ patHash = Array.foldl' addCksum 0 patArr
1877+
1878+ step (SplitOnSeqEmpty s) x = do
1879+ res <- fstep s x
1880+ case res of
1881+ Partial s1 -> return $ Partial $ SplitOnSeqEmpty s1
1882+ Done b -> return $ Done b
1883+ step (SplitOnSeqSingle s pat) x = do
1884+ if pat /= x
1885+ then do
1886+ res <- fstep s x
1887+ case res of
1888+ Partial s1 -> return $ Partial $ SplitOnSeqSingle s1 pat
1889+ Done b -> return $ Done b
1890+ else Done <$> fextract s
1891+ step (SplitOnSeqWord s idx wrd) x = do
1892+ let wrd1 = addToWord wrd x
1893+ if idx == maxIndex
1894+ then do
1895+ if wrd1 .&. wordMask == wordPat
1896+ then Done <$> fextract s
1897+ else return $ Partial $ SplitOnSeqWordLoop s wrd1
1898+ else return $ Partial $ SplitOnSeqWord s (idx + 1) wrd1
1899+ step (SplitOnSeqWordLoop s wrd) x = do
1900+ let wrd1 = addToWord wrd x
1901+ old = (wordMask .&. wrd)
1902+ `shiftR` (elemBits * (patLen - 1))
1903+ res <- fstep s (toEnum $ fromIntegral old)
1904+ case res of
1905+ Partial s1
1906+ | wrd1 .&. wordMask == wordPat ->
1907+ Done <$> fextract s1
1908+ | otherwise ->
1909+ return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1910+ Done b -> return $ Done b
1911+ step (SplitOnSeqKR s idx rb rh) x = do
1912+ rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1913+ if idx == maxIndex
1914+ then do
1915+ let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
1916+ let !ringHash = fld addCksum 0 rb
1917+ if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1918+ then Done <$> fextract s
1919+ else return $ Partial $ SplitOnSeqKRLoop s ringHash rb rh1
1920+ else return $ Partial $ SplitOnSeqKR s (idx + 1) rb rh1
1921+ step (SplitOnSeqKRLoop s cksum rb rh) x = do
1922+ old <- liftIO $ peek rh
1923+ res <- fstep s old
1924+ case res of
1925+ Partial s1 -> do
1926+ rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1927+ let ringHash = deltaCksum cksum old x
1928+ if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1929+ then Done <$> fextract s1
1930+ else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1931+ Done b -> return $ Done b
1932+
1933+ -- XXX extract should return backtrack count as well. If the fold
1934+ -- terminates early inside extract, we may still have buffered data
1935+ -- remaining which will be lost if we do not communicate that to the
1936+ -- driver.
1937+ extract state = do
1938+ let consumeWord s n wrd = do
1939+ if n == 0
1940+ then fextract s
1941+ else do
1942+ let old = elemMask .&. (wrd `shiftR` (elemBits * (n - 1)))
1943+ r <- fstep s (toEnum $ fromIntegral old)
1944+ case r of
1945+ Partial s1 -> consumeWord s1 (n - 1) wrd
1946+ Done b -> return b
1947+
1948+ let consumeRing s n rb rh =
1949+ if n == 0
1950+ then fextract s
1951+ else do
1952+ old <- liftIO $ peek rh
1953+ let rh1 = Ring.advance rb rh
1954+ r <- fstep s old
1955+ case r of
1956+ Partial s1 -> consumeRing s1 (n - 1) rb rh1
1957+ Done b -> return b
1958+
1959+ case state of
1960+ SplitOnSeqEmpty s -> fextract s
1961+ SplitOnSeqSingle s _ -> fextract s
1962+ SplitOnSeqWord s idx wrd -> consumeWord s idx wrd
1963+ SplitOnSeqWordLoop s wrd -> consumeWord s patLen wrd
1964+ SplitOnSeqKR s idx rb _ -> consumeRing s idx rb (Ring.startOf rb)
1965+ SplitOnSeqKRLoop s _ rb rh -> consumeRing s patLen rb rh
1966+
1967+------------------------------------------------------------------------------
1968+-- Distributing
1969+------------------------------------------------------------------------------
1970+--
1971+-- | Distribute one copy of the stream to each fold and zip the results.
1972+--
1973+-- @
1974+-- |-------Fold m a b--------|
1975+-- ---stream m a---| |---m (b,c)
1976+-- |-------Fold m a c--------|
1977+-- @
1978+--
1979+-- Definition:
1980+--
1981+-- >>> tee = Fold.teeWith (,)
1982+--
1983+-- Example:
1984+--
1985+-- >>> t = Fold.tee Fold.sum Fold.length
1986+-- >>> Stream.fold t (Stream.enumerateFromTo 1.0 100.0)
1987+-- (5050.0,100)
1988+--
1989+{-# INLINE tee #-}
1990+tee :: Monad m => Fold m a b -> Fold m a c -> Fold m a (b,c)
1991+tee = teeWith (,)
1992+
1993+-- XXX use "List" instead of "[]"?, use Array for output to scale it to a large
1994+-- number of consumers? For polymorphic case a vector could be helpful. For
1995+-- Storables we can use arrays. Will need separate APIs for those.
1996+--
1997+-- | Distribute one copy of the stream to each fold and collect the results in
1998+-- a container.
1999+--
2000+-- @
2001+--
2002+-- |-------Fold m a b--------|
2003+-- ---stream m a---| |---m [b]
2004+-- |-------Fold m a b--------|
2005+-- | |
2006+-- ...
2007+-- @
2008+--
2009+-- >>> Stream.fold (Fold.distribute [Fold.sum, Fold.length]) (Stream.enumerateFromTo 1 5)
2010+-- [15,5]
2011+--
2012+-- >>> distribute = Prelude.foldr (Fold.teeWith (:)) (Fold.fromPure [])
2013+--
2014+-- This is the consumer side dual of the producer side 'sequence' operation.
2015+--
2016+-- Stops when all the folds stop.
2017+--
2018+{-# INLINE distribute #-}
2019+distribute :: Monad m => [Fold m a b] -> Fold m a [b]
2020+distribute = Prelude.foldr (teeWith (:)) (fromPure [])
2021+
2022+------------------------------------------------------------------------------
2023+-- Partitioning
2024+------------------------------------------------------------------------------
2025+
2026+{-# INLINE partitionByMUsing #-}
2027+partitionByMUsing :: Monad m =>
2028+ ( (x -> y -> (x, y))
2029+ -> Fold m (Either b c) x
2030+ -> Fold m (Either b c) y
2031+ -> Fold m (Either b c) (x, y)
2032+ )
2033+ -> (a -> m (Either b c))
2034+ -> Fold m b x
2035+ -> Fold m c y
2036+ -> Fold m a (x, y)
2037+partitionByMUsing t f fld1 fld2 =
2038+ let l = lmap (fromLeft undefined) fld1 -- :: Fold m (Either b c) x
2039+ r = lmap (fromRight undefined) fld2 -- :: Fold m (Either b c) y
2040+ in lmapM f (t (,) (filter isLeft l) (filter isRight r))
2041+
2042+-- | Partition the input over two folds using an 'Either' partitioning
2043+-- predicate.
2044+--
2045+-- @
2046+--
2047+-- |-------Fold b x--------|
2048+-- -----stream m a --> (Either b c)----| |----(x,y)
2049+-- |-------Fold c y--------|
2050+-- @
2051+--
2052+-- Example, send input to either fold randomly:
2053+--
2054+-- >>> :set -package random
2055+-- >>> import System.Random (randomIO)
2056+-- >>> randomly a = randomIO >>= \x -> return $ if x then Left a else Right a
2057+-- >>> f = Fold.partitionByM randomly Fold.length Fold.length
2058+-- >>> Stream.fold f (Stream.enumerateFromTo 1 100)
2059+-- ...
2060+--
2061+-- Example, send input to the two folds in a proportion of 2:1:
2062+--
2063+-- >>> :{
2064+-- proportionately m n = do
2065+-- ref <- newIORef $ cycle $ concat [replicate m Left, replicate n Right]
2066+-- return $ \a -> do
2067+-- r <- readIORef ref
2068+-- writeIORef ref $ tail r
2069+-- return $ Prelude.head r a
2070+-- :}
2071+--
2072+-- >>> :{
2073+-- main = do
2074+-- g <- proportionately 2 1
2075+-- let f = Fold.partitionByM g Fold.length Fold.length
2076+-- r <- Stream.fold f (Stream.enumerateFromTo (1 :: Int) 100)
2077+-- print r
2078+-- :}
2079+--
2080+-- >>> main
2081+-- (67,33)
2082+--
2083+--
2084+-- This is the consumer side dual of the producer side 'mergeBy' operation.
2085+--
2086+-- When one fold is done, any input meant for it is ignored until the other
2087+-- fold is also done.
2088+--
2089+-- Stops when both the folds stop.
2090+--
2091+-- /See also: 'partitionByFstM' and 'partitionByMinM'./
2092+--
2093+-- /Pre-release/
2094+{-# INLINE partitionByM #-}
2095+partitionByM :: Monad m
2096+ => (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2097+partitionByM = partitionByMUsing teeWith
2098+
2099+-- | Similar to 'partitionByM' but terminates when the first fold terminates.
2100+--
2101+{-# INLINE partitionByFstM #-}
2102+partitionByFstM :: Monad m
2103+ => (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2104+partitionByFstM = partitionByMUsing teeWithFst
2105+
2106+-- | Similar to 'partitionByM' but terminates when any fold terminates.
2107+--
2108+{-# INLINE partitionByMinM #-}
2109+partitionByMinM :: Monad m =>
2110+ (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2111+partitionByMinM = partitionByMUsing teeWithMin
2112+
2113+-- Note: we could use (a -> Bool) instead of (a -> Either b c), but the latter
2114+-- makes the signature clearer as to which case belongs to which fold.
2115+-- XXX need to check the performance in both cases.
2116+
2117+-- | Same as 'partitionByM' but with a pure partition function.
2118+--
2119+-- Example, count even and odd numbers in a stream:
2120+--
2121+-- >>> :{
2122+-- let f = Fold.partitionBy (\n -> if even n then Left n else Right n)
2123+-- (fmap (("Even " ++) . show) Fold.length)
2124+-- (fmap (("Odd " ++) . show) Fold.length)
2125+-- in Stream.fold f (Stream.enumerateFromTo 1 100)
2126+-- :}
2127+-- ("Even 50","Odd 50")
2128+--
2129+-- /Pre-release/
2130+{-# INLINE partitionBy #-}
2131+partitionBy :: Monad m
2132+ => (a -> Either b c) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2133+partitionBy f = partitionByM (return . f)
2134+
2135+-- | Compose two folds such that the combined fold accepts a stream of 'Either'
2136+-- and routes the 'Left' values to the first fold and 'Right' values to the
2137+-- second fold.
2138+--
2139+-- Definition:
2140+--
2141+-- >>> partition = Fold.partitionBy id
2142+--
2143+{-# INLINE partition #-}
2144+partition :: Monad m
2145+ => Fold m b x -> Fold m c y -> Fold m (Either b c) (x, y)
2146+partition = partitionBy id
2147+
2148+{-
2149+-- | Send one item to each fold in a round-robin fashion. This is the consumer
2150+-- side dual of producer side 'mergeN' operation.
2151+--
2152+-- partitionN :: Monad m => [Fold m a b] -> Fold m a [b]
2153+-- partitionN fs = Fold step begin done
2154+-}
2155+
2156+------------------------------------------------------------------------------
2157+-- Unzipping
2158+------------------------------------------------------------------------------
2159+
2160+{-# INLINE unzipWithMUsing #-}
2161+unzipWithMUsing :: Monad m =>
2162+ ( (x -> y -> (x, y))
2163+ -> Fold m (b, c) x
2164+ -> Fold m (b, c) y
2165+ -> Fold m (b, c) (x, y)
2166+ )
2167+ -> (a -> m (b, c))
2168+ -> Fold m b x
2169+ -> Fold m c y
2170+ -> Fold m a (x, y)
2171+unzipWithMUsing t f fld1 fld2 =
2172+ let f1 = lmap fst fld1 -- :: Fold m (b, c) b
2173+ f2 = lmap snd fld2 -- :: Fold m (b, c) c
2174+ in lmapM f (t (,) f1 f2)
2175+
2176+-- | Like 'unzipWith' but with a monadic splitter function.
2177+--
2178+-- Definition:
2179+--
2180+-- >>> unzipWithM k f1 f2 = Fold.lmapM k (Fold.unzip f1 f2)
2181+--
2182+-- /Pre-release/
2183+{-# INLINE unzipWithM #-}
2184+unzipWithM :: Monad m
2185+ => (a -> m (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2186+unzipWithM = unzipWithMUsing teeWith
2187+
2188+-- | Similar to 'unzipWithM' but terminates when the first fold terminates.
2189+--
2190+{-# INLINE unzipWithFstM #-}
2191+unzipWithFstM :: Monad m =>
2192+ (a -> m (b, c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2193+unzipWithFstM = unzipWithMUsing teeWithFst
2194+
2195+-- | Similar to 'unzipWithM' but terminates when any fold terminates.
2196+--
2197+{-# INLINE unzipWithMinM #-}
2198+unzipWithMinM :: Monad m =>
2199+ (a -> m (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2200+unzipWithMinM = unzipWithMUsing teeWithMin
2201+
2202+-- | Split elements in the input stream into two parts using a pure splitter
2203+-- function, direct each part to a different fold and zip the results.
2204+--
2205+-- Definitions:
2206+--
2207+-- >>> unzipWith f = Fold.unzipWithM (return . f)
2208+-- >>> unzipWith f fld1 fld2 = Fold.lmap f (Fold.unzip fld1 fld2)
2209+--
2210+-- This fold terminates when both the input folds terminate.
2211+--
2212+-- /Pre-release/
2213+{-# INLINE unzipWith #-}
2214+unzipWith :: Monad m
2215+ => (a -> (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2216+unzipWith f = unzipWithM (return . f)
2217+
2218+-- | Send the elements of tuples in a stream of tuples through two different
2219+-- folds.
2220+--
2221+-- @
2222+--
2223+-- |-------Fold m a x--------|
2224+-- ---------stream of (a,b)--| |----m (x,y)
2225+-- |-------Fold m b y--------|
2226+--
2227+-- @
2228+--
2229+-- Definition:
2230+--
2231+-- >>> unzip = Fold.unzipWith id
2232+--
2233+-- This is the consumer side dual of the producer side 'zip' operation.
2234+--
2235+{-# INLINE unzip #-}
2236+unzip :: Monad m => Fold m a x -> Fold m b y -> Fold m (a,b) (x,y)
2237+unzip = unzipWith id
2238+
2239+------------------------------------------------------------------------------
2240+-- Combining streams and folds - Zipping
2241+------------------------------------------------------------------------------
2242+
2243+-- XXX These can be implemented using the fold scan, using the stream as a
2244+-- state.
2245+-- XXX Stream Skip state cannot be efficiently handled in folds but can be
2246+-- handled in parsers using the Continue facility. See zipWithM in the Parser
2247+-- module.
2248+--
2249+-- cmpBy, eqBy, isPrefixOf, isSubsequenceOf etc can be implemented using
2250+-- zipStream.
2251+
2252+-- | Zip a stream with the input of a fold using the supplied function.
2253+--
2254+-- /Unimplemented/
2255+--
2256+{-# INLINE zipStreamWithM #-}
2257+zipStreamWithM :: -- Monad m =>
2258+ (a -> b -> m c) -> Stream m a -> Fold m c x -> Fold m b x
2259+zipStreamWithM = undefined
2260+
2261+-- | Zip a stream with the input of a fold.
2262+--
2263+-- >>> zip = Fold.zipStreamWithM (curry return)
2264+--
2265+-- /Unimplemented/
2266+--
2267+{-# INLINE zipStream #-}
2268+zipStream :: Monad m => Stream m a -> Fold m (a, b) x -> Fold m b x
2269+zipStream = zipStreamWithM (curry return)
2270+
2271+-- | Pair each element of a fold input with its index, starting from index 0.
2272+--
2273+{-# INLINE indexingWith #-}
2274+indexingWith :: Monad m => Int -> (Int -> Int) -> Fold m a (Maybe (Int, a))
2275+indexingWith i f = fmap toMaybe $ foldl' step initial
2276+
2277+ where
2278+
2279+ initial = Nothing'
2280+
2281+ step Nothing' a = Just' (i, a)
2282+ step (Just' (n, _)) a = Just' (f n, a)
2283+
2284+-- |
2285+-- >>> indexing = Fold.indexingWith 0 (+ 1)
2286+--
2287+{-# INLINE indexing #-}
2288+indexing :: Monad m => Fold m a (Maybe (Int, a))
2289+indexing = indexingWith 0 (+ 1)
2290+
2291+-- |
2292+-- >>> indexingRev n = Fold.indexingWith n (subtract 1)
2293+--
2294+{-# INLINE indexingRev #-}
2295+indexingRev :: Monad m => Int -> Fold m a (Maybe (Int, a))
2296+indexingRev n = indexingWith n (subtract 1)
2297+
2298+-- | Pair each element of a fold input with its index, starting from index 0.
2299+--
2300+-- >>> indexed = Fold.scanMaybe Fold.indexing
2301+--
2302+{-# INLINE indexed #-}
2303+indexed :: Monad m => Fold m (Int, a) b -> Fold m a b
2304+indexed = scanMaybe indexing
2305+
2306+-- | Change the predicate function of a Fold from @a -> b@ to accept an
2307+-- additional state input @(s, a) -> b@. Convenient to filter with an
2308+-- addiitonal index or time input.
2309+--
2310+-- >>> filterWithIndex = Fold.with Fold.indexed Fold.filter
2311+--
2312+-- @
2313+-- filterWithAbsTime = with timestamped filter
2314+-- filterWithRelTime = with timeIndexed filter
2315+-- @
2316+--
2317+-- /Pre-release/
2318+{-# INLINE with #-}
2319+with ::
2320+ (Fold m (s, a) b -> Fold m a b)
2321+ -> (((s, a) -> c) -> Fold m (s, a) b -> Fold m (s, a) b)
2322+ -> (((s, a) -> c) -> Fold m a b -> Fold m a b)
2323+with f comb g = f . comb g . lmap snd
2324+
2325+-- XXX Implement as a filter
2326+-- sampleFromthen :: Monad m => Int -> Int -> Fold m a (Maybe a)
2327+
2328+-- | @sampleFromthen offset stride@ samples the element at @offset@ index and
2329+-- then every element at strides of @stride@.
2330+--
2331+{-# INLINE sampleFromthen #-}
2332+sampleFromthen :: Monad m => Int -> Int -> Fold m a b -> Fold m a b
2333+sampleFromthen offset size =
2334+ with indexed filter (\(i, _) -> (i + offset) `mod` size == 0)
2335+
2336+------------------------------------------------------------------------------
2337+-- Nesting
2338+------------------------------------------------------------------------------
2339+
2340+-- | @concatSequence f t@ applies folds from stream @t@ sequentially and
2341+-- collects the results using the fold @f@.
2342+--
2343+-- /Unimplemented/
2344+--
2345+{-# INLINE concatSequence #-}
2346+concatSequence ::
2347+ -- IsStream t =>
2348+ Fold m b c -> t (Fold m a b) -> Fold m a c
2349+concatSequence _f _p = undefined
2350+
2351+-- | Group the input stream into groups of elements between @low@ and @high@.
2352+-- Collection starts in chunks of @low@ and then keeps doubling until we reach
2353+-- @high@. Each chunk is folded using the provided fold function.
2354+--
2355+-- This could be useful, for example, when we are folding a stream of unknown
2356+-- size to a stream of arrays and we want to minimize the number of
2357+-- allocations.
2358+--
2359+-- NOTE: this would be an application of "many" using a terminating fold.
2360+--
2361+-- /Unimplemented/
2362+--
2363+{-# INLINE chunksBetween #-}
2364+chunksBetween :: -- Monad m =>
2365+ Int -> Int -> Fold m a b -> Fold m b c -> Fold m a c
2366+chunksBetween _low _high _f1 _f2 = undefined
2367+
2368+-- | A fold that buffers its input to a pure stream.
2369+--
2370+-- /Warning!/ working on large streams accumulated as buffers in memory could
2371+-- be very inefficient, consider using "Streamly.Data.Array" instead.
2372+--
2373+-- >>> toStream = fmap Stream.fromList Fold.toList
2374+--
2375+-- /Pre-release/
2376+{-# INLINE toStream #-}
2377+toStream :: (Monad m, Monad n) => Fold m a (Stream n a)
2378+toStream = fmap StreamD.fromList toList
2379+
2380+-- This is more efficient than 'toStream'. toStream is exactly the same as
2381+-- reversing the stream after toStreamRev.
2382+--
2383+-- | Buffers the input stream to a pure stream in the reverse order of the
2384+-- input.
2385+--
2386+-- >>> toStreamRev = fmap Stream.fromList Fold.toListRev
2387+--
2388+-- /Warning!/ working on large streams accumulated as buffers in memory could
2389+-- be very inefficient, consider using "Streamly.Data.Array" instead.
2390+--
2391+-- /Pre-release/
2392+
2393+-- xn : ... : x2 : x1 : []
2394+{-# INLINE toStreamRev #-}
2395+toStreamRev :: (Monad m, Monad n) => Fold m a (Stream n a)
2396+toStreamRev = fmap StreamD.fromList toListRev
2397+
2398+-- XXX This does not fuse. It contains a recursive step function. We will need
2399+-- a Skip input constructor in the fold type to make it fuse.
2400+--
2401+-- | Unfold and flatten the input stream of a fold.
2402+--
2403+-- @
2404+-- Stream.fold (unfoldMany u f) = Stream.fold f . Stream.unfoldMany u
2405+-- @
2406+--
2407+-- /Pre-release/
2408+{-# INLINE unfoldMany #-}
2409+unfoldMany :: Monad m => Unfold m a b -> Fold m b c -> Fold m a c
2410+unfoldMany (Unfold ustep inject) (Fold fstep initial extract) =
2411+ Fold consume initial extract
2412+
2413+ where
2414+
2415+ {-# INLINE produce #-}
2416+ produce fs us = do
2417+ ures <- ustep us
2418+ case ures of
2419+ StreamD.Yield b us1 -> do
2420+ fres <- fstep fs b
2421+ case fres of
2422+ Partial fs1 -> produce fs1 us1
2423+ -- XXX What to do with the remaining stream?
2424+ Done c -> return $ Done c
2425+ StreamD.Skip us1 -> produce fs us1
2426+ StreamD.Stop -> return $ Partial fs
2427+
2428+ {-# INLINE_LATE consume #-}
2429+ consume s a = inject a >>= produce s
2430+
2431+-- | Get the bottom most @n@ elements using the supplied comparison function.
2432+--
2433+{-# INLINE bottomBy #-}
2434+bottomBy :: (MonadIO m, Unbox a) =>
2435+ (a -> a -> Ordering)
2436+ -> Int
2437+ -> Fold m a (MutArray a)
2438+bottomBy cmp n = Fold step initial extract
2439+
2440+ where
2441+
2442+ initial = do
2443+ arr <- MA.newPinned n
2444+ if n <= 0
2445+ then return $ Done arr
2446+ else return $ Partial (arr, 0)
2447+
2448+ step (arr, i) x =
2449+ if i < n
2450+ then do
2451+ arr' <- MA.snoc arr x
2452+ MA.bubble cmp arr'
2453+ return $ Partial (arr', i + 1)
2454+ else do
2455+ x1 <- MA.getIndexUnsafe (i - 1) arr
2456+ case x `cmp` x1 of
2457+ LT -> do
2458+ MA.putIndexUnsafe (i - 1) arr x
2459+ MA.bubble cmp arr
2460+ return $ Partial (arr, i)
2461+ _ -> return $ Partial (arr, i)
2462+
2463+ extract = return . fst
2464+
2465+-- | Get the top @n@ elements using the supplied comparison function.
2466+--
2467+-- To get bottom n elements instead:
2468+--
2469+-- >>> bottomBy cmp = Fold.topBy (flip cmp)
2470+--
2471+-- Example:
2472+--
2473+-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2474+-- >>> Stream.fold (Fold.topBy compare 3) stream >>= MutArray.toList
2475+-- [17,11,9]
2476+--
2477+-- /Pre-release/
2478+--
2479+{-# INLINE topBy #-}
2480+topBy :: (MonadIO m, Unbox a) =>
2481+ (a -> a -> Ordering)
2482+ -> Int
2483+ -> Fold m a (MutArray a)
2484+topBy cmp = bottomBy (flip cmp)
2485+
2486+-- | Fold the input stream to top n elements.
2487+--
2488+-- Definition:
2489+--
2490+-- >>> top = Fold.topBy compare
2491+--
2492+-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2493+-- >>> Stream.fold (Fold.top 3) stream >>= MutArray.toList
2494+-- [17,11,9]
2495+--
2496+-- /Pre-release/
2497+{-# INLINE top #-}
2498+top :: (MonadIO m, Unbox a, Ord a) => Int -> Fold m a (MutArray a)
2499+top = bottomBy $ flip compare
2500+
2501+-- | Fold the input stream to bottom n elements.
2502+--
2503+-- Definition:
2504+--
2505+-- >>> bottom = Fold.bottomBy compare
2506+--
2507+-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2508+-- >>> Stream.fold (Fold.bottom 3) stream >>= MutArray.toList
2509+-- [1,2,3]
2510+--
2511+-- /Pre-release/
2512+{-# INLINE bottom #-}
2513+bottom :: (MonadIO m, Unbox a, Ord a) => Int -> Fold m a (MutArray a)
2514+bottom = bottomBy compare
2515+
2516+------------------------------------------------------------------------------
2517+-- Interspersed parsing
2518+------------------------------------------------------------------------------
2519+
2520+data IntersperseQState fs ps =
2521+ IntersperseQUnquoted !fs !ps
2522+ | IntersperseQQuoted !fs !ps
2523+ | IntersperseQQuotedEsc !fs !ps
2524+
2525+-- Useful for parsing CSV with quoting and escaping
2526+{-# INLINE intersperseWithQuotes #-}
2527+intersperseWithQuotes :: (Monad m, Eq a) =>
2528+ a -> a -> a -> Fold m a b -> Fold m b c -> Fold m a c
2529+intersperseWithQuotes
2530+ quote
2531+ esc
2532+ separator
2533+ (Fold stepL initialL extractL)
2534+ (Fold stepR initialR extractR) = Fold step initial extract
2535+
2536+ where
2537+
2538+ errMsg p status =
2539+ error $ "intersperseWithQuotes: " ++ p ++ " parsing fold cannot "
2540+ ++ status ++ " without input"
2541+
2542+ {-# INLINE initL #-}
2543+ initL mkState = do
2544+ resL <- initialL
2545+ case resL of
2546+ Partial sL ->
2547+ return $ Partial $ mkState sL
2548+ Done _ ->
2549+ errMsg "content" "succeed"
2550+
2551+ initial = do
2552+ res <- initialR
2553+ case res of
2554+ Partial sR -> initL (IntersperseQUnquoted sR)
2555+ Done b -> return $ Done b
2556+
2557+ {-# INLINE collect #-}
2558+ collect nextS sR b = do
2559+ res <- stepR sR b
2560+ case res of
2561+ Partial s ->
2562+ initL (nextS s)
2563+ Done c -> return (Done c)
2564+
2565+ {-# INLINE process #-}
2566+ process a sL sR nextState = do
2567+ r <- stepL sL a
2568+ case r of
2569+ Partial s -> return $ Partial (nextState sR s)
2570+ Done b -> collect nextState sR b
2571+
2572+ {-# INLINE processQuoted #-}
2573+ processQuoted a sL sR nextState = do
2574+ r <- stepL sL a
2575+ case r of
2576+ Partial s -> return $ Partial (nextState sR s)
2577+ Done _ -> error "Collecting fold finished inside quote"
2578+
2579+ step (IntersperseQUnquoted sR sL) a
2580+ | a == separator = do
2581+ b <- extractL sL
2582+ collect IntersperseQUnquoted sR b
2583+ | a == quote = processQuoted a sL sR IntersperseQQuoted
2584+ | otherwise = process a sL sR IntersperseQUnquoted
2585+
2586+ step (IntersperseQQuoted sR sL) a
2587+ | a == esc = processQuoted a sL sR IntersperseQQuotedEsc
2588+ | a == quote = process a sL sR IntersperseQUnquoted
2589+ | otherwise = processQuoted a sL sR IntersperseQQuoted
2590+
2591+ step (IntersperseQQuotedEsc sR sL) a =
2592+ processQuoted a sL sR IntersperseQQuoted
2593+
2594+ extract (IntersperseQUnquoted sR _) = extractR sR
2595+ extract (IntersperseQQuoted _ _) =
2596+ error "intersperseWithQuotes: finished inside quote"
2597+ extract (IntersperseQQuotedEsc _ _) =
2598+ error "intersperseWithQuotes: finished inside quote, at escape char"
core/src/Streamly/Internal/Data/Fold/Chunked.hs+18−20
@@ -6,7 +6,7 @@
66-- Stability : experimental
77-- Portability : GHC
88--
9−-- Use "Streamly.Data.Parser" instead.
9+-- Use "Streamly.Data.Parser.Chunked" instead.
1010--
1111-- Fold a stream of foreign arrays. @Fold m a b@ in this module works
1212-- on a stream of "Array a" and produces an output of type @b@.
@@ -19,19 +19,18 @@
1919-- folds in Data.Fold to correctly work on an array stream as if it is an
2020-- element stream. For example:
2121--
22−-- >> import qualified Streamly.Data.Fold as Fold
23−-- >> import qualified Streamly.Internal.Data.Array.Stream as ArrayStream
24−-- >> import qualified Streamly.Internal.Data.Fold.Chunked as ChunkFold
25−-- >> import qualified Streamly.Data.Stream as Stream
26−-- >> import qualified Streamly.Data.StreamK as StreamK
22+-- >>> import qualified Streamly.Data.Fold as Fold
23+-- >>> import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
24+-- >>> import qualified Streamly.Internal.Data.Fold.Chunked as ChunkFold
25+-- >>> import qualified Streamly.Data.Stream as Stream
26+-- >>> import qualified Streamly.Data.StreamK as StreamK
2727--
28−-- >> f = ChunkFold.fromFold (Fold.take 7 Fold.toList)
29−-- >> s = Stream.chunksOf 5 $ Stream.fromList "hello world"
30−-- >> ArrayStream.runArrayFold f (StreamK.fromStream s)
28+-- >>> f = ChunkFold.fromFold (Fold.take 7 Fold.toList)
29+-- >>> s = Stream.chunksOf 5 $ Stream.fromList "hello world"
30+-- >>> ArrayStream.runArrayFold f (StreamK.fromStream s)
3131-- Right "hello w"
3232--
3333module Streamly.Internal.Data.Fold.Chunked
34− {-# DEPRECATED "Please use Streamly.Data.Parser instead." #-}
3534 (
3635 ChunkFold (..)
3736
@@ -59,22 +58,22 @@ where
5958
6059#include "ArrayMacros.h"
6160
62−#if !MIN_VERSION_base(4,18,0)
6361import Control.Applicative (liftA2)
64−#endif
6562import Control.Exception (assert)
6663import Control.Monad.IO.Class (MonadIO(..))
6764import Data.Bifunctor (first)
6865import Data.Proxy (Proxy(..))
6966import Streamly.Internal.Data.Unbox (Unbox(..))
7067import GHC.Types (SPEC(..))
71−import Streamly.Internal.Data.Array (Array(..))
68+import Streamly.Internal.Data.Array.Mut.Type (touch)
69+import Streamly.Internal.Data.Array.Type (Array(..))
7270import Streamly.Internal.Data.Parser (Initial(..), Step(..))
7371import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
7472
75−import qualified Streamly.Internal.Data.Array.Type as Array
73+import qualified Streamly.Internal.Data.Array as Array
7674import qualified Streamly.Internal.Data.Fold as Fold
7775import qualified Streamly.Internal.Data.Parser as ParserD
76+import qualified Streamly.Internal.Data.Parser.ParserD.Type as ParserD
7877import qualified Streamly.Internal.Data.Parser as Parser
7978
8079import Prelude hiding (concatMap, take)
@@ -103,8 +102,8 @@ newtype ChunkFold m a b = ChunkFold (ParserD.Parser (Array a) m b)
103102{-# INLINE fromFold #-}
104103fromFold :: forall m a b. (MonadIO m, Unbox a) =>
105104 Fold.Fold m a b -> ChunkFold m a b
106−fromFold (Fold.Fold fstep finitial _ ffinal) =
107− ChunkFold (ParserD.Parser step initial extract)
105+fromFold (Fold.Fold fstep finitial fextract) =
106+ ChunkFold (ParserD.Parser step initial (fmap (Done 0) . fextract))
108107
109108 where
110109
@@ -124,7 +123,7 @@ fromFold (Fold.Fold fstep finitial _ ffinal) =
124123 assert (cur == end) (return ())
125124 return $ Partial 0 fs
126125 goArray !_ !cur !fs = do
127− x <- liftIO $ peekAt cur contents
126+ x <- liftIO $ peekByteIndex cur contents
128127 res <- fstep fs x
129128 let elemSize = SIZE_OF(a)
130129 next = INDEX_NEXT(cur,a)
@@ -134,8 +133,6 @@ fromFold (Fold.Fold fstep finitial _ ffinal) =
134133 Fold.Partial fs1 ->
135134 goArray SPEC next fs1
136135
137− extract = fmap (Done 0) . ffinal
138−
139136-- | Convert an element 'ParserD.Parser' into an array stream fold. If the
140137-- parser fails the fold would throw an exception.
141138--
@@ -163,7 +160,8 @@ fromParserD (ParserD.Parser step1 initial1 extract1) =
163160 else return $ st (arrRem + n) fs1
164161
165162 goArray !_ !cur !fs = do
166− x <- liftIO $ peekAt cur contents
163+ x <- liftIO $ peekByteIndex cur contents
164+ liftIO $ touch contents
167165 res <- step1 fs x
168166 let elemSize = SIZE_OF(a)
169167 next = INDEX_NEXT(cur,a)
core/src/Streamly/Internal/Data/Fold/Combinators.hsdeleted−2452
@@ -1,2452 +0,0 @@
1−{-# LANGUAGE CPP #-}
2−-- |
3−-- Module : Streamly.Internal.Data.Fold.Combinators
4−-- Copyright : (c) 2019 Composewell Technologies
5−-- (c) 2013 Gabriel Gonzalez
6−-- License : BSD3
7−-- Maintainer : streamly@composewell.com
8−-- Stability : experimental
9−-- Portability : GHC
10−--
11−-- See "Streamly.Data.Fold" for an overview and
12−-- "Streamly.Internal.Data.Fold.Type" for design notes.
13−
14−module Streamly.Internal.Data.Fold.Combinators
15− (
16− -- * Mappers
17− -- | Monadic functions useful with mapM/lmapM on folds or streams.
18− tracing
19− , trace
20−
21− -- * Folds
22−
23− -- ** Accumulators
24− -- *** Semigroups and Monoids
25− , sconcat
26− , mconcat
27− , foldMap
28− , foldMapM
29−
30− -- *** Reducers
31− , drainMapM
32− , the
33− , mean
34− , rollingHash
35− , defaultSalt
36− , rollingHashWithSalt
37− , rollingHashFirstN
38− -- , rollingHashLastN
39−
40− -- *** Saturating Reducers
41− -- | 'product' terminates if it becomes 0. Other folds can theoretically
42− -- saturate on bounded types, and therefore terminate, however, they will
43− -- run forever on unbounded types like Integer/Double.
44− , sum
45− , product
46− , maximumBy
47− , maximum
48− , minimumBy
49− , minimum
50−
51− -- *** Collectors
52− -- | Avoid using these folds in scalable or performance critical
53− -- applications, they buffer all the input in GC memory which can be
54− -- detrimental to performance if the input is large.
55− , toListRev
56− -- $toListRev
57− , toStream
58− , toStreamRev
59− , topBy
60− , top
61− , bottomBy
62− , bottom
63−
64− -- *** Scanners
65− -- | Stateful transformation of the elements. Useful in combination with
66− -- the 'scanMaybe' combinator. For scanners the result of the fold is
67− -- usually a transformation of the current element rather than an
68− -- aggregation of all elements till now.
69− , latest
70− -- , nthLast -- using Ring array
71− , indexingWith
72− , indexing
73− , indexingRev
74− , rollingMapM
75−
76− -- *** Filters
77− -- | Useful in combination with the 'scanMaybe' combinator.
78− , deleteBy
79− , uniqBy
80− , uniq
81− , repeated
82− , findIndices
83− , elemIndices
84−
85− -- *** Singleton folds
86− -- | Folds that terminate after consuming exactly one input element. All
87− -- these can be implemented in terms of the 'maybe' fold.
88− , one
89− , null -- XXX not very useful and could be problematic, remove it?
90− , satisfy
91− , maybe
92−
93− -- *** Multi folds
94− -- | Terminate after consuming one or more elements.
95− , drainN
96− -- , lastN
97− -- , (!!)
98− , indexGeneric
99− , index
100− , findM
101− , find
102− , lookup
103− , findIndex
104− , elemIndex
105− , elem
106− , notElem
107− , all
108− , any
109− , and
110− , or
111−
112− -- ** Trimmers
113− -- | Useful in combination with the 'scanMaybe' combinator.
114− , takingEndByM
115− , takingEndBy
116− , takingEndByM_
117− , takingEndBy_
118− , droppingWhileM
119− , droppingWhile
120− , prune
121−
122− -- * Running A Fold
123− , drive
124− -- , breakStream
125−
126− -- * Building Incrementally
127− , addStream
128−
129− -- * Combinators
130− -- ** Utilities
131− , with
132−
133− -- ** Mapping on Input
134− , transform
135−
136− -- ** Sliding Window
137− , slide2
138−
139− -- ** Scanning Input
140− , scan
141− , scanMany
142− , indexed
143−
144− -- ** Zipping Input
145− , zipStreamWithM
146− , zipStream
147−
148− -- ** Filtering Input
149− , mapMaybeM
150− , mapMaybe
151− , sampleFromthen
152−
153− {-
154− -- ** Insertion
155− -- | Insertion adds more elements to the stream.
156−
157− , insertBy
158− , intersperseM
159−
160− -- ** Reordering
161− , reverse
162− -}
163−
164− -- ** Trimming
165−
166− -- By elements
167− , takeEndBySeq
168− , takeEndBySeq_
169− {-
170− , drop
171− , dropWhile
172− , dropWhileM
173− -}
174−
175− -- ** Serial Append
176− -- , tail
177− -- , init
178− , splitAt -- spanN
179− -- , splitIn -- sessionN
180−
181− -- ** Parallel Distribution
182− , tee
183− , distribute
184− -- , distributeFst
185− -- , distributeMin
186−
187− -- ** Unzipping
188− , unzip
189− -- These two can be expressed using lmap/lmapM and unzip
190− , unzipWith
191− , unzipWithM
192− , unzipWithFstM
193− , unzipWithMinM
194−
195− -- ** Partitioning
196− , partitionByM
197− , partitionByFstM
198− , partitionByMinM
199− , partitionBy
200− , partition
201−
202− -- ** Splitting
203− , chunksBetween
204− , intersperseWithQuotes
205−
206− -- ** Nesting
207− , unfoldMany
208− , concatSequence
209−
210− -- * Deprecated
211− , drainBy
212− , last
213− , head
214− , sequence
215− , mapM
216− , variance
217− , stdDev
218− )
219−where
220−
221−#include "inline.hs"
222−#include "ArrayMacros.h"
223−
224−import Control.Monad (void)
225−import Control.Monad.IO.Class (MonadIO(..))
226−import Data.Bifunctor (first)
227−import Data.Bits (shiftL, shiftR, (.|.), (.&.))
228−import Data.Either (isLeft, isRight, fromLeft, fromRight)
229−import Data.Int (Int64)
230−import Data.Proxy (Proxy(..))
231−import Data.Word (Word32)
232−import Foreign.Storable (Storable, peek)
233−import Streamly.Internal.Data.MutArray.Type (MutArray(..))
234−import Streamly.Internal.Data.Maybe.Strict (Maybe'(..), toMaybe)
235−import Streamly.Internal.Data.Pipe.Type (Pipe (..), PipeState(..))
236−import Streamly.Internal.Data.Unbox (Unbox, sizeOf)
237−import Streamly.Internal.Data.Unfold.Type (Unfold(..))
238−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3'(..))
239−import Streamly.Internal.Data.Stream.Type (Stream)
240−
241−import qualified Prelude
242−import qualified Streamly.Internal.Data.MutArray.Type as MA
243−import qualified Streamly.Internal.Data.Array.Type as Array
244−import qualified Streamly.Internal.Data.Fold.Window as Fold
245−import qualified Streamly.Internal.Data.Pipe.Type as Pipe
246−import qualified Streamly.Internal.Data.Ring as Ring
247−import qualified Streamly.Internal.Data.Stream.Type as StreamD
248−
249−import Prelude hiding
250− ( Foldable(..), filter, drop, dropWhile, take, takeWhile, zipWith
251− , map, mapM_, sequence, all, any
252− , notElem, head, last, tail
253− , reverse, iterate, init, and, or, lookup, (!!)
254− , scanl, scanl1, replicate, concatMap, mconcat, unzip
255− , span, splitAt, break, mapM, zip, maybe)
256−import Streamly.Internal.Data.Fold.Type
257−
258−#include "DocTestDataFold.hs"
259−
260−------------------------------------------------------------------------------
261−-- Running
262−------------------------------------------------------------------------------
263−
264−-- | Drive a fold using the supplied 'Stream', reducing the resulting
265−-- expression strictly at each step.
266−--
267−-- Definition:
268−--
269−-- >>> drive = flip Stream.fold
270−--
271−-- Example:
272−--
273−-- >>> Fold.drive (Stream.enumerateFromTo 1 100) Fold.sum
274−-- 5050
275−--
276−{-# INLINE drive #-}
277−drive :: Monad m => Stream m a -> Fold m a b -> m b
278−drive = flip StreamD.fold
279−
280−{-
281−-- | Like 'drive' but also returns the remaining stream. The resulting stream
282−-- would be 'Stream.nil' if the stream finished before the fold.
283−--
284−-- Definition:
285−--
286−-- >>> breakStream = flip Stream.foldBreak
287−--
288−-- /CPS/
289−--
290−{-# INLINE breakStreamK #-}
291−breakStreamK :: Monad m => StreamK m a -> Fold m a b -> m (b, StreamK m a)
292−breakStreamK strm fl = fmap f $ K.foldBreak fl (Stream.toStreamK strm)
293−
294− where
295−
296− f (b, str) = (b, Stream.fromStreamK str)
297−-}
298−
299−-- | Append a stream to a fold to build the fold accumulator incrementally. We
300−-- can repeatedly call 'addStream' on the same fold to continue building the
301−-- fold and finally use 'drive' to finish the fold and extract the result. Also
302−-- see the 'Streamly.Data.Fold.addOne' operation which is a singleton version
303−-- of 'addStream'.
304−--
305−-- Definitions:
306−--
307−-- >>> addStream stream = Fold.drive stream . Fold.duplicate
308−--
309−-- Example, build a list incrementally:
310−--
311−-- >>> :{
312−-- pure (Fold.toList :: Fold IO Int [Int])
313−-- >>= Fold.addOne 1
314−-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
315−-- >>= Fold.drive Stream.nil
316−-- >>= print
317−-- :}
318−-- [1,2,3,4]
319−--
320−-- This can be used as an O(n) list append compared to the O(n^2) @++@ when
321−-- used for incrementally building a list.
322−--
323−-- Example, build a stream incrementally:
324−--
325−-- >>> :{
326−-- pure (Fold.toStream :: Fold IO Int (Stream Identity Int))
327−-- >>= Fold.addOne 1
328−-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
329−-- >>= Fold.drive Stream.nil
330−-- >>= print
331−-- :}
332−-- fromList [1,2,3,4]
333−--
334−-- This can be used as an O(n) stream append compared to the O(n^2) @<>@ when
335−-- used for incrementally building a stream.
336−--
337−-- Example, build an array incrementally:
338−--
339−-- >>> :{
340−-- pure (Array.write :: Fold IO Int (Array Int))
341−-- >>= Fold.addOne 1
342−-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
343−-- >>= Fold.drive Stream.nil
344−-- >>= print
345−-- :}
346−-- fromList [1,2,3,4]
347−--
348−-- Example, build an array stream incrementally:
349−--
350−-- >>> :{
351−-- let f :: Fold IO Int (Stream Identity (Array Int))
352−-- f = Fold.groupsOf 2 (Array.writeN 3) Fold.toStream
353−-- in pure f
354−-- >>= Fold.addOne 1
355−-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
356−-- >>= Fold.drive Stream.nil
357−-- >>= print
358−-- :}
359−-- fromList [fromList [1,2],fromList [3,4]]
360−--
361−addStream :: Monad m => Stream m a -> Fold m a b -> m (Fold m a b)
362−addStream stream = drive stream . duplicate
363−
364−------------------------------------------------------------------------------
365−-- Transformations on fold inputs
366−------------------------------------------------------------------------------
367−
368−-- | Flatten the monadic output of a fold to pure output.
369−--
370−{-# DEPRECATED sequence "Use \"rmapM id\" instead" #-}
371−{-# INLINE sequence #-}
372−sequence :: Monad m => Fold m a (m b) -> Fold m a b
373−sequence = rmapM id
374−
375−-- | Map a monadic function on the output of a fold.
376−--
377−{-# DEPRECATED mapM "Use rmapM instead" #-}
378−{-# INLINE mapM #-}
379−mapM :: Monad m => (b -> m c) -> Fold m a b -> Fold m a c
380−mapM = rmapM
381−
382−-- |
383−-- >>> mapMaybeM f = Fold.lmapM f . Fold.catMaybes
384−--
385−{-# INLINE mapMaybeM #-}
386−mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Fold m b r -> Fold m a r
387−mapMaybeM f = lmapM f . catMaybes
388−
389−-- | @mapMaybe f fold@ maps a 'Maybe' returning function @f@ on the input of
390−-- the fold, filters out 'Nothing' elements, and return the values extracted
391−-- from 'Just'.
392−--
393−-- >>> mapMaybe f = Fold.lmap f . Fold.catMaybes
394−-- >>> mapMaybe f = Fold.mapMaybeM (return . f)
395−--
396−-- >>> f x = if even x then Just x else Nothing
397−-- >>> fld = Fold.mapMaybe f Fold.toList
398−-- >>> Stream.fold fld (Stream.enumerateFromTo 1 10)
399−-- [2,4,6,8,10]
400−--
401−{-# INLINE mapMaybe #-}
402−mapMaybe :: Monad m => (a -> Maybe b) -> Fold m b r -> Fold m a r
403−mapMaybe f = lmap f . catMaybes
404−
405−------------------------------------------------------------------------------
406−-- Transformations on fold inputs
407−------------------------------------------------------------------------------
408−
409−-- | Apply a monadic function on the input and return the input.
410−--
411−-- >>> Stream.fold (Fold.lmapM (Fold.tracing print) Fold.drain) $ (Stream.enumerateFromTo (1 :: Int) 2)
412−-- 1
413−-- 2
414−--
415−-- /Pre-release/
416−--
417−{-# INLINE tracing #-}
418−tracing :: Monad m => (a -> m b) -> (a -> m a)
419−tracing f x = void (f x) >> return x
420−
421−-- | Apply a monadic function to each element flowing through and discard the
422−-- results.
423−--
424−-- >>> Stream.fold (Fold.trace print Fold.drain) $ (Stream.enumerateFromTo (1 :: Int) 2)
425−-- 1
426−-- 2
427−--
428−-- >>> trace f = Fold.lmapM (Fold.tracing f)
429−--
430−-- /Pre-release/
431−{-# INLINE trace #-}
432−trace :: Monad m => (a -> m b) -> Fold m a r -> Fold m a r
433−trace f = lmapM (tracing f)
434−
435−-- rename to lpipe?
436−--
437−-- | Apply a transformation on a 'Fold' using a 'Pipe'.
438−--
439−-- /Pre-release/
440−{-# INLINE transform #-}
441−transform :: Monad m => Pipe m a b -> Fold m b c -> Fold m a c
442−transform (Pipe pstep1 pstep2 pinitial) (Fold fstep finitial fextract ffinal) =
443− Fold step initial extract final
444−
445− where
446−
447− initial = first (Tuple' pinitial) <$> finitial
448−
449− step (Tuple' ps fs) x = do
450− r <- pstep1 ps x
451− go fs r
452−
453− where
454−
455− -- XXX use SPEC?
456− go acc (Pipe.Yield b (Consume ps')) = do
457− acc' <- fstep acc b
458− return
459− $ case acc' of
460− Partial s -> Partial $ Tuple' ps' s
461− Done b2 -> Done b2
462− go acc (Pipe.Yield b (Produce ps')) = do
463− acc' <- fstep acc b
464− r <- pstep2 ps'
465− case acc' of
466− Partial s -> go s r
467− Done b2 -> return $ Done b2
468− go acc (Pipe.Continue (Consume ps')) =
469− return $ Partial $ Tuple' ps' acc
470− go acc (Pipe.Continue (Produce ps')) = do
471− r <- pstep2 ps'
472− go acc r
473−
474− extract (Tuple' _ fs) = fextract fs
475−
476− final (Tuple' _ fs) = ffinal fs
477−
478−{-# INLINE scanWith #-}
479−scanWith :: Monad m => Bool -> Fold m a b -> Fold m b c -> Fold m a c
480−scanWith isMany
481− (Fold stepL initialL extractL finalL)
482− (Fold stepR initialR extractR finalR) =
483− Fold step initial extract final
484−
485− where
486−
487− {-# INLINE runStep #-}
488− runStep actionL sR = do
489− rL <- actionL
490− case rL of
491− Done bL -> do
492− rR <- stepR sR bL
493− case rR of
494− Partial sR1 ->
495− if isMany
496− then runStep initialL sR1
497− else Done <$> finalR sR1
498− Done bR -> return $ Done bR
499− Partial sL -> do
500− !b <- extractL sL
501− rR <- stepR sR b
502− case rR of
503− Partial sR1 -> return $ Partial (sL, sR1)
504− Done bR -> finalL sL >> return (Done bR)
505−
506− initial = do
507− r <- initialR
508− case r of
509− Partial sR -> runStep initialL sR
510− Done b -> return $ Done b
511−
512− step (sL, sR) x = runStep (stepL sL x) sR
513−
514− extract = extractR . snd
515−
516− final (sL, sR) = finalL sL *> finalR sR
517−
518−-- | Scan the input of a 'Fold' to change it in a stateful manner using another
519−-- 'Fold'. The scan stops as soon as the fold terminates.
520−--
521−-- /Pre-release/
522−{-# INLINE scan #-}
523−scan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
524−scan = scanWith False
525−
526−-- XXX This does not fuse beacuse of the recursive step. Need to investigate.
527−--
528−-- | Scan the input of a 'Fold' to change it in a stateful manner using another
529−-- 'Fold'. The scan restarts with a fresh state if the fold terminates.
530−--
531−-- /Pre-release/
532−{-# INLINE scanMany #-}
533−scanMany :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
534−scanMany = scanWith True
535−
536−------------------------------------------------------------------------------
537−-- Filters
538−------------------------------------------------------------------------------
539−
540−-- | Returns the latest element omitting the first occurrence that satisfies
541−-- the given equality predicate.
542−--
543−-- Example:
544−--
545−-- >>> input = Stream.fromList [1,3,3,5]
546−-- >>> Stream.fold Fold.toList $ Stream.scanMaybe (Fold.deleteBy (==) 3) input
547−-- [1,3,5]
548−--
549−{-# INLINE_NORMAL deleteBy #-}
550−deleteBy :: Monad m => (a -> a -> Bool) -> a -> Fold m a (Maybe a)
551−deleteBy eq x0 = fmap extract $ foldl' step (Tuple' False Nothing)
552−
553− where
554−
555− step (Tuple' False _) x =
556− if eq x x0
557− then Tuple' True Nothing
558− else Tuple' False (Just x)
559− step (Tuple' True _) x = Tuple' True (Just x)
560−
561− extract (Tuple' _ x) = x
562−
563−-- | Provide a sliding window of length 2 elements.
564−--
565−-- See "Streamly.Internal.Data.Fold.Window".
566−--
567−{-# INLINE slide2 #-}
568−slide2 :: Monad m => Fold m (a, Maybe a) b -> Fold m a b
569−slide2 (Fold step1 initial1 extract1 final1) = Fold step initial extract final
570−
571− where
572−
573− initial =
574− first (Tuple' Nothing) <$> initial1
575−
576− step (Tuple' prev s) cur =
577− first (Tuple' (Just cur)) <$> step1 s (cur, prev)
578−
579− extract (Tuple' _ s) = extract1 s
580−
581− final (Tuple' _ s) = final1 s
582−
583−-- | Return the latest unique element using the supplied comparison function.
584−-- Returns 'Nothing' if the current element is same as the last element
585−-- otherwise returns 'Just'.
586−--
587−-- Example, strip duplicate path separators:
588−--
589−-- >>> input = Stream.fromList "//a//b"
590−-- >>> f x y = x == '/' && y == '/'
591−-- >>> Stream.fold Fold.toList $ Stream.scanMaybe (Fold.uniqBy f) input
592−-- "/a/b"
593−--
594−-- Space: @O(1)@
595−--
596−-- /Pre-release/
597−--
598−{-# INLINE uniqBy #-}
599−uniqBy :: Monad m => (a -> a -> Bool) -> Fold m a (Maybe a)
600−uniqBy eq = rollingMap f
601−
602− where
603−
604− f pre curr =
605− case pre of
606− Nothing -> Just curr
607− Just x -> if x `eq` curr then Nothing else Just curr
608−
609−-- | See 'uniqBy'.
610−--
611−-- Definition:
612−--
613−-- >>> uniq = Fold.uniqBy (==)
614−--
615−{-# INLINE uniq #-}
616−uniq :: (Monad m, Eq a) => Fold m a (Maybe a)
617−uniq = uniqBy (==)
618−
619−-- | Strip all leading and trailing occurrences of an element passing a
620−-- predicate and make all other consecutive occurrences uniq.
621−--
622−-- >> prune p = Stream.dropWhileAround p $ Stream.uniqBy (x y -> p x && p y)
623−--
624−-- @
625−-- > Stream.prune isSpace (Stream.fromList " hello world! ")
626−-- "hello world!"
627−--
628−-- @
629−--
630−-- Space: @O(1)@
631−--
632−-- /Unimplemented/
633−{-# INLINE prune #-}
634−prune ::
635− -- (Monad m, Eq a) =>
636− (a -> Bool) -> Fold m a (Maybe a)
637−prune = error "Not implemented yet!"
638−
639−-- | Emit only repeated elements, once.
640−--
641−-- /Unimplemented/
642−repeated :: -- (Monad m, Eq a) =>
643− Fold m a (Maybe a)
644−repeated = error "Not implemented yet!"
645−
646−------------------------------------------------------------------------------
647−-- Left folds
648−------------------------------------------------------------------------------
649−
650−------------------------------------------------------------------------------
651−-- Run Effects
652−------------------------------------------------------------------------------
653−
654−-- |
655−-- Definitions:
656−--
657−-- >>> drainMapM f = Fold.lmapM f Fold.drain
658−-- >>> drainMapM f = Fold.foldMapM (void . f)
659−--
660−-- Drain all input after passing it through a monadic function. This is the
661−-- dual of mapM_ on stream producers.
662−--
663−{-# INLINE drainMapM #-}
664−drainMapM :: Monad m => (a -> m b) -> Fold m a ()
665−drainMapM f = lmapM f drain
666−
667−{-# DEPRECATED drainBy "Please use 'drainMapM' instead." #-}
668−{-# INLINE drainBy #-}
669−drainBy :: Monad m => (a -> m b) -> Fold m a ()
670−drainBy = drainMapM
671−
672−-- | Returns the latest element of the input stream, if any.
673−--
674−-- >>> latest = Fold.foldl1' (\_ x -> x)
675−-- >>> latest = fmap getLast $ Fold.foldMap (Last . Just)
676−--
677−{-# INLINE latest #-}
678−latest :: Monad m => Fold m a (Maybe a)
679−latest = foldl1' (\_ x -> x)
680−
681−{-# DEPRECATED last "Please use 'latest' instead." #-}
682−{-# INLINE last #-}
683−last :: Monad m => Fold m a (Maybe a)
684−last = latest
685−
686−-- | Terminates with 'Nothing' as soon as it finds an element different than
687−-- the previous one, returns 'the' element if the entire input consists of the
688−-- same element.
689−--
690−{-# INLINE the #-}
691−the :: (Monad m, Eq a) => Fold m a (Maybe a)
692−the = foldt' step initial id
693−
694− where
695−
696− initial = Partial Nothing
697−
698− step Nothing x = Partial (Just x)
699− step old@(Just x0) x =
700− if x0 == x
701− then Partial old
702− else Done Nothing
703−
704−------------------------------------------------------------------------------
705−-- To Summary
706−------------------------------------------------------------------------------
707−
708−-- | Determine the sum of all elements of a stream of numbers. Returns additive
709−-- identity (@0@) when the stream is empty. Note that this is not numerically
710−-- stable for floating point numbers.
711−--
712−-- >>> sum = Fold.cumulative Fold.windowSum
713−--
714−-- Same as following but numerically stable:
715−--
716−-- >>> sum = Fold.foldl' (+) 0
717−-- >>> sum = fmap Data.Monoid.getSum $ Fold.foldMap Data.Monoid.Sum
718−--
719−{-# INLINE sum #-}
720−sum :: (Monad m, Num a) => Fold m a a
721−sum = Fold.cumulative Fold.windowSum
722−
723−-- | Determine the product of all elements of a stream of numbers. Returns
724−-- multiplicative identity (@1@) when the stream is empty. The fold terminates
725−-- when it encounters (@0@) in its input.
726−--
727−-- Same as the following but terminates on multiplication by @0@:
728−--
729−-- >>> product = fmap Data.Monoid.getProduct $ Fold.foldMap Data.Monoid.Product
730−--
731−{-# INLINE product #-}
732−product :: (Monad m, Num a, Eq a) => Fold m a a
733−product = foldt' step (Partial 1) id
734−
735− where
736−
737− step x a =
738− if a == 0
739− then Done 0
740− else Partial $ x * a
741−
742−------------------------------------------------------------------------------
743−-- To Summary (Maybe)
744−------------------------------------------------------------------------------
745−
746−-- | Determine the maximum element in a stream using the supplied comparison
747−-- function.
748−--
749−{-# INLINE maximumBy #-}
750−maximumBy :: Monad m => (a -> a -> Ordering) -> Fold m a (Maybe a)
751−maximumBy cmp = foldl1' max'
752−
753− where
754−
755− max' x y =
756− case cmp x y of
757− GT -> x
758− _ -> y
759−
760−-- | Determine the maximum element in a stream.
761−--
762−-- Definitions:
763−--
764−-- >>> maximum = Fold.maximumBy compare
765−-- >>> maximum = Fold.foldl1' max
766−--
767−-- Same as the following but without a default maximum. The 'Max' Monoid uses
768−-- the 'minBound' as the default maximum:
769−--
770−-- >>> maximum = fmap Data.Semigroup.getMax $ Fold.foldMap Data.Semigroup.Max
771−--
772−{-# INLINE maximum #-}
773−maximum :: (Monad m, Ord a) => Fold m a (Maybe a)
774−maximum = foldl1' max
775−
776−-- | Computes the minimum element with respect to the given comparison function
777−--
778−{-# INLINE minimumBy #-}
779−minimumBy :: Monad m => (a -> a -> Ordering) -> Fold m a (Maybe a)
780−minimumBy cmp = foldl1' min'
781−
782− where
783−
784− min' x y =
785− case cmp x y of
786− GT -> y
787− _ -> x
788−
789−-- | Determine the minimum element in a stream using the supplied comparison
790−-- function.
791−--
792−-- Definitions:
793−--
794−-- >>> minimum = Fold.minimumBy compare
795−-- >>> minimum = Fold.foldl1' min
796−--
797−-- Same as the following but without a default minimum. The 'Min' Monoid uses the
798−-- 'maxBound' as the default maximum:
799−--
800−-- >>> maximum = fmap Data.Semigroup.getMin $ Fold.foldMap Data.Semigroup.Min
801−--
802−{-# INLINE minimum #-}
803−minimum :: (Monad m, Ord a) => Fold m a (Maybe a)
804−minimum = foldl1' min
805−
806−------------------------------------------------------------------------------
807−-- To Summary (Statistical)
808−------------------------------------------------------------------------------
809−
810−-- | Compute a numerically stable arithmetic mean of all elements in the input
811−-- stream.
812−--
813−{-# INLINE mean #-}
814−mean :: (Monad m, Fractional a) => Fold m a a
815−mean = fmap done $ foldl' step begin
816−
817− where
818−
819− begin = Tuple' 0 0
820−
821− step (Tuple' x n) y =
822− let n1 = n + 1
823− in Tuple' (x + (y - x) / n1) n1
824−
825− done (Tuple' x _) = x
826−
827−-- | Compute a numerically stable (population) variance over all elements in
828−-- the input stream.
829−--
830−{-# DEPRECATED variance "Use the streamly-statistics package instead" #-}
831−{-# INLINE variance #-}
832−variance :: (Monad m, Fractional a) => Fold m a a
833−variance = fmap done $ foldl' step begin
834−
835− where
836−
837− begin = Tuple3' 0 0 0
838−
839− step (Tuple3' n mean_ m2) x = Tuple3' n' mean' m2'
840−
841− where
842−
843− n' = n + 1
844− mean' = (n * mean_ + x) / (n + 1)
845− delta = x - mean_
846− m2' = m2 + delta * delta * n / (n + 1)
847−
848− done (Tuple3' n _ m2) = m2 / n
849−
850−-- | Compute a numerically stable (population) standard deviation over all
851−-- elements in the input stream.
852−--
853−{-# DEPRECATED stdDev "Use the streamly-statistics package instead" #-}
854−{-# INLINE stdDev #-}
855−stdDev :: (Monad m, Floating a) => Fold m a a
856−stdDev = sqrt <$> variance
857−
858−-- | Compute an 'Int' sized polynomial rolling hash
859−--
860−-- > H = salt * k ^ n + c1 * k ^ (n - 1) + c2 * k ^ (n - 2) + ... + cn * k ^ 0
861−--
862−-- Where @c1@, @c2@, @cn@ are the elements in the input stream and @k@ is a
863−-- constant.
864−--
865−-- This hash is often used in Rabin-Karp string search algorithm.
866−--
867−-- See https://en.wikipedia.org/wiki/Rolling_hash
868−--
869−{-# INLINE rollingHashWithSalt #-}
870−rollingHashWithSalt :: (Monad m, Enum a) => Int64 -> Fold m a Int64
871−rollingHashWithSalt = foldl' step
872−
873− where
874−
875− k = 2891336453 :: Int64
876−
877− step cksum a = cksum * k + fromIntegral (fromEnum a)
878−
879−-- | A default salt used in the implementation of 'rollingHash'.
880−{-# INLINE defaultSalt #-}
881−defaultSalt :: Int64
882−defaultSalt = -2578643520546668380
883−
884−-- | Compute an 'Int' sized polynomial rolling hash of a stream.
885−--
886−-- >>> rollingHash = Fold.rollingHashWithSalt Fold.defaultSalt
887−--
888−{-# INLINE rollingHash #-}
889−rollingHash :: (Monad m, Enum a) => Fold m a Int64
890−rollingHash = rollingHashWithSalt defaultSalt
891−
892−-- | Compute an 'Int' sized polynomial rolling hash of the first n elements of
893−-- a stream.
894−--
895−-- >>> rollingHashFirstN n = Fold.take n Fold.rollingHash
896−--
897−-- /Pre-release/
898−{-# INLINE rollingHashFirstN #-}
899−rollingHashFirstN :: (Monad m, Enum a) => Int -> Fold m a Int64
900−rollingHashFirstN n = take n rollingHash
901−
902−-- XXX Compare this with the implementation in Fold.Window, preferrably use the
903−-- latter if performance is good.
904−
905−-- | Apply a function on every two successive elements of a stream. The first
906−-- argument of the map function is the previous element and the second argument
907−-- is the current element. When processing the very first element in the
908−-- stream, the previous element is 'Nothing'.
909−--
910−-- /Pre-release/
911−--
912−{-# INLINE rollingMapM #-}
913−rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Fold m a b
914−rollingMapM f = Fold step initial extract extract
915−
916− where
917−
918− -- XXX We need just a postscan. We do not need an initial result here.
919− -- Or we can supply a default initial result as an argument to rollingMapM.
920− initial = return $ Partial (Nothing, error "Empty stream")
921−
922− step (prev, _) cur = do
923− x <- f prev cur
924− return $ Partial (Just cur, x)
925−
926− extract = return . snd
927−
928−-- |
929−-- >>> rollingMap f = Fold.rollingMapM (\x y -> return $ f x y)
930−--
931−{-# INLINE rollingMap #-}
932−rollingMap :: Monad m => (Maybe a -> a -> b) -> Fold m a b
933−rollingMap f = rollingMapM (\x y -> return $ f x y)
934−
935−------------------------------------------------------------------------------
936−-- Monoidal left folds
937−------------------------------------------------------------------------------
938−
939−-- | Semigroup concat. Append the elements of an input stream to a provided
940−-- starting value.
941−--
942−-- Definition:
943−--
944−-- >>> sconcat = Fold.foldl' (<>)
945−--
946−-- >>> semigroups = fmap Data.Monoid.Sum $ Stream.enumerateFromTo 1 10
947−-- >>> Stream.fold (Fold.sconcat 10) semigroups
948−-- Sum {getSum = 65}
949−--
950−{-# INLINE sconcat #-}
951−sconcat :: (Monad m, Semigroup a) => a -> Fold m a a
952−sconcat = foldl' (<>)
953−
954−-- | Monoid concat. Fold an input stream consisting of monoidal elements using
955−-- 'mappend' and 'mempty'.
956−--
957−-- Definition:
958−--
959−-- >>> mconcat = Fold.sconcat mempty
960−--
961−-- >>> monoids = fmap Data.Monoid.Sum $ Stream.enumerateFromTo 1 10
962−-- >>> Stream.fold Fold.mconcat monoids
963−-- Sum {getSum = 55}
964−--
965−{-# INLINE mconcat #-}
966−mconcat ::
967− ( Monad m
968− , Monoid a) => Fold m a a
969−mconcat = sconcat mempty
970−
971−-- |
972−-- Definition:
973−--
974−-- >>> foldMap f = Fold.lmap f Fold.mconcat
975−--
976−-- Make a fold from a pure function that folds the output of the function
977−-- using 'mappend' and 'mempty'.
978−--
979−-- >>> sum = Fold.foldMap Data.Monoid.Sum
980−-- >>> Stream.fold sum $ Stream.enumerateFromTo 1 10
981−-- Sum {getSum = 55}
982−--
983−{-# INLINE foldMap #-}
984−foldMap :: (Monad m, Monoid b) => (a -> b) -> Fold m a b
985−foldMap f = lmap f mconcat
986−
987−-- |
988−-- Definition:
989−--
990−-- >>> foldMapM f = Fold.lmapM f Fold.mconcat
991−--
992−-- Make a fold from a monadic function that folds the output of the function
993−-- using 'mappend' and 'mempty'.
994−--
995−-- >>> sum = Fold.foldMapM (return . Data.Monoid.Sum)
996−-- >>> Stream.fold sum $ Stream.enumerateFromTo 1 10
997−-- Sum {getSum = 55}
998−--
999−{-# INLINE foldMapM #-}
1000−foldMapM :: (Monad m, Monoid b) => (a -> m b) -> Fold m a b
1001−foldMapM act = foldlM' step (pure mempty)
1002−
1003− where
1004−
1005− step m a = do
1006− m' <- act a
1007− return $! mappend m m'
1008−
1009−------------------------------------------------------------------------------
1010−-- To Containers
1011−------------------------------------------------------------------------------
1012−
1013−-- $toListRev
1014−-- This is more efficient than 'Streamly.Internal.Data.Fold.toList'. toList is
1015−-- exactly the same as reversing the list after 'toListRev'.
1016−
1017−-- | Buffers the input stream to a list in the reverse order of the input.
1018−--
1019−-- Definition:
1020−--
1021−-- >>> toListRev = Fold.foldl' (flip (:)) []
1022−--
1023−-- /Warning!/ working on large lists accumulated as buffers in memory could be
1024−-- very inefficient, consider using "Streamly.Array" instead.
1025−--
1026−
1027−-- xn : ... : x2 : x1 : []
1028−{-# INLINE toListRev #-}
1029−toListRev :: Monad m => Fold m a [a]
1030−toListRev = foldl' (flip (:)) []
1031−
1032−------------------------------------------------------------------------------
1033−-- Partial Folds
1034−------------------------------------------------------------------------------
1035−
1036−-- | A fold that drains the first n elements of its input, running the effects
1037−-- and discarding the results.
1038−--
1039−-- Definition:
1040−--
1041−-- >>> drainN n = Fold.take n Fold.drain
1042−--
1043−-- /Pre-release/
1044−{-# INLINE drainN #-}
1045−drainN :: Monad m => Int -> Fold m a ()
1046−drainN n = take n drain
1047−
1048−------------------------------------------------------------------------------
1049−-- To Elements
1050−------------------------------------------------------------------------------
1051−
1052−-- | Like 'index', except with a more general 'Integral' argument
1053−--
1054−-- /Pre-release/
1055−{-# INLINE indexGeneric #-}
1056−indexGeneric :: (Integral i, Monad m) => i -> Fold m a (Maybe a)
1057−indexGeneric i = foldt' step (Partial 0) (const Nothing)
1058−
1059− where
1060−
1061− step j a =
1062− if i == j
1063− then Done $ Just a
1064− else Partial (j + 1)
1065−
1066−-- | Return the element at the given index.
1067−--
1068−-- Definition:
1069−--
1070−-- >>> index = Fold.indexGeneric
1071−--
1072−{-# INLINE index #-}
1073−index :: Monad m => Int -> Fold m a (Maybe a)
1074−index = indexGeneric
1075−
1076−-- | Consume a single input and transform it using the supplied 'Maybe'
1077−-- returning function.
1078−--
1079−-- /Pre-release/
1080−--
1081−{-# INLINE maybe #-}
1082−maybe :: Monad m => (a -> Maybe b) -> Fold m a (Maybe b)
1083−maybe f = foldt' (const (Done . f)) (Partial Nothing) id
1084−
1085−-- | Consume a single element and return it if it passes the predicate else
1086−-- return 'Nothing'.
1087−--
1088−-- Definition:
1089−--
1090−-- >>> satisfy f = Fold.maybe (\a -> if f a then Just a else Nothing)
1091−--
1092−-- /Pre-release/
1093−{-# INLINE satisfy #-}
1094−satisfy :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1095−satisfy f = maybe (\a -> if f a then Just a else Nothing)
1096−{-
1097−satisfy f = Fold step (return $ Partial ()) (const (return Nothing))
1098−
1099− where
1100−
1101− step () a = return $ Done $ if f a then Just a else Nothing
1102−-}
1103−
1104−-- Naming notes:
1105−--
1106−-- "head" and "next" are two alternative names for the same API. head sounds
1107−-- apt in the context of lists but next sounds more apt in the context of
1108−-- streams where we think in terms of generating and consuming the next element
1109−-- rather than taking the head of some static/persistent structure.
1110−--
1111−-- We also want to keep the nomenclature consistent across folds and parsers,
1112−-- "head" becomes even more unintuitive for parsers because there are two
1113−-- possible variants viz. peek and next.
1114−--
1115−-- Also, the "head" fold creates confusion in situations like
1116−-- https://github.com/composewell/streamly/issues/1404 where intuitive
1117−-- expectation from head is to consume the entire stream and just give us the
1118−-- head. There we want to convey the notion that we consume one element from
1119−-- the stream and stop. The name "one" already being used in parsers for this
1120−-- purpose sounds more apt from this perspective.
1121−--
1122−-- The source of confusion is perhaps due to the fact that some folds consume
1123−-- the entire stream and others terminate early. It may have been clearer if we
1124−-- had separate abstractions for the two use cases.
1125−
1126−-- XXX We can possibly use "head" for the purposes of reducing the entire
1127−-- stream to the head element i.e. take the head and drain the rest.
1128−
1129−-- | Take one element from the stream and stop.
1130−--
1131−-- Definition:
1132−--
1133−-- >>> one = Fold.maybe Just
1134−--
1135−-- This is similar to the stream 'Stream.uncons' operation.
1136−--
1137−{-# INLINE one #-}
1138−one :: Monad m => Fold m a (Maybe a)
1139−one = maybe Just
1140−
1141−-- | Extract the first element of the stream, if any.
1142−--
1143−-- >>> head = Fold.one
1144−--
1145−{-# DEPRECATED head "Please use \"one\" instead" #-}
1146−{-# INLINE head #-}
1147−head :: Monad m => Fold m a (Maybe a)
1148−head = one
1149−
1150−-- | Returns the first element that satisfies the given predicate.
1151−--
1152−-- /Pre-release/
1153−{-# INLINE findM #-}
1154−findM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1155−findM predicate =
1156− Fold step (return $ Partial ()) extract extract
1157−
1158− where
1159−
1160− step () a =
1161− let f r =
1162− if r
1163− then Done (Just a)
1164− else Partial ()
1165− in f <$> predicate a
1166−
1167− extract = const $ return Nothing
1168−
1169−-- | Returns the first element that satisfies the given predicate.
1170−--
1171−{-# INLINE find #-}
1172−find :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1173−find p = findM (return . p)
1174−
1175−-- | In a stream of (key-value) pairs @(a, b)@, return the value @b@ of the
1176−-- first pair where the key equals the given value @a@.
1177−--
1178−-- Definition:
1179−--
1180−-- >>> lookup x = fmap snd <$> Fold.find ((== x) . fst)
1181−--
1182−{-# INLINE lookup #-}
1183−lookup :: (Eq a, Monad m) => a -> Fold m (a,b) (Maybe b)
1184−lookup a0 = foldt' step (Partial ()) (const Nothing)
1185−
1186− where
1187−
1188− step () (a, b) =
1189− if a == a0
1190− then Done $ Just b
1191− else Partial ()
1192−
1193−-- | Returns the first index that satisfies the given predicate.
1194−--
1195−{-# INLINE findIndex #-}
1196−findIndex :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
1197−findIndex predicate = foldt' step (Partial 0) (const Nothing)
1198−
1199− where
1200−
1201− step i a =
1202− if predicate a
1203− then Done $ Just i
1204− else Partial (i + 1)
1205−
1206−-- | Returns the index of the latest element if the element satisfies the given
1207−-- predicate.
1208−--
1209−{-# INLINE findIndices #-}
1210−findIndices :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
1211−findIndices predicate =
1212− -- XXX implement by combining indexing and filtering scans
1213− fmap (either (const Nothing) Just) $ foldl' step (Left (-1))
1214−
1215− where
1216−
1217− step i a =
1218− if predicate a
1219− then Right (either id id i + 1)
1220− else Left (either id id i + 1)
1221−
1222−-- | Returns the index of the latest element if the element matches the given
1223−-- value.
1224−--
1225−-- Definition:
1226−--
1227−-- >>> elemIndices a = Fold.findIndices (== a)
1228−--
1229−{-# INLINE elemIndices #-}
1230−elemIndices :: (Monad m, Eq a) => a -> Fold m a (Maybe Int)
1231−elemIndices a = findIndices (== a)
1232−
1233−-- | Returns the first index where a given value is found in the stream.
1234−--
1235−-- Definition:
1236−--
1237−-- >>> elemIndex a = Fold.findIndex (== a)
1238−--
1239−{-# INLINE elemIndex #-}
1240−elemIndex :: (Eq a, Monad m) => a -> Fold m a (Maybe Int)
1241−elemIndex a = findIndex (== a)
1242−
1243−------------------------------------------------------------------------------
1244−-- To Boolean
1245−------------------------------------------------------------------------------
1246−
1247−-- Similar to 'eof' parser, but the fold consumes and discards an input element
1248−-- when not at eof. XXX Remove or Rename to "eof"?
1249−
1250−-- | Consume one element, return 'True' if successful else return 'False'. In
1251−-- other words, test if the input is empty or not.
1252−--
1253−-- WARNING! It consumes one element if the stream is not empty. If that is not
1254−-- what you want please use the eof parser instead.
1255−--
1256−-- Definition:
1257−--
1258−-- >>> null = fmap isJust Fold.one
1259−--
1260−{-# INLINE null #-}
1261−null :: Monad m => Fold m a Bool
1262−null = foldt' (\() _ -> Done False) (Partial ()) (const True)
1263−
1264−-- | Returns 'True' if any element of the input satisfies the predicate.
1265−--
1266−-- Definition:
1267−--
1268−-- >>> any p = Fold.lmap p Fold.or
1269−--
1270−-- Example:
1271−--
1272−-- >>> Stream.fold (Fold.any (== 0)) $ Stream.fromList [1,0,1]
1273−-- True
1274−--
1275−{-# INLINE any #-}
1276−any :: Monad m => (a -> Bool) -> Fold m a Bool
1277−any predicate = foldt' step initial id
1278−
1279− where
1280−
1281− initial = Partial False
1282−
1283− step _ a =
1284− if predicate a
1285− then Done True
1286− else Partial False
1287−
1288−-- | Return 'True' if the given element is present in the stream.
1289−--
1290−-- Definition:
1291−--
1292−-- >>> elem a = Fold.any (== a)
1293−--
1294−{-# INLINE elem #-}
1295−elem :: (Eq a, Monad m) => a -> Fold m a Bool
1296−elem a = any (== a)
1297−
1298−-- | Returns 'True' if all elements of the input satisfy the predicate.
1299−--
1300−-- Definition:
1301−--
1302−-- >>> all p = Fold.lmap p Fold.and
1303−--
1304−-- Example:
1305−--
1306−-- >>> Stream.fold (Fold.all (== 0)) $ Stream.fromList [1,0,1]
1307−-- False
1308−--
1309−{-# INLINE all #-}
1310−all :: Monad m => (a -> Bool) -> Fold m a Bool
1311−all predicate = foldt' step initial id
1312−
1313− where
1314−
1315− initial = Partial True
1316−
1317− step _ a =
1318− if predicate a
1319− then Partial True
1320− else Done False
1321−
1322−-- | Returns 'True' if the given element is not present in the stream.
1323−--
1324−-- Definition:
1325−--
1326−-- >>> notElem a = Fold.all (/= a)
1327−--
1328−{-# INLINE notElem #-}
1329−notElem :: (Eq a, Monad m) => a -> Fold m a Bool
1330−notElem a = all (/= a)
1331−
1332−-- | Returns 'True' if all elements are 'True', 'False' otherwise
1333−--
1334−-- Definition:
1335−--
1336−-- >>> and = Fold.all (== True)
1337−--
1338−{-# INLINE and #-}
1339−and :: Monad m => Fold m Bool Bool
1340−and = all (== True)
1341−
1342−-- | Returns 'True' if any element is 'True', 'False' otherwise
1343−--
1344−-- Definition:
1345−--
1346−-- >>> or = Fold.any (== True)
1347−--
1348−{-# INLINE or #-}
1349−or :: Monad m => Fold m Bool Bool
1350−or = any (== True)
1351−
1352−------------------------------------------------------------------------------
1353−-- Grouping/Splitting
1354−------------------------------------------------------------------------------
1355−
1356−------------------------------------------------------------------------------
1357−-- Grouping without looking at elements
1358−------------------------------------------------------------------------------
1359−
1360−------------------------------------------------------------------------------
1361−-- Binary APIs
1362−------------------------------------------------------------------------------
1363−
1364−-- | @splitAt n f1 f2@ composes folds @f1@ and @f2@ such that first @n@
1365−-- elements of its input are consumed by fold @f1@ and the rest of the stream
1366−-- is consumed by fold @f2@.
1367−--
1368−-- >>> let splitAt_ n xs = Stream.fold (Fold.splitAt n Fold.toList Fold.toList) $ Stream.fromList xs
1369−--
1370−-- >>> splitAt_ 6 "Hello World!"
1371−-- ("Hello ","World!")
1372−--
1373−-- >>> splitAt_ (-1) [1,2,3]
1374−-- ([],[1,2,3])
1375−--
1376−-- >>> splitAt_ 0 [1,2,3]
1377−-- ([],[1,2,3])
1378−--
1379−-- >>> splitAt_ 1 [1,2,3]
1380−-- ([1],[2,3])
1381−--
1382−-- >>> splitAt_ 3 [1,2,3]
1383−-- ([1,2,3],[])
1384−--
1385−-- >>> splitAt_ 4 [1,2,3]
1386−-- ([1,2,3],[])
1387−--
1388−-- > splitAt n f1 f2 = Fold.splitWith (,) (Fold.take n f1) f2
1389−--
1390−-- /Internal/
1391−
1392−{-# INLINE splitAt #-}
1393−splitAt
1394− :: Monad m
1395− => Int
1396− -> Fold m a b
1397− -> Fold m a c
1398− -> Fold m a (b, c)
1399−splitAt n fld = splitWith (,) (take n fld)
1400−
1401−------------------------------------------------------------------------------
1402−-- Element Aware APIs
1403−------------------------------------------------------------------------------
1404−--
1405−------------------------------------------------------------------------------
1406−-- Binary APIs
1407−------------------------------------------------------------------------------
1408−
1409−{-# INLINE takingEndByM #-}
1410−takingEndByM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1411−takingEndByM p = Fold step initial extract extract
1412−
1413− where
1414−
1415− initial = return $ Partial Nothing'
1416−
1417− step _ a = do
1418− r <- p a
1419− return
1420− $ if r
1421− then Done $ Just a
1422− else Partial $ Just' a
1423−
1424− extract = return . toMaybe
1425−
1426−-- |
1427−--
1428−-- >>> takingEndBy p = Fold.takingEndByM (return . p)
1429−--
1430−{-# INLINE takingEndBy #-}
1431−takingEndBy :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1432−takingEndBy p = takingEndByM (return . p)
1433−
1434−{-# INLINE takingEndByM_ #-}
1435−takingEndByM_ :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1436−takingEndByM_ p = Fold step initial extract extract
1437−
1438− where
1439−
1440− initial = return $ Partial Nothing'
1441−
1442− step _ a = do
1443− r <- p a
1444− return
1445− $ if r
1446− then Done Nothing
1447− else Partial $ Just' a
1448−
1449− extract = return . toMaybe
1450−
1451−-- |
1452−--
1453−-- >>> takingEndBy_ p = Fold.takingEndByM_ (return . p)
1454−--
1455−{-# INLINE takingEndBy_ #-}
1456−takingEndBy_ :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1457−takingEndBy_ p = takingEndByM_ (return . p)
1458−
1459−{-# INLINE droppingWhileM #-}
1460−droppingWhileM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1461−droppingWhileM p = Fold step initial extract extract
1462−
1463− where
1464−
1465− initial = return $ Partial Nothing'
1466−
1467− step Nothing' a = do
1468− r <- p a
1469− return
1470− $ Partial
1471− $ if r
1472− then Nothing'
1473− else Just' a
1474− step _ a = return $ Partial $ Just' a
1475−
1476− extract = return . toMaybe
1477−
1478−-- |
1479−-- >>> droppingWhile p = Fold.droppingWhileM (return . p)
1480−--
1481−{-# INLINE droppingWhile #-}
1482−droppingWhile :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1483−droppingWhile p = droppingWhileM (return . p)
1484−
1485−------------------------------------------------------------------------------
1486−-- Binary splitting on a separator
1487−------------------------------------------------------------------------------
1488−
1489−data SplitOnSeqState acc a rb rh w ck =
1490− SplitOnSeqEmpty !acc
1491− | SplitOnSeqSingle !acc !a
1492− | SplitOnSeqWord !acc !Int !w
1493− | SplitOnSeqWordLoop !acc !w
1494− | SplitOnSeqKR !acc !Int !rb !rh
1495− | SplitOnSeqKRLoop !acc !ck !rb !rh
1496−
1497−-- XXX Need to add tests for takeEndBySeq, we have tests for takeEndBySeq_ .
1498−
1499−-- | Continue taking the input until the input sequence matches the supplied
1500−-- sequence, taking the supplied sequence as well. If the pattern is empty this
1501−-- acts as an identity fold.
1502−--
1503−-- >>> s = Stream.fromList "hello there. How are you?"
1504−-- >>> f = Fold.takeEndBySeq (Array.fromList "re") Fold.toList
1505−-- >>> Stream.fold f s
1506−-- "hello there"
1507−--
1508−-- >>> Stream.fold Fold.toList $ Stream.foldMany f s
1509−-- ["hello there",". How are"," you?"]
1510−--
1511−-- /Pre-release/
1512−{-# INLINE takeEndBySeq #-}
1513−takeEndBySeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
1514− Array.Array a
1515− -> Fold m a b
1516− -> Fold m a b
1517−takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
1518− Fold step initial extract final
1519−
1520− where
1521−
1522− patLen = Array.length patArr
1523−
1524− initial = do
1525− res <- finitial
1526− case res of
1527− Partial acc
1528− | patLen == 0 ->
1529− -- XXX Should we match nothing or everything on empty
1530− -- pattern?
1531− -- Done <$> ffinal acc
1532− return $ Partial $ SplitOnSeqEmpty acc
1533− | patLen == 1 -> do
1534− pat <- liftIO $ Array.unsafeIndexIO 0 patArr
1535− return $ Partial $ SplitOnSeqSingle acc pat
1536− | SIZE_OF(a) * patLen <= sizeOf (Proxy :: Proxy Word) ->
1537− return $ Partial $ SplitOnSeqWord acc 0 0
1538− | otherwise -> do
1539− (rb, rhead) <- liftIO $ Ring.new patLen
1540− return $ Partial $ SplitOnSeqKR acc 0 rb rhead
1541− Done b -> return $ Done b
1542−
1543− -- Word pattern related
1544− maxIndex = patLen - 1
1545−
1546− elemBits = SIZE_OF(a) * 8
1547−
1548− wordMask :: Word
1549− wordMask = (1 `shiftL` (elemBits * patLen)) - 1
1550−
1551− wordPat :: Word
1552− wordPat = wordMask .&. Array.foldl' addToWord 0 patArr
1553−
1554− addToWord wd a = (wd `shiftL` elemBits) .|. fromIntegral (fromEnum a)
1555−
1556− -- For Rabin-Karp search
1557− k = 2891336453 :: Word32
1558− coeff = k ^ patLen
1559−
1560− addCksum cksum a = cksum * k + fromIntegral (fromEnum a)
1561−
1562− deltaCksum cksum old new =
1563− addCksum cksum new - coeff * fromIntegral (fromEnum old)
1564−
1565− -- XXX shall we use a random starting hash or 1 instead of 0?
1566− -- XXX Need to keep this cached across fold calls in foldmany
1567− -- XXX We may need refold to inject the cached state instead of
1568− -- initializing the state every time.
1569− -- XXX Allocation of ring buffer should also be done once
1570− patHash = Array.foldl' addCksum 0 patArr
1571−
1572− step (SplitOnSeqEmpty s) x = do
1573− res <- fstep s x
1574− case res of
1575− Partial s1 -> return $ Partial $ SplitOnSeqEmpty s1
1576− Done b -> return $ Done b
1577− step (SplitOnSeqSingle s pat) x = do
1578− res <- fstep s x
1579− case res of
1580− Partial s1
1581− | pat /= x -> return $ Partial $ SplitOnSeqSingle s1 pat
1582− | otherwise -> Done <$> ffinal s1
1583− Done b -> return $ Done b
1584− step (SplitOnSeqWord s idx wrd) x = do
1585− res <- fstep s x
1586− let wrd1 = addToWord wrd x
1587− case res of
1588− Partial s1
1589− | idx == maxIndex -> do
1590− if wrd1 .&. wordMask == wordPat
1591− then Done <$> ffinal s1
1592− else return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1593− | otherwise ->
1594− return $ Partial $ SplitOnSeqWord s1 (idx + 1) wrd1
1595− Done b -> return $ Done b
1596− step (SplitOnSeqWordLoop s wrd) x = do
1597− res <- fstep s x
1598− let wrd1 = addToWord wrd x
1599− case res of
1600− Partial s1
1601− | wrd1 .&. wordMask == wordPat ->
1602− Done <$> ffinal s1
1603− | otherwise ->
1604− return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1605− Done b -> return $ Done b
1606− step (SplitOnSeqKR s idx rb rh) x = do
1607− res <- fstep s x
1608− case res of
1609− Partial s1 -> do
1610− rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1611− if idx == maxIndex
1612− then do
1613− let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
1614− let !ringHash = fld addCksum 0 rb
1615− if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1616− then Done <$> ffinal s1
1617− else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1618− else
1619− return $ Partial $ SplitOnSeqKR s1 (idx + 1) rb rh1
1620− Done b -> return $ Done b
1621− step (SplitOnSeqKRLoop s cksum rb rh) x = do
1622− res <- fstep s x
1623− case res of
1624− Partial s1 -> do
1625− old <- liftIO $ peek rh
1626− rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1627− let ringHash = deltaCksum cksum old x
1628− if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1629− then Done <$> ffinal s1
1630− else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1631− Done b -> return $ Done b
1632−
1633− extractFunc fex state =
1634− let st =
1635− case state of
1636− SplitOnSeqEmpty s -> s
1637− SplitOnSeqSingle s _ -> s
1638− SplitOnSeqWord s _ _ -> s
1639− SplitOnSeqWordLoop s _ -> s
1640− SplitOnSeqKR s _ _ _ -> s
1641− SplitOnSeqKRLoop s _ _ _ -> s
1642− in fex st
1643−
1644− extract state = extractFunc fextract state
1645−
1646− final state = extractFunc ffinal state
1647−
1648−-- | Like 'takeEndBySeq' but discards the matched sequence.
1649−--
1650−-- /Pre-release/
1651−--
1652−{-# INLINE takeEndBySeq_ #-}
1653−takeEndBySeq_ :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
1654− Array.Array a
1655− -> Fold m a b
1656− -> Fold m a b
1657−takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
1658− Fold step initial extract final
1659−
1660− where
1661−
1662− patLen = Array.length patArr
1663−
1664− initial = do
1665− res <- finitial
1666− case res of
1667− Partial acc
1668− | patLen == 0 ->
1669− -- XXX Should we match nothing or everything on empty
1670− -- pattern?
1671− -- Done <$> ffinal acc
1672− return $ Partial $ SplitOnSeqEmpty acc
1673− | patLen == 1 -> do
1674− pat <- liftIO $ Array.unsafeIndexIO 0 patArr
1675− return $ Partial $ SplitOnSeqSingle acc pat
1676− -- XXX Need to add tests for this case
1677− | SIZE_OF(a) * patLen <= sizeOf (Proxy :: Proxy Word) ->
1678− return $ Partial $ SplitOnSeqWord acc 0 0
1679− | otherwise -> do
1680− (rb, rhead) <- liftIO $ Ring.new patLen
1681− return $ Partial $ SplitOnSeqKR acc 0 rb rhead
1682− Done b -> return $ Done b
1683−
1684− -- Word pattern related
1685− maxIndex = patLen - 1
1686−
1687− elemBits = SIZE_OF(a) * 8
1688−
1689− wordMask :: Word
1690− wordMask = (1 `shiftL` (elemBits * patLen)) - 1
1691−
1692− elemMask :: Word
1693− elemMask = (1 `shiftL` elemBits) - 1
1694−
1695− wordPat :: Word
1696− wordPat = wordMask .&. Array.foldl' addToWord 0 patArr
1697−
1698− addToWord wd a = (wd `shiftL` elemBits) .|. fromIntegral (fromEnum a)
1699−
1700− -- For Rabin-Karp search
1701− k = 2891336453 :: Word32
1702− coeff = k ^ patLen
1703−
1704− addCksum cksum a = cksum * k + fromIntegral (fromEnum a)
1705−
1706− deltaCksum cksum old new =
1707− addCksum cksum new - coeff * fromIntegral (fromEnum old)
1708−
1709− -- XXX shall we use a random starting hash or 1 instead of 0?
1710− -- XXX Need to keep this cached across fold calls in foldMany
1711− -- XXX We may need refold to inject the cached state instead of
1712− -- initializing the state every time.
1713− -- XXX Allocation of ring buffer should also be done once
1714− patHash = Array.foldl' addCksum 0 patArr
1715−
1716− step (SplitOnSeqEmpty s) x = do
1717− res <- fstep s x
1718− case res of
1719− Partial s1 -> return $ Partial $ SplitOnSeqEmpty s1
1720− Done b -> return $ Done b
1721− step (SplitOnSeqSingle s pat) x = do
1722− if pat /= x
1723− then do
1724− res <- fstep s x
1725− case res of
1726− Partial s1 -> return $ Partial $ SplitOnSeqSingle s1 pat
1727− Done b -> return $ Done b
1728− else Done <$> ffinal s
1729− step (SplitOnSeqWord s idx wrd) x = do
1730− let wrd1 = addToWord wrd x
1731− if idx == maxIndex
1732− then do
1733− if wrd1 .&. wordMask == wordPat
1734− then Done <$> ffinal s
1735− else return $ Partial $ SplitOnSeqWordLoop s wrd1
1736− else return $ Partial $ SplitOnSeqWord s (idx + 1) wrd1
1737− step (SplitOnSeqWordLoop s wrd) x = do
1738− let wrd1 = addToWord wrd x
1739− old = (wordMask .&. wrd)
1740− `shiftR` (elemBits * (patLen - 1))
1741− res <- fstep s (toEnum $ fromIntegral old)
1742− case res of
1743− Partial s1
1744− | wrd1 .&. wordMask == wordPat ->
1745− Done <$> ffinal s1
1746− | otherwise ->
1747− return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1748− Done b -> return $ Done b
1749− step (SplitOnSeqKR s idx rb rh) x = do
1750− rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1751− if idx == maxIndex
1752− then do
1753− let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
1754− let !ringHash = fld addCksum 0 rb
1755− if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1756− then Done <$> ffinal s
1757− else return $ Partial $ SplitOnSeqKRLoop s ringHash rb rh1
1758− else return $ Partial $ SplitOnSeqKR s (idx + 1) rb rh1
1759− step (SplitOnSeqKRLoop s cksum rb rh) x = do
1760− old <- liftIO $ peek rh
1761− res <- fstep s old
1762− case res of
1763− Partial s1 -> do
1764− rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1765− let ringHash = deltaCksum cksum old x
1766− if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1767− then Done <$> ffinal s1
1768− else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1769− Done b -> return $ Done b
1770−
1771− -- XXX extract should return backtrack count as well. If the fold
1772− -- terminates early inside extract, we may still have buffered data
1773− -- remaining which will be lost if we do not communicate that to the
1774− -- driver.
1775− extractFunc fex state = do
1776− let consumeWord s n wrd = do
1777− if n == 0
1778− then fex s
1779− else do
1780− let old = elemMask .&. (wrd `shiftR` (elemBits * (n - 1)))
1781− r <- fstep s (toEnum $ fromIntegral old)
1782− case r of
1783− Partial s1 -> consumeWord s1 (n - 1) wrd
1784− Done b -> return b
1785−
1786− let consumeRing s n rb rh =
1787− if n == 0
1788− then fex s
1789− else do
1790− old <- liftIO $ peek rh
1791− let rh1 = Ring.advance rb rh
1792− r <- fstep s old
1793− case r of
1794− Partial s1 -> consumeRing s1 (n - 1) rb rh1
1795− Done b -> return b
1796−
1797− case state of
1798− SplitOnSeqEmpty s -> fex s
1799− SplitOnSeqSingle s _ -> fex s
1800− SplitOnSeqWord s idx wrd -> consumeWord s idx wrd
1801− SplitOnSeqWordLoop s wrd -> consumeWord s patLen wrd
1802− SplitOnSeqKR s idx rb _ -> consumeRing s idx rb (Ring.startOf rb)
1803− SplitOnSeqKRLoop s _ rb rh -> consumeRing s patLen rb rh
1804−
1805− extract state = extractFunc fextract state
1806−
1807− final state = extractFunc ffinal state
1808−
1809−------------------------------------------------------------------------------
1810−-- Distributing
1811−------------------------------------------------------------------------------
1812−--
1813−-- | Distribute one copy of the stream to each fold and zip the results.
1814−--
1815−-- @
1816−-- |-------Fold m a b--------|
1817−-- ---stream m a---| |---m (b,c)
1818−-- |-------Fold m a c--------|
1819−-- @
1820−--
1821−-- Definition:
1822−--
1823−-- >>> tee = Fold.teeWith (,)
1824−--
1825−-- Example:
1826−--
1827−-- >>> t = Fold.tee Fold.sum Fold.length
1828−-- >>> Stream.fold t (Stream.enumerateFromTo 1.0 100.0)
1829−-- (5050.0,100)
1830−--
1831−{-# INLINE tee #-}
1832−tee :: Monad m => Fold m a b -> Fold m a c -> Fold m a (b,c)
1833−tee = teeWith (,)
1834−
1835−-- XXX use "List" instead of "[]"?, use Array for output to scale it to a large
1836−-- number of consumers? For polymorphic case a vector could be helpful. For
1837−-- Storables we can use arrays. Will need separate APIs for those.
1838−--
1839−-- | Distribute one copy of the stream to each fold and collect the results in
1840−-- a container.
1841−--
1842−-- @
1843−--
1844−-- |-------Fold m a b--------|
1845−-- ---stream m a---| |---m [b]
1846−-- |-------Fold m a b--------|
1847−-- | |
1848−-- ...
1849−-- @
1850−--
1851−-- >>> Stream.fold (Fold.distribute [Fold.sum, Fold.length]) (Stream.enumerateFromTo 1 5)
1852−-- [15,5]
1853−--
1854−-- >>> distribute = Prelude.foldr (Fold.teeWith (:)) (Fold.fromPure [])
1855−--
1856−-- This is the consumer side dual of the producer side 'sequence' operation.
1857−--
1858−-- Stops when all the folds stop.
1859−--
1860−{-# INLINE distribute #-}
1861−distribute :: Monad m => [Fold m a b] -> Fold m a [b]
1862−distribute = Prelude.foldr (teeWith (:)) (fromPure [])
1863−
1864−------------------------------------------------------------------------------
1865−-- Partitioning
1866−------------------------------------------------------------------------------
1867−
1868−{-# INLINE partitionByMUsing #-}
1869−partitionByMUsing :: Monad m =>
1870− ( (x -> y -> (x, y))
1871− -> Fold m (Either b c) x
1872− -> Fold m (Either b c) y
1873− -> Fold m (Either b c) (x, y)
1874− )
1875− -> (a -> m (Either b c))
1876− -> Fold m b x
1877− -> Fold m c y
1878− -> Fold m a (x, y)
1879−partitionByMUsing t f fld1 fld2 =
1880− let l = lmap (fromLeft undefined) fld1 -- :: Fold m (Either b c) x
1881− r = lmap (fromRight undefined) fld2 -- :: Fold m (Either b c) y
1882− in lmapM f (t (,) (filter isLeft l) (filter isRight r))
1883−
1884−-- | Partition the input over two folds using an 'Either' partitioning
1885−-- predicate.
1886−--
1887−-- @
1888−--
1889−-- |-------Fold b x--------|
1890−-- -----stream m a --> (Either b c)----| |----(x,y)
1891−-- |-------Fold c y--------|
1892−-- @
1893−--
1894−-- Example, send input to either fold randomly:
1895−--
1896−-- >>> :set -package random
1897−-- >>> import System.Random (randomIO)
1898−-- >>> randomly a = randomIO >>= \x -> return $ if x then Left a else Right a
1899−-- >>> f = Fold.partitionByM randomly Fold.length Fold.length
1900−-- >>> Stream.fold f (Stream.enumerateFromTo 1 100)
1901−-- ...
1902−--
1903−-- Example, send input to the two folds in a proportion of 2:1:
1904−--
1905−-- >>> :{
1906−-- proportionately m n = do
1907−-- ref <- newIORef $ cycle $ concat [replicate m Left, replicate n Right]
1908−-- return $ \a -> do
1909−-- r <- readIORef ref
1910−-- writeIORef ref $ tail r
1911−-- return $ Prelude.head r a
1912−-- :}
1913−--
1914−-- >>> :{
1915−-- main = do
1916−-- g <- proportionately 2 1
1917−-- let f = Fold.partitionByM g Fold.length Fold.length
1918−-- r <- Stream.fold f (Stream.enumerateFromTo (1 :: Int) 100)
1919−-- print r
1920−-- :}
1921−--
1922−-- >>> main
1923−-- (67,33)
1924−--
1925−--
1926−-- This is the consumer side dual of the producer side 'mergeBy' operation.
1927−--
1928−-- When one fold is done, any input meant for it is ignored until the other
1929−-- fold is also done.
1930−--
1931−-- Stops when both the folds stop.
1932−--
1933−-- /See also: 'partitionByFstM' and 'partitionByMinM'./
1934−--
1935−-- /Pre-release/
1936−{-# INLINE partitionByM #-}
1937−partitionByM :: Monad m
1938− => (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
1939−partitionByM = partitionByMUsing teeWith
1940−
1941−-- | Similar to 'partitionByM' but terminates when the first fold terminates.
1942−--
1943−{-# INLINE partitionByFstM #-}
1944−partitionByFstM :: Monad m
1945− => (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
1946−partitionByFstM = partitionByMUsing teeWithFst
1947−
1948−-- | Similar to 'partitionByM' but terminates when any fold terminates.
1949−--
1950−{-# INLINE partitionByMinM #-}
1951−partitionByMinM :: Monad m =>
1952− (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
1953−partitionByMinM = partitionByMUsing teeWithMin
1954−
1955−-- Note: we could use (a -> Bool) instead of (a -> Either b c), but the latter
1956−-- makes the signature clearer as to which case belongs to which fold.
1957−-- XXX need to check the performance in both cases.
1958−
1959−-- | Same as 'partitionByM' but with a pure partition function.
1960−--
1961−-- Example, count even and odd numbers in a stream:
1962−--
1963−-- >>> :{
1964−-- let f = Fold.partitionBy (\n -> if even n then Left n else Right n)
1965−-- (fmap (("Even " ++) . show) Fold.length)
1966−-- (fmap (("Odd " ++) . show) Fold.length)
1967−-- in Stream.fold f (Stream.enumerateFromTo 1 100)
1968−-- :}
1969−-- ("Even 50","Odd 50")
1970−--
1971−-- /Pre-release/
1972−{-# INLINE partitionBy #-}
1973−partitionBy :: Monad m
1974− => (a -> Either b c) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
1975−partitionBy f = partitionByM (return . f)
1976−
1977−-- | Compose two folds such that the combined fold accepts a stream of 'Either'
1978−-- and routes the 'Left' values to the first fold and 'Right' values to the
1979−-- second fold.
1980−--
1981−-- Definition:
1982−--
1983−-- >>> partition = Fold.partitionBy id
1984−--
1985−{-# INLINE partition #-}
1986−partition :: Monad m
1987− => Fold m b x -> Fold m c y -> Fold m (Either b c) (x, y)
1988−partition = partitionBy id
1989−
1990−{-
1991−-- | Send one item to each fold in a round-robin fashion. This is the consumer
1992−-- side dual of producer side 'mergeN' operation.
1993−--
1994−-- partitionN :: Monad m => [Fold m a b] -> Fold m a [b]
1995−-- partitionN fs = Fold step begin done
1996−-}
1997−
1998−------------------------------------------------------------------------------
1999−-- Unzipping
2000−------------------------------------------------------------------------------
2001−
2002−{-# INLINE unzipWithMUsing #-}
2003−unzipWithMUsing :: Monad m =>
2004− ( (x -> y -> (x, y))
2005− -> Fold m (b, c) x
2006− -> Fold m (b, c) y
2007− -> Fold m (b, c) (x, y)
2008− )
2009− -> (a -> m (b, c))
2010− -> Fold m b x
2011− -> Fold m c y
2012− -> Fold m a (x, y)
2013−unzipWithMUsing t f fld1 fld2 =
2014− let f1 = lmap fst fld1 -- :: Fold m (b, c) b
2015− f2 = lmap snd fld2 -- :: Fold m (b, c) c
2016− in lmapM f (t (,) f1 f2)
2017−
2018−-- | Like 'unzipWith' but with a monadic splitter function.
2019−--
2020−-- Definition:
2021−--
2022−-- >>> unzipWithM k f1 f2 = Fold.lmapM k (Fold.unzip f1 f2)
2023−--
2024−-- /Pre-release/
2025−{-# INLINE unzipWithM #-}
2026−unzipWithM :: Monad m
2027− => (a -> m (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2028−unzipWithM = unzipWithMUsing teeWith
2029−
2030−-- | Similar to 'unzipWithM' but terminates when the first fold terminates.
2031−--
2032−{-# INLINE unzipWithFstM #-}
2033−unzipWithFstM :: Monad m =>
2034− (a -> m (b, c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2035−unzipWithFstM = unzipWithMUsing teeWithFst
2036−
2037−-- | Similar to 'unzipWithM' but terminates when any fold terminates.
2038−--
2039−{-# INLINE unzipWithMinM #-}
2040−unzipWithMinM :: Monad m =>
2041− (a -> m (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2042−unzipWithMinM = unzipWithMUsing teeWithMin
2043−
2044−-- | Split elements in the input stream into two parts using a pure splitter
2045−-- function, direct each part to a different fold and zip the results.
2046−--
2047−-- Definitions:
2048−--
2049−-- >>> unzipWith f = Fold.unzipWithM (return . f)
2050−-- >>> unzipWith f fld1 fld2 = Fold.lmap f (Fold.unzip fld1 fld2)
2051−--
2052−-- This fold terminates when both the input folds terminate.
2053−--
2054−-- /Pre-release/
2055−{-# INLINE unzipWith #-}
2056−unzipWith :: Monad m
2057− => (a -> (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2058−unzipWith f = unzipWithM (return . f)
2059−
2060−-- | Send the elements of tuples in a stream of tuples through two different
2061−-- folds.
2062−--
2063−-- @
2064−--
2065−-- |-------Fold m a x--------|
2066−-- ---------stream of (a,b)--| |----m (x,y)
2067−-- |-------Fold m b y--------|
2068−--
2069−-- @
2070−--
2071−-- Definition:
2072−--
2073−-- >>> unzip = Fold.unzipWith id
2074−--
2075−-- This is the consumer side dual of the producer side 'zip' operation.
2076−--
2077−{-# INLINE unzip #-}
2078−unzip :: Monad m => Fold m a x -> Fold m b y -> Fold m (a,b) (x,y)
2079−unzip = unzipWith id
2080−
2081−------------------------------------------------------------------------------
2082−-- Combining streams and folds - Zipping
2083−------------------------------------------------------------------------------
2084−
2085−-- XXX These can be implemented using the fold scan, using the stream as a
2086−-- state.
2087−-- XXX Stream Skip state cannot be efficiently handled in folds but can be
2088−-- handled in parsers using the Continue facility. See zipWithM in the Parser
2089−-- module.
2090−--
2091−-- cmpBy, eqBy, isPrefixOf, isSubsequenceOf etc can be implemented using
2092−-- zipStream.
2093−
2094−-- | Zip a stream with the input of a fold using the supplied function.
2095−--
2096−-- /Unimplemented/
2097−--
2098−{-# INLINE zipStreamWithM #-}
2099−zipStreamWithM :: -- Monad m =>
2100− (a -> b -> m c) -> Stream m a -> Fold m c x -> Fold m b x
2101−zipStreamWithM = undefined
2102−
2103−-- | Zip a stream with the input of a fold.
2104−--
2105−-- >>> zip = Fold.zipStreamWithM (curry return)
2106−--
2107−-- /Unimplemented/
2108−--
2109−{-# INLINE zipStream #-}
2110−zipStream :: Monad m => Stream m a -> Fold m (a, b) x -> Fold m b x
2111−zipStream = zipStreamWithM (curry return)
2112−
2113−-- | Pair each element of a fold input with its index, starting from index 0.
2114−--
2115−{-# INLINE indexingWith #-}
2116−indexingWith :: Monad m => Int -> (Int -> Int) -> Fold m a (Maybe (Int, a))
2117−indexingWith i f = fmap toMaybe $ foldl' step initial
2118−
2119− where
2120−
2121− initial = Nothing'
2122−
2123− step Nothing' a = Just' (i, a)
2124− step (Just' (n, _)) a = Just' (f n, a)
2125−
2126−-- |
2127−-- >>> indexing = Fold.indexingWith 0 (+ 1)
2128−--
2129−{-# INLINE indexing #-}
2130−indexing :: Monad m => Fold m a (Maybe (Int, a))
2131−indexing = indexingWith 0 (+ 1)
2132−
2133−-- |
2134−-- >>> indexingRev n = Fold.indexingWith n (subtract 1)
2135−--
2136−{-# INLINE indexingRev #-}
2137−indexingRev :: Monad m => Int -> Fold m a (Maybe (Int, a))
2138−indexingRev n = indexingWith n (subtract 1)
2139−
2140−-- | Pair each element of a fold input with its index, starting from index 0.
2141−--
2142−-- >>> indexed = Fold.scanMaybe Fold.indexing
2143−--
2144−{-# INLINE indexed #-}
2145−indexed :: Monad m => Fold m (Int, a) b -> Fold m a b
2146−indexed = scanMaybe indexing
2147−
2148−-- | Change the predicate function of a Fold from @a -> b@ to accept an
2149−-- additional state input @(s, a) -> b@. Convenient to filter with an
2150−-- addiitonal index or time input.
2151−--
2152−-- >>> filterWithIndex = Fold.with Fold.indexed Fold.filter
2153−--
2154−-- @
2155−-- filterWithAbsTime = with timestamped filter
2156−-- filterWithRelTime = with timeIndexed filter
2157−-- @
2158−--
2159−-- /Pre-release/
2160−{-# INLINE with #-}
2161−with ::
2162− (Fold m (s, a) b -> Fold m a b)
2163− -> (((s, a) -> c) -> Fold m (s, a) b -> Fold m (s, a) b)
2164− -> (((s, a) -> c) -> Fold m a b -> Fold m a b)
2165−with f comb g = f . comb g . lmap snd
2166−
2167−-- XXX Implement as a filter
2168−-- sampleFromthen :: Monad m => Int -> Int -> Fold m a (Maybe a)
2169−
2170−-- | @sampleFromthen offset stride@ samples the element at @offset@ index and
2171−-- then every element at strides of @stride@.
2172−--
2173−{-# INLINE sampleFromthen #-}
2174−sampleFromthen :: Monad m => Int -> Int -> Fold m a b -> Fold m a b
2175−sampleFromthen offset size =
2176− with indexed filter (\(i, _) -> (i + offset) `mod` size == 0)
2177−
2178−------------------------------------------------------------------------------
2179−-- Nesting
2180−------------------------------------------------------------------------------
2181−
2182−-- | @concatSequence f t@ applies folds from stream @t@ sequentially and
2183−-- collects the results using the fold @f@.
2184−--
2185−-- /Unimplemented/
2186−--
2187−{-# INLINE concatSequence #-}
2188−concatSequence ::
2189− -- IsStream t =>
2190− Fold m b c -> t (Fold m a b) -> Fold m a c
2191−concatSequence _f _p = undefined
2192−
2193−-- | Group the input stream into groups of elements between @low@ and @high@.
2194−-- Collection starts in chunks of @low@ and then keeps doubling until we reach
2195−-- @high@. Each chunk is folded using the provided fold function.
2196−--
2197−-- This could be useful, for example, when we are folding a stream of unknown
2198−-- size to a stream of arrays and we want to minimize the number of
2199−-- allocations.
2200−--
2201−-- NOTE: this would be an application of "many" using a terminating fold.
2202−--
2203−-- /Unimplemented/
2204−--
2205−{-# INLINE chunksBetween #-}
2206−chunksBetween :: -- Monad m =>
2207− Int -> Int -> Fold m a b -> Fold m b c -> Fold m a c
2208−chunksBetween _low _high _f1 _f2 = undefined
2209−
2210−-- | A fold that buffers its input to a pure stream.
2211−--
2212−-- /Warning!/ working on large streams accumulated as buffers in memory could
2213−-- be very inefficient, consider using "Streamly.Data.Array" instead.
2214−--
2215−-- >>> toStream = fmap Stream.fromList Fold.toList
2216−--
2217−-- /Pre-release/
2218−{-# INLINE toStream #-}
2219−toStream :: (Monad m, Monad n) => Fold m a (Stream n a)
2220−toStream = fmap StreamD.fromList toList
2221−
2222−-- This is more efficient than 'toStream'. toStream is exactly the same as
2223−-- reversing the stream after toStreamRev.
2224−--
2225−-- | Buffers the input stream to a pure stream in the reverse order of the
2226−-- input.
2227−--
2228−-- >>> toStreamRev = fmap Stream.fromList Fold.toListRev
2229−--
2230−-- /Warning!/ working on large streams accumulated as buffers in memory could
2231−-- be very inefficient, consider using "Streamly.Data.Array" instead.
2232−--
2233−-- /Pre-release/
2234−
2235−-- xn : ... : x2 : x1 : []
2236−{-# INLINE toStreamRev #-}
2237−toStreamRev :: (Monad m, Monad n) => Fold m a (Stream n a)
2238−toStreamRev = fmap StreamD.fromList toListRev
2239−
2240−-- XXX This does not fuse. It contains a recursive step function. We will need
2241−-- a Skip input constructor in the fold type to make it fuse.
2242−--
2243−-- | Unfold and flatten the input stream of a fold.
2244−--
2245−-- @
2246−-- Stream.fold (unfoldMany u f) = Stream.fold f . Stream.unfoldMany u
2247−-- @
2248−--
2249−-- /Pre-release/
2250−{-# INLINE unfoldMany #-}
2251−unfoldMany :: Monad m => Unfold m a b -> Fold m b c -> Fold m a c
2252−unfoldMany (Unfold ustep inject) (Fold fstep initial extract final) =
2253− Fold consume initial extract final
2254−
2255− where
2256−
2257− {-# INLINE produce #-}
2258− produce fs us = do
2259− ures <- ustep us
2260− case ures of
2261− StreamD.Yield b us1 -> do
2262− fres <- fstep fs b
2263− case fres of
2264− Partial fs1 -> produce fs1 us1
2265− -- XXX What to do with the remaining stream?
2266− Done c -> return $ Done c
2267− StreamD.Skip us1 -> produce fs us1
2268− StreamD.Stop -> return $ Partial fs
2269−
2270− {-# INLINE_LATE consume #-}
2271− consume s a = inject a >>= produce s
2272−
2273−-- | Get the bottom most @n@ elements using the supplied comparison function.
2274−--
2275−{-# INLINE bottomBy #-}
2276−bottomBy :: (MonadIO m, Unbox a) =>
2277− (a -> a -> Ordering)
2278− -> Int
2279− -> Fold m a (MutArray a)
2280−bottomBy cmp n = Fold step initial extract extract
2281−
2282− where
2283−
2284− initial = do
2285− arr <- MA.pinnedNew n
2286− if n <= 0
2287− then return $ Done arr
2288− else return $ Partial (arr, 0)
2289−
2290− step (arr, i) x =
2291− if i < n
2292− then do
2293− arr' <- MA.snoc arr x
2294− MA.bubble cmp arr'
2295− return $ Partial (arr', i + 1)
2296− else do
2297− x1 <- MA.getIndexUnsafe (i - 1) arr
2298− case x `cmp` x1 of
2299− LT -> do
2300− MA.putIndexUnsafe (i - 1) arr x
2301− MA.bubble cmp arr
2302− return $ Partial (arr, i)
2303− _ -> return $ Partial (arr, i)
2304−
2305− extract = return . fst
2306−
2307−-- | Get the top @n@ elements using the supplied comparison function.
2308−--
2309−-- To get bottom n elements instead:
2310−--
2311−-- >>> bottomBy cmp = Fold.topBy (flip cmp)
2312−--
2313−-- Example:
2314−--
2315−-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2316−-- >>> Stream.fold (Fold.topBy compare 3) stream >>= MutArray.toList
2317−-- [17,11,9]
2318−--
2319−-- /Pre-release/
2320−--
2321−{-# INLINE topBy #-}
2322−topBy :: (MonadIO m, Unbox a) =>
2323− (a -> a -> Ordering)
2324− -> Int
2325− -> Fold m a (MutArray a)
2326−topBy cmp = bottomBy (flip cmp)
2327−
2328−-- | Fold the input stream to top n elements.
2329−--
2330−-- Definition:
2331−--
2332−-- >>> top = Fold.topBy compare
2333−--
2334−-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2335−-- >>> Stream.fold (Fold.top 3) stream >>= MutArray.toList
2336−-- [17,11,9]
2337−--
2338−-- /Pre-release/
2339−{-# INLINE top #-}
2340−top :: (MonadIO m, Unbox a, Ord a) => Int -> Fold m a (MutArray a)
2341−top = bottomBy $ flip compare
2342−
2343−-- | Fold the input stream to bottom n elements.
2344−--
2345−-- Definition:
2346−--
2347−-- >>> bottom = Fold.bottomBy compare
2348−--
2349−-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2350−-- >>> Stream.fold (Fold.bottom 3) stream >>= MutArray.toList
2351−-- [1,2,3]
2352−--
2353−-- /Pre-release/
2354−{-# INLINE bottom #-}
2355−bottom :: (MonadIO m, Unbox a, Ord a) => Int -> Fold m a (MutArray a)
2356−bottom = bottomBy compare
2357−
2358−------------------------------------------------------------------------------
2359−-- Interspersed parsing
2360−------------------------------------------------------------------------------
2361−
2362−data IntersperseQState fs ps =
2363− IntersperseQUnquoted !fs !ps
2364− | IntersperseQQuoted !fs !ps
2365− | IntersperseQQuotedEsc !fs !ps
2366−
2367−-- Useful for parsing CSV with quoting and escaping
2368−{-# INLINE intersperseWithQuotes #-}
2369−intersperseWithQuotes :: (Monad m, Eq a) =>
2370− a -> a -> a -> Fold m a b -> Fold m b c -> Fold m a c
2371−intersperseWithQuotes
2372− quote
2373− esc
2374− separator
2375− (Fold stepL initialL _ finalL)
2376− (Fold stepR initialR extractR finalR) = Fold step initial extract final
2377−
2378− where
2379−
2380− errMsg p status =
2381− error $ "intersperseWithQuotes: " ++ p ++ " parsing fold cannot "
2382− ++ status ++ " without input"
2383−
2384− {-# INLINE initL #-}
2385− initL mkState = do
2386− resL <- initialL
2387− case resL of
2388− Partial sL ->
2389− return $ Partial $ mkState sL
2390− Done _ ->
2391− errMsg "content" "succeed"
2392−
2393− initial = do
2394− res <- initialR
2395− case res of
2396− Partial sR -> initL (IntersperseQUnquoted sR)
2397− Done b -> return $ Done b
2398−
2399− {-# INLINE collect #-}
2400− collect nextS sR b = do
2401− res <- stepR sR b
2402− case res of
2403− Partial s ->
2404− initL (nextS s)
2405− Done c -> return (Done c)
2406−
2407− {-# INLINE process #-}
2408− process a sL sR nextState = do
2409− r <- stepL sL a
2410− case r of
2411− Partial s -> return $ Partial (nextState sR s)
2412− Done b -> collect nextState sR b
2413−
2414− {-# INLINE processQuoted #-}
2415− processQuoted a sL sR nextState = do
2416− r <- stepL sL a
2417− case r of
2418− Partial s -> return $ Partial (nextState sR s)
2419− Done _ -> do
2420− _ <- finalR sR
2421− error "Collecting fold finished inside quote"
2422−
2423− step (IntersperseQUnquoted sR sL) a
2424− | a == separator = do
2425− b <- finalL sL
2426− collect IntersperseQUnquoted sR b
2427− | a == quote = processQuoted a sL sR IntersperseQQuoted
2428− | otherwise = process a sL sR IntersperseQUnquoted
2429−
2430− step (IntersperseQQuoted sR sL) a
2431− | a == esc = processQuoted a sL sR IntersperseQQuotedEsc
2432− | a == quote = process a sL sR IntersperseQUnquoted
2433− | otherwise = processQuoted a sL sR IntersperseQQuoted
2434−
2435− step (IntersperseQQuotedEsc sR sL) a =
2436− processQuoted a sL sR IntersperseQQuoted
2437−
2438− extract (IntersperseQUnquoted sR _) = extractR sR
2439− extract (IntersperseQQuoted _ _) =
2440− error "intersperseWithQuotes: finished inside quote"
2441− extract (IntersperseQQuotedEsc _ _) =
2442− error "intersperseWithQuotes: finished inside quote, at escape char"
2443−
2444− final (IntersperseQUnquoted sR sL) = finalL sL *> finalR sR
2445− final (IntersperseQQuoted sR sL) = do
2446− _ <- finalR sR
2447− _ <- finalL sL
2448− error "intersperseWithQuotes: finished inside quote"
2449− final (IntersperseQQuotedEsc sR sL) = do
2450− _ <- finalR sR
2451− _ <- finalL sL
2452− error "intersperseWithQuotes: finished inside quote, at escape char"
core/src/Streamly/Internal/Data/Fold/Container.hs+32−75
@@ -9,6 +9,9 @@
99
1010module Streamly.Internal.Data.Fold.Container
1111 (
12+ -- * Imports
13+ -- $setup
14+
1215 -- * Set operations
1316 toSet
1417 , toIntSet
@@ -100,8 +103,7 @@ import qualified Data.Set as Set
100103import qualified Streamly.Internal.Data.IsMap as IsMap
101104
102105import Prelude hiding (Foldable(..))
103−import Streamly.Internal.Data.Fold.Type
104−import Streamly.Internal.Data.Fold.Combinators
106+import Streamly.Internal.Data.Fold
105107
106108-- $setup
107109-- >>> :m
@@ -270,15 +272,14 @@ demuxGeneric :: (Monad m, IsMap f, Traversable f) =>
270272 (a -> Key f)
271273 -> (a -> m (Fold m a b))
272274 -> Fold m a (m (f b), Maybe (Key f, b))
273−demuxGeneric getKey getFold =
274− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
275+demuxGeneric getKey getFold = fmap extract $ foldlM' step initial
275276
276277 where
277278
278279 initial = return $ Tuple' IsMap.mapEmpty Nothing
279280
280281 {-# INLINE runFold #-}
281− runFold kv (Fold step1 initial1 extract1 final1) (k, a) = do
282+ runFold kv (Fold step1 initial1 extract1) (k, a) = do
282283 res <- initial1
283284 case res of
284285 Partial s -> do
@@ -286,7 +287,7 @@ demuxGeneric getKey getFold =
286287 return
287288 $ case res1 of
288289 Partial _ ->
289− let fld = Fold step1 (return res1) extract1 final1
290+ let fld = Fold step1 (return res1) extract1
290291 in Tuple' (IsMap.mapInsert k fld kv) Nothing
291292 Done b -> Tuple' (IsMap.mapDelete k kv) (Just (k, b))
292293 Done b -> return $ Tuple' kv (Just (k, b))
@@ -299,25 +300,15 @@ demuxGeneric getKey getFold =
299300 runFold kv fld (k, a)
300301 Just f -> runFold kv f (k, a)
301302
302− extract (Tuple' kv x) = return (Prelude.mapM f kv, x)
303+ extract (Tuple' kv x) = (Prelude.mapM f kv, x)
303304
304305 where
305306
306− f (Fold _ i e _) = do
307+ f (Fold _ i e) = do
307308 r <- i
308309 case r of
309310 Partial s -> e s
310− _ -> error "demuxGeneric: unreachable code"
311−
312− final (Tuple' kv x) = return (Prelude.mapM f kv, x)
313−
314− where
315−
316− f (Fold _ i _ fin) = do
317− r <- i
318− case r of
319− Partial s -> fin s
320− _ -> error "demuxGeneric: unreachable code"
311+ Done b -> return b
321312
322313-- | @demux getKey getFold@: In a key value stream, fold values corresponding
323314-- to each key using a key specific fold. @getFold@ is invoked to generate a
@@ -359,36 +350,35 @@ demuxGenericIO :: (MonadIO m, IsMap f, Traversable f) =>
359350 (a -> Key f)
360351 -> (a -> m (Fold m a b))
361352 -> Fold m a (m (f b), Maybe (Key f, b))
362−demuxGenericIO getKey getFold =
363− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
353+demuxGenericIO getKey getFold = fmap extract $ foldlM' step initial
364354
365355 where
366356
367357 initial = return $ Tuple' IsMap.mapEmpty Nothing
368358
369359 {-# INLINE initFold #-}
370− initFold kv (Fold step1 initial1 extract1 final1) (k, a) = do
360+ initFold kv (Fold step1 initial1 extract1) (k, a) = do
371361 res <- initial1
372362 case res of
373363 Partial s -> do
374364 res1 <- step1 s a
375365 case res1 of
376366 Partial _ -> do
377− let fld = Fold step1 (return res1) extract1 final1
367+ let fld = Fold step1 (return res1) extract1
378368 ref <- liftIO $ newIORef fld
379369 return $ Tuple' (IsMap.mapInsert k ref kv) Nothing
380370 Done b -> return $ Tuple' kv (Just (k, b))
381371 Done b -> return $ Tuple' kv (Just (k, b))
382372
383373 {-# INLINE runFold #-}
384− runFold kv ref (Fold step1 initial1 extract1 final1) (k, a) = do
374+ runFold kv ref (Fold step1 initial1 extract1) (k, a) = do
385375 res <- initial1
386376 case res of
387377 Partial s -> do
388378 res1 <- step1 s a
389379 case res1 of
390380 Partial _ -> do
391− let fld = Fold step1 (return res1) extract1 final1
381+ let fld = Fold step1 (return res1) extract1
392382 liftIO $ writeIORef ref fld
393383 return $ Tuple' kv Nothing
394384 Done b ->
@@ -406,27 +396,16 @@ demuxGenericIO getKey getFold =
406396 f <- liftIO $ readIORef ref
407397 runFold kv ref f (k, a)
408398
409− extract (Tuple' kv x) = return (Prelude.mapM f kv, x)
399+ extract (Tuple' kv x) = (Prelude.mapM f kv, x)
410400
411401 where
412402
413403 f ref = do
414− Fold _ i e _ <- liftIO $ readIORef ref
404+ (Fold _ i e) <- liftIO $ readIORef ref
415405 r <- i
416406 case r of
417407 Partial s -> e s
418− _ -> error "demuxGenericIO: unreachable code"
419−
420− final (Tuple' kv x) = return (Prelude.mapM f kv, x)
421−
422− where
423−
424− f ref = do
425− Fold _ i _ fin <- liftIO $ readIORef ref
426− r <- i
427− case r of
428− Partial s -> fin s
429− _ -> error "demuxGenericIO: unreachable code"
408+ Done b -> return b
430409
431410-- | This is specialized version of 'demux' that uses mutable IO cells as
432411-- fold accumulators for better performance.
@@ -441,13 +420,6 @@ demuxIO :: (MonadIO m, Ord k) =>
441420 -> Fold m a (m (Map k b), Maybe (k, b))
442421demuxIO = demuxGenericIO
443422
444−-- | Fold a key value stream to a key-value Map. If the same key appears
445−-- multiple times, only the last value is retained.
446−{-# INLINE kvToMapOverwriteGeneric #-}
447−kvToMapOverwriteGeneric :: (Monad m, IsMap f) => Fold m (Key f, a) (f a)
448−kvToMapOverwriteGeneric =
449− foldl' (\kv (k, v) -> IsMap.mapInsert k v kv) IsMap.mapEmpty
450−
451423{-# INLINE demuxToContainer #-}
452424demuxToContainer :: (Monad m, IsMap f, Traversable f) =>
453425 (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
@@ -535,8 +507,8 @@ classifyGeneric :: (Monad m, IsMap f, Traversable f, Ord (Key f)) =>
535507 -- for that use case. We return an action because we want it to be lazy so
536508 -- that the downstream consumers can choose to process or discard it.
537509 (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
538−classifyGeneric f (Fold step1 initial1 extract1 final1) =
539− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
510+classifyGeneric f (Fold step1 initial1 extract1) =
511+ fmap extract $ foldlM' step initial
540512
541513 where
542514
@@ -573,16 +545,7 @@ classifyGeneric f (Fold step1 initial1 extract1 final1) =
573545 let kv1 = IsMap.mapDelete k kv
574546 in Tuple3' kv1 (Set.insert k set) (Just (k, b))
575547
576− extract (Tuple3' kv _ x) = return (Prelude.mapM extract1 kv, x)
577−
578− final (Tuple3' kv set x) = return (IsMap.mapTraverseWithKey f1 kv, x)
579−
580− where
581−
582− f1 k s = do
583− if Set.member k set
584− then extract1 s
585− else final1 s
548+ extract (Tuple3' kv _ x) = (Prelude.mapM extract1 kv, x)
586549
587550-- | Folds the values for each key using the supplied fold. When scanning, as
588551-- soon as the fold is complete, its result is available in the second
@@ -607,8 +570,8 @@ classify = classifyGeneric
607570{-# INLINE classifyGenericIO #-}
608571classifyGenericIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) =>
609572 (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
610−classifyGenericIO f (Fold step1 initial1 extract1 final1) =
611− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
573+classifyGenericIO f (Fold step1 initial1 extract1) =
574+ fmap extract $ foldlM' step initial
612575
613576 where
614577
@@ -647,21 +610,8 @@ classifyGenericIO f (Fold step1 initial1 extract1 final1) =
647610 in return
648611 $ Tuple3' kv1 (Set.insert k set) (Just (k, b))
649612
650− extract (Tuple3' kv _ x) = return (Prelude.mapM g kv, x)
651−
652− where
653−
654− g ref = liftIO (readIORef ref) >>= extract1
655−
656− final (Tuple3' kv set x) = return (IsMap.mapTraverseWithKey g kv, x)
657−
658− where
659−
660− g k ref = do
661− s <- liftIO $ readIORef ref
662− if Set.member k set
663− then extract1 s
664− else final1 s
613+ extract (Tuple3' kv _ x) =
614+ (Prelude.mapM (\ref -> liftIO (readIORef ref) >>= extract1) kv, x)
665615
666616-- | Same as classify except that it uses mutable IORef cells in the
667617-- Map providing better performance. Be aware that if this is used as a scan,
@@ -676,6 +626,13 @@ classifyIO :: (MonadIO m, Ord k) =>
676626 (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
677627classifyIO = classifyGenericIO
678628
629+-- | Fold a key value stream to a key-value Map. If the same key appears
630+-- multiple times, only the last value is retained.
631+{-# INLINE kvToMapOverwriteGeneric #-}
632+kvToMapOverwriteGeneric :: (Monad m, IsMap f) => Fold m (Key f, a) (f a)
633+kvToMapOverwriteGeneric =
634+ foldl' (\kv (k, v) -> IsMap.mapInsert k v kv) IsMap.mapEmpty
635+
679636{-# INLINE toContainer #-}
680637toContainer :: (Monad m, IsMap f, Traversable f, Ord (Key f)) =>
681638 (a -> Key f) -> Fold m a b -> Fold m a (f b)
core/src/Streamly/Internal/Data/Fold/Step.hs+2−2
@@ -8,7 +8,7 @@
88--
99module Streamly.Internal.Data.Fold.Step
1010 (
11− -- * Step Type
11+ -- * Types
1212 Step (..)
1313
1414 , mapMStep
@@ -40,7 +40,7 @@ data Step s b
4040 = Partial !s
4141 | Done !b
4242
43−-- | 'first' maps over the fold state and 'second' maps over the fold result.
43+-- | 'first' maps over 'Partial' and 'second' maps over 'Done'.
4444--
4545instance Bifunctor Step where
4646 {-# INLINE bimap #-}
core/src/Streamly/Internal/Data/Fold/Tee.hs−2
@@ -16,9 +16,7 @@ module Streamly.Internal.Data.Fold.Tee
1616 )
1717where
1818
19−#if !MIN_VERSION_base(4,18,0)
2019import Control.Applicative (liftA2)
21−#endif
2220import Streamly.Internal.Data.Fold.Type (Fold)
2321
2422import qualified Streamly.Internal.Data.Fold.Type as Fold
core/src/Streamly/Internal/Data/Fold/Type.hs+120−344
@@ -344,9 +344,11 @@
344344--
345345module Streamly.Internal.Data.Fold.Type
346346 (
347− module Streamly.Internal.Data.Fold.Step
347+ -- * Imports
348+ -- $setup
348349
349− -- * Fold Type
350+ -- * Types
351+ Step (..)
350352 , Fold (..)
351353
352354 -- * Constructors
@@ -367,8 +369,6 @@ module Streamly.Internal.Data.Fold.Type
367369 , toList
368370 , toStreamK
369371 , toStreamKRev
370− , lengthGeneric
371− , length
372372
373373 -- * Combinators
374374
@@ -393,8 +393,6 @@ module Streamly.Internal.Data.Fold.Type
393393 -- ** Trimming
394394 , take
395395 , taking
396− , takeEndBy_
397− , takeEndBy
398396 , dropping
399397
400398 -- ** Sequential application
@@ -446,25 +444,21 @@ where
446444
447445#include "inline.hs"
448446
449−#if !MIN_VERSION_base(4,18,0)
450447import Control.Applicative (liftA2)
451−#endif
452−import Control.Monad ((>=>), void)
448+import Control.Monad ((>=>))
453449import Data.Bifunctor (Bifunctor(..))
454450import Data.Either (fromLeft, fromRight, isLeft, isRight)
455451import Data.Functor.Identity (Identity(..))
456452import Fusion.Plugin.Types (Fuse(..))
453+import Streamly.Internal.Data.Fold.Step (Step(..), mapMStep, chainStepM)
457454import Streamly.Internal.Data.Maybe.Strict (Maybe'(..), toMaybe)
458455import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
459456import Streamly.Internal.Data.Refold.Type (Refold(..))
460457
461−import qualified Streamly.Internal.Data.StreamK.Type as K
458+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
462459
463460import Prelude hiding (Foldable(..), concatMap, filter, map, take)
464461
465−-- Entire module is exported, do not import selectively
466−import Streamly.Internal.Data.Fold.Step
467−
468462#include "DocTestDataFold.hs"
469463
470464------------------------------------------------------------------------------
@@ -475,64 +469,23 @@ import Streamly.Internal.Data.Fold.Step
475469-- The type @b@ is the accumulator of the writer. That's the reason the
476470-- default folds in various modules are called "write".
477471
478−-- An alternative to using an "extract" function is to use "Partial s b" style
479−-- partial value so that we always emit the output value and there is no need
480−-- to extract. Then extract can be used for cleanup purposes. But in this case
481−-- in some cases we may need a "Continue" constructor where an output value is
482−-- not available, this was implicit earlier. Also, "b" should be lazy here so
483−-- that we do not always compute it even if we do not need it.
484−--
485−-- Partial s b --> extract :: s -> b
486−-- Continue --> extract :: s -> Maybe b
487−--
488−-- But keeping 'b' lazy does not let the fold optimize well. It leads to
489−-- significant regressions in the key-value folds.
490−--
491−-- The "final" function complicates combinators that take other folds as
492−-- argument because we need to call their finalizers at right places. An
493−-- alternative to reduce this complexity where it is not required is to use a
494−-- separate type for bracketed folds but then we need to manage the complexity
495−-- of two different fold types.
496−
497−-- The "final" function could be (s -> m (Step s b)), like in parsers so that
498−-- it can be called in a loop to drain the fold.
499−
500−-- | The type @Fold m a b@ represents a consumer of an input stream of values
501−-- of type @a@ and returning a final value of type @b@ in 'Monad' @m@. The
502−-- constructor of a fold is @Fold step initial extract final@.
503−--
504−-- The fold uses an internal state of type @s@. The initial value of the state
505−-- @s@ is created by @initial@. This function is called once and only once
506−-- before the fold starts consuming input. Any resource allocation can be done
507−-- in this function.
508−--
509−-- The @step@ function is called on each input, it consumes an input and
510−-- returns the next intermediate state (see 'Step') or the final result @b@ if
511−-- the fold terminates.
512−--
513−-- If the fold is used as a scan, the @extract@ function is used by the scan
514−-- driver to map the current state @s@ of the fold to the fold result. Thus
515−-- @extract@ can be called multiple times. In some folds, where scanning does
516−-- not make sense, this function is left unimplemented; such folds cannot be
517−-- used as scans.
518−--
519−-- Before a fold terminates, @final@ is called once and only once (unless the
520−-- fold terminated in @initial@ itself). Any resources allocated by @initial@
521−-- can be released in @final@. In folds that do not require any cleanup
522−-- @extract@ and @final@ are typically the same.
523−--
524−-- When implementing fold combinators, care should be taken to cleanup any
525−-- state of the argument folds held by the fold by calling the respective
526−-- @final@ at all exit points of the fold. Also, @final@ should not be called
527−-- more than once. Note that if a fold terminates by 'Done' constructor, there
528−-- is no state to cleanup.
472+-- | The type @Fold m a b@ having constructor @Fold step initial extract@
473+-- represents a fold over an input stream of values of type @a@ to a final
474+-- value of type @b@ in 'Monad' @m@.
475+--
476+-- The fold uses an intermediate state @s@ as accumulator, the type @s@ is
477+-- internal to the specific fold definition. The initial value of the fold
478+-- state @s@ is returned by @initial@. The @step@ function consumes an input
479+-- and either returns the final result @b@ if the fold is done or the next
480+-- intermediate state (see 'Step'). At any point the fold driver can extract
481+-- the result from the intermediate state using the @extract@ function.
529482--
530483-- NOTE: The constructor is not yet released, smart constructors are provided
531484-- to create folds.
532485--
533486data Fold m a b =
534− -- | @Fold@ @step@ @initial@ @extract@ @final@
535− forall s. Fold (s -> a -> m (Step s b)) (m (Step s b)) (s -> m b) (s -> m b)
487+ -- | @Fold @ @ step @ @ initial @ @ extract@
488+ forall s. Fold (s -> a -> m (Step s b)) (m (Step s b)) (s -> m b)
536489
537490------------------------------------------------------------------------------
538491-- Mapping on the output
@@ -542,8 +495,7 @@ data Fold m a b =
542495--
543496{-# INLINE rmapM #-}
544497rmapM :: Monad m => (b -> m c) -> Fold m a b -> Fold m a c
545−rmapM f (Fold step initial extract final) =
546− Fold step1 initial1 (extract >=> f) (final >=> f)
498+rmapM f (Fold step initial extract) = Fold step1 initial1 (extract >=> f)
547499
548500 where
549501
@@ -574,7 +526,6 @@ foldl' step initial =
574526 (\s a -> return $ Partial $ step s a)
575527 (return (Partial initial))
576528 return
577− return
578529
579530-- | Make a fold from a left fold style monadic step function and initial value
580531-- of the accumulator.
@@ -589,7 +540,7 @@ foldl' step initial =
589540{-# INLINE foldlM' #-}
590541foldlM' :: Monad m => (b -> a -> m b) -> m b -> Fold m a b
591542foldlM' step initial =
592− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) return return
543+ Fold (\s a -> Partial <$> step s a) (Partial <$> initial) return
593544
594545-- | Make a strict left fold, for non-empty streams, using first element as the
595546-- starting value. Returns Nothing if the stream is empty.
@@ -687,11 +638,7 @@ foldrM' g z =
687638{-# INLINE foldt' #-}
688639foldt' :: Monad m => (s -> a -> Step s b) -> Step s b -> (s -> b) -> Fold m a b
689640foldt' step initial extract =
690− Fold
691− (\s a -> return $ step s a)
692− (return initial)
693− (return . extract)
694− (return . extract)
641+ Fold (\s a -> return $ step s a) (return initial) (return . extract)
695642
696643-- | Make a terminating fold with an effectful step function and initial state,
697644-- and a state extraction function.
@@ -704,7 +651,7 @@ foldt' step initial extract =
704651--
705652{-# INLINE foldtM' #-}
706653foldtM' :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> Fold m a b
707−foldtM' step initial extract = Fold step initial extract extract
654+foldtM' = Fold
708655
709656------------------------------------------------------------------------------
710657-- Refold
@@ -718,7 +665,7 @@ foldtM' step initial extract = Fold step initial extract extract
718665-- /Internal/
719666fromRefold :: Refold m c a b -> c -> Fold m a b
720667fromRefold (Refold step inject extract) c =
721− Fold step (inject c) extract extract
668+ Fold step (inject c) extract
722669
723670------------------------------------------------------------------------------
724671-- Basic Folds
@@ -771,29 +718,6 @@ toStreamKRev = foldl' (flip K.cons) K.nil
771718toStreamK :: Monad m => Fold m a (K.StreamK n a)
772719toStreamK = foldr K.cons K.nil
773720
774−-- | Like 'length', except with a more general 'Num' return value
775−--
776−-- Definition:
777−--
778−-- >>> lengthGeneric = fmap getSum $ Fold.foldMap (Sum . const 1)
779−-- >>> lengthGeneric = Fold.foldl' (\n _ -> n + 1) 0
780−--
781−-- /Pre-release/
782−{-# INLINE lengthGeneric #-}
783−lengthGeneric :: (Monad m, Num b) => Fold m a b
784−lengthGeneric = foldl' (\n _ -> n + 1) 0
785−
786−-- | Determine the length of the input stream.
787−--
788−-- Definition:
789−--
790−-- >>> length = Fold.lengthGeneric
791−-- >>> length = fmap getSum $ Fold.foldMap (Sum . const 1)
792−--
793−{-# INLINE length #-}
794−length :: Monad m => Fold m a Int
795−length = lengthGeneric
796−
797721------------------------------------------------------------------------------
798722-- Instances
799723------------------------------------------------------------------------------
@@ -801,8 +725,7 @@ length = lengthGeneric
801725-- | Maps a function on the output of the fold (the type @b@).
802726instance Functor m => Functor (Fold m a) where
803727 {-# INLINE fmap #-}
804− fmap f (Fold step1 initial1 extract final) =
805− Fold step initial (fmap2 f extract) (fmap2 f final)
728+ fmap f (Fold step1 initial1 extract) = Fold step initial (fmap2 f extract)
806729
807730 where
808731
@@ -837,7 +760,7 @@ instance Functor m => Functor (Fold m a) where
837760--
838761{-# INLINE fromPure #-}
839762fromPure :: Applicative m => b -> Fold m a b
840−fromPure b = Fold undefined (pure $ Done b) pure pure
763+fromPure b = Fold undefined (pure $ Done b) pure
841764
842765-- | Make a fold that yields the result of the supplied effectful action
843766-- without consuming any further input.
@@ -846,7 +769,7 @@ fromPure b = Fold undefined (pure $ Done b) pure pure
846769--
847770{-# INLINE fromEffect #-}
848771fromEffect :: Applicative m => m b -> Fold m a b
849−fromEffect b = Fold undefined (Done <$> b) pure pure
772+fromEffect b = Fold undefined (Done <$> b) pure
850773
851774{-# ANN type SeqFoldState Fuse #-}
852775data SeqFoldState sl f sr = SeqFoldL !sl | SeqFoldR !f !sr
@@ -871,20 +794,16 @@ data SeqFoldState sl f sr = SeqFoldL !sl | SeqFoldR !f !sr
871794-- complexity, because each composition adds a new branch that each subsequent
872795-- fold's input element has to traverse, therefore, it cannot scale to a large
873796-- number of compositions. After around 100 compositions the performance starts
874−-- dipping rapidly compared to a CPS style implementation.
875−--
876−-- For larger number of compositions you can convert the fold to a parser and
877−-- use ParserK.
797+-- dipping rapidly compared to a CPS style implementation. When you need
798+-- scaling use parser monad instead.
878799--
879800-- /Time: O(n^2) where n is the number of compositions./
880801--
881802{-# INLINE splitWith #-}
882803splitWith :: Monad m =>
883804 (a -> b -> c) -> Fold m x a -> Fold m x b -> Fold m x c
884−splitWith func
885− (Fold stepL initialL _ finalL)
886− (Fold stepR initialR _ finalR) =
887− Fold step initial extract final
805+splitWith func (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
806+ Fold step initial extract
888807
889808 where
890809
@@ -901,18 +820,13 @@ splitWith func
901820 step (SeqFoldL st) a = runL (stepL st a)
902821 step (SeqFoldR f st) a = runR (stepR st a) f
903822
904− -- XXX splitWith should not be used for scanning
905− -- It would rarely make sense and resource tracking and cleanup would be
906− -- expensive. especially when multiple splitWith are chained.
907− extract _ = error "splitWith: cannot be used for scanning"
908−
909− final (SeqFoldR f sR) = fmap f (finalR sR)
910− final (SeqFoldL sL) = do
911− rL <- finalL sL
823+ extract (SeqFoldR f sR) = fmap f (extractR sR)
824+ extract (SeqFoldL sL) = do
825+ rL <- extractL sL
912826 res <- initialR
913827 fmap (func rL)
914828 $ case res of
915− Partial sR -> finalR sR
829+ Partial sR -> extractR sR
916830 Done rR -> return rR
917831
918832{-# DEPRECATED serialWith "Please use \"splitWith\" instead" #-}
@@ -932,8 +846,8 @@ data SeqFoldState_ sl sr = SeqFoldL_ !sl | SeqFoldR_ !sr
932846--
933847{-# INLINE split_ #-}
934848split_ :: Monad m => Fold m x a -> Fold m x b -> Fold m x b
935−split_ (Fold stepL initialL _ finalL) (Fold stepR initialR _ finalR) =
936− Fold step initial extract final
849+split_ (Fold stepL initialL _) (Fold stepR initialR extractR) =
850+ Fold step initial extract
937851
938852 where
939853
@@ -956,16 +870,11 @@ split_ (Fold stepL initialL _ finalL) (Fold stepR initialR _ finalR) =
956870 resR <- stepR st a
957871 return $ first SeqFoldR_ resR
958872
959− -- XXX split_ should not be used for scanning
960− -- See splitWith for more details.
961− extract _ = error "split_: cannot be used for scanning"
962−
963− final (SeqFoldR_ sR) = finalR sR
964− final (SeqFoldL_ sL) = do
965− _ <- finalL sL
873+ extract (SeqFoldR_ sR) = extractR sR
874+ extract (SeqFoldL_ _) = do
966875 res <- initialR
967876 case res of
968− Partial sR -> finalR sR
877+ Partial sR -> extractR sR
969878 Done rR -> return rR
970879
971880-- | 'Applicative' form of 'splitWith'. Split the input serially over two
@@ -1013,10 +922,8 @@ data TeeState sL sR bL bR
1013922--
1014923{-# INLINE teeWith #-}
1015924teeWith :: Monad m => (a -> b -> c) -> Fold m x a -> Fold m x b -> Fold m x c
1016−teeWith f
1017− (Fold stepL initialL extractL finalL)
1018− (Fold stepR initialR extractR finalR) =
1019− Fold step initial extract final
925+teeWith f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
926+ Fold step initial extract
1020927
1021928 where
1022929
@@ -1043,10 +950,6 @@ teeWith f
1043950 extract (TeeLeft bR sL) = (`f` bR) <$> extractL sL
1044951 extract (TeeRight bL sR) = f bL <$> extractR sR
1045952
1046− final (TeeBoth sL sR) = f <$> finalL sL <*> finalR sR
1047− final (TeeLeft bR sL) = (`f` bR) <$> finalL sL
1048− final (TeeRight bL sR) = f bL <$> finalR sR
1049−
1050953{-# ANN type TeeFstState Fuse #-}
1051954data TeeFstState sL sR b
1052955 = TeeFstBoth !sL !sR
@@ -1059,10 +962,8 @@ data TeeFstState sL sR b
1059962{-# INLINE teeWithFst #-}
1060963teeWithFst :: Monad m =>
1061964 (b -> c -> d) -> Fold m a b -> Fold m a c -> Fold m a d
1062−teeWithFst f
1063− (Fold stepL initialL extractL finalL)
1064− (Fold stepR initialR extractR finalR) =
1065− Fold step initial extract final
965+teeWithFst f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
966+ Fold step initial extract
1066967
1067968 where
1068969
@@ -1081,7 +982,7 @@ teeWithFst f
1081982 Done bl -> do
1082983 Done . f bl <$>
1083984 case resR of
1084− Partial sr -> finalR sr
985+ Partial sr -> extractR sr
1085986 Done br -> return br
1086987
1087988 initial = runBoth initialL initialR
@@ -1092,9 +993,6 @@ teeWithFst f
1092993 extract (TeeFstBoth sL sR) = f <$> extractL sL <*> extractR sR
1093994 extract (TeeFstLeft bR sL) = (`f` bR) <$> extractL sL
1094995
1095− final (TeeFstBoth sL sR) = f <$> finalL sL <*> finalR sR
1096− final (TeeFstLeft bR sL) = (`f` bR) <$> finalL sL
1097−
1098996-- | Like 'teeWith' but terminates as soon as any one of the two folds
1099997-- terminates.
1100998--
@@ -1103,10 +1001,8 @@ teeWithFst f
11031001{-# INLINE teeWithMin #-}
11041002teeWithMin :: Monad m =>
11051003 (b -> c -> d) -> Fold m a b -> Fold m a c -> Fold m a d
1106−teeWithMin f
1107− (Fold stepL initialL extractL finalL)
1108− (Fold stepR initialR extractR finalR) =
1109− Fold step initial extract final
1004+teeWithMin f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1005+ Fold step initial extract
11101006
11111007 where
11121008
@@ -1118,12 +1014,12 @@ teeWithMin f
11181014 Partial sl -> do
11191015 case resR of
11201016 Partial sr -> return $ Partial $ Tuple' sl sr
1121− Done br -> Done . (`f` br) <$> finalL sl
1017+ Done br -> Done . (`f` br) <$> extractL sl
11221018
11231019 Done bl -> do
11241020 Done . f bl <$>
11251021 case resR of
1126− Partial sr -> finalR sr
1022+ Partial sr -> extractR sr
11271023 Done br -> return br
11281024
11291025 initial = runBoth initialL initialR
@@ -1132,8 +1028,6 @@ teeWithMin f
11321028
11331029 extract (Tuple' sL sR) = f <$> extractL sL <*> extractR sR
11341030
1135− final (Tuple' sL sR) = f <$> finalL sL <*> finalR sR
1136−
11371031-- | Shortest alternative. Apply both folds in parallel but choose the result
11381032-- from the one which consumed least input i.e. take the shortest succeeding
11391033-- fold.
@@ -1145,8 +1039,8 @@ teeWithMin f
11451039--
11461040{-# INLINE shortest #-}
11471041shortest :: Monad m => Fold m x a -> Fold m x b -> Fold m x (Either a b)
1148−shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
1149− Fold step initial extract final
1042+shortest (Fold stepL initialL extractL) (Fold stepR initialR _) =
1043+ Fold step initial extract
11501044
11511045 where
11521046
@@ -1154,16 +1048,10 @@ shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
11541048 runBoth actionL actionR = do
11551049 resL <- actionL
11561050 resR <- actionR
1157− case resL of
1158− Partial sL ->
1159− case resR of
1160− Partial sR -> return $ Partial $ Tuple' sL sR
1161− Done bR -> finalL sL >> return (Done (Right bR))
1162− Done bL -> do
1163− case resR of
1164− Partial sR -> void (finalR sR)
1165− Done _ -> return ()
1166− return (Done (Left bL))
1051+ return $
1052+ case resL of
1053+ Partial sL -> bimap (Tuple' sL) Right resR
1054+ Done bL -> Done $ Left bL
11671055
11681056 initial = runBoth initialL initialR
11691057
@@ -1171,8 +1059,6 @@ shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
11711059
11721060 extract (Tuple' sL _) = Left <$> extractL sL
11731061
1174− final (Tuple' sL sR) = Left <$> finalL sL <* finalR sR
1175−
11761062{-# ANN type LongestState Fuse #-}
11771063data LongestState sL sR
11781064 = LongestBoth !sL !sR
@@ -1190,10 +1076,8 @@ data LongestState sL sR
11901076--
11911077{-# INLINE longest #-}
11921078longest :: Monad m => Fold m x a -> Fold m x b -> Fold m x (Either a b)
1193−longest
1194− (Fold stepL initialL _ finalL)
1195− (Fold stepR initialR _ finalR) =
1196− Fold step initial extract final
1079+longest (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1080+ Fold step initial extract
11971081
11981082 where
11991083
@@ -1216,18 +1100,14 @@ longest
12161100 step (LongestLeft sL) a = bimap LongestLeft Left <$> stepL sL a
12171101 step (LongestRight sR) a = bimap LongestRight Right <$> stepR sR a
12181102
1219− -- XXX Scan with this may not make sense as we cannot determine the longest
1220− -- until one of them have exhausted.
1221− extract _ = error $ "longest: scan is not allowed as longest cannot be "
1222− ++ "determined until one fold has exhausted."
1223−
1224− final (LongestLeft sL) = Left <$> finalL sL
1225− final (LongestRight sR) = Right <$> finalR sR
1226− final (LongestBoth sL sR) = Left <$> finalL sL <* finalR sR
1103+ left sL = Left <$> extractL sL
1104+ extract (LongestLeft sL) = left sL
1105+ extract (LongestRight sR) = Right <$> extractR sR
1106+ extract (LongestBoth sL _) = left sL
12271107
1228−data ConcatMapState m sa a b c
1229− = B !sa (sa -> m b)
1230− | forall s. C (s -> a -> m (Step s c)) !s (s -> m c) (s -> m c)
1108+data ConcatMapState m sa a c
1109+ = B !sa
1110+ | forall s. C (s -> a -> m (Step s c)) !s (s -> m c)
12311111
12321112-- | Map a 'Fold' returning function on the result of a 'Fold' and run the
12331113-- returned fold. This operation can be used to express data dependencies
@@ -1250,47 +1130,43 @@ data ConcatMapState m sa a b c
12501130--
12511131{-# INLINE concatMap #-}
12521132concatMap :: Monad m => (b -> Fold m a c) -> Fold m a b -> Fold m a c
1253−concatMap f (Fold stepa initiala _ finala) =
1254− Fold stepc initialc extractc finalc
1133+concatMap f (Fold stepa initiala extracta) = Fold stepc initialc extractc
12551134 where
12561135 initialc = do
12571136 r <- initiala
12581137 case r of
1259− Partial s -> return $ Partial (B s finala)
1138+ Partial s -> return $ Partial (B s)
12601139 Done b -> initInnerFold (f b)
12611140
1262− stepc (B s fin) a = do
1141+ stepc (B s) a = do
12631142 r <- stepa s a
12641143 case r of
1265− Partial s1 -> return $ Partial (B s1 fin)
1144+ Partial s1 -> return $ Partial (B s1)
12661145 Done b -> initInnerFold (f b)
12671146
1268− stepc (C stepInner s extractInner fin) a = do
1147+ stepc (C stepInner s extractInner) a = do
12691148 r <- stepInner s a
12701149 return $ case r of
1271− Partial sc -> Partial (C stepInner sc extractInner fin)
1150+ Partial sc -> Partial (C stepInner sc extractInner)
12721151 Done c -> Done c
12731152
1274− -- XXX Cannot use for scanning
1275− extractc _ = error "concatMap: cannot be used for scanning"
1153+ extractc (B s) = do
1154+ r <- extracta s
1155+ initExtract (f r)
1156+ extractc (C _ sInner extractInner) = extractInner sInner
12761157
1277− initInnerFold (Fold step i e fin) = do
1158+ initInnerFold (Fold step i e) = do
12781159 r <- i
12791160 return $ case r of
1280− Partial s -> Partial (C step s e fin)
1161+ Partial s -> Partial (C step s e)
12811162 Done c -> Done c
12821163
1283− initFinalize (Fold _ i _ fin) = do
1164+ initExtract (Fold _ i e) = do
12841165 r <- i
12851166 case r of
1286− Partial s -> fin s
1167+ Partial s -> e s
12871168 Done c -> return c
12881169
1289− finalc (B s fin) = do
1290− r <- fin s
1291− initFinalize (f r)
1292− finalc (C _ sInner _ fin) = fin sInner
1293−
12941170------------------------------------------------------------------------------
12951171-- Mapping on input
12961172------------------------------------------------------------------------------
@@ -1309,7 +1185,7 @@ concatMap f (Fold stepa initiala _ finala) =
13091185--
13101186{-# INLINE lmap #-}
13111187lmap :: (a -> b) -> Fold m b r -> Fold m a r
1312−lmap f (Fold step begin done final) = Fold step' begin done final
1188+lmap f (Fold step begin done) = Fold step' begin done
13131189 where
13141190 step' x a = step x (f a)
13151191
@@ -1317,7 +1193,7 @@ lmap f (Fold step begin done final) = Fold step' begin done final
13171193--
13181194{-# INLINE lmapM #-}
13191195lmapM :: Monad m => (a -> m b) -> Fold m b r -> Fold m a r
1320−lmapM f (Fold step begin done final) = Fold step' begin done final
1196+lmapM f (Fold step begin done) = Fold step' begin done
13211197 where
13221198 step' x a = f a >>= step x
13231199
@@ -1329,10 +1205,8 @@ lmapM f (Fold step begin done final) = Fold step' begin done final
13291205-- /Pre-release/
13301206{-# INLINE postscan #-}
13311207postscan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1332−postscan
1333− (Fold stepL initialL extractL finalL)
1334− (Fold stepR initialR extractR finalR) =
1335− Fold step initial extract final
1208+postscan (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1209+ Fold step initial extract
13361210
13371211 where
13381212
@@ -1343,32 +1217,30 @@ postscan
13431217 Done bL -> do
13441218 rR <- stepR sR bL
13451219 case rR of
1346− Partial sR1 -> Done <$> finalR sR1
1220+ Partial sR1 -> Done <$> extractR sR1
13471221 Done bR -> return $ Done bR
13481222 Partial sL -> do
13491223 !b <- extractL sL
13501224 rR <- stepR sR b
1351− case rR of
1352− Partial sR1 -> return $ Partial (sL, sR1)
1353− Done bR -> finalL sL >> return (Done bR)
1225+ return
1226+ $ case rR of
1227+ Partial sR1 -> Partial (sL, sR1)
1228+ Done bR -> Done bR
13541229
13551230 initial = do
1356− rR <- initialR
1357− case rR of
1358− Partial sR -> do
1359− rL <- initialL
1231+ r <- initialR
1232+ rL <- initialL
1233+ case r of
1234+ Partial sR ->
13601235 case rL of
1361− Done _ -> Done <$> finalR sR
1236+ Done _ -> Done <$> extractR sR
13621237 Partial sL -> return $ Partial (sL, sR)
13631238 Done b -> return $ Done b
13641239
1365− -- XXX should use Tuple'
13661240 step (sL, sR) x = runStep (stepL sL x) sR
13671241
13681242 extract = extractR . snd
13691243
1370− final (sL, sR) = finalL sL *> finalR sR
1371−
13721244------------------------------------------------------------------------------
13731245-- Filtering
13741246------------------------------------------------------------------------------
@@ -1381,7 +1253,7 @@ postscan
13811253--
13821254{-# INLINE_NORMAL catMaybes #-}
13831255catMaybes :: Monad m => Fold m a b -> Fold m (Maybe a) b
1384−catMaybes (Fold step initial extract final) = Fold step1 initial extract final
1256+catMaybes (Fold step initial extract) = Fold step1 initial extract
13851257
13861258 where
13871259
@@ -1421,7 +1293,7 @@ filtering f = foldl' step Nothing
14211293{-# INLINE filter #-}
14221294filter :: Monad m => (a -> Bool) -> Fold m a r -> Fold m a r
14231295-- filter p = scanMaybe (filtering p)
1424−filter f (Fold step begin extract final) = Fold step' begin extract final
1296+filter f (Fold step begin done) = Fold step' begin done
14251297 where
14261298 step' x a = if f a then step x a else return $ Partial x
14271299
@@ -1432,7 +1304,7 @@ filter f (Fold step begin extract final) = Fold step' begin extract final
14321304--
14331305{-# INLINE filterM #-}
14341306filterM :: Monad m => (a -> m Bool) -> Fold m a r -> Fold m a r
1435−filterM f (Fold step begin extract final) = Fold step' begin extract final
1307+filterM f (Fold step begin done) = Fold step' begin done
14361308 where
14371309 step' x a = do
14381310 use <- f a
@@ -1521,7 +1393,7 @@ dropping n = foldt' step initial extract
15211393{-# INLINE take #-}
15221394take :: Monad m => Int -> Fold m a b -> Fold m a b
15231395-- take n = scanMaybe (taking n)
1524−take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
1396+take n (Fold fstep finitial fextract) = Fold step initial extract
15251397
15261398 where
15271399
@@ -1533,7 +1405,7 @@ take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
15331405 s1 = Tuple'Fused i1 s
15341406 if i1 < n
15351407 then return $ Partial s1
1536− else Done <$> ffinal s
1408+ else Done <$> fextract s
15371409 Done b -> return $ Done b
15381410
15391411 initial = finitial >>= next (-1)
@@ -1542,72 +1414,6 @@ take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
15421414
15431415 extract (Tuple'Fused _ r) = fextract r
15441416
1545− final (Tuple'Fused _ r) = ffinal r
1546−
1547−-- Note: Keep this consistent with S.splitOn. In fact we should eliminate
1548−-- S.splitOn in favor of the fold.
1549−--
1550−-- XXX Use Fold.many instead once it is fixed.
1551−-- > Stream.splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
1552−
1553−-- | Like 'takeEndBy' but drops the element on which the predicate succeeds.
1554−--
1555−-- Example:
1556−--
1557−-- >>> input = Stream.fromList "hello\nthere\n"
1558−-- >>> line = Fold.takeEndBy_ (== '\n') Fold.toList
1559−-- >>> Stream.fold line input
1560−-- "hello"
1561−--
1562−-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1563−-- ["hello","there"]
1564−--
1565−{-# INLINE takeEndBy_ #-}
1566−takeEndBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1567−-- takeEndBy_ predicate = scanMaybe (takingEndBy_ predicate)
1568−takeEndBy_ predicate (Fold fstep finitial fextract ffinal) =
1569− Fold step finitial fextract ffinal
1570−
1571− where
1572−
1573− step s a =
1574− if not (predicate a)
1575− then fstep s a
1576− else Done <$> ffinal s
1577−
1578−-- Note:
1579−-- > Stream.splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
1580−
1581−-- | Take the input, stop when the predicate succeeds taking the succeeding
1582−-- element as well.
1583−--
1584−-- Example:
1585−--
1586−-- >>> input = Stream.fromList "hello\nthere\n"
1587−-- >>> line = Fold.takeEndBy (== '\n') Fold.toList
1588−-- >>> Stream.fold line input
1589−-- "hello\n"
1590−--
1591−-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1592−-- ["hello\n","there\n"]
1593−--
1594−{-# INLINE takeEndBy #-}
1595−takeEndBy :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1596−-- takeEndBy predicate = scanMaybe (takingEndBy predicate)
1597−takeEndBy predicate (Fold fstep finitial fextract ffinal) =
1598− Fold step finitial fextract ffinal
1599−
1600− where
1601−
1602− step s a = do
1603− res <- fstep s a
1604− if not (predicate a)
1605− then return res
1606− else do
1607− case res of
1608− Partial s1 -> Done <$> ffinal s1
1609− Done b -> return $ Done b
1610−
16111417------------------------------------------------------------------------------
16121418-- Nesting
16131419------------------------------------------------------------------------------
@@ -1625,8 +1431,8 @@ takeEndBy predicate (Fold fstep finitial fextract ffinal) =
16251431-- /Pre-release/
16261432{-# INLINE duplicate #-}
16271433duplicate :: Monad m => Fold m a b -> Fold m a (Fold m a b)
1628−duplicate (Fold step1 initial1 extract1 final1) =
1629− Fold step initial extract final
1434+duplicate (Fold step1 initial1 extract1) =
1435+ Fold step initial (\s -> pure $ Fold step1 (pure $ Partial s) extract1)
16301436
16311437 where
16321438
@@ -1634,11 +1440,6 @@ duplicate (Fold step1 initial1 extract1 final1) =
16341440
16351441 step s a = second fromPure <$> step1 s a
16361442
1637− -- Scanning may be problematic due to multiple finalizations.
1638− extract = error "duplicate: scanning may be problematic"
1639−
1640− final s = pure $ Fold step1 (pure $ Partial s) extract1 final1
1641−
16421443-- If there were a finalize/flushing action in the stream type that would be
16431444-- equivalent to running initialize in Fold. But we do not have a flushing
16441445-- action in streams.
@@ -1650,9 +1451,9 @@ duplicate (Fold step1 initial1 extract1 final1) =
16501451-- /Pre-release/
16511452{-# INLINE reduce #-}
16521453reduce :: Monad m => Fold m a b -> m (Fold m a b)
1653−reduce (Fold step initial extract final) = do
1454+reduce (Fold step initial extract) = do
16541455 i <- initial
1655− return $ Fold step (return i) extract final
1456+ return $ Fold step (return i) extract
16561457
16571458-- This is the dual of Stream @cons@.
16581459
@@ -1662,8 +1463,7 @@ reduce (Fold step initial extract final) = do
16621463-- /Pre-release/
16631464{-# INLINE snoclM #-}
16641465snoclM :: Monad m => Fold m a b -> m a -> Fold m a b
1665−snoclM (Fold fstep finitial fextract ffinal) action =
1666− Fold fstep initial fextract ffinal
1466+snoclM (Fold fstep finitial fextract) action = Fold fstep initial fextract
16671467
16681468 where
16691469
@@ -1690,8 +1490,7 @@ snoclM (Fold fstep finitial fextract ffinal) action =
16901490{-# INLINE snocl #-}
16911491snocl :: Monad m => Fold m a b -> a -> Fold m a b
16921492-- snocl f = snoclM f . return
1693−snocl (Fold fstep finitial fextract ffinal) a =
1694− Fold fstep initial fextract ffinal
1493+snocl (Fold fstep finitial fextract) a = Fold fstep initial fextract
16951494
16961495 where
16971496
@@ -1711,12 +1510,12 @@ snocl (Fold fstep finitial fextract ffinal) a =
17111510-- /Pre-release/
17121511{-# INLINE snocM #-}
17131512snocM :: Monad m => Fold m a b -> m a -> m (Fold m a b)
1714−snocM (Fold step initial extract final) action = do
1513+snocM (Fold step initial extract) action = do
17151514 res <- initial
17161515 r <- case res of
17171516 Partial fs -> action >>= step fs
17181517 Done _ -> return res
1719− return $ Fold step (return r) extract final
1518+ return $ Fold step (return r) extract
17201519
17211520-- Definitions:
17221521--
@@ -1735,12 +1534,12 @@ snocM (Fold step initial extract final) action = do
17351534-- /Pre-release/
17361535{-# INLINE snoc #-}
17371536snoc :: Monad m => Fold m a b -> a -> m (Fold m a b)
1738−snoc (Fold step initial extract final) a = do
1537+snoc (Fold step initial extract) a = do
17391538 res <- initial
17401539 r <- case res of
17411540 Partial fs -> step fs a
17421541 Done _ -> return res
1743− return $ Fold step (return r) extract final
1542+ return $ Fold step (return r) extract
17441543
17451544-- | Append a singleton value to the fold.
17461545--
@@ -1768,7 +1567,7 @@ addOne = flip snoc
17681567-- /Pre-release/
17691568{-# INLINE extractM #-}
17701569extractM :: Monad m => Fold m a b -> m b
1771−extractM (Fold _ initial extract _) = do
1570+extractM (Fold _ initial extract) = do
17721571 res <- initial
17731572 case res of
17741573 Partial fs -> extract fs
@@ -1777,15 +1576,14 @@ extractM (Fold _ initial extract _) = do
17771576-- | Close a fold so that it does not accept any more input.
17781577{-# INLINE close #-}
17791578close :: Monad m => Fold m a b -> Fold m a b
1780−close (Fold _ initial1 _ final1) =
1781− Fold undefined initial undefined undefined
1579+close (Fold _ initial1 extract1) = Fold undefined initial undefined
17821580
17831581 where
17841582
17851583 initial = do
17861584 res <- initial1
17871585 case res of
1788− Partial s -> Done <$> final1 s
1586+ Partial s -> Done <$> extract1 s
17891587 Done b -> return $ Done b
17901588
17911589-- Corresponds to the null check for streams.
@@ -1795,7 +1593,7 @@ close (Fold _ initial1 _ final1) =
17951593-- /Pre-release/
17961594{-# INLINE isClosed #-}
17971595isClosed :: Monad m => Fold m a b -> m Bool
1798−isClosed (Fold _ initial _ _) = do
1596+isClosed (Fold _ initial _) = do
17991597 res <- initial
18001598 return $ case res of
18011599 Partial _ -> False
@@ -1829,10 +1627,8 @@ data ManyState s1 s2
18291627--
18301628{-# INLINE many #-}
18311629many :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1832−many
1833− (Fold sstep sinitial sextract sfinal)
1834− (Fold cstep cinitial cextract cfinal) =
1835− Fold step initial extract final
1630+many (Fold sstep sinitial sextract) (Fold cstep cinitial cextract) =
1631+ Fold step initial extract
18361632
18371633 where
18381634
@@ -1881,13 +1677,6 @@ many
18811677 Partial s -> cextract s
18821678 Done b -> return b
18831679
1884− final (ManyFirst ss cs) = sfinal ss *> cfinal cs
1885− final (ManyLoop ss cs) = do
1886− cres <- sfinal ss >>= cstep cs
1887− case cres of
1888− Partial s -> cfinal s
1889− Done b -> return b
1890−
18911680-- | Like many, but the "first" fold emits an output at the end even if no
18921681-- input is received.
18931682--
@@ -1897,10 +1686,8 @@ many
18971686--
18981687{-# INLINE manyPost #-}
18991688manyPost :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1900−manyPost
1901− (Fold sstep sinitial sextract sfinal)
1902− (Fold cstep cinitial cextract cfinal) =
1903− Fold step initial extract final
1689+manyPost (Fold sstep sinitial sextract) (Fold cstep cinitial cextract) =
1690+ Fold step initial extract
19041691
19051692 where
19061693
@@ -1936,12 +1723,6 @@ manyPost
19361723 Partial s -> cextract s
19371724 Done b -> return b
19381725
1939− final (Tuple' ss cs) = do
1940− cres <- sfinal ss >>= cstep cs
1941− case cres of
1942− Partial s -> cfinal s
1943− Done b -> return b
1944−
19451726-- | @groupsOf n split collect@ repeatedly applies the @split@ fold to chunks
19461727-- of @n@ items in the input stream and supplies the result to the @collect@
19471728-- fold.
@@ -1970,10 +1751,7 @@ groupsOf n split = many (take n split)
19701751--
19711752{-# INLINE refoldMany #-}
19721753refoldMany :: Monad m => Fold m a b -> Refold m x b c -> Refold m x a c
1973−refoldMany
1974− (Fold sstep sinitial sextract _sfinal)
1975− -- XXX We will need a "final" in refold as well
1976− (Refold cstep cinject cextract) =
1754+refoldMany (Fold sstep sinitial sextract) (Refold cstep cinject cextract) =
19771755 Refold step inject extract
19781756
19791757 where
@@ -2024,9 +1802,7 @@ data ConsumeManyState x cs ss = ConsumeMany x cs (Either ss ss)
20241802-- /Internal/
20251803{-# INLINE refoldMany1 #-}
20261804refoldMany1 :: Monad m => Refold m x a b -> Fold m b c -> Refold m x a c
2027−refoldMany1
2028− (Refold sstep sinject sextract)
2029− (Fold cstep cinitial cextract _cfinal) =
1805+refoldMany1 (Refold sstep sinject sextract) (Fold cstep cinitial cextract) =
20301806 Refold step inject extract
20311807
20321808 where
@@ -2077,7 +1853,7 @@ refoldMany1
20771853{-# INLINE refold #-}
20781854refold :: Monad m => Refold m b a c -> Fold m a b -> Fold m a c
20791855refold (Refold step inject extract) f =
2080− Fold step (extractM f >>= inject) extract extract
1856+ Fold step (extractM f >>= inject) extract
20811857
20821858------------------------------------------------------------------------------
20831859-- morphInner
@@ -2087,8 +1863,8 @@ refold (Refold step inject extract) f =
20871863--
20881864-- /Pre-release/
20891865morphInner :: (forall x. m x -> n x) -> Fold m a b -> Fold n a b
2090−morphInner f (Fold step initial extract final) =
2091− Fold (\x a -> f $ step x a) (f initial) (f . extract) (f . final)
1866+morphInner f (Fold step initial extract) =
1867+ Fold (\x a -> f $ step x a) (f initial) (f . extract)
20921868
20931869-- | Adapt a pure fold to any monad.
20941870--
core/src/Streamly/Internal/Data/Fold/Window.hs+53−59

This diff is too big to show here.

core/src/Streamly/Internal/Data/IOFinalizer.hs+2−12

This diff is too big to show here.

core/src/Streamly/Internal/Data/IORef/Unboxed.hs+20−23

This diff is too big to show here.

core/src/Streamly/Internal/Data/IsMap.hs−8

This diff is too big to show here.

core/src/Streamly/Internal/Data/List.hs+15−81

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutArray.hsdeleted−207

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutArray/Stream.hsdeleted−145

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutByteArray.hsdeleted−251

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutByteArray/Type.hsdeleted−321

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser.hs+143−135

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/Tee.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Tee.hsrenamed+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Type.hsrenamed+71−94

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/ParserDK.hsnew file+1544

This diff is too big to show here.

core/src/Streamly/Internal/Data/ParserK/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserK/Type.hsrenamed+84−291

This diff is too big to show here.

core/src/Streamly/Internal/Data/ParserK.hsdeleted−16

This diff is too big to show here.

core/src/Streamly/Internal/Data/Pipe.hs+8−5

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer.hs+13−6

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer/Source.hs+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer/Type.hs+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Refold/Type.hs+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring.hs+134−589

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring/Generic.hsdeleted−185

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring/Unboxed.hsnew file+616

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize.hsnew file+157

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH.hsdeleted−530

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hsdeleted−475

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/Common.hsdeleted−70

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/RecHeader.hsdeleted−411

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/Type.hsdeleted−317

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream.hs+20−20

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Bottom.hsnew file+670

This diff is too big to show here.

core/src/Streamly/Internal/Data/Array/Stream.hs → core/src/Streamly/Internal/Data/Stream/Chunked.hsrenamed+577−74

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Common.hs → core/src/Streamly/Internal/Data/Stream/Common.hsrenamed+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Cross.hsnew file+143

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Eliminate.hs+227−699

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Enumerate.hsnew file+560

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Exception.hs+90−369

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Expand.hsnew file+893

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Generate.hs+217−948

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Lift.hs+36−70

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Reduce.hsnew file+444

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Container.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Container.hsrenamed+58−66

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Eliminate.hsnew file+843

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Exception.hsnew file+479

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Generate.hsnew file+1190

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Lift.hsnew file+129

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Nesting.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Nesting.hsrenamed+228−115

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Step.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Step.hsrenamed+2−2

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Top.hsnew file+353

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Transform.hsnew file+1945

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Transformer.hsnew file+182

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Type.hsnew file+2085

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamDK.hsnew file+52

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Alt.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Alt.hsrenamed+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Transformer.hsrenamed+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Type.hsrenamed+41−95

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Top.hsdeleted−480

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Transform.hs+282−1197

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Transformer.hs+43−91

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Type.hs+229−1958

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Zip.hsnew file+91

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK.hs+127−435

This diff is too big to show here.

core/src/Streamly/Internal/Data/Time/Clock.hs+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Time/Units.hs+3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unbox.hs+258−313

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unbox/TH.hsdeleted−498

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold.hs+57−19

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold/Enumeration.hs+1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold/Type.hs+8−4

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Dir.hs+2−2

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/File.hs+16−14

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Handle.hs+48−39

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Path.hsdeleted−515

This diff is too big to show here.

core/src/Streamly/Internal/Data/Binary/Parser.hs → core/src/Streamly/Internal/Serialize/FromBytes.hsrenamed+4−10

This diff is too big to show here.

core/src/Streamly/Internal/Data/Binary/Stream.hs → core/src/Streamly/Internal/Serialize/ToBytes.hsrenamed+2−12

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Array.hs−6

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Parser.hs+9−160

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Stream.hs+25−53

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/String.hs+12−17

This diff is too big to show here.

core/src/Streamly/Unicode/Parser.hs+2−11

This diff is too big to show here.

core/src/Streamly/Unicode/Stream.hs−9

This diff is too big to show here.

core/src/Streamly/Unicode/String.hs+1−11

This diff is too big to show here.

core/streamly-core.cabal+94−150

This diff is too big to show here.

docs/Developer/module-organization.md+1−1

This diff is too big to show here.

docs/Developer/optimization-guidelines.md−12

This diff is too big to show here.

docs/Developer/paths.rst+2−25

This diff is too big to show here.

docs/User/HowTo/Compiling.md+3−5

This diff is too big to show here.

docs/User/Project/ApiChangelogs/0.9.0-0.10.0.txtdeleted−424

This diff is too big to show here.

docs/User/Project/CONTRIBUTORS.md−8

This diff is too big to show here.

docs/User/Project/Changelog.md+9−26

This diff is too big to show here.

docs/User/Project/README.md+2−9

This diff is too big to show here.

docs/User/Project/Upgrading-0.8-to-0.9.md → docs/User/Project/Upgrading.mdrenamed+9−87

This diff is too big to show here.

docs/User/Tutorials/quick-overview.md+14−10

This diff is too big to show here.

docs/streamly-docs.cabal+1−1

This diff is too big to show here.

hie.yaml+6−6

This diff is too big to show here.

src/Streamly.hsnew file+624

This diff is too big to show here.

src/Streamly/Data/Fold/Prelude.hs+14−32

This diff is too big to show here.

src/Streamly/Data/Fold/Tee.hs+1−1

This diff is too big to show here.

src/Streamly/Data/Stream/Concurrent.hsnew file+194

This diff is too big to show here.

src/Streamly/Data/Stream/Exception.hsnew file+24

This diff is too big to show here.

src/Streamly/Data/Stream/MkType.hs+10−41

This diff is too big to show here.

src/Streamly/Data/Stream/Prelude.hs+15−217

This diff is too big to show here.

src/Streamly/Data/Stream/Time.hsnew file+37

This diff is too big to show here.

src/Streamly/Data/Unicode/Stream.hsnew file+99

This diff is too big to show here.

src/Streamly/Internal/Data/Array/ArrayMacros.hnew file+40

This diff is too big to show here.

src/Streamly/Internal/Data/Channel.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Async.hs+3−6

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent.hs+44−1

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent/Channel.hs+10−29

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent/Channel/Type.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Prelude.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/SVar.hs+7−8

This diff is too big to show here.

src/Streamly/Internal/Data/IsMap/HashMap.hs−6

This diff is too big to show here.

src/Streamly/Internal/Data/SVar.hs−1

This diff is too big to show here.

src/Streamly/Internal/Data/SmallArray.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Ahead.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Async.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Dispatcher.hs → src/Streamly/Internal/Data/Stream/Channel/Dispatcher.hsrenamed+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Types.hs → src/Streamly/Internal/Data/Stream/Channel/Types.hsrenamed+37−12

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Worker.hs → src/Streamly/Internal/Data/Stream/Channel/Worker.hsrenamed+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent.hs+42−51

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel.hs+14−17

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Append.hs+11−11

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Consumer.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Dispatcher.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Interleave.hs+9−9

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Operations.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Type.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Lifted.hs → src/Streamly/Internal/Data/Stream/Exception/Lifted.hsrenamed+10−14

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream.hs−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Combinators.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Common.hs+6−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Exception.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Expand.hs+3−7

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Generate.hs+6−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Reduce.hs+7−7

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Top.hs+3−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Transform.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Type.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/MkType.hs+34−33

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Parallel.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Prelude.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar.hsdeleted−18

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar/Eliminate.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar/Generate.hs+2−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Serial.hs+5−8

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Time.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Zip.hs+2−41

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Zip/Concurrent.hsnew file+50

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/ZipAsync.hs+2−3

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/Exception.hs+3−1

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/Prelude.hsdeleted−17

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/SVar.hs+2−3

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event.hs+2−5

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Darwin.hs+5−5

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Linux.hs+9−15

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Windows.hs+8−11

This diff is too big to show here.

src/Streamly/Internal/FileSystem/FD.hs+13−14

This diff is too big to show here.

src/Streamly/Internal/Network/Inet/TCP.hs+48−83

This diff is too big to show here.

src/Streamly/Internal/Network/Socket.hs+33−34

This diff is too big to show here.

src/Streamly/Internal/System/IOVec.hs+6−6

This diff is too big to show here.

src/Streamly/Internal/Unicode/Char.hs+7−9

This diff is too big to show here.

src/Streamly/Memory/Array.hsnew file+74

This diff is too big to show here.

src/Streamly/Network/Inet/TCP.hs+30−41

This diff is too big to show here.

src/Streamly/Network/Socket.hs+30−16

This diff is too big to show here.

streamly.cabal+78−104

This diff is too big to show here.

targets/Targets.hs+4−10

This diff is too big to show here.

test/Streamly/Test/Data/Array.hs+10−9

This diff is too big to show here.

test/Streamly/Test/Data/MutArray.hs → test/Streamly/Test/Data/Array/Mut.hsrenamed+5−5

This diff is too big to show here.

test/Streamly/Test/Data/Array/Stream.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Data/Fold.hs+8−8

This diff is too big to show here.

test/Streamly/Test/Data/Fold/Window.hs+9−8

This diff is too big to show here.

test/Streamly/Test/Data/Parser.hs+114−263

This diff is too big to show here.

test/Streamly/Test/Data/ParserK.hs → test/Streamly/Test/Data/Parser/ParserD.hsrenamed+10−14

This diff is too big to show here.

test/Streamly/Test/Data/Ring/Unboxed.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Data/Serialize.hsdeleted−306

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/CompatV0.hsdeleted−46

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/CompatV1.hsdeleted−48

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/TH.hsdeleted−82

This diff is too big to show here.

test/Streamly/Test/Data/Stream/Concurrent.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Data/Unbox.hs+66−164

This diff is too big to show here.

test/Streamly/Test/Data/Unbox/TH.hsdeleted−146

This diff is too big to show here.

test/Streamly/Test/FileSystem/Event/Common.hs+1−1

This diff is too big to show here.

test/Streamly/Test/FileSystem/Handle.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Prelude/Serial.hs+3−7

This diff is too big to show here.

test/Streamly/Test/Prelude/Top.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Serialize/Serializable.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Unicode/Char.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Unicode/Parser.hs+93−130

This diff is too big to show here.

test/Streamly/Test/Unicode/Stream.hs+1−1

This diff is too big to show here.

test/lib/Streamly/Test/Prelude/Common.hs+2−9

This diff is too big to show here.

test/streamly-tests.cabal+16−63

This diff is too big to show here.