git.iko.soy/streamly
on e301ee3 help sign in
30c5a56
swap
0 ahead, 336 behind
269 files changed+9649−16198
since splitsnapshots
unifiedsplit
.circleci/config.yml+277−73 .github/workflows/haskell.yml+91−84 .github/workflows/regression-check.yml+2−2 .hlint.ignore−5 benchmark/NanoBenchmarks.hs+1−1 benchmark/Streamly/Benchmark/Data/Array/CommonImports.hs+1−1 benchmark/Streamly/Benchmark/Data/Array/Stream.hs+3−3 benchmark/Streamly/Benchmark/Data/Fold.hs+4−28 benchmark/Streamly/Benchmark/Data/Fold/Window.hs+40−40 benchmark/Streamly/Benchmark/Data/MutArray.hs+1−1 benchmark/Streamly/Benchmark/Data/Parser.hs+10−10 benchmark/Streamly/Benchmark/Data/ParserK.hs+36−89 benchmark/Streamly/Benchmark/Data/Ring/Unboxed.hs+4−3 benchmark/Streamly/Benchmark/Data/Serialize.hs−591 benchmark/Streamly/Benchmark/Data/Serialize/RecCompatible.hs−24 benchmark/Streamly/Benchmark/Data/Serialize/RecNonCompatible.hs−21 benchmark/Streamly/Benchmark/Data/Serialize/TH.hs−76 benchmark/Streamly/Benchmark/Data/Stream/Common.hs+1−3 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentCommon.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentEager.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentInterleaved.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentOrdered.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Eliminate.hs+1−3 benchmark/Streamly/Benchmark/Data/Stream/Exceptions.hs+3−12 benchmark/Streamly/Benchmark/Data/Stream/Expand.hs+1−9 benchmark/Streamly/Benchmark/Data/Stream/Generate.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Lift.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Reduce.hs+2−2 benchmark/Streamly/Benchmark/Data/Stream/Split.hs+2−2 benchmark/Streamly/Benchmark/Data/Stream/StreamD.hs+2−4 benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs+3−4 benchmark/Streamly/Benchmark/Data/Stream/StreamKAlt.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/ToStreamK.hs+4−3 benchmark/Streamly/Benchmark/Data/Stream/Transform.hs+2−2 benchmark/Streamly/Benchmark/Data/Unbox.hs+132 benchmark/Streamly/Benchmark/Data/Unfold.hs+2−11 benchmark/Streamly/Benchmark/FileSystem/Handle.hs+1−1 benchmark/Streamly/Benchmark/FileSystem/Handle/Read.hs+9−8 benchmark/Streamly/Benchmark/FileSystem/Handle/ReadWrite.hs+10−9 benchmark/Streamly/Benchmark/Prelude/Adaptive.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Ahead.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Async.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Concurrent.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Merge.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Parallel.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Rate.hs+1−1 benchmark/Streamly/Benchmark/Prelude/WAsync.hs+1−1 benchmark/Streamly/Benchmark/Prelude/WSerial.hs+1−1 benchmark/Streamly/Benchmark/Prelude/ZipAsync.hs+1−1 benchmark/Streamly/Benchmark/Prelude/ZipSerial.hs+1−1 benchmark/Streamly/Benchmark/Unicode/Char.hs+1−1 benchmark/Streamly/Benchmark/Unicode/Parser.hs+1−1 benchmark/Streamly/Benchmark/Unicode/Stream.hs+7−25 benchmark/Streamly/Benchmark/Unicode/Utf8.hs+1−1 benchmark/bench-runner/Main.hs+6−34 benchmark/lib/Streamly/Benchmark/Common.hs+1−1 benchmark/lib/Streamly/Benchmark/Common/Handle.hs+1−3 benchmark/lib/Streamly/Benchmark/Prelude.hs+1−1 benchmark/streamly-benchmarks.cabal+41−58 configure+9−9 configure.ac+1−1 core/configure+9−9 core/configure.ac+1−1 core/docs/ApiChangelogs/0.1.0-0.2.0.txt−1471 core/docs/ApiChangelogs/0.2.0-0.2.2.txt−306 core/docs/Changelog.md+12−60 core/src/DocTestDataArray.hs−8 core/src/DocTestDataFold.hs+1 core/src/DocTestDataParserK.hs−15 core/src/DocTestDataStream.hs+1−2 core/src/DocTestFileSystemHandle.hs+1−1 core/src/Streamly/Console/Stdio.hs+27−1 core/src/Streamly/Data/Array.hs+48−48 core/src/Streamly/Data/Array/Generic.hs+13−16 core/src/Streamly/Data/Fold.hs+118−150 core/src/Streamly/Data/MutArray.hs+12−18 core/src/Streamly/Data/MutArray/Generic.hs+3−8 core/src/Streamly/Data/MutByteArray.hs−106 core/src/Streamly/Data/Parser.hs+60−130 core/src/Streamly/Data/ParserK.hs+10−105 core/src/Streamly/Data/Stream.hs+107−199 core/src/Streamly/Data/StreamK.hs+19−37 core/src/Streamly/Data/Unbox.hs+16 core/src/Streamly/Data/Unfold.hs+1−3 core/src/Streamly/FileSystem/Handle.hs+30−7 core/src/Streamly/Internal/Data/Array.hs+163−610 core/src/Streamly/Internal/Data/Array/Generic.hs+11−49 core/src/Streamly/Internal/Data/Array/Type.hs+108−538 core/src/Streamly/Internal/Data/Builder.hs+9−11 core/src/Streamly/Internal/Data/Fold.hs+5−3 core/src/Streamly/Internal/Data/Fold/Chunked.hs+18−20 core/src/Streamly/Internal/Data/Fold/Combinators.hs+234−88 core/src/Streamly/Internal/Data/Fold/Container.hs+31−73 core/src/Streamly/Internal/Data/Fold/Step.hs+2−2 core/src/Streamly/Internal/Data/Fold/Tee.hs−2 core/src/Streamly/Internal/Data/Fold/Type.hs+120−344 core/src/Streamly/Internal/Data/Fold/Window.hs+53−59 core/src/Streamly/Internal/Data/IORef/Unboxed.hs+20−23 core/src/Streamly/Internal/Data/IsMap.hs−8 core/src/Streamly/Internal/Data/List.hs+5−5 core/src/Streamly/Internal/Data/MutArray.hs+33−154 core/src/Streamly/Internal/Data/MutArray/Generic.hs+49−172 core/src/Streamly/Internal/Data/MutArray/Stream.hs−145 core/src/Streamly/Internal/Data/MutArray/Type.hs+441−1308 core/src/Streamly/Internal/Data/MutByteArray.hs−251 core/src/Streamly/Internal/Data/MutByteArray/Type.hs−321 core/src/Streamly/Internal/Data/Parser.hs+143−135 core/src/Streamly/Internal/Data/Parser/Tee.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Tee.hs+1−1 core/src/Streamly/Internal/Data/Parser/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Type.hs+71−94 core/src/Streamly/Internal/Data/Parser/ParserDK.hs+1544 core/src/Streamly/Internal/Data/ParserK/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserK/Type.hs+83−288 core/src/Streamly/Internal/Data/ParserK.hs−16 core/src/Streamly/Internal/Data/Pipe.hs+8−5 core/src/Streamly/Internal/Data/Producer.hs+13−6 core/src/Streamly/Internal/Data/Producer/Source.hs+1−1 core/src/Streamly/Internal/Data/Producer/Type.hs+1−1 core/src/Streamly/Internal/Data/Ring.hs+134−589 core/src/Streamly/Internal/Data/Ring/Generic.hs−185 core/src/Streamly/Internal/Data/Ring/Unboxed.hs+616 core/src/Streamly/Internal/Data/Serialize.hs+157 core/src/Streamly/Internal/Data/Serialize/TH.hs−530 core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hs−475 core/src/Streamly/Internal/Data/Serialize/TH/Common.hs−70 core/src/Streamly/Internal/Data/Serialize/TH/RecHeader.hs−411 core/src/Streamly/Internal/Data/Serialize/Type.hs−317 core/src/Streamly/Internal/Data/Stream.hs+20−20 core/src/Streamly/Internal/Data/Array/Stream.hs → core/src/Streamly/Internal/Data/Stream/Chunked.hs+587−74 src/Streamly/Internal/Data/Stream/Common.hs → core/src/Streamly/Internal/Data/Stream/Common.hs+3−3 core/src/Streamly/Internal/Data/Stream/MutChunked.hs+323 core/src/Streamly/Internal/Data/Stream/Container.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Container.hs+57−65 core/src/Streamly/Internal/Data/Stream/Eliminate.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Eliminate.hs+33−39 core/src/Streamly/Internal/Data/Stream/Exception.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Exception.hs+9−12 core/src/Streamly/Internal/Data/Stream/Generate.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Generate.hs+26−27 core/src/Streamly/Internal/Data/Stream/Lift.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Lift.hs+3−3 core/src/Streamly/Internal/Data/Stream/Nesting.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Nesting.hs+228−115 core/src/Streamly/Internal/Data/Stream/Step.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Step.hs+2−2 core/src/Streamly/Internal/Data/Stream/StreamD/Top.hs+353 core/src/Streamly/Internal/Data/Stream/Transform.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Transform.hs+40−66 core/src/Streamly/Internal/Data/Stream/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Transformer.hs+4−5 core/src/Streamly/Internal/Data/Stream/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Type.hs+34−169 core/src/Streamly/Internal/Data/StreamK/Alt.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Alt.hs+1−1 core/src/Streamly/Internal/Data/StreamK/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Transformer.hs+3−3 core/src/Streamly/Internal/Data/StreamK/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Type.hs+41−95 core/src/Streamly/Internal/Data/Stream/Top.hs−480 core/src/Streamly/Internal/Data/StreamK.hs+134−350 core/src/Streamly/Internal/Data/Time/Clock.hs+3−3 core/src/Streamly/Internal/Data/Time/Units.hs+3 core/src/Streamly/Internal/Data/Unbox.hs+260−195 core/src/Streamly/Internal/Data/Unbox/TH.hs+82−85 core/src/Streamly/Internal/Data/Unfold.hs+58−20 core/src/Streamly/Internal/Data/Unfold/Enumeration.hs+1 core/src/Streamly/Internal/Data/Unfold/Type.hs+8−4 core/src/Streamly/Internal/FileSystem/File.hs+13−11 core/src/Streamly/Internal/FileSystem/Handle.hs+30−23 core/src/Streamly/Internal/FileSystem/Path.hs−515 core/src/Streamly/Internal/Data/Binary/Parser.hs → core/src/Streamly/Internal/Serialize/FromBytes.hs+4−10 core/src/Streamly/Internal/Data/Binary/Stream.hs → core/src/Streamly/Internal/Serialize/ToBytes.hs+2−12 core/src/Streamly/Internal/Unicode/Parser.hs−3 core/src/Streamly/Internal/Unicode/Stream.hs+15−43 core/src/Streamly/Internal/Unicode/String.hs+7−7 core/streamly-core.cabal+75−137 docs/Developer/optimization-guidelines.md−12 docs/Developer/paths.rst+2−25 docs/User/HowTo/Compiling.md+3−5 docs/User/Project/ApiChangelogs/0.9.0-0.10.0.txt−424 docs/User/Project/CONTRIBUTORS.md−8 docs/User/Project/Changelog.md+10−25 docs/User/Project/README.md+2−9 docs/User/Project/Upgrading-0.8-to-0.9.md → docs/User/Project/Upgrading.md+9−87 docs/User/Tutorials/quick-overview.md+14−10 docs/streamly-docs.cabal+1−1 hie.yaml+2−2 src/Streamly.hs+625 src/Streamly/Data/Fold/Prelude.hs+13−31 src/Streamly/Data/Fold/Tee.hs+1−1 src/Streamly/Data/Stream/Concurrent.hs+194 src/Streamly/Data/Stream/Exception.hs+24 src/Streamly/Data/Stream/MkType.hs+10−41 src/Streamly/Data/Stream/Prelude.hs+15−217 src/Streamly/Data/Stream/Time.hs+37 src/Streamly/Data/Unicode/Stream.hs+99 src/Streamly/Internal/Data/Array/ArrayMacros.h+40 src/Streamly/Internal/Data/Channel.hs−19 src/Streamly/Internal/Data/Fold/Async.hs+2−5 src/Streamly/Internal/Data/Fold/Concurrent.hs+44−1 src/Streamly/Internal/Data/Fold/Concurrent/Channel.hs+10−29 src/Streamly/Internal/Data/Fold/Concurrent/Channel/Type.hs+3−3 src/Streamly/Internal/Data/Fold/Prelude.hs−19 src/Streamly/Internal/Data/Fold/SVar.hs+5−6 src/Streamly/Internal/Data/IsMap/HashMap.hs−6 src/Streamly/Internal/Data/SVar.hs−1 src/Streamly/Internal/Data/SmallArray.hs+2−2 src/Streamly/Internal/Data/Stream/Ahead.hs+2−2 src/Streamly/Internal/Data/Stream/Async.hs+2−2 src/Streamly/Internal/Data/Channel/Dispatcher.hs → src/Streamly/Internal/Data/Stream/Channel/Dispatcher.hs+3−3 src/Streamly/Internal/Data/Channel/Types.hs → src/Streamly/Internal/Data/Stream/Channel/Types.hs+37−12 src/Streamly/Internal/Data/Channel/Worker.hs → src/Streamly/Internal/Data/Stream/Channel/Worker.hs+3−3 src/Streamly/Internal/Data/Stream/Concurrent.hs+42−51 src/Streamly/Internal/Data/Stream/Concurrent/Channel.hs+14−17 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Append.hs+11−11 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Consumer.hs+2−2 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Dispatcher.hs+3−3 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Interleave.hs+9−9 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Operations.hs+2−2 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Type.hs+4−4 src/Streamly/Internal/Data/Stream/Lifted.hs → src/Streamly/Internal/Data/Stream/Exception/Lifted.hs+2−2 src/Streamly/Internal/Data/Stream/IsStream.hs−6 src/Streamly/Internal/Data/Stream/IsStream/Combinators.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Common.hs+5−5 src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs+2−2 src/Streamly/Internal/Data/Stream/IsStream/Exception.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Expand.hs+3−7 src/Streamly/Internal/Data/Stream/IsStream/Generate.hs+6−6 src/Streamly/Internal/Data/Stream/IsStream/Reduce.hs+6−6 src/Streamly/Internal/Data/Stream/IsStream/Top.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Transform.hs+2−2 src/Streamly/Internal/Data/Stream/IsStream/Type.hs+1−1 src/Streamly/Internal/Data/Stream/MkType.hs+34−33 src/Streamly/Internal/Data/Stream/Parallel.hs+1−1 src/Streamly/Internal/Data/Stream/Prelude.hs−19 src/Streamly/Internal/Data/Stream/SVar.hs−18 src/Streamly/Internal/Data/Stream/SVar/Eliminate.hs+1−1 src/Streamly/Internal/Data/Stream/SVar/Generate.hs+1−3 src/Streamly/Internal/Data/Stream/Serial.hs+2−7 src/Streamly/Internal/Data/Stream/Time.hs+2−2 src/Streamly/Internal/Data/Stream/Zip.hs+2−41 src/Streamly/Internal/Data/Stream/Zip/Concurrent.hs+50 src/Streamly/Internal/Data/Stream/ZipAsync.hs+2−3 src/Streamly/Internal/Data/Unfold/Exception.hs+3−1 src/Streamly/Internal/Data/Unfold/Prelude.hs−17 src/Streamly/Internal/Data/Unfold/SVar.hs+1−2 src/Streamly/Internal/FileSystem/Event.hs+2−5 src/Streamly/Internal/FileSystem/Event/Darwin.hs+5−5 src/Streamly/Internal/FileSystem/Event/Linux.hs+9−15 src/Streamly/Internal/FileSystem/Event/Windows.hs+8−11 src/Streamly/Internal/FileSystem/FD.hs+10−11 src/Streamly/Internal/Network/Inet/TCP.hs+46−81 src/Streamly/Internal/Network/Socket.hs+30−33 src/Streamly/Internal/System/IOVec.hs+3−3 src/Streamly/Internal/Unicode/Char.hs+7−9 src/Streamly/Memory/Array.hs+74 src/Streamly/Network/Inet/TCP.hs+30−41 src/Streamly/Network/Socket.hs+30−16 streamly.cabal+62−84 targets/Targets.hs+3−8 test/Streamly/Test/Data/Array.hs+8−7 test/Streamly/Test/Data/Array/Stream.hs+1−1 test/Streamly/Test/Data/Fold.hs+6−7 test/Streamly/Test/Data/Fold/Window.hs+9−8 test/Streamly/Test/Data/MutArray.hs+1−1 test/Streamly/Test/Data/Parser.hs+13−18 test/Streamly/Test/Data/ParserK.hs+9−13 test/Streamly/Test/Data/Ring/Unboxed.hs+2−2 test/Streamly/Test/Data/Serialize.hs+13−251 test/Streamly/Test/Data/Serialize/CompatV0.hs−46 test/Streamly/Test/Data/Serialize/CompatV1.hs−48 test/Streamly/Test/Data/Serialize/TH.hs−82 test/Streamly/Test/Data/Stream/Concurrent.hs+1−1 test/Streamly/Test/Data/Unbox.hs+26−125 test/Streamly/Test/Data/Unbox/TH.hs+10−11 test/Streamly/Test/FileSystem/Event/Common.hs+1−1 test/Streamly/Test/FileSystem/Handle.hs+2−2 test/Streamly/Test/Prelude/Serial.hs+3−7 test/Streamly/Test/Serialize/Serializable.hs+2−2 test/Streamly/Test/Unicode/Char.hs+1−1 test/Streamly/Test/Unicode/Parser.hs+16−6 test/Streamly/Test/Unicode/Stream.hs+1−1 test/lib/Streamly/Test/Prelude/Common.hs+2−9 test/streamly-tests.cabal+11−38
.circleci/config.yml+277−73
@@ -1,7 +1,7 @@
11version: 2.1
22
33#-----------------------------------------------------------------------------
4−# packcheck-0.7.0
4+# packcheck-0.4.2
55# Packcheck global environment variables
66#-----------------------------------------------------------------------------
77
@@ -11,8 +11,6 @@ env: &env
1111 # Common options
1212 # ------------------------------------------------------------------------
1313 # GHC_OPTIONS: "-Werror"
14− # For updating see: https://downloads.haskell.org/~ghcup/
15− GHCUP_VERSION: 0.1.20.0
1614 CABAL_REINIT_CONFIG: "y"
1715 LC_ALL: "C.UTF-8"
1816
@@ -24,6 +22,7 @@ env: &env
2422 # DISABLE_DOCS: "y"
2523 # DISABLE_SDIST_BUILD: "y"
2624 # DISABLE_DIST_CHECKS: "y"
25+ ENABLE_INSTALL: "y"
2726
2827 # ------------------------------------------------------------------------
2928 # stack options
@@ -39,12 +38,15 @@ env: &env
3938 # cabal options
4039 # ------------------------------------------------------------------------
4140 CABAL_CHECK_RELAX: "y"
41+ CABAL_NO_SANDBOX: "y"
42+ CABAL_HACKAGE_MIRROR: "hackage.haskell.org:http://hackage.fpcomplete.com"
4243 CABAL_BUILD_OPTIONS: "-j1 --flag limit-build-mem"
4344
4445 # ------------------------------------------------------------------------
4546 # Where to find the required tools
4647 # ------------------------------------------------------------------------
47− PATH: /sbin:/usr/sbin:/bin:/usr/bin
48+ PATH: /opt/ghc/bin:/opt/ghcjs/bin:/sbin:/usr/sbin:/bin:/usr/bin
49+ TOOLS_DIR: /opt
4850
4951 # ------------------------------------------------------------------------
5052 # Location of packcheck.sh (the shell script invoked to perform CI tests ).
@@ -57,15 +59,15 @@ env: &env
5759 # If you have not committed packcheck.sh in your repo at PACKCHECK
5860 # then it is automatically pulled from this URL.
5961 PACKCHECK_GITHUB_URL: "https://raw.githubusercontent.com/composewell/packcheck"
60− PACKCHECK_GITHUB_COMMIT: "f6cfd7b7a33b4bf6fd63cce6a3fb3d26f5031403"
62+ PACKCHECK_GITHUB_COMMIT: "a302772c8f3791c477edd1764b82f8d1f5b22fdf"
6163
6264executors:
6365 amd64-executor:
6466 docker:
65− - image: ubuntu:latest
67+ - image: ubuntu:xenial
6668 x86-executor:
6769 docker:
68− - image: i386/ubuntu:focal
70+ - image: i386/ubuntu:eoan
6971
7072#-----------------------------------------------------------------------------
7173# Common utility stuff, not to be modified usually
@@ -77,50 +79,38 @@ preinstall: &preinstall
7779 # required for https/cache save and restore
7880 apt-get install -y ca-certificates
7981
80− # Alternative way of installing ghc and cabal, directly from
81− # haskell.org instead of using ghcup. NOTE: this is for Debian
82− # only and is debian release specific.
83− # gnupg is required for apt-key to work
84− #apt-get install -y gnupg
85− #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574
86− #echo "deb http://downloads.haskell.org/debian buster main" >> /etc/apt/sources.list
87− #apt-get update
88−
89− # required for outbound https for stack and for stack setup
90− apt-get install -y netbase xz-utils make
91− apt-get install -y zlib1g-dev
82+ # required to (re)generate the configure script
83+ apt-get install -y autoconf
9284
93− # For ghcup to install ghc
94− if test -n "$GHCUP_VERSION"
95− then
96− apt-get install -y gcc
97− apt-get install -y g++
98− fi
85+ # For ghc and cabal-install packages from hvr's ppa
86+ # gnupg is required for apt-key to work
87+ apt-get install -y gnupg
88+ apt-get install -y apt-transport-https
89+ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574
90+ apt-get install -y software-properties-common
91+ add-apt-repository -y ppa:hvr/ghc
92+ # echo "deb http://downloads.haskell.org/debian stretch main" >> /etc/apt/sources.list
93+ echo "deb-src http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main" >> /etc/apt/sources.list
94+ apt-get update
9995
100− # libgmp required by ghc for linking
101− apt-get install -y libgmp-dev
102− apt-get install -y libtinfo-dev
96+ # required for packcheck
97+ apt-get install -y curl
10398
104− # Required by cabal when git URL is specified in project file
99+ # required for circleci.
100+ apt-get install -y libcurl4-gnutls-dev
105101 apt-get install -y git
102+ apt-get install -y libtinfo-dev
106103
107− # Required for and by packcheck
108− apt-get install -y curl
104+ # required for outbound https for stack and for stack setup
105+ apt-get install -y netbase xz-utils make
109106
107+project-preinstall: &project-preinstall
108+ run: |
110109 # Get packcheck if needed
111− if test ! -e "$PACKCHECK"
112− then
113− if test -z "$PACKCHECK_GITHUB_COMMIT"
114− then
115− die "PACKCHECK_GITHUB_COMMIT is not specified."
116− fi
117− PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
118− curl --fail -sL -o "$PACKCHECK" $PACKCHECK_URL || exit 1
119− chmod +x $PACKCHECK
120− elif test ! -x "$PACKCHECK"
121− then
122− chmod +x $PACKCHECK
123− fi
110+ CURL=$(which curl)
111+ PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
112+ if test ! -e "$PACKCHECK"; then $CURL -sL -o "$PACKCHECK" $PACKCHECK_URL; fi;
113+ chmod +x $PACKCHECK
124114
125115restore: &restore
126116 # Needs to happen after installing ca-certificates
@@ -136,71 +126,271 @@ save: &save
136126 save_cache:
137127 key: v1-{{ .Environment.CIRCLE_JOB }}-{{ .Branch }}
138128 paths:
139− - ~/.local
140129 - ~/.cabal
130+ - ~/.ghc
131+ - ~/.local
141132 - ~/.stack
142− - ~/.ghcup
143133
144134#-----------------------------------------------------------------------------
145135# Build matrix
146136#-----------------------------------------------------------------------------
147137
148138jobs:
149− cabal-ghc-8_6_5:
139+ cabal-ghc-8_10_4:
150140 <<: *env
151141 executor: amd64-executor
152142 steps:
143+ - *preinstall
153144 - checkout
145+ - *project-preinstall
154146 - *restore
155− - *preinstall
156147 - run:
157148 environment:
158− GHCVER: "8.6.5"
159− CABAL_VERSION: "3.6.2.0"
160149 CABAL_PROJECT: "cabal.project"
150+ DISABLE_TEST: "yes"
151+ DISABLE_BENCH: "yes"
161152 DISABLE_SDIST_BUILD: "yes"
162− CABAL_BUILD_OPTIONS: "--flag debug --flag -opt"
163153 command: |
154+ apt-get install -y ghc-8.10.4
155+ apt-get install -y cabal-install-3.4
164156 bash -c "$PACKCHECK cabal"
165157 - *save
166− cabal-ghc-9_8_1-docspec:
158+ stack-ghc-8_8:
167159 <<: *env
168160 executor: amd64-executor
169161 steps:
162+ - *preinstall
170163 - checkout
164+ - *project-preinstall
171165 - *restore
166+ - run:
167+ name: GHC 8.8 + stack lts-16.12
168+ environment:
169+ BUILD: "stack"
170+ RESOLVER: "lts-16.12"
171+ SDIST_OPTIONS: "--ignore-check"
172+ STACK_BUILD_OPTIONS: "--flag streamly-benchmarks:-opt"
173+ command: |
174+ apt-get build-dep -y ghc-8.8.4
175+ apt-get install -y cabal-install-3.2
176+ bash -c "$PACKCHECK $BUILD"
177+ - *save
178+ cabal-ghc-8_6_5:
179+ <<: *env
180+ executor: amd64-executor
181+ steps:
172182 - *preinstall
183+ - checkout
184+ - *project-preinstall
185+ - *restore
186+ - run: |
187+ apt-get install -y ghc-8.6.3
188+ apt-get install -y cabal-install-2.4
189+ bash -c "$PACKCHECK cabal-v2"
190+ - *save
191+ cabal-ghc-8_4_4:
192+ <<: *env
193+ executor: x86-executor
194+ steps:
195+ - *preinstall
196+ - checkout
197+ - *project-preinstall
198+ - *restore
199+ - run:
200+ name: GHC 8.4.4 + x86 + debug
201+ environment:
202+ BUILD: "cabal-v2"
203+ GHCVER: "8.4.4"
204+ CABAL_BUILD_OPTIONS: "--flag debug --flag limit-build-mem"
205+ command: |
206+ apt-get install -y ghc-8.4.4
207+ apt-get install -y cabal-install-3.2
208+ bash -c "$PACKCHECK $BUILD"
209+ - *save
210+ cabal-ghc-8_2_2:
211+ <<: *env
212+ executor: amd64-executor
213+ steps:
214+ - *preinstall
215+ - checkout
216+ - *project-preinstall
217+ - *restore
173218 - run:
174219 environment:
175− GHCVER: "9.8.1"
176− CABAL_VERSION: "3.10.1.0"
177− CABAL_PROJECT: "cabal.project.doctest"
178− DISABLE_SDIST_BUILD: "y"
179− DISABLE_TEST: "y"
180− DISABLE_BENCH: "y"
181− DISABLE_DOCS: "y"
182− ENABLE_DOCSPEC: "y"
183− DOCSPEC_URL: https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz
184− DOCSPEC_OPTIONS: "--timeout 60 --check-properties --property-variables xs"
220+ BUILD: "cabal-v2"
221+ DISABLE_TEST: "yes"
222+ DISABLE_BENCH: "yes"
223+ DISABLE_DOCS: "yes"
224+ DISABLE_SDIST_BUILD: "yes"
225+ DISABLE_DIST_CHECKS: "yes"
185226 command: |
186− bash -c "$PACKCHECK cabal"
227+ apt-get install -y ghc-8.2.2
228+ apt-get install -y cabal-install-3.2
229+ bash -c "$PACKCHECK $BUILD"
187230 - *save
188− hlint-trailing-spaces:
231+ cabal-ghc-8_0_2:
189232 <<: *env
190233 executor: amd64-executor
191234 steps:
235+ - *preinstall
192236 - checkout
237+ - *project-preinstall
193238 - *restore
239+ - run:
240+ environment:
241+ BUILD: "cabal-v2"
242+ DISABLE_TEST: "yes"
243+ DISABLE_BENCH: "yes"
244+ DISABLE_DOCS: "yes"
245+ DISABLE_SDIST_BUILD: "yes"
246+ DISABLE_DIST_CHECKS: "yes"
247+ command: |
248+ apt-get install -y ghc-8.0.2
249+ apt-get install -y cabal-install-3.2
250+ bash -c "$PACKCHECK $BUILD"
251+ - *save
252+ cabal-ghc-7_10_3:
253+ <<: *env
254+ executor: amd64-executor
255+ steps:
194256 - *preinstall
257+ - checkout
258+ - *project-preinstall
259+ - *restore
260+ - run: |
261+ apt-get install -y ghc-7.10.3
262+ apt-get install -y cabal-install-2.4
263+ bash -c "$PACKCHECK cabal-v2"
264+ - *save
265+ stack-ghc-8_4:
266+ <<: *env
267+ executor: amd64-executor
268+ steps:
269+ - *preinstall
270+ - checkout
271+ - *project-preinstall
272+ - *restore
273+ - run: |
274+ bash -c "$PACKCHECK stack RESOLVER=lts-12"
275+ - *save
276+ stack-ghc-8_2:
277+ <<: *env
278+ executor: amd64-executor
279+ steps:
280+ - *preinstall
281+ - checkout
282+ - *project-preinstall
283+ - *restore
284+ - run: |
285+ bash -c "$PACKCHECK stack RESOLVER=lts-11"
286+ - *save
287+ cabal-ghcjs-8_4:
288+ <<: *env
289+ executor: amd64-executor
290+ steps:
291+ - *preinstall
292+ - checkout
293+ - *project-preinstall
294+ - *restore
195295 - run:
196− name: hlint and trailing spaces
296+ name: GHCJS 8.4+no-test+no-docs
197297 environment:
298+ BUILD: cabal-v2
299+ ENABLE_GHCJS: "yes"
300+ DISABLE_TEST: "yes"
301+ ENABLE_INSTALL: ""
302+ DISABLE_DOCS: "yes"
198303 DISABLE_SDIST_BUILD: "yes"
199− HLINT_VERSION: 3.6.1
304+ command: |
305+ add-apt-repository -y ppa:hvr/ghcjs
306+ add-apt-repository -y 'deb https://deb.nodesource.com/node_11.x xenial main'
307+ curl -sSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -
308+ apt-get update
309+ apt-get install -y cabal-install-3.2
310+ apt-get install -y nodejs
311+ apt-get install -y ghcjs-8.4
312+ cabal user-config update -a "jobs: 1"
313+ bash -c "$PACKCHECK $BUILD"
314+ - *save
315+ coveralls-ghc-8_2_2:
316+ <<: *env
317+ executor: amd64-executor
318+ steps:
319+ - *preinstall
320+ - checkout
321+ - *project-preinstall
322+ - *restore
323+ - run:
324+ environment:
325+ BUILD: cabal
326+ COVERALLS_OPTIONS: "--repo-token=\"$REPO_TOKEN\" --coverage-mode=StrictlyFullLines --exclude-dir=test test"
327+ GHC_OPTIONS: "-DCOVERAGE_BUILD"
328+ command: |
329+ apt-get install -y ghc-8.2.2
330+ apt-get install -y cabal-install-2.4
331+ bash -c "$PACKCHECK cabal-v1"
332+ - *save
333+ coveralls-ghc-8_8_3:
334+ <<: *env
335+ executor: amd64-executor
336+ steps:
337+ - *preinstall
338+ - checkout
339+ - *project-preinstall
340+ - *restore
341+ - run:
342+ environment:
343+ BUILD: cabal-v2
344+ COVERALLS_OPTIONS: "--repo-token=\"$REPO_TOKEN\" --coverage-mode=StrictlyFullLines --exclude-dir=test"
345+ CABAL_PROJECT: "cabal.project.coverage"
346+ CABAL_BUILD_TARGETS: ""
347+ command: |
348+ apt-get install -y ghc-8.8.3
349+ apt-get install -y cabal-install-3.0
350+ bash -c "$PACKCHECK cabal-v2"
351+ no_output_timeout: 25m
352+ - *save
353+ coveralls-ghc-8_10_2:
354+ <<: *env
355+ executor: amd64-executor
356+ steps:
357+ - *preinstall
358+ - checkout
359+ - *project-preinstall
360+ - *restore
361+ - run:
362+ name: Build and test with inspection, coverage, Werror
363+ command: |
364+ apt-get install -y ghc-8.10.2
365+ apt-get install -y cabal-install-3.2
366+ # Run tests with coverage
367+ cabal update
368+ cabal user-config update -a "jobs: 1"
369+ bin/test.sh --coverage --raw
370+ # Upload results to coveralls.io
371+ PATH=$HOME/.cabal/bin:$PATH
372+ export PATH
373+ which hpc-coveralls 2>/dev/null || cabal install --project-file cabal.project.hpc-coveralls hpc-coveralls
374+ hpc-coveralls --repo-token="$REPO_TOKEN" --coverage-mode=StrictlyFullLines
375+ no_output_timeout: 25m
376+ - *save
377+ hlint-src:
378+ <<: *env
379+ executor: amd64-executor
380+ steps:
381+ - *preinstall
382+ - checkout
383+ - *project-preinstall
384+ - *restore
385+ - run:
386+ name: Hlint src
387+ environment:
388+ BUILD: hlint
200389 HLINT_OPTIONS: lint
390+ DISABLE_SDIST_BUILD: "yes"
201391 HLINT_TARGETS: core/src src test benchmark
202392 command: |
203− bash -c "$PACKCHECK hlint" || exit 1
393+ bash -c "$PACKCHECK $BUILD" || exit 1
204394 echo "Checking trailing spaces..."
205395 count=$(find . -name "*.hs" -exec grep -H '\ $' {} \; | tee /dev/tty | wc -l)
206396 exit $count
@@ -210,9 +400,23 @@ workflows:
210400 version: 2
211401 build:
212402 jobs:
213− #- cabal-ghc-8_6_5:
214− # name: 8.6.5-debug-unoptimized
215− - cabal-ghc-9_8_1-docspec:
216− name: ghc-9.8.1-docspec
217− - hlint-trailing-spaces:
218− name: hlint and trailing spaces
403+ - cabal-ghc-8_10_4:
404+ name: GHC 8.10.4 + sdist + no-test + no-bench
405+ #- cabal-ghc-8.6.5
406+ #- cabal-ghc-8_4_4:
407+ # name: GHC 8.4.4 + x86 + debug
408+ #- cabal-ghc-8_2_2:
409+ # name: GHC 8.2.2 + no-test + no-bench + no-docs
410+ #- cabal-ghc-8_0_2:
411+ # name: GHC 8.0.2 + no-test + no-bench + no-docs
412+ #- cabal-ghc-7.10.3
413+ # - cabal-ghcjs-8_4:
414+ # name: GHCJS 8.4 + no-test + no-docs
415+ #- stack-ghc-8.4
416+ #- stack-ghc-8.2
417+ #- coveralls-ghc-8.2.2
418+ #- coveralls-ghc-8.8.3
419+ #- coveralls-ghc-8_10_2:
420+ # name: GHC 8.10.2 + inspection + coverage + Werror
421+ - hlint-src:
422+ name: Hlint src + Trailing Spaces
.github/workflows/haskell.yml+91−84
@@ -13,15 +13,9 @@ jobs:
1313 # packcheck environment variables
1414 LC_ALL: C.UTF-8
1515 BUILD: ${{ matrix.build }}
16− # For updating see: https://downloads.haskell.org/~ghcup/
17− GHCUP_VERSION: 0.1.20.0
18− DOCSPEC_URL: https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz
19− DOCSPEC_OPTIONS: "--timeout 60 --check-properties --property-variables xs"
20− GHCUP_GHC_OPTIONS: ${{ matrix.ghcup_ghc_options }}
2116 GHCVER: ${{ matrix.ghc_version }}
2217 CABALVER: ${{ matrix.cabal_version }}
2318 DISABLE_DOCS: ${{ matrix.disable_docs }}
24− ENABLE_DOCSPEC: ${{ matrix.enable_docspec }}
2519 DISABLE_TEST: ${{ matrix.disable_test }}
2620 DISABLE_BENCH: ${{ matrix.disable_bench }}
2721 DISABLE_DIST_CHECKS: ${{ matrix.disable_dist_checks }}
@@ -45,18 +39,14 @@ jobs:
4539 STACK_BUILD_OPTIONS: ${{ matrix.stack_build_options }}
4640
4741 # packcheck location and revision
48− PACKCHECK: "./packcheck.sh"
42+ PACKCHECK_LOCAL_PATH: "./packcheck.sh"
4943 PACKCHECK_GITHUB_URL: "https://raw.githubusercontent.com/composewell/packcheck"
50− PACKCHECK_GITHUB_COMMIT: "v0.7.0"
44+ PACKCHECK_GITHUB_COMMIT: "a302772c8f3791c477edd1764b82f8d1f5b22fdf"
5145
5246 # Pull token from "secrets" setting of the github repo
5347 COVERALLS_TOKEN: ${{ secrets.COVERALLS_TOKEN }}
5448 COVERAGE: ${{ matrix.coverage }}
55−
56− # hlint
57− HLINT_VERSION: 3.6.1
58− HLINT_OPTIONS: "lint"
59− HLINT_TARGETS: "core/src src test benchmark"
49+ DOCTEST: ${{ matrix.doctest }}
6050
6151 # Subdir
6252 SUBDIR: ${{ matrix.subdir }}
@@ -68,8 +58,8 @@ jobs:
6858 matrix:
6959 # The order is important to optimize fail-fast.
7060 name:
71− - 9.8.1-Werror
72− # - 9.8.1-docspec
61+ - 9.4.4-Werror
62+ - 9.2.7-docspec
7363 # - 8.10.7-coverage
7464
7565 # Note: if cabal.project is not specified benchmarks and tests won't
@@ -78,76 +68,53 @@ jobs:
7868 include:
7969 - name: head
8070 ghc_version: head
81− # The URL may change, to find a working URL go to https://gitlab.haskell.org/ghc/ghc/-/jobs/
82− # Find a debian10/11/12 job, click on a passed/failed status, at the
83− # end of the output you will find the tar.xz name, put that tar
84− # name after "raw/", and put the job name after "job=".
85− # Also see https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix
86− #
87− # May also use ghcup for installing ghc head version, use the
88− # version "LatestNightly", and the following config:
89− # ghcup config add-release-channel https://ghc.gitlab.haskell.org/ghcup-metadata/ghcup-nightlies-0.0.7.yaml
90− ghcup_ghc_options: "-u https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-deb10-int_native-validate.tar.xz?job=x86_64-linux-deb10-int_native-validate"
9171 runner: ubuntu-latest
9272 build: cabal
93− cabal_version: 3.10.2.0
73+ cabal_version: 3.8.1.0
9474 cabal_project: cabal.project.ghc-head
9575 disable_sdist_build: "y"
9676 ignore_error: true
97− # Note: use linux for warning build for convenient dev testing
98− - name: 9.8.1-Werror
99− ghc_version: 9.8.1
100− runner: ubuntu-latest
101− build: cabal
102− cabal_version: 3.10.1.0
103− disable_sdist_build: "y"
104− cabal_project: cabal.project.Werror
105− ignore_error: false
106− #- name: 9.8.1-docspec
107− # ghc_version: 9.8.1
108− # runner: ubuntu-latest
109− # build: cabal
110− # cabal_version: 3.10.1.0
111− # cabal_project: cabal.project.doctest
112− # disable_test: "y"
113− # disable_bench: "y"
114− # disable_docs: "y"
115− # enable_docspec: "y"
116− # disable_sdist_build: "y"
117− # ignore_error: false
118− - name: 9.8.1-fusion-inspection
119− ghc_version: 9.8.1
77+ - name: 9.6.1
78+ ghc_version: 9.6.1
12079 runner: ubuntu-latest
12180 build: cabal
12281 cabal_version: 3.10.1.0
12382 disable_sdist_build: "y"
12483 cabal_project: cabal.project
125− cabal_build_options: "--flag fusion-plugin --flag inspection"
126− ignore_error: false
127− - name: 9.6.3-macos
128− ghc_version: 9.6.3
129− runner: macos-latest
84+ ignore_error: true
85+ - name: 9.4.4-Werror
86+ ghc_version: 9.4.4
87+ runner: ubuntu-latest
13088 build: cabal
131− cabal_version: 3.10.1.0
89+ cabal_version: 3.8.1.0
13290 disable_sdist_build: "y"
133− cabal_project: cabal.project
91+ cabal_project: cabal.project.Werror
13492 ignore_error: false
135− - name: 9.4.7
136− ghc_version: 9.4.7
93+ - name: 9.4.4-fusion-inspection
94+ ghc_version: 9.4.4
13795 runner: ubuntu-latest
13896 build: cabal
13997 cabal_version: 3.8.1.0
14098 disable_sdist_build: "y"
14199 cabal_project: cabal.project
100+ cabal_build_options: "--flag fusion-plugin --flag inspection"
142101 ignore_error: false
143− - name: 9.2.8
144− ghc_version: 9.2.8
102+ - name: 9.2.7
103+ ghc_version: 9.2.7
145104 runner: ubuntu-latest
146105 build: cabal
147106 cabal_project: cabal.project
148107 cabal_version: 3.6.2.0
149108 disable_sdist_build: "y"
150109 ignore_error: false
110+ - name: 9.2.7-docspec
111+ ghc_version: 9.2.7
112+ runner: ubuntu-latest
113+ doctest: "y"
114+ cabal_version: 3.6.2.0
115+ cabal_project: cabal.project
116+ disable_sdist_build: "y"
117+ ignore_error: false
151118 - name: 9.0.2-streamly-sdist
152119 ghc_version: 9.0.2
153120 runner: ubuntu-latest
@@ -171,6 +138,14 @@ jobs:
171138 stack_build_options: "--flag streamly-benchmarks:-opt"
172139 cabal_version: 3.6.2.0
173140 ignore_error: false
141+ - name: 8.10.7-macos
142+ ghc_version: 8.10.7
143+ runner: macos-latest
144+ build: cabal
145+ cabal_version: 3.6.2.0
146+ cabal_project: cabal.project
147+ disable_sdist_build: "y"
148+ ignore_error: false
174149 # - name: 8.10.7-coverage
175150 # ghc_version: 8.10.7
176151 # runner: ubuntu-latest
@@ -185,6 +160,15 @@ jobs:
185160 cabal_project: cabal.project
186161 disable_sdist_build: "y"
187162 ignore_error: false
163+ - name: 8.6.5-no-fusion-unoptimized
164+ ghc_version: 8.6.5
165+ runner: ubuntu-latest
166+ build: cabal
167+ cabal_version: 3.6.2.0
168+ cabal_project: cabal.project
169+ cabal_build_options: "--flag no-fusion --flag -opt"
170+ disable_sdist_build: "y"
171+ ignore_error: false
188172 - name: 8.6.5-debug-unoptimized
189173 ghc_version: 8.6.5
190174 runner: ubuntu-latest
@@ -194,44 +178,66 @@ jobs:
194178 cabal_build_options: "--flag debug --flag -opt"
195179 disable_sdist_build: "y"
196180 ignore_error: false
197− - name: hlint
198− build: hlint
199− runner: ubuntu-latest
200− ignore_error: true
201181
202182 steps:
203183 - uses: actions/checkout@v2
204184
205185 - uses: actions/cache@v1
206− name: Cache common directories
186+ name: Cache ~/.cabal
207187 with:
208− path: |
209− ~/.local
210− ~/.cabal
211− ~/.stack
212− ~/.ghcup
188+ path: ~/.cabal
213189 # Bump the key version to clear the cache
214190 key: ${{ runner.os }}-${{ matrix.ghc_version }}-cabal-v2
215191
216192 - name: Download packcheck
217193 run: |
218− if test ! -e "$PACKCHECK"
219− then
220− if test -z "$PACKCHECK_GITHUB_COMMIT"
221− then
222− die "PACKCHECK_GITHUB_COMMIT is not specified."
223− fi
224− PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
225− curl --fail -sL -o "$PACKCHECK" $PACKCHECK_URL || exit 1
226− chmod +x $PACKCHECK
227− elif test ! -x "$PACKCHECK"
194+ # Get packcheck if needed
195+ CURL=$(which curl)
196+ PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
197+ if test ! -e "$PACKCHECK_LOCAL_PATH"; then $CURL -sL -o "$PACKCHECK_LOCAL_PATH" $PACKCHECK_URL; fi;
198+ chmod +x $PACKCHECK_LOCAL_PATH
199+
200+ os=$(uname -s -m)
201+ case "$os" in
202+ "Linux x86_64") GHCUP_ARCH="x86_64-linux" ;;
203+ "Darwin x86_64") GHCUP_ARCH="x86_64-apple-darwin" ;;
204+ *) echo "Unknown OS/Arch: $os"; exit 1;;
205+ esac
206+
207+ # Check available versions here: https://downloads.haskell.org/~ghcup/
208+ GHCUP_VER=0.1.18.0
209+ $CURL -sL -o ./ghcup https://downloads.haskell.org/~ghcup/$GHCUP_VER/${GHCUP_ARCH}-ghcup-$GHCUP_VER
210+ chmod +x ./ghcup
211+ if test "$GHCVER" = "head"
228212 then
229− chmod +x $PACKCHECK
213+ # The URL may change, to find a working URL go to https://gitlab.haskell.org/ghc/ghc/-/jobs/
214+ # Find a debian10 job, click on a passed/failed job, at the
215+ # end of the output you will find the tar.xz name, put that after
216+ # "raw/", and put the job name after "job=".
217+ # Also see https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix
218+ ./ghcup install ghc -u https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-deb10-int_native-validate.tar.xz?job=x86_64-linux-deb10-int_native-validate head
219+ else
220+ ./ghcup install ghc $GHCVER
230221 fi
222+ ./ghcup install cabal $CABALVER
223+ ./ghcup set ghc $GHCVER
224+ ./ghcup set cabal $CABALVER
231225
232226 - name: Run tests
233227 run: |
234− if test -n "$COVERAGE"
228+ if test -n "$DOCTEST"
229+ then
230+ mkdir -p $HOME/.cabal/bin
231+ curl -sL https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz > cabal-docspec.xz
232+ echo '0829bd034fba901cbcfe491d98ed8b28fd54f9cb5c91fa8e1ac62dc4413c9562 cabal-docspec.xz' | sha256sum -c -
233+ xz -d < cabal-docspec.xz > $HOME/.cabal/bin/cabal-docspec
234+ rm -f cabal-docspec.xz
235+ chmod a+x $HOME/.cabal/bin/cabal-docspec
236+ cabal update
237+ cabal build --project-file cabal.project.doctest all
238+ $HOME/.cabal/bin/cabal-docspec --version
239+ $HOME/.cabal/bin/cabal-docspec --timeout 60 --check-properties --property-variables xs
240+ elif test -n "$COVERAGE"
235241 then
236242 # Run tests with coverage
237243 cabal update
@@ -244,15 +250,16 @@ jobs:
244250 # XXX Uncomment this and fix it properly later
245251 # hpc-coveralls --repo-token="$COVERALLS_TOKEN" --coverage-mode=StrictlyFullLines
246252 else
253+ export TOOLS_DIR=/opt
247254 # /usr/local/opt/curl/bin for macOS
248255 export PATH=$HOME/.local/bin:$HOME/.ghcup/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/opt/curl/bin
249256 if test -n "$SUBDIR"
250257 then
251− mv "$PACKCHECK" "$SUBDIR/packcheck.sh"
258+ mv "$PACKCHECK_LOCAL_PATH" "$SUBDIR/packcheck.sh"
252259 # This is required as dist-newstyle will be created on the top level
253260 # directory as it is considered the working dir.
254261 rm cabal.project
255262 cd "$SUBDIR"
256263 fi
257− bash -c "$PACKCHECK $BUILD"
264+ bash -c "$PACKCHECK_LOCAL_PATH $BUILD"
258265 fi
.github/workflows/regression-check.yml+2−2
@@ -26,8 +26,8 @@ jobs:
2626 Data.Fold.Window
2727 Data.Parser
2828 Data.ParserK
29− Data.StreamD
30− Data.StreamK:6
29+ Data.Stream.StreamD
30+ Data.Stream.StreamK:6
3131 Data.Unfold
3232 Data.Stream
3333 Data.Stream.Concurrent
.hlint.ignore−5
@@ -2,9 +2,6 @@ core/src/Streamly/Internal/Data/Stream/StreamK/Type.hs
22core/src/Streamly/Internal/Data/Pipe/Type.hs
33core/src/Streamly/Internal/Unicode/Stream.hs
44src/Streamly/Internal/Data/SmallArray/Type.hs
5−src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs
6−src/Streamly/Internal/Data/Stream/Serial.hs
7−src/Streamly/Internal/Data/Stream/Zip.hs
85test/Streamly/Test/Data/Array.hs
96test/Streamly/Test/Data/Parser.hs
107test/Streamly/Test/Data/ParserK.hs
@@ -13,7 +10,6 @@ test/Streamly/Test/Data/Unfold.hs
1310test/Streamly/Test/FileSystem/Event.hs
1411test/Streamly/Test/Prelude/Concurrent.hs
1512test/Streamly/Test/Data/Stream/Concurrent.hs
16−test/lib/Streamly/Test/Prelude/Common.hs
1713test/Streamly/Test/Prelude/Fold.hs
1814test/Streamly/Test/Prelude/Rate.hs
1915test/Streamly/Test/Prelude/Serial.hs
@@ -31,7 +27,6 @@ benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs
3127benchmark/Streamly/Benchmark/Data/Unfold.hs
3228benchmark/Streamly/Benchmark/FileSystem/Handle.hs
3329benchmark/Streamly/Benchmark/Prelude/Async.hs
34−benchmark/Streamly/Benchmark/Prelude/Concurrent.hs
3530benchmark/Streamly/Benchmark/Prelude/Merge.hs
3631benchmark/Streamly/Benchmark/Prelude/Parallel.hs
3732benchmark/Streamly/Benchmark/Prelude/Rate.hs
benchmark/NanoBenchmarks.hs+1−1
@@ -18,7 +18,7 @@ import qualified Streamly.Data.Fold as FL
1818import qualified Streamly.Internal.Data.Stream as S
1919import qualified Streamly.Internal.Data.Stream.IsStream as Internal
2020
21−import Test.Tasty.Bench
21+import Gauge
2222import System.Random
2323
2424maxValue :: Int
benchmark/Streamly/Benchmark/Data/Array/CommonImports.hs+1−1
@@ -9,7 +9,7 @@ import qualified Streamly.Data.Fold as Fold
99import qualified Streamly.Data.Stream as S
1010import qualified Streamly.Internal.Data.Stream as Stream
1111
12−import Test.Tasty.Bench
12+import Gauge
1313import Streamly.Benchmark.Common hiding (benchPureSrc)
1414import qualified Stream.Common as P
1515
benchmark/Streamly/Benchmark/Data/Array/Stream.hs+3−3
@@ -39,7 +39,7 @@ import Prelude hiding ()
3939
4040import qualified Streamly.Data.Stream as Stream
4141import qualified Streamly.Internal.Data.Array as Array
42−import qualified Streamly.Internal.Data.Array.Stream as ArrayStream
42+import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
4343import qualified Streamly.Internal.Data.Fold as Fold
4444import qualified Streamly.Internal.Data.Parser as Parser
4545import qualified Streamly.Internal.Data.Stream as Stream
@@ -47,13 +47,13 @@ import qualified Streamly.Internal.Data.StreamK as StreamK
4747import qualified Streamly.Internal.FileSystem.Handle as Handle
4848import qualified Streamly.Internal.Unicode.Stream as Unicode
4949
50−import Test.Tasty.Bench hiding (env)
50+import Gauge hiding (env)
5151import Streamly.Benchmark.Common
5252import Streamly.Benchmark.Common.Handle
5353import Control.Monad.IO.Class (MonadIO)
5454
5555#ifdef INSPECTION
56−import Streamly.Internal.Data.MutByteArray (Unbox)
56+import Streamly.Internal.Data.Unbox (Unbox)
5757import Streamly.Internal.Data.Stream (Step(..))
5858import Test.Inspection
5959#endif
benchmark/Streamly/Benchmark/Data/Fold.hs+4−28
@@ -12,11 +12,6 @@
1212{-# OPTIONS_GHC -fno-warn-warnings-deprecations #-}
1313{-# OPTIONS_GHC -Wno-orphans #-}
1414
15−#undef FUSION_CHECK
16−#ifdef FUSION_CHECK
17−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
18−#endif
19−
2015module Main (main) where
2116
2217import Control.DeepSeq (NFData(..))
@@ -41,7 +36,7 @@ import qualified Streamly.Internal.Data.Unfold as Unfold
4136import qualified Streamly.Internal.Data.Pipe as Pipe
4237import qualified Streamly.Internal.Data.Stream as Stream
4338
44−import Test.Tasty.Bench
39+import Gauge
4540import Streamly.Benchmark.Common
4641import Prelude hiding (all, any, take, unzip, sequence_, filter)
4742
@@ -269,13 +264,13 @@ unzip = Stream.fold $ FL.lmap (\a -> (a, a)) (FL.unzip FL.sum FL.length)
269264{-# INLINE unzipWithFstM #-}
270265unzipWithFstM :: Monad m => Stream m Int -> m (Int, Int)
271266unzipWithFstM = do
272− let f a = return (a + 1, a)
267+ let f = \a -> return (a + 1, a)
273268 Stream.fold (FL.unzipWithFstM f FL.sum FL.length)
274269
275270{-# INLINE unzipWithMinM #-}
276271unzipWithMinM :: Monad m => Stream m Int -> m (Int, Int)
277272unzipWithMinM = do
278− let f a = return (a + 1, a)
273+ let f = \a -> return (a + 1, a)
279274 Stream.fold (FL.unzipWithMinM f FL.sum FL.length)
280275
281276-------------------------------------------------------------------------------
@@ -507,9 +502,7 @@ o_n_heap_serial value =
507502-------------------------------------------------------------------------------
508503
509504main :: IO ()
510−main = do
511−#ifndef FUSION_CHECK
512− runWithCLIOpts defaultStreamSize allBenchmarks
505+main = runWithCLIOpts defaultStreamSize allBenchmarks
513506
514507 where
515508
@@ -522,20 +515,3 @@ main = do
522515 , bgroup (o_n_space_prefix moduleName) (o_n_space_serial value)
523516 , bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
524517 ]
525−#else
526− -- Enable FUSION_CHECK macro at the beginning of the file
527− -- Enable one benchmark below, and run the benchmark
528− -- Check the .dump-simpl output
529− let value = 100000
530− let input = source value 1
531− let getKey buckets = (`mod` buckets)
532− let getFold k =
533− return $ case k of
534− 0 -> FL.sum
535− 1 -> FL.length
536− _ -> FL.length
537−
538− -- demuxToMap (getKey 64) (getFold . getKey 64) input
539− toIntMapIO (getKey 64) input
540− return ()
541−#endif
benchmark/Streamly/Benchmark/Data/Fold/Window.hs+40−40
@@ -8,12 +8,12 @@ import Streamly.Internal.Data.Stream (Stream)
88import System.Random (randomRIO)
99
1010import qualified Streamly.Data.Fold as Fold
11−import qualified Streamly.Internal.Data.Fold as Window
12−import qualified Streamly.Internal.Data.Ring as Ring
11+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
12+import qualified Streamly.Internal.Data.Fold.Window as Window
1313import qualified Streamly.Internal.Data.Stream as Stream
1414
1515import Streamly.Benchmark.Common
16−import Test.Tasty.Bench
16+import Gauge
1717
1818{-# INLINE source #-}
1919source :: (Monad m, Num a, Stream.Enumerable a) =>
@@ -70,55 +70,55 @@ o_1_space_folds :: Int -> [Benchmark]
7070o_1_space_folds numElements =
7171 [ bgroup "fold"
7272 [ benchWithFold numElements "minimum (window size 100)"
73− (Window.windowMinimum 100)
73+ (Window.minimum 100)
7474 , benchWithFold numElements "minimum (window size 1000)"
75− (Window.windowMinimum 1000)
75+ (Window.minimum 1000)
7676 , benchWith sourceDescendingInt numElements
7777 "minimum descending (window size 1000)"
78− (Window.windowMinimum 1000)
78+ (Window.minimum 1000)
7979
8080 , benchWithFold numElements "maximum (window size 100)"
81− (Window.windowMaximum 100)
81+ (Window.maximum 100)
8282 , benchWithFold numElements "maximum (window size 1000)"
83− (Window.windowMaximum 1000)
83+ (Window.maximum 1000)
8484 , benchWith sourceDescendingInt numElements
8585 "maximum descending (window size 1000)"
86− (Window.windowMaximum 1000)
86+ (Window.maximum 1000)
8787
8888 , benchWithFold numElements "range (window size 100)"
89− (Window.windowRange 100)
89+ (Window.range 100)
9090 , benchWithFold numElements "range (window size 1000)"
91− (Window.windowRange 1000)
91+ (Window.range 1000)
9292 , benchWith sourceDescendingInt numElements
9393 "range descending (window size 1000)"
94− (Window.windowRange 1000)
94+ (Window.range 1000)
9595
9696 , benchWithFoldInt numElements "sumInt (window size 100)"
97− (Ring.slidingWindow 100 Window.windowSumInt)
97+ (Ring.slidingWindow 100 Window.sumInt)
9898 , benchWithFoldInt numElements "sum for Int (window size 100)"
99− (Ring.slidingWindow 100 Window.windowSum)
99+ (Ring.slidingWindow 100 Window.sum)
100100 , benchWithFold numElements "sum (window size 100)"
101− (Ring.slidingWindow 100 Window.windowSum)
101+ (Ring.slidingWindow 100 Window.sum)
102102 , benchWithFold numElements "sum (window size 1000)"
103− (Ring.slidingWindow 1000 Window.windowSum)
103+ (Ring.slidingWindow 1000 Window.sum)
104104 , benchWithFold numElements "sum (entire stream)"
105− (Window.cumulative Window.windowSum)
105+ (Window.cumulative Window.sum)
106106 , benchWithFold numElements "sum (Data.Fold)"
107107 Fold.sum
108108
109109 , benchWithFold numElements "mean (window size 100)"
110− (Ring.slidingWindow 100 Window.windowMean)
110+ (Ring.slidingWindow 100 Window.mean)
111111 , benchWithFold numElements "mean (window size 1000)"
112− (Ring.slidingWindow 1000 Window.windowMean)
112+ (Ring.slidingWindow 1000 Window.mean)
113113 , benchWithFold numElements "mean (entire stream)"
114− (Window.cumulative Window.windowMean)
114+ (Window.cumulative Window.mean)
115115 , benchWithFold numElements "mean (Data.Fold)"
116116 Fold.mean
117117
118118 , benchWithFold numElements "powerSum 2 (window size 100)"
119− (Ring.slidingWindow 100 (Window.windowPowerSum 2))
119+ (Ring.slidingWindow 100 (Window.powerSum 2))
120120 , benchWithFold numElements "powerSum 2 (entire stream)"
121− (Window.cumulative (Window.windowPowerSum 2))
121+ (Window.cumulative (Window.powerSum 2))
122122
123123 ]
124124 ]
@@ -127,51 +127,51 @@ o_1_space_scans :: Int -> [Benchmark]
127127o_1_space_scans numElements =
128128 [ bgroup "scan"
129129 [ benchWithPostscan numElements "minimum (window size 10)"
130− (Window.windowMinimum 10)
130+ (Window.minimum 10)
131131 -- Below window size 30 the linear search based impl performs better
132132 -- than the dequeue based implementation.
133133 , benchWithPostscan numElements "minimum (window size 30)"
134− (Window.windowMinimum 30)
134+ (Window.minimum 30)
135135 , benchWithPostscan numElements "minimum (window size 1000)"
136− (Window.windowMinimum 1000)
136+ (Window.minimum 1000)
137137 , benchScanWith sourceDescendingInt numElements
138138 "minimum descending (window size 1000)"
139− (Window.windowMinimum 1000)
139+ (Window.minimum 1000)
140140
141141 , benchWithPostscan numElements "maximum (window size 10)"
142− (Window.windowMaximum 10)
142+ (Window.maximum 10)
143143 , benchWithPostscan numElements "maximum (window size 30)"
144− (Window.windowMaximum 30)
144+ (Window.maximum 30)
145145 , benchWithPostscan numElements "maximum (window size 1000)"
146− (Window.windowMaximum 1000)
146+ (Window.maximum 1000)
147147 , benchScanWith sourceDescendingInt numElements
148148 "maximum descending (window size 1000)"
149− (Window.windowMaximum 1000)
149+ (Window.maximum 1000)
150150
151151 , benchWithPostscan numElements "range (window size 10)"
152− (Window.windowRange 10)
152+ (Window.range 10)
153153 , benchWithPostscan numElements "range (window size 30)"
154− (Window.windowRange 30)
154+ (Window.range 30)
155155 , benchWithPostscan numElements "range (window size 1000)"
156− (Window.windowRange 1000)
156+ (Window.range 1000)
157157 , benchScanWith sourceDescendingInt numElements
158158 "range descending (window size 1000)"
159− (Window.windowRange 1000)
159+ (Window.range 1000)
160160
161161 , benchWithPostscan numElements "sum (window size 100)"
162− (Ring.slidingWindow 100 Window.windowSum)
162+ (Ring.slidingWindow 100 Window.sum)
163163 , benchWithPostscan numElements "sum (window size 1000)"
164− (Ring.slidingWindow 1000 Window.windowSum)
164+ (Ring.slidingWindow 1000 Window.sum)
165165
166166 , benchWithPostscan numElements "mean (window size 100)"
167− (Ring.slidingWindow 100 Window.windowMean)
167+ (Ring.slidingWindow 100 Window.mean)
168168 , benchWithPostscan numElements "mean (window size 1000)"
169− (Ring.slidingWindow 1000 Window.windowMean)
169+ (Ring.slidingWindow 1000 Window.mean)
170170
171171 , benchWithPostscan numElements "powerSum 2 (window size 100)"
172− (Ring.slidingWindow 100 (Window.windowPowerSum 2))
172+ (Ring.slidingWindow 100 (Window.powerSum 2))
173173 , benchWithPostscan numElements "powerSum 2 (window size 1000)"
174− (Ring.slidingWindow 1000 (Window.windowPowerSum 2))
174+ (Ring.slidingWindow 1000 (Window.powerSum 2))
175175 ]
176176 ]
177177
benchmark/Streamly/Benchmark/Data/MutArray.hs+1−1
@@ -49,7 +49,7 @@ import qualified Streamly.Internal.Data.MutArray as MArray
4949import qualified Streamly.Internal.Data.Fold as Fold
5050import qualified Streamly.Internal.Data.Stream as Stream
5151
52−import Test.Tasty.Bench
52+import Gauge
5353import Streamly.Benchmark.Common hiding (benchPureSrc)
5454
5555type Stream = MutArray
benchmark/Streamly/Benchmark/Data/Parser.hs+10−10
@@ -47,9 +47,10 @@ import qualified Streamly.Internal.Data.Fold as Fold
4747import qualified Streamly.Internal.Data.Parser as PR
4848import qualified Streamly.Data.Stream as Stream
4949import qualified Streamly.Internal.Data.Producer as Producer
50+import qualified Streamly.Internal.Data.Producer.Source as Source
5051import qualified Streamly.Internal.Data.Stream as Stream
5152
52−import Test.Tasty.Bench hiding (env)
53+import Gauge hiding (env)
5354import Streamly.Benchmark.Common
5455import Streamly.Benchmark.Common.Handle
5556
@@ -348,7 +349,7 @@ split_ value =
348349-- PR.dropWhile (<= (value * 1 `div` 4)) *> PR.die "alt"
349350{-# INLINE takeWhileFail #-}
350351takeWhileFail :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
351−takeWhileFail predicate (Fold fstep finitial _ ffinal) =
352+takeWhileFail predicate (Fold fstep finitial fextract) =
352353 Parser step initial extract
353354
354355 where
@@ -369,7 +370,7 @@ takeWhileFail predicate (Fold fstep finitial _ ffinal) =
369370 Fold.Done b -> Done 0 b
370371 else return $ Error "fail"
371372
372− extract s = fmap (Done 0) (ffinal s)
373+ extract s = fmap (Done 0) (fextract s)
373374
374375{-# INLINE alt2 #-}
375376alt2 :: Monad m
@@ -634,13 +635,13 @@ choice value =
634635{-# INLINE parseManyUnfoldArrays #-}
635636parseManyUnfoldArrays :: Int -> [Array.Array Int] -> IO ()
636637parseManyUnfoldArrays count arrays = do
637− let src = Producer.source (Just (Producer.OuterLoop arrays))
638+ let src = Source.source (Just (Producer.OuterLoop arrays))
638639 let parser = PR.fromFold (Fold.take count Fold.drain)
639640 let readSrc =
640− Producer.producer
641+ Source.producer
641642 $ Producer.concat Producer.fromList Array.producer
642643 let streamParser =
643− Producer.simplify (Producer.parseMany parser readSrc)
644+ Producer.simplify (Source.parseMany parser readSrc)
644645 Stream.fold Fold.drain $ Stream.unfold streamParser src
645646
646647-------------------------------------------------------------------------------
@@ -714,6 +715,8 @@ instance NFData ParseError where
714715o_1_space_serial :: Int -> [Benchmark]
715716o_1_space_serial value =
716717 [ benchIOSink value "takeBetween" $ takeBetween value
718+ , benchIOSink value "takeEQ" $ takeEQ value
719+ , benchIOSink value "takeGE" $ takeGE value
717720 , benchIOSink value "takeWhile" $ takeWhile value
718721 , benchIOSink value "takeWhileP" $ takeWhileP value
719722 , benchIOSink value "takeP" $ takeP value
@@ -810,11 +813,8 @@ o_1_space_serial_unfold bound arrays =
810813o_n_heap_serial :: Int -> [Benchmark]
811814o_n_heap_serial value =
812815 [
813− benchIOSink value "takeEQ" $ takeEQ value
814− , benchIOSink value "takeGE" $ takeGE value
815−
816816 -- lookahead benchmark holds the entire input till end
817− , benchIOSink value "lookAhead" $ lookAhead value
817+ benchIOSink value "lookAhead" $ lookAhead value
818818
819819 -- o-n-heap because of backtracking
820820 , benchIOSrc sourceEscapedFrames value "takeFramedByEsc_"
benchmark/Streamly/Benchmark/Data/ParserK.hs+36−89
@@ -5,10 +5,6 @@
55-- License : BSD-3-Clause
66-- Maintainer : streamly@composewell.com
77
8−-- BENCH_CHUNKED -> adaptC
9−-- BENCH_CHUNKED_GENERIC -> adaptCG
10−-- BENCH_SINGULAR -> adapt
11−
128{-# LANGUAGE CPP #-}
139{-# LANGUAGE FlexibleContexts #-}
1410{-# OPTIONS_GHC -Wno-orphans #-}
@@ -22,13 +18,7 @@ import Control.Applicative ((<|>))
2218import Control.DeepSeq (NFData(..))
2319import Control.Monad.IO.Class (MonadIO)
2420import Data.Foldable (asum)
25−#ifdef BENCH_CHUNKED
2621import Streamly.Data.Array (Array, Unbox)
27−#endif
28−#ifdef BENCH_CHUNKED_GENERIC
29−import Streamly.Data.Array.Generic (Array)
30−import qualified Streamly.Internal.Data.Array.Generic as GenArr (chunksOf)
31−#endif
3222import Streamly.Internal.Data.Fold (Fold(..))
3323import Streamly.Data.StreamK (StreamK)
3424import Streamly.Internal.Data.Parser
@@ -45,52 +35,12 @@ import qualified Streamly.Data.Stream as Stream
4535import qualified Streamly.Data.Fold as FL
4636import qualified Streamly.Internal.Data.Fold as Fold
4737import qualified Streamly.Data.Parser as PRD
48−import qualified Streamly.Internal.Data.ParserK as PR
38+import qualified Streamly.Data.ParserK as PR
4939import qualified Streamly.Internal.Data.StreamK as StreamK
5040
51−import Test.Tasty.Bench
41+import Gauge
5242import Streamly.Benchmark.Common
5343
54−-------------------------------------------------------------------------------
55−-- CPP Helpers
56−-------------------------------------------------------------------------------
57−
58−#ifdef BENCH_CHUNKED
59−
60−#define PARSE_OP StreamK.parseChunks
61−#define FROM_PARSER adaptC
62−#define INPUT (Array a)
63−#define PARSE_ELEM (Array Int)
64−#define CONSTRAINT_IO (MonadIO m, Unbox a)
65−#define CONSTRAINT (Monad m, Unbox a)
66−#define MODULE_NAME "Data.ParserK.Chunked"
67−
68−#endif
69−
70−#ifdef BENCH_CHUNKED_GENERIC
71−
72−#define PARSE_OP StreamK.parseChunksGeneric
73−#define FROM_PARSER adaptCG
74−#define INPUT (Array a)
75−#define PARSE_ELEM (Array Int)
76−#define CONSTRAINT_IO (MonadIO m)
77−#define CONSTRAINT (Monad m)
78−#define MODULE_NAME "Data.ParserK.Chunked.Generic"
79−
80−#endif
81−
82−#ifdef BENCH_SINGULAR
83−
84−#define PARSE_OP StreamK.parse
85−#define FROM_PARSER adapt
86−#define INPUT a
87−#define PARSE_ELEM Int
88−#define CONSTRAINT_IO (MonadIO m)
89−#define CONSTRAINT (Monad m)
90−#define MODULE_NAME "Data.ParserK"
91−
92−#endif
93−
9444-------------------------------------------------------------------------------
9545-- Utilities
9646-------------------------------------------------------------------------------
@@ -110,52 +60,49 @@ sourceUnfoldrM value n = Stream.unfoldrM step n
11060{-# INLINE benchIOSink #-}
11161benchIOSink
11262 :: NFData b
113− => Int -> String -> (StreamK IO PARSE_ELEM -> IO b) -> Benchmark
63+ => Int -> String -> (StreamK IO (Array Int) -> IO b) -> Benchmark
11464benchIOSink value name f =
11565 bench name $ nfIO $ randomRIO (1,1)
11666 >>= f
11767 . StreamK.fromStream
118−#ifdef BENCH_CHUNKED
11968 . Stream.chunksOf 4000
120−#endif
121−#ifdef BENCH_CHUNKED_GENERIC
122− . GenArr.chunksOf 4000
123−#endif
12469 . sourceUnfoldrM value
12570
12671-------------------------------------------------------------------------------
12772-- Parsers
12873-------------------------------------------------------------------------------
12974
75+#define PARSE_OP StreamK.parseChunks
76+
13077{-# INLINE one #-}
13178one :: MonadIO m =>
132− Int -> StreamK m PARSE_ELEM -> m (Either ParseError (Maybe Int))
133−one value = PARSE_OP p
79+ Int -> StreamK m (Array Int) -> m (Either ParseError (Maybe Int))
80+one value = StreamK.parseChunks p
13481
13582 where
13683
13784 p = do
138− m <- PR.FROM_PARSER (PRD.fromFold FL.one)
85+ m <- PR.fromFold FL.one
13986 case m of
14087 Just i -> if i >= value then pure m else p
14188 Nothing -> pure Nothing
14289
14390{-# INLINE satisfy #-}
144−satisfy :: CONSTRAINT_IO => (a -> Bool) -> PR.ParserK INPUT m a
145−satisfy = PR.FROM_PARSER . PRD.satisfy
91+satisfy :: (MonadIO m, Unbox a) => (a -> Bool) -> PR.ParserK a m a
92+satisfy = PR.fromParser . PRD.satisfy
14693
14794{-# INLINE takeWhile #-}
148−takeWhile :: CONSTRAINT_IO => (a -> Bool) -> PR.ParserK INPUT m ()
149−takeWhile p = PR.FROM_PARSER $ PRD.takeWhile p FL.drain
95+takeWhile :: (MonadIO m, Unbox a) => (a -> Bool) -> PR.ParserK a m ()
96+takeWhile p = PR.fromParser $ PRD.takeWhile p FL.drain
15097
15198{-# INLINE takeWhileK #-}
15299takeWhileK :: MonadIO m =>
153− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
100+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
154101takeWhileK value = PARSE_OP (takeWhile (<= value))
155102
156103{-# INLINE splitAp2 #-}
157104splitAp2 :: MonadIO m
158− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ((), ()))
105+ => Int -> StreamK m (Array Int) -> m (Either ParseError ((), ()))
159106splitAp2 value =
160107 PARSE_OP
161108 ((,)
@@ -165,7 +112,7 @@ splitAp2 value =
165112
166113{-# INLINE splitAp8 #-}
167114splitAp8 :: MonadIO m
168− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
115+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
169116splitAp8 value =
170117 PARSE_OP
171118 ( (\() () () () () () () () -> ())
@@ -180,7 +127,7 @@ splitAp8 value =
180127 )
181128
182129{-# INLINE sequenceA #-}
183−sequenceA :: MonadIO m => Int -> StreamK m PARSE_ELEM -> m Int
130+sequenceA :: MonadIO m => Int -> StreamK m (Array Int) -> m Int
184131sequenceA value xs = do
185132 let parser = satisfy (> 0)
186133 list = Prelude.replicate value parser
@@ -189,14 +136,14 @@ sequenceA value xs = do
189136
190137{-# INLINE sequenceA_ #-}
191138sequenceA_ :: MonadIO m =>
192− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
139+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
193140sequenceA_ value xs = do
194141 let parser = satisfy (> 0)
195142 list = Prelude.replicate value parser
196143 PARSE_OP (F.sequenceA_ list) xs
197144
198145{-# INLINE sequence #-}
199−sequence :: MonadIO m => Int -> StreamK m PARSE_ELEM -> m Int
146+sequence :: MonadIO m => Int -> StreamK m (Array Int) -> m Int
200147sequence value xs = do
201148 let parser = satisfy (> 0)
202149 list = Prelude.replicate value parser
@@ -205,7 +152,7 @@ sequence value xs = do
205152
206153{-# INLINE sequence_ #-}
207154sequence_ :: MonadIO m =>
208− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
155+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
209156sequence_ value =
210157 let parser = satisfy (> 0)
211158 list = Prelude.replicate value parser
@@ -213,7 +160,7 @@ sequence_ value =
213160
214161{-# INLINE takeWhileFailD #-}
215162takeWhileFailD :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
216−takeWhileFailD predicate (Fold fstep finitial _ ffinal) =
163+takeWhileFailD predicate (Fold fstep finitial fextract) =
217164 Parser step initial extract
218165
219166 where
@@ -234,16 +181,16 @@ takeWhileFailD predicate (Fold fstep finitial _ ffinal) =
234181 Fold.Done b -> Done 0 b
235182 else return $ Error "fail"
236183
237− extract s = fmap (Done 0) (ffinal s)
184+ extract s = fmap (Done 0) (fextract s)
238185
239186{-# INLINE takeWhileFail #-}
240−takeWhileFail :: CONSTRAINT =>
241− (a -> Bool) -> Fold m a b -> PR.ParserK INPUT m b
242−takeWhileFail p f = PR.FROM_PARSER (takeWhileFailD p f)
187+takeWhileFail :: (Monad m, Unbox a) =>
188+ (a -> Bool) -> Fold m a b -> PR.ParserK a m b
189+takeWhileFail p f = PR.fromParser (takeWhileFailD p f)
243190
244191{-# INLINE alt2 #-}
245192alt2 :: MonadIO m
246− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
193+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
247194alt2 value =
248195 PARSE_OP
249196 ( takeWhileFail (<= (value `div` 2)) Fold.drain
@@ -252,7 +199,7 @@ alt2 value =
252199
253200{-# INLINE alt8 #-}
254201alt8 :: MonadIO m
255− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
202+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
256203alt8 value =
257204 PARSE_OP
258205 ( takeWhileFail (<= ( value `div` 8)) Fold.drain
@@ -267,7 +214,7 @@ alt8 value =
267214
268215{-# INLINE alt16 #-}
269216alt16 :: MonadIO m
270− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
217+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
271218alt16 value =
272219 PARSE_OP
273220 ( takeWhileFail (<= ( value `div` 16)) Fold.drain
@@ -289,27 +236,27 @@ alt16 value =
289236 )
290237
291238{-# INLINE manyAlt #-}
292−manyAlt :: MonadIO m => StreamK m PARSE_ELEM -> m Int
239+manyAlt :: MonadIO m => StreamK m (Array Int) -> m Int
293240manyAlt xs = do
294241 x <- PARSE_OP (AP.many (satisfy (> 0))) xs
295242 return $ Prelude.length x
296243
297244{-# INLINE someAlt #-}
298−someAlt :: MonadIO m => StreamK m PARSE_ELEM -> m Int
245+someAlt :: MonadIO m => StreamK m (Array Int) -> m Int
299246someAlt xs = do
300247 x <- PARSE_OP (AP.some (satisfy (> 0))) xs
301248 return $ Prelude.length x
302249
303250{-# INLINE choice #-}
304251choice :: MonadIO m =>
305− Int -> StreamK m PARSE_ELEM -> m (Either ParseError Int)
252+ Int -> StreamK m (Array Int) -> m (Either ParseError Int)
306253choice value =
307254 PARSE_OP (asum (replicate value (satisfy (< 0)))
308255 AP.<|> satisfy (> 0))
309256
310257{-# INLINE monad2 #-}
311258monad2 :: MonadIO m
312− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
259+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
313260monad2 value =
314261 PARSE_OP $ do
315262 takeWhile (<= (value `div` 2))
@@ -317,7 +264,7 @@ monad2 value =
317264
318265{-# INLINE monad4 #-}
319266monad4 :: MonadIO m
320− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
267+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
321268monad4 value =
322269 PARSE_OP $ do
323270 takeWhile (<= ( value `div` 4))
@@ -327,7 +274,7 @@ monad4 value =
327274
328275{-# INLINE monad8 #-}
329276monad8 :: MonadIO m
330− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
277+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
331278monad8 value =
332279 PARSE_OP $ do
333280 takeWhile (<= ( value `div` 8))
@@ -341,7 +288,7 @@ monad8 value =
341288
342289{-# INLINE monad16 #-}
343290monad16 :: MonadIO m
344− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
291+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
345292monad16 value =
346293 PARSE_OP $ do
347294 takeWhile (<= ( value `div` 16))
@@ -366,7 +313,7 @@ monad16 value =
366313-------------------------------------------------------------------------------
367314
368315moduleName :: String
369−moduleName = MODULE_NAME
316+moduleName = "Data.ParserK"
370317
371318instance NFData ParseError where
372319 {-# INLINE rnf #-}
@@ -388,7 +335,7 @@ o_1_space_serial value =
388335 ]
389336
390337{-# INLINE sepBy1 #-}
391−sepBy1 :: MonadIO m => StreamK m PARSE_ELEM -> m Int
338+sepBy1 :: MonadIO m => StreamK m (Array Int) -> m Int
392339sepBy1 xs = do
393340 x <- PARSE_OP (parser (satisfy odd) (satisfy even)) xs
394341 return $ Prelude.length x
benchmark/Streamly/Benchmark/Data/Ring/Unboxed.hs+4−3
@@ -12,11 +12,12 @@ module Main (main) where
1212
1313import Control.Monad (void)
1414import GHC.Ptr (Ptr(..))
15−import qualified Streamly.Internal.Data.Array as Array
16−import qualified Streamly.Internal.Data.Ring as Ring
15+
16+import qualified Streamly.Internal.Data.Array.Type as Array
17+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
1718import qualified Data.Foldable as P
1819
19−import Test.Tasty.Bench
20+import Gauge
2021import Streamly.Benchmark.Common
2122
2223import Prelude as P
benchmark/Streamly/Benchmark/Data/Serialize.hsdeleted−591
@@ -1,591 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−{- HLINT ignore -}
3−
4−#undef FUSION_CHECK
5−#ifdef FUSION_CHECK
6−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
7−#endif
8−
9−-- |
10−-- Module : Streamly.Benchmark.Data.Serialize
11−-- Copyright : (c) 2023 Composewell
12−-- License : BSD-3-Clause
13−-- Maintainer : streamly@composewell.com
14−
15−module Main (main) where
16−
17−-------------------------------------------------------------------------------
18−-- Imports
19−-------------------------------------------------------------------------------
20−
21−import Control.DeepSeq (NFData(..), deepseq)
22−import Control.Monad (when, void)
23−import GHC.Generics (Generic)
24−import System.Random (randomRIO)
25−#ifndef USE_UNBOX
26−import Test.QuickCheck (Arbitrary, arbitrary)
27−#endif
28−
29−import Streamly.Internal.Data.MutByteArray
30−import qualified Streamly.Internal.Data.MutByteArray as MBA
31−#ifdef USE_UNBOX
32−import Data.Proxy (Proxy(..))
33−#else
34−import Control.DeepSeq (force)
35−import Test.QuickCheck (oneof, generate)
36−#endif
37−
38−import Test.Tasty.Bench
39−import Streamly.Benchmark.Common
40−
41−#ifndef USE_UNBOX
42−import Streamly.Benchmark.Data.Serialize.RecCompatible
43−import Streamly.Benchmark.Data.Serialize.RecNonCompatible
44−#endif
45−
46−#ifdef USE_UNBOX
47−#define SERIALIZE_CLASS Unbox
48−#define DERIVE_CLASS(typ) $(deriveUnbox [d|instance Unbox typ|])
49−#define SERIALIZE_OP pokeAt
50−#define DESERIALIZE_OP peekAt
51−#else
52−#define SERIALIZE_CLASS Serialize
53−#define DERIVE_CLASS(typ) $(deriveSerialize [d|instance Serialize typ|])
54−#define SERIALIZE_OP serializeAt
55−#define DESERIALIZE_OP deserializeAt
56−#endif
57−
58−-------------------------------------------------------------------------------
59−-- Types
60−-------------------------------------------------------------------------------
61−
62−data Unit = Unit
63− deriving (Generic, Show, Eq)
64−
65−#ifndef USE_TH
66−instance SERIALIZE_CLASS Unit
67−#else
68−DERIVE_CLASS(Unit)
69−#endif
70−
71−instance NFData Unit where
72− {-# INLINE rnf #-}
73− rnf _ = ()
74−
75−data Sum2
76− = Sum21
77− | Sum22
78− deriving (Generic, Show, Eq)
79−
80−#ifndef USE_TH
81−instance SERIALIZE_CLASS Sum2
82−#else
83−DERIVE_CLASS(Sum2)
84−#endif
85−
86−instance NFData Sum2 where
87− {-# INLINE rnf #-}
88− rnf _ = ()
89−
90−data Sum25
91− = Sum251
92− | Sum252
93− | Sum253
94− | Sum254
95− | Sum255
96− | Sum256
97− | Sum257
98− | Sum258
99− | Sum259
100− | Sum2510
101− | Sum2511
102− | Sum2512
103− | Sum2513
104− | Sum2514
105− | Sum2515
106− | Sum2516
107− | Sum2517
108− | Sum2518
109− | Sum2519
110− | Sum2520
111− | Sum2521
112− | Sum2522
113− | Sum2523
114− | Sum2524
115− | Sum2525
116− deriving (Generic, Show, Eq)
117−
118−#ifndef USE_TH
119−instance SERIALIZE_CLASS Sum25
120−#else
121−DERIVE_CLASS(Sum25)
122−#endif
123−
124−instance NFData Sum25 where
125− {-# INLINE rnf #-}
126− rnf _ = ()
127−
128−data Product25
129− = Product25
130− Int
131− Int
132− Int
133− Int
134− Int
135− Int
136− Int
137− Int
138− Int
139− Int
140− Int
141− Int
142− Int
143− Int
144− Int
145− Int
146− Int
147− Int
148− Int
149− Int
150− Int
151− Int
152− Int
153− Int
154− Int
155− deriving (Generic, Show)
156−
157−#ifndef USE_TH
158−instance SERIALIZE_CLASS Product25
159−#else
160−DERIVE_CLASS(Product25)
161−#endif
162−
163−instance NFData Product25 where
164− {-# INLINE rnf #-}
165− rnf
166− (Product25 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20 a21 a22 a23 a24 a25)
167− = a1
168− `seq` a2
169− `seq` a3
170− `seq` a4
171− `seq` a5
172− `seq` a6
173− `seq` a7
174− `seq` a8
175− `seq` a9
176− `seq` a10
177− `seq` a11
178− `seq` a12
179− `seq` a13
180− `seq` a14
181− `seq` a15
182− `seq` a16
183− `seq` a17
184− `seq` a18
185− `seq` a19
186− `seq` a20
187− `seq` a21
188− `seq` a22
189− `seq` a23
190− `seq` a24
191− `seq` a25
192− `seq` ()
193−
194−-- XXX derived Eq instance is not inlined
195−instance Eq Product25 where
196− {-# INLINE (==) #-}
197− (==)
198− (Product25 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20 a21 a22 a23 a24 a25)
199− (Product25 b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 b17 b18 b19 b20 b21 b22 b23 b24 b25)
200− = a1 == b1
201− && a2 == b2
202− && a3 == b3
203− && a4 == b4
204− && a5 == b5
205− && a6 == b6
206− && a7 == b7
207− && a8 == b8
208− && a9 == b9
209− && a10 == b10
210− && a11 == b11
211− && a12 == b12
212− && a13 == b13
213− && a14 == b14
214− && a15 == b15
215− && a16 == b16
216− && a17 == b17
217− && a18 == b18
218− && a19 == b19
219− && a20 == b20
220− && a21 == b21
221− && a22 == b22
222− && a23 == b23
223− && a24 == b24
224− && a25 == b25
225−
226−-------------------------------------------------------------------------------
227−-- Simple non-recursive ADT
228−-------------------------------------------------------------------------------
229−
230−{-
231−data CustomDT1
232− = CDT1C1
233− | CDT1C2 Int
234− | CDT1C3 Int Int
235− deriving (Generic, Show, Eq)
236−
237−#ifndef USE_TH
238−instance SERIALIZE_CLASS CustomDT1
239−#else
240−DERIVE_CLASS(CustomDT1)
241−#endif
242−
243−instance NFData CustomDT1 where
244− {-# INLINE rnf #-}
245− rnf CDT1C1 = ()
246− rnf (CDT1C2 i) = i `seq` ()
247− rnf (CDT1C3 i1 i2) = i1 `seq` i2 `seq` ()
248−-}
249−
250−-------------------------------------------------------------------------------
251−-- Recursive ADT
252−-------------------------------------------------------------------------------
253−
254−#ifndef USE_UNBOX
255−data BinTree a
256− = Tree (BinTree a) (BinTree a)
257− | Leaf a
258− deriving (Show, Read, Eq, Generic)
259−
260−#ifndef USE_TH
261−instance Serialize (BinTree a)
262−#else
263−$(deriveSerialize [d|instance Serialize a => Serialize (BinTree a)|])
264−#endif
265−
266−instance NFData a => NFData (BinTree a) where
267− {-# INLINE rnf #-}
268− rnf (Leaf a) = rnf a `seq` ()
269− rnf (Tree l r) = rnf l `seq` rnf r
270−
271−-- XXX This may not terminate, or could become really large.
272−instance Arbitrary a => Arbitrary (BinTree a) where
273− arbitrary = oneof [Leaf <$> arbitrary, Tree <$> arbitrary <*> arbitrary]
274−
275−mkBinTree :: (Arbitrary a) => Int -> IO (BinTree a)
276−mkBinTree = go (generate arbitrary)
277−
278− where
279−
280− go r 0 = Leaf <$> r
281− go r n = Tree <$> go r (n - 1) <*> go r (n - 1)
282−
283−#endif
284−
285−-------------------------------------------------------------------------------
286−-- Size helpers
287−-------------------------------------------------------------------------------
288−
289−{-# INLINE getSize #-}
290−getSize :: forall a. SERIALIZE_CLASS a => a -> Int
291−#ifdef USE_UNBOX
292−getSize _ = sizeOf (Proxy :: Proxy a)
293−#else
294−getSize = addSizeTo 0
295−#endif
296−
297−-------------------------------------------------------------------------------
298−-- Common helpers
299−-------------------------------------------------------------------------------
300−
301−{- HLINT ignore "Eta reduce" -}
302−-- Parts of "f" that are dependent on val will not be optimized out.
303−{-# INLINE loop #-}
304−loop :: Int -> (a -> IO b) -> a -> IO ()
305−loop count f val = go count val
306− where
307−
308− go n x = when (n > 0) $ f x >> go (n-1) x
309−
310−-- The first arg of "f" is the environment which is not threaded around in the
311−-- loop.
312−{-# INLINE loopWith #-}
313−loopWith :: Int -> (e -> a -> IO b) -> e -> a -> IO ()
314−loopWith count f e val = go count val
315− where
316−
317− go n x = when (n > 0) $ f e x >> go (n-1) x
318−
319−benchSink :: NFData b => String -> Int -> (Int -> IO b) -> Benchmark
320−benchSink name times f = bench name (nfIO (randomRIO (times, times) >>= f))
321−
322−-------------------------------------------------------------------------------
323−-- Serialization Helpers
324−-------------------------------------------------------------------------------
325−
326−{-
327−{-# INLINE pokeWithSize #-}
328−pokeWithSize :: SERIALIZE_CLASS a => MutByteArray -> a -> IO ()
329−pokeWithSize arr val = do
330− let n = getSize val
331− n `seq` SERIALIZE_OP 0 arr val >> return ()
332−
333−{-# INLINE pokeTimesWithSize #-}
334−pokeTimesWithSize :: SERIALIZE_CLASS a => a -> Int -> IO ()
335−pokeTimesWithSize val times = do
336− let n = getSize val
337− arr <- MBA.new n
338− loopWith times pokeWithSize arr val
339−-}
340−
341−{-# INLINE poke #-}
342−poke :: SERIALIZE_CLASS a => MutByteArray -> a -> IO ()
343−poke arr val = void (SERIALIZE_OP 0 arr val)
344−
345−{-# INLINE pokeTimes #-}
346−pokeTimes :: SERIALIZE_CLASS a => a -> Int -> IO ()
347−pokeTimes val times = do
348− let n = getSize val
349− arr <- MBA.new n
350− loopWith times poke arr val
351−
352−{-# INLINE encode #-}
353−encode :: SERIALIZE_CLASS a => a -> IO ()
354−encode val = do
355− let n = getSize val
356− arr <- MBA.new n
357− void (SERIALIZE_OP 0 arr val)
358−
359−{-# INLINE encodeTimes #-}
360−encodeTimes :: SERIALIZE_CLASS a => a -> Int -> IO ()
361−encodeTimes val times = loop times encode val
362−
363−{-# INLINE peek #-}
364−peek :: forall a. (NFData a, SERIALIZE_CLASS a) =>
365− (a, Int) -> MutByteArray -> IO ()
366−#ifdef USE_UNBOX
367−peek (_val, _) arr = do
368− (val1 :: a) <- DESERIALIZE_OP 0 arr
369−#else
370−peek (_val, n) arr = do
371− (_, val1 :: a) <- DESERIALIZE_OP 0 arr n
372−#endif
373− -- If the datatype is not deeply strict or deepseq is not used then use
374− -- Equality.
375− -- Ensure that we are actually constructing the type and using it. This
376− -- is important, otherwise the structure is created and discarded, the
377− -- cost of creation of the structure is not accounted. Otherwise we may
378− -- just read the values and discard them. The comparison adds to the
379− -- cost though. We could use deepseq but then we need to write
380− -- instances of NFData and ensure that they are correct and perform
381− -- well. Equality check also ensures correctness.
382− {-
383− if (val1 /= val)
384− then error "peek: no match"
385− else return ()
386− -}
387− val1 `deepseq` return ()
388−
389−{-# INLINE peekTimes #-}
390−peekTimes :: (NFData a, SERIALIZE_CLASS a) => Int -> a -> Int -> IO ()
391−peekTimes n val times = do
392− arr <- MBA.new n
393− _ <- SERIALIZE_OP 0 arr val
394− loopWith times peek (val, n) arr
395−
396−{-# INLINE trip #-}
397−trip :: forall a. (NFData a, SERIALIZE_CLASS a) => a -> IO ()
398−trip val = do
399− let n = getSize val
400− arr <- MBA.new n
401− _ <- SERIALIZE_OP 0 arr val
402−#ifdef USE_UNBOX
403− (val1 :: a) <- DESERIALIZE_OP 0 arr
404−#else
405− (_, val1 :: a) <- DESERIALIZE_OP 0 arr n
406−#endif
407− -- Do not remove this or use deepseq, see the comments in peek.
408− {-
409− if (val1 /= val)
410− then error "roundtrip: no match"
411− else return ()
412− -}
413− -- Note: deepseq is not needed if the datatype is strict
414− val1 `deepseq` return ()
415−
416−{-# INLINE roundtrip #-}
417−roundtrip :: (NFData a, SERIALIZE_CLASS a) => a -> Int -> IO ()
418−roundtrip val times = loop times trip val
419−
420−-------------------------------------------------------------------------------
421−-- Benchmarks
422−-------------------------------------------------------------------------------
423−
424−{-# INLINE benchConst #-}
425−benchConst ::
426− String
427− -> (forall a. (NFData a, SERIALIZE_CLASS a) =>
428− Int -> a -> Int -> IO ())
429− -> Int
430− -> Benchmark
431−benchConst gname f times =
432− bgroup gname
433− [ let !n = getSize Unit
434− in benchSink "Unit" times (f n Unit)
435− {-
436− , let !n = getSize CDT1C1
437− in benchSink "C1" times (f n CDT1C1)
438− , let val = CDT1C2 5
439− !n = getSize val
440− in benchSink "C2" (times `div` 2) (f n val)
441− , let val = CDT1C3 5 2
442− !n = getSize val
443− in benchSink "C3" (times `div` 3) (f n val)
444− -}
445− , let !n = getSize Sum21
446− in benchSink "Sum2" times (f n Sum21)
447− , let !n = getSize Sum2525
448− in benchSink "Sum25" times (f n Sum2525)
449− , let val = Product25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
450− !n = getSize val
451− in benchSink "Product25" (times `div` 26) (f n val)
452− ]
453−
454−#ifndef USE_UNBOX
455−{-# INLINE benchVar #-}
456−benchVar ::
457− String
458− -> (forall a. (NFData a, SERIALIZE_CLASS a) =>
459− Int -> a -> Int -> IO ())
460− -> BinTree Int
461− -> [Int]
462− -> Int
463− -> Benchmark
464−benchVar gname f tInt lInt times =
465− bgroup gname
466− [ let !n = getSize tInt
467− in benchSink "bintree-int" times (f n tInt)
468− , let !n = getSize lInt
469− in benchSink "list-int" times (f n lInt)
470− ]
471−
472−{-
473−{-# INLINE benchGeneric #-}
474−benchGeneric
475− :: (NFData b, Serialize b)
476− => String
477− -> String
478− -> (forall a. (NFData a, Serialize a) =>
479− Int -> a -> Int -> IO ())
480− -> b
481− -> Int
482− -> Benchmark
483−benchGeneric gname valId f val times =
484− bgroup gname
485− [ let !n = getSize val
486− in benchSink valId times (f n val)
487− ]
488−-}
489−
490−{-# INLINE benchLargeRec #-}
491−benchLargeRec ::
492− String
493− -> (forall a. (NFData a, Serialize a) =>
494− Int -> a -> Int -> IO ())
495− -> RecNonCompatible
496− -> RecCompatible
497− -> Int
498− -> Benchmark
499−benchLargeRec gname f recL recR times =
500− bgroup gname
501− [ let !n = getSize recL
502− in benchSink "RecNonCompatible" times (f n recL)
503− , let !n = getSize recR
504− in benchSink "RecCompatible" times (f n recR)
505− ]
506−#endif
507−
508−-- Times is scaled by the number of constructors to normalize
509−#ifdef USE_UNBOX
510−allBenchmarks :: Int -> [Benchmark]
511−allBenchmarks times =
512−#else
513−allBenchmarks ::
514− BinTree Int -> [Int] -> RecNonCompatible -> RecCompatible -> Int -> [Benchmark]
515−allBenchmarks tInt lInt recL recR times =
516−#endif
517− [ bgroup "sizeOf"
518− [
519−#ifndef USE_UNBOX
520− bench "bintree-int" $ nf getSize tInt
521− , bench "list-int" $ nf getSize lInt
522−#endif
523− ]
524− , benchConst "poke" (const pokeTimes) times
525− -- , benchConst "pokeWithSize" (const pokeTimesWithSize) times
526− , benchConst "encode" (const encodeTimes) times
527− , benchConst "peek" peekTimes times
528− , benchConst "roundtrip" (const roundtrip) times
529−#ifndef USE_UNBOX
530− , benchVar "poke" (const pokeTimes) tInt lInt 1
531− -- , benchVar "pokeWithSize" (const pokeTimesWithSize) tInt lInt 1
532− , benchVar "encode" (const encodeTimes) tInt lInt 1
533− , benchVar "peek" peekTimes tInt lInt 1
534− , benchVar "roundtrip" (const roundtrip) tInt lInt 1
535− , benchLargeRec "poke" (const pokeTimes) recL recR times
536− , benchLargeRec "encode" (const encodeTimes) recL recR times
537− , benchLargeRec "peek" peekTimes recL recR times
538− , benchLargeRec "roundtrip" (const roundtrip) recL recR times
539−#endif
540− ]
541−
542−-------------------------------------------------------------------------------
543−-- Driver
544−-------------------------------------------------------------------------------
545−
546−main :: IO ()
547−main = do
548−#ifndef USE_UNBOX
549− -- Approximately 100000 constructors
550− -- Assuming Leaf nodes constitute two constructors (Leaf, Int) and internal
551− -- nodes 1 level = log_2 (100001/3) + 1 = 16
552− !(tInt :: BinTree Int) <- force <$> mkBinTree 16
553−
554− let !recL = valRecNonCompatible
555− let !recR = valRecCompatible
556−
557− -- Approximately 100000 constructors, assuming two constructors (Cons, Int)
558− -- per element.
559− let lInt = [1..50000 :: Int]
560− let !len = length lInt -- evaluate the list
561−#endif
562−#ifndef FUSION_CHECK
563−#ifdef USE_UNBOX
564− runWithCLIOpts defaultStreamSize allBenchmarks
565−#else
566− len
567− `seq` runWithCLIOpts
568− defaultStreamSize
569− (allBenchmarks tInt lInt recL recR)
570−#endif
571−#else
572− -- Enable FUSION_CHECK macro at the beginning of the file
573− -- Enable one benchmark below, and run the benchmark
574− -- Check the .dump-simpl output
575− let value = 100000
576− -- print $ getSize (CDT1C3 4 2)
577− -- print $ getSize tInt
578− -- print $ getSize lInt
579−
580− -- pokeTimes tInt 1
581−
582− -- peekTimes ((CDT1C2 (5 :: Int)) :: CustomDT1) value
583− -- peekTimes (Sum2525) value
584− -- peekTimes (Product25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25) value
585− -- peekTimes tInt 1
586− let !n = getSize lInt
587− peekTimes n lInt 1
588−
589− -- roundtrip ((CDT1C2 (5 :: Int)) :: CustomDT1) value
590− return ()
591−#endif
benchmark/Streamly/Benchmark/Data/Serialize/RecCompatible.hsdeleted−24
@@ -1,24 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.RecCompatible
4− ( RecCompatible(..)
5− , valRecCompatible
6− ) where
7−
8−--------------------------------------------------------------------------------
9−-- Imports
10−--------------------------------------------------------------------------------
11−
12−import Streamly.Internal.Data.MutByteArray (Serialize)
13−import Streamly.Benchmark.Data.Serialize.TH (genLargeRecord)
14−
15−import qualified Streamly.Internal.Data.MutByteArray as Serialize
16−
17−--------------------------------------------------------------------------------
18−-- Code
19−--------------------------------------------------------------------------------
20−
21−$(genLargeRecord "RecCompatible" 50)
22−$(Serialize.deriveSerializeWith
23− (Serialize.encodeRecordFields True)
24− [d|instance Serialize RecCompatible|])
benchmark/Streamly/Benchmark/Data/Serialize/RecNonCompatible.hsdeleted−21
@@ -1,21 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.RecNonCompatible
4− ( RecNonCompatible(..)
5− , valRecNonCompatible
6− ) where
7−
8−--------------------------------------------------------------------------------
9−-- Imports
10−--------------------------------------------------------------------------------
11−
12−import Streamly.Benchmark.Data.Serialize.TH (genLargeRecord)
13−
14−import qualified Streamly.Internal.Data.MutByteArray as Serialize
15−
16−--------------------------------------------------------------------------------
17−-- Code
18−--------------------------------------------------------------------------------
19−
20−$(genLargeRecord "RecNonCompatible" 50)
21−$(Serialize.deriveSerialize [d|instance Serialize.Serialize RecNonCompatible|])
benchmark/Streamly/Benchmark/Data/Serialize/TH.hsdeleted−76
@@ -1,76 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.TH (genLargeRecord) where
4−
5−--------------------------------------------------------------------------------
6−-- Imports
7−--------------------------------------------------------------------------------
8−
9−import Language.Haskell.TH
10−
11−import Streamly.Internal.Data.MutByteArray (makeI)
12−import Control.DeepSeq (NFData(..))
13−
14−--------------------------------------------------------------------------------
15−-- Large Record
16−--------------------------------------------------------------------------------
17−
18−genLargeRecord :: String -> Int -> Q [Dec]
19−genLargeRecord tyName numFields =
20− sequence
21− [ dataD
22− (pure [])
23− (mkName tyName)
24− []
25− Nothing
26− [mkCon tyName]
27− [derivClause Nothing [conT ''Eq, conT ''Show]]
28− , mkValueSigDec
29− , mkValueDec
30− , nfDataInstance tyName
31− ]
32−
33− where
34−
35− fieldTypeChoices = [conT ''()]
36− chooseCycle i xs = xs !! (i `mod` length xs)
37− nfDataInstance nm =
38− instanceD
39− (pure [])
40− (appT (conT ''NFData) (conT (mkName nm)))
41− [ funD
42− 'rnf
43− [ clause
44− [ conP
45− (mkName nm)
46− (varP . makeI <$> [0 .. (numFields - 1)])
47− ]
48− (normalB
49− (foldl
50− (\b a -> [|rnf $(b) `seq` rnf $(a)|])
51− [|()|]
52− (varE . makeI <$> [0 .. (numFields - 1)])))
53− []
54− ]
55− ]
56− valueName = mkName $ "val" ++ tyName
57− mkValueSigDec = sigD valueName [t|$(conT (mkName tyName))|]
58− mkValueDec =
59− funD
60− valueName
61− [ clause
62− []
63− (normalB
64− (foldl
65− (\b a -> [|$(b) $(a)|])
66− (conE (mkName tyName))
67− (conE '() <$ [0 .. (numFields - 1)])))
68− []
69− ]
70− mkCon nm = recC (mkName nm) (mkField <$> [0 .. (numFields - 1)])
71− mkField i =
72− varBangType
73− (mkName ("field" ++ show i))
74− (bangType
75− (bang noSourceUnpackedness noSourceStrictness)
76− (chooseCycle i fieldTypeChoices))
benchmark/Streamly/Benchmark/Data/Stream/Common.hs+1−3
@@ -71,9 +71,7 @@ module Stream.Common
7171 )
7272where
7373
74−#if !MIN_VERSION_base(4,18,0)
7574import Control.Applicative (liftA2)
76−#endif
7775import Control.DeepSeq (NFData)
7876import Control.Exception (try)
7977import GHC.Exception (ErrorCall)
@@ -104,7 +102,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
104102#endif
105103#endif
106104
107−import Test.Tasty.Bench
105+import Gauge
108106import Prelude hiding (Foldable(..), mapM, replicate)
109107
110108#ifdef USE_STREAMK
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentCommon.hs+3−3
@@ -14,14 +14,14 @@ where
1414import Stream.Common
1515 (composeN, benchIO, benchIOSink, benchIOSrc, sourceUnfoldrM)
1616import Streamly.Data.Stream (Stream)
17−import Streamly.Internal.Data.Stream.Prelude (MonadAsync, Config)
17+import Streamly.Internal.Data.Stream.Concurrent (MonadAsync, Config)
1818
1919import qualified Data.List as List
2020import qualified Streamly.Data.Fold as Fold
2121import qualified Streamly.Data.Stream as Stream
22−import qualified Streamly.Internal.Data.Stream.Prelude as Async
22+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
2323
24−import Test.Tasty.Bench
24+import Gauge
2525import Prelude hiding (mapM)
2626import Streamly.Benchmark.Common
2727
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentEager.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentEager"
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentInterleaved.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentInterleaved"
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentOrdered.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentOrdered"
benchmark/Streamly/Benchmark/Data/Stream/Eliminate.hs+1−3
@@ -57,7 +57,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
5757#endif
5858#endif
5959
60−import Test.Tasty.Bench
60+import Gauge
6161import Streamly.Benchmark.Common
6262import Prelude hiding (length, sum, or, and, any, all, notElem, elem, (!!),
6363 lookup, repeat, minimum, maximum, product, last, mapM_, init)
@@ -132,14 +132,12 @@ foldableAny :: Int -> Int -> Bool
132132foldableAny value n =
133133 Prelude.any (> (value + 1)) (sourceUnfoldr value n :: Stream Identity Int)
134134
135−{- HLINT ignore "Use all"-}
136135{-# INLINE foldableAnd #-}
137136foldableAnd :: Int -> Int -> Bool
138137foldableAnd value n =
139138 Prelude.and $ fmap
140139 (<= (value + 1)) (sourceUnfoldr value n :: Stream Identity Int)
141140
142−{- HLINT ignore "Use any"-}
143141{-# INLINE foldableOr #-}
144142foldableOr :: Int -> Int -> Bool
145143foldableOr value n =
benchmark/Streamly/Benchmark/Data/Stream/Exceptions.hs+3−12
@@ -36,22 +36,21 @@ import System.IO (Handle, hClose, hPutChar)
3636import qualified Streamly.FileSystem.Handle as FH
3737import qualified Streamly.Internal.FileSystem.Handle as IFH
3838import qualified Streamly.Internal.Data.Unfold as IUF
39−import qualified Streamly.Internal.Data.Unfold.Prelude as IUF
39+import qualified Streamly.Internal.Data.Unfold.Exception as IUF
4040
4141#ifdef USE_PRELUDE
4242import qualified Streamly.Internal.Data.Stream.IsStream as Stream
4343#else
4444import qualified Streamly.Internal.Data.Stream as Stream
45−import qualified Streamly.Internal.Data.Stream.Prelude as Stream
45+import qualified Streamly.Internal.Data.Stream.Exception.Lifted as Stream
4646#endif
4747
48−import Test.Tasty.Bench hiding (env)
48+import Gauge hiding (env)
4949import Prelude hiding (last, length)
5050import Streamly.Benchmark.Common
5151import Streamly.Benchmark.Common.Handle
5252
5353#ifdef INSPECTION
54−import Control.Monad.Catch (MonadCatch)
5554import Test.Inspection
5655
5756import qualified Streamly.Internal.Data.Stream as D
@@ -287,12 +286,8 @@ readChunksOnException inh devNull =
287286 in IUF.fold (IFH.writeChunks devNull) readEx inh
288287
289288#ifdef INSPECTION
290−#if __GLASGOW_HASKELL__ >= 906
291−inspect $ hasNoTypeClassesExcept 'readChunksOnException [''MonadCatch]
292−#else
293289inspect $ hasNoTypeClasses 'readChunksOnException
294290#endif
295−#endif
296291
297292-- | Send the file contents to /dev/null with exception handling
298293readChunksBracket_ :: Handle -> Handle -> IO ()
@@ -301,12 +296,8 @@ readChunksBracket_ inh devNull =
301296 in IUF.fold (IFH.writeChunks devNull) readEx inh
302297
303298#ifdef INSPECTION
304−#if __GLASGOW_HASKELL__ >= 906
305−inspect $ hasNoTypeClassesExcept 'readChunksBracket_ [''MonadCatch]
306−#else
307299inspect $ hasNoTypeClasses 'readChunksBracket_
308300#endif
309−#endif
310301
311302readChunksBracket :: Handle -> Handle -> IO ()
312303readChunksBracket inh devNull =
benchmark/Streamly/Benchmark/Data/Stream/Expand.hs+1−9
@@ -54,7 +54,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
5454#endif
5555#endif
5656
57−import Test.Tasty.Bench
57+import Gauge
5858import Stream.Common
5959import Streamly.Benchmark.Common
6060import Prelude hiding (concatMap)
@@ -198,11 +198,7 @@ concatMapPure outer inner n =
198198 (fromStream $ sourceUnfoldr outer n)
199199
200200#ifdef INSPECTION
201−#if __GLASGOW_HASKELL__ >= 906
202−inspect $ hasNoTypeClassesExcept 'concatMapPure [''Applicative]
203−#else
204201inspect $ hasNoTypeClasses 'concatMapPure
205−#endif
206202inspect $ 'concatMapPure `hasNoType` ''SPEC
207203#endif
208204
@@ -215,11 +211,7 @@ concatMapRepl outer inner n =
215211 (fromStream . S.replicate inner) (fromStream $ sourceUnfoldrM outer n)
216212
217213#ifdef INSPECTION
218−#if __GLASGOW_HASKELL__ >= 906
219−inspect $ hasNoTypeClassesExcept 'concatMapRepl [''Applicative]
220−#else
221214inspect $ hasNoTypeClasses 'concatMapRepl
222−#endif
223215inspect $ 'concatMapRepl `hasNoType` ''SPEC
224216#endif
225217
benchmark/Streamly/Benchmark/Data/Stream/Generate.hs+1−1
@@ -40,7 +40,7 @@ import qualified Stream.Common as Common
4040#endif
4141#endif
4242
43−import Test.Tasty.Bench
43+import Gauge
4444import Streamly.Benchmark.Common
4545import qualified Prelude
4646
benchmark/Streamly/Benchmark/Data/Stream/Lift.hs+1−1
@@ -37,7 +37,7 @@ import qualified Control.Monad.State.Strict as State
3737#endif
3838#endif
3939
40−import Test.Tasty.Bench
40+import Gauge
4141import Streamly.Benchmark.Common
4242import Prelude hiding (reverse, tail)
4343
benchmark/Streamly/Benchmark/Data/Stream/Reduce.hs+2−2
@@ -48,7 +48,7 @@ import Streamly.Internal.Data.Stream (Stream)
4848import qualified Streamly.Internal.Data.Stream as S
4949#ifndef USE_STREAMLY_CORE
5050import qualified Streamly.Data.Stream.Prelude as S
51−import qualified Streamly.Internal.Data.Stream.Prelude as S
51+import qualified Streamly.Internal.Data.Stream.Time as S
5252#endif
5353
5454#ifdef USE_STREAMK
@@ -61,7 +61,7 @@ import qualified Streamly.Internal.Data.Stream as K
6161
6262#endif
6363
64−import Test.Tasty.Bench
64+import Gauge
6565import Streamly.Benchmark.Common
6666import Stream.Common
6767import Prelude hiding (reverse, tail)
benchmark/Streamly/Benchmark/Data/Stream/Split.hs+2−2
@@ -35,7 +35,7 @@ import qualified Streamly.Internal.FileSystem.Handle as IFH
3535import qualified Streamly.Internal.Unicode.Stream as IUS
3636import qualified Streamly.Prelude as S
3737
38−import Test.Tasty.Bench hiding (env)
38+import Gauge hiding (env)
3939import Prelude hiding (last, length)
4040import Streamly.Benchmark.Common
4141import Streamly.Benchmark.Common.Handle
@@ -43,7 +43,7 @@ import Streamly.Benchmark.Common.Handle
4343#ifdef INSPECTION
4444import Streamly.Internal.Data.Stream (Step(..))
4545
46−import qualified Streamly.Internal.Data.MutArray as MA
46+import qualified Streamly.Internal.Data.MutArray.Type as MA
4747import qualified Streamly.Internal.Data.Unfold as IUF
4848
4949import Test.Inspection
benchmark/Streamly/Benchmark/Data/Stream/StreamD.hs+2−4
@@ -22,12 +22,10 @@
2222
2323module Main (main) where
2424
25−#if !MIN_VERSION_base(4,18,0)
2625import Control.Applicative (liftA2)
27−#endif
2826import Control.Monad (when)
2927import Data.Maybe (isJust)
30−import Test.Tasty.Bench (bench, nfIO, bgroup, Benchmark, defaultMain)
28+import Gauge (bench, nfIO, bgroup, Benchmark, defaultMain)
3129import System.Random (randomRIO)
3230import Prelude hiding (tail, mapM_, foldl, last, map, mapM, concatMap, zip)
3331
@@ -463,7 +461,7 @@ inspect $ 'unfoldManyRepl `hasNoType` ''SPEC
463461-------------------------------------------------------------------------------
464462
465463moduleName :: String
466−moduleName = "Data.StreamD"
464+moduleName = "Data.Stream.StreamD"
467465
468466o_1_space :: [Benchmark]
469467o_1_space =
benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs+3−4
@@ -21,9 +21,7 @@
2121
2222module Main (main) where
2323
24−#if !MIN_VERSION_base(4,18,0)
2524import Control.Applicative (liftA2)
26−#endif
2725import Control.Monad (when)
2826import Data.Maybe (isJust)
2927import System.Random (randomRIO)
@@ -35,9 +33,10 @@ import Prelude hiding
3533import qualified Prelude as P
3634import qualified Data.List as List
3735
36+import qualified Streamly.Internal.Data.Stream.StreamK.Type as S
3837import qualified Streamly.Internal.Data.StreamK as S
3938
40−import Test.Tasty.Bench (bench, nfIO, bgroup, Benchmark, defaultMain)
39+import Gauge (bench, nfIO, bgroup, Benchmark, defaultMain)
4140
4241import Streamly.Benchmark.Common
4342
@@ -634,7 +633,7 @@ filterAllInNestedList str = do
634633-------------------------------------------------------------------------------
635634
636635moduleName :: String
637−moduleName = "Data.StreamK"
636+moduleName = "Data.Stream.StreamK"
638637
639638o_1_space_generation :: Int -> Benchmark
640639o_1_space_generation streamLen =
benchmark/Streamly/Benchmark/Data/Stream/StreamKAlt.hs+3−3
@@ -18,12 +18,12 @@ import Prelude hiding ()
1818-- import qualified Prelude as P
1919-- import qualified Data.List as List
2020
21−import qualified Streamly.Internal.Data.StreamK.Alt as S
21+import qualified Streamly.Internal.Data.Stream.StreamK.Alt as S
2222-- import qualified Streamly.Internal.Data.Stream.Common as SP
2323-- import qualified Streamly.Internal.Data.SVar as S
2424
2525import Streamly.Benchmark.Common
26−import Test.Tasty.Bench (bgroup, Benchmark, defaultMain)
26+import Gauge (bgroup, Benchmark, defaultMain)
2727
2828value :: Int
2929value = 100000
@@ -452,4 +452,4 @@ o_1_space =
452452 ]
453453
454454main :: IO ()
455−main = defaultMain [o_1_space]
455+main = defaultMain $ concat [o_1_space]
benchmark/Streamly/Benchmark/Data/Stream/ToStreamK.hs+4−3
@@ -1,5 +1,5 @@
11-- |
2−-- Module : Streamly.Benchmark.Data.StreamK
2+-- Module : Streamly.Benchmark.Data.Stream.StreamK
33-- Copyright : (c) 2018 Composewell Technologies
44--
55-- License : BSD3
@@ -33,11 +33,12 @@ import Prelude hiding
3333import qualified Prelude as P
3434-- import qualified Data.List as List
3535
36+import qualified Streamly.Internal.Data.Stream.StreamK.Type as S
3637import qualified Streamly.Internal.Data.StreamK as S
3738import qualified Streamly.Internal.Data.Stream as D
3839import qualified Streamly.Internal.Data.Fold as Fold
3940
40−import Test.Tasty.Bench (bgroup, Benchmark, defaultMain)
41+import Gauge (bgroup, Benchmark, defaultMain)
4142
4243import Streamly.Benchmark.Common
4344
@@ -120,7 +121,7 @@ fromFoldable streamLen n = S.fromFoldable [n..n+streamLen]
120121{-# INLINE fromFoldableM #-}
121122fromFoldableM :: Monad m => Int -> Int -> Stream m Int
122123fromFoldableM streamLen n =
123− Prelude.foldr (S.consM . return) S.nil [n .. n + streamLen]
124+ Prelude.foldr S.consM S.nil (Prelude.fmap return [n..n+streamLen])
124125
125126{-
126127{-# INLINABLE concatMapFoldableWith #-}
benchmark/Streamly/Benchmark/Data/Stream/Transform.hs+2−2
@@ -46,7 +46,7 @@ import Streamly.Internal.Data.Time.Units
4646import Streamly.Internal.Data.Stream (Stream)
4747import qualified Streamly.Internal.Data.Stream as Stream
4848#ifndef USE_STREAMLY_CORE
49−import qualified Streamly.Internal.Data.Stream.Prelude as Stream
49+import qualified Streamly.Internal.Data.Stream.Time as Stream
5050#endif
5151#ifdef USE_STREAMK
5252import Control.DeepSeq (NFData(..))
@@ -58,7 +58,7 @@ import qualified Streamly.Internal.Data.StreamK as StreamK
5858#endif
5959#endif
6060
61−import Test.Tasty.Bench
61+import Gauge
6262import Stream.Common hiding (scanl')
6363import Streamly.Benchmark.Common
6464import Prelude hiding (sequence, mapM)
benchmark/Streamly/Benchmark/Data/Unbox.hsnew file+132
@@ -0,0 +1,132 @@
1+{-# LANGUAGE TemplateHaskell #-}
2+
3+#undef FUSION_CHECK
4+#ifdef FUSION_CHECK
5+{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
6+#endif
7+
8+
9+-- |
10+-- Module : Streamly.Benchmark.Data.Unbox
11+-- Copyright : (c) 2023 Composewell
12+-- License : BSD-3-Clause
13+-- Maintainer : streamly@composewell.com
14+
15+module Main (main) where
16+
17+-------------------------------------------------------------------------------
18+-- Imports
19+-------------------------------------------------------------------------------
20+
21+import Control.DeepSeq (NFData(..))
22+import Control.Exception (assert)
23+import Control.Monad (replicateM_)
24+import Data.Proxy (Proxy(..))
25+import GHC.Generics (Generic)
26+import System.Random (randomRIO)
27+
28+import Streamly.Internal.Data.Unbox
29+#ifdef USE_TH
30+import Streamly.Internal.Data.Unbox.TH
31+#endif
32+
33+import Gauge
34+import Streamly.Benchmark.Common
35+
36+-------------------------------------------------------------------------------
37+-- Types
38+-------------------------------------------------------------------------------
39+
40+data CustomDT1
41+ = CDT1C1
42+ | CDT1C2 Int
43+ | CDT1C3 Int Bool
44+ deriving (Generic, Show, Eq)
45+
46+#ifndef USE_TH
47+instance Unbox CustomDT1
48+#else
49+$(deriveUnbox ''CustomDT1)
50+#endif
51+
52+-------------------------------------------------------------------------------
53+-- Helpers
54+-------------------------------------------------------------------------------
55+
56+{-# INLINE pokeTimes #-}
57+pokeTimes :: forall a. Unbox a => a -> Int -> IO ()
58+pokeTimes val times = do
59+ arr <- newBytes (sizeOf (Proxy :: Proxy a))
60+ replicateM_ times $ do
61+ pokeByteIndex 0 arr val
62+
63+{-# INLINE peekTimes #-}
64+peekTimes :: forall a. Unbox a => a -> Int -> IO ()
65+peekTimes val times = do
66+ arr <- newBytes (sizeOf (Proxy :: Proxy a))
67+ pokeByteIndex 0 arr val
68+ replicateM_ times $ do
69+ (_ :: a) <- peekByteIndex 0 arr
70+ return ()
71+
72+{-# INLINE roundtrip #-}
73+roundtrip :: forall a. (Eq a, Unbox a) => a -> Int -> IO ()
74+roundtrip val times = do
75+ arr <- newBytes (sizeOf (Proxy :: Proxy a))
76+ replicateM_ times $ do
77+ pokeByteIndex 0 arr val
78+ val1 <- peekByteIndex 0 arr
79+ assert (val == val1) (pure ())
80+
81+benchSink :: NFData b => String -> Int -> (Int -> IO b) -> Benchmark
82+benchSink name times f = bench name (nfIO (randomRIO (times, times) >>= f))
83+
84+-------------------------------------------------------------------------------
85+-- Benchmarks
86+-------------------------------------------------------------------------------
87+
88+allBenchmarks :: Int -> [Benchmark]
89+allBenchmarks times =
90+ [ bgroup "poke"
91+ [ benchSink "C1" times
92+ (pokeTimes (CDT1C1 :: CustomDT1))
93+ , benchSink "C2" times
94+ (pokeTimes ((CDT1C2 (5 :: Int)) :: CustomDT1))
95+ , benchSink "C3" times
96+ (pokeTimes ((CDT1C3 (5 :: Int) True) :: CustomDT1))
97+ ]
98+ , bgroup "peek"
99+ [ benchSink "C1" times
100+ (peekTimes (CDT1C1 :: CustomDT1))
101+ , benchSink "C2" times
102+ (peekTimes ((CDT1C2 (5 :: Int)) :: CustomDT1))
103+ , benchSink "C3" times
104+ (peekTimes ((CDT1C3 (5 :: Int) True) :: CustomDT1))
105+ ]
106+ , bgroup "roundtrip"
107+ [ benchSink "C1" times
108+ (roundtrip (CDT1C1 :: CustomDT1))
109+ , benchSink "C2" times
110+ (roundtrip ((CDT1C2 (5 :: Int)) :: CustomDT1))
111+ , benchSink "C3" times
112+ (roundtrip ((CDT1C3 (5 :: Int) True) :: CustomDT1))
113+ ]
114+ ]
115+
116+-------------------------------------------------------------------------------
117+-- Driver
118+-------------------------------------------------------------------------------
119+
120+main :: IO ()
121+main = do
122+#ifndef FUSION_CHECK
123+ runWithCLIOpts defaultStreamSize allBenchmarks
124+#else
125+ -- Enable FUSION_CHECK macro at the beginning of the file
126+ -- Enable one benchmark below, and run the benchmark
127+ -- Check the .dump-simpl output
128+ let value = 100000
129+ -- peekTimes ((CDT1C2 (5 :: Int)) :: CustomDT1) value
130+ roundtrip ((CDT1C2 (5 :: Int)) :: CustomDT1) value
131+ return ()
132+#endif
benchmark/Streamly/Benchmark/Data/Unfold.hs+2−11
@@ -31,18 +31,17 @@ import qualified Prelude
3131import qualified Streamly.FileSystem.Handle as FH
3232import qualified Streamly.Internal.Data.Fold as FL
3333import qualified Streamly.Internal.Data.Unfold as UF
34−import qualified Streamly.Internal.Data.Unfold.Prelude as UF
34+import qualified Streamly.Internal.Data.Unfold.Exception as UF
3535import qualified Streamly.Internal.Data.Stream as S
3636import qualified Streamly.Internal.Data.Stream as D
3737import qualified Streamly.Internal.Data.StreamK as K
3838
39−import Test.Tasty.Bench hiding (env)
39+import Gauge hiding (env)
4040import Prelude hiding (take, filter, zipWith, map, mapM, takeWhile)
4141import Streamly.Benchmark.Common
4242import Streamly.Benchmark.Common.Handle
4343
4444#ifdef INSPECTION
45−import Control.Monad.Catch (MonadCatch)
4645import Test.Inspection
4746#endif
4847
@@ -777,11 +776,7 @@ readWriteFinally_Unfold inh devNull =
777776 in S.fold (FH.write devNull) $ S.unfold readEx inh
778777
779778#ifdef INSPECTION
780−#if __GLASGOW_HASKELL__ >= 906
781−inspect $ hasNoTypeClassesExcept 'readWriteFinally_Unfold [''MonadCatch]
782−#else
783779inspect $ hasNoTypeClasses 'readWriteFinally_Unfold
784−#endif
785780-- inspect $ 'readWriteFinallyUnfold `hasNoType` ''Step
786781#endif
787782
@@ -797,11 +792,7 @@ readWriteBracket_Unfold inh devNull =
797792 in S.fold (FH.write devNull) $ S.unfold readEx inh
798793
799794#ifdef INSPECTION
800−#if __GLASGOW_HASKELL__ >= 906
801−inspect $ hasNoTypeClassesExcept 'readWriteBracket_Unfold [''MonadCatch]
802−#else
803795inspect $ hasNoTypeClasses 'readWriteBracket_Unfold
804−#endif
805796-- inspect $ 'readWriteBracketUnfold `hasNoType` ''Step
806797#endif
807798
benchmark/Streamly/Benchmark/FileSystem/Handle.hs+1−1
@@ -22,7 +22,7 @@ import Streamly.Benchmark.Common.Handle (mkHandleBenchEnv)
2222import qualified Handle.ReadWrite as RW
2323import qualified Handle.Read as RO
2424
25−import Test.Tasty.Bench hiding (env)
25+import Gauge hiding (env)
2626import Prelude hiding (last, length)
2727import Streamly.Benchmark.Common
2828
benchmark/Streamly/Benchmark/FileSystem/Handle/Read.hs+9−8
@@ -33,6 +33,7 @@ import qualified Streamly.Data.Stream as Stream
3333import qualified Streamly.Data.Fold as Fold
3434import qualified Streamly.FileSystem.Handle as FH
3535import qualified Streamly.Internal.Data.Array as A
36+import qualified Streamly.Internal.Data.Array.Type as AT
3637import qualified Streamly.Internal.Data.Fold as FL
3738import qualified Streamly.Internal.Data.Stream as IP
3839import qualified Streamly.Internal.FileSystem.Handle as IFH
@@ -40,14 +41,14 @@ import qualified Streamly.Internal.Unicode.Stream as IUS
4041import qualified Streamly.Data.Stream.Prelude as S
4142import qualified Streamly.Unicode.Stream as SS
4243
43−import Test.Tasty.Bench hiding (env)
44+import Gauge hiding (env)
4445import Prelude hiding (last, length)
4546import Streamly.Benchmark.Common.Handle
4647
4748#ifdef INSPECTION
4849import Streamly.Internal.Data.Stream (Step(..), FoldMany)
4950
50−import qualified Streamly.Internal.Data.MutArray as MutArray
51+import qualified Streamly.Internal.Data.MutArray.Type as MA
5152import qualified Streamly.Internal.Data.Stream as D
5253import qualified Streamly.Internal.Data.Unfold as IUF
5354
@@ -68,7 +69,7 @@ readLast = S.fold Fold.last . S.unfold FH.reader
6869inspect $ hasNoTypeClasses 'readLast
6970inspect $ 'readLast `hasNoType` ''Step -- S.unfold
7071inspect $ 'readLast `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
71−inspect $ 'readLast `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
72+inspect $ 'readLast `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
7273#endif
7374
7475-- assert that flattenArrays constructors are not present
@@ -80,7 +81,7 @@ readCountBytes = S.fold Fold.length . S.unfold FH.reader
8081inspect $ hasNoTypeClasses 'readCountBytes
8182inspect $ 'readCountBytes `hasNoType` ''Step -- S.unfold
8283inspect $ 'readCountBytes `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
83−inspect $ 'readCountBytes `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
84+inspect $ 'readCountBytes `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
8485#endif
8586
8687-- | Count the number of lines in a file.
@@ -95,7 +96,7 @@ readCountLines =
9596inspect $ hasNoTypeClasses 'readCountLines
9697inspect $ 'readCountLines `hasNoType` ''Step
9798inspect $ 'readCountLines `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
98−inspect $ 'readCountLines `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
99+inspect $ 'readCountLines `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
99100#endif
100101
101102-- | Count the number of words in a file.
@@ -119,7 +120,7 @@ readSumBytes = S.fold Fold.sum . S.unfold FH.reader
119120inspect $ hasNoTypeClasses 'readSumBytes
120121inspect $ 'readSumBytes `hasNoType` ''Step
121122inspect $ 'readSumBytes `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
122−inspect $ 'readSumBytes `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
123+inspect $ 'readSumBytes `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
123124#endif
124125
125126-- XXX When we mark this with INLINE and we have two benchmarks using S.drain
@@ -235,13 +236,13 @@ groupsOf :: Int -> Handle -> IO Int
235236groupsOf n inh =
236237 -- writeNUnsafe gives 2.5x boost here over writeN.
237238 S.fold Fold.length
238− $ IP.groupsOf n (A.writeNUnsafe n) (S.unfold FH.reader inh)
239+ $ IP.groupsOf n (AT.writeNUnsafe n) (S.unfold FH.reader inh)
239240
240241#ifdef INSPECTION
241242inspect $ hasNoTypeClasses 'groupsOf
242243inspect $ 'groupsOf `hasNoType` ''Step
243244inspect $ 'groupsOf `hasNoType` ''FoldMany
244−inspect $ 'groupsOf `hasNoType` ''MutArray.ArrayUnsafe -- AT.writeNUnsafe
245+inspect $ 'groupsOf `hasNoType` ''AT.ArrayUnsafe -- AT.writeNUnsafe
245246 -- FH.read/A.read
246247inspect $ 'groupsOf `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
247248#endif
benchmark/Streamly/Benchmark/FileSystem/Handle/ReadWrite.hs+10−9
@@ -34,7 +34,7 @@ import qualified Streamly.Internal.FileSystem.Handle as IFH
3434import qualified Streamly.Data.Array as A
3535import qualified Streamly.Data.Stream.Prelude as S
3636
37−import Test.Tasty.Bench hiding (env)
37+import Gauge hiding (env)
3838import Streamly.Benchmark.Common.Handle
3939
4040#ifdef INSPECTION
@@ -42,7 +42,8 @@ import Streamly.Internal.Data.Stream (Step(..))
4242
4343import qualified Streamly.Internal.Data.Stream as D
4444import qualified Streamly.Internal.Data.Tuple.Strict as Strict
45−import qualified Streamly.Internal.Data.MutArray as MutArray
45+import qualified Streamly.Internal.Data.Stream.MutChunked as MAS
46+import qualified Streamly.Internal.Data.Array.Type as AT
4647
4748import Test.Inspection
4849#endif
@@ -82,7 +83,7 @@ copyStream inh outh = S.fold (FH.write outh) (S.unfold FH.reader inh)
8283inspect $ hasNoTypeClasses 'copyStream
8384inspect $ 'copyStream `hasNoType` ''Step -- S.unfold
8485inspect $ 'copyStream `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
85−inspect $ 'copyStream `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
86+inspect $ 'copyStream `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
8687 -- FH.read/A.read
8788inspect $ 'copyStream `hasNoType` ''Strict.Tuple3' -- FH.write/chunksOf
8889#endif
@@ -108,8 +109,8 @@ readFromBytesNull inh devNull = IFH.putBytes devNull $ S.unfold FH.reader inh
108109#ifdef INSPECTION
109110inspect $ hasNoTypeClasses 'readFromBytesNull
110111inspect $ 'readFromBytesNull `hasNoType` ''Step
111−inspect $ 'readFromBytesNull `hasNoType` ''MutArray.SpliceState
112−inspect $ 'readFromBytesNull `hasNoType` ''MutArray.ArrayUnsafe -- FH.fromBytes/S.chunksOf
112+inspect $ 'readFromBytesNull `hasNoType` ''MAS.SpliceState
113+inspect $ 'readFromBytesNull `hasNoType` ''AT.ArrayUnsafe -- FH.fromBytes/S.chunksOf
113114inspect $ 'readFromBytesNull `hasNoType` ''D.FoldMany
114115#endif
115116
@@ -122,8 +123,8 @@ readWithFromBytesNull inh devNull =
122123#ifdef INSPECTION
123124inspect $ hasNoTypeClasses 'readWithFromBytesNull
124125inspect $ 'readWithFromBytesNull `hasNoType` ''Step
125−inspect $ 'readWithFromBytesNull `hasNoType` ''MutArray.SpliceState
126−inspect $ 'readWithFromBytesNull `hasNoType` ''MutArray.ArrayUnsafe -- FH.fromBytes/S.chunksOf
126+inspect $ 'readWithFromBytesNull `hasNoType` ''MAS.SpliceState
127+inspect $ 'readWithFromBytesNull `hasNoType` ''AT.ArrayUnsafe -- FH.fromBytes/S.chunksOf
127128inspect $ 'readWithFromBytesNull `hasNoType` ''D.FoldMany
128129#endif
129130
@@ -170,7 +171,7 @@ writeReadWith inh devNull = IUF.fold fld unf (defaultChunkSize, inh)
170171inspect $ hasNoTypeClasses 'writeReadWith
171172inspect $ 'writeReadWith `hasNoType` ''Step
172173inspect $ 'writeReadWith `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
173−inspect $ 'writeReadWith `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
174+inspect $ 'writeReadWith `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
174175 -- FH.read/A.read
175176#endif
176177
@@ -187,7 +188,7 @@ writeRead inh devNull = IUF.fold fld unf inh
187188inspect $ hasNoTypeClasses 'writeRead
188189inspect $ 'writeRead `hasNoType` ''Step
189190inspect $ 'writeRead `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
190−inspect $ 'writeRead `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
191+inspect $ 'writeRead `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
191192 -- FH.read/A.read
192193#endif
193194
benchmark/Streamly/Benchmark/Prelude/Adaptive.hs+1−1
@@ -10,7 +10,7 @@
1010import Control.Concurrent (threadDelay)
1111import Control.Monad (when)
1212import Control.Monad.IO.Class (liftIO)
13−import Test.Tasty.Bench
13+import Gauge
1414import Streamly.Prelude as S
1515import System.Random (randomRIO)
1616
benchmark/Streamly/Benchmark/Prelude/Ahead.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.Ahead"
benchmark/Streamly/Benchmark/Prelude/Async.hs+1−1
@@ -16,7 +16,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream.Transform as Transform
1616import Streamly.Benchmark.Common
1717import Streamly.Benchmark.Prelude
1818
19−import Test.Tasty.Bench
19+import Gauge
2020
2121moduleName :: String
2222moduleName = "Prelude.Async"
benchmark/Streamly/Benchmark/Prelude/Concurrent.hs+1−1
@@ -15,7 +15,7 @@ import Streamly.Prelude
1515 , fromWAsync, parallel, fromParallel
1616 )
1717
18−import Test.Tasty.Bench
18+import Gauge
1919import qualified Streamly.Prelude as S
2020
2121-------------------------------------------------------------------------------
benchmark/Streamly/Benchmark/Prelude/Merge.hs+1−1
@@ -30,7 +30,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream as Stream
3030import Streamly.Benchmark.Common
3131import Streamly.Benchmark.Prelude
3232
33−import Test.Tasty.Bench
33+import Gauge
3434
3535#ifdef INSPECTION
3636import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Prelude/Parallel.hs+1−1
@@ -22,7 +22,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream as Internal
2222import Streamly.Benchmark.Common
2323import Streamly.Benchmark.Prelude
2424
25−import Test.Tasty.Bench
25+import Gauge
2626
2727moduleName :: String
2828moduleName = "Prelude.Parallel"
benchmark/Streamly/Benchmark/Prelude/Rate.hs+1−1
@@ -14,7 +14,7 @@ import qualified Streamly.Prelude as S
1414import Streamly.Benchmark.Common
1515import Streamly.Benchmark.Prelude
1616
17−import Test.Tasty.Bench
17+import Gauge
1818
1919moduleName :: String
2020moduleName = "Prelude.Rate"
benchmark/Streamly/Benchmark/Prelude/WAsync.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.WAsync"
benchmark/Streamly/Benchmark/Prelude/WSerial.hs+1−1
@@ -29,7 +29,7 @@ import qualified Streamly.Internal.Data.Unfold as UF
2929import Streamly.Benchmark.Common
3030import Streamly.Benchmark.Prelude
3131
32−import Test.Tasty.Bench
32+import Gauge
3333
3434#ifdef INSPECTION
3535import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Prelude/ZipAsync.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.ZipAsync"
benchmark/Streamly/Benchmark/Prelude/ZipSerial.hs+1−1
@@ -29,7 +29,7 @@ import qualified Streamly.Prelude as S
2929import Streamly.Benchmark.Common
3030import Streamly.Benchmark.Prelude hiding (sourceUnfoldrM)
3131
32−import Test.Tasty.Bench
32+import Gauge
3333
3434#ifdef INSPECTION
3535import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Unicode/Char.hs+1−1
@@ -16,7 +16,7 @@ import Control.DeepSeq (NFData(..))
1616import Streamly.Internal.Data.Array (Array)
1717import System.FilePath (dropExtensions, takeFileName)
1818import System.FilePath.Posix ((</>))
19−import Test.Tasty.Bench (Benchmark, bench, bgroup, defaultMain, env, nfIO)
19+import Gauge.Main (Benchmark, bench, bgroup, defaultMain, env, nfIO)
2020import Streamly.Internal.Unicode.Char
2121 ( NormalizationMode(NFC, NFD, NFKC, NFKD)
2222 , normalize
benchmark/Streamly/Benchmark/Unicode/Parser.hs+1−1
@@ -31,7 +31,7 @@ import qualified Streamly.Data.Array as Array
3131import qualified Streamly.Data.Stream as Stream
3232import qualified Streamly.Internal.Unicode.Parser as PRU
3333
34−import Test.Tasty.Bench hiding (env)
34+import Gauge hiding (env)
3535import Streamly.Benchmark.Common
3636
3737runParser :: Int -> (Stream IO Char -> IO a) -> IO ()
benchmark/Streamly/Benchmark/Unicode/Stream.hs+7−25
@@ -11,11 +11,6 @@
1111{-# LANGUAGE CPP #-}
1212{-# LANGUAGE ScopedTypeVariables #-}
1313
14−#undef FUSION_CHECK
15−#ifdef FUSION_CHECK
16−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
17−#endif
18−
1914#ifdef __HADDOCK_VERSION__
2015#undef INSPECTION
2116#endif
@@ -38,17 +33,16 @@ import qualified Streamly.Internal.FileSystem.Handle as Handle
3833import qualified Streamly.Internal.Unicode.Array as UnicodeArr
3934import qualified Streamly.Internal.Unicode.Stream as Unicode
4035
41−import Test.Tasty.Bench hiding (env)
36+import Gauge hiding (env)
4237import Streamly.Benchmark.Common
4338import Streamly.Benchmark.Common.Handle
4439
4540#ifdef INSPECTION
46−import Streamly.Internal.Data.MutByteArray (Unbox)
47−import Streamly.Internal.Data.Stream (Step(..))
48−import qualified Streamly.Internal.Data.Array as Array
49−import qualified Streamly.Internal.Data.MutArray as MutArray
41+import Streamly.Internal.Data.Unbox (Unbox)
42+import Streamly.Internal.Data.Stream.StreamD (Step(..))
5043import qualified Streamly.Internal.Data.Fold as Fold
5144import qualified Streamly.Internal.Data.Tuple.Strict as Strict
45+import qualified Streamly.Internal.Data.Array.Type as Array
5246
5347import Test.Inspection
5448#endif
@@ -224,7 +218,7 @@ inspect $ 'copyStreamLatin1' `hasNoType` ''Step
224218inspect $ 'copyStreamLatin1' `hasNoType` ''Unfold.ConcatState -- Handle.read/UF.many
225219
226220inspect $ 'copyStreamLatin1' `hasNoType` ''Fold.Step
227−inspect $ 'copyStreamLatin1' `hasNoType` ''MutArray.ArrayUnsafe -- Handle.write/writeNUnsafe
221+inspect $ 'copyStreamLatin1' `hasNoType` ''Array.ArrayUnsafe -- Handle.write/writeNUnsafe
228222 -- Handle.read/Array.read
229223inspect $ 'copyStreamLatin1' `hasNoType` ''Strict.Tuple3' -- Handle.write/chunksOf
230224#endif
@@ -245,7 +239,7 @@ inspect $ 'copyStreamLatin1 `hasNoType` ''Unfold.ConcatState -- Handle.read/UF.m
245239
246240inspect $ 'copyStreamLatin1 `hasNoType` ''Fold.ManyState
247241inspect $ 'copyStreamLatin1 `hasNoType` ''Fold.Step
248−inspect $ 'copyStreamLatin1 `hasNoType` ''MutArray.ArrayUnsafe -- Handle.write/writeNUnsafe
242+inspect $ 'copyStreamLatin1 `hasNoType` ''Array.ArrayUnsafe -- Handle.write/writeNUnsafe
249243 -- Handle.read/Array.read
250244inspect $ 'copyStreamLatin1 `hasNoType` ''Strict.Tuple3' -- Handle.write/chunksOf
251245#endif
@@ -286,8 +280,7 @@ _copyStreamUtf8'Fold :: Handle -> Handle -> IO ()
286280_copyStreamUtf8'Fold inh outh =
287281 Stream.fold (Handle.write outh)
288282 $ Unicode.encodeUtf8
289− $ Stream.catRights
290− $ Stream.parseMany Unicode.writeCharUtf8'
283+ $ Stream.foldMany Unicode.writeCharUtf8'
291284 $ Stream.unfold Handle.reader inh
292285
293286{-# NOINLINE _copyStreamUtf8Parser #-}
@@ -324,7 +317,6 @@ o_1_space_decode_encode_read env =
324317
325318main :: IO ()
326319main = do
327−#ifndef FUSION_CHECK
328320 env <- mkHandleBenchEnv
329321 defaultMain (allBenchmarks env)
330322
@@ -337,13 +329,3 @@ main = do
337329 , o_1_space_decode_encode_read env
338330 ]
339331 ]
340−#else
341− -- Enable FUSION_CHECK macro at the beginning of the file
342− -- Enable one benchmark below, and run the benchmark
343− -- Check the .dump-simpl output
344− env <- mkHandleBenchEnv
345− let mkHandles (RefHandles {bigInH = inh, outputH = outh}) = Handles inh outh
346− (Handles inh outh) <- getHandles env mkHandles
347− copyStreamLatin1' inh outh
348− return ()
349−#endif
benchmark/Streamly/Benchmark/Unicode/Utf8.hs+1−1
@@ -13,7 +13,7 @@
1313import Data.Char (chr)
1414import Streamly.Internal.Unicode.Utf8 (Utf8)
1515
16−import Test.Tasty.Bench hiding (env)
16+import Gauge hiding (env)
1717import Streamly.Benchmark.Common
1818
1919import qualified Streamly.Internal.Unicode.Utf8 as Utf8
benchmark/bench-runner/Main.hs+6−34
@@ -16,11 +16,9 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
1616 benchName = drop 4 benchName0
1717
1818 general
19− -- GHC-9.6 requires 32M heap vs 16M for GHC-9.4
20− | "o-1-sp" `isInfixOf` benchName = "-K36K -M32M"
19+ | "o-1-sp" `isInfixOf` benchName = "-K36K -M16M"
2120 | "o-n-h" `isInfixOf` benchName = "-K36K -M32M"
22− -- GHC-9.6 requires 32M heap vs 16M for GHC-9.4
23− | "o-n-st" `isInfixOf` benchName = "-K1M -M32M"
21+ | "o-n-st" `isInfixOf` benchName = "-K1M -M16M"
2422 | "o-n-sp" `isInfixOf` benchName = "-K1M -M32M"
2523 | otherwise = ""
2624
@@ -29,17 +27,6 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
2927 | otherwise = ""
3028
3129 benchSpecific
32− -- GHC-9.6 requires 64M, earlier it was 16M
33− | "Data.Fold/o-n-heap.key-value.toHashMapIO (max buckets) sum" == benchName =
34− "-M64M"
35−
36− -- This is required only for the --long case because we allocate all
37− -- the arrays upfront. depends on the size of the stream.
38− | "Data.Parser/o-1-space" `isPrefixOf` benchName =
39− "-M128M"
40−
41− -----------------------------------------------------------------------
42−
4330 | "Prelude.Parallel/o-n-heap.mapping.mapM" == benchName = "-M256M"
4431 | "Prelude.Parallel/o-n-heap.monad-outer-product."
4532 `isPrefixOf` benchName = "-M256M"
@@ -70,18 +57,12 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
7057 -- GHC 9.4.4 requires 4M
7158 | "Data.Stream.StreamDK/o-n-space.traversable."
7259 `isPrefixOf` benchName = "-K4M"
73− -- GHC-9.6 requires 64M, earlier it was 32M
74− | "Data.Stream.StreamDK/o-n-heap.buffered.readsPrec pure streams" == benchName =
75− "-M64M"
76− -- GHC-9.6 requires 64M, earlier it was 32M
77− | "Data.Stream.StreamDK/o-n-heap.buffered.showPrec Haskell lists" == benchName =
78− "-M64M"
7960
8061 -----------------------------------------------------------------------
8162
82− | "Data.StreamD/o-n-space.elimination.toList" == benchName =
63+ | "Data.Stream.StreamD/o-n-space.elimination.toList" == benchName =
8364 "-K2M"
84− | "Data.StreamK/o-n-space.elimination.toList" == benchName =
65+ | "Data.Stream.StreamK/o-n-space.elimination.toList" == benchName =
8566 "-K2M"
8667
8768 -----------------------------------------------------------------------
@@ -93,13 +74,6 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
9374 | "Data.Stream/o-n-space.iterated."
9475 `isPrefixOf` benchName = "-K4M"
9576
96− -- GHC-9.6 requires 64M, earlier it was 32M
97− | "Data.Stream/o-n-heap.buffered.showPrec Haskell lists" == benchName =
98− "-M64M"
99− -- GHC-9.6 requires 64M, earlier it was 32M
100− | "Data.Stream/o-n-heap.buffered.readsPrec pure streams" == benchName =
101− "-M64M"
102−
10377 | "Data.Stream.ConcurrentEager/o-n-heap.monad-outer-product.toNullAp"
10478 `isPrefixOf` benchName = "-M1024M"
10579 | "Data.Stream.ConcurrentEager/o-1-space."
@@ -112,12 +86,10 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
11286
11387 | "Data.Array" `isPrefixOf` benchName
11488 && "/o-1-space.generation.read" `isSuffixOf` benchName = "-M32M"
115− -- XXX GHC 9.6 onwards needs 64M, earlier it was 32M
11689 | "Data.Array" `isPrefixOf` benchName
117− && "/o-1-space.generation.show" `isSuffixOf` benchName = "-M64M"
118− -- XXX GHC 9.6 onwards needs 64M, earlier it was 32M
90+ && "/o-1-space.generation.show" `isSuffixOf` benchName = "-M32M"
11991 | "Data.Array.Generic/o-1-space.transformationX4.map"
120− `isPrefixOf` benchName = "-M64M"
92+ `isPrefixOf` benchName = "-M32M"
12193
12294 -- XXX For --long option, need to check why so much heap is required.
12395 -- Note, if we remove the chunked stream module we need to keep the
benchmark/lib/Streamly/Benchmark/Common.hs+1−1
@@ -56,7 +56,7 @@ import System.Random (randomRIO)
5656import qualified Streamly.Internal.Data.Fold as Fold
5757import qualified Streamly.Internal.Data.Stream as S
5858
59−import Test.Tasty.Bench
59+import Gauge
6060
6161-------------------------------------------------------------------------------
6262-- Benchmark Prefixes
benchmark/lib/Streamly/Benchmark/Common/Handle.hs+1−3
@@ -30,8 +30,6 @@ module Streamly.Benchmark.Common.Handle
3030 , isSpace
3131 , isSp
3232 , mkHandleBenchEnv
33− , Handles(..)
34− , getHandles
3533 )
3634where
3735
@@ -48,7 +46,7 @@ import System.Process (callCommand)
4846
4947import Data.IORef
5048import Prelude hiding (last, length)
51−import Test.Tasty.Bench hiding (env)
49+import Gauge hiding (env)
5250
5351
5452scratchDir :: String
benchmark/lib/Streamly/Benchmark/Prelude.hs+1−1
@@ -82,7 +82,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream.Type as IsStream
8282import qualified Streamly.Internal.Data.Pipe as Pipe
8383import qualified Streamly.Internal.Data.Stream.Serial as Serial
8484
85−import Test.Tasty.Bench
85+import Gauge
8686import Streamly.Internal.Data.Time.Units
8787
8888-- Common polymorphic stream APIs used across multiple stream modules
benchmark/streamly-benchmarks.cabal+41−58
@@ -54,6 +54,11 @@ flag opt
5454 manual: True
5555 default: True
5656
57+flag use-gauge
58+ description: Use gauge instead of tasty-bench for benchmarking
59+ manual: True
60+ default: False
61+
5762flag use-streamly-core
5863 description: Benchmark only core package
5964 manual: True
@@ -140,10 +145,6 @@ common compile-options
140145 -Wno-missing-kind-signatures
141146 -Wno-operator-whitespace
142147
143− if impl(ghc >= 9.8)
144− ghc-options:
145− -Wno-missing-role-annotations
146−
147148 if flag(has-llvm)
148149 ghc-options: -fllvm
149150
@@ -151,17 +152,15 @@ common compile-options
151152 ghc-options: -Wmissed-specialisations
152153 -Wall-missed-specialisations
153154
155+ if flag(dev) || flag(debug)
156+ ghc-options: -fno-ignore-asserts
157+
154158common optimization-options
155159 if flag(opt)
156160 ghc-options: -O2
157161 -fdicts-strict
158162 -fspec-constr-recursive=16
159163 -fmax-worker-args=16
160−
161− -- For this to be effective it must come after the -O2 option
162− if flag(dev) || flag(debug)
163− ghc-options: -fno-ignore-asserts
164−
165164 if flag(fusion-plugin) && !impl(ghcjs) && !impl(ghc < 8.6)
166165 ghc-options: -fplugin Fusion.Plugin
167166 else
@@ -175,34 +174,43 @@ common bench-depends
175174 -- Core libraries shipped with ghc, the min and max
176175 -- constraints of these libraries should match with
177176 -- the GHC versions we support
178− base >= 4.9 && < 4.20
179− , deepseq >= 1.4.1 && < 1.6
177+ base >= 4.9 && < 4.19
178+ , deepseq >= 1.4.1 && < 1.5
180179 , mtl >= 2.2 && < 2.3.2
181180
182181 -- other libraries
183− , exceptions >= 0.8 && < 0.11
184182 , random >= 1.0 && < 2.0
185183 , transformers >= 0.4 && < 0.7
186− , containers >= 0.5 && < 0.8
184+ , containers >= 0.5 && < 0.7
187185 , hashable >= 1.3 && < 1.5
188186 , unordered-containers >= 0.2 && < 0.3
189187 , process >= 1.4 && < 1.7
190188 , directory >= 1.2.2 && < 1.4
191− , filepath >= 1.4.1 && < 1.6
192− , ghc-prim >= 0.4 && < 0.12
193− , tasty-bench >= 0.3 && < 0.4
194− , tasty >= 1.4.1 && < 1.6
195− , streamly-core
189+ , filepath >= 1.4.1 && < 1.5
190+ , ghc-prim >= 0.4 && < 0.11
196191
197− if !flag(use-streamly-core)
192+ if flag(use-streamly-core)
193+ build-depends: streamly-core
194+ else
198195 build-depends: streamly
196+ , streamly-core
197+
198+ if flag(use-gauge)
199+ build-depends: gauge >= 0.2.4 && < 0.3
200+ else
201+ build-depends: tasty-bench >= 0.3 && < 0.4
202+ , tasty >= 1.4.1 && < 1.5
203+ mixins: tasty-bench
204+ (Test.Tasty.Bench as Gauge
205+ , Test.Tasty.Bench as Gauge.Main
206+ )
199207
200208 if flag(fusion-plugin) && !impl(ghcjs) && !impl(ghc < 8.6)
201209 build-depends:
202210 fusion-plugin >= 0.2 && < 0.3
203211 if flag(inspection)
204− build-depends: template-haskell >= 2.14 && < 2.22
205− , inspection-testing >= 0.4 && < 0.6
212+ build-depends: template-haskell >= 2.14 && < 2.21
213+ , inspection-testing >= 0.4 && < 0.5
206214 -- Array uses a Storable constraint in dev build making several inspection
207215 -- tests fail
208216 if flag(dev) && flag(inspection)
@@ -352,30 +360,14 @@ benchmark Data.Unbox
352360 import: bench-options
353361 type: exitcode-stdio-1.0
354362 hs-source-dirs: .
355− cpp-options: -DUSE_UNBOX
356− main-is: Streamly/Benchmark/Data/Serialize.hs
363+ main-is: Streamly/Benchmark/Data/Unbox.hs
357364
358365benchmark Data.Unbox.Derive.TH
359366 import: bench-options
360367 type: exitcode-stdio-1.0
361368 hs-source-dirs: .
362− cpp-options: -DUSE_UNBOX
363369 cpp-options: -DUSE_TH
364− main-is: Streamly/Benchmark/Data/Serialize.hs
365−
366−benchmark Data.Serialize
367− import: bench-options
368− type: exitcode-stdio-1.0
369− hs-source-dirs: .
370− cpp-options: -DUSE_TH
371− main-is: Streamly/Benchmark/Data/Serialize.hs
372− build-depends: QuickCheck, template-haskell
373− other-modules:
374− Streamly.Benchmark.Data.Serialize.TH
375− Streamly.Benchmark.Data.Serialize.RecCompatible
376− Streamly.Benchmark.Data.Serialize.RecNonCompatible
377− if flag(limit-build-mem)
378− ghc-options: +RTS -M1000M -RTS
370+ main-is: Streamly/Benchmark/Data/Unbox.hs
379371
380372benchmark Data.Unfold
381373 import: bench-options
@@ -427,7 +419,6 @@ benchmark Data.ParserK
427419 import: bench-options
428420 type: exitcode-stdio-1.0
429421 hs-source-dirs: Streamly/Benchmark/Data
430− cpp-options: -DBENCH_SINGULAR
431422 main-is: ParserK.hs
432423 if impl(ghcjs)
433424 buildable: False
@@ -435,25 +426,17 @@ benchmark Data.ParserK
435426 buildable: True
436427 build-depends: exceptions >= 0.8 && < 0.11
437428
438−benchmark Data.ParserK.Chunked
429+-- Note: to use this we have to set DISABLE_FUSION in ParserD.hs so that
430+-- the rewrite rules are disabled. We can also use the "no-fusion" build
431+-- flag but we need to keep in mind that it disables fusion for streams
432+-- as well.
433+benchmark Data.FromParserK
439434 import: bench-options
440435 type: exitcode-stdio-1.0
441436 hs-source-dirs: Streamly/Benchmark/Data
442− cpp-options: -DBENCH_CHUNKED
437+ cpp-options: -DFROM_PARSERK
443438 main-is: ParserK.hs
444− if impl(ghcjs)
445− buildable: False
446− else
447− buildable: True
448− build-depends: exceptions >= 0.8 && < 0.11
449−
450−benchmark Data.ParserK.Chunked.Generic
451− import: bench-options
452− type: exitcode-stdio-1.0
453− hs-source-dirs: Streamly/Benchmark/Data
454− cpp-options: -DBENCH_CHUNKED_GENERIC
455− main-is: ParserK.hs
456− if impl(ghcjs)
439+ if flag(use-streamly-core) || impl(ghcjs)
457440 buildable: False
458441 else
459442 buildable: True
@@ -463,7 +446,7 @@ benchmark Data.ParserK.Chunked.Generic
463446-- Raw Streams
464447-------------------------------------------------------------------------------
465448
466−benchmark Data.StreamD
449+benchmark Data.Stream.StreamD
467450 import: bench-options
468451 type: exitcode-stdio-1.0
469452 hs-source-dirs: Streamly/Benchmark/Data/Stream
@@ -471,7 +454,7 @@ benchmark Data.StreamD
471454 if impl(ghcjs)
472455 buildable: False
473456
474−benchmark Data.StreamK
457+benchmark Data.Stream.StreamK
475458 import: bench-options
476459 type: exitcode-stdio-1.0
477460 hs-source-dirs: Streamly/Benchmark/Data/Stream
@@ -487,7 +470,7 @@ benchmark Data.Stream.ToStreamK
487470 if !flag(dev) || impl(ghcjs)
488471 buildable: False
489472
490−benchmark Data.StreamK.Alt
473+benchmark Data.Stream.StreamK.Alt
491474 import: bench-options
492475 type: exitcode-stdio-1.0
493476 hs-source-dirs: Streamly/Benchmark/Data/Stream
configure+9−9
@@ -1,6 +1,6 @@
11#! /bin/sh
22# Guess values for system-dependent variables and create Makefiles.
3−# Generated by GNU Autoconf 2.71 for streamly 0.10.1.
3+# Generated by GNU Autoconf 2.71 for streamly 0.10.0.
44#
55# Report bugs to <streamly@composewell.com>.
66#
@@ -610,8 +610,8 @@ MAKEFLAGS=
610610# Identity of this package.
611611PACKAGE_NAME='streamly'
612612PACKAGE_TARNAME='streamly'
613−PACKAGE_VERSION='0.10.1'
614−PACKAGE_STRING='streamly 0.10.1'
613+PACKAGE_VERSION='0.10.0'
614+PACKAGE_STRING='streamly 0.10.0'
615615PACKAGE_BUGREPORT='streamly@composewell.com'
616616PACKAGE_URL='https://streamly.composewell.com'
617617
@@ -1224,7 +1224,7 @@ if test "$ac_init_help" = "long"; then
12241224 # Omit some internal or obsolete options to make the list less imposing.
12251225 # This message is too long to be a string in the A/UX 3.1 sh.
12261226 cat <<_ACEOF
1227−\`configure' configures streamly 0.10.1 to adapt to many kinds of systems.
1227+\`configure' configures streamly 0.10.0 to adapt to many kinds of systems.
12281228
12291229Usage: $0 [OPTION]... [VAR=VALUE]...
12301230
@@ -1286,7 +1286,7 @@ fi
12861286
12871287if test -n "$ac_init_help"; then
12881288 case $ac_init_help in
1289− short | recursive ) echo "Configuration of streamly 0.10.1:";;
1289+ short | recursive ) echo "Configuration of streamly 0.10.0:";;
12901290 esac
12911291 cat <<\_ACEOF
12921292
@@ -1372,7 +1372,7 @@ fi
13721372test -n "$ac_init_help" && exit $ac_status
13731373if $ac_init_version; then
13741374 cat <<\_ACEOF
1375−streamly configure 0.10.1
1375+streamly configure 0.10.0
13761376generated by GNU Autoconf 2.71
13771377
13781378Copyright (C) 2021 Free Software Foundation, Inc.
@@ -1500,7 +1500,7 @@ cat >config.log <<_ACEOF
15001500This file contains any messages produced by compilers while
15011501running configure, to aid debugging if configure makes a mistake.
15021502
1503−It was created by streamly $as_me 0.10.1, which was
1503+It was created by streamly $as_me 0.10.0, which was
15041504generated by GNU Autoconf 2.71. Invocation command line was
15051505
15061506 $ $0$ac_configure_args_raw
@@ -3800,7 +3800,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
38003800# report actual input values of CONFIG_FILES etc. instead of their
38013801# values after options handling.
38023802ac_log="
3803−This file was extended by streamly $as_me 0.10.1, which was
3803+This file was extended by streamly $as_me 0.10.0, which was
38043804generated by GNU Autoconf 2.71. Invocation command line was
38053805
38063806 CONFIG_FILES = $CONFIG_FILES
@@ -3856,7 +3856,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\
38563856cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
38573857ac_cs_config='$ac_cs_config_escaped'
38583858ac_cs_version="\\
3859−streamly config.status 0.10.1
3859+streamly config.status 0.10.0
38603860configured by $0, generated by GNU Autoconf 2.71,
38613861 with options \\"\$ac_cs_config\\"
38623862
configure.ac+1−1
@@ -3,7 +3,7 @@
33# See https://www.gnu.org/software/autoconf/manual/autoconf.html for help on
44# the macros used in this file.
55
6−AC_INIT([streamly], [0.10.1], [streamly@composewell.com], [streamly], [https://streamly.composewell.com])
6+AC_INIT([streamly], [0.10.0], [streamly@composewell.com], [streamly], [https://streamly.composewell.com])
77
88# To suppress "WARNING: unrecognized options: --with-compiler"
99AC_ARG_WITH([compiler], [GHC])
core/configure+9−9
@@ -1,6 +1,6 @@
11#! /bin/sh
22# Guess values for system-dependent variables and create Makefiles.
3−# Generated by GNU Autoconf 2.71 for streamly-core 0.2.2.
3+# Generated by GNU Autoconf 2.71 for streamly-core 0.2.0.
44#
55# Report bugs to <streamly@composewell.com>.
66#
@@ -610,8 +610,8 @@ MAKEFLAGS=
610610# Identity of this package.
611611PACKAGE_NAME='streamly-core'
612612PACKAGE_TARNAME='streamly-core'
613−PACKAGE_VERSION='0.2.2'
614−PACKAGE_STRING='streamly-core 0.2.2'
613+PACKAGE_VERSION='0.2.0'
614+PACKAGE_STRING='streamly-core 0.2.0'
615615PACKAGE_BUGREPORT='streamly@composewell.com'
616616PACKAGE_URL='https://streamly.composewell.com'
617617
@@ -1256,7 +1256,7 @@ if test "$ac_init_help" = "long"; then
12561256 # Omit some internal or obsolete options to make the list less imposing.
12571257 # This message is too long to be a string in the A/UX 3.1 sh.
12581258 cat <<_ACEOF
1259−\`configure' configures streamly-core 0.2.2 to adapt to many kinds of systems.
1259+\`configure' configures streamly-core 0.2.0 to adapt to many kinds of systems.
12601260
12611261Usage: $0 [OPTION]... [VAR=VALUE]...
12621262
@@ -1318,7 +1318,7 @@ fi
13181318
13191319if test -n "$ac_init_help"; then
13201320 case $ac_init_help in
1321− short | recursive ) echo "Configuration of streamly-core 0.2.2:";;
1321+ short | recursive ) echo "Configuration of streamly-core 0.2.0:";;
13221322 esac
13231323 cat <<\_ACEOF
13241324
@@ -1404,7 +1404,7 @@ fi
14041404test -n "$ac_init_help" && exit $ac_status
14051405if $ac_init_version; then
14061406 cat <<\_ACEOF
1407−streamly-core configure 0.2.2
1407+streamly-core configure 0.2.0
14081408generated by GNU Autoconf 2.71
14091409
14101410Copyright (C) 2021 Free Software Foundation, Inc.
@@ -1622,7 +1622,7 @@ cat >config.log <<_ACEOF
16221622This file contains any messages produced by compilers while
16231623running configure, to aid debugging if configure makes a mistake.
16241624
1625−It was created by streamly-core $as_me 0.2.2, which was
1625+It was created by streamly-core $as_me 0.2.0, which was
16261626generated by GNU Autoconf 2.71. Invocation command line was
16271627
16281628 $ $0$ac_configure_args_raw
@@ -3832,7 +3832,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
38323832# report actual input values of CONFIG_FILES etc. instead of their
38333833# values after options handling.
38343834ac_log="
3835−This file was extended by streamly-core $as_me 0.2.2, which was
3835+This file was extended by streamly-core $as_me 0.2.0, which was
38363836generated by GNU Autoconf 2.71. Invocation command line was
38373837
38383838 CONFIG_FILES = $CONFIG_FILES
@@ -3888,7 +3888,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\
38883888cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
38893889ac_cs_config='$ac_cs_config_escaped'
38903890ac_cs_version="\\
3891−streamly-core config.status 0.2.2
3891+streamly-core config.status 0.2.0
38923892configured by $0, generated by GNU Autoconf 2.71,
38933893 with options \\"\$ac_cs_config\\"
38943894
core/configure.ac+1−1
@@ -3,7 +3,7 @@
33# See https://www.gnu.org/software/autoconf/manual/autoconf.html for help on
44# the macros used in this file.
55
6−AC_INIT([streamly-core], [0.2.2], [streamly@composewell.com], [streamly-core], [https://streamly.composewell.com])
6+AC_INIT([streamly-core], [0.2.0], [streamly@composewell.com], [streamly-core], [https://streamly.composewell.com])
77
88# To suppress "WARNING: unrecognized options: --with-compiler"
99AC_ARG_WITH([compiler], [GHC])
core/docs/ApiChangelogs/0.1.0-0.2.0.txtdeleted−1471
@@ -1,1471 +0,0 @@
1−---------------------------------
2−API Annotations
3−---------------------------------
4−
5−[A] : Added
6−[R] : Removed
7−[C] : Changed
8−[O] : Old definition
9−[N] : New definition
10−[D] : Deprecated
11−
12−---------------------------------
13−API diff
14−---------------------------------
15−
16−[C] Streamly.Unicode.Parser
17− [A] double :: Monad m => Parser Char m Double
18−[C] Streamly.FileSystem.Handle
19− [A] readWith :: MonadIO m => Int -> Handle -> Stream m Word8
20− [A] readChunksWith :: MonadIO m => Int -> Handle -> Stream m (Array Word8)
21−[C] Streamly.FileSystem.Dir
22− [C] readEither
23− [O] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
24− [N] readEither :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
25− [C] read
26− [O] read :: MonadIO m => FilePath -> Stream m FilePath
27− [N] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
28−[C] Streamly.Data.Unfold
29− [A] class Enum a => Enumerable a
30− [A] second :: b -> Unfold m (a, b) c -> Unfold m a c
31− [A] first :: a -> Unfold m (a, b) c -> Unfold m b c
32− [A] enumerateFromTo :: (Enumerable a, Monad m) => Unfold m (a, a) a
33− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => Unfold m (a, a, a) a
34− [A] enumerateFromThen :: (Enumerable a, Monad m) => Unfold m (a, a) a
35− [A] enumerateFrom :: (Enumerable a, Monad m) => Unfold m a a
36−[C] Streamly.Data.StreamK
37− [C] parseChunks
38− [O] parseChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b)
39− [N] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
40− [C] parseBreakChunks
41− [O] parseBreakChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
42− [N] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
43− [A] parseBreak :: forall m a b. Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
44− [A] parse :: Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b)
45− [A] handle :: (MonadCatch m, Exception e) => (e -> m (StreamK m a)) -> StreamK m a -> StreamK m a
46− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> StreamK m a) -> StreamK m a
47−[C] Streamly.Data.Stream
48− [A] wordsBy :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
49− [A] splitOn :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
50− [C] handle
51− [O] handle :: (MonadCatch m, Exception e) => (e -> Stream m a) -> Stream m a -> Stream m a
52− [N] handle :: (MonadCatch m, Exception e) => (e -> m (Stream m a)) -> Stream m a -> Stream m a
53− [A] groupsOf :: Monad m => Int -> Fold m a b -> Stream m a -> Stream m b
54−[C] Streamly.Data.ParserK
55− [D] fromParser :: (MonadIO m, Unbox a) => Parser a m b -> ParserK (Array a) m b
56− [D] fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK (Array a) m b
57− [A] adaptCG :: Monad m => Parser a m b -> ParserK (Array a) m b
58− [A] adaptC :: (Monad m, Unbox a) => Parser a m b -> ParserK (Array a) m b
59− [A] adapt :: Monad m => Parser a m b -> ParserK a m b
60−[C] Streamly.Data.Parser
61− [A] groupByRollingEither :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
62− [A] groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
63−[A] Streamly.Data.MutByteArray
64− [A] class Unbox a
65− [A] class Serialize a
66− [A] SerializeConfig
67− [A] MutByteArray
68− [A] unpin :: MutByteArray -> IO MutByteArray
69− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
70− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
71− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
72− [A] pinnedNew :: Int -> IO MutByteArray
73− [A] pin :: MutByteArray -> IO MutByteArray
74− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
75− [A] new :: Int -> IO MutByteArray
76− [A] isPinned :: MutByteArray -> Bool
77− [A] inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
78− [A] inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
79− [A] inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig
80− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
81− [A] deriveUnbox :: Q [Dec] -> Q [Dec]
82− [A] deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]
83− [A] deriveSerialize :: Q [Dec] -> Q [Dec]
84− [A] addSizeTo :: Serialize a => Int -> a -> Int
85−[C] Streamly.Data.MutArray.Generic
86− [A] write :: MonadIO m => Fold m a (MutArray a)
87− [A] readRev :: MonadIO m => MutArray a -> Stream m a
88− [A] read :: MonadIO m => MutArray a -> Stream m a
89− [A] putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
90− [A] modifyIndexUnsafe :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
91− [A] length :: MutArray a -> Int
92− [A] getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
93− [C] getIndex
94− [O] getIndex :: MonadIO m => Int -> MutArray a -> m a
95− [N] getIndex :: MonadIO m => Int -> MutArray a -> m (Maybe a)
96− [A] fromListN :: MonadIO m => Int -> [a] -> m (MutArray a)
97− [A] fromList :: MonadIO m => [a] -> m (MutArray a)
98−[C] Streamly.Data.MutArray
99− [A] unpin :: MutArray a -> IO (MutArray a)
100− [A] readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
101− [A] read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
102− [A] putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
103− [D] pokeByteIndex :: Unbox a => Int -> MutByteArray -> a -> IO ()
104− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
105− [A] pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
106− [A] pin :: MutArray a -> IO (MutArray a)
107− [D] peekByteIndex :: Unbox a => Int -> MutByteArray -> IO a
108− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
109− [D] newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
110− [A] modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
111− [A] modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
112− [A] modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()
113− [A] isPinned :: MutArray a -> Bool
114− [A] getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
115− [C] getIndex
116− [O] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
117− [N] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
118−[C] Streamly.Data.Array.Generic
119− [A] toList :: Array a -> [a]
120− [A] getIndex :: Int -> Array a -> Maybe a
121−[C] Streamly.Data.Array
122− [A] unpin :: Array a -> IO (Array a)
123− [D] pokeByteIndex :: Unbox a => Int -> MutByteArray -> a -> IO ()
124− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
125− [A] pin :: Array a -> IO (Array a)
126− [D] peekByteIndex :: Unbox a => Int -> MutByteArray -> IO a
127− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
128− [A] isPinned :: Array a -> Bool
129−[C] Streamly.Console.Stdio
130−
131−---------------------------------
132−Internal API diff
133−---------------------------------
134−
135−[C] Streamly.Internal.Unicode.Stream
136− [C] writeCharUtf8'
137− [O] writeCharUtf8' :: Monad m => Fold m Word8 Char
138− [N] writeCharUtf8' :: Monad m => Parser Word8 m Char
139−[C] Streamly.Internal.Unicode.Parser
140− [A] number :: Monad m => Parser Char m (Integer, Int)
141− [A] mkDouble :: Integer -> Int -> Double
142− [A] doubleParser :: Monad m => Parser Char m (Int, Int)
143− [C] double
144− [O] double :: Parser Char m Double
145− [N] double :: Monad m => Parser Char m Double
146−[R] Streamly.Internal.Serialize.ToBytes
147−[R] Streamly.Internal.Serialize.FromBytes
148−[C] Streamly.Internal.FileSystem.File
149− [A] writeAppendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()
150− [A] writeAppendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()
151− [A] writeAppendArray :: FilePath -> Array a -> IO ()
152− [A] writeAppend :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()
153− [R] appendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()
154− [R] appendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()
155− [R] appendArray :: FilePath -> Array a -> IO ()
156− [R] append :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()
157−[C] Streamly.Internal.FileSystem.Dir
158− [C] reader
159− [O] reader :: MonadIO m => Unfold m FilePath FilePath
160− [N] reader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
161− [C] readFiles
162− [O] readFiles :: MonadIO m => FilePath -> Stream m FilePath
163− [N] readFiles :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
164− [C] readEitherPaths
165− [O] readEitherPaths :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
166− [N] readEitherPaths :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
167− [C] readEither
168− [O] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
169− [N] readEither :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
170− [C] readDirs
171− [O] readDirs :: MonadIO m => FilePath -> Stream m FilePath
172− [N] readDirs :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
173− [C] read
174− [O] read :: MonadIO m => FilePath -> Stream m FilePath
175− [N] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
176− [C] fileReader
177− [O] fileReader :: MonadIO m => Unfold m FilePath FilePath
178− [N] fileReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
179− [C] eitherReaderPaths
180− [O] eitherReaderPaths :: MonadIO m => Unfold m FilePath (Either FilePath FilePath)
181− [N] eitherReaderPaths :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Either FilePath FilePath)
182− [C] eitherReader
183− [O] eitherReader :: MonadIO m => Unfold m FilePath (Either FilePath FilePath)
184− [N] eitherReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Either FilePath FilePath)
185− [C] dirReader
186− [O] dirReader :: MonadIO m => Unfold m FilePath FilePath
187− [N] dirReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
188−[R] Streamly.Internal.Data.Unfold.Type
189−[R] Streamly.Internal.Data.Unfold.Enumeration
190−[C] Streamly.Internal.Data.Unfold
191− [C] Unfold
192− [A] Unfold :: (s -> m (Step s b)) -> (a -> m s) -> Unfold m a b
193− [A] takeWhileMWithInput :: Monad m => (a -> b -> m Bool) -> Unfold m a b -> Unfold m a b
194− [A] manyInterleave :: Monad m => Unfold m a b -> Unfold m c a -> Unfold m c b
195− [A] enumerateFromStepIntegral :: (Monad m, Integral a) => Unfold m (a, a) a
196− [A] crossApplySnd :: Unfold m a b -> Unfold m a c -> Unfold m a c
197− [A] crossApplyFst :: Unfold m a b -> Unfold m a c -> Unfold m a b
198− [A] concatMap :: Monad m => (b -> Unfold m a c) -> Unfold m a b -> Unfold m a c
199−[R] Streamly.Internal.Data.Unboxed
200−[C] Streamly.Internal.Data.Time.Units
201− [R] Streamly.Internal.Data.Unboxed.Unbox
202− [A] Streamly.Internal.Data.Unbox.Unbox
203− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.NanoSecond64
204− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.MilliSecond64
205− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.MicroSecond64
206−[R] Streamly.Internal.Data.Time.Clock.Type
207−[A] Streamly.Internal.Data.StreamK
208− [A] CrossStreamK
209− [A] (FixityR,6)
210− [A] (FixityR,6)
211− [A] (FixityR,6)
212− [A] (FixityR,5)
213− [A] (FixityR,5)
214− [A] (FixityR,6)
215− [A] (FixityR,5)
216− [A] StreamK
217− [A] MkStream :: (forall r. State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
218− [A] zipWithM :: Monad m => (a -> b -> m c) -> StreamK m a -> StreamK m b -> StreamK m c
219− [A] zipWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
220− [A] unfoldrMWith :: Monad m => (m a -> StreamK m a -> StreamK m a) -> (b -> m (Maybe (a, b))) -> b -> StreamK m a
221− [A] unfoldrM :: Monad m => (b -> m (Maybe (a, b))) -> b -> StreamK m a
222− [A] unfoldr :: (b -> Maybe (a, b)) -> b -> StreamK m a
223− [A] uncons :: Applicative m => StreamK m a -> m (Maybe (a, StreamK m a))
224− [A] unShare :: StreamK m a -> StreamK m a
225− [A] unCross :: CrossStreamK m a -> StreamK m a
226− [A] toStream :: Applicative m => StreamK m a -> Stream m a
227− [A] toList :: Monad m => StreamK m a -> m [a]
228− [A] the :: (Eq a, Monad m) => StreamK m a -> m (Maybe a)
229− [A] takeWhile :: (a -> Bool) -> StreamK m a -> StreamK m a
230− [A] take :: Int -> StreamK m a -> StreamK m a
231− [A] tail :: Applicative m => StreamK m a -> m (Maybe (StreamK m a))
232− [A] sortBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> StreamK m a
233− [A] sequence :: Monad m => StreamK m (m a) -> StreamK m a
234− [A] scanlx' :: (x -> a -> x) -> x -> (x -> b) -> StreamK m a -> StreamK m b
235− [A] scanl' :: (b -> a -> b) -> b -> StreamK m a -> StreamK m b
236− [A] reverse :: StreamK m a -> StreamK m a
237− [A] replicateMWith :: (m a -> StreamK m a -> StreamK m a) -> Int -> m a -> StreamK m a
238− [A] replicateM :: Monad m => Int -> m a -> StreamK m a
239− [A] replicate :: Int -> a -> StreamK m a
240− [A] repeatMWith :: (m a -> t m a -> t m a) -> m a -> t m a
241− [A] repeatM :: Monad m => m a -> StreamK m a
242− [A] repeat :: a -> StreamK m a
243− [A] parseDBreak :: Monad m => Parser a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
244− [A] parseD :: Monad m => Parser a m b -> StreamK m a -> m (Either ParseError b)
245− [A] parseChunksGeneric :: Monad m => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
246− [A] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
247− [A] parseBreakChunksGeneric :: forall m a b. Monad m => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
248− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
249− [A] parseBreak :: forall m a b. Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
250− [A] parse :: Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b)
251− [A] null :: Monad m => StreamK m a -> m Bool
252− [A] notElem :: (Monad m, Eq a) => a -> StreamK m a -> m Bool
253− [A] nilM :: Applicative m => m b -> StreamK m a
254− [A] nil :: StreamK m a
255− [A] mkStream :: (forall r. State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
256− [A] mkCross :: StreamK m a -> CrossStreamK m a
257− [A] minimumBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> m (Maybe a)
258− [A] minimum :: (Monad m, Ord a) => StreamK m a -> m (Maybe a)
259− [A] mfix :: Monad m => (m a -> StreamK m a) -> StreamK m a
260− [A] mergeMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
261− [A] mergeIterateWith :: (StreamK m a -> StreamK m a -> StreamK m a) -> (a -> StreamK m a) -> StreamK m a -> StreamK m a
262− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
263− [A] mergeBy :: (a -> a -> Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
264− [A] maximumBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> m (Maybe a)
265− [A] maximum :: (Monad m, Ord a) => StreamK m a -> m (Maybe a)
266− [A] mapMaybe :: (a -> Maybe b) -> StreamK m a -> StreamK m b
267− [A] mapM_ :: Monad m => (a -> m b) -> StreamK m a -> m ()
268− [A] mapMWith :: (m b -> StreamK m b -> StreamK m b) -> (a -> m b) -> StreamK m a -> StreamK m b
269− [A] mapMSerial :: Monad m => (a -> m b) -> StreamK m a -> StreamK m b
270− [A] mapM :: Monad m => (a -> m b) -> StreamK m a -> StreamK m b
271− [A] map :: (a -> b) -> StreamK m a -> StreamK m b
272− [A] lookup :: (Monad m, Eq a) => a -> StreamK m (a, b) -> m (Maybe b)
273− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => StreamK m a -> StreamK (t m) a
274− [A] last :: Monad m => StreamK m a -> m (Maybe a)
275− [A] iterateMWith :: Monad m => (m a -> StreamK m a -> StreamK m a) -> (a -> m a) -> m a -> StreamK m a
276− [A] iterateM :: Monad m => (a -> m a) -> m a -> StreamK m a
277− [A] iterate :: (a -> a) -> a -> StreamK m a
278− [A] intersperseM :: Monad m => m a -> StreamK m a -> StreamK m a
279− [A] intersperse :: Monad m => a -> StreamK m a -> StreamK m a
280− [A] interleaveMin :: StreamK m a -> StreamK m a -> StreamK m a
281− [A] interleaveFst :: StreamK m a -> StreamK m a -> StreamK m a
282− [A] interleave :: StreamK m a -> StreamK m a -> StreamK m a
283− [A] insertBy :: (a -> a -> Ordering) -> a -> StreamK m a -> StreamK m a
284− [A] init :: Applicative m => StreamK m a -> m (Maybe (StreamK m a))
285− [A] hoist :: (Monad m, Monad n) => (forall x. m x -> n x) -> StreamK m a -> StreamK n a
286− [A] head :: Monad m => StreamK m a -> m (Maybe a)
287− [A] handle :: (MonadCatch m, Exception e) => (e -> m (StreamK m a)) -> StreamK m a -> StreamK m a
288− [A] fromYieldK :: YieldK m a -> StreamK m a
289− [A] fromStream :: Monad m => Stream m a -> StreamK m a
290− [A] fromStopK :: StopK m -> StreamK m a
291− [A] fromPure :: a -> StreamK m a
292− [A] fromList :: [a] -> StreamK m a
293− [A] fromIndicesMWith :: (m a -> StreamK m a -> StreamK m a) -> (Int -> m a) -> StreamK m a
294− [A] fromIndicesM :: Monad m => (Int -> m a) -> StreamK m a
295− [A] fromIndices :: (Int -> a) -> StreamK m a
296− [A] fromFoldableM :: (Foldable f, Monad m) => f (m a) -> StreamK m a
297− [A] fromFoldable :: Foldable f => f a -> StreamK m a
298− [A] fromEffect :: Monad m => m a -> StreamK m a
299− [A] foldrT :: (Monad m, Monad (s m), MonadTrans s) => (a -> s m b -> s m b) -> s m b -> StreamK m a -> s m b
300− [A] foldrSShared :: (a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
301− [A] foldrSM :: Monad m => (m a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
302− [A] foldrS :: (a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
303− [A] foldrM :: (a -> m b -> m b) -> m b -> StreamK m a -> m b
304− [A] foldr1 :: Monad m => (a -> a -> a) -> StreamK m a -> m (Maybe a)
305− [A] foldr :: Monad m => (a -> b -> b) -> b -> StreamK m a -> m b
306− [A] foldlx' :: forall m a b x. Monad m => (x -> a -> x) -> x -> (x -> b) -> StreamK m a -> m b
307− [A] foldlT :: (Monad m, Monad (s m), MonadTrans s) => (s m b -> a -> s m b) -> s m b -> StreamK m a -> s m b
308− [A] foldlS :: (StreamK m b -> a -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
309− [A] foldlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> StreamK m a -> m b
310− [A] foldlM' :: Monad m => (b -> a -> m b) -> m b -> StreamK m a -> m b
311− [A] foldl' :: Monad m => (b -> a -> b) -> b -> StreamK m a -> m b
312− [A] foldStreamShared :: State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> StreamK m a -> m r
313− [A] foldStream :: State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> StreamK m a -> m r
314− [A] foldEither :: Monad m => Fold m a b -> StreamK m a -> m (Either (Fold m a b) (b, StreamK m a))
315− [A] foldConcat :: Monad m => Producer m a b -> Fold m b c -> StreamK m a -> m (c, StreamK m a)
316− [A] foldBreak :: Monad m => Fold m a b -> StreamK m a -> m (b, StreamK m a)
317− [A] fold :: Monad m => Fold m a b -> StreamK m a -> m b
318− [A] findM :: Monad m => (a -> m Bool) -> StreamK m a -> m (Maybe a)
319− [A] findIndices :: (a -> Bool) -> StreamK m a -> StreamK m Int
320− [A] find :: Monad m => (a -> Bool) -> StreamK m a -> m (Maybe a)
321− [A] filter :: (a -> Bool) -> StreamK m a -> StreamK m a
322− [A] evalStateT :: Monad m => m s -> StreamK (StateT s m) a -> StreamK m a
323− [A] elem :: (Monad m, Eq a) => a -> StreamK m a -> m Bool
324− [A] dropWhile :: (a -> Bool) -> StreamK m a -> StreamK m a
325− [A] drop :: Int -> StreamK m a -> StreamK m a
326− [A] drain :: Monad m => StreamK m a -> m ()
327− [A] deleteBy :: (a -> a -> Bool) -> a -> StreamK m a -> StreamK m a
328− [A] crossWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
329− [A] crossApplyWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> StreamK m (a -> b) -> StreamK m a -> StreamK m b
330− [A] crossApplySnd :: StreamK m a -> StreamK m b -> StreamK m b
331− [A] crossApplyFst :: StreamK m a -> StreamK m b -> StreamK m a
332− [A] crossApply :: StreamK m (a -> b) -> StreamK m a -> StreamK m b
333− [A] cross :: Monad m => StreamK m a -> StreamK m b -> StreamK m (a, b)
334− [A] consMBy :: Monad m => (StreamK m a -> StreamK m a -> StreamK m a) -> m a -> StreamK m a -> StreamK m a
335− [A] consM :: Monad m => m a -> StreamK m a -> StreamK m a
336− [A] consK :: YieldK m a -> StreamK m a -> StreamK m a
337− [A] cons :: a -> StreamK m a -> StreamK m a
338− [A] conjoin :: Monad m => StreamK m a -> StreamK m a -> StreamK m a
339− [A] concatMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
340− [A] concatMapEffect :: Monad m => (b -> StreamK m a) -> m b -> StreamK m a
341− [A] concatMap :: (a -> StreamK m b) -> StreamK m a -> StreamK m b
342− [A] concatIterateWith :: (StreamK m a -> StreamK m a -> StreamK m a) -> (a -> StreamK m a) -> StreamK m a -> StreamK m a
343− [A] concatIterateScanWith :: Monad m => (StreamK m a -> StreamK m a -> StreamK m a) -> (b -> a -> m (b, StreamK m a)) -> m b -> StreamK m a -> StreamK m a
344− [A] concatIterateLeftsWith :: b ~ Either a c => (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m b -> StreamK m b
345− [A] concatEffect :: Monad m => m (StreamK m a) -> StreamK m a
346− [A] buildSM :: Monad m => ((m a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a
347− [A] buildS :: ((a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a
348− [A] buildM :: Monad m => (forall r. (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
349− [A] build :: forall m a. (forall b. (a -> b -> b) -> b -> b) -> StreamK m a
350− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> StreamK m a) -> StreamK m a
351− [A] bindWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> StreamK m a -> (a -> StreamK m b) -> StreamK m b
352− [A] before :: Monad m => m b -> StreamK m a -> StreamK m a
353− [A] augmentSM :: Monad m => ((m a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a
354− [A] augmentS :: ((a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a
355− [A] append :: StreamK m a -> StreamK m a -> StreamK m a
356− [A] any :: Monad m => (a -> Bool) -> StreamK m a -> m Bool
357− [A] all :: Monad m => (a -> Bool) -> StreamK m a -> m Bool
358− [A] (.:) :: a -> StreamK m a -> StreamK m a
359− [A] (!!) :: Monad m => StreamK m a -> Int -> m (Maybe a)
360−[R] Streamly.Internal.Data.Stream.StreamK.Type
361−[R] Streamly.Internal.Data.Stream.StreamK.Transformer
362−[R] Streamly.Internal.Data.Stream.StreamK
363−[R] Streamly.Internal.Data.Stream.StreamD.Type
364−[R] Streamly.Internal.Data.Stream.StreamD.Transformer
365−[R] Streamly.Internal.Data.Stream.StreamD.Transform
366−[R] Streamly.Internal.Data.Stream.StreamD.Top
367−[R] Streamly.Internal.Data.Stream.StreamD.Step
368−[R] Streamly.Internal.Data.Stream.StreamD.Nesting
369−[R] Streamly.Internal.Data.Stream.StreamD.Lift
370−[R] Streamly.Internal.Data.Stream.StreamD.Generate
371−[R] Streamly.Internal.Data.Stream.StreamD.Exception
372−[R] Streamly.Internal.Data.Stream.StreamD.Eliminate
373−[R] Streamly.Internal.Data.Stream.StreamD.Container
374−[D] Streamly.Internal.Data.Stream.StreamD
375−[R] Streamly.Internal.Data.Stream.Common
376−[R] Streamly.Internal.Data.Stream.Chunked
377−[C] Streamly.Internal.Data.Stream
378− [A] class Enum a => Enumerable a
379− [A] Stream
380− [A] UnStream :: (State StreamK m a -> s -> m (Step s a)) -> s -> Stream m a
381− [A] Step
382− [A] Yield :: a -> s -> Step s a
383− [A] Stop :: Step s a
384− [A] Skip :: s -> Step s a
385− [A] InterleaveState
386− [A] InterleaveSecondOnly :: s2 -> InterleaveState s1 s2
387− [A] InterleaveSecond :: s1 -> s2 -> InterleaveState s1 s2
388− [A] InterleaveFirstOnly :: s1 -> InterleaveState s1 s2
389− [A] InterleaveFirst :: s1 -> s2 -> InterleaveState s1 s2
390− [A] FoldManyPost
391− [A] FoldManyPostYield :: b -> FoldManyPost s fs b a -> FoldManyPost s fs b a
392− [A] FoldManyPostStart :: s -> FoldManyPost s fs b a
393− [A] FoldManyPostLoop :: s -> fs -> FoldManyPost s fs b a
394− [A] FoldManyPostDone :: FoldManyPost s fs b a
395− [A] FoldMany
396− [A] FoldManyYield :: b -> FoldMany s fs b a -> FoldMany s fs b a
397− [A] FoldManyStart :: s -> FoldMany s fs b a
398− [A] FoldManyLoop :: s -> fs -> FoldMany s fs b a
399− [A] FoldManyFirst :: fs -> s -> FoldMany s fs b a
400− [A] FoldManyDone :: FoldMany s fs b a
401− [A] CrossStream
402− [A] ConcatUnfoldInterleaveState
403− [A] ConcatUnfoldInterleaveOuter :: o -> [i] -> ConcatUnfoldInterleaveState o i
404− [A] ConcatUnfoldInterleaveInnerR :: [i] -> [i] -> ConcatUnfoldInterleaveState o i
405− [A] ConcatUnfoldInterleaveInnerL :: [i] -> [i] -> ConcatUnfoldInterleaveState o i
406− [A] ConcatUnfoldInterleaveInner :: o -> [i] -> ConcatUnfoldInterleaveState o i
407− [A] ConcatMapUState
408− [A] ConcatMapUOuter :: o -> ConcatMapUState o i
409− [A] ConcatMapUInner :: o -> i -> ConcatMapUState o i
410− [A] AppendState
411− [A] AppendSecond :: s2 -> AppendState s1 s2
412− [A] AppendFirst :: s1 -> AppendState s1 s2
413− [A] pattern Stream :: (State StreamK m a -> s -> m (Step s a)) -> s -> Stream m a
414− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Stream m b -> Stream m c
415− [A] zipWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
416− [A] wordsBy :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
417− [A] with :: Monad m => (Stream m a -> Stream m (s, a)) -> (((s, a) -> b) -> Stream m (s, a) -> Stream m (s, a)) -> ((s, a) -> b) -> Stream m a -> Stream m a
418− [A] usingStateT :: Monad m => m s -> (Stream (StateT s m) a -> Stream (StateT s m) a) -> Stream m a -> Stream m a
419− [A] usingReaderT :: Monad m => m r -> (Stream (ReaderT r m) a -> Stream (ReaderT r m) a) -> Stream m a -> Stream m a
420− [A] uniqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a
421− [A] uniq :: (Eq a, Monad m) => Stream m a -> Stream m a
422− [A] unionWithStreamGenericBy :: MonadIO m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
423− [A] unionWithStreamAscBy :: (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
424− [A] unfoldrM :: Monad m => (s -> m (Maybe (a, s))) -> s -> Stream m a
425− [A] unfoldr :: Monad m => (s -> Maybe (a, s)) -> s -> Stream m a
426− [A] unfoldRoundRobin :: Monad m => Unfold m a b -> Stream m a -> Stream m b
427− [A] unfoldMany :: Monad m => Unfold m a b -> Stream m a -> Stream m b
428− [A] unfoldIterateDfs :: Monad m => Unfold m a a -> Stream m a -> Stream m a
429− [A] unfoldIterateBfsRev :: Monad m => Unfold m a a -> Stream m a -> Stream m a
430− [A] unfoldIterateBfs :: Monad m => Unfold m a a -> Stream m a -> Stream m a
431− [A] unfoldInterleave :: Monad m => Unfold m a b -> Stream m a -> Stream m b
432− [A] unfold :: Applicative m => Unfold m a b -> a -> Stream m b
433− [A] uncons :: Monad m => Stream m a -> m (Maybe (a, Stream m a))
434− [A] unCross :: CrossStream m a -> Stream m a
435− [A] transform :: Monad m => Pipe m a b -> Stream m a -> Stream m b
436− [A] trace_ :: Monad m => m b -> Stream m a -> Stream m a
437− [A] trace :: Monad m => (a -> m b) -> Stream m a -> Stream m a
438− [A] toStreamK :: Monad m => Stream m a -> StreamK m a
439− [A] toListRev :: Monad m => Stream m a -> m [a]
440− [A] toList :: Monad m => Stream m a -> m [a]
441− [A] timestamped :: MonadIO m => Stream m a -> Stream m (AbsTime, a)
442− [A] timestampWith :: MonadIO m => Double -> Stream m a -> Stream m (AbsTime, a)
443− [A] timesWith :: MonadIO m => Double -> Stream m (AbsTime, RelTime64)
444− [A] times :: MonadIO m => Stream m (AbsTime, RelTime64)
445− [A] timeout :: AbsTime -> t m ()
446− [A] timeIndexed :: MonadIO m => Stream m a -> Stream m (RelTime64, a)
447− [A] timeIndexWith :: MonadIO m => Double -> Stream m a -> Stream m (RelTime64, a)
448− [A] the :: (Eq a, Monad m) => Stream m a -> m (Maybe a)
449− [A] tapOffsetEvery :: Monad m => Int -> Int -> Fold m a b -> Stream m a -> Stream m a
450− [A] tap :: Monad m => Fold m a b -> Stream m a -> Stream m a
451− [A] takeWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
452− [A] takeWhileLast :: (a -> Bool) -> Stream m a -> Stream m a
453− [A] takeWhileAround :: (a -> Bool) -> Stream m a -> Stream m a
454− [A] takeWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
455− [A] takeEndByM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
456− [A] takeEndBy :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
457− [A] take :: Applicative m => Int -> Stream m a -> Stream m a
458− [A] tail :: Monad m => Stream m a -> m (Maybe (Stream m a))
459− [A] stripSuffixUnbox :: (MonadIO m, Eq a, Unbox a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
460− [A] stripSuffix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
461− [A] stripPrefix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
462− [A] strideFromThen :: Monad m => Int -> Int -> Stream m a -> Stream m a
463− [A] splitOnSuffixSeqAny :: [Array a] -> Fold m a b -> Stream m a -> Stream m b
464− [A] splitOnSuffixSeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) => Bool -> Array a -> Fold m a b -> Stream m a -> Stream m b
465− [A] splitOnSeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) => Array a -> Fold m a b -> Stream m a -> Stream m b
466− [A] splitOnPrefix :: (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
467− [A] splitOnAny :: [Array a] -> Fold m a b -> Stream m a -> Stream m b
468− [A] splitOn :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
469− [A] splitInnerBySuffix :: (Monad m, Eq (f a), Monoid (f a)) => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
470− [A] splitInnerBy :: Monad m => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
471− [A] slicesBy :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
472− [A] sliceOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
473− [A] sequence :: Monad m => Stream m (m a) -> Stream m a
474− [A] scanlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> Stream m b
475− [A] scanlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> Stream m b
476− [A] scanlMAfter' :: Monad m => (b -> a -> m b) -> m b -> (b -> m b) -> Stream m a -> Stream m b
477− [A] scanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
478− [A] scanlM :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
479− [A] scanl1M' :: Monad m => (a -> a -> m a) -> Stream m a -> Stream m a
480− [A] scanl1M :: Monad m => (a -> a -> m a) -> Stream m a -> Stream m a
481− [A] scanl1' :: Monad m => (a -> a -> a) -> Stream m a -> Stream m a
482− [A] scanl1 :: Monad m => (a -> a -> a) -> Stream m a -> Stream m a
483− [A] scanl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
484− [A] scanl :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
485− [A] scanMaybe :: Monad m => Fold m a (Maybe b) -> Stream m a -> Stream m b
486− [A] scanMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
487− [A] scan :: Monad m => Fold m a b -> Stream m a -> Stream m b
488− [A] runStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m (s, a)
489− [A] runReaderT :: Monad m => m s -> Stream (ReaderT s m) a -> Stream m a
490− [A] runInnerWithState :: Monad m => (forall b. s -> t m b -> m (b, s)) -> m s -> Stream (t m) a -> Stream m (s, a)
491− [A] runInnerWith :: Monad m => (forall b. t m b -> m b) -> Stream (t m) a -> Stream m a
492− [A] roundRobin :: Monad m => Stream m a -> Stream m a -> Stream m a
493− [A] rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Stream m a -> Stream m b
494− [A] rollingMap2 :: Monad m => (a -> a -> b) -> Stream m a -> Stream m b
495− [A] rollingMap :: Monad m => (Maybe a -> a -> b) -> Stream m a -> Stream m b
496− [A] reverseUnbox :: (MonadIO m, Unbox a) => Stream m a -> Stream m a
497− [A] reverse :: Monad m => Stream m a -> Stream m a
498− [A] replicateM :: Monad m => Int -> m a -> Stream m a
499− [A] replicate :: Monad m => Int -> a -> Stream m a
500− [A] repeated :: Stream m a -> Stream m a
501− [A] repeatM :: Monad m => m a -> Stream m a
502− [A] repeat :: Monad m => a -> Stream m a
503− [A] relTimesWith :: MonadIO m => Double -> Stream m RelTime64
504− [A] relTimes :: MonadIO m => Stream m RelTime64
505− [A] refoldMany :: Monad m => Refold m x a b -> m x -> Stream m a -> Stream m b
506− [A] refoldIterateM :: Monad m => Refold m b a b -> m b -> Stream m a -> Stream m b
507− [A] reduceIterateBfs :: Monad m => (a -> a -> m a) -> Stream m a -> m (Maybe a)
508− [A] reassembleBy :: Fold m a b -> (a -> a -> Int) -> Stream m a -> Stream m b
509− [A] prune :: (a -> Bool) -> Stream m a -> Stream m a
510− [A] prescanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
511− [A] prescanl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
512− [A] postscanlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> Stream m b
513− [A] postscanlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> Stream m b
514− [A] postscanlMAfter' :: Monad m => (b -> a -> m b) -> m b -> (b -> m b) -> Stream m a -> Stream m b
515− [A] postscanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
516− [A] postscanlM :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
517− [A] postscanl' :: Monad m => (a -> b -> a) -> a -> Stream m b -> Stream m a
518− [A] postscanl :: Monad m => (a -> b -> a) -> a -> Stream m b -> Stream m a
519− [A] postscan :: Monad m => Fold m a b -> Stream m a -> Stream m b
520− [A] parseSequence :: Stream m (Parser a m b) -> Stream m a -> Stream m b
521− [A] parseManyTill :: Parser a m b -> Parser a m x -> Stream m a -> Stream m b
522− [A] parseManyD :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
523− [A] parseMany :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
524− [A] parseIterateD :: Monad m => (b -> Parser a m b) -> b -> Stream m a -> Stream m (Either ParseError b)
525− [A] parseIterate :: Monad m => (b -> Parser a m b) -> b -> Stream m a -> Stream m (Either ParseError b)
526− [A] parseD :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
527− [A] parseBreakD :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b, Stream m a)
528− [A] parseBreak :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b, Stream m a)
529− [A] parse :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
530− [A] onException :: MonadCatch m => m b -> Stream m a -> Stream m a
531− [A] null :: Monad m => Stream m a -> m Bool
532− [A] nub :: (Monad m, Ord a) => Stream m a -> Stream m a
533− [A] notElem :: (Monad m, Eq a) => a -> Stream m a -> m Bool
534− [A] nilM :: Applicative m => m b -> Stream m a
535− [A] nil :: Applicative m => Stream m a
536− [A] morphInner :: Monad n => (forall x. m x -> n x) -> Stream m a -> Stream n a
537− [A] mkCross :: Stream m a -> CrossStream m a
538− [A] minimumBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> m (Maybe a)
539− [A] minimum :: (Monad m, Ord a) => Stream m a -> m (Maybe a)
540− [A] mergeMinBy :: (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
541− [A] mergeFstBy :: (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
542− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
543− [A] mergeBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
544− [A] maximumBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> m (Maybe a)
545− [A] maximum :: (Monad m, Ord a) => Stream m a -> m (Maybe a)
546− [A] mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Stream m a -> Stream m b
547− [A] mapMaybe :: Monad m => (a -> Maybe b) -> Stream m a -> Stream m b
548− [A] mapM_ :: Monad m => (a -> m b) -> Stream m a -> m ()
549− [A] mapM :: Monad m => (a -> m b) -> Stream m a -> Stream m b
550− [A] map :: Monad m => (a -> b) -> Stream m a -> Stream m b
551− [A] lookup :: (Monad m, Eq a) => a -> Stream m (a, b) -> m (Maybe b)
552− [A] liftInnerWith :: Monad (t m) => (forall b. m b -> t m b) -> Stream m a -> Stream (t m) a
553− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => Stream m a -> Stream (t m) a
554− [A] last :: Monad m => Stream m a -> m (Maybe a)
555− [A] joinOuterGeneric :: MonadIO m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (Maybe a, Maybe b)
556− [A] joinOuterAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (Maybe a, Maybe b)
557− [A] joinOuter :: (Ord k, MonadIO m) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, Maybe a, Maybe b)
558− [A] joinLeftGeneric :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (a, Maybe b)
559− [A] joinLeftAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (a, Maybe b)
560− [A] joinLeft :: (Ord k, Monad m) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, a, Maybe b)
561− [A] joinInnerGeneric :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (a, b)
562− [A] joinInnerAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (a, b)
563− [A] joinInner :: (Monad m, Ord k) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, a, b)
564− [A] iterateM :: Monad m => (a -> m a) -> m a -> Stream m a
565− [A] iterate :: Monad m => (a -> a) -> a -> Stream m a
566− [A] isSuffixOfUnbox :: (MonadIO m, Eq a, Unbox a) => Stream m a -> Stream m a -> m Bool
567− [A] isSuffixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
568− [A] isSubsequenceOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
569− [A] isPrefixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
570− [A] isInfixOf :: (MonadIO m, Eq a, Enum a, Storable a, Unbox a) => Stream m a -> Stream m a -> m Bool
571− [A] intersperseM_ :: Monad m => m b -> Stream m a -> Stream m a
572− [A] intersperseMWith :: Int -> m a -> Stream m a -> Stream m a
573− [A] intersperseMSuffix_ :: Monad m => m b -> Stream m a -> Stream m a
574− [A] intersperseMSuffixWith :: forall m a. Monad m => Int -> m a -> Stream m a -> Stream m a
575− [A] intersperseMSuffix :: forall m a. Monad m => m a -> Stream m a -> Stream m a
576− [A] intersperseMPrefix_ :: Monad m => m b -> Stream m a -> Stream m a
577− [A] intersperseM :: Monad m => m a -> Stream m a -> Stream m a
578− [A] intersperse :: Monad m => a -> Stream m a -> Stream m a
579− [A] intersectBySorted :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
580− [A] interposeSuffixM :: Monad m => m c -> Unfold m b c -> Stream m b -> Stream m c
581− [A] interposeSuffix :: Monad m => c -> Unfold m b c -> Stream m b -> Stream m c
582− [A] interposeM :: Monad m => m c -> Unfold m b c -> Stream m b -> Stream m c
583− [A] interpose :: Monad m => c -> Unfold m b c -> Stream m b -> Stream m c
584− [A] interleaveMin :: Monad m => Stream m a -> Stream m a -> Stream m a
585− [A] interleaveFstSuffix :: Monad m => Stream m a -> Stream m a -> Stream m a
586− [A] interleaveFst :: Monad m => Stream m a -> Stream m a -> Stream m a
587− [A] interleave :: Monad m => Stream m a -> Stream m a -> Stream m a
588− [A] intercalateSuffix :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
589− [A] intercalate :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
590− [A] insertBy :: Monad m => (a -> a -> Ordering) -> a -> Stream m a -> Stream m a
591− [A] indexedR :: Monad m => Int -> Stream m a -> Stream m (Int, a)
592− [A] indexed :: Monad m => Stream m a -> Stream m (Int, a)
593− [A] headElse :: Monad m => a -> Stream m a -> m a
594− [A] head :: Monad m => Stream m a -> m (Maybe a)
595− [A] handle :: (MonadCatch m, Exception e) => (e -> m (Stream m a)) -> Stream m a -> Stream m a
596− [A] groupsWhile :: Monad m => (a -> a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
597− [A] groupsRollingBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
598− [A] groupsOf :: Monad m => Int -> Fold m a b -> Stream m a -> Stream m b
599− [A] gintercalateSuffix :: Monad m => Unfold m a c -> Stream m a -> Unfold m b c -> Stream m b -> Stream m c
600− [A] gintercalate :: Monad m => Unfold m a c -> Stream m a -> Unfold m b c -> Stream m b -> Stream m c
601− [A] ghandle :: (MonadCatch m, Exception e) => (e -> Stream m a -> m (Stream m a)) -> Stream m a -> Stream m a
602− [A] generateM :: Monad m => Int -> (Int -> m a) -> Stream m a
603− [A] generate :: Monad m => Int -> (Int -> a) -> Stream m a
604− [A] generalizeInner :: Monad m => Stream Identity a -> Stream m a
605− [A] gbracket_ :: Monad m => m c -> (c -> m d) -> (c -> e -> Stream m b -> m (Stream m b)) -> (forall s. m s -> m (Either e s)) -> (c -> Stream m b) -> Stream m b
606− [A] gbracket :: MonadIO m => IO c -> (c -> IO d1) -> (c -> e -> Stream m b -> IO (Stream m b)) -> (c -> IO d2) -> (forall s. m s -> m (Either e s)) -> (c -> Stream m b) -> Stream m b
607− [A] fromStreamK :: Applicative m => StreamK m a -> Stream m a
608− [A] fromPure :: Applicative m => a -> Stream m a
609− [A] fromPtrN :: (Monad m, Storable a) => Int -> Ptr a -> Stream m a
610− [A] fromPtr :: forall m a. (Monad m, Storable a) => Ptr a -> Stream m a
611− [A] fromListM :: Monad m => [m a] -> Stream m a
612− [A] fromList :: Applicative m => [a] -> Stream m a
613− [A] fromIndicesM :: Monad m => (Int -> m a) -> Stream m a
614− [A] fromIndices :: Monad m => (Int -> a) -> Stream m a
615− [A] fromFoldableM :: (Monad m, Foldable f) => f (m a) -> Stream m a
616− [A] fromFoldable :: (Monad m, Foldable f) => f a -> Stream m a
617− [A] fromEffect :: Applicative m => m a -> Stream m a
618− [A] fromByteStr# :: Monad m => Addr# -> Stream m Word8
619− [A] foldrT :: (Monad m, Monad (t m), MonadTrans t) => (a -> t m b -> t m b) -> t m b -> Stream m a -> t m b
620− [A] foldrS :: Monad m => (a -> Stream m b -> Stream m b) -> Stream m b -> Stream m a -> Stream m b
621− [A] foldrMx :: Monad m => (a -> m x -> m x) -> m x -> (m x -> m b) -> Stream m a -> m b
622− [A] foldrM :: Monad m => (a -> m b -> m b) -> m b -> Stream m a -> m b
623− [A] foldr1 :: Monad m => (a -> a -> a) -> Stream m a -> m (Maybe a)
624− [A] foldr :: Monad m => (a -> b -> b) -> b -> Stream m a -> m b
625− [A] foldlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> m b
626− [A] foldlT :: (Monad m, Monad (s m), MonadTrans s) => (s m b -> a -> s m b) -> s m b -> Stream m a -> s m b
627− [A] foldlS :: Monad m => (Stream m b -> a -> Stream m b) -> Stream m b -> Stream m a -> Stream m b
628− [A] foldlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> m b
629− [A] foldlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> m b
630− [A] foldl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> m b
631− [A] foldSequence :: Stream m (Fold m a b) -> Stream m a -> Stream m b
632− [A] foldManyPost :: Monad m => Fold m a b -> Stream m a -> Stream m b
633− [A] foldMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
634− [A] foldIterateM :: Monad m => (b -> m (Fold m a b)) -> m b -> Stream m a -> Stream m b
635− [A] foldIterateBfs :: Fold m a (Either a a) -> Stream m a -> m (Maybe a)
636− [A] foldEither :: Monad m => Fold m a b -> Stream m a -> m (Either (Fold m a b) (b, Stream m a))
637− [A] foldBreak :: Monad m => Fold m a b -> Stream m a -> m (b, Stream m a)
638− [A] foldAddLazy :: Monad m => Fold m a b -> Stream m a -> Fold m a b
639− [A] foldAdd :: Monad m => Fold m a b -> Stream m a -> m (Fold m a b)
640− [A] fold :: Monad m => Fold m a b -> Stream m a -> m b
641− [A] findM :: Monad m => (a -> m Bool) -> Stream m a -> m (Maybe a)
642− [A] findIndices :: Monad m => (a -> Bool) -> Stream m a -> Stream m Int
643− [A] find :: Monad m => (a -> Bool) -> Stream m a -> m (Maybe a)
644− [A] finallyUnsafe :: MonadCatch m => m b -> Stream m a -> Stream m a
645− [A] finallyIO :: (MonadIO m, MonadCatch m) => IO b -> Stream m a -> Stream m a
646− [A] filterM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
647− [A] filterInStreamGenericBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
648− [A] filterInStreamAscBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
649− [A] filter :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
650− [A] evalStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m a
651− [A] eqBy :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> m Bool
652− [A] enumerateTo :: (Monad m, Bounded a, Enumerable a) => a -> Stream m a
653− [A] enumerateFromToSmall :: (Monad m, Enum a) => a -> a -> Stream m a
654− [A] enumerateFromToIntegral :: (Monad m, Integral a) => a -> a -> Stream m a
655− [A] enumerateFromToFractional :: (Monad m, Fractional a, Ord a) => a -> a -> Stream m a
656− [A] enumerateFromTo :: (Enumerable a, Monad m) => a -> a -> Stream m a
657− [A] enumerateFromThenToSmall :: (Monad m, Enum a) => a -> a -> a -> Stream m a
658− [A] enumerateFromThenToIntegral :: (Monad m, Integral a) => a -> a -> a -> Stream m a
659− [A] enumerateFromThenToFractional :: (Monad m, Fractional a, Ord a) => a -> a -> a -> Stream m a
660− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => a -> a -> a -> Stream m a
661− [A] enumerateFromThenSmallBounded :: (Monad m, Enumerable a, Bounded a) => a -> a -> Stream m a
662− [A] enumerateFromThenNum :: (Monad m, Num a) => a -> a -> Stream m a
663− [A] enumerateFromThenIntegral :: (Monad m, Integral a, Bounded a) => a -> a -> Stream m a
664− [A] enumerateFromThenFractional :: (Monad m, Fractional a) => a -> a -> Stream m a
665− [A] enumerateFromThen :: (Enumerable a, Monad m) => a -> a -> Stream m a
666− [A] enumerateFromStepNum :: (Monad m, Num a) => a -> a -> Stream m a
667− [A] enumerateFromStepIntegral :: (Integral a, Monad m) => a -> a -> Stream m a
668− [A] enumerateFromNum :: (Monad m, Num a) => a -> Stream m a
669− [A] enumerateFromIntegral :: (Monad m, Integral a, Bounded a) => a -> Stream m a
670− [A] enumerateFromFractional :: (Monad m, Fractional a) => a -> Stream m a
671− [A] enumerateFromBounded :: (Monad m, Enumerable a, Bounded a) => a -> Stream m a
672− [A] enumerateFrom :: (Enumerable a, Monad m) => a -> Stream m a
673− [A] enumerate :: (Monad m, Bounded a, Enumerable a) => Stream m a
674− [A] elemIndices :: (Monad m, Eq a) => a -> Stream m a -> Stream m Int
675− [A] elem :: (Monad m, Eq a) => a -> Stream m a -> m Bool
676− [A] durations :: Double -> t m RelTime64
677− [A] dropWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
678− [A] dropWhileLast :: (a -> Bool) -> Stream m a -> Stream m a
679− [A] dropWhileAround :: (a -> Bool) -> Stream m a -> Stream m a
680− [A] dropWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
681− [A] dropSuffix :: Stream m a -> Stream m a -> Stream m a
682− [A] dropPrefix :: Stream m a -> Stream m a -> Stream m a
683− [A] dropLast :: Int -> Stream m a -> Stream m a
684− [A] dropInfix :: Stream m a -> Stream m a -> Stream m a
685− [A] drop :: Monad m => Int -> Stream m a -> Stream m a
686− [A] drain :: Monad m => Stream m a -> m ()
687− [A] deleteInStreamGenericBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
688− [A] deleteInStreamAscBy :: (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
689− [A] deleteBy :: Monad m => (a -> a -> Bool) -> a -> Stream m a -> Stream m a
690− [A] delayPre :: MonadIO m => Double -> Stream m a -> Stream m a
691− [A] delayPost :: MonadIO m => Double -> Stream m a -> Stream m a
692− [A] delay :: MonadIO m => Double -> Stream m a -> Stream m a
693− [A] crossWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
694− [A] crossApplySnd :: Functor f => Stream f a -> Stream f b -> Stream f b
695− [A] crossApplyFst :: Functor f => Stream f a -> Stream f b -> Stream f a
696− [A] crossApply :: Functor f => Stream f (a -> b) -> Stream f a -> Stream f b
697− [A] cross :: Monad m => Stream m a -> Stream m b -> Stream m (a, b)
698− [A] consM :: Applicative m => m a -> Stream m a -> Stream m a
699− [A] cons :: Applicative m => a -> Stream m a -> Stream m a
700− [A] concatMapM :: Monad m => (a -> m (Stream m b)) -> Stream m a -> Stream m b
701− [A] concatMap :: Monad m => (a -> Stream m b) -> Stream m a -> Stream m b
702− [A] concatIterateScan :: Monad m => (b -> a -> m b) -> (b -> m (Maybe (b, Stream m a))) -> b -> Stream m a
703− [A] concatIterateDfs :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
704− [A] concatIterateBfsRev :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
705− [A] concatIterateBfs :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
706− [A] concatEffect :: Monad m => m (Stream m a) -> Stream m a
707− [A] concat :: Monad m => Stream m (Stream m a) -> Stream m a
708− [A] cmpBy :: Monad m => (a -> b -> Ordering) -> Stream m a -> Stream m b -> m Ordering
709− [A] catRights :: Monad m => Stream m (Either a b) -> Stream m b
710− [A] catMaybes :: Monad m => Stream m (Maybe a) -> Stream m a
711− [A] catLefts :: Monad m => Stream m (Either a b) -> Stream m a
712− [A] catEithers :: Monad m => Stream m (Either a a) -> Stream m a
713− [A] bracketUnsafe :: MonadCatch m => m b -> (b -> m c) -> (b -> Stream m a) -> Stream m a
714− [A] bracketIO3 :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> IO d) -> (b -> IO e) -> (b -> Stream m a) -> Stream m a
715− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> Stream m a) -> Stream m a
716− [A] before :: Monad m => m b -> Stream m a -> Stream m a
717− [A] append :: Monad m => Stream m a -> Stream m a -> Stream m a
718− [A] any :: Monad m => (a -> Bool) -> Stream m a -> m Bool
719− [A] all :: Monad m => (a -> Bool) -> Stream m a -> m Bool
720− [A] afterUnsafe :: Monad m => m b -> Stream m a -> Stream m a
721− [A] afterIO :: MonadIO m => IO b -> Stream m a -> Stream m a
722− [A] absTimesWith :: MonadIO m => Double -> Stream m AbsTime
723− [A] absTimes :: MonadIO m => Stream m AbsTime
724− [A] (!!) :: Monad m => Stream m a -> Int -> m (Maybe a)
725−[R] Streamly.Internal.Data.Ring.Unboxed
726−[A] Streamly.Internal.Data.Ring.Generic
727− [A] Ring
728− [A] [ringMax] :: Ring a -> !Int
729− [A] [ringHead] :: Ring a -> !Int
730− [A] [ringArr] :: Ring a -> MutArray a
731− [A] Ring :: MutArray a -> !Int -> !Int -> Ring a
732− [A] writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
733− [A] unsafeInsertRingWith :: Ring a -> a -> IO Int
734− [A] toStreamWith :: Int -> Ring a -> Stream m a
735− [A] toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
736− [A] seek :: MonadIO m => Int -> Ring a -> m (Ring a)
737− [A] createRing :: MonadIO m => Int -> m (Ring a)
738− [A] copyToMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
739−[C] Streamly.Internal.Data.Ring
740− [C] Ring
741− [A] [ringStart] :: Ring a -> {-# UNPACK #-} !ForeignPtr a
742− [R] [ringMax] :: Ring a -> !Int
743− [R] [ringHead] :: Ring a -> !Int
744− [A] [ringBound] :: Ring a -> {-# UNPACK #-} !Ptr a
745− [R] [ringArr] :: Ring a -> MutArray a
746− [C] Ring
747− [O] Ring :: MutArray a -> !Int -> !Int -> Ring a
748− [N] Ring :: {-# UNPACK #-} !ForeignPtr a -> {-# UNPACK #-} !Ptr a -> Ring a
749− [A] GHC.Show.Show
750− [A] instance (GHC.Show.Show a, GHC.Show.Show b, GHC.Show.Show c, GHC.Show.Show d) => GHC.Show.Show (Streamly.Internal.Data.Ring.Tuple4' a b c d)
751− [A] writeN :: Int -> Fold m a (Ring a)
752− [R] writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
753− [R] unsafeInsertRingWith :: Ring a -> a -> IO Int
754− [A] unsafeInsert :: Storable a => Ring a -> Ptr a -> a -> IO (Ptr a)
755− [A] unsafeFoldRingNM :: forall m a b. (MonadIO m, Storable a) => Int -> Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
756− [A] unsafeFoldRingM :: forall m a b. (MonadIO m, Storable a) => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
757− [A] unsafeFoldRingFullM :: forall m a b. (MonadIO m, Storable a) => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
758− [A] unsafeFoldRing :: forall a b. Storable a => Ptr a -> (b -> a -> b) -> b -> Ring a -> b
759− [A] unsafeEqArrayN :: Ring a -> Ptr a -> Array a -> Int -> Bool
760− [A] unsafeEqArray :: Ring a -> Ptr a -> Array a -> Bool
761− [R] toStreamWith :: Int -> Ring a -> Stream m a
762− [R] toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
763− [A] startOf :: Ring a -> Ptr a
764− [A] slidingWindowWith :: forall m a b. (MonadIO m, Storable a, Unbox a) => Int -> Fold m ((a, Maybe a), m (MutArray a)) b -> Fold m a b
765− [A] slidingWindow :: forall m a b. (MonadIO m, Storable a, Unbox a) => Int -> Fold m (a, Maybe a) b -> Fold m a b
766− [A] slide :: Ring a -> a -> m (Ring a)
767− [R] seek :: MonadIO m => Int -> Ring a -> m (Ring a)
768− [A] ringsOf :: Int -> Stream m a -> Stream m (MutArray a)
769− [A] readRev :: Unfold m (MutArray a) a
770− [A] read :: forall m a. (MonadIO m, Storable a) => Unfold m (Ring a, Ptr a, Int) a
771− [A] putIndex :: Ring a -> Int -> a -> m ()
772− [A] newRing :: Int -> m (Ring a)
773− [A] new :: forall a. Storable a => Int -> IO (Ring a, Ptr a)
774− [A] moveBy :: forall a. Storable a => Int -> Ring a -> Ptr a -> Ptr a
775− [A] modifyIndex :: Ring a -> Int -> (a -> (a, b)) -> m b
776− [A] length :: Ring a -> Int
777− [A] getIndexUnsafe :: Ring a -> Int -> m a
778− [A] getIndexRev :: Ring a -> Int -> m a
779− [A] getIndex :: Ring a -> Int -> m a
780− [A] fromArray :: MutArray a -> Ring a
781− [R] createRing :: MonadIO m => Int -> m (Ring a)
782− [A] castUnsafe :: Ring a -> Ring b
783− [A] cast :: forall a b. Storable b => Ring a -> Maybe (Ring b)
784− [A] bytesFree :: Ring a -> Int
785− [A] byteLength :: Ring a -> Int
786− [A] byteCapacity :: Ring a -> Int
787− [A] asBytes :: Ring a -> Ring Word8
788− [A] advance :: forall a. Storable a => Ring a -> Ptr a -> Ptr a
789−[R] Streamly.Internal.Data.Producer.Type
790−[R] Streamly.Internal.Data.Producer.Source
791−[C] Streamly.Internal.Data.Producer
792− [A] Source
793− [A] unread :: [b] -> Source a b -> Source a b
794− [A] translate :: Functor m => (a -> c) -> (c -> a) -> Producer m c b -> Producer m a b
795− [A] source :: Maybe a -> Source a b
796− [A] producer :: Monad m => Producer m a b -> Producer m (Source a b) b
797− [A] parseManyD :: Monad m => Parser a m b -> Producer m (Source x a) a -> Producer m (Source x a) (Either ParseError b)
798− [A] parseMany :: Monad m => Parser a m b -> Producer m (Source x a) a -> Producer m (Source x a) (Either ParseError b)
799− [A] parse :: Monad m => Parser a m b -> Producer m (Source s a) a -> Source s a -> m (Either ParseError b, Source s a)
800− [A] lmap :: (a -> a) -> Producer m a b -> Producer m a b
801− [A] isEmpty :: Source a b -> Bool
802−[R] Streamly.Internal.Data.Pipe.Type
803−[C] Streamly.Internal.Data.Pipe
804− [A] Step
805− [A] Yield :: a -> s -> Step s a
806− [A] Continue :: s -> Step s a
807− [A] PipeState
808− [A] Produce :: s2 -> PipeState s1 s2
809− [A] Consume :: s1 -> PipeState s1 s2
810− [C] Pipe
811− [A] Pipe :: (s1 -> a -> m (Step (PipeState s1 s2) b)) -> (s2 -> m (Step (PipeState s1 s2) b)) -> s1 -> Pipe m a b
812−[A] Streamly.Internal.Data.ParserK
813− [A] Step
814− [A] Partial :: !Int -> (Input a -> m (Step a m r)) -> Step a m r
815− [A] Error :: !Int -> String -> Step a m r
816− [A] Done :: !Int -> r -> Step a m r
817− [A] Continue :: !Int -> (Input a -> m (Step a m r)) -> Step a m r
818− [A] ParseResult
819− [A] Success :: !Int -> !b -> ParseResult b
820− [A] Failure :: !Int -> !String -> ParseResult b
821− [A] Input
822− [A] None :: Input a
823− [A] Chunk :: a -> Input a
824− [A] ParserK
825− [A] [runParser] :: ParserK a m b -> forall r. (ParseResult b -> Int -> Input a -> m (Step a m r)) -> Int -> Int -> Input a -> m (Step a m r)
826− [A] MkParser :: (forall r. (ParseResult b -> Int -> Input a -> m (Step a m r)) -> Int -> Int -> Input a -> m (Step a m r)) -> ParserK a m b
827− [A] fromPure :: b -> ParserK a m b
828− [A] fromEffect :: Monad m => m b -> ParserK a m b
829− [A] die :: String -> ParserK a m b
830− [A] adaptCG :: Monad m => Parser a m b -> ParserK (Array a) m b
831− [A] adaptC :: (Monad m, Unbox a) => Parser a m b -> ParserK (Array a) m b
832− [A] adapt :: Monad m => Parser a m b -> ParserK a m b
833−[R] Streamly.Internal.Data.Parser.ParserK.Type
834−[R] Streamly.Internal.Data.Parser.ParserD.Type
835−[R] Streamly.Internal.Data.Parser.ParserD.Tee
836−[R] Streamly.Internal.Data.Parser.ParserD
837−[C] Streamly.Internal.Data.Parser
838− [A] Step
839− [A] Partial :: !Int -> !s -> Step s b
840− [A] Error :: !String -> Step s b
841− [A] Done :: !Int -> !b -> Step s b
842− [A] Continue :: !Int -> !s -> Step s b
843− [A] Parser
844− [A] Parser :: (s -> a -> m (Step s b)) -> m (Initial s b) -> (s -> m (Step s b)) -> Parser a m b
845− [A] Initial
846− [A] IPartial :: !s -> Initial s b
847− [A] IError :: !String -> Initial s b
848− [A] IDone :: !b -> Initial s b
849− [A] GHC.Show.Show
850− [A] instance (GHC.Show.Show a, GHC.Show.Show b) => GHC.Show.Show (Streamly.Internal.Data.Parser.Tuple'Fused a b)
851− [A] ParseError
852− [A] ParseError :: String -> ParseError
853− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Fold m c x -> Parser b m x
854− [A] zip :: Monad m => Stream m a -> Fold m (a, b) x -> Parser b m x
855− [A] wordWithQuotes :: (Monad m, Eq a) => Bool -> (a -> a -> Maybe a) -> a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
856− [A] wordProcessQuotes :: (Monad m, Eq a) => a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
857− [A] wordKeepQuotes :: (Monad m, Eq a) => a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
858− [A] wordFramedBy :: Monad m => (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
859− [A] wordBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
860− [A] toFold :: Monad m => Parser a m b -> Fold m a b
861− [A] takeWhileP :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
862− [A] takeWhile1 :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
863− [A] takeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
864− [A] takeStartBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
865− [A] takeStartBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
866− [A] takeP :: Monad m => Int -> Parser a m b -> Parser a m b
867− [A] takeGE :: Monad m => Int -> Fold m a b -> Parser a m b
868− [A] takeFramedBy_ :: Monad m => (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
869− [A] takeFramedByGeneric :: Monad m => Maybe (a -> Bool) -> Maybe (a -> Bool) -> Maybe (a -> Bool) -> Fold m a b -> Parser a m b
870− [A] takeFramedByEsc_ :: Monad m => (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
871− [A] takeEndBy_ :: (a -> Bool) -> Parser a m b -> Parser a m b
872− [A] takeEndByEsc :: Monad m => (a -> Bool) -> (a -> Bool) -> Parser a m b -> Parser a m b
873− [A] takeEndBy :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
874− [A] takeEitherSepBy :: (a -> Bool) -> Fold m (Either a b) c -> Parser a m c
875− [A] takeEQ :: Monad m => Int -> Fold m a b -> Parser a m b
876− [A] takeBetween :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
877− [A] subsequenceBy :: (a -> a -> Bool) -> Stream m a -> Parser a m ()
878− [A] streamEqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Parser a m ()
879− [A] split_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
880− [A] splitWith :: Monad m => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
881− [A] splitSome :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
882− [A] splitManyPost :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
883− [A] splitMany :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
884− [A] spanByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
885− [A] spanBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
886− [A] span :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
887− [A] some :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
888− [A] sequence :: Monad m => Stream m (Parser a m b) -> Fold m b c -> Parser a m c
889− [A] sepByAll :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
890− [A] sepBy1 :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
891− [A] sepBy :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
892− [A] satisfy :: Monad m => (a -> Bool) -> Parser a m a
893− [A] sampleFromthen :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
894− [A] roundRobin :: t (Parser a m b) -> Fold m b c -> Parser a m c
895− [A] rmapM :: Monad m => (b -> m c) -> Parser a m b -> Parser a m c
896− [A] retryMaxTotal :: Int -> Parser a m b -> Fold m b c -> Parser a m c
897− [A] retryMaxSuccessive :: Int -> Parser a m b -> Fold m b c -> Parser a m c
898− [A] retry :: Parser a m b -> Parser a m b
899− [A] postscan :: Fold m a b -> Parser b m c -> Parser a m c
900− [A] peek :: Monad m => Parser a m a
901− [A] oneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
902− [A] oneNotEq :: (Monad m, Eq a) => a -> Parser a m a
903− [A] oneEq :: (Monad m, Eq a) => a -> Parser a m a
904− [A] one :: Monad m => Parser a m a
905− [A] noneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
906− [A] noErrorUnsafeSplit_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
907− [A] noErrorUnsafeSplitWith :: Monad m => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
908− [A] noErrorUnsafeConcatMap :: Monad m => (b -> Parser a m c) -> Parser a m b -> Parser a m c
909− [A] maybe :: Monad m => (a -> Maybe b) -> Parser a m b
910− [A] manyTillP :: Parser a m b -> Parser a m x -> Parser b m c -> Parser a m c
911− [A] manyTill :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
912− [A] manyThen :: Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
913− [A] manyP :: Parser a m b -> Parser b m c -> Parser a m c
914− [A] many :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
915− [A] makeIndexFilter :: (Fold m (s, a) b -> Parser a m b) -> (((s, a) -> Bool) -> Fold m (s, a) b -> Fold m (s, a) b) -> ((s, a) -> Bool) -> Fold m a b -> Parser a m b
916− [A] lookAhead :: Monad m => Parser a m b -> Parser a m b
917− [A] lmapM :: Monad m => (a -> m b) -> Parser b m r -> Parser a m r
918− [A] lmap :: (a -> b) -> Parser b m r -> Parser a m r
919− [A] listEqBy :: Monad m => (a -> a -> Bool) -> [a] -> Parser a m [a]
920− [A] listEq :: (Monad m, Eq a) => [a] -> Parser a m [a]
921− [A] indexed :: forall m a b. Monad m => Fold m (Int, a) b -> Parser a m b
922− [A] groupByRollingEither :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
923− [A] groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
924− [A] groupBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
925− [A] fromPure :: Monad m => b -> Parser a m b
926− [A] fromFoldMaybe :: Monad m => String -> Fold m a (Maybe b) -> Parser a m b
927− [A] fromFold :: Monad m => Fold m a b -> Parser a m b
928− [A] fromEffect :: Monad m => m b -> Parser a m b
929− [A] filter :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
930− [A] extractStep :: Monad m => (s -> m (Step s1 b)) -> Step s b -> m (Step s1 b)
931− [A] eof :: Monad m => Parser a m ()
932− [A] either :: Monad m => (a -> Either String b) -> Parser a m b
933− [A] dropWhile :: Monad m => (a -> Bool) -> Parser a m ()
934− [A] dieM :: Monad m => m String -> Parser a m b
935− [A] die :: Monad m => String -> Parser a m b
936− [A] deintercalateAll :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
937− [A] deintercalate1 :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
938− [A] deintercalate :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
939− [A] countBetween :: Int -> Int -> Parser a m b -> Fold m b c -> Parser a m c
940− [A] count :: Int -> Parser a m b -> Fold m b c -> Parser a m c
941− [A] concatMap :: Monad m => (b -> Parser a m c) -> Parser a m b -> Parser a m c
942− [A] blockWithQuotes :: (Monad m, Eq a) => (a -> Bool) -> (a -> Bool) -> a -> a -> Fold m a b -> Parser a m b
943− [A] bimapOverrideCount :: Int -> (s -> s1) -> (b -> b1) -> Step s b -> Step s1 b1
944− [A] alt :: Monad m => Parser x m a -> Parser x m a -> Parser x m a
945−[A] Streamly.Internal.Data.MutByteArray
946− [A] class Unbox a
947− [A] class SizeOfRep (f :: Type -> Type)
948− [A] class Serialize a
949− [A] TypeOfType
950− [A] UnitType :: Name -> TypeOfType
951− [A] TheType :: SimpleDataCon -> TypeOfType
952− [A] MultiType :: [SimpleDataCon] -> TypeOfType
953− [A] SimpleDataCon
954− [A] SimpleDataCon :: Name -> [Field] -> SimpleDataCon
955− [A] SerializeConfig
956− [A] [cfgRecordSyntaxWithHeader] :: SerializeConfig -> Bool
957− [A] [cfgInlineSize] :: SerializeConfig -> Maybe Inline
958− [A] [cfgInlineSerialize] :: SerializeConfig -> Maybe Inline
959− [A] [cfgInlineDeserialize] :: SerializeConfig -> Maybe Inline
960− [A] [cfgConstructorTagAsString] :: SerializeConfig -> Bool
961− [A] SerializeConfig :: Maybe Inline -> Maybe Inline -> Maybe Inline -> Bool -> Bool -> SerializeConfig
962− [A] PinnedState
963− [A] Unpinned :: PinnedState
964− [A] Pinned :: PinnedState
965− [A] MutByteArray
966− [A] MutByteArray :: MutableByteArray# RealWorld -> MutByteArray
967− [A] DataType
968− [A] [dtTvs] :: DataType -> [Name]
969− [A] [dtName] :: DataType -> Name
970− [A] [dtCxt] :: DataType -> Cxt
971− [A] [dtCons] :: DataType -> [DataCon]
972− [A] DataType :: Name -> [Name] -> Cxt -> [DataCon] -> DataType
973− [A] DataCon
974− [A] [dcTvs] :: DataCon -> [Name]
975− [A] [dcName] :: DataCon -> Name
976− [A] [dcFields] :: DataCon -> [(Maybe Name, Type)]
977− [A] [dcCxt] :: DataCon -> Cxt
978− [A] DataCon :: Name -> [Name] -> Cxt -> [(Maybe Name, Type)] -> DataCon
979− [A] BoundedPtr
980− [A] BoundedPtr :: MutByteArray -> Int -> Int -> BoundedPtr
981− [A] Streamly.Internal.Data.Serialize.Type.Serialize
982− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize Streamly.Internal.Data.MutByteArray.LiftedInteger
983− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize GHC.Num.Integer.Integer
984− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize a => Streamly.Internal.Data.Serialize.Type.Serialize (GHC.Maybe.Maybe a)
985− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize (Data.Proxy.Proxy a)
986− [A] instance (Streamly.Internal.Data.Serialize.Type.Serialize a, Streamly.Internal.Data.Serialize.Type.Serialize b) => Streamly.Internal.Data.Serialize.Type.Serialize (Data.Either.Either a b)
987− [A] Peeker
988− [A] Peeker :: Builder BoundedPtr IO a -> Peeker a
989− [A] type Field = (Maybe Name, Type)
990− [A] xorCmp :: [Word8] -> Name -> Name -> Q Exp
991− [A] wListToString :: [Word8] -> String
992− [A] w8_int :: Word8 -> Int
993− [A] w32_int :: Word32 -> Int
994− [A] unpin :: MutByteArray -> IO MutByteArray
995− [A] typeOfType :: Type -> [DataCon] -> TypeOfType
996− [A] skipByte :: Peeker ()
997− [A] sizeOfRep :: SizeOfRep f => f x -> Int
998− [A] sizeOfMutableByteArray :: MutByteArray -> IO Int
999− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
1000− [A] simplifyDataCon :: DataCon -> SimpleDataCon
1001− [A] serializeW8List :: Name -> Name -> [Word8] -> Q Exp
1002− [A] serializeConfig :: SerializeConfig
1003− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
1004− [A] runPeeker :: Peeker a -> BoundedPtr -> IO a
1005− [A] reifyDataType :: Name -> Q DataType
1006− [A] readUnsafe :: Unbox a => Peeker a
1007− [A] read :: Unbox a => Peeker a
1008− [A] putSliceUnsafe :: MonadIO m => MutByteArray -> Int -> MutByteArray -> Int -> Int -> m ()
1009− [A] pokeRep :: PokeRep f => f a -> BoundedPtr -> IO BoundedPtr
1010− [A] pokeBoundedPtrUnsafe :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr
1011− [A] pokeBoundedPtr :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr
1012− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
1013− [A] pinnedNewAlignedBytes :: Int -> Int -> IO MutByteArray
1014− [A] pinnedNew :: Int -> IO MutByteArray
1015− [A] pinnedCloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
1016− [A] pin :: MutByteArray -> IO MutByteArray
1017− [A] peekRep :: PeekRep f => Peeker (f x)
1018− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
1019− [A] openConstructor :: Name -> Int -> Q Pat
1020− [A] nil :: MutByteArray
1021− [A] newBytesAs :: PinnedState -> Int -> IO MutByteArray
1022− [A] new :: Int -> IO MutByteArray
1023− [A] mkSerializeExprFields :: Name -> [Field] -> Q Exp
1024− [A] mkRecSizeOfExpr :: SimpleDataCon -> Q Exp
1025− [A] mkRecSerializeExpr :: Name -> SimpleDataCon -> Q Exp
1026− [A] mkRecDeserializeExpr :: Name -> Name -> Name -> SimpleDataCon -> Q Exp
1027− [A] mkFieldName :: Int -> Name
1028− [A] mkDeserializeKeysDec :: Name -> Name -> SimpleDataCon -> Q [Dec]
1029− [A] mkDeserializeExprOne :: Name -> SimpleDataCon -> Q Exp
1030− [A] matchConstructor :: Name -> Int -> Q Exp -> Q Match
1031− [A] makeN :: Int -> Name
1032− [A] makeI :: Int -> Name
1033− [A] makeA :: Int -> Name
1034− [A] litProxy :: Unbox a => Proxy a -> Q Exp
1035− [A] litIntegral :: Integral a => a -> Q Exp
1036− [A] isUnitType :: [DataCon] -> Bool
1037− [A] isRecordSyntax :: SimpleDataCon -> Bool
1038− [A] isPinned :: MutByteArray -> Bool
1039− [A] int_w8 :: Int -> Word8
1040− [A] int_w32 :: Int -> Word32
1041− [A] inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
1042− [A] inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
1043− [A] inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig
1044− [A] getMutableByteArray# :: MutByteArray -> MutableByteArray# RealWorld
1045− [A] genericSizeOf :: forall a. SizeOfRep (Rep a) => Proxy a -> Int
1046− [A] genericPokeByteIndex :: (Generic a, PokeRep (Rep a)) => MutByteArray -> Int -> a -> IO ()
1047− [A] genericPeekByteIndex :: (Generic a, PeekRep (Rep a)) => MutByteArray -> Int -> IO a
1048− [A] errorUnsupported :: String -> a
1049− [A] errorUnimplemented :: a
1050− [A] encodeRecordFields :: Bool -> SerializeConfig -> SerializeConfig
1051− [A] encodeConstrNames :: Bool -> SerializeConfig -> SerializeConfig
1052− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
1053− [A] deriveUnbox :: Q [Dec] -> Q [Dec]
1054− [A] deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]
1055− [A] deriveSerialize :: Q [Dec] -> Q [Dec]
1056− [A] conUpdateFuncDec :: Name -> [Field] -> Q [Dec]
1057− [A] cloneSliceUnsafeAs :: MonadIO m => PinnedState -> Int -> Int -> MutByteArray -> m MutByteArray
1058− [A] cloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
1059− [A] c2w :: Char -> Word8
1060− [A] asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
1061− [A] addSizeTo :: Serialize a => Int -> a -> Int
1062− [A] _x :: Name
1063− [A] _val :: Name
1064− [A] _tag :: Name
1065− [A] _initialOffset :: Name
1066− [A] _endOffset :: Name
1067− [A] _arr :: Name
1068− [A] _acc :: Name
1069−[A] Streamly.Internal.Data.MutArray.Stream
1070− [A] SpliceState
1071− [A] SpliceYielding :: arr -> SpliceState s arr -> SpliceState s arr
1072− [A] SpliceInitial :: s -> SpliceState s arr
1073− [A] SpliceFinish :: SpliceState s arr
1074− [A] SpliceBuffering :: s -> arr -> SpliceState s arr
1075− [A] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
1076− [A] splitOn :: (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> Stream m (MutArray a)
1077− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1078− [A] packArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1079− [A] lpackArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
1080− [A] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
1081− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1082− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1083− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (Either ParseError (MutArray a))
1084− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1085− [A] compactEQ :: Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1086− [A] compact :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1087− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1088−[A] Streamly.Internal.Data.MutArray.Generic
1089− [A] MutArray
1090− [A] [arrTrueLen] :: MutArray a -> {-# UNPACK #-} !Int
1091− [A] [arrStart] :: MutArray a -> {-# UNPACK #-} !Int
1092− [A] [arrLen] :: MutArray a -> {-# UNPACK #-} !Int
1093− [A] [arrContents#] :: MutArray a -> MutableArray# RealWorld a
1094− [A] MutArray :: MutableArray# RealWorld a -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> MutArray a
1095− [A] writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
1096− [A] writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
1097− [A] writeN :: MonadIO m => Int -> Fold m a (MutArray a)
1098− [A] write :: MonadIO m => Fold m a (MutArray a)
1099− [A] uninit :: MonadIO m => MutArray a -> Int -> m (MutArray a)
1100− [A] toStreamK :: MonadIO m => MutArray a -> StreamK m a
1101− [A] toList :: MonadIO m => MutArray a -> m [a]
1102− [A] strip :: MonadIO m => (a -> Bool) -> MutArray a -> m (MutArray a)
1103− [A] snocWith :: MonadIO m => (Int -> Int) -> MutArray a -> a -> m (MutArray a)
1104− [A] snocUnsafe :: MonadIO m => MutArray a -> a -> m (MutArray a)
1105− [A] snoc :: MonadIO m => MutArray a -> a -> m (MutArray a)
1106− [A] realloc :: MonadIO m => Int -> MutArray a -> m (MutArray a)
1107− [A] reader :: MonadIO m => Unfold m (MutArray a) a
1108− [A] readRev :: MonadIO m => MutArray a -> Stream m a
1109− [A] read :: MonadIO m => MutArray a -> Stream m a
1110− [A] putSliceUnsafe :: MonadIO m => MutArray a -> Int -> MutArray a -> Int -> Int -> m ()
1111− [A] putIndices :: MonadIO m => MutArray a -> Fold m (Int, a) ()
1112− [A] putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
1113− [A] putIndex :: MonadIO m => Int -> MutArray a -> a -> m ()
1114− [A] producerWith :: Monad m => (forall b. IO b -> m b) -> Producer m (MutArray a) a
1115− [A] producer :: MonadIO m => Producer m (MutArray a) a
1116− [A] nil :: MonadIO m => m (MutArray a)
1117− [A] new :: MonadIO m => Int -> m (MutArray a)
1118− [A] modifyIndexUnsafe :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
1119− [A] modifyIndex :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
1120− [A] length :: MutArray a -> Int
1121− [A] getSliceUnsafe :: Int -> Int -> MutArray a -> MutArray a
1122− [A] getSlice :: Int -> Int -> MutArray a -> MutArray a
1123− [A] getIndexUnsafeWith :: MonadIO m => MutableArray# RealWorld a -> Int -> m a
1124− [A] getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
1125− [A] getIndex :: MonadIO m => Int -> MutArray a -> m (Maybe a)
1126− [A] fromStreamN :: MonadIO m => Int -> Stream m a -> m (MutArray a)
1127− [A] fromStream :: MonadIO m => Stream m a -> m (MutArray a)
1128− [A] fromPureStream :: MonadIO m => Stream Identity a -> m (MutArray a)
1129− [A] fromListN :: MonadIO m => Int -> [a] -> m (MutArray a)
1130− [A] fromList :: MonadIO m => [a] -> m (MutArray a)
1131− [A] eq :: (MonadIO m, Eq a) => MutArray a -> MutArray a -> m Bool
1132− [A] cmp :: (MonadIO m, Ord a) => MutArray a -> MutArray a -> m Ordering
1133− [A] clone :: MonadIO m => MutArray a -> m (MutArray a)
1134− [A] chunksOf :: forall m a. MonadIO m => Int -> Stream m a -> Stream m (MutArray a)
1135−[A] Streamly.Internal.Data.MutArray
1136− [A] MutByteArray
1137− [A] MutArray
1138− [A] [arrStart] :: MutArray a -> {-# UNPACK #-} !Int
1139− [A] [arrEnd] :: MutArray a -> {-# UNPACK #-} !Int
1140− [A] [arrContents] :: MutArray a -> {-# UNPACK #-} !MutByteArray
1141− [A] [arrBound] :: MutArray a -> {-# UNPACK #-} !Int
1142− [A] MutArray :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> MutArray a
1143− [A] IORef
1144− [A] ArrayUnsafe
1145− [A] ArrayUnsafe :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> ArrayUnsafe a
1146− [A] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1147− [A] writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1148− [A] writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
1149− [A] writeNWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
1150− [A] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1151− [A] writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1152− [A] writeIORef :: Unbox a => IORef a -> a -> IO ()
1153− [A] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
1154− [A] writeAppendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
1155− [A] writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
1156− [A] writeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
1157− [A] writeAppend :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
1158− [A] write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
1159− [A] unsafeSwapIndices :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> MutArray a -> m ()
1160− [A] unpin :: MutArray a -> IO (MutArray a)
1161− [A] toStreamKWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> StreamK m a
1162− [A] toStreamKRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> StreamK m a
1163− [A] toStreamKRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
1164− [A] toStreamK :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
1165− [A] toStreamDWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
1166− [A] toStreamDRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
1167− [A] toList :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m [a]
1168− [A] swapIndices :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> MutArray a -> m ()
1169− [A] strip :: forall a m. (Unbox a, MonadIO m) => (a -> Bool) -> MutArray a -> m (MutArray a)
1170− [A] splitOn :: (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> Stream m (MutArray a)
1171− [A] splitAt :: forall a. Unbox a => Int -> MutArray a -> (MutArray a, MutArray a)
1172− [A] spliceWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int -> Int) -> MutArray a -> MutArray a -> m (MutArray a)
1173− [A] spliceUnsafe :: MonadIO m => MutArray a -> MutArray a -> m (MutArray a)
1174− [A] spliceExp :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
1175− [A] spliceCopy :: forall m a. MonadIO m => MutArray a -> MutArray a -> m (MutArray a)
1176− [A] splice :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
1177− [A] snocWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> MutArray a -> a -> m (MutArray a)
1178− [A] snocUnsafe :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1179− [A] snocMay :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (Maybe (MutArray a))
1180− [A] snocLinear :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1181− [A] snoc :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1182− [A] shuffleBy :: (a -> a -> m Bool) -> MutArray a -> MutArray a -> m ()
1183− [A] roundUpToPower2 :: Int -> Int
1184− [A] rightSize :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m (MutArray a)
1185− [A] reverse :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m ()
1186− [A] resizeExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1187− [A] resize :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1188− [A] realloc :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1189− [A] readerRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Unfold m (MutArray a) a
1190− [A] readerRev :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
1191− [A] reader :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
1192− [A] readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
1193− [A] readIORef :: Unbox a => IORef a -> IO a
1194− [A] read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
1195− [A] putIndices :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Fold m (Int, a) ()
1196− [A] putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
1197− [A] putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
1198− [A] producerWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Producer m (MutArray a) a
1199− [A] producer :: forall m a. (MonadIO m, Unbox a) => Producer m (MutArray a) a
1200− [A] pollIntIORef :: (MonadIO m, Unbox a) => IORef a -> Stream m a
1201− [A] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1202− [A] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (MutArray a)
1203− [A] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1204− [A] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
1205− [A] pinnedNewBytes :: MonadIO m => Int -> m (MutArray a)
1206− [A] pinnedNewAligned :: (MonadIO m, Unbox a) => Int -> Int -> m (MutArray a)
1207− [A] pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
1208− [A] pinnedFromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1209− [A] pinnedFromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1210− [A] pinnedClone :: MonadIO m => MutArray a -> m (MutArray a)
1211− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1212− [A] pin :: MutArray a -> IO (MutArray a)
1213− [A] permute :: MutArray a -> m Bool
1214− [A] partitionBy :: forall m a. (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> m (MutArray a, MutArray a)
1215− [A] nil :: MutArray a
1216− [A] newIORef :: forall a. Unbox a => a -> IO (IORef a)
1217− [A] newArrayWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int -> m MutByteArray) -> Int -> Int -> m (MutArray a)
1218− [A] new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
1219− [A] modifyIndices :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (Int -> a -> a) -> Fold m Int ()
1220− [A] modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
1221− [A] modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
1222− [A] modifyIORef' :: Unbox a => IORef a -> (a -> a) -> IO ()
1223− [A] modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()
1224− [A] mergeBy :: Int -> (MutArray a -> MutArray a -> m ()) -> MutArray a -> m ()
1225− [A] memcpy :: Ptr Word8 -> Ptr Word8 -> Int -> IO ()
1226− [A] memcmp :: Ptr Word8 -> Ptr Word8 -> Int -> IO Bool
1227− [A] length :: forall a. Unbox a => MutArray a -> Int
1228− [A] isPinned :: MutArray a -> Bool
1229− [A] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
1230− [A] getSliceUnsafe :: forall a. Unbox a => Int -> Int -> MutArray a -> MutArray a
1231− [A] getSlice :: forall a. Unbox a => Int -> Int -> MutArray a -> MutArray a
1232− [A] getIndicesD :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
1233− [A] getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1234− [A] getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1235− [A] getIndexRev :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1236− [A] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
1237− [A] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
1238− [A] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
1239− [A] fromStreamD :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
1240− [A] fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
1241− [A] fromPureStream :: (MonadIO m, Unbox a) => Stream Identity a -> m (MutArray a)
1242− [A] fromListRevN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1243− [A] fromListRev :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1244− [A] fromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1245− [A] fromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1246− [A] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
1247− [A] foldr :: (MonadIO m, Unbox a) => (a -> b -> b) -> b -> MutArray a -> m b
1248− [A] foldl' :: (MonadIO m, Unbox a) => (b -> a -> b) -> b -> MutArray a -> m b
1249− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1250− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1251− [A] divideBy :: Int -> (MutArray a -> m (MutArray a, MutArray a)) -> MutArray a -> m ()
1252− [A] cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
1253− [A] clone :: MonadIO m => MutArray a -> m (MutArray a)
1254− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1255− [A] castUnsafe :: MutArray a -> MutArray b
1256− [A] cast :: forall a b. Unbox b => MutArray a -> Maybe (MutArray b)
1257− [A] c_memchr :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)
1258− [A] bytesFree :: MutArray a -> Int
1259− [A] byteLength :: MutArray a -> Int
1260− [A] byteCapacity :: MutArray a -> Int
1261− [A] bubble :: (MonadIO m, Unbox a) => (a -> a -> Ordering) -> MutArray a -> m ()
1262− [A] breakOn :: MonadIO m => Word8 -> MutArray Word8 -> m (MutArray Word8, Maybe (MutArray Word8))
1263− [A] blockSize :: Int
1264− [A] asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
1265− [A] asBytes :: MutArray a -> MutArray Word8
1266− [A] arrayChunkBytes :: Int
1267− [A] allocBytesToElemCount :: Unbox a => a -> Int -> Int
1268−[C] Streamly.Internal.Data.IsMap
1269− [A] mapTraverseWithKey :: (IsMap f, Applicative t) => (Key f -> a -> t b) -> f a -> t (f b)
1270−[R] Streamly.Internal.Data.IORef.Unboxed
1271−[R] Streamly.Internal.Data.Fold.Window
1272−[R] Streamly.Internal.Data.Fold.Type
1273−[R] Streamly.Internal.Data.Fold.Tee
1274−[R] Streamly.Internal.Data.Fold.Step
1275−[R] Streamly.Internal.Data.Fold.Container
1276−[D] Streamly.Internal.Data.Fold.Chunked
1277−[C] Streamly.Internal.Data.Fold
1278− [A] ManyState
1279− [C] Fold
1280− [C] Fold
1281− [O] Fold :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> Fold m a b
1282− [N] Fold :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> (s -> m b) -> Fold m a b
1283− [A] windowSumInt :: forall m a. (Monad m, Integral a) => Fold m (a, Maybe a) a
1284− [A] windowSum :: forall m a. (Monad m, Num a) => Fold m (a, Maybe a) a
1285− [A] windowRollingMapM :: Monad m => (Maybe a -> a -> m (Maybe b)) -> Fold m (a, Maybe a) (Maybe b)
1286− [A] windowRollingMap :: Monad m => (Maybe a -> a -> Maybe b) -> Fold m (a, Maybe a) (Maybe b)
1287− [A] windowRange :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe (a, a))
1288− [A] windowPowerSumFrac :: (Monad m, Floating a) => a -> Fold m (a, Maybe a) a
1289− [A] windowPowerSum :: (Monad m, Num a) => Int -> Fold m (a, Maybe a) a
1290− [A] windowMinimum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
1291− [A] windowMean :: forall m a. (Monad m, Fractional a) => Fold m (a, Maybe a) a
1292− [A] windowMaximum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
1293− [A] windowLmap :: (c -> a) -> Fold m (a, Maybe a) b -> Fold m (c, Maybe c) b
1294− [A] windowLength :: (Monad m, Num b) => Fold m (a, Maybe a) b
1295− [A] toSet :: (Monad m, Ord a) => Fold m a (Set a)
1296− [A] toMapIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
1297− [A] toMap :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
1298− [A] toIntSet :: Monad m => Fold m Int IntSet
1299− [A] toContainerIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (f b)
1300− [A] toContainer :: (Monad m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (f b)
1301− [A] nubInt :: Monad m => Fold m Int (Maybe Int)
1302− [A] nub :: (Monad m, Ord a) => Fold m a (Maybe a)
1303− [A] mapMStep :: Applicative m => (a -> m b) -> Step s a -> m (Step s b)
1304− [A] kvToMap :: (Monad m, Ord k) => Fold m a b -> Fold m (k, a) (Map k b)
1305− [A] frequency :: (Monad m, Ord a) => Fold m a (Map a Int)
1306− [A] demuxToMapIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
1307− [A] demuxToMap :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
1308− [A] demuxToContainerIO :: (MonadIO m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
1309− [A] demuxToContainer :: (Monad m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
1310− [A] demuxKvToMap :: (Monad m, Ord k) => (k -> m (Fold m a b)) -> Fold m (k, a) (Map k b)
1311− [A] demuxKvToContainer :: (Monad m, IsMap f, Traversable f) => (Key f -> m (Fold m a b)) -> Fold m (Key f, a) (f b)
1312− [A] demuxIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
1313− [A] demuxGenericIO :: (MonadIO m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (m (f b), Maybe (Key f, b))
1314− [A] demuxGeneric :: (Monad m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (m (f b), Maybe (Key f, b))
1315− [A] demux :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
1316− [A] cumulative :: Fold m (a, Maybe a) b -> Fold m a b
1317− [A] countDistinctInt :: Monad m => Fold m Int Int
1318− [A] countDistinct :: (Monad m, Ord a) => Fold m a Int
1319− [A] classifyIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
1320− [A] classifyGenericIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
1321− [A] classifyGeneric :: (Monad m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
1322− [A] classify :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
1323− [A] chainStepM :: Applicative m => (s1 -> m s2) -> (a -> m (Step s2 b)) -> Step s1 a -> m (Step s2 b)
1324−[C] Streamly.Internal.Data.Builder
1325− [C] Builder
1326− [C] Builder
1327− [O] Builder :: (s -> m (s, a)) -> Builder s m a
1328− [N] Builder :: (s -> m (a, s)) -> Builder s m a
1329−[A] Streamly.Internal.Data.Binary.Stream
1330− [A] class ToBytes a
1331− [A] word8 :: Applicative m => Word8 -> Stream m Word8
1332− [A] word64le :: Monad m => Word64 -> Stream m Word8
1333− [A] word64host :: Monad m => Word64 -> Stream m Word8
1334− [A] word64be :: Monad m => Word64 -> Stream m Word8
1335− [A] word32le :: Monad m => Word32 -> Stream m Word8
1336− [A] word32be :: Monad m => Word32 -> Stream m Word8
1337− [A] word16le :: Monad m => Word16 -> Stream m Word8
1338− [A] word16be :: Monad m => Word16 -> Stream m Word8
1339− [A] unit :: Applicative m => Stream m Word8
1340− [A] toBytes :: ToBytes a => a -> Stream m Word8
1341− [A] ordering :: Applicative m => Ordering -> Stream m Word8
1342− [A] int8 :: Applicative m => Int8 -> Stream m Word8
1343− [A] int64le :: Monad m => Int64 -> Stream m Word8
1344− [A] int64be :: Monad m => Int64 -> Stream m Word8
1345− [A] int32le :: Monad m => Int32 -> Stream m Word8
1346− [A] int32be :: Monad m => Int32 -> Stream m Word8
1347− [A] int16le :: Monad m => Int16 -> Stream m Word8
1348− [A] int16be :: Monad m => Int16 -> Stream m Word8
1349− [A] float32le :: Monad m => Float -> Stream m Word8
1350− [A] float32be :: Monad m => Float -> Stream m Word8
1351− [A] double64le :: Monad m => Double -> Stream m Word8
1352− [A] double64be :: Monad m => Double -> Stream m Word8
1353− [A] charUtf8 :: Monad m => Char -> Stream m Word8
1354− [A] charLatin1 :: Applicative m => Char -> Stream m Word8
1355− [A] bool :: Applicative m => Bool -> Stream m Word8
1356−[A] Streamly.Internal.Data.Binary.Parser
1357− [A] class FromBytes a
1358− [A] word8 :: Monad m => Parser Word8 m Word8
1359− [A] word64le :: Monad m => Parser Word8 m Word64
1360− [A] word64host :: MonadIO m => Parser Word8 m Word64
1361− [A] word64be :: Monad m => Parser Word8 m Word64
1362− [A] word32le :: Monad m => Parser Word8 m Word32
1363− [A] word32be :: Monad m => Parser Word8 m Word32
1364− [A] word16le :: Monad m => Parser Word8 m Word16
1365− [A] word16be :: Monad m => Parser Word8 m Word16
1366− [A] unit :: Monad m => Parser Word8 m ()
1367− [A] ordering :: Monad m => Parser Word8 m Ordering
1368− [A] int8 :: Monad m => Parser Word8 m Int8
1369− [A] int64le :: Monad m => Parser Word8 m Int64
1370− [A] int64be :: Monad m => Parser Word8 m Int64
1371− [A] int32le :: Monad m => Parser Word8 m Int32
1372− [A] int32be :: Monad m => Parser Word8 m Int32
1373− [A] int16le :: Monad m => Parser Word8 m Int16
1374− [A] int16be :: Monad m => Parser Word8 m Int16
1375− [A] fromBytes :: FromBytes a => Parser Word8 m a
1376− [A] float32le :: MonadIO m => Parser Word8 m Float
1377− [A] float32be :: MonadIO m => Parser Word8 m Float
1378− [A] eqWord8 :: Monad m => Word8 -> Parser Word8 m Word8
1379− [A] double64le :: MonadIO m => Parser Word8 m Double
1380− [A] double64be :: MonadIO m => Parser Word8 m Double
1381− [A] charLatin1 :: Monad m => Parser Word8 m Char
1382− [A] bool :: Monad m => Parser Word8 m Bool
1383−[R] Streamly.Internal.Data.Array.Type
1384−[A] Streamly.Internal.Data.Array.Stream
1385− [A] unlines :: forall m a. (MonadIO m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1386− [A] toArray :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
1387− [A] splitOnSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
1388− [A] splitOn :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
1389− [A] runArrayParserDBreak :: forall m a b. (MonadIO m, Unbox a) => Parser (Array a) m b -> Stream m (Array a) -> m (Either ParseError b, Stream m (Array a))
1390− [A] runArrayFoldMany :: (Monad m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> StreamK m (Either ParseError b)
1391− [A] runArrayFoldBreak :: (MonadIO m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1392− [A] runArrayFold :: (MonadIO m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> m (Either ParseError b)
1393− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1394− [A] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
1395− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1396− [A] parseBreak :: (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1397− [A] lpackArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
1398− [A] interposeSuffix :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1399− [A] interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1400− [A] intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a
1401− [A] foldBreakD :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
1402− [A] foldBreak :: (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
1403− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1404− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1405− [A] concatRev :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
1406− [A] concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
1407− [A] compact :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
1408− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1409− [A] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
1410−[R] Streamly.Internal.Data.Array.Mut.Type
1411−[R] Streamly.Internal.Data.Array.Mut.Stream
1412−[R] Streamly.Internal.Data.Array.Mut
1413−[R] Streamly.Internal.Data.Array.Generic.Mut.Type
1414−[C] Streamly.Internal.Data.Array.Generic
1415− [A] getIndex :: Int -> Array a -> Maybe a
1416− [A] fromPureStream :: Stream Identity a -> Array a
1417− [A] fromByteStr# :: Addr# -> Array Word8
1418− [A] chunksOf :: forall m a. MonadIO m => Int -> Stream m a -> Stream m (Array a)
1419−[C] Streamly.Internal.Data.Array
1420− [A] ArrayUnsafe
1421− [A] ArrayUnsafe :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> ArrayUnsafe a
1422− [C] Array
1423− [A] [arrStart] :: Array a -> {-# UNPACK #-} !Int
1424− [A] [arrEnd] :: Array a -> {-# UNPACK #-} !Int
1425− [A] [arrContents] :: Array a -> {-# UNPACK #-} !MutByteArray
1426− [A] Array :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> Array a
1427− [A] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1428− [A] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1429− [R] writeNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
1430− [A] unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b
1431− [A] unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a
1432− [D] unsafeIndex :: forall a. Unbox a => Int -> Array a -> a
1433− [A] unsafeFreezeWithShrink :: Unbox a => MutArray a -> Array a
1434− [A] unpin :: Array a -> IO (Array a)
1435− [A] toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a
1436− [A] toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a
1437− [A] toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
1438− [A] toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
1439− [A] splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)
1440− [A] splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
1441− [A] serialize :: Serialize a => a -> Array Word8
1442− [A] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1443− [A] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
1444− [A] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1445− [A] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
1446− [A] pinnedSerialize :: Serialize a => a -> Array Word8
1447− [A] pinnedFromListN :: Unbox a => Int -> [a] -> Array a
1448− [A] pinnedFromList :: Unbox a => [a] -> Array a
1449− [A] pinnedClone :: MonadIO m => Array a -> m (Array a)
1450− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1451− [A] pin :: Array a -> IO (Array a)
1452− [A] nil :: Array a
1453− [A] isPinned :: Array a -> Bool
1454− [A] getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a
1455− [A] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
1456− [A] fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)
1457− [A] fromPureStream :: Unbox a => Stream Identity a -> Array a
1458− [A] fromListRevN :: Unbox a => Int -> [a] -> Array a
1459− [A] fromListRev :: Unbox a => [a] -> Array a
1460− [A] fromByteStr# :: Addr# -> Array Word8
1461− [A] foldr :: Unbox a => (a -> b -> b) -> b -> Array a -> b
1462− [A] foldl' :: forall a b. Unbox a => (b -> a -> b) -> b -> Array a -> b
1463− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1464− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1465− [A] encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8
1466− [A] deserialize :: Serialize a => Array Word8 -> a
1467− [A] clone :: MonadIO m => Array a -> m (Array a)
1468− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1469− [A] byteLength :: Array a -> Int
1470− [A] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
1471− [A] breakOn :: MonadIO m => Word8 -> Array Word8 -> m (Array Word8, Maybe (Array Word8))
core/docs/ApiChangelogs/0.2.0-0.2.2.txtdeleted−306
@@ -1,306 +0,0 @@
1−---------------------------------
2−API Annotations
3−---------------------------------
4−
5−[A] : Added
6−[R] : Removed
7−[C] : Changed
8−[O] : Old definition
9−[N] : New definition
10−[D] : Deprecated
11−
12−---------------------------------
13−API diff
14−---------------------------------
15−
16−[C] Streamly.Data.Stream
17− [A] (FixityR,5)
18− [A] (FixityR,5)
19−[C] Streamly.Data.MutArray.Generic
20− [A] emptyOf :: MonadIO m => Int -> m (MutArray a)
21− [A] createOf :: MonadIO m => Int -> Fold m a (MutArray a)
22− [A] create :: MonadIO m => Fold m a (MutArray a)
23−[C] Streamly.Data.MutArray
24− [A] pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
25− [A] emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
26− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
27− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
28− [A] appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
29− [A] append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
30−[C] Streamly.Data.Array.Generic
31− [A] createOf :: MonadIO m => Int -> Fold m a (Array a)
32− [A] create :: MonadIO m => Fold m a (Array a)
33−[C] Streamly.Data.Array
34− [A] class Serialize a
35− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
36− [A] pinnedSerialize :: Serialize a => a -> Array Word8
37− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
38− [A] deserialize :: Serialize a => Array Word8 -> a
39− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
40− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
41− [A] addSizeTo :: Serialize a => Int -> a -> Int
42−
43−---------------------------------
44−Internal API diff
45−---------------------------------
46−
47−[C] Streamly.Internal.Unicode.Stream
48− [A] encodeUtf16le' :: Stream m Char -> Stream m Word16
49− [A] decodeUtf16le' :: Stream m Word16 -> Stream m Char
50−[A] Streamly.Internal.FileSystem.Path
51− [A] class IsPath a
52− [A] Rel
53− [A] File
54− [A] Dir
55− [A] Abs
56− [A] Streamly.Internal.FileSystem.Path.IsPath
57− [A] instance Streamly.Internal.FileSystem.Path.IsPath Streamly.Internal.FileSystem.Path.Path
58− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel Streamly.Internal.FileSystem.Path.Path)
59− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path))
60− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path))
61− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path)
62− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path)
63− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs Streamly.Internal.FileSystem.Path.Path)
64− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path))
65− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path))
66− [A] GHC.Show.Show
67− [A] instance GHC.Show.Show Streamly.Internal.FileSystem.Path.PathException
68− [A] GHC.Exception.Type.Exception
69− [A] instance GHC.Exception.Type.Exception Streamly.Internal.FileSystem.Path.PathException
70− [A] GHC.Classes.Eq
71− [A] instance GHC.Classes.Eq Streamly.Internal.FileSystem.Path.PathException
72− [A] Path
73− [A] Path :: Array Word8 -> Path
74− [A] toString :: Path -> [Char]
75− [A] toPath :: IsPath a => a -> Path
76− [A] toChunk :: Path -> Array Word8
77− [A] toChars :: Monad m => Path -> Stream m Char
78− [A] relfile :: QuasiQuoter
79− [A] reldir :: QuasiQuoter
80− [A] rel :: QuasiQuoter
81− [A] primarySeparator :: Char
82− [A] path :: QuasiQuoter
83− [A] mkRelFile :: String -> Q Exp
84− [A] mkRelDir :: String -> Q Exp
85− [A] mkRel :: String -> Q Exp
86− [A] mkPath :: String -> Q Exp
87− [A] mkFile :: String -> Q Exp
88− [A] mkDir :: String -> Q Exp
89− [A] mkAbsFile :: String -> Q Exp
90− [A] mkAbsDir :: String -> Q Exp
91− [A] mkAbs :: String -> Q Exp
92− [A] isSeparator :: Char -> Bool
93− [A] fromString :: MonadThrow m => [Char] -> m Path
94− [A] fromPathUnsafe :: IsPath a => Path -> a
95− [A] fromPath :: (IsPath a, MonadThrow m) => Path -> m a
96− [A] fromChunkUnsafe :: Array Word8 -> Path
97− [A] fromChunk :: MonadThrow m => Array Word8 -> m Path
98− [A] fromChars :: MonadThrow m => Stream Identity Char -> m Path
99− [A] file :: QuasiQuoter
100− [A] extendPath :: Path -> Path -> Path
101− [A] extendDir :: (IsPath (a (Dir Path)), IsPath b, IsPath (a b)) => a (Dir Path) -> Rel b -> a b
102− [A] dir :: QuasiQuoter
103− [A] adaptPath :: (MonadThrow m, IsPath a, IsPath b) => a -> m b
104− [A] absfile :: QuasiQuoter
105− [A] absdir :: QuasiQuoter
106− [A] abs :: QuasiQuoter
107−[C] Streamly.Internal.Data.Stream
108− [A] (FixityR,5)
109− [A] (FixityR,5)
110− [C] splitInnerBySuffix
111− [O] splitInnerBySuffix :: (Monad m, Eq (f a), Monoid (f a)) => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
112− [N] splitInnerBySuffix :: Monad m => (f a -> Bool) -> (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
113− [D] sliceOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
114− [A] indexOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
115−[C] Streamly.Internal.Data.MutByteArray
116− [A] unsafePinnedAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
117− [A] unsafeAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
118− [D] nil :: MutByteArray
119− [A] empty :: MutByteArray
120− [D] asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
121−[D] Streamly.Internal.Data.MutArray.Stream
122− [D] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
123− [D] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
124− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
125− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
126−[C] Streamly.Internal.Data.MutArray.Generic
127− [D] writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
128− [D] writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
129− [A] unsafeCreateOf :: MonadIO m => Int -> Fold m a (MutArray a)
130− [A] emptyOf :: MonadIO m => Int -> m (MutArray a)
131− [A] createWith :: MonadIO m => Int -> Fold m a (MutArray a)
132− [A] createOf :: MonadIO m => Int -> Fold m a (MutArray a)
133− [A] create :: MonadIO m => Fold m a (MutArray a)
134−[C] Streamly.Internal.Data.MutArray
135− [A] SpliceState
136− [A] SpliceYielding :: arr -> SpliceState s arr -> SpliceState s arr
137− [A] SpliceInitial :: s -> SpliceState s arr
138− [A] SpliceFinish :: SpliceState s arr
139− [A] SpliceBuffering :: s -> arr -> SpliceState s arr
140− [R] MutByteArray
141− [D] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
142− [D] writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
143− [D] writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
144− [D] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
145− [D] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
146− [D] writeAppendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
147− [D] writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
148− [A] unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
149− [A] unsafePinnedAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
150− [A] unsafeCreateOfWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
151− [A] unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
152− [A] unsafeAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
153− [A] unsafeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
154− [A] toStreamWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
155− [A] toStreamRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
156− [R] toStreamDWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
157− [R] toStreamDRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
158− [A] slicerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
159− [A] sliceIndexerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
160− [A] revCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
161− [D] resizeExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
162− [D] resize :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
163− [A] pokeSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
164− [A] pokeAppendMay :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> a -> m (Maybe (MutArray Word8))
165− [A] pokeAppend :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> a -> m (MutArray Word8)
166− [D] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
167− [D] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
168− [D] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
169− [D] pinnedNewBytes :: MonadIO m => Int -> m (MutArray a)
170− [A] pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
171− [A] pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
172− [A] pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
173− [A] pinnedCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
174− [A] peekUnconsUnsafe :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> m (a, MutArray Word8)
175− [A] peekUncons :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> m (Maybe a, MutArray Word8)
176− [A] peekSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
177− [A] pPinnedCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Parser (MutArray a) m (MutArray a)
178− [A] pCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Parser (MutArray a) m (MutArray a)
179− [D] nil :: MutArray a
180− [A] lPinnedCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
181− [A] lCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
182− [A] indexReaderWith :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
183− [A] indexReader :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
184− [A] growExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
185− [A] grow :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
186− [D] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
187− [R] getIndicesD :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
188− [D] getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
189− [D] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
190− [A] fromStreamN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
191− [D] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
192− [D] fromStreamD :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
193− [A] fromPureStreamN :: (MonadIO m, Unbox a) => Int -> Stream Identity a -> m (MutArray a)
194− [A] fromPtrN :: MonadIO m => Int -> Ptr Word8 -> m (MutArray Word8)
195− [A] fromChunksRealloced :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> m (MutArray a)
196− [A] fromChunksK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
197− [A] fromByteStr# :: MonadIO m => Addr# -> m (MutArray Word8)
198− [D] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
199− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
200− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
201− [A] fPinnedCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) (MutArray a)
202− [A] fCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) (MutArray a)
203− [A] emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
204− [A] empty :: MutArray a
205− [A] createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
206− [A] createOfWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
207− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
208− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
209− [A] concatWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m (MutArray a) -> Stream m a
210− [A] concatRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m (MutArray a) -> Stream m a
211− [A] concatRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
212− [A] concat :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
213− [A] compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (MutArray Word8) -> Stream m (MutArray Word8)
214− [A] compactOnByte :: MonadIO m => Word8 -> Stream m (MutArray Word8) -> Stream m (MutArray Word8)
215− [A] compactLeAs :: forall m a. (MonadIO m, Unbox a) => PinnedState -> Int -> Stream m (MutArray a) -> Stream m (MutArray a)
216− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
217− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
218− [A] compactEQ :: Int -> Stream m (MutArray a) -> Stream m (MutArray a)
219− [D] cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
220− [A] byteEq :: MonadIO m => MutArray a -> MutArray a -> m Bool
221− [A] byteCmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
222− [A] buildChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
223− [D] asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
224− [A] appendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
225− [A] appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
226− [A] append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
227−[D] Streamly.Internal.Data.Array.Stream
228− [C] interposeSuffix
229− [O] interposeSuffix :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
230− [N] interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
231− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
232− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
233− [C] concatRev
234− [O] concatRev :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
235− [N] concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
236− [D] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
237−[C] Streamly.Internal.Data.Array.Generic
238− [A] createOf :: MonadIO m => Int -> Fold m a (Array a)
239− [A] create :: MonadIO m => Fold m a (Array a)
240−[C] Streamly.Internal.Data.Array
241− [R] ArrayUnsafe
242− [D] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
243− [D] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
244− [A] unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
245− [A] unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
246− [A] unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
247− [D] toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
248− [D] toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
249− [C] splice
250− [O] splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
251− [N] splice :: MonadIO m => Array a -> Array a -> m (Array a)
252− [A] slicerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Array a)
253− [A] sliceIndexerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Int, Int)
254− [D] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
255− [D] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
256− [D] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
257− [D] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
258− [A] pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
259− [A] pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
260− [A] pinnedCompactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
261− [A] parseBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
262− [D] nil :: Array a
263− [A] lPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
264− [A] lCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
265− [A] interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
266− [A] interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
267− [A] intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a
268− [A] indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a
269− [A] indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
270− [A] indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int
271− [D] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Array a)
272− [R] getIndicesFromThenTo :: Unfold m (Int, Int, Int, Array a) a
273− [D] getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
274− [D] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Int, Int)
275− [D] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
276− [D] fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)
277− [A] fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a
278− [A] fromPtrN :: Int -> Ptr Word8 -> Array Word8
279− [A] fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)
280− [A] fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
281− [A] fromByteStr :: Ptr Word8 -> Array Word8
282− [A] foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b
283− [A] foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
284− [A] foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
285− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
286− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
287− [C] findIndicesOf
288− [O] findIndicesOf :: (a -> Bool) -> Unfold Identity (Array a) Int
289− [N] findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int
290− [A] fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
291− [A] fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
292− [A] empty :: Array a
293− [A] createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
294− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
295− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
296− [A] concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
297− [A] concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
298− [A] compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
299− [A] compactOnByte :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
300− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
301− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
302− [A] byteEq :: Array a -> Array a -> Bool
303− [A] byteCmp :: Array a -> Array a -> Ordering
304− [A] buildChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
305− [D] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
306− [D] asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
core/docs/Changelog.md+12−60
@@ -1,73 +1,25 @@
11# Changelog
22
3−## 0.2.2 (Jan 2024)
3+## Unreleased
44
5−* Add fixities `infixr 5` for `cons` and `consM` functions.
6−* Fix a bug in Array `Eq` instance when the type is a sum type with
7− differently sized constructors.
8−* lpackArraysChunksOf, compact, writeChunksWith, putChunksWith now take the
9− buffer size in number of array elements instead of bytes.
5+* Arrays are now created unpinned by default, they were created pinned
6+ earlier. During IO operations unpinned arrays are automatically copied
7+ to pinned memory. When arrays are directly passed to IO operations
8+ programmers can choose to create them pinned to avoid a copy. To
9+ create pinned arrays, use the internal APIs with the `pinned*` prefix.
1010
11−## 0.2.1 (Dec 2023)
12−
13−* Make the serialization of the unit constructor deterministic.
14−* Expose `pinnedSerialize` & `deserialize` via `Streamly.Data.Array`.
15−
16−## 0.2.0 (Nov 2023)
17−
18−See [0.1.0-0.2.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0-0.2.0.txt)
19−for a full list of API changes in this release. Only a few significant
20−changes are mentioned here.
21−
22−### Breaking Changes
23−
24−* `ParserK` in `Streamly.Data.ParserK` is not implicitly specialized
25− to arrays anymore. To adapt to the new code, change `ParserK a m
26− b` to `ParserK (Array a) m b` where the `Array` type comes from
27− `Streamly.Data.Array`. This change also affected the signatures of
28− `parseChunks` and `parseBreakChunks`.
29−* Changed the signature of 'Streamly.Data.Stream.handle' to make the
30− exception handler monadic.
31−* Behavior change: Exceptions are now rethrown promptly in `bracketIO`.
32−
33−### Enhancements
34−
35−* __Serialization__: Added a `Streamly.Data.MutByteArray` module with a
36− `Serialize` type class for fast binary serialization. The Data.Array
37− module supplies the `serialize` and `deserialize` operations for arrays.
38−* __Unpinned Arrays__: Unboxed arrays are now created unpinned by default,
39− they were created pinned earlier. During IO operations, unpinned arrays
40− are automatically copied to pinned memory. When arrays are directly
41− passed to IO operations programmers can choose to create them pinned to
42− avoid a copy. To create pinned arrays, use the internal APIs with the
43− `pinned*` prefix.
44−* StreamK now supports native exception handling routines (handle, bracketIO).
45− Earlier we had to convert it to the `Stream` type for exception handling.
11+* Changed the signature of 'Streamly.Data.Stream.handle' to make the exception
12+ handler monadic.
13+* Rethrow the exception promptly in bracketIO.
4614
4715### Deprecations
4816
49−See [0.1.0-0.2.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0-0.2.0.txt)
50−for a full list of deprecations.
51−
52−### Internal API Changes
53−
54−* Fold constructor has changed, added a `final` field to support
55− finalization and cleanup of a chain of folds. The `extract` field is
56− now used only for mapping the fold internal state to fold result for
57− scanning purposes. If your fold does not require cleanup you can just use
58− your existing `extract` function as `final` as well to adapt to this change.
59−* Many low level internal modules have been removed, they are entirely
60− exported from higher level internal modules. If you were importing any
61− of the missing low level modules then import the higher level modules instead.
62−* Internal module changes:
63− * Streamly.Internal.Serialize.FromBytes -> Streamly.Internal.Data.Binary.Parser
64− * Streamly.Internal.Serialize.ToBytes -> Streamly.Internal.Data.Binary.Stream
65− * Streamly.Internal.Data.Unbox is now exported via Streamly.Internal.Data.Serialize
66− * Streamly.Internal.Data.IORef.Unboxed is now exported via Streamly.Internal.Data.Serialize
17+* `Streamly.Data.MutArray.newUnpinned` is renamed to
18+ `Streamly.Data.MutArray.pinnedNew`
6719
6820## 0.1.0 (March 2023)
6921
70−Also see [streamly-core-0.1.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0.txt) or
22+Also see [streamly-core-0.1.0 API Changelog](/core/docs/ApiChangelogs/0.1.0.txt) or
7123https://hackage.haskell.org/package/streamly-core-0.1.0/docs/docs/ApiChangelogs/0.1.0.txt
7224
7325`streamly` package is split into two packages, (1) `streamly-core` that
core/src/DocTestDataArray.hs−8
@@ -1,7 +1,6 @@
11{- $setup
22>>> :m
33>>> :set -XFlexibleContexts
4−>>> :set -XMagicHash
54>>> import Data.Function ((&))
65>>> import Data.Functor.Identity (Identity(..))
76>>> import System.IO.Unsafe (unsafePerformIO)
@@ -11,12 +10,5 @@
1110
1211>>> import qualified Streamly.Data.Array as Array
1312>>> import qualified Streamly.Data.Fold as Fold
14−>>> import qualified Streamly.Data.ParserK as ParserK
1513>>> import qualified Streamly.Data.Stream as Stream
16−>>> import qualified Streamly.Data.StreamK as StreamK
17−
18−For APIs that have not been released yet.
19−
20−>>> import qualified Streamly.Internal.Data.Array as Array
21−>>> import qualified Streamly.Internal.Data.Stream as Stream
2214-}
core/src/DocTestDataFold.hs+1
@@ -24,4 +24,5 @@
2424For APIs that have not been released yet.
2525
2626>>> import qualified Streamly.Internal.Data.Fold as Fold
27+>>> import qualified Streamly.Internal.Data.Fold.Window as FoldW
2728-}
core/src/DocTestDataParserK.hsdeleted−15
@@ -1,15 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import Control.Applicative ((<|>))
4−>>> import Data.Char (isDigit, isAlpha)
5−
6−>>> import Streamly.Data.Parser (Parser)
7−>>> import Streamly.Data.ParserK (ParserK)
8−
9−>>> import qualified Streamly.Data.Parser as Parser
10−>>> import qualified Streamly.Data.ParserK as ParserK
11−
12−For APIs that have not been released yet.
13−
14−>>> import qualified Streamly.Internal.Data.ParserK as ParserK
15−-}
core/src/DocTestDataStream.hs+1−2
@@ -6,7 +6,6 @@
66>>> import Control.Monad.IO.Class (MonadIO (liftIO))
77>>> import Control.Monad.Trans.Class (lift)
88>>> import Control.Monad.Trans.Identity (runIdentityT)
9−>>> import Data.Char (isSpace)
109>>> import Data.Either (fromLeft, fromRight, isLeft, isRight, either)
1110>>> import Data.Maybe (fromJust, isJust)
1211>>> import Data.Function (fix, (&))
@@ -31,9 +30,9 @@
3130For APIs that have not been released yet.
3231
3332>>> import qualified Streamly.Internal.Data.Fold as Fold
33+>>> import qualified Streamly.Internal.Data.Fold.Window as Window
3434>>> import qualified Streamly.Internal.Data.Parser as Parser
3535>>> import qualified Streamly.Internal.Data.Stream as Stream
36−>>> import qualified Streamly.Internal.Data.StreamK as StreamK
3736>>> import qualified Streamly.Internal.Data.Unfold as Unfold
3837>>> import qualified Streamly.Internal.FileSystem.Dir as Dir
3938-}
core/src/DocTestFileSystemHandle.hs+1−1
@@ -8,7 +8,7 @@
88
99For APIs that have not been released yet.
1010
11−>>> import qualified Streamly.Internal.Data.Array as Array (unsafeCreateOf)
11+>>> import qualified Streamly.Internal.Data.Array.Type as Array (writeNUnsafe)
1212>>> import qualified Streamly.Internal.Data.Unfold as Unfold (first)
1313>>> import qualified Streamly.Internal.FileSystem.Handle as Handle
1414>>> import qualified Streamly.Internal.System.IO as IO (defaultChunkSize)
core/src/Streamly/Console/Stdio.hs+27−1
@@ -24,7 +24,33 @@ module Streamly.Console.Stdio
2424 -- * Write (stderr)
2525 , writeErr
2626 , writeErrChunks
27+
28+ -- * Deprecated
29+ , read
30+ , readChunks
2731 )
2832where
2933
30−import Streamly.Internal.Console.Stdio
34+import Control.Monad.IO.Class (MonadIO(..))
35+import Data.Word (Word8)
36+import Streamly.Internal.Data.Array.Type (Array)
37+import Streamly.Internal.Data.Unfold (Unfold)
38+
39+import Streamly.Internal.Console.Stdio hiding (read, readChunks)
40+import Prelude hiding (read)
41+
42+-- Same as 'reader'
43+--
44+-- @since 0.8.0
45+{-# DEPRECATED read "Please use 'reader' instead" #-}
46+{-# INLINE read #-}
47+read :: MonadIO m => Unfold m () Word8
48+read = reader
49+
50+-- Same as 'chunkReader'
51+--
52+-- @since 0.8.0
53+{-# DEPRECATED readChunks "Please use 'chunkReader' instead" #-}
54+{-# INLINE readChunks #-}
55+readChunks :: MonadIO m => Unfold m () (Array Word8)
56+readChunks = chunkReader
core/src/Streamly/Data/Array.hs+48−48
@@ -29,80 +29,65 @@ module Streamly.Data.Array
2929 -- $overview
3030
3131 -- * The Array Type
32− Array
33−
34− -- * Pinning & Unpinning
35− -- | Arrays are created unpinned by default unless pinned versions of
36− -- creation APIs are used. Look for APIs with @pinned@ prefix in
37− -- "Streamly.Internal.Data.Array" for some unreleased pinned creation APIs.
38− -- If an array is to be sent to the OS without any further modification
39− -- then it should be created pinned in the first place instead of pinning
40− -- it later. Pinning an unpinned array has a copy overhead. OS interfacing
41− -- APIs create a pinned array directly or convert an unpinned array to
42− -- pinned array before sending it to the OS.
43− , pin
44− , unpin
45− , isPinned
32+ A.Array
4633
4734 -- * Construction
4835 -- | When performance matters, the fastest way to generate an array is
49− -- 'createOf'. 'IsList' and 'IsString' instances can be
36+ -- 'writeN'. 'IsList' and 'IsString' instances can be
5037 -- used to conveniently construct arrays from literal values.
5138 -- 'OverloadedLists' extension or 'fromList' can be used to construct an
5239 -- array from a list literal. Similarly, 'OverloadedStrings' extension or
5340 -- 'fromList' can be used to construct an array from a string literal.
5441
55− -- ** From Stream
56− , createOf
57− , create
58− , writeLastN -- drop old (ring buffer)
42+ -- Pure List APIs
43+ , A.fromListN
44+ , A.fromList
5945
60− -- ** From List
61− , fromListN
62− , fromList
46+ -- Monadic APIs
47+ , A.writeN -- drop new
48+ , A.write -- full buffer
49+ , writeLastN -- drop old (ring buffer)
6350
64− -- * To List
51+ -- * Conversion
6552 -- 'GHC.Exts.toList' from "GHC.Exts" can be used to convert an array to a
6653 -- list.
67− , toList
68−
69− -- * To Stream
70− , read
71− , readRev
54+ , A.toList
7255
7356 -- * Unfolds
74− , reader
75− , readerRev
57+ , A.reader
58+ , A.readerRev
7659
7760 -- * Casting
7861 , cast
7962 , asBytes
8063
8164 -- * Random Access
82− , length
65+ , A.length
8366 -- , (!!)
84− , getIndex
85−
86− -- * Serialization
87− , pinnedSerialize
88− , deserialize
67+ , A.getIndex
8968
90− -- * Re-exports
69+ -- * Unbox Type Class
9170 , Unbox (..)
92− , Serialize(..)
71+
72+ -- * Pinning & Unpinning
73+ , pin
74+ , unpin
75+ , isPinned
9376
9477 -- * Deprecated
95− , writeN -- drop new
96− , write -- full buffer
78+ , read
79+ , readRev
9780 )
9881where
9982
10083#include "inline.hs"
10184
102−import Streamly.Internal.Data.Array
103−import Streamly.Internal.Data.MutByteArray (Unbox(..), Serialize(..))
85+import Streamly.Internal.Data.Unfold (Unfold)
86+import Streamly.Internal.Data.Array as A hiding (read, readRev)
87+import Streamly.Internal.Data.Array.Type (pin, unpin, isPinned)
10488
105−import Prelude hiding (read, length)
89+import Streamly.Internal.Data.Unbox (Unbox (..))
90+import Prelude hiding (read)
10691
10792#include "DocTestDataArray.hs"
10893
@@ -118,7 +103,7 @@ import Prelude hiding (read, length)
118103--
119104-- Convert array to stream, and fold the stream:
120105--
121−-- >>> fold f arr = Array.read arr & Stream.fold f
106+-- >>> fold f arr = Stream.unfold Array.reader arr & Stream.fold f
122107-- >>> fold Fold.sum (Array.fromList [1,2,3::Int])
123108-- 6
124109--
@@ -126,7 +111,7 @@ import Prelude hiding (read, length)
126111--
127112-- Convert array to stream, transform, and fold back to array:
128113--
129−-- >>> amap f arr = Array.read arr & fmap f & Stream.fold (Array.createOf (Array.length arr))
114+-- >>> amap f arr = Stream.unfold Array.reader arr & fmap f & Stream.fold Array.write
130115-- >>> amap (+1) (Array.fromList [1,2,3::Int])
131116-- fromList [2,3,4]
132117--
@@ -153,11 +138,12 @@ import Prelude hiding (read, length)
153138-- See the 'fromPureStream' unreleased API to generate an array from an
154139-- Identity stream safely without using MonadIO constraint.
155140--
141+--
156142-- Note that 'Identity' streams can be generalized to IO streams:
157143--
158−-- >>> stream = Stream.fromList [1,2,3] :: Stream Identity Int
159−-- >>> pure2IO s = Stream.morphInner (return . runIdentity) s :: Stream IO Int
160−-- >>> Stream.fold Array.create (pure2IO stream)
144+-- >>> pure = Stream.fromList [1,2,3] :: Stream Identity Int
145+-- >>> generally = Stream.morphInner (return . runIdentity)
146+-- >>> Stream.fold Array.write (generally pure :: Stream IO Int)
161147-- fromList [1,2,3]
162148--
163149-- == Programming Tips
@@ -165,3 +151,17 @@ import Prelude hiding (read, length)
165151-- This module is designed to be imported qualified:
166152--
167153-- >>> import qualified Streamly.Data.Array as Array
154+
155+-- | Same as 'reader'
156+--
157+{-# DEPRECATED read "Please use 'reader' instead" #-}
158+{-# INLINE_NORMAL read #-}
159+read :: (Monad m, Unbox a) => Unfold m (Array a) a
160+read = reader
161+
162+-- | Same as 'readerRev'
163+--
164+{-# DEPRECATED readRev "Please use 'readerRev' instead" #-}
165+{-# INLINE_NORMAL readRev #-}
166+readRev :: (Monad m, Unbox a) => Unfold m (Array a) a
167+readRev = readerRev
core/src/Streamly/Data/Array/Generic.hs+13−16
@@ -15,37 +15,34 @@ module Streamly.Data.Array.Generic
1515 ( Array
1616
1717 -- * Construction
18− , fromListN
19− , fromList
18+ , A.fromListN
19+ , A.fromList
2020
2121 -- MonadicAPIs
22− , createOf
23− , create
22+ , A.writeN
23+ , A.write
2424
2525 -- * Conversion
26− , toList
26+ , A.toList
2727
2828 -- * Streams
29− , read
30− , readRev
29+ , A.read
30+ , A.readRev
3131
3232 -- * Unfolds
33− , reader
33+ , A.reader
3434 -- , A.readerRev
3535
3636 -- * Random Access
37− , length
38− , getIndex
37+ , A.length
38+ , A.getIndex
3939
4040 -- -- * Folding Arrays
4141 -- , A.streamFold
4242 -- , A.fold
43−
44− -- * Deprecated
45− , writeN
46− , write
4743 )
4844where
4945
50−import Streamly.Internal.Data.Array.Generic
51−import Prelude hiding (length, read)
46+import Streamly.Internal.Data.Array.Generic (Array)
47+
48+import qualified Streamly.Internal.Data.Array.Generic as A
core/src/Streamly/Data/Fold.hs+118−150
@@ -10,156 +10,6 @@
1010-- Fast, composable stream consumers with ability to terminate, supporting
1111-- stream fusion.
1212--
13−-- == Using Folds
14−--
15−-- This module provides elementary folds and fold combinators that can be used
16−-- to consume a stream of data and reduce it to a final value, or transform it
17−-- in a stateful manner using scans. A data stream can be reduced into a stream
18−-- of folded data elements by folding segments of the stream. Fold combinators
19−-- can be used to compose multiple folds in parallel or to create a pipeline of
20−-- folds such that the next fold consumes the result of the previous fold. To
21−-- run these folds on a stream see 'Streamly.Data.Stream.fold',
22−-- 'Streamly.Data.Stream.scan', 'Streamly.Data.Stream.postscan',
23−-- 'Streamly.Data.Stream.scanMaybe', 'Streamly.Data.Stream.foldMany' and other
24−-- operations accepting 'Fold' type as argument "Streamly.Data.Stream".
25−--
26−-- == Reducing a Stream
27−--
28−-- A 'Fold' is a consumer of a stream of values. A fold driver (such as
29−-- 'Streamly.Data.Stream.fold') initializes the fold @accumulator@, runs the
30−-- fold @step@ function in a loop, processing the input stream one element at a
31−-- time and accumulating the result. The loop continues until the fold
32−-- terminates, at which point the accumulated result is returned.
33−--
34−-- For example, a 'sum' Fold represents a stream consumer that adds the values
35−-- in the input stream:
36−--
37−-- >>> Stream.fold Fold.sum $ Stream.fromList [1..100]
38−-- 5050
39−--
40−-- Conceptually, a 'Fold' is a data type that mimics a strict left fold
41−-- ('Data.List.foldl'). The above example is similar to a left fold using
42−-- @(+)@ as the step and @0@ as the initial value of the accumulator:
43−--
44−-- >>> Data.List.foldl' (+) 0 [1..100]
45−-- 5050
46−--
47−-- 'Fold's have an early termination capability e.g. the 'one' fold terminates
48−-- after consuming one element:
49−--
50−-- >>> Stream.fold Fold.one $ Stream.fromList [1..]
51−-- Just 1
52−--
53−-- The above example is similar to the following right fold:
54−--
55−-- >>> Prelude.foldr (\x _ -> Just x) Nothing [1..]
56−-- Just 1
57−--
58−-- 'Fold's can be combined together using combinators. For example, to create a
59−-- fold that sums first two elements in a stream:
60−--
61−-- >>> sumTwo = Fold.take 2 Fold.sum
62−-- >>> Stream.fold sumTwo $ Stream.fromList [1..100]
63−-- 3
64−--
65−-- == Parallel Composition
66−--
67−-- Folds can be combined to run in parallel on the same input. For example, to
68−-- compute the average of numbers in a stream without going through the stream
69−-- twice:
70−--
71−-- >>> avg = Fold.teeWith (/) Fold.sum (fmap fromIntegral Fold.length)
72−-- >>> Stream.fold avg $ Stream.fromList [1.0..100.0]
73−-- 50.5
74−--
75−-- Folds can be combined so as to partition the input stream over multiple
76−-- folds. For example, to count even and odd numbers in a stream:
77−--
78−-- >>> split n = if even n then Left n else Right n
79−-- >>> stream = fmap split $ Stream.fromList [1..100]
80−-- >>> countEven = fmap (("Even " ++) . show) Fold.length
81−-- >>> countOdd = fmap (("Odd " ++) . show) Fold.length
82−-- >>> f = Fold.partition countEven countOdd
83−-- >>> Stream.fold f stream
84−-- ("Even 50","Odd 50")
85−--
86−-- == Sequential Composition
87−--
88−-- Terminating folds can be combined to parse the stream serially such that the
89−-- first fold consumes the input until it terminates and the second fold
90−-- consumes the rest of the input until it terminates:
91−--
92−-- >>> f = Fold.splitWith (,) (Fold.take 8 Fold.toList) (Fold.takeEndBy (== '\n') Fold.toList)
93−-- >>> Stream.fold f $ Stream.fromList "header: hello\n"
94−-- ("header: ","hello\n")
95−--
96−-- == Splitting a Stream
97−--
98−-- A 'Fold' can be applied repeatedly on a stream to transform it to a stream
99−-- of fold results. To split a stream on newlines:
100−--
101−-- >>> f = Fold.takeEndBy (== '\n') Fold.toList
102−-- >>> Stream.fold Fold.toList $ Stream.foldMany f $ Stream.fromList "Hello there!\nHow are you\n"
103−-- ["Hello there!\n","How are you\n"]
104−--
105−-- Similarly, we can split the input of a fold too:
106−--
107−-- >>> Stream.fold (Fold.many f Fold.toList) $ Stream.fromList "Hello there!\nHow are you\n"
108−-- ["Hello there!\n","How are you\n"]
109−--
110−-- == Folds vs. Streams
111−--
112−-- We can often use streams or folds to achieve the same goal. However, streams
113−-- are more efficient in composition of producers (e.g.
114−-- 'Data.Stream.append' or 'Data.Stream.mergeBy') whereas folds are
115−-- more efficient in composition of consumers (e.g. 'splitWith', 'partition'
116−-- or 'teeWith').
117−--
118−-- Streams are producers, transformations on streams happen on the output side:
119−--
120−-- >>> :{
121−-- f stream =
122−-- Stream.filter odd stream
123−-- & fmap (+1)
124−-- & Stream.fold Fold.sum
125−-- :}
126−--
127−-- >>> f $ Stream.fromList [1..100 :: Int]
128−-- 2550
129−--
130−-- Folds are stream consumers with an input stream and an output value, stream
131−-- transformations on folds happen on the input side:
132−--
133−-- >>> :{
134−-- f =
135−-- Fold.filter odd
136−-- $ Fold.lmap (+1)
137−-- $ Fold.sum
138−-- :}
139−--
140−-- >>> Stream.fold f $ Stream.fromList [1..100 :: Int]
141−-- 2550
142−--
143−-- Notice the similiarity in the definition of @f@ in both cases, the only
144−-- difference is the composition by @&@ vs @$@ and the use @lmap@ vs @map@, the
145−-- difference is due to output vs input side transformations.
146−--
147−-- == Fusion Limitations
148−--
149−-- Folds support stream fusion for generating loops comparable to the speed of
150−-- C. However, it has some limitations. For fusion to work, the folds must be
151−-- inlined, folds must be statically known and not generated dynamically, folds
152−-- should not be passed recursively.
153−--
154−-- Another limitation is due to the quadratic complexity causing slowdown when
155−-- too many nested compositions are used. Especially, the performance of the
156−-- Applicative instance and splitting operations (e.g. 'splitWith') degrades
157−-- quadratically (O(n^2)) when combined @n@ times, roughly 8 or less sequenced
158−-- operations are fine. For these cases folds can be converted to parsers and
159−-- then used as ParserK.
160−--
161−-- == Experimental APIs
162−--
16313-- Please refer to "Streamly.Internal.Data.Fold" for more functions that have
16414-- not yet been released.
16515
@@ -171,6 +21,9 @@ module Streamly.Data.Fold
17121 --
17222 -- $setup
17323
24+ -- * Overview
25+ -- $overview
26+
17427 -- * Running A Fold
17528 drive
17629 -- XXX Should we have a stream returning function in fold module?
@@ -405,6 +258,121 @@ import Streamly.Internal.Data.Fold
405258
406259#include "DocTestDataFold.hs"
407260
261+-- $overview
262+--
263+-- A 'Fold' is a consumer of a stream of values. A fold driver (such as
264+-- 'Streamly.Data.Stream.fold') initializes the fold @accumulator@, runs the
265+-- fold @step@ function in a loop, processing the input stream one element at a
266+-- time and accumulating the result. The loop continues until the fold
267+-- terminates, at which point the accumulated result is returned.
268+--
269+-- For example, a 'sum' Fold represents a stream consumer that adds the values
270+-- in the input stream:
271+--
272+-- >>> Stream.fold Fold.sum $ Stream.fromList [1..100]
273+-- 5050
274+--
275+-- Conceptually, a 'Fold' is a data type that mimics a strict left fold
276+-- ('Data.List.foldl'). The above example is similar to a left fold using
277+-- @(+)@ as the step and @0@ as the initial value of the accumulator:
278+--
279+-- >>> Data.List.foldl' (+) 0 [1..100]
280+-- 5050
281+--
282+-- 'Fold's have an early termination capability e.g. the 'one' fold terminates
283+-- after consuming one element:
284+--
285+-- >>> Stream.fold Fold.one $ Stream.fromList [1..]
286+-- Just 1
287+--
288+-- The above example is similar to the following right fold:
289+--
290+-- >>> Prelude.foldr (\x _ -> Just x) Nothing [1..]
291+-- Just 1
292+--
293+-- 'Fold's can be combined together using combinators. For example, to create a
294+-- fold that sums first two elements in a stream:
295+--
296+-- >>> sumTwo = Fold.take 2 Fold.sum
297+-- >>> Stream.fold sumTwo $ Stream.fromList [1..100]
298+-- 3
299+--
300+-- Folds can be combined to run in parallel on the same input. For example, to
301+-- compute the average of numbers in a stream without going through the stream
302+-- twice:
303+--
304+-- >>> avg = Fold.teeWith (/) Fold.sum (fmap fromIntegral Fold.length)
305+-- >>> Stream.fold avg $ Stream.fromList [1.0..100.0]
306+-- 50.5
307+--
308+-- Folds can be combined so as to partition the input stream over multiple
309+-- folds. For example, to count even and odd numbers in a stream:
310+--
311+-- >>> split n = if even n then Left n else Right n
312+-- >>> stream = fmap split $ Stream.fromList [1..100]
313+-- >>> countEven = fmap (("Even " ++) . show) Fold.length
314+-- >>> countOdd = fmap (("Odd " ++) . show) Fold.length
315+-- >>> f = Fold.partition countEven countOdd
316+-- >>> Stream.fold f stream
317+-- ("Even 50","Odd 50")
318+--
319+-- Terminating folds can be combined to parse the stream serially such that the
320+-- first fold consumes the input until it terminates and the second fold
321+-- consumes the rest of the input until it terminates:
322+--
323+-- >>> f = Fold.splitWith (,) (Fold.take 8 Fold.toList) (Fold.takeEndBy (== '\n') Fold.toList)
324+-- >>> Stream.fold f $ Stream.fromList "header: hello\n"
325+-- ("header: ","hello\n")
326+--
327+-- A 'Fold' can be applied repeatedly on a stream to transform it to a stream
328+-- of fold results. To split a stream on newlines:
329+--
330+-- >>> f = Fold.takeEndBy (== '\n') Fold.toList
331+-- >>> Stream.fold Fold.toList $ Stream.foldMany f $ Stream.fromList "Hello there!\nHow are you\n"
332+-- ["Hello there!\n","How are you\n"]
333+--
334+-- Similarly, we can split the input of a fold too:
335+--
336+-- >>> Stream.fold (Fold.many f Fold.toList) $ Stream.fromList "Hello there!\nHow are you\n"
337+-- ["Hello there!\n","How are you\n"]
338+--
339+-- = Folds vs. Streams
340+--
341+-- We can often use streams or folds to achieve the same goal. However, streams
342+-- are more efficient in composition of producers (e.g.
343+-- 'Data.Stream.append' or 'Data.Stream.mergeBy') whereas folds are
344+-- more efficient in composition of consumers (e.g. 'splitWith', 'partition'
345+-- or 'teeWith').
346+--
347+-- Streams are producers, transformations on streams happen on the output side:
348+--
349+-- >>> :{
350+-- f stream =
351+-- Stream.filter odd stream
352+-- & fmap (+1)
353+-- & Stream.fold Fold.sum
354+-- :}
355+--
356+-- >>> f $ Stream.fromList [1..100 :: Int]
357+-- 2550
358+--
359+-- Folds are stream consumers with an input stream and an output value, stream
360+-- transformations on folds happen on the input side:
361+--
362+-- >>> :{
363+-- f =
364+-- Fold.filter odd
365+-- $ Fold.lmap (+1)
366+-- $ Fold.sum
367+-- :}
368+--
369+-- >>> Stream.fold f $ Stream.fromList [1..100 :: Int]
370+-- 2550
371+--
372+-- Notice the similiarity in the definition of @f@ in both cases, the only
373+-- difference is the composition by @&@ vs @$@ and the use @lmap@ vs @map@, the
374+-- difference is due to output vs input side transformations.
375+
408376--------------------------------------------------------------------------------
409377-- Deprecated
410378--------------------------------------------------------------------------------
core/src/Streamly/Data/MutArray.hs+12−18
@@ -32,27 +32,22 @@ module Streamly.Data.MutArray
3232 -- * Construction
3333
3434 -- Uninitialized Arrays
35− , emptyOf
36− , pinnedEmptyOf
35+ , new
36+ , pinnedNew
3737
3838 -- From containers
3939 , fromListN
4040 , fromList
41− , createOf
42− , create
41+ , writeN -- drop new
42+ , write -- full buffer
4343 -- writeLastN
4444
45− -- * Pinning & Unpinning
46− , pin
47− , unpin
48− , isPinned
49−
5045 -- * Appending elements
5146 , snoc
5247
5348 -- * Appending streams
54− , appendN
55− , append
49+ , writeAppendN -- XXX Rename to writeSnocN?
50+ , writeAppend -- XXX Rename to writeSnoc?
5651
5752 -- * Inplace mutation
5853 , putIndex
@@ -83,17 +78,16 @@ module Streamly.Data.MutArray
8378 -- * Size
8479 , length
8580
86− -- * Re-exports
81+ -- * Unbox Type Class
8782 , Unbox (..)
8883
84+ -- * Pinning & Unpinning
85+ , pin
86+ , unpin
87+ , isPinned
88+
8989 -- * Deprecated
9090 , newPinned
91− , new
92− , pinnedNew
93− , writeN
94− , write
95− , writeAppendN
96− , writeAppend
9791 )
9892where
9993
core/src/Streamly/Data/MutArray/Generic.hs+3−8
@@ -23,11 +23,11 @@ module Streamly.Data.MutArray.Generic
2323 MutArray
2424
2525 -- * Construction
26− , emptyOf
26+ , new
2727 , fromListN
2828 , fromList
29− , createOf
30− , create
29+ , writeN
30+ , write
3131
3232 -- * Appending elements
3333 , snoc
@@ -56,11 +56,6 @@ module Streamly.Data.MutArray.Generic
5656
5757 -- * Size
5858 , length
59−
60− -- * Deprecated
61− , new
62− , writeN
63− , write
6459 )
6560where
6661
core/src/Streamly/Data/MutByteArray.hsdeleted−106
@@ -1,106 +0,0 @@
1−-- |
2−-- Module : Streamly.Data.MutByteArray
3−-- Copyright : (c) 2023 Composewell Technologies
4−-- License : BSD3-3-Clause
5−-- Maintainer : streamly@composewell.com
6−-- Portability : GHC
7−--
8−-- A low level byte Array type 'MutByteArray', along with type classes 'Unbox'
9−-- and 'Serialize' for fast binary serialization and deserialization of Haskell
10−-- values. Serialization, deserialization performance is similar to, and in
11−-- some cases many times better than the store package. Conceptually, the
12−-- 'Serialize' type class works in the same way as store.
13−--
14−-- Serialize instances are configurable to use constructor names (see
15−-- 'Streamly.Internal.Data.MutByteArray.encodeConstrNames'), record field names (see
16−-- 'Streamly.Internal.Data.MutByteArray.encodeRecordFields') instead of binary
17−-- encoded values. This is an experimental feature which allows JSON like
18−-- properties with faster speed. For example, you can change the order of
19−-- constructors or record fields without affecting serialized value.
20−--
21−-- Higher level unboxed array modules "Streamly.Data.Array" and
22−-- "Streamly.Data.MutArray" are built on top of this module. Unboxed arrays are
23−-- essentially serialized Haskell values. Array modules provide higher level
24−-- serialization routines like 'Streamly.Internal.Data.Array.pinnedSerialize'
25−-- and 'Streamly.Internal.Data.Array.deserialize' from the
26−-- "Streamly.Internal.Data.Array" module.
27−--
28−-- == Mutable Byte Array
29−--
30−-- 'MutByteArray' is a primitive mutable array in the IO monad. 'Unbox' and
31−-- 'Serialize' type classes use this primitive array to serialize data to and
32−-- deserialize it from. This array is used to build higher level unboxed
33−-- array types 'Streamly.Data.MutArray.MutArray' and 'Streamly.Data.Array.Array'.
34−--
35−-- == Using Unbox
36−--
37−-- The 'Unbox' type class is simple and used to serialize non-recursive fixed
38−-- size data types. This type class is primarily used to implement unboxed
39−-- arrays. Unboxed arrays are just a sequence of serialized fixed length
40−-- Haskell data types. Instances of this type class can be derived using
41−-- 'Generic' or template haskell based deriving functions provided in this
42−-- module.
43−--
44−-- Writing a data type to an array using the array creation routines in
45−-- "Streamly.Data.Array" or "Streamly.Data.MutArray" (e.g. @writeN@ or
46−-- @fromListN@), serializes the type to the array. Similarly, reading the data
47−-- type from the array deserializes it. You can also serialize and deserialize
48−-- directly to and from a 'MutByteArray', using the type class methods.
49−--
50−-- == Using Serialize
51−--
52−-- The 'Serialize' type class is a superset of the 'Unbox' type class, it can
53−-- serialize variable length data types as well e.g. Haskell lists. Use
54−-- 'deriveSerialize' to derive the instances of the type class automatically
55−-- and then use the type class methods to serialize and deserialize to and from
56−-- a 'MutByteArray'.
57−--
58−-- See 'Streamly.Internal.Data.Array.pinnedSerialize' and
59−-- 'Streamly.Internal.Data.Array.deserialize' for 'Array' type based
60−-- serialization.
61−--
62−-- == Comparing serialized values
63−--
64−-- When using the `Unbox` type class the same value may result in differing
65−-- serialized bytes because of unused uninitialized data in case of sum types.
66−-- Therefore, byte comparison of serialized values is not reliable.
67−--
68−-- However, the 'Serialize' type class guarantees that the serialized values
69−-- are always exactly the same and byte comparison of serialized is reliable.
70−--
71−module Streamly.Data.MutByteArray
72− (
73−
74− -- * Mutable Byte Array
75− -- | The standard way to read from or write to a 'MutByteArray' is by using
76− -- the 'Unbox' or 'Serialize' type class methods.
77− MutByteArray
78− , isPinned
79− , pin
80− , unpin
81− , new
82− , pinnedNew
83−
84− -- * Unbox
85− , Unbox(..)
86− , deriveUnbox
87−
88− -- * Serialize
89− , Serialize(..)
90−
91− -- ** Instance Config
92− , SerializeConfig
93− , inlineAddSizeTo
94− , inlineSerializeAt
95− , inlineDeserializeAt
96−
97− -- ** Instance Deriving
98− , deriveSerialize
99− , deriveSerializeWith
100− ) where
101−
102−--------------------------------------------------------------------------------
103−-- Imports
104−--------------------------------------------------------------------------------
105−
106−import Streamly.Internal.Data.MutByteArray
core/src/Streamly/Data/Parser.hs+60−130
@@ -7,137 +7,12 @@
77-- Stability : pre-release
88-- Portability : GHC
99--
10−-- Parsers are stream consumers like folds with the following differences:
10+-- Fast, composable stream consumers with ability to terminate, backtrack and
11+-- fail, supporting stream fusion. Parsers are a natural extension of
12+-- "Streamly.Data.Fold". Parsers and folds can be interconverted.
1113--
12−-- * folds cannot fail but parsers can fail and backtrack.
13−-- * folds can be composed as a Tee but parsers cannot.
14−-- * folds can be used for scanning but parsers cannot.
15−-- * folds can be converted to parsers.
16−--
17−-- This module implements parsers with stream fusion which compile to efficient
18−-- loops comparable to the speed of C.
19−--
20−-- == Using Parsers
21−--
22−-- This module provides elementary parsers and parser combinators that can be
23−-- used to parse a stream of data. Additionally, all the folds from the
24−-- "Streamly.Data.Fold" module can be converted to parsers using 'fromFold'.
25−-- All the parsing functionality provided by popular parsing libraries, and
26−-- more is available. Also see "Streamly.Unicode.Parser" module for Char stream
27−-- parsers.
28−--
29−-- A data stream can be transformed to a stream of parsed data elements. Parser
30−-- combinators can be used to create a pipeline of folds or parsers such that
31−-- the next fold or parser consumes the result of the previous parser. See
32−-- 'Streamly.Data.Stream.parse' and 'Streamly.Data.Stream.parseMany' to run
33−-- these parsers on a stream.
34−--
35−-- == Parser vs ParserK
36−--
37−-- There are two functionally equivalent parsing modules,
38−-- "Streamly.Data.Parser" (this module) and "Streamly.Data.ParserK". The latter
39−-- is a CPS based wrapper over the former, and can be used for parsing in
40−-- general. "Streamly.Data.Parser" enables stream fusion and should be
41−-- preferred over "Streamly.Data.ParserK" for high performance stream parsing
42−-- use cases. However, there are a few cases where this module is not
43−-- suitable and ParserK should be used instead.
44−--
45−-- For static fusion, parser combinators have to use strict pattern matching on
46−-- arguments of type Parser. This leads to infinte loop when a parser is
47−-- defined recursively, due to strict evaluation of the recursive call. For
48−-- example, the following implementation loops infinitely because of the
49−-- recursive use of parser @p@ in the @*>@ combinator:
50−--
51−-- >>> import Streamly.Data.Parser (Parser)
52−-- >>> import qualified Streamly.Data.Fold as Fold
53−-- >>> import qualified Streamly.Data.Parser as Parser
54−-- >>> import qualified Streamly.Data.Stream as Stream
55−-- >>> import Control.Applicative ((<|>))
56−--
57−-- >>> :{
58−-- >>> p :: Monad m => Parser Char m String
59−-- >>> p = Parser.satisfy (== '(') *> p <|> Parser.fromFold Fold.toList
60−-- >>> :}
61−--
62−-- Use ParserK when recursive use is required:
63−--
64−-- >>> import Streamly.Data.ParserK (ParserK)
65−-- >>> import qualified Streamly.Data.StreamK as StreamK
66−-- >>> import qualified Streamly.Internal.Data.StreamK as StreamK (parse)
67−-- >>> import qualified Streamly.Internal.Data.ParserK as ParserK (adapt)
68−--
69−-- >>> :{
70−-- >>> p :: Monad m => ParserK Char m String
71−-- >>> p = ParserK.adapt (Parser.satisfy (== '(')) *> p <|> ParserK.adapt (Parser.fromFold Fold.toList)
72−-- >>> :}
73−--
74−-- >>> StreamK.parse p $ StreamK.fromStream $ Stream.fromList "hello"
75−-- Right "hello"
76−--
77−-- For this reason Applicative, Alternative or Monad compositions with
78−-- recursion cannot be used with the 'Parser' type. Alternative type class based
79−-- operations like 'asum' and Alternative based generic parser combinators use
80−-- recursion. Similarly, Applicative type class based operations like
81−-- 'Prelude.sequence' use recursion. Custom implementations of many such
82−-- operations are provided in this module (e.g. 'some', 'many'), and those
83−-- should be used instead.
84−--
85−-- Another limitation of Parser type is due to the quadratic complexity causing
86−-- slowdown when too many nested compositions are used. Especially Applicative,
87−-- Monad, Alternative instances, and sequenced parsing operations (e.g. nested
88−-- 'one', and 'splitWith') degrade the performance quadratically (O(n^2)) when
89−-- combined @n@ times, roughly 8 or less sequenced parsers are fine. READ THE
90−-- DOCS OF APPLICATIVE, MONAD AND ALTERNATIVE INSTANCES.
91−--
92−-- == Streaming Parsers
93−--
94−-- With 'Streamly.Data.ParserK.ParserK' you can use the generic Alternative
95−-- type class based parsers from the
96−-- <https://hackage.haskell.org/package/parser-combinators parser-combinators>
97−-- library or similar. However, we recommend that you use the equivalent
98−-- functionality from this module for better performance and for streaming
99−-- behavior.
100−--
101−-- Firstly, the combinators in this module are faster due to stream fusion.
102−-- Secondly, these are streaming in nature as the results can be passed
103−-- directly to other stream consumers (folds or parsers). The Alternative type
104−-- class based parsers would end up buffering all the results in lists before
105−-- they can be consumed.
106−--
107−-- When recursion or heavy nesting is needed use ParserK.
108−--
109−-- == Error Reporting
110−--
111−-- These parsers do not report the error context (e.g. line number or column).
112−-- This may be supported in future.
113−--
114−-- == Monad Transformer Stack
115−--
116−-- 'MonadTrans' instance is not provided. If the 'Parser' type is the top most
117−-- layer (which should be the case almost always) you can just use 'fromEffect'
118−-- to execute the lower layer monad effects.
119−--
120−-- == Parser vs ParserK Implementation
121−--
122−-- The 'Parser' type represents a stream consumer by composing state as data
123−-- which enables stream fusion. Stream fusion generates a tight loop without
124−-- any constructor allocations between the stages, providing C like performance
125−-- for the loop. Stream fusion works when multiple functions are combined in a
126−-- pipeline statically. Therefore, the operations in this module must be
127−-- inlined and must not be used recursively to allow for stream fusion.
128−--
129−-- The 'ParserK' type represents a stream consumer by composing function calls,
130−-- therefore, a function call overhead is incurred at each composition. It is
131−-- quite fast in general but may be a few times slower than a fused parser.
132−-- However, it allows for scalable dynamic composition especially parsers can
133−-- be used in recursive calls. Using the 'ParserK' type operations like
134−-- 'splitWith' provide linear (O(n)) performance with respect to the number of
135−-- compositions.
136−--
137−-- == Experimental APIs
138−--
139−-- Please refer to "Streamly.Internal.Data.Parser" for functions that have not
140−-- yet been released.
14+-- Please refer to "Streamly.Internal.Data.Parser" for functions that have
15+-- not yet been released.
14116--
14217module Streamly.Data.Parser
14318 (
@@ -147,6 +22,9 @@ module Streamly.Data.Parser
14722 --
14823 -- $setup
14924
25+ -- * Overview
26+ -- $overview
27+
15028 -- * Parser Type
15129 Parser
15230
@@ -245,3 +123,55 @@ import Streamly.Internal.Data.Parser
245123import Prelude hiding (dropWhile, takeWhile, filter)
246124
247125#include "DocTestDataParser.hs"
126+
127+-- $overview
128+--
129+-- Several combinators in this module can be many times faster than CPS based
130+-- parsers because of stream fusion. For example,
131+-- 'Streamly.Internal.Data.Parser.many' combinator in this module is much
132+-- faster than the 'Control.Applicative.many' combinator of
133+-- 'Control.Applicative.Alternative' type class used by CPS based parsers.
134+--
135+-- The use of 'Alternative' type class, in parsers has another drawback.
136+-- Alternative based parsers use plain Haskell lists to collect the results. In
137+-- a strict Monad like IO, the results are necessarily buffered before they can
138+-- be consumed. This may not perform optimally in streaming applications
139+-- processing large amounts of data. Equivalent combinators in this module can
140+-- consume the results of parsing using a 'Fold' or another parser, thus
141+-- providing a scalable and composable consumer.
142+--
143+-- Note that these parsers do not report the error context (e.g. line number or
144+-- column). This may be supported in future.
145+--
146+-- mtl instances are not provided. If the 'Parser' type is the top most layer
147+-- (which should be the case almost always) you can just use 'fromEffect' to
148+-- execute the lower layer monad effects.
149+--
150+-- == Performance Notes
151+--
152+-- The 'Parser' type represents a stream consumer by composing state as data
153+-- which enables stream fusion. Stream fusion generates a tight loop without
154+-- any constructor allocations between the stages, providing C like performance
155+-- for the loop. Stream fusion works when multiple functions are combined in a
156+-- pipeline statically. Therefore, the operations in this module must be
157+-- inlined and must not be used recursively to allow for stream fusion. Note
158+-- that operations like 'sequence', and 'asum' that compose pasrers using
159+-- recursion should be avoided with these parsers. You can use these with the
160+-- 'ParserK' module instead.
161+--
162+-- Using the 'Parser' type, parsing operations like 'one', 'splitWith' etc.
163+-- degrade quadratically (O(n^2)) when combined many times. If you need to
164+-- combine these operations, say more than 8 times in a single loop, then you
165+-- should consider using the continuation style parser type 'ParserK' instead.
166+-- Also, if you need to use these operations in a recursive loop you should use
167+-- 'ParserK' instead.
168+--
169+-- The 'ParserK' type represents a stream consumer by composing function calls,
170+-- therefore, a function call overhead is incurred at each composition. It is
171+-- quite fast in general but may be a few times slower than a fused parser.
172+-- However, it allows for scalable dynamic composition especially parsers can
173+-- be used in recursive calls. Using the 'ParserK' type operations like
174+-- 'splitWith' provide linear (O(n)) performance with respect to the number of
175+-- compositions..
176+--
177+-- 'Parser' and 'ParserK' types can be interconverted.
core/src/Streamly/Data/ParserK.hs+10−105
@@ -1,4 +1,3 @@
1−{-# LANGUAGE CPP #-}
21-- |
32-- Module : Streamly.Data.ParserK
43-- Copyright : (c) 2023 Composewell Technologies
@@ -7,112 +6,26 @@
76-- Stability : pre-release
87-- Portability : GHC
98--
10−-- See the general notes about parsing in the "Streamly.Data.Parser" module.
11−-- This module implements a using Continuation Passing Style (CPS) wrapper over
12−-- the "Streamly.Data.Parser" module. It is as fast or faster than attoparsec.
9+-- Parsers using Continuation Passing Style (CPS). See notes in
10+-- "Streamly.Data.Parser" module to know when to use this module.
1311--
14−-- == Parser vs ParserK
15−--
16−-- 'ParserK' is preferred over 'Streamly.Data.Parser.Parser' when extensive
17−-- applicative, alternative and monadic composition is required, or when
18−-- recursive or dynamic composition of parsers is required. The
19−-- 'Streamly.Data.Parser.Parser' type fuses statically and creates efficient
20−-- loops whereas 'ParserK' uses function call based composition and has
21−-- comparatively larger runtime overhead but it is better suited to the
22−-- specific use cases mentioned above. 'ParserK' also allows to efficient parse
23−-- a stream of arrays, it can also break the input stream into a parse result
24−-- and remaining stream so that the stream can be parsed independently in
25−-- segments.
26−--
27−-- == Using ParserK
28−--
29−-- All the parsers from the "Streamly.Data.Parser" module can be adapted to
30−-- ParserK using the 'Streamly.Data.ParserK.adaptC',
31−-- 'Streamly.Internal.Data.ParserK.adapt', and
32−-- 'Streamly.Internal.Data.ParserK.adaptCG' combinators.
33−--
34−-- 'Streamly.Data.StreamK.parseChunks' runs a parser on a stream of unboxed
35−-- arrays, this is the preferred and most efficient way to parse chunked input.
36−-- The more general 'Streamly.Data.StreamK.parseBreakChunks' function returns
37−-- the remaining stream as well along with the parse result. There are
38−-- 'Streamly.Internal.Data.StreamK.parseChunksGeneric',
39−-- 'Streamly.Internal.Data.StreamK.parseBreakChunksGeneric' as well to run
40−-- parsers on boxed arrays. 'Streamly.Internal.Data.StreamK.parse',
41−-- 'Streamly.Internal.Data.StreamK.parseBreak' run parsers on a stream of
42−-- individual elements instead of stream of arrays.
43−--
44−-- == Monadic Composition
45−--
46−-- Monad composition can be used for lookbehind parsers, we can dynamically
47−-- compose new parsers based on the results of the previously parsed values.
48−--
49−-- If we have to parse "a9" or "9a" but not "99" or "aa" we can use the
50−-- following non-monadic, backtracking parser:
51−--
52−-- >>> digits p1 p2 = ((:) <$> p1 <*> ((:) <$> p2 <*> pure []))
53−-- >>> :{
54−-- backtracking :: Monad m => ParserK Char m String
55−-- backtracking = ParserK.adapt $
56−-- digits (Parser.satisfy isDigit) (Parser.satisfy isAlpha)
57−-- <|>
58−-- digits (Parser.satisfy isAlpha) (Parser.satisfy isDigit)
59−-- :}
60−--
61−-- We know that if the first parse resulted in a digit at the first place then
62−-- the second parse is going to fail. However, we waste that information and
63−-- parse the first character again in the second parse only to know that it is
64−-- not an alphabetic char. By using lookbehind in a 'Monad' composition we can
65−-- avoid redundant work:
66−--
67−-- >>> data DigitOrAlpha = Digit Char | Alpha Char
68−--
69−-- >>> :{
70−-- lookbehind :: Monad m => ParserK Char m String
71−-- lookbehind = do
72−-- x1 <- ParserK.adapt $
73−-- Digit <$> Parser.satisfy isDigit
74−-- <|> Alpha <$> Parser.satisfy isAlpha
75−-- -- Note: the parse depends on what we parsed already
76−-- x2 <- ParserK.adapt $
77−-- case x1 of
78−-- Digit _ -> Parser.satisfy isAlpha
79−-- Alpha _ -> Parser.satisfy isDigit
80−-- return $ case x1 of
81−-- Digit x -> [x,x2]
82−-- Alpha x -> [x,x2]
83−-- :}
84−--
85−-- == Experimental APIs
86−--
87−-- Please refer to "Streamly.Internal.Data.ParserK" for functions that have
88−-- not yet been released.
12+-- To run a 'ParserK' use 'Streamly.Data.StreamK.parseChunks'.
8913--
9014module Streamly.Data.ParserK
9115 (
92− -- * Setup
93− -- | To execute the code examples provided in this module in ghci, please
94− -- run the following commands first.
95− --
96− -- $setup
97−
9816 -- * Parser Type
9917 ParserK
10018
10119 -- * Parsers
10220 -- ** Conversions
103− , adapt
104− , adaptC
105− , adaptCG
21+ , fromFold
22+ , fromParser
10623 -- , toParser
10724
10825 -- ** Without Input
10926 , fromPure
11027 , fromEffect
11128 , die
112−
113− -- * Deprecated
114− , fromFold
115− , fromParser
11629 )
11730
11831where
@@ -120,20 +33,12 @@ where
12033import Control.Monad.IO.Class (MonadIO)
12134import Streamly.Internal.Data.Fold (Fold)
12235import Streamly.Internal.Data.Unbox (Unbox)
123−import Streamly.Internal.Data.Array (Array)
12436import qualified Streamly.Internal.Data.Parser as ParserD
12537
126−import Streamly.Internal.Data.ParserK.Type
127−
128−#include "DocTestDataParserK.hs"
38+import Streamly.Internal.Data.Parser.ParserK.Type
12939
130−{-# DEPRECATED fromFold "Please use \"ParserK.adaptC . Parser.fromFold\" instead." #-}
40+-- | Convert a 'Fold' to a 'ParserK'.
41+--
13142{-# INLINE fromFold #-}
132−fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK (Array a) m b
133−fromFold = adaptC . ParserD.fromFold
134−
135−{-# DEPRECATED fromParser "Please use \"adaptC\" instead." #-}
136−{-# INLINE fromParser #-}
137−fromParser ::
138− (MonadIO m, Unbox a) => ParserD.Parser a m b -> ParserK (Array a) m b
139−fromParser = adaptC
43+fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK a m b
44+fromFold = fromParser . ParserD.fromFold
core/src/Streamly/Data/Stream.hs+107−199
@@ -8,20 +8,15 @@
88-- Stability : released
99-- Portability : GHC
1010--
11−-- Streams represented as state machines, that fuse together when composed
12−-- statically, eliminating function calls or intermediate constructor
13−-- allocations - generating tight, efficient loops. Suitable for high
14−-- performance looping operations.
15−--
16−-- If you need to call these operations recursively in a loop (i.e. composed
17−-- dynamically) then it is recommended to use the continuation passing style
18−-- (CPS) stream operations from the "Streamly.Data.StreamK" module. 'Stream'
19−-- and 'StreamK' types are interconvertible. See more details in the
20−-- documentation below regarding 'Stream' vs 'StreamK'.
11+-- Fast, composable stream producers with ability to terminate, supporting
12+-- stream fusion.
2113--
2214-- Please refer to "Streamly.Internal.Data.Stream" for more functions that have
2315-- not yet been released.
2416--
17+-- For continuation passing style (CPS) stream type, please refer to
18+-- the "Streamly.Data.StreamK" module.
19+--
2520-- Checkout the <https://github.com/composewell/streamly-examples>
2621-- repository for many more real world examples of stream programming.
2722
@@ -42,6 +37,9 @@ module Streamly.Data.Stream
4237 -- * Construction
4338 -- | Functions ending in the general shape @b -> Stream m a@.
4439 --
40+ -- See also: "Streamly.Internal.Data.Stream.StreamD.Generate" for
41+ -- @Pre-release@ functions.
42+ --
4543 -- Useful Idioms:
4644 --
4745 -- >>> fromIndices f = fmap f $ Stream.enumerateFrom 0
@@ -51,12 +49,11 @@ module Streamly.Data.Stream
5149 -- >>> fromFoldableM = Stream.sequence . fromFoldable
5250
5351 -- ** Primitives
54− -- | A fused 'Stream' is never constructed using these primitives, they are
55− -- typically generated by converting containers like list into streams, or
56− -- generated using custom functions provided in this module. The 'cons'
57− -- primitive in this module has a rare use in fusing a small number of
58− -- elements. On the other hand, it is common to construct 'StreamK' stream
59− -- using the StreamK.'StreamK.cons' primitive.
52+ -- | A fused 'Stream' is never constructed using 'cons', we always convert
53+ -- other type of containers like list into streams, or generate it using
54+ -- custom functions provided in this module. The 'cons' primitive has a
55+ -- rare use in fusing a small number of elements. On the other hand, it is
56+ -- common to construct 'StreamK' stream using the 'StreamK.cons' primitive.
6057 , nil
6158 , nilM
6259 , cons
@@ -70,15 +67,6 @@ module Streamly.Data.Stream
7067
7168 -- ** From Values
7269 -- | Generate a monadic stream from a seed value or values.
73− --
74− -- Non-empty pure streams can be cycled using repeat and unfold:
75− --
76− -- >>> cycle = Stream.unfoldMany Unfold.fromList . Stream.repeat
77− --
78− -- Non-empty StreamK can be cycled using append:
79− --
80− -- >>> cycle xs = let ys = xs `StreamK.append` ys in ys
81− --
8270 , fromPure
8371 , fromEffect
8472 , repeat
@@ -123,6 +111,8 @@ module Streamly.Data.Stream
123111 -- | Functions ending in the general shape @Stream m a -> m b@ or @Stream m
124112 -- a -> m (b, Stream m a)@
125113 --
114+ -- See also: "Streamly.Internal.Data.Stream.StreamD.Eliminate" for @Pre-release@
115+ -- functions.
126116
127117-- EXPLANATION: In imperative terms a fold can be considered as a loop over the stream
128118-- that reduces the stream to a single value.
@@ -221,57 +211,21 @@ module Streamly.Data.Stream
221211 -- operations like mapping a function over the stream.
222212 , foldrM
223213 , foldr
224− -- foldr1
225214
226215 -- ** Specific Folds
227− -- | Streams are folded using folds in "Streamly.Data.Fold". Here are some
228− -- idioms and equivalents of Data.List APIs using folds:
216+ -- | Usually you can use the folds in "Streamly.Data.Fold". However, some
217+ -- folds that may be commonly used or may have an edge in performance in
218+ -- some cases are provided here.
219+ --
220+ -- Useful idioms:
229221 --
230222 -- >>> foldlM' f a = Stream.fold (Fold.foldlM' f a)
231− -- >>> foldl1' f = Stream.fold (Fold.foldl1' f)
223+ -- >>> foldl1 f = Stream.fold (Fold.foldl1' f)
232224 -- >>> foldl' f a = Stream.fold (Fold.foldl' f a)
233225 -- >>> drain = Stream.fold Fold.drain
234226 -- >>> mapM_ f = Stream.fold (Fold.drainMapM f)
235227 -- >>> length = Stream.fold Fold.length
236− -- >>> genericLength = Stream.fold Fold.lengthGeneric
237228 -- >>> head = Stream.fold Fold.one
238− -- >>> last = Stream.fold Fold.latest
239− -- >>> null = Stream.fold Fold.null
240− -- >>> and = Stream.fold Fold.and
241− -- >>> or = Stream.fold Fold.or
242− -- >>> any p = Stream.fold (Fold.any p)
243− -- >>> all p = Stream.fold (Fold.all p)
244− -- >>> sum = Stream.fold Fold.sum
245− -- >>> product = Stream.fold Fold.product
246− -- >>> maximum = Stream.fold Fold.maximum
247− -- >>> maximumBy cmp = Stream.fold (Fold.maximumBy cmp)
248− -- >>> minimum = Stream.fold Fold.minimum
249− -- >>> minimumBy cmp = Stream.fold (Fold.minimumBy cmp)
250− -- >>> elem x = Stream.fold (Fold.elem x)
251− -- >>> notElem x = Stream.fold (Fold.notElem x)
252− -- >>> lookup x = Stream.fold (Fold.lookup x)
253− -- >>> find p = Stream.fold (Fold.find p)
254− -- >>> (!?) i = Stream.fold (Fold.index i)
255− -- >>> genericIndex i = Stream.fold (Fold.indexGeneric i)
256− -- >>> elemIndex x = Stream.fold (Fold.elemIndex x)
257− -- >>> findIndex p = Stream.fold (Fold.findIndex p)
258− --
259− -- Some equivalents of Data.List APIs from the Stream module:
260− --
261− -- >>> head = fmap (fmap fst) . Stream.uncons
262− -- >>> tail = fmap (fmap snd) . Stream.uncons
263− -- >>> tail = Stream.tail -- unreleased API
264− -- >>> init = Stream.init -- unreleased API
265− --
266− -- A Stream based toList fold implementation is provided below because it
267− -- has a better performance compared to the fold.
268−
269− -- Functions in Data.List, missing here:
270− -- unsnoc = Stream.parseBreak (Parser.init Fold.toList)
271− -- genericTake
272− -- genericDrop
273− -- genericSplitAt
274− -- genericReplicate
275229 , toList
276230
277231 -- * Mapping
@@ -299,6 +253,8 @@ module Streamly.Data.Stream
299253 -- * Scanning
300254 -- | Stateful one-to-one transformations.
301255 --
256+ -- See also: "Streamly.Internal.Data.Stream.StreamD.Transform" for
257+ -- @Pre-release@ functions.
302258
303259 {-
304260 -- ** Left scans
@@ -325,15 +281,6 @@ module Streamly.Data.Stream
325281 , scan
326282 , postscan
327283 -- XXX postscan1 can be implemented using Monoids or Refolds.
328− -- The following scans from Data.List are not provided.
329− -- XXX scanl
330− -- XXX scanl1
331− -- XXX scanr
332− -- XXX scanr1
333− -- XXX mapAccumL
334− -- XXX mapAccumR
335− -- XXX inits
336− -- XXX tails
337284
338285 -- ** Specific scans
339286 -- Indexing can be considered as a special type of zipping where we zip a
@@ -342,8 +289,6 @@ module Streamly.Data.Stream
342289
343290 -- * Insertion
344291 -- | Add elements to the stream.
345− --
346− -- >>> insert = Stream.insertBy compare
347292
348293 -- Inserting elements is a special case of interleaving/merging streams.
349294 , insertBy
@@ -379,15 +324,12 @@ module Streamly.Data.Stream
379324 -- "Streamly.Internal.Data.Fold" can be used along with 'scanMaybe' to
380325 -- perform stateful filtering operations in general.
381326 --
382− -- Idioms and equivalents of Data.List APIs:
327+ -- Useful idioms:
383328 --
384329 -- >>> deleteBy cmp x = Stream.scanMaybe (Fold.deleteBy cmp x)
385− -- >>> deleteBy = Stream.deleteBy -- unreleased API
386− -- >>> delete = deleteBy (==)
387330 -- >>> findIndices p = Stream.scanMaybe (Fold.findIndices p)
388331 -- >>> elemIndices a = findIndices (== a)
389332 -- >>> uniq = Stream.scanMaybe (Fold.uniqBy (==))
390− -- >>> partition p = Stream.fold (Fold.partition Fold.toList Fold.toList) . fmap (if p then Left else Right)
391333 , scanMaybe
392334 , take
393335 , takeWhile
@@ -395,8 +337,6 @@ module Streamly.Data.Stream
395337 , drop
396338 , dropWhile
397339 , dropWhileM
398− -- XXX write to an array in reverse and then read in reverse
399− -- > dropWhileEnd = reverse . dropWhile p . reverse
400340
401341 -- XXX These are available as scans in folds. We need to check the
402342 -- performance though. If these are common and we need convenient stream
@@ -418,42 +358,34 @@ module Streamly.Data.Stream
418358 -- , elemIndices
419359
420360 -- * Combining Two Streams
421− -- | Note that these operations are suitable for statically fusing a few
422− -- streams, they have a quadratic O(n^2) time complexity wrt to the number
423− -- of streams. If you want to compose many streams dynamically using binary
424− -- combining operations see the corresponding operations in
425− -- "Streamly.Data.StreamK".
426− --
427− -- When fusing more than two streams it is more efficient if the binary
428− -- operations are composed as a balanced tree rather than a right
429− -- associative or left associative one e.g.:
430− --
431− -- >>> s1 = Stream.fromList [1,2] `Stream.append` Stream.fromList [3,4]
432− -- >>> s2 = Stream.fromList [4,5] `Stream.append` Stream.fromList [6,7]
433− -- >>> s = s1 `Stream.append` s2
434−
435361 -- ** Appending
436− -- | Equivalent of Data.List append:
437− --
438− -- >>> (++) = Stream.append
439362 , append
440363
441364 -- ** Interleaving
365+ -- | When interleaving more than two streams you may want to interleave
366+ -- them pairwise creating a balanced binary merge tree.
442367 , interleave
443368
444369 -- ** Merging
370+ -- | When merging more than two streams you may want to merging them
371+ -- pairwise creating a balanced binary merge tree.
372+ --
373+ -- Merging of @n@ streams can be performed by combining the streams pair
374+ -- wise using 'mergeMapWith' to give O(n * log n) time complexity. If used
375+ -- with 'concatMapWith' it will have O(n^2) performance.
376+
445377 , mergeBy
446378 , mergeByM
447379
448380 -- ** Zipping
449− -- | Idioms and equivalents of Data.List APIs:
381+ -- | When zipping more than two streams you may want to zip them
382+ -- pairwise creating a balanced binary tree.
450383 --
451− -- >>> zip = Stream.zipWith (,)
452− -- >>> unzip = Stream.fold (Fold.unzip Fold.toList Fold.toList)
384+ -- Zipping of @n@ streams can be performed by combining the streams pair
385+ -- wise using 'mergeMapWith' with O(n * log n) time complexity. If used
386+ -- with 'concatMapWith' it will have O(n^2) performance.
453387 , zipWith
454388 , zipWithM
455− -- XXX zipWith3,4,5,6,7
456− -- XXX unzip3,4,5,6,7
457389 -- , ZipStream (..)
458390
459391 -- ** Cross Product
@@ -486,89 +418,48 @@ module Streamly.Data.Stream
486418 -- One dimension loops are just a special case of nested loops. For
487419 -- example, 'concatMap' can degenerate to a simple map operation:
488420 --
489− -- >>> map f m = Stream.concatMap (\x -> Stream.fromPure (f x)) m
421+ -- > map f m = S.concatMap (\x -> S.fromPure (f x)) m
490422 --
491423 -- Similarly, 'concatMap' can perform filtering by mapping an element to a
492424 -- 'nil' stream:
493425 --
494− -- >>> filter p m = Stream.concatMap (\x -> if p x then Stream.fromPure x else Stream.nil) m
426+ -- > filter p m = S.concatMap (\x -> if p x then S.fromPure x else S.nil) m
495427 --
496− -- Idioms and equivalents of Data.List APIs:
497− --
498− -- >>> concat = Stream.concatMap id
499− -- >>> unlines = Stream.interposeSuffix '\n'
500− -- >>> unlines = Stream.intercalateSuffix Unfold.fromList "\n"
501− -- >>> unwords = Stream.interpose ' '
502− -- >>> unwords = Stream.intercalate Unfold.fromList " "
503428
504429 , concatEffect
505430 , concatMap
506431 , concatMapM
507432
508433 -- * Repeated Fold
509− -- | Idioms and equivalents of Data.List APIs:
434+ -- | Useful idioms:
510435 --
511− -- >>> groupsOf n = Stream.foldMany (Fold.take n Fold.toList)
512− -- >>> groupBy eq = Stream.groupsWhile eq Fold.toList
513− -- >>> groupBy eq = Stream.parseMany (Parser.groupBy eq Fold.toList)
514− -- >>> groupsByRolling eq = Stream.parseMany (Parser.groupByRolling eq Fold.toList)
515− -- >>> groups = groupBy (==)
436+ -- >>> splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
437+ -- >>> splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
438+ -- >>> groupsBy eq f = Stream.parseMany (Parser.groupBy eq f)
439+ -- >>> groupsByRolling eq f = Stream.parseMany (Parser.groupByRolling eq f)
440+ -- >>> groupsOf n f = Stream.foldMany (Fold.take n f)
516441 , foldMany -- XXX Rename to foldRepeat
517442 , groupsOf
518443 , parseMany
519444
520445 -- * Splitting
521− -- | Idioms and equivalents of Data.List APIs:
522− --
523− -- >>> splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
524− -- >>> splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
525− -- >>> lines = splitOnSuffix (== '\n')
526− -- >>> words = Stream.wordsBy isSpace
527− -- >>> splitAt n = Stream.fold (Fold.splitAt n Fold.toList Fold.toList)
528− -- >>> span p = Parser.splitWith (,) (Parser.takeWhile p Fold.toList) (Parser.fromFold Fold.toList)
529− -- >>> break p = span (not . p)
530446 , splitOn
531447 , wordsBy
532448
533449 -- * Buffered Operations
534450 -- | Operations that require buffering of the stream.
535451 -- Reverse is essentially a left fold followed by an unfold.
536− --
537− -- Idioms and equivalents of Data.List APIs:
538− --
539− -- >>> nub = Stream.fold Fold.toList . Stream.scanMaybe Fold.nub
540− -- >>> nub = Stream.ordNub -- unreleased API
541− -- >>> sortBy = StreamK.sortBy
542− -- >>> sortOn f = StreamK.sortOn -- unreleased API
543− -- >>> deleteFirstsBy = Stream.deleteFirstsBy -- unreleased
544− -- >>> (\\) = Stream.deleteFirstsBy (==) -- unreleased
545− -- >>> intersectBy = Stream.intersectBy -- unreleased
546− -- >>> intersect = Stream.intersectBy (==) -- unreleased
547− -- >>> unionBy = Stream.unionBy -- unreleased
548− -- >>> union = Stream.unionBy (==) -- unreleased
549− --
550452 , reverse
551− -- XXX transpose: write the streams to arrays and then stream transposed.
552− -- XXX subsequences
553− -- XXX permutations
554− -- , nub
555− -- , ordNub
556− -- , nubBy
557453
558454 -- * Multi-Stream folds
559455 -- | Operations that consume multiple streams at the same time.
560456 , eqBy
561457 , cmpBy
562458 , isPrefixOf
563− , isInfixOf
564− -- , isSuffixOf
565− -- , isSuffixOfUnbox
566459 , isSubsequenceOf
567460
568461 -- trimming sequences
569− -- , stripPrefix
570− -- , stripSuffix
571− -- , stripSuffixUnbox
462+ , stripPrefix
572463
573464 -- Exceptions and resource management depend on the "exceptions" package
574465 -- XXX We can have IO Stream operations not depending on "exceptions"
@@ -589,6 +480,9 @@ module Streamly.Data.Stream
589480 -- For example, instead of calling them on a stream of chars call them on a
590481 -- stream of arrays before flattening it to a stream of chars.
591482 --
483+ -- See also: "Streamly.Internal.Data.Stream.StreamD.Exception" for
484+ -- @Pre-release@ functions.
485+
592486 , onException
593487 , handle
594488
@@ -622,20 +516,25 @@ module Streamly.Data.Stream
622516 , runReaderT
623517 , runStateT
624518
625− -- XXX Arrays could be different types, therefore, this should be in
626− -- specific array module. Or maybe we should abstract over array types.
627519 -- * Stream of Arrays
628520 , Array.chunksOf
521+ , chunksSplitOn
629522 )
630523where
631524
525+import Control.Monad.IO.Class (MonadIO)
526+import Data.Word (Word8)
527+import Streamly.Data.Array (Array)
632528import Streamly.Internal.Data.Stream
633529import Prelude
634530 hiding (filter, drop, dropWhile, take, takeWhile, zipWith, foldr,
635− mapM, sequence, reverse, iterate, foldr1, repeat, replicate,
636− concatMap)
531+ foldl, map, mapM, mapM_, sequence, all, any, sum, product, elem,
532+ notElem, maximum, minimum, head, last, tail, length, null,
533+ reverse, iterate, init, and, or, lookup, foldr1, (!!),
534+ scanl, scanl1, repeat, replicate, concatMap, span)
637535
638536import qualified Streamly.Internal.Data.Array.Type as Array
537+import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
639538
640539#include "DocTestDataStream.hs"
641540
@@ -674,11 +573,11 @@ import qualified Streamly.Internal.Data.Array.Type as Array
674573--
675574-- == Stream Fusion
676575--
677−-- The fused 'Stream' type in this module employs stream fusion for C-like
678−-- performance when looping over data. It represents the stream as a state
679−-- machine using an explicit state, and a step function working on the state. A
680−-- typical stream operation consumes elements from the previous state machine
681−-- in a stream pipeline, transforms the elements and yields new values for the
576+-- The fused 'Stream' type employs stream fusion for C-like performance when
577+-- looping over data. It represents a stream source or transformation by
578+-- defining a state machine with explicit state, and a step function working on
579+-- the state. A typical stream operation consumes elements from the previous
580+-- state machine in the pipeline, transforms them and yields new values for the
682581-- next stage to consume. The stream operations are modular and represent a
683582-- single task, they have no knowledge of previous or next operation on the
684583-- elements.
@@ -687,7 +586,7 @@ import qualified Streamly.Internal.Data.Array.Type as Array
687586-- transformation operations and a stream consumer. All these operations taken
688587-- together form a closed loop processing the stream elements. Elements are
689588-- transferred between stages using a boxed data constructor. However, all the
690−-- stages of the pipeline are fused together by GHC, eliminating the boxing of
589+-- stages of the pipeline are fused together by GHC, eliminating the
691590-- intermediate constructors, and thus forming a tight C like loop without any
692591-- boxed data being used in the loop.
693592--
@@ -702,45 +601,54 @@ import qualified Streamly.Internal.Data.Array.Type as Array
702601--
703602-- == Stream vs StreamK
704603--
705−-- The fused stream model avoids constructor allocations and function call
706−-- overheads. However, the stream is represented as a state machine, and to
707−-- generate stream elements it has to navigate the decision tree of the state
708−-- machine. Moreover, the state machine is cranked for each element in the
709−-- stream. This performs extremely well when the number of states are limited.
710−-- The state machine starts getting expensive as the number of states increase.
711−-- For example, generating a stream from a list requires a single state and is
712−-- very efficient, even if it has millions of elements. However, using 'cons'
713−-- to construct a million element stream would be a disaster.
604+-- The fused stream model avoids constructor allocations or function call
605+-- overheads. However, the stream is represented as a state machine and to
606+-- generate elements it has to navigate the decision tree of the state machine.
607+-- Moreover, the state machine is cranked for each element in the stream. This
608+-- performs extremely well when the number of states are limited. The state
609+-- machine starts getting expensive as the number of states increase. For
610+-- example, generating a million element stream from a list requires a single
611+-- state and is very efficient. However, using fused 'cons' to generate a
612+-- million element stream would be a disaster.
714613--
715614-- A typical worst case scenario for fused stream model is a large number of
716615-- `cons` or `append` operations. A few static `cons` or `append` operations
717−-- are very fast and much faster than a CPS style stream because CPS involves a
718−-- function call for each element whereas fused stream involves a few
719−-- conditional branches in the state machine. However, constructing a large
720−-- stream using `cons` introduces as many states in the state machine as the
721−-- number of elements. If we compose `cons` as a balanced binary tree it will
722−-- take @n * log n@ time to navigate the tree, and @n * n@ if it is a right
723−-- associative composition.
724−--
616+-- are very fast and much faster than a CPS style stream. However, if we
617+-- construct a large stream using `cons` it introduces as many states in the
618+-- state machine as the number of elements. If we compose the `cons` as a
619+-- binary tree it will take @n * log n@ time to navigate the tree, and @n * n@
620+-- if it is a right associative composition.
621+--
622+-- For quadratic cases of fused stream, after a certain threshold the CPS
623+-- stream would perform much better and exhibit linear performance behavior.
725624-- Operations like 'cons' or 'append'; are typically recursively called to
726625-- construct a lazy infinite stream. For such use cases the CPS style 'StreamK'
727−-- should be used. CPS streams do not have a state machine that needs to be
626+-- type is provided. CPS streams do not have a state machine that needs to be
728627-- cranked for each element, past state has no effect on the future element
729−-- processing. However, CPS incurs a function call overhead for each element
730−-- processed, the overhead could be large compared to a fused state machine
731−-- even if it has many states. However, because of its linear performance
732−-- characterstics, after a certain threshold of stream compositions the CPS
733−-- stream would perform much better than the quadratic fused stream operations.
734−--
735−-- As a general guideline, you need to use 'StreamK' when you have to use
736−-- 'cons', 'append' or other operations having quadratic complexity at a large
737−-- scale. Typically, in such cases you need to compose the stream recursively,
738−-- by calling an operation in a loop. The decision to compose the stream is
739−-- taken at run time rather than statically at compile time.
740−--
741−-- Typically you would compose a 'StreamK' of chunks of data so that the
742−-- StreamK overhead is not high, and then process the chunks using 'Stream' by
743−-- using statically fused stream pipeline operations on the chunks.
628+-- processing. However, it incurs a function call overhead for each operation
629+-- for each element, which could be very large overhead compared to fused state
630+-- machines even if it has many states and cranks it for each element. But in
631+-- some cases scales tip in favor of the CPS stream. In those cases even though
632+-- CPS has a large constant overhead, it has a linear performance rather than
633+-- quadratic.
634+--
635+-- As a general guideline, if you have to use 'cons' or 'append' or operations
636+-- of similar nature, at a large scale, then 'StreamK' should be used. When you
637+-- need to compose the stream dynamically or recursively, then 'StreamK' should
638+-- be used. Typically you would use a dynamically generated 'StreamK' with
639+-- chunks of data which can then be processed by statically fused stream
640+-- pipeline operations.
744641--
745642-- 'Stream' and 'StreamK' types can be interconverted. See
746643-- "Streamly.Data.StreamK" module for conversion operations.
644+
645+-- | Split a stream of arrays on a given separator byte, dropping the separator
646+-- and coalescing all the arrays between two separators into a single array.
647+--
648+{-# INLINE chunksSplitOn #-}
649+chunksSplitOn
650+ :: (MonadIO m)
651+ => Word8
652+ -> Stream m (Array Word8)
653+ -> Stream m (Array Word8)
654+chunksSplitOn = ArrayStream.splitOn
core/src/Streamly/Data/StreamK.hs+19−37
@@ -8,30 +8,18 @@
88-- Stability : released
99-- Portability : GHC
1010--
11−-- Streams represented as chains of functions calls using Continuation Passing
12−-- Style (CPS), suitable for dynamically composing potentially large number of
13−-- streams.
14−--
15−-- Unlike the statically fused operations in "Streamly.Data.Stream", StreamK
16−-- operations are less efficient, involving a function call overhead for each
17−-- element, but they exhibit linear O(n) time complexity wrt to the number of
18−-- stream compositions. Therefore, they are suitable for dynamically composing
19−-- streams e.g. appending potentially infinite streams in recursive loops.
20−-- While fused streams can be used to efficiently process elements as small as
21−-- a single byte, CPS streams are typically used on bigger chunks of data to
22−-- avoid the larger overhead per element. For more details See the @Stream vs
23−-- StreamK@ section in the "Streamly.Data.Stream" module.
24−--
25−-- In addition to the combinators in this module, you can use operations from
26−-- "Streamly.Data.Stream" for StreamK as well by converting StreamK to Stream
27−-- ('toStream'), and vice-versa ('fromStream'). Please refer to
28−-- "Streamly.Internal.Data.StreamK" for more functions that have not yet been
29−-- released.
30−--
31−-- For documentation see the corresponding combinators in
32−-- "Streamly.Data.Stream". Documentation has been omitted in this module unless
33−-- there is a difference worth mentioning or if the combinator does not exist
34−-- in "Streamly.Data.Stream".
11+-- Streams using Continuation Passing Style (CPS). See the @Stream vs StreamK@
12+-- section in the "Streamly.Data.Stream" module to know when to use this
13+-- module.
14+--
15+-- Please refer to "Streamly.Internal.Data.StreamK" for more functions
16+-- that have not yet been released.
17+--
18+-- Some useful idioms:
19+--
20+-- >>> concatFoldableWith f = Prelude.foldr f StreamK.nil
21+-- >>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
22+-- >>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
3523
3624-- Notes:
3725--
@@ -95,8 +83,7 @@ module Streamly.Data.StreamK
9583 -- , foldBreak
9684
9785 -- ** Parsing
98− , parse
99− , parseBreak
86+ -- , parseBreak
10087 , parseBreakChunks
10188 , parseChunks
10289
@@ -106,11 +93,6 @@ module Streamly.Data.StreamK
10693 , take
10794
10895 -- * Combining Two Streams
109− -- | Unlike the operations in "Streamly.Data.Stream", these operations can
110− -- be used to dynamically compose large number of streams e.g. using the
111− -- 'concatMapWith' and 'mergeMapWith' operations. They have a linear O(n)
112− -- time complexity wrt to the number of streams being composed.
113−
11496 -- ** Appending
11597 , append
11698
@@ -134,12 +116,6 @@ module Streamly.Data.StreamK
134116 -- , CrossStreamK (..)
135117
136118 -- * Stream of streams
137− -- | Some useful idioms:
138− --
139− -- >>> concatFoldableWith f = Prelude.foldr f StreamK.nil
140− -- >>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
141− -- >>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
142− --
143119 , concatEffect
144120 -- , concatMap
145121 , concatMapWith
@@ -192,3 +168,9 @@ import Prelude hiding (reverse, zipWith, mapM, dropWhile, take)
192168-- 1
193169-- 2
194170-- [1,2]
171+--
172+-- == Exception Handling
173+--
174+-- There are no native exception handling operations in the StreamK module,
175+-- please convert to 'Stream' type and use exception handling operations from
176+-- "Streamly.Data.Stream".
core/src/Streamly/Data/Unbox.hsnew file+16
@@ -0,0 +1,16 @@
1+-- |
2+-- Module : Streamly.Data.Unbox
3+-- Copyright : (c) 2023 Composewell Technologies
4+-- License : BSD3-3-Clause
5+-- Maintainer : streamly@composewell.com
6+-- Portability : GHC
7+--
8+module Streamly.Data.Unbox
9+ ( Unbox(..)
10+ , MutableByteArray
11+ , deriveUnbox
12+ , deriveUnboxWith
13+ ) where
14+
15+import Streamly.Internal.Data.Unbox
16+import Streamly.Internal.Data.Unbox.TH
core/src/Streamly/Data/Unfold.hs+1−3
@@ -39,7 +39,7 @@ module Streamly.Data.Unfold
3939
4040 -- * Unfolds
4141 -- One to one correspondence with
42− -- "Streamly.Internal.Data.Stream.Generate"
42+ -- "Streamly.Internal.Data.Stream.StreamD.Generate"
4343
4444 -- ** Basic Constructors
4545 , unfoldrM
@@ -65,8 +65,6 @@ module Streamly.Data.Unfold
6565 -- ** Mapping on Input
6666 , lmap
6767 , lmapM
68− , first
69− , second
7068
7169 -- ** Mapping on Output
7270 , mapM
core/src/Streamly/FileSystem/Handle.hs+30−7
@@ -1,4 +1,5 @@
11{-# LANGUAGE CPP #-}
2+#include "inline.hs"
23
34-- |
45-- Module : Streamly.FileSystem.Handle
@@ -81,13 +82,13 @@ module Streamly.FileSystem.Handle
8182 -- position of the file handle. The stream ends as soon as EOF is
8283 -- encountered.
8384
84− -- *** Streams
85− , read
86− , readWith
87− , readChunks
88− , readChunksWith
85+ -- -- *** Streams
86+ -- , read
87+ -- , readWith
88+ -- , readChunks
89+ -- , readChunksWith
8990
90− -- *** Unfolds
91+ -- -- *** Unfolds
9192 , reader
9293 , readerWith
9394 , chunkReader
@@ -104,14 +105,36 @@ module Streamly.FileSystem.Handle
104105 , writeChunks
105106
106107 -- * Deprecated
108+ , read
107109 , readWithBufferOf
110+ , readChunks
108111 , readChunksWithBufferOf
109112 , writeChunksWithBufferOf
110113 , writeWithBufferOf
111114 )
112115where
113116
114−import Streamly.Internal.FileSystem.Handle
117+import Control.Monad.IO.Class (MonadIO(..))
118+import Data.Word (Word8)
119+import Streamly.Internal.Data.Array.Type (Array)
120+import Streamly.Internal.Data.Unfold.Type (Unfold)
121+import System.IO (Handle)
122+
123+import Streamly.Internal.FileSystem.Handle hiding (read, readChunks)
115124import Prelude hiding (read)
116125
117126#include "DocTestFileSystemHandle.hs"
127+
128+-- | Same as 'reader'
129+--
130+{-# DEPRECATED read "Please use 'reader' instead" #-}
131+{-# INLINE read #-}
132+read :: MonadIO m => Unfold m Handle Word8
133+read = reader
134+
135+-- | Same as 'chunkReader'
136+--
137+{-# DEPRECATED readChunks "Please use 'chunkReader' instead" #-}
138+{-# INLINE readChunks #-}
139+readChunks :: MonadIO m => Unfold m Handle (Array Word8)
140+readChunks = chunkReader
core/src/Streamly/Internal/Data/Array.hs+163−610
@@ -2,6 +2,7 @@
22-- |
33-- Module : Streamly.Internal.Data.Array
44-- Copyright : (c) 2019 Composewell Technologies
5+--
56-- License : BSD3
67-- Maintainer : streamly@composewell.com
78-- Stability : experimental
@@ -16,33 +17,59 @@ module Streamly.Internal.Data.Array
1617 -- $design
1718
1819 -- * The Array Type
19− module Streamly.Internal.Data.Array.Type
20+ Array
2021
2122 -- * Construction
23+
24+ -- Pure List APIs
25+ , A.fromListN
26+ , A.fromList
27+
28+ -- Stream Folds
29+ , fromStreamN
30+ , fromStream
31+
2232 -- Monadic Folds
33+ , A.writeN -- drop new
34+ , A.pinnedWriteNAligned
35+ , A.write -- full buffer
2336 , writeLastN
2437
38+ -- * Elimination
39+ -- ** Conversion
40+ , A.toList
41+
42+ -- ** Streams
43+ , A.read
44+ , A.readRev
45+
46+ -- ** Unfolds
47+ , reader
48+ , readerUnsafe
49+ , A.readerRev
50+ , producer -- experimental
51+
2552 -- * Random Access
2653 -- , (!!)
2754 , getIndex
55+ , A.unsafeIndex -- XXX Rename to getIndexUnsafe??
2856 , getIndexRev
29− , last -- XXX getLastIndex?
57+ , last -- XXX getIndexLast?
58+ , getIndices
59+ , getIndicesFromThenTo
3060 -- , getIndicesFrom -- read from a given position to the end of file
3161 -- , getIndicesUpto -- read from beginning up to the given position
3262 -- , getIndicesFromTo
3363 -- , getIndicesFromRev -- read from a given position to the beginning of file
3464 -- , getIndicesUptoRev -- read from end to the given position in file
35− , indexReader
36− , indexReaderFromThenTo
3765
3866 -- * Size
67+ , length
3968 , null
4069
4170 -- * Search
4271 , binarySearch
4372 , findIndicesOf
44− -- getIndicesOf
45− , indexFinder -- see splitOn
4673 -- , findIndexOf
4774 -- , find
4875
@@ -50,96 +77,66 @@ module Streamly.Internal.Data.Array
5077 , cast
5178 , asBytes
5279 , castUnsafe
80+ , asPtrUnsafe
5381 , asCStringUnsafe
82+ , A.unsafeFreeze -- asImmutableUnsafe?
83+ , A.unsafeThaw -- asMutableUnsafe?
5484
5585 -- * Subarrays
5686 , getSliceUnsafe
5787 -- , getSlice
58− , sliceIndexerFromLen
59− , slicerFromLen
88+ , genSlicesFromLen
89+ , getSlicesFromLen
6090 , splitOn
6191
6292 -- * Streaming Operations
6393 , streamTransform
6494
65− -- * Folding
95+ -- ** Folding
6696 , streamFold
6797 , fold
6898
69− -- * Stream of Arrays
70−
71− -- XXX these are probably not very useful to have in this module as we can
72− -- express these idiomatically using streams.
73− , interpose
74− , interposeSuffix
75− , intercalateSuffix
76−
77− , compactLE
78− , pinnedCompactLE
79− , compactOnByte
80− , compactOnByteSuffix
81−
82− , foldBreakChunks
83− , foldChunks
84− , foldBreakChunksK
85− , parseBreakChunksK
86−
87− -- * Serialization
88− , encodeAs
89− , serialize
90− , pinnedSerialize
91− , deserialize
92−
9399 -- * Deprecated
94− , genSlicesFromLen
95− , getSlicesFromLen
96− , getIndices
100+ , A.toStream
101+ , A.toStreamRev
97102 )
98103where
99104
100−#include "assert.hs"
101105#include "inline.hs"
102106#include "ArrayMacros.h"
103107
108+import Control.Exception (assert)
109+import Control.Monad (when)
104110import Control.Monad.IO.Class (MonadIO(..))
105−-- import Data.Bifunctor (first)
106−-- import Data.Either (fromRight)
107111import Data.Functor.Identity (Identity)
108112import Data.Proxy (Proxy(..))
109113import Data.Word (Word8)
110114import Foreign.C.String (CString)
111115import Foreign.Ptr (castPtr)
112116import Foreign.Storable (Storable)
113−import GHC.Types (SPEC(..))
114117import Streamly.Internal.Data.Unbox (Unbox(..))
115118import Prelude hiding (length, null, last, map, (!!), read, concat)
116119
117−import Streamly.Internal.Data.MutByteArray.Type (PinnedState(..), MutByteArray)
118−import Streamly.Internal.Data.Serialize.Type (Serialize)
120+import Streamly.Internal.Data.MutArray.Type (ArrayUnsafe(..))
121+import Streamly.Internal.Data.Array.Type
122+ (Array(..), length, asPtrUnsafe)
119123import Streamly.Internal.Data.Fold.Type (Fold(..))
120−import Streamly.Internal.Data.Parser (Parser(..), Initial(..), ParseError(..))
121−import Streamly.Internal.Data.Stream (Stream(..))
122−import Streamly.Internal.Data.StreamK (StreamK)
123−import Streamly.Internal.Data.SVar.Type (adaptState, defState)
124−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3Fused'(..))
124+import Streamly.Internal.Data.Producer.Type (Producer(..))
125+import Streamly.Internal.Data.Stream (Stream)
126+import Streamly.Internal.Data.Tuple.Strict (Tuple3Fused'(..))
125127import Streamly.Internal.Data.Unfold.Type (Unfold(..))
126128import Streamly.Internal.System.IO (unsafeInlineIO)
127129
128−import qualified Streamly.Internal.Data.Fold.Type as Fold
129−import qualified Streamly.Internal.Data.Serialize.Type as Serialize
130−import qualified Streamly.Internal.Data.MutByteArray.Type as MBA
130+import qualified Streamly.Internal.Data.MutArray.Type as MA
131131import qualified Streamly.Internal.Data.MutArray as MA
132+import qualified Streamly.Internal.Data.Array.Type as A
132133import qualified Streamly.Internal.Data.Fold as FL
133−import qualified Streamly.Internal.Data.Ring as RB
134−import qualified Streamly.Internal.Data.Parser as Parser
135−-- import qualified Streamly.Internal.Data.ParserK as ParserK
134+import qualified Streamly.Internal.Data.Producer.Type as Producer
135+import qualified Streamly.Internal.Data.Producer as Producer
136+import qualified Streamly.Internal.Data.Ring.Unboxed as RB
136137import qualified Streamly.Internal.Data.Stream as D
137138import qualified Streamly.Internal.Data.Stream as Stream
138−import qualified Streamly.Internal.Data.StreamK as StreamK
139139import qualified Streamly.Internal.Data.Unfold as Unfold
140−import qualified Prelude
141−
142−import Streamly.Internal.Data.Array.Type
143140
144141#include "DocTestDataArray.hs"
145142
@@ -166,18 +163,91 @@ import Streamly.Internal.Data.Array.Type
166163-- if we use one array for each chunk. This is still significant to add
167164-- pressure to GC.
168165
166+-------------------------------------------------------------------------------
167+-- Construction
168+-------------------------------------------------------------------------------
169+
170+-- | Create an 'Array' from the first N elements of a stream. The array is
171+-- allocated to size N, if the stream terminates before N elements then the
172+-- array may hold less than N elements.
173+--
174+-- /Pre-release/
175+{-# INLINE fromStreamN #-}
176+fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
177+fromStreamN n m = do
178+ when (n < 0) $ error "writeN: negative write count specified"
179+ A.fromStreamDN n m
180+
181+-- | Create an 'Array' from a stream. This is useful when we want to create a
182+-- single array from a stream of unknown size. 'writeN' is at least twice
183+-- as efficient when the size is already known.
184+--
185+-- Note that if the input stream is too large memory allocation for the array
186+-- may fail. When the stream size is not known, `chunksOf` followed by
187+-- processing of indvidual arrays in the resulting stream should be preferred.
188+--
189+-- /Pre-release/
190+{-# INLINE fromStream #-}
191+fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
192+fromStream = Stream.fold A.write
193+-- write m = A.fromStreamD $ D.fromStreamK m
194+
169195-------------------------------------------------------------------------------
170196-- Elimination
171197-------------------------------------------------------------------------------
172198
199+{-# INLINE_NORMAL producer #-}
200+producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a
201+producer =
202+ Producer.translate A.unsafeThaw A.unsafeFreeze
203+ $ MA.producerWith (return . unsafeInlineIO)
204+
205+-- | Unfold an array into a stream.
206+--
207+{-# INLINE_NORMAL reader #-}
208+reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
209+reader = Producer.simplify producer
210+
211+-- | Unfold an array into a stream, does not check the end of the array, the
212+-- user is responsible for terminating the stream within the array bounds. For
213+-- high performance application where the end condition can be determined by
214+-- a terminating fold.
215+--
216+-- Written in the hope that it may be faster than "read", however, in the case
217+-- for which this was written, "read" proves to be faster even though the core
218+-- generated with unsafeRead looks simpler.
219+--
220+-- /Pre-release/
221+--
222+{-# INLINE_NORMAL readerUnsafe #-}
223+readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
224+readerUnsafe = Unfold step inject
225+ where
226+
227+ inject (Array contents start end) =
228+ return (ArrayUnsafe contents end start)
229+
230+ {-# INLINE_LATE step #-}
231+ step (ArrayUnsafe contents end p) = do
232+ -- unsafeInlineIO allows us to run this in Identity monad for pure
233+ -- toList/foldr case which makes them much faster due to not
234+ -- accumulating the list and fusing better with the pure consumers.
235+ --
236+ -- This should be safe as the array contents are guaranteed to be
237+ -- evaluated/written to before we peek at them.
238+ let !x = unsafeInlineIO $ peekByteIndex p contents
239+ let !p1 = INDEX_NEXT(p,a)
240+ return $ D.Yield x (ArrayUnsafe contents end p1)
241+
173242-- |
174243--
244+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
175245-- >>> null arr = Array.byteLength arr == 0
176246--
177247-- /Pre-release/
178248{-# INLINE null #-}
179249null :: Array a -> Bool
180−null arr = byteLength arr == 0
250+null arr = A.byteLength arr == 0
181251
182252-- | Like 'getIndex' but indexes the array in reverse from the end.
183253--
@@ -189,11 +259,12 @@ getIndexRev i arr =
189259 $ do
190260 let elemPtr = RINDEX_OF(arrEnd arr, i, a)
191261 if i >= 0 && elemPtr >= arrStart arr
192− then Just <$> peekAt elemPtr (arrContents arr)
262+ then Just <$> peekByteIndex elemPtr (arrContents arr)
193263 else return Nothing
194264
195265-- |
196266--
267+-- >>> import qualified Streamly.Internal.Data.Array as Array
197268-- >>> last arr = Array.getIndexRev arr 0
198269--
199270-- /Pre-release/
@@ -205,8 +276,6 @@ last = getIndexRev 0
205276-- Folds with Array as the container
206277-------------------------------------------------------------------------------
207278
208−-- XXX We should generate this from Ring.
209−
210279-- | @writeLastN n@ folds a maximum of @n@ elements from the end of the input
211280-- stream to an 'Array'.
212281--
@@ -215,7 +284,7 @@ writeLastN ::
215284 (Storable a, Unbox a, MonadIO m) => Int -> Fold m a (Array a)
216285writeLastN n
217286 | n <= 0 = fmap (const mempty) FL.drain
218− | otherwise = unsafeFreeze <$> Fold step initial done done
287+ | otherwise = A.unsafeFreeze <$> Fold step initial done
219288
220289 where
221290
@@ -228,9 +297,11 @@ writeLastN n
228297 in fmap f $ liftIO $ RB.new n
229298
230299 done (Tuple3Fused' rb rh i) = do
231− arr <- MA.new n
232− -- XXX We should write a read unfold for ring.
233− foldFunc i rh MA.snocUnsafe arr rb
300+ arr <- liftIO $ MA.new n
301+ foldFunc i rh snoc' arr rb
302+
303+ -- XXX We should write a read unfold for ring.
304+ snoc' b a = liftIO $ MA.snocUnsafe b a
234305
235306 foldFunc i
236307 | i < n = RB.unsafeFoldRingM
@@ -256,26 +327,19 @@ binarySearch = undefined
256327-- compared to streams by using SIMD instructions.
257328-- We can also return a bit array instead.
258329
259−-- Can use SIMD.
260−
261330-- | Perform a linear search to find all the indices where a given element is
262331-- present in an array.
263332--
264333-- /Unimplemented/
265−indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int
266−indexFinder = undefined
267−
268−-- |
269−-- /Unimplemented/
270−findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int
271−findIndicesOf p = Stream.unfold (indexFinder p)
334+findIndicesOf :: (a -> Bool) -> Unfold Identity (Array a) Int
335+findIndicesOf = undefined
272336
273337{-
274338findIndexOf :: (a -> Bool) -> Array a -> Maybe Int
275−findIndexOf p = Unfold.fold Fold.one . Stream.unfold (indexFinder p)
339+findIndexOf p = Unfold.fold Fold.one . Stream.unfold (findIndicesOf p)
276340
277341find :: (a -> Bool) -> Array a -> Bool
278−find = Unfold.fold Fold.null . Stream.unfold (indexFinder p)
342+find = Unfold.fold Fold.null . Stream.unfold (findIndicesOf p)
279343-}
280344
281345-------------------------------------------------------------------------------
@@ -317,45 +381,29 @@ splitOn :: (Monad m, Unbox a) =>
317381 (a -> Bool) -> Array a -> Stream m (Array a)
318382splitOn predicate arr =
319383 fmap (\(i, len) -> getSliceUnsafe i len arr)
320− $ D.indexOnSuffix predicate (read arr)
321−
322−{-# INLINE sliceIndexerFromLen #-}
323−sliceIndexerFromLen :: forall m a. (Monad m, Unbox a)
324− => Int -- ^ from index
325− -> Int -- ^ length of the slice
326− -> Unfold m (Array a) (Int, Int)
327−sliceIndexerFromLen from len =
328− Unfold.lmap unsafeThaw (MA.sliceIndexerFromLen from len)
384+ $ D.sliceOnSuffix predicate (A.toStreamD arr)
329385
330−{-# DEPRECATED genSlicesFromLen "Please use sliceIndexerFromLen instead." #-}
331386{-# INLINE genSlicesFromLen #-}
332387genSlicesFromLen :: forall m a. (Monad m, Unbox a)
333388 => Int -- ^ from index
334389 -> Int -- ^ length of the slice
335390 -> Unfold m (Array a) (Int, Int)
336−genSlicesFromLen = sliceIndexerFromLen
391+genSlicesFromLen from len =
392+ Unfold.lmap A.unsafeThaw (MA.genSlicesFromLen from len)
337393
338394-- | Generate a stream of slices of specified length from an array, starting
339395-- from the supplied array index. The last slice may be shorter than the
340396-- requested length.
341397--
342398-- /Pre-release//
343−{-# INLINE slicerFromLen #-}
344−slicerFromLen :: forall m a. (Monad m, Unbox a)
345− => Int -- ^ from index
346− -> Int -- ^ length of the slice
347− -> Unfold m (Array a) (Array a)
348−slicerFromLen from len =
349− fmap unsafeFreeze
350− $ Unfold.lmap unsafeThaw (MA.slicerFromLen from len)
351−
352−{-# DEPRECATED getSlicesFromLen "Please use slicerFromLen instead." #-}
353399{-# INLINE getSlicesFromLen #-}
354400getSlicesFromLen :: forall m a. (Monad m, Unbox a)
355401 => Int -- ^ from index
356402 -> Int -- ^ length of the slice
357403 -> Unfold m (Array a) (Array a)
358−getSlicesFromLen = slicerFromLen
404+getSlicesFromLen from len =
405+ fmap A.unsafeFreeze
406+ $ Unfold.lmap A.unsafeThaw (MA.getSlicesFromLen from len)
359407
360408-------------------------------------------------------------------------------
361409-- Random reads and writes
@@ -373,7 +421,7 @@ getIndex i arr =
373421 $ do
374422 let elemPtr = INDEX_OF(arrStart arr, i, a)
375423 if i >= 0 && INDEX_VALID(elemPtr, arrEnd arr, a)
376− then Just <$> peekAt elemPtr (arrContents arr)
424+ then Just <$> peekByteIndex elemPtr (arrContents arr)
377425 else return Nothing
378426
379427-- | Given a stream of array indices, read the elements on those indices from
@@ -385,25 +433,19 @@ getIndex i arr =
385433-- @
386434-- read =
387435-- let u = lmap (\arr -> (0, length arr - 1)) Unfold.enumerateFromTo
388−-- in Unfold.lmap f (indexReader arr)
436+-- in Unfold.lmap f (getIndices arr)
389437--
390438-- readRev =
391439-- let i = length arr - 1
392−-- in Unfold.lmap f (indexReaderFromThenTo i (i - 1) 0)
440+-- in Unfold.lmap f (getIndicesFromThenTo i (i - 1) 0)
393441-- @
394442--
395443-- /Pre-release/
396−{-# INLINE indexReader #-}
397−indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
398−indexReader m =
399− let unf = MA.indexReaderWith (return . unsafeInlineIO) m
400− in Unfold.lmap unsafeThaw unf
401−
402−-- XXX DO NOT REMOVE, change the signature to use Stream instead of unfold
403−{-# DEPRECATED getIndices "Please use getIndices instead." #-}
404444{-# INLINE getIndices #-}
405445getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
406−getIndices = indexReader
446+getIndices m =
447+ let unf = MA.getIndicesD (return . unsafeInlineIO) m
448+ in Unfold.lmap A.unsafeThaw unf
407449
408450-- | Unfolds @(from, then, to, array)@ generating a finite stream whose first
409451-- element is the array value from the index @from@ and the successive elements
@@ -419,9 +461,9 @@ getIndices = indexReader
419461-- @
420462--
421463-- /Unimplemented/
422−{-# INLINE indexReaderFromThenTo #-}
423−indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a
424−indexReaderFromThenTo = undefined
464+{-# INLINE getIndicesFromThenTo #-}
465+getIndicesFromThenTo :: Unfold m (Int, Int, Int, Array a) a
466+getIndicesFromThenTo = undefined
425467
426468-------------------------------------------------------------------------------
427469-- Transform via stream operations
@@ -438,7 +480,7 @@ indexReaderFromThenTo = undefined
438480{-# INLINE runPipe #-}
439481runPipe :: (MonadIO m, Unbox a, Unbox b)
440482 => Pipe m a b -> Array a -> m (Array b)
441−runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (read arr)
483+runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (A.read arr)
442484-}
443485
444486-- XXX For transformations that cannot change the number of elements e.g. "map"
@@ -452,7 +494,7 @@ runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (read arr)
452494streamTransform :: forall m a b. (MonadIO m, Unbox a, Unbox b)
453495 => (Stream m a -> Stream m b) -> Array a -> m (Array b)
454496streamTransform f arr =
455− Stream.fold (createWith (length arr)) $ f (read arr)
497+ Stream.fold (A.writeWith (length arr)) $ f (A.read arr)
456498
457499-------------------------------------------------------------------------------
458500-- Casts
@@ -486,7 +528,7 @@ asBytes = castUnsafe
486528--
487529cast :: forall a b. (Unbox b) => Array a -> Maybe (Array b)
488530cast arr =
489− let len = byteLength arr
531+ let len = A.byteLength arr
490532 r = len `mod` SIZE_OF(b)
491533 in if r /= 0
492534 then Nothing
@@ -504,9 +546,9 @@ cast arr =
504546--
505547asCStringUnsafe :: Array a -> (CString -> IO b) -> IO b
506548asCStringUnsafe arr act = do
507− let arr1 = asBytes arr <> fromList [0]
508− -- unsafePinnedAsPtr makes sure the array is pinned
509− unsafePinnedAsPtr arr1 $ \ptr -> act (castPtr ptr)
549+ let arr1 = asBytes arr <> A.fromList [0]
550+ -- asPtrUnsafe makes sure the array is pinned
551+ asPtrUnsafe arr1 $ \ptr -> act (castPtr ptr)
510552
511553-------------------------------------------------------------------------------
512554-- Folds
@@ -519,500 +561,11 @@ asCStringUnsafe arr act = do
519561-- /Pre-release/
520562{-# INLINE fold #-}
521563fold :: forall m a b. (Monad m, Unbox a) => Fold m a b -> Array a -> m b
522−fold f arr = Stream.fold f (read arr)
564+fold f arr = Stream.fold f (A.read arr)
523565
524566-- | Fold an array using a stream fold operation.
525567--
526568-- /Pre-release/
527569{-# INLINE streamFold #-}
528570streamFold :: (Monad m, Unbox a) => (Stream m a -> m b) -> Array a -> m b
529−streamFold f arr = f (read arr)
530−
531−--------------------------------------------------------------------------------
532−-- Serialization
533−--------------------------------------------------------------------------------
534−
535−{-# INLINE encodeAs #-}
536−encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8
537−encodeAs ps a =
538− unsafeInlineIO $ do
539− let len = Serialize.addSizeTo 0 a
540− mbarr <- MBA.newBytesAs ps len
541− off <- Serialize.serializeAt 0 mbarr a
542− assertM(len == off)
543− pure $ Array mbarr 0 off
544−
545−-- |
546−-- Properties:
547−-- 1. Identity: @deserialize . serialize == id@
548−-- 2. Encoded equivalence: @serialize a == serialize a@
549−{-# INLINE serialize #-}
550−serialize :: Serialize a => a -> Array Word8
551−serialize = encodeAs Unpinned
552−
553−-- | Serialize a Haskell type to a pinned byte array. The array is allocated
554−-- using pinned memory so that it can be used directly in OS APIs for writing
555−-- to file or sending over the network.
556−--
557−-- Properties:
558−-- 1. Identity: @deserialize . pinnedSerialize == id@
559−-- 2. Encoded equivalence: @pinnedSerialize a == pinnedSerialize a@
560−{-# INLINE pinnedSerialize #-}
561−pinnedSerialize :: Serialize a => a -> Array Word8
562−pinnedSerialize = encodeAs Pinned
563−
564−-- | Decode a Haskell type from a byte array containing its serialized
565−-- representation.
566−{-# INLINE deserialize #-}
567−deserialize :: Serialize a => Array Word8 -> a
568−deserialize arr@(Array {..}) = unsafeInlineIO $ do
569− let lenArr = length arr
570− (off, val) <-
571− Serialize.deserializeAt arrStart arrContents (arrStart + lenArr)
572− assertM(off == arrStart + lenArr)
573− pure val
574−
575−-------------------------------------------------------------------------------
576−-- Streams of Arrays
577−-------------------------------------------------------------------------------
578−
579−-- TODO: efficiently compare two streams of arrays. Two streams can have chunks
580−-- of different sizes, we can handle that in the stream comparison abstraction.
581−-- This could be useful e.g. to fast compare whether two files differ.
582−
583−-- | Insert the given element between arrays and flatten.
584−--
585−-- >>> interpose x = Stream.interpose x Array.reader
586−--
587−{-# INLINE interpose #-}
588−interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
589−interpose x = D.interpose x reader
590−
591−data FlattenState s =
592− OuterLoop s
593− | InnerLoop s !MutByteArray !Int !Int
594−
595−-- | Insert the given element after each array and flatten. This is similar to
596−-- unlines.
597−--
598−-- >>> interposeSuffix x = Stream.interposeSuffix x Array.reader
599−--
600−{-# INLINE_NORMAL interposeSuffix #-}
601−interposeSuffix :: forall m a. (Monad m, Unbox a)
602− => a -> Stream m (Array a) -> Stream m a
603−-- This does not require MonadIO constraint.
604−-- interposeSuffix x = D.interposeSuffix x reader
605−interposeSuffix sep (D.Stream step state) = D.Stream step' (OuterLoop state)
606−
607− where
608−
609− {-# INLINE_LATE step' #-}
610− step' gst (OuterLoop st) = do
611− r <- step (adaptState gst) st
612− return $ case r of
613− D.Yield Array{..} s ->
614− D.Skip (InnerLoop s arrContents arrStart arrEnd)
615− D.Skip s -> D.Skip (OuterLoop s)
616− D.Stop -> D.Stop
617−
618− step' _ (InnerLoop st _ p end) | p == end =
619− return $ D.Yield sep $ OuterLoop st
620−
621− step' _ (InnerLoop st contents p end) = do
622− x <- pure $ unsafeInlineIO $ peekAt p contents
623− return $ D.Yield x (InnerLoop st contents (INDEX_NEXT(p,a)) end)
624−
625−-- | Insert the given array after each array and flatten.
626−--
627−-- >>> intercalateSuffix = Stream.intercalateSuffix Array.reader
628−--
629−{-# INLINE intercalateSuffix #-}
630−intercalateSuffix :: (Monad m, Unbox a)
631− => Array a -> Stream m (Array a) -> Stream m a
632−intercalateSuffix = D.intercalateSuffix reader
633−
634−-- | @compactLE n@ coalesces adjacent arrays in the input stream
635−-- only if the combined size would be less than or equal to n.
636−--
637−-- Generates unpinned arrays irrespective of the pinning status of input
638−-- arrays.
639−{-# INLINE_NORMAL compactLE #-}
640−compactLE :: (MonadIO m, Unbox a)
641− => Int -> Stream m (Array a) -> Stream m (Array a)
642−compactLE n stream =
643− D.map unsafeFreeze $ MA.compactLE n $ D.map unsafeThaw stream
644−
645−-- | Pinned version of 'compactLE'.
646−{-# INLINE_NORMAL pinnedCompactLE #-}
647−pinnedCompactLE :: (MonadIO m, Unbox a)
648− => Int -> Stream m (Array a) -> Stream m (Array a)
649−pinnedCompactLE n stream =
650− D.map unsafeFreeze $ MA.pinnedCompactLE n $ D.map unsafeThaw stream
651−
652−-- | Split a stream of arrays on a given separator byte, dropping the separator
653−-- and coalescing all the arrays between two separators into a single array.
654−--
655−{-# INLINE compactOnByte #-}
656−compactOnByte
657− :: (MonadIO m)
658− => Word8
659− -> Stream m (Array Word8)
660− -> Stream m (Array Word8)
661−compactOnByte byte =
662− fmap unsafeFreeze . MA.compactOnByte byte . fmap unsafeThaw
663−
664−-- | Like 'compactOnByte' considers the separator in suffix position instead of
665−-- infix position.
666−{-# INLINE compactOnByteSuffix #-}
667−compactOnByteSuffix
668− :: (MonadIO m)
669− => Word8
670− -> Stream m (Array Word8)
671− -> Stream m (Array Word8)
672−compactOnByteSuffix byte =
673− fmap unsafeFreeze . MA.compactOnByteSuffix byte . fmap unsafeThaw
674−
675−-------------------------------------------------------------------------------
676−-- Folding Streams of Arrays
677−-------------------------------------------------------------------------------
678−
679−-- XXX This should not be used for breaking a stream as the D.cons used in
680−-- reconstructing the stream could be very bad for performance. This can only
681−-- be useful in folding without breaking.
682−{-# INLINE_NORMAL foldBreakChunks #-}
683−foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) =>
684− Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
685−foldBreakChunks (Fold fstep initial _ final) stream@(Stream step state) = do
686− res <- initial
687− case res of
688− Fold.Partial fs -> go SPEC state fs
689− Fold.Done fb -> return $! (fb, stream)
690−
691− where
692−
693− {-# INLINE go #-}
694− go !_ st !fs = do
695− r <- step defState st
696− case r of
697− Stream.Yield (Array contents start end) s ->
698− let fp = Tuple' end contents
699− in goArray SPEC s fp start fs
700− Stream.Skip s -> go SPEC s fs
701− Stream.Stop -> do
702− b <- final fs
703− return (b, D.nil)
704−
705− goArray !_ s (Tuple' end _) !cur !fs
706− | cur == end = do
707− go SPEC s fs
708− goArray !_ st fp@(Tuple' end contents) !cur !fs = do
709− x <- liftIO $ peekAt cur contents
710− res <- fstep fs x
711− let next = INDEX_NEXT(cur,a)
712− case res of
713− Fold.Done b -> do
714− let arr = Array contents next end
715− return $! (b, D.cons arr (D.Stream step st))
716− Fold.Partial fs1 -> goArray SPEC st fp next fs1
717−
718−-- This may be more robust wrt fusion compared to unfoldMany?
719−
720−-- | Fold a stream of arrays using a 'Fold'. This is equivalent to the
721−-- following:
722−--
723−-- >>> foldChunks f = Stream.fold f . Stream.unfoldMany Array.reader
724−--
725−foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b
726−foldChunks f s = fmap fst (foldBreakChunks f s)
727−-- foldStream f = Stream.fold f . Stream.unfoldMany reader
728−
729−-- | Fold a stream of arrays using a 'Fold' and return the remaining stream.
730−--
731−-- The following alternative to this function allows composing the fold using
732−-- the parser Monad:
733−--
734−-- @
735−-- foldBreakStreamK f s =
736−-- fmap (first (fromRight undefined))
737−-- $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s
738−-- @
739−--
740−-- We can compare perf and remove this one or define it in terms of that.
741−--
742−foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) =>
743− Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
744−{-
745−foldBreakChunksK f s =
746− fmap (first (fromRight undefined))
747− $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s
748−-}
749−foldBreakChunksK (Fold fstep initial _ final) stream = do
750− res <- initial
751− case res of
752− Fold.Partial fs -> go fs stream
753− Fold.Done fb -> return (fb, stream)
754−
755− where
756−
757− {-# INLINE go #-}
758− go !fs st = do
759− let stop = (, StreamK.nil) <$> final fs
760− single a = yieldk a StreamK.nil
761− yieldk (Array contents start end) r =
762− let fp = Tuple' end contents
763− in goArray fs r fp start
764− in StreamK.foldStream defState yieldk single stop st
765−
766− goArray !fs st (Tuple' end _) !cur
767− | cur == end = do
768− go fs st
769− goArray !fs st fp@(Tuple' end contents) !cur = do
770− x <- liftIO $ peekAt cur contents
771− res <- fstep fs x
772− let next = INDEX_NEXT(cur,a)
773− case res of
774− Fold.Done b -> do
775− let arr = Array contents next end
776− return $! (b, StreamK.cons arr st)
777− Fold.Partial fs1 -> goArray fs1 st fp next
778−
779−{-
780−-- This can be generalized to any type provided it can be unfolded to a stream
781−-- and it can be combined using a semigroup operation.
782−--
783−{-# INLINE_NORMAL parseBreakD #-}
784−parseBreakD ::
785− forall m a b. (MonadIO m, MonadThrow m, Unbox a)
786− => PRD.Parser a m b
787− -> D.Stream m (Array.Array a)
788− -> m (b, D.Stream m (Array.Array a))
789−parseBreakD
790− (PRD.Parser pstep initial extract) stream@(D.Stream step state) = do
791−
792− res <- initial
793− case res of
794− PRD.IPartial s -> go SPEC state (List []) s
795− PRD.IDone b -> return (b, stream)
796− PRD.IError err -> throwM $ ParseError err
797−
798− where
799−
800− -- "backBuf" contains last few items in the stream that we may have to
801− -- backtrack to.
802− --
803− -- XXX currently we are using a dumb list based approach for backtracking
804− -- buffer. This can be replaced by a sliding/ring buffer using Data.Array.
805− -- That will allow us more efficient random back and forth movement.
806− go !_ st backBuf !pst = do
807− r <- step defState st
808− case r of
809− D.Yield (Array contents start end) s ->
810− gobuf SPEC s backBuf
811− (Tuple' end contents) start pst
812− D.Skip s -> go SPEC s backBuf pst
813− D.Stop -> do
814− b <- extract pst
815− return (b, D.nil)
816−
817− -- Use strictness on "cur" to keep it unboxed
818− gobuf !_ s backBuf (Tuple' end _) !cur !pst
819− | cur == end = do
820− go SPEC s backBuf pst
821− gobuf !_ s backBuf fp@(Tuple' end contents) !cur !pst = do
822− x <- liftIO $ peekByteIndex contents cur
823− pRes <- pstep pst x
824− let next = INDEX_NEXT(cur,a)
825− case pRes of
826− PR.Partial 0 pst1 ->
827− gobuf SPEC s (List []) fp next pst1
828− PR.Partial n pst1 -> do
829− assert (n <= Prelude.length (x:getList backBuf)) (return ())
830− let src0 = Prelude.take n (x:getList backBuf)
831− arr0 = A.fromListN n (Prelude.reverse src0)
832− arr1 = Array contents next end
833− src = arr0 <> arr1
834− let !(Array cont1 start end1) = src
835− fp1 = Tuple' end1 cont1
836− gobuf SPEC s (List []) fp1 start pst1
837− PR.Continue 0 pst1 ->
838− gobuf SPEC s (List (x:getList backBuf)) fp next pst1
839− PR.Continue n pst1 -> do
840− assert (n <= Prelude.length (x:getList backBuf)) (return ())
841− let (src0, buf1) = splitAt n (x:getList backBuf)
842− arr0 = A.fromListN n (Prelude.reverse src0)
843− arr1 = Array contents next end
844− src = arr0 <> arr1
845− let !(Array cont1 start end1) = src
846− fp1 = Tuple' end1 cont1
847− gobuf SPEC s (List buf1) fp1 start pst1
848− PR.Done 0 b -> do
849− let arr = Array contents next end
850− return (b, D.cons arr (D.Stream step s))
851− PR.Done n b -> do
852− assert (n <= Prelude.length (x:getList backBuf)) (return ())
853− let src0 = Prelude.take n (x:getList backBuf)
854− -- XXX create the array in reverse instead
855− arr0 = A.fromListN n (Prelude.reverse src0)
856− arr1 = Array contents next end
857− -- XXX Use StreamK to avoid adding arbitrary layers of
858− -- constructors every time.
859− str = D.cons arr0 (D.cons arr1 (D.Stream step s))
860− return (b, str)
861− PR.Error err -> throwM $ ParseError err
862−-}
863−
864−-- | Parse an array stream using the supplied 'Parser'. Returns the parse
865−-- result and the unconsumed stream. Throws 'ParseError' if the parse fails.
866−--
867−-- The following alternative to this function allows composing the parser using
868−-- the parser Monad:
869−--
870−-- >>> parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)
871−--
872−-- We can compare perf and remove this one or define it in terms of that.
873−--
874−-- /Internal/
875−--
876−{-# INLINE_NORMAL parseBreakChunksK #-}
877−parseBreakChunksK ::
878− forall m a b. (MonadIO m, Unbox a)
879− => Parser a m b
880− -> StreamK m (Array a)
881− -> m (Either ParseError b, StreamK m (Array a))
882−-- parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)
883−parseBreakChunksK (Parser pstep initial extract) stream = do
884− res <- initial
885− case res of
886− IPartial s -> go s stream []
887− IDone b -> return (Right b, stream)
888− IError err -> return (Left (ParseError err), stream)
889−
890− where
891−
892− -- "backBuf" contains last few items in the stream that we may have to
893− -- backtrack to.
894− --
895− -- XXX currently we are using a dumb list based approach for backtracking
896− -- buffer. This can be replaced by a sliding/ring buffer using Data.Array.
897− -- That will allow us more efficient random back and forth movement.
898− go !pst st backBuf = do
899− let stop = goStop pst backBuf -- (, K.nil) <$> extract pst
900− single a = yieldk a StreamK.nil
901− yieldk arr r = goArray pst backBuf r arr
902− in StreamK.foldStream defState yieldk single stop st
903−
904− -- Use strictness on "cur" to keep it unboxed
905− goArray !pst backBuf st (Array _ cur end) | cur == end = go pst st backBuf
906− goArray !pst backBuf st (Array contents cur end) = do
907− x <- liftIO $ peekAt cur contents
908− pRes <- pstep pst x
909− let next = INDEX_NEXT(cur,a)
910− case pRes of
911− Parser.Partial 0 s ->
912− goArray s [] st (Array contents next end)
913− Parser.Partial n s -> do
914− assert (n <= Prelude.length (x:backBuf)) (return ())
915− let src0 = Prelude.take n (x:backBuf)
916− arr0 = fromListN n (Prelude.reverse src0)
917− arr1 = Array contents next end
918− src = arr0 <> arr1
919− goArray s [] st src
920− Parser.Continue 0 s ->
921− goArray s (x:backBuf) st (Array contents next end)
922− Parser.Continue n s -> do
923− assert (n <= Prelude.length (x:backBuf)) (return ())
924− let (src0, buf1) = Prelude.splitAt n (x:backBuf)
925− arr0 = fromListN n (Prelude.reverse src0)
926− arr1 = Array contents next end
927− src = arr0 <> arr1
928− goArray s buf1 st src
929− Parser.Done 0 b -> do
930− let arr = Array contents next end
931− return (Right b, StreamK.cons arr st)
932− Parser.Done n b -> do
933− assert (n <= Prelude.length (x:backBuf)) (return ())
934− let src0 = Prelude.take n (x:backBuf)
935− -- XXX Use fromListRevN once implemented
936− -- arr0 = A.fromListRevN n src0
937− arr0 = fromListN n (Prelude.reverse src0)
938− arr1 = Array contents next end
939− str = StreamK.cons arr0 (StreamK.cons arr1 st)
940− return (Right b, str)
941− Parser.Error err -> do
942− let n = Prelude.length backBuf
943− arr0 = fromListN n (Prelude.reverse backBuf)
944− arr1 = Array contents cur end
945− str = StreamK.cons arr0 (StreamK.cons arr1 stream)
946− return (Left (ParseError err), str)
947−
948− -- This is a simplified goArray
949− goExtract !pst backBuf (Array _ cur end)
950− | cur == end = goStop pst backBuf
951− goExtract !pst backBuf (Array contents cur end) = do
952− x <- liftIO $ peekAt cur contents
953− pRes <- pstep pst x
954− let next = INDEX_NEXT(cur,a)
955− case pRes of
956− Parser.Partial 0 s ->
957− goExtract s [] (Array contents next end)
958− Parser.Partial n s -> do
959− assert (n <= Prelude.length (x:backBuf)) (return ())
960− let src0 = Prelude.take n (x:backBuf)
961− arr0 = fromListN n (Prelude.reverse src0)
962− arr1 = Array contents next end
963− src = arr0 <> arr1
964− goExtract s [] src
965− Parser.Continue 0 s ->
966− goExtract s backBuf (Array contents next end)
967− Parser.Continue n s -> do
968− assert (n <= Prelude.length (x:backBuf)) (return ())
969− let (src0, buf1) = Prelude.splitAt n (x:backBuf)
970− arr0 = fromListN n (Prelude.reverse src0)
971− arr1 = Array contents next end
972− src = arr0 <> arr1
973− goExtract s buf1 src
974− Parser.Done 0 b -> do
975− let arr = Array contents next end
976− return (Right b, StreamK.fromPure arr)
977− Parser.Done n b -> do
978− assert (n <= Prelude.length backBuf) (return ())
979− let src0 = Prelude.take n backBuf
980− -- XXX Use fromListRevN once implemented
981− -- arr0 = A.fromListRevN n src0
982− arr0 = fromListN n (Prelude.reverse src0)
983− arr1 = Array contents next end
984− str = StreamK.cons arr0 (StreamK.fromPure arr1)
985− return (Right b, str)
986− Parser.Error err -> do
987− let n = Prelude.length backBuf
988− arr0 = fromListN n (Prelude.reverse backBuf)
989− arr1 = Array contents cur end
990− str = StreamK.cons arr0 (StreamK.cons arr1 stream)
991− return (Left (ParseError err), str)
992−
993− -- This is a simplified goExtract
994− {-# INLINE goStop #-}
995− goStop !pst backBuf = do
996− pRes <- extract pst
997− case pRes of
998− Parser.Partial _ _ -> error "Bug: parseBreak: Partial in extract"
999− Parser.Continue 0 s ->
1000− goStop s backBuf
1001− Parser.Continue n s -> do
1002− assert (n <= Prelude.length backBuf) (return ())
1003− let (src0, buf1) = Prelude.splitAt n backBuf
1004− arr = fromListN n (Prelude.reverse src0)
1005− goExtract s buf1 arr
1006− Parser.Done 0 b ->
1007− return (Right b, StreamK.nil)
1008− Parser.Done n b -> do
1009− assert (n <= Prelude.length backBuf) (return ())
1010− let src0 = Prelude.take n backBuf
1011− -- XXX Use fromListRevN once implemented
1012− -- arr0 = A.fromListRevN n src0
1013− arr0 = fromListN n (Prelude.reverse src0)
1014− return (Right b, StreamK.fromPure arr0)
1015− Parser.Error err -> do
1016− let n = Prelude.length backBuf
1017− arr0 = fromListN n (Prelude.reverse backBuf)
1018− return (Left (ParseError err), StreamK.fromPure arr0)
571+streamFold f arr = f (A.read arr)
core/src/Streamly/Internal/Data/Array/Generic.hs+11−49
@@ -11,8 +11,8 @@ module Streamly.Internal.Data.Array.Generic
1111
1212 -- * Construction
1313 , nil
14− , createOf
15− , create
14+ , writeN
15+ , write
1616 , writeWith
1717 , writeLastN
1818
@@ -24,8 +24,6 @@ module Streamly.Internal.Data.Array.Generic
2424 , fromListN
2525 , fromList
2626
27− , chunksOf
28−
2927 -- * Elimination
3028 , length
3129 , reader
@@ -44,10 +42,6 @@ module Streamly.Internal.Data.Array.Generic
4442 , getIndex
4543 , getSliceUnsafe
4644 , strip
47−
48− -- * Deprecated
49− , writeN
50− , write
5145 )
5246where
5347
@@ -63,7 +57,7 @@ import GHC.IO (unsafePerformIO)
6357import Text.Read (readPrec)
6458
6559import Streamly.Internal.Data.Fold.Type (Fold(..))
66−import Streamly.Internal.Data.Stream.Type (Stream)
60+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
6761import Streamly.Internal.Data.Unfold.Type (Unfold(..))
6862import Streamly.Internal.System.IO (unsafeInlineIO)
6963
@@ -71,9 +65,9 @@ import qualified Streamly.Internal.Data.MutArray.Generic as MArray
7165import qualified Streamly.Internal.Data.Fold.Type as FL
7266import qualified Streamly.Internal.Data.Producer.Type as Producer
7367import qualified Streamly.Internal.Data.Producer as Producer
74−import qualified Streamly.Internal.Data.Ring.Generic as RB
75−import qualified Streamly.Internal.Data.Stream.Type as D
76−import qualified Streamly.Internal.Data.Stream.Generate as D
68+import qualified Streamly.Internal.Data.Ring as RB
69+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
70+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
7771import qualified Text.ParserCombinators.ReadPrec as ReadPrec
7872
7973import Prelude hiding (Foldable(..), read)
@@ -108,33 +102,21 @@ nil = unsafePerformIO $ unsafeFreeze <$> MArray.nil
108102-- Construction - Folds
109103-------------------------------------------------------------------------------
110104
111−{-# INLINE_NORMAL createOf #-}
112−createOf :: MonadIO m => Int -> Fold m a (Array a)
113−createOf = fmap unsafeFreeze <$> MArray.createOf
114−
115−-- XXX Deprecate in major
116−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
117−{-# INLINE writeN #-}
105+{-# INLINE_NORMAL writeN #-}
118106writeN :: MonadIO m => Int -> Fold m a (Array a)
119−writeN = createOf
107+writeN = fmap unsafeFreeze <$> MArray.writeN
120108
121109{-# INLINE_NORMAL writeWith #-}
122110writeWith :: MonadIO m => Int -> Fold m a (Array a)
123−writeWith elemCount = unsafeFreeze <$> MArray.createWith elemCount
111+writeWith elemCount = unsafeFreeze <$> MArray.writeWith elemCount
124112
125113-- | Fold the whole input to a single array.
126114--
127115-- /Caution! Do not use this on infinite streams./
128116--
129−{-# INLINE create #-}
130−create :: MonadIO m => Fold m a (Array a)
131−create = fmap unsafeFreeze MArray.create
132−
133−-- XXX Deprecate in major
134−-- {-# DEPRECATED write "Please use create instead." #-}
135117{-# INLINE write #-}
136118write :: MonadIO m => Fold m a (Array a)
137−write = create
119+write = fmap unsafeFreeze MArray.write
138120
139121fromPureStream :: Stream Identity a -> Array a
140122fromPureStream x =
@@ -145,15 +127,6 @@ fromPureStream x =
145127fromByteStr# :: Addr# -> Array Word8
146128fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
147129
148−-------------------------------------------------------------------------------
149−-- Stream Ops
150−-------------------------------------------------------------------------------
151−
152−{-# INLINE_NORMAL chunksOf #-}
153−chunksOf :: forall m a. MonadIO m
154− => Int -> Stream m a -> Stream m (Array a)
155−chunksOf n strm = fmap unsafeFreeze $ MArray.chunksOf n strm
156−
157130-------------------------------------------------------------------------------
158131-- Construction - from streams
159132-------------------------------------------------------------------------------
@@ -258,17 +231,6 @@ getIndex i arr@Array {..} =
258231 then Just $ getIndexUnsafe i arr
259232 else Nothing
260233
261−-- >>> import qualified Streamly.Data.Stream as Stream
262−-- >>> import qualified Streamly.Data.Fold as Fold
263−-- >>> import qualified Streamly.Internal.Data.Array.Generic as Array
264−-- >>> import Data.Function ((&))
265−-- >>> :{
266−-- Stream.fromList [1,2,3,4,5::Int]
267−-- & Stream.scan (Array.writeLastN 2)
268−-- & Stream.fold Fold.toList
269−-- :}
270−-- [fromList [],fromList [1],fromList [1,2],fromList [2,3],fromList [3,4],fromList [4,5]]
271−--
272234{-# INLINE writeLastN #-}
273235writeLastN :: MonadIO m => Int -> Fold m a (Array a)
274236writeLastN n = FL.rmapM f (RB.writeLastN n)
@@ -276,7 +238,7 @@ writeLastN n = FL.rmapM f (RB.writeLastN n)
276238 where
277239
278240 f rb = do
279− arr <- RB.copyToMutArray 0 n rb
241+ arr <- RB.toMutArray 0 n rb
280242 return $ unsafeFreeze arr
281243
282244{-# INLINE getSliceUnsafe #-}
core/src/Streamly/Internal/Data/Array/Type.hs+108−538
@@ -1,11 +1,8 @@
11{-# LANGUAGE CPP #-}
2−{-# LANGUAGE TypeFamilies #-}
3−-- Must come after TypeFamilies, otherwise it is re-enabled.
4−-- MonoLocalBinds enabled by TypeFamilies causes perf regressions in general.
5−{-# LANGUAGE NoMonoLocalBinds #-}
62-- |
73-- Module : Streamly.Internal.Data.Array.Type
84-- Copyright : (c) 2020 Composewell Technologies
5+--
96-- License : BSD3-3-Clause
107-- Maintainer : streamly@composewell.com
118-- Stability : experimental
@@ -13,146 +10,79 @@
1310--
1411-- See notes in "Streamly.Internal.Data.MutArray.Type"
1512--
16−
1713module Streamly.Internal.Data.Array.Type
1814 (
19− -- ** Type
2015 -- $arrayNotes
2116 Array (..)
17+ , asPtrUnsafe
18+ , nil
2219
23− -- ** Conversion
24− -- *** Mutable and Immutable
20+ -- * Freezing and Thawing
2521 , unsafeFreeze
2622 , unsafeFreezeWithShrink
2723 , unsafeThaw
2824
29− -- *** Pinned and Unpinned
25+ -- * Pinning and Unpinning
3026 , pin
3127 , unpin
3228 , isPinned
3329
34− -- *** Casting
35− , unsafePinnedAsPtr
36−
37− -- ** Construction
38− , empty
30+ -- * Construction
31+ , splice
3932
40− -- *** Cloning
41− -- XXX Why would we clone an immutable array?
42− , clone
43− , pinnedClone
44−
45− -- *** Slicing
46− -- | Get a subarray without copying
47− , splitAt
48− , breakOn -- XXX requires MonadIO
49−
50− -- *** Stream Folds
51− , unsafeMakePure
52− , createOf
53− , pinnedCreateOf
54− , unsafeCreateOf
55− , unsafePinnedCreateOf
56− , create
57− , pinnedCreate
58− , createWith
59−
60− -- *** From containers
61− , fromListN
62− , pinnedFromListN
6333 , fromList
6434 , pinnedFromList
65− , fromListRevN
35+ , fromListN
36+ , pinnedFromListN
6637 , fromListRev
67− , fromStreamN
68− , fromStream
69− , fromPureStreamN
38+ , fromListRevN
39+ , fromStreamDN
40+ , fromStreamD
7041 , fromPureStream
7142 , fromByteStr#
72− , fromByteStr
73− , fromPtrN
74− , fromChunks
75− , fromChunksK
7643
77− -- ** Reading
44+ -- * Split
45+ , breakOn
7846
79− -- *** Indexing
80− , unsafeIndexIO -- XXX unsafeGetIndexIO
81− , getIndexUnsafe -- XXX unsafeGetIndex
82−
83− -- *** To Streams
84− , read
85− , readRev
86− , toStreamK
87− , toStreamKRev
88−
89− -- *** To Containers
90− , toList
91−
92− -- *** Unfolds
93− , producer -- experimental
94− , readerUnsafe
95− , reader
96− , readerRev
97−
98− -- *** Size
99− , length
47+ -- * Elimination
48+ , unsafeIndexIO
49+ , unsafeIndex -- getIndexUnsafe
10050 , byteLength
51+ , length
10152
102− -- ** Folding
10353 , foldl'
10454 , foldr
105− , byteCmp
106− , byteEq
107−
108− -- ** Appending
109− , splice -- XXX requires MonadIO
110−
111− -- ** Streams of arrays
112− -- *** Chunk
113− -- | Group a stream into arrays.
114− , chunksOf
115− , pinnedChunksOf
116− , buildChunks
117−
118− -- *** Split
119− -- | Split an array into slices.
120−
121− -- *** Concat
122− -- | Append the arrays in a stream to form a stream of elements.
123− , concat
124− , concatRev
125−
126− -- *** Compact
127− -- | Append the arrays in a stream to form a stream of larger arrays.
128− , fCompactGE
129− , fPinnedCompactGE
130− , lCompactGE
131− , lPinnedCompactGE
132− , compactGE
55+ , splitAt
13356
134− -- ** Deprecated
135− , asPtrUnsafe
136− , unsafeIndex
137− , bufferChunks
138− , flattenArrays
139− , flattenArraysRev
140− , fromArrayStreamK
141− , fromStreamDN
142− , fromStreamD
14357 , toStreamD
14458 , toStreamDRev
59+ , toStreamK
60+ , toStreamKRev
14561 , toStream
14662 , toStreamRev
147− , nil
63+ , read
64+ , readRev
65+ , readerRev
66+ , toList
67+
68+ -- * Folds
14869 , writeWith
14970 , writeN
15071 , pinnedWriteN
15172 , writeNUnsafe
15273 , pinnedWriteNUnsafe
74+ , MA.ArrayUnsafe (..)
15375 , pinnedWriteNAligned
15476 , write
15577 , pinnedWrite
78+ , unsafeMakePure
79+
80+ -- * Streams of arrays
81+ , chunksOf
82+ , pinnedChunksOf
83+ , bufferChunks
84+ , flattenArrays
85+ , flattenArraysRev
15686 )
15787where
15888
@@ -160,36 +90,31 @@ where
16090#include "inline.hs"
16191
16292import Control.Exception (assert)
163−import Control.Monad (replicateM, when)
93+import Control.Monad (replicateM)
16494import Control.Monad.IO.Class (MonadIO(..))
16595import Data.Functor.Identity (Identity(..))
166−import Data.Int (Int8, Int16, Int32, Int64)
16796import Data.Proxy (Proxy(..))
168−import Data.Word (Word8, Word16, Word32, Word64)
97+import Data.Word (Word8)
16998import GHC.Base (build)
17099import GHC.Exts (IsList, IsString(..), Addr#)
171100
172101import GHC.IO (unsafePerformIO)
173102import GHC.Ptr (Ptr(..))
174−import Streamly.Internal.Data.Producer.Type (Producer(..))
175−import Streamly.Internal.Data.MutArray.Type (MutArray(..))
176−import Streamly.Internal.Data.MutByteArray.Type (MutByteArray)
103+import Streamly.Internal.Data.MutArray.Type (MutArray(..), MutableByteArray)
177104import Streamly.Internal.Data.Fold.Type (Fold(..))
178−import Streamly.Internal.Data.Stream.Type (Stream)
179−import Streamly.Internal.Data.StreamK.Type (StreamK)
105+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
180106import Streamly.Internal.Data.Unbox (Unbox(..))
181107import Streamly.Internal.Data.Unfold.Type (Unfold(..))
182108import Text.Read (readPrec)
183109
184−import Prelude hiding (Foldable(..), concat, read, unlines, splitAt)
110+import Prelude hiding (Foldable(..), read, unlines, splitAt)
185111
186112import qualified GHC.Exts as Exts
187−import qualified Streamly.Internal.Data.Fold.Type as Fold
188113import qualified Streamly.Internal.Data.MutArray.Type as MA
189−import qualified Streamly.Internal.Data.Stream.Type as D
190−import qualified Streamly.Internal.Data.StreamK.Type as K
191−import qualified Streamly.Internal.Data.MutByteArray.Type as Unboxed
192−import qualified Streamly.Internal.Data.Producer as Producer
114+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
115+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
116+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
117+import qualified Streamly.Internal.Data.Unbox as Unboxed
193118import qualified Streamly.Internal.Data.Unfold.Type as Unfold
194119import qualified Text.ParserCombinators.ReadPrec as ReadPrec
195120
@@ -232,7 +157,7 @@ data Array a =
232157#endif
233158 -- All offsets are in terms of bytes from the start of arraycontents
234159 Array
235− { arrContents :: {-# UNPACK #-} !MutByteArray
160+ { arrContents :: {-# UNPACK #-} !MutableByteArray
236161 , arrStart :: {-# UNPACK #-} !Int -- offset
237162 , arrEnd :: {-# UNPACK #-} !Int -- offset + len
238163 }
@@ -243,20 +168,14 @@ data Array a =
243168
244169-- | Use an @Array a@ as @Ptr a@.
245170--
246−-- See 'MA.unsafePinnedAsPtr' in the Mutable array module for more details.
171+-- See 'MA.asPtrUnsafe' in the Mutable array module for more details.
247172--
248173-- /Unsafe/
249174--
250175-- /Pre-release/
251176--
252−{-# INLINE unsafePinnedAsPtr #-}
253−unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
254−unsafePinnedAsPtr arr = MA.unsafePinnedAsPtr (unsafeThaw arr)
255−
256−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
257−{-# INLINE asPtrUnsafe #-}
258177asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
259−asPtrUnsafe = unsafePinnedAsPtr
178+asPtrUnsafe arr = MA.asPtrUnsafe (unsafeThaw arr)
260179
261180-------------------------------------------------------------------------------
262181-- Freezing and Thawing
@@ -303,19 +222,18 @@ unsafeThaw (Array ac as ae) = MutArray ac as ae ae
303222-- Pinning & Unpinning
304223-------------------------------------------------------------------------------
305224
306−-- | Return a copy of the 'Array' in pinned memory if unpinned, else return the
307−-- original array.
225+-- | Copy the 'Array' to pinned memory if unpinned, else do nothing. The
226+-- overhead is a copy if the input array is unpinned.
308227{-# INLINE pin #-}
309228pin :: Array a -> IO (Array a)
310229pin = fmap unsafeFreeze . MA.pin . unsafeThaw
311230
312−-- | Return a copy of the 'Array' in unpinned memory if pinned, else return the
313−-- original array.
231+-- | Copy the 'Array' to unpinned memory if pinned, else do nothing. The
232+-- overhead is a copy if the input array is pinned.
314233{-# INLINE unpin #-}
315234unpin :: Array a -> IO (Array a)
316235unpin = fmap unsafeFreeze . MA.unpin . unsafeThaw
317236
318−-- | Return 'True' if the array is allocated in pinned memory.
319237{-# INLINE isPinned #-}
320238isPinned :: Array a -> Bool
321239isPinned = MA.isPinned . unsafeThaw
@@ -324,14 +242,11 @@ isPinned = MA.isPinned . unsafeThaw
324242-- Construction
325243-------------------------------------------------------------------------------
326244
327−-- | Copy two immutable arrays into a new array. If you want to splice more
328−-- than two arrays then this operation would be highly inefficient because it
329−-- would make a copy on every splice operation, instead use the
330−-- 'fromChunksK' operation to combine n immutable arrays.
245+-- Splice two immutable arrays creating a new array.
331246{-# INLINE splice #-}
332−splice :: MonadIO m => Array a -> Array a -> m (Array a)
247+splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
333248splice arr1 arr2 =
334− unsafeFreeze <$> MA.spliceCopy (unsafeThaw arr1) (unsafeThaw arr2)
249+ unsafeFreeze <$> MA.splice (unsafeThaw arr1) (unsafeThaw arr2)
335250
336251-- | Create an 'Array' from the first N elements of a list. The array is
337252-- allocated to size N, if the list terminates before N elements then the
@@ -375,83 +290,24 @@ pinnedFromList xs = unsafePerformIO $ unsafeFreeze <$> MA.pinnedFromList xs
375290fromListRev :: Unbox a => [a] -> Array a
376291fromListRev xs = unsafePerformIO $ unsafeFreeze <$> MA.fromListRev xs
377292
378−-- | Create an 'Array' from the first N elements of a stream. The array is
379−-- allocated to size N, if the stream terminates before N elements then the
380−-- array may hold less than N elements.
381−--
382−-- >>> fromStreamN n = Stream.fold (Array.writeN n)
383−--
384−-- /Pre-release/
385−{-# INLINE_NORMAL fromStreamN #-}
386−fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
387−fromStreamN n m = do
388− when (n < 0) $ error "writeN: negative write count specified"
389− unsafeFreeze <$> MA.fromStreamN n m
390−-- fromStreamN n = D.fold (writeN n)
391−
392−{-# DEPRECATED fromStreamDN "Please use fromStreamN instead." #-}
293+{-# INLINE_NORMAL fromStreamDN #-}
393294fromStreamDN :: forall m a. (MonadIO m, Unbox a)
394295 => Int -> D.Stream m a -> m (Array a)
395−fromStreamDN = fromStreamN
296+fromStreamDN limit str = unsafeFreeze <$> MA.fromStreamDN limit str
396297
397−-- | Create an 'Array' from a stream. This is useful when we want to create a
398−-- single array from a stream of unknown size. 'writeN' is at least twice
399−-- as efficient when the size is already known.
400−--
401−-- >>> fromStream = Stream.fold Array.write
402−--
403−-- Note that if the input stream is too large memory allocation for the array
404−-- may fail. When the stream size is not known, `chunksOf` followed by
405−-- processing of indvidual arrays in the resulting stream should be preferred.
406−--
407−-- /Pre-release/
408298{-# INLINE_NORMAL fromStreamD #-}
409−fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
410−fromStream = D.fold write
411−-- fromStreamD str = unsafeFreeze <$> MA.fromStream str
412−
413−{-# DEPRECATED fromStreamD "Please use fromStream instead." #-}
414299fromStreamD :: forall m a. (MonadIO m, Unbox a)
415300 => D.Stream m a -> m (Array a)
416−fromStreamD = fromStream
417−
418−-------------------------------------------------------------------------------
419−-- Cloning
420−-------------------------------------------------------------------------------
421−
422−{-# INLINE clone #-}
423−clone ::
424− ( MonadIO m
425−#ifdef DEVBUILD
426− , Unbox a
427−#endif
428− )
429− => Array a -> m (Array a)
430−clone = fmap unsafeFreeze . MA.clone . unsafeThaw
431−
432−{-# INLINE pinnedClone #-}
433−pinnedClone ::
434− ( MonadIO m
435−#ifdef DEVBUILD
436− , Unbox a
437−#endif
438− )
439− => Array a -> m (Array a)
440−pinnedClone = fmap unsafeFreeze . MA.pinnedClone . unsafeThaw
301+fromStreamD str = unsafeFreeze <$> MA.fromStreamD str
441302
442303-------------------------------------------------------------------------------
443304-- Streams of arrays
444305-------------------------------------------------------------------------------
445306
446−{-# INLINE buildChunks #-}
447−buildChunks :: (MonadIO m, Unbox a) =>
448− D.Stream m a -> m (K.StreamK m (Array a))
449−buildChunks m = D.foldr K.cons K.nil $ chunksOf defaultChunkSize m
450−
451−{-# DEPRECATED bufferChunks "Please use buildChunks instead." #-}
307+{-# INLINE bufferChunks #-}
452308bufferChunks :: (MonadIO m, Unbox a) =>
453309 D.Stream m a -> m (K.StreamK m (Array a))
454−bufferChunks = buildChunks
310+bufferChunks m = D.foldr K.cons K.nil $ chunksOf defaultChunkSize m
455311
456312-- | @chunksOf n stream@ groups the elements in the input stream into arrays of
457313-- @n@ elements each.
@@ -472,100 +328,27 @@ pinnedChunksOf :: forall m a. (MonadIO m, Unbox a)
472328 => Int -> D.Stream m a -> D.Stream m (Array a)
473329pinnedChunksOf n str = D.map unsafeFreeze $ MA.pinnedChunksOf n str
474330
475−-- | Convert a stream of arrays into a stream of their elements.
331+-- | Use the "read" unfold instead.
332+--
333+-- @flattenArrays = unfoldMany read@
476334--
477−-- >>> concat = Stream.unfoldMany Array.reader
335+-- We can try this if there are any fusion issues in the unfold.
478336--
479−{-# INLINE_NORMAL concat #-}
480−concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
481−-- XXX this requires MonadIO whereas the unfoldMany version does not
482−concat = MA.concatWith (pure . unsafeInlineIO) . D.map unsafeThaw
483−-- concat = D.unfoldMany reader
484−
485−{-# DEPRECATED flattenArrays "Please use \"unfoldMany reader\" instead." #-}
486−{-# INLINE flattenArrays #-}
337+{-# INLINE_NORMAL flattenArrays #-}
487338flattenArrays :: forall m a. (MonadIO m, Unbox a)
488339 => D.Stream m (Array a) -> D.Stream m a
489−flattenArrays = concat
340+flattenArrays = MA.flattenArrays . D.map unsafeThaw
490341
491−-- | Convert a stream of arrays into a stream of their elements reversing the
492−-- contents of each array before flattening.
342+-- | Use the "readRev" unfold instead.
493343--
494−-- >>> concatRev = Stream.unfoldMany Array.readerRev
344+-- @flattenArrays = unfoldMany readRev@
495345--
496−{-# INLINE_NORMAL concatRev #-}
497−concatRev :: forall m a. (Monad m, Unbox a)
498− => D.Stream m (Array a) -> D.Stream m a
499−-- XXX this requires MonadIO whereas the unfoldMany version does not
500−concatRev = MA.concatRevWith (pure . unsafeInlineIO) . D.map unsafeThaw
501−-- concatRev = D.unfoldMany readerRev
502−
503−{-# DEPRECATED flattenArraysRev "Please use \"unfoldMany readerRev\" instead." #-}
504−{-# INLINE flattenArraysRev #-}
346+-- We can try this if there are any fusion issues in the unfold.
347+--
348+{-# INLINE_NORMAL flattenArraysRev #-}
505349flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
506350 => D.Stream m (Array a) -> D.Stream m a
507−flattenArraysRev = concatRev
508−
509−-------------------------------------------------------------------------------
510−-- Compact
511−-------------------------------------------------------------------------------
512−
513−-- XXX Note that this thaws immutable arrays for appending, that may be
514−-- problematic if multiple users do the same thing, however, thawed immutable
515−-- arrays would have no capacity to append, therefore, a copy will be forced
516−-- anyway.
517−
518−-- | Fold @fCompactGE n@ coalesces adjacent arrays in the input stream
519−-- until the size becomes greater than or equal to n.
520−--
521−-- Generates unpinned arrays irrespective of the pinning status of input
522−-- arrays.
523−{-# INLINE_NORMAL fCompactGE #-}
524−fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
525−fCompactGE n = fmap unsafeFreeze $ Fold.lmap unsafeThaw $ MA.fCompactGE n
526−
527−-- | PInned version of 'fCompactGE'.
528−{-# INLINE_NORMAL fPinnedCompactGE #-}
529−fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
530−fPinnedCompactGE n =
531− fmap unsafeFreeze $ Fold.lmap unsafeThaw $ MA.fPinnedCompactGE n
532−
533−-- | @compactGE n stream@ coalesces adjacent arrays in the @stream@ until
534−-- the size becomes greater than or equal to @n@.
535−--
536−-- >>> compactGE n = Stream.foldMany (Array.fCompactGE n)
537−--
538−-- Generates unpinned arrays irrespective of the pinning status of input
539−-- arrays.
540−{-# INLINE compactGE #-}
541−compactGE ::
542− (MonadIO m, Unbox a)
543− => Int -> Stream m (Array a) -> Stream m (Array a)
544−compactGE n stream =
545− D.map unsafeFreeze $ MA.compactGE n $ D.map unsafeThaw stream
546−
547−-- | Like 'compactGE' but for transforming folds instead of stream.
548−--
549−-- >>> lCompactGE n = Fold.many (Array.fCompactGE n)
550−--
551−-- Generates unpinned arrays irrespective of the pinning status of input
552−-- arrays.
553−{-# INLINE_NORMAL lCompactGE #-}
554−lCompactGE :: (MonadIO m, Unbox a)
555− => Int -> Fold m (Array a) () -> Fold m (Array a) ()
556−lCompactGE n fld =
557− Fold.lmap unsafeThaw $ MA.lCompactGE n (Fold.lmap unsafeFreeze fld)
558−
559−-- | Pinned version of 'lCompactGE'.
560−{-# INLINE_NORMAL lPinnedCompactGE #-}
561−lPinnedCompactGE :: (MonadIO m, Unbox a)
562− => Int -> Fold m (Array a) () -> Fold m (Array a) ()
563−lPinnedCompactGE n fld =
564− Fold.lmap unsafeThaw $ MA.lPinnedCompactGE n (Fold.lmap unsafeFreeze fld)
565−
566−-------------------------------------------------------------------------------
567−-- Splitting
568−-------------------------------------------------------------------------------
351+flattenArraysRev = MA.flattenArraysRev . D.map unsafeThaw
569352
570353-- Drops the separator byte
571354{-# INLINE breakOn #-}
@@ -587,14 +370,9 @@ unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a
587370unsafeIndexIO i arr = MA.getIndexUnsafe i (unsafeThaw arr)
588371
589372-- | Return element at the specified index without checking the bounds.
590−{-# INLINE_NORMAL getIndexUnsafe #-}
591−getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a
592−getIndexUnsafe i arr = let !r = unsafeInlineIO $ unsafeIndexIO i arr in r
593−
594−{-# DEPRECATED unsafeIndex "Please use 'getIndexUnsafe' instead" #-}
595373{-# INLINE_NORMAL unsafeIndex #-}
596374unsafeIndex :: forall a. Unbox a => Int -> Array a -> a
597−unsafeIndex = getIndexUnsafe
375+unsafeIndex i arr = let !r = unsafeInlineIO $ unsafeIndexIO i arr in r
598376
599377-- | /O(1)/ Get the byte length of the array.
600378--
@@ -609,68 +387,24 @@ byteLength = MA.byteLength . unsafeThaw
609387length :: Unbox a => Array a -> Int
610388length arr = MA.length (unsafeThaw arr)
611389
612−{-# INLINE_NORMAL producer #-}
613−producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a
614−producer =
615− Producer.translate unsafeThaw unsafeFreeze
616− $ MA.producerWith (return . unsafeInlineIO)
617−
618−-- | Unfold an array into a stream.
619−--
620−{-# INLINE_NORMAL reader #-}
621−reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
622−reader = Producer.simplify producer
623−
624−-- | Unfold an array into a stream, does not check the end of the array, the
625−-- user is responsible for terminating the stream within the array bounds. For
626−-- high performance application where the end condition can be determined by
627−-- a terminating fold.
628−--
629−-- Written in the hope that it may be faster than "read", however, in the case
630−-- for which this was written, "read" proves to be faster even though the core
631−-- generated with unsafeRead looks simpler.
632−--
633−-- /Pre-release/
634−--
635−{-# INLINE_NORMAL readerUnsafe #-}
636−readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
637−readerUnsafe = Unfold step inject
638− where
639−
640− inject (Array contents start end) =
641− return (MA.ArrayUnsafe contents end start)
642−
643− {-# INLINE_LATE step #-}
644− step (MA.ArrayUnsafe contents end p) = do
645− -- unsafeInlineIO allows us to run this in Identity monad for pure
646− -- toList/foldr case which makes them much faster due to not
647− -- accumulating the list and fusing better with the pure consumers.
648− --
649− -- This should be safe as the array contents are guaranteed to be
650− -- evaluated/written to before we peek at them.
651− let !x = unsafeInlineIO $ peekAt p contents
652− let !p1 = INDEX_NEXT(p,a)
653− return $ D.Yield x (MA.ArrayUnsafe contents end p1)
654−
655390-- | Unfold an array into a stream in reverse order.
656391--
657392{-# INLINE_NORMAL readerRev #-}
658393readerRev :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
659394readerRev = Unfold.lmap unsafeThaw $ MA.readerRevWith (return . unsafeInlineIO)
660395
661−{-# DEPRECATED toStreamD "Please use 'read' instead." #-}
662396{-# INLINE_NORMAL toStreamD #-}
663397toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> D.Stream m a
664−toStreamD = read
398+toStreamD arr = MA.toStreamDWith (return . unsafeInlineIO) (unsafeThaw arr)
665399
666400{-# INLINE toStreamK #-}
667401toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> K.StreamK m a
668402toStreamK arr = MA.toStreamKWith (return . unsafeInlineIO) (unsafeThaw arr)
669403
670−{-# DEPRECATED toStreamDRev "Please use 'readRev' instead." #-}
671404{-# INLINE_NORMAL toStreamDRev #-}
672405toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> D.Stream m a
673−toStreamDRev = readRev
406+toStreamDRev arr =
407+ MA.toStreamDRevWith (return . unsafeInlineIO) (unsafeThaw arr)
674408
675409{-# INLINE toStreamKRev #-}
676410toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> K.StreamK m a
@@ -682,7 +416,7 @@ toStreamKRev arr =
682416-- /Pre-release/
683417{-# INLINE_EARLY read #-}
684418read :: (Monad m, Unbox a) => Array a -> Stream m a
685−read arr = MA.toStreamWith (return . unsafeInlineIO) (unsafeThaw arr)
419+read = toStreamD
686420
687421-- | Same as 'read'
688422--
@@ -699,7 +433,7 @@ toStream = read
699433-- /Pre-release/
700434{-# INLINE_EARLY readRev #-}
701435readRev :: (Monad m, Unbox a) => Array a -> Stream m a
702−readRev arr = MA.toStreamRevWith (return . unsafeInlineIO) (unsafeThaw arr)
436+readRev = toStreamDRev
703437
704438-- | Same as 'readRev'
705439--
@@ -723,7 +457,6 @@ foldr f z arr = runIdentity $ D.foldr f z $ toStreamD arr
723457-- | Create two slices of an array without copying the original array. The
724458-- specified index @i@ is the first index of the second slice.
725459--
726−{-# INLINE splitAt #-}
727460splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)
728461splitAt i arr = (unsafeFreeze a, unsafeFreeze b)
729462 where
@@ -743,8 +476,8 @@ toListFB c n Array{..} = go arrStart
743476 -- accumulating the list and fusing better with the pure consumers.
744477 --
745478 -- This should be safe as the array contents are guaranteed to be
746− -- evaluated/written to before we peekAt at them.
747− let !x = unsafeInlineIO $ peekAt p arrContents
479+ -- evaluated/written to before we peekByteIndex at them.
480+ let !x = unsafeInlineIO $ peekByteIndex p arrContents
748481 in c x (go (INDEX_NEXT(p,a)))
749482
750483-- | Convert an 'Array' into a list.
@@ -757,28 +490,17 @@ toList s = build (\c n -> toListFB c n s)
757490-- Folds
758491-------------------------------------------------------------------------------
759492
760−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
493+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
761494-- 'Array'.
762495--
763−{-# INLINE_NORMAL createOf #-}
764−createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
765−createOf = fmap unsafeFreeze . MA.createOf
766−
767−-- XXX Deprecate in major
768−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
769−{-# INLINE writeN #-}
496+{-# INLINE_NORMAL writeN #-}
770497writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
771−writeN = createOf
498+writeN = fmap unsafeFreeze . MA.writeN
772499
773−-- | Like 'createOf' but creates a pinned array.
774−{-# INLINE_NORMAL pinnedCreateOf #-}
775−pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
776−pinnedCreateOf = fmap unsafeFreeze . MA.pinnedCreateOf
777−
778−{-# DEPRECATED pinnedWriteN "Please use pinnedCreateOf instead." #-}
779−{-# INLINE pinnedWriteN #-}
500+-- | Like 'fromListN' but creates a pinned array.
501+{-# INLINE_NORMAL pinnedWriteN #-}
780502pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
781−pinnedWriteN = pinnedCreateOf
503+pinnedWriteN = fmap unsafeFreeze . MA.pinnedWriteN
782504
783505-- | @pinnedWriteNAligned alignment n@ folds a maximum of @n@ elements from the input
784506-- stream to an 'Array' aligned to the given size.
@@ -786,74 +508,44 @@ pinnedWriteN = pinnedCreateOf
786508-- /Pre-release/
787509--
788510{-# INLINE_NORMAL pinnedWriteNAligned #-}
789−{-# DEPRECATED pinnedWriteNAligned "To be removed." #-}
790511pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a)
791512 => Int -> Int -> Fold m a (Array a)
792513pinnedWriteNAligned alignSize = fmap unsafeFreeze . MA.pinnedWriteNAligned alignSize
793514
794−-- | Like 'createOf' but does not check the array bounds when writing. The fold
515+-- | Like 'writeN' but does not check the array bounds when writing. The fold
795516-- driver must not call the step function more than 'n' times otherwise it will
796517-- corrupt the memory and crash. This function exists mainly because any
797518-- conditional in the step function blocks fusion causing 10x performance
798519-- slowdown.
799520--
800−{-# INLINE_NORMAL unsafeCreateOf #-}
801−unsafeCreateOf :: forall m a. (MonadIO m, Unbox a)
802− => Int -> Fold m a (Array a)
803−unsafeCreateOf n = unsafeFreeze <$> MA.unsafeCreateOf n
804−
805−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
806−{-# INLINE writeNUnsafe #-}
521+{-# INLINE_NORMAL writeNUnsafe #-}
807522writeNUnsafe :: forall m a. (MonadIO m, Unbox a)
808523 => Int -> Fold m a (Array a)
809−writeNUnsafe = unsafeCreateOf
524+writeNUnsafe n = unsafeFreeze <$> MA.writeNUnsafe n
810525
811−{-# INLINE_NORMAL unsafePinnedCreateOf #-}
812−unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a)
813− => Int -> Fold m a (Array a)
814−unsafePinnedCreateOf n = unsafeFreeze <$> MA.unsafePinnedCreateOf n
815−
816−{-# DEPRECATED pinnedWriteNUnsafe "Please use unsafePinnedCreateOf instead." #-}
817−{-# INLINE pinnedWriteNUnsafe #-}
526+{-# INLINE_NORMAL pinnedWriteNUnsafe #-}
818527pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a)
819528 => Int -> Fold m a (Array a)
820−pinnedWriteNUnsafe = unsafePinnedCreateOf
821−
822−{-# INLINE_NORMAL createWith #-}
823−createWith :: forall m a. (MonadIO m, Unbox a)
824− => Int -> Fold m a (Array a)
825−-- createWith n = FL.rmapM spliceArrays $ toArraysOf n
826−createWith elemCount = unsafeFreeze <$> MA.createWith elemCount
529+pinnedWriteNUnsafe n = unsafeFreeze <$> MA.pinnedWriteNUnsafe n
827530
828−{-# DEPRECATED writeWith "Please use createWith instead." #-}
829−{-# INLINE writeWith #-}
531+{-# INLINE_NORMAL writeWith #-}
830532writeWith :: forall m a. (MonadIO m, Unbox a)
831533 => Int -> Fold m a (Array a)
832−writeWith = createWith
534+-- writeWith n = FL.rmapM spliceArrays $ toArraysOf n
535+writeWith elemCount = unsafeFreeze <$> MA.writeWith elemCount
833536
834537-- | Fold the whole input to a single array.
835538--
836539-- /Caution! Do not use this on infinite streams./
837540--
838−{-# INLINE create #-}
839−create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
840−create = fmap unsafeFreeze MA.create
841−
842−-- XXX Deprecate in major
843−-- {-# DEPRECATED write "Please use create instead." #-}
844541{-# INLINE write #-}
845542write :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
846−write = create
543+write = fmap unsafeFreeze MA.write
847544
848−-- | Like 'create' but creates a pinned array.
849−{-# INLINE pinnedCreate #-}
850−pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
851−pinnedCreate = fmap unsafeFreeze MA.pinnedCreate
852−
853−{-# DEPRECATED pinnedWrite "Please use pinnedCreate instead." #-}
545+-- | Like 'write' but creates a pinned array.
854546{-# INLINE pinnedWrite #-}
855547pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
856−pinnedWrite = pinnedCreate
548+pinnedWrite = fmap unsafeFreeze MA.pinnedWrite
857549
858550-- | Fold "step" has a dependency on "initial", and each step is dependent on
859551-- the previous invocation of step due to state passing, finally extract
@@ -875,20 +567,15 @@ pinnedWrite = pinnedCreate
875567-- could be unsafe and dangerous. This is dangerous especially when used with
876568-- foldMany like operations.
877569--
570+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
878571-- >>> unsafePureWrite = Array.unsafeMakePure Array.write
879572--
880573{-# INLINE unsafeMakePure #-}
881574unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b
882−unsafeMakePure (Fold step initial extract final) =
575+unsafeMakePure (Fold step initial extract) =
883576 Fold (\x a -> return $! unsafeInlineIO (step x a))
884577 (return $! unsafePerformIO initial)
885578 (\s -> return $! unsafeInlineIO $ extract s)
886− (\s -> return $! unsafeInlineIO $ final s)
887−
888−{-# INLINE fromPureStreamN #-}
889−fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a
890−fromPureStreamN n x =
891− unsafePerformIO $ fmap unsafeFreeze (MA.fromPureStreamN n x)
892579
893580-- | Convert a pure stream in Identity monad to an immutable array.
894581--
@@ -901,19 +588,6 @@ fromPureStream x = unsafePerformIO $ fmap (unsafeFreeze) (MA.fromPureStream x)
901588-- fromPureStream = runIdentity . D.fold (unsafeMakePure write)
902589-- fromPureStream = fromList . runIdentity . D.toList
903590
904−-- XXX This should be monadic.
905−
906−-- | Copy an immutable 'Ptr Word8' sequence into an array.
907−--
908−-- /Unsafe:/ The caller is responsible for safe addressing.
909−--
910−-- Note that this should be evaluated strictly to ensure that we do not hold
911−-- the reference to the pointer in a lazy thunk.
912−fromPtrN :: Int -> Ptr Word8 -> Array Word8
913−fromPtrN n addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromPtrN n addr)
914−
915−-- XXX This should be monadic.
916−
917591-- | Copy a null terminated immutable 'Addr#' Word8 sequence into an array.
918592--
919593-- /Unsafe:/ The caller is responsible for safe addressing.
@@ -921,49 +595,13 @@ fromPtrN n addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromPtrN n addr)
921595-- Note that this is completely safe when reading from Haskell string
922596-- literals because they are guaranteed to be NULL terminated:
923597--
598+-- >>> :set -XMagicHash
599+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
924600-- >>> Array.toList $ Array.fromByteStr# "\1\2\3\0"#
925601-- [1,2,3]
926602--
927−-- Note that this should be evaluated strictly to ensure that we do not hold
928−-- the reference to the pointer in a lazy thunk.
929603fromByteStr# :: Addr# -> Array Word8
930−fromByteStr# addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromByteStr# addr)
931−
932−-- XXX This should be monadic.
933−
934−-- | Note that this should be evaluated strictly to ensure that we do not hold
935−-- the reference to the pointer in a lazy thunk.
936−fromByteStr :: Ptr Word8 -> Array Word8
937−fromByteStr (Ptr addr#) = fromByteStr# addr#
938−
939−-- XXX implement fromChunks/fromChunkList instead?
940−
941−-- | Convert an array stream to an array. Note that this requires peak memory
942−-- that is double the size of the array stream.
943−--
944−{-# INLINE fromChunksK #-}
945−fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)
946−fromChunksK stream =
947− -- We buffer the entire stream and then allocate the target array of the
948− -- same size, thus requiring double the memory.
949− fmap unsafeFreeze $ MA.fromChunksK $ fmap unsafeThaw stream
950−
951−{-# DEPRECATED fromArrayStreamK "Please use fromChunksK instead." #-}
952−fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (Array a) -> m (Array a)
953−fromArrayStreamK = fromChunksK
954−
955−-- | Given a stream of arrays, splice them all together to generate a single
956−-- array. The stream must be /finite/.
957−--
958−{-# INLINE fromChunks #-}
959−fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
960−fromChunks s =
961− -- XXX Check which implementation is better
962− -- This may also require double the memory as we double the space every
963− -- time, when copying the last array we may have reallocated almost double
964− -- the space required before we right size it.
965− fmap unsafeFreeze $ MA.fromChunksRealloced (fmap unsafeThaw s)
966− -- fromChunkStreamK $ D.toStreamK s
604+fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
967605
968606-------------------------------------------------------------------------------
969607-- Instances
@@ -995,66 +633,12 @@ instance Unbox a => IsList (Array a) where
995633 {-# INLINE toList #-}
996634 toList = toList
997635
998−-- | Byte compare two arrays. Compare the length of the arrays. If the length
999−-- is equal, compare the lexicographical ordering of two underlying byte arrays
1000−-- otherwise return the result of length comparison.
1001−--
1002−-- /Unsafe/: Note that the 'Unbox' instance of sum types with constructors of
1003−-- different sizes may leave some memory uninitialized which can make byte
1004−-- comparison unreliable.
1005−--
1006−-- /Pre-release/
1007−{-# INLINE byteCmp #-}
1008−byteCmp :: Array a -> Array a -> Ordering
1009−byteCmp arr1 arr2 =
1010− -- unsafePerformIO?
1011− unsafeInlineIO $! unsafeThaw arr1 `MA.byteCmp` unsafeThaw arr2
1012−
1013−-- | Byte equality of two arrays.
1014−--
1015−-- >>> byteEq arr1 arr2 = (==) EQ $ Array.byteCmp arr1 arr2
1016−--
1017−-- /Unsafe/: See 'byteCmp'.
1018−{-# INLINE byteEq #-}
1019−byteEq :: Array a -> Array a -> Bool
1020−byteEq arr1 arr2 = (==) EQ $ byteCmp arr1 arr2
1021−
1022−#define MK_EQ_INSTANCE(typ) \
1023−instance {-# OVERLAPPING #-} Eq (Array typ) where { \
1024−; {-# INLINE (==) #-} \
1025−; (==) = byteEq \
1026−}
1027−
1028−MK_EQ_INSTANCE(Char)
1029−MK_EQ_INSTANCE(Word8)
1030−MK_EQ_INSTANCE(Word16)
1031−MK_EQ_INSTANCE(Word32)
1032−
1033−-- XXX The Word64 default instance should be as fast because we are comparing
1034−-- 64-bit at a time.
1035−MK_EQ_INSTANCE(Word64)
1036−MK_EQ_INSTANCE(Int)
1037−MK_EQ_INSTANCE(Int8)
1038−MK_EQ_INSTANCE(Int16)
1039−MK_EQ_INSTANCE(Int32)
1040−
1041−-- XXX The Int64 default instance should be as fast.
1042−MK_EQ_INSTANCE(Int64)
1043−
1044−-- | If the type allows a byte-by-byte comparison this instance can be
1045−-- overlapped by a more specific instance that uses 'byteCmp'. Byte comparison
1046−-- can be significantly faster.
1047−--
1048−instance {-# OVERLAPPABLE #-} (Unbox a, Eq a) => Eq (Array a) where
636+-- XXX we are assuming that Unboxed equality means element equality. This may
637+-- or may not be correct? arrcmp is 40% faster compared to stream equality.
638+instance (Unbox a, Eq a) => Eq (Array a) where
1049639 {-# INLINE (==) #-}
1050640 arr1 == arr2 =
1051− -- Does unboxed byte equality mean element equality?
1052− -- XXX This is incorrect for sum types, as we may have some
1053− -- uninitialized memory in that case. If we always initialize the
1054− -- unused memory to zero we can use this.
1055− -- Byte comparison is 40% faster compared to stream equality.
1056− -- (==) EQ $ unsafeInlineIO $! unsafeThaw arr1 `MA.cmp` unsafeThaw arr2
1057− (toStreamD arr1 :: Stream Identity a) == toStreamD arr2
641+ (==) EQ $ unsafeInlineIO $! unsafeThaw arr1 `MA.cmp` unsafeThaw arr2
1058642
1059643instance (Unbox a, Ord a) => Ord (Array a) where
1060644 {-# INLINE compare #-}
@@ -1096,7 +680,7 @@ _toStreamD_ size Array{..} = D.Stream step arrStart
1096680 {-# INLINE_LATE step #-}
1097681 step _ p | p == arrEnd = return D.Stop
1098682 step _ p = liftIO $ do
1099− x <- peekAt p arrContents
683+ x <- peekByteIndex p arrContents
1100684 return $ D.Yield x (p + size)
1101685
1102686{-
@@ -1119,29 +703,15 @@ instance Foldable Array where
1119703-- Semigroup and Monoid
1120704-------------------------------------------------------------------------------
1121705
1122−-- XXX Deprecate and remove the Semigroup and Monoid instances because of
1123−-- potential misuse chances.
1124−
1125−-- | This should not be used for combining many or N arrays as it would copy
1126−-- the two arrays everytime to a new array. For coalescing multiple arrays use
1127−-- 'fromChunksK' instead.
1128706instance Unbox a => Semigroup (Array a) where
1129707 arr1 <> arr2 = unsafePerformIO $ splice arr1 arr2
1130708
1131−empty ::
1132−#ifdef DEVBUILD
1133− Unbox a =>
1134−#endif
1135− Array a
1136−empty = Array Unboxed.empty 0 0
1137−
1138−{-# DEPRECATED nil "Please use empty instead." #-}
1139709nil ::
1140710#ifdef DEVBUILD
1141711 Unbox a =>
1142712#endif
1143713 Array a
1144−nil = empty
714+nil = Array Unboxed.nil 0 0
1145715
1146716instance Unbox a => Monoid (Array a) where
1147717 mempty = nil
core/src/Streamly/Internal/Data/Builder.hs+9−11
@@ -16,9 +16,7 @@ module Streamly.Internal.Data.Builder
1616 )
1717where
1818
19−#if !MIN_VERSION_base(4,18,0)
2019import Control.Applicative (liftA2)
21−#endif
2220
2321------------------------------------------------------------------------------
2422-- The Builder type
@@ -29,16 +27,16 @@ import Control.Applicative (liftA2)
2927-- or even a Fold. Unlike fold the step function is one-shot and not called in
3028-- a loop.
3129newtype Builder s m a =
32− Builder (s -> m (a, s))
30+ Builder (s -> m (s, a))
3331
3432-- | Maps a function on the output of the fold (the type @b@).
3533instance Functor m => Functor (Builder s m) where
3634 {-# INLINE fmap #-}
37− fmap f (Builder step1) = Builder (fmap (\ (a, s) -> (f a, s)) . step1)
35+ fmap f (Builder step1) = Builder (fmap (fmap f) . step1)
3836
3937{-# INLINE fromPure #-}
4038fromPure :: Applicative m => b -> Builder s m b
41−fromPure b = Builder (\s -> pure (b, s))
39+fromPure b = Builder (\s -> pure (s, b))
4240
4341-- | Chain the actions and zip the outputs.
4442{-# INLINE sequenceWith #-}
@@ -49,9 +47,9 @@ sequenceWith func (Builder stepL) (Builder stepR) = Builder step
4947 where
5048
5149 step s = do
52− (x, s1) <- stepL s
53− (y, s2) <- stepR s1
54− pure (func x y, s2)
50+ (s1, x) <- stepL s
51+ (s2, y) <- stepR s1
52+ pure (s2, func x y)
5553
5654instance Monad m => Applicative (Builder a m) where
5755 {-# INLINE pure #-}
@@ -76,7 +74,7 @@ instance Monad m => Monad (Builder a m) where
7674 where
7775
7876 step s = do
79− (x, s1) <- stepL s
77+ (s1, x) <- stepL s
8078 let Builder stepR = f x
81− (y, s2) <- stepR s1
82− pure (y, s2)
79+ (s2, y) <- stepR s1
80+ pure (s2, y)
core/src/Streamly/Internal/Data/Fold.hs+5−3
@@ -16,18 +16,20 @@ module Streamly.Internal.Data.Fold
1616 -- * Imports
1717 -- $setup
1818
19− module Streamly.Internal.Data.Fold.Type
19+ module Streamly.Internal.Data.Fold.Step
20+ , module Streamly.Internal.Data.Fold.Type
2021 , module Streamly.Internal.Data.Fold.Tee
2122 , module Streamly.Internal.Data.Fold.Combinators
2223 , module Streamly.Internal.Data.Fold.Container
23− , module Streamly.Internal.Data.Fold.Window
24+ -- , module Streamly.Internal.Data.Fold.Window
2425 )
2526where
2627
2728import Streamly.Internal.Data.Fold.Combinators
2829import Streamly.Internal.Data.Fold.Container
30+import Streamly.Internal.Data.Fold.Step
2931import Streamly.Internal.Data.Fold.Tee
3032import Streamly.Internal.Data.Fold.Type
31−import Streamly.Internal.Data.Fold.Window
33+-- import Streamly.Internal.Data.Fold.Window
3234
3335#include "DocTestDataFold.hs"
core/src/Streamly/Internal/Data/Fold/Chunked.hs+18−20
@@ -6,7 +6,7 @@
66-- Stability : experimental
77-- Portability : GHC
88--
9−-- Use "Streamly.Data.Parser" instead.
9+-- Use "Streamly.Data.Parser.Chunked" instead.
1010--
1111-- Fold a stream of foreign arrays. @Fold m a b@ in this module works
1212-- on a stream of "Array a" and produces an output of type @b@.
@@ -19,19 +19,18 @@
1919-- folds in Data.Fold to correctly work on an array stream as if it is an
2020-- element stream. For example:
2121--
22−-- >> import qualified Streamly.Data.Fold as Fold
23−-- >> import qualified Streamly.Internal.Data.Array.Stream as ArrayStream
24−-- >> import qualified Streamly.Internal.Data.Fold.Chunked as ChunkFold
25−-- >> import qualified Streamly.Data.Stream as Stream
26−-- >> import qualified Streamly.Data.StreamK as StreamK
22+-- >>> import qualified Streamly.Data.Fold as Fold
23+-- >>> import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
24+-- >>> import qualified Streamly.Internal.Data.Fold.Chunked as ChunkFold
25+-- >>> import qualified Streamly.Data.Stream as Stream
26+-- >>> import qualified Streamly.Data.StreamK as StreamK
2727--
28−-- >> f = ChunkFold.fromFold (Fold.take 7 Fold.toList)
29−-- >> s = Stream.chunksOf 5 $ Stream.fromList "hello world"
30−-- >> ArrayStream.runArrayFold f (StreamK.fromStream s)
28+-- >>> f = ChunkFold.fromFold (Fold.take 7 Fold.toList)
29+-- >>> s = Stream.chunksOf 5 $ Stream.fromList "hello world"
30+-- >>> ArrayStream.runArrayFold f (StreamK.fromStream s)
3131-- Right "hello w"
3232--
3333module Streamly.Internal.Data.Fold.Chunked
34− {-# DEPRECATED "Please use Streamly.Data.Parser instead." #-}
3534 (
3635 ChunkFold (..)
3736
@@ -59,22 +58,22 @@ where
5958
6059#include "ArrayMacros.h"
6160
62−#if !MIN_VERSION_base(4,18,0)
6361import Control.Applicative (liftA2)
64−#endif
6562import Control.Exception (assert)
6663import Control.Monad.IO.Class (MonadIO(..))
6764import Data.Bifunctor (first)
6865import Data.Proxy (Proxy(..))
6966import Streamly.Internal.Data.Unbox (Unbox(..))
7067import GHC.Types (SPEC(..))
71−import Streamly.Internal.Data.Array (Array(..))
68+import Streamly.Internal.Data.MutArray.Type (touch)
69+import Streamly.Internal.Data.Array.Type (Array(..))
7270import Streamly.Internal.Data.Parser (Initial(..), Step(..))
7371import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
7472
75−import qualified Streamly.Internal.Data.Array.Type as Array
73+import qualified Streamly.Internal.Data.Array as Array
7674import qualified Streamly.Internal.Data.Fold as Fold
7775import qualified Streamly.Internal.Data.Parser as ParserD
76+import qualified Streamly.Internal.Data.Parser.ParserD.Type as ParserD
7877import qualified Streamly.Internal.Data.Parser as Parser
7978
8079import Prelude hiding (concatMap, take)
@@ -103,8 +102,8 @@ newtype ChunkFold m a b = ChunkFold (ParserD.Parser (Array a) m b)
103102{-# INLINE fromFold #-}
104103fromFold :: forall m a b. (MonadIO m, Unbox a) =>
105104 Fold.Fold m a b -> ChunkFold m a b
106−fromFold (Fold.Fold fstep finitial _ ffinal) =
107− ChunkFold (ParserD.Parser step initial extract)
105+fromFold (Fold.Fold fstep finitial fextract) =
106+ ChunkFold (ParserD.Parser step initial (fmap (Done 0) . fextract))
108107
109108 where
110109
@@ -124,7 +123,7 @@ fromFold (Fold.Fold fstep finitial _ ffinal) =
124123 assert (cur == end) (return ())
125124 return $ Partial 0 fs
126125 goArray !_ !cur !fs = do
127− x <- liftIO $ peekAt cur contents
126+ x <- liftIO $ peekByteIndex cur contents
128127 res <- fstep fs x
129128 let elemSize = SIZE_OF(a)
130129 next = INDEX_NEXT(cur,a)
@@ -134,8 +133,6 @@ fromFold (Fold.Fold fstep finitial _ ffinal) =
134133 Fold.Partial fs1 ->
135134 goArray SPEC next fs1
136135
137− extract = fmap (Done 0) . ffinal
138−
139136-- | Convert an element 'ParserD.Parser' into an array stream fold. If the
140137-- parser fails the fold would throw an exception.
141138--
@@ -163,7 +160,8 @@ fromParserD (ParserD.Parser step1 initial1 extract1) =
163160 else return $ st (arrRem + n) fs1
164161
165162 goArray !_ !cur !fs = do
166− x <- liftIO $ peekAt cur contents
163+ x <- liftIO $ peekByteIndex cur contents
164+ liftIO $ touch contents
167165 res <- step1 fs x
168166 let elemSize = SIZE_OF(a)
169167 next = INDEX_NEXT(cur,a)
core/src/Streamly/Internal/Data/Fold/Combinators.hs+234−88
@@ -13,9 +13,34 @@
1313
1414module Streamly.Internal.Data.Fold.Combinators
1515 (
16+ -- * Imports
17+ -- $setup
18+
19+ -- * Fold Type
20+ Step (..)
21+ , Fold (..)
22+ , Tee (..)
23+
24+ -- * Constructors
25+ -- | Which constructor to use?
26+ --
27+ -- * @foldl*@: If the fold never terminates i.e. does not use the 'Done'
28+ -- constructor otherwise use the @foldt*@ variants.
29+ -- * @*M@: Use the @M@ suffix variants if any of the step, initial, or
30+ -- extract function is monadic, otherwise use the pure variants.
31+ --
32+ , foldl'
33+ , foldlM'
34+ , foldl1'
35+ , foldlM1'
36+ , foldt'
37+ , foldtM'
38+ , foldr'
39+ , foldrM'
40+
1641 -- * Mappers
1742 -- | Monadic functions useful with mapM/lmapM on folds or streams.
18− tracing
43+ , tracing
1944 , trace
2045
2146 -- * Folds
@@ -28,8 +53,11 @@ module Streamly.Internal.Data.Fold.Combinators
2853 , foldMapM
2954
3055 -- *** Reducers
56+ , drain
3157 , drainMapM
3258 , the
59+ , length
60+ , lengthGeneric
3361 , mean
3462 , rollingHash
3563 , defaultSalt
@@ -52,10 +80,13 @@ module Streamly.Internal.Data.Fold.Combinators
5280 -- | Avoid using these folds in scalable or performance critical
5381 -- applications, they buffer all the input in GC memory which can be
5482 -- detrimental to performance if the input is large.
83+ , toList
5584 , toListRev
5685 -- $toListRev
5786 , toStream
5887 , toStreamRev
88+ , toStreamK
89+ , toStreamKRev
5990 , topBy
6091 , top
6192 , bottomBy
@@ -75,6 +106,7 @@ module Streamly.Internal.Data.Fold.Combinators
75106
76107 -- *** Filters
77108 -- | Useful in combination with the 'scanMaybe' combinator.
109+ , filtering
78110 , deleteBy
79111 , uniqBy
80112 , uniq
@@ -82,6 +114,13 @@ module Streamly.Internal.Data.Fold.Combinators
82114 , findIndices
83115 , elemIndices
84116
117+ -- ** Terminating Folds
118+ -- *** Empty folds
119+ -- | Folds that return a result without consuming any input.
120+ , fromPure
121+ , fromEffect
122+ , fromRefold
123+
85124 -- *** Singleton folds
86125 -- | Folds that terminate after consuming exactly one input element. All
87126 -- these can be implemented in terms of the 'maybe' fold.
@@ -111,6 +150,8 @@ module Streamly.Internal.Data.Fold.Combinators
111150
112151 -- ** Trimmers
113152 -- | Useful in combination with the 'scanMaybe' combinator.
153+ , taking
154+ , dropping
114155 , takingEndByM
115156 , takingEndBy
116157 , takingEndByM_
@@ -124,14 +165,34 @@ module Streamly.Internal.Data.Fold.Combinators
124165 -- , breakStream
125166
126167 -- * Building Incrementally
168+ , extractM
169+ , reduce
170+ , close
171+ , isClosed
172+ , snoc
173+ , snocl
174+ , snocM
175+ , snoclM
176+
177+ , addOne
127178 , addStream
128179
129180 -- * Combinators
130181 -- ** Utilities
131182 , with
132183
184+ -- ** Transforming the Monad
185+ , morphInner
186+ , generalizeInner
187+
188+ -- ** Mapping on output
189+ , rmapM
190+
133191 -- ** Mapping on Input
134192 , transform
193+ , lmap
194+ --, lsequence
195+ , lmapM
135196
136197 -- ** Sliding Window
137198 , slide2
@@ -139,6 +200,7 @@ module Streamly.Internal.Data.Fold.Combinators
139200 -- ** Scanning Input
140201 , scan
141202 , scanMany
203+ , postscan
142204 , indexed
143205
144206 -- ** Zipping Input
@@ -146,10 +208,19 @@ module Streamly.Internal.Data.Fold.Combinators
146208 , zipStream
147209
148210 -- ** Filtering Input
211+ , catMaybes
149212 , mapMaybeM
150213 , mapMaybe
214+ , scanMaybe
215+ , filter
216+ , filterM
151217 , sampleFromthen
152218
219+ -- Either streams
220+ , catLefts
221+ , catRights
222+ , catEithers
223+
153224 {-
154225 -- ** Insertion
155226 -- | Insertion adds more elements to the stream.
@@ -162,8 +233,11 @@ module Streamly.Internal.Data.Fold.Combinators
162233 -}
163234
164235 -- ** Trimming
236+ , take
165237
166238 -- By elements
239+ , takeEndBy
240+ , takeEndBy_
167241 , takeEndBySeq
168242 , takeEndBySeq_
169243 {-
@@ -173,13 +247,18 @@ module Streamly.Internal.Data.Fold.Combinators
173247 -}
174248
175249 -- ** Serial Append
250+ , splitWith
251+ , split_
176252 -- , tail
177253 -- , init
178254 , splitAt -- spanN
179255 -- , splitIn -- sessionN
180256
181257 -- ** Parallel Distribution
258+ , teeWith
182259 , tee
260+ , teeWithFst
261+ , teeWithMin
183262 , distribute
184263 -- , distributeFst
185264 -- , distributeMin
@@ -192,6 +271,10 @@ module Streamly.Internal.Data.Fold.Combinators
192271 , unzipWithFstM
193272 , unzipWithMinM
194273
274+ -- ** Parallel Alternative
275+ , shortest
276+ , longest
277+
195278 -- ** Partitioning
196279 , partitionByM
197280 , partitionByFstM
@@ -200,14 +283,23 @@ module Streamly.Internal.Data.Fold.Combinators
200283 , partition
201284
202285 -- ** Splitting
286+ , many
287+ , manyPost
288+ , groupsOf
203289 , chunksBetween
290+ , refoldMany
291+ , refoldMany1
204292 , intersperseWithQuotes
205293
206294 -- ** Nesting
207295 , unfoldMany
208296 , concatSequence
297+ , concatMap
298+ , duplicate
299+ , refold
209300
210301 -- * Deprecated
302+ , foldr
211303 , drainBy
212304 , last
213305 , head
@@ -215,6 +307,7 @@ module Streamly.Internal.Data.Fold.Combinators
215307 , mapM
216308 , variance
217309 , stdDev
310+ , serialWith
218311 )
219312where
220313
@@ -236,15 +329,15 @@ import Streamly.Internal.Data.Pipe.Type (Pipe (..), PipeState(..))
236329import Streamly.Internal.Data.Unbox (Unbox, sizeOf)
237330import Streamly.Internal.Data.Unfold.Type (Unfold(..))
238331import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3'(..))
239−import Streamly.Internal.Data.Stream.Type (Stream)
332+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
240333
241334import qualified Prelude
242335import qualified Streamly.Internal.Data.MutArray.Type as MA
243336import qualified Streamly.Internal.Data.Array.Type as Array
244−import qualified Streamly.Internal.Data.Fold.Window as Fold
337+import qualified Streamly.Internal.Data.Fold.Window as FoldW
245338import qualified Streamly.Internal.Data.Pipe.Type as Pipe
246−import qualified Streamly.Internal.Data.Ring as Ring
247−import qualified Streamly.Internal.Data.Stream.Type as StreamD
339+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
340+import qualified Streamly.Internal.Data.Stream.StreamD.Type as StreamD
248341
249342import Prelude hiding
250343 ( Foldable(..), filter, drop, dropWhile, take, takeWhile, zipWith
@@ -254,6 +347,7 @@ import Prelude hiding
254347 , scanl, scanl1, replicate, concatMap, mconcat, unzip
255348 , span, splitAt, break, mapM, zip, maybe)
256349import Streamly.Internal.Data.Fold.Type
350+import Streamly.Internal.Data.Fold.Tee
257351
258352#include "DocTestDataFold.hs"
259353
@@ -439,8 +533,8 @@ trace f = lmapM (tracing f)
439533-- /Pre-release/
440534{-# INLINE transform #-}
441535transform :: Monad m => Pipe m a b -> Fold m b c -> Fold m a c
442−transform (Pipe pstep1 pstep2 pinitial) (Fold fstep finitial fextract ffinal) =
443− Fold step initial extract final
536+transform (Pipe pstep1 pstep2 pinitial) (Fold fstep finitial fextract) =
537+ Fold step initial extract
444538
445539 where
446540
@@ -473,14 +567,10 @@ transform (Pipe pstep1 pstep2 pinitial) (Fold fstep finitial fextract ffinal) =
473567
474568 extract (Tuple' _ fs) = fextract fs
475569
476− final (Tuple' _ fs) = ffinal fs
477−
478570{-# INLINE scanWith #-}
479571scanWith :: Monad m => Bool -> Fold m a b -> Fold m b c -> Fold m a c
480−scanWith isMany
481− (Fold stepL initialL extractL finalL)
482− (Fold stepR initialR extractR finalR) =
483− Fold step initial extract final
572+scanWith isMany (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
573+ Fold step initial extract
484574
485575 where
486576
@@ -494,14 +584,15 @@ scanWith isMany
494584 Partial sR1 ->
495585 if isMany
496586 then runStep initialL sR1
497− else Done <$> finalR sR1
587+ else Done <$> extractR sR1
498588 Done bR -> return $ Done bR
499589 Partial sL -> do
500590 !b <- extractL sL
501591 rR <- stepR sR b
502− case rR of
503− Partial sR1 -> return $ Partial (sL, sR1)
504− Done bR -> finalL sL >> return (Done bR)
592+ return
593+ $ case rR of
594+ Partial sR1 -> Partial (sL, sR1)
595+ Done bR -> Done bR
505596
506597 initial = do
507598 r <- initialR
@@ -513,8 +604,6 @@ scanWith isMany
513604
514605 extract = extractR . snd
515606
516− final (sL, sR) = finalL sL *> finalR sR
517−
518607-- | Scan the input of a 'Fold' to change it in a stateful manner using another
519608-- 'Fold'. The scan stops as soon as the fold terminates.
520609--
@@ -566,7 +655,7 @@ deleteBy eq x0 = fmap extract $ foldl' step (Tuple' False Nothing)
566655--
567656{-# INLINE slide2 #-}
568657slide2 :: Monad m => Fold m (a, Maybe a) b -> Fold m a b
569−slide2 (Fold step1 initial1 extract1 final1) = Fold step initial extract final
658+slide2 (Fold step1 initial1 extract1) = Fold step initial extract
570659
571660 where
572661
@@ -578,8 +667,6 @@ slide2 (Fold step1 initial1 extract1 final1) = Fold step initial extract final
578667
579668 extract (Tuple' _ s) = extract1 s
580669
581− final (Tuple' _ s) = final1 s
582−
583670-- | Return the latest unique element using the supplied comparison function.
584671-- Returns 'Nothing' if the current element is same as the last element
585672-- otherwise returns 'Just'.
@@ -705,11 +792,35 @@ the = foldt' step initial id
705792-- To Summary
706793------------------------------------------------------------------------------
707794
795+-- | Like 'length', except with a more general 'Num' return value
796+--
797+-- Definition:
798+--
799+-- >>> lengthGeneric = fmap getSum $ Fold.foldMap (Sum . const 1)
800+-- >>> lengthGeneric = Fold.foldl' (\n _ -> n + 1) 0
801+--
802+-- /Pre-release/
803+{-# INLINE lengthGeneric #-}
804+lengthGeneric :: (Monad m, Num b) => Fold m a b
805+lengthGeneric = foldl' (\n _ -> n + 1) 0
806+
807+-- | Determine the length of the input stream.
808+--
809+-- Definition:
810+--
811+-- >>> length = Fold.lengthGeneric
812+-- >>> length = fmap getSum $ Fold.foldMap (Sum . const 1)
813+--
814+{-# INLINE length #-}
815+length :: Monad m => Fold m a Int
816+length = lengthGeneric
817+
818+
708819-- | Determine the sum of all elements of a stream of numbers. Returns additive
709820-- identity (@0@) when the stream is empty. Note that this is not numerically
710821-- stable for floating point numbers.
711822--
712−-- >>> sum = Fold.cumulative Fold.windowSum
823+-- >>> sum = FoldW.cumulative FoldW.sum
713824--
714825-- Same as following but numerically stable:
715826--
@@ -718,7 +829,7 @@ the = foldt' step initial id
718829--
719830{-# INLINE sum #-}
720831sum :: (Monad m, Num a) => Fold m a a
721−sum = Fold.cumulative Fold.windowSum
832+sum = FoldW.cumulative FoldW.sum
722833
723834-- | Determine the product of all elements of a stream of numbers. Returns
724835-- multiplicative identity (@1@) when the stream is empty. The fold terminates
@@ -911,7 +1022,7 @@ rollingHashFirstN n = take n rollingHash
9111022--
9121023{-# INLINE rollingMapM #-}
9131024rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Fold m a b
914−rollingMapM f = Fold step initial extract extract
1025+rollingMapM f = Fold step initial extract
9151026
9161027 where
9171028
@@ -1152,8 +1263,7 @@ head = one
11521263-- /Pre-release/
11531264{-# INLINE findM #-}
11541265findM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1155−findM predicate =
1156− Fold step (return $ Partial ()) extract extract
1266+findM predicate = Fold step (return $ Partial ()) (const $ return Nothing)
11571267
11581268 where
11591269
@@ -1164,8 +1274,6 @@ findM predicate =
11641274 else Partial ()
11651275 in f <$> predicate a
11661276
1167− extract = const $ return Nothing
1168−
11691277-- | Returns the first element that satisfies the given predicate.
11701278--
11711279{-# INLINE find #-}
@@ -1408,7 +1516,7 @@ splitAt n fld = splitWith (,) (take n fld)
14081516
14091517{-# INLINE takingEndByM #-}
14101518takingEndByM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1411−takingEndByM p = Fold step initial extract extract
1519+takingEndByM p = Fold step initial (return . toMaybe)
14121520
14131521 where
14141522
@@ -1421,8 +1529,6 @@ takingEndByM p = Fold step initial extract extract
14211529 then Done $ Just a
14221530 else Partial $ Just' a
14231531
1424− extract = return . toMaybe
1425−
14261532-- |
14271533--
14281534-- >>> takingEndBy p = Fold.takingEndByM (return . p)
@@ -1433,7 +1539,7 @@ takingEndBy p = takingEndByM (return . p)
14331539
14341540{-# INLINE takingEndByM_ #-}
14351541takingEndByM_ :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1436−takingEndByM_ p = Fold step initial extract extract
1542+takingEndByM_ p = Fold step initial (return . toMaybe)
14371543
14381544 where
14391545
@@ -1446,8 +1552,6 @@ takingEndByM_ p = Fold step initial extract extract
14461552 then Done Nothing
14471553 else Partial $ Just' a
14481554
1449− extract = return . toMaybe
1450−
14511555-- |
14521556--
14531557-- >>> takingEndBy_ p = Fold.takingEndByM_ (return . p)
@@ -1458,7 +1562,7 @@ takingEndBy_ p = takingEndByM_ (return . p)
14581562
14591563{-# INLINE droppingWhileM #-}
14601564droppingWhileM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1461−droppingWhileM p = Fold step initial extract extract
1565+droppingWhileM p = Fold step initial (return . toMaybe)
14621566
14631567 where
14641568
@@ -1473,8 +1577,6 @@ droppingWhileM p = Fold step initial extract extract
14731577 else Just' a
14741578 step _ a = return $ Partial $ Just' a
14751579
1476− extract = return . toMaybe
1477−
14781580-- |
14791581-- >>> droppingWhile p = Fold.droppingWhileM (return . p)
14801582--
@@ -1482,6 +1584,70 @@ droppingWhileM p = Fold step initial extract extract
14821584droppingWhile :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
14831585droppingWhile p = droppingWhileM (return . p)
14841586
1587+-- Note: Keep this consistent with S.splitOn. In fact we should eliminate
1588+-- S.splitOn in favor of the fold.
1589+--
1590+-- XXX Use Fold.many instead once it is fixed.
1591+-- > Stream.splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
1592+
1593+-- | Like 'takeEndBy' but drops the element on which the predicate succeeds.
1594+--
1595+-- Example:
1596+--
1597+-- >>> input = Stream.fromList "hello\nthere\n"
1598+-- >>> line = Fold.takeEndBy_ (== '\n') Fold.toList
1599+-- >>> Stream.fold line input
1600+-- "hello"
1601+--
1602+-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1603+-- ["hello","there"]
1604+--
1605+{-# INLINE takeEndBy_ #-}
1606+takeEndBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1607+-- takeEndBy_ predicate = scanMaybe (takingEndBy_ predicate)
1608+takeEndBy_ predicate (Fold fstep finitial fextract) =
1609+ Fold step finitial fextract
1610+
1611+ where
1612+
1613+ step s a =
1614+ if not (predicate a)
1615+ then fstep s a
1616+ else Done <$> fextract s
1617+
1618+-- Note:
1619+-- > Stream.splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
1620+
1621+-- | Take the input, stop when the predicate succeeds taking the succeeding
1622+-- element as well.
1623+--
1624+-- Example:
1625+--
1626+-- >>> input = Stream.fromList "hello\nthere\n"
1627+-- >>> line = Fold.takeEndBy (== '\n') Fold.toList
1628+-- >>> Stream.fold line input
1629+-- "hello\n"
1630+--
1631+-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1632+-- ["hello\n","there\n"]
1633+--
1634+{-# INLINE takeEndBy #-}
1635+takeEndBy :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1636+-- takeEndBy predicate = scanMaybe (takingEndBy predicate)
1637+takeEndBy predicate (Fold fstep finitial fextract) =
1638+ Fold step finitial fextract
1639+
1640+ where
1641+
1642+ step s a = do
1643+ res <- fstep s a
1644+ if not (predicate a)
1645+ then return res
1646+ else do
1647+ case res of
1648+ Partial s1 -> Done <$> fextract s1
1649+ Done b -> return $ Done b
1650+
14851651------------------------------------------------------------------------------
14861652-- Binary splitting on a separator
14871653------------------------------------------------------------------------------
@@ -1514,8 +1680,8 @@ takeEndBySeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
15141680 Array.Array a
15151681 -> Fold m a b
15161682 -> Fold m a b
1517−takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
1518− Fold step initial extract final
1683+takeEndBySeq patArr (Fold fstep finitial fextract) =
1684+ Fold step initial extract
15191685
15201686 where
15211687
@@ -1528,7 +1694,7 @@ takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
15281694 | patLen == 0 ->
15291695 -- XXX Should we match nothing or everything on empty
15301696 -- pattern?
1531− -- Done <$> ffinal acc
1697+ -- Done <$> fextract acc
15321698 return $ Partial $ SplitOnSeqEmpty acc
15331699 | patLen == 1 -> do
15341700 pat <- liftIO $ Array.unsafeIndexIO 0 patArr
@@ -1579,7 +1745,7 @@ takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
15791745 case res of
15801746 Partial s1
15811747 | pat /= x -> return $ Partial $ SplitOnSeqSingle s1 pat
1582− | otherwise -> Done <$> ffinal s1
1748+ | otherwise -> Done <$> fextract s1
15831749 Done b -> return $ Done b
15841750 step (SplitOnSeqWord s idx wrd) x = do
15851751 res <- fstep s x
@@ -1588,7 +1754,7 @@ takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
15881754 Partial s1
15891755 | idx == maxIndex -> do
15901756 if wrd1 .&. wordMask == wordPat
1591− then Done <$> ffinal s1
1757+ then Done <$> fextract s1
15921758 else return $ Partial $ SplitOnSeqWordLoop s1 wrd1
15931759 | otherwise ->
15941760 return $ Partial $ SplitOnSeqWord s1 (idx + 1) wrd1
@@ -1599,7 +1765,7 @@ takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
15991765 case res of
16001766 Partial s1
16011767 | wrd1 .&. wordMask == wordPat ->
1602− Done <$> ffinal s1
1768+ Done <$> fextract s1
16031769 | otherwise ->
16041770 return $ Partial $ SplitOnSeqWordLoop s1 wrd1
16051771 Done b -> return $ Done b
@@ -1613,7 +1779,7 @@ takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
16131779 let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
16141780 let !ringHash = fld addCksum 0 rb
16151781 if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1616− then Done <$> ffinal s1
1782+ then Done <$> fextract s1
16171783 else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
16181784 else
16191785 return $ Partial $ SplitOnSeqKR s1 (idx + 1) rb rh1
@@ -1626,11 +1792,11 @@ takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
16261792 rh1 <- liftIO $ Ring.unsafeInsert rb rh x
16271793 let ringHash = deltaCksum cksum old x
16281794 if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1629− then Done <$> ffinal s1
1795+ then Done <$> fextract s1
16301796 else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
16311797 Done b -> return $ Done b
16321798
1633− extractFunc fex state =
1799+ extract state =
16341800 let st =
16351801 case state of
16361802 SplitOnSeqEmpty s -> s
@@ -1639,11 +1805,7 @@ takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
16391805 SplitOnSeqWordLoop s _ -> s
16401806 SplitOnSeqKR s _ _ _ -> s
16411807 SplitOnSeqKRLoop s _ _ _ -> s
1642− in fex st
1643−
1644− extract state = extractFunc fextract state
1645−
1646− final state = extractFunc ffinal state
1808+ in fextract st
16471809
16481810-- | Like 'takeEndBySeq' but discards the matched sequence.
16491811--
@@ -1654,8 +1816,8 @@ takeEndBySeq_ :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
16541816 Array.Array a
16551817 -> Fold m a b
16561818 -> Fold m a b
1657−takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
1658− Fold step initial extract final
1819+takeEndBySeq_ patArr (Fold fstep finitial fextract) =
1820+ Fold step initial extract
16591821
16601822 where
16611823
@@ -1668,7 +1830,7 @@ takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
16681830 | patLen == 0 ->
16691831 -- XXX Should we match nothing or everything on empty
16701832 -- pattern?
1671− -- Done <$> ffinal acc
1833+ -- Done <$> fextract acc
16721834 return $ Partial $ SplitOnSeqEmpty acc
16731835 | patLen == 1 -> do
16741836 pat <- liftIO $ Array.unsafeIndexIO 0 patArr
@@ -1725,13 +1887,13 @@ takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
17251887 case res of
17261888 Partial s1 -> return $ Partial $ SplitOnSeqSingle s1 pat
17271889 Done b -> return $ Done b
1728− else Done <$> ffinal s
1890+ else Done <$> fextract s
17291891 step (SplitOnSeqWord s idx wrd) x = do
17301892 let wrd1 = addToWord wrd x
17311893 if idx == maxIndex
17321894 then do
17331895 if wrd1 .&. wordMask == wordPat
1734− then Done <$> ffinal s
1896+ then Done <$> fextract s
17351897 else return $ Partial $ SplitOnSeqWordLoop s wrd1
17361898 else return $ Partial $ SplitOnSeqWord s (idx + 1) wrd1
17371899 step (SplitOnSeqWordLoop s wrd) x = do
@@ -1742,7 +1904,7 @@ takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
17421904 case res of
17431905 Partial s1
17441906 | wrd1 .&. wordMask == wordPat ->
1745− Done <$> ffinal s1
1907+ Done <$> fextract s1
17461908 | otherwise ->
17471909 return $ Partial $ SplitOnSeqWordLoop s1 wrd1
17481910 Done b -> return $ Done b
@@ -1753,7 +1915,7 @@ takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
17531915 let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
17541916 let !ringHash = fld addCksum 0 rb
17551917 if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1756− then Done <$> ffinal s
1918+ then Done <$> fextract s
17571919 else return $ Partial $ SplitOnSeqKRLoop s ringHash rb rh1
17581920 else return $ Partial $ SplitOnSeqKR s (idx + 1) rb rh1
17591921 step (SplitOnSeqKRLoop s cksum rb rh) x = do
@@ -1764,7 +1926,7 @@ takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
17641926 rh1 <- liftIO $ Ring.unsafeInsert rb rh x
17651927 let ringHash = deltaCksum cksum old x
17661928 if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1767− then Done <$> ffinal s1
1929+ then Done <$> fextract s1
17681930 else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
17691931 Done b -> return $ Done b
17701932
@@ -1772,10 +1934,10 @@ takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
17721934 -- terminates early inside extract, we may still have buffered data
17731935 -- remaining which will be lost if we do not communicate that to the
17741936 -- driver.
1775− extractFunc fex state = do
1937+ extract state = do
17761938 let consumeWord s n wrd = do
17771939 if n == 0
1778− then fex s
1940+ then fextract s
17791941 else do
17801942 let old = elemMask .&. (wrd `shiftR` (elemBits * (n - 1)))
17811943 r <- fstep s (toEnum $ fromIntegral old)
@@ -1785,7 +1947,7 @@ takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
17851947
17861948 let consumeRing s n rb rh =
17871949 if n == 0
1788− then fex s
1950+ then fextract s
17891951 else do
17901952 old <- liftIO $ peek rh
17911953 let rh1 = Ring.advance rb rh
@@ -1795,17 +1957,13 @@ takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
17951957 Done b -> return b
17961958
17971959 case state of
1798− SplitOnSeqEmpty s -> fex s
1799− SplitOnSeqSingle s _ -> fex s
1960+ SplitOnSeqEmpty s -> fextract s
1961+ SplitOnSeqSingle s _ -> fextract s
18001962 SplitOnSeqWord s idx wrd -> consumeWord s idx wrd
18011963 SplitOnSeqWordLoop s wrd -> consumeWord s patLen wrd
18021964 SplitOnSeqKR s idx rb _ -> consumeRing s idx rb (Ring.startOf rb)
18031965 SplitOnSeqKRLoop s _ rb rh -> consumeRing s patLen rb rh
18041966
1805− extract state = extractFunc fextract state
1806−
1807− final state = extractFunc ffinal state
1808−
18091967------------------------------------------------------------------------------
18101968-- Distributing
18111969------------------------------------------------------------------------------
@@ -2249,8 +2407,8 @@ toStreamRev = fmap StreamD.fromList toListRev
22492407-- /Pre-release/
22502408{-# INLINE unfoldMany #-}
22512409unfoldMany :: Monad m => Unfold m a b -> Fold m b c -> Fold m a c
2252−unfoldMany (Unfold ustep inject) (Fold fstep initial extract final) =
2253− Fold consume initial extract final
2410+unfoldMany (Unfold ustep inject) (Fold fstep initial extract) =
2411+ Fold consume initial extract
22542412
22552413 where
22562414
@@ -2277,7 +2435,7 @@ bottomBy :: (MonadIO m, Unbox a) =>
22772435 (a -> a -> Ordering)
22782436 -> Int
22792437 -> Fold m a (MutArray a)
2280−bottomBy cmp n = Fold step initial extract extract
2438+bottomBy cmp n = Fold step initial extract
22812439
22822440 where
22832441
@@ -2372,8 +2530,8 @@ intersperseWithQuotes
23722530 quote
23732531 esc
23742532 separator
2375− (Fold stepL initialL _ finalL)
2376− (Fold stepR initialR extractR finalR) = Fold step initial extract final
2533+ (Fold stepL initialL extractL)
2534+ (Fold stepR initialR extractR) = Fold step initial extract
23772535
23782536 where
23792537
@@ -2416,13 +2574,11 @@ intersperseWithQuotes
24162574 r <- stepL sL a
24172575 case r of
24182576 Partial s -> return $ Partial (nextState sR s)
2419− Done _ -> do
2420− _ <- finalR sR
2421− error "Collecting fold finished inside quote"
2577+ Done _ -> error "Collecting fold finished inside quote"
24222578
24232579 step (IntersperseQUnquoted sR sL) a
24242580 | a == separator = do
2425− b <- finalL sL
2581+ b <- extractL sL
24262582 collect IntersperseQUnquoted sR b
24272583 | a == quote = processQuoted a sL sR IntersperseQQuoted
24282584 | otherwise = process a sL sR IntersperseQUnquoted
@@ -2440,13 +2596,3 @@ intersperseWithQuotes
24402596 error "intersperseWithQuotes: finished inside quote"
24412597 extract (IntersperseQQuotedEsc _ _) =
24422598 error "intersperseWithQuotes: finished inside quote, at escape char"
2443−
2444− final (IntersperseQUnquoted sR sL) = finalL sL *> finalR sR
2445− final (IntersperseQQuoted sR sL) = do
2446− _ <- finalR sR
2447− _ <- finalL sL
2448− error "intersperseWithQuotes: finished inside quote"
2449− final (IntersperseQQuotedEsc sR sL) = do
2450− _ <- finalR sR
2451− _ <- finalL sL
2452− error "intersperseWithQuotes: finished inside quote, at escape char"
core/src/Streamly/Internal/Data/Fold/Container.hs+31−73
@@ -9,6 +9,9 @@
99
1010module Streamly.Internal.Data.Fold.Container
1111 (
12+ -- * Imports
13+ -- $setup
14+
1215 -- * Set operations
1316 toSet
1417 , toIntSet
@@ -270,15 +273,14 @@ demuxGeneric :: (Monad m, IsMap f, Traversable f) =>
270273 (a -> Key f)
271274 -> (a -> m (Fold m a b))
272275 -> Fold m a (m (f b), Maybe (Key f, b))
273−demuxGeneric getKey getFold =
274− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
276+demuxGeneric getKey getFold = fmap extract $ foldlM' step initial
275277
276278 where
277279
278280 initial = return $ Tuple' IsMap.mapEmpty Nothing
279281
280282 {-# INLINE runFold #-}
281− runFold kv (Fold step1 initial1 extract1 final1) (k, a) = do
283+ runFold kv (Fold step1 initial1 extract1) (k, a) = do
282284 res <- initial1
283285 case res of
284286 Partial s -> do
@@ -286,7 +288,7 @@ demuxGeneric getKey getFold =
286288 return
287289 $ case res1 of
288290 Partial _ ->
289− let fld = Fold step1 (return res1) extract1 final1
291+ let fld = Fold step1 (return res1) extract1
290292 in Tuple' (IsMap.mapInsert k fld kv) Nothing
291293 Done b -> Tuple' (IsMap.mapDelete k kv) (Just (k, b))
292294 Done b -> return $ Tuple' kv (Just (k, b))
@@ -299,25 +301,15 @@ demuxGeneric getKey getFold =
299301 runFold kv fld (k, a)
300302 Just f -> runFold kv f (k, a)
301303
302− extract (Tuple' kv x) = return (Prelude.mapM f kv, x)
304+ extract (Tuple' kv x) = (Prelude.mapM f kv, x)
303305
304306 where
305307
306− f (Fold _ i e _) = do
308+ f (Fold _ i e) = do
307309 r <- i
308310 case r of
309311 Partial s -> e s
310− _ -> error "demuxGeneric: unreachable code"
311−
312− final (Tuple' kv x) = return (Prelude.mapM f kv, x)
313−
314− where
315−
316− f (Fold _ i _ fin) = do
317− r <- i
318− case r of
319− Partial s -> fin s
320− _ -> error "demuxGeneric: unreachable code"
312+ Done b -> return b
321313
322314-- | @demux getKey getFold@: In a key value stream, fold values corresponding
323315-- to each key using a key specific fold. @getFold@ is invoked to generate a
@@ -359,36 +351,35 @@ demuxGenericIO :: (MonadIO m, IsMap f, Traversable f) =>
359351 (a -> Key f)
360352 -> (a -> m (Fold m a b))
361353 -> Fold m a (m (f b), Maybe (Key f, b))
362−demuxGenericIO getKey getFold =
363− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
354+demuxGenericIO getKey getFold = fmap extract $ foldlM' step initial
364355
365356 where
366357
367358 initial = return $ Tuple' IsMap.mapEmpty Nothing
368359
369360 {-# INLINE initFold #-}
370− initFold kv (Fold step1 initial1 extract1 final1) (k, a) = do
361+ initFold kv (Fold step1 initial1 extract1) (k, a) = do
371362 res <- initial1
372363 case res of
373364 Partial s -> do
374365 res1 <- step1 s a
375366 case res1 of
376367 Partial _ -> do
377− let fld = Fold step1 (return res1) extract1 final1
368+ let fld = Fold step1 (return res1) extract1
378369 ref <- liftIO $ newIORef fld
379370 return $ Tuple' (IsMap.mapInsert k ref kv) Nothing
380371 Done b -> return $ Tuple' kv (Just (k, b))
381372 Done b -> return $ Tuple' kv (Just (k, b))
382373
383374 {-# INLINE runFold #-}
384− runFold kv ref (Fold step1 initial1 extract1 final1) (k, a) = do
375+ runFold kv ref (Fold step1 initial1 extract1) (k, a) = do
385376 res <- initial1
386377 case res of
387378 Partial s -> do
388379 res1 <- step1 s a
389380 case res1 of
390381 Partial _ -> do
391− let fld = Fold step1 (return res1) extract1 final1
382+ let fld = Fold step1 (return res1) extract1
392383 liftIO $ writeIORef ref fld
393384 return $ Tuple' kv Nothing
394385 Done b ->
@@ -406,27 +397,16 @@ demuxGenericIO getKey getFold =
406397 f <- liftIO $ readIORef ref
407398 runFold kv ref f (k, a)
408399
409− extract (Tuple' kv x) = return (Prelude.mapM f kv, x)
400+ extract (Tuple' kv x) = (Prelude.mapM f kv, x)
410401
411402 where
412403
413404 f ref = do
414− Fold _ i e _ <- liftIO $ readIORef ref
405+ (Fold _ i e) <- liftIO $ readIORef ref
415406 r <- i
416407 case r of
417408 Partial s -> e s
418− _ -> error "demuxGenericIO: unreachable code"
419−
420− final (Tuple' kv x) = return (Prelude.mapM f kv, x)
421−
422− where
423−
424− f ref = do
425− Fold _ i _ fin <- liftIO $ readIORef ref
426− r <- i
427− case r of
428− Partial s -> fin s
429− _ -> error "demuxGenericIO: unreachable code"
409+ Done b -> return b
430410
431411-- | This is specialized version of 'demux' that uses mutable IO cells as
432412-- fold accumulators for better performance.
@@ -441,13 +421,6 @@ demuxIO :: (MonadIO m, Ord k) =>
441421 -> Fold m a (m (Map k b), Maybe (k, b))
442422demuxIO = demuxGenericIO
443423
444−-- | Fold a key value stream to a key-value Map. If the same key appears
445−-- multiple times, only the last value is retained.
446−{-# INLINE kvToMapOverwriteGeneric #-}
447−kvToMapOverwriteGeneric :: (Monad m, IsMap f) => Fold m (Key f, a) (f a)
448−kvToMapOverwriteGeneric =
449− foldl' (\kv (k, v) -> IsMap.mapInsert k v kv) IsMap.mapEmpty
450−
451424{-# INLINE demuxToContainer #-}
452425demuxToContainer :: (Monad m, IsMap f, Traversable f) =>
453426 (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
@@ -535,8 +508,8 @@ classifyGeneric :: (Monad m, IsMap f, Traversable f, Ord (Key f)) =>
535508 -- for that use case. We return an action because we want it to be lazy so
536509 -- that the downstream consumers can choose to process or discard it.
537510 (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
538−classifyGeneric f (Fold step1 initial1 extract1 final1) =
539− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
511+classifyGeneric f (Fold step1 initial1 extract1) =
512+ fmap extract $ foldlM' step initial
540513
541514 where
542515
@@ -573,16 +546,7 @@ classifyGeneric f (Fold step1 initial1 extract1 final1) =
573546 let kv1 = IsMap.mapDelete k kv
574547 in Tuple3' kv1 (Set.insert k set) (Just (k, b))
575548
576− extract (Tuple3' kv _ x) = return (Prelude.mapM extract1 kv, x)
577−
578− final (Tuple3' kv set x) = return (IsMap.mapTraverseWithKey f1 kv, x)
579−
580− where
581−
582− f1 k s = do
583− if Set.member k set
584− then extract1 s
585− else final1 s
549+ extract (Tuple3' kv _ x) = (Prelude.mapM extract1 kv, x)
586550
587551-- | Folds the values for each key using the supplied fold. When scanning, as
588552-- soon as the fold is complete, its result is available in the second
@@ -607,8 +571,8 @@ classify = classifyGeneric
607571{-# INLINE classifyGenericIO #-}
608572classifyGenericIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) =>
609573 (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
610−classifyGenericIO f (Fold step1 initial1 extract1 final1) =
611− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
574+classifyGenericIO f (Fold step1 initial1 extract1) =
575+ fmap extract $ foldlM' step initial
612576
613577 where
614578
@@ -647,21 +611,8 @@ classifyGenericIO f (Fold step1 initial1 extract1 final1) =
647611 in return
648612 $ Tuple3' kv1 (Set.insert k set) (Just (k, b))
649613
650− extract (Tuple3' kv _ x) = return (Prelude.mapM g kv, x)
651−
652− where
653−
654− g ref = liftIO (readIORef ref) >>= extract1
655−
656− final (Tuple3' kv set x) = return (IsMap.mapTraverseWithKey g kv, x)
657−
658− where
659−
660− g k ref = do
661− s <- liftIO $ readIORef ref
662− if Set.member k set
663− then extract1 s
664− else final1 s
614+ extract (Tuple3' kv _ x) =
615+ (Prelude.mapM (\ref -> liftIO (readIORef ref) >>= extract1) kv, x)
665616
666617-- | Same as classify except that it uses mutable IORef cells in the
667618-- Map providing better performance. Be aware that if this is used as a scan,
@@ -676,6 +627,13 @@ classifyIO :: (MonadIO m, Ord k) =>
676627 (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
677628classifyIO = classifyGenericIO
678629
630+-- | Fold a key value stream to a key-value Map. If the same key appears
631+-- multiple times, only the last value is retained.
632+{-# INLINE kvToMapOverwriteGeneric #-}
633+kvToMapOverwriteGeneric :: (Monad m, IsMap f) => Fold m (Key f, a) (f a)
634+kvToMapOverwriteGeneric =
635+ foldl' (\kv (k, v) -> IsMap.mapInsert k v kv) IsMap.mapEmpty
636+
679637{-# INLINE toContainer #-}
680638toContainer :: (Monad m, IsMap f, Traversable f, Ord (Key f)) =>
681639 (a -> Key f) -> Fold m a b -> Fold m a (f b)
core/src/Streamly/Internal/Data/Fold/Step.hs+2−2
@@ -8,7 +8,7 @@
88--
99module Streamly.Internal.Data.Fold.Step
1010 (
11− -- * Step Type
11+ -- * Types
1212 Step (..)
1313
1414 , mapMStep
@@ -40,7 +40,7 @@ data Step s b
4040 = Partial !s
4141 | Done !b
4242
43−-- | 'first' maps over the fold state and 'second' maps over the fold result.
43+-- | 'first' maps over 'Partial' and 'second' maps over 'Done'.
4444--
4545instance Bifunctor Step where
4646 {-# INLINE bimap #-}
core/src/Streamly/Internal/Data/Fold/Tee.hs−2
@@ -16,9 +16,7 @@ module Streamly.Internal.Data.Fold.Tee
1616 )
1717where
1818
19−#if !MIN_VERSION_base(4,18,0)
2019import Control.Applicative (liftA2)
21−#endif
2220import Streamly.Internal.Data.Fold.Type (Fold)
2321
2422import qualified Streamly.Internal.Data.Fold.Type as Fold
core/src/Streamly/Internal/Data/Fold/Type.hs+120−344
@@ -344,9 +344,11 @@
344344--
345345module Streamly.Internal.Data.Fold.Type
346346 (
347− module Streamly.Internal.Data.Fold.Step
347+ -- * Imports
348+ -- $setup
348349
349− -- * Fold Type
350+ -- * Types
351+ Step (..)
350352 , Fold (..)
351353
352354 -- * Constructors
@@ -367,8 +369,6 @@ module Streamly.Internal.Data.Fold.Type
367369 , toList
368370 , toStreamK
369371 , toStreamKRev
370− , lengthGeneric
371− , length
372372
373373 -- * Combinators
374374
@@ -393,8 +393,6 @@ module Streamly.Internal.Data.Fold.Type
393393 -- ** Trimming
394394 , take
395395 , taking
396− , takeEndBy_
397− , takeEndBy
398396 , dropping
399397
400398 -- ** Sequential application
@@ -446,25 +444,21 @@ where
446444
447445#include "inline.hs"
448446
449−#if !MIN_VERSION_base(4,18,0)
450447import Control.Applicative (liftA2)
451−#endif
452−import Control.Monad ((>=>), void)
448+import Control.Monad ((>=>))
453449import Data.Bifunctor (Bifunctor(..))
454450import Data.Either (fromLeft, fromRight, isLeft, isRight)
455451import Data.Functor.Identity (Identity(..))
456452import Fusion.Plugin.Types (Fuse(..))
453+import Streamly.Internal.Data.Fold.Step (Step(..), mapMStep, chainStepM)
457454import Streamly.Internal.Data.Maybe.Strict (Maybe'(..), toMaybe)
458455import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
459456import Streamly.Internal.Data.Refold.Type (Refold(..))
460457
461−import qualified Streamly.Internal.Data.StreamK.Type as K
458+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
462459
463460import Prelude hiding (Foldable(..), concatMap, filter, map, take)
464461
465−-- Entire module is exported, do not import selectively
466−import Streamly.Internal.Data.Fold.Step
467−
468462#include "DocTestDataFold.hs"
469463
470464------------------------------------------------------------------------------
@@ -475,64 +469,23 @@ import Streamly.Internal.Data.Fold.Step
475469-- The type @b@ is the accumulator of the writer. That's the reason the
476470-- default folds in various modules are called "write".
477471
478−-- An alternative to using an "extract" function is to use "Partial s b" style
479−-- partial value so that we always emit the output value and there is no need
480−-- to extract. Then extract can be used for cleanup purposes. But in this case
481−-- in some cases we may need a "Continue" constructor where an output value is
482−-- not available, this was implicit earlier. Also, "b" should be lazy here so
483−-- that we do not always compute it even if we do not need it.
484−--
485−-- Partial s b --> extract :: s -> b
486−-- Continue --> extract :: s -> Maybe b
487−--
488−-- But keeping 'b' lazy does not let the fold optimize well. It leads to
489−-- significant regressions in the key-value folds.
490−--
491−-- The "final" function complicates combinators that take other folds as
492−-- argument because we need to call their finalizers at right places. An
493−-- alternative to reduce this complexity where it is not required is to use a
494−-- separate type for bracketed folds but then we need to manage the complexity
495−-- of two different fold types.
496−
497−-- The "final" function could be (s -> m (Step s b)), like in parsers so that
498−-- it can be called in a loop to drain the fold.
499−
500−-- | The type @Fold m a b@ represents a consumer of an input stream of values
501−-- of type @a@ and returning a final value of type @b@ in 'Monad' @m@. The
502−-- constructor of a fold is @Fold step initial extract final@.
503−--
504−-- The fold uses an internal state of type @s@. The initial value of the state
505−-- @s@ is created by @initial@. This function is called once and only once
506−-- before the fold starts consuming input. Any resource allocation can be done
507−-- in this function.
508−--
509−-- The @step@ function is called on each input, it consumes an input and
510−-- returns the next intermediate state (see 'Step') or the final result @b@ if
511−-- the fold terminates.
512−--
513−-- If the fold is used as a scan, the @extract@ function is used by the scan
514−-- driver to map the current state @s@ of the fold to the fold result. Thus
515−-- @extract@ can be called multiple times. In some folds, where scanning does
516−-- not make sense, this function is left unimplemented; such folds cannot be
517−-- used as scans.
518−--
519−-- Before a fold terminates, @final@ is called once and only once (unless the
520−-- fold terminated in @initial@ itself). Any resources allocated by @initial@
521−-- can be released in @final@. In folds that do not require any cleanup
522−-- @extract@ and @final@ are typically the same.
523−--
524−-- When implementing fold combinators, care should be taken to cleanup any
525−-- state of the argument folds held by the fold by calling the respective
526−-- @final@ at all exit points of the fold. Also, @final@ should not be called
527−-- more than once. Note that if a fold terminates by 'Done' constructor, there
528−-- is no state to cleanup.
472+-- | The type @Fold m a b@ having constructor @Fold step initial extract@
473+-- represents a fold over an input stream of values of type @a@ to a final
474+-- value of type @b@ in 'Monad' @m@.
475+--
476+-- The fold uses an intermediate state @s@ as accumulator, the type @s@ is
477+-- internal to the specific fold definition. The initial value of the fold
478+-- state @s@ is returned by @initial@. The @step@ function consumes an input
479+-- and either returns the final result @b@ if the fold is done or the next
480+-- intermediate state (see 'Step'). At any point the fold driver can extract
481+-- the result from the intermediate state using the @extract@ function.
529482--
530483-- NOTE: The constructor is not yet released, smart constructors are provided
531484-- to create folds.
532485--
533486data Fold m a b =
534− -- | @Fold@ @step@ @initial@ @extract@ @final@
535− forall s. Fold (s -> a -> m (Step s b)) (m (Step s b)) (s -> m b) (s -> m b)
487+ -- | @Fold @ @ step @ @ initial @ @ extract@
488+ forall s. Fold (s -> a -> m (Step s b)) (m (Step s b)) (s -> m b)
536489
537490------------------------------------------------------------------------------
538491-- Mapping on the output
@@ -542,8 +495,7 @@ data Fold m a b =
542495--
543496{-# INLINE rmapM #-}
544497rmapM :: Monad m => (b -> m c) -> Fold m a b -> Fold m a c
545−rmapM f (Fold step initial extract final) =
546− Fold step1 initial1 (extract >=> f) (final >=> f)
498+rmapM f (Fold step initial extract) = Fold step1 initial1 (extract >=> f)
547499
548500 where
549501
@@ -574,7 +526,6 @@ foldl' step initial =
574526 (\s a -> return $ Partial $ step s a)
575527 (return (Partial initial))
576528 return
577− return
578529
579530-- | Make a fold from a left fold style monadic step function and initial value
580531-- of the accumulator.
@@ -589,7 +540,7 @@ foldl' step initial =
589540{-# INLINE foldlM' #-}
590541foldlM' :: Monad m => (b -> a -> m b) -> m b -> Fold m a b
591542foldlM' step initial =
592− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) return return
543+ Fold (\s a -> Partial <$> step s a) (Partial <$> initial) return
593544
594545-- | Make a strict left fold, for non-empty streams, using first element as the
595546-- starting value. Returns Nothing if the stream is empty.
@@ -687,11 +638,7 @@ foldrM' g z =
687638{-# INLINE foldt' #-}
688639foldt' :: Monad m => (s -> a -> Step s b) -> Step s b -> (s -> b) -> Fold m a b
689640foldt' step initial extract =
690− Fold
691− (\s a -> return $ step s a)
692− (return initial)
693− (return . extract)
694− (return . extract)
641+ Fold (\s a -> return $ step s a) (return initial) (return . extract)
695642
696643-- | Make a terminating fold with an effectful step function and initial state,
697644-- and a state extraction function.
@@ -704,7 +651,7 @@ foldt' step initial extract =
704651--
705652{-# INLINE foldtM' #-}
706653foldtM' :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> Fold m a b
707−foldtM' step initial extract = Fold step initial extract extract
654+foldtM' = Fold
708655
709656------------------------------------------------------------------------------
710657-- Refold
@@ -718,7 +665,7 @@ foldtM' step initial extract = Fold step initial extract extract
718665-- /Internal/
719666fromRefold :: Refold m c a b -> c -> Fold m a b
720667fromRefold (Refold step inject extract) c =
721− Fold step (inject c) extract extract
668+ Fold step (inject c) extract
722669
723670------------------------------------------------------------------------------
724671-- Basic Folds
@@ -771,29 +718,6 @@ toStreamKRev = foldl' (flip K.cons) K.nil
771718toStreamK :: Monad m => Fold m a (K.StreamK n a)
772719toStreamK = foldr K.cons K.nil
773720
774−-- | Like 'length', except with a more general 'Num' return value
775−--
776−-- Definition:
777−--
778−-- >>> lengthGeneric = fmap getSum $ Fold.foldMap (Sum . const 1)
779−-- >>> lengthGeneric = Fold.foldl' (\n _ -> n + 1) 0
780−--
781−-- /Pre-release/
782−{-# INLINE lengthGeneric #-}
783−lengthGeneric :: (Monad m, Num b) => Fold m a b
784−lengthGeneric = foldl' (\n _ -> n + 1) 0
785−
786−-- | Determine the length of the input stream.
787−--
788−-- Definition:
789−--
790−-- >>> length = Fold.lengthGeneric
791−-- >>> length = fmap getSum $ Fold.foldMap (Sum . const 1)
792−--
793−{-# INLINE length #-}
794−length :: Monad m => Fold m a Int
795−length = lengthGeneric
796−
797721------------------------------------------------------------------------------
798722-- Instances
799723------------------------------------------------------------------------------
@@ -801,8 +725,7 @@ length = lengthGeneric
801725-- | Maps a function on the output of the fold (the type @b@).
802726instance Functor m => Functor (Fold m a) where
803727 {-# INLINE fmap #-}
804− fmap f (Fold step1 initial1 extract final) =
805− Fold step initial (fmap2 f extract) (fmap2 f final)
728+ fmap f (Fold step1 initial1 extract) = Fold step initial (fmap2 f extract)
806729
807730 where
808731
@@ -837,7 +760,7 @@ instance Functor m => Functor (Fold m a) where
837760--
838761{-# INLINE fromPure #-}
839762fromPure :: Applicative m => b -> Fold m a b
840−fromPure b = Fold undefined (pure $ Done b) pure pure
763+fromPure b = Fold undefined (pure $ Done b) pure
841764
842765-- | Make a fold that yields the result of the supplied effectful action
843766-- without consuming any further input.
@@ -846,7 +769,7 @@ fromPure b = Fold undefined (pure $ Done b) pure pure
846769--
847770{-# INLINE fromEffect #-}
848771fromEffect :: Applicative m => m b -> Fold m a b
849−fromEffect b = Fold undefined (Done <$> b) pure pure
772+fromEffect b = Fold undefined (Done <$> b) pure
850773
851774{-# ANN type SeqFoldState Fuse #-}
852775data SeqFoldState sl f sr = SeqFoldL !sl | SeqFoldR !f !sr
@@ -871,20 +794,16 @@ data SeqFoldState sl f sr = SeqFoldL !sl | SeqFoldR !f !sr
871794-- complexity, because each composition adds a new branch that each subsequent
872795-- fold's input element has to traverse, therefore, it cannot scale to a large
873796-- number of compositions. After around 100 compositions the performance starts
874−-- dipping rapidly compared to a CPS style implementation.
875−--
876−-- For larger number of compositions you can convert the fold to a parser and
877−-- use ParserK.
797+-- dipping rapidly compared to a CPS style implementation. When you need
798+-- scaling use parser monad instead.
878799--
879800-- /Time: O(n^2) where n is the number of compositions./
880801--
881802{-# INLINE splitWith #-}
882803splitWith :: Monad m =>
883804 (a -> b -> c) -> Fold m x a -> Fold m x b -> Fold m x c
884−splitWith func
885− (Fold stepL initialL _ finalL)
886− (Fold stepR initialR _ finalR) =
887− Fold step initial extract final
805+splitWith func (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
806+ Fold step initial extract
888807
889808 where
890809
@@ -901,18 +820,13 @@ splitWith func
901820 step (SeqFoldL st) a = runL (stepL st a)
902821 step (SeqFoldR f st) a = runR (stepR st a) f
903822
904− -- XXX splitWith should not be used for scanning
905− -- It would rarely make sense and resource tracking and cleanup would be
906− -- expensive. especially when multiple splitWith are chained.
907− extract _ = error "splitWith: cannot be used for scanning"
908−
909− final (SeqFoldR f sR) = fmap f (finalR sR)
910− final (SeqFoldL sL) = do
911− rL <- finalL sL
823+ extract (SeqFoldR f sR) = fmap f (extractR sR)
824+ extract (SeqFoldL sL) = do
825+ rL <- extractL sL
912826 res <- initialR
913827 fmap (func rL)
914828 $ case res of
915− Partial sR -> finalR sR
829+ Partial sR -> extractR sR
916830 Done rR -> return rR
917831
918832{-# DEPRECATED serialWith "Please use \"splitWith\" instead" #-}
@@ -932,8 +846,8 @@ data SeqFoldState_ sl sr = SeqFoldL_ !sl | SeqFoldR_ !sr
932846--
933847{-# INLINE split_ #-}
934848split_ :: Monad m => Fold m x a -> Fold m x b -> Fold m x b
935−split_ (Fold stepL initialL _ finalL) (Fold stepR initialR _ finalR) =
936− Fold step initial extract final
849+split_ (Fold stepL initialL _) (Fold stepR initialR extractR) =
850+ Fold step initial extract
937851
938852 where
939853
@@ -956,16 +870,11 @@ split_ (Fold stepL initialL _ finalL) (Fold stepR initialR _ finalR) =
956870 resR <- stepR st a
957871 return $ first SeqFoldR_ resR
958872
959− -- XXX split_ should not be used for scanning
960− -- See splitWith for more details.
961− extract _ = error "split_: cannot be used for scanning"
962−
963− final (SeqFoldR_ sR) = finalR sR
964− final (SeqFoldL_ sL) = do
965− _ <- finalL sL
873+ extract (SeqFoldR_ sR) = extractR sR
874+ extract (SeqFoldL_ _) = do
966875 res <- initialR
967876 case res of
968− Partial sR -> finalR sR
877+ Partial sR -> extractR sR
969878 Done rR -> return rR
970879
971880-- | 'Applicative' form of 'splitWith'. Split the input serially over two
@@ -1013,10 +922,8 @@ data TeeState sL sR bL bR
1013922--
1014923{-# INLINE teeWith #-}
1015924teeWith :: Monad m => (a -> b -> c) -> Fold m x a -> Fold m x b -> Fold m x c
1016−teeWith f
1017− (Fold stepL initialL extractL finalL)
1018− (Fold stepR initialR extractR finalR) =
1019− Fold step initial extract final
925+teeWith f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
926+ Fold step initial extract
1020927
1021928 where
1022929
@@ -1043,10 +950,6 @@ teeWith f
1043950 extract (TeeLeft bR sL) = (`f` bR) <$> extractL sL
1044951 extract (TeeRight bL sR) = f bL <$> extractR sR
1045952
1046− final (TeeBoth sL sR) = f <$> finalL sL <*> finalR sR
1047− final (TeeLeft bR sL) = (`f` bR) <$> finalL sL
1048− final (TeeRight bL sR) = f bL <$> finalR sR
1049−
1050953{-# ANN type TeeFstState Fuse #-}
1051954data TeeFstState sL sR b
1052955 = TeeFstBoth !sL !sR
@@ -1059,10 +962,8 @@ data TeeFstState sL sR b
1059962{-# INLINE teeWithFst #-}
1060963teeWithFst :: Monad m =>
1061964 (b -> c -> d) -> Fold m a b -> Fold m a c -> Fold m a d
1062−teeWithFst f
1063− (Fold stepL initialL extractL finalL)
1064− (Fold stepR initialR extractR finalR) =
1065− Fold step initial extract final
965+teeWithFst f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
966+ Fold step initial extract
1066967
1067968 where
1068969
@@ -1081,7 +982,7 @@ teeWithFst f
1081982 Done bl -> do
1082983 Done . f bl <$>
1083984 case resR of
1084− Partial sr -> finalR sr
985+ Partial sr -> extractR sr
1085986 Done br -> return br
1086987
1087988 initial = runBoth initialL initialR
@@ -1092,9 +993,6 @@ teeWithFst f
1092993 extract (TeeFstBoth sL sR) = f <$> extractL sL <*> extractR sR
1093994 extract (TeeFstLeft bR sL) = (`f` bR) <$> extractL sL
1094995
1095− final (TeeFstBoth sL sR) = f <$> finalL sL <*> finalR sR
1096− final (TeeFstLeft bR sL) = (`f` bR) <$> finalL sL
1097−
1098996-- | Like 'teeWith' but terminates as soon as any one of the two folds
1099997-- terminates.
1100998--
@@ -1103,10 +1001,8 @@ teeWithFst f
11031001{-# INLINE teeWithMin #-}
11041002teeWithMin :: Monad m =>
11051003 (b -> c -> d) -> Fold m a b -> Fold m a c -> Fold m a d
1106−teeWithMin f
1107− (Fold stepL initialL extractL finalL)
1108− (Fold stepR initialR extractR finalR) =
1109− Fold step initial extract final
1004+teeWithMin f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1005+ Fold step initial extract
11101006
11111007 where
11121008
@@ -1118,12 +1014,12 @@ teeWithMin f
11181014 Partial sl -> do
11191015 case resR of
11201016 Partial sr -> return $ Partial $ Tuple' sl sr
1121− Done br -> Done . (`f` br) <$> finalL sl
1017+ Done br -> Done . (`f` br) <$> extractL sl
11221018
11231019 Done bl -> do
11241020 Done . f bl <$>
11251021 case resR of
1126− Partial sr -> finalR sr
1022+ Partial sr -> extractR sr
11271023 Done br -> return br
11281024
11291025 initial = runBoth initialL initialR
@@ -1132,8 +1028,6 @@ teeWithMin f
11321028
11331029 extract (Tuple' sL sR) = f <$> extractL sL <*> extractR sR
11341030
1135− final (Tuple' sL sR) = f <$> finalL sL <*> finalR sR
1136−
11371031-- | Shortest alternative. Apply both folds in parallel but choose the result
11381032-- from the one which consumed least input i.e. take the shortest succeeding
11391033-- fold.
@@ -1145,8 +1039,8 @@ teeWithMin f
11451039--
11461040{-# INLINE shortest #-}
11471041shortest :: Monad m => Fold m x a -> Fold m x b -> Fold m x (Either a b)
1148−shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
1149− Fold step initial extract final
1042+shortest (Fold stepL initialL extractL) (Fold stepR initialR _) =
1043+ Fold step initial extract
11501044
11511045 where
11521046
@@ -1154,16 +1048,10 @@ shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
11541048 runBoth actionL actionR = do
11551049 resL <- actionL
11561050 resR <- actionR
1157− case resL of
1158− Partial sL ->
1159− case resR of
1160− Partial sR -> return $ Partial $ Tuple' sL sR
1161− Done bR -> finalL sL >> return (Done (Right bR))
1162− Done bL -> do
1163− case resR of
1164− Partial sR -> void (finalR sR)
1165− Done _ -> return ()
1166− return (Done (Left bL))
1051+ return $
1052+ case resL of
1053+ Partial sL -> bimap (Tuple' sL) Right resR
1054+ Done bL -> Done $ Left bL
11671055
11681056 initial = runBoth initialL initialR
11691057
@@ -1171,8 +1059,6 @@ shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
11711059
11721060 extract (Tuple' sL _) = Left <$> extractL sL
11731061
1174− final (Tuple' sL sR) = Left <$> finalL sL <* finalR sR
1175−
11761062{-# ANN type LongestState Fuse #-}
11771063data LongestState sL sR
11781064 = LongestBoth !sL !sR
@@ -1190,10 +1076,8 @@ data LongestState sL sR
11901076--
11911077{-# INLINE longest #-}
11921078longest :: Monad m => Fold m x a -> Fold m x b -> Fold m x (Either a b)
1193−longest
1194− (Fold stepL initialL _ finalL)
1195− (Fold stepR initialR _ finalR) =
1196− Fold step initial extract final
1079+longest (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1080+ Fold step initial extract
11971081
11981082 where
11991083
@@ -1216,18 +1100,14 @@ longest
12161100 step (LongestLeft sL) a = bimap LongestLeft Left <$> stepL sL a
12171101 step (LongestRight sR) a = bimap LongestRight Right <$> stepR sR a
12181102
1219− -- XXX Scan with this may not make sense as we cannot determine the longest
1220− -- until one of them have exhausted.
1221− extract _ = error $ "longest: scan is not allowed as longest cannot be "
1222− ++ "determined until one fold has exhausted."
1223−
1224− final (LongestLeft sL) = Left <$> finalL sL
1225− final (LongestRight sR) = Right <$> finalR sR
1226− final (LongestBoth sL sR) = Left <$> finalL sL <* finalR sR
1103+ left sL = Left <$> extractL sL
1104+ extract (LongestLeft sL) = left sL
1105+ extract (LongestRight sR) = Right <$> extractR sR
1106+ extract (LongestBoth sL _) = left sL
12271107
1228−data ConcatMapState m sa a b c
1229− = B !sa (sa -> m b)
1230− | forall s. C (s -> a -> m (Step s c)) !s (s -> m c) (s -> m c)
1108+data ConcatMapState m sa a c
1109+ = B !sa
1110+ | forall s. C (s -> a -> m (Step s c)) !s (s -> m c)
12311111
12321112-- | Map a 'Fold' returning function on the result of a 'Fold' and run the
12331113-- returned fold. This operation can be used to express data dependencies
@@ -1250,47 +1130,43 @@ data ConcatMapState m sa a b c
12501130--
12511131{-# INLINE concatMap #-}
12521132concatMap :: Monad m => (b -> Fold m a c) -> Fold m a b -> Fold m a c
1253−concatMap f (Fold stepa initiala _ finala) =
1254− Fold stepc initialc extractc finalc
1133+concatMap f (Fold stepa initiala extracta) = Fold stepc initialc extractc
12551134 where
12561135 initialc = do
12571136 r <- initiala
12581137 case r of
1259− Partial s -> return $ Partial (B s finala)
1138+ Partial s -> return $ Partial (B s)
12601139 Done b -> initInnerFold (f b)
12611140
1262− stepc (B s fin) a = do
1141+ stepc (B s) a = do
12631142 r <- stepa s a
12641143 case r of
1265− Partial s1 -> return $ Partial (B s1 fin)
1144+ Partial s1 -> return $ Partial (B s1)
12661145 Done b -> initInnerFold (f b)
12671146
1268− stepc (C stepInner s extractInner fin) a = do
1147+ stepc (C stepInner s extractInner) a = do
12691148 r <- stepInner s a
12701149 return $ case r of
1271− Partial sc -> Partial (C stepInner sc extractInner fin)
1150+ Partial sc -> Partial (C stepInner sc extractInner)
12721151 Done c -> Done c
12731152
1274− -- XXX Cannot use for scanning
1275− extractc _ = error "concatMap: cannot be used for scanning"
1153+ extractc (B s) = do
1154+ r <- extracta s
1155+ initExtract (f r)
1156+ extractc (C _ sInner extractInner) = extractInner sInner
12761157
1277− initInnerFold (Fold step i e fin) = do
1158+ initInnerFold (Fold step i e) = do
12781159 r <- i
12791160 return $ case r of
1280− Partial s -> Partial (C step s e fin)
1161+ Partial s -> Partial (C step s e)
12811162 Done c -> Done c
12821163
1283− initFinalize (Fold _ i _ fin) = do
1164+ initExtract (Fold _ i e) = do
12841165 r <- i
12851166 case r of
1286− Partial s -> fin s
1167+ Partial s -> e s
12871168 Done c -> return c
12881169
1289− finalc (B s fin) = do
1290− r <- fin s
1291− initFinalize (f r)
1292− finalc (C _ sInner _ fin) = fin sInner
1293−
12941170------------------------------------------------------------------------------
12951171-- Mapping on input
12961172------------------------------------------------------------------------------
@@ -1309,7 +1185,7 @@ concatMap f (Fold stepa initiala _ finala) =
13091185--
13101186{-# INLINE lmap #-}
13111187lmap :: (a -> b) -> Fold m b r -> Fold m a r
1312−lmap f (Fold step begin done final) = Fold step' begin done final
1188+lmap f (Fold step begin done) = Fold step' begin done
13131189 where
13141190 step' x a = step x (f a)
13151191
@@ -1317,7 +1193,7 @@ lmap f (Fold step begin done final) = Fold step' begin done final
13171193--
13181194{-# INLINE lmapM #-}
13191195lmapM :: Monad m => (a -> m b) -> Fold m b r -> Fold m a r
1320−lmapM f (Fold step begin done final) = Fold step' begin done final
1196+lmapM f (Fold step begin done) = Fold step' begin done
13211197 where
13221198 step' x a = f a >>= step x
13231199
@@ -1329,10 +1205,8 @@ lmapM f (Fold step begin done final) = Fold step' begin done final
13291205-- /Pre-release/
13301206{-# INLINE postscan #-}
13311207postscan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1332−postscan
1333− (Fold stepL initialL extractL finalL)
1334− (Fold stepR initialR extractR finalR) =
1335− Fold step initial extract final
1208+postscan (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1209+ Fold step initial extract
13361210
13371211 where
13381212
@@ -1343,32 +1217,30 @@ postscan
13431217 Done bL -> do
13441218 rR <- stepR sR bL
13451219 case rR of
1346− Partial sR1 -> Done <$> finalR sR1
1220+ Partial sR1 -> Done <$> extractR sR1
13471221 Done bR -> return $ Done bR
13481222 Partial sL -> do
13491223 !b <- extractL sL
13501224 rR <- stepR sR b
1351− case rR of
1352− Partial sR1 -> return $ Partial (sL, sR1)
1353− Done bR -> finalL sL >> return (Done bR)
1225+ return
1226+ $ case rR of
1227+ Partial sR1 -> Partial (sL, sR1)
1228+ Done bR -> Done bR
13541229
13551230 initial = do
1356− rR <- initialR
1357− case rR of
1358− Partial sR -> do
1359− rL <- initialL
1231+ r <- initialR
1232+ rL <- initialL
1233+ case r of
1234+ Partial sR ->
13601235 case rL of
1361− Done _ -> Done <$> finalR sR
1236+ Done _ -> Done <$> extractR sR
13621237 Partial sL -> return $ Partial (sL, sR)
13631238 Done b -> return $ Done b
13641239
1365− -- XXX should use Tuple'
13661240 step (sL, sR) x = runStep (stepL sL x) sR
13671241
13681242 extract = extractR . snd
13691243
1370− final (sL, sR) = finalL sL *> finalR sR
1371−
13721244------------------------------------------------------------------------------
13731245-- Filtering
13741246------------------------------------------------------------------------------
@@ -1381,7 +1253,7 @@ postscan
13811253--
13821254{-# INLINE_NORMAL catMaybes #-}
13831255catMaybes :: Monad m => Fold m a b -> Fold m (Maybe a) b
1384−catMaybes (Fold step initial extract final) = Fold step1 initial extract final
1256+catMaybes (Fold step initial extract) = Fold step1 initial extract
13851257
13861258 where
13871259
@@ -1421,7 +1293,7 @@ filtering f = foldl' step Nothing
14211293{-# INLINE filter #-}
14221294filter :: Monad m => (a -> Bool) -> Fold m a r -> Fold m a r
14231295-- filter p = scanMaybe (filtering p)
1424−filter f (Fold step begin extract final) = Fold step' begin extract final
1296+filter f (Fold step begin done) = Fold step' begin done
14251297 where
14261298 step' x a = if f a then step x a else return $ Partial x
14271299
@@ -1432,7 +1304,7 @@ filter f (Fold step begin extract final) = Fold step' begin extract final
14321304--
14331305{-# INLINE filterM #-}
14341306filterM :: Monad m => (a -> m Bool) -> Fold m a r -> Fold m a r
1435−filterM f (Fold step begin extract final) = Fold step' begin extract final
1307+filterM f (Fold step begin done) = Fold step' begin done
14361308 where
14371309 step' x a = do
14381310 use <- f a
@@ -1521,7 +1393,7 @@ dropping n = foldt' step initial extract
15211393{-# INLINE take #-}
15221394take :: Monad m => Int -> Fold m a b -> Fold m a b
15231395-- take n = scanMaybe (taking n)
1524−take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
1396+take n (Fold fstep finitial fextract) = Fold step initial extract
15251397
15261398 where
15271399
@@ -1533,7 +1405,7 @@ take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
15331405 s1 = Tuple'Fused i1 s
15341406 if i1 < n
15351407 then return $ Partial s1
1536− else Done <$> ffinal s
1408+ else Done <$> fextract s
15371409 Done b -> return $ Done b
15381410
15391411 initial = finitial >>= next (-1)
@@ -1542,72 +1414,6 @@ take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
15421414
15431415 extract (Tuple'Fused _ r) = fextract r
15441416
1545− final (Tuple'Fused _ r) = ffinal r
1546−
1547−-- Note: Keep this consistent with S.splitOn. In fact we should eliminate
1548−-- S.splitOn in favor of the fold.
1549−--
1550−-- XXX Use Fold.many instead once it is fixed.
1551−-- > Stream.splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
1552−
1553−-- | Like 'takeEndBy' but drops the element on which the predicate succeeds.
1554−--
1555−-- Example:
1556−--
1557−-- >>> input = Stream.fromList "hello\nthere\n"
1558−-- >>> line = Fold.takeEndBy_ (== '\n') Fold.toList
1559−-- >>> Stream.fold line input
1560−-- "hello"
1561−--
1562−-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1563−-- ["hello","there"]
1564−--
1565−{-# INLINE takeEndBy_ #-}
1566−takeEndBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1567−-- takeEndBy_ predicate = scanMaybe (takingEndBy_ predicate)
1568−takeEndBy_ predicate (Fold fstep finitial fextract ffinal) =
1569− Fold step finitial fextract ffinal
1570−
1571− where
1572−
1573− step s a =
1574− if not (predicate a)
1575− then fstep s a
1576− else Done <$> ffinal s
1577−
1578−-- Note:
1579−-- > Stream.splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
1580−
1581−-- | Take the input, stop when the predicate succeeds taking the succeeding
1582−-- element as well.
1583−--
1584−-- Example:
1585−--
1586−-- >>> input = Stream.fromList "hello\nthere\n"
1587−-- >>> line = Fold.takeEndBy (== '\n') Fold.toList
1588−-- >>> Stream.fold line input
1589−-- "hello\n"
1590−--
1591−-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1592−-- ["hello\n","there\n"]
1593−--
1594−{-# INLINE takeEndBy #-}
1595−takeEndBy :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1596−-- takeEndBy predicate = scanMaybe (takingEndBy predicate)
1597−takeEndBy predicate (Fold fstep finitial fextract ffinal) =
1598− Fold step finitial fextract ffinal
1599−
1600− where
1601−
1602− step s a = do
1603− res <- fstep s a
1604− if not (predicate a)
1605− then return res
1606− else do
1607− case res of
1608− Partial s1 -> Done <$> ffinal s1
1609− Done b -> return $ Done b
1610−
16111417------------------------------------------------------------------------------
16121418-- Nesting
16131419------------------------------------------------------------------------------
@@ -1625,8 +1431,8 @@ takeEndBy predicate (Fold fstep finitial fextract ffinal) =
16251431-- /Pre-release/
16261432{-# INLINE duplicate #-}
16271433duplicate :: Monad m => Fold m a b -> Fold m a (Fold m a b)
1628−duplicate (Fold step1 initial1 extract1 final1) =
1629− Fold step initial extract final
1434+duplicate (Fold step1 initial1 extract1) =
1435+ Fold step initial (\s -> pure $ Fold step1 (pure $ Partial s) extract1)
16301436
16311437 where
16321438
@@ -1634,11 +1440,6 @@ duplicate (Fold step1 initial1 extract1 final1) =
16341440
16351441 step s a = second fromPure <$> step1 s a
16361442
1637− -- Scanning may be problematic due to multiple finalizations.
1638− extract = error "duplicate: scanning may be problematic"
1639−
1640− final s = pure $ Fold step1 (pure $ Partial s) extract1 final1
1641−
16421443-- If there were a finalize/flushing action in the stream type that would be
16431444-- equivalent to running initialize in Fold. But we do not have a flushing
16441445-- action in streams.
@@ -1650,9 +1451,9 @@ duplicate (Fold step1 initial1 extract1 final1) =
16501451-- /Pre-release/
16511452{-# INLINE reduce #-}
16521453reduce :: Monad m => Fold m a b -> m (Fold m a b)
1653−reduce (Fold step initial extract final) = do
1454+reduce (Fold step initial extract) = do
16541455 i <- initial
1655− return $ Fold step (return i) extract final
1456+ return $ Fold step (return i) extract
16561457
16571458-- This is the dual of Stream @cons@.
16581459
@@ -1662,8 +1463,7 @@ reduce (Fold step initial extract final) = do
16621463-- /Pre-release/
16631464{-# INLINE snoclM #-}
16641465snoclM :: Monad m => Fold m a b -> m a -> Fold m a b
1665−snoclM (Fold fstep finitial fextract ffinal) action =
1666− Fold fstep initial fextract ffinal
1466+snoclM (Fold fstep finitial fextract) action = Fold fstep initial fextract
16671467
16681468 where
16691469
@@ -1690,8 +1490,7 @@ snoclM (Fold fstep finitial fextract ffinal) action =
16901490{-# INLINE snocl #-}
16911491snocl :: Monad m => Fold m a b -> a -> Fold m a b
16921492-- snocl f = snoclM f . return
1693−snocl (Fold fstep finitial fextract ffinal) a =
1694− Fold fstep initial fextract ffinal
1493+snocl (Fold fstep finitial fextract) a = Fold fstep initial fextract
16951494
16961495 where
16971496
@@ -1711,12 +1510,12 @@ snocl (Fold fstep finitial fextract ffinal) a =
17111510-- /Pre-release/
17121511{-# INLINE snocM #-}
17131512snocM :: Monad m => Fold m a b -> m a -> m (Fold m a b)
1714−snocM (Fold step initial extract final) action = do
1513+snocM (Fold step initial extract) action = do
17151514 res <- initial
17161515 r <- case res of
17171516 Partial fs -> action >>= step fs
17181517 Done _ -> return res
1719− return $ Fold step (return r) extract final
1518+ return $ Fold step (return r) extract
17201519
17211520-- Definitions:
17221521--
@@ -1735,12 +1534,12 @@ snocM (Fold step initial extract final) action = do
17351534-- /Pre-release/
17361535{-# INLINE snoc #-}
17371536snoc :: Monad m => Fold m a b -> a -> m (Fold m a b)
1738−snoc (Fold step initial extract final) a = do
1537+snoc (Fold step initial extract) a = do
17391538 res <- initial
17401539 r <- case res of
17411540 Partial fs -> step fs a
17421541 Done _ -> return res
1743− return $ Fold step (return r) extract final
1542+ return $ Fold step (return r) extract
17441543
17451544-- | Append a singleton value to the fold.
17461545--
@@ -1768,7 +1567,7 @@ addOne = flip snoc
17681567-- /Pre-release/
17691568{-# INLINE extractM #-}
17701569extractM :: Monad m => Fold m a b -> m b
1771−extractM (Fold _ initial extract _) = do
1570+extractM (Fold _ initial extract) = do
17721571 res <- initial
17731572 case res of
17741573 Partial fs -> extract fs
@@ -1777,15 +1576,14 @@ extractM (Fold _ initial extract _) = do
17771576-- | Close a fold so that it does not accept any more input.
17781577{-# INLINE close #-}
17791578close :: Monad m => Fold m a b -> Fold m a b
1780−close (Fold _ initial1 _ final1) =
1781− Fold undefined initial undefined undefined
1579+close (Fold _ initial1 extract1) = Fold undefined initial undefined
17821580
17831581 where
17841582
17851583 initial = do
17861584 res <- initial1
17871585 case res of
1788− Partial s -> Done <$> final1 s
1586+ Partial s -> Done <$> extract1 s
17891587 Done b -> return $ Done b
17901588
17911589-- Corresponds to the null check for streams.
@@ -1795,7 +1593,7 @@ close (Fold _ initial1 _ final1) =
17951593-- /Pre-release/
17961594{-# INLINE isClosed #-}
17971595isClosed :: Monad m => Fold m a b -> m Bool
1798−isClosed (Fold _ initial _ _) = do
1596+isClosed (Fold _ initial _) = do
17991597 res <- initial
18001598 return $ case res of
18011599 Partial _ -> False
@@ -1829,10 +1627,8 @@ data ManyState s1 s2
18291627--
18301628{-# INLINE many #-}
18311629many :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1832−many
1833− (Fold sstep sinitial sextract sfinal)
1834− (Fold cstep cinitial cextract cfinal) =
1835− Fold step initial extract final
1630+many (Fold sstep sinitial sextract) (Fold cstep cinitial cextract) =
1631+ Fold step initial extract
18361632
18371633 where
18381634
@@ -1881,13 +1677,6 @@ many
18811677 Partial s -> cextract s
18821678 Done b -> return b
18831679
1884− final (ManyFirst ss cs) = sfinal ss *> cfinal cs
1885− final (ManyLoop ss cs) = do
1886− cres <- sfinal ss >>= cstep cs
1887− case cres of
1888− Partial s -> cfinal s
1889− Done b -> return b
1890−
18911680-- | Like many, but the "first" fold emits an output at the end even if no
18921681-- input is received.
18931682--
@@ -1897,10 +1686,8 @@ many
18971686--
18981687{-# INLINE manyPost #-}
18991688manyPost :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1900−manyPost
1901− (Fold sstep sinitial sextract sfinal)
1902− (Fold cstep cinitial cextract cfinal) =
1903− Fold step initial extract final
1689+manyPost (Fold sstep sinitial sextract) (Fold cstep cinitial cextract) =
1690+ Fold step initial extract
19041691
19051692 where
19061693
@@ -1936,12 +1723,6 @@ manyPost
19361723 Partial s -> cextract s
19371724 Done b -> return b
19381725
1939− final (Tuple' ss cs) = do
1940− cres <- sfinal ss >>= cstep cs
1941− case cres of
1942− Partial s -> cfinal s
1943− Done b -> return b
1944−
19451726-- | @groupsOf n split collect@ repeatedly applies the @split@ fold to chunks
19461727-- of @n@ items in the input stream and supplies the result to the @collect@
19471728-- fold.
@@ -1970,10 +1751,7 @@ groupsOf n split = many (take n split)
19701751--
19711752{-# INLINE refoldMany #-}
19721753refoldMany :: Monad m => Fold m a b -> Refold m x b c -> Refold m x a c
1973−refoldMany
1974− (Fold sstep sinitial sextract _sfinal)
1975− -- XXX We will need a "final" in refold as well
1976− (Refold cstep cinject cextract) =
1754+refoldMany (Fold sstep sinitial sextract) (Refold cstep cinject cextract) =
19771755 Refold step inject extract
19781756
19791757 where
@@ -2024,9 +1802,7 @@ data ConsumeManyState x cs ss = ConsumeMany x cs (Either ss ss)
20241802-- /Internal/
20251803{-# INLINE refoldMany1 #-}
20261804refoldMany1 :: Monad m => Refold m x a b -> Fold m b c -> Refold m x a c
2027−refoldMany1
2028− (Refold sstep sinject sextract)
2029− (Fold cstep cinitial cextract _cfinal) =
1805+refoldMany1 (Refold sstep sinject sextract) (Fold cstep cinitial cextract) =
20301806 Refold step inject extract
20311807
20321808 where
@@ -2077,7 +1853,7 @@ refoldMany1
20771853{-# INLINE refold #-}
20781854refold :: Monad m => Refold m b a c -> Fold m a b -> Fold m a c
20791855refold (Refold step inject extract) f =
2080− Fold step (extractM f >>= inject) extract extract
1856+ Fold step (extractM f >>= inject) extract
20811857
20821858------------------------------------------------------------------------------
20831859-- morphInner
@@ -2087,8 +1863,8 @@ refold (Refold step inject extract) f =
20871863--
20881864-- /Pre-release/
20891865morphInner :: (forall x. m x -> n x) -> Fold m a b -> Fold n a b
2090−morphInner f (Fold step initial extract final) =
2091− Fold (\x a -> f $ step x a) (f initial) (f . extract) (f . final)
1866+morphInner f (Fold step initial extract) =
1867+ Fold (\x a -> f $ step x a) (f initial) (f . extract)
20921868
20931869-- | Adapt a pure fold to any monad.
20941870--
core/src/Streamly/Internal/Data/Fold/Window.hs+53−59
@@ -36,24 +36,24 @@ module Streamly.Internal.Data.Fold.Window
3636 -- window folds by keeping the second element of the input tuple as
3737 -- @Nothing@.
3838 --
39− windowLmap
39+ lmap
4040 , cumulative
4141
42− , windowRollingMap
43− , windowRollingMapM
42+ , rollingMap
43+ , rollingMapM
4444
4545 -- ** Sums
46− , windowLength
47− , windowSum
48− , windowSumInt
49− , windowPowerSum
50− , windowPowerSumFrac
46+ , length
47+ , sum
48+ , sumInt
49+ , powerSum
50+ , powerSumFrac
5151
5252 -- ** Location
53− , windowMinimum
54− , windowMaximum
55− , windowRange
56− , windowMean
53+ , minimum
54+ , maximum
55+ , range
56+ , mean
5757 )
5858where
5959
@@ -65,18 +65,16 @@ import Streamly.Internal.Data.Fold.Type (Fold(..), Step(..))
6565import Streamly.Internal.Data.Tuple.Strict
6666 (Tuple'(..), Tuple3Fused' (Tuple3Fused'))
6767
68−import Foreign.ForeignPtr.Unsafe (unsafeForeignPtrToPtr)
69−
7068import qualified Streamly.Internal.Data.Fold.Type as Fold
71−import qualified Streamly.Internal.Data.Ring as Ring
69+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
7270
7371import Prelude hiding (length, sum, minimum, maximum)
7472
7573-- $setup
7674-- >>> import Data.Bifunctor(bimap)
7775-- >>> import qualified Streamly.Data.Fold as Fold
78−-- >>> import qualified Streamly.Internal.Data.Fold.Window as Fold
79−-- >>> import qualified Streamly.Internal.Data.Ring as Ring
76+-- >>> import qualified Streamly.Internal.Data.Fold.Window as FoldW
77+-- >>> import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
8078-- >>> import qualified Streamly.Data.Stream as Stream
8179-- >>> import Prelude hiding (length, sum, minimum, maximum)
8280
@@ -89,9 +87,9 @@ import Prelude hiding (length, sum, minimum, maximum)
8987--
9088-- >>> lmap f = Fold.lmap (bimap f (f <$>))
9189--
92−{-# INLINE windowLmap #-}
93−windowLmap :: (c -> a) -> Fold m (a, Maybe a) b -> Fold m (c, Maybe c) b
94−windowLmap f = Fold.lmap (bimap f (f <$>))
90+{-# INLINE lmap #-}
91+lmap :: (c -> a) -> Fold m (a, Maybe a) b -> Fold m (c, Maybe c) b
92+lmap f = Fold.lmap (bimap f (f <$>))
9593
9694-- | Convert an incremental fold to a cumulative fold using the entire input
9795-- stream as a single window.
@@ -107,10 +105,10 @@ cumulative = Fold.lmap (, Nothing)
107105
108106-- | Apply an effectful function on the latest and the oldest element of the
109107-- window.
110−{-# INLINE windowRollingMapM #-}
111−windowRollingMapM :: Monad m =>
108+{-# INLINE rollingMapM #-}
109+rollingMapM :: Monad m =>
112110 (Maybe a -> a -> m (Maybe b)) -> Fold m (a, Maybe a) (Maybe b)
113−windowRollingMapM f = Fold.foldlM' f1 initial
111+rollingMapM f = Fold.foldlM' f1 initial
114112
115113 where
116114
@@ -120,12 +118,12 @@ windowRollingMapM f = Fold.foldlM' f1 initial
120118
121119-- | Apply a pure function on the latest and the oldest element of the window.
122120--
123−-- >>> windowRollingMap f = Fold.windowRollingMapM (\x y -> return $ f x y)
121+-- >>> rollingMap f = FoldW.rollingMapM (\x y -> return $ f x y)
124122--
125−{-# INLINE windowRollingMap #-}
126−windowRollingMap :: Monad m =>
123+{-# INLINE rollingMap #-}
124+rollingMap :: Monad m =>
127125 (Maybe a -> a -> Maybe b) -> Fold m (a, Maybe a) (Maybe b)
128−windowRollingMap f = Fold.foldl' f1 initial
126+rollingMap f = Fold.foldl' f1 initial
129127
130128 where
131129
@@ -148,9 +146,9 @@ windowRollingMap f = Fold.foldl' f1 initial
148146--
149147-- /Internal/
150148--
151−{-# INLINE windowSumInt #-}
152−windowSumInt :: forall m a. (Monad m, Integral a) => Fold m (a, Maybe a) a
153−windowSumInt = Fold step initial extract extract
149+{-# INLINE sumInt #-}
150+sumInt :: forall m a. (Monad m, Integral a) => Fold m (a, Maybe a) a
151+sumInt = Fold step initial extract
154152
155153 where
156154
@@ -182,9 +180,9 @@ windowSumInt = Fold step initial extract extract
182180--
183181-- /Time/: \(\mathcal{O}(n)\)
184182--
185−{-# INLINE windowSum #-}
186−windowSum :: forall m a. (Monad m, Num a) => Fold m (a, Maybe a) a
187−windowSum = Fold step initial extract extract
183+{-# INLINE sum #-}
184+sum :: forall m a. (Monad m, Num a) => Fold m (a, Maybe a) a
185+sum = Fold step initial extract
188186
189187 where
190188
@@ -221,9 +219,9 @@ windowSum = Fold step initial extract extract
221219--
222220-- >>> length = powerSum 0
223221--
224−{-# INLINE windowLength #-}
225−windowLength :: (Monad m, Num b) => Fold m (a, Maybe a) b
226−windowLength = Fold.foldl' step 0
222+{-# INLINE length #-}
223+length :: (Monad m, Num b) => Fold m (a, Maybe a) b
224+length = Fold.foldl' step 0
227225
228226 where
229227
@@ -239,18 +237,18 @@ windowLength = Fold.foldl' step 0
239237-- /Space/: \(\mathcal{O}(1)\)
240238--
241239-- /Time/: \(\mathcal{O}(n)\)
242−{-# INLINE windowPowerSum #-}
243−windowPowerSum :: (Monad m, Num a) => Int -> Fold m (a, Maybe a) a
244−windowPowerSum k = windowLmap (^ k) windowSum
240+{-# INLINE powerSum #-}
241+powerSum :: (Monad m, Num a) => Int -> Fold m (a, Maybe a) a
242+powerSum k = lmap (^ k) sum
245243
246244-- | Like 'powerSum' but powers can be negative or fractional. This is slower
247245-- than 'powerSum' for positive intergal powers.
248246--
249247-- >>> powerSumFrac p = lmap (** p) sum
250248--
251−{-# INLINE windowPowerSumFrac #-}
252−windowPowerSumFrac :: (Monad m, Floating a) => a -> Fold m (a, Maybe a) a
253−windowPowerSumFrac p = windowLmap (** p) windowSum
249+{-# INLINE powerSumFrac #-}
250+powerSumFrac :: (Monad m, Floating a) => a -> Fold m (a, Maybe a) a
251+powerSumFrac p = lmap (** p) sum
254252
255253-------------------------------------------------------------------------------
256254-- Location
@@ -267,9 +265,9 @@ windowPowerSumFrac p = windowLmap (** p) windowSum
267265--
268266-- /Time/: \(\mathcal{O}(n*w)\) where \(w\) is the window size.
269267--
270−{-# INLINE windowRange #-}
271−windowRange :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe (a, a))
272−windowRange n = Fold step initial extract extract
268+{-# INLINE range #-}
269+range :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe (a, a))
270+range n = Fold step initial extract
273271
274272 where
275273
@@ -302,11 +300,7 @@ windowRange n = Fold step initial extract extract
302300 if i == 0
303301 then return Nothing
304302 else do
305− -- Here we use "ringStart" over "ringHead" as "ringHead" will be
306− -- uninitialized if the ring is not full.
307− -- Using "unsafeForeignPtrToPtr" here is safe as we touch the ring
308− -- again in "foldFunc".
309− x <- liftIO $ peek (unsafeForeignPtrToPtr (Ring.ringStart rb))
303+ x <- liftIO $ peek rh
310304 let accum (mn, mx) a = return (min mn a, max mx a)
311305 fmap Just $ foldFunc i rh accum (x, x) rb
312306
@@ -322,9 +316,9 @@ windowRange n = Fold step initial extract extract
322316--
323317-- /Time/: \(\mathcal{O}(n*w)\) where \(w\) is the window size.
324318--
325−{-# INLINE windowMinimum #-}
326−windowMinimum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
327−windowMinimum n = fmap (fmap fst) $ windowRange n
319+{-# INLINE minimum #-}
320+minimum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
321+minimum n = fmap (fmap fst) $ range n
328322
329323-- | The maximum element in a rolling window.
330324--
@@ -335,9 +329,9 @@ windowMinimum n = fmap (fmap fst) $ windowRange n
335329--
336330-- /Time/: \(\mathcal{O}(n*w)\) where \(w\) is the window size.
337331--
338−{-# INLINE windowMaximum #-}
339−windowMaximum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
340−windowMaximum n = fmap (fmap snd) $ windowRange n
332+{-# INLINE maximum #-}
333+maximum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
334+maximum n = fmap (fmap snd) $ range n
341335
342336-- | Arithmetic mean of elements in a sliding window:
343337--
@@ -352,6 +346,6 @@ windowMaximum n = fmap (fmap snd) $ windowRange n
352346-- /Space/: \(\mathcal{O}(1)\)
353347--
354348-- /Time/: \(\mathcal{O}(n)\)
355−{-# INLINE windowMean #-}
356−windowMean :: forall m a. (Monad m, Fractional a) => Fold m (a, Maybe a) a
357−windowMean = Fold.teeWith (/) windowSum windowLength
349+{-# INLINE mean #-}
350+mean :: forall m a. (Monad m, Fractional a) => Fold m (a, Maybe a) a
351+mean = Fold.teeWith (/) sum length
core/src/Streamly/Internal/Data/IORef/Unboxed.hs+20−23
@@ -23,31 +23,34 @@ module Streamly.Internal.Data.IORef.Unboxed
2323 (
2424 IORef
2525
26− -- Construction
26+ -- * Construction
2727 , newIORef
2828
29− -- Write
29+ -- * Write
3030 , writeIORef
3131 , modifyIORef'
3232
33− -- Read
33+ -- * Read
3434 , readIORef
35− , pollIntIORef
35+ , toStreamD
3636 )
3737where
3838
3939#include "inline.hs"
4040
41−import Control.Monad.IO.Class (MonadIO(..))
4241import Data.Proxy (Proxy(..))
43−import Streamly.Internal.Data.MutByteArray.Type (MutByteArray)
44−import Streamly.Internal.Data.Unbox (Unbox(..), sizeOf)
42+import Control.Monad.IO.Class (MonadIO(..))
43+import Streamly.Internal.Data.Unbox
44+ ( MutableByteArray(..)
45+ , Unbox(..)
46+ , newBytes
47+ , sizeOf
48+ )
4549
46−import qualified Streamly.Internal.Data.MutByteArray.Type as MBA
47−import qualified Streamly.Internal.Data.Stream.Type as D
50+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
4851
4952-- | An 'IORef' holds a single 'Unbox'-able value.
50−newtype IORef a = IORef MutByteArray
53+newtype IORef a = IORef MutableByteArray
5154
5255-- | Create a new 'IORef'.
5356--
@@ -55,8 +58,8 @@ newtype IORef a = IORef MutByteArray
5558{-# INLINE newIORef #-}
5659newIORef :: forall a. Unbox a => a -> IO (IORef a)
5760newIORef x = do
58− var <- MBA.new (sizeOf (Proxy :: Proxy a))
59− pokeAt 0 var x
61+ var <- newBytes (sizeOf (Proxy :: Proxy a))
62+ pokeByteIndex 0 var x
6063 return $ IORef var
6164
6265-- | Write a value to an 'IORef'.
@@ -64,14 +67,14 @@ newIORef x = do
6467-- /Pre-release/
6568{-# INLINE writeIORef #-}
6669writeIORef :: Unbox a => IORef a -> a -> IO ()
67−writeIORef (IORef var) = pokeAt 0 var
70+writeIORef (IORef var) = pokeByteIndex 0 var
6871
6972-- | Read a value from an 'IORef'.
7073--
7174-- /Pre-release/
7275{-# INLINE readIORef #-}
7376readIORef :: Unbox a => IORef a -> IO a
74−readIORef (IORef var) = peekAt 0 var
77+readIORef (IORef var) = peekByteIndex 0 var
7578
7679-- | Modify the value of an 'IORef' using a function with strict application.
7780--
@@ -84,16 +87,10 @@ modifyIORef' var g = do
8487
8588-- | Generate a stream by continuously reading the IORef.
8689--
87−-- This operation reads the IORef without any synchronization. It can be
88−-- assumed to be atomic because the IORef (MutableByteArray) is always aligned
89−-- to Int boundaries, we are assuming that compiler uses single instructions to
90−-- access the memory. It may read stale values though until caches are
91−-- synchronised in a multiprocessor architecture.
92−--
9390-- /Pre-release/
94−{-# INLINE_NORMAL pollIntIORef #-}
95−pollIntIORef :: (MonadIO m, Unbox a) => IORef a -> D.Stream m a
96−pollIntIORef var = D.Stream step ()
91+{-# INLINE_NORMAL toStreamD #-}
92+toStreamD :: (MonadIO m, Unbox a) => IORef a -> D.Stream m a
93+toStreamD var = D.Stream step ()
9794
9895 where
9996
core/src/Streamly/Internal/Data/IsMap.hs−8
@@ -1,7 +1,3 @@
1−{-# LANGUAGE TypeFamilies #-}
2−-- Must come after TypeFamilies, otherwise it is re-enabled.
3−-- MonoLocalBinds enabled by TypeFamilies causes perf regressions in general.
4−{-# LANGUAGE NoMonoLocalBinds #-}
51-- |
62-- Module : Streamly.Internal.Data.IsMap
73-- Copyright : (c) 2022 Composewell Technologies
@@ -32,8 +28,6 @@ class IsMap f where
3228 mapDelete :: Key f -> f a -> f a
3329 mapUnion :: f a -> f a -> f a
3430 mapNull :: f a -> Bool
35− mapTraverseWithKey ::
36− Applicative t => (Key f -> a -> t b) -> f a -> t (f b)
3731
3832instance Ord k => IsMap (Map k) where
3933 type Key (Map k) = k
@@ -45,7 +39,6 @@ instance Ord k => IsMap (Map k) where
4539 mapDelete = Map.delete
4640 mapUnion = Map.union
4741 mapNull = Map.null
48− mapTraverseWithKey = Map.traverseWithKey
4942
5043instance IsMap IntMap.IntMap where
5144 type Key IntMap.IntMap = Int
@@ -57,4 +50,3 @@ instance IsMap IntMap.IntMap where
5750 mapDelete = IntMap.delete
5851 mapUnion = IntMap.union
5952 mapNull = IntMap.null
60− mapTraverseWithKey = IntMap.traverseWithKey
core/src/Streamly/Internal/Data/List.hs+5−5
@@ -66,19 +66,19 @@ where
6666import Control.Arrow (second)
6767import Data.Functor.Identity (Identity, runIdentity)
6868import GHC.Exts (IsList(..), IsString(..))
69−import Streamly.Internal.Data.Stream.Type
69+import Streamly.Internal.Data.Stream.StreamD.Type
7070 ( CrossStream
7171 , mkCross
7272 , unCross
7373 )
74−import Streamly.Internal.Data.Stream.Type (Stream)
74+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
7575import Text.Read
7676 ( Lexeme(Ident), lexP, parens, prec, readPrec, readListPrec
7777 , readListPrecDefault)
7878
79−import qualified Streamly.Internal.Data.StreamK.Type as K
80−import qualified Streamly.Internal.Data.Stream.Type as Stream
81−import qualified Streamly.Internal.Data.Stream.Generate as Stream
79+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
80+import qualified Streamly.Internal.Data.Stream.StreamD.Type as Stream
81+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as Stream
8282
8383-- XXX Rename to PureStream.
8484
core/src/Streamly/Internal/Data/MutArray.hs+33−154
@@ -8,200 +8,79 @@
88--
99module Streamly.Internal.Data.MutArray
1010 (
11− -- * MutArray.Type module
1211 module Streamly.Internal.Data.MutArray.Type
13− -- * MutArray module
14− , sliceIndexerFromLen
15− , slicerFromLen
16− , compactLE
17− , pinnedCompactLE
18− , compactOnByte
19− , compactOnByteSuffix
20− -- * Unboxed IORef
21− , module Streamly.Internal.Data.IORef.Unboxed
22−
23− -- * Deprecated
12+ , splitOn
2413 , genSlicesFromLen
2514 , getSlicesFromLen
15+ , fromStream
2616 )
2717where
2818
2919#include "inline.hs"
3020
3121import Control.Monad.IO.Class (MonadIO(..))
32−import Data.Word (Word8)
33−import Streamly.Internal.Data.MutByteArray.Type (PinnedState(..))
34−import Streamly.Internal.Data.Stream.Type (Stream)
3522import Streamly.Internal.Data.Unbox (Unbox)
23+import Streamly.Internal.Data.Stream (Stream)
3624import Streamly.Internal.Data.Unfold.Type (Unfold(..))
3725
38−import qualified Streamly.Internal.Data.Stream.Type as Stream
39−import qualified Streamly.Internal.Data.Stream.Nesting as Stream
40−-- import qualified Streamly.Internal.Data.Stream.Transform as Stream
26+import qualified Streamly.Internal.Data.Stream as D
4127import qualified Streamly.Internal.Data.Unfold as Unfold
4228
4329import Prelude hiding (foldr, length, read, splitAt)
4430import Streamly.Internal.Data.MutArray.Type
45−import Streamly.Internal.Data.IORef.Unboxed
31+
32+-- | Split the array into a stream of slices using a predicate. The element
33+-- matching the predicate is dropped.
34+--
35+-- /Pre-release/
36+{-# INLINE splitOn #-}
37+splitOn :: (MonadIO m, Unbox a) =>
38+ (a -> Bool) -> MutArray a -> Stream m (MutArray a)
39+splitOn predicate arr =
40+ fmap (\(i, len) -> getSliceUnsafe i len arr)
41+ $ D.sliceOnSuffix predicate (read arr)
4642
4743-- | Generate a stream of array slice descriptors ((index, len)) of specified
4844-- length from an array, starting from the supplied array index. The last slice
4945-- may be shorter than the requested length depending on the array length.
5046--
5147-- /Pre-release/
52−{-# INLINE sliceIndexerFromLen #-}
53−sliceIndexerFromLen :: forall m a. (Monad m, Unbox a)
48+{-# INLINE genSlicesFromLen #-}
49+genSlicesFromLen :: forall m a. (Monad m, Unbox a)
5450 => Int -- ^ from index
5551 -> Int -- ^ length of the slice
5652 -> Unfold m (MutArray a) (Int, Int)
57−sliceIndexerFromLen from len =
53+genSlicesFromLen from len =
5854 let fromThenTo n = (from, from + len, n - 1)
5955 mkSlice n i = return (i, min len (n - i))
6056 in Unfold.lmap length
6157 $ Unfold.mapM2 mkSlice
6258 $ Unfold.lmap fromThenTo Unfold.enumerateFromThenTo
6359
64−{-# DEPRECATED genSlicesFromLen "Please use sliceIndexerFromLen instead." #-}
65−genSlicesFromLen :: forall m a. (Monad m, Unbox a)
66− => Int -- ^ from index
67− -> Int -- ^ length of the slice
68− -> Unfold m (MutArray a) (Int, Int)
69−genSlicesFromLen = sliceIndexerFromLen
70−
7160-- | Generate a stream of slices of specified length from an array, starting
7261-- from the supplied array index. The last slice may be shorter than the
7362-- requested length depending on the array length.
7463--
7564-- /Pre-release/
76−{-# INLINE slicerFromLen #-}
77−slicerFromLen :: forall m a. (Monad m, Unbox a)
78− => Int -- ^ from index
79− -> Int -- ^ length of the slice
80− -> Unfold m (MutArray a) (MutArray a)
81−slicerFromLen from len =
82− let mkSlice arr (i, n) = return $ getSliceUnsafe i n arr
83− in Unfold.mapM2 mkSlice (sliceIndexerFromLen from len)
84−
85−{-# DEPRECATED getSlicesFromLen "Please use slicerFromLen instead." #-}
65+{-# INLINE getSlicesFromLen #-}
8666getSlicesFromLen :: forall m a. (Monad m, Unbox a)
8767 => Int -- ^ from index
8868 -> Int -- ^ length of the slice
8969 -> Unfold m (MutArray a) (MutArray a)
90−getSlicesFromLen = slicerFromLen
91−
92−-------------------------------------------------------------------------------
93−-- Compacting Streams of Arrays
94−-------------------------------------------------------------------------------
70+getSlicesFromLen from len =
71+ let mkSlice arr (i, n) = return $ getSliceUnsafe i n arr
72+ in Unfold.mapM2 mkSlice (genSlicesFromLen from len)
9573
96−-- | @compactLE maxElems@ coalesces adjacent arrays in the input stream
97−-- only if the combined size would be less than or equal to @maxElems@
98−-- elements. Note that it won't split an array if the original array is already
99−-- larger than maxElems.
100−--
101−-- @maxElems@ must be greater than 0.
74+-- | Create an 'Array' from a stream. This is useful when we want to create a
75+-- single array from a stream of unknown size. 'writeN' is at least twice
76+-- as efficient when the size is already known.
10277--
103−-- Generates unpinned arrays irrespective of the pinning status of input
104−-- arrays.
105−{-# INLINE compactLE #-}
106−compactLE :: (MonadIO m, Unbox a) =>
107− Int -> Stream m (MutArray a) -> Stream m (MutArray a)
108−-- XXX compactLE can be moved to MutArray/Type if we are not using the parser
109−-- to implement it.
110−compactLE = compactLeAs Unpinned
111−-- The parser version turns out to be a little bit slower.
112−-- compactLE n = Stream.catRights . Stream.parseManyD (pCompactLE n)
113−
114−-- | Pinned version of 'compactLE'.
115−{-# INLINE pinnedCompactLE #-}
116−pinnedCompactLE :: forall m a. (MonadIO m, Unbox a)
117− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
118−pinnedCompactLE = compactLeAs Pinned
119−-- pinnedCompactLE n = Stream.catRights . Stream.parseManyD (pPinnedCompactLE n)
120−
121−data SplitState s arr
122− = Initial s
123− | Buffering s arr
124− | Splitting s arr
125− | Yielding arr (SplitState s arr)
126− | Finishing
127−
128−-- | Split a stream of arrays on a given separator byte, dropping the separator
129−-- and coalescing all the arrays between two separators into a single array.
78+-- Note that if the input stream is too large memory allocation for the array
79+-- may fail. When the stream size is not known, `chunksOf` followed by
80+-- processing of indvidual arrays in the resulting stream should be preferred.
13081--
131−{-# INLINE_NORMAL _compactOnByteCustom #-}
132−_compactOnByteCustom
133− :: MonadIO m
134− => Word8
135− -> Stream m (MutArray Word8)
136− -> Stream m (MutArray Word8)
137−_compactOnByteCustom byte (Stream.Stream step state) =
138− Stream.Stream step' (Initial state)
139−
140− where
141−
142− {-# INLINE_LATE step' #-}
143− step' gst (Initial st) = do
144− r <- step gst st
145− case r of
146− Stream.Yield arr s -> do
147− (arr1, marr2) <- breakOn byte arr
148− return $ case marr2 of
149− Nothing -> Stream.Skip (Buffering s arr1)
150− Just arr2 -> Stream.Skip (Yielding arr1 (Splitting s arr2))
151− Stream.Skip s -> return $ Stream.Skip (Initial s)
152− Stream.Stop -> return Stream.Stop
153−
154− step' gst (Buffering st buf) = do
155− r <- step gst st
156− case r of
157− Stream.Yield arr s -> do
158− (arr1, marr2) <- breakOn byte arr
159− -- XXX Use spliceExp instead and then rightSize?
160− buf1 <- splice buf arr1
161− return $ case marr2 of
162− Nothing -> Stream.Skip (Buffering s buf1)
163− Just x -> Stream.Skip (Yielding buf1 (Splitting s x))
164− Stream.Skip s -> return $ Stream.Skip (Buffering s buf)
165− Stream.Stop -> return $
166− if byteLength buf == 0
167− then Stream.Stop
168− else Stream.Skip (Yielding buf Finishing)
169−
170− step' _ (Splitting st buf) = do
171− (arr1, marr2) <- breakOn byte buf
172− return $ case marr2 of
173− Nothing -> Stream.Skip $ Buffering st arr1
174− Just arr2 -> Stream.Skip $ Yielding arr1 (Splitting st arr2)
175−
176− step' _ (Yielding arr next) = return $ Stream.Yield arr next
177− step' _ Finishing = return Stream.Stop
178−
179−-- XXX implement predicate based version of this
180−-- XXX Naming of predicate based vs custom version
181−
182−-- | Split a stream of arrays on a given separator byte, dropping the separator
183−-- and coalescing all the arrays between two separators into a single array.
184−--
185−{-# INLINE compactOnByte #-}
186−compactOnByte
187− :: (MonadIO m)
188− => Word8
189− -> Stream m (MutArray Word8)
190− -> Stream m (MutArray Word8)
191−-- XXX compare perf of custom vs idiomatic version
192−-- compactOnByte = _compactOnByteCustom
193−-- XXX use spliceExp and rightSize?
194−compactOnByte byte = Stream.splitInnerBy (breakOn byte) splice
195−
196−-- | Like 'compactOnByte' considers the separator in suffix position instead of
197−-- infix position.
198−{-# INLINE compactOnByteSuffix #-}
199−compactOnByteSuffix
200− :: (MonadIO m)
201− => Word8
202− -> Stream m (MutArray Word8)
203− -> Stream m (MutArray Word8)
204−compactOnByteSuffix byte =
205− -- XXX use spliceExp and rightSize?
206− Stream.splitInnerBySuffix
207− (\arr -> byteLength arr == 0) (breakOn byte) splice
82+-- /Pre-release/
83+{-# INLINE fromStream #-}
84+fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
85+fromStream = fromStreamD
86+-- fromStream (Stream m) = P.fold write m
core/src/Streamly/Internal/Data/MutArray/Generic.hs+49−172
@@ -19,14 +19,14 @@ module Streamly.Internal.Data.MutArray.Generic
1919 , nil
2020
2121 -- *** Uninitialized Arrays
22− , emptyOf
22+ , new
2323 -- , newArrayWith
2424
2525 -- *** From streams
26− , unsafeCreateOf
27− , createOf
28− , createWith -- createOfMin/createMin/createGE?
29− , create
26+ , writeNUnsafe
27+ , writeN
28+ , writeWith
29+ , write
3030 , fromStreamN
3131 , fromStream
3232 , fromPureStream
@@ -101,7 +101,6 @@ module Streamly.Internal.Data.MutArray.Generic
101101 -- ** Random reads
102102 , getIndex
103103 , getIndexUnsafe
104− , getIndexUnsafeWith
105104 -- , getIndices
106105 -- , getFromThenTo
107106 -- , getIndexRev
@@ -133,7 +132,7 @@ module Streamly.Internal.Data.MutArray.Generic
133132 -- multidimensional array representations.
134133
135134 -- ** Construct from streams
136− , chunksOf
135+ -- , chunksOf
137136 -- , arrayStreamKFromStreamD
138137 -- , writeChunks
139138
@@ -160,13 +159,6 @@ module Streamly.Internal.Data.MutArray.Generic
160159 -- , appendSliceFrom
161160
162161 , clone
163−
164− -- * Deprecated
165− , new
166− , writeNUnsafe
167− , writeN
168− , writeWith
169− , write
170162 )
171163where
172164
@@ -189,15 +181,14 @@ import GHC.Int (Int(..))
189181import Streamly.Internal.Data.Fold.Type (Fold(..))
190182import Streamly.Internal.Data.Producer.Type (Producer (..))
191183import Streamly.Internal.Data.Unfold.Type (Unfold(..))
192−import Streamly.Internal.Data.Stream.Type (Stream)
193−import Streamly.Internal.Data.SVar.Type (adaptState)
184+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
194185
195186import qualified Streamly.Internal.Data.Fold.Type as FL
196187import qualified Streamly.Internal.Data.Producer as Producer
197−import qualified Streamly.Internal.Data.Stream.Type as D
198−import qualified Streamly.Internal.Data.Stream.Generate as D
199−import qualified Streamly.Internal.Data.Stream.Lift as D
200−import qualified Streamly.Internal.Data.StreamK.Type as K
188+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
189+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
190+import qualified Streamly.Internal.Data.Stream.StreamD.Lift as D
191+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
201192
202193import Prelude hiding (read, length)
203194
@@ -243,13 +234,13 @@ bottomElement =
243234-- XXX Would be nice if GHC can provide something like newUninitializedArray# so
244235-- that we do not have to write undefined or error in the whole array.
245236
246−-- | @emptyOf count@ allocates a zero length array that can be extended to hold
237+-- | @new count@ allocates a zero length array that can be extended to hold
247238-- up to 'count' items without reallocating.
248239--
249240-- /Pre-release/
250−{-# INLINE emptyOf #-}
251−emptyOf :: MonadIO m => Int -> m (MutArray a)
252−emptyOf n@(I# n#) =
241+{-# INLINE new #-}
242+new :: MonadIO m => Int -> m (MutArray a)
243+new n@(I# n#) =
253244 liftIO
254245 $ IO
255246 $ \s# ->
@@ -258,12 +249,6 @@ emptyOf n@(I# n#) =
258249 let ma = MutArray arr# 0 0 n
259250 in (# s1#, ma #)
260251
261−-- XXX Deprecate in major
262−-- {-# DEPRECATED new "Please use emptyOf instead." #-}
263−{-# INLINE new #-}
264−new :: MonadIO m => Int -> m (MutArray a)
265−new = emptyOf
266−
267252-- XXX This could be pure?
268253
269254-- |
@@ -278,21 +263,6 @@ nil = new 0
278263-- Random writes
279264-------------------------------------------------------------------------------
280265
281−-- | Write the given element to the given index of the 'MutableArray#'. Does not
282−-- check if the index is out of bounds of the array.
283−--
284−-- /Pre-release/
285−{-# INLINE putIndexUnsafeWith #-}
286−putIndexUnsafeWith :: MonadIO m => Int -> MutableArray# RealWorld a -> a -> m ()
287−putIndexUnsafeWith n _arrContents# x =
288− liftIO
289− $ IO
290− $ \s# ->
291− case n of
292− I# n# ->
293− let s1# = writeArray# _arrContents# n# x s#
294− in (# s1#, () #)
295−
296266-- | Write the given element to the given index of the array. Does not check if
297267-- the index is out of bounds of the array.
298268--
@@ -301,7 +271,13 @@ putIndexUnsafeWith n _arrContents# x =
301271putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
302272putIndexUnsafe i MutArray {..} x =
303273 assert (i >= 0 && i < arrLen)
304− putIndexUnsafeWith (i + arrStart) arrContents# x
274+ (liftIO
275+ $ IO
276+ $ \s# ->
277+ case i + arrStart of
278+ I# n# ->
279+ let s1# = writeArray# arrContents# n# x s#
280+ in (# s1#, () #))
305281
306282invalidIndex :: String -> Int -> a
307283invalidIndex label i =
@@ -331,7 +307,7 @@ putIndices arr = FL.foldlM' step (return ())
331307
332308 where
333309
334− step () (i, x) = putIndex i arr x
310+ step () (i, x) = liftIO (putIndex i arr x)
335311
336312-- | Modify a given index of an array using a modifier function without checking
337313-- the bounds.
@@ -480,25 +456,17 @@ uninit arr@MutArray{..} len =
480456-- Random reads
481457-------------------------------------------------------------------------------
482458
483−-- | Return the element at the specified index without checking the bounds from
484−-- a @MutableArray# RealWorld@.
485−--
486−-- Unsafe because it does not check the bounds of the array.
487−{-# INLINE getIndexUnsafeWith #-}
488−getIndexUnsafeWith :: MonadIO m => MutableArray# RealWorld a -> Int -> m a
489−getIndexUnsafeWith _arrContents# n =
490− liftIO
491− $ IO
492− $ \s# ->
493− let !(I# i#) = n
494− in readArray# _arrContents# i# s#
495−
496459-- | Return the element at the specified index without checking the bounds.
497460--
498461-- Unsafe because it does not check the bounds of the array.
499462{-# INLINE_NORMAL getIndexUnsafe #-}
500463getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
501−getIndexUnsafe n MutArray {..} = getIndexUnsafeWith arrContents# (n + arrStart)
464+getIndexUnsafe n MutArray {..} =
465+ liftIO
466+ $ IO
467+ $ \s# ->
468+ let !(I# i#) = arrStart + n
469+ in readArray# arrContents# i# s#
502470
503471-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
504472--
@@ -602,16 +570,16 @@ readRev arr@MutArray{..} =
602570arrayChunkSize :: Int
603571arrayChunkSize = 1024
604572
605−-- | Like 'createOf' but does not check the array bounds when writing. The fold
573+-- | Like 'writeN' but does not check the array bounds when writing. The fold
606574-- driver must not call the step function more than 'n' times otherwise it will
607575-- corrupt the memory and crash. This function exists mainly because any
608576-- conditional in the step function blocks fusion causing 10x performance
609577-- slowdown.
610578--
611579-- /Pre-release/
612−{-# INLINE_NORMAL unsafeCreateOf #-}
613−unsafeCreateOf :: MonadIO m => Int -> Fold m a (MutArray a)
614−unsafeCreateOf n = Fold step initial return return
580+{-# INLINE_NORMAL writeNUnsafe #-}
581+writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
582+writeNUnsafe n = Fold step initial return
615583
616584 where
617585
@@ -619,81 +587,59 @@ unsafeCreateOf n = Fold step initial return return
619587
620588 step arr x = FL.Partial <$> snocUnsafe arr x
621589
622−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
623−{-# INLINE writeNUnsafe #-}
624−writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
625−writeNUnsafe = unsafeCreateOf
626−
627−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
590+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
628591-- 'Array'.
629592--
630−-- >>> createOf n = Fold.take n (MutArray.unsafeCreateOf n)
593+-- >>> writeN n = Fold.take n (MutArray.writeNUnsafe n)
631594--
632595-- /Pre-release/
633−{-# INLINE_NORMAL createOf #-}
634−createOf :: MonadIO m => Int -> Fold m a (MutArray a)
635−createOf n = FL.take n $ unsafeCreateOf n
636−
637−-- XXX Deprecate in major
638−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
639−{-# INLINE writeN #-}
596+{-# INLINE_NORMAL writeN #-}
640597writeN :: MonadIO m => Int -> Fold m a (MutArray a)
641−writeN = createOf
598+writeN n = FL.take n $ writeNUnsafe n
642599
643600-- >>> f n = MutArray.writeAppendWith (* 2) (MutArray.pinnedNew n)
644601-- >>> writeWith n = Fold.rmapM MutArray.rightSize (f n)
645602-- >>> writeWith n = Fold.rmapM MutArray.fromArrayStreamK (MutArray.writeChunks n)
646603
647−-- | @createWith minCount@ folds the whole input to a single array. The array
604+-- | @writeWith minCount@ folds the whole input to a single array. The array
648605-- starts at a size big enough to hold minCount elements, the size is doubled
649606-- every time the array needs to be grown.
650607--
651608-- /Caution! Do not use this on infinite streams./
652609--
653610-- /Pre-release/
654−{-# INLINE_NORMAL createWith #-}
655−createWith :: MonadIO m => Int -> Fold m a (MutArray a)
611+{-# INLINE_NORMAL writeWith #-}
612+writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
656613-- writeWith n = FL.rmapM rightSize $ writeAppendWith (* 2) (pinnedNew n)
657−createWith elemCount = FL.rmapM extract $ FL.foldlM' step initial
614+writeWith elemCount = FL.rmapM extract $ FL.foldlM' step initial
658615
659616 where
660617
661618 initial = do
662− when (elemCount < 0) $ error "createWith: elemCount is negative"
663− new elemCount
619+ when (elemCount < 0) $ error "writeWith: elemCount is negative"
620+ liftIO $ new elemCount
664621
665622 step arr@(MutArray _ start end bound) x
666623 | end == bound = do
667624 let oldSize = end - start
668625 newSize = max (oldSize * 2) 1
669− arr1 <- realloc newSize arr
626+ arr1 <- liftIO $ realloc newSize arr
670627 snocUnsafe arr1 x
671628 step arr x = snocUnsafe arr x
672629
673− -- extract = rightSize
630+ -- extract = liftIO . rightSize
674631 extract = return
675632
676−{-# DEPRECATED writeWith "Please use createWith instead." #-}
677−{-# INLINE writeWith #-}
678−writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
679−writeWith = createWith
680−
681633-- | Fold the whole input to a single array.
682634--
683−-- Same as 'createWith' using an initial array size of 'arrayChunkSize' bytes
635+-- Same as 'writeWith' using an initial array size of 'arrayChunkSize' bytes
684636-- rounded up to the element size.
685637--
686638-- /Caution! Do not use this on infinite streams./
687639--
688−{-# INLINE create #-}
689−create :: MonadIO m => Fold m a (MutArray a)
690−create = writeWith arrayChunkSize
691−
692−-- XXX Deprecate in major
693−-- {-# DEPRECATED write "Please use create instead." #-}
694640{-# INLINE write #-}
695641write :: MonadIO m => Fold m a (MutArray a)
696−write = create
642+write = writeWith arrayChunkSize
697643
698644-- | Create a 'MutArray' from the first @n@ elements of a stream. The
699645-- array is allocated to size @n@, if the stream terminates before @n@
@@ -718,76 +664,7 @@ fromList xs = fromStream $ D.fromList xs
718664{-# INLINABLE fromPureStream #-}
719665fromPureStream :: MonadIO m => Stream Identity a -> m (MutArray a)
720666fromPureStream xs =
721− D.fold write $ D.morphInner (return . runIdentity) xs
722−
723−-------------------------------------------------------------------------------
724−-- Chunking
725−-------------------------------------------------------------------------------
726−
727−data GroupState s a start end bound
728− = GroupStart s
729− | GroupBuffer s (MutableArray# RealWorld a) start end bound
730− | GroupYield
731− (MutableArray# RealWorld a)
732− start
733− end
734− bound
735− (GroupState s a start end bound)
736− | GroupFinish
737−
738−-- | @chunksOf n stream@ groups the input stream into a stream of
739−-- arrays of size n.
740−--
741−-- @chunksOf n = foldMany (MutArray.writeN n)@
742−--
743−-- /Pre-release/
744−{-# INLINE_NORMAL chunksOf #-}
745−chunksOf :: forall m a. MonadIO m
746− => Int -> D.Stream m a -> D.Stream m (MutArray a)
747−-- XXX the idiomatic implementation leads to large regression in the D.reverse'
748−-- benchmark. It seems it has difficulty producing optimized code when
749−-- converting to StreamK. Investigate GHC optimizations.
750−-- chunksOf n = D.foldMany (writeN n)
751−chunksOf n (D.Stream step state) =
752− D.Stream step' (GroupStart state)
753−
754− where
755−
756− -- start is always 0
757− -- end and len are always equal
758−
759− {-# INLINE_LATE step' #-}
760− step' _ (GroupStart st) = do
761− when (n <= 0) $
762− -- XXX we can pass the module string from the higher level API
763− error $ "Streamly.Internal.Data.Array.Generic.Mut.Type.chunksOf: "
764− ++ "the size of arrays [" ++ show n
765− ++ "] must be a natural number"
766− (MutArray contents start end bound :: MutArray a) <- new n
767− return $ D.Skip (GroupBuffer st contents start end bound)
768−
769− step' gst (GroupBuffer st contents start end bound) = do
770− r <- step (adaptState gst) st
771− case r of
772− D.Yield x s -> do
773− putIndexUnsafeWith end contents x
774− let end1 = end + 1
775− return $
776− if end1 >= bound
777− then D.Skip
778− (GroupYield
779− contents start end1 bound (GroupStart s))
780− else D.Skip (GroupBuffer s contents start end1 bound)
781− D.Skip s ->
782− return $ D.Skip (GroupBuffer s contents start end bound)
783− D.Stop ->
784− return
785− $ D.Skip (GroupYield contents start end bound GroupFinish)
786−
787− step' _ (GroupYield contents start end bound next) =
788− return $ D.Yield (MutArray contents start end bound) next
789−
790− step' _ GroupFinish = return D.Stop
667+ liftIO $ D.fold write $ D.morphInner (return . runIdentity) xs
791668
792669-------------------------------------------------------------------------------
793670-- Unfolds
@@ -850,7 +727,7 @@ putSliceUnsafe src srcStart dst dstStart len = liftIO $ do
850727
851728{-# INLINE clone #-}
852729clone :: MonadIO m => MutArray a -> m (MutArray a)
853−clone src = do
730+clone src = liftIO $ do
854731 let len = arrLen src
855732 dst <- new len
856733 putSliceUnsafe src 0 dst 0 len
core/src/Streamly/Internal/Data/MutArray/Stream.hsdeleted−145
@@ -1,145 +0,0 @@
1−-- |
2−-- Module : Streamly.Internal.Data.MutArray.Stream
3−-- Copyright : (c) 2019 Composewell Technologies
4−-- License : BSD3-3-Clause
5−-- Maintainer : streamly@composewell.com
6−-- Stability : experimental
7−-- Portability : GHC
8−--
9−-- Combinators to efficiently manipulate streams of mutable arrays.
10−--
11−-- We can either push these in the MutArray module with a "chunks" prefix or
12−-- keep this as a separate module and release it.
13−--
14−module Streamly.Internal.Data.MutArray.Stream
15−{-# DEPRECATED "Please use \"Streamly.Internal.Data.MutArray\" instead." #-}
16− (
17− -- * Generation
18− MArray.chunksOf
19− , MArray.pinnedChunksOf
20− , MArray.writeChunks -- chunksWrite?
21− , MArray.splitOn -- chunksSplitOn
22−
23− -- * Compaction
24− , packArraysChunksOf
25− , MArray.SpliceState (..)
26− , lpackArraysChunksOf
27− , compact -- chunksCompact
28− , compactLE
29− , compactEQ
30− , compactGE
31−
32− -- * Elimination
33− , MArray.flattenArrays -- chunksConcat
34− , MArray.flattenArraysRev -- chunksConcatRev
35− , MArray.fromArrayStreamK -- chunksCoalesce
36− )
37−where
38−
39−import Control.Monad.IO.Class (MonadIO(..))
40−import Streamly.Internal.Data.Unbox (Unbox)
41−import Streamly.Internal.Data.MutArray.Type (MutArray(..))
42−import Streamly.Internal.Data.Fold.Type (Fold(..))
43−import Streamly.Internal.Data.Parser (ParseError)
44−import Streamly.Internal.Data.Stream.Type (Stream)
45−
46−import qualified Streamly.Internal.Data.MutArray as MArray
47−import qualified Streamly.Internal.Data.Fold.Type as FL
48−import qualified Streamly.Internal.Data.Parser as ParserD
49−import qualified Streamly.Internal.Data.Stream as D
50−
51−-------------------------------------------------------------------------------
52−-- Compact
53−-------------------------------------------------------------------------------
54−
55−-- XXX This can be removed once compactLEFold/compactLE are implemented.
56−--
57−-- | This mutates the first array (if it has space) to append values from the
58−-- second one. This would work for immutable arrays as well because an
59−-- immutable array never has space so a new array is allocated instead of
60−-- mutating it.
61−--
62−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
63−-- maximum specified size. Note that if a single array is bigger than the
64−-- specified size we do not split it to fit. When we coalesce multiple arrays
65−-- if the size would exceed the specified size we do not coalesce therefore the
66−-- actual array size may be less than the specified chunk size.
67−--
68−-- @since 0.7.0
69−{-# INLINE packArraysChunksOf #-}
70−packArraysChunksOf :: (MonadIO m, Unbox a)
71− => Int -> D.Stream m (MutArray a) -> D.Stream m (MutArray a)
72−packArraysChunksOf = MArray.compactLE
73−
74−-- XXX Remove this once compactLEFold is implemented
75−-- lpackArraysChunksOf = Fold.many compactLEFold
76−--
77−{-# INLINE lpackArraysChunksOf #-}
78−lpackArraysChunksOf :: (MonadIO m, Unbox a)
79− => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
80−lpackArraysChunksOf = MArray.lCompactGE
81−
82−-- XXX Same as compactLE, to be removed once that is implemented.
83−--
84−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
85−-- maximum specified size in bytes.
86−--
87−-- /Internal/
88−{-# INLINE compact #-}
89−compact :: (MonadIO m, Unbox a)
90− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
91−compact = packArraysChunksOf
92−
93−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
94−-- maximum specified size. Note that if a single array is bigger than the
95−-- specified size we do not split it to fit. When we coalesce multiple arrays
96−-- if the size would exceed the specified size we do not coalesce therefore the
97−-- actual array size may be less than the specified chunk size.
98−--
99−-- /Internal/
100−{-# INLINE compactLEParserD #-}
101−compactLEParserD ::
102− forall m a. (MonadIO m, Unbox a)
103− => Int -> ParserD.Parser (MutArray a) m (MutArray a)
104−compactLEParserD = MArray.pCompactLE
105−
106−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
107−-- minimum specified size. Note that if all the arrays in the stream together
108−-- are smaller than the specified size the resulting array will be smaller than
109−-- the specified size. When we coalesce multiple arrays if the size would exceed
110−-- the specified size we stop coalescing further.
111−--
112−-- /Internal/
113−{-# INLINE compactGEFold #-}
114−compactGEFold ::
115− forall m a. (MonadIO m, Unbox a)
116− => Int -> FL.Fold m (MutArray a) (MutArray a)
117−compactGEFold = MArray.fCompactGE
118−
119−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
120−-- maximum specified size in bytes.
121−--
122−-- /Internal/
123−compactLE :: (MonadIO m, Unbox a) =>
124− Int -> Stream m (MutArray a) -> Stream m (Either ParseError (MutArray a))
125−compactLE n = D.parseManyD (compactLEParserD n)
126−
127−-- | Like 'compactLE' but generates arrays of exactly equal to the size
128−-- specified except for the last array in the stream which could be shorter.
129−--
130−-- /Unimplemented/
131−{-# INLINE compactEQ #-}
132−compactEQ :: -- (MonadIO m, Unbox a) =>
133− Int -> Stream m (MutArray a) -> Stream m (MutArray a)
134−compactEQ _n _xs = undefined
135− -- IsStream.fromStreamD $ D.foldMany (compactEQFold n) (IsStream.toStreamD xs)
136−
137−-- | Like 'compactLE' but generates arrays of size greater than or equal to the
138−-- specified except for the last array in the stream which could be shorter.
139−--
140−-- /Internal/
141−{-# INLINE compactGE #-}
142−compactGE ::
143− (MonadIO m, Unbox a)
144− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
145−compactGE n = D.foldMany (compactGEFold n)
core/src/Streamly/Internal/Data/MutArray/Type.hs+441−1308
@@ -28,80 +28,57 @@
2828
2929module Streamly.Internal.Data.MutArray.Type
3030 (
31− -- ** Type
31+ -- * Type
3232 -- $arrayNotes
3333 MutArray (..)
34−
35− -- ** Conversion
36− -- *** Pinned and Unpinned
34+ , MutableByteArray
35+ , touch
3736 , pin
3837 , unpin
3938 , isPinned
4039
41− -- ** Casting
42− , cast
43− , castUnsafe -- XXX unsafeCast
44− , asBytes
45− , unsafePinnedAsPtr
46− , unsafeAsPtr
47−
40+ -- * Constructing and Writing
4841 -- ** Construction
49− , empty
50−
51− -- *** New
52− -- | New arrays are always empty arrays with some reserve capacity to
53− -- extend the length without reallocating.
54− , emptyOf
55− , newArrayWith -- emptyAlignedWith
56− , pinnedEmptyOf
57− , pinnedNewAligned -- XXX not required
58− -- , new -- uninitialized array of specified length
59−
60− -- *** Cloning
61− , clone
62− , pinnedClone
63−
64− -- *** Slicing
65− -- | Get a subarray without copying
66− , getSliceUnsafe -- XXX unsafeGetSlice
67− , getSlice
68− , splitAt -- XXX should be able to express using getSlice
69− , breakOn
42+ , nil
43+
44+ -- *** Uninitialized Arrays
45+ , pinnedNew
46+ , pinnedNewBytes
47+ , pinnedNewAligned
48+ , new
49+ , newArrayWith
7050
71− -- *** Stream Folds
51+ -- *** From streams
7252 , ArrayUnsafe (..)
73− , unsafeCreateOfWith
74− , unsafeCreateOf
75− , unsafePinnedCreateOf
76− , pinnedCreateOf
77− , createOfWith
78− , createOf
79− , revCreateOf
80−
81− , pinnedCreate
82− , createWith
83− , create
84− -- , revCreate
85−
86− -- *** From containers
53+ , writeNWithUnsafe
54+ , writeNWith
55+ , writeNUnsafe
56+ , pinnedWriteNUnsafe
57+ , writeN
58+ , pinnedWriteN
59+ , pinnedWriteNAligned
60+
61+ , writeWith
62+ , write
63+ , pinnedWrite
64+
65+ , writeRevN
66+ -- , writeRev
67+
68+ -- ** From containers
8769 , fromListN
8870 , pinnedFromListN
8971 , fromList
9072 , pinnedFromList
9173 , fromListRevN
9274 , fromListRev
93− , fromStreamN
94− , fromStream
95− , fromPureStreamN
75+ , fromStreamDN
76+ , fromStreamD
9677 , fromPureStream
97− , fromByteStr#
98− , fromPtrN
99− , fromChunksK
100− , fromChunksRealloced -- fromSmallChunks
10178
102− -- ** Random writes
79+ -- * Random writes
10380 , putIndex
104− , putIndexUnsafe -- XXX unsafePutIndex
81+ , putIndexUnsafe
10582 , putIndices
10683 -- , putFromThenTo
10784 -- , putFrom -- start writing at the given position
@@ -109,71 +86,78 @@ module Streamly.Internal.Data.MutArray.Type
10986 -- , putFromTo
11087 -- , putFromRev
11188 -- , putUptoRev
112− , modifyIndexUnsafe -- XXX unsafeModifyIndex
89+ , modifyIndexUnsafe
11390 , modifyIndex
11491 , modifyIndices
11592 , modify
11693 , swapIndices
11794 , unsafeSwapIndices
11895
119− -- ** Reading
96+ -- * Growing and Shrinking
97+ -- Arrays grow only at the end, though it is possible to grow on both sides
98+ -- and therefore have a cons as well as snoc. But that will require two
99+ -- bounds in the array representation.
120100
121− -- *** Indexing
122− , getIndex
123− , getIndexUnsafe -- XXX unsafeGetIndex
124− -- , getFromThenTo
125− , getIndexRev -- getRevIndex?
126− , indexReader
127− , indexReaderWith
101+ -- ** Appending elements
102+ , snocWith
103+ , snoc
104+ , snocLinear
105+ , snocMay
106+ , snocUnsafe
107+
108+ -- ** Appending streams
109+ , writeAppendNUnsafe
110+ , writeAppendN
111+ , writeAppendWith
112+ , writeAppend
113+
114+ -- * Eliminating and Reading
128115
129− -- *** To Streams
116+ -- ** To streams
117+ , reader
118+ , readerRevWith
119+ , readerRev
120+
121+ -- ** To containers
122+ , toStreamDWith
123+ , toStreamDRevWith
124+ , toStreamKWith
125+ , toStreamKRevWith
130126 , read
131127 , readRev
132− , toStreamWith
133− , toStreamRevWith
134128 , toStreamK
135− , toStreamKWith
136129 , toStreamKRev
137− , toStreamKRevWith
138−
139− -- *** To Containers
140130 , toList
141131
142− -- *** Unfolds
143132 -- experimental
144133 , producerWith
145134 , producer
146135
147− , reader
148− , readerRevWith
149− , readerRev
150−
151− -- ** Size and Capacity
152− -- *** Size
153− , length
154− , byteLength
155−
156− -- *** Capacity
157− -- , capacity
158− , byteCapacity
159− , bytesFree
136+ -- ** Random reads
137+ , getIndex
138+ , getIndexUnsafe
139+ , getIndices
140+ , getIndicesD
141+ -- , getFromThenTo
142+ , getIndexRev
160143
161− -- *** Capacity Management
144+ -- * Memory Management
162145 , blockSize
163146 , arrayChunkBytes
164147 , allocBytesToElemCount
165− , realloc -- this is "resize"
166− , grow
167− , growExp
148+ , realloc
149+ , resize
150+ , resizeExp
168151 , rightSize
169152
170− -- ** Folding
171− , foldl'
172− , foldr
173− , byteCmp
174− , byteEq
153+ -- * Size
154+ , length
155+ , byteLength
156+ -- , capacity
157+ , byteCapacity
158+ , bytesFree
175159
176− -- ** In-place Mutation Algorithms
160+ -- * In-place Mutation Algorithms
177161 , strip
178162 , reverse
179163 , permute
@@ -183,121 +167,59 @@ module Streamly.Internal.Data.MutArray.Type
183167 , mergeBy
184168 , bubble
185169
186− -- ** Growing and Shrinking
187− -- | Arrays grow only at the end, though it is possible to grow on both
188− -- sides and therefore have a cons as well as snoc. But that will require
189− -- both lower and upper bound in the array representation.
190−
191− -- *** Appending elements
192− , snocWith
193− , snoc
194− , snocLinear
195− , snocMay
196− , snocUnsafe -- XXX unsafeSnoc
197−
198− -- *** Appending streams
199− , unsafeAppendN
200− , appendN
201− , appendWith
202− , append
203−
204− -- *** Appending arrays
205− , spliceCopy
206− , spliceWith
207− , splice
208− , spliceExp
209− , spliceUnsafe -- XXX unsafeSplice
210− -- , putSlice
211− -- , appendSlice
212− -- , appendSliceFrom
213−
214− -- ** Serialization using Unbox
215− , pokeAppend
216− , pokeAppendMay
217− , pokeSkipUnsafe -- unsafePokeSkip
170+ -- * Casting
171+ , cast
172+ , castUnsafe
173+ , asBytes
174+ , asPtrUnsafe
218175
219− -- ** Deserialization using Unbox
220− , peekUncons
221− , peekUnconsUnsafe -- unsafePeekUncons
222− , peekSkipUnsafe -- unsafePeekSkip
176+ -- * Folding
177+ , foldl'
178+ , foldr
179+ , cmp
223180
224− -- Arrays of arrays
181+ -- * Arrays of arrays
225182 -- We can add dimensionality parameter to the array type to get
226183 -- multidimensional arrays. Multidimensional arrays would just be a
227184 -- convenience wrapper on top of single dimensional arrays.
228185
229− -- ** Streams of Arrays
230− -- *** Chunk
231− -- | Group a stream into arrays.
186+ -- | Operations dealing with multiple arrays, streams of arrays or
187+ -- multidimensional array representations.
188+
189+ -- ** Construct from streams
232190 , chunksOf
233191 , pinnedChunksOf
234− , buildChunks
192+ , writeChunks
235193
236− -- *** Split
237− -- | Split an array into slices.
194+ -- ** Eliminate to streams
195+ , flattenArrays
196+ , flattenArraysRev
197+ , fromArrayStreamK
238198
199+ -- ** Construct from arrays
200+ -- get chunks without copying
201+ , getSliceUnsafe
202+ , getSlice
239203 -- , getSlicesFromLenN
240− , splitOn
241− -- , slicesOf
242−
243− -- *** Concat
244− -- | Append the arrays in a stream to form a stream of elements.
245− , concatWith
246− , concatRevWith
247− , concat
248− , concatRev
249−
250− -- *** Compact
251− -- | Append the arrays in a stream to form a stream of larger arrays.
252− , SpliceState (..)
253− , pCompactLE
254− , pPinnedCompactLE
255− , compactLeAs
256− , fCompactGE
257− , fPinnedCompactGE
258− , lCompactGE
259− , lPinnedCompactGE
260− , compactGE
261− , compactEQ
262−
263− -- ** Utilities
204+ , splitAt -- XXX should be able to express using getSlice
205+ , breakOn
206+
207+ -- ** Appending arrays
208+ , spliceCopy
209+ , spliceWith
210+ , splice
211+ , spliceExp
212+ , spliceUnsafe
213+ , putSliceUnsafe
214+ -- , putSlice
215+ -- , appendSlice
216+ -- , appendSliceFrom
217+
218+ -- * Utilities
264219 , roundUpToPower2
265220 , memcpy
266221 , memcmp
267222 , c_memchr
268−
269− -- * Deprecated
270− , asPtrUnsafe
271− , writeChunks
272− , flattenArrays
273− , flattenArraysRev
274− , fromArrayStreamK
275− , fromStreamDN
276− , fromStreamD
277− , cmp
278− , getIndices
279− , getIndicesWith
280− , resize
281− , resizeExp
282− , nil
283− , new
284− , pinnedNew
285− , pinnedNewBytes
286− , writeAppendNUnsafe
287− , writeAppendN
288− , writeAppendWith
289− , writeAppend
290− , writeNWithUnsafe
291− , writeNWith
292− , writeNUnsafe
293− , pinnedWriteNUnsafe
294− , writeN
295− , pinnedWriteN
296− , pinnedWriteNAligned -- XXX not required
297− , writeWith
298− , write
299− , pinnedWrite
300− , writeRevN
301223 )
302224where
303225
@@ -308,52 +230,48 @@ where
308230
309231import Control.Monad (when, void)
310232import Control.Monad.IO.Class (MonadIO(..))
311−import Data.Bifunctor (first)
312233import Data.Bits (shiftR, (.|.), (.&.))
313234import Data.Functor.Identity (Identity(..))
314235import Data.Proxy (Proxy(..))
315236import Data.Word (Word8)
316237import Foreign.C.Types (CSize(..), CInt(..))
317238import Foreign.Ptr (plusPtr, minusPtr, nullPtr)
318−import Streamly.Internal.Data.MutByteArray.Type
319− ( MutByteArray(..)
239+import Streamly.Internal.Data.Unbox
240+ ( MutableByteArray(..)
241+ , Unbox(..)
320242 , PinnedState(..)
321243 , getMutableByteArray#
322− , putSliceUnsafe
244+ , touch
323245 )
324−import Streamly.Internal.Data.Unbox (Unbox(..))
325246import GHC.Base
326247 ( IO(..)
327248 , Int(..)
249+ , byteArrayContents#
328250 , compareByteArrays#
329251 , copyMutableByteArray#
330252 )
331253import GHC.Base (noinline)
332−import GHC.Exts (unsafeCoerce#, Addr#)
254+import GHC.Exts (unsafeCoerce#)
333255import GHC.Ptr (Ptr(..))
334256
335257import Streamly.Internal.Data.Fold.Type (Fold(..))
336258import Streamly.Internal.Data.Producer.Type (Producer (..))
337−import Streamly.Internal.Data.Stream.Type (Stream)
338−import Streamly.Internal.Data.Parser.Type (Parser (..))
339−import Streamly.Internal.Data.StreamK.Type (StreamK)
259+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
260+import Streamly.Internal.Data.Stream.StreamK.Type (StreamK)
340261import Streamly.Internal.Data.SVar.Type (adaptState, defState)
341−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
342262import Streamly.Internal.Data.Unfold.Type (Unfold(..))
343263import Streamly.Internal.System.IO (arrayPayloadSize, defaultChunkSize)
344264
345265import qualified Streamly.Internal.Data.Fold.Type as FL
346−import qualified Streamly.Internal.Data.MutByteArray.Type as Unboxed
347−import qualified Streamly.Internal.Data.Parser.Type as Parser
348−-- import qualified Streamly.Internal.Data.Fold.Type as Fold
349266import qualified Streamly.Internal.Data.Producer as Producer
350−import qualified Streamly.Internal.Data.Stream.Type as D
351−import qualified Streamly.Internal.Data.Stream.Lift as D
352−import qualified Streamly.Internal.Data.StreamK.Type as K
267+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
268+import qualified Streamly.Internal.Data.Stream.StreamD.Lift as D
269+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
270+import qualified Streamly.Internal.Data.Unbox as Unboxed
353271import qualified Prelude
354272
355273import Prelude hiding
356− (Foldable(..), concat, read, unlines, splitAt, reverse, truncate)
274+ (Foldable(..), read, unlines, splitAt, reverse, truncate)
357275
358276#include "DocTestDataMutArray.hs"
359277
@@ -361,7 +279,6 @@ import Prelude hiding
361279-- Foreign helpers
362280-------------------------------------------------------------------------------
363281
364−-- NOTE: Have to be "ccall unsafe" so that we can pass unpinned memory to these
365282foreign import ccall unsafe "string.h memcpy" c_memcpy
366283 :: Ptr Word8 -> Ptr Word8 -> CSize -> IO (Ptr Word8)
367284
@@ -371,9 +288,6 @@ foreign import ccall unsafe "string.h memchr" c_memchr
371288foreign import ccall unsafe "string.h memcmp" c_memcmp
372289 :: Ptr Word8 -> Ptr Word8 -> CSize -> IO CInt
373290
374−foreign import ccall unsafe "string.h strlen" c_strlen
375− :: Ptr Word8 -> IO CSize
376−
377291-- | Given an 'Unboxed' type (unused first arg) and a number of bytes, return
378292-- how many elements of that type will completely fit in those bytes.
379293--
@@ -428,7 +342,7 @@ data MutArray a =
428342 -- The array is a range into arrContents. arrContents may be a superset of
429343 -- the slice represented by the array. All offsets are in bytes.
430344 MutArray
431− { arrContents :: {-# UNPACK #-} !MutByteArray
345+ { arrContents :: {-# UNPACK #-} !MutableByteArray
432346 , arrStart :: {-# UNPACK #-} !Int -- ^ index into arrContents
433347 , arrEnd :: {-# UNPACK #-} !Int -- ^ index into arrContents
434348 -- Represents the first invalid index of
@@ -440,25 +354,22 @@ data MutArray a =
440354-- Pinning & Unpinning
441355-------------------------------------------------------------------------------
442356
443−-- | Return a copy of the array in pinned memory if unpinned, else return the
444−-- original array.
357+-- | Copy the 'MutArray' to pinned memory if unpinned, else do nothing. The
358+-- overhead is a copy if the input array is unpinned.
445359{-# INLINE pin #-}
446360pin :: MutArray a -> IO (MutArray a)
447−pin arr@MutArray{..} =
448− if Unboxed.isPinned arrContents
449− then pure arr
450− else pinnedClone arr
361+pin arr@MutArray{..} = do
362+ contents <- Unboxed.pin arrContents
363+ return $ arr {arrContents = contents}
451364
452−-- | Return a copy of the array in unpinned memory if pinned, else return the
453−-- original array.
365+-- | Copy the 'MutArray' to unpinned memory if pinned, else do nothing. The
366+-- overhead is a copy if the input array is pinned.
454367{-# INLINE unpin #-}
455368unpin :: MutArray a -> IO (MutArray a)
456−unpin arr@MutArray{..} =
457− if Unboxed.isPinned arrContents
458− then clone arr
459− else pure arr
369+unpin arr@MutArray{..} = do
370+ contents <- Unboxed.unpin arrContents
371+ return $ arr {arrContents = contents}
460372
461−-- | Return 'True' if the array is allocated in pinned memory.
462373{-# INLINE isPinned #-}
463374isPinned :: MutArray a -> Bool
464375isPinned MutArray{..} = Unboxed.isPinned arrContents
@@ -479,8 +390,6 @@ isPinned MutArray{..} = Unboxed.isPinned arrContents
479390-- we need more than that. See stg_pinnedNewAlignedByteArrayzh and
480391-- allocatePinned in GHC source.
481392
482−-- XXX Rename to emptyAlignedWith, alignSize should be first arg.
483−
484393-- | @newArrayWith allocator alignment count@ allocates a new array of zero
485394-- length and with a capacity to hold @count@ elements, using @allocator
486395-- size alignment@ as the memory allocator function.
@@ -493,7 +402,7 @@ isPinned MutArray{..} = Unboxed.isPinned arrContents
493402-- /Pre-release/
494403{-# INLINE newArrayWith #-}
495404newArrayWith :: forall m a. (MonadIO m, Unbox a)
496− => (Int -> Int -> m MutByteArray) -> Int -> Int -> m (MutArray a)
405+ => (Int -> Int -> m MutableByteArray) -> Int -> Int -> m (MutArray a)
497406newArrayWith alloc alignSize count = do
498407 let size = max (count * SIZE_OF(a)) 0
499408 contents <- alloc size alignSize
@@ -504,25 +413,12 @@ newArrayWith alloc alignSize count = do
504413 , arrBound = size
505414 }
506415
507−-- For arrays "nil" sounds a bit odd. empty is better. The only problem with
508−-- empty is that it is also used by the Alternative type class. But assuming we
509−-- will mostly import the Array module qualified this should be fine.
510−
511−-- | Create an empty array.
512−empty ::
513−#ifdef DEVBUILD
514− Unbox a =>
515−#endif
516− MutArray a
517−empty = MutArray Unboxed.empty 0 0 0
518−
519−{-# DEPRECATED nil "Please use empty instead." #-}
520416nil ::
521417#ifdef DEVBUILD
522418 Unbox a =>
523419#endif
524420 MutArray a
525−nil = empty
421+nil = MutArray Unboxed.nil 0 0 0
526422
527423{-# INLINE newBytesAs #-}
528424newBytesAs :: MonadIO m =>
@@ -539,13 +435,12 @@ newBytesAs ps bytes = do
539435 , arrBound = bytes
540436 }
541437
542−-- | Allocates a pinned empty array that with a reserved capacity of bytes.
543−-- The memory of the array is uninitialized and the allocation is aligned as
544−-- per the 'Unboxed' instance of the type.
438+-- | Allocates a pinned empty array that can hold 'count' items. The memory of
439+-- the array is uninitialized and the allocation is aligned as per the
440+-- 'Unboxed' instance of the type.
545441--
546442-- /Pre-release/
547443{-# INLINE pinnedNewBytes #-}
548−{-# DEPRECATED pinnedNewBytes "Please use pinnedEmptyOf with appropriate calculation" #-}
549444pinnedNewBytes :: MonadIO m =>
550445#ifdef DEVBUILD
551446 Unbox a =>
@@ -570,31 +465,21 @@ newAs ps =
570465 (error "new: alignment is not used in unpinned arrays.")
571466
572467-- XXX can unaligned allocation be more efficient when alignment is not needed?
573−
574−-- | Allocates a pinned array of zero length but growable to the specified
575−-- capacity without reallocation.
576−{-# INLINE pinnedEmptyOf #-}
577−pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
578−pinnedEmptyOf = newAs Pinned
579−
580−-- XXX Deprecate in major
581−-- {-# DEPRECATED pinnedNew "Please use pinnedEmptyOf instead." #-}
468+--
469+-- | Allocates an empty pinned array that can hold 'count' items. The memory of
470+-- the array is uninitialized and the allocation is aligned as per the 'Unboxed'
471+-- instance of the type.
472+--
582473{-# INLINE pinnedNew #-}
583474pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
584−pinnedNew = pinnedEmptyOf
475+pinnedNew = newAs Pinned
585476
586−-- | Allocates an unpinned array of zero length but growable to the specified
587−-- capacity without reallocation.
477+-- | Allocates an empty unpinned array that can hold 'count' items. The memory
478+-- of the array is uninitialized.
588479--
589−{-# INLINE emptyOf #-}
590−emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
591−emptyOf = newAs Unpinned
592−
593−-- XXX Deprecate in major
594−-- {-# DEPRECATED new "Please use emptyOf instead." #-}
595480{-# INLINE new #-}
596481new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
597−new = emptyOf
482+new = newAs Unpinned
598483
599484-------------------------------------------------------------------------------
600485-- Random writes
@@ -610,7 +495,7 @@ putIndexUnsafe :: forall m a. (MonadIO m, Unbox a)
610495putIndexUnsafe i MutArray{..} x = do
611496 let index = INDEX_OF(arrStart, i, a)
612497 assert (i >= 0 && INDEX_VALID(index, arrEnd, a)) (return ())
613− liftIO $ pokeAt index arrContents x
498+ liftIO $ pokeByteIndex index arrContents x
614499
615500invalidIndex :: String -> Int -> a
616501invalidIndex label i =
@@ -628,7 +513,7 @@ putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
628513putIndex i MutArray{..} x = do
629514 let index = INDEX_OF(arrStart,i,a)
630515 if i >= 0 && INDEX_VALID(index,arrEnd,a)
631− then liftIO $ pokeAt index arrContents x
516+ then liftIO $ pokeByteIndex index arrContents x
632517 else invalidIndex "putIndex" i
633518
634519-- | Write an input stream of (index, value) pairs to an array. Throws an
@@ -642,7 +527,7 @@ putIndices arr = FL.foldlM' step (return ())
642527
643528 where
644529
645− step () (i, x) = putIndex i arr x
530+ step () (i, x) = liftIO (putIndex i arr x)
646531
647532-- | Modify a given index of an array using a modifier function.
648533--
@@ -654,9 +539,9 @@ modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) =>
654539modifyIndexUnsafe i MutArray{..} f = liftIO $ do
655540 let index = INDEX_OF(arrStart,i,a)
656541 assert (i >= 0 && INDEX_NEXT(index,a) <= arrEnd) (return ())
657− r <- peekAt index arrContents
542+ r <- peekByteIndex index arrContents
658543 let (x, res) = f r
659− pokeAt index arrContents x
544+ pokeByteIndex index arrContents x
660545 return res
661546
662547-- | Modify a given index of an array using a modifier function.
@@ -668,9 +553,9 @@ modifyIndex i MutArray{..} f = do
668553 let index = INDEX_OF(arrStart,i,a)
669554 if i >= 0 && INDEX_VALID(index,arrEnd,a)
670555 then liftIO $ do
671− r <- peekAt index arrContents
556+ r <- peekByteIndex index arrContents
672557 let (x, res) = f r
673− pokeAt index arrContents x
558+ pokeByteIndex index arrContents x
674559 return res
675560 else invalidIndex "modifyIndex" i
676561
@@ -704,8 +589,8 @@ modify MutArray{..} f = liftIO $
704589
705590 go i =
706591 when (INDEX_VALID(i,arrEnd,a)) $ do
707− r <- peekAt i arrContents
708− pokeAt i arrContents (f r)
592+ r <- peekByteIndex i arrContents
593+ pokeByteIndex i arrContents (f r)
709594 go (INDEX_NEXT(i,a))
710595
711596-- XXX We could specify the number of bytes to swap instead of Proxy. Need
@@ -714,15 +599,15 @@ modify MutArray{..} f = liftIO $
714599swapArrayByteIndices ::
715600 forall a. Unbox a
716601 => Proxy a
717− -> MutByteArray
602+ -> MutableByteArray
718603 -> Int
719604 -> Int
720605 -> IO ()
721606swapArrayByteIndices _ arrContents i1 i2 = do
722− r1 <- peekAt i1 arrContents
723− r2 <- peekAt i2 arrContents
724− pokeAt i1 arrContents (r2 :: a)
725− pokeAt i2 arrContents (r1 :: a)
607+ r1 <- peekByteIndex i1 arrContents
608+ r2 <- peekByteIndex i2 arrContents
609+ pokeByteIndex i1 arrContents (r2 :: a)
610+ pokeByteIndex i2 arrContents (r1 :: a)
726611
727612-- | Swap the elements at two indices without validating the indices.
728613--
@@ -838,23 +723,26 @@ arrayChunkBytes = 1024
838723roundDownTo :: Int -> Int -> Int
839724roundDownTo elemSize size = size - (size `mod` elemSize)
840725
726+-- XXX See if resizing can be implemented by reading the old array as a stream
727+-- and then using writeN to the new array.
728+--
841729-- NOTE: we are passing elemSize explicitly to avoid an Unboxed constraint.
842730-- Since this is not inlined Unboxed consrraint leads to dictionary passing
843731-- which complicates some inspection tests.
844732--
845−{-# NOINLINE reallocExplicitAs #-}
846−reallocExplicitAs :: PinnedState -> Int -> Int -> MutArray a -> IO (MutArray a)
847−reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
733+{-# NOINLINE reallocExplicit #-}
734+reallocExplicit :: Int -> Int -> MutArray a -> IO (MutArray a)
735+reallocExplicit elemSize newCapacityInBytes MutArray{..} = do
848736 assertM(arrEnd <= arrBound)
849737
850738 -- Allocate new array
851739 let newCapMaxInBytes = roundUpLargeArray newCapacityInBytes
852740 contents <-
853− if ps == Pinned
854− then Unboxed.pinnedNew newCapMaxInBytes
855− else Unboxed.new newCapMaxInBytes
856− let !(MutByteArray mbarrFrom#) = arrContents
857− !(MutByteArray mbarrTo#) = contents
741+ if Unboxed.isPinned arrContents
742+ then Unboxed.pinnedNewBytes newCapMaxInBytes
743+ else Unboxed.newBytes newCapMaxInBytes
744+ let !(MutableByteArray mbarrFrom#) = arrContents
745+ !(MutableByteArray mbarrTo#) = contents
858746
859747 -- Copy old data
860748 let oldStart = arrStart
@@ -875,10 +763,6 @@ reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
875763 , arrBound = newCapInBytes
876764 }
877765
878−-- XXX Should these be called reallocBytes etc?
879−-- XXX We may also need reallocAs to allocate as pinned/unpinned explicitly. In
880−-- fact clone/pinnedClone can be implemented using reallocAs.
881−
882766-- | @realloc newCapacity array@ reallocates the array to the specified
883767-- capacity in bytes.
884768--
@@ -890,12 +774,7 @@ reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
890774-- If the original array is pinned, the newly allocated array is also pinned.
891775{-# INLINABLE realloc #-}
892776realloc :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
893−realloc bytes arr =
894− let ps =
895− if isPinned arr
896− then Pinned
897− else Unpinned
898− in liftIO $ reallocExplicitAs ps (SIZE_OF(a)) bytes arr
777+realloc bytes arr = liftIO $ reallocExplicit (SIZE_OF(a)) bytes arr
899778
900779-- | @reallocWith label capSizer minIncrBytes array@. The label is used
901780-- in error messages and the capSizer is used to determine the capacity of the
@@ -918,14 +797,14 @@ reallocWith label capSizer minIncrBytes arr = do
918797 where
919798
920799 badSize newSize =
921− Prelude.concat
800+ concat
922801 [ label
923802 , ": new array size (in bytes) is less than required size "
924803 , show newSize
925804 , ". Please check the sizing function passed."
926805 ]
927806
928−-- | @grow newCapacity array@ changes the total capacity of the array so that
807+-- | @resize newCapacity array@ changes the total capacity of the array so that
929808-- it is enough to hold the specified number of elements. Nothing is done if
930809-- the specified capacity is less than the length of the array.
931810--
@@ -933,46 +812,34 @@ reallocWith label capSizer minIncrBytes arr = do
933812-- the block size (4K).
934813--
935814-- /Pre-release/
936−{-# INLINE grow #-}
937−grow :: forall m a. (MonadIO m, Unbox a) =>
815+{-# INLINE resize #-}
816+resize :: forall m a. (MonadIO m, Unbox a) =>
938817 Int -> MutArray a -> m (MutArray a)
939−grow nElems arr@MutArray{..} = do
818+resize nElems arr@MutArray{..} = do
940819 let req = SIZE_OF(a) * nElems
941− cap = arrBound - arrStart
942− if req < cap
820+ len = arrEnd - arrStart
821+ if req < len
943822 then return arr
944823 else realloc req arr
945824
946−{-# DEPRECATED resize "Please use grow instead." #-}
947−{-# INLINE resize #-}
948−resize :: forall m a. (MonadIO m, Unbox a) =>
949− Int -> MutArray a -> m (MutArray a)
950−resize = grow
951−
952−-- | Like 'grow' but if the requested byte capacity is more than
953−-- 'largeObjectThreshold' then it is rounded up to the closest power of 2.
825+-- | Like 'resize' but if the byte capacity is more than 'largeObjectThreshold'
826+-- then it is rounded up to the closest power of 2.
954827--
955828-- /Pre-release/
956−{-# INLINE growExp #-}
957−growExp :: forall m a. (MonadIO m, Unbox a) =>
829+{-# INLINE resizeExp #-}
830+resizeExp :: forall m a. (MonadIO m, Unbox a) =>
958831 Int -> MutArray a -> m (MutArray a)
959−growExp nElems arr@MutArray{..} = do
832+resizeExp nElems arr@MutArray{..} = do
960833 let req = roundUpLargeArray (SIZE_OF(a) * nElems)
961834 req1 =
962835 if req > largeObjectThreshold
963836 then roundUpToPower2 req
964837 else req
965− cap = arrBound - arrStart
966− if req1 < cap
838+ len = arrEnd - arrStart
839+ if req1 < len
967840 then return arr
968841 else realloc req1 arr
969842
970−{-# DEPRECATED resizeExp "Please use growExp instead." #-}
971−{-# INLINE resizeExp #-}
972−resizeExp :: forall m a. (MonadIO m, Unbox a) =>
973− Int -> MutArray a -> m (MutArray a)
974−resizeExp = growExp
975−
976843-- | Resize the allocated memory to drop any reserved free space at the end of
977844-- the array and reallocate it to reduce wastage.
978845--
@@ -1017,7 +884,7 @@ rightSize arr@MutArray{..} = do
1017884snocNewEnd :: (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m (MutArray a)
1018885snocNewEnd newEnd arr@MutArray{..} x = liftIO $ do
1019886 assert (newEnd <= arrBound) (return ())
1020− pokeAt arrEnd arrContents x
887+ pokeByteIndex arrEnd arrContents x
1021888 return $ arr {arrEnd = newEnd}
1022889
1023890-- | Really really unsafe, appends the element into the first array, may
@@ -1037,7 +904,7 @@ snocUnsafe arr@MutArray{..} = snocNewEnd (INDEX_NEXT(arrEnd,a)) arr
1037904{-# INLINE snocMay #-}
1038905snocMay :: forall m a. (MonadIO m, Unbox a) =>
1039906 MutArray a -> a -> m (Maybe (MutArray a))
1040−snocMay arr@MutArray{..} x = do
907+snocMay arr@MutArray{..} x = liftIO $ do
1041908 let newEnd = INDEX_NEXT(arrEnd,a)
1042909 if newEnd <= arrBound
1043910 then Just <$> snocNewEnd newEnd arr x
@@ -1051,7 +918,7 @@ snocWithRealloc :: forall m a. (MonadIO m, Unbox a) =>
1051918 -> a
1052919 -> m (MutArray a)
1053920snocWithRealloc sizer arr x = do
1054− arr1 <- reallocWith "snocWith" sizer (SIZE_OF(a)) arr
921+ arr1 <- liftIO $ reallocWith "snocWith" sizer (SIZE_OF(a)) arr
1055922 snocUnsafe arr1 x
1056923
1057924-- | @snocWith sizer arr elem@ mutates @arr@ to append @elem@. The length of
@@ -1073,7 +940,7 @@ snocWith :: forall m a. (MonadIO m, Unbox a) =>
1073940 -> MutArray a
1074941 -> a
1075942 -> m (MutArray a)
1076−snocWith allocSize arr x = do
943+snocWith allocSize arr x = liftIO $ do
1077944 let newEnd = INDEX_NEXT(arrEnd arr,a)
1078945 if newEnd <= arrBound arr
1079946 then snocNewEnd newEnd arr x
@@ -1117,126 +984,6 @@ snoc = snocWith f
1117984 then oldSize * 2
1118985 else roundUpToPower2 oldSize * 2
1119986
1120−-------------------------------------------------------------------------------
1121−-- Serialization/Deserialization
1122−-------------------------------------------------------------------------------
1123−
1124−{-# INLINE pokeNewEnd #-}
1125−pokeNewEnd :: (MonadIO m, Unbox a) =>
1126− Int -> MutArray Word8 -> a -> m (MutArray Word8)
1127−pokeNewEnd newEnd arr@MutArray{..} x = liftIO $ do
1128− assert (newEnd <= arrBound) (return ())
1129− liftIO $ pokeAt arrEnd arrContents x
1130− return $ arr {arrEnd = newEnd}
1131−
1132−-- | Really really unsafe, unboxes a Haskell type and appends the resulting
1133−-- bytes to the byte array, may cause silent data corruption or if you are
1134−-- lucky a segfault if the array does not have enough space to append the
1135−-- element.
1136−--
1137−-- /Internal/
1138−{-# INLINE pokeAppendUnsafe #-}
1139−pokeAppendUnsafe :: forall m a. (MonadIO m, Unbox a) =>
1140− MutArray Word8 -> a -> m (MutArray Word8)
1141−pokeAppendUnsafe arr@MutArray{..} = pokeNewEnd (arrEnd + SIZE_OF(a)) arr
1142−
1143−-- | Skip the specified number of bytes in the array. The data in the skipped
1144−-- region remains uninitialzed.
1145−{-# INLINE pokeSkipUnsafe #-}
1146−pokeSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
1147−pokeSkipUnsafe n arr@MutArray{..} = do
1148− let newEnd = arrEnd + n
1149− in assert (newEnd <= arrBound) (arr {arrEnd = newEnd})
1150−
1151−-- | Like 'pokeAppend' but does not grow the array when pre-allocated array
1152−-- capacity becomes full.
1153−--
1154−-- /Internal/
1155−{-# INLINE pokeAppendMay #-}
1156−pokeAppendMay :: forall m a. (MonadIO m, Unbox a) =>
1157− MutArray Word8 -> a -> m (Maybe (MutArray Word8))
1158−pokeAppendMay arr@MutArray{..} x = liftIO $ do
1159− let newEnd = arrEnd + SIZE_OF(a)
1160− if newEnd <= arrBound
1161− then Just <$> pokeNewEnd newEnd arr x
1162− else return Nothing
1163−
1164−{-# NOINLINE pokeWithRealloc #-}
1165−pokeWithRealloc :: forall m a. (MonadIO m, Unbox a) =>
1166− (Int -> Int)
1167− -> MutArray Word8
1168− -> a
1169− -> m (MutArray Word8)
1170−pokeWithRealloc sizer arr x = do
1171− arr1 <- liftIO $ reallocWith "pokeWithRealloc" sizer (SIZE_OF(a)) arr
1172− pokeAppendUnsafe arr1 x
1173−
1174−{-# INLINE pokeAppendWith #-}
1175−pokeAppendWith :: forall m a. (MonadIO m, Unbox a) =>
1176− (Int -> Int)
1177− -> MutArray Word8
1178− -> a
1179− -> m (MutArray Word8)
1180−pokeAppendWith allocSize arr x = liftIO $ do
1181− let newEnd = arrEnd arr + SIZE_OF(a)
1182− if newEnd <= arrBound arr
1183− then pokeNewEnd newEnd arr x
1184− else pokeWithRealloc allocSize arr x
1185−
1186−-- | Unbox a Haskell type and append the resulting bytes to a mutable byte
1187−-- array. The array is grown exponentially when more space is needed.
1188−--
1189−-- Definition:
1190−--
1191−-- >>> pokeAppend arr x = MutArray.castUnsafe <$> MutArray.snoc (MutArray.castUnsafe arr) x
1192−--
1193−{-# INLINE pokeAppend #-}
1194−pokeAppend :: forall m a. (MonadIO m, Unbox a) =>
1195− MutArray Word8 -> a -> m (MutArray Word8)
1196−pokeAppend = pokeAppendWith f
1197−
1198− where
1199−
1200− f oldSize =
1201− if isPower2 oldSize
1202− then oldSize * 2
1203− else roundUpToPower2 oldSize * 2
1204−
1205−-- | Really really unsafe, create a Haskell value from an unboxed byte array,
1206−-- does not check if the array is big enough, may return garbage or if you are
1207−-- lucky may cause a segfault.
1208−--
1209−-- /Internal/
1210−{-# INLINE peekUnconsUnsafe #-}
1211−peekUnconsUnsafe :: forall m a. (MonadIO m, Unbox a) =>
1212− MutArray Word8 -> m (a, MutArray Word8)
1213−peekUnconsUnsafe MutArray{..} = do
1214− let start1 = arrStart + SIZE_OF(a)
1215− assert (start1 <= arrEnd) (return ())
1216− liftIO $ do
1217− r <- peekAt arrStart arrContents
1218− return (r, MutArray arrContents start1 arrEnd arrBound)
1219−
1220−-- | Discard the specified number of bytes in the array.
1221−{-# INLINE peekSkipUnsafe #-}
1222−peekSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
1223−peekSkipUnsafe n MutArray{..} =
1224− let start1 = arrStart + n
1225− in assert (start1 <= arrEnd) (MutArray arrContents start1 arrEnd arrBound)
1226−
1227−-- | Create a Haskell value from its unboxed representation from the head of a
1228−-- byte array, return the value and the remaining array.
1229−{-# INLINE peekUncons #-}
1230−peekUncons :: forall m a. (MonadIO m, Unbox a) =>
1231− MutArray Word8 -> m (Maybe a, MutArray Word8)
1232−peekUncons arr@MutArray{..} = do
1233− let start1 = arrStart + SIZE_OF(a)
1234− if start1 > arrEnd
1235− then return (Nothing, arr)
1236− else liftIO $ do
1237− r <- peekAt arrStart arrContents
1238− return (Just r, MutArray arrContents start1 arrEnd arrBound)
1239−
1240987-------------------------------------------------------------------------------
1241988-- Random reads
1242989-------------------------------------------------------------------------------
@@ -1251,7 +998,7 @@ getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1251998getIndexUnsafe i MutArray{..} = do
1252999 let index = INDEX_OF(arrStart,i,a)
12531000 assert (i >= 0 && INDEX_VALID(index,arrEnd,a)) (return ())
1254− liftIO $ peekAt index arrContents
1001+ liftIO $ peekByteIndex index arrContents
12551002
12561003-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
12571004--
@@ -1260,7 +1007,7 @@ getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
12601007getIndex i MutArray{..} = do
12611008 let index = INDEX_OF(arrStart,i,a)
12621009 if i >= 0 && INDEX_VALID(index,arrEnd,a)
1263− then liftIO $ Just <$> peekAt index arrContents
1010+ then liftIO $ Just <$> peekByteIndex index arrContents
12641011 else return Nothing
12651012
12661013-- | /O(1)/ Lookup the element at the given index from the end of the array.
@@ -1273,16 +1020,21 @@ getIndexRev :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
12731020getIndexRev i MutArray{..} = do
12741021 let index = RINDEX_OF(arrEnd,i,a)
12751022 if i >= 0 && index >= arrStart
1276− then liftIO $ peekAt index arrContents
1023+ then liftIO $ peekByteIndex index arrContents
12771024 else invalidIndex "getIndexRev" i
12781025
12791026data GetIndicesState contents start end st =
12801027 GetIndicesState contents start end st
12811028
1282−{-# INLINE indexReaderWith #-}
1283−indexReaderWith :: (Monad m, Unbox a) =>
1029+-- | Given an unfold that generates array indices, read the elements on those
1030+-- indices from the supplied MutArray. An error is thrown if an index is out of
1031+-- bounds.
1032+--
1033+-- /Pre-release/
1034+{-# INLINE getIndicesD #-}
1035+getIndicesD :: (Monad m, Unbox a) =>
12841036 (forall b. IO b -> m b) -> D.Stream m Int -> Unfold m (MutArray a) a
1285−indexReaderWith liftio (D.Stream stepi sti) = Unfold step inject
1037+getIndicesD liftio (D.Stream stepi sti) = Unfold step inject
12861038
12871039 where
12881040
@@ -1301,26 +1053,9 @@ indexReaderWith liftio (D.Stream stepi sti) = Unfold step inject
13011053 D.Skip s -> return $ D.Skip (GetIndicesState contents start end s)
13021054 D.Stop -> return D.Stop
13031055
1304−{-# DEPRECATED getIndicesWith "Please use indexReaderWith instead." #-}
1305−{-# INLINE getIndicesWith #-}
1306−getIndicesWith :: (Monad m, Unbox a) =>
1307− (forall b. IO b -> m b) -> D.Stream m Int -> Unfold m (MutArray a) a
1308−getIndicesWith = indexReaderWith
1309−
1310−-- | Given an unfold that generates array indices, read the elements on those
1311−-- indices from the supplied MutArray. An error is thrown if an index is out of
1312−-- bounds.
1313−--
1314−-- /Pre-release/
1315−{-# INLINE indexReader #-}
1316−indexReader :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1317−indexReader = indexReaderWith liftIO
1318−
1319−-- XXX DO NOT REMOVE, change the signature to use Stream instead of unfold
1320−{-# DEPRECATED getIndices "Please use indexReader instead." #-}
13211056{-# INLINE getIndices #-}
13221057getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1323−getIndices = indexReader
1058+getIndices = getIndicesD liftIO
13241059
13251060-------------------------------------------------------------------------------
13261061-- Subarrays
@@ -1425,7 +1160,7 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14251160
14261161 -- Invariant low < high on entry, and on return as well
14271162 moveHigh low high = do
1428− h <- peekAt high arrContents
1163+ h <- peekByteIndex high arrContents
14291164 if f h
14301165 then
14311166 -- Correctly classified, continue the loop
@@ -1445,7 +1180,7 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14451180 -- low <= high
14461181 -- Both low and high are valid locations within the array
14471182 go low high = do
1448− l <- peekAt low arrContents
1183+ l <- peekByteIndex low arrContents
14491184 if f l
14501185 then
14511186 -- low is wrongly classified
@@ -1456,8 +1191,8 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14561191 case r of
14571192 Nothing -> return low
14581193 Just (high1, h) -> do -- low < high1
1459− pokeAt low arrContents h
1460− pokeAt high1 arrContents l
1194+ pokeByteIndex low arrContents h
1195+ pokeByteIndex high1 arrContents l
14611196 let low1 = INDEX_NEXT(low,a)
14621197 high2 = INDEX_PREV(high1,a)
14631198 if low1 <= high2
@@ -1591,7 +1326,7 @@ chunksOfAs ps n (D.Stream step state) =
15911326 r <- step (adaptState gst) st
15921327 case r of
15931328 D.Yield x s -> do
1594− liftIO $ pokeAt end contents x
1329+ liftIO $ pokeByteIndex end contents x
15951330 let end1 = INDEX_NEXT(end,a)
15961331 return $
15971332 if end1 >= bound
@@ -1610,12 +1345,10 @@ chunksOfAs ps n (D.Stream step state) =
16101345
16111346 step' _ GroupFinish = return D.Stop
16121347
1613−-- | @chunksOf n stream@ groups the elements in the input stream into arrays of
1614−-- @n@ elements each.
1348+-- | @chunksOf n stream@ groups the input stream into a stream of
1349+-- arrays of size n.
16151350--
1616−-- Same as the following but may be more efficient:
1617−--
1618−-- >>> chunksOf n = Stream.foldMany (MutArray.createOf n)
1351+-- @chunksOf n = StreamD.foldMany (MutArray.writeN n)@
16191352--
16201353-- /Pre-release/
16211354{-# INLINE_NORMAL chunksOf #-}
@@ -1624,29 +1357,17 @@ chunksOf :: forall m a. (MonadIO m, Unbox a)
16241357-- XXX the idiomatic implementation leads to large regression in the D.reverse'
16251358-- benchmark. It seems it has difficulty producing optimized code when
16261359-- converting to StreamK. Investigate GHC optimizations.
1627−-- chunksOf n = D.foldMany (createOf n)
1360+-- chunksOf n = D.foldMany (writeN n)
16281361chunksOf = chunksOfAs Unpinned
16291362
16301363-- | Like 'chunksOf' but creates pinned arrays.
16311364{-# INLINE_NORMAL pinnedChunksOf #-}
16321365pinnedChunksOf :: forall m a. (MonadIO m, Unbox a)
16331366 => Int -> D.Stream m a -> D.Stream m (MutArray a)
1634−-- pinnedChunksOf n = D.foldMany (pinnedCreateOf n)
1367+-- pinnedChunksOf n = D.foldMany (pinnedWriteN n)
16351368pinnedChunksOf = chunksOfAs Pinned
16361369
1637−-- | When we are buffering a stream of unknown size into an array we do not
1638−-- know how much space to pre-allocate. So we start with the min size and emit
1639−-- the array then keep on doubling the size every time. Thus we do not need to
1640−-- guess the optimum chunk size.
1641−--
1642−-- We can incorporate this in chunksOfAs if the additional size parameter does
1643−-- not impact perf.
1644−--
1645−{-# INLINE _chunksOfRange #-}
1646−_chunksOfRange :: -- (MonadIO m, Unbox a) =>
1647− PinnedState -> Int -> Int -> D.Stream m a -> D.Stream m (MutArray a)
1648−_chunksOfRange _ps _low _hi = undefined
1649−
1370+-- XXX This should take a PinnedState
16501371-- XXX buffer to a list instead?
16511372-- | Buffer the stream into arrays in memory.
16521373{-# INLINE arrayStreamKFromStreamDAs #-}
@@ -1664,10 +1385,16 @@ data FlattenState s contents a =
16641385 OuterLoop s
16651386 | InnerLoop s contents !Int !Int
16661387
1667−{-# INLINE_NORMAL concatWith #-}
1668−concatWith :: forall m a. (Monad m, Unbox a)
1669− => (forall b. IO b -> m b) -> D.Stream m (MutArray a) -> D.Stream m a
1670−concatWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
1388+-- | Use the "reader" unfold instead.
1389+--
1390+-- @flattenArrays = unfoldMany reader@
1391+--
1392+-- We can try this if there are any fusion issues in the unfold.
1393+--
1394+{-# INLINE_NORMAL flattenArrays #-}
1395+flattenArrays :: forall m a. (MonadIO m, Unbox a)
1396+ => D.Stream m (MutArray a) -> D.Stream m a
1397+flattenArrays (D.Stream step state) = D.Stream step' (OuterLoop state)
16711398
16721399 where
16731400
@@ -1684,30 +1411,19 @@ concatWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
16841411 return $ D.Skip $ OuterLoop st
16851412
16861413 step' _ (InnerLoop st contents p end) = do
1687− x <- liftio $ peekAt p contents
1414+ x <- liftIO $ peekByteIndex p contents
16881415 return $ D.Yield x (InnerLoop st contents (INDEX_NEXT(p,a)) end)
16891416
1690−-- | Use the "reader" unfold instead.
1417+-- | Use the "readerRev" unfold instead.
16911418--
1692−-- @concat = unfoldMany reader@
1419+-- @flattenArrays = unfoldMany readerRev@
16931420--
16941421-- We can try this if there are any fusion issues in the unfold.
16951422--
1696−{-# INLINE_NORMAL concat #-}
1697−concat :: forall m a. (MonadIO m, Unbox a)
1698− => D.Stream m (MutArray a) -> D.Stream m a
1699−concat = concatWith liftIO
1700−
1701−{-# DEPRECATED flattenArrays "Please use \"unfoldMany reader\" instead." #-}
1702−{-# INLINE flattenArrays #-}
1703−flattenArrays :: forall m a. (MonadIO m, Unbox a)
1423+{-# INLINE_NORMAL flattenArraysRev #-}
1424+flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
17041425 => D.Stream m (MutArray a) -> D.Stream m a
1705−flattenArrays = concat
1706−
1707−{-# INLINE_NORMAL concatRevWith #-}
1708−concatRevWith :: forall m a. (Monad m, Unbox a)
1709− => (forall b. IO b -> m b) -> D.Stream m (MutArray a) -> D.Stream m a
1710−concatRevWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
1426+flattenArraysRev (D.Stream step state) = D.Stream step' (OuterLoop state)
17111427
17121428 where
17131429
@@ -1725,33 +1441,16 @@ concatRevWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
17251441 return $ D.Skip $ OuterLoop st
17261442
17271443 step' _ (InnerLoop st contents p start) = do
1728− x <- liftio $ peekAt p contents
1444+ x <- liftIO $ peekByteIndex p contents
17291445 let cur = INDEX_PREV(p,a)
17301446 return $ D.Yield x (InnerLoop st contents cur start)
17311447
1732−-- | Use the "readerRev" unfold instead.
1733−--
1734−-- @concat = unfoldMany readerRev@
1735−--
1736−-- We can try this if there are any fusion issues in the unfold.
1737−--
1738−{-# INLINE_NORMAL concatRev #-}
1739−concatRev :: forall m a. (MonadIO m, Unbox a)
1740− => D.Stream m (MutArray a) -> D.Stream m a
1741−concatRev = concatRevWith liftIO
1742−
1743−{-# DEPRECATED flattenArraysRev "Please use \"unfoldMany readerRev\" instead." #-}
1744−{-# INLINE flattenArraysRev #-}
1745−flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
1746− => D.Stream m (MutArray a) -> D.Stream m a
1747−flattenArraysRev = concatRev
1748−
17491448-------------------------------------------------------------------------------
17501449-- Unfolds
17511450-------------------------------------------------------------------------------
17521451
17531452data ArrayUnsafe a = ArrayUnsafe
1754− {-# UNPACK #-} !MutByteArray -- contents
1453+ {-# UNPACK #-} !MutableByteArray -- contents
17551454 {-# UNPACK #-} !Int -- index 1
17561455 {-# UNPACK #-} !Int -- index 2
17571456
@@ -1781,7 +1480,7 @@ producerWith liftio = Producer step (return . toArrayUnsafe) extract
17811480 -- few actions for correctness and execution order sanity. We want
17821481 -- the peek to occur right here and not lazily at some later point
17831482 -- because we want the peek to be ordered with respect to the touch.
1784− !x <- liftio $ peekAt cur contents
1483+ !x <- liftio $ peekByteIndex cur contents
17851484 return $ D.Yield x (ArrayUnsafe contents (INDEX_NEXT(cur,a)) end)
17861485
17871486 extract = return . fromArrayUnsafe
@@ -1812,7 +1511,7 @@ readerRevWith liftio = Unfold step inject
18121511 {-# INLINE_LATE step #-}
18131512 step (ArrayUnsafe _ start p) | p < start = return D.Stop
18141513 step (ArrayUnsafe contents start p) = do
1815− !x <- liftio $ peekAt p contents
1514+ !x <- liftio $ peekByteIndex p contents
18161515 return $ D.Yield x (ArrayUnsafe contents start (INDEX_PREV(p,a)))
18171516
18181517-- | Unfold an array into a stream in reverse order.
@@ -1843,7 +1542,7 @@ toListFB c n MutArray{..} = go arrStart
18431542 -- evaluated/written to before we peek at them.
18441543 -- XXX
18451544 let !x = unsafeInlineIO $ do
1846− r <- peekAt arrContents p
1545+ r <- peekByteIndex arrContents p
18471546 return r
18481547 in c x (go (PTR_NEXT(p,a)))
18491548-}
@@ -1860,21 +1559,21 @@ toList MutArray{..} = liftIO $ go arrStart
18601559
18611560 go p | assert (p <= arrEnd) (p == arrEnd) = return []
18621561 go p = do
1863− x <- peekAt p arrContents
1562+ x <- peekByteIndex p arrContents
18641563 (:) x <$> go (INDEX_NEXT(p,a))
18651564
1866−{-# INLINE_NORMAL toStreamWith #-}
1867−toStreamWith ::
1565+{-# INLINE_NORMAL toStreamDWith #-}
1566+toStreamDWith ::
18681567 forall m a. (Monad m, Unbox a)
18691568 => (forall b. IO b -> m b) -> MutArray a -> D.Stream m a
1870−toStreamWith liftio MutArray{..} = D.Stream step arrStart
1569+toStreamDWith liftio MutArray{..} = D.Stream step arrStart
18711570
18721571 where
18731572
18741573 {-# INLINE_LATE step #-}
18751574 step _ p | assert (p <= arrEnd) (p == arrEnd) = return D.Stop
18761575 step _ p = liftio $ do
1877− r <- peekAt p arrContents
1576+ r <- peekByteIndex p arrContents
18781577 return $ D.Yield r (INDEX_NEXT(p,a))
18791578
18801579-- | Convert a 'MutArray' into a stream.
@@ -1883,7 +1582,7 @@ toStreamWith liftio MutArray{..} = D.Stream step arrStart
18831582--
18841583{-# INLINE_NORMAL read #-}
18851584read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> D.Stream m a
1886−read = toStreamWith liftIO
1585+read = toStreamDWith liftIO
18871586
18881587{-# INLINE toStreamKWith #-}
18891588toStreamKWith ::
@@ -1895,18 +1594,18 @@ toStreamKWith liftio MutArray{..} = go arrStart
18951594
18961595 go p | assert (p <= arrEnd) (p == arrEnd) = K.nil
18971596 | otherwise =
1898− let elemM = peekAt p arrContents
1597+ let elemM = peekByteIndex p arrContents
18991598 in liftio elemM `K.consM` go (INDEX_NEXT(p,a))
19001599
19011600{-# INLINE toStreamK #-}
19021601toStreamK :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
19031602toStreamK = toStreamKWith liftIO
19041603
1905−{-# INLINE_NORMAL toStreamRevWith #-}
1906−toStreamRevWith ::
1604+{-# INLINE_NORMAL toStreamDRevWith #-}
1605+toStreamDRevWith ::
19071606 forall m a. (Monad m, Unbox a)
19081607 => (forall b. IO b -> m b) -> MutArray a -> D.Stream m a
1909−toStreamRevWith liftio MutArray{..} =
1608+toStreamDRevWith liftio MutArray{..} =
19101609 let p = INDEX_PREV(arrEnd,a)
19111610 in D.Stream step p
19121611
@@ -1915,7 +1614,7 @@ toStreamRevWith liftio MutArray{..} =
19151614 {-# INLINE_LATE step #-}
19161615 step _ p | p < arrStart = return D.Stop
19171616 step _ p = liftio $ do
1918− r <- peekAt p arrContents
1617+ r <- peekByteIndex p arrContents
19191618 return $ D.Yield r (INDEX_PREV(p,a))
19201619
19211620-- | Convert a 'MutArray' into a stream in reverse order.
@@ -1924,7 +1623,7 @@ toStreamRevWith liftio MutArray{..} =
19241623--
19251624{-# INLINE_NORMAL readRev #-}
19261625readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> D.Stream m a
1927−readRev = toStreamRevWith liftIO
1626+readRev = toStreamDRevWith liftIO
19281627
19291628{-# INLINE toStreamKRevWith #-}
19301629toStreamKRevWith ::
@@ -1938,7 +1637,7 @@ toStreamKRevWith liftio MutArray {..} =
19381637
19391638 go p | p < arrStart = K.nil
19401639 | otherwise =
1941− let elemM = peekAt p arrContents
1640+ let elemM = peekByteIndex p arrContents
19421641 in liftio elemM `K.consM` go (INDEX_PREV(p,a))
19431642
19441643{-# INLINE toStreamKRev #-}
@@ -1973,7 +1672,7 @@ foldr f z arr = D.foldr f z $ read arr
19731672
19741673-- XXX Keep the bound intact to not lose any free space? Perf impact?
19751674
1976−-- | @unsafeAppendN n arr@ appends up to @n@ input items to the supplied
1675+-- | @writeAppendNUnsafe n arr@ appends up to @n@ input items to the supplied
19771676-- array.
19781677--
19791678-- Unsafe: Do not drive the fold beyond @n@ elements, it will lead to memory
@@ -1982,12 +1681,13 @@ foldr f z arr = D.foldr f z $ read arr
19821681-- Any free space left in the array after appending @n@ elements is lost.
19831682--
19841683-- /Internal/
1985−{-# INLINE_NORMAL unsafeAppendN #-}
1986−unsafeAppendN :: forall m a. (MonadIO m, Unbox a) =>
1684+{-# INLINE_NORMAL writeAppendNUnsafe #-}
1685+writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) =>
19871686 Int
19881687 -> m (MutArray a)
19891688 -> Fold m a (MutArray a)
1990−unsafeAppendN n action = fmap fromArrayUnsafe $ FL.foldlM' step initial
1689+writeAppendNUnsafe n action =
1690+ fmap fromArrayUnsafe $ FL.foldlM' step initial
19911691
19921692 where
19931693
@@ -2000,59 +1700,38 @@ unsafeAppendN n action = fmap fromArrayUnsafe $ FL.foldlM' step initial
20001700 -- otherwise we lose that space.
20011701 arr1 <-
20021702 if free < needed
2003− then noinline reallocWith "unsafeAppendN" (+ needed) needed arr
1703+ then noinline reallocWith "writeAppendNUnsafeWith" (+ needed) needed arr
20041704 else return arr
20051705 return $ toArrayUnsafe arr1
20061706
20071707 step (ArrayUnsafe contents start end) x = do
2008− liftIO $ pokeAt end contents x
1708+ liftIO $ pokeByteIndex end contents x
20091709 return $ ArrayUnsafe contents start (INDEX_NEXT(end,a))
20101710
2011−{-# DEPRECATED writeAppendNUnsafe "Please use unsafeAppendN instead." #-}
2012−{-# INLINE writeAppendNUnsafe #-}
2013−writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) =>
2014− Int
2015− -> m (MutArray a)
2016− -> Fold m a (MutArray a)
2017−writeAppendNUnsafe = unsafeAppendN
2018−
20191711-- | Append @n@ elements to an existing array. Any free space left in the array
20201712-- after appending @n@ elements is lost.
20211713--
2022−-- >>> appendN n initial = Fold.take n (MutArray.unsafeAppendN n initial)
1714+-- >>> writeAppendN n initial = Fold.take n (MutArray.writeAppendNUnsafe n initial)
20231715--
2024−{-# INLINE_NORMAL appendN #-}
2025−appendN :: forall m a. (MonadIO m, Unbox a) =>
2026− Int -> m (MutArray a) -> Fold m a (MutArray a)
2027−appendN n initial = FL.take n (unsafeAppendN n initial)
2028−
2029−-- XXX Deprecate in major
2030−-- {-# DEPRECATED writeAppendN "Please use appendN instead." #-}
2031−{-# INLINE writeAppendN #-}
1716+{-# INLINE_NORMAL writeAppendN #-}
20321717writeAppendN :: forall m a. (MonadIO m, Unbox a) =>
20331718 Int -> m (MutArray a) -> Fold m a (MutArray a)
2034−writeAppendN = appendN
1719+writeAppendN n initial = FL.take n (writeAppendNUnsafe n initial)
20351720
2036−-- | @appendWith realloc action@ mutates the array generated by @action@ to
1721+-- | @writeAppendWith realloc action@ mutates the array generated by @action@ to
20371722-- append the input stream. If there is no reserved space available in the
20381723-- array it is reallocated to a size in bytes determined by @realloc oldSize@,
20391724-- where @oldSize@ is the current size of the array in bytes.
20401725--
20411726-- Note that the returned array may be a mutated version of original array.
20421727--
2043−-- >>> appendWith sizer = Fold.foldlM' (MutArray.snocWith sizer)
1728+-- >>> writeAppendWith sizer = Fold.foldlM' (MutArray.snocWith sizer)
20441729--
20451730-- /Pre-release/
2046−{-# INLINE appendWith #-}
2047−appendWith :: forall m a. (MonadIO m, Unbox a) =>
2048− (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
2049−appendWith sizer = FL.foldlM' (snocWith sizer)
2050−
2051−{-# DEPRECATED writeAppendWith "Please use appendWith instead." #-}
20521731{-# INLINE writeAppendWith #-}
20531732writeAppendWith :: forall m a. (MonadIO m, Unbox a) =>
20541733 (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
2055−writeAppendWith = appendWith
1734+writeAppendWith sizer = FL.foldlM' (snocWith sizer)
20561735
20571736-- | @append action@ mutates the array generated by @action@ to append the
20581737-- input stream. If there is no reserved space available in the array it is
@@ -2060,101 +1739,70 @@ writeAppendWith = appendWith
20601739--
20611740-- Note that the returned array may be a mutated version of original array.
20621741--
2063−-- >>> append = MutArray.appendWith (* 2)
1742+-- >>> writeAppend = MutArray.writeAppendWith (* 2)
20641743--
2065−{-# INLINE append #-}
2066−append :: forall m a. (MonadIO m, Unbox a) =>
2067− m (MutArray a) -> Fold m a (MutArray a)
2068−append = appendWith (* 2)
2069−
2070−-- XXX Deprecate in major
2071−-- {-# DEPRECATED writeAppend "Please use append instead." #-}
20721744{-# INLINE writeAppend #-}
20731745writeAppend :: forall m a. (MonadIO m, Unbox a) =>
20741746 m (MutArray a) -> Fold m a (MutArray a)
2075−writeAppend = append
1747+writeAppend = writeAppendWith (* 2)
20761748
20771749-- XXX We can carry bound as well in the state to make sure we do not lose the
20781750-- remaining capacity. Need to check perf impact.
20791751--
2080−-- | Like 'unsafeCreateOf' but takes a new array allocator @alloc size@
2081−-- function as argument.
1752+-- | Like 'writeNUnsafe' but takes a new array allocator @alloc size@ function
1753+-- as argument.
20821754--
2083−-- >>> unsafeCreateOfWith alloc n = MutArray.unsafeAppendN (alloc n) n
1755+-- >>> writeNWithUnsafe alloc n = MutArray.writeAppendNUnsafe (alloc n) n
20841756--
20851757-- /Pre-release/
2086−{-# INLINE_NORMAL unsafeCreateOfWith #-}
2087−unsafeCreateOfWith :: forall m a. (MonadIO m, Unbox a)
1758+{-# INLINE_NORMAL writeNWithUnsafe #-}
1759+writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a)
20881760 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2089−unsafeCreateOfWith alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
1761+writeNWithUnsafe alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
20901762
20911763 where
20921764
20931765 initial = toArrayUnsafe <$> alloc (max n 0)
20941766
20951767 step (ArrayUnsafe contents start end) x = do
2096− liftIO $ pokeAt end contents x
1768+ liftIO $ pokeByteIndex end contents x
20971769 return
20981770 $ ArrayUnsafe contents start (INDEX_NEXT(end,a))
20991771
2100−{-# DEPRECATED writeNWithUnsafe "Please use unsafeCreateOfWith instead." #-}
2101−{-# INLINE writeNWithUnsafe #-}
2102−writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a)
2103− => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2104−writeNWithUnsafe = unsafeCreateOfWith
2105−
21061772{-# INLINE_NORMAL writeNUnsafeAs #-}
21071773writeNUnsafeAs :: forall m a. (MonadIO m, Unbox a)
21081774 => PinnedState -> Int -> Fold m a (MutArray a)
2109−writeNUnsafeAs ps = unsafeCreateOfWith (newAs ps)
1775+writeNUnsafeAs ps = writeNWithUnsafe (newAs ps)
21101776
2111−-- | Like 'createOf' but does not check the array bounds when writing. The fold
1777+-- | Like 'writeN' but does not check the array bounds when writing. The fold
21121778-- driver must not call the step function more than 'n' times otherwise it will
21131779-- corrupt the memory and crash. This function exists mainly because any
21141780-- conditional in the step function blocks fusion causing 10x performance
21151781-- slowdown.
21161782--
2117−-- >>> unsafeCreateOf = MutArray.unsafeCreateOfWith MutArray.emptyOf
1783+-- >>> writeNUnsafe = MutArray.writeNWithUnsafe MutArray.new
21181784--
2119−{-# INLINE_NORMAL unsafeCreateOf #-}
2120−unsafeCreateOf :: forall m a. (MonadIO m, Unbox a)
2121− => Int -> Fold m a (MutArray a)
2122−unsafeCreateOf = writeNUnsafeAs Unpinned
2123−
2124−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
2125−{-# INLINE writeNUnsafe #-}
1785+{-# INLINE_NORMAL writeNUnsafe #-}
21261786writeNUnsafe :: forall m a. (MonadIO m, Unbox a)
21271787 => Int -> Fold m a (MutArray a)
2128−writeNUnsafe = unsafeCreateOf
2129−
2130−-- | Like 'unsafeCreateOf' but creates a pinned array.
2131−{-# INLINE_NORMAL unsafePinnedCreateOf #-}
2132−unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a)
2133− => Int -> Fold m a (MutArray a)
2134−unsafePinnedCreateOf = writeNUnsafeAs Pinned
1788+writeNUnsafe = writeNUnsafeAs Unpinned
21351789
2136−{-# DEPRECATED pinnedWriteNUnsafe "Please use unsafePinnedCreateOf instead." #-}
2137−{-# INLINE pinnedWriteNUnsafe #-}
1790+-- | Like 'writeNUnsafe' but creates a pinned array.
1791+{-# INLINE_NORMAL pinnedWriteNUnsafe #-}
21381792pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a)
21391793 => Int -> Fold m a (MutArray a)
2140−pinnedWriteNUnsafe = unsafePinnedCreateOf
1794+pinnedWriteNUnsafe = writeNUnsafeAs Pinned
21411795
2142−-- | @createOfWith alloc n@ folds a maximum of @n@ elements into an array
1796+-- | @writeNWith alloc n@ folds a maximum of @n@ elements into an array
21431797-- allocated using the @alloc@ function.
21441798--
2145−-- >>> createOfWith alloc n = Fold.take n (MutArray.unsafeCreateOfWith alloc n)
2146−-- >>> createOfWith alloc n = MutArray.appendN (alloc n) n
1799+-- >>> writeNWith alloc n = Fold.take n (MutArray.writeNWithUnsafe alloc n)
1800+-- >>> writeNWith alloc n = MutArray.writeAppendN (alloc n) n
21471801--
2148−{-# INLINE_NORMAL createOfWith #-}
2149−createOfWith :: forall m a. (MonadIO m, Unbox a)
2150− => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2151−createOfWith alloc n = FL.take n (unsafeCreateOfWith alloc n)
2152−
2153−-- {-# DEPRECATED writeNWith "Please use createOfWith instead." #-}
2154−{-# INLINE writeNWith #-}
1802+{-# INLINE_NORMAL writeNWith #-}
21551803writeNWith :: forall m a. (MonadIO m, Unbox a)
21561804 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2157−writeNWith = createOfWith
1805+writeNWith alloc n = FL.take n (writeNWithUnsafe alloc n)
21581806
21591807{-# INLINE_NORMAL writeNAs #-}
21601808writeNAs ::
@@ -2162,42 +1810,28 @@ writeNAs ::
21621810 => PinnedState
21631811 -> Int
21641812 -> Fold m a (MutArray a)
2165−writeNAs ps = createOfWith (newAs ps)
1813+writeNAs ps = writeNWith (newAs ps)
21661814
2167−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
1815+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
21681816-- 'MutArray'.
21691817--
2170−-- >>> createOf = MutArray.createOfWith MutArray.new
2171−-- >>> createOf n = Fold.take n (MutArray.unsafeCreateOf n)
2172−-- >>> createOf n = MutArray.appendN n (MutArray.emptyOf n)
1818+-- >>> writeN = MutArray.writeNWith MutArray.new
1819+-- >>> writeN n = Fold.take n (MutArray.writeNUnsafe n)
1820+-- >>> writeN n = MutArray.writeAppendN n (MutArray.new n)
21731821--
2174−{-# INLINE_NORMAL createOf #-}
2175−createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2176−createOf = writeNAs Unpinned
2177−
2178−-- XXX Deprecate in major
2179−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
2180−{-# INLINE writeN #-}
1822+{-# INLINE_NORMAL writeN #-}
21811823writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2182−writeN = createOf
1824+writeN = writeNAs Unpinned
21831825
2184−-- | Like 'createOf' but creates a pinned array.
2185−{-# INLINE_NORMAL pinnedCreateOf #-}
2186−pinnedCreateOf ::
2187− forall m a. (MonadIO m, Unbox a)
2188− => Int
2189− -> Fold m a (MutArray a)
2190−pinnedCreateOf = writeNAs Pinned
2191−
2192−{-# DEPRECATED pinnedWriteN "Please use pinnedCreateOf instead." #-}
2193−{-# INLINE pinnedWriteN #-}
1826+-- | Like 'writeN' but creates a pinned array.
1827+{-# INLINE_NORMAL pinnedWriteN #-}
21941828pinnedWriteN ::
21951829 forall m a. (MonadIO m, Unbox a)
21961830 => Int
21971831 -> Fold m a (MutArray a)
2198−pinnedWriteN = pinnedCreateOf
1832+pinnedWriteN = writeNAs Pinned
21991833
2200−-- | Like unsafeCreateOfWith but writes the array in reverse order.
1834+-- | Like writeNWithUnsafe but writes the array in reverse order.
22011835--
22021836-- /Internal/
22031837{-# INLINE_NORMAL writeRevNWithUnsafe #-}
@@ -2214,11 +1848,11 @@ writeRevNWithUnsafe alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
22141848
22151849 step (ArrayUnsafe contents start end) x = do
22161850 let ptr = INDEX_PREV(start,a)
2217− liftIO $ pokeAt ptr contents x
1851+ liftIO $ pokeByteIndex ptr contents x
22181852 return
22191853 $ ArrayUnsafe contents ptr end
22201854
2221−-- | Like createOfWith but writes the array in reverse order.
1855+-- | Like writeNWith but writes the array in reverse order.
22221856--
22231857-- /Internal/
22241858{-# INLINE_NORMAL writeRevNWith #-}
@@ -2226,36 +1860,31 @@ writeRevNWith :: forall m a. (MonadIO m, Unbox a)
22261860 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
22271861writeRevNWith alloc n = FL.take n (writeRevNWithUnsafe alloc n)
22281862
2229−-- | Like 'createOf' but writes the array in reverse order.
1863+-- | Like writeN but writes the array in reverse order.
22301864--
22311865-- /Pre-release/
2232−{-# INLINE_NORMAL revCreateOf #-}
2233−revCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2234−revCreateOf = writeRevNWith new
2235−
2236−{-# DEPRECATED writeRevN "Please use revCreateOf instead." #-}
2237−{-# INLINE writeRevN #-}
1866+{-# INLINE_NORMAL writeRevN #-}
22381867writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2239−writeRevN = revCreateOf
1868+writeRevN = writeRevNWith new
22401869
22411870-- | @pinnedWriteNAligned align n@ folds a maximum of @n@ elements from the
22421871-- input stream to a 'MutArray' aligned to the given size.
22431872--
2244−-- >>> pinnedWriteNAligned align = MutArray.createOfWith (MutArray.pinnedNewAligned align)
2245−-- >>> pinnedWriteNAligned align n = MutArray.appendN n (MutArray.pinnedNewAligned align n)
1873+-- >>> pinnedWriteNAligned align = MutArray.writeNWith (MutArray.pinnedNewAligned align)
1874+-- >>> pinnedWriteNAligned align n = MutArray.writeAppendN n (MutArray.pinnedNewAligned align n)
22461875--
22471876-- /Pre-release/
22481877--
22491878{-# INLINE_NORMAL pinnedWriteNAligned #-}
22501879pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a)
22511880 => Int -> Int -> Fold m a (MutArray a)
2252−pinnedWriteNAligned align = createOfWith (pinnedNewAligned align)
1881+pinnedWriteNAligned align = writeNWith (pinnedNewAligned align)
22531882
22541883-- XXX Buffer to a list instead?
2255−
1884+--
22561885-- | Buffer a stream into a stream of arrays.
22571886--
2258−-- >>> buildChunks n = Fold.many (MutArray.createOf n) Fold.toStreamK
1887+-- >>> writeChunks n = Fold.many (MutArray.writeN n) Fold.toStreamK
22591888--
22601889-- Breaking an array into an array stream can be useful to consume a large
22611890-- array sequentially such that memory of the array is released incrementatlly.
@@ -2264,43 +1893,37 @@ pinnedWriteNAligned align = createOfWith (pinnedNewAligned align)
22641893--
22651894-- /Unimplemented/
22661895--
2267−{-# INLINE_NORMAL buildChunks #-}
2268−buildChunks :: (MonadIO m, Unbox a) =>
2269− Int -> Fold m a (StreamK n (MutArray a))
2270−buildChunks n = FL.many (createOf n) FL.toStreamK
2271−
2272−{-# DEPRECATED writeChunks "Please use buildChunks instead." #-}
2273−{-# INLINE writeChunks #-}
1896+{-# INLINE_NORMAL writeChunks #-}
22741897writeChunks :: (MonadIO m, Unbox a) =>
22751898 Int -> Fold m a (StreamK n (MutArray a))
2276−writeChunks = buildChunks
1899+writeChunks n = FL.many (writeN n) FL.toStreamK
22771900
22781901{-# INLINE_NORMAL writeWithAs #-}
22791902writeWithAs :: forall m a. (MonadIO m, Unbox a)
22801903 => PinnedState -> Int -> Fold m a (MutArray a)
2281−-- writeWithAs ps n = FL.rmapM rightSize $ appendWith (* 2) (newAs ps n)
1904+-- writeWithAs ps n = FL.rmapM rightSize $ writeAppendWith (* 2) (newAs ps n)
22821905writeWithAs ps elemCount =
22831906 FL.rmapM extract $ FL.foldlM' step initial
22841907
22851908 where
22861909
22871910 initial = do
2288− when (elemCount < 0) $ error "createWith: elemCount is negative"
2289− newAs ps elemCount
1911+ when (elemCount < 0) $ error "writeWith: elemCount is negative"
1912+ liftIO $ newAs ps elemCount
22901913
22911914 step arr@(MutArray _ start end bound) x
22921915 | INDEX_NEXT(end,a) > bound = do
22931916 let oldSize = end - start
22941917 newSize = max (oldSize * 2) 1
2295− arr1 <- liftIO $ reallocExplicitAs ps (SIZE_OF(a)) newSize arr
1918+ arr1 <- liftIO $ reallocExplicit (SIZE_OF(a)) newSize arr
22961919 snocUnsafe arr1 x
22971920 step arr x = snocUnsafe arr x
22981921
22991922 extract = liftIO . rightSize
23001923
2301−-- XXX Compare createWith with fromStreamD which uses an array of streams
2302−-- implementation. We can write this using buildChunks above if that is faster.
2303−-- If createWith is faster then we should use that to implement
1924+-- XXX Compare writeWith with fromStreamD which uses an array of streams
1925+-- implementation. We can write this using writeChunks above if that is faster.
1926+-- If writeWith is faster then we should use that to implement
23041927-- fromStreamD.
23051928--
23061929-- XXX The realloc based implementation needs to make one extra copy if we use
@@ -2315,56 +1938,39 @@ writeWithAs ps elemCount =
23151938-- XXX check if GHC's memory allocator is efficient enough. We can try the C
23161939-- malloc to compare against.
23171940
2318−-- | @createWith minCount@ folds the whole input to a single array. The array
1941+-- | @writeWith minCount@ folds the whole input to a single array. The array
23191942-- starts at a size big enough to hold minCount elements, the size is doubled
23201943-- every time the array needs to be grown.
23211944--
23221945-- /Caution! Do not use this on infinite streams./
23231946--
2324−-- >>> f n = MutArray.appendWith (* 2) (MutArray.emptyOf n)
2325−-- >>> createWith n = Fold.rmapM MutArray.rightSize (f n)
2326−-- >>> createWith n = Fold.rmapM MutArray.fromChunksK (MutArray.buildChunks n)
1947+-- >>> f n = MutArray.writeAppendWith (* 2) (MutArray.new n)
1948+-- >>> writeWith n = Fold.rmapM MutArray.rightSize (f n)
1949+-- >>> writeWith n = Fold.rmapM MutArray.fromArrayStreamK (MutArray.writeChunks n)
23271950--
23281951-- /Pre-release/
2329−{-# INLINE_NORMAL createWith #-}
2330−createWith :: forall m a. (MonadIO m, Unbox a)
2331− => Int -> Fold m a (MutArray a)
2332−-- createWith n = FL.rmapM rightSize $ appendWith (* 2) (emptyOf n)
2333−createWith = writeWithAs Unpinned
2334−
2335−{-# DEPRECATED writeWith "Please use createWith instead." #-}
2336−{-# INLINE writeWith #-}
1952+{-# INLINE_NORMAL writeWith #-}
23371953writeWith :: forall m a. (MonadIO m, Unbox a)
23381954 => Int -> Fold m a (MutArray a)
2339−writeWith = createWith
1955+-- writeWith n = FL.rmapM rightSize $ writeAppendWith (* 2) (new n)
1956+writeWith = writeWithAs Unpinned
23401957
23411958-- | Fold the whole input to a single array.
23421959--
2343−-- Same as 'createWith' using an initial array size of 'arrayChunkBytes' bytes
1960+-- Same as 'writeWith' using an initial array size of 'arrayChunkBytes' bytes
23441961-- rounded up to the element size.
23451962--
23461963-- /Caution! Do not use this on infinite streams./
23471964--
2348−{-# INLINE create #-}
2349−create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2350−create = createWith (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
2351−
2352−-- XXX Deprecate in major
2353−-- {-# DEPRECATED write "Please use create instead." #-}
23541965{-# INLINE write #-}
23551966write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2356−write = create
1967+write = writeWith (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
23571968
2358−-- | Like 'create' but creates a pinned array.
2359−{-# INLINE pinnedCreate #-}
2360−pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2361−pinnedCreate =
2362− writeWithAs Pinned (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
2363−
2364−{-# DEPRECATED pinnedWrite "Please use pinnedCreate instead." #-}
1969+-- | Like 'write' but creates a pinned array.
23651970{-# INLINE pinnedWrite #-}
23661971pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2367−pinnedWrite = pinnedCreate
1972+pinnedWrite =
1973+ writeWithAs Pinned (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
23681974
23691975-------------------------------------------------------------------------------
23701976-- construct from streams, known size
@@ -2374,31 +1980,25 @@ pinnedWrite = pinnedCreate
23741980fromStreamDNAs :: forall m a. (MonadIO m, Unbox a)
23751981 => PinnedState -> Int -> D.Stream m a -> m (MutArray a)
23761982fromStreamDNAs ps limit str = do
2377− (arr :: MutArray a) <- newAs ps limit
1983+ (arr :: MutArray a) <- liftIO $ newAs ps limit
23781984 end <- D.foldlM' (fwrite (arrContents arr)) (return $ arrEnd arr) $ D.take limit str
23791985 return $ arr {arrEnd = end}
23801986
23811987 where
23821988
23831989 fwrite arrContents ptr x = do
2384− liftIO $ pokeAt ptr arrContents x
1990+ liftIO $ pokeByteIndex ptr arrContents x
23851991 return $ INDEX_NEXT(ptr,a)
23861992
2387−-- | Use the 'createOf' fold instead.
1993+-- | Use the 'writeN' fold instead.
23881994--
2389−-- >>> fromStreamN n = Stream.fold (MutArray.createOf n)
1995+-- >>> fromStreamDN n = Stream.fold (MutArray.writeN n)
23901996--
2391−{-# INLINE_NORMAL fromStreamN #-}
2392−fromStreamN :: forall m a. (MonadIO m, Unbox a)
2393− => Int -> D.Stream m a -> m (MutArray a)
2394−-- fromStreamDN n = D.fold (createOf n)
2395−fromStreamN = fromStreamDNAs Unpinned
2396−
2397−{-# DEPRECATED fromStreamDN "Please use fromStreamN instead." #-}
2398−{-# INLINE fromStreamDN #-}
1997+{-# INLINE_NORMAL fromStreamDN #-}
23991998fromStreamDN :: forall m a. (MonadIO m, Unbox a)
24001999 => Int -> D.Stream m a -> m (MutArray a)
2401−fromStreamDN = fromStreamN
2000+-- fromStreamDN n = D.fold (writeN n)
2001+fromStreamDN = fromStreamDNAs Unpinned
24022002
24032003-- | Create a 'MutArray' from the first N elements of a list. The array is
24042004-- allocated to size N, if the list terminates before N elements then the
@@ -2418,70 +2018,16 @@ pinnedFromListN n xs = fromStreamDNAs Pinned n $ D.fromList xs
24182018-- /Pre-release/
24192019{-# INLINE fromListRevN #-}
24202020fromListRevN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
2421−fromListRevN n xs = D.fold (revCreateOf n) $ D.fromList xs
2422−
2423−-- | Convert a pure stream in Identity monad to a mutable array.
2424−{-# INLINABLE fromPureStreamN #-}
2425−fromPureStreamN :: (MonadIO m, Unbox a) =>
2426− Int -> Stream Identity a -> m (MutArray a)
2427−fromPureStreamN n xs =
2428− D.fold (createOf n) $ D.morphInner (return . runIdentity) xs
2021+fromListRevN n xs = D.fold (writeRevN n) $ D.fromList xs
24292022
24302023-- | Convert a pure stream in Identity monad to a mutable array.
24312024{-# INLINABLE fromPureStream #-}
24322025fromPureStream :: (MonadIO m, Unbox a) => Stream Identity a -> m (MutArray a)
24332026fromPureStream xs =
2434− D.fold create $ D.morphInner (return . runIdentity) xs
2435−
2436−{-# INLINABLE fromPtrN #-}
2437−fromPtrN :: MonadIO m => Int -> Ptr Word8 -> m (MutArray Word8)
2438−fromPtrN len addr = do
2439− -- memcpy is better than stream copy when the size is known.
2440− -- XXX We can implement a stream copy in a similar way by streaming Word64
2441− -- first and then remaining Word8.
2442− arr <- new len
2443− _ <- unsafeAsPtr arr
2444− (\ptr -> liftIO $ c_memcpy ptr addr (fromIntegral len))
2445− return (arr {arrEnd = len})
2446−
2447−{-# INLINABLE fromByteStr# #-}
2448−fromByteStr# :: MonadIO m => Addr# -> m (MutArray Word8)
2449−fromByteStr# addr = do
2450− -- It is better to count the size first and allocate exact space.
2451− -- Also, memcpy is better than stream copy when the size is known.
2452− -- C strlen compares 4 bytes at a time, so is better than the stream
2453− -- version. https://github.com/bminor/glibc/blob/master/string/strlen.c
2454− -- XXX We can possibly use a stream of Word64 to do the same.
2455− -- fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
2456− len <- liftIO $ c_strlen (Ptr addr)
2457− let lenInt = fromIntegral len
2458− arr <- new lenInt
2459− _ <- unsafeAsPtr arr (\ptr -> liftIO $ c_memcpy ptr (Ptr addr) len)
2460− return (arr {arrEnd = lenInt})
2027+ liftIO $ D.fold write $ D.morphInner (return . runIdentity) xs
24612028
24622029-------------------------------------------------------------------------------
2463−-- convert a stream of arrays to a single array by reallocating and copying
2464−-------------------------------------------------------------------------------
2465−
2466−-- XXX Both of these implementations of splicing seem to perform equally well.
2467−-- We need to perform benchmarks over a range of sizes though.
2468−
2469−-- | Also see 'fromChunksK'.
2470−{-# INLINE fromChunksRealloced #-}
2471−fromChunksRealloced :: forall m a. (MonadIO m, Unbox a)
2472− => Stream m (MutArray a) -> m (MutArray a)
2473−fromChunksRealloced s = do
2474− res <- D.uncons s
2475− case res of
2476− Just (a, strm) -> do
2477− arr <- D.foldlM' spliceExp (pure a) strm
2478− -- Reallocation is exponential so there may be 50% empty space in
2479− -- worst case. One more reallocation to reclaim the space.
2480− rightSize arr
2481− Nothing -> pure nil
2482−
2483−-------------------------------------------------------------------------------
2484−-- convert a stream of arrays to a single array by buffering arrays first
2030+-- convert stream to a single array
24852031-------------------------------------------------------------------------------
24862032
24872033{-# INLINE arrayStreamKLength #-}
@@ -2491,72 +2037,34 @@ arrayStreamKLength as = K.foldl' (+) 0 (K.map length as)
24912037-- | Convert an array stream to an array. Note that this requires peak memory
24922038-- that is double the size of the array stream.
24932039--
2494−{-# INLINE fromChunkskAs #-}
2495−fromChunkskAs :: (Unbox a, MonadIO m) =>
2496− PinnedState -> StreamK m (MutArray a) -> m (MutArray a)
2497−fromChunkskAs ps as = do
2498− len <- arrayStreamKLength as
2499− arr <- newAs ps len
2500− -- XXX is StreamK fold faster or StreamD fold?
2501− K.foldlM' spliceUnsafe (pure arr) as
2502− -- fromStreamDN len $ D.unfoldMany reader $ D.fromStreamK as
2503−
2504−-- XXX Need to compare this with fromChunks and fromChunkList and keep the
2505−-- fastest or simplest one if all are equally fast.
2506−
2507−-- | Convert an array stream to an array. Note that this requires peak memory
2508−-- that is double the size of the array stream.
2509−--
2510−-- Also see 'fromChunksRealloced'.
2511−--
2512−{-# INLINE fromChunksK #-}
2513−fromChunksK :: (Unbox a, MonadIO m) =>
2514− StreamK m (MutArray a) -> m (MutArray a)
2515−fromChunksK = fromChunkskAs Unpinned
2516−
2517−{-# DEPRECATED fromArrayStreamK "Please use fromChunksK instead." #-}
25182040{-# INLINE fromArrayStreamK #-}
25192041fromArrayStreamK :: (Unbox a, MonadIO m) =>
25202042 StreamK m (MutArray a) -> m (MutArray a)
2521−fromArrayStreamK = fromChunksK
2043+fromArrayStreamK as = do
2044+ len <- arrayStreamKLength as
2045+ fromStreamDN len $ D.unfoldMany reader $ D.fromStreamK as
25222046
25232047{-# INLINE fromStreamDAs #-}
25242048fromStreamDAs ::
25252049 (MonadIO m, Unbox a) => PinnedState -> D.Stream m a -> m (MutArray a)
2526−fromStreamDAs ps m =
2527− arrayStreamKFromStreamDAs Unpinned m >>= fromChunkskAs ps
2528−
2529−-- | Create an 'Array' from a stream. This is useful when we want to create a
2530−-- single array from a stream of unknown size. 'createOf' is at least twice
2531−-- as efficient when the size is already known.
2532−--
2533−-- Note that if the input stream is too large memory allocation for the array
2534−-- may fail. When the stream size is not known, `chunksOf` followed by
2535−-- processing of indvidual arrays in the resulting stream should be preferred.
2536−--
2537−-- /Pre-release/
2538−{-# INLINE fromStream #-}
2539−fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
2540−fromStream = fromStreamDAs Unpinned
2050+fromStreamDAs ps m = arrayStreamKFromStreamDAs ps m >>= fromArrayStreamK
25412051
2542−-- fromStream (Stream m) = P.fold create m
25432052-- CAUTION: a very large number (millions) of arrays can degrade performance
25442053-- due to GC overhead because we need to buffer the arrays before we flatten
25452054-- all the arrays.
25462055--
2547−-- XXX Compare if this is faster or "fold create".
2056+-- XXX Compare if this is faster or "fold write".
25482057--
25492058-- | We could take the approach of doubling the memory allocation on each
25502059-- overflow. This would result in more or less the same amount of copying as in
25512060-- the chunking approach. However, if we have to shrink in the end then it may
25522061-- result in an extra copy of the entire data.
25532062--
2554−-- >>> fromStreamD = StreamD.fold MutArray.create
2063+-- >>> fromStreamD = StreamD.fold MutArray.write
25552064--
25562065{-# INLINE fromStreamD #-}
2557−{-# DEPRECATED fromStreamD "Please use fromStream instead." #-}
25582066fromStreamD :: (MonadIO m, Unbox a) => D.Stream m a -> m (MutArray a)
2559−fromStreamD = fromStream
2067+fromStreamD = fromStreamDAs Unpinned
25602068
25612069-- | Create a 'MutArray' from a list. The list must be of finite size.
25622070--
@@ -2577,54 +2085,26 @@ pinnedFromList xs = fromStreamDAs Pinned $ D.fromList xs
25772085fromListRev :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
25782086fromListRev xs = fromListRevN (Prelude.length xs) xs
25792087
2580−-------------------------------------------------------------------------------
2581−-- Cloning
2582−-------------------------------------------------------------------------------
2583−
2584−-- Arrays are aligned on 64-bit boundaries. The fastest way to copy an array is
2585−-- to unsafeCast it to Word64, read it, write it to Word64 array and unsafeCast
2586−-- it again. We can use SIMD read/write as well.
2587−
2588−{-# INLINE cloneAs #-}
2589−cloneAs ::
2590− ( MonadIO m
2591−#ifdef DEVBUILD
2592− , Unbox a
2593−#endif
2594− )
2595− => PinnedState -> MutArray a -> m (MutArray a)
2596−cloneAs ps src =
2597− do
2598− let startSrc = arrStart src
2599− srcLen = arrEnd src - startSrc
2600− newArrContents <-
2601− Unboxed.cloneSliceUnsafeAs ps startSrc srcLen (arrContents src)
2602− return $ MutArray newArrContents 0 srcLen srcLen
2603−
2604−{-# INLINE clone #-}
2605−clone ::
2606− ( MonadIO m
2607−#ifdef DEVBUILD
2608− , Unbox a
2609−#endif
2610− )
2611− => MutArray a -> m (MutArray a)
2612−clone = cloneAs Unpinned
2613−
2614−{-# INLINE pinnedClone #-}
2615−pinnedClone ::
2616− ( MonadIO m
2617−#ifdef DEVBUILD
2618− , Unbox a
2619−#endif
2620− )
2621− => MutArray a -> m (MutArray a)
2622−pinnedClone = cloneAs Pinned
2623−
26242088-------------------------------------------------------------------------------
26252089-- Combining
26262090-------------------------------------------------------------------------------
26272091
2092+-- | Put a sub range of a source array into a subrange of a destination array.
2093+-- This is not safe as it does not check the bounds.
2094+{-# INLINE putSliceUnsafe #-}
2095+putSliceUnsafe :: MonadIO m => MutArray a -> Int -> MutArray a -> Int -> Int -> m ()
2096+putSliceUnsafe src srcStartBytes dst dstStartBytes lenBytes = liftIO $ do
2097+ assertM(lenBytes <= arrBound dst - dstStartBytes)
2098+ assertM(lenBytes <= arrEnd src - srcStartBytes)
2099+ let !(I# srcStartBytes#) = srcStartBytes
2100+ !(I# dstStartBytes#) = dstStartBytes
2101+ !(I# lenBytes#) = lenBytes
2102+ let arrS# = getMutableByteArray# (arrContents src)
2103+ arrD# = getMutableByteArray# (arrContents dst)
2104+ IO $ \s# -> (# copyMutableByteArray#
2105+ arrS# srcStartBytes# arrD# dstStartBytes# lenBytes# s#
2106+ , () #)
2107+
26282108-- | Copy two arrays into a newly allocated array. If the first array is pinned
26292109-- the spliced array is also pinned.
26302110{-# INLINE spliceCopy #-}
@@ -2633,19 +2113,21 @@ spliceCopy :: forall m a. MonadIO m =>
26332113 Unbox a =>
26342114#endif
26352115 MutArray a -> MutArray a -> m (MutArray a)
2636−spliceCopy arr1 arr2 = do
2116+spliceCopy arr1 arr2 = liftIO $ do
26372117 let start1 = arrStart arr1
26382118 start2 = arrStart arr2
26392119 len1 = arrEnd arr1 - start1
26402120 len2 = arrEnd arr2 - start2
2641− let len = len1 + len2
2121+ let newLen = len1 + len2
26422122 newArrContents <-
26432123 if Unboxed.isPinned (arrContents arr1)
2644− then liftIO $ Unboxed.pinnedNew len
2645− else liftIO $ Unboxed.new len
2646− putSliceUnsafe (arrContents arr1) start1 newArrContents 0 len1
2647− putSliceUnsafe (arrContents arr2) start2 newArrContents len1 len2
2648− return $ MutArray newArrContents 0 len len
2124+ then Unboxed.pinnedNewBytes newLen
2125+ else Unboxed.newBytes newLen
2126+ let len = len1 + len2
2127+ newArr = MutArray newArrContents 0 len len
2128+ putSliceUnsafe arr1 start1 newArr 0 len1
2129+ putSliceUnsafe arr2 start2 newArr len1 len2
2130+ return newArr
26492131
26502132-- | Really really unsafe, appends the second array into the first array. If
26512133-- the first array does not have enough space it may cause silent data
@@ -2654,13 +2136,12 @@ spliceCopy arr1 arr2 = do
26542136spliceUnsafe :: MonadIO m =>
26552137 MutArray a -> MutArray a -> m (MutArray a)
26562138spliceUnsafe dst src =
2657− do
2139+ liftIO $ do
26582140 let startSrc = arrStart src
26592141 srcLen = arrEnd src - startSrc
26602142 endDst = arrEnd dst
26612143 assertM(endDst + srcLen <= arrBound dst)
2662− putSliceUnsafe
2663− (arrContents src) startSrc (arrContents dst) endDst srcLen
2144+ putSliceUnsafe src startSrc dst endDst srcLen
26642145 return $ dst {arrEnd = endDst + srcLen}
26652146
26662147-- | @spliceWith sizer dst src@ mutates @dst@ to append @src@. If there is no
@@ -2676,7 +2157,7 @@ spliceWith :: forall m a. (MonadIO m, Unbox a) =>
26762157 (Int -> Int -> Int) -> MutArray a -> MutArray a -> m (MutArray a)
26772158spliceWith sizer dst@(MutArray _ start end bound) src = do
26782159{-
2679− let f = appendWith (`sizer` byteLength src) (return dst)
2160+ let f = writeAppendWith (`sizer` byteLength src) (return dst)
26802161 in D.fold f (toStreamD src)
26812162-}
26822163 assert (end <= bound) (return ())
@@ -2692,7 +2173,7 @@ spliceWith sizer dst@(MutArray _ start end bound) src = do
26922173 $ "splice: newSize is less than the total size "
26932174 ++ "of arrays being appended. Please check the "
26942175 ++ "sizer function passed."
2695− realloc newSizeInBytes dst
2176+ liftIO $ realloc newSizeInBytes dst
26962177 else return dst
26972178 spliceUnsafe dst1 src
26982179
@@ -2729,24 +2210,14 @@ spliceExp = spliceWith (\l1 l2 -> max (l1 * 2) (l1 + l2))
27292210-- Splitting
27302211-------------------------------------------------------------------------------
27312212
2732−-- | Generate a stream of array slices using a predicate. The array element
2733−-- matching the predicate is dropped.
2734−--
2735−-- /Pre-release/
2736−{-# INLINE splitOn #-}
2737−splitOn :: (MonadIO m, Unbox a) =>
2738− (a -> Bool) -> MutArray a -> Stream m (MutArray a)
2739−splitOn predicate arr =
2740− fmap (\(i, len) -> getSliceUnsafe i len arr)
2741− $ D.indexOnSuffix predicate (read arr)
2742−
27432213-- | Drops the separator byte
27442214{-# INLINE breakOn #-}
27452215breakOn :: MonadIO m
27462216 => Word8 -> MutArray Word8 -> m (MutArray Word8, Maybe (MutArray Word8))
2747−breakOn sep arr@MutArray{..} = unsafeAsPtr arr $ \p -> liftIO $ do
2217+breakOn sep arr@MutArray{..} = asPtrUnsafe arr $ \p -> liftIO $ do
2218+ -- XXX Instead of using asPtrUnsafe (pinning memory) we can pass unlifted
2219+ -- Addr# to memchr and it should be safe (from ghc 8.4).
27482220 -- XXX We do not need memchr here, we can use a Haskell equivalent.
2749− -- Need efficient stream based primitives that work on Word64.
27502221 loc <- c_memchr p sep (fromIntegral $ byteLength arr)
27512222 let sepIndex = loc `minusPtr` p
27522223 return $
@@ -2767,41 +2238,32 @@ breakOn sep arr@MutArray{..} = unsafeAsPtr arr $ \p -> liftIO $ do
27672238 }
27682239 )
27692240
2770−-- | Like 'splitAt' but does not check whether the index is valid.
2771−--
2772−{-# INLINE unsafeSplitAt #-}
2773−unsafeSplitAt :: forall a. Unbox a =>
2774− Int -> MutArray a -> (MutArray a, MutArray a)
2775−unsafeSplitAt i MutArray{..} =
2776− let off = i * SIZE_OF(a)
2777− p = arrStart + off
2778− in ( MutArray
2779− { arrContents = arrContents
2780− , arrStart = arrStart
2781− , arrEnd = p
2782− , arrBound = p
2783− }
2784− , MutArray
2785− { arrContents = arrContents
2786− , arrStart = p
2787− , arrEnd = arrEnd
2788− , arrBound = arrBound
2789− }
2790− )
2791−
27922241-- | Create two slices of an array without copying the original array. The
27932242-- specified index @i@ is the first index of the second slice.
27942243--
2795−{-# INLINE splitAt #-}
27962244splitAt :: forall a. Unbox a => Int -> MutArray a -> (MutArray a, MutArray a)
2797−splitAt i arr =
2245+splitAt i arr@MutArray{..} =
27982246 let maxIndex = length arr - 1
27992247 in if i < 0
28002248 then error "sliceAt: negative array index"
28012249 else if i > maxIndex
28022250 then error $ "sliceAt: specified array index " ++ show i
28032251 ++ " is beyond the maximum index " ++ show maxIndex
2804− else unsafeSplitAt i arr
2252+ else let off = i * SIZE_OF(a)
2253+ p = arrStart + off
2254+ in ( MutArray
2255+ { arrContents = arrContents
2256+ , arrStart = arrStart
2257+ , arrEnd = p
2258+ , arrBound = p
2259+ }
2260+ , MutArray
2261+ { arrContents = arrContents
2262+ , arrStart = p
2263+ , arrEnd = arrEnd
2264+ , arrBound = arrBound
2265+ }
2266+ )
28052267
28062268-------------------------------------------------------------------------------
28072269-- Casting
@@ -2851,16 +2313,13 @@ cast arr =
28512313-- guarantees that unsafe calls will be performed in the calling thread. Making
28522314-- it safe to pass heap-allocated objects to unsafe functions.
28532315
2854−-- XXX Should we just name it asPtr, the unsafety is implicit for any pointer
2855−-- operations. And we are safe from Haskell perspective because we will be
2856−-- pinning the memory.
2857−--
2858−-- XXX we cannot pass the length of the ptr here as in some cases it may not be
2859−-- available e.g. a null terminated C string. However, we can create another
2860−-- flavor of the API e.g. asPtrN.
2316+-- Unsafe because of direct pointer operations. The user must ensure that they
2317+-- are writing within the legal bounds of the array. Should we just name it
2318+-- asPtr, the unsafety is implicit for any pointer operations. And we are safe
2319+-- from Haskell perspective because we will be pinning the memory.
28612320
2862−-- | Use a @MutArray a@ as @Ptr a@. This is useful when we want to pass an
2863−-- array as a pointer to some operating system call or to a "safe" FFI call.
2321+-- | Use an @MutArray a@ as @Ptr a@. This is useful when we want to pass an array
2322+-- as a pointer to some operating system call or to a "safe" FFI call.
28642323--
28652324-- If the array is not pinned it is copied to pinned memory before passing it
28662325-- to the monadic action.
@@ -2870,382 +2329,56 @@ cast arr =
28702329-- opportunistically before this operation occurs, to avoid the cost of a copy
28712330-- if possible.
28722331--
2873−-- /Unsafe/ because of direct pointer operations. The user must ensure that
2874−-- they are writing within the legal bounds of the array.
2332+-- /Unsafe/
28752333--
28762334-- /Pre-release/
28772335--
2878−{-# INLINE unsafePinnedAsPtr #-}
2879−unsafePinnedAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2880−unsafePinnedAsPtr arr f =
2881− Unboxed.unsafePinnedAsPtr
2882− (arrContents arr) (\ptr -> f (ptr `plusPtr` arrStart arr))
2883−
2884−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
2885−{-# INLINE asPtrUnsafe #-}
28862336asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2887−asPtrUnsafe = unsafePinnedAsPtr
2337+asPtrUnsafe arr f = do
2338+ contents <- liftIO $ Unboxed.pin $ arrContents arr
2339+ let !ptr = Ptr (byteArrayContents#
2340+ (unsafeCoerce# (getMutableByteArray# contents)))
28882341
2889−{-# INLINE unsafeAsPtr #-}
2890−unsafeAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2891−unsafeAsPtr arr f =
2892− Unboxed.unsafeAsPtr
2893− (arrContents arr) (\ptr -> f (ptr `plusPtr` arrStart arr))
2342+ r <- f (ptr `plusPtr` arrStart arr)
2343+ liftIO $ touch contents
2344+ return r
28942345
28952346-------------------------------------------------------------------------------
28962347-- Equality
28972348-------------------------------------------------------------------------------
28982349
2899−-- | Byte compare two arrays. Compare the length of the arrays. If the length
2900−-- is equal, compare the lexicographical ordering of two underlying byte arrays
2901−-- otherwise return the result of length comparison.
2902−--
2903−-- /Unsafe/: Note that the 'Unbox' instance of sum types with constructors of
2904−-- different sizes may leave some memory uninitialized which can make byte
2905−-- comparison unreliable.
2350+-- | Compare the length of the arrays. If the length is equal, compare the
2351+-- lexicographical ordering of two underlying byte arrays otherwise return the
2352+-- result of length comparison.
29062353--
29072354-- /Pre-release/
2908−{-# INLINE byteCmp #-}
2909−byteCmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
2910−byteCmp arr1 arr2 = do
2911− let marr1 = getMutableByteArray# (arrContents arr1)
2912− marr2 = getMutableByteArray# (arrContents arr2)
2913− !(I# st1#) = arrStart arr1
2914− !(I# st2#) = arrStart arr2
2915− !(I# len#) = byteLength arr1
2916− case compare (byteLength arr1) (byteLength arr2) of
2917− EQ -> do
2918− r <- liftIO $ IO $ \s# ->
2919− let res =
2920− I#
2921− (compareByteArrays#
2922− (unsafeCoerce# marr1)
2923− st1#
2924− (unsafeCoerce# marr2)
2925− st2#
2926− len#)
2927− in (# s#, res #)
2928− return $ compare r 0
2929− x -> return x
2930−
29312355{-# INLINE cmp #-}
2932−{-# DEPRECATED cmp "Please use byteCmp instead." #-}
29332356cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
2934−cmp = byteCmp
2935−
2936−-- | Byte equality of two arrays.
2937−--
2938−-- >>> byteEq arr1 arr2 = (==) EQ $ MArray.byteCmp arr1 arr2
2939−--
2940−-- /Unsafe/: See 'byteCmp'.
2941−{-# INLINE byteEq #-}
2942−byteEq :: MonadIO m => MutArray a -> MutArray a -> m Bool
2943−byteEq arr1 arr2 = fmap (EQ ==) $ byteCmp arr1 arr2
2944−
2945−-------------------------------------------------------------------------------
2946−-- Compact
2947−-------------------------------------------------------------------------------
2948−
2949−-- Note: LE versions avoid an extra copy compared to GE. LE parser trades
2950−-- backtracking one array in lieu of avoiding a copy. However, LE and GE both
2951−-- can leave some memory unused. They can split the last array to fit it
2952−-- exactly in the space.
2953−
2954−{-# INLINE_NORMAL pCompactLeAs #-}
2955−pCompactLeAs ::
2956− forall m a. (MonadIO m, Unbox a)
2957− => PinnedState -> Int -> Parser (MutArray a) m (MutArray a)
2958−pCompactLeAs ps maxElems = Parser step initial extract
2959−
2960− where
2961−
2962− maxBytes = maxElems * SIZE_OF(a)
2963−
2964− functionName = "Streamly.Internal.Data.MutArray.pCompactLE"
2965−
2966− initial =
2967− return
2968− $ if maxElems <= 0
2969− then error
2970− $ functionName
2971− ++ ": the size of arrays ["
2972− ++ show maxElems ++ "] must be a natural number"
2973− else Parser.IPartial Nothing
2974−
2975− step Nothing arr =
2976− return
2977− $ let len = byteLength arr
2978− in if len >= maxBytes
2979− then Parser.Done 0 arr
2980− else Parser.Partial 0 (Just arr)
2981− -- XXX Split the last array to use the space more compactly.
2982− step (Just buf) arr =
2983− let len = byteLength buf + byteLength arr
2984− in if len > maxBytes
2985− then return $ Parser.Done 1 buf
2986− else do
2987− buf1 <-
2988− if byteCapacity buf < maxBytes
2989− then liftIO $ reallocExplicitAs
2990− ps (SIZE_OF(a)) maxBytes buf
2991− else return buf
2992− buf2 <- spliceUnsafe buf1 arr
2993− return $ Parser.Partial 0 (Just buf2)
2994−
2995− extract Nothing = return $ Parser.Done 0 nil
2996− extract (Just buf) = return $ Parser.Done 0 buf
2997−
2998−-- | Parser @pCompactLE maxElems@ coalesces adjacent arrays in the input stream
2999−-- only if the combined size would be less than or equal to @maxElems@
3000−-- elements. Note that it won't split an array if the original array is already
3001−-- larger than maxElems.
3002−--
3003−-- @maxElems@ must be greater than 0.
3004−--
3005−-- Generates unpinned arrays irrespective of the pinning status of input
3006−-- arrays.
3007−--
3008−-- /Internal/
3009−{-# INLINE pCompactLE #-}
3010−pCompactLE ::
3011− forall m a. (MonadIO m, Unbox a)
3012− => Int -> Parser (MutArray a) m (MutArray a)
3013−pCompactLE = pCompactLeAs Unpinned
3014−
3015−-- | Pinned version of 'pCompactLE'.
3016−{-# INLINE pPinnedCompactLE #-}
3017−pPinnedCompactLE ::
3018− forall m a. (MonadIO m, Unbox a)
3019− => Int -> Parser (MutArray a) m (MutArray a)
3020−pPinnedCompactLE = pCompactLeAs Pinned
3021−
3022−data SpliceState s arr
3023− = SpliceInitial s
3024− | SpliceBuffering s arr
3025− | SpliceYielding arr (SpliceState s arr)
3026− | SpliceFinish
3027−
3028−-- This mutates the first array (if it has space) to append values from the
3029−-- second one. This would work for immutable arrays as well because an
3030−-- immutable array never has additional space so a new array is allocated
3031−-- instead of mutating it.
3032−
3033−{-# INLINE_NORMAL compactLeAs #-}
3034−compactLeAs :: forall m a. (MonadIO m, Unbox a)
3035− => PinnedState -> Int -> D.Stream m (MutArray a) -> D.Stream m (MutArray a)
3036−compactLeAs ps maxElems (D.Stream step state) =
3037− D.Stream step' (SpliceInitial state)
3038−
3039− where
3040−
3041− maxBytes = maxElems * SIZE_OF(a)
3042−
3043− functionName = "Streamly.Internal.Data.MutArray.rCompactLE"
3044−
3045− {-# INLINE_LATE step' #-}
3046− step' gst (SpliceInitial st) = do
3047− when (maxElems <= 0) $
3048− -- XXX we can pass the module string from the higher level API
3049− error $ functionName ++ ": the size of arrays [" ++ show maxElems
3050− ++ "] must be a natural number"
3051− r <- step gst st
3052− case r of
3053− D.Yield arr s -> return $
3054− let len = byteLength arr
3055− in if len >= maxBytes
3056− then D.Skip (SpliceYielding arr (SpliceInitial s))
3057− else D.Skip (SpliceBuffering s arr)
3058− D.Skip s -> return $ D.Skip (SpliceInitial s)
3059− D.Stop -> return D.Stop
3060−
3061− -- XXX Split the last array to use the space more compactly.
3062− step' gst (SpliceBuffering st buf) = do
3063− r <- step gst st
3064− case r of
3065− D.Yield arr s -> do
3066− let len = byteLength buf + byteLength arr
3067− if len > maxBytes
3068− then return $
3069− D.Skip (SpliceYielding buf (SpliceBuffering s arr))
3070− else do
3071− buf1 <- if byteCapacity buf < maxBytes
3072− then liftIO $ reallocExplicitAs
3073− ps (SIZE_OF(a)) maxBytes buf
3074− else return buf
3075− buf2 <- spliceUnsafe buf1 arr
3076− return $ D.Skip (SpliceBuffering s buf2)
3077− D.Skip s -> return $ D.Skip (SpliceBuffering s buf)
3078− D.Stop -> return $ D.Skip (SpliceYielding buf SpliceFinish)
3079−
3080− step' _ SpliceFinish = return D.Stop
3081−
3082− step' _ (SpliceYielding arr next) = return $ D.Yield arr next
3083−
3084−
3085−{-# INLINE_NORMAL fCompactGeAs #-}
3086−fCompactGeAs ::
3087− forall m a. (MonadIO m, Unbox a)
3088− => PinnedState -> Int -> FL.Fold m (MutArray a) (MutArray a)
3089−fCompactGeAs ps minElems = Fold step initial extract extract
3090−
3091− where
3092−
3093− minBytes = minElems * SIZE_OF(a)
3094−
3095− functionName = "Streamly.Internal.Data.MutArray.fCompactGE"
3096−
3097− initial =
3098− return
3099− $ if minElems < 0
3100− then error
3101− $ functionName
3102− ++ ": the size of arrays ["
3103− ++ show minElems ++ "] must be a natural number"
3104− else FL.Partial Nothing
3105−
3106− step Nothing arr =
3107− return
3108− $ let len = byteLength arr
3109− in if len >= minBytes
3110− then FL.Done arr
3111− else FL.Partial (Just arr)
3112− -- XXX Buffer arrays as a list to avoid copy and reallocations
3113− step (Just buf) arr = do
3114− let len = byteLength buf + byteLength arr
3115− buf1 <-
3116− if byteCapacity buf < len
3117− then liftIO $ reallocExplicitAs
3118− ps (SIZE_OF(a)) (max minBytes len) buf
3119− else return buf
3120− buf2 <- spliceUnsafe buf1 arr
3121− if len >= minBytes
3122− then return $ FL.Done buf2
3123− else return $ FL.Partial (Just buf2)
3124−
3125− extract Nothing = return nil
3126− extract (Just buf) = return buf
3127−
3128−-- | Fold @fCompactGE minElems@ coalesces adjacent arrays in the input stream
3129−-- until the size becomes greater than or equal to @minElems@.
3130−--
3131−-- Generates unpinned arrays irrespective of the pinning status of input
3132−-- arrays.
3133−{-# INLINE fCompactGE #-}
3134−fCompactGE ::
3135− forall m a. (MonadIO m, Unbox a)
3136− => Int -> FL.Fold m (MutArray a) (MutArray a)
3137−fCompactGE = fCompactGeAs Unpinned
3138−
3139−-- | Pinned version of 'fCompactGE'.
3140−{-# INLINE fPinnedCompactGE #-}
3141−fPinnedCompactGE ::
3142− forall m a. (MonadIO m, Unbox a)
3143− => Int -> FL.Fold m (MutArray a) (MutArray a)
3144−fPinnedCompactGE = fCompactGeAs Pinned
3145−
3146−{-# INLINE_NORMAL lCompactGeAs #-}
3147−lCompactGeAs :: forall m a. (MonadIO m, Unbox a)
3148− => PinnedState -> Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3149−-- The fold version turns out to be a little bit slower.
3150−-- lCompactGeAs ps n = FL.many (fCompactGeAs ps n)
3151−lCompactGeAs ps minElems (Fold step1 initial1 _ final1) =
3152− Fold step initial extract final
3153−
3154− where
3155−
3156− minBytes = minElems * SIZE_OF(a)
3157−
3158− functionName = "Streamly.Internal.Data.MutArray.lCompactGE"
3159−
3160− initial = do
3161− when (minElems <= 0) $
3162− -- XXX we can pass the module string from the higher level API
3163− error $ functionName ++ ": the size of arrays ["
3164− ++ show minElems ++ "] must be a natural number"
3165−
3166− r <- initial1
3167− return $ first (Tuple' Nothing) r
3168−
3169− {-# INLINE runInner #-}
3170− runInner len acc buf =
3171− if len >= minBytes
3172− then do
3173− r <- step1 acc buf
3174− case r of
3175− FL.Done _ -> return $ FL.Done ()
3176− FL.Partial s -> do
3177− _ <- final1 s
3178− res <- initial1
3179− return $ first (Tuple' Nothing) res
3180− else return $ FL.Partial $ Tuple' (Just buf) acc
3181−
3182− step (Tuple' Nothing r1) arr =
3183− runInner (byteLength arr) r1 arr
3184−
3185− -- XXX Buffer arrays as a list to avoid copy and reallocations
3186− step (Tuple' (Just buf) r1) arr = do
3187− let len = byteLength buf + byteLength arr
3188− buf1 <- if byteCapacity buf < len
3189− then liftIO $ reallocExplicitAs
3190− ps (SIZE_OF(a)) (max minBytes len) buf
3191− else return buf
3192− buf2 <- spliceUnsafe buf1 arr
3193− runInner len r1 buf2
3194−
3195− -- XXX Several folds do extract >=> final, therefore, we need to make final
3196− -- return "m b" rather than using extract post it if we want extract to be
3197− -- partial.
3198− --
3199− -- extract forces the pending buffer to be sent to the fold which is not
3200− -- what we want.
3201− extract _ = error "lCompactGE: not designed for scanning"
3202−
3203− final (Tuple' Nothing r1) = final1 r1
3204− final (Tuple' (Just buf) r1) = do
3205− r <- step1 r1 buf
3206− case r of
3207− FL.Partial rr -> final1 rr
3208− FL.Done _ -> return ()
3209−
3210−-- | Like 'compactGE' but for transforming folds instead of stream.
3211−--
3212−-- >>> lCompactGE n = Fold.many (MutArray.fCompactGE n)
3213−--
3214−-- Generates unpinned arrays irrespective of the pinning status of input
3215−-- arrays.
3216−{-# INLINE lCompactGE #-}
3217−lCompactGE :: forall m a. (MonadIO m, Unbox a)
3218− => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3219−lCompactGE = lCompactGeAs Unpinned
3220−
3221−-- | Pinned version of 'lCompactGE'.
3222−{-# INLINE lPinnedCompactGE #-}
3223−lPinnedCompactGE :: forall m a. (MonadIO m, Unbox a)
3224− => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3225−lPinnedCompactGE = lCompactGeAs Pinned
3226−
3227−-- | @compactGE n stream@ coalesces adjacent arrays in the @stream@ until
3228−-- the size becomes greater than or equal to @n@.
3229−--
3230−-- >>> compactGE n = Stream.foldMany (MutArray.fCompactGE n)
3231−--
3232−{-# INLINE compactGE #-}
3233−compactGE ::
3234− (MonadIO m, Unbox a)
3235− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
3236−compactGE n = D.foldMany (fCompactGE n)
3237−
3238−-- | 'compactEQ n' coalesces adajacent arrays in the input stream to
3239−-- arrays of exact size @n@.
3240−--
3241−-- /Unimplemented/
3242−{-# INLINE compactEQ #-}
3243−compactEQ :: -- (MonadIO m, Unbox a) =>
3244− Int -> Stream m (MutArray a) -> Stream m (MutArray a)
3245−compactEQ _n = undefined -- D.parseManyD (pCompactEQ n)
2357+cmp arr1 arr2 =
2358+ liftIO
2359+ $ do
2360+ let marr1 = getMutableByteArray# (arrContents arr1)
2361+ marr2 = getMutableByteArray# (arrContents arr2)
2362+ !(I# st1#) = arrStart arr1
2363+ !(I# st2#) = arrStart arr2
2364+ !(I# len#) = byteLength arr1
2365+ case compare (byteLength arr1) (byteLength arr2) of
2366+ EQ -> do
2367+ r <- liftIO $ IO $ \s# ->
2368+ let res =
2369+ I#
2370+ (compareByteArrays#
2371+ (unsafeCoerce# marr1)
2372+ st1#
2373+ (unsafeCoerce# marr2)
2374+ st2#
2375+ len#)
2376+ in (# s#, res #)
2377+ return $ compare r 0
2378+ x -> return x
32462379
32472380-------------------------------------------------------------------------------
3248−-- In-place mutation algorithms
2381+-- NFData
32492382-------------------------------------------------------------------------------
32502383
32512384-- | Strip elements which match with predicate from both ends.
@@ -3274,7 +2407,7 @@ strip eq arr@MutArray{..} = liftIO $ do
32742407 getStart cur = do
32752408 if cur < arrEnd
32762409 then do
3277− r <- peekAt cur arrContents
2410+ r <- peekByteIndex cur arrContents
32782411 if eq r
32792412 then getStart (INDEX_NEXT(cur,a))
32802413 else return cur
@@ -3284,7 +2417,7 @@ strip eq arr@MutArray{..} = liftIO $ do
32842417 if cur > low
32852418 then do
32862419 let prev = INDEX_PREV(cur,a)
3287− r <- peekAt prev arrContents
2420+ r <- peekByteIndex prev arrContents
32882421 if eq r
32892422 then getLast prev low
32902423 else return cur
core/src/Streamly/Internal/Data/MutByteArray.hsdeleted−251
@@ -1,251 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−-- This is required as all the instances in this module are orphan instances.
4−{-# OPTIONS_GHC -fno-warn-orphans #-}
5−
6−-- |
7−-- Module : Streamly.Internal.Data.MutByteArray
8−-- Copyright : (c) 2023 Composewell Technologies
9−-- License : BSD3-3-Clause
10−-- Maintainer : streamly@composewell.com
11−-- Portability : GHC
12−--
13−
14−module Streamly.Internal.Data.MutByteArray
15− (
16− -- * MutByteArray
17− module Streamly.Internal.Data.MutByteArray.Type
18− -- * Unbox
19− , module Streamly.Internal.Data.Unbox
20− , module Streamly.Internal.Data.Unbox.TH
21− -- * Serialize
22− , module Streamly.Internal.Data.Serialize.Type
23− -- * Serialize TH
24− , module Streamly.Internal.Data.Serialize.TH
25− ) where
26−
27−--------------------------------------------------------------------------------
28−-- Imports
29−--------------------------------------------------------------------------------
30−
31−import Data.Proxy (Proxy(..))
32−import Streamly.Internal.Data.Array (Array(..))
33−import GHC.Exts (Int(..), sizeofByteArray#, unsafeCoerce#)
34−import GHC.Word (Word8)
35−
36−#if __GLASGOW_HASKELL__ >= 900
37−import GHC.Num.Integer (Integer(..))
38−#else
39−import GHC.Integer.GMP.Internals (Integer(..), BigNat(..))
40−#endif
41−
42−import Streamly.Internal.Data.MutByteArray.Type
43−import Streamly.Internal.Data.Serialize.TH
44−import Streamly.Internal.Data.Serialize.Type
45−import Streamly.Internal.Data.Unbox
46−import Streamly.Internal.Data.Unbox.TH
47−
48−--------------------------------------------------------------------------------
49−-- Common instances
50−--------------------------------------------------------------------------------
51−
52−-- Note
53−-- ====
54−--
55−-- Even a non-functional change such as changing the order of constructors will
56−-- change the instance derivation.
57−--
58−-- This will not pose a problem if both, encode, and decode are done by the same
59−-- version of the application. There *might* be a problem if version that
60−-- encodes differs from the version that decodes.
61−--
62−-- We need to add some compatibility tests using different versions of
63−-- dependencies.
64−--
65−-- Although such chages for the most basic types won't happen we need to detect
66−-- if it ever happens.
67−--
68−-- Should we worry about these kind of changes and this kind of compatibility?
69−-- This is a problem for all types of derivations that depend on the order of
70−-- constructors, for example, Enum.
71−
72−-- Note on Windows build
73−-- =====================
74−--
75−-- On Windows, having template haskell splices here fail the build with the
76−-- following error:
77−--
78−-- @
79−-- addLibrarySearchPath: C:\... (Win32 error 3): The system cannot find the path specified.
80−-- @
81−--
82−-- The error might be irrelavant but having these splices triggers it. We should
83−-- either fix the problem or avoid the use to template haskell splices in this
84−-- file.
85−--
86−-- Similar issue: https://github.com/haskell/cabal/issues/4741
87−
88−-- $(Serialize.deriveSerialize ''Maybe)
89−instance Serialize a => Serialize (Maybe a) where
90−
91− {-# INLINE addSizeTo #-}
92− addSizeTo acc x =
93− case x of
94− Nothing -> (acc + 1)
95− Just field0 -> (addSizeTo (acc + 1)) field0
96−
97− {-# INLINE deserializeAt #-}
98− deserializeAt initialOffset arr endOffset = do
99− (i0, tag) <- ((deserializeAt initialOffset) arr) endOffset
100− case tag :: Word8 of
101− 0 -> pure (i0, Nothing)
102− 1 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
103− pure (i1, Just a0)
104− _ -> error "Found invalid tag while peeking (Maybe a)"
105−
106− {-# INLINE serializeAt #-}
107− serializeAt initialOffset arr val =
108− case val of
109− Nothing -> do
110− i0 <- ((serializeAt initialOffset) arr) (0 :: Word8)
111− pure i0
112− Just field0 -> do
113− i0 <- ((serializeAt initialOffset) arr) (1 :: Word8)
114− i1 <- ((serializeAt i0) arr) field0
115− pure i1
116−
117−-- $(Serialize.deriveSerialize ''Either)
118−instance (Serialize a, Serialize b) => Serialize (Either a b) where
119−
120− {-# INLINE addSizeTo #-}
121− addSizeTo acc x =
122− case x of
123− Left field0 -> (addSizeTo (acc + 1)) field0
124− Right field0 -> (addSizeTo (acc + 1)) field0
125−
126− {-# INLINE deserializeAt #-}
127− deserializeAt initialOffset arr endOffset = do
128− (i0, tag) <- ((deserializeAt initialOffset) arr) endOffset
129− case tag :: Word8 of
130− 0 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
131− pure (i1, Left a0)
132− 1 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
133− pure (i1, Right a0)
134− _ -> error "Found invalid tag while peeking (Either a b)"
135−
136− {-# INLINE serializeAt #-}
137− serializeAt initialOffset arr val =
138− case val of
139− Left field0 -> do
140− i0 <- ((serializeAt initialOffset) arr) (0 :: Word8)
141− i1 <- ((serializeAt i0) arr) field0
142− pure i1
143− Right field0 -> do
144− i0 <- ((serializeAt initialOffset) arr) (1 :: Word8)
145− i1 <- ((serializeAt i0) arr) field0
146− pure i1
147−
148−instance Serialize (Proxy a) where
149−
150− {-# INLINE addSizeTo #-}
151− addSizeTo acc _ = (acc + 1)
152−
153− {-# INLINE deserializeAt #-}
154− deserializeAt initialOffset _ _ = pure ((initialOffset + 1), Proxy)
155−
156− {-# INLINE serializeAt #-}
157− serializeAt initialOffset _ _ = pure (initialOffset + 1)
158−
159−--------------------------------------------------------------------------------
160−-- Integer
161−--------------------------------------------------------------------------------
162−
163−data LiftedInteger
164− = LIS Int
165− | LIP (Array Word)
166− | LIN (Array Word)
167−
168−-- $(Serialize.deriveSerialize ''LiftedInteger)
169−instance Serialize LiftedInteger where
170−
171− {-# INLINE addSizeTo #-}
172− addSizeTo acc x =
173− case x of
174− LIS field0 -> (addSizeTo (acc + 1)) field0
175− LIP field0 -> (addSizeTo (acc + 1)) field0
176− LIN field0 -> (addSizeTo (acc + 1)) field0
177−
178− {-# INLINE deserializeAt #-}
179− deserializeAt initialOffset arr endOffset = do
180− (i0, tag) <- ((deserializeAt initialOffset) arr) endOffset
181− case tag :: Word8 of
182− 0 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
183− pure (i1, LIS a0)
184− 1 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
185− pure (i1, LIP a0)
186− 2 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
187− pure (i1, LIN a0)
188− _ -> error "Found invalid tag while peeking (LiftedInteger)"
189−
190− {-# INLINE serializeAt #-}
191− serializeAt initialOffset arr val =
192− case val of
193− LIS field0 -> do
194− i0 <- ((serializeAt initialOffset) arr) (0 :: Word8)
195− i1 <- ((serializeAt i0) arr) field0
196− pure i1
197− LIP field0 -> do
198− i0 <- ((serializeAt initialOffset) arr) (1 :: Word8)
199− i1 <- ((serializeAt i0) arr) field0
200− pure i1
201− LIN field0 -> do
202− i0 <- ((serializeAt initialOffset) arr) (2 :: Word8)
203− i1 <- ((serializeAt i0) arr) field0
204− pure i1
205−
206−#if __GLASGOW_HASKELL__ >= 900
207−
208−{-# INLINE liftInteger #-}
209−liftInteger :: Integer -> LiftedInteger
210−liftInteger (IS x) = LIS (I# x)
211−liftInteger (IP x) =
212− LIP (Array (MutByteArray (unsafeCoerce# x)) 0 (I# (sizeofByteArray# x)))
213−liftInteger (IN x) =
214− LIN (Array (MutByteArray (unsafeCoerce# x)) 0 (I# (sizeofByteArray# x)))
215−
216−{-# INLINE unliftInteger #-}
217−unliftInteger :: LiftedInteger -> Integer
218−unliftInteger (LIS (I# x)) = IS x
219−unliftInteger (LIP (Array (MutByteArray x) _ _)) = IP (unsafeCoerce# x)
220−unliftInteger (LIN (Array (MutByteArray x) _ _)) = IN (unsafeCoerce# x)
221−
222−#else
223−
224−{-# INLINE liftInteger #-}
225−liftInteger :: Integer -> LiftedInteger
226−liftInteger (S# x) = LIS (I# x)
227−liftInteger (Jp# (BN# x)) =
228− LIP (Array (MutByteArray (unsafeCoerce# x)) 0 (I# (sizeofByteArray# x)))
229−liftInteger (Jn# (BN# x)) =
230− LIN (Array (MutByteArray (unsafeCoerce# x)) 0 (I# (sizeofByteArray# x)))
231−
232−{-# INLINE unliftInteger #-}
233−unliftInteger :: LiftedInteger -> Integer
234−unliftInteger (LIS (I# x)) = S# x
235−unliftInteger (LIP (Array (MutByteArray x) _ _)) =
236− Jp# (BN# (unsafeCoerce# x))
237−unliftInteger (LIN (Array (MutByteArray x) _ _)) =
238− Jn# (BN# (unsafeCoerce# x))
239−
240−#endif
241−
242−instance Serialize Integer where
243− {-# INLINE addSizeTo #-}
244− addSizeTo i a = addSizeTo i (liftInteger a)
245−
246− {-# INLINE deserializeAt #-}
247− deserializeAt off arr end =
248− fmap unliftInteger <$> deserializeAt off arr end
249−
250− {-# INLINE serializeAt #-}
251− serializeAt off arr val = serializeAt off arr (liftInteger val)
core/src/Streamly/Internal/Data/MutByteArray/Type.hsdeleted−321
@@ -1,321 +0,0 @@
1−{-# LANGUAGE UnboxedTuples #-}
2−
3−-- |
4−-- Module : Streamly.Internal.Data.MutByteArray.Type
5−-- Copyright : (c) 2023 Composewell Technologies
6−-- License : BSD3-3-Clause
7−-- Maintainer : streamly@composewell.com
8−-- Stability : experimental
9−-- Portability : GHC
10−--
11−module Streamly.Internal.Data.MutByteArray.Type
12− (
13− -- ** MutByteArray
14− MutByteArray(..)
15− , MutableByteArray
16− , getMutableByteArray# -- XXX getMutByteArray#
17−
18− -- ** Pinning
19− , PinnedState(..)
20− , isPinned
21− , pin
22− , unpin
23−
24− -- ** Allocation
25− , empty
26− , newBytesAs -- XXX should be removed
27− , new
28− , pinnedNew
29− , pinnedNewAlignedBytes -- XXX should be removed
30−
31− -- ** Access
32− , sizeOfMutableByteArray -- XXX length
33− , putSliceUnsafe
34− , cloneSliceUnsafeAs
35− , cloneSliceUnsafe
36− , pinnedCloneSliceUnsafe
37− , unsafePinnedAsPtr
38− , unsafeAsPtr
39−
40− -- ** Deprecated
41− , asPtrUnsafe
42− , nil
43− ) where
44−
45−import Control.Monad.IO.Class (MonadIO(..))
46−#ifdef DEBUG
47−import Control.Monad (when)
48−import Debug.Trace (trace)
49−#endif
50−import GHC.Base (IO(..))
51−import System.IO.Unsafe (unsafePerformIO)
52−
53−import GHC.Exts
54−
55−--------------------------------------------------------------------------------
56−-- The ArrayContents type
57−--------------------------------------------------------------------------------
58−
59−data PinnedState
60− = Pinned
61− | Unpinned deriving (Show, Eq)
62−
63−-- XXX can use UnliftedNewtypes
64−
65−-- | A lifted mutable byte array type wrapping @MutableByteArray# RealWorld@.
66−-- This is a low level array used to back high level unboxed arrays and
67−-- serialized data.
68−data MutByteArray = MutByteArray (MutableByteArray# RealWorld)
69−
70−{-# DEPRECATED MutableByteArray "Please use MutByteArray instead" #-}
71−type MutableByteArray = MutByteArray
72−
73−{-# INLINE getMutableByteArray# #-}
74−getMutableByteArray# :: MutByteArray -> MutableByteArray# RealWorld
75−getMutableByteArray# (MutByteArray mbarr) = mbarr
76−
77−-- | Return the size of the array in bytes.
78−{-# INLINE sizeOfMutableByteArray #-}
79−sizeOfMutableByteArray :: MutByteArray -> IO Int
80−sizeOfMutableByteArray (MutByteArray arr) =
81− IO $ \s ->
82− case getSizeofMutableByteArray# arr s of
83− (# s1, i #) -> (# s1, I# i #)
84−
85−{-# INLINE touch #-}
86−touch :: MutByteArray -> IO ()
87−touch (MutByteArray contents) =
88− IO $ \s -> case touch# contents s of s' -> (# s', () #)
89−
90−-- XXX Some functions in this module are "IO" and others are "m", we need to
91−-- make it consistent.
92−
93−-- XXX We can provide another API for "unsafe" FFI calls passing an unlifted
94−-- pointer to the FFI call. For unsafe calls we do not need to pin the array.
95−-- We can pass an unlifted pointer to the FFI routine to avoid GC kicking in
96−-- before the pointer is wrapped.
97−--
98−-- From the GHC manual:
99−--
100−-- GHC, since version 8.4, guarantees that garbage collection will never occur
101−-- during an unsafe call, even in the bytecode interpreter, and further
102−-- guarantees that unsafe calls will be performed in the calling thread. Making
103−-- it safe to pass heap-allocated objects to unsafe functions.
104−
105−-- | Use a @MutByteArray@ as @Ptr a@. This is useful when we want to pass
106−-- an array as a pointer to some operating system call or to a "safe" FFI call.
107−--
108−-- If the array is not pinned it is copied to pinned memory before passing it
109−-- to the monadic action.
110−--
111−-- /Performance Notes:/ Forces a copy if the array is not pinned. It is advised
112−-- that the programmer keeps this in mind and creates a pinned array
113−-- opportunistically before this operation occurs, to avoid the cost of a copy
114−-- if possible.
115−--
116−-- /Unsafe/ because of direct pointer operations. The user must ensure that
117−-- they are writing within the legal bounds of the array.
118−--
119−-- /Pre-release/
120−--
121−{-# INLINE unsafePinnedAsPtr #-}
122−unsafePinnedAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
123−unsafePinnedAsPtr arr f = do
124− contents <- liftIO $ pin arr
125− let !ptr = Ptr (byteArrayContents#
126− (unsafeCoerce# (getMutableByteArray# contents)))
127− r <- f ptr
128− liftIO $ touch contents
129− return r
130−
131−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
132−{-# INLINE asPtrUnsafe #-}
133−asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
134−asPtrUnsafe = unsafePinnedAsPtr
135−
136−-- | For use with unsafe FFI functions. Does not force pin the array memory.
137−{-# INLINE unsafeAsPtr #-}
138−unsafeAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
139−unsafeAsPtr arr f = do
140− let !ptr = Ptr (byteArrayContents#
141− (unsafeCoerce# (getMutableByteArray# arr)))
142− r <- f ptr
143− liftIO $ touch arr
144− return r
145−
146−--------------------------------------------------------------------------------
147−-- Creation
148−--------------------------------------------------------------------------------
149−
150−{-# NOINLINE empty #-}
151−empty :: MutByteArray
152−empty = unsafePerformIO $ new 0
153−
154−{-# DEPRECATED nil "Please use empty instead" #-}
155−nil :: MutByteArray
156−nil = empty
157−
158−-- XXX add "newRounded" to round up the large size to the next page boundary
159−-- and return the allocated size.
160−{-# INLINE new #-}
161−new :: Int -> IO MutByteArray
162−new nbytes | nbytes < 0 =
163− errorWithoutStackTrace "newByteArray: size must be >= 0"
164−new (I# nbytes) = IO $ \s ->
165− case newByteArray# nbytes s of
166− (# s', mbarr# #) ->
167− let c = MutByteArray mbarr#
168− in (# s', c #)
169−
170−{-# INLINE pinnedNew #-}
171−pinnedNew :: Int -> IO MutByteArray
172−pinnedNew nbytes | nbytes < 0 =
173− errorWithoutStackTrace "pinnedNew: size must be >= 0"
174−pinnedNew (I# nbytes) = IO $ \s ->
175− case newPinnedByteArray# nbytes s of
176− (# s', mbarr# #) ->
177− let c = MutByteArray mbarr#
178− in (# s', c #)
179−
180−{-# INLINE pinnedNewAlignedBytes #-}
181−pinnedNewAlignedBytes :: Int -> Int -> IO MutByteArray
182−pinnedNewAlignedBytes nbytes _align | nbytes < 0 =
183− errorWithoutStackTrace "pinnedNewAlignedBytes: size must be >= 0"
184−pinnedNewAlignedBytes (I# nbytes) (I# align) = IO $ \s ->
185− case newAlignedPinnedByteArray# nbytes align s of
186− (# s', mbarr# #) ->
187− let c = MutByteArray mbarr#
188− in (# s', c #)
189−
190−{-# INLINE newBytesAs #-}
191−newBytesAs :: PinnedState -> Int -> IO MutByteArray
192−newBytesAs Unpinned = new
193−newBytesAs Pinned = pinnedNew
194−
195−-------------------------------------------------------------------------------
196−-- Copying
197−-------------------------------------------------------------------------------
198−
199−-- Note: Array copy is more efficient than streaming copy.
200−-- CopyMutableByteArray# translates to genMemcpy in GHC/CmmToAsm/X86/CodeGen.hs
201−-- glibc memcpy copies bytes/words/pages - unrolls the loops:
202−-- https://github.com/bminor/glibc/blob/4290aed05135ae4c0272006442d147f2155e70d7/string/memcpy.c
203−-- https://github.com/bminor/glibc/blob/4290aed05135ae4c0272006442d147f2155e70d7/string/wordcopy.c
204−
205−-- | Put a sub range of a source array into a subrange of a destination array.
206−-- This is not safe as it does not check the bounds of neither the src array
207−-- nor the destination array.
208−{-# INLINE putSliceUnsafe #-}
209−putSliceUnsafe ::
210− MonadIO m
211− => MutByteArray
212− -> Int
213− -> MutByteArray
214− -> Int
215− -> Int
216− -> m ()
217−putSliceUnsafe src srcStartBytes dst dstStartBytes lenBytes = liftIO $ do
218−#ifdef DEBUG
219− srcLen <- sizeOfMutableByteArray src
220− dstLen <- sizeOfMutableByteArray dst
221− when (srcLen - srcStartBytes < lenBytes)
222− $ error $ "putSliceUnsafe: src overflow: start" ++ show srcStartBytes
223− ++ " end " ++ show srcLen ++ " len " ++ show lenBytes
224− when (dstLen - dstStartBytes < lenBytes)
225− $ error $ "putSliceUnsafe: dst overflow: start" ++ show dstStartBytes
226− ++ " end " ++ show dstLen ++ " len " ++ show lenBytes
227−#endif
228− let !(I# srcStartBytes#) = srcStartBytes
229− !(I# dstStartBytes#) = dstStartBytes
230− !(I# lenBytes#) = lenBytes
231− let arrS# = getMutableByteArray# src
232− arrD# = getMutableByteArray# dst
233− IO $ \s# -> (# copyMutableByteArray#
234− arrS# srcStartBytes# arrD# dstStartBytes# lenBytes# s#
235− , () #)
236−
237−-- | Unsafe as it does not check whether the start offset and length supplied
238−-- are valid inside the array.
239−{-# INLINE cloneSliceUnsafeAs #-}
240−cloneSliceUnsafeAs :: MonadIO m =>
241− PinnedState -> Int -> Int -> MutByteArray -> m MutByteArray
242−cloneSliceUnsafeAs ps srcOff srcLen src =
243− liftIO $ do
244− mba <- newBytesAs ps srcLen
245− putSliceUnsafe src srcOff mba 0 srcLen
246− return mba
247−
248−-- | @cloneSliceUnsafe offset len arr@ clones a slice of the supplied array
249−-- starting at the given offset and equal to the given length.
250−{-# INLINE cloneSliceUnsafe #-}
251−cloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
252−cloneSliceUnsafe = cloneSliceUnsafeAs Unpinned
253−
254−-- | @pinnedCloneSliceUnsafe offset len arr@
255−{-# INLINE pinnedCloneSliceUnsafe #-}
256−pinnedCloneSliceUnsafe :: MonadIO m =>
257− Int -> Int -> MutByteArray -> m MutByteArray
258−pinnedCloneSliceUnsafe = cloneSliceUnsafeAs Pinned
259−
260−-------------------------------------------------------------------------------
261−-- Pinning & Unpinning
262−-------------------------------------------------------------------------------
263−
264−-- | Return 'True' if the array is allocated in pinned memory.
265−{-# INLINE isPinned #-}
266−isPinned :: MutByteArray -> Bool
267−isPinned (MutByteArray arr#) =
268− let pinnedInt = I# (isMutableByteArrayPinned# arr#)
269− in pinnedInt /= 0
270−
271−
272−{-# INLINE cloneMutableArrayWith# #-}
273−cloneMutableArrayWith#
274− :: (Int# -> State# RealWorld -> (# State# RealWorld
275− , MutableByteArray# RealWorld #))
276− -> MutableByteArray# RealWorld
277− -> State# RealWorld
278− -> (# State# RealWorld, MutableByteArray# RealWorld #)
279−cloneMutableArrayWith# alloc# arr# s# =
280− case getSizeofMutableByteArray# arr# s# of
281− (# s1#, i# #) ->
282− case alloc# i# s1# of
283− (# s2#, arr1# #) ->
284− case copyMutableByteArray# arr# 0# arr1# 0# i# s2# of
285− s3# -> (# s3#, arr1# #)
286−
287−-- | Return a copy of the array in pinned memory if unpinned, else return the
288−-- original array.
289−{-# INLINE pin #-}
290−pin :: MutByteArray -> IO MutByteArray
291−pin arr@(MutByteArray marr#) =
292− if isPinned arr
293− then return arr
294− else
295−#ifdef DEBUG
296− do
297− -- XXX dump stack trace
298− trace ("pin: Copying array") (return ())
299−#endif
300− IO
301− $ \s# ->
302− case cloneMutableArrayWith# newPinnedByteArray# marr# s# of
303− (# s1#, marr1# #) -> (# s1#, MutByteArray marr1# #)
304−
305−-- | Return a copy of the array in unpinned memory if pinned, else return the
306−-- original array.
307−{-# INLINE unpin #-}
308−unpin :: MutByteArray -> IO MutByteArray
309−unpin arr@(MutByteArray marr#) =
310− if not (isPinned arr)
311− then return arr
312− else
313−#ifdef DEBUG
314− do
315− -- XXX dump stack trace
316− trace ("unpin: Copying array") (return ())
317−#endif
318− IO
319− $ \s# ->
320− case cloneMutableArrayWith# newByteArray# marr# s# of
321− (# s1#, marr1# #) -> (# s1#, MutByteArray marr1# #)
core/src/Streamly/Internal/Data/Parser.hs+143−135
@@ -10,16 +10,10 @@
1010module Streamly.Internal.Data.Parser
1111 (
1212 -- * Setup
13− -- | To execute the code examples provided in this module in ghci, please
14− -- run the following commands first.
15− --
1613 -- $setup
1714
18− module Streamly.Internal.Data.Parser.Type
19− --, module Streamly.Internal.Data.Parser.Tee
20−
2115 -- * Types
22− , Parser (..)
16+ Parser (..)
2317 , ParseError (..)
2418 , Step (..)
2519 , Initial (..)
@@ -31,9 +25,19 @@ module Streamly.Internal.Data.Parser
3125 -- * Accumulators
3226 , fromFold
3327 , fromFoldMaybe
28+ , fromPure
29+ , fromEffect
30+ , die
31+ , dieM
3432
3533 -- * Map on input
34+ , lmap
35+ , lmapM
3636 , postscan
37+ , filter
38+
39+ -- * Map on output
40+ , rmapM
3741
3842 -- * Element parsers
3943 , peek
@@ -135,6 +139,11 @@ module Streamly.Internal.Data.Parser
135139
136140 -- Second order parsers (parsers using parsers)
137141 -- * Binary Combinators
142+
143+ -- ** Sequential Applicative
144+ , splitWith
145+ , split_
146+
138147 {-
139148 -- ** Parallel Applicatives
140149 , teeWith
@@ -143,6 +152,9 @@ module Streamly.Internal.Data.Parser
143152 -- , teeTill -- like manyTill but parallel
144153 -}
145154
155+ -- ** Sequential Alternative
156+ , alt
157+
146158 {-
147159 -- ** Parallel Alternatives
148160 , shortest
@@ -153,6 +165,7 @@ module Streamly.Internal.Data.Parser
153165 -- * N-ary Combinators
154166 -- ** Sequential Collection
155167 , sequence
168+ , concatMap
156169
157170 -- ** Sequential Repetition
158171 , count
@@ -253,19 +266,18 @@ import Streamly.Internal.Data.SVar.Type (defState)
253266import Streamly.Internal.Data.Either.Strict (Either'(..))
254267import Streamly.Internal.Data.Maybe.Strict (Maybe'(..))
255268import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
256−import Streamly.Internal.Data.Stream.Type (Stream)
269+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
257270
258271import qualified Data.Foldable as Foldable
259272import qualified Streamly.Internal.Data.Fold.Type as FL
260−import qualified Streamly.Internal.Data.Stream.Type as D
261−import qualified Streamly.Internal.Data.Stream.Generate as D
262−
263−import Streamly.Internal.Data.Parser.Type
264−--import Streamly.Internal.Data.Parser.Tee -- It's empty
273+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
274+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
265275
266276import Prelude hiding
267277 (any, all, take, takeWhile, sequence, concatMap, maybe, either, span
268278 , zip, filter, dropWhile)
279+-- import Streamly.Internal.Data.Parser.ParserD.Tee
280+import Streamly.Internal.Data.Parser.ParserD.Type
269281
270282#include "DocTestDataParser.hs"
271283
@@ -284,7 +296,7 @@ import Prelude hiding
284296--
285297{-# INLINE toFold #-}
286298toFold :: Monad m => Parser a m b -> Fold m a b
287−toFold (Parser pstep pinitial pextract) = Fold step initial extract final
299+toFold (Parser pstep pinitial pextract) = Fold step initial extract
288300
289301 where
290302
@@ -312,9 +324,7 @@ toFold (Parser pstep pinitial pextract) = Fold step initial extract final
312324 Done n _ -> derror n
313325 Error err -> eerror err
314326
315− extract = error "toFold: parser cannot be used for scanning"
316−
317− final st = do
327+ extract st = do
318328 r <- pextract st
319329 case r of
320330 Done 0 b -> return b
@@ -332,7 +342,7 @@ toFold (Parser pstep pinitial pextract) = Fold step initial extract final
332342--
333343{-# INLINE fromFold #-}
334344fromFold :: Monad m => Fold m a b -> Parser a m b
335−fromFold (Fold fstep finitial _ ffinal) = Parser step initial extract
345+fromFold (Fold fstep finitial fextract) = Parser step initial extract
336346
337347 where
338348
@@ -350,7 +360,7 @@ fromFold (Fold fstep finitial _ ffinal) = Parser step initial extract
350360 FL.Partial s1 -> Partial 0 s1
351361 FL.Done b -> Done 0 b
352362
353− extract = fmap (Done 0) . ffinal
363+ extract = fmap (Done 0) . fextract
354364
355365-- | Convert a Maybe returning fold to an error returning parser. The first
356366-- argument is the error message that the parser would return when the fold
@@ -360,7 +370,7 @@ fromFold (Fold fstep finitial _ ffinal) = Parser step initial extract
360370--
361371{-# INLINE fromFoldMaybe #-}
362372fromFoldMaybe :: Monad m => String -> Fold m a (Maybe b) -> Parser a m b
363−fromFoldMaybe errMsg (Fold fstep finitial _ ffinal) =
373+fromFoldMaybe errMsg (Fold fstep finitial fextract) =
364374 Parser step initial extract
365375
366376 where
@@ -386,7 +396,7 @@ fromFoldMaybe errMsg (Fold fstep finitial _ ffinal) =
386396 Nothing -> Error errMsg
387397
388398 extract s = do
389− res <- ffinal s
399+ res <- fextract s
390400 case res of
391401 Just x -> return $ Done 0 x
392402 Nothing -> return $ Error errMsg
@@ -635,7 +645,7 @@ data Tuple'Fused a b = Tuple'Fused !a !b deriving Show
635645--
636646{-# INLINE takeBetween #-}
637647takeBetween :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
638−takeBetween low high (Fold fstep finitial _ ffinal) =
648+takeBetween low high (Fold fstep finitial fextract) =
639649
640650 Parser step initial (extract streamErr)
641651
@@ -686,11 +696,9 @@ takeBetween low high (Fold fstep finitial _ ffinal) =
686696 in case res of
687697 FL.Partial s -> do
688698 let s1 = Tuple'Fused i1 s
689− if i1 < low
699+ if i1 < high
690700 then return $ Continue 0 s1
691− else if i1 < high
692− then return $ Partial 0 s1
693− else fmap (Done 0) (ffinal s)
701+ else extract foldErr s1
694702 FL.Done b ->
695703 return
696704 $ if i1 >= low
@@ -700,12 +708,12 @@ takeBetween low high (Fold fstep finitial _ ffinal) =
700708 step (Tuple'Fused i s) a = fstep s a >>= snext i
701709
702710 extract f (Tuple'Fused i s)
703− | i >= low && i <= high = fmap (Done 0) (ffinal s)
711+ | i >= low && i <= high = fmap (Done 0) (fextract s)
704712 | otherwise = return $ Error (f i)
705713
706714 -- XXX Need to make Initial return type Step to deduplicate this
707715 iextract f (Tuple'Fused i s)
708− | i >= low && i <= high = fmap IDone (ffinal s)
716+ | i >= low && i <= high = fmap IDone (fextract s)
709717 | otherwise = return $ IError (f i)
710718
711719-- | Stops after taking exactly @n@ input elements.
@@ -722,7 +730,7 @@ takeBetween low high (Fold fstep finitial _ ffinal) =
722730--
723731{-# INLINE takeEQ #-}
724732takeEQ :: Monad m => Int -> Fold m a b -> Parser a m b
725−takeEQ n (Fold fstep finitial _ ffinal) = Parser step initial extract
733+takeEQ n (Fold fstep finitial fextract) = Parser step initial extract
726734
727735 where
728736
@@ -732,7 +740,7 @@ takeEQ n (Fold fstep finitial _ ffinal) = Parser step initial extract
732740 FL.Partial s ->
733741 if n > 0
734742 then return $ IPartial $ Tuple'Fused 1 s
735− else fmap IDone (ffinal s)
743+ else fmap IDone (fextract s)
736744 FL.Done b -> return $
737745 if n > 0
738746 then IError
@@ -756,7 +764,7 @@ takeEQ n (Fold fstep finitial _ ffinal) = Parser step initial extract
756764 -- assert (n == i1)
757765 Done 0
758766 <$> case res of
759− FL.Partial s -> ffinal s
767+ FL.Partial s -> fextract s
760768 FL.Done b -> return b
761769
762770 extract (Tuple'Fused i _) =
@@ -788,7 +796,7 @@ data TakeGEState s =
788796--
789797{-# INLINE takeGE #-}
790798takeGE :: Monad m => Int -> Fold m a b -> Parser a m b
791−takeGE n (Fold fstep finitial _ ffinal) = Parser step initial extract
799+takeGE n (Fold fstep finitial fextract) = Parser step initial extract
792800
793801 where
794802
@@ -836,7 +844,7 @@ takeGE n (Fold fstep finitial _ ffinal) = Parser step initial extract
836844 $ Error
837845 $ "takeGE: Expecting at least " ++ show n
838846 ++ " elements, input terminated on " ++ show (i - 1)
839− extract (TakeGEGE r) = fmap (Done 0) $ ffinal r
847+ extract (TakeGEGE r) = fmap (Done 0) $ fextract r
840848
841849-------------------------------------------------------------------------------
842850-- Conditional splitting
@@ -897,7 +905,7 @@ takeWhileP predicate (Parser pstep pinitial pextract) =
897905{-# INLINE takeWhile #-}
898906takeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
899907-- takeWhile cond f = takeWhileP cond (fromFold f)
900−takeWhile predicate (Fold fstep finitial _ ffinal) =
908+takeWhile predicate (Fold fstep finitial fextract) =
901909 Parser step initial extract
902910
903911 where
@@ -916,9 +924,9 @@ takeWhile predicate (Fold fstep finitial _ ffinal) =
916924 $ case fres of
917925 FL.Partial s1 -> Partial 0 s1
918926 FL.Done b -> Done 0 b
919− else Done 1 <$> ffinal s
927+ else Done 1 <$> fextract s
920928
921− extract s = fmap (Done 0) (ffinal s)
929+ extract s = fmap (Done 0) (fextract s)
922930
923931{-
924932-- XXX This may not be composable because of the b argument. We can instead
@@ -938,7 +946,7 @@ takeWhile1 acc cond f = undefined
938946{-# INLINE takeWhile1 #-}
939947takeWhile1 :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
940948-- takeWhile1 cond f = takeWhileP cond (takeBetween 1 maxBound f)
941−takeWhile1 predicate (Fold fstep finitial _ ffinal) =
949+takeWhile1 predicate (Fold fstep finitial fextract) =
942950 Parser step initial extract
943951
944952 where
@@ -968,11 +976,11 @@ takeWhile1 predicate (Fold fstep finitial _ ffinal) =
968976 if predicate a
969977 then process s a
970978 else do
971− b <- ffinal s
979+ b <- fextract s
972980 return $ Done 1 b
973981
974982 extract (Left' _) = return $ Error "takeWhile1: end of input"
975− extract (Right' s) = fmap (Done 0) (ffinal s)
983+ extract (Right' s) = fmap (Done 0) (fextract s)
976984
977985-- | Drain the input as long as the predicate succeeds, running the effects and
978986-- discarding the results.
@@ -1001,7 +1009,7 @@ takeFramedByGeneric :: Monad m =>
10011009 -> Maybe (a -> Bool) -- is frame end?
10021010 -> Fold m a b
10031011 -> Parser a m b
1004−takeFramedByGeneric esc begin end (Fold fstep finitial _ ffinal) =
1012+takeFramedByGeneric esc begin end (Fold fstep finitial fextract) =
10051013
10061014 Parser step initial extract
10071015
@@ -1033,20 +1041,20 @@ takeFramedByGeneric esc begin end (Fold fstep finitial _ ffinal) =
10331041 if isEnd a
10341042 then
10351043 if n == 0
1036− then Done 0 <$> ffinal s
1044+ then Done 0 <$> fextract s
10371045 else process s a (n - 1)
10381046 else
10391047 let n1 = if isBegin a then n + 1 else n
10401048 in process s a n1
10411049 Nothing -> -- takeEndBy case
10421050 if isEnd a
1043− then Done 0 <$> ffinal s
1051+ then Done 0 <$> fextract s
10441052 else process s a n
10451053 Nothing -> -- takeStartBy case
10461054 case begin of
10471055 Just isBegin ->
10481056 if isBegin a
1049− then Done 0 <$> ffinal s
1057+ then Done 0 <$> fextract s
10501058 else process s a n
10511059 Nothing ->
10521060 error $ "takeFramedByGeneric: "
@@ -1071,7 +1079,7 @@ takeFramedByGeneric esc begin end (Fold fstep finitial _ ffinal) =
10711079 case end of
10721080 Just isEnd ->
10731081 if isEnd a
1074− then Done 0 <$> ffinal s
1082+ then Done 0 <$> fextract s
10751083 else processCheckEsc s a 0
10761084 Nothing ->
10771085 error "Both begin and end frame predicate missing"
@@ -1086,7 +1094,7 @@ takeFramedByGeneric esc begin end (Fold fstep finitial _ ffinal) =
10861094 case begin of
10871095 Just _ ->
10881096 case end of
1089− Nothing -> fmap (Done 0) $ ffinal s
1097+ Nothing -> fmap (Done 0) $ fextract s
10901098 Just _ -> err "takeFramedByGeneric: missing frame end"
10911099 Nothing -> err "takeFramedByGeneric: missing closing frame"
10921100 extract (FrameEscEsc _ _) = err "takeFramedByGeneric: trailing escape"
@@ -1125,7 +1133,7 @@ blockWithQuotes :: (Monad m, Eq a) =>
11251133 -> Fold m a b
11261134 -> Parser a m b
11271135blockWithQuotes isEsc isQuote bopen bclose
1128− (Fold fstep finitial _ ffinal) =
1136+ (Fold fstep finitial fextract) =
11291137 Parser step initial extract
11301138
11311139 where
@@ -1155,7 +1163,7 @@ blockWithQuotes isEsc isQuote bopen bclose
11551163 | a == bopen = process s a (BlockUnquoted (level + 1))
11561164 | a == bclose =
11571165 if level == 1
1158− then fmap (Done 0) (ffinal s)
1166+ then fmap (Done 0) (fextract s)
11591167 else process s a (BlockUnquoted (level - 1))
11601168 | isQuote a = process s a (BlockQuoted level)
11611169 | otherwise = process s a (BlockUnquoted level)
@@ -1169,7 +1177,7 @@ blockWithQuotes isEsc isQuote bopen bclose
11691177
11701178 err = return . Error
11711179
1172− extract (BlockInit s) = fmap (Done 0) $ ffinal s
1180+ extract (BlockInit s) = fmap (Done 0) $ fextract s
11731181 extract (BlockUnquoted level _) =
11741182 err $ "blockWithQuotes: finished at block nest level " ++ show level
11751183 extract (BlockQuoted level _) =
@@ -1304,7 +1312,7 @@ takeEitherSepBy _cond = undefined -- D.toParserK . D.takeEitherSepBy cond
13041312--
13051313{-# INLINE takeStartBy #-}
13061314takeStartBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
1307−takeStartBy cond (Fold fstep finitial _ ffinal) =
1315+takeStartBy cond (Fold fstep finitial fextract) =
13081316
13091317 Parser step initial extract
13101318
@@ -1332,10 +1340,10 @@ takeStartBy cond (Fold fstep finitial _ ffinal) =
13321340 step (Right' s) a =
13331341 if not (cond a)
13341342 then process s a
1335− else Done 1 <$> ffinal s
1343+ else Done 1 <$> fextract s
13361344
1337− extract (Left' s) = fmap (Done 0) $ ffinal s
1338− extract (Right' s) = fmap (Done 0) $ ffinal s
1345+ extract (Left' s) = fmap (Done 0) $ fextract s
1346+ extract (Right' s) = fmap (Done 0) $ fextract s
13391347
13401348-- | Like 'takeStartBy' but drops the separator.
13411349--
@@ -1371,7 +1379,7 @@ takeFramedByEsc_ :: Monad m =>
13711379 (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
13721380-- takeFramedByEsc_ isEsc isEnd p =
13731381-- takeFramedByGeneric (Just isEsc) Nothing (Just isEnd) (toFold p)
1374−takeFramedByEsc_ isEsc isBegin isEnd (Fold fstep finitial _ ffinal ) =
1382+takeFramedByEsc_ isEsc isBegin isEnd (Fold fstep finitial fextract) =
13751383
13761384 Parser step initial extract
13771385
@@ -1407,7 +1415,7 @@ takeFramedByEsc_ isEsc isBegin isEnd (Fold fstep finitial _ ffinal ) =
14071415 in process s a n1
14081416 else
14091417 if n == 0
1410− then Done 0 <$> ffinal s
1418+ then Done 0 <$> fextract s
14111419 else process s a (n - 1)
14121420 step (FrameEscEsc s n) a = process s a n
14131421
@@ -1429,7 +1437,7 @@ takeFramedBy_ :: Monad m =>
14291437 (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
14301438-- takeFramedBy_ isBegin isEnd =
14311439-- takeFramedByGeneric (Just (const False)) (Just isBegin) (Just isEnd)
1432−takeFramedBy_ isBegin isEnd (Fold fstep finitial _ ffinal) =
1440+takeFramedBy_ isBegin isEnd (Fold fstep finitial fextract) =
14331441
14341442 Parser step initial extract
14351443
@@ -1459,7 +1467,7 @@ takeFramedBy_ isBegin isEnd (Fold fstep finitial _ ffinal) =
14591467 | not (isEnd a) =
14601468 let n1 = if isBegin a then n + 1 else n
14611469 in process s a n1
1462− | n == 0 = Done 0 <$> ffinal s
1470+ | n == 0 = Done 0 <$> fextract s
14631471 | otherwise = process s a (n - 1)
14641472
14651473 err = return . Error
@@ -1494,7 +1502,7 @@ data WordByState s b = WBLeft !s | WBWord !s | WBRight !b
14941502--
14951503{-# INLINE wordBy #-}
14961504wordBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
1497−wordBy predicate (Fold fstep finitial _ ffinal) = Parser step initial extract
1505+wordBy predicate (Fold fstep finitial fextract) = Parser step initial extract
14981506
14991507 where
15001508
@@ -1521,7 +1529,7 @@ wordBy predicate (Fold fstep finitial _ ffinal) = Parser step initial extract
15211529 if not (predicate a)
15221530 then worder s a
15231531 else do
1524− b <- ffinal s
1532+ b <- fextract s
15251533 return $ Partial 0 $ WBRight b
15261534 step (WBRight b) a =
15271535 return
@@ -1529,8 +1537,8 @@ wordBy predicate (Fold fstep finitial _ ffinal) = Parser step initial extract
15291537 then Done 1 b
15301538 else Partial 0 $ WBRight b
15311539
1532− extract (WBLeft s) = fmap (Done 0) $ ffinal s
1533− extract (WBWord s) = fmap (Done 0) $ ffinal s
1540+ extract (WBLeft s) = fmap (Done 0) $ fextract s
1541+ extract (WBWord s) = fmap (Done 0) $ fextract s
15341542 extract (WBRight b) = return (Done 0 b)
15351543
15361544data WordFramedState s b =
@@ -1567,7 +1575,7 @@ wordFramedBy :: Monad m =>
15671575 -> Fold m a b
15681576 -> Parser a m b
15691577wordFramedBy isEsc isBegin isEnd isSep
1570− (Fold fstep finitial _ ffinal) =
1578+ (Fold fstep finitial fextract) =
15711579 Parser step initial extract
15721580
15731581 where
@@ -1598,7 +1606,7 @@ wordFramedBy isEsc isBegin isEnd isSep
15981606 step (WordFramedWord s n) a
15991607 | isEsc a = return $ Continue 0 $ WordFramedEsc s n
16001608 | n == 0 && isSep a = do
1601− b <- ffinal s
1609+ b <- fextract s
16021610 return $ Partial 0 $ WordFramedSkipPost b
16031611 | otherwise = do
16041612 -- We need to use different order for checking begin and end for
@@ -1631,10 +1639,10 @@ wordFramedBy isEsc isBegin isEnd isSep
16311639
16321640 err = return . Error
16331641
1634− extract (WordFramedSkipPre s) = fmap (Done 0) $ ffinal s
1642+ extract (WordFramedSkipPre s) = fmap (Done 0) $ fextract s
16351643 extract (WordFramedWord s n) =
16361644 if n == 0
1637− then fmap (Done 0) $ ffinal s
1645+ then fmap (Done 0) $ fextract s
16381646 else err "wordFramedBy: missing frame end"
16391647 extract (WordFramedEsc _ _) =
16401648 err "wordFramedBy: trailing escape"
@@ -1721,7 +1729,7 @@ wordWithQuotes :: (Monad m, Eq a) =>
17211729 -> Fold m a b
17221730 -> Parser a m b
17231731wordWithQuotes keepQuotes tr escChar toRight isSep
1724− (Fold fstep finitial _ ffinal) =
1732+ (Fold fstep finitial fextract) =
17251733 Parser step initial extract
17261734
17271735 where
@@ -1772,7 +1780,7 @@ wordWithQuotes keepQuotes tr escChar toRight isSep
17721780 step (WordUnquotedWord s) a
17731781 | isEsc a = return $ Continue 0 $ WordUnquotedEsc s
17741782 | isSep a = do
1775− b <- ffinal s
1783+ b <- fextract s
17761784 return $ Partial 0 $ WordQuotedSkipPost b
17771785 | otherwise = do
17781786 case toRight a of
@@ -1820,11 +1828,11 @@ wordWithQuotes keepQuotes tr escChar toRight isSep
18201828
18211829 err = return . Error
18221830
1823− extract (WordQuotedSkipPre s) = fmap (Done 0) $ ffinal s
1824− extract (WordUnquotedWord s) = fmap (Done 0) $ ffinal s
1831+ extract (WordQuotedSkipPre s) = fmap (Done 0) $ fextract s
1832+ extract (WordUnquotedWord s) = fmap (Done 0) $ fextract s
18251833 extract (WordQuotedWord s n _ _) =
18261834 if n == 0
1827− then fmap (Done 0) $ ffinal s
1835+ then fmap (Done 0) $ fextract s
18281836 else err "wordWithQuotes: missing frame end"
18291837 extract WordQuotedEsc {} =
18301838 err "wordWithQuotes: trailing escape"
@@ -1902,7 +1910,7 @@ data GroupByState a s
19021910--
19031911{-# INLINE groupBy #-}
19041912groupBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
1905−groupBy eq (Fold fstep finitial _ ffinal) = Parser step initial extract
1913+groupBy eq (Fold fstep finitial fextract) = Parser step initial extract
19061914
19071915 where
19081916
@@ -1925,10 +1933,10 @@ groupBy eq (Fold fstep finitial _ ffinal) = Parser step initial extract
19251933 step (GroupByGrouping a0 s) a =
19261934 if eq a0 a
19271935 then grouper s a0 a
1928− else Done 1 <$> ffinal s
1936+ else Done 1 <$> fextract s
19291937
1930− extract (GroupByInit s) = fmap (Done 0) $ ffinal s
1931− extract (GroupByGrouping _ s) = fmap (Done 0) $ ffinal s
1938+ extract (GroupByInit s) = fmap (Done 0) $ fextract s
1939+ extract (GroupByGrouping _ s) = fmap (Done 0) $ fextract s
19321940
19331941-- | Unlike 'groupBy' this combinator performs a rolling comparison of two
19341942-- successive elements in the input stream. Assuming the input stream
@@ -1962,7 +1970,7 @@ groupBy eq (Fold fstep finitial _ ffinal) = Parser step initial extract
19621970--
19631971{-# INLINE groupByRolling #-}
19641972groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
1965−groupByRolling eq (Fold fstep finitial _ ffinal) = Parser step initial extract
1973+groupByRolling eq (Fold fstep finitial fextract) = Parser step initial extract
19661974
19671975 where
19681976
@@ -1985,10 +1993,10 @@ groupByRolling eq (Fold fstep finitial _ ffinal) = Parser step initial extract
19851993 step (GroupByGrouping a0 s) a =
19861994 if eq a0 a
19871995 then grouper s a
1988− else Done 1 <$> ffinal s
1996+ else Done 1 <$> fextract s
19891997
1990− extract (GroupByInit s) = fmap (Done 0) $ ffinal s
1991− extract (GroupByGrouping _ s) = fmap (Done 0) $ ffinal s
1998+ extract (GroupByInit s) = fmap (Done 0) $ fextract s
1999+ extract (GroupByGrouping _ s) = fmap (Done 0) $ fextract s
19922000
19932001{-# ANN type GroupByStatePair Fuse #-}
19942002data GroupByStatePair a s1 s2
@@ -2012,8 +2020,8 @@ groupByRollingEither :: Monad m =>
20122020 (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
20132021groupByRollingEither
20142022 eq
2015− (Fold fstep1 finitial1 _ ffinal1)
2016− (Fold fstep2 finitial2 _ ffinal2) = Parser step initial extract
2023+ (Fold fstep1 finitial1 fextract1)
2024+ (Fold fstep2 finitial2 fextract2) = Parser step initial extract
20172025
20182026 where
20192027
@@ -2072,21 +2080,21 @@ groupByRollingEither
20722080 step (GroupByGroupingPairL a0 s1 s2) a =
20732081 if not (eq a0 a)
20742082 then grouperL2 s1 s2 a
2075− else Done 1 . Left <$> ffinal1 s1
2083+ else Done 1 . Left <$> fextract1 s1
20762084
20772085 step (GroupByGroupingPairR a0 s1 s2) a =
20782086 if eq a0 a
20792087 then grouperR2 s1 s2 a
2080− else Done 1 . Right <$> ffinal2 s2
2088+ else Done 1 . Right <$> fextract2 s2
20812089
2082− extract (GroupByInitPair s1 _) = Done 0 . Left <$> ffinal1 s1
2083− extract (GroupByGroupingPairL _ s1 _) = Done 0 . Left <$> ffinal1 s1
2084− extract (GroupByGroupingPairR _ _ s2) = Done 0 . Right <$> ffinal2 s2
2090+ extract (GroupByInitPair s1 _) = Done 0 . Left <$> fextract1 s1
2091+ extract (GroupByGroupingPairL _ s1 _) = Done 0 . Left <$> fextract1 s1
2092+ extract (GroupByGroupingPairR _ _ s2) = Done 0 . Right <$> fextract2 s2
20852093 extract (GroupByGroupingPair a s1 _) = do
20862094 res <- fstep1 s1 a
20872095 case res of
20882096 FL.Done b -> return $ Done 0 (Left b)
2089− FL.Partial s11 -> Done 0 . Left <$> ffinal1 s11
2097+ FL.Partial s11 -> Done 0 . Left <$> fextract1 s11
20902098
20912099-- XXX use an Unfold instead of a list?
20922100-- XXX custom combinators for matching list, array and stream?
@@ -2239,7 +2247,7 @@ postscan = undefined
22392247{-# INLINE zipWithM #-}
22402248zipWithM :: Monad m =>
22412249 (a -> b -> m c) -> D.Stream m a -> Fold m c x -> Parser b m x
2242−zipWithM zf (D.Stream sstep state) (Fold fstep finitial _ ffinal) =
2250+zipWithM zf (D.Stream sstep state) (Fold fstep finitial fextract) =
22432251 Parser step initial extract
22442252
22452253 where
@@ -2252,7 +2260,7 @@ zipWithM zf (D.Stream sstep state) (Fold fstep finitial _ ffinal) =
22522260 case r of
22532261 D.Yield x s -> return $ IPartial (Just' x, s, fs)
22542262 D.Stop -> do
2255− x <- ffinal fs
2263+ x <- fextract fs
22562264 return $ IDone x
22572265 -- Need Skip/Continue in initial to loop right here
22582266 D.Skip s -> return $ IPartial (Nothing', s, fs)
@@ -2267,7 +2275,7 @@ zipWithM zf (D.Stream sstep state) (Fold fstep finitial _ ffinal) =
22672275 case r of
22682276 D.Yield x1 s -> return $ Continue 0 (Just' x1, s, fs1)
22692277 D.Stop -> do
2270− x <- ffinal fs1
2278+ x <- fextract fs1
22712279 return $ Done 0 x
22722280 D.Skip s -> return $ Continue 1 (Nothing', s, fs1)
22732281 FL.Done x -> return $ Done 0 x
@@ -2282,7 +2290,7 @@ zipWithM zf (D.Stream sstep state) (Fold fstep finitial _ ffinal) =
22822290 return $ Continue 0 (Nothing', s, fs1)
22832291 FL.Done x -> return $ Done 0 x
22842292 D.Stop -> do
2285− x <- ffinal fs
2293+ x <- fextract fs
22862294 return $ Done 1 x
22872295 D.Skip s -> return $ Continue 1 (Nothing', s, fs)
22882296
@@ -2566,7 +2574,7 @@ deintercalateAll :: Monad m =>
25662574deintercalateAll
25672575 (Parser stepL initialL extractL)
25682576 (Parser stepR initialR _)
2569− (Fold fstep finitial _ ffinal) = Parser step initial extract
2577+ (Fold fstep finitial fextract) = Parser step initial extract
25702578
25712579 where
25722580
@@ -2637,9 +2645,9 @@ deintercalateAll
26372645 extractResult n fs r = do
26382646 res <- fstep fs r
26392647 case res of
2640− FL.Partial fs1 -> fmap (Done n) $ ffinal fs1
2648+ FL.Partial fs1 -> fmap (Done n) $ fextract fs1
26412649 FL.Done c -> return (Done n c)
2642− extract (DeintercalateAllInitL fs) = fmap (Done 0) $ ffinal fs
2650+ extract (DeintercalateAllInitL fs) = fmap (Done 0) $ fextract fs
26432651 extract (DeintercalateAllL fs sL) = do
26442652 r <- extractL sL
26452653 case r of
@@ -2647,7 +2655,7 @@ deintercalateAll
26472655 Error err -> return $ Error err
26482656 Continue n s -> return $ Continue n (DeintercalateAllL fs s)
26492657 Partial _ _ -> error "Partial in extract"
2650− extract (DeintercalateAllInitR fs) = fmap (Done 0) $ ffinal fs
2658+ extract (DeintercalateAllInitR fs) = fmap (Done 0) $ fextract fs
26512659 extract (DeintercalateAllR _ _) =
26522660 return $ Error "deintercalateAll: input ended at 'Right' value"
26532661
@@ -2690,7 +2698,7 @@ deintercalate :: Monad m =>
26902698deintercalate
26912699 (Parser stepL initialL extractL)
26922700 (Parser stepR initialR _)
2693− (Fold fstep finitial _ ffinal) = Parser step initial extract
2701+ (Fold fstep finitial fextract) = Parser step initial extract
26942702
26952703 where
26962704
@@ -2721,7 +2729,7 @@ deintercalate
27212729 Done n b ->
27222730 processL (fstep fs (Left b)) n DeintercalateInitR
27232731 Error _ -> do
2724− xs <- ffinal fs
2732+ xs <- fextract fs
27252733 return $ Done cnt1 xs
27262734
27272735 {-# INLINE processR #-}
@@ -2741,7 +2749,7 @@ deintercalate
27412749 Continue n s -> return $ Continue n (DeintercalateR (cnt1 - n) fs s)
27422750 Done n b -> processR (cnt1 - n) b fs n
27432751 Error _ -> do
2744− xs <- ffinal fs
2752+ xs <- fextract fs
27452753 return $ Done cnt1 xs
27462754
27472755 step (DeintercalateInitL fs) a = do
@@ -2776,17 +2784,17 @@ deintercalate
27762784 -- XXX We could have the fold accept pairs of (bR, bL)
27772785 FL.Done _ -> error "Fold terminated consuming partial input"
27782786 Error _ -> do
2779− xs <- ffinal fs
2787+ xs <- fextract fs
27802788 return $ Done cnt1 xs
27812789
27822790 {-# INLINE extractResult #-}
27832791 extractResult n fs r = do
27842792 res <- fstep fs r
27852793 case res of
2786− FL.Partial fs1 -> fmap (Done n) $ ffinal fs1
2794+ FL.Partial fs1 -> fmap (Done n) $ fextract fs1
27872795 FL.Done c -> return (Done n c)
27882796
2789− extract (DeintercalateInitL fs) = fmap (Done 0) $ ffinal fs
2797+ extract (DeintercalateInitL fs) = fmap (Done 0) $ fextract fs
27902798 extract (DeintercalateL cnt fs sL) = do
27912799 r <- extractL sL
27922800 case r of
@@ -2794,10 +2802,10 @@ deintercalate
27942802 Continue n s -> return $ Continue n (DeintercalateL (cnt - n) fs s)
27952803 Partial _ _ -> error "Partial in extract"
27962804 Error _ -> do
2797− xs <- ffinal fs
2805+ xs <- fextract fs
27982806 return $ Done cnt xs
2799− extract (DeintercalateInitR fs) = fmap (Done 0) $ ffinal fs
2800− extract (DeintercalateR cnt fs _) = fmap (Done cnt) $ ffinal fs
2807+ extract (DeintercalateInitR fs) = fmap (Done 0) $ fextract fs
2808+ extract (DeintercalateR cnt fs _) = fmap (Done cnt) $ fextract fs
28012809 extract (DeintercalateRL cnt bR fs sL) = do
28022810 r <- extractL sL
28032811 case r of
@@ -2809,7 +2817,7 @@ deintercalate
28092817 Continue n s -> return $ Continue n (DeintercalateRL (cnt - n) bR fs s)
28102818 Partial _ _ -> error "Partial in extract"
28112819 Error _ -> do
2812− xs <- ffinal fs
2820+ xs <- fextract fs
28132821 return $ Done cnt xs
28142822
28152823{-# ANN type Deintercalate1State Fuse #-}
@@ -2846,7 +2854,7 @@ deintercalate1 :: Monad m =>
28462854deintercalate1
28472855 (Parser stepL initialL extractL)
28482856 (Parser stepR initialR _)
2849− (Fold fstep finitial _ ffinal) = Parser step initial extract
2857+ (Fold fstep finitial fextract) = Parser step initial extract
28502858
28512859 where
28522860
@@ -2900,7 +2908,7 @@ deintercalate1
29002908 Continue n s -> return $ Continue n (Deintercalate1R (cnt1 - n) fs s)
29012909 Done n b -> processR (cnt1 - n) b fs n
29022910 Error _ -> do
2903− xs <- ffinal fs
2911+ xs <- fextract fs
29042912 return $ Done cnt1 xs
29052913
29062914 step (Deintercalate1InitL cnt fs sL) a = runStepInitL cnt fs sL a
@@ -2929,14 +2937,14 @@ deintercalate1
29292937 -- XXX We could have the fold accept pairs of (bR, bL)
29302938 FL.Done _ -> error "Fold terminated consuming partial input"
29312939 Error _ -> do
2932− xs <- ffinal fs
2940+ xs <- fextract fs
29332941 return $ Done cnt1 xs
29342942
29352943 {-# INLINE extractResult #-}
29362944 extractResult n fs r = do
29372945 res <- fstep fs r
29382946 case res of
2939− FL.Partial fs1 -> fmap (Done n) $ ffinal fs1
2947+ FL.Partial fs1 -> fmap (Done n) $ fextract fs1
29402948 FL.Done c -> return (Done n c)
29412949
29422950 extract (Deintercalate1InitL cnt fs sL) = do
@@ -2946,8 +2954,8 @@ deintercalate1
29462954 Continue n s -> return $ Continue n (Deintercalate1InitL (cnt - n) fs s)
29472955 Partial _ _ -> error "Partial in extract"
29482956 Error err -> return $ Error err
2949− extract (Deintercalate1InitR fs) = fmap (Done 0) $ ffinal fs
2950− extract (Deintercalate1R cnt fs _) = fmap (Done cnt) $ ffinal fs
2957+ extract (Deintercalate1InitR fs) = fmap (Done 0) $ fextract fs
2958+ extract (Deintercalate1R cnt fs _) = fmap (Done cnt) $ fextract fs
29512959 extract (Deintercalate1RL cnt bR fs sL) = do
29522960 r <- extractL sL
29532961 case r of
@@ -2959,7 +2967,7 @@ deintercalate1
29592967 Continue n s -> return $ Continue n (Deintercalate1RL (cnt - n) bR fs s)
29602968 Partial _ _ -> error "Partial in extract"
29612969 Error _ -> do
2962− xs <- ffinal fs
2970+ xs <- fextract fs
29632971 return $ Done cnt xs
29642972
29652973{-# ANN type SepByState Fuse #-}
@@ -3002,7 +3010,7 @@ sepBy :: Monad m =>
30023010sepBy
30033011 (Parser stepL initialL extractL)
30043012 (Parser stepR initialR _)
3005− (Fold fstep finitial _ ffinal) = Parser step initial extract
3013+ (Fold fstep finitial fextract) = Parser step initial extract
30063014
30073015 where
30083016
@@ -3033,7 +3041,7 @@ sepBy
30333041 Done n b ->
30343042 processL (fstep fs b) n SepByInitR
30353043 Error _ -> do
3036− xs <- ffinal fs
3044+ xs <- fextract fs
30373045 return $ Done cnt1 xs
30383046
30393047 {-# INLINE processR #-}
@@ -3053,7 +3061,7 @@ sepBy
30533061 Continue n s -> return $ Continue n (SepByR (cnt1 - n) fs s)
30543062 Done n _ -> processR (cnt1 - n) fs n
30553063 Error _ -> do
3056− xs <- ffinal fs
3064+ xs <- fextract fs
30573065 return $ Done cnt1 xs
30583066
30593067 step (SepByInitL fs) a = do
@@ -3075,10 +3083,10 @@ sepBy
30753083 extractResult n fs r = do
30763084 res <- fstep fs r
30773085 case res of
3078− FL.Partial fs1 -> fmap (Done n) $ ffinal fs1
3086+ FL.Partial fs1 -> fmap (Done n) $ fextract fs1
30793087 FL.Done c -> return (Done n c)
30803088
3081− extract (SepByInitL fs) = fmap (Done 0) $ ffinal fs
3089+ extract (SepByInitL fs) = fmap (Done 0) $ fextract fs
30823090 extract (SepByL cnt fs sL) = do
30833091 r <- extractL sL
30843092 case r of
@@ -3086,10 +3094,10 @@ sepBy
30863094 Continue n s -> return $ Continue n (SepByL (cnt - n) fs s)
30873095 Partial _ _ -> error "Partial in extract"
30883096 Error _ -> do
3089− xs <- ffinal fs
3097+ xs <- fextract fs
30903098 return $ Done cnt xs
3091− extract (SepByInitR fs) = fmap (Done 0) $ ffinal fs
3092− extract (SepByR cnt fs _) = fmap (Done cnt) $ ffinal fs
3099+ extract (SepByInitR fs) = fmap (Done 0) $ fextract fs
3100+ extract (SepByR cnt fs _) = fmap (Done cnt) $ fextract fs
30933101
30943102-- | Non-backtracking version of sepBy. Several times faster.
30953103{-# INLINE sepByAll #-}
@@ -3144,7 +3152,7 @@ sepBy1 :: Monad m =>
31443152sepBy1
31453153 (Parser stepL initialL extractL)
31463154 (Parser stepR initialR _)
3147− (Fold fstep finitial _ ffinal) = Parser step initial extract
3155+ (Fold fstep finitial fextract) = Parser step initial extract
31483156
31493157 where
31503158
@@ -3191,7 +3199,7 @@ sepBy1
31913199 Done n b ->
31923200 processL (fstep fs b) n SepBy1InitR
31933201 Error _ -> do
3194− xs <- ffinal fs
3202+ xs <- fextract fs
31953203 return $ Done cnt1 xs
31963204
31973205 {-# INLINE processR #-}
@@ -3211,7 +3219,7 @@ sepBy1
32113219 Continue n s -> return $ Continue n (SepBy1R (cnt1 - n) fs s)
32123220 Done n _ -> processR (cnt1 - n) fs n
32133221 Error _ -> do
3214− xs <- ffinal fs
3222+ xs <- fextract fs
32153223 return $ Done cnt1 xs
32163224
32173225 step (SepBy1InitL cnt fs sL) a = runStepInitL cnt fs sL a
@@ -3228,7 +3236,7 @@ sepBy1
32283236 extractResult n fs r = do
32293237 res <- fstep fs r
32303238 case res of
3231− FL.Partial fs1 -> fmap (Done n) $ ffinal fs1
3239+ FL.Partial fs1 -> fmap (Done n) $ fextract fs1
32323240 FL.Done c -> return (Done n c)
32333241
32343242 extract (SepBy1InitL cnt fs sL) = do
@@ -3245,10 +3253,10 @@ sepBy1
32453253 Continue n s -> return $ Continue n (SepBy1L (cnt - n) fs s)
32463254 Partial _ _ -> error "Partial in extract"
32473255 Error _ -> do
3248− xs <- ffinal fs
3256+ xs <- fextract fs
32493257 return $ Done cnt xs
3250− extract (SepBy1InitR fs) = fmap (Done 0) $ ffinal fs
3251− extract (SepBy1R cnt fs _) = fmap (Done cnt) $ ffinal fs
3258+ extract (SepBy1InitR fs) = fmap (Done 0) $ fextract fs
3259+ extract (SepBy1R cnt fs _) = fmap (Done cnt) $ fextract fs
32523260
32533261-------------------------------------------------------------------------------
32543262-- Interleaving a collection of parsers
@@ -3278,7 +3286,7 @@ roundRobin _ps _f = undefined
32783286{-# INLINE sequence #-}
32793287sequence :: Monad m =>
32803288 D.Stream m (Parser a m b) -> Fold m b c -> Parser a m c
3281−sequence (D.Stream sstep sstate) (Fold fstep finitial _ ffinal) =
3289+sequence (D.Stream sstep sstate) (Fold fstep finitial fextract) =
32823290 Parser step initial extract
32833291
32843292 where
@@ -3296,7 +3304,7 @@ sequence (D.Stream sstep sstate) (Fold fstep finitial _ ffinal) =
32963304 case sres of
32973305 D.Yield p ss1 -> return $ Continue 1 (Just' p, ss1, fs)
32983306 D.Stop -> do
3299− c <- ffinal fs
3307+ c <- fextract fs
33003308 return $ Done 1 c
33013309 D.Skip ss1 -> return $ Continue 1 (Nothing', ss1, fs)
33023310
@@ -3332,7 +3340,7 @@ sequence (D.Stream sstep sstate) (Fold fstep finitial _ ffinal) =
33323340 FL.Done c -> return $ Done 1 c
33333341 IError err -> return $ Error err
33343342
3335− extract (Nothing', _, fs) = fmap (Done 0) $ ffinal fs
3343+ extract (Nothing', _, fs) = fmap (Done 0) $ fextract fs
33363344 extract (Just' (Parser pstep pinit pextr), ss, fs) = do
33373345 ps <- pinit
33383346 case ps of
@@ -3342,7 +3350,7 @@ sequence (D.Stream sstep sstate) (Fold fstep finitial _ ffinal) =
33423350 Done n b -> do
33433351 res <- fstep fs b
33443352 case res of
3345− FL.Partial fs1 -> fmap (Done n) $ ffinal fs1
3353+ FL.Partial fs1 -> fmap (Done n) $ fextract fs1
33463354 FL.Done c -> return (Done n c)
33473355 Error err -> return $ Error err
33483356 Continue n s -> return $ Continue n (Just' (Parser pstep (return (IPartial s)) pextr), ss, fs)
@@ -3350,7 +3358,7 @@ sequence (D.Stream sstep sstate) (Fold fstep finitial _ ffinal) =
33503358 IDone b -> do
33513359 fres <- fstep fs b
33523360 case fres of
3353− FL.Partial fs1 -> fmap (Done 0) $ ffinal fs1
3361+ FL.Partial fs1 -> fmap (Done 0) $ fextract fs1
33543362 FL.Done c -> return (Done 0 c)
33553363 IError err -> return $ Error err
33563364
@@ -3485,7 +3493,7 @@ manyTill :: Monad m
34853493 => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
34863494manyTill (Parser stepL initialL extractL)
34873495 (Parser stepR initialR _)
3488− (Fold fstep finitial _ ffinal) =
3496+ (Fold fstep finitial fextract) =
34893497 Parser step initial extract
34903498
34913499 where
@@ -3507,7 +3515,7 @@ manyTill (Parser stepL initialL extractL)
35073515 resR <- initialR
35083516 case resR of
35093517 IPartial sr -> return $ p (ManyTillR 0 fs sr)
3510− IDone _ -> d <$> ffinal fs
3518+ IDone _ -> d <$> fextract fs
35113519 IError _ -> scrutL fs p c d e
35123520
35133521 initial = do
@@ -3524,7 +3532,7 @@ manyTill (Parser stepL initialL extractL)
35243532 assertM(cnt + 1 - n >= 0)
35253533 return $ Continue n (ManyTillR (cnt + 1 - n) fs s)
35263534 Done n _ -> do
3527− b <- ffinal fs
3535+ b <- fextract fs
35283536 return $ Done n b
35293537 Error _ -> do
35303538 resL <- initialL
@@ -3563,12 +3571,12 @@ manyTill (Parser stepL initialL extractL)
35633571 Done n b -> do
35643572 r <- fstep fs b
35653573 case r of
3566− FL.Partial fs1 -> fmap (Done n) $ ffinal fs1
3574+ FL.Partial fs1 -> fmap (Done n) $ fextract fs1
35673575 FL.Done c -> return (Done n c)
35683576 Error err -> return $ Error err
35693577 Continue n s -> return $ Continue n (ManyTillL fs s)
35703578 Partial _ _ -> error "Partial in extract"
3571− extract (ManyTillR _ fs _) = fmap (Done 0) $ ffinal fs
3579+ extract (ManyTillR _ fs _) = fmap (Done 0) $ fextract fs
35723580
35733581-- | @manyThen f collect recover@ repeats the parser @collect@ on the input and
35743582-- collects the output in the supplied fold. If the the parser @collect@ fails,
core/src/Streamly/Internal/Data/Parser/Tee.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Tee.hsrenamed+1−1
@@ -24,7 +24,7 @@
2424-- some legit warnings, therefore, we have segregated only the code that uses
2525-- uni-pattern matches in this module.
2626
27−module Streamly.Internal.Data.Parser.Tee
27+module Streamly.Internal.Data.Parser.ParserD.Tee
2828 (
2929 {-
3030 -- Parallel zipped
core/src/Streamly/Internal/Data/Parser/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Type.hsrenamed+71−94
@@ -173,8 +173,11 @@
173173-- * <http://hackage.haskell.org/package/attoparsec attoparsec>
174174-- * <http://hackage.haskell.org/package/parsec parsec>
175175
176−module Streamly.Internal.Data.Parser.Type
176+module Streamly.Internal.Data.Parser.ParserD.Type
177177 (
178+ -- * Setup
179+ -- $setup
180+
178181 -- * Types
179182 Initial (..)
180183 , Step (..)
@@ -213,10 +216,7 @@ where
213216#include "inline.hs"
214217#include "assert.hs"
215218
216−#if !MIN_VERSION_base(4,18,0)
217−import Control.Applicative (liftA2)
218−#endif
219−import Control.Applicative (Alternative(..))
219+import Control.Applicative (Alternative(..), liftA2)
220220import Control.Exception (Exception(..))
221221-- import Control.Monad (MonadPlus(..), (>=>))
222222import Control.Monad ((>=>))
@@ -438,7 +438,6 @@ newtype ParseError = ParseError String
438438instance Exception ParseError where
439439 displayException (ParseError err) = err
440440
441−-- | Map a function on the result i.e. on @b@ in @Parser a m b@.
442441instance Functor m => Functor (Parser a m) where
443442 {-# INLINE fmap #-}
444443 fmap f (Parser step1 initial1 extract) =
@@ -505,30 +504,26 @@ data SeqParseState sl f sr = SeqParseL !sl | SeqParseR !f !sr
505504-- 'Streamly.Data.Stream.append', it splits the streams using two parsers and
506505-- zips the results.
507506
508−-- | Sequential parser application. Apply two parsers sequentially to an input
509−-- stream. The first parser runs and processes the input, the remaining input
510−-- is then passed to the second parser. If both parsers succeed, their outputs
511−-- are combined using the supplied function. If either parser fails, the
512−-- operation fails.
507+-- | Sequential parser application.
513508--
514−-- This combinator delivers high performance by stream fusion but it comes with
515−-- some limitations. For those cases use the 'Applicative' instance of
516−-- 'Streamly.Data.ParserK.ParserK'.
509+-- Apply two parsers sequentially to an input stream. The first parser runs and
510+-- processes the input, the remaining input is then passed to the second
511+-- parser. If both parsers succeed, their outputs are combined using the
512+-- supplied function. If either parser fails, the operation fails.
517513--
518−-- CAVEAT 1: NO RECURSION. This function is strict in both arguments. As a
519−-- result, if a parser is defined recursively using this, it may cause an
520−-- infintie loop. The following example checks the strictness:
514+-- This implementation is strict in the second argument, therefore, the
515+-- following will fail:
521516--
522−-- >>> p = Parser.splitWith const (Parser.satisfy (> 0)) undefined
523−-- >>> Stream.parse p $ Stream.fromList [1]
517+-- >>> Stream.parse (Parser.splitWith const (Parser.satisfy (> 0)) undefined) $ Stream.fromList [1]
524518-- *** Exception: Prelude.undefined
525519-- ...
526520--
527−-- CAVEAT 2: QUADRATIC TIME COMPLEXITY. Static composition is fast due to
528−-- stream fusion, but it works well only for limited (e.g. up to 8)
529−-- compositions, use "Streamly.Data.ParserK" for larger compositions.
521+-- Although this implementation allows stream fusion, it has quadratic
522+-- complexity, making it suitable only for a small number of compositions.
523+-- As a thumb rule use it for less than 8 compositions, use ParserK otherwise.
530524--
531−-- Below are some common idioms that can be expressed using 'splitWith':
525+-- Below are some common idioms that can be expressed using 'splitWith' and
526+-- other parser primitives:
532527--
533528-- >>> span p f1 f2 = Parser.splitWith (,) (Parser.takeWhile p f1) (Parser.fromFold f2)
534529-- >>> spanBy eq f1 f2 = Parser.splitWith (,) (Parser.groupBy eq f1) (Parser.fromFold f2)
@@ -617,12 +612,7 @@ splitWith func (Parser stepL initialL extractL)
617612-- We can use the compiler to automatically annotate accumulators, terminating
618613-- folds, non-failing parsers and failing parsers.
619614
620−-- | Better performance 'splitWith' for non-failing parsers.
621−--
622−-- Does not work correctly for parsers that can fail.
623−--
624−-- ALL THE CAVEATS IN 'splitWith' APPLY HERE AS WELL.
625−--
615+-- | Works correctly only if both the parsers are guaranteed to never fail.
626616{-# INLINE noErrorUnsafeSplitWith #-}
627617noErrorUnsafeSplitWith :: Monad m
628618 => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
@@ -693,8 +683,6 @@ data SeqAState sl sr = SeqAL !sl | SeqAR !sr
693683-- second parser. The output of the parser is the output of the second parser.
694684-- The operation fails if any of the parsers fail.
695685--
696−-- ALL THE CAVEATS IN 'splitWith' APPLY HERE AS WELL.
697−--
698686-- This implementation is strict in the second argument, therefore, the
699687-- following will fail:
700688--
@@ -702,6 +690,10 @@ data SeqAState sl sr = SeqAL !sl | SeqAR !sr
702690-- *** Exception: Prelude.undefined
703691-- ...
704692--
693+-- Although this implementation allows stream fusion, it has quadratic
694+-- complexity, making it suitable only for a small number of compositions.
695+-- As a thumb rule use it for less than 8 compositions, use ParserK otherwise.
696+--
705697-- /Pre-release/
706698--
707699{-# INLINE split_ #-}
@@ -757,12 +749,7 @@ split_ (Parser stepL initialL extractL) (Parser stepR initialR extractR) =
757749 Partial _ _ -> error "split_: Partial"
758750 Continue n s -> return $ Continue n (SeqAL s)
759751
760−-- | Better performance 'split_' for non-failing parsers.
761−--
762−-- Does not work correctly for parsers that can fail.
763−--
764−-- ALL THE CAVEATS IN 'splitWith' APPLY HERE AS WELL.
765−--
752+-- For backtracking folds
766753{-# INLINE noErrorUnsafeSplit_ #-}
767754noErrorUnsafeSplit_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
768755noErrorUnsafeSplit_
@@ -817,11 +804,7 @@ noErrorUnsafeSplit_
817804 Partial _ _ -> error "split_: Partial"
818805 Continue n s -> return $ Continue n (SeqAL s)
819806
820−-- | READ THE CAVEATS in 'splitWith' before using this instance.
821−--
822−-- >>> pure = Parser.fromPure
823−-- >>> (<*>) = Parser.splitWith id
824−-- >>> (*>) = Parser.split_
807+-- | 'Applicative' form of 'splitWith'.
825808instance Monad m => Applicative (Parser a m) where
826809 {-# INLINE pure #-}
827810 pure = fromPure
@@ -847,26 +830,20 @@ data AltParseState sl sr = AltParseL !Int !sl | AltParseR !sr
847830-- each subsequent alternative's input element has to go through, therefore, it
848831-- cannot scale to a large number of compositions
849832
850−-- | Sequential alternative. The input is first passed to the first parser,
833+-- | Sequential alternative. The input is first passed to the first parser, and
851834-- if it succeeds, the result is returned. However, if the first parser fails,
852−-- the parser driver backtracks and tries the same input on the second
853−-- (alternative) parser, returning the result if it succeeds.
854−--
855−-- This combinator delivers high performance by stream fusion but it comes with
856−-- some limitations. For those cases use the 'Alternative' instance of
857−-- 'Streamly.Data.ParserK.ParserK'.
835+-- the parser driver backtracks and tries the same input on the second parser,
836+-- returning the result if it succeeds.
858837--
859−-- CAVEAT 1: NO RECURSION. This function is strict in both arguments. As a
860−-- result, if a parser is defined recursively using this, it may cause an
861−-- infintie loop. The following example checks the strictness:
838+-- Note: This implementation is not lazy in the second argument. The following
839+-- will fail:
862840--
863−-- >>> p = Parser.satisfy (> 0) `Parser.alt` undefined
864−-- >>> Stream.parse p $ Stream.fromList [1..10]
841+-- >> Stream.parse (Parser.satisfy (> 0) `Parser.alt` undefined) $ Stream.fromList [1..10]
865842-- *** Exception: Prelude.undefined
866843--
867−-- CAVEAT 2: QUADRATIC TIME COMPLEXITY. Static composition is fast due to
868−-- stream fusion, but it works well only for limited (e.g. up to 8)
869−-- compositions, use "Streamly.Data.ParserK" for larger compositions.
844+-- Although this implementation allows stream fusion, it has quadratic
845+-- complexity, making it suitable only for a small number of compositions.
846+-- As a thumb rule use it for less than 8 compositions, use ParserK otherwise.
870847--
871848-- /Time Complexity:/ O(n^2) where n is the number of compositions.
872849--
@@ -947,7 +924,7 @@ data Fused3 a b c = Fused3 !a !b !c
947924--
948925{-# INLINE splitMany #-}
949926splitMany :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
950−splitMany (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
927+splitMany (Parser step1 initial1 extract1) (Fold fstep finitial fextract) =
951928 Parser step initial extract
952929
953930 where
@@ -963,7 +940,7 @@ splitMany (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
963940 IPartial ps -> return $ partial $ Fused3 ps 0 fs
964941 IDone pb ->
965942 runCollectorWith (handleCollect partial done) fs pb
966− IError _ -> done <$> ffinal fs
943+ IError _ -> done <$> fextract fs
967944 FL.Done fb -> return $ done fb
968945
969946 runCollectorWith cont fs pb = fstep fs pb >>= cont
@@ -987,19 +964,19 @@ splitMany (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
987964 assertM(cnt1 - n >= 0)
988965 fstep fs b >>= handleCollect (Partial n) (Done n)
989966 Error _ -> do
990− xs <- ffinal fs
967+ xs <- fextract fs
991968 return $ Done cnt xs
992969
993− extract (Fused3 _ 0 fs) = fmap (Done 0) (ffinal fs)
970+ extract (Fused3 _ 0 fs) = fmap (Done 0) (fextract fs)
994971 extract (Fused3 s cnt fs) = do
995972 r <- extract1 s
996973 case r of
997− Error _ -> fmap (Done cnt) (ffinal fs)
974+ Error _ -> fmap (Done cnt) (fextract fs)
998975 Done n b -> do
999976 assertM(n <= cnt)
1000977 fs1 <- fstep fs b
1001978 case fs1 of
1002− FL.Partial s1 -> fmap (Done n) (ffinal s1)
979+ FL.Partial s1 -> fmap (Done n) (fextract s1)
1003980 FL.Done b1 -> return (Done n b1)
1004981 Partial _ _ -> error "splitMany: Partial in extract"
1005982 Continue n s1 -> do
@@ -1013,7 +990,7 @@ splitMany (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
1013990--
1014991{-# INLINE splitManyPost #-}
1015992splitManyPost :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
1016−splitManyPost (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
993+splitManyPost (Parser step1 initial1 extract1) (Fold fstep finitial fextract) =
1017994 Parser step initial extract
1018995
1019996 where
@@ -1029,7 +1006,7 @@ splitManyPost (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
10291006 IPartial ps -> return $ partial $ Fused3 ps 0 fs
10301007 IDone pb ->
10311008 runCollectorWith (handleCollect partial done) fs pb
1032− IError _ -> done <$> ffinal fs
1009+ IError _ -> done <$> fextract fs
10331010 FL.Done fb -> return $ done fb
10341011
10351012 runCollectorWith cont fs pb = fstep fs pb >>= cont
@@ -1051,18 +1028,18 @@ splitManyPost (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
10511028 assertM(cnt1 - n >= 0)
10521029 fstep fs b >>= handleCollect (Partial n) (Done n)
10531030 Error _ -> do
1054− xs <- ffinal fs
1031+ xs <- fextract fs
10551032 return $ Done cnt1 xs
10561033
10571034 extract (Fused3 s cnt fs) = do
10581035 r <- extract1 s
10591036 case r of
1060− Error _ -> fmap (Done cnt) (ffinal fs)
1037+ Error _ -> fmap (Done cnt) (fextract fs)
10611038 Done n b -> do
10621039 assertM(n <= cnt)
10631040 fs1 <- fstep fs b
10641041 case fs1 of
1065− FL.Partial s1 -> fmap (Done n) (ffinal s1)
1042+ FL.Partial s1 -> fmap (Done n) (fextract s1)
10661043 FL.Done b1 -> return (Done n b1)
10671044 Partial _ _ -> error "splitMany: Partial in extract"
10681045 Continue n s1 -> do
@@ -1075,7 +1052,7 @@ splitManyPost (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
10751052--
10761053{-# INLINE splitSome #-}
10771054splitSome :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
1078−splitSome (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
1055+splitSome (Parser step1 initial1 extract1) (Fold fstep finitial fextract) =
10791056 Parser step initial extract
10801057
10811058 where
@@ -1091,7 +1068,7 @@ splitSome (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
10911068 IPartial ps -> return $ partial $ Fused3 ps 0 $ Right fs
10921069 IDone pb ->
10931070 runCollectorWith (handleCollect partial done) fs pb
1094− IError _ -> done <$> ffinal fs
1071+ IError _ -> done <$> fextract fs
10951072 FL.Done fb -> return $ done fb
10961073
10971074 runCollectorWith cont fs pb = fstep fs pb >>= cont
@@ -1141,7 +1118,7 @@ splitSome (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
11411118 Done n b -> do
11421119 assertM(cnt1 - n >= 0)
11431120 fstep fs b >>= handleCollect (Partial n) (Done n)
1144− Error _ -> Done cnt1 <$> ffinal fs
1121+ Error _ -> Done cnt1 <$> fextract fs
11451122
11461123 extract (Fused3 s cnt (Left fs)) = do
11471124 r <- extract1 s
@@ -1151,7 +1128,7 @@ splitSome (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
11511128 assertM(n <= cnt)
11521129 fs1 <- fstep fs b
11531130 case fs1 of
1154− FL.Partial s1 -> fmap (Done n) (ffinal s1)
1131+ FL.Partial s1 -> fmap (Done n) (fextract s1)
11551132 FL.Done b1 -> return (Done n b1)
11561133 Partial _ _ -> error "splitSome: Partial in extract"
11571134 Continue n s1 -> do
@@ -1160,12 +1137,12 @@ splitSome (Parser step1 initial1 extract1) (Fold fstep finitial _ ffinal) =
11601137 extract (Fused3 s cnt (Right fs)) = do
11611138 r <- extract1 s
11621139 case r of
1163− Error _ -> fmap (Done cnt) (ffinal fs)
1140+ Error _ -> fmap (Done cnt) (fextract fs)
11641141 Done n b -> do
11651142 assertM(n <= cnt)
11661143 fs1 <- fstep fs b
11671144 case fs1 of
1168− FL.Partial s1 -> fmap (Done n) (ffinal s1)
1145+ FL.Partial s1 -> fmap (Done n) (fextract s1)
11691146 FL.Done b1 -> return (Done n b1)
11701147 Partial _ _ -> error "splitSome: Partial in extract"
11711148 Continue n s1 -> do
@@ -1196,12 +1173,23 @@ dieM err = Parser undefined (IError <$> err) undefined
11961173-- Note: With the direct style parser type, the list in "some" and "many" is
11971174-- accumulated strictly, it cannot be consumed lazily.
11981175
1199−-- | READ THE CAVEATS in 'alt' before using this instance.
1176+-- | Sequential alternative. The input is first passed to the first parser, and
1177+-- if it succeeds, the result is returned. However, if the first parser fails,
1178+-- the parser driver backtracks and tries the same input on the second parser,
1179+-- returning the result if it succeeds.
1180+--
1181+-- Note: The implementation of '<|>' is not lazy in the second
1182+-- argument. The following code will fail:
1183+--
1184+-- >>> Stream.parse (Parser.satisfy (> 0) <|> undefined) $ Stream.fromList [1..10]
1185+-- *** Exception: Prelude.undefined
1186+-- ...
12001187--
1201−-- >>> empty = Parser.die "empty"
1202−-- >>> (<|>) = Parser.alt
1203−-- >>> many = flip Parser.many Fold.toList
1204−-- >>> some = flip Parser.some Fold.toList
1188+-- WARNING! this is not suitable for large scale use. As a thumb rule stream
1189+-- fusion works well for less than 8 compositions of this operation, otherwise
1190+-- consider using 'ParserK'. Do not use recursive parser implementations based
1191+-- on this Alternative instance.
1192+
12051193instance Monad m => Alternative (Parser a m) where
12061194 {-# INLINE empty #-}
12071195 empty = die "empty"
@@ -1220,13 +1208,8 @@ data ConcatParseState sl m a b =
12201208 ConcatParseL !sl
12211209 | forall s. ConcatParseR (s -> a -> m (Step s b)) s (s -> m (Step s b))
12221210
1223−-- XXX Does it fuse completely? Check and update, it cannot fuse the
1224−-- dynamically generated parser.
1225−
12261211-- | Map a 'Parser' returning function on the result of a 'Parser'.
12271212--
1228−-- ALL THE CAVEATS IN 'splitWith' APPLY HERE AS WELL.
1229−--
12301213-- /Pre-release/
12311214--
12321215{-# INLINE concatMap #-}
@@ -1296,12 +1279,6 @@ concatMap func (Parser stepL initialL extractL) = Parser step initial extract
12961279 Partial _ _ -> error "concatMap: extract Partial"
12971280 Continue n s -> return $ Continue n (ConcatParseL s)
12981281
1299−-- | Better performance 'concatMap' for non-failing parsers.
1300−--
1301−-- Does not work correctly for parsers that can fail.
1302−--
1303−-- ALL THE CAVEATS IN 'splitWith' APPLY HERE AS WELL.
1304−--
13051282{-# INLINE noErrorUnsafeConcatMap #-}
13061283noErrorUnsafeConcatMap :: Monad m =>
13071284 (b -> Parser a m c) -> Parser a m b -> Parser a m c
@@ -1374,9 +1351,11 @@ noErrorUnsafeConcatMap func (Parser stepL initialL extractL) =
13741351-- for small number of compositions but for a scalable implementation we need a
13751352-- CPS version.
13761353
1377−-- | READ THE CAVEATS in 'concatMap' before using this instance.
1354+-- | See documentation of 'Streamly.Internal.Data.Parser.ParserK.Type.Parser'.
13781355--
1379−-- >>> (>>=) = flip Parser.concatMap
1356+-- Although this implementation allows stream fusion, it has quadratic
1357+-- complexity, making it suitable only for a small number of compositions. As a
1358+-- thumb rule use it for less than 8 compositions, use 'ParserK' otherwise.
13801359--
13811360instance Monad m => Monad (Parser a m) where
13821361 {-# INLINE return #-}
@@ -1388,7 +1367,6 @@ instance Monad m => Monad (Parser a m) where
13881367 {-# INLINE (>>) #-}
13891368 (>>) = (*>)
13901369
1391−-- | >>> fail = Parser.die
13921370instance Monad m => Fail.MonadFail (Parser a m) where
13931371 {-# INLINE fail #-}
13941372 fail = die
@@ -1404,7 +1382,6 @@ instance Monad m => MonadPlus (Parser a m) where
14041382 mplus = alt
14051383-}
14061384
1407−-- | >>> liftIO = Parser.fromEffect . liftIO
14081385instance (MonadIO m) => MonadIO (Parser a m) where
14091386 {-# INLINE liftIO #-}
14101387 liftIO = fromEffect . liftIO
core/src/Streamly/Internal/Data/Parser/ParserDK.hsnew file+1544
@@ -0,0 +1,1544 @@
1+{-# OPTIONS_GHC -Wno-orphans #-}
2+
3+-- |
4+-- Module : Streamly.Internal.Data.Parser
5+-- Copyright : (c) 2020 Composewell Technologies
6+-- License : BSD-3-Clause
7+-- Maintainer : streamly@composewell.com
8+-- Stability : pre-release
9+-- Portability : GHC
10+--
11+-- Fast backtracking parsers with stream fusion and native streaming
12+-- capability.
13+--
14+-- 'Applicative' and 'Control.Applicative.Alternative' type class based
15+-- combinators from the
16+-- <http://hackage.haskell.org/package/parser-combinators parser-combinators>
17+-- package can also be used with the 'Parser' type. However, there are two
18+-- important differences between @parser-combinators@ and the equivalent ones
19+-- provided in this module in terms of performance:
20+--
21+-- 1) @parser-combinators@ use plain Haskell lists to collect the results, in a
22+-- strict Monad like IO, the results are necessarily buffered before they can
23+-- be consumed. This may not perform optimally in streaming applications
24+-- processing large amounts of data. Equivalent combinators in this module can
25+-- consume the results of parsing using a 'Fold', thus providing a scalability
26+-- and a composable consumer.
27+--
28+-- 2) Several combinators in this module can be many times faster because of
29+-- stream fusion. For example, 'Streamly.Internal.Data.Parser.many' combinator
30+-- in this module is much faster than the 'Control.Applicative.many' combinator
31+-- of 'Control.Applicative.Alternative' type class.
32+--
33+-- = Errors
34+--
35+-- Failing parsers in this module throw the 'D.ParseError' exception.
36+--
37+-- = Naming
38+--
39+-- As far as possible, try that the names of the combinators in this module are
40+-- consistent with:
41+--
42+-- * <https://hackage.haskell.org/package/base/docs/Text-ParserCombinators-ReadP.html base/Text.ParserCombinators.ReadP>
43+-- * <http://hackage.haskell.org/package/parser-combinators parser-combinators>
44+-- * <http://hackage.haskell.org/package/megaparsec megaparsec>
45+-- * <http://hackage.haskell.org/package/attoparsec attoparsec>
46+-- * <http://hackage.haskell.org/package/parsec parsec>
47+
48+module Streamly.Internal.Data.Parser
49+ (
50+ K.Parser (..)
51+ , D.ParseError (..)
52+ , D.Step (..)
53+
54+ -- * Downgrade to Fold
55+ , toFold
56+
57+ -- First order parsers
58+ -- * Accumulators
59+ , fromFold
60+ , fromFoldMaybe
61+ , fromPure
62+ , fromEffect
63+ , die
64+ , dieM
65+
66+ -- * Map on input
67+ , lmap
68+ , lmapM
69+ , filter
70+
71+ -- * Map on output
72+ , rmapM
73+
74+ -- * Element parsers
75+ , peek
76+
77+ -- All of these can be expressed in terms of either
78+ , one
79+ , oneEq
80+ , oneNotEq
81+ , oneOf
82+ , noneOf
83+ , eof
84+ , satisfy
85+ , maybe
86+ , either
87+
88+ -- * Sequence parsers (tokenizers)
89+ --
90+ -- | Parsers chained in series, if one parser terminates the composition
91+ -- terminates.
92+
93+ , lookAhead
94+
95+ -- ** By length
96+ -- | Grab a sequence of input elements without inspecting them
97+ , takeBetween
98+ -- , take -- takeBetween 0 n
99+ , takeEQ -- takeBetween n n
100+ , takeGE -- takeBetween n maxBound
101+ , takeP
102+
103+ -- Grab a sequence of input elements by inspecting them
104+ -- ** Exact match
105+ , listEq
106+ , listEqBy
107+ , streamEqBy
108+ , subsequenceBy
109+
110+ -- ** By predicate
111+ , takeWhileP
112+ , takeWhile
113+ -- $takeWhile
114+ , takeWhile1
115+ , dropWhile
116+
117+ -- ** Separators
118+ , takeEndBy
119+ , takeEndBy_
120+ , takeEndByEsc
121+ -- , takeEndByEsc_
122+ , takeStartBy
123+ , takeStartBy_
124+ , takeEitherSepBy
125+ , wordBy
126+ -- , wordByEsc
127+
128+ -- ** By comparing
129+ , groupBy
130+ , groupByRolling
131+ , groupByRollingEither
132+
133+ -- ** Framing
134+ -- , takeFramedBy
135+ , takeFramedBy_
136+ , takeFramedByEsc_
137+ , takeFramedByGeneric
138+ , wordFramedBy
139+ , wordQuotedBy
140+
141+ -- | Unimplemented
142+ --
143+ -- @
144+ -- , prefixOf -- match any prefix of a given string
145+ -- , suffixOf -- match any suffix of a given string
146+ -- , infixOf -- match any substring of a given string
147+ -- @
148+
149+ -- Second order parsers (parsers using parsers)
150+ -- * Binary Combinators
151+
152+ -- ** Sequential Applicative
153+ , splitWith
154+ , split_
155+
156+{-
157+ -- ** Parallel Applicatives
158+ , teeWith
159+ , teeWithFst
160+ , teeWithMin
161+ -- , teeTill -- like manyTill but parallel
162+-}
163+
164+ -- ** Sequential Interleaving
165+ -- Use two folds, run a primary parser, its rejected values go to the
166+ -- secondary parser.
167+ , deintercalate
168+ -- , deintercalatePrefix
169+ -- , deintercalateSuffix
170+
171+ -- *** Special cases
172+ -- | TODO: traditional implmentations of these may be of limited use. For
173+ -- example, consider parsing lines separated by @\\r\\n@. The main parser
174+ -- will have to detect and exclude the sequence @\\r\\n@ anyway so that we
175+ -- can apply the "sep" parser.
176+ --
177+ -- We can instead implement these as special cases of deintercalate.
178+ --
179+ -- @
180+ -- , endBy
181+ -- , sepEndBy
182+ -- , beginBy
183+ -- , sepBeginBy
184+ -- , sepAroundBy
185+ -- @
186+ , sepBy1
187+ , sepBy
188+
189+ -- ** Sequential Alternative
190+ , alt
191+
192+{-
193+ -- ** Parallel Alternatives
194+ , shortest
195+ , longest
196+ -- , fastest
197+-}
198+
199+ -- * N-ary Combinators
200+ -- ** Sequential Collection
201+ , concatSequence
202+ , concatMap
203+
204+ -- ** Sequential Repetition
205+ , count
206+ , countBetween
207+
208+ , manyP
209+ , many
210+ , some
211+ , manyTillP
212+ , manyTill
213+ , manyThen
214+
215+ -- * Distribution
216+ --
217+ -- | A simple and stupid impl would be to just convert the stream to an
218+ -- array and give the array reference to all consumers. The array can be
219+ -- grown on demand by any consumer and truncated when nonbody needs it.
220+
221+ -- ** Distribute to collection
222+ -- ** Distribute to repetition
223+
224+ -- ** Interleaved collection
225+ -- |
226+ --
227+ -- 1. Round robin
228+ -- 2. Priority based
229+ , roundRobin
230+
231+ -- ** Collection of Alternatives
232+ -- | Unimplemented
233+ --
234+ -- @
235+ -- , shortestN
236+ -- , longestN
237+ -- , fastestN -- first N successful in time
238+ -- , choiceN -- first N successful in position
239+ -- @
240+ -- , choice -- first successful in position
241+
242+ -- ** Repeated Alternatives
243+ , retryMaxTotal
244+ , retryMaxSuccessive
245+ , retry
246+
247+ -- * Deprecated
248+ , next
249+ )
250+where
251+
252+import Streamly.Internal.Data.Fold.Type (Fold(..))
253+import Streamly.Internal.Data.Parser.ParserK.Type (Parser)
254+
255+import qualified Data.Foldable as Foldable
256+import qualified Streamly.Internal.Data.Fold.Type as FL
257+import qualified Streamly.Internal.Data.Parser as D
258+import qualified Streamly.Internal.Data.Parser.ParserK.Type as K
259+
260+#ifdef USE_STREAMK
261+import Streamly.Internal.Data.StreamK (Stream)
262+import qualified Streamly.Internal.Data.StreamK as Stream
263+#else
264+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
265+import qualified Streamly.Internal.Data.Stream.StreamD.Type as Stream
266+#endif
267+
268+import Prelude hiding
269+ ( any, all, dropWhile, take, takeWhile, sequence, concatMap, maybe, either
270+ , filter )
271+
272+--
273+-- $setup
274+-- >>> :m
275+-- >>> import Prelude hiding (any, all, dropWhile, take, takeWhile, sequence, concatMap, maybe, either, filter)
276+-- >>> import Control.Applicative ((<|>))
277+-- >>> import Data.Char (isSpace)
278+-- >>> import qualified Data.Maybe as Maybe
279+-- >>> import qualified Data.Foldable as Foldable
280+-- >>> import qualified Streamly.Data.Stream as Stream
281+-- >>> import qualified Streamly.Internal.Data.Stream as Stream (parse, parseMany)
282+-- >>> import qualified Streamly.Internal.Data.Fold as Fold
283+-- >>> import qualified Streamly.Internal.Data.Parser as Parser
284+
285+-------------------------------------------------------------------------------
286+-- Downgrade a parser to a Fold
287+-------------------------------------------------------------------------------
288+--
289+-- | Make a 'Fold' from a 'Parser'. The fold just throws an exception if the
290+-- parser fails or tries to backtrack.
291+--
292+-- This can be useful in combinators that accept a Fold and we know that a
293+-- Parser cannot fail or failure exception is acceptable as there is no way to
294+-- recover.
295+--
296+-- /Pre-release/
297+--
298+{-# INLINE toFold #-}
299+toFold :: Monad m => Parser a m b -> Fold m a b
300+toFold p = D.toFold $ D.fromParserK p
301+
302+-------------------------------------------------------------------------------
303+-- Upgrade folds to parses
304+-------------------------------------------------------------------------------
305+--
306+-- | Make a 'Parser' from a 'Fold'.
307+--
308+{-# INLINE fromFold #-}
309+fromFold :: Monad m => Fold m a b -> Parser a m b
310+fromFold = D.toParserK . D.fromFold
311+
312+-- | Convert a Maybe returning fold to an error returning parser. The first
313+-- argument is the error message that the parser would return when the fold
314+-- returns Nothing.
315+--
316+-- /Pre-release/
317+--
318+{-# INLINE fromFoldMaybe #-}
319+fromFoldMaybe :: Monad m => String -> Fold m a (Maybe b) -> Parser a m b
320+fromFoldMaybe err = D.toParserK . D.fromFoldMaybe err
321+
322+-------------------------------------------------------------------------------
323+-- Terminating but not failing folds
324+-------------------------------------------------------------------------------
325+--
326+-- This is the dual of stream "fromPure".
327+--
328+-- | A parser that always yields a pure value without consuming any input.
329+--
330+{-# INLINE [3] fromPure #-}
331+fromPure :: Monad m => b -> Parser a m b
332+fromPure = D.toParserK . D.fromPure
333+{-# RULES "fromPure fallback to CPS" [2]
334+ forall a. D.toParserK (D.fromPure a) = K.fromPure a #-}
335+
336+-- This is the dual of stream "fromEffect".
337+--
338+-- | A parser that always yields the result of an effectful action without
339+-- consuming any input.
340+--
341+{-# INLINE fromEffect #-}
342+fromEffect :: Monad m => m b -> Parser a m b
343+fromEffect = K.fromEffect -- D.toParserK . D.fromEffect
344+
345+-- This is the dual of "nil".
346+--
347+-- | A parser that always fails with an error message without consuming
348+-- any input.
349+--
350+{-# INLINE [3] die #-}
351+die :: Monad m => String -> Parser a m b
352+die = D.toParserK . D.die
353+{-# RULES "die fallback to CPS" [2]
354+ forall a. D.toParserK (D.die a) = K.die a #-}
355+
356+-- This is the dual of "nilM".
357+--
358+-- | A parser that always fails with an effectful error message and without
359+-- consuming any input.
360+--
361+-- /Pre-release/
362+--
363+{-# INLINE dieM #-}
364+dieM :: Monad m => m String -> Parser a m b
365+dieM = D.toParserK . D.dieM
366+
367+-------------------------------------------------------------------------------
368+-- Map on input
369+-------------------------------------------------------------------------------
370+
371+-- | @lmap f parser@ maps the function @f@ on the input of the parser.
372+--
373+-- >>> Stream.parse (Parser.lmap (\x -> x * x) (Parser.fromFold Fold.sum)) (Stream.enumerateFromTo 1 100)
374+-- Right 338350
375+--
376+-- > lmap = Parser.lmapM return
377+--
378+{-# INLINE lmap #-}
379+lmap :: Monad m => (a -> b) -> Parser b m r -> Parser a m r
380+lmap f p = D.toParserK $ D.lmap f $ D.fromParserK p
381+
382+-- | @lmapM f parser@ maps the monadic function @f@ on the input of the parser.
383+--
384+{-# INLINE lmapM #-}
385+lmapM :: Monad m => (a -> m b) -> Parser b m r -> Parser a m r
386+lmapM f p = D.toParserK $ D.lmapM f $ D.fromParserK p
387+
388+-- | @rmapM f parser@ maps the monadic function @f@ on the output of the parser.
389+--
390+-- >>> rmap = fmap
391+{-# INLINE rmapM #-}
392+rmapM :: Monad m => (b -> m c) -> Parser a m b -> Parser a m c
393+rmapM f p = D.toParserK $ D.rmapM f $ D.fromParserK p
394+
395+-- | Include only those elements that pass a predicate.
396+--
397+-- >>> Stream.parse (Parser.filter (> 5) (Parser.fromFold Fold.sum)) $ Stream.fromList [1..10]
398+-- Right 40
399+--
400+{-# INLINE filter #-}
401+filter :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
402+filter f p = D.toParserK $ D.filter f $ D.fromParserK p
403+
404+-------------------------------------------------------------------------------
405+-- Failing Parsers
406+-------------------------------------------------------------------------------
407+
408+-- | Peek the head element of a stream, without consuming it. Fails if it
409+-- encounters end of input.
410+--
411+-- >>> Stream.parse ((,) <$> Parser.peek <*> Parser.satisfy (> 0)) $ Stream.fromList [1]
412+-- Right (1,1)
413+--
414+-- @
415+-- peek = lookAhead (satisfy True)
416+-- @
417+--
418+{-# INLINE peek #-}
419+peek :: Monad m => Parser a m a
420+peek = D.toParserK D.peek
421+
422+-- | Succeeds if we are at the end of input, fails otherwise.
423+--
424+-- >>> Stream.parse ((,) <$> Parser.satisfy (> 0) <*> Parser.eof) $ Stream.fromList [1]
425+-- Right (1,())
426+--
427+{-# INLINE eof #-}
428+eof :: Monad m => Parser a m ()
429+eof = D.toParserK D.eof
430+
431+-- | Returns the next element if it passes the predicate, fails otherwise.
432+--
433+-- >>> Stream.parse (Parser.satisfy (== 1)) $ Stream.fromList [1,0,1]
434+-- Right 1
435+--
436+-- >>> toMaybe f x = if f x then Just x else Nothing
437+-- >>> satisfy f = Parser.maybe (toMaybe f)
438+--
439+{-# INLINE satisfy #-}
440+satisfy :: Monad m => (a -> Bool) -> Parser a m a
441+satisfy = D.toParserK . D.satisfy
442+
443+-- | Consume one element from the head of the stream. Fails if it encounters
444+-- end of input.
445+--
446+-- >>> one = Parser.satisfy $ const True
447+--
448+{-# INLINE one #-}
449+one :: Monad m => Parser a m a
450+one = satisfy $ const True
451+
452+-- Alternate names: "only", "onlyThis".
453+
454+-- | Match a specific element.
455+--
456+-- >>> oneEq x = Parser.satisfy (== x)
457+--
458+{-# INLINE oneEq #-}
459+oneEq :: (Monad m, Eq a) => a -> Parser a m a
460+oneEq x = satisfy (== x)
461+
462+-- Alternate names: "exclude", "notThis".
463+
464+-- | Match anything other than the supplied element.
465+--
466+-- >>> oneNotEq x = Parser.satisfy (/= x)
467+--
468+{-# INLINE oneNotEq #-}
469+oneNotEq :: (Monad m, Eq a) => a -> Parser a m a
470+oneNotEq x = satisfy (/= x)
471+
472+-- | Match any one of the elements in the supplied list.
473+--
474+-- >>> oneOf xs = Parser.satisfy (`Foldable.elem` xs)
475+--
476+-- When performance matters a pattern matching predicate could be more
477+-- efficient than a 'Foldable' datatype:
478+--
479+-- @
480+-- let p x =
481+-- case x of
482+-- 'a' -> True
483+-- 'e' -> True
484+-- _ -> False
485+-- in satisfy p
486+-- @
487+--
488+-- GHC may use a binary search instead of linear search in the list.
489+-- Alternatively, you can also use an array instead of list for storage and
490+-- search.
491+--
492+{-# INLINE oneOf #-}
493+oneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
494+oneOf xs = satisfy (`Foldable.elem` xs)
495+
496+-- | See performance notes in 'oneOf'.
497+--
498+-- >>> noneOf xs = Parser.satisfy (`Foldable.notElem` xs)
499+--
500+{-# INLINE noneOf #-}
501+noneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
502+noneOf xs = satisfy (`Foldable.notElem` xs)
503+
504+-- | Return the next element of the input. Returns 'Nothing'
505+-- on end of input. Also known as 'head'.
506+--
507+-- /Pre-release/
508+--
509+{-# DEPRECATED next "Please use \"fromFold Fold.one\" instead" #-}
510+{-# INLINE next #-}
511+next :: Monad m => Parser a m (Maybe a)
512+next = D.toParserK D.next
513+
514+-- | Map a 'Maybe' returning function on the next element in the stream. The
515+-- parser fails if the function returns 'Nothing' otherwise returns the 'Just'
516+-- value.
517+--
518+-- >>> toEither = Maybe.maybe (Left "maybe: predicate failed") Right
519+-- >>> maybe f = Parser.either (toEither . f)
520+--
521+-- >>> maybe f = Parser.fromFoldMaybe "maybe: predicate failed" (Fold.maybe f)
522+--
523+-- /Pre-release/
524+--
525+{-# INLINE maybe #-}
526+maybe :: Monad m => (a -> Maybe b) -> Parser a m b
527+maybe = D.toParserK . D.maybe
528+
529+-- | Map an 'Either' returning function on the next element in the stream. If
530+-- the function returns 'Left err', the parser fails with the error message
531+-- @err@ otherwise returns the 'Right' value.
532+--
533+-- /Pre-release/
534+--
535+{-# INLINE either #-}
536+either :: Monad m => (a -> Either String b) -> Parser a m b
537+either = D.toParserK . D.either
538+
539+-------------------------------------------------------------------------------
540+-- Taking elements
541+-------------------------------------------------------------------------------
542+
543+-- | @takeBetween m n@ takes a minimum of @m@ and a maximum of @n@ input
544+-- elements and folds them using the supplied fold.
545+--
546+-- Stops after @n@ elements.
547+-- Fails if the stream ends before @m@ elements could be taken.
548+--
549+-- Examples: -
550+--
551+-- @
552+-- >>> :{
553+-- takeBetween' low high ls = Stream.parse prsr (Stream.fromList ls)
554+-- where prsr = Parser.takeBetween low high Fold.toList
555+-- :}
556+--
557+-- @
558+--
559+-- >>> takeBetween' 2 4 [1, 2, 3, 4, 5]
560+-- Right [1,2,3,4]
561+--
562+-- >>> takeBetween' 2 4 [1, 2]
563+-- Right [1,2]
564+--
565+-- >>> takeBetween' 2 4 [1]
566+-- Left (ParseError "takeBetween: Expecting alteast 2 elements, got 1")
567+--
568+-- >>> takeBetween' 0 0 [1, 2]
569+-- Right []
570+--
571+-- >>> takeBetween' 0 1 []
572+-- Right []
573+--
574+-- @takeBetween@ is the most general take operation, other take operations can
575+-- be defined in terms of takeBetween. For example:
576+--
577+-- >>> take n = Parser.takeBetween 0 n
578+-- >>> takeEQ n = Parser.takeBetween n n
579+-- >>> takeGE n = Parser.takeBetween n maxBound
580+--
581+-- /Pre-release/
582+--
583+{-# INLINE takeBetween #-}
584+takeBetween :: Monad m =>
585+ Int -> Int -> Fold m a b -> Parser a m b
586+takeBetween m n = D.toParserK . D.takeBetween m n
587+
588+-- | Stops after taking exactly @n@ input elements.
589+--
590+-- * Stops - after consuming @n@ elements.
591+-- * Fails - if the stream or the collecting fold ends before it can collect
592+-- exactly @n@ elements.
593+--
594+-- >>> Stream.parse (Parser.takeEQ 4 Fold.toList) $ Stream.fromList [1,0,1]
595+-- Left (ParseError "takeEQ: Expecting exactly 4 elements, input terminated on 3")
596+--
597+{-# INLINE takeEQ #-}
598+takeEQ :: Monad m => Int -> Fold m a b -> Parser a m b
599+takeEQ n = D.toParserK . D.takeEQ n
600+
601+-- | Take at least @n@ input elements, but can collect more.
602+--
603+-- * Stops - when the collecting fold stops.
604+-- * Fails - if the stream or the collecting fold ends before producing @n@
605+-- elements.
606+--
607+-- >>> Stream.parse (Parser.takeGE 4 Fold.toList) $ Stream.fromList [1,0,1]
608+-- Left (ParseError "takeGE: Expecting at least 4 elements, input terminated on 3")
609+--
610+-- >>> Stream.parse (Parser.takeGE 4 Fold.toList) $ Stream.fromList [1,0,1,0,1]
611+-- Right [1,0,1,0,1]
612+--
613+-- /Pre-release/
614+--
615+{-# INLINE takeGE #-}
616+takeGE :: Monad m => Int -> Fold m a b -> Parser a m b
617+takeGE n = D.toParserK . D.takeGE n
618+
619+-------------------------------------------------------------------------------
620+-- Take until a condition
621+-------------------------------------------------------------------------------
622+
623+-- $takeWhile
624+-- Note: This is called @takeWhileP@ and @munch@ in some parser libraries.
625+
626+-- XXX We should perhaps use only takeWhileP and rename it to takeWhile.
627+--
628+-- | Like 'takeWhile' but uses a 'Parser' instead of a 'Fold' to collect the
629+-- input. The combinator stops when the condition fails or if the collecting
630+-- parser stops.
631+--
632+-- Other interesting parsers can be implemented in terms of this parser:
633+--
634+-- >>> takeWhile1 cond p = Parser.takeWhileP cond (Parser.takeBetween 1 maxBound p)
635+-- >>> takeWhileBetween cond m n p = Parser.takeWhileP cond (Parser.takeBetween m n p)
636+--
637+-- Stops: when the condition fails or the collecting parser stops.
638+-- Fails: when the collecting parser fails.
639+--
640+-- /Pre-release/
641+--
642+{-# INLINE takeWhileP #-}
643+takeWhileP :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
644+takeWhileP cond p = D.toParserK $ D.takeWhileP cond (D.fromParserK p)
645+
646+-- | Collect stream elements until an element fails the predicate. The element
647+-- on which the predicate fails is returned back to the input stream.
648+--
649+-- * Stops - when the predicate fails or the collecting fold stops.
650+-- * Fails - never.
651+--
652+-- >>> Stream.parse (Parser.takeWhile (== 0) Fold.toList) $ Stream.fromList [0,0,1,0,1]
653+-- Right [0,0]
654+--
655+-- >>> takeWhile cond f = Parser.takeWhileP cond (Parser.fromFold f)
656+--
657+-- We can implement a @breakOn@ using 'takeWhile':
658+--
659+-- @
660+-- breakOn p = takeWhile (not p)
661+-- @
662+--
663+{-# INLINE takeWhile #-}
664+takeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
665+takeWhile cond = D.toParserK . D.takeWhile cond
666+-- takeWhile cond f = takeWhileP cond (fromFold f)
667+
668+-- | Like 'takeWhile' but takes at least one element otherwise fails.
669+--
670+-- >>> takeWhile1 cond p = Parser.takeWhileP cond (Parser.takeBetween 1 maxBound p)
671+--
672+{-# INLINE takeWhile1 #-}
673+takeWhile1 :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
674+takeWhile1 cond = D.toParserK . D.takeWhile1 cond
675+-- takeWhile1 cond f = takeWhileP cond (takeBetween 1 maxBound f)
676+
677+-- | Drain the input as long as the predicate succeeds, running the effects and
678+-- discarding the results.
679+--
680+-- This is also called @skipWhile@ in some parsing libraries.
681+--
682+-- >>> dropWhile p = Parser.takeWhile p Fold.drain
683+--
684+{-# INLINE dropWhile #-}
685+dropWhile :: Monad m => (a -> Bool) -> Parser a m ()
686+dropWhile p = takeWhile p FL.drain
687+
688+-------------------------------------------------------------------------------
689+-- Separators
690+-------------------------------------------------------------------------------
691+
692+-- XXX We can remove Maybe from esc
693+{-# INLINE takeFramedByGeneric #-}
694+takeFramedByGeneric :: Monad m =>
695+ Maybe (a -> Bool)
696+ -> Maybe (a -> Bool)
697+ -> Maybe (a -> Bool)
698+ -> Fold m a b
699+ -> Parser a m b
700+takeFramedByGeneric esc begin end f =
701+ D.toParserK $ D.takeFramedByGeneric esc begin end f
702+
703+-- | @takeEndBy cond parser@ parses a token that ends by a separator chosen by
704+-- the supplied predicate. The separator is also taken with the token.
705+--
706+-- This can be combined with other parsers to implement other interesting
707+-- parsers as follows:
708+--
709+-- >>> takeEndByLE cond n p = Parser.takeEndBy cond (Parser.fromFold $ Fold.take n p)
710+-- >>> takeEndByBetween cond m n p = Parser.takeEndBy cond (Parser.takeBetween m n p)
711+--
712+-- >>> takeEndBy = Parser.takeEndByEsc (const False)
713+--
714+-- See also "Streamly.Data.Fold.takeEndBy". Unlike the fold, the collecting
715+-- parser in the takeEndBy parser can decide whether to fail or not if the
716+-- stream does not end with separator.
717+--
718+-- /Pre-release/
719+--
720+{-# INLINE takeEndBy #-}
721+takeEndBy :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
722+takeEndBy cond = D.toParserK . D.takeEndBy cond . D.fromParserK
723+-- takeEndBy = takeEndByEsc (const False)
724+
725+-- | Like 'takeEndBy' but the separator is dropped.
726+--
727+-- See also "Streamly.Data.Fold.takeEndBy_".
728+--
729+-- /Pre-release/
730+--
731+{-# INLINE takeEndBy_ #-}
732+takeEndBy_ :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
733+takeEndBy_ cond = D.toParserK . D.takeEndBy_ cond . D.fromParserK
734+{-
735+takeEndBy_ isEnd p =
736+ takeFramedByGeneric Nothing Nothing (Just isEnd) (toFold p)
737+-}
738+
739+-- | Take either the separator or the token. Separator is a Left value and
740+-- token is Right value.
741+--
742+-- /Unimplemented/
743+{-# INLINE takeEitherSepBy #-}
744+takeEitherSepBy :: -- Monad m =>
745+ (a -> Bool) -> Fold m (Either a b) c -> Parser a m c
746+takeEitherSepBy _cond = undefined -- D.toParserK . D.takeEitherSepBy cond
747+
748+-- | Parse a token that starts with an element chosen by the predicate. The
749+-- parser fails if the input does not start with the selected element.
750+--
751+-- * Stops - when the predicate succeeds in non-leading position.
752+-- * Fails - when the predicate fails in the leading position.
753+--
754+-- >>> splitWithPrefix p f = Stream.parseMany (Parser.takeStartBy p f)
755+--
756+-- Examples: -
757+--
758+-- >>> p = Parser.takeStartBy (== ',') Fold.toList
759+-- >>> leadingComma = Stream.parse p . Stream.fromList
760+-- >>> leadingComma "a,b"
761+-- Left (ParseError "takeStartBy: missing frame start")
762+-- ...
763+-- >>> leadingComma ",,"
764+-- Right ","
765+-- >>> leadingComma ",a,b"
766+-- Right ",a"
767+-- >>> leadingComma ""
768+-- Right ""
769+--
770+-- /Pre-release/
771+--
772+{-# INLINE takeStartBy #-}
773+takeStartBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
774+takeStartBy cond = D.toParserK . D.takeStartBy cond
775+
776+-- | Like 'takeStartBy' but drops the separator.
777+--
778+-- >>> takeStartBy_ isBegin = Parser.takeFramedByGeneric Nothing (Just isBegin) Nothing
779+--
780+{-# INLINE takeStartBy_ #-}
781+takeStartBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
782+takeStartBy_ isBegin = takeFramedByGeneric Nothing (Just isBegin) Nothing
783+
784+-------------------------------------------------------------------------------
785+-- Quoting and Escaping
786+-------------------------------------------------------------------------------
787+
788+-- | Like 'takeEndBy' but the separator elements can be escaped using an
789+-- escape char determined by the first predicate. The escape characters are
790+-- removed.
791+--
792+-- /pre-release/
793+{-# INLINE takeEndByEsc #-}
794+takeEndByEsc :: Monad m =>
795+ (a -> Bool) -> (a -> Bool) -> Parser a m b -> Parser a m b
796+takeEndByEsc isEsc isEnd p =
797+ D.toParserK $ D.takeEndByEsc isEsc isEnd (D.fromParserK p)
798+
799+-- | @takeFramedByEsc_ isEsc isBegin isEnd fold@ parses a token framed using a
800+-- begin and end predicate, and an escape character. The frame begin and end
801+-- characters lose their special meaning if preceded by the escape character.
802+--
803+-- Nested frames are allowed if begin and end markers are different, nested
804+-- frames must be balanced unless escaped, nested frame markers are emitted as
805+-- it is.
806+--
807+-- For example,
808+--
809+-- >>> p = Parser.takeFramedByEsc_ (== '\\') (== '{') (== '}') Fold.toList
810+-- >>> Stream.parse p $ Stream.fromList "{hello}"
811+-- Right "hello"
812+-- >>> Stream.parse p $ Stream.fromList "{hello {world}}"
813+-- Right "hello {world}"
814+-- >>> Stream.parse p $ Stream.fromList "{hello \\{world}"
815+-- Right "hello {world"
816+-- >>> Stream.parse p $ Stream.fromList "{hello {world}"
817+-- Left (ParseError "takeFramedByEsc_: missing frame end")
818+--
819+-- /Pre-release/
820+{-# INLINE takeFramedByEsc_ #-}
821+takeFramedByEsc_ :: Monad m =>
822+ (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
823+takeFramedByEsc_ isEsc isBegin isEnd f =
824+ D.toParserK $ D.takeFramedByEsc_ isEsc isBegin isEnd f
825+-- takeEndByEsc_ isEsc isEnd p =
826+-- takeFramedByGeneric (Just isEsc) Nothing (Just isEnd) (toFold p)
827+
828+-- | @takeFramedBy_ isBegin isEnd fold@ parses a token framed by a begin and an
829+-- end predicate.
830+--
831+-- >>> takeFramedBy_ = Parser.takeFramedByEsc_ (const False)
832+--
833+{-# INLINE takeFramedBy_ #-}
834+takeFramedBy_ :: Monad m =>
835+ (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
836+takeFramedBy_ isBegin isEnd f = D.toParserK $ D.takeFramedBy_ isBegin isEnd f
837+-- takeFramedBy_ isBegin isEnd =
838+-- takeFramedByGeneric (Just (const False)) (Just isBegin) (Just isEnd)
839+
840+-------------------------------------------------------------------------------
841+-- Grouping and words
842+-------------------------------------------------------------------------------
843+
844+-- Note we can also get words using something like:
845+-- sepBy FL.toList (takeWhile (not . p) Fold.toList) (dropWhile p)
846+--
847+-- But that won't be as efficient and ergonomic.
848+--
849+-- | Like 'splitOn' but strips leading, trailing, and repeated separators.
850+-- Therefore, @".a..b."@ having '.' as the separator would be parsed as
851+-- @["a","b"]@. In other words, its like parsing words from whitespace
852+-- separated text.
853+--
854+-- * Stops - when it finds a word separator after a non-word element
855+-- * Fails - never.
856+--
857+-- >>> wordBy = Parser.wordFramedBy (const False) (const False) (const False)
858+--
859+-- @
860+-- S.wordsBy pred f = S.parseMany (PR.wordBy pred f)
861+-- @
862+--
863+{-# INLINE wordBy #-}
864+wordBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
865+wordBy f = D.toParserK . D.wordBy f
866+
867+-- | Like 'wordBy' but treats anything inside a pair of quotes as a single
868+-- word, the quotes can be escaped by an escape character. Recursive quotes
869+-- are possible if quote begin and end characters are different, quotes must be
870+-- balanced. Outermost quotes are stripped.
871+--
872+-- >>> braces = Parser.wordFramedBy (== '\\') (== '{') (== '}') isSpace Fold.toList
873+-- >>> Stream.parse braces $ Stream.fromList "{ab} cd"
874+-- Right "ab"
875+-- >>> Stream.parse braces $ Stream.fromList "{ab}{cd}"
876+-- Right "abcd"
877+-- >>> Stream.parse braces $ Stream.fromList "a{b} cd"
878+-- Right "ab"
879+-- >>> Stream.parse braces $ Stream.fromList "a{{b}} cd"
880+-- Right "a{b}"
881+--
882+-- >>> quotes = Parser.wordFramedBy (== '\\') (== '"') (== '"') isSpace Fold.toList
883+-- >>> Stream.parse quotes $ Stream.fromList "\"a\"\"b\""
884+-- Right "ab"
885+--
886+{-# INLINE wordFramedBy #-}
887+wordFramedBy :: Monad m =>
888+ (a -> Bool) -- ^ Escape
889+ -> (a -> Bool) -- ^ left quote
890+ -> (a -> Bool) -- ^ right quote
891+ -> (a -> Bool) -- ^ word seperator
892+ -> Fold m a b
893+ -> Parser a m b
894+wordFramedBy isEsc isBegin isEnd isSpc =
895+ D.toParserK . D.wordFramedBy isEsc isBegin isEnd isSpc
896+
897+-- | Like 'wordFramedBy' but the closing quote is determined by the opening
898+-- quote. The first quote begin starts a quote that is closed by its
899+-- corresponding closing quote.
900+--
901+-- 'wordFramedBy' and 'wordQuotedBy' both allow multiple quote characters based
902+-- on the predicates but 'wordQuotedBy' always fixes the quote at the first
903+-- occurrence and then it is closed only by the corresponding closing quote.
904+-- Therefore, other quoting characters can be embedded inside it as normal
905+-- characters. On the other hand, 'wordFramedBy' would close the quote as soon
906+-- as it encounters any of the closing quotes.
907+--
908+-- >>> q = (`elem` ['"', '\''])
909+-- >>> p kQ = Parser.wordQuotedBy kQ (== '\\') q q id isSpace Fold.toList
910+--
911+-- >>> Stream.parse (p False) $ Stream.fromList "a\"b'c\";'d\"e'f ghi"
912+-- Right "ab'c;d\"ef"
913+--
914+-- >>> Stream.parse (p True) $ Stream.fromList "a\"b'c\";'d\"e'f ghi"
915+-- Right "a\"b'c\";'d\"e'f"
916+--
917+{-# INLINE wordQuotedBy #-}
918+wordQuotedBy :: (Monad m, Eq a) =>
919+ Bool -- ^ keep the quotes in the output
920+ -> (a -> Bool) -- ^ Escape
921+ -> (a -> Bool) -- ^ left quote
922+ -> (a -> Bool) -- ^ right quote
923+ -> (a -> a) -- ^ get right quote from left quote
924+ -> (a -> Bool) -- ^ word seperator
925+ -> Fold m a b
926+ -> Parser a m b
927+wordQuotedBy keepQuotes isEsc isBegin isEnd toRight isSpc =
928+ D.toParserK . D.wordQuotedBy keepQuotes isEsc isBegin isEnd toRight isSpc
929+
930+-- | Given an input stream @[a,b,c,...]@ and a comparison function @cmp@, the
931+-- parser assigns the element @a@ to the first group, then if @a \`cmp` b@ is
932+-- 'True' @b@ is also assigned to the same group. If @a \`cmp` c@ is 'True'
933+-- then @c@ is also assigned to the same group and so on. When the comparison
934+-- fails the parser is terminated. Each group is folded using the 'Fold' @f@ and
935+-- the result of the fold is the result of the parser.
936+--
937+-- * Stops - when the comparison fails.
938+-- * Fails - never.
939+--
940+-- >>> :{
941+-- runGroupsBy eq =
942+-- Stream.fold Fold.toList
943+-- . Stream.parseMany (Parser.groupBy eq Fold.toList)
944+-- . Stream.fromList
945+-- :}
946+--
947+-- >>> runGroupsBy (<) []
948+-- []
949+--
950+-- >>> runGroupsBy (<) [1]
951+-- [Right [1]]
952+--
953+-- >>> runGroupsBy (<) [3, 5, 4, 1, 2, 0]
954+-- [Right [3,5,4],Right [1,2],Right [0]]
955+--
956+{-# INLINE groupBy #-}
957+groupBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
958+groupBy eq = D.toParserK . D.groupBy eq
959+
960+-- | Unlike 'groupBy' this combinator performs a rolling comparison of two
961+-- successive elements in the input stream. Assuming the input stream
962+-- is @[a,b,c,...]@ and the comparison function is @cmp@, the parser
963+-- first assigns the element @a@ to the first group, then if @a \`cmp` b@ is
964+-- 'True' @b@ is also assigned to the same group. If @b \`cmp` c@ is 'True'
965+-- then @c@ is also assigned to the same group and so on. When the comparison
966+-- fails the parser is terminated. Each group is folded using the 'Fold' @f@ and
967+-- the result of the fold is the result of the parser.
968+--
969+-- * Stops - when the comparison fails.
970+-- * Fails - never.
971+--
972+-- >>> :{
973+-- runGroupsByRolling eq =
974+-- Stream.fold Fold.toList
975+-- . Stream.parseMany (Parser.groupByRolling eq Fold.toList)
976+-- . Stream.fromList
977+-- :}
978+--
979+-- >>> runGroupsByRolling (<) []
980+-- []
981+--
982+-- >>> runGroupsByRolling (<) [1]
983+-- [Right [1]]
984+--
985+-- >>> runGroupsByRolling (<) [3, 5, 4, 1, 2, 0]
986+-- [Right [3,5],Right [4],Right [1,2],Right [0]]
987+--
988+-- /Pre-release/
989+--
990+{-# INLINE groupByRolling #-}
991+groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
992+groupByRolling eq = D.toParserK . D.groupByRolling eq
993+
994+-- | Like 'groupByRolling', but if the predicate is 'True' then collects using
995+-- the first fold as long as the predicate holds 'True', if the predicate is
996+-- 'False' collects using the second fold as long as it remains 'False'.
997+-- Returns 'Left' for the first case and 'Right' for the second case.
998+--
999+-- For example, if we want to detect sorted sequences in a stream, both
1000+-- ascending and descending cases we can use 'groupByRollingEither (<=)
1001+-- Fold.toList Fold.toList'.
1002+--
1003+-- /Pre-release/
1004+{-# INLINE groupByRollingEither #-}
1005+groupByRollingEither :: Monad m =>
1006+ (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
1007+groupByRollingEither eq f1 = D.toParserK . D.groupByRollingEither eq f1
1008+
1009+-- | Like 'listEqBy' but uses a stream instead of a list and does not return
1010+-- the stream.
1011+--
1012+-- See also: "Streamly.Data.Stream.streamEqBy"
1013+--
1014+{-# INLINE streamEqBy #-}
1015+streamEqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Parser a m ()
1016+streamEqBy cmp = D.toParserK . D.streamEqBy cmp
1017+
1018+-- | Match the given sequence of elements using the given comparison function.
1019+-- Returns the original sequence if successful.
1020+--
1021+-- Definition:
1022+--
1023+-- >>> listEqBy cmp xs = Parser.streamEqBy cmp (Stream.fromList xs) *> Parser.fromPure xs
1024+--
1025+-- Examples:
1026+--
1027+-- >>> Stream.parse (Parser.listEqBy (==) "string") $ Stream.fromList "string"
1028+-- Right "string"
1029+--
1030+-- >>> Stream.parse (Parser.listEqBy (==) "mismatch") $ Stream.fromList "match"
1031+-- Left (ParseError "streamEqBy: mismtach occurred")
1032+--
1033+{-# INLINE listEqBy #-}
1034+listEqBy :: Monad m => (a -> a -> Bool) -> [a] -> Parser a m [a]
1035+-- listEqBy cmp xs = D.toParserK (D.listEqBy cmp xs)
1036+listEqBy cmp xs = streamEqBy cmp (Stream.fromList xs) *> fromPure xs
1037+
1038+-- Rename to "list".
1039+-- | Match the input sequence with the supplied list and return it if
1040+-- successful.
1041+--
1042+-- >>> listEq = Parser.listEqBy (==)
1043+--
1044+{-# INLINE listEq #-}
1045+listEq :: (Monad m, Eq a) => [a] -> Parser a m [a]
1046+listEq = listEqBy (==)
1047+
1048+-- | Match if the input stream is a subsequence of the argument stream i.e. all
1049+-- the elements of the input stream occur, in order, in the argument stream.
1050+-- The elements do not have to occur consecutively. A sequence is considered a
1051+-- subsequence of itself.
1052+{-# INLINE subsequenceBy #-}
1053+subsequenceBy :: -- Monad m =>
1054+ (a -> a -> Bool) -> Stream m a -> Parser a m ()
1055+subsequenceBy = undefined
1056+
1057+{-
1058+-- Should go in Data.Parser.Regex in streamly package so that it can depend on
1059+-- regex backends.
1060+{-# INLINE regexPosix #-}
1061+regexPosix :: -- Monad m =>
1062+ Regex -> Parser m a (Maybe (Array (MatchOffset, MatchLength)))
1063+regexPosix = undefined
1064+
1065+{-# INLINE regexPCRE #-}
1066+regexPCRE :: -- Monad m =>
1067+ Regex -> Parser m a (Maybe (Array (MatchOffset, MatchLength)))
1068+regexPCRE = undefined
1069+-}
1070+
1071+-------------------------------------------------------------------------------
1072+-- nested parsers
1073+-------------------------------------------------------------------------------
1074+
1075+-- | Sequential parser application. Apply two parsers sequentially to an input
1076+-- stream. The input is provided to the first parser, when it is done the
1077+-- remaining input is provided to the second parser. If both the parsers
1078+-- succeed their outputs are combined using the supplied function. The
1079+-- operation fails if any of the parsers fail.
1080+--
1081+-- Note: This is a parsing dual of appending streams using
1082+-- 'Streamly.Data.Stream.append', it splits the streams using two parsers and zips
1083+-- the results.
1084+--
1085+-- This implementation is strict in the second argument, therefore, the
1086+-- following will fail:
1087+--
1088+-- >>> Stream.parse (Parser.splitWith const (Parser.satisfy (> 0)) undefined) $ Stream.fromList [1]
1089+-- *** Exception: Prelude.undefined
1090+-- ...
1091+--
1092+-- Compare with 'Applicative' instance method '<*>'. This implementation allows
1093+-- stream fusion but has quadratic complexity. This can fuse with other
1094+-- operations and can be faster than 'Applicative' instance for small number
1095+-- (less than 8) of compositions.
1096+--
1097+-- Many combinators can be expressed using @splitWith@ and other parser
1098+-- primitives. Some common idioms are described below,
1099+--
1100+-- @
1101+-- span :: (a -> Bool) -> Fold m a b -> Fold m a b -> Parser a m b
1102+-- span pred f1 f2 = splitWith (,) ('takeWhile' pred f1) ('fromFold' f2)
1103+-- @
1104+--
1105+-- @
1106+-- spanBy :: (a -> a -> Bool) -> Fold m a b -> Fold m a b -> Parser a m b
1107+-- spanBy eq f1 f2 = splitWith (,) ('groupBy' eq f1) ('fromFold' f2)
1108+-- @
1109+--
1110+-- @
1111+-- spanByRolling :: (a -> a -> Bool) -> Fold m a b -> Fold m a b -> Parser a m b
1112+-- spanByRolling eq f1 f2 = splitWith (,) ('groupByRolling' eq f1) ('fromFold' f2)
1113+-- @
1114+--
1115+-- /Pre-release/
1116+--
1117+{-# INLINE splitWith #-}
1118+splitWith :: Monad m
1119+ => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
1120+splitWith f p1 p2 =
1121+ D.toParserK $ D.splitWith f (D.fromParserK p1) (D.fromParserK p2)
1122+
1123+-- | Sequential parser application ignoring the output of the first parser.
1124+-- Apply two parsers sequentially to an input stream. The input is provided to
1125+-- the first parser, when it is done the remaining input is provided to the
1126+-- second parser. The output of the parser is the output of the second parser.
1127+-- The operation fails if any of the parsers fail.
1128+--
1129+-- This implementation is strict in the second argument, therefore, the
1130+-- following will fail:
1131+--
1132+-- >>> Stream.parse (Parser.split_ (Parser.satisfy (> 0)) undefined) $ Stream.fromList [1]
1133+-- *** Exception: Prelude.undefined
1134+-- ...
1135+--
1136+-- Compare with 'Applicative' instance method '*>'. This implementation allows
1137+-- stream fusion but has quadratic complexity. This can fuse with other
1138+-- operations, and can be faster than 'Applicative' instance for small
1139+-- number (less than 8) of compositions.
1140+--
1141+-- /Pre-release/
1142+--
1143+{-# INLINE split_ #-}
1144+split_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
1145+split_ p1 p2 = D.toParserK $ D.split_ (D.fromParserK p1) (D.fromParserK p2)
1146+
1147+{-
1148+-- | @teeWith f p1 p2@ distributes its input to both @p1@ and @p2@ until both
1149+-- of them succeed or anyone of them fails and combines their output using @f@.
1150+-- The parser succeeds if both the parsers succeed.
1151+--
1152+-- /Pre-release/
1153+--
1154+{-# INLINE teeWith #-}
1155+teeWith :: Monad m
1156+ => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
1157+teeWith f p1 p2 =
1158+ D.toParserK $ D.teeWith f (D.fromParserK p1) (D.fromParserK p2)
1159+
1160+-- | Like 'teeWith' but ends parsing and zips the results, if available,
1161+-- whenever the first parser ends.
1162+--
1163+-- /Pre-release/
1164+--
1165+{-# INLINE teeWithFst #-}
1166+teeWithFst :: Monad m
1167+ => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
1168+teeWithFst f p1 p2 =
1169+ D.toParserK $ D.teeWithFst f (D.fromParserK p1) (D.fromParserK p2)
1170+
1171+-- | Like 'teeWith' but ends parsing and zips the results, if available,
1172+-- whenever any of the parsers ends or fails.
1173+--
1174+-- /Unimplemented/
1175+--
1176+{-# INLINE teeWithMin #-}
1177+teeWithMin :: Monad m
1178+ => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
1179+teeWithMin f p1 p2 =
1180+ D.toParserK $ D.teeWithMin f (D.fromParserK p1) (D.fromParserK p2)
1181+-}
1182+
1183+-- | Sequential alternative. Apply the input to the first parser and return the
1184+-- result if the parser succeeds. If the first parser fails then backtrack and
1185+-- apply the same input to the second parser and return the result.
1186+--
1187+-- Note: This implementation is not lazy in the second argument. The following
1188+-- will fail:
1189+--
1190+-- >>> Stream.parse (Parser.satisfy (> 0) `Parser.alt` undefined) $ Stream.fromList [1..10]
1191+-- Right 1
1192+--
1193+-- Compare with 'Alternative' instance method '<|>'. This implementation allows
1194+-- stream fusion but has quadratic complexity. This can fuse with other
1195+-- operations and can be much faster than 'Alternative' instance for small
1196+-- number (less than 8) of alternatives.
1197+--
1198+-- /Pre-release/
1199+--
1200+{-# INLINE alt #-}
1201+alt :: Monad m => Parser x m a -> Parser x m a -> Parser x m a
1202+alt p1 p2 = D.toParserK $ D.alt (D.fromParserK p1) (D.fromParserK p2)
1203+
1204+{-
1205+-- | Shortest alternative. Apply both parsers in parallel but choose the result
1206+-- from the one which consumed least input i.e. take the shortest succeeding
1207+-- parse.
1208+--
1209+-- /Pre-release/
1210+--
1211+{-# INLINE shortest #-}
1212+shortest :: Monad m
1213+ => Parser x m a -> Parser x m a -> Parser x m a
1214+shortest p1 p2 = D.toParserK $ D.shortest (D.fromParserK p1) (D.fromParserK p2)
1215+
1216+-- | Longest alternative. Apply both parsers in parallel but choose the result
1217+-- from the one which consumed more input i.e. take the longest succeeding
1218+-- parse.
1219+--
1220+-- /Pre-release/
1221+--
1222+{-# INLINE longest #-}
1223+longest :: Monad m
1224+ => Parser x m a -> Parser x m a -> Parser x m a
1225+longest p1 p2 = D.toParserK $ D.longest (D.fromParserK p1) (D.fromParserK p2)
1226+-}
1227+
1228+-- | Run a parser without consuming the input.
1229+--
1230+{-# INLINE lookAhead #-}
1231+lookAhead :: Monad m => Parser a m b -> Parser a m b
1232+lookAhead p = D.toParserK $ D.lookAhead $ D.fromParserK p
1233+
1234+-- | Takes at-most @n@ input elements.
1235+--
1236+-- * Stops - when the collecting parser stops.
1237+-- * Fails - when the collecting parser fails.
1238+--
1239+-- >>> Stream.parse (Parser.takeP 4 (Parser.takeEQ 2 Fold.toList)) $ Stream.fromList [1, 2, 3, 4, 5]
1240+-- Right [1,2]
1241+--
1242+-- >>> Stream.parse (Parser.takeP 4 (Parser.takeEQ 5 Fold.toList)) $ Stream.fromList [1, 2, 3, 4, 5]
1243+-- Left (ParseError "takeEQ: Expecting exactly 5 elements, input terminated on 4")
1244+--
1245+-- /Internal/
1246+{-# INLINE takeP #-}
1247+takeP :: Monad m => Int -> Parser a m b -> Parser a m b
1248+takeP i p = D.toParserK $ D.takeP i $ D.fromParserK p
1249+
1250+-------------------------------------------------------------------------------
1251+-- Sequential Collection
1252+-------------------------------------------------------------------------------
1253+--
1254+-- | @concatSequence f p@ collects sequential parses of parsers in a
1255+-- serial stream @p@ using the fold @f@. Fails if the input ends or any
1256+-- of the parsers fail.
1257+--
1258+-- An even more efficient implementation can use ParserD type Parser in
1259+-- the stream.
1260+--
1261+-- /Pre-release/
1262+--
1263+{-# INLINE concatSequence #-}
1264+concatSequence ::
1265+ Monad m =>
1266+ Fold m b c -> Stream m (Parser a m b) -> Parser a m c
1267+concatSequence f p =
1268+ let sp = fmap D.fromParserK p
1269+ in D.toParserK $ D.sequence sp f
1270+
1271+-- | Map a 'Parser' returning function on the result of a 'Parser'.
1272+--
1273+-- Compare with 'Monad' instance method '>>='. This implementation allows
1274+-- stream fusion but has quadratic complexity. This can fuse with other
1275+-- operations and can be much faster than 'Monad' instance for small number
1276+-- (less than 8) of compositions.
1277+--
1278+-- /Pre-release/
1279+--
1280+{-# INLINE concatMap #-}
1281+concatMap :: Monad m
1282+ => (b -> Parser a m c) -> Parser a m b -> Parser a m c
1283+concatMap f p = D.toParserK $ D.concatMap (D.fromParserK . f) (D.fromParserK p)
1284+
1285+{-
1286+-------------------------------------------------------------------------------
1287+-- Alternative Collection
1288+-------------------------------------------------------------------------------
1289+--
1290+-- | @choice parsers@ applies the @parsers@ in order and returns the first
1291+-- successful parse.
1292+--
1293+-- This is same as 'asum' but more efficient.
1294+--
1295+-- /Broken/
1296+--
1297+{-# INLINE choice #-}
1298+choice ::
1299+ (Functor t, Foldable t, Monad m) => t (Parser a m b) -> Parser a m b
1300+choice ps = D.toParserK $ D.choice $ D.fromParserK <$> ps
1301+-}
1302+
1303+-------------------------------------------------------------------------------
1304+-- Sequential Repetition
1305+-------------------------------------------------------------------------------
1306+
1307+-- | Like 'many' but uses a 'Parser' instead of a 'Fold' to collect the
1308+-- results. Parsing stops or fails if the collecting parser stops or fails.
1309+--
1310+-- /Unimplemented/
1311+--
1312+{-# INLINE manyP #-}
1313+manyP :: -- Monad m =>
1314+ Parser a m b -> Parser b m c -> Parser a m c
1315+manyP _p _f = undefined -- D.toParserK $ D.manyP (D.fromParserK p) f
1316+
1317+-- | Collect zero or more parses. Apply the supplied parser repeatedly on the
1318+-- input stream and push the parse results to a downstream fold.
1319+--
1320+-- Stops: when the downstream fold stops or the parser fails.
1321+-- Fails: never, produces zero or more results.
1322+--
1323+-- >>> many = Parser.countBetween 0 maxBound
1324+--
1325+-- Compare with 'Control.Applicative.many'.
1326+--
1327+{-# INLINE many #-}
1328+many :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
1329+many p f = D.toParserK $ D.many (D.fromParserK p) f
1330+
1331+-- Note: many1 would perhaps be a better name for this and consistent with
1332+-- other names like takeWhile1. But we retain the name "some" for
1333+-- compatibility.
1334+--
1335+-- | Collect one or more parses. Apply the supplied parser repeatedly on the
1336+-- input stream and push the parse results to a downstream fold.
1337+--
1338+-- Stops: when the downstream fold stops or the parser fails.
1339+-- Fails: if it stops without producing a single result.
1340+--
1341+-- >>> some p f = Parser.manyP p (Parser.takeGE 1 f)
1342+-- >>> some = Parser.countBetween 1 maxBound
1343+--
1344+-- Compare with 'Control.Applicative.some'.
1345+--
1346+{-# INLINE some #-}
1347+some :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
1348+some p f = D.toParserK $ D.some (D.fromParserK p) f
1349+-- some p f = manyP p (takeGE 1 f)
1350+-- some = countBetween 1 maxBound
1351+
1352+-- | @countBetween m n f p@ collects between @m@ and @n@ sequential parses of
1353+-- parser @p@ using the fold @f@. Stop after collecting @n@ results. Fails if
1354+-- the input ends or the parser fails before @m@ results are collected.
1355+--
1356+-- >>> countBetween m n p f = Parser.manyP p (Parser.takeBetween m n f)
1357+--
1358+-- /Unimplemented/
1359+--
1360+{-# INLINE countBetween #-}
1361+countBetween :: -- Monad m =>
1362+ Int -> Int -> Parser a m b -> Fold m b c -> Parser a m c
1363+countBetween _m _n _p = undefined
1364+-- countBetween m n p f = manyP p (takeBetween m n f)
1365+
1366+-- | @count n f p@ collects exactly @n@ sequential parses of parser @p@ using
1367+-- the fold @f@. Fails if the input ends or the parser fails before @n@
1368+-- results are collected.
1369+--
1370+-- >>> count n = Parser.countBetween n n
1371+-- >>> count n p f = Parser.manyP p (Parser.takeEQ n f)
1372+--
1373+-- /Unimplemented/
1374+--
1375+{-# INLINE count #-}
1376+count :: -- Monad m =>
1377+ Int -> Parser a m b -> Fold m b c -> Parser a m c
1378+count n = countBetween n n
1379+-- count n p f = manyP p (takeEQ n f)
1380+
1381+-- | Like 'manyTill' but uses a 'Parser' to collect the results instead of a
1382+-- 'Fold'. Parsing stops or fails if the collecting parser stops or fails.
1383+--
1384+-- We can implemnent parsers like the following using 'manyTillP':
1385+--
1386+-- @
1387+-- countBetweenTill m n f p = manyTillP (takeBetween m n f) p
1388+-- @
1389+--
1390+-- /Unimplemented/
1391+--
1392+{-# INLINE manyTillP #-}
1393+manyTillP :: -- Monad m =>
1394+ Parser a m b -> Parser a m x -> Parser b m c -> Parser a m c
1395+manyTillP _p1 _p2 _f = undefined
1396+ -- D.toParserK $ D.manyTillP (D.fromParserK p1) (D.fromParserK p2) f
1397+
1398+-- | @manyTill chunking test f@ tries the parser @test@ on the input, if @test@
1399+-- fails it backtracks and tries @chunking@, after @chunking@ succeeds @test@ is
1400+-- tried again and so on. The parser stops when @test@ succeeds. The output of
1401+-- @test@ is discarded and the output of @chunking@ is accumulated by the
1402+-- supplied fold. The parser fails if @chunking@ fails.
1403+--
1404+-- Stops when the fold @f@ stops.
1405+--
1406+{-# INLINE manyTill #-}
1407+manyTill :: Monad m
1408+ => Parser a m b -- ^ Chunking parser. Parses chunks of input.
1409+ -> Parser a m x -- ^ Test parser. Parsing stops when this parser succeeds
1410+ -- else backtract and run the chunking parser.
1411+ -> Fold m b c -- ^ Folds the output of the chunking parser.
1412+ -> Parser a m c
1413+manyTill collect test f =
1414+ D.toParserK $ D.manyTill (D.fromParserK collect) (D.fromParserK test) f
1415+
1416+-- | @manyThen f collect recover@ repeats the parser @collect@ on the input and
1417+-- collects the output in the supplied fold. If the the parser @collect@ fails,
1418+-- parser @recover@ is run until it stops and then we start repeating the
1419+-- parser @collect@ again. The parser fails if the recovery parser fails.
1420+--
1421+-- For example, this can be used to find a key frame in a video stream after an
1422+-- error.
1423+--
1424+-- /Unimplemented/
1425+--
1426+{-# INLINE manyThen #-}
1427+manyThen :: -- (Foldable t, Monad m) =>
1428+ Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
1429+manyThen _parser _recover _f = undefined
1430+
1431+-------------------------------------------------------------------------------
1432+-- Interleaving
1433+-------------------------------------------------------------------------------
1434+--
1435+-- To deinterleave we can chain two parsers one behind the other. The input is
1436+-- given to the first parser and the input definitively rejected by the first
1437+-- parser is given to the second parser.
1438+--
1439+-- We can either have the parsers themselves buffer the input or use the shared
1440+-- global buffer to hold it until none of the parsers need it. When the first
1441+-- parser returns Skip (i.e. rewind) we let the second parser consume the
1442+-- rejected input and when it is done we move the cursor forward to the first
1443+-- parser again. This will require a "move forward" command as well.
1444+--
1445+-- To implement grep we can use three parsers, one to find the pattern, one
1446+-- to store the context behind the pattern and one to store the context in
1447+-- front of the pattern. When a match occurs we need to emit the accumulator of
1448+-- all the three parsers. One parser can count the line numbers to provide the
1449+-- line number info.
1450+
1451+-- XXX rename this to intercalate
1452+-- | Apply two parsers alternately to an input stream. The input stream is
1453+-- considered an interleaving of two patterns. The two parsers represent the
1454+-- two patterns.
1455+--
1456+{-# INLINE deintercalate #-}
1457+deintercalate :: Monad m =>
1458+ Parser a m x
1459+ -> Parser a m y
1460+ -> Fold m (Either x y) z
1461+ -> Parser a m z
1462+deintercalate contentL contentR sink =
1463+ D.toParserK
1464+ $ D.deintercalate
1465+ (D.fromParserK contentL) (D.fromParserK contentR) sink
1466+
1467+-- | Parse items separated by a separator parsed by the supplied parser. At
1468+-- least one item must be present for the parser to succeed.
1469+--
1470+-- Note that this can go in infinite loop if both the parsers fail on some
1471+-- input. Detection of that would make the implementation more complex.
1472+--
1473+{-# INLINE sepBy1 #-}
1474+sepBy1 :: Monad m =>
1475+ Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
1476+sepBy1 p sep sink = do
1477+ x <- p
1478+ f <- fromEffect $ FL.reduce sink
1479+ f1 <- fromEffect $ FL.snoc f x
1480+ many (sep >> p) f1
1481+
1482+-- | Run the content parser first, when it is done, the separator parser is
1483+-- run, when it is done content parser is run again and so on. If none of the
1484+-- parsers consumes an input then parser returns a failure.
1485+--
1486+-- >>> sepBy p1 p2 sink = Parser.deintercalate p1 p2 (Fold.catLefts sink)
1487+-- >>> sepBy content sep sink = Parser.sepBy1 content sep sink <|> return mempty
1488+--
1489+{-# INLINE sepBy #-}
1490+sepBy :: Monad m =>
1491+ Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
1492+sepBy content sep sink =
1493+ D.toParserK $ D.sepBy (D.fromParserK content) (D.fromParserK sep) sink
1494+-- sepBy sink = deintercalate (FL.catLefts sink)
1495+
1496+-------------------------------------------------------------------------------
1497+-- Interleaving a collection of parsers
1498+-------------------------------------------------------------------------------
1499+--
1500+-- | Apply a collection of parsers to an input stream in a round robin fashion.
1501+-- Each parser is applied until it stops and then we repeat starting with the
1502+-- the first parser again.
1503+--
1504+-- /Unimplemented/
1505+--
1506+{-# INLINE roundRobin #-}
1507+roundRobin :: -- (Foldable t, Monad m) =>
1508+ t (Parser a m b) -> Fold m b c -> Parser a m c
1509+roundRobin _ps _f = undefined
1510+
1511+-------------------------------------------------------------------------------
1512+-- Repeated Alternatives
1513+-------------------------------------------------------------------------------
1514+
1515+-- | Keep trying a parser up to a maximum of @n@ failures. When the parser
1516+-- fails the input consumed till now is dropped and the new instance is tried
1517+-- on the fresh input.
1518+--
1519+-- /Unimplemented/
1520+--
1521+{-# INLINE retryMaxTotal #-}
1522+retryMaxTotal :: -- (Monad m) =>
1523+ Int -> Parser a m b -> Fold m b c -> Parser a m c
1524+retryMaxTotal _n _p _f = undefined
1525+
1526+-- | Like 'retryMaxTotal' but aborts after @n@ successive failures.
1527+--
1528+-- /Unimplemented/
1529+--
1530+{-# INLINE retryMaxSuccessive #-}
1531+retryMaxSuccessive :: -- (Monad m) =>
1532+ Int -> Parser a m b -> Fold m b c -> Parser a m c
1533+retryMaxSuccessive _n _p _f = undefined
1534+
1535+-- | Keep trying a parser until it succeeds. When the parser fails the input
1536+-- consumed till now is dropped and the new instance is tried on the fresh
1537+-- input.
1538+--
1539+-- /Unimplemented/
1540+--
1541+{-# INLINE retry #-}
1542+retry :: -- (Monad m) =>
1543+ Parser a m b -> Parser a m b
1544+retry _p = undefined
core/src/Streamly/Internal/Data/ParserK/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserK/Type.hsrenamed+83−288
@@ -21,15 +21,13 @@
2121-- <http://hackage.haskell.org/package/parser-combinators parser-combinators>
2222-- package can also be used with the 'ParserK' type.
2323
24−module Streamly.Internal.Data.ParserK.Type
24+module Streamly.Internal.Data.Parser.ParserK.Type
2525 (
2626 Step (..)
2727 , Input (..)
2828 , ParseResult (..)
2929 , ParserK (..)
30− , adaptC
31− , adapt
32− , adaptCG
30+ , fromParser
3331 -- , toParser
3432 , fromPure
3533 , fromEffect
@@ -41,10 +39,7 @@ where
4139#include "assert.hs"
4240#include "inline.hs"
4341
44−#if !MIN_VERSION_base(4,18,0)
45−import Control.Applicative (liftA2)
46−#endif
47−import Control.Applicative (Alternative(..))
42+import Control.Applicative (Alternative(..), liftA2)
4843import Control.Monad (MonadPlus(..), ap)
4944import Control.Monad.IO.Class (MonadIO, liftIO)
5045-- import Control.Monad.Trans.Class (MonadTrans(lift))
@@ -56,24 +51,12 @@ import Streamly.Internal.System.IO (unsafeInlineIO)
5651
5752import qualified Control.Monad.Fail as Fail
5853import qualified Streamly.Internal.Data.Array.Type as Array
59−import qualified Streamly.Internal.Data.MutArray.Generic as GenArr
60− ( getIndexUnsafeWith
61− )
62−import qualified Streamly.Internal.Data.Array.Generic as GenArr
63−import qualified Streamly.Internal.Data.Parser.Type as ParserD
54+import qualified Streamly.Internal.Data.Parser.ParserD.Type as ParserD
6455
6556-- Note: We cannot use an Array directly as input because we need to identify
6657-- the end of input case using None. We cannot do that using nil Array as nil
6758-- Arrays can be encountered in normal input as well.
68−--
69−-- We could specialize the ParserK type to use an Array directly, that provides
70−-- some performance improvement. The best advantage of that is when we consume
71−-- one element at a time from the array. If we really want that perf
72−-- improvement we can use a special ParserK type with the following Input.
73−--
74−-- data Input a = None | Chunk {-# UNPACK #-} !(Array a)
75−--
76−data Input a = None | Chunk a
59+data Input a = None | Chunk {-# UNPACK #-} !(Array a)
7760
7861-- | The intermediate result of running a parser step. The parser driver may
7962-- stop with a final result, pause with a continuation to resume, or fail with
@@ -89,6 +72,9 @@ data Step a m r =
8972 -- The Int is the current stream position index wrt to the start of the
9073 -- array.
9174 Done !Int r
75+ -- XXX we can use a "resume" and a "stop" continuations instead of Maybe.
76+ -- measure if that works any better.
77+ -- Array a -> m (Step a m r), m (Step a m r)
9278 | Partial !Int (Input a -> m (Step a m r))
9379 | Continue !Int (Input a -> m (Step a m r))
9480 | Error !Int String
@@ -168,8 +154,8 @@ newtype ParserK a m b = MkParser
168154-------------------------------------------------------------------------------
169155
170156-- XXX rewrite this using ParserD, expose rmapM from ParserD.
171−
172−-- | Map a function on the result i.e. on @b@ in @Parser a m b@.
157+-- | Maps a function over the output of the parser.
158+--
173159instance Functor m => Functor (ParserK a m) where
174160 {-# INLINE fmap #-}
175161 fmap f parser = MkParser $ \k n st arr ->
@@ -181,7 +167,7 @@ instance Functor m => Functor (ParserK a m) where
181167-------------------------------------------------------------------------------
182168
183169-- This is the dual of stream "fromPure".
184−
170+--
185171-- | A parser that always yields a pure value without consuming any input.
186172--
187173-- /Pre-release/
@@ -199,11 +185,9 @@ fromEffect :: Monad m => m b -> ParserK a m b
199185fromEffect eff =
200186 MkParser $ \k n st arr -> eff >>= \b -> k (Success n b) st arr
201187
202−-- | @f \<$> p1 \<*> p2@ applies parsers p1 and p2 sequentially to an input
203−-- stream. The first parser runs and processes the input, the remaining input
204−-- is then passed to the second parser. If both parsers succeed, their outputs
205−-- are applied to the function @f@. If either parser fails, the operation
206−-- fails.
188+-- | 'Applicative' form of 'Streamly.Internal.Data.Parser.splitWith'. Note that
189+-- this operation does not fuse, use 'Streamly.Internal.Data.Parser.splitWith'
190+-- when fusion is important.
207191--
208192instance Monad m => Applicative (ParserK a m) where
209193 {-# INLINE pure #-}
@@ -245,8 +229,48 @@ instance Monad m => Applicative (ParserK a m) where
245229die :: String -> ParserK a m b
246230die err = MkParser (\k n st arr -> k (Failure n err) st arr)
247231
248−-- | Monad composition can be used for lookbehind parsers, we can dynamically
249−-- compose new parsers based on the results of the previously parsed values.
232+-- | Monad composition can be used for lookbehind parsers, we can make the
233+-- future parses depend on the previously parsed values.
234+--
235+-- If we have to parse "a9" or "9a" but not "99" or "aa" we can use the
236+-- following parser:
237+--
238+-- @
239+-- backtracking :: MonadCatch m => PR.Parser Char m String
240+-- backtracking =
241+-- sequence [PR.satisfy isDigit, PR.satisfy isAlpha]
242+-- '<|>'
243+-- sequence [PR.satisfy isAlpha, PR.satisfy isDigit]
244+-- @
245+--
246+-- We know that if the first parse resulted in a digit at the first place then
247+-- the second parse is going to fail. However, we waste that information and
248+-- parse the first character again in the second parse only to know that it is
249+-- not an alphabetic char. By using lookbehind in a 'Monad' composition we can
250+-- avoid redundant work:
251+--
252+-- @
253+-- data DigitOrAlpha = Digit Char | Alpha Char
254+--
255+-- lookbehind :: MonadCatch m => PR.Parser Char m String
256+-- lookbehind = do
257+-- x1 \<- Digit '<$>' PR.satisfy isDigit
258+-- '<|>' Alpha '<$>' PR.satisfy isAlpha
259+--
260+-- -- Note: the parse depends on what we parsed already
261+-- x2 <- case x1 of
262+-- Digit _ -> PR.satisfy isAlpha
263+-- Alpha _ -> PR.satisfy isDigit
264+--
265+-- return $ case x1 of
266+-- Digit x -> [x,x2]
267+-- Alpha x -> [x,x2]
268+-- @
269+--
270+-- See also 'Streamly.Internal.Data.Parser.concatMap'. This monad instance
271+-- does not fuse, use 'Streamly.Internal.Data.Parser.concatMap' when you need
272+-- fusion.
273+--
250274instance Monad m => Monad (ParserK a m) where
251275 {-# INLINE return #-}
252276 return = pure
@@ -278,10 +302,18 @@ instance MonadIO m => MonadIO (ParserK a m) where
278302-- Alternative
279303-------------------------------------------------------------------------------
280304
281−-- | @p1 \<|> p2@ passes the input to parser p1, if it succeeds, the result is
282−-- returned. However, if p1 fails, the parser driver backtracks and tries the
283−-- same input on the alternative parser p2, returning the result if it
284−-- succeeds.
305+-- | 'Alternative' form of 'Streamly.Internal.Data.Parser.alt'. Backtrack and
306+-- run the second parser if the first one fails.
307+--
308+-- The "some" and "many" operations of alternative accumulate results in a pure
309+-- list which is not scalable and streaming. Instead use
310+-- 'Streamly.Internal.Data.Parser.some' and
311+-- 'Streamly.Internal.Data.Parser.many' for fusible operations with composable
312+-- accumulation of results.
313+--
314+-- See also 'Streamly.Internal.Data.Parser.alt'. This 'Alternative' instance
315+-- does not fuse, use 'Streamly.Internal.Data.Parser.alt' when you need
316+-- fusion.
285317--
286318instance Monad m => Alternative (ParserK a m) where
287319 {-# INLINE empty #-}
@@ -333,22 +365,18 @@ instance MonadTrans (ParserK a) where
333365-- Convert ParserD to ParserK
334366-------------------------------------------------------------------------------
335367
336−--------------------------------------------------------------------------------
337−-- Chunked
338−--------------------------------------------------------------------------------
339−
340−{-# INLINE adaptCWith #-}
341−adaptCWith
368+{-# INLINE parseDToK #-}
369+parseDToK
342370 :: forall m a s b r. (Monad m, Unbox a)
343371 => (s -> a -> m (ParserD.Step s b))
344372 -> m (ParserD.Initial s b)
345373 -> (s -> m (ParserD.Step s b))
346− -> (ParseResult b -> Int -> Input (Array a) -> m (Step (Array a) m r))
374+ -> (ParseResult b -> Int -> Input a -> m (Step a m r))
347375 -> Int
348376 -> Int
349− -> Input (Array a)
350− -> m (Step (Array a) m r)
351−adaptCWith pstep initial extract cont !offset0 !usedCount !input = do
377+ -> Input a
378+ -> m (Step a m r)
379+parseDToK pstep initial extract cont !offset0 !usedCount !input = do
352380 res <- initial
353381 case res of
354382 ParserD.IPartial pst -> do
@@ -404,7 +432,7 @@ adaptCWith pstep initial extract cont !offset0 !usedCount !input = do
404432 go !_ !cur !pst | cur >= end =
405433 onContinue ((end - start) `div` SIZE_OF(a)) pst
406434 go !_ !cur !pst = do
407− let !x = unsafeInlineIO $ peekAt cur contents
435+ let !x = unsafeInlineIO $ peekByteIndex cur contents
408436 pRes <- pstep pst x
409437 let elemSize = SIZE_OF(a)
410438 next = INDEX_NEXT(cur,a)
@@ -452,258 +480,23 @@ adaptCWith pstep initial extract cont !offset0 !usedCount !input = do
452480 -- XXX It is called only when there is no input arr. So using 0
453481 -- as the position is correct?
454482 cont (Failure 0 err) count None
455− ParserD.Partial _ _ -> error "Bug: adaptCWith Partial unreachable"
456−
457− -- XXX Maybe we can use two separate continuations instead of using
458− -- Just/Nothing cases here. That may help in avoiding the parseContJust
459− -- function call.
460− {-# INLINE parseCont #-}
461− parseCont !cnt !pst (Chunk arr) = parseContChunk cnt 0 pst arr
462− parseCont !cnt !pst None = parseContNothing cnt pst
463−
464−-- | Convert an element 'Parser' to a chunked 'ParserK'. A chunked parser is
465−-- more efficient than an element parser.
466−--
467−-- /Pre-release/
468−--
469−{-# INLINE_LATE adaptC #-}
470−adaptC :: (Monad m, Unbox a) => ParserD.Parser a m b -> ParserK (Array a) m b
471−adaptC (ParserD.Parser step initial extract) =
472− MkParser $ adaptCWith step initial extract
473−
474−--------------------------------------------------------------------------------
475−-- Singular
476−--------------------------------------------------------------------------------
477−
478−{-# INLINE adaptWith #-}
479−adaptWith
480− :: forall m a s b r. (Monad m)
481− => (s -> a -> m (ParserD.Step s b))
482− -> m (ParserD.Initial s b)
483− -> (s -> m (ParserD.Step s b))
484− -> (ParseResult b -> Int -> Input a -> m (Step a m r))
485− -> Int
486− -> Int
487− -> Input a
488− -> m (Step a m r)
489−adaptWith pstep initial extract cont !relPos !usedCount !input = do
490− res <- initial
491− case res of
492− ParserD.IPartial pst -> do
493− -- XXX can we come here with relPos 1?
494− if relPos == 0
495− then
496− case input of
497− Chunk arr -> parseContChunk usedCount pst arr
498− None -> parseContNothing usedCount pst
499− -- XXX Previous code was using Continue in this case
500− else pure $ Partial relPos (parseCont usedCount pst)
501− ParserD.IDone b -> cont (Success relPos b) usedCount input
502− ParserD.IError err -> cont (Failure relPos err) usedCount input
503−
504− where
505−
506− -- XXX We can maintain an absolute position instead of relative that will
507− -- help in reporting of error location in the stream.
508− {-# NOINLINE parseContChunk #-}
509− parseContChunk !count !state x = do
510− go SPEC state
511−
512− where
513−
514− go !_ !pst = do
515− pRes <- pstep pst x
516− case pRes of
517− ParserD.Done 0 b ->
518− cont (Success 1 b) (count + 1) (Chunk x)
519− ParserD.Done 1 b ->
520− cont (Success 0 b) count (Chunk x)
521− ParserD.Done n b ->
522− cont (Success (1 - n) b) (count + 1 - n) (Chunk x)
523− ParserD.Partial 0 pst1 ->
524− pure $ Partial 1 (parseCont (count + 1) pst1)
525− ParserD.Partial 1 pst1 ->
526− -- XXX Since we got Partial, the driver should drop the
527− -- buffer, we should call the driver here?
528− go SPEC pst1
529− ParserD.Partial n pst1 ->
530− pure $ Partial (1 - n) (parseCont (count + 1 - n) pst1)
531− ParserD.Continue 0 pst1 ->
532− pure $ Continue 1 (parseCont (count + 1) pst1)
533− ParserD.Continue 1 pst1 ->
534− go SPEC pst1
535− ParserD.Continue n pst1 ->
536− pure $ Continue (1 - n) (parseCont (count + 1 - n) pst1)
537− ParserD.Error err ->
538− -- XXX fix undefined
539− cont (Failure 0 err) count (Chunk x)
540−
541− {-# NOINLINE parseContNothing #-}
542− parseContNothing !count !pst = do
543− r <- extract pst
544− case r of
545− -- IMPORTANT: the n here is from the byte stream parser, that means
546− -- it is the backtrack element count and not the stream position
547− -- index into the current input array.
548− ParserD.Done n b ->
549− assert (n >= 0)
550− (cont (Success (- n) b) (count - n) None)
551− ParserD.Continue n pst1 ->
552− assert (n >= 0)
553− (return $ Continue (- n) (parseCont (count - n) pst1))
554− ParserD.Error err ->
555− -- XXX It is called only when there is no input arr. So using 0
556− -- as the position is correct?
557− cont (Failure 0 err) count None
558− ParserD.Partial _ _ -> error "Bug: adaptCWith Partial unreachable"
483+ ParserD.Partial _ _ -> error "Bug: parseDToK Partial unreachable"
559484
560485 -- XXX Maybe we can use two separate continuations instead of using
561486 -- Just/Nothing cases here. That may help in avoiding the parseContJust
562487 -- function call.
563− {-# INLINE parseCont #-}
564− parseCont !cnt !pst (Chunk arr) = parseContChunk cnt pst arr
565− parseCont !cnt !pst None = parseContNothing cnt pst
566−
567−-- | Convert a 'Parser' to 'ParserK'.
568−--
569−-- /Pre-release/
570−--
571−{-# INLINE_LATE adapt #-}
572−adapt :: Monad m => ParserD.Parser a m b -> ParserK a m b
573−adapt (ParserD.Parser step initial extract) =
574− MkParser $ adaptWith step initial extract
575−
576−--------------------------------------------------------------------------------
577−-- Chunked Generic
578−--------------------------------------------------------------------------------
579−
580−{-# INLINE adaptCGWith #-}
581−adaptCGWith
582− :: forall m a s b r. (Monad m)
583− => (s -> a -> m (ParserD.Step s b))
584− -> m (ParserD.Initial s b)
585− -> (s -> m (ParserD.Step s b))
586− -> (ParseResult b -> Int -> Input (GenArr.Array a) -> m (Step (GenArr.Array a) m r))
587− -> Int
588− -> Int
589− -> Input (GenArr.Array a)
590− -> m (Step (GenArr.Array a) m r)
591−adaptCGWith pstep initial extract cont !offset0 !usedCount !input = do
592− res <- initial
593− case res of
594− ParserD.IPartial pst -> do
595− case input of
596− Chunk arr -> parseContChunk usedCount offset0 pst arr
597− None -> parseContNothing usedCount pst
598− ParserD.IDone b -> cont (Success offset0 b) usedCount input
599− ParserD.IError err -> cont (Failure offset0 err) usedCount input
600−
601− where
602−
603− {-# NOINLINE parseContChunk #-}
604− parseContChunk !count !offset !state arr@(GenArr.Array contents start len) = do
605− if offset >= 0
606− then go SPEC (start + offset) state
607− else return $ Continue offset (parseCont count state)
608−
609− where
610−
611− {-# INLINE end #-}
612− end = start + len
613−
614− {-# INLINE onDone #-}
615− onDone n b =
616− assert (n <= GenArr.length arr)
617− (cont (Success n b) (count + n - offset) (Chunk arr))
618−
619− {-# INLINE callParseCont #-}
620− callParseCont constr n pst1 =
621− assert (n < 0 || n >= GenArr.length arr)
622− (return $ constr n (parseCont (count + n - offset) pst1))
623−
624− {-# INLINE onPartial #-}
625− onPartial = callParseCont Partial
626−
627− {-# INLINE onContinue #-}
628− onContinue = callParseCont Continue
629−
630− {-# INLINE onError #-}
631− onError n err =
632− cont (Failure n err) (count + n - offset) (Chunk arr)
633−
634− {-# INLINE onBack #-}
635− onBack offset1 constr pst = do
636− let pos = offset1 - start
637− in if pos >= 0
638− then go SPEC offset1 pst
639− else constr pos pst
640−
641− go !_ !cur !pst | cur >= end =
642− onContinue len pst
643− go !_ !cur !pst = do
644− let !x = unsafeInlineIO $ GenArr.getIndexUnsafeWith contents cur
645− pRes <- pstep pst x
646− let next = cur + 1
647− back n = next - n
648− curOff = cur - start
649− nextOff = next - start
650− -- The "n" here is stream position index wrt the array start, and
651− -- not the backtrack count as returned by byte stream parsers.
652− case pRes of
653− ParserD.Done 0 b ->
654− onDone nextOff b
655− ParserD.Done 1 b ->
656− onDone curOff b
657− ParserD.Done n b ->
658− onDone (back n - start) b
659− ParserD.Partial 0 pst1 ->
660− go SPEC next pst1
661− ParserD.Partial 1 pst1 ->
662− go SPEC cur pst1
663− ParserD.Partial n pst1 ->
664− onBack (back n) onPartial pst1
665− ParserD.Continue 0 pst1 ->
666− go SPEC next pst1
667− ParserD.Continue 1 pst1 ->
668− go SPEC cur pst1
669− ParserD.Continue n pst1 ->
670− onBack (back n) onContinue pst1
671− ParserD.Error err ->
672− onError curOff err
673−
674− {-# NOINLINE parseContNothing #-}
675− parseContNothing !count !pst = do
676− r <- extract pst
677− case r of
678− -- IMPORTANT: the n here is from the byte stream parser, that means
679− -- it is the backtrack element count and not the stream position
680− -- index into the current input array.
681− ParserD.Done n b ->
682− assert (n >= 0)
683− (cont (Success (- n) b) (count - n) None)
684− ParserD.Continue n pst1 ->
685− assert (n >= 0)
686− (return $ Continue (- n) (parseCont (count - n) pst1))
687− ParserD.Error err ->
688− -- XXX It is called only when there is no input arr. So using 0
689− -- as the position is correct?
690− cont (Failure 0 err) count None
691− ParserD.Partial _ _ -> error "Bug: adaptCGWith Partial unreachable"
692−
693488 {-# INLINE parseCont #-}
694489 parseCont !cnt !pst (Chunk arr) = parseContChunk cnt 0 pst arr
695490 parseCont !cnt !pst None = parseContNothing cnt pst
696491
697−-- | A generic 'adaptC'. Similar to 'adaptC' but is not constrained to 'Unbox'
698−-- types.
492+-- | Convert a raw byte 'Parser' to a chunked 'ParserK'.
699493--
700494-- /Pre-release/
701495--
702−{-# INLINE_LATE adaptCG #-}
703−adaptCG ::
704− Monad m => ParserD.Parser a m b -> ParserK (GenArr.Array a) m b
705−adaptCG (ParserD.Parser step initial extract) =
706− MkParser $ adaptCGWith step initial extract
496+{-# INLINE_LATE fromParser #-}
497+fromParser :: (Monad m, Unbox a) => ParserD.Parser a m b -> ParserK a m b
498+fromParser (ParserD.Parser step initial extract) =
499+ MkParser $ parseDToK step initial extract
707500
708501{-
709502-------------------------------------------------------------------------------
@@ -745,8 +538,10 @@ toParser parser = ParserD.Parser step initial extract
745538 Partial _ cont1 -> extract cont1
746539 Continue n cont1 -> return $ ParserD.Continue n cont1
747540
541+#ifndef DISABLE_FUSION
748542{-# RULES "fromParser/toParser fusion" [2]
749543 forall s. toParser (fromParser s) = s #-}
750544{-# RULES "toParser/fromParser fusion" [2]
751545 forall s. fromParser (toParser s) = s #-}
546+#endif
752547-}
core/src/Streamly/Internal/Data/ParserK.hsdeleted−16
@@ -1,16 +0,0 @@
1−{-# LANGUAGE CPP #-}
2−-- |
3−-- Module : Streamly.Internal.Data.ParserK
4−-- Copyright : (c) 2020 Composewell Technologies
5−-- License : BSD-3-Clause
6−-- Maintainer : streamly@composewell.com
7−-- Stability : experimental
8−-- Portability : GHC
9−
10−module Streamly.Internal.Data.ParserK
11− (
12− module Streamly.Internal.Data.ParserK.Type
13− )
14−where
15−
16−import Streamly.Internal.Data.ParserK.Type
core/src/Streamly/Internal/Data/Pipe.hs+8−5
@@ -24,10 +24,11 @@
2424module Streamly.Internal.Data.Pipe
2525 (
2626 -- * Pipe Type
27− module Streamly.Internal.Data.Pipe.Type
27+ Pipe
2828
2929 -- * Pipes
3030 -- ** Mapping
31+ , map
3132 , mapM
3233
3334 {-
@@ -111,8 +112,10 @@ module Streamly.Internal.Data.Pipe
111112 , groupsRollingBy
112113 -}
113114
114−
115−
115+ -- * Composing Pipes
116+ , tee
117+ , zipWith
118+ , compose
116119
117120 {-
118121 -- * Distributing
@@ -241,9 +244,9 @@ import Prelude
241244
242245-- import Streamly.Data.Fold.Types (Fold(..))
243246import Streamly.Internal.Data.Pipe.Type
244−
247+ (Pipe(..), PipeState(..), Step(..), zipWith, tee, map, compose)
245248-- import Streamly.Internal.Data.Array.Type (Array)
246−-- import Streamly.Internal.Data.Ring (Ring)
249+-- import Streamly.Internal.Data.Ring.Unboxed (Ring)
247250-- import Streamly.Internal.Data.Stream (Stream)
248251-- import Streamly.Internal.Data.Time.Units
249252-- (AbsTime, MilliSecond64(..), addToAbsTime, diffAbsTime, toRelTime,
core/src/Streamly/Internal/Data/Producer.hs+13−6
@@ -24,24 +24,31 @@
2424-- unecessary function calls can be avoided.
2525
2626module Streamly.Internal.Data.Producer
27− (
28− module Streamly.Internal.Data.Producer.Source
29− , module Streamly.Internal.Data.Producer.Type
27+ ( Producer (..)
3028
3129 -- * Converting
3230 , simplify
31+
32+ -- * Producers
33+ , nil
34+ , nilM
35+ , unfoldrM
3336 , fromStreamD
37+ , fromList
38+
39+ -- * Combinators
40+ , NestedLoop (..)
41+ , concat
3442 )
3543where
3644
3745#include "inline.hs"
3846
39−import Streamly.Internal.Data.Stream.Step (Step(..))
40−import Streamly.Internal.Data.Stream.Type (Stream(..))
47+import Streamly.Internal.Data.Stream.StreamD.Step (Step(..))
48+import Streamly.Internal.Data.Stream.StreamD.Type (Stream(..))
4149import Streamly.Internal.Data.SVar.Type (defState)
4250import Streamly.Internal.Data.Unfold.Type (Unfold(..))
4351
44−import Streamly.Internal.Data.Producer.Source
4552import Streamly.Internal.Data.Producer.Type
4653import Prelude hiding (concat)
4754
core/src/Streamly/Internal/Data/Producer/Source.hs+1−1
@@ -38,7 +38,7 @@ import GHC.Exts (SpecConstrAnnotation(..))
3838import GHC.Types (SPEC(..))
3939import Streamly.Internal.Data.Parser (ParseError(..), Step(..))
4040import Streamly.Internal.Data.Producer.Type (Producer(..))
41−import Streamly.Internal.Data.Stream.Step (Step(..))
41+import Streamly.Internal.Data.Stream.StreamD.Step (Step(..))
4242
4343import qualified Streamly.Internal.Data.Parser as ParserD
4444-- import qualified Streamly.Internal.Data.Parser.ParserK.Type as ParserK
core/src/Streamly/Internal/Data/Producer/Type.hs+1−1
@@ -33,7 +33,7 @@ where
3333#include "inline.hs"
3434
3535import Fusion.Plugin.Types (Fuse(..))
36−import Streamly.Internal.Data.Stream.Step (Step(..))
36+import Streamly.Internal.Data.Stream.StreamD.Step (Step(..))
3737import Prelude hiding (concat, map)
3838
3939------------------------------------------------------------------------------
core/src/Streamly/Internal/Data/Ring.hs+134−589
@@ -1,619 +1,164 @@
11-- |
22-- Module : Streamly.Internal.Data.Ring
3−-- Copyright : (c) 2019 Composewell Technologies
4−-- License : BSD3
3+-- Copyright : (c) 2021 Composewell Technologies
4+-- License : BSD-3-Clause
55-- Maintainer : streamly@composewell.com
66-- Stability : experimental
77-- Portability : GHC
88--
9−-- A ring array is a circular mutable array.
10−
11−-- XXX Write benchmarks
12−-- XXX Make the implementation similar to mutable array
13−-- XXX Rename this module to Data.RingArray.Storable
149
1510module Streamly.Internal.Data.Ring
1611 ( Ring(..)
1712
18− -- * Construction
19− , new
20− , newRing
21− , writeN
22−
23− , advance
24− , moveBy
25− , startOf
26−
27− -- * Random writes
28− , unsafeInsert
29− , slide
30− , putIndex
31− , modifyIndex
32−
33− -- * Unfolds
34− , read
35− , readRev
36−
37− -- * Random reads
38− , getIndex
39− , getIndexUnsafe
40− , getIndexRev
41−
42− -- * Size
43− , length
44− , byteLength
45− -- , capacity
46− , byteCapacity
47− , bytesFree
48−
49− -- * Casting
50− , cast
51− , castUnsafe
52− , asBytes
53− , fromArray
54−
55− -- * Folds
56− , unsafeFoldRing
57− , unsafeFoldRingM
58− , unsafeFoldRingFullM
59− , unsafeFoldRingNM
13+ -- * Generation
14+ , createRing
15+ , writeLastN
6016
61− -- * Stream of Arrays
62− , ringsOf
17+ -- * Modification
18+ , seek
19+ , unsafeInsertRingWith
6320
64− -- * Fast Byte Comparisons
65− , unsafeEqArray
66− , unsafeEqArrayN
67−
68− , slidingWindow
69− , slidingWindowWith
21+ -- * Conversion
22+ , toMutArray
23+ , toStreamWith
7024 ) where
7125
72−#include "ArrayMacros.h"
73−#include "inline.hs"
74−
75−import Control.Exception (assert)
76−import Control.Monad.IO.Class (MonadIO(..))
77−import Data.Word (Word8)
78−import Foreign.Storable
79−import Foreign.ForeignPtr (ForeignPtr, withForeignPtr, touchForeignPtr)
80−import Foreign.ForeignPtr.Unsafe (unsafeForeignPtrToPtr)
81−import Foreign.Ptr (plusPtr, minusPtr, castPtr)
82−import Streamly.Internal.Data.Unbox as Unboxed (Unbox(peekAt))
83−import GHC.ForeignPtr (mallocPlainForeignPtrAlignedBytes)
84−import GHC.Ptr (Ptr(..))
85−import Streamly.Internal.Data.MutArray.Type (MutArray)
86−import Streamly.Internal.Data.Fold.Type (Fold(..), Step(..), lmap)
87−import Streamly.Internal.Data.Stream.Type (Stream)
88−import Streamly.Internal.Data.Stream.Step (Step(..))
89−import Streamly.Internal.Data.Unfold.Type (Unfold(..))
90−import Streamly.Internal.System.IO (unsafeInlineIO)
91−
92−import qualified Streamly.Internal.Data.MutArray.Type as MA
93−import qualified Streamly.Internal.Data.Array.Type as A
94−
95−import Prelude hiding (length, concat, read)
96−
97−-- $setup
98−-- >>> :m
99−-- >>> import qualified Streamly.Internal.Data.Ring as Ring
26+#include "assert.hs"
10027
101−-- | A ring buffer is a mutable array of fixed size. Initially the array is
102−-- empty, with ringStart pointing at the start of allocated memory. We call the
103−-- next location to be written in the ring as ringHead. Initially ringHead ==
104−-- ringStart. When the first item is added, ringHead points to ringStart +
105−-- sizeof item. When the buffer becomes full ringHead would wrap around to
106−-- ringStart. When the buffer is full, ringHead always points at the oldest
107−-- item in the ring and the newest item added always overwrites the oldest
108−-- item.
109−--
110−-- When using it we should keep in mind that a ringBuffer is a mutable data
111−-- structure. We should not leak out references to it for immutable use.
112−--
28+import Control.Monad.IO.Class (liftIO, MonadIO)
29+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
30+import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
31+import Streamly.Internal.Data.Fold.Type (Fold(..))
32+import Streamly.Internal.Data.MutArray.Generic
33+ ( MutArray(..)
34+ , new
35+ , uninit
36+ , putIndexUnsafe
37+ , putSliceUnsafe
38+ )
39+-- import qualified Streamly.Internal.Data.Stream.StreamD.Type as Stream
40+import qualified Streamly.Internal.Data.Fold.Type as Fold
41+
42+-- XXX Use MutableArray rather than keeping a MutArray here.
11343data Ring a = Ring
114− { ringStart :: {-# UNPACK #-} !(ForeignPtr a) -- first address
115− , ringBound :: {-# UNPACK #-} !(Ptr a) -- first address beyond allocated memory
44+ { ringArr :: MutArray a
45+ -- XXX We can keep the current fill amount, Or we can keep a count of total
46+ -- elements inserted and compute ring head as well using mod on that,
47+ -- assuming it won't overflow. But mod could be expensive.
48+ , ringHead :: !Int -- current index to be over-written
49+ , ringMax :: !Int -- first index beyond allocated memory
11650 }
11751
11852-------------------------------------------------------------------------------
119−-- Construction
53+-- Generation
12054-------------------------------------------------------------------------------
12155
122−-- | Get the first address of the ring as a pointer.
123−startOf :: Ring a -> Ptr a
124−startOf = unsafeForeignPtrToPtr . ringStart
125−
126−-- | Create a new ringbuffer and return the ring buffer and the ringHead.
127−-- Returns the ring and the ringHead, the ringHead is same as ringStart.
128−{-# INLINE new #-}
129−new :: forall a. Storable a => Int -> IO (Ring a, Ptr a)
130−new count = do
131− let size = count * max 1 (sizeOf (undefined :: a))
132− fptr <- mallocPlainForeignPtrAlignedBytes size (alignment (undefined :: a))
133− let p = unsafeForeignPtrToPtr fptr
56+-- XXX If we align the ringMax to nearest power of two then computation of the
57+-- index to write could be cheaper.
58+{-# INLINE createRing #-}
59+createRing :: MonadIO m => Int -> m (Ring a)
60+createRing count = liftIO $ do
61+ arr <- new count
62+ arr1 <- uninit arr count
13463 return (Ring
135− { ringStart = fptr
136− , ringBound = p `plusPtr` size
137− }, p)
138−
139−-- XXX Rename this to "new".
140−--
141−-- | @newRing count@ allocates an empty array that can hold 'count' items. The
142−-- memory of the array is uninitialized and the allocation is aligned as per
143−-- the 'Storable' instance of the type.
144−--
145−-- /Unimplemented/
146−{-# INLINE newRing #-}
147−newRing :: Int -> m (Ring a)
148−newRing = undefined
149−
150−-- | Advance the ringHead by 1 item, wrap around if we hit the end of the
151−-- array.
152−{-# INLINE advance #-}
153−advance :: forall a. Storable a => Ring a -> Ptr a -> Ptr a
154−advance Ring{..} ringHead =
155− let ptr = PTR_NEXT(ringHead,a)
156− in if ptr < ringBound
157− then ptr
158− else unsafeForeignPtrToPtr ringStart
159−
160−-- | Move the ringHead by n items. The direction depends on the sign on whether
161−-- n is positive or negative. Wrap around if we hit the beginning or end of the
162−-- array.
163−{-# INLINE moveBy #-}
164−moveBy :: forall a. Storable a => Int -> Ring a -> Ptr a -> Ptr a
165−moveBy by Ring {..} ringHead = ringStartPtr `plusPtr` advanceFromHead
166−
167− where
168−
169− elemSize = STORABLE_SIZE_OF(a)
170− ringStartPtr = unsafeForeignPtrToPtr ringStart
171− lenInBytes = ringBound `minusPtr` ringStartPtr
172− offInBytes = ringHead `minusPtr` ringStartPtr
173− len = assert (lenInBytes `mod` elemSize == 0) $ lenInBytes `div` elemSize
174− off = assert (offInBytes `mod` elemSize == 0) $ offInBytes `div` elemSize
175− advanceFromHead = (off + by `mod` len) * elemSize
176−
177−-- XXX Move the writeLastN from array module here.
178−--
179−-- | @writeN n@ is a rolling fold that keeps the last n elements of the stream
180−-- in a ring array.
181−--
182−-- /Unimplemented/
183−{-# INLINE writeN #-}
184−writeN :: -- (Storable a, MonadIO m) =>
185− Int -> Fold m a (Ring a)
186−writeN = undefined
187−
188−-------------------------------------------------------------------------------
189−-- Conversions
190−-------------------------------------------------------------------------------
191−
192−-- | Cast a mutable array to a ring array.
193−fromArray :: MutArray a -> Ring a
194−fromArray = undefined
195−
196−-------------------------------------------------------------------------------
197−-- Conversion to/from array
198−-------------------------------------------------------------------------------
199−
200−-- | Modify a given index of a ring array using a modifier function.
201−--
202−-- /Unimplemented/
203−modifyIndex :: -- forall m a b. (MonadIO m, Storable a) =>
204− Ring a -> Int -> (a -> (a, b)) -> m b
205−modifyIndex = undefined
206−
207−-- | /O(1)/ Write the given element at the given index in the ring array.
208−-- Performs in-place mutation of the array.
209−--
210−-- >>> putIndex arr ix val = Ring.modifyIndex arr ix (const (val, ()))
211−--
212−-- /Unimplemented/
213−{-# INLINE putIndex #-}
214−putIndex :: -- (MonadIO m, Storable a) =>
215− Ring a -> Int -> a -> m ()
216−putIndex = undefined
217−
218−-- | Insert an item at the head of the ring, when the ring is full this
219−-- replaces the oldest item in the ring with the new item. This is unsafe
220−-- beause ringHead supplied is not verified to be within the Ring. Also,
221−-- the ringStart foreignPtr must be guaranteed to be alive by the caller.
222−{-# INLINE unsafeInsert #-}
223−unsafeInsert :: Storable a => Ring a -> Ptr a -> a -> IO (Ptr a)
224−unsafeInsert rb ringHead newVal = do
225− poke ringHead newVal
226− -- touchForeignPtr (ringStart rb)
227− return $ advance rb ringHead
228−
229−-- | Insert an item at the head of the ring, when the ring is full this
230−-- replaces the oldest item in the ring with the new item.
231−--
232−-- /Unimplemented/
233−slide :: -- forall m a. (MonadIO m, Storable a) =>
234− Ring a -> a -> m (Ring a)
235−slide = undefined
236−
237−-------------------------------------------------------------------------------
238−-- Random reads
239−-------------------------------------------------------------------------------
240−
241−-- | Return the element at the specified index without checking the bounds.
242−--
243−-- Unsafe because it does not check the bounds of the ring array.
244−{-# INLINE_NORMAL getIndexUnsafe #-}
245−getIndexUnsafe :: -- forall m a. (MonadIO m, Storable a) =>
246− Ring a -> Int -> m a
247−getIndexUnsafe = undefined
248−
249−-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
250−--
251−{-# INLINE getIndex #-}
252−getIndex :: -- (MonadIO m, Storable a) =>
253− Ring a -> Int -> m a
254−getIndex = undefined
255−
256−-- | /O(1)/ Lookup the element at the given index from the end of the array.
257−-- Index starts from 0.
258−--
259−-- Slightly faster than computing the forward index and using getIndex.
260−--
261−{-# INLINE getIndexRev #-}
262−getIndexRev :: -- (MonadIO m, Storable a) =>
263− Ring a -> Int -> m a
264−getIndexRev = undefined
265−
266−-------------------------------------------------------------------------------
267−-- Size
268−-------------------------------------------------------------------------------
269−
270−-- | /O(1)/ Get the byte length of the array.
271−--
272−-- /Unimplemented/
273−{-# INLINE byteLength #-}
274−byteLength :: Ring a -> Int
275−byteLength = undefined
276−
277−-- | /O(1)/ Get the length of the array i.e. the number of elements in the
278−-- array.
279−--
280−-- Note that 'byteLength' is less expensive than this operation, as 'length'
281−-- involves a costly division operation.
282−--
283−-- /Unimplemented/
284−{-# INLINE length #-}
285−length :: -- forall a. Storable a =>
286− Ring a -> Int
287−length = undefined
288−
289−-- | Get the total capacity of an array. An array may have space reserved
290−-- beyond the current used length of the array.
291−--
292−-- /Pre-release/
293−{-# INLINE byteCapacity #-}
294−byteCapacity :: Ring a -> Int
295−byteCapacity = undefined
296−
297−-- | The remaining capacity in the array for appending more elements without
298−-- reallocation.
299−--
300−-- /Pre-release/
301−{-# INLINE bytesFree #-}
302−bytesFree :: Ring a -> Int
303−bytesFree = undefined
304−
305−-------------------------------------------------------------------------------
306−-- Unfolds
307−-------------------------------------------------------------------------------
308−
309−-- XXX We can read the ring in a loop and use "take" to restrict the number of
310−-- elements to be taken.
311−--
312−-- | Read n elements from the ring starting at the supplied ring head. If n is
313−-- more than the ring size it keeps reading the ring in a circular fashion.
314−--
315−-- If the ring is not full the user must ensure than n is less than or equal to
316−-- the number of valid elements in the ring.
317−--
318−-- /Internal/
319−{-# INLINE_NORMAL read #-}
320−read :: forall m a. (MonadIO m, Storable a) => Unfold m (Ring a, Ptr a, Int) a
321−read = Unfold step return
322−
323− where
324−
325− step (rb, rh, n) = do
326− if n <= 0
327− then do
328− liftIO $ touchForeignPtr (ringStart rb)
329− return Stop
330− else do
331− x <- liftIO $ peek rh
332− let rh1 = advance rb rh
333− return $ Yield x (rb, rh1, n - 1)
334−
335−-- | Unfold a ring array into a stream in reverse order.
336−--
337−-- /Unimplemented/
338−{-# INLINE_NORMAL readRev #-}
339−readRev :: -- forall m a. (MonadIO m, Storable a) =>
340− Unfold m (MutArray a) a
341−readRev = undefined
342−
343−-------------------------------------------------------------------------------
344−-- Stream of arrays
345−-------------------------------------------------------------------------------
346−
347−-- XXX Move this module to a lower level Ring/Type module and move ringsOf to a
348−-- higher level ring module where we can import "scan".
349−
350−-- | @ringsOf n stream@ groups the input stream into a stream of
351−-- ring arrays of size n. Each ring is a sliding window of size n.
352−--
353−-- /Unimplemented/
354−{-# INLINE_NORMAL ringsOf #-}
355−ringsOf :: -- forall m a. (MonadIO m, Storable a) =>
356− Int -> Stream m a -> Stream m (MutArray a)
357−ringsOf = undefined -- Stream.scan (writeN n)
358−
359−-------------------------------------------------------------------------------
360−-- Casting
361−-------------------------------------------------------------------------------
64+ { ringArr = arr1
65+ , ringHead = 0
66+ , ringMax = count
67+ })
36268
363−-- | Cast an array having elements of type @a@ into an array having elements of
364−-- type @b@. The array size must be a multiple of the size of type @b@.
365−--
366−-- /Unimplemented/
367−--
368−castUnsafe :: Ring a -> Ring b
369−castUnsafe = undefined
370−
371−-- | Cast an @Array a@ into an @Array Word8@.
372−--
373−-- /Unimplemented/
374−--
375−asBytes :: Ring a -> Ring Word8
376−asBytes = castUnsafe
377−
378−-- | Cast an array having elements of type @a@ into an array having elements of
379−-- type @b@. The length of the array should be a multiple of the size of the
380−-- target element otherwise 'Nothing' is returned.
381−--
382−-- /Pre-release/
383−--
384−cast :: forall a b. Storable b => Ring a -> Maybe (Ring b)
385−cast arr =
386− let len = byteLength arr
387− r = len `mod` STORABLE_SIZE_OF(b)
388− in if r /= 0
389− then Nothing
390− else Just $ castUnsafe arr
391−
392−-------------------------------------------------------------------------------
393−-- Equality
394−-------------------------------------------------------------------------------
39569
396−-- XXX remove all usage of unsafeInlineIO
397−--
398−-- | Like 'unsafeEqArray' but compares only N bytes instead of entire length of
399−-- the ring buffer. This is unsafe because the ringHead Ptr is not checked to
400−-- be in range.
401−{-# INLINE unsafeEqArrayN #-}
402−unsafeEqArrayN :: Ring a -> Ptr a -> A.Array a -> Int -> Bool
403−unsafeEqArrayN Ring{..} rh A.Array{..} nBytes
404− | nBytes < 0 = error "unsafeEqArrayN: n should be >= 0"
405− | nBytes == 0 = True
406− | otherwise = unsafeInlineIO $ check (castPtr rh) 0
407−
408− where
409−
410− w8Contents = arrContents
411−
412− check p i = do
413− (relem :: Word8) <- peek p
414− aelem <- peekAt i w8Contents
415− if relem == aelem
416− then go (p `plusPtr` 1) (i + 1)
417− else return False
418−
419− go p i
420− | i == nBytes = return True
421− | castPtr p == ringBound =
422− go (castPtr (unsafeForeignPtrToPtr ringStart)) i
423− | castPtr p == rh = touchForeignPtr ringStart >> return True
424− | otherwise = check p i
425−
426−-- XXX This is not modular. We should probably just convert the array and the
427−-- ring buffer to streams and compare the two streams. Need to check perf
428−-- though.
429−
430−-- | Byte compare the entire length of ringBuffer with the given array,
431−-- starting at the supplied ringHead pointer. Returns true if the Array and
432−-- the ringBuffer have identical contents.
433−--
434−-- This is unsafe because the ringHead Ptr is not checked to be in range. The
435−-- supplied array must be equal to or bigger than the ringBuffer, ARRAY BOUNDS
436−-- ARE NOT CHECKED.
437−{-# INLINE unsafeEqArray #-}
438−unsafeEqArray :: Ring a -> Ptr a -> A.Array a -> Bool
439−unsafeEqArray Ring{..} rh A.Array{..} =
440− unsafeInlineIO $ check (castPtr rh) 0
441−
442− where
443−
444− w8Contents = arrContents
445−
446− check p i = do
447− (relem :: Word8) <- peek p
448− aelem <- peekAt i w8Contents
449− if relem == aelem
450− then go (p `plusPtr` 1) (i + 1)
451− else return False
452−
453− go p i
454− | castPtr p ==
455− ringBound = go (castPtr (unsafeForeignPtrToPtr ringStart)) i
456− | castPtr p == rh = touchForeignPtr ringStart >> return True
457− | otherwise = check p i
458−
459−-------------------------------------------------------------------------------
460−-- Folding
461−-------------------------------------------------------------------------------
462−
463−-- XXX We can unfold it into a stream and fold the stream instead.
464−-- XXX use MonadIO
465−--
466−-- | Fold the buffer starting from ringStart up to the given 'Ptr' using a pure
467−-- step function. This is useful to fold the items in the ring when the ring is
468−-- not full. The supplied pointer is usually the end of the ring.
469−--
470−-- Unsafe because the supplied Ptr is not checked to be in range.
471−{-# INLINE unsafeFoldRing #-}
472−unsafeFoldRing :: forall a b. Storable a
473− => Ptr a -> (b -> a -> b) -> b -> Ring a -> b
474−unsafeFoldRing ptr f z Ring{..} =
475− let !res = unsafeInlineIO $ withForeignPtr ringStart $ \p ->
476− go z p ptr
477− in res
478− where
479− go !acc !p !q
480− | p == q = return acc
481− | otherwise = do
482− x <- peek p
483− go (f acc x) (PTR_NEXT(p,a)) q
484−
485−-- XXX Can we remove MonadIO here?
486−withForeignPtrM :: MonadIO m => ForeignPtr a -> (Ptr a -> m b) -> m b
487−withForeignPtrM fp fn = do
488− r <- fn $ unsafeForeignPtrToPtr fp
489− liftIO $ touchForeignPtr fp
490− return r
491−
492−-- | Like unsafeFoldRing but with a monadic step function.
493−{-# INLINE unsafeFoldRingM #-}
494−unsafeFoldRingM :: forall m a b. (MonadIO m, Storable a)
495− => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
496−unsafeFoldRingM ptr f z Ring {..} =
497− withForeignPtrM ringStart $ \x -> go z x ptr
498− where
499− go !acc !start !end
500− | start == end = return acc
501− | otherwise = do
502− let !x = unsafeInlineIO $ peek start
503− acc1 <- f acc x
504− go acc1 (PTR_NEXT(start,a)) end
505−
506−-- | Fold the entire length of a ring buffer starting at the supplied ringHead
507−-- pointer. Assuming the supplied ringHead pointer points to the oldest item,
508−-- this would fold the ring starting from the oldest item to the newest item in
509−-- the ring.
510−--
511−-- Note, this will crash on ring of 0 size.
512−--
513−{-# INLINE unsafeFoldRingFullM #-}
514−unsafeFoldRingFullM :: forall m a b. (MonadIO m, Storable a)
515− => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
516−unsafeFoldRingFullM rh f z rb@Ring {..} =
517− withForeignPtrM ringStart $ \_ -> go z rh
518− where
519− go !acc !start = do
520− let !x = unsafeInlineIO $ peek start
521− acc' <- f acc x
522− let ptr = advance rb start
523− if ptr == rh
524− then return acc'
525− else go acc' ptr
526−
527−-- | Fold @Int@ items in the ring starting at @Ptr a@. Won't fold more
528−-- than the length of the ring.
529−--
530−-- Note, this will crash on ring of 0 size.
531−--
532−{-# INLINE unsafeFoldRingNM #-}
533−unsafeFoldRingNM :: forall m a b. (MonadIO m, Storable a)
534− => Int -> Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
535−unsafeFoldRingNM count rh f z rb@Ring {..} =
536− withForeignPtrM ringStart $ \_ -> go count z rh
537−
538− where
539−
540− go 0 acc _ = return acc
541− go !n !acc !start = do
542− let !x = unsafeInlineIO $ peek start
543− acc' <- f acc x
544− let ptr = advance rb start
545− if ptr == rh || n == 0
546− then return acc'
547− else go (n - 1) acc' ptr
548−
549−data Tuple4' a b c d = Tuple4' !a !b !c !d deriving Show
550−
551−-- | Like slidingWindow but also provides the entire ring contents as an Array.
552−-- The array reflects the state of the ring after inserting the incoming
553−-- element.
554−--
555−-- IMPORTANT NOTE: The ring is mutable, therefore, the result of @(m (Array
556−-- a))@ action depends on when it is executed. It does not capture the sanpshot
557−-- of the ring at a particular time.
558−{-# INLINE slidingWindowWith #-}
559−slidingWindowWith :: forall m a b. (MonadIO m, Storable a, Unbox a)
560− => Int -> Fold m ((a, Maybe a), m (MutArray a)) b -> Fold m a b
561−slidingWindowWith n (Fold step1 initial1 extract1 final1) =
562− Fold step initial extract final
70+{-# INLINE writeLastN #-}
71+writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
72+writeLastN n = Fold step initial extract
56373
56474 where
56575
56676 initial = do
56777 if n <= 0
568− then error "Window size must be > 0"
78+ then Fold.Done <$> createRing 0
56979 else do
570− r <- initial1
571− (rb, rh) <- liftIO $ new n
572− return $
573− case r of
574− Partial s -> Partial $ Tuple4' rb rh (0 :: Int) s
575− Done b -> Done b
576−
577− toArray foldRing rb rh = do
578− -- Using unpinned array here instead of pinned
579− arr <- liftIO $ MA.new n
580− let snoc' b a = liftIO $ MA.snocUnsafe b a
581− foldRing rh snoc' arr rb
582−
583− step (Tuple4' rb rh i st) a
584− | i < n = do
585− rh1 <- liftIO $ unsafeInsert rb rh a
586− liftIO $ touchForeignPtr (ringStart rb)
587− let action = toArray unsafeFoldRingM rb (PTR_NEXT(rh, a))
588− r <- step1 st ((a, Nothing), action)
589− return $
590− case r of
591− Partial s -> Partial $ Tuple4' rb rh1 (i + 1) s
592− Done b -> Done b
593− | otherwise = do
594− old <- liftIO $ peek rh
595− rh1 <- liftIO $ unsafeInsert rb rh a
596− liftIO $ touchForeignPtr (ringStart rb)
597− r <- step1 st ((a, Just old), toArray unsafeFoldRingFullM rb rh1)
598− return $
599− case r of
600− Partial s -> Partial $ Tuple4' rb rh1 (i + 1) s
601− Done b -> Done b
602−
603− extract (Tuple4' _ _ _ st) = extract1 st
604−
605− final (Tuple4' _ _ _ st) = final1 st
606−
607−-- | @slidingWindow collector@ is an incremental sliding window
608−-- fold that does not require all the intermediate elements in a computation.
609−-- This maintains @n@ elements in the window, when a new element comes it slides
610−-- out the oldest element and the new element along with the old element are
611−-- supplied to the collector fold.
612−--
613−-- The 'Maybe' type is for the case when initially the window is filling and
614−-- there is no old element.
615−--
616−{-# INLINE slidingWindow #-}
617−slidingWindow :: forall m a b. (MonadIO m, Storable a, Unbox a)
618− => Int -> Fold m (a, Maybe a) b -> Fold m a b
619−slidingWindow n f = slidingWindowWith n (lmap fst f)
80+ rb <- createRing n
81+ return $ Fold.Partial $ Tuple' rb (0 :: Int)
82+
83+ step (Tuple' rb cnt) x = do
84+ rh1 <- liftIO $ unsafeInsertRingWith rb x
85+ return $ Fold.Partial $ Tuple' (rb {ringHead = rh1}) (cnt + 1)
86+
87+ extract (Tuple' rb@Ring{..} cnt) =
88+ return $
89+ if cnt < ringMax
90+ then Ring ringArr 0 ringHead
91+ else rb
92+
93+-------------------------------------------------------------------------------
94+-- Modification
95+-------------------------------------------------------------------------------
96+
97+-- XXX This is safe
98+-- Take the ring head and return the new ring head.
99+{-# INLINE unsafeInsertRingWith #-}
100+unsafeInsertRingWith :: Ring a -> a -> IO Int
101+unsafeInsertRingWith Ring{..} x = do
102+ assertM(ringMax >= 1)
103+ assertM(ringHead < ringMax)
104+ putIndexUnsafe ringHead ringArr x
105+ let rh1 = ringHead + 1
106+ next = if rh1 == ringMax then 0 else rh1
107+ return next
108+
109+-- | Move the ring head clockwise (+ve adj) or counter clockwise (-ve adj) by
110+-- the given amount.
111+{-# INLINE seek #-}
112+seek :: MonadIO m => Int -> Ring a -> m (Ring a)
113+seek adj rng@Ring{..}
114+ | ringMax > 0 = liftIO $ do
115+ -- XXX try avoiding mod when in bounds
116+ let idx1 = ringHead + adj
117+ next = mod idx1 ringMax
118+ return $ Ring ringArr next ringMax
119+ | otherwise = pure rng
120+
121+-------------------------------------------------------------------------------
122+-- Conversion
123+-------------------------------------------------------------------------------
124+
125+-- | @toMutArray rignHeadAdjustment lengthToRead ring@.
126+-- Convert the ring into a boxed mutable array. Note that the returned MutArray
127+-- may share the same underlying memory as the Ring.
128+{-# INLINE toMutArray #-}
129+toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
130+toMutArray adj n Ring{..} = do
131+ let len = min ringMax n
132+ let idx = mod (ringHead + adj) ringMax
133+ end = idx + len
134+ if end <= ringMax
135+ then
136+ -- putSliceUnsafe ringArr idx arr1 0 len
137+ return $ ringArr { arrStart = idx, arrLen = len }
138+ else do
139+ -- XXX Just swap the elements in the existing ring and return the
140+ -- same array without reallocation.
141+ arr <- liftIO $ new len
142+ arr1 <- uninit arr len
143+ putSliceUnsafe ringArr idx arr1 0 (ringMax - idx)
144+ putSliceUnsafe ringArr 0 arr1 (ringMax - idx) (end - ringMax)
145+ return arr1
146+
147+-- This would be theoretically slower than toMutArray because of a branch
148+-- introduced for each element in the second half of the ring.
149+
150+-- | Seek by n and then read the entire ring. Use 'take' on the stream to
151+-- restrict the reads.
152+toStreamWith :: Int -> Ring a -> Stream m a
153+toStreamWith = undefined
154+{-
155+toStreamWith n Ring{..}
156+ | ringMax > 0 = concatEffect $ liftIO $ do
157+ idx <- readIORef ringHead
158+ let idx1 = idx + adj
159+ next = mod idx1 ringMax
160+ s1 = undefined -- stream initial slice
161+ s2 = undefined -- stream next slice
162+ return (s1 `Stream.append` s2)
163+ | otherwise = Stream.nil
164+-}
core/src/Streamly/Internal/Data/Ring/Generic.hsdeleted−185
@@ -1,185 +0,0 @@
1−-- |
2−-- Module : Streamly.Internal.Data.Ring.Generic
3−-- Copyright : (c) 2021 Composewell Technologies
4−-- License : BSD-3-Clause
5−-- Maintainer : streamly@composewell.com
6−-- Stability : experimental
7−-- Portability : GHC
8−--
9−
10−module Streamly.Internal.Data.Ring.Generic
11− ( Ring(..)
12−
13− -- * Generation
14− , createRing
15− , writeLastN
16−
17− -- * Modification
18− , seek
19− , unsafeInsertRingWith
20−
21− -- * Conversion
22− , toMutArray
23− , copyToMutArray
24− , toStreamWith
25− ) where
26−
27−#include "assert.hs"
28−
29−import Control.Monad.IO.Class (liftIO, MonadIO)
30−import Streamly.Internal.Data.Stream.Type (Stream)
31−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
32−import Streamly.Internal.Data.Fold.Type (Fold(..))
33−import Streamly.Internal.Data.MutArray.Generic (MutArray(..))
34−
35−-- import qualified Streamly.Internal.Data.Stream.Type as Stream
36−import qualified Streamly.Internal.Data.Fold.Type as Fold
37−import qualified Streamly.Internal.Data.MutArray.Generic as MutArray
38−
39−-- XXX Use MutableArray rather than keeping a MutArray here.
40−data Ring a = Ring
41− { ringArr :: MutArray a
42− -- XXX We can keep the current fill amount, Or we can keep a count of total
43− -- elements inserted and compute ring head as well using mod on that,
44− -- assuming it won't overflow. But mod could be expensive.
45− , ringHead :: !Int -- current index to be over-written
46− , ringMax :: !Int -- first index beyond allocated memory
47− }
48−
49−-------------------------------------------------------------------------------
50−-- Generation
51−-------------------------------------------------------------------------------
52−
53−-- XXX If we align the ringMax to nearest power of two then computation of the
54−-- index to write could be cheaper.
55−{-# INLINE createRing #-}
56−createRing :: MonadIO m => Int -> m (Ring a)
57−createRing count = liftIO $ do
58− arr <- MutArray.new count
59− arr1 <- MutArray.uninit arr count
60− return (Ring
61− { ringArr = arr1
62− , ringHead = 0
63− , ringMax = count
64− })
65−
66−
67−-- | Note that it is not safe to return a reference to the mutable Ring using a
68−-- scan as the Ring is continuously getting mutated. You could however copy out
69−-- the Ring.
70−{-# INLINE writeLastN #-}
71−writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
72−writeLastN n = Fold step initial extract extract
73−
74− where
75−
76− initial = do
77− if n <= 0
78− then Fold.Done <$> createRing 0
79− else do
80− rb <- createRing n
81− return $ Fold.Partial $ Tuple' rb (0 :: Int)
82−
83− step (Tuple' rb cnt) x = do
84− rh1 <- liftIO $ unsafeInsertRingWith rb x
85− return $ Fold.Partial $ Tuple' (rb {ringHead = rh1}) (cnt + 1)
86−
87− extract (Tuple' rb@Ring{..} cnt) =
88− return $
89− if cnt < ringMax
90− then Ring ringArr 0 ringHead
91− else rb
92−
93−-------------------------------------------------------------------------------
94−-- Modification
95−-------------------------------------------------------------------------------
96−
97−-- XXX This is safe
98−-- Take the ring head and return the new ring head.
99−{-# INLINE unsafeInsertRingWith #-}
100−unsafeInsertRingWith :: Ring a -> a -> IO Int
101−unsafeInsertRingWith Ring{..} x = do
102− assertM(ringMax >= 1)
103− assertM(ringHead < ringMax)
104− MutArray.putIndexUnsafe ringHead ringArr x
105− let rh1 = ringHead + 1
106− next = if rh1 == ringMax then 0 else rh1
107− return next
108−
109−-- | Move the ring head clockwise (+ve adj) or counter clockwise (-ve adj) by
110−-- the given amount.
111−{-# INLINE seek #-}
112−seek :: MonadIO m => Int -> Ring a -> m (Ring a)
113−seek adj rng@Ring{..}
114− | ringMax > 0 = liftIO $ do
115− -- XXX try avoiding mod when in bounds
116− let idx1 = ringHead + adj
117− next = mod idx1 ringMax
118− return $ Ring ringArr next ringMax
119− | otherwise = pure rng
120−
121−-------------------------------------------------------------------------------
122−-- Conversion
123−-------------------------------------------------------------------------------
124−
125−-- | @toMutArray rignHeadAdjustment lengthToRead ring@.
126−-- Convert the ring into a boxed mutable array. Note that the returned MutArray
127−-- shares the same underlying memory as the Ring, the user of this API needs to
128−-- ensure that the ring is not mutated during and after the conversion.
129−--
130−{-# INLINE toMutArray #-}
131−toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
132−toMutArray adj n Ring{..} =
133− -- XXX for empty Ring it will raise an Exception: divide by zero
134− if ringMax <= 0
135− then MutArray.nil
136− else do
137− let len = min ringMax n
138− let idx = mod (ringHead + adj) ringMax
139− end = idx + len
140− if end <= ringMax
141− then
142− return $ ringArr { arrStart = idx, arrLen = len }
143− else do
144− -- XXX Just swap the elements in the existing ring and return the
145− -- same array without reallocation.
146− arr <- liftIO $ MutArray.new len
147− arr1 <- MutArray.uninit arr len
148− MutArray.putSliceUnsafe ringArr idx arr1 0 (ringMax - idx)
149− MutArray.putSliceUnsafe ringArr 0 arr1 (ringMax - idx) (end - ringMax)
150− return arr1
151−
152−-- | Copy out the mutable ring to a mutable Array.
153−{-# INLINE copyToMutArray #-}
154−copyToMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
155−copyToMutArray adj n Ring{..} = do
156− if ringMax <= 0
157− then MutArray.nil
158− else do
159− let len = min ringMax n
160− let idx = mod (ringHead + adj) ringMax
161− end = idx + len
162− arr <- MutArray.new len
163− arr1 <- MutArray.uninit arr len
164− MutArray.putSliceUnsafe ringArr idx arr1 0 (ringMax - idx)
165− MutArray.putSliceUnsafe ringArr 0 arr1 (ringMax - idx) (end - ringMax)
166− return arr1
167−
168−-- This would be theoretically slower than toMutArray because of a branch
169−-- introduced for each element in the second half of the ring.
170−
171−-- | Seek by n and then read the entire ring. Use 'take' on the stream to
172−-- restrict the reads.
173−toStreamWith :: Int -> Ring a -> Stream m a
174−toStreamWith = undefined
175−{-
176−toStreamWith n Ring{..}
177− | ringMax > 0 = concatEffect $ liftIO $ do
178− idx <- readIORef ringHead
179− let idx1 = idx + adj
180− next = mod idx1 ringMax
181− s1 = undefined -- stream initial slice
182− s2 = undefined -- stream next slice
183− return (s1 `Stream.append` s2)
184− | otherwise = Stream.nil
185−-}
core/src/Streamly/Internal/Data/Ring/Unboxed.hsnew file+616

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize.hsnew file+157

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH.hsdeleted−530

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hsdeleted−475

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/Common.hsdeleted−70

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/RecHeader.hsdeleted−411

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/Type.hsdeleted−317

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream.hs+20−20

This diff is too big to show here.

core/src/Streamly/Internal/Data/Array/Stream.hs → core/src/Streamly/Internal/Data/Stream/Chunked.hsrenamed+587−74

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Common.hs → core/src/Streamly/Internal/Data/Stream/Common.hsrenamed+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/MutChunked.hsnew file+323

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Container.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Container.hsrenamed+57−65

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Eliminate.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Eliminate.hsrenamed+33−39

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Exception.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Exception.hsrenamed+9−12

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Generate.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Generate.hsrenamed+26−27

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Lift.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Lift.hsrenamed+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Nesting.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Nesting.hsrenamed+228−115

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Step.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Step.hsrenamed+2−2

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Top.hsnew file+353

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Transform.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Transform.hsrenamed+40−66

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Transformer.hsrenamed+4−5

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Type.hsrenamed+34−169

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Alt.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Alt.hsrenamed+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Transformer.hsrenamed+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Type.hsrenamed+41−95

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Top.hsdeleted−480

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK.hs+134−350

This diff is too big to show here.

core/src/Streamly/Internal/Data/Time/Clock.hs+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Time/Units.hs+3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unbox.hs+260−195

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unbox/TH.hs+82−85

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold.hs+58−20

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold/Enumeration.hs+1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold/Type.hs+8−4

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/File.hs+13−11

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Handle.hs+30−23

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Path.hsdeleted−515

This diff is too big to show here.

core/src/Streamly/Internal/Data/Binary/Parser.hs → core/src/Streamly/Internal/Serialize/FromBytes.hsrenamed+4−10

This diff is too big to show here.

core/src/Streamly/Internal/Data/Binary/Stream.hs → core/src/Streamly/Internal/Serialize/ToBytes.hsrenamed+2−12

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Parser.hs−3

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Stream.hs+15−43

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/String.hs+7−7

This diff is too big to show here.

core/streamly-core.cabal+75−137

This diff is too big to show here.

docs/Developer/optimization-guidelines.md−12

This diff is too big to show here.

docs/Developer/paths.rst+2−25

This diff is too big to show here.

docs/User/HowTo/Compiling.md+3−5

This diff is too big to show here.

docs/User/Project/ApiChangelogs/0.9.0-0.10.0.txtdeleted−424

This diff is too big to show here.

docs/User/Project/CONTRIBUTORS.md−8

This diff is too big to show here.

docs/User/Project/Changelog.md+10−25

This diff is too big to show here.

docs/User/Project/README.md+2−9

This diff is too big to show here.

docs/User/Project/Upgrading-0.8-to-0.9.md → docs/User/Project/Upgrading.mdrenamed+9−87

This diff is too big to show here.

docs/User/Tutorials/quick-overview.md+14−10

This diff is too big to show here.

docs/streamly-docs.cabal+1−1

This diff is too big to show here.

hie.yaml+2−2

This diff is too big to show here.

src/Streamly.hsnew file+625

This diff is too big to show here.

src/Streamly/Data/Fold/Prelude.hs+13−31

This diff is too big to show here.

src/Streamly/Data/Fold/Tee.hs+1−1

This diff is too big to show here.

src/Streamly/Data/Stream/Concurrent.hsnew file+194

This diff is too big to show here.

src/Streamly/Data/Stream/Exception.hsnew file+24

This diff is too big to show here.

src/Streamly/Data/Stream/MkType.hs+10−41

This diff is too big to show here.

src/Streamly/Data/Stream/Prelude.hs+15−217

This diff is too big to show here.

src/Streamly/Data/Stream/Time.hsnew file+37

This diff is too big to show here.

src/Streamly/Data/Unicode/Stream.hsnew file+99

This diff is too big to show here.

src/Streamly/Internal/Data/Array/ArrayMacros.hnew file+40

This diff is too big to show here.

src/Streamly/Internal/Data/Channel.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Async.hs+2−5

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent.hs+44−1

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent/Channel.hs+10−29

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent/Channel/Type.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Prelude.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/SVar.hs+5−6

This diff is too big to show here.

src/Streamly/Internal/Data/IsMap/HashMap.hs−6

This diff is too big to show here.

src/Streamly/Internal/Data/SVar.hs−1

This diff is too big to show here.

src/Streamly/Internal/Data/SmallArray.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Ahead.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Async.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Dispatcher.hs → src/Streamly/Internal/Data/Stream/Channel/Dispatcher.hsrenamed+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Types.hs → src/Streamly/Internal/Data/Stream/Channel/Types.hsrenamed+37−12

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Worker.hs → src/Streamly/Internal/Data/Stream/Channel/Worker.hsrenamed+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent.hs+42−51

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel.hs+14−17

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Append.hs+11−11

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Consumer.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Dispatcher.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Interleave.hs+9−9

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Operations.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Type.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Lifted.hs → src/Streamly/Internal/Data/Stream/Exception/Lifted.hsrenamed+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream.hs−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Combinators.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Common.hs+5−5

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Exception.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Expand.hs+3−7

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Generate.hs+6−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Reduce.hs+6−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Top.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Transform.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Type.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/MkType.hs+34−33

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Parallel.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Prelude.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar.hsdeleted−18

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar/Eliminate.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar/Generate.hs+1−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Serial.hs+2−7

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Time.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Zip.hs+2−41

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Zip/Concurrent.hsnew file+50

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/ZipAsync.hs+2−3

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/Exception.hs+3−1

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/Prelude.hsdeleted−17

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/SVar.hs+1−2

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event.hs+2−5

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Darwin.hs+5−5

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Linux.hs+9−15

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Windows.hs+8−11

This diff is too big to show here.

src/Streamly/Internal/FileSystem/FD.hs+10−11

This diff is too big to show here.

src/Streamly/Internal/Network/Inet/TCP.hs+46−81

This diff is too big to show here.

src/Streamly/Internal/Network/Socket.hs+30−33

This diff is too big to show here.

src/Streamly/Internal/System/IOVec.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Unicode/Char.hs+7−9

This diff is too big to show here.

src/Streamly/Memory/Array.hsnew file+74

This diff is too big to show here.

src/Streamly/Network/Inet/TCP.hs+30−41

This diff is too big to show here.

src/Streamly/Network/Socket.hs+30−16

This diff is too big to show here.

streamly.cabal+62−84

This diff is too big to show here.

targets/Targets.hs+3−8

This diff is too big to show here.

test/Streamly/Test/Data/Array.hs+8−7

This diff is too big to show here.

test/Streamly/Test/Data/Array/Stream.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Data/Fold.hs+6−7

This diff is too big to show here.

test/Streamly/Test/Data/Fold/Window.hs+9−8

This diff is too big to show here.

test/Streamly/Test/Data/MutArray.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Data/Parser.hs+13−18

This diff is too big to show here.

test/Streamly/Test/Data/ParserK.hs+9−13

This diff is too big to show here.

test/Streamly/Test/Data/Ring/Unboxed.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Data/Serialize.hs+13−251

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/CompatV0.hsdeleted−46

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/CompatV1.hsdeleted−48

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/TH.hsdeleted−82

This diff is too big to show here.

test/Streamly/Test/Data/Stream/Concurrent.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Data/Unbox.hs+26−125

This diff is too big to show here.

test/Streamly/Test/Data/Unbox/TH.hs+10−11

This diff is too big to show here.

test/Streamly/Test/FileSystem/Event/Common.hs+1−1

This diff is too big to show here.

test/Streamly/Test/FileSystem/Handle.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Prelude/Serial.hs+3−7

This diff is too big to show here.

test/Streamly/Test/Serialize/Serializable.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Unicode/Char.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Unicode/Parser.hs+16−6

This diff is too big to show here.

test/Streamly/Test/Unicode/Stream.hs+1−1

This diff is too big to show here.

test/lib/Streamly/Test/Prelude/Common.hs+2−9

This diff is too big to show here.

test/streamly-tests.cabal+11−38

This diff is too big to show here.