git.iko.soy/streamly
on d304fb8 help sign in
30c5a56
swap
0 ahead, 343 behind
277 files changed+12031−18831
since splitsnapshots
unifiedsplit
.circleci/config.yml+277−73 .github/workflows/haskell.yml+91−84 .github/workflows/regression-check.yml+2−2 .hlint.ignore−5 benchmark/NanoBenchmarks.hs+1−1 benchmark/Streamly/Benchmark/Data/Array/CommonImports.hs+1−1 benchmark/Streamly/Benchmark/Data/Array/Stream.hs+3−3 benchmark/Streamly/Benchmark/Data/Fold.hs+5−28 benchmark/Streamly/Benchmark/Data/Fold/Window.hs+40−40 benchmark/Streamly/Benchmark/Data/MutArray.hs+1−1 benchmark/Streamly/Benchmark/Data/Parser.hs+10−10 benchmark/Streamly/Benchmark/Data/ParserK.hs+37−90 benchmark/Streamly/Benchmark/Data/Ring/Unboxed.hs+4−3 benchmark/Streamly/Benchmark/Data/Serialize.hs−591 benchmark/Streamly/Benchmark/Data/Serialize/RecCompatible.hs−24 benchmark/Streamly/Benchmark/Data/Serialize/RecNonCompatible.hs−21 benchmark/Streamly/Benchmark/Data/Serialize/TH.hs−76 benchmark/Streamly/Benchmark/Data/Stream/Common.hs+1−3 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentCommon.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentEager.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentInterleaved.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentOrdered.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Eliminate.hs+1−3 benchmark/Streamly/Benchmark/Data/Stream/Exceptions.hs+3−12 benchmark/Streamly/Benchmark/Data/Stream/Expand.hs+1−9 benchmark/Streamly/Benchmark/Data/Stream/Generate.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Lift.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/Reduce.hs+2−2 benchmark/Streamly/Benchmark/Data/Stream/Split.hs+2−2 benchmark/Streamly/Benchmark/Data/Stream/StreamD.hs+2−4 benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs+3−4 benchmark/Streamly/Benchmark/Data/Stream/StreamKAlt.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/ToStreamK.hs+4−3 benchmark/Streamly/Benchmark/Data/Stream/Transform.hs+2−2 benchmark/Streamly/Benchmark/Data/Unbox.hs+102 benchmark/Streamly/Benchmark/Data/Unfold.hs+2−11 benchmark/Streamly/Benchmark/FileSystem/Handle.hs+1−1 benchmark/Streamly/Benchmark/FileSystem/Handle/Read.hs+9−8 benchmark/Streamly/Benchmark/FileSystem/Handle/ReadWrite.hs+10−9 benchmark/Streamly/Benchmark/Prelude/Adaptive.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Ahead.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Async.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Concurrent.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Merge.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Parallel.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Rate.hs+1−1 benchmark/Streamly/Benchmark/Prelude/WAsync.hs+1−1 benchmark/Streamly/Benchmark/Prelude/WSerial.hs+1−1 benchmark/Streamly/Benchmark/Prelude/ZipAsync.hs+1−1 benchmark/Streamly/Benchmark/Prelude/ZipSerial.hs+1−1 benchmark/Streamly/Benchmark/Unicode/Char.hs+1−1 benchmark/Streamly/Benchmark/Unicode/Parser.hs+1−1 benchmark/Streamly/Benchmark/Unicode/Stream.hs+8−26 benchmark/Streamly/Benchmark/Unicode/Utf8.hs+1−1 benchmark/bench-runner/Main.hs+6−34 benchmark/lib/Streamly/Benchmark/Common.hs+1−1 benchmark/lib/Streamly/Benchmark/Common/Handle.hs+1−3 benchmark/lib/Streamly/Benchmark/Prelude.hs+1−1 benchmark/streamly-benchmarks.cabal+40−64 configure+9−9 configure.ac+1−1 core/configure+9−9 core/configure.ac+1−1 core/docs/ApiChangelogs/0.1.0-0.2.0.txt−1471 core/docs/ApiChangelogs/0.2.0-0.2.2.txt−306 core/docs/Changelog.md+12−60 core/src/DocTestDataArray.hs−8 core/src/DocTestDataFold.hs+1 core/src/DocTestDataParserK.hs−15 core/src/DocTestDataStream.hs+1−2 core/src/DocTestFileSystemHandle.hs−15 core/src/DocTestUnicodeParser.hs−9 core/src/DocTestUnicodeStream.hs−12 core/src/DocTestUnicodeString.hs−5 core/src/Streamly/Console/Stdio.hs+27−1 core/src/Streamly/Data/Array.hs+48−48 core/src/Streamly/Data/Array/Generic.hs+13−16 core/src/Streamly/Data/Fold.hs+119−150 core/src/Streamly/Data/MutArray.hs+12−18 core/src/Streamly/Data/MutArray/Generic.hs+3−8 core/src/Streamly/Data/MutByteArray.hs−106 core/src/Streamly/Data/Parser.hs+60−130 core/src/Streamly/Data/ParserK.hs+10−105 core/src/Streamly/Data/Stream.hs+107−199 core/src/Streamly/Data/StreamK.hs+19−37 core/src/Streamly/Data/Unbox.hs+16 core/src/Streamly/Data/Unfold.hs+1−3 core/src/Streamly/FileSystem/Handle.hs+29−15 core/src/Streamly/Internal/Data/Array.hs+163−610 core/src/Streamly/Internal/Data/Array/Generic.hs+11−49 core/src/Streamly/Internal/Data/Array/Type.hs+108−538 core/src/Streamly/Internal/Data/Builder.hs+9−11 core/src/Streamly/Internal/Data/Fold.hs+2574−9 core/src/Streamly/Internal/Data/Fold/Chunked.hs+18−20 core/src/Streamly/Internal/Data/Fold/Combinators.hs−2452 core/src/Streamly/Internal/Data/Fold/Container.hs+32−75 core/src/Streamly/Internal/Data/Fold/Step.hs+2−2 core/src/Streamly/Internal/Data/Fold/Tee.hs−2 core/src/Streamly/Internal/Data/Fold/Type.hs+120−344 core/src/Streamly/Internal/Data/Fold/Window.hs+53−59 core/src/Streamly/Internal/Data/IORef/Unboxed.hs+20−23 core/src/Streamly/Internal/Data/IsMap.hs−8 core/src/Streamly/Internal/Data/List.hs+5−5 core/src/Streamly/Internal/Data/MutArray.hs+33−154 core/src/Streamly/Internal/Data/MutArray/Generic.hs+49−172 core/src/Streamly/Internal/Data/MutArray/Stream.hs−145 core/src/Streamly/Internal/Data/MutArray/Type.hs+441−1308 core/src/Streamly/Internal/Data/MutByteArray.hs−251 core/src/Streamly/Internal/Data/MutByteArray/Type.hs−321 core/src/Streamly/Internal/Data/Parser.hs+143−135 core/src/Streamly/Internal/Data/Parser/Tee.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Tee.hs+1−1 core/src/Streamly/Internal/Data/Parser/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Type.hs+71−94 core/src/Streamly/Internal/Data/Parser/ParserDK.hs+1544 core/src/Streamly/Internal/Data/ParserK/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserK/Type.hs+84−291 core/src/Streamly/Internal/Data/ParserK.hs−16 core/src/Streamly/Internal/Data/Pipe.hs+8−5 core/src/Streamly/Internal/Data/Producer.hs+13−6 core/src/Streamly/Internal/Data/Producer/Source.hs+1−1 core/src/Streamly/Internal/Data/Producer/Type.hs+1−1 core/src/Streamly/Internal/Data/Refold/Type.hs+1−1 core/src/Streamly/Internal/Data/Ring.hs+134−589 core/src/Streamly/Internal/Data/Ring/Generic.hs−185 core/src/Streamly/Internal/Data/Ring/Unboxed.hs+616 core/src/Streamly/Internal/Data/Serialize.hs+157 core/src/Streamly/Internal/Data/Serialize/TH.hs−530 core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hs−475 core/src/Streamly/Internal/Data/Serialize/TH/Common.hs−70 core/src/Streamly/Internal/Data/Serialize/TH/RecHeader.hs−411 core/src/Streamly/Internal/Data/Serialize/Type.hs−317 core/src/Streamly/Internal/Data/Stream.hs+20−20 core/src/Streamly/Internal/Data/Array/Stream.hs → core/src/Streamly/Internal/Data/Stream/Chunked.hs+587−74 src/Streamly/Internal/Data/Stream/Common.hs → core/src/Streamly/Internal/Data/Stream/Common.hs+3−3 core/src/Streamly/Internal/Data/Stream/MutChunked.hs+323 core/src/Streamly/Internal/Data/Stream/Container.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Container.hs+57−65 core/src/Streamly/Internal/Data/Stream/Eliminate.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Eliminate.hs+33−39 core/src/Streamly/Internal/Data/Stream/Exception.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Exception.hs+9−12 core/src/Streamly/Internal/Data/Stream/Generate.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Generate.hs+26−27 core/src/Streamly/Internal/Data/Stream/Lift.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Lift.hs+3−3 core/src/Streamly/Internal/Data/Stream/Nesting.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Nesting.hs+228−115 core/src/Streamly/Internal/Data/Stream/Step.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Step.hs+2−2 core/src/Streamly/Internal/Data/Stream/StreamD/Top.hs+353 core/src/Streamly/Internal/Data/Stream/Transform.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Transform.hs+40−66 core/src/Streamly/Internal/Data/Stream/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Transformer.hs+4−5 core/src/Streamly/Internal/Data/Stream/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Type.hs+34−169 core/src/Streamly/Internal/Data/StreamK/Alt.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Alt.hs+1−1 core/src/Streamly/Internal/Data/StreamK/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Transformer.hs+3−3 core/src/Streamly/Internal/Data/StreamK/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Type.hs+41−95 core/src/Streamly/Internal/Data/Stream/Top.hs−480 core/src/Streamly/Internal/Data/StreamK.hs+134−350 core/src/Streamly/Internal/Data/Time/Clock.hs+3−3 core/src/Streamly/Internal/Data/Time/Units.hs+3 core/src/Streamly/Internal/Data/Unbox.hs+280−295 core/src/Streamly/Internal/Data/Unbox/TH.hs+82−85 core/src/Streamly/Internal/Data/Unfold.hs+58−20 core/src/Streamly/Internal/Data/Unfold/Enumeration.hs+1 core/src/Streamly/Internal/Data/Unfold/Type.hs+8−4 core/src/Streamly/Internal/FileSystem/File.hs+13−11 core/src/Streamly/Internal/FileSystem/Handle.hs+42−33 core/src/Streamly/Internal/FileSystem/Path.hs−515 core/src/Streamly/Internal/Data/Binary/Parser.hs → core/src/Streamly/Internal/Serialize/FromBytes.hs+4−10 core/src/Streamly/Internal/Data/Binary/Stream.hs → core/src/Streamly/Internal/Serialize/ToBytes.hs+2−12 core/src/Streamly/Internal/Unicode/Array.hs−6 core/src/Streamly/Internal/Unicode/Parser.hs+4−12 core/src/Streamly/Internal/Unicode/Stream.hs+25−53 core/src/Streamly/Internal/Unicode/String.hs+12−17 core/src/Streamly/Unicode/Parser.hs+2−11 core/src/Streamly/Unicode/Stream.hs−9 core/src/Streamly/Unicode/String.hs+1−11 core/streamly-core.cabal+78−146 docs/Developer/optimization-guidelines.md−12 docs/Developer/paths.rst+2−25 docs/User/HowTo/Compiling.md+3−5 docs/User/Project/ApiChangelogs/0.9.0-0.10.0.txt−424 docs/User/Project/CONTRIBUTORS.md−8 docs/User/Project/Changelog.md+10−25 docs/User/Project/README.md+2−9 docs/User/Project/Upgrading-0.8-to-0.9.md → docs/User/Project/Upgrading.md+9−87 docs/User/Tutorials/quick-overview.md+14−10 docs/streamly-docs.cabal+1−1 hie.yaml+2−2 src/Streamly.hs+624 src/Streamly/Data/Fold/Prelude.hs+14−32 src/Streamly/Data/Fold/Tee.hs+1−1 src/Streamly/Data/Stream/Concurrent.hs+194 src/Streamly/Data/Stream/Exception.hs+24 src/Streamly/Data/Stream/MkType.hs+10−41 src/Streamly/Data/Stream/Prelude.hs+15−217 src/Streamly/Data/Stream/Time.hs+37 src/Streamly/Data/Unicode/Stream.hs+99 src/Streamly/Internal/Data/Array/ArrayMacros.h+40 src/Streamly/Internal/Data/Channel.hs−19 src/Streamly/Internal/Data/Fold/Async.hs+3−6 src/Streamly/Internal/Data/Fold/Concurrent.hs+44−1 src/Streamly/Internal/Data/Fold/Concurrent/Channel.hs+10−29 src/Streamly/Internal/Data/Fold/Concurrent/Channel/Type.hs+3−3 src/Streamly/Internal/Data/Fold/Prelude.hs−19 src/Streamly/Internal/Data/Fold/SVar.hs+7−8 src/Streamly/Internal/Data/IsMap/HashMap.hs−6 src/Streamly/Internal/Data/SVar.hs−1 src/Streamly/Internal/Data/SmallArray.hs+3−3 src/Streamly/Internal/Data/Stream/Ahead.hs+2−2 src/Streamly/Internal/Data/Stream/Async.hs+2−2 src/Streamly/Internal/Data/Channel/Dispatcher.hs → src/Streamly/Internal/Data/Stream/Channel/Dispatcher.hs+3−3 src/Streamly/Internal/Data/Channel/Types.hs → src/Streamly/Internal/Data/Stream/Channel/Types.hs+37−12 src/Streamly/Internal/Data/Channel/Worker.hs → src/Streamly/Internal/Data/Stream/Channel/Worker.hs+3−3 src/Streamly/Internal/Data/Stream/Concurrent.hs+42−51 src/Streamly/Internal/Data/Stream/Concurrent/Channel.hs+14−17 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Append.hs+11−11 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Consumer.hs+2−2 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Dispatcher.hs+3−3 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Interleave.hs+9−9 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Operations.hs+2−2 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Type.hs+4−4 src/Streamly/Internal/Data/Stream/Lifted.hs → src/Streamly/Internal/Data/Stream/Exception/Lifted.hs+2−2 src/Streamly/Internal/Data/Stream/IsStream.hs−6 src/Streamly/Internal/Data/Stream/IsStream/Combinators.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Common.hs+6−6 src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs+2−2 src/Streamly/Internal/Data/Stream/IsStream/Exception.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Expand.hs+3−7 src/Streamly/Internal/Data/Stream/IsStream/Generate.hs+6−6 src/Streamly/Internal/Data/Stream/IsStream/Reduce.hs+7−7 src/Streamly/Internal/Data/Stream/IsStream/Top.hs+3−2 src/Streamly/Internal/Data/Stream/IsStream/Transform.hs+3−3 src/Streamly/Internal/Data/Stream/IsStream/Type.hs+2−2 src/Streamly/Internal/Data/Stream/MkType.hs+34−33 src/Streamly/Internal/Data/Stream/Parallel.hs+1−1 src/Streamly/Internal/Data/Stream/Prelude.hs−19 src/Streamly/Internal/Data/Stream/SVar.hs−18 src/Streamly/Internal/Data/Stream/SVar/Eliminate.hs+2−2 src/Streamly/Internal/Data/Stream/SVar/Generate.hs+1−3 src/Streamly/Internal/Data/Stream/Serial.hs+2−7 src/Streamly/Internal/Data/Stream/Time.hs+3−3 src/Streamly/Internal/Data/Stream/Zip.hs+2−41 src/Streamly/Internal/Data/Stream/Zip/Concurrent.hs+50 src/Streamly/Internal/Data/Stream/ZipAsync.hs+2−3 src/Streamly/Internal/Data/Unfold/Exception.hs+3−1 src/Streamly/Internal/Data/Unfold/Prelude.hs−17 src/Streamly/Internal/Data/Unfold/SVar.hs+1−2 src/Streamly/Internal/FileSystem/Event.hs+2−5 src/Streamly/Internal/FileSystem/Event/Darwin.hs+5−5 src/Streamly/Internal/FileSystem/Event/Linux.hs+9−15 src/Streamly/Internal/FileSystem/Event/Windows.hs+8−11 src/Streamly/Internal/FileSystem/FD.hs+10−11 src/Streamly/Internal/Network/Inet/TCP.hs+47−82 src/Streamly/Internal/Network/Socket.hs+30−33 src/Streamly/Internal/System/IOVec.hs+3−3 src/Streamly/Internal/Unicode/Char.hs+7−9 src/Streamly/Memory/Array.hs+74 src/Streamly/Network/Inet/TCP.hs+30−41 src/Streamly/Network/Socket.hs+30−16 streamly.cabal+64−86 targets/Targets.hs+3−9 test/Streamly/Test/Data/Array.hs+8−7 test/Streamly/Test/Data/Array/Stream.hs+1−1 test/Streamly/Test/Data/Fold.hs+7−7 test/Streamly/Test/Data/Fold/Window.hs+9−8 test/Streamly/Test/Data/MutArray.hs+1−1 test/Streamly/Test/Data/Parser.hs+13−18 test/Streamly/Test/Data/ParserK.hs+9−13 test/Streamly/Test/Data/Ring/Unboxed.hs+2−2 test/Streamly/Test/Data/Serialize.hs+13−251 test/Streamly/Test/Data/Serialize/CompatV0.hs−46 test/Streamly/Test/Data/Serialize/CompatV1.hs−48 test/Streamly/Test/Data/Serialize/TH.hs−82 test/Streamly/Test/Data/Stream/Concurrent.hs+1−1 test/Streamly/Test/Data/Unbox.hs+26−125 test/Streamly/Test/Data/Unbox/TH.hs+10−11 test/Streamly/Test/FileSystem/Event/Common.hs+1−1 test/Streamly/Test/FileSystem/Handle.hs+2−2 test/Streamly/Test/Prelude/Serial.hs+3−7 test/Streamly/Test/Serialize/Serializable.hs+2−2 test/Streamly/Test/Unicode/Char.hs+1−1 test/Streamly/Test/Unicode/Parser.hs+16−6 test/Streamly/Test/Unicode/Stream.hs+1−1 test/lib/Streamly/Test/Prelude/Common.hs+2−9 test/streamly-tests.cabal+11−38
.circleci/config.yml+277−73
@@ -1,7 +1,7 @@
11version: 2.1
22
33#-----------------------------------------------------------------------------
4−# packcheck-0.7.0
4+# packcheck-0.4.2
55# Packcheck global environment variables
66#-----------------------------------------------------------------------------
77
@@ -11,8 +11,6 @@ env: &env
1111 # Common options
1212 # ------------------------------------------------------------------------
1313 # GHC_OPTIONS: "-Werror"
14− # For updating see: https://downloads.haskell.org/~ghcup/
15− GHCUP_VERSION: 0.1.20.0
1614 CABAL_REINIT_CONFIG: "y"
1715 LC_ALL: "C.UTF-8"
1816
@@ -24,6 +22,7 @@ env: &env
2422 # DISABLE_DOCS: "y"
2523 # DISABLE_SDIST_BUILD: "y"
2624 # DISABLE_DIST_CHECKS: "y"
25+ ENABLE_INSTALL: "y"
2726
2827 # ------------------------------------------------------------------------
2928 # stack options
@@ -39,12 +38,15 @@ env: &env
3938 # cabal options
4039 # ------------------------------------------------------------------------
4140 CABAL_CHECK_RELAX: "y"
41+ CABAL_NO_SANDBOX: "y"
42+ CABAL_HACKAGE_MIRROR: "hackage.haskell.org:http://hackage.fpcomplete.com"
4243 CABAL_BUILD_OPTIONS: "-j1 --flag limit-build-mem"
4344
4445 # ------------------------------------------------------------------------
4546 # Where to find the required tools
4647 # ------------------------------------------------------------------------
47− PATH: /sbin:/usr/sbin:/bin:/usr/bin
48+ PATH: /opt/ghc/bin:/opt/ghcjs/bin:/sbin:/usr/sbin:/bin:/usr/bin
49+ TOOLS_DIR: /opt
4850
4951 # ------------------------------------------------------------------------
5052 # Location of packcheck.sh (the shell script invoked to perform CI tests ).
@@ -57,15 +59,15 @@ env: &env
5759 # If you have not committed packcheck.sh in your repo at PACKCHECK
5860 # then it is automatically pulled from this URL.
5961 PACKCHECK_GITHUB_URL: "https://raw.githubusercontent.com/composewell/packcheck"
60− PACKCHECK_GITHUB_COMMIT: "f6cfd7b7a33b4bf6fd63cce6a3fb3d26f5031403"
62+ PACKCHECK_GITHUB_COMMIT: "a302772c8f3791c477edd1764b82f8d1f5b22fdf"
6163
6264executors:
6365 amd64-executor:
6466 docker:
65− - image: ubuntu:latest
67+ - image: ubuntu:xenial
6668 x86-executor:
6769 docker:
68− - image: i386/ubuntu:focal
70+ - image: i386/ubuntu:eoan
6971
7072#-----------------------------------------------------------------------------
7173# Common utility stuff, not to be modified usually
@@ -77,50 +79,38 @@ preinstall: &preinstall
7779 # required for https/cache save and restore
7880 apt-get install -y ca-certificates
7981
80− # Alternative way of installing ghc and cabal, directly from
81− # haskell.org instead of using ghcup. NOTE: this is for Debian
82− # only and is debian release specific.
83− # gnupg is required for apt-key to work
84− #apt-get install -y gnupg
85− #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574
86− #echo "deb http://downloads.haskell.org/debian buster main" >> /etc/apt/sources.list
87− #apt-get update
88−
89− # required for outbound https for stack and for stack setup
90− apt-get install -y netbase xz-utils make
91− apt-get install -y zlib1g-dev
82+ # required to (re)generate the configure script
83+ apt-get install -y autoconf
9284
93− # For ghcup to install ghc
94− if test -n "$GHCUP_VERSION"
95− then
96− apt-get install -y gcc
97− apt-get install -y g++
98− fi
85+ # For ghc and cabal-install packages from hvr's ppa
86+ # gnupg is required for apt-key to work
87+ apt-get install -y gnupg
88+ apt-get install -y apt-transport-https
89+ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574
90+ apt-get install -y software-properties-common
91+ add-apt-repository -y ppa:hvr/ghc
92+ # echo "deb http://downloads.haskell.org/debian stretch main" >> /etc/apt/sources.list
93+ echo "deb-src http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main" >> /etc/apt/sources.list
94+ apt-get update
9995
100− # libgmp required by ghc for linking
101− apt-get install -y libgmp-dev
102− apt-get install -y libtinfo-dev
96+ # required for packcheck
97+ apt-get install -y curl
10398
104− # Required by cabal when git URL is specified in project file
99+ # required for circleci.
100+ apt-get install -y libcurl4-gnutls-dev
105101 apt-get install -y git
102+ apt-get install -y libtinfo-dev
106103
107− # Required for and by packcheck
108− apt-get install -y curl
104+ # required for outbound https for stack and for stack setup
105+ apt-get install -y netbase xz-utils make
109106
107+project-preinstall: &project-preinstall
108+ run: |
110109 # Get packcheck if needed
111− if test ! -e "$PACKCHECK"
112− then
113− if test -z "$PACKCHECK_GITHUB_COMMIT"
114− then
115− die "PACKCHECK_GITHUB_COMMIT is not specified."
116− fi
117− PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
118− curl --fail -sL -o "$PACKCHECK" $PACKCHECK_URL || exit 1
119− chmod +x $PACKCHECK
120− elif test ! -x "$PACKCHECK"
121− then
122− chmod +x $PACKCHECK
123− fi
110+ CURL=$(which curl)
111+ PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
112+ if test ! -e "$PACKCHECK"; then $CURL -sL -o "$PACKCHECK" $PACKCHECK_URL; fi;
113+ chmod +x $PACKCHECK
124114
125115restore: &restore
126116 # Needs to happen after installing ca-certificates
@@ -136,71 +126,271 @@ save: &save
136126 save_cache:
137127 key: v1-{{ .Environment.CIRCLE_JOB }}-{{ .Branch }}
138128 paths:
139− - ~/.local
140129 - ~/.cabal
130+ - ~/.ghc
131+ - ~/.local
141132 - ~/.stack
142− - ~/.ghcup
143133
144134#-----------------------------------------------------------------------------
145135# Build matrix
146136#-----------------------------------------------------------------------------
147137
148138jobs:
149− cabal-ghc-8_6_5:
139+ cabal-ghc-8_10_4:
150140 <<: *env
151141 executor: amd64-executor
152142 steps:
143+ - *preinstall
153144 - checkout
145+ - *project-preinstall
154146 - *restore
155− - *preinstall
156147 - run:
157148 environment:
158− GHCVER: "8.6.5"
159− CABAL_VERSION: "3.6.2.0"
160149 CABAL_PROJECT: "cabal.project"
150+ DISABLE_TEST: "yes"
151+ DISABLE_BENCH: "yes"
161152 DISABLE_SDIST_BUILD: "yes"
162− CABAL_BUILD_OPTIONS: "--flag debug --flag -opt"
163153 command: |
154+ apt-get install -y ghc-8.10.4
155+ apt-get install -y cabal-install-3.4
164156 bash -c "$PACKCHECK cabal"
165157 - *save
166− cabal-ghc-9_8_1-docspec:
158+ stack-ghc-8_8:
167159 <<: *env
168160 executor: amd64-executor
169161 steps:
162+ - *preinstall
170163 - checkout
164+ - *project-preinstall
171165 - *restore
166+ - run:
167+ name: GHC 8.8 + stack lts-16.12
168+ environment:
169+ BUILD: "stack"
170+ RESOLVER: "lts-16.12"
171+ SDIST_OPTIONS: "--ignore-check"
172+ STACK_BUILD_OPTIONS: "--flag streamly-benchmarks:-opt"
173+ command: |
174+ apt-get build-dep -y ghc-8.8.4
175+ apt-get install -y cabal-install-3.2
176+ bash -c "$PACKCHECK $BUILD"
177+ - *save
178+ cabal-ghc-8_6_5:
179+ <<: *env
180+ executor: amd64-executor
181+ steps:
172182 - *preinstall
183+ - checkout
184+ - *project-preinstall
185+ - *restore
186+ - run: |
187+ apt-get install -y ghc-8.6.3
188+ apt-get install -y cabal-install-2.4
189+ bash -c "$PACKCHECK cabal-v2"
190+ - *save
191+ cabal-ghc-8_4_4:
192+ <<: *env
193+ executor: x86-executor
194+ steps:
195+ - *preinstall
196+ - checkout
197+ - *project-preinstall
198+ - *restore
199+ - run:
200+ name: GHC 8.4.4 + x86 + debug
201+ environment:
202+ BUILD: "cabal-v2"
203+ GHCVER: "8.4.4"
204+ CABAL_BUILD_OPTIONS: "--flag debug --flag limit-build-mem"
205+ command: |
206+ apt-get install -y ghc-8.4.4
207+ apt-get install -y cabal-install-3.2
208+ bash -c "$PACKCHECK $BUILD"
209+ - *save
210+ cabal-ghc-8_2_2:
211+ <<: *env
212+ executor: amd64-executor
213+ steps:
214+ - *preinstall
215+ - checkout
216+ - *project-preinstall
217+ - *restore
173218 - run:
174219 environment:
175− GHCVER: "9.8.1"
176− CABAL_VERSION: "3.10.1.0"
177− CABAL_PROJECT: "cabal.project.doctest"
178− DISABLE_SDIST_BUILD: "y"
179− DISABLE_TEST: "y"
180− DISABLE_BENCH: "y"
181− DISABLE_DOCS: "y"
182− ENABLE_DOCSPEC: "y"
183− DOCSPEC_URL: https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz
184− DOCSPEC_OPTIONS: "--timeout 60 --check-properties --property-variables xs"
220+ BUILD: "cabal-v2"
221+ DISABLE_TEST: "yes"
222+ DISABLE_BENCH: "yes"
223+ DISABLE_DOCS: "yes"
224+ DISABLE_SDIST_BUILD: "yes"
225+ DISABLE_DIST_CHECKS: "yes"
185226 command: |
186− bash -c "$PACKCHECK cabal"
227+ apt-get install -y ghc-8.2.2
228+ apt-get install -y cabal-install-3.2
229+ bash -c "$PACKCHECK $BUILD"
187230 - *save
188− hlint-trailing-spaces:
231+ cabal-ghc-8_0_2:
189232 <<: *env
190233 executor: amd64-executor
191234 steps:
235+ - *preinstall
192236 - checkout
237+ - *project-preinstall
193238 - *restore
239+ - run:
240+ environment:
241+ BUILD: "cabal-v2"
242+ DISABLE_TEST: "yes"
243+ DISABLE_BENCH: "yes"
244+ DISABLE_DOCS: "yes"
245+ DISABLE_SDIST_BUILD: "yes"
246+ DISABLE_DIST_CHECKS: "yes"
247+ command: |
248+ apt-get install -y ghc-8.0.2
249+ apt-get install -y cabal-install-3.2
250+ bash -c "$PACKCHECK $BUILD"
251+ - *save
252+ cabal-ghc-7_10_3:
253+ <<: *env
254+ executor: amd64-executor
255+ steps:
194256 - *preinstall
257+ - checkout
258+ - *project-preinstall
259+ - *restore
260+ - run: |
261+ apt-get install -y ghc-7.10.3
262+ apt-get install -y cabal-install-2.4
263+ bash -c "$PACKCHECK cabal-v2"
264+ - *save
265+ stack-ghc-8_4:
266+ <<: *env
267+ executor: amd64-executor
268+ steps:
269+ - *preinstall
270+ - checkout
271+ - *project-preinstall
272+ - *restore
273+ - run: |
274+ bash -c "$PACKCHECK stack RESOLVER=lts-12"
275+ - *save
276+ stack-ghc-8_2:
277+ <<: *env
278+ executor: amd64-executor
279+ steps:
280+ - *preinstall
281+ - checkout
282+ - *project-preinstall
283+ - *restore
284+ - run: |
285+ bash -c "$PACKCHECK stack RESOLVER=lts-11"
286+ - *save
287+ cabal-ghcjs-8_4:
288+ <<: *env
289+ executor: amd64-executor
290+ steps:
291+ - *preinstall
292+ - checkout
293+ - *project-preinstall
294+ - *restore
195295 - run:
196− name: hlint and trailing spaces
296+ name: GHCJS 8.4+no-test+no-docs
197297 environment:
298+ BUILD: cabal-v2
299+ ENABLE_GHCJS: "yes"
300+ DISABLE_TEST: "yes"
301+ ENABLE_INSTALL: ""
302+ DISABLE_DOCS: "yes"
198303 DISABLE_SDIST_BUILD: "yes"
199− HLINT_VERSION: 3.6.1
304+ command: |
305+ add-apt-repository -y ppa:hvr/ghcjs
306+ add-apt-repository -y 'deb https://deb.nodesource.com/node_11.x xenial main'
307+ curl -sSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -
308+ apt-get update
309+ apt-get install -y cabal-install-3.2
310+ apt-get install -y nodejs
311+ apt-get install -y ghcjs-8.4
312+ cabal user-config update -a "jobs: 1"
313+ bash -c "$PACKCHECK $BUILD"
314+ - *save
315+ coveralls-ghc-8_2_2:
316+ <<: *env
317+ executor: amd64-executor
318+ steps:
319+ - *preinstall
320+ - checkout
321+ - *project-preinstall
322+ - *restore
323+ - run:
324+ environment:
325+ BUILD: cabal
326+ COVERALLS_OPTIONS: "--repo-token=\"$REPO_TOKEN\" --coverage-mode=StrictlyFullLines --exclude-dir=test test"
327+ GHC_OPTIONS: "-DCOVERAGE_BUILD"
328+ command: |
329+ apt-get install -y ghc-8.2.2
330+ apt-get install -y cabal-install-2.4
331+ bash -c "$PACKCHECK cabal-v1"
332+ - *save
333+ coveralls-ghc-8_8_3:
334+ <<: *env
335+ executor: amd64-executor
336+ steps:
337+ - *preinstall
338+ - checkout
339+ - *project-preinstall
340+ - *restore
341+ - run:
342+ environment:
343+ BUILD: cabal-v2
344+ COVERALLS_OPTIONS: "--repo-token=\"$REPO_TOKEN\" --coverage-mode=StrictlyFullLines --exclude-dir=test"
345+ CABAL_PROJECT: "cabal.project.coverage"
346+ CABAL_BUILD_TARGETS: ""
347+ command: |
348+ apt-get install -y ghc-8.8.3
349+ apt-get install -y cabal-install-3.0
350+ bash -c "$PACKCHECK cabal-v2"
351+ no_output_timeout: 25m
352+ - *save
353+ coveralls-ghc-8_10_2:
354+ <<: *env
355+ executor: amd64-executor
356+ steps:
357+ - *preinstall
358+ - checkout
359+ - *project-preinstall
360+ - *restore
361+ - run:
362+ name: Build and test with inspection, coverage, Werror
363+ command: |
364+ apt-get install -y ghc-8.10.2
365+ apt-get install -y cabal-install-3.2
366+ # Run tests with coverage
367+ cabal update
368+ cabal user-config update -a "jobs: 1"
369+ bin/test.sh --coverage --raw
370+ # Upload results to coveralls.io
371+ PATH=$HOME/.cabal/bin:$PATH
372+ export PATH
373+ which hpc-coveralls 2>/dev/null || cabal install --project-file cabal.project.hpc-coveralls hpc-coveralls
374+ hpc-coveralls --repo-token="$REPO_TOKEN" --coverage-mode=StrictlyFullLines
375+ no_output_timeout: 25m
376+ - *save
377+ hlint-src:
378+ <<: *env
379+ executor: amd64-executor
380+ steps:
381+ - *preinstall
382+ - checkout
383+ - *project-preinstall
384+ - *restore
385+ - run:
386+ name: Hlint src
387+ environment:
388+ BUILD: hlint
200389 HLINT_OPTIONS: lint
390+ DISABLE_SDIST_BUILD: "yes"
201391 HLINT_TARGETS: core/src src test benchmark
202392 command: |
203− bash -c "$PACKCHECK hlint" || exit 1
393+ bash -c "$PACKCHECK $BUILD" || exit 1
204394 echo "Checking trailing spaces..."
205395 count=$(find . -name "*.hs" -exec grep -H '\ $' {} \; | tee /dev/tty | wc -l)
206396 exit $count
@@ -210,9 +400,23 @@ workflows:
210400 version: 2
211401 build:
212402 jobs:
213− #- cabal-ghc-8_6_5:
214− # name: 8.6.5-debug-unoptimized
215− - cabal-ghc-9_8_1-docspec:
216− name: ghc-9.8.1-docspec
217− - hlint-trailing-spaces:
218− name: hlint and trailing spaces
403+ - cabal-ghc-8_10_4:
404+ name: GHC 8.10.4 + sdist + no-test + no-bench
405+ #- cabal-ghc-8.6.5
406+ #- cabal-ghc-8_4_4:
407+ # name: GHC 8.4.4 + x86 + debug
408+ #- cabal-ghc-8_2_2:
409+ # name: GHC 8.2.2 + no-test + no-bench + no-docs
410+ #- cabal-ghc-8_0_2:
411+ # name: GHC 8.0.2 + no-test + no-bench + no-docs
412+ #- cabal-ghc-7.10.3
413+ # - cabal-ghcjs-8_4:
414+ # name: GHCJS 8.4 + no-test + no-docs
415+ #- stack-ghc-8.4
416+ #- stack-ghc-8.2
417+ #- coveralls-ghc-8.2.2
418+ #- coveralls-ghc-8.8.3
419+ #- coveralls-ghc-8_10_2:
420+ # name: GHC 8.10.2 + inspection + coverage + Werror
421+ - hlint-src:
422+ name: Hlint src + Trailing Spaces
.github/workflows/haskell.yml+91−84
@@ -13,15 +13,9 @@ jobs:
1313 # packcheck environment variables
1414 LC_ALL: C.UTF-8
1515 BUILD: ${{ matrix.build }}
16− # For updating see: https://downloads.haskell.org/~ghcup/
17− GHCUP_VERSION: 0.1.20.0
18− DOCSPEC_URL: https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz
19− DOCSPEC_OPTIONS: "--timeout 60 --check-properties --property-variables xs"
20− GHCUP_GHC_OPTIONS: ${{ matrix.ghcup_ghc_options }}
2116 GHCVER: ${{ matrix.ghc_version }}
2217 CABALVER: ${{ matrix.cabal_version }}
2318 DISABLE_DOCS: ${{ matrix.disable_docs }}
24− ENABLE_DOCSPEC: ${{ matrix.enable_docspec }}
2519 DISABLE_TEST: ${{ matrix.disable_test }}
2620 DISABLE_BENCH: ${{ matrix.disable_bench }}
2721 DISABLE_DIST_CHECKS: ${{ matrix.disable_dist_checks }}
@@ -45,18 +39,14 @@ jobs:
4539 STACK_BUILD_OPTIONS: ${{ matrix.stack_build_options }}
4640
4741 # packcheck location and revision
48− PACKCHECK: "./packcheck.sh"
42+ PACKCHECK_LOCAL_PATH: "./packcheck.sh"
4943 PACKCHECK_GITHUB_URL: "https://raw.githubusercontent.com/composewell/packcheck"
50− PACKCHECK_GITHUB_COMMIT: "v0.7.0"
44+ PACKCHECK_GITHUB_COMMIT: "a302772c8f3791c477edd1764b82f8d1f5b22fdf"
5145
5246 # Pull token from "secrets" setting of the github repo
5347 COVERALLS_TOKEN: ${{ secrets.COVERALLS_TOKEN }}
5448 COVERAGE: ${{ matrix.coverage }}
55−
56− # hlint
57− HLINT_VERSION: 3.6.1
58− HLINT_OPTIONS: "lint"
59− HLINT_TARGETS: "core/src src test benchmark"
49+ DOCTEST: ${{ matrix.doctest }}
6050
6151 # Subdir
6252 SUBDIR: ${{ matrix.subdir }}
@@ -68,8 +58,8 @@ jobs:
6858 matrix:
6959 # The order is important to optimize fail-fast.
7060 name:
71− - 9.8.1-Werror
72− # - 9.8.1-docspec
61+ - 9.4.4-Werror
62+ - 9.2.7-docspec
7363 # - 8.10.7-coverage
7464
7565 # Note: if cabal.project is not specified benchmarks and tests won't
@@ -78,76 +68,53 @@ jobs:
7868 include:
7969 - name: head
8070 ghc_version: head
81− # The URL may change, to find a working URL go to https://gitlab.haskell.org/ghc/ghc/-/jobs/
82− # Find a debian10/11/12 job, click on a passed/failed status, at the
83− # end of the output you will find the tar.xz name, put that tar
84− # name after "raw/", and put the job name after "job=".
85− # Also see https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix
86− #
87− # May also use ghcup for installing ghc head version, use the
88− # version "LatestNightly", and the following config:
89− # ghcup config add-release-channel https://ghc.gitlab.haskell.org/ghcup-metadata/ghcup-nightlies-0.0.7.yaml
90− ghcup_ghc_options: "-u https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-deb10-int_native-validate.tar.xz?job=x86_64-linux-deb10-int_native-validate"
9171 runner: ubuntu-latest
9272 build: cabal
93− cabal_version: 3.10.2.0
73+ cabal_version: 3.8.1.0
9474 cabal_project: cabal.project.ghc-head
9575 disable_sdist_build: "y"
9676 ignore_error: true
97− # Note: use linux for warning build for convenient dev testing
98− - name: 9.8.1-Werror
99− ghc_version: 9.8.1
100− runner: ubuntu-latest
101− build: cabal
102− cabal_version: 3.10.1.0
103− disable_sdist_build: "y"
104− cabal_project: cabal.project.Werror
105− ignore_error: false
106− #- name: 9.8.1-docspec
107− # ghc_version: 9.8.1
108− # runner: ubuntu-latest
109− # build: cabal
110− # cabal_version: 3.10.1.0
111− # cabal_project: cabal.project.doctest
112− # disable_test: "y"
113− # disable_bench: "y"
114− # disable_docs: "y"
115− # enable_docspec: "y"
116− # disable_sdist_build: "y"
117− # ignore_error: false
118− - name: 9.8.1-fusion-inspection
119− ghc_version: 9.8.1
77+ - name: 9.6.1
78+ ghc_version: 9.6.1
12079 runner: ubuntu-latest
12180 build: cabal
12281 cabal_version: 3.10.1.0
12382 disable_sdist_build: "y"
12483 cabal_project: cabal.project
125− cabal_build_options: "--flag fusion-plugin --flag inspection"
126− ignore_error: false
127− - name: 9.6.3-macos
128− ghc_version: 9.6.3
129− runner: macos-latest
84+ ignore_error: true
85+ - name: 9.4.4-Werror
86+ ghc_version: 9.4.4
87+ runner: ubuntu-latest
13088 build: cabal
131− cabal_version: 3.10.1.0
89+ cabal_version: 3.8.1.0
13290 disable_sdist_build: "y"
133− cabal_project: cabal.project
91+ cabal_project: cabal.project.Werror
13492 ignore_error: false
135− - name: 9.4.7
136− ghc_version: 9.4.7
93+ - name: 9.4.4-fusion-inspection
94+ ghc_version: 9.4.4
13795 runner: ubuntu-latest
13896 build: cabal
13997 cabal_version: 3.8.1.0
14098 disable_sdist_build: "y"
14199 cabal_project: cabal.project
100+ cabal_build_options: "--flag fusion-plugin --flag inspection"
142101 ignore_error: false
143− - name: 9.2.8
144− ghc_version: 9.2.8
102+ - name: 9.2.7
103+ ghc_version: 9.2.7
145104 runner: ubuntu-latest
146105 build: cabal
147106 cabal_project: cabal.project
148107 cabal_version: 3.6.2.0
149108 disable_sdist_build: "y"
150109 ignore_error: false
110+ - name: 9.2.7-docspec
111+ ghc_version: 9.2.7
112+ runner: ubuntu-latest
113+ doctest: "y"
114+ cabal_version: 3.6.2.0
115+ cabal_project: cabal.project
116+ disable_sdist_build: "y"
117+ ignore_error: false
151118 - name: 9.0.2-streamly-sdist
152119 ghc_version: 9.0.2
153120 runner: ubuntu-latest
@@ -171,6 +138,14 @@ jobs:
171138 stack_build_options: "--flag streamly-benchmarks:-opt"
172139 cabal_version: 3.6.2.0
173140 ignore_error: false
141+ - name: 8.10.7-macos
142+ ghc_version: 8.10.7
143+ runner: macos-latest
144+ build: cabal
145+ cabal_version: 3.6.2.0
146+ cabal_project: cabal.project
147+ disable_sdist_build: "y"
148+ ignore_error: false
174149 # - name: 8.10.7-coverage
175150 # ghc_version: 8.10.7
176151 # runner: ubuntu-latest
@@ -185,6 +160,15 @@ jobs:
185160 cabal_project: cabal.project
186161 disable_sdist_build: "y"
187162 ignore_error: false
163+ - name: 8.6.5-no-fusion-unoptimized
164+ ghc_version: 8.6.5
165+ runner: ubuntu-latest
166+ build: cabal
167+ cabal_version: 3.6.2.0
168+ cabal_project: cabal.project
169+ cabal_build_options: "--flag no-fusion --flag -opt"
170+ disable_sdist_build: "y"
171+ ignore_error: false
188172 - name: 8.6.5-debug-unoptimized
189173 ghc_version: 8.6.5
190174 runner: ubuntu-latest
@@ -194,44 +178,66 @@ jobs:
194178 cabal_build_options: "--flag debug --flag -opt"
195179 disable_sdist_build: "y"
196180 ignore_error: false
197− - name: hlint
198− build: hlint
199− runner: ubuntu-latest
200− ignore_error: true
201181
202182 steps:
203183 - uses: actions/checkout@v2
204184
205185 - uses: actions/cache@v1
206− name: Cache common directories
186+ name: Cache ~/.cabal
207187 with:
208− path: |
209− ~/.local
210− ~/.cabal
211− ~/.stack
212− ~/.ghcup
188+ path: ~/.cabal
213189 # Bump the key version to clear the cache
214190 key: ${{ runner.os }}-${{ matrix.ghc_version }}-cabal-v2
215191
216192 - name: Download packcheck
217193 run: |
218− if test ! -e "$PACKCHECK"
219− then
220− if test -z "$PACKCHECK_GITHUB_COMMIT"
221− then
222− die "PACKCHECK_GITHUB_COMMIT is not specified."
223− fi
224− PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
225− curl --fail -sL -o "$PACKCHECK" $PACKCHECK_URL || exit 1
226− chmod +x $PACKCHECK
227− elif test ! -x "$PACKCHECK"
194+ # Get packcheck if needed
195+ CURL=$(which curl)
196+ PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
197+ if test ! -e "$PACKCHECK_LOCAL_PATH"; then $CURL -sL -o "$PACKCHECK_LOCAL_PATH" $PACKCHECK_URL; fi;
198+ chmod +x $PACKCHECK_LOCAL_PATH
199+
200+ os=$(uname -s -m)
201+ case "$os" in
202+ "Linux x86_64") GHCUP_ARCH="x86_64-linux" ;;
203+ "Darwin x86_64") GHCUP_ARCH="x86_64-apple-darwin" ;;
204+ *) echo "Unknown OS/Arch: $os"; exit 1;;
205+ esac
206+
207+ # Check available versions here: https://downloads.haskell.org/~ghcup/
208+ GHCUP_VER=0.1.18.0
209+ $CURL -sL -o ./ghcup https://downloads.haskell.org/~ghcup/$GHCUP_VER/${GHCUP_ARCH}-ghcup-$GHCUP_VER
210+ chmod +x ./ghcup
211+ if test "$GHCVER" = "head"
228212 then
229− chmod +x $PACKCHECK
213+ # The URL may change, to find a working URL go to https://gitlab.haskell.org/ghc/ghc/-/jobs/
214+ # Find a debian10 job, click on a passed/failed job, at the
215+ # end of the output you will find the tar.xz name, put that after
216+ # "raw/", and put the job name after "job=".
217+ # Also see https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix
218+ ./ghcup install ghc -u https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-deb10-int_native-validate.tar.xz?job=x86_64-linux-deb10-int_native-validate head
219+ else
220+ ./ghcup install ghc $GHCVER
230221 fi
222+ ./ghcup install cabal $CABALVER
223+ ./ghcup set ghc $GHCVER
224+ ./ghcup set cabal $CABALVER
231225
232226 - name: Run tests
233227 run: |
234− if test -n "$COVERAGE"
228+ if test -n "$DOCTEST"
229+ then
230+ mkdir -p $HOME/.cabal/bin
231+ curl -sL https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz > cabal-docspec.xz
232+ echo '0829bd034fba901cbcfe491d98ed8b28fd54f9cb5c91fa8e1ac62dc4413c9562 cabal-docspec.xz' | sha256sum -c -
233+ xz -d < cabal-docspec.xz > $HOME/.cabal/bin/cabal-docspec
234+ rm -f cabal-docspec.xz
235+ chmod a+x $HOME/.cabal/bin/cabal-docspec
236+ cabal update
237+ cabal build --project-file cabal.project.doctest all
238+ $HOME/.cabal/bin/cabal-docspec --version
239+ $HOME/.cabal/bin/cabal-docspec --timeout 60 --check-properties --property-variables xs
240+ elif test -n "$COVERAGE"
235241 then
236242 # Run tests with coverage
237243 cabal update
@@ -244,15 +250,16 @@ jobs:
244250 # XXX Uncomment this and fix it properly later
245251 # hpc-coveralls --repo-token="$COVERALLS_TOKEN" --coverage-mode=StrictlyFullLines
246252 else
253+ export TOOLS_DIR=/opt
247254 # /usr/local/opt/curl/bin for macOS
248255 export PATH=$HOME/.local/bin:$HOME/.ghcup/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/opt/curl/bin
249256 if test -n "$SUBDIR"
250257 then
251− mv "$PACKCHECK" "$SUBDIR/packcheck.sh"
258+ mv "$PACKCHECK_LOCAL_PATH" "$SUBDIR/packcheck.sh"
252259 # This is required as dist-newstyle will be created on the top level
253260 # directory as it is considered the working dir.
254261 rm cabal.project
255262 cd "$SUBDIR"
256263 fi
257− bash -c "$PACKCHECK $BUILD"
264+ bash -c "$PACKCHECK_LOCAL_PATH $BUILD"
258265 fi
.github/workflows/regression-check.yml+2−2
@@ -26,8 +26,8 @@ jobs:
2626 Data.Fold.Window
2727 Data.Parser
2828 Data.ParserK
29− Data.StreamD
30− Data.StreamK:6
29+ Data.Stream.StreamD
30+ Data.Stream.StreamK:6
3131 Data.Unfold
3232 Data.Stream
3333 Data.Stream.Concurrent
.hlint.ignore−5
@@ -2,9 +2,6 @@ core/src/Streamly/Internal/Data/Stream/StreamK/Type.hs
22core/src/Streamly/Internal/Data/Pipe/Type.hs
33core/src/Streamly/Internal/Unicode/Stream.hs
44src/Streamly/Internal/Data/SmallArray/Type.hs
5−src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs
6−src/Streamly/Internal/Data/Stream/Serial.hs
7−src/Streamly/Internal/Data/Stream/Zip.hs
85test/Streamly/Test/Data/Array.hs
96test/Streamly/Test/Data/Parser.hs
107test/Streamly/Test/Data/ParserK.hs
@@ -13,7 +10,6 @@ test/Streamly/Test/Data/Unfold.hs
1310test/Streamly/Test/FileSystem/Event.hs
1411test/Streamly/Test/Prelude/Concurrent.hs
1512test/Streamly/Test/Data/Stream/Concurrent.hs
16−test/lib/Streamly/Test/Prelude/Common.hs
1713test/Streamly/Test/Prelude/Fold.hs
1814test/Streamly/Test/Prelude/Rate.hs
1915test/Streamly/Test/Prelude/Serial.hs
@@ -31,7 +27,6 @@ benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs
3127benchmark/Streamly/Benchmark/Data/Unfold.hs
3228benchmark/Streamly/Benchmark/FileSystem/Handle.hs
3329benchmark/Streamly/Benchmark/Prelude/Async.hs
34−benchmark/Streamly/Benchmark/Prelude/Concurrent.hs
3530benchmark/Streamly/Benchmark/Prelude/Merge.hs
3631benchmark/Streamly/Benchmark/Prelude/Parallel.hs
3732benchmark/Streamly/Benchmark/Prelude/Rate.hs
benchmark/NanoBenchmarks.hs+1−1
@@ -18,7 +18,7 @@ import qualified Streamly.Data.Fold as FL
1818import qualified Streamly.Internal.Data.Stream as S
1919import qualified Streamly.Internal.Data.Stream.IsStream as Internal
2020
21−import Test.Tasty.Bench
21+import Gauge
2222import System.Random
2323
2424maxValue :: Int
benchmark/Streamly/Benchmark/Data/Array/CommonImports.hs+1−1
@@ -9,7 +9,7 @@ import qualified Streamly.Data.Fold as Fold
99import qualified Streamly.Data.Stream as S
1010import qualified Streamly.Internal.Data.Stream as Stream
1111
12−import Test.Tasty.Bench
12+import Gauge
1313import Streamly.Benchmark.Common hiding (benchPureSrc)
1414import qualified Stream.Common as P
1515
benchmark/Streamly/Benchmark/Data/Array/Stream.hs+3−3
@@ -39,7 +39,7 @@ import Prelude hiding ()
3939
4040import qualified Streamly.Data.Stream as Stream
4141import qualified Streamly.Internal.Data.Array as Array
42−import qualified Streamly.Internal.Data.Array.Stream as ArrayStream
42+import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
4343import qualified Streamly.Internal.Data.Fold as Fold
4444import qualified Streamly.Internal.Data.Parser as Parser
4545import qualified Streamly.Internal.Data.Stream as Stream
@@ -47,13 +47,13 @@ import qualified Streamly.Internal.Data.StreamK as StreamK
4747import qualified Streamly.Internal.FileSystem.Handle as Handle
4848import qualified Streamly.Internal.Unicode.Stream as Unicode
4949
50−import Test.Tasty.Bench hiding (env)
50+import Gauge hiding (env)
5151import Streamly.Benchmark.Common
5252import Streamly.Benchmark.Common.Handle
5353import Control.Monad.IO.Class (MonadIO)
5454
5555#ifdef INSPECTION
56−import Streamly.Internal.Data.MutByteArray (Unbox)
56+import Streamly.Internal.Data.Unbox (Unbox)
5757import Streamly.Internal.Data.Stream (Step(..))
5858import Test.Inspection
5959#endif
benchmark/Streamly/Benchmark/Data/Fold.hs+5−28
@@ -12,11 +12,6 @@
1212{-# OPTIONS_GHC -fno-warn-warnings-deprecations #-}
1313{-# OPTIONS_GHC -Wno-orphans #-}
1414
15−#undef FUSION_CHECK
16−#ifdef FUSION_CHECK
17−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
18−#endif
19−
2015module Main (main) where
2116
2217import Control.DeepSeq (NFData(..))
@@ -37,11 +32,12 @@ import Streamly.Internal.Data.MutArray (MutArray)
3732
3833import qualified Streamly.Internal.Data.Fold as FL
3934import qualified Streamly.Data.Fold.Prelude as Fold
35+import qualified Streamly.Internal.Data.Fold.Container as FL
4036import qualified Streamly.Internal.Data.Unfold as Unfold
4137import qualified Streamly.Internal.Data.Pipe as Pipe
4238import qualified Streamly.Internal.Data.Stream as Stream
4339
44−import Test.Tasty.Bench
40+import Gauge
4541import Streamly.Benchmark.Common
4642import Prelude hiding (all, any, take, unzip, sequence_, filter)
4743
@@ -269,13 +265,13 @@ unzip = Stream.fold $ FL.lmap (\a -> (a, a)) (FL.unzip FL.sum FL.length)
269265{-# INLINE unzipWithFstM #-}
270266unzipWithFstM :: Monad m => Stream m Int -> m (Int, Int)
271267unzipWithFstM = do
272− let f a = return (a + 1, a)
268+ let f = \a -> return (a + 1, a)
273269 Stream.fold (FL.unzipWithFstM f FL.sum FL.length)
274270
275271{-# INLINE unzipWithMinM #-}
276272unzipWithMinM :: Monad m => Stream m Int -> m (Int, Int)
277273unzipWithMinM = do
278− let f a = return (a + 1, a)
274+ let f = \a -> return (a + 1, a)
279275 Stream.fold (FL.unzipWithMinM f FL.sum FL.length)
280276
281277-------------------------------------------------------------------------------
@@ -507,9 +503,7 @@ o_n_heap_serial value =
507503-------------------------------------------------------------------------------
508504
509505main :: IO ()
510−main = do
511−#ifndef FUSION_CHECK
512− runWithCLIOpts defaultStreamSize allBenchmarks
506+main = runWithCLIOpts defaultStreamSize allBenchmarks
513507
514508 where
515509
@@ -522,20 +516,3 @@ main = do
522516 , bgroup (o_n_space_prefix moduleName) (o_n_space_serial value)
523517 , bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
524518 ]
525−#else
526− -- Enable FUSION_CHECK macro at the beginning of the file
527− -- Enable one benchmark below, and run the benchmark
528− -- Check the .dump-simpl output
529− let value = 100000
530− let input = source value 1
531− let getKey buckets = (`mod` buckets)
532− let getFold k =
533− return $ case k of
534− 0 -> FL.sum
535− 1 -> FL.length
536− _ -> FL.length
537−
538− -- demuxToMap (getKey 64) (getFold . getKey 64) input
539− toIntMapIO (getKey 64) input
540− return ()
541−#endif
benchmark/Streamly/Benchmark/Data/Fold/Window.hs+40−40
@@ -8,12 +8,12 @@ import Streamly.Internal.Data.Stream (Stream)
88import System.Random (randomRIO)
99
1010import qualified Streamly.Data.Fold as Fold
11−import qualified Streamly.Internal.Data.Fold as Window
12−import qualified Streamly.Internal.Data.Ring as Ring
11+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
12+import qualified Streamly.Internal.Data.Fold.Window as Window
1313import qualified Streamly.Internal.Data.Stream as Stream
1414
1515import Streamly.Benchmark.Common
16−import Test.Tasty.Bench
16+import Gauge
1717
1818{-# INLINE source #-}
1919source :: (Monad m, Num a, Stream.Enumerable a) =>
@@ -70,55 +70,55 @@ o_1_space_folds :: Int -> [Benchmark]
7070o_1_space_folds numElements =
7171 [ bgroup "fold"
7272 [ benchWithFold numElements "minimum (window size 100)"
73− (Window.windowMinimum 100)
73+ (Window.minimum 100)
7474 , benchWithFold numElements "minimum (window size 1000)"
75− (Window.windowMinimum 1000)
75+ (Window.minimum 1000)
7676 , benchWith sourceDescendingInt numElements
7777 "minimum descending (window size 1000)"
78− (Window.windowMinimum 1000)
78+ (Window.minimum 1000)
7979
8080 , benchWithFold numElements "maximum (window size 100)"
81− (Window.windowMaximum 100)
81+ (Window.maximum 100)
8282 , benchWithFold numElements "maximum (window size 1000)"
83− (Window.windowMaximum 1000)
83+ (Window.maximum 1000)
8484 , benchWith sourceDescendingInt numElements
8585 "maximum descending (window size 1000)"
86− (Window.windowMaximum 1000)
86+ (Window.maximum 1000)
8787
8888 , benchWithFold numElements "range (window size 100)"
89− (Window.windowRange 100)
89+ (Window.range 100)
9090 , benchWithFold numElements "range (window size 1000)"
91− (Window.windowRange 1000)
91+ (Window.range 1000)
9292 , benchWith sourceDescendingInt numElements
9393 "range descending (window size 1000)"
94− (Window.windowRange 1000)
94+ (Window.range 1000)
9595
9696 , benchWithFoldInt numElements "sumInt (window size 100)"
97− (Ring.slidingWindow 100 Window.windowSumInt)
97+ (Ring.slidingWindow 100 Window.sumInt)
9898 , benchWithFoldInt numElements "sum for Int (window size 100)"
99− (Ring.slidingWindow 100 Window.windowSum)
99+ (Ring.slidingWindow 100 Window.sum)
100100 , benchWithFold numElements "sum (window size 100)"
101− (Ring.slidingWindow 100 Window.windowSum)
101+ (Ring.slidingWindow 100 Window.sum)
102102 , benchWithFold numElements "sum (window size 1000)"
103− (Ring.slidingWindow 1000 Window.windowSum)
103+ (Ring.slidingWindow 1000 Window.sum)
104104 , benchWithFold numElements "sum (entire stream)"
105− (Window.cumulative Window.windowSum)
105+ (Window.cumulative Window.sum)
106106 , benchWithFold numElements "sum (Data.Fold)"
107107 Fold.sum
108108
109109 , benchWithFold numElements "mean (window size 100)"
110− (Ring.slidingWindow 100 Window.windowMean)
110+ (Ring.slidingWindow 100 Window.mean)
111111 , benchWithFold numElements "mean (window size 1000)"
112− (Ring.slidingWindow 1000 Window.windowMean)
112+ (Ring.slidingWindow 1000 Window.mean)
113113 , benchWithFold numElements "mean (entire stream)"
114− (Window.cumulative Window.windowMean)
114+ (Window.cumulative Window.mean)
115115 , benchWithFold numElements "mean (Data.Fold)"
116116 Fold.mean
117117
118118 , benchWithFold numElements "powerSum 2 (window size 100)"
119− (Ring.slidingWindow 100 (Window.windowPowerSum 2))
119+ (Ring.slidingWindow 100 (Window.powerSum 2))
120120 , benchWithFold numElements "powerSum 2 (entire stream)"
121− (Window.cumulative (Window.windowPowerSum 2))
121+ (Window.cumulative (Window.powerSum 2))
122122
123123 ]
124124 ]
@@ -127,51 +127,51 @@ o_1_space_scans :: Int -> [Benchmark]
127127o_1_space_scans numElements =
128128 [ bgroup "scan"
129129 [ benchWithPostscan numElements "minimum (window size 10)"
130− (Window.windowMinimum 10)
130+ (Window.minimum 10)
131131 -- Below window size 30 the linear search based impl performs better
132132 -- than the dequeue based implementation.
133133 , benchWithPostscan numElements "minimum (window size 30)"
134− (Window.windowMinimum 30)
134+ (Window.minimum 30)
135135 , benchWithPostscan numElements "minimum (window size 1000)"
136− (Window.windowMinimum 1000)
136+ (Window.minimum 1000)
137137 , benchScanWith sourceDescendingInt numElements
138138 "minimum descending (window size 1000)"
139− (Window.windowMinimum 1000)
139+ (Window.minimum 1000)
140140
141141 , benchWithPostscan numElements "maximum (window size 10)"
142− (Window.windowMaximum 10)
142+ (Window.maximum 10)
143143 , benchWithPostscan numElements "maximum (window size 30)"
144− (Window.windowMaximum 30)
144+ (Window.maximum 30)
145145 , benchWithPostscan numElements "maximum (window size 1000)"
146− (Window.windowMaximum 1000)
146+ (Window.maximum 1000)
147147 , benchScanWith sourceDescendingInt numElements
148148 "maximum descending (window size 1000)"
149− (Window.windowMaximum 1000)
149+ (Window.maximum 1000)
150150
151151 , benchWithPostscan numElements "range (window size 10)"
152− (Window.windowRange 10)
152+ (Window.range 10)
153153 , benchWithPostscan numElements "range (window size 30)"
154− (Window.windowRange 30)
154+ (Window.range 30)
155155 , benchWithPostscan numElements "range (window size 1000)"
156− (Window.windowRange 1000)
156+ (Window.range 1000)
157157 , benchScanWith sourceDescendingInt numElements
158158 "range descending (window size 1000)"
159− (Window.windowRange 1000)
159+ (Window.range 1000)
160160
161161 , benchWithPostscan numElements "sum (window size 100)"
162− (Ring.slidingWindow 100 Window.windowSum)
162+ (Ring.slidingWindow 100 Window.sum)
163163 , benchWithPostscan numElements "sum (window size 1000)"
164− (Ring.slidingWindow 1000 Window.windowSum)
164+ (Ring.slidingWindow 1000 Window.sum)
165165
166166 , benchWithPostscan numElements "mean (window size 100)"
167− (Ring.slidingWindow 100 Window.windowMean)
167+ (Ring.slidingWindow 100 Window.mean)
168168 , benchWithPostscan numElements "mean (window size 1000)"
169− (Ring.slidingWindow 1000 Window.windowMean)
169+ (Ring.slidingWindow 1000 Window.mean)
170170
171171 , benchWithPostscan numElements "powerSum 2 (window size 100)"
172− (Ring.slidingWindow 100 (Window.windowPowerSum 2))
172+ (Ring.slidingWindow 100 (Window.powerSum 2))
173173 , benchWithPostscan numElements "powerSum 2 (window size 1000)"
174− (Ring.slidingWindow 1000 (Window.windowPowerSum 2))
174+ (Ring.slidingWindow 1000 (Window.powerSum 2))
175175 ]
176176 ]
177177
benchmark/Streamly/Benchmark/Data/MutArray.hs+1−1
@@ -49,7 +49,7 @@ import qualified Streamly.Internal.Data.MutArray as MArray
4949import qualified Streamly.Internal.Data.Fold as Fold
5050import qualified Streamly.Internal.Data.Stream as Stream
5151
52−import Test.Tasty.Bench
52+import Gauge
5353import Streamly.Benchmark.Common hiding (benchPureSrc)
5454
5555type Stream = MutArray
benchmark/Streamly/Benchmark/Data/Parser.hs+10−10
@@ -47,9 +47,10 @@ import qualified Streamly.Internal.Data.Fold as Fold
4747import qualified Streamly.Internal.Data.Parser as PR
4848import qualified Streamly.Data.Stream as Stream
4949import qualified Streamly.Internal.Data.Producer as Producer
50+import qualified Streamly.Internal.Data.Producer.Source as Source
5051import qualified Streamly.Internal.Data.Stream as Stream
5152
52−import Test.Tasty.Bench hiding (env)
53+import Gauge hiding (env)
5354import Streamly.Benchmark.Common
5455import Streamly.Benchmark.Common.Handle
5556
@@ -348,7 +349,7 @@ split_ value =
348349-- PR.dropWhile (<= (value * 1 `div` 4)) *> PR.die "alt"
349350{-# INLINE takeWhileFail #-}
350351takeWhileFail :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
351−takeWhileFail predicate (Fold fstep finitial _ ffinal) =
352+takeWhileFail predicate (Fold fstep finitial fextract) =
352353 Parser step initial extract
353354
354355 where
@@ -369,7 +370,7 @@ takeWhileFail predicate (Fold fstep finitial _ ffinal) =
369370 Fold.Done b -> Done 0 b
370371 else return $ Error "fail"
371372
372− extract s = fmap (Done 0) (ffinal s)
373+ extract s = fmap (Done 0) (fextract s)
373374
374375{-# INLINE alt2 #-}
375376alt2 :: Monad m
@@ -634,13 +635,13 @@ choice value =
634635{-# INLINE parseManyUnfoldArrays #-}
635636parseManyUnfoldArrays :: Int -> [Array.Array Int] -> IO ()
636637parseManyUnfoldArrays count arrays = do
637− let src = Producer.source (Just (Producer.OuterLoop arrays))
638+ let src = Source.source (Just (Producer.OuterLoop arrays))
638639 let parser = PR.fromFold (Fold.take count Fold.drain)
639640 let readSrc =
640− Producer.producer
641+ Source.producer
641642 $ Producer.concat Producer.fromList Array.producer
642643 let streamParser =
643− Producer.simplify (Producer.parseMany parser readSrc)
644+ Producer.simplify (Source.parseMany parser readSrc)
644645 Stream.fold Fold.drain $ Stream.unfold streamParser src
645646
646647-------------------------------------------------------------------------------
@@ -714,6 +715,8 @@ instance NFData ParseError where
714715o_1_space_serial :: Int -> [Benchmark]
715716o_1_space_serial value =
716717 [ benchIOSink value "takeBetween" $ takeBetween value
718+ , benchIOSink value "takeEQ" $ takeEQ value
719+ , benchIOSink value "takeGE" $ takeGE value
717720 , benchIOSink value "takeWhile" $ takeWhile value
718721 , benchIOSink value "takeWhileP" $ takeWhileP value
719722 , benchIOSink value "takeP" $ takeP value
@@ -810,11 +813,8 @@ o_1_space_serial_unfold bound arrays =
810813o_n_heap_serial :: Int -> [Benchmark]
811814o_n_heap_serial value =
812815 [
813− benchIOSink value "takeEQ" $ takeEQ value
814− , benchIOSink value "takeGE" $ takeGE value
815−
816816 -- lookahead benchmark holds the entire input till end
817− , benchIOSink value "lookAhead" $ lookAhead value
817+ benchIOSink value "lookAhead" $ lookAhead value
818818
819819 -- o-n-heap because of backtracking
820820 , benchIOSrc sourceEscapedFrames value "takeFramedByEsc_"
benchmark/Streamly/Benchmark/Data/ParserK.hs+37−90
@@ -5,10 +5,6 @@
55-- License : BSD-3-Clause
66-- Maintainer : streamly@composewell.com
77
8−-- BENCH_CHUNKED -> adaptC
9−-- BENCH_CHUNKED_GENERIC -> adaptCG
10−-- BENCH_SINGULAR -> adapt
11−
128{-# LANGUAGE CPP #-}
139{-# LANGUAGE FlexibleContexts #-}
1410{-# OPTIONS_GHC -Wno-orphans #-}
@@ -22,14 +18,8 @@ import Control.Applicative ((<|>))
2218import Control.DeepSeq (NFData(..))
2319import Control.Monad.IO.Class (MonadIO)
2420import Data.Foldable (asum)
25−#ifdef BENCH_CHUNKED
2621import Streamly.Data.Array (Array, Unbox)
27−#endif
28−#ifdef BENCH_CHUNKED_GENERIC
29−import Streamly.Data.Array.Generic (Array)
30−import qualified Streamly.Internal.Data.Array.Generic as GenArr (chunksOf)
31−#endif
32−import Streamly.Internal.Data.Fold (Fold(..))
22+import Streamly.Internal.Data.Fold.Type (Fold(..))
3323import Streamly.Data.StreamK (StreamK)
3424import Streamly.Internal.Data.Parser
3525 (ParseError(..), Parser(..), Initial(..), Step(..))
@@ -45,52 +35,12 @@ import qualified Streamly.Data.Stream as Stream
4535import qualified Streamly.Data.Fold as FL
4636import qualified Streamly.Internal.Data.Fold as Fold
4737import qualified Streamly.Data.Parser as PRD
48−import qualified Streamly.Internal.Data.ParserK as PR
38+import qualified Streamly.Data.ParserK as PR
4939import qualified Streamly.Internal.Data.StreamK as StreamK
5040
51−import Test.Tasty.Bench
41+import Gauge
5242import Streamly.Benchmark.Common
5343
54−-------------------------------------------------------------------------------
55−-- CPP Helpers
56−-------------------------------------------------------------------------------
57−
58−#ifdef BENCH_CHUNKED
59−
60−#define PARSE_OP StreamK.parseChunks
61−#define FROM_PARSER adaptC
62−#define INPUT (Array a)
63−#define PARSE_ELEM (Array Int)
64−#define CONSTRAINT_IO (MonadIO m, Unbox a)
65−#define CONSTRAINT (Monad m, Unbox a)
66−#define MODULE_NAME "Data.ParserK.Chunked"
67−
68−#endif
69−
70−#ifdef BENCH_CHUNKED_GENERIC
71−
72−#define PARSE_OP StreamK.parseChunksGeneric
73−#define FROM_PARSER adaptCG
74−#define INPUT (Array a)
75−#define PARSE_ELEM (Array Int)
76−#define CONSTRAINT_IO (MonadIO m)
77−#define CONSTRAINT (Monad m)
78−#define MODULE_NAME "Data.ParserK.Chunked.Generic"
79−
80−#endif
81−
82−#ifdef BENCH_SINGULAR
83−
84−#define PARSE_OP StreamK.parse
85−#define FROM_PARSER adapt
86−#define INPUT a
87−#define PARSE_ELEM Int
88−#define CONSTRAINT_IO (MonadIO m)
89−#define CONSTRAINT (Monad m)
90−#define MODULE_NAME "Data.ParserK"
91−
92−#endif
93−
9444-------------------------------------------------------------------------------
9545-- Utilities
9646-------------------------------------------------------------------------------
@@ -110,52 +60,49 @@ sourceUnfoldrM value n = Stream.unfoldrM step n
11060{-# INLINE benchIOSink #-}
11161benchIOSink
11262 :: NFData b
113− => Int -> String -> (StreamK IO PARSE_ELEM -> IO b) -> Benchmark
63+ => Int -> String -> (StreamK IO (Array Int) -> IO b) -> Benchmark
11464benchIOSink value name f =
11565 bench name $ nfIO $ randomRIO (1,1)
11666 >>= f
11767 . StreamK.fromStream
118−#ifdef BENCH_CHUNKED
11968 . Stream.chunksOf 4000
120−#endif
121−#ifdef BENCH_CHUNKED_GENERIC
122− . GenArr.chunksOf 4000
123−#endif
12469 . sourceUnfoldrM value
12570
12671-------------------------------------------------------------------------------
12772-- Parsers
12873-------------------------------------------------------------------------------
12974
75+#define PARSE_OP StreamK.parseChunks
76+
13077{-# INLINE one #-}
13178one :: MonadIO m =>
132− Int -> StreamK m PARSE_ELEM -> m (Either ParseError (Maybe Int))
133−one value = PARSE_OP p
79+ Int -> StreamK m (Array Int) -> m (Either ParseError (Maybe Int))
80+one value = StreamK.parseChunks p
13481
13582 where
13683
13784 p = do
138− m <- PR.FROM_PARSER (PRD.fromFold FL.one)
85+ m <- PR.fromFold FL.one
13986 case m of
14087 Just i -> if i >= value then pure m else p
14188 Nothing -> pure Nothing
14289
14390{-# INLINE satisfy #-}
144−satisfy :: CONSTRAINT_IO => (a -> Bool) -> PR.ParserK INPUT m a
145−satisfy = PR.FROM_PARSER . PRD.satisfy
91+satisfy :: (MonadIO m, Unbox a) => (a -> Bool) -> PR.ParserK a m a
92+satisfy = PR.fromParser . PRD.satisfy
14693
14794{-# INLINE takeWhile #-}
148−takeWhile :: CONSTRAINT_IO => (a -> Bool) -> PR.ParserK INPUT m ()
149−takeWhile p = PR.FROM_PARSER $ PRD.takeWhile p FL.drain
95+takeWhile :: (MonadIO m, Unbox a) => (a -> Bool) -> PR.ParserK a m ()
96+takeWhile p = PR.fromParser $ PRD.takeWhile p FL.drain
15097
15198{-# INLINE takeWhileK #-}
15299takeWhileK :: MonadIO m =>
153− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
100+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
154101takeWhileK value = PARSE_OP (takeWhile (<= value))
155102
156103{-# INLINE splitAp2 #-}
157104splitAp2 :: MonadIO m
158− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ((), ()))
105+ => Int -> StreamK m (Array Int) -> m (Either ParseError ((), ()))
159106splitAp2 value =
160107 PARSE_OP
161108 ((,)
@@ -165,7 +112,7 @@ splitAp2 value =
165112
166113{-# INLINE splitAp8 #-}
167114splitAp8 :: MonadIO m
168− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
115+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
169116splitAp8 value =
170117 PARSE_OP
171118 ( (\() () () () () () () () -> ())
@@ -180,7 +127,7 @@ splitAp8 value =
180127 )
181128
182129{-# INLINE sequenceA #-}
183−sequenceA :: MonadIO m => Int -> StreamK m PARSE_ELEM -> m Int
130+sequenceA :: MonadIO m => Int -> StreamK m (Array Int) -> m Int
184131sequenceA value xs = do
185132 let parser = satisfy (> 0)
186133 list = Prelude.replicate value parser
@@ -189,14 +136,14 @@ sequenceA value xs = do
189136
190137{-# INLINE sequenceA_ #-}
191138sequenceA_ :: MonadIO m =>
192− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
139+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
193140sequenceA_ value xs = do
194141 let parser = satisfy (> 0)
195142 list = Prelude.replicate value parser
196143 PARSE_OP (F.sequenceA_ list) xs
197144
198145{-# INLINE sequence #-}
199−sequence :: MonadIO m => Int -> StreamK m PARSE_ELEM -> m Int
146+sequence :: MonadIO m => Int -> StreamK m (Array Int) -> m Int
200147sequence value xs = do
201148 let parser = satisfy (> 0)
202149 list = Prelude.replicate value parser
@@ -205,7 +152,7 @@ sequence value xs = do
205152
206153{-# INLINE sequence_ #-}
207154sequence_ :: MonadIO m =>
208− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
155+ Int -> StreamK m (Array Int) -> m (Either ParseError ())
209156sequence_ value =
210157 let parser = satisfy (> 0)
211158 list = Prelude.replicate value parser
@@ -213,7 +160,7 @@ sequence_ value =
213160
214161{-# INLINE takeWhileFailD #-}
215162takeWhileFailD :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
216−takeWhileFailD predicate (Fold fstep finitial _ ffinal) =
163+takeWhileFailD predicate (Fold fstep finitial fextract) =
217164 Parser step initial extract
218165
219166 where
@@ -234,16 +181,16 @@ takeWhileFailD predicate (Fold fstep finitial _ ffinal) =
234181 Fold.Done b -> Done 0 b
235182 else return $ Error "fail"
236183
237− extract s = fmap (Done 0) (ffinal s)
184+ extract s = fmap (Done 0) (fextract s)
238185
239186{-# INLINE takeWhileFail #-}
240−takeWhileFail :: CONSTRAINT =>
241− (a -> Bool) -> Fold m a b -> PR.ParserK INPUT m b
242−takeWhileFail p f = PR.FROM_PARSER (takeWhileFailD p f)
187+takeWhileFail :: (Monad m, Unbox a) =>
188+ (a -> Bool) -> Fold m a b -> PR.ParserK a m b
189+takeWhileFail p f = PR.fromParser (takeWhileFailD p f)
243190
244191{-# INLINE alt2 #-}
245192alt2 :: MonadIO m
246− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
193+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
247194alt2 value =
248195 PARSE_OP
249196 ( takeWhileFail (<= (value `div` 2)) Fold.drain
@@ -252,7 +199,7 @@ alt2 value =
252199
253200{-# INLINE alt8 #-}
254201alt8 :: MonadIO m
255− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
202+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
256203alt8 value =
257204 PARSE_OP
258205 ( takeWhileFail (<= ( value `div` 8)) Fold.drain
@@ -267,7 +214,7 @@ alt8 value =
267214
268215{-# INLINE alt16 #-}
269216alt16 :: MonadIO m
270− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
217+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
271218alt16 value =
272219 PARSE_OP
273220 ( takeWhileFail (<= ( value `div` 16)) Fold.drain
@@ -289,27 +236,27 @@ alt16 value =
289236 )
290237
291238{-# INLINE manyAlt #-}
292−manyAlt :: MonadIO m => StreamK m PARSE_ELEM -> m Int
239+manyAlt :: MonadIO m => StreamK m (Array Int) -> m Int
293240manyAlt xs = do
294241 x <- PARSE_OP (AP.many (satisfy (> 0))) xs
295242 return $ Prelude.length x
296243
297244{-# INLINE someAlt #-}
298−someAlt :: MonadIO m => StreamK m PARSE_ELEM -> m Int
245+someAlt :: MonadIO m => StreamK m (Array Int) -> m Int
299246someAlt xs = do
300247 x <- PARSE_OP (AP.some (satisfy (> 0))) xs
301248 return $ Prelude.length x
302249
303250{-# INLINE choice #-}
304251choice :: MonadIO m =>
305− Int -> StreamK m PARSE_ELEM -> m (Either ParseError Int)
252+ Int -> StreamK m (Array Int) -> m (Either ParseError Int)
306253choice value =
307254 PARSE_OP (asum (replicate value (satisfy (< 0)))
308255 AP.<|> satisfy (> 0))
309256
310257{-# INLINE monad2 #-}
311258monad2 :: MonadIO m
312− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
259+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
313260monad2 value =
314261 PARSE_OP $ do
315262 takeWhile (<= (value `div` 2))
@@ -317,7 +264,7 @@ monad2 value =
317264
318265{-# INLINE monad4 #-}
319266monad4 :: MonadIO m
320− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
267+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
321268monad4 value =
322269 PARSE_OP $ do
323270 takeWhile (<= ( value `div` 4))
@@ -327,7 +274,7 @@ monad4 value =
327274
328275{-# INLINE monad8 #-}
329276monad8 :: MonadIO m
330− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
277+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
331278monad8 value =
332279 PARSE_OP $ do
333280 takeWhile (<= ( value `div` 8))
@@ -341,7 +288,7 @@ monad8 value =
341288
342289{-# INLINE monad16 #-}
343290monad16 :: MonadIO m
344− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
291+ => Int -> StreamK m (Array Int) -> m (Either ParseError ())
345292monad16 value =
346293 PARSE_OP $ do
347294 takeWhile (<= ( value `div` 16))
@@ -366,7 +313,7 @@ monad16 value =
366313-------------------------------------------------------------------------------
367314
368315moduleName :: String
369−moduleName = MODULE_NAME
316+moduleName = "Data.ParserK"
370317
371318instance NFData ParseError where
372319 {-# INLINE rnf #-}
@@ -388,7 +335,7 @@ o_1_space_serial value =
388335 ]
389336
390337{-# INLINE sepBy1 #-}
391−sepBy1 :: MonadIO m => StreamK m PARSE_ELEM -> m Int
338+sepBy1 :: MonadIO m => StreamK m (Array Int) -> m Int
392339sepBy1 xs = do
393340 x <- PARSE_OP (parser (satisfy odd) (satisfy even)) xs
394341 return $ Prelude.length x
benchmark/Streamly/Benchmark/Data/Ring/Unboxed.hs+4−3
@@ -12,11 +12,12 @@ module Main (main) where
1212
1313import Control.Monad (void)
1414import GHC.Ptr (Ptr(..))
15−import qualified Streamly.Internal.Data.Array as Array
16−import qualified Streamly.Internal.Data.Ring as Ring
15+
16+import qualified Streamly.Internal.Data.Array.Type as Array
17+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
1718import qualified Data.Foldable as P
1819
19−import Test.Tasty.Bench
20+import Gauge
2021import Streamly.Benchmark.Common
2122
2223import Prelude as P
benchmark/Streamly/Benchmark/Data/Serialize.hsdeleted−591
@@ -1,591 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−{- HLINT ignore -}
3−
4−#undef FUSION_CHECK
5−#ifdef FUSION_CHECK
6−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
7−#endif
8−
9−-- |
10−-- Module : Streamly.Benchmark.Data.Serialize
11−-- Copyright : (c) 2023 Composewell
12−-- License : BSD-3-Clause
13−-- Maintainer : streamly@composewell.com
14−
15−module Main (main) where
16−
17−-------------------------------------------------------------------------------
18−-- Imports
19−-------------------------------------------------------------------------------
20−
21−import Control.DeepSeq (NFData(..), deepseq)
22−import Control.Monad (when, void)
23−import GHC.Generics (Generic)
24−import System.Random (randomRIO)
25−#ifndef USE_UNBOX
26−import Test.QuickCheck (Arbitrary, arbitrary)
27−#endif
28−
29−import Streamly.Internal.Data.MutByteArray
30−import qualified Streamly.Internal.Data.MutByteArray as MBA
31−#ifdef USE_UNBOX
32−import Data.Proxy (Proxy(..))
33−#else
34−import Control.DeepSeq (force)
35−import Test.QuickCheck (oneof, generate)
36−#endif
37−
38−import Test.Tasty.Bench
39−import Streamly.Benchmark.Common
40−
41−#ifndef USE_UNBOX
42−import Streamly.Benchmark.Data.Serialize.RecCompatible
43−import Streamly.Benchmark.Data.Serialize.RecNonCompatible
44−#endif
45−
46−#ifdef USE_UNBOX
47−#define SERIALIZE_CLASS Unbox
48−#define DERIVE_CLASS(typ) $(deriveUnbox [d|instance Unbox typ|])
49−#define SERIALIZE_OP pokeAt
50−#define DESERIALIZE_OP peekAt
51−#else
52−#define SERIALIZE_CLASS Serialize
53−#define DERIVE_CLASS(typ) $(deriveSerialize [d|instance Serialize typ|])
54−#define SERIALIZE_OP serializeAt
55−#define DESERIALIZE_OP deserializeAt
56−#endif
57−
58−-------------------------------------------------------------------------------
59−-- Types
60−-------------------------------------------------------------------------------
61−
62−data Unit = Unit
63− deriving (Generic, Show, Eq)
64−
65−#ifndef USE_TH
66−instance SERIALIZE_CLASS Unit
67−#else
68−DERIVE_CLASS(Unit)
69−#endif
70−
71−instance NFData Unit where
72− {-# INLINE rnf #-}
73− rnf _ = ()
74−
75−data Sum2
76− = Sum21
77− | Sum22
78− deriving (Generic, Show, Eq)
79−
80−#ifndef USE_TH
81−instance SERIALIZE_CLASS Sum2
82−#else
83−DERIVE_CLASS(Sum2)
84−#endif
85−
86−instance NFData Sum2 where
87− {-# INLINE rnf #-}
88− rnf _ = ()
89−
90−data Sum25
91− = Sum251
92− | Sum252
93− | Sum253
94− | Sum254
95− | Sum255
96− | Sum256
97− | Sum257
98− | Sum258
99− | Sum259
100− | Sum2510
101− | Sum2511
102− | Sum2512
103− | Sum2513
104− | Sum2514
105− | Sum2515
106− | Sum2516
107− | Sum2517
108− | Sum2518
109− | Sum2519
110− | Sum2520
111− | Sum2521
112− | Sum2522
113− | Sum2523
114− | Sum2524
115− | Sum2525
116− deriving (Generic, Show, Eq)
117−
118−#ifndef USE_TH
119−instance SERIALIZE_CLASS Sum25
120−#else
121−DERIVE_CLASS(Sum25)
122−#endif
123−
124−instance NFData Sum25 where
125− {-# INLINE rnf #-}
126− rnf _ = ()
127−
128−data Product25
129− = Product25
130− Int
131− Int
132− Int
133− Int
134− Int
135− Int
136− Int
137− Int
138− Int
139− Int
140− Int
141− Int
142− Int
143− Int
144− Int
145− Int
146− Int
147− Int
148− Int
149− Int
150− Int
151− Int
152− Int
153− Int
154− Int
155− deriving (Generic, Show)
156−
157−#ifndef USE_TH
158−instance SERIALIZE_CLASS Product25
159−#else
160−DERIVE_CLASS(Product25)
161−#endif
162−
163−instance NFData Product25 where
164− {-# INLINE rnf #-}
165− rnf
166− (Product25 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20 a21 a22 a23 a24 a25)
167− = a1
168− `seq` a2
169− `seq` a3
170− `seq` a4
171− `seq` a5
172− `seq` a6
173− `seq` a7
174− `seq` a8
175− `seq` a9
176− `seq` a10
177− `seq` a11
178− `seq` a12
179− `seq` a13
180− `seq` a14
181− `seq` a15
182− `seq` a16
183− `seq` a17
184− `seq` a18
185− `seq` a19
186− `seq` a20
187− `seq` a21
188− `seq` a22
189− `seq` a23
190− `seq` a24
191− `seq` a25
192− `seq` ()
193−
194−-- XXX derived Eq instance is not inlined
195−instance Eq Product25 where
196− {-# INLINE (==) #-}
197− (==)
198− (Product25 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20 a21 a22 a23 a24 a25)
199− (Product25 b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 b17 b18 b19 b20 b21 b22 b23 b24 b25)
200− = a1 == b1
201− && a2 == b2
202− && a3 == b3
203− && a4 == b4
204− && a5 == b5
205− && a6 == b6
206− && a7 == b7
207− && a8 == b8
208− && a9 == b9
209− && a10 == b10
210− && a11 == b11
211− && a12 == b12
212− && a13 == b13
213− && a14 == b14
214− && a15 == b15
215− && a16 == b16
216− && a17 == b17
217− && a18 == b18
218− && a19 == b19
219− && a20 == b20
220− && a21 == b21
221− && a22 == b22
222− && a23 == b23
223− && a24 == b24
224− && a25 == b25
225−
226−-------------------------------------------------------------------------------
227−-- Simple non-recursive ADT
228−-------------------------------------------------------------------------------
229−
230−{-
231−data CustomDT1
232− = CDT1C1
233− | CDT1C2 Int
234− | CDT1C3 Int Int
235− deriving (Generic, Show, Eq)
236−
237−#ifndef USE_TH
238−instance SERIALIZE_CLASS CustomDT1
239−#else
240−DERIVE_CLASS(CustomDT1)
241−#endif
242−
243−instance NFData CustomDT1 where
244− {-# INLINE rnf #-}
245− rnf CDT1C1 = ()
246− rnf (CDT1C2 i) = i `seq` ()
247− rnf (CDT1C3 i1 i2) = i1 `seq` i2 `seq` ()
248−-}
249−
250−-------------------------------------------------------------------------------
251−-- Recursive ADT
252−-------------------------------------------------------------------------------
253−
254−#ifndef USE_UNBOX
255−data BinTree a
256− = Tree (BinTree a) (BinTree a)
257− | Leaf a
258− deriving (Show, Read, Eq, Generic)
259−
260−#ifndef USE_TH
261−instance Serialize (BinTree a)
262−#else
263−$(deriveSerialize [d|instance Serialize a => Serialize (BinTree a)|])
264−#endif
265−
266−instance NFData a => NFData (BinTree a) where
267− {-# INLINE rnf #-}
268− rnf (Leaf a) = rnf a `seq` ()
269− rnf (Tree l r) = rnf l `seq` rnf r
270−
271−-- XXX This may not terminate, or could become really large.
272−instance Arbitrary a => Arbitrary (BinTree a) where
273− arbitrary = oneof [Leaf <$> arbitrary, Tree <$> arbitrary <*> arbitrary]
274−
275−mkBinTree :: (Arbitrary a) => Int -> IO (BinTree a)
276−mkBinTree = go (generate arbitrary)
277−
278− where
279−
280− go r 0 = Leaf <$> r
281− go r n = Tree <$> go r (n - 1) <*> go r (n - 1)
282−
283−#endif
284−
285−-------------------------------------------------------------------------------
286−-- Size helpers
287−-------------------------------------------------------------------------------
288−
289−{-# INLINE getSize #-}
290−getSize :: forall a. SERIALIZE_CLASS a => a -> Int
291−#ifdef USE_UNBOX
292−getSize _ = sizeOf (Proxy :: Proxy a)
293−#else
294−getSize = addSizeTo 0
295−#endif
296−
297−-------------------------------------------------------------------------------
298−-- Common helpers
299−-------------------------------------------------------------------------------
300−
301−{- HLINT ignore "Eta reduce" -}
302−-- Parts of "f" that are dependent on val will not be optimized out.
303−{-# INLINE loop #-}
304−loop :: Int -> (a -> IO b) -> a -> IO ()
305−loop count f val = go count val
306− where
307−
308− go n x = when (n > 0) $ f x >> go (n-1) x
309−
310−-- The first arg of "f" is the environment which is not threaded around in the
311−-- loop.
312−{-# INLINE loopWith #-}
313−loopWith :: Int -> (e -> a -> IO b) -> e -> a -> IO ()
314−loopWith count f e val = go count val
315− where
316−
317− go n x = when (n > 0) $ f e x >> go (n-1) x
318−
319−benchSink :: NFData b => String -> Int -> (Int -> IO b) -> Benchmark
320−benchSink name times f = bench name (nfIO (randomRIO (times, times) >>= f))
321−
322−-------------------------------------------------------------------------------
323−-- Serialization Helpers
324−-------------------------------------------------------------------------------
325−
326−{-
327−{-# INLINE pokeWithSize #-}
328−pokeWithSize :: SERIALIZE_CLASS a => MutByteArray -> a -> IO ()
329−pokeWithSize arr val = do
330− let n = getSize val
331− n `seq` SERIALIZE_OP 0 arr val >> return ()
332−
333−{-# INLINE pokeTimesWithSize #-}
334−pokeTimesWithSize :: SERIALIZE_CLASS a => a -> Int -> IO ()
335−pokeTimesWithSize val times = do
336− let n = getSize val
337− arr <- MBA.new n
338− loopWith times pokeWithSize arr val
339−-}
340−
341−{-# INLINE poke #-}
342−poke :: SERIALIZE_CLASS a => MutByteArray -> a -> IO ()
343−poke arr val = void (SERIALIZE_OP 0 arr val)
344−
345−{-# INLINE pokeTimes #-}
346−pokeTimes :: SERIALIZE_CLASS a => a -> Int -> IO ()
347−pokeTimes val times = do
348− let n = getSize val
349− arr <- MBA.new n
350− loopWith times poke arr val
351−
352−{-# INLINE encode #-}
353−encode :: SERIALIZE_CLASS a => a -> IO ()
354−encode val = do
355− let n = getSize val
356− arr <- MBA.new n
357− void (SERIALIZE_OP 0 arr val)
358−
359−{-# INLINE encodeTimes #-}
360−encodeTimes :: SERIALIZE_CLASS a => a -> Int -> IO ()
361−encodeTimes val times = loop times encode val
362−
363−{-# INLINE peek #-}
364−peek :: forall a. (NFData a, SERIALIZE_CLASS a) =>
365− (a, Int) -> MutByteArray -> IO ()
366−#ifdef USE_UNBOX
367−peek (_val, _) arr = do
368− (val1 :: a) <- DESERIALIZE_OP 0 arr
369−#else
370−peek (_val, n) arr = do
371− (_, val1 :: a) <- DESERIALIZE_OP 0 arr n
372−#endif
373− -- If the datatype is not deeply strict or deepseq is not used then use
374− -- Equality.
375− -- Ensure that we are actually constructing the type and using it. This
376− -- is important, otherwise the structure is created and discarded, the
377− -- cost of creation of the structure is not accounted. Otherwise we may
378− -- just read the values and discard them. The comparison adds to the
379− -- cost though. We could use deepseq but then we need to write
380− -- instances of NFData and ensure that they are correct and perform
381− -- well. Equality check also ensures correctness.
382− {-
383− if (val1 /= val)
384− then error "peek: no match"
385− else return ()
386− -}
387− val1 `deepseq` return ()
388−
389−{-# INLINE peekTimes #-}
390−peekTimes :: (NFData a, SERIALIZE_CLASS a) => Int -> a -> Int -> IO ()
391−peekTimes n val times = do
392− arr <- MBA.new n
393− _ <- SERIALIZE_OP 0 arr val
394− loopWith times peek (val, n) arr
395−
396−{-# INLINE trip #-}
397−trip :: forall a. (NFData a, SERIALIZE_CLASS a) => a -> IO ()
398−trip val = do
399− let n = getSize val
400− arr <- MBA.new n
401− _ <- SERIALIZE_OP 0 arr val
402−#ifdef USE_UNBOX
403− (val1 :: a) <- DESERIALIZE_OP 0 arr
404−#else
405− (_, val1 :: a) <- DESERIALIZE_OP 0 arr n
406−#endif
407− -- Do not remove this or use deepseq, see the comments in peek.
408− {-
409− if (val1 /= val)
410− then error "roundtrip: no match"
411− else return ()
412− -}
413− -- Note: deepseq is not needed if the datatype is strict
414− val1 `deepseq` return ()
415−
416−{-# INLINE roundtrip #-}
417−roundtrip :: (NFData a, SERIALIZE_CLASS a) => a -> Int -> IO ()
418−roundtrip val times = loop times trip val
419−
420−-------------------------------------------------------------------------------
421−-- Benchmarks
422−-------------------------------------------------------------------------------
423−
424−{-# INLINE benchConst #-}
425−benchConst ::
426− String
427− -> (forall a. (NFData a, SERIALIZE_CLASS a) =>
428− Int -> a -> Int -> IO ())
429− -> Int
430− -> Benchmark
431−benchConst gname f times =
432− bgroup gname
433− [ let !n = getSize Unit
434− in benchSink "Unit" times (f n Unit)
435− {-
436− , let !n = getSize CDT1C1
437− in benchSink "C1" times (f n CDT1C1)
438− , let val = CDT1C2 5
439− !n = getSize val
440− in benchSink "C2" (times `div` 2) (f n val)
441− , let val = CDT1C3 5 2
442− !n = getSize val
443− in benchSink "C3" (times `div` 3) (f n val)
444− -}
445− , let !n = getSize Sum21
446− in benchSink "Sum2" times (f n Sum21)
447− , let !n = getSize Sum2525
448− in benchSink "Sum25" times (f n Sum2525)
449− , let val = Product25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
450− !n = getSize val
451− in benchSink "Product25" (times `div` 26) (f n val)
452− ]
453−
454−#ifndef USE_UNBOX
455−{-# INLINE benchVar #-}
456−benchVar ::
457− String
458− -> (forall a. (NFData a, SERIALIZE_CLASS a) =>
459− Int -> a -> Int -> IO ())
460− -> BinTree Int
461− -> [Int]
462− -> Int
463− -> Benchmark
464−benchVar gname f tInt lInt times =
465− bgroup gname
466− [ let !n = getSize tInt
467− in benchSink "bintree-int" times (f n tInt)
468− , let !n = getSize lInt
469− in benchSink "list-int" times (f n lInt)
470− ]
471−
472−{-
473−{-# INLINE benchGeneric #-}
474−benchGeneric
475− :: (NFData b, Serialize b)
476− => String
477− -> String
478− -> (forall a. (NFData a, Serialize a) =>
479− Int -> a -> Int -> IO ())
480− -> b
481− -> Int
482− -> Benchmark
483−benchGeneric gname valId f val times =
484− bgroup gname
485− [ let !n = getSize val
486− in benchSink valId times (f n val)
487− ]
488−-}
489−
490−{-# INLINE benchLargeRec #-}
491−benchLargeRec ::
492− String
493− -> (forall a. (NFData a, Serialize a) =>
494− Int -> a -> Int -> IO ())
495− -> RecNonCompatible
496− -> RecCompatible
497− -> Int
498− -> Benchmark
499−benchLargeRec gname f recL recR times =
500− bgroup gname
501− [ let !n = getSize recL
502− in benchSink "RecNonCompatible" times (f n recL)
503− , let !n = getSize recR
504− in benchSink "RecCompatible" times (f n recR)
505− ]
506−#endif
507−
508−-- Times is scaled by the number of constructors to normalize
509−#ifdef USE_UNBOX
510−allBenchmarks :: Int -> [Benchmark]
511−allBenchmarks times =
512−#else
513−allBenchmarks ::
514− BinTree Int -> [Int] -> RecNonCompatible -> RecCompatible -> Int -> [Benchmark]
515−allBenchmarks tInt lInt recL recR times =
516−#endif
517− [ bgroup "sizeOf"
518− [
519−#ifndef USE_UNBOX
520− bench "bintree-int" $ nf getSize tInt
521− , bench "list-int" $ nf getSize lInt
522−#endif
523− ]
524− , benchConst "poke" (const pokeTimes) times
525− -- , benchConst "pokeWithSize" (const pokeTimesWithSize) times
526− , benchConst "encode" (const encodeTimes) times
527− , benchConst "peek" peekTimes times
528− , benchConst "roundtrip" (const roundtrip) times
529−#ifndef USE_UNBOX
530− , benchVar "poke" (const pokeTimes) tInt lInt 1
531− -- , benchVar "pokeWithSize" (const pokeTimesWithSize) tInt lInt 1
532− , benchVar "encode" (const encodeTimes) tInt lInt 1
533− , benchVar "peek" peekTimes tInt lInt 1
534− , benchVar "roundtrip" (const roundtrip) tInt lInt 1
535− , benchLargeRec "poke" (const pokeTimes) recL recR times
536− , benchLargeRec "encode" (const encodeTimes) recL recR times
537− , benchLargeRec "peek" peekTimes recL recR times
538− , benchLargeRec "roundtrip" (const roundtrip) recL recR times
539−#endif
540− ]
541−
542−-------------------------------------------------------------------------------
543−-- Driver
544−-------------------------------------------------------------------------------
545−
546−main :: IO ()
547−main = do
548−#ifndef USE_UNBOX
549− -- Approximately 100000 constructors
550− -- Assuming Leaf nodes constitute two constructors (Leaf, Int) and internal
551− -- nodes 1 level = log_2 (100001/3) + 1 = 16
552− !(tInt :: BinTree Int) <- force <$> mkBinTree 16
553−
554− let !recL = valRecNonCompatible
555− let !recR = valRecCompatible
556−
557− -- Approximately 100000 constructors, assuming two constructors (Cons, Int)
558− -- per element.
559− let lInt = [1..50000 :: Int]
560− let !len = length lInt -- evaluate the list
561−#endif
562−#ifndef FUSION_CHECK
563−#ifdef USE_UNBOX
564− runWithCLIOpts defaultStreamSize allBenchmarks
565−#else
566− len
567− `seq` runWithCLIOpts
568− defaultStreamSize
569− (allBenchmarks tInt lInt recL recR)
570−#endif
571−#else
572− -- Enable FUSION_CHECK macro at the beginning of the file
573− -- Enable one benchmark below, and run the benchmark
574− -- Check the .dump-simpl output
575− let value = 100000
576− -- print $ getSize (CDT1C3 4 2)
577− -- print $ getSize tInt
578− -- print $ getSize lInt
579−
580− -- pokeTimes tInt 1
581−
582− -- peekTimes ((CDT1C2 (5 :: Int)) :: CustomDT1) value
583− -- peekTimes (Sum2525) value
584− -- peekTimes (Product25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25) value
585− -- peekTimes tInt 1
586− let !n = getSize lInt
587− peekTimes n lInt 1
588−
589− -- roundtrip ((CDT1C2 (5 :: Int)) :: CustomDT1) value
590− return ()
591−#endif
benchmark/Streamly/Benchmark/Data/Serialize/RecCompatible.hsdeleted−24
@@ -1,24 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.RecCompatible
4− ( RecCompatible(..)
5− , valRecCompatible
6− ) where
7−
8−--------------------------------------------------------------------------------
9−-- Imports
10−--------------------------------------------------------------------------------
11−
12−import Streamly.Internal.Data.MutByteArray (Serialize)
13−import Streamly.Benchmark.Data.Serialize.TH (genLargeRecord)
14−
15−import qualified Streamly.Internal.Data.MutByteArray as Serialize
16−
17−--------------------------------------------------------------------------------
18−-- Code
19−--------------------------------------------------------------------------------
20−
21−$(genLargeRecord "RecCompatible" 50)
22−$(Serialize.deriveSerializeWith
23− (Serialize.encodeRecordFields True)
24− [d|instance Serialize RecCompatible|])
benchmark/Streamly/Benchmark/Data/Serialize/RecNonCompatible.hsdeleted−21
@@ -1,21 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.RecNonCompatible
4− ( RecNonCompatible(..)
5− , valRecNonCompatible
6− ) where
7−
8−--------------------------------------------------------------------------------
9−-- Imports
10−--------------------------------------------------------------------------------
11−
12−import Streamly.Benchmark.Data.Serialize.TH (genLargeRecord)
13−
14−import qualified Streamly.Internal.Data.MutByteArray as Serialize
15−
16−--------------------------------------------------------------------------------
17−-- Code
18−--------------------------------------------------------------------------------
19−
20−$(genLargeRecord "RecNonCompatible" 50)
21−$(Serialize.deriveSerialize [d|instance Serialize.Serialize RecNonCompatible|])
benchmark/Streamly/Benchmark/Data/Serialize/TH.hsdeleted−76
@@ -1,76 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.TH (genLargeRecord) where
4−
5−--------------------------------------------------------------------------------
6−-- Imports
7−--------------------------------------------------------------------------------
8−
9−import Language.Haskell.TH
10−
11−import Streamly.Internal.Data.MutByteArray (makeI)
12−import Control.DeepSeq (NFData(..))
13−
14−--------------------------------------------------------------------------------
15−-- Large Record
16−--------------------------------------------------------------------------------
17−
18−genLargeRecord :: String -> Int -> Q [Dec]
19−genLargeRecord tyName numFields =
20− sequence
21− [ dataD
22− (pure [])
23− (mkName tyName)
24− []
25− Nothing
26− [mkCon tyName]
27− [derivClause Nothing [conT ''Eq, conT ''Show]]
28− , mkValueSigDec
29− , mkValueDec
30− , nfDataInstance tyName
31− ]
32−
33− where
34−
35− fieldTypeChoices = [conT ''()]
36− chooseCycle i xs = xs !! (i `mod` length xs)
37− nfDataInstance nm =
38− instanceD
39− (pure [])
40− (appT (conT ''NFData) (conT (mkName nm)))
41− [ funD
42− 'rnf
43− [ clause
44− [ conP
45− (mkName nm)
46− (varP . makeI <$> [0 .. (numFields - 1)])
47− ]
48− (normalB
49− (foldl
50− (\b a -> [|rnf $(b) `seq` rnf $(a)|])
51− [|()|]
52− (varE . makeI <$> [0 .. (numFields - 1)])))
53− []
54− ]
55− ]
56− valueName = mkName $ "val" ++ tyName
57− mkValueSigDec = sigD valueName [t|$(conT (mkName tyName))|]
58− mkValueDec =
59− funD
60− valueName
61− [ clause
62− []
63− (normalB
64− (foldl
65− (\b a -> [|$(b) $(a)|])
66− (conE (mkName tyName))
67− (conE '() <$ [0 .. (numFields - 1)])))
68− []
69− ]
70− mkCon nm = recC (mkName nm) (mkField <$> [0 .. (numFields - 1)])
71− mkField i =
72− varBangType
73− (mkName ("field" ++ show i))
74− (bangType
75− (bang noSourceUnpackedness noSourceStrictness)
76− (chooseCycle i fieldTypeChoices))
benchmark/Streamly/Benchmark/Data/Stream/Common.hs+1−3
@@ -71,9 +71,7 @@ module Stream.Common
7171 )
7272where
7373
74−#if !MIN_VERSION_base(4,18,0)
7574import Control.Applicative (liftA2)
76−#endif
7775import Control.DeepSeq (NFData)
7876import Control.Exception (try)
7977import GHC.Exception (ErrorCall)
@@ -104,7 +102,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
104102#endif
105103#endif
106104
107−import Test.Tasty.Bench
105+import Gauge
108106import Prelude hiding (Foldable(..), mapM, replicate)
109107
110108#ifdef USE_STREAMK
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentCommon.hs+3−3
@@ -14,14 +14,14 @@ where
1414import Stream.Common
1515 (composeN, benchIO, benchIOSink, benchIOSrc, sourceUnfoldrM)
1616import Streamly.Data.Stream (Stream)
17−import Streamly.Internal.Data.Stream.Prelude (MonadAsync, Config)
17+import Streamly.Internal.Data.Stream.Concurrent (MonadAsync, Config)
1818
1919import qualified Data.List as List
2020import qualified Streamly.Data.Fold as Fold
2121import qualified Streamly.Data.Stream as Stream
22−import qualified Streamly.Internal.Data.Stream.Prelude as Async
22+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
2323
24−import Test.Tasty.Bench
24+import Gauge
2525import Prelude hiding (mapM)
2626import Streamly.Benchmark.Common
2727
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentEager.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentEager"
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentInterleaved.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentInterleaved"
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentOrdered.hs+1−1
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentOrdered"
benchmark/Streamly/Benchmark/Data/Stream/Eliminate.hs+1−3
@@ -57,7 +57,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
5757#endif
5858#endif
5959
60−import Test.Tasty.Bench
60+import Gauge
6161import Streamly.Benchmark.Common
6262import Prelude hiding (length, sum, or, and, any, all, notElem, elem, (!!),
6363 lookup, repeat, minimum, maximum, product, last, mapM_, init)
@@ -132,14 +132,12 @@ foldableAny :: Int -> Int -> Bool
132132foldableAny value n =
133133 Prelude.any (> (value + 1)) (sourceUnfoldr value n :: Stream Identity Int)
134134
135−{- HLINT ignore "Use all"-}
136135{-# INLINE foldableAnd #-}
137136foldableAnd :: Int -> Int -> Bool
138137foldableAnd value n =
139138 Prelude.and $ fmap
140139 (<= (value + 1)) (sourceUnfoldr value n :: Stream Identity Int)
141140
142−{- HLINT ignore "Use any"-}
143141{-# INLINE foldableOr #-}
144142foldableOr :: Int -> Int -> Bool
145143foldableOr value n =
benchmark/Streamly/Benchmark/Data/Stream/Exceptions.hs+3−12
@@ -36,22 +36,21 @@ import System.IO (Handle, hClose, hPutChar)
3636import qualified Streamly.FileSystem.Handle as FH
3737import qualified Streamly.Internal.FileSystem.Handle as IFH
3838import qualified Streamly.Internal.Data.Unfold as IUF
39−import qualified Streamly.Internal.Data.Unfold.Prelude as IUF
39+import qualified Streamly.Internal.Data.Unfold.Exception as IUF
4040
4141#ifdef USE_PRELUDE
4242import qualified Streamly.Internal.Data.Stream.IsStream as Stream
4343#else
4444import qualified Streamly.Internal.Data.Stream as Stream
45−import qualified Streamly.Internal.Data.Stream.Prelude as Stream
45+import qualified Streamly.Internal.Data.Stream.Exception.Lifted as Stream
4646#endif
4747
48−import Test.Tasty.Bench hiding (env)
48+import Gauge hiding (env)
4949import Prelude hiding (last, length)
5050import Streamly.Benchmark.Common
5151import Streamly.Benchmark.Common.Handle
5252
5353#ifdef INSPECTION
54−import Control.Monad.Catch (MonadCatch)
5554import Test.Inspection
5655
5756import qualified Streamly.Internal.Data.Stream as D
@@ -287,12 +286,8 @@ readChunksOnException inh devNull =
287286 in IUF.fold (IFH.writeChunks devNull) readEx inh
288287
289288#ifdef INSPECTION
290−#if __GLASGOW_HASKELL__ >= 906
291−inspect $ hasNoTypeClassesExcept 'readChunksOnException [''MonadCatch]
292−#else
293289inspect $ hasNoTypeClasses 'readChunksOnException
294290#endif
295−#endif
296291
297292-- | Send the file contents to /dev/null with exception handling
298293readChunksBracket_ :: Handle -> Handle -> IO ()
@@ -301,12 +296,8 @@ readChunksBracket_ inh devNull =
301296 in IUF.fold (IFH.writeChunks devNull) readEx inh
302297
303298#ifdef INSPECTION
304−#if __GLASGOW_HASKELL__ >= 906
305−inspect $ hasNoTypeClassesExcept 'readChunksBracket_ [''MonadCatch]
306−#else
307299inspect $ hasNoTypeClasses 'readChunksBracket_
308300#endif
309−#endif
310301
311302readChunksBracket :: Handle -> Handle -> IO ()
312303readChunksBracket inh devNull =
benchmark/Streamly/Benchmark/Data/Stream/Expand.hs+1−9
@@ -54,7 +54,7 @@ import qualified Streamly.Internal.Data.Stream as StreamK
5454#endif
5555#endif
5656
57−import Test.Tasty.Bench
57+import Gauge
5858import Stream.Common
5959import Streamly.Benchmark.Common
6060import Prelude hiding (concatMap)
@@ -198,11 +198,7 @@ concatMapPure outer inner n =
198198 (fromStream $ sourceUnfoldr outer n)
199199
200200#ifdef INSPECTION
201−#if __GLASGOW_HASKELL__ >= 906
202−inspect $ hasNoTypeClassesExcept 'concatMapPure [''Applicative]
203−#else
204201inspect $ hasNoTypeClasses 'concatMapPure
205−#endif
206202inspect $ 'concatMapPure `hasNoType` ''SPEC
207203#endif
208204
@@ -215,11 +211,7 @@ concatMapRepl outer inner n =
215211 (fromStream . S.replicate inner) (fromStream $ sourceUnfoldrM outer n)
216212
217213#ifdef INSPECTION
218−#if __GLASGOW_HASKELL__ >= 906
219−inspect $ hasNoTypeClassesExcept 'concatMapRepl [''Applicative]
220−#else
221214inspect $ hasNoTypeClasses 'concatMapRepl
222−#endif
223215inspect $ 'concatMapRepl `hasNoType` ''SPEC
224216#endif
225217
benchmark/Streamly/Benchmark/Data/Stream/Generate.hs+1−1
@@ -40,7 +40,7 @@ import qualified Stream.Common as Common
4040#endif
4141#endif
4242
43−import Test.Tasty.Bench
43+import Gauge
4444import Streamly.Benchmark.Common
4545import qualified Prelude
4646
benchmark/Streamly/Benchmark/Data/Stream/Lift.hs+1−1
@@ -37,7 +37,7 @@ import qualified Control.Monad.State.Strict as State
3737#endif
3838#endif
3939
40−import Test.Tasty.Bench
40+import Gauge
4141import Streamly.Benchmark.Common
4242import Prelude hiding (reverse, tail)
4343
benchmark/Streamly/Benchmark/Data/Stream/Reduce.hs+2−2
@@ -48,7 +48,7 @@ import Streamly.Internal.Data.Stream (Stream)
4848import qualified Streamly.Internal.Data.Stream as S
4949#ifndef USE_STREAMLY_CORE
5050import qualified Streamly.Data.Stream.Prelude as S
51−import qualified Streamly.Internal.Data.Stream.Prelude as S
51+import qualified Streamly.Internal.Data.Stream.Time as S
5252#endif
5353
5454#ifdef USE_STREAMK
@@ -61,7 +61,7 @@ import qualified Streamly.Internal.Data.Stream as K
6161
6262#endif
6363
64−import Test.Tasty.Bench
64+import Gauge
6565import Streamly.Benchmark.Common
6666import Stream.Common
6767import Prelude hiding (reverse, tail)
benchmark/Streamly/Benchmark/Data/Stream/Split.hs+2−2
@@ -35,7 +35,7 @@ import qualified Streamly.Internal.FileSystem.Handle as IFH
3535import qualified Streamly.Internal.Unicode.Stream as IUS
3636import qualified Streamly.Prelude as S
3737
38−import Test.Tasty.Bench hiding (env)
38+import Gauge hiding (env)
3939import Prelude hiding (last, length)
4040import Streamly.Benchmark.Common
4141import Streamly.Benchmark.Common.Handle
@@ -43,7 +43,7 @@ import Streamly.Benchmark.Common.Handle
4343#ifdef INSPECTION
4444import Streamly.Internal.Data.Stream (Step(..))
4545
46−import qualified Streamly.Internal.Data.MutArray as MA
46+import qualified Streamly.Internal.Data.MutArray.Type as MA
4747import qualified Streamly.Internal.Data.Unfold as IUF
4848
4949import Test.Inspection
benchmark/Streamly/Benchmark/Data/Stream/StreamD.hs+2−4
@@ -22,12 +22,10 @@
2222
2323module Main (main) where
2424
25−#if !MIN_VERSION_base(4,18,0)
2625import Control.Applicative (liftA2)
27−#endif
2826import Control.Monad (when)
2927import Data.Maybe (isJust)
30−import Test.Tasty.Bench (bench, nfIO, bgroup, Benchmark, defaultMain)
28+import Gauge (bench, nfIO, bgroup, Benchmark, defaultMain)
3129import System.Random (randomRIO)
3230import Prelude hiding (tail, mapM_, foldl, last, map, mapM, concatMap, zip)
3331
@@ -463,7 +461,7 @@ inspect $ 'unfoldManyRepl `hasNoType` ''SPEC
463461-------------------------------------------------------------------------------
464462
465463moduleName :: String
466−moduleName = "Data.StreamD"
464+moduleName = "Data.Stream.StreamD"
467465
468466o_1_space :: [Benchmark]
469467o_1_space =
benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs+3−4
@@ -21,9 +21,7 @@
2121
2222module Main (main) where
2323
24−#if !MIN_VERSION_base(4,18,0)
2524import Control.Applicative (liftA2)
26−#endif
2725import Control.Monad (when)
2826import Data.Maybe (isJust)
2927import System.Random (randomRIO)
@@ -35,9 +33,10 @@ import Prelude hiding
3533import qualified Prelude as P
3634import qualified Data.List as List
3735
36+import qualified Streamly.Internal.Data.Stream.StreamK.Type as S
3837import qualified Streamly.Internal.Data.StreamK as S
3938
40−import Test.Tasty.Bench (bench, nfIO, bgroup, Benchmark, defaultMain)
39+import Gauge (bench, nfIO, bgroup, Benchmark, defaultMain)
4140
4241import Streamly.Benchmark.Common
4342
@@ -634,7 +633,7 @@ filterAllInNestedList str = do
634633-------------------------------------------------------------------------------
635634
636635moduleName :: String
637−moduleName = "Data.StreamK"
636+moduleName = "Data.Stream.StreamK"
638637
639638o_1_space_generation :: Int -> Benchmark
640639o_1_space_generation streamLen =
benchmark/Streamly/Benchmark/Data/Stream/StreamKAlt.hs+3−3
@@ -18,12 +18,12 @@ import Prelude hiding ()
1818-- import qualified Prelude as P
1919-- import qualified Data.List as List
2020
21−import qualified Streamly.Internal.Data.StreamK.Alt as S
21+import qualified Streamly.Internal.Data.Stream.StreamK.Alt as S
2222-- import qualified Streamly.Internal.Data.Stream.Common as SP
2323-- import qualified Streamly.Internal.Data.SVar as S
2424
2525import Streamly.Benchmark.Common
26−import Test.Tasty.Bench (bgroup, Benchmark, defaultMain)
26+import Gauge (bgroup, Benchmark, defaultMain)
2727
2828value :: Int
2929value = 100000
@@ -452,4 +452,4 @@ o_1_space =
452452 ]
453453
454454main :: IO ()
455−main = defaultMain [o_1_space]
455+main = defaultMain $ concat [o_1_space]
benchmark/Streamly/Benchmark/Data/Stream/ToStreamK.hs+4−3
@@ -1,5 +1,5 @@
11-- |
2−-- Module : Streamly.Benchmark.Data.StreamK
2+-- Module : Streamly.Benchmark.Data.Stream.StreamK
33-- Copyright : (c) 2018 Composewell Technologies
44--
55-- License : BSD3
@@ -33,11 +33,12 @@ import Prelude hiding
3333import qualified Prelude as P
3434-- import qualified Data.List as List
3535
36+import qualified Streamly.Internal.Data.Stream.StreamK.Type as S
3637import qualified Streamly.Internal.Data.StreamK as S
3738import qualified Streamly.Internal.Data.Stream as D
3839import qualified Streamly.Internal.Data.Fold as Fold
3940
40−import Test.Tasty.Bench (bgroup, Benchmark, defaultMain)
41+import Gauge (bgroup, Benchmark, defaultMain)
4142
4243import Streamly.Benchmark.Common
4344
@@ -120,7 +121,7 @@ fromFoldable streamLen n = S.fromFoldable [n..n+streamLen]
120121{-# INLINE fromFoldableM #-}
121122fromFoldableM :: Monad m => Int -> Int -> Stream m Int
122123fromFoldableM streamLen n =
123− Prelude.foldr (S.consM . return) S.nil [n .. n + streamLen]
124+ Prelude.foldr S.consM S.nil (Prelude.fmap return [n..n+streamLen])
124125
125126{-
126127{-# INLINABLE concatMapFoldableWith #-}
benchmark/Streamly/Benchmark/Data/Stream/Transform.hs+2−2
@@ -46,7 +46,7 @@ import Streamly.Internal.Data.Time.Units
4646import Streamly.Internal.Data.Stream (Stream)
4747import qualified Streamly.Internal.Data.Stream as Stream
4848#ifndef USE_STREAMLY_CORE
49−import qualified Streamly.Internal.Data.Stream.Prelude as Stream
49+import qualified Streamly.Internal.Data.Stream.Time as Stream
5050#endif
5151#ifdef USE_STREAMK
5252import Control.DeepSeq (NFData(..))
@@ -58,7 +58,7 @@ import qualified Streamly.Internal.Data.StreamK as StreamK
5858#endif
5959#endif
6060
61−import Test.Tasty.Bench
61+import Gauge
6262import Stream.Common hiding (scanl')
6363import Streamly.Benchmark.Common
6464import Prelude hiding (sequence, mapM)
benchmark/Streamly/Benchmark/Data/Unbox.hsnew file+102
@@ -0,0 +1,102 @@
1+{-# LANGUAGE TemplateHaskell #-}
2+
3+-- |
4+-- Module : Streamly.Benchmark.Data.Unbox
5+-- Copyright : (c) 2023 Composewell
6+-- License : BSD-3-Clause
7+-- Maintainer : streamly@composewell.com
8+
9+module Main (main) where
10+
11+-------------------------------------------------------------------------------
12+-- Imports
13+-------------------------------------------------------------------------------
14+
15+import Control.DeepSeq (NFData(..))
16+import Control.Exception (assert)
17+import Control.Monad (replicateM_)
18+import Data.Proxy (Proxy(..))
19+import GHC.Generics (Generic)
20+import System.Random (randomRIO)
21+
22+import Streamly.Internal.Data.Unbox
23+import Streamly.Internal.Data.Unbox.TH
24+
25+import Gauge
26+import Streamly.Benchmark.Common
27+
28+-------------------------------------------------------------------------------
29+-- Types
30+-------------------------------------------------------------------------------
31+
32+data CustomDT1 a b
33+ = CDT1C1
34+ | CDT1C2 a
35+ | CDT1C3 a b
36+ deriving (Generic, Show, Eq)
37+type CustomDT1_ = CustomDT1 Int Bool
38+
39+instance (Unbox a, Unbox b) => Unbox (CustomDT1 a b)
40+
41+data CustomDT2 a b
42+ = CDT2C1
43+ | CDT2C2 a
44+ | CDT2C3 a b
45+ deriving (Show, Eq)
46+type CustomDT2_ = CustomDT2 Int Bool
47+
48+$(deriveUnbox ''CustomDT2)
49+
50+-------------------------------------------------------------------------------
51+-- Helpers
52+-------------------------------------------------------------------------------
53+
54+serializeDeserializeTimes :: forall a. (Eq a, Unbox a) => a -> Int -> IO ()
55+serializeDeserializeTimes val times = do
56+ arr <- newBytes (sizeOf (Proxy :: Proxy a))
57+ replicateM_ times $ do
58+ pokeByteIndex 0 arr val
59+ val1 <- peekByteIndex 0 arr
60+ assert (val == val1) (pure ())
61+
62+benchSink :: NFData b => String -> Int -> (Int -> IO b) -> Benchmark
63+benchSink name times f = bench name (nfIO (randomRIO (times, times) >>= f))
64+
65+-------------------------------------------------------------------------------
66+-- Benchmarks
67+-------------------------------------------------------------------------------
68+
69+allBenchmarks :: Int -> [Benchmark]
70+allBenchmarks times =
71+ [ benchSink
72+ "serializeDeserializeTimes CDT1C1 (Generic)"
73+ times
74+ (serializeDeserializeTimes (CDT1C1 :: CustomDT1_))
75+ , benchSink
76+ "serializeDeserializeTimes CDT2C1 (TH)"
77+ times
78+ (serializeDeserializeTimes (CDT2C1 :: CustomDT2_))
79+ , benchSink
80+ "serializeDeserializeTimes CDT1C2 (Generic)"
81+ times
82+ (serializeDeserializeTimes ((CDT1C2 (5 :: Int)) :: CustomDT1_))
83+ , benchSink
84+ "serializeDeserializeTimes CDT2C2 (TH)"
85+ times
86+ (serializeDeserializeTimes ((CDT2C2 (5 :: Int)) :: CustomDT2_))
87+ , benchSink
88+ "serializeDeserializeTimes CDT1C3 (Generic)"
89+ times
90+ (serializeDeserializeTimes ((CDT1C3 (5 :: Int) True) :: CustomDT1_))
91+ , benchSink
92+ "serializeDeserializeTimes CDT2C3 (TH)"
93+ times
94+ (serializeDeserializeTimes ((CDT2C3 (5 :: Int) True) :: CustomDT2_))
95+ ]
96+
97+-------------------------------------------------------------------------------
98+-- Driver
99+-------------------------------------------------------------------------------
100+
101+main :: IO ()
102+main = runWithCLIOpts defaultStreamSize allBenchmarks
benchmark/Streamly/Benchmark/Data/Unfold.hs+2−11
@@ -31,18 +31,17 @@ import qualified Prelude
3131import qualified Streamly.FileSystem.Handle as FH
3232import qualified Streamly.Internal.Data.Fold as FL
3333import qualified Streamly.Internal.Data.Unfold as UF
34−import qualified Streamly.Internal.Data.Unfold.Prelude as UF
34+import qualified Streamly.Internal.Data.Unfold.Exception as UF
3535import qualified Streamly.Internal.Data.Stream as S
3636import qualified Streamly.Internal.Data.Stream as D
3737import qualified Streamly.Internal.Data.StreamK as K
3838
39−import Test.Tasty.Bench hiding (env)
39+import Gauge hiding (env)
4040import Prelude hiding (take, filter, zipWith, map, mapM, takeWhile)
4141import Streamly.Benchmark.Common
4242import Streamly.Benchmark.Common.Handle
4343
4444#ifdef INSPECTION
45−import Control.Monad.Catch (MonadCatch)
4645import Test.Inspection
4746#endif
4847
@@ -777,11 +776,7 @@ readWriteFinally_Unfold inh devNull =
777776 in S.fold (FH.write devNull) $ S.unfold readEx inh
778777
779778#ifdef INSPECTION
780−#if __GLASGOW_HASKELL__ >= 906
781−inspect $ hasNoTypeClassesExcept 'readWriteFinally_Unfold [''MonadCatch]
782−#else
783779inspect $ hasNoTypeClasses 'readWriteFinally_Unfold
784−#endif
785780-- inspect $ 'readWriteFinallyUnfold `hasNoType` ''Step
786781#endif
787782
@@ -797,11 +792,7 @@ readWriteBracket_Unfold inh devNull =
797792 in S.fold (FH.write devNull) $ S.unfold readEx inh
798793
799794#ifdef INSPECTION
800−#if __GLASGOW_HASKELL__ >= 906
801−inspect $ hasNoTypeClassesExcept 'readWriteBracket_Unfold [''MonadCatch]
802−#else
803795inspect $ hasNoTypeClasses 'readWriteBracket_Unfold
804−#endif
805796-- inspect $ 'readWriteBracketUnfold `hasNoType` ''Step
806797#endif
807798
benchmark/Streamly/Benchmark/FileSystem/Handle.hs+1−1
@@ -22,7 +22,7 @@ import Streamly.Benchmark.Common.Handle (mkHandleBenchEnv)
2222import qualified Handle.ReadWrite as RW
2323import qualified Handle.Read as RO
2424
25−import Test.Tasty.Bench hiding (env)
25+import Gauge hiding (env)
2626import Prelude hiding (last, length)
2727import Streamly.Benchmark.Common
2828
benchmark/Streamly/Benchmark/FileSystem/Handle/Read.hs+9−8
@@ -33,6 +33,7 @@ import qualified Streamly.Data.Stream as Stream
3333import qualified Streamly.Data.Fold as Fold
3434import qualified Streamly.FileSystem.Handle as FH
3535import qualified Streamly.Internal.Data.Array as A
36+import qualified Streamly.Internal.Data.Array.Type as AT
3637import qualified Streamly.Internal.Data.Fold as FL
3738import qualified Streamly.Internal.Data.Stream as IP
3839import qualified Streamly.Internal.FileSystem.Handle as IFH
@@ -40,14 +41,14 @@ import qualified Streamly.Internal.Unicode.Stream as IUS
4041import qualified Streamly.Data.Stream.Prelude as S
4142import qualified Streamly.Unicode.Stream as SS
4243
43−import Test.Tasty.Bench hiding (env)
44+import Gauge hiding (env)
4445import Prelude hiding (last, length)
4546import Streamly.Benchmark.Common.Handle
4647
4748#ifdef INSPECTION
4849import Streamly.Internal.Data.Stream (Step(..), FoldMany)
4950
50−import qualified Streamly.Internal.Data.MutArray as MutArray
51+import qualified Streamly.Internal.Data.MutArray.Type as MA
5152import qualified Streamly.Internal.Data.Stream as D
5253import qualified Streamly.Internal.Data.Unfold as IUF
5354
@@ -68,7 +69,7 @@ readLast = S.fold Fold.last . S.unfold FH.reader
6869inspect $ hasNoTypeClasses 'readLast
6970inspect $ 'readLast `hasNoType` ''Step -- S.unfold
7071inspect $ 'readLast `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
71−inspect $ 'readLast `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
72+inspect $ 'readLast `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
7273#endif
7374
7475-- assert that flattenArrays constructors are not present
@@ -80,7 +81,7 @@ readCountBytes = S.fold Fold.length . S.unfold FH.reader
8081inspect $ hasNoTypeClasses 'readCountBytes
8182inspect $ 'readCountBytes `hasNoType` ''Step -- S.unfold
8283inspect $ 'readCountBytes `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
83−inspect $ 'readCountBytes `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
84+inspect $ 'readCountBytes `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
8485#endif
8586
8687-- | Count the number of lines in a file.
@@ -95,7 +96,7 @@ readCountLines =
9596inspect $ hasNoTypeClasses 'readCountLines
9697inspect $ 'readCountLines `hasNoType` ''Step
9798inspect $ 'readCountLines `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
98−inspect $ 'readCountLines `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
99+inspect $ 'readCountLines `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
99100#endif
100101
101102-- | Count the number of words in a file.
@@ -119,7 +120,7 @@ readSumBytes = S.fold Fold.sum . S.unfold FH.reader
119120inspect $ hasNoTypeClasses 'readSumBytes
120121inspect $ 'readSumBytes `hasNoType` ''Step
121122inspect $ 'readSumBytes `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
122−inspect $ 'readSumBytes `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
123+inspect $ 'readSumBytes `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
123124#endif
124125
125126-- XXX When we mark this with INLINE and we have two benchmarks using S.drain
@@ -235,13 +236,13 @@ groupsOf :: Int -> Handle -> IO Int
235236groupsOf n inh =
236237 -- writeNUnsafe gives 2.5x boost here over writeN.
237238 S.fold Fold.length
238− $ IP.groupsOf n (A.writeNUnsafe n) (S.unfold FH.reader inh)
239+ $ IP.groupsOf n (AT.writeNUnsafe n) (S.unfold FH.reader inh)
239240
240241#ifdef INSPECTION
241242inspect $ hasNoTypeClasses 'groupsOf
242243inspect $ 'groupsOf `hasNoType` ''Step
243244inspect $ 'groupsOf `hasNoType` ''FoldMany
244−inspect $ 'groupsOf `hasNoType` ''MutArray.ArrayUnsafe -- AT.writeNUnsafe
245+inspect $ 'groupsOf `hasNoType` ''AT.ArrayUnsafe -- AT.writeNUnsafe
245246 -- FH.read/A.read
246247inspect $ 'groupsOf `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
247248#endif
benchmark/Streamly/Benchmark/FileSystem/Handle/ReadWrite.hs+10−9
@@ -34,7 +34,7 @@ import qualified Streamly.Internal.FileSystem.Handle as IFH
3434import qualified Streamly.Data.Array as A
3535import qualified Streamly.Data.Stream.Prelude as S
3636
37−import Test.Tasty.Bench hiding (env)
37+import Gauge hiding (env)
3838import Streamly.Benchmark.Common.Handle
3939
4040#ifdef INSPECTION
@@ -42,7 +42,8 @@ import Streamly.Internal.Data.Stream (Step(..))
4242
4343import qualified Streamly.Internal.Data.Stream as D
4444import qualified Streamly.Internal.Data.Tuple.Strict as Strict
45−import qualified Streamly.Internal.Data.MutArray as MutArray
45+import qualified Streamly.Internal.Data.Stream.MutChunked as MAS
46+import qualified Streamly.Internal.Data.Array.Type as AT
4647
4748import Test.Inspection
4849#endif
@@ -82,7 +83,7 @@ copyStream inh outh = S.fold (FH.write outh) (S.unfold FH.reader inh)
8283inspect $ hasNoTypeClasses 'copyStream
8384inspect $ 'copyStream `hasNoType` ''Step -- S.unfold
8485inspect $ 'copyStream `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
85−inspect $ 'copyStream `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
86+inspect $ 'copyStream `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
8687 -- FH.read/A.read
8788inspect $ 'copyStream `hasNoType` ''Strict.Tuple3' -- FH.write/chunksOf
8889#endif
@@ -108,8 +109,8 @@ readFromBytesNull inh devNull = IFH.putBytes devNull $ S.unfold FH.reader inh
108109#ifdef INSPECTION
109110inspect $ hasNoTypeClasses 'readFromBytesNull
110111inspect $ 'readFromBytesNull `hasNoType` ''Step
111−inspect $ 'readFromBytesNull `hasNoType` ''MutArray.SpliceState
112−inspect $ 'readFromBytesNull `hasNoType` ''MutArray.ArrayUnsafe -- FH.fromBytes/S.chunksOf
112+inspect $ 'readFromBytesNull `hasNoType` ''MAS.SpliceState
113+inspect $ 'readFromBytesNull `hasNoType` ''AT.ArrayUnsafe -- FH.fromBytes/S.chunksOf
113114inspect $ 'readFromBytesNull `hasNoType` ''D.FoldMany
114115#endif
115116
@@ -122,8 +123,8 @@ readWithFromBytesNull inh devNull =
122123#ifdef INSPECTION
123124inspect $ hasNoTypeClasses 'readWithFromBytesNull
124125inspect $ 'readWithFromBytesNull `hasNoType` ''Step
125−inspect $ 'readWithFromBytesNull `hasNoType` ''MutArray.SpliceState
126−inspect $ 'readWithFromBytesNull `hasNoType` ''MutArray.ArrayUnsafe -- FH.fromBytes/S.chunksOf
126+inspect $ 'readWithFromBytesNull `hasNoType` ''MAS.SpliceState
127+inspect $ 'readWithFromBytesNull `hasNoType` ''AT.ArrayUnsafe -- FH.fromBytes/S.chunksOf
127128inspect $ 'readWithFromBytesNull `hasNoType` ''D.FoldMany
128129#endif
129130
@@ -170,7 +171,7 @@ writeReadWith inh devNull = IUF.fold fld unf (defaultChunkSize, inh)
170171inspect $ hasNoTypeClasses 'writeReadWith
171172inspect $ 'writeReadWith `hasNoType` ''Step
172173inspect $ 'writeReadWith `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
173−inspect $ 'writeReadWith `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
174+inspect $ 'writeReadWith `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
174175 -- FH.read/A.read
175176#endif
176177
@@ -187,7 +188,7 @@ writeRead inh devNull = IUF.fold fld unf inh
187188inspect $ hasNoTypeClasses 'writeRead
188189inspect $ 'writeRead `hasNoType` ''Step
189190inspect $ 'writeRead `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
190−inspect $ 'writeRead `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
191+inspect $ 'writeRead `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
191192 -- FH.read/A.read
192193#endif
193194
benchmark/Streamly/Benchmark/Prelude/Adaptive.hs+1−1
@@ -10,7 +10,7 @@
1010import Control.Concurrent (threadDelay)
1111import Control.Monad (when)
1212import Control.Monad.IO.Class (liftIO)
13−import Test.Tasty.Bench
13+import Gauge
1414import Streamly.Prelude as S
1515import System.Random (randomRIO)
1616
benchmark/Streamly/Benchmark/Prelude/Ahead.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.Ahead"
benchmark/Streamly/Benchmark/Prelude/Async.hs+1−1
@@ -16,7 +16,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream.Transform as Transform
1616import Streamly.Benchmark.Common
1717import Streamly.Benchmark.Prelude
1818
19−import Test.Tasty.Bench
19+import Gauge
2020
2121moduleName :: String
2222moduleName = "Prelude.Async"
benchmark/Streamly/Benchmark/Prelude/Concurrent.hs+1−1
@@ -15,7 +15,7 @@ import Streamly.Prelude
1515 , fromWAsync, parallel, fromParallel
1616 )
1717
18−import Test.Tasty.Bench
18+import Gauge
1919import qualified Streamly.Prelude as S
2020
2121-------------------------------------------------------------------------------
benchmark/Streamly/Benchmark/Prelude/Merge.hs+1−1
@@ -30,7 +30,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream as Stream
3030import Streamly.Benchmark.Common
3131import Streamly.Benchmark.Prelude
3232
33−import Test.Tasty.Bench
33+import Gauge
3434
3535#ifdef INSPECTION
3636import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Prelude/Parallel.hs+1−1
@@ -22,7 +22,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream as Internal
2222import Streamly.Benchmark.Common
2323import Streamly.Benchmark.Prelude
2424
25−import Test.Tasty.Bench
25+import Gauge
2626
2727moduleName :: String
2828moduleName = "Prelude.Parallel"
benchmark/Streamly/Benchmark/Prelude/Rate.hs+1−1
@@ -14,7 +14,7 @@ import qualified Streamly.Prelude as S
1414import Streamly.Benchmark.Common
1515import Streamly.Benchmark.Prelude
1616
17−import Test.Tasty.Bench
17+import Gauge
1818
1919moduleName :: String
2020moduleName = "Prelude.Rate"
benchmark/Streamly/Benchmark/Prelude/WAsync.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.WAsync"
benchmark/Streamly/Benchmark/Prelude/WSerial.hs+1−1
@@ -29,7 +29,7 @@ import qualified Streamly.Internal.Data.Unfold as UF
2929import Streamly.Benchmark.Common
3030import Streamly.Benchmark.Prelude
3131
32−import Test.Tasty.Bench
32+import Gauge
3333
3434#ifdef INSPECTION
3535import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Prelude/ZipAsync.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.ZipAsync"
benchmark/Streamly/Benchmark/Prelude/ZipSerial.hs+1−1
@@ -29,7 +29,7 @@ import qualified Streamly.Prelude as S
2929import Streamly.Benchmark.Common
3030import Streamly.Benchmark.Prelude hiding (sourceUnfoldrM)
3131
32−import Test.Tasty.Bench
32+import Gauge
3333
3434#ifdef INSPECTION
3535import GHC.Types (SPEC(..))
benchmark/Streamly/Benchmark/Unicode/Char.hs+1−1
@@ -16,7 +16,7 @@ import Control.DeepSeq (NFData(..))
1616import Streamly.Internal.Data.Array (Array)
1717import System.FilePath (dropExtensions, takeFileName)
1818import System.FilePath.Posix ((</>))
19−import Test.Tasty.Bench (Benchmark, bench, bgroup, defaultMain, env, nfIO)
19+import Gauge.Main (Benchmark, bench, bgroup, defaultMain, env, nfIO)
2020import Streamly.Internal.Unicode.Char
2121 ( NormalizationMode(NFC, NFD, NFKC, NFKD)
2222 , normalize
benchmark/Streamly/Benchmark/Unicode/Parser.hs+1−1
@@ -31,7 +31,7 @@ import qualified Streamly.Data.Array as Array
3131import qualified Streamly.Data.Stream as Stream
3232import qualified Streamly.Internal.Unicode.Parser as PRU
3333
34−import Test.Tasty.Bench hiding (env)
34+import Gauge hiding (env)
3535import Streamly.Benchmark.Common
3636
3737runParser :: Int -> (Stream IO Char -> IO a) -> IO ()
benchmark/Streamly/Benchmark/Unicode/Stream.hs+8−26
@@ -11,11 +11,6 @@
1111{-# LANGUAGE CPP #-}
1212{-# LANGUAGE ScopedTypeVariables #-}
1313
14−#undef FUSION_CHECK
15−#ifdef FUSION_CHECK
16−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
17−#endif
18−
1914#ifdef __HADDOCK_VERSION__
2015#undef INSPECTION
2116#endif
@@ -38,17 +33,16 @@ import qualified Streamly.Internal.FileSystem.Handle as Handle
3833import qualified Streamly.Internal.Unicode.Array as UnicodeArr
3934import qualified Streamly.Internal.Unicode.Stream as Unicode
4035
41−import Test.Tasty.Bench hiding (env)
36+import Gauge hiding (env)
4237import Streamly.Benchmark.Common
4338import Streamly.Benchmark.Common.Handle
4439
4540#ifdef INSPECTION
46−import Streamly.Internal.Data.MutByteArray (Unbox)
47−import Streamly.Internal.Data.Stream (Step(..))
48−import qualified Streamly.Internal.Data.Array as Array
49−import qualified Streamly.Internal.Data.MutArray as MutArray
50−import qualified Streamly.Internal.Data.Fold as Fold
41+import Streamly.Internal.Data.Unbox (Unbox)
42+import Streamly.Internal.Data.Stream.StreamD (Step(..))
43+import qualified Streamly.Internal.Data.Fold.Type as Fold
5144import qualified Streamly.Internal.Data.Tuple.Strict as Strict
45+import qualified Streamly.Internal.Data.Array.Type as Array
5246
5347import Test.Inspection
5448#endif
@@ -224,7 +218,7 @@ inspect $ 'copyStreamLatin1' `hasNoType` ''Step
224218inspect $ 'copyStreamLatin1' `hasNoType` ''Unfold.ConcatState -- Handle.read/UF.many
225219
226220inspect $ 'copyStreamLatin1' `hasNoType` ''Fold.Step
227−inspect $ 'copyStreamLatin1' `hasNoType` ''MutArray.ArrayUnsafe -- Handle.write/writeNUnsafe
221+inspect $ 'copyStreamLatin1' `hasNoType` ''Array.ArrayUnsafe -- Handle.write/writeNUnsafe
228222 -- Handle.read/Array.read
229223inspect $ 'copyStreamLatin1' `hasNoType` ''Strict.Tuple3' -- Handle.write/chunksOf
230224#endif
@@ -245,7 +239,7 @@ inspect $ 'copyStreamLatin1 `hasNoType` ''Unfold.ConcatState -- Handle.read/UF.m
245239
246240inspect $ 'copyStreamLatin1 `hasNoType` ''Fold.ManyState
247241inspect $ 'copyStreamLatin1 `hasNoType` ''Fold.Step
248−inspect $ 'copyStreamLatin1 `hasNoType` ''MutArray.ArrayUnsafe -- Handle.write/writeNUnsafe
242+inspect $ 'copyStreamLatin1 `hasNoType` ''Array.ArrayUnsafe -- Handle.write/writeNUnsafe
249243 -- Handle.read/Array.read
250244inspect $ 'copyStreamLatin1 `hasNoType` ''Strict.Tuple3' -- Handle.write/chunksOf
251245#endif
@@ -286,8 +280,7 @@ _copyStreamUtf8'Fold :: Handle -> Handle -> IO ()
286280_copyStreamUtf8'Fold inh outh =
287281 Stream.fold (Handle.write outh)
288282 $ Unicode.encodeUtf8
289− $ Stream.catRights
290− $ Stream.parseMany Unicode.writeCharUtf8'
283+ $ Stream.foldMany Unicode.writeCharUtf8'
291284 $ Stream.unfold Handle.reader inh
292285
293286{-# NOINLINE _copyStreamUtf8Parser #-}
@@ -324,7 +317,6 @@ o_1_space_decode_encode_read env =
324317
325318main :: IO ()
326319main = do
327−#ifndef FUSION_CHECK
328320 env <- mkHandleBenchEnv
329321 defaultMain (allBenchmarks env)
330322
@@ -337,13 +329,3 @@ main = do
337329 , o_1_space_decode_encode_read env
338330 ]
339331 ]
340−#else
341− -- Enable FUSION_CHECK macro at the beginning of the file
342− -- Enable one benchmark below, and run the benchmark
343− -- Check the .dump-simpl output
344− env <- mkHandleBenchEnv
345− let mkHandles (RefHandles {bigInH = inh, outputH = outh}) = Handles inh outh
346− (Handles inh outh) <- getHandles env mkHandles
347− copyStreamLatin1' inh outh
348− return ()
349−#endif
benchmark/Streamly/Benchmark/Unicode/Utf8.hs+1−1
@@ -13,7 +13,7 @@
1313import Data.Char (chr)
1414import Streamly.Internal.Unicode.Utf8 (Utf8)
1515
16−import Test.Tasty.Bench hiding (env)
16+import Gauge hiding (env)
1717import Streamly.Benchmark.Common
1818
1919import qualified Streamly.Internal.Unicode.Utf8 as Utf8
benchmark/bench-runner/Main.hs+6−34
@@ -16,11 +16,9 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
1616 benchName = drop 4 benchName0
1717
1818 general
19− -- GHC-9.6 requires 32M heap vs 16M for GHC-9.4
20− | "o-1-sp" `isInfixOf` benchName = "-K36K -M32M"
19+ | "o-1-sp" `isInfixOf` benchName = "-K36K -M16M"
2120 | "o-n-h" `isInfixOf` benchName = "-K36K -M32M"
22− -- GHC-9.6 requires 32M heap vs 16M for GHC-9.4
23− | "o-n-st" `isInfixOf` benchName = "-K1M -M32M"
21+ | "o-n-st" `isInfixOf` benchName = "-K1M -M16M"
2422 | "o-n-sp" `isInfixOf` benchName = "-K1M -M32M"
2523 | otherwise = ""
2624
@@ -29,17 +27,6 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
2927 | otherwise = ""
3028
3129 benchSpecific
32− -- GHC-9.6 requires 64M, earlier it was 16M
33− | "Data.Fold/o-n-heap.key-value.toHashMapIO (max buckets) sum" == benchName =
34− "-M64M"
35−
36− -- This is required only for the --long case because we allocate all
37− -- the arrays upfront. depends on the size of the stream.
38− | "Data.Parser/o-1-space" `isPrefixOf` benchName =
39− "-M128M"
40−
41− -----------------------------------------------------------------------
42−
4330 | "Prelude.Parallel/o-n-heap.mapping.mapM" == benchName = "-M256M"
4431 | "Prelude.Parallel/o-n-heap.monad-outer-product."
4532 `isPrefixOf` benchName = "-M256M"
@@ -70,18 +57,12 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
7057 -- GHC 9.4.4 requires 4M
7158 | "Data.Stream.StreamDK/o-n-space.traversable."
7259 `isPrefixOf` benchName = "-K4M"
73− -- GHC-9.6 requires 64M, earlier it was 32M
74− | "Data.Stream.StreamDK/o-n-heap.buffered.readsPrec pure streams" == benchName =
75− "-M64M"
76− -- GHC-9.6 requires 64M, earlier it was 32M
77− | "Data.Stream.StreamDK/o-n-heap.buffered.showPrec Haskell lists" == benchName =
78− "-M64M"
7960
8061 -----------------------------------------------------------------------
8162
82− | "Data.StreamD/o-n-space.elimination.toList" == benchName =
63+ | "Data.Stream.StreamD/o-n-space.elimination.toList" == benchName =
8364 "-K2M"
84− | "Data.StreamK/o-n-space.elimination.toList" == benchName =
65+ | "Data.Stream.StreamK/o-n-space.elimination.toList" == benchName =
8566 "-K2M"
8667
8768 -----------------------------------------------------------------------
@@ -93,13 +74,6 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
9374 | "Data.Stream/o-n-space.iterated."
9475 `isPrefixOf` benchName = "-K4M"
9576
96− -- GHC-9.6 requires 64M, earlier it was 32M
97− | "Data.Stream/o-n-heap.buffered.showPrec Haskell lists" == benchName =
98− "-M64M"
99− -- GHC-9.6 requires 64M, earlier it was 32M
100− | "Data.Stream/o-n-heap.buffered.readsPrec pure streams" == benchName =
101− "-M64M"
102−
10377 | "Data.Stream.ConcurrentEager/o-n-heap.monad-outer-product.toNullAp"
10478 `isPrefixOf` benchName = "-M1024M"
10579 | "Data.Stream.ConcurrentEager/o-1-space."
@@ -112,12 +86,10 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
11286
11387 | "Data.Array" `isPrefixOf` benchName
11488 && "/o-1-space.generation.read" `isSuffixOf` benchName = "-M32M"
115− -- XXX GHC 9.6 onwards needs 64M, earlier it was 32M
11689 | "Data.Array" `isPrefixOf` benchName
117− && "/o-1-space.generation.show" `isSuffixOf` benchName = "-M64M"
118− -- XXX GHC 9.6 onwards needs 64M, earlier it was 32M
90+ && "/o-1-space.generation.show" `isSuffixOf` benchName = "-M32M"
11991 | "Data.Array.Generic/o-1-space.transformationX4.map"
120− `isPrefixOf` benchName = "-M64M"
92+ `isPrefixOf` benchName = "-M32M"
12193
12294 -- XXX For --long option, need to check why so much heap is required.
12395 -- Note, if we remove the chunked stream module we need to keep the
benchmark/lib/Streamly/Benchmark/Common.hs+1−1
@@ -56,7 +56,7 @@ import System.Random (randomRIO)
5656import qualified Streamly.Internal.Data.Fold as Fold
5757import qualified Streamly.Internal.Data.Stream as S
5858
59−import Test.Tasty.Bench
59+import Gauge
6060
6161-------------------------------------------------------------------------------
6262-- Benchmark Prefixes
benchmark/lib/Streamly/Benchmark/Common/Handle.hs+1−3
@@ -30,8 +30,6 @@ module Streamly.Benchmark.Common.Handle
3030 , isSpace
3131 , isSp
3232 , mkHandleBenchEnv
33− , Handles(..)
34− , getHandles
3533 )
3634where
3735
@@ -48,7 +46,7 @@ import System.Process (callCommand)
4846
4947import Data.IORef
5048import Prelude hiding (last, length)
51−import Test.Tasty.Bench hiding (env)
49+import Gauge hiding (env)
5250
5351
5452scratchDir :: String
benchmark/lib/Streamly/Benchmark/Prelude.hs+1−1
@@ -82,7 +82,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream.Type as IsStream
8282import qualified Streamly.Internal.Data.Pipe as Pipe
8383import qualified Streamly.Internal.Data.Stream.Serial as Serial
8484
85−import Test.Tasty.Bench
85+import Gauge
8686import Streamly.Internal.Data.Time.Units
8787
8888-- Common polymorphic stream APIs used across multiple stream modules
benchmark/streamly-benchmarks.cabal+40−64
@@ -54,6 +54,11 @@ flag opt
5454 manual: True
5555 default: True
5656
57+flag use-gauge
58+ description: Use gauge instead of tasty-bench for benchmarking
59+ manual: True
60+ default: False
61+
5762flag use-streamly-core
5863 description: Benchmark only core package
5964 manual: True
@@ -140,10 +145,6 @@ common compile-options
140145 -Wno-missing-kind-signatures
141146 -Wno-operator-whitespace
142147
143− if impl(ghc >= 9.8)
144− ghc-options:
145− -Wno-missing-role-annotations
146−
147148 if flag(has-llvm)
148149 ghc-options: -fllvm
149150
@@ -151,17 +152,15 @@ common compile-options
151152 ghc-options: -Wmissed-specialisations
152153 -Wall-missed-specialisations
153154
155+ if flag(dev) || flag(debug)
156+ ghc-options: -fno-ignore-asserts
157+
154158common optimization-options
155159 if flag(opt)
156160 ghc-options: -O2
157161 -fdicts-strict
158162 -fspec-constr-recursive=16
159163 -fmax-worker-args=16
160−
161− -- For this to be effective it must come after the -O2 option
162− if flag(dev) || flag(debug)
163− ghc-options: -fno-ignore-asserts
164−
165164 if flag(fusion-plugin) && !impl(ghcjs) && !impl(ghc < 8.6)
166165 ghc-options: -fplugin Fusion.Plugin
167166 else
@@ -175,34 +174,43 @@ common bench-depends
175174 -- Core libraries shipped with ghc, the min and max
176175 -- constraints of these libraries should match with
177176 -- the GHC versions we support
178− base >= 4.9 && < 4.20
179− , deepseq >= 1.4.1 && < 1.6
177+ base >= 4.9 && < 4.19
178+ , deepseq >= 1.4.1 && < 1.5
180179 , mtl >= 2.2 && < 2.3.2
181180
182181 -- other libraries
183− , exceptions >= 0.8 && < 0.11
184182 , random >= 1.0 && < 2.0
185183 , transformers >= 0.4 && < 0.7
186− , containers >= 0.5 && < 0.8
184+ , containers >= 0.5 && < 0.7
187185 , hashable >= 1.3 && < 1.5
188186 , unordered-containers >= 0.2 && < 0.3
189187 , process >= 1.4 && < 1.7
190188 , directory >= 1.2.2 && < 1.4
191− , filepath >= 1.4.1 && < 1.6
192− , ghc-prim >= 0.4 && < 0.12
193− , tasty-bench >= 0.3 && < 0.4
194− , tasty >= 1.4.1 && < 1.6
195− , streamly-core
189+ , filepath >= 1.4.1 && < 1.5
190+ , ghc-prim >= 0.4 && < 0.11
196191
197− if !flag(use-streamly-core)
192+ if flag(use-streamly-core)
193+ build-depends: streamly-core
194+ else
198195 build-depends: streamly
196+ , streamly-core
197+
198+ if flag(use-gauge)
199+ build-depends: gauge >= 0.2.4 && < 0.3
200+ else
201+ build-depends: tasty-bench >= 0.3 && < 0.4
202+ , tasty >= 1.4.1 && < 1.5
203+ mixins: tasty-bench
204+ (Test.Tasty.Bench as Gauge
205+ , Test.Tasty.Bench as Gauge.Main
206+ )
199207
200208 if flag(fusion-plugin) && !impl(ghcjs) && !impl(ghc < 8.6)
201209 build-depends:
202210 fusion-plugin >= 0.2 && < 0.3
203211 if flag(inspection)
204− build-depends: template-haskell >= 2.14 && < 2.22
205− , inspection-testing >= 0.4 && < 0.6
212+ build-depends: template-haskell >= 2.14 && < 2.21
213+ , inspection-testing >= 0.4 && < 0.5
206214 -- Array uses a Storable constraint in dev build making several inspection
207215 -- tests fail
208216 if flag(dev) && flag(inspection)
@@ -352,30 +360,7 @@ benchmark Data.Unbox
352360 import: bench-options
353361 type: exitcode-stdio-1.0
354362 hs-source-dirs: .
355− cpp-options: -DUSE_UNBOX
356− main-is: Streamly/Benchmark/Data/Serialize.hs
357−
358−benchmark Data.Unbox.Derive.TH
359− import: bench-options
360− type: exitcode-stdio-1.0
361− hs-source-dirs: .
362− cpp-options: -DUSE_UNBOX
363− cpp-options: -DUSE_TH
364− main-is: Streamly/Benchmark/Data/Serialize.hs
365−
366−benchmark Data.Serialize
367− import: bench-options
368− type: exitcode-stdio-1.0
369− hs-source-dirs: .
370− cpp-options: -DUSE_TH
371− main-is: Streamly/Benchmark/Data/Serialize.hs
372− build-depends: QuickCheck, template-haskell
373− other-modules:
374− Streamly.Benchmark.Data.Serialize.TH
375− Streamly.Benchmark.Data.Serialize.RecCompatible
376− Streamly.Benchmark.Data.Serialize.RecNonCompatible
377− if flag(limit-build-mem)
378− ghc-options: +RTS -M1000M -RTS
363+ main-is: Streamly/Benchmark/Data/Unbox.hs
379364
380365benchmark Data.Unfold
381366 import: bench-options
@@ -427,7 +412,6 @@ benchmark Data.ParserK
427412 import: bench-options
428413 type: exitcode-stdio-1.0
429414 hs-source-dirs: Streamly/Benchmark/Data
430− cpp-options: -DBENCH_SINGULAR
431415 main-is: ParserK.hs
432416 if impl(ghcjs)
433417 buildable: False
@@ -435,25 +419,17 @@ benchmark Data.ParserK
435419 buildable: True
436420 build-depends: exceptions >= 0.8 && < 0.11
437421
438−benchmark Data.ParserK.Chunked
422+-- Note: to use this we have to set DISABLE_FUSION in ParserD.hs so that
423+-- the rewrite rules are disabled. We can also use the "no-fusion" build
424+-- flag but we need to keep in mind that it disables fusion for streams
425+-- as well.
426+benchmark Data.FromParserK
439427 import: bench-options
440428 type: exitcode-stdio-1.0
441429 hs-source-dirs: Streamly/Benchmark/Data
442− cpp-options: -DBENCH_CHUNKED
430+ cpp-options: -DFROM_PARSERK
443431 main-is: ParserK.hs
444− if impl(ghcjs)
445− buildable: False
446− else
447− buildable: True
448− build-depends: exceptions >= 0.8 && < 0.11
449−
450−benchmark Data.ParserK.Chunked.Generic
451− import: bench-options
452− type: exitcode-stdio-1.0
453− hs-source-dirs: Streamly/Benchmark/Data
454− cpp-options: -DBENCH_CHUNKED_GENERIC
455− main-is: ParserK.hs
456− if impl(ghcjs)
432+ if flag(use-streamly-core) || impl(ghcjs)
457433 buildable: False
458434 else
459435 buildable: True
@@ -463,7 +439,7 @@ benchmark Data.ParserK.Chunked.Generic
463439-- Raw Streams
464440-------------------------------------------------------------------------------
465441
466−benchmark Data.StreamD
442+benchmark Data.Stream.StreamD
467443 import: bench-options
468444 type: exitcode-stdio-1.0
469445 hs-source-dirs: Streamly/Benchmark/Data/Stream
@@ -471,7 +447,7 @@ benchmark Data.StreamD
471447 if impl(ghcjs)
472448 buildable: False
473449
474−benchmark Data.StreamK
450+benchmark Data.Stream.StreamK
475451 import: bench-options
476452 type: exitcode-stdio-1.0
477453 hs-source-dirs: Streamly/Benchmark/Data/Stream
@@ -487,7 +463,7 @@ benchmark Data.Stream.ToStreamK
487463 if !flag(dev) || impl(ghcjs)
488464 buildable: False
489465
490−benchmark Data.StreamK.Alt
466+benchmark Data.Stream.StreamK.Alt
491467 import: bench-options
492468 type: exitcode-stdio-1.0
493469 hs-source-dirs: Streamly/Benchmark/Data/Stream
configure+9−9
@@ -1,6 +1,6 @@
11#! /bin/sh
22# Guess values for system-dependent variables and create Makefiles.
3−# Generated by GNU Autoconf 2.71 for streamly 0.10.1.
3+# Generated by GNU Autoconf 2.71 for streamly 0.10.0.
44#
55# Report bugs to <streamly@composewell.com>.
66#
@@ -610,8 +610,8 @@ MAKEFLAGS=
610610# Identity of this package.
611611PACKAGE_NAME='streamly'
612612PACKAGE_TARNAME='streamly'
613−PACKAGE_VERSION='0.10.1'
614−PACKAGE_STRING='streamly 0.10.1'
613+PACKAGE_VERSION='0.10.0'
614+PACKAGE_STRING='streamly 0.10.0'
615615PACKAGE_BUGREPORT='streamly@composewell.com'
616616PACKAGE_URL='https://streamly.composewell.com'
617617
@@ -1224,7 +1224,7 @@ if test "$ac_init_help" = "long"; then
12241224 # Omit some internal or obsolete options to make the list less imposing.
12251225 # This message is too long to be a string in the A/UX 3.1 sh.
12261226 cat <<_ACEOF
1227−\`configure' configures streamly 0.10.1 to adapt to many kinds of systems.
1227+\`configure' configures streamly 0.10.0 to adapt to many kinds of systems.
12281228
12291229Usage: $0 [OPTION]... [VAR=VALUE]...
12301230
@@ -1286,7 +1286,7 @@ fi
12861286
12871287if test -n "$ac_init_help"; then
12881288 case $ac_init_help in
1289− short | recursive ) echo "Configuration of streamly 0.10.1:";;
1289+ short | recursive ) echo "Configuration of streamly 0.10.0:";;
12901290 esac
12911291 cat <<\_ACEOF
12921292
@@ -1372,7 +1372,7 @@ fi
13721372test -n "$ac_init_help" && exit $ac_status
13731373if $ac_init_version; then
13741374 cat <<\_ACEOF
1375−streamly configure 0.10.1
1375+streamly configure 0.10.0
13761376generated by GNU Autoconf 2.71
13771377
13781378Copyright (C) 2021 Free Software Foundation, Inc.
@@ -1500,7 +1500,7 @@ cat >config.log <<_ACEOF
15001500This file contains any messages produced by compilers while
15011501running configure, to aid debugging if configure makes a mistake.
15021502
1503−It was created by streamly $as_me 0.10.1, which was
1503+It was created by streamly $as_me 0.10.0, which was
15041504generated by GNU Autoconf 2.71. Invocation command line was
15051505
15061506 $ $0$ac_configure_args_raw
@@ -3800,7 +3800,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
38003800# report actual input values of CONFIG_FILES etc. instead of their
38013801# values after options handling.
38023802ac_log="
3803−This file was extended by streamly $as_me 0.10.1, which was
3803+This file was extended by streamly $as_me 0.10.0, which was
38043804generated by GNU Autoconf 2.71. Invocation command line was
38053805
38063806 CONFIG_FILES = $CONFIG_FILES
@@ -3856,7 +3856,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\
38563856cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
38573857ac_cs_config='$ac_cs_config_escaped'
38583858ac_cs_version="\\
3859−streamly config.status 0.10.1
3859+streamly config.status 0.10.0
38603860configured by $0, generated by GNU Autoconf 2.71,
38613861 with options \\"\$ac_cs_config\\"
38623862
configure.ac+1−1
@@ -3,7 +3,7 @@
33# See https://www.gnu.org/software/autoconf/manual/autoconf.html for help on
44# the macros used in this file.
55
6−AC_INIT([streamly], [0.10.1], [streamly@composewell.com], [streamly], [https://streamly.composewell.com])
6+AC_INIT([streamly], [0.10.0], [streamly@composewell.com], [streamly], [https://streamly.composewell.com])
77
88# To suppress "WARNING: unrecognized options: --with-compiler"
99AC_ARG_WITH([compiler], [GHC])
core/configure+9−9
@@ -1,6 +1,6 @@
11#! /bin/sh
22# Guess values for system-dependent variables and create Makefiles.
3−# Generated by GNU Autoconf 2.71 for streamly-core 0.2.2.
3+# Generated by GNU Autoconf 2.71 for streamly-core 0.2.0.
44#
55# Report bugs to <streamly@composewell.com>.
66#
@@ -610,8 +610,8 @@ MAKEFLAGS=
610610# Identity of this package.
611611PACKAGE_NAME='streamly-core'
612612PACKAGE_TARNAME='streamly-core'
613−PACKAGE_VERSION='0.2.2'
614−PACKAGE_STRING='streamly-core 0.2.2'
613+PACKAGE_VERSION='0.2.0'
614+PACKAGE_STRING='streamly-core 0.2.0'
615615PACKAGE_BUGREPORT='streamly@composewell.com'
616616PACKAGE_URL='https://streamly.composewell.com'
617617
@@ -1256,7 +1256,7 @@ if test "$ac_init_help" = "long"; then
12561256 # Omit some internal or obsolete options to make the list less imposing.
12571257 # This message is too long to be a string in the A/UX 3.1 sh.
12581258 cat <<_ACEOF
1259−\`configure' configures streamly-core 0.2.2 to adapt to many kinds of systems.
1259+\`configure' configures streamly-core 0.2.0 to adapt to many kinds of systems.
12601260
12611261Usage: $0 [OPTION]... [VAR=VALUE]...
12621262
@@ -1318,7 +1318,7 @@ fi
13181318
13191319if test -n "$ac_init_help"; then
13201320 case $ac_init_help in
1321− short | recursive ) echo "Configuration of streamly-core 0.2.2:";;
1321+ short | recursive ) echo "Configuration of streamly-core 0.2.0:";;
13221322 esac
13231323 cat <<\_ACEOF
13241324
@@ -1404,7 +1404,7 @@ fi
14041404test -n "$ac_init_help" && exit $ac_status
14051405if $ac_init_version; then
14061406 cat <<\_ACEOF
1407−streamly-core configure 0.2.2
1407+streamly-core configure 0.2.0
14081408generated by GNU Autoconf 2.71
14091409
14101410Copyright (C) 2021 Free Software Foundation, Inc.
@@ -1622,7 +1622,7 @@ cat >config.log <<_ACEOF
16221622This file contains any messages produced by compilers while
16231623running configure, to aid debugging if configure makes a mistake.
16241624
1625−It was created by streamly-core $as_me 0.2.2, which was
1625+It was created by streamly-core $as_me 0.2.0, which was
16261626generated by GNU Autoconf 2.71. Invocation command line was
16271627
16281628 $ $0$ac_configure_args_raw
@@ -3832,7 +3832,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
38323832# report actual input values of CONFIG_FILES etc. instead of their
38333833# values after options handling.
38343834ac_log="
3835−This file was extended by streamly-core $as_me 0.2.2, which was
3835+This file was extended by streamly-core $as_me 0.2.0, which was
38363836generated by GNU Autoconf 2.71. Invocation command line was
38373837
38383838 CONFIG_FILES = $CONFIG_FILES
@@ -3888,7 +3888,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\
38883888cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
38893889ac_cs_config='$ac_cs_config_escaped'
38903890ac_cs_version="\\
3891−streamly-core config.status 0.2.2
3891+streamly-core config.status 0.2.0
38923892configured by $0, generated by GNU Autoconf 2.71,
38933893 with options \\"\$ac_cs_config\\"
38943894
core/configure.ac+1−1
@@ -3,7 +3,7 @@
33# See https://www.gnu.org/software/autoconf/manual/autoconf.html for help on
44# the macros used in this file.
55
6−AC_INIT([streamly-core], [0.2.2], [streamly@composewell.com], [streamly-core], [https://streamly.composewell.com])
6+AC_INIT([streamly-core], [0.2.0], [streamly@composewell.com], [streamly-core], [https://streamly.composewell.com])
77
88# To suppress "WARNING: unrecognized options: --with-compiler"
99AC_ARG_WITH([compiler], [GHC])
core/docs/ApiChangelogs/0.1.0-0.2.0.txtdeleted−1471
@@ -1,1471 +0,0 @@
1−---------------------------------
2−API Annotations
3−---------------------------------
4−
5−[A] : Added
6−[R] : Removed
7−[C] : Changed
8−[O] : Old definition
9−[N] : New definition
10−[D] : Deprecated
11−
12−---------------------------------
13−API diff
14−---------------------------------
15−
16−[C] Streamly.Unicode.Parser
17− [A] double :: Monad m => Parser Char m Double
18−[C] Streamly.FileSystem.Handle
19− [A] readWith :: MonadIO m => Int -> Handle -> Stream m Word8
20− [A] readChunksWith :: MonadIO m => Int -> Handle -> Stream m (Array Word8)
21−[C] Streamly.FileSystem.Dir
22− [C] readEither
23− [O] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
24− [N] readEither :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
25− [C] read
26− [O] read :: MonadIO m => FilePath -> Stream m FilePath
27− [N] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
28−[C] Streamly.Data.Unfold
29− [A] class Enum a => Enumerable a
30− [A] second :: b -> Unfold m (a, b) c -> Unfold m a c
31− [A] first :: a -> Unfold m (a, b) c -> Unfold m b c
32− [A] enumerateFromTo :: (Enumerable a, Monad m) => Unfold m (a, a) a
33− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => Unfold m (a, a, a) a
34− [A] enumerateFromThen :: (Enumerable a, Monad m) => Unfold m (a, a) a
35− [A] enumerateFrom :: (Enumerable a, Monad m) => Unfold m a a
36−[C] Streamly.Data.StreamK
37− [C] parseChunks
38− [O] parseChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b)
39− [N] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
40− [C] parseBreakChunks
41− [O] parseBreakChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
42− [N] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
43− [A] parseBreak :: forall m a b. Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
44− [A] parse :: Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b)
45− [A] handle :: (MonadCatch m, Exception e) => (e -> m (StreamK m a)) -> StreamK m a -> StreamK m a
46− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> StreamK m a) -> StreamK m a
47−[C] Streamly.Data.Stream
48− [A] wordsBy :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
49− [A] splitOn :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
50− [C] handle
51− [O] handle :: (MonadCatch m, Exception e) => (e -> Stream m a) -> Stream m a -> Stream m a
52− [N] handle :: (MonadCatch m, Exception e) => (e -> m (Stream m a)) -> Stream m a -> Stream m a
53− [A] groupsOf :: Monad m => Int -> Fold m a b -> Stream m a -> Stream m b
54−[C] Streamly.Data.ParserK
55− [D] fromParser :: (MonadIO m, Unbox a) => Parser a m b -> ParserK (Array a) m b
56− [D] fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK (Array a) m b
57− [A] adaptCG :: Monad m => Parser a m b -> ParserK (Array a) m b
58− [A] adaptC :: (Monad m, Unbox a) => Parser a m b -> ParserK (Array a) m b
59− [A] adapt :: Monad m => Parser a m b -> ParserK a m b
60−[C] Streamly.Data.Parser
61− [A] groupByRollingEither :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
62− [A] groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
63−[A] Streamly.Data.MutByteArray
64− [A] class Unbox a
65− [A] class Serialize a
66− [A] SerializeConfig
67− [A] MutByteArray
68− [A] unpin :: MutByteArray -> IO MutByteArray
69− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
70− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
71− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
72− [A] pinnedNew :: Int -> IO MutByteArray
73− [A] pin :: MutByteArray -> IO MutByteArray
74− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
75− [A] new :: Int -> IO MutByteArray
76− [A] isPinned :: MutByteArray -> Bool
77− [A] inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
78− [A] inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
79− [A] inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig
80− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
81− [A] deriveUnbox :: Q [Dec] -> Q [Dec]
82− [A] deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]
83− [A] deriveSerialize :: Q [Dec] -> Q [Dec]
84− [A] addSizeTo :: Serialize a => Int -> a -> Int
85−[C] Streamly.Data.MutArray.Generic
86− [A] write :: MonadIO m => Fold m a (MutArray a)
87− [A] readRev :: MonadIO m => MutArray a -> Stream m a
88− [A] read :: MonadIO m => MutArray a -> Stream m a
89− [A] putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
90− [A] modifyIndexUnsafe :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
91− [A] length :: MutArray a -> Int
92− [A] getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
93− [C] getIndex
94− [O] getIndex :: MonadIO m => Int -> MutArray a -> m a
95− [N] getIndex :: MonadIO m => Int -> MutArray a -> m (Maybe a)
96− [A] fromListN :: MonadIO m => Int -> [a] -> m (MutArray a)
97− [A] fromList :: MonadIO m => [a] -> m (MutArray a)
98−[C] Streamly.Data.MutArray
99− [A] unpin :: MutArray a -> IO (MutArray a)
100− [A] readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
101− [A] read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
102− [A] putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
103− [D] pokeByteIndex :: Unbox a => Int -> MutByteArray -> a -> IO ()
104− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
105− [A] pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
106− [A] pin :: MutArray a -> IO (MutArray a)
107− [D] peekByteIndex :: Unbox a => Int -> MutByteArray -> IO a
108− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
109− [D] newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
110− [A] modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
111− [A] modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
112− [A] modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()
113− [A] isPinned :: MutArray a -> Bool
114− [A] getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
115− [C] getIndex
116− [O] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
117− [N] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
118−[C] Streamly.Data.Array.Generic
119− [A] toList :: Array a -> [a]
120− [A] getIndex :: Int -> Array a -> Maybe a
121−[C] Streamly.Data.Array
122− [A] unpin :: Array a -> IO (Array a)
123− [D] pokeByteIndex :: Unbox a => Int -> MutByteArray -> a -> IO ()
124− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
125− [A] pin :: Array a -> IO (Array a)
126− [D] peekByteIndex :: Unbox a => Int -> MutByteArray -> IO a
127− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
128− [A] isPinned :: Array a -> Bool
129−[C] Streamly.Console.Stdio
130−
131−---------------------------------
132−Internal API diff
133−---------------------------------
134−
135−[C] Streamly.Internal.Unicode.Stream
136− [C] writeCharUtf8'
137− [O] writeCharUtf8' :: Monad m => Fold m Word8 Char
138− [N] writeCharUtf8' :: Monad m => Parser Word8 m Char
139−[C] Streamly.Internal.Unicode.Parser
140− [A] number :: Monad m => Parser Char m (Integer, Int)
141− [A] mkDouble :: Integer -> Int -> Double
142− [A] doubleParser :: Monad m => Parser Char m (Int, Int)
143− [C] double
144− [O] double :: Parser Char m Double
145− [N] double :: Monad m => Parser Char m Double
146−[R] Streamly.Internal.Serialize.ToBytes
147−[R] Streamly.Internal.Serialize.FromBytes
148−[C] Streamly.Internal.FileSystem.File
149− [A] writeAppendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()
150− [A] writeAppendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()
151− [A] writeAppendArray :: FilePath -> Array a -> IO ()
152− [A] writeAppend :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()
153− [R] appendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()
154− [R] appendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()
155− [R] appendArray :: FilePath -> Array a -> IO ()
156− [R] append :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()
157−[C] Streamly.Internal.FileSystem.Dir
158− [C] reader
159− [O] reader :: MonadIO m => Unfold m FilePath FilePath
160− [N] reader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
161− [C] readFiles
162− [O] readFiles :: MonadIO m => FilePath -> Stream m FilePath
163− [N] readFiles :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
164− [C] readEitherPaths
165− [O] readEitherPaths :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
166− [N] readEitherPaths :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
167− [C] readEither
168− [O] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
169− [N] readEither :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
170− [C] readDirs
171− [O] readDirs :: MonadIO m => FilePath -> Stream m FilePath
172− [N] readDirs :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
173− [C] read
174− [O] read :: MonadIO m => FilePath -> Stream m FilePath
175− [N] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
176− [C] fileReader
177− [O] fileReader :: MonadIO m => Unfold m FilePath FilePath
178− [N] fileReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
179− [C] eitherReaderPaths
180− [O] eitherReaderPaths :: MonadIO m => Unfold m FilePath (Either FilePath FilePath)
181− [N] eitherReaderPaths :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Either FilePath FilePath)
182− [C] eitherReader
183− [O] eitherReader :: MonadIO m => Unfold m FilePath (Either FilePath FilePath)
184− [N] eitherReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Either FilePath FilePath)
185− [C] dirReader
186− [O] dirReader :: MonadIO m => Unfold m FilePath FilePath
187− [N] dirReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
188−[R] Streamly.Internal.Data.Unfold.Type
189−[R] Streamly.Internal.Data.Unfold.Enumeration
190−[C] Streamly.Internal.Data.Unfold
191− [C] Unfold
192− [A] Unfold :: (s -> m (Step s b)) -> (a -> m s) -> Unfold m a b
193− [A] takeWhileMWithInput :: Monad m => (a -> b -> m Bool) -> Unfold m a b -> Unfold m a b
194− [A] manyInterleave :: Monad m => Unfold m a b -> Unfold m c a -> Unfold m c b
195− [A] enumerateFromStepIntegral :: (Monad m, Integral a) => Unfold m (a, a) a
196− [A] crossApplySnd :: Unfold m a b -> Unfold m a c -> Unfold m a c
197− [A] crossApplyFst :: Unfold m a b -> Unfold m a c -> Unfold m a b
198− [A] concatMap :: Monad m => (b -> Unfold m a c) -> Unfold m a b -> Unfold m a c
199−[R] Streamly.Internal.Data.Unboxed
200−[C] Streamly.Internal.Data.Time.Units
201− [R] Streamly.Internal.Data.Unboxed.Unbox
202− [A] Streamly.Internal.Data.Unbox.Unbox
203− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.NanoSecond64
204− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.MilliSecond64
205− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.MicroSecond64
206−[R] Streamly.Internal.Data.Time.Clock.Type
207−[A] Streamly.Internal.Data.StreamK
208− [A] CrossStreamK
209− [A] (FixityR,6)
210− [A] (FixityR,6)
211− [A] (FixityR,6)
212− [A] (FixityR,5)
213− [A] (FixityR,5)
214− [A] (FixityR,6)
215− [A] (FixityR,5)
216− [A] StreamK
217− [A] MkStream :: (forall r. State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
218− [A] zipWithM :: Monad m => (a -> b -> m c) -> StreamK m a -> StreamK m b -> StreamK m c
219− [A] zipWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
220− [A] unfoldrMWith :: Monad m => (m a -> StreamK m a -> StreamK m a) -> (b -> m (Maybe (a, b))) -> b -> StreamK m a
221− [A] unfoldrM :: Monad m => (b -> m (Maybe (a, b))) -> b -> StreamK m a
222− [A] unfoldr :: (b -> Maybe (a, b)) -> b -> StreamK m a
223− [A] uncons :: Applicative m => StreamK m a -> m (Maybe (a, StreamK m a))
224− [A] unShare :: StreamK m a -> StreamK m a
225− [A] unCross :: CrossStreamK m a -> StreamK m a
226− [A] toStream :: Applicative m => StreamK m a -> Stream m a
227− [A] toList :: Monad m => StreamK m a -> m [a]
228− [A] the :: (Eq a, Monad m) => StreamK m a -> m (Maybe a)
229− [A] takeWhile :: (a -> Bool) -> StreamK m a -> StreamK m a
230− [A] take :: Int -> StreamK m a -> StreamK m a
231− [A] tail :: Applicative m => StreamK m a -> m (Maybe (StreamK m a))
232− [A] sortBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> StreamK m a
233− [A] sequence :: Monad m => StreamK m (m a) -> StreamK m a
234− [A] scanlx' :: (x -> a -> x) -> x -> (x -> b) -> StreamK m a -> StreamK m b
235− [A] scanl' :: (b -> a -> b) -> b -> StreamK m a -> StreamK m b
236− [A] reverse :: StreamK m a -> StreamK m a
237− [A] replicateMWith :: (m a -> StreamK m a -> StreamK m a) -> Int -> m a -> StreamK m a
238− [A] replicateM :: Monad m => Int -> m a -> StreamK m a
239− [A] replicate :: Int -> a -> StreamK m a
240− [A] repeatMWith :: (m a -> t m a -> t m a) -> m a -> t m a
241− [A] repeatM :: Monad m => m a -> StreamK m a
242− [A] repeat :: a -> StreamK m a
243− [A] parseDBreak :: Monad m => Parser a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
244− [A] parseD :: Monad m => Parser a m b -> StreamK m a -> m (Either ParseError b)
245− [A] parseChunksGeneric :: Monad m => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
246− [A] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
247− [A] parseBreakChunksGeneric :: forall m a b. Monad m => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
248− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
249− [A] parseBreak :: forall m a b. Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
250− [A] parse :: Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b)
251− [A] null :: Monad m => StreamK m a -> m Bool
252− [A] notElem :: (Monad m, Eq a) => a -> StreamK m a -> m Bool
253− [A] nilM :: Applicative m => m b -> StreamK m a
254− [A] nil :: StreamK m a
255− [A] mkStream :: (forall r. State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
256− [A] mkCross :: StreamK m a -> CrossStreamK m a
257− [A] minimumBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> m (Maybe a)
258− [A] minimum :: (Monad m, Ord a) => StreamK m a -> m (Maybe a)
259− [A] mfix :: Monad m => (m a -> StreamK m a) -> StreamK m a
260− [A] mergeMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
261− [A] mergeIterateWith :: (StreamK m a -> StreamK m a -> StreamK m a) -> (a -> StreamK m a) -> StreamK m a -> StreamK m a
262− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
263− [A] mergeBy :: (a -> a -> Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
264− [A] maximumBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> m (Maybe a)
265− [A] maximum :: (Monad m, Ord a) => StreamK m a -> m (Maybe a)
266− [A] mapMaybe :: (a -> Maybe b) -> StreamK m a -> StreamK m b
267− [A] mapM_ :: Monad m => (a -> m b) -> StreamK m a -> m ()
268− [A] mapMWith :: (m b -> StreamK m b -> StreamK m b) -> (a -> m b) -> StreamK m a -> StreamK m b
269− [A] mapMSerial :: Monad m => (a -> m b) -> StreamK m a -> StreamK m b
270− [A] mapM :: Monad m => (a -> m b) -> StreamK m a -> StreamK m b
271− [A] map :: (a -> b) -> StreamK m a -> StreamK m b
272− [A] lookup :: (Monad m, Eq a) => a -> StreamK m (a, b) -> m (Maybe b)
273− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => StreamK m a -> StreamK (t m) a
274− [A] last :: Monad m => StreamK m a -> m (Maybe a)
275− [A] iterateMWith :: Monad m => (m a -> StreamK m a -> StreamK m a) -> (a -> m a) -> m a -> StreamK m a
276− [A] iterateM :: Monad m => (a -> m a) -> m a -> StreamK m a
277− [A] iterate :: (a -> a) -> a -> StreamK m a
278− [A] intersperseM :: Monad m => m a -> StreamK m a -> StreamK m a
279− [A] intersperse :: Monad m => a -> StreamK m a -> StreamK m a
280− [A] interleaveMin :: StreamK m a -> StreamK m a -> StreamK m a
281− [A] interleaveFst :: StreamK m a -> StreamK m a -> StreamK m a
282− [A] interleave :: StreamK m a -> StreamK m a -> StreamK m a
283− [A] insertBy :: (a -> a -> Ordering) -> a -> StreamK m a -> StreamK m a
284− [A] init :: Applicative m => StreamK m a -> m (Maybe (StreamK m a))
285− [A] hoist :: (Monad m, Monad n) => (forall x. m x -> n x) -> StreamK m a -> StreamK n a
286− [A] head :: Monad m => StreamK m a -> m (Maybe a)
287− [A] handle :: (MonadCatch m, Exception e) => (e -> m (StreamK m a)) -> StreamK m a -> StreamK m a
288− [A] fromYieldK :: YieldK m a -> StreamK m a
289− [A] fromStream :: Monad m => Stream m a -> StreamK m a
290− [A] fromStopK :: StopK m -> StreamK m a
291− [A] fromPure :: a -> StreamK m a
292− [A] fromList :: [a] -> StreamK m a
293− [A] fromIndicesMWith :: (m a -> StreamK m a -> StreamK m a) -> (Int -> m a) -> StreamK m a
294− [A] fromIndicesM :: Monad m => (Int -> m a) -> StreamK m a
295− [A] fromIndices :: (Int -> a) -> StreamK m a
296− [A] fromFoldableM :: (Foldable f, Monad m) => f (m a) -> StreamK m a
297− [A] fromFoldable :: Foldable f => f a -> StreamK m a
298− [A] fromEffect :: Monad m => m a -> StreamK m a
299− [A] foldrT :: (Monad m, Monad (s m), MonadTrans s) => (a -> s m b -> s m b) -> s m b -> StreamK m a -> s m b
300− [A] foldrSShared :: (a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
301− [A] foldrSM :: Monad m => (m a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
302− [A] foldrS :: (a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
303− [A] foldrM :: (a -> m b -> m b) -> m b -> StreamK m a -> m b
304− [A] foldr1 :: Monad m => (a -> a -> a) -> StreamK m a -> m (Maybe a)
305− [A] foldr :: Monad m => (a -> b -> b) -> b -> StreamK m a -> m b
306− [A] foldlx' :: forall m a b x. Monad m => (x -> a -> x) -> x -> (x -> b) -> StreamK m a -> m b
307− [A] foldlT :: (Monad m, Monad (s m), MonadTrans s) => (s m b -> a -> s m b) -> s m b -> StreamK m a -> s m b
308− [A] foldlS :: (StreamK m b -> a -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
309− [A] foldlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> StreamK m a -> m b
310− [A] foldlM' :: Monad m => (b -> a -> m b) -> m b -> StreamK m a -> m b
311− [A] foldl' :: Monad m => (b -> a -> b) -> b -> StreamK m a -> m b
312− [A] foldStreamShared :: State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> StreamK m a -> m r
313− [A] foldStream :: State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> StreamK m a -> m r
314− [A] foldEither :: Monad m => Fold m a b -> StreamK m a -> m (Either (Fold m a b) (b, StreamK m a))
315− [A] foldConcat :: Monad m => Producer m a b -> Fold m b c -> StreamK m a -> m (c, StreamK m a)
316− [A] foldBreak :: Monad m => Fold m a b -> StreamK m a -> m (b, StreamK m a)
317− [A] fold :: Monad m => Fold m a b -> StreamK m a -> m b
318− [A] findM :: Monad m => (a -> m Bool) -> StreamK m a -> m (Maybe a)
319− [A] findIndices :: (a -> Bool) -> StreamK m a -> StreamK m Int
320− [A] find :: Monad m => (a -> Bool) -> StreamK m a -> m (Maybe a)
321− [A] filter :: (a -> Bool) -> StreamK m a -> StreamK m a
322− [A] evalStateT :: Monad m => m s -> StreamK (StateT s m) a -> StreamK m a
323− [A] elem :: (Monad m, Eq a) => a -> StreamK m a -> m Bool
324− [A] dropWhile :: (a -> Bool) -> StreamK m a -> StreamK m a
325− [A] drop :: Int -> StreamK m a -> StreamK m a
326− [A] drain :: Monad m => StreamK m a -> m ()
327− [A] deleteBy :: (a -> a -> Bool) -> a -> StreamK m a -> StreamK m a
328− [A] crossWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
329− [A] crossApplyWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> StreamK m (a -> b) -> StreamK m a -> StreamK m b
330− [A] crossApplySnd :: StreamK m a -> StreamK m b -> StreamK m b
331− [A] crossApplyFst :: StreamK m a -> StreamK m b -> StreamK m a
332− [A] crossApply :: StreamK m (a -> b) -> StreamK m a -> StreamK m b
333− [A] cross :: Monad m => StreamK m a -> StreamK m b -> StreamK m (a, b)
334− [A] consMBy :: Monad m => (StreamK m a -> StreamK m a -> StreamK m a) -> m a -> StreamK m a -> StreamK m a
335− [A] consM :: Monad m => m a -> StreamK m a -> StreamK m a
336− [A] consK :: YieldK m a -> StreamK m a -> StreamK m a
337− [A] cons :: a -> StreamK m a -> StreamK m a
338− [A] conjoin :: Monad m => StreamK m a -> StreamK m a -> StreamK m a
339− [A] concatMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
340− [A] concatMapEffect :: Monad m => (b -> StreamK m a) -> m b -> StreamK m a
341− [A] concatMap :: (a -> StreamK m b) -> StreamK m a -> StreamK m b
342− [A] concatIterateWith :: (StreamK m a -> StreamK m a -> StreamK m a) -> (a -> StreamK m a) -> StreamK m a -> StreamK m a
343− [A] concatIterateScanWith :: Monad m => (StreamK m a -> StreamK m a -> StreamK m a) -> (b -> a -> m (b, StreamK m a)) -> m b -> StreamK m a -> StreamK m a
344− [A] concatIterateLeftsWith :: b ~ Either a c => (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m b -> StreamK m b
345− [A] concatEffect :: Monad m => m (StreamK m a) -> StreamK m a
346− [A] buildSM :: Monad m => ((m a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a
347− [A] buildS :: ((a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a
348− [A] buildM :: Monad m => (forall r. (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
349− [A] build :: forall m a. (forall b. (a -> b -> b) -> b -> b) -> StreamK m a
350− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> StreamK m a) -> StreamK m a
351− [A] bindWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> StreamK m a -> (a -> StreamK m b) -> StreamK m b
352− [A] before :: Monad m => m b -> StreamK m a -> StreamK m a
353− [A] augmentSM :: Monad m => ((m a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a
354− [A] augmentS :: ((a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a
355− [A] append :: StreamK m a -> StreamK m a -> StreamK m a
356− [A] any :: Monad m => (a -> Bool) -> StreamK m a -> m Bool
357− [A] all :: Monad m => (a -> Bool) -> StreamK m a -> m Bool
358− [A] (.:) :: a -> StreamK m a -> StreamK m a
359− [A] (!!) :: Monad m => StreamK m a -> Int -> m (Maybe a)
360−[R] Streamly.Internal.Data.Stream.StreamK.Type
361−[R] Streamly.Internal.Data.Stream.StreamK.Transformer
362−[R] Streamly.Internal.Data.Stream.StreamK
363−[R] Streamly.Internal.Data.Stream.StreamD.Type
364−[R] Streamly.Internal.Data.Stream.StreamD.Transformer
365−[R] Streamly.Internal.Data.Stream.StreamD.Transform
366−[R] Streamly.Internal.Data.Stream.StreamD.Top
367−[R] Streamly.Internal.Data.Stream.StreamD.Step
368−[R] Streamly.Internal.Data.Stream.StreamD.Nesting
369−[R] Streamly.Internal.Data.Stream.StreamD.Lift
370−[R] Streamly.Internal.Data.Stream.StreamD.Generate
371−[R] Streamly.Internal.Data.Stream.StreamD.Exception
372−[R] Streamly.Internal.Data.Stream.StreamD.Eliminate
373−[R] Streamly.Internal.Data.Stream.StreamD.Container
374−[D] Streamly.Internal.Data.Stream.StreamD
375−[R] Streamly.Internal.Data.Stream.Common
376−[R] Streamly.Internal.Data.Stream.Chunked
377−[C] Streamly.Internal.Data.Stream
378− [A] class Enum a => Enumerable a
379− [A] Stream
380− [A] UnStream :: (State StreamK m a -> s -> m (Step s a)) -> s -> Stream m a
381− [A] Step
382− [A] Yield :: a -> s -> Step s a
383− [A] Stop :: Step s a
384− [A] Skip :: s -> Step s a
385− [A] InterleaveState
386− [A] InterleaveSecondOnly :: s2 -> InterleaveState s1 s2
387− [A] InterleaveSecond :: s1 -> s2 -> InterleaveState s1 s2
388− [A] InterleaveFirstOnly :: s1 -> InterleaveState s1 s2
389− [A] InterleaveFirst :: s1 -> s2 -> InterleaveState s1 s2
390− [A] FoldManyPost
391− [A] FoldManyPostYield :: b -> FoldManyPost s fs b a -> FoldManyPost s fs b a
392− [A] FoldManyPostStart :: s -> FoldManyPost s fs b a
393− [A] FoldManyPostLoop :: s -> fs -> FoldManyPost s fs b a
394− [A] FoldManyPostDone :: FoldManyPost s fs b a
395− [A] FoldMany
396− [A] FoldManyYield :: b -> FoldMany s fs b a -> FoldMany s fs b a
397− [A] FoldManyStart :: s -> FoldMany s fs b a
398− [A] FoldManyLoop :: s -> fs -> FoldMany s fs b a
399− [A] FoldManyFirst :: fs -> s -> FoldMany s fs b a
400− [A] FoldManyDone :: FoldMany s fs b a
401− [A] CrossStream
402− [A] ConcatUnfoldInterleaveState
403− [A] ConcatUnfoldInterleaveOuter :: o -> [i] -> ConcatUnfoldInterleaveState o i
404− [A] ConcatUnfoldInterleaveInnerR :: [i] -> [i] -> ConcatUnfoldInterleaveState o i
405− [A] ConcatUnfoldInterleaveInnerL :: [i] -> [i] -> ConcatUnfoldInterleaveState o i
406− [A] ConcatUnfoldInterleaveInner :: o -> [i] -> ConcatUnfoldInterleaveState o i
407− [A] ConcatMapUState
408− [A] ConcatMapUOuter :: o -> ConcatMapUState o i
409− [A] ConcatMapUInner :: o -> i -> ConcatMapUState o i
410− [A] AppendState
411− [A] AppendSecond :: s2 -> AppendState s1 s2
412− [A] AppendFirst :: s1 -> AppendState s1 s2
413− [A] pattern Stream :: (State StreamK m a -> s -> m (Step s a)) -> s -> Stream m a
414− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Stream m b -> Stream m c
415− [A] zipWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
416− [A] wordsBy :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
417− [A] with :: Monad m => (Stream m a -> Stream m (s, a)) -> (((s, a) -> b) -> Stream m (s, a) -> Stream m (s, a)) -> ((s, a) -> b) -> Stream m a -> Stream m a
418− [A] usingStateT :: Monad m => m s -> (Stream (StateT s m) a -> Stream (StateT s m) a) -> Stream m a -> Stream m a
419− [A] usingReaderT :: Monad m => m r -> (Stream (ReaderT r m) a -> Stream (ReaderT r m) a) -> Stream m a -> Stream m a
420− [A] uniqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a
421− [A] uniq :: (Eq a, Monad m) => Stream m a -> Stream m a
422− [A] unionWithStreamGenericBy :: MonadIO m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
423− [A] unionWithStreamAscBy :: (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
424− [A] unfoldrM :: Monad m => (s -> m (Maybe (a, s))) -> s -> Stream m a
425− [A] unfoldr :: Monad m => (s -> Maybe (a, s)) -> s -> Stream m a
426− [A] unfoldRoundRobin :: Monad m => Unfold m a b -> Stream m a -> Stream m b
427− [A] unfoldMany :: Monad m => Unfold m a b -> Stream m a -> Stream m b
428− [A] unfoldIterateDfs :: Monad m => Unfold m a a -> Stream m a -> Stream m a
429− [A] unfoldIterateBfsRev :: Monad m => Unfold m a a -> Stream m a -> Stream m a
430− [A] unfoldIterateBfs :: Monad m => Unfold m a a -> Stream m a -> Stream m a
431− [A] unfoldInterleave :: Monad m => Unfold m a b -> Stream m a -> Stream m b
432− [A] unfold :: Applicative m => Unfold m a b -> a -> Stream m b
433− [A] uncons :: Monad m => Stream m a -> m (Maybe (a, Stream m a))
434− [A] unCross :: CrossStream m a -> Stream m a
435− [A] transform :: Monad m => Pipe m a b -> Stream m a -> Stream m b
436− [A] trace_ :: Monad m => m b -> Stream m a -> Stream m a
437− [A] trace :: Monad m => (a -> m b) -> Stream m a -> Stream m a
438− [A] toStreamK :: Monad m => Stream m a -> StreamK m a
439− [A] toListRev :: Monad m => Stream m a -> m [a]
440− [A] toList :: Monad m => Stream m a -> m [a]
441− [A] timestamped :: MonadIO m => Stream m a -> Stream m (AbsTime, a)
442− [A] timestampWith :: MonadIO m => Double -> Stream m a -> Stream m (AbsTime, a)
443− [A] timesWith :: MonadIO m => Double -> Stream m (AbsTime, RelTime64)
444− [A] times :: MonadIO m => Stream m (AbsTime, RelTime64)
445− [A] timeout :: AbsTime -> t m ()
446− [A] timeIndexed :: MonadIO m => Stream m a -> Stream m (RelTime64, a)
447− [A] timeIndexWith :: MonadIO m => Double -> Stream m a -> Stream m (RelTime64, a)
448− [A] the :: (Eq a, Monad m) => Stream m a -> m (Maybe a)
449− [A] tapOffsetEvery :: Monad m => Int -> Int -> Fold m a b -> Stream m a -> Stream m a
450− [A] tap :: Monad m => Fold m a b -> Stream m a -> Stream m a
451− [A] takeWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
452− [A] takeWhileLast :: (a -> Bool) -> Stream m a -> Stream m a
453− [A] takeWhileAround :: (a -> Bool) -> Stream m a -> Stream m a
454− [A] takeWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
455− [A] takeEndByM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
456− [A] takeEndBy :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
457− [A] take :: Applicative m => Int -> Stream m a -> Stream m a
458− [A] tail :: Monad m => Stream m a -> m (Maybe (Stream m a))
459− [A] stripSuffixUnbox :: (MonadIO m, Eq a, Unbox a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
460− [A] stripSuffix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
461− [A] stripPrefix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
462− [A] strideFromThen :: Monad m => Int -> Int -> Stream m a -> Stream m a
463− [A] splitOnSuffixSeqAny :: [Array a] -> Fold m a b -> Stream m a -> Stream m b
464− [A] splitOnSuffixSeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) => Bool -> Array a -> Fold m a b -> Stream m a -> Stream m b
465− [A] splitOnSeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) => Array a -> Fold m a b -> Stream m a -> Stream m b
466− [A] splitOnPrefix :: (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
467− [A] splitOnAny :: [Array a] -> Fold m a b -> Stream m a -> Stream m b
468− [A] splitOn :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
469− [A] splitInnerBySuffix :: (Monad m, Eq (f a), Monoid (f a)) => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
470− [A] splitInnerBy :: Monad m => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
471− [A] slicesBy :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
472− [A] sliceOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
473− [A] sequence :: Monad m => Stream m (m a) -> Stream m a
474− [A] scanlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> Stream m b
475− [A] scanlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> Stream m b
476− [A] scanlMAfter' :: Monad m => (b -> a -> m b) -> m b -> (b -> m b) -> Stream m a -> Stream m b
477− [A] scanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
478− [A] scanlM :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
479− [A] scanl1M' :: Monad m => (a -> a -> m a) -> Stream m a -> Stream m a
480− [A] scanl1M :: Monad m => (a -> a -> m a) -> Stream m a -> Stream m a
481− [A] scanl1' :: Monad m => (a -> a -> a) -> Stream m a -> Stream m a
482− [A] scanl1 :: Monad m => (a -> a -> a) -> Stream m a -> Stream m a
483− [A] scanl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
484− [A] scanl :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
485− [A] scanMaybe :: Monad m => Fold m a (Maybe b) -> Stream m a -> Stream m b
486− [A] scanMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
487− [A] scan :: Monad m => Fold m a b -> Stream m a -> Stream m b
488− [A] runStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m (s, a)
489− [A] runReaderT :: Monad m => m s -> Stream (ReaderT s m) a -> Stream m a
490− [A] runInnerWithState :: Monad m => (forall b. s -> t m b -> m (b, s)) -> m s -> Stream (t m) a -> Stream m (s, a)
491− [A] runInnerWith :: Monad m => (forall b. t m b -> m b) -> Stream (t m) a -> Stream m a
492− [A] roundRobin :: Monad m => Stream m a -> Stream m a -> Stream m a
493− [A] rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Stream m a -> Stream m b
494− [A] rollingMap2 :: Monad m => (a -> a -> b) -> Stream m a -> Stream m b
495− [A] rollingMap :: Monad m => (Maybe a -> a -> b) -> Stream m a -> Stream m b
496− [A] reverseUnbox :: (MonadIO m, Unbox a) => Stream m a -> Stream m a
497− [A] reverse :: Monad m => Stream m a -> Stream m a
498− [A] replicateM :: Monad m => Int -> m a -> Stream m a
499− [A] replicate :: Monad m => Int -> a -> Stream m a
500− [A] repeated :: Stream m a -> Stream m a
501− [A] repeatM :: Monad m => m a -> Stream m a
502− [A] repeat :: Monad m => a -> Stream m a
503− [A] relTimesWith :: MonadIO m => Double -> Stream m RelTime64
504− [A] relTimes :: MonadIO m => Stream m RelTime64
505− [A] refoldMany :: Monad m => Refold m x a b -> m x -> Stream m a -> Stream m b
506− [A] refoldIterateM :: Monad m => Refold m b a b -> m b -> Stream m a -> Stream m b
507− [A] reduceIterateBfs :: Monad m => (a -> a -> m a) -> Stream m a -> m (Maybe a)
508− [A] reassembleBy :: Fold m a b -> (a -> a -> Int) -> Stream m a -> Stream m b
509− [A] prune :: (a -> Bool) -> Stream m a -> Stream m a
510− [A] prescanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
511− [A] prescanl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
512− [A] postscanlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> Stream m b
513− [A] postscanlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> Stream m b
514− [A] postscanlMAfter' :: Monad m => (b -> a -> m b) -> m b -> (b -> m b) -> Stream m a -> Stream m b
515− [A] postscanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
516− [A] postscanlM :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
517− [A] postscanl' :: Monad m => (a -> b -> a) -> a -> Stream m b -> Stream m a
518− [A] postscanl :: Monad m => (a -> b -> a) -> a -> Stream m b -> Stream m a
519− [A] postscan :: Monad m => Fold m a b -> Stream m a -> Stream m b
520− [A] parseSequence :: Stream m (Parser a m b) -> Stream m a -> Stream m b
521− [A] parseManyTill :: Parser a m b -> Parser a m x -> Stream m a -> Stream m b
522− [A] parseManyD :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
523− [A] parseMany :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
524− [A] parseIterateD :: Monad m => (b -> Parser a m b) -> b -> Stream m a -> Stream m (Either ParseError b)
525− [A] parseIterate :: Monad m => (b -> Parser a m b) -> b -> Stream m a -> Stream m (Either ParseError b)
526− [A] parseD :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
527− [A] parseBreakD :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b, Stream m a)
528− [A] parseBreak :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b, Stream m a)
529− [A] parse :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
530− [A] onException :: MonadCatch m => m b -> Stream m a -> Stream m a
531− [A] null :: Monad m => Stream m a -> m Bool
532− [A] nub :: (Monad m, Ord a) => Stream m a -> Stream m a
533− [A] notElem :: (Monad m, Eq a) => a -> Stream m a -> m Bool
534− [A] nilM :: Applicative m => m b -> Stream m a
535− [A] nil :: Applicative m => Stream m a
536− [A] morphInner :: Monad n => (forall x. m x -> n x) -> Stream m a -> Stream n a
537− [A] mkCross :: Stream m a -> CrossStream m a
538− [A] minimumBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> m (Maybe a)
539− [A] minimum :: (Monad m, Ord a) => Stream m a -> m (Maybe a)
540− [A] mergeMinBy :: (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
541− [A] mergeFstBy :: (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
542− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
543− [A] mergeBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
544− [A] maximumBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> m (Maybe a)
545− [A] maximum :: (Monad m, Ord a) => Stream m a -> m (Maybe a)
546− [A] mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Stream m a -> Stream m b
547− [A] mapMaybe :: Monad m => (a -> Maybe b) -> Stream m a -> Stream m b
548− [A] mapM_ :: Monad m => (a -> m b) -> Stream m a -> m ()
549− [A] mapM :: Monad m => (a -> m b) -> Stream m a -> Stream m b
550− [A] map :: Monad m => (a -> b) -> Stream m a -> Stream m b
551− [A] lookup :: (Monad m, Eq a) => a -> Stream m (a, b) -> m (Maybe b)
552− [A] liftInnerWith :: Monad (t m) => (forall b. m b -> t m b) -> Stream m a -> Stream (t m) a
553− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => Stream m a -> Stream (t m) a
554− [A] last :: Monad m => Stream m a -> m (Maybe a)
555− [A] joinOuterGeneric :: MonadIO m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (Maybe a, Maybe b)
556− [A] joinOuterAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (Maybe a, Maybe b)
557− [A] joinOuter :: (Ord k, MonadIO m) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, Maybe a, Maybe b)
558− [A] joinLeftGeneric :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (a, Maybe b)
559− [A] joinLeftAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (a, Maybe b)
560− [A] joinLeft :: (Ord k, Monad m) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, a, Maybe b)
561− [A] joinInnerGeneric :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (a, b)
562− [A] joinInnerAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (a, b)
563− [A] joinInner :: (Monad m, Ord k) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, a, b)
564− [A] iterateM :: Monad m => (a -> m a) -> m a -> Stream m a
565− [A] iterate :: Monad m => (a -> a) -> a -> Stream m a
566− [A] isSuffixOfUnbox :: (MonadIO m, Eq a, Unbox a) => Stream m a -> Stream m a -> m Bool
567− [A] isSuffixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
568− [A] isSubsequenceOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
569− [A] isPrefixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
570− [A] isInfixOf :: (MonadIO m, Eq a, Enum a, Storable a, Unbox a) => Stream m a -> Stream m a -> m Bool
571− [A] intersperseM_ :: Monad m => m b -> Stream m a -> Stream m a
572− [A] intersperseMWith :: Int -> m a -> Stream m a -> Stream m a
573− [A] intersperseMSuffix_ :: Monad m => m b -> Stream m a -> Stream m a
574− [A] intersperseMSuffixWith :: forall m a. Monad m => Int -> m a -> Stream m a -> Stream m a
575− [A] intersperseMSuffix :: forall m a. Monad m => m a -> Stream m a -> Stream m a
576− [A] intersperseMPrefix_ :: Monad m => m b -> Stream m a -> Stream m a
577− [A] intersperseM :: Monad m => m a -> Stream m a -> Stream m a
578− [A] intersperse :: Monad m => a -> Stream m a -> Stream m a
579− [A] intersectBySorted :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
580− [A] interposeSuffixM :: Monad m => m c -> Unfold m b c -> Stream m b -> Stream m c
581− [A] interposeSuffix :: Monad m => c -> Unfold m b c -> Stream m b -> Stream m c
582− [A] interposeM :: Monad m => m c -> Unfold m b c -> Stream m b -> Stream m c
583− [A] interpose :: Monad m => c -> Unfold m b c -> Stream m b -> Stream m c
584− [A] interleaveMin :: Monad m => Stream m a -> Stream m a -> Stream m a
585− [A] interleaveFstSuffix :: Monad m => Stream m a -> Stream m a -> Stream m a
586− [A] interleaveFst :: Monad m => Stream m a -> Stream m a -> Stream m a
587− [A] interleave :: Monad m => Stream m a -> Stream m a -> Stream m a
588− [A] intercalateSuffix :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
589− [A] intercalate :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
590− [A] insertBy :: Monad m => (a -> a -> Ordering) -> a -> Stream m a -> Stream m a
591− [A] indexedR :: Monad m => Int -> Stream m a -> Stream m (Int, a)
592− [A] indexed :: Monad m => Stream m a -> Stream m (Int, a)
593− [A] headElse :: Monad m => a -> Stream m a -> m a
594− [A] head :: Monad m => Stream m a -> m (Maybe a)
595− [A] handle :: (MonadCatch m, Exception e) => (e -> m (Stream m a)) -> Stream m a -> Stream m a
596− [A] groupsWhile :: Monad m => (a -> a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
597− [A] groupsRollingBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
598− [A] groupsOf :: Monad m => Int -> Fold m a b -> Stream m a -> Stream m b
599− [A] gintercalateSuffix :: Monad m => Unfold m a c -> Stream m a -> Unfold m b c -> Stream m b -> Stream m c
600− [A] gintercalate :: Monad m => Unfold m a c -> Stream m a -> Unfold m b c -> Stream m b -> Stream m c
601− [A] ghandle :: (MonadCatch m, Exception e) => (e -> Stream m a -> m (Stream m a)) -> Stream m a -> Stream m a
602− [A] generateM :: Monad m => Int -> (Int -> m a) -> Stream m a
603− [A] generate :: Monad m => Int -> (Int -> a) -> Stream m a
604− [A] generalizeInner :: Monad m => Stream Identity a -> Stream m a
605− [A] gbracket_ :: Monad m => m c -> (c -> m d) -> (c -> e -> Stream m b -> m (Stream m b)) -> (forall s. m s -> m (Either e s)) -> (c -> Stream m b) -> Stream m b
606− [A] gbracket :: MonadIO m => IO c -> (c -> IO d1) -> (c -> e -> Stream m b -> IO (Stream m b)) -> (c -> IO d2) -> (forall s. m s -> m (Either e s)) -> (c -> Stream m b) -> Stream m b
607− [A] fromStreamK :: Applicative m => StreamK m a -> Stream m a
608− [A] fromPure :: Applicative m => a -> Stream m a
609− [A] fromPtrN :: (Monad m, Storable a) => Int -> Ptr a -> Stream m a
610− [A] fromPtr :: forall m a. (Monad m, Storable a) => Ptr a -> Stream m a
611− [A] fromListM :: Monad m => [m a] -> Stream m a
612− [A] fromList :: Applicative m => [a] -> Stream m a
613− [A] fromIndicesM :: Monad m => (Int -> m a) -> Stream m a
614− [A] fromIndices :: Monad m => (Int -> a) -> Stream m a
615− [A] fromFoldableM :: (Monad m, Foldable f) => f (m a) -> Stream m a
616− [A] fromFoldable :: (Monad m, Foldable f) => f a -> Stream m a
617− [A] fromEffect :: Applicative m => m a -> Stream m a
618− [A] fromByteStr# :: Monad m => Addr# -> Stream m Word8
619− [A] foldrT :: (Monad m, Monad (t m), MonadTrans t) => (a -> t m b -> t m b) -> t m b -> Stream m a -> t m b
620− [A] foldrS :: Monad m => (a -> Stream m b -> Stream m b) -> Stream m b -> Stream m a -> Stream m b
621− [A] foldrMx :: Monad m => (a -> m x -> m x) -> m x -> (m x -> m b) -> Stream m a -> m b
622− [A] foldrM :: Monad m => (a -> m b -> m b) -> m b -> Stream m a -> m b
623− [A] foldr1 :: Monad m => (a -> a -> a) -> Stream m a -> m (Maybe a)
624− [A] foldr :: Monad m => (a -> b -> b) -> b -> Stream m a -> m b
625− [A] foldlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> m b
626− [A] foldlT :: (Monad m, Monad (s m), MonadTrans s) => (s m b -> a -> s m b) -> s m b -> Stream m a -> s m b
627− [A] foldlS :: Monad m => (Stream m b -> a -> Stream m b) -> Stream m b -> Stream m a -> Stream m b
628− [A] foldlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> m b
629− [A] foldlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> m b
630− [A] foldl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> m b
631− [A] foldSequence :: Stream m (Fold m a b) -> Stream m a -> Stream m b
632− [A] foldManyPost :: Monad m => Fold m a b -> Stream m a -> Stream m b
633− [A] foldMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
634− [A] foldIterateM :: Monad m => (b -> m (Fold m a b)) -> m b -> Stream m a -> Stream m b
635− [A] foldIterateBfs :: Fold m a (Either a a) -> Stream m a -> m (Maybe a)
636− [A] foldEither :: Monad m => Fold m a b -> Stream m a -> m (Either (Fold m a b) (b, Stream m a))
637− [A] foldBreak :: Monad m => Fold m a b -> Stream m a -> m (b, Stream m a)
638− [A] foldAddLazy :: Monad m => Fold m a b -> Stream m a -> Fold m a b
639− [A] foldAdd :: Monad m => Fold m a b -> Stream m a -> m (Fold m a b)
640− [A] fold :: Monad m => Fold m a b -> Stream m a -> m b
641− [A] findM :: Monad m => (a -> m Bool) -> Stream m a -> m (Maybe a)
642− [A] findIndices :: Monad m => (a -> Bool) -> Stream m a -> Stream m Int
643− [A] find :: Monad m => (a -> Bool) -> Stream m a -> m (Maybe a)
644− [A] finallyUnsafe :: MonadCatch m => m b -> Stream m a -> Stream m a
645− [A] finallyIO :: (MonadIO m, MonadCatch m) => IO b -> Stream m a -> Stream m a
646− [A] filterM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
647− [A] filterInStreamGenericBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
648− [A] filterInStreamAscBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
649− [A] filter :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
650− [A] evalStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m a
651− [A] eqBy :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> m Bool
652− [A] enumerateTo :: (Monad m, Bounded a, Enumerable a) => a -> Stream m a
653− [A] enumerateFromToSmall :: (Monad m, Enum a) => a -> a -> Stream m a
654− [A] enumerateFromToIntegral :: (Monad m, Integral a) => a -> a -> Stream m a
655− [A] enumerateFromToFractional :: (Monad m, Fractional a, Ord a) => a -> a -> Stream m a
656− [A] enumerateFromTo :: (Enumerable a, Monad m) => a -> a -> Stream m a
657− [A] enumerateFromThenToSmall :: (Monad m, Enum a) => a -> a -> a -> Stream m a
658− [A] enumerateFromThenToIntegral :: (Monad m, Integral a) => a -> a -> a -> Stream m a
659− [A] enumerateFromThenToFractional :: (Monad m, Fractional a, Ord a) => a -> a -> a -> Stream m a
660− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => a -> a -> a -> Stream m a
661− [A] enumerateFromThenSmallBounded :: (Monad m, Enumerable a, Bounded a) => a -> a -> Stream m a
662− [A] enumerateFromThenNum :: (Monad m, Num a) => a -> a -> Stream m a
663− [A] enumerateFromThenIntegral :: (Monad m, Integral a, Bounded a) => a -> a -> Stream m a
664− [A] enumerateFromThenFractional :: (Monad m, Fractional a) => a -> a -> Stream m a
665− [A] enumerateFromThen :: (Enumerable a, Monad m) => a -> a -> Stream m a
666− [A] enumerateFromStepNum :: (Monad m, Num a) => a -> a -> Stream m a
667− [A] enumerateFromStepIntegral :: (Integral a, Monad m) => a -> a -> Stream m a
668− [A] enumerateFromNum :: (Monad m, Num a) => a -> Stream m a
669− [A] enumerateFromIntegral :: (Monad m, Integral a, Bounded a) => a -> Stream m a
670− [A] enumerateFromFractional :: (Monad m, Fractional a) => a -> Stream m a
671− [A] enumerateFromBounded :: (Monad m, Enumerable a, Bounded a) => a -> Stream m a
672− [A] enumerateFrom :: (Enumerable a, Monad m) => a -> Stream m a
673− [A] enumerate :: (Monad m, Bounded a, Enumerable a) => Stream m a
674− [A] elemIndices :: (Monad m, Eq a) => a -> Stream m a -> Stream m Int
675− [A] elem :: (Monad m, Eq a) => a -> Stream m a -> m Bool
676− [A] durations :: Double -> t m RelTime64
677− [A] dropWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
678− [A] dropWhileLast :: (a -> Bool) -> Stream m a -> Stream m a
679− [A] dropWhileAround :: (a -> Bool) -> Stream m a -> Stream m a
680− [A] dropWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
681− [A] dropSuffix :: Stream m a -> Stream m a -> Stream m a
682− [A] dropPrefix :: Stream m a -> Stream m a -> Stream m a
683− [A] dropLast :: Int -> Stream m a -> Stream m a
684− [A] dropInfix :: Stream m a -> Stream m a -> Stream m a
685− [A] drop :: Monad m => Int -> Stream m a -> Stream m a
686− [A] drain :: Monad m => Stream m a -> m ()
687− [A] deleteInStreamGenericBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
688− [A] deleteInStreamAscBy :: (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
689− [A] deleteBy :: Monad m => (a -> a -> Bool) -> a -> Stream m a -> Stream m a
690− [A] delayPre :: MonadIO m => Double -> Stream m a -> Stream m a
691− [A] delayPost :: MonadIO m => Double -> Stream m a -> Stream m a
692− [A] delay :: MonadIO m => Double -> Stream m a -> Stream m a
693− [A] crossWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
694− [A] crossApplySnd :: Functor f => Stream f a -> Stream f b -> Stream f b
695− [A] crossApplyFst :: Functor f => Stream f a -> Stream f b -> Stream f a
696− [A] crossApply :: Functor f => Stream f (a -> b) -> Stream f a -> Stream f b
697− [A] cross :: Monad m => Stream m a -> Stream m b -> Stream m (a, b)
698− [A] consM :: Applicative m => m a -> Stream m a -> Stream m a
699− [A] cons :: Applicative m => a -> Stream m a -> Stream m a
700− [A] concatMapM :: Monad m => (a -> m (Stream m b)) -> Stream m a -> Stream m b
701− [A] concatMap :: Monad m => (a -> Stream m b) -> Stream m a -> Stream m b
702− [A] concatIterateScan :: Monad m => (b -> a -> m b) -> (b -> m (Maybe (b, Stream m a))) -> b -> Stream m a
703− [A] concatIterateDfs :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
704− [A] concatIterateBfsRev :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
705− [A] concatIterateBfs :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
706− [A] concatEffect :: Monad m => m (Stream m a) -> Stream m a
707− [A] concat :: Monad m => Stream m (Stream m a) -> Stream m a
708− [A] cmpBy :: Monad m => (a -> b -> Ordering) -> Stream m a -> Stream m b -> m Ordering
709− [A] catRights :: Monad m => Stream m (Either a b) -> Stream m b
710− [A] catMaybes :: Monad m => Stream m (Maybe a) -> Stream m a
711− [A] catLefts :: Monad m => Stream m (Either a b) -> Stream m a
712− [A] catEithers :: Monad m => Stream m (Either a a) -> Stream m a
713− [A] bracketUnsafe :: MonadCatch m => m b -> (b -> m c) -> (b -> Stream m a) -> Stream m a
714− [A] bracketIO3 :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> IO d) -> (b -> IO e) -> (b -> Stream m a) -> Stream m a
715− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> Stream m a) -> Stream m a
716− [A] before :: Monad m => m b -> Stream m a -> Stream m a
717− [A] append :: Monad m => Stream m a -> Stream m a -> Stream m a
718− [A] any :: Monad m => (a -> Bool) -> Stream m a -> m Bool
719− [A] all :: Monad m => (a -> Bool) -> Stream m a -> m Bool
720− [A] afterUnsafe :: Monad m => m b -> Stream m a -> Stream m a
721− [A] afterIO :: MonadIO m => IO b -> Stream m a -> Stream m a
722− [A] absTimesWith :: MonadIO m => Double -> Stream m AbsTime
723− [A] absTimes :: MonadIO m => Stream m AbsTime
724− [A] (!!) :: Monad m => Stream m a -> Int -> m (Maybe a)
725−[R] Streamly.Internal.Data.Ring.Unboxed
726−[A] Streamly.Internal.Data.Ring.Generic
727− [A] Ring
728− [A] [ringMax] :: Ring a -> !Int
729− [A] [ringHead] :: Ring a -> !Int
730− [A] [ringArr] :: Ring a -> MutArray a
731− [A] Ring :: MutArray a -> !Int -> !Int -> Ring a
732− [A] writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
733− [A] unsafeInsertRingWith :: Ring a -> a -> IO Int
734− [A] toStreamWith :: Int -> Ring a -> Stream m a
735− [A] toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
736− [A] seek :: MonadIO m => Int -> Ring a -> m (Ring a)
737− [A] createRing :: MonadIO m => Int -> m (Ring a)
738− [A] copyToMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
739−[C] Streamly.Internal.Data.Ring
740− [C] Ring
741− [A] [ringStart] :: Ring a -> {-# UNPACK #-} !ForeignPtr a
742− [R] [ringMax] :: Ring a -> !Int
743− [R] [ringHead] :: Ring a -> !Int
744− [A] [ringBound] :: Ring a -> {-# UNPACK #-} !Ptr a
745− [R] [ringArr] :: Ring a -> MutArray a
746− [C] Ring
747− [O] Ring :: MutArray a -> !Int -> !Int -> Ring a
748− [N] Ring :: {-# UNPACK #-} !ForeignPtr a -> {-# UNPACK #-} !Ptr a -> Ring a
749− [A] GHC.Show.Show
750− [A] instance (GHC.Show.Show a, GHC.Show.Show b, GHC.Show.Show c, GHC.Show.Show d) => GHC.Show.Show (Streamly.Internal.Data.Ring.Tuple4' a b c d)
751− [A] writeN :: Int -> Fold m a (Ring a)
752− [R] writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
753− [R] unsafeInsertRingWith :: Ring a -> a -> IO Int
754− [A] unsafeInsert :: Storable a => Ring a -> Ptr a -> a -> IO (Ptr a)
755− [A] unsafeFoldRingNM :: forall m a b. (MonadIO m, Storable a) => Int -> Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
756− [A] unsafeFoldRingM :: forall m a b. (MonadIO m, Storable a) => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
757− [A] unsafeFoldRingFullM :: forall m a b. (MonadIO m, Storable a) => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
758− [A] unsafeFoldRing :: forall a b. Storable a => Ptr a -> (b -> a -> b) -> b -> Ring a -> b
759− [A] unsafeEqArrayN :: Ring a -> Ptr a -> Array a -> Int -> Bool
760− [A] unsafeEqArray :: Ring a -> Ptr a -> Array a -> Bool
761− [R] toStreamWith :: Int -> Ring a -> Stream m a
762− [R] toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
763− [A] startOf :: Ring a -> Ptr a
764− [A] slidingWindowWith :: forall m a b. (MonadIO m, Storable a, Unbox a) => Int -> Fold m ((a, Maybe a), m (MutArray a)) b -> Fold m a b
765− [A] slidingWindow :: forall m a b. (MonadIO m, Storable a, Unbox a) => Int -> Fold m (a, Maybe a) b -> Fold m a b
766− [A] slide :: Ring a -> a -> m (Ring a)
767− [R] seek :: MonadIO m => Int -> Ring a -> m (Ring a)
768− [A] ringsOf :: Int -> Stream m a -> Stream m (MutArray a)
769− [A] readRev :: Unfold m (MutArray a) a
770− [A] read :: forall m a. (MonadIO m, Storable a) => Unfold m (Ring a, Ptr a, Int) a
771− [A] putIndex :: Ring a -> Int -> a -> m ()
772− [A] newRing :: Int -> m (Ring a)
773− [A] new :: forall a. Storable a => Int -> IO (Ring a, Ptr a)
774− [A] moveBy :: forall a. Storable a => Int -> Ring a -> Ptr a -> Ptr a
775− [A] modifyIndex :: Ring a -> Int -> (a -> (a, b)) -> m b
776− [A] length :: Ring a -> Int
777− [A] getIndexUnsafe :: Ring a -> Int -> m a
778− [A] getIndexRev :: Ring a -> Int -> m a
779− [A] getIndex :: Ring a -> Int -> m a
780− [A] fromArray :: MutArray a -> Ring a
781− [R] createRing :: MonadIO m => Int -> m (Ring a)
782− [A] castUnsafe :: Ring a -> Ring b
783− [A] cast :: forall a b. Storable b => Ring a -> Maybe (Ring b)
784− [A] bytesFree :: Ring a -> Int
785− [A] byteLength :: Ring a -> Int
786− [A] byteCapacity :: Ring a -> Int
787− [A] asBytes :: Ring a -> Ring Word8
788− [A] advance :: forall a. Storable a => Ring a -> Ptr a -> Ptr a
789−[R] Streamly.Internal.Data.Producer.Type
790−[R] Streamly.Internal.Data.Producer.Source
791−[C] Streamly.Internal.Data.Producer
792− [A] Source
793− [A] unread :: [b] -> Source a b -> Source a b
794− [A] translate :: Functor m => (a -> c) -> (c -> a) -> Producer m c b -> Producer m a b
795− [A] source :: Maybe a -> Source a b
796− [A] producer :: Monad m => Producer m a b -> Producer m (Source a b) b
797− [A] parseManyD :: Monad m => Parser a m b -> Producer m (Source x a) a -> Producer m (Source x a) (Either ParseError b)
798− [A] parseMany :: Monad m => Parser a m b -> Producer m (Source x a) a -> Producer m (Source x a) (Either ParseError b)
799− [A] parse :: Monad m => Parser a m b -> Producer m (Source s a) a -> Source s a -> m (Either ParseError b, Source s a)
800− [A] lmap :: (a -> a) -> Producer m a b -> Producer m a b
801− [A] isEmpty :: Source a b -> Bool
802−[R] Streamly.Internal.Data.Pipe.Type
803−[C] Streamly.Internal.Data.Pipe
804− [A] Step
805− [A] Yield :: a -> s -> Step s a
806− [A] Continue :: s -> Step s a
807− [A] PipeState
808− [A] Produce :: s2 -> PipeState s1 s2
809− [A] Consume :: s1 -> PipeState s1 s2
810− [C] Pipe
811− [A] Pipe :: (s1 -> a -> m (Step (PipeState s1 s2) b)) -> (s2 -> m (Step (PipeState s1 s2) b)) -> s1 -> Pipe m a b
812−[A] Streamly.Internal.Data.ParserK
813− [A] Step
814− [A] Partial :: !Int -> (Input a -> m (Step a m r)) -> Step a m r
815− [A] Error :: !Int -> String -> Step a m r
816− [A] Done :: !Int -> r -> Step a m r
817− [A] Continue :: !Int -> (Input a -> m (Step a m r)) -> Step a m r
818− [A] ParseResult
819− [A] Success :: !Int -> !b -> ParseResult b
820− [A] Failure :: !Int -> !String -> ParseResult b
821− [A] Input
822− [A] None :: Input a
823− [A] Chunk :: a -> Input a
824− [A] ParserK
825− [A] [runParser] :: ParserK a m b -> forall r. (ParseResult b -> Int -> Input a -> m (Step a m r)) -> Int -> Int -> Input a -> m (Step a m r)
826− [A] MkParser :: (forall r. (ParseResult b -> Int -> Input a -> m (Step a m r)) -> Int -> Int -> Input a -> m (Step a m r)) -> ParserK a m b
827− [A] fromPure :: b -> ParserK a m b
828− [A] fromEffect :: Monad m => m b -> ParserK a m b
829− [A] die :: String -> ParserK a m b
830− [A] adaptCG :: Monad m => Parser a m b -> ParserK (Array a) m b
831− [A] adaptC :: (Monad m, Unbox a) => Parser a m b -> ParserK (Array a) m b
832− [A] adapt :: Monad m => Parser a m b -> ParserK a m b
833−[R] Streamly.Internal.Data.Parser.ParserK.Type
834−[R] Streamly.Internal.Data.Parser.ParserD.Type
835−[R] Streamly.Internal.Data.Parser.ParserD.Tee
836−[R] Streamly.Internal.Data.Parser.ParserD
837−[C] Streamly.Internal.Data.Parser
838− [A] Step
839− [A] Partial :: !Int -> !s -> Step s b
840− [A] Error :: !String -> Step s b
841− [A] Done :: !Int -> !b -> Step s b
842− [A] Continue :: !Int -> !s -> Step s b
843− [A] Parser
844− [A] Parser :: (s -> a -> m (Step s b)) -> m (Initial s b) -> (s -> m (Step s b)) -> Parser a m b
845− [A] Initial
846− [A] IPartial :: !s -> Initial s b
847− [A] IError :: !String -> Initial s b
848− [A] IDone :: !b -> Initial s b
849− [A] GHC.Show.Show
850− [A] instance (GHC.Show.Show a, GHC.Show.Show b) => GHC.Show.Show (Streamly.Internal.Data.Parser.Tuple'Fused a b)
851− [A] ParseError
852− [A] ParseError :: String -> ParseError
853− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Fold m c x -> Parser b m x
854− [A] zip :: Monad m => Stream m a -> Fold m (a, b) x -> Parser b m x
855− [A] wordWithQuotes :: (Monad m, Eq a) => Bool -> (a -> a -> Maybe a) -> a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
856− [A] wordProcessQuotes :: (Monad m, Eq a) => a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
857− [A] wordKeepQuotes :: (Monad m, Eq a) => a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
858− [A] wordFramedBy :: Monad m => (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
859− [A] wordBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
860− [A] toFold :: Monad m => Parser a m b -> Fold m a b
861− [A] takeWhileP :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
862− [A] takeWhile1 :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
863− [A] takeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
864− [A] takeStartBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
865− [A] takeStartBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
866− [A] takeP :: Monad m => Int -> Parser a m b -> Parser a m b
867− [A] takeGE :: Monad m => Int -> Fold m a b -> Parser a m b
868− [A] takeFramedBy_ :: Monad m => (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
869− [A] takeFramedByGeneric :: Monad m => Maybe (a -> Bool) -> Maybe (a -> Bool) -> Maybe (a -> Bool) -> Fold m a b -> Parser a m b
870− [A] takeFramedByEsc_ :: Monad m => (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
871− [A] takeEndBy_ :: (a -> Bool) -> Parser a m b -> Parser a m b
872− [A] takeEndByEsc :: Monad m => (a -> Bool) -> (a -> Bool) -> Parser a m b -> Parser a m b
873− [A] takeEndBy :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
874− [A] takeEitherSepBy :: (a -> Bool) -> Fold m (Either a b) c -> Parser a m c
875− [A] takeEQ :: Monad m => Int -> Fold m a b -> Parser a m b
876− [A] takeBetween :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
877− [A] subsequenceBy :: (a -> a -> Bool) -> Stream m a -> Parser a m ()
878− [A] streamEqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Parser a m ()
879− [A] split_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
880− [A] splitWith :: Monad m => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
881− [A] splitSome :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
882− [A] splitManyPost :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
883− [A] splitMany :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
884− [A] spanByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
885− [A] spanBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
886− [A] span :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
887− [A] some :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
888− [A] sequence :: Monad m => Stream m (Parser a m b) -> Fold m b c -> Parser a m c
889− [A] sepByAll :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
890− [A] sepBy1 :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
891− [A] sepBy :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
892− [A] satisfy :: Monad m => (a -> Bool) -> Parser a m a
893− [A] sampleFromthen :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
894− [A] roundRobin :: t (Parser a m b) -> Fold m b c -> Parser a m c
895− [A] rmapM :: Monad m => (b -> m c) -> Parser a m b -> Parser a m c
896− [A] retryMaxTotal :: Int -> Parser a m b -> Fold m b c -> Parser a m c
897− [A] retryMaxSuccessive :: Int -> Parser a m b -> Fold m b c -> Parser a m c
898− [A] retry :: Parser a m b -> Parser a m b
899− [A] postscan :: Fold m a b -> Parser b m c -> Parser a m c
900− [A] peek :: Monad m => Parser a m a
901− [A] oneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
902− [A] oneNotEq :: (Monad m, Eq a) => a -> Parser a m a
903− [A] oneEq :: (Monad m, Eq a) => a -> Parser a m a
904− [A] one :: Monad m => Parser a m a
905− [A] noneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
906− [A] noErrorUnsafeSplit_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
907− [A] noErrorUnsafeSplitWith :: Monad m => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
908− [A] noErrorUnsafeConcatMap :: Monad m => (b -> Parser a m c) -> Parser a m b -> Parser a m c
909− [A] maybe :: Monad m => (a -> Maybe b) -> Parser a m b
910− [A] manyTillP :: Parser a m b -> Parser a m x -> Parser b m c -> Parser a m c
911− [A] manyTill :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
912− [A] manyThen :: Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
913− [A] manyP :: Parser a m b -> Parser b m c -> Parser a m c
914− [A] many :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
915− [A] makeIndexFilter :: (Fold m (s, a) b -> Parser a m b) -> (((s, a) -> Bool) -> Fold m (s, a) b -> Fold m (s, a) b) -> ((s, a) -> Bool) -> Fold m a b -> Parser a m b
916− [A] lookAhead :: Monad m => Parser a m b -> Parser a m b
917− [A] lmapM :: Monad m => (a -> m b) -> Parser b m r -> Parser a m r
918− [A] lmap :: (a -> b) -> Parser b m r -> Parser a m r
919− [A] listEqBy :: Monad m => (a -> a -> Bool) -> [a] -> Parser a m [a]
920− [A] listEq :: (Monad m, Eq a) => [a] -> Parser a m [a]
921− [A] indexed :: forall m a b. Monad m => Fold m (Int, a) b -> Parser a m b
922− [A] groupByRollingEither :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
923− [A] groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
924− [A] groupBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
925− [A] fromPure :: Monad m => b -> Parser a m b
926− [A] fromFoldMaybe :: Monad m => String -> Fold m a (Maybe b) -> Parser a m b
927− [A] fromFold :: Monad m => Fold m a b -> Parser a m b
928− [A] fromEffect :: Monad m => m b -> Parser a m b
929− [A] filter :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
930− [A] extractStep :: Monad m => (s -> m (Step s1 b)) -> Step s b -> m (Step s1 b)
931− [A] eof :: Monad m => Parser a m ()
932− [A] either :: Monad m => (a -> Either String b) -> Parser a m b
933− [A] dropWhile :: Monad m => (a -> Bool) -> Parser a m ()
934− [A] dieM :: Monad m => m String -> Parser a m b
935− [A] die :: Monad m => String -> Parser a m b
936− [A] deintercalateAll :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
937− [A] deintercalate1 :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
938− [A] deintercalate :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
939− [A] countBetween :: Int -> Int -> Parser a m b -> Fold m b c -> Parser a m c
940− [A] count :: Int -> Parser a m b -> Fold m b c -> Parser a m c
941− [A] concatMap :: Monad m => (b -> Parser a m c) -> Parser a m b -> Parser a m c
942− [A] blockWithQuotes :: (Monad m, Eq a) => (a -> Bool) -> (a -> Bool) -> a -> a -> Fold m a b -> Parser a m b
943− [A] bimapOverrideCount :: Int -> (s -> s1) -> (b -> b1) -> Step s b -> Step s1 b1
944− [A] alt :: Monad m => Parser x m a -> Parser x m a -> Parser x m a
945−[A] Streamly.Internal.Data.MutByteArray
946− [A] class Unbox a
947− [A] class SizeOfRep (f :: Type -> Type)
948− [A] class Serialize a
949− [A] TypeOfType
950− [A] UnitType :: Name -> TypeOfType
951− [A] TheType :: SimpleDataCon -> TypeOfType
952− [A] MultiType :: [SimpleDataCon] -> TypeOfType
953− [A] SimpleDataCon
954− [A] SimpleDataCon :: Name -> [Field] -> SimpleDataCon
955− [A] SerializeConfig
956− [A] [cfgRecordSyntaxWithHeader] :: SerializeConfig -> Bool
957− [A] [cfgInlineSize] :: SerializeConfig -> Maybe Inline
958− [A] [cfgInlineSerialize] :: SerializeConfig -> Maybe Inline
959− [A] [cfgInlineDeserialize] :: SerializeConfig -> Maybe Inline
960− [A] [cfgConstructorTagAsString] :: SerializeConfig -> Bool
961− [A] SerializeConfig :: Maybe Inline -> Maybe Inline -> Maybe Inline -> Bool -> Bool -> SerializeConfig
962− [A] PinnedState
963− [A] Unpinned :: PinnedState
964− [A] Pinned :: PinnedState
965− [A] MutByteArray
966− [A] MutByteArray :: MutableByteArray# RealWorld -> MutByteArray
967− [A] DataType
968− [A] [dtTvs] :: DataType -> [Name]
969− [A] [dtName] :: DataType -> Name
970− [A] [dtCxt] :: DataType -> Cxt
971− [A] [dtCons] :: DataType -> [DataCon]
972− [A] DataType :: Name -> [Name] -> Cxt -> [DataCon] -> DataType
973− [A] DataCon
974− [A] [dcTvs] :: DataCon -> [Name]
975− [A] [dcName] :: DataCon -> Name
976− [A] [dcFields] :: DataCon -> [(Maybe Name, Type)]
977− [A] [dcCxt] :: DataCon -> Cxt
978− [A] DataCon :: Name -> [Name] -> Cxt -> [(Maybe Name, Type)] -> DataCon
979− [A] BoundedPtr
980− [A] BoundedPtr :: MutByteArray -> Int -> Int -> BoundedPtr
981− [A] Streamly.Internal.Data.Serialize.Type.Serialize
982− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize Streamly.Internal.Data.MutByteArray.LiftedInteger
983− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize GHC.Num.Integer.Integer
984− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize a => Streamly.Internal.Data.Serialize.Type.Serialize (GHC.Maybe.Maybe a)
985− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize (Data.Proxy.Proxy a)
986− [A] instance (Streamly.Internal.Data.Serialize.Type.Serialize a, Streamly.Internal.Data.Serialize.Type.Serialize b) => Streamly.Internal.Data.Serialize.Type.Serialize (Data.Either.Either a b)
987− [A] Peeker
988− [A] Peeker :: Builder BoundedPtr IO a -> Peeker a
989− [A] type Field = (Maybe Name, Type)
990− [A] xorCmp :: [Word8] -> Name -> Name -> Q Exp
991− [A] wListToString :: [Word8] -> String
992− [A] w8_int :: Word8 -> Int
993− [A] w32_int :: Word32 -> Int
994− [A] unpin :: MutByteArray -> IO MutByteArray
995− [A] typeOfType :: Type -> [DataCon] -> TypeOfType
996− [A] skipByte :: Peeker ()
997− [A] sizeOfRep :: SizeOfRep f => f x -> Int
998− [A] sizeOfMutableByteArray :: MutByteArray -> IO Int
999− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
1000− [A] simplifyDataCon :: DataCon -> SimpleDataCon
1001− [A] serializeW8List :: Name -> Name -> [Word8] -> Q Exp
1002− [A] serializeConfig :: SerializeConfig
1003− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
1004− [A] runPeeker :: Peeker a -> BoundedPtr -> IO a
1005− [A] reifyDataType :: Name -> Q DataType
1006− [A] readUnsafe :: Unbox a => Peeker a
1007− [A] read :: Unbox a => Peeker a
1008− [A] putSliceUnsafe :: MonadIO m => MutByteArray -> Int -> MutByteArray -> Int -> Int -> m ()
1009− [A] pokeRep :: PokeRep f => f a -> BoundedPtr -> IO BoundedPtr
1010− [A] pokeBoundedPtrUnsafe :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr
1011− [A] pokeBoundedPtr :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr
1012− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
1013− [A] pinnedNewAlignedBytes :: Int -> Int -> IO MutByteArray
1014− [A] pinnedNew :: Int -> IO MutByteArray
1015− [A] pinnedCloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
1016− [A] pin :: MutByteArray -> IO MutByteArray
1017− [A] peekRep :: PeekRep f => Peeker (f x)
1018− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
1019− [A] openConstructor :: Name -> Int -> Q Pat
1020− [A] nil :: MutByteArray
1021− [A] newBytesAs :: PinnedState -> Int -> IO MutByteArray
1022− [A] new :: Int -> IO MutByteArray
1023− [A] mkSerializeExprFields :: Name -> [Field] -> Q Exp
1024− [A] mkRecSizeOfExpr :: SimpleDataCon -> Q Exp
1025− [A] mkRecSerializeExpr :: Name -> SimpleDataCon -> Q Exp
1026− [A] mkRecDeserializeExpr :: Name -> Name -> Name -> SimpleDataCon -> Q Exp
1027− [A] mkFieldName :: Int -> Name
1028− [A] mkDeserializeKeysDec :: Name -> Name -> SimpleDataCon -> Q [Dec]
1029− [A] mkDeserializeExprOne :: Name -> SimpleDataCon -> Q Exp
1030− [A] matchConstructor :: Name -> Int -> Q Exp -> Q Match
1031− [A] makeN :: Int -> Name
1032− [A] makeI :: Int -> Name
1033− [A] makeA :: Int -> Name
1034− [A] litProxy :: Unbox a => Proxy a -> Q Exp
1035− [A] litIntegral :: Integral a => a -> Q Exp
1036− [A] isUnitType :: [DataCon] -> Bool
1037− [A] isRecordSyntax :: SimpleDataCon -> Bool
1038− [A] isPinned :: MutByteArray -> Bool
1039− [A] int_w8 :: Int -> Word8
1040− [A] int_w32 :: Int -> Word32
1041− [A] inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
1042− [A] inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
1043− [A] inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig
1044− [A] getMutableByteArray# :: MutByteArray -> MutableByteArray# RealWorld
1045− [A] genericSizeOf :: forall a. SizeOfRep (Rep a) => Proxy a -> Int
1046− [A] genericPokeByteIndex :: (Generic a, PokeRep (Rep a)) => MutByteArray -> Int -> a -> IO ()
1047− [A] genericPeekByteIndex :: (Generic a, PeekRep (Rep a)) => MutByteArray -> Int -> IO a
1048− [A] errorUnsupported :: String -> a
1049− [A] errorUnimplemented :: a
1050− [A] encodeRecordFields :: Bool -> SerializeConfig -> SerializeConfig
1051− [A] encodeConstrNames :: Bool -> SerializeConfig -> SerializeConfig
1052− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
1053− [A] deriveUnbox :: Q [Dec] -> Q [Dec]
1054− [A] deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]
1055− [A] deriveSerialize :: Q [Dec] -> Q [Dec]
1056− [A] conUpdateFuncDec :: Name -> [Field] -> Q [Dec]
1057− [A] cloneSliceUnsafeAs :: MonadIO m => PinnedState -> Int -> Int -> MutByteArray -> m MutByteArray
1058− [A] cloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
1059− [A] c2w :: Char -> Word8
1060− [A] asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
1061− [A] addSizeTo :: Serialize a => Int -> a -> Int
1062− [A] _x :: Name
1063− [A] _val :: Name
1064− [A] _tag :: Name
1065− [A] _initialOffset :: Name
1066− [A] _endOffset :: Name
1067− [A] _arr :: Name
1068− [A] _acc :: Name
1069−[A] Streamly.Internal.Data.MutArray.Stream
1070− [A] SpliceState
1071− [A] SpliceYielding :: arr -> SpliceState s arr -> SpliceState s arr
1072− [A] SpliceInitial :: s -> SpliceState s arr
1073− [A] SpliceFinish :: SpliceState s arr
1074− [A] SpliceBuffering :: s -> arr -> SpliceState s arr
1075− [A] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
1076− [A] splitOn :: (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> Stream m (MutArray a)
1077− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1078− [A] packArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1079− [A] lpackArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
1080− [A] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
1081− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1082− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1083− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (Either ParseError (MutArray a))
1084− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1085− [A] compactEQ :: Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1086− [A] compact :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1087− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1088−[A] Streamly.Internal.Data.MutArray.Generic
1089− [A] MutArray
1090− [A] [arrTrueLen] :: MutArray a -> {-# UNPACK #-} !Int
1091− [A] [arrStart] :: MutArray a -> {-# UNPACK #-} !Int
1092− [A] [arrLen] :: MutArray a -> {-# UNPACK #-} !Int
1093− [A] [arrContents#] :: MutArray a -> MutableArray# RealWorld a
1094− [A] MutArray :: MutableArray# RealWorld a -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> MutArray a
1095− [A] writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
1096− [A] writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
1097− [A] writeN :: MonadIO m => Int -> Fold m a (MutArray a)
1098− [A] write :: MonadIO m => Fold m a (MutArray a)
1099− [A] uninit :: MonadIO m => MutArray a -> Int -> m (MutArray a)
1100− [A] toStreamK :: MonadIO m => MutArray a -> StreamK m a
1101− [A] toList :: MonadIO m => MutArray a -> m [a]
1102− [A] strip :: MonadIO m => (a -> Bool) -> MutArray a -> m (MutArray a)
1103− [A] snocWith :: MonadIO m => (Int -> Int) -> MutArray a -> a -> m (MutArray a)
1104− [A] snocUnsafe :: MonadIO m => MutArray a -> a -> m (MutArray a)
1105− [A] snoc :: MonadIO m => MutArray a -> a -> m (MutArray a)
1106− [A] realloc :: MonadIO m => Int -> MutArray a -> m (MutArray a)
1107− [A] reader :: MonadIO m => Unfold m (MutArray a) a
1108− [A] readRev :: MonadIO m => MutArray a -> Stream m a
1109− [A] read :: MonadIO m => MutArray a -> Stream m a
1110− [A] putSliceUnsafe :: MonadIO m => MutArray a -> Int -> MutArray a -> Int -> Int -> m ()
1111− [A] putIndices :: MonadIO m => MutArray a -> Fold m (Int, a) ()
1112− [A] putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
1113− [A] putIndex :: MonadIO m => Int -> MutArray a -> a -> m ()
1114− [A] producerWith :: Monad m => (forall b. IO b -> m b) -> Producer m (MutArray a) a
1115− [A] producer :: MonadIO m => Producer m (MutArray a) a
1116− [A] nil :: MonadIO m => m (MutArray a)
1117− [A] new :: MonadIO m => Int -> m (MutArray a)
1118− [A] modifyIndexUnsafe :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
1119− [A] modifyIndex :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
1120− [A] length :: MutArray a -> Int
1121− [A] getSliceUnsafe :: Int -> Int -> MutArray a -> MutArray a
1122− [A] getSlice :: Int -> Int -> MutArray a -> MutArray a
1123− [A] getIndexUnsafeWith :: MonadIO m => MutableArray# RealWorld a -> Int -> m a
1124− [A] getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
1125− [A] getIndex :: MonadIO m => Int -> MutArray a -> m (Maybe a)
1126− [A] fromStreamN :: MonadIO m => Int -> Stream m a -> m (MutArray a)
1127− [A] fromStream :: MonadIO m => Stream m a -> m (MutArray a)
1128− [A] fromPureStream :: MonadIO m => Stream Identity a -> m (MutArray a)
1129− [A] fromListN :: MonadIO m => Int -> [a] -> m (MutArray a)
1130− [A] fromList :: MonadIO m => [a] -> m (MutArray a)
1131− [A] eq :: (MonadIO m, Eq a) => MutArray a -> MutArray a -> m Bool
1132− [A] cmp :: (MonadIO m, Ord a) => MutArray a -> MutArray a -> m Ordering
1133− [A] clone :: MonadIO m => MutArray a -> m (MutArray a)
1134− [A] chunksOf :: forall m a. MonadIO m => Int -> Stream m a -> Stream m (MutArray a)
1135−[A] Streamly.Internal.Data.MutArray
1136− [A] MutByteArray
1137− [A] MutArray
1138− [A] [arrStart] :: MutArray a -> {-# UNPACK #-} !Int
1139− [A] [arrEnd] :: MutArray a -> {-# UNPACK #-} !Int
1140− [A] [arrContents] :: MutArray a -> {-# UNPACK #-} !MutByteArray
1141− [A] [arrBound] :: MutArray a -> {-# UNPACK #-} !Int
1142− [A] MutArray :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> MutArray a
1143− [A] IORef
1144− [A] ArrayUnsafe
1145− [A] ArrayUnsafe :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> ArrayUnsafe a
1146− [A] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1147− [A] writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1148− [A] writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
1149− [A] writeNWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
1150− [A] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1151− [A] writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1152− [A] writeIORef :: Unbox a => IORef a -> a -> IO ()
1153− [A] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
1154− [A] writeAppendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
1155− [A] writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
1156− [A] writeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
1157− [A] writeAppend :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
1158− [A] write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
1159− [A] unsafeSwapIndices :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> MutArray a -> m ()
1160− [A] unpin :: MutArray a -> IO (MutArray a)
1161− [A] toStreamKWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> StreamK m a
1162− [A] toStreamKRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> StreamK m a
1163− [A] toStreamKRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
1164− [A] toStreamK :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
1165− [A] toStreamDWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
1166− [A] toStreamDRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
1167− [A] toList :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m [a]
1168− [A] swapIndices :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> MutArray a -> m ()
1169− [A] strip :: forall a m. (Unbox a, MonadIO m) => (a -> Bool) -> MutArray a -> m (MutArray a)
1170− [A] splitOn :: (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> Stream m (MutArray a)
1171− [A] splitAt :: forall a. Unbox a => Int -> MutArray a -> (MutArray a, MutArray a)
1172− [A] spliceWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int -> Int) -> MutArray a -> MutArray a -> m (MutArray a)
1173− [A] spliceUnsafe :: MonadIO m => MutArray a -> MutArray a -> m (MutArray a)
1174− [A] spliceExp :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
1175− [A] spliceCopy :: forall m a. MonadIO m => MutArray a -> MutArray a -> m (MutArray a)
1176− [A] splice :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
1177− [A] snocWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> MutArray a -> a -> m (MutArray a)
1178− [A] snocUnsafe :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1179− [A] snocMay :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (Maybe (MutArray a))
1180− [A] snocLinear :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1181− [A] snoc :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1182− [A] shuffleBy :: (a -> a -> m Bool) -> MutArray a -> MutArray a -> m ()
1183− [A] roundUpToPower2 :: Int -> Int
1184− [A] rightSize :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m (MutArray a)
1185− [A] reverse :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m ()
1186− [A] resizeExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1187− [A] resize :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1188− [A] realloc :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1189− [A] readerRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Unfold m (MutArray a) a
1190− [A] readerRev :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
1191− [A] reader :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
1192− [A] readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
1193− [A] readIORef :: Unbox a => IORef a -> IO a
1194− [A] read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
1195− [A] putIndices :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Fold m (Int, a) ()
1196− [A] putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
1197− [A] putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
1198− [A] producerWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Producer m (MutArray a) a
1199− [A] producer :: forall m a. (MonadIO m, Unbox a) => Producer m (MutArray a) a
1200− [A] pollIntIORef :: (MonadIO m, Unbox a) => IORef a -> Stream m a
1201− [A] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1202− [A] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (MutArray a)
1203− [A] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1204− [A] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
1205− [A] pinnedNewBytes :: MonadIO m => Int -> m (MutArray a)
1206− [A] pinnedNewAligned :: (MonadIO m, Unbox a) => Int -> Int -> m (MutArray a)
1207− [A] pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
1208− [A] pinnedFromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1209− [A] pinnedFromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1210− [A] pinnedClone :: MonadIO m => MutArray a -> m (MutArray a)
1211− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1212− [A] pin :: MutArray a -> IO (MutArray a)
1213− [A] permute :: MutArray a -> m Bool
1214− [A] partitionBy :: forall m a. (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> m (MutArray a, MutArray a)
1215− [A] nil :: MutArray a
1216− [A] newIORef :: forall a. Unbox a => a -> IO (IORef a)
1217− [A] newArrayWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int -> m MutByteArray) -> Int -> Int -> m (MutArray a)
1218− [A] new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
1219− [A] modifyIndices :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (Int -> a -> a) -> Fold m Int ()
1220− [A] modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
1221− [A] modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
1222− [A] modifyIORef' :: Unbox a => IORef a -> (a -> a) -> IO ()
1223− [A] modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()
1224− [A] mergeBy :: Int -> (MutArray a -> MutArray a -> m ()) -> MutArray a -> m ()
1225− [A] memcpy :: Ptr Word8 -> Ptr Word8 -> Int -> IO ()
1226− [A] memcmp :: Ptr Word8 -> Ptr Word8 -> Int -> IO Bool
1227− [A] length :: forall a. Unbox a => MutArray a -> Int
1228− [A] isPinned :: MutArray a -> Bool
1229− [A] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
1230− [A] getSliceUnsafe :: forall a. Unbox a => Int -> Int -> MutArray a -> MutArray a
1231− [A] getSlice :: forall a. Unbox a => Int -> Int -> MutArray a -> MutArray a
1232− [A] getIndicesD :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
1233− [A] getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1234− [A] getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1235− [A] getIndexRev :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1236− [A] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
1237− [A] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
1238− [A] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
1239− [A] fromStreamD :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
1240− [A] fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
1241− [A] fromPureStream :: (MonadIO m, Unbox a) => Stream Identity a -> m (MutArray a)
1242− [A] fromListRevN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1243− [A] fromListRev :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1244− [A] fromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1245− [A] fromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1246− [A] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
1247− [A] foldr :: (MonadIO m, Unbox a) => (a -> b -> b) -> b -> MutArray a -> m b
1248− [A] foldl' :: (MonadIO m, Unbox a) => (b -> a -> b) -> b -> MutArray a -> m b
1249− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1250− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1251− [A] divideBy :: Int -> (MutArray a -> m (MutArray a, MutArray a)) -> MutArray a -> m ()
1252− [A] cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
1253− [A] clone :: MonadIO m => MutArray a -> m (MutArray a)
1254− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1255− [A] castUnsafe :: MutArray a -> MutArray b
1256− [A] cast :: forall a b. Unbox b => MutArray a -> Maybe (MutArray b)
1257− [A] c_memchr :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)
1258− [A] bytesFree :: MutArray a -> Int
1259− [A] byteLength :: MutArray a -> Int
1260− [A] byteCapacity :: MutArray a -> Int
1261− [A] bubble :: (MonadIO m, Unbox a) => (a -> a -> Ordering) -> MutArray a -> m ()
1262− [A] breakOn :: MonadIO m => Word8 -> MutArray Word8 -> m (MutArray Word8, Maybe (MutArray Word8))
1263− [A] blockSize :: Int
1264− [A] asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
1265− [A] asBytes :: MutArray a -> MutArray Word8
1266− [A] arrayChunkBytes :: Int
1267− [A] allocBytesToElemCount :: Unbox a => a -> Int -> Int
1268−[C] Streamly.Internal.Data.IsMap
1269− [A] mapTraverseWithKey :: (IsMap f, Applicative t) => (Key f -> a -> t b) -> f a -> t (f b)
1270−[R] Streamly.Internal.Data.IORef.Unboxed
1271−[R] Streamly.Internal.Data.Fold.Window
1272−[R] Streamly.Internal.Data.Fold.Type
1273−[R] Streamly.Internal.Data.Fold.Tee
1274−[R] Streamly.Internal.Data.Fold.Step
1275−[R] Streamly.Internal.Data.Fold.Container
1276−[D] Streamly.Internal.Data.Fold.Chunked
1277−[C] Streamly.Internal.Data.Fold
1278− [A] ManyState
1279− [C] Fold
1280− [C] Fold
1281− [O] Fold :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> Fold m a b
1282− [N] Fold :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> (s -> m b) -> Fold m a b
1283− [A] windowSumInt :: forall m a. (Monad m, Integral a) => Fold m (a, Maybe a) a
1284− [A] windowSum :: forall m a. (Monad m, Num a) => Fold m (a, Maybe a) a
1285− [A] windowRollingMapM :: Monad m => (Maybe a -> a -> m (Maybe b)) -> Fold m (a, Maybe a) (Maybe b)
1286− [A] windowRollingMap :: Monad m => (Maybe a -> a -> Maybe b) -> Fold m (a, Maybe a) (Maybe b)
1287− [A] windowRange :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe (a, a))
1288− [A] windowPowerSumFrac :: (Monad m, Floating a) => a -> Fold m (a, Maybe a) a
1289− [A] windowPowerSum :: (Monad m, Num a) => Int -> Fold m (a, Maybe a) a
1290− [A] windowMinimum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
1291− [A] windowMean :: forall m a. (Monad m, Fractional a) => Fold m (a, Maybe a) a
1292− [A] windowMaximum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
1293− [A] windowLmap :: (c -> a) -> Fold m (a, Maybe a) b -> Fold m (c, Maybe c) b
1294− [A] windowLength :: (Monad m, Num b) => Fold m (a, Maybe a) b
1295− [A] toSet :: (Monad m, Ord a) => Fold m a (Set a)
1296− [A] toMapIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
1297− [A] toMap :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
1298− [A] toIntSet :: Monad m => Fold m Int IntSet
1299− [A] toContainerIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (f b)
1300− [A] toContainer :: (Monad m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (f b)
1301− [A] nubInt :: Monad m => Fold m Int (Maybe Int)
1302− [A] nub :: (Monad m, Ord a) => Fold m a (Maybe a)
1303− [A] mapMStep :: Applicative m => (a -> m b) -> Step s a -> m (Step s b)
1304− [A] kvToMap :: (Monad m, Ord k) => Fold m a b -> Fold m (k, a) (Map k b)
1305− [A] frequency :: (Monad m, Ord a) => Fold m a (Map a Int)
1306− [A] demuxToMapIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
1307− [A] demuxToMap :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
1308− [A] demuxToContainerIO :: (MonadIO m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
1309− [A] demuxToContainer :: (Monad m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
1310− [A] demuxKvToMap :: (Monad m, Ord k) => (k -> m (Fold m a b)) -> Fold m (k, a) (Map k b)
1311− [A] demuxKvToContainer :: (Monad m, IsMap f, Traversable f) => (Key f -> m (Fold m a b)) -> Fold m (Key f, a) (f b)
1312− [A] demuxIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
1313− [A] demuxGenericIO :: (MonadIO m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (m (f b), Maybe (Key f, b))
1314− [A] demuxGeneric :: (Monad m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (m (f b), Maybe (Key f, b))
1315− [A] demux :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
1316− [A] cumulative :: Fold m (a, Maybe a) b -> Fold m a b
1317− [A] countDistinctInt :: Monad m => Fold m Int Int
1318− [A] countDistinct :: (Monad m, Ord a) => Fold m a Int
1319− [A] classifyIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
1320− [A] classifyGenericIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
1321− [A] classifyGeneric :: (Monad m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
1322− [A] classify :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
1323− [A] chainStepM :: Applicative m => (s1 -> m s2) -> (a -> m (Step s2 b)) -> Step s1 a -> m (Step s2 b)
1324−[C] Streamly.Internal.Data.Builder
1325− [C] Builder
1326− [C] Builder
1327− [O] Builder :: (s -> m (s, a)) -> Builder s m a
1328− [N] Builder :: (s -> m (a, s)) -> Builder s m a
1329−[A] Streamly.Internal.Data.Binary.Stream
1330− [A] class ToBytes a
1331− [A] word8 :: Applicative m => Word8 -> Stream m Word8
1332− [A] word64le :: Monad m => Word64 -> Stream m Word8
1333− [A] word64host :: Monad m => Word64 -> Stream m Word8
1334− [A] word64be :: Monad m => Word64 -> Stream m Word8
1335− [A] word32le :: Monad m => Word32 -> Stream m Word8
1336− [A] word32be :: Monad m => Word32 -> Stream m Word8
1337− [A] word16le :: Monad m => Word16 -> Stream m Word8
1338− [A] word16be :: Monad m => Word16 -> Stream m Word8
1339− [A] unit :: Applicative m => Stream m Word8
1340− [A] toBytes :: ToBytes a => a -> Stream m Word8
1341− [A] ordering :: Applicative m => Ordering -> Stream m Word8
1342− [A] int8 :: Applicative m => Int8 -> Stream m Word8
1343− [A] int64le :: Monad m => Int64 -> Stream m Word8
1344− [A] int64be :: Monad m => Int64 -> Stream m Word8
1345− [A] int32le :: Monad m => Int32 -> Stream m Word8
1346− [A] int32be :: Monad m => Int32 -> Stream m Word8
1347− [A] int16le :: Monad m => Int16 -> Stream m Word8
1348− [A] int16be :: Monad m => Int16 -> Stream m Word8
1349− [A] float32le :: Monad m => Float -> Stream m Word8
1350− [A] float32be :: Monad m => Float -> Stream m Word8
1351− [A] double64le :: Monad m => Double -> Stream m Word8
1352− [A] double64be :: Monad m => Double -> Stream m Word8
1353− [A] charUtf8 :: Monad m => Char -> Stream m Word8
1354− [A] charLatin1 :: Applicative m => Char -> Stream m Word8
1355− [A] bool :: Applicative m => Bool -> Stream m Word8
1356−[A] Streamly.Internal.Data.Binary.Parser
1357− [A] class FromBytes a
1358− [A] word8 :: Monad m => Parser Word8 m Word8
1359− [A] word64le :: Monad m => Parser Word8 m Word64
1360− [A] word64host :: MonadIO m => Parser Word8 m Word64
1361− [A] word64be :: Monad m => Parser Word8 m Word64
1362− [A] word32le :: Monad m => Parser Word8 m Word32
1363− [A] word32be :: Monad m => Parser Word8 m Word32
1364− [A] word16le :: Monad m => Parser Word8 m Word16
1365− [A] word16be :: Monad m => Parser Word8 m Word16
1366− [A] unit :: Monad m => Parser Word8 m ()
1367− [A] ordering :: Monad m => Parser Word8 m Ordering
1368− [A] int8 :: Monad m => Parser Word8 m Int8
1369− [A] int64le :: Monad m => Parser Word8 m Int64
1370− [A] int64be :: Monad m => Parser Word8 m Int64
1371− [A] int32le :: Monad m => Parser Word8 m Int32
1372− [A] int32be :: Monad m => Parser Word8 m Int32
1373− [A] int16le :: Monad m => Parser Word8 m Int16
1374− [A] int16be :: Monad m => Parser Word8 m Int16
1375− [A] fromBytes :: FromBytes a => Parser Word8 m a
1376− [A] float32le :: MonadIO m => Parser Word8 m Float
1377− [A] float32be :: MonadIO m => Parser Word8 m Float
1378− [A] eqWord8 :: Monad m => Word8 -> Parser Word8 m Word8
1379− [A] double64le :: MonadIO m => Parser Word8 m Double
1380− [A] double64be :: MonadIO m => Parser Word8 m Double
1381− [A] charLatin1 :: Monad m => Parser Word8 m Char
1382− [A] bool :: Monad m => Parser Word8 m Bool
1383−[R] Streamly.Internal.Data.Array.Type
1384−[A] Streamly.Internal.Data.Array.Stream
1385− [A] unlines :: forall m a. (MonadIO m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1386− [A] toArray :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
1387− [A] splitOnSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
1388− [A] splitOn :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
1389− [A] runArrayParserDBreak :: forall m a b. (MonadIO m, Unbox a) => Parser (Array a) m b -> Stream m (Array a) -> m (Either ParseError b, Stream m (Array a))
1390− [A] runArrayFoldMany :: (Monad m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> StreamK m (Either ParseError b)
1391− [A] runArrayFoldBreak :: (MonadIO m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1392− [A] runArrayFold :: (MonadIO m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> m (Either ParseError b)
1393− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1394− [A] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
1395− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1396− [A] parseBreak :: (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1397− [A] lpackArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
1398− [A] interposeSuffix :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1399− [A] interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1400− [A] intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a
1401− [A] foldBreakD :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
1402− [A] foldBreak :: (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
1403− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1404− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1405− [A] concatRev :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
1406− [A] concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
1407− [A] compact :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
1408− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1409− [A] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
1410−[R] Streamly.Internal.Data.Array.Mut.Type
1411−[R] Streamly.Internal.Data.Array.Mut.Stream
1412−[R] Streamly.Internal.Data.Array.Mut
1413−[R] Streamly.Internal.Data.Array.Generic.Mut.Type
1414−[C] Streamly.Internal.Data.Array.Generic
1415− [A] getIndex :: Int -> Array a -> Maybe a
1416− [A] fromPureStream :: Stream Identity a -> Array a
1417− [A] fromByteStr# :: Addr# -> Array Word8
1418− [A] chunksOf :: forall m a. MonadIO m => Int -> Stream m a -> Stream m (Array a)
1419−[C] Streamly.Internal.Data.Array
1420− [A] ArrayUnsafe
1421− [A] ArrayUnsafe :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> ArrayUnsafe a
1422− [C] Array
1423− [A] [arrStart] :: Array a -> {-# UNPACK #-} !Int
1424− [A] [arrEnd] :: Array a -> {-# UNPACK #-} !Int
1425− [A] [arrContents] :: Array a -> {-# UNPACK #-} !MutByteArray
1426− [A] Array :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> Array a
1427− [A] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1428− [A] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1429− [R] writeNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
1430− [A] unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b
1431− [A] unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a
1432− [D] unsafeIndex :: forall a. Unbox a => Int -> Array a -> a
1433− [A] unsafeFreezeWithShrink :: Unbox a => MutArray a -> Array a
1434− [A] unpin :: Array a -> IO (Array a)
1435− [A] toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a
1436− [A] toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a
1437− [A] toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
1438− [A] toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
1439− [A] splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)
1440− [A] splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
1441− [A] serialize :: Serialize a => a -> Array Word8
1442− [A] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1443− [A] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
1444− [A] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1445− [A] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
1446− [A] pinnedSerialize :: Serialize a => a -> Array Word8
1447− [A] pinnedFromListN :: Unbox a => Int -> [a] -> Array a
1448− [A] pinnedFromList :: Unbox a => [a] -> Array a
1449− [A] pinnedClone :: MonadIO m => Array a -> m (Array a)
1450− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1451− [A] pin :: Array a -> IO (Array a)
1452− [A] nil :: Array a
1453− [A] isPinned :: Array a -> Bool
1454− [A] getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a
1455− [A] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
1456− [A] fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)
1457− [A] fromPureStream :: Unbox a => Stream Identity a -> Array a
1458− [A] fromListRevN :: Unbox a => Int -> [a] -> Array a
1459− [A] fromListRev :: Unbox a => [a] -> Array a
1460− [A] fromByteStr# :: Addr# -> Array Word8
1461− [A] foldr :: Unbox a => (a -> b -> b) -> b -> Array a -> b
1462− [A] foldl' :: forall a b. Unbox a => (b -> a -> b) -> b -> Array a -> b
1463− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1464− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1465− [A] encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8
1466− [A] deserialize :: Serialize a => Array Word8 -> a
1467− [A] clone :: MonadIO m => Array a -> m (Array a)
1468− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1469− [A] byteLength :: Array a -> Int
1470− [A] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
1471− [A] breakOn :: MonadIO m => Word8 -> Array Word8 -> m (Array Word8, Maybe (Array Word8))
core/docs/ApiChangelogs/0.2.0-0.2.2.txtdeleted−306
@@ -1,306 +0,0 @@
1−---------------------------------
2−API Annotations
3−---------------------------------
4−
5−[A] : Added
6−[R] : Removed
7−[C] : Changed
8−[O] : Old definition
9−[N] : New definition
10−[D] : Deprecated
11−
12−---------------------------------
13−API diff
14−---------------------------------
15−
16−[C] Streamly.Data.Stream
17− [A] (FixityR,5)
18− [A] (FixityR,5)
19−[C] Streamly.Data.MutArray.Generic
20− [A] emptyOf :: MonadIO m => Int -> m (MutArray a)
21− [A] createOf :: MonadIO m => Int -> Fold m a (MutArray a)
22− [A] create :: MonadIO m => Fold m a (MutArray a)
23−[C] Streamly.Data.MutArray
24− [A] pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
25− [A] emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
26− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
27− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
28− [A] appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
29− [A] append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
30−[C] Streamly.Data.Array.Generic
31− [A] createOf :: MonadIO m => Int -> Fold m a (Array a)
32− [A] create :: MonadIO m => Fold m a (Array a)
33−[C] Streamly.Data.Array
34− [A] class Serialize a
35− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
36− [A] pinnedSerialize :: Serialize a => a -> Array Word8
37− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
38− [A] deserialize :: Serialize a => Array Word8 -> a
39− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
40− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
41− [A] addSizeTo :: Serialize a => Int -> a -> Int
42−
43−---------------------------------
44−Internal API diff
45−---------------------------------
46−
47−[C] Streamly.Internal.Unicode.Stream
48− [A] encodeUtf16le' :: Stream m Char -> Stream m Word16
49− [A] decodeUtf16le' :: Stream m Word16 -> Stream m Char
50−[A] Streamly.Internal.FileSystem.Path
51− [A] class IsPath a
52− [A] Rel
53− [A] File
54− [A] Dir
55− [A] Abs
56− [A] Streamly.Internal.FileSystem.Path.IsPath
57− [A] instance Streamly.Internal.FileSystem.Path.IsPath Streamly.Internal.FileSystem.Path.Path
58− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel Streamly.Internal.FileSystem.Path.Path)
59− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path))
60− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path))
61− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path)
62− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path)
63− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs Streamly.Internal.FileSystem.Path.Path)
64− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path))
65− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path))
66− [A] GHC.Show.Show
67− [A] instance GHC.Show.Show Streamly.Internal.FileSystem.Path.PathException
68− [A] GHC.Exception.Type.Exception
69− [A] instance GHC.Exception.Type.Exception Streamly.Internal.FileSystem.Path.PathException
70− [A] GHC.Classes.Eq
71− [A] instance GHC.Classes.Eq Streamly.Internal.FileSystem.Path.PathException
72− [A] Path
73− [A] Path :: Array Word8 -> Path
74− [A] toString :: Path -> [Char]
75− [A] toPath :: IsPath a => a -> Path
76− [A] toChunk :: Path -> Array Word8
77− [A] toChars :: Monad m => Path -> Stream m Char
78− [A] relfile :: QuasiQuoter
79− [A] reldir :: QuasiQuoter
80− [A] rel :: QuasiQuoter
81− [A] primarySeparator :: Char
82− [A] path :: QuasiQuoter
83− [A] mkRelFile :: String -> Q Exp
84− [A] mkRelDir :: String -> Q Exp
85− [A] mkRel :: String -> Q Exp
86− [A] mkPath :: String -> Q Exp
87− [A] mkFile :: String -> Q Exp
88− [A] mkDir :: String -> Q Exp
89− [A] mkAbsFile :: String -> Q Exp
90− [A] mkAbsDir :: String -> Q Exp
91− [A] mkAbs :: String -> Q Exp
92− [A] isSeparator :: Char -> Bool
93− [A] fromString :: MonadThrow m => [Char] -> m Path
94− [A] fromPathUnsafe :: IsPath a => Path -> a
95− [A] fromPath :: (IsPath a, MonadThrow m) => Path -> m a
96− [A] fromChunkUnsafe :: Array Word8 -> Path
97− [A] fromChunk :: MonadThrow m => Array Word8 -> m Path
98− [A] fromChars :: MonadThrow m => Stream Identity Char -> m Path
99− [A] file :: QuasiQuoter
100− [A] extendPath :: Path -> Path -> Path
101− [A] extendDir :: (IsPath (a (Dir Path)), IsPath b, IsPath (a b)) => a (Dir Path) -> Rel b -> a b
102− [A] dir :: QuasiQuoter
103− [A] adaptPath :: (MonadThrow m, IsPath a, IsPath b) => a -> m b
104− [A] absfile :: QuasiQuoter
105− [A] absdir :: QuasiQuoter
106− [A] abs :: QuasiQuoter
107−[C] Streamly.Internal.Data.Stream
108− [A] (FixityR,5)
109− [A] (FixityR,5)
110− [C] splitInnerBySuffix
111− [O] splitInnerBySuffix :: (Monad m, Eq (f a), Monoid (f a)) => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
112− [N] splitInnerBySuffix :: Monad m => (f a -> Bool) -> (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
113− [D] sliceOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
114− [A] indexOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
115−[C] Streamly.Internal.Data.MutByteArray
116− [A] unsafePinnedAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
117− [A] unsafeAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
118− [D] nil :: MutByteArray
119− [A] empty :: MutByteArray
120− [D] asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
121−[D] Streamly.Internal.Data.MutArray.Stream
122− [D] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
123− [D] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
124− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
125− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
126−[C] Streamly.Internal.Data.MutArray.Generic
127− [D] writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
128− [D] writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
129− [A] unsafeCreateOf :: MonadIO m => Int -> Fold m a (MutArray a)
130− [A] emptyOf :: MonadIO m => Int -> m (MutArray a)
131− [A] createWith :: MonadIO m => Int -> Fold m a (MutArray a)
132− [A] createOf :: MonadIO m => Int -> Fold m a (MutArray a)
133− [A] create :: MonadIO m => Fold m a (MutArray a)
134−[C] Streamly.Internal.Data.MutArray
135− [A] SpliceState
136− [A] SpliceYielding :: arr -> SpliceState s arr -> SpliceState s arr
137− [A] SpliceInitial :: s -> SpliceState s arr
138− [A] SpliceFinish :: SpliceState s arr
139− [A] SpliceBuffering :: s -> arr -> SpliceState s arr
140− [R] MutByteArray
141− [D] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
142− [D] writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
143− [D] writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
144− [D] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
145− [D] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
146− [D] writeAppendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
147− [D] writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
148− [A] unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
149− [A] unsafePinnedAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
150− [A] unsafeCreateOfWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
151− [A] unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
152− [A] unsafeAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
153− [A] unsafeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
154− [A] toStreamWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
155− [A] toStreamRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
156− [R] toStreamDWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
157− [R] toStreamDRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
158− [A] slicerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
159− [A] sliceIndexerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
160− [A] revCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
161− [D] resizeExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
162− [D] resize :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
163− [A] pokeSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
164− [A] pokeAppendMay :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> a -> m (Maybe (MutArray Word8))
165− [A] pokeAppend :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> a -> m (MutArray Word8)
166− [D] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
167− [D] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
168− [D] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
169− [D] pinnedNewBytes :: MonadIO m => Int -> m (MutArray a)
170− [A] pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
171− [A] pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
172− [A] pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
173− [A] pinnedCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
174− [A] peekUnconsUnsafe :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> m (a, MutArray Word8)
175− [A] peekUncons :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> m (Maybe a, MutArray Word8)
176− [A] peekSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
177− [A] pPinnedCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Parser (MutArray a) m (MutArray a)
178− [A] pCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Parser (MutArray a) m (MutArray a)
179− [D] nil :: MutArray a
180− [A] lPinnedCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
181− [A] lCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
182− [A] indexReaderWith :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
183− [A] indexReader :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
184− [A] growExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
185− [A] grow :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
186− [D] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
187− [R] getIndicesD :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
188− [D] getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
189− [D] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
190− [A] fromStreamN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
191− [D] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
192− [D] fromStreamD :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
193− [A] fromPureStreamN :: (MonadIO m, Unbox a) => Int -> Stream Identity a -> m (MutArray a)
194− [A] fromPtrN :: MonadIO m => Int -> Ptr Word8 -> m (MutArray Word8)
195− [A] fromChunksRealloced :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> m (MutArray a)
196− [A] fromChunksK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
197− [A] fromByteStr# :: MonadIO m => Addr# -> m (MutArray Word8)
198− [D] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
199− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
200− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
201− [A] fPinnedCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) (MutArray a)
202− [A] fCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) (MutArray a)
203− [A] emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
204− [A] empty :: MutArray a
205− [A] createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
206− [A] createOfWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
207− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
208− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
209− [A] concatWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m (MutArray a) -> Stream m a
210− [A] concatRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m (MutArray a) -> Stream m a
211− [A] concatRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
212− [A] concat :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
213− [A] compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (MutArray Word8) -> Stream m (MutArray Word8)
214− [A] compactOnByte :: MonadIO m => Word8 -> Stream m (MutArray Word8) -> Stream m (MutArray Word8)
215− [A] compactLeAs :: forall m a. (MonadIO m, Unbox a) => PinnedState -> Int -> Stream m (MutArray a) -> Stream m (MutArray a)
216− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
217− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
218− [A] compactEQ :: Int -> Stream m (MutArray a) -> Stream m (MutArray a)
219− [D] cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
220− [A] byteEq :: MonadIO m => MutArray a -> MutArray a -> m Bool
221− [A] byteCmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
222− [A] buildChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
223− [D] asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
224− [A] appendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
225− [A] appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
226− [A] append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
227−[D] Streamly.Internal.Data.Array.Stream
228− [C] interposeSuffix
229− [O] interposeSuffix :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
230− [N] interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
231− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
232− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
233− [C] concatRev
234− [O] concatRev :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
235− [N] concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
236− [D] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
237−[C] Streamly.Internal.Data.Array.Generic
238− [A] createOf :: MonadIO m => Int -> Fold m a (Array a)
239− [A] create :: MonadIO m => Fold m a (Array a)
240−[C] Streamly.Internal.Data.Array
241− [R] ArrayUnsafe
242− [D] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
243− [D] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
244− [A] unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
245− [A] unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
246− [A] unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
247− [D] toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
248− [D] toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
249− [C] splice
250− [O] splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
251− [N] splice :: MonadIO m => Array a -> Array a -> m (Array a)
252− [A] slicerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Array a)
253− [A] sliceIndexerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Int, Int)
254− [D] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
255− [D] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
256− [D] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
257− [D] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
258− [A] pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
259− [A] pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
260− [A] pinnedCompactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
261− [A] parseBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
262− [D] nil :: Array a
263− [A] lPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
264− [A] lCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
265− [A] interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
266− [A] interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
267− [A] intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a
268− [A] indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a
269− [A] indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
270− [A] indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int
271− [D] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Array a)
272− [R] getIndicesFromThenTo :: Unfold m (Int, Int, Int, Array a) a
273− [D] getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
274− [D] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Int, Int)
275− [D] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
276− [D] fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)
277− [A] fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a
278− [A] fromPtrN :: Int -> Ptr Word8 -> Array Word8
279− [A] fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)
280− [A] fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
281− [A] fromByteStr :: Ptr Word8 -> Array Word8
282− [A] foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b
283− [A] foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
284− [A] foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
285− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
286− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
287− [C] findIndicesOf
288− [O] findIndicesOf :: (a -> Bool) -> Unfold Identity (Array a) Int
289− [N] findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int
290− [A] fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
291− [A] fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
292− [A] empty :: Array a
293− [A] createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
294− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
295− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
296− [A] concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
297− [A] concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
298− [A] compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
299− [A] compactOnByte :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
300− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
301− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
302− [A] byteEq :: Array a -> Array a -> Bool
303− [A] byteCmp :: Array a -> Array a -> Ordering
304− [A] buildChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
305− [D] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
306− [D] asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
core/docs/Changelog.md+12−60
@@ -1,73 +1,25 @@
11# Changelog
22
3−## 0.2.2 (Jan 2024)
3+## Unreleased
44
5−* Add fixities `infixr 5` for `cons` and `consM` functions.
6−* Fix a bug in Array `Eq` instance when the type is a sum type with
7− differently sized constructors.
8−* lpackArraysChunksOf, compact, writeChunksWith, putChunksWith now take the
9− buffer size in number of array elements instead of bytes.
5+* Arrays are now created unpinned by default, they were created pinned
6+ earlier. During IO operations unpinned arrays are automatically copied
7+ to pinned memory. When arrays are directly passed to IO operations
8+ programmers can choose to create them pinned to avoid a copy. To
9+ create pinned arrays, use the internal APIs with the `pinned*` prefix.
1010
11−## 0.2.1 (Dec 2023)
12−
13−* Make the serialization of the unit constructor deterministic.
14−* Expose `pinnedSerialize` & `deserialize` via `Streamly.Data.Array`.
15−
16−## 0.2.0 (Nov 2023)
17−
18−See [0.1.0-0.2.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0-0.2.0.txt)
19−for a full list of API changes in this release. Only a few significant
20−changes are mentioned here.
21−
22−### Breaking Changes
23−
24−* `ParserK` in `Streamly.Data.ParserK` is not implicitly specialized
25− to arrays anymore. To adapt to the new code, change `ParserK a m
26− b` to `ParserK (Array a) m b` where the `Array` type comes from
27− `Streamly.Data.Array`. This change also affected the signatures of
28− `parseChunks` and `parseBreakChunks`.
29−* Changed the signature of 'Streamly.Data.Stream.handle' to make the
30− exception handler monadic.
31−* Behavior change: Exceptions are now rethrown promptly in `bracketIO`.
32−
33−### Enhancements
34−
35−* __Serialization__: Added a `Streamly.Data.MutByteArray` module with a
36− `Serialize` type class for fast binary serialization. The Data.Array
37− module supplies the `serialize` and `deserialize` operations for arrays.
38−* __Unpinned Arrays__: Unboxed arrays are now created unpinned by default,
39− they were created pinned earlier. During IO operations, unpinned arrays
40− are automatically copied to pinned memory. When arrays are directly
41− passed to IO operations programmers can choose to create them pinned to
42− avoid a copy. To create pinned arrays, use the internal APIs with the
43− `pinned*` prefix.
44−* StreamK now supports native exception handling routines (handle, bracketIO).
45− Earlier we had to convert it to the `Stream` type for exception handling.
11+* Changed the signature of 'Streamly.Data.Stream.handle' to make the exception
12+ handler monadic.
13+* Rethrow the exception promptly in bracketIO.
4614
4715### Deprecations
4816
49−See [0.1.0-0.2.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0-0.2.0.txt)
50−for a full list of deprecations.
51−
52−### Internal API Changes
53−
54−* Fold constructor has changed, added a `final` field to support
55− finalization and cleanup of a chain of folds. The `extract` field is
56− now used only for mapping the fold internal state to fold result for
57− scanning purposes. If your fold does not require cleanup you can just use
58− your existing `extract` function as `final` as well to adapt to this change.
59−* Many low level internal modules have been removed, they are entirely
60− exported from higher level internal modules. If you were importing any
61− of the missing low level modules then import the higher level modules instead.
62−* Internal module changes:
63− * Streamly.Internal.Serialize.FromBytes -> Streamly.Internal.Data.Binary.Parser
64− * Streamly.Internal.Serialize.ToBytes -> Streamly.Internal.Data.Binary.Stream
65− * Streamly.Internal.Data.Unbox is now exported via Streamly.Internal.Data.Serialize
66− * Streamly.Internal.Data.IORef.Unboxed is now exported via Streamly.Internal.Data.Serialize
17+* `Streamly.Data.MutArray.newUnpinned` is renamed to
18+ `Streamly.Data.MutArray.pinnedNew`
6719
6820## 0.1.0 (March 2023)
6921
70−Also see [streamly-core-0.1.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0.txt) or
22+Also see [streamly-core-0.1.0 API Changelog](/core/docs/ApiChangelogs/0.1.0.txt) or
7123https://hackage.haskell.org/package/streamly-core-0.1.0/docs/docs/ApiChangelogs/0.1.0.txt
7224
7325`streamly` package is split into two packages, (1) `streamly-core` that
core/src/DocTestDataArray.hs−8
@@ -1,7 +1,6 @@
11{- $setup
22>>> :m
33>>> :set -XFlexibleContexts
4−>>> :set -XMagicHash
54>>> import Data.Function ((&))
65>>> import Data.Functor.Identity (Identity(..))
76>>> import System.IO.Unsafe (unsafePerformIO)
@@ -11,12 +10,5 @@
1110
1211>>> import qualified Streamly.Data.Array as Array
1312>>> import qualified Streamly.Data.Fold as Fold
14−>>> import qualified Streamly.Data.ParserK as ParserK
1513>>> import qualified Streamly.Data.Stream as Stream
16−>>> import qualified Streamly.Data.StreamK as StreamK
17−
18−For APIs that have not been released yet.
19−
20−>>> import qualified Streamly.Internal.Data.Array as Array
21−>>> import qualified Streamly.Internal.Data.Stream as Stream
2214-}
core/src/DocTestDataFold.hs+1
@@ -24,4 +24,5 @@
2424For APIs that have not been released yet.
2525
2626>>> import qualified Streamly.Internal.Data.Fold as Fold
27+>>> import qualified Streamly.Internal.Data.Fold.Window as FoldW
2728-}
core/src/DocTestDataParserK.hsdeleted−15
@@ -1,15 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import Control.Applicative ((<|>))
4−>>> import Data.Char (isDigit, isAlpha)
5−
6−>>> import Streamly.Data.Parser (Parser)
7−>>> import Streamly.Data.ParserK (ParserK)
8−
9−>>> import qualified Streamly.Data.Parser as Parser
10−>>> import qualified Streamly.Data.ParserK as ParserK
11−
12−For APIs that have not been released yet.
13−
14−>>> import qualified Streamly.Internal.Data.ParserK as ParserK
15−-}
core/src/DocTestDataStream.hs+1−2
@@ -6,7 +6,6 @@
66>>> import Control.Monad.IO.Class (MonadIO (liftIO))
77>>> import Control.Monad.Trans.Class (lift)
88>>> import Control.Monad.Trans.Identity (runIdentityT)
9−>>> import Data.Char (isSpace)
109>>> import Data.Either (fromLeft, fromRight, isLeft, isRight, either)
1110>>> import Data.Maybe (fromJust, isJust)
1211>>> import Data.Function (fix, (&))
@@ -31,9 +30,9 @@
3130For APIs that have not been released yet.
3231
3332>>> import qualified Streamly.Internal.Data.Fold as Fold
33+>>> import qualified Streamly.Internal.Data.Fold.Window as Window
3434>>> import qualified Streamly.Internal.Data.Parser as Parser
3535>>> import qualified Streamly.Internal.Data.Stream as Stream
36−>>> import qualified Streamly.Internal.Data.StreamK as StreamK
3736>>> import qualified Streamly.Internal.Data.Unfold as Unfold
3837>>> import qualified Streamly.Internal.FileSystem.Dir as Dir
3938-}
core/src/DocTestFileSystemHandle.hsdeleted−15
@@ -1,15 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import qualified Streamly.Data.Array as Array
4−>>> import qualified Streamly.FileSystem.Handle as Handle hiding (readChunks)
5−>>> import qualified Streamly.Data.Fold as Fold
6−>>> import qualified Streamly.Data.Stream as Stream
7−>>> import qualified Streamly.Data.Unfold as Unfold
8−
9−For APIs that have not been released yet.
10−
11−>>> import qualified Streamly.Internal.Data.Array as Array (unsafeCreateOf)
12−>>> import qualified Streamly.Internal.Data.Unfold as Unfold (first)
13−>>> import qualified Streamly.Internal.FileSystem.Handle as Handle
14−>>> import qualified Streamly.Internal.System.IO as IO (defaultChunkSize)
15−-}
core/src/DocTestUnicodeParser.hsdeleted−9
@@ -1,9 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import qualified Streamly.Data.Stream as Stream
4−>>> import qualified Streamly.Unicode.Parser as Unicode
5−
6−For APIs that have not been released yet.
7−
8−>>> import qualified Streamly.Internal.Unicode.Parser as Unicode (number, mkDouble)
9−-}
core/src/DocTestUnicodeStream.hsdeleted−12
@@ -1,12 +0,0 @@
1−{- $setup
2−>>> :m
3−
4−>>> import qualified Streamly.Data.Fold as Fold
5−>>> import qualified Streamly.Data.Stream as Stream
6−>>> import qualified Streamly.Unicode.Stream as Unicode
7−
8−For APIs that have not been released yet.
9−
10−>>> :set -XMagicHash
11−>>> import qualified Streamly.Internal.Unicode.Stream as Unicode
12−-}
core/src/DocTestUnicodeString.hsdeleted−5
@@ -1,5 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> :set -XQuasiQuotes
4−>>> import Streamly.Internal.Unicode.String
5−-}
core/src/Streamly/Console/Stdio.hs+27−1
@@ -24,7 +24,33 @@ module Streamly.Console.Stdio
2424 -- * Write (stderr)
2525 , writeErr
2626 , writeErrChunks
27+
28+ -- * Deprecated
29+ , read
30+ , readChunks
2731 )
2832where
2933
30−import Streamly.Internal.Console.Stdio
34+import Control.Monad.IO.Class (MonadIO(..))
35+import Data.Word (Word8)
36+import Streamly.Internal.Data.Array.Type (Array)
37+import Streamly.Internal.Data.Unfold (Unfold)
38+
39+import Streamly.Internal.Console.Stdio hiding (read, readChunks)
40+import Prelude hiding (read)
41+
42+-- Same as 'reader'
43+--
44+-- @since 0.8.0
45+{-# DEPRECATED read "Please use 'reader' instead" #-}
46+{-# INLINE read #-}
47+read :: MonadIO m => Unfold m () Word8
48+read = reader
49+
50+-- Same as 'chunkReader'
51+--
52+-- @since 0.8.0
53+{-# DEPRECATED readChunks "Please use 'chunkReader' instead" #-}
54+{-# INLINE readChunks #-}
55+readChunks :: MonadIO m => Unfold m () (Array Word8)
56+readChunks = chunkReader
core/src/Streamly/Data/Array.hs+48−48
@@ -29,80 +29,65 @@ module Streamly.Data.Array
2929 -- $overview
3030
3131 -- * The Array Type
32− Array
33−
34− -- * Pinning & Unpinning
35− -- | Arrays are created unpinned by default unless pinned versions of
36− -- creation APIs are used. Look for APIs with @pinned@ prefix in
37− -- "Streamly.Internal.Data.Array" for some unreleased pinned creation APIs.
38− -- If an array is to be sent to the OS without any further modification
39− -- then it should be created pinned in the first place instead of pinning
40− -- it later. Pinning an unpinned array has a copy overhead. OS interfacing
41− -- APIs create a pinned array directly or convert an unpinned array to
42− -- pinned array before sending it to the OS.
43− , pin
44− , unpin
45− , isPinned
32+ A.Array
4633
4734 -- * Construction
4835 -- | When performance matters, the fastest way to generate an array is
49− -- 'createOf'. 'IsList' and 'IsString' instances can be
36+ -- 'writeN'. 'IsList' and 'IsString' instances can be
5037 -- used to conveniently construct arrays from literal values.
5138 -- 'OverloadedLists' extension or 'fromList' can be used to construct an
5239 -- array from a list literal. Similarly, 'OverloadedStrings' extension or
5340 -- 'fromList' can be used to construct an array from a string literal.
5441
55− -- ** From Stream
56− , createOf
57− , create
58− , writeLastN -- drop old (ring buffer)
42+ -- Pure List APIs
43+ , A.fromListN
44+ , A.fromList
5945
60− -- ** From List
61− , fromListN
62− , fromList
46+ -- Monadic APIs
47+ , A.writeN -- drop new
48+ , A.write -- full buffer
49+ , writeLastN -- drop old (ring buffer)
6350
64− -- * To List
51+ -- * Conversion
6552 -- 'GHC.Exts.toList' from "GHC.Exts" can be used to convert an array to a
6653 -- list.
67− , toList
68−
69− -- * To Stream
70− , read
71− , readRev
54+ , A.toList
7255
7356 -- * Unfolds
74− , reader
75− , readerRev
57+ , A.reader
58+ , A.readerRev
7659
7760 -- * Casting
7861 , cast
7962 , asBytes
8063
8164 -- * Random Access
82− , length
65+ , A.length
8366 -- , (!!)
84− , getIndex
85−
86− -- * Serialization
87− , pinnedSerialize
88− , deserialize
67+ , A.getIndex
8968
90− -- * Re-exports
69+ -- * Unbox Type Class
9170 , Unbox (..)
92− , Serialize(..)
71+
72+ -- * Pinning & Unpinning
73+ , pin
74+ , unpin
75+ , isPinned
9376
9477 -- * Deprecated
95− , writeN -- drop new
96− , write -- full buffer
78+ , read
79+ , readRev
9780 )
9881where
9982
10083#include "inline.hs"
10184
102−import Streamly.Internal.Data.Array
103−import Streamly.Internal.Data.MutByteArray (Unbox(..), Serialize(..))
85+import Streamly.Internal.Data.Unfold (Unfold)
86+import Streamly.Internal.Data.Array as A hiding (read, readRev)
87+import Streamly.Internal.Data.Array.Type (pin, unpin, isPinned)
10488
105−import Prelude hiding (read, length)
89+import Streamly.Internal.Data.Unbox (Unbox (..))
90+import Prelude hiding (read)
10691
10792#include "DocTestDataArray.hs"
10893
@@ -118,7 +103,7 @@ import Prelude hiding (read, length)
118103--
119104-- Convert array to stream, and fold the stream:
120105--
121−-- >>> fold f arr = Array.read arr & Stream.fold f
106+-- >>> fold f arr = Stream.unfold Array.reader arr & Stream.fold f
122107-- >>> fold Fold.sum (Array.fromList [1,2,3::Int])
123108-- 6
124109--
@@ -126,7 +111,7 @@ import Prelude hiding (read, length)
126111--
127112-- Convert array to stream, transform, and fold back to array:
128113--
129−-- >>> amap f arr = Array.read arr & fmap f & Stream.fold (Array.createOf (Array.length arr))
114+-- >>> amap f arr = Stream.unfold Array.reader arr & fmap f & Stream.fold Array.write
130115-- >>> amap (+1) (Array.fromList [1,2,3::Int])
131116-- fromList [2,3,4]
132117--
@@ -153,11 +138,12 @@ import Prelude hiding (read, length)
153138-- See the 'fromPureStream' unreleased API to generate an array from an
154139-- Identity stream safely without using MonadIO constraint.
155140--
141+--
156142-- Note that 'Identity' streams can be generalized to IO streams:
157143--
158−-- >>> stream = Stream.fromList [1,2,3] :: Stream Identity Int
159−-- >>> pure2IO s = Stream.morphInner (return . runIdentity) s :: Stream IO Int
160−-- >>> Stream.fold Array.create (pure2IO stream)
144+-- >>> pure = Stream.fromList [1,2,3] :: Stream Identity Int
145+-- >>> generally = Stream.morphInner (return . runIdentity)
146+-- >>> Stream.fold Array.write (generally pure :: Stream IO Int)
161147-- fromList [1,2,3]
162148--
163149-- == Programming Tips
@@ -165,3 +151,17 @@ import Prelude hiding (read, length)
165151-- This module is designed to be imported qualified:
166152--
167153-- >>> import qualified Streamly.Data.Array as Array
154+
155+-- | Same as 'reader'
156+--
157+{-# DEPRECATED read "Please use 'reader' instead" #-}
158+{-# INLINE_NORMAL read #-}
159+read :: (Monad m, Unbox a) => Unfold m (Array a) a
160+read = reader
161+
162+-- | Same as 'readerRev'
163+--
164+{-# DEPRECATED readRev "Please use 'readerRev' instead" #-}
165+{-# INLINE_NORMAL readRev #-}
166+readRev :: (Monad m, Unbox a) => Unfold m (Array a) a
167+readRev = readerRev
core/src/Streamly/Data/Array/Generic.hs+13−16
@@ -15,37 +15,34 @@ module Streamly.Data.Array.Generic
1515 ( Array
1616
1717 -- * Construction
18− , fromListN
19− , fromList
18+ , A.fromListN
19+ , A.fromList
2020
2121 -- MonadicAPIs
22− , createOf
23− , create
22+ , A.writeN
23+ , A.write
2424
2525 -- * Conversion
26− , toList
26+ , A.toList
2727
2828 -- * Streams
29− , read
30− , readRev
29+ , A.read
30+ , A.readRev
3131
3232 -- * Unfolds
33− , reader
33+ , A.reader
3434 -- , A.readerRev
3535
3636 -- * Random Access
37− , length
38− , getIndex
37+ , A.length
38+ , A.getIndex
3939
4040 -- -- * Folding Arrays
4141 -- , A.streamFold
4242 -- , A.fold
43−
44− -- * Deprecated
45− , writeN
46− , write
4743 )
4844where
4945
50−import Streamly.Internal.Data.Array.Generic
51−import Prelude hiding (length, read)
46+import Streamly.Internal.Data.Array.Generic (Array)
47+
48+import qualified Streamly.Internal.Data.Array.Generic as A
core/src/Streamly/Data/Fold.hs+119−150
@@ -10,156 +10,6 @@
1010-- Fast, composable stream consumers with ability to terminate, supporting
1111-- stream fusion.
1212--
13−-- == Using Folds
14−--
15−-- This module provides elementary folds and fold combinators that can be used
16−-- to consume a stream of data and reduce it to a final value, or transform it
17−-- in a stateful manner using scans. A data stream can be reduced into a stream
18−-- of folded data elements by folding segments of the stream. Fold combinators
19−-- can be used to compose multiple folds in parallel or to create a pipeline of
20−-- folds such that the next fold consumes the result of the previous fold. To
21−-- run these folds on a stream see 'Streamly.Data.Stream.fold',
22−-- 'Streamly.Data.Stream.scan', 'Streamly.Data.Stream.postscan',
23−-- 'Streamly.Data.Stream.scanMaybe', 'Streamly.Data.Stream.foldMany' and other
24−-- operations accepting 'Fold' type as argument "Streamly.Data.Stream".
25−--
26−-- == Reducing a Stream
27−--
28−-- A 'Fold' is a consumer of a stream of values. A fold driver (such as
29−-- 'Streamly.Data.Stream.fold') initializes the fold @accumulator@, runs the
30−-- fold @step@ function in a loop, processing the input stream one element at a
31−-- time and accumulating the result. The loop continues until the fold
32−-- terminates, at which point the accumulated result is returned.
33−--
34−-- For example, a 'sum' Fold represents a stream consumer that adds the values
35−-- in the input stream:
36−--
37−-- >>> Stream.fold Fold.sum $ Stream.fromList [1..100]
38−-- 5050
39−--
40−-- Conceptually, a 'Fold' is a data type that mimics a strict left fold
41−-- ('Data.List.foldl'). The above example is similar to a left fold using
42−-- @(+)@ as the step and @0@ as the initial value of the accumulator:
43−--
44−-- >>> Data.List.foldl' (+) 0 [1..100]
45−-- 5050
46−--
47−-- 'Fold's have an early termination capability e.g. the 'one' fold terminates
48−-- after consuming one element:
49−--
50−-- >>> Stream.fold Fold.one $ Stream.fromList [1..]
51−-- Just 1
52−--
53−-- The above example is similar to the following right fold:
54−--
55−-- >>> Prelude.foldr (\x _ -> Just x) Nothing [1..]
56−-- Just 1
57−--
58−-- 'Fold's can be combined together using combinators. For example, to create a
59−-- fold that sums first two elements in a stream:
60−--
61−-- >>> sumTwo = Fold.take 2 Fold.sum
62−-- >>> Stream.fold sumTwo $ Stream.fromList [1..100]
63−-- 3
64−--
65−-- == Parallel Composition
66−--
67−-- Folds can be combined to run in parallel on the same input. For example, to
68−-- compute the average of numbers in a stream without going through the stream
69−-- twice:
70−--
71−-- >>> avg = Fold.teeWith (/) Fold.sum (fmap fromIntegral Fold.length)
72−-- >>> Stream.fold avg $ Stream.fromList [1.0..100.0]
73−-- 50.5
74−--
75−-- Folds can be combined so as to partition the input stream over multiple
76−-- folds. For example, to count even and odd numbers in a stream:
77−--
78−-- >>> split n = if even n then Left n else Right n
79−-- >>> stream = fmap split $ Stream.fromList [1..100]
80−-- >>> countEven = fmap (("Even " ++) . show) Fold.length
81−-- >>> countOdd = fmap (("Odd " ++) . show) Fold.length
82−-- >>> f = Fold.partition countEven countOdd
83−-- >>> Stream.fold f stream
84−-- ("Even 50","Odd 50")
85−--
86−-- == Sequential Composition
87−--
88−-- Terminating folds can be combined to parse the stream serially such that the
89−-- first fold consumes the input until it terminates and the second fold
90−-- consumes the rest of the input until it terminates:
91−--
92−-- >>> f = Fold.splitWith (,) (Fold.take 8 Fold.toList) (Fold.takeEndBy (== '\n') Fold.toList)
93−-- >>> Stream.fold f $ Stream.fromList "header: hello\n"
94−-- ("header: ","hello\n")
95−--
96−-- == Splitting a Stream
97−--
98−-- A 'Fold' can be applied repeatedly on a stream to transform it to a stream
99−-- of fold results. To split a stream on newlines:
100−--
101−-- >>> f = Fold.takeEndBy (== '\n') Fold.toList
102−-- >>> Stream.fold Fold.toList $ Stream.foldMany f $ Stream.fromList "Hello there!\nHow are you\n"
103−-- ["Hello there!\n","How are you\n"]
104−--
105−-- Similarly, we can split the input of a fold too:
106−--
107−-- >>> Stream.fold (Fold.many f Fold.toList) $ Stream.fromList "Hello there!\nHow are you\n"
108−-- ["Hello there!\n","How are you\n"]
109−--
110−-- == Folds vs. Streams
111−--
112−-- We can often use streams or folds to achieve the same goal. However, streams
113−-- are more efficient in composition of producers (e.g.
114−-- 'Data.Stream.append' or 'Data.Stream.mergeBy') whereas folds are
115−-- more efficient in composition of consumers (e.g. 'splitWith', 'partition'
116−-- or 'teeWith').
117−--
118−-- Streams are producers, transformations on streams happen on the output side:
119−--
120−-- >>> :{
121−-- f stream =
122−-- Stream.filter odd stream
123−-- & fmap (+1)
124−-- & Stream.fold Fold.sum
125−-- :}
126−--
127−-- >>> f $ Stream.fromList [1..100 :: Int]
128−-- 2550
129−--
130−-- Folds are stream consumers with an input stream and an output value, stream
131−-- transformations on folds happen on the input side:
132−--
133−-- >>> :{
134−-- f =
135−-- Fold.filter odd
136−-- $ Fold.lmap (+1)
137−-- $ Fold.sum
138−-- :}
139−--
140−-- >>> Stream.fold f $ Stream.fromList [1..100 :: Int]
141−-- 2550
142−--
143−-- Notice the similiarity in the definition of @f@ in both cases, the only
144−-- difference is the composition by @&@ vs @$@ and the use @lmap@ vs @map@, the
145−-- difference is due to output vs input side transformations.
146−--
147−-- == Fusion Limitations
148−--
149−-- Folds support stream fusion for generating loops comparable to the speed of
150−-- C. However, it has some limitations. For fusion to work, the folds must be
151−-- inlined, folds must be statically known and not generated dynamically, folds
152−-- should not be passed recursively.
153−--
154−-- Another limitation is due to the quadratic complexity causing slowdown when
155−-- too many nested compositions are used. Especially, the performance of the
156−-- Applicative instance and splitting operations (e.g. 'splitWith') degrades
157−-- quadratically (O(n^2)) when combined @n@ times, roughly 8 or less sequenced
158−-- operations are fine. For these cases folds can be converted to parsers and
159−-- then used as ParserK.
160−--
161−-- == Experimental APIs
162−--
16313-- Please refer to "Streamly.Internal.Data.Fold" for more functions that have
16414-- not yet been released.
16515
@@ -171,6 +21,9 @@ module Streamly.Data.Fold
17121 --
17222 -- $setup
17323
24+ -- * Overview
25+ -- $overview
26+
17427 -- * Running A Fold
17528 drive
17629 -- XXX Should we have a stream returning function in fold module?
@@ -402,9 +255,125 @@ import Prelude
402255 span, splitAt, break, mapM, maybe)
403256
404257import Streamly.Internal.Data.Fold
258+import Streamly.Internal.Data.Fold.Container
405259
406260#include "DocTestDataFold.hs"
407261
262+-- $overview
263+--
264+-- A 'Fold' is a consumer of a stream of values. A fold driver (such as
265+-- 'Streamly.Data.Stream.fold') initializes the fold @accumulator@, runs the
266+-- fold @step@ function in a loop, processing the input stream one element at a
267+-- time and accumulating the result. The loop continues until the fold
268+-- terminates, at which point the accumulated result is returned.
269+--
270+-- For example, a 'sum' Fold represents a stream consumer that adds the values
271+-- in the input stream:
272+--
273+-- >>> Stream.fold Fold.sum $ Stream.fromList [1..100]
274+-- 5050
275+--
276+-- Conceptually, a 'Fold' is a data type that mimics a strict left fold
277+-- ('Data.List.foldl'). The above example is similar to a left fold using
278+-- @(+)@ as the step and @0@ as the initial value of the accumulator:
279+--
280+-- >>> Data.List.foldl' (+) 0 [1..100]
281+-- 5050
282+--
283+-- 'Fold's have an early termination capability e.g. the 'one' fold terminates
284+-- after consuming one element:
285+--
286+-- >>> Stream.fold Fold.one $ Stream.fromList [1..]
287+-- Just 1
288+--
289+-- The above example is similar to the following right fold:
290+--
291+-- >>> Prelude.foldr (\x _ -> Just x) Nothing [1..]
292+-- Just 1
293+--
294+-- 'Fold's can be combined together using combinators. For example, to create a
295+-- fold that sums first two elements in a stream:
296+--
297+-- >>> sumTwo = Fold.take 2 Fold.sum
298+-- >>> Stream.fold sumTwo $ Stream.fromList [1..100]
299+-- 3
300+--
301+-- Folds can be combined to run in parallel on the same input. For example, to
302+-- compute the average of numbers in a stream without going through the stream
303+-- twice:
304+--
305+-- >>> avg = Fold.teeWith (/) Fold.sum (fmap fromIntegral Fold.length)
306+-- >>> Stream.fold avg $ Stream.fromList [1.0..100.0]
307+-- 50.5
308+--
309+-- Folds can be combined so as to partition the input stream over multiple
310+-- folds. For example, to count even and odd numbers in a stream:
311+--
312+-- >>> split n = if even n then Left n else Right n
313+-- >>> stream = fmap split $ Stream.fromList [1..100]
314+-- >>> countEven = fmap (("Even " ++) . show) Fold.length
315+-- >>> countOdd = fmap (("Odd " ++) . show) Fold.length
316+-- >>> f = Fold.partition countEven countOdd
317+-- >>> Stream.fold f stream
318+-- ("Even 50","Odd 50")
319+--
320+-- Terminating folds can be combined to parse the stream serially such that the
321+-- first fold consumes the input until it terminates and the second fold
322+-- consumes the rest of the input until it terminates:
323+--
324+-- >>> f = Fold.splitWith (,) (Fold.take 8 Fold.toList) (Fold.takeEndBy (== '\n') Fold.toList)
325+-- >>> Stream.fold f $ Stream.fromList "header: hello\n"
326+-- ("header: ","hello\n")
327+--
328+-- A 'Fold' can be applied repeatedly on a stream to transform it to a stream
329+-- of fold results. To split a stream on newlines:
330+--
331+-- >>> f = Fold.takeEndBy (== '\n') Fold.toList
332+-- >>> Stream.fold Fold.toList $ Stream.foldMany f $ Stream.fromList "Hello there!\nHow are you\n"
333+-- ["Hello there!\n","How are you\n"]
334+--
335+-- Similarly, we can split the input of a fold too:
336+--
337+-- >>> Stream.fold (Fold.many f Fold.toList) $ Stream.fromList "Hello there!\nHow are you\n"
338+-- ["Hello there!\n","How are you\n"]
339+--
340+-- = Folds vs. Streams
341+--
342+-- We can often use streams or folds to achieve the same goal. However, streams
343+-- are more efficient in composition of producers (e.g.
344+-- 'Data.Stream.append' or 'Data.Stream.mergeBy') whereas folds are
345+-- more efficient in composition of consumers (e.g. 'splitWith', 'partition'
346+-- or 'teeWith').
347+--
348+-- Streams are producers, transformations on streams happen on the output side:
349+--
350+-- >>> :{
351+-- f stream =
352+-- Stream.filter odd stream
353+-- & fmap (+1)
354+-- & Stream.fold Fold.sum
355+-- :}
356+--
357+-- >>> f $ Stream.fromList [1..100 :: Int]
358+-- 2550
359+--
360+-- Folds are stream consumers with an input stream and an output value, stream
361+-- transformations on folds happen on the input side:
362+--
363+-- >>> :{
364+-- f =
365+-- Fold.filter odd
366+-- $ Fold.lmap (+1)
367+-- $ Fold.sum
368+-- :}
369+--
370+-- >>> Stream.fold f $ Stream.fromList [1..100 :: Int]
371+-- 2550
372+--
373+-- Notice the similiarity in the definition of @f@ in both cases, the only
374+-- difference is the composition by @&@ vs @$@ and the use @lmap@ vs @map@, the
375+-- difference is due to output vs input side transformations.
376+
408377--------------------------------------------------------------------------------
409378-- Deprecated
410379--------------------------------------------------------------------------------
core/src/Streamly/Data/MutArray.hs+12−18
@@ -32,27 +32,22 @@ module Streamly.Data.MutArray
3232 -- * Construction
3333
3434 -- Uninitialized Arrays
35− , emptyOf
36− , pinnedEmptyOf
35+ , new
36+ , pinnedNew
3737
3838 -- From containers
3939 , fromListN
4040 , fromList
41− , createOf
42− , create
41+ , writeN -- drop new
42+ , write -- full buffer
4343 -- writeLastN
4444
45− -- * Pinning & Unpinning
46− , pin
47− , unpin
48− , isPinned
49−
5045 -- * Appending elements
5146 , snoc
5247
5348 -- * Appending streams
54− , appendN
55− , append
49+ , writeAppendN -- XXX Rename to writeSnocN?
50+ , writeAppend -- XXX Rename to writeSnoc?
5651
5752 -- * Inplace mutation
5853 , putIndex
@@ -83,17 +78,16 @@ module Streamly.Data.MutArray
8378 -- * Size
8479 , length
8580
86− -- * Re-exports
81+ -- * Unbox Type Class
8782 , Unbox (..)
8883
84+ -- * Pinning & Unpinning
85+ , pin
86+ , unpin
87+ , isPinned
88+
8989 -- * Deprecated
9090 , newPinned
91− , new
92− , pinnedNew
93− , writeN
94− , write
95− , writeAppendN
96− , writeAppend
9791 )
9892where
9993
core/src/Streamly/Data/MutArray/Generic.hs+3−8
@@ -23,11 +23,11 @@ module Streamly.Data.MutArray.Generic
2323 MutArray
2424
2525 -- * Construction
26− , emptyOf
26+ , new
2727 , fromListN
2828 , fromList
29− , createOf
30− , create
29+ , writeN
30+ , write
3131
3232 -- * Appending elements
3333 , snoc
@@ -56,11 +56,6 @@ module Streamly.Data.MutArray.Generic
5656
5757 -- * Size
5858 , length
59−
60− -- * Deprecated
61− , new
62− , writeN
63− , write
6459 )
6560where
6661
core/src/Streamly/Data/MutByteArray.hsdeleted−106
@@ -1,106 +0,0 @@
1−-- |
2−-- Module : Streamly.Data.MutByteArray
3−-- Copyright : (c) 2023 Composewell Technologies
4−-- License : BSD3-3-Clause
5−-- Maintainer : streamly@composewell.com
6−-- Portability : GHC
7−--
8−-- A low level byte Array type 'MutByteArray', along with type classes 'Unbox'
9−-- and 'Serialize' for fast binary serialization and deserialization of Haskell
10−-- values. Serialization, deserialization performance is similar to, and in
11−-- some cases many times better than the store package. Conceptually, the
12−-- 'Serialize' type class works in the same way as store.
13−--
14−-- Serialize instances are configurable to use constructor names (see
15−-- 'Streamly.Internal.Data.MutByteArray.encodeConstrNames'), record field names (see
16−-- 'Streamly.Internal.Data.MutByteArray.encodeRecordFields') instead of binary
17−-- encoded values. This is an experimental feature which allows JSON like
18−-- properties with faster speed. For example, you can change the order of
19−-- constructors or record fields without affecting serialized value.
20−--
21−-- Higher level unboxed array modules "Streamly.Data.Array" and
22−-- "Streamly.Data.MutArray" are built on top of this module. Unboxed arrays are
23−-- essentially serialized Haskell values. Array modules provide higher level
24−-- serialization routines like 'Streamly.Internal.Data.Array.pinnedSerialize'
25−-- and 'Streamly.Internal.Data.Array.deserialize' from the
26−-- "Streamly.Internal.Data.Array" module.
27−--
28−-- == Mutable Byte Array
29−--
30−-- 'MutByteArray' is a primitive mutable array in the IO monad. 'Unbox' and
31−-- 'Serialize' type classes use this primitive array to serialize data to and
32−-- deserialize it from. This array is used to build higher level unboxed
33−-- array types 'Streamly.Data.MutArray.MutArray' and 'Streamly.Data.Array.Array'.
34−--
35−-- == Using Unbox
36−--
37−-- The 'Unbox' type class is simple and used to serialize non-recursive fixed
38−-- size data types. This type class is primarily used to implement unboxed
39−-- arrays. Unboxed arrays are just a sequence of serialized fixed length
40−-- Haskell data types. Instances of this type class can be derived using
41−-- 'Generic' or template haskell based deriving functions provided in this
42−-- module.
43−--
44−-- Writing a data type to an array using the array creation routines in
45−-- "Streamly.Data.Array" or "Streamly.Data.MutArray" (e.g. @writeN@ or
46−-- @fromListN@), serializes the type to the array. Similarly, reading the data
47−-- type from the array deserializes it. You can also serialize and deserialize
48−-- directly to and from a 'MutByteArray', using the type class methods.
49−--
50−-- == Using Serialize
51−--
52−-- The 'Serialize' type class is a superset of the 'Unbox' type class, it can
53−-- serialize variable length data types as well e.g. Haskell lists. Use
54−-- 'deriveSerialize' to derive the instances of the type class automatically
55−-- and then use the type class methods to serialize and deserialize to and from
56−-- a 'MutByteArray'.
57−--
58−-- See 'Streamly.Internal.Data.Array.pinnedSerialize' and
59−-- 'Streamly.Internal.Data.Array.deserialize' for 'Array' type based
60−-- serialization.
61−--
62−-- == Comparing serialized values
63−--
64−-- When using the `Unbox` type class the same value may result in differing
65−-- serialized bytes because of unused uninitialized data in case of sum types.
66−-- Therefore, byte comparison of serialized values is not reliable.
67−--
68−-- However, the 'Serialize' type class guarantees that the serialized values
69−-- are always exactly the same and byte comparison of serialized is reliable.
70−--
71−module Streamly.Data.MutByteArray
72− (
73−
74− -- * Mutable Byte Array
75− -- | The standard way to read from or write to a 'MutByteArray' is by using
76− -- the 'Unbox' or 'Serialize' type class methods.
77− MutByteArray
78− , isPinned
79− , pin
80− , unpin
81− , new
82− , pinnedNew
83−
84− -- * Unbox
85− , Unbox(..)
86− , deriveUnbox
87−
88− -- * Serialize
89− , Serialize(..)
90−
91− -- ** Instance Config
92− , SerializeConfig
93− , inlineAddSizeTo
94− , inlineSerializeAt
95− , inlineDeserializeAt
96−
97− -- ** Instance Deriving
98− , deriveSerialize
99− , deriveSerializeWith
100− ) where
101−
102−--------------------------------------------------------------------------------
103−-- Imports
104−--------------------------------------------------------------------------------
105−
106−import Streamly.Internal.Data.MutByteArray
core/src/Streamly/Data/Parser.hs+60−130
@@ -7,137 +7,12 @@
77-- Stability : pre-release
88-- Portability : GHC
99--
10−-- Parsers are stream consumers like folds with the following differences:
10+-- Fast, composable stream consumers with ability to terminate, backtrack and
11+-- fail, supporting stream fusion. Parsers are a natural extension of
12+-- "Streamly.Data.Fold". Parsers and folds can be interconverted.
1113--
12−-- * folds cannot fail but parsers can fail and backtrack.
13−-- * folds can be composed as a Tee but parsers cannot.
14−-- * folds can be used for scanning but parsers cannot.
15−-- * folds can be converted to parsers.
16−--
17−-- This module implements parsers with stream fusion which compile to efficient
18−-- loops comparable to the speed of C.
19−--
20−-- == Using Parsers
21−--
22−-- This module provides elementary parsers and parser combinators that can be
23−-- used to parse a stream of data. Additionally, all the folds from the
24−-- "Streamly.Data.Fold" module can be converted to parsers using 'fromFold'.
25−-- All the parsing functionality provided by popular parsing libraries, and
26−-- more is available. Also see "Streamly.Unicode.Parser" module for Char stream
27−-- parsers.
28−--
29−-- A data stream can be transformed to a stream of parsed data elements. Parser
30−-- combinators can be used to create a pipeline of folds or parsers such that
31−-- the next fold or parser consumes the result of the previous parser. See
32−-- 'Streamly.Data.Stream.parse' and 'Streamly.Data.Stream.parseMany' to run
33−-- these parsers on a stream.
34−--
35−-- == Parser vs ParserK
36−--
37−-- There are two functionally equivalent parsing modules,
38−-- "Streamly.Data.Parser" (this module) and "Streamly.Data.ParserK". The latter
39−-- is a CPS based wrapper over the former, and can be used for parsing in
40−-- general. "Streamly.Data.Parser" enables stream fusion and should be
41−-- preferred over "Streamly.Data.ParserK" for high performance stream parsing
42−-- use cases. However, there are a few cases where this module is not
43−-- suitable and ParserK should be used instead.
44−--
45−-- For static fusion, parser combinators have to use strict pattern matching on
46−-- arguments of type Parser. This leads to infinte loop when a parser is
47−-- defined recursively, due to strict evaluation of the recursive call. For
48−-- example, the following implementation loops infinitely because of the
49−-- recursive use of parser @p@ in the @*>@ combinator:
50−--
51−-- >>> import Streamly.Data.Parser (Parser)
52−-- >>> import qualified Streamly.Data.Fold as Fold
53−-- >>> import qualified Streamly.Data.Parser as Parser
54−-- >>> import qualified Streamly.Data.Stream as Stream
55−-- >>> import Control.Applicative ((<|>))
56−--
57−-- >>> :{
58−-- >>> p :: Monad m => Parser Char m String
59−-- >>> p = Parser.satisfy (== '(') *> p <|> Parser.fromFold Fold.toList
60−-- >>> :}
61−--
62−-- Use ParserK when recursive use is required:
63−--
64−-- >>> import Streamly.Data.ParserK (ParserK)
65−-- >>> import qualified Streamly.Data.StreamK as StreamK
66−-- >>> import qualified Streamly.Internal.Data.StreamK as StreamK (parse)
67−-- >>> import qualified Streamly.Internal.Data.ParserK as ParserK (adapt)
68−--
69−-- >>> :{
70−-- >>> p :: Monad m => ParserK Char m String
71−-- >>> p = ParserK.adapt (Parser.satisfy (== '(')) *> p <|> ParserK.adapt (Parser.fromFold Fold.toList)
72−-- >>> :}
73−--
74−-- >>> StreamK.parse p $ StreamK.fromStream $ Stream.fromList "hello"
75−-- Right "hello"
76−--
77−-- For this reason Applicative, Alternative or Monad compositions with
78−-- recursion cannot be used with the 'Parser' type. Alternative type class based
79−-- operations like 'asum' and Alternative based generic parser combinators use
80−-- recursion. Similarly, Applicative type class based operations like
81−-- 'Prelude.sequence' use recursion. Custom implementations of many such
82−-- operations are provided in this module (e.g. 'some', 'many'), and those
83−-- should be used instead.
84−--
85−-- Another limitation of Parser type is due to the quadratic complexity causing
86−-- slowdown when too many nested compositions are used. Especially Applicative,
87−-- Monad, Alternative instances, and sequenced parsing operations (e.g. nested
88−-- 'one', and 'splitWith') degrade the performance quadratically (O(n^2)) when
89−-- combined @n@ times, roughly 8 or less sequenced parsers are fine. READ THE
90−-- DOCS OF APPLICATIVE, MONAD AND ALTERNATIVE INSTANCES.
91−--
92−-- == Streaming Parsers
93−--
94−-- With 'Streamly.Data.ParserK.ParserK' you can use the generic Alternative
95−-- type class based parsers from the
96−-- <https://hackage.haskell.org/package/parser-combinators parser-combinators>
97−-- library or similar. However, we recommend that you use the equivalent
98−-- functionality from this module for better performance and for streaming
99−-- behavior.
100−--
101−-- Firstly, the combinators in this module are faster due to stream fusion.
102−-- Secondly, these are streaming in nature as the results can be passed
103−-- directly to other stream consumers (folds or parsers). The Alternative type
104−-- class based parsers would end up buffering all the results in lists before
105−-- they can be consumed.
106−--
107−-- When recursion or heavy nesting is needed use ParserK.
108−--
109−-- == Error Reporting
110−--
111−-- These parsers do not report the error context (e.g. line number or column).
112−-- This may be supported in future.
113−--
114−-- == Monad Transformer Stack
115−--
116−-- 'MonadTrans' instance is not provided. If the 'Parser' type is the top most
117−-- layer (which should be the case almost always) you can just use 'fromEffect'
118−-- to execute the lower layer monad effects.
119−--
120−-- == Parser vs ParserK Implementation
121−--
122−-- The 'Parser' type represents a stream consumer by composing state as data
123−-- which enables stream fusion. Stream fusion generates a tight loop without
124−-- any constructor allocations between the stages, providing C like performance
125−-- for the loop. Stream fusion works when multiple functions are combined in a
126−-- pipeline statically. Therefore, the operations in this module must be
127−-- inlined and must not be used recursively to allow for stream fusion.
128−--
129−-- The 'ParserK' type represents a stream consumer by composing function calls,
130−-- therefore, a function call overhead is incurred at each composition. It is
131−-- quite fast in general but may be a few times slower than a fused parser.
132−-- However, it allows for scalable dynamic composition especially parsers can
133−-- be used in recursive calls. Using the 'ParserK' type operations like
134−-- 'splitWith' provide linear (O(n)) performance with respect to the number of
135−-- compositions.
136−--
137−-- == Experimental APIs
138−--
139−-- Please refer to "Streamly.Internal.Data.Parser" for functions that have not
140−-- yet been released.
14+-- Please refer to "Streamly.Internal.Data.Parser" for functions that have
15+-- not yet been released.
14116--
14217module Streamly.Data.Parser
14318 (
@@ -147,6 +22,9 @@ module Streamly.Data.Parser
14722 --
14823 -- $setup
14924
25+ -- * Overview
26+ -- $overview
27+
15028 -- * Parser Type
15129 Parser
15230
@@ -245,3 +123,55 @@ import Streamly.Internal.Data.Parser
245123import Prelude hiding (dropWhile, takeWhile, filter)
246124
247125#include "DocTestDataParser.hs"
126+
127+-- $overview
128+--
129+-- Several combinators in this module can be many times faster than CPS based
130+-- parsers because of stream fusion. For example,
131+-- 'Streamly.Internal.Data.Parser.many' combinator in this module is much
132+-- faster than the 'Control.Applicative.many' combinator of
133+-- 'Control.Applicative.Alternative' type class used by CPS based parsers.
134+--
135+-- The use of 'Alternative' type class, in parsers has another drawback.
136+-- Alternative based parsers use plain Haskell lists to collect the results. In
137+-- a strict Monad like IO, the results are necessarily buffered before they can
138+-- be consumed. This may not perform optimally in streaming applications
139+-- processing large amounts of data. Equivalent combinators in this module can
140+-- consume the results of parsing using a 'Fold' or another parser, thus
141+-- providing a scalable and composable consumer.
142+--
143+-- Note that these parsers do not report the error context (e.g. line number or
144+-- column). This may be supported in future.
145+--
146+-- mtl instances are not provided. If the 'Parser' type is the top most layer
147+-- (which should be the case almost always) you can just use 'fromEffect' to
148+-- execute the lower layer monad effects.
149+--
150+-- == Performance Notes
151+--
152+-- The 'Parser' type represents a stream consumer by composing state as data
153+-- which enables stream fusion. Stream fusion generates a tight loop without
154+-- any constructor allocations between the stages, providing C like performance
155+-- for the loop. Stream fusion works when multiple functions are combined in a
156+-- pipeline statically. Therefore, the operations in this module must be
157+-- inlined and must not be used recursively to allow for stream fusion. Note
158+-- that operations like 'sequence', and 'asum' that compose pasrers using
159+-- recursion should be avoided with these parsers. You can use these with the
160+-- 'ParserK' module instead.
161+--
162+-- Using the 'Parser' type, parsing operations like 'one', 'splitWith' etc.
163+-- degrade quadratically (O(n^2)) when combined many times. If you need to
164+-- combine these operations, say more than 8 times in a single loop, then you
165+-- should consider using the continuation style parser type 'ParserK' instead.
166+-- Also, if you need to use these operations in a recursive loop you should use
167+-- 'ParserK' instead.
168+--
169+-- The 'ParserK' type represents a stream consumer by composing function calls,
170+-- therefore, a function call overhead is incurred at each composition. It is
171+-- quite fast in general but may be a few times slower than a fused parser.
172+-- However, it allows for scalable dynamic composition especially parsers can
173+-- be used in recursive calls. Using the 'ParserK' type operations like
174+-- 'splitWith' provide linear (O(n)) performance with respect to the number of
175+-- compositions..
176+--
177+-- 'Parser' and 'ParserK' types can be interconverted.
core/src/Streamly/Data/ParserK.hs+10−105
@@ -1,4 +1,3 @@
1−{-# LANGUAGE CPP #-}
21-- |
32-- Module : Streamly.Data.ParserK
43-- Copyright : (c) 2023 Composewell Technologies
@@ -7,112 +6,26 @@
76-- Stability : pre-release
87-- Portability : GHC
98--
10−-- See the general notes about parsing in the "Streamly.Data.Parser" module.
11−-- This module implements a using Continuation Passing Style (CPS) wrapper over
12−-- the "Streamly.Data.Parser" module. It is as fast or faster than attoparsec.
9+-- Parsers using Continuation Passing Style (CPS). See notes in
10+-- "Streamly.Data.Parser" module to know when to use this module.
1311--
14−-- == Parser vs ParserK
15−--
16−-- 'ParserK' is preferred over 'Streamly.Data.Parser.Parser' when extensive
17−-- applicative, alternative and monadic composition is required, or when
18−-- recursive or dynamic composition of parsers is required. The
19−-- 'Streamly.Data.Parser.Parser' type fuses statically and creates efficient
20−-- loops whereas 'ParserK' uses function call based composition and has
21−-- comparatively larger runtime overhead but it is better suited to the
22−-- specific use cases mentioned above. 'ParserK' also allows to efficient parse
23−-- a stream of arrays, it can also break the input stream into a parse result
24−-- and remaining stream so that the stream can be parsed independently in
25−-- segments.
26−--
27−-- == Using ParserK
28−--
29−-- All the parsers from the "Streamly.Data.Parser" module can be adapted to
30−-- ParserK using the 'Streamly.Data.ParserK.adaptC',
31−-- 'Streamly.Internal.Data.ParserK.adapt', and
32−-- 'Streamly.Internal.Data.ParserK.adaptCG' combinators.
33−--
34−-- 'Streamly.Data.StreamK.parseChunks' runs a parser on a stream of unboxed
35−-- arrays, this is the preferred and most efficient way to parse chunked input.
36−-- The more general 'Streamly.Data.StreamK.parseBreakChunks' function returns
37−-- the remaining stream as well along with the parse result. There are
38−-- 'Streamly.Internal.Data.StreamK.parseChunksGeneric',
39−-- 'Streamly.Internal.Data.StreamK.parseBreakChunksGeneric' as well to run
40−-- parsers on boxed arrays. 'Streamly.Internal.Data.StreamK.parse',
41−-- 'Streamly.Internal.Data.StreamK.parseBreak' run parsers on a stream of
42−-- individual elements instead of stream of arrays.
43−--
44−-- == Monadic Composition
45−--
46−-- Monad composition can be used for lookbehind parsers, we can dynamically
47−-- compose new parsers based on the results of the previously parsed values.
48−--
49−-- If we have to parse "a9" or "9a" but not "99" or "aa" we can use the
50−-- following non-monadic, backtracking parser:
51−--
52−-- >>> digits p1 p2 = ((:) <$> p1 <*> ((:) <$> p2 <*> pure []))
53−-- >>> :{
54−-- backtracking :: Monad m => ParserK Char m String
55−-- backtracking = ParserK.adapt $
56−-- digits (Parser.satisfy isDigit) (Parser.satisfy isAlpha)
57−-- <|>
58−-- digits (Parser.satisfy isAlpha) (Parser.satisfy isDigit)
59−-- :}
60−--
61−-- We know that if the first parse resulted in a digit at the first place then
62−-- the second parse is going to fail. However, we waste that information and
63−-- parse the first character again in the second parse only to know that it is
64−-- not an alphabetic char. By using lookbehind in a 'Monad' composition we can
65−-- avoid redundant work:
66−--
67−-- >>> data DigitOrAlpha = Digit Char | Alpha Char
68−--
69−-- >>> :{
70−-- lookbehind :: Monad m => ParserK Char m String
71−-- lookbehind = do
72−-- x1 <- ParserK.adapt $
73−-- Digit <$> Parser.satisfy isDigit
74−-- <|> Alpha <$> Parser.satisfy isAlpha
75−-- -- Note: the parse depends on what we parsed already
76−-- x2 <- ParserK.adapt $
77−-- case x1 of
78−-- Digit _ -> Parser.satisfy isAlpha
79−-- Alpha _ -> Parser.satisfy isDigit
80−-- return $ case x1 of
81−-- Digit x -> [x,x2]
82−-- Alpha x -> [x,x2]
83−-- :}
84−--
85−-- == Experimental APIs
86−--
87−-- Please refer to "Streamly.Internal.Data.ParserK" for functions that have
88−-- not yet been released.
12+-- To run a 'ParserK' use 'Streamly.Data.StreamK.parseChunks'.
8913--
9014module Streamly.Data.ParserK
9115 (
92− -- * Setup
93− -- | To execute the code examples provided in this module in ghci, please
94− -- run the following commands first.
95− --
96− -- $setup
97−
9816 -- * Parser Type
9917 ParserK
10018
10119 -- * Parsers
10220 -- ** Conversions
103− , adapt
104− , adaptC
105− , adaptCG
21+ , fromFold
22+ , fromParser
10623 -- , toParser
10724
10825 -- ** Without Input
10926 , fromPure
11027 , fromEffect
11128 , die
112−
113− -- * Deprecated
114− , fromFold
115− , fromParser
11629 )
11730
11831where
@@ -120,20 +33,12 @@ where
12033import Control.Monad.IO.Class (MonadIO)
12134import Streamly.Internal.Data.Fold (Fold)
12235import Streamly.Internal.Data.Unbox (Unbox)
123−import Streamly.Internal.Data.Array (Array)
12436import qualified Streamly.Internal.Data.Parser as ParserD
12537
126−import Streamly.Internal.Data.ParserK.Type
127−
128−#include "DocTestDataParserK.hs"
38+import Streamly.Internal.Data.Parser.ParserK.Type
12939
130−{-# DEPRECATED fromFold "Please use \"ParserK.adaptC . Parser.fromFold\" instead." #-}
40+-- | Convert a 'Fold' to a 'ParserK'.
41+--
13142{-# INLINE fromFold #-}
132−fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK (Array a) m b
133−fromFold = adaptC . ParserD.fromFold
134−
135−{-# DEPRECATED fromParser "Please use \"adaptC\" instead." #-}
136−{-# INLINE fromParser #-}
137−fromParser ::
138− (MonadIO m, Unbox a) => ParserD.Parser a m b -> ParserK (Array a) m b
139−fromParser = adaptC
43+fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK a m b
44+fromFold = fromParser . ParserD.fromFold
core/src/Streamly/Data/Stream.hs+107−199
@@ -8,20 +8,15 @@
88-- Stability : released
99-- Portability : GHC
1010--
11−-- Streams represented as state machines, that fuse together when composed
12−-- statically, eliminating function calls or intermediate constructor
13−-- allocations - generating tight, efficient loops. Suitable for high
14−-- performance looping operations.
15−--
16−-- If you need to call these operations recursively in a loop (i.e. composed
17−-- dynamically) then it is recommended to use the continuation passing style
18−-- (CPS) stream operations from the "Streamly.Data.StreamK" module. 'Stream'
19−-- and 'StreamK' types are interconvertible. See more details in the
20−-- documentation below regarding 'Stream' vs 'StreamK'.
11+-- Fast, composable stream producers with ability to terminate, supporting
12+-- stream fusion.
2113--
2214-- Please refer to "Streamly.Internal.Data.Stream" for more functions that have
2315-- not yet been released.
2416--
17+-- For continuation passing style (CPS) stream type, please refer to
18+-- the "Streamly.Data.StreamK" module.
19+--
2520-- Checkout the <https://github.com/composewell/streamly-examples>
2621-- repository for many more real world examples of stream programming.
2722
@@ -42,6 +37,9 @@ module Streamly.Data.Stream
4237 -- * Construction
4338 -- | Functions ending in the general shape @b -> Stream m a@.
4439 --
40+ -- See also: "Streamly.Internal.Data.Stream.StreamD.Generate" for
41+ -- @Pre-release@ functions.
42+ --
4543 -- Useful Idioms:
4644 --
4745 -- >>> fromIndices f = fmap f $ Stream.enumerateFrom 0
@@ -51,12 +49,11 @@ module Streamly.Data.Stream
5149 -- >>> fromFoldableM = Stream.sequence . fromFoldable
5250
5351 -- ** Primitives
54− -- | A fused 'Stream' is never constructed using these primitives, they are
55− -- typically generated by converting containers like list into streams, or
56− -- generated using custom functions provided in this module. The 'cons'
57− -- primitive in this module has a rare use in fusing a small number of
58− -- elements. On the other hand, it is common to construct 'StreamK' stream
59− -- using the StreamK.'StreamK.cons' primitive.
52+ -- | A fused 'Stream' is never constructed using 'cons', we always convert
53+ -- other type of containers like list into streams, or generate it using
54+ -- custom functions provided in this module. The 'cons' primitive has a
55+ -- rare use in fusing a small number of elements. On the other hand, it is
56+ -- common to construct 'StreamK' stream using the 'StreamK.cons' primitive.
6057 , nil
6158 , nilM
6259 , cons
@@ -70,15 +67,6 @@ module Streamly.Data.Stream
7067
7168 -- ** From Values
7269 -- | Generate a monadic stream from a seed value or values.
73− --
74− -- Non-empty pure streams can be cycled using repeat and unfold:
75− --
76− -- >>> cycle = Stream.unfoldMany Unfold.fromList . Stream.repeat
77− --
78− -- Non-empty StreamK can be cycled using append:
79− --
80− -- >>> cycle xs = let ys = xs `StreamK.append` ys in ys
81− --
8270 , fromPure
8371 , fromEffect
8472 , repeat
@@ -123,6 +111,8 @@ module Streamly.Data.Stream
123111 -- | Functions ending in the general shape @Stream m a -> m b@ or @Stream m
124112 -- a -> m (b, Stream m a)@
125113 --
114+ -- See also: "Streamly.Internal.Data.Stream.StreamD.Eliminate" for @Pre-release@
115+ -- functions.
126116
127117-- EXPLANATION: In imperative terms a fold can be considered as a loop over the stream
128118-- that reduces the stream to a single value.
@@ -221,57 +211,21 @@ module Streamly.Data.Stream
221211 -- operations like mapping a function over the stream.
222212 , foldrM
223213 , foldr
224− -- foldr1
225214
226215 -- ** Specific Folds
227− -- | Streams are folded using folds in "Streamly.Data.Fold". Here are some
228− -- idioms and equivalents of Data.List APIs using folds:
216+ -- | Usually you can use the folds in "Streamly.Data.Fold". However, some
217+ -- folds that may be commonly used or may have an edge in performance in
218+ -- some cases are provided here.
219+ --
220+ -- Useful idioms:
229221 --
230222 -- >>> foldlM' f a = Stream.fold (Fold.foldlM' f a)
231− -- >>> foldl1' f = Stream.fold (Fold.foldl1' f)
223+ -- >>> foldl1 f = Stream.fold (Fold.foldl1' f)
232224 -- >>> foldl' f a = Stream.fold (Fold.foldl' f a)
233225 -- >>> drain = Stream.fold Fold.drain
234226 -- >>> mapM_ f = Stream.fold (Fold.drainMapM f)
235227 -- >>> length = Stream.fold Fold.length
236− -- >>> genericLength = Stream.fold Fold.lengthGeneric
237228 -- >>> head = Stream.fold Fold.one
238− -- >>> last = Stream.fold Fold.latest
239− -- >>> null = Stream.fold Fold.null
240− -- >>> and = Stream.fold Fold.and
241− -- >>> or = Stream.fold Fold.or
242− -- >>> any p = Stream.fold (Fold.any p)
243− -- >>> all p = Stream.fold (Fold.all p)
244− -- >>> sum = Stream.fold Fold.sum
245− -- >>> product = Stream.fold Fold.product
246− -- >>> maximum = Stream.fold Fold.maximum
247− -- >>> maximumBy cmp = Stream.fold (Fold.maximumBy cmp)
248− -- >>> minimum = Stream.fold Fold.minimum
249− -- >>> minimumBy cmp = Stream.fold (Fold.minimumBy cmp)
250− -- >>> elem x = Stream.fold (Fold.elem x)
251− -- >>> notElem x = Stream.fold (Fold.notElem x)
252− -- >>> lookup x = Stream.fold (Fold.lookup x)
253− -- >>> find p = Stream.fold (Fold.find p)
254− -- >>> (!?) i = Stream.fold (Fold.index i)
255− -- >>> genericIndex i = Stream.fold (Fold.indexGeneric i)
256− -- >>> elemIndex x = Stream.fold (Fold.elemIndex x)
257− -- >>> findIndex p = Stream.fold (Fold.findIndex p)
258− --
259− -- Some equivalents of Data.List APIs from the Stream module:
260− --
261− -- >>> head = fmap (fmap fst) . Stream.uncons
262− -- >>> tail = fmap (fmap snd) . Stream.uncons
263− -- >>> tail = Stream.tail -- unreleased API
264− -- >>> init = Stream.init -- unreleased API
265− --
266− -- A Stream based toList fold implementation is provided below because it
267− -- has a better performance compared to the fold.
268−
269− -- Functions in Data.List, missing here:
270− -- unsnoc = Stream.parseBreak (Parser.init Fold.toList)
271− -- genericTake
272− -- genericDrop
273− -- genericSplitAt
274− -- genericReplicate
275229 , toList
276230
277231 -- * Mapping
@@ -299,6 +253,8 @@ module Streamly.Data.Stream
299253 -- * Scanning
300254 -- | Stateful one-to-one transformations.
301255 --
256+ -- See also: "Streamly.Internal.Data.Stream.StreamD.Transform" for
257+ -- @Pre-release@ functions.
302258
303259 {-
304260 -- ** Left scans
@@ -325,15 +281,6 @@ module Streamly.Data.Stream
325281 , scan
326282 , postscan
327283 -- XXX postscan1 can be implemented using Monoids or Refolds.
328− -- The following scans from Data.List are not provided.
329− -- XXX scanl
330− -- XXX scanl1
331− -- XXX scanr
332− -- XXX scanr1
333− -- XXX mapAccumL
334− -- XXX mapAccumR
335− -- XXX inits
336− -- XXX tails
337284
338285 -- ** Specific scans
339286 -- Indexing can be considered as a special type of zipping where we zip a
@@ -342,8 +289,6 @@ module Streamly.Data.Stream
342289
343290 -- * Insertion
344291 -- | Add elements to the stream.
345− --
346− -- >>> insert = Stream.insertBy compare
347292
348293 -- Inserting elements is a special case of interleaving/merging streams.
349294 , insertBy
@@ -379,15 +324,12 @@ module Streamly.Data.Stream
379324 -- "Streamly.Internal.Data.Fold" can be used along with 'scanMaybe' to
380325 -- perform stateful filtering operations in general.
381326 --
382− -- Idioms and equivalents of Data.List APIs:
327+ -- Useful idioms:
383328 --
384329 -- >>> deleteBy cmp x = Stream.scanMaybe (Fold.deleteBy cmp x)
385− -- >>> deleteBy = Stream.deleteBy -- unreleased API
386− -- >>> delete = deleteBy (==)
387330 -- >>> findIndices p = Stream.scanMaybe (Fold.findIndices p)
388331 -- >>> elemIndices a = findIndices (== a)
389332 -- >>> uniq = Stream.scanMaybe (Fold.uniqBy (==))
390− -- >>> partition p = Stream.fold (Fold.partition Fold.toList Fold.toList) . fmap (if p then Left else Right)
391333 , scanMaybe
392334 , take
393335 , takeWhile
@@ -395,8 +337,6 @@ module Streamly.Data.Stream
395337 , drop
396338 , dropWhile
397339 , dropWhileM
398− -- XXX write to an array in reverse and then read in reverse
399− -- > dropWhileEnd = reverse . dropWhile p . reverse
400340
401341 -- XXX These are available as scans in folds. We need to check the
402342 -- performance though. If these are common and we need convenient stream
@@ -418,42 +358,34 @@ module Streamly.Data.Stream
418358 -- , elemIndices
419359
420360 -- * Combining Two Streams
421− -- | Note that these operations are suitable for statically fusing a few
422− -- streams, they have a quadratic O(n^2) time complexity wrt to the number
423− -- of streams. If you want to compose many streams dynamically using binary
424− -- combining operations see the corresponding operations in
425− -- "Streamly.Data.StreamK".
426− --
427− -- When fusing more than two streams it is more efficient if the binary
428− -- operations are composed as a balanced tree rather than a right
429− -- associative or left associative one e.g.:
430− --
431− -- >>> s1 = Stream.fromList [1,2] `Stream.append` Stream.fromList [3,4]
432− -- >>> s2 = Stream.fromList [4,5] `Stream.append` Stream.fromList [6,7]
433− -- >>> s = s1 `Stream.append` s2
434−
435361 -- ** Appending
436− -- | Equivalent of Data.List append:
437− --
438− -- >>> (++) = Stream.append
439362 , append
440363
441364 -- ** Interleaving
365+ -- | When interleaving more than two streams you may want to interleave
366+ -- them pairwise creating a balanced binary merge tree.
442367 , interleave
443368
444369 -- ** Merging
370+ -- | When merging more than two streams you may want to merging them
371+ -- pairwise creating a balanced binary merge tree.
372+ --
373+ -- Merging of @n@ streams can be performed by combining the streams pair
374+ -- wise using 'mergeMapWith' to give O(n * log n) time complexity. If used
375+ -- with 'concatMapWith' it will have O(n^2) performance.
376+
445377 , mergeBy
446378 , mergeByM
447379
448380 -- ** Zipping
449− -- | Idioms and equivalents of Data.List APIs:
381+ -- | When zipping more than two streams you may want to zip them
382+ -- pairwise creating a balanced binary tree.
450383 --
451− -- >>> zip = Stream.zipWith (,)
452− -- >>> unzip = Stream.fold (Fold.unzip Fold.toList Fold.toList)
384+ -- Zipping of @n@ streams can be performed by combining the streams pair
385+ -- wise using 'mergeMapWith' with O(n * log n) time complexity. If used
386+ -- with 'concatMapWith' it will have O(n^2) performance.
453387 , zipWith
454388 , zipWithM
455− -- XXX zipWith3,4,5,6,7
456− -- XXX unzip3,4,5,6,7
457389 -- , ZipStream (..)
458390
459391 -- ** Cross Product
@@ -486,89 +418,48 @@ module Streamly.Data.Stream
486418 -- One dimension loops are just a special case of nested loops. For
487419 -- example, 'concatMap' can degenerate to a simple map operation:
488420 --
489− -- >>> map f m = Stream.concatMap (\x -> Stream.fromPure (f x)) m
421+ -- > map f m = S.concatMap (\x -> S.fromPure (f x)) m
490422 --
491423 -- Similarly, 'concatMap' can perform filtering by mapping an element to a
492424 -- 'nil' stream:
493425 --
494− -- >>> filter p m = Stream.concatMap (\x -> if p x then Stream.fromPure x else Stream.nil) m
426+ -- > filter p m = S.concatMap (\x -> if p x then S.fromPure x else S.nil) m
495427 --
496− -- Idioms and equivalents of Data.List APIs:
497− --
498− -- >>> concat = Stream.concatMap id
499− -- >>> unlines = Stream.interposeSuffix '\n'
500− -- >>> unlines = Stream.intercalateSuffix Unfold.fromList "\n"
501− -- >>> unwords = Stream.interpose ' '
502− -- >>> unwords = Stream.intercalate Unfold.fromList " "
503428
504429 , concatEffect
505430 , concatMap
506431 , concatMapM
507432
508433 -- * Repeated Fold
509− -- | Idioms and equivalents of Data.List APIs:
434+ -- | Useful idioms:
510435 --
511− -- >>> groupsOf n = Stream.foldMany (Fold.take n Fold.toList)
512− -- >>> groupBy eq = Stream.groupsWhile eq Fold.toList
513− -- >>> groupBy eq = Stream.parseMany (Parser.groupBy eq Fold.toList)
514− -- >>> groupsByRolling eq = Stream.parseMany (Parser.groupByRolling eq Fold.toList)
515− -- >>> groups = groupBy (==)
436+ -- >>> splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
437+ -- >>> splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
438+ -- >>> groupsBy eq f = Stream.parseMany (Parser.groupBy eq f)
439+ -- >>> groupsByRolling eq f = Stream.parseMany (Parser.groupByRolling eq f)
440+ -- >>> groupsOf n f = Stream.foldMany (Fold.take n f)
516441 , foldMany -- XXX Rename to foldRepeat
517442 , groupsOf
518443 , parseMany
519444
520445 -- * Splitting
521− -- | Idioms and equivalents of Data.List APIs:
522− --
523− -- >>> splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
524− -- >>> splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
525− -- >>> lines = splitOnSuffix (== '\n')
526− -- >>> words = Stream.wordsBy isSpace
527− -- >>> splitAt n = Stream.fold (Fold.splitAt n Fold.toList Fold.toList)
528− -- >>> span p = Parser.splitWith (,) (Parser.takeWhile p Fold.toList) (Parser.fromFold Fold.toList)
529− -- >>> break p = span (not . p)
530446 , splitOn
531447 , wordsBy
532448
533449 -- * Buffered Operations
534450 -- | Operations that require buffering of the stream.
535451 -- Reverse is essentially a left fold followed by an unfold.
536− --
537− -- Idioms and equivalents of Data.List APIs:
538− --
539− -- >>> nub = Stream.fold Fold.toList . Stream.scanMaybe Fold.nub
540− -- >>> nub = Stream.ordNub -- unreleased API
541− -- >>> sortBy = StreamK.sortBy
542− -- >>> sortOn f = StreamK.sortOn -- unreleased API
543− -- >>> deleteFirstsBy = Stream.deleteFirstsBy -- unreleased
544− -- >>> (\\) = Stream.deleteFirstsBy (==) -- unreleased
545− -- >>> intersectBy = Stream.intersectBy -- unreleased
546− -- >>> intersect = Stream.intersectBy (==) -- unreleased
547− -- >>> unionBy = Stream.unionBy -- unreleased
548− -- >>> union = Stream.unionBy (==) -- unreleased
549− --
550452 , reverse
551− -- XXX transpose: write the streams to arrays and then stream transposed.
552− -- XXX subsequences
553− -- XXX permutations
554− -- , nub
555− -- , ordNub
556− -- , nubBy
557453
558454 -- * Multi-Stream folds
559455 -- | Operations that consume multiple streams at the same time.
560456 , eqBy
561457 , cmpBy
562458 , isPrefixOf
563− , isInfixOf
564− -- , isSuffixOf
565− -- , isSuffixOfUnbox
566459 , isSubsequenceOf
567460
568461 -- trimming sequences
569− -- , stripPrefix
570− -- , stripSuffix
571− -- , stripSuffixUnbox
462+ , stripPrefix
572463
573464 -- Exceptions and resource management depend on the "exceptions" package
574465 -- XXX We can have IO Stream operations not depending on "exceptions"
@@ -589,6 +480,9 @@ module Streamly.Data.Stream
589480 -- For example, instead of calling them on a stream of chars call them on a
590481 -- stream of arrays before flattening it to a stream of chars.
591482 --
483+ -- See also: "Streamly.Internal.Data.Stream.StreamD.Exception" for
484+ -- @Pre-release@ functions.
485+
592486 , onException
593487 , handle
594488
@@ -622,20 +516,25 @@ module Streamly.Data.Stream
622516 , runReaderT
623517 , runStateT
624518
625− -- XXX Arrays could be different types, therefore, this should be in
626− -- specific array module. Or maybe we should abstract over array types.
627519 -- * Stream of Arrays
628520 , Array.chunksOf
521+ , chunksSplitOn
629522 )
630523where
631524
525+import Control.Monad.IO.Class (MonadIO)
526+import Data.Word (Word8)
527+import Streamly.Data.Array (Array)
632528import Streamly.Internal.Data.Stream
633529import Prelude
634530 hiding (filter, drop, dropWhile, take, takeWhile, zipWith, foldr,
635− mapM, sequence, reverse, iterate, foldr1, repeat, replicate,
636− concatMap)
531+ foldl, map, mapM, mapM_, sequence, all, any, sum, product, elem,
532+ notElem, maximum, minimum, head, last, tail, length, null,
533+ reverse, iterate, init, and, or, lookup, foldr1, (!!),
534+ scanl, scanl1, repeat, replicate, concatMap, span)
637535
638536import qualified Streamly.Internal.Data.Array.Type as Array
537+import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
639538
640539#include "DocTestDataStream.hs"
641540
@@ -674,11 +573,11 @@ import qualified Streamly.Internal.Data.Array.Type as Array
674573--
675574-- == Stream Fusion
676575--
677−-- The fused 'Stream' type in this module employs stream fusion for C-like
678−-- performance when looping over data. It represents the stream as a state
679−-- machine using an explicit state, and a step function working on the state. A
680−-- typical stream operation consumes elements from the previous state machine
681−-- in a stream pipeline, transforms the elements and yields new values for the
576+-- The fused 'Stream' type employs stream fusion for C-like performance when
577+-- looping over data. It represents a stream source or transformation by
578+-- defining a state machine with explicit state, and a step function working on
579+-- the state. A typical stream operation consumes elements from the previous
580+-- state machine in the pipeline, transforms them and yields new values for the
682581-- next stage to consume. The stream operations are modular and represent a
683582-- single task, they have no knowledge of previous or next operation on the
684583-- elements.
@@ -687,7 +586,7 @@ import qualified Streamly.Internal.Data.Array.Type as Array
687586-- transformation operations and a stream consumer. All these operations taken
688587-- together form a closed loop processing the stream elements. Elements are
689588-- transferred between stages using a boxed data constructor. However, all the
690−-- stages of the pipeline are fused together by GHC, eliminating the boxing of
589+-- stages of the pipeline are fused together by GHC, eliminating the
691590-- intermediate constructors, and thus forming a tight C like loop without any
692591-- boxed data being used in the loop.
693592--
@@ -702,45 +601,54 @@ import qualified Streamly.Internal.Data.Array.Type as Array
702601--
703602-- == Stream vs StreamK
704603--
705−-- The fused stream model avoids constructor allocations and function call
706−-- overheads. However, the stream is represented as a state machine, and to
707−-- generate stream elements it has to navigate the decision tree of the state
708−-- machine. Moreover, the state machine is cranked for each element in the
709−-- stream. This performs extremely well when the number of states are limited.
710−-- The state machine starts getting expensive as the number of states increase.
711−-- For example, generating a stream from a list requires a single state and is
712−-- very efficient, even if it has millions of elements. However, using 'cons'
713−-- to construct a million element stream would be a disaster.
604+-- The fused stream model avoids constructor allocations or function call
605+-- overheads. However, the stream is represented as a state machine and to
606+-- generate elements it has to navigate the decision tree of the state machine.
607+-- Moreover, the state machine is cranked for each element in the stream. This
608+-- performs extremely well when the number of states are limited. The state
609+-- machine starts getting expensive as the number of states increase. For
610+-- example, generating a million element stream from a list requires a single
611+-- state and is very efficient. However, using fused 'cons' to generate a
612+-- million element stream would be a disaster.
714613--
715614-- A typical worst case scenario for fused stream model is a large number of
716615-- `cons` or `append` operations. A few static `cons` or `append` operations
717−-- are very fast and much faster than a CPS style stream because CPS involves a
718−-- function call for each element whereas fused stream involves a few
719−-- conditional branches in the state machine. However, constructing a large
720−-- stream using `cons` introduces as many states in the state machine as the
721−-- number of elements. If we compose `cons` as a balanced binary tree it will
722−-- take @n * log n@ time to navigate the tree, and @n * n@ if it is a right
723−-- associative composition.
724−--
616+-- are very fast and much faster than a CPS style stream. However, if we
617+-- construct a large stream using `cons` it introduces as many states in the
618+-- state machine as the number of elements. If we compose the `cons` as a
619+-- binary tree it will take @n * log n@ time to navigate the tree, and @n * n@
620+-- if it is a right associative composition.
621+--
622+-- For quadratic cases of fused stream, after a certain threshold the CPS
623+-- stream would perform much better and exhibit linear performance behavior.
725624-- Operations like 'cons' or 'append'; are typically recursively called to
726625-- construct a lazy infinite stream. For such use cases the CPS style 'StreamK'
727−-- should be used. CPS streams do not have a state machine that needs to be
626+-- type is provided. CPS streams do not have a state machine that needs to be
728627-- cranked for each element, past state has no effect on the future element
729−-- processing. However, CPS incurs a function call overhead for each element
730−-- processed, the overhead could be large compared to a fused state machine
731−-- even if it has many states. However, because of its linear performance
732−-- characterstics, after a certain threshold of stream compositions the CPS
733−-- stream would perform much better than the quadratic fused stream operations.
734−--
735−-- As a general guideline, you need to use 'StreamK' when you have to use
736−-- 'cons', 'append' or other operations having quadratic complexity at a large
737−-- scale. Typically, in such cases you need to compose the stream recursively,
738−-- by calling an operation in a loop. The decision to compose the stream is
739−-- taken at run time rather than statically at compile time.
740−--
741−-- Typically you would compose a 'StreamK' of chunks of data so that the
742−-- StreamK overhead is not high, and then process the chunks using 'Stream' by
743−-- using statically fused stream pipeline operations on the chunks.
628+-- processing. However, it incurs a function call overhead for each operation
629+-- for each element, which could be very large overhead compared to fused state
630+-- machines even if it has many states and cranks it for each element. But in
631+-- some cases scales tip in favor of the CPS stream. In those cases even though
632+-- CPS has a large constant overhead, it has a linear performance rather than
633+-- quadratic.
634+--
635+-- As a general guideline, if you have to use 'cons' or 'append' or operations
636+-- of similar nature, at a large scale, then 'StreamK' should be used. When you
637+-- need to compose the stream dynamically or recursively, then 'StreamK' should
638+-- be used. Typically you would use a dynamically generated 'StreamK' with
639+-- chunks of data which can then be processed by statically fused stream
640+-- pipeline operations.
744641--
745642-- 'Stream' and 'StreamK' types can be interconverted. See
746643-- "Streamly.Data.StreamK" module for conversion operations.
644+
645+-- | Split a stream of arrays on a given separator byte, dropping the separator
646+-- and coalescing all the arrays between two separators into a single array.
647+--
648+{-# INLINE chunksSplitOn #-}
649+chunksSplitOn
650+ :: (MonadIO m)
651+ => Word8
652+ -> Stream m (Array Word8)
653+ -> Stream m (Array Word8)
654+chunksSplitOn = ArrayStream.splitOn
core/src/Streamly/Data/StreamK.hs+19−37
@@ -8,30 +8,18 @@
88-- Stability : released
99-- Portability : GHC
1010--
11−-- Streams represented as chains of functions calls using Continuation Passing
12−-- Style (CPS), suitable for dynamically composing potentially large number of
13−-- streams.
14−--
15−-- Unlike the statically fused operations in "Streamly.Data.Stream", StreamK
16−-- operations are less efficient, involving a function call overhead for each
17−-- element, but they exhibit linear O(n) time complexity wrt to the number of
18−-- stream compositions. Therefore, they are suitable for dynamically composing
19−-- streams e.g. appending potentially infinite streams in recursive loops.
20−-- While fused streams can be used to efficiently process elements as small as
21−-- a single byte, CPS streams are typically used on bigger chunks of data to
22−-- avoid the larger overhead per element. For more details See the @Stream vs
23−-- StreamK@ section in the "Streamly.Data.Stream" module.
24−--
25−-- In addition to the combinators in this module, you can use operations from
26−-- "Streamly.Data.Stream" for StreamK as well by converting StreamK to Stream
27−-- ('toStream'), and vice-versa ('fromStream'). Please refer to
28−-- "Streamly.Internal.Data.StreamK" for more functions that have not yet been
29−-- released.
30−--
31−-- For documentation see the corresponding combinators in
32−-- "Streamly.Data.Stream". Documentation has been omitted in this module unless
33−-- there is a difference worth mentioning or if the combinator does not exist
34−-- in "Streamly.Data.Stream".
11+-- Streams using Continuation Passing Style (CPS). See the @Stream vs StreamK@
12+-- section in the "Streamly.Data.Stream" module to know when to use this
13+-- module.
14+--
15+-- Please refer to "Streamly.Internal.Data.StreamK" for more functions
16+-- that have not yet been released.
17+--
18+-- Some useful idioms:
19+--
20+-- >>> concatFoldableWith f = Prelude.foldr f StreamK.nil
21+-- >>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
22+-- >>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
3523
3624-- Notes:
3725--
@@ -95,8 +83,7 @@ module Streamly.Data.StreamK
9583 -- , foldBreak
9684
9785 -- ** Parsing
98− , parse
99− , parseBreak
86+ -- , parseBreak
10087 , parseBreakChunks
10188 , parseChunks
10289
@@ -106,11 +93,6 @@ module Streamly.Data.StreamK
10693 , take
10794
10895 -- * Combining Two Streams
109− -- | Unlike the operations in "Streamly.Data.Stream", these operations can
110− -- be used to dynamically compose large number of streams e.g. using the
111− -- 'concatMapWith' and 'mergeMapWith' operations. They have a linear O(n)
112− -- time complexity wrt to the number of streams being composed.
113−
11496 -- ** Appending
11597 , append
11698
@@ -134,12 +116,6 @@ module Streamly.Data.StreamK
134116 -- , CrossStreamK (..)
135117
136118 -- * Stream of streams
137− -- | Some useful idioms:
138− --
139− -- >>> concatFoldableWith f = Prelude.foldr f StreamK.nil
140− -- >>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
141− -- >>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
142− --
143119 , concatEffect
144120 -- , concatMap
145121 , concatMapWith
@@ -192,3 +168,9 @@ import Prelude hiding (reverse, zipWith, mapM, dropWhile, take)
192168-- 1
193169-- 2
194170-- [1,2]
171+--
172+-- == Exception Handling
173+--
174+-- There are no native exception handling operations in the StreamK module,
175+-- please convert to 'Stream' type and use exception handling operations from
176+-- "Streamly.Data.Stream".
core/src/Streamly/Data/Unbox.hsnew file+16
@@ -0,0 +1,16 @@
1+-- |
2+-- Module : Streamly.Data.Unbox
3+-- Copyright : (c) 2023 Composewell Technologies
4+-- License : BSD3-3-Clause
5+-- Maintainer : streamly@composewell.com
6+-- Portability : GHC
7+--
8+module Streamly.Data.Unbox
9+ ( Unbox(..)
10+ , MutableByteArray
11+ , deriveUnbox
12+ , deriveUnboxWith
13+ ) where
14+
15+import Streamly.Internal.Data.Unbox
16+import Streamly.Internal.Data.Unbox.TH
core/src/Streamly/Data/Unfold.hs+1−3
@@ -39,7 +39,7 @@ module Streamly.Data.Unfold
3939
4040 -- * Unfolds
4141 -- One to one correspondence with
42− -- "Streamly.Internal.Data.Stream.Generate"
42+ -- "Streamly.Internal.Data.Stream.StreamD.Generate"
4343
4444 -- ** Basic Constructors
4545 , unfoldrM
@@ -65,8 +65,6 @@ module Streamly.Data.Unfold
6565 -- ** Mapping on Input
6666 , lmap
6767 , lmapM
68− , first
69− , second
7068
7169 -- ** Mapping on Output
7270 , mapM
core/src/Streamly/FileSystem/Handle.hs+29−15
@@ -1,4 +1,4 @@
1−{-# LANGUAGE CPP #-}
1+#include "inline.hs"
22
33-- |
44-- Module : Streamly.FileSystem.Handle
@@ -40,12 +40,6 @@
4040--
4141module Streamly.FileSystem.Handle
4242 (
43− -- * Setup
44− -- | To execute the code examples provided in this module in ghci, please
45− -- run the following commands first.
46− --
47− -- $setup
48−
4943 -- * Singleton IO
5044 -- | Read or write a single buffer.
5145 getChunk
@@ -81,13 +75,13 @@ module Streamly.FileSystem.Handle
8175 -- position of the file handle. The stream ends as soon as EOF is
8276 -- encountered.
8377
84− -- *** Streams
85− , read
86− , readWith
87− , readChunks
88− , readChunksWith
78+ -- -- *** Streams
79+ -- , read
80+ -- , readWith
81+ -- , readChunks
82+ -- , readChunksWith
8983
90− -- *** Unfolds
84+ -- -- *** Unfolds
9185 , reader
9286 , readerWith
9387 , chunkReader
@@ -104,14 +98,34 @@ module Streamly.FileSystem.Handle
10498 , writeChunks
10599
106100 -- * Deprecated
101+ , read
107102 , readWithBufferOf
103+ , readChunks
108104 , readChunksWithBufferOf
109105 , writeChunksWithBufferOf
110106 , writeWithBufferOf
111107 )
112108where
113109
114−import Streamly.Internal.FileSystem.Handle
110+import Control.Monad.IO.Class (MonadIO(..))
111+import Data.Word (Word8)
112+import Streamly.Internal.Data.Array.Type (Array)
113+import Streamly.Internal.Data.Unfold.Type (Unfold)
114+import System.IO (Handle)
115+
116+import Streamly.Internal.FileSystem.Handle hiding (read, readChunks)
115117import Prelude hiding (read)
116118
117−#include "DocTestFileSystemHandle.hs"
119+-- | Same as 'reader'
120+--
121+{-# DEPRECATED read "Please use 'reader' instead" #-}
122+{-# INLINE read #-}
123+read :: MonadIO m => Unfold m Handle Word8
124+read = reader
125+
126+-- | Same as 'chunkReader'
127+--
128+{-# DEPRECATED readChunks "Please use 'chunkReader' instead" #-}
129+{-# INLINE readChunks #-}
130+readChunks :: MonadIO m => Unfold m Handle (Array Word8)
131+readChunks = chunkReader
core/src/Streamly/Internal/Data/Array.hs+163−610
@@ -2,6 +2,7 @@
22-- |
33-- Module : Streamly.Internal.Data.Array
44-- Copyright : (c) 2019 Composewell Technologies
5+--
56-- License : BSD3
67-- Maintainer : streamly@composewell.com
78-- Stability : experimental
@@ -16,33 +17,59 @@ module Streamly.Internal.Data.Array
1617 -- $design
1718
1819 -- * The Array Type
19− module Streamly.Internal.Data.Array.Type
20+ Array
2021
2122 -- * Construction
23+
24+ -- Pure List APIs
25+ , A.fromListN
26+ , A.fromList
27+
28+ -- Stream Folds
29+ , fromStreamN
30+ , fromStream
31+
2232 -- Monadic Folds
33+ , A.writeN -- drop new
34+ , A.pinnedWriteNAligned
35+ , A.write -- full buffer
2336 , writeLastN
2437
38+ -- * Elimination
39+ -- ** Conversion
40+ , A.toList
41+
42+ -- ** Streams
43+ , A.read
44+ , A.readRev
45+
46+ -- ** Unfolds
47+ , reader
48+ , readerUnsafe
49+ , A.readerRev
50+ , producer -- experimental
51+
2552 -- * Random Access
2653 -- , (!!)
2754 , getIndex
55+ , A.unsafeIndex -- XXX Rename to getIndexUnsafe??
2856 , getIndexRev
29− , last -- XXX getLastIndex?
57+ , last -- XXX getIndexLast?
58+ , getIndices
59+ , getIndicesFromThenTo
3060 -- , getIndicesFrom -- read from a given position to the end of file
3161 -- , getIndicesUpto -- read from beginning up to the given position
3262 -- , getIndicesFromTo
3363 -- , getIndicesFromRev -- read from a given position to the beginning of file
3464 -- , getIndicesUptoRev -- read from end to the given position in file
35− , indexReader
36− , indexReaderFromThenTo
3765
3866 -- * Size
67+ , length
3968 , null
4069
4170 -- * Search
4271 , binarySearch
4372 , findIndicesOf
44− -- getIndicesOf
45− , indexFinder -- see splitOn
4673 -- , findIndexOf
4774 -- , find
4875
@@ -50,96 +77,66 @@ module Streamly.Internal.Data.Array
5077 , cast
5178 , asBytes
5279 , castUnsafe
80+ , asPtrUnsafe
5381 , asCStringUnsafe
82+ , A.unsafeFreeze -- asImmutableUnsafe?
83+ , A.unsafeThaw -- asMutableUnsafe?
5484
5585 -- * Subarrays
5686 , getSliceUnsafe
5787 -- , getSlice
58− , sliceIndexerFromLen
59− , slicerFromLen
88+ , genSlicesFromLen
89+ , getSlicesFromLen
6090 , splitOn
6191
6292 -- * Streaming Operations
6393 , streamTransform
6494
65− -- * Folding
95+ -- ** Folding
6696 , streamFold
6797 , fold
6898
69− -- * Stream of Arrays
70−
71− -- XXX these are probably not very useful to have in this module as we can
72− -- express these idiomatically using streams.
73− , interpose
74− , interposeSuffix
75− , intercalateSuffix
76−
77− , compactLE
78− , pinnedCompactLE
79− , compactOnByte
80− , compactOnByteSuffix
81−
82− , foldBreakChunks
83− , foldChunks
84− , foldBreakChunksK
85− , parseBreakChunksK
86−
87− -- * Serialization
88− , encodeAs
89− , serialize
90− , pinnedSerialize
91− , deserialize
92−
9399 -- * Deprecated
94− , genSlicesFromLen
95− , getSlicesFromLen
96− , getIndices
100+ , A.toStream
101+ , A.toStreamRev
97102 )
98103where
99104
100−#include "assert.hs"
101105#include "inline.hs"
102106#include "ArrayMacros.h"
103107
108+import Control.Exception (assert)
109+import Control.Monad (when)
104110import Control.Monad.IO.Class (MonadIO(..))
105−-- import Data.Bifunctor (first)
106−-- import Data.Either (fromRight)
107111import Data.Functor.Identity (Identity)
108112import Data.Proxy (Proxy(..))
109113import Data.Word (Word8)
110114import Foreign.C.String (CString)
111115import Foreign.Ptr (castPtr)
112116import Foreign.Storable (Storable)
113−import GHC.Types (SPEC(..))
114117import Streamly.Internal.Data.Unbox (Unbox(..))
115118import Prelude hiding (length, null, last, map, (!!), read, concat)
116119
117−import Streamly.Internal.Data.MutByteArray.Type (PinnedState(..), MutByteArray)
118−import Streamly.Internal.Data.Serialize.Type (Serialize)
120+import Streamly.Internal.Data.MutArray.Type (ArrayUnsafe(..))
121+import Streamly.Internal.Data.Array.Type
122+ (Array(..), length, asPtrUnsafe)
119123import Streamly.Internal.Data.Fold.Type (Fold(..))
120−import Streamly.Internal.Data.Parser (Parser(..), Initial(..), ParseError(..))
121−import Streamly.Internal.Data.Stream (Stream(..))
122−import Streamly.Internal.Data.StreamK (StreamK)
123−import Streamly.Internal.Data.SVar.Type (adaptState, defState)
124−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3Fused'(..))
124+import Streamly.Internal.Data.Producer.Type (Producer(..))
125+import Streamly.Internal.Data.Stream (Stream)
126+import Streamly.Internal.Data.Tuple.Strict (Tuple3Fused'(..))
125127import Streamly.Internal.Data.Unfold.Type (Unfold(..))
126128import Streamly.Internal.System.IO (unsafeInlineIO)
127129
128−import qualified Streamly.Internal.Data.Fold.Type as Fold
129−import qualified Streamly.Internal.Data.Serialize.Type as Serialize
130−import qualified Streamly.Internal.Data.MutByteArray.Type as MBA
130+import qualified Streamly.Internal.Data.MutArray.Type as MA
131131import qualified Streamly.Internal.Data.MutArray as MA
132+import qualified Streamly.Internal.Data.Array.Type as A
132133import qualified Streamly.Internal.Data.Fold as FL
133−import qualified Streamly.Internal.Data.Ring as RB
134−import qualified Streamly.Internal.Data.Parser as Parser
135−-- import qualified Streamly.Internal.Data.ParserK as ParserK
134+import qualified Streamly.Internal.Data.Producer.Type as Producer
135+import qualified Streamly.Internal.Data.Producer as Producer
136+import qualified Streamly.Internal.Data.Ring.Unboxed as RB
136137import qualified Streamly.Internal.Data.Stream as D
137138import qualified Streamly.Internal.Data.Stream as Stream
138−import qualified Streamly.Internal.Data.StreamK as StreamK
139139import qualified Streamly.Internal.Data.Unfold as Unfold
140−import qualified Prelude
141−
142−import Streamly.Internal.Data.Array.Type
143140
144141#include "DocTestDataArray.hs"
145142
@@ -166,18 +163,91 @@ import Streamly.Internal.Data.Array.Type
166163-- if we use one array for each chunk. This is still significant to add
167164-- pressure to GC.
168165
166+-------------------------------------------------------------------------------
167+-- Construction
168+-------------------------------------------------------------------------------
169+
170+-- | Create an 'Array' from the first N elements of a stream. The array is
171+-- allocated to size N, if the stream terminates before N elements then the
172+-- array may hold less than N elements.
173+--
174+-- /Pre-release/
175+{-# INLINE fromStreamN #-}
176+fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
177+fromStreamN n m = do
178+ when (n < 0) $ error "writeN: negative write count specified"
179+ A.fromStreamDN n m
180+
181+-- | Create an 'Array' from a stream. This is useful when we want to create a
182+-- single array from a stream of unknown size. 'writeN' is at least twice
183+-- as efficient when the size is already known.
184+--
185+-- Note that if the input stream is too large memory allocation for the array
186+-- may fail. When the stream size is not known, `chunksOf` followed by
187+-- processing of indvidual arrays in the resulting stream should be preferred.
188+--
189+-- /Pre-release/
190+{-# INLINE fromStream #-}
191+fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
192+fromStream = Stream.fold A.write
193+-- write m = A.fromStreamD $ D.fromStreamK m
194+
169195-------------------------------------------------------------------------------
170196-- Elimination
171197-------------------------------------------------------------------------------
172198
199+{-# INLINE_NORMAL producer #-}
200+producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a
201+producer =
202+ Producer.translate A.unsafeThaw A.unsafeFreeze
203+ $ MA.producerWith (return . unsafeInlineIO)
204+
205+-- | Unfold an array into a stream.
206+--
207+{-# INLINE_NORMAL reader #-}
208+reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
209+reader = Producer.simplify producer
210+
211+-- | Unfold an array into a stream, does not check the end of the array, the
212+-- user is responsible for terminating the stream within the array bounds. For
213+-- high performance application where the end condition can be determined by
214+-- a terminating fold.
215+--
216+-- Written in the hope that it may be faster than "read", however, in the case
217+-- for which this was written, "read" proves to be faster even though the core
218+-- generated with unsafeRead looks simpler.
219+--
220+-- /Pre-release/
221+--
222+{-# INLINE_NORMAL readerUnsafe #-}
223+readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
224+readerUnsafe = Unfold step inject
225+ where
226+
227+ inject (Array contents start end) =
228+ return (ArrayUnsafe contents end start)
229+
230+ {-# INLINE_LATE step #-}
231+ step (ArrayUnsafe contents end p) = do
232+ -- unsafeInlineIO allows us to run this in Identity monad for pure
233+ -- toList/foldr case which makes them much faster due to not
234+ -- accumulating the list and fusing better with the pure consumers.
235+ --
236+ -- This should be safe as the array contents are guaranteed to be
237+ -- evaluated/written to before we peek at them.
238+ let !x = unsafeInlineIO $ peekByteIndex p contents
239+ let !p1 = INDEX_NEXT(p,a)
240+ return $ D.Yield x (ArrayUnsafe contents end p1)
241+
173242-- |
174243--
244+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
175245-- >>> null arr = Array.byteLength arr == 0
176246--
177247-- /Pre-release/
178248{-# INLINE null #-}
179249null :: Array a -> Bool
180−null arr = byteLength arr == 0
250+null arr = A.byteLength arr == 0
181251
182252-- | Like 'getIndex' but indexes the array in reverse from the end.
183253--
@@ -189,11 +259,12 @@ getIndexRev i arr =
189259 $ do
190260 let elemPtr = RINDEX_OF(arrEnd arr, i, a)
191261 if i >= 0 && elemPtr >= arrStart arr
192− then Just <$> peekAt elemPtr (arrContents arr)
262+ then Just <$> peekByteIndex elemPtr (arrContents arr)
193263 else return Nothing
194264
195265-- |
196266--
267+-- >>> import qualified Streamly.Internal.Data.Array as Array
197268-- >>> last arr = Array.getIndexRev arr 0
198269--
199270-- /Pre-release/
@@ -205,8 +276,6 @@ last = getIndexRev 0
205276-- Folds with Array as the container
206277-------------------------------------------------------------------------------
207278
208−-- XXX We should generate this from Ring.
209−
210279-- | @writeLastN n@ folds a maximum of @n@ elements from the end of the input
211280-- stream to an 'Array'.
212281--
@@ -215,7 +284,7 @@ writeLastN ::
215284 (Storable a, Unbox a, MonadIO m) => Int -> Fold m a (Array a)
216285writeLastN n
217286 | n <= 0 = fmap (const mempty) FL.drain
218− | otherwise = unsafeFreeze <$> Fold step initial done done
287+ | otherwise = A.unsafeFreeze <$> Fold step initial done
219288
220289 where
221290
@@ -228,9 +297,11 @@ writeLastN n
228297 in fmap f $ liftIO $ RB.new n
229298
230299 done (Tuple3Fused' rb rh i) = do
231− arr <- MA.new n
232− -- XXX We should write a read unfold for ring.
233− foldFunc i rh MA.snocUnsafe arr rb
300+ arr <- liftIO $ MA.new n
301+ foldFunc i rh snoc' arr rb
302+
303+ -- XXX We should write a read unfold for ring.
304+ snoc' b a = liftIO $ MA.snocUnsafe b a
234305
235306 foldFunc i
236307 | i < n = RB.unsafeFoldRingM
@@ -256,26 +327,19 @@ binarySearch = undefined
256327-- compared to streams by using SIMD instructions.
257328-- We can also return a bit array instead.
258329
259−-- Can use SIMD.
260−
261330-- | Perform a linear search to find all the indices where a given element is
262331-- present in an array.
263332--
264333-- /Unimplemented/
265−indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int
266−indexFinder = undefined
267−
268−-- |
269−-- /Unimplemented/
270−findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int
271−findIndicesOf p = Stream.unfold (indexFinder p)
334+findIndicesOf :: (a -> Bool) -> Unfold Identity (Array a) Int
335+findIndicesOf = undefined
272336
273337{-
274338findIndexOf :: (a -> Bool) -> Array a -> Maybe Int
275−findIndexOf p = Unfold.fold Fold.one . Stream.unfold (indexFinder p)
339+findIndexOf p = Unfold.fold Fold.one . Stream.unfold (findIndicesOf p)
276340
277341find :: (a -> Bool) -> Array a -> Bool
278−find = Unfold.fold Fold.null . Stream.unfold (indexFinder p)
342+find = Unfold.fold Fold.null . Stream.unfold (findIndicesOf p)
279343-}
280344
281345-------------------------------------------------------------------------------
@@ -317,45 +381,29 @@ splitOn :: (Monad m, Unbox a) =>
317381 (a -> Bool) -> Array a -> Stream m (Array a)
318382splitOn predicate arr =
319383 fmap (\(i, len) -> getSliceUnsafe i len arr)
320− $ D.indexOnSuffix predicate (read arr)
321−
322−{-# INLINE sliceIndexerFromLen #-}
323−sliceIndexerFromLen :: forall m a. (Monad m, Unbox a)
324− => Int -- ^ from index
325− -> Int -- ^ length of the slice
326− -> Unfold m (Array a) (Int, Int)
327−sliceIndexerFromLen from len =
328− Unfold.lmap unsafeThaw (MA.sliceIndexerFromLen from len)
384+ $ D.sliceOnSuffix predicate (A.toStreamD arr)
329385
330−{-# DEPRECATED genSlicesFromLen "Please use sliceIndexerFromLen instead." #-}
331386{-# INLINE genSlicesFromLen #-}
332387genSlicesFromLen :: forall m a. (Monad m, Unbox a)
333388 => Int -- ^ from index
334389 -> Int -- ^ length of the slice
335390 -> Unfold m (Array a) (Int, Int)
336−genSlicesFromLen = sliceIndexerFromLen
391+genSlicesFromLen from len =
392+ Unfold.lmap A.unsafeThaw (MA.genSlicesFromLen from len)
337393
338394-- | Generate a stream of slices of specified length from an array, starting
339395-- from the supplied array index. The last slice may be shorter than the
340396-- requested length.
341397--
342398-- /Pre-release//
343−{-# INLINE slicerFromLen #-}
344−slicerFromLen :: forall m a. (Monad m, Unbox a)
345− => Int -- ^ from index
346− -> Int -- ^ length of the slice
347− -> Unfold m (Array a) (Array a)
348−slicerFromLen from len =
349− fmap unsafeFreeze
350− $ Unfold.lmap unsafeThaw (MA.slicerFromLen from len)
351−
352−{-# DEPRECATED getSlicesFromLen "Please use slicerFromLen instead." #-}
353399{-# INLINE getSlicesFromLen #-}
354400getSlicesFromLen :: forall m a. (Monad m, Unbox a)
355401 => Int -- ^ from index
356402 -> Int -- ^ length of the slice
357403 -> Unfold m (Array a) (Array a)
358−getSlicesFromLen = slicerFromLen
404+getSlicesFromLen from len =
405+ fmap A.unsafeFreeze
406+ $ Unfold.lmap A.unsafeThaw (MA.getSlicesFromLen from len)
359407
360408-------------------------------------------------------------------------------
361409-- Random reads and writes
@@ -373,7 +421,7 @@ getIndex i arr =
373421 $ do
374422 let elemPtr = INDEX_OF(arrStart arr, i, a)
375423 if i >= 0 && INDEX_VALID(elemPtr, arrEnd arr, a)
376− then Just <$> peekAt elemPtr (arrContents arr)
424+ then Just <$> peekByteIndex elemPtr (arrContents arr)
377425 else return Nothing
378426
379427-- | Given a stream of array indices, read the elements on those indices from
@@ -385,25 +433,19 @@ getIndex i arr =
385433-- @
386434-- read =
387435-- let u = lmap (\arr -> (0, length arr - 1)) Unfold.enumerateFromTo
388−-- in Unfold.lmap f (indexReader arr)
436+-- in Unfold.lmap f (getIndices arr)
389437--
390438-- readRev =
391439-- let i = length arr - 1
392−-- in Unfold.lmap f (indexReaderFromThenTo i (i - 1) 0)
440+-- in Unfold.lmap f (getIndicesFromThenTo i (i - 1) 0)
393441-- @
394442--
395443-- /Pre-release/
396−{-# INLINE indexReader #-}
397−indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
398−indexReader m =
399− let unf = MA.indexReaderWith (return . unsafeInlineIO) m
400− in Unfold.lmap unsafeThaw unf
401−
402−-- XXX DO NOT REMOVE, change the signature to use Stream instead of unfold
403−{-# DEPRECATED getIndices "Please use getIndices instead." #-}
404444{-# INLINE getIndices #-}
405445getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
406−getIndices = indexReader
446+getIndices m =
447+ let unf = MA.getIndicesD (return . unsafeInlineIO) m
448+ in Unfold.lmap A.unsafeThaw unf
407449
408450-- | Unfolds @(from, then, to, array)@ generating a finite stream whose first
409451-- element is the array value from the index @from@ and the successive elements
@@ -419,9 +461,9 @@ getIndices = indexReader
419461-- @
420462--
421463-- /Unimplemented/
422−{-# INLINE indexReaderFromThenTo #-}
423−indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a
424−indexReaderFromThenTo = undefined
464+{-# INLINE getIndicesFromThenTo #-}
465+getIndicesFromThenTo :: Unfold m (Int, Int, Int, Array a) a
466+getIndicesFromThenTo = undefined
425467
426468-------------------------------------------------------------------------------
427469-- Transform via stream operations
@@ -438,7 +480,7 @@ indexReaderFromThenTo = undefined
438480{-# INLINE runPipe #-}
439481runPipe :: (MonadIO m, Unbox a, Unbox b)
440482 => Pipe m a b -> Array a -> m (Array b)
441−runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (read arr)
483+runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (A.read arr)
442484-}
443485
444486-- XXX For transformations that cannot change the number of elements e.g. "map"
@@ -452,7 +494,7 @@ runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (read arr)
452494streamTransform :: forall m a b. (MonadIO m, Unbox a, Unbox b)
453495 => (Stream m a -> Stream m b) -> Array a -> m (Array b)
454496streamTransform f arr =
455− Stream.fold (createWith (length arr)) $ f (read arr)
497+ Stream.fold (A.writeWith (length arr)) $ f (A.read arr)
456498
457499-------------------------------------------------------------------------------
458500-- Casts
@@ -486,7 +528,7 @@ asBytes = castUnsafe
486528--
487529cast :: forall a b. (Unbox b) => Array a -> Maybe (Array b)
488530cast arr =
489− let len = byteLength arr
531+ let len = A.byteLength arr
490532 r = len `mod` SIZE_OF(b)
491533 in if r /= 0
492534 then Nothing
@@ -504,9 +546,9 @@ cast arr =
504546--
505547asCStringUnsafe :: Array a -> (CString -> IO b) -> IO b
506548asCStringUnsafe arr act = do
507− let arr1 = asBytes arr <> fromList [0]
508− -- unsafePinnedAsPtr makes sure the array is pinned
509− unsafePinnedAsPtr arr1 $ \ptr -> act (castPtr ptr)
549+ let arr1 = asBytes arr <> A.fromList [0]
550+ -- asPtrUnsafe makes sure the array is pinned
551+ asPtrUnsafe arr1 $ \ptr -> act (castPtr ptr)
510552
511553-------------------------------------------------------------------------------
512554-- Folds
@@ -519,500 +561,11 @@ asCStringUnsafe arr act = do
519561-- /Pre-release/
520562{-# INLINE fold #-}
521563fold :: forall m a b. (Monad m, Unbox a) => Fold m a b -> Array a -> m b
522−fold f arr = Stream.fold f (read arr)
564+fold f arr = Stream.fold f (A.read arr)
523565
524566-- | Fold an array using a stream fold operation.
525567--
526568-- /Pre-release/
527569{-# INLINE streamFold #-}
528570streamFold :: (Monad m, Unbox a) => (Stream m a -> m b) -> Array a -> m b
529−streamFold f arr = f (read arr)
530−
531−--------------------------------------------------------------------------------
532−-- Serialization
533−--------------------------------------------------------------------------------
534−
535−{-# INLINE encodeAs #-}
536−encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8
537−encodeAs ps a =
538− unsafeInlineIO $ do
539− let len = Serialize.addSizeTo 0 a
540− mbarr <- MBA.newBytesAs ps len
541− off <- Serialize.serializeAt 0 mbarr a
542− assertM(len == off)
543− pure $ Array mbarr 0 off
544−
545−-- |
546−-- Properties:
547−-- 1. Identity: @deserialize . serialize == id@
548−-- 2. Encoded equivalence: @serialize a == serialize a@
549−{-# INLINE serialize #-}
550−serialize :: Serialize a => a -> Array Word8
551−serialize = encodeAs Unpinned
552−
553−-- | Serialize a Haskell type to a pinned byte array. The array is allocated
554−-- using pinned memory so that it can be used directly in OS APIs for writing
555−-- to file or sending over the network.
556−--
557−-- Properties:
558−-- 1. Identity: @deserialize . pinnedSerialize == id@
559−-- 2. Encoded equivalence: @pinnedSerialize a == pinnedSerialize a@
560−{-# INLINE pinnedSerialize #-}
561−pinnedSerialize :: Serialize a => a -> Array Word8
562−pinnedSerialize = encodeAs Pinned
563−
564−-- | Decode a Haskell type from a byte array containing its serialized
565−-- representation.
566−{-# INLINE deserialize #-}
567−deserialize :: Serialize a => Array Word8 -> a
568−deserialize arr@(Array {..}) = unsafeInlineIO $ do
569− let lenArr = length arr
570− (off, val) <-
571− Serialize.deserializeAt arrStart arrContents (arrStart + lenArr)
572− assertM(off == arrStart + lenArr)
573− pure val
574−
575−-------------------------------------------------------------------------------
576−-- Streams of Arrays
577−-------------------------------------------------------------------------------
578−
579−-- TODO: efficiently compare two streams of arrays. Two streams can have chunks
580−-- of different sizes, we can handle that in the stream comparison abstraction.
581−-- This could be useful e.g. to fast compare whether two files differ.
582−
583−-- | Insert the given element between arrays and flatten.
584−--
585−-- >>> interpose x = Stream.interpose x Array.reader
586−--
587−{-# INLINE interpose #-}
588−interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
589−interpose x = D.interpose x reader
590−
591−data FlattenState s =
592− OuterLoop s
593− | InnerLoop s !MutByteArray !Int !Int
594−
595−-- | Insert the given element after each array and flatten. This is similar to
596−-- unlines.
597−--
598−-- >>> interposeSuffix x = Stream.interposeSuffix x Array.reader
599−--
600−{-# INLINE_NORMAL interposeSuffix #-}
601−interposeSuffix :: forall m a. (Monad m, Unbox a)
602− => a -> Stream m (Array a) -> Stream m a
603−-- This does not require MonadIO constraint.
604−-- interposeSuffix x = D.interposeSuffix x reader
605−interposeSuffix sep (D.Stream step state) = D.Stream step' (OuterLoop state)
606−
607− where
608−
609− {-# INLINE_LATE step' #-}
610− step' gst (OuterLoop st) = do
611− r <- step (adaptState gst) st
612− return $ case r of
613− D.Yield Array{..} s ->
614− D.Skip (InnerLoop s arrContents arrStart arrEnd)
615− D.Skip s -> D.Skip (OuterLoop s)
616− D.Stop -> D.Stop
617−
618− step' _ (InnerLoop st _ p end) | p == end =
619− return $ D.Yield sep $ OuterLoop st
620−
621− step' _ (InnerLoop st contents p end) = do
622− x <- pure $ unsafeInlineIO $ peekAt p contents
623− return $ D.Yield x (InnerLoop st contents (INDEX_NEXT(p,a)) end)
624−
625−-- | Insert the given array after each array and flatten.
626−--
627−-- >>> intercalateSuffix = Stream.intercalateSuffix Array.reader
628−--
629−{-# INLINE intercalateSuffix #-}
630−intercalateSuffix :: (Monad m, Unbox a)
631− => Array a -> Stream m (Array a) -> Stream m a
632−intercalateSuffix = D.intercalateSuffix reader
633−
634−-- | @compactLE n@ coalesces adjacent arrays in the input stream
635−-- only if the combined size would be less than or equal to n.
636−--
637−-- Generates unpinned arrays irrespective of the pinning status of input
638−-- arrays.
639−{-# INLINE_NORMAL compactLE #-}
640−compactLE :: (MonadIO m, Unbox a)
641− => Int -> Stream m (Array a) -> Stream m (Array a)
642−compactLE n stream =
643− D.map unsafeFreeze $ MA.compactLE n $ D.map unsafeThaw stream
644−
645−-- | Pinned version of 'compactLE'.
646−{-# INLINE_NORMAL pinnedCompactLE #-}
647−pinnedCompactLE :: (MonadIO m, Unbox a)
648− => Int -> Stream m (Array a) -> Stream m (Array a)
649−pinnedCompactLE n stream =
650− D.map unsafeFreeze $ MA.pinnedCompactLE n $ D.map unsafeThaw stream
651−
652−-- | Split a stream of arrays on a given separator byte, dropping the separator
653−-- and coalescing all the arrays between two separators into a single array.
654−--
655−{-# INLINE compactOnByte #-}
656−compactOnByte
657− :: (MonadIO m)
658− => Word8
659− -> Stream m (Array Word8)
660− -> Stream m (Array Word8)
661−compactOnByte byte =
662− fmap unsafeFreeze . MA.compactOnByte byte . fmap unsafeThaw
663−
664−-- | Like 'compactOnByte' considers the separator in suffix position instead of
665−-- infix position.
666−{-# INLINE compactOnByteSuffix #-}
667−compactOnByteSuffix
668− :: (MonadIO m)
669− => Word8
670− -> Stream m (Array Word8)
671− -> Stream m (Array Word8)
672−compactOnByteSuffix byte =
673− fmap unsafeFreeze . MA.compactOnByteSuffix byte . fmap unsafeThaw
674−
675−-------------------------------------------------------------------------------
676−-- Folding Streams of Arrays
677−-------------------------------------------------------------------------------
678−
679−-- XXX This should not be used for breaking a stream as the D.cons used in
680−-- reconstructing the stream could be very bad for performance. This can only
681−-- be useful in folding without breaking.
682−{-# INLINE_NORMAL foldBreakChunks #-}
683−foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) =>
684− Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
685−foldBreakChunks (Fold fstep initial _ final) stream@(Stream step state) = do
686− res <- initial
687− case res of
688− Fold.Partial fs -> go SPEC state fs
689− Fold.Done fb -> return $! (fb, stream)
690−
691− where
692−
693− {-# INLINE go #-}
694− go !_ st !fs = do
695− r <- step defState st
696− case r of
697− Stream.Yield (Array contents start end) s ->
698− let fp = Tuple' end contents
699− in goArray SPEC s fp start fs
700− Stream.Skip s -> go SPEC s fs
701− Stream.Stop -> do
702− b <- final fs
703− return (b, D.nil)
704−
705− goArray !_ s (Tuple' end _) !cur !fs
706− | cur == end = do
707− go SPEC s fs
708− goArray !_ st fp@(Tuple' end contents) !cur !fs = do
709− x <- liftIO $ peekAt cur contents
710− res <- fstep fs x
711− let next = INDEX_NEXT(cur,a)
712− case res of
713− Fold.Done b -> do
714− let arr = Array contents next end
715− return $! (b, D.cons arr (D.Stream step st))
716− Fold.Partial fs1 -> goArray SPEC st fp next fs1
717−
718−-- This may be more robust wrt fusion compared to unfoldMany?
719−
720−-- | Fold a stream of arrays using a 'Fold'. This is equivalent to the
721−-- following:
722−--
723−-- >>> foldChunks f = Stream.fold f . Stream.unfoldMany Array.reader
724−--
725−foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b
726−foldChunks f s = fmap fst (foldBreakChunks f s)
727−-- foldStream f = Stream.fold f . Stream.unfoldMany reader
728−
729−-- | Fold a stream of arrays using a 'Fold' and return the remaining stream.
730−--
731−-- The following alternative to this function allows composing the fold using
732−-- the parser Monad:
733−--
734−-- @
735−-- foldBreakStreamK f s =
736−-- fmap (first (fromRight undefined))
737−-- $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s
738−-- @
739−--
740−-- We can compare perf and remove this one or define it in terms of that.
741−--
742−foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) =>
743− Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
744−{-
745−foldBreakChunksK f s =
746− fmap (first (fromRight undefined))
747− $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s
748−-}
749−foldBreakChunksK (Fold fstep initial _ final) stream = do
750− res <- initial
751− case res of
752− Fold.Partial fs -> go fs stream
753− Fold.Done fb -> return (fb, stream)
754−
755− where
756−
757− {-# INLINE go #-}
758− go !fs st = do
759− let stop = (, StreamK.nil) <$> final fs
760− single a = yieldk a StreamK.nil
761− yieldk (Array contents start end) r =
762− let fp = Tuple' end contents
763− in goArray fs r fp start
764− in StreamK.foldStream defState yieldk single stop st
765−
766− goArray !fs st (Tuple' end _) !cur
767− | cur == end = do
768− go fs st
769− goArray !fs st fp@(Tuple' end contents) !cur = do
770− x <- liftIO $ peekAt cur contents
771− res <- fstep fs x
772− let next = INDEX_NEXT(cur,a)
773− case res of
774− Fold.Done b -> do
775− let arr = Array contents next end
776− return $! (b, StreamK.cons arr st)
777− Fold.Partial fs1 -> goArray fs1 st fp next
778−
779−{-
780−-- This can be generalized to any type provided it can be unfolded to a stream
781−-- and it can be combined using a semigroup operation.
782−--
783−{-# INLINE_NORMAL parseBreakD #-}
784−parseBreakD ::
785− forall m a b. (MonadIO m, MonadThrow m, Unbox a)
786− => PRD.Parser a m b
787− -> D.Stream m (Array.Array a)
788− -> m (b, D.Stream m (Array.Array a))
789−parseBreakD
790− (PRD.Parser pstep initial extract) stream@(D.Stream step state) = do
791−
792− res <- initial
793− case res of
794− PRD.IPartial s -> go SPEC state (List []) s
795− PRD.IDone b -> return (b, stream)
796− PRD.IError err -> throwM $ ParseError err
797−
798− where
799−
800− -- "backBuf" contains last few items in the stream that we may have to
801− -- backtrack to.
802− --
803− -- XXX currently we are using a dumb list based approach for backtracking
804− -- buffer. This can be replaced by a sliding/ring buffer using Data.Array.
805− -- That will allow us more efficient random back and forth movement.
806− go !_ st backBuf !pst = do
807− r <- step defState st
808− case r of
809− D.Yield (Array contents start end) s ->
810− gobuf SPEC s backBuf
811− (Tuple' end contents) start pst
812− D.Skip s -> go SPEC s backBuf pst
813− D.Stop -> do
814− b <- extract pst
815− return (b, D.nil)
816−
817− -- Use strictness on "cur" to keep it unboxed
818− gobuf !_ s backBuf (Tuple' end _) !cur !pst
819− | cur == end = do
820− go SPEC s backBuf pst
821− gobuf !_ s backBuf fp@(Tuple' end contents) !cur !pst = do
822− x <- liftIO $ peekByteIndex contents cur
823− pRes <- pstep pst x
824− let next = INDEX_NEXT(cur,a)
825− case pRes of
826− PR.Partial 0 pst1 ->
827− gobuf SPEC s (List []) fp next pst1
828− PR.Partial n pst1 -> do
829− assert (n <= Prelude.length (x:getList backBuf)) (return ())
830− let src0 = Prelude.take n (x:getList backBuf)
831− arr0 = A.fromListN n (Prelude.reverse src0)
832− arr1 = Array contents next end
833− src = arr0 <> arr1
834− let !(Array cont1 start end1) = src
835− fp1 = Tuple' end1 cont1
836− gobuf SPEC s (List []) fp1 start pst1
837− PR.Continue 0 pst1 ->
838− gobuf SPEC s (List (x:getList backBuf)) fp next pst1
839− PR.Continue n pst1 -> do
840− assert (n <= Prelude.length (x:getList backBuf)) (return ())
841− let (src0, buf1) = splitAt n (x:getList backBuf)
842− arr0 = A.fromListN n (Prelude.reverse src0)
843− arr1 = Array contents next end
844− src = arr0 <> arr1
845− let !(Array cont1 start end1) = src
846− fp1 = Tuple' end1 cont1
847− gobuf SPEC s (List buf1) fp1 start pst1
848− PR.Done 0 b -> do
849− let arr = Array contents next end
850− return (b, D.cons arr (D.Stream step s))
851− PR.Done n b -> do
852− assert (n <= Prelude.length (x:getList backBuf)) (return ())
853− let src0 = Prelude.take n (x:getList backBuf)
854− -- XXX create the array in reverse instead
855− arr0 = A.fromListN n (Prelude.reverse src0)
856− arr1 = Array contents next end
857− -- XXX Use StreamK to avoid adding arbitrary layers of
858− -- constructors every time.
859− str = D.cons arr0 (D.cons arr1 (D.Stream step s))
860− return (b, str)
861− PR.Error err -> throwM $ ParseError err
862−-}
863−
864−-- | Parse an array stream using the supplied 'Parser'. Returns the parse
865−-- result and the unconsumed stream. Throws 'ParseError' if the parse fails.
866−--
867−-- The following alternative to this function allows composing the parser using
868−-- the parser Monad:
869−--
870−-- >>> parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)
871−--
872−-- We can compare perf and remove this one or define it in terms of that.
873−--
874−-- /Internal/
875−--
876−{-# INLINE_NORMAL parseBreakChunksK #-}
877−parseBreakChunksK ::
878− forall m a b. (MonadIO m, Unbox a)
879− => Parser a m b
880− -> StreamK m (Array a)
881− -> m (Either ParseError b, StreamK m (Array a))
882−-- parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)
883−parseBreakChunksK (Parser pstep initial extract) stream = do
884− res <- initial
885− case res of
886− IPartial s -> go s stream []
887− IDone b -> return (Right b, stream)
888− IError err -> return (Left (ParseError err), stream)
889−
890− where
891−
892− -- "backBuf" contains last few items in the stream that we may have to
893− -- backtrack to.
894− --
895− -- XXX currently we are using a dumb list based approach for backtracking
896− -- buffer. This can be replaced by a sliding/ring buffer using Data.Array.
897− -- That will allow us more efficient random back and forth movement.
898− go !pst st backBuf = do
899− let stop = goStop pst backBuf -- (, K.nil) <$> extract pst
900− single a = yieldk a StreamK.nil
901− yieldk arr r = goArray pst backBuf r arr
902− in StreamK.foldStream defState yieldk single stop st
903−
904− -- Use strictness on "cur" to keep it unboxed
905− goArray !pst backBuf st (Array _ cur end) | cur == end = go pst st backBuf
906− goArray !pst backBuf st (Array contents cur end) = do
907− x <- liftIO $ peekAt cur contents
908− pRes <- pstep pst x
909− let next = INDEX_NEXT(cur,a)
910− case pRes of
911− Parser.Partial 0 s ->
912− goArray s [] st (Array contents next end)
913− Parser.Partial n s -> do
914− assert (n <= Prelude.length (x:backBuf)) (return ())
915− let src0 = Prelude.take n (x:backBuf)
916− arr0 = fromListN n (Prelude.reverse src0)
917− arr1 = Array contents next end
918− src = arr0 <> arr1
919− goArray s [] st src
920− Parser.Continue 0 s ->
921− goArray s (x:backBuf) st (Array contents next end)
922− Parser.Continue n s -> do
923− assert (n <= Prelude.length (x:backBuf)) (return ())
924− let (src0, buf1) = Prelude.splitAt n (x:backBuf)
925− arr0 = fromListN n (Prelude.reverse src0)
926− arr1 = Array contents next end
927− src = arr0 <> arr1
928− goArray s buf1 st src
929− Parser.Done 0 b -> do
930− let arr = Array contents next end
931− return (Right b, StreamK.cons arr st)
932− Parser.Done n b -> do
933− assert (n <= Prelude.length (x:backBuf)) (return ())
934− let src0 = Prelude.take n (x:backBuf)
935− -- XXX Use fromListRevN once implemented
936− -- arr0 = A.fromListRevN n src0
937− arr0 = fromListN n (Prelude.reverse src0)
938− arr1 = Array contents next end
939− str = StreamK.cons arr0 (StreamK.cons arr1 st)
940− return (Right b, str)
941− Parser.Error err -> do
942− let n = Prelude.length backBuf
943− arr0 = fromListN n (Prelude.reverse backBuf)
944− arr1 = Array contents cur end
945− str = StreamK.cons arr0 (StreamK.cons arr1 stream)
946− return (Left (ParseError err), str)
947−
948− -- This is a simplified goArray
949− goExtract !pst backBuf (Array _ cur end)
950− | cur == end = goStop pst backBuf
951− goExtract !pst backBuf (Array contents cur end) = do
952− x <- liftIO $ peekAt cur contents
953− pRes <- pstep pst x
954− let next = INDEX_NEXT(cur,a)
955− case pRes of
956− Parser.Partial 0 s ->
957− goExtract s [] (Array contents next end)
958− Parser.Partial n s -> do
959− assert (n <= Prelude.length (x:backBuf)) (return ())
960− let src0 = Prelude.take n (x:backBuf)
961− arr0 = fromListN n (Prelude.reverse src0)
962− arr1 = Array contents next end
963− src = arr0 <> arr1
964− goExtract s [] src
965− Parser.Continue 0 s ->
966− goExtract s backBuf (Array contents next end)
967− Parser.Continue n s -> do
968− assert (n <= Prelude.length (x:backBuf)) (return ())
969− let (src0, buf1) = Prelude.splitAt n (x:backBuf)
970− arr0 = fromListN n (Prelude.reverse src0)
971− arr1 = Array contents next end
972− src = arr0 <> arr1
973− goExtract s buf1 src
974− Parser.Done 0 b -> do
975− let arr = Array contents next end
976− return (Right b, StreamK.fromPure arr)
977− Parser.Done n b -> do
978− assert (n <= Prelude.length backBuf) (return ())
979− let src0 = Prelude.take n backBuf
980− -- XXX Use fromListRevN once implemented
981− -- arr0 = A.fromListRevN n src0
982− arr0 = fromListN n (Prelude.reverse src0)
983− arr1 = Array contents next end
984− str = StreamK.cons arr0 (StreamK.fromPure arr1)
985− return (Right b, str)
986− Parser.Error err -> do
987− let n = Prelude.length backBuf
988− arr0 = fromListN n (Prelude.reverse backBuf)
989− arr1 = Array contents cur end
990− str = StreamK.cons arr0 (StreamK.cons arr1 stream)
991− return (Left (ParseError err), str)
992−
993− -- This is a simplified goExtract
994− {-# INLINE goStop #-}
995− goStop !pst backBuf = do
996− pRes <- extract pst
997− case pRes of
998− Parser.Partial _ _ -> error "Bug: parseBreak: Partial in extract"
999− Parser.Continue 0 s ->
1000− goStop s backBuf
1001− Parser.Continue n s -> do
1002− assert (n <= Prelude.length backBuf) (return ())
1003− let (src0, buf1) = Prelude.splitAt n backBuf
1004− arr = fromListN n (Prelude.reverse src0)
1005− goExtract s buf1 arr
1006− Parser.Done 0 b ->
1007− return (Right b, StreamK.nil)
1008− Parser.Done n b -> do
1009− assert (n <= Prelude.length backBuf) (return ())
1010− let src0 = Prelude.take n backBuf
1011− -- XXX Use fromListRevN once implemented
1012− -- arr0 = A.fromListRevN n src0
1013− arr0 = fromListN n (Prelude.reverse src0)
1014− return (Right b, StreamK.fromPure arr0)
1015− Parser.Error err -> do
1016− let n = Prelude.length backBuf
1017− arr0 = fromListN n (Prelude.reverse backBuf)
1018− return (Left (ParseError err), StreamK.fromPure arr0)
571+streamFold f arr = f (A.read arr)
core/src/Streamly/Internal/Data/Array/Generic.hs+11−49
@@ -11,8 +11,8 @@ module Streamly.Internal.Data.Array.Generic
1111
1212 -- * Construction
1313 , nil
14− , createOf
15− , create
14+ , writeN
15+ , write
1616 , writeWith
1717 , writeLastN
1818
@@ -24,8 +24,6 @@ module Streamly.Internal.Data.Array.Generic
2424 , fromListN
2525 , fromList
2626
27− , chunksOf
28−
2927 -- * Elimination
3028 , length
3129 , reader
@@ -44,10 +42,6 @@ module Streamly.Internal.Data.Array.Generic
4442 , getIndex
4543 , getSliceUnsafe
4644 , strip
47−
48− -- * Deprecated
49− , writeN
50− , write
5145 )
5246where
5347
@@ -63,7 +57,7 @@ import GHC.IO (unsafePerformIO)
6357import Text.Read (readPrec)
6458
6559import Streamly.Internal.Data.Fold.Type (Fold(..))
66−import Streamly.Internal.Data.Stream.Type (Stream)
60+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
6761import Streamly.Internal.Data.Unfold.Type (Unfold(..))
6862import Streamly.Internal.System.IO (unsafeInlineIO)
6963
@@ -71,9 +65,9 @@ import qualified Streamly.Internal.Data.MutArray.Generic as MArray
7165import qualified Streamly.Internal.Data.Fold.Type as FL
7266import qualified Streamly.Internal.Data.Producer.Type as Producer
7367import qualified Streamly.Internal.Data.Producer as Producer
74−import qualified Streamly.Internal.Data.Ring.Generic as RB
75−import qualified Streamly.Internal.Data.Stream.Type as D
76−import qualified Streamly.Internal.Data.Stream.Generate as D
68+import qualified Streamly.Internal.Data.Ring as RB
69+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
70+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
7771import qualified Text.ParserCombinators.ReadPrec as ReadPrec
7872
7973import Prelude hiding (Foldable(..), read)
@@ -108,33 +102,21 @@ nil = unsafePerformIO $ unsafeFreeze <$> MArray.nil
108102-- Construction - Folds
109103-------------------------------------------------------------------------------
110104
111−{-# INLINE_NORMAL createOf #-}
112−createOf :: MonadIO m => Int -> Fold m a (Array a)
113−createOf = fmap unsafeFreeze <$> MArray.createOf
114−
115−-- XXX Deprecate in major
116−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
117−{-# INLINE writeN #-}
105+{-# INLINE_NORMAL writeN #-}
118106writeN :: MonadIO m => Int -> Fold m a (Array a)
119−writeN = createOf
107+writeN = fmap unsafeFreeze <$> MArray.writeN
120108
121109{-# INLINE_NORMAL writeWith #-}
122110writeWith :: MonadIO m => Int -> Fold m a (Array a)
123−writeWith elemCount = unsafeFreeze <$> MArray.createWith elemCount
111+writeWith elemCount = unsafeFreeze <$> MArray.writeWith elemCount
124112
125113-- | Fold the whole input to a single array.
126114--
127115-- /Caution! Do not use this on infinite streams./
128116--
129−{-# INLINE create #-}
130−create :: MonadIO m => Fold m a (Array a)
131−create = fmap unsafeFreeze MArray.create
132−
133−-- XXX Deprecate in major
134−-- {-# DEPRECATED write "Please use create instead." #-}
135117{-# INLINE write #-}
136118write :: MonadIO m => Fold m a (Array a)
137−write = create
119+write = fmap unsafeFreeze MArray.write
138120
139121fromPureStream :: Stream Identity a -> Array a
140122fromPureStream x =
@@ -145,15 +127,6 @@ fromPureStream x =
145127fromByteStr# :: Addr# -> Array Word8
146128fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
147129
148−-------------------------------------------------------------------------------
149−-- Stream Ops
150−-------------------------------------------------------------------------------
151−
152−{-# INLINE_NORMAL chunksOf #-}
153−chunksOf :: forall m a. MonadIO m
154− => Int -> Stream m a -> Stream m (Array a)
155−chunksOf n strm = fmap unsafeFreeze $ MArray.chunksOf n strm
156−
157130-------------------------------------------------------------------------------
158131-- Construction - from streams
159132-------------------------------------------------------------------------------
@@ -258,17 +231,6 @@ getIndex i arr@Array {..} =
258231 then Just $ getIndexUnsafe i arr
259232 else Nothing
260233
261−-- >>> import qualified Streamly.Data.Stream as Stream
262−-- >>> import qualified Streamly.Data.Fold as Fold
263−-- >>> import qualified Streamly.Internal.Data.Array.Generic as Array
264−-- >>> import Data.Function ((&))
265−-- >>> :{
266−-- Stream.fromList [1,2,3,4,5::Int]
267−-- & Stream.scan (Array.writeLastN 2)
268−-- & Stream.fold Fold.toList
269−-- :}
270−-- [fromList [],fromList [1],fromList [1,2],fromList [2,3],fromList [3,4],fromList [4,5]]
271−--
272234{-# INLINE writeLastN #-}
273235writeLastN :: MonadIO m => Int -> Fold m a (Array a)
274236writeLastN n = FL.rmapM f (RB.writeLastN n)
@@ -276,7 +238,7 @@ writeLastN n = FL.rmapM f (RB.writeLastN n)
276238 where
277239
278240 f rb = do
279− arr <- RB.copyToMutArray 0 n rb
241+ arr <- RB.toMutArray 0 n rb
280242 return $ unsafeFreeze arr
281243
282244{-# INLINE getSliceUnsafe #-}
core/src/Streamly/Internal/Data/Array/Type.hs+108−538
@@ -1,11 +1,8 @@
11{-# LANGUAGE CPP #-}
2−{-# LANGUAGE TypeFamilies #-}
3−-- Must come after TypeFamilies, otherwise it is re-enabled.
4−-- MonoLocalBinds enabled by TypeFamilies causes perf regressions in general.
5−{-# LANGUAGE NoMonoLocalBinds #-}
62-- |
73-- Module : Streamly.Internal.Data.Array.Type
84-- Copyright : (c) 2020 Composewell Technologies
5+--
96-- License : BSD3-3-Clause
107-- Maintainer : streamly@composewell.com
118-- Stability : experimental
@@ -13,146 +10,79 @@
1310--
1411-- See notes in "Streamly.Internal.Data.MutArray.Type"
1512--
16−
1713module Streamly.Internal.Data.Array.Type
1814 (
19− -- ** Type
2015 -- $arrayNotes
2116 Array (..)
17+ , asPtrUnsafe
18+ , nil
2219
23− -- ** Conversion
24− -- *** Mutable and Immutable
20+ -- * Freezing and Thawing
2521 , unsafeFreeze
2622 , unsafeFreezeWithShrink
2723 , unsafeThaw
2824
29− -- *** Pinned and Unpinned
25+ -- * Pinning and Unpinning
3026 , pin
3127 , unpin
3228 , isPinned
3329
34− -- *** Casting
35− , unsafePinnedAsPtr
36−
37− -- ** Construction
38− , empty
30+ -- * Construction
31+ , splice
3932
40− -- *** Cloning
41− -- XXX Why would we clone an immutable array?
42− , clone
43− , pinnedClone
44−
45− -- *** Slicing
46− -- | Get a subarray without copying
47− , splitAt
48− , breakOn -- XXX requires MonadIO
49−
50− -- *** Stream Folds
51− , unsafeMakePure
52− , createOf
53− , pinnedCreateOf
54− , unsafeCreateOf
55− , unsafePinnedCreateOf
56− , create
57− , pinnedCreate
58− , createWith
59−
60− -- *** From containers
61− , fromListN
62− , pinnedFromListN
6333 , fromList
6434 , pinnedFromList
65− , fromListRevN
35+ , fromListN
36+ , pinnedFromListN
6637 , fromListRev
67− , fromStreamN
68− , fromStream
69− , fromPureStreamN
38+ , fromListRevN
39+ , fromStreamDN
40+ , fromStreamD
7041 , fromPureStream
7142 , fromByteStr#
72− , fromByteStr
73− , fromPtrN
74− , fromChunks
75− , fromChunksK
7643
77− -- ** Reading
44+ -- * Split
45+ , breakOn
7846
79− -- *** Indexing
80− , unsafeIndexIO -- XXX unsafeGetIndexIO
81− , getIndexUnsafe -- XXX unsafeGetIndex
82−
83− -- *** To Streams
84− , read
85− , readRev
86− , toStreamK
87− , toStreamKRev
88−
89− -- *** To Containers
90− , toList
91−
92− -- *** Unfolds
93− , producer -- experimental
94− , readerUnsafe
95− , reader
96− , readerRev
97−
98− -- *** Size
99− , length
47+ -- * Elimination
48+ , unsafeIndexIO
49+ , unsafeIndex -- getIndexUnsafe
10050 , byteLength
51+ , length
10152
102− -- ** Folding
10353 , foldl'
10454 , foldr
105− , byteCmp
106− , byteEq
107−
108− -- ** Appending
109− , splice -- XXX requires MonadIO
110−
111− -- ** Streams of arrays
112− -- *** Chunk
113− -- | Group a stream into arrays.
114− , chunksOf
115− , pinnedChunksOf
116− , buildChunks
117−
118− -- *** Split
119− -- | Split an array into slices.
120−
121− -- *** Concat
122− -- | Append the arrays in a stream to form a stream of elements.
123− , concat
124− , concatRev
125−
126− -- *** Compact
127− -- | Append the arrays in a stream to form a stream of larger arrays.
128− , fCompactGE
129− , fPinnedCompactGE
130− , lCompactGE
131− , lPinnedCompactGE
132− , compactGE
55+ , splitAt
13356
134− -- ** Deprecated
135− , asPtrUnsafe
136− , unsafeIndex
137− , bufferChunks
138− , flattenArrays
139− , flattenArraysRev
140− , fromArrayStreamK
141− , fromStreamDN
142− , fromStreamD
14357 , toStreamD
14458 , toStreamDRev
59+ , toStreamK
60+ , toStreamKRev
14561 , toStream
14662 , toStreamRev
147− , nil
63+ , read
64+ , readRev
65+ , readerRev
66+ , toList
67+
68+ -- * Folds
14869 , writeWith
14970 , writeN
15071 , pinnedWriteN
15172 , writeNUnsafe
15273 , pinnedWriteNUnsafe
74+ , MA.ArrayUnsafe (..)
15375 , pinnedWriteNAligned
15476 , write
15577 , pinnedWrite
78+ , unsafeMakePure
79+
80+ -- * Streams of arrays
81+ , chunksOf
82+ , pinnedChunksOf
83+ , bufferChunks
84+ , flattenArrays
85+ , flattenArraysRev
15686 )
15787where
15888
@@ -160,36 +90,31 @@ where
16090#include "inline.hs"
16191
16292import Control.Exception (assert)
163−import Control.Monad (replicateM, when)
93+import Control.Monad (replicateM)
16494import Control.Monad.IO.Class (MonadIO(..))
16595import Data.Functor.Identity (Identity(..))
166−import Data.Int (Int8, Int16, Int32, Int64)
16796import Data.Proxy (Proxy(..))
168−import Data.Word (Word8, Word16, Word32, Word64)
97+import Data.Word (Word8)
16998import GHC.Base (build)
17099import GHC.Exts (IsList, IsString(..), Addr#)
171100
172101import GHC.IO (unsafePerformIO)
173102import GHC.Ptr (Ptr(..))
174−import Streamly.Internal.Data.Producer.Type (Producer(..))
175−import Streamly.Internal.Data.MutArray.Type (MutArray(..))
176−import Streamly.Internal.Data.MutByteArray.Type (MutByteArray)
103+import Streamly.Internal.Data.MutArray.Type (MutArray(..), MutableByteArray)
177104import Streamly.Internal.Data.Fold.Type (Fold(..))
178−import Streamly.Internal.Data.Stream.Type (Stream)
179−import Streamly.Internal.Data.StreamK.Type (StreamK)
105+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
180106import Streamly.Internal.Data.Unbox (Unbox(..))
181107import Streamly.Internal.Data.Unfold.Type (Unfold(..))
182108import Text.Read (readPrec)
183109
184−import Prelude hiding (Foldable(..), concat, read, unlines, splitAt)
110+import Prelude hiding (Foldable(..), read, unlines, splitAt)
185111
186112import qualified GHC.Exts as Exts
187−import qualified Streamly.Internal.Data.Fold.Type as Fold
188113import qualified Streamly.Internal.Data.MutArray.Type as MA
189−import qualified Streamly.Internal.Data.Stream.Type as D
190−import qualified Streamly.Internal.Data.StreamK.Type as K
191−import qualified Streamly.Internal.Data.MutByteArray.Type as Unboxed
192−import qualified Streamly.Internal.Data.Producer as Producer
114+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
115+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
116+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
117+import qualified Streamly.Internal.Data.Unbox as Unboxed
193118import qualified Streamly.Internal.Data.Unfold.Type as Unfold
194119import qualified Text.ParserCombinators.ReadPrec as ReadPrec
195120
@@ -232,7 +157,7 @@ data Array a =
232157#endif
233158 -- All offsets are in terms of bytes from the start of arraycontents
234159 Array
235− { arrContents :: {-# UNPACK #-} !MutByteArray
160+ { arrContents :: {-# UNPACK #-} !MutableByteArray
236161 , arrStart :: {-# UNPACK #-} !Int -- offset
237162 , arrEnd :: {-# UNPACK #-} !Int -- offset + len
238163 }
@@ -243,20 +168,14 @@ data Array a =
243168
244169-- | Use an @Array a@ as @Ptr a@.
245170--
246−-- See 'MA.unsafePinnedAsPtr' in the Mutable array module for more details.
171+-- See 'MA.asPtrUnsafe' in the Mutable array module for more details.
247172--
248173-- /Unsafe/
249174--
250175-- /Pre-release/
251176--
252−{-# INLINE unsafePinnedAsPtr #-}
253−unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
254−unsafePinnedAsPtr arr = MA.unsafePinnedAsPtr (unsafeThaw arr)
255−
256−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
257−{-# INLINE asPtrUnsafe #-}
258177asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
259−asPtrUnsafe = unsafePinnedAsPtr
178+asPtrUnsafe arr = MA.asPtrUnsafe (unsafeThaw arr)
260179
261180-------------------------------------------------------------------------------
262181-- Freezing and Thawing
@@ -303,19 +222,18 @@ unsafeThaw (Array ac as ae) = MutArray ac as ae ae
303222-- Pinning & Unpinning
304223-------------------------------------------------------------------------------
305224
306−-- | Return a copy of the 'Array' in pinned memory if unpinned, else return the
307−-- original array.
225+-- | Copy the 'Array' to pinned memory if unpinned, else do nothing. The
226+-- overhead is a copy if the input array is unpinned.
308227{-# INLINE pin #-}
309228pin :: Array a -> IO (Array a)
310229pin = fmap unsafeFreeze . MA.pin . unsafeThaw
311230
312−-- | Return a copy of the 'Array' in unpinned memory if pinned, else return the
313−-- original array.
231+-- | Copy the 'Array' to unpinned memory if pinned, else do nothing. The
232+-- overhead is a copy if the input array is pinned.
314233{-# INLINE unpin #-}
315234unpin :: Array a -> IO (Array a)
316235unpin = fmap unsafeFreeze . MA.unpin . unsafeThaw
317236
318−-- | Return 'True' if the array is allocated in pinned memory.
319237{-# INLINE isPinned #-}
320238isPinned :: Array a -> Bool
321239isPinned = MA.isPinned . unsafeThaw
@@ -324,14 +242,11 @@ isPinned = MA.isPinned . unsafeThaw
324242-- Construction
325243-------------------------------------------------------------------------------
326244
327−-- | Copy two immutable arrays into a new array. If you want to splice more
328−-- than two arrays then this operation would be highly inefficient because it
329−-- would make a copy on every splice operation, instead use the
330−-- 'fromChunksK' operation to combine n immutable arrays.
245+-- Splice two immutable arrays creating a new array.
331246{-# INLINE splice #-}
332−splice :: MonadIO m => Array a -> Array a -> m (Array a)
247+splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
333248splice arr1 arr2 =
334− unsafeFreeze <$> MA.spliceCopy (unsafeThaw arr1) (unsafeThaw arr2)
249+ unsafeFreeze <$> MA.splice (unsafeThaw arr1) (unsafeThaw arr2)
335250
336251-- | Create an 'Array' from the first N elements of a list. The array is
337252-- allocated to size N, if the list terminates before N elements then the
@@ -375,83 +290,24 @@ pinnedFromList xs = unsafePerformIO $ unsafeFreeze <$> MA.pinnedFromList xs
375290fromListRev :: Unbox a => [a] -> Array a
376291fromListRev xs = unsafePerformIO $ unsafeFreeze <$> MA.fromListRev xs
377292
378−-- | Create an 'Array' from the first N elements of a stream. The array is
379−-- allocated to size N, if the stream terminates before N elements then the
380−-- array may hold less than N elements.
381−--
382−-- >>> fromStreamN n = Stream.fold (Array.writeN n)
383−--
384−-- /Pre-release/
385−{-# INLINE_NORMAL fromStreamN #-}
386−fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
387−fromStreamN n m = do
388− when (n < 0) $ error "writeN: negative write count specified"
389− unsafeFreeze <$> MA.fromStreamN n m
390−-- fromStreamN n = D.fold (writeN n)
391−
392−{-# DEPRECATED fromStreamDN "Please use fromStreamN instead." #-}
293+{-# INLINE_NORMAL fromStreamDN #-}
393294fromStreamDN :: forall m a. (MonadIO m, Unbox a)
394295 => Int -> D.Stream m a -> m (Array a)
395−fromStreamDN = fromStreamN
296+fromStreamDN limit str = unsafeFreeze <$> MA.fromStreamDN limit str
396297
397−-- | Create an 'Array' from a stream. This is useful when we want to create a
398−-- single array from a stream of unknown size. 'writeN' is at least twice
399−-- as efficient when the size is already known.
400−--
401−-- >>> fromStream = Stream.fold Array.write
402−--
403−-- Note that if the input stream is too large memory allocation for the array
404−-- may fail. When the stream size is not known, `chunksOf` followed by
405−-- processing of indvidual arrays in the resulting stream should be preferred.
406−--
407−-- /Pre-release/
408298{-# INLINE_NORMAL fromStreamD #-}
409−fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
410−fromStream = D.fold write
411−-- fromStreamD str = unsafeFreeze <$> MA.fromStream str
412−
413−{-# DEPRECATED fromStreamD "Please use fromStream instead." #-}
414299fromStreamD :: forall m a. (MonadIO m, Unbox a)
415300 => D.Stream m a -> m (Array a)
416−fromStreamD = fromStream
417−
418−-------------------------------------------------------------------------------
419−-- Cloning
420−-------------------------------------------------------------------------------
421−
422−{-# INLINE clone #-}
423−clone ::
424− ( MonadIO m
425−#ifdef DEVBUILD
426− , Unbox a
427−#endif
428− )
429− => Array a -> m (Array a)
430−clone = fmap unsafeFreeze . MA.clone . unsafeThaw
431−
432−{-# INLINE pinnedClone #-}
433−pinnedClone ::
434− ( MonadIO m
435−#ifdef DEVBUILD
436− , Unbox a
437−#endif
438− )
439− => Array a -> m (Array a)
440−pinnedClone = fmap unsafeFreeze . MA.pinnedClone . unsafeThaw
301+fromStreamD str = unsafeFreeze <$> MA.fromStreamD str
441302
442303-------------------------------------------------------------------------------
443304-- Streams of arrays
444305-------------------------------------------------------------------------------
445306
446−{-# INLINE buildChunks #-}
447−buildChunks :: (MonadIO m, Unbox a) =>
448− D.Stream m a -> m (K.StreamK m (Array a))
449−buildChunks m = D.foldr K.cons K.nil $ chunksOf defaultChunkSize m
450−
451−{-# DEPRECATED bufferChunks "Please use buildChunks instead." #-}
307+{-# INLINE bufferChunks #-}
452308bufferChunks :: (MonadIO m, Unbox a) =>
453309 D.Stream m a -> m (K.StreamK m (Array a))
454−bufferChunks = buildChunks
310+bufferChunks m = D.foldr K.cons K.nil $ chunksOf defaultChunkSize m
455311
456312-- | @chunksOf n stream@ groups the elements in the input stream into arrays of
457313-- @n@ elements each.
@@ -472,100 +328,27 @@ pinnedChunksOf :: forall m a. (MonadIO m, Unbox a)
472328 => Int -> D.Stream m a -> D.Stream m (Array a)
473329pinnedChunksOf n str = D.map unsafeFreeze $ MA.pinnedChunksOf n str
474330
475−-- | Convert a stream of arrays into a stream of their elements.
331+-- | Use the "read" unfold instead.
332+--
333+-- @flattenArrays = unfoldMany read@
476334--
477−-- >>> concat = Stream.unfoldMany Array.reader
335+-- We can try this if there are any fusion issues in the unfold.
478336--
479−{-# INLINE_NORMAL concat #-}
480−concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
481−-- XXX this requires MonadIO whereas the unfoldMany version does not
482−concat = MA.concatWith (pure . unsafeInlineIO) . D.map unsafeThaw
483−-- concat = D.unfoldMany reader
484−
485−{-# DEPRECATED flattenArrays "Please use \"unfoldMany reader\" instead." #-}
486−{-# INLINE flattenArrays #-}
337+{-# INLINE_NORMAL flattenArrays #-}
487338flattenArrays :: forall m a. (MonadIO m, Unbox a)
488339 => D.Stream m (Array a) -> D.Stream m a
489−flattenArrays = concat
340+flattenArrays = MA.flattenArrays . D.map unsafeThaw
490341
491−-- | Convert a stream of arrays into a stream of their elements reversing the
492−-- contents of each array before flattening.
342+-- | Use the "readRev" unfold instead.
493343--
494−-- >>> concatRev = Stream.unfoldMany Array.readerRev
344+-- @flattenArrays = unfoldMany readRev@
495345--
496−{-# INLINE_NORMAL concatRev #-}
497−concatRev :: forall m a. (Monad m, Unbox a)
498− => D.Stream m (Array a) -> D.Stream m a
499−-- XXX this requires MonadIO whereas the unfoldMany version does not
500−concatRev = MA.concatRevWith (pure . unsafeInlineIO) . D.map unsafeThaw
501−-- concatRev = D.unfoldMany readerRev
502−
503−{-# DEPRECATED flattenArraysRev "Please use \"unfoldMany readerRev\" instead." #-}
504−{-# INLINE flattenArraysRev #-}
346+-- We can try this if there are any fusion issues in the unfold.
347+--
348+{-# INLINE_NORMAL flattenArraysRev #-}
505349flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
506350 => D.Stream m (Array a) -> D.Stream m a
507−flattenArraysRev = concatRev
508−
509−-------------------------------------------------------------------------------
510−-- Compact
511−-------------------------------------------------------------------------------
512−
513−-- XXX Note that this thaws immutable arrays for appending, that may be
514−-- problematic if multiple users do the same thing, however, thawed immutable
515−-- arrays would have no capacity to append, therefore, a copy will be forced
516−-- anyway.
517−
518−-- | Fold @fCompactGE n@ coalesces adjacent arrays in the input stream
519−-- until the size becomes greater than or equal to n.
520−--
521−-- Generates unpinned arrays irrespective of the pinning status of input
522−-- arrays.
523−{-# INLINE_NORMAL fCompactGE #-}
524−fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
525−fCompactGE n = fmap unsafeFreeze $ Fold.lmap unsafeThaw $ MA.fCompactGE n
526−
527−-- | PInned version of 'fCompactGE'.
528−{-# INLINE_NORMAL fPinnedCompactGE #-}
529−fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
530−fPinnedCompactGE n =
531− fmap unsafeFreeze $ Fold.lmap unsafeThaw $ MA.fPinnedCompactGE n
532−
533−-- | @compactGE n stream@ coalesces adjacent arrays in the @stream@ until
534−-- the size becomes greater than or equal to @n@.
535−--
536−-- >>> compactGE n = Stream.foldMany (Array.fCompactGE n)
537−--
538−-- Generates unpinned arrays irrespective of the pinning status of input
539−-- arrays.
540−{-# INLINE compactGE #-}
541−compactGE ::
542− (MonadIO m, Unbox a)
543− => Int -> Stream m (Array a) -> Stream m (Array a)
544−compactGE n stream =
545− D.map unsafeFreeze $ MA.compactGE n $ D.map unsafeThaw stream
546−
547−-- | Like 'compactGE' but for transforming folds instead of stream.
548−--
549−-- >>> lCompactGE n = Fold.many (Array.fCompactGE n)
550−--
551−-- Generates unpinned arrays irrespective of the pinning status of input
552−-- arrays.
553−{-# INLINE_NORMAL lCompactGE #-}
554−lCompactGE :: (MonadIO m, Unbox a)
555− => Int -> Fold m (Array a) () -> Fold m (Array a) ()
556−lCompactGE n fld =
557− Fold.lmap unsafeThaw $ MA.lCompactGE n (Fold.lmap unsafeFreeze fld)
558−
559−-- | Pinned version of 'lCompactGE'.
560−{-# INLINE_NORMAL lPinnedCompactGE #-}
561−lPinnedCompactGE :: (MonadIO m, Unbox a)
562− => Int -> Fold m (Array a) () -> Fold m (Array a) ()
563−lPinnedCompactGE n fld =
564− Fold.lmap unsafeThaw $ MA.lPinnedCompactGE n (Fold.lmap unsafeFreeze fld)
565−
566−-------------------------------------------------------------------------------
567−-- Splitting
568−-------------------------------------------------------------------------------
351+flattenArraysRev = MA.flattenArraysRev . D.map unsafeThaw
569352
570353-- Drops the separator byte
571354{-# INLINE breakOn #-}
@@ -587,14 +370,9 @@ unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a
587370unsafeIndexIO i arr = MA.getIndexUnsafe i (unsafeThaw arr)
588371
589372-- | Return element at the specified index without checking the bounds.
590−{-# INLINE_NORMAL getIndexUnsafe #-}
591−getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a
592−getIndexUnsafe i arr = let !r = unsafeInlineIO $ unsafeIndexIO i arr in r
593−
594−{-# DEPRECATED unsafeIndex "Please use 'getIndexUnsafe' instead" #-}
595373{-# INLINE_NORMAL unsafeIndex #-}
596374unsafeIndex :: forall a. Unbox a => Int -> Array a -> a
597−unsafeIndex = getIndexUnsafe
375+unsafeIndex i arr = let !r = unsafeInlineIO $ unsafeIndexIO i arr in r
598376
599377-- | /O(1)/ Get the byte length of the array.
600378--
@@ -609,68 +387,24 @@ byteLength = MA.byteLength . unsafeThaw
609387length :: Unbox a => Array a -> Int
610388length arr = MA.length (unsafeThaw arr)
611389
612−{-# INLINE_NORMAL producer #-}
613−producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a
614−producer =
615− Producer.translate unsafeThaw unsafeFreeze
616− $ MA.producerWith (return . unsafeInlineIO)
617−
618−-- | Unfold an array into a stream.
619−--
620−{-# INLINE_NORMAL reader #-}
621−reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
622−reader = Producer.simplify producer
623−
624−-- | Unfold an array into a stream, does not check the end of the array, the
625−-- user is responsible for terminating the stream within the array bounds. For
626−-- high performance application where the end condition can be determined by
627−-- a terminating fold.
628−--
629−-- Written in the hope that it may be faster than "read", however, in the case
630−-- for which this was written, "read" proves to be faster even though the core
631−-- generated with unsafeRead looks simpler.
632−--
633−-- /Pre-release/
634−--
635−{-# INLINE_NORMAL readerUnsafe #-}
636−readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
637−readerUnsafe = Unfold step inject
638− where
639−
640− inject (Array contents start end) =
641− return (MA.ArrayUnsafe contents end start)
642−
643− {-# INLINE_LATE step #-}
644− step (MA.ArrayUnsafe contents end p) = do
645− -- unsafeInlineIO allows us to run this in Identity monad for pure
646− -- toList/foldr case which makes them much faster due to not
647− -- accumulating the list and fusing better with the pure consumers.
648− --
649− -- This should be safe as the array contents are guaranteed to be
650− -- evaluated/written to before we peek at them.
651− let !x = unsafeInlineIO $ peekAt p contents
652− let !p1 = INDEX_NEXT(p,a)
653− return $ D.Yield x (MA.ArrayUnsafe contents end p1)
654−
655390-- | Unfold an array into a stream in reverse order.
656391--
657392{-# INLINE_NORMAL readerRev #-}
658393readerRev :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
659394readerRev = Unfold.lmap unsafeThaw $ MA.readerRevWith (return . unsafeInlineIO)
660395
661−{-# DEPRECATED toStreamD "Please use 'read' instead." #-}
662396{-# INLINE_NORMAL toStreamD #-}
663397toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> D.Stream m a
664−toStreamD = read
398+toStreamD arr = MA.toStreamDWith (return . unsafeInlineIO) (unsafeThaw arr)
665399
666400{-# INLINE toStreamK #-}
667401toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> K.StreamK m a
668402toStreamK arr = MA.toStreamKWith (return . unsafeInlineIO) (unsafeThaw arr)
669403
670−{-# DEPRECATED toStreamDRev "Please use 'readRev' instead." #-}
671404{-# INLINE_NORMAL toStreamDRev #-}
672405toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> D.Stream m a
673−toStreamDRev = readRev
406+toStreamDRev arr =
407+ MA.toStreamDRevWith (return . unsafeInlineIO) (unsafeThaw arr)
674408
675409{-# INLINE toStreamKRev #-}
676410toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> K.StreamK m a
@@ -682,7 +416,7 @@ toStreamKRev arr =
682416-- /Pre-release/
683417{-# INLINE_EARLY read #-}
684418read :: (Monad m, Unbox a) => Array a -> Stream m a
685−read arr = MA.toStreamWith (return . unsafeInlineIO) (unsafeThaw arr)
419+read = toStreamD
686420
687421-- | Same as 'read'
688422--
@@ -699,7 +433,7 @@ toStream = read
699433-- /Pre-release/
700434{-# INLINE_EARLY readRev #-}
701435readRev :: (Monad m, Unbox a) => Array a -> Stream m a
702−readRev arr = MA.toStreamRevWith (return . unsafeInlineIO) (unsafeThaw arr)
436+readRev = toStreamDRev
703437
704438-- | Same as 'readRev'
705439--
@@ -723,7 +457,6 @@ foldr f z arr = runIdentity $ D.foldr f z $ toStreamD arr
723457-- | Create two slices of an array without copying the original array. The
724458-- specified index @i@ is the first index of the second slice.
725459--
726−{-# INLINE splitAt #-}
727460splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)
728461splitAt i arr = (unsafeFreeze a, unsafeFreeze b)
729462 where
@@ -743,8 +476,8 @@ toListFB c n Array{..} = go arrStart
743476 -- accumulating the list and fusing better with the pure consumers.
744477 --
745478 -- This should be safe as the array contents are guaranteed to be
746− -- evaluated/written to before we peekAt at them.
747− let !x = unsafeInlineIO $ peekAt p arrContents
479+ -- evaluated/written to before we peekByteIndex at them.
480+ let !x = unsafeInlineIO $ peekByteIndex p arrContents
748481 in c x (go (INDEX_NEXT(p,a)))
749482
750483-- | Convert an 'Array' into a list.
@@ -757,28 +490,17 @@ toList s = build (\c n -> toListFB c n s)
757490-- Folds
758491-------------------------------------------------------------------------------
759492
760−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
493+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
761494-- 'Array'.
762495--
763−{-# INLINE_NORMAL createOf #-}
764−createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
765−createOf = fmap unsafeFreeze . MA.createOf
766−
767−-- XXX Deprecate in major
768−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
769−{-# INLINE writeN #-}
496+{-# INLINE_NORMAL writeN #-}
770497writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
771−writeN = createOf
498+writeN = fmap unsafeFreeze . MA.writeN
772499
773−-- | Like 'createOf' but creates a pinned array.
774−{-# INLINE_NORMAL pinnedCreateOf #-}
775−pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
776−pinnedCreateOf = fmap unsafeFreeze . MA.pinnedCreateOf
777−
778−{-# DEPRECATED pinnedWriteN "Please use pinnedCreateOf instead." #-}
779−{-# INLINE pinnedWriteN #-}
500+-- | Like 'fromListN' but creates a pinned array.
501+{-# INLINE_NORMAL pinnedWriteN #-}
780502pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
781−pinnedWriteN = pinnedCreateOf
503+pinnedWriteN = fmap unsafeFreeze . MA.pinnedWriteN
782504
783505-- | @pinnedWriteNAligned alignment n@ folds a maximum of @n@ elements from the input
784506-- stream to an 'Array' aligned to the given size.
@@ -786,74 +508,44 @@ pinnedWriteN = pinnedCreateOf
786508-- /Pre-release/
787509--
788510{-# INLINE_NORMAL pinnedWriteNAligned #-}
789−{-# DEPRECATED pinnedWriteNAligned "To be removed." #-}
790511pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a)
791512 => Int -> Int -> Fold m a (Array a)
792513pinnedWriteNAligned alignSize = fmap unsafeFreeze . MA.pinnedWriteNAligned alignSize
793514
794−-- | Like 'createOf' but does not check the array bounds when writing. The fold
515+-- | Like 'writeN' but does not check the array bounds when writing. The fold
795516-- driver must not call the step function more than 'n' times otherwise it will
796517-- corrupt the memory and crash. This function exists mainly because any
797518-- conditional in the step function blocks fusion causing 10x performance
798519-- slowdown.
799520--
800−{-# INLINE_NORMAL unsafeCreateOf #-}
801−unsafeCreateOf :: forall m a. (MonadIO m, Unbox a)
802− => Int -> Fold m a (Array a)
803−unsafeCreateOf n = unsafeFreeze <$> MA.unsafeCreateOf n
804−
805−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
806−{-# INLINE writeNUnsafe #-}
521+{-# INLINE_NORMAL writeNUnsafe #-}
807522writeNUnsafe :: forall m a. (MonadIO m, Unbox a)
808523 => Int -> Fold m a (Array a)
809−writeNUnsafe = unsafeCreateOf
524+writeNUnsafe n = unsafeFreeze <$> MA.writeNUnsafe n
810525
811−{-# INLINE_NORMAL unsafePinnedCreateOf #-}
812−unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a)
813− => Int -> Fold m a (Array a)
814−unsafePinnedCreateOf n = unsafeFreeze <$> MA.unsafePinnedCreateOf n
815−
816−{-# DEPRECATED pinnedWriteNUnsafe "Please use unsafePinnedCreateOf instead." #-}
817−{-# INLINE pinnedWriteNUnsafe #-}
526+{-# INLINE_NORMAL pinnedWriteNUnsafe #-}
818527pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a)
819528 => Int -> Fold m a (Array a)
820−pinnedWriteNUnsafe = unsafePinnedCreateOf
821−
822−{-# INLINE_NORMAL createWith #-}
823−createWith :: forall m a. (MonadIO m, Unbox a)
824− => Int -> Fold m a (Array a)
825−-- createWith n = FL.rmapM spliceArrays $ toArraysOf n
826−createWith elemCount = unsafeFreeze <$> MA.createWith elemCount
529+pinnedWriteNUnsafe n = unsafeFreeze <$> MA.pinnedWriteNUnsafe n
827530
828−{-# DEPRECATED writeWith "Please use createWith instead." #-}
829−{-# INLINE writeWith #-}
531+{-# INLINE_NORMAL writeWith #-}
830532writeWith :: forall m a. (MonadIO m, Unbox a)
831533 => Int -> Fold m a (Array a)
832−writeWith = createWith
534+-- writeWith n = FL.rmapM spliceArrays $ toArraysOf n
535+writeWith elemCount = unsafeFreeze <$> MA.writeWith elemCount
833536
834537-- | Fold the whole input to a single array.
835538--
836539-- /Caution! Do not use this on infinite streams./
837540--
838−{-# INLINE create #-}
839−create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
840−create = fmap unsafeFreeze MA.create
841−
842−-- XXX Deprecate in major
843−-- {-# DEPRECATED write "Please use create instead." #-}
844541{-# INLINE write #-}
845542write :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
846−write = create
543+write = fmap unsafeFreeze MA.write
847544
848−-- | Like 'create' but creates a pinned array.
849−{-# INLINE pinnedCreate #-}
850−pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
851−pinnedCreate = fmap unsafeFreeze MA.pinnedCreate
852−
853−{-# DEPRECATED pinnedWrite "Please use pinnedCreate instead." #-}
545+-- | Like 'write' but creates a pinned array.
854546{-# INLINE pinnedWrite #-}
855547pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
856−pinnedWrite = pinnedCreate
548+pinnedWrite = fmap unsafeFreeze MA.pinnedWrite
857549
858550-- | Fold "step" has a dependency on "initial", and each step is dependent on
859551-- the previous invocation of step due to state passing, finally extract
@@ -875,20 +567,15 @@ pinnedWrite = pinnedCreate
875567-- could be unsafe and dangerous. This is dangerous especially when used with
876568-- foldMany like operations.
877569--
570+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
878571-- >>> unsafePureWrite = Array.unsafeMakePure Array.write
879572--
880573{-# INLINE unsafeMakePure #-}
881574unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b
882−unsafeMakePure (Fold step initial extract final) =
575+unsafeMakePure (Fold step initial extract) =
883576 Fold (\x a -> return $! unsafeInlineIO (step x a))
884577 (return $! unsafePerformIO initial)
885578 (\s -> return $! unsafeInlineIO $ extract s)
886− (\s -> return $! unsafeInlineIO $ final s)
887−
888−{-# INLINE fromPureStreamN #-}
889−fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a
890−fromPureStreamN n x =
891− unsafePerformIO $ fmap unsafeFreeze (MA.fromPureStreamN n x)
892579
893580-- | Convert a pure stream in Identity monad to an immutable array.
894581--
@@ -901,19 +588,6 @@ fromPureStream x = unsafePerformIO $ fmap (unsafeFreeze) (MA.fromPureStream x)
901588-- fromPureStream = runIdentity . D.fold (unsafeMakePure write)
902589-- fromPureStream = fromList . runIdentity . D.toList
903590
904−-- XXX This should be monadic.
905−
906−-- | Copy an immutable 'Ptr Word8' sequence into an array.
907−--
908−-- /Unsafe:/ The caller is responsible for safe addressing.
909−--
910−-- Note that this should be evaluated strictly to ensure that we do not hold
911−-- the reference to the pointer in a lazy thunk.
912−fromPtrN :: Int -> Ptr Word8 -> Array Word8
913−fromPtrN n addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromPtrN n addr)
914−
915−-- XXX This should be monadic.
916−
917591-- | Copy a null terminated immutable 'Addr#' Word8 sequence into an array.
918592--
919593-- /Unsafe:/ The caller is responsible for safe addressing.
@@ -921,49 +595,13 @@ fromPtrN n addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromPtrN n addr)
921595-- Note that this is completely safe when reading from Haskell string
922596-- literals because they are guaranteed to be NULL terminated:
923597--
598+-- >>> :set -XMagicHash
599+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
924600-- >>> Array.toList $ Array.fromByteStr# "\1\2\3\0"#
925601-- [1,2,3]
926602--
927−-- Note that this should be evaluated strictly to ensure that we do not hold
928−-- the reference to the pointer in a lazy thunk.
929603fromByteStr# :: Addr# -> Array Word8
930−fromByteStr# addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromByteStr# addr)
931−
932−-- XXX This should be monadic.
933−
934−-- | Note that this should be evaluated strictly to ensure that we do not hold
935−-- the reference to the pointer in a lazy thunk.
936−fromByteStr :: Ptr Word8 -> Array Word8
937−fromByteStr (Ptr addr#) = fromByteStr# addr#
938−
939−-- XXX implement fromChunks/fromChunkList instead?
940−
941−-- | Convert an array stream to an array. Note that this requires peak memory
942−-- that is double the size of the array stream.
943−--
944−{-# INLINE fromChunksK #-}
945−fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)
946−fromChunksK stream =
947− -- We buffer the entire stream and then allocate the target array of the
948− -- same size, thus requiring double the memory.
949− fmap unsafeFreeze $ MA.fromChunksK $ fmap unsafeThaw stream
950−
951−{-# DEPRECATED fromArrayStreamK "Please use fromChunksK instead." #-}
952−fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (Array a) -> m (Array a)
953−fromArrayStreamK = fromChunksK
954−
955−-- | Given a stream of arrays, splice them all together to generate a single
956−-- array. The stream must be /finite/.
957−--
958−{-# INLINE fromChunks #-}
959−fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
960−fromChunks s =
961− -- XXX Check which implementation is better
962− -- This may also require double the memory as we double the space every
963− -- time, when copying the last array we may have reallocated almost double
964− -- the space required before we right size it.
965− fmap unsafeFreeze $ MA.fromChunksRealloced (fmap unsafeThaw s)
966− -- fromChunkStreamK $ D.toStreamK s
604+fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
967605
968606-------------------------------------------------------------------------------
969607-- Instances
@@ -995,66 +633,12 @@ instance Unbox a => IsList (Array a) where
995633 {-# INLINE toList #-}
996634 toList = toList
997635
998−-- | Byte compare two arrays. Compare the length of the arrays. If the length
999−-- is equal, compare the lexicographical ordering of two underlying byte arrays
1000−-- otherwise return the result of length comparison.
1001−--
1002−-- /Unsafe/: Note that the 'Unbox' instance of sum types with constructors of
1003−-- different sizes may leave some memory uninitialized which can make byte
1004−-- comparison unreliable.
1005−--
1006−-- /Pre-release/
1007−{-# INLINE byteCmp #-}
1008−byteCmp :: Array a -> Array a -> Ordering
1009−byteCmp arr1 arr2 =
1010− -- unsafePerformIO?
1011− unsafeInlineIO $! unsafeThaw arr1 `MA.byteCmp` unsafeThaw arr2
1012−
1013−-- | Byte equality of two arrays.
1014−--
1015−-- >>> byteEq arr1 arr2 = (==) EQ $ Array.byteCmp arr1 arr2
1016−--
1017−-- /Unsafe/: See 'byteCmp'.
1018−{-# INLINE byteEq #-}
1019−byteEq :: Array a -> Array a -> Bool
1020−byteEq arr1 arr2 = (==) EQ $ byteCmp arr1 arr2
1021−
1022−#define MK_EQ_INSTANCE(typ) \
1023−instance {-# OVERLAPPING #-} Eq (Array typ) where { \
1024−; {-# INLINE (==) #-} \
1025−; (==) = byteEq \
1026−}
1027−
1028−MK_EQ_INSTANCE(Char)
1029−MK_EQ_INSTANCE(Word8)
1030−MK_EQ_INSTANCE(Word16)
1031−MK_EQ_INSTANCE(Word32)
1032−
1033−-- XXX The Word64 default instance should be as fast because we are comparing
1034−-- 64-bit at a time.
1035−MK_EQ_INSTANCE(Word64)
1036−MK_EQ_INSTANCE(Int)
1037−MK_EQ_INSTANCE(Int8)
1038−MK_EQ_INSTANCE(Int16)
1039−MK_EQ_INSTANCE(Int32)
1040−
1041−-- XXX The Int64 default instance should be as fast.
1042−MK_EQ_INSTANCE(Int64)
1043−
1044−-- | If the type allows a byte-by-byte comparison this instance can be
1045−-- overlapped by a more specific instance that uses 'byteCmp'. Byte comparison
1046−-- can be significantly faster.
1047−--
1048−instance {-# OVERLAPPABLE #-} (Unbox a, Eq a) => Eq (Array a) where
636+-- XXX we are assuming that Unboxed equality means element equality. This may
637+-- or may not be correct? arrcmp is 40% faster compared to stream equality.
638+instance (Unbox a, Eq a) => Eq (Array a) where
1049639 {-# INLINE (==) #-}
1050640 arr1 == arr2 =
1051− -- Does unboxed byte equality mean element equality?
1052− -- XXX This is incorrect for sum types, as we may have some
1053− -- uninitialized memory in that case. If we always initialize the
1054− -- unused memory to zero we can use this.
1055− -- Byte comparison is 40% faster compared to stream equality.
1056− -- (==) EQ $ unsafeInlineIO $! unsafeThaw arr1 `MA.cmp` unsafeThaw arr2
1057− (toStreamD arr1 :: Stream Identity a) == toStreamD arr2
641+ (==) EQ $ unsafeInlineIO $! unsafeThaw arr1 `MA.cmp` unsafeThaw arr2
1058642
1059643instance (Unbox a, Ord a) => Ord (Array a) where
1060644 {-# INLINE compare #-}
@@ -1096,7 +680,7 @@ _toStreamD_ size Array{..} = D.Stream step arrStart
1096680 {-# INLINE_LATE step #-}
1097681 step _ p | p == arrEnd = return D.Stop
1098682 step _ p = liftIO $ do
1099− x <- peekAt p arrContents
683+ x <- peekByteIndex p arrContents
1100684 return $ D.Yield x (p + size)
1101685
1102686{-
@@ -1119,29 +703,15 @@ instance Foldable Array where
1119703-- Semigroup and Monoid
1120704-------------------------------------------------------------------------------
1121705
1122−-- XXX Deprecate and remove the Semigroup and Monoid instances because of
1123−-- potential misuse chances.
1124−
1125−-- | This should not be used for combining many or N arrays as it would copy
1126−-- the two arrays everytime to a new array. For coalescing multiple arrays use
1127−-- 'fromChunksK' instead.
1128706instance Unbox a => Semigroup (Array a) where
1129707 arr1 <> arr2 = unsafePerformIO $ splice arr1 arr2
1130708
1131−empty ::
1132−#ifdef DEVBUILD
1133− Unbox a =>
1134−#endif
1135− Array a
1136−empty = Array Unboxed.empty 0 0
1137−
1138−{-# DEPRECATED nil "Please use empty instead." #-}
1139709nil ::
1140710#ifdef DEVBUILD
1141711 Unbox a =>
1142712#endif
1143713 Array a
1144−nil = empty
714+nil = Array Unboxed.nil 0 0
1145715
1146716instance Unbox a => Monoid (Array a) where
1147717 mempty = nil
core/src/Streamly/Internal/Data/Builder.hs+9−11
@@ -16,9 +16,7 @@ module Streamly.Internal.Data.Builder
1616 )
1717where
1818
19−#if !MIN_VERSION_base(4,18,0)
2019import Control.Applicative (liftA2)
21−#endif
2220
2321------------------------------------------------------------------------------
2422-- The Builder type
@@ -29,16 +27,16 @@ import Control.Applicative (liftA2)
2927-- or even a Fold. Unlike fold the step function is one-shot and not called in
3028-- a loop.
3129newtype Builder s m a =
32− Builder (s -> m (a, s))
30+ Builder (s -> m (s, a))
3331
3432-- | Maps a function on the output of the fold (the type @b@).
3533instance Functor m => Functor (Builder s m) where
3634 {-# INLINE fmap #-}
37− fmap f (Builder step1) = Builder (fmap (\ (a, s) -> (f a, s)) . step1)
35+ fmap f (Builder step1) = Builder (fmap (fmap f) . step1)
3836
3937{-# INLINE fromPure #-}
4038fromPure :: Applicative m => b -> Builder s m b
41−fromPure b = Builder (\s -> pure (b, s))
39+fromPure b = Builder (\s -> pure (s, b))
4240
4341-- | Chain the actions and zip the outputs.
4442{-# INLINE sequenceWith #-}
@@ -49,9 +47,9 @@ sequenceWith func (Builder stepL) (Builder stepR) = Builder step
4947 where
5048
5149 step s = do
52− (x, s1) <- stepL s
53− (y, s2) <- stepR s1
54− pure (func x y, s2)
50+ (s1, x) <- stepL s
51+ (s2, y) <- stepR s1
52+ pure (s2, func x y)
5553
5654instance Monad m => Applicative (Builder a m) where
5755 {-# INLINE pure #-}
@@ -76,7 +74,7 @@ instance Monad m => Monad (Builder a m) where
7674 where
7775
7876 step s = do
79− (x, s1) <- stepL s
77+ (s1, x) <- stepL s
8078 let Builder stepR = f x
81− (y, s2) <- stepR s1
82− pure (y, s2)
79+ (s2, y) <- stepR s1
80+ pure (s2, y)
core/src/Streamly/Internal/Data/Fold.hs+2574−9
@@ -16,18 +16,2583 @@ module Streamly.Internal.Data.Fold
1616 -- * Imports
1717 -- $setup
1818
19− module Streamly.Internal.Data.Fold.Type
20− , module Streamly.Internal.Data.Fold.Tee
21− , module Streamly.Internal.Data.Fold.Combinators
22− , module Streamly.Internal.Data.Fold.Container
23− , module Streamly.Internal.Data.Fold.Window
19+ -- * Fold Type
20+ Step (..)
21+ , Fold (..)
22+ , Tee (..)
23+
24+ -- * Constructors
25+ -- | Which constructor to use?
26+ --
27+ -- * @foldl*@: If the fold never terminates i.e. does not use the 'Done'
28+ -- constructor otherwise use the @foldt*@ variants.
29+ -- * @*M@: Use the @M@ suffix variants if any of the step, initial, or
30+ -- extract function is monadic, otherwise use the pure variants.
31+ --
32+ , foldl'
33+ , foldlM'
34+ , foldl1'
35+ , foldlM1'
36+ , foldt'
37+ , foldtM'
38+ , foldr'
39+ , foldrM'
40+
41+ -- * Mappers
42+ -- | Monadic functions useful with mapM/lmapM on folds or streams.
43+ , tracing
44+ , trace
45+
46+ -- * Folds
47+
48+ -- ** Accumulators
49+ -- *** Semigroups and Monoids
50+ , sconcat
51+ , mconcat
52+ , foldMap
53+ , foldMapM
54+
55+ -- *** Reducers
56+ , drain
57+ , drainMapM
58+ , the
59+ , length
60+ , lengthGeneric
61+ , mean
62+ , rollingHash
63+ , defaultSalt
64+ , rollingHashWithSalt
65+ , rollingHashFirstN
66+ -- , rollingHashLastN
67+
68+ -- *** Saturating Reducers
69+ -- | 'product' terminates if it becomes 0. Other folds can theoretically
70+ -- saturate on bounded types, and therefore terminate, however, they will
71+ -- run forever on unbounded types like Integer/Double.
72+ , sum
73+ , product
74+ , maximumBy
75+ , maximum
76+ , minimumBy
77+ , minimum
78+
79+ -- *** Collectors
80+ -- | Avoid using these folds in scalable or performance critical
81+ -- applications, they buffer all the input in GC memory which can be
82+ -- detrimental to performance if the input is large.
83+ , toList
84+ , toListRev
85+ -- $toListRev
86+ , toStream
87+ , toStreamRev
88+ , toStreamK
89+ , toStreamKRev
90+ , topBy
91+ , top
92+ , bottomBy
93+ , bottom
94+
95+ -- *** Scanners
96+ -- | Stateful transformation of the elements. Useful in combination with
97+ -- the 'scanMaybe' combinator. For scanners the result of the fold is
98+ -- usually a transformation of the current element rather than an
99+ -- aggregation of all elements till now.
100+ , latest
101+ -- , nthLast -- using Ring array
102+ , indexingWith
103+ , indexing
104+ , indexingRev
105+ , rollingMapM
106+
107+ -- *** Filters
108+ -- | Useful in combination with the 'scanMaybe' combinator.
109+ , filtering
110+ , deleteBy
111+ , uniqBy
112+ , uniq
113+ , repeated
114+ , findIndices
115+ , elemIndices
116+
117+ -- ** Terminating Folds
118+ -- *** Empty folds
119+ -- | Folds that return a result without consuming any input.
120+ , fromPure
121+ , fromEffect
122+ , fromRefold
123+
124+ -- *** Singleton folds
125+ -- | Folds that terminate after consuming exactly one input element. All
126+ -- these can be implemented in terms of the 'maybe' fold.
127+ , one
128+ , null -- XXX not very useful and could be problematic, remove it?
129+ , satisfy
130+ , maybe
131+
132+ -- *** Multi folds
133+ -- | Terminate after consuming one or more elements.
134+ , drainN
135+ -- , lastN
136+ -- , (!!)
137+ , indexGeneric
138+ , index
139+ , findM
140+ , find
141+ , lookup
142+ , findIndex
143+ , elemIndex
144+ , elem
145+ , notElem
146+ , all
147+ , any
148+ , and
149+ , or
150+
151+ -- ** Trimmers
152+ -- | Useful in combination with the 'scanMaybe' combinator.
153+ , taking
154+ , dropping
155+ , takingEndByM
156+ , takingEndBy
157+ , takingEndByM_
158+ , takingEndBy_
159+ , droppingWhileM
160+ , droppingWhile
161+ , prune
162+
163+ -- * Running A Fold
164+ , drive
165+ -- , breakStream
166+
167+ -- * Building Incrementally
168+ , extractM
169+ , reduce
170+ , close
171+ , isClosed
172+ , snoc
173+ , snocl
174+ , snocM
175+ , snoclM
176+
177+ , addOne
178+ , addStream
179+
180+ -- * Combinators
181+ -- ** Utilities
182+ , with
183+
184+ -- ** Transforming the Monad
185+ , morphInner
186+ , generalizeInner
187+
188+ -- ** Mapping on output
189+ , rmapM
190+
191+ -- ** Mapping on Input
192+ , transform
193+ , lmap
194+ --, lsequence
195+ , lmapM
196+
197+ -- ** Sliding Window
198+ , slide2
199+
200+ -- ** Scanning Input
201+ , scan
202+ , scanMany
203+ , postscan
204+ , indexed
205+
206+ -- ** Zipping Input
207+ , zipStreamWithM
208+ , zipStream
209+
210+ -- ** Filtering Input
211+ , catMaybes
212+ , mapMaybeM
213+ , mapMaybe
214+ , scanMaybe
215+ , filter
216+ , filterM
217+ , sampleFromthen
218+
219+ -- Either streams
220+ , catLefts
221+ , catRights
222+ , catEithers
223+
224+ {-
225+ -- ** Insertion
226+ -- | Insertion adds more elements to the stream.
227+
228+ , insertBy
229+ , intersperseM
230+
231+ -- ** Reordering
232+ , reverse
233+ -}
234+
235+ -- ** Trimming
236+ , take
237+
238+ -- By elements
239+ , takeEndBy
240+ , takeEndBy_
241+ , takeEndBySeq
242+ , takeEndBySeq_
243+ {-
244+ , drop
245+ , dropWhile
246+ , dropWhileM
247+ -}
248+
249+ -- ** Serial Append
250+ , splitWith
251+ , split_
252+ -- , tail
253+ -- , init
254+ , splitAt -- spanN
255+ -- , splitIn -- sessionN
256+
257+ -- ** Parallel Distribution
258+ , teeWith
259+ , tee
260+ , teeWithFst
261+ , teeWithMin
262+ , distribute
263+ -- , distributeFst
264+ -- , distributeMin
265+
266+ -- ** Unzipping
267+ , unzip
268+ -- These two can be expressed using lmap/lmapM and unzip
269+ , unzipWith
270+ , unzipWithM
271+ , unzipWithFstM
272+ , unzipWithMinM
273+
274+ -- ** Parallel Alternative
275+ , shortest
276+ , longest
277+
278+ -- ** Partitioning
279+ , partitionByM
280+ , partitionByFstM
281+ , partitionByMinM
282+ , partitionBy
283+ , partition
284+
285+ -- ** Splitting
286+ , many
287+ , manyPost
288+ , groupsOf
289+ , chunksBetween
290+ , refoldMany
291+ , refoldMany1
292+ , intersperseWithQuotes
293+
294+ -- ** Nesting
295+ , unfoldMany
296+ , concatSequence
297+ , concatMap
298+ , duplicate
299+ , refold
300+
301+ -- * Deprecated
302+ , foldr
303+ , drainBy
304+ , last
305+ , head
306+ , sequence
307+ , mapM
308+ , variance
309+ , stdDev
310+ , serialWith
24311 )
25312where
26313
27−import Streamly.Internal.Data.Fold.Combinators
28−import Streamly.Internal.Data.Fold.Container
29−import Streamly.Internal.Data.Fold.Tee
314+#include "inline.hs"
315+#include "ArrayMacros.h"
316+
317+import Control.Monad (void)
318+import Control.Monad.IO.Class (MonadIO(..))
319+import Data.Bifunctor (first)
320+import Data.Bits (shiftL, shiftR, (.|.), (.&.))
321+import Data.Either (isLeft, isRight, fromLeft, fromRight)
322+import Data.Int (Int64)
323+import Data.Proxy (Proxy(..))
324+import Data.Word (Word32)
325+import Foreign.Storable (Storable, peek)
326+import Streamly.Internal.Data.MutArray.Type (MutArray(..))
327+import Streamly.Internal.Data.Maybe.Strict (Maybe'(..), toMaybe)
328+import Streamly.Internal.Data.Pipe.Type (Pipe (..), PipeState(..))
329+import Streamly.Internal.Data.Unbox (Unbox, sizeOf)
330+import Streamly.Internal.Data.Unfold.Type (Unfold(..))
331+import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3'(..))
332+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
333+
334+import qualified Prelude
335+import qualified Streamly.Internal.Data.MutArray.Type as MA
336+import qualified Streamly.Internal.Data.Array.Type as Array
337+import qualified Streamly.Internal.Data.Fold.Window as FoldW
338+import qualified Streamly.Internal.Data.Pipe.Type as Pipe
339+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
340+import qualified Streamly.Internal.Data.Stream.StreamD.Type as StreamD
341+
342+import Prelude hiding
343+ ( Foldable(..), filter, drop, dropWhile, take, takeWhile, zipWith
344+ , map, mapM_, sequence, all, any
345+ , notElem, head, last, tail
346+ , reverse, iterate, init, and, or, lookup, (!!)
347+ , scanl, scanl1, replicate, concatMap, mconcat, unzip
348+ , span, splitAt, break, mapM, zip, maybe)
30349import Streamly.Internal.Data.Fold.Type
31−import Streamly.Internal.Data.Fold.Window
350+import Streamly.Internal.Data.Fold.Tee
32351
33352#include "DocTestDataFold.hs"
353+
354+------------------------------------------------------------------------------
355+-- Running
356+------------------------------------------------------------------------------
357+
358+-- | Drive a fold using the supplied 'Stream', reducing the resulting
359+-- expression strictly at each step.
360+--
361+-- Definition:
362+--
363+-- >>> drive = flip Stream.fold
364+--
365+-- Example:
366+--
367+-- >>> Fold.drive (Stream.enumerateFromTo 1 100) Fold.sum
368+-- 5050
369+--
370+{-# INLINE drive #-}
371+drive :: Monad m => Stream m a -> Fold m a b -> m b
372+drive = flip StreamD.fold
373+
374+{-
375+-- | Like 'drive' but also returns the remaining stream. The resulting stream
376+-- would be 'Stream.nil' if the stream finished before the fold.
377+--
378+-- Definition:
379+--
380+-- >>> breakStream = flip Stream.foldBreak
381+--
382+-- /CPS/
383+--
384+{-# INLINE breakStreamK #-}
385+breakStreamK :: Monad m => StreamK m a -> Fold m a b -> m (b, StreamK m a)
386+breakStreamK strm fl = fmap f $ K.foldBreak fl (Stream.toStreamK strm)
387+
388+ where
389+
390+ f (b, str) = (b, Stream.fromStreamK str)
391+-}
392+
393+-- | Append a stream to a fold to build the fold accumulator incrementally. We
394+-- can repeatedly call 'addStream' on the same fold to continue building the
395+-- fold and finally use 'drive' to finish the fold and extract the result. Also
396+-- see the 'Streamly.Data.Fold.addOne' operation which is a singleton version
397+-- of 'addStream'.
398+--
399+-- Definitions:
400+--
401+-- >>> addStream stream = Fold.drive stream . Fold.duplicate
402+--
403+-- Example, build a list incrementally:
404+--
405+-- >>> :{
406+-- pure (Fold.toList :: Fold IO Int [Int])
407+-- >>= Fold.addOne 1
408+-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
409+-- >>= Fold.drive Stream.nil
410+-- >>= print
411+-- :}
412+-- [1,2,3,4]
413+--
414+-- This can be used as an O(n) list append compared to the O(n^2) @++@ when
415+-- used for incrementally building a list.
416+--
417+-- Example, build a stream incrementally:
418+--
419+-- >>> :{
420+-- pure (Fold.toStream :: Fold IO Int (Stream Identity Int))
421+-- >>= Fold.addOne 1
422+-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
423+-- >>= Fold.drive Stream.nil
424+-- >>= print
425+-- :}
426+-- fromList [1,2,3,4]
427+--
428+-- This can be used as an O(n) stream append compared to the O(n^2) @<>@ when
429+-- used for incrementally building a stream.
430+--
431+-- Example, build an array incrementally:
432+--
433+-- >>> :{
434+-- pure (Array.write :: Fold IO Int (Array Int))
435+-- >>= Fold.addOne 1
436+-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
437+-- >>= Fold.drive Stream.nil
438+-- >>= print
439+-- :}
440+-- fromList [1,2,3,4]
441+--
442+-- Example, build an array stream incrementally:
443+--
444+-- >>> :{
445+-- let f :: Fold IO Int (Stream Identity (Array Int))
446+-- f = Fold.groupsOf 2 (Array.writeN 3) Fold.toStream
447+-- in pure f
448+-- >>= Fold.addOne 1
449+-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
450+-- >>= Fold.drive Stream.nil
451+-- >>= print
452+-- :}
453+-- fromList [fromList [1,2],fromList [3,4]]
454+--
455+addStream :: Monad m => Stream m a -> Fold m a b -> m (Fold m a b)
456+addStream stream = drive stream . duplicate
457+
458+------------------------------------------------------------------------------
459+-- Transformations on fold inputs
460+------------------------------------------------------------------------------
461+
462+-- | Flatten the monadic output of a fold to pure output.
463+--
464+{-# DEPRECATED sequence "Use \"rmapM id\" instead" #-}
465+{-# INLINE sequence #-}
466+sequence :: Monad m => Fold m a (m b) -> Fold m a b
467+sequence = rmapM id
468+
469+-- | Map a monadic function on the output of a fold.
470+--
471+{-# DEPRECATED mapM "Use rmapM instead" #-}
472+{-# INLINE mapM #-}
473+mapM :: Monad m => (b -> m c) -> Fold m a b -> Fold m a c
474+mapM = rmapM
475+
476+-- |
477+-- >>> mapMaybeM f = Fold.lmapM f . Fold.catMaybes
478+--
479+{-# INLINE mapMaybeM #-}
480+mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Fold m b r -> Fold m a r
481+mapMaybeM f = lmapM f . catMaybes
482+
483+-- | @mapMaybe f fold@ maps a 'Maybe' returning function @f@ on the input of
484+-- the fold, filters out 'Nothing' elements, and return the values extracted
485+-- from 'Just'.
486+--
487+-- >>> mapMaybe f = Fold.lmap f . Fold.catMaybes
488+-- >>> mapMaybe f = Fold.mapMaybeM (return . f)
489+--
490+-- >>> f x = if even x then Just x else Nothing
491+-- >>> fld = Fold.mapMaybe f Fold.toList
492+-- >>> Stream.fold fld (Stream.enumerateFromTo 1 10)
493+-- [2,4,6,8,10]
494+--
495+{-# INLINE mapMaybe #-}
496+mapMaybe :: Monad m => (a -> Maybe b) -> Fold m b r -> Fold m a r
497+mapMaybe f = lmap f . catMaybes
498+
499+------------------------------------------------------------------------------
500+-- Transformations on fold inputs
501+------------------------------------------------------------------------------
502+
503+-- | Apply a monadic function on the input and return the input.
504+--
505+-- >>> Stream.fold (Fold.lmapM (Fold.tracing print) Fold.drain) $ (Stream.enumerateFromTo (1 :: Int) 2)
506+-- 1
507+-- 2
508+--
509+-- /Pre-release/
510+--
511+{-# INLINE tracing #-}
512+tracing :: Monad m => (a -> m b) -> (a -> m a)
513+tracing f x = void (f x) >> return x
514+
515+-- | Apply a monadic function to each element flowing through and discard the
516+-- results.
517+--
518+-- >>> Stream.fold (Fold.trace print Fold.drain) $ (Stream.enumerateFromTo (1 :: Int) 2)
519+-- 1
520+-- 2
521+--
522+-- >>> trace f = Fold.lmapM (Fold.tracing f)
523+--
524+-- /Pre-release/
525+{-# INLINE trace #-}
526+trace :: Monad m => (a -> m b) -> Fold m a r -> Fold m a r
527+trace f = lmapM (tracing f)
528+
529+-- rename to lpipe?
530+--
531+-- | Apply a transformation on a 'Fold' using a 'Pipe'.
532+--
533+-- /Pre-release/
534+{-# INLINE transform #-}
535+transform :: Monad m => Pipe m a b -> Fold m b c -> Fold m a c
536+transform (Pipe pstep1 pstep2 pinitial) (Fold fstep finitial fextract) =
537+ Fold step initial extract
538+
539+ where
540+
541+ initial = first (Tuple' pinitial) <$> finitial
542+
543+ step (Tuple' ps fs) x = do
544+ r <- pstep1 ps x
545+ go fs r
546+
547+ where
548+
549+ -- XXX use SPEC?
550+ go acc (Pipe.Yield b (Consume ps')) = do
551+ acc' <- fstep acc b
552+ return
553+ $ case acc' of
554+ Partial s -> Partial $ Tuple' ps' s
555+ Done b2 -> Done b2
556+ go acc (Pipe.Yield b (Produce ps')) = do
557+ acc' <- fstep acc b
558+ r <- pstep2 ps'
559+ case acc' of
560+ Partial s -> go s r
561+ Done b2 -> return $ Done b2
562+ go acc (Pipe.Continue (Consume ps')) =
563+ return $ Partial $ Tuple' ps' acc
564+ go acc (Pipe.Continue (Produce ps')) = do
565+ r <- pstep2 ps'
566+ go acc r
567+
568+ extract (Tuple' _ fs) = fextract fs
569+
570+{-# INLINE scanWith #-}
571+scanWith :: Monad m => Bool -> Fold m a b -> Fold m b c -> Fold m a c
572+scanWith isMany (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
573+ Fold step initial extract
574+
575+ where
576+
577+ {-# INLINE runStep #-}
578+ runStep actionL sR = do
579+ rL <- actionL
580+ case rL of
581+ Done bL -> do
582+ rR <- stepR sR bL
583+ case rR of
584+ Partial sR1 ->
585+ if isMany
586+ then runStep initialL sR1
587+ else Done <$> extractR sR1
588+ Done bR -> return $ Done bR
589+ Partial sL -> do
590+ !b <- extractL sL
591+ rR <- stepR sR b
592+ return
593+ $ case rR of
594+ Partial sR1 -> Partial (sL, sR1)
595+ Done bR -> Done bR
596+
597+ initial = do
598+ r <- initialR
599+ case r of
600+ Partial sR -> runStep initialL sR
601+ Done b -> return $ Done b
602+
603+ step (sL, sR) x = runStep (stepL sL x) sR
604+
605+ extract = extractR . snd
606+
607+-- | Scan the input of a 'Fold' to change it in a stateful manner using another
608+-- 'Fold'. The scan stops as soon as the fold terminates.
609+--
610+-- /Pre-release/
611+{-# INLINE scan #-}
612+scan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
613+scan = scanWith False
614+
615+-- XXX This does not fuse beacuse of the recursive step. Need to investigate.
616+--
617+-- | Scan the input of a 'Fold' to change it in a stateful manner using another
618+-- 'Fold'. The scan restarts with a fresh state if the fold terminates.
619+--
620+-- /Pre-release/
621+{-# INLINE scanMany #-}
622+scanMany :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
623+scanMany = scanWith True
624+
625+------------------------------------------------------------------------------
626+-- Filters
627+------------------------------------------------------------------------------
628+
629+-- | Returns the latest element omitting the first occurrence that satisfies
630+-- the given equality predicate.
631+--
632+-- Example:
633+--
634+-- >>> input = Stream.fromList [1,3,3,5]
635+-- >>> Stream.fold Fold.toList $ Stream.scanMaybe (Fold.deleteBy (==) 3) input
636+-- [1,3,5]
637+--
638+{-# INLINE_NORMAL deleteBy #-}
639+deleteBy :: Monad m => (a -> a -> Bool) -> a -> Fold m a (Maybe a)
640+deleteBy eq x0 = fmap extract $ foldl' step (Tuple' False Nothing)
641+
642+ where
643+
644+ step (Tuple' False _) x =
645+ if eq x x0
646+ then Tuple' True Nothing
647+ else Tuple' False (Just x)
648+ step (Tuple' True _) x = Tuple' True (Just x)
649+
650+ extract (Tuple' _ x) = x
651+
652+-- | Provide a sliding window of length 2 elements.
653+--
654+-- See "Streamly.Internal.Data.Fold.Window".
655+--
656+{-# INLINE slide2 #-}
657+slide2 :: Monad m => Fold m (a, Maybe a) b -> Fold m a b
658+slide2 (Fold step1 initial1 extract1) = Fold step initial extract
659+
660+ where
661+
662+ initial =
663+ first (Tuple' Nothing) <$> initial1
664+
665+ step (Tuple' prev s) cur =
666+ first (Tuple' (Just cur)) <$> step1 s (cur, prev)
667+
668+ extract (Tuple' _ s) = extract1 s
669+
670+-- | Return the latest unique element using the supplied comparison function.
671+-- Returns 'Nothing' if the current element is same as the last element
672+-- otherwise returns 'Just'.
673+--
674+-- Example, strip duplicate path separators:
675+--
676+-- >>> input = Stream.fromList "//a//b"
677+-- >>> f x y = x == '/' && y == '/'
678+-- >>> Stream.fold Fold.toList $ Stream.scanMaybe (Fold.uniqBy f) input
679+-- "/a/b"
680+--
681+-- Space: @O(1)@
682+--
683+-- /Pre-release/
684+--
685+{-# INLINE uniqBy #-}
686+uniqBy :: Monad m => (a -> a -> Bool) -> Fold m a (Maybe a)
687+uniqBy eq = rollingMap f
688+
689+ where
690+
691+ f pre curr =
692+ case pre of
693+ Nothing -> Just curr
694+ Just x -> if x `eq` curr then Nothing else Just curr
695+
696+-- | See 'uniqBy'.
697+--
698+-- Definition:
699+--
700+-- >>> uniq = Fold.uniqBy (==)
701+--
702+{-# INLINE uniq #-}
703+uniq :: (Monad m, Eq a) => Fold m a (Maybe a)
704+uniq = uniqBy (==)
705+
706+-- | Strip all leading and trailing occurrences of an element passing a
707+-- predicate and make all other consecutive occurrences uniq.
708+--
709+-- >> prune p = Stream.dropWhileAround p $ Stream.uniqBy (x y -> p x && p y)
710+--
711+-- @
712+-- > Stream.prune isSpace (Stream.fromList " hello world! ")
713+-- "hello world!"
714+--
715+-- @
716+--
717+-- Space: @O(1)@
718+--
719+-- /Unimplemented/
720+{-# INLINE prune #-}
721+prune ::
722+ -- (Monad m, Eq a) =>
723+ (a -> Bool) -> Fold m a (Maybe a)
724+prune = error "Not implemented yet!"
725+
726+-- | Emit only repeated elements, once.
727+--
728+-- /Unimplemented/
729+repeated :: -- (Monad m, Eq a) =>
730+ Fold m a (Maybe a)
731+repeated = error "Not implemented yet!"
732+
733+------------------------------------------------------------------------------
734+-- Left folds
735+------------------------------------------------------------------------------
736+
737+------------------------------------------------------------------------------
738+-- Run Effects
739+------------------------------------------------------------------------------
740+
741+-- |
742+-- Definitions:
743+--
744+-- >>> drainMapM f = Fold.lmapM f Fold.drain
745+-- >>> drainMapM f = Fold.foldMapM (void . f)
746+--
747+-- Drain all input after passing it through a monadic function. This is the
748+-- dual of mapM_ on stream producers.
749+--
750+{-# INLINE drainMapM #-}
751+drainMapM :: Monad m => (a -> m b) -> Fold m a ()
752+drainMapM f = lmapM f drain
753+
754+{-# DEPRECATED drainBy "Please use 'drainMapM' instead." #-}
755+{-# INLINE drainBy #-}
756+drainBy :: Monad m => (a -> m b) -> Fold m a ()
757+drainBy = drainMapM
758+
759+-- | Returns the latest element of the input stream, if any.
760+--
761+-- >>> latest = Fold.foldl1' (\_ x -> x)
762+-- >>> latest = fmap getLast $ Fold.foldMap (Last . Just)
763+--
764+{-# INLINE latest #-}
765+latest :: Monad m => Fold m a (Maybe a)
766+latest = foldl1' (\_ x -> x)
767+
768+{-# DEPRECATED last "Please use 'latest' instead." #-}
769+{-# INLINE last #-}
770+last :: Monad m => Fold m a (Maybe a)
771+last = latest
772+
773+-- | Terminates with 'Nothing' as soon as it finds an element different than
774+-- the previous one, returns 'the' element if the entire input consists of the
775+-- same element.
776+--
777+{-# INLINE the #-}
778+the :: (Monad m, Eq a) => Fold m a (Maybe a)
779+the = foldt' step initial id
780+
781+ where
782+
783+ initial = Partial Nothing
784+
785+ step Nothing x = Partial (Just x)
786+ step old@(Just x0) x =
787+ if x0 == x
788+ then Partial old
789+ else Done Nothing
790+
791+------------------------------------------------------------------------------
792+-- To Summary
793+------------------------------------------------------------------------------
794+
795+-- | Like 'length', except with a more general 'Num' return value
796+--
797+-- Definition:
798+--
799+-- >>> lengthGeneric = fmap getSum $ Fold.foldMap (Sum . const 1)
800+-- >>> lengthGeneric = Fold.foldl' (\n _ -> n + 1) 0
801+--
802+-- /Pre-release/
803+{-# INLINE lengthGeneric #-}
804+lengthGeneric :: (Monad m, Num b) => Fold m a b
805+lengthGeneric = foldl' (\n _ -> n + 1) 0
806+
807+-- | Determine the length of the input stream.
808+--
809+-- Definition:
810+--
811+-- >>> length = Fold.lengthGeneric
812+-- >>> length = fmap getSum $ Fold.foldMap (Sum . const 1)
813+--
814+{-# INLINE length #-}
815+length :: Monad m => Fold m a Int
816+length = lengthGeneric
817+
818+
819+-- | Determine the sum of all elements of a stream of numbers. Returns additive
820+-- identity (@0@) when the stream is empty. Note that this is not numerically
821+-- stable for floating point numbers.
822+--
823+-- >>> sum = FoldW.cumulative FoldW.sum
824+--
825+-- Same as following but numerically stable:
826+--
827+-- >>> sum = Fold.foldl' (+) 0
828+-- >>> sum = fmap Data.Monoid.getSum $ Fold.foldMap Data.Monoid.Sum
829+--
830+{-# INLINE sum #-}
831+sum :: (Monad m, Num a) => Fold m a a
832+sum = FoldW.cumulative FoldW.sum
833+
834+-- | Determine the product of all elements of a stream of numbers. Returns
835+-- multiplicative identity (@1@) when the stream is empty. The fold terminates
836+-- when it encounters (@0@) in its input.
837+--
838+-- Same as the following but terminates on multiplication by @0@:
839+--
840+-- >>> product = fmap Data.Monoid.getProduct $ Fold.foldMap Data.Monoid.Product
841+--
842+{-# INLINE product #-}
843+product :: (Monad m, Num a, Eq a) => Fold m a a
844+product = foldt' step (Partial 1) id
845+
846+ where
847+
848+ step x a =
849+ if a == 0
850+ then Done 0
851+ else Partial $ x * a
852+
853+------------------------------------------------------------------------------
854+-- To Summary (Maybe)
855+------------------------------------------------------------------------------
856+
857+-- | Determine the maximum element in a stream using the supplied comparison
858+-- function.
859+--
860+{-# INLINE maximumBy #-}
861+maximumBy :: Monad m => (a -> a -> Ordering) -> Fold m a (Maybe a)
862+maximumBy cmp = foldl1' max'
863+
864+ where
865+
866+ max' x y =
867+ case cmp x y of
868+ GT -> x
869+ _ -> y
870+
871+-- | Determine the maximum element in a stream.
872+--
873+-- Definitions:
874+--
875+-- >>> maximum = Fold.maximumBy compare
876+-- >>> maximum = Fold.foldl1' max
877+--
878+-- Same as the following but without a default maximum. The 'Max' Monoid uses
879+-- the 'minBound' as the default maximum:
880+--
881+-- >>> maximum = fmap Data.Semigroup.getMax $ Fold.foldMap Data.Semigroup.Max
882+--
883+{-# INLINE maximum #-}
884+maximum :: (Monad m, Ord a) => Fold m a (Maybe a)
885+maximum = foldl1' max
886+
887+-- | Computes the minimum element with respect to the given comparison function
888+--
889+{-# INLINE minimumBy #-}
890+minimumBy :: Monad m => (a -> a -> Ordering) -> Fold m a (Maybe a)
891+minimumBy cmp = foldl1' min'
892+
893+ where
894+
895+ min' x y =
896+ case cmp x y of
897+ GT -> y
898+ _ -> x
899+
900+-- | Determine the minimum element in a stream using the supplied comparison
901+-- function.
902+--
903+-- Definitions:
904+--
905+-- >>> minimum = Fold.minimumBy compare
906+-- >>> minimum = Fold.foldl1' min
907+--
908+-- Same as the following but without a default minimum. The 'Min' Monoid uses the
909+-- 'maxBound' as the default maximum:
910+--
911+-- >>> maximum = fmap Data.Semigroup.getMin $ Fold.foldMap Data.Semigroup.Min
912+--
913+{-# INLINE minimum #-}
914+minimum :: (Monad m, Ord a) => Fold m a (Maybe a)
915+minimum = foldl1' min
916+
917+------------------------------------------------------------------------------
918+-- To Summary (Statistical)
919+------------------------------------------------------------------------------
920+
921+-- | Compute a numerically stable arithmetic mean of all elements in the input
922+-- stream.
923+--
924+{-# INLINE mean #-}
925+mean :: (Monad m, Fractional a) => Fold m a a
926+mean = fmap done $ foldl' step begin
927+
928+ where
929+
930+ begin = Tuple' 0 0
931+
932+ step (Tuple' x n) y =
933+ let n1 = n + 1
934+ in Tuple' (x + (y - x) / n1) n1
935+
936+ done (Tuple' x _) = x
937+
938+-- | Compute a numerically stable (population) variance over all elements in
939+-- the input stream.
940+--
941+{-# DEPRECATED variance "Use the streamly-statistics package instead" #-}
942+{-# INLINE variance #-}
943+variance :: (Monad m, Fractional a) => Fold m a a
944+variance = fmap done $ foldl' step begin
945+
946+ where
947+
948+ begin = Tuple3' 0 0 0
949+
950+ step (Tuple3' n mean_ m2) x = Tuple3' n' mean' m2'
951+
952+ where
953+
954+ n' = n + 1
955+ mean' = (n * mean_ + x) / (n + 1)
956+ delta = x - mean_
957+ m2' = m2 + delta * delta * n / (n + 1)
958+
959+ done (Tuple3' n _ m2) = m2 / n
960+
961+-- | Compute a numerically stable (population) standard deviation over all
962+-- elements in the input stream.
963+--
964+{-# DEPRECATED stdDev "Use the streamly-statistics package instead" #-}
965+{-# INLINE stdDev #-}
966+stdDev :: (Monad m, Floating a) => Fold m a a
967+stdDev = sqrt <$> variance
968+
969+-- | Compute an 'Int' sized polynomial rolling hash
970+--
971+-- > H = salt * k ^ n + c1 * k ^ (n - 1) + c2 * k ^ (n - 2) + ... + cn * k ^ 0
972+--
973+-- Where @c1@, @c2@, @cn@ are the elements in the input stream and @k@ is a
974+-- constant.
975+--
976+-- This hash is often used in Rabin-Karp string search algorithm.
977+--
978+-- See https://en.wikipedia.org/wiki/Rolling_hash
979+--
980+{-# INLINE rollingHashWithSalt #-}
981+rollingHashWithSalt :: (Monad m, Enum a) => Int64 -> Fold m a Int64
982+rollingHashWithSalt = foldl' step
983+
984+ where
985+
986+ k = 2891336453 :: Int64
987+
988+ step cksum a = cksum * k + fromIntegral (fromEnum a)
989+
990+-- | A default salt used in the implementation of 'rollingHash'.
991+{-# INLINE defaultSalt #-}
992+defaultSalt :: Int64
993+defaultSalt = -2578643520546668380
994+
995+-- | Compute an 'Int' sized polynomial rolling hash of a stream.
996+--
997+-- >>> rollingHash = Fold.rollingHashWithSalt Fold.defaultSalt
998+--
999+{-# INLINE rollingHash #-}
1000+rollingHash :: (Monad m, Enum a) => Fold m a Int64
1001+rollingHash = rollingHashWithSalt defaultSalt
1002+
1003+-- | Compute an 'Int' sized polynomial rolling hash of the first n elements of
1004+-- a stream.
1005+--
1006+-- >>> rollingHashFirstN n = Fold.take n Fold.rollingHash
1007+--
1008+-- /Pre-release/
1009+{-# INLINE rollingHashFirstN #-}
1010+rollingHashFirstN :: (Monad m, Enum a) => Int -> Fold m a Int64
1011+rollingHashFirstN n = take n rollingHash
1012+
1013+-- XXX Compare this with the implementation in Fold.Window, preferrably use the
1014+-- latter if performance is good.
1015+
1016+-- | Apply a function on every two successive elements of a stream. The first
1017+-- argument of the map function is the previous element and the second argument
1018+-- is the current element. When processing the very first element in the
1019+-- stream, the previous element is 'Nothing'.
1020+--
1021+-- /Pre-release/
1022+--
1023+{-# INLINE rollingMapM #-}
1024+rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Fold m a b
1025+rollingMapM f = Fold step initial extract
1026+
1027+ where
1028+
1029+ -- XXX We need just a postscan. We do not need an initial result here.
1030+ -- Or we can supply a default initial result as an argument to rollingMapM.
1031+ initial = return $ Partial (Nothing, error "Empty stream")
1032+
1033+ step (prev, _) cur = do
1034+ x <- f prev cur
1035+ return $ Partial (Just cur, x)
1036+
1037+ extract = return . snd
1038+
1039+-- |
1040+-- >>> rollingMap f = Fold.rollingMapM (\x y -> return $ f x y)
1041+--
1042+{-# INLINE rollingMap #-}
1043+rollingMap :: Monad m => (Maybe a -> a -> b) -> Fold m a b
1044+rollingMap f = rollingMapM (\x y -> return $ f x y)
1045+
1046+------------------------------------------------------------------------------
1047+-- Monoidal left folds
1048+------------------------------------------------------------------------------
1049+
1050+-- | Semigroup concat. Append the elements of an input stream to a provided
1051+-- starting value.
1052+--
1053+-- Definition:
1054+--
1055+-- >>> sconcat = Fold.foldl' (<>)
1056+--
1057+-- >>> semigroups = fmap Data.Monoid.Sum $ Stream.enumerateFromTo 1 10
1058+-- >>> Stream.fold (Fold.sconcat 10) semigroups
1059+-- Sum {getSum = 65}
1060+--
1061+{-# INLINE sconcat #-}
1062+sconcat :: (Monad m, Semigroup a) => a -> Fold m a a
1063+sconcat = foldl' (<>)
1064+
1065+-- | Monoid concat. Fold an input stream consisting of monoidal elements using
1066+-- 'mappend' and 'mempty'.
1067+--
1068+-- Definition:
1069+--
1070+-- >>> mconcat = Fold.sconcat mempty
1071+--
1072+-- >>> monoids = fmap Data.Monoid.Sum $ Stream.enumerateFromTo 1 10
1073+-- >>> Stream.fold Fold.mconcat monoids
1074+-- Sum {getSum = 55}
1075+--
1076+{-# INLINE mconcat #-}
1077+mconcat ::
1078+ ( Monad m
1079+ , Monoid a) => Fold m a a
1080+mconcat = sconcat mempty
1081+
1082+-- |
1083+-- Definition:
1084+--
1085+-- >>> foldMap f = Fold.lmap f Fold.mconcat
1086+--
1087+-- Make a fold from a pure function that folds the output of the function
1088+-- using 'mappend' and 'mempty'.
1089+--
1090+-- >>> sum = Fold.foldMap Data.Monoid.Sum
1091+-- >>> Stream.fold sum $ Stream.enumerateFromTo 1 10
1092+-- Sum {getSum = 55}
1093+--
1094+{-# INLINE foldMap #-}
1095+foldMap :: (Monad m, Monoid b) => (a -> b) -> Fold m a b
1096+foldMap f = lmap f mconcat
1097+
1098+-- |
1099+-- Definition:
1100+--
1101+-- >>> foldMapM f = Fold.lmapM f Fold.mconcat
1102+--
1103+-- Make a fold from a monadic function that folds the output of the function
1104+-- using 'mappend' and 'mempty'.
1105+--
1106+-- >>> sum = Fold.foldMapM (return . Data.Monoid.Sum)
1107+-- >>> Stream.fold sum $ Stream.enumerateFromTo 1 10
1108+-- Sum {getSum = 55}
1109+--
1110+{-# INLINE foldMapM #-}
1111+foldMapM :: (Monad m, Monoid b) => (a -> m b) -> Fold m a b
1112+foldMapM act = foldlM' step (pure mempty)
1113+
1114+ where
1115+
1116+ step m a = do
1117+ m' <- act a
1118+ return $! mappend m m'
1119+
1120+------------------------------------------------------------------------------
1121+-- To Containers
1122+------------------------------------------------------------------------------
1123+
1124+-- $toListRev
1125+-- This is more efficient than 'Streamly.Internal.Data.Fold.toList'. toList is
1126+-- exactly the same as reversing the list after 'toListRev'.
1127+
1128+-- | Buffers the input stream to a list in the reverse order of the input.
1129+--
1130+-- Definition:
1131+--
1132+-- >>> toListRev = Fold.foldl' (flip (:)) []
1133+--
1134+-- /Warning!/ working on large lists accumulated as buffers in memory could be
1135+-- very inefficient, consider using "Streamly.Array" instead.
1136+--
1137+
1138+-- xn : ... : x2 : x1 : []
1139+{-# INLINE toListRev #-}
1140+toListRev :: Monad m => Fold m a [a]
1141+toListRev = foldl' (flip (:)) []
1142+
1143+------------------------------------------------------------------------------
1144+-- Partial Folds
1145+------------------------------------------------------------------------------
1146+
1147+-- | A fold that drains the first n elements of its input, running the effects
1148+-- and discarding the results.
1149+--
1150+-- Definition:
1151+--
1152+-- >>> drainN n = Fold.take n Fold.drain
1153+--
1154+-- /Pre-release/
1155+{-# INLINE drainN #-}
1156+drainN :: Monad m => Int -> Fold m a ()
1157+drainN n = take n drain
1158+
1159+------------------------------------------------------------------------------
1160+-- To Elements
1161+------------------------------------------------------------------------------
1162+
1163+-- | Like 'index', except with a more general 'Integral' argument
1164+--
1165+-- /Pre-release/
1166+{-# INLINE indexGeneric #-}
1167+indexGeneric :: (Integral i, Monad m) => i -> Fold m a (Maybe a)
1168+indexGeneric i = foldt' step (Partial 0) (const Nothing)
1169+
1170+ where
1171+
1172+ step j a =
1173+ if i == j
1174+ then Done $ Just a
1175+ else Partial (j + 1)
1176+
1177+-- | Return the element at the given index.
1178+--
1179+-- Definition:
1180+--
1181+-- >>> index = Fold.indexGeneric
1182+--
1183+{-# INLINE index #-}
1184+index :: Monad m => Int -> Fold m a (Maybe a)
1185+index = indexGeneric
1186+
1187+-- | Consume a single input and transform it using the supplied 'Maybe'
1188+-- returning function.
1189+--
1190+-- /Pre-release/
1191+--
1192+{-# INLINE maybe #-}
1193+maybe :: Monad m => (a -> Maybe b) -> Fold m a (Maybe b)
1194+maybe f = foldt' (const (Done . f)) (Partial Nothing) id
1195+
1196+-- | Consume a single element and return it if it passes the predicate else
1197+-- return 'Nothing'.
1198+--
1199+-- Definition:
1200+--
1201+-- >>> satisfy f = Fold.maybe (\a -> if f a then Just a else Nothing)
1202+--
1203+-- /Pre-release/
1204+{-# INLINE satisfy #-}
1205+satisfy :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1206+satisfy f = maybe (\a -> if f a then Just a else Nothing)
1207+{-
1208+satisfy f = Fold step (return $ Partial ()) (const (return Nothing))
1209+
1210+ where
1211+
1212+ step () a = return $ Done $ if f a then Just a else Nothing
1213+-}
1214+
1215+-- Naming notes:
1216+--
1217+-- "head" and "next" are two alternative names for the same API. head sounds
1218+-- apt in the context of lists but next sounds more apt in the context of
1219+-- streams where we think in terms of generating and consuming the next element
1220+-- rather than taking the head of some static/persistent structure.
1221+--
1222+-- We also want to keep the nomenclature consistent across folds and parsers,
1223+-- "head" becomes even more unintuitive for parsers because there are two
1224+-- possible variants viz. peek and next.
1225+--
1226+-- Also, the "head" fold creates confusion in situations like
1227+-- https://github.com/composewell/streamly/issues/1404 where intuitive
1228+-- expectation from head is to consume the entire stream and just give us the
1229+-- head. There we want to convey the notion that we consume one element from
1230+-- the stream and stop. The name "one" already being used in parsers for this
1231+-- purpose sounds more apt from this perspective.
1232+--
1233+-- The source of confusion is perhaps due to the fact that some folds consume
1234+-- the entire stream and others terminate early. It may have been clearer if we
1235+-- had separate abstractions for the two use cases.
1236+
1237+-- XXX We can possibly use "head" for the purposes of reducing the entire
1238+-- stream to the head element i.e. take the head and drain the rest.
1239+
1240+-- | Take one element from the stream and stop.
1241+--
1242+-- Definition:
1243+--
1244+-- >>> one = Fold.maybe Just
1245+--
1246+-- This is similar to the stream 'Stream.uncons' operation.
1247+--
1248+{-# INLINE one #-}
1249+one :: Monad m => Fold m a (Maybe a)
1250+one = maybe Just
1251+
1252+-- | Extract the first element of the stream, if any.
1253+--
1254+-- >>> head = Fold.one
1255+--
1256+{-# DEPRECATED head "Please use \"one\" instead" #-}
1257+{-# INLINE head #-}
1258+head :: Monad m => Fold m a (Maybe a)
1259+head = one
1260+
1261+-- | Returns the first element that satisfies the given predicate.
1262+--
1263+-- /Pre-release/
1264+{-# INLINE findM #-}
1265+findM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1266+findM predicate = Fold step (return $ Partial ()) (const $ return Nothing)
1267+
1268+ where
1269+
1270+ step () a =
1271+ let f r =
1272+ if r
1273+ then Done (Just a)
1274+ else Partial ()
1275+ in f <$> predicate a
1276+
1277+-- | Returns the first element that satisfies the given predicate.
1278+--
1279+{-# INLINE find #-}
1280+find :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1281+find p = findM (return . p)
1282+
1283+-- | In a stream of (key-value) pairs @(a, b)@, return the value @b@ of the
1284+-- first pair where the key equals the given value @a@.
1285+--
1286+-- Definition:
1287+--
1288+-- >>> lookup x = fmap snd <$> Fold.find ((== x) . fst)
1289+--
1290+{-# INLINE lookup #-}
1291+lookup :: (Eq a, Monad m) => a -> Fold m (a,b) (Maybe b)
1292+lookup a0 = foldt' step (Partial ()) (const Nothing)
1293+
1294+ where
1295+
1296+ step () (a, b) =
1297+ if a == a0
1298+ then Done $ Just b
1299+ else Partial ()
1300+
1301+-- | Returns the first index that satisfies the given predicate.
1302+--
1303+{-# INLINE findIndex #-}
1304+findIndex :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
1305+findIndex predicate = foldt' step (Partial 0) (const Nothing)
1306+
1307+ where
1308+
1309+ step i a =
1310+ if predicate a
1311+ then Done $ Just i
1312+ else Partial (i + 1)
1313+
1314+-- | Returns the index of the latest element if the element satisfies the given
1315+-- predicate.
1316+--
1317+{-# INLINE findIndices #-}
1318+findIndices :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
1319+findIndices predicate =
1320+ -- XXX implement by combining indexing and filtering scans
1321+ fmap (either (const Nothing) Just) $ foldl' step (Left (-1))
1322+
1323+ where
1324+
1325+ step i a =
1326+ if predicate a
1327+ then Right (either id id i + 1)
1328+ else Left (either id id i + 1)
1329+
1330+-- | Returns the index of the latest element if the element matches the given
1331+-- value.
1332+--
1333+-- Definition:
1334+--
1335+-- >>> elemIndices a = Fold.findIndices (== a)
1336+--
1337+{-# INLINE elemIndices #-}
1338+elemIndices :: (Monad m, Eq a) => a -> Fold m a (Maybe Int)
1339+elemIndices a = findIndices (== a)
1340+
1341+-- | Returns the first index where a given value is found in the stream.
1342+--
1343+-- Definition:
1344+--
1345+-- >>> elemIndex a = Fold.findIndex (== a)
1346+--
1347+{-# INLINE elemIndex #-}
1348+elemIndex :: (Eq a, Monad m) => a -> Fold m a (Maybe Int)
1349+elemIndex a = findIndex (== a)
1350+
1351+------------------------------------------------------------------------------
1352+-- To Boolean
1353+------------------------------------------------------------------------------
1354+
1355+-- Similar to 'eof' parser, but the fold consumes and discards an input element
1356+-- when not at eof. XXX Remove or Rename to "eof"?
1357+
1358+-- | Consume one element, return 'True' if successful else return 'False'. In
1359+-- other words, test if the input is empty or not.
1360+--
1361+-- WARNING! It consumes one element if the stream is not empty. If that is not
1362+-- what you want please use the eof parser instead.
1363+--
1364+-- Definition:
1365+--
1366+-- >>> null = fmap isJust Fold.one
1367+--
1368+{-# INLINE null #-}
1369+null :: Monad m => Fold m a Bool
1370+null = foldt' (\() _ -> Done False) (Partial ()) (const True)
1371+
1372+-- | Returns 'True' if any element of the input satisfies the predicate.
1373+--
1374+-- Definition:
1375+--
1376+-- >>> any p = Fold.lmap p Fold.or
1377+--
1378+-- Example:
1379+--
1380+-- >>> Stream.fold (Fold.any (== 0)) $ Stream.fromList [1,0,1]
1381+-- True
1382+--
1383+{-# INLINE any #-}
1384+any :: Monad m => (a -> Bool) -> Fold m a Bool
1385+any predicate = foldt' step initial id
1386+
1387+ where
1388+
1389+ initial = Partial False
1390+
1391+ step _ a =
1392+ if predicate a
1393+ then Done True
1394+ else Partial False
1395+
1396+-- | Return 'True' if the given element is present in the stream.
1397+--
1398+-- Definition:
1399+--
1400+-- >>> elem a = Fold.any (== a)
1401+--
1402+{-# INLINE elem #-}
1403+elem :: (Eq a, Monad m) => a -> Fold m a Bool
1404+elem a = any (== a)
1405+
1406+-- | Returns 'True' if all elements of the input satisfy the predicate.
1407+--
1408+-- Definition:
1409+--
1410+-- >>> all p = Fold.lmap p Fold.and
1411+--
1412+-- Example:
1413+--
1414+-- >>> Stream.fold (Fold.all (== 0)) $ Stream.fromList [1,0,1]
1415+-- False
1416+--
1417+{-# INLINE all #-}
1418+all :: Monad m => (a -> Bool) -> Fold m a Bool
1419+all predicate = foldt' step initial id
1420+
1421+ where
1422+
1423+ initial = Partial True
1424+
1425+ step _ a =
1426+ if predicate a
1427+ then Partial True
1428+ else Done False
1429+
1430+-- | Returns 'True' if the given element is not present in the stream.
1431+--
1432+-- Definition:
1433+--
1434+-- >>> notElem a = Fold.all (/= a)
1435+--
1436+{-# INLINE notElem #-}
1437+notElem :: (Eq a, Monad m) => a -> Fold m a Bool
1438+notElem a = all (/= a)
1439+
1440+-- | Returns 'True' if all elements are 'True', 'False' otherwise
1441+--
1442+-- Definition:
1443+--
1444+-- >>> and = Fold.all (== True)
1445+--
1446+{-# INLINE and #-}
1447+and :: Monad m => Fold m Bool Bool
1448+and = all (== True)
1449+
1450+-- | Returns 'True' if any element is 'True', 'False' otherwise
1451+--
1452+-- Definition:
1453+--
1454+-- >>> or = Fold.any (== True)
1455+--
1456+{-# INLINE or #-}
1457+or :: Monad m => Fold m Bool Bool
1458+or = any (== True)
1459+
1460+------------------------------------------------------------------------------
1461+-- Grouping/Splitting
1462+------------------------------------------------------------------------------
1463+
1464+------------------------------------------------------------------------------
1465+-- Grouping without looking at elements
1466+------------------------------------------------------------------------------
1467+
1468+------------------------------------------------------------------------------
1469+-- Binary APIs
1470+------------------------------------------------------------------------------
1471+
1472+-- | @splitAt n f1 f2@ composes folds @f1@ and @f2@ such that first @n@
1473+-- elements of its input are consumed by fold @f1@ and the rest of the stream
1474+-- is consumed by fold @f2@.
1475+--
1476+-- >>> let splitAt_ n xs = Stream.fold (Fold.splitAt n Fold.toList Fold.toList) $ Stream.fromList xs
1477+--
1478+-- >>> splitAt_ 6 "Hello World!"
1479+-- ("Hello ","World!")
1480+--
1481+-- >>> splitAt_ (-1) [1,2,3]
1482+-- ([],[1,2,3])
1483+--
1484+-- >>> splitAt_ 0 [1,2,3]
1485+-- ([],[1,2,3])
1486+--
1487+-- >>> splitAt_ 1 [1,2,3]
1488+-- ([1],[2,3])
1489+--
1490+-- >>> splitAt_ 3 [1,2,3]
1491+-- ([1,2,3],[])
1492+--
1493+-- >>> splitAt_ 4 [1,2,3]
1494+-- ([1,2,3],[])
1495+--
1496+-- > splitAt n f1 f2 = Fold.splitWith (,) (Fold.take n f1) f2
1497+--
1498+-- /Internal/
1499+
1500+{-# INLINE splitAt #-}
1501+splitAt
1502+ :: Monad m
1503+ => Int
1504+ -> Fold m a b
1505+ -> Fold m a c
1506+ -> Fold m a (b, c)
1507+splitAt n fld = splitWith (,) (take n fld)
1508+
1509+------------------------------------------------------------------------------
1510+-- Element Aware APIs
1511+------------------------------------------------------------------------------
1512+--
1513+------------------------------------------------------------------------------
1514+-- Binary APIs
1515+------------------------------------------------------------------------------
1516+
1517+{-# INLINE takingEndByM #-}
1518+takingEndByM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1519+takingEndByM p = Fold step initial (return . toMaybe)
1520+
1521+ where
1522+
1523+ initial = return $ Partial Nothing'
1524+
1525+ step _ a = do
1526+ r <- p a
1527+ return
1528+ $ if r
1529+ then Done $ Just a
1530+ else Partial $ Just' a
1531+
1532+-- |
1533+--
1534+-- >>> takingEndBy p = Fold.takingEndByM (return . p)
1535+--
1536+{-# INLINE takingEndBy #-}
1537+takingEndBy :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1538+takingEndBy p = takingEndByM (return . p)
1539+
1540+{-# INLINE takingEndByM_ #-}
1541+takingEndByM_ :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1542+takingEndByM_ p = Fold step initial (return . toMaybe)
1543+
1544+ where
1545+
1546+ initial = return $ Partial Nothing'
1547+
1548+ step _ a = do
1549+ r <- p a
1550+ return
1551+ $ if r
1552+ then Done Nothing
1553+ else Partial $ Just' a
1554+
1555+-- |
1556+--
1557+-- >>> takingEndBy_ p = Fold.takingEndByM_ (return . p)
1558+--
1559+{-# INLINE takingEndBy_ #-}
1560+takingEndBy_ :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1561+takingEndBy_ p = takingEndByM_ (return . p)
1562+
1563+{-# INLINE droppingWhileM #-}
1564+droppingWhileM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1565+droppingWhileM p = Fold step initial (return . toMaybe)
1566+
1567+ where
1568+
1569+ initial = return $ Partial Nothing'
1570+
1571+ step Nothing' a = do
1572+ r <- p a
1573+ return
1574+ $ Partial
1575+ $ if r
1576+ then Nothing'
1577+ else Just' a
1578+ step _ a = return $ Partial $ Just' a
1579+
1580+-- |
1581+-- >>> droppingWhile p = Fold.droppingWhileM (return . p)
1582+--
1583+{-# INLINE droppingWhile #-}
1584+droppingWhile :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1585+droppingWhile p = droppingWhileM (return . p)
1586+
1587+-- Note: Keep this consistent with S.splitOn. In fact we should eliminate
1588+-- S.splitOn in favor of the fold.
1589+--
1590+-- XXX Use Fold.many instead once it is fixed.
1591+-- > Stream.splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
1592+
1593+-- | Like 'takeEndBy' but drops the element on which the predicate succeeds.
1594+--
1595+-- Example:
1596+--
1597+-- >>> input = Stream.fromList "hello\nthere\n"
1598+-- >>> line = Fold.takeEndBy_ (== '\n') Fold.toList
1599+-- >>> Stream.fold line input
1600+-- "hello"
1601+--
1602+-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1603+-- ["hello","there"]
1604+--
1605+{-# INLINE takeEndBy_ #-}
1606+takeEndBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1607+-- takeEndBy_ predicate = scanMaybe (takingEndBy_ predicate)
1608+takeEndBy_ predicate (Fold fstep finitial fextract) =
1609+ Fold step finitial fextract
1610+
1611+ where
1612+
1613+ step s a =
1614+ if not (predicate a)
1615+ then fstep s a
1616+ else Done <$> fextract s
1617+
1618+-- Note:
1619+-- > Stream.splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
1620+
1621+-- | Take the input, stop when the predicate succeeds taking the succeeding
1622+-- element as well.
1623+--
1624+-- Example:
1625+--
1626+-- >>> input = Stream.fromList "hello\nthere\n"
1627+-- >>> line = Fold.takeEndBy (== '\n') Fold.toList
1628+-- >>> Stream.fold line input
1629+-- "hello\n"
1630+--
1631+-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1632+-- ["hello\n","there\n"]
1633+--
1634+{-# INLINE takeEndBy #-}
1635+takeEndBy :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1636+-- takeEndBy predicate = scanMaybe (takingEndBy predicate)
1637+takeEndBy predicate (Fold fstep finitial fextract) =
1638+ Fold step finitial fextract
1639+
1640+ where
1641+
1642+ step s a = do
1643+ res <- fstep s a
1644+ if not (predicate a)
1645+ then return res
1646+ else do
1647+ case res of
1648+ Partial s1 -> Done <$> fextract s1
1649+ Done b -> return $ Done b
1650+
1651+------------------------------------------------------------------------------
1652+-- Binary splitting on a separator
1653+------------------------------------------------------------------------------
1654+
1655+data SplitOnSeqState acc a rb rh w ck =
1656+ SplitOnSeqEmpty !acc
1657+ | SplitOnSeqSingle !acc !a
1658+ | SplitOnSeqWord !acc !Int !w
1659+ | SplitOnSeqWordLoop !acc !w
1660+ | SplitOnSeqKR !acc !Int !rb !rh
1661+ | SplitOnSeqKRLoop !acc !ck !rb !rh
1662+
1663+-- XXX Need to add tests for takeEndBySeq, we have tests for takeEndBySeq_ .
1664+
1665+-- | Continue taking the input until the input sequence matches the supplied
1666+-- sequence, taking the supplied sequence as well. If the pattern is empty this
1667+-- acts as an identity fold.
1668+--
1669+-- >>> s = Stream.fromList "hello there. How are you?"
1670+-- >>> f = Fold.takeEndBySeq (Array.fromList "re") Fold.toList
1671+-- >>> Stream.fold f s
1672+-- "hello there"
1673+--
1674+-- >>> Stream.fold Fold.toList $ Stream.foldMany f s
1675+-- ["hello there",". How are"," you?"]
1676+--
1677+-- /Pre-release/
1678+{-# INLINE takeEndBySeq #-}
1679+takeEndBySeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
1680+ Array.Array a
1681+ -> Fold m a b
1682+ -> Fold m a b
1683+takeEndBySeq patArr (Fold fstep finitial fextract) =
1684+ Fold step initial extract
1685+
1686+ where
1687+
1688+ patLen = Array.length patArr
1689+
1690+ initial = do
1691+ res <- finitial
1692+ case res of
1693+ Partial acc
1694+ | patLen == 0 ->
1695+ -- XXX Should we match nothing or everything on empty
1696+ -- pattern?
1697+ -- Done <$> fextract acc
1698+ return $ Partial $ SplitOnSeqEmpty acc
1699+ | patLen == 1 -> do
1700+ pat <- liftIO $ Array.unsafeIndexIO 0 patArr
1701+ return $ Partial $ SplitOnSeqSingle acc pat
1702+ | SIZE_OF(a) * patLen <= sizeOf (Proxy :: Proxy Word) ->
1703+ return $ Partial $ SplitOnSeqWord acc 0 0
1704+ | otherwise -> do
1705+ (rb, rhead) <- liftIO $ Ring.new patLen
1706+ return $ Partial $ SplitOnSeqKR acc 0 rb rhead
1707+ Done b -> return $ Done b
1708+
1709+ -- Word pattern related
1710+ maxIndex = patLen - 1
1711+
1712+ elemBits = SIZE_OF(a) * 8
1713+
1714+ wordMask :: Word
1715+ wordMask = (1 `shiftL` (elemBits * patLen)) - 1
1716+
1717+ wordPat :: Word
1718+ wordPat = wordMask .&. Array.foldl' addToWord 0 patArr
1719+
1720+ addToWord wd a = (wd `shiftL` elemBits) .|. fromIntegral (fromEnum a)
1721+
1722+ -- For Rabin-Karp search
1723+ k = 2891336453 :: Word32
1724+ coeff = k ^ patLen
1725+
1726+ addCksum cksum a = cksum * k + fromIntegral (fromEnum a)
1727+
1728+ deltaCksum cksum old new =
1729+ addCksum cksum new - coeff * fromIntegral (fromEnum old)
1730+
1731+ -- XXX shall we use a random starting hash or 1 instead of 0?
1732+ -- XXX Need to keep this cached across fold calls in foldmany
1733+ -- XXX We may need refold to inject the cached state instead of
1734+ -- initializing the state every time.
1735+ -- XXX Allocation of ring buffer should also be done once
1736+ patHash = Array.foldl' addCksum 0 patArr
1737+
1738+ step (SplitOnSeqEmpty s) x = do
1739+ res <- fstep s x
1740+ case res of
1741+ Partial s1 -> return $ Partial $ SplitOnSeqEmpty s1
1742+ Done b -> return $ Done b
1743+ step (SplitOnSeqSingle s pat) x = do
1744+ res <- fstep s x
1745+ case res of
1746+ Partial s1
1747+ | pat /= x -> return $ Partial $ SplitOnSeqSingle s1 pat
1748+ | otherwise -> Done <$> fextract s1
1749+ Done b -> return $ Done b
1750+ step (SplitOnSeqWord s idx wrd) x = do
1751+ res <- fstep s x
1752+ let wrd1 = addToWord wrd x
1753+ case res of
1754+ Partial s1
1755+ | idx == maxIndex -> do
1756+ if wrd1 .&. wordMask == wordPat
1757+ then Done <$> fextract s1
1758+ else return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1759+ | otherwise ->
1760+ return $ Partial $ SplitOnSeqWord s1 (idx + 1) wrd1
1761+ Done b -> return $ Done b
1762+ step (SplitOnSeqWordLoop s wrd) x = do
1763+ res <- fstep s x
1764+ let wrd1 = addToWord wrd x
1765+ case res of
1766+ Partial s1
1767+ | wrd1 .&. wordMask == wordPat ->
1768+ Done <$> fextract s1
1769+ | otherwise ->
1770+ return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1771+ Done b -> return $ Done b
1772+ step (SplitOnSeqKR s idx rb rh) x = do
1773+ res <- fstep s x
1774+ case res of
1775+ Partial s1 -> do
1776+ rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1777+ if idx == maxIndex
1778+ then do
1779+ let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
1780+ let !ringHash = fld addCksum 0 rb
1781+ if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1782+ then Done <$> fextract s1
1783+ else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1784+ else
1785+ return $ Partial $ SplitOnSeqKR s1 (idx + 1) rb rh1
1786+ Done b -> return $ Done b
1787+ step (SplitOnSeqKRLoop s cksum rb rh) x = do
1788+ res <- fstep s x
1789+ case res of
1790+ Partial s1 -> do
1791+ old <- liftIO $ peek rh
1792+ rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1793+ let ringHash = deltaCksum cksum old x
1794+ if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1795+ then Done <$> fextract s1
1796+ else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1797+ Done b -> return $ Done b
1798+
1799+ extract state =
1800+ let st =
1801+ case state of
1802+ SplitOnSeqEmpty s -> s
1803+ SplitOnSeqSingle s _ -> s
1804+ SplitOnSeqWord s _ _ -> s
1805+ SplitOnSeqWordLoop s _ -> s
1806+ SplitOnSeqKR s _ _ _ -> s
1807+ SplitOnSeqKRLoop s _ _ _ -> s
1808+ in fextract st
1809+
1810+-- | Like 'takeEndBySeq' but discards the matched sequence.
1811+--
1812+-- /Pre-release/
1813+--
1814+{-# INLINE takeEndBySeq_ #-}
1815+takeEndBySeq_ :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
1816+ Array.Array a
1817+ -> Fold m a b
1818+ -> Fold m a b
1819+takeEndBySeq_ patArr (Fold fstep finitial fextract) =
1820+ Fold step initial extract
1821+
1822+ where
1823+
1824+ patLen = Array.length patArr
1825+
1826+ initial = do
1827+ res <- finitial
1828+ case res of
1829+ Partial acc
1830+ | patLen == 0 ->
1831+ -- XXX Should we match nothing or everything on empty
1832+ -- pattern?
1833+ -- Done <$> fextract acc
1834+ return $ Partial $ SplitOnSeqEmpty acc
1835+ | patLen == 1 -> do
1836+ pat <- liftIO $ Array.unsafeIndexIO 0 patArr
1837+ return $ Partial $ SplitOnSeqSingle acc pat
1838+ -- XXX Need to add tests for this case
1839+ | SIZE_OF(a) * patLen <= sizeOf (Proxy :: Proxy Word) ->
1840+ return $ Partial $ SplitOnSeqWord acc 0 0
1841+ | otherwise -> do
1842+ (rb, rhead) <- liftIO $ Ring.new patLen
1843+ return $ Partial $ SplitOnSeqKR acc 0 rb rhead
1844+ Done b -> return $ Done b
1845+
1846+ -- Word pattern related
1847+ maxIndex = patLen - 1
1848+
1849+ elemBits = SIZE_OF(a) * 8
1850+
1851+ wordMask :: Word
1852+ wordMask = (1 `shiftL` (elemBits * patLen)) - 1
1853+
1854+ elemMask :: Word
1855+ elemMask = (1 `shiftL` elemBits) - 1
1856+
1857+ wordPat :: Word
1858+ wordPat = wordMask .&. Array.foldl' addToWord 0 patArr
1859+
1860+ addToWord wd a = (wd `shiftL` elemBits) .|. fromIntegral (fromEnum a)
1861+
1862+ -- For Rabin-Karp search
1863+ k = 2891336453 :: Word32
1864+ coeff = k ^ patLen
1865+
1866+ addCksum cksum a = cksum * k + fromIntegral (fromEnum a)
1867+
1868+ deltaCksum cksum old new =
1869+ addCksum cksum new - coeff * fromIntegral (fromEnum old)
1870+
1871+ -- XXX shall we use a random starting hash or 1 instead of 0?
1872+ -- XXX Need to keep this cached across fold calls in foldMany
1873+ -- XXX We may need refold to inject the cached state instead of
1874+ -- initializing the state every time.
1875+ -- XXX Allocation of ring buffer should also be done once
1876+ patHash = Array.foldl' addCksum 0 patArr
1877+
1878+ step (SplitOnSeqEmpty s) x = do
1879+ res <- fstep s x
1880+ case res of
1881+ Partial s1 -> return $ Partial $ SplitOnSeqEmpty s1
1882+ Done b -> return $ Done b
1883+ step (SplitOnSeqSingle s pat) x = do
1884+ if pat /= x
1885+ then do
1886+ res <- fstep s x
1887+ case res of
1888+ Partial s1 -> return $ Partial $ SplitOnSeqSingle s1 pat
1889+ Done b -> return $ Done b
1890+ else Done <$> fextract s
1891+ step (SplitOnSeqWord s idx wrd) x = do
1892+ let wrd1 = addToWord wrd x
1893+ if idx == maxIndex
1894+ then do
1895+ if wrd1 .&. wordMask == wordPat
1896+ then Done <$> fextract s
1897+ else return $ Partial $ SplitOnSeqWordLoop s wrd1
1898+ else return $ Partial $ SplitOnSeqWord s (idx + 1) wrd1
1899+ step (SplitOnSeqWordLoop s wrd) x = do
1900+ let wrd1 = addToWord wrd x
1901+ old = (wordMask .&. wrd)
1902+ `shiftR` (elemBits * (patLen - 1))
1903+ res <- fstep s (toEnum $ fromIntegral old)
1904+ case res of
1905+ Partial s1
1906+ | wrd1 .&. wordMask == wordPat ->
1907+ Done <$> fextract s1
1908+ | otherwise ->
1909+ return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1910+ Done b -> return $ Done b
1911+ step (SplitOnSeqKR s idx rb rh) x = do
1912+ rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1913+ if idx == maxIndex
1914+ then do
1915+ let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
1916+ let !ringHash = fld addCksum 0 rb
1917+ if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1918+ then Done <$> fextract s
1919+ else return $ Partial $ SplitOnSeqKRLoop s ringHash rb rh1
1920+ else return $ Partial $ SplitOnSeqKR s (idx + 1) rb rh1
1921+ step (SplitOnSeqKRLoop s cksum rb rh) x = do
1922+ old <- liftIO $ peek rh
1923+ res <- fstep s old
1924+ case res of
1925+ Partial s1 -> do
1926+ rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1927+ let ringHash = deltaCksum cksum old x
1928+ if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1929+ then Done <$> fextract s1
1930+ else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1931+ Done b -> return $ Done b
1932+
1933+ -- XXX extract should return backtrack count as well. If the fold
1934+ -- terminates early inside extract, we may still have buffered data
1935+ -- remaining which will be lost if we do not communicate that to the
1936+ -- driver.
1937+ extract state = do
1938+ let consumeWord s n wrd = do
1939+ if n == 0
1940+ then fextract s
1941+ else do
1942+ let old = elemMask .&. (wrd `shiftR` (elemBits * (n - 1)))
1943+ r <- fstep s (toEnum $ fromIntegral old)
1944+ case r of
1945+ Partial s1 -> consumeWord s1 (n - 1) wrd
1946+ Done b -> return b
1947+
1948+ let consumeRing s n rb rh =
1949+ if n == 0
1950+ then fextract s
1951+ else do
1952+ old <- liftIO $ peek rh
1953+ let rh1 = Ring.advance rb rh
1954+ r <- fstep s old
1955+ case r of
1956+ Partial s1 -> consumeRing s1 (n - 1) rb rh1
1957+ Done b -> return b
1958+
1959+ case state of
1960+ SplitOnSeqEmpty s -> fextract s
1961+ SplitOnSeqSingle s _ -> fextract s
1962+ SplitOnSeqWord s idx wrd -> consumeWord s idx wrd
1963+ SplitOnSeqWordLoop s wrd -> consumeWord s patLen wrd
1964+ SplitOnSeqKR s idx rb _ -> consumeRing s idx rb (Ring.startOf rb)
1965+ SplitOnSeqKRLoop s _ rb rh -> consumeRing s patLen rb rh
1966+
1967+------------------------------------------------------------------------------
1968+-- Distributing
1969+------------------------------------------------------------------------------
1970+--
1971+-- | Distribute one copy of the stream to each fold and zip the results.
1972+--
1973+-- @
1974+-- |-------Fold m a b--------|
1975+-- ---stream m a---| |---m (b,c)
1976+-- |-------Fold m a c--------|
1977+-- @
1978+--
1979+-- Definition:
1980+--
1981+-- >>> tee = Fold.teeWith (,)
1982+--
1983+-- Example:
1984+--
1985+-- >>> t = Fold.tee Fold.sum Fold.length
1986+-- >>> Stream.fold t (Stream.enumerateFromTo 1.0 100.0)
1987+-- (5050.0,100)
1988+--
1989+{-# INLINE tee #-}
1990+tee :: Monad m => Fold m a b -> Fold m a c -> Fold m a (b,c)
1991+tee = teeWith (,)
1992+
1993+-- XXX use "List" instead of "[]"?, use Array for output to scale it to a large
1994+-- number of consumers? For polymorphic case a vector could be helpful. For
1995+-- Storables we can use arrays. Will need separate APIs for those.
1996+--
1997+-- | Distribute one copy of the stream to each fold and collect the results in
1998+-- a container.
1999+--
2000+-- @
2001+--
2002+-- |-------Fold m a b--------|
2003+-- ---stream m a---| |---m [b]
2004+-- |-------Fold m a b--------|
2005+-- | |
2006+-- ...
2007+-- @
2008+--
2009+-- >>> Stream.fold (Fold.distribute [Fold.sum, Fold.length]) (Stream.enumerateFromTo 1 5)
2010+-- [15,5]
2011+--
2012+-- >>> distribute = Prelude.foldr (Fold.teeWith (:)) (Fold.fromPure [])
2013+--
2014+-- This is the consumer side dual of the producer side 'sequence' operation.
2015+--
2016+-- Stops when all the folds stop.
2017+--
2018+{-# INLINE distribute #-}
2019+distribute :: Monad m => [Fold m a b] -> Fold m a [b]
2020+distribute = Prelude.foldr (teeWith (:)) (fromPure [])
2021+
2022+------------------------------------------------------------------------------
2023+-- Partitioning
2024+------------------------------------------------------------------------------
2025+
2026+{-# INLINE partitionByMUsing #-}
2027+partitionByMUsing :: Monad m =>
2028+ ( (x -> y -> (x, y))
2029+ -> Fold m (Either b c) x
2030+ -> Fold m (Either b c) y
2031+ -> Fold m (Either b c) (x, y)
2032+ )
2033+ -> (a -> m (Either b c))
2034+ -> Fold m b x
2035+ -> Fold m c y
2036+ -> Fold m a (x, y)
2037+partitionByMUsing t f fld1 fld2 =
2038+ let l = lmap (fromLeft undefined) fld1 -- :: Fold m (Either b c) x
2039+ r = lmap (fromRight undefined) fld2 -- :: Fold m (Either b c) y
2040+ in lmapM f (t (,) (filter isLeft l) (filter isRight r))
2041+
2042+-- | Partition the input over two folds using an 'Either' partitioning
2043+-- predicate.
2044+--
2045+-- @
2046+--
2047+-- |-------Fold b x--------|
2048+-- -----stream m a --> (Either b c)----| |----(x,y)
2049+-- |-------Fold c y--------|
2050+-- @
2051+--
2052+-- Example, send input to either fold randomly:
2053+--
2054+-- >>> :set -package random
2055+-- >>> import System.Random (randomIO)
2056+-- >>> randomly a = randomIO >>= \x -> return $ if x then Left a else Right a
2057+-- >>> f = Fold.partitionByM randomly Fold.length Fold.length
2058+-- >>> Stream.fold f (Stream.enumerateFromTo 1 100)
2059+-- ...
2060+--
2061+-- Example, send input to the two folds in a proportion of 2:1:
2062+--
2063+-- >>> :{
2064+-- proportionately m n = do
2065+-- ref <- newIORef $ cycle $ concat [replicate m Left, replicate n Right]
2066+-- return $ \a -> do
2067+-- r <- readIORef ref
2068+-- writeIORef ref $ tail r
2069+-- return $ Prelude.head r a
2070+-- :}
2071+--
2072+-- >>> :{
2073+-- main = do
2074+-- g <- proportionately 2 1
2075+-- let f = Fold.partitionByM g Fold.length Fold.length
2076+-- r <- Stream.fold f (Stream.enumerateFromTo (1 :: Int) 100)
2077+-- print r
2078+-- :}
2079+--
2080+-- >>> main
2081+-- (67,33)
2082+--
2083+--
2084+-- This is the consumer side dual of the producer side 'mergeBy' operation.
2085+--
2086+-- When one fold is done, any input meant for it is ignored until the other
2087+-- fold is also done.
2088+--
2089+-- Stops when both the folds stop.
2090+--
2091+-- /See also: 'partitionByFstM' and 'partitionByMinM'./
2092+--
2093+-- /Pre-release/
2094+{-# INLINE partitionByM #-}
2095+partitionByM :: Monad m
2096+ => (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2097+partitionByM = partitionByMUsing teeWith
2098+
2099+-- | Similar to 'partitionByM' but terminates when the first fold terminates.
2100+--
2101+{-# INLINE partitionByFstM #-}
2102+partitionByFstM :: Monad m
2103+ => (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2104+partitionByFstM = partitionByMUsing teeWithFst
2105+
2106+-- | Similar to 'partitionByM' but terminates when any fold terminates.
2107+--
2108+{-# INLINE partitionByMinM #-}
2109+partitionByMinM :: Monad m =>
2110+ (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2111+partitionByMinM = partitionByMUsing teeWithMin
2112+
2113+-- Note: we could use (a -> Bool) instead of (a -> Either b c), but the latter
2114+-- makes the signature clearer as to which case belongs to which fold.
2115+-- XXX need to check the performance in both cases.
2116+
2117+-- | Same as 'partitionByM' but with a pure partition function.
2118+--
2119+-- Example, count even and odd numbers in a stream:
2120+--
2121+-- >>> :{
2122+-- let f = Fold.partitionBy (\n -> if even n then Left n else Right n)
2123+-- (fmap (("Even " ++) . show) Fold.length)
2124+-- (fmap (("Odd " ++) . show) Fold.length)
2125+-- in Stream.fold f (Stream.enumerateFromTo 1 100)
2126+-- :}
2127+-- ("Even 50","Odd 50")
2128+--
2129+-- /Pre-release/
2130+{-# INLINE partitionBy #-}
2131+partitionBy :: Monad m
2132+ => (a -> Either b c) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2133+partitionBy f = partitionByM (return . f)
2134+
2135+-- | Compose two folds such that the combined fold accepts a stream of 'Either'
2136+-- and routes the 'Left' values to the first fold and 'Right' values to the
2137+-- second fold.
2138+--
2139+-- Definition:
2140+--
2141+-- >>> partition = Fold.partitionBy id
2142+--
2143+{-# INLINE partition #-}
2144+partition :: Monad m
2145+ => Fold m b x -> Fold m c y -> Fold m (Either b c) (x, y)
2146+partition = partitionBy id
2147+
2148+{-
2149+-- | Send one item to each fold in a round-robin fashion. This is the consumer
2150+-- side dual of producer side 'mergeN' operation.
2151+--
2152+-- partitionN :: Monad m => [Fold m a b] -> Fold m a [b]
2153+-- partitionN fs = Fold step begin done
2154+-}
2155+
2156+------------------------------------------------------------------------------
2157+-- Unzipping
2158+------------------------------------------------------------------------------
2159+
2160+{-# INLINE unzipWithMUsing #-}
2161+unzipWithMUsing :: Monad m =>
2162+ ( (x -> y -> (x, y))
2163+ -> Fold m (b, c) x
2164+ -> Fold m (b, c) y
2165+ -> Fold m (b, c) (x, y)
2166+ )
2167+ -> (a -> m (b, c))
2168+ -> Fold m b x
2169+ -> Fold m c y
2170+ -> Fold m a (x, y)
2171+unzipWithMUsing t f fld1 fld2 =
2172+ let f1 = lmap fst fld1 -- :: Fold m (b, c) b
2173+ f2 = lmap snd fld2 -- :: Fold m (b, c) c
2174+ in lmapM f (t (,) f1 f2)
2175+
2176+-- | Like 'unzipWith' but with a monadic splitter function.
2177+--
2178+-- Definition:
2179+--
2180+-- >>> unzipWithM k f1 f2 = Fold.lmapM k (Fold.unzip f1 f2)
2181+--
2182+-- /Pre-release/
2183+{-# INLINE unzipWithM #-}
2184+unzipWithM :: Monad m
2185+ => (a -> m (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2186+unzipWithM = unzipWithMUsing teeWith
2187+
2188+-- | Similar to 'unzipWithM' but terminates when the first fold terminates.
2189+--
2190+{-# INLINE unzipWithFstM #-}
2191+unzipWithFstM :: Monad m =>
2192+ (a -> m (b, c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2193+unzipWithFstM = unzipWithMUsing teeWithFst
2194+
2195+-- | Similar to 'unzipWithM' but terminates when any fold terminates.
2196+--
2197+{-# INLINE unzipWithMinM #-}
2198+unzipWithMinM :: Monad m =>
2199+ (a -> m (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2200+unzipWithMinM = unzipWithMUsing teeWithMin
2201+
2202+-- | Split elements in the input stream into two parts using a pure splitter
2203+-- function, direct each part to a different fold and zip the results.
2204+--
2205+-- Definitions:
2206+--
2207+-- >>> unzipWith f = Fold.unzipWithM (return . f)
2208+-- >>> unzipWith f fld1 fld2 = Fold.lmap f (Fold.unzip fld1 fld2)
2209+--
2210+-- This fold terminates when both the input folds terminate.
2211+--
2212+-- /Pre-release/
2213+{-# INLINE unzipWith #-}
2214+unzipWith :: Monad m
2215+ => (a -> (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2216+unzipWith f = unzipWithM (return . f)
2217+
2218+-- | Send the elements of tuples in a stream of tuples through two different
2219+-- folds.
2220+--
2221+-- @
2222+--
2223+-- |-------Fold m a x--------|
2224+-- ---------stream of (a,b)--| |----m (x,y)
2225+-- |-------Fold m b y--------|
2226+--
2227+-- @
2228+--
2229+-- Definition:
2230+--
2231+-- >>> unzip = Fold.unzipWith id
2232+--
2233+-- This is the consumer side dual of the producer side 'zip' operation.
2234+--
2235+{-# INLINE unzip #-}
2236+unzip :: Monad m => Fold m a x -> Fold m b y -> Fold m (a,b) (x,y)
2237+unzip = unzipWith id
2238+
2239+------------------------------------------------------------------------------
2240+-- Combining streams and folds - Zipping
2241+------------------------------------------------------------------------------
2242+
2243+-- XXX These can be implemented using the fold scan, using the stream as a
2244+-- state.
2245+-- XXX Stream Skip state cannot be efficiently handled in folds but can be
2246+-- handled in parsers using the Continue facility. See zipWithM in the Parser
2247+-- module.
2248+--
2249+-- cmpBy, eqBy, isPrefixOf, isSubsequenceOf etc can be implemented using
2250+-- zipStream.
2251+
2252+-- | Zip a stream with the input of a fold using the supplied function.
2253+--
2254+-- /Unimplemented/
2255+--
2256+{-# INLINE zipStreamWithM #-}
2257+zipStreamWithM :: -- Monad m =>
2258+ (a -> b -> m c) -> Stream m a -> Fold m c x -> Fold m b x
2259+zipStreamWithM = undefined
2260+
2261+-- | Zip a stream with the input of a fold.
2262+--
2263+-- >>> zip = Fold.zipStreamWithM (curry return)
2264+--
2265+-- /Unimplemented/
2266+--
2267+{-# INLINE zipStream #-}
2268+zipStream :: Monad m => Stream m a -> Fold m (a, b) x -> Fold m b x
2269+zipStream = zipStreamWithM (curry return)
2270+
2271+-- | Pair each element of a fold input with its index, starting from index 0.
2272+--
2273+{-# INLINE indexingWith #-}
2274+indexingWith :: Monad m => Int -> (Int -> Int) -> Fold m a (Maybe (Int, a))
2275+indexingWith i f = fmap toMaybe $ foldl' step initial
2276+
2277+ where
2278+
2279+ initial = Nothing'
2280+
2281+ step Nothing' a = Just' (i, a)
2282+ step (Just' (n, _)) a = Just' (f n, a)
2283+
2284+-- |
2285+-- >>> indexing = Fold.indexingWith 0 (+ 1)
2286+--
2287+{-# INLINE indexing #-}
2288+indexing :: Monad m => Fold m a (Maybe (Int, a))
2289+indexing = indexingWith 0 (+ 1)
2290+
2291+-- |
2292+-- >>> indexingRev n = Fold.indexingWith n (subtract 1)
2293+--
2294+{-# INLINE indexingRev #-}
2295+indexingRev :: Monad m => Int -> Fold m a (Maybe (Int, a))
2296+indexingRev n = indexingWith n (subtract 1)
2297+
2298+-- | Pair each element of a fold input with its index, starting from index 0.
2299+--
2300+-- >>> indexed = Fold.scanMaybe Fold.indexing
2301+--
2302+{-# INLINE indexed #-}
2303+indexed :: Monad m => Fold m (Int, a) b -> Fold m a b
2304+indexed = scanMaybe indexing
2305+
2306+-- | Change the predicate function of a Fold from @a -> b@ to accept an
2307+-- additional state input @(s, a) -> b@. Convenient to filter with an
2308+-- addiitonal index or time input.
2309+--
2310+-- >>> filterWithIndex = Fold.with Fold.indexed Fold.filter
2311+--
2312+-- @
2313+-- filterWithAbsTime = with timestamped filter
2314+-- filterWithRelTime = with timeIndexed filter
2315+-- @
2316+--
2317+-- /Pre-release/
2318+{-# INLINE with #-}
2319+with ::
2320+ (Fold m (s, a) b -> Fold m a b)
2321+ -> (((s, a) -> c) -> Fold m (s, a) b -> Fold m (s, a) b)
2322+ -> (((s, a) -> c) -> Fold m a b -> Fold m a b)
2323+with f comb g = f . comb g . lmap snd
2324+
2325+-- XXX Implement as a filter
2326+-- sampleFromthen :: Monad m => Int -> Int -> Fold m a (Maybe a)
2327+
2328+-- | @sampleFromthen offset stride@ samples the element at @offset@ index and
2329+-- then every element at strides of @stride@.
2330+--
2331+{-# INLINE sampleFromthen #-}
2332+sampleFromthen :: Monad m => Int -> Int -> Fold m a b -> Fold m a b
2333+sampleFromthen offset size =
2334+ with indexed filter (\(i, _) -> (i + offset) `mod` size == 0)
2335+
2336+------------------------------------------------------------------------------
2337+-- Nesting
2338+------------------------------------------------------------------------------
2339+
2340+-- | @concatSequence f t@ applies folds from stream @t@ sequentially and
2341+-- collects the results using the fold @f@.
2342+--
2343+-- /Unimplemented/
2344+--
2345+{-# INLINE concatSequence #-}
2346+concatSequence ::
2347+ -- IsStream t =>
2348+ Fold m b c -> t (Fold m a b) -> Fold m a c
2349+concatSequence _f _p = undefined
2350+
2351+-- | Group the input stream into groups of elements between @low@ and @high@.
2352+-- Collection starts in chunks of @low@ and then keeps doubling until we reach
2353+-- @high@. Each chunk is folded using the provided fold function.
2354+--
2355+-- This could be useful, for example, when we are folding a stream of unknown
2356+-- size to a stream of arrays and we want to minimize the number of
2357+-- allocations.
2358+--
2359+-- NOTE: this would be an application of "many" using a terminating fold.
2360+--
2361+-- /Unimplemented/
2362+--
2363+{-# INLINE chunksBetween #-}
2364+chunksBetween :: -- Monad m =>
2365+ Int -> Int -> Fold m a b -> Fold m b c -> Fold m a c
2366+chunksBetween _low _high _f1 _f2 = undefined
2367+
2368+-- | A fold that buffers its input to a pure stream.
2369+--
2370+-- /Warning!/ working on large streams accumulated as buffers in memory could
2371+-- be very inefficient, consider using "Streamly.Data.Array" instead.
2372+--
2373+-- >>> toStream = fmap Stream.fromList Fold.toList
2374+--
2375+-- /Pre-release/
2376+{-# INLINE toStream #-}
2377+toStream :: (Monad m, Monad n) => Fold m a (Stream n a)
2378+toStream = fmap StreamD.fromList toList
2379+
2380+-- This is more efficient than 'toStream'. toStream is exactly the same as
2381+-- reversing the stream after toStreamRev.
2382+--
2383+-- | Buffers the input stream to a pure stream in the reverse order of the
2384+-- input.
2385+--
2386+-- >>> toStreamRev = fmap Stream.fromList Fold.toListRev
2387+--
2388+-- /Warning!/ working on large streams accumulated as buffers in memory could
2389+-- be very inefficient, consider using "Streamly.Data.Array" instead.
2390+--
2391+-- /Pre-release/
2392+
2393+-- xn : ... : x2 : x1 : []
2394+{-# INLINE toStreamRev #-}
2395+toStreamRev :: (Monad m, Monad n) => Fold m a (Stream n a)
2396+toStreamRev = fmap StreamD.fromList toListRev
2397+
2398+-- XXX This does not fuse. It contains a recursive step function. We will need
2399+-- a Skip input constructor in the fold type to make it fuse.
2400+--
2401+-- | Unfold and flatten the input stream of a fold.
2402+--
2403+-- @
2404+-- Stream.fold (unfoldMany u f) = Stream.fold f . Stream.unfoldMany u
2405+-- @
2406+--
2407+-- /Pre-release/
2408+{-# INLINE unfoldMany #-}
2409+unfoldMany :: Monad m => Unfold m a b -> Fold m b c -> Fold m a c
2410+unfoldMany (Unfold ustep inject) (Fold fstep initial extract) =
2411+ Fold consume initial extract
2412+
2413+ where
2414+
2415+ {-# INLINE produce #-}
2416+ produce fs us = do
2417+ ures <- ustep us
2418+ case ures of
2419+ StreamD.Yield b us1 -> do
2420+ fres <- fstep fs b
2421+ case fres of
2422+ Partial fs1 -> produce fs1 us1
2423+ -- XXX What to do with the remaining stream?
2424+ Done c -> return $ Done c
2425+ StreamD.Skip us1 -> produce fs us1
2426+ StreamD.Stop -> return $ Partial fs
2427+
2428+ {-# INLINE_LATE consume #-}
2429+ consume s a = inject a >>= produce s
2430+
2431+-- | Get the bottom most @n@ elements using the supplied comparison function.
2432+--
2433+{-# INLINE bottomBy #-}
2434+bottomBy :: (MonadIO m, Unbox a) =>
2435+ (a -> a -> Ordering)
2436+ -> Int
2437+ -> Fold m a (MutArray a)
2438+bottomBy cmp n = Fold step initial extract
2439+
2440+ where
2441+
2442+ initial = do
2443+ arr <- MA.pinnedNew n
2444+ if n <= 0
2445+ then return $ Done arr
2446+ else return $ Partial (arr, 0)
2447+
2448+ step (arr, i) x =
2449+ if i < n
2450+ then do
2451+ arr' <- MA.snoc arr x
2452+ MA.bubble cmp arr'
2453+ return $ Partial (arr', i + 1)
2454+ else do
2455+ x1 <- MA.getIndexUnsafe (i - 1) arr
2456+ case x `cmp` x1 of
2457+ LT -> do
2458+ MA.putIndexUnsafe (i - 1) arr x
2459+ MA.bubble cmp arr
2460+ return $ Partial (arr, i)
2461+ _ -> return $ Partial (arr, i)
2462+
2463+ extract = return . fst
2464+
2465+-- | Get the top @n@ elements using the supplied comparison function.
2466+--
2467+-- To get bottom n elements instead:
2468+--
2469+-- >>> bottomBy cmp = Fold.topBy (flip cmp)
2470+--
2471+-- Example:
2472+--
2473+-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2474+-- >>> Stream.fold (Fold.topBy compare 3) stream >>= MutArray.toList
2475+-- [17,11,9]
2476+--
2477+-- /Pre-release/
2478+--
2479+{-# INLINE topBy #-}
2480+topBy :: (MonadIO m, Unbox a) =>
2481+ (a -> a -> Ordering)
2482+ -> Int
2483+ -> Fold m a (MutArray a)
2484+topBy cmp = bottomBy (flip cmp)
2485+
2486+-- | Fold the input stream to top n elements.
2487+--
2488+-- Definition:
2489+--
2490+-- >>> top = Fold.topBy compare
2491+--
2492+-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2493+-- >>> Stream.fold (Fold.top 3) stream >>= MutArray.toList
2494+-- [17,11,9]
2495+--
2496+-- /Pre-release/
2497+{-# INLINE top #-}
2498+top :: (MonadIO m, Unbox a, Ord a) => Int -> Fold m a (MutArray a)
2499+top = bottomBy $ flip compare
2500+
2501+-- | Fold the input stream to bottom n elements.
2502+--
2503+-- Definition:
2504+--
2505+-- >>> bottom = Fold.bottomBy compare
2506+--
2507+-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2508+-- >>> Stream.fold (Fold.bottom 3) stream >>= MutArray.toList
2509+-- [1,2,3]
2510+--
2511+-- /Pre-release/
2512+{-# INLINE bottom #-}
2513+bottom :: (MonadIO m, Unbox a, Ord a) => Int -> Fold m a (MutArray a)
2514+bottom = bottomBy compare
2515+
2516+------------------------------------------------------------------------------
2517+-- Interspersed parsing
2518+------------------------------------------------------------------------------
2519+
2520+data IntersperseQState fs ps =
2521+ IntersperseQUnquoted !fs !ps
2522+ | IntersperseQQuoted !fs !ps
2523+ | IntersperseQQuotedEsc !fs !ps
2524+
2525+-- Useful for parsing CSV with quoting and escaping
2526+{-# INLINE intersperseWithQuotes #-}
2527+intersperseWithQuotes :: (Monad m, Eq a) =>
2528+ a -> a -> a -> Fold m a b -> Fold m b c -> Fold m a c
2529+intersperseWithQuotes
2530+ quote
2531+ esc
2532+ separator
2533+ (Fold stepL initialL extractL)
2534+ (Fold stepR initialR extractR) = Fold step initial extract
2535+
2536+ where
2537+
2538+ errMsg p status =
2539+ error $ "intersperseWithQuotes: " ++ p ++ " parsing fold cannot "
2540+ ++ status ++ " without input"
2541+
2542+ {-# INLINE initL #-}
2543+ initL mkState = do
2544+ resL <- initialL
2545+ case resL of
2546+ Partial sL ->
2547+ return $ Partial $ mkState sL
2548+ Done _ ->
2549+ errMsg "content" "succeed"
2550+
2551+ initial = do
2552+ res <- initialR
2553+ case res of
2554+ Partial sR -> initL (IntersperseQUnquoted sR)
2555+ Done b -> return $ Done b
2556+
2557+ {-# INLINE collect #-}
2558+ collect nextS sR b = do
2559+ res <- stepR sR b
2560+ case res of
2561+ Partial s ->
2562+ initL (nextS s)
2563+ Done c -> return (Done c)
2564+
2565+ {-# INLINE process #-}
2566+ process a sL sR nextState = do
2567+ r <- stepL sL a
2568+ case r of
2569+ Partial s -> return $ Partial (nextState sR s)
2570+ Done b -> collect nextState sR b
2571+
2572+ {-# INLINE processQuoted #-}
2573+ processQuoted a sL sR nextState = do
2574+ r <- stepL sL a
2575+ case r of
2576+ Partial s -> return $ Partial (nextState sR s)
2577+ Done _ -> error "Collecting fold finished inside quote"
2578+
2579+ step (IntersperseQUnquoted sR sL) a
2580+ | a == separator = do
2581+ b <- extractL sL
2582+ collect IntersperseQUnquoted sR b
2583+ | a == quote = processQuoted a sL sR IntersperseQQuoted
2584+ | otherwise = process a sL sR IntersperseQUnquoted
2585+
2586+ step (IntersperseQQuoted sR sL) a
2587+ | a == esc = processQuoted a sL sR IntersperseQQuotedEsc
2588+ | a == quote = process a sL sR IntersperseQUnquoted
2589+ | otherwise = processQuoted a sL sR IntersperseQQuoted
2590+
2591+ step (IntersperseQQuotedEsc sR sL) a =
2592+ processQuoted a sL sR IntersperseQQuoted
2593+
2594+ extract (IntersperseQUnquoted sR _) = extractR sR
2595+ extract (IntersperseQQuoted _ _) =
2596+ error "intersperseWithQuotes: finished inside quote"
2597+ extract (IntersperseQQuotedEsc _ _) =
2598+ error "intersperseWithQuotes: finished inside quote, at escape char"
core/src/Streamly/Internal/Data/Fold/Chunked.hs+18−20
@@ -6,7 +6,7 @@
66-- Stability : experimental
77-- Portability : GHC
88--
9−-- Use "Streamly.Data.Parser" instead.
9+-- Use "Streamly.Data.Parser.Chunked" instead.
1010--
1111-- Fold a stream of foreign arrays. @Fold m a b@ in this module works
1212-- on a stream of "Array a" and produces an output of type @b@.
@@ -19,19 +19,18 @@
1919-- folds in Data.Fold to correctly work on an array stream as if it is an
2020-- element stream. For example:
2121--
22−-- >> import qualified Streamly.Data.Fold as Fold
23−-- >> import qualified Streamly.Internal.Data.Array.Stream as ArrayStream
24−-- >> import qualified Streamly.Internal.Data.Fold.Chunked as ChunkFold
25−-- >> import qualified Streamly.Data.Stream as Stream
26−-- >> import qualified Streamly.Data.StreamK as StreamK
22+-- >>> import qualified Streamly.Data.Fold as Fold
23+-- >>> import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
24+-- >>> import qualified Streamly.Internal.Data.Fold.Chunked as ChunkFold
25+-- >>> import qualified Streamly.Data.Stream as Stream
26+-- >>> import qualified Streamly.Data.StreamK as StreamK
2727--
28−-- >> f = ChunkFold.fromFold (Fold.take 7 Fold.toList)
29−-- >> s = Stream.chunksOf 5 $ Stream.fromList "hello world"
30−-- >> ArrayStream.runArrayFold f (StreamK.fromStream s)
28+-- >>> f = ChunkFold.fromFold (Fold.take 7 Fold.toList)
29+-- >>> s = Stream.chunksOf 5 $ Stream.fromList "hello world"
30+-- >>> ArrayStream.runArrayFold f (StreamK.fromStream s)
3131-- Right "hello w"
3232--
3333module Streamly.Internal.Data.Fold.Chunked
34− {-# DEPRECATED "Please use Streamly.Data.Parser instead." #-}
3534 (
3635 ChunkFold (..)
3736
@@ -59,22 +58,22 @@ where
5958
6059#include "ArrayMacros.h"
6160
62−#if !MIN_VERSION_base(4,18,0)
6361import Control.Applicative (liftA2)
64−#endif
6562import Control.Exception (assert)
6663import Control.Monad.IO.Class (MonadIO(..))
6764import Data.Bifunctor (first)
6865import Data.Proxy (Proxy(..))
6966import Streamly.Internal.Data.Unbox (Unbox(..))
7067import GHC.Types (SPEC(..))
71−import Streamly.Internal.Data.Array (Array(..))
68+import Streamly.Internal.Data.MutArray.Type (touch)
69+import Streamly.Internal.Data.Array.Type (Array(..))
7270import Streamly.Internal.Data.Parser (Initial(..), Step(..))
7371import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
7472
75−import qualified Streamly.Internal.Data.Array.Type as Array
73+import qualified Streamly.Internal.Data.Array as Array
7674import qualified Streamly.Internal.Data.Fold as Fold
7775import qualified Streamly.Internal.Data.Parser as ParserD
76+import qualified Streamly.Internal.Data.Parser.ParserD.Type as ParserD
7877import qualified Streamly.Internal.Data.Parser as Parser
7978
8079import Prelude hiding (concatMap, take)
@@ -103,8 +102,8 @@ newtype ChunkFold m a b = ChunkFold (ParserD.Parser (Array a) m b)
103102{-# INLINE fromFold #-}
104103fromFold :: forall m a b. (MonadIO m, Unbox a) =>
105104 Fold.Fold m a b -> ChunkFold m a b
106−fromFold (Fold.Fold fstep finitial _ ffinal) =
107− ChunkFold (ParserD.Parser step initial extract)
105+fromFold (Fold.Fold fstep finitial fextract) =
106+ ChunkFold (ParserD.Parser step initial (fmap (Done 0) . fextract))
108107
109108 where
110109
@@ -124,7 +123,7 @@ fromFold (Fold.Fold fstep finitial _ ffinal) =
124123 assert (cur == end) (return ())
125124 return $ Partial 0 fs
126125 goArray !_ !cur !fs = do
127− x <- liftIO $ peekAt cur contents
126+ x <- liftIO $ peekByteIndex cur contents
128127 res <- fstep fs x
129128 let elemSize = SIZE_OF(a)
130129 next = INDEX_NEXT(cur,a)
@@ -134,8 +133,6 @@ fromFold (Fold.Fold fstep finitial _ ffinal) =
134133 Fold.Partial fs1 ->
135134 goArray SPEC next fs1
136135
137− extract = fmap (Done 0) . ffinal
138−
139136-- | Convert an element 'ParserD.Parser' into an array stream fold. If the
140137-- parser fails the fold would throw an exception.
141138--
@@ -163,7 +160,8 @@ fromParserD (ParserD.Parser step1 initial1 extract1) =
163160 else return $ st (arrRem + n) fs1
164161
165162 goArray !_ !cur !fs = do
166− x <- liftIO $ peekAt cur contents
163+ x <- liftIO $ peekByteIndex cur contents
164+ liftIO $ touch contents
167165 res <- step1 fs x
168166 let elemSize = SIZE_OF(a)
169167 next = INDEX_NEXT(cur,a)
core/src/Streamly/Internal/Data/Fold/Combinators.hsdeleted−2452
@@ -1,2452 +0,0 @@
1−{-# LANGUAGE CPP #-}
2−-- |
3−-- Module : Streamly.Internal.Data.Fold.Combinators
4−-- Copyright : (c) 2019 Composewell Technologies
5−-- (c) 2013 Gabriel Gonzalez
6−-- License : BSD3
7−-- Maintainer : streamly@composewell.com
8−-- Stability : experimental
9−-- Portability : GHC
10−--
11−-- See "Streamly.Data.Fold" for an overview and
12−-- "Streamly.Internal.Data.Fold.Type" for design notes.
13−
14−module Streamly.Internal.Data.Fold.Combinators
15− (
16− -- * Mappers
17− -- | Monadic functions useful with mapM/lmapM on folds or streams.
18− tracing
19− , trace
20−
21− -- * Folds
22−
23− -- ** Accumulators
24− -- *** Semigroups and Monoids
25− , sconcat
26− , mconcat
27− , foldMap
28− , foldMapM
29−
30− -- *** Reducers
31− , drainMapM
32− , the
33− , mean
34− , rollingHash
35− , defaultSalt
36− , rollingHashWithSalt
37− , rollingHashFirstN
38− -- , rollingHashLastN
39−
40− -- *** Saturating Reducers
41− -- | 'product' terminates if it becomes 0. Other folds can theoretically
42− -- saturate on bounded types, and therefore terminate, however, they will
43− -- run forever on unbounded types like Integer/Double.
44− , sum
45− , product
46− , maximumBy
47− , maximum
48− , minimumBy
49− , minimum
50−
51− -- *** Collectors
52− -- | Avoid using these folds in scalable or performance critical
53− -- applications, they buffer all the input in GC memory which can be
54− -- detrimental to performance if the input is large.
55− , toListRev
56− -- $toListRev
57− , toStream
58− , toStreamRev
59− , topBy
60− , top
61− , bottomBy
62− , bottom
63−
64− -- *** Scanners
65− -- | Stateful transformation of the elements. Useful in combination with
66− -- the 'scanMaybe' combinator. For scanners the result of the fold is
67− -- usually a transformation of the current element rather than an
68− -- aggregation of all elements till now.
69− , latest
70− -- , nthLast -- using Ring array
71− , indexingWith
72− , indexing
73− , indexingRev
74− , rollingMapM
75−
76− -- *** Filters
77− -- | Useful in combination with the 'scanMaybe' combinator.
78− , deleteBy
79− , uniqBy
80− , uniq
81− , repeated
82− , findIndices
83− , elemIndices
84−
85− -- *** Singleton folds
86− -- | Folds that terminate after consuming exactly one input element. All
87− -- these can be implemented in terms of the 'maybe' fold.
88− , one
89− , null -- XXX not very useful and could be problematic, remove it?
90− , satisfy
91− , maybe
92−
93− -- *** Multi folds
94− -- | Terminate after consuming one or more elements.
95− , drainN
96− -- , lastN
97− -- , (!!)
98− , indexGeneric
99− , index
100− , findM
101− , find
102− , lookup
103− , findIndex
104− , elemIndex
105− , elem
106− , notElem
107− , all
108− , any
109− , and
110− , or
111−
112− -- ** Trimmers
113− -- | Useful in combination with the 'scanMaybe' combinator.
114− , takingEndByM
115− , takingEndBy
116− , takingEndByM_
117− , takingEndBy_
118− , droppingWhileM
119− , droppingWhile
120− , prune
121−
122− -- * Running A Fold
123− , drive
124− -- , breakStream
125−
126− -- * Building Incrementally
127− , addStream
128−
129− -- * Combinators
130− -- ** Utilities
131− , with
132−
133− -- ** Mapping on Input
134− , transform
135−
136− -- ** Sliding Window
137− , slide2
138−
139− -- ** Scanning Input
140− , scan
141− , scanMany
142− , indexed
143−
144− -- ** Zipping Input
145− , zipStreamWithM
146− , zipStream
147−
148− -- ** Filtering Input
149− , mapMaybeM
150− , mapMaybe
151− , sampleFromthen
152−
153− {-
154− -- ** Insertion
155− -- | Insertion adds more elements to the stream.
156−
157− , insertBy
158− , intersperseM
159−
160− -- ** Reordering
161− , reverse
162− -}
163−
164− -- ** Trimming
165−
166− -- By elements
167− , takeEndBySeq
168− , takeEndBySeq_
169− {-
170− , drop
171− , dropWhile
172− , dropWhileM
173− -}
174−
175− -- ** Serial Append
176− -- , tail
177− -- , init
178− , splitAt -- spanN
179− -- , splitIn -- sessionN
180−
181− -- ** Parallel Distribution
182− , tee
183− , distribute
184− -- , distributeFst
185− -- , distributeMin
186−
187− -- ** Unzipping
188− , unzip
189− -- These two can be expressed using lmap/lmapM and unzip
190− , unzipWith
191− , unzipWithM
192− , unzipWithFstM
193− , unzipWithMinM
194−
195− -- ** Partitioning
196− , partitionByM
197− , partitionByFstM
198− , partitionByMinM
199− , partitionBy
200− , partition
201−
202− -- ** Splitting
203− , chunksBetween
204− , intersperseWithQuotes
205−
206− -- ** Nesting
207− , unfoldMany
208− , concatSequence
209−
210− -- * Deprecated
211− , drainBy
212− , last
213− , head
214− , sequence
215− , mapM
216− , variance
217− , stdDev
218− )
219−where
220−
221−#include "inline.hs"
222−#include "ArrayMacros.h"
223−
224−import Control.Monad (void)
225−import Control.Monad.IO.Class (MonadIO(..))
226−import Data.Bifunctor (first)
227−import Data.Bits (shiftL, shiftR, (.|.), (.&.))
228−import Data.Either (isLeft, isRight, fromLeft, fromRight)
229−import Data.Int (Int64)
230−import Data.Proxy (Proxy(..))
231−import Data.Word (Word32)
232−import Foreign.Storable (Storable, peek)
233−import Streamly.Internal.Data.MutArray.Type (MutArray(..))
234−import Streamly.Internal.Data.Maybe.Strict (Maybe'(..), toMaybe)
235−import Streamly.Internal.Data.Pipe.Type (Pipe (..), PipeState(..))
236−import Streamly.Internal.Data.Unbox (Unbox, sizeOf)
237−import Streamly.Internal.Data.Unfold.Type (Unfold(..))
238−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3'(..))
239−import Streamly.Internal.Data.Stream.Type (Stream)
240−
241−import qualified Prelude
242−import qualified Streamly.Internal.Data.MutArray.Type as MA
243−import qualified Streamly.Internal.Data.Array.Type as Array
244−import qualified Streamly.Internal.Data.Fold.Window as Fold
245−import qualified Streamly.Internal.Data.Pipe.Type as Pipe
246−import qualified Streamly.Internal.Data.Ring as Ring
247−import qualified Streamly.Internal.Data.Stream.Type as StreamD
248−
249−import Prelude hiding
250− ( Foldable(..), filter, drop, dropWhile, take, takeWhile, zipWith
251− , map, mapM_, sequence, all, any
252− , notElem, head, last, tail
253− , reverse, iterate, init, and, or, lookup, (!!)
254− , scanl, scanl1, replicate, concatMap, mconcat, unzip
255− , span, splitAt, break, mapM, zip, maybe)
256−import Streamly.Internal.Data.Fold.Type
257−
258−#include "DocTestDataFold.hs"
259−
260−------------------------------------------------------------------------------
261−-- Running
262−------------------------------------------------------------------------------
263−
264−-- | Drive a fold using the supplied 'Stream', reducing the resulting
265−-- expression strictly at each step.
266−--
267−-- Definition:
268−--
269−-- >>> drive = flip Stream.fold
270−--
271−-- Example:
272−--
273−-- >>> Fold.drive (Stream.enumerateFromTo 1 100) Fold.sum
274−-- 5050
275−--
276−{-# INLINE drive #-}
277−drive :: Monad m => Stream m a -> Fold m a b -> m b
278−drive = flip StreamD.fold
279−
280−{-
281−-- | Like 'drive' but also returns the remaining stream. The resulting stream
282−-- would be 'Stream.nil' if the stream finished before the fold.
283−--
284−-- Definition:
285−--
286−-- >>> breakStream = flip Stream.foldBreak
287−--
288−-- /CPS/
289−--
290−{-# INLINE breakStreamK #-}
291−breakStreamK :: Monad m => StreamK m a -> Fold m a b -> m (b, StreamK m a)
292−breakStreamK strm fl = fmap f $ K.foldBreak fl (Stream.toStreamK strm)
293−
294− where
295−
296− f (b, str) = (b, Stream.fromStreamK str)
297−-}
298−
299−-- | Append a stream to a fold to build the fold accumulator incrementally. We
300−-- can repeatedly call 'addStream' on the same fold to continue building the
301−-- fold and finally use 'drive' to finish the fold and extract the result. Also
302−-- see the 'Streamly.Data.Fold.addOne' operation which is a singleton version
303−-- of 'addStream'.
304−--
305−-- Definitions:
306−--
307−-- >>> addStream stream = Fold.drive stream . Fold.duplicate
308−--
309−-- Example, build a list incrementally:
310−--
311−-- >>> :{
312−-- pure (Fold.toList :: Fold IO Int [Int])
313−-- >>= Fold.addOne 1
314−-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
315−-- >>= Fold.drive Stream.nil
316−-- >>= print
317−-- :}
318−-- [1,2,3,4]
319−--
320−-- This can be used as an O(n) list append compared to the O(n^2) @++@ when
321−-- used for incrementally building a list.
322−--
323−-- Example, build a stream incrementally:
324−--
325−-- >>> :{
326−-- pure (Fold.toStream :: Fold IO Int (Stream Identity Int))
327−-- >>= Fold.addOne 1
328−-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
329−-- >>= Fold.drive Stream.nil
330−-- >>= print
331−-- :}
332−-- fromList [1,2,3,4]
333−--
334−-- This can be used as an O(n) stream append compared to the O(n^2) @<>@ when
335−-- used for incrementally building a stream.
336−--
337−-- Example, build an array incrementally:
338−--
339−-- >>> :{
340−-- pure (Array.write :: Fold IO Int (Array Int))
341−-- >>= Fold.addOne 1
342−-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
343−-- >>= Fold.drive Stream.nil
344−-- >>= print
345−-- :}
346−-- fromList [1,2,3,4]
347−--
348−-- Example, build an array stream incrementally:
349−--
350−-- >>> :{
351−-- let f :: Fold IO Int (Stream Identity (Array Int))
352−-- f = Fold.groupsOf 2 (Array.writeN 3) Fold.toStream
353−-- in pure f
354−-- >>= Fold.addOne 1
355−-- >>= Fold.addStream (Stream.enumerateFromTo 2 4)
356−-- >>= Fold.drive Stream.nil
357−-- >>= print
358−-- :}
359−-- fromList [fromList [1,2],fromList [3,4]]
360−--
361−addStream :: Monad m => Stream m a -> Fold m a b -> m (Fold m a b)
362−addStream stream = drive stream . duplicate
363−
364−------------------------------------------------------------------------------
365−-- Transformations on fold inputs
366−------------------------------------------------------------------------------
367−
368−-- | Flatten the monadic output of a fold to pure output.
369−--
370−{-# DEPRECATED sequence "Use \"rmapM id\" instead" #-}
371−{-# INLINE sequence #-}
372−sequence :: Monad m => Fold m a (m b) -> Fold m a b
373−sequence = rmapM id
374−
375−-- | Map a monadic function on the output of a fold.
376−--
377−{-# DEPRECATED mapM "Use rmapM instead" #-}
378−{-# INLINE mapM #-}
379−mapM :: Monad m => (b -> m c) -> Fold m a b -> Fold m a c
380−mapM = rmapM
381−
382−-- |
383−-- >>> mapMaybeM f = Fold.lmapM f . Fold.catMaybes
384−--
385−{-# INLINE mapMaybeM #-}
386−mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Fold m b r -> Fold m a r
387−mapMaybeM f = lmapM f . catMaybes
388−
389−-- | @mapMaybe f fold@ maps a 'Maybe' returning function @f@ on the input of
390−-- the fold, filters out 'Nothing' elements, and return the values extracted
391−-- from 'Just'.
392−--
393−-- >>> mapMaybe f = Fold.lmap f . Fold.catMaybes
394−-- >>> mapMaybe f = Fold.mapMaybeM (return . f)
395−--
396−-- >>> f x = if even x then Just x else Nothing
397−-- >>> fld = Fold.mapMaybe f Fold.toList
398−-- >>> Stream.fold fld (Stream.enumerateFromTo 1 10)
399−-- [2,4,6,8,10]
400−--
401−{-# INLINE mapMaybe #-}
402−mapMaybe :: Monad m => (a -> Maybe b) -> Fold m b r -> Fold m a r
403−mapMaybe f = lmap f . catMaybes
404−
405−------------------------------------------------------------------------------
406−-- Transformations on fold inputs
407−------------------------------------------------------------------------------
408−
409−-- | Apply a monadic function on the input and return the input.
410−--
411−-- >>> Stream.fold (Fold.lmapM (Fold.tracing print) Fold.drain) $ (Stream.enumerateFromTo (1 :: Int) 2)
412−-- 1
413−-- 2
414−--
415−-- /Pre-release/
416−--
417−{-# INLINE tracing #-}
418−tracing :: Monad m => (a -> m b) -> (a -> m a)
419−tracing f x = void (f x) >> return x
420−
421−-- | Apply a monadic function to each element flowing through and discard the
422−-- results.
423−--
424−-- >>> Stream.fold (Fold.trace print Fold.drain) $ (Stream.enumerateFromTo (1 :: Int) 2)
425−-- 1
426−-- 2
427−--
428−-- >>> trace f = Fold.lmapM (Fold.tracing f)
429−--
430−-- /Pre-release/
431−{-# INLINE trace #-}
432−trace :: Monad m => (a -> m b) -> Fold m a r -> Fold m a r
433−trace f = lmapM (tracing f)
434−
435−-- rename to lpipe?
436−--
437−-- | Apply a transformation on a 'Fold' using a 'Pipe'.
438−--
439−-- /Pre-release/
440−{-# INLINE transform #-}
441−transform :: Monad m => Pipe m a b -> Fold m b c -> Fold m a c
442−transform (Pipe pstep1 pstep2 pinitial) (Fold fstep finitial fextract ffinal) =
443− Fold step initial extract final
444−
445− where
446−
447− initial = first (Tuple' pinitial) <$> finitial
448−
449− step (Tuple' ps fs) x = do
450− r <- pstep1 ps x
451− go fs r
452−
453− where
454−
455− -- XXX use SPEC?
456− go acc (Pipe.Yield b (Consume ps')) = do
457− acc' <- fstep acc b
458− return
459− $ case acc' of
460− Partial s -> Partial $ Tuple' ps' s
461− Done b2 -> Done b2
462− go acc (Pipe.Yield b (Produce ps')) = do
463− acc' <- fstep acc b
464− r <- pstep2 ps'
465− case acc' of
466− Partial s -> go s r
467− Done b2 -> return $ Done b2
468− go acc (Pipe.Continue (Consume ps')) =
469− return $ Partial $ Tuple' ps' acc
470− go acc (Pipe.Continue (Produce ps')) = do
471− r <- pstep2 ps'
472− go acc r
473−
474− extract (Tuple' _ fs) = fextract fs
475−
476− final (Tuple' _ fs) = ffinal fs
477−
478−{-# INLINE scanWith #-}
479−scanWith :: Monad m => Bool -> Fold m a b -> Fold m b c -> Fold m a c
480−scanWith isMany
481− (Fold stepL initialL extractL finalL)
482− (Fold stepR initialR extractR finalR) =
483− Fold step initial extract final
484−
485− where
486−
487− {-# INLINE runStep #-}
488− runStep actionL sR = do
489− rL <- actionL
490− case rL of
491− Done bL -> do
492− rR <- stepR sR bL
493− case rR of
494− Partial sR1 ->
495− if isMany
496− then runStep initialL sR1
497− else Done <$> finalR sR1
498− Done bR -> return $ Done bR
499− Partial sL -> do
500− !b <- extractL sL
501− rR <- stepR sR b
502− case rR of
503− Partial sR1 -> return $ Partial (sL, sR1)
504− Done bR -> finalL sL >> return (Done bR)
505−
506− initial = do
507− r <- initialR
508− case r of
509− Partial sR -> runStep initialL sR
510− Done b -> return $ Done b
511−
512− step (sL, sR) x = runStep (stepL sL x) sR
513−
514− extract = extractR . snd
515−
516− final (sL, sR) = finalL sL *> finalR sR
517−
518−-- | Scan the input of a 'Fold' to change it in a stateful manner using another
519−-- 'Fold'. The scan stops as soon as the fold terminates.
520−--
521−-- /Pre-release/
522−{-# INLINE scan #-}
523−scan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
524−scan = scanWith False
525−
526−-- XXX This does not fuse beacuse of the recursive step. Need to investigate.
527−--
528−-- | Scan the input of a 'Fold' to change it in a stateful manner using another
529−-- 'Fold'. The scan restarts with a fresh state if the fold terminates.
530−--
531−-- /Pre-release/
532−{-# INLINE scanMany #-}
533−scanMany :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
534−scanMany = scanWith True
535−
536−------------------------------------------------------------------------------
537−-- Filters
538−------------------------------------------------------------------------------
539−
540−-- | Returns the latest element omitting the first occurrence that satisfies
541−-- the given equality predicate.
542−--
543−-- Example:
544−--
545−-- >>> input = Stream.fromList [1,3,3,5]
546−-- >>> Stream.fold Fold.toList $ Stream.scanMaybe (Fold.deleteBy (==) 3) input
547−-- [1,3,5]
548−--
549−{-# INLINE_NORMAL deleteBy #-}
550−deleteBy :: Monad m => (a -> a -> Bool) -> a -> Fold m a (Maybe a)
551−deleteBy eq x0 = fmap extract $ foldl' step (Tuple' False Nothing)
552−
553− where
554−
555− step (Tuple' False _) x =
556− if eq x x0
557− then Tuple' True Nothing
558− else Tuple' False (Just x)
559− step (Tuple' True _) x = Tuple' True (Just x)
560−
561− extract (Tuple' _ x) = x
562−
563−-- | Provide a sliding window of length 2 elements.
564−--
565−-- See "Streamly.Internal.Data.Fold.Window".
566−--
567−{-# INLINE slide2 #-}
568−slide2 :: Monad m => Fold m (a, Maybe a) b -> Fold m a b
569−slide2 (Fold step1 initial1 extract1 final1) = Fold step initial extract final
570−
571− where
572−
573− initial =
574− first (Tuple' Nothing) <$> initial1
575−
576− step (Tuple' prev s) cur =
577− first (Tuple' (Just cur)) <$> step1 s (cur, prev)
578−
579− extract (Tuple' _ s) = extract1 s
580−
581− final (Tuple' _ s) = final1 s
582−
583−-- | Return the latest unique element using the supplied comparison function.
584−-- Returns 'Nothing' if the current element is same as the last element
585−-- otherwise returns 'Just'.
586−--
587−-- Example, strip duplicate path separators:
588−--
589−-- >>> input = Stream.fromList "//a//b"
590−-- >>> f x y = x == '/' && y == '/'
591−-- >>> Stream.fold Fold.toList $ Stream.scanMaybe (Fold.uniqBy f) input
592−-- "/a/b"
593−--
594−-- Space: @O(1)@
595−--
596−-- /Pre-release/
597−--
598−{-# INLINE uniqBy #-}
599−uniqBy :: Monad m => (a -> a -> Bool) -> Fold m a (Maybe a)
600−uniqBy eq = rollingMap f
601−
602− where
603−
604− f pre curr =
605− case pre of
606− Nothing -> Just curr
607− Just x -> if x `eq` curr then Nothing else Just curr
608−
609−-- | See 'uniqBy'.
610−--
611−-- Definition:
612−--
613−-- >>> uniq = Fold.uniqBy (==)
614−--
615−{-# INLINE uniq #-}
616−uniq :: (Monad m, Eq a) => Fold m a (Maybe a)
617−uniq = uniqBy (==)
618−
619−-- | Strip all leading and trailing occurrences of an element passing a
620−-- predicate and make all other consecutive occurrences uniq.
621−--
622−-- >> prune p = Stream.dropWhileAround p $ Stream.uniqBy (x y -> p x && p y)
623−--
624−-- @
625−-- > Stream.prune isSpace (Stream.fromList " hello world! ")
626−-- "hello world!"
627−--
628−-- @
629−--
630−-- Space: @O(1)@
631−--
632−-- /Unimplemented/
633−{-# INLINE prune #-}
634−prune ::
635− -- (Monad m, Eq a) =>
636− (a -> Bool) -> Fold m a (Maybe a)
637−prune = error "Not implemented yet!"
638−
639−-- | Emit only repeated elements, once.
640−--
641−-- /Unimplemented/
642−repeated :: -- (Monad m, Eq a) =>
643− Fold m a (Maybe a)
644−repeated = error "Not implemented yet!"
645−
646−------------------------------------------------------------------------------
647−-- Left folds
648−------------------------------------------------------------------------------
649−
650−------------------------------------------------------------------------------
651−-- Run Effects
652−------------------------------------------------------------------------------
653−
654−-- |
655−-- Definitions:
656−--
657−-- >>> drainMapM f = Fold.lmapM f Fold.drain
658−-- >>> drainMapM f = Fold.foldMapM (void . f)
659−--
660−-- Drain all input after passing it through a monadic function. This is the
661−-- dual of mapM_ on stream producers.
662−--
663−{-# INLINE drainMapM #-}
664−drainMapM :: Monad m => (a -> m b) -> Fold m a ()
665−drainMapM f = lmapM f drain
666−
667−{-# DEPRECATED drainBy "Please use 'drainMapM' instead." #-}
668−{-# INLINE drainBy #-}
669−drainBy :: Monad m => (a -> m b) -> Fold m a ()
670−drainBy = drainMapM
671−
672−-- | Returns the latest element of the input stream, if any.
673−--
674−-- >>> latest = Fold.foldl1' (\_ x -> x)
675−-- >>> latest = fmap getLast $ Fold.foldMap (Last . Just)
676−--
677−{-# INLINE latest #-}
678−latest :: Monad m => Fold m a (Maybe a)
679−latest = foldl1' (\_ x -> x)
680−
681−{-# DEPRECATED last "Please use 'latest' instead." #-}
682−{-# INLINE last #-}
683−last :: Monad m => Fold m a (Maybe a)
684−last = latest
685−
686−-- | Terminates with 'Nothing' as soon as it finds an element different than
687−-- the previous one, returns 'the' element if the entire input consists of the
688−-- same element.
689−--
690−{-# INLINE the #-}
691−the :: (Monad m, Eq a) => Fold m a (Maybe a)
692−the = foldt' step initial id
693−
694− where
695−
696− initial = Partial Nothing
697−
698− step Nothing x = Partial (Just x)
699− step old@(Just x0) x =
700− if x0 == x
701− then Partial old
702− else Done Nothing
703−
704−------------------------------------------------------------------------------
705−-- To Summary
706−------------------------------------------------------------------------------
707−
708−-- | Determine the sum of all elements of a stream of numbers. Returns additive
709−-- identity (@0@) when the stream is empty. Note that this is not numerically
710−-- stable for floating point numbers.
711−--
712−-- >>> sum = Fold.cumulative Fold.windowSum
713−--
714−-- Same as following but numerically stable:
715−--
716−-- >>> sum = Fold.foldl' (+) 0
717−-- >>> sum = fmap Data.Monoid.getSum $ Fold.foldMap Data.Monoid.Sum
718−--
719−{-# INLINE sum #-}
720−sum :: (Monad m, Num a) => Fold m a a
721−sum = Fold.cumulative Fold.windowSum
722−
723−-- | Determine the product of all elements of a stream of numbers. Returns
724−-- multiplicative identity (@1@) when the stream is empty. The fold terminates
725−-- when it encounters (@0@) in its input.
726−--
727−-- Same as the following but terminates on multiplication by @0@:
728−--
729−-- >>> product = fmap Data.Monoid.getProduct $ Fold.foldMap Data.Monoid.Product
730−--
731−{-# INLINE product #-}
732−product :: (Monad m, Num a, Eq a) => Fold m a a
733−product = foldt' step (Partial 1) id
734−
735− where
736−
737− step x a =
738− if a == 0
739− then Done 0
740− else Partial $ x * a
741−
742−------------------------------------------------------------------------------
743−-- To Summary (Maybe)
744−------------------------------------------------------------------------------
745−
746−-- | Determine the maximum element in a stream using the supplied comparison
747−-- function.
748−--
749−{-# INLINE maximumBy #-}
750−maximumBy :: Monad m => (a -> a -> Ordering) -> Fold m a (Maybe a)
751−maximumBy cmp = foldl1' max'
752−
753− where
754−
755− max' x y =
756− case cmp x y of
757− GT -> x
758− _ -> y
759−
760−-- | Determine the maximum element in a stream.
761−--
762−-- Definitions:
763−--
764−-- >>> maximum = Fold.maximumBy compare
765−-- >>> maximum = Fold.foldl1' max
766−--
767−-- Same as the following but without a default maximum. The 'Max' Monoid uses
768−-- the 'minBound' as the default maximum:
769−--
770−-- >>> maximum = fmap Data.Semigroup.getMax $ Fold.foldMap Data.Semigroup.Max
771−--
772−{-# INLINE maximum #-}
773−maximum :: (Monad m, Ord a) => Fold m a (Maybe a)
774−maximum = foldl1' max
775−
776−-- | Computes the minimum element with respect to the given comparison function
777−--
778−{-# INLINE minimumBy #-}
779−minimumBy :: Monad m => (a -> a -> Ordering) -> Fold m a (Maybe a)
780−minimumBy cmp = foldl1' min'
781−
782− where
783−
784− min' x y =
785− case cmp x y of
786− GT -> y
787− _ -> x
788−
789−-- | Determine the minimum element in a stream using the supplied comparison
790−-- function.
791−--
792−-- Definitions:
793−--
794−-- >>> minimum = Fold.minimumBy compare
795−-- >>> minimum = Fold.foldl1' min
796−--
797−-- Same as the following but without a default minimum. The 'Min' Monoid uses the
798−-- 'maxBound' as the default maximum:
799−--
800−-- >>> maximum = fmap Data.Semigroup.getMin $ Fold.foldMap Data.Semigroup.Min
801−--
802−{-# INLINE minimum #-}
803−minimum :: (Monad m, Ord a) => Fold m a (Maybe a)
804−minimum = foldl1' min
805−
806−------------------------------------------------------------------------------
807−-- To Summary (Statistical)
808−------------------------------------------------------------------------------
809−
810−-- | Compute a numerically stable arithmetic mean of all elements in the input
811−-- stream.
812−--
813−{-# INLINE mean #-}
814−mean :: (Monad m, Fractional a) => Fold m a a
815−mean = fmap done $ foldl' step begin
816−
817− where
818−
819− begin = Tuple' 0 0
820−
821− step (Tuple' x n) y =
822− let n1 = n + 1
823− in Tuple' (x + (y - x) / n1) n1
824−
825− done (Tuple' x _) = x
826−
827−-- | Compute a numerically stable (population) variance over all elements in
828−-- the input stream.
829−--
830−{-# DEPRECATED variance "Use the streamly-statistics package instead" #-}
831−{-# INLINE variance #-}
832−variance :: (Monad m, Fractional a) => Fold m a a
833−variance = fmap done $ foldl' step begin
834−
835− where
836−
837− begin = Tuple3' 0 0 0
838−
839− step (Tuple3' n mean_ m2) x = Tuple3' n' mean' m2'
840−
841− where
842−
843− n' = n + 1
844− mean' = (n * mean_ + x) / (n + 1)
845− delta = x - mean_
846− m2' = m2 + delta * delta * n / (n + 1)
847−
848− done (Tuple3' n _ m2) = m2 / n
849−
850−-- | Compute a numerically stable (population) standard deviation over all
851−-- elements in the input stream.
852−--
853−{-# DEPRECATED stdDev "Use the streamly-statistics package instead" #-}
854−{-# INLINE stdDev #-}
855−stdDev :: (Monad m, Floating a) => Fold m a a
856−stdDev = sqrt <$> variance
857−
858−-- | Compute an 'Int' sized polynomial rolling hash
859−--
860−-- > H = salt * k ^ n + c1 * k ^ (n - 1) + c2 * k ^ (n - 2) + ... + cn * k ^ 0
861−--
862−-- Where @c1@, @c2@, @cn@ are the elements in the input stream and @k@ is a
863−-- constant.
864−--
865−-- This hash is often used in Rabin-Karp string search algorithm.
866−--
867−-- See https://en.wikipedia.org/wiki/Rolling_hash
868−--
869−{-# INLINE rollingHashWithSalt #-}
870−rollingHashWithSalt :: (Monad m, Enum a) => Int64 -> Fold m a Int64
871−rollingHashWithSalt = foldl' step
872−
873− where
874−
875− k = 2891336453 :: Int64
876−
877− step cksum a = cksum * k + fromIntegral (fromEnum a)
878−
879−-- | A default salt used in the implementation of 'rollingHash'.
880−{-# INLINE defaultSalt #-}
881−defaultSalt :: Int64
882−defaultSalt = -2578643520546668380
883−
884−-- | Compute an 'Int' sized polynomial rolling hash of a stream.
885−--
886−-- >>> rollingHash = Fold.rollingHashWithSalt Fold.defaultSalt
887−--
888−{-# INLINE rollingHash #-}
889−rollingHash :: (Monad m, Enum a) => Fold m a Int64
890−rollingHash = rollingHashWithSalt defaultSalt
891−
892−-- | Compute an 'Int' sized polynomial rolling hash of the first n elements of
893−-- a stream.
894−--
895−-- >>> rollingHashFirstN n = Fold.take n Fold.rollingHash
896−--
897−-- /Pre-release/
898−{-# INLINE rollingHashFirstN #-}
899−rollingHashFirstN :: (Monad m, Enum a) => Int -> Fold m a Int64
900−rollingHashFirstN n = take n rollingHash
901−
902−-- XXX Compare this with the implementation in Fold.Window, preferrably use the
903−-- latter if performance is good.
904−
905−-- | Apply a function on every two successive elements of a stream. The first
906−-- argument of the map function is the previous element and the second argument
907−-- is the current element. When processing the very first element in the
908−-- stream, the previous element is 'Nothing'.
909−--
910−-- /Pre-release/
911−--
912−{-# INLINE rollingMapM #-}
913−rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Fold m a b
914−rollingMapM f = Fold step initial extract extract
915−
916− where
917−
918− -- XXX We need just a postscan. We do not need an initial result here.
919− -- Or we can supply a default initial result as an argument to rollingMapM.
920− initial = return $ Partial (Nothing, error "Empty stream")
921−
922− step (prev, _) cur = do
923− x <- f prev cur
924− return $ Partial (Just cur, x)
925−
926− extract = return . snd
927−
928−-- |
929−-- >>> rollingMap f = Fold.rollingMapM (\x y -> return $ f x y)
930−--
931−{-# INLINE rollingMap #-}
932−rollingMap :: Monad m => (Maybe a -> a -> b) -> Fold m a b
933−rollingMap f = rollingMapM (\x y -> return $ f x y)
934−
935−------------------------------------------------------------------------------
936−-- Monoidal left folds
937−------------------------------------------------------------------------------
938−
939−-- | Semigroup concat. Append the elements of an input stream to a provided
940−-- starting value.
941−--
942−-- Definition:
943−--
944−-- >>> sconcat = Fold.foldl' (<>)
945−--
946−-- >>> semigroups = fmap Data.Monoid.Sum $ Stream.enumerateFromTo 1 10
947−-- >>> Stream.fold (Fold.sconcat 10) semigroups
948−-- Sum {getSum = 65}
949−--
950−{-# INLINE sconcat #-}
951−sconcat :: (Monad m, Semigroup a) => a -> Fold m a a
952−sconcat = foldl' (<>)
953−
954−-- | Monoid concat. Fold an input stream consisting of monoidal elements using
955−-- 'mappend' and 'mempty'.
956−--
957−-- Definition:
958−--
959−-- >>> mconcat = Fold.sconcat mempty
960−--
961−-- >>> monoids = fmap Data.Monoid.Sum $ Stream.enumerateFromTo 1 10
962−-- >>> Stream.fold Fold.mconcat monoids
963−-- Sum {getSum = 55}
964−--
965−{-# INLINE mconcat #-}
966−mconcat ::
967− ( Monad m
968− , Monoid a) => Fold m a a
969−mconcat = sconcat mempty
970−
971−-- |
972−-- Definition:
973−--
974−-- >>> foldMap f = Fold.lmap f Fold.mconcat
975−--
976−-- Make a fold from a pure function that folds the output of the function
977−-- using 'mappend' and 'mempty'.
978−--
979−-- >>> sum = Fold.foldMap Data.Monoid.Sum
980−-- >>> Stream.fold sum $ Stream.enumerateFromTo 1 10
981−-- Sum {getSum = 55}
982−--
983−{-# INLINE foldMap #-}
984−foldMap :: (Monad m, Monoid b) => (a -> b) -> Fold m a b
985−foldMap f = lmap f mconcat
986−
987−-- |
988−-- Definition:
989−--
990−-- >>> foldMapM f = Fold.lmapM f Fold.mconcat
991−--
992−-- Make a fold from a monadic function that folds the output of the function
993−-- using 'mappend' and 'mempty'.
994−--
995−-- >>> sum = Fold.foldMapM (return . Data.Monoid.Sum)
996−-- >>> Stream.fold sum $ Stream.enumerateFromTo 1 10
997−-- Sum {getSum = 55}
998−--
999−{-# INLINE foldMapM #-}
1000−foldMapM :: (Monad m, Monoid b) => (a -> m b) -> Fold m a b
1001−foldMapM act = foldlM' step (pure mempty)
1002−
1003− where
1004−
1005− step m a = do
1006− m' <- act a
1007− return $! mappend m m'
1008−
1009−------------------------------------------------------------------------------
1010−-- To Containers
1011−------------------------------------------------------------------------------
1012−
1013−-- $toListRev
1014−-- This is more efficient than 'Streamly.Internal.Data.Fold.toList'. toList is
1015−-- exactly the same as reversing the list after 'toListRev'.
1016−
1017−-- | Buffers the input stream to a list in the reverse order of the input.
1018−--
1019−-- Definition:
1020−--
1021−-- >>> toListRev = Fold.foldl' (flip (:)) []
1022−--
1023−-- /Warning!/ working on large lists accumulated as buffers in memory could be
1024−-- very inefficient, consider using "Streamly.Array" instead.
1025−--
1026−
1027−-- xn : ... : x2 : x1 : []
1028−{-# INLINE toListRev #-}
1029−toListRev :: Monad m => Fold m a [a]
1030−toListRev = foldl' (flip (:)) []
1031−
1032−------------------------------------------------------------------------------
1033−-- Partial Folds
1034−------------------------------------------------------------------------------
1035−
1036−-- | A fold that drains the first n elements of its input, running the effects
1037−-- and discarding the results.
1038−--
1039−-- Definition:
1040−--
1041−-- >>> drainN n = Fold.take n Fold.drain
1042−--
1043−-- /Pre-release/
1044−{-# INLINE drainN #-}
1045−drainN :: Monad m => Int -> Fold m a ()
1046−drainN n = take n drain
1047−
1048−------------------------------------------------------------------------------
1049−-- To Elements
1050−------------------------------------------------------------------------------
1051−
1052−-- | Like 'index', except with a more general 'Integral' argument
1053−--
1054−-- /Pre-release/
1055−{-# INLINE indexGeneric #-}
1056−indexGeneric :: (Integral i, Monad m) => i -> Fold m a (Maybe a)
1057−indexGeneric i = foldt' step (Partial 0) (const Nothing)
1058−
1059− where
1060−
1061− step j a =
1062− if i == j
1063− then Done $ Just a
1064− else Partial (j + 1)
1065−
1066−-- | Return the element at the given index.
1067−--
1068−-- Definition:
1069−--
1070−-- >>> index = Fold.indexGeneric
1071−--
1072−{-# INLINE index #-}
1073−index :: Monad m => Int -> Fold m a (Maybe a)
1074−index = indexGeneric
1075−
1076−-- | Consume a single input and transform it using the supplied 'Maybe'
1077−-- returning function.
1078−--
1079−-- /Pre-release/
1080−--
1081−{-# INLINE maybe #-}
1082−maybe :: Monad m => (a -> Maybe b) -> Fold m a (Maybe b)
1083−maybe f = foldt' (const (Done . f)) (Partial Nothing) id
1084−
1085−-- | Consume a single element and return it if it passes the predicate else
1086−-- return 'Nothing'.
1087−--
1088−-- Definition:
1089−--
1090−-- >>> satisfy f = Fold.maybe (\a -> if f a then Just a else Nothing)
1091−--
1092−-- /Pre-release/
1093−{-# INLINE satisfy #-}
1094−satisfy :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1095−satisfy f = maybe (\a -> if f a then Just a else Nothing)
1096−{-
1097−satisfy f = Fold step (return $ Partial ()) (const (return Nothing))
1098−
1099− where
1100−
1101− step () a = return $ Done $ if f a then Just a else Nothing
1102−-}
1103−
1104−-- Naming notes:
1105−--
1106−-- "head" and "next" are two alternative names for the same API. head sounds
1107−-- apt in the context of lists but next sounds more apt in the context of
1108−-- streams where we think in terms of generating and consuming the next element
1109−-- rather than taking the head of some static/persistent structure.
1110−--
1111−-- We also want to keep the nomenclature consistent across folds and parsers,
1112−-- "head" becomes even more unintuitive for parsers because there are two
1113−-- possible variants viz. peek and next.
1114−--
1115−-- Also, the "head" fold creates confusion in situations like
1116−-- https://github.com/composewell/streamly/issues/1404 where intuitive
1117−-- expectation from head is to consume the entire stream and just give us the
1118−-- head. There we want to convey the notion that we consume one element from
1119−-- the stream and stop. The name "one" already being used in parsers for this
1120−-- purpose sounds more apt from this perspective.
1121−--
1122−-- The source of confusion is perhaps due to the fact that some folds consume
1123−-- the entire stream and others terminate early. It may have been clearer if we
1124−-- had separate abstractions for the two use cases.
1125−
1126−-- XXX We can possibly use "head" for the purposes of reducing the entire
1127−-- stream to the head element i.e. take the head and drain the rest.
1128−
1129−-- | Take one element from the stream and stop.
1130−--
1131−-- Definition:
1132−--
1133−-- >>> one = Fold.maybe Just
1134−--
1135−-- This is similar to the stream 'Stream.uncons' operation.
1136−--
1137−{-# INLINE one #-}
1138−one :: Monad m => Fold m a (Maybe a)
1139−one = maybe Just
1140−
1141−-- | Extract the first element of the stream, if any.
1142−--
1143−-- >>> head = Fold.one
1144−--
1145−{-# DEPRECATED head "Please use \"one\" instead" #-}
1146−{-# INLINE head #-}
1147−head :: Monad m => Fold m a (Maybe a)
1148−head = one
1149−
1150−-- | Returns the first element that satisfies the given predicate.
1151−--
1152−-- /Pre-release/
1153−{-# INLINE findM #-}
1154−findM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1155−findM predicate =
1156− Fold step (return $ Partial ()) extract extract
1157−
1158− where
1159−
1160− step () a =
1161− let f r =
1162− if r
1163− then Done (Just a)
1164− else Partial ()
1165− in f <$> predicate a
1166−
1167− extract = const $ return Nothing
1168−
1169−-- | Returns the first element that satisfies the given predicate.
1170−--
1171−{-# INLINE find #-}
1172−find :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1173−find p = findM (return . p)
1174−
1175−-- | In a stream of (key-value) pairs @(a, b)@, return the value @b@ of the
1176−-- first pair where the key equals the given value @a@.
1177−--
1178−-- Definition:
1179−--
1180−-- >>> lookup x = fmap snd <$> Fold.find ((== x) . fst)
1181−--
1182−{-# INLINE lookup #-}
1183−lookup :: (Eq a, Monad m) => a -> Fold m (a,b) (Maybe b)
1184−lookup a0 = foldt' step (Partial ()) (const Nothing)
1185−
1186− where
1187−
1188− step () (a, b) =
1189− if a == a0
1190− then Done $ Just b
1191− else Partial ()
1192−
1193−-- | Returns the first index that satisfies the given predicate.
1194−--
1195−{-# INLINE findIndex #-}
1196−findIndex :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
1197−findIndex predicate = foldt' step (Partial 0) (const Nothing)
1198−
1199− where
1200−
1201− step i a =
1202− if predicate a
1203− then Done $ Just i
1204− else Partial (i + 1)
1205−
1206−-- | Returns the index of the latest element if the element satisfies the given
1207−-- predicate.
1208−--
1209−{-# INLINE findIndices #-}
1210−findIndices :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
1211−findIndices predicate =
1212− -- XXX implement by combining indexing and filtering scans
1213− fmap (either (const Nothing) Just) $ foldl' step (Left (-1))
1214−
1215− where
1216−
1217− step i a =
1218− if predicate a
1219− then Right (either id id i + 1)
1220− else Left (either id id i + 1)
1221−
1222−-- | Returns the index of the latest element if the element matches the given
1223−-- value.
1224−--
1225−-- Definition:
1226−--
1227−-- >>> elemIndices a = Fold.findIndices (== a)
1228−--
1229−{-# INLINE elemIndices #-}
1230−elemIndices :: (Monad m, Eq a) => a -> Fold m a (Maybe Int)
1231−elemIndices a = findIndices (== a)
1232−
1233−-- | Returns the first index where a given value is found in the stream.
1234−--
1235−-- Definition:
1236−--
1237−-- >>> elemIndex a = Fold.findIndex (== a)
1238−--
1239−{-# INLINE elemIndex #-}
1240−elemIndex :: (Eq a, Monad m) => a -> Fold m a (Maybe Int)
1241−elemIndex a = findIndex (== a)
1242−
1243−------------------------------------------------------------------------------
1244−-- To Boolean
1245−------------------------------------------------------------------------------
1246−
1247−-- Similar to 'eof' parser, but the fold consumes and discards an input element
1248−-- when not at eof. XXX Remove or Rename to "eof"?
1249−
1250−-- | Consume one element, return 'True' if successful else return 'False'. In
1251−-- other words, test if the input is empty or not.
1252−--
1253−-- WARNING! It consumes one element if the stream is not empty. If that is not
1254−-- what you want please use the eof parser instead.
1255−--
1256−-- Definition:
1257−--
1258−-- >>> null = fmap isJust Fold.one
1259−--
1260−{-# INLINE null #-}
1261−null :: Monad m => Fold m a Bool
1262−null = foldt' (\() _ -> Done False) (Partial ()) (const True)
1263−
1264−-- | Returns 'True' if any element of the input satisfies the predicate.
1265−--
1266−-- Definition:
1267−--
1268−-- >>> any p = Fold.lmap p Fold.or
1269−--
1270−-- Example:
1271−--
1272−-- >>> Stream.fold (Fold.any (== 0)) $ Stream.fromList [1,0,1]
1273−-- True
1274−--
1275−{-# INLINE any #-}
1276−any :: Monad m => (a -> Bool) -> Fold m a Bool
1277−any predicate = foldt' step initial id
1278−
1279− where
1280−
1281− initial = Partial False
1282−
1283− step _ a =
1284− if predicate a
1285− then Done True
1286− else Partial False
1287−
1288−-- | Return 'True' if the given element is present in the stream.
1289−--
1290−-- Definition:
1291−--
1292−-- >>> elem a = Fold.any (== a)
1293−--
1294−{-# INLINE elem #-}
1295−elem :: (Eq a, Monad m) => a -> Fold m a Bool
1296−elem a = any (== a)
1297−
1298−-- | Returns 'True' if all elements of the input satisfy the predicate.
1299−--
1300−-- Definition:
1301−--
1302−-- >>> all p = Fold.lmap p Fold.and
1303−--
1304−-- Example:
1305−--
1306−-- >>> Stream.fold (Fold.all (== 0)) $ Stream.fromList [1,0,1]
1307−-- False
1308−--
1309−{-# INLINE all #-}
1310−all :: Monad m => (a -> Bool) -> Fold m a Bool
1311−all predicate = foldt' step initial id
1312−
1313− where
1314−
1315− initial = Partial True
1316−
1317− step _ a =
1318− if predicate a
1319− then Partial True
1320− else Done False
1321−
1322−-- | Returns 'True' if the given element is not present in the stream.
1323−--
1324−-- Definition:
1325−--
1326−-- >>> notElem a = Fold.all (/= a)
1327−--
1328−{-# INLINE notElem #-}
1329−notElem :: (Eq a, Monad m) => a -> Fold m a Bool
1330−notElem a = all (/= a)
1331−
1332−-- | Returns 'True' if all elements are 'True', 'False' otherwise
1333−--
1334−-- Definition:
1335−--
1336−-- >>> and = Fold.all (== True)
1337−--
1338−{-# INLINE and #-}
1339−and :: Monad m => Fold m Bool Bool
1340−and = all (== True)
1341−
1342−-- | Returns 'True' if any element is 'True', 'False' otherwise
1343−--
1344−-- Definition:
1345−--
1346−-- >>> or = Fold.any (== True)
1347−--
1348−{-# INLINE or #-}
1349−or :: Monad m => Fold m Bool Bool
1350−or = any (== True)
1351−
1352−------------------------------------------------------------------------------
1353−-- Grouping/Splitting
1354−------------------------------------------------------------------------------
1355−
1356−------------------------------------------------------------------------------
1357−-- Grouping without looking at elements
1358−------------------------------------------------------------------------------
1359−
1360−------------------------------------------------------------------------------
1361−-- Binary APIs
1362−------------------------------------------------------------------------------
1363−
1364−-- | @splitAt n f1 f2@ composes folds @f1@ and @f2@ such that first @n@
1365−-- elements of its input are consumed by fold @f1@ and the rest of the stream
1366−-- is consumed by fold @f2@.
1367−--
1368−-- >>> let splitAt_ n xs = Stream.fold (Fold.splitAt n Fold.toList Fold.toList) $ Stream.fromList xs
1369−--
1370−-- >>> splitAt_ 6 "Hello World!"
1371−-- ("Hello ","World!")
1372−--
1373−-- >>> splitAt_ (-1) [1,2,3]
1374−-- ([],[1,2,3])
1375−--
1376−-- >>> splitAt_ 0 [1,2,3]
1377−-- ([],[1,2,3])
1378−--
1379−-- >>> splitAt_ 1 [1,2,3]
1380−-- ([1],[2,3])
1381−--
1382−-- >>> splitAt_ 3 [1,2,3]
1383−-- ([1,2,3],[])
1384−--
1385−-- >>> splitAt_ 4 [1,2,3]
1386−-- ([1,2,3],[])
1387−--
1388−-- > splitAt n f1 f2 = Fold.splitWith (,) (Fold.take n f1) f2
1389−--
1390−-- /Internal/
1391−
1392−{-# INLINE splitAt #-}
1393−splitAt
1394− :: Monad m
1395− => Int
1396− -> Fold m a b
1397− -> Fold m a c
1398− -> Fold m a (b, c)
1399−splitAt n fld = splitWith (,) (take n fld)
1400−
1401−------------------------------------------------------------------------------
1402−-- Element Aware APIs
1403−------------------------------------------------------------------------------
1404−--
1405−------------------------------------------------------------------------------
1406−-- Binary APIs
1407−------------------------------------------------------------------------------
1408−
1409−{-# INLINE takingEndByM #-}
1410−takingEndByM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1411−takingEndByM p = Fold step initial extract extract
1412−
1413− where
1414−
1415− initial = return $ Partial Nothing'
1416−
1417− step _ a = do
1418− r <- p a
1419− return
1420− $ if r
1421− then Done $ Just a
1422− else Partial $ Just' a
1423−
1424− extract = return . toMaybe
1425−
1426−-- |
1427−--
1428−-- >>> takingEndBy p = Fold.takingEndByM (return . p)
1429−--
1430−{-# INLINE takingEndBy #-}
1431−takingEndBy :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1432−takingEndBy p = takingEndByM (return . p)
1433−
1434−{-# INLINE takingEndByM_ #-}
1435−takingEndByM_ :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1436−takingEndByM_ p = Fold step initial extract extract
1437−
1438− where
1439−
1440− initial = return $ Partial Nothing'
1441−
1442− step _ a = do
1443− r <- p a
1444− return
1445− $ if r
1446− then Done Nothing
1447− else Partial $ Just' a
1448−
1449− extract = return . toMaybe
1450−
1451−-- |
1452−--
1453−-- >>> takingEndBy_ p = Fold.takingEndByM_ (return . p)
1454−--
1455−{-# INLINE takingEndBy_ #-}
1456−takingEndBy_ :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1457−takingEndBy_ p = takingEndByM_ (return . p)
1458−
1459−{-# INLINE droppingWhileM #-}
1460−droppingWhileM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
1461−droppingWhileM p = Fold step initial extract extract
1462−
1463− where
1464−
1465− initial = return $ Partial Nothing'
1466−
1467− step Nothing' a = do
1468− r <- p a
1469− return
1470− $ Partial
1471− $ if r
1472− then Nothing'
1473− else Just' a
1474− step _ a = return $ Partial $ Just' a
1475−
1476− extract = return . toMaybe
1477−
1478−-- |
1479−-- >>> droppingWhile p = Fold.droppingWhileM (return . p)
1480−--
1481−{-# INLINE droppingWhile #-}
1482−droppingWhile :: Monad m => (a -> Bool) -> Fold m a (Maybe a)
1483−droppingWhile p = droppingWhileM (return . p)
1484−
1485−------------------------------------------------------------------------------
1486−-- Binary splitting on a separator
1487−------------------------------------------------------------------------------
1488−
1489−data SplitOnSeqState acc a rb rh w ck =
1490− SplitOnSeqEmpty !acc
1491− | SplitOnSeqSingle !acc !a
1492− | SplitOnSeqWord !acc !Int !w
1493− | SplitOnSeqWordLoop !acc !w
1494− | SplitOnSeqKR !acc !Int !rb !rh
1495− | SplitOnSeqKRLoop !acc !ck !rb !rh
1496−
1497−-- XXX Need to add tests for takeEndBySeq, we have tests for takeEndBySeq_ .
1498−
1499−-- | Continue taking the input until the input sequence matches the supplied
1500−-- sequence, taking the supplied sequence as well. If the pattern is empty this
1501−-- acts as an identity fold.
1502−--
1503−-- >>> s = Stream.fromList "hello there. How are you?"
1504−-- >>> f = Fold.takeEndBySeq (Array.fromList "re") Fold.toList
1505−-- >>> Stream.fold f s
1506−-- "hello there"
1507−--
1508−-- >>> Stream.fold Fold.toList $ Stream.foldMany f s
1509−-- ["hello there",". How are"," you?"]
1510−--
1511−-- /Pre-release/
1512−{-# INLINE takeEndBySeq #-}
1513−takeEndBySeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
1514− Array.Array a
1515− -> Fold m a b
1516− -> Fold m a b
1517−takeEndBySeq patArr (Fold fstep finitial fextract ffinal) =
1518− Fold step initial extract final
1519−
1520− where
1521−
1522− patLen = Array.length patArr
1523−
1524− initial = do
1525− res <- finitial
1526− case res of
1527− Partial acc
1528− | patLen == 0 ->
1529− -- XXX Should we match nothing or everything on empty
1530− -- pattern?
1531− -- Done <$> ffinal acc
1532− return $ Partial $ SplitOnSeqEmpty acc
1533− | patLen == 1 -> do
1534− pat <- liftIO $ Array.unsafeIndexIO 0 patArr
1535− return $ Partial $ SplitOnSeqSingle acc pat
1536− | SIZE_OF(a) * patLen <= sizeOf (Proxy :: Proxy Word) ->
1537− return $ Partial $ SplitOnSeqWord acc 0 0
1538− | otherwise -> do
1539− (rb, rhead) <- liftIO $ Ring.new patLen
1540− return $ Partial $ SplitOnSeqKR acc 0 rb rhead
1541− Done b -> return $ Done b
1542−
1543− -- Word pattern related
1544− maxIndex = patLen - 1
1545−
1546− elemBits = SIZE_OF(a) * 8
1547−
1548− wordMask :: Word
1549− wordMask = (1 `shiftL` (elemBits * patLen)) - 1
1550−
1551− wordPat :: Word
1552− wordPat = wordMask .&. Array.foldl' addToWord 0 patArr
1553−
1554− addToWord wd a = (wd `shiftL` elemBits) .|. fromIntegral (fromEnum a)
1555−
1556− -- For Rabin-Karp search
1557− k = 2891336453 :: Word32
1558− coeff = k ^ patLen
1559−
1560− addCksum cksum a = cksum * k + fromIntegral (fromEnum a)
1561−
1562− deltaCksum cksum old new =
1563− addCksum cksum new - coeff * fromIntegral (fromEnum old)
1564−
1565− -- XXX shall we use a random starting hash or 1 instead of 0?
1566− -- XXX Need to keep this cached across fold calls in foldmany
1567− -- XXX We may need refold to inject the cached state instead of
1568− -- initializing the state every time.
1569− -- XXX Allocation of ring buffer should also be done once
1570− patHash = Array.foldl' addCksum 0 patArr
1571−
1572− step (SplitOnSeqEmpty s) x = do
1573− res <- fstep s x
1574− case res of
1575− Partial s1 -> return $ Partial $ SplitOnSeqEmpty s1
1576− Done b -> return $ Done b
1577− step (SplitOnSeqSingle s pat) x = do
1578− res <- fstep s x
1579− case res of
1580− Partial s1
1581− | pat /= x -> return $ Partial $ SplitOnSeqSingle s1 pat
1582− | otherwise -> Done <$> ffinal s1
1583− Done b -> return $ Done b
1584− step (SplitOnSeqWord s idx wrd) x = do
1585− res <- fstep s x
1586− let wrd1 = addToWord wrd x
1587− case res of
1588− Partial s1
1589− | idx == maxIndex -> do
1590− if wrd1 .&. wordMask == wordPat
1591− then Done <$> ffinal s1
1592− else return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1593− | otherwise ->
1594− return $ Partial $ SplitOnSeqWord s1 (idx + 1) wrd1
1595− Done b -> return $ Done b
1596− step (SplitOnSeqWordLoop s wrd) x = do
1597− res <- fstep s x
1598− let wrd1 = addToWord wrd x
1599− case res of
1600− Partial s1
1601− | wrd1 .&. wordMask == wordPat ->
1602− Done <$> ffinal s1
1603− | otherwise ->
1604− return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1605− Done b -> return $ Done b
1606− step (SplitOnSeqKR s idx rb rh) x = do
1607− res <- fstep s x
1608− case res of
1609− Partial s1 -> do
1610− rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1611− if idx == maxIndex
1612− then do
1613− let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
1614− let !ringHash = fld addCksum 0 rb
1615− if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1616− then Done <$> ffinal s1
1617− else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1618− else
1619− return $ Partial $ SplitOnSeqKR s1 (idx + 1) rb rh1
1620− Done b -> return $ Done b
1621− step (SplitOnSeqKRLoop s cksum rb rh) x = do
1622− res <- fstep s x
1623− case res of
1624− Partial s1 -> do
1625− old <- liftIO $ peek rh
1626− rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1627− let ringHash = deltaCksum cksum old x
1628− if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1629− then Done <$> ffinal s1
1630− else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1631− Done b -> return $ Done b
1632−
1633− extractFunc fex state =
1634− let st =
1635− case state of
1636− SplitOnSeqEmpty s -> s
1637− SplitOnSeqSingle s _ -> s
1638− SplitOnSeqWord s _ _ -> s
1639− SplitOnSeqWordLoop s _ -> s
1640− SplitOnSeqKR s _ _ _ -> s
1641− SplitOnSeqKRLoop s _ _ _ -> s
1642− in fex st
1643−
1644− extract state = extractFunc fextract state
1645−
1646− final state = extractFunc ffinal state
1647−
1648−-- | Like 'takeEndBySeq' but discards the matched sequence.
1649−--
1650−-- /Pre-release/
1651−--
1652−{-# INLINE takeEndBySeq_ #-}
1653−takeEndBySeq_ :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) =>
1654− Array.Array a
1655− -> Fold m a b
1656− -> Fold m a b
1657−takeEndBySeq_ patArr (Fold fstep finitial fextract ffinal) =
1658− Fold step initial extract final
1659−
1660− where
1661−
1662− patLen = Array.length patArr
1663−
1664− initial = do
1665− res <- finitial
1666− case res of
1667− Partial acc
1668− | patLen == 0 ->
1669− -- XXX Should we match nothing or everything on empty
1670− -- pattern?
1671− -- Done <$> ffinal acc
1672− return $ Partial $ SplitOnSeqEmpty acc
1673− | patLen == 1 -> do
1674− pat <- liftIO $ Array.unsafeIndexIO 0 patArr
1675− return $ Partial $ SplitOnSeqSingle acc pat
1676− -- XXX Need to add tests for this case
1677− | SIZE_OF(a) * patLen <= sizeOf (Proxy :: Proxy Word) ->
1678− return $ Partial $ SplitOnSeqWord acc 0 0
1679− | otherwise -> do
1680− (rb, rhead) <- liftIO $ Ring.new patLen
1681− return $ Partial $ SplitOnSeqKR acc 0 rb rhead
1682− Done b -> return $ Done b
1683−
1684− -- Word pattern related
1685− maxIndex = patLen - 1
1686−
1687− elemBits = SIZE_OF(a) * 8
1688−
1689− wordMask :: Word
1690− wordMask = (1 `shiftL` (elemBits * patLen)) - 1
1691−
1692− elemMask :: Word
1693− elemMask = (1 `shiftL` elemBits) - 1
1694−
1695− wordPat :: Word
1696− wordPat = wordMask .&. Array.foldl' addToWord 0 patArr
1697−
1698− addToWord wd a = (wd `shiftL` elemBits) .|. fromIntegral (fromEnum a)
1699−
1700− -- For Rabin-Karp search
1701− k = 2891336453 :: Word32
1702− coeff = k ^ patLen
1703−
1704− addCksum cksum a = cksum * k + fromIntegral (fromEnum a)
1705−
1706− deltaCksum cksum old new =
1707− addCksum cksum new - coeff * fromIntegral (fromEnum old)
1708−
1709− -- XXX shall we use a random starting hash or 1 instead of 0?
1710− -- XXX Need to keep this cached across fold calls in foldMany
1711− -- XXX We may need refold to inject the cached state instead of
1712− -- initializing the state every time.
1713− -- XXX Allocation of ring buffer should also be done once
1714− patHash = Array.foldl' addCksum 0 patArr
1715−
1716− step (SplitOnSeqEmpty s) x = do
1717− res <- fstep s x
1718− case res of
1719− Partial s1 -> return $ Partial $ SplitOnSeqEmpty s1
1720− Done b -> return $ Done b
1721− step (SplitOnSeqSingle s pat) x = do
1722− if pat /= x
1723− then do
1724− res <- fstep s x
1725− case res of
1726− Partial s1 -> return $ Partial $ SplitOnSeqSingle s1 pat
1727− Done b -> return $ Done b
1728− else Done <$> ffinal s
1729− step (SplitOnSeqWord s idx wrd) x = do
1730− let wrd1 = addToWord wrd x
1731− if idx == maxIndex
1732− then do
1733− if wrd1 .&. wordMask == wordPat
1734− then Done <$> ffinal s
1735− else return $ Partial $ SplitOnSeqWordLoop s wrd1
1736− else return $ Partial $ SplitOnSeqWord s (idx + 1) wrd1
1737− step (SplitOnSeqWordLoop s wrd) x = do
1738− let wrd1 = addToWord wrd x
1739− old = (wordMask .&. wrd)
1740− `shiftR` (elemBits * (patLen - 1))
1741− res <- fstep s (toEnum $ fromIntegral old)
1742− case res of
1743− Partial s1
1744− | wrd1 .&. wordMask == wordPat ->
1745− Done <$> ffinal s1
1746− | otherwise ->
1747− return $ Partial $ SplitOnSeqWordLoop s1 wrd1
1748− Done b -> return $ Done b
1749− step (SplitOnSeqKR s idx rb rh) x = do
1750− rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1751− if idx == maxIndex
1752− then do
1753− let fld = Ring.unsafeFoldRing (Ring.ringBound rb)
1754− let !ringHash = fld addCksum 0 rb
1755− if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1756− then Done <$> ffinal s
1757− else return $ Partial $ SplitOnSeqKRLoop s ringHash rb rh1
1758− else return $ Partial $ SplitOnSeqKR s (idx + 1) rb rh1
1759− step (SplitOnSeqKRLoop s cksum rb rh) x = do
1760− old <- liftIO $ peek rh
1761− res <- fstep s old
1762− case res of
1763− Partial s1 -> do
1764− rh1 <- liftIO $ Ring.unsafeInsert rb rh x
1765− let ringHash = deltaCksum cksum old x
1766− if ringHash == patHash && Ring.unsafeEqArray rb rh1 patArr
1767− then Done <$> ffinal s1
1768− else return $ Partial $ SplitOnSeqKRLoop s1 ringHash rb rh1
1769− Done b -> return $ Done b
1770−
1771− -- XXX extract should return backtrack count as well. If the fold
1772− -- terminates early inside extract, we may still have buffered data
1773− -- remaining which will be lost if we do not communicate that to the
1774− -- driver.
1775− extractFunc fex state = do
1776− let consumeWord s n wrd = do
1777− if n == 0
1778− then fex s
1779− else do
1780− let old = elemMask .&. (wrd `shiftR` (elemBits * (n - 1)))
1781− r <- fstep s (toEnum $ fromIntegral old)
1782− case r of
1783− Partial s1 -> consumeWord s1 (n - 1) wrd
1784− Done b -> return b
1785−
1786− let consumeRing s n rb rh =
1787− if n == 0
1788− then fex s
1789− else do
1790− old <- liftIO $ peek rh
1791− let rh1 = Ring.advance rb rh
1792− r <- fstep s old
1793− case r of
1794− Partial s1 -> consumeRing s1 (n - 1) rb rh1
1795− Done b -> return b
1796−
1797− case state of
1798− SplitOnSeqEmpty s -> fex s
1799− SplitOnSeqSingle s _ -> fex s
1800− SplitOnSeqWord s idx wrd -> consumeWord s idx wrd
1801− SplitOnSeqWordLoop s wrd -> consumeWord s patLen wrd
1802− SplitOnSeqKR s idx rb _ -> consumeRing s idx rb (Ring.startOf rb)
1803− SplitOnSeqKRLoop s _ rb rh -> consumeRing s patLen rb rh
1804−
1805− extract state = extractFunc fextract state
1806−
1807− final state = extractFunc ffinal state
1808−
1809−------------------------------------------------------------------------------
1810−-- Distributing
1811−------------------------------------------------------------------------------
1812−--
1813−-- | Distribute one copy of the stream to each fold and zip the results.
1814−--
1815−-- @
1816−-- |-------Fold m a b--------|
1817−-- ---stream m a---| |---m (b,c)
1818−-- |-------Fold m a c--------|
1819−-- @
1820−--
1821−-- Definition:
1822−--
1823−-- >>> tee = Fold.teeWith (,)
1824−--
1825−-- Example:
1826−--
1827−-- >>> t = Fold.tee Fold.sum Fold.length
1828−-- >>> Stream.fold t (Stream.enumerateFromTo 1.0 100.0)
1829−-- (5050.0,100)
1830−--
1831−{-# INLINE tee #-}
1832−tee :: Monad m => Fold m a b -> Fold m a c -> Fold m a (b,c)
1833−tee = teeWith (,)
1834−
1835−-- XXX use "List" instead of "[]"?, use Array for output to scale it to a large
1836−-- number of consumers? For polymorphic case a vector could be helpful. For
1837−-- Storables we can use arrays. Will need separate APIs for those.
1838−--
1839−-- | Distribute one copy of the stream to each fold and collect the results in
1840−-- a container.
1841−--
1842−-- @
1843−--
1844−-- |-------Fold m a b--------|
1845−-- ---stream m a---| |---m [b]
1846−-- |-------Fold m a b--------|
1847−-- | |
1848−-- ...
1849−-- @
1850−--
1851−-- >>> Stream.fold (Fold.distribute [Fold.sum, Fold.length]) (Stream.enumerateFromTo 1 5)
1852−-- [15,5]
1853−--
1854−-- >>> distribute = Prelude.foldr (Fold.teeWith (:)) (Fold.fromPure [])
1855−--
1856−-- This is the consumer side dual of the producer side 'sequence' operation.
1857−--
1858−-- Stops when all the folds stop.
1859−--
1860−{-# INLINE distribute #-}
1861−distribute :: Monad m => [Fold m a b] -> Fold m a [b]
1862−distribute = Prelude.foldr (teeWith (:)) (fromPure [])
1863−
1864−------------------------------------------------------------------------------
1865−-- Partitioning
1866−------------------------------------------------------------------------------
1867−
1868−{-# INLINE partitionByMUsing #-}
1869−partitionByMUsing :: Monad m =>
1870− ( (x -> y -> (x, y))
1871− -> Fold m (Either b c) x
1872− -> Fold m (Either b c) y
1873− -> Fold m (Either b c) (x, y)
1874− )
1875− -> (a -> m (Either b c))
1876− -> Fold m b x
1877− -> Fold m c y
1878− -> Fold m a (x, y)
1879−partitionByMUsing t f fld1 fld2 =
1880− let l = lmap (fromLeft undefined) fld1 -- :: Fold m (Either b c) x
1881− r = lmap (fromRight undefined) fld2 -- :: Fold m (Either b c) y
1882− in lmapM f (t (,) (filter isLeft l) (filter isRight r))
1883−
1884−-- | Partition the input over two folds using an 'Either' partitioning
1885−-- predicate.
1886−--
1887−-- @
1888−--
1889−-- |-------Fold b x--------|
1890−-- -----stream m a --> (Either b c)----| |----(x,y)
1891−-- |-------Fold c y--------|
1892−-- @
1893−--
1894−-- Example, send input to either fold randomly:
1895−--
1896−-- >>> :set -package random
1897−-- >>> import System.Random (randomIO)
1898−-- >>> randomly a = randomIO >>= \x -> return $ if x then Left a else Right a
1899−-- >>> f = Fold.partitionByM randomly Fold.length Fold.length
1900−-- >>> Stream.fold f (Stream.enumerateFromTo 1 100)
1901−-- ...
1902−--
1903−-- Example, send input to the two folds in a proportion of 2:1:
1904−--
1905−-- >>> :{
1906−-- proportionately m n = do
1907−-- ref <- newIORef $ cycle $ concat [replicate m Left, replicate n Right]
1908−-- return $ \a -> do
1909−-- r <- readIORef ref
1910−-- writeIORef ref $ tail r
1911−-- return $ Prelude.head r a
1912−-- :}
1913−--
1914−-- >>> :{
1915−-- main = do
1916−-- g <- proportionately 2 1
1917−-- let f = Fold.partitionByM g Fold.length Fold.length
1918−-- r <- Stream.fold f (Stream.enumerateFromTo (1 :: Int) 100)
1919−-- print r
1920−-- :}
1921−--
1922−-- >>> main
1923−-- (67,33)
1924−--
1925−--
1926−-- This is the consumer side dual of the producer side 'mergeBy' operation.
1927−--
1928−-- When one fold is done, any input meant for it is ignored until the other
1929−-- fold is also done.
1930−--
1931−-- Stops when both the folds stop.
1932−--
1933−-- /See also: 'partitionByFstM' and 'partitionByMinM'./
1934−--
1935−-- /Pre-release/
1936−{-# INLINE partitionByM #-}
1937−partitionByM :: Monad m
1938− => (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
1939−partitionByM = partitionByMUsing teeWith
1940−
1941−-- | Similar to 'partitionByM' but terminates when the first fold terminates.
1942−--
1943−{-# INLINE partitionByFstM #-}
1944−partitionByFstM :: Monad m
1945− => (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
1946−partitionByFstM = partitionByMUsing teeWithFst
1947−
1948−-- | Similar to 'partitionByM' but terminates when any fold terminates.
1949−--
1950−{-# INLINE partitionByMinM #-}
1951−partitionByMinM :: Monad m =>
1952− (a -> m (Either b c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
1953−partitionByMinM = partitionByMUsing teeWithMin
1954−
1955−-- Note: we could use (a -> Bool) instead of (a -> Either b c), but the latter
1956−-- makes the signature clearer as to which case belongs to which fold.
1957−-- XXX need to check the performance in both cases.
1958−
1959−-- | Same as 'partitionByM' but with a pure partition function.
1960−--
1961−-- Example, count even and odd numbers in a stream:
1962−--
1963−-- >>> :{
1964−-- let f = Fold.partitionBy (\n -> if even n then Left n else Right n)
1965−-- (fmap (("Even " ++) . show) Fold.length)
1966−-- (fmap (("Odd " ++) . show) Fold.length)
1967−-- in Stream.fold f (Stream.enumerateFromTo 1 100)
1968−-- :}
1969−-- ("Even 50","Odd 50")
1970−--
1971−-- /Pre-release/
1972−{-# INLINE partitionBy #-}
1973−partitionBy :: Monad m
1974− => (a -> Either b c) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
1975−partitionBy f = partitionByM (return . f)
1976−
1977−-- | Compose two folds such that the combined fold accepts a stream of 'Either'
1978−-- and routes the 'Left' values to the first fold and 'Right' values to the
1979−-- second fold.
1980−--
1981−-- Definition:
1982−--
1983−-- >>> partition = Fold.partitionBy id
1984−--
1985−{-# INLINE partition #-}
1986−partition :: Monad m
1987− => Fold m b x -> Fold m c y -> Fold m (Either b c) (x, y)
1988−partition = partitionBy id
1989−
1990−{-
1991−-- | Send one item to each fold in a round-robin fashion. This is the consumer
1992−-- side dual of producer side 'mergeN' operation.
1993−--
1994−-- partitionN :: Monad m => [Fold m a b] -> Fold m a [b]
1995−-- partitionN fs = Fold step begin done
1996−-}
1997−
1998−------------------------------------------------------------------------------
1999−-- Unzipping
2000−------------------------------------------------------------------------------
2001−
2002−{-# INLINE unzipWithMUsing #-}
2003−unzipWithMUsing :: Monad m =>
2004− ( (x -> y -> (x, y))
2005− -> Fold m (b, c) x
2006− -> Fold m (b, c) y
2007− -> Fold m (b, c) (x, y)
2008− )
2009− -> (a -> m (b, c))
2010− -> Fold m b x
2011− -> Fold m c y
2012− -> Fold m a (x, y)
2013−unzipWithMUsing t f fld1 fld2 =
2014− let f1 = lmap fst fld1 -- :: Fold m (b, c) b
2015− f2 = lmap snd fld2 -- :: Fold m (b, c) c
2016− in lmapM f (t (,) f1 f2)
2017−
2018−-- | Like 'unzipWith' but with a monadic splitter function.
2019−--
2020−-- Definition:
2021−--
2022−-- >>> unzipWithM k f1 f2 = Fold.lmapM k (Fold.unzip f1 f2)
2023−--
2024−-- /Pre-release/
2025−{-# INLINE unzipWithM #-}
2026−unzipWithM :: Monad m
2027− => (a -> m (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2028−unzipWithM = unzipWithMUsing teeWith
2029−
2030−-- | Similar to 'unzipWithM' but terminates when the first fold terminates.
2031−--
2032−{-# INLINE unzipWithFstM #-}
2033−unzipWithFstM :: Monad m =>
2034− (a -> m (b, c)) -> Fold m b x -> Fold m c y -> Fold m a (x, y)
2035−unzipWithFstM = unzipWithMUsing teeWithFst
2036−
2037−-- | Similar to 'unzipWithM' but terminates when any fold terminates.
2038−--
2039−{-# INLINE unzipWithMinM #-}
2040−unzipWithMinM :: Monad m =>
2041− (a -> m (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2042−unzipWithMinM = unzipWithMUsing teeWithMin
2043−
2044−-- | Split elements in the input stream into two parts using a pure splitter
2045−-- function, direct each part to a different fold and zip the results.
2046−--
2047−-- Definitions:
2048−--
2049−-- >>> unzipWith f = Fold.unzipWithM (return . f)
2050−-- >>> unzipWith f fld1 fld2 = Fold.lmap f (Fold.unzip fld1 fld2)
2051−--
2052−-- This fold terminates when both the input folds terminate.
2053−--
2054−-- /Pre-release/
2055−{-# INLINE unzipWith #-}
2056−unzipWith :: Monad m
2057− => (a -> (b,c)) -> Fold m b x -> Fold m c y -> Fold m a (x,y)
2058−unzipWith f = unzipWithM (return . f)
2059−
2060−-- | Send the elements of tuples in a stream of tuples through two different
2061−-- folds.
2062−--
2063−-- @
2064−--
2065−-- |-------Fold m a x--------|
2066−-- ---------stream of (a,b)--| |----m (x,y)
2067−-- |-------Fold m b y--------|
2068−--
2069−-- @
2070−--
2071−-- Definition:
2072−--
2073−-- >>> unzip = Fold.unzipWith id
2074−--
2075−-- This is the consumer side dual of the producer side 'zip' operation.
2076−--
2077−{-# INLINE unzip #-}
2078−unzip :: Monad m => Fold m a x -> Fold m b y -> Fold m (a,b) (x,y)
2079−unzip = unzipWith id
2080−
2081−------------------------------------------------------------------------------
2082−-- Combining streams and folds - Zipping
2083−------------------------------------------------------------------------------
2084−
2085−-- XXX These can be implemented using the fold scan, using the stream as a
2086−-- state.
2087−-- XXX Stream Skip state cannot be efficiently handled in folds but can be
2088−-- handled in parsers using the Continue facility. See zipWithM in the Parser
2089−-- module.
2090−--
2091−-- cmpBy, eqBy, isPrefixOf, isSubsequenceOf etc can be implemented using
2092−-- zipStream.
2093−
2094−-- | Zip a stream with the input of a fold using the supplied function.
2095−--
2096−-- /Unimplemented/
2097−--
2098−{-# INLINE zipStreamWithM #-}
2099−zipStreamWithM :: -- Monad m =>
2100− (a -> b -> m c) -> Stream m a -> Fold m c x -> Fold m b x
2101−zipStreamWithM = undefined
2102−
2103−-- | Zip a stream with the input of a fold.
2104−--
2105−-- >>> zip = Fold.zipStreamWithM (curry return)
2106−--
2107−-- /Unimplemented/
2108−--
2109−{-# INLINE zipStream #-}
2110−zipStream :: Monad m => Stream m a -> Fold m (a, b) x -> Fold m b x
2111−zipStream = zipStreamWithM (curry return)
2112−
2113−-- | Pair each element of a fold input with its index, starting from index 0.
2114−--
2115−{-# INLINE indexingWith #-}
2116−indexingWith :: Monad m => Int -> (Int -> Int) -> Fold m a (Maybe (Int, a))
2117−indexingWith i f = fmap toMaybe $ foldl' step initial
2118−
2119− where
2120−
2121− initial = Nothing'
2122−
2123− step Nothing' a = Just' (i, a)
2124− step (Just' (n, _)) a = Just' (f n, a)
2125−
2126−-- |
2127−-- >>> indexing = Fold.indexingWith 0 (+ 1)
2128−--
2129−{-# INLINE indexing #-}
2130−indexing :: Monad m => Fold m a (Maybe (Int, a))
2131−indexing = indexingWith 0 (+ 1)
2132−
2133−-- |
2134−-- >>> indexingRev n = Fold.indexingWith n (subtract 1)
2135−--
2136−{-# INLINE indexingRev #-}
2137−indexingRev :: Monad m => Int -> Fold m a (Maybe (Int, a))
2138−indexingRev n = indexingWith n (subtract 1)
2139−
2140−-- | Pair each element of a fold input with its index, starting from index 0.
2141−--
2142−-- >>> indexed = Fold.scanMaybe Fold.indexing
2143−--
2144−{-# INLINE indexed #-}
2145−indexed :: Monad m => Fold m (Int, a) b -> Fold m a b
2146−indexed = scanMaybe indexing
2147−
2148−-- | Change the predicate function of a Fold from @a -> b@ to accept an
2149−-- additional state input @(s, a) -> b@. Convenient to filter with an
2150−-- addiitonal index or time input.
2151−--
2152−-- >>> filterWithIndex = Fold.with Fold.indexed Fold.filter
2153−--
2154−-- @
2155−-- filterWithAbsTime = with timestamped filter
2156−-- filterWithRelTime = with timeIndexed filter
2157−-- @
2158−--
2159−-- /Pre-release/
2160−{-# INLINE with #-}
2161−with ::
2162− (Fold m (s, a) b -> Fold m a b)
2163− -> (((s, a) -> c) -> Fold m (s, a) b -> Fold m (s, a) b)
2164− -> (((s, a) -> c) -> Fold m a b -> Fold m a b)
2165−with f comb g = f . comb g . lmap snd
2166−
2167−-- XXX Implement as a filter
2168−-- sampleFromthen :: Monad m => Int -> Int -> Fold m a (Maybe a)
2169−
2170−-- | @sampleFromthen offset stride@ samples the element at @offset@ index and
2171−-- then every element at strides of @stride@.
2172−--
2173−{-# INLINE sampleFromthen #-}
2174−sampleFromthen :: Monad m => Int -> Int -> Fold m a b -> Fold m a b
2175−sampleFromthen offset size =
2176− with indexed filter (\(i, _) -> (i + offset) `mod` size == 0)
2177−
2178−------------------------------------------------------------------------------
2179−-- Nesting
2180−------------------------------------------------------------------------------
2181−
2182−-- | @concatSequence f t@ applies folds from stream @t@ sequentially and
2183−-- collects the results using the fold @f@.
2184−--
2185−-- /Unimplemented/
2186−--
2187−{-# INLINE concatSequence #-}
2188−concatSequence ::
2189− -- IsStream t =>
2190− Fold m b c -> t (Fold m a b) -> Fold m a c
2191−concatSequence _f _p = undefined
2192−
2193−-- | Group the input stream into groups of elements between @low@ and @high@.
2194−-- Collection starts in chunks of @low@ and then keeps doubling until we reach
2195−-- @high@. Each chunk is folded using the provided fold function.
2196−--
2197−-- This could be useful, for example, when we are folding a stream of unknown
2198−-- size to a stream of arrays and we want to minimize the number of
2199−-- allocations.
2200−--
2201−-- NOTE: this would be an application of "many" using a terminating fold.
2202−--
2203−-- /Unimplemented/
2204−--
2205−{-# INLINE chunksBetween #-}
2206−chunksBetween :: -- Monad m =>
2207− Int -> Int -> Fold m a b -> Fold m b c -> Fold m a c
2208−chunksBetween _low _high _f1 _f2 = undefined
2209−
2210−-- | A fold that buffers its input to a pure stream.
2211−--
2212−-- /Warning!/ working on large streams accumulated as buffers in memory could
2213−-- be very inefficient, consider using "Streamly.Data.Array" instead.
2214−--
2215−-- >>> toStream = fmap Stream.fromList Fold.toList
2216−--
2217−-- /Pre-release/
2218−{-# INLINE toStream #-}
2219−toStream :: (Monad m, Monad n) => Fold m a (Stream n a)
2220−toStream = fmap StreamD.fromList toList
2221−
2222−-- This is more efficient than 'toStream'. toStream is exactly the same as
2223−-- reversing the stream after toStreamRev.
2224−--
2225−-- | Buffers the input stream to a pure stream in the reverse order of the
2226−-- input.
2227−--
2228−-- >>> toStreamRev = fmap Stream.fromList Fold.toListRev
2229−--
2230−-- /Warning!/ working on large streams accumulated as buffers in memory could
2231−-- be very inefficient, consider using "Streamly.Data.Array" instead.
2232−--
2233−-- /Pre-release/
2234−
2235−-- xn : ... : x2 : x1 : []
2236−{-# INLINE toStreamRev #-}
2237−toStreamRev :: (Monad m, Monad n) => Fold m a (Stream n a)
2238−toStreamRev = fmap StreamD.fromList toListRev
2239−
2240−-- XXX This does not fuse. It contains a recursive step function. We will need
2241−-- a Skip input constructor in the fold type to make it fuse.
2242−--
2243−-- | Unfold and flatten the input stream of a fold.
2244−--
2245−-- @
2246−-- Stream.fold (unfoldMany u f) = Stream.fold f . Stream.unfoldMany u
2247−-- @
2248−--
2249−-- /Pre-release/
2250−{-# INLINE unfoldMany #-}
2251−unfoldMany :: Monad m => Unfold m a b -> Fold m b c -> Fold m a c
2252−unfoldMany (Unfold ustep inject) (Fold fstep initial extract final) =
2253− Fold consume initial extract final
2254−
2255− where
2256−
2257− {-# INLINE produce #-}
2258− produce fs us = do
2259− ures <- ustep us
2260− case ures of
2261− StreamD.Yield b us1 -> do
2262− fres <- fstep fs b
2263− case fres of
2264− Partial fs1 -> produce fs1 us1
2265− -- XXX What to do with the remaining stream?
2266− Done c -> return $ Done c
2267− StreamD.Skip us1 -> produce fs us1
2268− StreamD.Stop -> return $ Partial fs
2269−
2270− {-# INLINE_LATE consume #-}
2271− consume s a = inject a >>= produce s
2272−
2273−-- | Get the bottom most @n@ elements using the supplied comparison function.
2274−--
2275−{-# INLINE bottomBy #-}
2276−bottomBy :: (MonadIO m, Unbox a) =>
2277− (a -> a -> Ordering)
2278− -> Int
2279− -> Fold m a (MutArray a)
2280−bottomBy cmp n = Fold step initial extract extract
2281−
2282− where
2283−
2284− initial = do
2285− arr <- MA.pinnedNew n
2286− if n <= 0
2287− then return $ Done arr
2288− else return $ Partial (arr, 0)
2289−
2290− step (arr, i) x =
2291− if i < n
2292− then do
2293− arr' <- MA.snoc arr x
2294− MA.bubble cmp arr'
2295− return $ Partial (arr', i + 1)
2296− else do
2297− x1 <- MA.getIndexUnsafe (i - 1) arr
2298− case x `cmp` x1 of
2299− LT -> do
2300− MA.putIndexUnsafe (i - 1) arr x
2301− MA.bubble cmp arr
2302− return $ Partial (arr, i)
2303− _ -> return $ Partial (arr, i)
2304−
2305− extract = return . fst
2306−
2307−-- | Get the top @n@ elements using the supplied comparison function.
2308−--
2309−-- To get bottom n elements instead:
2310−--
2311−-- >>> bottomBy cmp = Fold.topBy (flip cmp)
2312−--
2313−-- Example:
2314−--
2315−-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2316−-- >>> Stream.fold (Fold.topBy compare 3) stream >>= MutArray.toList
2317−-- [17,11,9]
2318−--
2319−-- /Pre-release/
2320−--
2321−{-# INLINE topBy #-}
2322−topBy :: (MonadIO m, Unbox a) =>
2323− (a -> a -> Ordering)
2324− -> Int
2325− -> Fold m a (MutArray a)
2326−topBy cmp = bottomBy (flip cmp)
2327−
2328−-- | Fold the input stream to top n elements.
2329−--
2330−-- Definition:
2331−--
2332−-- >>> top = Fold.topBy compare
2333−--
2334−-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2335−-- >>> Stream.fold (Fold.top 3) stream >>= MutArray.toList
2336−-- [17,11,9]
2337−--
2338−-- /Pre-release/
2339−{-# INLINE top #-}
2340−top :: (MonadIO m, Unbox a, Ord a) => Int -> Fold m a (MutArray a)
2341−top = bottomBy $ flip compare
2342−
2343−-- | Fold the input stream to bottom n elements.
2344−--
2345−-- Definition:
2346−--
2347−-- >>> bottom = Fold.bottomBy compare
2348−--
2349−-- >>> stream = Stream.fromList [2::Int,7,9,3,1,5,6,11,17]
2350−-- >>> Stream.fold (Fold.bottom 3) stream >>= MutArray.toList
2351−-- [1,2,3]
2352−--
2353−-- /Pre-release/
2354−{-# INLINE bottom #-}
2355−bottom :: (MonadIO m, Unbox a, Ord a) => Int -> Fold m a (MutArray a)
2356−bottom = bottomBy compare
2357−
2358−------------------------------------------------------------------------------
2359−-- Interspersed parsing
2360−------------------------------------------------------------------------------
2361−
2362−data IntersperseQState fs ps =
2363− IntersperseQUnquoted !fs !ps
2364− | IntersperseQQuoted !fs !ps
2365− | IntersperseQQuotedEsc !fs !ps
2366−
2367−-- Useful for parsing CSV with quoting and escaping
2368−{-# INLINE intersperseWithQuotes #-}
2369−intersperseWithQuotes :: (Monad m, Eq a) =>
2370− a -> a -> a -> Fold m a b -> Fold m b c -> Fold m a c
2371−intersperseWithQuotes
2372− quote
2373− esc
2374− separator
2375− (Fold stepL initialL _ finalL)
2376− (Fold stepR initialR extractR finalR) = Fold step initial extract final
2377−
2378− where
2379−
2380− errMsg p status =
2381− error $ "intersperseWithQuotes: " ++ p ++ " parsing fold cannot "
2382− ++ status ++ " without input"
2383−
2384− {-# INLINE initL #-}
2385− initL mkState = do
2386− resL <- initialL
2387− case resL of
2388− Partial sL ->
2389− return $ Partial $ mkState sL
2390− Done _ ->
2391− errMsg "content" "succeed"
2392−
2393− initial = do
2394− res <- initialR
2395− case res of
2396− Partial sR -> initL (IntersperseQUnquoted sR)
2397− Done b -> return $ Done b
2398−
2399− {-# INLINE collect #-}
2400− collect nextS sR b = do
2401− res <- stepR sR b
2402− case res of
2403− Partial s ->
2404− initL (nextS s)
2405− Done c -> return (Done c)
2406−
2407− {-# INLINE process #-}
2408− process a sL sR nextState = do
2409− r <- stepL sL a
2410− case r of
2411− Partial s -> return $ Partial (nextState sR s)
2412− Done b -> collect nextState sR b
2413−
2414− {-# INLINE processQuoted #-}
2415− processQuoted a sL sR nextState = do
2416− r <- stepL sL a
2417− case r of
2418− Partial s -> return $ Partial (nextState sR s)
2419− Done _ -> do
2420− _ <- finalR sR
2421− error "Collecting fold finished inside quote"
2422−
2423− step (IntersperseQUnquoted sR sL) a
2424− | a == separator = do
2425− b <- finalL sL
2426− collect IntersperseQUnquoted sR b
2427− | a == quote = processQuoted a sL sR IntersperseQQuoted
2428− | otherwise = process a sL sR IntersperseQUnquoted
2429−
2430− step (IntersperseQQuoted sR sL) a
2431− | a == esc = processQuoted a sL sR IntersperseQQuotedEsc
2432− | a == quote = process a sL sR IntersperseQUnquoted
2433− | otherwise = processQuoted a sL sR IntersperseQQuoted
2434−
2435− step (IntersperseQQuotedEsc sR sL) a =
2436− processQuoted a sL sR IntersperseQQuoted
2437−
2438− extract (IntersperseQUnquoted sR _) = extractR sR
2439− extract (IntersperseQQuoted _ _) =
2440− error "intersperseWithQuotes: finished inside quote"
2441− extract (IntersperseQQuotedEsc _ _) =
2442− error "intersperseWithQuotes: finished inside quote, at escape char"
2443−
2444− final (IntersperseQUnquoted sR sL) = finalL sL *> finalR sR
2445− final (IntersperseQQuoted sR sL) = do
2446− _ <- finalR sR
2447− _ <- finalL sL
2448− error "intersperseWithQuotes: finished inside quote"
2449− final (IntersperseQQuotedEsc sR sL) = do
2450− _ <- finalR sR
2451− _ <- finalL sL
2452− error "intersperseWithQuotes: finished inside quote, at escape char"
core/src/Streamly/Internal/Data/Fold/Container.hs+32−75
@@ -9,6 +9,9 @@
99
1010module Streamly.Internal.Data.Fold.Container
1111 (
12+ -- * Imports
13+ -- $setup
14+
1215 -- * Set operations
1316 toSet
1417 , toIntSet
@@ -100,8 +103,7 @@ import qualified Data.Set as Set
100103import qualified Streamly.Internal.Data.IsMap as IsMap
101104
102105import Prelude hiding (Foldable(..))
103−import Streamly.Internal.Data.Fold.Type
104−import Streamly.Internal.Data.Fold.Combinators
106+import Streamly.Internal.Data.Fold
105107
106108-- $setup
107109-- >>> :m
@@ -270,15 +272,14 @@ demuxGeneric :: (Monad m, IsMap f, Traversable f) =>
270272 (a -> Key f)
271273 -> (a -> m (Fold m a b))
272274 -> Fold m a (m (f b), Maybe (Key f, b))
273−demuxGeneric getKey getFold =
274− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
275+demuxGeneric getKey getFold = fmap extract $ foldlM' step initial
275276
276277 where
277278
278279 initial = return $ Tuple' IsMap.mapEmpty Nothing
279280
280281 {-# INLINE runFold #-}
281− runFold kv (Fold step1 initial1 extract1 final1) (k, a) = do
282+ runFold kv (Fold step1 initial1 extract1) (k, a) = do
282283 res <- initial1
283284 case res of
284285 Partial s -> do
@@ -286,7 +287,7 @@ demuxGeneric getKey getFold =
286287 return
287288 $ case res1 of
288289 Partial _ ->
289− let fld = Fold step1 (return res1) extract1 final1
290+ let fld = Fold step1 (return res1) extract1
290291 in Tuple' (IsMap.mapInsert k fld kv) Nothing
291292 Done b -> Tuple' (IsMap.mapDelete k kv) (Just (k, b))
292293 Done b -> return $ Tuple' kv (Just (k, b))
@@ -299,25 +300,15 @@ demuxGeneric getKey getFold =
299300 runFold kv fld (k, a)
300301 Just f -> runFold kv f (k, a)
301302
302− extract (Tuple' kv x) = return (Prelude.mapM f kv, x)
303+ extract (Tuple' kv x) = (Prelude.mapM f kv, x)
303304
304305 where
305306
306− f (Fold _ i e _) = do
307+ f (Fold _ i e) = do
307308 r <- i
308309 case r of
309310 Partial s -> e s
310− _ -> error "demuxGeneric: unreachable code"
311−
312− final (Tuple' kv x) = return (Prelude.mapM f kv, x)
313−
314− where
315−
316− f (Fold _ i _ fin) = do
317− r <- i
318− case r of
319− Partial s -> fin s
320− _ -> error "demuxGeneric: unreachable code"
311+ Done b -> return b
321312
322313-- | @demux getKey getFold@: In a key value stream, fold values corresponding
323314-- to each key using a key specific fold. @getFold@ is invoked to generate a
@@ -359,36 +350,35 @@ demuxGenericIO :: (MonadIO m, IsMap f, Traversable f) =>
359350 (a -> Key f)
360351 -> (a -> m (Fold m a b))
361352 -> Fold m a (m (f b), Maybe (Key f, b))
362−demuxGenericIO getKey getFold =
363− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
353+demuxGenericIO getKey getFold = fmap extract $ foldlM' step initial
364354
365355 where
366356
367357 initial = return $ Tuple' IsMap.mapEmpty Nothing
368358
369359 {-# INLINE initFold #-}
370− initFold kv (Fold step1 initial1 extract1 final1) (k, a) = do
360+ initFold kv (Fold step1 initial1 extract1) (k, a) = do
371361 res <- initial1
372362 case res of
373363 Partial s -> do
374364 res1 <- step1 s a
375365 case res1 of
376366 Partial _ -> do
377− let fld = Fold step1 (return res1) extract1 final1
367+ let fld = Fold step1 (return res1) extract1
378368 ref <- liftIO $ newIORef fld
379369 return $ Tuple' (IsMap.mapInsert k ref kv) Nothing
380370 Done b -> return $ Tuple' kv (Just (k, b))
381371 Done b -> return $ Tuple' kv (Just (k, b))
382372
383373 {-# INLINE runFold #-}
384− runFold kv ref (Fold step1 initial1 extract1 final1) (k, a) = do
374+ runFold kv ref (Fold step1 initial1 extract1) (k, a) = do
385375 res <- initial1
386376 case res of
387377 Partial s -> do
388378 res1 <- step1 s a
389379 case res1 of
390380 Partial _ -> do
391− let fld = Fold step1 (return res1) extract1 final1
381+ let fld = Fold step1 (return res1) extract1
392382 liftIO $ writeIORef ref fld
393383 return $ Tuple' kv Nothing
394384 Done b ->
@@ -406,27 +396,16 @@ demuxGenericIO getKey getFold =
406396 f <- liftIO $ readIORef ref
407397 runFold kv ref f (k, a)
408398
409− extract (Tuple' kv x) = return (Prelude.mapM f kv, x)
399+ extract (Tuple' kv x) = (Prelude.mapM f kv, x)
410400
411401 where
412402
413403 f ref = do
414− Fold _ i e _ <- liftIO $ readIORef ref
404+ (Fold _ i e) <- liftIO $ readIORef ref
415405 r <- i
416406 case r of
417407 Partial s -> e s
418− _ -> error "demuxGenericIO: unreachable code"
419−
420− final (Tuple' kv x) = return (Prelude.mapM f kv, x)
421−
422− where
423−
424− f ref = do
425− Fold _ i _ fin <- liftIO $ readIORef ref
426− r <- i
427− case r of
428− Partial s -> fin s
429− _ -> error "demuxGenericIO: unreachable code"
408+ Done b -> return b
430409
431410-- | This is specialized version of 'demux' that uses mutable IO cells as
432411-- fold accumulators for better performance.
@@ -441,13 +420,6 @@ demuxIO :: (MonadIO m, Ord k) =>
441420 -> Fold m a (m (Map k b), Maybe (k, b))
442421demuxIO = demuxGenericIO
443422
444−-- | Fold a key value stream to a key-value Map. If the same key appears
445−-- multiple times, only the last value is retained.
446−{-# INLINE kvToMapOverwriteGeneric #-}
447−kvToMapOverwriteGeneric :: (Monad m, IsMap f) => Fold m (Key f, a) (f a)
448−kvToMapOverwriteGeneric =
449− foldl' (\kv (k, v) -> IsMap.mapInsert k v kv) IsMap.mapEmpty
450−
451423{-# INLINE demuxToContainer #-}
452424demuxToContainer :: (Monad m, IsMap f, Traversable f) =>
453425 (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
@@ -535,8 +507,8 @@ classifyGeneric :: (Monad m, IsMap f, Traversable f, Ord (Key f)) =>
535507 -- for that use case. We return an action because we want it to be lazy so
536508 -- that the downstream consumers can choose to process or discard it.
537509 (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
538−classifyGeneric f (Fold step1 initial1 extract1 final1) =
539− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
510+classifyGeneric f (Fold step1 initial1 extract1) =
511+ fmap extract $ foldlM' step initial
540512
541513 where
542514
@@ -573,16 +545,7 @@ classifyGeneric f (Fold step1 initial1 extract1 final1) =
573545 let kv1 = IsMap.mapDelete k kv
574546 in Tuple3' kv1 (Set.insert k set) (Just (k, b))
575547
576− extract (Tuple3' kv _ x) = return (Prelude.mapM extract1 kv, x)
577−
578− final (Tuple3' kv set x) = return (IsMap.mapTraverseWithKey f1 kv, x)
579−
580− where
581−
582− f1 k s = do
583− if Set.member k set
584− then extract1 s
585− else final1 s
548+ extract (Tuple3' kv _ x) = (Prelude.mapM extract1 kv, x)
586549
587550-- | Folds the values for each key using the supplied fold. When scanning, as
588551-- soon as the fold is complete, its result is available in the second
@@ -607,8 +570,8 @@ classify = classifyGeneric
607570{-# INLINE classifyGenericIO #-}
608571classifyGenericIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) =>
609572 (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
610−classifyGenericIO f (Fold step1 initial1 extract1 final1) =
611− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) extract final
573+classifyGenericIO f (Fold step1 initial1 extract1) =
574+ fmap extract $ foldlM' step initial
612575
613576 where
614577
@@ -647,21 +610,8 @@ classifyGenericIO f (Fold step1 initial1 extract1 final1) =
647610 in return
648611 $ Tuple3' kv1 (Set.insert k set) (Just (k, b))
649612
650− extract (Tuple3' kv _ x) = return (Prelude.mapM g kv, x)
651−
652− where
653−
654− g ref = liftIO (readIORef ref) >>= extract1
655−
656− final (Tuple3' kv set x) = return (IsMap.mapTraverseWithKey g kv, x)
657−
658− where
659−
660− g k ref = do
661− s <- liftIO $ readIORef ref
662− if Set.member k set
663− then extract1 s
664− else final1 s
613+ extract (Tuple3' kv _ x) =
614+ (Prelude.mapM (\ref -> liftIO (readIORef ref) >>= extract1) kv, x)
665615
666616-- | Same as classify except that it uses mutable IORef cells in the
667617-- Map providing better performance. Be aware that if this is used as a scan,
@@ -676,6 +626,13 @@ classifyIO :: (MonadIO m, Ord k) =>
676626 (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
677627classifyIO = classifyGenericIO
678628
629+-- | Fold a key value stream to a key-value Map. If the same key appears
630+-- multiple times, only the last value is retained.
631+{-# INLINE kvToMapOverwriteGeneric #-}
632+kvToMapOverwriteGeneric :: (Monad m, IsMap f) => Fold m (Key f, a) (f a)
633+kvToMapOverwriteGeneric =
634+ foldl' (\kv (k, v) -> IsMap.mapInsert k v kv) IsMap.mapEmpty
635+
679636{-# INLINE toContainer #-}
680637toContainer :: (Monad m, IsMap f, Traversable f, Ord (Key f)) =>
681638 (a -> Key f) -> Fold m a b -> Fold m a (f b)
core/src/Streamly/Internal/Data/Fold/Step.hs+2−2
@@ -8,7 +8,7 @@
88--
99module Streamly.Internal.Data.Fold.Step
1010 (
11− -- * Step Type
11+ -- * Types
1212 Step (..)
1313
1414 , mapMStep
@@ -40,7 +40,7 @@ data Step s b
4040 = Partial !s
4141 | Done !b
4242
43−-- | 'first' maps over the fold state and 'second' maps over the fold result.
43+-- | 'first' maps over 'Partial' and 'second' maps over 'Done'.
4444--
4545instance Bifunctor Step where
4646 {-# INLINE bimap #-}
core/src/Streamly/Internal/Data/Fold/Tee.hs−2
@@ -16,9 +16,7 @@ module Streamly.Internal.Data.Fold.Tee
1616 )
1717where
1818
19−#if !MIN_VERSION_base(4,18,0)
2019import Control.Applicative (liftA2)
21−#endif
2220import Streamly.Internal.Data.Fold.Type (Fold)
2321
2422import qualified Streamly.Internal.Data.Fold.Type as Fold
core/src/Streamly/Internal/Data/Fold/Type.hs+120−344
@@ -344,9 +344,11 @@
344344--
345345module Streamly.Internal.Data.Fold.Type
346346 (
347− module Streamly.Internal.Data.Fold.Step
347+ -- * Imports
348+ -- $setup
348349
349− -- * Fold Type
350+ -- * Types
351+ Step (..)
350352 , Fold (..)
351353
352354 -- * Constructors
@@ -367,8 +369,6 @@ module Streamly.Internal.Data.Fold.Type
367369 , toList
368370 , toStreamK
369371 , toStreamKRev
370− , lengthGeneric
371− , length
372372
373373 -- * Combinators
374374
@@ -393,8 +393,6 @@ module Streamly.Internal.Data.Fold.Type
393393 -- ** Trimming
394394 , take
395395 , taking
396− , takeEndBy_
397− , takeEndBy
398396 , dropping
399397
400398 -- ** Sequential application
@@ -446,25 +444,21 @@ where
446444
447445#include "inline.hs"
448446
449−#if !MIN_VERSION_base(4,18,0)
450447import Control.Applicative (liftA2)
451−#endif
452−import Control.Monad ((>=>), void)
448+import Control.Monad ((>=>))
453449import Data.Bifunctor (Bifunctor(..))
454450import Data.Either (fromLeft, fromRight, isLeft, isRight)
455451import Data.Functor.Identity (Identity(..))
456452import Fusion.Plugin.Types (Fuse(..))
453+import Streamly.Internal.Data.Fold.Step (Step(..), mapMStep, chainStepM)
457454import Streamly.Internal.Data.Maybe.Strict (Maybe'(..), toMaybe)
458455import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
459456import Streamly.Internal.Data.Refold.Type (Refold(..))
460457
461−import qualified Streamly.Internal.Data.StreamK.Type as K
458+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
462459
463460import Prelude hiding (Foldable(..), concatMap, filter, map, take)
464461
465−-- Entire module is exported, do not import selectively
466−import Streamly.Internal.Data.Fold.Step
467−
468462#include "DocTestDataFold.hs"
469463
470464------------------------------------------------------------------------------
@@ -475,64 +469,23 @@ import Streamly.Internal.Data.Fold.Step
475469-- The type @b@ is the accumulator of the writer. That's the reason the
476470-- default folds in various modules are called "write".
477471
478−-- An alternative to using an "extract" function is to use "Partial s b" style
479−-- partial value so that we always emit the output value and there is no need
480−-- to extract. Then extract can be used for cleanup purposes. But in this case
481−-- in some cases we may need a "Continue" constructor where an output value is
482−-- not available, this was implicit earlier. Also, "b" should be lazy here so
483−-- that we do not always compute it even if we do not need it.
484−--
485−-- Partial s b --> extract :: s -> b
486−-- Continue --> extract :: s -> Maybe b
487−--
488−-- But keeping 'b' lazy does not let the fold optimize well. It leads to
489−-- significant regressions in the key-value folds.
490−--
491−-- The "final" function complicates combinators that take other folds as
492−-- argument because we need to call their finalizers at right places. An
493−-- alternative to reduce this complexity where it is not required is to use a
494−-- separate type for bracketed folds but then we need to manage the complexity
495−-- of two different fold types.
496−
497−-- The "final" function could be (s -> m (Step s b)), like in parsers so that
498−-- it can be called in a loop to drain the fold.
499−
500−-- | The type @Fold m a b@ represents a consumer of an input stream of values
501−-- of type @a@ and returning a final value of type @b@ in 'Monad' @m@. The
502−-- constructor of a fold is @Fold step initial extract final@.
503−--
504−-- The fold uses an internal state of type @s@. The initial value of the state
505−-- @s@ is created by @initial@. This function is called once and only once
506−-- before the fold starts consuming input. Any resource allocation can be done
507−-- in this function.
508−--
509−-- The @step@ function is called on each input, it consumes an input and
510−-- returns the next intermediate state (see 'Step') or the final result @b@ if
511−-- the fold terminates.
512−--
513−-- If the fold is used as a scan, the @extract@ function is used by the scan
514−-- driver to map the current state @s@ of the fold to the fold result. Thus
515−-- @extract@ can be called multiple times. In some folds, where scanning does
516−-- not make sense, this function is left unimplemented; such folds cannot be
517−-- used as scans.
518−--
519−-- Before a fold terminates, @final@ is called once and only once (unless the
520−-- fold terminated in @initial@ itself). Any resources allocated by @initial@
521−-- can be released in @final@. In folds that do not require any cleanup
522−-- @extract@ and @final@ are typically the same.
523−--
524−-- When implementing fold combinators, care should be taken to cleanup any
525−-- state of the argument folds held by the fold by calling the respective
526−-- @final@ at all exit points of the fold. Also, @final@ should not be called
527−-- more than once. Note that if a fold terminates by 'Done' constructor, there
528−-- is no state to cleanup.
472+-- | The type @Fold m a b@ having constructor @Fold step initial extract@
473+-- represents a fold over an input stream of values of type @a@ to a final
474+-- value of type @b@ in 'Monad' @m@.
475+--
476+-- The fold uses an intermediate state @s@ as accumulator, the type @s@ is
477+-- internal to the specific fold definition. The initial value of the fold
478+-- state @s@ is returned by @initial@. The @step@ function consumes an input
479+-- and either returns the final result @b@ if the fold is done or the next
480+-- intermediate state (see 'Step'). At any point the fold driver can extract
481+-- the result from the intermediate state using the @extract@ function.
529482--
530483-- NOTE: The constructor is not yet released, smart constructors are provided
531484-- to create folds.
532485--
533486data Fold m a b =
534− -- | @Fold@ @step@ @initial@ @extract@ @final@
535− forall s. Fold (s -> a -> m (Step s b)) (m (Step s b)) (s -> m b) (s -> m b)
487+ -- | @Fold @ @ step @ @ initial @ @ extract@
488+ forall s. Fold (s -> a -> m (Step s b)) (m (Step s b)) (s -> m b)
536489
537490------------------------------------------------------------------------------
538491-- Mapping on the output
@@ -542,8 +495,7 @@ data Fold m a b =
542495--
543496{-# INLINE rmapM #-}
544497rmapM :: Monad m => (b -> m c) -> Fold m a b -> Fold m a c
545−rmapM f (Fold step initial extract final) =
546− Fold step1 initial1 (extract >=> f) (final >=> f)
498+rmapM f (Fold step initial extract) = Fold step1 initial1 (extract >=> f)
547499
548500 where
549501
@@ -574,7 +526,6 @@ foldl' step initial =
574526 (\s a -> return $ Partial $ step s a)
575527 (return (Partial initial))
576528 return
577− return
578529
579530-- | Make a fold from a left fold style monadic step function and initial value
580531-- of the accumulator.
@@ -589,7 +540,7 @@ foldl' step initial =
589540{-# INLINE foldlM' #-}
590541foldlM' :: Monad m => (b -> a -> m b) -> m b -> Fold m a b
591542foldlM' step initial =
592− Fold (\s a -> Partial <$> step s a) (Partial <$> initial) return return
543+ Fold (\s a -> Partial <$> step s a) (Partial <$> initial) return
593544
594545-- | Make a strict left fold, for non-empty streams, using first element as the
595546-- starting value. Returns Nothing if the stream is empty.
@@ -687,11 +638,7 @@ foldrM' g z =
687638{-# INLINE foldt' #-}
688639foldt' :: Monad m => (s -> a -> Step s b) -> Step s b -> (s -> b) -> Fold m a b
689640foldt' step initial extract =
690− Fold
691− (\s a -> return $ step s a)
692− (return initial)
693− (return . extract)
694− (return . extract)
641+ Fold (\s a -> return $ step s a) (return initial) (return . extract)
695642
696643-- | Make a terminating fold with an effectful step function and initial state,
697644-- and a state extraction function.
@@ -704,7 +651,7 @@ foldt' step initial extract =
704651--
705652{-# INLINE foldtM' #-}
706653foldtM' :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> Fold m a b
707−foldtM' step initial extract = Fold step initial extract extract
654+foldtM' = Fold
708655
709656------------------------------------------------------------------------------
710657-- Refold
@@ -718,7 +665,7 @@ foldtM' step initial extract = Fold step initial extract extract
718665-- /Internal/
719666fromRefold :: Refold m c a b -> c -> Fold m a b
720667fromRefold (Refold step inject extract) c =
721− Fold step (inject c) extract extract
668+ Fold step (inject c) extract
722669
723670------------------------------------------------------------------------------
724671-- Basic Folds
@@ -771,29 +718,6 @@ toStreamKRev = foldl' (flip K.cons) K.nil
771718toStreamK :: Monad m => Fold m a (K.StreamK n a)
772719toStreamK = foldr K.cons K.nil
773720
774−-- | Like 'length', except with a more general 'Num' return value
775−--
776−-- Definition:
777−--
778−-- >>> lengthGeneric = fmap getSum $ Fold.foldMap (Sum . const 1)
779−-- >>> lengthGeneric = Fold.foldl' (\n _ -> n + 1) 0
780−--
781−-- /Pre-release/
782−{-# INLINE lengthGeneric #-}
783−lengthGeneric :: (Monad m, Num b) => Fold m a b
784−lengthGeneric = foldl' (\n _ -> n + 1) 0
785−
786−-- | Determine the length of the input stream.
787−--
788−-- Definition:
789−--
790−-- >>> length = Fold.lengthGeneric
791−-- >>> length = fmap getSum $ Fold.foldMap (Sum . const 1)
792−--
793−{-# INLINE length #-}
794−length :: Monad m => Fold m a Int
795−length = lengthGeneric
796−
797721------------------------------------------------------------------------------
798722-- Instances
799723------------------------------------------------------------------------------
@@ -801,8 +725,7 @@ length = lengthGeneric
801725-- | Maps a function on the output of the fold (the type @b@).
802726instance Functor m => Functor (Fold m a) where
803727 {-# INLINE fmap #-}
804− fmap f (Fold step1 initial1 extract final) =
805− Fold step initial (fmap2 f extract) (fmap2 f final)
728+ fmap f (Fold step1 initial1 extract) = Fold step initial (fmap2 f extract)
806729
807730 where
808731
@@ -837,7 +760,7 @@ instance Functor m => Functor (Fold m a) where
837760--
838761{-# INLINE fromPure #-}
839762fromPure :: Applicative m => b -> Fold m a b
840−fromPure b = Fold undefined (pure $ Done b) pure pure
763+fromPure b = Fold undefined (pure $ Done b) pure
841764
842765-- | Make a fold that yields the result of the supplied effectful action
843766-- without consuming any further input.
@@ -846,7 +769,7 @@ fromPure b = Fold undefined (pure $ Done b) pure pure
846769--
847770{-# INLINE fromEffect #-}
848771fromEffect :: Applicative m => m b -> Fold m a b
849−fromEffect b = Fold undefined (Done <$> b) pure pure
772+fromEffect b = Fold undefined (Done <$> b) pure
850773
851774{-# ANN type SeqFoldState Fuse #-}
852775data SeqFoldState sl f sr = SeqFoldL !sl | SeqFoldR !f !sr
@@ -871,20 +794,16 @@ data SeqFoldState sl f sr = SeqFoldL !sl | SeqFoldR !f !sr
871794-- complexity, because each composition adds a new branch that each subsequent
872795-- fold's input element has to traverse, therefore, it cannot scale to a large
873796-- number of compositions. After around 100 compositions the performance starts
874−-- dipping rapidly compared to a CPS style implementation.
875−--
876−-- For larger number of compositions you can convert the fold to a parser and
877−-- use ParserK.
797+-- dipping rapidly compared to a CPS style implementation. When you need
798+-- scaling use parser monad instead.
878799--
879800-- /Time: O(n^2) where n is the number of compositions./
880801--
881802{-# INLINE splitWith #-}
882803splitWith :: Monad m =>
883804 (a -> b -> c) -> Fold m x a -> Fold m x b -> Fold m x c
884−splitWith func
885− (Fold stepL initialL _ finalL)
886− (Fold stepR initialR _ finalR) =
887− Fold step initial extract final
805+splitWith func (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
806+ Fold step initial extract
888807
889808 where
890809
@@ -901,18 +820,13 @@ splitWith func
901820 step (SeqFoldL st) a = runL (stepL st a)
902821 step (SeqFoldR f st) a = runR (stepR st a) f
903822
904− -- XXX splitWith should not be used for scanning
905− -- It would rarely make sense and resource tracking and cleanup would be
906− -- expensive. especially when multiple splitWith are chained.
907− extract _ = error "splitWith: cannot be used for scanning"
908−
909− final (SeqFoldR f sR) = fmap f (finalR sR)
910− final (SeqFoldL sL) = do
911− rL <- finalL sL
823+ extract (SeqFoldR f sR) = fmap f (extractR sR)
824+ extract (SeqFoldL sL) = do
825+ rL <- extractL sL
912826 res <- initialR
913827 fmap (func rL)
914828 $ case res of
915− Partial sR -> finalR sR
829+ Partial sR -> extractR sR
916830 Done rR -> return rR
917831
918832{-# DEPRECATED serialWith "Please use \"splitWith\" instead" #-}
@@ -932,8 +846,8 @@ data SeqFoldState_ sl sr = SeqFoldL_ !sl | SeqFoldR_ !sr
932846--
933847{-# INLINE split_ #-}
934848split_ :: Monad m => Fold m x a -> Fold m x b -> Fold m x b
935−split_ (Fold stepL initialL _ finalL) (Fold stepR initialR _ finalR) =
936− Fold step initial extract final
849+split_ (Fold stepL initialL _) (Fold stepR initialR extractR) =
850+ Fold step initial extract
937851
938852 where
939853
@@ -956,16 +870,11 @@ split_ (Fold stepL initialL _ finalL) (Fold stepR initialR _ finalR) =
956870 resR <- stepR st a
957871 return $ first SeqFoldR_ resR
958872
959− -- XXX split_ should not be used for scanning
960− -- See splitWith for more details.
961− extract _ = error "split_: cannot be used for scanning"
962−
963− final (SeqFoldR_ sR) = finalR sR
964− final (SeqFoldL_ sL) = do
965− _ <- finalL sL
873+ extract (SeqFoldR_ sR) = extractR sR
874+ extract (SeqFoldL_ _) = do
966875 res <- initialR
967876 case res of
968− Partial sR -> finalR sR
877+ Partial sR -> extractR sR
969878 Done rR -> return rR
970879
971880-- | 'Applicative' form of 'splitWith'. Split the input serially over two
@@ -1013,10 +922,8 @@ data TeeState sL sR bL bR
1013922--
1014923{-# INLINE teeWith #-}
1015924teeWith :: Monad m => (a -> b -> c) -> Fold m x a -> Fold m x b -> Fold m x c
1016−teeWith f
1017− (Fold stepL initialL extractL finalL)
1018− (Fold stepR initialR extractR finalR) =
1019− Fold step initial extract final
925+teeWith f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
926+ Fold step initial extract
1020927
1021928 where
1022929
@@ -1043,10 +950,6 @@ teeWith f
1043950 extract (TeeLeft bR sL) = (`f` bR) <$> extractL sL
1044951 extract (TeeRight bL sR) = f bL <$> extractR sR
1045952
1046− final (TeeBoth sL sR) = f <$> finalL sL <*> finalR sR
1047− final (TeeLeft bR sL) = (`f` bR) <$> finalL sL
1048− final (TeeRight bL sR) = f bL <$> finalR sR
1049−
1050953{-# ANN type TeeFstState Fuse #-}
1051954data TeeFstState sL sR b
1052955 = TeeFstBoth !sL !sR
@@ -1059,10 +962,8 @@ data TeeFstState sL sR b
1059962{-# INLINE teeWithFst #-}
1060963teeWithFst :: Monad m =>
1061964 (b -> c -> d) -> Fold m a b -> Fold m a c -> Fold m a d
1062−teeWithFst f
1063− (Fold stepL initialL extractL finalL)
1064− (Fold stepR initialR extractR finalR) =
1065− Fold step initial extract final
965+teeWithFst f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
966+ Fold step initial extract
1066967
1067968 where
1068969
@@ -1081,7 +982,7 @@ teeWithFst f
1081982 Done bl -> do
1082983 Done . f bl <$>
1083984 case resR of
1084− Partial sr -> finalR sr
985+ Partial sr -> extractR sr
1085986 Done br -> return br
1086987
1087988 initial = runBoth initialL initialR
@@ -1092,9 +993,6 @@ teeWithFst f
1092993 extract (TeeFstBoth sL sR) = f <$> extractL sL <*> extractR sR
1093994 extract (TeeFstLeft bR sL) = (`f` bR) <$> extractL sL
1094995
1095− final (TeeFstBoth sL sR) = f <$> finalL sL <*> finalR sR
1096− final (TeeFstLeft bR sL) = (`f` bR) <$> finalL sL
1097−
1098996-- | Like 'teeWith' but terminates as soon as any one of the two folds
1099997-- terminates.
1100998--
@@ -1103,10 +1001,8 @@ teeWithFst f
11031001{-# INLINE teeWithMin #-}
11041002teeWithMin :: Monad m =>
11051003 (b -> c -> d) -> Fold m a b -> Fold m a c -> Fold m a d
1106−teeWithMin f
1107− (Fold stepL initialL extractL finalL)
1108− (Fold stepR initialR extractR finalR) =
1109− Fold step initial extract final
1004+teeWithMin f (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1005+ Fold step initial extract
11101006
11111007 where
11121008
@@ -1118,12 +1014,12 @@ teeWithMin f
11181014 Partial sl -> do
11191015 case resR of
11201016 Partial sr -> return $ Partial $ Tuple' sl sr
1121− Done br -> Done . (`f` br) <$> finalL sl
1017+ Done br -> Done . (`f` br) <$> extractL sl
11221018
11231019 Done bl -> do
11241020 Done . f bl <$>
11251021 case resR of
1126− Partial sr -> finalR sr
1022+ Partial sr -> extractR sr
11271023 Done br -> return br
11281024
11291025 initial = runBoth initialL initialR
@@ -1132,8 +1028,6 @@ teeWithMin f
11321028
11331029 extract (Tuple' sL sR) = f <$> extractL sL <*> extractR sR
11341030
1135− final (Tuple' sL sR) = f <$> finalL sL <*> finalR sR
1136−
11371031-- | Shortest alternative. Apply both folds in parallel but choose the result
11381032-- from the one which consumed least input i.e. take the shortest succeeding
11391033-- fold.
@@ -1145,8 +1039,8 @@ teeWithMin f
11451039--
11461040{-# INLINE shortest #-}
11471041shortest :: Monad m => Fold m x a -> Fold m x b -> Fold m x (Either a b)
1148−shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
1149− Fold step initial extract final
1042+shortest (Fold stepL initialL extractL) (Fold stepR initialR _) =
1043+ Fold step initial extract
11501044
11511045 where
11521046
@@ -1154,16 +1048,10 @@ shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
11541048 runBoth actionL actionR = do
11551049 resL <- actionL
11561050 resR <- actionR
1157− case resL of
1158− Partial sL ->
1159− case resR of
1160− Partial sR -> return $ Partial $ Tuple' sL sR
1161− Done bR -> finalL sL >> return (Done (Right bR))
1162− Done bL -> do
1163− case resR of
1164− Partial sR -> void (finalR sR)
1165− Done _ -> return ()
1166− return (Done (Left bL))
1051+ return $
1052+ case resL of
1053+ Partial sL -> bimap (Tuple' sL) Right resR
1054+ Done bL -> Done $ Left bL
11671055
11681056 initial = runBoth initialL initialR
11691057
@@ -1171,8 +1059,6 @@ shortest (Fold stepL initialL extractL finalL) (Fold stepR initialR _ finalR) =
11711059
11721060 extract (Tuple' sL _) = Left <$> extractL sL
11731061
1174− final (Tuple' sL sR) = Left <$> finalL sL <* finalR sR
1175−
11761062{-# ANN type LongestState Fuse #-}
11771063data LongestState sL sR
11781064 = LongestBoth !sL !sR
@@ -1190,10 +1076,8 @@ data LongestState sL sR
11901076--
11911077{-# INLINE longest #-}
11921078longest :: Monad m => Fold m x a -> Fold m x b -> Fold m x (Either a b)
1193−longest
1194− (Fold stepL initialL _ finalL)
1195− (Fold stepR initialR _ finalR) =
1196− Fold step initial extract final
1079+longest (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1080+ Fold step initial extract
11971081
11981082 where
11991083
@@ -1216,18 +1100,14 @@ longest
12161100 step (LongestLeft sL) a = bimap LongestLeft Left <$> stepL sL a
12171101 step (LongestRight sR) a = bimap LongestRight Right <$> stepR sR a
12181102
1219− -- XXX Scan with this may not make sense as we cannot determine the longest
1220− -- until one of them have exhausted.
1221− extract _ = error $ "longest: scan is not allowed as longest cannot be "
1222− ++ "determined until one fold has exhausted."
1223−
1224− final (LongestLeft sL) = Left <$> finalL sL
1225− final (LongestRight sR) = Right <$> finalR sR
1226− final (LongestBoth sL sR) = Left <$> finalL sL <* finalR sR
1103+ left sL = Left <$> extractL sL
1104+ extract (LongestLeft sL) = left sL
1105+ extract (LongestRight sR) = Right <$> extractR sR
1106+ extract (LongestBoth sL _) = left sL
12271107
1228−data ConcatMapState m sa a b c
1229− = B !sa (sa -> m b)
1230− | forall s. C (s -> a -> m (Step s c)) !s (s -> m c) (s -> m c)
1108+data ConcatMapState m sa a c
1109+ = B !sa
1110+ | forall s. C (s -> a -> m (Step s c)) !s (s -> m c)
12311111
12321112-- | Map a 'Fold' returning function on the result of a 'Fold' and run the
12331113-- returned fold. This operation can be used to express data dependencies
@@ -1250,47 +1130,43 @@ data ConcatMapState m sa a b c
12501130--
12511131{-# INLINE concatMap #-}
12521132concatMap :: Monad m => (b -> Fold m a c) -> Fold m a b -> Fold m a c
1253−concatMap f (Fold stepa initiala _ finala) =
1254− Fold stepc initialc extractc finalc
1133+concatMap f (Fold stepa initiala extracta) = Fold stepc initialc extractc
12551134 where
12561135 initialc = do
12571136 r <- initiala
12581137 case r of
1259− Partial s -> return $ Partial (B s finala)
1138+ Partial s -> return $ Partial (B s)
12601139 Done b -> initInnerFold (f b)
12611140
1262− stepc (B s fin) a = do
1141+ stepc (B s) a = do
12631142 r <- stepa s a
12641143 case r of
1265− Partial s1 -> return $ Partial (B s1 fin)
1144+ Partial s1 -> return $ Partial (B s1)
12661145 Done b -> initInnerFold (f b)
12671146
1268− stepc (C stepInner s extractInner fin) a = do
1147+ stepc (C stepInner s extractInner) a = do
12691148 r <- stepInner s a
12701149 return $ case r of
1271− Partial sc -> Partial (C stepInner sc extractInner fin)
1150+ Partial sc -> Partial (C stepInner sc extractInner)
12721151 Done c -> Done c
12731152
1274− -- XXX Cannot use for scanning
1275− extractc _ = error "concatMap: cannot be used for scanning"
1153+ extractc (B s) = do
1154+ r <- extracta s
1155+ initExtract (f r)
1156+ extractc (C _ sInner extractInner) = extractInner sInner
12761157
1277− initInnerFold (Fold step i e fin) = do
1158+ initInnerFold (Fold step i e) = do
12781159 r <- i
12791160 return $ case r of
1280− Partial s -> Partial (C step s e fin)
1161+ Partial s -> Partial (C step s e)
12811162 Done c -> Done c
12821163
1283− initFinalize (Fold _ i _ fin) = do
1164+ initExtract (Fold _ i e) = do
12841165 r <- i
12851166 case r of
1286− Partial s -> fin s
1167+ Partial s -> e s
12871168 Done c -> return c
12881169
1289− finalc (B s fin) = do
1290− r <- fin s
1291− initFinalize (f r)
1292− finalc (C _ sInner _ fin) = fin sInner
1293−
12941170------------------------------------------------------------------------------
12951171-- Mapping on input
12961172------------------------------------------------------------------------------
@@ -1309,7 +1185,7 @@ concatMap f (Fold stepa initiala _ finala) =
13091185--
13101186{-# INLINE lmap #-}
13111187lmap :: (a -> b) -> Fold m b r -> Fold m a r
1312−lmap f (Fold step begin done final) = Fold step' begin done final
1188+lmap f (Fold step begin done) = Fold step' begin done
13131189 where
13141190 step' x a = step x (f a)
13151191
@@ -1317,7 +1193,7 @@ lmap f (Fold step begin done final) = Fold step' begin done final
13171193--
13181194{-# INLINE lmapM #-}
13191195lmapM :: Monad m => (a -> m b) -> Fold m b r -> Fold m a r
1320−lmapM f (Fold step begin done final) = Fold step' begin done final
1196+lmapM f (Fold step begin done) = Fold step' begin done
13211197 where
13221198 step' x a = f a >>= step x
13231199
@@ -1329,10 +1205,8 @@ lmapM f (Fold step begin done final) = Fold step' begin done final
13291205-- /Pre-release/
13301206{-# INLINE postscan #-}
13311207postscan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1332−postscan
1333− (Fold stepL initialL extractL finalL)
1334− (Fold stepR initialR extractR finalR) =
1335− Fold step initial extract final
1208+postscan (Fold stepL initialL extractL) (Fold stepR initialR extractR) =
1209+ Fold step initial extract
13361210
13371211 where
13381212
@@ -1343,32 +1217,30 @@ postscan
13431217 Done bL -> do
13441218 rR <- stepR sR bL
13451219 case rR of
1346− Partial sR1 -> Done <$> finalR sR1
1220+ Partial sR1 -> Done <$> extractR sR1
13471221 Done bR -> return $ Done bR
13481222 Partial sL -> do
13491223 !b <- extractL sL
13501224 rR <- stepR sR b
1351− case rR of
1352− Partial sR1 -> return $ Partial (sL, sR1)
1353− Done bR -> finalL sL >> return (Done bR)
1225+ return
1226+ $ case rR of
1227+ Partial sR1 -> Partial (sL, sR1)
1228+ Done bR -> Done bR
13541229
13551230 initial = do
1356− rR <- initialR
1357− case rR of
1358− Partial sR -> do
1359− rL <- initialL
1231+ r <- initialR
1232+ rL <- initialL
1233+ case r of
1234+ Partial sR ->
13601235 case rL of
1361− Done _ -> Done <$> finalR sR
1236+ Done _ -> Done <$> extractR sR
13621237 Partial sL -> return $ Partial (sL, sR)
13631238 Done b -> return $ Done b
13641239
1365− -- XXX should use Tuple'
13661240 step (sL, sR) x = runStep (stepL sL x) sR
13671241
13681242 extract = extractR . snd
13691243
1370− final (sL, sR) = finalL sL *> finalR sR
1371−
13721244------------------------------------------------------------------------------
13731245-- Filtering
13741246------------------------------------------------------------------------------
@@ -1381,7 +1253,7 @@ postscan
13811253--
13821254{-# INLINE_NORMAL catMaybes #-}
13831255catMaybes :: Monad m => Fold m a b -> Fold m (Maybe a) b
1384−catMaybes (Fold step initial extract final) = Fold step1 initial extract final
1256+catMaybes (Fold step initial extract) = Fold step1 initial extract
13851257
13861258 where
13871259
@@ -1421,7 +1293,7 @@ filtering f = foldl' step Nothing
14211293{-# INLINE filter #-}
14221294filter :: Monad m => (a -> Bool) -> Fold m a r -> Fold m a r
14231295-- filter p = scanMaybe (filtering p)
1424−filter f (Fold step begin extract final) = Fold step' begin extract final
1296+filter f (Fold step begin done) = Fold step' begin done
14251297 where
14261298 step' x a = if f a then step x a else return $ Partial x
14271299
@@ -1432,7 +1304,7 @@ filter f (Fold step begin extract final) = Fold step' begin extract final
14321304--
14331305{-# INLINE filterM #-}
14341306filterM :: Monad m => (a -> m Bool) -> Fold m a r -> Fold m a r
1435−filterM f (Fold step begin extract final) = Fold step' begin extract final
1307+filterM f (Fold step begin done) = Fold step' begin done
14361308 where
14371309 step' x a = do
14381310 use <- f a
@@ -1521,7 +1393,7 @@ dropping n = foldt' step initial extract
15211393{-# INLINE take #-}
15221394take :: Monad m => Int -> Fold m a b -> Fold m a b
15231395-- take n = scanMaybe (taking n)
1524−take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
1396+take n (Fold fstep finitial fextract) = Fold step initial extract
15251397
15261398 where
15271399
@@ -1533,7 +1405,7 @@ take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
15331405 s1 = Tuple'Fused i1 s
15341406 if i1 < n
15351407 then return $ Partial s1
1536− else Done <$> ffinal s
1408+ else Done <$> fextract s
15371409 Done b -> return $ Done b
15381410
15391411 initial = finitial >>= next (-1)
@@ -1542,72 +1414,6 @@ take n (Fold fstep finitial fextract ffinal) = Fold step initial extract final
15421414
15431415 extract (Tuple'Fused _ r) = fextract r
15441416
1545− final (Tuple'Fused _ r) = ffinal r
1546−
1547−-- Note: Keep this consistent with S.splitOn. In fact we should eliminate
1548−-- S.splitOn in favor of the fold.
1549−--
1550−-- XXX Use Fold.many instead once it is fixed.
1551−-- > Stream.splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
1552−
1553−-- | Like 'takeEndBy' but drops the element on which the predicate succeeds.
1554−--
1555−-- Example:
1556−--
1557−-- >>> input = Stream.fromList "hello\nthere\n"
1558−-- >>> line = Fold.takeEndBy_ (== '\n') Fold.toList
1559−-- >>> Stream.fold line input
1560−-- "hello"
1561−--
1562−-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1563−-- ["hello","there"]
1564−--
1565−{-# INLINE takeEndBy_ #-}
1566−takeEndBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1567−-- takeEndBy_ predicate = scanMaybe (takingEndBy_ predicate)
1568−takeEndBy_ predicate (Fold fstep finitial fextract ffinal) =
1569− Fold step finitial fextract ffinal
1570−
1571− where
1572−
1573− step s a =
1574− if not (predicate a)
1575− then fstep s a
1576− else Done <$> ffinal s
1577−
1578−-- Note:
1579−-- > Stream.splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
1580−
1581−-- | Take the input, stop when the predicate succeeds taking the succeeding
1582−-- element as well.
1583−--
1584−-- Example:
1585−--
1586−-- >>> input = Stream.fromList "hello\nthere\n"
1587−-- >>> line = Fold.takeEndBy (== '\n') Fold.toList
1588−-- >>> Stream.fold line input
1589−-- "hello\n"
1590−--
1591−-- >>> Stream.fold Fold.toList $ Stream.foldMany line input
1592−-- ["hello\n","there\n"]
1593−--
1594−{-# INLINE takeEndBy #-}
1595−takeEndBy :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a b
1596−-- takeEndBy predicate = scanMaybe (takingEndBy predicate)
1597−takeEndBy predicate (Fold fstep finitial fextract ffinal) =
1598− Fold step finitial fextract ffinal
1599−
1600− where
1601−
1602− step s a = do
1603− res <- fstep s a
1604− if not (predicate a)
1605− then return res
1606− else do
1607− case res of
1608− Partial s1 -> Done <$> ffinal s1
1609− Done b -> return $ Done b
1610−
16111417------------------------------------------------------------------------------
16121418-- Nesting
16131419------------------------------------------------------------------------------
@@ -1625,8 +1431,8 @@ takeEndBy predicate (Fold fstep finitial fextract ffinal) =
16251431-- /Pre-release/
16261432{-# INLINE duplicate #-}
16271433duplicate :: Monad m => Fold m a b -> Fold m a (Fold m a b)
1628−duplicate (Fold step1 initial1 extract1 final1) =
1629− Fold step initial extract final
1434+duplicate (Fold step1 initial1 extract1) =
1435+ Fold step initial (\s -> pure $ Fold step1 (pure $ Partial s) extract1)
16301436
16311437 where
16321438
@@ -1634,11 +1440,6 @@ duplicate (Fold step1 initial1 extract1 final1) =
16341440
16351441 step s a = second fromPure <$> step1 s a
16361442
1637− -- Scanning may be problematic due to multiple finalizations.
1638− extract = error "duplicate: scanning may be problematic"
1639−
1640− final s = pure $ Fold step1 (pure $ Partial s) extract1 final1
1641−
16421443-- If there were a finalize/flushing action in the stream type that would be
16431444-- equivalent to running initialize in Fold. But we do not have a flushing
16441445-- action in streams.
@@ -1650,9 +1451,9 @@ duplicate (Fold step1 initial1 extract1 final1) =
16501451-- /Pre-release/
16511452{-# INLINE reduce #-}
16521453reduce :: Monad m => Fold m a b -> m (Fold m a b)
1653−reduce (Fold step initial extract final) = do
1454+reduce (Fold step initial extract) = do
16541455 i <- initial
1655− return $ Fold step (return i) extract final
1456+ return $ Fold step (return i) extract
16561457
16571458-- This is the dual of Stream @cons@.
16581459
@@ -1662,8 +1463,7 @@ reduce (Fold step initial extract final) = do
16621463-- /Pre-release/
16631464{-# INLINE snoclM #-}
16641465snoclM :: Monad m => Fold m a b -> m a -> Fold m a b
1665−snoclM (Fold fstep finitial fextract ffinal) action =
1666− Fold fstep initial fextract ffinal
1466+snoclM (Fold fstep finitial fextract) action = Fold fstep initial fextract
16671467
16681468 where
16691469
@@ -1690,8 +1490,7 @@ snoclM (Fold fstep finitial fextract ffinal) action =
16901490{-# INLINE snocl #-}
16911491snocl :: Monad m => Fold m a b -> a -> Fold m a b
16921492-- snocl f = snoclM f . return
1693−snocl (Fold fstep finitial fextract ffinal) a =
1694− Fold fstep initial fextract ffinal
1493+snocl (Fold fstep finitial fextract) a = Fold fstep initial fextract
16951494
16961495 where
16971496
@@ -1711,12 +1510,12 @@ snocl (Fold fstep finitial fextract ffinal) a =
17111510-- /Pre-release/
17121511{-# INLINE snocM #-}
17131512snocM :: Monad m => Fold m a b -> m a -> m (Fold m a b)
1714−snocM (Fold step initial extract final) action = do
1513+snocM (Fold step initial extract) action = do
17151514 res <- initial
17161515 r <- case res of
17171516 Partial fs -> action >>= step fs
17181517 Done _ -> return res
1719− return $ Fold step (return r) extract final
1518+ return $ Fold step (return r) extract
17201519
17211520-- Definitions:
17221521--
@@ -1735,12 +1534,12 @@ snocM (Fold step initial extract final) action = do
17351534-- /Pre-release/
17361535{-# INLINE snoc #-}
17371536snoc :: Monad m => Fold m a b -> a -> m (Fold m a b)
1738−snoc (Fold step initial extract final) a = do
1537+snoc (Fold step initial extract) a = do
17391538 res <- initial
17401539 r <- case res of
17411540 Partial fs -> step fs a
17421541 Done _ -> return res
1743− return $ Fold step (return r) extract final
1542+ return $ Fold step (return r) extract
17441543
17451544-- | Append a singleton value to the fold.
17461545--
@@ -1768,7 +1567,7 @@ addOne = flip snoc
17681567-- /Pre-release/
17691568{-# INLINE extractM #-}
17701569extractM :: Monad m => Fold m a b -> m b
1771−extractM (Fold _ initial extract _) = do
1570+extractM (Fold _ initial extract) = do
17721571 res <- initial
17731572 case res of
17741573 Partial fs -> extract fs
@@ -1777,15 +1576,14 @@ extractM (Fold _ initial extract _) = do
17771576-- | Close a fold so that it does not accept any more input.
17781577{-# INLINE close #-}
17791578close :: Monad m => Fold m a b -> Fold m a b
1780−close (Fold _ initial1 _ final1) =
1781− Fold undefined initial undefined undefined
1579+close (Fold _ initial1 extract1) = Fold undefined initial undefined
17821580
17831581 where
17841582
17851583 initial = do
17861584 res <- initial1
17871585 case res of
1788− Partial s -> Done <$> final1 s
1586+ Partial s -> Done <$> extract1 s
17891587 Done b -> return $ Done b
17901588
17911589-- Corresponds to the null check for streams.
@@ -1795,7 +1593,7 @@ close (Fold _ initial1 _ final1) =
17951593-- /Pre-release/
17961594{-# INLINE isClosed #-}
17971595isClosed :: Monad m => Fold m a b -> m Bool
1798−isClosed (Fold _ initial _ _) = do
1596+isClosed (Fold _ initial _) = do
17991597 res <- initial
18001598 return $ case res of
18011599 Partial _ -> False
@@ -1829,10 +1627,8 @@ data ManyState s1 s2
18291627--
18301628{-# INLINE many #-}
18311629many :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1832−many
1833− (Fold sstep sinitial sextract sfinal)
1834− (Fold cstep cinitial cextract cfinal) =
1835− Fold step initial extract final
1630+many (Fold sstep sinitial sextract) (Fold cstep cinitial cextract) =
1631+ Fold step initial extract
18361632
18371633 where
18381634
@@ -1881,13 +1677,6 @@ many
18811677 Partial s -> cextract s
18821678 Done b -> return b
18831679
1884− final (ManyFirst ss cs) = sfinal ss *> cfinal cs
1885− final (ManyLoop ss cs) = do
1886− cres <- sfinal ss >>= cstep cs
1887− case cres of
1888− Partial s -> cfinal s
1889− Done b -> return b
1890−
18911680-- | Like many, but the "first" fold emits an output at the end even if no
18921681-- input is received.
18931682--
@@ -1897,10 +1686,8 @@ many
18971686--
18981687{-# INLINE manyPost #-}
18991688manyPost :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
1900−manyPost
1901− (Fold sstep sinitial sextract sfinal)
1902− (Fold cstep cinitial cextract cfinal) =
1903− Fold step initial extract final
1689+manyPost (Fold sstep sinitial sextract) (Fold cstep cinitial cextract) =
1690+ Fold step initial extract
19041691
19051692 where
19061693
@@ -1936,12 +1723,6 @@ manyPost
19361723 Partial s -> cextract s
19371724 Done b -> return b
19381725
1939− final (Tuple' ss cs) = do
1940− cres <- sfinal ss >>= cstep cs
1941− case cres of
1942− Partial s -> cfinal s
1943− Done b -> return b
1944−
19451726-- | @groupsOf n split collect@ repeatedly applies the @split@ fold to chunks
19461727-- of @n@ items in the input stream and supplies the result to the @collect@
19471728-- fold.
@@ -1970,10 +1751,7 @@ groupsOf n split = many (take n split)
19701751--
19711752{-# INLINE refoldMany #-}
19721753refoldMany :: Monad m => Fold m a b -> Refold m x b c -> Refold m x a c
1973−refoldMany
1974− (Fold sstep sinitial sextract _sfinal)
1975− -- XXX We will need a "final" in refold as well
1976− (Refold cstep cinject cextract) =
1754+refoldMany (Fold sstep sinitial sextract) (Refold cstep cinject cextract) =
19771755 Refold step inject extract
19781756
19791757 where
@@ -2024,9 +1802,7 @@ data ConsumeManyState x cs ss = ConsumeMany x cs (Either ss ss)
20241802-- /Internal/
20251803{-# INLINE refoldMany1 #-}
20261804refoldMany1 :: Monad m => Refold m x a b -> Fold m b c -> Refold m x a c
2027−refoldMany1
2028− (Refold sstep sinject sextract)
2029− (Fold cstep cinitial cextract _cfinal) =
1805+refoldMany1 (Refold sstep sinject sextract) (Fold cstep cinitial cextract) =
20301806 Refold step inject extract
20311807
20321808 where
@@ -2077,7 +1853,7 @@ refoldMany1
20771853{-# INLINE refold #-}
20781854refold :: Monad m => Refold m b a c -> Fold m a b -> Fold m a c
20791855refold (Refold step inject extract) f =
2080− Fold step (extractM f >>= inject) extract extract
1856+ Fold step (extractM f >>= inject) extract
20811857
20821858------------------------------------------------------------------------------
20831859-- morphInner
@@ -2087,8 +1863,8 @@ refold (Refold step inject extract) f =
20871863--
20881864-- /Pre-release/
20891865morphInner :: (forall x. m x -> n x) -> Fold m a b -> Fold n a b
2090−morphInner f (Fold step initial extract final) =
2091− Fold (\x a -> f $ step x a) (f initial) (f . extract) (f . final)
1866+morphInner f (Fold step initial extract) =
1867+ Fold (\x a -> f $ step x a) (f initial) (f . extract)
20921868
20931869-- | Adapt a pure fold to any monad.
20941870--
core/src/Streamly/Internal/Data/Fold/Window.hs+53−59
@@ -36,24 +36,24 @@ module Streamly.Internal.Data.Fold.Window
3636 -- window folds by keeping the second element of the input tuple as
3737 -- @Nothing@.
3838 --
39− windowLmap
39+ lmap
4040 , cumulative
4141
42− , windowRollingMap
43− , windowRollingMapM
42+ , rollingMap
43+ , rollingMapM
4444
4545 -- ** Sums
46− , windowLength
47− , windowSum
48− , windowSumInt
49− , windowPowerSum
50− , windowPowerSumFrac
46+ , length
47+ , sum
48+ , sumInt
49+ , powerSum
50+ , powerSumFrac
5151
5252 -- ** Location
53− , windowMinimum
54− , windowMaximum
55− , windowRange
56− , windowMean
53+ , minimum
54+ , maximum
55+ , range
56+ , mean
5757 )
5858where
5959
@@ -65,18 +65,16 @@ import Streamly.Internal.Data.Fold.Type (Fold(..), Step(..))
6565import Streamly.Internal.Data.Tuple.Strict
6666 (Tuple'(..), Tuple3Fused' (Tuple3Fused'))
6767
68−import Foreign.ForeignPtr.Unsafe (unsafeForeignPtrToPtr)
69−
7068import qualified Streamly.Internal.Data.Fold.Type as Fold
71−import qualified Streamly.Internal.Data.Ring as Ring
69+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
7270
7371import Prelude hiding (length, sum, minimum, maximum)
7472
7573-- $setup
7674-- >>> import Data.Bifunctor(bimap)
7775-- >>> import qualified Streamly.Data.Fold as Fold
78−-- >>> import qualified Streamly.Internal.Data.Fold.Window as Fold
79−-- >>> import qualified Streamly.Internal.Data.Ring as Ring
76+-- >>> import qualified Streamly.Internal.Data.Fold.Window as FoldW
77+-- >>> import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
8078-- >>> import qualified Streamly.Data.Stream as Stream
8179-- >>> import Prelude hiding (length, sum, minimum, maximum)
8280
@@ -89,9 +87,9 @@ import Prelude hiding (length, sum, minimum, maximum)
8987--
9088-- >>> lmap f = Fold.lmap (bimap f (f <$>))
9189--
92−{-# INLINE windowLmap #-}
93−windowLmap :: (c -> a) -> Fold m (a, Maybe a) b -> Fold m (c, Maybe c) b
94−windowLmap f = Fold.lmap (bimap f (f <$>))
90+{-# INLINE lmap #-}
91+lmap :: (c -> a) -> Fold m (a, Maybe a) b -> Fold m (c, Maybe c) b
92+lmap f = Fold.lmap (bimap f (f <$>))
9593
9694-- | Convert an incremental fold to a cumulative fold using the entire input
9795-- stream as a single window.
@@ -107,10 +105,10 @@ cumulative = Fold.lmap (, Nothing)
107105
108106-- | Apply an effectful function on the latest and the oldest element of the
109107-- window.
110−{-# INLINE windowRollingMapM #-}
111−windowRollingMapM :: Monad m =>
108+{-# INLINE rollingMapM #-}
109+rollingMapM :: Monad m =>
112110 (Maybe a -> a -> m (Maybe b)) -> Fold m (a, Maybe a) (Maybe b)
113−windowRollingMapM f = Fold.foldlM' f1 initial
111+rollingMapM f = Fold.foldlM' f1 initial
114112
115113 where
116114
@@ -120,12 +118,12 @@ windowRollingMapM f = Fold.foldlM' f1 initial
120118
121119-- | Apply a pure function on the latest and the oldest element of the window.
122120--
123−-- >>> windowRollingMap f = Fold.windowRollingMapM (\x y -> return $ f x y)
121+-- >>> rollingMap f = FoldW.rollingMapM (\x y -> return $ f x y)
124122--
125−{-# INLINE windowRollingMap #-}
126−windowRollingMap :: Monad m =>
123+{-# INLINE rollingMap #-}
124+rollingMap :: Monad m =>
127125 (Maybe a -> a -> Maybe b) -> Fold m (a, Maybe a) (Maybe b)
128−windowRollingMap f = Fold.foldl' f1 initial
126+rollingMap f = Fold.foldl' f1 initial
129127
130128 where
131129
@@ -148,9 +146,9 @@ windowRollingMap f = Fold.foldl' f1 initial
148146--
149147-- /Internal/
150148--
151−{-# INLINE windowSumInt #-}
152−windowSumInt :: forall m a. (Monad m, Integral a) => Fold m (a, Maybe a) a
153−windowSumInt = Fold step initial extract extract
149+{-# INLINE sumInt #-}
150+sumInt :: forall m a. (Monad m, Integral a) => Fold m (a, Maybe a) a
151+sumInt = Fold step initial extract
154152
155153 where
156154
@@ -182,9 +180,9 @@ windowSumInt = Fold step initial extract extract
182180--
183181-- /Time/: \(\mathcal{O}(n)\)
184182--
185−{-# INLINE windowSum #-}
186−windowSum :: forall m a. (Monad m, Num a) => Fold m (a, Maybe a) a
187−windowSum = Fold step initial extract extract
183+{-# INLINE sum #-}
184+sum :: forall m a. (Monad m, Num a) => Fold m (a, Maybe a) a
185+sum = Fold step initial extract
188186
189187 where
190188
@@ -221,9 +219,9 @@ windowSum = Fold step initial extract extract
221219--
222220-- >>> length = powerSum 0
223221--
224−{-# INLINE windowLength #-}
225−windowLength :: (Monad m, Num b) => Fold m (a, Maybe a) b
226−windowLength = Fold.foldl' step 0
222+{-# INLINE length #-}
223+length :: (Monad m, Num b) => Fold m (a, Maybe a) b
224+length = Fold.foldl' step 0
227225
228226 where
229227
@@ -239,18 +237,18 @@ windowLength = Fold.foldl' step 0
239237-- /Space/: \(\mathcal{O}(1)\)
240238--
241239-- /Time/: \(\mathcal{O}(n)\)
242−{-# INLINE windowPowerSum #-}
243−windowPowerSum :: (Monad m, Num a) => Int -> Fold m (a, Maybe a) a
244−windowPowerSum k = windowLmap (^ k) windowSum
240+{-# INLINE powerSum #-}
241+powerSum :: (Monad m, Num a) => Int -> Fold m (a, Maybe a) a
242+powerSum k = lmap (^ k) sum
245243
246244-- | Like 'powerSum' but powers can be negative or fractional. This is slower
247245-- than 'powerSum' for positive intergal powers.
248246--
249247-- >>> powerSumFrac p = lmap (** p) sum
250248--
251−{-# INLINE windowPowerSumFrac #-}
252−windowPowerSumFrac :: (Monad m, Floating a) => a -> Fold m (a, Maybe a) a
253−windowPowerSumFrac p = windowLmap (** p) windowSum
249+{-# INLINE powerSumFrac #-}
250+powerSumFrac :: (Monad m, Floating a) => a -> Fold m (a, Maybe a) a
251+powerSumFrac p = lmap (** p) sum
254252
255253-------------------------------------------------------------------------------
256254-- Location
@@ -267,9 +265,9 @@ windowPowerSumFrac p = windowLmap (** p) windowSum
267265--
268266-- /Time/: \(\mathcal{O}(n*w)\) where \(w\) is the window size.
269267--
270−{-# INLINE windowRange #-}
271−windowRange :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe (a, a))
272−windowRange n = Fold step initial extract extract
268+{-# INLINE range #-}
269+range :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe (a, a))
270+range n = Fold step initial extract
273271
274272 where
275273
@@ -302,11 +300,7 @@ windowRange n = Fold step initial extract extract
302300 if i == 0
303301 then return Nothing
304302 else do
305− -- Here we use "ringStart" over "ringHead" as "ringHead" will be
306− -- uninitialized if the ring is not full.
307− -- Using "unsafeForeignPtrToPtr" here is safe as we touch the ring
308− -- again in "foldFunc".
309− x <- liftIO $ peek (unsafeForeignPtrToPtr (Ring.ringStart rb))
303+ x <- liftIO $ peek rh
310304 let accum (mn, mx) a = return (min mn a, max mx a)
311305 fmap Just $ foldFunc i rh accum (x, x) rb
312306
@@ -322,9 +316,9 @@ windowRange n = Fold step initial extract extract
322316--
323317-- /Time/: \(\mathcal{O}(n*w)\) where \(w\) is the window size.
324318--
325−{-# INLINE windowMinimum #-}
326−windowMinimum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
327−windowMinimum n = fmap (fmap fst) $ windowRange n
319+{-# INLINE minimum #-}
320+minimum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
321+minimum n = fmap (fmap fst) $ range n
328322
329323-- | The maximum element in a rolling window.
330324--
@@ -335,9 +329,9 @@ windowMinimum n = fmap (fmap fst) $ windowRange n
335329--
336330-- /Time/: \(\mathcal{O}(n*w)\) where \(w\) is the window size.
337331--
338−{-# INLINE windowMaximum #-}
339−windowMaximum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
340−windowMaximum n = fmap (fmap snd) $ windowRange n
332+{-# INLINE maximum #-}
333+maximum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
334+maximum n = fmap (fmap snd) $ range n
341335
342336-- | Arithmetic mean of elements in a sliding window:
343337--
@@ -352,6 +346,6 @@ windowMaximum n = fmap (fmap snd) $ windowRange n
352346-- /Space/: \(\mathcal{O}(1)\)
353347--
354348-- /Time/: \(\mathcal{O}(n)\)
355−{-# INLINE windowMean #-}
356−windowMean :: forall m a. (Monad m, Fractional a) => Fold m (a, Maybe a) a
357−windowMean = Fold.teeWith (/) windowSum windowLength
349+{-# INLINE mean #-}
350+mean :: forall m a. (Monad m, Fractional a) => Fold m (a, Maybe a) a
351+mean = Fold.teeWith (/) sum length
core/src/Streamly/Internal/Data/IORef/Unboxed.hs+20−23
@@ -23,31 +23,34 @@ module Streamly.Internal.Data.IORef.Unboxed
2323 (
2424 IORef
2525
26− -- Construction
26+ -- * Construction
2727 , newIORef
2828
29− -- Write
29+ -- * Write
3030 , writeIORef
3131 , modifyIORef'
3232
33− -- Read
33+ -- * Read
3434 , readIORef
35− , pollIntIORef
35+ , toStreamD
3636 )
3737where
3838
3939#include "inline.hs"
4040
41−import Control.Monad.IO.Class (MonadIO(..))
4241import Data.Proxy (Proxy(..))
43−import Streamly.Internal.Data.MutByteArray.Type (MutByteArray)
44−import Streamly.Internal.Data.Unbox (Unbox(..), sizeOf)
42+import Control.Monad.IO.Class (MonadIO(..))
43+import Streamly.Internal.Data.Unbox
44+ ( MutableByteArray(..)
45+ , Unbox(..)
46+ , newBytes
47+ , sizeOf
48+ )
4549
46−import qualified Streamly.Internal.Data.MutByteArray.Type as MBA
47−import qualified Streamly.Internal.Data.Stream.Type as D
50+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
4851
4952-- | An 'IORef' holds a single 'Unbox'-able value.
50−newtype IORef a = IORef MutByteArray
53+newtype IORef a = IORef MutableByteArray
5154
5255-- | Create a new 'IORef'.
5356--
@@ -55,8 +58,8 @@ newtype IORef a = IORef MutByteArray
5558{-# INLINE newIORef #-}
5659newIORef :: forall a. Unbox a => a -> IO (IORef a)
5760newIORef x = do
58− var <- MBA.new (sizeOf (Proxy :: Proxy a))
59− pokeAt 0 var x
61+ var <- newBytes (sizeOf (Proxy :: Proxy a))
62+ pokeByteIndex 0 var x
6063 return $ IORef var
6164
6265-- | Write a value to an 'IORef'.
@@ -64,14 +67,14 @@ newIORef x = do
6467-- /Pre-release/
6568{-# INLINE writeIORef #-}
6669writeIORef :: Unbox a => IORef a -> a -> IO ()
67−writeIORef (IORef var) = pokeAt 0 var
70+writeIORef (IORef var) = pokeByteIndex 0 var
6871
6972-- | Read a value from an 'IORef'.
7073--
7174-- /Pre-release/
7275{-# INLINE readIORef #-}
7376readIORef :: Unbox a => IORef a -> IO a
74−readIORef (IORef var) = peekAt 0 var
77+readIORef (IORef var) = peekByteIndex 0 var
7578
7679-- | Modify the value of an 'IORef' using a function with strict application.
7780--
@@ -84,16 +87,10 @@ modifyIORef' var g = do
8487
8588-- | Generate a stream by continuously reading the IORef.
8689--
87−-- This operation reads the IORef without any synchronization. It can be
88−-- assumed to be atomic because the IORef (MutableByteArray) is always aligned
89−-- to Int boundaries, we are assuming that compiler uses single instructions to
90−-- access the memory. It may read stale values though until caches are
91−-- synchronised in a multiprocessor architecture.
92−--
9390-- /Pre-release/
94−{-# INLINE_NORMAL pollIntIORef #-}
95−pollIntIORef :: (MonadIO m, Unbox a) => IORef a -> D.Stream m a
96−pollIntIORef var = D.Stream step ()
91+{-# INLINE_NORMAL toStreamD #-}
92+toStreamD :: (MonadIO m, Unbox a) => IORef a -> D.Stream m a
93+toStreamD var = D.Stream step ()
9794
9895 where
9996
core/src/Streamly/Internal/Data/IsMap.hs−8
@@ -1,7 +1,3 @@
1−{-# LANGUAGE TypeFamilies #-}
2−-- Must come after TypeFamilies, otherwise it is re-enabled.
3−-- MonoLocalBinds enabled by TypeFamilies causes perf regressions in general.
4−{-# LANGUAGE NoMonoLocalBinds #-}
51-- |
62-- Module : Streamly.Internal.Data.IsMap
73-- Copyright : (c) 2022 Composewell Technologies
@@ -32,8 +28,6 @@ class IsMap f where
3228 mapDelete :: Key f -> f a -> f a
3329 mapUnion :: f a -> f a -> f a
3430 mapNull :: f a -> Bool
35− mapTraverseWithKey ::
36− Applicative t => (Key f -> a -> t b) -> f a -> t (f b)
3731
3832instance Ord k => IsMap (Map k) where
3933 type Key (Map k) = k
@@ -45,7 +39,6 @@ instance Ord k => IsMap (Map k) where
4539 mapDelete = Map.delete
4640 mapUnion = Map.union
4741 mapNull = Map.null
48− mapTraverseWithKey = Map.traverseWithKey
4942
5043instance IsMap IntMap.IntMap where
5144 type Key IntMap.IntMap = Int
@@ -57,4 +50,3 @@ instance IsMap IntMap.IntMap where
5750 mapDelete = IntMap.delete
5851 mapUnion = IntMap.union
5952 mapNull = IntMap.null
60− mapTraverseWithKey = IntMap.traverseWithKey
core/src/Streamly/Internal/Data/List.hs+5−5
@@ -66,19 +66,19 @@ where
6666import Control.Arrow (second)
6767import Data.Functor.Identity (Identity, runIdentity)
6868import GHC.Exts (IsList(..), IsString(..))
69−import Streamly.Internal.Data.Stream.Type
69+import Streamly.Internal.Data.Stream.StreamD.Type
7070 ( CrossStream
7171 , mkCross
7272 , unCross
7373 )
74−import Streamly.Internal.Data.Stream.Type (Stream)
74+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
7575import Text.Read
7676 ( Lexeme(Ident), lexP, parens, prec, readPrec, readListPrec
7777 , readListPrecDefault)
7878
79−import qualified Streamly.Internal.Data.StreamK.Type as K
80−import qualified Streamly.Internal.Data.Stream.Type as Stream
81−import qualified Streamly.Internal.Data.Stream.Generate as Stream
79+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
80+import qualified Streamly.Internal.Data.Stream.StreamD.Type as Stream
81+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as Stream
8282
8383-- XXX Rename to PureStream.
8484
core/src/Streamly/Internal/Data/MutArray.hs+33−154
@@ -8,200 +8,79 @@
88--
99module Streamly.Internal.Data.MutArray
1010 (
11− -- * MutArray.Type module
1211 module Streamly.Internal.Data.MutArray.Type
13− -- * MutArray module
14− , sliceIndexerFromLen
15− , slicerFromLen
16− , compactLE
17− , pinnedCompactLE
18− , compactOnByte
19− , compactOnByteSuffix
20− -- * Unboxed IORef
21− , module Streamly.Internal.Data.IORef.Unboxed
22−
23− -- * Deprecated
12+ , splitOn
2413 , genSlicesFromLen
2514 , getSlicesFromLen
15+ , fromStream
2616 )
2717where
2818
2919#include "inline.hs"
3020
3121import Control.Monad.IO.Class (MonadIO(..))
32−import Data.Word (Word8)
33−import Streamly.Internal.Data.MutByteArray.Type (PinnedState(..))
34−import Streamly.Internal.Data.Stream.Type (Stream)
3522import Streamly.Internal.Data.Unbox (Unbox)
23+import Streamly.Internal.Data.Stream (Stream)
3624import Streamly.Internal.Data.Unfold.Type (Unfold(..))
3725
38−import qualified Streamly.Internal.Data.Stream.Type as Stream
39−import qualified Streamly.Internal.Data.Stream.Nesting as Stream
40−-- import qualified Streamly.Internal.Data.Stream.Transform as Stream
26+import qualified Streamly.Internal.Data.Stream as D
4127import qualified Streamly.Internal.Data.Unfold as Unfold
4228
4329import Prelude hiding (foldr, length, read, splitAt)
4430import Streamly.Internal.Data.MutArray.Type
45−import Streamly.Internal.Data.IORef.Unboxed
31+
32+-- | Split the array into a stream of slices using a predicate. The element
33+-- matching the predicate is dropped.
34+--
35+-- /Pre-release/
36+{-# INLINE splitOn #-}
37+splitOn :: (MonadIO m, Unbox a) =>
38+ (a -> Bool) -> MutArray a -> Stream m (MutArray a)
39+splitOn predicate arr =
40+ fmap (\(i, len) -> getSliceUnsafe i len arr)
41+ $ D.sliceOnSuffix predicate (read arr)
4642
4743-- | Generate a stream of array slice descriptors ((index, len)) of specified
4844-- length from an array, starting from the supplied array index. The last slice
4945-- may be shorter than the requested length depending on the array length.
5046--
5147-- /Pre-release/
52−{-# INLINE sliceIndexerFromLen #-}
53−sliceIndexerFromLen :: forall m a. (Monad m, Unbox a)
48+{-# INLINE genSlicesFromLen #-}
49+genSlicesFromLen :: forall m a. (Monad m, Unbox a)
5450 => Int -- ^ from index
5551 -> Int -- ^ length of the slice
5652 -> Unfold m (MutArray a) (Int, Int)
57−sliceIndexerFromLen from len =
53+genSlicesFromLen from len =
5854 let fromThenTo n = (from, from + len, n - 1)
5955 mkSlice n i = return (i, min len (n - i))
6056 in Unfold.lmap length
6157 $ Unfold.mapM2 mkSlice
6258 $ Unfold.lmap fromThenTo Unfold.enumerateFromThenTo
6359
64−{-# DEPRECATED genSlicesFromLen "Please use sliceIndexerFromLen instead." #-}
65−genSlicesFromLen :: forall m a. (Monad m, Unbox a)
66− => Int -- ^ from index
67− -> Int -- ^ length of the slice
68− -> Unfold m (MutArray a) (Int, Int)
69−genSlicesFromLen = sliceIndexerFromLen
70−
7160-- | Generate a stream of slices of specified length from an array, starting
7261-- from the supplied array index. The last slice may be shorter than the
7362-- requested length depending on the array length.
7463--
7564-- /Pre-release/
76−{-# INLINE slicerFromLen #-}
77−slicerFromLen :: forall m a. (Monad m, Unbox a)
78− => Int -- ^ from index
79− -> Int -- ^ length of the slice
80− -> Unfold m (MutArray a) (MutArray a)
81−slicerFromLen from len =
82− let mkSlice arr (i, n) = return $ getSliceUnsafe i n arr
83− in Unfold.mapM2 mkSlice (sliceIndexerFromLen from len)
84−
85−{-# DEPRECATED getSlicesFromLen "Please use slicerFromLen instead." #-}
65+{-# INLINE getSlicesFromLen #-}
8666getSlicesFromLen :: forall m a. (Monad m, Unbox a)
8767 => Int -- ^ from index
8868 -> Int -- ^ length of the slice
8969 -> Unfold m (MutArray a) (MutArray a)
90−getSlicesFromLen = slicerFromLen
91−
92−-------------------------------------------------------------------------------
93−-- Compacting Streams of Arrays
94−-------------------------------------------------------------------------------
70+getSlicesFromLen from len =
71+ let mkSlice arr (i, n) = return $ getSliceUnsafe i n arr
72+ in Unfold.mapM2 mkSlice (genSlicesFromLen from len)
9573
96−-- | @compactLE maxElems@ coalesces adjacent arrays in the input stream
97−-- only if the combined size would be less than or equal to @maxElems@
98−-- elements. Note that it won't split an array if the original array is already
99−-- larger than maxElems.
100−--
101−-- @maxElems@ must be greater than 0.
74+-- | Create an 'Array' from a stream. This is useful when we want to create a
75+-- single array from a stream of unknown size. 'writeN' is at least twice
76+-- as efficient when the size is already known.
10277--
103−-- Generates unpinned arrays irrespective of the pinning status of input
104−-- arrays.
105−{-# INLINE compactLE #-}
106−compactLE :: (MonadIO m, Unbox a) =>
107− Int -> Stream m (MutArray a) -> Stream m (MutArray a)
108−-- XXX compactLE can be moved to MutArray/Type if we are not using the parser
109−-- to implement it.
110−compactLE = compactLeAs Unpinned
111−-- The parser version turns out to be a little bit slower.
112−-- compactLE n = Stream.catRights . Stream.parseManyD (pCompactLE n)
113−
114−-- | Pinned version of 'compactLE'.
115−{-# INLINE pinnedCompactLE #-}
116−pinnedCompactLE :: forall m a. (MonadIO m, Unbox a)
117− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
118−pinnedCompactLE = compactLeAs Pinned
119−-- pinnedCompactLE n = Stream.catRights . Stream.parseManyD (pPinnedCompactLE n)
120−
121−data SplitState s arr
122− = Initial s
123− | Buffering s arr
124− | Splitting s arr
125− | Yielding arr (SplitState s arr)
126− | Finishing
127−
128−-- | Split a stream of arrays on a given separator byte, dropping the separator
129−-- and coalescing all the arrays between two separators into a single array.
78+-- Note that if the input stream is too large memory allocation for the array
79+-- may fail. When the stream size is not known, `chunksOf` followed by
80+-- processing of indvidual arrays in the resulting stream should be preferred.
13081--
131−{-# INLINE_NORMAL _compactOnByteCustom #-}
132−_compactOnByteCustom
133− :: MonadIO m
134− => Word8
135− -> Stream m (MutArray Word8)
136− -> Stream m (MutArray Word8)
137−_compactOnByteCustom byte (Stream.Stream step state) =
138− Stream.Stream step' (Initial state)
139−
140− where
141−
142− {-# INLINE_LATE step' #-}
143− step' gst (Initial st) = do
144− r <- step gst st
145− case r of
146− Stream.Yield arr s -> do
147− (arr1, marr2) <- breakOn byte arr
148− return $ case marr2 of
149− Nothing -> Stream.Skip (Buffering s arr1)
150− Just arr2 -> Stream.Skip (Yielding arr1 (Splitting s arr2))
151− Stream.Skip s -> return $ Stream.Skip (Initial s)
152− Stream.Stop -> return Stream.Stop
153−
154− step' gst (Buffering st buf) = do
155− r <- step gst st
156− case r of
157− Stream.Yield arr s -> do
158− (arr1, marr2) <- breakOn byte arr
159− -- XXX Use spliceExp instead and then rightSize?
160− buf1 <- splice buf arr1
161− return $ case marr2 of
162− Nothing -> Stream.Skip (Buffering s buf1)
163− Just x -> Stream.Skip (Yielding buf1 (Splitting s x))
164− Stream.Skip s -> return $ Stream.Skip (Buffering s buf)
165− Stream.Stop -> return $
166− if byteLength buf == 0
167− then Stream.Stop
168− else Stream.Skip (Yielding buf Finishing)
169−
170− step' _ (Splitting st buf) = do
171− (arr1, marr2) <- breakOn byte buf
172− return $ case marr2 of
173− Nothing -> Stream.Skip $ Buffering st arr1
174− Just arr2 -> Stream.Skip $ Yielding arr1 (Splitting st arr2)
175−
176− step' _ (Yielding arr next) = return $ Stream.Yield arr next
177− step' _ Finishing = return Stream.Stop
178−
179−-- XXX implement predicate based version of this
180−-- XXX Naming of predicate based vs custom version
181−
182−-- | Split a stream of arrays on a given separator byte, dropping the separator
183−-- and coalescing all the arrays between two separators into a single array.
184−--
185−{-# INLINE compactOnByte #-}
186−compactOnByte
187− :: (MonadIO m)
188− => Word8
189− -> Stream m (MutArray Word8)
190− -> Stream m (MutArray Word8)
191−-- XXX compare perf of custom vs idiomatic version
192−-- compactOnByte = _compactOnByteCustom
193−-- XXX use spliceExp and rightSize?
194−compactOnByte byte = Stream.splitInnerBy (breakOn byte) splice
195−
196−-- | Like 'compactOnByte' considers the separator in suffix position instead of
197−-- infix position.
198−{-# INLINE compactOnByteSuffix #-}
199−compactOnByteSuffix
200− :: (MonadIO m)
201− => Word8
202− -> Stream m (MutArray Word8)
203− -> Stream m (MutArray Word8)
204−compactOnByteSuffix byte =
205− -- XXX use spliceExp and rightSize?
206− Stream.splitInnerBySuffix
207− (\arr -> byteLength arr == 0) (breakOn byte) splice
82+-- /Pre-release/
83+{-# INLINE fromStream #-}
84+fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
85+fromStream = fromStreamD
86+-- fromStream (Stream m) = P.fold write m
core/src/Streamly/Internal/Data/MutArray/Generic.hs+49−172
@@ -19,14 +19,14 @@ module Streamly.Internal.Data.MutArray.Generic
1919 , nil
2020
2121 -- *** Uninitialized Arrays
22− , emptyOf
22+ , new
2323 -- , newArrayWith
2424
2525 -- *** From streams
26− , unsafeCreateOf
27− , createOf
28− , createWith -- createOfMin/createMin/createGE?
29− , create
26+ , writeNUnsafe
27+ , writeN
28+ , writeWith
29+ , write
3030 , fromStreamN
3131 , fromStream
3232 , fromPureStream
@@ -101,7 +101,6 @@ module Streamly.Internal.Data.MutArray.Generic
101101 -- ** Random reads
102102 , getIndex
103103 , getIndexUnsafe
104− , getIndexUnsafeWith
105104 -- , getIndices
106105 -- , getFromThenTo
107106 -- , getIndexRev
@@ -133,7 +132,7 @@ module Streamly.Internal.Data.MutArray.Generic
133132 -- multidimensional array representations.
134133
135134 -- ** Construct from streams
136− , chunksOf
135+ -- , chunksOf
137136 -- , arrayStreamKFromStreamD
138137 -- , writeChunks
139138
@@ -160,13 +159,6 @@ module Streamly.Internal.Data.MutArray.Generic
160159 -- , appendSliceFrom
161160
162161 , clone
163−
164− -- * Deprecated
165− , new
166− , writeNUnsafe
167− , writeN
168− , writeWith
169− , write
170162 )
171163where
172164
@@ -189,15 +181,14 @@ import GHC.Int (Int(..))
189181import Streamly.Internal.Data.Fold.Type (Fold(..))
190182import Streamly.Internal.Data.Producer.Type (Producer (..))
191183import Streamly.Internal.Data.Unfold.Type (Unfold(..))
192−import Streamly.Internal.Data.Stream.Type (Stream)
193−import Streamly.Internal.Data.SVar.Type (adaptState)
184+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
194185
195186import qualified Streamly.Internal.Data.Fold.Type as FL
196187import qualified Streamly.Internal.Data.Producer as Producer
197−import qualified Streamly.Internal.Data.Stream.Type as D
198−import qualified Streamly.Internal.Data.Stream.Generate as D
199−import qualified Streamly.Internal.Data.Stream.Lift as D
200−import qualified Streamly.Internal.Data.StreamK.Type as K
188+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
189+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
190+import qualified Streamly.Internal.Data.Stream.StreamD.Lift as D
191+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
201192
202193import Prelude hiding (read, length)
203194
@@ -243,13 +234,13 @@ bottomElement =
243234-- XXX Would be nice if GHC can provide something like newUninitializedArray# so
244235-- that we do not have to write undefined or error in the whole array.
245236
246−-- | @emptyOf count@ allocates a zero length array that can be extended to hold
237+-- | @new count@ allocates a zero length array that can be extended to hold
247238-- up to 'count' items without reallocating.
248239--
249240-- /Pre-release/
250−{-# INLINE emptyOf #-}
251−emptyOf :: MonadIO m => Int -> m (MutArray a)
252−emptyOf n@(I# n#) =
241+{-# INLINE new #-}
242+new :: MonadIO m => Int -> m (MutArray a)
243+new n@(I# n#) =
253244 liftIO
254245 $ IO
255246 $ \s# ->
@@ -258,12 +249,6 @@ emptyOf n@(I# n#) =
258249 let ma = MutArray arr# 0 0 n
259250 in (# s1#, ma #)
260251
261−-- XXX Deprecate in major
262−-- {-# DEPRECATED new "Please use emptyOf instead." #-}
263−{-# INLINE new #-}
264−new :: MonadIO m => Int -> m (MutArray a)
265−new = emptyOf
266−
267252-- XXX This could be pure?
268253
269254-- |
@@ -278,21 +263,6 @@ nil = new 0
278263-- Random writes
279264-------------------------------------------------------------------------------
280265
281−-- | Write the given element to the given index of the 'MutableArray#'. Does not
282−-- check if the index is out of bounds of the array.
283−--
284−-- /Pre-release/
285−{-# INLINE putIndexUnsafeWith #-}
286−putIndexUnsafeWith :: MonadIO m => Int -> MutableArray# RealWorld a -> a -> m ()
287−putIndexUnsafeWith n _arrContents# x =
288− liftIO
289− $ IO
290− $ \s# ->
291− case n of
292− I# n# ->
293− let s1# = writeArray# _arrContents# n# x s#
294− in (# s1#, () #)
295−
296266-- | Write the given element to the given index of the array. Does not check if
297267-- the index is out of bounds of the array.
298268--
@@ -301,7 +271,13 @@ putIndexUnsafeWith n _arrContents# x =
301271putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
302272putIndexUnsafe i MutArray {..} x =
303273 assert (i >= 0 && i < arrLen)
304− putIndexUnsafeWith (i + arrStart) arrContents# x
274+ (liftIO
275+ $ IO
276+ $ \s# ->
277+ case i + arrStart of
278+ I# n# ->
279+ let s1# = writeArray# arrContents# n# x s#
280+ in (# s1#, () #))
305281
306282invalidIndex :: String -> Int -> a
307283invalidIndex label i =
@@ -331,7 +307,7 @@ putIndices arr = FL.foldlM' step (return ())
331307
332308 where
333309
334− step () (i, x) = putIndex i arr x
310+ step () (i, x) = liftIO (putIndex i arr x)
335311
336312-- | Modify a given index of an array using a modifier function without checking
337313-- the bounds.
@@ -480,25 +456,17 @@ uninit arr@MutArray{..} len =
480456-- Random reads
481457-------------------------------------------------------------------------------
482458
483−-- | Return the element at the specified index without checking the bounds from
484−-- a @MutableArray# RealWorld@.
485−--
486−-- Unsafe because it does not check the bounds of the array.
487−{-# INLINE getIndexUnsafeWith #-}
488−getIndexUnsafeWith :: MonadIO m => MutableArray# RealWorld a -> Int -> m a
489−getIndexUnsafeWith _arrContents# n =
490− liftIO
491− $ IO
492− $ \s# ->
493− let !(I# i#) = n
494− in readArray# _arrContents# i# s#
495−
496459-- | Return the element at the specified index without checking the bounds.
497460--
498461-- Unsafe because it does not check the bounds of the array.
499462{-# INLINE_NORMAL getIndexUnsafe #-}
500463getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
501−getIndexUnsafe n MutArray {..} = getIndexUnsafeWith arrContents# (n + arrStart)
464+getIndexUnsafe n MutArray {..} =
465+ liftIO
466+ $ IO
467+ $ \s# ->
468+ let !(I# i#) = arrStart + n
469+ in readArray# arrContents# i# s#
502470
503471-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
504472--
@@ -602,16 +570,16 @@ readRev arr@MutArray{..} =
602570arrayChunkSize :: Int
603571arrayChunkSize = 1024
604572
605−-- | Like 'createOf' but does not check the array bounds when writing. The fold
573+-- | Like 'writeN' but does not check the array bounds when writing. The fold
606574-- driver must not call the step function more than 'n' times otherwise it will
607575-- corrupt the memory and crash. This function exists mainly because any
608576-- conditional in the step function blocks fusion causing 10x performance
609577-- slowdown.
610578--
611579-- /Pre-release/
612−{-# INLINE_NORMAL unsafeCreateOf #-}
613−unsafeCreateOf :: MonadIO m => Int -> Fold m a (MutArray a)
614−unsafeCreateOf n = Fold step initial return return
580+{-# INLINE_NORMAL writeNUnsafe #-}
581+writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
582+writeNUnsafe n = Fold step initial return
615583
616584 where
617585
@@ -619,81 +587,59 @@ unsafeCreateOf n = Fold step initial return return
619587
620588 step arr x = FL.Partial <$> snocUnsafe arr x
621589
622−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
623−{-# INLINE writeNUnsafe #-}
624−writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
625−writeNUnsafe = unsafeCreateOf
626−
627−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
590+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
628591-- 'Array'.
629592--
630−-- >>> createOf n = Fold.take n (MutArray.unsafeCreateOf n)
593+-- >>> writeN n = Fold.take n (MutArray.writeNUnsafe n)
631594--
632595-- /Pre-release/
633−{-# INLINE_NORMAL createOf #-}
634−createOf :: MonadIO m => Int -> Fold m a (MutArray a)
635−createOf n = FL.take n $ unsafeCreateOf n
636−
637−-- XXX Deprecate in major
638−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
639−{-# INLINE writeN #-}
596+{-# INLINE_NORMAL writeN #-}
640597writeN :: MonadIO m => Int -> Fold m a (MutArray a)
641−writeN = createOf
598+writeN n = FL.take n $ writeNUnsafe n
642599
643600-- >>> f n = MutArray.writeAppendWith (* 2) (MutArray.pinnedNew n)
644601-- >>> writeWith n = Fold.rmapM MutArray.rightSize (f n)
645602-- >>> writeWith n = Fold.rmapM MutArray.fromArrayStreamK (MutArray.writeChunks n)
646603
647−-- | @createWith minCount@ folds the whole input to a single array. The array
604+-- | @writeWith minCount@ folds the whole input to a single array. The array
648605-- starts at a size big enough to hold minCount elements, the size is doubled
649606-- every time the array needs to be grown.
650607--
651608-- /Caution! Do not use this on infinite streams./
652609--
653610-- /Pre-release/
654−{-# INLINE_NORMAL createWith #-}
655−createWith :: MonadIO m => Int -> Fold m a (MutArray a)
611+{-# INLINE_NORMAL writeWith #-}
612+writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
656613-- writeWith n = FL.rmapM rightSize $ writeAppendWith (* 2) (pinnedNew n)
657−createWith elemCount = FL.rmapM extract $ FL.foldlM' step initial
614+writeWith elemCount = FL.rmapM extract $ FL.foldlM' step initial
658615
659616 where
660617
661618 initial = do
662− when (elemCount < 0) $ error "createWith: elemCount is negative"
663− new elemCount
619+ when (elemCount < 0) $ error "writeWith: elemCount is negative"
620+ liftIO $ new elemCount
664621
665622 step arr@(MutArray _ start end bound) x
666623 | end == bound = do
667624 let oldSize = end - start
668625 newSize = max (oldSize * 2) 1
669− arr1 <- realloc newSize arr
626+ arr1 <- liftIO $ realloc newSize arr
670627 snocUnsafe arr1 x
671628 step arr x = snocUnsafe arr x
672629
673− -- extract = rightSize
630+ -- extract = liftIO . rightSize
674631 extract = return
675632
676−{-# DEPRECATED writeWith "Please use createWith instead." #-}
677−{-# INLINE writeWith #-}
678−writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
679−writeWith = createWith
680−
681633-- | Fold the whole input to a single array.
682634--
683−-- Same as 'createWith' using an initial array size of 'arrayChunkSize' bytes
635+-- Same as 'writeWith' using an initial array size of 'arrayChunkSize' bytes
684636-- rounded up to the element size.
685637--
686638-- /Caution! Do not use this on infinite streams./
687639--
688−{-# INLINE create #-}
689−create :: MonadIO m => Fold m a (MutArray a)
690−create = writeWith arrayChunkSize
691−
692−-- XXX Deprecate in major
693−-- {-# DEPRECATED write "Please use create instead." #-}
694640{-# INLINE write #-}
695641write :: MonadIO m => Fold m a (MutArray a)
696−write = create
642+write = writeWith arrayChunkSize
697643
698644-- | Create a 'MutArray' from the first @n@ elements of a stream. The
699645-- array is allocated to size @n@, if the stream terminates before @n@
@@ -718,76 +664,7 @@ fromList xs = fromStream $ D.fromList xs
718664{-# INLINABLE fromPureStream #-}
719665fromPureStream :: MonadIO m => Stream Identity a -> m (MutArray a)
720666fromPureStream xs =
721− D.fold write $ D.morphInner (return . runIdentity) xs
722−
723−-------------------------------------------------------------------------------
724−-- Chunking
725−-------------------------------------------------------------------------------
726−
727−data GroupState s a start end bound
728− = GroupStart s
729− | GroupBuffer s (MutableArray# RealWorld a) start end bound
730− | GroupYield
731− (MutableArray# RealWorld a)
732− start
733− end
734− bound
735− (GroupState s a start end bound)
736− | GroupFinish
737−
738−-- | @chunksOf n stream@ groups the input stream into a stream of
739−-- arrays of size n.
740−--
741−-- @chunksOf n = foldMany (MutArray.writeN n)@
742−--
743−-- /Pre-release/
744−{-# INLINE_NORMAL chunksOf #-}
745−chunksOf :: forall m a. MonadIO m
746− => Int -> D.Stream m a -> D.Stream m (MutArray a)
747−-- XXX the idiomatic implementation leads to large regression in the D.reverse'
748−-- benchmark. It seems it has difficulty producing optimized code when
749−-- converting to StreamK. Investigate GHC optimizations.
750−-- chunksOf n = D.foldMany (writeN n)
751−chunksOf n (D.Stream step state) =
752− D.Stream step' (GroupStart state)
753−
754− where
755−
756− -- start is always 0
757− -- end and len are always equal
758−
759− {-# INLINE_LATE step' #-}
760− step' _ (GroupStart st) = do
761− when (n <= 0) $
762− -- XXX we can pass the module string from the higher level API
763− error $ "Streamly.Internal.Data.Array.Generic.Mut.Type.chunksOf: "
764− ++ "the size of arrays [" ++ show n
765− ++ "] must be a natural number"
766− (MutArray contents start end bound :: MutArray a) <- new n
767− return $ D.Skip (GroupBuffer st contents start end bound)
768−
769− step' gst (GroupBuffer st contents start end bound) = do
770− r <- step (adaptState gst) st
771− case r of
772− D.Yield x s -> do
773− putIndexUnsafeWith end contents x
774− let end1 = end + 1
775− return $
776− if end1 >= bound
777− then D.Skip
778− (GroupYield
779− contents start end1 bound (GroupStart s))
780− else D.Skip (GroupBuffer s contents start end1 bound)
781− D.Skip s ->
782− return $ D.Skip (GroupBuffer s contents start end bound)
783− D.Stop ->
784− return
785− $ D.Skip (GroupYield contents start end bound GroupFinish)
786−
787− step' _ (GroupYield contents start end bound next) =
788− return $ D.Yield (MutArray contents start end bound) next
789−
790− step' _ GroupFinish = return D.Stop
667+ liftIO $ D.fold write $ D.morphInner (return . runIdentity) xs
791668
792669-------------------------------------------------------------------------------
793670-- Unfolds
@@ -850,7 +727,7 @@ putSliceUnsafe src srcStart dst dstStart len = liftIO $ do
850727
851728{-# INLINE clone #-}
852729clone :: MonadIO m => MutArray a -> m (MutArray a)
853−clone src = do
730+clone src = liftIO $ do
854731 let len = arrLen src
855732 dst <- new len
856733 putSliceUnsafe src 0 dst 0 len
core/src/Streamly/Internal/Data/MutArray/Stream.hsdeleted−145
@@ -1,145 +0,0 @@
1−-- |
2−-- Module : Streamly.Internal.Data.MutArray.Stream
3−-- Copyright : (c) 2019 Composewell Technologies
4−-- License : BSD3-3-Clause
5−-- Maintainer : streamly@composewell.com
6−-- Stability : experimental
7−-- Portability : GHC
8−--
9−-- Combinators to efficiently manipulate streams of mutable arrays.
10−--
11−-- We can either push these in the MutArray module with a "chunks" prefix or
12−-- keep this as a separate module and release it.
13−--
14−module Streamly.Internal.Data.MutArray.Stream
15−{-# DEPRECATED "Please use \"Streamly.Internal.Data.MutArray\" instead." #-}
16− (
17− -- * Generation
18− MArray.chunksOf
19− , MArray.pinnedChunksOf
20− , MArray.writeChunks -- chunksWrite?
21− , MArray.splitOn -- chunksSplitOn
22−
23− -- * Compaction
24− , packArraysChunksOf
25− , MArray.SpliceState (..)
26− , lpackArraysChunksOf
27− , compact -- chunksCompact
28− , compactLE
29− , compactEQ
30− , compactGE
31−
32− -- * Elimination
33− , MArray.flattenArrays -- chunksConcat
34− , MArray.flattenArraysRev -- chunksConcatRev
35− , MArray.fromArrayStreamK -- chunksCoalesce
36− )
37−where
38−
39−import Control.Monad.IO.Class (MonadIO(..))
40−import Streamly.Internal.Data.Unbox (Unbox)
41−import Streamly.Internal.Data.MutArray.Type (MutArray(..))
42−import Streamly.Internal.Data.Fold.Type (Fold(..))
43−import Streamly.Internal.Data.Parser (ParseError)
44−import Streamly.Internal.Data.Stream.Type (Stream)
45−
46−import qualified Streamly.Internal.Data.MutArray as MArray
47−import qualified Streamly.Internal.Data.Fold.Type as FL
48−import qualified Streamly.Internal.Data.Parser as ParserD
49−import qualified Streamly.Internal.Data.Stream as D
50−
51−-------------------------------------------------------------------------------
52−-- Compact
53−-------------------------------------------------------------------------------
54−
55−-- XXX This can be removed once compactLEFold/compactLE are implemented.
56−--
57−-- | This mutates the first array (if it has space) to append values from the
58−-- second one. This would work for immutable arrays as well because an
59−-- immutable array never has space so a new array is allocated instead of
60−-- mutating it.
61−--
62−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
63−-- maximum specified size. Note that if a single array is bigger than the
64−-- specified size we do not split it to fit. When we coalesce multiple arrays
65−-- if the size would exceed the specified size we do not coalesce therefore the
66−-- actual array size may be less than the specified chunk size.
67−--
68−-- @since 0.7.0
69−{-# INLINE packArraysChunksOf #-}
70−packArraysChunksOf :: (MonadIO m, Unbox a)
71− => Int -> D.Stream m (MutArray a) -> D.Stream m (MutArray a)
72−packArraysChunksOf = MArray.compactLE
73−
74−-- XXX Remove this once compactLEFold is implemented
75−-- lpackArraysChunksOf = Fold.many compactLEFold
76−--
77−{-# INLINE lpackArraysChunksOf #-}
78−lpackArraysChunksOf :: (MonadIO m, Unbox a)
79− => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
80−lpackArraysChunksOf = MArray.lCompactGE
81−
82−-- XXX Same as compactLE, to be removed once that is implemented.
83−--
84−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
85−-- maximum specified size in bytes.
86−--
87−-- /Internal/
88−{-# INLINE compact #-}
89−compact :: (MonadIO m, Unbox a)
90− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
91−compact = packArraysChunksOf
92−
93−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
94−-- maximum specified size. Note that if a single array is bigger than the
95−-- specified size we do not split it to fit. When we coalesce multiple arrays
96−-- if the size would exceed the specified size we do not coalesce therefore the
97−-- actual array size may be less than the specified chunk size.
98−--
99−-- /Internal/
100−{-# INLINE compactLEParserD #-}
101−compactLEParserD ::
102− forall m a. (MonadIO m, Unbox a)
103− => Int -> ParserD.Parser (MutArray a) m (MutArray a)
104−compactLEParserD = MArray.pCompactLE
105−
106−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
107−-- minimum specified size. Note that if all the arrays in the stream together
108−-- are smaller than the specified size the resulting array will be smaller than
109−-- the specified size. When we coalesce multiple arrays if the size would exceed
110−-- the specified size we stop coalescing further.
111−--
112−-- /Internal/
113−{-# INLINE compactGEFold #-}
114−compactGEFold ::
115− forall m a. (MonadIO m, Unbox a)
116− => Int -> FL.Fold m (MutArray a) (MutArray a)
117−compactGEFold = MArray.fCompactGE
118−
119−-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
120−-- maximum specified size in bytes.
121−--
122−-- /Internal/
123−compactLE :: (MonadIO m, Unbox a) =>
124− Int -> Stream m (MutArray a) -> Stream m (Either ParseError (MutArray a))
125−compactLE n = D.parseManyD (compactLEParserD n)
126−
127−-- | Like 'compactLE' but generates arrays of exactly equal to the size
128−-- specified except for the last array in the stream which could be shorter.
129−--
130−-- /Unimplemented/
131−{-# INLINE compactEQ #-}
132−compactEQ :: -- (MonadIO m, Unbox a) =>
133− Int -> Stream m (MutArray a) -> Stream m (MutArray a)
134−compactEQ _n _xs = undefined
135− -- IsStream.fromStreamD $ D.foldMany (compactEQFold n) (IsStream.toStreamD xs)
136−
137−-- | Like 'compactLE' but generates arrays of size greater than or equal to the
138−-- specified except for the last array in the stream which could be shorter.
139−--
140−-- /Internal/
141−{-# INLINE compactGE #-}
142−compactGE ::
143− (MonadIO m, Unbox a)
144− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
145−compactGE n = D.foldMany (compactGEFold n)
core/src/Streamly/Internal/Data/MutArray/Type.hs+441−1308
@@ -28,80 +28,57 @@
2828
2929module Streamly.Internal.Data.MutArray.Type
3030 (
31− -- ** Type
31+ -- * Type
3232 -- $arrayNotes
3333 MutArray (..)
34−
35− -- ** Conversion
36− -- *** Pinned and Unpinned
34+ , MutableByteArray
35+ , touch
3736 , pin
3837 , unpin
3938 , isPinned
4039
41− -- ** Casting
42− , cast
43− , castUnsafe -- XXX unsafeCast
44− , asBytes
45− , unsafePinnedAsPtr
46− , unsafeAsPtr
47−
40+ -- * Constructing and Writing
4841 -- ** Construction
49− , empty
50−
51− -- *** New
52− -- | New arrays are always empty arrays with some reserve capacity to
53− -- extend the length without reallocating.
54− , emptyOf
55− , newArrayWith -- emptyAlignedWith
56− , pinnedEmptyOf
57− , pinnedNewAligned -- XXX not required
58− -- , new -- uninitialized array of specified length
59−
60− -- *** Cloning
61− , clone
62− , pinnedClone
63−
64− -- *** Slicing
65− -- | Get a subarray without copying
66− , getSliceUnsafe -- XXX unsafeGetSlice
67− , getSlice
68− , splitAt -- XXX should be able to express using getSlice
69− , breakOn
42+ , nil
43+
44+ -- *** Uninitialized Arrays
45+ , pinnedNew
46+ , pinnedNewBytes
47+ , pinnedNewAligned
48+ , new
49+ , newArrayWith
7050
71− -- *** Stream Folds
51+ -- *** From streams
7252 , ArrayUnsafe (..)
73− , unsafeCreateOfWith
74− , unsafeCreateOf
75− , unsafePinnedCreateOf
76− , pinnedCreateOf
77− , createOfWith
78− , createOf
79− , revCreateOf
80−
81− , pinnedCreate
82− , createWith
83− , create
84− -- , revCreate
85−
86− -- *** From containers
53+ , writeNWithUnsafe
54+ , writeNWith
55+ , writeNUnsafe
56+ , pinnedWriteNUnsafe
57+ , writeN
58+ , pinnedWriteN
59+ , pinnedWriteNAligned
60+
61+ , writeWith
62+ , write
63+ , pinnedWrite
64+
65+ , writeRevN
66+ -- , writeRev
67+
68+ -- ** From containers
8769 , fromListN
8870 , pinnedFromListN
8971 , fromList
9072 , pinnedFromList
9173 , fromListRevN
9274 , fromListRev
93− , fromStreamN
94− , fromStream
95− , fromPureStreamN
75+ , fromStreamDN
76+ , fromStreamD
9677 , fromPureStream
97− , fromByteStr#
98− , fromPtrN
99− , fromChunksK
100− , fromChunksRealloced -- fromSmallChunks
10178
102− -- ** Random writes
79+ -- * Random writes
10380 , putIndex
104− , putIndexUnsafe -- XXX unsafePutIndex
81+ , putIndexUnsafe
10582 , putIndices
10683 -- , putFromThenTo
10784 -- , putFrom -- start writing at the given position
@@ -109,71 +86,78 @@ module Streamly.Internal.Data.MutArray.Type
10986 -- , putFromTo
11087 -- , putFromRev
11188 -- , putUptoRev
112− , modifyIndexUnsafe -- XXX unsafeModifyIndex
89+ , modifyIndexUnsafe
11390 , modifyIndex
11491 , modifyIndices
11592 , modify
11693 , swapIndices
11794 , unsafeSwapIndices
11895
119− -- ** Reading
96+ -- * Growing and Shrinking
97+ -- Arrays grow only at the end, though it is possible to grow on both sides
98+ -- and therefore have a cons as well as snoc. But that will require two
99+ -- bounds in the array representation.
120100
121− -- *** Indexing
122− , getIndex
123− , getIndexUnsafe -- XXX unsafeGetIndex
124− -- , getFromThenTo
125− , getIndexRev -- getRevIndex?
126− , indexReader
127− , indexReaderWith
101+ -- ** Appending elements
102+ , snocWith
103+ , snoc
104+ , snocLinear
105+ , snocMay
106+ , snocUnsafe
107+
108+ -- ** Appending streams
109+ , writeAppendNUnsafe
110+ , writeAppendN
111+ , writeAppendWith
112+ , writeAppend
113+
114+ -- * Eliminating and Reading
128115
129− -- *** To Streams
116+ -- ** To streams
117+ , reader
118+ , readerRevWith
119+ , readerRev
120+
121+ -- ** To containers
122+ , toStreamDWith
123+ , toStreamDRevWith
124+ , toStreamKWith
125+ , toStreamKRevWith
130126 , read
131127 , readRev
132− , toStreamWith
133− , toStreamRevWith
134128 , toStreamK
135− , toStreamKWith
136129 , toStreamKRev
137− , toStreamKRevWith
138−
139− -- *** To Containers
140130 , toList
141131
142− -- *** Unfolds
143132 -- experimental
144133 , producerWith
145134 , producer
146135
147− , reader
148− , readerRevWith
149− , readerRev
150−
151− -- ** Size and Capacity
152− -- *** Size
153− , length
154− , byteLength
155−
156− -- *** Capacity
157− -- , capacity
158− , byteCapacity
159− , bytesFree
136+ -- ** Random reads
137+ , getIndex
138+ , getIndexUnsafe
139+ , getIndices
140+ , getIndicesD
141+ -- , getFromThenTo
142+ , getIndexRev
160143
161− -- *** Capacity Management
144+ -- * Memory Management
162145 , blockSize
163146 , arrayChunkBytes
164147 , allocBytesToElemCount
165− , realloc -- this is "resize"
166− , grow
167− , growExp
148+ , realloc
149+ , resize
150+ , resizeExp
168151 , rightSize
169152
170− -- ** Folding
171− , foldl'
172− , foldr
173− , byteCmp
174− , byteEq
153+ -- * Size
154+ , length
155+ , byteLength
156+ -- , capacity
157+ , byteCapacity
158+ , bytesFree
175159
176− -- ** In-place Mutation Algorithms
160+ -- * In-place Mutation Algorithms
177161 , strip
178162 , reverse
179163 , permute
@@ -183,121 +167,59 @@ module Streamly.Internal.Data.MutArray.Type
183167 , mergeBy
184168 , bubble
185169
186− -- ** Growing and Shrinking
187− -- | Arrays grow only at the end, though it is possible to grow on both
188− -- sides and therefore have a cons as well as snoc. But that will require
189− -- both lower and upper bound in the array representation.
190−
191− -- *** Appending elements
192− , snocWith
193− , snoc
194− , snocLinear
195− , snocMay
196− , snocUnsafe -- XXX unsafeSnoc
197−
198− -- *** Appending streams
199− , unsafeAppendN
200− , appendN
201− , appendWith
202− , append
203−
204− -- *** Appending arrays
205− , spliceCopy
206− , spliceWith
207− , splice
208− , spliceExp
209− , spliceUnsafe -- XXX unsafeSplice
210− -- , putSlice
211− -- , appendSlice
212− -- , appendSliceFrom
213−
214− -- ** Serialization using Unbox
215− , pokeAppend
216− , pokeAppendMay
217− , pokeSkipUnsafe -- unsafePokeSkip
170+ -- * Casting
171+ , cast
172+ , castUnsafe
173+ , asBytes
174+ , asPtrUnsafe
218175
219− -- ** Deserialization using Unbox
220− , peekUncons
221− , peekUnconsUnsafe -- unsafePeekUncons
222− , peekSkipUnsafe -- unsafePeekSkip
176+ -- * Folding
177+ , foldl'
178+ , foldr
179+ , cmp
223180
224− -- Arrays of arrays
181+ -- * Arrays of arrays
225182 -- We can add dimensionality parameter to the array type to get
226183 -- multidimensional arrays. Multidimensional arrays would just be a
227184 -- convenience wrapper on top of single dimensional arrays.
228185
229− -- ** Streams of Arrays
230− -- *** Chunk
231− -- | Group a stream into arrays.
186+ -- | Operations dealing with multiple arrays, streams of arrays or
187+ -- multidimensional array representations.
188+
189+ -- ** Construct from streams
232190 , chunksOf
233191 , pinnedChunksOf
234− , buildChunks
192+ , writeChunks
235193
236− -- *** Split
237− -- | Split an array into slices.
194+ -- ** Eliminate to streams
195+ , flattenArrays
196+ , flattenArraysRev
197+ , fromArrayStreamK
238198
199+ -- ** Construct from arrays
200+ -- get chunks without copying
201+ , getSliceUnsafe
202+ , getSlice
239203 -- , getSlicesFromLenN
240− , splitOn
241− -- , slicesOf
242−
243− -- *** Concat
244− -- | Append the arrays in a stream to form a stream of elements.
245− , concatWith
246− , concatRevWith
247− , concat
248− , concatRev
249−
250− -- *** Compact
251− -- | Append the arrays in a stream to form a stream of larger arrays.
252− , SpliceState (..)
253− , pCompactLE
254− , pPinnedCompactLE
255− , compactLeAs
256− , fCompactGE
257− , fPinnedCompactGE
258− , lCompactGE
259− , lPinnedCompactGE
260− , compactGE
261− , compactEQ
262−
263− -- ** Utilities
204+ , splitAt -- XXX should be able to express using getSlice
205+ , breakOn
206+
207+ -- ** Appending arrays
208+ , spliceCopy
209+ , spliceWith
210+ , splice
211+ , spliceExp
212+ , spliceUnsafe
213+ , putSliceUnsafe
214+ -- , putSlice
215+ -- , appendSlice
216+ -- , appendSliceFrom
217+
218+ -- * Utilities
264219 , roundUpToPower2
265220 , memcpy
266221 , memcmp
267222 , c_memchr
268−
269− -- * Deprecated
270− , asPtrUnsafe
271− , writeChunks
272− , flattenArrays
273− , flattenArraysRev
274− , fromArrayStreamK
275− , fromStreamDN
276− , fromStreamD
277− , cmp
278− , getIndices
279− , getIndicesWith
280− , resize
281− , resizeExp
282− , nil
283− , new
284− , pinnedNew
285− , pinnedNewBytes
286− , writeAppendNUnsafe
287− , writeAppendN
288− , writeAppendWith
289− , writeAppend
290− , writeNWithUnsafe
291− , writeNWith
292− , writeNUnsafe
293− , pinnedWriteNUnsafe
294− , writeN
295− , pinnedWriteN
296− , pinnedWriteNAligned -- XXX not required
297− , writeWith
298− , write
299− , pinnedWrite
300− , writeRevN
301223 )
302224where
303225
@@ -308,52 +230,48 @@ where
308230
309231import Control.Monad (when, void)
310232import Control.Monad.IO.Class (MonadIO(..))
311−import Data.Bifunctor (first)
312233import Data.Bits (shiftR, (.|.), (.&.))
313234import Data.Functor.Identity (Identity(..))
314235import Data.Proxy (Proxy(..))
315236import Data.Word (Word8)
316237import Foreign.C.Types (CSize(..), CInt(..))
317238import Foreign.Ptr (plusPtr, minusPtr, nullPtr)
318−import Streamly.Internal.Data.MutByteArray.Type
319− ( MutByteArray(..)
239+import Streamly.Internal.Data.Unbox
240+ ( MutableByteArray(..)
241+ , Unbox(..)
320242 , PinnedState(..)
321243 , getMutableByteArray#
322− , putSliceUnsafe
244+ , touch
323245 )
324−import Streamly.Internal.Data.Unbox (Unbox(..))
325246import GHC.Base
326247 ( IO(..)
327248 , Int(..)
249+ , byteArrayContents#
328250 , compareByteArrays#
329251 , copyMutableByteArray#
330252 )
331253import GHC.Base (noinline)
332−import GHC.Exts (unsafeCoerce#, Addr#)
254+import GHC.Exts (unsafeCoerce#)
333255import GHC.Ptr (Ptr(..))
334256
335257import Streamly.Internal.Data.Fold.Type (Fold(..))
336258import Streamly.Internal.Data.Producer.Type (Producer (..))
337−import Streamly.Internal.Data.Stream.Type (Stream)
338−import Streamly.Internal.Data.Parser.Type (Parser (..))
339−import Streamly.Internal.Data.StreamK.Type (StreamK)
259+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
260+import Streamly.Internal.Data.Stream.StreamK.Type (StreamK)
340261import Streamly.Internal.Data.SVar.Type (adaptState, defState)
341−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
342262import Streamly.Internal.Data.Unfold.Type (Unfold(..))
343263import Streamly.Internal.System.IO (arrayPayloadSize, defaultChunkSize)
344264
345265import qualified Streamly.Internal.Data.Fold.Type as FL
346−import qualified Streamly.Internal.Data.MutByteArray.Type as Unboxed
347−import qualified Streamly.Internal.Data.Parser.Type as Parser
348−-- import qualified Streamly.Internal.Data.Fold.Type as Fold
349266import qualified Streamly.Internal.Data.Producer as Producer
350−import qualified Streamly.Internal.Data.Stream.Type as D
351−import qualified Streamly.Internal.Data.Stream.Lift as D
352−import qualified Streamly.Internal.Data.StreamK.Type as K
267+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
268+import qualified Streamly.Internal.Data.Stream.StreamD.Lift as D
269+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
270+import qualified Streamly.Internal.Data.Unbox as Unboxed
353271import qualified Prelude
354272
355273import Prelude hiding
356− (Foldable(..), concat, read, unlines, splitAt, reverse, truncate)
274+ (Foldable(..), read, unlines, splitAt, reverse, truncate)
357275
358276#include "DocTestDataMutArray.hs"
359277
@@ -361,7 +279,6 @@ import Prelude hiding
361279-- Foreign helpers
362280-------------------------------------------------------------------------------
363281
364−-- NOTE: Have to be "ccall unsafe" so that we can pass unpinned memory to these
365282foreign import ccall unsafe "string.h memcpy" c_memcpy
366283 :: Ptr Word8 -> Ptr Word8 -> CSize -> IO (Ptr Word8)
367284
@@ -371,9 +288,6 @@ foreign import ccall unsafe "string.h memchr" c_memchr
371288foreign import ccall unsafe "string.h memcmp" c_memcmp
372289 :: Ptr Word8 -> Ptr Word8 -> CSize -> IO CInt
373290
374−foreign import ccall unsafe "string.h strlen" c_strlen
375− :: Ptr Word8 -> IO CSize
376−
377291-- | Given an 'Unboxed' type (unused first arg) and a number of bytes, return
378292-- how many elements of that type will completely fit in those bytes.
379293--
@@ -428,7 +342,7 @@ data MutArray a =
428342 -- The array is a range into arrContents. arrContents may be a superset of
429343 -- the slice represented by the array. All offsets are in bytes.
430344 MutArray
431− { arrContents :: {-# UNPACK #-} !MutByteArray
345+ { arrContents :: {-# UNPACK #-} !MutableByteArray
432346 , arrStart :: {-# UNPACK #-} !Int -- ^ index into arrContents
433347 , arrEnd :: {-# UNPACK #-} !Int -- ^ index into arrContents
434348 -- Represents the first invalid index of
@@ -440,25 +354,22 @@ data MutArray a =
440354-- Pinning & Unpinning
441355-------------------------------------------------------------------------------
442356
443−-- | Return a copy of the array in pinned memory if unpinned, else return the
444−-- original array.
357+-- | Copy the 'MutArray' to pinned memory if unpinned, else do nothing. The
358+-- overhead is a copy if the input array is unpinned.
445359{-# INLINE pin #-}
446360pin :: MutArray a -> IO (MutArray a)
447−pin arr@MutArray{..} =
448− if Unboxed.isPinned arrContents
449− then pure arr
450− else pinnedClone arr
361+pin arr@MutArray{..} = do
362+ contents <- Unboxed.pin arrContents
363+ return $ arr {arrContents = contents}
451364
452−-- | Return a copy of the array in unpinned memory if pinned, else return the
453−-- original array.
365+-- | Copy the 'MutArray' to unpinned memory if pinned, else do nothing. The
366+-- overhead is a copy if the input array is pinned.
454367{-# INLINE unpin #-}
455368unpin :: MutArray a -> IO (MutArray a)
456−unpin arr@MutArray{..} =
457− if Unboxed.isPinned arrContents
458− then clone arr
459− else pure arr
369+unpin arr@MutArray{..} = do
370+ contents <- Unboxed.unpin arrContents
371+ return $ arr {arrContents = contents}
460372
461−-- | Return 'True' if the array is allocated in pinned memory.
462373{-# INLINE isPinned #-}
463374isPinned :: MutArray a -> Bool
464375isPinned MutArray{..} = Unboxed.isPinned arrContents
@@ -479,8 +390,6 @@ isPinned MutArray{..} = Unboxed.isPinned arrContents
479390-- we need more than that. See stg_pinnedNewAlignedByteArrayzh and
480391-- allocatePinned in GHC source.
481392
482−-- XXX Rename to emptyAlignedWith, alignSize should be first arg.
483−
484393-- | @newArrayWith allocator alignment count@ allocates a new array of zero
485394-- length and with a capacity to hold @count@ elements, using @allocator
486395-- size alignment@ as the memory allocator function.
@@ -493,7 +402,7 @@ isPinned MutArray{..} = Unboxed.isPinned arrContents
493402-- /Pre-release/
494403{-# INLINE newArrayWith #-}
495404newArrayWith :: forall m a. (MonadIO m, Unbox a)
496− => (Int -> Int -> m MutByteArray) -> Int -> Int -> m (MutArray a)
405+ => (Int -> Int -> m MutableByteArray) -> Int -> Int -> m (MutArray a)
497406newArrayWith alloc alignSize count = do
498407 let size = max (count * SIZE_OF(a)) 0
499408 contents <- alloc size alignSize
@@ -504,25 +413,12 @@ newArrayWith alloc alignSize count = do
504413 , arrBound = size
505414 }
506415
507−-- For arrays "nil" sounds a bit odd. empty is better. The only problem with
508−-- empty is that it is also used by the Alternative type class. But assuming we
509−-- will mostly import the Array module qualified this should be fine.
510−
511−-- | Create an empty array.
512−empty ::
513−#ifdef DEVBUILD
514− Unbox a =>
515−#endif
516− MutArray a
517−empty = MutArray Unboxed.empty 0 0 0
518−
519−{-# DEPRECATED nil "Please use empty instead." #-}
520416nil ::
521417#ifdef DEVBUILD
522418 Unbox a =>
523419#endif
524420 MutArray a
525−nil = empty
421+nil = MutArray Unboxed.nil 0 0 0
526422
527423{-# INLINE newBytesAs #-}
528424newBytesAs :: MonadIO m =>
@@ -539,13 +435,12 @@ newBytesAs ps bytes = do
539435 , arrBound = bytes
540436 }
541437
542−-- | Allocates a pinned empty array that with a reserved capacity of bytes.
543−-- The memory of the array is uninitialized and the allocation is aligned as
544−-- per the 'Unboxed' instance of the type.
438+-- | Allocates a pinned empty array that can hold 'count' items. The memory of
439+-- the array is uninitialized and the allocation is aligned as per the
440+-- 'Unboxed' instance of the type.
545441--
546442-- /Pre-release/
547443{-# INLINE pinnedNewBytes #-}
548−{-# DEPRECATED pinnedNewBytes "Please use pinnedEmptyOf with appropriate calculation" #-}
549444pinnedNewBytes :: MonadIO m =>
550445#ifdef DEVBUILD
551446 Unbox a =>
@@ -570,31 +465,21 @@ newAs ps =
570465 (error "new: alignment is not used in unpinned arrays.")
571466
572467-- XXX can unaligned allocation be more efficient when alignment is not needed?
573−
574−-- | Allocates a pinned array of zero length but growable to the specified
575−-- capacity without reallocation.
576−{-# INLINE pinnedEmptyOf #-}
577−pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
578−pinnedEmptyOf = newAs Pinned
579−
580−-- XXX Deprecate in major
581−-- {-# DEPRECATED pinnedNew "Please use pinnedEmptyOf instead." #-}
468+--
469+-- | Allocates an empty pinned array that can hold 'count' items. The memory of
470+-- the array is uninitialized and the allocation is aligned as per the 'Unboxed'
471+-- instance of the type.
472+--
582473{-# INLINE pinnedNew #-}
583474pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
584−pinnedNew = pinnedEmptyOf
475+pinnedNew = newAs Pinned
585476
586−-- | Allocates an unpinned array of zero length but growable to the specified
587−-- capacity without reallocation.
477+-- | Allocates an empty unpinned array that can hold 'count' items. The memory
478+-- of the array is uninitialized.
588479--
589−{-# INLINE emptyOf #-}
590−emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
591−emptyOf = newAs Unpinned
592−
593−-- XXX Deprecate in major
594−-- {-# DEPRECATED new "Please use emptyOf instead." #-}
595480{-# INLINE new #-}
596481new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
597−new = emptyOf
482+new = newAs Unpinned
598483
599484-------------------------------------------------------------------------------
600485-- Random writes
@@ -610,7 +495,7 @@ putIndexUnsafe :: forall m a. (MonadIO m, Unbox a)
610495putIndexUnsafe i MutArray{..} x = do
611496 let index = INDEX_OF(arrStart, i, a)
612497 assert (i >= 0 && INDEX_VALID(index, arrEnd, a)) (return ())
613− liftIO $ pokeAt index arrContents x
498+ liftIO $ pokeByteIndex index arrContents x
614499
615500invalidIndex :: String -> Int -> a
616501invalidIndex label i =
@@ -628,7 +513,7 @@ putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
628513putIndex i MutArray{..} x = do
629514 let index = INDEX_OF(arrStart,i,a)
630515 if i >= 0 && INDEX_VALID(index,arrEnd,a)
631− then liftIO $ pokeAt index arrContents x
516+ then liftIO $ pokeByteIndex index arrContents x
632517 else invalidIndex "putIndex" i
633518
634519-- | Write an input stream of (index, value) pairs to an array. Throws an
@@ -642,7 +527,7 @@ putIndices arr = FL.foldlM' step (return ())
642527
643528 where
644529
645− step () (i, x) = putIndex i arr x
530+ step () (i, x) = liftIO (putIndex i arr x)
646531
647532-- | Modify a given index of an array using a modifier function.
648533--
@@ -654,9 +539,9 @@ modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) =>
654539modifyIndexUnsafe i MutArray{..} f = liftIO $ do
655540 let index = INDEX_OF(arrStart,i,a)
656541 assert (i >= 0 && INDEX_NEXT(index,a) <= arrEnd) (return ())
657− r <- peekAt index arrContents
542+ r <- peekByteIndex index arrContents
658543 let (x, res) = f r
659− pokeAt index arrContents x
544+ pokeByteIndex index arrContents x
660545 return res
661546
662547-- | Modify a given index of an array using a modifier function.
@@ -668,9 +553,9 @@ modifyIndex i MutArray{..} f = do
668553 let index = INDEX_OF(arrStart,i,a)
669554 if i >= 0 && INDEX_VALID(index,arrEnd,a)
670555 then liftIO $ do
671− r <- peekAt index arrContents
556+ r <- peekByteIndex index arrContents
672557 let (x, res) = f r
673− pokeAt index arrContents x
558+ pokeByteIndex index arrContents x
674559 return res
675560 else invalidIndex "modifyIndex" i
676561
@@ -704,8 +589,8 @@ modify MutArray{..} f = liftIO $
704589
705590 go i =
706591 when (INDEX_VALID(i,arrEnd,a)) $ do
707− r <- peekAt i arrContents
708− pokeAt i arrContents (f r)
592+ r <- peekByteIndex i arrContents
593+ pokeByteIndex i arrContents (f r)
709594 go (INDEX_NEXT(i,a))
710595
711596-- XXX We could specify the number of bytes to swap instead of Proxy. Need
@@ -714,15 +599,15 @@ modify MutArray{..} f = liftIO $
714599swapArrayByteIndices ::
715600 forall a. Unbox a
716601 => Proxy a
717− -> MutByteArray
602+ -> MutableByteArray
718603 -> Int
719604 -> Int
720605 -> IO ()
721606swapArrayByteIndices _ arrContents i1 i2 = do
722− r1 <- peekAt i1 arrContents
723− r2 <- peekAt i2 arrContents
724− pokeAt i1 arrContents (r2 :: a)
725− pokeAt i2 arrContents (r1 :: a)
607+ r1 <- peekByteIndex i1 arrContents
608+ r2 <- peekByteIndex i2 arrContents
609+ pokeByteIndex i1 arrContents (r2 :: a)
610+ pokeByteIndex i2 arrContents (r1 :: a)
726611
727612-- | Swap the elements at two indices without validating the indices.
728613--
@@ -838,23 +723,26 @@ arrayChunkBytes = 1024
838723roundDownTo :: Int -> Int -> Int
839724roundDownTo elemSize size = size - (size `mod` elemSize)
840725
726+-- XXX See if resizing can be implemented by reading the old array as a stream
727+-- and then using writeN to the new array.
728+--
841729-- NOTE: we are passing elemSize explicitly to avoid an Unboxed constraint.
842730-- Since this is not inlined Unboxed consrraint leads to dictionary passing
843731-- which complicates some inspection tests.
844732--
845−{-# NOINLINE reallocExplicitAs #-}
846−reallocExplicitAs :: PinnedState -> Int -> Int -> MutArray a -> IO (MutArray a)
847−reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
733+{-# NOINLINE reallocExplicit #-}
734+reallocExplicit :: Int -> Int -> MutArray a -> IO (MutArray a)
735+reallocExplicit elemSize newCapacityInBytes MutArray{..} = do
848736 assertM(arrEnd <= arrBound)
849737
850738 -- Allocate new array
851739 let newCapMaxInBytes = roundUpLargeArray newCapacityInBytes
852740 contents <-
853− if ps == Pinned
854− then Unboxed.pinnedNew newCapMaxInBytes
855− else Unboxed.new newCapMaxInBytes
856− let !(MutByteArray mbarrFrom#) = arrContents
857− !(MutByteArray mbarrTo#) = contents
741+ if Unboxed.isPinned arrContents
742+ then Unboxed.pinnedNewBytes newCapMaxInBytes
743+ else Unboxed.newBytes newCapMaxInBytes
744+ let !(MutableByteArray mbarrFrom#) = arrContents
745+ !(MutableByteArray mbarrTo#) = contents
858746
859747 -- Copy old data
860748 let oldStart = arrStart
@@ -875,10 +763,6 @@ reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
875763 , arrBound = newCapInBytes
876764 }
877765
878−-- XXX Should these be called reallocBytes etc?
879−-- XXX We may also need reallocAs to allocate as pinned/unpinned explicitly. In
880−-- fact clone/pinnedClone can be implemented using reallocAs.
881−
882766-- | @realloc newCapacity array@ reallocates the array to the specified
883767-- capacity in bytes.
884768--
@@ -890,12 +774,7 @@ reallocExplicitAs ps elemSize newCapacityInBytes MutArray{..} = do
890774-- If the original array is pinned, the newly allocated array is also pinned.
891775{-# INLINABLE realloc #-}
892776realloc :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
893−realloc bytes arr =
894− let ps =
895− if isPinned arr
896− then Pinned
897− else Unpinned
898− in liftIO $ reallocExplicitAs ps (SIZE_OF(a)) bytes arr
777+realloc bytes arr = liftIO $ reallocExplicit (SIZE_OF(a)) bytes arr
899778
900779-- | @reallocWith label capSizer minIncrBytes array@. The label is used
901780-- in error messages and the capSizer is used to determine the capacity of the
@@ -918,14 +797,14 @@ reallocWith label capSizer minIncrBytes arr = do
918797 where
919798
920799 badSize newSize =
921− Prelude.concat
800+ concat
922801 [ label
923802 , ": new array size (in bytes) is less than required size "
924803 , show newSize
925804 , ". Please check the sizing function passed."
926805 ]
927806
928−-- | @grow newCapacity array@ changes the total capacity of the array so that
807+-- | @resize newCapacity array@ changes the total capacity of the array so that
929808-- it is enough to hold the specified number of elements. Nothing is done if
930809-- the specified capacity is less than the length of the array.
931810--
@@ -933,46 +812,34 @@ reallocWith label capSizer minIncrBytes arr = do
933812-- the block size (4K).
934813--
935814-- /Pre-release/
936−{-# INLINE grow #-}
937−grow :: forall m a. (MonadIO m, Unbox a) =>
815+{-# INLINE resize #-}
816+resize :: forall m a. (MonadIO m, Unbox a) =>
938817 Int -> MutArray a -> m (MutArray a)
939−grow nElems arr@MutArray{..} = do
818+resize nElems arr@MutArray{..} = do
940819 let req = SIZE_OF(a) * nElems
941− cap = arrBound - arrStart
942− if req < cap
820+ len = arrEnd - arrStart
821+ if req < len
943822 then return arr
944823 else realloc req arr
945824
946−{-# DEPRECATED resize "Please use grow instead." #-}
947−{-# INLINE resize #-}
948−resize :: forall m a. (MonadIO m, Unbox a) =>
949− Int -> MutArray a -> m (MutArray a)
950−resize = grow
951−
952−-- | Like 'grow' but if the requested byte capacity is more than
953−-- 'largeObjectThreshold' then it is rounded up to the closest power of 2.
825+-- | Like 'resize' but if the byte capacity is more than 'largeObjectThreshold'
826+-- then it is rounded up to the closest power of 2.
954827--
955828-- /Pre-release/
956−{-# INLINE growExp #-}
957−growExp :: forall m a. (MonadIO m, Unbox a) =>
829+{-# INLINE resizeExp #-}
830+resizeExp :: forall m a. (MonadIO m, Unbox a) =>
958831 Int -> MutArray a -> m (MutArray a)
959−growExp nElems arr@MutArray{..} = do
832+resizeExp nElems arr@MutArray{..} = do
960833 let req = roundUpLargeArray (SIZE_OF(a) * nElems)
961834 req1 =
962835 if req > largeObjectThreshold
963836 then roundUpToPower2 req
964837 else req
965− cap = arrBound - arrStart
966− if req1 < cap
838+ len = arrEnd - arrStart
839+ if req1 < len
967840 then return arr
968841 else realloc req1 arr
969842
970−{-# DEPRECATED resizeExp "Please use growExp instead." #-}
971−{-# INLINE resizeExp #-}
972−resizeExp :: forall m a. (MonadIO m, Unbox a) =>
973− Int -> MutArray a -> m (MutArray a)
974−resizeExp = growExp
975−
976843-- | Resize the allocated memory to drop any reserved free space at the end of
977844-- the array and reallocate it to reduce wastage.
978845--
@@ -1017,7 +884,7 @@ rightSize arr@MutArray{..} = do
1017884snocNewEnd :: (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m (MutArray a)
1018885snocNewEnd newEnd arr@MutArray{..} x = liftIO $ do
1019886 assert (newEnd <= arrBound) (return ())
1020− pokeAt arrEnd arrContents x
887+ pokeByteIndex arrEnd arrContents x
1021888 return $ arr {arrEnd = newEnd}
1022889
1023890-- | Really really unsafe, appends the element into the first array, may
@@ -1037,7 +904,7 @@ snocUnsafe arr@MutArray{..} = snocNewEnd (INDEX_NEXT(arrEnd,a)) arr
1037904{-# INLINE snocMay #-}
1038905snocMay :: forall m a. (MonadIO m, Unbox a) =>
1039906 MutArray a -> a -> m (Maybe (MutArray a))
1040−snocMay arr@MutArray{..} x = do
907+snocMay arr@MutArray{..} x = liftIO $ do
1041908 let newEnd = INDEX_NEXT(arrEnd,a)
1042909 if newEnd <= arrBound
1043910 then Just <$> snocNewEnd newEnd arr x
@@ -1051,7 +918,7 @@ snocWithRealloc :: forall m a. (MonadIO m, Unbox a) =>
1051918 -> a
1052919 -> m (MutArray a)
1053920snocWithRealloc sizer arr x = do
1054− arr1 <- reallocWith "snocWith" sizer (SIZE_OF(a)) arr
921+ arr1 <- liftIO $ reallocWith "snocWith" sizer (SIZE_OF(a)) arr
1055922 snocUnsafe arr1 x
1056923
1057924-- | @snocWith sizer arr elem@ mutates @arr@ to append @elem@. The length of
@@ -1073,7 +940,7 @@ snocWith :: forall m a. (MonadIO m, Unbox a) =>
1073940 -> MutArray a
1074941 -> a
1075942 -> m (MutArray a)
1076−snocWith allocSize arr x = do
943+snocWith allocSize arr x = liftIO $ do
1077944 let newEnd = INDEX_NEXT(arrEnd arr,a)
1078945 if newEnd <= arrBound arr
1079946 then snocNewEnd newEnd arr x
@@ -1117,126 +984,6 @@ snoc = snocWith f
1117984 then oldSize * 2
1118985 else roundUpToPower2 oldSize * 2
1119986
1120−-------------------------------------------------------------------------------
1121−-- Serialization/Deserialization
1122−-------------------------------------------------------------------------------
1123−
1124−{-# INLINE pokeNewEnd #-}
1125−pokeNewEnd :: (MonadIO m, Unbox a) =>
1126− Int -> MutArray Word8 -> a -> m (MutArray Word8)
1127−pokeNewEnd newEnd arr@MutArray{..} x = liftIO $ do
1128− assert (newEnd <= arrBound) (return ())
1129− liftIO $ pokeAt arrEnd arrContents x
1130− return $ arr {arrEnd = newEnd}
1131−
1132−-- | Really really unsafe, unboxes a Haskell type and appends the resulting
1133−-- bytes to the byte array, may cause silent data corruption or if you are
1134−-- lucky a segfault if the array does not have enough space to append the
1135−-- element.
1136−--
1137−-- /Internal/
1138−{-# INLINE pokeAppendUnsafe #-}
1139−pokeAppendUnsafe :: forall m a. (MonadIO m, Unbox a) =>
1140− MutArray Word8 -> a -> m (MutArray Word8)
1141−pokeAppendUnsafe arr@MutArray{..} = pokeNewEnd (arrEnd + SIZE_OF(a)) arr
1142−
1143−-- | Skip the specified number of bytes in the array. The data in the skipped
1144−-- region remains uninitialzed.
1145−{-# INLINE pokeSkipUnsafe #-}
1146−pokeSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
1147−pokeSkipUnsafe n arr@MutArray{..} = do
1148− let newEnd = arrEnd + n
1149− in assert (newEnd <= arrBound) (arr {arrEnd = newEnd})
1150−
1151−-- | Like 'pokeAppend' but does not grow the array when pre-allocated array
1152−-- capacity becomes full.
1153−--
1154−-- /Internal/
1155−{-# INLINE pokeAppendMay #-}
1156−pokeAppendMay :: forall m a. (MonadIO m, Unbox a) =>
1157− MutArray Word8 -> a -> m (Maybe (MutArray Word8))
1158−pokeAppendMay arr@MutArray{..} x = liftIO $ do
1159− let newEnd = arrEnd + SIZE_OF(a)
1160− if newEnd <= arrBound
1161− then Just <$> pokeNewEnd newEnd arr x
1162− else return Nothing
1163−
1164−{-# NOINLINE pokeWithRealloc #-}
1165−pokeWithRealloc :: forall m a. (MonadIO m, Unbox a) =>
1166− (Int -> Int)
1167− -> MutArray Word8
1168− -> a
1169− -> m (MutArray Word8)
1170−pokeWithRealloc sizer arr x = do
1171− arr1 <- liftIO $ reallocWith "pokeWithRealloc" sizer (SIZE_OF(a)) arr
1172− pokeAppendUnsafe arr1 x
1173−
1174−{-# INLINE pokeAppendWith #-}
1175−pokeAppendWith :: forall m a. (MonadIO m, Unbox a) =>
1176− (Int -> Int)
1177− -> MutArray Word8
1178− -> a
1179− -> m (MutArray Word8)
1180−pokeAppendWith allocSize arr x = liftIO $ do
1181− let newEnd = arrEnd arr + SIZE_OF(a)
1182− if newEnd <= arrBound arr
1183− then pokeNewEnd newEnd arr x
1184− else pokeWithRealloc allocSize arr x
1185−
1186−-- | Unbox a Haskell type and append the resulting bytes to a mutable byte
1187−-- array. The array is grown exponentially when more space is needed.
1188−--
1189−-- Definition:
1190−--
1191−-- >>> pokeAppend arr x = MutArray.castUnsafe <$> MutArray.snoc (MutArray.castUnsafe arr) x
1192−--
1193−{-# INLINE pokeAppend #-}
1194−pokeAppend :: forall m a. (MonadIO m, Unbox a) =>
1195− MutArray Word8 -> a -> m (MutArray Word8)
1196−pokeAppend = pokeAppendWith f
1197−
1198− where
1199−
1200− f oldSize =
1201− if isPower2 oldSize
1202− then oldSize * 2
1203− else roundUpToPower2 oldSize * 2
1204−
1205−-- | Really really unsafe, create a Haskell value from an unboxed byte array,
1206−-- does not check if the array is big enough, may return garbage or if you are
1207−-- lucky may cause a segfault.
1208−--
1209−-- /Internal/
1210−{-# INLINE peekUnconsUnsafe #-}
1211−peekUnconsUnsafe :: forall m a. (MonadIO m, Unbox a) =>
1212− MutArray Word8 -> m (a, MutArray Word8)
1213−peekUnconsUnsafe MutArray{..} = do
1214− let start1 = arrStart + SIZE_OF(a)
1215− assert (start1 <= arrEnd) (return ())
1216− liftIO $ do
1217− r <- peekAt arrStart arrContents
1218− return (r, MutArray arrContents start1 arrEnd arrBound)
1219−
1220−-- | Discard the specified number of bytes in the array.
1221−{-# INLINE peekSkipUnsafe #-}
1222−peekSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
1223−peekSkipUnsafe n MutArray{..} =
1224− let start1 = arrStart + n
1225− in assert (start1 <= arrEnd) (MutArray arrContents start1 arrEnd arrBound)
1226−
1227−-- | Create a Haskell value from its unboxed representation from the head of a
1228−-- byte array, return the value and the remaining array.
1229−{-# INLINE peekUncons #-}
1230−peekUncons :: forall m a. (MonadIO m, Unbox a) =>
1231− MutArray Word8 -> m (Maybe a, MutArray Word8)
1232−peekUncons arr@MutArray{..} = do
1233− let start1 = arrStart + SIZE_OF(a)
1234− if start1 > arrEnd
1235− then return (Nothing, arr)
1236− else liftIO $ do
1237− r <- peekAt arrStart arrContents
1238− return (Just r, MutArray arrContents start1 arrEnd arrBound)
1239−
1240987-------------------------------------------------------------------------------
1241988-- Random reads
1242989-------------------------------------------------------------------------------
@@ -1251,7 +998,7 @@ getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1251998getIndexUnsafe i MutArray{..} = do
1252999 let index = INDEX_OF(arrStart,i,a)
12531000 assert (i >= 0 && INDEX_VALID(index,arrEnd,a)) (return ())
1254− liftIO $ peekAt index arrContents
1001+ liftIO $ peekByteIndex index arrContents
12551002
12561003-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
12571004--
@@ -1260,7 +1007,7 @@ getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
12601007getIndex i MutArray{..} = do
12611008 let index = INDEX_OF(arrStart,i,a)
12621009 if i >= 0 && INDEX_VALID(index,arrEnd,a)
1263− then liftIO $ Just <$> peekAt index arrContents
1010+ then liftIO $ Just <$> peekByteIndex index arrContents
12641011 else return Nothing
12651012
12661013-- | /O(1)/ Lookup the element at the given index from the end of the array.
@@ -1273,16 +1020,21 @@ getIndexRev :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
12731020getIndexRev i MutArray{..} = do
12741021 let index = RINDEX_OF(arrEnd,i,a)
12751022 if i >= 0 && index >= arrStart
1276− then liftIO $ peekAt index arrContents
1023+ then liftIO $ peekByteIndex index arrContents
12771024 else invalidIndex "getIndexRev" i
12781025
12791026data GetIndicesState contents start end st =
12801027 GetIndicesState contents start end st
12811028
1282−{-# INLINE indexReaderWith #-}
1283−indexReaderWith :: (Monad m, Unbox a) =>
1029+-- | Given an unfold that generates array indices, read the elements on those
1030+-- indices from the supplied MutArray. An error is thrown if an index is out of
1031+-- bounds.
1032+--
1033+-- /Pre-release/
1034+{-# INLINE getIndicesD #-}
1035+getIndicesD :: (Monad m, Unbox a) =>
12841036 (forall b. IO b -> m b) -> D.Stream m Int -> Unfold m (MutArray a) a
1285−indexReaderWith liftio (D.Stream stepi sti) = Unfold step inject
1037+getIndicesD liftio (D.Stream stepi sti) = Unfold step inject
12861038
12871039 where
12881040
@@ -1301,26 +1053,9 @@ indexReaderWith liftio (D.Stream stepi sti) = Unfold step inject
13011053 D.Skip s -> return $ D.Skip (GetIndicesState contents start end s)
13021054 D.Stop -> return D.Stop
13031055
1304−{-# DEPRECATED getIndicesWith "Please use indexReaderWith instead." #-}
1305−{-# INLINE getIndicesWith #-}
1306−getIndicesWith :: (Monad m, Unbox a) =>
1307− (forall b. IO b -> m b) -> D.Stream m Int -> Unfold m (MutArray a) a
1308−getIndicesWith = indexReaderWith
1309−
1310−-- | Given an unfold that generates array indices, read the elements on those
1311−-- indices from the supplied MutArray. An error is thrown if an index is out of
1312−-- bounds.
1313−--
1314−-- /Pre-release/
1315−{-# INLINE indexReader #-}
1316−indexReader :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1317−indexReader = indexReaderWith liftIO
1318−
1319−-- XXX DO NOT REMOVE, change the signature to use Stream instead of unfold
1320−{-# DEPRECATED getIndices "Please use indexReader instead." #-}
13211056{-# INLINE getIndices #-}
13221057getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1323−getIndices = indexReader
1058+getIndices = getIndicesD liftIO
13241059
13251060-------------------------------------------------------------------------------
13261061-- Subarrays
@@ -1425,7 +1160,7 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14251160
14261161 -- Invariant low < high on entry, and on return as well
14271162 moveHigh low high = do
1428− h <- peekAt high arrContents
1163+ h <- peekByteIndex high arrContents
14291164 if f h
14301165 then
14311166 -- Correctly classified, continue the loop
@@ -1445,7 +1180,7 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14451180 -- low <= high
14461181 -- Both low and high are valid locations within the array
14471182 go low high = do
1448− l <- peekAt low arrContents
1183+ l <- peekByteIndex low arrContents
14491184 if f l
14501185 then
14511186 -- low is wrongly classified
@@ -1456,8 +1191,8 @@ partitionBy f arr@MutArray{..} = liftIO $ do
14561191 case r of
14571192 Nothing -> return low
14581193 Just (high1, h) -> do -- low < high1
1459− pokeAt low arrContents h
1460− pokeAt high1 arrContents l
1194+ pokeByteIndex low arrContents h
1195+ pokeByteIndex high1 arrContents l
14611196 let low1 = INDEX_NEXT(low,a)
14621197 high2 = INDEX_PREV(high1,a)
14631198 if low1 <= high2
@@ -1591,7 +1326,7 @@ chunksOfAs ps n (D.Stream step state) =
15911326 r <- step (adaptState gst) st
15921327 case r of
15931328 D.Yield x s -> do
1594− liftIO $ pokeAt end contents x
1329+ liftIO $ pokeByteIndex end contents x
15951330 let end1 = INDEX_NEXT(end,a)
15961331 return $
15971332 if end1 >= bound
@@ -1610,12 +1345,10 @@ chunksOfAs ps n (D.Stream step state) =
16101345
16111346 step' _ GroupFinish = return D.Stop
16121347
1613−-- | @chunksOf n stream@ groups the elements in the input stream into arrays of
1614−-- @n@ elements each.
1348+-- | @chunksOf n stream@ groups the input stream into a stream of
1349+-- arrays of size n.
16151350--
1616−-- Same as the following but may be more efficient:
1617−--
1618−-- >>> chunksOf n = Stream.foldMany (MutArray.createOf n)
1351+-- @chunksOf n = StreamD.foldMany (MutArray.writeN n)@
16191352--
16201353-- /Pre-release/
16211354{-# INLINE_NORMAL chunksOf #-}
@@ -1624,29 +1357,17 @@ chunksOf :: forall m a. (MonadIO m, Unbox a)
16241357-- XXX the idiomatic implementation leads to large regression in the D.reverse'
16251358-- benchmark. It seems it has difficulty producing optimized code when
16261359-- converting to StreamK. Investigate GHC optimizations.
1627−-- chunksOf n = D.foldMany (createOf n)
1360+-- chunksOf n = D.foldMany (writeN n)
16281361chunksOf = chunksOfAs Unpinned
16291362
16301363-- | Like 'chunksOf' but creates pinned arrays.
16311364{-# INLINE_NORMAL pinnedChunksOf #-}
16321365pinnedChunksOf :: forall m a. (MonadIO m, Unbox a)
16331366 => Int -> D.Stream m a -> D.Stream m (MutArray a)
1634−-- pinnedChunksOf n = D.foldMany (pinnedCreateOf n)
1367+-- pinnedChunksOf n = D.foldMany (pinnedWriteN n)
16351368pinnedChunksOf = chunksOfAs Pinned
16361369
1637−-- | When we are buffering a stream of unknown size into an array we do not
1638−-- know how much space to pre-allocate. So we start with the min size and emit
1639−-- the array then keep on doubling the size every time. Thus we do not need to
1640−-- guess the optimum chunk size.
1641−--
1642−-- We can incorporate this in chunksOfAs if the additional size parameter does
1643−-- not impact perf.
1644−--
1645−{-# INLINE _chunksOfRange #-}
1646−_chunksOfRange :: -- (MonadIO m, Unbox a) =>
1647− PinnedState -> Int -> Int -> D.Stream m a -> D.Stream m (MutArray a)
1648−_chunksOfRange _ps _low _hi = undefined
1649−
1370+-- XXX This should take a PinnedState
16501371-- XXX buffer to a list instead?
16511372-- | Buffer the stream into arrays in memory.
16521373{-# INLINE arrayStreamKFromStreamDAs #-}
@@ -1664,10 +1385,16 @@ data FlattenState s contents a =
16641385 OuterLoop s
16651386 | InnerLoop s contents !Int !Int
16661387
1667−{-# INLINE_NORMAL concatWith #-}
1668−concatWith :: forall m a. (Monad m, Unbox a)
1669− => (forall b. IO b -> m b) -> D.Stream m (MutArray a) -> D.Stream m a
1670−concatWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
1388+-- | Use the "reader" unfold instead.
1389+--
1390+-- @flattenArrays = unfoldMany reader@
1391+--
1392+-- We can try this if there are any fusion issues in the unfold.
1393+--
1394+{-# INLINE_NORMAL flattenArrays #-}
1395+flattenArrays :: forall m a. (MonadIO m, Unbox a)
1396+ => D.Stream m (MutArray a) -> D.Stream m a
1397+flattenArrays (D.Stream step state) = D.Stream step' (OuterLoop state)
16711398
16721399 where
16731400
@@ -1684,30 +1411,19 @@ concatWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
16841411 return $ D.Skip $ OuterLoop st
16851412
16861413 step' _ (InnerLoop st contents p end) = do
1687− x <- liftio $ peekAt p contents
1414+ x <- liftIO $ peekByteIndex p contents
16881415 return $ D.Yield x (InnerLoop st contents (INDEX_NEXT(p,a)) end)
16891416
1690−-- | Use the "reader" unfold instead.
1417+-- | Use the "readerRev" unfold instead.
16911418--
1692−-- @concat = unfoldMany reader@
1419+-- @flattenArrays = unfoldMany readerRev@
16931420--
16941421-- We can try this if there are any fusion issues in the unfold.
16951422--
1696−{-# INLINE_NORMAL concat #-}
1697−concat :: forall m a. (MonadIO m, Unbox a)
1698− => D.Stream m (MutArray a) -> D.Stream m a
1699−concat = concatWith liftIO
1700−
1701−{-# DEPRECATED flattenArrays "Please use \"unfoldMany reader\" instead." #-}
1702−{-# INLINE flattenArrays #-}
1703−flattenArrays :: forall m a. (MonadIO m, Unbox a)
1423+{-# INLINE_NORMAL flattenArraysRev #-}
1424+flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
17041425 => D.Stream m (MutArray a) -> D.Stream m a
1705−flattenArrays = concat
1706−
1707−{-# INLINE_NORMAL concatRevWith #-}
1708−concatRevWith :: forall m a. (Monad m, Unbox a)
1709− => (forall b. IO b -> m b) -> D.Stream m (MutArray a) -> D.Stream m a
1710−concatRevWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
1426+flattenArraysRev (D.Stream step state) = D.Stream step' (OuterLoop state)
17111427
17121428 where
17131429
@@ -1725,33 +1441,16 @@ concatRevWith liftio (D.Stream step state) = D.Stream step' (OuterLoop state)
17251441 return $ D.Skip $ OuterLoop st
17261442
17271443 step' _ (InnerLoop st contents p start) = do
1728− x <- liftio $ peekAt p contents
1444+ x <- liftIO $ peekByteIndex p contents
17291445 let cur = INDEX_PREV(p,a)
17301446 return $ D.Yield x (InnerLoop st contents cur start)
17311447
1732−-- | Use the "readerRev" unfold instead.
1733−--
1734−-- @concat = unfoldMany readerRev@
1735−--
1736−-- We can try this if there are any fusion issues in the unfold.
1737−--
1738−{-# INLINE_NORMAL concatRev #-}
1739−concatRev :: forall m a. (MonadIO m, Unbox a)
1740− => D.Stream m (MutArray a) -> D.Stream m a
1741−concatRev = concatRevWith liftIO
1742−
1743−{-# DEPRECATED flattenArraysRev "Please use \"unfoldMany readerRev\" instead." #-}
1744−{-# INLINE flattenArraysRev #-}
1745−flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
1746− => D.Stream m (MutArray a) -> D.Stream m a
1747−flattenArraysRev = concatRev
1748−
17491448-------------------------------------------------------------------------------
17501449-- Unfolds
17511450-------------------------------------------------------------------------------
17521451
17531452data ArrayUnsafe a = ArrayUnsafe
1754− {-# UNPACK #-} !MutByteArray -- contents
1453+ {-# UNPACK #-} !MutableByteArray -- contents
17551454 {-# UNPACK #-} !Int -- index 1
17561455 {-# UNPACK #-} !Int -- index 2
17571456
@@ -1781,7 +1480,7 @@ producerWith liftio = Producer step (return . toArrayUnsafe) extract
17811480 -- few actions for correctness and execution order sanity. We want
17821481 -- the peek to occur right here and not lazily at some later point
17831482 -- because we want the peek to be ordered with respect to the touch.
1784− !x <- liftio $ peekAt cur contents
1483+ !x <- liftio $ peekByteIndex cur contents
17851484 return $ D.Yield x (ArrayUnsafe contents (INDEX_NEXT(cur,a)) end)
17861485
17871486 extract = return . fromArrayUnsafe
@@ -1812,7 +1511,7 @@ readerRevWith liftio = Unfold step inject
18121511 {-# INLINE_LATE step #-}
18131512 step (ArrayUnsafe _ start p) | p < start = return D.Stop
18141513 step (ArrayUnsafe contents start p) = do
1815− !x <- liftio $ peekAt p contents
1514+ !x <- liftio $ peekByteIndex p contents
18161515 return $ D.Yield x (ArrayUnsafe contents start (INDEX_PREV(p,a)))
18171516
18181517-- | Unfold an array into a stream in reverse order.
@@ -1843,7 +1542,7 @@ toListFB c n MutArray{..} = go arrStart
18431542 -- evaluated/written to before we peek at them.
18441543 -- XXX
18451544 let !x = unsafeInlineIO $ do
1846− r <- peekAt arrContents p
1545+ r <- peekByteIndex arrContents p
18471546 return r
18481547 in c x (go (PTR_NEXT(p,a)))
18491548-}
@@ -1860,21 +1559,21 @@ toList MutArray{..} = liftIO $ go arrStart
18601559
18611560 go p | assert (p <= arrEnd) (p == arrEnd) = return []
18621561 go p = do
1863− x <- peekAt p arrContents
1562+ x <- peekByteIndex p arrContents
18641563 (:) x <$> go (INDEX_NEXT(p,a))
18651564
1866−{-# INLINE_NORMAL toStreamWith #-}
1867−toStreamWith ::
1565+{-# INLINE_NORMAL toStreamDWith #-}
1566+toStreamDWith ::
18681567 forall m a. (Monad m, Unbox a)
18691568 => (forall b. IO b -> m b) -> MutArray a -> D.Stream m a
1870−toStreamWith liftio MutArray{..} = D.Stream step arrStart
1569+toStreamDWith liftio MutArray{..} = D.Stream step arrStart
18711570
18721571 where
18731572
18741573 {-# INLINE_LATE step #-}
18751574 step _ p | assert (p <= arrEnd) (p == arrEnd) = return D.Stop
18761575 step _ p = liftio $ do
1877− r <- peekAt p arrContents
1576+ r <- peekByteIndex p arrContents
18781577 return $ D.Yield r (INDEX_NEXT(p,a))
18791578
18801579-- | Convert a 'MutArray' into a stream.
@@ -1883,7 +1582,7 @@ toStreamWith liftio MutArray{..} = D.Stream step arrStart
18831582--
18841583{-# INLINE_NORMAL read #-}
18851584read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> D.Stream m a
1886−read = toStreamWith liftIO
1585+read = toStreamDWith liftIO
18871586
18881587{-# INLINE toStreamKWith #-}
18891588toStreamKWith ::
@@ -1895,18 +1594,18 @@ toStreamKWith liftio MutArray{..} = go arrStart
18951594
18961595 go p | assert (p <= arrEnd) (p == arrEnd) = K.nil
18971596 | otherwise =
1898− let elemM = peekAt p arrContents
1597+ let elemM = peekByteIndex p arrContents
18991598 in liftio elemM `K.consM` go (INDEX_NEXT(p,a))
19001599
19011600{-# INLINE toStreamK #-}
19021601toStreamK :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
19031602toStreamK = toStreamKWith liftIO
19041603
1905−{-# INLINE_NORMAL toStreamRevWith #-}
1906−toStreamRevWith ::
1604+{-# INLINE_NORMAL toStreamDRevWith #-}
1605+toStreamDRevWith ::
19071606 forall m a. (Monad m, Unbox a)
19081607 => (forall b. IO b -> m b) -> MutArray a -> D.Stream m a
1909−toStreamRevWith liftio MutArray{..} =
1608+toStreamDRevWith liftio MutArray{..} =
19101609 let p = INDEX_PREV(arrEnd,a)
19111610 in D.Stream step p
19121611
@@ -1915,7 +1614,7 @@ toStreamRevWith liftio MutArray{..} =
19151614 {-# INLINE_LATE step #-}
19161615 step _ p | p < arrStart = return D.Stop
19171616 step _ p = liftio $ do
1918− r <- peekAt p arrContents
1617+ r <- peekByteIndex p arrContents
19191618 return $ D.Yield r (INDEX_PREV(p,a))
19201619
19211620-- | Convert a 'MutArray' into a stream in reverse order.
@@ -1924,7 +1623,7 @@ toStreamRevWith liftio MutArray{..} =
19241623--
19251624{-# INLINE_NORMAL readRev #-}
19261625readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> D.Stream m a
1927−readRev = toStreamRevWith liftIO
1626+readRev = toStreamDRevWith liftIO
19281627
19291628{-# INLINE toStreamKRevWith #-}
19301629toStreamKRevWith ::
@@ -1938,7 +1637,7 @@ toStreamKRevWith liftio MutArray {..} =
19381637
19391638 go p | p < arrStart = K.nil
19401639 | otherwise =
1941− let elemM = peekAt p arrContents
1640+ let elemM = peekByteIndex p arrContents
19421641 in liftio elemM `K.consM` go (INDEX_PREV(p,a))
19431642
19441643{-# INLINE toStreamKRev #-}
@@ -1973,7 +1672,7 @@ foldr f z arr = D.foldr f z $ read arr
19731672
19741673-- XXX Keep the bound intact to not lose any free space? Perf impact?
19751674
1976−-- | @unsafeAppendN n arr@ appends up to @n@ input items to the supplied
1675+-- | @writeAppendNUnsafe n arr@ appends up to @n@ input items to the supplied
19771676-- array.
19781677--
19791678-- Unsafe: Do not drive the fold beyond @n@ elements, it will lead to memory
@@ -1982,12 +1681,13 @@ foldr f z arr = D.foldr f z $ read arr
19821681-- Any free space left in the array after appending @n@ elements is lost.
19831682--
19841683-- /Internal/
1985−{-# INLINE_NORMAL unsafeAppendN #-}
1986−unsafeAppendN :: forall m a. (MonadIO m, Unbox a) =>
1684+{-# INLINE_NORMAL writeAppendNUnsafe #-}
1685+writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) =>
19871686 Int
19881687 -> m (MutArray a)
19891688 -> Fold m a (MutArray a)
1990−unsafeAppendN n action = fmap fromArrayUnsafe $ FL.foldlM' step initial
1689+writeAppendNUnsafe n action =
1690+ fmap fromArrayUnsafe $ FL.foldlM' step initial
19911691
19921692 where
19931693
@@ -2000,59 +1700,38 @@ unsafeAppendN n action = fmap fromArrayUnsafe $ FL.foldlM' step initial
20001700 -- otherwise we lose that space.
20011701 arr1 <-
20021702 if free < needed
2003− then noinline reallocWith "unsafeAppendN" (+ needed) needed arr
1703+ then noinline reallocWith "writeAppendNUnsafeWith" (+ needed) needed arr
20041704 else return arr
20051705 return $ toArrayUnsafe arr1
20061706
20071707 step (ArrayUnsafe contents start end) x = do
2008− liftIO $ pokeAt end contents x
1708+ liftIO $ pokeByteIndex end contents x
20091709 return $ ArrayUnsafe contents start (INDEX_NEXT(end,a))
20101710
2011−{-# DEPRECATED writeAppendNUnsafe "Please use unsafeAppendN instead." #-}
2012−{-# INLINE writeAppendNUnsafe #-}
2013−writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) =>
2014− Int
2015− -> m (MutArray a)
2016− -> Fold m a (MutArray a)
2017−writeAppendNUnsafe = unsafeAppendN
2018−
20191711-- | Append @n@ elements to an existing array. Any free space left in the array
20201712-- after appending @n@ elements is lost.
20211713--
2022−-- >>> appendN n initial = Fold.take n (MutArray.unsafeAppendN n initial)
1714+-- >>> writeAppendN n initial = Fold.take n (MutArray.writeAppendNUnsafe n initial)
20231715--
2024−{-# INLINE_NORMAL appendN #-}
2025−appendN :: forall m a. (MonadIO m, Unbox a) =>
2026− Int -> m (MutArray a) -> Fold m a (MutArray a)
2027−appendN n initial = FL.take n (unsafeAppendN n initial)
2028−
2029−-- XXX Deprecate in major
2030−-- {-# DEPRECATED writeAppendN "Please use appendN instead." #-}
2031−{-# INLINE writeAppendN #-}
1716+{-# INLINE_NORMAL writeAppendN #-}
20321717writeAppendN :: forall m a. (MonadIO m, Unbox a) =>
20331718 Int -> m (MutArray a) -> Fold m a (MutArray a)
2034−writeAppendN = appendN
1719+writeAppendN n initial = FL.take n (writeAppendNUnsafe n initial)
20351720
2036−-- | @appendWith realloc action@ mutates the array generated by @action@ to
1721+-- | @writeAppendWith realloc action@ mutates the array generated by @action@ to
20371722-- append the input stream. If there is no reserved space available in the
20381723-- array it is reallocated to a size in bytes determined by @realloc oldSize@,
20391724-- where @oldSize@ is the current size of the array in bytes.
20401725--
20411726-- Note that the returned array may be a mutated version of original array.
20421727--
2043−-- >>> appendWith sizer = Fold.foldlM' (MutArray.snocWith sizer)
1728+-- >>> writeAppendWith sizer = Fold.foldlM' (MutArray.snocWith sizer)
20441729--
20451730-- /Pre-release/
2046−{-# INLINE appendWith #-}
2047−appendWith :: forall m a. (MonadIO m, Unbox a) =>
2048− (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
2049−appendWith sizer = FL.foldlM' (snocWith sizer)
2050−
2051−{-# DEPRECATED writeAppendWith "Please use appendWith instead." #-}
20521731{-# INLINE writeAppendWith #-}
20531732writeAppendWith :: forall m a. (MonadIO m, Unbox a) =>
20541733 (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
2055−writeAppendWith = appendWith
1734+writeAppendWith sizer = FL.foldlM' (snocWith sizer)
20561735
20571736-- | @append action@ mutates the array generated by @action@ to append the
20581737-- input stream. If there is no reserved space available in the array it is
@@ -2060,101 +1739,70 @@ writeAppendWith = appendWith
20601739--
20611740-- Note that the returned array may be a mutated version of original array.
20621741--
2063−-- >>> append = MutArray.appendWith (* 2)
1742+-- >>> writeAppend = MutArray.writeAppendWith (* 2)
20641743--
2065−{-# INLINE append #-}
2066−append :: forall m a. (MonadIO m, Unbox a) =>
2067− m (MutArray a) -> Fold m a (MutArray a)
2068−append = appendWith (* 2)
2069−
2070−-- XXX Deprecate in major
2071−-- {-# DEPRECATED writeAppend "Please use append instead." #-}
20721744{-# INLINE writeAppend #-}
20731745writeAppend :: forall m a. (MonadIO m, Unbox a) =>
20741746 m (MutArray a) -> Fold m a (MutArray a)
2075−writeAppend = append
1747+writeAppend = writeAppendWith (* 2)
20761748
20771749-- XXX We can carry bound as well in the state to make sure we do not lose the
20781750-- remaining capacity. Need to check perf impact.
20791751--
2080−-- | Like 'unsafeCreateOf' but takes a new array allocator @alloc size@
2081−-- function as argument.
1752+-- | Like 'writeNUnsafe' but takes a new array allocator @alloc size@ function
1753+-- as argument.
20821754--
2083−-- >>> unsafeCreateOfWith alloc n = MutArray.unsafeAppendN (alloc n) n
1755+-- >>> writeNWithUnsafe alloc n = MutArray.writeAppendNUnsafe (alloc n) n
20841756--
20851757-- /Pre-release/
2086−{-# INLINE_NORMAL unsafeCreateOfWith #-}
2087−unsafeCreateOfWith :: forall m a. (MonadIO m, Unbox a)
1758+{-# INLINE_NORMAL writeNWithUnsafe #-}
1759+writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a)
20881760 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2089−unsafeCreateOfWith alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
1761+writeNWithUnsafe alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
20901762
20911763 where
20921764
20931765 initial = toArrayUnsafe <$> alloc (max n 0)
20941766
20951767 step (ArrayUnsafe contents start end) x = do
2096− liftIO $ pokeAt end contents x
1768+ liftIO $ pokeByteIndex end contents x
20971769 return
20981770 $ ArrayUnsafe contents start (INDEX_NEXT(end,a))
20991771
2100−{-# DEPRECATED writeNWithUnsafe "Please use unsafeCreateOfWith instead." #-}
2101−{-# INLINE writeNWithUnsafe #-}
2102−writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a)
2103− => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2104−writeNWithUnsafe = unsafeCreateOfWith
2105−
21061772{-# INLINE_NORMAL writeNUnsafeAs #-}
21071773writeNUnsafeAs :: forall m a. (MonadIO m, Unbox a)
21081774 => PinnedState -> Int -> Fold m a (MutArray a)
2109−writeNUnsafeAs ps = unsafeCreateOfWith (newAs ps)
1775+writeNUnsafeAs ps = writeNWithUnsafe (newAs ps)
21101776
2111−-- | Like 'createOf' but does not check the array bounds when writing. The fold
1777+-- | Like 'writeN' but does not check the array bounds when writing. The fold
21121778-- driver must not call the step function more than 'n' times otherwise it will
21131779-- corrupt the memory and crash. This function exists mainly because any
21141780-- conditional in the step function blocks fusion causing 10x performance
21151781-- slowdown.
21161782--
2117−-- >>> unsafeCreateOf = MutArray.unsafeCreateOfWith MutArray.emptyOf
1783+-- >>> writeNUnsafe = MutArray.writeNWithUnsafe MutArray.new
21181784--
2119−{-# INLINE_NORMAL unsafeCreateOf #-}
2120−unsafeCreateOf :: forall m a. (MonadIO m, Unbox a)
2121− => Int -> Fold m a (MutArray a)
2122−unsafeCreateOf = writeNUnsafeAs Unpinned
2123−
2124−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
2125−{-# INLINE writeNUnsafe #-}
1785+{-# INLINE_NORMAL writeNUnsafe #-}
21261786writeNUnsafe :: forall m a. (MonadIO m, Unbox a)
21271787 => Int -> Fold m a (MutArray a)
2128−writeNUnsafe = unsafeCreateOf
2129−
2130−-- | Like 'unsafeCreateOf' but creates a pinned array.
2131−{-# INLINE_NORMAL unsafePinnedCreateOf #-}
2132−unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a)
2133− => Int -> Fold m a (MutArray a)
2134−unsafePinnedCreateOf = writeNUnsafeAs Pinned
1788+writeNUnsafe = writeNUnsafeAs Unpinned
21351789
2136−{-# DEPRECATED pinnedWriteNUnsafe "Please use unsafePinnedCreateOf instead." #-}
2137−{-# INLINE pinnedWriteNUnsafe #-}
1790+-- | Like 'writeNUnsafe' but creates a pinned array.
1791+{-# INLINE_NORMAL pinnedWriteNUnsafe #-}
21381792pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a)
21391793 => Int -> Fold m a (MutArray a)
2140−pinnedWriteNUnsafe = unsafePinnedCreateOf
1794+pinnedWriteNUnsafe = writeNUnsafeAs Pinned
21411795
2142−-- | @createOfWith alloc n@ folds a maximum of @n@ elements into an array
1796+-- | @writeNWith alloc n@ folds a maximum of @n@ elements into an array
21431797-- allocated using the @alloc@ function.
21441798--
2145−-- >>> createOfWith alloc n = Fold.take n (MutArray.unsafeCreateOfWith alloc n)
2146−-- >>> createOfWith alloc n = MutArray.appendN (alloc n) n
1799+-- >>> writeNWith alloc n = Fold.take n (MutArray.writeNWithUnsafe alloc n)
1800+-- >>> writeNWith alloc n = MutArray.writeAppendN (alloc n) n
21471801--
2148−{-# INLINE_NORMAL createOfWith #-}
2149−createOfWith :: forall m a. (MonadIO m, Unbox a)
2150− => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2151−createOfWith alloc n = FL.take n (unsafeCreateOfWith alloc n)
2152−
2153−-- {-# DEPRECATED writeNWith "Please use createOfWith instead." #-}
2154−{-# INLINE writeNWith #-}
1802+{-# INLINE_NORMAL writeNWith #-}
21551803writeNWith :: forall m a. (MonadIO m, Unbox a)
21561804 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
2157−writeNWith = createOfWith
1805+writeNWith alloc n = FL.take n (writeNWithUnsafe alloc n)
21581806
21591807{-# INLINE_NORMAL writeNAs #-}
21601808writeNAs ::
@@ -2162,42 +1810,28 @@ writeNAs ::
21621810 => PinnedState
21631811 -> Int
21641812 -> Fold m a (MutArray a)
2165−writeNAs ps = createOfWith (newAs ps)
1813+writeNAs ps = writeNWith (newAs ps)
21661814
2167−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
1815+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
21681816-- 'MutArray'.
21691817--
2170−-- >>> createOf = MutArray.createOfWith MutArray.new
2171−-- >>> createOf n = Fold.take n (MutArray.unsafeCreateOf n)
2172−-- >>> createOf n = MutArray.appendN n (MutArray.emptyOf n)
1818+-- >>> writeN = MutArray.writeNWith MutArray.new
1819+-- >>> writeN n = Fold.take n (MutArray.writeNUnsafe n)
1820+-- >>> writeN n = MutArray.writeAppendN n (MutArray.new n)
21731821--
2174−{-# INLINE_NORMAL createOf #-}
2175−createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2176−createOf = writeNAs Unpinned
2177−
2178−-- XXX Deprecate in major
2179−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
2180−{-# INLINE writeN #-}
1822+{-# INLINE_NORMAL writeN #-}
21811823writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2182−writeN = createOf
1824+writeN = writeNAs Unpinned
21831825
2184−-- | Like 'createOf' but creates a pinned array.
2185−{-# INLINE_NORMAL pinnedCreateOf #-}
2186−pinnedCreateOf ::
2187− forall m a. (MonadIO m, Unbox a)
2188− => Int
2189− -> Fold m a (MutArray a)
2190−pinnedCreateOf = writeNAs Pinned
2191−
2192−{-# DEPRECATED pinnedWriteN "Please use pinnedCreateOf instead." #-}
2193−{-# INLINE pinnedWriteN #-}
1826+-- | Like 'writeN' but creates a pinned array.
1827+{-# INLINE_NORMAL pinnedWriteN #-}
21941828pinnedWriteN ::
21951829 forall m a. (MonadIO m, Unbox a)
21961830 => Int
21971831 -> Fold m a (MutArray a)
2198−pinnedWriteN = pinnedCreateOf
1832+pinnedWriteN = writeNAs Pinned
21991833
2200−-- | Like unsafeCreateOfWith but writes the array in reverse order.
1834+-- | Like writeNWithUnsafe but writes the array in reverse order.
22011835--
22021836-- /Internal/
22031837{-# INLINE_NORMAL writeRevNWithUnsafe #-}
@@ -2214,11 +1848,11 @@ writeRevNWithUnsafe alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
22141848
22151849 step (ArrayUnsafe contents start end) x = do
22161850 let ptr = INDEX_PREV(start,a)
2217− liftIO $ pokeAt ptr contents x
1851+ liftIO $ pokeByteIndex ptr contents x
22181852 return
22191853 $ ArrayUnsafe contents ptr end
22201854
2221−-- | Like createOfWith but writes the array in reverse order.
1855+-- | Like writeNWith but writes the array in reverse order.
22221856--
22231857-- /Internal/
22241858{-# INLINE_NORMAL writeRevNWith #-}
@@ -2226,36 +1860,31 @@ writeRevNWith :: forall m a. (MonadIO m, Unbox a)
22261860 => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
22271861writeRevNWith alloc n = FL.take n (writeRevNWithUnsafe alloc n)
22281862
2229−-- | Like 'createOf' but writes the array in reverse order.
1863+-- | Like writeN but writes the array in reverse order.
22301864--
22311865-- /Pre-release/
2232−{-# INLINE_NORMAL revCreateOf #-}
2233−revCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2234−revCreateOf = writeRevNWith new
2235−
2236−{-# DEPRECATED writeRevN "Please use revCreateOf instead." #-}
2237−{-# INLINE writeRevN #-}
1866+{-# INLINE_NORMAL writeRevN #-}
22381867writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
2239−writeRevN = revCreateOf
1868+writeRevN = writeRevNWith new
22401869
22411870-- | @pinnedWriteNAligned align n@ folds a maximum of @n@ elements from the
22421871-- input stream to a 'MutArray' aligned to the given size.
22431872--
2244−-- >>> pinnedWriteNAligned align = MutArray.createOfWith (MutArray.pinnedNewAligned align)
2245−-- >>> pinnedWriteNAligned align n = MutArray.appendN n (MutArray.pinnedNewAligned align n)
1873+-- >>> pinnedWriteNAligned align = MutArray.writeNWith (MutArray.pinnedNewAligned align)
1874+-- >>> pinnedWriteNAligned align n = MutArray.writeAppendN n (MutArray.pinnedNewAligned align n)
22461875--
22471876-- /Pre-release/
22481877--
22491878{-# INLINE_NORMAL pinnedWriteNAligned #-}
22501879pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a)
22511880 => Int -> Int -> Fold m a (MutArray a)
2252−pinnedWriteNAligned align = createOfWith (pinnedNewAligned align)
1881+pinnedWriteNAligned align = writeNWith (pinnedNewAligned align)
22531882
22541883-- XXX Buffer to a list instead?
2255−
1884+--
22561885-- | Buffer a stream into a stream of arrays.
22571886--
2258−-- >>> buildChunks n = Fold.many (MutArray.createOf n) Fold.toStreamK
1887+-- >>> writeChunks n = Fold.many (MutArray.writeN n) Fold.toStreamK
22591888--
22601889-- Breaking an array into an array stream can be useful to consume a large
22611890-- array sequentially such that memory of the array is released incrementatlly.
@@ -2264,43 +1893,37 @@ pinnedWriteNAligned align = createOfWith (pinnedNewAligned align)
22641893--
22651894-- /Unimplemented/
22661895--
2267−{-# INLINE_NORMAL buildChunks #-}
2268−buildChunks :: (MonadIO m, Unbox a) =>
2269− Int -> Fold m a (StreamK n (MutArray a))
2270−buildChunks n = FL.many (createOf n) FL.toStreamK
2271−
2272−{-# DEPRECATED writeChunks "Please use buildChunks instead." #-}
2273−{-# INLINE writeChunks #-}
1896+{-# INLINE_NORMAL writeChunks #-}
22741897writeChunks :: (MonadIO m, Unbox a) =>
22751898 Int -> Fold m a (StreamK n (MutArray a))
2276−writeChunks = buildChunks
1899+writeChunks n = FL.many (writeN n) FL.toStreamK
22771900
22781901{-# INLINE_NORMAL writeWithAs #-}
22791902writeWithAs :: forall m a. (MonadIO m, Unbox a)
22801903 => PinnedState -> Int -> Fold m a (MutArray a)
2281−-- writeWithAs ps n = FL.rmapM rightSize $ appendWith (* 2) (newAs ps n)
1904+-- writeWithAs ps n = FL.rmapM rightSize $ writeAppendWith (* 2) (newAs ps n)
22821905writeWithAs ps elemCount =
22831906 FL.rmapM extract $ FL.foldlM' step initial
22841907
22851908 where
22861909
22871910 initial = do
2288− when (elemCount < 0) $ error "createWith: elemCount is negative"
2289− newAs ps elemCount
1911+ when (elemCount < 0) $ error "writeWith: elemCount is negative"
1912+ liftIO $ newAs ps elemCount
22901913
22911914 step arr@(MutArray _ start end bound) x
22921915 | INDEX_NEXT(end,a) > bound = do
22931916 let oldSize = end - start
22941917 newSize = max (oldSize * 2) 1
2295− arr1 <- liftIO $ reallocExplicitAs ps (SIZE_OF(a)) newSize arr
1918+ arr1 <- liftIO $ reallocExplicit (SIZE_OF(a)) newSize arr
22961919 snocUnsafe arr1 x
22971920 step arr x = snocUnsafe arr x
22981921
22991922 extract = liftIO . rightSize
23001923
2301−-- XXX Compare createWith with fromStreamD which uses an array of streams
2302−-- implementation. We can write this using buildChunks above if that is faster.
2303−-- If createWith is faster then we should use that to implement
1924+-- XXX Compare writeWith with fromStreamD which uses an array of streams
1925+-- implementation. We can write this using writeChunks above if that is faster.
1926+-- If writeWith is faster then we should use that to implement
23041927-- fromStreamD.
23051928--
23061929-- XXX The realloc based implementation needs to make one extra copy if we use
@@ -2315,56 +1938,39 @@ writeWithAs ps elemCount =
23151938-- XXX check if GHC's memory allocator is efficient enough. We can try the C
23161939-- malloc to compare against.
23171940
2318−-- | @createWith minCount@ folds the whole input to a single array. The array
1941+-- | @writeWith minCount@ folds the whole input to a single array. The array
23191942-- starts at a size big enough to hold minCount elements, the size is doubled
23201943-- every time the array needs to be grown.
23211944--
23221945-- /Caution! Do not use this on infinite streams./
23231946--
2324−-- >>> f n = MutArray.appendWith (* 2) (MutArray.emptyOf n)
2325−-- >>> createWith n = Fold.rmapM MutArray.rightSize (f n)
2326−-- >>> createWith n = Fold.rmapM MutArray.fromChunksK (MutArray.buildChunks n)
1947+-- >>> f n = MutArray.writeAppendWith (* 2) (MutArray.new n)
1948+-- >>> writeWith n = Fold.rmapM MutArray.rightSize (f n)
1949+-- >>> writeWith n = Fold.rmapM MutArray.fromArrayStreamK (MutArray.writeChunks n)
23271950--
23281951-- /Pre-release/
2329−{-# INLINE_NORMAL createWith #-}
2330−createWith :: forall m a. (MonadIO m, Unbox a)
2331− => Int -> Fold m a (MutArray a)
2332−-- createWith n = FL.rmapM rightSize $ appendWith (* 2) (emptyOf n)
2333−createWith = writeWithAs Unpinned
2334−
2335−{-# DEPRECATED writeWith "Please use createWith instead." #-}
2336−{-# INLINE writeWith #-}
1952+{-# INLINE_NORMAL writeWith #-}
23371953writeWith :: forall m a. (MonadIO m, Unbox a)
23381954 => Int -> Fold m a (MutArray a)
2339−writeWith = createWith
1955+-- writeWith n = FL.rmapM rightSize $ writeAppendWith (* 2) (new n)
1956+writeWith = writeWithAs Unpinned
23401957
23411958-- | Fold the whole input to a single array.
23421959--
2343−-- Same as 'createWith' using an initial array size of 'arrayChunkBytes' bytes
1960+-- Same as 'writeWith' using an initial array size of 'arrayChunkBytes' bytes
23441961-- rounded up to the element size.
23451962--
23461963-- /Caution! Do not use this on infinite streams./
23471964--
2348−{-# INLINE create #-}
2349−create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2350−create = createWith (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
2351−
2352−-- XXX Deprecate in major
2353−-- {-# DEPRECATED write "Please use create instead." #-}
23541965{-# INLINE write #-}
23551966write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2356−write = create
1967+write = writeWith (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
23571968
2358−-- | Like 'create' but creates a pinned array.
2359−{-# INLINE pinnedCreate #-}
2360−pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2361−pinnedCreate =
2362− writeWithAs Pinned (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
2363−
2364−{-# DEPRECATED pinnedWrite "Please use pinnedCreate instead." #-}
1969+-- | Like 'write' but creates a pinned array.
23651970{-# INLINE pinnedWrite #-}
23661971pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
2367−pinnedWrite = pinnedCreate
1972+pinnedWrite =
1973+ writeWithAs Pinned (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
23681974
23691975-------------------------------------------------------------------------------
23701976-- construct from streams, known size
@@ -2374,31 +1980,25 @@ pinnedWrite = pinnedCreate
23741980fromStreamDNAs :: forall m a. (MonadIO m, Unbox a)
23751981 => PinnedState -> Int -> D.Stream m a -> m (MutArray a)
23761982fromStreamDNAs ps limit str = do
2377− (arr :: MutArray a) <- newAs ps limit
1983+ (arr :: MutArray a) <- liftIO $ newAs ps limit
23781984 end <- D.foldlM' (fwrite (arrContents arr)) (return $ arrEnd arr) $ D.take limit str
23791985 return $ arr {arrEnd = end}
23801986
23811987 where
23821988
23831989 fwrite arrContents ptr x = do
2384− liftIO $ pokeAt ptr arrContents x
1990+ liftIO $ pokeByteIndex ptr arrContents x
23851991 return $ INDEX_NEXT(ptr,a)
23861992
2387−-- | Use the 'createOf' fold instead.
1993+-- | Use the 'writeN' fold instead.
23881994--
2389−-- >>> fromStreamN n = Stream.fold (MutArray.createOf n)
1995+-- >>> fromStreamDN n = Stream.fold (MutArray.writeN n)
23901996--
2391−{-# INLINE_NORMAL fromStreamN #-}
2392−fromStreamN :: forall m a. (MonadIO m, Unbox a)
2393− => Int -> D.Stream m a -> m (MutArray a)
2394−-- fromStreamDN n = D.fold (createOf n)
2395−fromStreamN = fromStreamDNAs Unpinned
2396−
2397−{-# DEPRECATED fromStreamDN "Please use fromStreamN instead." #-}
2398−{-# INLINE fromStreamDN #-}
1997+{-# INLINE_NORMAL fromStreamDN #-}
23991998fromStreamDN :: forall m a. (MonadIO m, Unbox a)
24001999 => Int -> D.Stream m a -> m (MutArray a)
2401−fromStreamDN = fromStreamN
2000+-- fromStreamDN n = D.fold (writeN n)
2001+fromStreamDN = fromStreamDNAs Unpinned
24022002
24032003-- | Create a 'MutArray' from the first N elements of a list. The array is
24042004-- allocated to size N, if the list terminates before N elements then the
@@ -2418,70 +2018,16 @@ pinnedFromListN n xs = fromStreamDNAs Pinned n $ D.fromList xs
24182018-- /Pre-release/
24192019{-# INLINE fromListRevN #-}
24202020fromListRevN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
2421−fromListRevN n xs = D.fold (revCreateOf n) $ D.fromList xs
2422−
2423−-- | Convert a pure stream in Identity monad to a mutable array.
2424−{-# INLINABLE fromPureStreamN #-}
2425−fromPureStreamN :: (MonadIO m, Unbox a) =>
2426− Int -> Stream Identity a -> m (MutArray a)
2427−fromPureStreamN n xs =
2428− D.fold (createOf n) $ D.morphInner (return . runIdentity) xs
2021+fromListRevN n xs = D.fold (writeRevN n) $ D.fromList xs
24292022
24302023-- | Convert a pure stream in Identity monad to a mutable array.
24312024{-# INLINABLE fromPureStream #-}
24322025fromPureStream :: (MonadIO m, Unbox a) => Stream Identity a -> m (MutArray a)
24332026fromPureStream xs =
2434− D.fold create $ D.morphInner (return . runIdentity) xs
2435−
2436−{-# INLINABLE fromPtrN #-}
2437−fromPtrN :: MonadIO m => Int -> Ptr Word8 -> m (MutArray Word8)
2438−fromPtrN len addr = do
2439− -- memcpy is better than stream copy when the size is known.
2440− -- XXX We can implement a stream copy in a similar way by streaming Word64
2441− -- first and then remaining Word8.
2442− arr <- new len
2443− _ <- unsafeAsPtr arr
2444− (\ptr -> liftIO $ c_memcpy ptr addr (fromIntegral len))
2445− return (arr {arrEnd = len})
2446−
2447−{-# INLINABLE fromByteStr# #-}
2448−fromByteStr# :: MonadIO m => Addr# -> m (MutArray Word8)
2449−fromByteStr# addr = do
2450− -- It is better to count the size first and allocate exact space.
2451− -- Also, memcpy is better than stream copy when the size is known.
2452− -- C strlen compares 4 bytes at a time, so is better than the stream
2453− -- version. https://github.com/bminor/glibc/blob/master/string/strlen.c
2454− -- XXX We can possibly use a stream of Word64 to do the same.
2455− -- fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
2456− len <- liftIO $ c_strlen (Ptr addr)
2457− let lenInt = fromIntegral len
2458− arr <- new lenInt
2459− _ <- unsafeAsPtr arr (\ptr -> liftIO $ c_memcpy ptr (Ptr addr) len)
2460− return (arr {arrEnd = lenInt})
2027+ liftIO $ D.fold write $ D.morphInner (return . runIdentity) xs
24612028
24622029-------------------------------------------------------------------------------
2463−-- convert a stream of arrays to a single array by reallocating and copying
2464−-------------------------------------------------------------------------------
2465−
2466−-- XXX Both of these implementations of splicing seem to perform equally well.
2467−-- We need to perform benchmarks over a range of sizes though.
2468−
2469−-- | Also see 'fromChunksK'.
2470−{-# INLINE fromChunksRealloced #-}
2471−fromChunksRealloced :: forall m a. (MonadIO m, Unbox a)
2472− => Stream m (MutArray a) -> m (MutArray a)
2473−fromChunksRealloced s = do
2474− res <- D.uncons s
2475− case res of
2476− Just (a, strm) -> do
2477− arr <- D.foldlM' spliceExp (pure a) strm
2478− -- Reallocation is exponential so there may be 50% empty space in
2479− -- worst case. One more reallocation to reclaim the space.
2480− rightSize arr
2481− Nothing -> pure nil
2482−
2483−-------------------------------------------------------------------------------
2484−-- convert a stream of arrays to a single array by buffering arrays first
2030+-- convert stream to a single array
24852031-------------------------------------------------------------------------------
24862032
24872033{-# INLINE arrayStreamKLength #-}
@@ -2491,72 +2037,34 @@ arrayStreamKLength as = K.foldl' (+) 0 (K.map length as)
24912037-- | Convert an array stream to an array. Note that this requires peak memory
24922038-- that is double the size of the array stream.
24932039--
2494−{-# INLINE fromChunkskAs #-}
2495−fromChunkskAs :: (Unbox a, MonadIO m) =>
2496− PinnedState -> StreamK m (MutArray a) -> m (MutArray a)
2497−fromChunkskAs ps as = do
2498− len <- arrayStreamKLength as
2499− arr <- newAs ps len
2500− -- XXX is StreamK fold faster or StreamD fold?
2501− K.foldlM' spliceUnsafe (pure arr) as
2502− -- fromStreamDN len $ D.unfoldMany reader $ D.fromStreamK as
2503−
2504−-- XXX Need to compare this with fromChunks and fromChunkList and keep the
2505−-- fastest or simplest one if all are equally fast.
2506−
2507−-- | Convert an array stream to an array. Note that this requires peak memory
2508−-- that is double the size of the array stream.
2509−--
2510−-- Also see 'fromChunksRealloced'.
2511−--
2512−{-# INLINE fromChunksK #-}
2513−fromChunksK :: (Unbox a, MonadIO m) =>
2514− StreamK m (MutArray a) -> m (MutArray a)
2515−fromChunksK = fromChunkskAs Unpinned
2516−
2517−{-# DEPRECATED fromArrayStreamK "Please use fromChunksK instead." #-}
25182040{-# INLINE fromArrayStreamK #-}
25192041fromArrayStreamK :: (Unbox a, MonadIO m) =>
25202042 StreamK m (MutArray a) -> m (MutArray a)
2521−fromArrayStreamK = fromChunksK
2043+fromArrayStreamK as = do
2044+ len <- arrayStreamKLength as
2045+ fromStreamDN len $ D.unfoldMany reader $ D.fromStreamK as
25222046
25232047{-# INLINE fromStreamDAs #-}
25242048fromStreamDAs ::
25252049 (MonadIO m, Unbox a) => PinnedState -> D.Stream m a -> m (MutArray a)
2526−fromStreamDAs ps m =
2527− arrayStreamKFromStreamDAs Unpinned m >>= fromChunkskAs ps
2528−
2529−-- | Create an 'Array' from a stream. This is useful when we want to create a
2530−-- single array from a stream of unknown size. 'createOf' is at least twice
2531−-- as efficient when the size is already known.
2532−--
2533−-- Note that if the input stream is too large memory allocation for the array
2534−-- may fail. When the stream size is not known, `chunksOf` followed by
2535−-- processing of indvidual arrays in the resulting stream should be preferred.
2536−--
2537−-- /Pre-release/
2538−{-# INLINE fromStream #-}
2539−fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
2540−fromStream = fromStreamDAs Unpinned
2050+fromStreamDAs ps m = arrayStreamKFromStreamDAs ps m >>= fromArrayStreamK
25412051
2542−-- fromStream (Stream m) = P.fold create m
25432052-- CAUTION: a very large number (millions) of arrays can degrade performance
25442053-- due to GC overhead because we need to buffer the arrays before we flatten
25452054-- all the arrays.
25462055--
2547−-- XXX Compare if this is faster or "fold create".
2056+-- XXX Compare if this is faster or "fold write".
25482057--
25492058-- | We could take the approach of doubling the memory allocation on each
25502059-- overflow. This would result in more or less the same amount of copying as in
25512060-- the chunking approach. However, if we have to shrink in the end then it may
25522061-- result in an extra copy of the entire data.
25532062--
2554−-- >>> fromStreamD = StreamD.fold MutArray.create
2063+-- >>> fromStreamD = StreamD.fold MutArray.write
25552064--
25562065{-# INLINE fromStreamD #-}
2557−{-# DEPRECATED fromStreamD "Please use fromStream instead." #-}
25582066fromStreamD :: (MonadIO m, Unbox a) => D.Stream m a -> m (MutArray a)
2559−fromStreamD = fromStream
2067+fromStreamD = fromStreamDAs Unpinned
25602068
25612069-- | Create a 'MutArray' from a list. The list must be of finite size.
25622070--
@@ -2577,54 +2085,26 @@ pinnedFromList xs = fromStreamDAs Pinned $ D.fromList xs
25772085fromListRev :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
25782086fromListRev xs = fromListRevN (Prelude.length xs) xs
25792087
2580−-------------------------------------------------------------------------------
2581−-- Cloning
2582−-------------------------------------------------------------------------------
2583−
2584−-- Arrays are aligned on 64-bit boundaries. The fastest way to copy an array is
2585−-- to unsafeCast it to Word64, read it, write it to Word64 array and unsafeCast
2586−-- it again. We can use SIMD read/write as well.
2587−
2588−{-# INLINE cloneAs #-}
2589−cloneAs ::
2590− ( MonadIO m
2591−#ifdef DEVBUILD
2592− , Unbox a
2593−#endif
2594− )
2595− => PinnedState -> MutArray a -> m (MutArray a)
2596−cloneAs ps src =
2597− do
2598− let startSrc = arrStart src
2599− srcLen = arrEnd src - startSrc
2600− newArrContents <-
2601− Unboxed.cloneSliceUnsafeAs ps startSrc srcLen (arrContents src)
2602− return $ MutArray newArrContents 0 srcLen srcLen
2603−
2604−{-# INLINE clone #-}
2605−clone ::
2606− ( MonadIO m
2607−#ifdef DEVBUILD
2608− , Unbox a
2609−#endif
2610− )
2611− => MutArray a -> m (MutArray a)
2612−clone = cloneAs Unpinned
2613−
2614−{-# INLINE pinnedClone #-}
2615−pinnedClone ::
2616− ( MonadIO m
2617−#ifdef DEVBUILD
2618− , Unbox a
2619−#endif
2620− )
2621− => MutArray a -> m (MutArray a)
2622−pinnedClone = cloneAs Pinned
2623−
26242088-------------------------------------------------------------------------------
26252089-- Combining
26262090-------------------------------------------------------------------------------
26272091
2092+-- | Put a sub range of a source array into a subrange of a destination array.
2093+-- This is not safe as it does not check the bounds.
2094+{-# INLINE putSliceUnsafe #-}
2095+putSliceUnsafe :: MonadIO m => MutArray a -> Int -> MutArray a -> Int -> Int -> m ()
2096+putSliceUnsafe src srcStartBytes dst dstStartBytes lenBytes = liftIO $ do
2097+ assertM(lenBytes <= arrBound dst - dstStartBytes)
2098+ assertM(lenBytes <= arrEnd src - srcStartBytes)
2099+ let !(I# srcStartBytes#) = srcStartBytes
2100+ !(I# dstStartBytes#) = dstStartBytes
2101+ !(I# lenBytes#) = lenBytes
2102+ let arrS# = getMutableByteArray# (arrContents src)
2103+ arrD# = getMutableByteArray# (arrContents dst)
2104+ IO $ \s# -> (# copyMutableByteArray#
2105+ arrS# srcStartBytes# arrD# dstStartBytes# lenBytes# s#
2106+ , () #)
2107+
26282108-- | Copy two arrays into a newly allocated array. If the first array is pinned
26292109-- the spliced array is also pinned.
26302110{-# INLINE spliceCopy #-}
@@ -2633,19 +2113,21 @@ spliceCopy :: forall m a. MonadIO m =>
26332113 Unbox a =>
26342114#endif
26352115 MutArray a -> MutArray a -> m (MutArray a)
2636−spliceCopy arr1 arr2 = do
2116+spliceCopy arr1 arr2 = liftIO $ do
26372117 let start1 = arrStart arr1
26382118 start2 = arrStart arr2
26392119 len1 = arrEnd arr1 - start1
26402120 len2 = arrEnd arr2 - start2
2641− let len = len1 + len2
2121+ let newLen = len1 + len2
26422122 newArrContents <-
26432123 if Unboxed.isPinned (arrContents arr1)
2644− then liftIO $ Unboxed.pinnedNew len
2645− else liftIO $ Unboxed.new len
2646− putSliceUnsafe (arrContents arr1) start1 newArrContents 0 len1
2647− putSliceUnsafe (arrContents arr2) start2 newArrContents len1 len2
2648− return $ MutArray newArrContents 0 len len
2124+ then Unboxed.pinnedNewBytes newLen
2125+ else Unboxed.newBytes newLen
2126+ let len = len1 + len2
2127+ newArr = MutArray newArrContents 0 len len
2128+ putSliceUnsafe arr1 start1 newArr 0 len1
2129+ putSliceUnsafe arr2 start2 newArr len1 len2
2130+ return newArr
26492131
26502132-- | Really really unsafe, appends the second array into the first array. If
26512133-- the first array does not have enough space it may cause silent data
@@ -2654,13 +2136,12 @@ spliceCopy arr1 arr2 = do
26542136spliceUnsafe :: MonadIO m =>
26552137 MutArray a -> MutArray a -> m (MutArray a)
26562138spliceUnsafe dst src =
2657− do
2139+ liftIO $ do
26582140 let startSrc = arrStart src
26592141 srcLen = arrEnd src - startSrc
26602142 endDst = arrEnd dst
26612143 assertM(endDst + srcLen <= arrBound dst)
2662− putSliceUnsafe
2663− (arrContents src) startSrc (arrContents dst) endDst srcLen
2144+ putSliceUnsafe src startSrc dst endDst srcLen
26642145 return $ dst {arrEnd = endDst + srcLen}
26652146
26662147-- | @spliceWith sizer dst src@ mutates @dst@ to append @src@. If there is no
@@ -2676,7 +2157,7 @@ spliceWith :: forall m a. (MonadIO m, Unbox a) =>
26762157 (Int -> Int -> Int) -> MutArray a -> MutArray a -> m (MutArray a)
26772158spliceWith sizer dst@(MutArray _ start end bound) src = do
26782159{-
2679− let f = appendWith (`sizer` byteLength src) (return dst)
2160+ let f = writeAppendWith (`sizer` byteLength src) (return dst)
26802161 in D.fold f (toStreamD src)
26812162-}
26822163 assert (end <= bound) (return ())
@@ -2692,7 +2173,7 @@ spliceWith sizer dst@(MutArray _ start end bound) src = do
26922173 $ "splice: newSize is less than the total size "
26932174 ++ "of arrays being appended. Please check the "
26942175 ++ "sizer function passed."
2695− realloc newSizeInBytes dst
2176+ liftIO $ realloc newSizeInBytes dst
26962177 else return dst
26972178 spliceUnsafe dst1 src
26982179
@@ -2729,24 +2210,14 @@ spliceExp = spliceWith (\l1 l2 -> max (l1 * 2) (l1 + l2))
27292210-- Splitting
27302211-------------------------------------------------------------------------------
27312212
2732−-- | Generate a stream of array slices using a predicate. The array element
2733−-- matching the predicate is dropped.
2734−--
2735−-- /Pre-release/
2736−{-# INLINE splitOn #-}
2737−splitOn :: (MonadIO m, Unbox a) =>
2738− (a -> Bool) -> MutArray a -> Stream m (MutArray a)
2739−splitOn predicate arr =
2740− fmap (\(i, len) -> getSliceUnsafe i len arr)
2741− $ D.indexOnSuffix predicate (read arr)
2742−
27432213-- | Drops the separator byte
27442214{-# INLINE breakOn #-}
27452215breakOn :: MonadIO m
27462216 => Word8 -> MutArray Word8 -> m (MutArray Word8, Maybe (MutArray Word8))
2747−breakOn sep arr@MutArray{..} = unsafeAsPtr arr $ \p -> liftIO $ do
2217+breakOn sep arr@MutArray{..} = asPtrUnsafe arr $ \p -> liftIO $ do
2218+ -- XXX Instead of using asPtrUnsafe (pinning memory) we can pass unlifted
2219+ -- Addr# to memchr and it should be safe (from ghc 8.4).
27482220 -- XXX We do not need memchr here, we can use a Haskell equivalent.
2749− -- Need efficient stream based primitives that work on Word64.
27502221 loc <- c_memchr p sep (fromIntegral $ byteLength arr)
27512222 let sepIndex = loc `minusPtr` p
27522223 return $
@@ -2767,41 +2238,32 @@ breakOn sep arr@MutArray{..} = unsafeAsPtr arr $ \p -> liftIO $ do
27672238 }
27682239 )
27692240
2770−-- | Like 'splitAt' but does not check whether the index is valid.
2771−--
2772−{-# INLINE unsafeSplitAt #-}
2773−unsafeSplitAt :: forall a. Unbox a =>
2774− Int -> MutArray a -> (MutArray a, MutArray a)
2775−unsafeSplitAt i MutArray{..} =
2776− let off = i * SIZE_OF(a)
2777− p = arrStart + off
2778− in ( MutArray
2779− { arrContents = arrContents
2780− , arrStart = arrStart
2781− , arrEnd = p
2782− , arrBound = p
2783− }
2784− , MutArray
2785− { arrContents = arrContents
2786− , arrStart = p
2787− , arrEnd = arrEnd
2788− , arrBound = arrBound
2789− }
2790− )
2791−
27922241-- | Create two slices of an array without copying the original array. The
27932242-- specified index @i@ is the first index of the second slice.
27942243--
2795−{-# INLINE splitAt #-}
27962244splitAt :: forall a. Unbox a => Int -> MutArray a -> (MutArray a, MutArray a)
2797−splitAt i arr =
2245+splitAt i arr@MutArray{..} =
27982246 let maxIndex = length arr - 1
27992247 in if i < 0
28002248 then error "sliceAt: negative array index"
28012249 else if i > maxIndex
28022250 then error $ "sliceAt: specified array index " ++ show i
28032251 ++ " is beyond the maximum index " ++ show maxIndex
2804− else unsafeSplitAt i arr
2252+ else let off = i * SIZE_OF(a)
2253+ p = arrStart + off
2254+ in ( MutArray
2255+ { arrContents = arrContents
2256+ , arrStart = arrStart
2257+ , arrEnd = p
2258+ , arrBound = p
2259+ }
2260+ , MutArray
2261+ { arrContents = arrContents
2262+ , arrStart = p
2263+ , arrEnd = arrEnd
2264+ , arrBound = arrBound
2265+ }
2266+ )
28052267
28062268-------------------------------------------------------------------------------
28072269-- Casting
@@ -2851,16 +2313,13 @@ cast arr =
28512313-- guarantees that unsafe calls will be performed in the calling thread. Making
28522314-- it safe to pass heap-allocated objects to unsafe functions.
28532315
2854−-- XXX Should we just name it asPtr, the unsafety is implicit for any pointer
2855−-- operations. And we are safe from Haskell perspective because we will be
2856−-- pinning the memory.
2857−--
2858−-- XXX we cannot pass the length of the ptr here as in some cases it may not be
2859−-- available e.g. a null terminated C string. However, we can create another
2860−-- flavor of the API e.g. asPtrN.
2316+-- Unsafe because of direct pointer operations. The user must ensure that they
2317+-- are writing within the legal bounds of the array. Should we just name it
2318+-- asPtr, the unsafety is implicit for any pointer operations. And we are safe
2319+-- from Haskell perspective because we will be pinning the memory.
28612320
2862−-- | Use a @MutArray a@ as @Ptr a@. This is useful when we want to pass an
2863−-- array as a pointer to some operating system call or to a "safe" FFI call.
2321+-- | Use an @MutArray a@ as @Ptr a@. This is useful when we want to pass an array
2322+-- as a pointer to some operating system call or to a "safe" FFI call.
28642323--
28652324-- If the array is not pinned it is copied to pinned memory before passing it
28662325-- to the monadic action.
@@ -2870,382 +2329,56 @@ cast arr =
28702329-- opportunistically before this operation occurs, to avoid the cost of a copy
28712330-- if possible.
28722331--
2873−-- /Unsafe/ because of direct pointer operations. The user must ensure that
2874−-- they are writing within the legal bounds of the array.
2332+-- /Unsafe/
28752333--
28762334-- /Pre-release/
28772335--
2878−{-# INLINE unsafePinnedAsPtr #-}
2879−unsafePinnedAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2880−unsafePinnedAsPtr arr f =
2881− Unboxed.unsafePinnedAsPtr
2882− (arrContents arr) (\ptr -> f (ptr `plusPtr` arrStart arr))
2883−
2884−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
2885−{-# INLINE asPtrUnsafe #-}
28862336asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2887−asPtrUnsafe = unsafePinnedAsPtr
2337+asPtrUnsafe arr f = do
2338+ contents <- liftIO $ Unboxed.pin $ arrContents arr
2339+ let !ptr = Ptr (byteArrayContents#
2340+ (unsafeCoerce# (getMutableByteArray# contents)))
28882341
2889−{-# INLINE unsafeAsPtr #-}
2890−unsafeAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
2891−unsafeAsPtr arr f =
2892− Unboxed.unsafeAsPtr
2893− (arrContents arr) (\ptr -> f (ptr `plusPtr` arrStart arr))
2342+ r <- f (ptr `plusPtr` arrStart arr)
2343+ liftIO $ touch contents
2344+ return r
28942345
28952346-------------------------------------------------------------------------------
28962347-- Equality
28972348-------------------------------------------------------------------------------
28982349
2899−-- | Byte compare two arrays. Compare the length of the arrays. If the length
2900−-- is equal, compare the lexicographical ordering of two underlying byte arrays
2901−-- otherwise return the result of length comparison.
2902−--
2903−-- /Unsafe/: Note that the 'Unbox' instance of sum types with constructors of
2904−-- different sizes may leave some memory uninitialized which can make byte
2905−-- comparison unreliable.
2350+-- | Compare the length of the arrays. If the length is equal, compare the
2351+-- lexicographical ordering of two underlying byte arrays otherwise return the
2352+-- result of length comparison.
29062353--
29072354-- /Pre-release/
2908−{-# INLINE byteCmp #-}
2909−byteCmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
2910−byteCmp arr1 arr2 = do
2911− let marr1 = getMutableByteArray# (arrContents arr1)
2912− marr2 = getMutableByteArray# (arrContents arr2)
2913− !(I# st1#) = arrStart arr1
2914− !(I# st2#) = arrStart arr2
2915− !(I# len#) = byteLength arr1
2916− case compare (byteLength arr1) (byteLength arr2) of
2917− EQ -> do
2918− r <- liftIO $ IO $ \s# ->
2919− let res =
2920− I#
2921− (compareByteArrays#
2922− (unsafeCoerce# marr1)
2923− st1#
2924− (unsafeCoerce# marr2)
2925− st2#
2926− len#)
2927− in (# s#, res #)
2928− return $ compare r 0
2929− x -> return x
2930−
29312355{-# INLINE cmp #-}
2932−{-# DEPRECATED cmp "Please use byteCmp instead." #-}
29332356cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
2934−cmp = byteCmp
2935−
2936−-- | Byte equality of two arrays.
2937−--
2938−-- >>> byteEq arr1 arr2 = (==) EQ $ MArray.byteCmp arr1 arr2
2939−--
2940−-- /Unsafe/: See 'byteCmp'.
2941−{-# INLINE byteEq #-}
2942−byteEq :: MonadIO m => MutArray a -> MutArray a -> m Bool
2943−byteEq arr1 arr2 = fmap (EQ ==) $ byteCmp arr1 arr2
2944−
2945−-------------------------------------------------------------------------------
2946−-- Compact
2947−-------------------------------------------------------------------------------
2948−
2949−-- Note: LE versions avoid an extra copy compared to GE. LE parser trades
2950−-- backtracking one array in lieu of avoiding a copy. However, LE and GE both
2951−-- can leave some memory unused. They can split the last array to fit it
2952−-- exactly in the space.
2953−
2954−{-# INLINE_NORMAL pCompactLeAs #-}
2955−pCompactLeAs ::
2956− forall m a. (MonadIO m, Unbox a)
2957− => PinnedState -> Int -> Parser (MutArray a) m (MutArray a)
2958−pCompactLeAs ps maxElems = Parser step initial extract
2959−
2960− where
2961−
2962− maxBytes = maxElems * SIZE_OF(a)
2963−
2964− functionName = "Streamly.Internal.Data.MutArray.pCompactLE"
2965−
2966− initial =
2967− return
2968− $ if maxElems <= 0
2969− then error
2970− $ functionName
2971− ++ ": the size of arrays ["
2972− ++ show maxElems ++ "] must be a natural number"
2973− else Parser.IPartial Nothing
2974−
2975− step Nothing arr =
2976− return
2977− $ let len = byteLength arr
2978− in if len >= maxBytes
2979− then Parser.Done 0 arr
2980− else Parser.Partial 0 (Just arr)
2981− -- XXX Split the last array to use the space more compactly.
2982− step (Just buf) arr =
2983− let len = byteLength buf + byteLength arr
2984− in if len > maxBytes
2985− then return $ Parser.Done 1 buf
2986− else do
2987− buf1 <-
2988− if byteCapacity buf < maxBytes
2989− then liftIO $ reallocExplicitAs
2990− ps (SIZE_OF(a)) maxBytes buf
2991− else return buf
2992− buf2 <- spliceUnsafe buf1 arr
2993− return $ Parser.Partial 0 (Just buf2)
2994−
2995− extract Nothing = return $ Parser.Done 0 nil
2996− extract (Just buf) = return $ Parser.Done 0 buf
2997−
2998−-- | Parser @pCompactLE maxElems@ coalesces adjacent arrays in the input stream
2999−-- only if the combined size would be less than or equal to @maxElems@
3000−-- elements. Note that it won't split an array if the original array is already
3001−-- larger than maxElems.
3002−--
3003−-- @maxElems@ must be greater than 0.
3004−--
3005−-- Generates unpinned arrays irrespective of the pinning status of input
3006−-- arrays.
3007−--
3008−-- /Internal/
3009−{-# INLINE pCompactLE #-}
3010−pCompactLE ::
3011− forall m a. (MonadIO m, Unbox a)
3012− => Int -> Parser (MutArray a) m (MutArray a)
3013−pCompactLE = pCompactLeAs Unpinned
3014−
3015−-- | Pinned version of 'pCompactLE'.
3016−{-# INLINE pPinnedCompactLE #-}
3017−pPinnedCompactLE ::
3018− forall m a. (MonadIO m, Unbox a)
3019− => Int -> Parser (MutArray a) m (MutArray a)
3020−pPinnedCompactLE = pCompactLeAs Pinned
3021−
3022−data SpliceState s arr
3023− = SpliceInitial s
3024− | SpliceBuffering s arr
3025− | SpliceYielding arr (SpliceState s arr)
3026− | SpliceFinish
3027−
3028−-- This mutates the first array (if it has space) to append values from the
3029−-- second one. This would work for immutable arrays as well because an
3030−-- immutable array never has additional space so a new array is allocated
3031−-- instead of mutating it.
3032−
3033−{-# INLINE_NORMAL compactLeAs #-}
3034−compactLeAs :: forall m a. (MonadIO m, Unbox a)
3035− => PinnedState -> Int -> D.Stream m (MutArray a) -> D.Stream m (MutArray a)
3036−compactLeAs ps maxElems (D.Stream step state) =
3037− D.Stream step' (SpliceInitial state)
3038−
3039− where
3040−
3041− maxBytes = maxElems * SIZE_OF(a)
3042−
3043− functionName = "Streamly.Internal.Data.MutArray.rCompactLE"
3044−
3045− {-# INLINE_LATE step' #-}
3046− step' gst (SpliceInitial st) = do
3047− when (maxElems <= 0) $
3048− -- XXX we can pass the module string from the higher level API
3049− error $ functionName ++ ": the size of arrays [" ++ show maxElems
3050− ++ "] must be a natural number"
3051− r <- step gst st
3052− case r of
3053− D.Yield arr s -> return $
3054− let len = byteLength arr
3055− in if len >= maxBytes
3056− then D.Skip (SpliceYielding arr (SpliceInitial s))
3057− else D.Skip (SpliceBuffering s arr)
3058− D.Skip s -> return $ D.Skip (SpliceInitial s)
3059− D.Stop -> return D.Stop
3060−
3061− -- XXX Split the last array to use the space more compactly.
3062− step' gst (SpliceBuffering st buf) = do
3063− r <- step gst st
3064− case r of
3065− D.Yield arr s -> do
3066− let len = byteLength buf + byteLength arr
3067− if len > maxBytes
3068− then return $
3069− D.Skip (SpliceYielding buf (SpliceBuffering s arr))
3070− else do
3071− buf1 <- if byteCapacity buf < maxBytes
3072− then liftIO $ reallocExplicitAs
3073− ps (SIZE_OF(a)) maxBytes buf
3074− else return buf
3075− buf2 <- spliceUnsafe buf1 arr
3076− return $ D.Skip (SpliceBuffering s buf2)
3077− D.Skip s -> return $ D.Skip (SpliceBuffering s buf)
3078− D.Stop -> return $ D.Skip (SpliceYielding buf SpliceFinish)
3079−
3080− step' _ SpliceFinish = return D.Stop
3081−
3082− step' _ (SpliceYielding arr next) = return $ D.Yield arr next
3083−
3084−
3085−{-# INLINE_NORMAL fCompactGeAs #-}
3086−fCompactGeAs ::
3087− forall m a. (MonadIO m, Unbox a)
3088− => PinnedState -> Int -> FL.Fold m (MutArray a) (MutArray a)
3089−fCompactGeAs ps minElems = Fold step initial extract extract
3090−
3091− where
3092−
3093− minBytes = minElems * SIZE_OF(a)
3094−
3095− functionName = "Streamly.Internal.Data.MutArray.fCompactGE"
3096−
3097− initial =
3098− return
3099− $ if minElems < 0
3100− then error
3101− $ functionName
3102− ++ ": the size of arrays ["
3103− ++ show minElems ++ "] must be a natural number"
3104− else FL.Partial Nothing
3105−
3106− step Nothing arr =
3107− return
3108− $ let len = byteLength arr
3109− in if len >= minBytes
3110− then FL.Done arr
3111− else FL.Partial (Just arr)
3112− -- XXX Buffer arrays as a list to avoid copy and reallocations
3113− step (Just buf) arr = do
3114− let len = byteLength buf + byteLength arr
3115− buf1 <-
3116− if byteCapacity buf < len
3117− then liftIO $ reallocExplicitAs
3118− ps (SIZE_OF(a)) (max minBytes len) buf
3119− else return buf
3120− buf2 <- spliceUnsafe buf1 arr
3121− if len >= minBytes
3122− then return $ FL.Done buf2
3123− else return $ FL.Partial (Just buf2)
3124−
3125− extract Nothing = return nil
3126− extract (Just buf) = return buf
3127−
3128−-- | Fold @fCompactGE minElems@ coalesces adjacent arrays in the input stream
3129−-- until the size becomes greater than or equal to @minElems@.
3130−--
3131−-- Generates unpinned arrays irrespective of the pinning status of input
3132−-- arrays.
3133−{-# INLINE fCompactGE #-}
3134−fCompactGE ::
3135− forall m a. (MonadIO m, Unbox a)
3136− => Int -> FL.Fold m (MutArray a) (MutArray a)
3137−fCompactGE = fCompactGeAs Unpinned
3138−
3139−-- | Pinned version of 'fCompactGE'.
3140−{-# INLINE fPinnedCompactGE #-}
3141−fPinnedCompactGE ::
3142− forall m a. (MonadIO m, Unbox a)
3143− => Int -> FL.Fold m (MutArray a) (MutArray a)
3144−fPinnedCompactGE = fCompactGeAs Pinned
3145−
3146−{-# INLINE_NORMAL lCompactGeAs #-}
3147−lCompactGeAs :: forall m a. (MonadIO m, Unbox a)
3148− => PinnedState -> Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3149−-- The fold version turns out to be a little bit slower.
3150−-- lCompactGeAs ps n = FL.many (fCompactGeAs ps n)
3151−lCompactGeAs ps minElems (Fold step1 initial1 _ final1) =
3152− Fold step initial extract final
3153−
3154− where
3155−
3156− minBytes = minElems * SIZE_OF(a)
3157−
3158− functionName = "Streamly.Internal.Data.MutArray.lCompactGE"
3159−
3160− initial = do
3161− when (minElems <= 0) $
3162− -- XXX we can pass the module string from the higher level API
3163− error $ functionName ++ ": the size of arrays ["
3164− ++ show minElems ++ "] must be a natural number"
3165−
3166− r <- initial1
3167− return $ first (Tuple' Nothing) r
3168−
3169− {-# INLINE runInner #-}
3170− runInner len acc buf =
3171− if len >= minBytes
3172− then do
3173− r <- step1 acc buf
3174− case r of
3175− FL.Done _ -> return $ FL.Done ()
3176− FL.Partial s -> do
3177− _ <- final1 s
3178− res <- initial1
3179− return $ first (Tuple' Nothing) res
3180− else return $ FL.Partial $ Tuple' (Just buf) acc
3181−
3182− step (Tuple' Nothing r1) arr =
3183− runInner (byteLength arr) r1 arr
3184−
3185− -- XXX Buffer arrays as a list to avoid copy and reallocations
3186− step (Tuple' (Just buf) r1) arr = do
3187− let len = byteLength buf + byteLength arr
3188− buf1 <- if byteCapacity buf < len
3189− then liftIO $ reallocExplicitAs
3190− ps (SIZE_OF(a)) (max minBytes len) buf
3191− else return buf
3192− buf2 <- spliceUnsafe buf1 arr
3193− runInner len r1 buf2
3194−
3195− -- XXX Several folds do extract >=> final, therefore, we need to make final
3196− -- return "m b" rather than using extract post it if we want extract to be
3197− -- partial.
3198− --
3199− -- extract forces the pending buffer to be sent to the fold which is not
3200− -- what we want.
3201− extract _ = error "lCompactGE: not designed for scanning"
3202−
3203− final (Tuple' Nothing r1) = final1 r1
3204− final (Tuple' (Just buf) r1) = do
3205− r <- step1 r1 buf
3206− case r of
3207− FL.Partial rr -> final1 rr
3208− FL.Done _ -> return ()
3209−
3210−-- | Like 'compactGE' but for transforming folds instead of stream.
3211−--
3212−-- >>> lCompactGE n = Fold.many (MutArray.fCompactGE n)
3213−--
3214−-- Generates unpinned arrays irrespective of the pinning status of input
3215−-- arrays.
3216−{-# INLINE lCompactGE #-}
3217−lCompactGE :: forall m a. (MonadIO m, Unbox a)
3218− => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3219−lCompactGE = lCompactGeAs Unpinned
3220−
3221−-- | Pinned version of 'lCompactGE'.
3222−{-# INLINE lPinnedCompactGE #-}
3223−lPinnedCompactGE :: forall m a. (MonadIO m, Unbox a)
3224− => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
3225−lPinnedCompactGE = lCompactGeAs Pinned
3226−
3227−-- | @compactGE n stream@ coalesces adjacent arrays in the @stream@ until
3228−-- the size becomes greater than or equal to @n@.
3229−--
3230−-- >>> compactGE n = Stream.foldMany (MutArray.fCompactGE n)
3231−--
3232−{-# INLINE compactGE #-}
3233−compactGE ::
3234− (MonadIO m, Unbox a)
3235− => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
3236−compactGE n = D.foldMany (fCompactGE n)
3237−
3238−-- | 'compactEQ n' coalesces adajacent arrays in the input stream to
3239−-- arrays of exact size @n@.
3240−--
3241−-- /Unimplemented/
3242−{-# INLINE compactEQ #-}
3243−compactEQ :: -- (MonadIO m, Unbox a) =>
3244− Int -> Stream m (MutArray a) -> Stream m (MutArray a)
3245−compactEQ _n = undefined -- D.parseManyD (pCompactEQ n)
2357+cmp arr1 arr2 =
2358+ liftIO
2359+ $ do
2360+ let marr1 = getMutableByteArray# (arrContents arr1)
2361+ marr2 = getMutableByteArray# (arrContents arr2)
2362+ !(I# st1#) = arrStart arr1
2363+ !(I# st2#) = arrStart arr2
2364+ !(I# len#) = byteLength arr1
2365+ case compare (byteLength arr1) (byteLength arr2) of
2366+ EQ -> do
2367+ r <- liftIO $ IO $ \s# ->
2368+ let res =
2369+ I#
2370+ (compareByteArrays#
2371+ (unsafeCoerce# marr1)
2372+ st1#
2373+ (unsafeCoerce# marr2)
2374+ st2#
2375+ len#)
2376+ in (# s#, res #)
2377+ return $ compare r 0
2378+ x -> return x
32462379
32472380-------------------------------------------------------------------------------
3248−-- In-place mutation algorithms
2381+-- NFData
32492382-------------------------------------------------------------------------------
32502383
32512384-- | Strip elements which match with predicate from both ends.
@@ -3274,7 +2407,7 @@ strip eq arr@MutArray{..} = liftIO $ do
32742407 getStart cur = do
32752408 if cur < arrEnd
32762409 then do
3277− r <- peekAt cur arrContents
2410+ r <- peekByteIndex cur arrContents
32782411 if eq r
32792412 then getStart (INDEX_NEXT(cur,a))
32802413 else return cur
@@ -3284,7 +2417,7 @@ strip eq arr@MutArray{..} = liftIO $ do
32842417 if cur > low
32852418 then do
32862419 let prev = INDEX_PREV(cur,a)
3287− r <- peekAt prev arrContents
2420+ r <- peekByteIndex prev arrContents
32882421 if eq r
32892422 then getLast prev low
32902423 else return cur
core/src/Streamly/Internal/Data/MutByteArray.hsdeleted−251
@@ -1,251 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−-- This is required as all the instances in this module are orphan instances.
4−{-# OPTIONS_GHC -fno-warn-orphans #-}
5−
6−-- |
7−-- Module : Streamly.Internal.Data.MutByteArray
8−-- Copyright : (c) 2023 Composewell Technologies
9−-- License : BSD3-3-Clause
10−-- Maintainer : streamly@composewell.com
11−-- Portability : GHC
12−--
13−
14−module Streamly.Internal.Data.MutByteArray
15− (
16− -- * MutByteArray
17− module Streamly.Internal.Data.MutByteArray.Type
18− -- * Unbox
19− , module Streamly.Internal.Data.Unbox
20− , module Streamly.Internal.Data.Unbox.TH
21− -- * Serialize
22− , module Streamly.Internal.Data.Serialize.Type
23− -- * Serialize TH
24− , module Streamly.Internal.Data.Serialize.TH
25− ) where
26−
27−--------------------------------------------------------------------------------
28−-- Imports
29−--------------------------------------------------------------------------------
30−
31−import Data.Proxy (Proxy(..))
32−import Streamly.Internal.Data.Array (Array(..))
33−import GHC.Exts (Int(..), sizeofByteArray#, unsafeCoerce#)
34−import GHC.Word (Word8)
35−
36−#if __GLASGOW_HASKELL__ >= 900
37−import GHC.Num.Integer (Integer(..))
38−#else
39−import GHC.Integer.GMP.Internals (Integer(..), BigNat(..))
40−#endif
41−
42−import Streamly.Internal.Data.MutByteArray.Type
43−import Streamly.Internal.Data.Serialize.TH
44−import Streamly.Internal.Data.Serialize.Type
45−import Streamly.Internal.Data.Unbox
46−import Streamly.Internal.Data.Unbox.TH
47−
48−--------------------------------------------------------------------------------
49−-- Common instances
50−--------------------------------------------------------------------------------
51−
52−-- Note
53−-- ====
54−--
55−-- Even a non-functional change such as changing the order of constructors will
56−-- change the instance derivation.
57−--
58−-- This will not pose a problem if both, encode, and decode are done by the same
59−-- version of the application. There *might* be a problem if version that
60−-- encodes differs from the version that decodes.
61−--
62−-- We need to add some compatibility tests using different versions of
63−-- dependencies.
64−--
65−-- Although such chages for the most basic types won't happen we need to detect
66−-- if it ever happens.
67−--
68−-- Should we worry about these kind of changes and this kind of compatibility?
69−-- This is a problem for all types of derivations that depend on the order of
70−-- constructors, for example, Enum.
71−
72−-- Note on Windows build
73−-- =====================
74−--
75−-- On Windows, having template haskell splices here fail the build with the
76−-- following error:
77−--
78−-- @
79−-- addLibrarySearchPath: C:\... (Win32 error 3): The system cannot find the path specified.
80−-- @
81−--
82−-- The error might be irrelavant but having these splices triggers it. We should
83−-- either fix the problem or avoid the use to template haskell splices in this
84−-- file.
85−--
86−-- Similar issue: https://github.com/haskell/cabal/issues/4741
87−
88−-- $(Serialize.deriveSerialize ''Maybe)
89−instance Serialize a => Serialize (Maybe a) where
90−
91− {-# INLINE addSizeTo #-}
92− addSizeTo acc x =
93− case x of
94− Nothing -> (acc + 1)
95− Just field0 -> (addSizeTo (acc + 1)) field0
96−
97− {-# INLINE deserializeAt #-}
98− deserializeAt initialOffset arr endOffset = do
99− (i0, tag) <- ((deserializeAt initialOffset) arr) endOffset
100− case tag :: Word8 of
101− 0 -> pure (i0, Nothing)
102− 1 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
103− pure (i1, Just a0)
104− _ -> error "Found invalid tag while peeking (Maybe a)"
105−
106− {-# INLINE serializeAt #-}
107− serializeAt initialOffset arr val =
108− case val of
109− Nothing -> do
110− i0 <- ((serializeAt initialOffset) arr) (0 :: Word8)
111− pure i0
112− Just field0 -> do
113− i0 <- ((serializeAt initialOffset) arr) (1 :: Word8)
114− i1 <- ((serializeAt i0) arr) field0
115− pure i1
116−
117−-- $(Serialize.deriveSerialize ''Either)
118−instance (Serialize a, Serialize b) => Serialize (Either a b) where
119−
120− {-# INLINE addSizeTo #-}
121− addSizeTo acc x =
122− case x of
123− Left field0 -> (addSizeTo (acc + 1)) field0
124− Right field0 -> (addSizeTo (acc + 1)) field0
125−
126− {-# INLINE deserializeAt #-}
127− deserializeAt initialOffset arr endOffset = do
128− (i0, tag) <- ((deserializeAt initialOffset) arr) endOffset
129− case tag :: Word8 of
130− 0 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
131− pure (i1, Left a0)
132− 1 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
133− pure (i1, Right a0)
134− _ -> error "Found invalid tag while peeking (Either a b)"
135−
136− {-# INLINE serializeAt #-}
137− serializeAt initialOffset arr val =
138− case val of
139− Left field0 -> do
140− i0 <- ((serializeAt initialOffset) arr) (0 :: Word8)
141− i1 <- ((serializeAt i0) arr) field0
142− pure i1
143− Right field0 -> do
144− i0 <- ((serializeAt initialOffset) arr) (1 :: Word8)
145− i1 <- ((serializeAt i0) arr) field0
146− pure i1
147−
148−instance Serialize (Proxy a) where
149−
150− {-# INLINE addSizeTo #-}
151− addSizeTo acc _ = (acc + 1)
152−
153− {-# INLINE deserializeAt #-}
154− deserializeAt initialOffset _ _ = pure ((initialOffset + 1), Proxy)
155−
156− {-# INLINE serializeAt #-}
157− serializeAt initialOffset _ _ = pure (initialOffset + 1)
158−
159−--------------------------------------------------------------------------------
160−-- Integer
161−--------------------------------------------------------------------------------
162−
163−data LiftedInteger
164− = LIS Int
165− | LIP (Array Word)
166− | LIN (Array Word)
167−
168−-- $(Serialize.deriveSerialize ''LiftedInteger)
169−instance Serialize LiftedInteger where
170−
171− {-# INLINE addSizeTo #-}
172− addSizeTo acc x =
173− case x of
174− LIS field0 -> (addSizeTo (acc + 1)) field0
175− LIP field0 -> (addSizeTo (acc + 1)) field0
176− LIN field0 -> (addSizeTo (acc + 1)) field0
177−
178− {-# INLINE deserializeAt #-}
179− deserializeAt initialOffset arr endOffset = do
180− (i0, tag) <- ((deserializeAt initialOffset) arr) endOffset
181− case tag :: Word8 of
182− 0 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
183− pure (i1, LIS a0)
184− 1 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
185− pure (i1, LIP a0)
186− 2 -> do (i1, a0) <- ((deserializeAt i0) arr) endOffset
187− pure (i1, LIN a0)
188− _ -> error "Found invalid tag while peeking (LiftedInteger)"
189−
190− {-# INLINE serializeAt #-}
191− serializeAt initialOffset arr val =
192− case val of
193− LIS field0 -> do
194− i0 <- ((serializeAt initialOffset) arr) (0 :: Word8)
195− i1 <- ((serializeAt i0) arr) field0
196− pure i1
197− LIP field0 -> do
198− i0 <- ((serializeAt initialOffset) arr) (1 :: Word8)
199− i1 <- ((serializeAt i0) arr) field0
200− pure i1
201− LIN field0 -> do
202− i0 <- ((serializeAt initialOffset) arr) (2 :: Word8)
203− i1 <- ((serializeAt i0) arr) field0
204− pure i1
205−
206−#if __GLASGOW_HASKELL__ >= 900
207−
208−{-# INLINE liftInteger #-}
209−liftInteger :: Integer -> LiftedInteger
210−liftInteger (IS x) = LIS (I# x)
211−liftInteger (IP x) =
212− LIP (Array (MutByteArray (unsafeCoerce# x)) 0 (I# (sizeofByteArray# x)))
213−liftInteger (IN x) =
214− LIN (Array (MutByteArray (unsafeCoerce# x)) 0 (I# (sizeofByteArray# x)))
215−
216−{-# INLINE unliftInteger #-}
217−unliftInteger :: LiftedInteger -> Integer
218−unliftInteger (LIS (I# x)) = IS x
219−unliftInteger (LIP (Array (MutByteArray x) _ _)) = IP (unsafeCoerce# x)
220−unliftInteger (LIN (Array (MutByteArray x) _ _)) = IN (unsafeCoerce# x)
221−
222−#else
223−
224−{-# INLINE liftInteger #-}
225−liftInteger :: Integer -> LiftedInteger
226−liftInteger (S# x) = LIS (I# x)
227−liftInteger (Jp# (BN# x)) =
228− LIP (Array (MutByteArray (unsafeCoerce# x)) 0 (I# (sizeofByteArray# x)))
229−liftInteger (Jn# (BN# x)) =
230− LIN (Array (MutByteArray (unsafeCoerce# x)) 0 (I# (sizeofByteArray# x)))
231−
232−{-# INLINE unliftInteger #-}
233−unliftInteger :: LiftedInteger -> Integer
234−unliftInteger (LIS (I# x)) = S# x
235−unliftInteger (LIP (Array (MutByteArray x) _ _)) =
236− Jp# (BN# (unsafeCoerce# x))
237−unliftInteger (LIN (Array (MutByteArray x) _ _)) =
238− Jn# (BN# (unsafeCoerce# x))
239−
240−#endif
241−
242−instance Serialize Integer where
243− {-# INLINE addSizeTo #-}
244− addSizeTo i a = addSizeTo i (liftInteger a)
245−
246− {-# INLINE deserializeAt #-}
247− deserializeAt off arr end =
248− fmap unliftInteger <$> deserializeAt off arr end
249−
250− {-# INLINE serializeAt #-}
251− serializeAt off arr val = serializeAt off arr (liftInteger val)
core/src/Streamly/Internal/Data/MutByteArray/Type.hsdeleted−321
@@ -1,321 +0,0 @@
1−{-# LANGUAGE UnboxedTuples #-}
2−
3−-- |
4−-- Module : Streamly.Internal.Data.MutByteArray.Type
5−-- Copyright : (c) 2023 Composewell Technologies
6−-- License : BSD3-3-Clause
7−-- Maintainer : streamly@composewell.com
8−-- Stability : experimental
9−-- Portability : GHC
10−--
11−module Streamly.Internal.Data.MutByteArray.Type
12− (
13− -- ** MutByteArray
14− MutByteArray(..)
15− , MutableByteArray
16− , getMutableByteArray# -- XXX getMutByteArray#
17−
18− -- ** Pinning
19− , PinnedState(..)
20− , isPinned
21− , pin
22− , unpin
23−
24− -- ** Allocation
25− , empty
26− , newBytesAs -- XXX should be removed
27− , new
28− , pinnedNew
29− , pinnedNewAlignedBytes -- XXX should be removed
30−
31− -- ** Access
32− , sizeOfMutableByteArray -- XXX length
33− , putSliceUnsafe
34− , cloneSliceUnsafeAs
35− , cloneSliceUnsafe
36− , pinnedCloneSliceUnsafe
37− , unsafePinnedAsPtr
38− , unsafeAsPtr
39−
40− -- ** Deprecated
41− , asPtrUnsafe
42− , nil
43− ) where
44−
45−import Control.Monad.IO.Class (MonadIO(..))
46−#ifdef DEBUG
47−import Control.Monad (when)
48−import Debug.Trace (trace)
49−#endif
50−import GHC.Base (IO(..))
51−import System.IO.Unsafe (unsafePerformIO)
52−
53−import GHC.Exts
54−
55−--------------------------------------------------------------------------------
56−-- The ArrayContents type
57−--------------------------------------------------------------------------------
58−
59−data PinnedState
60− = Pinned
61− | Unpinned deriving (Show, Eq)
62−
63−-- XXX can use UnliftedNewtypes
64−
65−-- | A lifted mutable byte array type wrapping @MutableByteArray# RealWorld@.
66−-- This is a low level array used to back high level unboxed arrays and
67−-- serialized data.
68−data MutByteArray = MutByteArray (MutableByteArray# RealWorld)
69−
70−{-# DEPRECATED MutableByteArray "Please use MutByteArray instead" #-}
71−type MutableByteArray = MutByteArray
72−
73−{-# INLINE getMutableByteArray# #-}
74−getMutableByteArray# :: MutByteArray -> MutableByteArray# RealWorld
75−getMutableByteArray# (MutByteArray mbarr) = mbarr
76−
77−-- | Return the size of the array in bytes.
78−{-# INLINE sizeOfMutableByteArray #-}
79−sizeOfMutableByteArray :: MutByteArray -> IO Int
80−sizeOfMutableByteArray (MutByteArray arr) =
81− IO $ \s ->
82− case getSizeofMutableByteArray# arr s of
83− (# s1, i #) -> (# s1, I# i #)
84−
85−{-# INLINE touch #-}
86−touch :: MutByteArray -> IO ()
87−touch (MutByteArray contents) =
88− IO $ \s -> case touch# contents s of s' -> (# s', () #)
89−
90−-- XXX Some functions in this module are "IO" and others are "m", we need to
91−-- make it consistent.
92−
93−-- XXX We can provide another API for "unsafe" FFI calls passing an unlifted
94−-- pointer to the FFI call. For unsafe calls we do not need to pin the array.
95−-- We can pass an unlifted pointer to the FFI routine to avoid GC kicking in
96−-- before the pointer is wrapped.
97−--
98−-- From the GHC manual:
99−--
100−-- GHC, since version 8.4, guarantees that garbage collection will never occur
101−-- during an unsafe call, even in the bytecode interpreter, and further
102−-- guarantees that unsafe calls will be performed in the calling thread. Making
103−-- it safe to pass heap-allocated objects to unsafe functions.
104−
105−-- | Use a @MutByteArray@ as @Ptr a@. This is useful when we want to pass
106−-- an array as a pointer to some operating system call or to a "safe" FFI call.
107−--
108−-- If the array is not pinned it is copied to pinned memory before passing it
109−-- to the monadic action.
110−--
111−-- /Performance Notes:/ Forces a copy if the array is not pinned. It is advised
112−-- that the programmer keeps this in mind and creates a pinned array
113−-- opportunistically before this operation occurs, to avoid the cost of a copy
114−-- if possible.
115−--
116−-- /Unsafe/ because of direct pointer operations. The user must ensure that
117−-- they are writing within the legal bounds of the array.
118−--
119−-- /Pre-release/
120−--
121−{-# INLINE unsafePinnedAsPtr #-}
122−unsafePinnedAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
123−unsafePinnedAsPtr arr f = do
124− contents <- liftIO $ pin arr
125− let !ptr = Ptr (byteArrayContents#
126− (unsafeCoerce# (getMutableByteArray# contents)))
127− r <- f ptr
128− liftIO $ touch contents
129− return r
130−
131−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
132−{-# INLINE asPtrUnsafe #-}
133−asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
134−asPtrUnsafe = unsafePinnedAsPtr
135−
136−-- | For use with unsafe FFI functions. Does not force pin the array memory.
137−{-# INLINE unsafeAsPtr #-}
138−unsafeAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
139−unsafeAsPtr arr f = do
140− let !ptr = Ptr (byteArrayContents#
141− (unsafeCoerce# (getMutableByteArray# arr)))
142− r <- f ptr
143− liftIO $ touch arr
144− return r
145−
146−--------------------------------------------------------------------------------
147−-- Creation
148−--------------------------------------------------------------------------------
149−
150−{-# NOINLINE empty #-}
151−empty :: MutByteArray
152−empty = unsafePerformIO $ new 0
153−
154−{-# DEPRECATED nil "Please use empty instead" #-}
155−nil :: MutByteArray
156−nil = empty
157−
158−-- XXX add "newRounded" to round up the large size to the next page boundary
159−-- and return the allocated size.
160−{-# INLINE new #-}
161−new :: Int -> IO MutByteArray
162−new nbytes | nbytes < 0 =
163− errorWithoutStackTrace "newByteArray: size must be >= 0"
164−new (I# nbytes) = IO $ \s ->
165− case newByteArray# nbytes s of
166− (# s', mbarr# #) ->
167− let c = MutByteArray mbarr#
168− in (# s', c #)
169−
170−{-# INLINE pinnedNew #-}
171−pinnedNew :: Int -> IO MutByteArray
172−pinnedNew nbytes | nbytes < 0 =
173− errorWithoutStackTrace "pinnedNew: size must be >= 0"
174−pinnedNew (I# nbytes) = IO $ \s ->
175− case newPinnedByteArray# nbytes s of
176− (# s', mbarr# #) ->
177− let c = MutByteArray mbarr#
178− in (# s', c #)
179−
180−{-# INLINE pinnedNewAlignedBytes #-}
181−pinnedNewAlignedBytes :: Int -> Int -> IO MutByteArray
182−pinnedNewAlignedBytes nbytes _align | nbytes < 0 =
183− errorWithoutStackTrace "pinnedNewAlignedBytes: size must be >= 0"
184−pinnedNewAlignedBytes (I# nbytes) (I# align) = IO $ \s ->
185− case newAlignedPinnedByteArray# nbytes align s of
186− (# s', mbarr# #) ->
187− let c = MutByteArray mbarr#
188− in (# s', c #)
189−
190−{-# INLINE newBytesAs #-}
191−newBytesAs :: PinnedState -> Int -> IO MutByteArray
192−newBytesAs Unpinned = new
193−newBytesAs Pinned = pinnedNew
194−
195−-------------------------------------------------------------------------------
196−-- Copying
197−-------------------------------------------------------------------------------
198−
199−-- Note: Array copy is more efficient than streaming copy.
200−-- CopyMutableByteArray# translates to genMemcpy in GHC/CmmToAsm/X86/CodeGen.hs
201−-- glibc memcpy copies bytes/words/pages - unrolls the loops:
202−-- https://github.com/bminor/glibc/blob/4290aed05135ae4c0272006442d147f2155e70d7/string/memcpy.c
203−-- https://github.com/bminor/glibc/blob/4290aed05135ae4c0272006442d147f2155e70d7/string/wordcopy.c
204−
205−-- | Put a sub range of a source array into a subrange of a destination array.
206−-- This is not safe as it does not check the bounds of neither the src array
207−-- nor the destination array.
208−{-# INLINE putSliceUnsafe #-}
209−putSliceUnsafe ::
210− MonadIO m
211− => MutByteArray
212− -> Int
213− -> MutByteArray
214− -> Int
215− -> Int
216− -> m ()
217−putSliceUnsafe src srcStartBytes dst dstStartBytes lenBytes = liftIO $ do
218−#ifdef DEBUG
219− srcLen <- sizeOfMutableByteArray src
220− dstLen <- sizeOfMutableByteArray dst
221− when (srcLen - srcStartBytes < lenBytes)
222− $ error $ "putSliceUnsafe: src overflow: start" ++ show srcStartBytes
223− ++ " end " ++ show srcLen ++ " len " ++ show lenBytes
224− when (dstLen - dstStartBytes < lenBytes)
225− $ error $ "putSliceUnsafe: dst overflow: start" ++ show dstStartBytes
226− ++ " end " ++ show dstLen ++ " len " ++ show lenBytes
227−#endif
228− let !(I# srcStartBytes#) = srcStartBytes
229− !(I# dstStartBytes#) = dstStartBytes
230− !(I# lenBytes#) = lenBytes
231− let arrS# = getMutableByteArray# src
232− arrD# = getMutableByteArray# dst
233− IO $ \s# -> (# copyMutableByteArray#
234− arrS# srcStartBytes# arrD# dstStartBytes# lenBytes# s#
235− , () #)
236−
237−-- | Unsafe as it does not check whether the start offset and length supplied
238−-- are valid inside the array.
239−{-# INLINE cloneSliceUnsafeAs #-}
240−cloneSliceUnsafeAs :: MonadIO m =>
241− PinnedState -> Int -> Int -> MutByteArray -> m MutByteArray
242−cloneSliceUnsafeAs ps srcOff srcLen src =
243− liftIO $ do
244− mba <- newBytesAs ps srcLen
245− putSliceUnsafe src srcOff mba 0 srcLen
246− return mba
247−
248−-- | @cloneSliceUnsafe offset len arr@ clones a slice of the supplied array
249−-- starting at the given offset and equal to the given length.
250−{-# INLINE cloneSliceUnsafe #-}
251−cloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
252−cloneSliceUnsafe = cloneSliceUnsafeAs Unpinned
253−
254−-- | @pinnedCloneSliceUnsafe offset len arr@
255−{-# INLINE pinnedCloneSliceUnsafe #-}
256−pinnedCloneSliceUnsafe :: MonadIO m =>
257− Int -> Int -> MutByteArray -> m MutByteArray
258−pinnedCloneSliceUnsafe = cloneSliceUnsafeAs Pinned
259−
260−-------------------------------------------------------------------------------
261−-- Pinning & Unpinning
262−-------------------------------------------------------------------------------
263−
264−-- | Return 'True' if the array is allocated in pinned memory.
265−{-# INLINE isPinned #-}
266−isPinned :: MutByteArray -> Bool
267−isPinned (MutByteArray arr#) =
268− let pinnedInt = I# (isMutableByteArrayPinned# arr#)
269− in pinnedInt /= 0
270−
271−
272−{-# INLINE cloneMutableArrayWith# #-}
273−cloneMutableArrayWith#
274− :: (Int# -> State# RealWorld -> (# State# RealWorld
275− , MutableByteArray# RealWorld #))
276− -> MutableByteArray# RealWorld
277− -> State# RealWorld
278− -> (# State# RealWorld, MutableByteArray# RealWorld #)
279−cloneMutableArrayWith# alloc# arr# s# =
280− case getSizeofMutableByteArray# arr# s# of
281− (# s1#, i# #) ->
282− case alloc# i# s1# of
283− (# s2#, arr1# #) ->
284− case copyMutableByteArray# arr# 0# arr1# 0# i# s2# of
285− s3# -> (# s3#, arr1# #)
286−
287−-- | Return a copy of the array in pinned memory if unpinned, else return the
288−-- original array.
289−{-# INLINE pin #-}
290−pin :: MutByteArray -> IO MutByteArray
291−pin arr@(MutByteArray marr#) =
292− if isPinned arr
293− then return arr
294− else
295−#ifdef DEBUG
296− do
297− -- XXX dump stack trace
298− trace ("pin: Copying array") (return ())
299−#endif
300− IO
301− $ \s# ->
302− case cloneMutableArrayWith# newPinnedByteArray# marr# s# of
303− (# s1#, marr1# #) -> (# s1#, MutByteArray marr1# #)
304−
305−-- | Return a copy of the array in unpinned memory if pinned, else return the
306−-- original array.
307−{-# INLINE unpin #-}
308−unpin :: MutByteArray -> IO MutByteArray
309−unpin arr@(MutByteArray marr#) =
310− if not (isPinned arr)
311− then return arr
312− else
313−#ifdef DEBUG
314− do
315− -- XXX dump stack trace
316− trace ("unpin: Copying array") (return ())
317−#endif
318− IO
319− $ \s# ->
320− case cloneMutableArrayWith# newByteArray# marr# s# of
321− (# s1#, marr1# #) -> (# s1#, MutByteArray marr1# #)
core/src/Streamly/Internal/Data/Parser.hs+143−135

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/Tee.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Tee.hsrenamed+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Type.hsrenamed+71−94

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/ParserDK.hsnew file+1544

This diff is too big to show here.

core/src/Streamly/Internal/Data/ParserK/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserK/Type.hsrenamed+84−291

This diff is too big to show here.

core/src/Streamly/Internal/Data/ParserK.hsdeleted−16

This diff is too big to show here.

core/src/Streamly/Internal/Data/Pipe.hs+8−5

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer.hs+13−6

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer/Source.hs+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer/Type.hs+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Refold/Type.hs+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring.hs+134−589

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring/Generic.hsdeleted−185

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring/Unboxed.hsnew file+616

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize.hsnew file+157

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH.hsdeleted−530

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hsdeleted−475

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/Common.hsdeleted−70

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/RecHeader.hsdeleted−411

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/Type.hsdeleted−317

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream.hs+20−20

This diff is too big to show here.

core/src/Streamly/Internal/Data/Array/Stream.hs → core/src/Streamly/Internal/Data/Stream/Chunked.hsrenamed+587−74

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Common.hs → core/src/Streamly/Internal/Data/Stream/Common.hsrenamed+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/MutChunked.hsnew file+323

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Container.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Container.hsrenamed+57−65

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Eliminate.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Eliminate.hsrenamed+33−39

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Exception.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Exception.hsrenamed+9−12

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Generate.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Generate.hsrenamed+26−27

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Lift.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Lift.hsrenamed+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Nesting.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Nesting.hsrenamed+228−115

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Step.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Step.hsrenamed+2−2

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Top.hsnew file+353

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Transform.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Transform.hsrenamed+40−66

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Transformer.hsrenamed+4−5

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Type.hsrenamed+34−169

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Alt.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Alt.hsrenamed+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Transformer.hsrenamed+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Type.hsrenamed+41−95

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Top.hsdeleted−480

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK.hs+134−350

This diff is too big to show here.

core/src/Streamly/Internal/Data/Time/Clock.hs+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Time/Units.hs+3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unbox.hs+280−295

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unbox/TH.hs+82−85

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold.hs+58−20

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold/Enumeration.hs+1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold/Type.hs+8−4

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/File.hs+13−11

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Handle.hs+42−33

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Path.hsdeleted−515

This diff is too big to show here.

core/src/Streamly/Internal/Data/Binary/Parser.hs → core/src/Streamly/Internal/Serialize/FromBytes.hsrenamed+4−10

This diff is too big to show here.

core/src/Streamly/Internal/Data/Binary/Stream.hs → core/src/Streamly/Internal/Serialize/ToBytes.hsrenamed+2−12

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Array.hs−6

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Parser.hs+4−12

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Stream.hs+25−53

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/String.hs+12−17

This diff is too big to show here.

core/src/Streamly/Unicode/Parser.hs+2−11

This diff is too big to show here.

core/src/Streamly/Unicode/Stream.hs−9

This diff is too big to show here.

core/src/Streamly/Unicode/String.hs+1−11

This diff is too big to show here.

core/streamly-core.cabal+78−146

This diff is too big to show here.

docs/Developer/optimization-guidelines.md−12

This diff is too big to show here.

docs/Developer/paths.rst+2−25

This diff is too big to show here.

docs/User/HowTo/Compiling.md+3−5

This diff is too big to show here.

docs/User/Project/ApiChangelogs/0.9.0-0.10.0.txtdeleted−424

This diff is too big to show here.

docs/User/Project/CONTRIBUTORS.md−8

This diff is too big to show here.

docs/User/Project/Changelog.md+10−25

This diff is too big to show here.

docs/User/Project/README.md+2−9

This diff is too big to show here.

docs/User/Project/Upgrading-0.8-to-0.9.md → docs/User/Project/Upgrading.mdrenamed+9−87

This diff is too big to show here.

docs/User/Tutorials/quick-overview.md+14−10

This diff is too big to show here.

docs/streamly-docs.cabal+1−1

This diff is too big to show here.

hie.yaml+2−2

This diff is too big to show here.

src/Streamly.hsnew file+624

This diff is too big to show here.

src/Streamly/Data/Fold/Prelude.hs+14−32

This diff is too big to show here.

src/Streamly/Data/Fold/Tee.hs+1−1

This diff is too big to show here.

src/Streamly/Data/Stream/Concurrent.hsnew file+194

This diff is too big to show here.

src/Streamly/Data/Stream/Exception.hsnew file+24

This diff is too big to show here.

src/Streamly/Data/Stream/MkType.hs+10−41

This diff is too big to show here.

src/Streamly/Data/Stream/Prelude.hs+15−217

This diff is too big to show here.

src/Streamly/Data/Stream/Time.hsnew file+37

This diff is too big to show here.

src/Streamly/Data/Unicode/Stream.hsnew file+99

This diff is too big to show here.

src/Streamly/Internal/Data/Array/ArrayMacros.hnew file+40

This diff is too big to show here.

src/Streamly/Internal/Data/Channel.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Async.hs+3−6

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent.hs+44−1

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent/Channel.hs+10−29

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent/Channel/Type.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Prelude.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/SVar.hs+7−8

This diff is too big to show here.

src/Streamly/Internal/Data/IsMap/HashMap.hs−6

This diff is too big to show here.

src/Streamly/Internal/Data/SVar.hs−1

This diff is too big to show here.

src/Streamly/Internal/Data/SmallArray.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Ahead.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Async.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Dispatcher.hs → src/Streamly/Internal/Data/Stream/Channel/Dispatcher.hsrenamed+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Types.hs → src/Streamly/Internal/Data/Stream/Channel/Types.hsrenamed+37−12

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Worker.hs → src/Streamly/Internal/Data/Stream/Channel/Worker.hsrenamed+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent.hs+42−51

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel.hs+14−17

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Append.hs+11−11

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Consumer.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Dispatcher.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Interleave.hs+9−9

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Operations.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Type.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Lifted.hs → src/Streamly/Internal/Data/Stream/Exception/Lifted.hsrenamed+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream.hs−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Combinators.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Common.hs+6−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Exception.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Expand.hs+3−7

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Generate.hs+6−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Reduce.hs+7−7

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Top.hs+3−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Transform.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Type.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/MkType.hs+34−33

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Parallel.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Prelude.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar.hsdeleted−18

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar/Eliminate.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar/Generate.hs+1−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Serial.hs+2−7

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Time.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Zip.hs+2−41

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Zip/Concurrent.hsnew file+50

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/ZipAsync.hs+2−3

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/Exception.hs+3−1

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/Prelude.hsdeleted−17

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/SVar.hs+1−2

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event.hs+2−5

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Darwin.hs+5−5

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Linux.hs+9−15

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Windows.hs+8−11

This diff is too big to show here.

src/Streamly/Internal/FileSystem/FD.hs+10−11

This diff is too big to show here.

src/Streamly/Internal/Network/Inet/TCP.hs+47−82

This diff is too big to show here.

src/Streamly/Internal/Network/Socket.hs+30−33

This diff is too big to show here.

src/Streamly/Internal/System/IOVec.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Unicode/Char.hs+7−9

This diff is too big to show here.

src/Streamly/Memory/Array.hsnew file+74

This diff is too big to show here.

src/Streamly/Network/Inet/TCP.hs+30−41

This diff is too big to show here.

src/Streamly/Network/Socket.hs+30−16

This diff is too big to show here.

streamly.cabal+64−86

This diff is too big to show here.

targets/Targets.hs+3−9

This diff is too big to show here.

test/Streamly/Test/Data/Array.hs+8−7

This diff is too big to show here.

test/Streamly/Test/Data/Array/Stream.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Data/Fold.hs+7−7

This diff is too big to show here.

test/Streamly/Test/Data/Fold/Window.hs+9−8

This diff is too big to show here.

test/Streamly/Test/Data/MutArray.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Data/Parser.hs+13−18

This diff is too big to show here.

test/Streamly/Test/Data/ParserK.hs+9−13

This diff is too big to show here.

test/Streamly/Test/Data/Ring/Unboxed.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Data/Serialize.hs+13−251

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/CompatV0.hsdeleted−46

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/CompatV1.hsdeleted−48

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/TH.hsdeleted−82

This diff is too big to show here.

test/Streamly/Test/Data/Stream/Concurrent.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Data/Unbox.hs+26−125

This diff is too big to show here.

test/Streamly/Test/Data/Unbox/TH.hs+10−11

This diff is too big to show here.

test/Streamly/Test/FileSystem/Event/Common.hs+1−1

This diff is too big to show here.

test/Streamly/Test/FileSystem/Handle.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Prelude/Serial.hs+3−7

This diff is too big to show here.

test/Streamly/Test/Serialize/Serializable.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Unicode/Char.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Unicode/Parser.hs+16−6

This diff is too big to show here.

test/Streamly/Test/Unicode/Stream.hs+1−1

This diff is too big to show here.

test/lib/Streamly/Test/Prelude/Common.hs+2−9

This diff is too big to show here.

test/streamly-tests.cabal+11−38

This diff is too big to show here.