on e249c4d help sign in
363 files changed+31802−38096
since splitsnapshots
unifiedsplit
.circleci/config.yml+277−73 .github/workflows/haskell.yml+89−108 .github/workflows/regression-check.yml+26−15 .hlint.ignore+2−8 .hlint.yaml+3−1 CHANGELOG.md+1−1 LICENSE+254 README.md+1−1 appveyor.yml−2 benchmark/NanoBenchmarks.hs+1−1 benchmark/Streamly/Benchmark/Data/Array/CommonImports.hs+2−2 benchmark/Streamly/Benchmark/Data/MutArray.hs → benchmark/Streamly/Benchmark/Data/Array/Mut.hs+9−10 benchmark/Streamly/Benchmark/Data/Array/Stream.hs+11−11 benchmark/Streamly/Benchmark/Data/Fold.hs+13−37 benchmark/Streamly/Benchmark/Data/Fold/Window.hs+53−66 benchmark/Streamly/Benchmark/Data/Parser.hs+90−421 benchmark/Streamly/Benchmark/Data/Parser/ParserD.hs+461 benchmark/Streamly/Benchmark/Data/Parser/ParserK.hs+167 benchmark/Streamly/Benchmark/Data/ParserK.hs−437 benchmark/Streamly/Benchmark/Data/Ring/Unboxed.hs+4−3 benchmark/Streamly/Benchmark/Data/Serialize.hs−591 benchmark/Streamly/Benchmark/Data/Serialize/RecCompatible.hs−24 benchmark/Streamly/Benchmark/Data/Serialize/RecNonCompatible.hs−21 benchmark/Streamly/Benchmark/Data/Serialize/TH.hs−76 benchmark/Streamly/Benchmark/Data/Stream.hs−49 benchmark/Streamly/Benchmark/Data/Stream/Common.hs+11−13 benchmark/Streamly/Benchmark/Data/Stream/Concurrent.hs+1−1 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentCommon.hs+8−9 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentEager.hs+2−2 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentInterleaved.hs+2−2 benchmark/Streamly/Benchmark/Data/Stream/ConcurrentOrdered.hs+2−2 benchmark/Streamly/Benchmark/Data/Stream/Eliminate.hs+7−9 benchmark/Streamly/Benchmark/Data/Stream/Exceptions.hs+6−16 benchmark/Streamly/Benchmark/Data/Stream/Expand.hs+7−15 benchmark/Streamly/Benchmark/Data/Stream/Generate.hs+4−4 benchmark/Streamly/Benchmark/Data/Stream/Lift.hs+5−5 benchmark/Streamly/Benchmark/Data/Stream/Reduce.hs+23−42 benchmark/Streamly/Benchmark/Data/Stream/Split.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/StreamD.hs+4−6 benchmark/Streamly/Benchmark/Data/Stream/StreamKAlt.hs → benchmark/Streamly/Benchmark/Data/Stream/StreamDK.hs+3−3 benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs+11−12 benchmark/Streamly/Benchmark/Data/Stream/ToStreamK.hs+6−5 benchmark/Streamly/Benchmark/Data/Stream/Transform.hs+6−6 benchmark/Streamly/Benchmark/Data/Unfold.hs+7−17 benchmark/Streamly/Benchmark/FileSystem/Handle.hs+1−1 benchmark/Streamly/Benchmark/FileSystem/Handle/Read.hs+34−33 benchmark/Streamly/Benchmark/FileSystem/Handle/ReadWrite.hs+12−11 benchmark/Streamly/Benchmark/Prelude/Adaptive.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Ahead.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Async.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Concurrent.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Merge.hs+2−2 benchmark/Streamly/Benchmark/Prelude/Parallel.hs+1−1 benchmark/Streamly/Benchmark/Prelude/Rate.hs+1−1 benchmark/Streamly/Benchmark/Prelude/WAsync.hs+1−1 benchmark/Streamly/Benchmark/Prelude/WSerial.hs+2−2 benchmark/Streamly/Benchmark/Prelude/ZipAsync.hs+1−1 benchmark/Streamly/Benchmark/Prelude/ZipSerial.hs+2−2 benchmark/Streamly/Benchmark/Unicode/Char.hs+2−2 benchmark/Streamly/Benchmark/Unicode/Parser.hs−107 benchmark/Streamly/Benchmark/Unicode/Stream.hs+10−28 benchmark/Streamly/Benchmark/Unicode/Utf8.hs+1−1 benchmark/bench-runner/Main.hs+19−59 benchmark/bench-runner/cabal.project−3 benchmark/bench-runner/default.nix+1−1 benchmark/lib/Streamly/Benchmark/Common.hs+2−2 benchmark/lib/Streamly/Benchmark/Common/Handle.hs+1−3 benchmark/lib/Streamly/Benchmark/Prelude.hs+1−1 benchmark/streamly-benchmarks.cabal+60−80 bin/mk-hscope.sh+1−1 bin/run-repl-quick.sh+1−1 cabal.project.packdiff+16−2 cabal.project.report+4−2 configure+9−9 configure.ac+1−1 core/Changelog.md−1 core/LICENSE−322 core/configure+9−9 core/configure.ac+1−1 core/docs/ApiChangelogs/0.1.0-0.2.0.txt−1471 core/docs/ApiChangelogs/0.1.0.txt−405 core/docs/ApiChangelogs/0.2.0-0.2.2.txt−306 core/docs/Changelog.md−97 core/docs/Readme.md−1 core/src/DocTestDataArray.hs−22 core/src/DocTestDataFold.hs−27 core/src/DocTestDataMutArray.hs−10 core/src/DocTestDataMutArrayGeneric.hs−10 core/src/DocTestDataParser.hs−20 core/src/DocTestDataParserK.hs−15 core/src/DocTestDataStream.hs−39 core/src/DocTestDataStreamK.hs−21 core/src/DocTestDataUnfold.hs−12 core/src/DocTestFileSystemHandle.hs−15 core/src/DocTestUnicodeParser.hs−9 core/src/DocTestUnicodeStream.hs−12 core/src/DocTestUnicodeString.hs−5 core/src/Streamly/Console/Stdio.hs+27−1 core/src/Streamly/Data/Array.hs+111−115 core/src/Streamly/Data/Array/Generic.hs+11−19 core/src/Streamly/Data/Array/Generic/Mut.hs+38 core/src/Streamly/Data/Array/Mut.hs+66 core/src/Streamly/Data/Fold.hs+54−105 core/src/Streamly/Data/MutArray.hs−110 core/src/Streamly/Data/MutArray/Generic.hs−70 core/src/Streamly/Data/MutByteArray.hs−106 core/src/Streamly/Data/Parser.hs+12−147 core/src/Streamly/Data/ParserK.hs−139 core/src/Streamly/Data/Stream.hs+125−371 core/src/Streamly/Data/Stream/StreamDK.hs+505 core/src/Streamly/Data/Stream/StreamK.hs+93 core/src/Streamly/Data/Stream/Zip.hs+16 core/src/Streamly/Data/StreamK.hs−194 core/src/Streamly/Data/Unfold.hs+85−98 core/src/Streamly/FileSystem/Handle.hs+29−15 core/src/Streamly/Internal/Console/Stdio.hs+2−2 core/src/Streamly/Internal/Data/Array.hs+187−644 core/src/Streamly/Internal/Data/Array/Generic.hs+185−152 core/src/Streamly/Internal/Data/Array/Generic/Mut/Type.hs+608 core/src/Streamly/Internal/Data/Array/Mut.hs+100 core/src/Streamly/Internal/Data/Array/Mut/Stream.hs+323 core/src/Streamly/Internal/Data/Array/Mut/Type.hs+2350 core/src/Streamly/Internal/Data/Array/Type.hs+117−668 core/src/Streamly/Internal/Data/Builder.hs+9−11 core/src/Streamly/Internal/Data/Fold.hs+2507−11 core/src/Streamly/Internal/Data/Fold/Chunked.hs+22−24 core/src/Streamly/Internal/Data/Fold/Combinators.hs−2452 core/src/Streamly/Internal/Data/Fold/Container.hs+40−119 core/src/Streamly/Internal/Data/Fold/Step.hs+2−2 core/src/Streamly/Internal/Data/Fold/Tee.hs+7−8 core/src/Streamly/Internal/Data/Fold/Type.hs+167−402 core/src/Streamly/Internal/Data/Fold/Window.hs+53−59 core/src/Streamly/Internal/Data/IOFinalizer.hs+2−12 core/src/Streamly/Internal/Data/IORef/Unboxed.hs+22−23 core/src/Streamly/Internal/Data/IsMap.hs−8 core/src/Streamly/Internal/Data/List.hs+16−81 core/src/Streamly/Internal/Data/MutArray.hs−207 core/src/Streamly/Internal/Data/MutArray/Generic.hs−947 core/src/Streamly/Internal/Data/MutArray/Stream.hs−145 core/src/Streamly/Internal/Data/MutArray/Type.hs−3318 core/src/Streamly/Internal/Data/MutByteArray.hs−251 core/src/Streamly/Internal/Data/MutByteArray/Type.hs−321 core/src/Streamly/Internal/Data/Parser.hs+594−2671 core/src/Streamly/Internal/Data/Parser/Chunked.hs+327 core/src/Streamly/Internal/Data/Parser/Chunked/Type.hs+329 core/src/Streamly/Internal/Data/Parser/ParserD.hs+2247 core/src/Streamly/Internal/Data/Parser/Tee.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Tee.hs+1−1 core/src/Streamly/Internal/Data/Parser/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Type.hs+271−212 core/src/Streamly/Internal/Data/Parser/ParserK/Type.hs+313 core/src/Streamly/Internal/Data/ParserK.hs−16 core/src/Streamly/Internal/Data/ParserK/Type.hs−752 core/src/Streamly/Internal/Data/Pipe.hs+10−7 core/src/Streamly/Internal/Data/Producer.hs+13−6 core/src/Streamly/Internal/Data/Producer/Source.hs+19−34 core/src/Streamly/Internal/Data/Producer/Type.hs+1−1 core/src/Streamly/Internal/Data/Refold/Type.hs+2−2 core/src/Streamly/Internal/Data/Ring.hs+31−603 core/src/Streamly/Internal/Data/Ring/Generic.hs−185 core/src/Streamly/Internal/Data/Ring/Unboxed.hs+615 core/src/Streamly/Internal/Data/Serialize/TH.hs−530 core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hs−475 core/src/Streamly/Internal/Data/Serialize/TH/Common.hs−70 core/src/Streamly/Internal/Data/Serialize/TH/RecHeader.hs−411 core/src/Streamly/Internal/Data/Serialize/Type.hs−317 core/src/Streamly/Internal/Data/Stream.hs+3−31 core/src/Streamly/Internal/Data/Stream/Bottom.hs+670 core/src/Streamly/Internal/Data/Array/Stream.hs → core/src/Streamly/Internal/Data/Stream/Chunked.hs+581−92 src/Streamly/Internal/Data/Stream/Common.hs → core/src/Streamly/Internal/Data/Stream/Common.hs+12−12 core/src/Streamly/Internal/Data/Stream/Cross.hs+143 core/src/Streamly/Internal/Data/Stream/Eliminate.hs+228−697 core/src/Streamly/Internal/Data/Stream/Enumerate.hs+560 core/src/Streamly/Internal/Data/Stream/Exception.hs+90−369 core/src/Streamly/Internal/Data/Stream/Expand.hs+893 core/src/Streamly/Internal/Data/Stream/Generate.hs+217−948 core/src/Streamly/Internal/Data/Stream/Lift.hs+36−70 core/src/Streamly/Internal/Data/Stream/Reduce.hs+444 core/src/Streamly/Internal/Data/Stream/StreamD.hs+29−3 core/src/Streamly/Internal/Data/Stream/Container.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Container.hs+63−70 core/src/Streamly/Internal/Data/Stream/StreamD/Eliminate.hs+826 core/src/Streamly/Internal/Data/Stream/StreamD/Exception.hs+480 core/src/Streamly/Internal/Data/Stream/StreamD/Generate.hs+1211 core/src/Streamly/Internal/Data/Stream/StreamD/Lift.hs+131 core/src/Streamly/Internal/Data/Stream/Nesting.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Nesting.hs+252−154 core/src/Streamly/Internal/Data/Stream/Step.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Step.hs+2−2 core/src/Streamly/Internal/Data/Stream/StreamD/Top.hs+356 core/src/Streamly/Internal/Data/Stream/StreamD/Transform.hs+1844 core/src/Streamly/Internal/Data/Stream/StreamD/Transformer.hs+185 core/src/Streamly/Internal/Data/Stream/StreamD/Type.hs+2022 core/src/Streamly/Internal/Data/Stream/StreamDK.hs+52 core/src/Streamly/Internal/Data/StreamK.hs → core/src/Streamly/Internal/Data/Stream/StreamK.hs+249−713 core/src/Streamly/Internal/Data/StreamK/Alt.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Alt.hs+1−1 core/src/Streamly/Internal/Data/StreamK/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Transformer.hs+8−8 core/src/Streamly/Internal/Data/StreamK/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Type.hs+256−336 core/src/Streamly/Internal/Data/Stream/Top.hs−480 core/src/Streamly/Internal/Data/Stream/Transform.hs+282−1197 core/src/Streamly/Internal/Data/Stream/Transformer.hs+43−91 core/src/Streamly/Internal/Data/Stream/Type.hs+229−1958 core/src/Streamly/Internal/Data/Stream/Zip.hs+91 core/src/Streamly/Internal/Data/Time/Clock.hs+3−3 core/src/Streamly/Internal/Data/Time/Units.hs+4−1 core/src/Streamly/Internal/Data/Unbox/TH.hs−498 core/src/Streamly/Internal/Data/Unbox.hs → core/src/Streamly/Internal/Data/Unboxed.hs+276−344 core/src/Streamly/Internal/Data/Unfold.hs+194−32 core/src/Streamly/Internal/Data/Unfold/Enumeration.hs+45−14 core/src/Streamly/Internal/Data/Unfold/Type.hs+20−127 core/src/Streamly/Internal/FileSystem/Dir.hs+23−70 core/src/Streamly/Internal/FileSystem/File.hs+32−31 core/src/Streamly/Internal/FileSystem/Handle.hs+52−41 core/src/Streamly/Internal/FileSystem/Path.hs−515 core/src/Streamly/Internal/Data/Binary/Parser.hs → core/src/Streamly/Internal/Serialize/FromBytes.hs+13−29 core/src/Streamly/Internal/Data/Binary/Stream.hs → core/src/Streamly/Internal/Serialize/ToBytes.hs+9−31 core/src/Streamly/Internal/Unicode/Array.hs−6 core/src/Streamly/Internal/Unicode/Parser.hs+25−354 core/src/Streamly/Internal/Unicode/Stream.hs+68−79 core/src/Streamly/Internal/Unicode/String.hs+12−17 core/src/Streamly/Unicode/Parser.hs+2−12 core/src/Streamly/Unicode/Stream.hs+10−14 core/src/Streamly/Unicode/String.hs+1−15 core/src/inline.hs+1−1 core/streamly-core.cabal+115−197 docs/Developer/Benchmarks.md−8 docs/Developer/MAINTAINING.md+10−7 docs/Developer/module-organization.md+1−1 docs/Developer/optimization-guidelines.md+1−13 docs/Developer/paths.rst+2−25 docs/Documentation.md−5 docs/User/Explanatory/design.md−42 docs/User/HowTo/Compiling.md+14−36 docs/User/HowTo/streamly-vs-async.md−2 docs/User/HowTo/streamly-vs-lists.md−2 docs/User/Project/ApiChangelogs/0.8.3-0.9.0.txt−175 docs/User/Project/ApiChangelogs/0.9.0-0.10.0.txt−424 docs/User/Project/README.md−141 docs/User/Project/Upgrading-0.8-to-0.9.md−384 docs/User/Project/ApiChangelogs/0.8.3.txt → docs/User/ProjectRelated/API-changelog.txt docs/User/Project/CONTRIBUTORS.md → docs/User/ProjectRelated/CONTRIBUTORS.md+1−9 docs/User/Project/Changelog.md → docs/User/ProjectRelated/Changelog.md+32−92 docs/User/Project/Credits.md → docs/User/ProjectRelated/Credits.md docs/User/Project/Github.link → docs/User/ProjectRelated/Github.link docs/User/ProjectRelated/README.md+166 docs/User/Project/Roadmap.link → docs/User/ProjectRelated/Roadmap.link docs/User/Tutorials/ConcurrentStreams.hs−2 docs/User/Tutorials/ReactiveProgramming.hs−2 docs/User/Tutorials/Tutorial.hs−2 docs/User/Tutorials/before-you-begin.md−77 docs/User/Tutorials/haskell-lazy-evaluation.md−215 docs/User/Tutorials/quick-overview.md+42−21 docs/User/Tutorials/getting-started.md → docs/User/Tutorials/setup-and-usage.md+25−28 docs/User/Tutorials/why-haskell-streamly.md−5 docs/streamly-docs.cabal+6−8 hie.yaml+14−16 src/Streamly.hs+624 src/Streamly/Data/Fold/Prelude.hs−65 src/Streamly/Data/Fold/Tee.hs+1−2 src/Streamly/Data/Stream/Concurrent.hs+184 src/Streamly/Data/Stream/Exception.hs+24 src/Streamly/Data/Stream/MkType.hs+10−41 src/Streamly/Data/Stream/Prelude.hs+15−217 src/Streamly/Data/Stream/Time.hs+37 src/Streamly/Data/Unicode/Stream.hs+99 src/Streamly/Internal/Control/ForkLifted.hs+4−17 src/Streamly/Internal/Data/Array/ArrayMacros.h+40 src/Streamly/Internal/Data/Channel.hs−19 src/Streamly/Internal/Data/Fold/Async.hs+6−18 src/Streamly/Internal/Data/Fold/Concurrent.hs+49−37 src/Streamly/Internal/Data/Fold/Concurrent/Channel.hs+10−31 src/Streamly/Internal/Data/Fold/Concurrent/Channel/Type.hs+7−8 src/Streamly/Internal/Data/Fold/Prelude.hs−19 src/Streamly/Internal/Data/Fold/SVar.hs+7−8 src/Streamly/Internal/Data/IsMap/HashMap.hs−10 src/Streamly/Internal/Data/SVar.hs−1 src/Streamly/Internal/Data/SmallArray.hs+4−4 src/Streamly/Internal/Data/Stream/Ahead.hs+3−15 src/Streamly/Internal/Data/Stream/Async.hs+3−22 src/Streamly/Internal/Data/Channel/Dispatcher.hs → src/Streamly/Internal/Data/Stream/Channel/Dispatcher.hs+3−3 src/Streamly/Internal/Data/Channel/Types.hs → src/Streamly/Internal/Data/Stream/Channel/Types.hs+32−22 src/Streamly/Internal/Data/Channel/Worker.hs → src/Streamly/Internal/Data/Stream/Channel/Worker.hs+3−3 src/Streamly/Internal/Data/Stream/Concurrent.hs+90−219 src/Streamly/Internal/Data/Stream/Concurrent/Channel.hs+19−22 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Append.hs+37−37 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Consumer.hs+2−2 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Dispatcher.hs+3−3 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Interleave.hs+14−16 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Operations.hs+8−8 src/Streamly/Internal/Data/Stream/Concurrent/Channel/Type.hs+5−5 src/Streamly/Internal/Data/Stream/Lifted.hs → src/Streamly/Internal/Data/Stream/Exception/Lifted.hs+12−16 src/Streamly/Internal/Data/Stream/Instances.hs+9−6 src/Streamly/Internal/Data/Stream/IsStream.hs+2−8 src/Streamly/Internal/Data/Stream/IsStream/Combinators.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Common.hs+8−8 src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs+11−13 src/Streamly/Internal/Data/Stream/IsStream/Enumeration.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Exception.hs+4−4 src/Streamly/Internal/Data/Stream/IsStream/Expand.hs+4−8 src/Streamly/Internal/Data/Stream/IsStream/Generate.hs+7−7 src/Streamly/Internal/Data/Stream/IsStream/Lift.hs+1−1 src/Streamly/Internal/Data/Stream/IsStream/Reduce.hs+16−15 src/Streamly/Internal/Data/Stream/IsStream/Top.hs+5−4 src/Streamly/Internal/Data/Stream/IsStream/Transform.hs+5−5 src/Streamly/Internal/Data/Stream/IsStream/Type.hs+21−22 src/Streamly/Internal/Data/Stream/MkType.hs+34−33 src/Streamly/Internal/Data/Stream/Parallel.hs+19−31 src/Streamly/Internal/Data/Stream/Prelude.hs−19 src/Streamly/Internal/Data/Stream/SVar.hs−18 src/Streamly/Internal/Data/Stream/SVar/Eliminate.hs+4−4 src/Streamly/Internal/Data/Stream/SVar/Generate.hs+6−8 src/Streamly/Internal/Data/Stream/Serial.hs+6−19 src/Streamly/Internal/Data/Stream/Time.hs+10−10 src/Streamly/Internal/Data/Stream/Zip.hs+3−42 src/Streamly/Internal/Data/Stream/Zip/Concurrent.hs+50 src/Streamly/Internal/Data/Stream/ZipAsync.hs+4−5 src/Streamly/Internal/Data/Unfold/Exception.hs+3−1 src/Streamly/Internal/Data/Unfold/Prelude.hs−17 src/Streamly/Internal/Data/Unfold/SVar.hs+2−3 src/Streamly/Internal/FileSystem/Event.hs+2−5 src/Streamly/Internal/FileSystem/Event/Darwin.hs+5−5 src/Streamly/Internal/FileSystem/Event/Linux.hs+9−15 src/Streamly/Internal/FileSystem/Event/Windows.hs+8−11 src/Streamly/Internal/FileSystem/FD.hs+15−16 src/Streamly/Internal/Network/Inet/TCP.hs+48−84 src/Streamly/Internal/Network/Socket.hs+31−34 src/Streamly/Internal/System/IOVec.hs+7−7 src/Streamly/Internal/Unicode/Char.hs+8−10 src/Streamly/Memory/Array.hs+74 src/Streamly/Network/Inet/TCP.hs+30−41 src/Streamly/Network/Socket.hs+30−16 src/Streamly/Prelude.hs+4−4 src/inline.hs+1−1 stack.yaml+2−1 streamly.cabal+90−117 targets/Targets.hs+16−17 test/Streamly/Test/Data/Array.hs+10−10 test/Streamly/Test/Data/MutArray.hs → test/Streamly/Test/Data/Array/Mut.hs+5−5 test/Streamly/Test/Data/Array/Stream.hs+3−4 test/Streamly/Test/Data/Fold.hs+8−8 test/Streamly/Test/Data/Fold/Window.hs+9−8 test/Streamly/Test/Data/Parser.hs+141−282 test/Streamly/Test/Data/Parser/Chunked.hs+1239 test/Streamly/Test/Data/ParserK.hs → test/Streamly/Test/Data/Parser/ParserD.hs+14−19 test/Streamly/Test/Data/Ring/Unboxed.hs+2−2 test/Streamly/Test/Data/Serialize.hs−306 test/Streamly/Test/Data/Serialize/CompatV0.hs−46 test/Streamly/Test/Data/Serialize/CompatV1.hs−48 test/Streamly/Test/Data/Serialize/TH.hs−82 test/Streamly/Test/Data/Stream.hs−33 test/Streamly/Test/Data/Stream/Concurrent.hs+82−126 test/Streamly/Test/Data/Unbox.hs+50−318 test/Streamly/Test/Data/Unbox/TH.hs−146 test/Streamly/Test/Data/Unfold.hs+2−2 test/Streamly/Test/FileSystem/Event/Common.hs+3−5 test/Streamly/Test/FileSystem/Event/Linux.hs+3−16 test/Streamly/Test/FileSystem/Handle.hs+2−2 test/Streamly/Test/Network/Inet/TCP.hs+4−5 test/Streamly/Test/Network/Socket.hs+4−5 test/Streamly/Test/Prelude/Serial.hs+35−42 test/Streamly/Test/Prelude/Top.hs+1−1 test/Streamly/Test/Serialize/Serializable.hs−237 test/Streamly/Test/Unicode/Char.hs+1−1 test/Streamly/Test/Unicode/Parser.hs−231 test/Streamly/Test/Unicode/Stream.hs+28−30 test/lib/Streamly/Test/Common.hs−4 test/lib/Streamly/Test/Prelude/Common.hs+2−9 test/streamly-tests.cabal+29−99
.circleci/config.yml+277−73
@@ -1,7 +1,7 @@
11version: 2.1
22
33#-----------------------------------------------------------------------------
4−# packcheck-0.7.0
4+# packcheck-0.4.2
55# Packcheck global environment variables
66#-----------------------------------------------------------------------------
77
@@ -11,8 +11,6 @@ env: &env
1111 # Common options
1212 # ------------------------------------------------------------------------
1313 # GHC_OPTIONS: "-Werror"
14− # For updating see: https://downloads.haskell.org/~ghcup/
15− GHCUP_VERSION: 0.1.20.0
1614 CABAL_REINIT_CONFIG: "y"
1715 LC_ALL: "C.UTF-8"
1816
@@ -24,6 +22,7 @@ env: &env
2422 # DISABLE_DOCS: "y"
2523 # DISABLE_SDIST_BUILD: "y"
2624 # DISABLE_DIST_CHECKS: "y"
25+ ENABLE_INSTALL: "y"
2726
2827 # ------------------------------------------------------------------------
2928 # stack options
@@ -39,12 +38,15 @@ env: &env
3938 # cabal options
4039 # ------------------------------------------------------------------------
4140 CABAL_CHECK_RELAX: "y"
41+ CABAL_NO_SANDBOX: "y"
42+ CABAL_HACKAGE_MIRROR: "hackage.haskell.org:http://hackage.fpcomplete.com"
4243 CABAL_BUILD_OPTIONS: "-j1 --flag limit-build-mem"
4344
4445 # ------------------------------------------------------------------------
4546 # Where to find the required tools
4647 # ------------------------------------------------------------------------
47− PATH: /sbin:/usr/sbin:/bin:/usr/bin
48+ PATH: /opt/ghc/bin:/opt/ghcjs/bin:/sbin:/usr/sbin:/bin:/usr/bin
49+ TOOLS_DIR: /opt
4850
4951 # ------------------------------------------------------------------------
5052 # Location of packcheck.sh (the shell script invoked to perform CI tests ).
@@ -57,15 +59,15 @@ env: &env
5759 # If you have not committed packcheck.sh in your repo at PACKCHECK
5860 # then it is automatically pulled from this URL.
5961 PACKCHECK_GITHUB_URL: "https://raw.githubusercontent.com/composewell/packcheck"
60− PACKCHECK_GITHUB_COMMIT: "f6cfd7b7a33b4bf6fd63cce6a3fb3d26f5031403"
62+ PACKCHECK_GITHUB_COMMIT: "a302772c8f3791c477edd1764b82f8d1f5b22fdf"
6163
6264executors:
6365 amd64-executor:
6466 docker:
65− - image: ubuntu:latest
67+ - image: ubuntu:xenial
6668 x86-executor:
6769 docker:
68− - image: i386/ubuntu:focal
70+ - image: i386/ubuntu:eoan
6971
7072#-----------------------------------------------------------------------------
7173# Common utility stuff, not to be modified usually
@@ -77,50 +79,38 @@ preinstall: &preinstall
7779 # required for https/cache save and restore
7880 apt-get install -y ca-certificates
7981
80− # Alternative way of installing ghc and cabal, directly from
81− # haskell.org instead of using ghcup. NOTE: this is for Debian
82− # only and is debian release specific.
83− # gnupg is required for apt-key to work
84− #apt-get install -y gnupg
85− #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574
86− #echo "deb http://downloads.haskell.org/debian buster main" >> /etc/apt/sources.list
87− #apt-get update
88−
89− # required for outbound https for stack and for stack setup
90− apt-get install -y netbase xz-utils make
91− apt-get install -y zlib1g-dev
82+ # required to (re)generate the configure script
83+ apt-get install -y autoconf
9284
93− # For ghcup to install ghc
94− if test -n "$GHCUP_VERSION"
95− then
96− apt-get install -y gcc
97− apt-get install -y g++
98− fi
85+ # For ghc and cabal-install packages from hvr's ppa
86+ # gnupg is required for apt-key to work
87+ apt-get install -y gnupg
88+ apt-get install -y apt-transport-https
89+ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574
90+ apt-get install -y software-properties-common
91+ add-apt-repository -y ppa:hvr/ghc
92+ # echo "deb http://downloads.haskell.org/debian stretch main" >> /etc/apt/sources.list
93+ echo "deb-src http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main" >> /etc/apt/sources.list
94+ apt-get update
9995
100− # libgmp required by ghc for linking
101− apt-get install -y libgmp-dev
102− apt-get install -y libtinfo-dev
96+ # required for packcheck
97+ apt-get install -y curl
10398
104− # Required by cabal when git URL is specified in project file
99+ # required for circleci.
100+ apt-get install -y libcurl4-gnutls-dev
105101 apt-get install -y git
102+ apt-get install -y libtinfo-dev
106103
107− # Required for and by packcheck
108− apt-get install -y curl
104+ # required for outbound https for stack and for stack setup
105+ apt-get install -y netbase xz-utils make
109106
107+project-preinstall: &project-preinstall
108+ run: |
110109 # Get packcheck if needed
111− if test ! -e "$PACKCHECK"
112− then
113− if test -z "$PACKCHECK_GITHUB_COMMIT"
114− then
115− die "PACKCHECK_GITHUB_COMMIT is not specified."
116− fi
117− PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
118− curl --fail -sL -o "$PACKCHECK" $PACKCHECK_URL || exit 1
119− chmod +x $PACKCHECK
120− elif test ! -x "$PACKCHECK"
121− then
122− chmod +x $PACKCHECK
123− fi
110+ CURL=$(which curl)
111+ PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
112+ if test ! -e "$PACKCHECK"; then $CURL -sL -o "$PACKCHECK" $PACKCHECK_URL; fi;
113+ chmod +x $PACKCHECK
124114
125115restore: &restore
126116 # Needs to happen after installing ca-certificates
@@ -136,71 +126,271 @@ save: &save
136126 save_cache:
137127 key: v1-{{ .Environment.CIRCLE_JOB }}-{{ .Branch }}
138128 paths:
139− - ~/.local
140129 - ~/.cabal
130+ - ~/.ghc
131+ - ~/.local
141132 - ~/.stack
142− - ~/.ghcup
143133
144134#-----------------------------------------------------------------------------
145135# Build matrix
146136#-----------------------------------------------------------------------------
147137
148138jobs:
149− cabal-ghc-8_6_5:
139+ cabal-ghc-8_10_4:
150140 <<: *env
151141 executor: amd64-executor
152142 steps:
143+ - *preinstall
153144 - checkout
145+ - *project-preinstall
154146 - *restore
155− - *preinstall
156147 - run:
157148 environment:
158− GHCVER: "8.6.5"
159− CABAL_VERSION: "3.6.2.0"
160149 CABAL_PROJECT: "cabal.project"
150+ DISABLE_TEST: "yes"
151+ DISABLE_BENCH: "yes"
161152 DISABLE_SDIST_BUILD: "yes"
162− CABAL_BUILD_OPTIONS: "--flag debug --flag -opt"
163153 command: |
154+ apt-get install -y ghc-8.10.4
155+ apt-get install -y cabal-install-3.4
164156 bash -c "$PACKCHECK cabal"
165157 - *save
166− cabal-ghc-9_8_1-docspec:
158+ stack-ghc-8_8:
167159 <<: *env
168160 executor: amd64-executor
169161 steps:
162+ - *preinstall
170163 - checkout
164+ - *project-preinstall
171165 - *restore
166+ - run:
167+ name: GHC 8.8 + stack lts-16.12
168+ environment:
169+ BUILD: "stack"
170+ RESOLVER: "lts-16.12"
171+ SDIST_OPTIONS: "--ignore-check"
172+ STACK_BUILD_OPTIONS: "--flag streamly-benchmarks:-opt"
173+ command: |
174+ apt-get build-dep -y ghc-8.8.4
175+ apt-get install -y cabal-install-3.2
176+ bash -c "$PACKCHECK $BUILD"
177+ - *save
178+ cabal-ghc-8_6_5:
179+ <<: *env
180+ executor: amd64-executor
181+ steps:
172182 - *preinstall
183+ - checkout
184+ - *project-preinstall
185+ - *restore
186+ - run: |
187+ apt-get install -y ghc-8.6.3
188+ apt-get install -y cabal-install-2.4
189+ bash -c "$PACKCHECK cabal-v2"
190+ - *save
191+ cabal-ghc-8_4_4:
192+ <<: *env
193+ executor: x86-executor
194+ steps:
195+ - *preinstall
196+ - checkout
197+ - *project-preinstall
198+ - *restore
199+ - run:
200+ name: GHC 8.4.4 + x86 + debug
201+ environment:
202+ BUILD: "cabal-v2"
203+ GHCVER: "8.4.4"
204+ CABAL_BUILD_OPTIONS: "--flag debug --flag limit-build-mem"
205+ command: |
206+ apt-get install -y ghc-8.4.4
207+ apt-get install -y cabal-install-3.2
208+ bash -c "$PACKCHECK $BUILD"
209+ - *save
210+ cabal-ghc-8_2_2:
211+ <<: *env
212+ executor: amd64-executor
213+ steps:
214+ - *preinstall
215+ - checkout
216+ - *project-preinstall
217+ - *restore
173218 - run:
174219 environment:
175− GHCVER: "9.8.1"
176− CABAL_VERSION: "3.10.1.0"
177− CABAL_PROJECT: "cabal.project.doctest"
178− DISABLE_SDIST_BUILD: "y"
179− DISABLE_TEST: "y"
180− DISABLE_BENCH: "y"
181− DISABLE_DOCS: "y"
182− ENABLE_DOCSPEC: "y"
183− DOCSPEC_URL: https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz
184− DOCSPEC_OPTIONS: "--timeout 60 --check-properties --property-variables xs"
220+ BUILD: "cabal-v2"
221+ DISABLE_TEST: "yes"
222+ DISABLE_BENCH: "yes"
223+ DISABLE_DOCS: "yes"
224+ DISABLE_SDIST_BUILD: "yes"
225+ DISABLE_DIST_CHECKS: "yes"
185226 command: |
186− bash -c "$PACKCHECK cabal"
227+ apt-get install -y ghc-8.2.2
228+ apt-get install -y cabal-install-3.2
229+ bash -c "$PACKCHECK $BUILD"
187230 - *save
188− hlint-trailing-spaces:
231+ cabal-ghc-8_0_2:
189232 <<: *env
190233 executor: amd64-executor
191234 steps:
235+ - *preinstall
192236 - checkout
237+ - *project-preinstall
193238 - *restore
239+ - run:
240+ environment:
241+ BUILD: "cabal-v2"
242+ DISABLE_TEST: "yes"
243+ DISABLE_BENCH: "yes"
244+ DISABLE_DOCS: "yes"
245+ DISABLE_SDIST_BUILD: "yes"
246+ DISABLE_DIST_CHECKS: "yes"
247+ command: |
248+ apt-get install -y ghc-8.0.2
249+ apt-get install -y cabal-install-3.2
250+ bash -c "$PACKCHECK $BUILD"
251+ - *save
252+ cabal-ghc-7_10_3:
253+ <<: *env
254+ executor: amd64-executor
255+ steps:
194256 - *preinstall
257+ - checkout
258+ - *project-preinstall
259+ - *restore
260+ - run: |
261+ apt-get install -y ghc-7.10.3
262+ apt-get install -y cabal-install-2.4
263+ bash -c "$PACKCHECK cabal-v2"
264+ - *save
265+ stack-ghc-8_4:
266+ <<: *env
267+ executor: amd64-executor
268+ steps:
269+ - *preinstall
270+ - checkout
271+ - *project-preinstall
272+ - *restore
273+ - run: |
274+ bash -c "$PACKCHECK stack RESOLVER=lts-12"
275+ - *save
276+ stack-ghc-8_2:
277+ <<: *env
278+ executor: amd64-executor
279+ steps:
280+ - *preinstall
281+ - checkout
282+ - *project-preinstall
283+ - *restore
284+ - run: |
285+ bash -c "$PACKCHECK stack RESOLVER=lts-11"
286+ - *save
287+ cabal-ghcjs-8_4:
288+ <<: *env
289+ executor: amd64-executor
290+ steps:
291+ - *preinstall
292+ - checkout
293+ - *project-preinstall
294+ - *restore
195295 - run:
196− name: hlint and trailing spaces
296+ name: GHCJS 8.4+no-test+no-docs
197297 environment:
298+ BUILD: cabal-v2
299+ ENABLE_GHCJS: "yes"
300+ DISABLE_TEST: "yes"
301+ ENABLE_INSTALL: ""
302+ DISABLE_DOCS: "yes"
198303 DISABLE_SDIST_BUILD: "yes"
199− HLINT_VERSION: 3.6.1
304+ command: |
305+ add-apt-repository -y ppa:hvr/ghcjs
306+ add-apt-repository -y 'deb https://deb.nodesource.com/node_11.x xenial main'
307+ curl -sSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -
308+ apt-get update
309+ apt-get install -y cabal-install-3.2
310+ apt-get install -y nodejs
311+ apt-get install -y ghcjs-8.4
312+ cabal user-config update -a "jobs: 1"
313+ bash -c "$PACKCHECK $BUILD"
314+ - *save
315+ coveralls-ghc-8_2_2:
316+ <<: *env
317+ executor: amd64-executor
318+ steps:
319+ - *preinstall
320+ - checkout
321+ - *project-preinstall
322+ - *restore
323+ - run:
324+ environment:
325+ BUILD: cabal
326+ COVERALLS_OPTIONS: "--repo-token=\"$REPO_TOKEN\" --coverage-mode=StrictlyFullLines --exclude-dir=test test"
327+ GHC_OPTIONS: "-DCOVERAGE_BUILD"
328+ command: |
329+ apt-get install -y ghc-8.2.2
330+ apt-get install -y cabal-install-2.4
331+ bash -c "$PACKCHECK cabal-v1"
332+ - *save
333+ coveralls-ghc-8_8_3:
334+ <<: *env
335+ executor: amd64-executor
336+ steps:
337+ - *preinstall
338+ - checkout
339+ - *project-preinstall
340+ - *restore
341+ - run:
342+ environment:
343+ BUILD: cabal-v2
344+ COVERALLS_OPTIONS: "--repo-token=\"$REPO_TOKEN\" --coverage-mode=StrictlyFullLines --exclude-dir=test"
345+ CABAL_PROJECT: "cabal.project.coverage"
346+ CABAL_BUILD_TARGETS: ""
347+ command: |
348+ apt-get install -y ghc-8.8.3
349+ apt-get install -y cabal-install-3.0
350+ bash -c "$PACKCHECK cabal-v2"
351+ no_output_timeout: 25m
352+ - *save
353+ coveralls-ghc-8_10_2:
354+ <<: *env
355+ executor: amd64-executor
356+ steps:
357+ - *preinstall
358+ - checkout
359+ - *project-preinstall
360+ - *restore
361+ - run:
362+ name: Build and test with inspection, coverage, Werror
363+ command: |
364+ apt-get install -y ghc-8.10.2
365+ apt-get install -y cabal-install-3.2
366+ # Run tests with coverage
367+ cabal update
368+ cabal user-config update -a "jobs: 1"
369+ bin/test.sh --coverage --raw
370+ # Upload results to coveralls.io
371+ PATH=$HOME/.cabal/bin:$PATH
372+ export PATH
373+ which hpc-coveralls 2>/dev/null || cabal install --project-file cabal.project.hpc-coveralls hpc-coveralls
374+ hpc-coveralls --repo-token="$REPO_TOKEN" --coverage-mode=StrictlyFullLines
375+ no_output_timeout: 25m
376+ - *save
377+ hlint-src:
378+ <<: *env
379+ executor: amd64-executor
380+ steps:
381+ - *preinstall
382+ - checkout
383+ - *project-preinstall
384+ - *restore
385+ - run:
386+ name: Hlint src
387+ environment:
388+ BUILD: hlint
200389 HLINT_OPTIONS: lint
390+ DISABLE_SDIST_BUILD: "yes"
201391 HLINT_TARGETS: core/src src test benchmark
202392 command: |
203− bash -c "$PACKCHECK hlint" || exit 1
393+ bash -c "$PACKCHECK $BUILD" || exit 1
204394 echo "Checking trailing spaces..."
205395 count=$(find . -name "*.hs" -exec grep -H '\ $' {} \; | tee /dev/tty | wc -l)
206396 exit $count
@@ -210,9 +400,23 @@ workflows:
210400 version: 2
211401 build:
212402 jobs:
213− #- cabal-ghc-8_6_5:
214− # name: 8.6.5-debug-unoptimized
215− - cabal-ghc-9_8_1-docspec:
216− name: ghc-9.8.1-docspec
217− - hlint-trailing-spaces:
218− name: hlint and trailing spaces
403+ - cabal-ghc-8_10_4:
404+ name: GHC 8.10.4 + sdist + no-test + no-bench
405+ #- cabal-ghc-8.6.5
406+ #- cabal-ghc-8_4_4:
407+ # name: GHC 8.4.4 + x86 + debug
408+ #- cabal-ghc-8_2_2:
409+ # name: GHC 8.2.2 + no-test + no-bench + no-docs
410+ #- cabal-ghc-8_0_2:
411+ # name: GHC 8.0.2 + no-test + no-bench + no-docs
412+ #- cabal-ghc-7.10.3
413+ # - cabal-ghcjs-8_4:
414+ # name: GHCJS 8.4 + no-test + no-docs
415+ #- stack-ghc-8.4
416+ #- stack-ghc-8.2
417+ #- coveralls-ghc-8.2.2
418+ #- coveralls-ghc-8.8.3
419+ #- coveralls-ghc-8_10_2:
420+ # name: GHC 8.10.2 + inspection + coverage + Werror
421+ - hlint-src:
422+ name: Hlint src + Trailing Spaces
.github/workflows/haskell.yml+89−108
@@ -13,15 +13,9 @@ jobs:
1313 # packcheck environment variables
1414 LC_ALL: C.UTF-8
1515 BUILD: ${{ matrix.build }}
16− # For updating see: https://downloads.haskell.org/~ghcup/
17− GHCUP_VERSION: 0.1.20.0
18− DOCSPEC_URL: https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz
19− DOCSPEC_OPTIONS: "--timeout 60 --check-properties --property-variables xs"
20− GHCUP_GHC_OPTIONS: ${{ matrix.ghcup_ghc_options }}
2116 GHCVER: ${{ matrix.ghc_version }}
2217 CABALVER: ${{ matrix.cabal_version }}
2318 DISABLE_DOCS: ${{ matrix.disable_docs }}
24− ENABLE_DOCSPEC: ${{ matrix.enable_docspec }}
2519 DISABLE_TEST: ${{ matrix.disable_test }}
2620 DISABLE_BENCH: ${{ matrix.disable_bench }}
2721 DISABLE_DIST_CHECKS: ${{ matrix.disable_dist_checks }}
@@ -45,21 +39,14 @@ jobs:
4539 STACK_BUILD_OPTIONS: ${{ matrix.stack_build_options }}
4640
4741 # packcheck location and revision
48− PACKCHECK: "./packcheck.sh"
42+ PACKCHECK_LOCAL_PATH: "./packcheck.sh"
4943 PACKCHECK_GITHUB_URL: "https://raw.githubusercontent.com/composewell/packcheck"
50− PACKCHECK_GITHUB_COMMIT: "v0.7.0"
44+ PACKCHECK_GITHUB_COMMIT: "a302772c8f3791c477edd1764b82f8d1f5b22fdf"
5145
5246 # Pull token from "secrets" setting of the github repo
5347 COVERALLS_TOKEN: ${{ secrets.COVERALLS_TOKEN }}
5448 COVERAGE: ${{ matrix.coverage }}
55−
56− # hlint
57− HLINT_VERSION: 3.6.1
58− HLINT_OPTIONS: "lint"
59− HLINT_TARGETS: "core/src src test benchmark"
60−
61− # Subdir
62− SUBDIR: ${{ matrix.subdir }}
49+ DOCTEST: ${{ matrix.doctest }}
6350
6451 runs-on: ${{ matrix.runner }}
6552 continue-on-error: ${{ matrix.ignore_error }}
@@ -68,8 +55,8 @@ jobs:
6855 matrix:
6956 # The order is important to optimize fail-fast.
7057 name:
71− - 9.8.1-Werror
72− # - 9.8.1-docspec
58+ - 8.10.7-Werror
59+ - 9.2.4-docspec
7360 # - 8.10.7-coverage
7461
7562 # Note: if cabal.project is not specified benchmarks and tests won't
@@ -78,74 +65,35 @@ jobs:
7865 include:
7966 - name: head
8067 ghc_version: head
81− # The URL may change, to find a working URL go to https://gitlab.haskell.org/ghc/ghc/-/jobs/
82− # Find a debian10/11/12 job, click on a passed/failed status, at the
83− # end of the output you will find the tar.xz name, put that tar
84− # name after "raw/", and put the job name after "job=".
85− # Also see https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix
86− #
87− # May also use ghcup for installing ghc head version, use the
88− # version "LatestNightly", and the following config:
89− # ghcup config add-release-channel https://ghc.gitlab.haskell.org/ghcup-metadata/ghcup-nightlies-0.0.7.yaml
90− ghcup_ghc_options: "-u https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-deb10-int_native-validate.tar.xz?job=x86_64-linux-deb10-int_native-validate"
9168 runner: ubuntu-latest
9269 build: cabal
93− cabal_version: 3.10.2.0
70+ cabal_version: 3.8.1.0
9471 cabal_project: cabal.project.ghc-head
9572 disable_sdist_build: "y"
9673 ignore_error: true
97− # Note: use linux for warning build for convenient dev testing
98− - name: 9.8.1-Werror
99− ghc_version: 9.8.1
100− runner: ubuntu-latest
101− build: cabal
102− cabal_version: 3.10.1.0
103− disable_sdist_build: "y"
104− cabal_project: cabal.project.Werror
105− ignore_error: false
106− #- name: 9.8.1-docspec
107− # ghc_version: 9.8.1
108− # runner: ubuntu-latest
109− # build: cabal
110− # cabal_version: 3.10.1.0
111− # cabal_project: cabal.project.doctest
112− # disable_test: "y"
113− # disable_bench: "y"
114− # disable_docs: "y"
115− # enable_docspec: "y"
116− # disable_sdist_build: "y"
117− # ignore_error: false
118− - name: 9.8.1-fusion-inspection
119− ghc_version: 9.8.1
74+ - name: 9.4.2-fusion-inspection
75+ ghc_version: 9.4.2
12076 runner: ubuntu-latest
12177 build: cabal
122− cabal_version: 3.10.1.0
78+ cabal_version: 3.8.1.0
12379 disable_sdist_build: "y"
12480 cabal_project: cabal.project
12581 cabal_build_options: "--flag fusion-plugin --flag inspection"
12682 ignore_error: false
127− - name: 9.6.3-macos
128− ghc_version: 9.6.3
129− runner: macos-latest
130− build: cabal
131− cabal_version: 3.10.1.0
132− disable_sdist_build: "y"
133− cabal_project: cabal.project
134− ignore_error: false
135− - name: 9.4.7
136− ghc_version: 9.4.7
83+ - name: 9.2.4
84+ ghc_version: 9.2.4
13785 runner: ubuntu-latest
13886 build: cabal
139− cabal_version: 3.8.1.0
140− disable_sdist_build: "y"
14187 cabal_project: cabal.project
88+ cabal_version: 3.6.2.0
89+ disable_sdist_build: "y"
14290 ignore_error: false
143− - name: 9.2.8
144− ghc_version: 9.2.8
91+ - name: 9.2.4-docspec
92+ ghc_version: 9.2.4
14593 runner: ubuntu-latest
146− build: cabal
147− cabal_project: cabal.project
94+ doctest: "y"
14895 cabal_version: 3.6.2.0
96+ cabal_project: cabal.project
14997 disable_sdist_build: "y"
15098 ignore_error: false
15199 - name: 9.0.2-streamly-sdist
@@ -155,28 +103,37 @@ jobs:
155103 cabal_version: 3.6.2.0
156104 cabal_project: cabal.project.streamly
157105 ignore_error: true
158− - name: 9.0.2-streamly-core-sdist
159− ghc_version: 9.0.2
160− runner: ubuntu-latest
161− build: cabal
162− cabal_version: 3.6.2.0
163− subdir: core
164− ignore_error: false
165106 - name: 8.10.7-streamly-sdist-stack
166107 runner: ubuntu-latest
167108 build: stack
168− resolver: lts-20.13
109+ resolver: lts-18.23
169110 stack_yaml: stack.yaml
170111 sdist_options: "--ignore-check"
171112 stack_build_options: "--flag streamly-benchmarks:-opt"
172113 cabal_version: 3.6.2.0
173114 ignore_error: false
115+ - name: 8.10.7-macos
116+ ghc_version: 8.10.7
117+ runner: macos-latest
118+ build: cabal
119+ cabal_version: 3.6.2.0
120+ cabal_project: cabal.project
121+ disable_sdist_build: "y"
122+ ignore_error: false
174123 # - name: 8.10.7-coverage
175124 # ghc_version: 8.10.7
176125 # runner: ubuntu-latest
177126 # coverage: "y"
178127 # cabal_version: 3.6.2.0
179128 # ignore_error: false
129+ - name: 8.10.7-Werror
130+ ghc_version: 8.10.7
131+ runner: ubuntu-latest
132+ build: cabal
133+ cabal_version: 3.6.2.0
134+ cabal_project: cabal.project.Werror
135+ disable_sdist_build: "y"
136+ ignore_error: false
180137 - name: 8.8.4
181138 ghc_version: 8.8.4
182139 runner: ubuntu-latest
@@ -185,6 +142,15 @@ jobs:
185142 cabal_project: cabal.project
186143 disable_sdist_build: "y"
187144 ignore_error: false
145+ - name: 8.6.5-no-fusion-unoptimized
146+ ghc_version: 8.6.5
147+ runner: ubuntu-latest
148+ build: cabal
149+ cabal_version: 3.6.2.0
150+ cabal_project: cabal.project
151+ cabal_build_options: "--flag no-fusion --flag -opt"
152+ disable_sdist_build: "y"
153+ ignore_error: false
188154 - name: 8.6.5-debug-unoptimized
189155 ghc_version: 8.6.5
190156 runner: ubuntu-latest
@@ -194,44 +160,66 @@ jobs:
194160 cabal_build_options: "--flag debug --flag -opt"
195161 disable_sdist_build: "y"
196162 ignore_error: false
197− - name: hlint
198− build: hlint
199− runner: ubuntu-latest
200− ignore_error: true
201163
202164 steps:
203165 - uses: actions/checkout@v2
204166
205167 - uses: actions/cache@v1
206− name: Cache common directories
168+ name: Cache ~/.cabal
207169 with:
208− path: |
209− ~/.local
210− ~/.cabal
211− ~/.stack
212− ~/.ghcup
170+ path: ~/.cabal
213171 # Bump the key version to clear the cache
214172 key: ${{ runner.os }}-${{ matrix.ghc_version }}-cabal-v2
215173
216174 - name: Download packcheck
217175 run: |
218− if test ! -e "$PACKCHECK"
219− then
220− if test -z "$PACKCHECK_GITHUB_COMMIT"
221− then
222− die "PACKCHECK_GITHUB_COMMIT is not specified."
223− fi
224− PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
225− curl --fail -sL -o "$PACKCHECK" $PACKCHECK_URL || exit 1
226− chmod +x $PACKCHECK
227− elif test ! -x "$PACKCHECK"
176+ # Get packcheck if needed
177+ CURL=$(which curl)
178+ PACKCHECK_URL=${PACKCHECK_GITHUB_URL}/${PACKCHECK_GITHUB_COMMIT}/packcheck.sh
179+ if test ! -e "$PACKCHECK_LOCAL_PATH"; then $CURL -sL -o "$PACKCHECK_LOCAL_PATH" $PACKCHECK_URL; fi;
180+ chmod +x $PACKCHECK_LOCAL_PATH
181+
182+ os=$(uname -s -m)
183+ case "$os" in
184+ "Linux x86_64") GHCUP_ARCH="x86_64-linux" ;;
185+ "Darwin x86_64") GHCUP_ARCH="x86_64-apple-darwin" ;;
186+ *) echo "Unknown OS/Arch: $os"; exit 1;;
187+ esac
188+
189+ # Check available versions here: https://downloads.haskell.org/~ghcup/
190+ GHCUP_VER=0.1.18.0
191+ $CURL -sL -o ./ghcup https://downloads.haskell.org/~ghcup/$GHCUP_VER/${GHCUP_ARCH}-ghcup-$GHCUP_VER
192+ chmod +x ./ghcup
193+ if test "$GHCVER" = "head"
228194 then
229− chmod +x $PACKCHECK
195+ # The URL may change, to find a working URL go to https://gitlab.haskell.org/ghc/ghc/-/jobs/
196+ # Find a debian10 job, click on a passed/failed job, at the
197+ # end of the output you will find the tar.xz name, put that after
198+ # "raw/", and put the job name after "job=".
199+ # Also see https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix
200+ ./ghcup install ghc -u https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-deb10-int_native-validate.tar.xz?job=x86_64-linux-deb10-int_native-validate head
201+ else
202+ ./ghcup install ghc $GHCVER
230203 fi
204+ ./ghcup install cabal $CABALVER
205+ ./ghcup set ghc $GHCVER
206+ ./ghcup set cabal $CABALVER
231207
232208 - name: Run tests
233209 run: |
234− if test -n "$COVERAGE"
210+ if test -n "$DOCTEST"
211+ then
212+ mkdir -p $HOME/.cabal/bin
213+ curl -sL https://github.com/phadej/cabal-extras/releases/download/cabal-docspec-0.0.0.20210111/cabal-docspec-0.0.0.20210111.xz > cabal-docspec.xz
214+ echo '0829bd034fba901cbcfe491d98ed8b28fd54f9cb5c91fa8e1ac62dc4413c9562 cabal-docspec.xz' | sha256sum -c -
215+ xz -d < cabal-docspec.xz > $HOME/.cabal/bin/cabal-docspec
216+ rm -f cabal-docspec.xz
217+ chmod a+x $HOME/.cabal/bin/cabal-docspec
218+ cabal update
219+ cabal build --project-file cabal.project.doctest all
220+ $HOME/.cabal/bin/cabal-docspec --version
221+ $HOME/.cabal/bin/cabal-docspec --timeout 60 --check-properties --property-variables xs
222+ elif test -n "$COVERAGE"
235223 then
236224 # Run tests with coverage
237225 cabal update
@@ -244,15 +232,8 @@ jobs:
244232 # XXX Uncomment this and fix it properly later
245233 # hpc-coveralls --repo-token="$COVERALLS_TOKEN" --coverage-mode=StrictlyFullLines
246234 else
235+ export TOOLS_DIR=/opt
247236 # /usr/local/opt/curl/bin for macOS
248237 export PATH=$HOME/.local/bin:$HOME/.ghcup/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/opt/curl/bin
249− if test -n "$SUBDIR"
250− then
251− mv "$PACKCHECK" "$SUBDIR/packcheck.sh"
252− # This is required as dist-newstyle will be created on the top level
253− # directory as it is considered the working dir.
254− rm cabal.project
255− cd "$SUBDIR"
256− fi
257− bash -c "$PACKCHECK $BUILD"
238+ bash -c "$PACKCHECK_LOCAL_PATH $BUILD"
258239 fi
.github/workflows/regression-check.yml+26−15
@@ -19,15 +19,16 @@ jobs:
1919 env:
2020 CI_BENCHMARKS_WITH_CUTOFF: >-
2121 Data.Array
22− Data.MutArray
22+ Data.Array.Mut
2323 Data.Array.Generic
2424 Data.Ring.Unboxed
2525 Data.Fold
2626 Data.Fold.Window
2727 Data.Parser
28− Data.ParserK
29− Data.StreamD
30− Data.StreamK:6
28+ Data.Parser.ParserD
29+ Data.Parser.ParserK
30+ Data.Stream.StreamD
31+ Data.Stream.StreamK:6
3132 Data.Unfold
3233 Data.Stream
3334 Data.Stream.Concurrent
@@ -60,7 +61,7 @@ jobs:
6061 GHCUP_VER=0.1.18.0
6162 curl -sL -o ./ghcup https://downloads.haskell.org/~ghcup/$GHCUP_VER/x86_64-linux-ghcup-$GHCUP_VER
6263 chmod +x ./ghcup
63− GHCVER=9.2.7
64+ GHCVER=8.10.7
6465 ./ghcup install ghc $GHCVER
6566 ./ghcup set ghc $GHCVER
6667 cabal update
@@ -73,6 +74,14 @@ jobs:
7374 # Bump the key version to clear the cache
7475 key: cache-v2
7576
77+ - name: Cache charts from master
78+ id: cache-charts-master
79+ uses: actions/cache@v2
80+ with:
81+ path: charts-master
82+ # Bump the key version to clear the cache
83+ key: charts-master-v1
84+
7685 - name: Cache bench-runner from pr
7786 id: cache-bench-runner-pr
7887 uses: actions/cache@v2
@@ -91,6 +100,7 @@ jobs:
91100 clean: false
92101
93102 - name: Install bench-runner
103+ if: steps.cache-bench-runner-pr.outputs.cache-hit != 'true'
94104 run: |
95105 cabal install bench-runner --project-file=cabal.project.report --installdir=./
96106
@@ -99,16 +109,19 @@ jobs:
99109 # -----------------------------------------------------------------
100110
101111 - name: Checkout the base branch
112+ if: steps.cache-charts-master.outputs.cache-hit != 'true'
102113 uses: actions/checkout@v2
103114 with:
104115 ref: master
105116 clean: false
106117
107118 - name: Run benchmarks
119+ if: steps.cache-charts-master.outputs.cache-hit != 'true'
108120 run: |
109− ./bench-runner --package-name streamly-benchmarks --package-version 0.0.0 --targets "$CI_BENCHMARKS" --raw
121+ ./bench-runner --targets "$CI_BENCHMARKS" --raw
110122
111123 - name: Move charts to charts-master
124+ if: steps.cache-charts-master.outputs.cache-hit != 'true'
112125 run: mv charts charts-master
113126
114127 # -----------------------------------------------------------------
@@ -125,7 +138,7 @@ jobs:
125138
126139 - name: Run benchmarks and append
127140 run: |
128− ./bench-runner --package-name streamly-benchmarks --package-version 0.0.0 --targets "$CI_BENCHMARKS" --raw --append
141+ ./bench-runner --targets "$CI_BENCHMARKS" --raw --append
129142
130143 # -----------------------------------------------------------------
131144 # -- Compare
@@ -133,7 +146,7 @@ jobs:
133146
134147 - name: List all benchmarks
135148 run: |
136− ./bench-runner --package-name streamly-benchmarks --package-version 0.0.0 --targets "$CI_BENCHMARKS" --no-measure
149+ ./bench-runner --targets "$CI_BENCHMARKS" --no-measure
137150
138151 - name: Compare benchmarks
139152 run: |
@@ -151,13 +164,11 @@ jobs:
151164
152165 echo
153166 echo "Checking $bname for regressions greater than $cutoff percent"
154− ! ./bench-runner \
155− --package-name streamly-benchmarks \
156− --package-version 0.0.0 \
157− --targets "$bname" \
158− --fields "$CI_FIELDS" \
159− --no-measure --silent \
160− --diff-cutoff-percent $cutoff \
167+ ! ./bench-runner \
168+ --targets "$bname" \
169+ --fields "$CI_FIELDS" \
170+ --no-measure --silent \
171+ --diff-cutoff-percent $cutoff \
161172 | grep -v "^$"
162173 test $? -eq 1 && EXIT_STATUS=1
163174 done
.hlint.ignore+2−8
@@ -2,18 +2,13 @@ core/src/Streamly/Internal/Data/Stream/StreamK/Type.hs
22core/src/Streamly/Internal/Data/Pipe/Type.hs
33core/src/Streamly/Internal/Unicode/Stream.hs
44src/Streamly/Internal/Data/SmallArray/Type.hs
5−src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs
6−src/Streamly/Internal/Data/Stream/Serial.hs
7−src/Streamly/Internal/Data/Stream/Zip.hs
85test/Streamly/Test/Data/Array.hs
96test/Streamly/Test/Data/Parser.hs
10−test/Streamly/Test/Data/ParserK.hs
7+test/Streamly/Test/Data/Parser/ParserD.hs
118test/Streamly/Test/Data/SmallArray.hs
129test/Streamly/Test/Data/Unfold.hs
1310test/Streamly/Test/FileSystem/Event.hs
1411test/Streamly/Test/Prelude/Concurrent.hs
15−test/Streamly/Test/Data/Stream/Concurrent.hs
16−test/lib/Streamly/Test/Prelude/Common.hs
1712test/Streamly/Test/Prelude/Fold.hs
1813test/Streamly/Test/Prelude/Rate.hs
1914test/Streamly/Test/Prelude/Serial.hs
@@ -25,13 +20,12 @@ benchmark/lib/Streamly/Benchmark/Prelude.hs
2520benchmark/NanoBenchmarks.hs
2621benchmark/Streamly/Benchmark/Data/Array.hs
2722benchmark/Streamly/Benchmark/Data/Parser.hs
28−benchmark/Streamly/Benchmark/Data/ParserK.hs
23+benchmark/Streamly/Benchmark/Data/Parser/ParserD.hs
2924benchmark/Streamly/Benchmark/Data/Stream/StreamDK.hs
3025benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs
3126benchmark/Streamly/Benchmark/Data/Unfold.hs
3227benchmark/Streamly/Benchmark/FileSystem/Handle.hs
3328benchmark/Streamly/Benchmark/Prelude/Async.hs
34−benchmark/Streamly/Benchmark/Prelude/Concurrent.hs
3529benchmark/Streamly/Benchmark/Prelude/Merge.hs
3630benchmark/Streamly/Benchmark/Prelude/Parallel.hs
3731benchmark/Streamly/Benchmark/Prelude/Rate.hs
.hlint.yaml+3−1
@@ -24,12 +24,14 @@
2424- ignore: {name: "Use uncurry"}
2525- ignore: {name: "Redundant $!"}
2626- ignore: {name: "Use fmap"}
27−- ignore: {name: "Use ++"}
2827
2928# Warnings ignored in specific places
29+- ignore: {name: "Use ++", within: Stream.Transform}
3030- ignore: {name: "Use mapM", within: Stream.Transform}
3131- ignore: {name: "Use traverse", within: Stream.Transform}
3232- ignore: {name: "Redundant <*", within: Stream.Expand}
33+- ignore: {name: "Use ++", within: Stream.Reduce}
34+- ignore: {name: "Use ++", within: Stream.Split}
3335- ignore: {name: "Redundant bracket", within: Stream.Split}
3436- ignore: {name: "Use isDigit", within: Streamly.Internal.Unicode.Parser}
3537
CHANGELOG.md+1−1
@@ -1 +1 @@
1−docs/User/Project/Changelog.md
1+docs/User/ProjectRelated/Changelog.md
LICENSE+254
@@ -26,6 +26,172 @@ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
2626OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
2727OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
2828
29+-------------------------------------------------------------------------------
30+base-4.12.0.0 (http://hackage.haskell.org/package/base-4.12.0.0)
31+-------------------------------------------------------------------------------
32+This library (libraries/base) is derived from code from several
33+sources:
34+
35+ * Code from the GHC project which is largely (c) The University of
36+ Glasgow, and distributable under a BSD-style license (see below),
37+
38+ * Code from the Haskell 98 Report which is (c) Simon Peyton Jones
39+ and freely redistributable (but see the full license for
40+ restrictions).
41+
42+ * Code from the Haskell Foreign Function Interface specification,
43+ which is (c) Manuel M. T. Chakravarty and freely redistributable
44+ (but see the full license for restrictions).
45+
46+The full text of these licenses is reproduced below. All of the
47+licenses are BSD-style or compatible.
48+
49+The Glasgow Haskell Compiler License
50+
51+Copyright 2004, The University Court of the University of Glasgow.
52+All rights reserved.
53+
54+Redistribution and use in source and binary forms, with or without
55+modification, are permitted provided that the following conditions are met:
56+
57+- Redistributions of source code must retain the above copyright notice,
58+this list of conditions and the following disclaimer.
59+
60+- Redistributions in binary form must reproduce the above copyright notice,
61+this list of conditions and the following disclaimer in the documentation
62+and/or other materials provided with the distribution.
63+
64+- Neither name of the University nor the names of its contributors may be
65+used to endorse or promote products derived from this software without
66+specific prior written permission.
67+
68+THIS SOFTWARE IS PROVIDED BY THE UNIVERSITY COURT OF THE UNIVERSITY OF
69+GLASGOW AND THE CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
70+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
71+FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
72+UNIVERSITY COURT OF THE UNIVERSITY OF GLASGOW OR THE CONTRIBUTORS BE LIABLE
73+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
74+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
75+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
76+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
77+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
78+OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
79+DAMAGE.
80+
81+Code derived from the document "Report on the Programming Language
82+Haskell 98", is distributed under the following license:
83+
84+ Copyright (c) 2002 Simon Peyton Jones
85+
86+ The authors intend this Report to belong to the entire Haskell
87+ community, and so we grant permission to copy and distribute it for
88+ any purpose, provided that it is reproduced in its entirety,
89+ including this Notice. Modified versions of this Report may also be
90+ copied and distributed for any purpose, provided that the modified
91+ version is clearly presented as such, and that it does not claim to
92+ be a definition of the Haskell 98 Language.
93+
94+Code derived from the document "The Haskell 98 Foreign Function
95+Interface, An Addendum to the Haskell 98 Report" is distributed under
96+the following license:
97+
98+ Copyright (c) 2002 Manuel M. T. Chakravarty
99+
100+ The authors intend this Report to belong to the entire Haskell
101+ community, and so we grant permission to copy and distribute it for
102+ any purpose, provided that it is reproduced in its entirety,
103+ including this Notice. Modified versions of this Report may also be
104+ copied and distributed for any purpose, provided that the modified
105+ version is clearly presented as such, and that it does not claim to
106+ be a definition of the Haskell 98 Foreign Function Interface.
107+
108+-------------------------------------------------------------------------------
109+bjoern-2008-2009 (http://bjoern.hoehrmann.de/utf-8/decoder/dfa/)
110+-------------------------------------------------------------------------------
111+Copyright (c) 2008-2009 Bjoern Hoehrmann <bjoern@hoehrmann.de>
112+
113+Permission is hereby granted, free of charge, to any person obtaining a copy of
114+this software and associated documentation files (the "Software"), to deal in
115+the Software without restriction, including without limitation the rights to
116+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
117+of the Software, and to permit persons to whom the Software is furnished to do
118+so, subject to the following conditions:
119+
120+The above copyright notice and this permission notice shall be included in all
121+copies or substantial portions of the Software.
122+
123+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
124+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
125+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
126+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
127+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
128+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
129+SOFTWARE.
130+
131+-------------------------------------------------------------------------------
132+clock-0.7.2 (http://hackage.haskell.org/package/clock-0.7.2)
133+-------------------------------------------------------------------------------
134+Copyright (c) 2009-2012, Cetin Sert
135+Copyright (c) 2010, Eugene Kirpichov
136+
137+All rights reserved.
138+
139+Redistribution and use in source and binary forms, with or without
140+modification, are permitted provided that the following conditions are
141+met:
142+
143+ * Redistributions of source code must retain the above copyright
144+ notice, this list of conditions and the following disclaimer.
145+
146+ * Redistributions in binary form must reproduce the above
147+ copyright notice, this list of conditions and the following
148+ disclaimer in the documentation and/or other materials provided
149+ with the distribution.
150+
151+ * The names of contributors may not be used to endorse or promote
152+ products derived from this software without specific prior
153+ written permission.
154+
155+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
156+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
157+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
158+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
159+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
160+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
161+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
162+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
163+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
164+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
165+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
166+
167+-------------------------------------------------------------------------------
168+foldl-1.4.5 (http://hackage.haskell.org/package/foldl-1.4.5)
169+-------------------------------------------------------------------------------
170+Copyright (c) 2013 Gabriel Gonzalez
171+All rights reserved.
172+
173+Redistribution and use in source and binary forms, with or without modification,
174+are permitted provided that the following conditions are met:
175+ * Redistributions of source code must retain the above copyright notice,
176+ this list of conditions and the following disclaimer.
177+ * Redistributions in binary form must reproduce the above copyright notice,
178+ this list of conditions and the following disclaimer in the documentation
179+ and/or other materials provided with the distribution.
180+ * Neither the name of Gabriel Gonzalez nor the names of other contributors
181+ may be used to endorse or promote products derived from this software
182+ without specific prior written permission.
183+
184+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
185+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
186+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
187+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
188+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
189+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
190+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
191+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
192+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
193+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
194+
29195-------------------------------------------------------------------------------
30196fsnotify-0.3.0.1 (http://hackage.haskell.org/package/fsnotify-0.3.0.1/)
31197-------------------------------------------------------------------------------
@@ -93,3 +259,91 @@ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
93259THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
94260(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
95261OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
262+
263+-------------------------------------------------------------------------------
264+primitive-0.7.0.0 (https://hackage.haskell.org/package/primitive-0.7.0.0)
265+-------------------------------------------------------------------------------
266+Copyright (c) 2008-2009, Roman Leshchinskiy
267+All rights reserved.
268+
269+Redistribution and use in source and binary forms, with or without
270+modification, are permitted provided that the following conditions are met:
271+
272+- Redistributions of source code must retain the above copyright notice,
273+this list of conditions and the following disclaimer.
274+
275+- Redistributions in binary form must reproduce the above copyright notice,
276+this list of conditions and the following disclaimer in the documentation
277+and/or other materials provided with the distribution.
278+
279+- Neither name of the University nor the names of its contributors may be
280+used to endorse or promote products derived from this software without
281+specific prior written permission.
282+
283+THIS SOFTWARE IS PROVIDED BY THE UNIVERSITY COURT OF THE UNIVERSITY OF
284+GLASGOW AND THE CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
285+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
286+FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
287+UNIVERSITY COURT OF THE UNIVERSITY OF GLASGOW OR THE CONTRIBUTORS BE LIABLE
288+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
289+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
290+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
291+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
292+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
293+OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
294+DAMAGE.
295+
296+-------------------------------------------------------------------------------
297+transient-0.5.5 (http://hackage.haskell.org/package/transient-0.5.5)
298+-------------------------------------------------------------------------------
299+Copyright © 2014-2016 Alberto G. Corona <https://github.com/agocorona>
300+
301+Permission is hereby granted, free of charge, to any person obtaining a copy of
302+this software and associated documentation files (the "Software"), to deal in
303+the Software without restriction, including without limitation the rights to
304+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
305+the Software, and to permit persons to whom the Software is furnished to do so,
306+subject to the following conditions:
307+
308+The above copyright notice and this permission notice shall be included in all
309+copies or substantial portions of the Software.
310+
311+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
312+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
313+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
314+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
315+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
316+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
317+
318+-------------------------------------------------------------------------------
319+vector-0.12.0.2 (http://hackage.haskell.org/package/vector-0.12.0.2)
320+-------------------------------------------------------------------------------
321+Copyright (c) 2008-2012, Roman Leshchinskiy
322+All rights reserved.
323+
324+Redistribution and use in source and binary forms, with or without
325+modification, are permitted provided that the following conditions are met:
326+
327+- Redistributions of source code must retain the above copyright notice,
328+this list of conditions and the following disclaimer.
329+
330+- Redistributions in binary form must reproduce the above copyright notice,
331+this list of conditions and the following disclaimer in the documentation
332+and/or other materials provided with the distribution.
333+
334+- Neither name of the University nor the names of its contributors may be
335+used to endorse or promote products derived from this software without
336+specific prior written permission.
337+
338+THIS SOFTWARE IS PROVIDED BY THE UNIVERSITY COURT OF THE UNIVERSITY OF
339+GLASGOW AND THE CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
340+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
341+FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
342+UNIVERSITY COURT OF THE UNIVERSITY OF GLASGOW OR THE CONTRIBUTORS BE LIABLE
343+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
344+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
345+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
346+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
347+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
348+OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
349+DAMAGE.
README.md+1−1
@@ -1 +1 @@
1−docs/User/Project/README.md
1+docs/User/ProjectRelated/README.md
appveyor.yml−2
@@ -14,8 +14,6 @@ environment:
1414 # ------------------------------------------------------------------------
1515 # Common options
1616 # ------------------------------------------------------------------------
17− # GHC 9.2.7 onwards redundant constraint warnings are incompatible with
18− # older versions
1917 GHC_OPTIONS: "-Werror"
2018 CABAL_REINIT_CONFIG: "y"
2119 LC_ALL: "C.UTF-8"
benchmark/NanoBenchmarks.hs+1−1
@@ -18,7 +18,7 @@ import qualified Streamly.Data.Fold as FL
1818import qualified Streamly.Internal.Data.Stream as S
1919import qualified Streamly.Internal.Data.Stream.IsStream as Internal
2020
21−import Test.Tasty.Bench
21+import Gauge
2222import System.Random
2323
2424maxValue :: Int
benchmark/Streamly/Benchmark/Data/Array/CommonImports.hs+2−2
@@ -7,9 +7,9 @@ import System.Random (randomRIO)
77
88import qualified Streamly.Data.Fold as Fold
99import qualified Streamly.Data.Stream as S
10−import qualified Streamly.Internal.Data.Stream as Stream
10+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
1111
12−import Test.Tasty.Bench
12+import Gauge
1313import Streamly.Benchmark.Common hiding (benchPureSrc)
1414import qualified Stream.Common as P
1515
benchmark/Streamly/Benchmark/Data/MutArray.hs → benchmark/Streamly/Benchmark/Data/Array/Mut.hsrenamed+9−10
@@ -42,19 +42,18 @@ import Prelude
4242 , id
4343 , undefined
4444 )
45−import Streamly.Internal.Data.MutArray (MutArray)
4645
4746import qualified Streamly.Internal.Data.Array as Array
48−import qualified Streamly.Internal.Data.MutArray as MArray
47+import qualified Streamly.Internal.Data.Array.Mut as MArray
4948import qualified Streamly.Internal.Data.Fold as Fold
50−import qualified Streamly.Internal.Data.Stream as Stream
49+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
5150
52−import Test.Tasty.Bench
51+import Gauge
5352import Streamly.Benchmark.Common hiding (benchPureSrc)
5453
55−type Stream = MutArray
54+type Stream = MArray.Array
5655
57−instance NFData (MutArray a) where
56+instance NFData (MArray.Array a) where
5857 {-# INLINE rnf #-}
5958 rnf _ = ()
6059
@@ -181,11 +180,11 @@ unfoldReadRevDrain = drain . Stream.unfold MArray.readerRev
181180
182181{-# INLINE toStreamDRevDrain #-}
183182toStreamDRevDrain :: MonadIO m => Stream Int -> m ()
184−toStreamDRevDrain = drain . MArray.readRev
183+toStreamDRevDrain = drain . MArray.toStreamDRev
185184
186185{-# INLINE toStreamDDrain #-}
187186toStreamDDrain :: MonadIO m => Stream Int -> m ()
188−toStreamDDrain = drain . MArray.read
187+toStreamDDrain = drain . MArray.toStreamD
189188
190189{-# INLINE unfoldFold #-}
191190unfoldFold :: MonadIO m => Stream Int -> m Int
@@ -250,7 +249,7 @@ o_1_space_transformationX4 value =
250249 ]
251250
252251o_1_space_serial_marray ::
253− Int -> (MutArray Int, Array.Array Int) -> [Benchmark]
252+ Int -> (MArray.Array Int, Array.Array Int) -> [Benchmark]
254253o_1_space_serial_marray value ~(array, indices) =
255254 [ benchIO' "partitionBy (< 0)" (const (return array))
256255 $ MArray.partitionBy (< 0)
@@ -274,7 +273,7 @@ o_1_space_serial_marray value ~(array, indices) =
274273-------------------------------------------------------------------------------
275274
276275moduleName :: String
277−moduleName = "Data.MutArray"
276+moduleName = "Data.Array.Mut"
278277
279278main :: IO ()
280279main = do
benchmark/Streamly/Benchmark/Data/Array/Stream.hs+11−11
@@ -31,30 +31,30 @@ import Control.Monad.Catch (MonadCatch)
3131import Data.Functor.Identity (runIdentity)
3232import Data.Maybe (isJust)
3333import Data.Word (Word8)
34−import Streamly.Internal.Data.Stream (Stream)
35−import Streamly.Internal.Data.StreamK (StreamK)
34+import Streamly.Internal.Data.Stream.StreamD (Stream)
35+import Streamly.Internal.Data.Stream.StreamK (StreamK)
3636import System.IO (Handle)
3737import System.Random (randomRIO)
3838import Prelude hiding ()
3939
4040import qualified Streamly.Data.Stream as Stream
4141import qualified Streamly.Internal.Data.Array as Array
42−import qualified Streamly.Internal.Data.Array.Stream as ArrayStream
42+import qualified Streamly.Internal.Data.Stream.Chunked as ArrayStream
4343import qualified Streamly.Internal.Data.Fold as Fold
4444import qualified Streamly.Internal.Data.Parser as Parser
45−import qualified Streamly.Internal.Data.Stream as Stream
46−import qualified Streamly.Internal.Data.StreamK as StreamK
45+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
46+import qualified Streamly.Internal.Data.Stream.StreamK as StreamK
4747import qualified Streamly.Internal.FileSystem.Handle as Handle
4848import qualified Streamly.Internal.Unicode.Stream as Unicode
4949
50−import Test.Tasty.Bench hiding (env)
50+import Gauge hiding (env)
5151import Streamly.Benchmark.Common
5252import Streamly.Benchmark.Common.Handle
5353import Control.Monad.IO.Class (MonadIO)
5454
5555#ifdef INSPECTION
56−import Streamly.Internal.Data.MutByteArray (Unbox)
57−import Streamly.Internal.Data.Stream (Step(..))
56+import Streamly.Internal.Data.Unboxed (Unbox)
57+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..))
5858import Test.Inspection
5959#endif
6060
@@ -123,7 +123,7 @@ inspect $ 'toChunksCountBytes `hasNoType` ''Step
123123
124124toChunksDecodeUtf8Arrays :: Handle -> IO ()
125125toChunksDecodeUtf8Arrays =
126− Stream.drain . Unicode.decodeUtf8Chunks . Handle.readChunks
126+ Stream.drain . Unicode.decodeUtf8Arrays . Handle.readChunks
127127
128128#ifdef INSPECTION
129129inspect $ hasNoTypeClasses 'toChunksDecodeUtf8Arrays
@@ -295,8 +295,8 @@ main = do
295295 then return (undefined, undefined)
296296 else
297297 do
298− small <- Stream.toList $ Stream.chunksOf 100 $ sourceUnfoldrM value 0
299− big <- Stream.toList $ Stream.chunksOf value $ sourceUnfoldrM value 0
298+ small <- Stream.toList $ Stream.arraysOf 100 $ sourceUnfoldrM value 0
299+ big <- Stream.toList $ Stream.arraysOf value $ sourceUnfoldrM value 0
300300 return (small, big)
301301
302302 allBenchmarks env arrays value =
benchmark/Streamly/Benchmark/Data/Fold.hs+13−37
@@ -12,11 +12,6 @@
1212{-# OPTIONS_GHC -fno-warn-warnings-deprecations #-}
1313{-# OPTIONS_GHC -Wno-orphans #-}
1414
15−#undef FUSION_CHECK
16−#ifdef FUSION_CHECK
17−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
18−#endif
19−
2015module Main (main) where
2116
2217import Control.DeepSeq (NFData(..))
@@ -30,18 +25,18 @@ import Data.IntMap.Strict (IntMap)
3025import Data.Monoid (Last(..), Sum(..))
3126import System.Random (randomRIO)
3227
33−import Streamly.Internal.Data.Stream (Stream)
28+import Streamly.Internal.Data.Stream.StreamD (Stream)
3429import Streamly.Internal.Data.Fold (Fold(..))
3530import Streamly.Internal.Data.IsMap.HashMap ()
36−import Streamly.Internal.Data.MutArray (MutArray)
3731
32+import qualified Streamly.Internal.Data.Array.Mut.Type as MArray
3833import qualified Streamly.Internal.Data.Fold as FL
39−import qualified Streamly.Data.Fold.Prelude as Fold
34+import qualified Streamly.Internal.Data.Fold.Container as FL
4035import qualified Streamly.Internal.Data.Unfold as Unfold
4136import qualified Streamly.Internal.Data.Pipe as Pipe
42−import qualified Streamly.Internal.Data.Stream as Stream
37+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
4338
44−import Test.Tasty.Bench
39+import Gauge
4540import Streamly.Benchmark.Common
4641import Prelude hiding (all, any, take, unzip, sequence_, filter)
4742
@@ -219,7 +214,7 @@ demuxToIntMap :: Monad m =>
219214demuxToIntMap f g = Stream.fold (FL.demuxToContainer f g)
220215
221216{-# INLINE demuxToHashMap #-}
222−demuxToHashMap :: (Monad m, Hashable k) =>
217+demuxToHashMap :: (Monad m, Ord k, Hashable k) =>
223218 (a -> k) -> (a -> m (Fold m a b)) -> Stream m a -> m (HashMap k b)
224219demuxToHashMap f g = Stream.fold (FL.demuxToContainer f g)
225220
@@ -229,7 +224,7 @@ demuxToMapIO :: (MonadIO m, Ord k) =>
229224demuxToMapIO f g = Stream.fold (FL.demuxToContainerIO f g)
230225
231226{-# INLINE demuxToHashMapIO #-}
232−demuxToHashMapIO :: (MonadIO m, Hashable k) =>
227+demuxToHashMapIO :: (MonadIO m, Ord k, Hashable k) =>
233228 (a -> k) -> (a -> m (Fold m a b)) -> Stream m a -> m (HashMap k b)
234229demuxToHashMapIO f g = Stream.fold (FL.demuxToContainerIO f g)
235230
@@ -256,7 +251,7 @@ toIntMapIO f = Stream.fold (FL.toContainerIO f FL.sum)
256251{-# INLINE toHashMapIO #-}
257252toHashMapIO :: (MonadIO m, Ord k, Num a, Hashable k) =>
258253 (a -> k) -> Stream m a -> m (HashMap k a)
259−toHashMapIO f = Stream.fold (Fold.toHashMapIO f FL.sum)
254+toHashMapIO f = Stream.fold (FL.toContainerIO f FL.sum)
260255
261256-------------------------------------------------------------------------------
262257-- unzip
@@ -269,13 +264,13 @@ unzip = Stream.fold $ FL.lmap (\a -> (a, a)) (FL.unzip FL.sum FL.length)
269264{-# INLINE unzipWithFstM #-}
270265unzipWithFstM :: Monad m => Stream m Int -> m (Int, Int)
271266unzipWithFstM = do
272− let f a = return (a + 1, a)
267+ let f = \a -> return (a + 1, a)
273268 Stream.fold (FL.unzipWithFstM f FL.sum FL.length)
274269
275270{-# INLINE unzipWithMinM #-}
276271unzipWithMinM :: Monad m => Stream m Int -> m (Int, Int)
277272unzipWithMinM = do
278− let f a = return (a + 1, a)
273+ let f = \a -> return (a + 1, a)
279274 Stream.fold (FL.unzipWithMinM f FL.sum FL.length)
280275
281276-------------------------------------------------------------------------------
@@ -296,7 +291,7 @@ unfoldMany val =
296291moduleName :: String
297292moduleName = "Data.Fold"
298293
299−instance NFData (MutArray a) where
294+instance NFData (MArray.Array a) where
300295 {-# INLINE rnf #-}
301296 rnf _ = ()
302297
@@ -311,6 +306,7 @@ o_1_space_serial_elimination value =
311306 , benchIOSink value "drainBy" (Stream.fold (FL.drainBy return))
312307 , benchIOSink value "drainN" (Stream.fold (FL.drainN value))
313308 , benchIOSink value "last" (Stream.fold FL.last)
309+ , benchIOSink value "nub" (Stream.fold FL.nub)
314310 , benchIOSink value "length" (Stream.fold FL.length)
315311 , benchIOSink value "top" (Stream.fold $ FL.top 10)
316312 , benchIOSink value "bottom" (Stream.fold $ FL.bottom 10)
@@ -453,7 +449,6 @@ o_n_heap_serial value =
453449 , benchIOSink value "toStreamRev"
454450 (Stream.fold FL.toStreamRev
455451 :: Stream IO a -> IO (Stream Identity a))
456− , benchIOSink value "nub" (Stream.fold FL.nub)
457452 ]
458453 , bgroup "key-value"
459454 [
@@ -507,9 +502,7 @@ o_n_heap_serial value =
507502-------------------------------------------------------------------------------
508503
509504main :: IO ()
510−main = do
511−#ifndef FUSION_CHECK
512− runWithCLIOpts defaultStreamSize allBenchmarks
505+main = runWithCLIOpts defaultStreamSize allBenchmarks
513506
514507 where
515508
@@ -522,20 +515,3 @@ main = do
522515 , bgroup (o_n_space_prefix moduleName) (o_n_space_serial value)
523516 , bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
524517 ]
525−#else
526− -- Enable FUSION_CHECK macro at the beginning of the file
527− -- Enable one benchmark below, and run the benchmark
528− -- Check the .dump-simpl output
529− let value = 100000
530− let input = source value 1
531− let getKey buckets = (`mod` buckets)
532− let getFold k =
533− return $ case k of
534− 0 -> FL.sum
535− 1 -> FL.length
536− _ -> FL.length
537−
538− -- demuxToMap (getKey 64) (getFold . getKey 64) input
539− toIntMapIO (getKey 64) input
540− return ()
541−#endif
benchmark/Streamly/Benchmark/Data/Fold/Window.hs+53−66
@@ -4,16 +4,15 @@ module Main (main) where
44
55import Control.DeepSeq (NFData)
66import Streamly.Data.Fold (Fold)
7−import Streamly.Internal.Data.Stream (Stream)
7+import Streamly.Internal.Data.Stream.StreamD (Stream)
88import System.Random (randomRIO)
99
1010import qualified Streamly.Data.Fold as Fold
11−import qualified Streamly.Internal.Data.Fold as Window
12−import qualified Streamly.Internal.Data.Ring as Ring
13−import qualified Streamly.Internal.Data.Stream as Stream
11+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
12+import qualified Streamly.Internal.Data.Fold.Window as Window
13+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
1414
15−import Streamly.Benchmark.Common
16−import Test.Tasty.Bench
15+import Gauge
1716
1817{-# INLINE source #-}
1918source :: (Monad m, Num a, Stream.Enumerable a) =>
@@ -66,126 +65,114 @@ benchScanWith src len name f =
6665benchWithPostscan :: Int -> String -> Fold IO Double a -> Benchmark
6766benchWithPostscan = benchScanWith source
6867
69−o_1_space_folds :: Int -> [Benchmark]
70−o_1_space_folds numElements =
71− [ bgroup "fold"
68+{-# INLINE numElements #-}
69+numElements :: Int
70+numElements = 100000
71+
72+main :: IO ()
73+main =
74+ defaultMain
75+ [ bgroup
76+ "fold"
7277 [ benchWithFold numElements "minimum (window size 100)"
73− (Window.windowMinimum 100)
78+ (Window.minimum 100)
7479 , benchWithFold numElements "minimum (window size 1000)"
75− (Window.windowMinimum 1000)
80+ (Window.minimum 1000)
7681 , benchWith sourceDescendingInt numElements
7782 "minimum descending (window size 1000)"
78− (Window.windowMinimum 1000)
83+ (Window.minimum 1000)
7984
8085 , benchWithFold numElements "maximum (window size 100)"
81− (Window.windowMaximum 100)
86+ (Window.maximum 100)
8287 , benchWithFold numElements "maximum (window size 1000)"
83− (Window.windowMaximum 1000)
88+ (Window.maximum 1000)
8489 , benchWith sourceDescendingInt numElements
8590 "maximum descending (window size 1000)"
86− (Window.windowMaximum 1000)
91+ (Window.maximum 1000)
8792
8893 , benchWithFold numElements "range (window size 100)"
89− (Window.windowRange 100)
94+ (Window.range 100)
9095 , benchWithFold numElements "range (window size 1000)"
91− (Window.windowRange 1000)
96+ (Window.range 1000)
9297 , benchWith sourceDescendingInt numElements
9398 "range descending (window size 1000)"
94− (Window.windowRange 1000)
99+ (Window.range 1000)
95100
96101 , benchWithFoldInt numElements "sumInt (window size 100)"
97− (Ring.slidingWindow 100 Window.windowSumInt)
102+ (Ring.slidingWindow 100 Window.sumInt)
98103 , benchWithFoldInt numElements "sum for Int (window size 100)"
99− (Ring.slidingWindow 100 Window.windowSum)
104+ (Ring.slidingWindow 100 Window.sum)
100105 , benchWithFold numElements "sum (window size 100)"
101− (Ring.slidingWindow 100 Window.windowSum)
106+ (Ring.slidingWindow 100 Window.sum)
102107 , benchWithFold numElements "sum (window size 1000)"
103− (Ring.slidingWindow 1000 Window.windowSum)
108+ (Ring.slidingWindow 1000 Window.sum)
104109 , benchWithFold numElements "sum (entire stream)"
105− (Window.cumulative Window.windowSum)
110+ (Window.cumulative Window.sum)
106111 , benchWithFold numElements "sum (Data.Fold)"
107112 Fold.sum
108113
109114 , benchWithFold numElements "mean (window size 100)"
110− (Ring.slidingWindow 100 Window.windowMean)
115+ (Ring.slidingWindow 100 Window.mean)
111116 , benchWithFold numElements "mean (window size 1000)"
112− (Ring.slidingWindow 1000 Window.windowMean)
117+ (Ring.slidingWindow 1000 Window.mean)
113118 , benchWithFold numElements "mean (entire stream)"
114− (Window.cumulative Window.windowMean)
119+ (Window.cumulative Window.mean)
115120 , benchWithFold numElements "mean (Data.Fold)"
116121 Fold.mean
117122
118123 , benchWithFold numElements "powerSum 2 (window size 100)"
119− (Ring.slidingWindow 100 (Window.windowPowerSum 2))
124+ (Ring.slidingWindow 100 (Window.powerSum 2))
120125 , benchWithFold numElements "powerSum 2 (entire stream)"
121− (Window.cumulative (Window.windowPowerSum 2))
126+ (Window.cumulative (Window.powerSum 2))
122127
123128 ]
124− ]
125−
126−o_1_space_scans :: Int -> [Benchmark]
127−o_1_space_scans numElements =
128− [ bgroup "scan"
129+ , bgroup
130+ "scan"
129131 [ benchWithPostscan numElements "minimum (window size 10)"
130− (Window.windowMinimum 10)
132+ (Window.minimum 10)
131133 -- Below window size 30 the linear search based impl performs better
132134 -- than the dequeue based implementation.
133135 , benchWithPostscan numElements "minimum (window size 30)"
134− (Window.windowMinimum 30)
136+ (Window.minimum 30)
135137 , benchWithPostscan numElements "minimum (window size 1000)"
136− (Window.windowMinimum 1000)
138+ (Window.minimum 1000)
137139 , benchScanWith sourceDescendingInt numElements
138140 "minimum descending (window size 1000)"
139− (Window.windowMinimum 1000)
141+ (Window.minimum 1000)
140142
141143 , benchWithPostscan numElements "maximum (window size 10)"
142− (Window.windowMaximum 10)
144+ (Window.maximum 10)
143145 , benchWithPostscan numElements "maximum (window size 30)"
144− (Window.windowMaximum 30)
146+ (Window.maximum 30)
145147 , benchWithPostscan numElements "maximum (window size 1000)"
146− (Window.windowMaximum 1000)
148+ (Window.maximum 1000)
147149 , benchScanWith sourceDescendingInt numElements
148150 "maximum descending (window size 1000)"
149− (Window.windowMaximum 1000)
151+ (Window.maximum 1000)
150152
151153 , benchWithPostscan numElements "range (window size 10)"
152− (Window.windowRange 10)
154+ (Window.range 10)
153155 , benchWithPostscan numElements "range (window size 30)"
154− (Window.windowRange 30)
156+ (Window.range 30)
155157 , benchWithPostscan numElements "range (window size 1000)"
156− (Window.windowRange 1000)
158+ (Window.range 1000)
157159 , benchScanWith sourceDescendingInt numElements
158160 "range descending (window size 1000)"
159− (Window.windowRange 1000)
161+ (Window.range 1000)
160162
161163 , benchWithPostscan numElements "sum (window size 100)"
162− (Ring.slidingWindow 100 Window.windowSum)
164+ (Ring.slidingWindow 100 Window.sum)
163165 , benchWithPostscan numElements "sum (window size 1000)"
164− (Ring.slidingWindow 1000 Window.windowSum)
166+ (Ring.slidingWindow 1000 Window.sum)
165167
166168 , benchWithPostscan numElements "mean (window size 100)"
167− (Ring.slidingWindow 100 Window.windowMean)
169+ (Ring.slidingWindow 100 Window.mean)
168170 , benchWithPostscan numElements "mean (window size 1000)"
169− (Ring.slidingWindow 1000 Window.windowMean)
171+ (Ring.slidingWindow 1000 Window.mean)
170172
171173 , benchWithPostscan numElements "powerSum 2 (window size 100)"
172− (Ring.slidingWindow 100 (Window.windowPowerSum 2))
174+ (Ring.slidingWindow 100 (Window.powerSum 2))
173175 , benchWithPostscan numElements "powerSum 2 (window size 1000)"
174− (Ring.slidingWindow 1000 (Window.windowPowerSum 2))
176+ (Ring.slidingWindow 1000 (Window.powerSum 2))
175177 ]
176178 ]
177−
178−moduleName :: String
179−moduleName = "Data.Fold.Window"
180−
181−main :: IO ()
182−main = runWithCLIOpts defaultStreamSize allBenchmarks
183−
184− where
185−
186− allBenchmarks value =
187− [ bgroup (o_1_space_prefix moduleName) $ concat
188− [ o_1_space_folds value
189− , o_1_space_scans value
190− ]
191− ]
benchmark/Streamly/Benchmark/Data/Parser.hs+90−421
@@ -1,10 +1,5 @@
11{-# OPTIONS_GHC -Wno-deprecations #-}
22
3−#undef FUSION_CHECK
4−#ifdef FUSION_CHECK
5−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
6−#endif
7−
83-- |
94-- Module : Streamly.Benchmark.Data.Parser
105-- Copyright : (c) 2020 Composewell Technologies
@@ -22,34 +17,32 @@ module Main
2217 main
2318 ) where
2419
25−import Control.Applicative ((<|>))
2620import Control.DeepSeq (NFData(..))
27−import Data.Function ((&))
21+import Data.Foldable (asum)
2822import Data.Functor (($>))
2923import Data.Monoid (Sum(..))
3024import GHC.Magic (inline)
3125import GHC.Magic (noinline)
3226import System.IO (Handle)
3327import System.Random (randomRIO)
34−import Streamly.Internal.Data.Fold (Fold(..))
35−import Streamly.Internal.Data.Parser
36− (ParseError(..), Parser(..), Initial(..), Step(..))
37−import Streamly.Internal.Data.Stream (Stream)
28+import Streamly.Internal.Data.Parser (ParseError(..))
29+import Streamly.Internal.Data.Stream.StreamD (Stream)
3830import Prelude hiding
39− (any, all, take, sequence, sequence_, sequenceA, takeWhile, dropWhile, span)
31+ (any, all, take, sequence, sequence_, sequenceA, takeWhile, dropWhile)
4032
41−import qualified Control.Applicative as AP
42−import qualified Data.Foldable as F
4333import qualified Data.Traversable as TR
34+import qualified Data.Foldable as F
35+import qualified Control.Applicative as AP
4436import qualified Streamly.FileSystem.Handle as Handle
4537import qualified Streamly.Internal.Data.Array as Array
4638import qualified Streamly.Internal.Data.Fold as Fold
4739import qualified Streamly.Internal.Data.Parser as PR
4840import qualified Streamly.Data.Stream as Stream
4941import qualified Streamly.Internal.Data.Producer as Producer
50−import qualified Streamly.Internal.Data.Stream as Stream
42+import qualified Streamly.Internal.Data.Producer.Source as Source
43+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
5144
52−import Test.Tasty.Bench hiding (env)
45+import Gauge hiding (env)
5346import Streamly.Benchmark.Common
5447import Streamly.Benchmark.Common.Handle
5548
@@ -120,6 +113,18 @@ benchIOSink value name f =
120113-- Parsers
121114-------------------------------------------------------------------------------
122115
116+{-# INLINE one #-}
117+one :: Monad m => Int -> Stream m Int -> m (Either ParseError (Maybe Int))
118+one value = Stream.parse p
119+
120+ where
121+
122+ p = do
123+ m <- PR.fromFold Fold.one
124+ case m of
125+ Just i -> if i >= value then pure m else p
126+ Nothing -> pure Nothing
127+
123128{-# INLINE takeBetween #-}
124129takeBetween :: Monad m => Int -> Stream m a -> m (Either ParseError ())
125130takeBetween value = Stream.parse (PR.takeBetween 0 value Fold.drain)
@@ -128,10 +133,6 @@ takeBetween value = Stream.parse (PR.takeBetween 0 value Fold.drain)
128133takeEQ :: Monad m => Int -> Stream m a -> m (Either ParseError ())
129134takeEQ value = Stream.parse (PR.takeEQ value Fold.drain)
130135
131−{-# INLINE takeGE #-}
132−takeGE :: Monad m => Int -> Stream m a -> m (Either ParseError ())
133−takeGE value = Stream.parse (PR.takeGE value Fold.drain)
134−
135136{-# INLINE dropWhile #-}
136137dropWhile :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
137138dropWhile value = Stream.parse (PR.dropWhile (<= value))
@@ -186,26 +187,14 @@ wordBy :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
186187wordBy value = Stream.parse (PR.wordBy (>= value) Fold.drain)
187188
188189{-# INLINE sepByWords #-}
189−sepByWords :: Monad m => Stream m Int -> m (Either ParseError ())
190−sepByWords = Stream.parse (wrds even Fold.drain)
190+sepByWords :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
191+sepByWords _ = Stream.parse (wrds even Fold.drain)
191192 where
192193 wrds p = PR.sepBy (PR.takeWhile (not . p) Fold.drain) (PR.dropWhile p)
193194
194−{-# INLINE sepByAllWords #-}
195−sepByAllWords :: Monad m => Stream m Int -> m (Either ParseError ())
196−sepByAllWords = Stream.parse (wrds even Fold.drain)
197− where
198− wrds p = PR.sepByAll (PR.takeWhile (not . p) Fold.drain) (PR.dropWhile p)
199−
200−-- Returning a list to compare with the sepBy1 in ParserK
201−{-# INLINE sepBy1 #-}
202−sepBy1 :: Monad m => Stream m Int -> m (Either ParseError [Int])
203−sepBy1 xs = do
204− Stream.parse (PR.sepBy1 (PR.satisfy odd) (PR.satisfy even) Fold.toList) xs
205−
206195{-# INLINE sepByWords1 #-}
207−sepByWords1 :: Monad m => Stream m Int -> m (Either ParseError ())
208−sepByWords1 = Stream.parse (wrds even Fold.drain)
196+sepByWords1 :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
197+sepByWords1 _ = Stream.parse (wrds even Fold.drain)
209198 where
210199 wrds p = PR.sepBy1 (PR.takeWhile (not . p) Fold.drain) (PR.dropWhile p)
211200
@@ -219,26 +208,6 @@ deintercalate _ = Stream.parse (partition even)
219208 PR.deintercalate
220209 (PR.takeWhile (not . p) Fold.sum) (PR.takeWhile p Fold.sum) Fold.drain
221210
222−{-# INLINE deintercalate1 #-}
223−deintercalate1 :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
224−deintercalate1 _ = Stream.parse (partition even)
225−
226− where
227−
228− partition p =
229− PR.deintercalate1
230− (PR.takeWhile (not . p) Fold.sum) (PR.takeWhile p Fold.sum) Fold.drain
231−
232−{-# INLINE deintercalateAll #-}
233−deintercalateAll :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
234−deintercalateAll _ = Stream.parse (partition even)
235−
236− where
237−
238− partition p =
239− PR.deintercalateAll
240− (PR.takeWhile (not . p) Fold.sum) (PR.takeWhile p Fold.sum) Fold.drain
241−
242211{-# INLINE manyWordByEven #-}
243212manyWordByEven :: Monad m => Stream m Int -> m (Either ParseError ())
244213manyWordByEven = Stream.parse (PR.many (PR.wordBy even Fold.drain) Fold.drain)
@@ -268,44 +237,16 @@ manyTill :: Monad m => Int -> Stream m Int -> m (Either ParseError Int)
268237manyTill value =
269238 Stream.parse (PR.manyTill (PR.satisfy (> 0)) (PR.satisfy (== value)) Fold.length)
270239
271−{-# INLINE splitAp2 #-}
272−splitAp2 :: Monad m
240+{-# INLINE splitAp #-}
241+splitAp :: Monad m
273242 => Int -> Stream m Int -> m (Either ParseError ((), ()))
274−splitAp2 value =
243+splitAp value =
275244 Stream.parse
276245 ((,)
277246 <$> PR.dropWhile (<= (value `div` 2))
278247 <*> PR.dropWhile (<= value)
279248 )
280249
281−{-# INLINE splitAp4 #-}
282−splitAp4 :: Monad m
283− => Int -> Stream m Int -> m (Either ParseError ())
284−splitAp4 value =
285− Stream.parse
286− ( (\() () () () -> ())
287− <$> PR.dropWhile (<= (value * 1 `div` 4))
288− <*> PR.dropWhile (<= (value * 2 `div` 4))
289− <*> PR.dropWhile (<= (value * 3 `div` 4))
290− <*> PR.dropWhile (<= value)
291− )
292−
293−{-# INLINE splitAp8 #-}
294−splitAp8 :: Monad m
295− => Int -> Stream m Int -> m (Either ParseError ())
296−splitAp8 value =
297− Stream.parse
298− ( (\() () () () () () () () -> ())
299− <$> PR.dropWhile (<= (value * 1 `div` 8))
300− <*> PR.dropWhile (<= (value * 2 `div` 8))
301− <*> PR.dropWhile (<= (value * 3 `div` 8))
302− <*> PR.dropWhile (<= (value * 4 `div` 8))
303− <*> PR.dropWhile (<= (value * 5 `div` 8))
304− <*> PR.dropWhile (<= (value * 6 `div` 8))
305− <*> PR.dropWhile (<= (value * 7 `div` 8))
306− <*> PR.dropWhile (<= value)
307− )
308−
309250{-# INLINE splitApBefore #-}
310251splitApBefore :: Monad m
311252 => Int -> Stream m Int -> m (Either ParseError ())
@@ -324,10 +265,10 @@ splitApAfter value =
324265 <* PR.dropWhile (<= value)
325266 )
326267
327−{-# INLINE splitWith2 #-}
328−splitWith2 :: Monad m
268+{-# INLINE splitWith #-}
269+splitWith :: Monad m
329270 => Int -> Stream m Int -> m (Either ParseError ((), ()))
330−splitWith2 value =
271+splitWith value =
331272 Stream.parse
332273 (PR.splitWith (,)
333274 (PR.dropWhile (<= (value `div` 2)))
@@ -344,157 +285,6 @@ split_ value =
344285 (PR.dropWhile (<= value))
345286 )
346287
347−-- XXX dropWhile with applicative does not fuse
348−-- PR.dropWhile (<= (value * 1 `div` 4)) *> PR.die "alt"
349−{-# INLINE takeWhileFail #-}
350−takeWhileFail :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
351−takeWhileFail predicate (Fold fstep finitial _ ffinal) =
352− Parser step initial extract
353−
354− where
355−
356− initial = do
357− res <- finitial
358− return $ case res of
359− Fold.Partial s -> IPartial s
360− Fold.Done b -> IDone b
361−
362− step s a =
363− if predicate a
364− then do
365− fres <- fstep s a
366− return
367− $ case fres of
368− Fold.Partial s1 -> Partial 0 s1
369− Fold.Done b -> Done 0 b
370− else return $ Error "fail"
371−
372− extract s = fmap (Done 0) (ffinal s)
373−
374−{-# INLINE alt2 #-}
375−alt2 :: Monad m
376− => Int -> Stream m Int -> m (Either ParseError ())
377−alt2 value =
378− Stream.parse
379− (PR.alt
380− (takeWhileFail (<= (value `div` 2)) Fold.drain)
381− (PR.dropWhile (<= value))
382− )
383−
384−{-# INLINE alt4 #-}
385−alt4 :: Monad m
386− => Int -> Stream m Int -> m (Either ParseError ())
387−alt4 value =
388− Stream.parse
389− ( takeWhileFail (<= (value * 1 `div` 4)) Fold.drain
390− <|> takeWhileFail (<= (value * 2 `div` 4)) Fold.drain
391− <|> takeWhileFail (<= (value * 3 `div` 4)) Fold.drain
392− <|> PR.dropWhile (<= value)
393− )
394−
395−{-# INLINE alt8 #-}
396−alt8 :: Monad m
397− => Int -> Stream m Int -> m (Either ParseError ())
398−alt8 value =
399− Stream.parse
400− ( takeWhileFail (<= (value * 1 `div` 8)) Fold.drain
401− <|> takeWhileFail (<= (value * 2 `div` 8)) Fold.drain
402− <|> takeWhileFail (<= (value * 3 `div` 8)) Fold.drain
403− <|> takeWhileFail (<= (value * 4 `div` 8)) Fold.drain
404− <|> takeWhileFail (<= (value * 5 `div` 8)) Fold.drain
405− <|> takeWhileFail (<= (value * 6 `div` 8)) Fold.drain
406− <|> takeWhileFail (<= (value * 7 `div` 8)) Fold.drain
407− <|> PR.dropWhile (<= value)
408− )
409−
410−{-# INLINE alt16 #-}
411−alt16 :: Monad m
412− => Int -> Stream m Int -> m (Either ParseError ())
413−alt16 value =
414− Stream.parse
415− ( takeWhileFail (<= (value * 1 `div` 16)) Fold.drain
416− <|> takeWhileFail (<= (value * 2 `div` 16)) Fold.drain
417− <|> takeWhileFail (<= (value * 3 `div` 16)) Fold.drain
418− <|> takeWhileFail (<= (value * 4 `div` 16)) Fold.drain
419− <|> takeWhileFail (<= (value * 5 `div` 16)) Fold.drain
420− <|> takeWhileFail (<= (value * 6 `div` 16)) Fold.drain
421− <|> takeWhileFail (<= (value * 8 `div` 16)) Fold.drain
422− <|> takeWhileFail (<= (value * 9 `div` 16)) Fold.drain
423− <|> takeWhileFail (<= (value * 10 `div` 16)) Fold.drain
424− <|> takeWhileFail (<= (value * 11 `div` 16)) Fold.drain
425− <|> takeWhileFail (<= (value * 12 `div` 16)) Fold.drain
426− <|> takeWhileFail (<= (value * 13 `div` 16)) Fold.drain
427− <|> takeWhileFail (<= (value * 14 `div` 16)) Fold.drain
428− <|> takeWhileFail (<= (value * 15 `div` 16)) Fold.drain
429− <|> PR.dropWhile (<= value)
430− )
431−
432−{-# INLINE altSmall #-}
433−altSmall :: Monad m
434− => Int -> Stream m Int -> m ()
435−altSmall value =
436− Stream.fold Fold.drain .
437− Stream.parseMany
438− (PR.alt
439− (PR.satisfy (>= value) *> PR.die "alt")
440− (PR.satisfy (<= value))
441− )
442−
443−{-# INLINE monad #-}
444−monad :: Monad m
445− => Int -> Stream m Int -> m (Either ParseError ())
446−monad value =
447− Stream.parse
448− $ do
449− PR.dropWhile (<= (value `div` 2))
450− PR.dropWhile (<= value)
451−
452−{-# INLINE monad4 #-}
453−monad4 :: Monad m
454− => Int -> Stream m Int -> m (Either ParseError ())
455−monad4 value =
456− Stream.parse $ do
457− PR.dropWhile (<= (value `div` 4))
458− PR.dropWhile (<= (value `div` 2))
459− PR.dropWhile (<= (value * 3 `div` 4))
460− PR.dropWhile (<= value)
461−
462−{-# INLINE monad8 #-}
463−monad8 :: Monad m
464− => Int -> Stream m Int -> m (Either ParseError ())
465−monad8 value =
466− Stream.parse $ do
467− PR.dropWhile (<= (value * 1 `div` 8))
468− PR.dropWhile (<= (value * 2 `div` 8))
469− PR.dropWhile (<= (value * 3 `div` 8))
470− PR.dropWhile (<= (value * 4 `div` 8))
471− PR.dropWhile (<= (value * 5 `div` 8))
472− PR.dropWhile (<= (value * 6 `div` 8))
473− PR.dropWhile (<= (value * 7 `div` 8))
474− PR.dropWhile (<= value)
475−
476−{-# INLINE monad16 #-}
477−monad16 :: Monad m
478− => Int -> Stream m Int -> m (Either ParseError ())
479−monad16 value =
480− Stream.parse $ do
481− PR.dropWhile (<= (value * 1 `div` 16))
482− PR.dropWhile (<= (value * 2 `div` 16))
483− PR.dropWhile (<= (value * 3 `div` 16))
484− PR.dropWhile (<= (value * 4 `div` 16))
485− PR.dropWhile (<= (value * 5 `div` 16))
486− PR.dropWhile (<= (value * 6 `div` 16))
487− PR.dropWhile (<= (value * 7 `div` 16))
488− PR.dropWhile (<= (value * 8 `div` 16))
489− PR.dropWhile (<= (value * 9 `div` 16))
490− PR.dropWhile (<= (value * 10 `div` 16))
491− PR.dropWhile (<= (value * 11 `div` 16))
492− PR.dropWhile (<= (value * 12 `div` 16))
493− PR.dropWhile (<= (value * 13 `div` 16))
494− PR.dropWhile (<= (value * 14 `div` 16))
495− PR.dropWhile (<= (value * 15 `div` 16))
496− PR.dropWhile (<= value)
497−
498288{-# INLINE takeEndBy_ #-}
499289takeEndBy_ :: Monad m
500290 => Int -> Stream m Int -> m (Either ParseError ())
@@ -542,24 +332,6 @@ longestAllAny value =
542332 )
543333-}
544334
545−-------------------------------------------------------------------------------
546−-- Spanning
547−-------------------------------------------------------------------------------
548−
549−{-# INLINE span #-}
550−span :: Monad m => Int -> Stream m Int -> m (Either ParseError ((), ()))
551−span value = Stream.parse (PR.span (<= (value `div` 2)) Fold.drain Fold.drain)
552−
553−{-# INLINE spanBy #-}
554−spanBy :: Monad m => Int -> Stream m Int -> m (Either ParseError ((), ()))
555−spanBy value =
556− Stream.parse (PR.spanBy (\_ i -> i <= (value `div` 2)) Fold.drain Fold.drain)
557−
558−{-# INLINE spanByRolling #-}
559−spanByRolling :: Monad m => Int -> Stream m Int -> m (Either ParseError ((), ()))
560−spanByRolling value =
561− Stream.parse (PR.spanByRolling (\_ i -> i <= value `div` 2) Fold.drain Fold.drain)
562−
563335parseManyChunksOfSum :: Int -> Handle -> IO Int
564336parseManyChunksOfSum n inh =
565337 Stream.fold Fold.length
@@ -567,6 +339,22 @@ parseManyChunksOfSum n inh =
567339 (PR.fromFold $ Fold.take n Fold.sum)
568340 (Stream.unfold Handle.reader inh)
569341
342+-------------------------------------------------------------------------------
343+-- Parsing with unfolds
344+-------------------------------------------------------------------------------
345+
346+{-# INLINE parseManyUnfoldArrays #-}
347+parseManyUnfoldArrays :: Int -> [Array.Array Int] -> IO ()
348+parseManyUnfoldArrays count arrays = do
349+ let src = Source.source (Just (Producer.OuterLoop arrays))
350+ let parser = PR.fromFold (Fold.take count Fold.drain)
351+ let readSrc =
352+ Source.producer
353+ $ Producer.concat Producer.fromList Array.producer
354+ let streamParser =
355+ Producer.simplify (Source.parseMany parser readSrc)
356+ Stream.fold Fold.drain $ Stream.unfold streamParser src
357+
570358-------------------------------------------------------------------------------
571359-- Parsers in which -fspec-constr-recursive=16 is problematic
572360-------------------------------------------------------------------------------
@@ -580,20 +368,17 @@ lookAhead :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
580368lookAhead value =
581369 Stream.parse (PR.lookAhead (PR.takeWhile (<= value) Fold.drain) $> ())
582370
583−-- XXX The timing of this increased 3x after the stepify extract changes.
584−{-# INLINE sequenceA_ #-}
585−sequenceA_ :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
586−sequenceA_ value =
587− Stream.parse (F.sequenceA_ $ replicate value (PR.satisfy (> 0)))
588−
589−-- quadratic complexity
590371{-# INLINE sequenceA #-}
591372sequenceA :: Monad m => Int -> Stream m Int -> m Int
592373sequenceA value xs = do
593374 x <- Stream.parse (TR.sequenceA (replicate value (PR.satisfy (> 0)))) xs
594375 return $ length x
595376
596−-- quadratic complexity
377+{-# INLINE sequenceA_ #-}
378+sequenceA_ :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
379+sequenceA_ value =
380+ Stream.parse (F.sequenceA_ $ replicate value (PR.satisfy (> 0)))
381+
597382{-# INLINE sequence #-}
598383sequence :: Monad m => Int -> Stream m Int -> m Int
599384sequence value xs = do
@@ -602,21 +387,13 @@ sequence value xs = do
602387
603388{-# INLINE sequence_ #-}
604389sequence_ :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
605−sequence_ value xs =
606− Stream.parse (foldr f (return ()) (replicate value (PR.takeBetween 0 1 Fold.drain))) xs
607−
608− where
390+sequence_ value =
391+ Stream.parse (F.sequence_ $ replicate value (PR.satisfy (> 0)))
609392
610− {-# INLINE f #-}
611− f m k = m >>= (\_ -> k)
612−
613−-- choice using the "Alternative" instance with direct style parser type has
614−-- quadratic performance complexity.
615−--
616393{-# INLINE choiceAsum #-}
617394choiceAsum :: Monad m => Int -> Stream m Int -> m (Either ParseError Int)
618395choiceAsum value =
619− Stream.parse (F.asum (replicate value (PR.satisfy (< 0)))
396+ Stream.parse (asum (replicate value (PR.satisfy (< 0)))
620397 AP.<|> PR.satisfy (> 0))
621398
622399{-
@@ -627,22 +404,6 @@ choice value =
627404 (PR.choice (replicate value (PR.satisfy (< 0))) AP.<|> PR.satisfy (> 0))
628405-}
629406
630−-------------------------------------------------------------------------------
631−-- Parsing with unfolds
632−-------------------------------------------------------------------------------
633−
634−{-# INLINE parseManyUnfoldArrays #-}
635−parseManyUnfoldArrays :: Int -> [Array.Array Int] -> IO ()
636−parseManyUnfoldArrays count arrays = do
637− let src = Producer.source (Just (Producer.OuterLoop arrays))
638− let parser = PR.fromFold (Fold.take count Fold.drain)
639− let readSrc =
640− Producer.producer
641− $ Producer.concat Producer.fromList Array.producer
642− let streamParser =
643− Producer.simplify (Producer.parseMany parser readSrc)
644− Stream.fold Fold.drain $ Stream.unfold streamParser src
645−
646407-------------------------------------------------------------------------------
647408-- Stream transformation
648409-------------------------------------------------------------------------------
@@ -655,35 +416,6 @@ parseMany n =
655416 . Stream.catRights . Stream.parseMany (PR.fromFold $ Fold.take n Fold.mconcat)
656417 . fmap Sum
657418
658−{-# INLINE parseManyGroupBy #-}
659−parseManyGroupBy :: Monad m => (Int -> Int -> Bool) -> Stream m Int -> m ()
660−parseManyGroupBy cmp =
661− Stream.fold Fold.drain . Stream.parseMany (PR.groupBy cmp Fold.drain)
662−
663−{-# INLINE parseManyGroupsRolling #-}
664−parseManyGroupsRolling :: Monad m => Bool -> Stream m Int -> m ()
665−parseManyGroupsRolling b =
666− Stream.fold Fold.drain
667− . Stream.parseMany (PR.groupByRolling (\_ _ -> b) Fold.drain)
668−
669−{-# INLINE parseManyGroupsRollingEither #-}
670−parseManyGroupsRollingEither :: Monad m =>
671− (Int -> Int -> Bool) -> Int -> m ()
672−parseManyGroupsRollingEither cmp value = do
673− sourceUnfoldrM value 1
674− & Stream.parseMany (PR.groupByRollingEither cmp Fold.drain Fold.drain)
675− & Stream.fold Fold.drain
676−
677−{-# INLINE parseManyGroupsRollingEitherAlt #-}
678−parseManyGroupsRollingEitherAlt :: Monad m =>
679− (Int -> Int -> Bool) -> Int -> m ()
680−parseManyGroupsRollingEitherAlt cmp value = do
681− sourceUnfoldrM value 1
682− -- Make the input unsorted.
683− & fmap (\x -> if even x then x + 2 else x)
684− & Stream.parseMany (PR.groupByRollingEither cmp Fold.drain Fold.drain)
685− & Stream.fold Fold.drain
686−
687419{-# INLINE parseIterate #-}
688420parseIterate :: Monad m => Int -> Stream m Int -> m ()
689421parseIterate n =
@@ -697,8 +429,12 @@ parseIterate n =
697429
698430{-# INLINE concatSequence #-}
699431concatSequence :: Monad m => Stream m Int -> m (Either ParseError ())
700−concatSequence =
701− Stream.parse $ PR.sequence (Stream.repeat PR.one) Fold.drain
432+concatSequence = Stream.parse $ PR.concatSequence Fold.drain $ Stream.repeat PR.one
433+
434+{-# INLINE parseManyGroupBy #-}
435+parseManyGroupBy :: Monad m => (Int -> Int -> Bool) -> Stream m Int -> m ()
436+parseManyGroupBy cmp =
437+ Stream.fold Fold.drain . Stream.parseMany (PR.groupBy cmp Fold.drain)
702438
703439-------------------------------------------------------------------------------
704440-- Benchmarks
@@ -713,84 +449,48 @@ instance NFData ParseError where
713449
714450o_1_space_serial :: Int -> [Benchmark]
715451o_1_space_serial value =
716− [ benchIOSink value "takeBetween" $ takeBetween value
452+ [ benchIOSink value "one (fold)" $ one value
453+ , benchIOSink value "takeBetween" $ takeBetween value
454+ , benchIOSink value "takeEQ" $ takeEQ value
717455 , benchIOSink value "takeWhile" $ takeWhile value
718456 , benchIOSink value "takeWhileP" $ takeWhileP value
719457 , benchIOSink value "takeP" $ takeP value
720458 , benchIOSink value "dropWhile" $ dropWhile value
721459 , benchIOSink value "takeStartBy" $ takeStartBy value
722− , benchIOSink value "takeEndBy_" $ takeEndBy_ value
460+ , benchIOSrc sourceEscapedFrames value "takeFramedByEsc_"
461+ $ takeFramedByEsc_ value
723462 , benchIOSink value "groupBy" $ groupBy
724463 , benchIOSink value "groupByRolling" $ groupByRolling
725464 , benchIOSink value "wordBy" $ wordBy value
726− , benchIOSink value "sepBy (words)" sepByWords
727− , benchIOSink value "sepByAll (words)" sepByAllWords
728− , benchIOSink value "sepBy1 (words)" sepByWords1
465+ , benchIOSink value "sepBy (words)" $ sepByWords value
466+ , benchIOSink value "sepBy1 (words)" $ sepByWords1 value
729467 , benchIOSink value "deintercalate" $ deintercalate value
730− , benchIOSink value "deintercalate1" $ deintercalate1 value
731− , benchIOSink value "deintercalateAll" $ deintercalateAll value
732− -- Applicative and Monad
733− , benchIOSink value "splitAp2" $ splitAp2 value
734− , benchIOSink value "splitAp4" $ splitAp4 value
735− , benchIOSink value "splitAp8" $ splitAp8 value
468+ , benchIOSink value "splitAp" $ splitAp value
736469 , benchIOSink value "splitApBefore" $ splitApBefore value
737470 , benchIOSink value "splitApAfter" $ splitApAfter value
738− , benchIOSink value "splitWith2" $ splitWith2 value
739− , benchIOSink value "span" $ span value
740− , benchIOSink value "spanBy" $ spanBy value
741− , benchIOSink value "spanByRolling" $ spanByRolling value
742− , benchIOSink value "monad2" $ monad value
743− , benchIOSink value "monad4" $ monad4 value
744− , benchIOSink value "monad8" $ monad8 value
745− -- Alternative
746− , benchIOSink value "alt2parseMany" $ altSmall value
747− , benchIOSink value "alt2" $ alt2 value
748− , benchIOSink value "alt4" $ alt4 value
749− , benchIOSink value "alt8" $ alt8 value
750− , benchIOSink value "alt16" $ alt16 value
471+ , benchIOSink value "splitWith" $ splitWith value
472+ , benchIOSink value "takeEndBy_" $ takeEndBy_ value
751473 , benchIOSink value "many" many
752474 , benchIOSink value "many (wordBy even)" $ manyWordByEven
753475 , benchIOSink value "some" some
754476 , benchIOSink value "manyTill" $ manyTill value
755− , benchIOSink value "parseMany" $ parseMany value
756− , benchIOSink value "parseMany (take 1)" (parseMany 1)
757− , benchIOSink value "parseMany (take all)" (parseMany value)
758− , benchIOSink value "parseMany (groupBy (<))" (parseManyGroupBy (<))
759− , benchIOSink value "parseMany (groupBy (==))" (parseManyGroupBy (==))
760− , benchIOSink value "parseMany groupRollingBy (bound groups)"
761− $ parseManyGroupsRolling False
762− , benchIOSink value "parseMany groupRollingBy (1 group)"
763− $ parseManyGroupsRolling True
764− , bench "parseMany groupRollingByEither (Left)"
765− $ nfIO $ parseManyGroupsRollingEitherLeft
766− , bench "parseMany groupRollingByEither (Right)"
767− $ nfIO $ parseManyGroupsRollingEitherRight
768− , bench "parseMany groupRollingByEither (Alternating)"
769− $ nfIO $ parseManyGroupsRollingEitherAlt1
770− , benchIOSink value "parseIterate (take 1)" (parseIterate 1)
771− , benchIOSink value "parseIterate (take all)" (parseIterate value)
772− , benchIOSink value "concatSequence" concatSequence
773477 {-
774478 , benchIOSink value "tee" $ teeAllAny value
775479 , benchIOSink value "teeFst" $ teeFstAllAny value
776480 , benchIOSink value "shortest" $ shortestAllAny value
777481 , benchIOSink value "longest" $ longestAllAny value
778482 -}
483+ , benchIOSink value "parseMany (take 1)" (parseMany 1)
484+ , benchIOSink value "parseMany (take all)" (parseMany value)
485+ , benchIOSink value "parseIterate (take 1)" (parseIterate 1)
486+ , benchIOSink value "parseIterate (take all)" (parseIterate value)
487+ , benchIOSink value "concatSequence" concatSequence
488+ , benchIOSink value "parseMany (groupBy (<))" (parseManyGroupBy (<))
489+ , benchIOSink value "parseMany (groupBy (==))" (parseManyGroupBy (==))
490+ , benchIOSink value "listEqBy" (listEqBy value)
779491 , benchIOSink value "streamEqBy" (streamEqBy value)
780492 ]
781493
782− where
783−
784− {-# NOINLINE parseManyGroupsRollingEitherLeft #-}
785− parseManyGroupsRollingEitherLeft = parseManyGroupsRollingEither (<) value
786−
787− {-# NOINLINE parseManyGroupsRollingEitherRight #-}
788− parseManyGroupsRollingEitherRight = parseManyGroupsRollingEither (>) value
789−
790− {-# NOINLINE parseManyGroupsRollingEitherAlt1 #-}
791− parseManyGroupsRollingEitherAlt1 =
792− parseManyGroupsRollingEitherAlt (>) value
793−
794494o_1_space_filesystem :: BenchEnv -> [Benchmark]
795495o_1_space_filesystem env =
796496 [ mkBench ("S.parseMany (Fold.take " ++ show (bigSize env) ++ " Fold.sum)") env
@@ -810,38 +510,26 @@ o_1_space_serial_unfold bound arrays =
810510o_n_heap_serial :: Int -> [Benchmark]
811511o_n_heap_serial value =
812512 [
813− benchIOSink value "takeEQ" $ takeEQ value
814− , benchIOSink value "takeGE" $ takeGE value
815−
816513 -- lookahead benchmark holds the entire input till end
817− , benchIOSink value "lookAhead" $ lookAhead value
514+ benchIOSink value "lookAhead" $ lookAhead value
818515
819− -- o-n-heap because of backtracking
820− , benchIOSrc sourceEscapedFrames value "takeFramedByEsc_"
821− $ takeFramedByEsc_ value
516+ -- accumulates the results in a list
517+ , benchIOSink value "sequence" $ sequence value
518+ , benchIOSink value "sequenceA" $ sequenceA value
822519
520+ -- XXX why should this take O(n) heap, it discards the results?
521+ , benchIOSink value "sequence_" $ sequence_ value
522+ , benchIOSink value "sequenceA_" $ sequenceA_ value
823523 -- non-linear time complexity (parserD)
824524 , benchIOSink value "split_" $ split_ value
825− -- XXX Takes lot of space when run on a long stream, why?
826− , benchIOSink value "monad16" $ monad16 value
525+ -- XXX why O(n) heap?
526+ , benchIOSink value "choice (asum)" $ choiceAsum value
527+ -- , benchIOSink value "choice" $ choice value
827528
828529 -- These show non-linear time complexity.
829530 -- They accumulate the results in a list.
830− , benchIOSink value "sepBy1" sepBy1
831531 , benchIOSink value "manyAlt" manyAlt
832532 , benchIOSink value "someAlt" someAlt
833− , benchIOSink value "listEqBy" (listEqBy value)
834− ]
835−
836−-- accumulate results in a list in IO
837−o_n_space_serial :: Int -> [Benchmark]
838−o_n_space_serial value =
839− [ benchIOSink value "sequenceA/100" $ sequenceA (value `div` 100)
840− , benchIOSink value "sequenceA_/100" $ sequenceA_ (value `div` 100)
841− , benchIOSink value "sequence/100" $ sequence (value `div` 100)
842− , benchIOSink value "sequence_/100" $ sequence_ (value `div` 100)
843− , benchIOSink value "choice (asum)/100" $ choiceAsum (value `div` 100)
844− -- , benchIOSink value "choice/100" $ choice (value `div` 100)
845533 ]
846534
847535-------------------------------------------------------------------------------
@@ -850,13 +538,12 @@ o_n_space_serial value =
850538
851539main :: IO ()
852540main = do
853−#ifndef FUSION_CHECK
854541 env <- mkHandleBenchEnv
855542 runWithCLIOptsEnv defaultStreamSize alloc (allBenchmarks env)
856543
857544 where
858545
859− alloc value = Stream.fold Fold.toList $ Stream.chunksOf 100 $ sourceUnfoldrM value 0
546+ alloc value = Stream.fold Fold.toList $ Stream.arraysOf 100 $ sourceUnfoldrM value 0
860547
861548 allBenchmarks env arrays value =
862549 [ bgroup (o_1_space_prefix moduleName) (o_1_space_serial value)
@@ -866,22 +553,4 @@ main = do
866553 , bgroup (o_1_space_prefix moduleName)
867554 (o_1_space_serial_unfold value arrays)
868555 , bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
869− , bgroup (o_n_space_prefix moduleName) (o_n_space_serial value)
870556 ]
871−#else
872− -- Enable FUSION_CHECK macro at the beginning of the file
873− -- Enable one benchmark below, and run the benchmark
874− -- Check the .dump-simpl output
875− let value = 100000
876− -- let input = sourceUnfoldrM value 1
877− -- manyTill value input
878− -- deintercalate value input
879− -- deintercalate1 value input
880− -- deintercalateAll value input
881− -- sepByWords input
882− -- sepByAllWords input
883− -- sepBy1 input
884− -- sepByWords1 input
885− takeFramedByEsc_ value (sourceEscapedFrames value 1)
886− return ()
887−#endif
benchmark/Streamly/Benchmark/Data/Parser/ParserD.hsnew file+461
@@ -0,0 +1,461 @@
1+{-# OPTIONS_GHC -Wno-deprecations #-}
2+
3+-- |
4+-- Module : Streamly.Benchmark.Data.ParserD
5+-- Copyright : (c) 2020 Composewell Technologies
6+--
7+-- License : BSD-3-Clause
8+-- Maintainer : streamly@composewell.com
9+
10+{-# LANGUAGE FlexibleContexts #-}
11+{-# LANGUAGE ScopedTypeVariables #-}
12+{-# OPTIONS_GHC -fspec-constr-recursive=4 #-}
13+{-# OPTIONS_GHC -Wno-orphans #-}
14+
15+module Main
16+ (
17+ main
18+ ) where
19+
20+import Control.DeepSeq (NFData(..))
21+import Data.Foldable (asum)
22+import Data.Function ((&))
23+import Data.Functor (($>))
24+import Streamly.Internal.Data.Parser (ParseError(..))
25+import Streamly.Internal.Data.Stream.StreamD (Stream)
26+import System.Random (randomRIO)
27+import Prelude hiding (any, all, take, sequence, sequenceA, sequence_, takeWhile, span)
28+
29+import qualified Data.Traversable as TR
30+import qualified Data.Foldable as F
31+import qualified Control.Applicative as AP
32+import qualified Streamly.Internal.Data.Array as Array
33+import qualified Streamly.Internal.Data.Fold as Fold
34+import qualified Streamly.Internal.Data.Parser.ParserD as PR
35+import qualified Streamly.Internal.Data.Producer as Producer
36+import qualified Streamly.Internal.Data.Producer.Source as Source
37+import qualified Streamly.Data.Stream as Stream
38+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
39+
40+import Gauge
41+import Streamly.Benchmark.Common
42+
43+-------------------------------------------------------------------------------
44+-- Utilities
45+-------------------------------------------------------------------------------
46+
47+-- XXX these can be moved to the common module
48+
49+{-# INLINE sourceUnfoldrM #-}
50+sourceUnfoldrM :: Monad m => Int -> Int -> Stream m Int
51+sourceUnfoldrM value n = Stream.unfoldrM step n
52+ where
53+ step cnt =
54+ if cnt > n + value
55+ then return Nothing
56+ else return (Just (cnt, cnt + 1))
57+
58+-- | Takes a fold method, and uses it with a default source.
59+{-# INLINE benchIOSink #-}
60+benchIOSink
61+ :: (NFData b)
62+ => Int -> String -> (Stream IO Int -> IO b) -> Benchmark
63+benchIOSink value name f =
64+ bench name $ nfIO $ randomRIO (1,1) >>= f . sourceUnfoldrM value
65+
66+-------------------------------------------------------------------------------
67+-- Parsers
68+-------------------------------------------------------------------------------
69+
70+{-# INLINE listEqBy #-}
71+listEqBy :: Int -> Stream IO Int -> IO (Either ParseError [Int])
72+listEqBy len = Stream.parseD (PR.listEqBy (==) [1 .. len])
73+
74+{-# INLINE streamEqBy #-}
75+streamEqBy :: Int -> Stream IO Int -> IO (Either ParseError ())
76+streamEqBy len = Stream.parseD (PR.streamEqBy (==) (Stream.enumerateFromToIntegral 1 len))
77+
78+{-# INLINE drainWhile #-}
79+drainWhile :: Monad m => (a -> Bool) -> PR.Parser a m ()
80+drainWhile p = PR.takeWhile p Fold.drain
81+
82+{-# INLINE takeStartBy #-}
83+takeStartBy :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
84+takeStartBy value stream = do
85+ let stream2 = value `Stream.cons` stream
86+ Stream.parseD (PR.takeStartBy (== value) Fold.drain) stream2
87+
88+{-# INLINE takeWhile #-}
89+takeWhile :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
90+takeWhile value = Stream.parseD (drainWhile (<= value))
91+
92+{-# INLINE takeP #-}
93+takeP :: Monad m => Int -> Stream m a -> m (Either ParseError ())
94+takeP value = Stream.parseD (PR.takeP value (PR.fromFold Fold.drain))
95+
96+{-# INLINE takeBetween #-}
97+takeBetween :: Monad m => Int -> Stream m a -> m (Either ParseError ())
98+takeBetween value = Stream.parseD (PR.takeBetween 0 value Fold.drain)
99+
100+{-# INLINE groupBy #-}
101+groupBy :: Monad m => Stream m Int -> m (Either ParseError ())
102+groupBy = Stream.parseD (PR.groupBy (<=) Fold.drain)
103+
104+{-# INLINE groupByRolling #-}
105+groupByRolling :: Monad m => Stream m Int -> m (Either ParseError ())
106+groupByRolling = Stream.parseD (PR.groupByRolling (<=) Fold.drain)
107+
108+{-# INLINE wordBy #-}
109+wordBy :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
110+wordBy value = Stream.parseD (PR.wordBy (>= value) Fold.drain)
111+
112+{-# INLINE manyWordByEven #-}
113+manyWordByEven :: Monad m => Stream m Int -> m (Either ParseError ())
114+manyWordByEven =
115+ Stream.parseD (PR.many (PR.wordBy (Prelude.even) Fold.drain) Fold.drain)
116+
117+{-# INLINE many #-}
118+many :: Monad m => Stream m Int -> m (Either ParseError Int)
119+many = Stream.parseD (PR.many (PR.satisfy (> 0)) Fold.length)
120+
121+{-# INLINE manyAlt #-}
122+manyAlt :: Monad m => Stream m Int -> m Int
123+manyAlt xs = do
124+ x <- Stream.parseD (AP.many (PR.satisfy (> 0))) xs
125+ return $ Prelude.length x
126+
127+{-# INLINE some #-}
128+some :: Monad m => Stream m Int -> m (Either ParseError Int)
129+some = Stream.parseD (PR.some (PR.satisfy (> 0)) Fold.length)
130+
131+{-# INLINE someAlt #-}
132+someAlt :: Monad m => Stream m Int -> m Int
133+someAlt xs = do
134+ x <- Stream.parseD (AP.some (PR.satisfy (> 0))) xs
135+ return $ Prelude.length x
136+
137+{-#INLINE takeEndBy_ #-}
138+takeEndBy_ :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
139+takeEndBy_ value = Stream.parseD (PR.takeEndBy_ (>= value) (PR.fromFold Fold.drain))
140+
141+{-# INLINE manyTill #-}
142+manyTill :: Monad m => Int -> Stream m Int -> m (Either ParseError Int)
143+manyTill value =
144+ let p = PR.satisfy (> 0)
145+ pcond = PR.satisfy (== value)
146+ in Stream.parseD (PR.manyTill p pcond Fold.length)
147+
148+{-# INLINE splitWith #-}
149+splitWith :: Monad m
150+ => Int -> Stream m Int -> m (Either ParseError ((), ()))
151+splitWith value =
152+ Stream.parseD
153+ ((,)
154+ <$> drainWhile (<= (value `div` 2))
155+ <*> drainWhile (<= value)
156+ )
157+
158+{-
159+{-# INLINE teeAllAny #-}
160+teeAllAny :: Monad m
161+ => Int -> Stream m Int -> m ((), ())
162+teeAllAny value =
163+ Stream.parseD
164+ (PR.teeWith (,)
165+ (drainWhile (<= value))
166+ (drainWhile (<= value))
167+ )
168+
169+{-# INLINE teeFstAllAny #-}
170+teeFstAllAny :: Monad m
171+ => Int -> Stream m Int -> m ((), ())
172+teeFstAllAny value =
173+ Stream.parseD
174+ (PR.teeWithFst (,)
175+ (drainWhile (<= value))
176+ (drainWhile (<= value))
177+ )
178+
179+{-# INLINE shortestAllAny #-}
180+shortestAllAny :: Monad m
181+ => Int -> Stream m Int -> m ()
182+shortestAllAny value =
183+ Stream.parseD
184+ (PR.shortest
185+ (drainWhile (<= value))
186+ (drainWhile (<= value))
187+ )
188+
189+{-# INLINE longestAllAny #-}
190+longestAllAny :: Monad m
191+ => Int -> Stream m Int -> m ()
192+longestAllAny value =
193+ Stream.parseD
194+ (PR.longest
195+ (drainWhile (<= value))
196+ (drainWhile (<= value))
197+ )
198+-}
199+
200+{-# INLINE sequenceParser #-}
201+sequenceParser :: Monad m => Stream m Int -> m (Either ParseError ())
202+sequenceParser =
203+ Stream.parseD
204+ (PR.sequence (Stream.repeat (PR.satisfy $ const True)) Fold.drain)
205+
206+-------------------------------------------------------------------------------
207+-- Spanning
208+-------------------------------------------------------------------------------
209+
210+{-# INLINE span #-}
211+span :: Monad m => Int -> Stream m Int -> m (Either ParseError ((), ()))
212+span value = Stream.parseD (PR.span (<= (value `div` 2)) Fold.drain Fold.drain)
213+
214+{-# INLINE spanBy #-}
215+spanBy :: Monad m => Int -> Stream m Int -> m (Either ParseError ((), ()))
216+spanBy value =
217+ Stream.parseD (PR.spanBy (\_ i -> i <= (value `div` 2)) Fold.drain Fold.drain)
218+
219+{-# INLINE spanByRolling #-}
220+spanByRolling :: Monad m => Int -> Stream m Int -> m (Either ParseError ((), ()))
221+spanByRolling value =
222+ Stream.parseD (PR.spanByRolling (\_ i -> i <= value `div` 2) Fold.drain Fold.drain)
223+
224+-------------------------------------------------------------------------------
225+-- Nested
226+-------------------------------------------------------------------------------
227+
228+{-# INLINE parseMany #-}
229+parseMany :: Monad m => Int -> Stream m Int -> m ()
230+parseMany value = Stream.fold Fold.drain . Stream.parseManyD (drainWhile (<= value + 1))
231+
232+{-# INLINE parseManyGroups #-}
233+parseManyGroups :: Monad m => Bool -> Stream m Int -> m ()
234+parseManyGroups b = Stream.fold Fold.drain . Stream.parseManyD (PR.groupBy (\_ _ -> b) Fold.drain)
235+
236+{-# INLINE parseManyGroupsRolling #-}
237+parseManyGroupsRolling :: Monad m => Bool -> Stream m Int -> m ()
238+parseManyGroupsRolling b =
239+ Stream.fold Fold.drain . Stream.parseManyD (PR.groupByRolling (\_ _ -> b) Fold.drain)
240+
241+{-# INLINE parseManyGroupsRollingEither #-}
242+parseManyGroupsRollingEither :: Monad m =>
243+ (Int -> Int -> Bool) -> Int -> m ()
244+parseManyGroupsRollingEither cmp value = do
245+ sourceUnfoldrM value 1
246+ & Stream.parseManyD (PR.groupByRollingEither cmp Fold.drain Fold.drain)
247+ & Stream.fold Fold.drain
248+
249+{-# INLINE parseManyGroupsRollingEitherAlt #-}
250+parseManyGroupsRollingEitherAlt :: Monad m =>
251+ (Int -> Int -> Bool) -> Int -> m ()
252+parseManyGroupsRollingEitherAlt cmp value = do
253+ sourceUnfoldrM value 1
254+ -- Make the input unsorted.
255+ & fmap (\x -> if even x then x + 2 else x)
256+ & Stream.parseManyD (PR.groupByRollingEither cmp Fold.drain Fold.drain)
257+ & Stream.fold Fold.drain
258+
259+-------------------------------------------------------------------------------
260+-- Parsing with unfolds
261+-------------------------------------------------------------------------------
262+
263+{-# INLINE parseManyUnfoldArrays #-}
264+parseManyUnfoldArrays :: Int -> [Array.Array Int] -> IO ()
265+parseManyUnfoldArrays count arrays = do
266+ let src = Source.source (Just (Producer.OuterLoop arrays))
267+ let parser = PR.fromFold (Fold.take count Fold.drain)
268+ let readSrc =
269+ Source.producer
270+ $ Producer.concat Producer.fromList Array.producer
271+ let streamParser =
272+ Producer.simplify (Source.parseManyD parser readSrc)
273+ Stream.fold Fold.drain $ Stream.unfold streamParser src
274+
275+-------------------------------------------------------------------------------
276+-- Parsers in which -fspec-constr-recursive=16 is problematic
277+-------------------------------------------------------------------------------
278+
279+-- XXX -fspec-constr-recursive=16 makes GHC go beserk when compiling these.
280+-- We need to fix GHC so that we can have better control over that option or do
281+-- not have to rely on it.
282+--
283+{-# INLINE lookAhead #-}
284+lookAhead :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
285+lookAhead value =
286+ Stream.parseD (PR.lookAhead (PR.takeWhile (<= value) Fold.drain) $> ())
287+
288+-- XXX The timing of this increased 3x after the stepify extract changes.
289+{-# INLINE sequenceA_ #-}
290+sequenceA_ :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
291+sequenceA_ value =
292+ Stream.parseD (F.sequenceA_ $ replicate value (PR.satisfy (> 0)))
293+
294+-- quadratic complexity
295+{-# INLINE sequenceA #-}
296+sequenceA :: Monad m => Int -> Stream m Int -> m Int
297+sequenceA value xs = do
298+ x <- Stream.parseD (TR.sequenceA (replicate value (PR.satisfy (> 0)))) xs
299+ return $ length x
300+
301+-- quadratic complexity
302+{-# INLINE sequence #-}
303+sequence :: Monad m => Int -> Stream m Int -> m Int
304+sequence value xs = do
305+ x <- Stream.parseD (TR.sequence (replicate value (PR.satisfy (> 0)))) xs
306+ return $ length x
307+
308+{-# INLINE sequence_ #-}
309+sequence_ :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
310+sequence_ value xs =
311+ Stream.parseD (foldr f (return ()) (replicate value (PR.takeBetween 0 1 Fold.drain))) xs
312+
313+ where
314+
315+ {-# INLINE f #-}
316+ f m k = m >>= (\_ -> k)
317+
318+-- choice using the "Alternative" instance with direct style parser type has
319+-- quadratic performance complexity.
320+--
321+{-# INLINE choiceAsum #-}
322+choiceAsum :: Monad m => Int -> Stream m Int -> m (Either ParseError Int)
323+choiceAsum value =
324+ Stream.parseD (asum (replicate value (PR.satisfy (< 0)))
325+ AP.<|> PR.satisfy (> 0))
326+
327+{-
328+{-# INLINE choice #-}
329+choice :: Monad m => Int -> Stream m Int -> m Int
330+choice value =
331+ Stream.parseD
332+ (PR.choice (replicate value (PR.satisfy (< 0))) AP.<|> PR.satisfy (> 0))
333+-}
334+
335+-------------------------------------------------------------------------------
336+-- Benchmarks
337+-------------------------------------------------------------------------------
338+
339+moduleName :: String
340+moduleName = "Data.Parser.ParserD"
341+
342+instance NFData ParseError where
343+ {-# INLINE rnf #-}
344+ rnf (ParseError x) = rnf x
345+
346+o_1_space_serial :: Int -> [Benchmark]
347+o_1_space_serial value =
348+ [ benchIOSink value "takeWhile" $ takeWhile value
349+ , benchIOSink value "takeP" $ takeP value
350+ , benchIOSink value "takeBetween" $ takeBetween value
351+ , benchIOSink value "takeStartBy" $ takeStartBy value
352+ , benchIOSink value "groupBy" $ groupBy
353+ , benchIOSink value "groupByRolling" $ groupByRolling
354+ , benchIOSink value "wordBy" $ wordBy value
355+ , benchIOSink value "splitWith" $ splitWith value
356+ , benchIOSink value "many" many
357+ , benchIOSink value "many (wordBy even)" $ manyWordByEven
358+ , benchIOSink value "some" some
359+ , benchIOSink value "takeEndBy_" $ takeEndBy_ value
360+ , benchIOSink value "manyTill" $ manyTill value
361+ {-
362+ , benchIOSink value "tee (all,any)" $ teeAllAny value
363+ , benchIOSink value "teeFst (all,any)" $ teeFstAllAny value
364+ , benchIOSink value "shortest (all,any)" $ shortestAllAny value
365+ , benchIOSink value "longest (all,any)" $ longestAllAny value
366+ -}
367+ , benchIOSink value "sequenceParser" sequenceParser
368+ , benchIOSink value "listEqBy" (listEqBy value)
369+ , benchIOSink value "streamEqBy" (streamEqBy value)
370+ ]
371+
372+o_1_space_serial_spanning :: Int -> [Benchmark]
373+o_1_space_serial_spanning value =
374+ [ benchIOSink value "span" $ span value
375+ , benchIOSink value "spanBy" $ spanBy value
376+ , benchIOSink value "spanByRolling" $ spanByRolling value
377+ ]
378+
379+o_1_space_serial_nested :: Int -> [Benchmark]
380+o_1_space_serial_nested value =
381+ [ benchIOSink value "parseMany" $ parseMany value
382+ , benchIOSink value "parseMany groupBy (bound groups)"
383+ $ parseManyGroups False
384+ , benchIOSink value "parseMany groupRollingBy (bound groups)"
385+ $ parseManyGroupsRolling False
386+ , benchIOSink value "parseMany groupBy (1 group)" $ parseManyGroups True
387+ , benchIOSink value "parseMany groupRollingBy (1 group)"
388+ $ parseManyGroupsRolling True
389+ , bench "parseMany groupRollingByEither (Left)"
390+ $ nfIO $ parseManyGroupsRollingEitherLeft
391+ , bench "parseMany groupRollingByEither (Right)"
392+ $ nfIO $ parseManyGroupsRollingEitherRight
393+ , bench "parseMany groupRollingByEither (Alternating)"
394+ $ nfIO $ parseManyGroupsRollingEitherAlt1
395+ ]
396+
397+ where
398+
399+ {-# NOINLINE parseManyGroupsRollingEitherLeft #-}
400+ parseManyGroupsRollingEitherLeft = parseManyGroupsRollingEither (<) value
401+
402+ {-# NOINLINE parseManyGroupsRollingEitherRight #-}
403+ parseManyGroupsRollingEitherRight = parseManyGroupsRollingEither (>) value
404+
405+ {-# NOINLINE parseManyGroupsRollingEitherAlt1 #-}
406+ parseManyGroupsRollingEitherAlt1 =
407+ parseManyGroupsRollingEitherAlt (>) value
408+
409+o_1_space_serial_unfold :: Int -> [Array.Array Int] -> [Benchmark]
410+o_1_space_serial_unfold bound arrays =
411+ [ bench "parseMany/Unfold/1000 arrays/1 parse"
412+ $ nfIO $ parseManyUnfoldArrays bound arrays
413+ , bench "parseMany/Unfold/1000 arrays/100000 parses"
414+ $ nfIO $ parseManyUnfoldArrays 1 arrays
415+ ]
416+
417+o_n_heap_serial :: Int -> [Benchmark]
418+o_n_heap_serial value =
419+ [
420+ -- lookahead benchmark holds the entire input till end
421+ benchIOSink value "lookAhead" $ lookAhead value
422+
423+ -- These show non-linear time complexity
424+ -- They accumulate the results in a list.
425+ , benchIOSink value "manyAlt" manyAlt
426+ , benchIOSink value "someAlt" someAlt
427+ ]
428+
429+-- accumulate results in a list in IO
430+o_n_space_serial :: Int -> [Benchmark]
431+o_n_space_serial value =
432+ [ benchIOSink value "sequenceA/100" $ sequenceA (value `div` 100)
433+ , benchIOSink value "sequenceA_/100" $ sequenceA_ (value `div` 100)
434+ , benchIOSink value "sequence/100" $ sequence (value `div` 100)
435+ , benchIOSink value "sequence_/100" $ sequence_ (value `div` 100)
436+ , benchIOSink value "choice (asum)/100" $ choiceAsum (value `div` 100)
437+ -- , benchIOSink value "choice/100" $ choice (value `div` 100)
438+ ]
439+
440+-------------------------------------------------------------------------------
441+-- Driver
442+-------------------------------------------------------------------------------
443+
444+main :: IO ()
445+main = do
446+ x <- randomRIO (1,1)
447+ runWithCLIOptsEnv (defaultStreamSize + x) alloc allBenchmarks
448+
449+ where
450+
451+ alloc value = Stream.fold Fold.toList $ Stream.arraysOf 100 $ sourceUnfoldrM value 0
452+
453+ allBenchmarks arraysSmall value =
454+ [ bgroup (o_1_space_prefix moduleName) (o_1_space_serial value)
455+ , bgroup (o_1_space_prefix moduleName) (o_1_space_serial_spanning value)
456+ , bgroup (o_1_space_prefix moduleName) (o_1_space_serial_nested value)
457+ , bgroup (o_1_space_prefix moduleName)
458+ (o_1_space_serial_unfold value arraysSmall)
459+ , bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
460+ , bgroup (o_n_space_prefix moduleName) (o_n_space_serial value)
461+ ]
benchmark/Streamly/Benchmark/Data/Parser/ParserK.hsnew file+167
@@ -0,0 +1,167 @@
1+-- |
2+-- Module : Streamly.Benchmark.Data.ParserK
3+-- Copyright : (c) 2020 Composewell Technologies
4+--
5+-- License : BSD-3-Clause
6+-- Maintainer : streamly@composewell.com
7+
8+{-# LANGUAGE CPP #-}
9+{-# LANGUAGE FlexibleContexts #-}
10+{-# OPTIONS_GHC -Wno-orphans #-}
11+
12+module Main
13+ (
14+ main
15+ ) where
16+
17+import Control.DeepSeq (NFData(..))
18+import Data.Foldable (asum)
19+import Streamly.Internal.Data.Parser (ParseError(..))
20+import Streamly.Internal.Data.Stream.StreamD (Stream)
21+import System.Random (randomRIO)
22+import Prelude hiding (any, all, take, sequence, sequenceA, takeWhile)
23+
24+import qualified Control.Applicative as AP
25+import qualified Data.Foldable as F
26+import qualified Data.Traversable as TR
27+import qualified Streamly.Internal.Data.Fold as FL
28+import qualified Streamly.Internal.Data.Parser.ParserK.Type as PR
29+import qualified Streamly.Internal.Data.Parser.ParserD as PRD
30+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
31+
32+import Gauge
33+import Streamly.Benchmark.Common
34+
35+-------------------------------------------------------------------------------
36+-- Utilities
37+-------------------------------------------------------------------------------
38+
39+-- XXX these can be moved to the common module
40+
41+{-# INLINE sourceUnfoldrM #-}
42+sourceUnfoldrM :: Monad m => Int -> Int -> Stream m Int
43+sourceUnfoldrM value n = Stream.unfoldrM step n
44+ where
45+ step cnt =
46+ if cnt > n + value
47+ then return Nothing
48+ else return (Just (cnt, cnt + 1))
49+
50+-- | Takes a fold method, and uses it with a default source.
51+{-# INLINE benchIOSink #-}
52+benchIOSink
53+ :: NFData b
54+ => Int -> String -> (Stream IO Int -> IO b) -> Benchmark
55+benchIOSink value name f =
56+ bench name $ nfIO $ randomRIO (1,1) >>= f . sourceUnfoldrM value
57+
58+-------------------------------------------------------------------------------
59+-- Parsers
60+-------------------------------------------------------------------------------
61+
62+#ifdef FROM_PARSERK
63+#define PARSE_OP (Stream.parseD . PRD.fromParserK)
64+#else
65+#define PARSE_OP Stream.parse
66+#endif
67+
68+{-# INLINE satisfy #-}
69+satisfy :: Monad m => (a -> Bool) -> PR.Parser a m a
70+satisfy = PRD.toParserK . PRD.satisfy
71+
72+{-# INLINE takeWhile #-}
73+takeWhile :: Monad m => (a -> Bool) -> PR.Parser a m ()
74+takeWhile p = PRD.toParserK $ PRD.takeWhile p FL.drain
75+
76+{-# INLINE takeWhileK #-}
77+takeWhileK :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
78+takeWhileK value = PARSE_OP (takeWhile (<= value))
79+
80+{-# INLINE splitApp #-}
81+splitApp :: Monad m
82+ => Int -> Stream m Int -> m (Either ParseError ((), ()))
83+splitApp value =
84+ PARSE_OP ((,) <$> takeWhile (<= (value `div` 2)) <*> takeWhile (<= value))
85+
86+{-# INLINE sequenceA #-}
87+sequenceA :: Monad m => Int -> Stream m Int -> m Int
88+sequenceA value xs = do
89+ let parser = satisfy (> 0)
90+ list = Prelude.replicate value parser
91+ x <- PARSE_OP (TR.sequenceA list) xs
92+ return $ Prelude.length x
93+
94+{-# INLINE sequenceA_ #-}
95+sequenceA_ :: Monad m => Int -> Stream m Int -> m (Either ParseError ())
96+sequenceA_ value xs = do
97+ let parser = satisfy (> 0)
98+ list = Prelude.replicate value parser
99+ PARSE_OP (F.sequenceA_ list) xs
100+
101+{-# INLINE sequence #-}
102+sequence :: Monad m => Int -> Stream m Int -> m Int
103+sequence value xs = do
104+ let parser = satisfy (> 0)
105+ list = Prelude.replicate value parser
106+ x <- PARSE_OP (TR.sequence list) xs
107+ return $ Prelude.length x
108+
109+{-# INLINE manyAlt #-}
110+manyAlt :: Monad m => Stream m Int -> m Int
111+manyAlt xs = do
112+ x <- PARSE_OP (AP.many (satisfy (> 0))) xs
113+ return $ Prelude.length x
114+
115+{-# INLINE someAlt #-}
116+someAlt :: Monad m => Stream m Int -> m Int
117+someAlt xs = do
118+ x <- PARSE_OP (AP.some (satisfy (> 0))) xs
119+ return $ Prelude.length x
120+
121+{-# INLINE choice #-}
122+choice :: Monad m => Int -> Stream m Int -> m (Either ParseError Int)
123+choice value =
124+ PARSE_OP (asum (replicate value (satisfy (< 0)))
125+ AP.<|> satisfy (> 0))
126+
127+-------------------------------------------------------------------------------
128+-- Benchmarks
129+-------------------------------------------------------------------------------
130+
131+moduleName :: String
132+moduleName = "Data.Parser.ParserK"
133+
134+instance NFData ParseError where
135+ {-# INLINE rnf #-}
136+ rnf (ParseError x) = rnf x
137+
138+o_1_space_serial :: Int -> [Benchmark]
139+o_1_space_serial value =
140+ [ benchIOSink value "takeWhile" $ takeWhileK value
141+ , benchIOSink value "splitApp" $ splitApp value
142+ ]
143+
144+-- O(n) heap beacuse of accumulation of the list in strict IO monad?
145+o_n_heap_serial :: Int -> [Benchmark]
146+o_n_heap_serial value =
147+ [ benchIOSink value "sequenceA" $ sequenceA value
148+ , benchIOSink value "sequenceA_" $ sequenceA_ value
149+ , benchIOSink value "sequence" $ sequence value
150+ , benchIOSink value "manyAlt" manyAlt
151+ , benchIOSink value "someAlt" someAlt
152+ , benchIOSink value "choice" $ choice value
153+ ]
154+
155+-------------------------------------------------------------------------------
156+-- Driver
157+-------------------------------------------------------------------------------
158+
159+main :: IO ()
160+main = runWithCLIOpts defaultStreamSize allBenchmarks
161+
162+ where
163+
164+ allBenchmarks value =
165+ [ bgroup (o_1_space_prefix moduleName) (o_1_space_serial value)
166+ , bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
167+ ]
benchmark/Streamly/Benchmark/Data/ParserK.hsdeleted−437
@@ -1,437 +0,0 @@
1−-- |
2−-- Module : Streamly.Benchmark.Data.ParserK
3−-- Copyright : (c) 2020 Composewell Technologies
4−--
5−-- License : BSD-3-Clause
6−-- Maintainer : streamly@composewell.com
7−
8−-- BENCH_CHUNKED -> adaptC
9−-- BENCH_CHUNKED_GENERIC -> adaptCG
10−-- BENCH_SINGULAR -> adapt
11−
12−{-# LANGUAGE CPP #-}
13−{-# LANGUAGE FlexibleContexts #-}
14−{-# OPTIONS_GHC -Wno-orphans #-}
15−
16−module Main
17− (
18− main
19− ) where
20−
21−import Control.Applicative ((<|>))
22−import Control.DeepSeq (NFData(..))
23−import Control.Monad.IO.Class (MonadIO)
24−import Data.Foldable (asum)
25−#ifdef BENCH_CHUNKED
26−import Streamly.Data.Array (Array, Unbox)
27−#endif
28−#ifdef BENCH_CHUNKED_GENERIC
29−import Streamly.Data.Array.Generic (Array)
30−import qualified Streamly.Internal.Data.Array.Generic as GenArr (chunksOf)
31−#endif
32−import Streamly.Internal.Data.Fold (Fold(..))
33−import Streamly.Data.StreamK (StreamK)
34−import Streamly.Internal.Data.Parser
35− (ParseError(..), Parser(..), Initial(..), Step(..))
36−import Streamly.Internal.Data.Stream (Stream)
37−import System.Random (randomRIO)
38−import Prelude hiding
39− (any, all, take, sequence, sequence_, sequenceA, takeWhile)
40−
41−import qualified Control.Applicative as AP
42−import qualified Data.Foldable as F
43−import qualified Data.Traversable as TR
44−import qualified Streamly.Data.Stream as Stream
45−import qualified Streamly.Data.Fold as FL
46−import qualified Streamly.Internal.Data.Fold as Fold
47−import qualified Streamly.Data.Parser as PRD
48−import qualified Streamly.Internal.Data.ParserK as PR
49−import qualified Streamly.Internal.Data.StreamK as StreamK
50−
51−import Test.Tasty.Bench
52−import Streamly.Benchmark.Common
53−
54−-------------------------------------------------------------------------------
55−-- CPP Helpers
56−-------------------------------------------------------------------------------
57−
58−#ifdef BENCH_CHUNKED
59−
60−#define PARSE_OP StreamK.parseChunks
61−#define FROM_PARSER adaptC
62−#define INPUT (Array a)
63−#define PARSE_ELEM (Array Int)
64−#define CONSTRAINT_IO (MonadIO m, Unbox a)
65−#define CONSTRAINT (Monad m, Unbox a)
66−#define MODULE_NAME "Data.ParserK.Chunked"
67−
68−#endif
69−
70−#ifdef BENCH_CHUNKED_GENERIC
71−
72−#define PARSE_OP StreamK.parseChunksGeneric
73−#define FROM_PARSER adaptCG
74−#define INPUT (Array a)
75−#define PARSE_ELEM (Array Int)
76−#define CONSTRAINT_IO (MonadIO m)
77−#define CONSTRAINT (Monad m)
78−#define MODULE_NAME "Data.ParserK.Chunked.Generic"
79−
80−#endif
81−
82−#ifdef BENCH_SINGULAR
83−
84−#define PARSE_OP StreamK.parse
85−#define FROM_PARSER adapt
86−#define INPUT a
87−#define PARSE_ELEM Int
88−#define CONSTRAINT_IO (MonadIO m)
89−#define CONSTRAINT (Monad m)
90−#define MODULE_NAME "Data.ParserK"
91−
92−#endif
93−
94−-------------------------------------------------------------------------------
95−-- Utilities
96−-------------------------------------------------------------------------------
97−
98−-- XXX these can be moved to the common module
99−
100−{-# INLINE sourceUnfoldrM #-}
101−sourceUnfoldrM :: Monad m => Int -> Int -> Stream m Int
102−sourceUnfoldrM value n = Stream.unfoldrM step n
103− where
104− step cnt =
105− if cnt > n + value
106− then return Nothing
107− else return (Just (cnt, cnt + 1))
108−
109−-- | Takes a fold method, and uses it with a default source.
110−{-# INLINE benchIOSink #-}
111−benchIOSink
112− :: NFData b
113− => Int -> String -> (StreamK IO PARSE_ELEM -> IO b) -> Benchmark
114−benchIOSink value name f =
115− bench name $ nfIO $ randomRIO (1,1)
116− >>= f
117− . StreamK.fromStream
118−#ifdef BENCH_CHUNKED
119− . Stream.chunksOf 4000
120−#endif
121−#ifdef BENCH_CHUNKED_GENERIC
122− . GenArr.chunksOf 4000
123−#endif
124− . sourceUnfoldrM value
125−
126−-------------------------------------------------------------------------------
127−-- Parsers
128−-------------------------------------------------------------------------------
129−
130−{-# INLINE one #-}
131−one :: MonadIO m =>
132− Int -> StreamK m PARSE_ELEM -> m (Either ParseError (Maybe Int))
133−one value = PARSE_OP p
134−
135− where
136−
137− p = do
138− m <- PR.FROM_PARSER (PRD.fromFold FL.one)
139− case m of
140− Just i -> if i >= value then pure m else p
141− Nothing -> pure Nothing
142−
143−{-# INLINE satisfy #-}
144−satisfy :: CONSTRAINT_IO => (a -> Bool) -> PR.ParserK INPUT m a
145−satisfy = PR.FROM_PARSER . PRD.satisfy
146−
147−{-# INLINE takeWhile #-}
148−takeWhile :: CONSTRAINT_IO => (a -> Bool) -> PR.ParserK INPUT m ()
149−takeWhile p = PR.FROM_PARSER $ PRD.takeWhile p FL.drain
150−
151−{-# INLINE takeWhileK #-}
152−takeWhileK :: MonadIO m =>
153− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
154−takeWhileK value = PARSE_OP (takeWhile (<= value))
155−
156−{-# INLINE splitAp2 #-}
157−splitAp2 :: MonadIO m
158− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ((), ()))
159−splitAp2 value =
160− PARSE_OP
161− ((,)
162− <$> takeWhile (<= (value `div` 2))
163− <*> takeWhile (<= value)
164− )
165−
166−{-# INLINE splitAp8 #-}
167−splitAp8 :: MonadIO m
168− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
169−splitAp8 value =
170− PARSE_OP
171− ( (\() () () () () () () () -> ())
172− <$> takeWhile (<= ( value `div` 8))
173− <*> takeWhile (<= ((value * 2) `div` 8))
174− <*> takeWhile (<= ((value * 3) `div` 8))
175− <*> takeWhile (<= ((value * 4) `div` 8))
176− <*> takeWhile (<= ((value * 5) `div` 8))
177− <*> takeWhile (<= ((value * 6) `div` 8))
178− <*> takeWhile (<= ((value * 7) `div` 8))
179− <*> takeWhile (<= value)
180− )
181−
182−{-# INLINE sequenceA #-}
183−sequenceA :: MonadIO m => Int -> StreamK m PARSE_ELEM -> m Int
184−sequenceA value xs = do
185− let parser = satisfy (> 0)
186− list = Prelude.replicate value parser
187− x <- PARSE_OP (TR.sequenceA list) xs
188− return $ Prelude.length x
189−
190−{-# INLINE sequenceA_ #-}
191−sequenceA_ :: MonadIO m =>
192− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
193−sequenceA_ value xs = do
194− let parser = satisfy (> 0)
195− list = Prelude.replicate value parser
196− PARSE_OP (F.sequenceA_ list) xs
197−
198−{-# INLINE sequence #-}
199−sequence :: MonadIO m => Int -> StreamK m PARSE_ELEM -> m Int
200−sequence value xs = do
201− let parser = satisfy (> 0)
202− list = Prelude.replicate value parser
203− x <- PARSE_OP (TR.sequence list) xs
204− return $ Prelude.length x
205−
206−{-# INLINE sequence_ #-}
207−sequence_ :: MonadIO m =>
208− Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
209−sequence_ value =
210− let parser = satisfy (> 0)
211− list = Prelude.replicate value parser
212− in PARSE_OP (F.sequence_ list)
213−
214−{-# INLINE takeWhileFailD #-}
215−takeWhileFailD :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
216−takeWhileFailD predicate (Fold fstep finitial _ ffinal) =
217− Parser step initial extract
218−
219− where
220−
221− initial = do
222− res <- finitial
223− return $ case res of
224− Fold.Partial s -> IPartial s
225− Fold.Done b -> IDone b
226−
227− step s a =
228− if predicate a
229− then do
230− fres <- fstep s a
231− return
232− $ case fres of
233− Fold.Partial s1 -> Partial 0 s1
234− Fold.Done b -> Done 0 b
235− else return $ Error "fail"
236−
237− extract s = fmap (Done 0) (ffinal s)
238−
239−{-# INLINE takeWhileFail #-}
240−takeWhileFail :: CONSTRAINT =>
241− (a -> Bool) -> Fold m a b -> PR.ParserK INPUT m b
242−takeWhileFail p f = PR.FROM_PARSER (takeWhileFailD p f)
243−
244−{-# INLINE alt2 #-}
245−alt2 :: MonadIO m
246− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
247−alt2 value =
248− PARSE_OP
249− ( takeWhileFail (<= (value `div` 2)) Fold.drain
250− <|> takeWhile (<= value)
251− )
252−
253−{-# INLINE alt8 #-}
254−alt8 :: MonadIO m
255− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
256−alt8 value =
257− PARSE_OP
258− ( takeWhileFail (<= ( value `div` 8)) Fold.drain
259− <|> takeWhileFail (<= ((value * 2) `div` 8)) Fold.drain
260− <|> takeWhileFail (<= ((value * 3) `div` 8)) Fold.drain
261− <|> takeWhileFail (<= ((value * 4) `div` 8)) Fold.drain
262− <|> takeWhileFail (<= ((value * 5) `div` 8)) Fold.drain
263− <|> takeWhileFail (<= ((value * 6) `div` 8)) Fold.drain
264− <|> takeWhileFail (<= ((value * 7) `div` 8)) Fold.drain
265− <|> takeWhile (<= value)
266− )
267−
268−{-# INLINE alt16 #-}
269−alt16 :: MonadIO m
270− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
271−alt16 value =
272− PARSE_OP
273− ( takeWhileFail (<= ( value `div` 16)) Fold.drain
274− <|> takeWhileFail (<= ((value * 2) `div` 16)) Fold.drain
275− <|> takeWhileFail (<= ((value * 3) `div` 16)) Fold.drain
276− <|> takeWhileFail (<= ((value * 4) `div` 16)) Fold.drain
277− <|> takeWhileFail (<= ((value * 5) `div` 16)) Fold.drain
278− <|> takeWhileFail (<= ((value * 6) `div` 16)) Fold.drain
279− <|> takeWhileFail (<= ((value * 7) `div` 16)) Fold.drain
280− <|> takeWhileFail (<= ((value * 8) `div` 16)) Fold.drain
281− <|> takeWhileFail (<= ((value * 9) `div` 16)) Fold.drain
282− <|> takeWhileFail (<= ((value * 10) `div` 16)) Fold.drain
283− <|> takeWhileFail (<= ((value * 11) `div` 16)) Fold.drain
284− <|> takeWhileFail (<= ((value * 12) `div` 16)) Fold.drain
285− <|> takeWhileFail (<= ((value * 13) `div` 16)) Fold.drain
286− <|> takeWhileFail (<= ((value * 14) `div` 16)) Fold.drain
287− <|> takeWhileFail (<= ((value * 15) `div` 16)) Fold.drain
288− <|> takeWhile (<= value)
289− )
290−
291−{-# INLINE manyAlt #-}
292−manyAlt :: MonadIO m => StreamK m PARSE_ELEM -> m Int
293−manyAlt xs = do
294− x <- PARSE_OP (AP.many (satisfy (> 0))) xs
295− return $ Prelude.length x
296−
297−{-# INLINE someAlt #-}
298−someAlt :: MonadIO m => StreamK m PARSE_ELEM -> m Int
299−someAlt xs = do
300− x <- PARSE_OP (AP.some (satisfy (> 0))) xs
301− return $ Prelude.length x
302−
303−{-# INLINE choice #-}
304−choice :: MonadIO m =>
305− Int -> StreamK m PARSE_ELEM -> m (Either ParseError Int)
306−choice value =
307− PARSE_OP (asum (replicate value (satisfy (< 0)))
308− AP.<|> satisfy (> 0))
309−
310−{-# INLINE monad2 #-}
311−monad2 :: MonadIO m
312− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
313−monad2 value =
314− PARSE_OP $ do
315− takeWhile (<= (value `div` 2))
316− takeWhile (<= value)
317−
318−{-# INLINE monad4 #-}
319−monad4 :: MonadIO m
320− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
321−monad4 value =
322− PARSE_OP $ do
323− takeWhile (<= ( value `div` 4))
324− takeWhile (<= ((value * 2) `div` 4))
325− takeWhile (<= ((value * 3) `div` 4))
326− takeWhile (<= value)
327−
328−{-# INLINE monad8 #-}
329−monad8 :: MonadIO m
330− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
331−monad8 value =
332− PARSE_OP $ do
333− takeWhile (<= ( value `div` 8))
334− takeWhile (<= ((value * 2) `div` 8))
335− takeWhile (<= ((value * 3) `div` 8))
336− takeWhile (<= ((value * 4) `div` 8))
337− takeWhile (<= ((value * 5) `div` 8))
338− takeWhile (<= ((value * 6) `div` 8))
339− takeWhile (<= ((value * 7) `div` 8))
340− takeWhile (<= value)
341−
342−{-# INLINE monad16 #-}
343−monad16 :: MonadIO m
344− => Int -> StreamK m PARSE_ELEM -> m (Either ParseError ())
345−monad16 value =
346− PARSE_OP $ do
347− takeWhile (<= ( value `div` 16))
348− takeWhile (<= ((value * 2) `div` 16))
349− takeWhile (<= ((value * 3) `div` 16))
350− takeWhile (<= ((value * 4) `div` 16))
351− takeWhile (<= ((value * 5) `div` 16))
352− takeWhile (<= ((value * 6) `div` 16))
353− takeWhile (<= ((value * 7) `div` 16))
354− takeWhile (<= ((value * 8) `div` 16))
355− takeWhile (<= ((value * 9) `div` 16))
356− takeWhile (<= ((value * 10) `div` 16))
357− takeWhile (<= ((value * 11) `div` 16))
358− takeWhile (<= ((value * 12) `div` 16))
359− takeWhile (<= ((value * 13) `div` 16))
360− takeWhile (<= ((value * 14) `div` 16))
361− takeWhile (<= ((value * 15) `div` 16))
362− takeWhile (<= value)
363−
364−-------------------------------------------------------------------------------
365−-- Benchmarks
366−-------------------------------------------------------------------------------
367−
368−moduleName :: String
369−moduleName = MODULE_NAME
370−
371−instance NFData ParseError where
372− {-# INLINE rnf #-}
373− rnf (ParseError x) = rnf x
374−
375−o_1_space_serial :: Int -> [Benchmark]
376−o_1_space_serial value =
377− [ benchIOSink value "drain" (Stream.fold Fold.drain . StreamK.toStream)
378− , benchIOSink value "takeWhile" $ takeWhileK value
379− , benchIOSink value "splitAp2" $ splitAp2 value
380− , benchIOSink value "splitAp8" $ splitAp8 value
381− , benchIOSink value "alt2" $ alt2 value
382− , benchIOSink value "alt8" $ alt8 value
383− , benchIOSink value "alt16" $ alt16 value
384− , benchIOSink value "monad2" $ monad2 value
385− , benchIOSink value "monad4" $ monad4 value
386− , benchIOSink value "monad8" $ monad8 value
387− , benchIOSink value "monad16" $ monad16 value
388− ]
389−
390−{-# INLINE sepBy1 #-}
391−sepBy1 :: MonadIO m => StreamK m PARSE_ELEM -> m Int
392−sepBy1 xs = do
393− x <- PARSE_OP (parser (satisfy odd) (satisfy even)) xs
394− return $ Prelude.length x
395−
396− where
397−
398− parser p sep = do
399− x <- p
400− fmap (x :) $ AP.many (sep >> p)
401−
402−-- O(n) heap beacuse of accumulation of the list in strict IO monad?
403−o_n_heap_serial :: Int -> [Benchmark]
404−o_n_heap_serial value =
405− [
406− -- accumulates the results in a list
407− -- XXX why should this take O(n) heap, it discards the results?
408− benchIOSink value "sequence_" $ sequence_ value
409− , benchIOSink value "sequenceA_" $ sequenceA_ value
410− , benchIOSink value "sequence" $ sequence value
411− , benchIOSink value "sequenceA" $ sequenceA value
412− , benchIOSink value "manyAlt" manyAlt
413− , benchIOSink value "sepBy1" sepBy1
414− , benchIOSink value "someAlt" someAlt
415− , benchIOSink value "choice" $ choice value
416− ]
417−
418−-- O(n) heap beacuse of accumulation of the list in strict IO monad?
419−o_1_space_recursive :: Int -> [Benchmark]
420−o_1_space_recursive value =
421− [ benchIOSink value "one (recursive)" $ one value
422− ]
423−
424−-------------------------------------------------------------------------------
425−-- Driver
426−-------------------------------------------------------------------------------
427−
428−main :: IO ()
429−main = runWithCLIOpts defaultStreamSize allBenchmarks
430−
431− where
432−
433− allBenchmarks value =
434− [ bgroup (o_1_space_prefix moduleName) (o_1_space_serial value)
435− , bgroup (o_1_space_prefix moduleName) (o_1_space_recursive value)
436− , bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
437− ]
benchmark/Streamly/Benchmark/Data/Ring/Unboxed.hs+4−3
@@ -12,11 +12,12 @@ module Main (main) where
1212
1313import Control.Monad (void)
1414import GHC.Ptr (Ptr(..))
15−import qualified Streamly.Internal.Data.Array as Array
16−import qualified Streamly.Internal.Data.Ring as Ring
15+
16+import qualified Streamly.Internal.Data.Array.Type as Array
17+import qualified Streamly.Internal.Data.Ring.Unboxed as Ring
1718import qualified Data.Foldable as P
1819
19−import Test.Tasty.Bench
20+import Gauge
2021import Streamly.Benchmark.Common
2122
2223import Prelude as P
benchmark/Streamly/Benchmark/Data/Serialize.hsdeleted−591
@@ -1,591 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−{- HLINT ignore -}
3−
4−#undef FUSION_CHECK
5−#ifdef FUSION_CHECK
6−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
7−#endif
8−
9−-- |
10−-- Module : Streamly.Benchmark.Data.Serialize
11−-- Copyright : (c) 2023 Composewell
12−-- License : BSD-3-Clause
13−-- Maintainer : streamly@composewell.com
14−
15−module Main (main) where
16−
17−-------------------------------------------------------------------------------
18−-- Imports
19−-------------------------------------------------------------------------------
20−
21−import Control.DeepSeq (NFData(..), deepseq)
22−import Control.Monad (when, void)
23−import GHC.Generics (Generic)
24−import System.Random (randomRIO)
25−#ifndef USE_UNBOX
26−import Test.QuickCheck (Arbitrary, arbitrary)
27−#endif
28−
29−import Streamly.Internal.Data.MutByteArray
30−import qualified Streamly.Internal.Data.MutByteArray as MBA
31−#ifdef USE_UNBOX
32−import Data.Proxy (Proxy(..))
33−#else
34−import Control.DeepSeq (force)
35−import Test.QuickCheck (oneof, generate)
36−#endif
37−
38−import Test.Tasty.Bench
39−import Streamly.Benchmark.Common
40−
41−#ifndef USE_UNBOX
42−import Streamly.Benchmark.Data.Serialize.RecCompatible
43−import Streamly.Benchmark.Data.Serialize.RecNonCompatible
44−#endif
45−
46−#ifdef USE_UNBOX
47−#define SERIALIZE_CLASS Unbox
48−#define DERIVE_CLASS(typ) $(deriveUnbox [d|instance Unbox typ|])
49−#define SERIALIZE_OP pokeAt
50−#define DESERIALIZE_OP peekAt
51−#else
52−#define SERIALIZE_CLASS Serialize
53−#define DERIVE_CLASS(typ) $(deriveSerialize [d|instance Serialize typ|])
54−#define SERIALIZE_OP serializeAt
55−#define DESERIALIZE_OP deserializeAt
56−#endif
57−
58−-------------------------------------------------------------------------------
59−-- Types
60−-------------------------------------------------------------------------------
61−
62−data Unit = Unit
63− deriving (Generic, Show, Eq)
64−
65−#ifndef USE_TH
66−instance SERIALIZE_CLASS Unit
67−#else
68−DERIVE_CLASS(Unit)
69−#endif
70−
71−instance NFData Unit where
72− {-# INLINE rnf #-}
73− rnf _ = ()
74−
75−data Sum2
76− = Sum21
77− | Sum22
78− deriving (Generic, Show, Eq)
79−
80−#ifndef USE_TH
81−instance SERIALIZE_CLASS Sum2
82−#else
83−DERIVE_CLASS(Sum2)
84−#endif
85−
86−instance NFData Sum2 where
87− {-# INLINE rnf #-}
88− rnf _ = ()
89−
90−data Sum25
91− = Sum251
92− | Sum252
93− | Sum253
94− | Sum254
95− | Sum255
96− | Sum256
97− | Sum257
98− | Sum258
99− | Sum259
100− | Sum2510
101− | Sum2511
102− | Sum2512
103− | Sum2513
104− | Sum2514
105− | Sum2515
106− | Sum2516
107− | Sum2517
108− | Sum2518
109− | Sum2519
110− | Sum2520
111− | Sum2521
112− | Sum2522
113− | Sum2523
114− | Sum2524
115− | Sum2525
116− deriving (Generic, Show, Eq)
117−
118−#ifndef USE_TH
119−instance SERIALIZE_CLASS Sum25
120−#else
121−DERIVE_CLASS(Sum25)
122−#endif
123−
124−instance NFData Sum25 where
125− {-# INLINE rnf #-}
126− rnf _ = ()
127−
128−data Product25
129− = Product25
130− Int
131− Int
132− Int
133− Int
134− Int
135− Int
136− Int
137− Int
138− Int
139− Int
140− Int
141− Int
142− Int
143− Int
144− Int
145− Int
146− Int
147− Int
148− Int
149− Int
150− Int
151− Int
152− Int
153− Int
154− Int
155− deriving (Generic, Show)
156−
157−#ifndef USE_TH
158−instance SERIALIZE_CLASS Product25
159−#else
160−DERIVE_CLASS(Product25)
161−#endif
162−
163−instance NFData Product25 where
164− {-# INLINE rnf #-}
165− rnf
166− (Product25 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20 a21 a22 a23 a24 a25)
167− = a1
168− `seq` a2
169− `seq` a3
170− `seq` a4
171− `seq` a5
172− `seq` a6
173− `seq` a7
174− `seq` a8
175− `seq` a9
176− `seq` a10
177− `seq` a11
178− `seq` a12
179− `seq` a13
180− `seq` a14
181− `seq` a15
182− `seq` a16
183− `seq` a17
184− `seq` a18
185− `seq` a19
186− `seq` a20
187− `seq` a21
188− `seq` a22
189− `seq` a23
190− `seq` a24
191− `seq` a25
192− `seq` ()
193−
194−-- XXX derived Eq instance is not inlined
195−instance Eq Product25 where
196− {-# INLINE (==) #-}
197− (==)
198− (Product25 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a20 a21 a22 a23 a24 a25)
199− (Product25 b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 b17 b18 b19 b20 b21 b22 b23 b24 b25)
200− = a1 == b1
201− && a2 == b2
202− && a3 == b3
203− && a4 == b4
204− && a5 == b5
205− && a6 == b6
206− && a7 == b7
207− && a8 == b8
208− && a9 == b9
209− && a10 == b10
210− && a11 == b11
211− && a12 == b12
212− && a13 == b13
213− && a14 == b14
214− && a15 == b15
215− && a16 == b16
216− && a17 == b17
217− && a18 == b18
218− && a19 == b19
219− && a20 == b20
220− && a21 == b21
221− && a22 == b22
222− && a23 == b23
223− && a24 == b24
224− && a25 == b25
225−
226−-------------------------------------------------------------------------------
227−-- Simple non-recursive ADT
228−-------------------------------------------------------------------------------
229−
230−{-
231−data CustomDT1
232− = CDT1C1
233− | CDT1C2 Int
234− | CDT1C3 Int Int
235− deriving (Generic, Show, Eq)
236−
237−#ifndef USE_TH
238−instance SERIALIZE_CLASS CustomDT1
239−#else
240−DERIVE_CLASS(CustomDT1)
241−#endif
242−
243−instance NFData CustomDT1 where
244− {-# INLINE rnf #-}
245− rnf CDT1C1 = ()
246− rnf (CDT1C2 i) = i `seq` ()
247− rnf (CDT1C3 i1 i2) = i1 `seq` i2 `seq` ()
248−-}
249−
250−-------------------------------------------------------------------------------
251−-- Recursive ADT
252−-------------------------------------------------------------------------------
253−
254−#ifndef USE_UNBOX
255−data BinTree a
256− = Tree (BinTree a) (BinTree a)
257− | Leaf a
258− deriving (Show, Read, Eq, Generic)
259−
260−#ifndef USE_TH
261−instance Serialize (BinTree a)
262−#else
263−$(deriveSerialize [d|instance Serialize a => Serialize (BinTree a)|])
264−#endif
265−
266−instance NFData a => NFData (BinTree a) where
267− {-# INLINE rnf #-}
268− rnf (Leaf a) = rnf a `seq` ()
269− rnf (Tree l r) = rnf l `seq` rnf r
270−
271−-- XXX This may not terminate, or could become really large.
272−instance Arbitrary a => Arbitrary (BinTree a) where
273− arbitrary = oneof [Leaf <$> arbitrary, Tree <$> arbitrary <*> arbitrary]
274−
275−mkBinTree :: (Arbitrary a) => Int -> IO (BinTree a)
276−mkBinTree = go (generate arbitrary)
277−
278− where
279−
280− go r 0 = Leaf <$> r
281− go r n = Tree <$> go r (n - 1) <*> go r (n - 1)
282−
283−#endif
284−
285−-------------------------------------------------------------------------------
286−-- Size helpers
287−-------------------------------------------------------------------------------
288−
289−{-# INLINE getSize #-}
290−getSize :: forall a. SERIALIZE_CLASS a => a -> Int
291−#ifdef USE_UNBOX
292−getSize _ = sizeOf (Proxy :: Proxy a)
293−#else
294−getSize = addSizeTo 0
295−#endif
296−
297−-------------------------------------------------------------------------------
298−-- Common helpers
299−-------------------------------------------------------------------------------
300−
301−{- HLINT ignore "Eta reduce" -}
302−-- Parts of "f" that are dependent on val will not be optimized out.
303−{-# INLINE loop #-}
304−loop :: Int -> (a -> IO b) -> a -> IO ()
305−loop count f val = go count val
306− where
307−
308− go n x = when (n > 0) $ f x >> go (n-1) x
309−
310−-- The first arg of "f" is the environment which is not threaded around in the
311−-- loop.
312−{-# INLINE loopWith #-}
313−loopWith :: Int -> (e -> a -> IO b) -> e -> a -> IO ()
314−loopWith count f e val = go count val
315− where
316−
317− go n x = when (n > 0) $ f e x >> go (n-1) x
318−
319−benchSink :: NFData b => String -> Int -> (Int -> IO b) -> Benchmark
320−benchSink name times f = bench name (nfIO (randomRIO (times, times) >>= f))
321−
322−-------------------------------------------------------------------------------
323−-- Serialization Helpers
324−-------------------------------------------------------------------------------
325−
326−{-
327−{-# INLINE pokeWithSize #-}
328−pokeWithSize :: SERIALIZE_CLASS a => MutByteArray -> a -> IO ()
329−pokeWithSize arr val = do
330− let n = getSize val
331− n `seq` SERIALIZE_OP 0 arr val >> return ()
332−
333−{-# INLINE pokeTimesWithSize #-}
334−pokeTimesWithSize :: SERIALIZE_CLASS a => a -> Int -> IO ()
335−pokeTimesWithSize val times = do
336− let n = getSize val
337− arr <- MBA.new n
338− loopWith times pokeWithSize arr val
339−-}
340−
341−{-# INLINE poke #-}
342−poke :: SERIALIZE_CLASS a => MutByteArray -> a -> IO ()
343−poke arr val = void (SERIALIZE_OP 0 arr val)
344−
345−{-# INLINE pokeTimes #-}
346−pokeTimes :: SERIALIZE_CLASS a => a -> Int -> IO ()
347−pokeTimes val times = do
348− let n = getSize val
349− arr <- MBA.new n
350− loopWith times poke arr val
351−
352−{-# INLINE encode #-}
353−encode :: SERIALIZE_CLASS a => a -> IO ()
354−encode val = do
355− let n = getSize val
356− arr <- MBA.new n
357− void (SERIALIZE_OP 0 arr val)
358−
359−{-# INLINE encodeTimes #-}
360−encodeTimes :: SERIALIZE_CLASS a => a -> Int -> IO ()
361−encodeTimes val times = loop times encode val
362−
363−{-# INLINE peek #-}
364−peek :: forall a. (NFData a, SERIALIZE_CLASS a) =>
365− (a, Int) -> MutByteArray -> IO ()
366−#ifdef USE_UNBOX
367−peek (_val, _) arr = do
368− (val1 :: a) <- DESERIALIZE_OP 0 arr
369−#else
370−peek (_val, n) arr = do
371− (_, val1 :: a) <- DESERIALIZE_OP 0 arr n
372−#endif
373− -- If the datatype is not deeply strict or deepseq is not used then use
374− -- Equality.
375− -- Ensure that we are actually constructing the type and using it. This
376− -- is important, otherwise the structure is created and discarded, the
377− -- cost of creation of the structure is not accounted. Otherwise we may
378− -- just read the values and discard them. The comparison adds to the
379− -- cost though. We could use deepseq but then we need to write
380− -- instances of NFData and ensure that they are correct and perform
381− -- well. Equality check also ensures correctness.
382− {-
383− if (val1 /= val)
384− then error "peek: no match"
385− else return ()
386− -}
387− val1 `deepseq` return ()
388−
389−{-# INLINE peekTimes #-}
390−peekTimes :: (NFData a, SERIALIZE_CLASS a) => Int -> a -> Int -> IO ()
391−peekTimes n val times = do
392− arr <- MBA.new n
393− _ <- SERIALIZE_OP 0 arr val
394− loopWith times peek (val, n) arr
395−
396−{-# INLINE trip #-}
397−trip :: forall a. (NFData a, SERIALIZE_CLASS a) => a -> IO ()
398−trip val = do
399− let n = getSize val
400− arr <- MBA.new n
401− _ <- SERIALIZE_OP 0 arr val
402−#ifdef USE_UNBOX
403− (val1 :: a) <- DESERIALIZE_OP 0 arr
404−#else
405− (_, val1 :: a) <- DESERIALIZE_OP 0 arr n
406−#endif
407− -- Do not remove this or use deepseq, see the comments in peek.
408− {-
409− if (val1 /= val)
410− then error "roundtrip: no match"
411− else return ()
412− -}
413− -- Note: deepseq is not needed if the datatype is strict
414− val1 `deepseq` return ()
415−
416−{-# INLINE roundtrip #-}
417−roundtrip :: (NFData a, SERIALIZE_CLASS a) => a -> Int -> IO ()
418−roundtrip val times = loop times trip val
419−
420−-------------------------------------------------------------------------------
421−-- Benchmarks
422−-------------------------------------------------------------------------------
423−
424−{-# INLINE benchConst #-}
425−benchConst ::
426− String
427− -> (forall a. (NFData a, SERIALIZE_CLASS a) =>
428− Int -> a -> Int -> IO ())
429− -> Int
430− -> Benchmark
431−benchConst gname f times =
432− bgroup gname
433− [ let !n = getSize Unit
434− in benchSink "Unit" times (f n Unit)
435− {-
436− , let !n = getSize CDT1C1
437− in benchSink "C1" times (f n CDT1C1)
438− , let val = CDT1C2 5
439− !n = getSize val
440− in benchSink "C2" (times `div` 2) (f n val)
441− , let val = CDT1C3 5 2
442− !n = getSize val
443− in benchSink "C3" (times `div` 3) (f n val)
444− -}
445− , let !n = getSize Sum21
446− in benchSink "Sum2" times (f n Sum21)
447− , let !n = getSize Sum2525
448− in benchSink "Sum25" times (f n Sum2525)
449− , let val = Product25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
450− !n = getSize val
451− in benchSink "Product25" (times `div` 26) (f n val)
452− ]
453−
454−#ifndef USE_UNBOX
455−{-# INLINE benchVar #-}
456−benchVar ::
457− String
458− -> (forall a. (NFData a, SERIALIZE_CLASS a) =>
459− Int -> a -> Int -> IO ())
460− -> BinTree Int
461− -> [Int]
462− -> Int
463− -> Benchmark
464−benchVar gname f tInt lInt times =
465− bgroup gname
466− [ let !n = getSize tInt
467− in benchSink "bintree-int" times (f n tInt)
468− , let !n = getSize lInt
469− in benchSink "list-int" times (f n lInt)
470− ]
471−
472−{-
473−{-# INLINE benchGeneric #-}
474−benchGeneric
475− :: (NFData b, Serialize b)
476− => String
477− -> String
478− -> (forall a. (NFData a, Serialize a) =>
479− Int -> a -> Int -> IO ())
480− -> b
481− -> Int
482− -> Benchmark
483−benchGeneric gname valId f val times =
484− bgroup gname
485− [ let !n = getSize val
486− in benchSink valId times (f n val)
487− ]
488−-}
489−
490−{-# INLINE benchLargeRec #-}
491−benchLargeRec ::
492− String
493− -> (forall a. (NFData a, Serialize a) =>
494− Int -> a -> Int -> IO ())
495− -> RecNonCompatible
496− -> RecCompatible
497− -> Int
498− -> Benchmark
499−benchLargeRec gname f recL recR times =
500− bgroup gname
501− [ let !n = getSize recL
502− in benchSink "RecNonCompatible" times (f n recL)
503− , let !n = getSize recR
504− in benchSink "RecCompatible" times (f n recR)
505− ]
506−#endif
507−
508−-- Times is scaled by the number of constructors to normalize
509−#ifdef USE_UNBOX
510−allBenchmarks :: Int -> [Benchmark]
511−allBenchmarks times =
512−#else
513−allBenchmarks ::
514− BinTree Int -> [Int] -> RecNonCompatible -> RecCompatible -> Int -> [Benchmark]
515−allBenchmarks tInt lInt recL recR times =
516−#endif
517− [ bgroup "sizeOf"
518− [
519−#ifndef USE_UNBOX
520− bench "bintree-int" $ nf getSize tInt
521− , bench "list-int" $ nf getSize lInt
522−#endif
523− ]
524− , benchConst "poke" (const pokeTimes) times
525− -- , benchConst "pokeWithSize" (const pokeTimesWithSize) times
526− , benchConst "encode" (const encodeTimes) times
527− , benchConst "peek" peekTimes times
528− , benchConst "roundtrip" (const roundtrip) times
529−#ifndef USE_UNBOX
530− , benchVar "poke" (const pokeTimes) tInt lInt 1
531− -- , benchVar "pokeWithSize" (const pokeTimesWithSize) tInt lInt 1
532− , benchVar "encode" (const encodeTimes) tInt lInt 1
533− , benchVar "peek" peekTimes tInt lInt 1
534− , benchVar "roundtrip" (const roundtrip) tInt lInt 1
535− , benchLargeRec "poke" (const pokeTimes) recL recR times
536− , benchLargeRec "encode" (const encodeTimes) recL recR times
537− , benchLargeRec "peek" peekTimes recL recR times
538− , benchLargeRec "roundtrip" (const roundtrip) recL recR times
539−#endif
540− ]
541−
542−-------------------------------------------------------------------------------
543−-- Driver
544−-------------------------------------------------------------------------------
545−
546−main :: IO ()
547−main = do
548−#ifndef USE_UNBOX
549− -- Approximately 100000 constructors
550− -- Assuming Leaf nodes constitute two constructors (Leaf, Int) and internal
551− -- nodes 1 level = log_2 (100001/3) + 1 = 16
552− !(tInt :: BinTree Int) <- force <$> mkBinTree 16
553−
554− let !recL = valRecNonCompatible
555− let !recR = valRecCompatible
556−
557− -- Approximately 100000 constructors, assuming two constructors (Cons, Int)
558− -- per element.
559− let lInt = [1..50000 :: Int]
560− let !len = length lInt -- evaluate the list
561−#endif
562−#ifndef FUSION_CHECK
563−#ifdef USE_UNBOX
564− runWithCLIOpts defaultStreamSize allBenchmarks
565−#else
566− len
567− `seq` runWithCLIOpts
568− defaultStreamSize
569− (allBenchmarks tInt lInt recL recR)
570−#endif
571−#else
572− -- Enable FUSION_CHECK macro at the beginning of the file
573− -- Enable one benchmark below, and run the benchmark
574− -- Check the .dump-simpl output
575− let value = 100000
576− -- print $ getSize (CDT1C3 4 2)
577− -- print $ getSize tInt
578− -- print $ getSize lInt
579−
580− -- pokeTimes tInt 1
581−
582− -- peekTimes ((CDT1C2 (5 :: Int)) :: CustomDT1) value
583− -- peekTimes (Sum2525) value
584− -- peekTimes (Product25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25) value
585− -- peekTimes tInt 1
586− let !n = getSize lInt
587− peekTimes n lInt 1
588−
589− -- roundtrip ((CDT1C2 (5 :: Int)) :: CustomDT1) value
590− return ()
591−#endif
benchmark/Streamly/Benchmark/Data/Serialize/RecCompatible.hsdeleted−24
@@ -1,24 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.RecCompatible
4− ( RecCompatible(..)
5− , valRecCompatible
6− ) where
7−
8−--------------------------------------------------------------------------------
9−-- Imports
10−--------------------------------------------------------------------------------
11−
12−import Streamly.Internal.Data.MutByteArray (Serialize)
13−import Streamly.Benchmark.Data.Serialize.TH (genLargeRecord)
14−
15−import qualified Streamly.Internal.Data.MutByteArray as Serialize
16−
17−--------------------------------------------------------------------------------
18−-- Code
19−--------------------------------------------------------------------------------
20−
21−$(genLargeRecord "RecCompatible" 50)
22−$(Serialize.deriveSerializeWith
23− (Serialize.encodeRecordFields True)
24− [d|instance Serialize RecCompatible|])
benchmark/Streamly/Benchmark/Data/Serialize/RecNonCompatible.hsdeleted−21
@@ -1,21 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.RecNonCompatible
4− ( RecNonCompatible(..)
5− , valRecNonCompatible
6− ) where
7−
8−--------------------------------------------------------------------------------
9−-- Imports
10−--------------------------------------------------------------------------------
11−
12−import Streamly.Benchmark.Data.Serialize.TH (genLargeRecord)
13−
14−import qualified Streamly.Internal.Data.MutByteArray as Serialize
15−
16−--------------------------------------------------------------------------------
17−-- Code
18−--------------------------------------------------------------------------------
19−
20−$(genLargeRecord "RecNonCompatible" 50)
21−$(Serialize.deriveSerialize [d|instance Serialize.Serialize RecNonCompatible|])
benchmark/Streamly/Benchmark/Data/Serialize/TH.hsdeleted−76
@@ -1,76 +0,0 @@
1−{-# LANGUAGE TemplateHaskell #-}
2−
3−module Streamly.Benchmark.Data.Serialize.TH (genLargeRecord) where
4−
5−--------------------------------------------------------------------------------
6−-- Imports
7−--------------------------------------------------------------------------------
8−
9−import Language.Haskell.TH
10−
11−import Streamly.Internal.Data.MutByteArray (makeI)
12−import Control.DeepSeq (NFData(..))
13−
14−--------------------------------------------------------------------------------
15−-- Large Record
16−--------------------------------------------------------------------------------
17−
18−genLargeRecord :: String -> Int -> Q [Dec]
19−genLargeRecord tyName numFields =
20− sequence
21− [ dataD
22− (pure [])
23− (mkName tyName)
24− []
25− Nothing
26− [mkCon tyName]
27− [derivClause Nothing [conT ''Eq, conT ''Show]]
28− , mkValueSigDec
29− , mkValueDec
30− , nfDataInstance tyName
31− ]
32−
33− where
34−
35− fieldTypeChoices = [conT ''()]
36− chooseCycle i xs = xs !! (i `mod` length xs)
37− nfDataInstance nm =
38− instanceD
39− (pure [])
40− (appT (conT ''NFData) (conT (mkName nm)))
41− [ funD
42− 'rnf
43− [ clause
44− [ conP
45− (mkName nm)
46− (varP . makeI <$> [0 .. (numFields - 1)])
47− ]
48− (normalB
49− (foldl
50− (\b a -> [|rnf $(b) `seq` rnf $(a)|])
51− [|()|]
52− (varE . makeI <$> [0 .. (numFields - 1)])))
53− []
54− ]
55− ]
56− valueName = mkName $ "val" ++ tyName
57− mkValueSigDec = sigD valueName [t|$(conT (mkName tyName))|]
58− mkValueDec =
59− funD
60− valueName
61− [ clause
62− []
63− (normalB
64− (foldl
65− (\b a -> [|$(b) $(a)|])
66− (conE (mkName tyName))
67− (conE '() <$ [0 .. (numFields - 1)])))
68− []
69− ]
70− mkCon nm = recC (mkName nm) (mkField <$> [0 .. (numFields - 1)])
71− mkField i =
72− varBangType
73− (mkName ("field" ++ show i))
74− (bangType
75− (bang noSourceUnpackedness noSourceStrictness)
76− (chooseCycle i fieldTypeChoices))
benchmark/Streamly/Benchmark/Data/Stream.hs−49
@@ -9,11 +9,6 @@
99{-# LANGUAGE ScopedTypeVariables #-}
1010{-# LANGUAGE RankNTypes #-}
1111
12−#undef FUSION_CHECK
13−#ifdef FUSION_CHECK
14−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
15−#endif
16−
1712module Main (main) where
1813
1914import Streamly.Benchmark.Common.Handle (mkHandleBenchEnv)
@@ -33,15 +28,6 @@ import qualified Stream.Transform as Transformation
3328
3429import Streamly.Benchmark.Common
3530
36−#ifdef FUSION_CHECK
37−import Data.Function ((&))
38−import Streamly.Data.Stream (Stream)
39−import qualified Streamly.Internal.Data.Stream as Stream
40−import qualified Streamly.Internal.Data.Fold as Fold
41−import qualified Data.Foldable as Foldable
42−import Stream.Common
43−#endif
44−
4531moduleName :: String
4632#ifdef USE_STREAMK
4733moduleName = "Data.Stream.StreamDK"
@@ -58,7 +44,6 @@ moduleName = "Data.Stream"
5844--
5945main :: IO ()
6046main = do
61−#ifndef FUSION_CHECK
6247 env <- mkHandleBenchEnv
6348 runWithCLIOpts defaultStreamSize (allBenchmarks env)
6449
@@ -78,37 +63,3 @@ main = do
7863 , Lift.benchmarks moduleName size
7964 , NestedStream.benchmarks moduleName size
8065 ]
81−#else
82− -- Enable FUSION_CHECK macro at the beginning of the file
83− -- Enable one benchmark below, and run the benchmark
84− -- Check the .dump-simpl output
85− let value = 100000
86− let input = sourceUnfoldrM value 1
87−
88− -- Stream.foldr (:) [] input >>= print . Prelude.length
89− -- Stream.fold (Fold.foldr' (:) []) input >>= print . Prelude.length
90−
91− -- Stream.toList input >>= print . Prelude.length
92− -- Stream.fold Fold.toList input >>= print . Prelude.length
93−
94− -- We can automatically assert that these generate identical core
95−
96− -- Stream.drain input >>= print
97− -- Stream.fold Fold.drain input >>= print
98−
99− -- Stream.last input >>= print
100− -- Stream.fold Fold.latest input >>= print
101−
102− -- Stream.length input >>= print
103− -- Stream.fold Fold.length input >>= print
104−
105− -- Stream.maximum input >>= print
106− -- Stream.fold Fold.maximum input >>= print
107−
108− -- Stream.head input >>= print
109− -- Stream.fold Fold.one input >>= print
110−
111− -- composeN 4 (Stream.scanl' (+) 0) input
112− composeN 4 (Stream.scan (Fold.foldl' (+) 0)) input
113− return ()
114−#endif
benchmark/Streamly/Benchmark/Data/Stream/Common.hs+11−13
@@ -71,9 +71,7 @@ module Stream.Common
7171 )
7272where
7373
74−#if !MIN_VERSION_base(4,18,0)
7574import Control.Applicative (liftA2)
76−#endif
7775import Control.DeepSeq (NFData)
7876import Control.Exception (try)
7977import GHC.Exception (ErrorCall)
@@ -92,20 +90,20 @@ import Streamly.Benchmark.Prelude
9290 , concatStreamsWith
9391 )
9492#else
95−import Streamly.Internal.Data.Stream (Stream)
96−import qualified Streamly.Internal.Data.Stream as D
93+import Streamly.Internal.Data.Stream.StreamD (Stream)
94+import qualified Streamly.Internal.Data.Stream.StreamD as D
9795#ifdef USE_STREAMK
98−import Streamly.Internal.Data.StreamK (StreamK)
99−import qualified Streamly.Internal.Data.Stream as Stream
100−import qualified Streamly.Internal.Data.StreamK as StreamK
96+import Streamly.Internal.Data.Stream.StreamK (StreamK)
97+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
98+import qualified Streamly.Internal.Data.Stream.StreamK as StreamK
10199#else
102−import qualified Streamly.Internal.Data.Stream as Stream
103−import qualified Streamly.Internal.Data.Stream as StreamK
100+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
101+import qualified Streamly.Internal.Data.Stream.StreamD as StreamK
104102#endif
105103#endif
106104
107−import Test.Tasty.Bench
108−import Prelude hiding (Foldable(..), mapM, replicate)
105+import Gauge
106+import Prelude hiding (mapM, replicate)
109107
110108#ifdef USE_STREAMK
111109toStream :: Applicative m => StreamK m a -> Stream m a
@@ -130,10 +128,10 @@ unCross = id
130128type MonadAsync = Monad
131129
132130#ifdef USE_STREAMK
133−mkCross :: StreamK m a -> StreamK.CrossStreamK m a
131+mkCross :: StreamK.Stream m a -> StreamK.CrossStreamK m a
134132mkCross = StreamK.mkCross
135133
136−unCross :: StreamK.CrossStreamK m a -> StreamK m a
134+unCross :: StreamK.CrossStreamK m a -> StreamK.Stream m a
137135unCross = StreamK.unCross
138136#else
139137mkCross :: Stream m a -> Stream.CrossStream m a
benchmark/Streamly/Benchmark/Data/Stream/Concurrent.hs+1−1
@@ -17,4 +17,4 @@ moduleName = "Data.Stream.Concurrent"
1717-------------------------------------------------------------------------------
1818
1919main :: IO ()
20−main = runWithCLIOpts defaultStreamSize (allBenchmarks moduleName False id)
20+main = runWithCLIOpts defaultStreamSize (allBenchmarks moduleName id)
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentCommon.hs+8−9
@@ -14,14 +14,14 @@ where
1414import Stream.Common
1515 (composeN, benchIO, benchIOSink, benchIOSrc, sourceUnfoldrM)
1616import Streamly.Data.Stream (Stream)
17−import Streamly.Internal.Data.Stream.Prelude (MonadAsync, Config)
17+import Streamly.Internal.Data.Stream.Concurrent (MonadAsync, Config)
1818
1919import qualified Data.List as List
2020import qualified Streamly.Data.Fold as Fold
2121import qualified Streamly.Data.Stream as Stream
22−import qualified Streamly.Internal.Data.Stream.Prelude as Async
22+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
2323
24−import Test.Tasty.Bench
24+import Gauge
2525import Prelude hiding (mapM)
2626import Streamly.Benchmark.Common
2727
@@ -193,16 +193,15 @@ o_1_space_outerProduct value f =
193193-- Benchmark sets
194194-------------------------------------------------------------------------------
195195
196−allBenchmarks :: String -> Bool -> (Config -> Config) -> Int -> [Benchmark]
197−allBenchmarks moduleName wide modifier value =
196+allBenchmarks :: String -> (Config -> Config) -> Int -> [Benchmark]
197+allBenchmarks moduleName modifier value =
198198 [ bgroup (o_1_space_prefix moduleName) $ concat
199199 [ o_1_space_mapping value modifier
200200 , o_1_space_concatFoldable value modifier
201201 , o_1_space_concatMap value modifier
202+ , o_1_space_outerProduct value modifier
202203 , o_1_space_joining value modifier
203− ] ++ if wide then [] else o_1_space_outerProduct value modifier
204− , bgroup (o_n_heap_prefix moduleName) $ concat
205− [ if wide then o_1_space_outerProduct value modifier else []
206− , o_n_heap_buffering value modifier
207204 ]
205+ , bgroup (o_n_heap_prefix moduleName)
206+ (o_n_heap_buffering value modifier)
208207 ]
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentEager.hs+2−2
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentEager"
@@ -22,4 +22,4 @@ main :: IO ()
2222main =
2323 runWithCLIOpts
2424 defaultStreamSize
25− (allBenchmarks moduleName True (Async.eager True))
25+ (allBenchmarks moduleName (Async.eager True))
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentInterleaved.hs+2−2
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentInterleaved"
@@ -22,4 +22,4 @@ main :: IO ()
2222main =
2323 runWithCLIOpts
2424 defaultStreamSize
25− (allBenchmarks moduleName True (Async.interleaved True))
25+ (allBenchmarks moduleName (Async.interleaved True))
benchmark/Streamly/Benchmark/Data/Stream/ConcurrentOrdered.hs+2−2
@@ -9,7 +9,7 @@
99import Stream.ConcurrentCommon (allBenchmarks)
1010import Streamly.Benchmark.Common (runWithCLIOpts, defaultStreamSize)
1111
12−import qualified Streamly.Internal.Data.Stream.Prelude as Async
12+import qualified Streamly.Internal.Data.Stream.Concurrent as Async
1313
1414moduleName :: String
1515moduleName = "Data.Stream.ConcurrentOrdered"
@@ -22,4 +22,4 @@ main :: IO ()
2222main =
2323 runWithCLIOpts
2424 defaultStreamSize
25− (allBenchmarks moduleName False (Async.ordered True))
25+ (allBenchmarks moduleName (Async.ordered True))
benchmark/Streamly/Benchmark/Data/Stream/Eliminate.hs+7−9
@@ -37,7 +37,7 @@ import qualified Streamly.Internal.Data.Fold as Fold
3737#ifdef INSPECTION
3838import GHC.Types (SPEC(..))
3939import Test.Inspection
40−import qualified Streamly.Internal.Data.Stream as D
40+import qualified Streamly.Internal.Data.Stream.StreamD as D
4141#endif
4242
4343#ifdef USE_PRELUDE
@@ -47,17 +47,17 @@ import qualified Streamly.Internal.Data.Stream.IsStream as S
4747import qualified Streamly.Internal.Data.Stream.IsStream as StreamK
4848#else
4949import Stream.Common
50−import Streamly.Internal.Data.Stream (Stream)
51−import qualified Streamly.Internal.Data.Stream as S
50+import Streamly.Internal.Data.Stream.StreamD (Stream)
51+import qualified Streamly.Internal.Data.Stream.StreamD as S
5252#ifdef USE_STREAMK
53−import Streamly.Internal.Data.StreamK (StreamK)
54−import qualified Streamly.Data.StreamK as StreamK
53+import Streamly.Internal.Data.Stream.StreamK (StreamK)
54+import qualified Streamly.Data.Stream.StreamK as StreamK
5555#else
56−import qualified Streamly.Internal.Data.Stream as StreamK
56+import qualified Streamly.Internal.Data.Stream.StreamD as StreamK
5757#endif
5858#endif
5959
60−import Test.Tasty.Bench
60+import Gauge
6161import Streamly.Benchmark.Common
6262import Prelude hiding (length, sum, or, and, any, all, notElem, elem, (!!),
6363 lookup, repeat, minimum, maximum, product, last, mapM_, init)
@@ -132,14 +132,12 @@ foldableAny :: Int -> Int -> Bool
132132foldableAny value n =
133133 Prelude.any (> (value + 1)) (sourceUnfoldr value n :: Stream Identity Int)
134134
135−{- HLINT ignore "Use all"-}
136135{-# INLINE foldableAnd #-}
137136foldableAnd :: Int -> Int -> Bool
138137foldableAnd value n =
139138 Prelude.and $ fmap
140139 (<= (value + 1)) (sourceUnfoldr value n :: Stream Identity Int)
141140
142−{- HLINT ignore "Use any"-}
143141{-# INLINE foldableOr #-}
144142foldableOr :: Int -> Int -> Bool
145143foldableOr value n =
benchmark/Streamly/Benchmark/Data/Stream/Exceptions.hs+6−16
@@ -36,25 +36,24 @@ import System.IO (Handle, hClose, hPutChar)
3636import qualified Streamly.FileSystem.Handle as FH
3737import qualified Streamly.Internal.FileSystem.Handle as IFH
3838import qualified Streamly.Internal.Data.Unfold as IUF
39−import qualified Streamly.Internal.Data.Unfold.Prelude as IUF
39+import qualified Streamly.Internal.Data.Unfold.Exception as IUF
4040
4141#ifdef USE_PRELUDE
4242import qualified Streamly.Internal.Data.Stream.IsStream as Stream
4343#else
44−import qualified Streamly.Internal.Data.Stream as Stream
45−import qualified Streamly.Internal.Data.Stream.Prelude as Stream
44+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
45+import qualified Streamly.Internal.Data.Stream.Exception.Lifted as Stream
4646#endif
4747
48−import Test.Tasty.Bench hiding (env)
48+import Gauge hiding (env)
4949import Prelude hiding (last, length)
5050import Streamly.Benchmark.Common
5151import Streamly.Benchmark.Common.Handle
5252
5353#ifdef INSPECTION
54−import Control.Monad.Catch (MonadCatch)
5554import Test.Inspection
5655
57−import qualified Streamly.Internal.Data.Stream as D
56+import qualified Streamly.Internal.Data.Stream.StreamD as D
5857#endif
5958
6059#ifdef USE_PRELUDE
@@ -178,8 +177,7 @@ inspect $ hasNoTypeClasses 'readWriteOnExceptionStream
178177-- | Send the file contents to /dev/null with exception handling
179178readWriteHandleExceptionStream :: Handle -> Handle -> IO ()
180179readWriteHandleExceptionStream inh devNull =
181− let handler (_e :: SomeException) =
182− return $ Stream.fromEffect (hClose inh >> return 10)
180+ let handler (_e :: SomeException) = Stream.fromEffect (hClose inh >> return 10)
183181 readEx = Stream.handle handler (Stream.unfold FH.reader inh)
184182 in Stream.fold (FH.write devNull) readEx
185183
@@ -287,12 +285,8 @@ readChunksOnException inh devNull =
287285 in IUF.fold (IFH.writeChunks devNull) readEx inh
288286
289287#ifdef INSPECTION
290−#if __GLASGOW_HASKELL__ >= 906
291−inspect $ hasNoTypeClassesExcept 'readChunksOnException [''MonadCatch]
292−#else
293288inspect $ hasNoTypeClasses 'readChunksOnException
294289#endif
295−#endif
296290
297291-- | Send the file contents to /dev/null with exception handling
298292readChunksBracket_ :: Handle -> Handle -> IO ()
@@ -301,12 +295,8 @@ readChunksBracket_ inh devNull =
301295 in IUF.fold (IFH.writeChunks devNull) readEx inh
302296
303297#ifdef INSPECTION
304−#if __GLASGOW_HASKELL__ >= 906
305−inspect $ hasNoTypeClassesExcept 'readChunksBracket_ [''MonadCatch]
306−#else
307298inspect $ hasNoTypeClasses 'readChunksBracket_
308299#endif
309−#endif
310300
311301readChunksBracket :: Handle -> Handle -> IO ()
312302readChunksBracket inh devNull =
benchmark/Streamly/Benchmark/Data/Stream/Expand.hs+7−15
@@ -29,7 +29,7 @@ module Stream.Expand (benchmarks) where
2929import GHC.Types (SPEC(..))
3030import Test.Inspection
3131
32−import qualified Streamly.Internal.Data.Stream as D
32+import qualified Streamly.Internal.Data.Stream.StreamD as D
3333#endif
3434
3535import qualified Stream.Common as Common
@@ -42,19 +42,19 @@ import Streamly.Benchmark.Prelude
4242 ( sourceFoldMapM, sourceFoldMapWith, sourceFoldMapWithM
4343 , sourceFoldMapWithStream, concatFoldableWith, concatForFoldableWith)
4444#else
45−import qualified Streamly.Internal.Data.Stream as S
45+import qualified Streamly.Internal.Data.Stream.StreamD as S
4646#ifdef USE_STREAMK
47−import Streamly.Internal.Data.Stream (Stream)
48−import Streamly.Internal.Data.StreamK (StreamK, CrossStreamK)
47+import Streamly.Internal.Data.Stream.StreamD (Stream)
48+import Streamly.Internal.Data.Stream.StreamK (StreamK, CrossStreamK)
4949import qualified Control.Applicative as AP
5050import qualified Streamly.Internal.Data.Fold as Fold
51−import qualified Streamly.Internal.Data.StreamK as StreamK
51+import qualified Streamly.Internal.Data.Stream.StreamK as StreamK
5252#else
53−import qualified Streamly.Internal.Data.Stream as StreamK
53+import qualified Streamly.Internal.Data.Stream.StreamD as StreamK
5454#endif
5555#endif
5656
57−import Test.Tasty.Bench
57+import Gauge
5858import Stream.Common
5959import Streamly.Benchmark.Common
6060import Prelude hiding (concatMap)
@@ -198,11 +198,7 @@ concatMapPure outer inner n =
198198 (fromStream $ sourceUnfoldr outer n)
199199
200200#ifdef INSPECTION
201−#if __GLASGOW_HASKELL__ >= 906
202−inspect $ hasNoTypeClassesExcept 'concatMapPure [''Applicative]
203−#else
204201inspect $ hasNoTypeClasses 'concatMapPure
205−#endif
206202inspect $ 'concatMapPure `hasNoType` ''SPEC
207203#endif
208204
@@ -215,11 +211,7 @@ concatMapRepl outer inner n =
215211 (fromStream . S.replicate inner) (fromStream $ sourceUnfoldrM outer n)
216212
217213#ifdef INSPECTION
218−#if __GLASGOW_HASKELL__ >= 906
219−inspect $ hasNoTypeClassesExcept 'concatMapRepl [''Applicative]
220−#else
221214inspect $ hasNoTypeClasses 'concatMapRepl
222−#endif
223215inspect $ 'concatMapRepl `hasNoType` ''SPEC
224216#endif
225217
benchmark/Streamly/Benchmark/Data/Stream/Generate.hs+4−4
@@ -31,16 +31,16 @@ import qualified Streamly.Prelude as S
3131import qualified Streamly.Internal.Data.Stream.IsStream as Stream
3232#else
3333import Stream.Common
34−import Streamly.Internal.Data.Stream (Stream)
35−import qualified Streamly.Internal.Data.Stream as Stream
34+import Streamly.Internal.Data.Stream.StreamD (Stream)
35+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
3636#ifdef USE_STREAMK
3737import System.IO.Unsafe (unsafeInterleaveIO)
38−import qualified Streamly.Internal.Data.StreamK as StreamK
38+import qualified Streamly.Internal.Data.Stream.StreamK as StreamK
3939import qualified Stream.Common as Common
4040#endif
4141#endif
4242
43−import Test.Tasty.Bench
43+import Gauge
4444import Streamly.Benchmark.Common
4545import qualified Prelude
4646
benchmark/Streamly/Benchmark/Data/Stream/Lift.hs+5−5
@@ -27,17 +27,17 @@ import qualified Streamly.Internal.Data.Fold as Fold
2727#ifdef USE_PRELUDE
2828import qualified Streamly.Internal.Data.Stream.IsStream as Stream
2929#else
30−import Streamly.Internal.Data.Stream (Stream)
31−import qualified Streamly.Internal.Data.Stream as Stream
30+import Streamly.Internal.Data.Stream.StreamD (Stream)
31+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
3232#ifdef USE_STREAMK
3333import Stream.Common (benchIO, drain)
34−import Streamly.Internal.Data.StreamK (StreamK)
35−import qualified Streamly.Internal.Data.StreamK as StreamK
34+import Streamly.Internal.Data.Stream.StreamK (StreamK)
35+import qualified Streamly.Internal.Data.Stream.StreamK as StreamK
3636import qualified Control.Monad.State.Strict as State
3737#endif
3838#endif
3939
40−import Test.Tasty.Bench
40+import Gauge
4141import Streamly.Benchmark.Common
4242import Prelude hiding (reverse, tail)
4343
benchmark/Streamly/Benchmark/Data/Stream/Reduce.hs+23−42
@@ -44,24 +44,24 @@ import Streamly.Benchmark.Prelude hiding
4444 , transformComposeMapM, transformTeeMapM, transformZipMapM)
4545#else
4646
47−import Streamly.Internal.Data.Stream (Stream)
48−import qualified Streamly.Internal.Data.Stream as S
47+import Streamly.Internal.Data.Stream.StreamD (Stream)
48+import qualified Streamly.Internal.Data.Stream.StreamD as S
4949#ifndef USE_STREAMLY_CORE
5050import qualified Streamly.Data.Stream.Prelude as S
51−import qualified Streamly.Internal.Data.Stream.Prelude as S
51+import qualified Streamly.Internal.Data.Stream.Time as S
5252#endif
5353
5454#ifdef USE_STREAMK
55−import Streamly.Internal.Data.StreamK (StreamK)
55+import Streamly.Internal.Data.Stream.StreamK (StreamK)
5656import qualified Streamly.Internal.Data.Parser as PR
57−import qualified Streamly.Internal.Data.StreamK as K
57+import qualified Streamly.Internal.Data.Stream.StreamK as K
5858#else
59−import qualified Streamly.Internal.Data.Stream as K
59+import qualified Streamly.Internal.Data.Stream.StreamD as K
6060#endif
6161
6262#endif
6363
64−import Test.Tasty.Bench
64+import Gauge
6565import Streamly.Benchmark.Common
6666import Stream.Common
6767import Prelude hiding (reverse, tail)
@@ -107,50 +107,30 @@ iterateSource g count len n = f count (sourceUnfoldrM len n)
107107-- Grouping transformations
108108-------------------------------------------------------------------------------
109109
110+#ifdef USE_PRELUDE
110111{-# INLINE groups #-}
111112groups :: MonadIO m => Stream m Int -> m ()
112−groups =
113−#ifdef USE_PRELUDE
114− Common.drain . S.groups FL.drain
115−#else
116− Common.drain . S.groupsWhile (==) FL.drain
117−#endif
113+groups = Common.drain . S.groups FL.drain
118114
119−{-# INLINE groupsWhileLT #-}
120−groupsWhileLT :: MonadIO m => Stream m Int -> m ()
121−groupsWhileLT =
122−#ifdef USE_PRELUDE
123− Common.drain . S.groupsBy (>) FL.drain
124−#else
125− Common.drain . S.groupsWhile (<) FL.drain
126−#endif
127−
128−{-# INLINE groupsWhileEq #-}
129−groupsWhileEq :: MonadIO m => Stream m Int -> m ()
130−groupsWhileEq =
131−#ifdef USE_PRELUDE
132− Common.drain . S.groupsBy (==) FL.drain
133−#else
134− Common.drain . S.groupsWhile (==) FL.drain
135−#endif
115+-- XXX Change this test when the order of comparison is later changed
116+{-# INLINE groupsByGT #-}
117+groupsByGT :: MonadIO m => Stream m Int -> m ()
118+groupsByGT = Common.drain . S.groupsBy (>) FL.drain
136119
120+{-# INLINE groupsByEq #-}
121+groupsByEq :: MonadIO m => Stream m Int -> m ()
122+groupsByEq = Common.drain . S.groupsBy (==) FL.drain
137123
124+-- XXX Change this test when the order of comparison is later changed
138125{-# INLINE groupsByRollingLT #-}
139126groupsByRollingLT :: MonadIO m => Stream m Int -> m ()
140127groupsByRollingLT =
141−#ifdef USE_PRELUDE
142128 Common.drain . S.groupsByRolling (<) FL.drain
143−#else
144− Common.drain . S.groupsRollingBy (<) FL.drain
145−#endif
146129
147130{-# INLINE groupsByRollingEq #-}
148131groupsByRollingEq :: MonadIO m => Stream m Int -> m ()
149132groupsByRollingEq =
150−#ifdef USE_PRELUDE
151133 Common.drain . S.groupsByRolling (==) FL.drain
152−#else
153− Common.drain . S.groupsRollingBy (==) FL.drain
154134#endif
155135
156136{-# INLINE foldMany #-}
@@ -199,7 +179,7 @@ refoldIterateM =
199179{-# INLINE parseBreak #-}
200180parseBreak :: Monad m => StreamK m Int -> m ()
201181parseBreak s = do
202− r <- K.parseDBreak PR.one s
182+ r <- K.parseBreak PR.one s
203183 case r of
204184 (Left _, _) -> return ()
205185 (Right _, s1) -> parseBreak s1
@@ -210,15 +190,16 @@ o_1_space_grouping value =
210190 -- Buffering operations using heap proportional to group/window sizes.
211191 [ bgroup "grouping"
212192 [
193+#ifdef USE_PRELUDE
213194 benchIOSink value "groups" groups
214− , benchIOSink value "groupsWhileLT" groupsWhileLT
215− , benchIOSink value "groupsWhileEq" groupsWhileEq
195+ , benchIOSink value "groupsByGT" groupsByGT
196+ , benchIOSink value "groupsByEq" groupsByEq
216197 , benchIOSink value "groupsByRollingLT" groupsByRollingLT
217198 , benchIOSink value "groupsByRollingEq" groupsByRollingEq
218199 ,
219−
200+#endif
220201 -- XXX parseMany/parseIterate benchmarks are in the Parser/ParserD
221− -- modules we can bring those here. chunksOf benchmarks are in
202+ -- modules we can bring those here. arraysOf benchmarks are in
222203 -- Parser/ParserD/Array.Stream/FileSystem.Handle.
223204 benchIOSink value "foldMany" foldMany
224205 , benchIOSink value "foldManyPost" foldManyPost
benchmark/Streamly/Benchmark/Data/Stream/Split.hs+3−3
@@ -35,15 +35,15 @@ import qualified Streamly.Internal.FileSystem.Handle as IFH
3535import qualified Streamly.Internal.Unicode.Stream as IUS
3636import qualified Streamly.Prelude as S
3737
38−import Test.Tasty.Bench hiding (env)
38+import Gauge hiding (env)
3939import Prelude hiding (last, length)
4040import Streamly.Benchmark.Common
4141import Streamly.Benchmark.Common.Handle
4242
4343#ifdef INSPECTION
44−import Streamly.Internal.Data.Stream (Step(..))
44+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..))
4545
46−import qualified Streamly.Internal.Data.MutArray as MA
46+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
4747import qualified Streamly.Internal.Data.Unfold as IUF
4848
4949import Test.Inspection
benchmark/Streamly/Benchmark/Data/Stream/StreamD.hs+4−6
@@ -22,17 +22,15 @@
2222
2323module Main (main) where
2424
25−#if !MIN_VERSION_base(4,18,0)
2625import Control.Applicative (liftA2)
27−#endif
2826import Control.Monad (when)
2927import Data.Maybe (isJust)
30−import Test.Tasty.Bench (bench, nfIO, bgroup, Benchmark, defaultMain)
28+import Gauge (bench, nfIO, bgroup, Benchmark, defaultMain)
3129import System.Random (randomRIO)
3230import Prelude hiding (tail, mapM_, foldl, last, map, mapM, concatMap, zip)
3331
3432import qualified Prelude as P
35−import qualified Streamly.Internal.Data.Stream as S
33+import qualified Streamly.Internal.Data.Stream.StreamD as S
3634import qualified Streamly.Internal.Data.Unfold as UF
3735import qualified Streamly.Internal.Data.Fold as FL
3836
@@ -326,7 +324,7 @@ filterMap n = composeN n $ S.map (subtract 1) . S.filter (<= maxValue)
326324-- Nested Composition
327325-------------------------------------------------------------------------------
328326
329−instance Monad f => Applicative (S.Stream f) where
327+instance Applicative f => Applicative (S.Stream f) where
330328 {-# INLINE pure #-}
331329 pure = S.fromPure
332330
@@ -463,7 +461,7 @@ inspect $ 'unfoldManyRepl `hasNoType` ''SPEC
463461-------------------------------------------------------------------------------
464462
465463moduleName :: String
466−moduleName = "Data.StreamD"
464+moduleName = "Data.Stream.StreamD"
467465
468466o_1_space :: [Benchmark]
469467o_1_space =
benchmark/Streamly/Benchmark/Data/Stream/StreamKAlt.hs → benchmark/Streamly/Benchmark/Data/Stream/StreamDK.hsrenamed+3−3
@@ -18,12 +18,12 @@ import Prelude hiding ()
1818-- import qualified Prelude as P
1919-- import qualified Data.List as List
2020
21−import qualified Streamly.Internal.Data.StreamK.Alt as S
21+import qualified Streamly.Internal.Data.Stream.StreamDK as S
2222-- import qualified Streamly.Internal.Data.Stream.Common as SP
2323-- import qualified Streamly.Internal.Data.SVar as S
2424
2525import Streamly.Benchmark.Common
26−import Test.Tasty.Bench (bgroup, Benchmark, defaultMain)
26+import Gauge (bgroup, Benchmark, defaultMain)
2727
2828value :: Int
2929value = 100000
@@ -452,4 +452,4 @@ o_1_space =
452452 ]
453453
454454main :: IO ()
455−main = defaultMain [o_1_space]
455+main = defaultMain $ concat [o_1_space]
benchmark/Streamly/Benchmark/Data/Stream/StreamK.hs+11−12
@@ -21,23 +21,22 @@
2121
2222module Main (main) where
2323
24−#if !MIN_VERSION_base(4,18,0)
2524import Control.Applicative (liftA2)
26−#endif
2725import Control.Monad (when)
2826import Data.Maybe (isJust)
2927import System.Random (randomRIO)
3028import Prelude hiding
31− ( Foldable(..), tail, mapM_, last, map, mapM, concatMap, zipWith, init, iterate
29+ ( tail, mapM_, foldl, last, map, mapM, concatMap, zipWith, init, iterate
3230 , repeat, replicate
3331 )
3432
3533import qualified Prelude as P
3634import qualified Data.List as List
3735
38−import qualified Streamly.Internal.Data.StreamK as S
36+import qualified Streamly.Internal.Data.Stream.StreamK.Type as S
37+import qualified Streamly.Internal.Data.Stream.StreamK as S
3938
40−import Test.Tasty.Bench (bench, nfIO, bgroup, Benchmark, defaultMain)
39+import Gauge (bench, nfIO, bgroup, Benchmark, defaultMain)
4140
4241import Streamly.Benchmark.Common
4342
@@ -117,7 +116,7 @@ Benchmarks that need to be added
117116-- Stream generation and elimination
118117-------------------------------------------------------------------------------
119118
120−type Stream m a = S.StreamK m a
119+type Stream m a = S.Stream m a
121120
122121{-# INLINE unfoldr #-}
123122unfoldr :: Int -> Int -> Stream m Int
@@ -168,15 +167,15 @@ fromFoldable streamLen n = S.fromFoldable [n..n+streamLen]
168167{-# INLINE fromFoldableM #-}
169168fromFoldableM :: Monad m => Int -> Int -> Stream m Int
170169fromFoldableM streamLen n =
171− List.foldr S.consM S.nil (Prelude.fmap return [n..n+streamLen])
170+ Prelude.foldr S.consM S.nil (Prelude.fmap return [n..n+streamLen])
172171
173172{-# INLINABLE concatMapFoldableWith #-}
174−concatMapFoldableWith :: P.Foldable f
173+concatMapFoldableWith :: Foldable f
175174 => (Stream m b -> Stream m b -> Stream m b)
176175 -> (a -> Stream m b)
177176 -> f a
178177 -> Stream m b
179−concatMapFoldableWith f g = P.foldr (f . g) S.nil
178+concatMapFoldableWith f g = Prelude.foldr (f . g) S.nil
180179
181180{-# INLINE concatMapFoldableSerial #-}
182181concatMapFoldableSerial :: Int -> Int -> Stream m Int
@@ -502,7 +501,7 @@ concatMapBySerial outer inner n =
502501-- Nested Composition
503502-------------------------------------------------------------------------------
504503
505−instance Monad m => Applicative (S.StreamK m) where
504+instance Monad m => Applicative (S.Stream m) where
506505 {-# INLINE pure #-}
507506 pure = S.fromPure
508507
@@ -521,7 +520,7 @@ instance Monad m => Applicative (S.StreamK m) where
521520-- NOTE: even though concatMap for StreamD is 3x faster compared to StreamK,
522521-- the monad instance of StreamD is slower than StreamK after foldr/build
523522-- fusion.
524−instance Monad m => Monad (S.StreamK m) where
523+instance Monad m => Monad (S.Stream m) where
525524 {-# INLINE return #-}
526525 return = pure
527526
@@ -634,7 +633,7 @@ filterAllInNestedList str = do
634633-------------------------------------------------------------------------------
635634
636635moduleName :: String
637−moduleName = "Data.StreamK"
636+moduleName = "Data.Stream.StreamK"
638637
639638o_1_space_generation :: Int -> Benchmark
640639o_1_space_generation streamLen =
benchmark/Streamly/Benchmark/Data/Stream/ToStreamK.hs+6−5
@@ -1,5 +1,5 @@
11-- |
2−-- Module : Streamly.Benchmark.Data.StreamK
2+-- Module : Streamly.Benchmark.Data.Stream.StreamK
33-- Copyright : (c) 2018 Composewell Technologies
44--
55-- License : BSD3
@@ -33,11 +33,12 @@ import Prelude hiding
3333import qualified Prelude as P
3434-- import qualified Data.List as List
3535
36−import qualified Streamly.Internal.Data.StreamK as S
37−import qualified Streamly.Internal.Data.Stream as D
36+import qualified Streamly.Internal.Data.Stream.StreamK.Type as S
37+import qualified Streamly.Internal.Data.Stream.StreamK as S
38+import qualified Streamly.Internal.Data.Stream.StreamD as D
3839import qualified Streamly.Internal.Data.Fold as Fold
3940
40−import Test.Tasty.Bench (bgroup, Benchmark, defaultMain)
41+import Gauge (bgroup, Benchmark, defaultMain)
4142
4243import Streamly.Benchmark.Common
4344
@@ -120,7 +121,7 @@ fromFoldable streamLen n = S.fromFoldable [n..n+streamLen]
120121{-# INLINE fromFoldableM #-}
121122fromFoldableM :: Monad m => Int -> Int -> Stream m Int
122123fromFoldableM streamLen n =
123− Prelude.foldr (S.consM . return) S.nil [n .. n + streamLen]
124+ Prelude.foldr S.consM S.nil (Prelude.fmap return [n..n+streamLen])
124125
125126{-
126127{-# INLINABLE concatMapFoldableWith #-}
benchmark/Streamly/Benchmark/Data/Stream/Transform.hs+6−6
@@ -43,22 +43,22 @@ import qualified Streamly.Internal.Data.Fold as Fold
4343import qualified Streamly.Internal.Data.Stream.IsStream as Stream
4444import Streamly.Internal.Data.Time.Units
4545#else
46−import Streamly.Internal.Data.Stream (Stream)
47−import qualified Streamly.Internal.Data.Stream as Stream
46+import Streamly.Internal.Data.Stream.StreamD (Stream)
47+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
4848#ifndef USE_STREAMLY_CORE
49−import qualified Streamly.Internal.Data.Stream.Prelude as Stream
49+import qualified Streamly.Internal.Data.Stream.Time as Stream
5050#endif
5151#ifdef USE_STREAMK
5252import Control.DeepSeq (NFData(..))
5353import Data.Functor.Identity (Identity(..))
5454import qualified Prelude
5555import qualified Streamly.Internal.Data.Fold as Fold
56−import Streamly.Internal.Data.StreamK (StreamK)
57−import qualified Streamly.Internal.Data.StreamK as StreamK
56+import Streamly.Internal.Data.Stream.StreamK (StreamK)
57+import qualified Streamly.Internal.Data.Stream.StreamK as StreamK
5858#endif
5959#endif
6060
61−import Test.Tasty.Bench
61+import Gauge
6262import Stream.Common hiding (scanl')
6363import Streamly.Benchmark.Common
6464import Prelude hiding (sequence, mapM)
benchmark/Streamly/Benchmark/Data/Unfold.hs+7−17
@@ -31,18 +31,17 @@ import qualified Prelude
3131import qualified Streamly.FileSystem.Handle as FH
3232import qualified Streamly.Internal.Data.Fold as FL
3333import qualified Streamly.Internal.Data.Unfold as UF
34−import qualified Streamly.Internal.Data.Unfold.Prelude as UF
35−import qualified Streamly.Internal.Data.Stream as S
36−import qualified Streamly.Internal.Data.Stream as D
37−import qualified Streamly.Internal.Data.StreamK as K
34+import qualified Streamly.Internal.Data.Unfold.Exception as UF
35+import qualified Streamly.Internal.Data.Stream.StreamD as S
36+import qualified Streamly.Internal.Data.Stream.StreamD as D
37+import qualified Streamly.Internal.Data.Stream.StreamK as K
3838
39−import Test.Tasty.Bench hiding (env)
39+import Gauge hiding (env)
4040import Prelude hiding (take, filter, zipWith, map, mapM, takeWhile)
4141import Streamly.Benchmark.Common
4242import Streamly.Benchmark.Common.Handle
4343
4444#ifdef INSPECTION
45−import Control.Monad.Catch (MonadCatch)
4645import Test.Inspection
4746#endif
4847
@@ -753,7 +752,7 @@ readWriteOnExceptionUnfold inh devNull =
753752 in S.fold (FH.write devNull) $ S.unfold readEx inh
754753
755754#ifdef INSPECTION
756−-- inspect $ hasNoTypeClasses 'readWriteOnExceptionUnfold
755+inspect $ hasNoTypeClasses 'readWriteOnExceptionUnfold
757756-- inspect $ 'readWriteOnExceptionUnfold `hasNoType` ''Step
758757#endif
759758
@@ -765,8 +764,7 @@ readWriteHandleExceptionUnfold inh devNull =
765764 in S.fold (FH.write devNull) $ S.unfold readEx inh
766765
767766#ifdef INSPECTION
768−-- hasNoTypeClasses started failing in GHC 9.4.4
769−-- inspect $ hasNoTypeClasses 'readWriteHandleExceptionUnfold
767+inspect $ hasNoTypeClasses 'readWriteHandleExceptionUnfold
770768-- inspect $ 'readWriteHandleExceptionUnfold `hasNoType` ''Step
771769#endif
772770
@@ -777,11 +775,7 @@ readWriteFinally_Unfold inh devNull =
777775 in S.fold (FH.write devNull) $ S.unfold readEx inh
778776
779777#ifdef INSPECTION
780−#if __GLASGOW_HASKELL__ >= 906
781−inspect $ hasNoTypeClassesExcept 'readWriteFinally_Unfold [''MonadCatch]
782−#else
783778inspect $ hasNoTypeClasses 'readWriteFinally_Unfold
784−#endif
785779-- inspect $ 'readWriteFinallyUnfold `hasNoType` ''Step
786780#endif
787781
@@ -797,11 +791,7 @@ readWriteBracket_Unfold inh devNull =
797791 in S.fold (FH.write devNull) $ S.unfold readEx inh
798792
799793#ifdef INSPECTION
800−#if __GLASGOW_HASKELL__ >= 906
801−inspect $ hasNoTypeClassesExcept 'readWriteBracket_Unfold [''MonadCatch]
802−#else
803794inspect $ hasNoTypeClasses 'readWriteBracket_Unfold
804−#endif
805795-- inspect $ 'readWriteBracketUnfold `hasNoType` ''Step
806796#endif
807797
benchmark/Streamly/Benchmark/FileSystem/Handle.hs+1−1
@@ -22,7 +22,7 @@ import Streamly.Benchmark.Common.Handle (mkHandleBenchEnv)
2222import qualified Handle.ReadWrite as RW
2323import qualified Handle.Read as RO
2424
25−import Test.Tasty.Bench hiding (env)
25+import Gauge hiding (env)
2626import Prelude hiding (last, length)
2727import Streamly.Benchmark.Common
2828
benchmark/Streamly/Benchmark/FileSystem/Handle/Read.hs+34−33
@@ -33,22 +33,23 @@ import qualified Streamly.Data.Stream as Stream
3333import qualified Streamly.Data.Fold as Fold
3434import qualified Streamly.FileSystem.Handle as FH
3535import qualified Streamly.Internal.Data.Array as A
36+import qualified Streamly.Internal.Data.Array.Type as AT
3637import qualified Streamly.Internal.Data.Fold as FL
37−import qualified Streamly.Internal.Data.Stream as IP
38+import qualified Streamly.Internal.Data.Stream.StreamD as IP
3839import qualified Streamly.Internal.FileSystem.Handle as IFH
3940import qualified Streamly.Internal.Unicode.Stream as IUS
4041import qualified Streamly.Data.Stream.Prelude as S
4142import qualified Streamly.Unicode.Stream as SS
4243
43−import Test.Tasty.Bench hiding (env)
44+import Gauge hiding (env)
4445import Prelude hiding (last, length)
4546import Streamly.Benchmark.Common.Handle
4647
4748#ifdef INSPECTION
48−import Streamly.Internal.Data.Stream (Step(..), FoldMany)
49+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..), FoldMany)
4950
50−import qualified Streamly.Internal.Data.MutArray as MutArray
51−import qualified Streamly.Internal.Data.Stream as D
51+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
52+import qualified Streamly.Internal.Data.Stream.StreamD as D
5253import qualified Streamly.Internal.Data.Unfold as IUF
5354
5455import Test.Inspection
@@ -68,7 +69,7 @@ readLast = S.fold Fold.last . S.unfold FH.reader
6869inspect $ hasNoTypeClasses 'readLast
6970inspect $ 'readLast `hasNoType` ''Step -- S.unfold
7071inspect $ 'readLast `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
71−inspect $ 'readLast `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
72+inspect $ 'readLast `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
7273#endif
7374
7475-- assert that flattenArrays constructors are not present
@@ -80,7 +81,7 @@ readCountBytes = S.fold Fold.length . S.unfold FH.reader
8081inspect $ hasNoTypeClasses 'readCountBytes
8182inspect $ 'readCountBytes `hasNoType` ''Step -- S.unfold
8283inspect $ 'readCountBytes `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
83−inspect $ 'readCountBytes `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
84+inspect $ 'readCountBytes `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
8485#endif
8586
8687-- | Count the number of lines in a file.
@@ -95,7 +96,7 @@ readCountLines =
9596inspect $ hasNoTypeClasses 'readCountLines
9697inspect $ 'readCountLines `hasNoType` ''Step
9798inspect $ 'readCountLines `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
98−inspect $ 'readCountLines `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
99+inspect $ 'readCountLines `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
99100#endif
100101
101102-- | Count the number of words in a file.
@@ -119,7 +120,7 @@ readSumBytes = S.fold Fold.sum . S.unfold FH.reader
119120inspect $ hasNoTypeClasses 'readSumBytes
120121inspect $ 'readSumBytes `hasNoType` ''Step
121122inspect $ 'readSumBytes `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
122−inspect $ 'readSumBytes `hasNoType` ''MutArray.ArrayUnsafe -- FH.read/A.read
123+inspect $ 'readSumBytes `hasNoType` ''MA.ArrayUnsafe -- FH.read/A.read
123124#endif
124125
125126-- XXX When we mark this with INLINE and we have two benchmarks using S.drain
@@ -214,7 +215,7 @@ o_1_space_reduce_toBytes env =
214215
215216chunksOfSum :: Int -> Handle -> IO Int
216217chunksOfSum n inh =
217− S.fold Fold.length $ IP.groupsOf n FL.sum (S.unfold FH.reader inh)
218+ S.fold Fold.length $ IP.chunksOf n FL.sum (S.unfold FH.reader inh)
218219
219220foldManyPostChunksOfSum :: Int -> Handle -> IO Int
220221foldManyPostChunksOfSum n inh =
@@ -230,34 +231,34 @@ foldManyChunksOfSum n inh =
230231-- Even though allocations remain the same in both cases inlining improves time
231232-- by 4x.
232233-- | Slice in chunks of size n and get the count of chunks.
233−{-# INLINE groupsOf #-}
234−groupsOf :: Int -> Handle -> IO Int
235−groupsOf n inh =
234+{-# INLINE chunksOf #-}
235+chunksOf :: Int -> Handle -> IO Int
236+chunksOf n inh =
236237 -- writeNUnsafe gives 2.5x boost here over writeN.
237238 S.fold Fold.length
238− $ IP.groupsOf n (A.writeNUnsafe n) (S.unfold FH.reader inh)
239+ $ IP.chunksOf n (AT.writeNUnsafe n) (S.unfold FH.reader inh)
239240
240241#ifdef INSPECTION
241−inspect $ hasNoTypeClasses 'groupsOf
242−inspect $ 'groupsOf `hasNoType` ''Step
243−inspect $ 'groupsOf `hasNoType` ''FoldMany
244−inspect $ 'groupsOf `hasNoType` ''MutArray.ArrayUnsafe -- AT.writeNUnsafe
242+inspect $ hasNoTypeClasses 'chunksOf
243+inspect $ 'chunksOf `hasNoType` ''Step
244+inspect $ 'chunksOf `hasNoType` ''FoldMany
245+inspect $ 'chunksOf `hasNoType` ''AT.ArrayUnsafe -- AT.writeNUnsafe
245246 -- FH.read/A.read
246−inspect $ 'groupsOf `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
247+inspect $ 'chunksOf `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
247248#endif
248249
249−{-# INLINE chunksOf #-}
250−chunksOf :: Int -> Handle -> IO Int
251−chunksOf n inh =
252− S.fold Fold.length $ Stream.chunksOf n (S.unfold FH.reader inh)
250+{-# INLINE arraysOf #-}
251+arraysOf :: Int -> Handle -> IO Int
252+arraysOf n inh =
253+ S.fold Fold.length $ Stream.arraysOf n (S.unfold FH.reader inh)
253254
254255o_1_space_reduce_read_grouped :: BenchEnv -> [Benchmark]
255256o_1_space_reduce_read_grouped env =
256257 [ bgroup "reduce/read/chunks"
257− [ mkBench ("S.groupsOf " ++ show (bigSize env) ++ " FL.sum") env $
258+ [ mkBench ("S.chunksOf " ++ show (bigSize env) ++ " FL.sum") env $
258259 \inh _ ->
259260 chunksOfSum (bigSize env) inh
260− , mkBench "S.groupsOf 1 FL.sum" env $ \inh _ ->
261+ , mkBench "S.chunksOf 1 FL.sum" env $ \inh _ ->
261262 chunksOfSum 1 inh
262263
263264 -- XXX investigate why we need inline/noinline in these cases (GHC)
@@ -280,20 +281,20 @@ o_1_space_reduce_read_grouped env =
280281 $ \inh _ -> inline foldManyChunksOfSum 1 inh
281282
282283 -- folding chunks to arrays
283− , mkBenchSmall "S.groupsOf 1" env $ \inh _ ->
284− groupsOf 1 inh
285− , mkBench "S.groupsOf 10" env $ \inh _ ->
286− groupsOf 10 inh
287− , mkBench "S.groupsOf 1000" env $ \inh _ ->
288− groupsOf 1000 inh
289−
290− -- chunksOf may use a different impl than groupsOf
291284 , mkBenchSmall "S.chunksOf 1" env $ \inh _ ->
292285 chunksOf 1 inh
293286 , mkBench "S.chunksOf 10" env $ \inh _ ->
294287 chunksOf 10 inh
295288 , mkBench "S.chunksOf 1000" env $ \inh _ ->
296289 chunksOf 1000 inh
290+
291+ -- arraysOf may use a different impl than chunksOf
292+ , mkBenchSmall "S.arraysOf 1" env $ \inh _ ->
293+ arraysOf 1 inh
294+ , mkBench "S.arraysOf 10" env $ \inh _ ->
295+ arraysOf 10 inh
296+ , mkBench "S.arraysOf 1000" env $ \inh _ ->
297+ arraysOf 1000 inh
297298 ]
298299 ]
299300
benchmark/Streamly/Benchmark/FileSystem/Handle/ReadWrite.hs+12−11
@@ -34,15 +34,16 @@ import qualified Streamly.Internal.FileSystem.Handle as IFH
3434import qualified Streamly.Data.Array as A
3535import qualified Streamly.Data.Stream.Prelude as S
3636
37−import Test.Tasty.Bench hiding (env)
37+import Gauge hiding (env)
3838import Streamly.Benchmark.Common.Handle
3939
4040#ifdef INSPECTION
41−import Streamly.Internal.Data.Stream (Step(..))
41+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..))
4242
43−import qualified Streamly.Internal.Data.Stream as D
43+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
4444import qualified Streamly.Internal.Data.Tuple.Strict as Strict
45−import qualified Streamly.Internal.Data.MutArray as MutArray
45+import qualified Streamly.Internal.Data.Array.Mut.Stream as MAS
46+import qualified Streamly.Internal.Data.Array.Type as AT
4647
4748import Test.Inspection
4849#endif
@@ -82,7 +83,7 @@ copyStream inh outh = S.fold (FH.write outh) (S.unfold FH.reader inh)
8283inspect $ hasNoTypeClasses 'copyStream
8384inspect $ 'copyStream `hasNoType` ''Step -- S.unfold
8485inspect $ 'copyStream `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
85−inspect $ 'copyStream `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
86+inspect $ 'copyStream `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
8687 -- FH.read/A.read
8788inspect $ 'copyStream `hasNoType` ''Strict.Tuple3' -- FH.write/chunksOf
8889#endif
@@ -108,8 +109,8 @@ readFromBytesNull inh devNull = IFH.putBytes devNull $ S.unfold FH.reader inh
108109#ifdef INSPECTION
109110inspect $ hasNoTypeClasses 'readFromBytesNull
110111inspect $ 'readFromBytesNull `hasNoType` ''Step
111−inspect $ 'readFromBytesNull `hasNoType` ''MutArray.SpliceState
112−inspect $ 'readFromBytesNull `hasNoType` ''MutArray.ArrayUnsafe -- FH.fromBytes/S.chunksOf
112+inspect $ 'readFromBytesNull `hasNoType` ''MAS.SpliceState
113+inspect $ 'readFromBytesNull `hasNoType` ''AT.ArrayUnsafe -- FH.fromBytes/S.arraysOf
113114inspect $ 'readFromBytesNull `hasNoType` ''D.FoldMany
114115#endif
115116
@@ -122,8 +123,8 @@ readWithFromBytesNull inh devNull =
122123#ifdef INSPECTION
123124inspect $ hasNoTypeClasses 'readWithFromBytesNull
124125inspect $ 'readWithFromBytesNull `hasNoType` ''Step
125−inspect $ 'readWithFromBytesNull `hasNoType` ''MutArray.SpliceState
126−inspect $ 'readWithFromBytesNull `hasNoType` ''MutArray.ArrayUnsafe -- FH.fromBytes/S.chunksOf
126+inspect $ 'readWithFromBytesNull `hasNoType` ''MAS.SpliceState
127+inspect $ 'readWithFromBytesNull `hasNoType` ''AT.ArrayUnsafe -- FH.fromBytes/S.arraysOf
127128inspect $ 'readWithFromBytesNull `hasNoType` ''D.FoldMany
128129#endif
129130
@@ -170,7 +171,7 @@ writeReadWith inh devNull = IUF.fold fld unf (defaultChunkSize, inh)
170171inspect $ hasNoTypeClasses 'writeReadWith
171172inspect $ 'writeReadWith `hasNoType` ''Step
172173inspect $ 'writeReadWith `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
173−inspect $ 'writeReadWith `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
174+inspect $ 'writeReadWith `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
174175 -- FH.read/A.read
175176#endif
176177
@@ -187,7 +188,7 @@ writeRead inh devNull = IUF.fold fld unf inh
187188inspect $ hasNoTypeClasses 'writeRead
188189inspect $ 'writeRead `hasNoType` ''Step
189190inspect $ 'writeRead `hasNoType` ''IUF.ConcatState -- FH.read/UF.many
190−inspect $ 'writeRead `hasNoType` ''MutArray.ArrayUnsafe -- FH.write/writeNUnsafe
191+inspect $ 'writeRead `hasNoType` ''AT.ArrayUnsafe -- FH.write/writeNUnsafe
191192 -- FH.read/A.read
192193#endif
193194
benchmark/Streamly/Benchmark/Prelude/Adaptive.hs+1−1
@@ -10,7 +10,7 @@
1010import Control.Concurrent (threadDelay)
1111import Control.Monad (when)
1212import Control.Monad.IO.Class (liftIO)
13−import Test.Tasty.Bench
13+import Gauge
1414import Streamly.Prelude as S
1515import System.Random (randomRIO)
1616
benchmark/Streamly/Benchmark/Prelude/Ahead.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.Ahead"
benchmark/Streamly/Benchmark/Prelude/Async.hs+1−1
@@ -16,7 +16,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream.Transform as Transform
1616import Streamly.Benchmark.Common
1717import Streamly.Benchmark.Prelude
1818
19−import Test.Tasty.Bench
19+import Gauge
2020
2121moduleName :: String
2222moduleName = "Prelude.Async"
benchmark/Streamly/Benchmark/Prelude/Concurrent.hs+1−1
@@ -15,7 +15,7 @@ import Streamly.Prelude
1515 , fromWAsync, parallel, fromParallel
1616 )
1717
18−import Test.Tasty.Bench
18+import Gauge
1919import qualified Streamly.Prelude as S
2020
2121-------------------------------------------------------------------------------
benchmark/Streamly/Benchmark/Prelude/Merge.hs+2−2
@@ -30,13 +30,13 @@ import qualified Streamly.Internal.Data.Stream.IsStream as Stream
3030import Streamly.Benchmark.Common
3131import Streamly.Benchmark.Prelude
3232
33−import Test.Tasty.Bench
33+import Gauge
3434
3535#ifdef INSPECTION
3636import GHC.Types (SPEC(..))
3737import Test.Inspection
3838
39−import qualified Streamly.Internal.Data.Stream as D
39+import qualified Streamly.Internal.Data.Stream.StreamD as D
4040#endif
4141
4242moduleName :: String
benchmark/Streamly/Benchmark/Prelude/Parallel.hs+1−1
@@ -22,7 +22,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream as Internal
2222import Streamly.Benchmark.Common
2323import Streamly.Benchmark.Prelude
2424
25−import Test.Tasty.Bench
25+import Gauge
2626
2727moduleName :: String
2828moduleName = "Prelude.Parallel"
benchmark/Streamly/Benchmark/Prelude/Rate.hs+1−1
@@ -14,7 +14,7 @@ import qualified Streamly.Prelude as S
1414import Streamly.Benchmark.Common
1515import Streamly.Benchmark.Prelude
1616
17−import Test.Tasty.Bench
17+import Gauge
1818
1919moduleName :: String
2020moduleName = "Prelude.Rate"
benchmark/Streamly/Benchmark/Prelude/WAsync.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.WAsync"
benchmark/Streamly/Benchmark/Prelude/WSerial.hs+2−2
@@ -29,13 +29,13 @@ import qualified Streamly.Internal.Data.Unfold as UF
2929import Streamly.Benchmark.Common
3030import Streamly.Benchmark.Prelude
3131
32−import Test.Tasty.Bench
32+import Gauge
3333
3434#ifdef INSPECTION
3535import GHC.Types (SPEC(..))
3636import Test.Inspection
3737
38−import qualified Streamly.Internal.Data.Stream as D
38+import qualified Streamly.Internal.Data.Stream.StreamD as D
3939#endif
4040
4141moduleName :: String
benchmark/Streamly/Benchmark/Prelude/ZipAsync.hs+1−1
@@ -15,7 +15,7 @@ import qualified Streamly.Prelude as S
1515import Streamly.Benchmark.Common
1616import Streamly.Benchmark.Prelude
1717
18−import Test.Tasty.Bench
18+import Gauge
1919
2020moduleName :: String
2121moduleName = "Prelude.ZipAsync"
benchmark/Streamly/Benchmark/Prelude/ZipSerial.hs+2−2
@@ -29,13 +29,13 @@ import qualified Streamly.Prelude as S
2929import Streamly.Benchmark.Common
3030import Streamly.Benchmark.Prelude hiding (sourceUnfoldrM)
3131
32−import Test.Tasty.Bench
32+import Gauge
3333
3434#ifdef INSPECTION
3535import GHC.Types (SPEC(..))
3636import Test.Inspection
3737
38−import qualified Streamly.Internal.Data.Stream as D
38+import qualified Streamly.Internal.Data.Stream.StreamD as D
3939#endif
4040
4141moduleName :: String
benchmark/Streamly/Benchmark/Unicode/Char.hs+2−2
@@ -16,7 +16,7 @@ import Control.DeepSeq (NFData(..))
1616import Streamly.Internal.Data.Array (Array)
1717import System.FilePath (dropExtensions, takeFileName)
1818import System.FilePath.Posix ((</>))
19−import Test.Tasty.Bench (Benchmark, bench, bgroup, defaultMain, env, nfIO)
19+import Gauge.Main (Benchmark, bench, bgroup, defaultMain, env, nfIO)
2020import Streamly.Internal.Unicode.Char
2121 ( NormalizationMode(NFC, NFD, NFKC, NFKD)
2222 , normalize
@@ -24,7 +24,7 @@ import Streamly.Internal.Unicode.Char
2424import Streamly.Benchmark.Common (o_1_space_prefix)
2525
2626import qualified Streamly.Internal.Data.Array as Array
27−import qualified Streamly.Internal.Data.Stream as IsStream
27+import qualified Streamly.Internal.Data.Stream.StreamD as IsStream
2828import qualified System.Directory as Dir
2929
3030--------------------------------------------------------------------------------
benchmark/Streamly/Benchmark/Unicode/Parser.hsdeleted−107
@@ -1,107 +0,0 @@
1−-- |
2−-- Module : Streamly.Benchmark.Data.Parser
3−-- Copyright : (c) 2020 Composewell Technologies
4−--
5−-- License : BSD-3-Clause
6−-- Maintainer : streamly@composewell.com
7−
8−{-# LANGUAGE CPP #-}
9−{-# LANGUAGE FlexibleContexts #-}
10−{-# LANGUAGE ScopedTypeVariables #-}
11−{-# OPTIONS_GHC -Wno-orphans #-}
12−
13−#undef FUSION_CHECK
14−#ifdef FUSION_CHECK
15−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
16−#endif
17−
18−module Main
19− (
20− main
21− ) where
22−
23−import Control.DeepSeq (NFData(..))
24−import Control.Monad (replicateM_)
25−import Streamly.Internal.Data.Parser (ParseError(..))
26−import Streamly.Internal.Data.Stream (Stream)
27−import Prelude hiding
28− (any, all, take, sequence, sequence_, sequenceA, takeWhile, dropWhile)
29−
30−import qualified Streamly.Data.Array as Array
31−import qualified Streamly.Data.Stream as Stream
32−import qualified Streamly.Internal.Unicode.Parser as PRU
33−
34−import Test.Tasty.Bench hiding (env)
35−import Streamly.Benchmark.Common
36−
37−runParser :: Int -> (Stream IO Char -> IO a) -> IO ()
38−runParser count p = do
39− let v = "+123456789.123456789e-123"
40− let !(arr :: Array.Array Char) = Array.fromListN (length v) v
41− let s = Stream.unfold Array.reader arr
42− replicateM_ count (p s)
43−
44−-- | Takes a fold method, and uses it with a default source.
45−{-# INLINE benchIOSink #-}
46−benchIOSink :: Int -> String -> (Stream IO Char -> IO b) -> Benchmark
47−benchIOSink value name f = bench name $ nfIO $ runParser value f
48−
49−{-# INLINE doubleParser #-}
50−doubleParser :: Monad m => Stream m Char -> m (Either ParseError (Int, Int))
51−doubleParser = Stream.parse PRU.doubleParser
52−
53−{-# INLINE number #-}
54−number :: Monad m => Stream m Char -> m (Either ParseError (Integer, Int))
55−number = Stream.parse PRU.number
56−
57−{-# INLINE double #-}
58−double :: Monad m => Stream m Char -> m (Either ParseError Double)
59−double = Stream.parse PRU.double
60−
61−{-# INLINE numberDouble #-}
62−numberDouble :: Monad m => Stream m Char -> m (Either ParseError Double)
63−numberDouble = Stream.parse p
64− where p = uncurry PRU.mkDouble <$> PRU.number
65−
66−-------------------------------------------------------------------------------
67−-- Benchmarks
68−-------------------------------------------------------------------------------
69−
70−moduleName :: String
71−moduleName = "Unicode.Parser"
72−
73−instance NFData ParseError where
74− {-# INLINE rnf #-}
75− rnf (ParseError x) = rnf x
76−
77−o_n_heap_serial :: Int -> [Benchmark]
78−o_n_heap_serial value =
79− [
80− benchIOSink value "doubleParser" doubleParser
81− , benchIOSink value "number" number
82− , benchIOSink value "double (doubleParser)" double
83− , benchIOSink value "double (number)" numberDouble
84− ]
85−
86−-------------------------------------------------------------------------------
87−-- Driver
88−-------------------------------------------------------------------------------
89−
90−main :: IO ()
91−main = do
92−#ifndef FUSION_CHECK
93− runWithCLIOpts defaultStreamSize allBenchmarks
94−
95− where
96−
97− allBenchmarks value =
98− [ bgroup (o_n_heap_prefix moduleName) (o_n_heap_serial value)
99− ]
100−#else
101− -- Enable FUSION_CHECK macro at the beginning of the file
102− -- Enable one benchmark below, and run the benchmark
103− -- Check the .dump-simpl output
104− let value = 100000
105− runParser value double
106− return ()
107−#endif
benchmark/Streamly/Benchmark/Unicode/Stream.hs+10−28
@@ -11,11 +11,6 @@
1111{-# LANGUAGE CPP #-}
1212{-# LANGUAGE ScopedTypeVariables #-}
1313
14−#undef FUSION_CHECK
15−#ifdef FUSION_CHECK
16−{-# OPTIONS_GHC -ddump-simpl -ddump-to-file -dsuppress-all #-}
17−#endif
18−
1914#ifdef __HADDOCK_VERSION__
2015#undef INSPECTION
2116#endif
@@ -32,23 +27,22 @@ import System.IO (Handle)
3227
3328import qualified Streamly.Data.Array as Array
3429import qualified Streamly.Data.Fold as Fold
35−import qualified Streamly.Internal.Data.Stream as Stream
30+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
3631import qualified Streamly.Internal.Data.Unfold as Unfold
3732import qualified Streamly.Internal.FileSystem.Handle as Handle
3833import qualified Streamly.Internal.Unicode.Array as UnicodeArr
3934import qualified Streamly.Internal.Unicode.Stream as Unicode
4035
41−import Test.Tasty.Bench hiding (env)
36+import Gauge hiding (env)
4237import Streamly.Benchmark.Common
4338import Streamly.Benchmark.Common.Handle
4439
4540#ifdef INSPECTION
46−import Streamly.Internal.Data.MutByteArray (Unbox)
47−import Streamly.Internal.Data.Stream (Step(..))
48−import qualified Streamly.Internal.Data.Array as Array
49−import qualified Streamly.Internal.Data.MutArray as MutArray
50−import qualified Streamly.Internal.Data.Fold as Fold
41+import Streamly.Internal.Data.Unboxed (Unbox)
42+import Streamly.Internal.Data.Stream.StreamD.Type (Step(..))
43+import qualified Streamly.Internal.Data.Fold.Type as Fold
5144import qualified Streamly.Internal.Data.Tuple.Strict as Strict
45+import qualified Streamly.Internal.Data.Array.Type as Array
5246
5347import Test.Inspection
5448#endif
@@ -62,7 +56,7 @@ copyCodecUtf8ArraysLenient :: Handle -> Handle -> IO ()
6256copyCodecUtf8ArraysLenient inh outh =
6357 Stream.fold (Handle.write outh)
6458 $ Unicode.encodeUtf8'
65− $ Unicode.decodeUtf8Chunks
59+ $ Unicode.decodeUtf8Arrays
6660 $ Handle.readChunks inh
6761
6862#ifdef INSPECTION
@@ -224,7 +218,7 @@ inspect $ 'copyStreamLatin1' `hasNoType` ''Step
224218inspect $ 'copyStreamLatin1' `hasNoType` ''Unfold.ConcatState -- Handle.read/UF.many
225219
226220inspect $ 'copyStreamLatin1' `hasNoType` ''Fold.Step
227−inspect $ 'copyStreamLatin1' `hasNoType` ''MutArray.ArrayUnsafe -- Handle.write/writeNUnsafe
221+inspect $ 'copyStreamLatin1' `hasNoType` ''Array.ArrayUnsafe -- Handle.write/writeNUnsafe
228222 -- Handle.read/Array.read
229223inspect $ 'copyStreamLatin1' `hasNoType` ''Strict.Tuple3' -- Handle.write/chunksOf
230224#endif
@@ -245,7 +239,7 @@ inspect $ 'copyStreamLatin1 `hasNoType` ''Unfold.ConcatState -- Handle.read/UF.m
245239
246240inspect $ 'copyStreamLatin1 `hasNoType` ''Fold.ManyState
247241inspect $ 'copyStreamLatin1 `hasNoType` ''Fold.Step
248−inspect $ 'copyStreamLatin1 `hasNoType` ''MutArray.ArrayUnsafe -- Handle.write/writeNUnsafe
242+inspect $ 'copyStreamLatin1 `hasNoType` ''Array.ArrayUnsafe -- Handle.write/writeNUnsafe
249243 -- Handle.read/Array.read
250244inspect $ 'copyStreamLatin1 `hasNoType` ''Strict.Tuple3' -- Handle.write/chunksOf
251245#endif
@@ -286,8 +280,7 @@ _copyStreamUtf8'Fold :: Handle -> Handle -> IO ()
286280_copyStreamUtf8'Fold inh outh =
287281 Stream.fold (Handle.write outh)
288282 $ Unicode.encodeUtf8
289− $ Stream.catRights
290− $ Stream.parseMany Unicode.writeCharUtf8'
283+ $ Stream.foldMany Unicode.writeCharUtf8'
291284 $ Stream.unfold Handle.reader inh
292285
293286{-# NOINLINE _copyStreamUtf8Parser #-}
@@ -324,7 +317,6 @@ o_1_space_decode_encode_read env =
324317
325318main :: IO ()
326319main = do
327−#ifndef FUSION_CHECK
328320 env <- mkHandleBenchEnv
329321 defaultMain (allBenchmarks env)
330322
@@ -337,13 +329,3 @@ main = do
337329 , o_1_space_decode_encode_read env
338330 ]
339331 ]
340−#else
341− -- Enable FUSION_CHECK macro at the beginning of the file
342− -- Enable one benchmark below, and run the benchmark
343− -- Check the .dump-simpl output
344− env <- mkHandleBenchEnv
345− let mkHandles (RefHandles {bigInH = inh, outputH = outh}) = Handles inh outh
346− (Handles inh outh) <- getHandles env mkHandles
347− copyStreamLatin1' inh outh
348− return ()
349−#endif
benchmark/Streamly/Benchmark/Unicode/Utf8.hs+1−1
@@ -13,7 +13,7 @@
1313import Data.Char (chr)
1414import Streamly.Internal.Unicode.Utf8 (Utf8)
1515
16−import Test.Tasty.Bench hiding (env)
16+import Gauge hiding (env)
1717import Streamly.Benchmark.Common
1818
1919import qualified Streamly.Internal.Unicode.Utf8 as Utf8
benchmark/bench-runner/Main.hs+19−59
@@ -16,11 +16,9 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
1616 benchName = drop 4 benchName0
1717
1818 general
19− -- GHC-9.6 requires 32M heap vs 16M for GHC-9.4
20− | "o-1-sp" `isInfixOf` benchName = "-K36K -M32M"
19+ | "o-1-sp" `isInfixOf` benchName = "-K36K -M16M"
2120 | "o-n-h" `isInfixOf` benchName = "-K36K -M32M"
22− -- GHC-9.6 requires 32M heap vs 16M for GHC-9.4
23− | "o-n-st" `isInfixOf` benchName = "-K1M -M32M"
21+ | "o-n-st" `isInfixOf` benchName = "-K1M -M16M"
2422 | "o-n-sp" `isInfixOf` benchName = "-K1M -M32M"
2523 | otherwise = ""
2624
@@ -29,17 +27,6 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
2927 | otherwise = ""
3028
3129 benchSpecific
32− -- GHC-9.6 requires 64M, earlier it was 16M
33− | "Data.Fold/o-n-heap.key-value.toHashMapIO (max buckets) sum" == benchName =
34− "-M64M"
35−
36− -- This is required only for the --long case because we allocate all
37− -- the arrays upfront. depends on the size of the stream.
38− | "Data.Parser/o-1-space" `isPrefixOf` benchName =
39− "-M128M"
40−
41− -----------------------------------------------------------------------
42−
4330 | "Prelude.Parallel/o-n-heap.mapping.mapM" == benchName = "-M256M"
4431 | "Prelude.Parallel/o-n-heap.monad-outer-product."
4532 `isPrefixOf` benchName = "-M256M"
@@ -60,28 +47,11 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
6047 | "Prelude.WAsync/o-n-space.monad-outer-product." `isPrefixOf` benchName =
6148 "-K4M"
6249
63− -- This module is dev only now, and can be removed at some point
64− | "Data.Stream.StreamDK/o-1-space.grouping.classifySessionsOf"
65− `isPrefixOf` benchName = "-K512K"
66− | "Data.Stream.StreamDK/o-n-space.foldr.foldrM/"
67− `isPrefixOf` benchName = "-K4M"
68− | "Data.Stream.StreamDK/o-n-space.iterated."
69− `isPrefixOf` benchName = "-K4M -M64M"
70− -- GHC 9.4.4 requires 4M
71− | "Data.Stream.StreamDK/o-n-space.traversable."
72− `isPrefixOf` benchName = "-K4M"
73− -- GHC-9.6 requires 64M, earlier it was 32M
74− | "Data.Stream.StreamDK/o-n-heap.buffered.readsPrec pure streams" == benchName =
75− "-M64M"
76− -- GHC-9.6 requires 64M, earlier it was 32M
77− | "Data.Stream.StreamDK/o-n-heap.buffered.showPrec Haskell lists" == benchName =
78− "-M64M"
79−
8050 -----------------------------------------------------------------------
8151
82− | "Data.StreamD/o-n-space.elimination.toList" == benchName =
52+ | "Data.Stream.StreamD/o-n-space.elimination.toList" == benchName =
8353 "-K2M"
84− | "Data.StreamK/o-n-space.elimination.toList" == benchName =
54+ | "Data.Stream.StreamK/o-n-space.elimination.toList" == benchName =
8555 "-K2M"
8656
8757 -----------------------------------------------------------------------
@@ -93,41 +63,31 @@ rtsOpts exeName benchName0 = unwords [general, exeSpecific, benchSpecific]
9363 | "Data.Stream/o-n-space.iterated."
9464 `isPrefixOf` benchName = "-K4M"
9565
96− -- GHC-9.6 requires 64M, earlier it was 32M
97− | "Data.Stream/o-n-heap.buffered.showPrec Haskell lists" == benchName =
98− "-M64M"
99− -- GHC-9.6 requires 64M, earlier it was 32M
100− | "Data.Stream/o-n-heap.buffered.readsPrec pure streams" == benchName =
101− "-M64M"
66+ | "Data.Stream.StreamDK/o-1-space.grouping.classifySessionsOf"
67+ `isPrefixOf` benchName = "-K512K"
68+ | "Data.Stream.StreamDK/o-n-space.foldr.foldrM/"
69+ `isPrefixOf` benchName = "-K4M"
70+ | "Data.Stream.StreamDK/o-n-space.iterated."
71+ `isPrefixOf` benchName = "-K4M -M64M"
72+ | "Data.Stream.StreamDK/o-n-space.traversable."
73+ `isPrefixOf` benchName = "-K2M"
74+
75+ | "Data.Stream.ConcurrentInterleaved/o-1-space.monad-outer-product.toNullAp"
76+ `isPrefixOf` benchName = "-M32M"
10277
103− | "Data.Stream.ConcurrentEager/o-n-heap.monad-outer-product.toNullAp"
104− `isPrefixOf` benchName = "-M1024M"
78+ | "Data.Stream.ConcurrentEager/o-1-space.monad-outer-product.toNullAp"
79+ `isPrefixOf` benchName = "-M512M"
10580 | "Data.Stream.ConcurrentEager/o-1-space."
10681 `isPrefixOf` benchName = "-M128M"
10782
108− | "Data.Stream.ConcurrentOrdered/o-1-space.concat-foldable.foldMapWith"
109− `isPrefixOf` benchName = "-K128K"
110−
11183 ----------------------------------------------------------------------
11284
11385 | "Data.Array" `isPrefixOf` benchName
11486 && "/o-1-space.generation.read" `isSuffixOf` benchName = "-M32M"
115− -- XXX GHC 9.6 onwards needs 64M, earlier it was 32M
11687 | "Data.Array" `isPrefixOf` benchName
117− && "/o-1-space.generation.show" `isSuffixOf` benchName = "-M64M"
118− -- XXX GHC 9.6 onwards needs 64M, earlier it was 32M
88+ && "/o-1-space.generation.show" `isSuffixOf` benchName = "-M32M"
11989 | "Data.Array.Generic/o-1-space.transformationX4.map"
120− `isPrefixOf` benchName = "-M64M"
121−
122− -- XXX For --long option, need to check why so much heap is required.
123− -- Note, if we remove the chunked stream module we need to keep the
124− -- chunked stream benchmarks in the stream module.
125− | "Data.Array.Stream/o-1-space"
126− `isPrefixOf` benchName = "-K4M -M512M"
127− -- XXX Takes up to 160MB heap for --long, we use chunked stream for
128− -- this, so the reason may be related to chunked streams.
129− | "Data.ParserK/o-1-space"
130− `isPrefixOf` benchName = "-K4M -M256M"
90+ `isPrefixOf` benchName = "-M32M"
13191
13292 -----------------------------------------------------------------------
13393
benchmark/bench-runner/cabal.project−3
@@ -1,4 +1 @@
11packages: . ../../targets
2−
3−package bench-report
4− flags: +no-charts
benchmark/bench-runner/default.nix+1−1
@@ -49,7 +49,7 @@ let haskellPackages =
4949 nixpkgs.haskell.lib.overrideCabal
5050 (let src = fetchGit {
5151 url = "git@github.com:composewell/bench-report.git";
52− rev = "3b50bf0ad06f746bca42f30c8d40a0426e159af3";
52+ rev = "ca3872acd4b817731b951c6ed91bfa6c58f26c5a";
5353 }; in super.callCabal2nix "bench-report" src {})
5454 (old:
5555 { enableLibraryProfiling = false;
benchmark/lib/Streamly/Benchmark/Common.hs+2−2
@@ -54,9 +54,9 @@ import Data.Functor.Identity (Identity, runIdentity)
5454import System.Random (randomRIO)
5555
5656import qualified Streamly.Internal.Data.Fold as Fold
57−import qualified Streamly.Internal.Data.Stream as S
57+import qualified Streamly.Internal.Data.Stream.StreamD as S
5858
59−import Test.Tasty.Bench
59+import Gauge
6060
6161-------------------------------------------------------------------------------
6262-- Benchmark Prefixes
benchmark/lib/Streamly/Benchmark/Common/Handle.hs+1−3
@@ -30,8 +30,6 @@ module Streamly.Benchmark.Common.Handle
3030 , isSpace
3131 , isSp
3232 , mkHandleBenchEnv
33− , Handles(..)
34− , getHandles
3533 )
3634where
3735
@@ -48,7 +46,7 @@ import System.Process (callCommand)
4846
4947import Data.IORef
5048import Prelude hiding (last, length)
51−import Test.Tasty.Bench hiding (env)
49+import Gauge hiding (env)
5250
5351
5452scratchDir :: String
benchmark/lib/Streamly/Benchmark/Prelude.hs+1−1
@@ -82,7 +82,7 @@ import qualified Streamly.Internal.Data.Stream.IsStream.Type as IsStream
8282import qualified Streamly.Internal.Data.Pipe as Pipe
8383import qualified Streamly.Internal.Data.Stream.Serial as Serial
8484
85−import Test.Tasty.Bench
85+import Gauge
8686import Streamly.Internal.Data.Time.Units
8787
8888-- Common polymorphic stream APIs used across multiple stream modules
benchmark/streamly-benchmarks.cabal+60−80
@@ -54,6 +54,11 @@ flag opt
5454 manual: True
5555 default: True
5656
57+flag use-gauge
58+ description: Use gauge instead of tasty-bench for benchmarking
59+ manual: True
60+ default: False
61+
5762flag use-streamly-core
5863 description: Benchmark only core package
5964 manual: True
@@ -140,10 +145,6 @@ common compile-options
140145 -Wno-missing-kind-signatures
141146 -Wno-operator-whitespace
142147
143− if impl(ghc >= 9.8)
144− ghc-options:
145− -Wno-missing-role-annotations
146−
147148 if flag(has-llvm)
148149 ghc-options: -fllvm
149150
@@ -151,17 +152,15 @@ common compile-options
151152 ghc-options: -Wmissed-specialisations
152153 -Wall-missed-specialisations
153154
155+ if flag(dev) || flag(debug)
156+ ghc-options: -fno-ignore-asserts
157+
154158common optimization-options
155159 if flag(opt)
156160 ghc-options: -O2
157161 -fdicts-strict
158162 -fspec-constr-recursive=16
159163 -fmax-worker-args=16
160−
161− -- For this to be effective it must come after the -O2 option
162− if flag(dev) || flag(debug)
163− ghc-options: -fno-ignore-asserts
164−
165164 if flag(fusion-plugin) && !impl(ghcjs) && !impl(ghc < 8.6)
166165 ghc-options: -fplugin Fusion.Plugin
167166 else
@@ -175,34 +174,43 @@ common bench-depends
175174 -- Core libraries shipped with ghc, the min and max
176175 -- constraints of these libraries should match with
177176 -- the GHC versions we support
178− base >= 4.9 && < 4.20
179− , deepseq >= 1.4.1 && < 1.6
180− , mtl >= 2.2 && < 2.3.2
177+ base >= 4.9 && < 4.18
178+ , deepseq >= 1.4.1 && < 1.5
179+ , mtl >= 2.2 && < 2.3
181180
182181 -- other libraries
183− , exceptions >= 0.8 && < 0.11
184182 , random >= 1.0 && < 2.0
185183 , transformers >= 0.4 && < 0.7
186− , containers >= 0.5 && < 0.8
184+ , containers >= 0.5 && < 0.7
187185 , hashable >= 1.3 && < 1.5
188186 , unordered-containers >= 0.2 && < 0.3
189187 , process >= 1.4 && < 1.7
190188 , directory >= 1.2.2 && < 1.4
191− , filepath >= 1.4.1 && < 1.6
192− , ghc-prim >= 0.4 && < 0.12
193− , tasty-bench >= 0.3 && < 0.4
194− , tasty >= 1.4.1 && < 1.6
195− , streamly-core
189+ , filepath >= 1.4.1 && < 1.5
190+ , ghc-prim >= 0.4 && < 0.10
196191
197− if !flag(use-streamly-core)
192+ if flag(use-streamly-core)
193+ build-depends: streamly-core
194+ else
198195 build-depends: streamly
196+ , streamly-core
197+
198+ if flag(use-gauge)
199+ build-depends: gauge >= 0.2.4 && < 0.3
200+ else
201+ build-depends: tasty-bench >= 0.3 && < 0.4
202+ , tasty >= 1.4.1 && < 1.5
203+ mixins: tasty-bench
204+ (Test.Tasty.Bench as Gauge
205+ , Test.Tasty.Bench as Gauge.Main
206+ )
199207
200208 if flag(fusion-plugin) && !impl(ghcjs) && !impl(ghc < 8.6)
201209 build-depends:
202210 fusion-plugin >= 0.2 && < 0.3
203211 if flag(inspection)
204− build-depends: template-haskell >= 2.14 && < 2.22
205− , inspection-testing >= 0.4 && < 0.6
212+ build-depends: template-haskell >= 2.14 && < 2.20
213+ , inspection-testing >= 0.4 && < 0.5
206214 -- Array uses a Storable constraint in dev build making several inspection
207215 -- tests fail
208216 if flag(dev) && flag(inspection)
@@ -269,7 +277,6 @@ benchmark Data.Stream
269277 else
270278 ghc-options: +RTS -M2500M -RTS
271279
272−-- Use StreamK functions where possible, converting to StreaK to StreamD
273280benchmark Data.Stream.StreamDK
274281 import: bench-options
275282 type: exitcode-stdio-1.0
@@ -348,34 +355,6 @@ benchmark Data.Stream.ConcurrentOrdered
348355 else
349356 buildable: True
350357
351−benchmark Data.Unbox
352− import: bench-options
353− type: exitcode-stdio-1.0
354− hs-source-dirs: .
355− cpp-options: -DUSE_UNBOX
356− main-is: Streamly/Benchmark/Data/Serialize.hs
357−
358−benchmark Data.Unbox.Derive.TH
359− import: bench-options
360− type: exitcode-stdio-1.0
361− hs-source-dirs: .
362− cpp-options: -DUSE_UNBOX
363− cpp-options: -DUSE_TH
364− main-is: Streamly/Benchmark/Data/Serialize.hs
365−
366−benchmark Data.Serialize
367− import: bench-options
368− type: exitcode-stdio-1.0
369− hs-source-dirs: .
370− cpp-options: -DUSE_TH
371− main-is: Streamly/Benchmark/Data/Serialize.hs
372− build-depends: QuickCheck, template-haskell
373− other-modules:
374− Streamly.Benchmark.Data.Serialize.TH
375− Streamly.Benchmark.Data.Serialize.RecCompatible
376− Streamly.Benchmark.Data.Serialize.RecNonCompatible
377− if flag(limit-build-mem)
378− ghc-options: +RTS -M1000M -RTS
379358
380359benchmark Data.Unfold
381360 import: bench-options
@@ -407,12 +386,12 @@ benchmark Data.Fold.Window
407386 else
408387 buildable: True
409388
410−benchmark Data.Parser
389+benchmark Data.Parser.ParserD
411390 import: bench-options
412391 type: exitcode-stdio-1.0
413− hs-source-dirs: Streamly/Benchmark/Data
414− main-is: Parser.hs
415− if impl(ghcjs)
392+ hs-source-dirs: Streamly/Benchmark/Data/Parser
393+ main-is: ParserD.hs
394+ if flag(use-streamly-core) || impl(ghcjs)
416395 buildable: False
417396 else
418397 buildable: True
@@ -421,49 +400,56 @@ benchmark Data.Parser
421400 if flag(dev)
422401 ghc-options: +RTS -M3000M -RTS
423402 else
424− ghc-options: +RTS -M2000M -RTS
403+ ghc-options: +RTS -M750M -RTS
425404
426−benchmark Data.ParserK
405+benchmark Data.Parser.ParserK
427406 import: bench-options
428407 type: exitcode-stdio-1.0
429− hs-source-dirs: Streamly/Benchmark/Data
430− cpp-options: -DBENCH_SINGULAR
408+ hs-source-dirs: Streamly/Benchmark/Data/Parser
431409 main-is: ParserK.hs
432− if impl(ghcjs)
410+ if flag(use-streamly-core) || impl(ghcjs)
433411 buildable: False
434412 else
435413 buildable: True
436414 build-depends: exceptions >= 0.8 && < 0.11
437415
438−benchmark Data.ParserK.Chunked
416+-- Note: to use this we have to set DISABLE_FUSION in ParserD.hs so that
417+-- the rewrite rules are disabled. We can also use the "no-fusion" build
418+-- flag but we need to keep in mind that it disables fusion for streams
419+-- as well.
420+benchmark Data.Parser.FromParserK
439421 import: bench-options
440422 type: exitcode-stdio-1.0
441− hs-source-dirs: Streamly/Benchmark/Data
442− cpp-options: -DBENCH_CHUNKED
423+ hs-source-dirs: Streamly/Benchmark/Data/Parser
424+ cpp-options: -DFROM_PARSERK
443425 main-is: ParserK.hs
444− if impl(ghcjs)
426+ if flag(use-streamly-core) || impl(ghcjs)
445427 buildable: False
446428 else
447429 buildable: True
448430 build-depends: exceptions >= 0.8 && < 0.11
449431
450−benchmark Data.ParserK.Chunked.Generic
432+benchmark Data.Parser
451433 import: bench-options
452434 type: exitcode-stdio-1.0
453435 hs-source-dirs: Streamly/Benchmark/Data
454− cpp-options: -DBENCH_CHUNKED_GENERIC
455− main-is: ParserK.hs
456− if impl(ghcjs)
436+ main-is: Parser.hs
437+ if flag(use-streamly-core) || impl(ghcjs)
457438 buildable: False
458439 else
459440 buildable: True
460441 build-depends: exceptions >= 0.8 && < 0.11
442+ if flag(limit-build-mem)
443+ if flag(dev)
444+ ghc-options: +RTS -M3000M -RTS
445+ else
446+ ghc-options: +RTS -M1000M -RTS
461447
462448-------------------------------------------------------------------------------
463449-- Raw Streams
464450-------------------------------------------------------------------------------
465451
466−benchmark Data.StreamD
452+benchmark Data.Stream.StreamD
467453 import: bench-options
468454 type: exitcode-stdio-1.0
469455 hs-source-dirs: Streamly/Benchmark/Data/Stream
@@ -471,7 +457,7 @@ benchmark Data.StreamD
471457 if impl(ghcjs)
472458 buildable: False
473459
474−benchmark Data.StreamK
460+benchmark Data.Stream.StreamK
475461 import: bench-options
476462 type: exitcode-stdio-1.0
477463 hs-source-dirs: Streamly/Benchmark/Data/Stream
@@ -487,11 +473,11 @@ benchmark Data.Stream.ToStreamK
487473 if !flag(dev) || impl(ghcjs)
488474 buildable: False
489475
490−benchmark Data.StreamK.Alt
476+benchmark Data.Stream.StreamK.Alt
491477 import: bench-options
492478 type: exitcode-stdio-1.0
493479 hs-source-dirs: Streamly/Benchmark/Data/Stream
494− main-is: StreamKAlt.hs
480+ main-is: StreamDK.hs
495481 if !flag(dev) || impl(ghcjs)
496482 buildable: False
497483
@@ -680,10 +666,10 @@ benchmark Data.Array
680666 if flag(limit-build-mem)
681667 ghc-options: +RTS -M1500M -RTS
682668
683−benchmark Data.MutArray
669+benchmark Data.Array.Mut
684670 import: bench-options
685671 type: exitcode-stdio-1.0
686− main-is: Streamly/Benchmark/Data/MutArray.hs
672+ main-is: Streamly/Benchmark/Data/Array/Mut.hs
687673 if flag(limit-build-mem)
688674 ghc-options: +RTS -M1500M -RTS
689675 if flag(use-streamly-core)
@@ -770,9 +756,3 @@ benchmark Unicode.Utf8
770756 buildable: False
771757 else
772758 buildable: True
773−
774−benchmark Unicode.Parser
775− import: bench-options
776− type: exitcode-stdio-1.0
777− hs-source-dirs: Streamly/Benchmark/Unicode
778− main-is: Parser.hs
bin/mk-hscope.sh+1−1
@@ -1,7 +1,7 @@
11#!/usr/bin/env bash
22
33GHC_VERSION=8.10.4
4−STREAMLY_VERSION=0.9.0
4+STREAMLY_VERSION=0.8.2
55
66case `uname` in
77 Darwin) SYSTEM=x86_64-osx;;
bin/run-repl-quick.sh+1−1
@@ -13,7 +13,7 @@ set -e
1313
1414SCRIPT_DIR=$(dirname $0)
1515
16−STREAMLY_VERSION=0.9.0
16+STREAMLY_VERSION=0.8.2
1717BENCH_REPORT_DIR=benchmark/bench-report
1818source $SCRIPT_DIR/targets.sh
1919
cabal.project.packdiff+16−2
@@ -1,9 +1,23 @@
1+packages:
2+ streamly.cabal
3+ core/streamly-core.cabal
4+
15source-repository-package
26 type: git
37 location: https://github.com/composewell/packdiff.git
4− tag: b11a50c9c0523382a19eebc0b12efdae194406a9
8+ tag: 98201422ea62974400a910331928d71a5fc90801
9+
10+source-repository-package
11+ type: git
12+ location: https://github.com/composewell/streamly-shell.git
13+ tag: b0c6318119d1cc5e2106343f17b5cff2ad94417f
14+
15+source-repository-package
16+ type: git
17+ location: https://github.com/composewell/streamly-coreutils.git
18+ tag: 5dbb8bcefac3aefd5eb298dc410b11e36e79f35a
519
620source-repository-package
721 type: git
822 location: https://github.com/composewell/streamly-process.git
9− tag: d80b860d9d8ea98e4f7f63390442b3155c34dd08
23+ tag: 794a4102f82a13a94c66f6453c20703a5296229c
cabal.project.report+4−2
@@ -2,6 +2,8 @@ packages:
22 targets/streamly-targets.cabal
33 benchmark/bench-runner/bench-runner.cabal
44 test/test-runner/test-runner.cabal
5+ streamly.cabal
6+ core/streamly-core.cabal
57
68package bench-report
79 flags: +no-charts
@@ -12,7 +14,7 @@ package bench-show
1214source-repository-package
1315 type: git
1416 location: https://github.com/composewell/bench-report.git
15− tag: 3b50bf0ad06f746bca42f30c8d40a0426e159af3
17+ tag: 11713fde39d9973d8210e84de801f250a05a6735
1618
1719source-repository-package
1820 type: git
@@ -27,4 +29,4 @@ source-repository-package
2729source-repository-package
2830 type: git
2931 location: https://github.com/composewell/streamly-process.git
30− tag: d80b860d9d8ea98e4f7f63390442b3155c34dd08
32+ tag: 794a4102f82a13a94c66f6453c20703a5296229c
configure+9−9
@@ -1,6 +1,6 @@
11#! /bin/sh
22# Guess values for system-dependent variables and create Makefiles.
3−# Generated by GNU Autoconf 2.71 for streamly 0.10.1.
3+# Generated by GNU Autoconf 2.71 for streamly 0.9.0.
44#
55# Report bugs to <streamly@composewell.com>.
66#
@@ -610,8 +610,8 @@ MAKEFLAGS=
610610# Identity of this package.
611611PACKAGE_NAME='streamly'
612612PACKAGE_TARNAME='streamly'
613−PACKAGE_VERSION='0.10.1'
614−PACKAGE_STRING='streamly 0.10.1'
613+PACKAGE_VERSION='0.9.0'
614+PACKAGE_STRING='streamly 0.9.0'
615615PACKAGE_BUGREPORT='streamly@composewell.com'
616616PACKAGE_URL='https://streamly.composewell.com'
617617
@@ -1224,7 +1224,7 @@ if test "$ac_init_help" = "long"; then
12241224 # Omit some internal or obsolete options to make the list less imposing.
12251225 # This message is too long to be a string in the A/UX 3.1 sh.
12261226 cat <<_ACEOF
1227−\`configure' configures streamly 0.10.1 to adapt to many kinds of systems.
1227+\`configure' configures streamly 0.9.0 to adapt to many kinds of systems.
12281228
12291229Usage: $0 [OPTION]... [VAR=VALUE]...
12301230
@@ -1286,7 +1286,7 @@ fi
12861286
12871287if test -n "$ac_init_help"; then
12881288 case $ac_init_help in
1289− short | recursive ) echo "Configuration of streamly 0.10.1:";;
1289+ short | recursive ) echo "Configuration of streamly 0.9.0:";;
12901290 esac
12911291 cat <<\_ACEOF
12921292
@@ -1372,7 +1372,7 @@ fi
13721372test -n "$ac_init_help" && exit $ac_status
13731373if $ac_init_version; then
13741374 cat <<\_ACEOF
1375−streamly configure 0.10.1
1375+streamly configure 0.9.0
13761376generated by GNU Autoconf 2.71
13771377
13781378Copyright (C) 2021 Free Software Foundation, Inc.
@@ -1500,7 +1500,7 @@ cat >config.log <<_ACEOF
15001500This file contains any messages produced by compilers while
15011501running configure, to aid debugging if configure makes a mistake.
15021502
1503−It was created by streamly $as_me 0.10.1, which was
1503+It was created by streamly $as_me 0.9.0, which was
15041504generated by GNU Autoconf 2.71. Invocation command line was
15051505
15061506 $ $0$ac_configure_args_raw
@@ -3800,7 +3800,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
38003800# report actual input values of CONFIG_FILES etc. instead of their
38013801# values after options handling.
38023802ac_log="
3803−This file was extended by streamly $as_me 0.10.1, which was
3803+This file was extended by streamly $as_me 0.9.0, which was
38043804generated by GNU Autoconf 2.71. Invocation command line was
38053805
38063806 CONFIG_FILES = $CONFIG_FILES
@@ -3856,7 +3856,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\
38563856cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
38573857ac_cs_config='$ac_cs_config_escaped'
38583858ac_cs_version="\\
3859−streamly config.status 0.10.1
3859+streamly config.status 0.9.0
38603860configured by $0, generated by GNU Autoconf 2.71,
38613861 with options \\"\$ac_cs_config\\"
38623862
configure.ac+1−1
@@ -3,7 +3,7 @@
33# See https://www.gnu.org/software/autoconf/manual/autoconf.html for help on
44# the macros used in this file.
55
6−AC_INIT([streamly], [0.10.1], [streamly@composewell.com], [streamly], [https://streamly.composewell.com])
6+AC_INIT([streamly], [0.9.0], [streamly@composewell.com], [streamly], [https://streamly.composewell.com])
77
88# To suppress "WARNING: unrecognized options: --with-compiler"
99AC_ARG_WITH([compiler], [GHC])
core/Changelog.mddeleted−1
@@ -1 +0,0 @@
1−docs/Changelog.md
core/LICENSEdeleted−322
@@ -1,322 +0,0 @@
1−Copyright (c) 2017, Composewell Technologies
2−All rights reserved.
3−
4−Redistribution and use in source and binary forms, with or without
5−modification, are permitted provided that the following conditions are met:
6−
7−1. Redistributions of source code must retain the above copyright notice, this
8−list of conditions and the following disclaimer.
9−
10−2. Redistributions in binary form must reproduce the above copyright notice,
11−this list of conditions and the following disclaimer in the documentation
12−and/or other materials provided with the distribution.
13−
14−3. Neither the name of the copyright holder nor the names of its contributors
15−may be used to endorse or promote products derived from this software without
16−specific prior written permission.
17−
18−THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
19−ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
20−WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
21−DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
22−FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
23−DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
24−SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
25−CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
26−OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
27−OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
28−
29−-------------------------------------------------------------------------------
30−base-4.12.0.0 (http://hackage.haskell.org/package/base-4.12.0.0)
31−-------------------------------------------------------------------------------
32−This library (libraries/base) is derived from code from several
33−sources:
34−
35− * Code from the GHC project which is largely (c) The University of
36− Glasgow, and distributable under a BSD-style license (see below),
37−
38− * Code from the Haskell 98 Report which is (c) Simon Peyton Jones
39− and freely redistributable (but see the full license for
40− restrictions).
41−
42− * Code from the Haskell Foreign Function Interface specification,
43− which is (c) Manuel M. T. Chakravarty and freely redistributable
44− (but see the full license for restrictions).
45−
46−The full text of these licenses is reproduced below. All of the
47−licenses are BSD-style or compatible.
48−
49−The Glasgow Haskell Compiler License
50−
51−Copyright 2004, The University Court of the University of Glasgow.
52−All rights reserved.
53−
54−Redistribution and use in source and binary forms, with or without
55−modification, are permitted provided that the following conditions are met:
56−
57−- Redistributions of source code must retain the above copyright notice,
58−this list of conditions and the following disclaimer.
59−
60−- Redistributions in binary form must reproduce the above copyright notice,
61−this list of conditions and the following disclaimer in the documentation
62−and/or other materials provided with the distribution.
63−
64−- Neither name of the University nor the names of its contributors may be
65−used to endorse or promote products derived from this software without
66−specific prior written permission.
67−
68−THIS SOFTWARE IS PROVIDED BY THE UNIVERSITY COURT OF THE UNIVERSITY OF
69−GLASGOW AND THE CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
70−INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
71−FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
72−UNIVERSITY COURT OF THE UNIVERSITY OF GLASGOW OR THE CONTRIBUTORS BE LIABLE
73−FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
74−DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
75−SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
76−CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
77−LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
78−OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
79−DAMAGE.
80−
81−Code derived from the document "Report on the Programming Language
82−Haskell 98", is distributed under the following license:
83−
84− Copyright (c) 2002 Simon Peyton Jones
85−
86− The authors intend this Report to belong to the entire Haskell
87− community, and so we grant permission to copy and distribute it for
88− any purpose, provided that it is reproduced in its entirety,
89− including this Notice. Modified versions of this Report may also be
90− copied and distributed for any purpose, provided that the modified
91− version is clearly presented as such, and that it does not claim to
92− be a definition of the Haskell 98 Language.
93−
94−Code derived from the document "The Haskell 98 Foreign Function
95−Interface, An Addendum to the Haskell 98 Report" is distributed under
96−the following license:
97−
98− Copyright (c) 2002 Manuel M. T. Chakravarty
99−
100− The authors intend this Report to belong to the entire Haskell
101− community, and so we grant permission to copy and distribute it for
102− any purpose, provided that it is reproduced in its entirety,
103− including this Notice. Modified versions of this Report may also be
104− copied and distributed for any purpose, provided that the modified
105− version is clearly presented as such, and that it does not claim to
106− be a definition of the Haskell 98 Foreign Function Interface.
107−
108−-------------------------------------------------------------------------------
109−bjoern-2008-2009 (http://bjoern.hoehrmann.de/utf-8/decoder/dfa/)
110−-------------------------------------------------------------------------------
111−Copyright (c) 2008-2009 Bjoern Hoehrmann <bjoern@hoehrmann.de>
112−
113−Permission is hereby granted, free of charge, to any person obtaining a copy of
114−this software and associated documentation files (the "Software"), to deal in
115−the Software without restriction, including without limitation the rights to
116−use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
117−of the Software, and to permit persons to whom the Software is furnished to do
118−so, subject to the following conditions:
119−
120−The above copyright notice and this permission notice shall be included in all
121−copies or substantial portions of the Software.
122−
123−THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
124−IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
125−FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
126−AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
127−LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
128−OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
129−SOFTWARE.
130−
131−-------------------------------------------------------------------------------
132−clock-0.7.2 (http://hackage.haskell.org/package/clock-0.7.2)
133−-------------------------------------------------------------------------------
134−Copyright (c) 2009-2012, Cetin Sert
135−Copyright (c) 2010, Eugene Kirpichov
136−
137−All rights reserved.
138−
139−Redistribution and use in source and binary forms, with or without
140−modification, are permitted provided that the following conditions are
141−met:
142−
143− * Redistributions of source code must retain the above copyright
144− notice, this list of conditions and the following disclaimer.
145−
146− * Redistributions in binary form must reproduce the above
147− copyright notice, this list of conditions and the following
148− disclaimer in the documentation and/or other materials provided
149− with the distribution.
150−
151− * The names of contributors may not be used to endorse or promote
152− products derived from this software without specific prior
153− written permission.
154−
155−THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
156−"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
157−LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
158−A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
159−OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
160−SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
161−LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
162−DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
163−THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
164−(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
165−OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
166−
167−-------------------------------------------------------------------------------
168−foldl-1.4.5 (http://hackage.haskell.org/package/foldl-1.4.5)
169−-------------------------------------------------------------------------------
170−Copyright (c) 2013 Gabriel Gonzalez
171−All rights reserved.
172−
173−Redistribution and use in source and binary forms, with or without modification,
174−are permitted provided that the following conditions are met:
175− * Redistributions of source code must retain the above copyright notice,
176− this list of conditions and the following disclaimer.
177− * Redistributions in binary form must reproduce the above copyright notice,
178− this list of conditions and the following disclaimer in the documentation
179− and/or other materials provided with the distribution.
180− * Neither the name of Gabriel Gonzalez nor the names of other contributors
181− may be used to endorse or promote products derived from this software
182− without specific prior written permission.
183−
184−THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
185−ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
186−WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
187−DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
188−ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
189−(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
190−LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
191−ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
192−(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
193−SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
194−
195−-------------------------------------------------------------------------------
196−primitive-0.7.0.0 (https://hackage.haskell.org/package/primitive-0.7.0.0)
197−-------------------------------------------------------------------------------
198−Copyright (c) 2008-2009, Roman Leshchinskiy
199−All rights reserved.
200−
201−Redistribution and use in source and binary forms, with or without
202−modification, are permitted provided that the following conditions are met:
203−
204−- Redistributions of source code must retain the above copyright notice,
205−this list of conditions and the following disclaimer.
206−
207−- Redistributions in binary form must reproduce the above copyright notice,
208−this list of conditions and the following disclaimer in the documentation
209−and/or other materials provided with the distribution.
210−
211−- Neither name of the University nor the names of its contributors may be
212−used to endorse or promote products derived from this software without
213−specific prior written permission.
214−
215−THIS SOFTWARE IS PROVIDED BY THE UNIVERSITY COURT OF THE UNIVERSITY OF
216−GLASGOW AND THE CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
217−INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
218−FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
219−UNIVERSITY COURT OF THE UNIVERSITY OF GLASGOW OR THE CONTRIBUTORS BE LIABLE
220−FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
221−DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
222−SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
223−CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
224−LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
225−OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
226−DAMAGE.
227−
228−-------------------------------------------------------------------------------
229−transient-0.5.5 (http://hackage.haskell.org/package/transient-0.5.5)
230−-------------------------------------------------------------------------------
231−Copyright © 2014-2016 Alberto G. Corona <https://github.com/agocorona>
232−
233−Permission is hereby granted, free of charge, to any person obtaining a copy of
234−this software and associated documentation files (the "Software"), to deal in
235−the Software without restriction, including without limitation the rights to
236−use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
237−the Software, and to permit persons to whom the Software is furnished to do so,
238−subject to the following conditions:
239−
240−The above copyright notice and this permission notice shall be included in all
241−copies or substantial portions of the Software.
242−
243−THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
244−IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
245−FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
246−COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
247−IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
248−CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
249−
250−-------------------------------------------------------------------------------
251−vector-0.12.0.2 (http://hackage.haskell.org/package/vector-0.12.0.2)
252−-------------------------------------------------------------------------------
253−Copyright (c) 2008-2012, Roman Leshchinskiy
254−All rights reserved.
255−
256−Redistribution and use in source and binary forms, with or without
257−modification, are permitted provided that the following conditions are met:
258−
259−- Redistributions of source code must retain the above copyright notice,
260−this list of conditions and the following disclaimer.
261−
262−- Redistributions in binary form must reproduce the above copyright notice,
263−this list of conditions and the following disclaimer in the documentation
264−and/or other materials provided with the distribution.
265−
266−- Neither name of the University nor the names of its contributors may be
267−used to endorse or promote products derived from this software without
268−specific prior written permission.
269−
270−THIS SOFTWARE IS PROVIDED BY THE UNIVERSITY COURT OF THE UNIVERSITY OF
271−GLASGOW AND THE CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
272−INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
273−FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
274−UNIVERSITY COURT OF THE UNIVERSITY OF GLASGOW OR THE CONTRIBUTORS BE LIABLE
275−FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
276−DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
277−SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
278−CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
279−LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
280−OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
281−DAMAGE.
282−
283−-------------------------------------------------------------------------------
284−th-utilities-0.2.5.0 (https://hackage.haskell.org/package/th-utilities)
285−-------------------------------------------------------------------------------
286−Copyright (c) 2016 FP Complete Corporation.
287−
288−Permission is hereby granted, free of charge, to any person obtaining
289−a copy of this software and associated documentation files (the
290−"Software"), to deal in the Software without restriction, including
291−without limitation the rights to use, copy, modify, merge, publish,
292−distribute, sublicense, and/or sell copies of the Software, and to
293−permit persons to whom the Software is furnished to do so, subject to
294−the following conditions:
295−
296−The above copyright notice and this permission notice shall be
297−included in all copies or substantial portions of the Software.
298−
299−THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
300−EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
301−MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
302−NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
303−LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
304−OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
305−WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
306−
307−-------------------------------------------------------------------------------
308−th-abstraction-0.5.0.0 (https://hackage.haskell.org/package/th-abstraction)
309−-------------------------------------------------------------------------------
310−Copyright (c) 2017-2020 Eric Mertens
311−
312−Permission to use, copy, modify, and/or distribute this software for any purpose
313−with or without fee is hereby granted, provided that the above copyright notice
314−and this permission notice appear in all copies.
315−
316−THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
317−REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
318−FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
319−INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
320−OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
321−TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
322−THIS SOFTWARE.
core/configure+9−9
@@ -1,6 +1,6 @@
11#! /bin/sh
22# Guess values for system-dependent variables and create Makefiles.
3−# Generated by GNU Autoconf 2.71 for streamly-core 0.2.2.
3+# Generated by GNU Autoconf 2.71 for streamly-core 0.8.2.
44#
55# Report bugs to <streamly@composewell.com>.
66#
@@ -610,8 +610,8 @@ MAKEFLAGS=
610610# Identity of this package.
611611PACKAGE_NAME='streamly-core'
612612PACKAGE_TARNAME='streamly-core'
613−PACKAGE_VERSION='0.2.2'
614−PACKAGE_STRING='streamly-core 0.2.2'
613+PACKAGE_VERSION='0.8.2'
614+PACKAGE_STRING='streamly-core 0.8.2'
615615PACKAGE_BUGREPORT='streamly@composewell.com'
616616PACKAGE_URL='https://streamly.composewell.com'
617617
@@ -1256,7 +1256,7 @@ if test "$ac_init_help" = "long"; then
12561256 # Omit some internal or obsolete options to make the list less imposing.
12571257 # This message is too long to be a string in the A/UX 3.1 sh.
12581258 cat <<_ACEOF
1259−\`configure' configures streamly-core 0.2.2 to adapt to many kinds of systems.
1259+\`configure' configures streamly-core 0.8.2 to adapt to many kinds of systems.
12601260
12611261Usage: $0 [OPTION]... [VAR=VALUE]...
12621262
@@ -1318,7 +1318,7 @@ fi
13181318
13191319if test -n "$ac_init_help"; then
13201320 case $ac_init_help in
1321− short | recursive ) echo "Configuration of streamly-core 0.2.2:";;
1321+ short | recursive ) echo "Configuration of streamly-core 0.8.2:";;
13221322 esac
13231323 cat <<\_ACEOF
13241324
@@ -1404,7 +1404,7 @@ fi
14041404test -n "$ac_init_help" && exit $ac_status
14051405if $ac_init_version; then
14061406 cat <<\_ACEOF
1407−streamly-core configure 0.2.2
1407+streamly-core configure 0.8.2
14081408generated by GNU Autoconf 2.71
14091409
14101410Copyright (C) 2021 Free Software Foundation, Inc.
@@ -1622,7 +1622,7 @@ cat >config.log <<_ACEOF
16221622This file contains any messages produced by compilers while
16231623running configure, to aid debugging if configure makes a mistake.
16241624
1625−It was created by streamly-core $as_me 0.2.2, which was
1625+It was created by streamly-core $as_me 0.8.2, which was
16261626generated by GNU Autoconf 2.71. Invocation command line was
16271627
16281628 $ $0$ac_configure_args_raw
@@ -3832,7 +3832,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
38323832# report actual input values of CONFIG_FILES etc. instead of their
38333833# values after options handling.
38343834ac_log="
3835−This file was extended by streamly-core $as_me 0.2.2, which was
3835+This file was extended by streamly-core $as_me 0.8.2, which was
38363836generated by GNU Autoconf 2.71. Invocation command line was
38373837
38383838 CONFIG_FILES = $CONFIG_FILES
@@ -3888,7 +3888,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\
38883888cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
38893889ac_cs_config='$ac_cs_config_escaped'
38903890ac_cs_version="\\
3891−streamly-core config.status 0.2.2
3891+streamly-core config.status 0.8.2
38923892configured by $0, generated by GNU Autoconf 2.71,
38933893 with options \\"\$ac_cs_config\\"
38943894
core/configure.ac+1−1
@@ -3,7 +3,7 @@
33# See https://www.gnu.org/software/autoconf/manual/autoconf.html for help on
44# the macros used in this file.
55
6−AC_INIT([streamly-core], [0.2.2], [streamly@composewell.com], [streamly-core], [https://streamly.composewell.com])
6+AC_INIT([streamly-core], [0.8.2], [streamly@composewell.com], [streamly-core], [https://streamly.composewell.com])
77
88# To suppress "WARNING: unrecognized options: --with-compiler"
99AC_ARG_WITH([compiler], [GHC])
core/docs/ApiChangelogs/0.1.0-0.2.0.txtdeleted−1471
@@ -1,1471 +0,0 @@
1−---------------------------------
2−API Annotations
3−---------------------------------
4−
5−[A] : Added
6−[R] : Removed
7−[C] : Changed
8−[O] : Old definition
9−[N] : New definition
10−[D] : Deprecated
11−
12−---------------------------------
13−API diff
14−---------------------------------
15−
16−[C] Streamly.Unicode.Parser
17− [A] double :: Monad m => Parser Char m Double
18−[C] Streamly.FileSystem.Handle
19− [A] readWith :: MonadIO m => Int -> Handle -> Stream m Word8
20− [A] readChunksWith :: MonadIO m => Int -> Handle -> Stream m (Array Word8)
21−[C] Streamly.FileSystem.Dir
22− [C] readEither
23− [O] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
24− [N] readEither :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
25− [C] read
26− [O] read :: MonadIO m => FilePath -> Stream m FilePath
27− [N] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
28−[C] Streamly.Data.Unfold
29− [A] class Enum a => Enumerable a
30− [A] second :: b -> Unfold m (a, b) c -> Unfold m a c
31− [A] first :: a -> Unfold m (a, b) c -> Unfold m b c
32− [A] enumerateFromTo :: (Enumerable a, Monad m) => Unfold m (a, a) a
33− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => Unfold m (a, a, a) a
34− [A] enumerateFromThen :: (Enumerable a, Monad m) => Unfold m (a, a) a
35− [A] enumerateFrom :: (Enumerable a, Monad m) => Unfold m a a
36−[C] Streamly.Data.StreamK
37− [C] parseChunks
38− [O] parseChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b)
39− [N] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
40− [C] parseBreakChunks
41− [O] parseBreakChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
42− [N] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
43− [A] parseBreak :: forall m a b. Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
44− [A] parse :: Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b)
45− [A] handle :: (MonadCatch m, Exception e) => (e -> m (StreamK m a)) -> StreamK m a -> StreamK m a
46− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> StreamK m a) -> StreamK m a
47−[C] Streamly.Data.Stream
48− [A] wordsBy :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
49− [A] splitOn :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
50− [C] handle
51− [O] handle :: (MonadCatch m, Exception e) => (e -> Stream m a) -> Stream m a -> Stream m a
52− [N] handle :: (MonadCatch m, Exception e) => (e -> m (Stream m a)) -> Stream m a -> Stream m a
53− [A] groupsOf :: Monad m => Int -> Fold m a b -> Stream m a -> Stream m b
54−[C] Streamly.Data.ParserK
55− [D] fromParser :: (MonadIO m, Unbox a) => Parser a m b -> ParserK (Array a) m b
56− [D] fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK (Array a) m b
57− [A] adaptCG :: Monad m => Parser a m b -> ParserK (Array a) m b
58− [A] adaptC :: (Monad m, Unbox a) => Parser a m b -> ParserK (Array a) m b
59− [A] adapt :: Monad m => Parser a m b -> ParserK a m b
60−[C] Streamly.Data.Parser
61− [A] groupByRollingEither :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
62− [A] groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
63−[A] Streamly.Data.MutByteArray
64− [A] class Unbox a
65− [A] class Serialize a
66− [A] SerializeConfig
67− [A] MutByteArray
68− [A] unpin :: MutByteArray -> IO MutByteArray
69− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
70− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
71− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
72− [A] pinnedNew :: Int -> IO MutByteArray
73− [A] pin :: MutByteArray -> IO MutByteArray
74− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
75− [A] new :: Int -> IO MutByteArray
76− [A] isPinned :: MutByteArray -> Bool
77− [A] inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
78− [A] inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
79− [A] inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig
80− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
81− [A] deriveUnbox :: Q [Dec] -> Q [Dec]
82− [A] deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]
83− [A] deriveSerialize :: Q [Dec] -> Q [Dec]
84− [A] addSizeTo :: Serialize a => Int -> a -> Int
85−[C] Streamly.Data.MutArray.Generic
86− [A] write :: MonadIO m => Fold m a (MutArray a)
87− [A] readRev :: MonadIO m => MutArray a -> Stream m a
88− [A] read :: MonadIO m => MutArray a -> Stream m a
89− [A] putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
90− [A] modifyIndexUnsafe :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
91− [A] length :: MutArray a -> Int
92− [A] getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
93− [C] getIndex
94− [O] getIndex :: MonadIO m => Int -> MutArray a -> m a
95− [N] getIndex :: MonadIO m => Int -> MutArray a -> m (Maybe a)
96− [A] fromListN :: MonadIO m => Int -> [a] -> m (MutArray a)
97− [A] fromList :: MonadIO m => [a] -> m (MutArray a)
98−[C] Streamly.Data.MutArray
99− [A] unpin :: MutArray a -> IO (MutArray a)
100− [A] readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
101− [A] read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
102− [A] putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
103− [D] pokeByteIndex :: Unbox a => Int -> MutByteArray -> a -> IO ()
104− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
105− [A] pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
106− [A] pin :: MutArray a -> IO (MutArray a)
107− [D] peekByteIndex :: Unbox a => Int -> MutByteArray -> IO a
108− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
109− [D] newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
110− [A] modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
111− [A] modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
112− [A] modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()
113− [A] isPinned :: MutArray a -> Bool
114− [A] getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
115− [C] getIndex
116− [O] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
117− [N] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
118−[C] Streamly.Data.Array.Generic
119− [A] toList :: Array a -> [a]
120− [A] getIndex :: Int -> Array a -> Maybe a
121−[C] Streamly.Data.Array
122− [A] unpin :: Array a -> IO (Array a)
123− [D] pokeByteIndex :: Unbox a => Int -> MutByteArray -> a -> IO ()
124− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
125− [A] pin :: Array a -> IO (Array a)
126− [D] peekByteIndex :: Unbox a => Int -> MutByteArray -> IO a
127− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
128− [A] isPinned :: Array a -> Bool
129−[C] Streamly.Console.Stdio
130−
131−---------------------------------
132−Internal API diff
133−---------------------------------
134−
135−[C] Streamly.Internal.Unicode.Stream
136− [C] writeCharUtf8'
137− [O] writeCharUtf8' :: Monad m => Fold m Word8 Char
138− [N] writeCharUtf8' :: Monad m => Parser Word8 m Char
139−[C] Streamly.Internal.Unicode.Parser
140− [A] number :: Monad m => Parser Char m (Integer, Int)
141− [A] mkDouble :: Integer -> Int -> Double
142− [A] doubleParser :: Monad m => Parser Char m (Int, Int)
143− [C] double
144− [O] double :: Parser Char m Double
145− [N] double :: Monad m => Parser Char m Double
146−[R] Streamly.Internal.Serialize.ToBytes
147−[R] Streamly.Internal.Serialize.FromBytes
148−[C] Streamly.Internal.FileSystem.File
149− [A] writeAppendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()
150− [A] writeAppendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()
151− [A] writeAppendArray :: FilePath -> Array a -> IO ()
152− [A] writeAppend :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()
153− [R] appendWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m Word8 -> m ()
154− [R] appendChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array a) -> m ()
155− [R] appendArray :: FilePath -> Array a -> IO ()
156− [R] append :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8 -> m ()
157−[C] Streamly.Internal.FileSystem.Dir
158− [C] reader
159− [O] reader :: MonadIO m => Unfold m FilePath FilePath
160− [N] reader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
161− [C] readFiles
162− [O] readFiles :: MonadIO m => FilePath -> Stream m FilePath
163− [N] readFiles :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
164− [C] readEitherPaths
165− [O] readEitherPaths :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
166− [N] readEitherPaths :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
167− [C] readEither
168− [O] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
169− [N] readEither :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Either FilePath FilePath)
170− [C] readDirs
171− [O] readDirs :: MonadIO m => FilePath -> Stream m FilePath
172− [N] readDirs :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
173− [C] read
174− [O] read :: MonadIO m => FilePath -> Stream m FilePath
175− [N] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m FilePath
176− [C] fileReader
177− [O] fileReader :: MonadIO m => Unfold m FilePath FilePath
178− [N] fileReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
179− [C] eitherReaderPaths
180− [O] eitherReaderPaths :: MonadIO m => Unfold m FilePath (Either FilePath FilePath)
181− [N] eitherReaderPaths :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Either FilePath FilePath)
182− [C] eitherReader
183− [O] eitherReader :: MonadIO m => Unfold m FilePath (Either FilePath FilePath)
184− [N] eitherReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath (Either FilePath FilePath)
185− [C] dirReader
186− [O] dirReader :: MonadIO m => Unfold m FilePath FilePath
187− [N] dirReader :: (MonadIO m, MonadCatch m) => Unfold m FilePath FilePath
188−[R] Streamly.Internal.Data.Unfold.Type
189−[R] Streamly.Internal.Data.Unfold.Enumeration
190−[C] Streamly.Internal.Data.Unfold
191− [C] Unfold
192− [A] Unfold :: (s -> m (Step s b)) -> (a -> m s) -> Unfold m a b
193− [A] takeWhileMWithInput :: Monad m => (a -> b -> m Bool) -> Unfold m a b -> Unfold m a b
194− [A] manyInterleave :: Monad m => Unfold m a b -> Unfold m c a -> Unfold m c b
195− [A] enumerateFromStepIntegral :: (Monad m, Integral a) => Unfold m (a, a) a
196− [A] crossApplySnd :: Unfold m a b -> Unfold m a c -> Unfold m a c
197− [A] crossApplyFst :: Unfold m a b -> Unfold m a c -> Unfold m a b
198− [A] concatMap :: Monad m => (b -> Unfold m a c) -> Unfold m a b -> Unfold m a c
199−[R] Streamly.Internal.Data.Unboxed
200−[C] Streamly.Internal.Data.Time.Units
201− [R] Streamly.Internal.Data.Unboxed.Unbox
202− [A] Streamly.Internal.Data.Unbox.Unbox
203− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.NanoSecond64
204− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.MilliSecond64
205− [A] instance Streamly.Internal.Data.Unbox.Unbox Streamly.Internal.Data.Time.Units.MicroSecond64
206−[R] Streamly.Internal.Data.Time.Clock.Type
207−[A] Streamly.Internal.Data.StreamK
208− [A] CrossStreamK
209− [A] (FixityR,6)
210− [A] (FixityR,6)
211− [A] (FixityR,6)
212− [A] (FixityR,5)
213− [A] (FixityR,5)
214− [A] (FixityR,6)
215− [A] (FixityR,5)
216− [A] StreamK
217− [A] MkStream :: (forall r. State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
218− [A] zipWithM :: Monad m => (a -> b -> m c) -> StreamK m a -> StreamK m b -> StreamK m c
219− [A] zipWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
220− [A] unfoldrMWith :: Monad m => (m a -> StreamK m a -> StreamK m a) -> (b -> m (Maybe (a, b))) -> b -> StreamK m a
221− [A] unfoldrM :: Monad m => (b -> m (Maybe (a, b))) -> b -> StreamK m a
222− [A] unfoldr :: (b -> Maybe (a, b)) -> b -> StreamK m a
223− [A] uncons :: Applicative m => StreamK m a -> m (Maybe (a, StreamK m a))
224− [A] unShare :: StreamK m a -> StreamK m a
225− [A] unCross :: CrossStreamK m a -> StreamK m a
226− [A] toStream :: Applicative m => StreamK m a -> Stream m a
227− [A] toList :: Monad m => StreamK m a -> m [a]
228− [A] the :: (Eq a, Monad m) => StreamK m a -> m (Maybe a)
229− [A] takeWhile :: (a -> Bool) -> StreamK m a -> StreamK m a
230− [A] take :: Int -> StreamK m a -> StreamK m a
231− [A] tail :: Applicative m => StreamK m a -> m (Maybe (StreamK m a))
232− [A] sortBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> StreamK m a
233− [A] sequence :: Monad m => StreamK m (m a) -> StreamK m a
234− [A] scanlx' :: (x -> a -> x) -> x -> (x -> b) -> StreamK m a -> StreamK m b
235− [A] scanl' :: (b -> a -> b) -> b -> StreamK m a -> StreamK m b
236− [A] reverse :: StreamK m a -> StreamK m a
237− [A] replicateMWith :: (m a -> StreamK m a -> StreamK m a) -> Int -> m a -> StreamK m a
238− [A] replicateM :: Monad m => Int -> m a -> StreamK m a
239− [A] replicate :: Int -> a -> StreamK m a
240− [A] repeatMWith :: (m a -> t m a -> t m a) -> m a -> t m a
241− [A] repeatM :: Monad m => m a -> StreamK m a
242− [A] repeat :: a -> StreamK m a
243− [A] parseDBreak :: Monad m => Parser a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
244− [A] parseD :: Monad m => Parser a m b -> StreamK m a -> m (Either ParseError b)
245− [A] parseChunksGeneric :: Monad m => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
246− [A] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
247− [A] parseBreakChunksGeneric :: forall m a b. Monad m => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
248− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
249− [A] parseBreak :: forall m a b. Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b, StreamK m a)
250− [A] parse :: Monad m => ParserK a m b -> StreamK m a -> m (Either ParseError b)
251− [A] null :: Monad m => StreamK m a -> m Bool
252− [A] notElem :: (Monad m, Eq a) => a -> StreamK m a -> m Bool
253− [A] nilM :: Applicative m => m b -> StreamK m a
254− [A] nil :: StreamK m a
255− [A] mkStream :: (forall r. State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
256− [A] mkCross :: StreamK m a -> CrossStreamK m a
257− [A] minimumBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> m (Maybe a)
258− [A] minimum :: (Monad m, Ord a) => StreamK m a -> m (Maybe a)
259− [A] mfix :: Monad m => (m a -> StreamK m a) -> StreamK m a
260− [A] mergeMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
261− [A] mergeIterateWith :: (StreamK m a -> StreamK m a -> StreamK m a) -> (a -> StreamK m a) -> StreamK m a -> StreamK m a
262− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
263− [A] mergeBy :: (a -> a -> Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
264− [A] maximumBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> m (Maybe a)
265− [A] maximum :: (Monad m, Ord a) => StreamK m a -> m (Maybe a)
266− [A] mapMaybe :: (a -> Maybe b) -> StreamK m a -> StreamK m b
267− [A] mapM_ :: Monad m => (a -> m b) -> StreamK m a -> m ()
268− [A] mapMWith :: (m b -> StreamK m b -> StreamK m b) -> (a -> m b) -> StreamK m a -> StreamK m b
269− [A] mapMSerial :: Monad m => (a -> m b) -> StreamK m a -> StreamK m b
270− [A] mapM :: Monad m => (a -> m b) -> StreamK m a -> StreamK m b
271− [A] map :: (a -> b) -> StreamK m a -> StreamK m b
272− [A] lookup :: (Monad m, Eq a) => a -> StreamK m (a, b) -> m (Maybe b)
273− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => StreamK m a -> StreamK (t m) a
274− [A] last :: Monad m => StreamK m a -> m (Maybe a)
275− [A] iterateMWith :: Monad m => (m a -> StreamK m a -> StreamK m a) -> (a -> m a) -> m a -> StreamK m a
276− [A] iterateM :: Monad m => (a -> m a) -> m a -> StreamK m a
277− [A] iterate :: (a -> a) -> a -> StreamK m a
278− [A] intersperseM :: Monad m => m a -> StreamK m a -> StreamK m a
279− [A] intersperse :: Monad m => a -> StreamK m a -> StreamK m a
280− [A] interleaveMin :: StreamK m a -> StreamK m a -> StreamK m a
281− [A] interleaveFst :: StreamK m a -> StreamK m a -> StreamK m a
282− [A] interleave :: StreamK m a -> StreamK m a -> StreamK m a
283− [A] insertBy :: (a -> a -> Ordering) -> a -> StreamK m a -> StreamK m a
284− [A] init :: Applicative m => StreamK m a -> m (Maybe (StreamK m a))
285− [A] hoist :: (Monad m, Monad n) => (forall x. m x -> n x) -> StreamK m a -> StreamK n a
286− [A] head :: Monad m => StreamK m a -> m (Maybe a)
287− [A] handle :: (MonadCatch m, Exception e) => (e -> m (StreamK m a)) -> StreamK m a -> StreamK m a
288− [A] fromYieldK :: YieldK m a -> StreamK m a
289− [A] fromStream :: Monad m => Stream m a -> StreamK m a
290− [A] fromStopK :: StopK m -> StreamK m a
291− [A] fromPure :: a -> StreamK m a
292− [A] fromList :: [a] -> StreamK m a
293− [A] fromIndicesMWith :: (m a -> StreamK m a -> StreamK m a) -> (Int -> m a) -> StreamK m a
294− [A] fromIndicesM :: Monad m => (Int -> m a) -> StreamK m a
295− [A] fromIndices :: (Int -> a) -> StreamK m a
296− [A] fromFoldableM :: (Foldable f, Monad m) => f (m a) -> StreamK m a
297− [A] fromFoldable :: Foldable f => f a -> StreamK m a
298− [A] fromEffect :: Monad m => m a -> StreamK m a
299− [A] foldrT :: (Monad m, Monad (s m), MonadTrans s) => (a -> s m b -> s m b) -> s m b -> StreamK m a -> s m b
300− [A] foldrSShared :: (a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
301− [A] foldrSM :: Monad m => (m a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
302− [A] foldrS :: (a -> StreamK m b -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
303− [A] foldrM :: (a -> m b -> m b) -> m b -> StreamK m a -> m b
304− [A] foldr1 :: Monad m => (a -> a -> a) -> StreamK m a -> m (Maybe a)
305− [A] foldr :: Monad m => (a -> b -> b) -> b -> StreamK m a -> m b
306− [A] foldlx' :: forall m a b x. Monad m => (x -> a -> x) -> x -> (x -> b) -> StreamK m a -> m b
307− [A] foldlT :: (Monad m, Monad (s m), MonadTrans s) => (s m b -> a -> s m b) -> s m b -> StreamK m a -> s m b
308− [A] foldlS :: (StreamK m b -> a -> StreamK m b) -> StreamK m b -> StreamK m a -> StreamK m b
309− [A] foldlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> StreamK m a -> m b
310− [A] foldlM' :: Monad m => (b -> a -> m b) -> m b -> StreamK m a -> m b
311− [A] foldl' :: Monad m => (b -> a -> b) -> b -> StreamK m a -> m b
312− [A] foldStreamShared :: State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> StreamK m a -> m r
313− [A] foldStream :: State StreamK m a -> (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> StreamK m a -> m r
314− [A] foldEither :: Monad m => Fold m a b -> StreamK m a -> m (Either (Fold m a b) (b, StreamK m a))
315− [A] foldConcat :: Monad m => Producer m a b -> Fold m b c -> StreamK m a -> m (c, StreamK m a)
316− [A] foldBreak :: Monad m => Fold m a b -> StreamK m a -> m (b, StreamK m a)
317− [A] fold :: Monad m => Fold m a b -> StreamK m a -> m b
318− [A] findM :: Monad m => (a -> m Bool) -> StreamK m a -> m (Maybe a)
319− [A] findIndices :: (a -> Bool) -> StreamK m a -> StreamK m Int
320− [A] find :: Monad m => (a -> Bool) -> StreamK m a -> m (Maybe a)
321− [A] filter :: (a -> Bool) -> StreamK m a -> StreamK m a
322− [A] evalStateT :: Monad m => m s -> StreamK (StateT s m) a -> StreamK m a
323− [A] elem :: (Monad m, Eq a) => a -> StreamK m a -> m Bool
324− [A] dropWhile :: (a -> Bool) -> StreamK m a -> StreamK m a
325− [A] drop :: Int -> StreamK m a -> StreamK m a
326− [A] drain :: Monad m => StreamK m a -> m ()
327− [A] deleteBy :: (a -> a -> Bool) -> a -> StreamK m a -> StreamK m a
328− [A] crossWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
329− [A] crossApplyWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> StreamK m (a -> b) -> StreamK m a -> StreamK m b
330− [A] crossApplySnd :: StreamK m a -> StreamK m b -> StreamK m b
331− [A] crossApplyFst :: StreamK m a -> StreamK m b -> StreamK m a
332− [A] crossApply :: StreamK m (a -> b) -> StreamK m a -> StreamK m b
333− [A] cross :: Monad m => StreamK m a -> StreamK m b -> StreamK m (a, b)
334− [A] consMBy :: Monad m => (StreamK m a -> StreamK m a -> StreamK m a) -> m a -> StreamK m a -> StreamK m a
335− [A] consM :: Monad m => m a -> StreamK m a -> StreamK m a
336− [A] consK :: YieldK m a -> StreamK m a -> StreamK m a
337− [A] cons :: a -> StreamK m a -> StreamK m a
338− [A] conjoin :: Monad m => StreamK m a -> StreamK m a -> StreamK m a
339− [A] concatMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
340− [A] concatMapEffect :: Monad m => (b -> StreamK m a) -> m b -> StreamK m a
341− [A] concatMap :: (a -> StreamK m b) -> StreamK m a -> StreamK m b
342− [A] concatIterateWith :: (StreamK m a -> StreamK m a -> StreamK m a) -> (a -> StreamK m a) -> StreamK m a -> StreamK m a
343− [A] concatIterateScanWith :: Monad m => (StreamK m a -> StreamK m a -> StreamK m a) -> (b -> a -> m (b, StreamK m a)) -> m b -> StreamK m a -> StreamK m a
344− [A] concatIterateLeftsWith :: b ~ Either a c => (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m b -> StreamK m b
345− [A] concatEffect :: Monad m => m (StreamK m a) -> StreamK m a
346− [A] buildSM :: Monad m => ((m a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a
347− [A] buildS :: ((a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a
348− [A] buildM :: Monad m => (forall r. (a -> StreamK m a -> m r) -> (a -> m r) -> m r -> m r) -> StreamK m a
349− [A] build :: forall m a. (forall b. (a -> b -> b) -> b -> b) -> StreamK m a
350− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> StreamK m a) -> StreamK m a
351− [A] bindWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> StreamK m a -> (a -> StreamK m b) -> StreamK m b
352− [A] before :: Monad m => m b -> StreamK m a -> StreamK m a
353− [A] augmentSM :: Monad m => ((m a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a
354− [A] augmentS :: ((a -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a) -> StreamK m a -> StreamK m a
355− [A] append :: StreamK m a -> StreamK m a -> StreamK m a
356− [A] any :: Monad m => (a -> Bool) -> StreamK m a -> m Bool
357− [A] all :: Monad m => (a -> Bool) -> StreamK m a -> m Bool
358− [A] (.:) :: a -> StreamK m a -> StreamK m a
359− [A] (!!) :: Monad m => StreamK m a -> Int -> m (Maybe a)
360−[R] Streamly.Internal.Data.Stream.StreamK.Type
361−[R] Streamly.Internal.Data.Stream.StreamK.Transformer
362−[R] Streamly.Internal.Data.Stream.StreamK
363−[R] Streamly.Internal.Data.Stream.StreamD.Type
364−[R] Streamly.Internal.Data.Stream.StreamD.Transformer
365−[R] Streamly.Internal.Data.Stream.StreamD.Transform
366−[R] Streamly.Internal.Data.Stream.StreamD.Top
367−[R] Streamly.Internal.Data.Stream.StreamD.Step
368−[R] Streamly.Internal.Data.Stream.StreamD.Nesting
369−[R] Streamly.Internal.Data.Stream.StreamD.Lift
370−[R] Streamly.Internal.Data.Stream.StreamD.Generate
371−[R] Streamly.Internal.Data.Stream.StreamD.Exception
372−[R] Streamly.Internal.Data.Stream.StreamD.Eliminate
373−[R] Streamly.Internal.Data.Stream.StreamD.Container
374−[D] Streamly.Internal.Data.Stream.StreamD
375−[R] Streamly.Internal.Data.Stream.Common
376−[R] Streamly.Internal.Data.Stream.Chunked
377−[C] Streamly.Internal.Data.Stream
378− [A] class Enum a => Enumerable a
379− [A] Stream
380− [A] UnStream :: (State StreamK m a -> s -> m (Step s a)) -> s -> Stream m a
381− [A] Step
382− [A] Yield :: a -> s -> Step s a
383− [A] Stop :: Step s a
384− [A] Skip :: s -> Step s a
385− [A] InterleaveState
386− [A] InterleaveSecondOnly :: s2 -> InterleaveState s1 s2
387− [A] InterleaveSecond :: s1 -> s2 -> InterleaveState s1 s2
388− [A] InterleaveFirstOnly :: s1 -> InterleaveState s1 s2
389− [A] InterleaveFirst :: s1 -> s2 -> InterleaveState s1 s2
390− [A] FoldManyPost
391− [A] FoldManyPostYield :: b -> FoldManyPost s fs b a -> FoldManyPost s fs b a
392− [A] FoldManyPostStart :: s -> FoldManyPost s fs b a
393− [A] FoldManyPostLoop :: s -> fs -> FoldManyPost s fs b a
394− [A] FoldManyPostDone :: FoldManyPost s fs b a
395− [A] FoldMany
396− [A] FoldManyYield :: b -> FoldMany s fs b a -> FoldMany s fs b a
397− [A] FoldManyStart :: s -> FoldMany s fs b a
398− [A] FoldManyLoop :: s -> fs -> FoldMany s fs b a
399− [A] FoldManyFirst :: fs -> s -> FoldMany s fs b a
400− [A] FoldManyDone :: FoldMany s fs b a
401− [A] CrossStream
402− [A] ConcatUnfoldInterleaveState
403− [A] ConcatUnfoldInterleaveOuter :: o -> [i] -> ConcatUnfoldInterleaveState o i
404− [A] ConcatUnfoldInterleaveInnerR :: [i] -> [i] -> ConcatUnfoldInterleaveState o i
405− [A] ConcatUnfoldInterleaveInnerL :: [i] -> [i] -> ConcatUnfoldInterleaveState o i
406− [A] ConcatUnfoldInterleaveInner :: o -> [i] -> ConcatUnfoldInterleaveState o i
407− [A] ConcatMapUState
408− [A] ConcatMapUOuter :: o -> ConcatMapUState o i
409− [A] ConcatMapUInner :: o -> i -> ConcatMapUState o i
410− [A] AppendState
411− [A] AppendSecond :: s2 -> AppendState s1 s2
412− [A] AppendFirst :: s1 -> AppendState s1 s2
413− [A] pattern Stream :: (State StreamK m a -> s -> m (Step s a)) -> s -> Stream m a
414− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Stream m b -> Stream m c
415− [A] zipWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
416− [A] wordsBy :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
417− [A] with :: Monad m => (Stream m a -> Stream m (s, a)) -> (((s, a) -> b) -> Stream m (s, a) -> Stream m (s, a)) -> ((s, a) -> b) -> Stream m a -> Stream m a
418− [A] usingStateT :: Monad m => m s -> (Stream (StateT s m) a -> Stream (StateT s m) a) -> Stream m a -> Stream m a
419− [A] usingReaderT :: Monad m => m r -> (Stream (ReaderT r m) a -> Stream (ReaderT r m) a) -> Stream m a -> Stream m a
420− [A] uniqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a
421− [A] uniq :: (Eq a, Monad m) => Stream m a -> Stream m a
422− [A] unionWithStreamGenericBy :: MonadIO m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
423− [A] unionWithStreamAscBy :: (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
424− [A] unfoldrM :: Monad m => (s -> m (Maybe (a, s))) -> s -> Stream m a
425− [A] unfoldr :: Monad m => (s -> Maybe (a, s)) -> s -> Stream m a
426− [A] unfoldRoundRobin :: Monad m => Unfold m a b -> Stream m a -> Stream m b
427− [A] unfoldMany :: Monad m => Unfold m a b -> Stream m a -> Stream m b
428− [A] unfoldIterateDfs :: Monad m => Unfold m a a -> Stream m a -> Stream m a
429− [A] unfoldIterateBfsRev :: Monad m => Unfold m a a -> Stream m a -> Stream m a
430− [A] unfoldIterateBfs :: Monad m => Unfold m a a -> Stream m a -> Stream m a
431− [A] unfoldInterleave :: Monad m => Unfold m a b -> Stream m a -> Stream m b
432− [A] unfold :: Applicative m => Unfold m a b -> a -> Stream m b
433− [A] uncons :: Monad m => Stream m a -> m (Maybe (a, Stream m a))
434− [A] unCross :: CrossStream m a -> Stream m a
435− [A] transform :: Monad m => Pipe m a b -> Stream m a -> Stream m b
436− [A] trace_ :: Monad m => m b -> Stream m a -> Stream m a
437− [A] trace :: Monad m => (a -> m b) -> Stream m a -> Stream m a
438− [A] toStreamK :: Monad m => Stream m a -> StreamK m a
439− [A] toListRev :: Monad m => Stream m a -> m [a]
440− [A] toList :: Monad m => Stream m a -> m [a]
441− [A] timestamped :: MonadIO m => Stream m a -> Stream m (AbsTime, a)
442− [A] timestampWith :: MonadIO m => Double -> Stream m a -> Stream m (AbsTime, a)
443− [A] timesWith :: MonadIO m => Double -> Stream m (AbsTime, RelTime64)
444− [A] times :: MonadIO m => Stream m (AbsTime, RelTime64)
445− [A] timeout :: AbsTime -> t m ()
446− [A] timeIndexed :: MonadIO m => Stream m a -> Stream m (RelTime64, a)
447− [A] timeIndexWith :: MonadIO m => Double -> Stream m a -> Stream m (RelTime64, a)
448− [A] the :: (Eq a, Monad m) => Stream m a -> m (Maybe a)
449− [A] tapOffsetEvery :: Monad m => Int -> Int -> Fold m a b -> Stream m a -> Stream m a
450− [A] tap :: Monad m => Fold m a b -> Stream m a -> Stream m a
451− [A] takeWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
452− [A] takeWhileLast :: (a -> Bool) -> Stream m a -> Stream m a
453− [A] takeWhileAround :: (a -> Bool) -> Stream m a -> Stream m a
454− [A] takeWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
455− [A] takeEndByM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
456− [A] takeEndBy :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
457− [A] take :: Applicative m => Int -> Stream m a -> Stream m a
458− [A] tail :: Monad m => Stream m a -> m (Maybe (Stream m a))
459− [A] stripSuffixUnbox :: (MonadIO m, Eq a, Unbox a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
460− [A] stripSuffix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
461− [A] stripPrefix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
462− [A] strideFromThen :: Monad m => Int -> Int -> Stream m a -> Stream m a
463− [A] splitOnSuffixSeqAny :: [Array a] -> Fold m a b -> Stream m a -> Stream m b
464− [A] splitOnSuffixSeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) => Bool -> Array a -> Fold m a b -> Stream m a -> Stream m b
465− [A] splitOnSeq :: forall m a b. (MonadIO m, Storable a, Unbox a, Enum a, Eq a) => Array a -> Fold m a b -> Stream m a -> Stream m b
466− [A] splitOnPrefix :: (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
467− [A] splitOnAny :: [Array a] -> Fold m a b -> Stream m a -> Stream m b
468− [A] splitOn :: Monad m => (a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
469− [A] splitInnerBySuffix :: (Monad m, Eq (f a), Monoid (f a)) => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
470− [A] splitInnerBy :: Monad m => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
471− [A] slicesBy :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
472− [A] sliceOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
473− [A] sequence :: Monad m => Stream m (m a) -> Stream m a
474− [A] scanlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> Stream m b
475− [A] scanlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> Stream m b
476− [A] scanlMAfter' :: Monad m => (b -> a -> m b) -> m b -> (b -> m b) -> Stream m a -> Stream m b
477− [A] scanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
478− [A] scanlM :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
479− [A] scanl1M' :: Monad m => (a -> a -> m a) -> Stream m a -> Stream m a
480− [A] scanl1M :: Monad m => (a -> a -> m a) -> Stream m a -> Stream m a
481− [A] scanl1' :: Monad m => (a -> a -> a) -> Stream m a -> Stream m a
482− [A] scanl1 :: Monad m => (a -> a -> a) -> Stream m a -> Stream m a
483− [A] scanl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
484− [A] scanl :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
485− [A] scanMaybe :: Monad m => Fold m a (Maybe b) -> Stream m a -> Stream m b
486− [A] scanMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
487− [A] scan :: Monad m => Fold m a b -> Stream m a -> Stream m b
488− [A] runStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m (s, a)
489− [A] runReaderT :: Monad m => m s -> Stream (ReaderT s m) a -> Stream m a
490− [A] runInnerWithState :: Monad m => (forall b. s -> t m b -> m (b, s)) -> m s -> Stream (t m) a -> Stream m (s, a)
491− [A] runInnerWith :: Monad m => (forall b. t m b -> m b) -> Stream (t m) a -> Stream m a
492− [A] roundRobin :: Monad m => Stream m a -> Stream m a -> Stream m a
493− [A] rollingMapM :: Monad m => (Maybe a -> a -> m b) -> Stream m a -> Stream m b
494− [A] rollingMap2 :: Monad m => (a -> a -> b) -> Stream m a -> Stream m b
495− [A] rollingMap :: Monad m => (Maybe a -> a -> b) -> Stream m a -> Stream m b
496− [A] reverseUnbox :: (MonadIO m, Unbox a) => Stream m a -> Stream m a
497− [A] reverse :: Monad m => Stream m a -> Stream m a
498− [A] replicateM :: Monad m => Int -> m a -> Stream m a
499− [A] replicate :: Monad m => Int -> a -> Stream m a
500− [A] repeated :: Stream m a -> Stream m a
501− [A] repeatM :: Monad m => m a -> Stream m a
502− [A] repeat :: Monad m => a -> Stream m a
503− [A] relTimesWith :: MonadIO m => Double -> Stream m RelTime64
504− [A] relTimes :: MonadIO m => Stream m RelTime64
505− [A] refoldMany :: Monad m => Refold m x a b -> m x -> Stream m a -> Stream m b
506− [A] refoldIterateM :: Monad m => Refold m b a b -> m b -> Stream m a -> Stream m b
507− [A] reduceIterateBfs :: Monad m => (a -> a -> m a) -> Stream m a -> m (Maybe a)
508− [A] reassembleBy :: Fold m a b -> (a -> a -> Int) -> Stream m a -> Stream m b
509− [A] prune :: (a -> Bool) -> Stream m a -> Stream m a
510− [A] prescanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
511− [A] prescanl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> Stream m b
512− [A] postscanlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> Stream m b
513− [A] postscanlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> Stream m b
514− [A] postscanlMAfter' :: Monad m => (b -> a -> m b) -> m b -> (b -> m b) -> Stream m a -> Stream m b
515− [A] postscanlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
516− [A] postscanlM :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> Stream m b
517− [A] postscanl' :: Monad m => (a -> b -> a) -> a -> Stream m b -> Stream m a
518− [A] postscanl :: Monad m => (a -> b -> a) -> a -> Stream m b -> Stream m a
519− [A] postscan :: Monad m => Fold m a b -> Stream m a -> Stream m b
520− [A] parseSequence :: Stream m (Parser a m b) -> Stream m a -> Stream m b
521− [A] parseManyTill :: Parser a m b -> Parser a m x -> Stream m a -> Stream m b
522− [A] parseManyD :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
523− [A] parseMany :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
524− [A] parseIterateD :: Monad m => (b -> Parser a m b) -> b -> Stream m a -> Stream m (Either ParseError b)
525− [A] parseIterate :: Monad m => (b -> Parser a m b) -> b -> Stream m a -> Stream m (Either ParseError b)
526− [A] parseD :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
527− [A] parseBreakD :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b, Stream m a)
528− [A] parseBreak :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b, Stream m a)
529− [A] parse :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
530− [A] onException :: MonadCatch m => m b -> Stream m a -> Stream m a
531− [A] null :: Monad m => Stream m a -> m Bool
532− [A] nub :: (Monad m, Ord a) => Stream m a -> Stream m a
533− [A] notElem :: (Monad m, Eq a) => a -> Stream m a -> m Bool
534− [A] nilM :: Applicative m => m b -> Stream m a
535− [A] nil :: Applicative m => Stream m a
536− [A] morphInner :: Monad n => (forall x. m x -> n x) -> Stream m a -> Stream n a
537− [A] mkCross :: Stream m a -> CrossStream m a
538− [A] minimumBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> m (Maybe a)
539− [A] minimum :: (Monad m, Ord a) => Stream m a -> m (Maybe a)
540− [A] mergeMinBy :: (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
541− [A] mergeFstBy :: (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
542− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
543− [A] mergeBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
544− [A] maximumBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> m (Maybe a)
545− [A] maximum :: (Monad m, Ord a) => Stream m a -> m (Maybe a)
546− [A] mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Stream m a -> Stream m b
547− [A] mapMaybe :: Monad m => (a -> Maybe b) -> Stream m a -> Stream m b
548− [A] mapM_ :: Monad m => (a -> m b) -> Stream m a -> m ()
549− [A] mapM :: Monad m => (a -> m b) -> Stream m a -> Stream m b
550− [A] map :: Monad m => (a -> b) -> Stream m a -> Stream m b
551− [A] lookup :: (Monad m, Eq a) => a -> Stream m (a, b) -> m (Maybe b)
552− [A] liftInnerWith :: Monad (t m) => (forall b. m b -> t m b) -> Stream m a -> Stream (t m) a
553− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => Stream m a -> Stream (t m) a
554− [A] last :: Monad m => Stream m a -> m (Maybe a)
555− [A] joinOuterGeneric :: MonadIO m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (Maybe a, Maybe b)
556− [A] joinOuterAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (Maybe a, Maybe b)
557− [A] joinOuter :: (Ord k, MonadIO m) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, Maybe a, Maybe b)
558− [A] joinLeftGeneric :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (a, Maybe b)
559− [A] joinLeftAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (a, Maybe b)
560− [A] joinLeft :: (Ord k, Monad m) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, a, Maybe b)
561− [A] joinInnerGeneric :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> Stream m (a, b)
562− [A] joinInnerAscBy :: (a -> b -> Ordering) -> Stream m a -> Stream m b -> Stream m (a, b)
563− [A] joinInner :: (Monad m, Ord k) => Stream m (k, a) -> Stream m (k, b) -> Stream m (k, a, b)
564− [A] iterateM :: Monad m => (a -> m a) -> m a -> Stream m a
565− [A] iterate :: Monad m => (a -> a) -> a -> Stream m a
566− [A] isSuffixOfUnbox :: (MonadIO m, Eq a, Unbox a) => Stream m a -> Stream m a -> m Bool
567− [A] isSuffixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
568− [A] isSubsequenceOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
569− [A] isPrefixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
570− [A] isInfixOf :: (MonadIO m, Eq a, Enum a, Storable a, Unbox a) => Stream m a -> Stream m a -> m Bool
571− [A] intersperseM_ :: Monad m => m b -> Stream m a -> Stream m a
572− [A] intersperseMWith :: Int -> m a -> Stream m a -> Stream m a
573− [A] intersperseMSuffix_ :: Monad m => m b -> Stream m a -> Stream m a
574− [A] intersperseMSuffixWith :: forall m a. Monad m => Int -> m a -> Stream m a -> Stream m a
575− [A] intersperseMSuffix :: forall m a. Monad m => m a -> Stream m a -> Stream m a
576− [A] intersperseMPrefix_ :: Monad m => m b -> Stream m a -> Stream m a
577− [A] intersperseM :: Monad m => m a -> Stream m a -> Stream m a
578− [A] intersperse :: Monad m => a -> Stream m a -> Stream m a
579− [A] intersectBySorted :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
580− [A] interposeSuffixM :: Monad m => m c -> Unfold m b c -> Stream m b -> Stream m c
581− [A] interposeSuffix :: Monad m => c -> Unfold m b c -> Stream m b -> Stream m c
582− [A] interposeM :: Monad m => m c -> Unfold m b c -> Stream m b -> Stream m c
583− [A] interpose :: Monad m => c -> Unfold m b c -> Stream m b -> Stream m c
584− [A] interleaveMin :: Monad m => Stream m a -> Stream m a -> Stream m a
585− [A] interleaveFstSuffix :: Monad m => Stream m a -> Stream m a -> Stream m a
586− [A] interleaveFst :: Monad m => Stream m a -> Stream m a -> Stream m a
587− [A] interleave :: Monad m => Stream m a -> Stream m a -> Stream m a
588− [A] intercalateSuffix :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
589− [A] intercalate :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
590− [A] insertBy :: Monad m => (a -> a -> Ordering) -> a -> Stream m a -> Stream m a
591− [A] indexedR :: Monad m => Int -> Stream m a -> Stream m (Int, a)
592− [A] indexed :: Monad m => Stream m a -> Stream m (Int, a)
593− [A] headElse :: Monad m => a -> Stream m a -> m a
594− [A] head :: Monad m => Stream m a -> m (Maybe a)
595− [A] handle :: (MonadCatch m, Exception e) => (e -> m (Stream m a)) -> Stream m a -> Stream m a
596− [A] groupsWhile :: Monad m => (a -> a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
597− [A] groupsRollingBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Stream m a -> Stream m b
598− [A] groupsOf :: Monad m => Int -> Fold m a b -> Stream m a -> Stream m b
599− [A] gintercalateSuffix :: Monad m => Unfold m a c -> Stream m a -> Unfold m b c -> Stream m b -> Stream m c
600− [A] gintercalate :: Monad m => Unfold m a c -> Stream m a -> Unfold m b c -> Stream m b -> Stream m c
601− [A] ghandle :: (MonadCatch m, Exception e) => (e -> Stream m a -> m (Stream m a)) -> Stream m a -> Stream m a
602− [A] generateM :: Monad m => Int -> (Int -> m a) -> Stream m a
603− [A] generate :: Monad m => Int -> (Int -> a) -> Stream m a
604− [A] generalizeInner :: Monad m => Stream Identity a -> Stream m a
605− [A] gbracket_ :: Monad m => m c -> (c -> m d) -> (c -> e -> Stream m b -> m (Stream m b)) -> (forall s. m s -> m (Either e s)) -> (c -> Stream m b) -> Stream m b
606− [A] gbracket :: MonadIO m => IO c -> (c -> IO d1) -> (c -> e -> Stream m b -> IO (Stream m b)) -> (c -> IO d2) -> (forall s. m s -> m (Either e s)) -> (c -> Stream m b) -> Stream m b
607− [A] fromStreamK :: Applicative m => StreamK m a -> Stream m a
608− [A] fromPure :: Applicative m => a -> Stream m a
609− [A] fromPtrN :: (Monad m, Storable a) => Int -> Ptr a -> Stream m a
610− [A] fromPtr :: forall m a. (Monad m, Storable a) => Ptr a -> Stream m a
611− [A] fromListM :: Monad m => [m a] -> Stream m a
612− [A] fromList :: Applicative m => [a] -> Stream m a
613− [A] fromIndicesM :: Monad m => (Int -> m a) -> Stream m a
614− [A] fromIndices :: Monad m => (Int -> a) -> Stream m a
615− [A] fromFoldableM :: (Monad m, Foldable f) => f (m a) -> Stream m a
616− [A] fromFoldable :: (Monad m, Foldable f) => f a -> Stream m a
617− [A] fromEffect :: Applicative m => m a -> Stream m a
618− [A] fromByteStr# :: Monad m => Addr# -> Stream m Word8
619− [A] foldrT :: (Monad m, Monad (t m), MonadTrans t) => (a -> t m b -> t m b) -> t m b -> Stream m a -> t m b
620− [A] foldrS :: Monad m => (a -> Stream m b -> Stream m b) -> Stream m b -> Stream m a -> Stream m b
621− [A] foldrMx :: Monad m => (a -> m x -> m x) -> m x -> (m x -> m b) -> Stream m a -> m b
622− [A] foldrM :: Monad m => (a -> m b -> m b) -> m b -> Stream m a -> m b
623− [A] foldr1 :: Monad m => (a -> a -> a) -> Stream m a -> m (Maybe a)
624− [A] foldr :: Monad m => (a -> b -> b) -> b -> Stream m a -> m b
625− [A] foldlx' :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream m a -> m b
626− [A] foldlT :: (Monad m, Monad (s m), MonadTrans s) => (s m b -> a -> s m b) -> s m b -> Stream m a -> s m b
627− [A] foldlS :: Monad m => (Stream m b -> a -> Stream m b) -> Stream m b -> Stream m a -> Stream m b
628− [A] foldlMx' :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream m a -> m b
629− [A] foldlM' :: Monad m => (b -> a -> m b) -> m b -> Stream m a -> m b
630− [A] foldl' :: Monad m => (b -> a -> b) -> b -> Stream m a -> m b
631− [A] foldSequence :: Stream m (Fold m a b) -> Stream m a -> Stream m b
632− [A] foldManyPost :: Monad m => Fold m a b -> Stream m a -> Stream m b
633− [A] foldMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
634− [A] foldIterateM :: Monad m => (b -> m (Fold m a b)) -> m b -> Stream m a -> Stream m b
635− [A] foldIterateBfs :: Fold m a (Either a a) -> Stream m a -> m (Maybe a)
636− [A] foldEither :: Monad m => Fold m a b -> Stream m a -> m (Either (Fold m a b) (b, Stream m a))
637− [A] foldBreak :: Monad m => Fold m a b -> Stream m a -> m (b, Stream m a)
638− [A] foldAddLazy :: Monad m => Fold m a b -> Stream m a -> Fold m a b
639− [A] foldAdd :: Monad m => Fold m a b -> Stream m a -> m (Fold m a b)
640− [A] fold :: Monad m => Fold m a b -> Stream m a -> m b
641− [A] findM :: Monad m => (a -> m Bool) -> Stream m a -> m (Maybe a)
642− [A] findIndices :: Monad m => (a -> Bool) -> Stream m a -> Stream m Int
643− [A] find :: Monad m => (a -> Bool) -> Stream m a -> m (Maybe a)
644− [A] finallyUnsafe :: MonadCatch m => m b -> Stream m a -> Stream m a
645− [A] finallyIO :: (MonadIO m, MonadCatch m) => IO b -> Stream m a -> Stream m a
646− [A] filterM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
647− [A] filterInStreamGenericBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
648− [A] filterInStreamAscBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
649− [A] filter :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
650− [A] evalStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m a
651− [A] eqBy :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> m Bool
652− [A] enumerateTo :: (Monad m, Bounded a, Enumerable a) => a -> Stream m a
653− [A] enumerateFromToSmall :: (Monad m, Enum a) => a -> a -> Stream m a
654− [A] enumerateFromToIntegral :: (Monad m, Integral a) => a -> a -> Stream m a
655− [A] enumerateFromToFractional :: (Monad m, Fractional a, Ord a) => a -> a -> Stream m a
656− [A] enumerateFromTo :: (Enumerable a, Monad m) => a -> a -> Stream m a
657− [A] enumerateFromThenToSmall :: (Monad m, Enum a) => a -> a -> a -> Stream m a
658− [A] enumerateFromThenToIntegral :: (Monad m, Integral a) => a -> a -> a -> Stream m a
659− [A] enumerateFromThenToFractional :: (Monad m, Fractional a, Ord a) => a -> a -> a -> Stream m a
660− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => a -> a -> a -> Stream m a
661− [A] enumerateFromThenSmallBounded :: (Monad m, Enumerable a, Bounded a) => a -> a -> Stream m a
662− [A] enumerateFromThenNum :: (Monad m, Num a) => a -> a -> Stream m a
663− [A] enumerateFromThenIntegral :: (Monad m, Integral a, Bounded a) => a -> a -> Stream m a
664− [A] enumerateFromThenFractional :: (Monad m, Fractional a) => a -> a -> Stream m a
665− [A] enumerateFromThen :: (Enumerable a, Monad m) => a -> a -> Stream m a
666− [A] enumerateFromStepNum :: (Monad m, Num a) => a -> a -> Stream m a
667− [A] enumerateFromStepIntegral :: (Integral a, Monad m) => a -> a -> Stream m a
668− [A] enumerateFromNum :: (Monad m, Num a) => a -> Stream m a
669− [A] enumerateFromIntegral :: (Monad m, Integral a, Bounded a) => a -> Stream m a
670− [A] enumerateFromFractional :: (Monad m, Fractional a) => a -> Stream m a
671− [A] enumerateFromBounded :: (Monad m, Enumerable a, Bounded a) => a -> Stream m a
672− [A] enumerateFrom :: (Enumerable a, Monad m) => a -> Stream m a
673− [A] enumerate :: (Monad m, Bounded a, Enumerable a) => Stream m a
674− [A] elemIndices :: (Monad m, Eq a) => a -> Stream m a -> Stream m Int
675− [A] elem :: (Monad m, Eq a) => a -> Stream m a -> m Bool
676− [A] durations :: Double -> t m RelTime64
677− [A] dropWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
678− [A] dropWhileLast :: (a -> Bool) -> Stream m a -> Stream m a
679− [A] dropWhileAround :: (a -> Bool) -> Stream m a -> Stream m a
680− [A] dropWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
681− [A] dropSuffix :: Stream m a -> Stream m a -> Stream m a
682− [A] dropPrefix :: Stream m a -> Stream m a -> Stream m a
683− [A] dropLast :: Int -> Stream m a -> Stream m a
684− [A] dropInfix :: Stream m a -> Stream m a -> Stream m a
685− [A] drop :: Monad m => Int -> Stream m a -> Stream m a
686− [A] drain :: Monad m => Stream m a -> m ()
687− [A] deleteInStreamGenericBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Stream m a -> Stream m a
688− [A] deleteInStreamAscBy :: (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
689− [A] deleteBy :: Monad m => (a -> a -> Bool) -> a -> Stream m a -> Stream m a
690− [A] delayPre :: MonadIO m => Double -> Stream m a -> Stream m a
691− [A] delayPost :: MonadIO m => Double -> Stream m a -> Stream m a
692− [A] delay :: MonadIO m => Double -> Stream m a -> Stream m a
693− [A] crossWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
694− [A] crossApplySnd :: Functor f => Stream f a -> Stream f b -> Stream f b
695− [A] crossApplyFst :: Functor f => Stream f a -> Stream f b -> Stream f a
696− [A] crossApply :: Functor f => Stream f (a -> b) -> Stream f a -> Stream f b
697− [A] cross :: Monad m => Stream m a -> Stream m b -> Stream m (a, b)
698− [A] consM :: Applicative m => m a -> Stream m a -> Stream m a
699− [A] cons :: Applicative m => a -> Stream m a -> Stream m a
700− [A] concatMapM :: Monad m => (a -> m (Stream m b)) -> Stream m a -> Stream m b
701− [A] concatMap :: Monad m => (a -> Stream m b) -> Stream m a -> Stream m b
702− [A] concatIterateScan :: Monad m => (b -> a -> m b) -> (b -> m (Maybe (b, Stream m a))) -> b -> Stream m a
703− [A] concatIterateDfs :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
704− [A] concatIterateBfsRev :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
705− [A] concatIterateBfs :: Monad m => (a -> Maybe (Stream m a)) -> Stream m a -> Stream m a
706− [A] concatEffect :: Monad m => m (Stream m a) -> Stream m a
707− [A] concat :: Monad m => Stream m (Stream m a) -> Stream m a
708− [A] cmpBy :: Monad m => (a -> b -> Ordering) -> Stream m a -> Stream m b -> m Ordering
709− [A] catRights :: Monad m => Stream m (Either a b) -> Stream m b
710− [A] catMaybes :: Monad m => Stream m (Maybe a) -> Stream m a
711− [A] catLefts :: Monad m => Stream m (Either a b) -> Stream m a
712− [A] catEithers :: Monad m => Stream m (Either a a) -> Stream m a
713− [A] bracketUnsafe :: MonadCatch m => m b -> (b -> m c) -> (b -> Stream m a) -> Stream m a
714− [A] bracketIO3 :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> IO d) -> (b -> IO e) -> (b -> Stream m a) -> Stream m a
715− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> Stream m a) -> Stream m a
716− [A] before :: Monad m => m b -> Stream m a -> Stream m a
717− [A] append :: Monad m => Stream m a -> Stream m a -> Stream m a
718− [A] any :: Monad m => (a -> Bool) -> Stream m a -> m Bool
719− [A] all :: Monad m => (a -> Bool) -> Stream m a -> m Bool
720− [A] afterUnsafe :: Monad m => m b -> Stream m a -> Stream m a
721− [A] afterIO :: MonadIO m => IO b -> Stream m a -> Stream m a
722− [A] absTimesWith :: MonadIO m => Double -> Stream m AbsTime
723− [A] absTimes :: MonadIO m => Stream m AbsTime
724− [A] (!!) :: Monad m => Stream m a -> Int -> m (Maybe a)
725−[R] Streamly.Internal.Data.Ring.Unboxed
726−[A] Streamly.Internal.Data.Ring.Generic
727− [A] Ring
728− [A] [ringMax] :: Ring a -> !Int
729− [A] [ringHead] :: Ring a -> !Int
730− [A] [ringArr] :: Ring a -> MutArray a
731− [A] Ring :: MutArray a -> !Int -> !Int -> Ring a
732− [A] writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
733− [A] unsafeInsertRingWith :: Ring a -> a -> IO Int
734− [A] toStreamWith :: Int -> Ring a -> Stream m a
735− [A] toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
736− [A] seek :: MonadIO m => Int -> Ring a -> m (Ring a)
737− [A] createRing :: MonadIO m => Int -> m (Ring a)
738− [A] copyToMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
739−[C] Streamly.Internal.Data.Ring
740− [C] Ring
741− [A] [ringStart] :: Ring a -> {-# UNPACK #-} !ForeignPtr a
742− [R] [ringMax] :: Ring a -> !Int
743− [R] [ringHead] :: Ring a -> !Int
744− [A] [ringBound] :: Ring a -> {-# UNPACK #-} !Ptr a
745− [R] [ringArr] :: Ring a -> MutArray a
746− [C] Ring
747− [O] Ring :: MutArray a -> !Int -> !Int -> Ring a
748− [N] Ring :: {-# UNPACK #-} !ForeignPtr a -> {-# UNPACK #-} !Ptr a -> Ring a
749− [A] GHC.Show.Show
750− [A] instance (GHC.Show.Show a, GHC.Show.Show b, GHC.Show.Show c, GHC.Show.Show d) => GHC.Show.Show (Streamly.Internal.Data.Ring.Tuple4' a b c d)
751− [A] writeN :: Int -> Fold m a (Ring a)
752− [R] writeLastN :: MonadIO m => Int -> Fold m a (Ring a)
753− [R] unsafeInsertRingWith :: Ring a -> a -> IO Int
754− [A] unsafeInsert :: Storable a => Ring a -> Ptr a -> a -> IO (Ptr a)
755− [A] unsafeFoldRingNM :: forall m a b. (MonadIO m, Storable a) => Int -> Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
756− [A] unsafeFoldRingM :: forall m a b. (MonadIO m, Storable a) => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
757− [A] unsafeFoldRingFullM :: forall m a b. (MonadIO m, Storable a) => Ptr a -> (b -> a -> m b) -> b -> Ring a -> m b
758− [A] unsafeFoldRing :: forall a b. Storable a => Ptr a -> (b -> a -> b) -> b -> Ring a -> b
759− [A] unsafeEqArrayN :: Ring a -> Ptr a -> Array a -> Int -> Bool
760− [A] unsafeEqArray :: Ring a -> Ptr a -> Array a -> Bool
761− [R] toStreamWith :: Int -> Ring a -> Stream m a
762− [R] toMutArray :: MonadIO m => Int -> Int -> Ring a -> m (MutArray a)
763− [A] startOf :: Ring a -> Ptr a
764− [A] slidingWindowWith :: forall m a b. (MonadIO m, Storable a, Unbox a) => Int -> Fold m ((a, Maybe a), m (MutArray a)) b -> Fold m a b
765− [A] slidingWindow :: forall m a b. (MonadIO m, Storable a, Unbox a) => Int -> Fold m (a, Maybe a) b -> Fold m a b
766− [A] slide :: Ring a -> a -> m (Ring a)
767− [R] seek :: MonadIO m => Int -> Ring a -> m (Ring a)
768− [A] ringsOf :: Int -> Stream m a -> Stream m (MutArray a)
769− [A] readRev :: Unfold m (MutArray a) a
770− [A] read :: forall m a. (MonadIO m, Storable a) => Unfold m (Ring a, Ptr a, Int) a
771− [A] putIndex :: Ring a -> Int -> a -> m ()
772− [A] newRing :: Int -> m (Ring a)
773− [A] new :: forall a. Storable a => Int -> IO (Ring a, Ptr a)
774− [A] moveBy :: forall a. Storable a => Int -> Ring a -> Ptr a -> Ptr a
775− [A] modifyIndex :: Ring a -> Int -> (a -> (a, b)) -> m b
776− [A] length :: Ring a -> Int
777− [A] getIndexUnsafe :: Ring a -> Int -> m a
778− [A] getIndexRev :: Ring a -> Int -> m a
779− [A] getIndex :: Ring a -> Int -> m a
780− [A] fromArray :: MutArray a -> Ring a
781− [R] createRing :: MonadIO m => Int -> m (Ring a)
782− [A] castUnsafe :: Ring a -> Ring b
783− [A] cast :: forall a b. Storable b => Ring a -> Maybe (Ring b)
784− [A] bytesFree :: Ring a -> Int
785− [A] byteLength :: Ring a -> Int
786− [A] byteCapacity :: Ring a -> Int
787− [A] asBytes :: Ring a -> Ring Word8
788− [A] advance :: forall a. Storable a => Ring a -> Ptr a -> Ptr a
789−[R] Streamly.Internal.Data.Producer.Type
790−[R] Streamly.Internal.Data.Producer.Source
791−[C] Streamly.Internal.Data.Producer
792− [A] Source
793− [A] unread :: [b] -> Source a b -> Source a b
794− [A] translate :: Functor m => (a -> c) -> (c -> a) -> Producer m c b -> Producer m a b
795− [A] source :: Maybe a -> Source a b
796− [A] producer :: Monad m => Producer m a b -> Producer m (Source a b) b
797− [A] parseManyD :: Monad m => Parser a m b -> Producer m (Source x a) a -> Producer m (Source x a) (Either ParseError b)
798− [A] parseMany :: Monad m => Parser a m b -> Producer m (Source x a) a -> Producer m (Source x a) (Either ParseError b)
799− [A] parse :: Monad m => Parser a m b -> Producer m (Source s a) a -> Source s a -> m (Either ParseError b, Source s a)
800− [A] lmap :: (a -> a) -> Producer m a b -> Producer m a b
801− [A] isEmpty :: Source a b -> Bool
802−[R] Streamly.Internal.Data.Pipe.Type
803−[C] Streamly.Internal.Data.Pipe
804− [A] Step
805− [A] Yield :: a -> s -> Step s a
806− [A] Continue :: s -> Step s a
807− [A] PipeState
808− [A] Produce :: s2 -> PipeState s1 s2
809− [A] Consume :: s1 -> PipeState s1 s2
810− [C] Pipe
811− [A] Pipe :: (s1 -> a -> m (Step (PipeState s1 s2) b)) -> (s2 -> m (Step (PipeState s1 s2) b)) -> s1 -> Pipe m a b
812−[A] Streamly.Internal.Data.ParserK
813− [A] Step
814− [A] Partial :: !Int -> (Input a -> m (Step a m r)) -> Step a m r
815− [A] Error :: !Int -> String -> Step a m r
816− [A] Done :: !Int -> r -> Step a m r
817− [A] Continue :: !Int -> (Input a -> m (Step a m r)) -> Step a m r
818− [A] ParseResult
819− [A] Success :: !Int -> !b -> ParseResult b
820− [A] Failure :: !Int -> !String -> ParseResult b
821− [A] Input
822− [A] None :: Input a
823− [A] Chunk :: a -> Input a
824− [A] ParserK
825− [A] [runParser] :: ParserK a m b -> forall r. (ParseResult b -> Int -> Input a -> m (Step a m r)) -> Int -> Int -> Input a -> m (Step a m r)
826− [A] MkParser :: (forall r. (ParseResult b -> Int -> Input a -> m (Step a m r)) -> Int -> Int -> Input a -> m (Step a m r)) -> ParserK a m b
827− [A] fromPure :: b -> ParserK a m b
828− [A] fromEffect :: Monad m => m b -> ParserK a m b
829− [A] die :: String -> ParserK a m b
830− [A] adaptCG :: Monad m => Parser a m b -> ParserK (Array a) m b
831− [A] adaptC :: (Monad m, Unbox a) => Parser a m b -> ParserK (Array a) m b
832− [A] adapt :: Monad m => Parser a m b -> ParserK a m b
833−[R] Streamly.Internal.Data.Parser.ParserK.Type
834−[R] Streamly.Internal.Data.Parser.ParserD.Type
835−[R] Streamly.Internal.Data.Parser.ParserD.Tee
836−[R] Streamly.Internal.Data.Parser.ParserD
837−[C] Streamly.Internal.Data.Parser
838− [A] Step
839− [A] Partial :: !Int -> !s -> Step s b
840− [A] Error :: !String -> Step s b
841− [A] Done :: !Int -> !b -> Step s b
842− [A] Continue :: !Int -> !s -> Step s b
843− [A] Parser
844− [A] Parser :: (s -> a -> m (Step s b)) -> m (Initial s b) -> (s -> m (Step s b)) -> Parser a m b
845− [A] Initial
846− [A] IPartial :: !s -> Initial s b
847− [A] IError :: !String -> Initial s b
848− [A] IDone :: !b -> Initial s b
849− [A] GHC.Show.Show
850− [A] instance (GHC.Show.Show a, GHC.Show.Show b) => GHC.Show.Show (Streamly.Internal.Data.Parser.Tuple'Fused a b)
851− [A] ParseError
852− [A] ParseError :: String -> ParseError
853− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Fold m c x -> Parser b m x
854− [A] zip :: Monad m => Stream m a -> Fold m (a, b) x -> Parser b m x
855− [A] wordWithQuotes :: (Monad m, Eq a) => Bool -> (a -> a -> Maybe a) -> a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
856− [A] wordProcessQuotes :: (Monad m, Eq a) => a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
857− [A] wordKeepQuotes :: (Monad m, Eq a) => a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
858− [A] wordFramedBy :: Monad m => (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
859− [A] wordBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
860− [A] toFold :: Monad m => Parser a m b -> Fold m a b
861− [A] takeWhileP :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
862− [A] takeWhile1 :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
863− [A] takeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
864− [A] takeStartBy_ :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
865− [A] takeStartBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
866− [A] takeP :: Monad m => Int -> Parser a m b -> Parser a m b
867− [A] takeGE :: Monad m => Int -> Fold m a b -> Parser a m b
868− [A] takeFramedBy_ :: Monad m => (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
869− [A] takeFramedByGeneric :: Monad m => Maybe (a -> Bool) -> Maybe (a -> Bool) -> Maybe (a -> Bool) -> Fold m a b -> Parser a m b
870− [A] takeFramedByEsc_ :: Monad m => (a -> Bool) -> (a -> Bool) -> (a -> Bool) -> Fold m a b -> Parser a m b
871− [A] takeEndBy_ :: (a -> Bool) -> Parser a m b -> Parser a m b
872− [A] takeEndByEsc :: Monad m => (a -> Bool) -> (a -> Bool) -> Parser a m b -> Parser a m b
873− [A] takeEndBy :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
874− [A] takeEitherSepBy :: (a -> Bool) -> Fold m (Either a b) c -> Parser a m c
875− [A] takeEQ :: Monad m => Int -> Fold m a b -> Parser a m b
876− [A] takeBetween :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
877− [A] subsequenceBy :: (a -> a -> Bool) -> Stream m a -> Parser a m ()
878− [A] streamEqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Parser a m ()
879− [A] split_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
880− [A] splitWith :: Monad m => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
881− [A] splitSome :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
882− [A] splitManyPost :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
883− [A] splitMany :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
884− [A] spanByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
885− [A] spanBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
886− [A] span :: Monad m => (a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (b, c)
887− [A] some :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
888− [A] sequence :: Monad m => Stream m (Parser a m b) -> Fold m b c -> Parser a m c
889− [A] sepByAll :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
890− [A] sepBy1 :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
891− [A] sepBy :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
892− [A] satisfy :: Monad m => (a -> Bool) -> Parser a m a
893− [A] sampleFromthen :: Monad m => Int -> Int -> Fold m a b -> Parser a m b
894− [A] roundRobin :: t (Parser a m b) -> Fold m b c -> Parser a m c
895− [A] rmapM :: Monad m => (b -> m c) -> Parser a m b -> Parser a m c
896− [A] retryMaxTotal :: Int -> Parser a m b -> Fold m b c -> Parser a m c
897− [A] retryMaxSuccessive :: Int -> Parser a m b -> Fold m b c -> Parser a m c
898− [A] retry :: Parser a m b -> Parser a m b
899− [A] postscan :: Fold m a b -> Parser b m c -> Parser a m c
900− [A] peek :: Monad m => Parser a m a
901− [A] oneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
902− [A] oneNotEq :: (Monad m, Eq a) => a -> Parser a m a
903− [A] oneEq :: (Monad m, Eq a) => a -> Parser a m a
904− [A] one :: Monad m => Parser a m a
905− [A] noneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
906− [A] noErrorUnsafeSplit_ :: Monad m => Parser x m a -> Parser x m b -> Parser x m b
907− [A] noErrorUnsafeSplitWith :: Monad m => (a -> b -> c) -> Parser x m a -> Parser x m b -> Parser x m c
908− [A] noErrorUnsafeConcatMap :: Monad m => (b -> Parser a m c) -> Parser a m b -> Parser a m c
909− [A] maybe :: Monad m => (a -> Maybe b) -> Parser a m b
910− [A] manyTillP :: Parser a m b -> Parser a m x -> Parser b m c -> Parser a m c
911− [A] manyTill :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
912− [A] manyThen :: Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
913− [A] manyP :: Parser a m b -> Parser b m c -> Parser a m c
914− [A] many :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
915− [A] makeIndexFilter :: (Fold m (s, a) b -> Parser a m b) -> (((s, a) -> Bool) -> Fold m (s, a) b -> Fold m (s, a) b) -> ((s, a) -> Bool) -> Fold m a b -> Parser a m b
916− [A] lookAhead :: Monad m => Parser a m b -> Parser a m b
917− [A] lmapM :: Monad m => (a -> m b) -> Parser b m r -> Parser a m r
918− [A] lmap :: (a -> b) -> Parser b m r -> Parser a m r
919− [A] listEqBy :: Monad m => (a -> a -> Bool) -> [a] -> Parser a m [a]
920− [A] listEq :: (Monad m, Eq a) => [a] -> Parser a m [a]
921− [A] indexed :: forall m a b. Monad m => Fold m (Int, a) b -> Parser a m b
922− [A] groupByRollingEither :: Monad m => (a -> a -> Bool) -> Fold m a b -> Fold m a c -> Parser a m (Either b c)
923− [A] groupByRolling :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
924− [A] groupBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
925− [A] fromPure :: Monad m => b -> Parser a m b
926− [A] fromFoldMaybe :: Monad m => String -> Fold m a (Maybe b) -> Parser a m b
927− [A] fromFold :: Monad m => Fold m a b -> Parser a m b
928− [A] fromEffect :: Monad m => m b -> Parser a m b
929− [A] filter :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
930− [A] extractStep :: Monad m => (s -> m (Step s1 b)) -> Step s b -> m (Step s1 b)
931− [A] eof :: Monad m => Parser a m ()
932− [A] either :: Monad m => (a -> Either String b) -> Parser a m b
933− [A] dropWhile :: Monad m => (a -> Bool) -> Parser a m ()
934− [A] dieM :: Monad m => m String -> Parser a m b
935− [A] die :: Monad m => String -> Parser a m b
936− [A] deintercalateAll :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
937− [A] deintercalate1 :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
938− [A] deintercalate :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
939− [A] countBetween :: Int -> Int -> Parser a m b -> Fold m b c -> Parser a m c
940− [A] count :: Int -> Parser a m b -> Fold m b c -> Parser a m c
941− [A] concatMap :: Monad m => (b -> Parser a m c) -> Parser a m b -> Parser a m c
942− [A] blockWithQuotes :: (Monad m, Eq a) => (a -> Bool) -> (a -> Bool) -> a -> a -> Fold m a b -> Parser a m b
943− [A] bimapOverrideCount :: Int -> (s -> s1) -> (b -> b1) -> Step s b -> Step s1 b1
944− [A] alt :: Monad m => Parser x m a -> Parser x m a -> Parser x m a
945−[A] Streamly.Internal.Data.MutByteArray
946− [A] class Unbox a
947− [A] class SizeOfRep (f :: Type -> Type)
948− [A] class Serialize a
949− [A] TypeOfType
950− [A] UnitType :: Name -> TypeOfType
951− [A] TheType :: SimpleDataCon -> TypeOfType
952− [A] MultiType :: [SimpleDataCon] -> TypeOfType
953− [A] SimpleDataCon
954− [A] SimpleDataCon :: Name -> [Field] -> SimpleDataCon
955− [A] SerializeConfig
956− [A] [cfgRecordSyntaxWithHeader] :: SerializeConfig -> Bool
957− [A] [cfgInlineSize] :: SerializeConfig -> Maybe Inline
958− [A] [cfgInlineSerialize] :: SerializeConfig -> Maybe Inline
959− [A] [cfgInlineDeserialize] :: SerializeConfig -> Maybe Inline
960− [A] [cfgConstructorTagAsString] :: SerializeConfig -> Bool
961− [A] SerializeConfig :: Maybe Inline -> Maybe Inline -> Maybe Inline -> Bool -> Bool -> SerializeConfig
962− [A] PinnedState
963− [A] Unpinned :: PinnedState
964− [A] Pinned :: PinnedState
965− [A] MutByteArray
966− [A] MutByteArray :: MutableByteArray# RealWorld -> MutByteArray
967− [A] DataType
968− [A] [dtTvs] :: DataType -> [Name]
969− [A] [dtName] :: DataType -> Name
970− [A] [dtCxt] :: DataType -> Cxt
971− [A] [dtCons] :: DataType -> [DataCon]
972− [A] DataType :: Name -> [Name] -> Cxt -> [DataCon] -> DataType
973− [A] DataCon
974− [A] [dcTvs] :: DataCon -> [Name]
975− [A] [dcName] :: DataCon -> Name
976− [A] [dcFields] :: DataCon -> [(Maybe Name, Type)]
977− [A] [dcCxt] :: DataCon -> Cxt
978− [A] DataCon :: Name -> [Name] -> Cxt -> [(Maybe Name, Type)] -> DataCon
979− [A] BoundedPtr
980− [A] BoundedPtr :: MutByteArray -> Int -> Int -> BoundedPtr
981− [A] Streamly.Internal.Data.Serialize.Type.Serialize
982− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize Streamly.Internal.Data.MutByteArray.LiftedInteger
983− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize GHC.Num.Integer.Integer
984− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize a => Streamly.Internal.Data.Serialize.Type.Serialize (GHC.Maybe.Maybe a)
985− [A] instance Streamly.Internal.Data.Serialize.Type.Serialize (Data.Proxy.Proxy a)
986− [A] instance (Streamly.Internal.Data.Serialize.Type.Serialize a, Streamly.Internal.Data.Serialize.Type.Serialize b) => Streamly.Internal.Data.Serialize.Type.Serialize (Data.Either.Either a b)
987− [A] Peeker
988− [A] Peeker :: Builder BoundedPtr IO a -> Peeker a
989− [A] type Field = (Maybe Name, Type)
990− [A] xorCmp :: [Word8] -> Name -> Name -> Q Exp
991− [A] wListToString :: [Word8] -> String
992− [A] w8_int :: Word8 -> Int
993− [A] w32_int :: Word32 -> Int
994− [A] unpin :: MutByteArray -> IO MutByteArray
995− [A] typeOfType :: Type -> [DataCon] -> TypeOfType
996− [A] skipByte :: Peeker ()
997− [A] sizeOfRep :: SizeOfRep f => f x -> Int
998− [A] sizeOfMutableByteArray :: MutByteArray -> IO Int
999− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
1000− [A] simplifyDataCon :: DataCon -> SimpleDataCon
1001− [A] serializeW8List :: Name -> Name -> [Word8] -> Q Exp
1002− [A] serializeConfig :: SerializeConfig
1003− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
1004− [A] runPeeker :: Peeker a -> BoundedPtr -> IO a
1005− [A] reifyDataType :: Name -> Q DataType
1006− [A] readUnsafe :: Unbox a => Peeker a
1007− [A] read :: Unbox a => Peeker a
1008− [A] putSliceUnsafe :: MonadIO m => MutByteArray -> Int -> MutByteArray -> Int -> Int -> m ()
1009− [A] pokeRep :: PokeRep f => f a -> BoundedPtr -> IO BoundedPtr
1010− [A] pokeBoundedPtrUnsafe :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr
1011− [A] pokeBoundedPtr :: forall a. Unbox a => a -> BoundedPtr -> IO BoundedPtr
1012− [A] pokeAt :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutByteArray -> a -> IO ()
1013− [A] pinnedNewAlignedBytes :: Int -> Int -> IO MutByteArray
1014− [A] pinnedNew :: Int -> IO MutByteArray
1015− [A] pinnedCloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
1016− [A] pin :: MutByteArray -> IO MutByteArray
1017− [A] peekRep :: PeekRep f => Peeker (f x)
1018− [A] peekAt :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutByteArray -> IO a
1019− [A] openConstructor :: Name -> Int -> Q Pat
1020− [A] nil :: MutByteArray
1021− [A] newBytesAs :: PinnedState -> Int -> IO MutByteArray
1022− [A] new :: Int -> IO MutByteArray
1023− [A] mkSerializeExprFields :: Name -> [Field] -> Q Exp
1024− [A] mkRecSizeOfExpr :: SimpleDataCon -> Q Exp
1025− [A] mkRecSerializeExpr :: Name -> SimpleDataCon -> Q Exp
1026− [A] mkRecDeserializeExpr :: Name -> Name -> Name -> SimpleDataCon -> Q Exp
1027− [A] mkFieldName :: Int -> Name
1028− [A] mkDeserializeKeysDec :: Name -> Name -> SimpleDataCon -> Q [Dec]
1029− [A] mkDeserializeExprOne :: Name -> SimpleDataCon -> Q Exp
1030− [A] matchConstructor :: Name -> Int -> Q Exp -> Q Match
1031− [A] makeN :: Int -> Name
1032− [A] makeI :: Int -> Name
1033− [A] makeA :: Int -> Name
1034− [A] litProxy :: Unbox a => Proxy a -> Q Exp
1035− [A] litIntegral :: Integral a => a -> Q Exp
1036− [A] isUnitType :: [DataCon] -> Bool
1037− [A] isRecordSyntax :: SimpleDataCon -> Bool
1038− [A] isPinned :: MutByteArray -> Bool
1039− [A] int_w8 :: Int -> Word8
1040− [A] int_w32 :: Int -> Word32
1041− [A] inlineSerializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
1042− [A] inlineDeserializeAt :: Maybe Inline -> SerializeConfig -> SerializeConfig
1043− [A] inlineAddSizeTo :: Maybe Inline -> SerializeConfig -> SerializeConfig
1044− [A] getMutableByteArray# :: MutByteArray -> MutableByteArray# RealWorld
1045− [A] genericSizeOf :: forall a. SizeOfRep (Rep a) => Proxy a -> Int
1046− [A] genericPokeByteIndex :: (Generic a, PokeRep (Rep a)) => MutByteArray -> Int -> a -> IO ()
1047− [A] genericPeekByteIndex :: (Generic a, PeekRep (Rep a)) => MutByteArray -> Int -> IO a
1048− [A] errorUnsupported :: String -> a
1049− [A] errorUnimplemented :: a
1050− [A] encodeRecordFields :: Bool -> SerializeConfig -> SerializeConfig
1051− [A] encodeConstrNames :: Bool -> SerializeConfig -> SerializeConfig
1052− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
1053− [A] deriveUnbox :: Q [Dec] -> Q [Dec]
1054− [A] deriveSerializeWith :: (SerializeConfig -> SerializeConfig) -> Q [Dec] -> Q [Dec]
1055− [A] deriveSerialize :: Q [Dec] -> Q [Dec]
1056− [A] conUpdateFuncDec :: Name -> [Field] -> Q [Dec]
1057− [A] cloneSliceUnsafeAs :: MonadIO m => PinnedState -> Int -> Int -> MutByteArray -> m MutByteArray
1058− [A] cloneSliceUnsafe :: MonadIO m => Int -> Int -> MutByteArray -> m MutByteArray
1059− [A] c2w :: Char -> Word8
1060− [A] asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
1061− [A] addSizeTo :: Serialize a => Int -> a -> Int
1062− [A] _x :: Name
1063− [A] _val :: Name
1064− [A] _tag :: Name
1065− [A] _initialOffset :: Name
1066− [A] _endOffset :: Name
1067− [A] _arr :: Name
1068− [A] _acc :: Name
1069−[A] Streamly.Internal.Data.MutArray.Stream
1070− [A] SpliceState
1071− [A] SpliceYielding :: arr -> SpliceState s arr -> SpliceState s arr
1072− [A] SpliceInitial :: s -> SpliceState s arr
1073− [A] SpliceFinish :: SpliceState s arr
1074− [A] SpliceBuffering :: s -> arr -> SpliceState s arr
1075− [A] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
1076− [A] splitOn :: (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> Stream m (MutArray a)
1077− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1078− [A] packArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1079− [A] lpackArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
1080− [A] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
1081− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1082− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1083− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (Either ParseError (MutArray a))
1084− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1085− [A] compactEQ :: Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1086− [A] compact :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
1087− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1088−[A] Streamly.Internal.Data.MutArray.Generic
1089− [A] MutArray
1090− [A] [arrTrueLen] :: MutArray a -> {-# UNPACK #-} !Int
1091− [A] [arrStart] :: MutArray a -> {-# UNPACK #-} !Int
1092− [A] [arrLen] :: MutArray a -> {-# UNPACK #-} !Int
1093− [A] [arrContents#] :: MutArray a -> MutableArray# RealWorld a
1094− [A] MutArray :: MutableArray# RealWorld a -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> MutArray a
1095− [A] writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
1096− [A] writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
1097− [A] writeN :: MonadIO m => Int -> Fold m a (MutArray a)
1098− [A] write :: MonadIO m => Fold m a (MutArray a)
1099− [A] uninit :: MonadIO m => MutArray a -> Int -> m (MutArray a)
1100− [A] toStreamK :: MonadIO m => MutArray a -> StreamK m a
1101− [A] toList :: MonadIO m => MutArray a -> m [a]
1102− [A] strip :: MonadIO m => (a -> Bool) -> MutArray a -> m (MutArray a)
1103− [A] snocWith :: MonadIO m => (Int -> Int) -> MutArray a -> a -> m (MutArray a)
1104− [A] snocUnsafe :: MonadIO m => MutArray a -> a -> m (MutArray a)
1105− [A] snoc :: MonadIO m => MutArray a -> a -> m (MutArray a)
1106− [A] realloc :: MonadIO m => Int -> MutArray a -> m (MutArray a)
1107− [A] reader :: MonadIO m => Unfold m (MutArray a) a
1108− [A] readRev :: MonadIO m => MutArray a -> Stream m a
1109− [A] read :: MonadIO m => MutArray a -> Stream m a
1110− [A] putSliceUnsafe :: MonadIO m => MutArray a -> Int -> MutArray a -> Int -> Int -> m ()
1111− [A] putIndices :: MonadIO m => MutArray a -> Fold m (Int, a) ()
1112− [A] putIndexUnsafe :: forall m a. MonadIO m => Int -> MutArray a -> a -> m ()
1113− [A] putIndex :: MonadIO m => Int -> MutArray a -> a -> m ()
1114− [A] producerWith :: Monad m => (forall b. IO b -> m b) -> Producer m (MutArray a) a
1115− [A] producer :: MonadIO m => Producer m (MutArray a) a
1116− [A] nil :: MonadIO m => m (MutArray a)
1117− [A] new :: MonadIO m => Int -> m (MutArray a)
1118− [A] modifyIndexUnsafe :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
1119− [A] modifyIndex :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
1120− [A] length :: MutArray a -> Int
1121− [A] getSliceUnsafe :: Int -> Int -> MutArray a -> MutArray a
1122− [A] getSlice :: Int -> Int -> MutArray a -> MutArray a
1123− [A] getIndexUnsafeWith :: MonadIO m => MutableArray# RealWorld a -> Int -> m a
1124− [A] getIndexUnsafe :: MonadIO m => Int -> MutArray a -> m a
1125− [A] getIndex :: MonadIO m => Int -> MutArray a -> m (Maybe a)
1126− [A] fromStreamN :: MonadIO m => Int -> Stream m a -> m (MutArray a)
1127− [A] fromStream :: MonadIO m => Stream m a -> m (MutArray a)
1128− [A] fromPureStream :: MonadIO m => Stream Identity a -> m (MutArray a)
1129− [A] fromListN :: MonadIO m => Int -> [a] -> m (MutArray a)
1130− [A] fromList :: MonadIO m => [a] -> m (MutArray a)
1131− [A] eq :: (MonadIO m, Eq a) => MutArray a -> MutArray a -> m Bool
1132− [A] cmp :: (MonadIO m, Ord a) => MutArray a -> MutArray a -> m Ordering
1133− [A] clone :: MonadIO m => MutArray a -> m (MutArray a)
1134− [A] chunksOf :: forall m a. MonadIO m => Int -> Stream m a -> Stream m (MutArray a)
1135−[A] Streamly.Internal.Data.MutArray
1136− [A] MutByteArray
1137− [A] MutArray
1138− [A] [arrStart] :: MutArray a -> {-# UNPACK #-} !Int
1139− [A] [arrEnd] :: MutArray a -> {-# UNPACK #-} !Int
1140− [A] [arrContents] :: MutArray a -> {-# UNPACK #-} !MutByteArray
1141− [A] [arrBound] :: MutArray a -> {-# UNPACK #-} !Int
1142− [A] MutArray :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> MutArray a
1143− [A] IORef
1144− [A] ArrayUnsafe
1145− [A] ArrayUnsafe :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> ArrayUnsafe a
1146− [A] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1147− [A] writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1148− [A] writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
1149− [A] writeNWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
1150− [A] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1151− [A] writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1152− [A] writeIORef :: Unbox a => IORef a -> a -> IO ()
1153− [A] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
1154− [A] writeAppendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
1155− [A] writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
1156− [A] writeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
1157− [A] writeAppend :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
1158− [A] write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
1159− [A] unsafeSwapIndices :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> MutArray a -> m ()
1160− [A] unpin :: MutArray a -> IO (MutArray a)
1161− [A] toStreamKWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> StreamK m a
1162− [A] toStreamKRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> StreamK m a
1163− [A] toStreamKRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
1164− [A] toStreamK :: forall m a. (MonadIO m, Unbox a) => MutArray a -> StreamK m a
1165− [A] toStreamDWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
1166− [A] toStreamDRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
1167− [A] toList :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m [a]
1168− [A] swapIndices :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> MutArray a -> m ()
1169− [A] strip :: forall a m. (Unbox a, MonadIO m) => (a -> Bool) -> MutArray a -> m (MutArray a)
1170− [A] splitOn :: (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> Stream m (MutArray a)
1171− [A] splitAt :: forall a. Unbox a => Int -> MutArray a -> (MutArray a, MutArray a)
1172− [A] spliceWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int -> Int) -> MutArray a -> MutArray a -> m (MutArray a)
1173− [A] spliceUnsafe :: MonadIO m => MutArray a -> MutArray a -> m (MutArray a)
1174− [A] spliceExp :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
1175− [A] spliceCopy :: forall m a. MonadIO m => MutArray a -> MutArray a -> m (MutArray a)
1176− [A] splice :: (MonadIO m, Unbox a) => MutArray a -> MutArray a -> m (MutArray a)
1177− [A] snocWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> MutArray a -> a -> m (MutArray a)
1178− [A] snocUnsafe :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1179− [A] snocMay :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (Maybe (MutArray a))
1180− [A] snocLinear :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1181− [A] snoc :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
1182− [A] shuffleBy :: (a -> a -> m Bool) -> MutArray a -> MutArray a -> m ()
1183− [A] roundUpToPower2 :: Int -> Int
1184− [A] rightSize :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m (MutArray a)
1185− [A] reverse :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m ()
1186− [A] resizeExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1187− [A] resize :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1188− [A] realloc :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
1189− [A] readerRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Unfold m (MutArray a) a
1190− [A] readerRev :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
1191− [A] reader :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
1192− [A] readRev :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
1193− [A] readIORef :: Unbox a => IORef a -> IO a
1194− [A] read :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Stream m a
1195− [A] putIndices :: forall m a. (MonadIO m, Unbox a) => MutArray a -> Fold m (Int, a) ()
1196− [A] putIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
1197− [A] putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
1198− [A] producerWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Producer m (MutArray a) a
1199− [A] producer :: forall m a. (MonadIO m, Unbox a) => Producer m (MutArray a) a
1200− [A] pollIntIORef :: (MonadIO m, Unbox a) => IORef a -> Stream m a
1201− [A] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1202− [A] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (MutArray a)
1203− [A] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
1204− [A] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
1205− [A] pinnedNewBytes :: MonadIO m => Int -> m (MutArray a)
1206− [A] pinnedNewAligned :: (MonadIO m, Unbox a) => Int -> Int -> m (MutArray a)
1207− [A] pinnedNew :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
1208− [A] pinnedFromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1209− [A] pinnedFromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1210− [A] pinnedClone :: MonadIO m => MutArray a -> m (MutArray a)
1211− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1212− [A] pin :: MutArray a -> IO (MutArray a)
1213− [A] permute :: MutArray a -> m Bool
1214− [A] partitionBy :: forall m a. (MonadIO m, Unbox a) => (a -> Bool) -> MutArray a -> m (MutArray a, MutArray a)
1215− [A] nil :: MutArray a
1216− [A] newIORef :: forall a. Unbox a => a -> IO (IORef a)
1217− [A] newArrayWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int -> m MutByteArray) -> Int -> Int -> m (MutArray a)
1218− [A] new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
1219− [A] modifyIndices :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (Int -> a -> a) -> Fold m Int ()
1220− [A] modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
1221− [A] modifyIndex :: forall m a b. (MonadIO m, Unbox a) => Int -> MutArray a -> (a -> (a, b)) -> m b
1222− [A] modifyIORef' :: Unbox a => IORef a -> (a -> a) -> IO ()
1223− [A] modify :: forall m a. (MonadIO m, Unbox a) => MutArray a -> (a -> a) -> m ()
1224− [A] mergeBy :: Int -> (MutArray a -> MutArray a -> m ()) -> MutArray a -> m ()
1225− [A] memcpy :: Ptr Word8 -> Ptr Word8 -> Int -> IO ()
1226− [A] memcmp :: Ptr Word8 -> Ptr Word8 -> Int -> IO Bool
1227− [A] length :: forall a. Unbox a => MutArray a -> Int
1228− [A] isPinned :: MutArray a -> Bool
1229− [A] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
1230− [A] getSliceUnsafe :: forall a. Unbox a => Int -> Int -> MutArray a -> MutArray a
1231− [A] getSlice :: forall a. Unbox a => Int -> Int -> MutArray a -> MutArray a
1232− [A] getIndicesD :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
1233− [A] getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
1234− [A] getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1235− [A] getIndexRev :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
1236− [A] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (Maybe a)
1237− [A] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
1238− [A] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
1239− [A] fromStreamD :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
1240− [A] fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
1241− [A] fromPureStream :: (MonadIO m, Unbox a) => Stream Identity a -> m (MutArray a)
1242− [A] fromListRevN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1243− [A] fromListRev :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1244− [A] fromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
1245− [A] fromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
1246− [A] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
1247− [A] foldr :: (MonadIO m, Unbox a) => (a -> b -> b) -> b -> MutArray a -> m b
1248− [A] foldl' :: (MonadIO m, Unbox a) => (b -> a -> b) -> b -> MutArray a -> m b
1249− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1250− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
1251− [A] divideBy :: Int -> (MutArray a -> m (MutArray a, MutArray a)) -> MutArray a -> m ()
1252− [A] cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
1253− [A] clone :: MonadIO m => MutArray a -> m (MutArray a)
1254− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (MutArray a)
1255− [A] castUnsafe :: MutArray a -> MutArray b
1256− [A] cast :: forall a b. Unbox b => MutArray a -> Maybe (MutArray b)
1257− [A] c_memchr :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)
1258− [A] bytesFree :: MutArray a -> Int
1259− [A] byteLength :: MutArray a -> Int
1260− [A] byteCapacity :: MutArray a -> Int
1261− [A] bubble :: (MonadIO m, Unbox a) => (a -> a -> Ordering) -> MutArray a -> m ()
1262− [A] breakOn :: MonadIO m => Word8 -> MutArray Word8 -> m (MutArray Word8, Maybe (MutArray Word8))
1263− [A] blockSize :: Int
1264− [A] asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
1265− [A] asBytes :: MutArray a -> MutArray Word8
1266− [A] arrayChunkBytes :: Int
1267− [A] allocBytesToElemCount :: Unbox a => a -> Int -> Int
1268−[C] Streamly.Internal.Data.IsMap
1269− [A] mapTraverseWithKey :: (IsMap f, Applicative t) => (Key f -> a -> t b) -> f a -> t (f b)
1270−[R] Streamly.Internal.Data.IORef.Unboxed
1271−[R] Streamly.Internal.Data.Fold.Window
1272−[R] Streamly.Internal.Data.Fold.Type
1273−[R] Streamly.Internal.Data.Fold.Tee
1274−[R] Streamly.Internal.Data.Fold.Step
1275−[R] Streamly.Internal.Data.Fold.Container
1276−[D] Streamly.Internal.Data.Fold.Chunked
1277−[C] Streamly.Internal.Data.Fold
1278− [A] ManyState
1279− [C] Fold
1280− [C] Fold
1281− [O] Fold :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> Fold m a b
1282− [N] Fold :: (s -> a -> m (Step s b)) -> m (Step s b) -> (s -> m b) -> (s -> m b) -> Fold m a b
1283− [A] windowSumInt :: forall m a. (Monad m, Integral a) => Fold m (a, Maybe a) a
1284− [A] windowSum :: forall m a. (Monad m, Num a) => Fold m (a, Maybe a) a
1285− [A] windowRollingMapM :: Monad m => (Maybe a -> a -> m (Maybe b)) -> Fold m (a, Maybe a) (Maybe b)
1286− [A] windowRollingMap :: Monad m => (Maybe a -> a -> Maybe b) -> Fold m (a, Maybe a) (Maybe b)
1287− [A] windowRange :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe (a, a))
1288− [A] windowPowerSumFrac :: (Monad m, Floating a) => a -> Fold m (a, Maybe a) a
1289− [A] windowPowerSum :: (Monad m, Num a) => Int -> Fold m (a, Maybe a) a
1290− [A] windowMinimum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
1291− [A] windowMean :: forall m a. (Monad m, Fractional a) => Fold m (a, Maybe a) a
1292− [A] windowMaximum :: (MonadIO m, Storable a, Ord a) => Int -> Fold m a (Maybe a)
1293− [A] windowLmap :: (c -> a) -> Fold m (a, Maybe a) b -> Fold m (c, Maybe c) b
1294− [A] windowLength :: (Monad m, Num b) => Fold m (a, Maybe a) b
1295− [A] toSet :: (Monad m, Ord a) => Fold m a (Set a)
1296− [A] toMapIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
1297− [A] toMap :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
1298− [A] toIntSet :: Monad m => Fold m Int IntSet
1299− [A] toContainerIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (f b)
1300− [A] toContainer :: (Monad m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (f b)
1301− [A] nubInt :: Monad m => Fold m Int (Maybe Int)
1302− [A] nub :: (Monad m, Ord a) => Fold m a (Maybe a)
1303− [A] mapMStep :: Applicative m => (a -> m b) -> Step s a -> m (Step s b)
1304− [A] kvToMap :: (Monad m, Ord k) => Fold m a b -> Fold m (k, a) (Map k b)
1305− [A] frequency :: (Monad m, Ord a) => Fold m a (Map a Int)
1306− [A] demuxToMapIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
1307− [A] demuxToMap :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
1308− [A] demuxToContainerIO :: (MonadIO m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
1309− [A] demuxToContainer :: (Monad m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (f b)
1310− [A] demuxKvToMap :: (Monad m, Ord k) => (k -> m (Fold m a b)) -> Fold m (k, a) (Map k b)
1311− [A] demuxKvToContainer :: (Monad m, IsMap f, Traversable f) => (Key f -> m (Fold m a b)) -> Fold m (Key f, a) (f b)
1312− [A] demuxIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
1313− [A] demuxGenericIO :: (MonadIO m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (m (f b), Maybe (Key f, b))
1314− [A] demuxGeneric :: (Monad m, IsMap f, Traversable f) => (a -> Key f) -> (a -> m (Fold m a b)) -> Fold m a (m (f b), Maybe (Key f, b))
1315− [A] demux :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
1316− [A] cumulative :: Fold m (a, Maybe a) b -> Fold m a b
1317− [A] countDistinctInt :: Monad m => Fold m Int Int
1318− [A] countDistinct :: (Monad m, Ord a) => Fold m a Int
1319− [A] classifyIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
1320− [A] classifyGenericIO :: (MonadIO m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
1321− [A] classifyGeneric :: (Monad m, IsMap f, Traversable f, Ord (Key f)) => (a -> Key f) -> Fold m a b -> Fold m a (m (f b), Maybe (Key f, b))
1322− [A] classify :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
1323− [A] chainStepM :: Applicative m => (s1 -> m s2) -> (a -> m (Step s2 b)) -> Step s1 a -> m (Step s2 b)
1324−[C] Streamly.Internal.Data.Builder
1325− [C] Builder
1326− [C] Builder
1327− [O] Builder :: (s -> m (s, a)) -> Builder s m a
1328− [N] Builder :: (s -> m (a, s)) -> Builder s m a
1329−[A] Streamly.Internal.Data.Binary.Stream
1330− [A] class ToBytes a
1331− [A] word8 :: Applicative m => Word8 -> Stream m Word8
1332− [A] word64le :: Monad m => Word64 -> Stream m Word8
1333− [A] word64host :: Monad m => Word64 -> Stream m Word8
1334− [A] word64be :: Monad m => Word64 -> Stream m Word8
1335− [A] word32le :: Monad m => Word32 -> Stream m Word8
1336− [A] word32be :: Monad m => Word32 -> Stream m Word8
1337− [A] word16le :: Monad m => Word16 -> Stream m Word8
1338− [A] word16be :: Monad m => Word16 -> Stream m Word8
1339− [A] unit :: Applicative m => Stream m Word8
1340− [A] toBytes :: ToBytes a => a -> Stream m Word8
1341− [A] ordering :: Applicative m => Ordering -> Stream m Word8
1342− [A] int8 :: Applicative m => Int8 -> Stream m Word8
1343− [A] int64le :: Monad m => Int64 -> Stream m Word8
1344− [A] int64be :: Monad m => Int64 -> Stream m Word8
1345− [A] int32le :: Monad m => Int32 -> Stream m Word8
1346− [A] int32be :: Monad m => Int32 -> Stream m Word8
1347− [A] int16le :: Monad m => Int16 -> Stream m Word8
1348− [A] int16be :: Monad m => Int16 -> Stream m Word8
1349− [A] float32le :: Monad m => Float -> Stream m Word8
1350− [A] float32be :: Monad m => Float -> Stream m Word8
1351− [A] double64le :: Monad m => Double -> Stream m Word8
1352− [A] double64be :: Monad m => Double -> Stream m Word8
1353− [A] charUtf8 :: Monad m => Char -> Stream m Word8
1354− [A] charLatin1 :: Applicative m => Char -> Stream m Word8
1355− [A] bool :: Applicative m => Bool -> Stream m Word8
1356−[A] Streamly.Internal.Data.Binary.Parser
1357− [A] class FromBytes a
1358− [A] word8 :: Monad m => Parser Word8 m Word8
1359− [A] word64le :: Monad m => Parser Word8 m Word64
1360− [A] word64host :: MonadIO m => Parser Word8 m Word64
1361− [A] word64be :: Monad m => Parser Word8 m Word64
1362− [A] word32le :: Monad m => Parser Word8 m Word32
1363− [A] word32be :: Monad m => Parser Word8 m Word32
1364− [A] word16le :: Monad m => Parser Word8 m Word16
1365− [A] word16be :: Monad m => Parser Word8 m Word16
1366− [A] unit :: Monad m => Parser Word8 m ()
1367− [A] ordering :: Monad m => Parser Word8 m Ordering
1368− [A] int8 :: Monad m => Parser Word8 m Int8
1369− [A] int64le :: Monad m => Parser Word8 m Int64
1370− [A] int64be :: Monad m => Parser Word8 m Int64
1371− [A] int32le :: Monad m => Parser Word8 m Int32
1372− [A] int32be :: Monad m => Parser Word8 m Int32
1373− [A] int16le :: Monad m => Parser Word8 m Int16
1374− [A] int16be :: Monad m => Parser Word8 m Int16
1375− [A] fromBytes :: FromBytes a => Parser Word8 m a
1376− [A] float32le :: MonadIO m => Parser Word8 m Float
1377− [A] float32be :: MonadIO m => Parser Word8 m Float
1378− [A] eqWord8 :: Monad m => Word8 -> Parser Word8 m Word8
1379− [A] double64le :: MonadIO m => Parser Word8 m Double
1380− [A] double64be :: MonadIO m => Parser Word8 m Double
1381− [A] charLatin1 :: Monad m => Parser Word8 m Char
1382− [A] bool :: Monad m => Parser Word8 m Bool
1383−[R] Streamly.Internal.Data.Array.Type
1384−[A] Streamly.Internal.Data.Array.Stream
1385− [A] unlines :: forall m a. (MonadIO m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1386− [A] toArray :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
1387− [A] splitOnSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
1388− [A] splitOn :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
1389− [A] runArrayParserDBreak :: forall m a b. (MonadIO m, Unbox a) => Parser (Array a) m b -> Stream m (Array a) -> m (Either ParseError b, Stream m (Array a))
1390− [A] runArrayFoldMany :: (Monad m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> StreamK m (Either ParseError b)
1391− [A] runArrayFoldBreak :: (MonadIO m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1392− [A] runArrayFold :: (MonadIO m, Unbox a) => ChunkFold m a b -> StreamK m (Array a) -> m (Either ParseError b)
1393− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1394− [A] parseChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b)
1395− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK (Array a) m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1396− [A] parseBreak :: (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
1397− [A] lpackArraysChunksOf :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
1398− [A] interposeSuffix :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1399− [A] interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
1400− [A] intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a
1401− [A] foldBreakD :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
1402− [A] foldBreak :: (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
1403− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1404− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1405− [A] concatRev :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
1406− [A] concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
1407− [A] compact :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
1408− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1409− [A] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
1410−[R] Streamly.Internal.Data.Array.Mut.Type
1411−[R] Streamly.Internal.Data.Array.Mut.Stream
1412−[R] Streamly.Internal.Data.Array.Mut
1413−[R] Streamly.Internal.Data.Array.Generic.Mut.Type
1414−[C] Streamly.Internal.Data.Array.Generic
1415− [A] getIndex :: Int -> Array a -> Maybe a
1416− [A] fromPureStream :: Stream Identity a -> Array a
1417− [A] fromByteStr# :: Addr# -> Array Word8
1418− [A] chunksOf :: forall m a. MonadIO m => Int -> Stream m a -> Stream m (Array a)
1419−[C] Streamly.Internal.Data.Array
1420− [A] ArrayUnsafe
1421− [A] ArrayUnsafe :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> ArrayUnsafe a
1422− [C] Array
1423− [A] [arrStart] :: Array a -> {-# UNPACK #-} !Int
1424− [A] [arrEnd] :: Array a -> {-# UNPACK #-} !Int
1425− [A] [arrContents] :: Array a -> {-# UNPACK #-} !MutByteArray
1426− [A] Array :: {-# UNPACK #-} !MutByteArray -> {-# UNPACK #-} !Int -> {-# UNPACK #-} !Int -> Array a
1427− [A] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1428− [A] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1429− [R] writeNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
1430− [A] unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b
1431− [A] unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a
1432− [D] unsafeIndex :: forall a. Unbox a => Int -> Array a -> a
1433− [A] unsafeFreezeWithShrink :: Unbox a => MutArray a -> Array a
1434− [A] unpin :: Array a -> IO (Array a)
1435− [A] toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a
1436− [A] toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a
1437− [A] toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
1438− [A] toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
1439− [A] splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)
1440− [A] splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
1441− [A] serialize :: Serialize a => a -> Array Word8
1442− [A] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1443− [A] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
1444− [A] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1445− [A] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
1446− [A] pinnedSerialize :: Serialize a => a -> Array Word8
1447− [A] pinnedFromListN :: Unbox a => Int -> [a] -> Array a
1448− [A] pinnedFromList :: Unbox a => [a] -> Array a
1449− [A] pinnedClone :: MonadIO m => Array a -> m (Array a)
1450− [A] pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1451− [A] pin :: Array a -> IO (Array a)
1452− [A] nil :: Array a
1453− [A] isPinned :: Array a -> Bool
1454− [A] getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a
1455− [A] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
1456− [A] fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)
1457− [A] fromPureStream :: Unbox a => Stream Identity a -> Array a
1458− [A] fromListRevN :: Unbox a => Int -> [a] -> Array a
1459− [A] fromListRev :: Unbox a => [a] -> Array a
1460− [A] fromByteStr# :: Addr# -> Array Word8
1461− [A] foldr :: Unbox a => (a -> b -> b) -> b -> Array a -> b
1462− [A] foldl' :: forall a b. Unbox a => (b -> a -> b) -> b -> Array a -> b
1463− [A] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1464− [A] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
1465− [A] encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8
1466− [A] deserialize :: Serialize a => Array Word8 -> a
1467− [A] clone :: MonadIO m => Array a -> m (Array a)
1468− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
1469− [A] byteLength :: Array a -> Int
1470− [A] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
1471− [A] breakOn :: MonadIO m => Word8 -> Array Word8 -> m (Array Word8, Maybe (Array Word8))
core/docs/ApiChangelogs/0.1.0.txtdeleted−405
@@ -1,405 +0,0 @@
1−---------------------------------
2−Terminology
3−---------------------------------
4−
5−[A]: Added
6−[R]: Removed
7−[C]: Changed
8−[D]: Deprecated
9−[O]: Old
10−[N]: New
11−
12−---------------------------------
13−Difference
14−---------------------------------
15−
16−The moved modules have been compared with their version in streamly v0.8.3
17−
18−[A] Streamly.Unicode.String
19− [A] str :: QuasiQuoter
20−[C] Streamly.Unicode.Stream
21− [C] encodeUtf8'
22− [O] encodeUtf8' :: (Monad m, IsStream t) => t m Char -> t m Word8
23− [N] encodeUtf8' :: Monad m => Stream m Char -> Stream m Word8
24− [C] encodeUtf8
25− [O] encodeUtf8 :: (Monad m, IsStream t) => t m Char -> t m Word8
26− [N] encodeUtf8 :: Monad m => Stream m Char -> Stream m Word8
27− [C] encodeStrings
28− [O] encodeStrings :: (MonadIO m, IsStream t) => (SerialT m Char -> SerialT m Word8) -> t m String -> t m (Array Word8)
29− [N] encodeStrings :: MonadIO m => (Stream m Char -> Stream m Word8) -> Stream m String -> Stream m (Array Word8)
30− [C] encodeLatin1'
31− [O] encodeLatin1' :: (IsStream t, Monad m) => t m Char -> t m Word8
32− [N] encodeLatin1' :: Monad m => Stream m Char -> Stream m Word8
33− [C] encodeLatin1
34− [O] encodeLatin1 :: (IsStream t, Monad m) => t m Char -> t m Word8
35− [N] encodeLatin1 :: Monad m => Stream m Char -> Stream m Word8
36− [A] decodeUtf8Chunks :: MonadIO m => Stream m (Array Word8) -> Stream m Char
37− [C] decodeUtf8'
38− [O] decodeUtf8' :: (Monad m, IsStream t) => t m Word8 -> t m Char
39− [N] decodeUtf8' :: Monad m => Stream m Word8 -> Stream m Char
40− [C] decodeUtf8
41− [O] decodeUtf8 :: (Monad m, IsStream t) => t m Word8 -> t m Char
42− [N] decodeUtf8 :: Monad m => Stream m Word8 -> Stream m Char
43− [C] decodeLatin1
44− [O] decodeLatin1 :: (IsStream t, Monad m) => t m Word8 -> t m Char
45− [N] decodeLatin1 :: Monad m => Stream m Word8 -> Stream m Char
46−[A] Streamly.Unicode.Parser
47− [A] upper :: Monad m => Parser Char m Char
48− [A] symbol :: Monad m => Parser Char m Char
49− [A] stringIgnoreCase :: Monad m => String -> Parser Char m String
50− [A] string :: Monad m => String -> Parser Char m String
51− [A] space :: Monad m => Parser Char m Char
52− [A] signed :: (Num a, Monad m) => Parser Char m a -> Parser Char m a
53− [A] separator :: Monad m => Parser Char m Char
54− [A] punctuation :: Monad m => Parser Char m Char
55− [A] printable :: Monad m => Parser Char m Char
56− [A] octDigit :: Monad m => Parser Char m Char
57− [A] numeric :: Monad m => Parser Char m Char
58− [A] mark :: Monad m => Parser Char m Char
59− [A] lower :: Monad m => Parser Char m Char
60− [A] letter :: Monad m => Parser Char m Char
61− [A] latin1 :: Monad m => Parser Char m Char
62− [A] hexadecimal :: (Monad m, Integral a, Bits a) => Parser Char m a
63− [A] hexDigit :: Monad m => Parser Char m Char
64− [A] dropSpace1 :: Monad m => Parser Char m ()
65− [A] dropSpace :: Monad m => Parser Char m ()
66− [A] digit :: Monad m => Parser Char m Char
67− [A] decimal :: (Monad m, Integral a) => Parser Char m a
68− [A] charIgnoreCase :: Monad m => Char -> Parser Char m Char
69− [A] char :: Monad m => Char -> Parser Char m Char
70− [A] asciiUpper :: Monad m => Parser Char m Char
71− [A] asciiLower :: Monad m => Parser Char m Char
72− [A] ascii :: Monad m => Parser Char m Char
73− [A] alphaNum :: Monad m => Parser Char m Char
74− [A] alpha :: Monad m => Parser Char m Char
75−[C] Streamly.FileSystem.Handle
76− [D] writeWithBufferOf :: MonadIO m => Int -> Handle -> Fold m Word8 ()
77− [A] writeWith :: MonadIO m => Int -> Handle -> Fold m Word8 ()
78− [C] writeChunks
79− [O] writeChunks :: (MonadIO m, Storable a) => Handle -> Fold m (Array a) ()
80− [N] writeChunks :: MonadIO m => Handle -> Fold m (Array a) ()
81− [A] readerWith :: MonadIO m => Unfold m (Int, Handle) Word8
82− [A] reader :: MonadIO m => Unfold m Handle Word8
83− [D] readWithBufferOf :: MonadIO m => Unfold m (Int, Handle) Word8
84− [D] readChunksWithBufferOf :: MonadIO m => Unfold m (Int, Handle) (Array Word8)
85− [D] readChunks :: MonadIO m => Unfold m Handle (Array Word8)
86− [D] read :: MonadIO m => Unfold m Handle Word8
87− [C] putChunk
88− [O] putChunk :: (MonadIO m, Storable a) => Handle -> Array a -> m ()
89− [N] putChunk :: MonadIO m => Handle -> Array a -> m ()
90− [A] chunkReaderWith :: MonadIO m => Unfold m (Int, Handle) (Array Word8)
91− [A] chunkReader :: MonadIO m => Unfold m Handle (Array Word8)
92−[A] Streamly.FileSystem.File
93− [A] writeWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Fold m Word8 ()
94− [A] writeChunks :: (MonadIO m, MonadCatch m) => FilePath -> Fold m (Array a) ()
95− [A] write :: (MonadIO m, MonadCatch m) => FilePath -> Fold m Word8 ()
96− [A] withFile :: (MonadIO m, MonadCatch m) => FilePath -> IOMode -> (Handle -> Stream m a) -> Stream m a
97− [A] readChunksWith :: (MonadIO m, MonadCatch m) => Int -> FilePath -> Stream m (Array Word8)
98− [A] readChunks :: (MonadIO m, MonadCatch m) => FilePath -> Stream m (Array Word8)
99− [A] read :: (MonadIO m, MonadCatch m) => FilePath -> Stream m Word8
100−[A] Streamly.FileSystem.Dir
101− [A] readEither :: MonadIO m => FilePath -> Stream m (Either FilePath FilePath)
102− [A] read :: MonadIO m => FilePath -> Stream m FilePath
103−[C] Streamly.Data.Unfold
104− [C] take
105− [O] take :: Monad m => Int -> Unfold m a b -> Unfold m a b
106− [N] take :: Applicative m => Int -> Unfold m a b -> Unfold m a b
107− [C] replicateM
108− [O] replicateM :: Monad m => Int -> Unfold m (m a) a
109− [N] replicateM :: Applicative m => Unfold m (Int, m a) a
110− [C] repeatM
111− [O] repeatM :: Monad m => Unfold m (m a) a
112− [N] repeatM :: Applicative m => Unfold m (m a) a
113− [C] many
114− [O] many :: Monad m => Unfold m a b -> Unfold m b c -> Unfold m a c
115− [N] many :: Monad m => Unfold m b c -> Unfold m a b -> Unfold m a c
116− [C] iterateM
117− [O] iterateM :: Monad m => (a -> m a) -> Unfold m (m a) a
118− [N] iterateM :: Applicative m => (a -> m a) -> Unfold m (m a) a
119− [C] fromStream
120− [O] fromStream :: (IsStream t, Monad m) => Unfold m (t m a) a
121− [N] fromStream :: Applicative m => Unfold m (Stream m a) a
122− [C] fromListM
123− [O] fromListM :: Monad m => Unfold m [m a] a
124− [N] fromListM :: Applicative m => Unfold m [m a] a
125− [C] fromList
126− [O] fromList :: Monad m => Unfold m [a] a
127− [N] fromList :: Applicative m => Unfold m [a] a
128− [C] drop
129− [O] drop :: Monad m => Int -> Unfold m a b -> Unfold m a b
130− [N] drop :: Applicative m => Int -> Unfold m a b -> Unfold m a b
131−[A] Streamly.Data.StreamK
132− [A] StreamK
133− [A] zipWithM :: Monad m => (a -> b -> m c) -> StreamK m a -> StreamK m b -> StreamK m c
134− [A] zipWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
135− [A] uncons :: Applicative m => StreamK m a -> m (Maybe (a, StreamK m a))
136− [A] toStream :: Applicative m => StreamK m a -> Stream m a
137− [A] sortBy :: Monad m => (a -> a -> Ordering) -> StreamK m a -> StreamK m a
138− [A] reverse :: StreamK m a -> StreamK m a
139− [A] parseChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b)
140− [A] parseBreakChunks :: (Monad m, Unbox a) => ParserK a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
141− [A] nilM :: Applicative m => m b -> StreamK m a
142− [A] nil :: StreamK m a
143− [A] mergeMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
144− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
145− [A] mergeBy :: (a -> a -> Ordering) -> StreamK m a -> StreamK m a -> StreamK m a
146− [A] interleave :: StreamK m a -> StreamK m a -> StreamK m a
147− [A] fromStream :: Monad m => Stream m a -> StreamK m a
148− [A] fromPure :: a -> StreamK m a
149− [A] fromFoldable :: Foldable f => f a -> StreamK m a
150− [A] fromEffect :: Monad m => m a -> StreamK m a
151− [A] crossWith :: Monad m => (a -> b -> c) -> StreamK m a -> StreamK m b -> StreamK m c
152− [A] consM :: Monad m => m a -> StreamK m a -> StreamK m a
153− [A] cons :: a -> StreamK m a -> StreamK m a
154− [A] concatMapWith :: (StreamK m b -> StreamK m b -> StreamK m b) -> (a -> StreamK m b) -> StreamK m a -> StreamK m b
155− [A] concatEffect :: Monad m => m (StreamK m a) -> StreamK m a
156− [A] append :: StreamK m a -> StreamK m a -> StreamK m a
157−[A] Streamly.Data.Stream
158− [A] class Enum a => Enumerable a
159− [A] Stream
160− [A] zipWithM :: Monad m => (a -> b -> m c) -> Stream m a -> Stream m b -> Stream m c
161− [A] zipWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
162− [A] unfoldrM :: Monad m => (s -> m (Maybe (a, s))) -> s -> Stream m a
163− [A] unfoldr :: Monad m => (s -> Maybe (a, s)) -> s -> Stream m a
164− [A] unfoldMany :: Monad m => Unfold m a b -> Stream m a -> Stream m b
165− [A] unfold :: Applicative m => Unfold m a b -> a -> Stream m b
166− [A] uncons :: Monad m => Stream m a -> m (Maybe (a, Stream m a))
167− [A] trace :: Monad m => (a -> m b) -> Stream m a -> Stream m a
168− [A] toList :: Monad m => Stream m a -> m [a]
169− [A] tap :: Monad m => Fold m a b -> Stream m a -> Stream m a
170− [A] takeWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
171− [A] takeWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
172− [A] take :: Applicative m => Int -> Stream m a -> Stream m a
173− [A] stripPrefix :: (Monad m, Eq a) => Stream m a -> Stream m a -> m (Maybe (Stream m a))
174− [A] sequence :: Monad m => Stream m (m a) -> Stream m a
175− [A] scanMaybe :: Monad m => Fold m a (Maybe b) -> Stream m a -> Stream m b
176− [A] scan :: Monad m => Fold m a b -> Stream m a -> Stream m b
177− [A] runStateT :: Monad m => m s -> Stream (StateT s m) a -> Stream m (s, a)
178− [A] runReaderT :: Monad m => m s -> Stream (ReaderT s m) a -> Stream m a
179− [A] reverse :: Monad m => Stream m a -> Stream m a
180− [A] replicateM :: Monad m => Int -> m a -> Stream m a
181− [A] replicate :: Monad m => Int -> a -> Stream m a
182− [A] repeatM :: Monad m => m a -> Stream m a
183− [A] repeat :: Monad m => a -> Stream m a
184− [A] postscan :: Monad m => Fold m a b -> Stream m a -> Stream m b
185− [A] parseMany :: Monad m => Parser a m b -> Stream m a -> Stream m (Either ParseError b)
186− [A] parse :: Monad m => Parser a m b -> Stream m a -> m (Either ParseError b)
187− [A] onException :: MonadCatch m => m b -> Stream m a -> Stream m a
188− [A] nilM :: Applicative m => m b -> Stream m a
189− [A] nil :: Applicative m => Stream m a
190− [A] morphInner :: Monad n => (forall x. m x -> n x) -> Stream m a -> Stream n a
191− [A] mergeByM :: Monad m => (a -> a -> m Ordering) -> Stream m a -> Stream m a -> Stream m a
192− [A] mergeBy :: Monad m => (a -> a -> Ordering) -> Stream m a -> Stream m a -> Stream m a
193− [A] mapMaybeM :: Monad m => (a -> m (Maybe b)) -> Stream m a -> Stream m b
194− [A] mapMaybe :: Monad m => (a -> Maybe b) -> Stream m a -> Stream m b
195− [A] mapM :: Monad m => (a -> m b) -> Stream m a -> Stream m b
196− [A] liftInner :: (Monad m, MonadTrans t, Monad (t m)) => Stream m a -> Stream (t m) a
197− [A] iterateM :: Monad m => (a -> m a) -> m a -> Stream m a
198− [A] iterate :: Monad m => (a -> a) -> a -> Stream m a
199− [A] isSubsequenceOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
200− [A] isPrefixOf :: (Monad m, Eq a) => Stream m a -> Stream m a -> m Bool
201− [A] intersperseM_ :: Monad m => m b -> Stream m a -> Stream m a
202− [A] intersperseM :: Monad m => m a -> Stream m a -> Stream m a
203− [A] intersperse :: Monad m => a -> Stream m a -> Stream m a
204− [A] interleave :: Monad m => Stream m a -> Stream m a -> Stream m a
205− [A] intercalateSuffix :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
206− [A] intercalate :: Monad m => Unfold m b c -> b -> Stream m b -> Stream m c
207− [A] insertBy :: Monad m => (a -> a -> Ordering) -> a -> Stream m a -> Stream m a
208− [A] indexed :: Monad m => Stream m a -> Stream m (Int, a)
209− [A] handle :: (MonadCatch m, Exception e) => (e -> Stream m a) -> Stream m a -> Stream m a
210− [A] fromPure :: Applicative m => a -> Stream m a
211− [A] fromList :: Applicative m => [a] -> Stream m a
212− [A] fromEffect :: Applicative m => m a -> Stream m a
213− [A] foldrM :: Monad m => (a -> m b -> m b) -> m b -> Stream m a -> m b
214− [A] foldr :: Monad m => (a -> b -> b) -> b -> Stream m a -> m b
215− [A] foldMany :: Monad m => Fold m a b -> Stream m a -> Stream m b
216− [A] foldBreak :: Monad m => Fold m a b -> Stream m a -> m (b, Stream m a)
217− [A] fold :: Monad m => Fold m a b -> Stream m a -> m b
218− [A] finallyIO :: (MonadIO m, MonadCatch m) => IO b -> Stream m a -> Stream m a
219− [A] filterM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
220− [A] filter :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
221− [A] eqBy :: Monad m => (a -> b -> Bool) -> Stream m a -> Stream m b -> m Bool
222− [A] enumerateTo :: (Monad m, Bounded a, Enumerable a) => a -> Stream m a
223− [A] enumerateFromTo :: (Enumerable a, Monad m) => a -> a -> Stream m a
224− [A] enumerateFromThenTo :: (Enumerable a, Monad m) => a -> a -> a -> Stream m a
225− [A] enumerateFromThen :: (Enumerable a, Monad m) => a -> a -> Stream m a
226− [A] enumerateFrom :: (Enumerable a, Monad m) => a -> Stream m a
227− [A] enumerate :: (Monad m, Bounded a, Enumerable a) => Stream m a
228− [A] dropWhileM :: Monad m => (a -> m Bool) -> Stream m a -> Stream m a
229− [A] dropWhile :: Monad m => (a -> Bool) -> Stream m a -> Stream m a
230− [A] drop :: Monad m => Int -> Stream m a -> Stream m a
231− [A] delay :: MonadIO m => Double -> Stream m a -> Stream m a
232− [A] crossWith :: Monad m => (a -> b -> c) -> Stream m a -> Stream m b -> Stream m c
233− [A] consM :: Applicative m => m a -> Stream m a -> Stream m a
234− [A] cons :: Applicative m => a -> Stream m a -> Stream m a
235− [A] concatMapM :: Monad m => (a -> m (Stream m b)) -> Stream m a -> Stream m b
236− [A] concatMap :: Monad m => (a -> Stream m b) -> Stream m a -> Stream m b
237− [A] concatEffect :: Monad m => m (Stream m a) -> Stream m a
238− [A] cmpBy :: Monad m => (a -> b -> Ordering) -> Stream m a -> Stream m b -> m Ordering
239− [A] chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)
240− [A] catRights :: Monad m => Stream m (Either a b) -> Stream m b
241− [A] catMaybes :: Monad m => Stream m (Maybe a) -> Stream m a
242− [A] catLefts :: Monad m => Stream m (Either a b) -> Stream m a
243− [A] catEithers :: Monad m => Stream m (Either a a) -> Stream m a
244− [A] bracketIO3 :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> IO d) -> (b -> IO e) -> (b -> Stream m a) -> Stream m a
245− [A] bracketIO :: (MonadIO m, MonadCatch m) => IO b -> (b -> IO c) -> (b -> Stream m a) -> Stream m a
246− [A] before :: Monad m => m b -> Stream m a -> Stream m a
247− [A] append :: Monad m => Stream m a -> Stream m a -> Stream m a
248− [A] afterIO :: MonadIO m => IO b -> Stream m a -> Stream m a
249−[A] Streamly.Data.ParserK
250− [A] ParserK
251− [A] fromPure :: b -> ParserK a m b
252− [A] fromParser :: (Monad m, Unbox a) => Parser a m b -> ParserK a m b
253− [A] fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK a m b
254− [A] fromEffect :: Monad m => m b -> ParserK a m b
255− [A] die :: String -> ParserK a m b
256−[A] Streamly.Data.Parser
257− [A] Parser
258− [A] wordWithQuotes :: (Monad m, Eq a) => Bool -> (a -> a -> Maybe a) -> a -> (a -> Maybe a) -> (a -> Bool) -> Fold m a b -> Parser a m b
259− [A] wordBy :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
260− [A] takeWhile1 :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
261− [A] takeWhile :: Monad m => (a -> Bool) -> Fold m a b -> Parser a m b
262− [A] takeEQ :: Monad m => Int -> Fold m a b -> Parser a m b
263− [A] streamEqBy :: Monad m => (a -> a -> Bool) -> Stream m a -> Parser a m ()
264− [A] some :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
265− [A] satisfy :: Monad m => (a -> Bool) -> Parser a m a
266− [A] rmapM :: Monad m => (b -> m c) -> Parser a m b -> Parser a m c
267− [A] peek :: Monad m => Parser a m a
268− [A] oneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
269− [A] one :: Monad m => Parser a m a
270− [A] noneOf :: (Monad m, Eq a, Foldable f) => f a -> Parser a m a
271− [A] manyTill :: Monad m => Parser a m b -> Parser a m x -> Fold m b c -> Parser a m c
272− [A] many :: Monad m => Parser a m b -> Fold m b c -> Parser a m c
273− [A] lookAhead :: Monad m => Parser a m b -> Parser a m b
274− [A] lmapM :: Monad m => (a -> m b) -> Parser b m r -> Parser a m r
275− [A] lmap :: (a -> b) -> Parser b m r -> Parser a m r
276− [A] listEqBy :: Monad m => (a -> a -> Bool) -> [a] -> Parser a m [a]
277− [A] listEq :: (Monad m, Eq a) => [a] -> Parser a m [a]
278− [A] groupBy :: Monad m => (a -> a -> Bool) -> Fold m a b -> Parser a m b
279− [A] fromPure :: Monad m => b -> Parser a m b
280− [A] fromFold :: Monad m => Fold m a b -> Parser a m b
281− [A] fromEffect :: Monad m => m b -> Parser a m b
282− [A] filter :: Monad m => (a -> Bool) -> Parser a m b -> Parser a m b
283− [A] eof :: Monad m => Parser a m ()
284− [A] dropWhile :: Monad m => (a -> Bool) -> Parser a m ()
285− [A] die :: Monad m => String -> Parser a m b
286− [A] deintercalate :: Monad m => Parser a m x -> Parser a m y -> Fold m (Either x y) z -> Parser a m z
287−[A] Streamly.Data.MutArray.Generic
288− [A] MutArray
289− [A] writeN :: MonadIO m => Int -> Fold m a (MutArray a)
290− [A] toList :: MonadIO m => MutArray a -> m [a]
291− [A] snoc :: MonadIO m => MutArray a -> a -> m (MutArray a)
292− [A] reader :: MonadIO m => Unfold m (MutArray a) a
293− [A] putIndex :: MonadIO m => Int -> MutArray a -> a -> m ()
294− [A] new :: forall m a. MonadIO m => Int -> m (MutArray a)
295− [A] modifyIndex :: MonadIO m => Int -> MutArray a -> (a -> (a, b)) -> m b
296− [A] getIndex :: MonadIO m => Int -> MutArray a -> m a
297−[A] Streamly.Data.MutArray
298− [A] class Unbox a
299− [A] MutArray
300− [A] writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
301− [A] writeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
302− [A] writeAppend :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
303− [A] write :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
304− [A] toList :: forall m a. (MonadIO m, Unbox a) => MutArray a -> m [a]
305− [A] snoc :: forall m a. (MonadIO m, Unbox a) => MutArray a -> a -> m (MutArray a)
306− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
307− [A] readerRev :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
308− [A] reader :: forall m a. (MonadIO m, Unbox a) => Unfold m (MutArray a) a
309− [A] putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> a -> m ()
310− [A] pokeByteIndex :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutableByteArray -> a -> IO ()
311− [A] peekByteIndex :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutableByteArray -> IO a
312− [A] newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
313− [A] new :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
314− [A] length :: forall a. Unbox a => MutArray a -> Int
315− [A] getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m a
316− [A] fromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (MutArray a)
317− [A] fromList :: (MonadIO m, Unbox a) => [a] -> m (MutArray a)
318− [A] cast :: forall a b. Unbox b => MutArray a -> Maybe (MutArray b)
319− [A] asBytes :: MutArray a -> MutArray Word8
320−[C] Streamly.Data.Fold
321− [A] Tee
322− [A] [unTee] :: Tee m a b -> Fold m a b
323− [A] Tee :: Fold m a b -> Tee m a b
324− [D] variance :: (Monad m, Fractional a) => Fold m a a
325− [A] uniqBy :: Monad m => (a -> a -> Bool) -> Fold m a (Maybe a)
326− [A] topBy :: (MonadIO m, Unbox a) => (a -> a -> Ordering) -> Int -> Fold m a (MutArray a)
327− [A] toSet :: (Monad m, Ord a) => Fold m a (Set a)
328− [A] toMapIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
329− [A] toMap :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (Map k b)
330− [A] toIntSet :: Monad m => Fold m Int IntSet
331− [A] the :: (Monad m, Eq a) => Fold m a (Maybe a)
332− [D] stdDev :: (Monad m, Floating a) => Fold m a a
333− [A] splitWith :: Monad m => (a -> b -> c) -> Fold m x a -> Fold m x b -> Fold m x c
334− [D] serialWith :: Monad m => (a -> b -> c) -> Fold m x a -> Fold m x b -> Fold m x c
335− [A] scanMaybe :: Monad m => Fold m a (Maybe b) -> Fold m b c -> Fold m a c
336− [A] scan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
337− [A] postscan :: Monad m => Fold m a b -> Fold m b c -> Fold m a c
338− [A] one :: Monad m => Fold m a (Maybe a)
339− [A] nubInt :: Monad m => Fold m Int (Maybe Int)
340− [A] nub :: (Monad m, Ord a) => Fold m a (Maybe a)
341− [A] morphInner :: (forall x. m x -> n x) -> Fold m a b -> Fold n a b
342− [A] latest :: Monad m => Fold m a (Maybe a)
343− [D] last :: Monad m => Fold m a (Maybe a)
344− [D] head :: Monad m => Fold m a (Maybe a)
345− [A] groupsOf :: Monad m => Int -> Fold m a b -> Fold m b c -> Fold m a c
346− [A] frequency :: (Monad m, Ord a) => Fold m a (Map a Int)
347− [A] foldr' :: Monad m => (a -> b -> b) -> b -> Fold m a b
348− [D] foldr :: Monad m => (a -> b -> b) -> b -> Fold m a b
349− [A] foldlM1' :: Monad m => (a -> a -> m a) -> Fold m a (Maybe a)
350− [A] foldl1' :: Monad m => (a -> a -> a) -> Fold m a (Maybe a)
351− [A] findM :: Monad m => (a -> m Bool) -> Fold m a (Maybe a)
352− [A] findIndices :: Monad m => (a -> Bool) -> Fold m a (Maybe Int)
353− [A] elemIndices :: (Monad m, Eq a) => a -> Fold m a (Maybe Int)
354− [A] duplicate :: Monad m => Fold m a b -> Fold m a (Fold m a b)
355− [A] drive :: Monad m => Stream m a -> Fold m a b -> m b
356− [A] drainMapM :: Monad m => (a -> m b) -> Fold m a ()
357− [D] drainBy :: Monad m => (a -> m b) -> Fold m a ()
358− [A] demuxToMapIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
359− [A] demuxToMap :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (Map k b)
360− [A] demuxIO :: (MonadIO m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
361− [A] demux :: (Monad m, Ord k) => (a -> k) -> (a -> m (Fold m a b)) -> Fold m a (m (Map k b), Maybe (k, b))
362− [A] deleteBy :: Monad m => (a -> a -> Bool) -> a -> Fold m a (Maybe a)
363− [A] countDistinctInt :: Monad m => Fold m Int Int
364− [A] countDistinct :: (Monad m, Ord a) => Fold m a Int
365− [A] classifyIO :: (MonadIO m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
366− [A] classify :: (Monad m, Ord k) => (a -> k) -> Fold m a b -> Fold m a (m (Map k b), Maybe (k, b))
367− [D] chunksOf :: Monad m => Int -> Fold m a b -> Fold m b c -> Fold m a c
368− [A] catRights :: Monad m => Fold m b c -> Fold m (Either a b) c
369− [A] catLefts :: Monad m => Fold m a c -> Fold m (Either a b) c
370− [A] catEithers :: Fold m a b -> Fold m (Either a a) b
371− [A] addStream :: Monad m => Stream m a -> Fold m a b -> m (Fold m a b)
372− [A] addOne :: Monad m => a -> Fold m a b -> m (Fold m a b)
373−[A] Streamly.Data.Array.Generic
374− [A] Array
375− [A] writeN :: MonadIO m => Int -> Fold m a (Array a)
376− [A] write :: MonadIO m => Fold m a (Array a)
377− [A] reader :: Monad m => Unfold m (Array a) a
378− [A] readRev :: Monad m => Array a -> Stream m a
379− [A] read :: MonadIO m => Array a -> Stream m a
380− [A] length :: Array a -> Int
381− [A] fromListN :: Int -> [a] -> Array a
382− [A] fromList :: [a] -> Array a
383−[A] Streamly.Data.Array
384− [A] class Unbox a
385− [A] Array
386− [A] writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
387− [A] writeLastN :: (Storable a, Unbox a, MonadIO m) => Int -> Fold m a (Array a)
388− [A] write :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
389− [A] toList :: Unbox a => Array a -> [a]
390− [A] sizeOf :: (Unbox a, SizeOfRep (Rep a)) => Proxy a -> Int
391− [A] readerRev :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
392− [A] reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
393− [A] pokeByteIndex :: (Unbox a, Generic a, PokeRep (Rep a)) => Int -> MutableByteArray -> a -> IO ()
394− [A] peekByteIndex :: (Unbox a, Generic a, PeekRep (Rep a)) => Int -> MutableByteArray -> IO a
395− [A] length :: Unbox a => Array a -> Int
396− [A] getIndex :: forall a. Unbox a => Int -> Array a -> Maybe a
397− [A] fromListN :: Unbox a => Int -> [a] -> Array a
398− [A] fromList :: Unbox a => [a] -> Array a
399− [A] cast :: forall a b. Unbox b => Array a -> Maybe (Array b)
400− [A] asBytes :: Array a -> Array Word8
401−[C] Streamly.Console.Stdio
402− [A] reader :: MonadIO m => Unfold m () Word8
403− [D] readChunks :: MonadIO m => Unfold m () (Array Word8)
404− [D] read :: MonadIO m => Unfold m () Word8
405− [A] chunkReader :: MonadIO m => Unfold m () (Array Word8)
core/docs/ApiChangelogs/0.2.0-0.2.2.txtdeleted−306
@@ -1,306 +0,0 @@
1−---------------------------------
2−API Annotations
3−---------------------------------
4−
5−[A] : Added
6−[R] : Removed
7−[C] : Changed
8−[O] : Old definition
9−[N] : New definition
10−[D] : Deprecated
11−
12−---------------------------------
13−API diff
14−---------------------------------
15−
16−[C] Streamly.Data.Stream
17− [A] (FixityR,5)
18− [A] (FixityR,5)
19−[C] Streamly.Data.MutArray.Generic
20− [A] emptyOf :: MonadIO m => Int -> m (MutArray a)
21− [A] createOf :: MonadIO m => Int -> Fold m a (MutArray a)
22− [A] create :: MonadIO m => Fold m a (MutArray a)
23−[C] Streamly.Data.MutArray
24− [A] pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
25− [A] emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
26− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
27− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
28− [A] appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
29− [A] append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
30−[C] Streamly.Data.Array.Generic
31− [A] createOf :: MonadIO m => Int -> Fold m a (Array a)
32− [A] create :: MonadIO m => Fold m a (Array a)
33−[C] Streamly.Data.Array
34− [A] class Serialize a
35− [A] serializeAt :: Serialize a => Int -> MutByteArray -> a -> IO Int
36− [A] pinnedSerialize :: Serialize a => a -> Array Word8
37− [A] deserializeAt :: Serialize a => Int -> MutByteArray -> Int -> IO (Int, a)
38− [A] deserialize :: Serialize a => Array Word8 -> a
39− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
40− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
41− [A] addSizeTo :: Serialize a => Int -> a -> Int
42−
43−---------------------------------
44−Internal API diff
45−---------------------------------
46−
47−[C] Streamly.Internal.Unicode.Stream
48− [A] encodeUtf16le' :: Stream m Char -> Stream m Word16
49− [A] decodeUtf16le' :: Stream m Word16 -> Stream m Char
50−[A] Streamly.Internal.FileSystem.Path
51− [A] class IsPath a
52− [A] Rel
53− [A] File
54− [A] Dir
55− [A] Abs
56− [A] Streamly.Internal.FileSystem.Path.IsPath
57− [A] instance Streamly.Internal.FileSystem.Path.IsPath Streamly.Internal.FileSystem.Path.Path
58− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel Streamly.Internal.FileSystem.Path.Path)
59− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path))
60− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Rel (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path))
61− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path)
62− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path)
63− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs Streamly.Internal.FileSystem.Path.Path)
64− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs (Streamly.Internal.FileSystem.Path.File Streamly.Internal.FileSystem.Path.Path))
65− [A] instance Streamly.Internal.FileSystem.Path.IsPath (Streamly.Internal.FileSystem.Path.Abs (Streamly.Internal.FileSystem.Path.Dir Streamly.Internal.FileSystem.Path.Path))
66− [A] GHC.Show.Show
67− [A] instance GHC.Show.Show Streamly.Internal.FileSystem.Path.PathException
68− [A] GHC.Exception.Type.Exception
69− [A] instance GHC.Exception.Type.Exception Streamly.Internal.FileSystem.Path.PathException
70− [A] GHC.Classes.Eq
71− [A] instance GHC.Classes.Eq Streamly.Internal.FileSystem.Path.PathException
72− [A] Path
73− [A] Path :: Array Word8 -> Path
74− [A] toString :: Path -> [Char]
75− [A] toPath :: IsPath a => a -> Path
76− [A] toChunk :: Path -> Array Word8
77− [A] toChars :: Monad m => Path -> Stream m Char
78− [A] relfile :: QuasiQuoter
79− [A] reldir :: QuasiQuoter
80− [A] rel :: QuasiQuoter
81− [A] primarySeparator :: Char
82− [A] path :: QuasiQuoter
83− [A] mkRelFile :: String -> Q Exp
84− [A] mkRelDir :: String -> Q Exp
85− [A] mkRel :: String -> Q Exp
86− [A] mkPath :: String -> Q Exp
87− [A] mkFile :: String -> Q Exp
88− [A] mkDir :: String -> Q Exp
89− [A] mkAbsFile :: String -> Q Exp
90− [A] mkAbsDir :: String -> Q Exp
91− [A] mkAbs :: String -> Q Exp
92− [A] isSeparator :: Char -> Bool
93− [A] fromString :: MonadThrow m => [Char] -> m Path
94− [A] fromPathUnsafe :: IsPath a => Path -> a
95− [A] fromPath :: (IsPath a, MonadThrow m) => Path -> m a
96− [A] fromChunkUnsafe :: Array Word8 -> Path
97− [A] fromChunk :: MonadThrow m => Array Word8 -> m Path
98− [A] fromChars :: MonadThrow m => Stream Identity Char -> m Path
99− [A] file :: QuasiQuoter
100− [A] extendPath :: Path -> Path -> Path
101− [A] extendDir :: (IsPath (a (Dir Path)), IsPath b, IsPath (a b)) => a (Dir Path) -> Rel b -> a b
102− [A] dir :: QuasiQuoter
103− [A] adaptPath :: (MonadThrow m, IsPath a, IsPath b) => a -> m b
104− [A] absfile :: QuasiQuoter
105− [A] absdir :: QuasiQuoter
106− [A] abs :: QuasiQuoter
107−[C] Streamly.Internal.Data.Stream
108− [A] (FixityR,5)
109− [A] (FixityR,5)
110− [C] splitInnerBySuffix
111− [O] splitInnerBySuffix :: (Monad m, Eq (f a), Monoid (f a)) => (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
112− [N] splitInnerBySuffix :: Monad m => (f a -> Bool) -> (f a -> m (f a, Maybe (f a))) -> (f a -> f a -> m (f a)) -> Stream m (f a) -> Stream m (f a)
113− [D] sliceOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
114− [A] indexOnSuffix :: Monad m => (a -> Bool) -> Stream m a -> Stream m (Int, Int)
115−[C] Streamly.Internal.Data.MutByteArray
116− [A] unsafePinnedAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
117− [A] unsafeAsPtr :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
118− [D] nil :: MutByteArray
119− [A] empty :: MutByteArray
120− [D] asPtrUnsafe :: MonadIO m => MutByteArray -> (Ptr a -> m b) -> m b
121−[D] Streamly.Internal.Data.MutArray.Stream
122− [D] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
123− [D] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
124− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
125− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
126−[C] Streamly.Internal.Data.MutArray.Generic
127− [D] writeWith :: MonadIO m => Int -> Fold m a (MutArray a)
128− [D] writeNUnsafe :: MonadIO m => Int -> Fold m a (MutArray a)
129− [A] unsafeCreateOf :: MonadIO m => Int -> Fold m a (MutArray a)
130− [A] emptyOf :: MonadIO m => Int -> m (MutArray a)
131− [A] createWith :: MonadIO m => Int -> Fold m a (MutArray a)
132− [A] createOf :: MonadIO m => Int -> Fold m a (MutArray a)
133− [A] create :: MonadIO m => Fold m a (MutArray a)
134−[C] Streamly.Internal.Data.MutArray
135− [A] SpliceState
136− [A] SpliceYielding :: arr -> SpliceState s arr -> SpliceState s arr
137− [A] SpliceInitial :: s -> SpliceState s arr
138− [A] SpliceFinish :: SpliceState s arr
139− [A] SpliceBuffering :: s -> arr -> SpliceState s arr
140− [R] MutByteArray
141− [D] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
142− [D] writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
143− [D] writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
144− [D] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
145− [D] writeChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
146− [D] writeAppendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
147− [D] writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
148− [A] unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
149− [A] unsafePinnedAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
150− [A] unsafeCreateOfWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
151− [A] unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
152− [A] unsafeAsPtr :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
153− [A] unsafeAppendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
154− [A] toStreamWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
155− [A] toStreamRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
156− [R] toStreamDWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
157− [R] toStreamDRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> MutArray a -> Stream m a
158− [A] slicerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
159− [A] sliceIndexerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
160− [A] revCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
161− [D] resizeExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
162− [D] resize :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
163− [A] pokeSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
164− [A] pokeAppendMay :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> a -> m (Maybe (MutArray Word8))
165− [A] pokeAppend :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> a -> m (MutArray Word8)
166− [D] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
167− [D] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
168− [D] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
169− [D] pinnedNewBytes :: MonadIO m => Int -> m (MutArray a)
170− [A] pinnedEmptyOf :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
171− [A] pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
172− [A] pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
173− [A] pinnedCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
174− [A] peekUnconsUnsafe :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> m (a, MutArray Word8)
175− [A] peekUncons :: forall m a. (MonadIO m, Unbox a) => MutArray Word8 -> m (Maybe a, MutArray Word8)
176− [A] peekSkipUnsafe :: Int -> MutArray Word8 -> MutArray Word8
177− [A] pPinnedCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Parser (MutArray a) m (MutArray a)
178− [A] pCompactLE :: forall m a. (MonadIO m, Unbox a) => Int -> Parser (MutArray a) m (MutArray a)
179− [D] nil :: MutArray a
180− [A] lPinnedCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
181− [A] lCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) () -> Fold m (MutArray a) ()
182− [A] indexReaderWith :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
183− [A] indexReader :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
184− [A] growExp :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
185− [A] grow :: forall m a. (MonadIO m, Unbox a) => Int -> MutArray a -> m (MutArray a)
186− [D] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (MutArray a)
187− [R] getIndicesD :: (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m Int -> Unfold m (MutArray a) a
188− [D] getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (MutArray a) a
189− [D] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (MutArray a) (Int, Int)
190− [A] fromStreamN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
191− [D] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (MutArray a)
192− [D] fromStreamD :: (MonadIO m, Unbox a) => Stream m a -> m (MutArray a)
193− [A] fromPureStreamN :: (MonadIO m, Unbox a) => Int -> Stream Identity a -> m (MutArray a)
194− [A] fromPtrN :: MonadIO m => Int -> Ptr Word8 -> m (MutArray Word8)
195− [A] fromChunksRealloced :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> m (MutArray a)
196− [A] fromChunksK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
197− [A] fromByteStr# :: MonadIO m => Addr# -> m (MutArray Word8)
198− [D] fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (MutArray a) -> m (MutArray a)
199− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
200− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
201− [A] fPinnedCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) (MutArray a)
202− [A] fCompactGE :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m (MutArray a) (MutArray a)
203− [A] emptyOf :: (MonadIO m, Unbox a) => Int -> m (MutArray a)
204− [A] empty :: MutArray a
205− [A] createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
206− [A] createOfWith :: forall m a. (MonadIO m, Unbox a) => (Int -> m (MutArray a)) -> Int -> Fold m a (MutArray a)
207− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (MutArray a)
208− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (MutArray a)
209− [A] concatWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m (MutArray a) -> Stream m a
210− [A] concatRevWith :: forall m a. (Monad m, Unbox a) => (forall b. IO b -> m b) -> Stream m (MutArray a) -> Stream m a
211− [A] concatRev :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
212− [A] concat :: forall m a. (MonadIO m, Unbox a) => Stream m (MutArray a) -> Stream m a
213− [A] compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (MutArray Word8) -> Stream m (MutArray Word8)
214− [A] compactOnByte :: MonadIO m => Word8 -> Stream m (MutArray Word8) -> Stream m (MutArray Word8)
215− [A] compactLeAs :: forall m a. (MonadIO m, Unbox a) => PinnedState -> Int -> Stream m (MutArray a) -> Stream m (MutArray a)
216− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
217− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (MutArray a) -> Stream m (MutArray a)
218− [A] compactEQ :: Int -> Stream m (MutArray a) -> Stream m (MutArray a)
219− [D] cmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
220− [A] byteEq :: MonadIO m => MutArray a -> MutArray a -> m Bool
221− [A] byteCmp :: MonadIO m => MutArray a -> MutArray a -> m Ordering
222− [A] buildChunks :: (MonadIO m, Unbox a) => Int -> Fold m a (StreamK n (MutArray a))
223− [D] asPtrUnsafe :: MonadIO m => MutArray a -> (Ptr a -> m b) -> m b
224− [A] appendWith :: forall m a. (MonadIO m, Unbox a) => (Int -> Int) -> m (MutArray a) -> Fold m a (MutArray a)
225− [A] appendN :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a) -> Fold m a (MutArray a)
226− [A] append :: forall m a. (MonadIO m, Unbox a) => m (MutArray a) -> Fold m a (MutArray a)
227−[D] Streamly.Internal.Data.Array.Stream
228− [C] interposeSuffix
229− [O] interposeSuffix :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
230− [N] interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
231− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
232− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
233− [C] concatRev
234− [O] concatRev :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
235− [N] concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
236− [D] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
237−[C] Streamly.Internal.Data.Array.Generic
238− [A] createOf :: MonadIO m => Int -> Fold m a (Array a)
239− [A] create :: MonadIO m => Fold m a (Array a)
240−[C] Streamly.Internal.Data.Array
241− [R] ArrayUnsafe
242− [D] writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
243− [D] writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
244− [A] unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
245− [A] unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
246− [A] unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
247− [D] toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
248− [D] toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a
249− [C] splice
250− [O] splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
251− [N] splice :: MonadIO m => Array a -> Array a -> m (Array a)
252− [A] slicerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Array a)
253− [A] sliceIndexerFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Int, Int)
254− [D] pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
255− [D] pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)
256− [D] pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
257− [D] pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
258− [A] pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
259− [A] pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
260− [A] pinnedCompactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
261− [A] parseBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))
262− [D] nil :: Array a
263− [A] lPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
264− [A] lCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()
265− [A] interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
266− [A] interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
267− [A] intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a
268− [A] indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a
269− [A] indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
270− [A] indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int
271− [D] getSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Array a)
272− [R] getIndicesFromThenTo :: Unfold m (Int, Int, Int, Array a) a
273− [D] getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
274− [D] genSlicesFromLen :: forall m a. (Monad m, Unbox a) => Int -> Int -> Unfold m (Array a) (Int, Int)
275− [D] fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
276− [D] fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)
277− [A] fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a
278− [A] fromPtrN :: Int -> Ptr Word8 -> Array Word8
279− [A] fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)
280− [A] fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
281− [A] fromByteStr :: Ptr Word8 -> Array Word8
282− [A] foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b
283− [A] foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
284− [A] foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
285− [D] flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
286− [D] flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a
287− [C] findIndicesOf
288− [O] findIndicesOf :: (a -> Bool) -> Unfold Identity (Array a) Int
289− [N] findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int
290− [A] fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
291− [A] fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
292− [A] empty :: Array a
293− [A] createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
294− [A] createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
295− [A] create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
296− [A] concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
297− [A] concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
298− [A] compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
299− [A] compactOnByte :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)
300− [A] compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
301− [A] compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)
302− [A] byteEq :: Array a -> Array a -> Bool
303− [A] byteCmp :: Array a -> Array a -> Ordering
304− [A] buildChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
305− [D] bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))
306− [D] asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
core/docs/Changelog.mddeleted−97
@@ -1,97 +0,0 @@
1−# Changelog
2−
3−## 0.2.2 (Jan 2024)
4−
5−* Add fixities `infixr 5` for `cons` and `consM` functions.
6−* Fix a bug in Array `Eq` instance when the type is a sum type with
7− differently sized constructors.
8−* lpackArraysChunksOf, compact, writeChunksWith, putChunksWith now take the
9− buffer size in number of array elements instead of bytes.
10−
11−## 0.2.1 (Dec 2023)
12−
13−* Make the serialization of the unit constructor deterministic.
14−* Expose `pinnedSerialize` & `deserialize` via `Streamly.Data.Array`.
15−
16−## 0.2.0 (Nov 2023)
17−
18−See [0.1.0-0.2.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0-0.2.0.txt)
19−for a full list of API changes in this release. Only a few significant
20−changes are mentioned here.
21−
22−### Breaking Changes
23−
24−* `ParserK` in `Streamly.Data.ParserK` is not implicitly specialized
25− to arrays anymore. To adapt to the new code, change `ParserK a m
26− b` to `ParserK (Array a) m b` where the `Array` type comes from
27− `Streamly.Data.Array`. This change also affected the signatures of
28− `parseChunks` and `parseBreakChunks`.
29−* Changed the signature of 'Streamly.Data.Stream.handle' to make the
30− exception handler monadic.
31−* Behavior change: Exceptions are now rethrown promptly in `bracketIO`.
32−
33−### Enhancements
34−
35−* __Serialization__: Added a `Streamly.Data.MutByteArray` module with a
36− `Serialize` type class for fast binary serialization. The Data.Array
37− module supplies the `serialize` and `deserialize` operations for arrays.
38−* __Unpinned Arrays__: Unboxed arrays are now created unpinned by default,
39− they were created pinned earlier. During IO operations, unpinned arrays
40− are automatically copied to pinned memory. When arrays are directly
41− passed to IO operations programmers can choose to create them pinned to
42− avoid a copy. To create pinned arrays, use the internal APIs with the
43− `pinned*` prefix.
44−* StreamK now supports native exception handling routines (handle, bracketIO).
45− Earlier we had to convert it to the `Stream` type for exception handling.
46−
47−### Deprecations
48−
49−See [0.1.0-0.2.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0-0.2.0.txt)
50−for a full list of deprecations.
51−
52−### Internal API Changes
53−
54−* Fold constructor has changed, added a `final` field to support
55− finalization and cleanup of a chain of folds. The `extract` field is
56− now used only for mapping the fold internal state to fold result for
57− scanning purposes. If your fold does not require cleanup you can just use
58− your existing `extract` function as `final` as well to adapt to this change.
59−* Many low level internal modules have been removed, they are entirely
60− exported from higher level internal modules. If you were importing any
61− of the missing low level modules then import the higher level modules instead.
62−* Internal module changes:
63− * Streamly.Internal.Serialize.FromBytes -> Streamly.Internal.Data.Binary.Parser
64− * Streamly.Internal.Serialize.ToBytes -> Streamly.Internal.Data.Binary.Stream
65− * Streamly.Internal.Data.Unbox is now exported via Streamly.Internal.Data.Serialize
66− * Streamly.Internal.Data.IORef.Unboxed is now exported via Streamly.Internal.Data.Serialize
67−
68−## 0.1.0 (March 2023)
69−
70−Also see [streamly-core-0.1.0 API Changelog](https://github.com/composewell/streamly/blob/streamly-0.10.0/core/docs/ApiChangelogs/0.1.0.txt) or
71−https://hackage.haskell.org/package/streamly-core-0.1.0/docs/docs/ApiChangelogs/0.1.0.txt
72−
73−`streamly` package is split into two packages, (1) `streamly-core` that
74−has only GHC boot library depdendecies, and (2) `streamly` that contains
75−higher level operations (including concurrent ones) with additional
76−dependencies.
77−
78−* Moved the following modules from `streamly` package to the
79− `streamly-core` package:
80− * Streamly.Console.Stdio
81− * Streamly.Data.Fold
82− * Streamly.Data.Unfold
83− * Streamly.FileSystem.Handle
84− * Streamly.Unicode.Stream
85−* Added the following new modules:
86− * Streamly.Data.Array
87− * Streamly.Data.Array.Generic
88− * Streamly.Data.MutArray
89− * Streamly.Data.MutArray.Generic
90− * Streamly.Data.Parser
91− * Streamly.Data.ParserK
92− * Streamly.Data.Stream
93− * Streamly.Data.StreamK
94− * Streamly.FileSystem.Dir
95− * Streamly.FileSystem.File
96− * Streamly.Unicode.Parser
97− * Streamly.Unicode.String
core/docs/Readme.mddeleted−1
@@ -1 +0,0 @@
1−Please refer to the "streamly" package for tutorials and other documentation.
core/src/DocTestDataArray.hsdeleted−22
@@ -1,22 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> :set -XFlexibleContexts
4−>>> :set -XMagicHash
5−>>> import Data.Function ((&))
6−>>> import Data.Functor.Identity (Identity(..))
7−>>> import System.IO.Unsafe (unsafePerformIO)
8−
9−>>> import Streamly.Data.Array (Array)
10−>>> import Streamly.Data.Stream (Stream)
11−
12−>>> import qualified Streamly.Data.Array as Array
13−>>> import qualified Streamly.Data.Fold as Fold
14−>>> import qualified Streamly.Data.ParserK as ParserK
15−>>> import qualified Streamly.Data.Stream as Stream
16−>>> import qualified Streamly.Data.StreamK as StreamK
17−
18−For APIs that have not been released yet.
19−
20−>>> import qualified Streamly.Internal.Data.Array as Array
21−>>> import qualified Streamly.Internal.Data.Stream as Stream
22−-}
core/src/DocTestDataFold.hsdeleted−27
@@ -1,27 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> :set -XFlexibleContexts
4−>>> import Control.Monad (void)
5−>>> import qualified Data.Foldable as Foldable
6−>>> import Data.Function ((&))
7−>>> import Data.Functor.Identity (Identity, runIdentity)
8−>>> import Data.IORef (newIORef, readIORef, writeIORef)
9−>>> import Data.Maybe (fromJust, isJust)
10−>>> import Data.Monoid (Endo(..), Last(..), Sum(..))
11−
12−>>> import Streamly.Data.Array (Array)
13−>>> import Streamly.Data.Fold (Fold, Tee(..))
14−>>> import Streamly.Data.Stream (Stream)
15−
16−>>> import qualified Streamly.Data.Array as Array
17−>>> import qualified Streamly.Data.Fold as Fold
18−>>> import qualified Streamly.Data.MutArray as MutArray
19−>>> import qualified Streamly.Data.Parser as Parser
20−>>> import qualified Streamly.Data.Stream as Stream
21−>>> import qualified Streamly.Data.StreamK as StreamK
22−>>> import qualified Streamly.Data.Unfold as Unfold
23−
24−For APIs that have not been released yet.
25−
26−>>> import qualified Streamly.Internal.Data.Fold as Fold
27−-}
core/src/DocTestDataMutArray.hsdeleted−10
@@ -1,10 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import qualified Streamly.Data.Fold as Fold
4−>>> import qualified Streamly.Data.MutArray as MutArray
5−>>> import qualified Streamly.Data.Stream as Stream
6−
7−For APIs that have not been released yet.
8−
9−>>> import Streamly.Internal.Data.MutArray as MutArray
10−-}
core/src/DocTestDataMutArrayGeneric.hsdeleted−10
@@ -1,10 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import qualified Streamly.Data.Fold as Fold
4−>>> import qualified Streamly.Data.MutArray.Generic as MutArray
5−>>> import qualified Streamly.Data.Stream as Stream
6−
7−For APIs that have not been released yet.
8−
9−>>> import Streamly.Internal.Data.MutArray.Generic as MutArray
10−-}
core/src/DocTestDataParser.hsdeleted−20
@@ -1,20 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import Control.Applicative ((<|>))
4−>>> import Data.Bifunctor (second)
5−>>> import Data.Char (isSpace)
6−>>> import qualified Data.Foldable as Foldable
7−>>> import qualified Data.Maybe as Maybe
8−
9−>>> import Streamly.Data.Fold (Fold)
10−>>> import Streamly.Data.Parser (Parser)
11−
12−>>> import qualified Streamly.Data.Fold as Fold
13−>>> import qualified Streamly.Data.Parser as Parser
14−>>> import qualified Streamly.Data.Stream as Stream
15−
16−For APIs that have not been released yet.
17−
18−>>> import qualified Streamly.Internal.Data.Fold as Fold
19−>>> import qualified Streamly.Internal.Data.Parser as Parser
20−-}
core/src/DocTestDataParserK.hsdeleted−15
@@ -1,15 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import Control.Applicative ((<|>))
4−>>> import Data.Char (isDigit, isAlpha)
5−
6−>>> import Streamly.Data.Parser (Parser)
7−>>> import Streamly.Data.ParserK (ParserK)
8−
9−>>> import qualified Streamly.Data.Parser as Parser
10−>>> import qualified Streamly.Data.ParserK as ParserK
11−
12−For APIs that have not been released yet.
13−
14−>>> import qualified Streamly.Internal.Data.ParserK as ParserK
15−-}
core/src/DocTestDataStream.hsdeleted−39
@@ -1,39 +0,0 @@
1−{- $setup
2−
3−>>> :m
4−>>> import Control.Concurrent (threadDelay)
5−>>> import Control.Monad (void)
6−>>> import Control.Monad.IO.Class (MonadIO (liftIO))
7−>>> import Control.Monad.Trans.Class (lift)
8−>>> import Control.Monad.Trans.Identity (runIdentityT)
9−>>> import Data.Char (isSpace)
10−>>> import Data.Either (fromLeft, fromRight, isLeft, isRight, either)
11−>>> import Data.Maybe (fromJust, isJust)
12−>>> import Data.Function (fix, (&))
13−>>> import Data.Functor.Identity (runIdentity)
14−>>> import Data.IORef
15−>>> import Data.Semigroup (cycle1)
16−>>> import GHC.Exts (Ptr (Ptr))
17−>>> import System.IO (stdout, hSetBuffering, BufferMode(LineBuffering))
18−
19−>>> hSetBuffering stdout LineBuffering
20−>>> effect n = print n >> return n
21−
22−>>> import Streamly.Data.Stream (Stream)
23−>>> import qualified Streamly.Data.Array as Array
24−>>> import qualified Streamly.Data.Fold as Fold
25−>>> import qualified Streamly.Data.Stream as Stream
26−>>> import qualified Streamly.Data.StreamK as StreamK
27−>>> import qualified Streamly.Data.Unfold as Unfold
28−>>> import qualified Streamly.Data.Parser as Parser
29−>>> import qualified Streamly.FileSystem.Dir as Dir
30−
31−For APIs that have not been released yet.
32−
33−>>> import qualified Streamly.Internal.Data.Fold as Fold
34−>>> import qualified Streamly.Internal.Data.Parser as Parser
35−>>> import qualified Streamly.Internal.Data.Stream as Stream
36−>>> import qualified Streamly.Internal.Data.StreamK as StreamK
37−>>> import qualified Streamly.Internal.Data.Unfold as Unfold
38−>>> import qualified Streamly.Internal.FileSystem.Dir as Dir
39−-}
core/src/DocTestDataStreamK.hsdeleted−21
@@ -1,21 +0,0 @@
1−{- $setup
2−
3−>>> :m
4−>>> import Control.Concurrent (threadDelay)
5−>>> import Data.Function (fix, (&))
6−>>> import Data.Semigroup (cycle1)
7−
8−>>> effect n = print n >> return n
9−
10−>>> import Streamly.Data.StreamK (StreamK)
11−>>> import qualified Streamly.Data.Fold as Fold
12−>>> import qualified Streamly.Data.Parser as Parser
13−>>> import qualified Streamly.Data.Stream as Stream
14−>>> import qualified Streamly.Data.StreamK as StreamK
15−>>> import qualified Streamly.FileSystem.Dir as Dir
16−
17−For APIs that have not been released yet.
18−
19−>>> import qualified Streamly.Internal.Data.StreamK as StreamK
20−>>> import qualified Streamly.Internal.FileSystem.Dir as Dir
21−-}
core/src/DocTestDataUnfold.hsdeleted−12
@@ -1,12 +0,0 @@
1−{- $setup
2−
3−>>> :m
4−>>> import Streamly.Data.Unfold (Unfold)
5−>>> import qualified Streamly.Data.Fold as Fold
6−>>> import qualified Streamly.Data.Stream as Stream
7−>>> import qualified Streamly.Data.Unfold as Unfold
8−
9−For APIs that have not been released yet.
10−
11−>>> import qualified Streamly.Internal.Data.Unfold as Unfold
12−-}
core/src/DocTestFileSystemHandle.hsdeleted−15
@@ -1,15 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import qualified Streamly.Data.Array as Array
4−>>> import qualified Streamly.FileSystem.Handle as Handle hiding (readChunks)
5−>>> import qualified Streamly.Data.Fold as Fold
6−>>> import qualified Streamly.Data.Stream as Stream
7−>>> import qualified Streamly.Data.Unfold as Unfold
8−
9−For APIs that have not been released yet.
10−
11−>>> import qualified Streamly.Internal.Data.Array as Array (unsafeCreateOf)
12−>>> import qualified Streamly.Internal.Data.Unfold as Unfold (first)
13−>>> import qualified Streamly.Internal.FileSystem.Handle as Handle
14−>>> import qualified Streamly.Internal.System.IO as IO (defaultChunkSize)
15−-}
core/src/DocTestUnicodeParser.hsdeleted−9
@@ -1,9 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> import qualified Streamly.Data.Stream as Stream
4−>>> import qualified Streamly.Unicode.Parser as Unicode
5−
6−For APIs that have not been released yet.
7−
8−>>> import qualified Streamly.Internal.Unicode.Parser as Unicode (number, mkDouble)
9−-}
core/src/DocTestUnicodeStream.hsdeleted−12
@@ -1,12 +0,0 @@
1−{- $setup
2−>>> :m
3−
4−>>> import qualified Streamly.Data.Fold as Fold
5−>>> import qualified Streamly.Data.Stream as Stream
6−>>> import qualified Streamly.Unicode.Stream as Unicode
7−
8−For APIs that have not been released yet.
9−
10−>>> :set -XMagicHash
11−>>> import qualified Streamly.Internal.Unicode.Stream as Unicode
12−-}
core/src/DocTestUnicodeString.hsdeleted−5
@@ -1,5 +0,0 @@
1−{- $setup
2−>>> :m
3−>>> :set -XQuasiQuotes
4−>>> import Streamly.Internal.Unicode.String
5−-}
core/src/Streamly/Console/Stdio.hs+27−1
@@ -24,7 +24,33 @@ module Streamly.Console.Stdio
2424 -- * Write (stderr)
2525 , writeErr
2626 , writeErrChunks
27+
28+ -- * Deprecated
29+ , read
30+ , readChunks
2731 )
2832where
2933
30−import Streamly.Internal.Console.Stdio
34+import Control.Monad.IO.Class (MonadIO(..))
35+import Data.Word (Word8)
36+import Streamly.Internal.Data.Array.Type (Array)
37+import Streamly.Internal.Data.Unfold (Unfold)
38+
39+import Streamly.Internal.Console.Stdio hiding (read, readChunks)
40+import Prelude hiding (read)
41+
42+-- Same as 'reader'
43+--
44+-- @since 0.8.0
45+{-# DEPRECATED read "Please use 'reader' instead" #-}
46+{-# INLINE read #-}
47+read :: MonadIO m => Unfold m () Word8
48+read = reader
49+
50+-- Same as 'chunkReader'
51+--
52+-- @since 0.8.0
53+{-# DEPRECATED readChunks "Please use 'chunkReader' instead" #-}
54+{-# INLINE readChunks #-}
55+readChunks :: MonadIO m => Unfold m () (Array Word8)
56+readChunks = chunkReader
core/src/Streamly/Data/Array.hs+111−115
@@ -1,4 +1,3 @@
1−{-# LANGUAGE CPP #-}
21-- |
32-- Module : Streamly.Data.Array
43-- Copyright : (c) 2019 Composewell Technologies
@@ -8,160 +7,157 @@
87-- Stability : released
98-- Portability : GHC
109--
11−-- Unboxed immutable arrays with streaming interfaces.
10+-- This module provides APIs to create and use unboxed immutable arrays. Once
11+-- created, their contents cannot be modified. Only types that are unboxable
12+-- via the 'Unbox' type class can be stored in these arrays.
13+--
14+-- = Folding Arrays
15+--
16+-- Convert array to stream, and fold the stream:
17+--
18+-- >>> fold f arr = Stream.unfold Array.reader arr & Stream.fold f
19+-- >>> fold Fold.sum (Array.fromList [1,2,3::Int])
20+-- 6
21+--
22+-- = Transforming Arrays
23+--
24+-- Convert array to stream, transform, and fold back to array:
25+--
26+-- >>> amap f arr = Stream.unfold Array.reader arr & fmap f & Stream.fold Array.write
27+-- >>> amap (+1) (Array.fromList [1,2,3::Int])
28+-- fromList [2,3,4]
29+--
30+-- = Pinned and Unpinned Arrays
31+--
32+-- The array type can use both pinned and unpinned memory under the hood.
33+-- Currently the array creation APIs create arrays in pinned memory but it will
34+-- change to unpinned in future releases. The change should not affect users
35+-- functionally unless they are directly accessing the internal memory of the
36+-- array via internal APIs. As of now unpinned arrays can be created using
37+-- unreleased APIs.
38+--
39+-- Unpinned arrays have the advantage of allowing automatic defragmentation of
40+-- the memory by GC. Whereas pinned arrays have the advantage of not requiring
41+-- a copy by GC. Normally you would want to use unpinned arrays. However, in
42+-- some cases, for example, for long lived large data storage, and for
43+-- interfacing with the operating system or foreign (non-Haskell) consumers you
44+-- may want to use pinned arrays.
45+--
46+-- = Creating Arrays from Non-IO Streams
47+--
48+-- Array creation folds require 'MonadIO' because they need to sequence effects
49+-- in IO streams. To operate on streams in pure Monads like 'Identity' you can
50+-- morph it to IO monad as follows:
51+--
52+-- The 'MonadIO' based folds can be morphed to 'Identity' stream folds:
53+--
54+-- >>> purely = Fold.morphInner (Identity . unsafePerformIO)
55+-- >>> Stream.fold (purely Array.write) $ Stream.fromList [1,2,3::Int]
56+-- Identity fromList [1,2,3]
57+--
58+-- Alternatively, 'Identity' streams can be generalized to IO streams:
59+--
60+-- >>> pure = Stream.fromList [1,2,3] :: Stream Identity Int
61+-- >>> generally = Stream.morphInner (return . runIdentity)
62+-- >>> Stream.fold Array.write (generally pure :: Stream IO Int)
63+-- fromList [1,2,3]
1264--
13−-- Please refer to "Streamly.Internal.Data.Array" for functions that have not
14−-- yet been released.
65+-- Since it is a pure stream we can use 'unsafePerformIO' to extract the result
66+-- of fold from IO.
1567--
16−-- For arrays that work on boxed types, not requiring the 'Unbox' constraint,
17−-- please refer to "Streamly.Data.Array.Generic". For arrays that can be
18−-- mutated in-place, please see "Streamly.Data.MutArray".
68+-- = Programming Tips
69+--
70+-- This module is designed to be imported qualified:
71+--
72+-- >>> import qualified Streamly.Data.Array as Array
73+--
74+-- Use @QuantifiedConstraints@ extension to avoid the 'Storable' constraint
75+-- when using the 'Unbox' constraint.
76+--
77+-- For experimental APIs see "Streamly.Internal.Data.Array".
1978
2079module Streamly.Data.Array
2180 (
22− -- * Setup
23− -- | To execute the code examples provided in this module in ghci, please
24− -- run the following commands first.
25− --
81+ -- * Imports for Examples
2682 -- $setup
2783
28− -- * Overview
29− -- $overview
30−
3184 -- * The Array Type
32− Array
33−
34− -- * Pinning & Unpinning
35− -- | Arrays are created unpinned by default unless pinned versions of
36− -- creation APIs are used. Look for APIs with @pinned@ prefix in
37− -- "Streamly.Internal.Data.Array" for some unreleased pinned creation APIs.
38− -- If an array is to be sent to the OS without any further modification
39− -- then it should be created pinned in the first place instead of pinning
40− -- it later. Pinning an unpinned array has a copy overhead. OS interfacing
41− -- APIs create a pinned array directly or convert an unpinned array to
42− -- pinned array before sending it to the OS.
43− , pin
44− , unpin
45− , isPinned
85+ A.Array
4686
4787 -- * Construction
4888 -- | When performance matters, the fastest way to generate an array is
49− -- 'createOf'. 'IsList' and 'IsString' instances can be
89+ -- 'writeN'. 'IsList' and 'IsString' instances can be
5090 -- used to conveniently construct arrays from literal values.
5191 -- 'OverloadedLists' extension or 'fromList' can be used to construct an
5292 -- array from a list literal. Similarly, 'OverloadedStrings' extension or
5393 -- 'fromList' can be used to construct an array from a string literal.
5494
55− -- ** From Stream
56− , createOf
57− , create
58− , writeLastN -- drop old (ring buffer)
95+ -- Pure List APIs
96+ , A.fromListN
97+ , A.fromList
5998
60− -- ** From List
61− , fromListN
62− , fromList
99+ -- Monadic APIs
100+ , A.writeN -- drop new
101+ , A.write -- full buffer
102+ , writeLastN -- drop old (ring buffer)
63103
64− -- * To List
104+ -- * Conversion
65105 -- 'GHC.Exts.toList' from "GHC.Exts" can be used to convert an array to a
66106 -- list.
67− , toList
68−
69− -- * To Stream
70− , read
71− , readRev
107+ , A.toList
72108
73109 -- * Unfolds
74− , reader
75− , readerRev
110+ , A.reader
111+ , A.readerRev
76112
77113 -- * Casting
78114 , cast
79115 , asBytes
80116
81117 -- * Random Access
82− , length
118+ , A.length
83119 -- , (!!)
84− , getIndex
85−
86− -- * Serialization
87− , pinnedSerialize
88− , deserialize
120+ , A.getIndex
89121
90− -- * Re-exports
122+ -- * Unbox Type Class
91123 , Unbox (..)
92− , Serialize(..)
93124
94125 -- * Deprecated
95− , writeN -- drop new
96− , write -- full buffer
126+ , read
127+ , readRev
97128 )
98129where
99130
100131#include "inline.hs"
101132
102−import Streamly.Internal.Data.Array
103−import Streamly.Internal.Data.MutByteArray (Unbox(..), Serialize(..))
133+import Streamly.Internal.Data.Unfold (Unfold)
134+import Streamly.Internal.Data.Array as A hiding (read, readRev)
104135
105−import Prelude hiding (read, length)
136+import Streamly.Internal.Data.Unboxed (Unbox (..))
137+import Prelude hiding (read)
106138
107−#include "DocTestDataArray.hs"
139+-- $setup
140+-- >>> :m
141+-- >>> :set -XFlexibleContexts
142+-- >>> import Data.Function ((&))
143+-- >>> import Data.Functor.Identity (Identity(..))
144+-- >>> import Streamly.Data.Array (Array)
145+-- >>> import Streamly.Data.Stream (Stream)
146+-- >>> import System.IO.Unsafe (unsafePerformIO)
147+-- >>> import qualified Streamly.Data.Array as Array
148+-- >>> import qualified Streamly.Data.Fold as Fold
149+-- >>> import qualified Streamly.Data.Stream as Stream
108150
109−-- $overview
110−--
111−-- This module provides APIs to create and use unboxed immutable arrays. Once
112−-- created, their contents cannot be modified. Only types that are unboxable
113−-- via the 'Unbox' type class can be stored in these arrays. Note that the
114−-- array memory grows automatically when creating a new array, therefore, an
115−-- array can be created from a variable length stream.
116−--
117−-- == Folding Arrays
118−--
119−-- Convert array to stream, and fold the stream:
120−--
121−-- >>> fold f arr = Array.read arr & Stream.fold f
122−-- >>> fold Fold.sum (Array.fromList [1,2,3::Int])
123−-- 6
124−--
125−-- == Transforming Arrays
126−--
127−-- Convert array to stream, transform, and fold back to array:
128−--
129−-- >>> amap f arr = Array.read arr & fmap f & Stream.fold (Array.createOf (Array.length arr))
130−-- >>> amap (+1) (Array.fromList [1,2,3::Int])
131−-- fromList [2,3,4]
132−--
133−-- == Pinned and Unpinned Arrays
134−--
135−-- The array type can use both pinned and unpinned memory under the hood. The
136−-- default array creation operations create unpinned arrays. IO operations
137−-- automatically copy an array to pinned memory if the array passed to it is
138−-- unpinned. Programmers can use appropriate pinned array generation APIs to
139−-- reduce the copying if it happens.
140−--
141−-- Unpinned arrays have the advantage of allowing automatic defragmentation of
142−-- the memory by GC. Whereas pinned arrays have the advantage of not requiring
143−-- a copy by GC. Normally you would want to use unpinned arrays. However, in
144−-- some cases, for example, for long lived large data storage, and for
145−-- interfacing with the operating system or foreign (non-Haskell) consumers you
146−-- may want to use pinned arrays.
151+-- | Same as 'reader'
147152--
148−-- == Creating Arrays from Non-IO Streams
149−--
150−-- Array creation folds require 'MonadIO' otherwise the compiler may
151−-- incorrectly share the array memory thinking it is pure.
152−--
153−-- See the 'fromPureStream' unreleased API to generate an array from an
154−-- Identity stream safely without using MonadIO constraint.
155−--
156−-- Note that 'Identity' streams can be generalized to IO streams:
157−--
158−-- >>> stream = Stream.fromList [1,2,3] :: Stream Identity Int
159−-- >>> pure2IO s = Stream.morphInner (return . runIdentity) s :: Stream IO Int
160−-- >>> Stream.fold Array.create (pure2IO stream)
161−-- fromList [1,2,3]
162−--
163−-- == Programming Tips
164−--
165−-- This module is designed to be imported qualified:
153+{-# DEPRECATED read "Please use 'reader' instead" #-}
154+{-# INLINE_NORMAL read #-}
155+read :: (Monad m, Unbox a) => Unfold m (Array a) a
156+read = reader
157+
158+-- | Same as 'readerRev'
166159--
167−-- >>> import qualified Streamly.Data.Array as Array
160+{-# DEPRECATED readRev "Please use 'readerRev' instead" #-}
161+{-# INLINE_NORMAL readRev #-}
162+readRev :: (Monad m, Unbox a) => Unfold m (Array a) a
163+readRev = readerRev
core/src/Streamly/Data/Array/Generic.hs+11−19
@@ -15,37 +15,29 @@ module Streamly.Data.Array.Generic
1515 ( Array
1616
1717 -- * Construction
18− , fromListN
19− , fromList
18+ , A.fromListN
19+ , A.fromList
2020
2121 -- MonadicAPIs
22− , createOf
23− , create
24−
25− -- * Conversion
26− , toList
22+ , A.writeN
23+ , A.write
2724
2825 -- * Streams
29− , read
30− , readRev
26+ , A.read
27+ , A.readRev
3128
3229 -- * Unfolds
33− , reader
34− -- , A.readerRev
30+ , A.reader
3531
3632 -- * Random Access
37− , length
38− , getIndex
33+ , A.length
3934
4035 -- -- * Folding Arrays
4136 -- , A.streamFold
4237 -- , A.fold
43−
44− -- * Deprecated
45− , writeN
46− , write
4738 )
4839where
4940
50−import Streamly.Internal.Data.Array.Generic
51−import Prelude hiding (length, read)
41+import Streamly.Internal.Data.Array.Generic (Array)
42+
43+import qualified Streamly.Internal.Data.Array.Generic as A
core/src/Streamly/Data/Array/Generic/Mut.hsnew file+38
@@ -0,0 +1,38 @@
1+-- |
2+-- Module : Streamly.Data.Array.Generic.Mut
3+-- Copyright : (c) 2020 Composewell Technologies
4+-- License : BSD3-3-Clause
5+-- Maintainer : streamly@composewell.com
6+-- Stability : experimental
7+-- Portability : GHC
8+--
9+-- Mutable variant of the "Streamly.Data.Array.Generic" module.
10+--
11+module Streamly.Data.Array.Generic.Mut
12+(
13+ -- * Type
14+ Array
15+
16+ -- * Construction
17+ , writeN
18+
19+ -- * Appending elements
20+ , new
21+ , snoc
22+
23+ -- * Conversion
24+ , toList
25+
26+ -- * Unfolds
27+ , reader
28+
29+ -- * Random reads
30+ , getIndex
31+
32+ -- * Inplace mutation
33+ , putIndex
34+ , modifyIndex
35+ )
36+where
37+
38+import Streamly.Internal.Data.Array.Generic.Mut.Type
core/src/Streamly/Data/Array/Mut.hsnew file+66
@@ -0,0 +1,66 @@
1+-- |
2+-- Module : Streamly.Data.Array.Mut
3+-- Copyright : (c) 2022 Composewell Technologies
4+--
5+-- License : BSD3
6+-- Maintainer : streamly@composewell.com
7+-- Stability : released
8+-- Portability : GHC
9+--
10+-- This module provides mutable arrays suitable for long lived data storage,
11+-- random access, and mutation.
12+--
13+
14+module Streamly.Data.Array.Mut
15+ (
16+ -- * Types
17+ Array
18+
19+ -- * Construction
20+
21+ -- Uninitialized Arrays
22+ , new
23+ , newPinned
24+
25+ -- From containers
26+ , fromListN
27+ , fromList
28+ , writeN -- drop new
29+ , write -- full buffer
30+ -- writeLastN
31+
32+ -- * Appending elements
33+ , snoc
34+
35+ -- * Appending streams
36+ , writeAppendN
37+ , writeAppend
38+
39+ -- * Inplace mutation
40+ , putIndex
41+
42+ -- * Random access
43+ , getIndex
44+
45+ -- * Conversion
46+ , toList
47+
48+ -- * Unfolds
49+ , reader
50+ , readerRev
51+
52+ -- * Casting
53+ , cast
54+ , asBytes
55+
56+ -- * Size
57+ , length
58+
59+ -- * Unbox Type Class
60+ , Unbox (..)
61+ )
62+where
63+
64+import Prelude hiding (length, read)
65+import Streamly.Internal.Data.Array.Mut
66+import Streamly.Internal.Data.Unboxed (Unbox (..))
core/src/Streamly/Data/Fold.hs+54−105
@@ -1,4 +1,3 @@
1−{-# LANGUAGE CPP #-}
21-- |
32-- Module : Streamly.Data.Fold
43-- Copyright : (c) 2019 Composewell Technologies
@@ -7,45 +6,31 @@
76-- Stability : released
87-- Portability : GHC
98--
10−-- Fast, composable stream consumers with ability to terminate, supporting
11−-- stream fusion.
9+-- A 'Fold' is a sink or a consumer of a stream of values. The 'Fold' type
10+-- consists of an accumulator and an effectful action that absorbs a value into
11+-- the accumulator.
1212--
13−-- == Using Folds
13+-- >>> import Data.Function ((&))
14+-- >>> import qualified Streamly.Data.Fold as Fold
15+-- >>> import qualified Streamly.Data.Stream as Stream
1416--
15−-- This module provides elementary folds and fold combinators that can be used
16−-- to consume a stream of data and reduce it to a final value, or transform it
17−-- in a stateful manner using scans. A data stream can be reduced into a stream
18−-- of folded data elements by folding segments of the stream. Fold combinators
19−-- can be used to compose multiple folds in parallel or to create a pipeline of
20−-- folds such that the next fold consumes the result of the previous fold. To
21−-- run these folds on a stream see 'Streamly.Data.Stream.fold',
22−-- 'Streamly.Data.Stream.scan', 'Streamly.Data.Stream.postscan',
23−-- 'Streamly.Data.Stream.scanMaybe', 'Streamly.Data.Stream.foldMany' and other
24−-- operations accepting 'Fold' type as argument "Streamly.Data.Stream".
25−--
26−-- == Reducing a Stream
27−--
28−-- A 'Fold' is a consumer of a stream of values. A fold driver (such as
29−-- 'Streamly.Data.Stream.fold') initializes the fold @accumulator@, runs the
30−-- fold @step@ function in a loop, processing the input stream one element at a
31−-- time and accumulating the result. The loop continues until the fold
32−-- terminates, at which point the accumulated result is returned.
33−--
34−-- For example, a 'sum' Fold represents a stream consumer that adds the values
35−-- in the input stream:
17+-- For example, a 'sum' Fold represents adding the input to the accumulated
18+-- sum. A fold driver pushes values from a stream to the 'Fold' one at a time,
19+-- reducing the stream to a single value.
3620--
3721-- >>> Stream.fold Fold.sum $ Stream.fromList [1..100]
3822-- 5050
3923--
40−-- Conceptually, a 'Fold' is a data type that mimics a strict left fold
41−-- ('Data.List.foldl'). The above example is similar to a left fold using
42−-- @(+)@ as the step and @0@ as the initial value of the accumulator:
24+-- Conceptually, a 'Fold' is a data type that can mimic a strict left fold
25+-- ('Data.List.foldl') as well as lazy right fold ('Prelude.foldr'). The above
26+-- example is similar to a left fold using @(+)@ as the step and @0@ as the
27+-- initial value of the accumulator:
4328--
4429-- >>> Data.List.foldl' (+) 0 [1..100]
4530-- 5050
4631--
47−-- 'Fold's have an early termination capability e.g. the 'one' fold terminates
48−-- after consuming one element:
32+-- 'Fold's have an early termination capability e.g. the 'one' fold would
33+-- terminate on an infinite stream:
4934--
5035-- >>> Stream.fold Fold.one $ Stream.fromList [1..]
5136-- Just 1
@@ -62,8 +47,6 @@
6247-- >>> Stream.fold sumTwo $ Stream.fromList [1..100]
6348-- 3
6449--
65−-- == Parallel Composition
66−--
6750-- Folds can be combined to run in parallel on the same input. For example, to
6851-- compute the average of numbers in a stream without going through the stream
6952-- twice:
@@ -83,8 +66,6 @@
8366-- >>> Stream.fold f stream
8467-- ("Even 50","Odd 50")
8568--
86−-- == Sequential Composition
87−--
8869-- Terminating folds can be combined to parse the stream serially such that the
8970-- first fold consumes the input until it terminates and the second fold
9071-- consumes the rest of the input until it terminates:
@@ -93,8 +74,6 @@
9374-- >>> Stream.fold f $ Stream.fromList "header: hello\n"
9475-- ("header: ","hello\n")
9576--
96−-- == Splitting a Stream
97−--
9877-- A 'Fold' can be applied repeatedly on a stream to transform it to a stream
9978-- of fold results. To split a stream on newlines:
10079--
@@ -107,7 +86,10 @@
10786-- >>> Stream.fold (Fold.many f Fold.toList) $ Stream.fromList "Hello there!\nHow are you\n"
10887-- ["Hello there!\n","How are you\n"]
10988--
110−-- == Folds vs. Streams
89+-- Please see "Streamly.Internal.Data.Fold" for additional @Pre-release@
90+-- functions.
91+--
92+-- = Folds vs. Streams
11193--
11294-- We can often use streams or folds to achieve the same goal. However, streams
11395-- are more efficient in composition of producers (e.g.
@@ -143,49 +125,18 @@
143125-- Notice the similiarity in the definition of @f@ in both cases, the only
144126-- difference is the composition by @&@ vs @$@ and the use @lmap@ vs @map@, the
145127-- difference is due to output vs input side transformations.
146−--
147−-- == Fusion Limitations
148−--
149−-- Folds support stream fusion for generating loops comparable to the speed of
150−-- C. However, it has some limitations. For fusion to work, the folds must be
151−-- inlined, folds must be statically known and not generated dynamically, folds
152−-- should not be passed recursively.
153−--
154−-- Another limitation is due to the quadratic complexity causing slowdown when
155−-- too many nested compositions are used. Especially, the performance of the
156−-- Applicative instance and splitting operations (e.g. 'splitWith') degrades
157−-- quadratically (O(n^2)) when combined @n@ times, roughly 8 or less sequenced
158−-- operations are fine. For these cases folds can be converted to parsers and
159−-- then used as ParserK.
160−--
161−-- == Experimental APIs
162−--
163−-- Please refer to "Streamly.Internal.Data.Fold" for more functions that have
164−-- not yet been released.
165128
166129module Streamly.Data.Fold
167130 (
168− -- * Setup
169− -- | To execute the code examples provided in this module in ghci, please
170− -- run the following commands first.
171− --
172− -- $setup
173−
174− -- * Running A Fold
175− drive
176− -- XXX Should we have a stream returning function in fold module?
177− -- , breakStream
178−
179131 -- * Fold Type
180132
181− , Fold -- (..)
133+ Fold -- (..)
182134 , Tee (..)
183135
184136 -- * Constructors
185137 , foldl'
186138 , foldlM'
187139 , foldl1'
188− , foldlM1'
189140 , foldr'
190141
191142 -- * Folds
@@ -219,6 +170,10 @@ module Streamly.Data.Fold
219170 , toListRev
220171 , toSet
221172 , toIntSet
173+ , toMap
174+ , toMapIO
175+ , demuxToMap
176+ , demuxToMapIO
222177 , topBy
223178
224179 -- ** Non-Empty Accumulators
@@ -240,8 +195,14 @@ module Streamly.Data.Fold
240195 , uniqBy
241196 , nub
242197 , nubInt
198+ , classify
199+ , classifyIO
200+ , demux
201+ , demuxIO
243202
244203 -- ** Terminating Folds
204+ -- | These are much like lazy right folds.
205+
245206 , one
246207 , null
247208 -- , satisfy
@@ -250,7 +211,6 @@ module Streamly.Data.Fold
250211 , index
251212 , the
252213 , find
253− , findM
254214 , lookup
255215 , findIndex
256216 , elemIndex
@@ -261,10 +221,15 @@ module Streamly.Data.Fold
261221 , and
262222 , or
263223
224+ -- * Running A Fold
225+ , drive
226+ -- XXX Should we have a stream returning function in fold module?
227+ -- , breakStream
228+
264229 -- * Incremental builders
265− -- | Mutable arrays ("Streamly.Data.MutArray") are basic builders. You can
266− -- use the 'Streamly.Data.MutArray.snoc' or
267− -- 'Streamly.Data.MutArray.writeAppend' operations to incrementally build
230+ -- | Mutable arrays ("Streamly.Data.Array.Mut") are basic builders. You can
231+ -- use the 'Streamly.Data.Array.Mut.snoc' or
232+ -- 'Streamly.Data.Array.Mut.writeAppend' operations to incrementally build
268233 -- mutable arrays. The 'addOne' and 'addStream' combinators can be used to
269234 -- incrementally build any type of structure using a fold, including arrays
270235 -- or a stream of arrays.
@@ -332,11 +297,8 @@ module Streamly.Data.Fold
332297 , takeEndBy
333298 , takeEndBy_
334299
335− -- ** Splitting
300+ -- ** Serial Append
336301 , splitWith
337− , many
338− , groupsOf
339− -- , intervalsOf
340302
341303 -- ** Parallel Distribution
342304 -- | For applicative composition using distribution see
@@ -358,21 +320,14 @@ module Streamly.Data.Fold
358320 --, partitionByMinM
359321 --, partitionBy
360322
361− -- ** Key-value Collectors
362− , toMap
363− , toMapIO
364− , demuxToMap
365− , demuxToMapIO
366−
367− -- ** Key-value Scanners
368− , classify
369− , classifyIO
370− , demux
371− , demuxIO
372−
373323 -- ** Unzipping
374324 , unzip
375325
326+ -- ** Splitting
327+ , many
328+ , chunksOf
329+ -- , intervalsOf
330+
376331 -- ** Nesting
377332 , concatMap
378333
@@ -380,7 +335,6 @@ module Streamly.Data.Fold
380335 , morphInner
381336
382337 -- * Deprecated
383− , chunksOf
384338 , foldr
385339 , drainBy
386340 , last
@@ -394,22 +348,17 @@ module Streamly.Data.Fold
394348where
395349
396350import Prelude
397− hiding (Foldable(..), filter, drop, dropWhile, take, takeWhile, zipWith,
398− map, mapM_, sequence, all, any,
399− notElem, head, last, tail,
400− reverse, iterate, init, and, or, lookup, (!!),
401− scanl, scanl1, replicate, concatMap, mconcat, unzip,
351+ hiding (filter, drop, dropWhile, take, takeWhile, zipWith, foldr,
352+ foldl, map, mapM_, sequence, all, any, sum, product, elem,
353+ notElem, maximum, minimum, head, last, tail, length, null,
354+ reverse, iterate, init, and, or, lookup, foldr1, (!!),
355+ scanl, scanl1, replicate, concatMap, mconcat, foldMap, unzip,
402356 span, splitAt, break, mapM, maybe)
403357
404358import Streamly.Internal.Data.Fold
359+import Streamly.Internal.Data.Fold.Container
405360
406−#include "DocTestDataFold.hs"
407−
408−--------------------------------------------------------------------------------
409−-- Deprecated
410−--------------------------------------------------------------------------------
411−
412−{-# DEPRECATED chunksOf "Please use 'groupsOf' instead" #-}
413−{-# INLINE chunksOf #-}
414−chunksOf :: Monad m => Int -> Fold m a b -> Fold m b c -> Fold m a c
415−chunksOf = groupsOf
361+-- $setup
362+-- >>> import Data.Function ((&))
363+-- >>> import qualified Streamly.Data.Fold as Fold
364+-- >>> import qualified Streamly.Data.Stream as Stream
core/src/Streamly/Data/MutArray.hsdeleted−110
@@ -1,110 +0,0 @@
1−{-# LANGUAGE CPP #-}
2−-- |
3−-- Module : Streamly.Data.MutArray
4−-- Copyright : (c) 2022 Composewell Technologies
5−--
6−-- License : BSD3
7−-- Maintainer : streamly@composewell.com
8−-- Stability : released
9−-- Portability : GHC
10−--
11−-- This module provides a mutable version of "Streamly.Data.Array". The
12−-- contents of a mutable array can be modified in-place. For general
13−-- documentation, please refer to the original module.
14−--
15−-- Please refer to "Streamly.Internal.Data.MutArray" for functions that have
16−-- not yet been released.
17−--
18−-- For mutable arrays that work on boxed types, not requiring the 'Unbox'
19−-- constraint, please refer to "Streamly.Data.MutArray.Generic".
20−
21−module Streamly.Data.MutArray
22− (
23− -- * Setup
24− -- | To execute the code examples provided in this module in ghci, please
25− -- run the following commands first.
26− --
27− -- $setup
28−
29− -- * Mutable Array Type
30− MutArray
31−
32− -- * Construction
33−
34− -- Uninitialized Arrays
35− , emptyOf
36− , pinnedEmptyOf
37−
38− -- From containers
39− , fromListN
40− , fromList
41− , createOf
42− , create
43− -- writeLastN
44−
45− -- * Pinning & Unpinning
46− , pin
47− , unpin
48− , isPinned
49−
50− -- * Appending elements
51− , snoc
52−
53− -- * Appending streams
54− , appendN
55− , append
56−
57− -- * Inplace mutation
58− , putIndex
59− , putIndexUnsafe
60− , modifyIndex
61− , modifyIndexUnsafe
62− , modify
63−
64− -- * Random access
65− , getIndex
66− , getIndexUnsafe
67−
68− -- * Conversion
69− , toList
70−
71− -- * Streams
72− , read
73− , readRev
74−
75− -- * Unfolds
76− , reader
77− , readerRev
78−
79− -- * Casting
80− , cast
81− , asBytes
82−
83− -- * Size
84− , length
85−
86− -- * Re-exports
87− , Unbox (..)
88−
89− -- * Deprecated
90− , newPinned
91− , new
92− , pinnedNew
93− , writeN
94− , write
95− , writeAppendN
96− , writeAppend
97− )
98−where
99−
100−import Prelude hiding (length, read)
101−import Streamly.Internal.Data.MutArray
102−import Streamly.Internal.Data.Unbox (Unbox (..))
103−import Control.Monad.IO.Class (MonadIO)
104−
105−#include "DocTestDataMutArray.hs"
106−
107−{-# DEPRECATED newPinned "Please use pinnedNew instead." #-}
108−{-# INLINE newPinned #-}
109−newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (MutArray a)
110−newPinned = pinnedNew
core/src/Streamly/Data/MutArray/Generic.hsdeleted−70
@@ -1,70 +0,0 @@
1−{-# LANGUAGE CPP #-}
2−-- |
3−-- Module : Streamly.Data.MutArray.Generic
4−-- Copyright : (c) 2020 Composewell Technologies
5−-- License : BSD3-3-Clause
6−-- Maintainer : streamly@composewell.com
7−-- Stability : experimental
8−-- Portability : GHC
9−--
10−-- Unconstrained version of "Streamly.Data.MutArray" module.
11−--
12−-- See the "Streamly.Data.MutArray" module for documentation.
13−--
14−module Streamly.Data.MutArray.Generic
15−(
16− -- * Setup
17− -- | To execute the code examples provided in this module in ghci, please
18− -- run the following commands first.
19− --
20− -- $setup
21−
22− -- * Type
23− MutArray
24−
25− -- * Construction
26− , emptyOf
27− , fromListN
28− , fromList
29− , createOf
30− , create
31−
32− -- * Appending elements
33− , snoc
34−
35− -- * Inplace mutation
36− , putIndex
37− , putIndexUnsafe
38− , modifyIndex
39− , modifyIndexUnsafe
40− -- , modify
41−
42− -- * Random reads
43− , getIndex
44− , getIndexUnsafe
45−
46− -- * Conversion
47− , toList
48−
49− -- * Streams
50− , read
51− , readRev
52−
53− -- * Unfolds
54− , reader
55− -- , readerRev
56−
57− -- * Size
58− , length
59−
60− -- * Deprecated
61− , new
62− , writeN
63− , write
64− )
65−where
66−
67−import Streamly.Internal.Data.MutArray.Generic
68−import Prelude hiding (read, length)
69−
70−#include "DocTestDataMutArrayGeneric.hs"
core/src/Streamly/Data/MutByteArray.hsdeleted−106
@@ -1,106 +0,0 @@
1−-- |
2−-- Module : Streamly.Data.MutByteArray
3−-- Copyright : (c) 2023 Composewell Technologies
4−-- License : BSD3-3-Clause
5−-- Maintainer : streamly@composewell.com
6−-- Portability : GHC
7−--
8−-- A low level byte Array type 'MutByteArray', along with type classes 'Unbox'
9−-- and 'Serialize' for fast binary serialization and deserialization of Haskell
10−-- values. Serialization, deserialization performance is similar to, and in
11−-- some cases many times better than the store package. Conceptually, the
12−-- 'Serialize' type class works in the same way as store.
13−--
14−-- Serialize instances are configurable to use constructor names (see
15−-- 'Streamly.Internal.Data.MutByteArray.encodeConstrNames'), record field names (see
16−-- 'Streamly.Internal.Data.MutByteArray.encodeRecordFields') instead of binary
17−-- encoded values. This is an experimental feature which allows JSON like
18−-- properties with faster speed. For example, you can change the order of
19−-- constructors or record fields without affecting serialized value.
20−--
21−-- Higher level unboxed array modules "Streamly.Data.Array" and
22−-- "Streamly.Data.MutArray" are built on top of this module. Unboxed arrays are
23−-- essentially serialized Haskell values. Array modules provide higher level
24−-- serialization routines like 'Streamly.Internal.Data.Array.pinnedSerialize'
25−-- and 'Streamly.Internal.Data.Array.deserialize' from the
26−-- "Streamly.Internal.Data.Array" module.
27−--
28−-- == Mutable Byte Array
29−--
30−-- 'MutByteArray' is a primitive mutable array in the IO monad. 'Unbox' and
31−-- 'Serialize' type classes use this primitive array to serialize data to and
32−-- deserialize it from. This array is used to build higher level unboxed
33−-- array types 'Streamly.Data.MutArray.MutArray' and 'Streamly.Data.Array.Array'.
34−--
35−-- == Using Unbox
36−--
37−-- The 'Unbox' type class is simple and used to serialize non-recursive fixed
38−-- size data types. This type class is primarily used to implement unboxed
39−-- arrays. Unboxed arrays are just a sequence of serialized fixed length
40−-- Haskell data types. Instances of this type class can be derived using
41−-- 'Generic' or template haskell based deriving functions provided in this
42−-- module.
43−--
44−-- Writing a data type to an array using the array creation routines in
45−-- "Streamly.Data.Array" or "Streamly.Data.MutArray" (e.g. @writeN@ or
46−-- @fromListN@), serializes the type to the array. Similarly, reading the data
47−-- type from the array deserializes it. You can also serialize and deserialize
48−-- directly to and from a 'MutByteArray', using the type class methods.
49−--
50−-- == Using Serialize
51−--
52−-- The 'Serialize' type class is a superset of the 'Unbox' type class, it can
53−-- serialize variable length data types as well e.g. Haskell lists. Use
54−-- 'deriveSerialize' to derive the instances of the type class automatically
55−-- and then use the type class methods to serialize and deserialize to and from
56−-- a 'MutByteArray'.
57−--
58−-- See 'Streamly.Internal.Data.Array.pinnedSerialize' and
59−-- 'Streamly.Internal.Data.Array.deserialize' for 'Array' type based
60−-- serialization.
61−--
62−-- == Comparing serialized values
63−--
64−-- When using the `Unbox` type class the same value may result in differing
65−-- serialized bytes because of unused uninitialized data in case of sum types.
66−-- Therefore, byte comparison of serialized values is not reliable.
67−--
68−-- However, the 'Serialize' type class guarantees that the serialized values
69−-- are always exactly the same and byte comparison of serialized is reliable.
70−--
71−module Streamly.Data.MutByteArray
72− (
73−
74− -- * Mutable Byte Array
75− -- | The standard way to read from or write to a 'MutByteArray' is by using
76− -- the 'Unbox' or 'Serialize' type class methods.
77− MutByteArray
78− , isPinned
79− , pin
80− , unpin
81− , new
82− , pinnedNew
83−
84− -- * Unbox
85− , Unbox(..)
86− , deriveUnbox
87−
88− -- * Serialize
89− , Serialize(..)
90−
91− -- ** Instance Config
92− , SerializeConfig
93− , inlineAddSizeTo
94− , inlineSerializeAt
95− , inlineDeserializeAt
96−
97− -- ** Instance Deriving
98− , deriveSerialize
99− , deriveSerializeWith
100− ) where
101−
102−--------------------------------------------------------------------------------
103−-- Imports
104−--------------------------------------------------------------------------------
105−
106−import Streamly.Internal.Data.MutByteArray
core/src/Streamly/Data/Parser.hs+12−147
@@ -1,4 +1,3 @@
1−{-# LANGUAGE CPP #-}
21-- |
32-- Module : Streamly.Data.Parser
43-- Copyright : (c) 2020 Composewell Technologies
@@ -7,146 +6,19 @@
76-- Stability : pre-release
87-- Portability : GHC
98--
10−-- Parsers are stream consumers like folds with the following differences:
9+-- Fast backtracking parsers with stream fusion and native streaming
10+-- capability. Parsers are just more powerful folds, folds and parsers can be
11+-- interconverted.
1112--
12−-- * folds cannot fail but parsers can fail and backtrack.
13−-- * folds can be composed as a Tee but parsers cannot.
14−-- * folds can be used for scanning but parsers cannot.
15−-- * folds can be converted to parsers.
13+-- Note that these parsers do not report the error context (e.g. line number or
14+-- column). This may be supported in future.
1615--
17−-- This module implements parsers with stream fusion which compile to efficient
18−-- loops comparable to the speed of C.
19−--
20−-- == Using Parsers
21−--
22−-- This module provides elementary parsers and parser combinators that can be
23−-- used to parse a stream of data. Additionally, all the folds from the
24−-- "Streamly.Data.Fold" module can be converted to parsers using 'fromFold'.
25−-- All the parsing functionality provided by popular parsing libraries, and
26−-- more is available. Also see "Streamly.Unicode.Parser" module for Char stream
27−-- parsers.
28−--
29−-- A data stream can be transformed to a stream of parsed data elements. Parser
30−-- combinators can be used to create a pipeline of folds or parsers such that
31−-- the next fold or parser consumes the result of the previous parser. See
32−-- 'Streamly.Data.Stream.parse' and 'Streamly.Data.Stream.parseMany' to run
33−-- these parsers on a stream.
34−--
35−-- == Parser vs ParserK
36−--
37−-- There are two functionally equivalent parsing modules,
38−-- "Streamly.Data.Parser" (this module) and "Streamly.Data.ParserK". The latter
39−-- is a CPS based wrapper over the former, and can be used for parsing in
40−-- general. "Streamly.Data.Parser" enables stream fusion and should be
41−-- preferred over "Streamly.Data.ParserK" for high performance stream parsing
42−-- use cases. However, there are a few cases where this module is not
43−-- suitable and ParserK should be used instead.
44−--
45−-- For static fusion, parser combinators have to use strict pattern matching on
46−-- arguments of type Parser. This leads to infinte loop when a parser is
47−-- defined recursively, due to strict evaluation of the recursive call. For
48−-- example, the following implementation loops infinitely because of the
49−-- recursive use of parser @p@ in the @*>@ combinator:
50−--
51−-- >>> import Streamly.Data.Parser (Parser)
52−-- >>> import qualified Streamly.Data.Fold as Fold
53−-- >>> import qualified Streamly.Data.Parser as Parser
54−-- >>> import qualified Streamly.Data.Stream as Stream
55−-- >>> import Control.Applicative ((<|>))
56−--
57−-- >>> :{
58−-- >>> p :: Monad m => Parser Char m String
59−-- >>> p = Parser.satisfy (== '(') *> p <|> Parser.fromFold Fold.toList
60−-- >>> :}
61−--
62−-- Use ParserK when recursive use is required:
63−--
64−-- >>> import Streamly.Data.ParserK (ParserK)
65−-- >>> import qualified Streamly.Data.StreamK as StreamK
66−-- >>> import qualified Streamly.Internal.Data.StreamK as StreamK (parse)
67−-- >>> import qualified Streamly.Internal.Data.ParserK as ParserK (adapt)
68−--
69−-- >>> :{
70−-- >>> p :: Monad m => ParserK Char m String
71−-- >>> p = ParserK.adapt (Parser.satisfy (== '(')) *> p <|> ParserK.adapt (Parser.fromFold Fold.toList)
72−-- >>> :}
73−--
74−-- >>> StreamK.parse p $ StreamK.fromStream $ Stream.fromList "hello"
75−-- Right "hello"
76−--
77−-- For this reason Applicative, Alternative or Monad compositions with
78−-- recursion cannot be used with the 'Parser' type. Alternative type class based
79−-- operations like 'asum' and Alternative based generic parser combinators use
80−-- recursion. Similarly, Applicative type class based operations like
81−-- 'Prelude.sequence' use recursion. Custom implementations of many such
82−-- operations are provided in this module (e.g. 'some', 'many'), and those
83−-- should be used instead.
84−--
85−-- Another limitation of Parser type is due to the quadratic complexity causing
86−-- slowdown when too many nested compositions are used. Especially Applicative,
87−-- Monad, Alternative instances, and sequenced parsing operations (e.g. nested
88−-- 'one', and 'splitWith') degrade the performance quadratically (O(n^2)) when
89−-- combined @n@ times, roughly 8 or less sequenced parsers are fine. READ THE
90−-- DOCS OF APPLICATIVE, MONAD AND ALTERNATIVE INSTANCES.
91−--
92−-- == Streaming Parsers
93−--
94−-- With 'Streamly.Data.ParserK.ParserK' you can use the generic Alternative
95−-- type class based parsers from the
96−-- <https://hackage.haskell.org/package/parser-combinators parser-combinators>
97−-- library or similar. However, we recommend that you use the equivalent
98−-- functionality from this module for better performance and for streaming
99−-- behavior.
100−--
101−-- Firstly, the combinators in this module are faster due to stream fusion.
102−-- Secondly, these are streaming in nature as the results can be passed
103−-- directly to other stream consumers (folds or parsers). The Alternative type
104−-- class based parsers would end up buffering all the results in lists before
105−-- they can be consumed.
106−--
107−-- When recursion or heavy nesting is needed use ParserK.
108−--
109−-- == Error Reporting
110−--
111−-- These parsers do not report the error context (e.g. line number or column).
112−-- This may be supported in future.
113−--
114−-- == Monad Transformer Stack
115−--
116−-- 'MonadTrans' instance is not provided. If the 'Parser' type is the top most
117−-- layer (which should be the case almost always) you can just use 'fromEffect'
118−-- to execute the lower layer monad effects.
119−--
120−-- == Parser vs ParserK Implementation
121−--
122−-- The 'Parser' type represents a stream consumer by composing state as data
123−-- which enables stream fusion. Stream fusion generates a tight loop without
124−-- any constructor allocations between the stages, providing C like performance
125−-- for the loop. Stream fusion works when multiple functions are combined in a
126−-- pipeline statically. Therefore, the operations in this module must be
127−-- inlined and must not be used recursively to allow for stream fusion.
128−--
129−-- The 'ParserK' type represents a stream consumer by composing function calls,
130−-- therefore, a function call overhead is incurred at each composition. It is
131−-- quite fast in general but may be a few times slower than a fused parser.
132−-- However, it allows for scalable dynamic composition especially parsers can
133−-- be used in recursive calls. Using the 'ParserK' type operations like
134−-- 'splitWith' provide linear (O(n)) performance with respect to the number of
135−-- compositions.
136−--
137−-- == Experimental APIs
138−--
139−-- Please refer to "Streamly.Internal.Data.Parser" for functions that have not
140−-- yet been released.
16+-- mtl instances are not provided. If the 'Parser' type is the top most layer
17+-- (which should be the case almost always) you can just use 'fromEffect' to
18+-- execute the lower layer monad effects.
14119--
14220module Streamly.Data.Parser
14321 (
144− -- * Setup
145− -- | To execute the code examples provided in this module in ghci, please
146− -- run the following commands first.
147− --
148− -- $setup
149−
15022 -- * Parser Type
15123 Parser
15224
@@ -218,17 +90,12 @@ module Streamly.Data.Parser
21890
21991 -- ** Grouping
22092 , groupBy
221− , groupByRolling
222− , groupByRollingEither
93+ -- , groupByRolling
94+ -- , groupByRollingEither
22395
22496 -- ** Framing
225− -- , wordFramedBy
226− , wordWithQuotes
227− -- , wordProcessQuotes
228− -- , wordKeepQuotes
229−
230− -- -- * Alternative
231− -- , alt
97+ , wordFramedBy
98+ , wordQuotedBy
23299
233100 -- ** Splitting
234101 , many
@@ -243,5 +110,3 @@ where
243110
244111import Streamly.Internal.Data.Parser
245112import Prelude hiding (dropWhile, takeWhile, filter)
246−
247−#include "DocTestDataParser.hs"
core/src/Streamly/Data/ParserK.hsdeleted−139
@@ -1,139 +0,0 @@
1−{-# LANGUAGE CPP #-}
2−-- |
3−-- Module : Streamly.Data.ParserK
4−-- Copyright : (c) 2023 Composewell Technologies
5−-- License : BSD-3-Clause
6−-- Maintainer : streamly@composewell.com
7−-- Stability : pre-release
8−-- Portability : GHC
9−--
10−-- See the general notes about parsing in the "Streamly.Data.Parser" module.
11−-- This module implements a using Continuation Passing Style (CPS) wrapper over
12−-- the "Streamly.Data.Parser" module. It is as fast or faster than attoparsec.
13−--
14−-- == Parser vs ParserK
15−--
16−-- 'ParserK' is preferred over 'Streamly.Data.Parser.Parser' when extensive
17−-- applicative, alternative and monadic composition is required, or when
18−-- recursive or dynamic composition of parsers is required. The
19−-- 'Streamly.Data.Parser.Parser' type fuses statically and creates efficient
20−-- loops whereas 'ParserK' uses function call based composition and has
21−-- comparatively larger runtime overhead but it is better suited to the
22−-- specific use cases mentioned above. 'ParserK' also allows to efficient parse
23−-- a stream of arrays, it can also break the input stream into a parse result
24−-- and remaining stream so that the stream can be parsed independently in
25−-- segments.
26−--
27−-- == Using ParserK
28−--
29−-- All the parsers from the "Streamly.Data.Parser" module can be adapted to
30−-- ParserK using the 'Streamly.Data.ParserK.adaptC',
31−-- 'Streamly.Internal.Data.ParserK.adapt', and
32−-- 'Streamly.Internal.Data.ParserK.adaptCG' combinators.
33−--
34−-- 'Streamly.Data.StreamK.parseChunks' runs a parser on a stream of unboxed
35−-- arrays, this is the preferred and most efficient way to parse chunked input.
36−-- The more general 'Streamly.Data.StreamK.parseBreakChunks' function returns
37−-- the remaining stream as well along with the parse result. There are
38−-- 'Streamly.Internal.Data.StreamK.parseChunksGeneric',
39−-- 'Streamly.Internal.Data.StreamK.parseBreakChunksGeneric' as well to run
40−-- parsers on boxed arrays. 'Streamly.Internal.Data.StreamK.parse',
41−-- 'Streamly.Internal.Data.StreamK.parseBreak' run parsers on a stream of
42−-- individual elements instead of stream of arrays.
43−--
44−-- == Monadic Composition
45−--
46−-- Monad composition can be used for lookbehind parsers, we can dynamically
47−-- compose new parsers based on the results of the previously parsed values.
48−--
49−-- If we have to parse "a9" or "9a" but not "99" or "aa" we can use the
50−-- following non-monadic, backtracking parser:
51−--
52−-- >>> digits p1 p2 = ((:) <$> p1 <*> ((:) <$> p2 <*> pure []))
53−-- >>> :{
54−-- backtracking :: Monad m => ParserK Char m String
55−-- backtracking = ParserK.adapt $
56−-- digits (Parser.satisfy isDigit) (Parser.satisfy isAlpha)
57−-- <|>
58−-- digits (Parser.satisfy isAlpha) (Parser.satisfy isDigit)
59−-- :}
60−--
61−-- We know that if the first parse resulted in a digit at the first place then
62−-- the second parse is going to fail. However, we waste that information and
63−-- parse the first character again in the second parse only to know that it is
64−-- not an alphabetic char. By using lookbehind in a 'Monad' composition we can
65−-- avoid redundant work:
66−--
67−-- >>> data DigitOrAlpha = Digit Char | Alpha Char
68−--
69−-- >>> :{
70−-- lookbehind :: Monad m => ParserK Char m String
71−-- lookbehind = do
72−-- x1 <- ParserK.adapt $
73−-- Digit <$> Parser.satisfy isDigit
74−-- <|> Alpha <$> Parser.satisfy isAlpha
75−-- -- Note: the parse depends on what we parsed already
76−-- x2 <- ParserK.adapt $
77−-- case x1 of
78−-- Digit _ -> Parser.satisfy isAlpha
79−-- Alpha _ -> Parser.satisfy isDigit
80−-- return $ case x1 of
81−-- Digit x -> [x,x2]
82−-- Alpha x -> [x,x2]
83−-- :}
84−--
85−-- == Experimental APIs
86−--
87−-- Please refer to "Streamly.Internal.Data.ParserK" for functions that have
88−-- not yet been released.
89−--
90−module Streamly.Data.ParserK
91− (
92− -- * Setup
93− -- | To execute the code examples provided in this module in ghci, please
94− -- run the following commands first.
95− --
96− -- $setup
97−
98− -- * Parser Type
99− ParserK
100−
101− -- * Parsers
102− -- ** Conversions
103− , adapt
104− , adaptC
105− , adaptCG
106− -- , toParser
107−
108− -- ** Without Input
109− , fromPure
110− , fromEffect
111− , die
112−
113− -- * Deprecated
114− , fromFold
115− , fromParser
116− )
117−
118−where
119−
120−import Control.Monad.IO.Class (MonadIO)
121−import Streamly.Internal.Data.Fold (Fold)
122−import Streamly.Internal.Data.Unbox (Unbox)
123−import Streamly.Internal.Data.Array (Array)
124−import qualified Streamly.Internal.Data.Parser as ParserD
125−
126−import Streamly.Internal.Data.ParserK.Type
127−
128−#include "DocTestDataParserK.hs"
129−
130−{-# DEPRECATED fromFold "Please use \"ParserK.adaptC . Parser.fromFold\" instead." #-}
131−{-# INLINE fromFold #-}
132−fromFold :: (MonadIO m, Unbox a) => Fold m a b -> ParserK (Array a) m b
133−fromFold = adaptC . ParserD.fromFold
134−
135−{-# DEPRECATED fromParser "Please use \"adaptC\" instead." #-}
136−{-# INLINE fromParser #-}
137−fromParser ::
138− (MonadIO m, Unbox a) => ParserD.Parser a m b -> ParserK (Array a) m b
139−fromParser = adaptC
core/src/Streamly/Data/Stream.hs+125−371
@@ -1,4 +1,3 @@
1−{-# LANGUAGE CPP #-}
21-- |
32-- Module : Streamly.Data.Stream
43-- Copyright : (c) 2017 Composewell Technologies
@@ -8,55 +7,99 @@
87-- Stability : released
98-- Portability : GHC
109--
11−-- Streams represented as state machines, that fuse together when composed
12−-- statically, eliminating function calls or intermediate constructor
13−-- allocations - generating tight, efficient loops. Suitable for high
14−-- performance looping operations.
10+-- To run examples in this module:
1511--
16−-- If you need to call these operations recursively in a loop (i.e. composed
17−-- dynamically) then it is recommended to use the continuation passing style
18−-- (CPS) stream operations from the "Streamly.Data.StreamK" module. 'Stream'
19−-- and 'StreamK' types are interconvertible. See more details in the
20−-- documentation below regarding 'Stream' vs 'StreamK'.
12+-- >>> import qualified Streamly.Data.Fold as Fold
13+-- >>> import qualified Streamly.Data.Stream as Stream
2114--
22−-- Please refer to "Streamly.Internal.Data.Stream" for more functions that have
23−-- not yet been released.
15+-- We will add some more imports in the examples as needed.
2416--
25−-- Checkout the <https://github.com/composewell/streamly-examples>
17+-- For effectful streams we will use the following IO action:
18+--
19+-- >>> effect n = print n >> return n
20+--
21+-- = Overview
22+--
23+-- Streamly is a framework for modular data flow based programming and
24+-- declarative concurrency. Powerful stream fusion framework in streamly
25+-- allows high performance combinatorial programming even when using byte level
26+-- streams. Streamly API is similar to Haskell lists.
27+--
28+-- == Console Echo Example
29+--
30+-- In the following example, 'repeatM' generates an infinite stream of 'String'
31+-- by repeatedly performing the 'getLine' IO action. 'mapM' then applies
32+-- 'putStrLn' on each element in the stream converting it to stream of '()'.
33+-- Finally, 'drain' folds the stream to IO discarding the () values, thus
34+-- producing only effects.
35+--
36+-- >>> import Data.Function ((&))
37+--
38+-- >>> :{
39+-- echo =
40+-- Stream.repeatM getLine -- Stream IO String
41+-- & Stream.mapM putStrLn -- Stream IO ()
42+-- & Stream.fold Fold.drain -- IO ()
43+-- :}
44+--
45+-- This is a console echo program. It is an example of a declarative loop
46+-- written using streaming combinators. Compare it with an imperative @while@
47+-- loop.
48+--
49+-- Hopefully, this gives you an idea how we can program declaratively by
50+-- representing loops using streams. In this module, you can find all
51+-- "Data.List" like functions and many more powerful combinators to perform
52+-- common programming tasks.
53+--
54+-- == Performance Notes
55+--
56+-- The 'Stream' type represents a stream by composing state as data which
57+-- enables stream fusion. Stream fusion generates a tight loop without any
58+-- constructor allocations between the stages, providing C like performance for
59+-- the loop. Stream fusion works when multiple functions are combined in a
60+-- pipeline statically. Therefore, the operations in this module must be
61+-- inlined and must not be used recursively to allow for stream fusion.
62+--
63+-- Using the 'Stream' type binary stream construction operations like 'cons',
64+-- 'append' etc. degrade quadratically (O(n^2)) when combined many times. If
65+-- you need to combine these operations, say more than 50 times in a single
66+-- loop, then you should use the continuation style stream type 'StreamK'
67+-- instead. Also, if you need to use these operations in a recursive loop you
68+-- should use 'StreamK' instead.
69+--
70+-- The 'StreamK' type represents a stream by composing function calls,
71+-- therefore, a function call overhead is incurred at each composition. It is
72+-- quite fast in general but may be a few times slower than a fused stream.
73+-- However, it allows for scalable dynamic composition and control flow
74+-- manipulation. Using the 'StreamK' type binary operations like 'cons' and
75+-- 'append' provide linear (O(n)) performance.
76+--
77+-- 'Stream' and 'StreamK' types can be interconverted.
78+--
79+-- == Useful Idioms
80+--
81+-- >>> fromListM = Stream.sequence . Stream.fromList
82+-- >>> fromIndices f = fmap f $ Stream.enumerateFrom 0
83+--
84+-- Also see "Streamly.Internal.Data.Stream" module for many more @Pre-release@
85+-- combinators. See the <https://github.com/composewell/streamly-examples>
2686-- repository for many more real world examples of stream programming.
27−
87+--
2888module Streamly.Data.Stream
2989 (
30− -- * Setup
31− -- | To execute the code examples provided in this module in ghci, please
32− -- run the following commands first.
33− --
34− -- $setup
35−
36− -- * Overview
37− -- $overview
38−
39− -- * The Stream Type
4090 Stream
41−
4291 -- * Construction
4392 -- | Functions ending in the general shape @b -> Stream m a@.
4493 --
45− -- Useful Idioms:
46− --
47− -- >>> fromIndices f = fmap f $ Stream.enumerateFrom 0
48− -- >>> fromIndicesM f = Stream.mapM f $ Stream.enumerateFrom 0
49− -- >>> fromListM = Stream.sequence . Stream.fromList
50− -- >>> fromFoldable = StreamK.toStream . StreamK.fromFoldable
51− -- >>> fromFoldableM = Stream.sequence . fromFoldable
94+ -- See also: "Streamly.Internal.Data.Stream.Generate" for
95+ -- @Pre-release@ functions.
5296
5397 -- ** Primitives
54− -- | A fused 'Stream' is never constructed using these primitives, they are
55− -- typically generated by converting containers like list into streams, or
56− -- generated using custom functions provided in this module. The 'cons'
57− -- primitive in this module has a rare use in fusing a small number of
58− -- elements. On the other hand, it is common to construct 'StreamK' stream
59− -- using the StreamK.'StreamK.cons' primitive.
98+ -- | Primitives to construct a stream from pure values or monadic actions.
99+ -- All other stream construction and generation combinators described later
100+ -- can be expressed in terms of these primitives. However, the special
101+ -- versions provided in this module can be much more efficient in most
102+ -- cases. Users can create custom combinators using these primitives.
60103 , nil
61104 , nilM
62105 , cons
@@ -70,15 +113,6 @@ module Streamly.Data.Stream
70113
71114 -- ** From Values
72115 -- | Generate a monadic stream from a seed value or values.
73− --
74− -- Non-empty pure streams can be cycled using repeat and unfold:
75− --
76− -- >>> cycle = Stream.unfoldMany Unfold.fromList . Stream.repeat
77− --
78− -- Non-empty StreamK can be cycled using append:
79− --
80− -- >>> cycle xs = let ys = xs `StreamK.append` ys in ys
81− --
82116 , fromPure
83117 , fromEffect
84118 , repeat
@@ -123,6 +157,8 @@ module Streamly.Data.Stream
123157 -- | Functions ending in the general shape @Stream m a -> m b@ or @Stream m
124158 -- a -> m (b, Stream m a)@
125159 --
160+ -- See also: "Streamly.Internal.Data.Stream.Eliminate" for @Pre-release@
161+ -- functions.
126162
127163-- EXPLANATION: In imperative terms a fold can be considered as a loop over the stream
128164-- that reduces the stream to a single value.
@@ -196,7 +232,7 @@ module Streamly.Data.Stream
196232 -- ending in the general shape @Stream m a -> m (b, Stream m a)@
197233 , uncons
198234
199− -- ** Strict Left Folds
235+ -- ** Folding
200236 -- XXX Need to have a general parse operation here which can be used to
201237 -- express all others.
202238 , fold -- XXX rename to run? We can have a Stream.run and Fold.run.
@@ -210,69 +246,17 @@ module Streamly.Data.Stream
210246
211247 -- ** Parsing
212248 , parse
213− -- , parseBreak
249+ , parseBreak
214250
215− -- ** Lazy Right Folds
216− -- | Consuming a stream to build a right associated expression, suitable
251+ -- -- ** Lazy Right Folds
252+ -- Consuming a stream to build a right associated expression, suitable
217253 -- for lazy evaluation. Evaluation of the input happens when the output of
218254 -- the fold is evaluated, the fold output is a lazy thunk.
219255 --
220256 -- This is suitable for stream transformation operations, for example,
221257 -- operations like mapping a function over the stream.
222− , foldrM
223− , foldr
224− -- foldr1
225−
226− -- ** Specific Folds
227− -- | Streams are folded using folds in "Streamly.Data.Fold". Here are some
228− -- idioms and equivalents of Data.List APIs using folds:
229− --
230− -- >>> foldlM' f a = Stream.fold (Fold.foldlM' f a)
231− -- >>> foldl1' f = Stream.fold (Fold.foldl1' f)
232− -- >>> foldl' f a = Stream.fold (Fold.foldl' f a)
233− -- >>> drain = Stream.fold Fold.drain
234− -- >>> mapM_ f = Stream.fold (Fold.drainMapM f)
235− -- >>> length = Stream.fold Fold.length
236− -- >>> genericLength = Stream.fold Fold.lengthGeneric
237− -- >>> head = Stream.fold Fold.one
238− -- >>> last = Stream.fold Fold.latest
239− -- >>> null = Stream.fold Fold.null
240− -- >>> and = Stream.fold Fold.and
241− -- >>> or = Stream.fold Fold.or
242− -- >>> any p = Stream.fold (Fold.any p)
243− -- >>> all p = Stream.fold (Fold.all p)
244− -- >>> sum = Stream.fold Fold.sum
245− -- >>> product = Stream.fold Fold.product
246− -- >>> maximum = Stream.fold Fold.maximum
247− -- >>> maximumBy cmp = Stream.fold (Fold.maximumBy cmp)
248− -- >>> minimum = Stream.fold Fold.minimum
249− -- >>> minimumBy cmp = Stream.fold (Fold.minimumBy cmp)
250− -- >>> elem x = Stream.fold (Fold.elem x)
251− -- >>> notElem x = Stream.fold (Fold.notElem x)
252− -- >>> lookup x = Stream.fold (Fold.lookup x)
253− -- >>> find p = Stream.fold (Fold.find p)
254− -- >>> (!?) i = Stream.fold (Fold.index i)
255− -- >>> genericIndex i = Stream.fold (Fold.indexGeneric i)
256− -- >>> elemIndex x = Stream.fold (Fold.elemIndex x)
257− -- >>> findIndex p = Stream.fold (Fold.findIndex p)
258− --
259− -- Some equivalents of Data.List APIs from the Stream module:
260− --
261− -- >>> head = fmap (fmap fst) . Stream.uncons
262− -- >>> tail = fmap (fmap snd) . Stream.uncons
263− -- >>> tail = Stream.tail -- unreleased API
264− -- >>> init = Stream.init -- unreleased API
265− --
266− -- A Stream based toList fold implementation is provided below because it
267− -- has a better performance compared to the fold.
268−
269− -- Functions in Data.List, missing here:
270− -- unsnoc = Stream.parseBreak (Parser.init Fold.toList)
271− -- genericTake
272− -- genericDrop
273− -- genericSplitAt
274− -- genericReplicate
275− , toList
258+ -- , foldrM
259+ -- , foldr
276260
277261 -- * Mapping
278262 -- | Stateless one-to-one transformations. Use 'fmap' for mapping a pure
@@ -292,58 +276,24 @@ module Streamly.Data.Stream
292276 -- the stream, it can merely transform them.
293277 , sequence
294278 , mapM
295− , trace
279+ -- , trace -- XXX Use "tracing" map instead?
296280 , tap
297281 , delay
298282
299283 -- * Scanning
300284 -- | Stateful one-to-one transformations.
301285 --
302−
303− {-
304− -- ** Left scans
305− -- | We can perform scans using folds with the 'scan' combinator in the
306− -- next section. However, the combinators supplied in this section are
307− -- better amenable to stream fusion when combined with other operations.
308− -- Note that 'postscan' using folds fuses well and does not require custom
309− -- combinators like these.
310− , scanl'
311− , scanlM'
312− , scanl1'
313− , scanl1M'
314− -}
315−
316− -- ** Scanning By 'Fold'
317− -- | Useful idioms:
318− --
319− -- >>> scanl' f z = Stream.scan (Fold.foldl' f z)
320− -- >>> scanlM' f z = Stream.scan (Fold.foldlM' f z)
321− -- >>> postscanl' f z = Stream.postscan (Fold.foldl' f z)
322− -- >>> postscanlM' f z = Stream.postscan (Fold.foldlM' f z)
323− -- >>> scanl1' f = Stream.catMaybes . Stream.scan (Fold.foldl1' f)
324− -- >>> scanl1M' f = Stream.catMaybes . Stream.scan (Fold.foldlM1' f)
286+ -- See also: "Streamly.Internal.Data.Stream.Transform" for
287+ -- @Pre-release@ functions.
325288 , scan
326289 , postscan
327290 -- XXX postscan1 can be implemented using Monoids or Refolds.
328− -- The following scans from Data.List are not provided.
329− -- XXX scanl
330− -- XXX scanl1
331− -- XXX scanr
332− -- XXX scanr1
333− -- XXX mapAccumL
334− -- XXX mapAccumR
335− -- XXX inits
336− -- XXX tails
337−
338− -- ** Specific scans
339291 -- Indexing can be considered as a special type of zipping where we zip a
340292 -- stream with an index stream.
341293 , indexed
342294
343295 -- * Insertion
344296 -- | Add elements to the stream.
345− --
346− -- >>> insert = Stream.insertBy compare
347297
348298 -- Inserting elements is a special case of interleaving/merging streams.
349299 , insertBy
@@ -378,16 +328,6 @@ module Streamly.Data.Stream
378328 -- filtering folds (folds returning a 'Maybe' type) in
379329 -- "Streamly.Internal.Data.Fold" can be used along with 'scanMaybe' to
380330 -- perform stateful filtering operations in general.
381− --
382− -- Idioms and equivalents of Data.List APIs:
383− --
384− -- >>> deleteBy cmp x = Stream.scanMaybe (Fold.deleteBy cmp x)
385− -- >>> deleteBy = Stream.deleteBy -- unreleased API
386− -- >>> delete = deleteBy (==)
387− -- >>> findIndices p = Stream.scanMaybe (Fold.findIndices p)
388− -- >>> elemIndices a = findIndices (== a)
389− -- >>> uniq = Stream.scanMaybe (Fold.uniqBy (==))
390− -- >>> partition p = Stream.fold (Fold.partition Fold.toList Fold.toList) . fmap (if p then Left else Right)
391331 , scanMaybe
392332 , take
393333 , takeWhile
@@ -395,8 +335,6 @@ module Streamly.Data.Stream
395335 , drop
396336 , dropWhile
397337 , dropWhileM
398− -- XXX write to an array in reverse and then read in reverse
399− -- > dropWhileEnd = reverse . dropWhile p . reverse
400338
401339 -- XXX These are available as scans in folds. We need to check the
402340 -- performance though. If these are common and we need convenient stream
@@ -418,42 +356,34 @@ module Streamly.Data.Stream
418356 -- , elemIndices
419357
420358 -- * Combining Two Streams
421− -- | Note that these operations are suitable for statically fusing a few
422− -- streams, they have a quadratic O(n^2) time complexity wrt to the number
423− -- of streams. If you want to compose many streams dynamically using binary
424− -- combining operations see the corresponding operations in
425− -- "Streamly.Data.StreamK".
426− --
427− -- When fusing more than two streams it is more efficient if the binary
428− -- operations are composed as a balanced tree rather than a right
429− -- associative or left associative one e.g.:
430− --
431− -- >>> s1 = Stream.fromList [1,2] `Stream.append` Stream.fromList [3,4]
432− -- >>> s2 = Stream.fromList [4,5] `Stream.append` Stream.fromList [6,7]
433− -- >>> s = s1 `Stream.append` s2
434−
435359 -- ** Appending
436− -- | Equivalent of Data.List append:
437− --
438− -- >>> (++) = Stream.append
439360 , append
440361
441362 -- ** Interleaving
363+ -- | When interleaving more than two streams you may want to interleave
364+ -- them pairwise creating a balanced binary merge tree.
442365 , interleave
443366
444367 -- ** Merging
368+ -- | When merging more than two streams you may want to merging them
369+ -- pairwise creating a balanced binary merge tree.
370+ --
371+ -- Merging of @n@ streams can be performed by combining the streams pair
372+ -- wise using 'mergeMapWith' to give O(n * log n) time complexity. If used
373+ -- with 'concatMapWith' it will have O(n^2) performance.
374+
445375 , mergeBy
446376 , mergeByM
447377
448378 -- ** Zipping
449− -- | Idioms and equivalents of Data.List APIs:
379+ -- | When zipping more than two streams you may want to zip them
380+ -- pairwise creating a balanced binary tree.
450381 --
451− -- >>> zip = Stream.zipWith (,)
452− -- >>> unzip = Stream.fold (Fold.unzip Fold.toList Fold.toList)
382+ -- Zipping of @n@ streams can be performed by combining the streams pair
383+ -- wise using 'mergeMapWith' with O(n * log n) time complexity. If used
384+ -- with 'concatMapWith' it will have O(n^2) performance.
453385 , zipWith
454386 , zipWithM
455− -- XXX zipWith3,4,5,6,7
456− -- XXX unzip3,4,5,6,7
457387 -- , ZipStream (..)
458388
459389 -- ** Cross Product
@@ -486,109 +416,51 @@ module Streamly.Data.Stream
486416 -- One dimension loops are just a special case of nested loops. For
487417 -- example, 'concatMap' can degenerate to a simple map operation:
488418 --
489− -- >>> map f m = Stream.concatMap (\x -> Stream.fromPure (f x)) m
419+ -- > map f m = S.concatMap (\x -> S.fromPure (f x)) m
490420 --
491421 -- Similarly, 'concatMap' can perform filtering by mapping an element to a
492422 -- 'nil' stream:
493423 --
494− -- >>> filter p m = Stream.concatMap (\x -> if p x then Stream.fromPure x else Stream.nil) m
495− --
496− -- Idioms and equivalents of Data.List APIs:
424+ -- > filter p m = S.concatMap (\x -> if p x then S.fromPure x else S.nil) m
497425 --
498− -- >>> concat = Stream.concatMap id
499− -- >>> unlines = Stream.interposeSuffix '\n'
500− -- >>> unlines = Stream.intercalateSuffix Unfold.fromList "\n"
501− -- >>> unwords = Stream.interpose ' '
502− -- >>> unwords = Stream.intercalate Unfold.fromList " "
503426
504427 , concatEffect
505428 , concatMap
506429 , concatMapM
507430
508431 -- * Repeated Fold
509− -- | Idioms and equivalents of Data.List APIs:
510− --
511− -- >>> groupsOf n = Stream.foldMany (Fold.take n Fold.toList)
512− -- >>> groupBy eq = Stream.groupsWhile eq Fold.toList
513− -- >>> groupBy eq = Stream.parseMany (Parser.groupBy eq Fold.toList)
514− -- >>> groupsByRolling eq = Stream.parseMany (Parser.groupByRolling eq Fold.toList)
515− -- >>> groups = groupBy (==)
516432 , foldMany -- XXX Rename to foldRepeat
517− , groupsOf
518433 , parseMany
519−
520− -- * Splitting
521− -- | Idioms and equivalents of Data.List APIs:
522− --
523− -- >>> splitWithSuffix p f = Stream.foldMany (Fold.takeEndBy p f)
524− -- >>> splitOnSuffix p f = Stream.foldMany (Fold.takeEndBy_ p f)
525− -- >>> lines = splitOnSuffix (== '\n')
526− -- >>> words = Stream.wordsBy isSpace
527− -- >>> splitAt n = Stream.fold (Fold.splitAt n Fold.toList Fold.toList)
528− -- >>> span p = Parser.splitWith (,) (Parser.takeWhile p Fold.toList) (Parser.fromFold Fold.toList)
529− -- >>> break p = span (not . p)
530− , splitOn
531− , wordsBy
434+ , Array.arraysOf
532435
533436 -- * Buffered Operations
534437 -- | Operations that require buffering of the stream.
535438 -- Reverse is essentially a left fold followed by an unfold.
536− --
537− -- Idioms and equivalents of Data.List APIs:
538− --
539− -- >>> nub = Stream.fold Fold.toList . Stream.scanMaybe Fold.nub
540− -- >>> nub = Stream.ordNub -- unreleased API
541− -- >>> sortBy = StreamK.sortBy
542− -- >>> sortOn f = StreamK.sortOn -- unreleased API
543− -- >>> deleteFirstsBy = Stream.deleteFirstsBy -- unreleased
544− -- >>> (\\) = Stream.deleteFirstsBy (==) -- unreleased
545− -- >>> intersectBy = Stream.intersectBy -- unreleased
546− -- >>> intersect = Stream.intersectBy (==) -- unreleased
547− -- >>> unionBy = Stream.unionBy -- unreleased
548− -- >>> union = Stream.unionBy (==) -- unreleased
549− --
550439 , reverse
551− -- XXX transpose: write the streams to arrays and then stream transposed.
552− -- XXX subsequences
553− -- XXX permutations
554− -- , nub
555− -- , ordNub
556− -- , nubBy
557440
558441 -- * Multi-Stream folds
559442 -- | Operations that consume multiple streams at the same time.
560443 , eqBy
561444 , cmpBy
562445 , isPrefixOf
563− , isInfixOf
564− -- , isSuffixOf
565− -- , isSuffixOfUnbox
566446 , isSubsequenceOf
567447
568448 -- trimming sequences
569− -- , stripPrefix
570− -- , stripSuffix
571− -- , stripSuffixUnbox
449+ , stripPrefix
572450
573451 -- Exceptions and resource management depend on the "exceptions" package
574452 -- XXX We can have IO Stream operations not depending on "exceptions"
575453 -- in Exception.Base
576454
577455 -- * Exceptions
578− -- | Note that the stream exception handling routines catch and handle
579− -- exceptions only in the stream generation steps and not in the consumer
580− -- of the stream. For example, if we are folding or parsing a stream - any
581− -- exceptions in the fold or parse steps won't be observed by the stream
582− -- exception handlers. Exceptions in the fold or parse steps can be handled
583− -- using the fold or parse exception handling routines. You can wrap the
584− -- stream elimination function in the monad exception handler to observe
585− -- exceptions in the stream as well as the consumer.
586− --
587− -- Most of these combinators inhibit stream fusion, therefore, when
456+ -- | Most of these combinators inhibit stream fusion, therefore, when
588457 -- possible, they should be called in an outer loop to mitigate the cost.
589458 -- For example, instead of calling them on a stream of chars call them on a
590459 -- stream of arrays before flattening it to a stream of chars.
591460 --
461+ -- See also: "Streamly.Internal.Data.Stream.Exception" for
462+ -- @Pre-release@ functions.
463+
592464 , onException
593465 , handle
594466
@@ -596,19 +468,8 @@ module Streamly.Data.Stream
596468 -- | 'bracket' is the most general resource management operation, all other
597469 -- operations can be expressed using it. These functions have IO suffix
598470 -- because the allocation and cleanup functions are IO actions. For
599− -- generalized allocation and cleanup functions, see the functions without
471+ -- generalized allocation and cleanup functions see the functions without
600472 -- the IO suffix in the "streamly" package.
601− --
602− -- Note that these operations bracket the stream generation only, they do
603− -- not cover the stream consumer. This means if an exception occurs in
604− -- the consumer of the stream (e.g. in a fold or parse step) then the
605− -- exception won't be observed by the stream resource handlers, in that
606− -- case the resource cleanup handler runs when the stream is garbage
607− -- collected.
608− --
609− -- Monad level resource management can always be used around the stream
610− -- elimination functions, such a function can observe exceptions in both
611− -- the stream and its consumer.
612473 , before
613474 , afterIO
614475 , finallyIO
@@ -622,125 +483,18 @@ module Streamly.Data.Stream
622483 , runReaderT
623484 , runStateT
624485
625− -- XXX Arrays could be different types, therefore, this should be in
626− -- specific array module. Or maybe we should abstract over array types.
627− -- * Stream of Arrays
628− , Array.chunksOf
486+ -- -- * Stream Types
487+ -- $serial
488+ -- , Interleave
489+ -- , Zip
629490 )
630491where
631492
632−import Streamly.Internal.Data.Stream
493+import qualified Streamly.Internal.Data.Array.Type as Array
494+import Streamly.Internal.Data.Stream.StreamD
633495import Prelude
634496 hiding (filter, drop, dropWhile, take, takeWhile, zipWith, foldr,
635− mapM, sequence, reverse, iterate, foldr1, repeat, replicate,
636− concatMap)
637−
638−import qualified Streamly.Internal.Data.Array.Type as Array
639−
640−#include "DocTestDataStream.hs"
641−
642−-- $overview
643−--
644−-- Streamly is a framework for modular data flow based programming and
645−-- declarative concurrency. Powerful stream fusion framework in streamly
646−-- allows high performance combinatorial programming even when using byte level
647−-- streams. Streamly API is similar to Haskell lists.
648−--
649−-- == Console Echo Example
650−--
651−-- In the following example, 'repeatM' generates an infinite stream of 'String'
652−-- by repeatedly performing the 'getLine' IO action. 'mapM' then applies
653−-- 'putStrLn' on each element in the stream converting it to stream of '()'.
654−-- Finally, 'drain' folds the stream to IO discarding the () values, thus
655−-- producing only effects.
656−--
657−-- >>> import Data.Function ((&))
658−--
659−-- >>> :{
660−-- echo =
661−-- Stream.repeatM getLine -- Stream IO String
662−-- & Stream.mapM putStrLn -- Stream IO ()
663−-- & Stream.fold Fold.drain -- IO ()
664−-- :}
665−--
666−-- This is a console echo program. It is an example of a declarative loop
667−-- written using streaming combinators. Compare it with an imperative @while@
668−-- loop.
669−--
670−-- Hopefully, this gives you an idea how we can program declaratively by
671−-- representing loops using streams. In this module, you can find all
672−-- "Data.List" like functions and many more powerful combinators to perform
673−-- common programming tasks.
674−--
675−-- == Stream Fusion
676−--
677−-- The fused 'Stream' type in this module employs stream fusion for C-like
678−-- performance when looping over data. It represents the stream as a state
679−-- machine using an explicit state, and a step function working on the state. A
680−-- typical stream operation consumes elements from the previous state machine
681−-- in a stream pipeline, transforms the elements and yields new values for the
682−-- next stage to consume. The stream operations are modular and represent a
683−-- single task, they have no knowledge of previous or next operation on the
684−-- elements.
685−--
686−-- A typical stream pipeline consists of a stream producer, several stream
687−-- transformation operations and a stream consumer. All these operations taken
688−-- together form a closed loop processing the stream elements. Elements are
689−-- transferred between stages using a boxed data constructor. However, all the
690−-- stages of the pipeline are fused together by GHC, eliminating the boxing of
691−-- intermediate constructors, and thus forming a tight C like loop without any
692−-- boxed data being used in the loop.
693−--
694−-- Stream fusion works effectively when:
695−--
696−-- * the stream pipeline is composed statically (known at compile time)
697−-- * all the operations forming the loop are inlined
698−-- * the loop is not recursively defined, recursion breaks inlining
699−--
700−-- If these conditions cannot be met, the CPS style stream type 'StreamK' may
701−-- turn out to be a better choice than the fused stream type 'Stream'.
702−--
703−-- == Stream vs StreamK
704−--
705−-- The fused stream model avoids constructor allocations and function call
706−-- overheads. However, the stream is represented as a state machine, and to
707−-- generate stream elements it has to navigate the decision tree of the state
708−-- machine. Moreover, the state machine is cranked for each element in the
709−-- stream. This performs extremely well when the number of states are limited.
710−-- The state machine starts getting expensive as the number of states increase.
711−-- For example, generating a stream from a list requires a single state and is
712−-- very efficient, even if it has millions of elements. However, using 'cons'
713−-- to construct a million element stream would be a disaster.
714−--
715−-- A typical worst case scenario for fused stream model is a large number of
716−-- `cons` or `append` operations. A few static `cons` or `append` operations
717−-- are very fast and much faster than a CPS style stream because CPS involves a
718−-- function call for each element whereas fused stream involves a few
719−-- conditional branches in the state machine. However, constructing a large
720−-- stream using `cons` introduces as many states in the state machine as the
721−-- number of elements. If we compose `cons` as a balanced binary tree it will
722−-- take @n * log n@ time to navigate the tree, and @n * n@ if it is a right
723−-- associative composition.
724−--
725−-- Operations like 'cons' or 'append'; are typically recursively called to
726−-- construct a lazy infinite stream. For such use cases the CPS style 'StreamK'
727−-- should be used. CPS streams do not have a state machine that needs to be
728−-- cranked for each element, past state has no effect on the future element
729−-- processing. However, CPS incurs a function call overhead for each element
730−-- processed, the overhead could be large compared to a fused state machine
731−-- even if it has many states. However, because of its linear performance
732−-- characterstics, after a certain threshold of stream compositions the CPS
733−-- stream would perform much better than the quadratic fused stream operations.
734−--
735−-- As a general guideline, you need to use 'StreamK' when you have to use
736−-- 'cons', 'append' or other operations having quadratic complexity at a large
737−-- scale. Typically, in such cases you need to compose the stream recursively,
738−-- by calling an operation in a loop. The decision to compose the stream is
739−-- taken at run time rather than statically at compile time.
740−--
741−-- Typically you would compose a 'StreamK' of chunks of data so that the
742−-- StreamK overhead is not high, and then process the chunks using 'Stream' by
743−-- using statically fused stream pipeline operations on the chunks.
744−--
745−-- 'Stream' and 'StreamK' types can be interconverted. See
746−-- "Streamly.Data.StreamK" module for conversion operations.
497+ foldl, map, mapM, mapM_, sequence, all, any, sum, product, elem,
498+ notElem, maximum, minimum, head, last, tail, length, null,
499+ reverse, iterate, init, and, or, lookup, foldr1, (!!),
500+ scanl, scanl1, repeat, replicate, concatMap, span)
core/src/Streamly/Data/Stream/StreamDK.hsnew file+505
@@ -0,0 +1,505 @@
1+-- |
2+-- Module : Streamly.Data.Stream.StreamDK
3+-- Copyright : (c) 2017 Composewell Technologies
4+--
5+-- License : BSD3
6+-- Maintainer : streamly@composewell.com
7+-- Stability : released
8+-- Portability : GHC
9+--
10+-- To run examples in this module:
11+--
12+-- >>> import qualified Streamly.Data.Fold as Fold
13+-- >>> import qualified Streamly.Data.Stream.StreamDK as Stream
14+--
15+-- We will add some more imports in the examples as needed.
16+--
17+-- For effectful streams we will use the following IO action:
18+--
19+-- >>> effect n = print n >> return n
20+--
21+-- = Overview
22+--
23+-- Streamly is a framework for modular data flow based programming and
24+-- declarative concurrency. Powerful stream fusion framework in streamly
25+-- allows high performance combinatorial programming even when using byte level
26+-- streams. Streamly API is similar to Haskell lists.
27+--
28+-- Streams can be constructed like lists, except that they use 'nil' instead of
29+-- '[]' and 'cons' instead of ':'.
30+--
31+-- `cons` adds a pure value at the head of the stream:
32+--
33+-- >>> import Streamly.Data.Stream.StreamDK (Stream, cons, consM, nil)
34+-- >>> stream = 1 `cons` 2 `cons` nil :: Stream IO Int
35+-- >>> Stream.fold Fold.toList stream -- IO [Int]
36+-- [1,2]
37+--
38+-- `consM` adds an effect at the head of the stream:
39+--
40+-- >>> stream = effect 1 `consM` effect 2 `consM` nil
41+-- >>> Stream.fold Fold.toList stream
42+-- 1
43+-- 2
44+-- [1,2]
45+--
46+-- == Console Echo Example
47+--
48+-- In the following example, 'repeatM' generates an infinite stream of 'String'
49+-- by repeatedly performing the 'getLine' IO action. 'mapM' then applies
50+-- 'putStrLn' on each element in the stream converting it to stream of '()'.
51+-- Finally, 'drain' folds the stream to IO discarding the () values, thus
52+-- producing only effects.
53+--
54+-- >>> import Data.Function ((&))
55+--
56+-- >>> :{
57+-- echo =
58+-- Stream.repeatM getLine -- Stream IO String
59+-- & Stream.mapM putStrLn -- Stream IO ()
60+-- & Stream.fold Fold.drain -- IO ()
61+-- :}
62+--
63+-- This is a console echo program. It is an example of a declarative loop
64+-- written using streaming combinators. Compare it with an imperative @while@
65+-- loop.
66+--
67+-- Hopefully, this gives you an idea how we can program declaratively by
68+-- representing loops using streams. In this module, you can find all
69+-- "Data.List" like functions and many more powerful combinators to perform
70+-- common programming tasks.
71+--
72+-- == Useful Idioms
73+--
74+-- >>> fromListM = Stream.sequence . Stream.fromList
75+-- >>> fromFoldableM = Stream.sequence . Stream.fromFoldable
76+-- >>> fromIndices f = fmap f $ Stream.enumerateFrom 0
77+--
78+-- Also see "Streamly.Internal.Data.Stream.StreamDK" module for many more @Pre-release@
79+-- combinators. See the <https://github.com/composewell/streamly-examples>
80+-- repository for many more real world examples of stream programming.
81+--
82+-- == Performance Notes
83+--
84+-- Operations annotated as /CPS/ force the use of continuation passing style
85+-- streams. Therefore, such operations are not subjected to stream fusion.
86+-- However, for some of these operations you can find fusible alternatives in
87+-- the internal modules, which are perfectly fine to use but you need to
88+-- understand the implications especially the O(n^2) nature of those
89+-- operations..
90+--
91+module Streamly.Data.Stream.StreamDK
92+ (
93+ Stream
94+ -- * Construction
95+ -- | Functions ending in the general shape @b -> Stream m a@.
96+ --
97+ -- See also: "Streamly.Internal.Data.Stream.Generate" for
98+ -- @Pre-release@ functions.
99+
100+ -- ** Primitives
101+ -- | Primitives to construct a stream from pure values or monadic actions.
102+ -- All other stream construction and generation combinators described later
103+ -- can be expressed in terms of these primitives. However, the special
104+ -- versions provided in this module can be much more efficient in most
105+ -- cases. Users can create custom combinators using these primitives.
106+ , nil
107+ , nilM
108+ , cons
109+ , consM
110+ -- , cons2 -- fused version
111+ -- , consM2 -- fused version
112+
113+ -- ** Unfolding
114+ -- | 'unfoldrM' is the most general way of generating a stream efficiently.
115+ -- All other generation operations can be expressed using it.
116+ , unfoldr
117+ , unfoldrM
118+
119+ -- ** From Values
120+ -- | Generate a monadic stream from a seed value or values.
121+ , fromPure
122+ , fromEffect
123+ , repeat
124+ , repeatM
125+ , replicate
126+ , replicateM
127+
128+ -- Note: Using enumeration functions e.g. 'Prelude.enumFromThen' turns out
129+ -- to be slightly faster than the idioms like @[from, then..]@.
130+ --
131+ -- ** Enumeration
132+ -- | We can use the 'Enum' type class to enumerate a type producing a list
133+ -- and then convert it to a stream:
134+ --
135+ -- @
136+ -- 'fromList' $ 'Prelude.enumFromThen' from then
137+ -- @
138+ --
139+ -- However, this is not particularly efficient.
140+ -- The 'Enumerable' type class provides corresponding functions that
141+ -- generate a stream instead of a list, efficiently.
142+
143+ , Enumerable (..)
144+ , enumerate
145+ , enumerateTo
146+
147+ -- ** Iteration
148+ , iterate
149+ , iterateM
150+
151+ -- ** From Containers
152+ -- | Convert an input structure, container or source into a stream. All of
153+ -- these can be expressed in terms of primitives.
154+ , fromList
155+ , fromFoldable
156+
157+ -- ** From Unfolds
158+ -- | Most of the above stream generation operations can also be expressed
159+ -- using the corresponding unfolds in the "Streamly.Data.Unfold" module.
160+ , unfold -- XXX rename to fromUnfold?
161+
162+ -- * Elimination
163+ -- | Functions ending in the general shape @Stream m a -> m b@ or @Stream m
164+ -- a -> m (b, Stream m a)@
165+ --
166+ -- See also: "Streamly.Internal.Data.Stream.Eliminate" for @Pre-release@
167+ -- functions.
168+
169+-- EXPLANATION: In imperative terms a fold can be considered as a loop over the stream
170+-- that reduces the stream to a single value.
171+-- Left and right folds both use a fold function @f@ and an identity element
172+-- @z@ (@zero@) to deconstruct a recursive data structure and reconstruct a
173+-- new data structure. The new structure may be a recursive construction (a
174+-- container) or a non-recursive single value reduction of the original
175+-- structure.
176+--
177+-- Both right and left folds are mathematical duals of each other, they are
178+-- functionally equivalent. Operationally, a left fold on a left associated
179+-- structure behaves exactly in the same way as a right fold on a right
180+-- associated structure. Similarly, a left fold on a right associated structure
181+-- behaves in the same way as a right fold on a left associated structure.
182+-- However, the behavior of a right fold on a right associated structure is
183+-- operationally different (even though functionally equivalent) than a left
184+-- fold on the same structure.
185+--
186+-- On right associated structures like Haskell @cons@ lists or Streamly
187+-- streams, a lazy right fold is naturally suitable for lazy recursive
188+-- reconstruction of a new structure, while a strict left fold is naturally
189+-- suitable for efficient reduction. In right folds control is in the hand of
190+-- the @puller@ whereas in left folds the control is in the hand of the
191+-- @pusher@.
192+--
193+-- The behavior of right and left folds are described in detail in the
194+-- individual fold's documentation. To illustrate the two folds for right
195+-- associated @cons@ lists:
196+--
197+-- > foldr :: (a -> b -> b) -> b -> [a] -> b
198+-- > foldr f z [] = z
199+-- > foldr f z (x:xs) = x `f` foldr f z xs
200+-- >
201+-- > foldl :: (b -> a -> b) -> b -> [a] -> b
202+-- > foldl f z [] = z
203+-- > foldl f z (x:xs) = foldl f (z `f` x) xs
204+--
205+-- @foldr@ is conceptually equivalent to:
206+--
207+-- > foldr f z [] = z
208+-- > foldr f z [x] = f x z
209+-- > foldr f z xs = foldr f (foldr f z (tail xs)) [head xs]
210+--
211+-- @foldl@ is conceptually equivalent to:
212+--
213+-- > foldl f z [] = z
214+-- > foldl f z [x] = f z x
215+-- > foldl f z xs = foldl f (foldl f z (init xs)) [last xs]
216+--
217+-- Left and right folds are duals of each other.
218+--
219+-- @
220+-- foldr f z xs = foldl (flip f) z (reverse xs)
221+-- foldl f z xs = foldr (flip f) z (reverse xs)
222+-- @
223+--
224+-- More generally:
225+--
226+-- @
227+-- foldr f z xs = foldl g id xs z where g k x = k . f x
228+-- foldl f z xs = foldr g id xs z where g x k = k . flip f x
229+-- @
230+--
231+
232+-- NOTE: Folds are inherently serial as each step needs to use the result of
233+-- the previous step. However, it is possible to fold parts of the stream in
234+-- parallel and then combine the results using a monoid.
235+
236+ -- ** Primitives
237+ -- Consuming a part of the stream and returning the rest. Functions
238+ -- ending in the general shape @Stream m a -> m (b, Stream m a)@
239+ , uncons
240+
241+ -- ** Folding
242+ -- XXX Need to have a general parse operation here which can be used to
243+ -- express all others.
244+ , fold -- XXX rename to run? We can have a Stream.run and Fold.run.
245+ -- XXX fold1 can be achieved using Monoids or Refolds.
246+ -- XXX We can call this just "break" and parseBreak as "munch"
247+ , foldBreak
248+
249+ -- XXX should we have a Fold returning function in stream module?
250+ -- , foldAdd
251+ -- , buildl
252+
253+ -- ** Parsing
254+ , parse
255+ , parseBreak
256+
257+ -- -- ** Lazy Right Folds
258+ -- Consuming a stream to build a right associated expression, suitable
259+ -- for lazy evaluation. Evaluation of the input happens when the output of
260+ -- the fold is evaluated, the fold output is a lazy thunk.
261+ --
262+ -- This is suitable for stream transformation operations, for example,
263+ -- operations like mapping a function over the stream.
264+ -- , foldrM
265+ -- , foldr
266+
267+ -- * Mapping
268+ -- | Stateless one-to-one transformations. Use 'fmap' for mapping a pure
269+ -- function on a stream.
270+
271+ -- EXPLANATION:
272+ -- In imperative terms a map operation can be considered as a loop over
273+ -- the stream that transforms the stream into another stream by performing
274+ -- an operation on each element of the stream.
275+ --
276+ -- 'map' is the least powerful transformation operation with strictest
277+ -- guarantees. A map, (1) is a stateless loop which means that no state is
278+ -- allowed to be carried from one iteration to another, therefore,
279+ -- operations on different elements are guaranteed to not affect each
280+ -- other, (2) is a strictly one-to-one transformation of stream elements
281+ -- which means it guarantees that no elements can be added or removed from
282+ -- the stream, it can merely transform them.
283+ , sequence
284+ , mapM
285+ -- , trace -- XXX Use "tracing" map instead?
286+ , tap
287+ , delay
288+
289+ -- * Scanning
290+ -- | Stateful one-to-one transformations.
291+ --
292+ -- See also: "Streamly.Internal.Data.Stream.Transform" for
293+ -- @Pre-release@ functions.
294+ , scan
295+ , postscan
296+ -- XXX postscan1 can be implemented using Monoids or Refolds.
297+ -- Indexing can be considered as a special type of zipping where we zip a
298+ -- stream with an index stream.
299+ , indexed
300+
301+ -- * Insertion
302+ -- | Add elements to the stream.
303+
304+ -- Inserting elements is a special case of interleaving/merging streams.
305+ , insertBy
306+ , intersperseM
307+ , intersperseM_
308+ , intersperse
309+
310+ -- * Filtering
311+ -- | Remove elements from the stream.
312+
313+ -- ** Stateless Filters
314+ -- | 'mapMaybeM' is the most general stateless filtering operation. All
315+ -- other filtering operations can be expressed using it.
316+
317+ -- EXPLANATION:
318+ -- In imperative terms a filter over a stream corresponds to a loop with a
319+ -- @continue@ clause for the cases when the predicate fails.
320+
321+ , mapMaybe
322+ , mapMaybeM
323+ , filter
324+ , filterM
325+
326+ -- Filter and concat
327+ , catMaybes
328+ , catLefts
329+ , catRights
330+ , catEithers
331+
332+ -- ** Stateful Filters
333+ -- | 'scanMaybe' is the most general stateful filtering operation. The
334+ -- filtering folds (folds returning a 'Maybe' type) in
335+ -- "Streamly.Internal.Data.Fold" can be used along with 'scanMaybe' to
336+ -- perform stateful filtering operations in general.
337+ , scanMaybe
338+ , take
339+ , takeWhile
340+ , takeWhileM
341+ , drop
342+ , dropWhile
343+ , dropWhileM
344+
345+ -- XXX These are available as scans in folds. We need to check the
346+ -- performance though. If these are common and we need convenient stream
347+ -- ops then we can expose these.
348+
349+ -- , deleteBy
350+ -- , uniq
351+ -- , uniqBy
352+
353+ -- -- ** Sampling
354+ -- , strideFromThen
355+
356+ -- -- ** Searching
357+ -- Finding the presence or location of an element, a sequence of elements
358+ -- or another stream within a stream.
359+
360+ -- -- ** Searching Elements
361+ -- , findIndices
362+ -- , elemIndices
363+
364+ -- * Combining Two Streams
365+ -- ** Appending
366+ , append
367+
368+ -- ** Interleaving
369+ , interleave
370+ -- , interleave2
371+
372+ -- ** Merging
373+ -- | Merging of @n@ streams can be performed by combining the streams pair
374+ -- wise using 'mergeMapWith' to give O(n * log n) time complexity. If used
375+ -- with 'concatMapWith' it will have O(n^2) performance.
376+
377+ , mergeBy
378+ , mergeByM
379+ -- , mergeBy2
380+ -- , mergeByM2
381+
382+ -- ** Zipping
383+ -- | Zipping of @n@ streams can be performed by combining the streams pair
384+ -- wise using 'mergeMapWith' with O(n * log n) time complexity. If used
385+ -- with 'concatMapWith' it will have O(n^2) performance.
386+ , zipWith
387+ , zipWithM
388+ -- , zipWith2
389+ -- , zipWithM2
390+ , ZipStream (..)
391+
392+ -- ** Cross Product
393+ -- XXX The argument order in this operation is such that it seems we are
394+ -- transforming the first stream using the second stream because the second
395+ -- stream is evaluated many times or buffered and better be finite, first
396+ -- stream could potentially be infinite. In the tradition of using the
397+ -- transformed stream at the end we can have a flipped version called
398+ -- "crossMap" or "nestWith".
399+ , crossWith
400+ -- , cross
401+ -- , joinInner
402+ , CrossStream (..)
403+
404+ -- * Unfold Each
405+ , unfoldMany
406+ , intercalate
407+ , intercalateSuffix
408+
409+ -- * Stream of streams
410+ -- | Stream operations like map and filter represent loop processing in
411+ -- imperative programming terms. Similarly, the imperative concept of
412+ -- nested loops are represented by streams of streams. The 'concatMap'
413+ -- operation represents nested looping.
414+ -- A 'concatMap' operation loops over the input stream and then for each
415+ -- element of the input stream generates another stream and then loops over
416+ -- that inner stream as well producing effects and generating a single
417+ -- output stream.
418+ --
419+ -- One dimension loops are just a special case of nested loops. For
420+ -- example, 'concatMap' can degenerate to a simple map operation:
421+ --
422+ -- > map f m = S.concatMap (\x -> S.fromPure (f x)) m
423+ --
424+ -- Similarly, 'concatMap' can perform filtering by mapping an element to a
425+ -- 'nil' stream:
426+ --
427+ -- > filter p m = S.concatMap (\x -> if p x then S.fromPure x else S.nil) m
428+ --
429+
430+ , concatEffect
431+ , concatMapWith
432+ , concatMap
433+ , concatMapM
434+ , mergeMapWith
435+
436+ -- * Repeated Fold
437+ , foldMany -- XXX Rename to foldRepeat
438+ , parseMany
439+ , arraysOf
440+
441+ -- * Buffered Operations
442+ -- | Operations that require buffering of the stream.
443+ -- Reverse is essentially a left fold followed by an unfold.
444+ , reverse
445+ , sortBy
446+
447+ -- * Multi-Stream folds
448+ -- | Operations that consume multiple streams at the same time.
449+ , eqBy
450+ , cmpBy
451+ , isPrefixOf
452+ , isSubsequenceOf
453+
454+ -- trimming sequences
455+ , stripPrefix
456+
457+ -- Exceptions and resource management depend on the "exceptions" package
458+ -- XXX We can have IO Stream operations not depending on "exceptions"
459+ -- in Exception.Base
460+
461+ -- * Exceptions
462+ -- | Most of these combinators inhibit stream fusion, therefore, when
463+ -- possible, they should be called in an outer loop to mitigate the cost.
464+ -- For example, instead of calling them on a stream of chars call them on a
465+ -- stream of arrays before flattening it to a stream of chars.
466+ --
467+ -- See also: "Streamly.Internal.Data.Stream.Exception" for
468+ -- @Pre-release@ functions.
469+
470+ , onException
471+ , handle
472+
473+ -- * Resource Management
474+ -- | 'bracket' is the most general resource management operation, all other
475+ -- operations can be expressed using it. These functions have IO suffix
476+ -- because the allocation and cleanup functions are IO actions. For
477+ -- generalized allocation and cleanup functions see the functions without
478+ -- the IO suffix in the "streamly" package.
479+ , before
480+ , afterIO
481+ , finallyIO
482+ , bracketIO
483+ , bracketIO3
484+
485+ -- * Transforming Inner Monad
486+
487+ , morphInner
488+ , liftInner
489+ , runReaderT
490+ , runStateT
491+
492+ -- -- * Stream Types
493+ -- $serial
494+ -- , Interleave
495+ -- , Zip
496+ )
497+where
498+
499+import Streamly.Internal.Data.Stream.StreamDK
500+import Prelude
501+ hiding (filter, drop, dropWhile, take, takeWhile, zipWith, foldr,
502+ foldl, map, mapM, mapM_, sequence, all, any, sum, product, elem,
503+ notElem, maximum, minimum, head, last, tail, length, null,
504+ reverse, iterate, init, and, or, lookup, foldr1, (!!),
505+ scanl, scanl1, repeat, replicate, concatMap, span)
core/src/Streamly/Data/Stream/StreamK.hsnew file+93
@@ -0,0 +1,93 @@
1+-- |
2+-- Module : Streamly.Data.Stream.StreamK
3+-- Copyright : (c) 2017 Composewell Technologies
4+--
5+-- License : BSD3
6+-- Maintainer : streamly@composewell.com
7+-- Stability : released
8+-- Portability : GHC
9+--
10+-- Streams using Continuation Passing Style (CPS). See notes in "Data.Stream"
11+-- module to know when to use this module.
12+
13+-- primitive/open loop operations that can be used recursively e.g. uncons,
14+-- foldBreak, parseBreak should not be converted from StreamD for use in
15+-- StreamK, instead native StreamK impl should be used.
16+--
17+-- Closed loop operations like repeat, replicate, iterate etc can be converted
18+-- from StreamD.
19+--
20+-- In the last phase any operation like (toStreamK . f . toStreamD) should be
21+-- rewritten to a K version of f.
22+-- XXX Need to add rewrite rules for all missing StreamD operations.
23+--
24+module Streamly.Data.Stream.StreamK
25+ (
26+ StreamK
27+
28+ -- * Construction
29+ -- ** Primitives
30+ , nil
31+ , nilM
32+ , cons
33+ , consM
34+
35+ -- ** From Values
36+ , fromPure
37+ , fromEffect
38+
39+ -- ** From Stream
40+ , fromStream
41+ , toStream
42+
43+ -- ** From Containers
44+ , fromFoldable
45+
46+ -- * Elimination
47+
48+ -- ** Primitives
49+ , uncons
50+
51+ -- -- ** Folding
52+ -- , foldBreak
53+
54+ -- ** Parsing
55+ , parseBreak
56+
57+ -- * Combining Two Streams
58+ -- ** Appending
59+ , append
60+
61+ -- ** Interleaving
62+ , interleave
63+
64+ -- ** Merging
65+ , mergeBy
66+ , mergeByM
67+
68+ -- ** Zipping
69+ , zipWith
70+ , zipWithM
71+
72+ -- ** Cross Product
73+ -- XXX is "bind/concatFor" better to have than crossWith?
74+ -- crossWith f xs1 xs2 = concatFor xs1 (\x -> fmap (f x) xs2)
75+ , crossWith
76+ -- , cross
77+ -- , joinInner
78+ -- , CrossStreamK (..)
79+
80+ -- * Stream of streams
81+ , concatEffect
82+ -- , concatMap
83+ , concatMapWith
84+ , mergeMapWith
85+
86+ -- * Buffered Operations
87+ , reverse
88+ , sortBy
89+ )
90+where
91+
92+import Streamly.Internal.Data.Stream.StreamK
93+import Prelude hiding (reverse, zipWith)
core/src/Streamly/Data/Stream/Zip.hsnew file+16
@@ -0,0 +1,16 @@
1+-- |
2+-- Module : Streamly.Data.Stream.Zip
3+-- Copyright : (c) 2017 Composewell Technologies
4+--
5+-- License : BSD3
6+-- Maintainer : streamly@composewell.com
7+-- Stability : released
8+-- Portability : GHC
9+--
10+module Streamly.Data.Stream.Zip
11+ (
12+ ZipStream (..)
13+ )
14+where
15+
16+import Streamly.Internal.Data.Stream.Zip
core/src/Streamly/Data/StreamK.hsdeleted−194
@@ -1,194 +0,0 @@
1−{-# LANGUAGE CPP #-}
2−-- |
3−-- Module : Streamly.Data.StreamK
4−-- Copyright : (c) 2017 Composewell Technologies
5−--
6−-- License : BSD3
7−-- Maintainer : streamly@composewell.com
8−-- Stability : released
9−-- Portability : GHC
10−--
11−-- Streams represented as chains of functions calls using Continuation Passing
12−-- Style (CPS), suitable for dynamically composing potentially large number of
13−-- streams.
14−--
15−-- Unlike the statically fused operations in "Streamly.Data.Stream", StreamK
16−-- operations are less efficient, involving a function call overhead for each
17−-- element, but they exhibit linear O(n) time complexity wrt to the number of
18−-- stream compositions. Therefore, they are suitable for dynamically composing
19−-- streams e.g. appending potentially infinite streams in recursive loops.
20−-- While fused streams can be used to efficiently process elements as small as
21−-- a single byte, CPS streams are typically used on bigger chunks of data to
22−-- avoid the larger overhead per element. For more details See the @Stream vs
23−-- StreamK@ section in the "Streamly.Data.Stream" module.
24−--
25−-- In addition to the combinators in this module, you can use operations from
26−-- "Streamly.Data.Stream" for StreamK as well by converting StreamK to Stream
27−-- ('toStream'), and vice-versa ('fromStream'). Please refer to
28−-- "Streamly.Internal.Data.StreamK" for more functions that have not yet been
29−-- released.
30−--
31−-- For documentation see the corresponding combinators in
32−-- "Streamly.Data.Stream". Documentation has been omitted in this module unless
33−-- there is a difference worth mentioning or if the combinator does not exist
34−-- in "Streamly.Data.Stream".
35−
36−-- Notes:
37−--
38−-- primitive/open loop operations that can be used recursively e.g. uncons,
39−-- foldBreak, parseBreak should not be converted from StreamD for use in
40−-- StreamK, instead native StreamK impl should be used.
41−--
42−-- Closed loop operations like repeat, replicate, iterate etc can be converted
43−-- from StreamD.
44−--
45−-- In the last phase any operation like (toStreamK . f . toStreamD) should be
46−-- rewritten to a K version of f.
47−-- XXX Need to add rewrite rules for all missing StreamD operations.
48−--
49−module Streamly.Data.StreamK
50− (
51− -- * Setup
52− -- | To execute the code examples provided in this module in ghci, please
53− -- run the following commands first.
54− --
55− -- $setup
56−
57− -- * Overview
58− -- $overview
59−
60− -- * Type
61− StreamK
62−
63− -- * Construction
64− -- ** Primitives
65− -- | Primitives to construct a stream from pure values or monadic actions.
66− -- All other stream construction and generation combinators described later
67− -- can be expressed in terms of these primitives. However, the special
68− -- versions provided in this module can be much more efficient in some
69− -- cases. Users can create custom combinators using these primitives.
70− , nil
71− , nilM
72− , cons
73− , consM
74−
75− -- ** From Values
76− , fromPure
77− , fromEffect
78−
79− -- ** From Stream
80− -- | Please note that 'Stream' type does not observe any exceptions from
81− -- the consumer of the stream whereas 'StreamK' does.
82− , fromStream
83− , toStream
84−
85− -- ** From Containers
86− , fromFoldable
87−
88− -- * Elimination
89−
90− -- ** Primitives
91− , uncons
92− , drain
93−
94− -- -- ** Folding
95− -- , foldBreak
96−
97− -- ** Parsing
98− , parse
99− , parseBreak
100− , parseBreakChunks
101− , parseChunks
102−
103− -- * Transformation
104− , mapM
105− , dropWhile
106− , take
107−
108− -- * Combining Two Streams
109− -- | Unlike the operations in "Streamly.Data.Stream", these operations can
110− -- be used to dynamically compose large number of streams e.g. using the
111− -- 'concatMapWith' and 'mergeMapWith' operations. They have a linear O(n)
112− -- time complexity wrt to the number of streams being composed.
113−
114− -- ** Appending
115− , append
116−
117− -- ** Interleaving
118− , interleave
119−
120− -- ** Merging
121− , mergeBy
122− , mergeByM
123−
124− -- ** Zipping
125− , zipWith
126− , zipWithM
127−
128− -- ** Cross Product
129− -- XXX is "bind/concatFor" better to have than crossWith?
130− -- crossWith f xs1 xs2 = concatFor xs1 (\x -> fmap (f x) xs2)
131− , crossWith
132− -- , cross
133− -- , joinInner
134− -- , CrossStreamK (..)
135−
136− -- * Stream of streams
137− -- | Some useful idioms:
138− --
139− -- >>> concatFoldableWith f = Prelude.foldr f StreamK.nil
140− -- >>> concatMapFoldableWith f g = Prelude.foldr (f . g) StreamK.nil
141− -- >>> concatForFoldableWith f xs g = Prelude.foldr (f . g) StreamK.nil xs
142− --
143− , concatEffect
144− -- , concatMap
145− , concatMapWith
146− , mergeMapWith
147−
148− -- * Buffered Operations
149− , reverse
150− , sortBy
151−
152− -- * Exceptions
153− -- | Please note that 'Stream' type does not observe any exceptions from
154− -- the consumer of the stream whereas 'StreamK' does.
155− , handle
156−
157− -- * Resource Management
158− -- | Please note that 'Stream' type does not observe any exceptions from
159− -- the consumer of the stream whereas 'StreamK' does.
160− , bracketIO
161− )
162−where
163−
164−import Streamly.Internal.Data.StreamK
165−import Prelude hiding (reverse, zipWith, mapM, dropWhile, take)
166−
167−#include "DocTestDataStreamK.hs"
168−
169−-- $overview
170−--
171−-- Continuation passing style (CPS) stream implementation. The 'K' in 'StreamK'
172−-- stands for Kontinuation.
173−--
174−-- StreamK can be constructed like lists, except that they use 'nil' instead of
175−-- '[]' and 'cons' instead of ':'.
176−--
177−-- `cons` adds a pure value at the head of the stream:
178−--
179−-- >>> import Streamly.Data.StreamK (StreamK, cons, consM, nil)
180−-- >>> stream = 1 `cons` 2 `cons` nil :: StreamK IO Int
181−--
182−-- You can use operations from "Streamly.Data.Stream" for StreamK as well by
183−-- converting StreamK to Stream ('toStream'), and vice-versa ('fromStream').
184−--
185−-- >>> Stream.fold Fold.toList $ StreamK.toStream stream -- IO [Int]
186−-- [1,2]
187−--
188−-- `consM` adds an effect at the head of the stream:
189−--
190−-- >>> stream = effect 1 `consM` effect 2 `consM` nil
191−-- >>> Stream.fold Fold.toList $ StreamK.toStream stream
192−-- 1
193−-- 2
194−-- [1,2]
core/src/Streamly/Data/Unfold.hs+85−98
@@ -1,4 +1,4 @@
1−{-# LANGUAGE CPP #-}
1+#include "inline.hs"
22
33-- |
44-- Module : Streamly.Data.Unfold
@@ -8,104 +8,13 @@
88-- Stability : released
99-- Portability : GHC
1010--
11−-- Fast, composable stream producers with ability to terminate, supporting
12−-- nested stream fusion. Nested stream operations like
13−-- 'Streamly.Data.Stream.concatMap' in the "Streamly.Data.Stream" module do not
14−-- fuse, however, the 'Streamly.Data.Stream.unfoldMany' operation, using the
15−-- 'Unfold' type, is a fully fusible alternative to
16−-- 'Streamly.Data.Stream.concatMap'.
17−--
18−-- Please refer to "Streamly.Internal.Data.Unfold" for more functions that have
19−-- not yet been released.
20−--
21−-- Exception combinators are not exposed, we would like to encourage the use of
22−-- 'Stream' type instead whenever exception handling is required. We can
23−-- consider exposing the unfold exception functions if there is a compelling
24−-- use case to use unfolds instead of stream.
25−
26−module Streamly.Data.Unfold
27− (
28− -- * Setup
29− -- | To execute the code examples provided in this module in ghci, please
30− -- run the following commands first.
31− --
32− -- $setup
33−
34− -- * Overview
35− -- $overview
36−
37− -- * Unfold Type
38− Unfold
39−
40− -- * Unfolds
41− -- One to one correspondence with
42− -- "Streamly.Internal.Data.Stream.Generate"
43−
44− -- ** Basic Constructors
45− , unfoldrM
46− , unfoldr
47− , function
48− , functionM
49−
50− -- ** Generators
51− -- | Generate a monadic stream from a seed.
52− , repeatM
53− , replicateM
54− , iterateM
55−
56− -- ** Enumeration
57− , Enumerable (..)
58−
59− -- ** From Containers
60− , fromList
61− , fromListM
62− , fromStream
63−
64− -- * Combinators
65− -- ** Mapping on Input
66− , lmap
67− , lmapM
68− , first
69− , second
70−
71− -- ** Mapping on Output
72− , mapM
73−
74− -- ** Filtering
75− , takeWhileM
76− , takeWhile
77− , take
78− , filter
79− , filterM
80− , drop
81− , dropWhile
82− , dropWhileM
83−
84− -- ** Zipping
85− , zipWith
86−
87− -- ** Cross Product
88− , crossWith
89−
90− -- ** Nesting
91− , many
92−
93− )
94−where
95−
96−import Prelude hiding
97− ( concat, map, mapM, takeWhile, take, filter, const, drop, dropWhile
98− , zipWith
99− )
100−import Streamly.Internal.Data.Unfold
101−
102−#include "DocTestDataUnfold.hs"
103−
104−-- $overview
105−--
10611-- An 'Unfold' is a source or a producer of a stream of values. It takes a
10712-- seed value as an input and unfolds it into a sequence of values.
10813--
14+-- >>> import qualified Streamly.Data.Fold as Fold
15+-- >>> import qualified Streamly.Data.Unfold as Unfold
16+-- >>> import qualified Streamly.Internal.Data.Stream as Stream
17+--
10918-- For example, the 'fromList' Unfold generates a stream of values from a
11019-- supplied list. Unfolds can be converted to 'Streamly.Internal.Data.Stream'
11120-- using the 'Stream.unfold' operation.
@@ -149,7 +58,7 @@ import Streamly.Internal.Data.Unfold
14958-- Please see "Streamly.Internal.Data.Unfold" for additional @Pre-release@
15059-- functions.
15160--
152−-- == Creating New Unfolds
61+-- = Creating New Unfolds
15362--
15463-- There are many commonly used unfolds provided in this module. However, you
15564-- can always create your own as well. An 'Unfold' is just a data
@@ -186,7 +95,7 @@ import Streamly.Internal.Data.Unfold
18695-- >>> Data.List.unfoldr f [1,2,3]
18796-- [1,2,3]
18897--
189−-- == Unfolds vs. Streams
98+-- = Unfolds vs. Streams
19099--
191100-- The 'Unfold' abstraction for representing streams was introduced in Streamly
192101-- to provide C like performance for nested looping of streams. 'Unfold' and
@@ -231,3 +140,81 @@ import Streamly.Internal.Data.Unfold
231140-- streams are written, it is easy to adapt a stream to an unfold. If you are
232141-- writing an unfold you can convert it to stream for free using
233142-- 'Stream.unfold'.
143+
144+-- What makes streams less efficient is also what makes them more conveient to
145+-- use and powerful. The stream data type (Stream m a) bundles the state along
146+-- with the stream generator function making it opaque, whereas an unfold
147+-- exposes the state (Unfold m s a) to the user. This allows the Unfold to be
148+-- unfolded (inlined) inside a nested loop without having to bundle the state
149+-- and the generator together, the stream state can be saved and passed
150+-- independent of the generator function. On the other hand in a stream type we
151+-- have to bundle the stream state and the generator function together to save
152+-- the stream. This makes it inefficient because it requires boxing and
153+-- constructor allocation. However, this makes streams more convenient as we do
154+-- not need to pass around the state/seed separately.
155+--
156+module Streamly.Data.Unfold
157+ (
158+ -- * Unfold Type
159+ Unfold
160+
161+ -- * Unfolds
162+ -- One to one correspondence with
163+ -- "Streamly.Internal.Data.Stream.Generate"
164+
165+ -- ** Basic Constructors
166+ , unfoldrM
167+ , unfoldr
168+ , function
169+ , functionM
170+
171+ -- ** Generators
172+ -- | Generate a monadic stream from a seed.
173+ , repeatM
174+ , replicateM
175+ , iterateM
176+
177+ -- ** From Containers
178+ , fromList
179+ , fromListM
180+ , fromStream
181+
182+ -- * Combinators
183+ -- ** Mapping on Input
184+ , lmap
185+ , lmapM
186+
187+ -- ** Mapping on Output
188+ , mapM
189+
190+ -- ** Filtering
191+ , takeWhileM
192+ , takeWhile
193+ , take
194+ , filter
195+ , filterM
196+ , drop
197+ , dropWhile
198+ , dropWhileM
199+
200+ -- ** Zipping
201+ , zipWith
202+
203+ -- ** Cross Product
204+ , crossWith
205+
206+ -- ** Nesting
207+ , many
208+
209+ -- Exception combinators are not exposed, we would like to encourage the
210+ -- use of Stream type instead where exception handling is required. We can
211+ -- consider exposing the unfold exception functions if there is a
212+ -- compelling use case to use unfolds instead of stream.
213+ )
214+where
215+
216+import Prelude hiding
217+ ( concat, map, mapM, takeWhile, take, filter, const, drop, dropWhile
218+ , zipWith
219+ )
220+import Streamly.Internal.Data.Unfold
core/src/Streamly/FileSystem/Handle.hs+29−15
@@ -1,4 +1,4 @@
1−{-# LANGUAGE CPP #-}
1+#include "inline.hs"
22
33-- |
44-- Module : Streamly.FileSystem.Handle
@@ -40,12 +40,6 @@
4040--
4141module Streamly.FileSystem.Handle
4242 (
43− -- * Setup
44− -- | To execute the code examples provided in this module in ghci, please
45− -- run the following commands first.
46− --
47− -- $setup
48−
4943 -- * Singleton IO
5044 -- | Read or write a single buffer.
5145 getChunk
@@ -81,13 +75,13 @@ module Streamly.FileSystem.Handle
8175 -- position of the file handle. The stream ends as soon as EOF is
8276 -- encountered.
8377
84− -- *** Streams
85− , read
86− , readWith
87− , readChunks
88− , readChunksWith
78+ -- -- *** Streams
79+ -- , read
80+ -- , readWith
81+ -- , readChunks
82+ -- , readChunksWith
8983
90− -- *** Unfolds
84+ -- -- *** Unfolds
9185 , reader
9286 , readerWith
9387 , chunkReader
@@ -104,14 +98,34 @@ module Streamly.FileSystem.Handle
10498 , writeChunks
10599
106100 -- * Deprecated
101+ , read
107102 , readWithBufferOf
103+ , readChunks
108104 , readChunksWithBufferOf
109105 , writeChunksWithBufferOf
110106 , writeWithBufferOf
111107 )
112108where
113109
114−import Streamly.Internal.FileSystem.Handle
110+import Control.Monad.IO.Class (MonadIO(..))
111+import Data.Word (Word8)
112+import Streamly.Internal.Data.Array.Type (Array)
113+import Streamly.Internal.Data.Unfold.Type (Unfold)
114+import System.IO (Handle)
115+
116+import Streamly.Internal.FileSystem.Handle hiding (read, readChunks)
115117import Prelude hiding (read)
116118
117−#include "DocTestFileSystemHandle.hs"
119+-- | Same as 'reader'
120+--
121+{-# DEPRECATED read "Please use 'reader' instead" #-}
122+{-# INLINE read #-}
123+read :: MonadIO m => Unfold m Handle Word8
124+read = reader
125+
126+-- | Same as 'chunkReader'
127+--
128+{-# DEPRECATED readChunks "Please use 'chunkReader' instead" #-}
129+{-# INLINE readChunks #-}
130+readChunks :: MonadIO m => Unfold m Handle (Array Word8)
131+readChunks = chunkReader
core/src/Streamly/Internal/Console/Stdio.hs+2−2
@@ -47,12 +47,12 @@ import System.IO (stdin, stdout, stderr)
4747import Prelude hiding (read)
4848
4949import Streamly.Internal.Data.Array.Type (Array(..))
50−import Streamly.Internal.Data.Stream (Stream)
50+import Streamly.Internal.Data.Stream.StreamD (Stream)
5151import Streamly.Internal.Data.Unfold (Unfold)
5252import Streamly.Internal.Data.Fold (Fold)
5353
5454import qualified Streamly.Internal.Data.Array as Array
55−import qualified Streamly.Internal.Data.Stream as Stream
55+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
5656 (intersperseMSuffix)
5757import qualified Streamly.Internal.Data.Unfold as Unfold
5858import qualified Streamly.Internal.FileSystem.Handle as Handle
core/src/Streamly/Internal/Data/Array.hs+187−644
@@ -1,48 +1,88 @@
1−{-# LANGUAGE CPP #-}
21-- |
32-- Module : Streamly.Internal.Data.Array
43-- Copyright : (c) 2019 Composewell Technologies
4+--
55-- License : BSD3
66-- Maintainer : streamly@composewell.com
77-- Stability : experimental
88-- Portability : GHC
99--
10+-- To summarize:
11+--
12+-- * Arrays are finite and fixed in size
13+-- * provide /O(1)/ access to elements
14+-- * store only data and not functions
15+-- * provide efficient IO interfacing
16+--
17+-- 'Foldable' instance is not provided because the implementation would be much
18+-- less efficient compared to folding via streams. 'Semigroup' and 'Monoid'
19+-- instances should be used with care; concatenating arrays using binary
20+-- operations can be highly inefficient. Instead, use
21+-- 'Streamly.Internal.Data.Stream.Chunked.toArray' to concatenate N
22+-- arrays at once.
23+--
24+-- Each array is one pointer visible to the GC. Too many small arrays (e.g.
25+-- single byte) are only as good as holding those elements in a Haskell list.
26+-- However, small arrays can be compacted into large ones to reduce the
27+-- overhead. To hold 32GB memory in 32k sized buffers we need 1 million arrays
28+-- if we use one array for each chunk. This is still significant to add
29+-- pressure to GC.
30+
1031module Streamly.Internal.Data.Array
1132 (
12− -- * Setup
13− -- $setup
33+ Array
1434
15− -- * Design Notes
16− -- $design
35+ -- * Construction
1736
18− -- * The Array Type
19− module Streamly.Internal.Data.Array.Type
37+ -- Pure List APIs
38+ , A.fromListN
39+ , A.fromList
40+
41+ -- Stream Folds
42+ , fromStreamN
43+ , fromStream
2044
21− -- * Construction
2245 -- Monadic Folds
46+ , A.writeN -- drop new
47+ , A.writeNAligned
48+ , A.write -- full buffer
2349 , writeLastN
2450
51+ -- * Elimination
52+ -- ** Conversion
53+ , A.toList
54+
55+ -- ** Streams
56+ , A.read
57+ , A.readRev
58+
59+ -- ** Unfolds
60+ , reader
61+ , readerUnsafe
62+ , A.readerRev
63+ , producer -- experimental
64+
2565 -- * Random Access
2666 -- , (!!)
2767 , getIndex
68+ , A.unsafeIndex -- XXX Rename to getIndexUnsafe??
2869 , getIndexRev
29− , last -- XXX getLastIndex?
70+ , last -- XXX getIndexLast?
71+ , getIndices
72+ , getIndicesFromThenTo
3073 -- , getIndicesFrom -- read from a given position to the end of file
3174 -- , getIndicesUpto -- read from beginning up to the given position
3275 -- , getIndicesFromTo
3376 -- , getIndicesFromRev -- read from a given position to the beginning of file
3477 -- , getIndicesUptoRev -- read from end to the given position in file
35− , indexReader
36− , indexReaderFromThenTo
3778
3879 -- * Size
80+ , length
3981 , null
4082
4183 -- * Search
4284 , binarySearch
4385 , findIndicesOf
44− -- getIndicesOf
45− , indexFinder -- see splitOn
4686 -- , findIndexOf
4787 -- , find
4888
@@ -50,134 +90,156 @@ module Streamly.Internal.Data.Array
5090 , cast
5191 , asBytes
5292 , castUnsafe
93+ , asPtrUnsafe
5394 , asCStringUnsafe
95+ , A.unsafeFreeze -- asImmutableUnsafe?
96+ , A.unsafeThaw -- asMutableUnsafe?
5497
5598 -- * Subarrays
5699 , getSliceUnsafe
57100 -- , getSlice
58− , sliceIndexerFromLen
59− , slicerFromLen
101+ , genSlicesFromLen
102+ , getSlicesFromLen
60103 , splitOn
61104
62105 -- * Streaming Operations
63106 , streamTransform
64107
65− -- * Folding
108+ -- ** Folding
66109 , streamFold
67110 , fold
68111
69− -- * Stream of Arrays
70−
71− -- XXX these are probably not very useful to have in this module as we can
72− -- express these idiomatically using streams.
73− , interpose
74− , interposeSuffix
75− , intercalateSuffix
76−
77− , compactLE
78− , pinnedCompactLE
79− , compactOnByte
80− , compactOnByteSuffix
81−
82− , foldBreakChunks
83− , foldChunks
84− , foldBreakChunksK
85− , parseBreakChunksK
86−
87− -- * Serialization
88− , encodeAs
89− , serialize
90− , pinnedSerialize
91− , deserialize
92−
93112 -- * Deprecated
94− , genSlicesFromLen
95− , getSlicesFromLen
96− , getIndices
113+ , A.toStream
114+ , A.toStreamRev
97115 )
98116where
99117
100−#include "assert.hs"
101118#include "inline.hs"
102119#include "ArrayMacros.h"
103120
121+import Control.Exception (assert)
122+import Control.Monad (when)
104123import Control.Monad.IO.Class (MonadIO(..))
105−-- import Data.Bifunctor (first)
106−-- import Data.Either (fromRight)
107124import Data.Functor.Identity (Identity)
108125import Data.Proxy (Proxy(..))
109126import Data.Word (Word8)
110127import Foreign.C.String (CString)
111128import Foreign.Ptr (castPtr)
112129import Foreign.Storable (Storable)
113−import GHC.Types (SPEC(..))
114−import Streamly.Internal.Data.Unbox (Unbox(..))
130+import Streamly.Internal.Data.Unboxed
131+ ( Unbox
132+ , peekWith
133+ , sizeOf
134+ )
115135import Prelude hiding (length, null, last, map, (!!), read, concat)
116136
117−import Streamly.Internal.Data.MutByteArray.Type (PinnedState(..), MutByteArray)
118−import Streamly.Internal.Data.Serialize.Type (Serialize)
137+import Streamly.Internal.Data.Array.Mut.Type (ArrayUnsafe(..))
138+import Streamly.Internal.Data.Array.Type
139+ (Array(..), length, asPtrUnsafe)
119140import Streamly.Internal.Data.Fold.Type (Fold(..))
120−import Streamly.Internal.Data.Parser (Parser(..), Initial(..), ParseError(..))
121−import Streamly.Internal.Data.Stream (Stream(..))
122−import Streamly.Internal.Data.StreamK (StreamK)
123−import Streamly.Internal.Data.SVar.Type (adaptState, defState)
124−import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3Fused'(..))
141+import Streamly.Internal.Data.Producer.Type (Producer(..))
142+import Streamly.Internal.Data.Stream.StreamD (Stream)
143+import Streamly.Internal.Data.Tuple.Strict (Tuple3Fused'(..))
125144import Streamly.Internal.Data.Unfold.Type (Unfold(..))
126145import Streamly.Internal.System.IO (unsafeInlineIO)
127146
128−import qualified Streamly.Internal.Data.Fold.Type as Fold
129−import qualified Streamly.Internal.Data.Serialize.Type as Serialize
130−import qualified Streamly.Internal.Data.MutByteArray.Type as MBA
131−import qualified Streamly.Internal.Data.MutArray as MA
147+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
148+import qualified Streamly.Internal.Data.Array.Mut as MA
149+import qualified Streamly.Internal.Data.Array.Type as A
132150import qualified Streamly.Internal.Data.Fold as FL
133−import qualified Streamly.Internal.Data.Ring as RB
134−import qualified Streamly.Internal.Data.Parser as Parser
135−-- import qualified Streamly.Internal.Data.ParserK as ParserK
136−import qualified Streamly.Internal.Data.Stream as D
137−import qualified Streamly.Internal.Data.Stream as Stream
138−import qualified Streamly.Internal.Data.StreamK as StreamK
151+import qualified Streamly.Internal.Data.Producer.Type as Producer
152+import qualified Streamly.Internal.Data.Producer as Producer
153+import qualified Streamly.Internal.Data.Ring.Unboxed as RB
154+import qualified Streamly.Internal.Data.Stream.StreamD as D
155+import qualified Streamly.Internal.Data.Stream.StreamD as Stream
139156import qualified Streamly.Internal.Data.Unfold as Unfold
140−import qualified Prelude
141−
142−import Streamly.Internal.Data.Array.Type
143157
144−#include "DocTestDataArray.hs"
158+-------------------------------------------------------------------------------
159+-- Construction
160+-------------------------------------------------------------------------------
145161
146−-- $design
147−--
148−-- To summarize:
162+-- | Create an 'Array' from the first N elements of a stream. The array is
163+-- allocated to size N, if the stream terminates before N elements then the
164+-- array may hold less than N elements.
149165--
150−-- * Arrays are finite and fixed in size
151−-- * provide /O(1)/ access to elements
152−-- * store only data and not functions
153−-- * provide efficient IO interfacing
166+-- /Pre-release/
167+{-# INLINE fromStreamN #-}
168+fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
169+fromStreamN n m = do
170+ when (n < 0) $ error "writeN: negative write count specified"
171+ A.fromStreamDN n m
172+
173+-- | Create an 'Array' from a stream. This is useful when we want to create a
174+-- single array from a stream of unknown size. 'writeN' is at least twice
175+-- as efficient when the size is already known.
154176--
155−-- 'Foldable' instance is not provided because the implementation would be much
156−-- less efficient compared to folding via streams. 'Semigroup' and 'Monoid'
157−-- instances should be used with care; concatenating arrays using binary
158−-- operations can be highly inefficient. Instead, use
159−-- 'Streamly.Internal.Data.Stream.Chunked.toArray' to concatenate N
160−-- arrays at once.
177+-- Note that if the input stream is too large memory allocation for the array
178+-- may fail. When the stream size is not known, `arraysOf` followed by
179+-- processing of indvidual arrays in the resulting stream should be preferred.
161180--
162−-- Each array is one pointer visible to the GC. Too many small arrays (e.g.
163−-- single byte) are only as good as holding those elements in a Haskell list.
164−-- However, small arrays can be compacted into large ones to reduce the
165−-- overhead. To hold 32GB memory in 32k sized buffers we need 1 million arrays
166−-- if we use one array for each chunk. This is still significant to add
167−-- pressure to GC.
181+-- /Pre-release/
182+{-# INLINE fromStream #-}
183+fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
184+fromStream = Stream.fold A.write
185+-- write m = A.fromStreamD $ D.fromStreamK m
168186
169187-------------------------------------------------------------------------------
170188-- Elimination
171189-------------------------------------------------------------------------------
172190
191+{-# INLINE_NORMAL producer #-}
192+producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a
193+producer =
194+ Producer.translate A.unsafeThaw A.unsafeFreeze
195+ $ MA.producerWith (return . unsafeInlineIO)
196+
197+-- | Unfold an array into a stream.
198+--
199+{-# INLINE_NORMAL reader #-}
200+reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
201+reader = Producer.simplify producer
202+
203+-- | Unfold an array into a stream, does not check the end of the array, the
204+-- user is responsible for terminating the stream within the array bounds. For
205+-- high performance application where the end condition can be determined by
206+-- a terminating fold.
207+--
208+-- Written in the hope that it may be faster than "read", however, in the case
209+-- for which this was written, "read" proves to be faster even though the core
210+-- generated with unsafeRead looks simpler.
211+--
212+-- /Pre-release/
213+--
214+{-# INLINE_NORMAL readerUnsafe #-}
215+readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
216+readerUnsafe = Unfold step inject
217+ where
218+
219+ inject (Array contents start end) =
220+ return (ArrayUnsafe contents end start)
221+
222+ {-# INLINE_LATE step #-}
223+ step (ArrayUnsafe contents end p) = do
224+ -- unsafeInlineIO allows us to run this in Identity monad for pure
225+ -- toList/foldr case which makes them much faster due to not
226+ -- accumulating the list and fusing better with the pure consumers.
227+ --
228+ -- This should be safe as the array contents are guaranteed to be
229+ -- evaluated/written to before we peek at them.
230+ let !x = unsafeInlineIO $ peekWith contents p
231+ let !p1 = INDEX_NEXT(p,a)
232+ return $ D.Yield x (ArrayUnsafe contents end p1)
233+
173234-- |
174235--
236+-- >>> import qualified Streamly.Internal.Data.Array.Type as Array
175237-- >>> null arr = Array.byteLength arr == 0
176238--
177239-- /Pre-release/
178240{-# INLINE null #-}
179241null :: Array a -> Bool
180−null arr = byteLength arr == 0
242+null arr = A.byteLength arr == 0
181243
182244-- | Like 'getIndex' but indexes the array in reverse from the end.
183245--
@@ -189,11 +251,12 @@ getIndexRev i arr =
189251 $ do
190252 let elemPtr = RINDEX_OF(arrEnd arr, i, a)
191253 if i >= 0 && elemPtr >= arrStart arr
192− then Just <$> peekAt elemPtr (arrContents arr)
254+ then Just <$> peekWith (arrContents arr) elemPtr
193255 else return Nothing
194256
195257-- |
196258--
259+-- >>> import qualified Streamly.Internal.Data.Array as Array
197260-- >>> last arr = Array.getIndexRev arr 0
198261--
199262-- /Pre-release/
@@ -205,8 +268,6 @@ last = getIndexRev 0
205268-- Folds with Array as the container
206269-------------------------------------------------------------------------------
207270
208−-- XXX We should generate this from Ring.
209−
210271-- | @writeLastN n@ folds a maximum of @n@ elements from the end of the input
211272-- stream to an 'Array'.
212273--
@@ -215,7 +276,7 @@ writeLastN ::
215276 (Storable a, Unbox a, MonadIO m) => Int -> Fold m a (Array a)
216277writeLastN n
217278 | n <= 0 = fmap (const mempty) FL.drain
218− | otherwise = unsafeFreeze <$> Fold step initial done done
279+ | otherwise = A.unsafeFreeze <$> Fold step initial done
219280
220281 where
221282
@@ -228,9 +289,11 @@ writeLastN n
228289 in fmap f $ liftIO $ RB.new n
229290
230291 done (Tuple3Fused' rb rh i) = do
231− arr <- MA.new n
232− -- XXX We should write a read unfold for ring.
233− foldFunc i rh MA.snocUnsafe arr rb
292+ arr <- liftIO $ MA.newPinned n
293+ foldFunc i rh snoc' arr rb
294+
295+ -- XXX We should write a read unfold for ring.
296+ snoc' b a = liftIO $ MA.snocUnsafe b a
234297
235298 foldFunc i
236299 | i < n = RB.unsafeFoldRingM
@@ -256,26 +319,19 @@ binarySearch = undefined
256319-- compared to streams by using SIMD instructions.
257320-- We can also return a bit array instead.
258321
259−-- Can use SIMD.
260−
261322-- | Perform a linear search to find all the indices where a given element is
262323-- present in an array.
263324--
264325-- /Unimplemented/
265−indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int
266−indexFinder = undefined
267−
268−-- |
269−-- /Unimplemented/
270−findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int
271−findIndicesOf p = Stream.unfold (indexFinder p)
326+findIndicesOf :: (a -> Bool) -> Unfold Identity (Array a) Int
327+findIndicesOf = undefined
272328
273329{-
274330findIndexOf :: (a -> Bool) -> Array a -> Maybe Int
275−findIndexOf p = Unfold.fold Fold.one . Stream.unfold (indexFinder p)
331+findIndexOf p = Unfold.fold Fold.one . Stream.unfold (findIndicesOf p)
276332
277333find :: (a -> Bool) -> Array a -> Bool
278−find = Unfold.fold Fold.null . Stream.unfold (indexFinder p)
334+find = Unfold.fold Fold.null . Stream.unfold (findIndicesOf p)
279335-}
280336
281337-------------------------------------------------------------------------------
@@ -317,45 +373,29 @@ splitOn :: (Monad m, Unbox a) =>
317373 (a -> Bool) -> Array a -> Stream m (Array a)
318374splitOn predicate arr =
319375 fmap (\(i, len) -> getSliceUnsafe i len arr)
320− $ D.indexOnSuffix predicate (read arr)
376+ $ D.sliceOnSuffix predicate (A.toStreamD arr)
321377
322−{-# INLINE sliceIndexerFromLen #-}
323−sliceIndexerFromLen :: forall m a. (Monad m, Unbox a)
324− => Int -- ^ from index
325− -> Int -- ^ length of the slice
326− -> Unfold m (Array a) (Int, Int)
327−sliceIndexerFromLen from len =
328− Unfold.lmap unsafeThaw (MA.sliceIndexerFromLen from len)
329−
330−{-# DEPRECATED genSlicesFromLen "Please use sliceIndexerFromLen instead." #-}
331378{-# INLINE genSlicesFromLen #-}
332379genSlicesFromLen :: forall m a. (Monad m, Unbox a)
333380 => Int -- ^ from index
334381 -> Int -- ^ length of the slice
335382 -> Unfold m (Array a) (Int, Int)
336−genSlicesFromLen = sliceIndexerFromLen
383+genSlicesFromLen from len =
384+ Unfold.lmap A.unsafeThaw (MA.genSlicesFromLen from len)
337385
338386-- | Generate a stream of slices of specified length from an array, starting
339387-- from the supplied array index. The last slice may be shorter than the
340388-- requested length.
341389--
342390-- /Pre-release//
343−{-# INLINE slicerFromLen #-}
344−slicerFromLen :: forall m a. (Monad m, Unbox a)
345− => Int -- ^ from index
346− -> Int -- ^ length of the slice
347− -> Unfold m (Array a) (Array a)
348−slicerFromLen from len =
349− fmap unsafeFreeze
350− $ Unfold.lmap unsafeThaw (MA.slicerFromLen from len)
351−
352−{-# DEPRECATED getSlicesFromLen "Please use slicerFromLen instead." #-}
353391{-# INLINE getSlicesFromLen #-}
354392getSlicesFromLen :: forall m a. (Monad m, Unbox a)
355393 => Int -- ^ from index
356394 -> Int -- ^ length of the slice
357395 -> Unfold m (Array a) (Array a)
358−getSlicesFromLen = slicerFromLen
396+getSlicesFromLen from len =
397+ fmap A.unsafeFreeze
398+ $ Unfold.lmap A.unsafeThaw (MA.getSlicesFromLen from len)
359399
360400-------------------------------------------------------------------------------
361401-- Random reads and writes
@@ -373,7 +413,7 @@ getIndex i arr =
373413 $ do
374414 let elemPtr = INDEX_OF(arrStart arr, i, a)
375415 if i >= 0 && INDEX_VALID(elemPtr, arrEnd arr, a)
376− then Just <$> peekAt elemPtr (arrContents arr)
416+ then Just <$> peekWith (arrContents arr) elemPtr
377417 else return Nothing
378418
379419-- | Given a stream of array indices, read the elements on those indices from
@@ -385,25 +425,19 @@ getIndex i arr =
385425-- @
386426-- read =
387427-- let u = lmap (\arr -> (0, length arr - 1)) Unfold.enumerateFromTo
388−-- in Unfold.lmap f (indexReader arr)
428+-- in Unfold.lmap f (getIndices arr)
389429--
390430-- readRev =
391431-- let i = length arr - 1
392−-- in Unfold.lmap f (indexReaderFromThenTo i (i - 1) 0)
432+-- in Unfold.lmap f (getIndicesFromThenTo i (i - 1) 0)
393433-- @
394434--
395435-- /Pre-release/
396−{-# INLINE indexReader #-}
397−indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
398−indexReader m =
399− let unf = MA.indexReaderWith (return . unsafeInlineIO) m
400− in Unfold.lmap unsafeThaw unf
401−
402−-- XXX DO NOT REMOVE, change the signature to use Stream instead of unfold
403−{-# DEPRECATED getIndices "Please use getIndices instead." #-}
404436{-# INLINE getIndices #-}
405437getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a
406−getIndices = indexReader
438+getIndices m =
439+ let unf = MA.getIndicesD (return . unsafeInlineIO) m
440+ in Unfold.lmap A.unsafeThaw unf
407441
408442-- | Unfolds @(from, then, to, array)@ generating a finite stream whose first
409443-- element is the array value from the index @from@ and the successive elements
@@ -419,9 +453,9 @@ getIndices = indexReader
419453-- @
420454--
421455-- /Unimplemented/
422−{-# INLINE indexReaderFromThenTo #-}
423−indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a
424−indexReaderFromThenTo = undefined
456+{-# INLINE getIndicesFromThenTo #-}
457+getIndicesFromThenTo :: Unfold m (Int, Int, Int, Array a) a
458+getIndicesFromThenTo = undefined
425459
426460-------------------------------------------------------------------------------
427461-- Transform via stream operations
@@ -438,7 +472,7 @@ indexReaderFromThenTo = undefined
438472{-# INLINE runPipe #-}
439473runPipe :: (MonadIO m, Unbox a, Unbox b)
440474 => Pipe m a b -> Array a -> m (Array b)
441−runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (read arr)
475+runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (A.read arr)
442476-}
443477
444478-- XXX For transformations that cannot change the number of elements e.g. "map"
@@ -452,7 +486,7 @@ runPipe f arr = P.runPipe (toArrayMinChunk (length arr)) $ f (read arr)
452486streamTransform :: forall m a b. (MonadIO m, Unbox a, Unbox b)
453487 => (Stream m a -> Stream m b) -> Array a -> m (Array b)
454488streamTransform f arr =
455− Stream.fold (createWith (length arr)) $ f (read arr)
489+ Stream.fold (A.writeWith (length arr)) $ f (A.read arr)
456490
457491-------------------------------------------------------------------------------
458492-- Casts
@@ -486,15 +520,13 @@ asBytes = castUnsafe
486520--
487521cast :: forall a b. (Unbox b) => Array a -> Maybe (Array b)
488522cast arr =
489− let len = byteLength arr
523+ let len = A.byteLength arr
490524 r = len `mod` SIZE_OF(b)
491525 in if r /= 0
492526 then Nothing
493527 else Just $ castUnsafe arr
494528
495−-- | Convert an array of any type into a null terminated CString Ptr. If the
496−-- array is unpinned it is first converted to a pinned array which requires a
497−-- copy.
529+-- | Convert an array of any type into a null terminated CString Ptr.
498530--
499531-- /Unsafe/
500532--
@@ -504,9 +536,9 @@ cast arr =
504536--
505537asCStringUnsafe :: Array a -> (CString -> IO b) -> IO b
506538asCStringUnsafe arr act = do
507− let arr1 = asBytes arr <> fromList [0]
508− -- unsafePinnedAsPtr makes sure the array is pinned
509− unsafePinnedAsPtr arr1 $ \ptr -> act (castPtr ptr)
539+ -- XXX Ensure a pinned allocation here.
540+ let arr1 = asBytes arr <> A.fromList [0]
541+ asPtrUnsafe arr1 $ \ptr -> act (castPtr ptr)
510542
511543-------------------------------------------------------------------------------
512544-- Folds
@@ -519,500 +551,11 @@ asCStringUnsafe arr act = do
519551-- /Pre-release/
520552{-# INLINE fold #-}
521553fold :: forall m a b. (Monad m, Unbox a) => Fold m a b -> Array a -> m b
522−fold f arr = Stream.fold f (read arr)
554+fold f arr = Stream.fold f (A.read arr)
523555
524556-- | Fold an array using a stream fold operation.
525557--
526558-- /Pre-release/
527559{-# INLINE streamFold #-}
528560streamFold :: (Monad m, Unbox a) => (Stream m a -> m b) -> Array a -> m b
529−streamFold f arr = f (read arr)
530−
531−--------------------------------------------------------------------------------
532−-- Serialization
533−--------------------------------------------------------------------------------
534−
535−{-# INLINE encodeAs #-}
536−encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8
537−encodeAs ps a =
538− unsafeInlineIO $ do
539− let len = Serialize.addSizeTo 0 a
540− mbarr <- MBA.newBytesAs ps len
541− off <- Serialize.serializeAt 0 mbarr a
542− assertM(len == off)
543− pure $ Array mbarr 0 off
544−
545−-- |
546−-- Properties:
547−-- 1. Identity: @deserialize . serialize == id@
548−-- 2. Encoded equivalence: @serialize a == serialize a@
549−{-# INLINE serialize #-}
550−serialize :: Serialize a => a -> Array Word8
551−serialize = encodeAs Unpinned
552−
553−-- | Serialize a Haskell type to a pinned byte array. The array is allocated
554−-- using pinned memory so that it can be used directly in OS APIs for writing
555−-- to file or sending over the network.
556−--
557−-- Properties:
558−-- 1. Identity: @deserialize . pinnedSerialize == id@
559−-- 2. Encoded equivalence: @pinnedSerialize a == pinnedSerialize a@
560−{-# INLINE pinnedSerialize #-}
561−pinnedSerialize :: Serialize a => a -> Array Word8
562−pinnedSerialize = encodeAs Pinned
563−
564−-- | Decode a Haskell type from a byte array containing its serialized
565−-- representation.
566−{-# INLINE deserialize #-}
567−deserialize :: Serialize a => Array Word8 -> a
568−deserialize arr@(Array {..}) = unsafeInlineIO $ do
569− let lenArr = length arr
570− (off, val) <-
571− Serialize.deserializeAt arrStart arrContents (arrStart + lenArr)
572− assertM(off == arrStart + lenArr)
573− pure val
574−
575−-------------------------------------------------------------------------------
576−-- Streams of Arrays
577−-------------------------------------------------------------------------------
578−
579−-- TODO: efficiently compare two streams of arrays. Two streams can have chunks
580−-- of different sizes, we can handle that in the stream comparison abstraction.
581−-- This could be useful e.g. to fast compare whether two files differ.
582−
583−-- | Insert the given element between arrays and flatten.
584−--
585−-- >>> interpose x = Stream.interpose x Array.reader
586−--
587−{-# INLINE interpose #-}
588−interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a
589−interpose x = D.interpose x reader
590−
591−data FlattenState s =
592− OuterLoop s
593− | InnerLoop s !MutByteArray !Int !Int
594−
595−-- | Insert the given element after each array and flatten. This is similar to
596−-- unlines.
597−--
598−-- >>> interposeSuffix x = Stream.interposeSuffix x Array.reader
599−--
600−{-# INLINE_NORMAL interposeSuffix #-}
601−interposeSuffix :: forall m a. (Monad m, Unbox a)
602− => a -> Stream m (Array a) -> Stream m a
603−-- This does not require MonadIO constraint.
604−-- interposeSuffix x = D.interposeSuffix x reader
605−interposeSuffix sep (D.Stream step state) = D.Stream step' (OuterLoop state)
606−
607− where
608−
609− {-# INLINE_LATE step' #-}
610− step' gst (OuterLoop st) = do
611− r <- step (adaptState gst) st
612− return $ case r of
613− D.Yield Array{..} s ->
614− D.Skip (InnerLoop s arrContents arrStart arrEnd)
615− D.Skip s -> D.Skip (OuterLoop s)
616− D.Stop -> D.Stop
617−
618− step' _ (InnerLoop st _ p end) | p == end =
619− return $ D.Yield sep $ OuterLoop st
620−
621− step' _ (InnerLoop st contents p end) = do
622− x <- pure $ unsafeInlineIO $ peekAt p contents
623− return $ D.Yield x (InnerLoop st contents (INDEX_NEXT(p,a)) end)
624−
625−-- | Insert the given array after each array and flatten.
626−--
627−-- >>> intercalateSuffix = Stream.intercalateSuffix Array.reader
628−--
629−{-# INLINE intercalateSuffix #-}
630−intercalateSuffix :: (Monad m, Unbox a)
631− => Array a -> Stream m (Array a) -> Stream m a
632−intercalateSuffix = D.intercalateSuffix reader
633−
634−-- | @compactLE n@ coalesces adjacent arrays in the input stream
635−-- only if the combined size would be less than or equal to n.
636−--
637−-- Generates unpinned arrays irrespective of the pinning status of input
638−-- arrays.
639−{-# INLINE_NORMAL compactLE #-}
640−compactLE :: (MonadIO m, Unbox a)
641− => Int -> Stream m (Array a) -> Stream m (Array a)
642−compactLE n stream =
643− D.map unsafeFreeze $ MA.compactLE n $ D.map unsafeThaw stream
644−
645−-- | Pinned version of 'compactLE'.
646−{-# INLINE_NORMAL pinnedCompactLE #-}
647−pinnedCompactLE :: (MonadIO m, Unbox a)
648− => Int -> Stream m (Array a) -> Stream m (Array a)
649−pinnedCompactLE n stream =
650− D.map unsafeFreeze $ MA.pinnedCompactLE n $ D.map unsafeThaw stream
651−
652−-- | Split a stream of arrays on a given separator byte, dropping the separator
653−-- and coalescing all the arrays between two separators into a single array.
654−--
655−{-# INLINE compactOnByte #-}
656−compactOnByte
657− :: (MonadIO m)
658− => Word8
659− -> Stream m (Array Word8)
660− -> Stream m (Array Word8)
661−compactOnByte byte =
662− fmap unsafeFreeze . MA.compactOnByte byte . fmap unsafeThaw
663−
664−-- | Like 'compactOnByte' considers the separator in suffix position instead of
665−-- infix position.
666−{-# INLINE compactOnByteSuffix #-}
667−compactOnByteSuffix
668− :: (MonadIO m)
669− => Word8
670− -> Stream m (Array Word8)
671− -> Stream m (Array Word8)
672−compactOnByteSuffix byte =
673− fmap unsafeFreeze . MA.compactOnByteSuffix byte . fmap unsafeThaw
674−
675−-------------------------------------------------------------------------------
676−-- Folding Streams of Arrays
677−-------------------------------------------------------------------------------
678−
679−-- XXX This should not be used for breaking a stream as the D.cons used in
680−-- reconstructing the stream could be very bad for performance. This can only
681−-- be useful in folding without breaking.
682−{-# INLINE_NORMAL foldBreakChunks #-}
683−foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) =>
684− Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))
685−foldBreakChunks (Fold fstep initial _ final) stream@(Stream step state) = do
686− res <- initial
687− case res of
688− Fold.Partial fs -> go SPEC state fs
689− Fold.Done fb -> return $! (fb, stream)
690−
691− where
692−
693− {-# INLINE go #-}
694− go !_ st !fs = do
695− r <- step defState st
696− case r of
697− Stream.Yield (Array contents start end) s ->
698− let fp = Tuple' end contents
699− in goArray SPEC s fp start fs
700− Stream.Skip s -> go SPEC s fs
701− Stream.Stop -> do
702− b <- final fs
703− return (b, D.nil)
704−
705− goArray !_ s (Tuple' end _) !cur !fs
706− | cur == end = do
707− go SPEC s fs
708− goArray !_ st fp@(Tuple' end contents) !cur !fs = do
709− x <- liftIO $ peekAt cur contents
710− res <- fstep fs x
711− let next = INDEX_NEXT(cur,a)
712− case res of
713− Fold.Done b -> do
714− let arr = Array contents next end
715− return $! (b, D.cons arr (D.Stream step st))
716− Fold.Partial fs1 -> goArray SPEC st fp next fs1
717−
718−-- This may be more robust wrt fusion compared to unfoldMany?
719−
720−-- | Fold a stream of arrays using a 'Fold'. This is equivalent to the
721−-- following:
722−--
723−-- >>> foldChunks f = Stream.fold f . Stream.unfoldMany Array.reader
724−--
725−foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b
726−foldChunks f s = fmap fst (foldBreakChunks f s)
727−-- foldStream f = Stream.fold f . Stream.unfoldMany reader
728−
729−-- | Fold a stream of arrays using a 'Fold' and return the remaining stream.
730−--
731−-- The following alternative to this function allows composing the fold using
732−-- the parser Monad:
733−--
734−-- @
735−-- foldBreakStreamK f s =
736−-- fmap (first (fromRight undefined))
737−-- $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s
738−-- @
739−--
740−-- We can compare perf and remove this one or define it in terms of that.
741−--
742−foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) =>
743− Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))
744−{-
745−foldBreakChunksK f s =
746− fmap (first (fromRight undefined))
747− $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s
748−-}
749−foldBreakChunksK (Fold fstep initial _ final) stream = do
750− res <- initial
751− case res of
752− Fold.Partial fs -> go fs stream
753− Fold.Done fb -> return (fb, stream)
754−
755− where
756−
757− {-# INLINE go #-}
758− go !fs st = do
759− let stop = (, StreamK.nil) <$> final fs
760− single a = yieldk a StreamK.nil
761− yieldk (Array contents start end) r =
762− let fp = Tuple' end contents
763− in goArray fs r fp start
764− in StreamK.foldStream defState yieldk single stop st
765−
766− goArray !fs st (Tuple' end _) !cur
767− | cur == end = do
768− go fs st
769− goArray !fs st fp@(Tuple' end contents) !cur = do
770− x <- liftIO $ peekAt cur contents
771− res <- fstep fs x
772− let next = INDEX_NEXT(cur,a)
773− case res of
774− Fold.Done b -> do
775− let arr = Array contents next end
776− return $! (b, StreamK.cons arr st)
777− Fold.Partial fs1 -> goArray fs1 st fp next
778−
779−{-
780−-- This can be generalized to any type provided it can be unfolded to a stream
781−-- and it can be combined using a semigroup operation.
782−--
783−{-# INLINE_NORMAL parseBreakD #-}
784−parseBreakD ::
785− forall m a b. (MonadIO m, MonadThrow m, Unbox a)
786− => PRD.Parser a m b
787− -> D.Stream m (Array.Array a)
788− -> m (b, D.Stream m (Array.Array a))
789−parseBreakD
790− (PRD.Parser pstep initial extract) stream@(D.Stream step state) = do
791−
792− res <- initial
793− case res of
794− PRD.IPartial s -> go SPEC state (List []) s
795− PRD.IDone b -> return (b, stream)
796− PRD.IError err -> throwM $ ParseError err
797−
798− where
799−
800− -- "backBuf" contains last few items in the stream that we may have to
801− -- backtrack to.
802− --
803− -- XXX currently we are using a dumb list based approach for backtracking
804− -- buffer. This can be replaced by a sliding/ring buffer using Data.Array.
805− -- That will allow us more efficient random back and forth movement.
806− go !_ st backBuf !pst = do
807− r <- step defState st
808− case r of
809− D.Yield (Array contents start end) s ->
810− gobuf SPEC s backBuf
811− (Tuple' end contents) start pst
812− D.Skip s -> go SPEC s backBuf pst
813− D.Stop -> do
814− b <- extract pst
815− return (b, D.nil)
816−
817− -- Use strictness on "cur" to keep it unboxed
818− gobuf !_ s backBuf (Tuple' end _) !cur !pst
819− | cur == end = do
820− go SPEC s backBuf pst
821− gobuf !_ s backBuf fp@(Tuple' end contents) !cur !pst = do
822− x <- liftIO $ peekByteIndex contents cur
823− pRes <- pstep pst x
824− let next = INDEX_NEXT(cur,a)
825− case pRes of
826− PR.Partial 0 pst1 ->
827− gobuf SPEC s (List []) fp next pst1
828− PR.Partial n pst1 -> do
829− assert (n <= Prelude.length (x:getList backBuf)) (return ())
830− let src0 = Prelude.take n (x:getList backBuf)
831− arr0 = A.fromListN n (Prelude.reverse src0)
832− arr1 = Array contents next end
833− src = arr0 <> arr1
834− let !(Array cont1 start end1) = src
835− fp1 = Tuple' end1 cont1
836− gobuf SPEC s (List []) fp1 start pst1
837− PR.Continue 0 pst1 ->
838− gobuf SPEC s (List (x:getList backBuf)) fp next pst1
839− PR.Continue n pst1 -> do
840− assert (n <= Prelude.length (x:getList backBuf)) (return ())
841− let (src0, buf1) = splitAt n (x:getList backBuf)
842− arr0 = A.fromListN n (Prelude.reverse src0)
843− arr1 = Array contents next end
844− src = arr0 <> arr1
845− let !(Array cont1 start end1) = src
846− fp1 = Tuple' end1 cont1
847− gobuf SPEC s (List buf1) fp1 start pst1
848− PR.Done 0 b -> do
849− let arr = Array contents next end
850− return (b, D.cons arr (D.Stream step s))
851− PR.Done n b -> do
852− assert (n <= Prelude.length (x:getList backBuf)) (return ())
853− let src0 = Prelude.take n (x:getList backBuf)
854− -- XXX create the array in reverse instead
855− arr0 = A.fromListN n (Prelude.reverse src0)
856− arr1 = Array contents next end
857− -- XXX Use StreamK to avoid adding arbitrary layers of
858− -- constructors every time.
859− str = D.cons arr0 (D.cons arr1 (D.Stream step s))
860− return (b, str)
861− PR.Error err -> throwM $ ParseError err
862−-}
863−
864−-- | Parse an array stream using the supplied 'Parser'. Returns the parse
865−-- result and the unconsumed stream. Throws 'ParseError' if the parse fails.
866−--
867−-- The following alternative to this function allows composing the parser using
868−-- the parser Monad:
869−--
870−-- >>> parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)
871−--
872−-- We can compare perf and remove this one or define it in terms of that.
873−--
874−-- /Internal/
875−--
876−{-# INLINE_NORMAL parseBreakChunksK #-}
877−parseBreakChunksK ::
878− forall m a b. (MonadIO m, Unbox a)
879− => Parser a m b
880− -> StreamK m (Array a)
881− -> m (Either ParseError b, StreamK m (Array a))
882−-- parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)
883−parseBreakChunksK (Parser pstep initial extract) stream = do
884− res <- initial
885− case res of
886− IPartial s -> go s stream []
887− IDone b -> return (Right b, stream)
888− IError err -> return (Left (ParseError err), stream)
889−
890− where
891−
892− -- "backBuf" contains last few items in the stream that we may have to
893− -- backtrack to.
894− --
895− -- XXX currently we are using a dumb list based approach for backtracking
896− -- buffer. This can be replaced by a sliding/ring buffer using Data.Array.
897− -- That will allow us more efficient random back and forth movement.
898− go !pst st backBuf = do
899− let stop = goStop pst backBuf -- (, K.nil) <$> extract pst
900− single a = yieldk a StreamK.nil
901− yieldk arr r = goArray pst backBuf r arr
902− in StreamK.foldStream defState yieldk single stop st
903−
904− -- Use strictness on "cur" to keep it unboxed
905− goArray !pst backBuf st (Array _ cur end) | cur == end = go pst st backBuf
906− goArray !pst backBuf st (Array contents cur end) = do
907− x <- liftIO $ peekAt cur contents
908− pRes <- pstep pst x
909− let next = INDEX_NEXT(cur,a)
910− case pRes of
911− Parser.Partial 0 s ->
912− goArray s [] st (Array contents next end)
913− Parser.Partial n s -> do
914− assert (n <= Prelude.length (x:backBuf)) (return ())
915− let src0 = Prelude.take n (x:backBuf)
916− arr0 = fromListN n (Prelude.reverse src0)
917− arr1 = Array contents next end
918− src = arr0 <> arr1
919− goArray s [] st src
920− Parser.Continue 0 s ->
921− goArray s (x:backBuf) st (Array contents next end)
922− Parser.Continue n s -> do
923− assert (n <= Prelude.length (x:backBuf)) (return ())
924− let (src0, buf1) = Prelude.splitAt n (x:backBuf)
925− arr0 = fromListN n (Prelude.reverse src0)
926− arr1 = Array contents next end
927− src = arr0 <> arr1
928− goArray s buf1 st src
929− Parser.Done 0 b -> do
930− let arr = Array contents next end
931− return (Right b, StreamK.cons arr st)
932− Parser.Done n b -> do
933− assert (n <= Prelude.length (x:backBuf)) (return ())
934− let src0 = Prelude.take n (x:backBuf)
935− -- XXX Use fromListRevN once implemented
936− -- arr0 = A.fromListRevN n src0
937− arr0 = fromListN n (Prelude.reverse src0)
938− arr1 = Array contents next end
939− str = StreamK.cons arr0 (StreamK.cons arr1 st)
940− return (Right b, str)
941− Parser.Error err -> do
942− let n = Prelude.length backBuf
943− arr0 = fromListN n (Prelude.reverse backBuf)
944− arr1 = Array contents cur end
945− str = StreamK.cons arr0 (StreamK.cons arr1 stream)
946− return (Left (ParseError err), str)
947−
948− -- This is a simplified goArray
949− goExtract !pst backBuf (Array _ cur end)
950− | cur == end = goStop pst backBuf
951− goExtract !pst backBuf (Array contents cur end) = do
952− x <- liftIO $ peekAt cur contents
953− pRes <- pstep pst x
954− let next = INDEX_NEXT(cur,a)
955− case pRes of
956− Parser.Partial 0 s ->
957− goExtract s [] (Array contents next end)
958− Parser.Partial n s -> do
959− assert (n <= Prelude.length (x:backBuf)) (return ())
960− let src0 = Prelude.take n (x:backBuf)
961− arr0 = fromListN n (Prelude.reverse src0)
962− arr1 = Array contents next end
963− src = arr0 <> arr1
964− goExtract s [] src
965− Parser.Continue 0 s ->
966− goExtract s backBuf (Array contents next end)
967− Parser.Continue n s -> do
968− assert (n <= Prelude.length (x:backBuf)) (return ())
969− let (src0, buf1) = Prelude.splitAt n (x:backBuf)
970− arr0 = fromListN n (Prelude.reverse src0)
971− arr1 = Array contents next end
972− src = arr0 <> arr1
973− goExtract s buf1 src
974− Parser.Done 0 b -> do
975− let arr = Array contents next end
976− return (Right b, StreamK.fromPure arr)
977− Parser.Done n b -> do
978− assert (n <= Prelude.length backBuf) (return ())
979− let src0 = Prelude.take n backBuf
980− -- XXX Use fromListRevN once implemented
981− -- arr0 = A.fromListRevN n src0
982− arr0 = fromListN n (Prelude.reverse src0)
983− arr1 = Array contents next end
984− str = StreamK.cons arr0 (StreamK.fromPure arr1)
985− return (Right b, str)
986− Parser.Error err -> do
987− let n = Prelude.length backBuf
988− arr0 = fromListN n (Prelude.reverse backBuf)
989− arr1 = Array contents cur end
990− str = StreamK.cons arr0 (StreamK.cons arr1 stream)
991− return (Left (ParseError err), str)
992−
993− -- This is a simplified goExtract
994− {-# INLINE goStop #-}
995− goStop !pst backBuf = do
996− pRes <- extract pst
997− case pRes of
998− Parser.Partial _ _ -> error "Bug: parseBreak: Partial in extract"
999− Parser.Continue 0 s ->
1000− goStop s backBuf
1001− Parser.Continue n s -> do
1002− assert (n <= Prelude.length backBuf) (return ())
1003− let (src0, buf1) = Prelude.splitAt n backBuf
1004− arr = fromListN n (Prelude.reverse src0)
1005− goExtract s buf1 arr
1006− Parser.Done 0 b ->
1007− return (Right b, StreamK.nil)
1008− Parser.Done n b -> do
1009− assert (n <= Prelude.length backBuf) (return ())
1010− let src0 = Prelude.take n backBuf
1011− -- XXX Use fromListRevN once implemented
1012− -- arr0 = A.fromListRevN n src0
1013− arr0 = fromListN n (Prelude.reverse src0)
1014− return (Right b, StreamK.fromPure arr0)
1015− Parser.Error err -> do
1016− let n = Prelude.length backBuf
1017− arr0 = fromListN n (Prelude.reverse backBuf)
1018− return (Left (ParseError err), StreamK.fromPure arr0)
561+streamFold f arr = f (A.read arr)
core/src/Streamly/Internal/Data/Array/Generic.hs+185−152
@@ -4,6 +4,7 @@
44--
55-- License : BSD-3-Clause
66-- Maintainer : streamly@composewell.com
7+-- Stability : pre-release
78-- Portability : GHC
89--
910module Streamly.Internal.Data.Array.Generic
@@ -11,26 +12,25 @@ module Streamly.Internal.Data.Array.Generic
1112
1213 -- * Construction
1314 , nil
14− , createOf
15− , create
16− , writeWith
15+ , writeN
16+ , write
1717 , writeLastN
1818
19+ , fromStreamDN
20+ , fromStreamD
1921 , fromStreamN
2022 , fromStream
21− , fromPureStream
22− , fromByteStr#
2323
2424 , fromListN
2525 , fromList
2626
27− , chunksOf
28−
2927 -- * Elimination
3028 , length
3129 , reader
3230
3331 , toList
32+ , readStreamD
33+ , readRevStreamD
3434 , read
3535 , readRev
3636
@@ -41,42 +41,36 @@ module Streamly.Internal.Data.Array.Generic
4141
4242 -- * Random Access
4343 , getIndexUnsafe
44− , getIndex
45− , getSliceUnsafe
4644 , strip
47−
48− -- * Deprecated
49− , writeN
50− , write
45+ , putIndices
5146 )
5247where
5348
5449#include "inline.hs"
5550
56−import Control.Monad (replicateM)
57−import Control.Monad.IO.Class (MonadIO)
58−import Data.Functor.Identity (Identity(..))
59−import Data.Word (Word8)
51+import Control.Monad (when, replicateM)
52+import Control.Monad.IO.Class (liftIO, MonadIO)
6053import GHC.Base (MutableArray#, RealWorld)
61−import GHC.Exts (Addr#)
6254import GHC.IO (unsafePerformIO)
6355import Text.Read (readPrec)
6456
6557import Streamly.Internal.Data.Fold.Type (Fold(..))
66−import Streamly.Internal.Data.Stream.Type (Stream)
58+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
59+import Streamly.Internal.Data.Tuple.Strict (Tuple'(..), Tuple3'(..))
6760import Streamly.Internal.Data.Unfold.Type (Unfold(..))
6861import Streamly.Internal.System.IO (unsafeInlineIO)
6962
70−import qualified Streamly.Internal.Data.MutArray.Generic as MArray
63+import qualified Streamly.Internal.Data.Array.Generic.Mut.Type as MArray
7164import qualified Streamly.Internal.Data.Fold.Type as FL
7265import qualified Streamly.Internal.Data.Producer.Type as Producer
7366import qualified Streamly.Internal.Data.Producer as Producer
74−import qualified Streamly.Internal.Data.Ring.Generic as RB
75−import qualified Streamly.Internal.Data.Stream.Type as D
76−import qualified Streamly.Internal.Data.Stream.Generate as D
67+import qualified Streamly.Internal.Data.Ring as RB
68+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
69+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
7770import qualified Text.ParserCombinators.ReadPrec as ReadPrec
7871
79−import Prelude hiding (Foldable(..), read)
72+import Data.IORef
73+import Prelude hiding (foldr, length, read)
8074
8175-------------------------------------------------------------------------------
8276-- Array Data Type
@@ -94,87 +88,81 @@ data Array a =
9488 -- ^ The length of this slice.
9589 }
9690
97−unsafeFreeze :: MArray.MutArray a -> Array a
98−unsafeFreeze (MArray.MutArray cont# arrS arrL _) = Array cont# arrS arrL
91+unsafeFreeze :: MArray.Array a -> Array a
92+unsafeFreeze (MArray.Array cont# arrS arrL _) = Array cont# arrS arrL
9993
100−unsafeThaw :: Array a -> MArray.MutArray a
101−unsafeThaw (Array cont# arrS arrL) = MArray.MutArray cont# arrS arrL arrL
94+unsafeThaw :: Array a -> MArray.Array a
95+unsafeThaw (Array cont# arrS arrL) = MArray.Array cont# arrS arrL arrL
10296
10397{-# NOINLINE nil #-}
10498nil :: Array a
105−nil = unsafePerformIO $ unsafeFreeze <$> MArray.nil
99+nil = unsafePerformIO $ do
100+ marr <- MArray.new 0
101+ return $ unsafeFreeze marr
106102
107103-------------------------------------------------------------------------------
108104-- Construction - Folds
109105-------------------------------------------------------------------------------
110106
111−{-# INLINE_NORMAL createOf #-}
112−createOf :: MonadIO m => Int -> Fold m a (Array a)
113−createOf = fmap unsafeFreeze <$> MArray.createOf
114−
115−-- XXX Deprecate in major
116−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
117−{-# INLINE writeN #-}
107+{-# INLINE_NORMAL writeN #-}
118108writeN :: MonadIO m => Int -> Fold m a (Array a)
119−writeN = createOf
120−
121−{-# INLINE_NORMAL writeWith #-}
122−writeWith :: MonadIO m => Int -> Fold m a (Array a)
123−writeWith elemCount = unsafeFreeze <$> MArray.createWith elemCount
124−
125−-- | Fold the whole input to a single array.
126−--
127−-- /Caution! Do not use this on infinite streams./
128−--
129−{-# INLINE create #-}
130−create :: MonadIO m => Fold m a (Array a)
131−create = fmap unsafeFreeze MArray.create
109+writeN = fmap unsafeFreeze <$> MArray.writeN
132110
133−-- XXX Deprecate in major
134−-- {-# DEPRECATED write "Please use create instead." #-}
135−{-# INLINE write #-}
111+{-# INLINE_NORMAL write #-}
136112write :: MonadIO m => Fold m a (Array a)
137−write = create
138−
139−fromPureStream :: Stream Identity a -> Array a
140−fromPureStream x =
141− unsafePerformIO $ fmap (unsafeFreeze) (MArray.fromPureStream x)
142−-- fromPureStream = runIdentity . D.fold (unsafeMakePure write)
143−-- fromPureStream = fromList . runIdentity . D.toList
144−
145−fromByteStr# :: Addr# -> Array Word8
146−fromByteStr# addr = fromPureStream (D.fromByteStr# addr)
147−
148−-------------------------------------------------------------------------------
149−-- Stream Ops
150−-------------------------------------------------------------------------------
151−
152−{-# INLINE_NORMAL chunksOf #-}
153−chunksOf :: forall m a. MonadIO m
154− => Int -> Stream m a -> Stream m (Array a)
155−chunksOf n strm = fmap unsafeFreeze $ MArray.chunksOf n strm
113+write = Fold step initial extract
114+ where
115+ initial = do
116+ marr <- liftIO $ MArray.new 0
117+ return $ FL.Partial (Tuple3' marr 0 0)
118+ step (Tuple3' marr i capacity) x
119+ | i == capacity =
120+ let newCapacity = max (capacity * 2) 1
121+ in do newMarr <- liftIO $ MArray.realloc newCapacity marr
122+ marr1 <- liftIO $ MArray.snocUnsafe newMarr x
123+ return $ FL.Partial $ Tuple3' marr1 (i + 1) newCapacity
124+ | otherwise = do
125+ marr1 <- liftIO $ MArray.snocUnsafe marr x
126+ return $ FL.Partial $ Tuple3' marr1 (i + 1) capacity
127+ extract (Tuple3' marr _ _) =
128+ return $ unsafeFreeze marr
156129
157130-------------------------------------------------------------------------------
158131-- Construction - from streams
159132-------------------------------------------------------------------------------
160133
134+{-# INLINE_NORMAL fromStreamDN #-}
135+fromStreamDN :: MonadIO m => Int -> D.Stream m a -> m (Array a)
136+fromStreamDN limit str = do
137+ marr <- liftIO $ MArray.new (max limit 0)
138+ i <-
139+ D.foldlM'
140+ (\i x -> i `seq` liftIO $ MArray.putIndexUnsafe i marr x >> return (i + 1))
141+ (return 0) $
142+ D.take limit str
143+ return $ unsafeFreeze $ marr { MArray.arrLen = i }
144+
145+{-# INLINE fromStreamD #-}
146+fromStreamD :: MonadIO m => D.Stream m a -> m (Array a)
147+fromStreamD = D.fold write
148+
161149{-# INLINE fromStreamN #-}
162150fromStreamN :: MonadIO m => Int -> Stream m a -> m (Array a)
163−fromStreamN n = D.fold (writeN n)
151+fromStreamN n m = do
152+ when (n < 0) $ error "fromStreamN: negative write count specified"
153+ fromStreamDN n m
164154
165155{-# INLINE fromStream #-}
166156fromStream :: MonadIO m => Stream m a -> m (Array a)
167−fromStream = D.fold write
168−
169−-- XXX Consider foldr/build fusion in toList/fromList
157+fromStream = fromStreamD
170158
171159{-# INLINABLE fromListN #-}
172160fromListN :: Int -> [a] -> Array a
173−fromListN n xs = unsafePerformIO $ fromStreamN n $ D.fromList xs
161+fromListN n xs = unsafePerformIO $ fromStreamDN n $ D.fromList xs
174162
175163{-# INLINABLE fromList #-}
176164fromList :: [a] -> Array a
177−fromList xs = unsafePerformIO $ fromStream $ D.fromList xs
165+fromList xs = unsafePerformIO $ fromStreamD $ D.fromList xs
178166
179167-------------------------------------------------------------------------------
180168-- Elimination - Unfolds
@@ -203,37 +191,44 @@ toList arr = loop 0
203191
204192 len = length arr
205193 loop i | i == len = []
206− loop i = getIndexUnsafe i arr : loop (i + 1)
194+ loop i = getIndexUnsafe arr i : loop (i + 1)
207195
208−{-# INLINE_NORMAL read #-}
209−read :: Monad m => Array a -> Stream m a
210−read arr@Array{..} =
211− D.map (`getIndexUnsafe` arr) $ D.enumerateFromToIntegral 0 (arrLen - 1)
196+{-# INLINE_NORMAL readStreamD #-}
197+readStreamD :: MonadIO m => Array a -> D.Stream m a
198+readStreamD = MArray.toStreamD . unsafeThaw
212199
213−{-# INLINE_NORMAL readRev #-}
214−readRev :: Monad m => Array a -> Stream m a
215−readRev arr@Array{..} =
216− D.map (`getIndexUnsafe` arr)
200+{-# INLINE_NORMAL readRevStreamD #-}
201+readRevStreamD :: Monad m => Array a -> D.Stream m a
202+readRevStreamD arr@Array{..} =
203+ D.map (getIndexUnsafe arr)
217204 $ D.enumerateFromThenToIntegral (arrLen - 1) (arrLen - 2) 0
218205
206+{-# INLINE_EARLY read #-}
207+read :: MonadIO m => Array a -> Stream m a
208+read = readStreamD
209+
210+{-# INLINE_EARLY readRev #-}
211+readRev :: Monad m => Array a -> Stream m a
212+readRev = readRevStreamD
213+
219214-------------------------------------------------------------------------------
220215-- Elimination - using Folds
221216-------------------------------------------------------------------------------
222217
223218{-# INLINE_NORMAL foldl' #-}
224219foldl' :: (b -> a -> b) -> b -> Array a -> b
225−foldl' f z arr = unsafePerformIO $ D.foldl' f z $ read arr
220+foldl' f z arr = unsafePerformIO $ D.foldl' f z $ readStreamD arr
226221
227222{-# INLINE_NORMAL foldr #-}
228223foldr :: (a -> b -> b) -> b -> Array a -> b
229−foldr f z arr = unsafePerformIO $ D.foldr f z $ read arr
224+foldr f z arr = unsafePerformIO $ D.foldr f z $ readStreamD arr
230225
231226{-# INLINE fold #-}
232−fold :: Monad m => Fold m a b -> Array a -> m b
233−fold f arr = D.fold f (read arr)
227+fold :: MonadIO m => Fold m a b -> Array a -> m b
228+fold f arr = D.fold f (readStreamD arr)
234229
235230{-# INLINE streamFold #-}
236−streamFold :: Monad m => (Stream m a -> m b) -> Array a -> m b
231+streamFold :: MonadIO m => (Stream m a -> m b) -> Array a -> m b
237232streamFold f arr = f (read arr)
238233
239234-------------------------------------------------------------------------------
@@ -245,96 +240,134 @@ streamFold f arr = f (read arr)
245240--
246241-- @since 0.8.0
247242{-# INLINE getIndexUnsafe #-}
248−getIndexUnsafe :: Int -> Array a -> a
249−getIndexUnsafe i arr =
243+getIndexUnsafe :: Array a -> Int -> a
244+getIndexUnsafe arr i =
250245 unsafePerformIO $ MArray.getIndexUnsafe i (unsafeThaw arr)
251246
252−-- | Lookup the element at the given index. Index starts from 0.
253−--
254−{-# INLINE getIndex #-}
255−getIndex :: Int -> Array a -> Maybe a
256−getIndex i arr@Array {..} =
257− if i >= 0 && i < arrLen
258− then Just $ getIndexUnsafe i arr
259− else Nothing
260−
261−-- >>> import qualified Streamly.Data.Stream as Stream
262−-- >>> import qualified Streamly.Data.Fold as Fold
263−-- >>> import qualified Streamly.Internal.Data.Array.Generic as Array
264−-- >>> import Data.Function ((&))
265−-- >>> :{
266−-- Stream.fromList [1,2,3,4,5::Int]
267−-- & Stream.scan (Array.writeLastN 2)
268−-- & Stream.fold Fold.toList
269−-- :}
270−-- [fromList [],fromList [1],fromList [1,2],fromList [2,3],fromList [3,4],fromList [4,5]]
271−--
272247{-# INLINE writeLastN #-}
273248writeLastN :: MonadIO m => Int -> Fold m a (Array a)
274−writeLastN n = FL.rmapM f (RB.writeLastN n)
249+writeLastN n
250+ | n <= 0 = fmap (const nil) FL.drain
251+ | otherwise = Fold step initial done
275252
276253 where
277254
278− f rb = do
279− arr <- RB.copyToMutArray 0 n rb
280− return $ unsafeFreeze arr
255+ initial = do
256+ rb <- liftIO $ RB.createRing n
257+ return $ FL.Partial $ Tuple' rb (0 :: Int)
281258
282−{-# INLINE getSliceUnsafe #-}
283−getSliceUnsafe :: Int -> Int -> Array a -> Array a
284−getSliceUnsafe offset len (Array cont off1 _) = Array cont (off1 + offset) len
259+ step (Tuple' rb rh) x = do
260+ liftIO $ RB.unsafeInsertRing rb rh x
261+ return $ FL.Partial $ Tuple' rb (rh + 1)
262+
263+ done (Tuple' rb rh) = do
264+ arr' <- liftIO $ MArray.new (min rh n)
265+ ref <- liftIO $ readIORef $ RB.ringHead rb
266+ if rh < n
267+ then
268+ MArray.putSliceUnsafe (RB.arr rb) 0 arr' 0 ref
269+ else do
270+ MArray.putSliceUnsafe (RB.arr rb) ref arr' 0 (n - ref)
271+ MArray.putSliceUnsafe (RB.arr rb) 0 arr' (n - ref) ref
272+ return $ unsafeFreeze arr'
273+
274+sliceUnsafe :: Int -> Int -> Array a -> Array a
275+sliceUnsafe offset len (Array cont off1 _) = Array cont (off1 + offset) len
285276
286277-- XXX This is not efficient as it copies the array. We should support array
287278-- slicing so that we can just refer to the underlying array memory instead of
288279-- copying.
289−
280+--
290281-- | Truncate the array at the beginning and end as long as the predicate
291−-- holds true. Returns a slice of the original array.
292−{-# INLINE strip #-}
282+-- holds true.
293283strip :: (a -> Bool) -> Array a -> Array a
294−strip p arr = unsafeFreeze $ unsafePerformIO $ MArray.strip p (unsafeThaw arr)
284+strip p arr =
285+ let lastIndex = length arr - 1
286+ indexR = getIndexR lastIndex -- last predicate failing index
287+ in if indexR == -1
288+ then nil
289+ else
290+ let indexL = getIndexL 0 -- first predicate failing index
291+ in if indexL == 0 && indexR == lastIndex
292+ then arr
293+ else unsafeFreeze $ unsafePerformIO $ do
294+ let newLen = indexR - indexL + 1
295+ arrThawed = unsafeThaw (sliceUnsafe indexL newLen arr)
296+ MArray.clone arrThawed
297+
298+ where
299+
300+ getIndexR idx
301+ | idx < 0 = idx
302+ | otherwise =
303+ if p (getIndexUnsafe arr idx) then getIndexR (idx - 1) else idx
304+
305+ getIndexL idx = if p (getIndexUnsafe arr idx)
306+ then getIndexL (idx + 1)
307+ else idx
308+
309+-- | Write an input stream of (index, value) pairs to an array. Throws an
310+-- error if any index is out of bounds.
311+--
312+-- /Pre-release/
313+{-# INLINE putIndices #-}
314+putIndices :: MonadIO m
315+ => Array a -> Fold m (Int, a) ()
316+putIndices arr = FL.rmapM (\ _ -> return ()) (FL.foldlM' step initial)
317+
318+ where
319+
320+ initial = return $ unsafeThaw arr
321+
322+ step marr (i, x) = do
323+ MArray.putIndexUnsafe i marr x
324+ return marr
295325
296326-------------------------------------------------------------------------------
297327-- Instances
298328-------------------------------------------------------------------------------
299329
300330instance Eq a => Eq (Array a) where
301− {-# INLINE (==) #-}
302− arr1 == arr2 =
303− unsafeInlineIO $! unsafeThaw arr1 `MArray.eq` unsafeThaw arr2
331+ a1 == a2 = lenA1 == lenA2 && loop (lenA1 - 1)
304332
305−instance Ord a => Ord (Array a) where
306− {-# INLINE compare #-}
307− compare arr1 arr2 =
308− unsafeInlineIO $! unsafeThaw arr1 `MArray.cmp` unsafeThaw arr2
309−
310− -- Default definitions defined in base do not have an INLINE on them, so we
311− -- replicate them here with an INLINE.
312− {-# INLINE (<) #-}
313− x < y = case compare x y of { LT -> True; _ -> False }
314−
315− {-# INLINE (<=) #-}
316− x <= y = case compare x y of { GT -> False; _ -> True }
333+ where
317334
318− {-# INLINE (>) #-}
319− x > y = case compare x y of { GT -> True; _ -> False }
335+ lenA1 = length a1
336+ lenA2 = length a2
320337
321− {-# INLINE (>=) #-}
322− x >= y = case compare x y of { LT -> False; _ -> True }
338+ loop i
339+ | i < 0 = True
340+ | otherwise =
341+ let v1 = getIndexUnsafe a1 i
342+ v2 = getIndexUnsafe a2 i
343+ in v1 == v2 && loop (i - 1)
323344
324− -- These two default methods use '<=' rather than 'compare'
325− -- because the latter is often more expensive
326− {-# INLINE max #-}
327− max x y = if x <= y then y else x
328−
329− {-# INLINE min #-}
330− min x y = if x <= y then x else y
345+instance Ord a => Ord (Array a) where
346+ compare a1 a2 =
347+ case compare lenA1 lenA2 of
348+ LT -> LT
349+ GT -> GT
350+ EQ -> loop 0
351+
352+ where
353+
354+ lenA1 = length a1
355+ lenA2 = length a2
356+
357+ loop i
358+ | i >= lenA1 = EQ
359+ | otherwise =
360+ let v1 = getIndexUnsafe a1 i
361+ v2 = getIndexUnsafe a2 i
362+ in case compare v1 v2 of
363+ LT -> LT
364+ GT -> GT
365+ EQ -> loop (i + 1)
331366
332367instance Show a => Show (Array a) where
333− {-# INLINE show #-}
334368 show arr = "fromList " ++ show (toList arr)
335369
336370instance Read a => Read (Array a) where
337− {-# INLINE readPrec #-}
338371 readPrec = do
339372 fromListWord <- replicateM 9 ReadPrec.get
340373 if fromListWord == "fromList "
core/src/Streamly/Internal/Data/Array/Generic/Mut/Type.hsnew file+608
@@ -0,0 +1,608 @@
1+{-# LANGUAGE UnboxedTuples #-}
2+
3+-- |
4+-- Module : Streamly.Internal.Data.Array.Generic.Mut.Type
5+-- Copyright : (c) 2020 Composewell Technologies
6+-- License : BSD3-3-Clause
7+-- Maintainer : streamly@composewell.com
8+-- Stability : experimental
9+-- Portability : GHC
10+--
11+module Streamly.Internal.Data.Array.Generic.Mut.Type
12+(
13+ -- * Type
14+ -- $arrayNotes
15+ Array (..)
16+
17+ -- * Constructing and Writing
18+ -- ** Construction
19+ -- , nil
20+
21+ -- *** Uninitialized Arrays
22+ , new
23+ -- , newArrayWith
24+
25+ -- *** From streams
26+ , writeNUnsafe
27+ , writeN
28+
29+ -- , writeWith
30+ -- , write
31+
32+ -- , writeRevN
33+ -- , writeRev
34+
35+ -- ** From containers
36+ -- , fromListN
37+ -- , fromList
38+ -- , fromStreamDN
39+ -- , fromStreamD
40+
41+ -- * Random writes
42+ , putIndex
43+ , putIndexUnsafe
44+ -- , putIndices
45+ -- , putFromThenTo
46+ -- , putFrom -- start writing at the given position
47+ -- , putUpto -- write from beginning up to the given position
48+ -- , putFromTo
49+ -- , putFromRev
50+ -- , putUptoRev
51+ , modifyIndexUnsafe
52+ , modifyIndex
53+ -- , modifyIndices
54+ -- , modify
55+ -- , swapIndices
56+
57+ -- * Growing and Shrinking
58+ -- Arrays grow only at the end, though it is possible to grow on both sides
59+ -- and therefore have a cons as well as snoc. But that will require two
60+ -- bounds in the array representation.
61+
62+ -- ** Reallocation
63+ , realloc
64+
65+ -- ** Appending elements
66+ , snocWith
67+ , snoc
68+ -- , snocLinear
69+ -- , snocMay
70+ , snocUnsafe
71+
72+ -- ** Appending streams
73+ -- , writeAppendNUnsafe
74+ -- , writeAppendN
75+ -- , writeAppendWith
76+ -- , writeAppend
77+
78+ -- ** Truncation
79+ -- These are not the same as slicing the array at the beginning, they may
80+ -- reduce the length as well as the capacity of the array.
81+ -- , truncateWith
82+ -- , truncate
83+ -- , truncateExp
84+
85+ -- * Eliminating and Reading
86+
87+ -- ** Unfolds
88+ , reader
89+ -- , readerRev
90+ , producerWith -- experimental
91+ , producer -- experimental
92+
93+ -- ** To containers
94+ , toStreamD
95+ -- , toStreamDRev
96+ , toStreamK
97+ -- , toStreamKRev
98+ , toList
99+
100+ -- ** Random reads
101+ , getIndex
102+ , getIndexUnsafe
103+ -- , getIndices
104+ -- , getFromThenTo
105+ -- , getIndexRev
106+
107+ -- * In-place Mutation Algorithms
108+ -- , reverse
109+ -- , permute
110+ -- , partitionBy
111+ -- , shuffleBy
112+ -- , divideBy
113+ -- , mergeBy
114+
115+ -- * Folding
116+ -- , foldl'
117+ -- , foldr
118+ -- , cmp
119+
120+ -- * Arrays of arrays
121+ -- We can add dimensionality parameter to the array type to get
122+ -- multidimensional arrays. Multidimensional arrays would just be a
123+ -- convenience wrapper on top of single dimensional arrays.
124+
125+ -- | Operations dealing with multiple arrays, streams of arrays or
126+ -- multidimensional array representations.
127+
128+ -- ** Construct from streams
129+ -- , arraysOf
130+ -- , arrayStreamKFromStreamD
131+ -- , writeChunks
132+
133+ -- ** Eliminate to streams
134+ -- , flattenArrays
135+ -- , flattenArraysRev
136+ -- , fromArrayStreamK
137+
138+ -- ** Construct from arrays
139+ -- get chunks without copying
140+ , getSliceUnsafe
141+ , getSlice
142+ -- , getSlicesFromLenN
143+ -- , splitAt -- XXX should be able to express using getSlice
144+ -- , breakOn
145+
146+ -- ** Appending arrays
147+ -- , spliceCopy
148+ -- , spliceWith
149+ -- , splice
150+ -- , spliceExp
151+ , putSliceUnsafe
152+ -- , appendSlice
153+ -- , appendSliceFrom
154+
155+ , clone
156+ )
157+where
158+
159+#include "inline.hs"
160+#include "assert.hs"
161+
162+import Control.Monad.IO.Class (MonadIO(..))
163+import GHC.Base
164+ ( MutableArray#
165+ , RealWorld
166+ , copyMutableArray#
167+ , newArray#
168+ , readArray#
169+ , writeArray#
170+ )
171+import GHC.IO (IO(..))
172+import GHC.Int (Int(..))
173+import Streamly.Internal.Data.Fold.Type (Fold(..))
174+import Streamly.Internal.Data.Producer.Type (Producer (..))
175+import Streamly.Internal.Data.Unfold.Type (Unfold(..))
176+
177+import qualified Streamly.Internal.Data.Fold.Type as FL
178+import qualified Streamly.Internal.Data.Producer as Producer
179+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
180+import qualified Streamly.Internal.Data.Stream.StreamD.Generate as D
181+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
182+
183+import Prelude hiding (read)
184+
185+-- $setup
186+-- >>> :m
187+-- >>> import qualified Streamly.Internal.Data.Array.Generic.Mut.Type as Array
188+-- >>> import qualified Streamly.Internal.Data.Stream as Stream
189+-- >>> import qualified Streamly.Internal.Data.Stream.StreamD as StreamD
190+-- >>> import qualified Streamly.Internal.Data.Fold as Fold
191+-- >>> import qualified Streamly.Internal.Data.Fold.Type as Fold
192+
193+-------------------------------------------------------------------------------
194+-- Array Data Type
195+-------------------------------------------------------------------------------
196+
197+data Array a =
198+ Array
199+ { arrContents# :: MutableArray# RealWorld a
200+ -- ^ The internal contents of the array representing the entire array.
201+
202+ , arrStart :: {-# UNPACK #-}!Int
203+ -- ^ The starting index of this slice.
204+
205+ , arrLen :: {-# UNPACK #-}!Int
206+ -- ^ The length of this slice.
207+
208+ , arrTrueLen :: {-# UNPACK #-}!Int
209+ -- ^ This is the true length of the array. Coincidentally, this also
210+ -- represents the first index beyond the maximum acceptable index of
211+ -- the array. This is specific to the array contents itself and not
212+ -- dependent on the slice. This value should not change and is shared
213+ -- across all the slices.
214+ }
215+
216+{-# INLINE bottomElement #-}
217+bottomElement :: a
218+bottomElement =
219+ error
220+ $ unwords
221+ [ funcName
222+ , "This is the bottom element of the array."
223+ , "This is a place holder and should never be reached!"
224+ ]
225+
226+ where
227+
228+ funcName = "Streamly.Internal.Data.Array.Generic.Mut.Type.bottomElement:"
229+
230+-- XXX Would be nice if GHC can provide something like newUninitializedArray# so
231+-- that we do not have to write undefined or error in the whole array.
232+-- | @new count@ allocates an empty array that can hold 'count' items.
233+--
234+-- /Pre-release/
235+{-# INLINE new #-}
236+new :: forall m a. MonadIO m => Int -> m (Array a)
237+new n@(I# n#) =
238+ liftIO
239+ $ IO
240+ $ \s# ->
241+ case newArray# n# bottomElement s# of
242+ (# s1#, arr# #) ->
243+ let ma = Array arr# 0 0 n
244+ in (# s1#, ma #)
245+
246+-------------------------------------------------------------------------------
247+-- Random writes
248+-------------------------------------------------------------------------------
249+
250+-- | Write the given element to the given index of the array. Does not check if
251+-- the index is out of bounds of the array.
252+--
253+-- /Pre-release/
254+{-# INLINE putIndexUnsafe #-}
255+putIndexUnsafe :: forall m a. MonadIO m => Int -> Array a -> a -> m ()
256+putIndexUnsafe i Array {..} x =
257+ liftIO
258+ $ IO
259+ $ \s# ->
260+ case i + arrStart of
261+ I# n# ->
262+ let s1# = writeArray# arrContents# n# x s#
263+ in (# s1#, () #)
264+
265+invalidIndex :: String -> Int -> a
266+invalidIndex label i =
267+ error $ label ++ ": invalid array index " ++ show i
268+
269+-- | /O(1)/ Write the given element at the given index in the array.
270+-- Performs in-place mutation of the array.
271+--
272+-- >>> putIndex ix arr val = Array.modifyIndex ix arr (const (val, ()))
273+--
274+-- /Pre-release/
275+{-# INLINE putIndex #-}
276+putIndex :: MonadIO m => Int -> Array a -> a -> m ()
277+putIndex i arr@Array {..} x =
278+ if i >= 0 && i < arrLen
279+ then putIndexUnsafe i arr x
280+ else invalidIndex "putIndex" i
281+
282+-- | Modify a given index of an array using a modifier function without checking
283+-- the bounds.
284+--
285+-- Unsafe because it does not check the bounds of the array.
286+--
287+-- /Pre-release/
288+modifyIndexUnsafe :: MonadIO m => Int -> Array a -> (a -> (a, b)) -> m b
289+modifyIndexUnsafe i Array {..} f = do
290+ liftIO
291+ $ IO
292+ $ \s# ->
293+ case i + arrStart of
294+ I# n# ->
295+ case readArray# arrContents# n# s# of
296+ (# s1#, a #) ->
297+ let (a1, b) = f a
298+ s2# = writeArray# arrContents# n# a1 s1#
299+ in (# s2#, b #)
300+
301+-- | Modify a given index of an array using a modifier function.
302+--
303+-- /Pre-release/
304+modifyIndex :: MonadIO m => Int -> Array a -> (a -> (a, b)) -> m b
305+modifyIndex i arr@Array {..} f = do
306+ if i >= 0 && i < arrLen
307+ then modifyIndexUnsafe i arr f
308+ else invalidIndex "modifyIndex" i
309+
310+-------------------------------------------------------------------------------
311+-- Resizing
312+-------------------------------------------------------------------------------
313+
314+-- | Reallocates the array according to the new size. This is a safe function
315+-- that always creates a new array and copies the old array into the new one. If
316+-- the reallocated size is less than the original array it results in a
317+-- truncated version of the original array.
318+--
319+realloc :: MonadIO m => Int -> Array a -> m (Array a)
320+realloc n arr = do
321+ arr1 <- new n
322+ let !newLen@(I# newLen#) = min n (arrLen arr)
323+ !(I# arrS#) = arrStart arr
324+ !(I# arr1S#) = arrStart arr1
325+ arrC# = arrContents# arr
326+ arr1C# = arrContents# arr1
327+ liftIO
328+ $ IO
329+ $ \s# ->
330+ let s1# = copyMutableArray# arrC# arrS# arr1C# arr1S# newLen# s#
331+ in (# s1#, arr1 {arrLen = newLen, arrTrueLen = n} #)
332+
333+reallocWith ::
334+ MonadIO m => String -> (Int -> Int) -> Int -> Array a -> m (Array a)
335+reallocWith label sizer reqSize arr = do
336+ let oldSize = arrLen arr
337+ newSize = sizer oldSize
338+ safeSize = max newSize (oldSize + reqSize)
339+ assert (newSize >= oldSize + reqSize || error badSize) (return ())
340+ realloc safeSize arr
341+
342+ where
343+
344+ badSize = concat
345+ [ label
346+ , ": new array size is less than required size "
347+ , show reqSize
348+ , ". Please check the sizing function passed."
349+ ]
350+
351+-------------------------------------------------------------------------------
352+-- Snoc
353+-------------------------------------------------------------------------------
354+
355+-- XXX Not sure of the behavior of writeArray# if we specify an index which is
356+-- out of bounds. This comment should be rewritten based on that.
357+-- | Really really unsafe, appends the element into the first array, may
358+-- cause silent data corruption or if you are lucky a segfault if the index
359+-- is out of bounds.
360+--
361+-- /Internal/
362+{-# INLINE snocUnsafe #-}
363+snocUnsafe :: MonadIO m => Array a -> a -> m (Array a)
364+snocUnsafe arr@Array {..} a = do
365+ assert (arrStart + arrLen < arrTrueLen) (return ())
366+ putIndexUnsafe arrLen arr a
367+ return $ arr {arrLen = arrLen + 1}
368+
369+-- NOINLINE to move it out of the way and not pollute the instruction cache.
370+{-# NOINLINE snocWithRealloc #-}
371+snocWithRealloc :: MonadIO m => (Int -> Int) -> Array a -> a -> m (Array a)
372+snocWithRealloc sizer arr x = do
373+ arr1 <- reallocWith "snocWithRealloc" sizer 1 arr
374+ snocUnsafe arr1 x
375+
376+-- | @snocWith sizer arr elem@ mutates @arr@ to append @elem@. The length of
377+-- the array increases by 1.
378+--
379+-- If there is no reserved space available in @arr@ it is reallocated to a size
380+-- in bytes determined by the @sizer oldSize@ function, where @oldSize@ is the
381+-- original size of the array.
382+--
383+-- Note that the returned array may be a mutated version of the original array.
384+--
385+-- /Pre-release/
386+{-# INLINE snocWith #-}
387+snocWith :: MonadIO m => (Int -> Int) -> Array a -> a -> m (Array a)
388+snocWith sizer arr@Array {..} x = do
389+ if arrStart + arrLen < arrTrueLen
390+ then snocUnsafe arr x
391+ else snocWithRealloc sizer arr x
392+
393+-- XXX round it to next power of 2.
394+--
395+-- | The array is mutated to append an additional element to it. If there is no
396+-- reserved space available in the array then it is reallocated to double the
397+-- original size.
398+--
399+-- This is useful to reduce allocations when appending unknown number of
400+-- elements.
401+--
402+-- Note that the returned array may be a mutated version of the original array.
403+--
404+-- >>> snoc = Array.snocWith (* 2)
405+--
406+-- Performs O(n * log n) copies to grow, but is liberal with memory allocation.
407+--
408+-- /Pre-release/
409+{-# INLINE snoc #-}
410+snoc :: MonadIO m => Array a -> a -> m (Array a)
411+snoc = snocWith (* 2)
412+
413+-------------------------------------------------------------------------------
414+-- Random reads
415+-------------------------------------------------------------------------------
416+
417+-- | Return the element at the specified index without checking the bounds.
418+--
419+-- Unsafe because it does not check the bounds of the array.
420+{-# INLINE_NORMAL getIndexUnsafe #-}
421+getIndexUnsafe :: MonadIO m => Int -> Array a -> m a
422+getIndexUnsafe n Array {..} =
423+ liftIO
424+ $ IO
425+ $ \s# ->
426+ let !(I# i#) = arrStart + n
427+ in readArray# arrContents# i# s#
428+
429+-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
430+--
431+{-# INLINE getIndex #-}
432+getIndex :: MonadIO m => Int -> Array a -> m a
433+getIndex i arr@Array {..} =
434+ if i >= 0 && i < arrLen
435+ then getIndexUnsafe i arr
436+ else invalidIndex "getIndex" i
437+
438+-------------------------------------------------------------------------------
439+-- Subarrays
440+-------------------------------------------------------------------------------
441+
442+-- XXX We can also get immutable slices.
443+
444+-- | /O(1)/ Slice an array in constant time.
445+--
446+-- Unsafe: The bounds of the slice are not checked.
447+--
448+-- /Unsafe/
449+--
450+-- /Pre-release/
451+{-# INLINE getSliceUnsafe #-}
452+getSliceUnsafe
453+ :: Int -- ^ from index
454+ -> Int -- ^ length of the slice
455+ -> Array a
456+ -> Array a
457+getSliceUnsafe index len arr@Array {..} =
458+ assert (index >= 0 && len >= 0 && index + len <= arrLen)
459+ $ arr {arrStart = arrStart + index, arrLen = len}
460+
461+
462+-- | /O(1)/ Slice an array in constant time. Throws an error if the slice
463+-- extends out of the array bounds.
464+--
465+-- /Pre-release/
466+{-# INLINE getSlice #-}
467+getSlice
468+ :: Int -- ^ from index
469+ -> Int -- ^ length of the slice
470+ -> Array a
471+ -> Array a
472+getSlice index len arr@Array{..} =
473+ if index >= 0 && len >= 0 && index + len <= arrLen
474+ then arr {arrStart = arrStart + index, arrLen = len}
475+ else error
476+ $ "getSlice: invalid slice, index "
477+ ++ show index ++ " length " ++ show len
478+
479+-------------------------------------------------------------------------------
480+-- to Lists and streams
481+-------------------------------------------------------------------------------
482+
483+-- | Convert an 'Array' into a list.
484+--
485+-- /Pre-release/
486+{-# INLINE toList #-}
487+toList :: MonadIO m => Array a -> m [a]
488+toList arr@Array{..} = mapM (`getIndexUnsafe` arr) [0 .. (arrLen - 1)]
489+
490+-- | Use the 'read' unfold instead.
491+--
492+-- @toStreamD = D.unfold read@
493+--
494+-- We can try this if the unfold has any performance issues.
495+{-# INLINE_NORMAL toStreamD #-}
496+toStreamD :: MonadIO m => Array a -> D.Stream m a
497+toStreamD arr@Array{..} =
498+ D.mapM (`getIndexUnsafe` arr) $ D.enumerateFromToIntegral 0 (arrLen - 1)
499+
500+{-# INLINE toStreamK #-}
501+toStreamK :: MonadIO m => Array a -> K.Stream m a
502+toStreamK arr@Array{..} = K.unfoldrM step 0
503+
504+ where
505+
506+ step i
507+ | i == arrLen = return Nothing
508+ | otherwise = do
509+ x <- getIndexUnsafe i arr
510+ return $ Just (x, i + 1)
511+
512+
513+-------------------------------------------------------------------------------
514+-- Folds
515+-------------------------------------------------------------------------------
516+
517+-- | Like 'writeN' but does not check the array bounds when writing. The fold
518+-- driver must not call the step function more than 'n' times otherwise it will
519+-- corrupt the memory and crash. This function exists mainly because any
520+-- conditional in the step function blocks fusion causing 10x performance
521+-- slowdown.
522+--
523+-- /Pre-release/
524+{-# INLINE_NORMAL writeNUnsafe #-}
525+writeNUnsafe :: MonadIO m => Int -> Fold m a (Array a)
526+writeNUnsafe n = Fold step initial return
527+
528+ where
529+
530+ initial = FL.Partial <$> new (max n 0)
531+
532+ step arr x = FL.Partial <$> snocUnsafe arr x
533+
534+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
535+-- 'Array'.
536+--
537+-- >>> writeN n = Fold.take n (Array.writeNUnsafe n)
538+--
539+-- /Pre-release/
540+{-# INLINE_NORMAL writeN #-}
541+writeN :: MonadIO m => Int -> Fold m a (Array a)
542+writeN n = FL.take n $ writeNUnsafe n
543+
544+-------------------------------------------------------------------------------
545+-- Unfolds
546+-------------------------------------------------------------------------------
547+
548+-- | Resumable unfold of an array.
549+--
550+{-# INLINE_NORMAL producerWith #-}
551+producerWith :: Monad m => (forall b. IO b -> m b) -> Producer m (Array a) a
552+producerWith liftio = Producer step inject extract
553+
554+ where
555+
556+ {-# INLINE inject #-}
557+ inject arr = return (arr, 0)
558+
559+ {-# INLINE extract #-}
560+ extract (arr, i) =
561+ return $ arr {arrStart = arrStart arr + i, arrLen = arrLen arr - i}
562+
563+ {-# INLINE_LATE step #-}
564+ step (arr, i)
565+ | assert (arrLen arr >= 0) (i == arrLen arr) = return D.Stop
566+ step (arr, i) = do
567+ x <- liftio $ getIndexUnsafe i arr
568+ return $ D.Yield x (arr, i + 1)
569+
570+-- | Resumable unfold of an array.
571+--
572+{-# INLINE_NORMAL producer #-}
573+producer :: MonadIO m => Producer m (Array a) a
574+producer = producerWith liftIO
575+
576+-- | Unfold an array into a stream.
577+--
578+{-# INLINE_NORMAL reader #-}
579+reader :: MonadIO m => Unfold m (Array a) a
580+reader = Producer.simplify producer
581+
582+--------------------------------------------------------------------------------
583+-- Appending arrays
584+--------------------------------------------------------------------------------
585+
586+-- | Put a sub range of a source array into a subrange of a destination array.
587+-- This is not safe as it does not check the bounds.
588+{-# INLINE putSliceUnsafe #-}
589+putSliceUnsafe :: MonadIO m => Array a -> Int -> Array a -> Int -> Int -> m ()
590+putSliceUnsafe src srcStart dst dstStart len = liftIO $ do
591+ assertM(len <= arrLen dst)
592+ assertM(len <= arrLen src)
593+ let !(I# srcStart#) = srcStart + arrStart src
594+ !(I# dstStart#) = dstStart + arrStart dst
595+ !(I# len#) = len
596+ let arrS# = arrContents# src
597+ arrD# = arrContents# dst
598+ IO $ \s# -> (# copyMutableArray#
599+ arrS# srcStart# arrD# dstStart# len# s#
600+ , () #)
601+
602+{-# INLINE clone #-}
603+clone :: MonadIO m => Array a -> m (Array a)
604+clone src = liftIO $ do
605+ let len = arrLen src
606+ dst <- new len
607+ putSliceUnsafe src 0 dst 0 len
608+ return dst
core/src/Streamly/Internal/Data/Array/Mut.hsnew file+100
@@ -0,0 +1,100 @@
1+#include "inline.hs"
2+
3+-- |
4+-- Module : Streamly.Internal.Data.Array.Mut
5+-- Copyright : (c) 2020 Composewell Technologies
6+-- License : BSD-3-Clause
7+-- Maintainer : streamly@composewell.com
8+-- Stability : experimental
9+-- Portability : GHC
10+--
11+-- Unboxed pinned mutable array type for 'Unboxed' types with an option to use
12+-- foreign (non-GHC) memory allocators. Fulfils the following goals:
13+--
14+-- * Random access (array)
15+-- * Efficient storage (unboxed)
16+-- * Performance (unboxed access)
17+-- * Performance - in-place operations (mutable)
18+-- * Performance - GC (pinned, mutable)
19+-- * interfacing with OS (pinned)
20+-- * Fragmentation control (foreign allocators)
21+--
22+-- Stream and Fold APIs allow easy, efficient and convenient operations on
23+-- arrays.
24+
25+module Streamly.Internal.Data.Array.Mut
26+ (
27+ module Streamly.Internal.Data.Array.Mut.Type
28+ , splitOn
29+ , genSlicesFromLen
30+ , getSlicesFromLen
31+ , fromStream
32+ )
33+where
34+
35+import Control.Monad.IO.Class (MonadIO(..))
36+import Streamly.Internal.Data.Unboxed (Unbox)
37+import Streamly.Internal.Data.Stream.StreamD (Stream)
38+import Streamly.Internal.Data.Unfold.Type (Unfold(..))
39+
40+import qualified Streamly.Internal.Data.Stream.StreamD as D
41+import qualified Streamly.Internal.Data.Unfold as Unfold
42+
43+import Prelude hiding (foldr, length, read, splitAt)
44+import Streamly.Internal.Data.Array.Mut.Type
45+
46+-- | Split the array into a stream of slices using a predicate. The element
47+-- matching the predicate is dropped.
48+--
49+-- /Pre-release/
50+{-# INLINE splitOn #-}
51+splitOn :: (MonadIO m, Unbox a) =>
52+ (a -> Bool) -> Array a -> Stream m (Array a)
53+splitOn predicate arr =
54+ fmap (\(i, len) -> getSliceUnsafe i len arr)
55+ $ D.sliceOnSuffix predicate (toStreamD arr)
56+
57+-- | Generate a stream of array slice descriptors ((index, len)) of specified
58+-- length from an array, starting from the supplied array index. The last slice
59+-- may be shorter than the requested length depending on the array length.
60+--
61+-- /Pre-release/
62+{-# INLINE genSlicesFromLen #-}
63+genSlicesFromLen :: forall m a. (Monad m, Unbox a)
64+ => Int -- ^ from index
65+ -> Int -- ^ length of the slice
66+ -> Unfold m (Array a) (Int, Int)
67+genSlicesFromLen from len =
68+ let fromThenTo n = (from, from + len, n - 1)
69+ mkSlice n i = return (i, min len (n - i))
70+ in Unfold.lmap length
71+ $ Unfold.mapM2 mkSlice
72+ $ Unfold.lmap fromThenTo Unfold.enumerateFromThenTo
73+
74+-- | Generate a stream of slices of specified length from an array, starting
75+-- from the supplied array index. The last slice may be shorter than the
76+-- requested length depending on the array length.
77+--
78+-- /Pre-release/
79+{-# INLINE getSlicesFromLen #-}
80+getSlicesFromLen :: forall m a. (Monad m, Unbox a)
81+ => Int -- ^ from index
82+ -> Int -- ^ length of the slice
83+ -> Unfold m (Array a) (Array a)
84+getSlicesFromLen from len =
85+ let mkSlice arr (i, n) = return $ getSliceUnsafe i n arr
86+ in Unfold.mapM2 mkSlice (genSlicesFromLen from len)
87+
88+-- | Create an 'Array' from a stream. This is useful when we want to create a
89+-- single array from a stream of unknown size. 'writeN' is at least twice
90+-- as efficient when the size is already known.
91+--
92+-- Note that if the input stream is too large memory allocation for the array
93+-- may fail. When the stream size is not known, `arraysOf` followed by
94+-- processing of indvidual arrays in the resulting stream should be preferred.
95+--
96+-- /Pre-release/
97+{-# INLINE fromStream #-}
98+fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
99+fromStream = fromStreamD
100+-- fromStream (Stream m) = P.fold write m
core/src/Streamly/Internal/Data/Array/Mut/Stream.hsnew file+323
@@ -0,0 +1,323 @@
1+-- |
2+-- Module : Streamly.Internal.Data.Array.Mut.Stream
3+-- Copyright : (c) 2019 Composewell Technologies
4+-- License : BSD3-3-Clause
5+-- Maintainer : streamly@composewell.com
6+-- Stability : experimental
7+-- Portability : GHC
8+--
9+-- Combinators to efficiently manipulate streams of mutable arrays.
10+--
11+module Streamly.Internal.Data.Array.Mut.Stream
12+ (
13+ -- * Generation
14+ arraysOf
15+
16+ -- * Compaction
17+ , packArraysChunksOf
18+ , SpliceState (..)
19+ , lpackArraysChunksOf
20+ , compact
21+ , compactLE
22+ , compactEQ
23+ , compactGE
24+ )
25+where
26+
27+#include "inline.hs"
28+#include "ArrayMacros.h"
29+
30+import Control.Monad.IO.Class (MonadIO(..))
31+import Control.Monad (when)
32+import Data.Bifunctor (first)
33+import Data.Proxy (Proxy(..))
34+import Streamly.Internal.Data.Unboxed (Unbox, sizeOf)
35+import Streamly.Internal.Data.Array.Mut.Type (Array(..))
36+import Streamly.Internal.Data.Fold.Type (Fold(..))
37+import Streamly.Internal.Data.Parser (ParseError)
38+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
39+import Streamly.Internal.Data.Tuple.Strict (Tuple'(..))
40+
41+import qualified Streamly.Internal.Data.Array.Mut.Type as MArray
42+import qualified Streamly.Internal.Data.Fold.Type as FL
43+import qualified Streamly.Internal.Data.Stream.StreamD as D
44+import qualified Streamly.Internal.Data.Parser.ParserD as ParserD
45+
46+-- | @arraysOf n stream@ groups the elements in the input stream into arrays of
47+-- @n@ elements each.
48+--
49+-- Same as the following but may be more efficient:
50+--
51+-- > arraysOf n = Stream.foldMany (MArray.writeN n)
52+--
53+-- /Pre-release/
54+{-# INLINE arraysOf #-}
55+arraysOf :: (MonadIO m, Unbox a)
56+ => Int -> Stream m a -> Stream m (Array a)
57+arraysOf = MArray.arraysOf
58+
59+-------------------------------------------------------------------------------
60+-- Compact
61+-------------------------------------------------------------------------------
62+
63+data SpliceState s arr
64+ = SpliceInitial s
65+ | SpliceBuffering s arr
66+ | SpliceYielding arr (SpliceState s arr)
67+ | SpliceFinish
68+
69+-- XXX This can be removed once compactLEFold/compactLE are implemented.
70+--
71+-- | This mutates the first array (if it has space) to append values from the
72+-- second one. This would work for immutable arrays as well because an
73+-- immutable array never has space so a new array is allocated instead of
74+-- mutating it.
75+--
76+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
77+-- maximum specified size. Note that if a single array is bigger than the
78+-- specified size we do not split it to fit. When we coalesce multiple arrays
79+-- if the size would exceed the specified size we do not coalesce therefore the
80+-- actual array size may be less than the specified chunk size.
81+--
82+-- @since 0.7.0
83+{-# INLINE_NORMAL packArraysChunksOf #-}
84+packArraysChunksOf :: (MonadIO m, Unbox a)
85+ => Int -> D.Stream m (Array a) -> D.Stream m (Array a)
86+packArraysChunksOf n (D.Stream step state) =
87+ D.Stream step' (SpliceInitial state)
88+
89+ where
90+
91+ {-# INLINE_LATE step' #-}
92+ step' gst (SpliceInitial st) = do
93+ when (n <= 0) $
94+ -- XXX we can pass the module string from the higher level API
95+ error $ "Streamly.Internal.Data.Array.Mut.Type.packArraysChunksOf: the size of "
96+ ++ "arrays [" ++ show n ++ "] must be a natural number"
97+ r <- step gst st
98+ case r of
99+ D.Yield arr s -> return $
100+ let len = MArray.byteLength arr
101+ in if len >= n
102+ then D.Skip (SpliceYielding arr (SpliceInitial s))
103+ else D.Skip (SpliceBuffering s arr)
104+ D.Skip s -> return $ D.Skip (SpliceInitial s)
105+ D.Stop -> return D.Stop
106+
107+ step' gst (SpliceBuffering st buf) = do
108+ r <- step gst st
109+ case r of
110+ D.Yield arr s -> do
111+ let len = MArray.byteLength buf + MArray.byteLength arr
112+ if len > n
113+ then return $
114+ D.Skip (SpliceYielding buf (SpliceBuffering s arr))
115+ else do
116+ buf' <- if MArray.byteCapacity buf < n
117+ then liftIO $ MArray.realloc n buf
118+ else return buf
119+ buf'' <- MArray.splice buf' arr
120+ return $ D.Skip (SpliceBuffering s buf'')
121+ D.Skip s -> return $ D.Skip (SpliceBuffering s buf)
122+ D.Stop -> return $ D.Skip (SpliceYielding buf SpliceFinish)
123+
124+ step' _ SpliceFinish = return D.Stop
125+
126+ step' _ (SpliceYielding arr next) = return $ D.Yield arr next
127+
128+-- XXX Remove this once compactLEFold is implemented
129+-- lpackArraysChunksOf = Fold.many compactLEFold
130+--
131+{-# INLINE_NORMAL lpackArraysChunksOf #-}
132+lpackArraysChunksOf :: (MonadIO m, Unbox a)
133+ => Int -> Fold m (Array a) () -> Fold m (Array a) ()
134+lpackArraysChunksOf n (Fold step1 initial1 extract1) =
135+ Fold step initial extract
136+
137+ where
138+
139+ initial = do
140+ when (n <= 0) $
141+ -- XXX we can pass the module string from the higher level API
142+ error $ "Streamly.Internal.Data.Array.Mut.Type.packArraysChunksOf: the size of "
143+ ++ "arrays [" ++ show n ++ "] must be a natural number"
144+
145+ r <- initial1
146+ return $ first (Tuple' Nothing) r
147+
148+ extract (Tuple' Nothing r1) = extract1 r1
149+ extract (Tuple' (Just buf) r1) = do
150+ r <- step1 r1 buf
151+ case r of
152+ FL.Partial rr -> extract1 rr
153+ FL.Done _ -> return ()
154+
155+ step (Tuple' Nothing r1) arr =
156+ let len = MArray.byteLength arr
157+ in if len >= n
158+ then do
159+ r <- step1 r1 arr
160+ case r of
161+ FL.Done _ -> return $ FL.Done ()
162+ FL.Partial s -> do
163+ extract1 s
164+ res <- initial1
165+ return $ first (Tuple' Nothing) res
166+ else return $ FL.Partial $ Tuple' (Just arr) r1
167+
168+ step (Tuple' (Just buf) r1) arr = do
169+ let len = MArray.byteLength buf + MArray.byteLength arr
170+ buf' <- if MArray.byteCapacity buf < len
171+ then liftIO $ MArray.realloc (max n len) buf
172+ else return buf
173+ buf'' <- MArray.splice buf' arr
174+
175+ -- XXX this is common in both the equations of step
176+ if len >= n
177+ then do
178+ r <- step1 r1 buf''
179+ case r of
180+ FL.Done _ -> return $ FL.Done ()
181+ FL.Partial s -> do
182+ extract1 s
183+ res <- initial1
184+ return $ first (Tuple' Nothing) res
185+ else return $ FL.Partial $ Tuple' (Just buf'') r1
186+
187+-- XXX Same as compactLE, to be removed once that is implemented.
188+--
189+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
190+-- maximum specified size in bytes.
191+--
192+-- /Internal/
193+{-# INLINE compact #-}
194+compact :: (MonadIO m, Unbox a)
195+ => Int -> Stream m (Array a) -> Stream m (Array a)
196+compact = packArraysChunksOf
197+
198+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
199+-- maximum specified size. Note that if a single array is bigger than the
200+-- specified size we do not split it to fit. When we coalesce multiple arrays
201+-- if the size would exceed the specified size we do not coalesce therefore the
202+-- actual array size may be less than the specified chunk size.
203+--
204+-- /Internal/
205+{-# INLINE_NORMAL compactLEParserD #-}
206+compactLEParserD ::
207+ forall m a. (MonadIO m, Unbox a)
208+ => Int -> ParserD.Parser (Array a) m (Array a)
209+compactLEParserD n = ParserD.Parser step initial extract
210+
211+ where
212+
213+ nBytes = n * SIZE_OF(a)
214+
215+ initial =
216+ return
217+ $ if n <= 0
218+ then error
219+ $ functionPath
220+ ++ ": the size of arrays ["
221+ ++ show n ++ "] must be a natural number"
222+ else ParserD.IPartial Nothing
223+
224+ step Nothing arr =
225+ return
226+ $ let len = MArray.byteLength arr
227+ in if len >= nBytes
228+ then ParserD.Done 0 arr
229+ else ParserD.Partial 0 (Just arr)
230+ step (Just buf) arr =
231+ let len = MArray.byteLength buf + MArray.byteLength arr
232+ in if len > nBytes
233+ then return $ ParserD.Done 1 buf
234+ else do
235+ buf1 <-
236+ if MArray.byteCapacity buf < nBytes
237+ then liftIO $ MArray.realloc nBytes buf
238+ else return buf
239+ buf2 <- MArray.splice buf1 arr
240+ return $ ParserD.Partial 0 (Just buf2)
241+
242+ extract Nothing = return $ ParserD.Done 0 MArray.nil
243+ extract (Just buf) = return $ ParserD.Done 0 buf
244+
245+ functionPath =
246+ "Streamly.Internal.Data.Array.Mut.Stream.compactLEParserD"
247+
248+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
249+-- minimum specified size. Note that if all the arrays in the stream together
250+-- are smaller than the specified size the resulting array will be smaller than
251+-- the specified size. When we coalesce multiple arrays if the size would exceed
252+-- the specified size we stop coalescing further.
253+--
254+-- /Internal/
255+{-# INLINE_NORMAL compactGEFold #-}
256+compactGEFold ::
257+ forall m a. (MonadIO m, Unbox a)
258+ => Int -> FL.Fold m (Array a) (Array a)
259+compactGEFold n = Fold step initial extract
260+
261+ where
262+
263+ nBytes = n * SIZE_OF(a)
264+
265+ initial =
266+ return
267+ $ if n < 0
268+ then error
269+ $ functionPath
270+ ++ ": the size of arrays ["
271+ ++ show n ++ "] must be a natural number"
272+ else FL.Partial Nothing
273+
274+ step Nothing arr =
275+ return
276+ $ let len = MArray.byteLength arr
277+ in if len >= nBytes
278+ then FL.Done arr
279+ else FL.Partial (Just arr)
280+ step (Just buf) arr = do
281+ let len = MArray.byteLength buf + MArray.byteLength arr
282+ buf1 <-
283+ if MArray.byteCapacity buf < len
284+ then liftIO $ MArray.realloc (max len nBytes) buf
285+ else return buf
286+ buf2 <- MArray.splice buf1 arr
287+ if len >= n
288+ then return $ FL.Done buf2
289+ else return $ FL.Partial (Just buf2)
290+
291+ extract Nothing = return MArray.nil
292+ extract (Just buf) = return buf
293+
294+ functionPath =
295+ "Streamly.Internal.Data.Array.Mut.Stream.compactGEFold"
296+
297+-- | Coalesce adjacent arrays in incoming stream to form bigger arrays of a
298+-- maximum specified size in bytes.
299+--
300+-- /Internal/
301+compactLE :: (MonadIO m, Unbox a) =>
302+ Int -> Stream m (Array a) -> Stream m (Either ParseError (Array a))
303+compactLE n = D.parseManyD (compactLEParserD n)
304+
305+-- | Like 'compactLE' but generates arrays of exactly equal to the size
306+-- specified except for the last array in the stream which could be shorter.
307+--
308+-- /Unimplemented/
309+{-# INLINE compactEQ #-}
310+compactEQ :: -- (MonadIO m, Unbox a) =>
311+ Int -> Stream m (Array a) -> Stream m (Array a)
312+compactEQ _n _xs = undefined
313+ -- IsStream.fromStreamD $ D.foldMany (compactEQFold n) (IsStream.toStreamD xs)
314+
315+-- | Like 'compactLE' but generates arrays of size greater than or equal to the
316+-- specified except for the last array in the stream which could be shorter.
317+--
318+-- /Internal/
319+{-# INLINE compactGE #-}
320+compactGE ::
321+ (MonadIO m, Unbox a)
322+ => Int -> Stream m (Array a) -> Stream m (Array a)
323+compactGE n = D.foldMany (compactGEFold n)
core/src/Streamly/Internal/Data/Array/Mut/Type.hsnew file+2350
@@ -0,0 +1,2350 @@
1+{-# LANGUAGE UnboxedTuples #-}
2+-- |
3+-- Module : Streamly.Internal.Data.Array.Mut.Type
4+-- Copyright : (c) 2020 Composewell Technologies
5+-- License : BSD3-3-Clause
6+-- Maintainer : streamly@composewell.com
7+-- Stability : experimental
8+-- Portability : GHC
9+--
10+-- Mutable arrays and file system files are quite similar, they can grow and
11+-- their content is mutable. Therefore, both have similar APIs as well. We
12+-- strive to keep the API consistent for both. Ideally, you should be able to
13+-- replace one with another with little changes to the code.
14+
15+module Streamly.Internal.Data.Array.Mut.Type
16+ (
17+ -- * Type
18+ -- $arrayNotes
19+ Array (..)
20+ , MutableByteArray
21+ , touch
22+ , pin
23+ , unpin
24+
25+ -- * Constructing and Writing
26+ -- ** Construction
27+ , nil
28+
29+ -- *** Uninitialized Arrays
30+ , newPinned
31+ , newPinnedBytes
32+ , newAlignedPinned
33+ , new
34+ , newArrayWith
35+
36+ -- *** Initialized Arrays
37+ , withNewArrayUnsafe
38+
39+ -- *** From streams
40+ , ArrayUnsafe (..)
41+ , writeNWithUnsafe
42+ , writeNWith
43+ , writeNUnsafe
44+ , writeN
45+ , writeNAligned
46+
47+ , writeWith
48+ , write
49+
50+ , writeRevN
51+ -- , writeRev
52+
53+ -- ** From containers
54+ , fromListN
55+ , fromList
56+ , fromListRevN
57+ , fromListRev
58+ , fromStreamDN
59+ , fromStreamD
60+
61+ -- * Random writes
62+ , putIndex
63+ , putIndexUnsafe
64+ , putIndices
65+ -- , putFromThenTo
66+ -- , putFrom -- start writing at the given position
67+ -- , putUpto -- write from beginning up to the given position
68+ -- , putFromTo
69+ -- , putFromRev
70+ -- , putUptoRev
71+ , modifyIndexUnsafe
72+ , modifyIndex
73+ , modifyIndices
74+ , modify
75+ , swapIndices
76+ , unsafeSwapIndices
77+
78+ -- * Growing and Shrinking
79+ -- Arrays grow only at the end, though it is possible to grow on both sides
80+ -- and therefore have a cons as well as snoc. But that will require two
81+ -- bounds in the array representation.
82+
83+ -- ** Appending elements
84+ , snocWith
85+ , snoc
86+ , snocLinear
87+ , snocMay
88+ , snocUnsafe
89+
90+ -- ** Appending streams
91+ , writeAppendNUnsafe
92+ , writeAppendN
93+ , writeAppendWith
94+ , writeAppend
95+
96+ -- * Eliminating and Reading
97+
98+ -- ** To streams
99+ , reader
100+ , readerRevWith
101+ , readerRev
102+
103+ -- ** To containers
104+ , toStreamDWith
105+ , toStreamDRevWith
106+ , toStreamKWith
107+ , toStreamKRevWith
108+ , toStreamD
109+ , toStreamDRev
110+ , toStreamK
111+ , toStreamKRev
112+ , toList
113+
114+ -- experimental
115+ , producerWith
116+ , producer
117+
118+ -- ** Random reads
119+ , getIndex
120+ , getIndexUnsafe
121+ , getIndices
122+ , getIndicesD
123+ -- , getFromThenTo
124+ , getIndexRev
125+
126+ -- * Memory Management
127+ , blockSize
128+ , arrayChunkBytes
129+ , allocBytesToElemCount
130+ , realloc
131+ , resize
132+ , resizeExp
133+ , rightSize
134+
135+ -- * Size
136+ , length
137+ , byteLength
138+ -- , capacity
139+ , byteCapacity
140+ , bytesFree
141+
142+ -- * In-place Mutation Algorithms
143+ , strip
144+ , reverse
145+ , permute
146+ , partitionBy
147+ , shuffleBy
148+ , divideBy
149+ , mergeBy
150+ , bubble
151+
152+ -- * Casting
153+ , cast
154+ , castUnsafe
155+ , asBytes
156+ , asPtrUnsafe
157+
158+ -- * Folding
159+ , foldl'
160+ , foldr
161+ , cmp
162+
163+ -- * Arrays of arrays
164+ -- We can add dimensionality parameter to the array type to get
165+ -- multidimensional arrays. Multidimensional arrays would just be a
166+ -- convenience wrapper on top of single dimensional arrays.
167+
168+ -- | Operations dealing with multiple arrays, streams of arrays or
169+ -- multidimensional array representations.
170+
171+ -- ** Construct from streams
172+ , arraysOf
173+ , arrayStreamKFromStreamD
174+ , writeChunks
175+
176+ -- ** Eliminate to streams
177+ , flattenArrays
178+ , flattenArraysRev
179+ , fromArrayStreamK
180+
181+ -- ** Construct from arrays
182+ -- get chunks without copying
183+ , getSliceUnsafe
184+ , getSlice
185+ -- , getSlicesFromLenN
186+ , splitAt -- XXX should be able to express using getSlice
187+ , breakOn
188+
189+ -- ** Appending arrays
190+ , spliceCopy
191+ , spliceWith
192+ , splice
193+ , spliceExp
194+ , spliceUnsafe
195+ , putSliceUnsafe
196+ -- , putSlice
197+ -- , appendSlice
198+ -- , appendSliceFrom
199+
200+ -- * Utilities
201+ , roundUpToPower2
202+ , memcpy
203+ , memcmp
204+ , c_memchr
205+ )
206+where
207+
208+-- Notes:
209+-- When we use a purely lazy Monad like Identity, we need to force ordering of
210+-- some actions for correctness.
211+
212+#include "assert.hs"
213+#include "inline.hs"
214+#include "ArrayMacros.h"
215+#include "MachDeps.h"
216+
217+import Control.Monad (when, void)
218+import Control.Monad.IO.Class (MonadIO(..))
219+import Data.Bits (shiftR, (.|.), (.&.))
220+import Data.Proxy (Proxy(..))
221+import Data.Word (Word8)
222+import Foreign.C.Types (CSize(..), CInt(..))
223+import Foreign.Ptr (plusPtr, minusPtr, nullPtr)
224+import Streamly.Internal.Data.Unboxed
225+ ( MutableByteArray(..)
226+ , Unbox
227+ , getMutableByteArray#
228+ , peekWith
229+ , pokeWith
230+ , sizeOf
231+ , touch
232+ )
233+import GHC.Base
234+ ( IO(..)
235+ , Int(..)
236+ , byteArrayContents#
237+ , compareByteArrays#
238+ , copyMutableByteArray#
239+ )
240+import GHC.Base (noinline)
241+import GHC.Exts (unsafeCoerce#)
242+import GHC.Ptr (Ptr(..))
243+
244+import Streamly.Internal.Data.Fold.Type (Fold(..))
245+import Streamly.Internal.Data.Producer.Type (Producer (..))
246+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
247+import Streamly.Internal.Data.SVar.Type (adaptState, defState)
248+import Streamly.Internal.Data.Unfold.Type (Unfold(..))
249+import Streamly.Internal.System.IO (arrayPayloadSize, defaultChunkSize)
250+
251+import qualified Streamly.Internal.Data.Fold.Type as FL
252+import qualified Streamly.Internal.Data.Producer as Producer
253+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
254+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
255+import qualified Streamly.Internal.Data.Unboxed as Unboxed
256+import qualified Prelude
257+
258+import Prelude hiding
259+ (length, foldr, read, unlines, splitAt, reverse, truncate)
260+
261+-- $setup
262+-- >>> :m
263+-- >>> import qualified Streamly.Internal.Data.Array.Mut.Type as Array
264+-- >>> import qualified Streamly.Internal.Data.Stream as Stream
265+-- >>> import qualified Streamly.Internal.Data.Stream.StreamD as StreamD
266+-- >>> import qualified Streamly.Internal.Data.Fold as Fold
267+-- >>> import qualified Streamly.Internal.Data.Fold.Type as Fold
268+
269+-------------------------------------------------------------------------------
270+-- Foreign helpers
271+-------------------------------------------------------------------------------
272+
273+foreign import ccall unsafe "string.h memcpy" c_memcpy
274+ :: Ptr Word8 -> Ptr Word8 -> CSize -> IO (Ptr Word8)
275+
276+foreign import ccall unsafe "string.h memchr" c_memchr
277+ :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)
278+
279+foreign import ccall unsafe "string.h memcmp" c_memcmp
280+ :: Ptr Word8 -> Ptr Word8 -> CSize -> IO CInt
281+
282+-- | Given an 'Unboxed' type (unused first arg) and a number of bytes, return
283+-- how many elements of that type will completely fit in those bytes.
284+--
285+{-# INLINE bytesToElemCount #-}
286+bytesToElemCount :: forall a. Unbox a => a -> Int -> Int
287+bytesToElemCount _ n = n `div` SIZE_OF(a)
288+
289+-- XXX we are converting Int to CSize
290+memcpy :: Ptr Word8 -> Ptr Word8 -> Int -> IO ()
291+memcpy dst src len = void (c_memcpy dst src (fromIntegral len))
292+
293+-- XXX we are converting Int to CSize
294+-- return True if the memory locations have identical contents
295+{-# INLINE memcmp #-}
296+memcmp :: Ptr Word8 -> Ptr Word8 -> Int -> IO Bool
297+memcmp p1 p2 len = do
298+ r <- c_memcmp p1 p2 (fromIntegral len)
299+ return $ r == 0
300+
301+-------------------------------------------------------------------------------
302+-- Array Data Type
303+-------------------------------------------------------------------------------
304+
305+-- $arrayNotes
306+--
307+-- We can use an 'Unboxed' constraint in the Array type and the constraint can
308+-- be automatically provided to a function that pattern matches on the Array
309+-- type. However, it has huge performance cost, so we do not use it.
310+-- Investigate a GHC improvement possiblity.
311+
312+-- XXX Rename the fields to better names.
313+
314+-- | An unboxed, pinned mutable array. An array is created with a given length
315+-- and capacity. Length is the number of valid elements in the array. Capacity
316+-- is the maximum number of elements that the array can be expanded to without
317+-- having to reallocate the memory.
318+--
319+-- The elements in the array can be mutated in-place without changing the
320+-- reference (constructor). However, the length of the array cannot be mutated
321+-- in-place. A new array reference is generated when the length changes. When
322+-- the length is increased (upto the maximum reserved capacity of the array),
323+-- the array is not reallocated and the new reference uses the same underlying
324+-- memory as the old one.
325+--
326+-- Several routines in this module allow the programmer to control the capacity
327+-- of the array. The programmer can control the trade-off between memory usage
328+-- and performance impact due to reallocations when growing or shrinking the
329+-- array.
330+--
331+data Array a =
332+#ifdef DEVBUILD
333+ Unbox a =>
334+#endif
335+ -- The array is a range into arrContents. arrContents may be a superset of
336+ -- the slice represented by the array. All offsets are in bytes.
337+ Array
338+ { arrContents :: {-# UNPACK #-} !MutableByteArray
339+ , arrStart :: {-# UNPACK #-} !Int -- ^ index into arrContents
340+ , arrEnd :: {-# UNPACK #-} !Int -- ^ index into arrContents
341+ -- Represents the first invalid index of
342+ -- the array.
343+ , arrBound :: {-# UNPACK #-} !Int -- ^ first invalid index of arrContents.
344+ }
345+
346+-------------------------------------------------------------------------------
347+-- Pinning & Unpinning
348+-------------------------------------------------------------------------------
349+
350+{-# INLINE pin #-}
351+pin :: Array a -> IO (Array a)
352+pin arr@Array{..} = do
353+ contents <- Unboxed.pin arrContents
354+ return $ arr {arrContents = contents}
355+
356+{-# INLINE unpin #-}
357+unpin :: Array a -> IO (Array a)
358+unpin arr@Array{..} = do
359+ contents <- Unboxed.unpin arrContents
360+ return $ arr {arrContents = contents}
361+
362+-------------------------------------------------------------------------------
363+-- Construction
364+-------------------------------------------------------------------------------
365+
366+-- XXX Change the names to use "new" instead of "newArray". That way we can use
367+-- the same names for managed file system objects as well. For unmanaged ones
368+-- we can use open/create etc as usual.
369+--
370+-- A new array is similar to "touch" creating a zero length file. An mmapped
371+-- array would be similar to a sparse file with holes. TBD: support mmapped
372+-- files and arrays.
373+
374+-- GHC always guarantees word-aligned memory, alignment is important only when
375+-- we need more than that. See stg_newAlignedPinnedByteArrayzh and
376+-- allocatePinned in GHC source.
377+
378+-- | @newArrayWith allocator alignment count@ allocates a new array of zero
379+-- length and with a capacity to hold @count@ elements, using @allocator
380+-- size alignment@ as the memory allocator function.
381+--
382+-- Alignment must be greater than or equal to machine word size and a power of
383+-- 2.
384+--
385+-- Alignment is ignored if the allocator allocates unpinned memory.
386+--
387+-- /Pre-release/
388+{-# INLINE newArrayWith #-}
389+newArrayWith :: forall m a. (MonadIO m, Unbox a)
390+ => (Int -> Int -> m MutableByteArray) -> Int -> Int -> m (Array a)
391+newArrayWith alloc alignSize count = do
392+ let size = max (count * SIZE_OF(a)) 0
393+ contents <- alloc size alignSize
394+ return $ Array
395+ { arrContents = contents
396+ , arrStart = 0
397+ , arrEnd = 0
398+ , arrBound = size
399+ }
400+
401+nil ::
402+#ifdef DEVBUILD
403+ Unbox a =>
404+#endif
405+ Array a
406+nil = Array Unboxed.nil 0 0 0
407+
408+
409+-- | Allocates a pinned empty array that can hold 'count' items. The memory of
410+-- the array is uninitialized and the allocation is aligned as per the
411+-- 'Unboxed' instance of the type.
412+--
413+-- /Pre-release/
414+{-# INLINE newPinnedBytes #-}
415+newPinnedBytes :: MonadIO m =>
416+#ifdef DEVBUILD
417+ Unbox a =>
418+#endif
419+ Int -> m (Array a)
420+newPinnedBytes bytes = do
421+ contents <- liftIO $ Unboxed.newPinnedBytes bytes
422+ return $ Array
423+ { arrContents = contents
424+ , arrStart = 0
425+ , arrEnd = 0
426+ , arrBound = bytes
427+ }
428+
429+-- | Like 'newArrayWith' but using an allocator is a pinned memory allocator and
430+-- the alignment is dictated by the 'Unboxed' instance of the type.
431+--
432+-- /Internal/
433+{-# INLINE newAlignedPinned #-}
434+newAlignedPinned :: (MonadIO m, Unbox a) => Int -> Int -> m (Array a)
435+newAlignedPinned =
436+ newArrayWith (\s a -> liftIO $ Unboxed.newAlignedPinnedBytes s a)
437+
438+-- XXX can unaligned allocation be more efficient when alignment is not needed?
439+--
440+-- | Allocates an empty pinned array that can hold 'count' items. The memory of
441+-- the array is uninitialized and the allocation is aligned as per the 'Unboxed'
442+-- instance of the type.
443+--
444+{-# INLINE newPinned #-}
445+newPinned :: forall m a. (MonadIO m, Unbox a) => Int -> m (Array a)
446+newPinned =
447+ newArrayWith
448+ (\s _ -> liftIO $ Unboxed.newPinnedBytes s)
449+ (error "newPinned: alignSize is not used")
450+
451+-- | Allocates an empty unpinned array that can hold 'count' items. The memory
452+-- of the array is uninitialized.
453+--
454+{-# INLINE new #-}
455+new :: (MonadIO m, Unbox a) => Int -> m (Array a)
456+new =
457+ newArrayWith
458+ (\s _ -> liftIO $ Unboxed.newUnpinnedBytes s)
459+ (error "new: alignment is not used in unpinned arrays.")
460+
461+-- | Allocate an Array of the given size and run an IO action passing the array
462+-- start pointer.
463+--
464+-- /Internal/
465+{-# INLINE withNewArrayUnsafe #-}
466+withNewArrayUnsafe ::
467+ (MonadIO m, Unbox a) => Int -> (Ptr a -> m ()) -> m (Array a)
468+withNewArrayUnsafe count f = do
469+ arr <- newPinned count
470+ asPtrUnsafe arr
471+ $ \p -> f p >> return arr
472+
473+-------------------------------------------------------------------------------
474+-- Random writes
475+-------------------------------------------------------------------------------
476+
477+-- | Write the given element to the given index of the array. Does not check if
478+-- the index is out of bounds of the array.
479+--
480+-- /Pre-release/
481+{-# INLINE putIndexUnsafe #-}
482+putIndexUnsafe :: forall m a. (MonadIO m, Unbox a)
483+ => Int -> Array a -> a -> m ()
484+putIndexUnsafe i Array{..} x = do
485+ let index = INDEX_OF(arrStart, i, a)
486+ assert (i >= 0 && INDEX_VALID(index, arrEnd, a)) (return ())
487+ liftIO $ pokeWith arrContents index x
488+
489+invalidIndex :: String -> Int -> a
490+invalidIndex label i =
491+ error $ label ++ ": invalid array index " ++ show i
492+
493+-- | /O(1)/ Write the given element at the given index in the array.
494+-- Performs in-place mutation of the array.
495+--
496+-- >>> putIndex ix arr val = Array.modifyIndex ix arr (const (val, ()))
497+-- >>> f = Array.putIndices
498+-- >>> putIndex ix arr val = Stream.fold (f arr) (Stream.fromPure (ix, val))
499+--
500+{-# INLINE putIndex #-}
501+putIndex :: forall m a. (MonadIO m, Unbox a) => Int -> Array a -> a -> m ()
502+putIndex i Array{..} x = do
503+ let index = INDEX_OF(arrStart,i,a)
504+ if i >= 0 && INDEX_VALID(index,arrEnd,a)
505+ then liftIO $ pokeWith arrContents index x
506+ else invalidIndex "putIndex" i
507+
508+-- | Write an input stream of (index, value) pairs to an array. Throws an
509+-- error if any index is out of bounds.
510+--
511+-- /Pre-release/
512+{-# INLINE putIndices #-}
513+putIndices :: forall m a. (MonadIO m, Unbox a)
514+ => Array a -> Fold m (Int, a) ()
515+putIndices arr = FL.foldlM' step (return ())
516+
517+ where
518+
519+ step () (i, x) = liftIO (putIndex i arr x)
520+
521+-- | Modify a given index of an array using a modifier function.
522+--
523+-- /Pre-release/
524+modifyIndexUnsafe :: forall m a b. (MonadIO m, Unbox a) =>
525+ Int -> Array a -> (a -> (a, b)) -> m b
526+modifyIndexUnsafe i Array{..} f = liftIO $ do
527+ let index = INDEX_OF(arrStart,i,a)
528+ assert (i >= 0 && INDEX_NEXT(index,a) <= arrEnd) (return ())
529+ r <- peekWith arrContents index
530+ let (x, res) = f r
531+ pokeWith arrContents index x
532+ return res
533+
534+-- | Modify a given index of an array using a modifier function.
535+--
536+-- /Pre-release/
537+modifyIndex :: forall m a b. (MonadIO m, Unbox a) =>
538+ Int -> Array a -> (a -> (a, b)) -> m b
539+modifyIndex i Array{..} f = do
540+ let index = INDEX_OF(arrStart,i,a)
541+ if i >= 0 && INDEX_VALID(index,arrEnd,a)
542+ then liftIO $ do
543+ r <- peekWith arrContents index
544+ let (x, res) = f r
545+ pokeWith arrContents index x
546+ return res
547+ else invalidIndex "modifyIndex" i
548+
549+
550+-- | Modify the array indices generated by the supplied stream.
551+--
552+-- /Pre-release/
553+{-# INLINE modifyIndices #-}
554+modifyIndices :: forall m a . (MonadIO m, Unbox a)
555+ => Array a -> (Int -> a -> a) -> Fold m Int ()
556+modifyIndices arr f = FL.foldlM' step initial
557+
558+ where
559+
560+ initial = return ()
561+
562+ step () i =
563+ let f1 x = (f i x, ())
564+ in modifyIndex i arr f1
565+
566+-- | Modify each element of an array using the supplied modifier function.
567+--
568+-- /Pre-release/
569+modify :: forall m a. (MonadIO m, Unbox a)
570+ => Array a -> (a -> a) -> m ()
571+modify Array{..} f = liftIO $
572+ go arrStart
573+
574+ where
575+
576+ go i =
577+ when (INDEX_VALID(i,arrEnd,a)) $ do
578+ r <- peekWith arrContents i
579+ pokeWith arrContents i (f r)
580+ go (INDEX_NEXT(i,a))
581+
582+-- XXX We could specify the number of bytes to swap instead of Proxy. Need
583+-- to ensure that the memory does not overlap.
584+{-# INLINE swapArrayByteIndices #-}
585+swapArrayByteIndices ::
586+ forall a. Unbox a
587+ => Proxy a
588+ -> MutableByteArray
589+ -> Int
590+ -> Int
591+ -> IO ()
592+swapArrayByteIndices _ arrContents i1 i2 = do
593+ r1 <- peekWith arrContents i1
594+ r2 <- peekWith arrContents i2
595+ pokeWith arrContents i1 (r2 :: a)
596+ pokeWith arrContents i2 (r1 :: a)
597+
598+-- | Swap the elements at two indices without validating the indices.
599+--
600+-- /Unsafe/: This could result in memory corruption if indices are not valid.
601+--
602+-- /Pre-release/
603+{-# INLINE unsafeSwapIndices #-}
604+unsafeSwapIndices :: forall m a. (MonadIO m, Unbox a)
605+ => Int -> Int -> Array a -> m ()
606+unsafeSwapIndices i1 i2 Array{..} = liftIO $ do
607+ let t1 = INDEX_OF(arrStart,i1,a)
608+ t2 = INDEX_OF(arrStart,i2,a)
609+ swapArrayByteIndices (Proxy :: Proxy a) arrContents t1 t2
610+
611+-- | Swap the elements at two indices.
612+--
613+-- /Pre-release/
614+swapIndices :: forall m a. (MonadIO m, Unbox a)
615+ => Int -> Int -> Array a -> m ()
616+swapIndices i1 i2 Array{..} = liftIO $ do
617+ let t1 = INDEX_OF(arrStart,i1,a)
618+ t2 = INDEX_OF(arrStart,i2,a)
619+ when (i1 < 0 || INDEX_INVALID(t1,arrEnd,a))
620+ $ invalidIndex "swapIndices" i1
621+ when (i2 < 0 || INDEX_INVALID(t2,arrEnd,a))
622+ $ invalidIndex "swapIndices" i2
623+ swapArrayByteIndices (Proxy :: Proxy a) arrContents t1 t2
624+
625+-------------------------------------------------------------------------------
626+-- Rounding
627+-------------------------------------------------------------------------------
628+
629+-- XXX Should we use bitshifts in calculations or it gets optimized by the
630+-- compiler/processor itself?
631+--
632+-- | The page or block size used by the GHC allocator. Allocator allocates at
633+-- least a block and then allocates smaller allocations from within a block.
634+blockSize :: Int
635+blockSize = 4 * 1024
636+
637+-- | Allocations larger than 'largeObjectThreshold' are in multiples of block
638+-- size and are always pinned. The space beyond the end of a large object up to
639+-- the end of the block is unused.
640+largeObjectThreshold :: Int
641+largeObjectThreshold = (blockSize * 8) `div` 10
642+
643+-- XXX Should be done only when we are using the GHC allocator.
644+-- | Round up an array larger than 'largeObjectThreshold' to use the whole
645+-- block.
646+{-# INLINE roundUpLargeArray #-}
647+roundUpLargeArray :: Int -> Int
648+roundUpLargeArray size =
649+ if size >= largeObjectThreshold
650+ then
651+ assert
652+ (blockSize /= 0 && ((blockSize .&. (blockSize - 1)) == 0))
653+ ((size + blockSize - 1) .&. negate blockSize)
654+ else size
655+
656+{-# INLINE isPower2 #-}
657+isPower2 :: Int -> Bool
658+isPower2 n = n .&. (n - 1) == 0
659+
660+{-# INLINE roundUpToPower2 #-}
661+roundUpToPower2 :: Int -> Int
662+roundUpToPower2 n =
663+#if WORD_SIZE_IN_BITS == 64
664+ 1 + z6
665+#else
666+ 1 + z5
667+#endif
668+
669+ where
670+
671+ z0 = n - 1
672+ z1 = z0 .|. z0 `shiftR` 1
673+ z2 = z1 .|. z1 `shiftR` 2
674+ z3 = z2 .|. z2 `shiftR` 4
675+ z4 = z3 .|. z3 `shiftR` 8
676+ z5 = z4 .|. z4 `shiftR` 16
677+ z6 = z5 .|. z5 `shiftR` 32
678+
679+-- | @allocBytesToBytes elem allocatedBytes@ returns the array size in bytes
680+-- such that the real allocation is less than or equal to @allocatedBytes@,
681+-- unless @allocatedBytes@ is less than the size of one array element in which
682+-- case it returns one element's size.
683+--
684+{-# INLINE allocBytesToBytes #-}
685+allocBytesToBytes :: forall a. Unbox a => a -> Int -> Int
686+allocBytesToBytes _ n = max (arrayPayloadSize n) (SIZE_OF(a))
687+
688+-- | Given an 'Unboxed' type (unused first arg) and real allocation size
689+-- (including overhead), return how many elements of that type will completely
690+-- fit in it, returns at least 1.
691+--
692+{-# INLINE allocBytesToElemCount #-}
693+allocBytesToElemCount :: Unbox a => a -> Int -> Int
694+allocBytesToElemCount x bytes =
695+ let n = bytesToElemCount x (allocBytesToBytes x bytes)
696+ in assert (n >= 1) n
697+
698+-- | The default chunk size by which the array creation routines increase the
699+-- size of the array when the array is grown linearly.
700+arrayChunkBytes :: Int
701+arrayChunkBytes = 1024
702+
703+-------------------------------------------------------------------------------
704+-- Resizing
705+-------------------------------------------------------------------------------
706+
707+-- | Round the second argument down to multiples of the first argument.
708+{-# INLINE roundDownTo #-}
709+roundDownTo :: Int -> Int -> Int
710+roundDownTo elemSize size = size - (size `mod` elemSize)
711+
712+-- XXX See if resizing can be implemented by reading the old array as a stream
713+-- and then using writeN to the new array.
714+--
715+-- NOTE: we are passing elemSize explicitly to avoid an Unboxed constraint.
716+-- Since this is not inlined Unboxed consrraint leads to dictionary passing
717+-- which complicates some inspection tests.
718+--
719+{-# NOINLINE reallocExplicit #-}
720+reallocExplicit :: Int -> Int -> Array a -> IO (Array a)
721+reallocExplicit elemSize newCapacityInBytes Array{..} = do
722+ assertM(arrEnd <= arrBound)
723+
724+ -- Allocate new array
725+ let newCapMaxInBytes = roundUpLargeArray newCapacityInBytes
726+ contents <- Unboxed.newPinnedBytes newCapMaxInBytes
727+ let !(MutableByteArray mbarrFrom#) = arrContents
728+ !(MutableByteArray mbarrTo#) = contents
729+
730+ -- Copy old data
731+ let oldStart = arrStart
732+ !(I# oldStartInBytes#) = oldStart
733+ oldSizeInBytes = arrEnd - oldStart
734+ newCapInBytes = roundDownTo elemSize newCapMaxInBytes
735+ !newLenInBytes@(I# newLenInBytes#) = min oldSizeInBytes newCapInBytes
736+ assert (oldSizeInBytes `mod` elemSize == 0) (return ())
737+ assert (newLenInBytes >= 0) (return ())
738+ assert (newLenInBytes `mod` elemSize == 0) (return ())
739+ IO $ \s# -> (# copyMutableByteArray# mbarrFrom# oldStartInBytes#
740+ mbarrTo# 0# newLenInBytes# s#, () #)
741+
742+ return $ Array
743+ { arrStart = 0
744+ , arrContents = contents
745+ , arrEnd = newLenInBytes
746+ , arrBound = newCapInBytes
747+ }
748+
749+-- | @realloc newCapacity array@ reallocates the array to the specified
750+-- capacity in bytes.
751+--
752+-- If the new size is less than the original array the array gets truncated.
753+-- If the new size is not a multiple of array element size then it is rounded
754+-- down to multiples of array size. If the new size is more than
755+-- 'largeObjectThreshold' then it is rounded up to the block size (4K).
756+--
757+{-# INLINABLE realloc #-}
758+realloc :: forall m a. (MonadIO m, Unbox a) => Int -> Array a -> m (Array a)
759+realloc bytes arr = liftIO $ reallocExplicit (SIZE_OF(a)) bytes arr
760+
761+-- | @reallocWith label capSizer minIncrBytes array@. The label is used
762+-- in error messages and the capSizer is used to determine the capacity of the
763+-- new array in bytes given the current byte length of the array.
764+reallocWith :: forall m a. (MonadIO m , Unbox a) =>
765+ String
766+ -> (Int -> Int)
767+ -> Int
768+ -> Array a
769+ -> m (Array a)
770+reallocWith label capSizer minIncrBytes arr = do
771+ let oldSizeBytes = arrEnd arr - arrStart arr
772+ newCapBytes = capSizer oldSizeBytes
773+ newSizeBytes = oldSizeBytes + minIncrBytes
774+ safeCapBytes = max newCapBytes newSizeBytes
775+ assertM(safeCapBytes >= newSizeBytes || error (badSize newSizeBytes))
776+
777+ realloc safeCapBytes arr
778+
779+ where
780+
781+ badSize newSize =
782+ concat
783+ [ label
784+ , ": new array size (in bytes) is less than required size "
785+ , show newSize
786+ , ". Please check the sizing function passed."
787+ ]
788+
789+-- | @resize newCapacity array@ changes the total capacity of the array so that
790+-- it is enough to hold the specified number of elements. Nothing is done if
791+-- the specified capacity is less than the length of the array.
792+--
793+-- If the capacity is more than 'largeObjectThreshold' then it is rounded up to
794+-- the block size (4K).
795+--
796+-- /Pre-release/
797+{-# INLINE resize #-}
798+resize :: forall m a. (MonadIO m, Unbox a) =>
799+ Int -> Array a -> m (Array a)
800+resize nElems arr@Array{..} = do
801+ let req = SIZE_OF(a) * nElems
802+ len = arrEnd - arrStart
803+ if req < len
804+ then return arr
805+ else realloc req arr
806+
807+-- | Like 'resize' but if the byte capacity is more than 'largeObjectThreshold'
808+-- then it is rounded up to the closest power of 2.
809+--
810+-- /Pre-release/
811+{-# INLINE resizeExp #-}
812+resizeExp :: forall m a. (MonadIO m, Unbox a) =>
813+ Int -> Array a -> m (Array a)
814+resizeExp nElems arr@Array{..} = do
815+ let req = roundUpLargeArray (SIZE_OF(a) * nElems)
816+ req1 =
817+ if req > largeObjectThreshold
818+ then roundUpToPower2 req
819+ else req
820+ len = arrEnd - arrStart
821+ if req1 < len
822+ then return arr
823+ else realloc req1 arr
824+
825+-- | Resize the allocated memory to drop any reserved free space at the end of
826+-- the array and reallocate it to reduce wastage.
827+--
828+-- Up to 25% wastage is allowed to avoid reallocations. If the capacity is
829+-- more than 'largeObjectThreshold' then free space up to the 'blockSize' is
830+-- retained.
831+--
832+-- /Pre-release/
833+{-# INLINE rightSize #-}
834+rightSize :: forall m a. (MonadIO m, Unbox a) => Array a -> m (Array a)
835+rightSize arr@Array{..} = do
836+ assert (arrEnd <= arrBound) (return ())
837+ let start = arrStart
838+ len = arrEnd - start
839+ capacity = arrBound - start
840+ target = roundUpLargeArray len
841+ waste = arrBound - arrEnd
842+ assert (target >= len) (return ())
843+ assert (len `mod` SIZE_OF(a) == 0) (return ())
844+ -- We trade off some wastage (25%) to avoid reallocations and copying.
845+ if target < capacity && len < 3 * waste
846+ then realloc target arr
847+ else return arr
848+
849+-------------------------------------------------------------------------------
850+-- Snoc
851+-------------------------------------------------------------------------------
852+
853+-- XXX We can possibly use a smallMutableByteArray to hold the start, end,
854+-- bound pointers. Using fully mutable handle will ensure that we do not have
855+-- multiple references to the same array of different lengths lying around and
856+-- potentially misused. In that case "snoc" need not return a new array (snoc
857+-- :: Array a -> a -> m ()), it will just modify the old reference. The array
858+-- length will be mutable. This means the length function would also be
859+-- monadic. Mutable arrays would behave more like files that grow in that
860+-- case.
861+
862+-- | Snoc using a 'Ptr'. Low level reusable function.
863+--
864+-- /Internal/
865+{-# INLINE snocNewEnd #-}
866+snocNewEnd :: (MonadIO m, Unbox a) => Int -> Array a -> a -> m (Array a)
867+snocNewEnd newEnd arr@Array{..} x = liftIO $ do
868+ assert (newEnd <= arrBound) (return ())
869+ pokeWith arrContents arrEnd x
870+ return $ arr {arrEnd = newEnd}
871+
872+-- | Really really unsafe, appends the element into the first array, may
873+-- cause silent data corruption or if you are lucky a segfault if the first
874+-- array does not have enough space to append the element.
875+--
876+-- /Internal/
877+{-# INLINE snocUnsafe #-}
878+snocUnsafe :: forall m a. (MonadIO m, Unbox a) =>
879+ Array a -> a -> m (Array a)
880+snocUnsafe arr@Array{..} = snocNewEnd (INDEX_NEXT(arrEnd,a)) arr
881+
882+-- | Like 'snoc' but does not reallocate when pre-allocated array capacity
883+-- becomes full.
884+--
885+-- /Internal/
886+{-# INLINE snocMay #-}
887+snocMay :: forall m a. (MonadIO m, Unbox a) =>
888+ Array a -> a -> m (Maybe (Array a))
889+snocMay arr@Array{..} x = liftIO $ do
890+ let newEnd = INDEX_NEXT(arrEnd,a)
891+ if newEnd <= arrBound
892+ then Just <$> snocNewEnd newEnd arr x
893+ else return Nothing
894+
895+-- NOINLINE to move it out of the way and not pollute the instruction cache.
896+{-# NOINLINE snocWithRealloc #-}
897+snocWithRealloc :: forall m a. (MonadIO m, Unbox a) =>
898+ (Int -> Int)
899+ -> Array a
900+ -> a
901+ -> m (Array a)
902+snocWithRealloc sizer arr x = do
903+ arr1 <- liftIO $ reallocWith "snocWith" sizer (SIZE_OF(a)) arr
904+ snocUnsafe arr1 x
905+
906+-- | @snocWith sizer arr elem@ mutates @arr@ to append @elem@. The length of
907+-- the array increases by 1.
908+--
909+-- If there is no reserved space available in @arr@ it is reallocated to a size
910+-- in bytes determined by the @sizer oldSizeBytes@ function, where
911+-- @oldSizeBytes@ is the original size of the array in bytes.
912+--
913+-- If the new array size is more than 'largeObjectThreshold' we automatically
914+-- round it up to 'blockSize'.
915+--
916+-- Note that the returned array may be a mutated version of the original array.
917+--
918+-- /Pre-release/
919+{-# INLINE snocWith #-}
920+snocWith :: forall m a. (MonadIO m, Unbox a) =>
921+ (Int -> Int)
922+ -> Array a
923+ -> a
924+ -> m (Array a)
925+snocWith allocSize arr x = liftIO $ do
926+ let newEnd = INDEX_NEXT(arrEnd arr,a)
927+ if newEnd <= arrBound arr
928+ then snocNewEnd newEnd arr x
929+ else snocWithRealloc allocSize arr x
930+
931+-- | The array is mutated to append an additional element to it. If there
932+-- is no reserved space available in the array then it is reallocated to grow
933+-- it by 'arrayChunkBytes' rounded up to 'blockSize' when the size becomes more
934+-- than 'largeObjectThreshold'.
935+--
936+-- Note that the returned array may be a mutated version of the original array.
937+--
938+-- Performs O(n^2) copies to grow but is thrifty on memory.
939+--
940+-- /Pre-release/
941+{-# INLINE snocLinear #-}
942+snocLinear :: forall m a. (MonadIO m, Unbox a) => Array a -> a -> m (Array a)
943+snocLinear = snocWith (+ allocBytesToBytes (undefined :: a) arrayChunkBytes)
944+
945+-- | The array is mutated to append an additional element to it. If there is no
946+-- reserved space available in the array then it is reallocated to double the
947+-- original size.
948+--
949+-- This is useful to reduce allocations when appending unknown number of
950+-- elements.
951+--
952+-- Note that the returned array may be a mutated version of the original array.
953+--
954+-- >>> snoc = Array.snocWith (* 2)
955+--
956+-- Performs O(n * log n) copies to grow, but is liberal with memory allocation.
957+--
958+{-# INLINE snoc #-}
959+snoc :: forall m a. (MonadIO m, Unbox a) => Array a -> a -> m (Array a)
960+snoc = snocWith f
961+
962+ where
963+
964+ f oldSize =
965+ if isPower2 oldSize
966+ then oldSize * 2
967+ else roundUpToPower2 oldSize * 2
968+
969+-------------------------------------------------------------------------------
970+-- Random reads
971+-------------------------------------------------------------------------------
972+
973+-- XXX Can this be deduplicated with array/foreign
974+
975+-- | Return the element at the specified index without checking the bounds.
976+--
977+-- Unsafe because it does not check the bounds of the array.
978+{-# INLINE_NORMAL getIndexUnsafe #-}
979+getIndexUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Array a -> m a
980+getIndexUnsafe i Array{..} = do
981+ let index = INDEX_OF(arrStart,i,a)
982+ assert (i >= 0 && INDEX_VALID(index,arrEnd,a)) (return ())
983+ liftIO $ peekWith arrContents index
984+
985+-- | /O(1)/ Lookup the element at the given index. Index starts from 0.
986+--
987+{-# INLINE getIndex #-}
988+getIndex :: forall m a. (MonadIO m, Unbox a) => Int -> Array a -> m a
989+getIndex i Array{..} = do
990+ let index = INDEX_OF(arrStart,i,a)
991+ if i >= 0 && INDEX_VALID(index,arrEnd,a)
992+ then liftIO $ peekWith arrContents index
993+ else invalidIndex "getIndex" i
994+
995+-- | /O(1)/ Lookup the element at the given index from the end of the array.
996+-- Index starts from 0.
997+--
998+-- Slightly faster than computing the forward index and using getIndex.
999+--
1000+{-# INLINE getIndexRev #-}
1001+getIndexRev :: forall m a. (MonadIO m, Unbox a) => Int -> Array a -> m a
1002+getIndexRev i Array{..} = do
1003+ let index = RINDEX_OF(arrEnd,i,a)
1004+ if i >= 0 && index >= arrStart
1005+ then liftIO $ peekWith arrContents index
1006+ else invalidIndex "getIndexRev" i
1007+
1008+data GetIndicesState contents start end st =
1009+ GetIndicesState contents start end st
1010+
1011+-- | Given an unfold that generates array indices, read the elements on those
1012+-- indices from the supplied Array. An error is thrown if an index is out of
1013+-- bounds.
1014+--
1015+-- /Pre-release/
1016+{-# INLINE getIndicesD #-}
1017+getIndicesD :: (Monad m, Unbox a) =>
1018+ (forall b. IO b -> m b) -> D.Stream m Int -> Unfold m (Array a) a
1019+getIndicesD liftio (D.Stream stepi sti) = Unfold step inject
1020+
1021+ where
1022+
1023+ inject (Array contents start end _) =
1024+ return $ GetIndicesState contents start end sti
1025+
1026+ {-# INLINE_LATE step #-}
1027+ step (GetIndicesState contents start end st) = do
1028+ r <- stepi defState st
1029+ case r of
1030+ D.Yield i s -> do
1031+ x <- liftio $ getIndex i (Array contents start end undefined)
1032+ return $ D.Yield x (GetIndicesState contents start end s)
1033+ D.Skip s -> return $ D.Skip (GetIndicesState contents start end s)
1034+ D.Stop -> return D.Stop
1035+
1036+{-# INLINE getIndices #-}
1037+getIndices :: (MonadIO m, Unbox a) => Stream m Int -> Unfold m (Array a) a
1038+getIndices = getIndicesD liftIO
1039+
1040+-------------------------------------------------------------------------------
1041+-- Subarrays
1042+-------------------------------------------------------------------------------
1043+
1044+-- XXX We can also get immutable slices.
1045+
1046+-- | /O(1)/ Slice an array in constant time.
1047+--
1048+-- Unsafe: The bounds of the slice are not checked.
1049+--
1050+-- /Unsafe/
1051+--
1052+-- /Pre-release/
1053+{-# INLINE getSliceUnsafe #-}
1054+getSliceUnsafe :: forall a. Unbox a
1055+ => Int -- ^ from index
1056+ -> Int -- ^ length of the slice
1057+ -> Array a
1058+ -> Array a
1059+getSliceUnsafe index len (Array contents start e _) =
1060+ let fp1 = INDEX_OF(start,index,a)
1061+ end = fp1 + (len * SIZE_OF(a))
1062+ in assert
1063+ (index >= 0 && len >= 0 && end <= e)
1064+ -- Note: In a slice we always use bound = end so that the slice
1065+ -- user cannot overwrite elements beyond the end of the slice.
1066+ (Array contents fp1 end end)
1067+
1068+-- | /O(1)/ Slice an array in constant time. Throws an error if the slice
1069+-- extends out of the array bounds.
1070+--
1071+-- /Pre-release/
1072+{-# INLINE getSlice #-}
1073+getSlice :: forall a. Unbox a =>
1074+ Int -- ^ from index
1075+ -> Int -- ^ length of the slice
1076+ -> Array a
1077+ -> Array a
1078+getSlice index len (Array contents start e _) =
1079+ let fp1 = INDEX_OF(start,index,a)
1080+ end = fp1 + (len * SIZE_OF(a))
1081+ in if index >= 0 && len >= 0 && end <= e
1082+ -- Note: In a slice we always use bound = end so that the slice user
1083+ -- cannot overwrite elements beyond the end of the slice.
1084+ then Array contents fp1 end end
1085+ else error
1086+ $ "getSlice: invalid slice, index "
1087+ ++ show index ++ " length " ++ show len
1088+
1089+-------------------------------------------------------------------------------
1090+-- In-place mutation algorithms
1091+-------------------------------------------------------------------------------
1092+
1093+-- XXX consider the bulk update/accumulation/permutation APIs from vector.
1094+
1095+-- | You may not need to reverse an array because you can consume it in reverse
1096+-- using 'readerRev'. To reverse large arrays you can read in reverse and write
1097+-- to another array. However, in-place reverse can be useful to take adavantage
1098+-- of cache locality and when you do not want to allocate additional memory.
1099+--
1100+{-# INLINE reverse #-}
1101+reverse :: forall m a. (MonadIO m, Unbox a) => Array a -> m ()
1102+reverse Array{..} = liftIO $ do
1103+ let l = arrStart
1104+ h = INDEX_PREV(arrEnd,a)
1105+ in swap l h
1106+
1107+ where
1108+
1109+ swap l h = do
1110+ when (l < h) $ do
1111+ swapArrayByteIndices (Proxy :: Proxy a) arrContents l h
1112+ swap (INDEX_NEXT(l,a)) (INDEX_PREV(h,a))
1113+
1114+-- | Generate the next permutation of the sequence, returns False if this is
1115+-- the last permutation.
1116+--
1117+-- /Unimplemented/
1118+{-# INLINE permute #-}
1119+permute :: Array a -> m Bool
1120+permute = undefined
1121+
1122+-- | Partition an array into two halves using a partitioning predicate. The
1123+-- first half retains values where the predicate is 'False' and the second half
1124+-- retains values where the predicate is 'True'.
1125+--
1126+-- /Pre-release/
1127+{-# INLINE partitionBy #-}
1128+partitionBy :: forall m a. (MonadIO m, Unbox a)
1129+ => (a -> Bool) -> Array a -> m (Array a, Array a)
1130+partitionBy f arr@Array{..} = liftIO $ do
1131+ if arrStart >= arrEnd
1132+ then return (arr, arr)
1133+ else do
1134+ ptr <- go arrStart (INDEX_PREV(arrEnd,a))
1135+ let pl = Array arrContents arrStart ptr ptr
1136+ pr = Array arrContents ptr arrEnd arrEnd
1137+ return (pl, pr)
1138+
1139+ where
1140+
1141+ -- Invariant low < high on entry, and on return as well
1142+ moveHigh low high = do
1143+ h <- peekWith arrContents high
1144+ if f h
1145+ then
1146+ -- Correctly classified, continue the loop
1147+ let high1 = INDEX_PREV(high,a)
1148+ in if low == high1
1149+ then return Nothing
1150+ else moveHigh low high1
1151+ else return (Just (high, h)) -- incorrectly classified
1152+
1153+ -- Keep a low pointer starting at the start of the array (first partition)
1154+ -- and a high pointer starting at the end of the array (second partition).
1155+ -- Keep incrementing the low ptr and decrementing the high ptr until both
1156+ -- are wrongly classified, at that point swap the two and continue until
1157+ -- the two pointer cross each other.
1158+ --
1159+ -- Invariants when entering this loop:
1160+ -- low <= high
1161+ -- Both low and high are valid locations within the array
1162+ go low high = do
1163+ l <- peekWith arrContents low
1164+ if f l
1165+ then
1166+ -- low is wrongly classified
1167+ if low == high
1168+ then return low
1169+ else do -- low < high
1170+ r <- moveHigh low high
1171+ case r of
1172+ Nothing -> return low
1173+ Just (high1, h) -> do -- low < high1
1174+ pokeWith arrContents low h
1175+ pokeWith arrContents high1 l
1176+ let low1 = INDEX_NEXT(low,a)
1177+ high2 = INDEX_PREV(high1,a)
1178+ if low1 <= high2
1179+ then go low1 high2
1180+ else return low1 -- low1 > high2
1181+
1182+ else do
1183+ -- low is correctly classified
1184+ let low1 = INDEX_NEXT(low,a)
1185+ if low == high
1186+ then return low1
1187+ else go low1 high
1188+
1189+-- | Shuffle corresponding elements from two arrays using a shuffle function.
1190+-- If the shuffle function returns 'False' then do nothing otherwise swap the
1191+-- elements. This can be used in a bottom up fold to shuffle or reorder the
1192+-- elements.
1193+--
1194+-- /Unimplemented/
1195+{-# INLINE shuffleBy #-}
1196+shuffleBy :: (a -> a -> m Bool) -> Array a -> Array a -> m ()
1197+shuffleBy = undefined
1198+
1199+-- XXX we can also make the folds partial by stopping at a certain level.
1200+--
1201+-- | @divideBy level partition array@ performs a top down hierarchical
1202+-- recursive partitioning fold of items in the container using the given
1203+-- function as the partition function. Level indicates the level in the tree
1204+-- where the fold would stop.
1205+--
1206+-- This performs a quick sort if the partition function is
1207+-- 'partitionBy (< pivot)'.
1208+--
1209+-- /Unimplemented/
1210+{-# INLINABLE divideBy #-}
1211+divideBy ::
1212+ Int -> (Array a -> m (Array a, Array a)) -> Array a -> m ()
1213+divideBy = undefined
1214+
1215+-- | @mergeBy level merge array@ performs a pairwise bottom up fold recursively
1216+-- merging the pairs using the supplied merge function. Level indicates the
1217+-- level in the tree where the fold would stop.
1218+--
1219+-- This performs a random shuffle if the merge function is random. If we
1220+-- stop at level 0 and repeatedly apply the function then we can do a bubble
1221+-- sort.
1222+--
1223+-- /Unimplemented/
1224+mergeBy :: Int -> (Array a -> Array a -> m ()) -> Array a -> m ()
1225+mergeBy = undefined
1226+
1227+-------------------------------------------------------------------------------
1228+-- Size
1229+-------------------------------------------------------------------------------
1230+
1231+-- | /O(1)/ Get the byte length of the array.
1232+--
1233+{-# INLINE byteLength #-}
1234+byteLength :: Array a -> Int
1235+byteLength Array{..} =
1236+ let len = arrEnd - arrStart
1237+ in assert (len >= 0) len
1238+
1239+-- Note: try to avoid the use of length in performance sensitive internal
1240+-- routines as it involves a costly 'div' operation. Instead use the end ptr
1241+-- in the array to check the bounds etc.
1242+--
1243+-- | /O(1)/ Get the length of the array i.e. the number of elements in the
1244+-- array.
1245+--
1246+-- Note that 'byteLength' is less expensive than this operation, as 'length'
1247+-- involves a costly division operation.
1248+--
1249+{-# INLINE length #-}
1250+length :: forall a. Unbox a => Array a -> Int
1251+length arr =
1252+ let elemSize = SIZE_OF(a)
1253+ blen = byteLength arr
1254+ in assert (blen `mod` elemSize == 0) (blen `div` elemSize)
1255+
1256+-- | Get the total capacity of an array. An array may have space reserved
1257+-- beyond the current used length of the array.
1258+--
1259+-- /Pre-release/
1260+{-# INLINE byteCapacity #-}
1261+byteCapacity :: Array a -> Int
1262+byteCapacity Array{..} =
1263+ let len = arrBound - arrStart
1264+ in assert (len >= 0) len
1265+
1266+-- | The remaining capacity in the array for appending more elements without
1267+-- reallocation.
1268+--
1269+-- /Pre-release/
1270+{-# INLINE bytesFree #-}
1271+bytesFree :: Array a -> Int
1272+bytesFree Array{..} =
1273+ let n = arrBound - arrEnd
1274+ in assert (n >= 0) n
1275+
1276+-------------------------------------------------------------------------------
1277+-- Streams of arrays - Creation
1278+-------------------------------------------------------------------------------
1279+
1280+data GroupState s contents start end bound
1281+ = GroupStart s
1282+ | GroupBuffer s contents start end bound
1283+ | GroupYield
1284+ contents start end bound (GroupState s contents start end bound)
1285+ | GroupFinish
1286+
1287+-- | @arraysOf n stream@ groups the input stream into a stream of
1288+-- arrays of size n.
1289+--
1290+-- @arraysOf n = StreamD.foldMany (Array.writeN n)@
1291+--
1292+-- /Pre-release/
1293+{-# INLINE_NORMAL arraysOf #-}
1294+arraysOf :: forall m a. (MonadIO m, Unbox a)
1295+ => Int -> D.Stream m a -> D.Stream m (Array a)
1296+-- XXX the idiomatic implementation leads to large regression in the D.reverse'
1297+-- benchmark. It seems it has difficulty producing optimized code when
1298+-- converting to StreamK. Investigate GHC optimizations.
1299+-- arraysOf n = D.foldMany (writeN n)
1300+arraysOf n (D.Stream step state) =
1301+ D.Stream step' (GroupStart state)
1302+
1303+ where
1304+
1305+ {-# INLINE_LATE step' #-}
1306+ step' _ (GroupStart st) = do
1307+ when (n <= 0) $
1308+ -- XXX we can pass the module string from the higher level API
1309+ error $ "Streamly.Internal.Data.Array.Mut.Type.arraysOf: "
1310+ ++ "the size of arrays [" ++ show n
1311+ ++ "] must be a natural number"
1312+ (Array contents start end bound :: Array a) <- liftIO $ newPinned n
1313+ return $ D.Skip (GroupBuffer st contents start end bound)
1314+
1315+ step' gst (GroupBuffer st contents start end bound) = do
1316+ r <- step (adaptState gst) st
1317+ case r of
1318+ D.Yield x s -> do
1319+ liftIO $ pokeWith contents end x
1320+ let end1 = INDEX_NEXT(end,a)
1321+ return $
1322+ if end1 >= bound
1323+ then D.Skip
1324+ (GroupYield
1325+ contents start end1 bound (GroupStart s))
1326+ else D.Skip (GroupBuffer s contents start end1 bound)
1327+ D.Skip s ->
1328+ return $ D.Skip (GroupBuffer s contents start end bound)
1329+ D.Stop ->
1330+ return
1331+ $ D.Skip (GroupYield contents start end bound GroupFinish)
1332+
1333+ step' _ (GroupYield contents start end bound next) =
1334+ return $ D.Yield (Array contents start end bound) next
1335+
1336+ step' _ GroupFinish = return D.Stop
1337+
1338+-- XXX buffer to a list instead?
1339+-- | Buffer the stream into arrays in memory.
1340+{-# INLINE arrayStreamKFromStreamD #-}
1341+arrayStreamKFromStreamD :: forall m a. (MonadIO m, Unbox a) =>
1342+ D.Stream m a -> m (K.Stream m (Array a))
1343+arrayStreamKFromStreamD =
1344+ let n = allocBytesToElemCount (undefined :: a) defaultChunkSize
1345+ in D.foldr K.cons K.nil . arraysOf n
1346+
1347+-------------------------------------------------------------------------------
1348+-- Streams of arrays - Flattening
1349+-------------------------------------------------------------------------------
1350+
1351+data FlattenState s contents a =
1352+ OuterLoop s
1353+ | InnerLoop s contents !Int !Int
1354+
1355+-- | Use the "reader" unfold instead.
1356+--
1357+-- @flattenArrays = unfoldMany reader@
1358+--
1359+-- We can try this if there are any fusion issues in the unfold.
1360+--
1361+{-# INLINE_NORMAL flattenArrays #-}
1362+flattenArrays :: forall m a. (MonadIO m, Unbox a)
1363+ => D.Stream m (Array a) -> D.Stream m a
1364+flattenArrays (D.Stream step state) = D.Stream step' (OuterLoop state)
1365+
1366+ where
1367+
1368+ {-# INLINE_LATE step' #-}
1369+ step' gst (OuterLoop st) = do
1370+ r <- step (adaptState gst) st
1371+ return $ case r of
1372+ D.Yield Array{..} s ->
1373+ D.Skip (InnerLoop s arrContents arrStart arrEnd)
1374+ D.Skip s -> D.Skip (OuterLoop s)
1375+ D.Stop -> D.Stop
1376+
1377+ step' _ (InnerLoop st _ p end) | assert (p <= end) (p == end) =
1378+ return $ D.Skip $ OuterLoop st
1379+
1380+ step' _ (InnerLoop st contents p end) = do
1381+ x <- liftIO $ peekWith contents p
1382+ return $ D.Yield x (InnerLoop st contents (INDEX_NEXT(p,a)) end)
1383+
1384+-- | Use the "readerRev" unfold instead.
1385+--
1386+-- @flattenArrays = unfoldMany readerRev@
1387+--
1388+-- We can try this if there are any fusion issues in the unfold.
1389+--
1390+{-# INLINE_NORMAL flattenArraysRev #-}
1391+flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
1392+ => D.Stream m (Array a) -> D.Stream m a
1393+flattenArraysRev (D.Stream step state) = D.Stream step' (OuterLoop state)
1394+
1395+ where
1396+
1397+ {-# INLINE_LATE step' #-}
1398+ step' gst (OuterLoop st) = do
1399+ r <- step (adaptState gst) st
1400+ return $ case r of
1401+ D.Yield Array{..} s ->
1402+ let p = INDEX_PREV(arrEnd,a)
1403+ in D.Skip (InnerLoop s arrContents p arrStart)
1404+ D.Skip s -> D.Skip (OuterLoop s)
1405+ D.Stop -> D.Stop
1406+
1407+ step' _ (InnerLoop st _ p start) | p < start =
1408+ return $ D.Skip $ OuterLoop st
1409+
1410+ step' _ (InnerLoop st contents p start) = do
1411+ x <- liftIO $ peekWith contents p
1412+ let cur = INDEX_PREV(p,a)
1413+ return $ D.Yield x (InnerLoop st contents cur start)
1414+
1415+-------------------------------------------------------------------------------
1416+-- Unfolds
1417+-------------------------------------------------------------------------------
1418+
1419+data ArrayUnsafe a = ArrayUnsafe
1420+ {-# UNPACK #-} !MutableByteArray -- contents
1421+ {-# UNPACK #-} !Int -- index 1
1422+ {-# UNPACK #-} !Int -- index 2
1423+
1424+toArrayUnsafe :: Array a -> ArrayUnsafe a
1425+toArrayUnsafe (Array contents start end _) = ArrayUnsafe contents start end
1426+
1427+fromArrayUnsafe ::
1428+#ifdef DEVBUILD
1429+ Unbox a =>
1430+#endif
1431+ ArrayUnsafe a -> Array a
1432+fromArrayUnsafe (ArrayUnsafe contents start end) =
1433+ Array contents start end end
1434+
1435+{-# INLINE_NORMAL producerWith #-}
1436+producerWith ::
1437+ forall m a. (Monad m, Unbox a)
1438+ => (forall b. IO b -> m b) -> Producer m (Array a) a
1439+producerWith liftio = Producer step (return . toArrayUnsafe) extract
1440+ where
1441+
1442+ {-# INLINE_LATE step #-}
1443+ step (ArrayUnsafe _ cur end)
1444+ | assert (cur <= end) (cur == end) = return D.Stop
1445+ step (ArrayUnsafe contents cur end) = do
1446+ -- When we use a purely lazy Monad like Identity, we need to force a
1447+ -- few actions for correctness and execution order sanity. We want
1448+ -- the peek to occur right here and not lazily at some later point
1449+ -- because we want the peek to be ordered with respect to the touch.
1450+ !x <- liftio $ peekWith contents cur
1451+ return $ D.Yield x (ArrayUnsafe contents (INDEX_NEXT(cur,a)) end)
1452+
1453+ extract = return . fromArrayUnsafe
1454+
1455+-- | Resumable unfold of an array.
1456+--
1457+{-# INLINE_NORMAL producer #-}
1458+producer :: forall m a. (MonadIO m, Unbox a) => Producer m (Array a) a
1459+producer = producerWith liftIO
1460+
1461+-- | Unfold an array into a stream.
1462+--
1463+{-# INLINE_NORMAL reader #-}
1464+reader :: forall m a. (MonadIO m, Unbox a) => Unfold m (Array a) a
1465+reader = Producer.simplify producer
1466+
1467+{-# INLINE_NORMAL readerRevWith #-}
1468+readerRevWith ::
1469+ forall m a. (Monad m, Unbox a)
1470+ => (forall b. IO b -> m b) -> Unfold m (Array a) a
1471+readerRevWith liftio = Unfold step inject
1472+ where
1473+
1474+ inject (Array contents start end _) =
1475+ let p = INDEX_PREV(end,a)
1476+ in return $ ArrayUnsafe contents start p
1477+
1478+ {-# INLINE_LATE step #-}
1479+ step (ArrayUnsafe _ start p) | p < start = return D.Stop
1480+ step (ArrayUnsafe contents start p) = do
1481+ !x <- liftio $ peekWith contents p
1482+ return $ D.Yield x (ArrayUnsafe contents start (INDEX_PREV(p,a)))
1483+
1484+-- | Unfold an array into a stream in reverse order.
1485+--
1486+{-# INLINE_NORMAL readerRev #-}
1487+readerRev :: forall m a. (MonadIO m, Unbox a) => Unfold m (Array a) a
1488+readerRev = readerRevWith liftIO
1489+
1490+-------------------------------------------------------------------------------
1491+-- to Lists and streams
1492+-------------------------------------------------------------------------------
1493+
1494+{-
1495+-- Use foldr/build fusion to fuse with list consumers
1496+-- This can be useful when using the IsList instance
1497+{-# INLINE_LATE toListFB #-}
1498+toListFB :: forall a b. Unbox a => (a -> b -> b) -> b -> Array a -> b
1499+toListFB c n Array{..} = go arrStart
1500+ where
1501+
1502+ go p | assert (p <= arrEnd) (p == arrEnd) = n
1503+ go p =
1504+ -- unsafeInlineIO allows us to run this in Identity monad for pure
1505+ -- toList/foldr case which makes them much faster due to not
1506+ -- accumulating the list and fusing better with the pure consumers.
1507+ --
1508+ -- This should be safe as the array contents are guaranteed to be
1509+ -- evaluated/written to before we peek at them.
1510+ -- XXX
1511+ let !x = unsafeInlineIO $ do
1512+ r <- peekWith arrContents p
1513+ return r
1514+ in c x (go (PTR_NEXT(p,a)))
1515+-}
1516+
1517+-- XXX Monadic foldr/build fusion?
1518+-- Reference: https://www.researchgate.net/publication/220676509_Monadic_augment_and_generalised_short_cut_fusion
1519+-- | Convert an 'Array' into a list.
1520+--
1521+{-# INLINE toList #-}
1522+toList :: forall m a. (MonadIO m, Unbox a) => Array a -> m [a]
1523+toList Array{..} = liftIO $ go arrStart
1524+ where
1525+
1526+ go p | assert (p <= arrEnd) (p == arrEnd) = return []
1527+ go p = do
1528+ x <- peekWith arrContents p
1529+ (:) x <$> go (INDEX_NEXT(p,a))
1530+
1531+{-# INLINE_NORMAL toStreamDWith #-}
1532+toStreamDWith ::
1533+ forall m a. (Monad m, Unbox a)
1534+ => (forall b. IO b -> m b) -> Array a -> D.Stream m a
1535+toStreamDWith liftio Array{..} = D.Stream step arrStart
1536+
1537+ where
1538+
1539+ {-# INLINE_LATE step #-}
1540+ step _ p | assert (p <= arrEnd) (p == arrEnd) = return D.Stop
1541+ step _ p = liftio $ do
1542+ r <- peekWith arrContents p
1543+ return $ D.Yield r (INDEX_NEXT(p,a))
1544+
1545+-- | Use the 'reader' unfold instead.
1546+--
1547+-- @toStreamD = D.unfold reader@
1548+--
1549+-- We can try this if the unfold has any performance issues.
1550+{-# INLINE_NORMAL toStreamD #-}
1551+toStreamD :: forall m a. (MonadIO m, Unbox a) => Array a -> D.Stream m a
1552+toStreamD = toStreamDWith liftIO
1553+
1554+{-# INLINE toStreamKWith #-}
1555+toStreamKWith ::
1556+ forall m a. (Monad m, Unbox a)
1557+ => (forall b. IO b -> m b) -> Array a -> K.Stream m a
1558+toStreamKWith liftio Array{..} = go arrStart
1559+
1560+ where
1561+
1562+ go p | assert (p <= arrEnd) (p == arrEnd) = K.nil
1563+ | otherwise =
1564+ let elemM = peekWith arrContents p
1565+ in liftio elemM `K.consM` go (INDEX_NEXT(p,a))
1566+
1567+{-# INLINE toStreamK #-}
1568+toStreamK :: forall m a. (MonadIO m, Unbox a) => Array a -> K.Stream m a
1569+toStreamK = toStreamKWith liftIO
1570+
1571+{-# INLINE_NORMAL toStreamDRevWith #-}
1572+toStreamDRevWith ::
1573+ forall m a. (Monad m, Unbox a)
1574+ => (forall b. IO b -> m b) -> Array a -> D.Stream m a
1575+toStreamDRevWith liftio Array{..} =
1576+ let p = INDEX_PREV(arrEnd,a)
1577+ in D.Stream step p
1578+
1579+ where
1580+
1581+ {-# INLINE_LATE step #-}
1582+ step _ p | p < arrStart = return D.Stop
1583+ step _ p = liftio $ do
1584+ r <- peekWith arrContents p
1585+ return $ D.Yield r (INDEX_PREV(p,a))
1586+
1587+-- | Use the 'readerRev' unfold instead.
1588+--
1589+-- @toStreamDRev = D.unfold readerRev@
1590+--
1591+-- We can try this if the unfold has any perf issues.
1592+{-# INLINE_NORMAL toStreamDRev #-}
1593+toStreamDRev :: forall m a. (MonadIO m, Unbox a) => Array a -> D.Stream m a
1594+toStreamDRev = toStreamDRevWith liftIO
1595+
1596+{-# INLINE toStreamKRevWith #-}
1597+toStreamKRevWith ::
1598+ forall m a. (Monad m, Unbox a)
1599+ => (forall b. IO b -> m b) -> Array a -> K.Stream m a
1600+toStreamKRevWith liftio Array {..} =
1601+ let p = INDEX_PREV(arrEnd,a)
1602+ in go p
1603+
1604+ where
1605+
1606+ go p | p < arrStart = K.nil
1607+ | otherwise =
1608+ let elemM = peekWith arrContents p
1609+ in liftio elemM `K.consM` go (INDEX_PREV(p,a))
1610+
1611+{-# INLINE toStreamKRev #-}
1612+toStreamKRev :: forall m a. (MonadIO m, Unbox a) => Array a -> K.Stream m a
1613+toStreamKRev = toStreamKRevWith liftIO
1614+
1615+-------------------------------------------------------------------------------
1616+-- Folding
1617+-------------------------------------------------------------------------------
1618+
1619+-- XXX Need something like "Array m a" enforcing monadic action to avoid the
1620+-- possibility of such APIs.
1621+--
1622+-- | Strict left fold of an array.
1623+{-# INLINE_NORMAL foldl' #-}
1624+foldl' :: (MonadIO m, Unbox a) => (b -> a -> b) -> b -> Array a -> m b
1625+foldl' f z arr = D.foldl' f z $ toStreamD arr
1626+
1627+-- | Right fold of an array.
1628+{-# INLINE_NORMAL foldr #-}
1629+foldr :: (MonadIO m, Unbox a) => (a -> b -> b) -> b -> Array a -> m b
1630+foldr f z arr = D.foldr f z $ toStreamD arr
1631+
1632+-------------------------------------------------------------------------------
1633+-- Folds
1634+-------------------------------------------------------------------------------
1635+
1636+-- Note: Arrays may be allocated with a specific alignment at the beginning of
1637+-- the array. If you need to maintain that alignment on reallocations then you
1638+-- can resize the array manually before append, using an aligned resize
1639+-- operation.
1640+
1641+-- XXX Keep the bound intact to not lose any free space? Perf impact?
1642+
1643+-- | @writeAppendNUnsafe n alloc@ appends up to @n@ input items to the supplied
1644+-- array.
1645+--
1646+-- Unsafe: Do not drive the fold beyond @n@ elements, it will lead to memory
1647+-- corruption or segfault.
1648+--
1649+-- Any free space left in the array after appending @n@ elements is lost.
1650+--
1651+-- /Internal/
1652+{-# INLINE_NORMAL writeAppendNUnsafe #-}
1653+writeAppendNUnsafe :: forall m a. (MonadIO m, Unbox a) =>
1654+ Int
1655+ -> m (Array a)
1656+ -> Fold m a (Array a)
1657+writeAppendNUnsafe n action =
1658+ fmap fromArrayUnsafe $ FL.foldlM' step initial
1659+
1660+ where
1661+
1662+ initial = do
1663+ assert (n >= 0) (return ())
1664+ arr@(Array _ _ end bound) <- action
1665+ let free = bound - end
1666+ needed = n * SIZE_OF(a)
1667+ -- XXX We can also reallocate if the array has too much free space,
1668+ -- otherwise we lose that space.
1669+ arr1 <-
1670+ if free < needed
1671+ then noinline reallocWith "writeAppendNUnsafeWith" (+ needed) needed arr
1672+ else return arr
1673+ return $ toArrayUnsafe arr1
1674+
1675+ step (ArrayUnsafe contents start end) x = do
1676+ liftIO $ pokeWith contents end x
1677+ return $ ArrayUnsafe contents start (INDEX_NEXT(end,a))
1678+
1679+-- | Append @n@ elements to an existing array. Any free space left in the array
1680+-- after appending @n@ elements is lost.
1681+--
1682+-- >>> writeAppendN n initial = Fold.take n (Array.writeAppendNUnsafe n initial)
1683+--
1684+{-# INLINE_NORMAL writeAppendN #-}
1685+writeAppendN :: forall m a. (MonadIO m, Unbox a) =>
1686+ Int -> m (Array a) -> Fold m a (Array a)
1687+writeAppendN n initial = FL.take n (writeAppendNUnsafe n initial)
1688+
1689+-- | @writeAppendWith realloc action@ mutates the array generated by @action@ to
1690+-- append the input stream. If there is no reserved space available in the
1691+-- array it is reallocated to a size in bytes determined by @realloc oldSize@,
1692+-- where @oldSize@ is the current size of the array in bytes.
1693+--
1694+-- Note that the returned array may be a mutated version of original array.
1695+--
1696+-- >>> writeAppendWith sizer = Fold.foldlM' (Array.snocWith sizer)
1697+--
1698+-- /Pre-release/
1699+{-# INLINE writeAppendWith #-}
1700+writeAppendWith :: forall m a. (MonadIO m, Unbox a) =>
1701+ (Int -> Int) -> m (Array a) -> Fold m a (Array a)
1702+writeAppendWith sizer = FL.foldlM' (snocWith sizer)
1703+
1704+-- | @append action@ mutates the array generated by @action@ to append the
1705+-- input stream. If there is no reserved space available in the array it is
1706+-- reallocated to double the size.
1707+--
1708+-- Note that the returned array may be a mutated version of original array.
1709+--
1710+-- >>> writeAppend = Array.writeAppendWith (* 2)
1711+--
1712+{-# INLINE writeAppend #-}
1713+writeAppend :: forall m a. (MonadIO m, Unbox a) =>
1714+ m (Array a) -> Fold m a (Array a)
1715+writeAppend = writeAppendWith (* 2)
1716+
1717+-- XXX We can carry bound as well in the state to make sure we do not lose the
1718+-- remaining capacity. Need to check perf impact.
1719+--
1720+-- | Like 'writeNUnsafe' but takes a new array allocator @alloc size@ function
1721+-- as argument.
1722+--
1723+-- >>> writeNWithUnsafe alloc n = Array.writeAppendNUnsafe (alloc n) n
1724+--
1725+-- /Pre-release/
1726+{-# INLINE_NORMAL writeNWithUnsafe #-}
1727+writeNWithUnsafe :: forall m a. (MonadIO m, Unbox a)
1728+ => (Int -> m (Array a)) -> Int -> Fold m a (Array a)
1729+writeNWithUnsafe alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
1730+
1731+ where
1732+
1733+ initial = toArrayUnsafe <$> alloc (max n 0)
1734+
1735+ step (ArrayUnsafe contents start end) x = do
1736+ liftIO $ pokeWith contents end x
1737+ return
1738+ $ ArrayUnsafe contents start (INDEX_NEXT(end,a))
1739+
1740+-- | Like 'writeN' but does not check the array bounds when writing. The fold
1741+-- driver must not call the step function more than 'n' times otherwise it will
1742+-- corrupt the memory and crash. This function exists mainly because any
1743+-- conditional in the step function blocks fusion causing 10x performance
1744+-- slowdown.
1745+--
1746+-- >>> writeNUnsafe = Array.writeNWithUnsafe Array.newPinned
1747+--
1748+{-# INLINE_NORMAL writeNUnsafe #-}
1749+writeNUnsafe :: forall m a. (MonadIO m, Unbox a)
1750+ => Int -> Fold m a (Array a)
1751+writeNUnsafe = writeNWithUnsafe newPinned
1752+
1753+-- | @writeNWith alloc n@ folds a maximum of @n@ elements into an array
1754+-- allocated using the @alloc@ function.
1755+--
1756+-- >>> writeNWith alloc n = Fold.take n (Array.writeNWithUnsafe alloc n)
1757+-- >>> writeNWith alloc n = Array.writeAppendN (alloc n) n
1758+--
1759+{-# INLINE_NORMAL writeNWith #-}
1760+writeNWith :: forall m a. (MonadIO m, Unbox a)
1761+ => (Int -> m (Array a)) -> Int -> Fold m a (Array a)
1762+writeNWith alloc n = FL.take n (writeNWithUnsafe alloc n)
1763+
1764+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
1765+-- 'Array'.
1766+--
1767+-- >>> writeN = Array.writeNWith Array.newPinned
1768+-- >>> writeN n = Fold.take n (Array.writeNUnsafe n)
1769+-- >>> writeN n = Array.writeAppendN n (Array.newPinned n)
1770+--
1771+{-# INLINE_NORMAL writeN #-}
1772+writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1773+writeN = writeNWith newPinned
1774+
1775+-- | Like writeNWithUnsafe but writes the array in reverse order.
1776+--
1777+-- /Internal/
1778+{-# INLINE_NORMAL writeRevNWithUnsafe #-}
1779+writeRevNWithUnsafe :: forall m a. (MonadIO m, Unbox a)
1780+ => (Int -> m (Array a)) -> Int -> Fold m a (Array a)
1781+writeRevNWithUnsafe alloc n = fromArrayUnsafe <$> FL.foldlM' step initial
1782+
1783+ where
1784+
1785+ toArrayUnsafeRev (Array contents _ _ bound) =
1786+ ArrayUnsafe contents bound bound
1787+
1788+ initial = toArrayUnsafeRev <$> alloc (max n 0)
1789+
1790+ step (ArrayUnsafe contents start end) x = do
1791+ let ptr = INDEX_PREV(start,a)
1792+ liftIO $ pokeWith contents ptr x
1793+ return
1794+ $ ArrayUnsafe contents ptr end
1795+
1796+-- | Like writeNWith but writes the array in reverse order.
1797+--
1798+-- /Internal/
1799+{-# INLINE_NORMAL writeRevNWith #-}
1800+writeRevNWith :: forall m a. (MonadIO m, Unbox a)
1801+ => (Int -> m (Array a)) -> Int -> Fold m a (Array a)
1802+writeRevNWith alloc n = FL.take n (writeRevNWithUnsafe alloc n)
1803+
1804+-- | Like writeN but writes the array in reverse order.
1805+--
1806+-- /Pre-release/
1807+{-# INLINE_NORMAL writeRevN #-}
1808+writeRevN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
1809+writeRevN = writeRevNWith newPinned
1810+
1811+-- | @writeNAligned align n@ folds a maximum of @n@ elements from the input
1812+-- stream to an 'Array' aligned to the given size.
1813+--
1814+-- >>> writeNAligned align = Array.writeNWith (Array.newAlignedPinned align)
1815+-- >>> writeNAligned align n = Array.writeAppendN n (Array.newAlignedPinned align n)
1816+--
1817+-- /Pre-release/
1818+--
1819+{-# INLINE_NORMAL writeNAligned #-}
1820+writeNAligned :: forall m a. (MonadIO m, Unbox a)
1821+ => Int -> Int -> Fold m a (Array a)
1822+writeNAligned align = writeNWith (newAlignedPinned align)
1823+
1824+-- XXX Buffer to a list instead?
1825+--
1826+-- | Buffer a stream into a stream of arrays.
1827+--
1828+-- >>> writeChunks n = Fold.many (Array.writeN n) Fold.toStreamK
1829+--
1830+-- Breaking an array into an array stream can be useful to consume a large
1831+-- array sequentially such that memory of the array is released incrementatlly.
1832+--
1833+-- See also: 'arrayStreamKFromStreamD'.
1834+--
1835+-- /Unimplemented/
1836+--
1837+{-# INLINE_NORMAL writeChunks #-}
1838+writeChunks :: (MonadIO m, Unbox a) =>
1839+ Int -> Fold m a (K.Stream n (Array a))
1840+writeChunks n = FL.many (writeN n) FL.toStreamK
1841+
1842+-- XXX Compare writeWith with fromStreamD which uses an array of streams
1843+-- implementation. We can write this using writeChunks above if that is faster.
1844+-- If writeWith is faster then we should use that to implement
1845+-- fromStreamD.
1846+--
1847+-- XXX The realloc based implementation needs to make one extra copy if we use
1848+-- shrinkToFit. On the other hand, the stream of arrays implementation may
1849+-- buffer the array chunk pointers in memory but it does not have to shrink as
1850+-- we know the exact size in the end. However, memory copying does not seem to
1851+-- be as expensive as the allocations. Therefore, we need to reduce the number
1852+-- of allocations instead. Also, the size of allocations matters, right sizing
1853+-- an allocation even at the cost of copying sems to help. Should be measured
1854+-- on a big stream with heavy calls to toArray to see the effect.
1855+--
1856+-- XXX check if GHC's memory allocator is efficient enough. We can try the C
1857+-- malloc to compare against.
1858+
1859+-- | @writeWith minCount@ folds the whole input to a single array. The array
1860+-- starts at a size big enough to hold minCount elements, the size is doubled
1861+-- every time the array needs to be grown.
1862+--
1863+-- /Caution! Do not use this on infinite streams./
1864+--
1865+-- >>> f n = Array.writeAppendWith (* 2) (Array.newPinned n)
1866+-- >>> writeWith n = Fold.rmapM Array.rightSize (f n)
1867+-- >>> writeWith n = Fold.rmapM Array.fromArrayStreamK (Array.writeChunks n)
1868+--
1869+-- /Pre-release/
1870+{-# INLINE_NORMAL writeWith #-}
1871+writeWith :: forall m a. (MonadIO m, Unbox a)
1872+ => Int -> Fold m a (Array a)
1873+-- writeWith n = FL.rmapM rightSize $ writeAppendWith (* 2) (newPinned n)
1874+writeWith elemCount =
1875+ FL.rmapM extract $ FL.foldlM' step initial
1876+
1877+ where
1878+
1879+ insertElem (Array contents start end bound) x = do
1880+ liftIO $ pokeWith contents end x
1881+ return $ Array contents start (INDEX_NEXT(end,a)) bound
1882+
1883+ initial = do
1884+ when (elemCount < 0) $ error "writeWith: elemCount is negative"
1885+ liftIO $ newPinned elemCount
1886+ step arr@(Array _ start end bound) x
1887+ | INDEX_NEXT(end,a) > bound = do
1888+ let oldSize = end - start
1889+ newSize = max (oldSize * 2) 1
1890+ arr1 <- liftIO $ reallocExplicit (SIZE_OF(a)) newSize arr
1891+ insertElem arr1 x
1892+ step arr x = insertElem arr x
1893+ extract = liftIO . rightSize
1894+
1895+-- | Fold the whole input to a single array.
1896+--
1897+-- Same as 'writeWith' using an initial array size of 'arrayChunkBytes' bytes
1898+-- rounded up to the element size.
1899+--
1900+-- /Caution! Do not use this on infinite streams./
1901+--
1902+{-# INLINE write #-}
1903+write :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
1904+write = writeWith (allocBytesToElemCount (undefined :: a) arrayChunkBytes)
1905+
1906+-------------------------------------------------------------------------------
1907+-- construct from streams, known size
1908+-------------------------------------------------------------------------------
1909+
1910+-- | Use the 'writeN' fold instead.
1911+--
1912+-- >>> fromStreamDN n = StreamD.fold (Array.writeN n)
1913+--
1914+{-# INLINE_NORMAL fromStreamDN #-}
1915+fromStreamDN :: forall m a. (MonadIO m, Unbox a)
1916+ => Int -> D.Stream m a -> m (Array a)
1917+-- fromStreamDN n = D.fold (writeN n)
1918+fromStreamDN limit str = do
1919+ (arr :: Array a) <- liftIO $ newPinned limit
1920+ end <- D.foldlM' (fwrite (arrContents arr)) (return $ arrEnd arr) $ D.take limit str
1921+ return $ arr {arrEnd = end}
1922+
1923+ where
1924+
1925+ fwrite arrContents ptr x = do
1926+ liftIO $ pokeWith arrContents ptr x
1927+ return $ INDEX_NEXT(ptr,a)
1928+
1929+-- | Create an 'Array' from the first N elements of a list. The array is
1930+-- allocated to size N, if the list terminates before N elements then the
1931+-- array may hold less than N elements.
1932+--
1933+{-# INLINABLE fromListN #-}
1934+fromListN :: (MonadIO m, Unbox a) => Int -> [a] -> m (Array a)
1935+fromListN n xs = fromStreamDN n $ D.fromList xs
1936+
1937+-- | Like fromListN but writes the array in reverse order.
1938+--
1939+-- /Pre-release/
1940+{-# INLINE fromListRevN #-}
1941+fromListRevN :: (MonadIO m, Unbox a) => Int -> [a] -> m (Array a)
1942+fromListRevN n xs = D.fold (writeRevN n) $ D.fromList xs
1943+
1944+-------------------------------------------------------------------------------
1945+-- convert stream to a single array
1946+-------------------------------------------------------------------------------
1947+
1948+{-# INLINE arrayStreamKLength #-}
1949+arrayStreamKLength :: (Monad m, Unbox a) => K.Stream m (Array a) -> m Int
1950+arrayStreamKLength as = K.foldl' (+) 0 (K.map length as)
1951+
1952+-- | Convert an array stream to an array. Note that this requires peak memory
1953+-- that is double the size of the array stream.
1954+--
1955+{-# INLINE fromArrayStreamK #-}
1956+fromArrayStreamK :: (Unbox a, MonadIO m) =>
1957+ K.Stream m (Array a) -> m (Array a)
1958+fromArrayStreamK as = do
1959+ len <- arrayStreamKLength as
1960+ fromStreamDN len $ D.unfoldMany reader $ D.fromStreamK as
1961+
1962+-- CAUTION: a very large number (millions) of arrays can degrade performance
1963+-- due to GC overhead because we need to buffer the arrays before we flatten
1964+-- all the arrays.
1965+--
1966+-- XXX Compare if this is faster or "fold write".
1967+--
1968+-- | We could take the approach of doubling the memory allocation on each
1969+-- overflow. This would result in more or less the same amount of copying as in
1970+-- the chunking approach. However, if we have to shrink in the end then it may
1971+-- result in an extra copy of the entire data.
1972+--
1973+-- >>> fromStreamD = StreamD.fold Array.write
1974+--
1975+{-# INLINE fromStreamD #-}
1976+fromStreamD :: (MonadIO m, Unbox a) => D.Stream m a -> m (Array a)
1977+fromStreamD m = arrayStreamKFromStreamD m >>= fromArrayStreamK
1978+
1979+-- | Create an 'Array' from a list. The list must be of finite size.
1980+--
1981+{-# INLINE fromList #-}
1982+fromList :: (MonadIO m, Unbox a) => [a] -> m (Array a)
1983+fromList xs = fromStreamD $ D.fromList xs
1984+
1985+-- XXX We are materializing the whole list first for getting the length. Check
1986+-- if the 'fromList' like chunked implementation would fare better.
1987+
1988+-- | Like 'fromList' but writes the contents of the list in reverse order.
1989+{-# INLINE fromListRev #-}
1990+fromListRev :: (MonadIO m, Unbox a) => [a] -> m (Array a)
1991+fromListRev xs = fromListRevN (Prelude.length xs) xs
1992+
1993+-------------------------------------------------------------------------------
1994+-- Combining
1995+-------------------------------------------------------------------------------
1996+
1997+-- | Put a sub range of a source array into a subrange of a destination array.
1998+-- This is not safe as it does not check the bounds.
1999+{-# INLINE putSliceUnsafe #-}
2000+putSliceUnsafe :: MonadIO m => Array a -> Int -> Array a -> Int -> Int -> m ()
2001+putSliceUnsafe src srcStartBytes dst dstStartBytes lenBytes = liftIO $ do
2002+ assertM(lenBytes <= arrBound dst - dstStartBytes)
2003+ assertM(lenBytes <= arrEnd src - srcStartBytes)
2004+ let !(I# srcStartBytes#) = srcStartBytes
2005+ !(I# dstStartBytes#) = dstStartBytes
2006+ !(I# lenBytes#) = lenBytes
2007+ let arrS# = getMutableByteArray# (arrContents src)
2008+ arrD# = getMutableByteArray# (arrContents dst)
2009+ IO $ \s# -> (# copyMutableByteArray#
2010+ arrS# srcStartBytes# arrD# dstStartBytes# lenBytes# s#
2011+ , () #)
2012+
2013+-- | Copy two arrays into a newly allocated array.
2014+{-# INLINE spliceCopy #-}
2015+spliceCopy :: forall m a. MonadIO m =>
2016+#ifdef DEVBUILD
2017+ Unbox a =>
2018+#endif
2019+ Array a -> Array a -> m (Array a)
2020+spliceCopy arr1 arr2 = liftIO $ do
2021+ let start1 = arrStart arr1
2022+ start2 = arrStart arr2
2023+ len1 = arrEnd arr1 - start1
2024+ len2 = arrEnd arr2 - start2
2025+ newArrContents <- liftIO $ Unboxed.newPinnedBytes (len1 + len2)
2026+ let len = len1 + len2
2027+ newArr = Array newArrContents 0 len len
2028+ putSliceUnsafe arr1 start1 newArr 0 len1
2029+ putSliceUnsafe arr2 start2 newArr len1 len2
2030+ return newArr
2031+
2032+-- | Really really unsafe, appends the second array into the first array. If
2033+-- the first array does not have enough space it may cause silent data
2034+-- corruption or if you are lucky a segfault.
2035+{-# INLINE spliceUnsafe #-}
2036+spliceUnsafe :: MonadIO m =>
2037+ Array a -> Array a -> m (Array a)
2038+spliceUnsafe dst src =
2039+ liftIO $ do
2040+ let startSrc = arrStart src
2041+ srcLen = arrEnd src - startSrc
2042+ endDst = arrEnd dst
2043+ assertM(endDst + srcLen <= arrBound dst)
2044+ putSliceUnsafe src startSrc dst endDst srcLen
2045+ return $ dst {arrEnd = endDst + srcLen}
2046+
2047+-- | @spliceWith sizer dst src@ mutates @dst@ to append @src@. If there is no
2048+-- reserved space available in @dst@ it is reallocated to a size determined by
2049+-- the @sizer dstBytes srcBytes@ function, where @dstBytes@ is the size of the
2050+-- first array and @srcBytes@ is the size of the second array, in bytes.
2051+--
2052+-- Note that the returned array may be a mutated version of first array.
2053+--
2054+-- /Pre-release/
2055+{-# INLINE spliceWith #-}
2056+spliceWith :: forall m a. (MonadIO m, Unbox a) =>
2057+ (Int -> Int -> Int) -> Array a -> Array a -> m (Array a)
2058+spliceWith sizer dst@(Array _ start end bound) src = do
2059+{-
2060+ let f = writeAppendWith (`sizer` byteLength src) (return dst)
2061+ in D.fold f (toStreamD src)
2062+-}
2063+ assert (end <= bound) (return ())
2064+ let srcBytes = arrEnd src - arrStart src
2065+
2066+ dst1 <-
2067+ if end + srcBytes >= bound
2068+ then do
2069+ let dstBytes = end - start
2070+ newSizeInBytes = sizer dstBytes srcBytes
2071+ when (newSizeInBytes < dstBytes + srcBytes)
2072+ $ error
2073+ $ "splice: newSize is less than the total size "
2074+ ++ "of arrays being appended. Please check the "
2075+ ++ "sizer function passed."
2076+ liftIO $ realloc newSizeInBytes dst
2077+ else return dst
2078+ spliceUnsafe dst1 src
2079+
2080+-- | The first array is mutated to append the second array. If there is no
2081+-- reserved space available in the first array a new allocation of exact
2082+-- required size is done.
2083+--
2084+-- Note that the returned array may be a mutated version of first array.
2085+--
2086+-- >>> splice = Array.spliceWith (+)
2087+--
2088+-- /Pre-release/
2089+{-# INLINE splice #-}
2090+splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
2091+splice = spliceWith (+)
2092+
2093+-- | Like 'append' but the growth of the array is exponential. Whenever a new
2094+-- allocation is required the previous array size is at least doubled.
2095+--
2096+-- This is useful to reduce allocations when folding many arrays together.
2097+--
2098+-- Note that the returned array may be a mutated version of first array.
2099+--
2100+-- >>> spliceExp = Array.spliceWith (\l1 l2 -> max (l1 * 2) (l1 + l2))
2101+--
2102+-- /Pre-release/
2103+{-# INLINE spliceExp #-}
2104+spliceExp :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
2105+spliceExp = spliceWith (\l1 l2 -> max (l1 * 2) (l1 + l2))
2106+
2107+-------------------------------------------------------------------------------
2108+-- Splitting
2109+-------------------------------------------------------------------------------
2110+
2111+-- | Drops the separator byte
2112+{-# INLINE breakOn #-}
2113+breakOn :: MonadIO m
2114+ => Word8 -> Array Word8 -> m (Array Word8, Maybe (Array Word8))
2115+breakOn sep arr@Array{..} = asPtrUnsafe arr $ \p -> liftIO $ do
2116+ -- XXX Instead of using asPtrUnsafe (pinning memory) we can pass unlifted
2117+ -- Addr# to memchr and it should be safe (from ghc 8.4).
2118+ -- XXX We do not need memchr here, we can use a Haskell equivalent.
2119+ loc <- c_memchr p sep (fromIntegral $ byteLength arr)
2120+ let sepIndex = loc `minusPtr` p
2121+ return $
2122+ if loc == nullPtr
2123+ then (arr, Nothing)
2124+ else
2125+ ( Array
2126+ { arrContents = arrContents
2127+ , arrStart = arrStart
2128+ , arrEnd = arrStart + sepIndex -- exclude the separator
2129+ , arrBound = arrStart + sepIndex
2130+ }
2131+ , Just $ Array
2132+ { arrContents = arrContents
2133+ , arrStart = arrStart + (sepIndex + 1)
2134+ , arrEnd = arrEnd
2135+ , arrBound = arrBound
2136+ }
2137+ )
2138+
2139+-- | Create two slices of an array without copying the original array. The
2140+-- specified index @i@ is the first index of the second slice.
2141+--
2142+splitAt :: forall a. Unbox a => Int -> Array a -> (Array a, Array a)
2143+splitAt i arr@Array{..} =
2144+ let maxIndex = length arr - 1
2145+ in if i < 0
2146+ then error "sliceAt: negative array index"
2147+ else if i > maxIndex
2148+ then error $ "sliceAt: specified array index " ++ show i
2149+ ++ " is beyond the maximum index " ++ show maxIndex
2150+ else let off = i * SIZE_OF(a)
2151+ p = arrStart + off
2152+ in ( Array
2153+ { arrContents = arrContents
2154+ , arrStart = arrStart
2155+ , arrEnd = p
2156+ , arrBound = p
2157+ }
2158+ , Array
2159+ { arrContents = arrContents
2160+ , arrStart = p
2161+ , arrEnd = arrEnd
2162+ , arrBound = arrBound
2163+ }
2164+ )
2165+
2166+-------------------------------------------------------------------------------
2167+-- Casting
2168+-------------------------------------------------------------------------------
2169+
2170+-- | Cast an array having elements of type @a@ into an array having elements of
2171+-- type @b@. The array size must be a multiple of the size of type @b@
2172+-- otherwise accessing the last element of the array may result into a crash or
2173+-- a random value.
2174+--
2175+-- /Pre-release/
2176+--
2177+castUnsafe ::
2178+#ifdef DEVBUILD
2179+ Unbox b =>
2180+#endif
2181+ Array a -> Array b
2182+castUnsafe (Array contents start end bound) =
2183+ Array contents start end bound
2184+
2185+-- | Cast an @Array a@ into an @Array Word8@.
2186+--
2187+asBytes :: Array a -> Array Word8
2188+asBytes = castUnsafe
2189+
2190+-- | Cast an array having elements of type @a@ into an array having elements of
2191+-- type @b@. The length of the array should be a multiple of the size of the
2192+-- target element otherwise 'Nothing' is returned.
2193+--
2194+cast :: forall a b. Unbox b => Array a -> Maybe (Array b)
2195+cast arr =
2196+ let len = byteLength arr
2197+ r = len `mod` SIZE_OF(b)
2198+ in if r /= 0
2199+ then Nothing
2200+ else Just $ castUnsafe arr
2201+
2202+-- XXX We can provide another API for "unsafe" FFI calls passing an unlifted
2203+-- pointer to the FFI call. For unsafe calls we do not need to pin the array.
2204+-- We can pass an unlifted pointer to the FFI routine to avoid GC kicking in
2205+-- before the pointer is wrapped.
2206+--
2207+-- From the GHC manual:
2208+--
2209+-- GHC, since version 8.4, guarantees that garbage collection will never occur
2210+-- during an unsafe call, even in the bytecode interpreter, and further
2211+-- guarantees that unsafe calls will be performed in the calling thread. Making
2212+-- it safe to pass heap-allocated objects to unsafe functions.
2213+
2214+-- Unsafe because of direct pointer operations. The user must ensure that they
2215+-- are writing within the legal bounds of the array. Should we just name it
2216+-- asPtr, the unsafety is implicit for any pointer operations. And we are safe
2217+-- from Haskell perspective because we will be pinning the memory.
2218+
2219+-- | Use an @Array a@ as @Ptr a@. This is useful when we want to pass an array
2220+-- as a pointer to some operating system call or to a "safe" FFI call.
2221+--
2222+-- If the array is not pinned it is copied to pinned memory before passing it
2223+-- to the monadic action.
2224+--
2225+-- /Performance Notes:/ Forces a copy if the array is not pinned. It is advised
2226+-- that the programmer keeps this in mind and creates a pinned array
2227+-- opportunistically before this operation occurs, to avoid the cost of a copy
2228+-- if possible.
2229+--
2230+-- /Unsafe/
2231+--
2232+-- /Pre-release/
2233+--
2234+asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
2235+asPtrUnsafe arr f = do
2236+ let contents = arrContents arr
2237+ !ptr = Ptr (byteArrayContents#
2238+ (unsafeCoerce# (getMutableByteArray# contents)))
2239+ -- XXX Check if the array is pinned, if not, copy it to a pinned array
2240+ -- XXX We should probably pass to the IO action the byte length of the array
2241+ -- as well so that bounds can be checked.
2242+ r <- f (ptr `plusPtr` arrStart arr)
2243+ liftIO $ touch contents
2244+ return r
2245+
2246+-------------------------------------------------------------------------------
2247+-- Equality
2248+-------------------------------------------------------------------------------
2249+
2250+-- | Compare the length of the arrays. If the length is equal, compare the
2251+-- lexicographical ordering of two underlying byte arrays
2252+--
2253+-- /Pre-release/
2254+{-# INLINE cmp #-}
2255+cmp :: MonadIO m => Array a -> Array a -> m Ordering
2256+cmp arr1 arr2 =
2257+ liftIO
2258+ $ do
2259+ let marr1 = getMutableByteArray# (arrContents arr1)
2260+ marr2 = getMutableByteArray# (arrContents arr2)
2261+ !(I# st1#) = arrStart arr1
2262+ !(I# st2#) = arrStart arr2
2263+ !(I# len#) = byteLength arr1
2264+ case compare (byteLength arr1) (byteLength arr2) of
2265+ EQ -> do
2266+ r <- liftIO $ IO $ \s# ->
2267+ let res =
2268+ I#
2269+ (compareByteArrays#
2270+ (unsafeCoerce# marr1)
2271+ st1#
2272+ (unsafeCoerce# marr2)
2273+ st2#
2274+ len#)
2275+ in (# s#, res #)
2276+ return $ compare r 0
2277+ x -> return x
2278+
2279+-------------------------------------------------------------------------------
2280+-- NFData
2281+-------------------------------------------------------------------------------
2282+
2283+-- | Strip elements which match with predicate from both ends.
2284+--
2285+-- /Pre-release/
2286+{-# INLINE strip #-}
2287+strip :: forall a m. (Unbox a, MonadIO m) =>
2288+ (a -> Bool) -> Array a -> m (Array a)
2289+strip eq arr@Array{..} = liftIO $ do
2290+ st <- getStart arrStart
2291+ end <- getLast arrEnd st
2292+ return arr {arrStart = st, arrEnd = end, arrBound = end}
2293+
2294+ where
2295+
2296+ {-
2297+ -- XXX This should have the same perf but it does not, investigate.
2298+ getStart = do
2299+ r <- liftIO $ D.head $ D.findIndices (not . eq) $ toStreamD arr
2300+ pure $
2301+ case r of
2302+ Nothing -> arrEnd
2303+ Just i -> PTR_INDEX(arrStart,i,a)
2304+ -}
2305+
2306+ getStart cur = do
2307+ if cur < arrEnd
2308+ then do
2309+ r <- peekWith arrContents cur
2310+ if eq r
2311+ then getStart (INDEX_NEXT(cur,a))
2312+ else return cur
2313+ else return cur
2314+
2315+ getLast cur low = do
2316+ if cur > low
2317+ then do
2318+ let prev = INDEX_PREV(cur,a)
2319+ r <- peekWith arrContents prev
2320+ if eq r
2321+ then getLast prev low
2322+ else return cur
2323+ else return cur
2324+
2325+-- | Given an array sorted in ascending order except the last element being out
2326+-- of order, use bubble sort to place the last element at the right place such
2327+-- that the array remains sorted in ascending order.
2328+--
2329+-- /Pre-release/
2330+{-# INLINE bubble #-}
2331+bubble :: (MonadIO m, Unbox a) => (a -> a -> Ordering) -> Array a -> m ()
2332+bubble cmp0 arr =
2333+ when (l > 1) $ do
2334+ x <- getIndexUnsafe (l - 1) arr
2335+ go x (l - 2)
2336+
2337+ where
2338+
2339+ l = length arr
2340+
2341+ go x i =
2342+ if i >= 0
2343+ then do
2344+ x1 <- getIndexUnsafe i arr
2345+ case x `cmp0` x1 of
2346+ LT -> do
2347+ putIndexUnsafe (i + 1) arr x1
2348+ go x (i - 1)
2349+ _ -> putIndexUnsafe (i + 1) arr x
2350+ else putIndexUnsafe (i + 1) arr x
core/src/Streamly/Internal/Data/Array/Type.hs+117−668
@@ -1,158 +1,76 @@
1−{-# LANGUAGE CPP #-}
2−{-# LANGUAGE TypeFamilies #-}
3−-- Must come after TypeFamilies, otherwise it is re-enabled.
4−-- MonoLocalBinds enabled by TypeFamilies causes perf regressions in general.
5−{-# LANGUAGE NoMonoLocalBinds #-}
61-- |
72-- Module : Streamly.Internal.Data.Array.Type
83-- Copyright : (c) 2020 Composewell Technologies
4+--
95-- License : BSD3-3-Clause
106-- Maintainer : streamly@composewell.com
117-- Stability : experimental
128-- Portability : GHC
139--
14−-- See notes in "Streamly.Internal.Data.MutArray.Type"
10+-- See notes in "Streamly.Internal.Data.Array.Mut.Type"
1511--
16−
1712module Streamly.Internal.Data.Array.Type
1813 (
19− -- ** Type
2014 -- $arrayNotes
2115 Array (..)
16+ , asPtrUnsafe
2217
23− -- ** Conversion
24− -- *** Mutable and Immutable
18+ -- * Freezing and Thawing
2519 , unsafeFreeze
2620 , unsafeFreezeWithShrink
2721 , unsafeThaw
2822
29− -- *** Pinned and Unpinned
23+ -- * Pinning and Unpinning
3024 , pin
3125 , unpin
32− , isPinned
33−
34− -- *** Casting
35− , unsafePinnedAsPtr
36−
37− -- ** Construction
38− , empty
3926
40− -- *** Cloning
41− -- XXX Why would we clone an immutable array?
42− , clone
43− , pinnedClone
27+ -- * Construction
28+ , splice
4429
45− -- *** Slicing
46− -- | Get a subarray without copying
47− , splitAt
48− , breakOn -- XXX requires MonadIO
49−
50− -- *** Stream Folds
51− , unsafeMakePure
52− , createOf
53− , pinnedCreateOf
54− , unsafeCreateOf
55− , unsafePinnedCreateOf
56− , create
57− , pinnedCreate
58− , createWith
59−
60− -- *** From containers
61− , fromListN
62− , pinnedFromListN
6330 , fromList
64− , pinnedFromList
65− , fromListRevN
31+ , fromListN
6632 , fromListRev
67− , fromStreamN
68− , fromStream
69− , fromPureStreamN
70− , fromPureStream
71− , fromByteStr#
72− , fromByteStr
73− , fromPtrN
74− , fromChunks
75− , fromChunksK
76−
77− -- ** Reading
78−
79− -- *** Indexing
80− , unsafeIndexIO -- XXX unsafeGetIndexIO
81− , getIndexUnsafe -- XXX unsafeGetIndex
82−
83− -- *** To Streams
84− , read
85− , readRev
86− , toStreamK
87− , toStreamKRev
88−
89− -- *** To Containers
90− , toList
33+ , fromListRevN
34+ , fromStreamDN
35+ , fromStreamD
9136
92− -- *** Unfolds
93− , producer -- experimental
94− , readerUnsafe
95− , reader
96− , readerRev
37+ -- * Split
38+ , breakOn
9739
98− -- *** Size
99− , length
40+ -- * Elimination
41+ , unsafeIndexIO
42+ , unsafeIndex -- getIndexUnsafe
10043 , byteLength
44+ , length
10145
102− -- ** Folding
10346 , foldl'
10447 , foldr
105− , byteCmp
106− , byteEq
107−
108− -- ** Appending
109− , splice -- XXX requires MonadIO
110−
111− -- ** Streams of arrays
112− -- *** Chunk
113− -- | Group a stream into arrays.
114− , chunksOf
115− , pinnedChunksOf
116− , buildChunks
117−
118− -- *** Split
119− -- | Split an array into slices.
120−
121− -- *** Concat
122− -- | Append the arrays in a stream to form a stream of elements.
123− , concat
124− , concatRev
125−
126− -- *** Compact
127− -- | Append the arrays in a stream to form a stream of larger arrays.
128− , fCompactGE
129− , fPinnedCompactGE
130− , lCompactGE
131− , lPinnedCompactGE
132− , compactGE
133−
134− -- ** Deprecated
135− , asPtrUnsafe
136− , unsafeIndex
137− , bufferChunks
138− , flattenArrays
139− , flattenArraysRev
140− , fromArrayStreamK
141− , fromStreamDN
142− , fromStreamD
48+ , splitAt
49+
14350 , toStreamD
14451 , toStreamDRev
52+ , toStreamK
53+ , toStreamKRev
14554 , toStream
14655 , toStreamRev
147− , nil
56+ , read
57+ , readRev
58+ , readerRev
59+ , toList
60+
61+ -- * Folds
14862 , writeWith
14963 , writeN
150− , pinnedWriteN
15164 , writeNUnsafe
152− , pinnedWriteNUnsafe
153− , pinnedWriteNAligned
65+ , MA.ArrayUnsafe (..)
66+ , writeNAligned
15467 , write
155− , pinnedWrite
68+
69+ -- * Streams of arrays
70+ , arraysOf
71+ , bufferChunks
72+ , flattenArrays
73+ , flattenArraysRev
15674 )
15775where
15876
@@ -160,60 +78,42 @@ where
16078#include "inline.hs"
16179
16280import Control.Exception (assert)
163−import Control.Monad (replicateM, when)
81+import Control.Monad (replicateM)
16482import Control.Monad.IO.Class (MonadIO(..))
16583import Data.Functor.Identity (Identity(..))
166−import Data.Int (Int8, Int16, Int32, Int64)
16784import Data.Proxy (Proxy(..))
168−import Data.Word (Word8, Word16, Word32, Word64)
85+import Data.Word (Word8)
16986import GHC.Base (build)
170−import GHC.Exts (IsList, IsString(..), Addr#)
87+import GHC.Exts (IsList, IsString(..))
17188
17289import GHC.IO (unsafePerformIO)
17390import GHC.Ptr (Ptr(..))
174−import Streamly.Internal.Data.Producer.Type (Producer(..))
175−import Streamly.Internal.Data.MutArray.Type (MutArray(..))
176−import Streamly.Internal.Data.MutByteArray.Type (MutByteArray)
91+import Streamly.Internal.Data.Array.Mut.Type (MutableByteArray)
17792import Streamly.Internal.Data.Fold.Type (Fold(..))
178−import Streamly.Internal.Data.Stream.Type (Stream)
179−import Streamly.Internal.Data.StreamK.Type (StreamK)
180−import Streamly.Internal.Data.Unbox (Unbox(..))
93+import Streamly.Internal.Data.Stream.StreamD.Type (Stream)
94+import Streamly.Internal.Data.Unboxed (Unbox, peekWith, sizeOf)
18195import Streamly.Internal.Data.Unfold.Type (Unfold(..))
18296import Text.Read (readPrec)
18397
184−import Prelude hiding (Foldable(..), concat, read, unlines, splitAt)
98+import Prelude hiding (length, foldr, read, unlines, splitAt)
18599
186100import qualified GHC.Exts as Exts
187−import qualified Streamly.Internal.Data.Fold.Type as Fold
188−import qualified Streamly.Internal.Data.MutArray.Type as MA
189−import qualified Streamly.Internal.Data.Stream.Type as D
190−import qualified Streamly.Internal.Data.StreamK.Type as K
191−import qualified Streamly.Internal.Data.MutByteArray.Type as Unboxed
192−import qualified Streamly.Internal.Data.Producer as Producer
101+import qualified Streamly.Internal.Data.Array.Mut.Type as MA
102+import qualified Streamly.Internal.Data.Stream.StreamD.Type as D
103+import qualified Streamly.Internal.Data.Stream.StreamK.Type as K
104+import qualified Streamly.Internal.Data.Unboxed as Unboxed
193105import qualified Streamly.Internal.Data.Unfold.Type as Unfold
194106import qualified Text.ParserCombinators.ReadPrec as ReadPrec
195107
196108import Streamly.Internal.System.IO (unsafeInlineIO, defaultChunkSize)
197109
198−#include "DocTestDataArray.hs"
199−
200−-------------------------------------------------------------------------------
201−-- Notes
202−-------------------------------------------------------------------------------
203−
204−-- IMPORTANT:
205−
206−-- We need to be careful while using unsafePerformIO when array creation is
207−-- involved.
208−--
209−-- * We need to make sure the unsafe IO line does not float out of the binding.
210−-- * The order of the IO actions should be sane. For example, `touch` after `f`.
211110--
212−-- Assume the unsafe IO action floats up. If it makes sense given this
213−-- assumption, it's probably OK to use usafe IO.
214−--
215−-- A general approach should be never to use unsafe IO where Array creation is
216−-- involved or touch is involved.
111+-- $setup
112+-- >>> :m
113+-- >>> :set -XMagicHash
114+-- >>> import Prelude hiding (length, foldr, read, unlines, splitAt)
115+-- >>> import Streamly.Data.Stream as Stream
116+-- >>> import Streamly.Internal.Data.Array as Array
217117
218118-------------------------------------------------------------------------------
219119-- Array Data Type
@@ -232,7 +132,7 @@ data Array a =
232132#endif
233133 -- All offsets are in terms of bytes from the start of arraycontents
234134 Array
235− { arrContents :: {-# UNPACK #-} !MutByteArray
135+ { arrContents :: {-# UNPACK #-} !MutableByteArray
236136 , arrStart :: {-# UNPACK #-} !Int -- offset
237137 , arrEnd :: {-# UNPACK #-} !Int -- offset + len
238138 }
@@ -243,20 +143,14 @@ data Array a =
243143
244144-- | Use an @Array a@ as @Ptr a@.
245145--
246−-- See 'MA.unsafePinnedAsPtr' in the Mutable array module for more details.
146+-- See 'MA.asPtrUnsafe' in the Mutable array module for more details.
247147--
248148-- /Unsafe/
249149--
250150-- /Pre-release/
251151--
252−{-# INLINE unsafePinnedAsPtr #-}
253−unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
254−unsafePinnedAsPtr arr = MA.unsafePinnedAsPtr (unsafeThaw arr)
255−
256−{-# DEPRECATED asPtrUnsafe "Please use unsafePinnedAsPtr instead." #-}
257−{-# INLINE asPtrUnsafe #-}
258152asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b
259−asPtrUnsafe = unsafePinnedAsPtr
153+asPtrUnsafe arr = MA.asPtrUnsafe (unsafeThaw arr)
260154
261155-------------------------------------------------------------------------------
262156-- Freezing and Thawing
@@ -276,15 +170,15 @@ asPtrUnsafe = unsafePinnedAsPtr
276170--
277171-- /Pre-release/
278172{-# INLINE unsafeFreeze #-}
279−unsafeFreeze :: MutArray a -> Array a
280−unsafeFreeze (MutArray ac as ae _) = Array ac as ae
173+unsafeFreeze :: MA.Array a -> Array a
174+unsafeFreeze (MA.Array ac as ae _) = Array ac as ae
281175
282176-- | Similar to 'unsafeFreeze' but uses 'MA.rightSize' on the mutable array
283177-- first.
284178{-# INLINE unsafeFreezeWithShrink #-}
285−unsafeFreezeWithShrink :: Unbox a => MutArray a -> Array a
179+unsafeFreezeWithShrink :: Unbox a => MA.Array a -> Array a
286180unsafeFreezeWithShrink arr = unsafePerformIO $ do
287− MutArray ac as ae _ <- MA.rightSize arr
181+ MA.Array ac as ae _ <- MA.rightSize arr
288182 return $ Array ac as ae
289183
290184-- | Makes a mutable array using the underlying memory of the immutable array.
@@ -296,42 +190,30 @@ unsafeFreezeWithShrink arr = unsafePerformIO $ do
296190--
297191-- /Pre-release/
298192{-# INLINE unsafeThaw #-}
299−unsafeThaw :: Array a -> MutArray a
300−unsafeThaw (Array ac as ae) = MutArray ac as ae ae
193+unsafeThaw :: Array a -> MA.Array a
194+unsafeThaw (Array ac as ae) = MA.Array ac as ae ae
301195
302196-------------------------------------------------------------------------------
303197-- Pinning & Unpinning
304198-------------------------------------------------------------------------------
305199
306−-- | Return a copy of the 'Array' in pinned memory if unpinned, else return the
307−-- original array.
308200{-# INLINE pin #-}
309201pin :: Array a -> IO (Array a)
310202pin = fmap unsafeFreeze . MA.pin . unsafeThaw
311203
312−-- | Return a copy of the 'Array' in unpinned memory if pinned, else return the
313−-- original array.
314204{-# INLINE unpin #-}
315205unpin :: Array a -> IO (Array a)
316206unpin = fmap unsafeFreeze . MA.unpin . unsafeThaw
317207
318−-- | Return 'True' if the array is allocated in pinned memory.
319−{-# INLINE isPinned #-}
320−isPinned :: Array a -> Bool
321−isPinned = MA.isPinned . unsafeThaw
322−
323208-------------------------------------------------------------------------------
324209-- Construction
325210-------------------------------------------------------------------------------
326211
327−-- | Copy two immutable arrays into a new array. If you want to splice more
328−-- than two arrays then this operation would be highly inefficient because it
329−-- would make a copy on every splice operation, instead use the
330−-- 'fromChunksK' operation to combine n immutable arrays.
212+-- Splice two immutable arrays creating a new array.
331213{-# INLINE splice #-}
332−splice :: MonadIO m => Array a -> Array a -> m (Array a)
214+splice :: (MonadIO m, Unbox a) => Array a -> Array a -> m (Array a)
333215splice arr1 arr2 =
334− unsafeFreeze <$> MA.spliceCopy (unsafeThaw arr1) (unsafeThaw arr2)
216+ unsafeFreeze <$> MA.splice (unsafeThaw arr1) (unsafeThaw arr2)
335217
336218-- | Create an 'Array' from the first N elements of a list. The array is
337219-- allocated to size N, if the list terminates before N elements then the
@@ -341,12 +223,6 @@ splice arr1 arr2 =
341223fromListN :: Unbox a => Int -> [a] -> Array a
342224fromListN n xs = unsafePerformIO $ unsafeFreeze <$> MA.fromListN n xs
343225
344−-- | Like 'fromListN' but creates a pinned array.
345−{-# INLINABLE pinnedFromListN #-}
346−pinnedFromListN :: Unbox a => Int -> [a] -> Array a
347−pinnedFromListN n xs =
348− unsafePerformIO $ unsafeFreeze <$> MA.pinnedFromListN n xs
349−
350226-- | Create an 'Array' from the first N elements of a list in reverse order.
351227-- The array is allocated to size N, if the list terminates before N elements
352228-- then the array may hold less than N elements.
@@ -362,11 +238,6 @@ fromListRevN n xs = unsafePerformIO $ unsafeFreeze <$> MA.fromListRevN n xs
362238fromList :: Unbox a => [a] -> Array a
363239fromList xs = unsafePerformIO $ unsafeFreeze <$> MA.fromList xs
364240
365−-- | Like 'fromList' but creates a pinned array.
366−{-# INLINE pinnedFromList #-}
367−pinnedFromList :: Unbox a => [a] -> Array a
368−pinnedFromList xs = unsafePerformIO $ unsafeFreeze <$> MA.pinnedFromList xs
369−
370241-- | Create an 'Array' from a list in reverse order. The list must be of finite
371242-- size.
372243--
@@ -375,197 +246,59 @@ pinnedFromList xs = unsafePerformIO $ unsafeFreeze <$> MA.pinnedFromList xs
375246fromListRev :: Unbox a => [a] -> Array a
376247fromListRev xs = unsafePerformIO $ unsafeFreeze <$> MA.fromListRev xs
377248
378−-- | Create an 'Array' from the first N elements of a stream. The array is
379−-- allocated to size N, if the stream terminates before N elements then the
380−-- array may hold less than N elements.
381−--
382−-- >>> fromStreamN n = Stream.fold (Array.writeN n)
383−--
384−-- /Pre-release/
385−{-# INLINE_NORMAL fromStreamN #-}
386−fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)
387−fromStreamN n m = do
388− when (n < 0) $ error "writeN: negative write count specified"
389− unsafeFreeze <$> MA.fromStreamN n m
390−-- fromStreamN n = D.fold (writeN n)
391−
392−{-# DEPRECATED fromStreamDN "Please use fromStreamN instead." #-}
249+{-# INLINE_NORMAL fromStreamDN #-}
393250fromStreamDN :: forall m a. (MonadIO m, Unbox a)
394251 => Int -> D.Stream m a -> m (Array a)
395−fromStreamDN = fromStreamN
252+fromStreamDN limit str = unsafeFreeze <$> MA.fromStreamDN limit str
396253
397−-- | Create an 'Array' from a stream. This is useful when we want to create a
398−-- single array from a stream of unknown size. 'writeN' is at least twice
399−-- as efficient when the size is already known.
400−--
401−-- >>> fromStream = Stream.fold Array.write
402−--
403−-- Note that if the input stream is too large memory allocation for the array
404−-- may fail. When the stream size is not known, `chunksOf` followed by
405−-- processing of indvidual arrays in the resulting stream should be preferred.
406−--
407−-- /Pre-release/
408254{-# INLINE_NORMAL fromStreamD #-}
409−fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)
410−fromStream = D.fold write
411−-- fromStreamD str = unsafeFreeze <$> MA.fromStream str
412−
413−{-# DEPRECATED fromStreamD "Please use fromStream instead." #-}
414255fromStreamD :: forall m a. (MonadIO m, Unbox a)
415256 => D.Stream m a -> m (Array a)
416−fromStreamD = fromStream
417−
418−-------------------------------------------------------------------------------
419−-- Cloning
420−-------------------------------------------------------------------------------
421−
422−{-# INLINE clone #-}
423−clone ::
424− ( MonadIO m
425−#ifdef DEVBUILD
426− , Unbox a
427−#endif
428− )
429− => Array a -> m (Array a)
430−clone = fmap unsafeFreeze . MA.clone . unsafeThaw
431−
432−{-# INLINE pinnedClone #-}
433−pinnedClone ::
434− ( MonadIO m
435−#ifdef DEVBUILD
436− , Unbox a
437−#endif
438− )
439− => Array a -> m (Array a)
440−pinnedClone = fmap unsafeFreeze . MA.pinnedClone . unsafeThaw
257+fromStreamD str = unsafeFreeze <$> MA.fromStreamD str
441258
442259-------------------------------------------------------------------------------
443260-- Streams of arrays
444261-------------------------------------------------------------------------------
445262
446−{-# INLINE buildChunks #-}
447−buildChunks :: (MonadIO m, Unbox a) =>
448− D.Stream m a -> m (K.StreamK m (Array a))
449−buildChunks m = D.foldr K.cons K.nil $ chunksOf defaultChunkSize m
450−
451−{-# DEPRECATED bufferChunks "Please use buildChunks instead." #-}
263+{-# INLINE bufferChunks #-}
452264bufferChunks :: (MonadIO m, Unbox a) =>
453− D.Stream m a -> m (K.StreamK m (Array a))
454−bufferChunks = buildChunks
265+ D.Stream m a -> m (K.Stream m (Array a))
266+bufferChunks m = D.foldr K.cons K.nil $ arraysOf defaultChunkSize m
455267
456−-- | @chunksOf n stream@ groups the elements in the input stream into arrays of
268+-- | @arraysOf n stream@ groups the elements in the input stream into arrays of
457269-- @n@ elements each.
458270--
459271-- Same as the following but may be more efficient:
460272--
461−-- >>> chunksOf n = Stream.foldMany (Array.writeN n)
273+-- >>> arraysOf n = Stream.foldMany (Array.writeN n)
462274--
463275-- /Pre-release/
464−{-# INLINE_NORMAL chunksOf #-}
465−chunksOf :: forall m a. (MonadIO m, Unbox a)
276+{-# INLINE_NORMAL arraysOf #-}
277+arraysOf :: forall m a. (MonadIO m, Unbox a)
466278 => Int -> D.Stream m a -> D.Stream m (Array a)
467−chunksOf n str = D.map unsafeFreeze $ MA.chunksOf n str
279+arraysOf n str = D.map unsafeFreeze $ MA.arraysOf n str
468280
469−-- | Like 'chunksOf' but creates pinned arrays.
470−{-# INLINE_NORMAL pinnedChunksOf #-}
471−pinnedChunksOf :: forall m a. (MonadIO m, Unbox a)
472− => Int -> D.Stream m a -> D.Stream m (Array a)
473−pinnedChunksOf n str = D.map unsafeFreeze $ MA.pinnedChunksOf n str
474−
475−-- | Convert a stream of arrays into a stream of their elements.
281+-- | Use the "read" unfold instead.
476282--
477−-- >>> concat = Stream.unfoldMany Array.reader
283+-- @flattenArrays = unfoldMany read@
478284--
479−{-# INLINE_NORMAL concat #-}
480−concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a
481−-- XXX this requires MonadIO whereas the unfoldMany version does not
482−concat = MA.concatWith (pure . unsafeInlineIO) . D.map unsafeThaw
483−-- concat = D.unfoldMany reader
484−
485−{-# DEPRECATED flattenArrays "Please use \"unfoldMany reader\" instead." #-}
486−{-# INLINE flattenArrays #-}
285+-- We can try this if there are any fusion issues in the unfold.
286+--
287+{-# INLINE_NORMAL flattenArrays #-}
487288flattenArrays :: forall m a. (MonadIO m, Unbox a)
488289 => D.Stream m (Array a) -> D.Stream m a
489−flattenArrays = concat
290+flattenArrays = MA.flattenArrays . D.map unsafeThaw
490291
491−-- | Convert a stream of arrays into a stream of their elements reversing the
492−-- contents of each array before flattening.
292+-- | Use the "readRev" unfold instead.
493293--
494−-- >>> concatRev = Stream.unfoldMany Array.readerRev
294+-- @flattenArrays = unfoldMany readRev@
495295--
496−{-# INLINE_NORMAL concatRev #-}
497−concatRev :: forall m a. (Monad m, Unbox a)
498− => D.Stream m (Array a) -> D.Stream m a
499−-- XXX this requires MonadIO whereas the unfoldMany version does not
500−concatRev = MA.concatRevWith (pure . unsafeInlineIO) . D.map unsafeThaw
501−-- concatRev = D.unfoldMany readerRev
502−
503−{-# DEPRECATED flattenArraysRev "Please use \"unfoldMany readerRev\" instead." #-}
504−{-# INLINE flattenArraysRev #-}
296+-- We can try this if there are any fusion issues in the unfold.
297+--
298+{-# INLINE_NORMAL flattenArraysRev #-}
505299flattenArraysRev :: forall m a. (MonadIO m, Unbox a)
506300 => D.Stream m (Array a) -> D.Stream m a
507−flattenArraysRev = concatRev
508−
509−-------------------------------------------------------------------------------
510−-- Compact
511−-------------------------------------------------------------------------------
512−
513−-- XXX Note that this thaws immutable arrays for appending, that may be
514−-- problematic if multiple users do the same thing, however, thawed immutable
515−-- arrays would have no capacity to append, therefore, a copy will be forced
516−-- anyway.
517−
518−-- | Fold @fCompactGE n@ coalesces adjacent arrays in the input stream
519−-- until the size becomes greater than or equal to n.
520−--
521−-- Generates unpinned arrays irrespective of the pinning status of input
522−-- arrays.
523−{-# INLINE_NORMAL fCompactGE #-}
524−fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
525−fCompactGE n = fmap unsafeFreeze $ Fold.lmap unsafeThaw $ MA.fCompactGE n
526−
527−-- | PInned version of 'fCompactGE'.
528−{-# INLINE_NORMAL fPinnedCompactGE #-}
529−fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)
530−fPinnedCompactGE n =
531− fmap unsafeFreeze $ Fold.lmap unsafeThaw $ MA.fPinnedCompactGE n
532−
533−-- | @compactGE n stream@ coalesces adjacent arrays in the @stream@ until
534−-- the size becomes greater than or equal to @n@.
535−--
536−-- >>> compactGE n = Stream.foldMany (Array.fCompactGE n)
537−--
538−-- Generates unpinned arrays irrespective of the pinning status of input
539−-- arrays.
540−{-# INLINE compactGE #-}
541−compactGE ::
542− (MonadIO m, Unbox a)
543− => Int -> Stream m (Array a) -> Stream m (Array a)
544−compactGE n stream =
545− D.map unsafeFreeze $ MA.compactGE n $ D.map unsafeThaw stream
546−
547−-- | Like 'compactGE' but for transforming folds instead of stream.
548−--
549−-- >>> lCompactGE n = Fold.many (Array.fCompactGE n)
550−--
551−-- Generates unpinned arrays irrespective of the pinning status of input
552−-- arrays.
553−{-# INLINE_NORMAL lCompactGE #-}
554−lCompactGE :: (MonadIO m, Unbox a)
555− => Int -> Fold m (Array a) () -> Fold m (Array a) ()
556−lCompactGE n fld =
557− Fold.lmap unsafeThaw $ MA.lCompactGE n (Fold.lmap unsafeFreeze fld)
558−
559−-- | Pinned version of 'lCompactGE'.
560−{-# INLINE_NORMAL lPinnedCompactGE #-}
561−lPinnedCompactGE :: (MonadIO m, Unbox a)
562− => Int -> Fold m (Array a) () -> Fold m (Array a) ()
563−lPinnedCompactGE n fld =
564− Fold.lmap unsafeThaw $ MA.lPinnedCompactGE n (Fold.lmap unsafeFreeze fld)
565−
566−-------------------------------------------------------------------------------
567−-- Splitting
568−-------------------------------------------------------------------------------
301+flattenArraysRev = MA.flattenArraysRev . D.map unsafeThaw
569302
570303-- Drops the separator byte
571304{-# INLINE breakOn #-}
@@ -587,14 +320,9 @@ unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a
587320unsafeIndexIO i arr = MA.getIndexUnsafe i (unsafeThaw arr)
588321
589322-- | Return element at the specified index without checking the bounds.
590−{-# INLINE_NORMAL getIndexUnsafe #-}
591−getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a
592−getIndexUnsafe i arr = let !r = unsafeInlineIO $ unsafeIndexIO i arr in r
593−
594−{-# DEPRECATED unsafeIndex "Please use 'getIndexUnsafe' instead" #-}
595323{-# INLINE_NORMAL unsafeIndex #-}
596324unsafeIndex :: forall a. Unbox a => Int -> Array a -> a
597−unsafeIndex = getIndexUnsafe
325+unsafeIndex i arr = let !r = unsafeInlineIO $ unsafeIndexIO i arr in r
598326
599327-- | /O(1)/ Get the byte length of the array.
600328--
@@ -609,71 +337,27 @@ byteLength = MA.byteLength . unsafeThaw
609337length :: Unbox a => Array a -> Int
610338length arr = MA.length (unsafeThaw arr)
611339
612−{-# INLINE_NORMAL producer #-}
613−producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a
614−producer =
615− Producer.translate unsafeThaw unsafeFreeze
616− $ MA.producerWith (return . unsafeInlineIO)
617−
618−-- | Unfold an array into a stream.
619−--
620−{-# INLINE_NORMAL reader #-}
621−reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
622−reader = Producer.simplify producer
623−
624−-- | Unfold an array into a stream, does not check the end of the array, the
625−-- user is responsible for terminating the stream within the array bounds. For
626−-- high performance application where the end condition can be determined by
627−-- a terminating fold.
628−--
629−-- Written in the hope that it may be faster than "read", however, in the case
630−-- for which this was written, "read" proves to be faster even though the core
631−-- generated with unsafeRead looks simpler.
632−--
633−-- /Pre-release/
634−--
635−{-# INLINE_NORMAL readerUnsafe #-}
636−readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
637−readerUnsafe = Unfold step inject
638− where
639−
640− inject (Array contents start end) =
641− return (MA.ArrayUnsafe contents end start)
642−
643− {-# INLINE_LATE step #-}
644− step (MA.ArrayUnsafe contents end p) = do
645− -- unsafeInlineIO allows us to run this in Identity monad for pure
646− -- toList/foldr case which makes them much faster due to not
647− -- accumulating the list and fusing better with the pure consumers.
648− --
649− -- This should be safe as the array contents are guaranteed to be
650− -- evaluated/written to before we peek at them.
651− let !x = unsafeInlineIO $ peekAt p contents
652− let !p1 = INDEX_NEXT(p,a)
653− return $ D.Yield x (MA.ArrayUnsafe contents end p1)
654−
655340-- | Unfold an array into a stream in reverse order.
656341--
657342{-# INLINE_NORMAL readerRev #-}
658343readerRev :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a
659344readerRev = Unfold.lmap unsafeThaw $ MA.readerRevWith (return . unsafeInlineIO)
660345
661−{-# DEPRECATED toStreamD "Please use 'read' instead." #-}
662346{-# INLINE_NORMAL toStreamD #-}
663347toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> D.Stream m a
664−toStreamD = read
348+toStreamD arr = MA.toStreamDWith (return . unsafeInlineIO) (unsafeThaw arr)
665349
666350{-# INLINE toStreamK #-}
667−toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> K.StreamK m a
351+toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> K.Stream m a
668352toStreamK arr = MA.toStreamKWith (return . unsafeInlineIO) (unsafeThaw arr)
669353
670−{-# DEPRECATED toStreamDRev "Please use 'readRev' instead." #-}
671354{-# INLINE_NORMAL toStreamDRev #-}
672355toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> D.Stream m a
673−toStreamDRev = readRev
356+toStreamDRev arr =
357+ MA.toStreamDRevWith (return . unsafeInlineIO) (unsafeThaw arr)
674358
675359{-# INLINE toStreamKRev #-}
676−toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> K.StreamK m a
360+toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> K.Stream m a
677361toStreamKRev arr =
678362 MA.toStreamKRevWith (return . unsafeInlineIO) (unsafeThaw arr)
679363
@@ -682,7 +366,7 @@ toStreamKRev arr =
682366-- /Pre-release/
683367{-# INLINE_EARLY read #-}
684368read :: (Monad m, Unbox a) => Array a -> Stream m a
685−read arr = MA.toStreamWith (return . unsafeInlineIO) (unsafeThaw arr)
369+read = toStreamD
686370
687371-- | Same as 'read'
688372--
@@ -699,7 +383,7 @@ toStream = read
699383-- /Pre-release/
700384{-# INLINE_EARLY readRev #-}
701385readRev :: (Monad m, Unbox a) => Array a -> Stream m a
702−readRev arr = MA.toStreamRevWith (return . unsafeInlineIO) (unsafeThaw arr)
386+readRev = toStreamDRev
703387
704388-- | Same as 'readRev'
705389--
@@ -723,7 +407,6 @@ foldr f z arr = runIdentity $ D.foldr f z $ toStreamD arr
723407-- | Create two slices of an array without copying the original array. The
724408-- specified index @i@ is the first index of the second slice.
725409--
726−{-# INLINE splitAt #-}
727410splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)
728411splitAt i arr = (unsafeFreeze a, unsafeFreeze b)
729412 where
@@ -743,8 +426,8 @@ toListFB c n Array{..} = go arrStart
743426 -- accumulating the list and fusing better with the pure consumers.
744427 --
745428 -- This should be safe as the array contents are guaranteed to be
746− -- evaluated/written to before we peekAt at them.
747− let !x = unsafeInlineIO $ peekAt p arrContents
429+ -- evaluated/written to before we peekWith at them.
430+ let !x = unsafeInlineIO $ peekWith arrContents p
748431 in c x (go (INDEX_NEXT(p,a)))
749432
750433-- | Convert an 'Array' into a list.
@@ -757,213 +440,47 @@ toList s = build (\c n -> toListFB c n s)
757440-- Folds
758441-------------------------------------------------------------------------------
759442
760−-- | @createOf n@ folds a maximum of @n@ elements from the input stream to an
443+-- | @writeN n@ folds a maximum of @n@ elements from the input stream to an
761444-- 'Array'.
762445--
763−{-# INLINE_NORMAL createOf #-}
764−createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
765−createOf = fmap unsafeFreeze . MA.createOf
766−
767−-- XXX Deprecate in major
768−-- {-# DEPRECATED writeN "Please use createOf instead." #-}
769−{-# INLINE writeN #-}
446+{-# INLINE_NORMAL writeN #-}
770447writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
771−writeN = createOf
772−
773−-- | Like 'createOf' but creates a pinned array.
774−{-# INLINE_NORMAL pinnedCreateOf #-}
775−pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
776−pinnedCreateOf = fmap unsafeFreeze . MA.pinnedCreateOf
448+writeN = fmap unsafeFreeze . MA.writeN
777449
778−{-# DEPRECATED pinnedWriteN "Please use pinnedCreateOf instead." #-}
779−{-# INLINE pinnedWriteN #-}
780−pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)
781−pinnedWriteN = pinnedCreateOf
782−
783−-- | @pinnedWriteNAligned alignment n@ folds a maximum of @n@ elements from the input
450+-- | @writeNAligned alignment n@ folds a maximum of @n@ elements from the input
784451-- stream to an 'Array' aligned to the given size.
785452--
786453-- /Pre-release/
787454--
788−{-# INLINE_NORMAL pinnedWriteNAligned #-}
789−{-# DEPRECATED pinnedWriteNAligned "To be removed." #-}
790−pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a)
455+{-# INLINE_NORMAL writeNAligned #-}
456+writeNAligned :: forall m a. (MonadIO m, Unbox a)
791457 => Int -> Int -> Fold m a (Array a)
792−pinnedWriteNAligned alignSize = fmap unsafeFreeze . MA.pinnedWriteNAligned alignSize
458+writeNAligned alignSize = fmap unsafeFreeze . MA.writeNAligned alignSize
793459
794−-- | Like 'createOf' but does not check the array bounds when writing. The fold
460+-- | Like 'writeN' but does not check the array bounds when writing. The fold
795461-- driver must not call the step function more than 'n' times otherwise it will
796462-- corrupt the memory and crash. This function exists mainly because any
797463-- conditional in the step function blocks fusion causing 10x performance
798464-- slowdown.
799465--
800−{-# INLINE_NORMAL unsafeCreateOf #-}
801−unsafeCreateOf :: forall m a. (MonadIO m, Unbox a)
802− => Int -> Fold m a (Array a)
803−unsafeCreateOf n = unsafeFreeze <$> MA.unsafeCreateOf n
804−
805−{-# DEPRECATED writeNUnsafe "Please use unsafeCreateOf instead." #-}
806−{-# INLINE writeNUnsafe #-}
466+{-# INLINE_NORMAL writeNUnsafe #-}
807467writeNUnsafe :: forall m a. (MonadIO m, Unbox a)
808468 => Int -> Fold m a (Array a)
809−writeNUnsafe = unsafeCreateOf
810−
811−{-# INLINE_NORMAL unsafePinnedCreateOf #-}
812−unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a)
813− => Int -> Fold m a (Array a)
814−unsafePinnedCreateOf n = unsafeFreeze <$> MA.unsafePinnedCreateOf n
815−
816−{-# DEPRECATED pinnedWriteNUnsafe "Please use unsafePinnedCreateOf instead." #-}
817−{-# INLINE pinnedWriteNUnsafe #-}
818−pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a)
819− => Int -> Fold m a (Array a)
820−pinnedWriteNUnsafe = unsafePinnedCreateOf
821−
822−{-# INLINE_NORMAL createWith #-}
823−createWith :: forall m a. (MonadIO m, Unbox a)
824− => Int -> Fold m a (Array a)
825−-- createWith n = FL.rmapM spliceArrays $ toArraysOf n
826−createWith elemCount = unsafeFreeze <$> MA.createWith elemCount
469+writeNUnsafe n = unsafeFreeze <$> MA.writeNUnsafe n
827470
828−{-# DEPRECATED writeWith "Please use createWith instead." #-}
829−{-# INLINE writeWith #-}
471+{-# INLINE_NORMAL writeWith #-}
830472writeWith :: forall m a. (MonadIO m, Unbox a)
831473 => Int -> Fold m a (Array a)
832−writeWith = createWith
474+-- writeWith n = FL.rmapM spliceArrays $ toArraysOf n
475+writeWith elemCount = unsafeFreeze <$> MA.writeWith elemCount
833476
834477-- | Fold the whole input to a single array.
835478--
836479-- /Caution! Do not use this on infinite streams./
837480--
838−{-# INLINE create #-}
839−create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
840−create = fmap unsafeFreeze MA.create
841−
842−-- XXX Deprecate in major
843−-- {-# DEPRECATED write "Please use create instead." #-}
844481{-# INLINE write #-}
845482write :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
846−write = create
847−
848−-- | Like 'create' but creates a pinned array.
849−{-# INLINE pinnedCreate #-}
850−pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
851−pinnedCreate = fmap unsafeFreeze MA.pinnedCreate
852−
853−{-# DEPRECATED pinnedWrite "Please use pinnedCreate instead." #-}
854−{-# INLINE pinnedWrite #-}
855−pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)
856−pinnedWrite = pinnedCreate
857−
858−-- | Fold "step" has a dependency on "initial", and each step is dependent on
859−-- the previous invocation of step due to state passing, finally extract
860−-- depends on the result of step, therefore, as long as the fold is driven in
861−-- the correct order the operations would be correctly ordered. We need to
862−-- ensure that we strictly evaluate the previous step completely before the
863−-- next step.
864−--
865−-- To not share the same array we need to make sure that the result of
866−-- "initial" is not shared. Existential type ensures that it does not get
867−-- shared across different folds. However, if we invoke "initial" multiple
868−-- times for the same fold, there is a possiblity of sharing the two because
869−-- the compiler would consider it as a pure value. One such example is the
870−-- chunksOf combinator, or using an array creation fold with foldMany
871−-- combinator. Is there a proper way in GHC to tell it to not share a pure
872−-- expression in a particular case?
873−--
874−-- For this reason array creation folds have a MonadIO constraint. Pure folds
875−-- could be unsafe and dangerous. This is dangerous especially when used with
876−-- foldMany like operations.
877−--
878−-- >>> unsafePureWrite = Array.unsafeMakePure Array.write
879−--
880−{-# INLINE unsafeMakePure #-}
881−unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b
882−unsafeMakePure (Fold step initial extract final) =
883− Fold (\x a -> return $! unsafeInlineIO (step x a))
884− (return $! unsafePerformIO initial)
885− (\s -> return $! unsafeInlineIO $ extract s)
886− (\s -> return $! unsafeInlineIO $ final s)
887−
888−{-# INLINE fromPureStreamN #-}
889−fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a
890−fromPureStreamN n x =
891− unsafePerformIO $ fmap unsafeFreeze (MA.fromPureStreamN n x)
892−
893−-- | Convert a pure stream in Identity monad to an immutable array.
894−--
895−-- Same as the following but with better performance:
896−--
897−-- >>> fromPureStream = Array.fromList . runIdentity . Stream.toList
898−--
899−fromPureStream :: Unbox a => Stream Identity a -> Array a
900−fromPureStream x = unsafePerformIO $ fmap (unsafeFreeze) (MA.fromPureStream x)
901−-- fromPureStream = runIdentity . D.fold (unsafeMakePure write)
902−-- fromPureStream = fromList . runIdentity . D.toList
903−
904−-- XXX This should be monadic.
905−
906−-- | Copy an immutable 'Ptr Word8' sequence into an array.
907−--
908−-- /Unsafe:/ The caller is responsible for safe addressing.
909−--
910−-- Note that this should be evaluated strictly to ensure that we do not hold
911−-- the reference to the pointer in a lazy thunk.
912−fromPtrN :: Int -> Ptr Word8 -> Array Word8
913−fromPtrN n addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromPtrN n addr)
914−
915−-- XXX This should be monadic.
916−
917−-- | Copy a null terminated immutable 'Addr#' Word8 sequence into an array.
918−--
919−-- /Unsafe:/ The caller is responsible for safe addressing.
920−--
921−-- Note that this is completely safe when reading from Haskell string
922−-- literals because they are guaranteed to be NULL terminated:
923−--
924−-- >>> Array.toList $ Array.fromByteStr# "\1\2\3\0"#
925−-- [1,2,3]
926−--
927−-- Note that this should be evaluated strictly to ensure that we do not hold
928−-- the reference to the pointer in a lazy thunk.
929−fromByteStr# :: Addr# -> Array Word8
930−fromByteStr# addr = unsafePerformIO $ fmap unsafeFreeze (MA.fromByteStr# addr)
931−
932−-- XXX This should be monadic.
933−
934−-- | Note that this should be evaluated strictly to ensure that we do not hold
935−-- the reference to the pointer in a lazy thunk.
936−fromByteStr :: Ptr Word8 -> Array Word8
937−fromByteStr (Ptr addr#) = fromByteStr# addr#
938−
939−-- XXX implement fromChunks/fromChunkList instead?
940−
941−-- | Convert an array stream to an array. Note that this requires peak memory
942−-- that is double the size of the array stream.
943−--
944−{-# INLINE fromChunksK #-}
945−fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)
946−fromChunksK stream =
947− -- We buffer the entire stream and then allocate the target array of the
948− -- same size, thus requiring double the memory.
949− fmap unsafeFreeze $ MA.fromChunksK $ fmap unsafeThaw stream
950−
951−{-# DEPRECATED fromArrayStreamK "Please use fromChunksK instead." #-}
952−fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (Array a) -> m (Array a)
953−fromArrayStreamK = fromChunksK
954−
955−-- | Given a stream of arrays, splice them all together to generate a single
956−-- array. The stream must be /finite/.
957−--
958−{-# INLINE fromChunks #-}
959−fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)
960−fromChunks s =
961− -- XXX Check which implementation is better
962− -- This may also require double the memory as we double the space every
963− -- time, when copying the last array we may have reallocated almost double
964− -- the space required before we right size it.
965− fmap unsafeFreeze $ MA.fromChunksRealloced (fmap unsafeThaw s)
966− -- fromChunkStreamK $ D.toStreamK s
483+write = fmap unsafeFreeze MA.write
967484
968485-------------------------------------------------------------------------------
969486-- Instances
@@ -995,66 +512,12 @@ instance Unbox a => IsList (Array a) where
995512 {-# INLINE toList #-}
996513 toList = toList
997514
998−-- | Byte compare two arrays. Compare the length of the arrays. If the length
999−-- is equal, compare the lexicographical ordering of two underlying byte arrays
1000−-- otherwise return the result of length comparison.
1001−--
1002−-- /Unsafe/: Note that the 'Unbox' instance of sum types with constructors of
1003−-- different sizes may leave some memory uninitialized which can make byte
1004−-- comparison unreliable.
1005−--
1006−-- /Pre-release/
1007−{-# INLINE byteCmp #-}
1008−byteCmp :: Array a -> Array a -> Ordering
1009−byteCmp arr1 arr2 =
1010− -- unsafePerformIO?
1011− unsafeInlineIO $! unsafeThaw arr1 `MA.byteCmp` unsafeThaw arr2
1012−
1013−-- | Byte equality of two arrays.
1014−--
1015−-- >>> byteEq arr1 arr2 = (==) EQ $ Array.byteCmp arr1 arr2
1016−--
1017−-- /Unsafe/: See 'byteCmp'.
1018−{-# INLINE byteEq #-}
1019−byteEq :: Array a -> Array a -> Bool
1020−byteEq arr1 arr2 = (==) EQ $ byteCmp arr1 arr2
1021−
1022−#define MK_EQ_INSTANCE(typ) \
1023−instance {-# OVERLAPPING #-} Eq (Array typ) where { \
1024−; {-# INLINE (==) #-} \
1025−; (==) = byteEq \
1026−}
1027−
1028−MK_EQ_INSTANCE(Char)
1029−MK_EQ_INSTANCE(Word8)
1030−MK_EQ_INSTANCE(Word16)
1031−MK_EQ_INSTANCE(Word32)
1032−
1033−-- XXX The Word64 default instance should be as fast because we are comparing
1034−-- 64-bit at a time.
1035−MK_EQ_INSTANCE(Word64)
1036−MK_EQ_INSTANCE(Int)
1037−MK_EQ_INSTANCE(Int8)
1038−MK_EQ_INSTANCE(Int16)
1039−MK_EQ_INSTANCE(Int32)
1040−
1041−-- XXX The Int64 default instance should be as fast.
1042−MK_EQ_INSTANCE(Int64)
1043−
1044−-- | If the type allows a byte-by-byte comparison this instance can be
1045−-- overlapped by a more specific instance that uses 'byteCmp'. Byte comparison
1046−-- can be significantly faster.
1047−--
1048−instance {-# OVERLAPPABLE #-} (Unbox a, Eq a) => Eq (Array a) where
515+-- XXX we are assuming that Unboxed equality means element equality. This may
516+-- or may not be correct? arrcmp is 40% faster compared to stream equality.
517+instance (Unbox a, Eq a) => Eq (Array a) where
1049518 {-# INLINE (==) #-}
1050519 arr1 == arr2 =
1051− -- Does unboxed byte equality mean element equality?
1052− -- XXX This is incorrect for sum types, as we may have some
1053− -- uninitialized memory in that case. If we always initialize the
1054− -- unused memory to zero we can use this.
1055− -- Byte comparison is 40% faster compared to stream equality.
1056− -- (==) EQ $ unsafeInlineIO $! unsafeThaw arr1 `MA.cmp` unsafeThaw arr2
1057− (toStreamD arr1 :: Stream Identity a) == toStreamD arr2
520+ (==) EQ $ unsafeInlineIO $! unsafeThaw arr1 `MA.cmp` unsafeThaw arr2
1058521
1059522instance (Unbox a, Ord a) => Ord (Array a) where
1060523 {-# INLINE compare #-}
@@ -1096,7 +559,7 @@ _toStreamD_ size Array{..} = D.Stream step arrStart
1096559 {-# INLINE_LATE step #-}
1097560 step _ p | p == arrEnd = return D.Stop
1098561 step _ p = liftIO $ do
1099− x <- peekAt p arrContents
562+ x <- peekWith arrContents p
1100563 return $ D.Yield x (p + size)
1101564
1102565{-
@@ -1119,29 +582,15 @@ instance Foldable Array where
1119582-- Semigroup and Monoid
1120583-------------------------------------------------------------------------------
1121584
1122−-- XXX Deprecate and remove the Semigroup and Monoid instances because of
1123−-- potential misuse chances.
1124−
1125−-- | This should not be used for combining many or N arrays as it would copy
1126−-- the two arrays everytime to a new array. For coalescing multiple arrays use
1127−-- 'fromChunksK' instead.
1128585instance Unbox a => Semigroup (Array a) where
1129586 arr1 <> arr2 = unsafePerformIO $ splice arr1 arr2
1130587
1131−empty ::
1132−#ifdef DEVBUILD
1133− Unbox a =>
1134−#endif
1135− Array a
1136−empty = Array Unboxed.empty 0 0
1137−
1138−{-# DEPRECATED nil "Please use empty instead." #-}
1139588nil ::
1140589#ifdef DEVBUILD
1141590 Unbox a =>
1142591#endif
1143592 Array a
1144−nil = empty
593+nil = Array Unboxed.nil 0 0
1145594
1146595instance Unbox a => Monoid (Array a) where
1147596 mempty = nil
core/src/Streamly/Internal/Data/Builder.hs+9−11
@@ -16,9 +16,7 @@ module Streamly.Internal.Data.Builder
1616 )
1717where
1818
19−#if !MIN_VERSION_base(4,18,0)
2019import Control.Applicative (liftA2)
21−#endif
2220
2321------------------------------------------------------------------------------
2422-- The Builder type
@@ -29,16 +27,16 @@ import Control.Applicative (liftA2)
2927-- or even a Fold. Unlike fold the step function is one-shot and not called in
3028-- a loop.
3129newtype Builder s m a =
32− Builder (s -> m (a, s))
30+ Builder (s -> m (s, a))
3331
3432-- | Maps a function on the output of the fold (the type @b@).
3533instance Functor m => Functor (Builder s m) where
3634 {-# INLINE fmap #-}
37− fmap f (Builder step1) = Builder (fmap (\ (a, s) -> (f a, s)) . step1)
35+ fmap f (Builder step1) = Builder (fmap (fmap f) . step1)
3836
3937{-# INLINE fromPure #-}
4038fromPure :: Applicative m => b -> Builder s m b
41−fromPure b = Builder (\s -> pure (b, s))
39+fromPure b = Builder (\s -> pure (s, b))
4240
4341-- | Chain the actions and zip the outputs.
4442{-# INLINE sequenceWith #-}
@@ -49,9 +47,9 @@ sequenceWith func (Builder stepL) (Builder stepR) = Builder step
4947 where
5048
5149 step s = do
52− (x, s1) <- stepL s
53− (y, s2) <- stepR s1
54− pure (func x y, s2)
50+ (s1, x) <- stepL s
51+ (s2, y) <- stepR s1
52+ pure (s2, func x y)
5553
5654instance Monad m => Applicative (Builder a m) where
5755 {-# INLINE pure #-}
@@ -76,7 +74,7 @@ instance Monad m => Monad (Builder a m) where
7674 where
7775
7876 step s = do
79− (x, s1) <- stepL s
77+ (s1, x) <- stepL s
8078 let Builder stepR = f x
81− (y, s2) <- stepR s1
82− pure (y, s2)
79+ (s2, y) <- stepR s1
80+ pure (s2, y)
core/src/Streamly/Internal/Data/Fold.hs+2507−11

This diff is too big to show here.

core/src/Streamly/Internal/Data/Fold/Chunked.hs+22−24

This diff is too big to show here.

core/src/Streamly/Internal/Data/Fold/Combinators.hsdeleted−2452

This diff is too big to show here.

core/src/Streamly/Internal/Data/Fold/Container.hs+40−119

This diff is too big to show here.

core/src/Streamly/Internal/Data/Fold/Step.hs+2−2

This diff is too big to show here.

core/src/Streamly/Internal/Data/Fold/Tee.hs+7−8

This diff is too big to show here.

core/src/Streamly/Internal/Data/Fold/Type.hs+167−402

This diff is too big to show here.

core/src/Streamly/Internal/Data/Fold/Window.hs+53−59

This diff is too big to show here.

core/src/Streamly/Internal/Data/IOFinalizer.hs+2−12

This diff is too big to show here.

core/src/Streamly/Internal/Data/IORef/Unboxed.hs+22−23

This diff is too big to show here.

core/src/Streamly/Internal/Data/IsMap.hs−8

This diff is too big to show here.

core/src/Streamly/Internal/Data/List.hs+16−81

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutArray.hsdeleted−207

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutArray/Generic.hsdeleted−947

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutArray/Stream.hsdeleted−145

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutArray/Type.hsdeleted−3318

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutByteArray.hsdeleted−251

This diff is too big to show here.

core/src/Streamly/Internal/Data/MutByteArray/Type.hsdeleted−321

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser.hs+594−2671

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/Chunked.hsnew file+327

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/Chunked/Type.hsnew file+329

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/ParserD.hsnew file+2247

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/Tee.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Tee.hsrenamed+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/Type.hs → core/src/Streamly/Internal/Data/Parser/ParserD/Type.hsrenamed+271−212

This diff is too big to show here.

core/src/Streamly/Internal/Data/Parser/ParserK/Type.hsnew file+313

This diff is too big to show here.

core/src/Streamly/Internal/Data/ParserK.hsdeleted−16

This diff is too big to show here.

core/src/Streamly/Internal/Data/ParserK/Type.hsdeleted−752

This diff is too big to show here.

core/src/Streamly/Internal/Data/Pipe.hs+10−7

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer.hs+13−6

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer/Source.hs+19−34

This diff is too big to show here.

core/src/Streamly/Internal/Data/Producer/Type.hs+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Refold/Type.hs+2−2

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring.hs+31−603

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring/Generic.hsdeleted−185

This diff is too big to show here.

core/src/Streamly/Internal/Data/Ring/Unboxed.hsnew file+615

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH.hsdeleted−530

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hsdeleted−475

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/Common.hsdeleted−70

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/TH/RecHeader.hsdeleted−411

This diff is too big to show here.

core/src/Streamly/Internal/Data/Serialize/Type.hsdeleted−317

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream.hs+3−31

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Bottom.hsnew file+670

This diff is too big to show here.

core/src/Streamly/Internal/Data/Array/Stream.hs → core/src/Streamly/Internal/Data/Stream/Chunked.hsrenamed+581−92

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Common.hs → core/src/Streamly/Internal/Data/Stream/Common.hsrenamed+12−12

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Cross.hsnew file+143

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Eliminate.hs+228−697

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Enumerate.hsnew file+560

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Exception.hs+90−369

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Expand.hsnew file+893

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Generate.hs+217−948

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Lift.hs+36−70

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Reduce.hsnew file+444

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD.hs+29−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Container.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Container.hsrenamed+63−70

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Eliminate.hsnew file+826

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Exception.hsnew file+480

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Generate.hsnew file+1211

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Lift.hsnew file+131

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Nesting.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Nesting.hsrenamed+252−154

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Step.hs → core/src/Streamly/Internal/Data/Stream/StreamD/Step.hsrenamed+2−2

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Top.hsnew file+356

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Transform.hsnew file+1844

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Transformer.hsnew file+185

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamD/Type.hsnew file+2022

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/StreamDK.hsnew file+52

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK.hs → core/src/Streamly/Internal/Data/Stream/StreamK.hsrenamed+249−713

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Alt.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Alt.hsrenamed+1−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Transformer.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Transformer.hsrenamed+8−8

This diff is too big to show here.

core/src/Streamly/Internal/Data/StreamK/Type.hs → core/src/Streamly/Internal/Data/Stream/StreamK/Type.hsrenamed+256−336

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Top.hsdeleted−480

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Transform.hs+282−1197

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Transformer.hs+43−91

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Type.hs+229−1958

This diff is too big to show here.

core/src/Streamly/Internal/Data/Stream/Zip.hsnew file+91

This diff is too big to show here.

core/src/Streamly/Internal/Data/Time/Clock.hs+3−3

This diff is too big to show here.

core/src/Streamly/Internal/Data/Time/Units.hs+4−1

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unbox/TH.hsdeleted−498

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unbox.hs → core/src/Streamly/Internal/Data/Unboxed.hsrenamed+276−344

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold.hs+194−32

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold/Enumeration.hs+45−14

This diff is too big to show here.

core/src/Streamly/Internal/Data/Unfold/Type.hs+20−127

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Dir.hs+23−70

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/File.hs+32−31

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Handle.hs+52−41

This diff is too big to show here.

core/src/Streamly/Internal/FileSystem/Path.hsdeleted−515

This diff is too big to show here.

core/src/Streamly/Internal/Data/Binary/Parser.hs → core/src/Streamly/Internal/Serialize/FromBytes.hsrenamed+13−29

This diff is too big to show here.

core/src/Streamly/Internal/Data/Binary/Stream.hs → core/src/Streamly/Internal/Serialize/ToBytes.hsrenamed+9−31

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Array.hs−6

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Parser.hs+25−354

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/Stream.hs+68−79

This diff is too big to show here.

core/src/Streamly/Internal/Unicode/String.hs+12−17

This diff is too big to show here.

core/src/Streamly/Unicode/Parser.hs+2−12

This diff is too big to show here.

core/src/Streamly/Unicode/Stream.hs+10−14

This diff is too big to show here.

core/src/Streamly/Unicode/String.hs+1−15

This diff is too big to show here.

core/src/inline.hs+1−1

This diff is too big to show here.

core/streamly-core.cabal+115−197

This diff is too big to show here.

docs/Developer/Benchmarks.md−8

This diff is too big to show here.

docs/Developer/MAINTAINING.md+10−7

This diff is too big to show here.

docs/Developer/module-organization.md+1−1

This diff is too big to show here.

docs/Developer/optimization-guidelines.md+1−13

This diff is too big to show here.

docs/Developer/paths.rst+2−25

This diff is too big to show here.

docs/Documentation.mddeleted−5

This diff is too big to show here.

docs/User/Explanatory/design.mddeleted−42

This diff is too big to show here.

docs/User/HowTo/Compiling.md+14−36

This diff is too big to show here.

docs/User/HowTo/streamly-vs-async.md−2

This diff is too big to show here.

docs/User/HowTo/streamly-vs-lists.md−2

This diff is too big to show here.

docs/User/Project/ApiChangelogs/0.8.3-0.9.0.txtdeleted−175

This diff is too big to show here.

docs/User/Project/ApiChangelogs/0.9.0-0.10.0.txtdeleted−424

This diff is too big to show here.

docs/User/Project/README.mddeleted−141

This diff is too big to show here.

docs/User/Project/Upgrading-0.8-to-0.9.mddeleted−384

This diff is too big to show here.

docs/User/Project/ApiChangelogs/0.8.3.txt → docs/User/ProjectRelated/API-changelog.txtrenamed

The content is the same.

docs/User/Project/CONTRIBUTORS.md → docs/User/ProjectRelated/CONTRIBUTORS.mdrenamed+1−9

This diff is too big to show here.

docs/User/Project/Changelog.md → docs/User/ProjectRelated/Changelog.mdrenamed+32−92

This diff is too big to show here.

docs/User/Project/Credits.md → docs/User/ProjectRelated/Credits.mdrenamed

The content is the same.

docs/User/Project/Github.link → docs/User/ProjectRelated/Github.linkrenamed

The content is the same.

docs/User/ProjectRelated/README.mdnew file+166

This diff is too big to show here.

docs/User/Project/Roadmap.link → docs/User/ProjectRelated/Roadmap.linkrenamed

The content is the same.

docs/User/Tutorials/ConcurrentStreams.hs−2

This diff is too big to show here.

docs/User/Tutorials/ReactiveProgramming.hs−2

This diff is too big to show here.

docs/User/Tutorials/Tutorial.hs−2

This diff is too big to show here.

docs/User/Tutorials/before-you-begin.mddeleted−77

This diff is too big to show here.

docs/User/Tutorials/haskell-lazy-evaluation.mddeleted−215

This diff is too big to show here.

docs/User/Tutorials/quick-overview.md+42−21

This diff is too big to show here.

docs/User/Tutorials/getting-started.md → docs/User/Tutorials/setup-and-usage.mdrenamed+25−28

This diff is too big to show here.

docs/User/Tutorials/why-haskell-streamly.md−5

This diff is too big to show here.

docs/streamly-docs.cabal+6−8

This diff is too big to show here.

hie.yaml+14−16

This diff is too big to show here.

src/Streamly.hsnew file+624

This diff is too big to show here.

src/Streamly/Data/Fold/Prelude.hsdeleted−65

This diff is too big to show here.

src/Streamly/Data/Fold/Tee.hs+1−2

This diff is too big to show here.

src/Streamly/Data/Stream/Concurrent.hsnew file+184

This diff is too big to show here.

src/Streamly/Data/Stream/Exception.hsnew file+24

This diff is too big to show here.

src/Streamly/Data/Stream/MkType.hs+10−41

This diff is too big to show here.

src/Streamly/Data/Stream/Prelude.hs+15−217

This diff is too big to show here.

src/Streamly/Data/Stream/Time.hsnew file+37

This diff is too big to show here.

src/Streamly/Data/Unicode/Stream.hsnew file+99

This diff is too big to show here.

src/Streamly/Internal/Control/ForkLifted.hs+4−17

This diff is too big to show here.

src/Streamly/Internal/Data/Array/ArrayMacros.hnew file+40

This diff is too big to show here.

src/Streamly/Internal/Data/Channel.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Async.hs+6−18

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent.hs+49−37

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent/Channel.hs+10−31

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Concurrent/Channel/Type.hs+7−8

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/Prelude.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Fold/SVar.hs+7−8

This diff is too big to show here.

src/Streamly/Internal/Data/IsMap/HashMap.hs−10

This diff is too big to show here.

src/Streamly/Internal/Data/SVar.hs−1

This diff is too big to show here.

src/Streamly/Internal/Data/SmallArray.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Ahead.hs+3−15

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Async.hs+3−22

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Dispatcher.hs → src/Streamly/Internal/Data/Stream/Channel/Dispatcher.hsrenamed+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Types.hs → src/Streamly/Internal/Data/Stream/Channel/Types.hsrenamed+32−22

This diff is too big to show here.

src/Streamly/Internal/Data/Channel/Worker.hs → src/Streamly/Internal/Data/Stream/Channel/Worker.hsrenamed+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent.hs+90−219

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel.hs+19−22

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Append.hs+37−37

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Consumer.hs+2−2

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Dispatcher.hs+3−3

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Interleave.hs+14−16

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Operations.hs+8−8

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Concurrent/Channel/Type.hs+5−5

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Lifted.hs → src/Streamly/Internal/Data/Stream/Exception/Lifted.hsrenamed+12−16

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Instances.hs+9−6

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream.hs+2−8

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Combinators.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Common.hs+8−8

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Eliminate.hs+11−13

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Enumeration.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Exception.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Expand.hs+4−8

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Generate.hs+7−7

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Lift.hs+1−1

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Reduce.hs+16−15

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Top.hs+5−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Transform.hs+5−5

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/IsStream/Type.hs+21−22

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/MkType.hs+34−33

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Parallel.hs+19−31

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Prelude.hsdeleted−19

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar.hsdeleted−18

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar/Eliminate.hs+4−4

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/SVar/Generate.hs+6−8

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Serial.hs+6−19

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Time.hs+10−10

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Zip.hs+3−42

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/Zip/Concurrent.hsnew file+50

This diff is too big to show here.

src/Streamly/Internal/Data/Stream/ZipAsync.hs+4−5

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/Exception.hs+3−1

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/Prelude.hsdeleted−17

This diff is too big to show here.

src/Streamly/Internal/Data/Unfold/SVar.hs+2−3

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event.hs+2−5

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Darwin.hs+5−5

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Linux.hs+9−15

This diff is too big to show here.

src/Streamly/Internal/FileSystem/Event/Windows.hs+8−11

This diff is too big to show here.

src/Streamly/Internal/FileSystem/FD.hs+15−16

This diff is too big to show here.

src/Streamly/Internal/Network/Inet/TCP.hs+48−84

This diff is too big to show here.

src/Streamly/Internal/Network/Socket.hs+31−34

This diff is too big to show here.

src/Streamly/Internal/System/IOVec.hs+7−7

This diff is too big to show here.

src/Streamly/Internal/Unicode/Char.hs+8−10

This diff is too big to show here.

src/Streamly/Memory/Array.hsnew file+74

This diff is too big to show here.

src/Streamly/Network/Inet/TCP.hs+30−41

This diff is too big to show here.

src/Streamly/Network/Socket.hs+30−16

This diff is too big to show here.

src/Streamly/Prelude.hs+4−4

This diff is too big to show here.

src/inline.hs+1−1

This diff is too big to show here.

stack.yaml+2−1

This diff is too big to show here.

streamly.cabal+90−117

This diff is too big to show here.

targets/Targets.hs+16−17

This diff is too big to show here.

test/Streamly/Test/Data/Array.hs+10−10

This diff is too big to show here.

test/Streamly/Test/Data/MutArray.hs → test/Streamly/Test/Data/Array/Mut.hsrenamed+5−5

This diff is too big to show here.

test/Streamly/Test/Data/Array/Stream.hs+3−4

This diff is too big to show here.

test/Streamly/Test/Data/Fold.hs+8−8

This diff is too big to show here.

test/Streamly/Test/Data/Fold/Window.hs+9−8

This diff is too big to show here.

test/Streamly/Test/Data/Parser.hs+141−282

This diff is too big to show here.

test/Streamly/Test/Data/Parser/Chunked.hsnew file+1239

This diff is too big to show here.

test/Streamly/Test/Data/ParserK.hs → test/Streamly/Test/Data/Parser/ParserD.hsrenamed+14−19

This diff is too big to show here.

test/Streamly/Test/Data/Ring/Unboxed.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Data/Serialize.hsdeleted−306

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/CompatV0.hsdeleted−46

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/CompatV1.hsdeleted−48

This diff is too big to show here.

test/Streamly/Test/Data/Serialize/TH.hsdeleted−82

This diff is too big to show here.

test/Streamly/Test/Data/Stream.hsdeleted−33

This diff is too big to show here.

test/Streamly/Test/Data/Stream/Concurrent.hs+82−126

This diff is too big to show here.

test/Streamly/Test/Data/Unbox.hs+50−318

This diff is too big to show here.

test/Streamly/Test/Data/Unbox/TH.hsdeleted−146

This diff is too big to show here.

test/Streamly/Test/Data/Unfold.hs+2−2

This diff is too big to show here.

test/Streamly/Test/FileSystem/Event/Common.hs+3−5

This diff is too big to show here.

test/Streamly/Test/FileSystem/Event/Linux.hs+3−16

This diff is too big to show here.

test/Streamly/Test/FileSystem/Handle.hs+2−2

This diff is too big to show here.

test/Streamly/Test/Network/Inet/TCP.hs+4−5

This diff is too big to show here.

test/Streamly/Test/Network/Socket.hs+4−5

This diff is too big to show here.

test/Streamly/Test/Prelude/Serial.hs+35−42

This diff is too big to show here.

test/Streamly/Test/Prelude/Top.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Serialize/Serializable.hsdeleted−237

This diff is too big to show here.

test/Streamly/Test/Unicode/Char.hs+1−1

This diff is too big to show here.

test/Streamly/Test/Unicode/Parser.hsdeleted−231

This diff is too big to show here.

test/Streamly/Test/Unicode/Stream.hs+28−30

This diff is too big to show here.

test/lib/Streamly/Test/Common.hs−4

This diff is too big to show here.

test/lib/Streamly/Test/Prelude/Common.hs+2−9

This diff is too big to show here.

test/streamly-tests.cabal+29−99

This diff is too big to show here.