composewell/streamly
1
1
Fork 0
mirror of https://github.com/composewell/streamly.git synced 2024-09-19 23:48:00 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,250 Commits 488 Branches 34 Tags 28 MiB
51f12c9d22
Commit Graph

1250 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Harendra Kumar
ef779032d9 pass the utf-8 lookup table instead of using it as global 
The core gets simplified by 10x (2000 lines vs 200 lines).
Stream benchmarks improve considerably readStream/utf8 changed from 12 sec to 7
sec.
 2019-09-19 08:59:59 +05:30
Harendra Kumar
00e979329b Add support for aligned array creation. 2019-09-19 08:59:59 +05:30
Harendra Kumar
d6f08aa6d7 Fast path the ascii utf8 decoding case 
Gives more than 2x (600 ms vs 1300 ms) performance boost on mostly ascii files.
 2019-09-19 08:59:59 +05:30
Harendra Kumar
e692bad3fe Make the utf-8 lookup table more compact 
It should improve cache efficiency. Though measured improvement is modest,
5-6%.
 2019-09-19 08:59:59 +05:30
Harendra Kumar
9e46ec65fc Add some tests and benchmarks for decode-utf8 2019-09-19 08:59:59 +05:30
Pranay Sashank
3b44db7522 Fix passing of state in decodeUtf8Lenient. 2019-09-19 08:59:59 +05:30
Harendra Kumar
f5d83bc866 Add benchmark for decodeUtf8Lenient 2019-09-19 08:59:59 +05:30
Pranay Sashank
3e30855a93 Add link and copyright to the decode function. 2019-09-19 08:59:59 +05:30
Pranay Sashank
3544b94864 Add decodeUtf8ArraysLenient, strictify fields in encodeUtf8. 
- Change decodeUtf8 to new implementation.
- Strictify fields in flattenArrays, ~30% perf improvement.
- concatMapU readU instead of flattenArrays, ~10% perf loss.
 2019-09-19 08:59:59 +05:30
Pranay Sashank
4337d196f1 Add decodeUtf8Arrays, simplify decodeUtf8. 2019-09-19 08:59:59 +05:30
Pranay Sashank
1a12bb130d Add decodeUtf8Lenient to String module. 2019-09-19 08:59:59 +05:30
Harendra Kumar
5da4037cba update the upper bound of clock in maxrate test 2019-09-18 13:02:23 +05:30
Harendra Kumar
349c9d7a64 Use an IORef for the counts in hashmap 
This reduces the garbage generated by frequently updating the hashmap.
 2019-09-18 10:26:14 +05:30
Harendra Kumar
df9335085d disable -funfolding-use-threshold option in fileio 
This option gets good results on some benchmarks (notably lines-unlines for
streams) but other benchmarks degrade. So disabling for now.
 2019-09-16 21:46:55 +05:30
Harendra Kumar
c3ff5d378a replace unsafeInlineIO with unsafePerformIO 
Since we are performing a side-effect (malloc) inside we need to ensure that it
will be performed uniquely every time the code is called.
 2019-09-16 14:19:11 +05:30
Harendra Kumar
661f84262e Add some word/unword, word counting benchmarks 2019-09-15 23:48:21 +05:30
Harendra Kumar
0b2f615930 Add splitInnerBySuffix to split lines in inner container 2019-09-15 23:24:39 +05:30
Harendra Kumar
53a6a873c0 Add a splitInnerBy combinator to split chunked streams 2019-09-15 23:09:46 +05:30
Harendra Kumar
4a367b8d99 Use -funfolding-use-threshold=150 in fileio benchmark 
The lines-unlines benchmark does not fuse well without this option.
 2019-09-14 18:57:43 +05:30
Harendra Kumar
d4495c3298 shrink an array only if the wastage is more than 25% 2019-09-14 18:57:06 +05:30
Harendra Kumar
1c339acfc6 Hide some array stream combinators 
* Those that can already be expressed in terms of streams
* Those that are not used anywhere, if needed they can be expressed in terms of
* streams.
 2019-09-13 14:02:58 +05:30
Harendra Kumar
006fc5fcfe Reallocate only once in lpackArraysChunksOf 2019-09-13 11:44:19 +05:30
Harendra Kumar
7a8227eae4 implement Array intercalate in terms of stream op 
We do not need a custom Array unlines function anymore. We can just use stream
operations to do that job in general.
 2019-09-13 07:43:29 +05:30
Harendra Kumar
7cf3784326 refactor to allow using array write fold in fromStreamD 
Move the "write" fold to the Array.Types module so that we can use it in
"fromStreamD"/fromList. However, it is found to be slightly less efficient than
the current implementation using "fromStreamD".
 2019-09-10 21:37:01 +05:30
Harendra Kumar
b38198c3f5 Add support to switch to C malloc vs GHC malloc 2019-09-10 21:37:01 +05:30
Harendra Kumar
7d9c88b06f Use String instead of Array as word type 
improves performance and memory utilization significantly.
 2019-09-10 15:36:51 +05:30
Harendra Kumar
d1a12545ae Fix orphan instance warning 2019-09-10 15:18:12 +05:30
Harendra Kumar
6f3fda8a4a Make toLower and isAlpha more efficient 2019-09-06 23:02:22 +05:30
Harendra Kumar
b9e99db0bd Add word classifier example 2019-09-06 00:38:53 +05:30
Harendra Kumar
f503496a4d Add rolling hash folds 2019-09-05 20:43:45 +05:30
Harendra Kumar
307fa9f5d5 Add direct style append/interleave/concatUnfoldInterleave etc 2019-09-04 13:03:07 +05:30
Harendra Kumar
f030c0a462 Clarify monad transformers' usage recommendation 2019-09-04 13:01:17 +05:30
Harendra Kumar
874a2017b1 update comments on stream types 2019-09-04 12:40:24 +05:30
Harendra Kumar
cafa593cfe update the library overview in the cabal file 2019-09-02 09:56:22 +05:30
Harendra Kumar
cccc7a4f34 Add a direct style version of StreamK 2019-08-30 13:43:27 +05:30
Harendra Kumar
f0c11b5490 implement foldr/build fusion for unfoldr/foldrM 2019-08-30 12:38:57 +05:30
Harendra Kumar
a014617f5e
Merge pull request #261 from composewell/streamly-module-doc 
update documentation for different stream types
 2019-08-30 12:36:11 +05:30
Harendra Kumar
379732da24 use bench-show 0.3 2019-08-27 23:34:35 +05:30
Harendra Kumar
c6f8df019d disable inspection tests for haddock build 2019-08-27 22:39:34 +05:30
Harendra Kumar
1a8bc296ae Add the inspection testing flag in CI 2019-08-27 22:39:33 +05:30
Harendra Kumar
6f06bdd7fe move the concatMap inspection test to benchmarks 2019-08-27 21:29:57 +05:30
Harendra Kumar
f5614c725d separate inspection testing build from benchmark build 
Sometimes we may want to just build benchmarks without doing inspection
testing, for example GHC head version cannot build inspection-testing package,
but we still want to run benchmarks.
 2019-08-27 21:02:44 +05:30
Harendra Kumar
7c9c1c48b5 Add support for GHC 8.8 
Fix warnings.
Add a travis CI build matrix entry
 2019-08-27 08:53:25 +05:30
Harendra Kumar
f2fe19185b Add Prelude.arraysOf to Internal module for testing 2019-08-26 08:18:47 +05:30
Harendra Kumar
ca7490737e remove unnecessary ops from Streamly.Internal 
arraysOf can now as efficiently expressed by chunksOf
writeN can be used in place of writeNUnsafe with -funfolding-use-threshold
write is now as efficient as writeS in all cases
 2019-08-26 08:00:08 +05:30
Harendra Kumar
c0d4e04a1d Use (concatMapU A.readU) instead of flattenArrays 
Use a modular composition using concatMap and Unfold to express efficient
flattening of arrays into a stream instead of using a custom flattenArrays. The
generated code is almost identical in both the cases and measured performance
is also similar.

Also simplified readU to use one less variable threaded around the loop.
 2019-08-26 07:38:34 +05:30
Harendra Kumar
bca03590a8 Add -funfolding-use-threshold GHC option 
update the optimization options' description.
 2019-08-25 12:40:10 +05:30
Harendra Kumar
1b8439d641 change writeNUnsafe to use strict unboxed data 
Now all benchmarks that use A.writeN/A.writeNUnsafe with concatMap perform as
well as the custom flattenArray implementation.
 2019-08-25 10:57:19 +05:30
Harendra Kumar
0ef35be05b Add inspection tests for some benchmarks (eq/cmp/merge/zip) 2019-08-23 17:43:00 +05:30
Harendra Kumar
11b10b9c0c Move LinearOps.hs benchmark file to streamly library 
To facilitate inspection testing of all benchmarks.
 2019-08-23 14:57:29 +05:30