composewell/streamly
1
1
Fork 0
mirror of https://github.com/composewell/streamly.git synced 2024-09-20 07:58:27 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,219 Commits 488 Branches 34 Tags 28 MiB
1475c2fb7d
Commit Graph

1219 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Harendra Kumar
1475c2fb7d Move Streamly.Unfold to Streamly.Internal.Data.Unfold 2019-09-24 21:28:25 +05:30
Harendra Kumar
2af9c97500 move Unfold module files to internal 2019-09-24 21:14:51 +05:30
Harendra Kumar
ae847b821f remove fold API exports from Streamly.Internal 
They are now exported via Streamly.Internal.Data.Fold
 2019-09-24 21:14:51 +05:30
Harendra Kumar
885a66f45a merge Streamly.Fold.Internal with Streamly.Internal.Data.Fold 2019-09-24 21:14:51 +05:30
Harendra Kumar
042b677592 Move Streamly.Fold module 
* Move exposed Streamly.Fold to Streamly.Data.Fold
* Move the internal implementation to Streamly.Inetrnal.Data.Fold
 2019-09-24 21:14:51 +05:30
Harendra Kumar
a8dd99dc75 move Fold modules to Internal 2019-09-24 20:55:49 +05:30
Harendra Kumar
ee6be006b2 Stop exposing internal Prelude functions via Internal 
They are now exposed via Internal.Prelude
 2019-09-24 20:55:41 +05:30
Harendra Kumar
6b98e076d2 Fix build with "dev" flag 2019-09-24 20:55:31 +05:30
Harendra Kumar
b051167f42 Use Internal.Prelude module in place of Internal 2019-09-24 20:55:07 +05:30
Harendra Kumar
beeed6b1c7 import Streamly.Internal.Prelude in place of Streamly.Prelude.Internal 2019-09-24 20:54:28 +05:30
Harendra Kumar
c1aa574d1e move Prelude/Internal.hs to Internal.Prelude.hs 2019-09-24 20:49:24 +05:30
Harendra Kumar
aea974c0d2 import the renamed ArrayStream module 2019-09-24 15:32:02 +05:30
Harendra Kumar
52d0bfdaa3 Move ArrayStream to Internal module 2019-09-24 15:26:43 +05:30
Harendra Kumar
b96d73517e move touch at the end of the loop 2019-09-21 01:06:09 +05:30
David Feuer
b438759fec Improve demuxing 
* Use `alterF` to speed up `Map` modification.

* Bump lower bound on `containers`.

* Make `demuxWith` and `demuxWith_` just powerful enough to implement
`demux` and `demux_` without extra mapping. We could actually go
further if we wanted.
 2019-09-20 23:42:07 +05:30
David Feuer
f312ccb596 Cleanup cleanup 2019-09-20 23:39:30 +05:30
Harendra Kumar
3a4f72af7e Simplify and optimize utf-8 decoding 
* Simplify the state machine to carry minimal state
* reduce branches taken and improve branch prediction

With this change decode-utf8 has almost the same performance as decodeChar8 for
the ascii case. For the mixed unicode chars case as well the performance is
pretty good and not too far away from the ascii case.

Stream and array versions both fuse to similar core which pretty small and both
cases show exactly the same performance.
 2019-09-20 01:46:39 +05:30
Harendra Kumar
069728c621 Use cacheline aligned lookup table 
It makes a barely noticeable difference.
 2019-09-19 08:59:59 +05:30
Harendra Kumar
e12c86463f make some performance improvements for utf8 decoding 
1) Change the Codepoint type to Int for cheaper conversion to Char
2) Add strictness annotations in decode
3) Reorder the constructor arguments
4) Move foreign ptr touch to only once after the array is done
 2019-09-19 08:59:59 +05:30
Harendra Kumar
ef779032d9 pass the utf-8 lookup table instead of using it as global 
The core gets simplified by 10x (2000 lines vs 200 lines).
Stream benchmarks improve considerably readStream/utf8 changed from 12 sec to 7
sec.
 2019-09-19 08:59:59 +05:30
Harendra Kumar
00e979329b Add support for aligned array creation. 2019-09-19 08:59:59 +05:30
Harendra Kumar
d6f08aa6d7 Fast path the ascii utf8 decoding case 
Gives more than 2x (600 ms vs 1300 ms) performance boost on mostly ascii files.
 2019-09-19 08:59:59 +05:30
Harendra Kumar
e692bad3fe Make the utf-8 lookup table more compact 
It should improve cache efficiency. Though measured improvement is modest,
5-6%.
 2019-09-19 08:59:59 +05:30
Harendra Kumar
9e46ec65fc Add some tests and benchmarks for decode-utf8 2019-09-19 08:59:59 +05:30
Pranay Sashank
3b44db7522 Fix passing of state in decodeUtf8Lenient. 2019-09-19 08:59:59 +05:30
Harendra Kumar
f5d83bc866 Add benchmark for decodeUtf8Lenient 2019-09-19 08:59:59 +05:30
Pranay Sashank
3e30855a93 Add link and copyright to the decode function. 2019-09-19 08:59:59 +05:30
Pranay Sashank
3544b94864 Add decodeUtf8ArraysLenient, strictify fields in encodeUtf8. 
- Change decodeUtf8 to new implementation.
- Strictify fields in flattenArrays, ~30% perf improvement.
- concatMapU readU instead of flattenArrays, ~10% perf loss.
 2019-09-19 08:59:59 +05:30
Pranay Sashank
4337d196f1 Add decodeUtf8Arrays, simplify decodeUtf8. 2019-09-19 08:59:59 +05:30
Pranay Sashank
1a12bb130d Add decodeUtf8Lenient to String module. 2019-09-19 08:59:59 +05:30
Harendra Kumar
5da4037cba update the upper bound of clock in maxrate test 2019-09-18 13:02:23 +05:30
Harendra Kumar
349c9d7a64 Use an IORef for the counts in hashmap 
This reduces the garbage generated by frequently updating the hashmap.
 2019-09-18 10:26:14 +05:30
Harendra Kumar
df9335085d disable -funfolding-use-threshold option in fileio 
This option gets good results on some benchmarks (notably lines-unlines for
streams) but other benchmarks degrade. So disabling for now.
 2019-09-16 21:46:55 +05:30
Harendra Kumar
c3ff5d378a replace unsafeInlineIO with unsafePerformIO 
Since we are performing a side-effect (malloc) inside we need to ensure that it
will be performed uniquely every time the code is called.
 2019-09-16 14:19:11 +05:30
Harendra Kumar
661f84262e Add some word/unword, word counting benchmarks 2019-09-15 23:48:21 +05:30
Harendra Kumar
0b2f615930 Add splitInnerBySuffix to split lines in inner container 2019-09-15 23:24:39 +05:30
Harendra Kumar
53a6a873c0 Add a splitInnerBy combinator to split chunked streams 2019-09-15 23:09:46 +05:30
Harendra Kumar
4a367b8d99 Use -funfolding-use-threshold=150 in fileio benchmark 
The lines-unlines benchmark does not fuse well without this option.
 2019-09-14 18:57:43 +05:30
Harendra Kumar
d4495c3298 shrink an array only if the wastage is more than 25% 2019-09-14 18:57:06 +05:30
Harendra Kumar
1c339acfc6 Hide some array stream combinators 
* Those that can already be expressed in terms of streams
* Those that are not used anywhere, if needed they can be expressed in terms of
* streams.
 2019-09-13 14:02:58 +05:30
Harendra Kumar
006fc5fcfe Reallocate only once in lpackArraysChunksOf 2019-09-13 11:44:19 +05:30
Harendra Kumar
7a8227eae4 implement Array intercalate in terms of stream op 
We do not need a custom Array unlines function anymore. We can just use stream
operations to do that job in general.
 2019-09-13 07:43:29 +05:30
Harendra Kumar
7cf3784326 refactor to allow using array write fold in fromStreamD 
Move the "write" fold to the Array.Types module so that we can use it in
"fromStreamD"/fromList. However, it is found to be slightly less efficient than
the current implementation using "fromStreamD".
 2019-09-10 21:37:01 +05:30
Harendra Kumar
b38198c3f5 Add support to switch to C malloc vs GHC malloc 2019-09-10 21:37:01 +05:30
Harendra Kumar
7d9c88b06f Use String instead of Array as word type 
improves performance and memory utilization significantly.
 2019-09-10 15:36:51 +05:30
Harendra Kumar
d1a12545ae Fix orphan instance warning 2019-09-10 15:18:12 +05:30
Harendra Kumar
6f3fda8a4a Make toLower and isAlpha more efficient 2019-09-06 23:02:22 +05:30
Harendra Kumar
b9e99db0bd Add word classifier example 2019-09-06 00:38:53 +05:30
Harendra Kumar
f503496a4d Add rolling hash folds 2019-09-05 20:43:45 +05:30
Harendra Kumar
307fa9f5d5 Add direct style append/interleave/concatUnfoldInterleave etc 2019-09-04 13:03:07 +05:30