e265852 · streamly · git.iko.soy

git.iko.soy/streamly

Add skeleton for groupByRollingEither

The driving use case for this is to make the sorting faster by detecting sorted sequences in the input.

commite265852671d2bcbb764fea45fb5a50322de79762 authorHarendra Kumar <harendra@composewell.com> dateMon 27 Sep 2021, 16:11 (5 years ago) parent6865c64

3 files changed+24

compare with master

unifiedsplit

benchmark/Streamly/Benchmark/Prelude/Merge.hs+5 src/Streamly/Internal/Data/Parser.hs+16 src/Streamly/Internal/Data/Stream/IsStream/Top.hs+3

benchmark/Streamly/Benchmark/Prelude/Merge.hs+5

@@ -238,6 +238,11 @@ o_n_heap_concat value =

238238 "concatPairsWithMergeByFused compare (sqrtVal of sqrtVal)"

239239 (concatPairsWithMergeByFused compare sqrtVal sqrtVal)

240240 ]

241+ -- TODO: change sourceUnfoldrM to generate alternating bigger and lower

242+ -- numbers to simulate a random input for a worst case sort benchmark. We

243+ -- can use 0 and value as two ints in the state and alternate each in the

244+ -- output streams, incrementing the lower number of decrementing the higher

245+ -- number.

241246 , bgroup "sorting"

242247 [ benchIOSink value "sortBy compare" (sortBy compare)

243248 , benchIOSink value "sortBy (flip compare)" (sortBy (flip compare))

src/Streamly/Internal/Data/Parser.hs+16

@@ -97,6 +97,7 @@ module Streamly.Internal.Data.Parser

9797 , wordBy

9898 , groupBy

9999 , groupByRolling

100+ , groupByRollingEither

100101 , eqBy

101102 -- | Unimplemented

102103 --

@@ -699,6 +700,21 @@ groupBy eq = K.toParserK . D.groupBy eq

699700groupByRolling :: MonadCatch m => (a -> a -> Bool) -> Fold m a b -> Parser m a b

700701groupByRolling eq = K.toParserK . D.groupByRolling eq

701702

703+-- | Like 'groupByRolling', but if the predicate is 'True' then collects using

704+-- the first fold as long as the predicate holds 'True', if the predicate is

705+-- 'False' collects using the second fold as long as it remains 'False'.

706+-- Returns 'Left' for the first case and 'Right' for the second case.

707+--

708+-- For example, if we want to detect sorted sequences in a stream, both

709+-- ascending and descending cases we can use 'groupByRollingEither (<=)

710+-- Fold.toList Fold.toList'.

711+--

712+-- /Unimplemented/

713+{-# INLINABLE groupByRollingEither #-}

714+groupByRollingEither :: -- MonadCatch m =>

715+ (a -> a -> Bool) -> Fold m a b -> Fold m a b -> Parser m a (Either b b)

716+groupByRollingEither = undefined

717+

702718-- | Match the given sequence of elements using the given comparison function.

703719--

704720-- >>> Stream.parse (Parser.eqBy (==) "string") $ Stream.fromList "string"

src/Streamly/Internal/Data/Stream/IsStream/Top.hs+3

@@ -181,6 +181,9 @@ sampleBurstStart gap =

181181-- input stream is almost sorted (ascending/descending) or random. We could

182182-- serialize the stream to an array and use quicksort.

183183--

184+-- Use 'groupByRollingEither (\x -> (< GT) . f x) Fold.toList Fold.toListRev'

185+-- to generate sorted segments before merging. Compare the perf.

186+--

184187-- | Sort the input stream using a supplied comparison function.

185188--

186189-- /O(n) space/