git.iko.soy/streamly
on master help sign in
update FileIO section
8b9b1aeHarendra Kumar
README.md+83−44
compare with master
unifiedsplit
@@ -397,64 +397,103 @@ powerful ways of composing and applying them.
397
397
398## File IO
398## File IO
399
399
400−The following code snippet implements some common Unix command line utilities
400+The following code snippets implement some common Unix command line utilities
401−using streamly. To get an idea about IO streaming performance, you can
401+using streamly. You can compile these with `ghc -O2
402−benchmark these against the regular unix utilities using the `time` command.
402+-fspec-constr-recursive=10` and compare the performance with regular GNU
403−Make sure to use a big enough input file and compile with
403+coreutils available on your system. Source file
404−`ghc -O2 -fspec-constr-recursive=10` when benchmarking. Use `+RTS -s` flags on
404+[HandleIO.hs](https://github.com/composewell/streamly/tree/master/examples/HandleIO.hs)
405−the executable to check the space usage, look for `maximum residency` in the
405+in the examples directory includes these examples.
406−output.
407−
408−Note that `grep -c` counts the number of lines where the pattern occurs whereas
409−the snippet below counts the total number of occurrences of the pattern,
410−therefore, the output may differ.
411
406
412``` haskell
407``` haskell
408+module Main where
409+
413import qualified Streamly.Prelude as S
410import qualified Streamly.Prelude as S
414import qualified Streamly.Fold as FL
411import qualified Streamly.Fold as FL
415import qualified Streamly.Mem.Array as A
412import qualified Streamly.Mem.Array as A
416−import qualified Streamly.FileSystem.File as File
413+import qualified Streamly.FileSystem.Handle as FH
414+import qualified System.IO as FH
417
415
418import Data.Char (ord)
416import Data.Char (ord)
419import System.Environment (getArgs)
417import System.Environment (getArgs)
420import System.IO (openFile, IOMode(..), stdout)
418import System.IO (openFile, IOMode(..), stdout)
421
419
420+withArg f = do
421+ (name : _) <- getArgs
422+ src <- openFile name ReadMode
423+ f src
424+
425+withArg2 f = do
426+ (sname : dname : _) <- getArgs
427+ src <- openFile sname ReadMode
428+ dst <- openFile dname WriteMode
429+ f src dst
430+```
431+
432+### cat
433+
434+``` haskell
422−cat src = File.writeArrays stdout $ File.readArraysUpto (256*1024) src
435+cat = FH.writeArrays stdout . FH.readArraysOfUpto (256*1024)
436+main = withArg cat
437+```
438+
439+### cp
440+
441+``` haskell
423−cp src dst = File.writeArrays dst $ File.readArraysUpto (256*1024) src
442+cp src dst = FH.writeArrays dst $ FH.readArraysOfUpto (256*1024) src
424−wcl src = print =<<
443+main = withArg2 cp
444+```
445+
446+### wc -l
447+
448+``` haskell
449+wcl =
425− ( S.length
450+ S.length
426− $ FL.splitSuffixBy (== fromIntegral (ord '\n')) FL.drain
451+ . FL.splitSuffixBy (== fromIntegral (ord '\n')) FL.drain
427− $ File.read src)
452+ . FH.read
428−grepc pat src = print . (subtract 1) =<<
453+main = withArg wcl >>= print
454+```
455+
456+### grep -c
457+
458+Note that `grep -c` counts the number of lines where the pattern occurs whereas
459+the snippet below counts the total number of occurrences of the pattern,
460+therefore, the output may differ.
461+
462+``` haskell
463+grepc pat =
429− ( S.length
464+ S.length
430− $ FL.splitOn (A.fromList (map (fromIntegral . ord) pat)) FL.drain
465+ . FL.splitOn (A.fromList (map (fromIntegral . ord) pat)) FL.drain
431− $ File.read src)
466+ . FH.read
467+main = withArg (grepc "pattern") >>= print . subtract 1
468+```
469+
470+### Average Line Length
471+
472+``` haskell
432−avgll src = print =<<
473+avgll =
433− ( FL.foldl' avg
474+ FL.foldl' avg
434− $ FL.splitSuffixBy (== fromIntegral (ord '\n')) FL.length
475+ . FL.splitSuffixBy (== fromIntegral (ord '\n')) FL.length
435− $ File.read src)
476+ . FH.read
477+
436− where avg = (/) <$> toDouble FL.sum <*> toDouble FL.length
478+ where avg = (/) <$> toDouble FL.sum <*> toDouble FL.length
437 toDouble = fmap (fromIntegral :: Int -> Double)
479 toDouble = fmap (fromIntegral :: Int -> Double)
438−llhisto src = print =<<
439− ( FL.foldl' (FL.classify FL.length)
440− $ S.map bucket
441− $ FL.splitSuffixBy (== fromIntegral (ord '\n')) FL.length
442− $ File.read src)
443− where
444− bucket n = let i = n `div` 10 in if i > 9 then (9,n) else (i,n)
445
480
446−main = do
481+main = withArg avgll >>= print
447− name <- fmap head getArgs
482+```
448− src <- openFile name ReadMode
449− -- cat src -- Unix cat program
450− -- wcl src -- Unix wc -l program
451− -- grepc "aaaa" src -- Unix grep -c program
452
483
453− -- dst <- openFile "dst.txt" WriteMode
484+### Line Length Histogram
454− -- cp src dst -- Unix cp program
485+
486+``` haskell
487+llhisto =
488+ FL.foldl' (FL.classify FL.length)
489+ . S.map bucket
490+ . FL.splitSuffixBy (== fromIntegral (ord '\n')) FL.length
491+ . FH.read
492+
493+ where
494+ bucket n = let i = n `div` 10 in if i > 9 then (9,n) else (i,n)
455
495
456− -- avgll src -- get average line length
496+main = withArg llhisto >>= print
457− llhisto src -- get line length histogram
458```
497```
459
498
460## Socket IO
499## Socket IO