@@ -7,53 +7,82 @@
7-- Stability : released
7-- Stability : released
8-- Portability : GHC
8-- Portability : GHC
10−-- This module provides immutable arrays in pinned memory (non GC memory)
10+-- This module provides APIs to create and use unboxed immutable arrays. Once
11−-- suitable for long lived data storage, random access and for interfacing with
11+-- created their contents cannot be modified. Only types that are unboxable via
12−-- the operating system.
12+-- the 'Unbox' type class can be stored in these arrays.
14−-- Arrays in this module are chunks of pinned memory that hold a sequence of
15−-- 'Storable' values of a given type, they cannot store non-serializable data
16−-- like functions. Once created an array cannot be modified. Pinned memory
17−-- allows efficient buffering of long lived data without adding any impact to
18−-- GC. One array is just one pointer visible to GC and it does not have to be
19−-- copied across generations. Moreover, pinned memory allows communication
20−-- with foreign consumers and producers (e.g. file or network IO) without
25−-- Array creation APIs require a 'MonadIO' Monad, except 'fromList' which is a
26−-- pure API. To operate on streams in pure Monads like 'Identity' you can hoist
27−-- it to IO monad as follows:
29−-- >>> import Data.Functor.Identity (Identity, runIdentity)
30−-- >>> s = Stream.fromList [1..10] :: Stream Identity Int
31−-- >>> s1 = Stream.hoist (return . runIdentity) s :: Stream IO Int
32−-- >>> Stream.fold Array.write s1 :: IO (Array Int)
33−-- fromList [1,2,3,4,5,6,7,8,9,10]
35−-- 'unsafePerformIO' can be used to get a pure API from IO, as long as you know
36−-- it is safe to do so:
38−-- >>> import System.IO.Unsafe (unsafePerformIO)
14+-- = Folding Arrays
39−-- >>> unsafePerformIO $ Stream.fold Array.write s1 :: Array Int
15+--
16+-- Convert array to stream, and fold the stream:
18+-- >>> fold f arr = Stream.unfold Array.reader arr & Stream.fold f
19+-- >>> fold Fold.sum (Array.fromList [1,2,3::Int])
22+-- = Transforming Arrays
24+-- Convert array to stream, transform, and fold back to array:
26+-- >>> amap f arr = Stream.unfold Array.reader arr & fmap f & Stream.fold Array.write
27+-- >>> amap (+1) (Array.fromList [1,2,3::Int])
40−-- fromList [1,2,3,4,5,6,7,8,9,10]
28+-- fromList [2,3,4]
30+-- = Pinned and Unpinned Arrays
32+-- The array type can use both pinned and unpinned memory under the hood.
33+-- Currently the array creation APIs create arrays in pinned memory but it will
34+-- change to unpinned in future releases. The change should not affect users
35+-- functionally unless they are directly accessing the internal memory of the
36+-- array via internal APIs. As of now unpinned arrays can be created using
39+-- Unpinned arrays have the advantage of allowing automatic defragmentation of
40+-- the memory by GC. Whereas pinned arrays have the advantage of not requiring
41+-- a copy by GC. Normally you would want to use unpinned arrays. However, in
42+-- some cases, for example, for long lived large data storage, and for
43+-- interfacing with the operating system or foreign (non-Haskell) consumers you
44+-- may want to use pinned arrays.
46+-- = Creating Arrays from Non-IO Streams
48+-- Array creation folds require 'MonadIO' because they need to sequence effects
49+-- in IO streams. To operate on streams in pure Monads like 'Identity' you can
50+-- hoist it to IO monad as follows:
42−-- To apply a transformation to an array use 'read' to unfold the array into a
52+-- The 'MonadIO' based folds can be hoisted to 'Identity' stream folds:
43−-- stream, apply a transformation on the stream and then use 'write' to fold it
53+--
44−-- back to an array.
54+-- >>> purely = Fold.hoist (Identity . unsafePerformIO)
55+-- >>> Stream.fold (purely Array.write) $ Stream.fromList [1,2,3::Int]
56+-- Identity fromList [1,2,3]
58+-- Alternatively, 'Identity' streams can be generalized to IO streams:
60+-- >>> pure = Stream.fromList [1,2,3] :: Stream Identity Int
61+-- >>> generally = Stream.hoist (return . runIdentity)
62+-- >>> Stream.fold Array.write (generally pure :: Stream IO Int)
65+-- Since it is a pure stream we can use 'unsafePerformIO' to extract the result
46-- This module is designed to be imported qualified:
70-- This module is designed to be imported qualified:
48−-- > import qualified Streamly.Data.Array as Array
72+-- >>> import qualified Streamly.Data.Array as Array
74+-- Use @QuantifiedConstraints@ extension to avoid the 'Storable' constraint
75+-- when using the 'Unbox' constraint.
50-- For experimental APIs see "Streamly.Internal.Data.Array".
77-- For experimental APIs see "Streamly.Internal.Data.Array".
52module Streamly.Data.Array
79module Streamly.Data.Array
54− -- * Types
81+ -- * Imports for Examples
55− Unbox (..)
82+ -- $setup
58 -- * Construction
87 -- * Construction
59 -- | When performance matters, the fastest way to generate an array is
88 -- | When performance matters, the fastest way to generate an array is
@@ -90,6 +119,9 @@ module Streamly.Data.Array
90 -- , (!!)
119 -- , (!!)
91 , A.getIndex
120 , A.getIndex
122+ -- * Unbox Type Class
93 -- * Deprecated
125 -- * Deprecated
95 , readRev
127 , readRev
@@ -107,10 +139,16 @@ import Prelude hiding (read)
109-- >>> :set -XFlexibleContexts
141-- >>> :set -XFlexibleContexts
110−-- >>> import Streamly.Internal.Data.Stream (Stream)
142+-- >>> import Data.Function ((&))
143+-- >>> import Data.Functor.Identity (Identity(..))
111-- >>> import Streamly.Data.Array (Array)
144-- >>> import Streamly.Data.Array (Array)
112−-- >>> import qualified Streamly.Internal.Data.Stream as Stream
145+-- >>> import Streamly.Data.Stream (Stream)
146+-- >>> import System.IO.Unsafe (unsafePerformIO)
113-- >>> import qualified Streamly.Data.Array as Array
147-- >>> import qualified Streamly.Data.Array as Array
148+-- >>> import qualified Streamly.Data.Fold as Fold
149+-- >>> import qualified Streamly.Internal.Data.Fold as Fold (hoist)
150+-- >>> import qualified Streamly.Data.Stream as Stream
151+-- >>> import qualified Streamly.Internal.Data.Stream as Stream (hoist)
115-- | Same as 'reader'
153-- | Same as 'reader'