git.iko.soy/streamly
on 8dadcad help sign in
streamly/src/Streamly/Memory/Array.hs
102 lines 3.8 kB history raw
1{-# LANGUAGE BangPatterns #-}
2{-# LANGUAGE CPP #-}
3{-# LANGUAGE MagicHash #-}
4{-# LANGUAGE RecordWildCards #-}
5{-# LANGUAGE UnboxedTuples #-}
6{-# LANGUAGE ScopedTypeVariables #-}
8#include "inline.hs"
10-- |
11-- Module : Streamly.Memory.Array
12-- Copyright : (c) 2019 Composewell Technologies
13--
14-- License : BSD3
15-- Maintainer : streamly@composewell.com
16-- Stability : experimental
17-- Portability : GHC
18--
19-- This module provides immutable arrays suitable for storage and random
20-- access. Once created an array cannot be mutated without copying. Arrays in
21-- this module are chunks of memory that hold a sequence of 'Storable' values
22-- of a given type. They are designed to store serializable data, they cannot
23-- store non-serializable data like functions, this is reflected by the
24-- 'Storable' constraint. For efficient buffering of data, arrays use pinned
25-- memory and therefore can hold arbitrary number of elements with a single
26-- movable pointer visible to GC, therefore, adding no pressure to GC.
27-- Moreover, pinned memory allows communication with foreign consumers and
28-- producers (e.g. file or network IO) without copying the data.
29--
30-- By design, there are no array transformation operations provided in this
31-- module, only conversion to and from stream is provided. Arrays can be
32-- operated upon /efficiently/ by converting them into a stream, applying the
33-- desired stream transformations from "Streamly.Prelude" and then converting
34-- it back to an array.
35--
36-- This module is designed to be imported qualified:
37--
38-- > import qualified Streamly.Array as A
40-- To summarize:
41--
42-- Arrays are finite and fixed in size
43-- provide /O(1)/ access to elements
44-- store only data and not functions
45-- provide efficient IO interfacing
46--
47-- 'Foldable' instance is not provided because the implementation would be much
48-- less efficient compared to folding via streams. 'Semigroup' and 'Monoid'
49-- instances are deliberately not provided to avoid misuse; concatenating
50-- arrays using binary operations can be highly inefficient. Instead, use
51-- 'Streamly.Memory.Array.Stream.toArray' to concatenate N arrays at once.
52--
53-- Each array is one pointer visible to the GC. Too many small arrays (e.g.
54-- single byte) are only as good as holding those elements in a Haskell list.
55-- However, small arrays can be compacted into large ones to reduce the
56-- overhead. To hold 32GB memory in 32k sized buffers we need 1 million arrays
57-- if we use one array for each chunk. This is still significant to add
58-- pressure to GC. However, we can create arrays of arrays (trees) to scale to
59-- arbitrarily large amounts of memory but still using small chunks of
60-- contiguous memory.
62module Streamly.Memory.Array
63 (
64 A.Array
66 -- , defaultChunkSize
68 -- * Arrays
69 -- ** Construction
70 -- | When performance matters, the fastest way to generate an array is
71 -- 'writeN'. For regular use, 'IsList' and 'IsString' instances can be
72 -- used to conveniently construct arrays from literal values.
73 -- 'OverloadedLists' extension or 'fromList' can be used to construct an
74 -- array from a list literal. Similarly, 'OverloadedStrings' extension or
75 -- 'fromList' can be used to construct an array from a string literal.
77 -- Pure List APIs
78 , A.fromListN
79 , A.fromList
81 -- Monadic APIs
82 -- , newArray
83 , A.writeN -- drop new
84 , A.write -- full buffer
85 -- , writeLastN -- drop old (ring buffer)
87 -- ** Elimination
88 -- 'GHC.Exts.toList' from "GHC.Exts" can be used to convert an array to a
89 -- list.
91 , A.toList
92 , A.read
94 -- ** Random Access
95 , A.length
96 -- , (!!)
98 , A.readIndex
99 )
100where
102import Streamly.Internal.Memory.Array as A