19d52fb · rowtype-yoga/purescript-arraybuffer-builder

git.iko.soy/rowtype-yoga/purescript-arraybuffer-builder

on master help sign in

Bytes type for appending DataView

19d52fbJames Brock5 years ago

CHANGELOG.md+4−2

compare with master

unifiedsplit

@@ -1,6 +1,8 @@

1−## 2021-04-28

1+## v2.0.0 2021-04-29

3+Added the `Bytes` type.

3−Build for PureScript v0.14.1

5+Spago build for PureScript v0.14.1.

5## v1.1.0 2020-09-10

7## v1.1.0 2020-09-10

README.md+39−8

@@ -30,8 +30,10 @@ All `ArrayBuffer` building must occur in `Effect`.

31Create a two-byte `arraybuffer :: ArrayBuffer` which contains the number *-10* encoded as big-endian 16-bit two’s-complement.

32```purescript

33+import Data.ArrayBuffer.Builder (execPut, putInt16be)

34+

33do

35do

34− arraybuffer <- execPut $ putInt16be (-10)

36+ arraybuffer :: ArrayBuffer <- execPut $ putInt16be (-10)

35```

37```

37### Serialize three floats

39### Serialize three floats

@@ -40,8 +42,10 @@ Create a 24-byte `arraybuffer :: ArrayBuffer` which contains three big-endian

40IEEE-754 double-precision floats.

42IEEE-754 double-precision floats.

42```purescript

44```purescript

45+import Data.ArrayBuffer.Builder (execPut, putFloat64be)

46+

43do

47do

44− arraybuffer <- execPut $ do

48+ arraybuffer :: ArrayBuffer <- execPut $ do

45 putFloat64be 1.0

49 putFloat64be 1.0

46 putFloat64be 2.0

50 putFloat64be 2.0

47 putFloat64be 3.0

51 putFloat64be 3.0

@@ -49,22 +53,27 @@ do

50### Serialize a `String` as UTF8

54### Serialize a `String` as UTF8

52−Encode a `String` as UTF8 with a length prefix.

56+Encode a `String` as UTF8 with a length prefix into our `Builder`.

54We give this as an example, rather than supporting it in the library, because

58We give this as an example, rather than supporting it in the library, because

55it depends on

59it depends on

56[`Data.TextEncoding.encodeUtf8`](https://pursuit.purescript.org/packages/purescript-text-encoding/docs/Data.TextEncoding#v:encodeUtf8).

60[`Data.TextEncoding.encodeUtf8`](https://pursuit.purescript.org/packages/purescript-text-encoding/docs/Data.TextEncoding#v:encodeUtf8).

58```purescript

62```purescript

63+import Data.ArrayBuffer.Builder (PutM, putArrayBuffer, execPut)

64+import Data.ArrayBuffer.Typed (buffer)

65+import Data.TextEncoding (encodeUtf8)

66+import Data.ArrayBuffer.ArrayBuffer (byteLength)

67+

59putStringUtf8 :: forall m. (MonadEffect m) => String -> PutM m Unit

68putStringUtf8 :: forall m. (MonadEffect m) => String -> PutM m Unit

60putStringUtf8 s = do

69putStringUtf8 s = do

61− let stringbuf = Data.ArrayBuffer.Typed.buffer $ Data.TextEncoding.encodeUtf8 s

70+ let stringbuf = buffer $ encodeUtf8 s

62 -- Put a 32-bit big-endian length prefix for the length of the utf8 string, in bytes.

71 -- Put a 32-bit big-endian length prefix for the length of the utf8 string, in bytes.

63− putUint32be $ Data.Uint.fromInt $ Data.ArrayBuffer.byteLength stringbuf

72+ putInt32be $ byteLength stringbuf

64 putArrayBuffer stringbuf

73 putArrayBuffer stringbuf

66do

75do

67− arraybuffer <- execPut $ putStringUtf8 "BLM"

76+ arraybuffer :: ArrayBuffer <- execPut $ putStringUtf8 "BLM"

68```

77```

70### Serialize an `Array Int`

79### Serialize an `Array Int`

@@ -77,17 +86,38 @@ from the Haskell

77library.

86library.

79```purescript

88```purescript

89+import Data.ArrayBuffer.Builder (execPut, putInt32be)

90+import Data.Foldable (traverse_)

91+import Data.Array (length)

92+

80putArrayInt32 :: forall m. (MonadEffect m) => Array Int -> PutM m Unit

93putArrayInt32 :: forall m. (MonadEffect m) => Array Int -> PutM m Unit

81putArrayInt32 xs = do

94putArrayInt32 xs = do

82 -- Put a 64-bit big-endian length prefix for the length of the array.

95 -- Put a 64-bit big-endian length prefix for the length of the array.

83− putUint32be 0

96+ putInt32be 0

84− putUint32be $ Data.Uint.fromInt $ Data.Array.length xs

97+ putInt32be $ length xs

85 traverse_ putInt32be xs

98 traverse_ putInt32be xs

87do

100do

88 arraybuffer <- execPut $ putArrayInt32 [1,2,3]

101 arraybuffer <- execPut $ putArrayInt32 [1,2,3]

89```

102```

103

104+

105+## `unsafePerformEffect execPut`

106+

107+Dear computer programmer, you can call `execPut` inside `unsafePerformEffect`.

108+

109+The reason why this whole library is `Effect`ful is that `ArrayBuffer`s are mutable, so the `ArrayBuffer` library functions do everything in `Effect` to to confirm that any mutations are properly sequenced.

110+

111+However, all of the `put` functions in this library are actually side-effect free, so we can use `unsafePerformEffect` to tell the compiler to trust us that we don't perform any side-effects while we're building an `ArrayBuffer`.

112+

113+```purescript

114+let serialized :: ArrayBuffer

115+ serialized = unsafePerformEffect $ execPut do

116+ putInt32be 3 -- ✅ Good, this has no side-effects

117+ putInt32be =<< getCurrentPosixTime -- ❌ No don't do this

118+ putArrayBuffer =<< readFileFromMyFilesystem -- ❌ No don’t do this either

119+```

120+

91## Deserialization

121## Deserialization

122

93This package is only for writing `ArrayBuffer`s, not reading them.

123This package is only for writing `ArrayBuffer`s, not reading them.

@@ -104,3 +134,4 @@ for a way to deserialize from `ArrayBuffer` back to Purescript data.

104* [__dynamic-buffers__](https://pursuit.purescript.org/packages/purescript-dynamic-buffers)

134* [__dynamic-buffers__](https://pursuit.purescript.org/packages/purescript-dynamic-buffers)

105* [__node-buffer__](https://pursuit.purescript.org/packages/purescript-node-buffer)

135* [__node-buffer__](https://pursuit.purescript.org/packages/purescript-node-buffer)

106* [__arraybuffer-class__](https://pursuit.purescript.org/packages/purescript-arraybuffer-class)

136* [__arraybuffer-class__](https://pursuit.purescript.org/packages/purescript-arraybuffer-class)

137+* [__bytestrings__](https://pursuit.purescript.org/packages/purescript-bytestrings/)

src/Data/ArrayBuffer/Builder.purs+63−14

@@ -16,6 +16,7 @@ module Data.ArrayBuffer.Builder

16, execPut

17, subBuilder

18, putArrayBuffer

19+, putDataView

19, putUint8

20, putUint8

20, putInt8

21, putInt8

21, putUint16be

22, putUint16be

@@ -32,11 +33,16 @@ module Data.ArrayBuffer.Builder

32, putFloat64le

33, putFloat64le

33, Builder

34, Builder

34, (<>>)

35, (<>>)

36+, Bytes

37+, toView

35, execBuilder

38, execBuilder

36, length

39, length

37, singleton

40, singleton

41+, singleton_

38, cons

42, cons

43+, cons_

39, snoc

44, snoc

45+, snoc_

40, encodeUint8

46, encodeUint8

41, encodeInt8

47, encodeInt8

42, encodeUint16be

48, encodeUint16be

@@ -60,7 +66,7 @@ import Control.Monad.Writer.Trans (WriterT, execWriterT, tell, lift)

60import Data.ArrayBuffer.ArrayBuffer as AB

66import Data.ArrayBuffer.ArrayBuffer as AB

61import Data.ArrayBuffer.DataView as DV

67import Data.ArrayBuffer.DataView as DV

62import Data.ArrayBuffer.Typed as AT

68import Data.ArrayBuffer.Typed as AT

63−import Data.ArrayBuffer.Types (ArrayBuffer, Uint8Array)

69+import Data.ArrayBuffer.Types (ArrayBuffer, Uint8Array, DataView, ByteLength)

64import Data.Float32 (Float32)

70import Data.Float32 (Float32)

65import Data.Maybe (Maybe(..))

71import Data.Maybe (Maybe(..))

66import Data.UInt (UInt)

72import Data.UInt (UInt)

@@ -80,6 +86,15 @@ type Put = PutM Effect

81-- | Monoidal builder for `ArrayBuffer`s.

87-- | Monoidal builder for `ArrayBuffer`s.

82-- |

88-- |

89+-- | We can add two types of things to the `Builder`: `ArrayBuffer`s and

90+-- | `DataView`s.

91+-- | We might prefer

92+-- | to add a `DataView` to a `Builder` when we’re adding a large slice of data

93+-- | from some other `ArrayBuffer`, so that we don’t

94+-- | need an extra intermediate copy of the slice.

95+-- | Our convention is that functions which add `DataView`s to

96+-- | the `Builder` are suffixed with underscore (`_`).

97+-- |

83-- | ### Left-associative `<>>` append operator

98-- | ### Left-associative `<>>` append operator

84-- |

99-- |

85-- | __TL;DR__ You probably don't want to use the `Builder` monoid directly

100-- | __TL;DR__ You probably don't want to use the `Builder` monoid directly

@@ -114,7 +129,7 @@ type Put = PutM Effect

114-- | `snoc` case.

129-- | `snoc` case.

115-- |

130-- |

116data Builder

131data Builder

117− = Node Builder ArrayBuffer Builder

132+ = Node Builder Bytes Builder

118 | Null

133 | Null

119

134

120instance semigroupBuilder :: Semigroup Builder where

135instance semigroupBuilder :: Semigroup Builder where

@@ -134,36 +149,60 @@ infixl 5 append as <>>

134instance monoidBuilder :: Monoid Builder where

149instance monoidBuilder :: Monoid Builder where

135 mempty = Null

150 mempty = Null

136

151

152+-- | For distinguishing between `ArrayBuffer` and `DataView`.

153+data Bytes

154+ = ByteBuff ArrayBuffer

155+ | ByteView DataView

156+

157+toView :: Bytes -> DataView

158+toView (ByteBuff ab) = DV.whole ab

159+toView (ByteView dv) = dv

160+

137-- | `Builder` is not an instance of `Foldable` because `Builder` is monomorphic.

161-- | `Builder` is not an instance of `Foldable` because `Builder` is monomorphic.

138−foldl :: forall b. (b -> ArrayBuffer -> b) -> b -> Builder -> b

162+foldl :: forall b. (b -> Bytes -> b) -> b -> Builder -> b

139−foldl f a Null = a

163+foldl _ a Null = a

140foldl f a (Node l x r) = foldl f (f (foldl f a l) x) r

164foldl f a (Node l x r) = foldl f (f (foldl f a l) x) r

141

165

142-- | Monomorphic foldM copied from

166-- | Monomorphic foldM copied from

143-- | [`Data.Foldable.foldM`](https://pursuit.purescript.org/packages/purescript-foldable-traversable/4.1.1/docs/Data.Foldable#v:foldM)

167-- | [`Data.Foldable.foldM`](https://pursuit.purescript.org/packages/purescript-foldable-traversable/4.1.1/docs/Data.Foldable#v:foldM)

144−foldM :: forall b m. (Monad m) => (b -> ArrayBuffer -> m b) -> b -> Builder -> m b

168+foldM :: forall b m. (Monad m) => (b -> Bytes -> m b) -> b -> Builder -> m b

145foldM f a0 = foldl (\ma b -> ma >>= flip f b) (pure a0)

169foldM f a0 = foldl (\ma b -> ma >>= flip f b) (pure a0)

146

170

147-- | Construct a `Builder` with a single `ArrayBuffer`. *O(1)*

171-- | Construct a `Builder` with a single `ArrayBuffer`. *O(1)*

148singleton :: ArrayBuffer -> Builder

172singleton :: ArrayBuffer -> Builder

149−singleton buf = Node Null buf Null

173+singleton buf = Node Null (ByteBuff buf) Null

174+

175+-- | Construct a `Builder` with a single `DataView`. *O(1)*

176+singleton_ :: DataView -> Builder

177+singleton_ buf = Node Null (ByteView buf) Null

150

178

151-- | Prepend an `ArrayBuffer` to the beginning of the `Builder`. *O(1)*

179-- | Prepend an `ArrayBuffer` to the beginning of the `Builder`. *O(1)*

152cons :: ArrayBuffer -> Builder -> Builder

180cons :: ArrayBuffer -> Builder -> Builder

153−cons x bs = Node Null x bs

181+cons x bs = Node Null (ByteBuff x) bs

182+

183+-- | Prepend a `DataView` to the beginning of the `Builder`. *O(1)*

184+cons_ :: DataView -> Builder -> Builder

185+cons_ x bs = Node Null (ByteView x) bs

154

186

155-- | Append an `ArrayBuffer` to the end of the `Builder`. *O(1)*

187-- | Append an `ArrayBuffer` to the end of the `Builder`. *O(1)*

156snoc :: Builder -> ArrayBuffer -> Builder

188snoc :: Builder -> ArrayBuffer -> Builder

157−snoc bs x = Node bs x Null

189+snoc bs x = Node bs (ByteBuff x) Null

190+

191+-- | Append a `DataView` to the end of the `Builder`. *O(1)*

192+snoc_ :: Builder -> DataView -> Builder

193+snoc_ bs x = Node bs (ByteView x) Null

158

194

159-- | Calculate the total byte length of the `Builder`. *O(n)*

195-- | Calculate the total byte length of the `Builder`. *O(n)*

160−length :: Builder -> Int

196+length :: Builder -> ByteLength

161−length bldr = foldl (\b a -> b + AB.byteLength a) 0 bldr

197+length bldr = foldl (\b a -> b + len a) 0 bldr

198+ where

199+ len (ByteBuff ab) = AB.byteLength ab

200+ len (ByteView dv) = DV.byteLength dv

162

201

163-- | Build a single `ArrayBuffer` from a `Builder`. *O(n)*

202-- | Build a single `ArrayBuffer` from a `Builder`. *O(n)*

164execBuilder :: forall m. (MonadEffect m) => Builder -> m ArrayBuffer

203execBuilder :: forall m. (MonadEffect m) => Builder -> m ArrayBuffer

165execBuilder bldr = do

204execBuilder bldr = do

166− let buflen = foldl (\b a -> b + AB.byteLength a) 0 bldr

205+ let buflen = length bldr

167 -- Allocate the final ArrayBuffer

206 -- Allocate the final ArrayBuffer

168 buf <- liftEffect $ AB.empty buflen

207 buf <- liftEffect $ AB.empty buflen

169 -- Then copy each ArrayBuffer into the final ArrayBuffer.

208 -- Then copy each ArrayBuffer into the final ArrayBuffer.

@@ -172,11 +211,16 @@ execBuilder bldr = do

172 newview <- liftEffect (AT.whole buf :: Effect Uint8Array)

211 newview <- liftEffect (AT.whole buf :: Effect Uint8Array)

173 _ <- foldM

212 _ <- foldM

174 ( \offset a -> do

213 ( \offset a -> do

175− aview <- liftEffect (AT.whole a :: Effect Uint8Array)

214+ aview <- liftEffect $ toUint8Array a

176 _ <- liftEffect $ AT.setTyped newview (Just offset) aview

215 _ <- liftEffect $ AT.setTyped newview (Just offset) aview

177− pure $ offset + AB.byteLength a

216+ pure $ offset + AT.byteLength aview

178 ) 0 bldr

217 ) 0 bldr

179 pure buf

218 pure buf

219+ where

220+ toUint8Array :: Bytes -> Effect Uint8Array

221+ toUint8Array (ByteBuff ab) = AT.whole ab

222+ toUint8Array (ByteView dv) =

223+ AT.part (DV.buffer dv) (DV.byteOffset dv) (DV.byteLength dv)

180

224

181-- | Build an `ArrayBuffer` with do-notation in any `MonadEffect`. *O(n)*

225-- | Build an `ArrayBuffer` with do-notation in any `MonadEffect`. *O(n)*

182execPutM :: forall m. (MonadEffect m) => PutM m Unit -> m ArrayBuffer

226execPutM :: forall m. (MonadEffect m) => PutM m Unit -> m ArrayBuffer

@@ -212,6 +256,10 @@ subBuilder = lift <<< execWriterT

212putArrayBuffer :: forall m. (MonadEffect m) => ArrayBuffer -> PutM m Unit

256putArrayBuffer :: forall m. (MonadEffect m) => ArrayBuffer -> PutM m Unit

213putArrayBuffer = tell <<< singleton

257putArrayBuffer = tell <<< singleton

214

258

259+-- | Append a `DataView` to the builder.

260+putDataView :: forall m. (MonadEffect m) => DataView -> PutM m Unit

261+putDataView = tell <<< singleton_

262+

215-- | Serialize an 8-bit unsigned integer (byte) into a new `ArrayBuffer`.

263-- | Serialize an 8-bit unsigned integer (byte) into a new `ArrayBuffer`.

216encodeUint8 :: forall m. (MonadEffect m) => UInt -> m ArrayBuffer

264encodeUint8 :: forall m. (MonadEffect m) => UInt -> m ArrayBuffer

217encodeUint8 x = do

265encodeUint8 x = do

@@ -379,7 +427,7 @@ putFloat64le = putArrayBuffer <=< encodeFloat64le

379-- left or right tree is a singleton. If that's not true, then in the

427-- left or right tree is a singleton. If that's not true, then in the

380-- unlikely worst case `append` might be *O(n)*.

428-- unlikely worst case `append` might be *O(n)*.

381--

429--

382−-- `Builder` is optimized what we consider to be normal usage, that is,

430+-- `Builder` is optimized for what we consider to be normal usage, that is,

383-- `snoc`ing singleton elements to the end of the `Builder`.

431-- `snoc`ing singleton elements to the end of the `Builder`.

384--

432--

385-- If a Builder is built entirely by `snoc`ing, it will look like a

433-- If a Builder is built entirely by `snoc`ing, it will look like a

@@ -427,6 +475,7 @@ putFloat64le = putArrayBuffer <=< encodeFloat64le

427--

475--

428-- One relatively cheap and simple performance improvement for this library would be to

476-- One relatively cheap and simple performance improvement for this library would be to

429-- remove the Null constructor of `Builder` and instead use Javascript nulls.

477-- remove the Null constructor of `Builder` and instead use Javascript nulls.

478+-- UPDATE Actually no, we need the Null constructor for the Monoid instance.

430--

479--

431-- In the longer term, it might make sense to try to change the Builder so

480-- In the longer term, it might make sense to try to change the Builder so

432-- that it works like the

481-- that it works like the

test/Main.purs+13

@@ -3,6 +3,7 @@ module Test.Main where

3import Prelude (Unit, bind, pure, discard, map, negate, ($), (<$>), (=<<), (<>))

4import Data.ArrayBuffer.Types (ArrayBuffer,Uint8Array)

5import Data.ArrayBuffer.Typed as AT

6+import Data.ArrayBuffer.DataView as DV

6import Data.UInt as UInt

7import Data.UInt as UInt

7import Effect (Effect)

8import Effect (Effect)

8import Test.Assert (assertEqual')

9import Test.Assert (assertEqual')

@@ -98,3 +99,15 @@ main = do

98 putInt8 $ length x

99 putInt8 $ length x

99 tell x

100 tell x

100

101

102+ buildTest "Test 10" [1,2,3,4,5,6,7,8] $ do

103+ b1 <- DV.whole <$> encodeInt8 1

104+ b2 <- DV.whole <$> encodeInt8 2

105+ b3 <- DV.whole <$> encodeInt8 3

106+ b4 <- DV.whole <$> encodeInt8 4

107+ b5 <- DV.whole <$> encodeInt8 5

108+ b6 <- DV.whole <$> encodeInt8 6

109+ b7 <- DV.whole <$> encodeInt8 7

110+ b8 <- DV.whole <$> encodeInt8 8

111+ pure $ ((singleton_ b1 <> singleton_ b2) <> (singleton_ b3 <> singleton_ b4))

112+ <> ((singleton_ b5 <> singleton_ b6) <> (singleton_ b7 <> singleton_ b8))

113+