on master help sign in
Update serialization docs
commit97c1aba1b766fbd562370960e154e5a3bc28e565 authorHarendra Kumar <harendra@composewell.com> dateTue 28 Nov 2023, 08:52 (3 years ago) parentd55d68a
3 files changed+42−27
core/src/Streamly/Data/MutByteArray.hs+20−13 core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hs+17−11 streamly.cabal+5−3
core/src/Streamly/Data/MutByteArray.hs+20−13
@@ -5,18 +5,25 @@
55-- Maintainer : streamly@composewell.com
66-- Portability : GHC
77--
8−-- A low level byte Array type for fast binary serialization and
9−-- deserialization of Haskell values. Higher level unboxed array modules
10−-- "Streamly.Data.Array" and "Streamly.Data.MutArray" are built on top of this
11−-- module. This module provides two type classes for serialization, 'Unbox' and
12−-- 'Serialize'. The speed is similar to, and in some cases many times faster
13−-- than the store package. Conceptually, the 'Serialize' type class works in
14−-- the same way as store.
8+-- A low level byte Array type 'MutByteArray', along with type classes 'Unbox'
9+-- and 'Serialize' for fast binary serialization and deserialization of Haskell
10+-- values. Serialization, deserialization performance is similar to, and in
11+-- some cases many times better than the store package. Conceptually, the
12+-- 'Serialize' type class works in the same way as store.
1513--
16−-- You probably want the higher level
17−-- 'Streamly.Internal.Data.Array.pinnedSerialize' and
18−-- 'Streamly.Internal.Data.Array.deserialize' from the "Streamly.Data.Array"
19−-- module.
14+-- Serialize instances are configurable to use constructor names (see
15+-- 'Streamly.Internal.Data.MutByteArray.encodeConstrNames'), record field names (see
16+-- 'Streamly.Internal.Data.MutByteArray.encodeRecordFields') instead of binary
17+-- encoded values. This is an experimental feature which allows JSON like
18+-- properties with faster speed. For example, you can change the order of
19+-- constructors or record fields without affecting serialized value.
20+--
21+-- Higher level unboxed array modules "Streamly.Data.Array" and
22+-- "Streamly.Data.MutArray" are built on top of this module. Unboxed arrays are
23+-- essentially serialized Haskell values. Array modules provide higher level
24+-- serialization routines like 'Streamly.Data.Array.pinnedSerialize'
25+-- and 'Streamly.Data.Array.deserialize' from the
26+-- "Streamly.Data.Array" module.
2027--
2128-- == Mutable Byte Array
2229--
@@ -48,8 +55,8 @@
4855-- and then use the type class methods to serialize and deserialize to and from
4956-- a 'MutByteArray'.
5057--
51−-- See 'Streamly.Internal.Data.Array.pinnedSerialize' and
52−-- 'Streamly.Internal.Data.Array.deserialize' for 'Array' type based
58+-- See 'Streamly.Data.Array.pinnedSerialize' and
59+-- 'Streamly.Data.Array.deserialize' for 'Array' type based
5360-- serialization.
5461--
5562module Streamly.Data.MutByteArray
core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hs+17−11
@@ -92,6 +92,11 @@ import Streamly.Internal.Data.Unbox.TH (DataCon(..))
9292-- * 'inlineSerializeAt' (Just Inline)
9393-- * 'inlineDeserializeAt' (Just Inline)
9494--
95+-- The following experimental options are also available:
96+--
97+-- * 'encodeConstrNames' False
98+-- * 'encodeRecordFields' False
99+--
95100data SerializeConfig =
96101 SerializeConfig
97102 { cfgInlineSize :: Maybe Inline
@@ -127,25 +132,26 @@ inlineDeserializeAt v cfg = cfg {cfgInlineDeserialize = v}
127132-- | __Experimental__
128133--
129134-- In sum types, use Latin-1 encoded original constructor names rather than
130−-- binary values to identify constructors. This option is not useful on a
131−-- product type.
135+-- binary values to identify constructors. This option is not applicable to
136+-- product types.
132137--
133−-- This option enables the following backward compatible behavior:
138+-- This option enables the following behavior:
134139--
135140-- * __Reordering__: Order of the fields can be changed without affecting
136141-- serialization.
137142-- * __Addition__: If a field is added in the new version, the old version of
138143-- the data type can still be deserialized by the new version. The new value
139144-- would never occur in the old one.
140−-- * __Deletion__: If a field is deleted in the new version and deserialization
145+-- * __Deletion__: If a field is deleted in the new version, deserialization
141146-- of the old version will result in an error. TBD: We can possibly designate a
142−-- catch-all field to handle this case.
147+-- catch-all case to handle this scenario.
143148--
144149-- Note that if you change a type, change the semantics of a type, or delete a
145150-- field and add a new field with the same name, deserialization of old data
146151-- may result in silent unexpected behavior.
147152--
148153-- This option has to be the same on both encoding and decoding side.
154+--
149155-- The default is 'False'.
150156--
151157encodeConstrNames :: Bool -> SerializeConfig -> SerializeConfig
@@ -161,11 +167,11 @@ encodeConstrNames v cfg = cfg {cfgConstructorTagAsString = v}
161167-- | __Experimental__
162168--
163169-- In explicit record types, use Latin-1 encoded record field names rather than
164−-- binary values to identify the record fields. Note that this option cannot be
165−-- used on a sum type. Also, it does not work on a product type which is not a
166−-- record, because there are no field names to begin with.
170+-- binary values to identify the record fields. Note that this option is not
171+-- applicable to sum types. Also, it does not work on a product type which is
172+-- not a record, because there are no field names to begin with.
167173--
168−-- This option enables the following backward compatible behavior:
174+-- This option enables the following behavior:
169175--
170176-- * __Reordering__: Order of the fields can be changed without affecting
171177-- serialization.
@@ -173,10 +179,10 @@ encodeConstrNames v cfg = cfg {cfgConstructorTagAsString = v}
173179-- version of the data type can still be deserialized by the new version, the
174180-- field value in the older version is assumed to be 'Nothing'. If any other
175181-- type of field is added, deserialization of the older version results in an
176−-- error but only when that field is accessed in the deserialized record.
182+-- error but only when that field is actually accessed in the deserialized
183+-- record.
177184-- * __Deletion__: If a field is deleted in the new version and it is
178185-- encountered in a previously serialized version then the field is discarded.
179−-- TBD: We can possibly designate a catch-all field to handle this case.
180186--
181187-- This option has to be the same on both encoding and decoding side.
182188--
streamly.cabal+5−3
@@ -18,10 +18,10 @@ description:
1818 high-performance, concurrent, scalable applications in Haskell. Stream
1919 fusion optimizations in streamly enable exceptional modularity with
2020 high performance comparable to C. Streamly complements the Haskell
21− <https://hackage.haskell.org/package/base base> package, supplying the
22− missing functionality to quickly build general-purpose applications
21+ <https://hackage.haskell.org/package/base base> package, supplying
22+ additional functionality to quickly build general-purpose applications
2323 in Haskell. For high-level functionality built over streamly like
24− streaming system processes, shell programming, GNU coreutils,
24+ streaming OS processes, shell programming, GNU coreutils,
2525 statistics, and compression libraries please see the
2626 <https://streamly.composewell.com/ecosystem.html streamly ecosystem packages>.
2727 .
@@ -31,6 +31,7 @@ description:
3131 * API close to standard Haskell lists (<https://github.com/composewell/streamly-examples Examples>)
3232 * Declarative concurrency with automatic scaling
3333 * Filesystem, fsnotify, network, and Unicode support included
34+ * Fast binary serialization (with optional JSON like features)
3435 * More functionality is provided via many (<https://streamly.composewell.com/ecosystem.html ecosystem packages>)
3536 .
3637 Unified and powerful abstractions:
@@ -39,6 +40,7 @@ description:
3940 * Unifies @Data.List@, @list-t@, and @logict@ with streaming
4041 * Unifies concurrency with standard streaming abstractions
4142 * Unifies reactive, time-domain programming with streaming
43+ * Unifies binary serialization and unboxed arrays
4244 * Interworks with other streaming libraries
4345
4446homepage: https://streamly.composewell.com