on master help sign in
Update serialization docs
97c1abaHarendra Kumar
core/src/Streamly/Data/MutByteArray.hs+20−13
@@ -5,18 +5,25 @@
5-- Maintainer : streamly@composewell.com
5-- Maintainer : streamly@composewell.com
6-- Portability : GHC
6-- Portability : GHC
7--
7--
8+-- A low level byte Array type 'MutByteArray', along with type classes 'Unbox'
8−-- A low level byte Array type for fast binary serialization and
9+-- and 'Serialize' for fast binary serialization and deserialization of Haskell
9−-- deserialization of Haskell values. Higher level unboxed array modules
10+-- values. Serialization, deserialization performance is similar to, and in
10−-- "Streamly.Data.Array" and "Streamly.Data.MutArray" are built on top of this
11−-- module. This module provides two type classes for serialization, 'Unbox' and
12−-- 'Serialize'. The speed is similar to, and in some cases many times faster
13−-- than the store package. Conceptually, the 'Serialize' type class works in
11+-- some cases many times better than the store package. Conceptually, the
14−-- the same way as store.
12+-- 'Serialize' type class works in the same way as store.
15--
13--
16−-- You probably want the higher level
14+-- Serialize instances are configurable to use constructor names (see
15+-- 'Streamly.Internal.Data.MutByteArray.encodeConstrNames'), record field names (see
16+-- 'Streamly.Internal.Data.MutByteArray.encodeRecordFields') instead of binary
17+-- encoded values. This is an experimental feature which allows JSON like
18+-- properties with faster speed. For example, you can change the order of
19+-- constructors or record fields without affecting serialized value.
20+--
21+-- Higher level unboxed array modules "Streamly.Data.Array" and
22+-- "Streamly.Data.MutArray" are built on top of this module. Unboxed arrays are
23+-- essentially serialized Haskell values. Array modules provide higher level
17−-- 'Streamly.Internal.Data.Array.pinnedSerialize' and
24+-- serialization routines like 'Streamly.Data.Array.pinnedSerialize'
18−-- 'Streamly.Internal.Data.Array.deserialize' from the "Streamly.Data.Array"
25+-- and 'Streamly.Data.Array.deserialize' from the
19−-- module.
26+-- "Streamly.Data.Array" module.
20--
27--
21-- == Mutable Byte Array
28-- == Mutable Byte Array
22--
29--
@@ -48,8 +55,8 @@
48-- and then use the type class methods to serialize and deserialize to and from
55-- and then use the type class methods to serialize and deserialize to and from
49-- a 'MutByteArray'.
56-- a 'MutByteArray'.
50--
57--
51−-- See 'Streamly.Internal.Data.Array.pinnedSerialize' and
58+-- See 'Streamly.Data.Array.pinnedSerialize' and
52−-- 'Streamly.Internal.Data.Array.deserialize' for 'Array' type based
59+-- 'Streamly.Data.Array.deserialize' for 'Array' type based
53-- serialization.
60-- serialization.
54--
61--
55module Streamly.Data.MutByteArray
62module Streamly.Data.MutByteArray
core/src/Streamly/Internal/Data/Serialize/TH/Bottom.hs+17−11
@@ -92,6 +92,11 @@ import Streamly.Internal.Data.Unbox.TH (DataCon(..))
92-- * 'inlineSerializeAt' (Just Inline)
92-- * 'inlineSerializeAt' (Just Inline)
93-- * 'inlineDeserializeAt' (Just Inline)
93-- * 'inlineDeserializeAt' (Just Inline)
94--
94--
95+-- The following experimental options are also available:
96+--
97+-- * 'encodeConstrNames' False
98+-- * 'encodeRecordFields' False
99+--
95data SerializeConfig =
100data SerializeConfig =
96 SerializeConfig
101 SerializeConfig
97 { cfgInlineSize :: Maybe Inline
102 { cfgInlineSize :: Maybe Inline
@@ -127,25 +132,26 @@ inlineDeserializeAt v cfg = cfg {cfgInlineDeserialize = v}
127-- | __Experimental__
132-- | __Experimental__
128--
133--
129-- In sum types, use Latin-1 encoded original constructor names rather than
134-- In sum types, use Latin-1 encoded original constructor names rather than
130−-- binary values to identify constructors. This option is not useful on a
135+-- binary values to identify constructors. This option is not applicable to
131−-- product type.
136+-- product types.
132--
137--
133−-- This option enables the following backward compatible behavior:
138+-- This option enables the following behavior:
134--
139--
135-- * __Reordering__: Order of the fields can be changed without affecting
140-- * __Reordering__: Order of the fields can be changed without affecting
136-- serialization.
141-- serialization.
137-- * __Addition__: If a field is added in the new version, the old version of
142-- * __Addition__: If a field is added in the new version, the old version of
138-- the data type can still be deserialized by the new version. The new value
143-- the data type can still be deserialized by the new version. The new value
139-- would never occur in the old one.
144-- would never occur in the old one.
140−-- * __Deletion__: If a field is deleted in the new version and deserialization
145+-- * __Deletion__: If a field is deleted in the new version, deserialization
141-- of the old version will result in an error. TBD: We can possibly designate a
146-- of the old version will result in an error. TBD: We can possibly designate a
142−-- catch-all field to handle this case.
147+-- catch-all case to handle this scenario.
143--
148--
144-- Note that if you change a type, change the semantics of a type, or delete a
149-- Note that if you change a type, change the semantics of a type, or delete a
145-- field and add a new field with the same name, deserialization of old data
150-- field and add a new field with the same name, deserialization of old data
146-- may result in silent unexpected behavior.
151-- may result in silent unexpected behavior.
147--
152--
148-- This option has to be the same on both encoding and decoding side.
153-- This option has to be the same on both encoding and decoding side.
154+--
149-- The default is 'False'.
155-- The default is 'False'.
150--
156--
151encodeConstrNames :: Bool -> SerializeConfig -> SerializeConfig
157encodeConstrNames :: Bool -> SerializeConfig -> SerializeConfig
@@ -161,11 +167,11 @@ encodeConstrNames v cfg = cfg {cfgConstructorTagAsString = v}
161-- | __Experimental__
167-- | __Experimental__
162--
168--
163-- In explicit record types, use Latin-1 encoded record field names rather than
169-- In explicit record types, use Latin-1 encoded record field names rather than
164−-- binary values to identify the record fields. Note that this option cannot be
170+-- binary values to identify the record fields. Note that this option is not
165−-- used on a sum type. Also, it does not work on a product type which is not a
171+-- applicable to sum types. Also, it does not work on a product type which is
166−-- record, because there are no field names to begin with.
172+-- not a record, because there are no field names to begin with.
167--
173--
168−-- This option enables the following backward compatible behavior:
174+-- This option enables the following behavior:
169--
175--
170-- * __Reordering__: Order of the fields can be changed without affecting
176-- * __Reordering__: Order of the fields can be changed without affecting
171-- serialization.
177-- serialization.
@@ -173,10 +179,10 @@ encodeConstrNames v cfg = cfg {cfgConstructorTagAsString = v}
173-- version of the data type can still be deserialized by the new version, the
179-- version of the data type can still be deserialized by the new version, the
174-- field value in the older version is assumed to be 'Nothing'. If any other
180-- field value in the older version is assumed to be 'Nothing'. If any other
175-- type of field is added, deserialization of the older version results in an
181-- type of field is added, deserialization of the older version results in an
176−-- error but only when that field is accessed in the deserialized record.
182+-- error but only when that field is actually accessed in the deserialized
183+-- record.
177-- * __Deletion__: If a field is deleted in the new version and it is
184-- * __Deletion__: If a field is deleted in the new version and it is
178-- encountered in a previously serialized version then the field is discarded.
185-- encountered in a previously serialized version then the field is discarded.
179−-- TBD: We can possibly designate a catch-all field to handle this case.
180--
186--
181-- This option has to be the same on both encoding and decoding side.
187-- This option has to be the same on both encoding and decoding side.
182--
188--
streamly.cabal+5−3
@@ -18,10 +18,10 @@ description:
18 high-performance, concurrent, scalable applications in Haskell. Stream
18 high-performance, concurrent, scalable applications in Haskell. Stream
19 fusion optimizations in streamly enable exceptional modularity with
19 fusion optimizations in streamly enable exceptional modularity with
20 high performance comparable to C. Streamly complements the Haskell
20 high performance comparable to C. Streamly complements the Haskell
21− <https://hackage.haskell.org/package/base base> package, supplying the
21+ <https://hackage.haskell.org/package/base base> package, supplying
22− missing functionality to quickly build general-purpose applications
22+ additional functionality to quickly build general-purpose applications
23 in Haskell. For high-level functionality built over streamly like
23 in Haskell. For high-level functionality built over streamly like
24− streaming system processes, shell programming, GNU coreutils,
24+ streaming OS processes, shell programming, GNU coreutils,
25 statistics, and compression libraries please see the
25 statistics, and compression libraries please see the
26 <https://streamly.composewell.com/ecosystem.html streamly ecosystem packages>.
26 <https://streamly.composewell.com/ecosystem.html streamly ecosystem packages>.
27 .
27 .
@@ -31,6 +31,7 @@ description:
31 * API close to standard Haskell lists (<https://github.com/composewell/streamly-examples Examples>)
31 * API close to standard Haskell lists (<https://github.com/composewell/streamly-examples Examples>)
32 * Declarative concurrency with automatic scaling
32 * Declarative concurrency with automatic scaling
33 * Filesystem, fsnotify, network, and Unicode support included
33 * Filesystem, fsnotify, network, and Unicode support included
34+ * Fast binary serialization (with optional JSON like features)
34 * More functionality is provided via many (<https://streamly.composewell.com/ecosystem.html ecosystem packages>)
35 * More functionality is provided via many (<https://streamly.composewell.com/ecosystem.html ecosystem packages>)
35 .
36 .
36 Unified and powerful abstractions:
37 Unified and powerful abstractions:
@@ -39,6 +40,7 @@ description:
39 * Unifies @Data.List@, @list-t@, and @logict@ with streaming
40 * Unifies @Data.List@, @list-t@, and @logict@ with streaming
40 * Unifies concurrency with standard streaming abstractions
41 * Unifies concurrency with standard streaming abstractions
41 * Unifies reactive, time-domain programming with streaming
42 * Unifies reactive, time-domain programming with streaming
43+ * Unifies binary serialization and unboxed arrays
42 * Interworks with other streaming libraries
44 * Interworks with other streaming libraries
43
45
44homepage: https://streamly.composewell.com
46homepage: https://streamly.composewell.com