git.iko.soy/typeable/schematic
on clean_and_union help sign in
typeable/schematic/README.md
255 lines 9.3 kB history raw
1# schematic
3[![Build Status](https://travis-ci.org/typeable/schematic.svg?branch=master)](https://travis-ci.org/typeable/schematic)
5## Goal
7The goal of the library is to provide a type-safe transport layer for serializing and validating JSON. It can be thought of as a subset of [json-schema](http://json-schema.org), which is basically a specification of a JSON document. The other goal is getting as much as possible from this specification for free. Right now the following bits are prototyped:
9* All of the instantiations of the transport types structurally follow the schema provided by user
10* Serializers are generic, so they follow from the type-level schema and the're supposed to have a roundtrip property by an implementation.
11* Runtime value validators are generated from the schema. Validation errors are reported as a pairs of a json-path to the element and an error message.
12* There are migrations. It's possible to describe a series of migrations to the schema and have all the necessary machinery to deserialize a user specified version of the schema if there are multiple versions available.
13* Schematic schemas can be exported to [json-schema](http://json-schema.org)
15Be aware that library is experimental and subject to change a lot. The current state can be viewed as a prototype.
17## Installation
19* Install [Stack](https://github.com/commercialhaskell/stack)
21```
22$ stack install schematic
23```
25## GHC Extensions
27To use this library without any hassle you should add a few GHC extension
28either to a module or a cabal file:
30```
31DataKinds
32OverloadedLists
33OverloadedStrings
34TypeApplications
35```
37`Overloaded`-extensions are being used only by `field` combinator,
38so if you don't use it - feel free to disable it.
40## GHC Options
42I recommend using `{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}`
43pragma in modules which declare schemas and schema migrations, because `schematic`
44heavily uses promoted types which should be prefixed with "`" signs and they can
45be dropped with this option, making it less noisy visually.
46It's entirely optional and totally opinionated, so feel free to ignore this.
47Through the documentation, the code is written without any explicit ticks for
48the same reason.
49Notice that they're still needed for type-level lists and tuples in schematic code.
51## Basic Examples
53```haskell
54type SchemaExample
55 = SchemaObject
56 '[ '("foo", SchemaArray '[AEq 1] (SchemaNumber '[NGt 10]))
57 , '("bar", SchemaOptional (SchemaText '[TEnum '["foo", "bar"]]))]
58```
60* This schema says that's structurally it's a JSON object which has two fields:
61 * field "foo" contains an array of numeric values
62 * field "bar" contains an optional text value
63* It also allows you to validate JSON values itself:
64 * an array in "foo" contains exactly one element
65 * that element must be strictly greater than 10
66 * value of "bar" field must be either "foo" or "bar" if presented
69This one is valid for example
71```haskell
72schemaJson :: ByteString
73schemaJson = "{\"foo\": [13], \"bar\": null}"
74```
76Also valid
78```haskell
79schemaJson :: ByteString
80schemaJson = "{\"foo\": [13], \"bar\": \"bar\"}"
81```
83it can be parsed and validated like this:
85```haskell
86decodeAndValidateJson schemaJson :: ParseResult (JsonRepr SchemaExample)
87```
89`ParseResult` type encodes three possible situations:
90* json is structurally consistent with a schema and runtime validations pass
91* json source is malformed or doesn't correspond to a schema structurally
92* some of the runtime validations have failed, but it's structurally correct
93 overall
95It implies a transport layer representation of that data that can be traversed
96and transformed to whatever internal types user has.
97It's called `JsonRepr`. Type parameter is the schema itself.
99```haskell
100jsonExample :: JsonRepr SchemaExample
101jsonExample = withRepr @SchemaExample
102 $ field @"foo" [12]
103 :& field @"bar" (Just "bar")
104 :& RNil
105```
107The most basic and sugar-free way of constructing the same object will look like this:
109```
110jsonExample' :: JsonRepr SchemaExample
111jsonExample' = ReprObject $
112 FieldRepr (ReprArray [ReprNumber 12])
113 :& FieldRepr (ReprOptional (Just (ReprText "bar")))
114 :& RNil
115```
117## Lens-compatibility
119It's possible to use `flens` to construct a field lens for a typed object in schematic. Let's suppose you have a schema like this:
121```haskell
122type ArraySchema = SchemaArray '[AEq 1] (SchemaNumber '[NGt 10])
124type ArrayField = '("foo", ArraySchema)
126type FieldsSchema =
127 '[ ArrayField, '("bar", SchemaOptional (SchemaText '[TEnum '["foo", "bar"]]))]
129type SchemaExample = 'SchemaObject FieldsSchema
131```
133There are two ways of working with named fields in the objects:
135* Using the `fget` and `fset` functions: `fget @"foo" (fput newFooVal objectData) == newFooVal`
136* Using the lens library: `set (flens @"foo") newFooVal objectData ^. flens @"foo" == newFooVal`
138## Migrations
140Usually migrations are implicit and hard to deal with, `schematic` deals with it
141by giving tools to deal with it to a programmer, being as explicit as possible,
142notifying the developer of a missing transition with a compile-time error.
143It allows to build versioned HTTP APIs or migrate data from the older versions
144when reading from the JSON storages.
146It's possible to represent schema changes as a series of migrations,
147which describes a series of json-path/change pairs. Migrations can be applied in
148succession.
150This piece of code will apply a migration to the schema, decode and validate the latest version.
152```haskell
153type SchemaExample
154 = SchemaObject
155 '[ '("foo", SchemaArray '[AEq 1] (SchemaNumber '[NGt 10]))
156 , '("bar", SchemaOptional (SchemaText '[TEnum '["foo", "bar"]]))]
158type TestMigration =
159 'Migration "test_revision"
160 '[ Diff '[ PKey "bar" ] (Update (SchemaText '[]))
161 , Diff '[ PKey "foo" ] (Update (SchemaNumber '[])) ]
163type VS = 'Versioned SchemaExample '[ TestMigration ]
165schemaJsonTopVersion :: ByteString
166schemaJsonTopVersion = "{ \"foo\": 42, \"bar\": \"bar\" }"
167```
169It's possible to decode the latest version like this:
171```
172decodeAndValidateJson schemaJsonTopVersion :: ParseResult (JsonRepr (TopVersion (AllVersions VS)))
173```
175It's important to differentiate between schema migrations and data migrations.
176Schema migration is a change of a schema acceptable by a JSON consumer. It transforms one schema to another.
177Data migrations on the other hand are functions on the data itself. They transform values
178acceptable by one schema to values acceptable by another.
179Let's look at the situation when we have to add a field to a JSON Object of a current schema:
181```haskell
182type SchemaExample = 'SchemaObject
183 '[ '("foo", 'SchemaArray '[ 'AEq 1] ('SchemaNumber '[ 'NGt 10]))
184 , '("bar", 'SchemaOptional ('SchemaText '[ 'TEnum '["foo", "bar"]]))]
186jsonExample :: JsonRepr SchemaExample
187jsonExample = withRepr @SchemaExample
188 $ field @"bar" (Just "bar")
189 :& field @"foo" [12]
190 :& RNil
192type AddQuuz =
193 'Migration "add_field_quuz"
194 '[ 'Diff '[] ('AddKey "quuz" (SchemaNumber '[])) ]
196type DeleteQuuz =
197 'Migration "remove_field_quuz"
198 '[ 'Diff '[] ( 'DeleteKey "quuz") ]
200type Migrations = '[ AddQuuz
201 , DeleteQuuz ]
203type VersionedJson = 'Versioned SchemaExample Migrations
205migrationList :: MigrationList Identity VersionedJson
206migrationList
207 = (migrateObject (\r -> Identity $ field @"quuz" 42 :& r))
208 :&& shrinkObject
209 :&& MNil
211```
213In this instance `Migrations` is a list of schema migrations and
214`MigrationList Identity VersionedJson` is a list of data migrations corresponding
215to the list of schema migrations: `migrateObject (\r -> Identity $ field @"quuz" 42 :& r))`
216will be used at runtime to transform `JsonRepr SchemaExample` to
217`JsonRepr (SchemaByRevision "add_quuz" VersionedJson). `Identity` in the type of
218`MigrationList` is a monad we choose to run our migrations in. If there's need to
219do database queries or something like that, it can be changed to something more
220appropriate by a user.
221`migrateObject` takes a function transforming old fields into new ones, `shrinkObject`
222allows to shrink the JSON object in case migration only removed fields.
224## How do I construct a value of `JsonRepr schema`?
226`JsonRepr schema` is a primary representation of a value serializable/deserializable
227to JSON, but with a twist. It's guaranteed to correspond a type-level schema,
228it's a type parameter of a same name.
229It makes constructing it a little bit more involved, but luckily `schematic`
230provides a DSL for doing so in a more straightforward fashion. An example would
231look like this:
233```haskell
234type SchemaExample = 'SchemaObject
235 '[ '("foo", SchemaArray '[ AEq 1] (SchemaNumber '[NGt 10]))
236 , '("bar", SchemaOptional (SchemaText '[TEnum '["foo", "bar"]]))]
238jsonExample = withRepr @SchemaExample
239 $ field @"bar" (Just "bar")
240 :& field @"foo" [12]
241 :& RNil
242```
244`@foo` syntax is an explicit type application, which is a feature of GHC 8+, it
245makes type inference possible without relying on a bunch of `Proxy`s, which
246makes it syntactically more terse.
248[GHC Type Applications](https://ghc.haskell.org/trac/ghc/wiki/TypeApplication)
250`schematic` provides instances for `OverloadedLists` and `OverloadedStrings`
251GHC extensions to make use of string and list literals for values in a previous
252example.
255## Export to json-schema (draft 4)