@@ -4,26 +4,26 @@ weight = 4
4template = "doc.html"
4template = "doc.html"
7−If one is working on a Gall agent with any kind of web interface, it's likely one will encounter the problem of converting Hoon data structures to JSON and vice versa. This is what we'll look at in this document.
7+If you are working on a Gall agent with any kind of web interface, it's likely you will encounter the problem of converting Hoon data structures to JSON and vice versa. This is what we'll look at in this document.
9Urbit represents JSON data with the `$json` structure (defined in `lull.hoon`). You can refer to the [json type](#the-json-type) section below for details.
9Urbit represents JSON data with the `$json` structure (defined in `lull.hoon`). You can refer to the [json type](#the-json-type) section below for details.
11JSON data on the web is encoded in text, so Urbit has two functions in `zuse.hoon` for dealing with this:
11JSON data on the web is encoded in text, so Urbit has two functions in `zuse.hoon` for dealing with this:
13−- [+en-json:html](/docs/hoon/reference/zuse/2e_2-3#en-jsonhtml) - For printing `$json` to a text-encoded form.
13+- [`+en-json:html`](/docs/hoon/reference/zuse/2e_2-3#en-jsonhtml) - For printing `$json` to a text-encoded form.
14−- [+de-json:html](/docs/hoon/reference/zuse/2e_2-3#de-jsonhtml) - For parsing text-encoded JSON to a `$json` structure.
14+- [`+de-json:html`](/docs/hoon/reference/zuse/2e_2-3#de-jsonhtml) - For parsing text-encoded JSON to a `$json` structure.
16−One typically wants `$json` data converted to some other `noun` structure or vice versa, so Urbit has three collections of functions for this purpose, also in `zuse.hoon`:
16+You typically want `$json` data converted to some other `noun` structure or vice versa, so Urbit has three collections of functions for this purpose, also in `zuse.hoon`:
18−- [+enjs:format](/docs/hoon/reference/zuse/2d_1-5#enjsformat) - Functions for converting various atoms and structures to `$json`.
18+- [`+enjs:format`](/docs/hoon/reference/zuse/2d_1-5#enjsformat) - Functions for converting various atoms and structures to `$json`.
19−- [+dejs:format](/docs/hoon/reference/zuse/2d_6#dejsformat) - Many "reparsers" for converting `$json` data to atoms and other structures.
19+- [`+dejs:format`](/docs/hoon/reference/zuse/2d_6#dejsformat) - Many "reparsers" for converting `$json` data to atoms and other structures.
20−- [+dejs-soft:format](/docs/hoon/reference/zuse/2d_7#dejs-softformat) - Largely the same as `+dejs:format` except its reparsers produce `unit`s which are null upon failure rather than simply crashing.
20+- [`+dejs-soft:format`](/docs/hoon/reference/zuse/2d_7#dejs-softformat) - Largely the same as `+dejs:format` except its reparsers produce `unit`s which are null upon failure rather than simply crashing.
22The relationship between these types and functions look like this:
22The relationship between these types and functions look like this:
24
24
26−Note this diagram is a simplification - the `+dejs:format` and `+enjs:format` collections in particular are tools to be used in writing conversion functions rather than simply being used by themselves, but it demonstrates the basic relationships. Additionally, it is less common to do printing/parsing manually - this would typically be handled automatically by Eyre, though it may be necessary if one is retrieving JSON data via the web client vane Iris.
26+Note this diagram is a simplification - the `+dejs:format` and `+enjs:format` collections in particular are tools to be used in writing conversion functions rather than simply being used by themselves, but it demonstrates the basic relationships. Additionally, it is less common to do printing/parsing manually - this would typically be handled automatically by Eyre, though it may be necessary if you're retrieving JSON data via the web client vane Iris.
28### In practice
28### In practice
@@ -31,9 +31,9 @@ A typical Gall agent will have a number of structures defined in a file in the `
32If the agent only interacts with other agents inside Urbit, it may just use a `%noun` `mark`. If, however, it needs to talk to a web interface of some kind, it usually must handle `$json` data with a `%json` mark.
32If the agent only interacts with other agents inside Urbit, it may just use a `%noun` `mark`. If, however, it needs to talk to a web interface of some kind, it usually must handle `$json` data with a `%json` mark.
34−Sometimes an agent's interactions with a web interface are totally distinct from its interactions with other agents. If so, the agent could just have separate scry endpoints, poke handlers, etc, that directly deal with `$json` data with a `%json` mark. In such a case, one can include `$json` encoding/decoding functions directly in the agent or associated libraries, using the general techniques demonstrated in the [$json encoding and decoding example](#json-encoding-and-decoding-example) section below.
34+Sometimes an agent's interactions with a web interface are totally distinct from its interactions with other agents. If so, the agent could just have separate scry endpoints, poke handlers, etc, that directly deal with `$json` data with a `%json` mark. In such a case, you can include `$json` encoding/decoding functions directly in the agent or associated libraries, using the general techniques demonstrated in the [$json encoding and decoding example](#json-encoding-and-decoding-example) section below.
36−If, on the other hand, one wants a unified interface (whether interacting with a web client or within Urbit), a different approach is necessary. Rather than taking or producing either `%noun` or `%json` marked data, custom `mark` files can be created which specify conversion methods for both `%noun` and `%json` marked data.
36+If, on the other hand, you want a unified interface (whether interacting with a web client or within Urbit), a different approach is necessary. Rather than taking or producing either `%noun` or `%json` marked data, custom `mark` files can be created which specify conversion methods for both `%noun` and `%json` marked data.
38With this approach, an agent would take and/or produce data with some `mark` like `%my-custom-mark`. Then, when the agent must interact with a web client, the webserver vane Eyre can automatically convert `%my-custom-mark` to `%json` or vice versa. This way the agent only ever has to handle the `%my-custom-mark` data. This approach is used by `%graph-store` with its `%graph-update-2` mark, for example, and a number of other agents.
38With this approach, an agent would take and/or produce data with some `mark` like `%my-custom-mark`. Then, when the agent must interact with a web client, the webserver vane Eyre can automatically convert `%my-custom-mark` to `%json` or vice versa. This way the agent only ever has to handle the `%my-custom-mark` data. This approach is used by `%graph-store` with its `%graph-update-2` mark, for example, and a number of other agents.
@@ -65,7 +65,7 @@ The correspondence of `$json` to JSON types is fairly self-evident, but here's a
65| Array | `[%a p=(list json)]` | `["foo",123]` | `[%a p=~[[%s p='foo'] [%n p=~.123]]]` |
65| Array | `[%a p=(list json)]` | `["foo",123]` | `[%a p=~[[%s p='foo'] [%n p=~.123]]]` |
66| Object | `[%o p=(map @t json)]` | `{"foo":"xyz","bar":123}` | `[%o p={[p='bar' q=[%n p=~.123]] [p='foo' q=[%s p='xyz']]}]` |
66| Object | `[%o p=(map @t json)]` | `{"foo":"xyz","bar":123}` | `[%o p={[p='bar' q=[%n p=~.123]] [p='foo' q=[%s p='xyz']]}]` |
68−Since the `$json` `%o` object and `%a` array types may themselves contain any `$json`, one can see how JSON structures of arbitrary complexity can be represented. Note the `%n` number type is a `@ta` rather than something like a `@ud` that one might expect. This is because JSON's number type may be either an integer or floating point, so it's left as a `knot` which can then be parsed to a `@ud` or `@rd` with the appropriate [+dejs:format](/docs/hoon/reference/zuse/2d_6) function.
68+Since the `$json` `%o` object and `%a` array types may themselves contain any `$json`, you can see how JSON structures of arbitrary complexity can be represented. Note the `%n` number type is a `@ta` rather than something like a `@ud` that you might expect. This is because JSON's number type may be either an integer or floating point, so it's left as a `knot` which can then be parsed to a `@ud` or `@rd` with the appropriate [`+dejs:format`](/docs/hoon/reference/zuse/2d_6) function.
70## `$json` encoding and decoding example
70## `$json` encoding and decoding example
@@ -84,9 +84,7 @@ Let's have a look at a practical example. Here's a core with three arms. It has
85 |= usr=user
85 |= usr=user
86 |^ ^- json
86 |^ ^- json
87− :- %o
87+ %- pairs:enjs:format
91 ['username' s+username.usr]
89 ['username' s+username.usr]
92 ['name' name]
90 ['name' name]
@@ -118,12 +116,12 @@ Let's have a look at a practical example. Here's a core with three arms. It has
119### Try it out
117### Try it out
121−First we'll try using our `$json` encoding/decoding library, and afterwards we'll take a closer look at its construction. To begin, save the code above in `/lib/json-test.hoon` and `|commit` it:
119+First we'll try using our `$json` encoding/decoding library, and afterwards we'll take a closer look at its construction. To begin, save the code above in `/lib/json-test.hoon` of the `%base` desk on a fake ship and `|commit` it:
124−> |commit %home
122+> |commit %base
126−+ /~zod/home/5/lib/json-test/hoon
124++ /~zod/base/5/lib/json-test/hoon
129Then we need to build it so we can use it. We'll give it a face of `user-lib`:
127Then we need to build it so we can use it. We'll give it a face of `user-lib`:
@@ -187,9 +185,7 @@ Here's our arm that converts a `$user` structure to `$json`:
188 |= usr=user
186 |= usr=user
189 |^ ^- json
187 |^ ^- json
190− :- %o
188+ %- pairs:enjs:format
194 ['username' s+username.usr]
190 ['username' s+username.usr]
195 ['name' name]
191 ['name' name]
@@ -208,9 +204,9 @@ Here's our arm that converts a `$user` structure to `$json`:
209There are different ways we could represent our `$user` structure as JSON, but in this case we've opted to encapsulate it in an object and have the `name` as an array (since JSON arrays preserve order).
205There are different ways we could represent our `$user` structure as JSON, but in this case we've opted to encapsulate it in an object and have the `name` as an array (since JSON arrays preserve order).
211−Since an object is represented as a `[%o p=(map @t json)]`, we can use `malt` to convert a `list` of key-value pairs to a `(map @t json)`.
207+[`+enjs:format`](/docs/hoon/reference/zuse/2d_1-5#enjsformat)includes the convenient [`+pairs`](/docs/hoon/reference/zuse/2d_1-5#pairsenjsformat) function, which converts a list of `[@t json]` to an object containing those key-value pairs. We've used this to assemble the final object. Note that if you happen to have only a single key-value pair rather than a list, you can use [`+frond`](/docs/hoon/reference/zuse/2d_1-5#frondenjsformat) instead of `+pairs`.
213−For the `joined` field, we've used the [+sect](/docs/hoon/reference/zuse/2d_1-5#sectenjsformat) function from [+enjs:format](/docs/hoon/reference/zuse/2d_1-5#enjsformat) to convert the `@da` to a Unix seconds timestamp in a `$json` number. The `+sect` function, like others in `+enjs`, takes in a noun (in this case a `@da`) and produces `$json` (in this case a `[%n @ta]` number). `+enjs` contains a handful of useful functions like this, but for the rest we've just hand-made the `$json` structure. This is fairly typical when encoding `$json`, it's usually [decoding](#converting-from-json) that makes more extensive use of the `$json` utility functions in `+format`.
209+For the `joined` field, we've used the [`+sect`](/docs/hoon/reference/zuse/2d_1-5#sectenjsformat) function from `+enjs` to convert the `@da` to a Unix seconds timestamp in a `$json` number. The `+sect` function, like others in `+enjs`, takes in a noun (in this case a `@da`) and produces `$json` (in this case a `[%n @ta]` number). `+enjs` contains a handful of useful functions like this, but for the rest we've just hand-made the `$json` structure. This is fairly typical when encoding `$json`, it's usually [decoding](#converting-from-json) that makes more extensive use of the `$json` utility functions in `+format`.
215For the `name` field we've just formed a cell of `%a` and a list of `$json` strings, since a `$json` array is `[%a p=(list json)]`. Note we've separated this part into its own arm and wrapped the whole thing in a `|^` - a core with a `$` arm that's computed immediately. This is simply for readability - our structure here is quite simple but when dealing with deeply-nested `$json` structures or complex logic, having a single giant function can quickly become unwieldy.
211For the `name` field we've just formed a cell of `%a` and a list of `$json` strings, since a `$json` array is `[%a p=(list json)]`. Note we've separated this part into its own arm and wrapped the whole thing in a `|^` - a core with a `$` arm that's computed immediately. This is simply for readability - our structure here is quite simple but when dealing with deeply-nested `$json` structures or complex logic, having a single giant function can quickly become unwieldy.
@@ -235,19 +231,113 @@ Here's our arm that converts `$json` to our `$user` structure:
236This is the inverse of the [encoding](#converting-to-json) function described in the previous section.
232This is the inverse of the [encoding](#converting-to-json) function described in the previous section.
238−We make extensive use of [+dejs:format](/docs/hoon/reference/zuse/2d_6) functions here, so we've used `=,` to expose the namespace and allow succinct `+dejs` function calls.
234+We make extensive use of [`+dejs:format`](/docs/hoon/reference/zuse/2d_6) functions here, so we've used `=,` to expose the namespace and allow succinct `+dejs` function calls.
236+We use the [`+ot`](/docs/hoon/reference/zuse/2d_6#otdejsformat) function from `+dejs:format` to decode the `$json` object to a n-tuple. It's a wet gate that takes a list of pairs of keys and other `+dejs` functions and produces a new gate that takes the `$json` to be decoded (which we've given it in `jon`).
238+The [`+so`](/docs/hoon/reference/zuse/2d_6#sodejsformat) functions just decode `$json` strings to `cord`s. The [`+at`](/docs/hoon/reference/zuse/2d_6#atdejsformat) function converts a `$json` array to a tuple, decoding each element with the respective function given in its argument list. Like `+ot`, `+at` is also a wet gate that produces a gate that takes `$json`. In our case we've used `+so` for each element, since they're all strings.
240+For `joined`, we've used the [`+du`](/docs/hoon/reference/zuse/2d_6#dudejsformat) function, which converts a Unix seconds timestamp in a `$json` number to a `@da` (it's basically the inverse of the `+sect:enjs:format` we used earlier).
242+Notice how `+ot` takes in other `+dejs` functions in its argument. One of its arguments includes the `+at` function which itself takes in other `+dejs` functions. There are several `+dejs` functions like this that allow complex nested JSON structures to be decoded. For other examples of common `+dejs` functions like this, see the [More `+dejs`](#more-dejs) section below.
244+There are dozens of different functions in [`+dejs:format`](/docs/hoon/reference/zuse/2d_6) that will cover a great many use cases. If there isn't a `+dejs` function for a particular case, you can also just write a custom function - it just has to take `$json`. Note there's also the [`+dejs-soft:format`](/docs/hoon/reference/zuse/2d_7) functions - these are similar to `+dejs` functions except they produce `unit`s rather than simply crashing if decoding fails.
248+We looked at the commonly used `+ot` function in the [first example](#converting-from-json), now let's look at a couple more common `+dejs` object functions.
252+The [`+of`](/docs/hoon/reference/zuse/2d_6#ofdejsformat) function takes an object containing a single key-value pair, decodes the value with the corresponding `+dejs` function in a key-function list, and produces a key-value tuple. This is useful when there are multiple possible objects you might receive, and tagged unions are a common data structure in hoon.
254+Let's look at an example. Here's a gate that takes in some `$json`, decodes it with an `+of` function that can handle three possible objects, casts the result to a tagged union, switches against its head with `?-`, performs some transformation and finally returns the result. You can save it as `gen/of-test.hoon` in the `%base` desk of a fake ship and `|commit %base`.
264+ %foo (cat 3 +.fbb '!!!')
265+ %bar ?:(+.fbb 'Yes' 'No')
266+ %baz :((cury cat 3) p.fbb q.fbb r.fbb)
271+ [%baz p=@t q=@t r=@t]
278+ baz+(at ~[so so so])
240−We use the [+ot](/docs/hoon/reference/zuse/2d_6#otdejsformat) function from `+dejs:format` to decode the `$json` object to a n-tuple. It's a wet gate that takes a list of pairs of keys and other `+dejs` functions and produces a new gate that takes the `$json` to be decoded (which we've given it in `jon`).
285+```
286+> +of-test (need (de-json:html '{"foo":"Hello"}'))
289+> +of-test (need (de-json:html '{"bar":true}'))
292+> +of-test (need (de-json:html '{"baz":["a","b","c"]}'))
298+The [`+ou`](/docs/hoon/reference/zuse/2d_6#oudejsformat) function decodes a `$json` object to an n-tuple using the matching functions in a key-function list. Additionally, it lets you set some key-value pairs in an object as optional and others as mandatory. The mandatory ones crash if they're missing and the optional ones are replaced with a given noun.
300+`+ou` is different to other `+dejs` functions - the functions it takes are `$-((unit json) grub)` rather than the usuall `$-(json grub)` of most `+dejs` functions. There are only two `+dejs` functions that fit this - [`+un`](/docs/hoon/reference/zuse/2d_6#undejsformat) and [`+uf`](/docs/hoon/reference/zuse/2d_6#ufdejsformat). These are intended to be used with `+ou` - you would wrap each function in the key-function list of `+ou` with either `+un` or `+uf`.
302+`+un` crashes if its argument is `~`. `+ou` gives functions a `~` if the matching key-value pair is missing in the `$json` object, so `+un` crashes if the key-value pair is missing. Therefore, `+un` lets you set key-value pairs as mandatory.
304+`+uf` takes two arguments - a noun and a `+dejs` function. If the `(unit json)` it's given by `+ou` is `~`, it produces the noun it was given rather than the product of the `+dejs` function. This lets you specify key-value pairs as optional, replacing missing ones with whatever you want.
242−The [+so](/docs/hoon/reference/zuse/2d_6#sodejsformat) functions just decode `$json` strings to `cord`s. The [+at](/docs/hoon/reference/zuse/2d_6#atdejsformat) function converts a `$json` array to a tuple, decoding each element with the respective function given in its argument list. Like `+ot`, `+at` is also a wet gate that produces a gate that takes `$json`. In our case we've used `+so` for each element, since they're all strings.
306+Let's look at a practical example. Here's a generator you can save in the `%base` desk of a fake ship in `gen/ou-test.hoon` and `|commit %base`. It takes in a `$json` object and produces a triple. The `+ou` in `+decode` has three key-function pairs - the first two are mandatory and the last is optional, producing the bunt of a set if the `%baz` key is missing.
244−For `joined`, we've used the [+du](/docs/hoon/reference/zuse/2d_6#dudejsformat) function, which converts a Unix seconds timestamp in a `$json` number to a `@da` (it's basically the inverse of the `+sect:enjs:format` we used earlier).
308+#### `ou-test.hoon`
246−There are dozens of different functions in [+dejs:format](/docs/hoon/reference/zuse/2d_6) that will cover a great many use cases. If there isn't a `+dejs` function for a particular case, one can also just write a custom function - it just has to take `$json`. Note there's also the [+dejs-soft:format](/docs/hoon/reference/zuse/2d_7) functions - these are similar to `+dejs` functions except they produce `unit`s rather than simply crashing if decoding fails.
310+```hoon
312+|^ ^- [@t ? (set @ud)]
319+ baz+(uf *(set @ud) (as ni))
327+> +ou-test (need (de-json:html '{"foo":"hello","bar":true,"baz":[1,2,3,4]}'))
328+['hello' %.y {1 2 3 4}]
330+> +ou-test (need (de-json:html '{"foo":"hello","bar":true}'))
333+> +ou-test (need (de-json:html '{"foo":"hello"}'))
335+dojo: hoon expression failed
248## `mark` file example
338## `mark` file example
250−Here's a simple `mark` file for the `$user` structure we created in the [previous section](#json-encoding-and-decoding-example). It imports the [json-test.hoon](#json-testhoon) library we created and saved in our `%home` desk's `/lib` directory.
340+Here's a simple `mark` file for the `$user` structure we created in the [previous section](#json-encoding-and-decoding-example). It imports the [json-test.hoon](#json-testhoon) library we created and saved in our `%base` desk's `/lib` directory.
252#### `user.hoon`
342#### `user.hoon`
@@ -268,7 +358,7 @@ Here's a simple `mark` file for the `$user` structure we created in the [previou
271−The [Marks section](/docs/arvo/clay/marks/marks) of the Clay documentation covers `mark` files comprehensively and is worth reading through if one wants to write a mark file.
361+The [Marks section](/docs/arvo/clay/marks/marks) of the Clay documentation covers `mark` files comprehensively and is worth reading through if you want to write a mark file.
273In brief, a mark file contains a `door` with three arms. The door's sample type is the type of the data in question - in our case the `$user` structure. The `+grab` arm contains methods for converting _to_ our mark, and the `+grow` arm contains methods for converting _from_ our mark. The `+noun` arms are mandatory, and then we've added `+json` arms which respectively call the `+from-js` and `+to-js` functions from our `json-test.hoon` library. The final `+grad` arm defines various revision control functions, in our case we've delegated these to the `%noun` mark.
363In brief, a mark file contains a `door` with three arms. The door's sample type is the type of the data in question - in our case the `$user` structure. The `+grab` arm contains methods for converting _to_ our mark, and the `+grow` arm contains methods for converting _from_ our mark. The `+noun` arms are mandatory, and then we've added `+json` arms which respectively call the `+from-js` and `+to-js` functions from our `json-test.hoon` library. The final `+grad` arm defines various revision control functions, in our case we've delegated these to the `%noun` mark.