on master help sign in
Merge pull request #1249 from tinnus-napbus/add-gall-guide

Add new gall guide.

commitab8c292312ddc155450e10b03fc5e95fe1a4f938 authorpoprox <51337059+drbeefsupreme@users.noreply.github.com> dateTue 14 Dec 2021, 18:58 (5 years ago) parent76b0d98fd084dc
20 files changed+4681−18
content/docs/arvo/concepts/scry.md+1−1 content/docs/development/develop.md+1−1 content/docs/userspace/_index.md+1−1 content/docs/userspace/gall-guide.md−14 content/docs/userspace/gall-guide/1-arvo.md+305 content/docs/userspace/gall-guide/10-scry.md+362 content/docs/userspace/gall-guide/11-fail.md+146 content/docs/userspace/gall-guide/12-next-steps.md+42 content/docs/userspace/gall-guide/2-agent.md+305 content/docs/userspace/gall-guide/3-imports-and-aliases.md+285 content/docs/userspace/gall-guide/4-lifecycle.md+487 content/docs/userspace/gall-guide/5-cards.md+244 content/docs/userspace/gall-guide/6-pokes.md+538 content/docs/userspace/gall-guide/7-sur-and-marks.md+408 content/docs/userspace/gall-guide/8-subscriptions.md+911 content/docs/userspace/gall-guide/9-vanes.md+284 content/docs/userspace/gall-guide/_index.md+45 content/docs/userspace/gall-guide/intro.md+83 content/docs/userspace/gall-guide/types.md+228 content/docs/userspace/overview.md+5−1
content/docs/arvo/concepts/scry.md+1−1
@@ -74,4 +74,4 @@ The webserver vane Eyre has a system which allows clients like web browsers to p
7474
7575[Eyre's External API Reference](/docs/arvo/eyre/external-api-ref#scry) - Documentation of Eyre's scry system which allows web clients to perform scries over HTTP.
7676
77−[The Gall Guide](/docs/userspace/gall-guide) - The Gall tutorial includes a section about `+on-peek` and writing scry endpoints.
77+[The Gall Guide](/docs/userspace/gall-guide/intro) - The Gall tutorial includes a section about `+on-peek` and writing scry endpoints.
content/docs/development/develop.md+1−1
@@ -70,7 +70,7 @@ to see how this pattern works in practice.
7070
7171If you want to learn how to build gall agents, your best bet is to first [learn
7272the Hoon programming language](/docs/hoon/hoon-school/), and then follow that
73−up with the [Gall guide](/docs/userspace/gall-guide).
73+up with the [Gall guide](/docs/userspace/gall-guide/intro).
7474
7575### Contribute to Urbit Core {#core}
7676
content/docs/userspace/_index.md+1−1
@@ -12,7 +12,7 @@ Urbit's userspace is managed by Arvo's Gall vane.
1212
1313An overview of userspace in Urbit.
1414
15−## [Gall Guide](/docs/userspace/gall-guide)
15+## [Gall Guide](/docs/userspace/gall-guide/intro)
1616
1717Learn how to develop Gall agents.
1818
content/docs/userspace/gall-guide.mddeleted−14
@@ -1,14 +0,0 @@
1−+++
2−title = "Gall Guide"
3−weight = 5
4−template = "doc.html"
5−+++
6−
7−[**The Complete Guide to Gall and Userspace Apps on
8−Urbit**](https://github.com/timlucmiptev/gall-guide), or more colloquially
9−referred to as just "the Gall Guide," was contributed by a member of our
10−community named `~timluc-miptev`.
11−
12−Our plan is ultimately to make this part of the official documentation found
13−here, but for now, know that it has our official seal of approval and is
14−actively maintained.
content/docs/userspace/gall-guide/1-arvo.mdnew file+305
@@ -0,0 +1,305 @@
1++++
2+title = "1. Arvo"
3+weight = 5
4+template = "doc.html"
5++++
6+
7+This document is a prologue to the Gall guide. If you've worked though [Hoon
8+School](/docs/hoon/hoon-school/intro) (or have otherwise learned the basics of
9+Hoon), you'll likely be familiar with generators, but not with all the other
10+parts of the Arvo operating system or the way it fits together. We'll go over
11+the basic details here so you're better oriented to learn Gall agent
12+development. We'll not go into the internal workings of the kernel much, but
13+just what is necessary to understand it from the perspective of userspace.
14+
15+## Arvo and its Vanes
16+
17+[Arvo](/docs/arvo/overview) is the Urbit OS and kernel which is written in
18+[Hoon](/docs/glossary/hoon), compiled to [Nock](/docs/gossary/nock), and
19+executed by the runtime environment and virtual machine
20+[Vere](/docs/glossary/vere). Arvo has eight kernel modules called vanes:
21+[Ames](/docs/arvo/ames/ames), [Behn](/docs/arvo/behn/behn),
22+[Clay](/docs/arvo/clay/clay), [Dill](/docs/arvo/dill/dill),
23+[Eyre](/docs/arvo/eyre/eyre), [Gall](/docs/arvo/gall/gall),
24+[Iris](/docs/arvo/iris/iris), and [Jael](/docs/arvo/jael/jael).
25+
26+Arvo itself has its own small codebase in `/sys/arvo.hoon` which primarily
27+implements the [transition function](/docs/arvo/overview#operating-function) `(State, Event) -> (State, Effects)` for
28+events injected by the runtime. It also handles inter-vane messaging, the
29+[scry](/docs/arvo/concepts/scry) system, and a couple of other things. Most of
30+the heavy lifting is done by the vanes themselves - Arvo itself typically just
31+routes events to the relevant vanes.
32+
33+Each vane has its own state. Gall's state contains the agents it's managing,
34+Clay's state contains all the desks and their files, Jael's state contains all
35+its PKI data, etc. All the vanes and their states live in Arvo's state, so
36+Arvo's state ultimately contains the entire OS and its data.
37+
38+Here's a brief summary of each of the vanes:
39+
40+- **Ames**: This is both the name of Urbit's networking protocol, as well as the
41+ vane that handles communications over it. All inter-ship communications are
42+ done with Ames, but you'd not typically deal with it directly in a Gall agent
43+ because Gall itself handles it for you.
44+- **Behn**: A simple timer vane. Behn lets your Gall agent set timers which go off
45+ at the time specified and notify your agent.
46+- **Clay**: Filesystem vane. Clay is a revision-controlled, typed filesystem with a
47+ built-in build system. Your agent's source code lives in Clay. Your agent's
48+ source code and relevant files are automatically built and loaded upon
49+ installation, so your Gall agent itself would not need to interact with Clay
50+ unless you specifically wanted to read and write files.
51+
52+- **Dill**: Terminal driver vane. You would not typically interact with Dill
53+ directly; printing debug messages to the terminal is usually done with hinting runes
54+ and functions rather than tasks to Dill, and CLI apps are mediated by a
55+ sub-module of the `%hood` system agent called `%drum`. CLI apps will not be touched
56+ on in this guide, but there's a separate [CLI
57+ Apps](/docs/hoon/guides/cli-tutorial) guide which covers them if you're
58+ interested.
59+- **Eyre**: Webserver vane. App web front-ends are served via Eyre. It's possible to
60+ handle HTTP requests directly in a Gall agent (see the [Eyre
61+ Guide](/docs/arvo/eyre/guide) for details), but usually you'd just serve a
62+ front-end [glob](/docs/userspace/dist/glob) via the `%docket` agent, so you'd
63+ not typically have your agent deal with Eyre directly.
64+
65+- **Gall**: App management vane; this is where you agent lives.
66+
67+- **Iris**: Web client vane. If you want your agent to query external web APIs and
68+ the like, it's done via Iris. Oftentimes web API interactions are
69+ spun out into [threads](/docs/userspace/threads/overview) to avoid
70+ complicating the Gall agent itself, so a Gall agent would not necessary deal
71+ with Iris directly, even if it made use of external APIs.
72+- **Jael**: Key infrastructure vane. Jael keeps track of PKI data for your ship and
73+ other ships on the network. Jael's data is most heavily used by Ames, and
74+ since Gall handles Ames communications for you, you'd not typically deal with
75+ Jael directly unless your were specifically writing something that made use of
76+ its data.
77+
78+## Userspace
79+
80+Gall agents live in "userspace" as opposed to "kernelspace". Kernelspace is Arvo
81+and its eight vanes. Userspace is primarily Gall agents, generators, threads,
82+front-ends, and all of their related files in Clay. The distinction looks
83+something like this:
84+
85+[![kernelspace/userspace diagram](https://media.urbit.org/docs/userspace/gall-guide/kernelspace-userspace-diagram.svg)](https://media.urbit.org/docs/userspace/gall-guide/kernelspace-userspace-diagram.svg)
86+
87+By and large, Gall _is_ the userspace vane - the majority of userspace is either
88+Gall agents, or things used by Gall agents. Apart from the agents themselves,
89+there's also:
90+
91+- **Generators**: These are basically scripts. You'll likely already be familiar
92+ with these from Hoon School. Aside from learning exercises, their main use is
93+ to make interacting with Gall agents easier from the dojo. Rather than having
94+ to manually craft `%poke`s to agents, generators can take a simpler input,
95+ reformat it into what the agent actually expects, and poke the agent with it.
96+ When you do something like `:dojo|wipe` in the dojo, you're actually running
97+ the `/gen/dojo/wipe.hoon` generator and poking the `%dojo` agent with its
98+ output.
99+- **Threads**: While generators are for strictly synchronous operations, threads
100+ make it easy to implement sequences of asynchronous operations. Threads are
101+ managed by the `%spider` agent. They can be used as mere scripts like
102+ generators, but their main purpose is for performing complex IO. For example,
103+ suppose you need to query some external web API, then with the data in its
104+ response you make another API call, and then another, before finally having
105+ the data you need. If one of the API calls fails, your Gall agent is
106+ potentially left in a strange intermediary state. Instead, you can put all the
107+ IO logic in a separate thread which is completely atomic. That way the Gall
108+ agent only has to deal with the two conditions of success or failure. Writing
109+ threads is covered in a [separate
110+ guide](/docs/userspace/threads/basics/fundamentals), which you might like to
111+ work through after completing the Gall Guide.
112+
113+- **Front-end**: Web UIs. It's possible for Gall agents to handle HTTP requests
114+ directly and dynamically produce responses, but it's also possible to have a
115+ static [glob](/docs/userspace/dist/glob) of HTML, CSS, Javascript, images,
116+ etc, which are served to the client like an ordinary web app. Such front-end
117+ files are typically managed by the `%docket` agent which serves them via Eyre.
118+ The [software distribution guide](/docs/userspace/dist/guide) covers this in
119+ detail, and you might like to work through it after completing the Gall Guide.
120+
121+## The Filesystem
122+
123+On an ordinary OS, you have persistent disk storage and volatile memory. An
124+application is launched by reading an executable file on disk, loading it into
125+memory and running it. The application will maybe read some more files from
126+disk, deserialize them into data structures in memory, perform some computations
127+and manipulate the data, then serialize the new data and write it back to disk.
128+This process is necessary because persistent storage is too slow to operate on
129+directly and the fast memory is wiped when it loses power. The result is that
130+all non-emphemeral data is ultimately stored as files in the filesystem on disk.
131+Arvo on the other hand is completely different.
132+
133+Arvo has no concept of volatile memory - its whole state is assumed to be
134+persistent. This means it's unnecessary for a Gall agent to write its data to
135+the filesystem or read it in from the filesystem - an agent can just modify its
136+state in situ and leave it there. The urbit runtime writes events to disk and
137+backs up Arvo's state on the host OS to ensure data integrity but Arvo itself
138+isn't concerned with such details.
139+
140+The result of this total persistence is that the filesystem—Clay—does not have
141+the same fundamental role as on an ordinary OS. In Arvo, very little of its data
142+is actually stored in Clay. The vast majority is just in the state of Gall
143+agents and vanes. For example, none of the chat messages, notebooks, etc, in the
144+Groups app exist in Clay - they're all in the state of the `%graph-store` agent.
145+For the most part, Clay just stores source code.
146+
147+Clay has a few unique features - it's a typed filesystem, with all file types
148+defined in `mark` files. It's revision controlled, in a similar way to git. It
149+also has a built-in build system (formerly a separate vane called Ford, but was
150+merged with Clay in 2020 to make atomicity of upgrades easier). We'll look at
151+some of these features in more detail later in the guide.
152+
153+## Desk Anatomy
154+
155+The fundamental unit in Clay is a desk. Desks are kind of like git repositories.
156+By default, new urbits come with the following desks included: `%base`, `%garden`, `%landscape`, `%webterm`
157+and `%bitcoin`.
158+
159+- `%base` - This desk contains the kernel as well as some core agents and utilities.
160+- `%garden` - This desk contains agents and utilities for managing apps, and the
161+ home screen that displays other app tiles.
162+- `%landscape` - This desk contains everything for the Groups app.
163+- `%webterm` - This desk is for the web dojo app.
164+- `%bitcoin` - This desk is for the bitcoin wallet app and bitcoin provider.
165+
166+You'll typically also have a `%kids` desk, which is just a copy of `%base` from
167+upstream that sponsored ships (moons in the case of a planet, planets in the
168+case of a star) sync their `%base` desk from. Any third-party apps you've
169+installed will also have their own desks.
170+
171+Desks are typically assumed to store their files according to the following directory structure:
172+
173+```
174+desk
175+├── app
176+├── gen
177+├── lib
178+├── mar
179+├── sur
180+├── sys
181+├── ted
182+└── tests
183+```
184+
185+- `app`: Gall agents.
186+- `gen`: Generators.
187+- `lib`: Libraries - these are imported with the `/+` Ford rune.
188+- `mar`: `mark` files, which are filetype definitions.
189+- `sur`: Structures - these typically contain type definitions and structures,
190+ and would be imported with the `/-` Ford rune.
191+- `sys`: Kernel files and standard library. Only the `%base` desk has this
192+ directory, it's omitted entirely in all other desks.
193+- `ted`: Threads.
194+- `tests`: Unit tests, to be run by the `%test` thread. This is often omitted in distributed desks.
195+
196+This directory hierarchy is not strictly enforced, but most tools expect things
197+to be in their right place. Any of these folders can be omitted if they'd
198+otherwise be empty.
199+
200+As mentioned, the `%base` desk alone includes a `/sys` directory containing the
201+kernel and standard libraries. It looks like this:
202+
203+```
204+sys
205+├── arvo.hoon
206+├── hoon.hoon
207+├── lull.hoon
208+├── vane
209+│ ├── ames.hoon
210+│ ├── behn.hoon
211+│ ├── clay.hoon
212+│ ├── dill.hoon
213+│ ├── eyre.hoon
214+│ ├── gall.hoon
215+│ ├── iris.hoon
216+│ └── jael.hoon
217+└── zuse.hoon
218+```
219+
220+- `arvo.hoon`: Source code for Arvo itself.
221+- `hoon.hoon`: Hoon standard library and compiler.
222+- `lull.hoon`: Mostly structures and type definitions for interacting with
223+ vanes.
224+- `vane`: This directory contains the source code for each of the vanes.
225+- `zuse.hoon`: This is an extra utility library. It mostly contains
226+ cryptographic functions and functions for dealing with web data like JSON.
227+
228+The chain of dependency for the core kernel files is `hoon.hoon` -> `arvo.hoon`
229+-> `lull.hoon` -> `zuse.hoon`. For more information, see the [Filesystem
230+Hierarchy](/docs/arvo/reference/filesystem) documentation.
231+
232+In addition to the directories discussed, there's a handful of special files a
233+desk might contain. All of them live in the root of the desk, and all are
234+optional in the general case, except for `sys.kelvin`, which is mandatory.
235+
236+- `sys.kelvin`: Specifies the kernel version with which the desk is compatible.
237+- `desk.bill`: Specifies Gall agents to be auto-started upon desk installation.
238+- `desk.ship`: If the desk is being republished, the original publisher can be
239+ specified here.
240+- `desk.docket-0`: Configures the front-end, tile, and other metadata for desks
241+ which include a home screen app.
242+
243+Each desk must be self-contained; it must include all the `mark`s, libraries,
244+threads, etc, that it needs. The one exception is the kernel and standard
245+libraries from the `%base` desk. Agents, threads and generators in other desks
246+all have these libraries available to them in their subject.
247+
248+## APIs
249+
250+You should now have a general idea of the different parts of Arvo, but how does
251+a Gall agent interact with these things?
252+
253+There are two basic ways of interacting with other parts of the system: by
254+scrying into them, and by passing them messages and receiving messages in response.
255+There are also two basic things to interact with: vanes, and other agents.
256+
257+- Scries: The scry system allows you to access the state of other agents and
258+ vanes in a read-only fashion. Scries can be performed from any context with
259+ the dotket (`.^`) rune. Each vane has "scry endpoints" which define what you
260+ can read, and these are comprehensively documented in the Scry Reference of
261+ each vane's section of the [Arvo documentation](/docs/arvo/overview). Agents
262+ define scry endpoints in the `+on-peek` arm of their agent core.
263+ Scries can only be done on the local ship; it is not yet possible to perform
264+ scries over the network (but this functionality is planned for the future). There is a separate [guide to
265+ scries](/docs/arvo/concepts/scry) which you might like to read through for
266+ more details.
267+- Messages:
268+ - Vanes: Each vane has a number of `task`s it can be passed and `gift`s it can
269+ respond with in its respective section of `lull.hoon`. These might do all
270+ manner of things, depending on the vane. For example, Iris might fetch an
271+ external HTTP resource for you, Clay might read or build a specified file,
272+ etc. The `task`s and `gift`s of each vane are comprehensively documented in the
273+ API Reference of each vane's section of the [Arvo
274+ documentation](/docs/arvo/overview).
275+ - Agents: These can be `%poke`d with some data, which is a request to perform a single action. They can also be `%watch`ed,
276+ which means to subscribe for updates. We'll discuss these in detail later in
277+ the guide.
278+
279+Here's a simplified diagram of the ways an agent can interact with other parts
280+of the system:
281+
282+![api diagram](https://media.urbit.org/docs/userspace/gall-guide/api-diagram.svg)
283+
284+Things like `on-poke` are arms of the agent core. Don't worry about their
285+meaning for now, we'll discuss them in detail later in the guide.
286+
287+Inter-agent messaging can occur over the network, so you can interact with
288+agents on other ships as well as local ones. You can only talk to local vanes,
289+but some vanes like Clay are able to make requests to other ships on your
290+behalf. Note this summary is simplified - vanes don't just talk in `task`s and
291+`gift`s in all cases. For example, requests from HTTP clients through Eyre (the
292+webserver vane) behave more like those from agents than vanes, and a couple of
293+other vanes also have some different behaviours. Agent interactions are also a
294+little more complicated, and we'll discuss that later, but the basic patterns
295+described here cover the majority of cases.
296+
297+## Environment Setup
298+
299+Before proceeding with the Gall Guide, you'll need to have an appropriate text
300+editor installed and configured, and know how to work with a fake ship for
301+development. Best practices are described in the [environment setup
302+guide](/docs/development/environment). Example agents and other code throughout
303+this guide will just be committed to the `%base` desk of a fake ship, but it's a
304+good idea to have a read through that guide for when you begin work on your own
305+apps.
content/docs/userspace/gall-guide/10-scry.mdnew file+362
@@ -0,0 +1,362 @@
1++++
2+title = "10. Scries"
3+weight = 50
4+template = "doc.html"
5++++
6+
7+In this lesson we'll look at scrying agents, as well as how agents handle such
8+scries. If you're not at all familiar with performing scries in general, have a
9+read through the [Scry Guide](/docs/arvo/concepts/scry), as well as the [dotket
10+rune documentation](/docs/hoon/reference/rune/dot#-dotket).
11+
12+## Scrying
13+
14+A scry is a read-only request to Arvo's global namespace. Vanes and agents
15+define _scry endpoints_ which allow data to be requested from their states. The
16+endpoints can process the data in any way before returning it, but they cannot
17+alter the actual state - scries can only read, not modify.
18+
19+Most of the time, scry requests are handled by Arvo, which routes the request to
20+the appropriate vane. When you scry a Gall agent you actually scry Gall itself.
21+Gall interprets the request, runs it on the specified agent, and then returns
22+the result. Scries are performed with the
23+[dotket](/docs/hoon/reference/rune/dot#-dotket) (`.^`) rune. Here's a summary of
24+their format:
25+
26+![scry summary diagram](https://media.urbit.org/docs/arvo/scry-diagram-v2.svg)
27+
28+A note on `care`s: Cares are most carefully implemented by Clay, where they specify
29+submodules and have tightly defined behaviors. For Gall agents, most of these
30+don't have any special behavior, and are just used to indicate the general kind
31+of data produced by the endpoint. There are a handful of exceptions to this:
32+`%d`, `%e`, `%u` and `%x`.
33+
34+#### `%d`
35+
36+A scry to Gall with a `%d` `care` and no `path` will produce the `desk` in which
37+the specified agent resides. For example:
38+
39+```
40+> .^(desk %gd /=hark-store=)
41+%garden
42+> .^(desk %gd /=hood=)
43+%base
44+```
45+
46+#### `%e`
47+
48+A scry to Gall with a `%e` `care`, a `desk` rather than agent in the `desk`
49+field of the above diagram, and no path, will produce a set of all installed
50+agents on that desk and their status. For example:
51+
52+```
53+> .^((set [=dude:gall live=?]) %ge /=garden=)
54+{ [dude=%hark-system-hook live=%.y]
55+ [dude=%treaty live=%.y]
56+ [dude=%docket live=%.y]
57+ [dude=%settings-store live=%.y]
58+ [dude=%hark-store live=%.y]
59+}
60+```
61+
62+#### `%u`
63+
64+A scry to Gall with a `%u` `care` and no `path` will check whether or not the
65+specified agent is installed and running:
66+
67+```
68+> .^(? %gu /=btc-wallet=)
69+%.y
70+> .^(? %gu /=btc-provider=)
71+%.n
72+> .^(? %gu /=foobar=)
73+%.n
74+```
75+
76+#### `%x`
77+
78+A scry to Gall with a `%x` `care` will be passed to the agent for handling. Gall
79+handles `%x` specially, and expects an extra field at the end of the `path` that
80+specifies the `mark` to return. Gall will take the data produced by the
81+specified endpoint and try to convert it to the given mark, crashing if the mark
82+conversion fails. The extra field specifying the mark is not passed through to
83+the agent itself. Here's a couple of examples:
84+
85+```
86+> =store -build-file /=landscape=/sur/graph-store/hoon
87+
88+> .^(update:store %gx /=graph-store=/keys/noun)
89+[p=~2021.11.18..10.50.41..c914 q=[%keys resources={[entity=~zod name=%dm-inbox]}]]
90+
91+> (crip (en-json:html .^(json %gx /=graph-store=/keys/json)))
92+'{"graph-update":{"keys":[{"name":"dm-inbox","ship":"zod"}]}}'
93+```
94+
95+The majority of Gall agents simply take `%x` `care`s in their scry endpoints,
96+but in principle it's possible for a Gall agent to define a scry endpoint that
97+takes any one of the `care`s listed in the diagram above. An agent's scry
98+endpoints are defined in its `on-peek` arm, which we'll look at next.
99+
100+## Handling scries
101+
102+When a scry is performed on a Gall agent, Gall will strip out some extraneous
103+parts, and deliver it to the agent's `on-peek` arm as a `path`. The `path` will
104+only have two components from the diagram above: The _care_ and the _path_. For
105+example, a scry of `.^(update:store %gx /=graph-store=/keys/noun)` will come
106+into the `on-peek` arm of `%graph-store` as `/x/keys`.
107+
108+The `on-peek` arm produces a `(unit (unit cage))`. The reason for the double
109+`unit` is that Arvo interprets `~` to mean the scry path couldn't be resolved,
110+and interprets `[~ ~]` to means it resolved to nothing. In either case the
111+dotket expression which initiated the scry will crash. The `cage` will contain
112+the actual data to return.
113+
114+An ordinary `on-peek` arm, therefore, begins like so:
115+
116+```hoon
117+++ on-peek
118+ |= =path
119+ ^- (unit (unit cage))
120+ ....
121+```
122+
123+Typically, you'd handle the `path` similarly to `on-watch`, as we discussed in
124+the lesson on subscriptions. You'd use something like a wutlus expression to
125+test the value of the `path`, defining your scry endpoints like so:
126+
127+```hoon
128+?+ path (on-peek:def path)
129+ [%x %some %path ~] ....
130+ [%x %foo ~] ....
131+ [%x %blah @ ~]
132+ =/ =ship (slav %p i.t.t.path)
133+ .....
134+....
135+```
136+
137+Each endpoint would then compose the `(unit (unit cage))`. The simplest way to
138+format it is like:
139+
140+```hoon
141+``noun+!>('some data')
142+```
143+
144+If it requires a more complex expression to retrieve or compose the data, you
145+can do something like:
146+
147+```hoon
148+:^ ~ ~ %some-mark
149+!> ^- some-type
150+:+ 'foo'
151+ 'bar'
152+'baz'
153+```
154+
155+Previously we discussed custom `mark` files. Such mark files are most commonly
156+used when the data might be accessed through Eyre's HTTP API, and therefore
157+required JSON conversion methods. We'll cover such things in part two of the
158+Gall Guide, but note that if that's the case for your agent, you may wish to
159+also have your scry endpoints return data with your custom `mark` so it can
160+easily be converted to JSON when accessed from the web.
161+
162+In some cases, typically with scry `path`s that contain wildcards like the `[%x %blah @ ~]` example above, your agent may not always be able to find the
163+requested data. In such cases, you can just produce a cell of `[~ ~]` for the
164+`(unit (unit cage))`. Keep in mind, however, that this will result in a crash
165+for the dotket expression which initiated the scry. In some cases you may want
166+that, but in other cases you may not, so instead you could wrap the data inside
167+the `vase` in a `unit` and have _that_ be null instead. It all depends on the
168+needs of your particular application and its clients.
169+
170+## Example
171+
172+Here's a simple example agent with three scry endpoints:
173+
174+#### `peeker.hoon`
175+
176+```hoon
177+/+ default-agent, dbug
178+|%
179++$ versioned-state
180+ $% state-0
181+ ==
182++$ state-0 [%0 data=(map @p @t)]
183++$ card card:agent:gall
184+--
185+%- agent:dbug
186+=| state-0
187+=* state -
188+^- agent:gall
189+|_ =bowl:gall
190++* this .
191+ def ~(. (default-agent this %.n) bowl)
192+::
193+++ on-init
194+ ^- (quip card _this)
195+ `this
196+::
197+++ on-save
198+ ^- vase
199+ !>(state)
200+::
201+++ on-load
202+ |= old-state=vase
203+ ^- (quip card _this)
204+ =/ old !<(versioned-state old-state)
205+ ?- -.old
206+ %0 `this(state old)
207+ ==
208+::
209+++ on-poke
210+ |= [=mark =vase]
211+ ^- (quip card _this)
212+ ?> =(src.bowl our.bowl)
213+ ?+ mark (on-poke:def mark vase)
214+ %noun
215+ `this(data (~(put by data) !<([@p @t] vase)))
216+ ==
217+::
218+++ on-watch on-watch:def
219+++ on-leave on-leave:def
220+::
221+++ on-peek
222+ |= =path
223+ ^- (unit (unit cage))
224+ ?+ path (on-peek:def path)
225+ [%x %all ~] ``noun+!>(data)
226+ ::
227+ [%x %has @ ~]
228+ =/ who=@p (slav %p i.t.t.path)
229+ ``noun+!>(`?`(~(has by data) who))
230+ ::
231+ [%x %get @ ~]
232+ =/ who=@p (slav %p i.t.t.path)
233+ =/ maybe-res (~(get by data) who)
234+ ?~ maybe-res
235+ [~ ~]
236+ ``noun+!>(`@t`u.maybe-res)
237+ ==
238+::
239+++ on-agent on-agent:def
240+++ on-arvo on-arvo:def
241+++ on-fail on-fail:def
242+--
243+```
244+
245+The agent's `on-poke` arm takes a cell of `[@p @t]` and saves it in the agent's
246+state, which contains a `(map @p @t)` called `data`. The `on-peek` arm is:
247+
248+```hoon
249+++ on-peek
250+ |= =path
251+ ^- (unit (unit cage))
252+ ?+ path (on-peek:def path)
253+ [%x %all ~] ``noun+!>(data)
254+ ::
255+ [%x %has @ ~]
256+ =/ who=@p (slav %p i.t.t.path)
257+ ``noun+!>(`?`(~(has by data) who))
258+ ::
259+ [%x %get @ ~]
260+ =/ who=@p (slav %p i.t.t.path)
261+ =/ maybe-res (~(get by data) who)
262+ ?~ maybe-res
263+ [~ ~]
264+ ``noun+!>(`@t`u.maybe-res)
265+ ==
266+```
267+
268+It defines three scry endpoints, all using a `%x` `care`: `/x/all`,
269+`/x/has/[ship]`, and `/x/get/[ship]`. The first will simply return the entire
270+`(map @p @t)` in the agent's state. The second will check whether the given ship
271+is in the map and produce a `?`. The third will produce the `@t` for the given
272+`@p` if it exists in the map, or else return `[~ ~]` to indicate the data
273+doesn't exist, producing a crash in the dotket expression.
274+
275+Let's try it out. Save the agent above as `/app/peeker.hoon` in the `%base`
276+desk, `|commit %base` and start the agent with `|rein %base [& %peeker]`.
277+
278+First, let's add some data to the map:
279+
280+```
281+> :peeker [~zod 'foo']
282+>=
283+> :peeker [~nut 'bar']
284+>=
285+> :peeker [~wet 'baz']
286+>=
287+```
288+
289+Now if we use `dbug` to inspect the state, we'll see the data has been added:
290+
291+```
292+> {[p=~wet q='baz'] [p=~nut q='bar'] [p=~zod q='foo']}
293+> :peeker +dbug [%state %data]
294+>=
295+```
296+
297+Next, let's try the `/x/all` scry endpoint:
298+
299+```
300+> .^((map @p @t) %gx /=peeker=/all/noun)
301+{[p=~wet q='baz'] [p=~nut q='bar'] [p=~zod q='foo']}
302+```
303+
304+The `/x/has/[ship]` endpoint:
305+
306+```
307+> .^(? %gx /=peeker=/has/~zod/noun)
308+%.y
309+> .^(? %gx /=peeker=/has/~wet/noun)
310+%.y
311+> .^(? %gx /=peeker=/has/~nes/noun)
312+%.n
313+```
314+
315+And finally, the `/x/get/[ship]` endpoint:
316+
317+```
318+> .^(@t %gx /=peeker=/get/~zod/noun)
319+'foo'
320+> .^(@t %gx /=peeker=/get/~wet/noun)
321+'baz'
322+```
323+
324+We'll now try scrying for a ship that doesn't exist in the map. Note that due to
325+a bug at the time of writing, the resulting crash will wipe the dojo's subject,
326+so don't try this if you've got anything pinned there that you want to keep.
327+
328+```
329+~zod:dojo> .^(@t %gx /=peeker=/get/~nes/noun)
330+crash!
331+```
332+
333+## Summary
334+
335+- Scries are read-only requests to vanes or agents which can be done inside any
336+ code, during its evaluation.
337+- Scries are performed with the dotket (`.^`) rune.
338+- Scries will fail if the scry endpoint does not exist, the requested data does
339+ not exist, or the data does not nest in the return type specified.
340+- Scries can only be performed on the local ship, not on remote ships.
341+- Gall scries with an agent name in the `desk` field will be passed to that
342+ agent's `on-peek` arm for handling.
343+- Gall scries with a `%x` `care` take a `mark` at the end of the scry `path`,
344+ telling Gall to convert the data returned by the scry endpoint to the mark
345+ specified.
346+- The `on-peek` arm takes a `path` with the `care` in the head and the `path` part
347+ of the scry in the tail, like `/x/some/path`.
348+- The `on-peek` arm produces a `(unit (unit cage))`. The outer `unit` is null if
349+ the scry endpoint does not exist, and the inner `unit` is null if the data
350+ does not exist.
351+
352+## Exercises
353+
354+- Have a read through the [Scry Guide](/docs/arvo/concepts/scry).
355+- Have a read through the [dotket rune
356+ documentation](/docs/hoon/reference/rune/dot#-dotket).
357+- Run through the [Example](#example) yourself if you've not done so already.
358+- Try adding another scry endpoint to the `peeker.hoon` agent, which uses a
359+ [`wyt:by`](/docs/hoon/reference/stdlib/2i#wytby) map function to produce the
360+ number of items in the `data` map.
361+- Have a look through the `on-peek` arms of some other agents on your ship, and
362+ try performing some scries to some of the endpoints.
content/docs/userspace/gall-guide/11-fail.mdnew file+146
@@ -0,0 +1,146 @@
1++++
2+title = "11. Failure"
3+weight = 55
4+template = "doc.html"
5++++
6+
7+In this lesson we'll cover the last agent arm we haven't touched on yet:
8+`on-fail`. We'll also touch on one last concept, which is the _helper core_.
9+
10+# Failures
11+
12+When crashes or errors occur in certain cases, Gall passes them to an agent's
13+`on-fail` arm for handling. This arm is very seldom used, almost all agents
14+leave it for `default-agent` to handle, which just prints the error message to
15+the terminal. While you're unlikely to use this arm, we'll briefly go over its
16+behavior for completeness.
17+
18+`on-fail` takes a `term` error message and a `tang`, typically containing a
19+stack trace, and often with additional messages about the error. If it weren't
20+delegated to `on-fail:def`, it would begin with:
21+
22+```hoon
23+++ on-fail
24+ |= [=term =tang]
25+ ^- (quip card _this)
26+ ....
27+```
28+
29+Gall calls `on-fail` in four cases:
30+
31+- When there's a crash in the `on-arvo` arm.
32+- When there's a crash in the `on-agent` arm.
33+- When there's a crash in the `on-leave` arm.
34+- When an agent produces a `%watch` card but the `wire`, ship, agent and `path`
35+ specified are the same as an existing subscription.
36+
37+For an `on-arvo` failure, the `term` will always be `%arvo-response`, and the
38+`tang` will contain a stack trace.
39+
40+For `on-agent`, the `term` will be the head of the `sign` (`%poke-ack`, `%fact`,
41+etc). The `tang` will contain a stack trace and a message of "closing
42+subscription".
43+
44+For an `on-leave` failure, the `term` will always be `%leave`, and the `tang`
45+will contain a stack trace.
46+
47+For a `%watch` failure, the `term` will be `%watch-not-unique`. The `tang` will
48+include a message of "subscribe wire not unique", as well as the agent name, the
49+`wire`, the target ship and the target agent.
50+
51+How you might handle these cases (if you wanted to manually handle them) depends
52+on the purpose of your particular agent.
53+
54+## Helper core
55+
56+Back in the lesson on lustar virtual arms, we briefly mentioned a common pattern
57+is to define a deferred expression for a helper core named `hc` like:
58+
59+```hoon
60++* this .
61+ def ~(. (default-agent this %.n) bowl)
62+ hc ~(. +> bowl)
63+```
64+
65+The name `do` is also used frequently besides `hc`.
66+
67+A helper core is a separate core composed into the subject of the agent core,
68+containing useful functions for use by the agent arms. Such a helper core would
69+typically contain functions that would only ever be used internally by the
70+agent - more general functions would usually be included in a separate `/lib`
71+library and imported with a [faslus](/docs/arvo/ford/ford#ford-runes) (`/+`)
72+rune. Additionally, you might recall that the example agent of the
73+[subscriptions lesson](/docs/userspace/gall-guide/8-subscriptions#example) used
74+a barket (`|^`) rune to create a door in the `on-poke` arm with a separate
75+`handle-poke` arm. That approach is typically used when functions will only be
76+used in that one arm. The helper core, on the other hand, is useful when
77+functions will be used by multiple agent arms.
78+
79+The conventional pattern is to have the helper core _below_ the agent core, so
80+the structure of the agent file is like:
81+
82+```
83+[imports]
84+[state types core]
85+[agent core]
86+[helper core]
87+```
88+
89+Recall that the build system will implicitly compose any discrete expressions.
90+If we simply added the helper core below the agent core, the agent core would be
91+composed into the subject of the helper core, which is the opposite of what we
92+want. Instead, we must inversely compose the two cores with a
93+[tisgal](/docs/hoon/reference/rune/tis#-tisgal) (`=<`) rune. We add the tisgal
94+rune directly above the agent core like:
95+
96+```hoon
97+.....
98+=<
99+|_ =bowl:gall
100++* this .
101+ def ~(. (default-agent this %.n) bowl)
102+ hc ~(. +> bowl)
103+++ on-init
104+.....
105+```
106+
107+We can then add the helper core below the agent core. The helper core is most
108+typically a door like the agent core, also with the `bowl` as its sample. This
109+is just so any functions you define in it have ready access to the `bowl`. It
110+would look like:
111+
112+```hoon
113+|_ =bowl:gall
114+++ some-function ...
115+++ another ....
116+++ etc ...
117+--
118+```
119+
120+Back in the lustar virtual arm of the agent core, we give it a deferred expression name of `hc`
121+and call it like so:
122+
123+```hoon
124+hc ~(. +> bowl)
125+```
126+
127+To get to the helper core we composed from within the door, we use a
128+[censig](/docs/hoon/reference/rune/cen#-censig) expression to call `+>` of the
129+subject (`.`) with the `bowl` as its sample. After that, any agent arms can make
130+use of helper core functions by calling them like `(some-function:hc ....)`.
131+
132+## Summary
133+
134+- `on-fail` is called in certain cases of crashes or failures.
135+- Crashes in the `on-agent`, `on-arvo`, or `on-watch` arms will trigger a call
136+ to `on-fail`.
137+- A non-unique `%watch` `card` will also trigger a call to `on-fail`.
138+- `on-fail` is seldom used - most agents just leave it to `%default-agent` to
139+ handle, which just prints the error to the terminal.
140+- A helper core is an extra core of useful functions, composed into the subject
141+ of the agent core.
142+- Helper cores are typically placed below the agent core, and composed with a
143+ tisgal (`=<`) rune.
144+- The helper core is typically a door with the `bowl` as a sample.
145+- The helper core is typically given a name of `hc` or `do` in the lustar virtual arm
146+ of the agent core.
content/docs/userspace/gall-guide/12-next-steps.mdnew file+42
@@ -0,0 +1,42 @@
1++++
2+title = "12. Next Steps"
3+weight = 60
4+template = "doc.html"
5++++
6+
7+We've now covered all the arms of a Gall agent, and everything you need to know
8+to start writing your own agent.
9+
10+The things haven't touched on yet are front-end development and integration,
11+Eyre's HTTP API for communicating with agents from the web, and dealing with
12+JSON data. A second part to the Gall Guide is coming soon, and will focus
13+exclusively on those aspects. In the mean time, you can refer to the following
14+documents for help writing a web front-end for your app:
15+
16+- [Eyre's external API reference](/docs/arvo/eyre/external-api-ref) - This
17+ explains Eyre's HTTP API, through which a browser or other HTTP client can
18+ interact with a Gall agent.
19+- [Eyre's usage guide](/docs/arvo/eyre/guide) - This walks through examples of
20+ using Eyre's HTTP API.
21+- [JSON guide](/docs/hoon/guides/json-guide) - This walks through the basics of
22+ converting Hoon data structures to JSON, for use with a web client. It also
23+ covers JSON conversion methods in `mark` files.
24+- [Zuse reference](/docs/hoon/reference/zuse/table-of-contents) - This contains
25+ documentation of all JSON encoding and decoding functions included in the
26+ `zuse.hoon` utility library.
27+- [The software distribution guide](/docs/userspace/dist/dist) - This covers
28+ everything you need to know to distribute apps to other ships. It includes
29+ details of bundling a web front-end and serving it to the user in the browser.
30+
31+In addition to these documents about creating a web-based user interface for
32+your app, there are some other guides you might like to have a look at:
33+
34+- [Threads guide](/docs/userspace/threads/overview) - Threads are like transient
35+ agents, typically used for handling complex I/O functionality for Gall
36+ agents - like interacting with an external HTTP API.
37+- [The software distribution guide](/docs/usespace/dist/dist) - This explains
38+ how to set up a desk for distribution, so other people can install your app.
39+
40+For more development resources, and for ways to get involved with the Urbit
41+development community, see the [Urbit Developers
42+site](https://developers.urbit.org/).
content/docs/userspace/gall-guide/2-agent.mdnew file+305
@@ -0,0 +1,305 @@
1++++
2+title = "2. The Agent Core"
3+weight = 10
4+template = "doc.html"
5++++
6+
7+In this lesson we'll look at the basic type and structure of a Gall agent.
8+
9+A Gall agent is a [door](/docs/glossary/door) with exactly ten [arms](/docs/glossary/arm). Each arm is responsible for
10+handling certain kinds of events that Gall feeds in to the agent. A door is
11+just a [core](/docs/glossary/core) with a sample - it's made with the
12+[barcab](/docs/hoon/reference/rune/bar#_-barcab) rune (`|_`) instead of the
13+usual [barcen](/docs/hoon/reference/rune/bar#-barcen) rune (`|%`).
14+
15+## The ten arms
16+
17+We'll discuss each of the arms in detail later. For now, here's a quick summary.
18+The arms of an agent can be be roughly grouped by purpose:
19+
20+### State management
21+
22+These arms are primarily for initializing and upgrading an agent.
23+
24+- `on-init`: Handles starting an agent for the first time.
25+- `on-save`: Handles exporting an agent's state - typically as part of the
26+ upgrade process but also when suspending, uninstalling and debugging.
27+- `on-load`: Handles loading a previously exported agent state - typically as
28+ part of the upgrade process but also when resuming or reinstalling an agent.
29+
30+### Request handlers
31+
32+These arms handle requests initiated by outside entities, e.g. other agents,
33+HTTP requests from the front-end, etc.
34+
35+- `on-poke`: Handles one-off requests, actions, etc.
36+- `on-watch`: Handles subscription requests from other entities.
37+- `on-leave`: Handles unsubscribe notifications from other, previously subscribed
38+ entities.
39+
40+### Response handlers
41+
42+These two arms handle responses to requests our agent previously initiated.
43+
44+- `on-agent`: Handles request acknowledgements and subscription updates from
45+ other agents.
46+- `on-arvo`: Handles responses from vanes.
47+
48+### Scry handler
49+
50+- `on-peek`: Handles local read-only requests.
51+
52+### Failure handler
53+
54+- `on-fail`: Handles certain kinds of crash reports from Gall.
55+
56+## Bowl
57+
58+The sample of a Gall agent door is always a `bowl:gall`. Every time an event triggers the
59+agent, Gall populates the bowl with things like the current date-time, fresh entropy,
60+subscription information, which ship the request came from, etc, so that all the
61+arms of the agent have access to that data. For the exact structure and contents
62+of the bowl, have a read through [its entry in the Gall vane types
63+documentation](/docs/arvo/gall/data-types#bowl).
64+
65+One important thing to note is that the bowl is only repopulated when there's a
66+new Arvo event. If a local agent or web client were to send multiple
67+messages to your agent at the same time, these would all arrive in the same
68+event. This means if your agent depended on a unique date-time or entropy to
69+process each message, you could run into problems if your agent doesn't account
70+for this possibility.
71+
72+## State
73+
74+If you've worked through [Hoon School](/docs/hoon/hoon-school/intro), you may
75+recall that a core is a cell of `[battery payload]`. The battery is the core
76+itself compiled to Nock, and the payload is the subject which it operates on.
77+
78+For an agent, the payload will at least contain the bowl, the usual Hoon and `zuse` standard
79+library functions, and the **state** of the agent. For example, if your agent
80+were for an address book app, it might keep a `map` of ships to address book
81+entries. It might add entries, delete entries, and modify entries. This address
82+book `map` would be part of the state stored in the payload.
83+
84+## Transition function
85+
86+If you recall from the prologue, the whole Arvo operating system works on the
87+basis of a simple transition function `(event, oldState) -> (effects, newState)`. Gall agents also function the same way. Eight of an agent's ten arms
88+produce the same thing, a cell of:
89+
90+- **Head**: A list of effects called `card`s (which we'll discuss later).
91+- **Tail**: A new agent core, possibly with a modified payload.
92+
93+It goes something like this:
94+
95+1. An event is routed to Gall.
96+2. Gall calls the appropriate arm of the agent, depending on the kind of event.
97+3. That arm processes the event, returning a list `card`s to be sent off, and
98+ the agent core itself with a modified state in the payload.
99+4. Gall sends the `card`s off and saves the modified agent core.
100+5. Rinse and repeat.
101+
102+## Virtualization
103+
104+When a crash occurs in the kernel, the system usually aborts the computation and
105+discards the event as though it never happened. Gall on the other hand
106+virtualizes all its agents, so this doesn't happen. Instead, when a crash occurs
107+in an agent, Gall intercepts the crash and takes appropriate action depending on
108+the kind of event that caused it. For example, if a poke from another ship
109+caused a crash in the `on-poke` arm, Gall will respond to the poke with a
110+"nack", a negative acknowledgement, telling the original ship the poke was
111+rejected.
112+
113+What this means is that you can intentionally design your agent to crash in
114+cases it can't handle. For example, if a poke comes in with an unexpected
115+`mark`, it crashes. If a permission check fails, it crashes. This is quite
116+different to most programs written in procedural languages, which must handle
117+all exceptions to avoid crashing.
118+
119+## Example
120+
121+Here's about the simplest valid Gall agent:
122+
123+```hoon
124+|_ =bowl:gall
125+++ on-init `..on-init
126+++ on-save !>(~)
127+++ on-load |=(vase `..on-init)
128+++ on-poke |=(cage !!)
129+++ on-watch |=(path !!)
130+++ on-leave |=(path `..on-init)
131+++ on-peek |=(path ~)
132+++ on-agent |=([wire sign:agent:gall] !!)
133+++ on-arvo |=([wire sign-arvo] !!)
134+++ on-fail |=([term tang] `..on-init)
135+--
136+```
137+
138+This is just a dummy agent that does absolutely nothing - it has no state and
139+rejects all messages by crashing. Typically we'd cast this to an `agent:gall`,
140+but in this instance we won't so it's easier to examine its structure in the
141+dojo. We'll get to what each of the arms do later. For now, we'll just consider
142+a few particular points.
143+
144+Firstly, note its structure - it's a door (created with `|_`) with a sample of
145+`bowl:gall` and the ten arms described earlier.
146+
147+Secondly, you'll notice some of the arms return:
148+
149+```hoon
150+`..on-init
151+```
152+
153+A backtick at the beginning is an irregular syntax meaning "prepend with null",
154+so for example, in the dojo:
155+
156+```
157+> `50
158+[~ 50]
159+```
160+
161+The next part has `..on-init`, which means "the subject of the `on-init` arm".
162+The subject of the `on-init` arm is our whole agent. In the [transition
163+function](#transition-function) section we mentioned that most arms return a
164+list of effects called `card`s and a new agent core. Since an empty list is `~`,
165+we've created a cell that fits that description.
166+
167+Let's examine our agent. In the dojo of a fake ship, mount the `%base` desk with
168+`|mount %base`. On the Unix side, navigate to `/path/to/fake/ship/base`, and save
169+the above agent in the `/app` directory as `skeleton.hoon`. Back in the dojo,
170+commit the file to the desk with `|commit %base`.
171+
172+For the moment we won't install our `%skeleton` agent. Instead, we'll use the
173+`%build-file` thread to build it and save it in the dojo's subject so we can
174+have a look. Run the following in the dojo:
175+
176+```
177+> =skeleton -build-file %/app/skeleton/hoon
178+```
179+
180+Now, let's have a look:
181+
182+```
183+> skeleton
184+< 10.fxw
185+ [ bowl
186+ [ [our=@p src=@p dap=@tas]
187+ [ wex=nlr([p=[wire=/ ship=@p term=@tas] q=[acked=?(%.y %.n) path=/]])
188+ sup=nlr([p=it(/) q=[p=@p q=/]])
189+ ]
190+ act=@ud
191+ eny=@uvJ
192+ now=@da
193+ byk=[p=@p q=@tas r=?([%da p=@da] [%tas p=@tas] [%ud p=@ud])]
194+ ]
195+ <17.zbp 33.wxp 14.dyd 53.vlb 77.wir 232.wfe 51.qbt 123.zao 46.hgz 1.pnw %140>
196+ ]
197+>
198+```
199+
200+The dojo pretty-prints cores with a format of `number-of-arms.hash`. You can see
201+the head of `skeleton` is `10.fxw` - that's the battery of the core, our 10-arm
202+agent. If we try printing the head of `skeleton` we'll see it's a whole lot of
203+compiled Nock:
204+
205+```
206+> -.skeleton
207+[ [ 11
208+ [ 1.953.460.339
209+ 1
210+ [ 7.368.801
211+ 7.957.707.045.546.060.659
212+ 1.852.796.776
213+ 0
214+ ]
215+ [7 15]
216+ 7
217+ 34
218+ ]
219+...(truncated for brevity)...
220+```
221+
222+The battery's not too important, it's not something we'd ever touch in practice.
223+Instead, let's have a look at the core's payload by printing the tail of
224+`skeleton`. We'll see its head is the `bowl:gall` sample we specified, and then
225+the tail is just all the usual standard library functions:
226+
227+```
228+> +.skeleton
229+[ bowl
230+ [ [our=~zod src=~zod dap=%$]
231+ [wex={} sup={}]
232+ act=0
233+ eny=0v0
234+ now=~2000.1.1
235+ byk=[p=~zod q=%$ r=[%ud p=0]]
236+ ]
237+ <17.zbp 33.wxp 14.dyd 53.vlb 77.wir 232.wfe 51.qbt 123.zao 46.hgz 1.pnw %140>
238+]
239+```
240+
241+Currently `skeleton` has no state, but of course in practice you'd want to store
242+some actual data. We'll add `foo=42` as our state with the `=+` rune at the
243+beginning of our agent:
244+
245+```hoon
246+=+ foo=42
247+|_ =bowl:gall
248+++ on-init `..on-init
249+++ on-save !>(~)
250+++ on-load |=(vase `..on-init)
251+++ on-poke |=(cage !!)
252+++ on-watch |=(path !!)
253+++ on-leave |=(path `..on-init)
254+++ on-peek |=(path ~)
255+++ on-agent |=([wire sign:agent:gall] !!)
256+++ on-arvo |=([wire sign-arvo] !!)
257+++ on-fail |=([term tang] `..on-init)
258+--
259+```
260+
261+Save the modified `skeleton.hoon` in `/app` on the `%base` desk like before, and run `|commit %base` again in the dojo. Then, rebuild it with the same `%build-file` command as before:
262+
263+```
264+> =skeleton -build-file %/app/skeleton/hoon
265+```
266+
267+If we again examine our agent core's payload by looking at the tail of
268+`skeleton`, we'll see `foo=42` is now included:
269+
270+```
271+> +.skeleton
272+[ bowl
273+ [ [our=~zod src=~zod dap=%$]
274+ [wex={} sup={}]
275+ act=0
276+ eny=0v0
277+ now=~2000.1.1
278+ byk=[p=~zod q=%$ r=[%ud p=0]]
279+ ]
280+ foo=42
281+ <17.zbp 33.wxp 14.dyd 53.vlb 77.wir 232.wfe 51.qbt 123.zao 46.hgz 1.pnw %140>
282+]
283+```
284+
285+## Summary
286+
287+- A Gall agent is a door with exactly ten specific arms and a sample of `bowl:gall`.
288+- Each of the ten arms handle different kinds of events - Gall calls the
289+ appropriate arm for the kind event it receives.
290+- The ten arms fit roughly into five categories:
291+ - State management.
292+ - Request handlers.
293+ - Response handlers.
294+ - Scry handler.
295+ - Failure handler.
296+- The state of an agent—the data it's storing—lives in the core's payload.
297+- Most arms produce a list of effects called `card`s, and a new agent core with
298+ a modified state in its payload.
299+
300+## Exercises
301+
302+- Run through the [Example](#example) yourself on a fake ship if you've not done
303+ so already.
304+- Have a look at the [`bowl` entry in the Gall data types
305+ documentation](/docs/arvo/gall/data-types#bowl) if you've not done so already.
content/docs/userspace/gall-guide/3-imports-and-aliases.mdnew file+285
@@ -0,0 +1,285 @@
1++++
2+title = "3. Imports and Aliases"
3+weight = 15
4+template = "doc.html"
5++++
6+
7+In the last lesson we looked at the most basic aspects of a Gall agent's
8+structure. Before we get into the different agent arms in detail, there's some
9+boilerplate to cover that makes life easier when writing Gall agents.
10+
11+## Useful libraries
12+
13+There are a couple of libraries that you'll very likely use in every agent you
14+write. These are [`default-agent`](#default-agent) and [`dbug`](#dbug). In
15+brief, `default-agent` provides simple default behaviours for each agent arm,
16+and `dbug` lets you inspect the state and bowl of an agent from the dojo, for
17+debugging purposes. Every example agent we look at from here on out will make
18+use of both libraries.
19+
20+Let's look at each in more detail:
21+
22+### `default-agent`
23+
24+The `default-agent` library contains a basic agent with sane default behaviours
25+for each arm. In some cases it just crashes and prints an error message to the
26+terminal, and in others it succeeds but does nothing. It has two primary uses:
27+
28+- For any agent arms you don't need, you can just have them call the matching
29+ function in `default-agent`, rather than having to manually handle events on
30+ those arms.
31+- A common pattern in an agent is to switch on the input of an arm with
32+ [wutlus](/docs/hoon/reference/rune/wut#-wutlus) (`?+`) runes or maybe
33+ [wutcol](/docs/hoon/reference/rune/wut#wutcol) (`?:`) runes. For any
34+ unexpected input, you can just pass it to the relevant arm of `default-agent`
35+ rather than handling it manually.
36+
37+The `default-agent` library lives in `/lib/default-agent/hoon` of the `%base`
38+desk, and you would typically include a copy in any new desk you created. It's
39+imported at the beginning of an agent with the
40+[faslus](/docs/arvo/ford/ford#ford-runes) (`/+`) rune.
41+
42+The library is a wet gate which takes two arguments: `agent` and `help`. The
43+first is your agent core itself, and the second is a `?`. If `help` is `%.y` (equivalently, `%&`), it
44+will crash in all cases. If `help` is `%.n` (equivalently, `%|`), it will use its defaults. You would
45+almost always have `help` as `%.n`.
46+
47+The wet gate returns an `agent:gall` door with a sample of `bowl:gall` - a
48+typical agent core. Usually you would define an alias for it in a virtual arm
49+([explained below](#virtual-arms)) so it's simple to call.
50+
51+### `dbug`
52+
53+The `dbug` library lets you inspect the state and `bowl` of your agent from the
54+dojo. It includes an `agent:dbug` function which wraps your whole `agent:gall`
55+door, adding its extra debugging functionality while transparently passing
56+events to your agent for handling like usual.
57+
58+To use it, you just import `dbug` with a
59+[faslus](/docs/arvo/ford/ford#ford-runes) (`/+`) rune at the beginning, then add
60+the following line directly before the door of your agent:
61+
62+```hoon
63+%- agent:dbug
64+```
65+
66+With that done, you can poke your agent with the `+dbug` generator from the dojo
67+and it will pretty-print its state, like:
68+
69+```
70+> :your-agent +dbug
71+```
72+
73+The generator also has a few useful optional arguments:
74+
75+- `%bowl`: Print the agent's bowl.
76+- `[%state 'some code']`: Evaluate some code with the agent's state as its
77+ subject and print the result. The most common case is `[%state %some-face]`,
78+ which will print the contents of the wing with the given face.
79+- `[%incoming ...]`: Print details of the matching incoming subscription, one
80+ of:
81+ - `[%incoming %ship ~some-ship]`
82+ - `[%incoming %path /part/of/path]`
83+ - `[%incoming %wire /part/of/wire]`
84+- `[%outgoing ...]`: Print details of the matching outgoing subscription, one
85+ of:
86+ - `[%outgoing %ship ~some-ship]`
87+ - `[%outgoing %path /part/of/path]`
88+ - `[%outgoing %wire /part/of/wire]`
89+ - `[outgoing %term %agent-name]`
90+
91+By default it will retrieve your agent's state by using its `on-save` arm, but
92+if your app implements a scry endpoint with a path of `/x/dbug/state`, it will
93+use that instead.
94+
95+We haven't yet covered some of the concepts described here, so don't worry if
96+you don't fully understand `dbug`'s functionality - you can refer back here
97+later.
98+
99+## Virtual arms
100+
101+An agent core must have exactly ten arms. However, there's a special kind of
102+"virtual arm" that can be added without actually increasing the core's arm
103+count, since it really just adds code to the other arms in the core. A virtual arm is created with the
104+[lustar](/docs/hoon/reference/rune/lus#-lustar) (`+*`) rune, and its purpose is
105+to define _deferred expressions_. It takes a list of pairs of names and Hoon
106+expressions. When compiled, the deferred expressions defined in the virtual arm are
107+implicitly inserted at the beginning of every other arm of the core, so they all
108+have access to them. Each time a name in a `+*` is called, the associated Hoon is evaluated in its place, similar to lazy evaluation except it is re-evaluated whenever needed. See the [tistar](/docs/hoon/reference/rune/tis#-tistar) reference for more information on deferred expressions.
109+
110+A virtual arm in an agent often looks something like this:
111+
112+```hoon
113++* this .
114+ def ~(. (default-agent this %.n) bowl)
115+```
116+
117+`this` and `def` are the deferred expressions, and next to each one is the Hoon
118+expression it evaluates whenever called. Notice that unlike most things that
119+take _n_ arguments, a virtual arm is not terminated with a `==`. You can define
120+as many aliases as you like. The two in this example are conventional ones you'd
121+use in most agents you write. Their purposes are:
122+
123+```hoon
124+this .
125+```
126+
127+Rather than having to return `..on-init` like we did in the last lesson,
128+instead our arms can just refer to `this` whenever modifying or returning the
129+agent core.
130+
131+```hoon
132+def ~(. (default-agent this %.n) bowl)
133+```
134+
135+This sets up the `default-agent` library we [described above](#default-agent),
136+so you can easily call its arms like `on-poke:def`, `on-agent:def`, etc.
137+
138+## Additional cores
139+
140+While Gall expects a single 10-arm agent core, it's possible to include
141+additional cores by composing them into the subject of the agent core itself.
142+The contents of these cores will then be available to arms of the agent core.
143+
144+Usually to compose cores in this way, you'd have to do something like insert
145+[tisgar](/docs/hoon/reference/rune/tis#-tisgar) (`=>`) runes in between them.
146+However, Clay's build system implicitly composes everything in a file by
147+wrapping it in a [tissig](/docs/hoon/reference/rune/tis#-tissig) (`=~`)
148+expression, which means you can just butt separate cores up against one another
149+and they'll all still get composed.
150+
151+You can add as many extra cores as you'd like before the agent core, but
152+typically you'd just add one containing type definitions for the agent's state,
153+as well as any other useful structures. We'll look at the state in more detail
154+in the next lesson.
155+
156+## Example
157+
158+Here's the `skeleton.hoon` dummy agent from the previous lesson, modified with
159+the concepts discussed here:
160+
161+```hoon
162+/+ default-agent, dbug
163+|%
164++$ card card:agent:gall
165+--
166+%- agent:dbug
167+^- agent:gall
168+|_ =bowl:gall
169++* this .
170+ def ~(. (default-agent this %.n) bowl)
171+++ on-init
172+ ^- (quip card _this)
173+ `this
174+++ on-save on-save:def
175+++ on-load on-load:def
176+++ on-poke on-poke:def
177+++ on-watch on-watch:def
178+++ on-leave on-leave:def
179+++ on-peek on-peek:def
180+++ on-agent on-agent:def
181+++ on-arvo on-arvo:def
182+++ on-fail on-fail:def
183+--
184+```
185+
186+The first line uses the faslus (`/+`) Ford rune to import
187+`/lib/default-agent.hoon` and `/lib/dbug.hoon`, building them and loading them
188+into the subject of our agent so they're available for use. You can read more
189+about Ford runes in the [Ford section of the vane
190+documenation](/docs/arvo/ford/ford#ford-runes).
191+
192+Next, we've added an extra core. Notice how it's not explicitly composed, since
193+the build system will do that for us. In this case we've just added a single
194+`card` arm, which makes it simpler to reference the `card:agent:gall` type.
195+
196+After that core, we call `agent:dbug` with our whole agent core as its argument.
197+This allows us to use the `dbug` features described earlier.
198+
199+Inside our agent door, we've added an extra virtual arm and defined a couple
200+deferred expressions:
201+
202+```hoon
203++* this .
204+ def ~(. (default-agent this %.n) bowl)
205+```
206+
207+In most of the arms, you see we've been able to replace the dummy code with
208+simple calls to the corresponding arms of `default-agent`, which we set up as a deferred
209+expression named `def` in the virtual arm. We've also replaced the old `..on-init`
210+with our deferred expression named `this` in the `on-init` arm as an example - it makes things a bit
211+simpler.
212+
213+You can save the code above in `/app/skeleton.hoon` of your `%base` desk like
214+before and `|commit %base` in the dojo. Additionally, you can start the agent so
215+we can try out `dbug`. To start it, run the following in the dojo:
216+
217+```
218+> |rein %base [& %skeleton]
219+```
220+
221+For details of using the `|rein` generator, see the [Dojo
222+Tools](/docs/userspace/dist/tools#rein) section of the software distribution
223+documentation.
224+
225+Now our agent should be running, so let's try out `dbug`. In the dojo, let's try
226+poking our agent with the `+dbug` generator:
227+
228+```
229+> ~
230+> :skeleton +dbug
231+>=
232+```
233+
234+It just printed out `~`. Our dummy `skeleton` agent doesn't have any state
235+defined, so it's printing out null as a result. Let's try printing the `bowl`
236+instead:
237+
238+```
239+> [ [our=~zod src=~zod dap=%skeleton]
240+ [wex={} sup={}]
241+ act=5
242+ eny
243+ 0v209.tg795.bc2e8.uja0d.11eq9.qp3b3.mlttd.gmf09.q7ro3.6unfh.16jiu.m9lh9.6jlt8.4f847.f0qfh.up08t.3h4l2.qm39h.r3qdd.k1r11.bja8l
244+ now=~2021.11.5..13.28.24..e20e
245+ byk=[p=~zod q=%base r=[%da p=~2021.11.5..12.02.22..f99b]]
246+]
247+> :skeleton +dbug %bowl
248+>=
249+```
250+
251+We'll use `dbug` more throughout the guide, but hopefully you should now have an
252+idea of its basic usage.
253+
254+## Summary
255+
256+The key takeaways are:
257+
258+- Libraries are imported with `/+`.
259+- `default-agent` is a library that provides default behaviors for Gall agent
260+ arms.
261+- `dbug` is a library that lets you inspect the state and `bowl` of an agent
262+ from the dojo, with the `+dbug` generator.
263+- Convenient deferred expressions for Hoon expressions can be defined in a virtual arm with
264+ the [lustar](/docs/hoon/reference/rune/lus#-lustar) (`+*`) rune.
265+- `this` is a conventional deferred expression name for the agent core itself.
266+- `def` is a conventional deferred expression name for accessing arms in the `default-agent`
267+ library.
268+- Extra cores can be composed into the subject of the agent core. The
269+ composition is done implicitly by the build system. Typically we'd include one
270+ extra core that defines types for our agent's state and maybe other useful
271+ types as well.
272+
273+## Exercises
274+
275+- Run through the [example](#example) yourself on a fake ship if you've not done
276+ so already.
277+- Have a read through the [Ford rune
278+ documentation](/docs/arvo/ford/ford#ford-runes) for details about importing
279+ libraries, structures and other things.
280+- Try the `+dbug` generator out on some other agents, like `:settings-store +dbug`, `:btc-wallet +dbug`, etc, and try some of its options [described
281+ above](#dbug).
282+- Have a quick look over the source of the `default-agent` library, located at
283+ `/lib/default-agent.hoon` in the `%base` desk. We've not yet covered what the
284+ different arms do but it's still useful to get a general idea, and you'll
285+ likely want to refer back to it later.
content/docs/userspace/gall-guide/4-lifecycle.mdnew file+487
@@ -0,0 +1,487 @@
1++++
2+title = "4. Lifecycle"
3+weight = 20
4+template = "doc.html"
5++++
6+
7+In the last lesson we looked at a couple of useful things used as boilerplate in
8+most agents. Now we're going to get into the guts of how agents work, and start
9+looking at what the agent arms do. The first thing we'll look at is the agent's
10+state, and the three arms for managing it: `on-init`, `on-save`, and `on-load`.
11+These arms handle what we call an agent's "lifecycle".
12+
13+## Lifecycle
14+
15+An agent's lifecycle starts when it's first installed. At this point, the
16+agent's `on-init` arm is called. This is the _only_ time `on-init` is ever
17+called - its purpose is just to initialize the agent. The `on-init` arm might be
18+very simple and just set an initial value for the state, or even do nothing at
19+all and return the agent core exactly as-is. It may also be more complicated,
20+and perform some [scries](/docs/arvo/concepts/scry) to obtain extra data or
21+check that another agent is also installed. It might send off some `card`s to
22+other agents or vanes to do things like load data in to the `%settings-store`
23+agent, bind an Eyre endpoint, or anything else. It all depends on the needs of
24+your particular application. If `on-init` fails for whatever reason, the agent
25+installation will fail and be aborted.
26+
27+Once initialized, an agent will just go on doing its thing - processing events,
28+updating its state, producing effects, etc. At some point, you'll likely want to
29+push an update for your agent. Maybe it's a bug fix, maybe you want to add extra
30+features. Whatever the reason, you need to change the source code of your agent,
31+so you commit a modified version of the file to Clay. When the commit completes, Gall updates the app as follows:
32+
33+- The agent's `on-save` arm is called, which packs the agent's state in a `vase`
34+ and exports it.
35+- The new version of the agent is built and loaded into Gall.
36+- The previously exported `vase` is passed to the `on-load` arm of the newly
37+ built agent. The `on-load` arm will process it, convert it to the new version
38+ of the state if necessary, and load it back into the state of the agent.
39+
40+A `vase` is just a cell of `[type-of-the-noun the-noun]`. Most data an agent
41+sends or receives will be encapsulated in a vase. A vase is made with the
42+[zapgar](/docs/hoon/reference/rune/zap#-zapgar) (`!>`) rune like
43+`!>(some-data)`, and unpacked with the
44+[zapgal](/docs/hoon/reference/rune/zap#-zapgal) (`!<`) rune like
45+`!<(type-to-extract vase)`. Have a read through the [`vase` section of the type
46+reference for details](/docs/userspace/gall-guide/types#vase).
47+
48+We'll look at the three arms described here in a little more detail, but first
49+we need to touch on the state itself.
50+
51+## Versioned state type
52+
53+In the previous lesson we introduced the idea of composing additional cores into
54+the subject of the agent core. Here we'll look at using such a core to define
55+the type of the agent's state. In principle, we could make it as simple as this:
56+
57+```hoon
58+|%
59++$ my-state-type @ud
60+--
61+```
62+
63+However, when you update your agent as described in the [Lifecycle](#lifecycle)
64+section, you may want to change the type of the state itself. This means
65+`on-load` might find different versions of the state in the `vase` it receives,
66+and it might not be able to distinguish between them.
67+
68+For example, if you were creating an agent for a To-Do task management app, your
69+tasks might initially have a `?(%todo %done)` union to specify whether they're
70+complete or not. Something like:
71+
72+```hoon
73+(map task=@t status=?(%todo %done))
74+```
75+
76+At some point, you might want to add a third status to represent "in progress",
77+which might involve changing `status` like:
78+
79+```hoon
80+(map title=@t status=?(%todo %done %work))
81+```
82+
83+The conventional way to keep this managable and reliably differentiate possible
84+state types is to have _versioned states_. The first version of the state would
85+typically be called `state-0`, and its head would be tagged with `%0`. Then,
86+when you change the state's type in an update, you'd add a new structure called
87+`state-1` and tag its head with `%1`. The next would then be `state-2`, and so
88+on.
89+
90+In addition to each of those individual state versions, you'd also define a
91+structure called `versioned-state`, which just contains a union of all the
92+possible states. This way, the vase `on-load` receives can be unpacked to a
93+`versioned-state` type, and then a
94+[wuthep](/docs/hoon/reference/rune/wut#--wuthep) (`?-`) expression can switch on
95+the head (`%0`, `%1`, `%2`, etc) and process each one appropriately.
96+
97+For example, your state definition core might initially look like:
98+
99+```hoon
100+|%
101++$ versioned-state
102+ $% state-0
103+ ==
104++$ state-0 [%0 tasks=(map title=@t status=?(%todo %done))]
105+--
106+```
107+
108+When you later update your agent with a new state version, you'd change it to:
109+
110+```hoon
111+|%
112++$ versioned-state
113+ $% state-0
114+ state-1
115+ ==
116++$ state-0 [%0 tasks=(map title=@t status=?(%todo %done))]
117++$ state-1 [%1 tasks=(map title=@t status=?(%todo %done %work))]
118+--
119+```
120+
121+Another reason for versioning the state type is that there may be cases where
122+the state type doesn't change, but you still want to apply special transition
123+logic for an old state during upgrade. For example, you may need to reprocess
124+the data for a new feature or to fix a bug.
125+
126+## Adding the state
127+
128+Along with a core defining the type of the state, we also need to actually add
129+it to the subject of the core. The conventional way to do this is by adding the following immediately before the agent core itself:
130+
131+```hoon
132+=| state-0
133+=* state -
134+```
135+
136+The first line bunts (produces the default value) of the state type we defined
137+in the previous core, and adds it to the head of the subject _without a face_.
138+The next line uses [tistar](/docs/hoon/reference/rune/tis#-tistar) to give it
139+the name of `state`. You might wonder why we don't just give it a face when we
140+bunt it and skip the tistar part. If we did that, we'd have to refer to `tasks`
141+as `tasks.state`. With tistar, we can just reference `tasks` while also being
142+able to reference the whole `state` when necessary.
143+
144+Note that adding the state like this only happens when the agent is built - from
145+then on the arms of our agent will just modify it.
146+
147+## State management arms
148+
149+We've described the basic lifecycle process and the purpose of each state
150+management arm. Now let's look at each arm in detail:
151+
152+### `on-init`
153+
154+This arm takes no argument, and produces a `(quip card _this)`. It's called
155+exactly once, when the agent is first installed. Its purpose is to initialize
156+the agent.
157+
158+`(quip a b)` is equivalent to `[(list a) b]`, see the [types
159+reference](/docs/userspace/gall-guide/types#quip) for details.
160+
161+A `card` is a message to another agent or vane. We'll discuss `card`s in detail
162+later.
163+
164+`this` is our agent core, which we give the `this` alias in the virtual arm
165+described in the previous lesson. The underscore at the beginning is the
166+irregular syntax for the [buccab](/docs/hoon/reference/rune/buc#_-buccab) (`$_`)
167+rune. Buccab is like an inverted bunt - instead of producing the default value
168+of a type, instead it produces the type of some value. So `_this` means "the
169+type of `this`" - the type of our agent core.
170+
171+Recall that in the last lesson, we said that most arms return a cell of
172+`[effects new-agent-core]`. That's exactly what `(quip card _this)` is.
173+
174+### `on-save`
175+
176+This arm takes no argument, and produces a `vase`. Its purpose is to export the
177+state of an agent - the state is packed into the vase it produces. The main time
178+it's called is when an agent is upgraded. When that happens, the agent's state
179+is exported with `on-save`, the new version of the agent is compiled and loaded,
180+and then the state is imported back into the new version of the agent via the
181+[`on-load`](#on-load) arm.
182+
183+As well as the agent upgrade process, `on-save` is also used when an agent is
184+suspended or an app is uninstalled, so that the state can be restored when it's
185+resumed or reinstalled.
186+
187+The state is packed in a vase with the
188+[zapgar](/docs/hoon/reference/rune/zap#-zapgar) (`!>`) rune, like `!>(state)`.
189+
190+### `on-load`
191+
192+This arm takes a `vase` and produces a `(quip card _this)`. Its purpose is to
193+import a state previously exported with [`on-save`](#on-save). Typically
194+you'd have used a [versioned state](#versioned-state-type) as described above,
195+so this arm would test which state version the imported data has, convert data
196+from an old version to the new version if necessary, and load it into the
197+`state` wing of the subject.
198+
199+The vase would be unpacked with a
200+[zapgal](/docs/hoon/reference/rune/zap#-zapgal) (`!<`) rune, and then typically
201+you'd test its version with a [wuthep](/docs/hoon/reference/rune/wut#--wuthep)
202+(`?-`) expression.
203+
204+## Example
205+
206+Here's a new agent to demonstrate the concepts we've discussed here:
207+
208+```hoon
209+/+ default-agent, dbug
210+|%
211++$ versioned-state
212+ $% state-0
213+ ==
214++$ state-0 [%0 val=@ud]
215++$ card card:agent:gall
216+--
217+%- agent:dbug
218+=| state-0
219+=* state -
220+^- agent:gall
221+|_ =bowl:gall
222++* this .
223+ def ~(. (default-agent this %.n) bowl)
224+::
225+++ on-init
226+ ^- (quip card _this)
227+ `this(val 42)
228+::
229+++ on-save
230+ ^- vase
231+ !>(state)
232+::
233+++ on-load
234+ |= old-state=vase
235+ ^- (quip card _this)
236+ =/ old !<(versioned-state old-state)
237+ ?- -.old
238+ %0 `this(state old)
239+ ==
240+::
241+++ on-poke on-poke:def
242+++ on-watch on-watch:def
243+++ on-leave on-leave:def
244+++ on-peek on-peek:def
245+++ on-agent on-agent:def
246+++ on-arvo on-arvo:def
247+++ on-fail on-fail:def
248+--
249+```
250+
251+Let's break it down and have a look at the new parts we've added. First, the
252+state core:
253+
254+```hoon
255+|%
256++$ versioned-state
257+ $% state-0
258+ ==
259++$ state-0 [%0 val=@ud]
260++$ card card:agent:gall
261+--
262+```
263+
264+In `state-0` we've defined the structure of our state, which is just a `@ud`.
265+We've tagged the head with a `%0` constant representing the version number, so
266+`on-load` can easily test the state version. In `versioned-state` we've created
267+a union and just added our `state-0` type. We've added an extra `card` arm as
268+well, just so we can use `card` as a type, rather than the unweildy
269+`card:agent:gall`.
270+
271+After that core, we have the usual `agent:dbug` call, and then we have this:
272+
273+```hoon
274+=| state-0
275+=* state -
276+```
277+
278+We've just bunted the `state-0` type, which will produce `[%0 val=0]`, pinning
279+it to the head of the subject. Then, we've use
280+[tistar](/docs/hoon/reference/rune/tis#-tistar) (`=*`) to give it a name of
281+`state`.
282+
283+Inside our agent core, we have `on-init`:
284+
285+```hoon
286+++ on-init
287+ ^- (quip card _this)
288+ `this(val 42)
289+```
290+
291+The `a(b c)` syntax is the irregular form of the
292+[centis](/docs/hoon/reference/rune/cen#-centis) (`%=`) rune. You'll likely be
293+familiar with this from recursive functions, where you'll typically call the buc
294+arm of a trap like `$(a b, c d, ...)`. It's the same concept here - we're saying
295+`this` (our agent core) with `val` replaced by `42`. Since `on-init` is only
296+called when the agent is first installed, we're just initializing the state.
297+
298+Next we have `on-save`:
299+
300+```hoon
301+++ on-save
302+ ^- vase
303+ !>(state)
304+```
305+
306+This exports our agent's state, and is called during upgrades, suspensions, etc.
307+We're having it pack the `state` value in a `vase`.
308+
309+Finally, we have `on-load`:
310+
311+```hoon
312+++ on-load
313+ |= old-state=vase
314+ ^- (quip card _this)
315+ =/ old !<(versioned-state old-state)
316+ ?- -.old
317+ %0 `this(state old)
318+ ==
319+```
320+
321+It takes in the old state in a `vase`, then unpacks it to the `versioned-state`
322+type we defined earlier. We test its head for the version, and load it back into
323+the state of our agent if it matches. This test is a bit redundant at this stage
324+since we only have one state version, but you'll soon see the purpose of it.
325+
326+You can save it as `/app/lifecycle.hoon` in the `%base` desk and `|commit %base`. Then, run `|rein %base [& %lifecycle]` to start it.
327+
328+Let's try inspecting our state with `dbug`:
329+
330+```
331+> [%0 val=42]
332+> :lifecycle +dbug
333+>=
334+```
335+
336+`dbug` can also dig into the state with the `%state` argument, printing the value of the specified face:
337+
338+```
339+> 42
340+> :lifecycle +dbug [%state %val]
341+>=
342+```
343+
344+Next, we're going to modify our agent and change the structure of the state so
345+we can test out the upgrade process. Here's a modified version, which you can
346+again save in `/app/lifecycle.hoon` and `|commit %base`:
347+
348+```hoon
349+/+ default-agent, dbug
350+|%
351++$ versioned-state
352+ $% state-0
353+ state-1
354+ ==
355++$ state-0 [%0 val=@ud]
356++$ state-1 [%1 val=[@ud @ud]]
357++$ card card:agent:gall
358+--
359+%- agent:dbug
360+=| state-1
361+=* state -
362+^- agent:gall
363+|_ =bowl:gall
364++* this .
365+ def ~(. (default-agent this %.n) bowl)
366+::
367+++ on-init
368+ ^- (quip card _this)
369+ `this(val [27 32])
370+::
371+++ on-save
372+ ^- vase
373+ !>(state)
374+::
375+++ on-load
376+ |= old-state=vase
377+ ^- (quip card _this)
378+ =/ old !<(versioned-state old-state)
379+ ?- -.old
380+ %1 `this(state old)
381+ %0 `this(state 1+[val.old val.old])
382+ ==
383+::
384+++ on-poke on-poke:def
385+++ on-watch on-watch:def
386+++ on-leave on-leave:def
387+++ on-peek on-peek:def
388+++ on-agent on-agent:def
389+++ on-arvo on-arvo:def
390+++ on-fail on-fail:def
391+--
392+```
393+
394+As soon as you `|commit` it, Gall will immediately export the existing state
395+with `on-save`, build the new version of the agent, then import the state back
396+in with `on-load`.
397+
398+In the state definition core, you'll see we've added a new state version with a different structure:
399+
400+```hoon
401++$ versioned-state
402+ $% state-0
403+ state-1
404+ ==
405++$ state-0 [%0 val=@ud]
406++$ state-1 [%1 val=[@ud @ud]]
407++$ card card:agent:gall
408+--
409+```
410+
411+We've also changed the part that adds the state, so it uses the new version instead:
412+
413+```hoon
414+=| state-1
415+=* state -
416+```
417+
418+In `on-init`, we've updated it to initialize the state with a value that fits the new type we've defined:
419+
420+```hoon
421+++ on-init
422+ ^- (quip card _this)
423+ `this(val [27 32])
424+```
425+
426+`on-init` won't be called in this case, but if someone were to directly install
427+this new version of the agent, it would be, so we still need to update it.
428+
429+`on-save` has been left unchanged, but `on-load` has been updated like so:
430+
431+```hoon
432+++ on-load
433+ |= old-state=vase
434+ ^- (quip card _this)
435+ =/ old !<(versioned-state old-state)
436+ ?- -.old
437+ %1 `this(state old)
438+ %0 `this(state 1+[val.old val.old])
439+ ==
440+```
441+
442+We've updated the `?-` expression with a new case that handles our new state
443+type, and for the old state type we've added a function that converts it to the
444+new type - in this case by duplicating `val` and changing the head-tag from `%0`
445+to `%1`. This is an extremely simple state type transition function - it would
446+likely be more complicated for an agent with real functionality.
447+
448+Let's now use `dbug` to confirm our state has successfully been updated to the new
449+type:
450+
451+```
452+> [%1 val=[42 42]]
453+> :lifecycle +dbug
454+>=
455+```
456+
457+## Summary
458+
459+- The app lifecycle rougly consists of initialization, state export, upgrade,
460+ state import and state version transition.
461+- This is managed by three arms: `on-init`, `on-save` and `on-load`.
462+- `on-init` initializes the agent and is called when it's first installed.
463+- `on-save` exports the agent's state and is called during upgrade or
464+ when an app is suspended.
465+- `on-load` imports an agent's state and is called during upgrade or
466+ when an app is unsuspended. It also handles converting data from old
467+ state versions to new state versions.
468+- The type of an agent's state is typically defined in a separate core.
469+- The state type is typically versioned, with a new type definition for each
470+ version of the state.
471+- The state is initially added by bunting the state type and then naming it
472+ `state` with the tistar (`=*`) rune, so its contents can be referenced
473+ directly.
474+- A `vase` is a cell of `[type-of-the-noun the-noun]`.
475+- `(quip a b)` is the same as `[(list a) b]`, and is the `[effects new-agent-core]` pair returned by many arms of an agent core.
476+
477+## Exercises
478+
479+- Run through the [example](#example) yourself on a fake ship if you've not done
480+ so already.
481+- Have a look at the [`vase` entry in the type
482+ reference](/docs/userspace/gall-guide/types#vase).
483+- Have a look at the [`quip` entry in the type
484+ reference](/docs/userspace/gall-guide/types#quip).
485+- Try modifying the second version of the agent in the [example](#example)
486+ section, adding a third state version. Include functions in the wuthep
487+ expression in `on-load` to convert old versions to your new state type.
content/docs/userspace/gall-guide/5-cards.mdnew file+244
@@ -0,0 +1,244 @@
1++++
2+title = "5. Cards"
3+weight = 25
4+template = "doc.html"
5++++
6+
7+As we previously discussed, most arms of an agent core produce a cell of
8+`[effects new-agent-core]`, and the type we use for this is typically `(quip card _this)`. We've covered `_this`, but we haven't yet looked at `card` effects
9+in detail. That's what we'll do here. In explaining `card`s we'll touch on some
10+concepts relating to the mechanics of pokes, subscriptions and other things
11+we've not yet covered. Don't worry if you don't understand how it all fits
12+together yet, we just want to give you a basic idea of `card`s so we can then
13+dig into how they work in practice.
14+
15+## `card` type
16+
17+The `card:agent:gall` type (henceforth just `card`) has a slightly complex
18+structure, so we'll walk through it step-by-step.
19+
20+`lull.hoon` defines a `card` like so:
21+
22+```hoon
23++$ card (wind note gift)
24+```
25+
26+A `wind` is defined in `arvo.hoon` as:
27+
28+```hoon
29+++ wind
30+ |$ [a b]
31+ $% [%pass p=wire q=a]
32+ [%slip p=a]
33+ [%give p=b]
34+ ==
35+```
36+
37+Gall will not accept a `%slip`, so we can ignore that. A `card`, then, is one
38+of:
39+
40+```hoon
41+[%pass wire note]
42+[%give gift]
43+```
44+
45+We'll consider each separately.
46+
47+## `%pass`
48+
49+```hoon
50+[%pass path note]
51+```
52+
53+The purpose of a `%pass` card is to send some kind of one-off request, action,
54+task, or what have you, to another agent or vane. A `%pass` card is a request
55+your agent _initiates_. This is in contrast to a [`%give`](#give) card, which is
56+sent in _response_ to another agent or vane.
57+
58+The type of the first field in a `%pass` card is a `wire`. A `wire` is just a
59+list of `@ta`, with a syntax of `/foo/bar/baz`. When you `%pass` something to an
60+agent or vane, the response will come back on the `wire` you specify here. Your
61+agent can then check the `wire` and maybe do different things depending on its
62+content. The [`wire`](/docs/userspace/gall-guide/types#wire) type is covered in
63+the [types reference](/docs/userspace/gall-guide/types). We'll show how `wire`s
64+are practically used later on.
65+
66+The type of the next field is a `note:agent:gall` (henceforth just `note`), which
67+`lull.hoon` defines as:
68+
69+```hoon
70++$ note
71+ $% [%agent [=ship name=term] =task]
72+ [%arvo note-arvo]
73+ [%pyre =tang]
74+ ==
75+```
76+
77+- An `%agent` `note` is a request to another Gall agent, either local or on a
78+ remote ship. The `ship` and `name` fields are just the target ship and agent
79+ name. The `task` is the request itself, we'll discuss it separately
80+ [below](#task).
81+- An `%arvo` `note` is a request to a vane. We'll discuss such requests
82+ [below](#note-arvo).
83+- A `%pyre` `note` is used to abort an event. It's mostly used internally by
84+ `kiln` (a submodule of `%hood`), it's unlikely you'd use it in your own agent. The `tang` contains an
85+ error message.
86+
87+### `task`
88+
89+A `task:agent:gall` (henceforth just `task`) is defined in `lull.hoon` as:
90+
91+```hoon
92++$ task
93+ $% [%watch =path]
94+ [%watch-as =mark =path]
95+ [%leave ~]
96+ [%poke =cage]
97+ [%poke-as =mark =cage]
98+ ==
99+```
100+
101+Note a few of these include a `path` field. The `path` type is exactly the same
102+as a `wire` - a list of `@ta` with a syntax of `/foo/bar/baz`. The reason for
103+the `wire`/`path` distinction is just to indicate their separate purposes. While
104+a `wire` is for _responses_, a `path` is for _requests_. The
105+[`path`](/docs/userspace/gall-guide/types#path) type is also covered in the
106+[types reference](/docs/userspace/gall-guide/types).
107+
108+The kinds of `task`s can be divided into two categories:
109+
110+#### Subscriptions
111+
112+`%watch`, `%watch-as` and `%leave` all pertain to subscriptions.
113+
114+- `%watch`: A request to subscribe to the specified `path`. Once subscribed,
115+ your agent will receive any updates the other agent sends out on that `path`.
116+ You can subscribe more than once to the same `path`, but each subscription
117+ must have a separate `wire` specified at the beginning of the [`%pass`
118+ card](#pass).
119+- `%watch-as`: This is the same as `%watch`, except Gall will convert updates to
120+ the given `mark` before delivering them to your agent.
121+- `%leave`: Unsubscribe. The subscription to cancel is determined by the `wire`
122+ at the beginning of the [`pass` card](#pass) rather than the subscription
123+ `path`, so its argument is just `~`.
124+
125+**Examples**
126+
127+![subscription card examples](https://media.urbit.org/docs/userspace/gall-guide/sub-cards.svg)
128+
129+#### Pokes
130+
131+Pokes are requests, actions, or just some data which you send to another agent.
132+Unlike subscriptions, these are just one-off messages.
133+
134+A `%poke` contains a `cage` of some data. A `cage` is a cell of `[mark vase]`.
135+The `mark` is just a `@tas` like `%foo`, and corresponds to a mark file in the
136+`/mar` directory. We'll cover `mark`s in greater detail later. The `vase` contains
137+the actual data you're sending.
138+
139+The `%poke-as` task is the same as `%poke` except Gall will convert the `mark`
140+in the `cage` to the `mark` you specify before sending it off.
141+
142+**Examples**
143+
144+![poke card examples](https://media.urbit.org/docs/userspace/gall-guide/poke-cards.svg)
145+
146+### `note-arvo`
147+
148+A `note-arvo` is defined in `lull.hoon` like so:
149+
150+```hoon
151++$ note-arvo
152+ $~ [%b %wake ~]
153+ $% [%a task:ames]
154+ [%b task:behn]
155+ [%c task:clay]
156+ [%d task:dill]
157+ [%e task:eyre]
158+ [%g task:gall]
159+ [%i task:iris]
160+ [%j task:jael]
161+ [%$ %whiz ~]
162+ [@tas %meta vase]
163+ ==
164+```
165+
166+The letter at the beginning corresponds to the vane - `%b` for Behn, `%c` for
167+Clay, etc. After then vane letter comes the task. Each vane has an API with a
168+set of tasks that it will accept, and are defined in each vane's section of
169+`lull.hoon`. Each vane's tasks are documented on the API Reference page of its
170+section in the [Arvo documentation](/docs/arvo/arvo).
171+
172+#### Examples
173+
174+![arvo card examples](https://media.urbit.org/docs/userspace/gall-guide/arvo-cards.svg)
175+
176+## `%give`
177+
178+```hoon
179+[%give gift]
180+```
181+
182+The purpose of a `%give` card is to respond to a request made by another agent
183+or vane. More specifically, it's either for acknowledging a request, or for
184+sending out updates to subscribers. This is in contrast to a [`%pass`](#give)
185+card, which is essentially unsolicited.
186+
187+A `%give` card contains a `gift:agent:gall` (henceforth just `gift`), which is
188+defined in `lull.hoon` as:
189+
190+```hoon
191++$ gift
192+ $% [%fact paths=(list path) =cage]
193+ [%kick paths=(list path) ship=(unit ship)]
194+ [%watch-ack p=(unit tang)]
195+ [%poke-ack p=(unit tang)]
196+ ==
197+```
198+
199+These can be divided into two categories:
200+
201+### Acknowledgements
202+
203+`%watch-ack` is sent in response to a `%watch` or `%watch-as` request, and
204+`%poke-ack` is sent in response to a `%poke` or `%poke-as` request. If the
205+`(unit tang)` is null, it's an ack - a positive acknowledgement. If the `(unit tang)` is non-null, it's a nack - a negative acknowledgement, and the `tang`
206+contains an error message. Gall automatically sends a nack with a stack trace if
207+your agent crashes while processing the request, and automatically sends an ack
208+if it does not. Therefore, you would not explicitly produce a `%watch-ack` or
209+`%poke-ack` gift.
210+
211+#### Examples
212+
213+![ack card examples](https://media.urbit.org/docs/userspace/gall-guide/ack-cards.svg)
214+
215+### Subscriptions
216+
217+`%fact` and `%kick` are both sent out to existing subscribers - entities that
218+have previously `%watch`ed a path on your ship.
219+
220+A `%kick` gift takes a list of subscription `path`s. and a `(unit ship)`, which
221+is the ship to kick from those paths. If the `unit` is null, all subscribers are
222+kicked from the specified paths. Note that sometimes Gall can produce `%kick`
223+gifts without your agent explicitly sending a card, due to networking
224+conditions.
225+
226+`%fact`s are how updates are sent out to subscribers. The `paths` field is a
227+list of subscription paths - all subscribers of the specified `path`s will
228+receive the `%fact`. The `cage` is the data itself - a cell of a `mark` and a
229+`vase`.
230+
231+#### Examples
232+
233+![gift card examples](https://media.urbit.org/docs/userspace/gall-guide/gift-cards.svg)
234+
235+## Summary
236+
237+Here's a diagram that summarizes the different kinds of `card`s:
238+
239+[![card diagram](https://media.urbit.org/docs/userspace/gall-guide/card-diagram.svg)](https://media.urbit.org/docs/userspace/gall-guide/card-diagram.svg)
240+
241+## Exercises
242+
243+- Have a read of the [`wire`](/docs/userspace/gall-guide/types#wire) and
244+ [`path`](/docs/userspace/gall-guide/types#path) entries in the type reference.
content/docs/userspace/gall-guide/6-pokes.mdnew file+538
@@ -0,0 +1,538 @@
1++++
2+title = "6. Pokes"
3+weight = 30
4+template = "doc.html"
5++++
6+
7+In this lesson we'll look at sending and receiving one-off messages called
8+`%poke`s. We'll look at the `on-poke` agent arm which handles incoming pokes.
9+We'll also introduce the `on-agent` arm, and look at the one kind of response it can
10+take - a `%poke-ack`.
11+
12+## Receiving a poke
13+
14+Whenever something tries to poke your agent, Gall calls your agent's `on-poke`
15+arm and give it the `cage` from the poke as its sample. The `on-poke` arm will
16+produce a `(quip card _this)`. Here's how it would typically begin:
17+
18+```hoon
19+++ on-poke
20+ |= [=mark =vase]
21+ ^- (quip card _this)
22+ ...
23+```
24+
25+The sample of the gate is usually specified as a cell of `mark` and `vase`
26+rather than just `cage`, simply because it's easier to work with.
27+
28+Typically, you'd first test the `mark` with something like a
29+[wutlus](/docs/hoon/reference/rune/wut#-wutlus) `?+` expression, passing
30+unexpected `mark`s to `default-agent`, which just crashes. We'll look at custom
31+`mark`s in a subsequent lesson, but the basic patten looks like:
32+
33+```hoon
34+?+ mark (on-poke:def mark vase)
35+ %noun ...
36+ %something-else ...
37+ ...
38+==
39+```
40+
41+After testing the `mark`, you'd usually extract the `vase` to the expected type,
42+and then apply whatever logic you need. For example:
43+
44+```hoon
45+=/ action !<(some-type vase)
46+?- -.action
47+ %foo ...
48+ %bar ...
49+ ...
50+==
51+```
52+
53+Your agent will then produce a list of `card`s to be sent off and a new,
54+modified state, as appropriate. We'll go into subscriptions in the next lesson,
55+but just to give you an idea of a typical pattern: An agent for a chat app might
56+take new messages as pokes, add them to the list of messages in its state, and
57+send out the new messages to subscribed chat participants as `gift`s.
58+
59+As discussed in the previous lesson, Gall will automatically send a `%poke-ack`
60+`gift` back to wherever the poke came from. The `%poke-ack` will be a nack if
61+your agent crashed while processing the poke, and an ack otherwise. If it's a
62+nack, the `tang` in the `%poke-ack` will contain a stack trace of the crash.
63+
64+As a result, you do not need to explicitly send a `%poke-ack`. Instead, you
65+would design your agent to handle only what you expect and crash in all other
66+cases. You can crash by passing the `cage` to `default-agent`, or just with a
67+`!!`. In the latter case, if you want to add an error message to the stack
68+trace, you can do so like:
69+
70+```hoon
71+~| "some error message"
72+!!
73+```
74+
75+This will produce a trace that looks something like:
76+
77+```
78+/sys/vane/gall/hoon:<[1.372 9].[1.372 37]>
79+/app/pokeme/hoon:<[31 3].[43 5]>
80+/app/pokeme/hoon:<[32 3].[43 5]>
81+/app/pokeme/hoon:<[34 5].[42 7]>
82+/app/pokeme/hoon:<[35 5].[42 7]>
83+/app/pokeme/hoon:<[38 7].[41 27]>
84+/app/pokeme/hoon:<[39 9].[40 11]>
85+"some error message"
86+/app/pokeme/hoon:<[40 9].[40 11]>
87+```
88+
89+Note that the `tang` in the nack is just for debugging purposes, you should not
90+try to pass actual data by encoding it in the nack `tang`.
91+
92+## Sending a poke
93+
94+An agent can send pokes to other agents by producing [`%poke`
95+`card`s](/docs/userspace/gall-guide/5-cards#pokes). Any agent arm apart from
96+`on-peek` and `on-save` can produce such `card`s. The arms would typically
97+produce the `(quip card _this)` like so:
98+
99+```hoon
100+:_ this
101+:~ [%pass /some/wire %agent [~target-ship %target-agent] %poke %some-mark !>('some data')]
102+==
103+```
104+
105+The [colcab](/docs/hoon/reference/rune/col#_-colcab) (`:_`) rune makes an
106+inverted cell, it's just `:-` but with the head and tail swapped. We use colcab
107+to produce the `(quip card _this)` because the list of cards is "heavier"
108+here than the new agent core expression (`this`), so it makes it more
109+readable.
110+
111+### Receiving the `%poke-ack`
112+
113+The pokes will be processed by their targets [as described in the previous
114+section](#receiving-a-poke), and they'll `%give` back a `%poke-ack` on the
115+`wire` you specified (`/some/wire` in the previous example). When Gall gets the
116+`%poke-ack` back, it will call the `on-agent` arm of your agent, with the `wire`
117+it came in on and the `%poke-ack` itself in a `sign:agent:gall`. Your `on-agent`
118+arm would therefore begin like so:
119+
120+```hoon
121+++ on-agent
122+ |= [=wire =sign:agent:gall]
123+ ^- (quip card _this)
124+ ...
125+```
126+
127+A `sign:agent:gall` (henceforth just `sign`) is defined in `lull.hoon` as:
128+
129+```hoon
130++$ sign
131+ $% [%poke-ack p=(unit tang)]
132+ [%watch-ack p=(unit tang)]
133+ [%fact =cage]
134+ [%kick ~]
135+ ==
136+```
137+
138+It's basically the same as a [`gift`](/docs/userspace/gall-guide/5-cards#give),
139+but incoming instead of outgoing.
140+
141+The simplest way to handle a `%poke-ack` by passing it to `default-agent`'s
142+`on-agent` arm, which will just print an error message to the terminal if it's a
143+nack, and otherwise do nothing. Sometimes you'll want your agent to do something
144+different depending on whether the poke failed or succeeded (and therefore
145+whether it's a nack or an ack).
146+
147+As stated in the [Precepts](/docs/development/precepts#specifics): "Route on wire before sign, never sign before wire.". Thus we first test the
148+`wire` so you can tell what the `%poke-ack` was for. You might do something
149+like:
150+
151+```hoon
152+?+ wire (on-agent:def wire sign)
153+ [%some %wire ~] ...
154+ ...
155+==
156+```
157+
158+After that, you'll need to see what kind of `sign` it is:
159+
160+```hoon
161+?+ -.sign (on-agent:def wire sign)
162+ %poke-ack ...
163+ ...
164+```
165+
166+Then, you can tell whether it's an ack or a nack by testing whether the `(unit tang)` in the `%poke-ack` is null:
167+
168+```hoon
169+?~ p.sign
170+ ...(what to do if the poke succeeded)...
171+...(what to do if the poke failed)...
172+```
173+
174+Finally, you can produce the `(quip card _this)`.
175+
176+## Example
177+
178+We're going to look at a couple of agents to demonstrate both sending and
179+receiving pokes. Here's the first, an agent that receives pokes:
180+
181+### `pokeme.hoon`
182+
183+```hoon
184+/+ default-agent, dbug
185+|%
186++$ versioned-state
187+ $% state-0
188+ ==
189++$ state-0 [%0 val=@ud]
190++$ card card:agent:gall
191+--
192+%- agent:dbug
193+=| state-0
194+=* state -
195+^- agent:gall
196+|_ =bowl:gall
197++* this .
198+ def ~(. (default-agent this %.n) bowl)
199+::
200+++ on-init
201+ ^- (quip card _this)
202+ `this
203+::
204+++ on-save
205+ ^- vase
206+ !>(state)
207+::
208+++ on-load
209+ |= old-state=vase
210+ ^- (quip card _this)
211+ =/ old !<(versioned-state old-state)
212+ ?- -.old
213+ %0 `this(state old)
214+ ==
215+::
216+++ on-poke
217+ |= [=mark =vase]
218+ ^- (quip card _this)
219+ ?+ mark (on-poke:def mark vase)
220+ %noun
221+ =/ action !<(?(%inc %dec) vase)
222+ ?- action
223+ %inc `this(val +(val))
224+ ::
225+ %dec
226+ ?: =(0 val)
227+ ~| "Can't decrement - already zero!"
228+ !!
229+ `this(val (dec val))
230+ ==
231+ ==
232+::
233+++ on-watch on-watch:def
234+++ on-leave on-leave:def
235+++ on-peek on-peek:def
236+++ on-agent on-agent:def
237+++ on-arvo on-arvo:def
238+++ on-fail on-fail:def
239+--
240+```
241+
242+This is a very simple agent that just has `val`, a number, in its state. It will
243+take pokes that either increment or decrement `val`. Here's its `on-poke` arm:
244+
245+```hoon
246+++ on-poke
247+ |= [=mark =vase]
248+ ^- (quip card _this)
249+ ?+ mark (on-poke:def mark vase)
250+ %noun
251+ =/ action !<(?(%inc %dec) vase)
252+ ?- action
253+ %inc `this(val +(val))
254+ %dec
255+ ?: =(0 val)
256+ ~| "Can't decrement - already zero!"
257+ !!
258+ `this(val (dec val))
259+ ==
260+ ==
261+```
262+
263+It only expects pokes with a `%noun` mark, and passes all others to
264+`on-poke:def`, which just crashes. For `%noun` pokes, it expects to receive
265+either `%inc` or `%dec` in the `vase`. If it's `%inc`, it produces a new `this`
266+with `val` incremented. If it's `%dec`, it produces `this` with `val`
267+decremented, or crashes if `val` is already zero.
268+
269+Let's try it out. Save the agent above as `/app/pokeme.hoon` in the `%base` desk
270+and `|commit %base`. Then, start it up with `|rein %base [& %pokeme]`. We can
271+check its initial state with `dbug`:
272+
273+```
274+> 0
275+> :pokeme +dbug [%state %val]
276+>=
277+```
278+
279+Next, we'll try poking it. The dojo lets you poke agents with the following syntax:
280+
281+```
282+:agent-name &some-mark ['some' 'noun']
283+```
284+
285+If the `mark` part is omitted, it'll just default to `%noun`. Since our agent
286+only takes a `%noun` mark, we can skip that. The rest will be packed in a vase
287+by the dojo and delivered as a poke, so we can do:
288+
289+```
290+> :pokeme %inc
291+>=
292+```
293+
294+If we now look at the state with `dbug`, we'll see the poke was successful and
295+it's been incremented:
296+
297+```
298+> 1
299+> :pokeme +dbug [%state %val]
300+>=
301+```
302+
303+Let's try decrement:
304+
305+```
306+> :pokeme %dec
307+>=
308+> 0
309+> :pokeme +dbug [%state %val]
310+>=
311+```
312+
313+As you can see, it's back at zero. If we try again, we'll see it fails, and the
314+dojo will print the `tang` in the `%poke-ack` nack:
315+
316+```
317+> :pokeme %dec
318+/sys/vane/gall/hoon:<[1.372 9].[1.372 37]>
319+/app/pokeme/hoon:<[31 3].[43 5]>
320+/app/pokeme/hoon:<[32 3].[43 5]>
321+/app/pokeme/hoon:<[34 5].[42 7]>
322+/app/pokeme/hoon:<[35 5].[42 7]>
323+/app/pokeme/hoon:<[38 7].[41 27]>
324+/app/pokeme/hoon:<[39 9].[40 11]>
325+"Can't decrement - already zero!"
326+/app/pokeme/hoon:<[40 9].[40 11]>
327+dojo: app poke failed
328+```
329+
330+### `pokeit.hoon`
331+
332+Here's a second agent. It takes a poke of `%inc` or `%dec` like before, but
333+rather than updating its own state, it sends two pokes to `%pokeme`, so
334+`%pokeme`'s state will be incremented or decremented by two.
335+
336+```hoon
337+/+ default-agent, dbug
338+|%
339++$ versioned-state
340+ $% state-0
341+ ==
342++$ state-0 [%0 ~]
343++$ card card:agent:gall
344+--
345+%- agent:dbug
346+=| state-0
347+=* state -
348+^- agent:gall
349+|_ =bowl:gall
350++* this .
351+ def ~(. (default-agent this %.n) bowl)
352+::
353+++ on-init
354+ ^- (quip card _this)
355+ `this
356+::
357+++ on-save
358+ ^- vase
359+ !>(state)
360+::
361+++ on-load
362+ |= old-state=vase
363+ ^- (quip card _this)
364+ =/ old !<(versioned-state old-state)
365+ ?- -.old
366+ %0 `this(state old)
367+ ==
368+::
369+++ on-poke
370+ |= [=mark =vase]
371+ ^- (quip card _this)
372+ ?+ mark (on-poke:def mark vase)
373+ %noun
374+ =/ action !<(?(%inc %dec) vase)
375+ ?- action
376+ %inc
377+ :_ this
378+ :~ [%pass /inc %agent [our.bowl %pokeme] %poke %noun !>(%inc)]
379+ [%pass /inc %agent [our.bowl %pokeme] %poke %noun !>(%inc)]
380+ ==
381+ ::
382+ %dec
383+ :_ this
384+ :~ [%pass /dec %agent [our.bowl %pokeme] %poke %noun !>(%dec)]
385+ [%pass /dec %agent [our.bowl %pokeme] %poke %noun !>(%dec)]
386+ ==
387+ ==
388+ ==
389+::
390+++ on-watch on-watch:def
391+++ on-leave on-leave:def
392+++ on-peek on-peek:def
393+::
394+++ on-agent
395+ |= [=wire =sign:agent:gall]
396+ ^- (quip card _this)
397+ ?+ wire (on-agent:def wire sign)
398+ [%inc ~]
399+ ?. ?=(%poke-ack -.sign)
400+ (on-agent:def wire sign)
401+ ?~ p.sign
402+ %- (slog '%pokeit: Increment poke succeeded!' ~)
403+ `this
404+ %- (slog '%pokeit: Increment poke failed!' ~)
405+ `this
406+ ::
407+ [%dec ~]
408+ ?. ?=(%poke-ack -.sign)
409+ (on-agent:def wire sign)
410+ ?~ p.sign
411+ %- (slog '%pokeit: Decrement poke succeeded!' ~)
412+ `this
413+ %- (slog '%pokeit: Decrement poke failed!' ~)
414+ `this
415+ ==
416+::
417+++ on-arvo on-arvo:def
418+++ on-fail on-fail:def
419+--
420+```
421+
422+Here's the `on-poke` arm:
423+
424+```hoon
425+++ on-poke
426+ |= [=mark =vase]
427+ ^- (quip card _this)
428+ ?+ mark (on-poke:def mark vase)
429+ %noun
430+ =/ action !<(?(%inc %dec) vase)
431+ ?- action
432+ %inc
433+ :_ this
434+ :~ [%pass /inc %agent [our.bowl %pokeme] %poke %noun !>(%inc)]
435+ [%pass /inc %agent [our.bowl %pokeme] %poke %noun !>(%inc)]
436+ ==
437+ %dec
438+ :_ this
439+ :~ [%pass /dec %agent [our.bowl %pokeme] %poke %noun !>(%dec)]
440+ [%pass /dec %agent [our.bowl %pokeme] %poke %noun !>(%dec)]
441+ ==
442+ ==
443+ ==
444+```
445+
446+It's similar to `%pokeme`, except it sends two `%poke` `card`s to `%pokeme` for
447+each case, rather than modifying its own state. The `%inc` pokes specify a
448+`wire` of `/inc`, and the `%dec` pokes specify a `wire` of `/dec`, so we can
449+differentiate the responses. It also has the following `on-agent`:
450+
451+```hoon
452+++ on-agent
453+ |= [=wire =sign:agent:gall]
454+ ^- (quip card _this)
455+ ?+ wire (on-agent wire sign)
456+ [%inc ~]
457+ ?. ?=(%poke-ack -.sign)
458+ (on-agent wire sign)
459+ ?~ p.sign
460+ %- (slog '%pokeit: Increment poke succeeded!' ~)
461+ `this
462+ %- (slog '%pokeit: Increment poke failed!' ~)
463+ `this
464+ ::
465+ [%dec ~]
466+ ?. ?=(%poke-ack -.sign)
467+ (on-agent wire sign)
468+ ?~ p.sign
469+ %- (slog '%pokeit: Decrement poke succeeded!' ~)
470+ `this
471+ %- (slog '%pokeit: Decrement poke failed!' ~)
472+ `this
473+ ==
474+```
475+
476+`on-agent` tests the `wire`, checks if it's a `%poke-ack`, and then prints to
477+the terminal whether it succeeded or failed.
478+
479+Save this agent to `/app/pokeit.hoon` on the `%base` desk, `|commit %base`, and
480+start it with `|rein %base [& %pokeme] [& %pokeit]`.
481+
482+Let's try it out:
483+
484+```
485+%pokeit: Increment poke succeeded!
486+%pokeit: Increment poke succeeded!
487+> :pokeit %inc
488+>=
489+```
490+
491+`%pokeit` has received positive `%poke-ack`s, which means both pokes succeeded.
492+It could tell they were increments because the `%poke-ack`s came back on the
493+`/inc` wire we specified. We can check the state of `%pokeme` to confirm:
494+
495+```
496+> 2
497+> :pokeme +dbug [%state %val]
498+>=
499+```
500+
501+Let's try decrementing `%pokeme` so it's an odd number, and then try a `%dec`
502+via `%pokeit`:
503+
504+```
505+> :pokeme %dec
506+>=
507+%pokeit: Decrement poke succeeded!
508+%pokeit: Decrement poke failed!
509+> :pokeit %dec
510+>=
511+```
512+
513+The `on-agent` arm of `%pokeit` has received one ack and one nack. The first
514+took `val` to zero, and the second crashed trying to decrement below zero.
515+
516+## Summary
517+
518+- Incoming pokes go to the `on-poke` arm of an agent.
519+- The `on-poke` arm takes a `cage` and produces an `(quip card _this)`.
520+- Gall will automatically return a `%poke-ack` to the poke's source, with a
521+ stack trace in the `(unit tang)` if your agent crashed while processing the
522+ poke.
523+- Outgoing pokes can be sent by including `%poke` `%pass` `card`s in the `quip`
524+ produced by most agent arms.
525+- `%poke-ack`s in response to pokes you've sent will come in to the `on-agent`
526+ arm in a `sign`, on the `wire` you specified in the original `%poke` `card`.
527+- You can poke agents from the dojo with a syntax of `:agent &mark ['some' 'noun']`.
528+
529+## Exercises
530+
531+- Run through the [example](#example) yourself on a fake ship if you've not done
532+ so already.
533+- Have a look at the `on-agent` arm of `/lib/default-agent.hoon` to see how
534+ `default-agent` handles incoming `sign`s.
535+- Try modifying the `%pokeme` agent with another action of your choice (in
536+ addition to `%inc` and `%dec`).
537+- Try modifying the `%pokeit` agent to send your new type of poke to `%pokeme`,
538+ and handle the `%poke-ack` it gets back.
content/docs/userspace/gall-guide/7-sur-and-marks.mdnew file+408
@@ -0,0 +1,408 @@
1++++
2+title = "7. Structures and Marks"
3+weight = 35
4+template = "doc.html"
5++++
6+
7+Before we get into subscription mechanics, there's three things we need to touch
8+on that are very commonly used in Gall agents. The first is defining an agent's
9+types in a `/sur` structure file, the second is `mark` files, and the third is
10+permissions. Note the example code presented in this lesson will not yet build a
11+fully functioning Gall agent, we'll get to that in the next lesson.
12+
13+## `/sur`
14+
15+In the [previous lesson on pokes](/docs/userspace/gall-guide/6-pokes), we used a
16+very simple union in the `vase` for incoming pokes:
17+
18+```hoon
19+=/ action !<(?(%inc %dec) vase)
20+```
21+
22+A real Gall agent is likely to have a more complicated API. The most common
23+approach is to define a head-tagged union of all possible poke types the agent
24+will accept, and another for all possible updates it might send out to
25+subscribers. Rather than defining these types in the agent itself, you would
26+typically define them in a separate core saved in the `/sur` directory of the
27+desk. The `/sur` directory is the canonical location for userspace type
28+definitions.
29+
30+With this approach, your agent can simply import the structures file and make use
31+of its types. Additionally, if someone else wants to write an agent that
32+interfaces with yours, they can include your structure file in their own desk
33+to interact with your agent's API in a type-safe way.
34+
35+#### Example
36+
37+Let's look at a practical example. If we were creating a simple To-Do app, our
38+agent might accept a few possible `action`s as pokes: Adding a new task,
39+deleting a task, toggling a task's "done" status, and renaming an existing task.
40+It might also be able to send `update`s out to subscribers when these events
41+occur. If our agent were named `%todo`, it might have the following structure in
42+`/sur/todo.hoon`:
43+
44+```hoon
45+|%
46++$ id @
47++$ name @t
48++$ task [=name done=?]
49++$ tasks (map id task)
50++$ action
51+ $% [%add =name]
52+ [%del =id]
53+ [%toggle =id]
54+ [%rename =id =name]
55+ ==
56++$ update
57+ $% [%add =id =name]
58+ [%del =id]
59+ [%toggle =id]
60+ [%rename =id =name]
61+ [%initial =tasks]
62+ ==
63+--
64+```
65+
66+Our `%todo` agent could then import this structure file with a [fashep ford
67+rune](/docs/arvo/ford/ford#ford-runes) (`/-`) at the beginning of the agent like
68+so:
69+
70+```hoon
71+/- todo
72+```
73+
74+The agent's state could be defined like:
75+
76+```hoon
77+|%
78++$ versioned-state
79+ $% state-0
80+ ==
81++$ state-0 [%0 =tasks:todo]
82++$ card card:agent:gall
83+--
84+```
85+
86+Then, in its `on-poke` arm, it could handle these actions in the following
87+manner:
88+
89+```hoon
90+++ on-poke
91+ |= [=mark =vase]
92+ ^- (quip card _this)
93+ |^
94+ ?> =(src.bowl our.bowl)
95+ ?+ mark (on-poke:def mark vase)
96+ %todo-action
97+ =^ cards state
98+ (handle-poke !<(action:todo vase))
99+ [cards this]
100+ ==
101+ ::
102+ ++ handle-poke
103+ |= =action:todo
104+ ^- (quip card _state)
105+ ?- -.action
106+ %add
107+ :_ state(tasks (~(put by tasks) now.bowl [name.action %.n]))
108+ :~ :* %give %fact ~[/updates] %todo-update
109+ !>(`update:todo`[%add now.bowl name.action])
110+ ==
111+ ==
112+ ::
113+ %del
114+ :_ state(tasks (~(del by tasks) id.action))
115+ :~ :* %give %fact ~[/updates] %todo-update
116+ !>(`update:todo`action)
117+ ==
118+ ==
119+ ::
120+ %toggle
121+ :_ %= state
122+ tasks %+ ~(jab by tasks)
123+ id.action
124+ |=(=task:todo task(done !done.task))
125+ ==
126+ :~ :* %give %fact ~[/updates] %todo-update
127+ !>(`update:todo`action)
128+ ==
129+ ==
130+ ::
131+ %rename
132+ :_ %= state
133+ tasks %+ ~(jab by tasks)
134+ id.action
135+ |=(=task:todo task(name name.action))
136+ ==
137+ :~ :* %give %fact ~[/updates] %todo-update
138+ !>(`update:todo`action)
139+ ==
140+ ==
141+ ::
142+ %allow
143+ `state(friends (~(put in friends) who.action))
144+ ::
145+ %kick
146+ :_ state(friends (~(del in friends) who.action))
147+ :~ [%give %kick ~[/updates] `who.action]
148+ ==
149+ ==
150+ --
151+```
152+
153+Let's break this down a bit. Firstly, our `on-poke` arm includes a
154+[barket](/docs/hoon/reference/rune/bar#-barket) (`|^`) rune. Barket creates a
155+core with a `$` arm that's computed immediately. We extract the `vase` to the
156+`action:todo` type and immediately pass it to the `handle-poke` arm of the core
157+created with the barket. This `handle-poke` arm tests what kind of `action` it's
158+received by checking its head. It then updates the state, and also sends an
159+update to subscribers, as appropriate. Don't worry too much about the `%give`
160+`card` for now - we'll cover subscriptions in the next lesson.
161+
162+Notice that the `handle-poke` arm produces a `(quip card _state)` rather than
163+`(quip card _this)`. The call to `handle-poke` is also part of the following
164+expression:
165+
166+```hoon
167+=^ cards state
168+ (handle-poke !<(action:todo vase))
169+[cards this]
170+```
171+
172+The [tisket](/docs/hoon/reference/rune/tis#-tisket) (`=^`) expression takes two
173+arguments: A new named noun to pin to the subject (`cards` in this case), and an
174+existing wing of the subject to modify (`state` in this case). Since
175+`handle-poke` produces `(quip card _state)`, we're saving the `card`s it
176+produces to `cards` and replacing the existing `state` with its new one.
177+Finally, we produce `[cards this]`, where `this` will now contain the modified
178+`state`. The `[cards this]` is a `(quip card _this)`, which our `on-poke` arm is
179+expected to produce.
180+
181+This might seem a little convoluted, but it's a common pattern we do for two
182+reasons. Firstly, it's not ideal to be passing around the entire `this` agent
183+core - it's much tidier just passing around the `state`, until you actually want
184+to return it to Gall. Secondly, It's much easier to read when the poke handling
185+logic is separated into its own arm. This is a fairly simple example but if your
186+agent is more complex, handling multiple marks and containing additional logic
187+before it gets to the actual contents of the `vase`, structuring things this way
188+can be useful.
189+
190+You can of course structure your `on-poke` arm differently than we've done
191+here - we're just demonstrating a typical pattern.
192+
193+## `mark` files
194+
195+So far we've just used a `%noun` mark for pokes - we haven't really delved into
196+what such `mark`s represent, or considered writing custom ones.
197+
198+Formally, marks are file types in the Clay filesystem. They correspond to mark
199+files in the `/mar` directory of a desk. The `%noun` mark, for example,
200+corresponds to the `/mar/noun.hoon` file. Mark files define the actual hoon data
201+type for the file (e.g. a `*` noun for the `%noun` mark), but they also specify
202+some extra things:
203+
204+- Methods for converting between the mark in question and other marks.
205+- Revision control functions like patching, diffing, merging, etc.
206+
207+Aside from their use by Clay for storing files in the filesystem, they're also
208+used extensively for exchanging data with the outside world, and for exchanging
209+data between Gall agents. When data comes in from a remote ship, destined for a
210+particular Gall agent, it will be validated by the file in `/mar` that
211+corresponds to its mark before being delivered to the agent. If the remote data
212+has no corresponding mark file in `/mar` or it fails validation, it will crash
213+before it touches the agent.
214+
215+A mark file is a door with exactly three arms. The door's sample is the data type the
216+mark will handle. For example, the sample of the `%noun` mark is just `non=*`,
217+since it handles any noun. The three arms are as follows:
218+
219+- `grab`: Methods for converting _to_ our mark _from_ other marks.
220+- `grow`: Methods for converting _from_ our mark _to_ other marks.
221+- `grad`: Revision control functions.
222+
223+In the context of Gall agents, you'll likely just use marks for sending and
224+receiving data, and not for actually storing files in Clay. Therefore, it's
225+unlikely you'll need to write custom revision control functions in the `grad`
226+arm. Instead, you can simply delegate `grad` functions to another mark -
227+typically `%noun`. If you want to learn more about writing such `grad`
228+functions, you can refer to the [Marks Guide](/docs/arvo/clay/marks/marks) in
229+the Clay vane documentation, which is much more comprehensive, but it's not
230+necessary for our purposes here.
231+
232+#### Example
233+
234+Here's a very simple mark file for the `action` structure we created in the
235+[previous section](#sur):
236+
237+```hoon
238+/- todo
239+|_ =action:todo
240+++ grab
241+ |%
242+ ++ noun action:todo
243+ --
244+++ grow
245+ |%
246+ ++ noun action
247+ --
248+++ grad %noun
249+--
250+```
251+
252+We've imported the `/sur/todo.hoon` structure library from the previous section,
253+and we've defined the sample of the door as `=action:todo`, since that's what
254+it will handle. Now let's consider the arms:
255+
256+- `grab`: This handles conversion methods _to_ our mark. It contains a core with
257+ arm names corresponding to other marks. In this case, it can only convert from
258+ a `noun` mark, so that's the core's only arm. The `noun` arm simply calls the
259+ `action` structure from our structure library. This is called "clamming" or
260+ "molding" - when some noun comes in, it gets called like `(action:todo [some-noun])` - producing data of the `action` type if it nests, and crashing
261+ otherwise.
262+- `grow`: This handles conversion methods _from_ our mark. Like `grab`, it
263+ contains a core with arm names corresponding to other marks. Here we've also
264+ only added an arm for a `%noun` mark. In this case, `action` data will come in
265+ as the sample of our door, and the `noun` arm simply returns it, since it's
266+ already a noun (as everything is in Hoon).
267+- `grad`: This is the revision control arm, and as you can see we've simply
268+ delegated it to the `%noun` mark.
269+
270+This mark file could be saved as `/mar/todo/action.hoon`, and then the `on-poke`
271+arm in the previous example could test for it instead of `%noun` like so:
272+
273+```hoon
274+++ on-poke
275+ |= [=mark =vase]
276+ |^ ^- (quip card _this)
277+ ?+ mark (on-poke:def mark vase)
278+ %todo-action
279+ ...
280+```
281+
282+Note how `%todo-action` will be resolved to `/mar/todo/action.hoon` - the hyphen
283+will be interpreted as `/` if there's not already a `/mar/todo-action.hoon`.
284+
285+This simple mark file isn't all that useful. Typically, you'd add `json` arms to
286+`grow` and `grab`, which allow your data to be converted to and from JSON, and
287+therefore allow your agent to communicate with a web front-end. Front-ends,
288+JSON, and Eyre's APIs which facilitate such communications will be covered in a
289+forthcoming part two of the Gall guide. For now though, it's still useful to use
290+marks and understand how they work.
291+
292+One further note on marks - while data from remote ships must have a matching
293+mark file in `/mar`, it's possible to exchange data between local agents with
294+"fake" marks - ones that don't exist in `/mar`. Your `on-poke` arm could, for
295+example, use a made-up mark like `%foobar` for actions initiated locally. This
296+is because marks come into play only at validation boundries, none of which are
297+crossed when doing local agent-to-agent communications.
298+
299+## Permissions
300+
301+In example agents so far, we haven't bothered to check where events such as
302+pokes are actually coming from - our example agents would accept data from
303+anywhere, including random foreign ships. We'll now have a look at how to handle
304+such permission checks.
305+
306+Back in [lesson 2](/docs/userspace/gall-guide/2-agent#bowl) we discussed the
307+[bowl](/docs/arvo/gall/data-types#bowl). The `bowl` includes a couple of useful
308+fields: `our` and `src`. The `our` field just contains the `@p` of the local
309+ship. The `src` field contains the `@p` of the ship from which the event
310+originated, and is updated for every new event.
311+
312+When messages come in over Ames from other ships on the network, they're
313+[encrypted](/docs/arvo/ames/cryptography) with our ship's public keys and signed by the ship which sent them.
314+The Ames vane decrypts and verifies the messages using keys in the Jael vane,
315+which are obtained from the [Azimuth Ethereum contract](/docs/azimuth/azimuth-eth) and [Layer 2 data](/docs/azimuth/l2/layer2) where Urbit ID ownership
316+and keys are recorded. This means the originating `@p` of all messages are
317+cryptographically validated before being passed on to Gall, so the `@p`
318+specified in the `src` field of the `bowl` can be trusted to be correct, which
319+makes checking permissions very simple.
320+
321+You're free to use whatever logic you want for this, but the most common way is
322+to use [wutgar](/docs/hoon/reference/rune/wut#-wutgar) (`?>`) and
323+[wutgal](/docs/hoon/reference/rune/wut#-wutgal) (`?<`) runes, which are
324+respectively True and False assertions that crash if they don't evaluate to the
325+expected truth value. To only allow messages from the local ship, you can just
326+do the following in the relevant agent arm:
327+
328+```hoon
329+?> =(src.bowl our.bowl)
330+```
331+
332+A common permission is to allow messages from the local ship, as well as all of
333+its moons, which can be done with the `team:title` standard library function:
334+
335+```hoon
336+?> (team:title our.bowl src.bowl)
337+```
338+
339+If we want to only allow messages from a particular set of ships, we could, for
340+example, have a `(set @p)` in our agent's state called `allowed`. Then, we can
341+use the `has:in` set function to check:
342+
343+```hoon
344+?> (~(has in allowed) src.bowl)
345+```
346+
347+If we wanted to check a ship was allowed in a particular group in the Groups
348+app, we could scry our ship's `%group-store` agent and compare:
349+
350+```hoon
351+?> .^(? %gx /(scot %p our.bowl)/group-store/(scot %da now.bowl)/groups/ship/~bitbet-bolbel/urbit-community/join/(scot %p src.bowl)/noun)
352+```
353+
354+There are many ways to handle permissions, it all depends on your particular use
355+case.
356+
357+## Summary
358+
359+Type definitions:
360+
361+- An agent's type definitions live in the `/sur` directory of a desk.
362+- The `/sur` file is a core, typically containing a number of lusbuc (`+$`)
363+ arms.
364+- `/sur` files are imported with the fashep (`/-`) Ford rune at the beginning
365+ of a file.
366+- Agent API types, for pokes and updates to subscribers, are commonly defined as
367+ head-tagged unions such as `[%foo bar=baz]`.
368+
369+Mark files:
370+
371+- Mark files live in the `/mar` directory of a desk.
372+- A mark like `%foo` corresponds to a file in `/mar` like `/mar/foo.hoon`
373+- Marks are file types in Clay, but are also used for passing data between
374+ agents as well as for external data generally.
375+- A mark file is a door with a sample of the data type it handles and exactly three
376+ arms: `grab`, `grow` and `grad`.
377+- `grab` and `grow` each contain a core with arm names corresponding to other marks.
378+- `grab` and `grow` define functions for converting to and from our mark,
379+ respectively.
380+- `grad` defines revision control functions for Clay, but you'd typically just
381+ delegate such functions to the `%noun` mark.
382+- Incoming data from remote ships will have their marks validated by the
383+ corresponding mark file in `/mar`.
384+- Messages passed between agents on a local ship don't necessarily need mark
385+ files in `/mar`.
386+- Mark files are most commonly used for converting an agent's native types to
387+ JSON, in order to interact with a web front-end (which we'll cover in part two
388+ of the Gall guide).
389+
390+Permissions:
391+
392+- The source of incoming messages from remote ships are cryptographically
393+ validated by Ames and provided to Gall, which then populates the `src` field
394+ of the `bowl` with the `@p`.
395+- Permissions are most commonly enforced with wutgar (`?>`) and wutgal (`?<`)
396+ assertions in the relevant agent arms.
397+- Messages can be restricted to the local ship with `?> =(src.bowl our.bowl)` or to
398+ its moons as well with `?> (team:title our.bowl src.bowl)`.
399+- There are many other ways to handle permissions, it just depends on the needs
400+ of the particular agent.
401+
402+## Exercises
403+
404+- Have a quick look at the [tisket
405+ documentation](/docs/hoon/reference/rune/tis#-tisket).
406+- Try writing a mark file for the `update:todo` type, in a similar fashion to
407+ the `action:todo` one in the [mark file section](#mark-file). You can compare
408+ yours to the one we'll use in the next lesson.
content/docs/userspace/gall-guide/8-subscriptions.mdnew file+911
@@ -0,0 +1,911 @@
1++++
2+title = "8. Subscriptions"
3+weight = 40
4+template = "doc.html"
5++++
6+
7+In this lesson we're going to look at subscriptions. Subscriptions are probably
8+the most complicated part of writing agents, so there's a fair bit to cover.
9+Before we get into the nitty-gritty details, we'll give a brief overview of
10+Gall's subscription mechanics.
11+
12+The basic unit of subscriptions is the _path_. An agent will typically define a
13+number of subscription paths in its `on-watch` arm, and other agents (local or
14+remote) can subscribe to those paths. The agent will then send out updates
15+called `%fact`s on one or more of its paths, and _all_ subscribers of those
16+paths will receive them. An agent cannot send out updates to specific
17+subscribers, it can only target its paths. An agent can kick subscribers from
18+its paths, and subscribers can unsubscribe from any paths.
19+
20+The subscription paths an agent defines can be simple and fixed like
21+`/foo/bar/baz`. They can also be dynamic, containing data of a particular atom
22+aura encoded in certain elements of the path. These paths can therefore be as
23+simple or complex as you need for your particular application.
24+
25+Note it's not strictly necessary to define subscription paths explicitly. As
26+long as the arm doesn't crash, the subscription will succeed. In practice,
27+however, it's nearly always appropriate to define them explicitly and crash on
28+unrecognized paths.
29+
30+For a deeper explanation of subscription mechanics in Arvo, you can refer to
31+Arvo's [Subscriptions](/docs/arvo/concepts/subscriptions) section.
32+
33+## Incoming subscriptions
34+
35+Subscription requests from other entities arrive in your agent's `on-watch` arm.
36+The `on-watch` arm takes the `path` to which they're subscribing, and produces a
37+`(quip card _this)`:
38+
39+```hoon
40+++ on-watch
41+ |= =path
42+ ^- (quip card _this)
43+ ...
44+```
45+
46+Your agent's subscription paths would be defined in this arm, typically in a
47+wutlus (`?+`) expression or similar:
48+
49+```hoon
50+?+ path (on-watch:def path)
51+ [%updates ~]
52+ ......
53+ ......
54+ [%blah %blah ~]
55+ ......
56+ ......
57+ [%foo @ ~]
58+ =/ when=@da (slav %da i.t.path)
59+ ......
60+ ......
61+ [%bar %baz *]
62+ ?+ t.t.path (on-watch:def path)
63+ ~ .....
64+ [%abc %def ~] .....
65+ [%blah ~] .....
66+ ==
67+==
68+```
69+
70+Subscription paths can be simple and fixed like the first two examples above:
71+`/updates` and `/blah/blah`. They can also contain "wildcard" elements, with an
72+atom of a particular aura encoded in an element of the `path`, as in the `[%foo @ ~]` example. The type pattern matcher is quite limited, so we just specify
73+such variable elements as `@`, and then decode them with something like `(slav %da i.t.path)` (for a `@da`), as in the example. The incoming `path` in this
74+example would look like `/foo/~2021.11.14..13.30.39..6b17`. For more information
75+on decoding atoms in strings, see the [Strings
76+Guide](/docs/hoon/guides/strings#decoding-from-text).
77+
78+In the last case of `[%bar baz *]`, we're allowing a variable number of elements
79+in the path. First we check it's `/foo/bar/...something...`, and then we check
80+what the "something" is in another wutlus expression and handle it
81+appropriately. In this case, it could be `/foo/bar`, `/foo/bar/abc/def`, or
82+`/foo/bar/blah`. You could of course also have "wildcard" elements here too, so
83+there's not really a limit to the complexity of your subscription paths, or the
84+data that might be encoded therein.
85+
86+Permissions can be checked as described in the previous lesson, comparing the
87+source `@p` of the request in `src.bowl` to `our.bowl` or any other logic you
88+find appropriate.
89+
90+If a permission check fails, the path is not valid, or any other reason you want
91+to reject the subscription request, your agent can simply crash. The behavior
92+here is the same as with `on-poke` - Gall will send a `%watch-ack` card in
93+response, which is either an ack (positive acknowledgement) or a nack (negative
94+acknowledgement). The `(unit tang)` in the `%watch-ack` will be null if
95+processing succeeded, and non-null if it crashed, with a stack trace in the
96+`tang`. Like with `poke-ack`s, you don't need to explicitly send a
97+`%watch-ack` - Gall will do it automatically.
98+
99+As well as sending a `%watch-ack`, Gall will also record the subscription in the
100+`sup` field of the `bowl`, if it succeeded. Then, when you send updates out to
101+subscribers of the `path` in question, the new subscriber will begin receiving
102+them as well.
103+
104+Updates to subscribers would usually be sent from other arms, but there's one
105+special case for `on-watch` which is very useful. Normally updates can only be
106+sent to all subscribers of a particular path - you can't target a specific
107+subscriber. There's one exception to this: In `on-watch`, when there's a new
108+subscription, you can send a `%fact` back with an empty `(list path)`, and it'll
109+only go to the new subscriber. This is most useful when you want to give the
110+subscriber some initial state, which you otherwise couldn't do without sending
111+it to everyone. It might look something like this:
112+
113+```hoon
114+:_ this
115+:~ [%give %fact ~ %todo-update !>(`update:todo`initial+tasks)]
116+==
117+```
118+
119+## Sending updates to subscribers
120+
121+Once your agent has subscribers, it's easy to send them out updates. All you
122+need to do is produce `card`s with `%fact`s in them:
123+
124+```hoon
125+:_ this
126+:~ [%give %fact ~[/some/path /another/path] %some-mark !>('some data')]
127+ [%give %fact ~[/some/path] %some-mark !>('more data')]
128+ ....
129+==
130+```
131+
132+The `(list path)` in the `%fact` specifies which subscription `path`s the
133+`%fact` should be sent on. All subscribers of all `path`s specified will receive
134+the `%fact`. Any agent arm which produces a `(quip card _this)` can send
135+`%fact`s to subscribers. Most often they will be produced in the `on-poke` arm,
136+since new data will often be added in `poke`s.
137+
138+## Kicking subscribers
139+
140+To kick a subscriber, you just send a `%kick` `card`:
141+
142+```hoon
143+[%give %kick ~[/some/path] `~sampel-palnet]
144+```
145+
146+The `(list path)` specifies which subscription `path`s the ship should be kicked
147+from, and the `(unit ship)` specifies which ship to kick. The `(unit ship)` can also be null, like so:
148+
149+```hoon
150+[%give %kick ~[/some/path] ~]
151+```
152+
153+In this case, all subscribers to the specified `path`s will be kicked.
154+
155+Note that `%kick`s are not exclusively sent by the agent itself - Gall itself
156+can also kick subscribers under certain network conditions. Because of this,
157+`%kick`s are not assumed to be intentional, and the usual behavior is for a
158+kicked agent to try and resubscribe. Therefore, if you want to disallow a
159+particular subscriber, your agent's `on-watch` arm should reject further
160+subscription requests from them - your agent should not just `%kick` them and
161+call it a day.
162+
163+## Outgoing subscriptions
164+
165+Now that we've covered incoming subscriptions, we'll look at the other side of
166+it: Subscribing to other agents. This is done by `%pass`ing the target agent a
167+`%watch` task in a `card`:
168+
169+```hoon
170+[%pass /some/wire %agent [~some-ship %some-agent] %watch /some/path]
171+```
172+
173+If your agent's subscription request is successful, updates will come in to your
174+agent's `on-agent` arm on the `wire` specified (`/some/wire` in this example).
175+The `wire` can be anything you like - its purpose is for your agent to figure
176+out which subscription the updates came from. The `[ship term]` pair specifies the
177+ship and agent you're trying to subscribe to, and the final `path` (`/some/path`
178+in this example) is the path you want to subscribe to - a `path` the target
179+agent has defined in its `on-watch` arm.
180+
181+Gall will deliver the `card` to the target agent and call that agent's
182+`on-watch` arm, which will process the request [as described
183+above](#incoming-subscription-requests), accept or reject it, and send back
184+either a positive or negative `%watch-ack`. The `%watch-ack` will come back in
185+to your agent's `on-agent` arm in a `sign`, along with the `wire` you specified
186+(`/some/wire` in this example). Recall in the lesson on pokes, the `on-agent`
187+arm starts with:
188+
189+```hoon
190+++ on-agent
191+ |= [=wire =sign:agent:gall]
192+ ^- (quip card _this)
193+ .....
194+```
195+
196+The `sign` will be of the following format:
197+
198+```hoon
199+[%watch-ack p=(unit tang)]
200+```
201+
202+How you want to handle the `%watch-ack` really depends on the particular agent.
203+In the simplest case, you can just pass it to the `on-agent` arm of
204+`default-agent`, which will just accept it and do nothing apart from printing
205+the error in the `%watch-ack` `tang` if it's a nack. You shouldn't have your
206+agent crash on a `%watch-ack` - even if it's a nack your agent should process it
207+successfully. If you wanted to apply some additional logic on receipt of the
208+`%watch-ack`, you'd typically first test the `wire`, then test whether it's a
209+`%watch-ack`, then test whether it's an ack or a nack and do whatever's
210+appropriate:
211+
212+```hoon
213+++ on-agent
214+ |= [=wire =sign:agent:gall]
215+ ^- (quip card _this)
216+ ?+ wire (on-watch:def wire sign)
217+ [%expected %wire ~]
218+ ?+ -.sign (on-watch:def wire sign)
219+ %watch-ack
220+ ?~ p.sign
221+ ...(do something if ack)...
222+ ...(do something if nack)...
223+ ......
224+```
225+
226+The `on-agent` arm produces a `(quip card _this)`, so you can produce new
227+`card`s and update your agent's state, as appropriate.
228+
229+One further thing to note with subscriptions is that you can subscribe multiple
230+times to the same `path` on the same ship and agent, as long as the `wire` is
231+unique. If the ship, agent, `path` and `wire` are all the same as an existing
232+subscription, Gall will not allow the request to be sent, and instead fail with
233+an error message fed into the `on-fail` arm of your agent.
234+
235+## Receiving updates
236+
237+Assuming the `%watch` succeeded, your agent will now begin receiving any
238+`%fact`s the other agent publishes on the `path` to which you've subscribed. These
239+`%fact`s will also come in to your agent's `on-agent` arm in a `sign`, just like
240+the initial `%watch-ack`. The `%fact` `sign` will have the following format:
241+
242+```hoon
243+[%fact =cage]
244+```
245+
246+You would typically handle such `%fact`s in the following manner: Test the
247+`wire`, test whether the `sign` is a `%fact`, test the `mark` in the `cage`,
248+extract the data from the `vase` in the `cage`, and apply your logic. Again, routing on `wire` before `sign` is one of the [Precepts](/docs/development/precepts#specifics). For example:
249+
250+```hoon
251+++ on-agent
252+ |= [=wire =sign:agent:gall]
253+ ^- (quip card _this)
254+ ?+ wire (on-watch:def wire sign)
255+ [%expected %wire ~]
256+ ?+ -.sign (on-watch:def wire sign)
257+ %fact
258+ ?+ p.cage.sign (on-watch:def wire sign)
259+ %expected-mark
260+ =/ foo !<(expected-type q.cage.sign)
261+ .....
262+ ......
263+```
264+
265+Note that Gall will not allow `sign`s to come into `on-agent` unsolicited, so
266+you don't necessarily need to include permission logic in this arm.
267+
268+The `on-agent` arm produces a `(quip card _this)`, so you can produce new
269+`card`s and update your agent's state, as appropriate.
270+
271+## Getting kicked
272+
273+For whatever reason, the agent you're `%watch`ing might want to kick your agent
274+from a `path` to which it's suscribed, ending your subscription and ceasing to
275+send your agent `%fact`s. To do this, it will send your agent a `%kick` card [as
276+described above](#kicking-subscribers). The `%kick` will come in to your agent's
277+`on-agent` arm in a `sign`, like `%watch-ack`s and `%fact`s do. The `%kick`
278+`sign` will have the following format:
279+
280+```hoon
281+[%kick ~]
282+```
283+
284+Since the `%kick` itself contains no information, you'll need to consider the
285+`wire` it comes in on to know what it pertains to. As explained previously,
286+`%kick`s aren't always intentional - sometimes Gall will kick subscribers due to
287+network issues. Your `on-agent` arm therefore has no way to know whether the
288+other agent actually intended to kick it. This means _your agent should almost
289+always try to resubscribe if it gets kicked_. Then, if the resubscribe `%watch`
290+request is rejected with a negative `%watch-ack`, you can conclude that it was
291+intentional and give up. The logic would look something like this:
292+
293+```hoon
294+++ on-agent
295+ |= [=wire =sign:agent:gall]
296+ ^- (quip card _this)
297+ ?+ wire (on-agent:def wire sign)
298+ [%some %wire ~]
299+ ?+ -.sign (on-agent:def wire sign)
300+ %kick
301+ :_ this
302+ :~ [%pass /some/wire %agent [src.bowl dap.bowl] %watch /some/path]
303+ ==
304+ .......
305+```
306+
307+## Leaving a subscription
308+
309+Eventually you may wish to unsubscribe from a `path` in another agent and stop
310+receiving updates. This is done by `%pass`ing a `%leave` task to the agent in
311+question:
312+
313+```hoon
314+[%pass /some/wire %agent [~some-ship %some-agent] %leave ~]
315+```
316+
317+The subcription to be ended is determined by the combination of the `wire`, ship
318+and agent, so the `%leave` task itself always just has `~` at the end.
319+
320+## Example
321+
322+Here we're going to give a pretty well fleshed out example. It will demonstrate
323+both inbound and outbound subscriptions, most of the concepts we've discussed
324+here, as well as some from the previous lesson - `/sur` files, `mark` files, and
325+permission checks.
326+
327+In previous lessons we've only dealt with things on a local ship - this example
328+will demonstrate messages being sent over the network.
329+
330+The example will be composed of two separate agents - a publisher called
331+`/app/todo.hoon` and a subscriber called `/app/todo-watcher.hoon`, which will
332+live on separate ships. It will be a very rudimentary To-Do app - to-do tasks
333+will be poked into the publisher and sent out to the subscriber as `%fact`s,
334+which will just print them to the dojo. It will have its types defined in
335+`/sur/todo.hoon`, and it will have a couple of `mark` files for pokes and
336+updates: `/mar/todo/action.hoon` and `/mar/todo/update.hoon`.
337+
338+Before we get into trying it out, we'll first walk through the `/sur` file, mark
339+files, and each agent.
340+
341+### Types and marks
342+
343+#### `/sur/todo.hoon`
344+
345+```hoon
346+|%
347++$ id @
348++$ name @t
349++$ task [=name done=?]
350++$ tasks (map id task)
351++$ who @p
352++$ friends (set who)
353++$ action
354+ $% [%add =name]
355+ [%del =id]
356+ [%toggle =id]
357+ [%rename =id =name]
358+ [%allow =who]
359+ [%kick =who]
360+ ==
361++$ update
362+ $% [%add =id =name]
363+ [%del =id]
364+ [%toggle =id]
365+ [%rename =id =name]
366+ [%initial =tasks]
367+ ==
368+--
369+```
370+
371+This file defines most of the types for the agents. The list of to-do tasks will
372+be stored in the state of the publisher agent as the `tasks` type, a `(map id task)`, where a `task` is a `[=name done=?]`. The set of ships allowed to
373+subscribe will be stored in `friends`, a `(set @p)`, also in the publisher's
374+state. After that, there are the head-tagged unions of accepted poke `action`s
375+and `update`s for subscribers.
376+
377+#### `/mar/todo/action.hoon`
378+
379+```hoon
380+/- todo
381+|_ =action:todo
382+++ grab
383+ |%
384+ ++ noun action:todo
385+ --
386+++ grow
387+ |%
388+ ++ noun action
389+ --
390+++ grad %noun
391+--
392+```
393+
394+This is a very simple mark file for the `action` type.
395+
396+#### `/mar/todo/update.hoon`
397+
398+```hoon
399+/- todo
400+|_ =update:todo
401+++ grab
402+ |%
403+ ++ noun update:todo
404+ --
405+++ grow
406+ |%
407+ ++ noun update
408+ --
409+++ grad %noun
410+--
411+```
412+
413+This is a very simple mark file for the `update` type.
414+
415+### Publisher
416+
417+#### `/app/todo.hoon`
418+
419+```hoon
420+/- todo
421+/+ default-agent, dbug
422+|%
423++$ versioned-state
424+ $% state-0
425+ ==
426++$ state-0 [%0 =friends:todo =tasks:todo]
427++$ card card:agent:gall
428+--
429+%- agent:dbug
430+=| state-0
431+=* state -
432+^- agent:gall
433+|_ =bowl:gall
434++* this .
435+ def ~(. (default-agent this %.n) bowl)
436+::
437+++ on-init
438+ ^- (quip card _this)
439+ `this
440+::
441+++ on-save
442+ ^- vase
443+ !>(state)
444+::
445+++ on-load
446+ |= old-state=vase
447+ ^- (quip card _this)
448+ =/ old !<(versioned-state old-state)
449+ ?- -.old
450+ %0 `this(state old)
451+ ==
452+::
453+++ on-poke
454+ |= [=mark =vase]
455+ ^- (quip card _this)
456+ |^
457+ ?> =(src.bowl our.bowl)
458+ ?+ mark (on-poke:def mark vase)
459+ %todo-action
460+ =^ cards state
461+ (handle-poke !<(action:todo vase))
462+ [cards this]
463+ ==
464+ ++ handle-poke
465+ |= =action:todo
466+ ^- (quip card _state)
467+ ?- -.action
468+ %add
469+ ?: (~(has by tasks) now.bowl)
470+ $(now.bowl (add now.bowl ~s0..0001))
471+ :_ state(tasks (~(put by tasks) now.bowl [name.action %.n]))
472+ :~ :* %give %fact ~[/updates] %todo-update
473+ !>(`update:todo`[%add now.bowl name.action])
474+ ==
475+ ==
476+ ::
477+ %del
478+ :_ state(tasks (~(del by tasks) id.action))
479+ :~ :* %give %fact ~[/updates] %todo-update
480+ !>(`update:todo`action)
481+ ==
482+ ==
483+ ::
484+ %toggle
485+ :- :~ :* %give %fact ~[/updates] %todo-update
486+ !>(`update:todo`action)
487+ == ==
488+ %= state
489+ tasks %+ ~(jab by tasks)
490+ id.action
491+ |=(=task:todo task(done !done.task))
492+ ==
493+ ::
494+ %rename
495+ :- :~ :* %give %fact ~[/updates] %todo-update
496+ !>(`update:todo`action)
497+ == ==
498+ %= state
499+ tasks %+ ~(jab by tasks)
500+ id.action
501+ |=(=task:todo task(name name.action))
502+ ==
503+ %allow
504+ `state(friends (~(put in friends) who.action))
505+ ::
506+ %kick
507+ :_ state(friends (~(del in friends) who.action))
508+ :~ [%give %kick ~[/updates] `who.action]
509+ ==
510+ ==
511+ --
512+::
513+++ on-watch
514+ |= =path
515+ ^- (quip card _this)
516+ ?+ path (on-watch:def path)
517+ [%updates ~]
518+ ?> (~(has in friends) src.bowl)
519+ :_ this
520+ :~ [%give %fact ~ %todo-update !>(`update:todo`initial+tasks)]
521+ ==
522+ ==
523+::
524+++ on-leave on-leave:def
525+++ on-peek on-peek:def
526+++ on-agent on-agent:def
527+++ on-arvo on-arvo:def
528+++ on-fail on-fail:def
529+--
530+```
531+
532+This is the publisher agent, `todo.hoon`. The bulk of its logic is in its
533+`on-poke` arm, where it handles the various possible actions like `%add`ing a
534+task, `%toggle`ing its "done" state, `%rename`ing a task, and so on. It also has
535+a couple of `action`s for `%allow`ing and `%kick`ing subscribers.
536+
537+Most of these cases both update the state of the agent, as well as producing
538+`%fact` cards to send out to subscribers with the new data.
539+
540+You'll notice it only allows these pokes from the local ship, and enforces this
541+in `on-poke` with:
542+
543+```hoon
544+?> =(src.bowl our.bowl)
545+```
546+
547+Additionally, you might notice the `%add` case in `handle-poke` begins with the
548+following:
549+
550+```hoon
551+?: (~(has by tasks) now.bowl)
552+ $(now.bowl (add now.bowl ~s0..0001))
553+```
554+
555+Back in lesson two, we mentioned that the bowl is only repopulated when there's
556+a new Arvo event, so simultaneous messages from a local agent or web client
557+would be processed with the same bowl. Since we're using `now.bowl` for the task
558+ID, this means multiple `%add` actions could collide. To handle this case, we
559+check if there's already an entry in the `tasks` map with the current date-time,
560+and if there is, we increase the time by a fraction of a second and try again.
561+
562+Let's now look at `on-watch`:
563+
564+```hoon
565+++ on-watch
566+ |= =path
567+ ^- (quip card _this)
568+ ?+ path (on-watch:def path)
569+ [%updates ~]
570+ ?> (~(has in friends) src.bowl)
571+ :_ this
572+ :~ [%give %fact ~ %todo-update !>(`update:todo`initial+tasks)]
573+ ==
574+ ==
575+```
576+
577+When `on-watch` gets a subscription request, it checks whether the requesting
578+ship is in the `friends` set, and crashes if it is not. If they're in `friends`,
579+it produces a `%fact` card with a null `(list path)`, which means it goes only
580+to the new subscriber. This `%fact` contains the entire `tasks` map as it
581+currently exists, getting the new subscriber up to date.
582+
583+### Subscriber
584+
585+#### `/app/todo-watcher.hoon`
586+
587+```hoon
588+/- todo
589+/+ default-agent, dbug
590+|%
591++$ versioned-state
592+ $% state-0
593+ ==
594++$ state-0 [%0 ~]
595++$ card card:agent:gall
596+--
597+%- agent:dbug
598+=| state-0
599+=* state -
600+^- agent:gall
601+|_ =bowl:gall
602++* this .
603+ def ~(. (default-agent this %.n) bowl)
604+::
605+++ on-init
606+ ^- (quip card _this)
607+ `this
608+::
609+++ on-save
610+ ^- vase
611+ !>(state)
612+::
613+++ on-load
614+ |= old-state=vase
615+ ^- (quip card _this)
616+ =/ old !<(versioned-state old-state)
617+ ?- -.old
618+ %0 `this(state old)
619+ ==
620+::
621+++ on-poke
622+ |= [=mark =vase]
623+ ^- (quip card _this)
624+ ?> =(src.bowl our.bowl)
625+ ?+ mark (on-poke:def mark vase)
626+ %noun
627+ =/ action !<(?([%sub @p] [%unsub @p]) vase)
628+ ?- -.action
629+ %sub
630+ :_ this
631+ :~ [%pass /todos %agent [+.action %todo] %watch /updates]
632+ ==
633+ %unsub
634+ :_ this
635+ :~ [%pass /todos %agent [+.action %todo] %leave ~]
636+ ==
637+ ==
638+ ==
639+::
640+++ on-watch on-watch:def
641+++ on-leave on-leave:def
642+++ on-peek on-peek:def
643+::
644+++ on-agent
645+ |= [=wire =sign:agent:gall]
646+ ^- (quip card _this)
647+ ?+ wire (on-agent:def wire sign)
648+ [%todos ~]
649+ ?+ -.sign (on-agent:def wire sign)
650+ %watch-ack
651+ ?~ p.sign
652+ ((slog '%todo-watcher: Subscribe succeeded!' ~) `this)
653+ ((slog '%todo-watcher: Subscribe failed!' ~) `this)
654+ ::
655+ %kick
656+ %- (slog '%todo-watcher: Got kick, resubscribing...' ~)
657+ :_ this
658+ :~ [%pass /todos %agent [src.bowl %todo] %watch /updates]
659+ ==
660+ ::
661+ %fact
662+ ?+ p.cage.sign (on-agent:def wire sign)
663+ %todo-update
664+ ~& !<(update:todo q.cage.sign)
665+ `this
666+ ==
667+ ==
668+ ==
669+::
670+++ on-arvo on-arvo:def
671+++ on-fail on-fail:def
672+--
673+```
674+
675+This is the subscriber agent. Since it's just for demonstrative purposes, it has
676+no state and just prints the updates it receives. In practice it would keep the
677+`tasks` map it receives in its own state, and then update it as it receives new
678+`%fact`s.
679+
680+The `on-poke` arm is fairly simple - it accepts two pokes, to either `[%sub ~some-ship]` or `[%unsub ~some-ship]`.
681+
682+The `on-agent` arm will print whether a subscription request succeeded or
683+failed, as well as printing a message when it gets kicked. When it receives a
684+`%fact` from the publisher agent, it will just print it to the terminal with a
685+`~&` expression.
686+
687+### Trying it out
688+
689+We're going to try this between two different ships. The first ship will be the
690+usual fakezod. We'll add both `mark` files, the `/sur` file, and the `todo.hoon`
691+agent to the `%base` desk of our fakezod, putting them in the following
692+directories:
693+
694+```
695+base
696+├── app
697+│ └── todo.hoon
698+├── mar
699+│ └── todo
700+│ ├── action.hoon
701+│ └── update.hoon
702+└── sur
703+ └── todo.hoon
704+```
705+
706+In `~zod`'s dojo, we can `|commit %base`, and then start the `%todo` agent:
707+
708+```
709+|rein %base [& %todo]
710+```
711+
712+Now we need to spin up another fake ship. We'll use `~nut` in this example:
713+
714+```
715+urbit -F nut
716+```
717+
718+Once it's booted, we can `|mount %base` and then add just the `update.hoon` mark
719+file, the `/sur` file, and the `todo-watcher.hoon` agent like so:
720+
721+```
722+base
723+├── app
724+│ └── todo-watcher.hoon
725+├── mar
726+│ └── todo
727+│ └── update.hoon
728+└── sur
729+ └── todo.hoon
730+```
731+
732+On `~nut` we can then `|commit %base`, and start the `%todo-watcher` agent:
733+
734+```
735+|rein %base [& %todo-watcher]
736+```
737+
738+Now, on `~nut`, let's try subscribing:
739+
740+```
741+> :todo-watcher [%sub ~zod]
742+>=
743+%todo-watcher: Subscribe failed!
744+```
745+
746+Our `%todo-watcher` agent tried, but received a negative `%watch-ack` from
747+`%todo`, because we haven't yet added `~nut` to the `friends` set of allowed
748+ships. Let's now remedy that on `~zod`:
749+
750+```
751+> :todo &todo-action [%allow ~nut]
752+>=
753+```
754+
755+Let's also add a couple of to-do tasks, on `~zod`:
756+
757+```
758+> :todo &todo-action [%add 'foo']
759+>=
760+> :todo &todo-action [%add 'bar']
761+>=
762+```
763+
764+If we now check its state with `+dbug`, we'll see they're in the `tasks` map,
765+and `~nut` will also now be in the `friends` set:
766+
767+```
768+> [ %0
769+ friends={~nut}
770+ tasks
771+ { [ p=170.141.184.505.349.079.206.522.766.950.035.095.552
772+ q=[name='foo' done=%.n]
773+ ]
774+ [ p=170.141.184.505.349.079.278.538.984.166.386.565.120
775+ q=[name='bar' done=%.n]
776+ ]
777+ }
778+]
779+> :todo +dbug
780+>=
781+```
782+
783+Let's now try subscribing again on `~nut`:
784+
785+```
786+> :todo-watcher [%sub ~zod]
787+>=
788+%todo-watcher: Subscribe succeeded!
789+[ %initial
790+ tasks
791+ { [ p=170.141.184.505.349.079.206.522.766.950.035.095.552
792+ q=[name='foo' done=%.n]
793+ ]
794+ [ p=170.141.184.505.349.079.278.538.984.166.386.565.120
795+ q=[name='bar' done=%.n]
796+ ]
797+ }
798+]
799+```
800+
801+As you can see, this time it's worked, and we've immediately received the
802+initial `tasks` map.
803+
804+Now, let's try adding another task on `~zod`:
805+
806+```
807+> :todo &todo-action [%add 'baz']
808+>=
809+```
810+
811+On `~nut`, we'll see it has received the `%fact` with the new task in it:
812+
813+```
814+[ %add
815+ id=170.141.184.505.349.082.779.030.192.959.445.270.528
816+ name='baz'
817+]
818+```
819+
820+Let's try toggle its done state on `~zod`:
821+
822+```
823+> :todo &todo-action [%toggle 170.141.184.505.349.082.779.030.192.959.445.270.528]
824+>=
825+```
826+
827+`~nut` will again get the `%fact`:
828+
829+```
830+[ %toggle
831+ id=170.141.184.505.349.082.779.030.192.959.445.270.528
832+]
833+```
834+
835+Recall that incoming subscriptions are stored in `sup.bowl`, and outgoing
836+subscriptions are stored in `wex.bowl`. Let's have a look at the incoming
837+subscription on `~zod`:
838+
839+```
840+> [ path=/updates
841+ from=~nut
842+ duct=~[/gall/sys/req/~nut/todo /ames/bone/~nut/1 //ames]
843+]
844+> :todo +dbug [%incoming %ship ~nut]
845+>=
846+```
847+
848+On `~nut`, let's look at the outgoing subscription:
849+
850+```
851+> [wire=/todos agnt=[~zod %todo] path=/updates ackd=%.y]
852+> :todo-watcher +dbug [%outgoing %ship ~zod]
853+>=
854+```
855+
856+Now on `~zod`, let's try kicking `~nut` and removing it from our `friends` set:
857+
858+```
859+> :todo &todo-action [%kick ~nut]
860+>=
861+```
862+
863+On `~nut`, we'll see it got the `%kick`, tried resubscribing automatically, but
864+was rejected because `~nut` is no longer in `friends`:
865+
866+```
867+%todo-watcher: Got kick, resubscribing...
868+%todo-watcher: Subscribe failed!
869+```
870+
871+## Summary
872+
873+- Incoming subscription requests arrive in an agent's `on-watch` arm.
874+- An agent will define various subscription `path`s in its `on-watch` arm, which
875+ others can subscribe to.
876+- Gall will automatically produce a negative `%watch-ack` if `on-watch` crashed,
877+ and a positive one if it was successful.
878+- Incoming subscribers are recorded in the `sup` field of the `bowl`.
879+- `on-watch` can produce a `%fact` with a null `(list path)` which will go only
880+ to the new subscriber.
881+- Updates are sent to subscribers in `%fact` cards, and contain a `cage` with a
882+ `mark` and some data in a `vase`.
883+- `%fact`s are sent to all subscribers of the paths specified in the `(list path)`.
884+- A subscriber can be kicked from subscription paths with a `%kick` card
885+ specifying the ship in the `(unit ship)`. All subscribers of the specified
886+ paths will be kicked if the `(unit ship)` is null.
887+- An outgoing subscription can be initiated with a `%watch` card.
888+- The `%watch-ack` will come back in to the subscriber's `on-agent` arm as a
889+ `sign`, and may be positive or negative, depending on whether the `(unit tang)` is null.
890+- `%kick`s will also arrive in the subscriber's `on-agent` arm as a `sign`.
891+ Since kicks may not be intentional, the subscriber should attempt to
892+ resubscribe and only give up if the subsequent `%watch-ack` is negative.
893+- `%fact`s will also arrive in the subscriber's `on-agent` arm.
894+- All such `sign`s that arrive in `on-agent` will also have a `wire`.
895+- The `wire` for subscription updates to arrive on is specified in the initial
896+ `%watch` card.
897+- A subscriber can unsubscribe by passing a `%leave` card on the original
898+ `wire`.
899+
900+## Exercises
901+
902+- Have a look at the [Strings Guide](/docs/hoon/guides/strings) if you're not
903+ already familiar with decoding/encoding atoms in strings.
904+- Try running through the [example](#example) yourself, if you've not done so
905+ already.
906+- Try modifying `%todo-watcher` to recording the data it receives in its state,
907+ rather than simply printing it to the terminal.
908+- If you'd like, try going back to [lesson
909+ 6](/docs/userspace/gall-guide/6-pokes) (on pokes) and modifying the agents
910+ with an appropriate permission system, and also try running them on separate
911+ ships.
content/docs/userspace/gall-guide/9-vanes.mdnew file+284
@@ -0,0 +1,284 @@
1++++
2+title = "9. Vanes"
3+weight = 45
4+template = "doc.html"
5++++
6+
7+In this lesson we're going to look at interacting with vanes (kernel modules).
8+The API for each vane consists of `task`s it can take, and `gift`s it can
9+return. The `task`s and `gift`s for each vane are defined in its section of
10+`lull.hoon`. Here's the `task:iris`s and `gift:iris`s for Iris, the HTTP client
11+vane, as an example:
12+
13+```hoon
14+|%
15++$ gift
16+ $% [%request id=@ud request=request:http]
17+ [%cancel-request id=@ud]
18+ [%http-response =client-response]
19+ ==
20++$ task
21+ $~ [%vega ~]
22+ $% $>(%born vane-task)
23+ $>(%trim vane-task)
24+ $>(%vega vane-task)
25+ [%request =request:http =outbound-config]
26+ [%cancel-request ~]
27+ [%receive id=@ud =http-event:http]
28+ ==
29+```
30+
31+The API of each vane is documented in its respective section of the [Arvo
32+documentation](/docs/arvo/overview). Each vane has a detailed API reference and
33+examples of their usage. There are far too many `task`s and `gift`s across the
34+vanes to cover here, so in the [`Example`](#example) section of this document,
35+we'll just look at a single, simple example with a Behn timer. The basic pattern
36+in the example is broadly applicable to the other vanes as well.
37+
38+## Sending a vane task
39+
40+A `task` can be sent to a vane by `%pass`ing it an `%arvo` card. We touched on
41+these in the [Cards](/docs/userspace/gall-guide/5-cards) lesson, but we'll
42+briefly recap it here. The type of the card is as follows:
43+
44+```hoon
45+[%pass path %arvo note-arvo]
46+```
47+
48+The `path` will just be the `wire` you want the response to arrive on. The
49+`note-arvo` is the following union:
50+
51+```hoon
52++$ note-arvo
53+ $~ [%b %wake ~]
54+ $% [%a task:ames]
55+ [%b task:behn]
56+ [%c task:clay]
57+ [%d task:dill]
58+ [%e task:eyre]
59+ [%g task:gall]
60+ [%i task:iris]
61+ [%j task:jael]
62+ [%$ %whiz ~]
63+ [@tas %meta vase]
64+ ==
65+```
66+
67+The letter tags just specify which vane it goes to, and then follows the `task`
68+itself. Here are a couple of examples. The first sends a `%wait` `task:behn` to
69+Behn, setting a timer to go off one minute in the future. The second sends a
70+`%warp` `task:clay` to Clay, asking whether `sys.kelvin` exists on the `%base`
71+desk.
72+
73+```hoon
74+[%pass /some/wire %arvo %b %wait (add ~m1 now.bowl)]
75+[%pass /some/wire %arvo %c %warp our.bowl %base ~ %sing %u da+now.bowl /sys/kelvin]
76+```
77+
78+## Receiving a vane gift
79+
80+Once a `task` has been sent to a vane, any `gift`s the vane sends back in
81+response will arrive in the `on-arvo` arm of your agent. The `on-arvo` arm
82+exclusively handles such vane `gift`s. The `gift`s will arrive in a `sign-arvo`,
83+along with the `wire` specified in the original request. The `on-arvo` arm
84+produces a `(quip card _this)` like usual, so it would look like:
85+
86+```hoon
87+++ on-arvo
88+ |= [=wire =sign-arvo]
89+ ^- (quip card _this)
90+ .....
91+```
92+
93+A `sign-arvo` is the following structure, defined in `lull.hoon`:
94+
95+```hoon
96++$ sign-arvo
97+ $% [%ames gift:ames]
98+ $: %behn
99+ $% gift:behn
100+ $>(%wris gift:clay)
101+ $>(%writ gift:clay)
102+ $>(%mere gift:clay)
103+ $>(%unto gift:gall)
104+ ==
105+ ==
106+ [%clay gift:clay]
107+ [%dill gift:dill]
108+ [%eyre gift:eyre]
109+ [%gall gift:gall]
110+ [%iris gift:iris]
111+ [%jael gift:jael]
112+ ==
113+```
114+
115+The head of the `sign-arvo` will be the name of the vane like `%behn`, `%clay`,
116+etc. The tail will be the `gift` itself. Here are a couple of `sign-arvo`
117+examples, and the responses to the example `task`s in the previous section:
118+
119+```hoon
120+[%behn %wake ~]
121+```
122+
123+```
124+[ %clay
125+ [ %writ
126+ p
127+ [ ~
128+ [ p=[p=%u q=[%da p=~2021.11.17..13.55.00..c195] r=%base]
129+ q=/sys/kelvin
130+ r=[p=%flag q=[#t/?(%.y %.n) q=0]]
131+ ]
132+ ]
133+ ]
134+]
135+```
136+
137+The typical pattern is to first test the `wire` with something like a wutlus
138+(`?+`) expression, and then test the `sign-arvo`. Since most `gift`s are
139+head-tagged, you can test both the vane and the gift at the same time like:
140+
141+```hoon
142+?+ sign-arvo (on-arvo:def wire sign-arvo)
143+ [%behn %wake *]
144+ .....
145+....
146+```
147+
148+## Example
149+
150+Here's a very simple example that takes a poke of a `@dr` (a relative date-time
151+value) and sends Behn a `%wait` `task:behn`, setting a timer to go off `@dr` in
152+the future. When the timer goes off, `on-arvo` will take the `%wake` `gift:behn`
153+and print "Ding!" to the terminal.
154+
155+#### `ding.hoon`
156+
157+```hoon
158+/+ default-agent, dbug
159+|%
160++$ card card:agent:gall
161+--
162+%- agent:dbug
163+^- agent:gall
164+|_ =bowl:gall
165++* this .
166+ def ~(. (default-agent this %.n) bowl)
167+++ on-init on-init:def
168+++ on-save on-save:def
169+++ on-load on-load:def
170+++ on-poke
171+ |= [=mark =vase]
172+ ^- (quip card _this)
173+ ?+ mark (on-poke:def mark vase)
174+ %noun
175+ :_ this
176+ :~ [%pass /timers %arvo %b %wait (add now.bowl !<(@dr vase))]
177+ ==
178+ ==
179+++ on-watch on-watch:def
180+++ on-leave on-leave:def
181+++ on-peek on-peek:def
182+++ on-agent on-agent:def
183+++ on-arvo
184+ |= [=wire =sign-arvo]
185+ ^- (quip card _this)
186+ ?+ wire (on-arvo:def wire sign-arvo)
187+ [%timers ~]
188+ ?+ sign-arvo (on-arvo:def wire sign-arvo)
189+ [%behn %wake *]
190+ ?~ error.sign-arvo
191+ ((slog 'Ding!' ~) `this)
192+ (on-arvo:def wire sign-arvo)
193+ ==
194+ ==
195+++ on-fail on-fail:def
196+--
197+```
198+
199+Let's examine the `on-poke` arm:
200+
201+```hoon
202+++ on-poke
203+ |= [=mark =vase]
204+ ^- (quip card _this)
205+ ?+ mark (on-poke:def mark vase)
206+ %noun
207+ :_ this
208+ :~ [%pass /timers %arvo %b %wait (add now.bowl !<(@dr vase))]
209+ ==
210+ ==
211+```
212+
213+A Behn `%wait` task has the format `[%wait @da]` - the `@da` (an absolute
214+date-time value) is the time the timer should go off. The `vase` of the poke
215+takes a `@dr`, so we extract it directly into an `add` expression, producing a
216+date-time `@dr` from now. Behn will receive the `%wait` task and set the timer
217+in Unix. When it fires, Behn will produce a `%wake` `gift:behn` and deliver it
218+to `on-arvo`, on the `wire` we specified (`/timers`). Here's the `on-arvo` arm:
219+
220+```hoon
221+++ on-arvo
222+ |= [=wire =sign-arvo]
223+ ^- (quip card _this)
224+ ?+ wire (on-arvo:def wire sign-arvo)
225+ [%timers ~]
226+ ?+ sign-arvo (on-arvo:def wire sign-arvo)
227+ [%behn %wake *]
228+ ?~ error.sign-arvo
229+ ((slog 'Ding!' ~) `this)
230+ (on-arvo:def wire sign-arvo)
231+ ==
232+ ==
233+```
234+
235+We remark that, just like in the case of agent-agent communication, `gift`s from Arvo are also routed `wire` before `sign-arvo`.
236+
237+First we check the `wire` is `/timers`, and then we check the `sign-arvo` begins
238+with `[%behn %wake ....]`. Behn's `%wake` gift has the following format:
239+
240+```hoon
241+[%wake error=(unit tang)]
242+```
243+
244+The `error` is null if the timer fired successfully, and contains an error in
245+the `tang` if it did not. We therefore test whether `error.sign-arvo` is `~`,
246+and if it is, we print `Ding!` to the terminal. If the `wire`, `sign-arvo` or
247+`error` are something unexpected, we pass it to `%default-agent`, which
248+will just crash and print an error message.
249+
250+Let's try it out. Save the agent above as `/app/ding.hoon` on the `%base` desk
251+and `|commit %base`. Then, start the agent with `|rein %base [& %ding]`.
252+
253+Next, in the dojo let's try poking our agent, setting a timer for five seconds
254+from now:
255+
256+```
257+> :ding ~s5
258+>=
259+```
260+
261+After approximately five seconds, we see the timer fired successfully:
262+
263+```
264+> Ding!
265+```
266+
267+## Summary
268+
269+- Each vane has an API composed of `task`s it takes and `gift`s it produces.
270+- Each vane's `task`s and `gift`s are defined in `lull.hoon`
271+- Each vane's section of the [Arvo documentation](/docs/arvo/overview) includes
272+ an API reference that explains its `task`s and `gift`s, as well as an Examples
273+ section demonstrating their usage.
274+- Vane `task`s can be sent to vanes by `%pass`ing them an `%arvo` `card`.
275+- Vane `gift`s come back to the `on-arvo` arm of the agent core in a
276+ `sign-arvo`.
277+
278+## Exercises
279+
280+- Run through the [Example](#example) yourself if you've not done so already.
281+- Have a look at some vane sections of `lull.hoon` to familiarize yourself with
282+ its structure.
283+- Have a quick look at the API reference sections of a couple of vanes in the
284+ [Arvo documentation](/docs/arvo/overview).
content/docs/userspace/gall-guide/_index.mdnew file+45
@@ -0,0 +1,45 @@
1++++
2+title = "Gall Guide"
3+weight = 5
4+sort_by = "weight"
5+template = "sections/docs/chapters.html"
6+insert_anchor_links = "right"
7++++
8+
9+## Table of Contents
10+
11+- [Introduction](/docs/userspace/gall-guide/intro)
12+
13+#### Lessons
14+
15+1. [Arvo](/docs/userspage/gall-guide/1-arvo) - This lesson provides an
16+ overview of the Arvo operating system, and some other useful background
17+ information.
18+2. [The Agent Core](/docs/userspace/gall-guide/2-agent) - This lesson goes over
19+ the basic structure of a Gall agent.
20+3. [Imports and Aliases](/docs/userspace/gall-guide/3-imports-and-aliases) -
21+ This lesson covers some useful libraries, concepts and boilerplate commonly
22+ used when writing Gall agents.
23+4. [Lifecycle](/docs/userspace/gall-guide/4-lifecycle) - This lesson introduces
24+ the state management arms of an agent.
25+5. [Cards](/docs/userspace/gall-guide/5-cards) - This lesson covers `card`s -
26+ the structure used to pass messages to other agents and vanes.
27+6. [Pokes](/docs/userspace/gall-guide/6-pokes) - This lesson covers sending and
28+ receiving one-off messages called "pokes" between agents.
29+7. [Structures and Marks](/docs/userspace/gall-guide/7-sur-and-marks) - This
30+ lesson talks about importing type defintions, and writing `mark` files.
31+8. [Subscriptions](/docs/userspace/gall-guide/8-subscriptions) - This lesson
32+ goes through the mechanics of subscriptions - both inbound and outbound.
33+9. [Vanes](/docs/userspace/gall-guide/9-vanes) - This lesson explains how to
34+ interact with vanes (kernel modules) from an agent.
35+10. [Scries](/docs/userspace/gall-guide/10-scry) - This lesson gives an overview
36+ of scrying Gall agents, and how scry endpoints are defined in agents.
37+11. [Failure](/docs/userspace/gall-guide/11-fail) - This lesson covers how Gall
38+ handles certain errors and crashes, as well as the concept of a helper core.
39+12. [Next Steps](/docs/userspace/gall-guide/12-next-steps) - The Gall Guide is
40+ now complete - here are some things you can look at next.
41+
42+#### Appendix
43+
44+- [Types](/docs/userspace/gall-guide/types) - A reference for a few of
45+ the types used in the Gall Guide.
content/docs/userspace/gall-guide/intro.mdnew file+83
@@ -0,0 +1,83 @@
1++++
2+title = "Introduction"
3+weight = 1
4+template = "doc.html"
5++++
6+
7+This guide will walk through everything you need to know to write your own Gall
8+agents.
9+
10+The Gall Guide is suitable for anyone with an intermediate knowledge of Hoon. If
11+you've worked through [Hoon School](/docs/hoon/hoon-school/intro) or something
12+equivalent, you should be fine.
13+
14+## What are Gall agents?
15+
16+Gall is one of the eight vanes (kernel modules) of Arvo, Urbit's operating
17+system. Gall's purpose is to manage userspace applications called _agents_.
18+
19+An agent is a piece of software that is primarily focused on maintaining and
20+distributing a piece of state with a defined structure. It exposes an interface
21+that lets programs read, subscribe to, and manipulate the state. Every event
22+happens in an atomic transaction, so the state is never inconsistent. Since the
23+state is permanent, when the agent is upgraded with a change to the structure of
24+the state, the developer provides a migration function from the old state type
25+to the new state type.
26+
27+It's not too far off to think of an agent as simply a database with
28+developer-defined logic. But an agent is significantly less constrained than a
29+database. Databases are usually tightly constrained in one or more ways because
30+they need to provide certain guarantees (like atomicity) or optimizations (like
31+indexes). Arvo is a [single-level store](/docs/arvo/overview#single-level-store), so atomicity comes for free. Many
32+applications don't use databases because they need relational indices; rather,
33+they use them for their guarantees around persistence. Some do need the indices,
34+though, and it's not hard to imagine an agent which provides a SQL-like
35+interface.
36+
37+On the other hand, an agent is also a lot like what many systems call a
38+"service". An agent is permanent and addressable -- a running program can talk
39+to an agent just by naming it. An agent can perform [IO](/blog/io-in-hoon), unlike most databases.
40+This is a critical part of an agent: it performs IO along the same transaction
41+boundaries as changes to its state, so if an effect happens, you know that the
42+associated state change has happened.
43+
44+But the best way to think about an agent is as a state machine. Like a state
45+machine, any input could happen at any time, and it must react coherently to
46+that input. Output (effects) and the next state of the machine are a pure
47+function of the previous state and the input event.
48+
49+## Table of Contents
50+
51+#### Lessons
52+
53+1. [Arvo](/docs/userspace/gall-guide/1-arvo) - This lesson provides an
54+ overview of the Arvo operating system, and some other useful background
55+ information.
56+2. [The Agent Core](/docs/userspace/gall-guide/2-agent) - This lesson goes over
57+ the basic structure of a Gall agent.
58+3. [Imports and Aliases](/docs/userspace/gall-guide/3-imports-and-aliases) -
59+ This lesson covers some useful libraries, concepts and boilerplate commonly
60+ used when writing Gall agents.
61+4. [Lifecycle](/docs/userspace/gall-guide/4-lifecycle) - This lesson introduces
62+ the state management arms of an agent.
63+5. [Cards](/docs/userspace/gall-guide/5-cards) - This lesson covers `card`s -
64+ the structure used to pass messages to other agents and vanes.
65+6. [Pokes](/docs/userspace/gall-guide/6-pokes) - This lesson covers sending and
66+ receiving one-off messages called "pokes" between agents.
67+7. [Structures and Marks](/docs/userspace/gall-guide/7-sur-and-marks) - This
68+ lesson talks about importing type defintions, and writing `mark` files.
69+8. [Subscriptions](/docs/userspace/gall-guide/8-subscriptions) - This lesson
70+ goes through the mechanics of subscriptions - both inbound and outbound.
71+9. [Vanes](/docs/userspace/gall-guide/9-vanes) - This lesson explains how to
72+ interact with vanes (kernel modules) from an agent.
73+10. [Scries](/docs/userspace/gall-guide/10-scry) - This lesson gives an overview
74+ of scrying Gall agents, and how scry endpoints are defined in agents.
75+11. [Failure](/docs/userspace/gall-guide/11-fail) - This lesson covers how Gall
76+ handles certain errors and crashes, as well as the concept of a helper core.
77+12. [Next Steps](/docs/userspace/gall-guide/12-next-steps) - The Gall Guide is
78+ now complete - here are some things you can look at next.
79+
80+#### Appendix
81+
82+- [Types](/docs/userspace/gall-guide/types) - A reference for a few of
83+ the types used in the Gall Guide.
content/docs/userspace/gall-guide/types.mdnew file+228
@@ -0,0 +1,228 @@
1++++
2+title = "Appendix: Types"
3+weight = 65
4+template = "doc.html"
5++++
6+
7+This document explains a few of the types commonly used in Gall agents. In
8+addition to these, the [Data Types](/docs/arvo/gall/data-types) section of the
9+Gall vane documentation is a useful reference. In particular, the whole
10+[`agent`](/docs/arvo/gall/data-types#agent) subsection, as well as
11+[`bowl`](/docs/arvo/gall/data-types#bowl),
12+[`boat`](/docs/arvo/gall/data-types#boat), and
13+[`bitt`](/docs/arvo/gall/data-types#bitt).
14+
15+## `vase`
16+
17+Vases are used to encapsulate _dynamically typed_ data - they let typed data be
18+moved around in contexts where you can't know the type ahead of time, and
19+therefore can't have a _static_ type.
20+
21+Vases are used extensively - almost all data your agent will send
22+and received is wrapped in a vase.
23+
24+A vase is just a cell with data in the tail and the type of the data in the
25+head. Its formal definition is:
26+
27+```hoon
28++$ vase [p=type q=*]
29+```
30+
31+Here's what it looks like if we bunt a vase in the dojo:
32+
33+```
34+> *vase
35+[#t/* q=0]
36+```
37+
38+There are two simple runes used to create and unpack vases. We'll look at each
39+of these next.
40+
41+### Create a `vase`
42+
43+The [zapgar](https://urbit.org/docs/hoon/reference/rune/zap#-zapgar) rune (`!>`)
44+takes a single argument of any noun, and wraps it in a vase. For example, in the
45+dojo:
46+
47+```
48+> !>([1 2 3])
49+[#t/[@ud @ud @ud] q=[1 2 3]]
50+
51+> !>('foo')
52+[#t/@t q=7.303.014]
53+
54+> !>([[0xdead 0xb33f] 'foo'])
55+[#t/[[@ux @ux] @t] q=[[57.005 45.887] 7.303.014]]
56+
57+> !>(foo='bar')
58+[#t/foo=@t q=7.496.034]
59+```
60+
61+You would typically use `!>` as part of a [`cage`](#cage) when you're
62+constructing a `card` like a poke or a `%fact` `gift` to be sent off.
63+
64+### Extract data from `vase`
65+
66+The [zapgal](https://urbit.org/docs/hoon/reference/rune/zap#-zapgal) rune (`!<`)
67+takes two arguments: A mold specifying the type to try and extract the data as,
68+and the vase to be extracted.
69+
70+Let's look at an example in the dojo. First, let's create a vase of `[@t @ux @ud]`:
71+
72+```
73+> =myvase !>(['foo' 0xabcd 123])
74+> myvase
75+[#t/[@t @ux @ud] q=[7.303.014 43.981 123]]
76+```
77+
78+Next, let's try extracting our vase:
79+
80+```
81+> !< [@t @ux @ud] myvase
82+['foo' 0xabcd 123]
83+```
84+
85+Now let's try asking for a `@p` rather than `@t`:
86+
87+```
88+> !< [@p @ux @ud] myvase
89+-need.@p
90+-have.@t
91+nest-fail
92+```
93+
94+As you can see, it will crash if the type does not nest. Note that
95+rather than using `!<`, you can also just clam the tail of the vase like:
96+
97+```
98+> ((trel @t @ux @ud) +.myvase)
99+[p='foo' q=0xabcd r=123]
100+```
101+
102+The only problem is that you can't tell if the auras were wrong:
103+
104+```
105+> ((trel @p @ud @ux) +.myvase)
106+[p=~sibtel-tallyd q=43.981 r=0x7b]
107+```
108+
109+You'd typically use `!<` on the data in `card`s that come in from other ships,
110+agents, etc.
111+
112+## `mark`
113+
114+The `mark` type is just a `@tas` like `%foo`, and specifies the Clay filetype of
115+some data. The `mark` corresponds to a mark file in the `/mar` directory, so a
116+`mark` of `%foo` corresponds to `/mar/foo/hoon`. Mark files are used for saving
117+data in Clay, validating data sent between agents or over the network, and
118+converting between different data types. For more information about mark files,
119+you can refer to the [Marks section of the Clay
120+documentation](/docs/arvo/clay/marks/marks).
121+
122+## `cage`
123+
124+A `cage` is a cell of a [`mark`](#mark) and a [`vase`](#vase), like `[%foo !>('bar')]`. The data in the vase should match the data type of the specified
125+mark.
126+
127+Most data an agent sends will be in a `cage`, and most data it receives will
128+arrive in a `cage`. The `mark` may be used to validate or convert the data in
129+the `vase`, depending on the context.
130+
131+## `quip`
132+
133+`quip` is a mold-builder. A `(quip a b)` is equivalent to `[(list a) b]`, it's
134+just a more convenient way to specify it. Most arms of an agent return a `(quip card _this)`, which is a list of effects and a new state.
135+
136+## `path`
137+
138+The `path` type is formally defined as:
139+
140+```hoon
141++$ path (list knot)
142+```
143+
144+A knot is a `@ta` text atom (see the [Strings guide](/docs/hoon/guides/strings)
145+for details), so a `path` is just a list of text. Rather than having to write
146+`[~.foo ~.bar ~.baz ~]` though, it has its own syntax which looks like
147+`/foo/bar/baz`.
148+
149+A `path` is similar to a filesystem path in Unix, giving data a location in a
150+nested hierarchy. In Arvo though, they're not only used for files, but are a
151+more general type used for several different purposes. Its elements have no
152+inherent significance, it depends on the context. In a Gall agent, a `path` is
153+most commonly a subscription path - you might subscribe for updates to
154+`/foo/bar` on another agent, or another agent might subscribe to `/baz` on your
155+agent.
156+
157+A `path` might just be a series of fixed `@ta` like `/foo/bar`, but some
158+elements might also be variable and include encoded atoms, or some other datum. For
159+example, you might like to include a date in the path like
160+`/updates/~2021.10.31..07.24.27..db68`. Other agents might create the path by
161+doing something like:
162+
163+```hoon
164+/update/(scot %da now.bowl)
165+```
166+
167+Then, when you get a subscription request, you might do something like:
168+
169+```hoon
170+?+ path !!
171+ [%updates @ ~]
172+ =/ date=@da (slav %da i.t.path)
173+ ...(rest of code)...
174+```
175+
176+See the [Encoding in text](/docs/hoon/guides/strings#encoding-in-text) and
177+[Decoding from text](/docs/hoon/guides/strings#decoding-from-text) sections of
178+the Strings guide for more information on dealing with atoms encoded in strings.
179+
180+Aside from using function calls when constructing a `path` as demonstrated
181+above, you can also insert text you're previously stored with `=/` or what have
182+you, simply by enclosing them in brackets. For example, in the dojo:
183+
184+```
185+> =const ~.bar
186+> `path`/foo/[const]/baz
187+/foo/bar/baz
188+```
189+
190+## `wire`
191+
192+The type of a wire is formally defined as:
193+
194+```hoon
195++$ wire path
196+```
197+
198+So, a `wire` is just a [`path`](#path), type-wise they're exactly the same. The
199+reason there's a separate `wire` type is just to differentiate their purpose. A
200+`wire` is a path for responses to requests an agent initiates. If you subscribe
201+to the `path` `/some/path` on another agent, you also specify `/some/wire`.
202+Then, when that agent sends out updates to subscribers of `/some/path`, your
203+agent receives them on `/some/wire`.
204+
205+More formally, `wire`s are used by Arvo to represent an event cause, and
206+therefore return path, in a call stack called a
207+[`duct`](/docs/arvo/overview#duct). Inter-vane communications happen over
208+`duct`s as [`move`](/docs/arvo/overview#moves)s, and Gall converts the `card`s
209+produced by agents into such `move`s behind the scenes. A detailed understanding
210+of this system is not necessary to write Gall agents, but if you're interested
211+it's comprehensively documented in the [Arvo overview](/docs/arvo/overview) and
212+[move trace tutorial](/docs/arvo/tutorials/move-trace).
213+
214+For agents, the `wire` is specified in the second argument of a `%pass` `card`.
215+It's used for anything you can `%pass`, such as `%poke`s, `%watch`es, and
216+`%arvo` notes. For example:
217+
218+```hoon
219+[%pass /this/is/wire %agent [~zod %foobar] %watch /this/is/path]
220+::
221+[%pass /this/is/wire %agent [~zod %foobar] %poke %foo !>('hello')]
222+::
223+[%pass /this/is/wire %arvo %b %wait (add now.bowl ~m1)]
224+```
225+
226+The `on-agent` and `on-arvo` arms of the agent core include a `wire` in their
227+respective sample. Responses from agents come in to the former, and responses
228+from vanes come in to the latter.
content/docs/userspace/overview.md+5−1
@@ -6,7 +6,7 @@ template = "doc.html"
66
77Urbit's userspace is managed by Arvo's Gall vane.
88
9−## [Gall Guide](/docs/userspace/gall-guide)
9+## [Gall Guide](/docs/userspace/gall-guide/intro)
1010
1111Learn how to develop Gall agents.
1212
@@ -22,3 +22,7 @@ Resources on the flagship web client Landcape, through which most users interact
2222## [Threads](/docs/userspace/threads/)
2323
2424Resources on threads - monadic functions useful for complex IO.
25+
26+## [Distribution](/docs/userspace/dist/dist)
27+
28+Learn how to distribute apps on Urbit.