5aliases = ["/docs/learn/arvo/gall/"]
8This document describes Gall, the Arvo userspace vane. It is accurate as of 2019.12.02.
12Urbit code lives in the following basic categories:
14- Runtime (Nock interpreter, persistence engine, IO drivers, jets)
15- Kernel vanes (managed by Arvo)
16- Userspace agents (managed by Gall, permanent state)
17- Userspace imps (managed by Spider, transient state)
19This lays out the framework for the third category: Agents.
21An agent is a piece of software that is primarily focused on maintaining
22and distributing a piece of state with a defined structure. It exposes
23an interface that lets programs read, subscribe to, and manipulate the
24state. Every event happens in an atomic transaction, so the state is
25never inconsistent. Since the state is permanent, when the agent is
26upgraded with a change to the structure of the state, the developer
27provides a migration function from the old state type to the new state
30It's not too far off to think of an agent as simply a database with
31developer-defined logic. But an agent is significantly less constrained
32than a database. Databases are usually tightly constrained in one or
33more ways because they need to provide certain guarantees (like
34atomicity) or optimizations (like indexes). Urbit is a single-level
35store, so atomicity comes for free. Many applications don't use
36databases because they need relational indices; rather, they use them
37for their guarantees around persistence. Some do need the indices,
38though, and it's not hard to imagine an agent which provides a SQL-like
41On the other hand, an agent is also a lot like what many systems call a
42"service". An agent is permanent and addressable -- a running program
43can talk to an agent just by naming it. An agent can perform IO, unlike
44most databases. This is a critical part of an agent: it performs IO
45along the same transaction boundaries as changes to its state, so if an
46effect happens, you know that the associated state change has happened.
47You should be careful, though, to avoid creating implicit non-atomicity
48-- eg if it comes into an inconsistent state pending a response to an IO
51But the best way to think about an agent is as a state machine. Like a
52state machine, any input could happen at any time, and it must react
53coherently to that input. Ouput (effects) and the next state of the
54machine are both pure functions of the previous state and the input
55event. Of course, it's important to ensure there isn't an order of
56events that could cause your agent to enter an inconsistent state.
58We often think of state machines as finite state machines, but of course
59in this instance, the state is not actually finite (though it should be
60definable in a regular recursive data type).
64An agent is defined as a [core](/docs/glossary/core/) with a set of [arms](/docs/glossary/arm/) to handle various
65events. These handlers usually produce a list of effects and the next
66state of the agent. The interface definition can be found in
67`sys/zuse.hoon`, which at the time of writing is:
73 +$ step (quip card form)
74 +$ card (wind note gift)
77 [%agent [=ship name=term] =task]
81 [%watch-as =mark =path]
84 [%poke-as =mark =cage]
87 $% [%fact path=(unit path) =cage]
88 [%kick path=(unit path) ship=(unit ship)]
89 [%watch-ack p=(unit tang)]
90 [%poke-ack p=(unit tang)]
93 $% [%poke-ack p=(unit tang)]
94 [%watch-ack p=(unit tang)]
102 *(quip card _^|(..on-init))
109 *(quip card _^|(..on-init))
113 *(quip card _^|(..on-init))
117 *(quip card _^|(..on-init))
121 *(quip card _^|(..on-init))
129 *(quip card _^|(..on-init))
133 *(quip card _^|(..on-init))
137 *(quip card _^|(..on-init))
141Here's a skeleton example of an implementation:
156 `this(state !<(@ old-state-vase))
161 ~& got-poked-with-data=mark
178 |= [wire sign:agent:gall]
191We also supply a `default-agent` library, which is useful if you don't
192want to handle some of the arms. Most of the above could also be
201 default ~(. (default-agent this %|) bowl)
203++ on-init on-init:default
204++ on-save on-save:default
205++ on-load on-load:default
209 ~& got-poked-with-data=mark
213++ on-watch on-watch:default
214++ on-leave on-leave:default
215++ on-peek on-peek:default
216++ on-agent on-agent:default
217++ on-arvo on-arvo:default
218++ on-fail on-fail:default
222So, an agent is a core with 10 arms. The handlers correspond to
223different sorts of input. We'll discuss each of these in detail, but
224first a few general concepts.
228### Interacting with an agent
230There are two basic ways programs can interact with an agent: you may
231subscribe to data as described above, or you may "poke" an agent to send
232it a command. Suppose there is an agent for a calendar service; then,
233pokes would likely include `%create-event`, `%modify-event`,
234`%delete-event`, and `%change-time-zone` (along with assocated data).
235Subscription paths could include `/next-event` and `/all-events`.
237Agents generally conform to CQRS -- pokes may change state, but
238subscriptions generally shouldn't. In practice, there may be state
239changes required to properly initialize the subscription, but these
240shouldn't change the essential state of the agent.
244Most of the handlers produce a `(quip card agent:gall)`, which is just
245`[(list card) agent:gall]`. The first allows us to produce effects, and
246the second allows us to maintain state.
248A card is one of two things: a `note` or a `gift`. You `%pass` `note`s
249and you `%give` `gift`s. When you pass a `note`, you expect zero or more
250responses; when you give a `gift`, you will not get a response. Since you
251may get a response to a `note`, you tag it with a `wire` so that you can
252identify the response.
254Thus, we say you "pass a `note` along a `wire`", or you "give a `gift`".
255These phrases correspond to producing a card that looks like:
258[%pass /my/wire a-note]
267When you give a `gift`, that's the end of the story, but when you pass a
268`note`, you may get a response. If you do, then the response will come
269tagged with the `wire` that you used to pass the `note`. It's generally
270good practice to make your `wire`s fairly unique within your agent, since
271otherwise you may not be able to distinguish the responses.
275An agent may pass `note`s to either Arvo or another agent. If the `note` is
276to another agent, then it should usually be one of these:
279[%pass /my/wire %agent [our.bowl agent-name] %watch /a/path]
280[%pass /my/wire %agent [our.bowl agent-name] %leave ~]
281[%pass /my/wire %agent [our.bowl agent-name] %poke %foo-mark !>(poke-data)]
284Note that to unsubscribe to a `path`, you must send the unsubscription on
285the same `wire` that sent for the original subscription. Don't subscribe
286to separate `path`s along the same `wire`, because then you can't properly
287distinguish them for cancelling (besides not being able to distinguish
288the subscription updates). In other words, besides letting *you*
289distinguish your `card`s, the `wire` also identifies requests for the system.
291If the `note` is not to another agent but to Arvo itself, then it is a
292request to one of the vanes, which are kernel modules that provide
293various system services, including IO. Here are some examples:
296[%pass /my/wire %arvo %b %wait (add now.hid ~s10)]
299This is a request to the `%b` vane (Behn, the timer vane) to set a timer
300for 10 seconds in the future. After 10 seconds, Behn will respond with
301a `%wake` card, which you will recieve in `+on-arvo`. It will come back
305=/ =path /(scot %p our.hid)/home/(scot %da now.hid)/my-file/txt
306=/ contents=cage [%txt !>(~['text file line 1' 'line 2'])]
307[%pass /my/wire %arvo %c %info (foal path contents)]
310This is a request to the `%c` vane (Clay, the filesystem vane) to write
311a file to `/my-file/txt` in the home desk. You will not receive a
312response to this `note`.
316=/ =moat [da+now.hid da+(add now.hid ~h1) path]
317[%pass /my/wire %arvo %c %warp our.hid %home ~ %many & moat]
320This is a request to the `%c` vane to send us a `%writ` `card` whenever
321`/my-file/txt` changes in the next hour. There may be many responses to
322this note if the file changes many times.
326 [%'GET' 'https://example.com' ~ ~]
327[%pass /my/wire %arvo %i %request request *outbound-config:iris]
330This is a request to the `%i` vane (Iris, the HTTP client vane) to make
331a GET HTTP request to example.com. The response will come as an
332`%http-response` card.
336In contrast to the many possible `note`s, there are only two types of
337`gift`s that an agent may give:
340[%give %fact (unit path) =cage]
341[%give %kick (unit path)]
344A subscription update is a new piece of subscription content for all
345subscribers on a given `path`. If no `path` is given, then the update is
346only given to the program that instigated this request. Typical use of
347this mode is in `+on-watch` to give an initial update to a new
348subscriber to get them up to date.
350A subscription close closes the subscription for all subscribers on a
351given `path`. If no `path` is given, then the update is only given to the
352program that instigated this request. Typical use of this mode would be
353in `+on-watch` to produce a single update to a subscription then close
358A `vase` is a piece of dynamic data. Structurally, it's a pair of an
359explicit reification of a type and an untyped [noun](/docs/glossary/noun/). This lets us
360represent a value which has a type that isn't known at compile time. A
361vase has three operations:
363- `!>` is a unary rune that lifts a statically typed value to a
364 dynamically-typed `vase`. For example, `!>('hi')` gives `[#t/@t q=26.984]`.
366- `!<` is a binary rune that takes a mold and a dynamically typed `vase`
367 and reduces it to a statically typed value. If the `vase` does not in
368 fact have the type of the mold you give it, then it nest-fails, else
369 it produces `value`. For example, `!<(@t !>('hi'))` produces `'hi'` while `!<(^ !>('hi'))` causes a nest fail..
371- The compiler takes text and converts it to a `vase` of the compiled
372 code. Agents shouldn't need this directly, but Gall uses this to
373 compile agents to `vase`s, on which it calls `!<(agent:gall
374 compiled-agent-vase)`.
376A `cage` is simply a pair of a mark and a vase. A mark is a textual tag
377that should correspond to the particular dynamic type in the vase.
379In agents, we use vases to represent types which Gall doesn't know about
380when it was compiled, but which nevertheless need to go outside the
381agent. In practice, there are three common cases:
383- The data in a "poke" request is of a type that is defined by each
384 agent, so it must by dynamic. This is the input to `++on-poke` as well
385 as the cage in the `%poke` case of the `%agent` note.
387- The data in a subscription update is defined by each agent, so it must
388 be dynamic. This is the cage in `%fact`, both when giving the update
389 and in the input to `+on-agent`.
391- The state of an agent is also unique to each agent. Most of the time,
392 Gall doesn't interact directly with agent's state, but when upgrading
393 an agent, it must pass the state of the old version of the agent to
394 the new version of the agent. This is the output of `+on-save` and
395 the input to `+on-load`.
399In the definition of an agent, the entire core is wrapped with the `^|`
400rune, which corresponds to the "iron" variance mode. This means it's
401"contravariant" in the input, which allows Gall to write to the new bowl
402on every event. It also means that the context of the core is opaque to
403Gall, which means you can store state in your context in whatever
406This is why most of the handler arms produce not just a list of cards
407but also a new `agent:gall`. Usually, this will be the "same" agent in
408the sense that the code will be the same, but you may have changed the
409state that's stored in its (opaque to Gall) context.
411The skeleton example above gives an example of keeping an atom as your
412state. Define the state in the context of your agent core, then you can
413change it with `=.` or `%=`, as long as you produce the new version of
416When you upgrade an agent, you need to extract the state from your
417opaque context and produce it to Gall as a dynamically typed vase.
418Usually, this will be easy: just call `!>` on whatever state you wish
419to preserve. This is `+on-save`.
421When the new agent is about to be started, Gall will call it with
422`+on-load` with the vase just produced above. This allows you to ingest
423your old state and continue right where you left off.
425If the type of your state hasn't changed, you can just
428=. state !<(state-type old-state-vase)
431If it has changed, then it should look more like:
434=. state (upgrade-state !<(old-state-type old-state-vase))
437It's useful to tag your state with a version number so that the
438`+upgrade-state` function can take a tagged union of all your old state
439types and upgrade from any of them to the current state type. For
440example, it may have the following structure:
449 |= old=old-state-types
460The core takes as input a `bowl`, which includes useful info like the
461current ship, the current time, and a renewable source of entropy. This
462information is available to any of the handlers.
466A description of each of the handler arms follows.
470This arm is called once when the agent is started. It has no input and
471lets you perform any initial IO.
475This arm is called immediately before the agent is upgraded. It
476packages the permament state of the agent in a `vase` for the next version
477of the agent. Unlike most handlers, this cannot produce effects.
481This arm is called immediately after the agent is upgraded. It receives
482a `vase` of the state of the previously-running version of the agent,
483which allows it to cleanly upgrade from the old agent.
487This arm is called when the agent is "poked". The input is a `cage`, so
488it's a pair of a mark and a dynamic `vase`.
492This arm is called when a program wants to subscribe to the agent on a
493particular `path`. The agent may or may not need to perform setup steps
494to intialize the subscription. It may produce a `%give`
495`%subscription-result` to the subscriber to get it up to date, but after
496this event is complete, it cannot give further updates to a specific
497subscriber. It must give all further updates to all subscribers on a
500If this arm crashes, then the subscription is immediately terminated.
501More specifcally, it never started -- the subscriber will receive a
502negative `%watch-ack`. You may also produce an explicit `%kick` to
503close the subscription without crashing -- for example, you could
504produce a single update followed by a `%kick`.
508This arm is called when a program becomes unsubscribed to you.
509Subscriptions may close because the subscriber intentionally
510unsubscribed, but they also could be closed by an intermediary. For
511example, if a subscription is from another ship which is currently
512unreachable, Ames may choose to close the subscription to avoid queueing
513updates indefinitely. If the program crashes while processing an
514update, this may also generate an unsubscription. You should consider
515subscriptions to be closable at any time.
519This arm is called when a program reads from the agent's "scry"
520namespace, which should be referentially transparent. Unlike most
521handlers, this cannot perform IO, and it cannot change the state. All
522it can do is produce a piece of data to the caller, or not.
524If this arm produces `[~ ~ data]`, then `data` is the value at the the
525given `path`. If it produces `[~ ~]`, then there is no data at the given
526`path` and never will be. If it produces `~`, then we don't know yet
527whether there is or will be data at the given `path`.
531This arm is called to handle responses to `%give` moves to other agents.
532It will be one of the following types of response:
534- `%poke-ack`: acknowledgment (positive or negative) of a poke. If the
535 value is `~`, then the poke succeeded. If the value is `[~ tang]`,
536 then the poke failed, and a printable explanation (e.g. a stack trace)
537 is given in the `tang`.
539- `%watch-ack`: acknowledgment (positive or negative) of a subscription.
540 If negative, the subscription is already ended (technically, it never
543- `%fact`: update from the publisher.
545- `%kick`: notification that the subscription has ended.
549This arm is called to handle responses to `%pass` `move`s to vanes. The
550list of possible responses from the system is statically defined in
551sys/zuse.hoon (grep for `+ sign-arvo`).
555If an error happens in `+on-poke`, the crash report goes into the
556`%poke-ack` response. Similarly, if an error happens in
557`+on-subscription`, the crash report goes into the `%watch-ack`
558response. If a crash happens in any of the other handlers, the report
559is passed into this arm.
keysesc to close
- j k
- move down, up
- enter
- open
- esc
- back, or up a level
- 1 2 3 4
- readme, files, commits, refs
- /
- filter
- y
- copy the clone command
- c
- compare the chosen commit with another (or shift-click both)
- ?
- show or hide this list