@@ -4,2315 +4,563 @@ weight = 6
4template = "doc.html"
4template = "doc.html"
5aliases = ["/docs/learn/arvo/gall/"]
5aliases = ["/docs/learn/arvo/gall/"]
7−This is documentation about how to write Gall apps. Some knowledge of Hoon is assumed.
9−> Note: the last major revision of these lessons was in 2016. They need to be
8+This document describes Gall, the Arvo userspace vane. It is accurate as of 2019.12.02.
10− rewritten; in some ways they are outdated, and in other ways they were not
9+
11− always accessible enough to beginners. We are planning to rewrite these in the
10+# Agents
12− near future. There is still useful information to be found in the meantime.
11+
13−
12+Urbit code lives in the following basic categories:
15−
14+- Runtime (Nock interpreter, persistence engine, IO drivers, jets)
16−> Note: This lesson depends on some code that needs to be updated before the examples will work correctly. This will be addressed in the near future. We apologize!
15+- Kernel vanes (managed by Arvo)
17−
16+- Userspace agents (managed by Gall, permanent state)
18−Most people have lots of data stored in online services, many of
17+- Userspace imps (managed by Spider, transient state)
19−which have APIs. API connectors allow the user to access this
18+
20−data from within their Urbit.
19+This lays out the framework for the third category: Agents.
22−A security driver allows the user to make authenticated requests
21+An agent is a piece of software that is primarily focused on maintaining
23−to the service's API, but the user still needs to know the API
22+and distributing a piece of state with a defined structure. It exposes
24−and send and receive JSON. An API connector puts a layer of
23+an interface that lets programs read, subscribe to, and manipulate the
25−porcelain over this to allow the user easier control over their
24+state. Every event happens in an atomic transaction, so the state is
26−data.
25+never inconsistent. Since the state is permanent, when the agent is
27−
26+upgraded with a change to the structure of the state, the developer
28−Each connector should perform two basic functions. First, it
27+provides a migration function from the old state type to the new state
29−should expose a tree of the data in the service that's accessible
28+type.
30−through `.^` and FUSE. Second, it should expose event streams as
29+
31−actions occur in the service. Let's take a look at how the `%gh`
30+It's not too far off to think of an agent as simply a database with
32−app accomplishes both for Github.
31+developer-defined logic. But an agent is significantly less constrained
33−
32+than a database. Databases are usually tightly constrained in one or
34−After starting `%gh` (`|start %gh`), let's look at the root of
33+more ways because they need to provide certain guarantees (like
35−the tree that `%gh` exposes:
34+atomicity) or optimizations (like indexes). Urbit is a single-level
36−
35+store, so atomicity comes for free. Many applications don't use
37−```
36+databases because they need relational indices; rather, they use them
38−~your-urbit:dojo> .^(arch %gy /=gh=)
37+for their guarantees around persistence. Some do need the indices,
39−[fil=~ dir={[p=~.issues q=~]}]
38+though, and it's not hard to imagine an agent which provides a SQL-like
42−`%gh` is currently a skeleton -- it contains examples of all the
41+On the other hand, an agent is also a lot like what many systems call a
43−functionality necessary for a connector, but many endpoints
42+"service". An agent is permanent and addressable -- a running program
44−aren't implemented. As we can see here, only issues are
43+can talk to an agent just by naming it. An agent can perform IO, unlike
45−implemented. We can explore this tree:
44+most databases. This is a critical part of an agent: it performs IO
46−
45+along the same transaction boundaries as changes to its state, so if an
47−```
46+effect happens, you know that the associated state change has happened.
48−~your-urbit:dojo> .^(arch %gy /=gh=/issues)
47+You should be careful, though, to avoid creating implicit non-atomicity
49−[fil=~ dir={[p=~.by-repo q=~] [p=~.mine q=~]}]
48+-- eg if it comes into an inconsistent state pending a response to an IO
52−And eventually:
51+But the best way to think about an agent is as a state machine. Like a
53−
52+state machine, any input could happen at any time, and it must react
54−```
53+coherently to that input. Ouput (effects) and the next state of the
55−/- gh
54+machine are both pure functions of the previous state and the input
56−.^(issue:gh %gx /=gh=/issues/by-repo/philipcmonktest/testing/7)
55+event. Of course, it's important to ensure there isn't an order of
57−<issue...>
56+events that could cause your agent to enter an inconsistent state.
58−login.user:.^(issue:gh %gx /=gh=/issues/by-repo/philipcmonktest/testing/7)
57+
59−'philipcmonktest'
58+We often think of state machines as finite state machines, but of course
60−```
59+in this instance, the state is not actually finite (though it should be
61−
60+definable in a regular recursive data type).
62−The other thing an API connector needs to do is expose event
61+
63−streams. One straightforward way to use this capacity is with
62+## Specification
65−
64+An agent is defined as a core with a set of arms to handle various
66−```
65+events. These handlers usually produce a list of effects and the next
67−:pipe|connect %gh /listen/philipcmonktest/testing/issues/'issue_comment' %public
66+state of the agent. The interface definition can be found in
68−```
67+`sys/zuse.hoon`, which at the time of writing is:
70−Now creating an issue results in the following message in your
71−`%public` `:talk` channel:
74−~your-urbit[philipcmonktest@github]: opened issue ##11: i found a bug!
77−You'll also get notifications of issue status changes and
80−As is often the case, the best way to write an API connector is
81−to copy an existing one and modify it. We'll dissect the Github
82−connector here. This code is in `/=home=/app/gh/hoon`, and it's
83−also reproduced at the bottom of this page (in case the code
84−drifts out of sync with this doc).
86−We'll go over the two parts to an API connector (one-time reading vs listening
87−for events) separately.
91−A connector exposes a tree of data. Every read request has a
92−`care`, which is generally either `%x` or `%y`. `%x` is a request
93−for a particular piece of data, while `%y` is a request for a
94−directory listing. Roughly, `%x` means Unix `cat` and `%y` means
97−> Sometimes you wish to expose a tree where a part of the path can't be enumerated. For example, a `%y` of `/issues/by-repo` "should" produce a list of all Github users, but we don't do that because it's too long. Instead, we just produce our own username (from the `web.plan` file). You can still access repos from other users, you just don't see them in the directory listing.
99−The usual flow for implementing this tree of data makes heavy use
100−of the `connector` library. This library is well documented in
101−the source, so check out `/=home=/lib/connector/hoon`.
103−The `connector` library needs to be initialized with definitions
104−of `move` and `sub-result` (all the types of data that can be
105−returned by a place). This can just be a line at the top of the
109−=+ connector=(connector move sub-result) :: Set up connector library
112−Most of the Github-specific logic is in `++places`, which is a
113−list of all the places we can request. A place consists of:
117− $: guard/mold
71+ =< form
118− read-x/$-(path move)
119− read-y/$-(path move)
120− sigh-x/$-(jon/json (unit sub-result))
121− sigh-y/$-(jon/json (unit arch))
125−- `guard`, the type of the paths we should match. For example, to match `/issues/<user>/<repo>` use `{$issues @t @t $~}`.
127−- `read-x`, called when someone tries to read the place with `care` `%x`. Should produce a single move, usually either a `%diff` response if we can immediately answer or a `%hiss` http request if we need to make a request to the api. See the `++read-*` functions in `++helpers` for some common handlers.
129−- `read-y`, same as `read-x` except with care `%y`.
131−- `sigh-x`, called when an http response comes back on this place. You're given the json of the result, and you should produce either a result or null. Null represents an error. If you didn't create an http request in `read-x`, then this should never be called. Use `++sigh-strange` from `++helpers` to unconditionally signal an error.
133−- `sigh-y`, same as `sigh-x` except with care `%y`. Note that a `%y` request must produce an arch, unlike a `%x` request, which may produce data of any mark.
135−Filling out the list of places is a lot of grunt work, but most
136−places are fairly straightforward, and the `++read-*` helper
137−functions are useful. Check out the library source for more
138−information on those.
140−Besides the list of places, we just need to handle the flow of
141−control. `++peek` and `++peer-scry` are the interface we expose
142−to the rest of the system while `++sigh-httr` and `++sigh-tang`
143−are used to handle responses when we make HTTP requests.
145−- `++peek` should usually just produce `~`. If there are cases where we can respond directly to a `.^` request without blocking on anything, we could do it here, but it's generally not worth the hassle since the same logic should be duplicated in `++peer-scry`.
147−- `++peer-scry` is where the actual handling for a read request goes. In general, we just need to call `++read` from the `connector` library with the bone, list of places, care, and path. This will match the path to the appropriate place and run either `++read-x` or `++read-y`, depending on the care.
149−- If `++read-x` or `++read-y` made a successful API request, then the response will come back on `++sigh-httr`. Here, we just need to parse out the wire and call `++sigh` from the `connector` library with the list of places, care, path, and HTTP result. This will match the path to the appropriate place and run either `++sigh-x` or `++sigh-y`, depending on the care.
151−- If `++read-x` or `++read-y` made an API request that failed, then we'll get a stack trace in `++sigh-tang`. Here, we just print it out and move on.
153−That's really all there is to the reading portion of API
156−One of the most accessible ways to jump into Arvo programming is
157−to just add more places to an existing API connector. It's
158−useful, small in scope, and comes in bite-sized chunks since most
159−places are less than ten lines of code.
163−Listening for events is fairly service-specific. In some
164−services, we poll for changes. The Twitter connector has an
165−example of this, but note that it predates the `connector`
166−library and is thus more complicated than it needs to be, and the
167−interface it exposes isn't standard. In Github, we power our
168−event streams with webhooks.
170−For Github, when someone subscribes to
171−`/listen/<user>/<repo>/<events...>`, we want to produce
172−well-typed results when they occur.
174−We only want to create one webhook per event, so in our state we
175−have `hook`, which is a map of event names to the set of bones
176−which are subscribed to that event. We must always keep this
179−Flow of control starts in `++peer-listen`, where we just call
180−`++listen`. In `++listen`, for each event in the list of events,
181−we check to see if we have that hook set up (by checking whether
182−it exists in `hook`). If so, we call `++update-hook` to add the
183−current bone to the set of listeners. Otherwise, we call
184−`++create-hook`, which sends a request to Github to set up the
185−new webhook. We also create an entry in `hook` with the current
188−When we created the webhook, we told Github to send the event to
189−`/~/to/gh/gh-<event>.json?anon&wire=/`. This turns into a poke.
190−Let's parse out this url. The first `gh` is the app name to
191−poke. The next portion, `gh-<event>` is the mark to convert the
192−JSON to. For example, when the "issues" event fires, it'll get
193−converted to mark `gh-issues`. This mark has a conversion
194−routine from `json` to the `issues` type, which is defined in the
195−`gh` structure (`/=home=/sur/gh/hoon`). Once this conversion
196−happens, the `gh` app will get poked with the correctly marked
199−In `++poke`, we check `hook` to find the subscribers to the event
200−that was fired, and we update them.
202−That's really all there is to it. Webhook flow isn't well
203−standardized, so even if your chosen service is powered by
204−webhooks, your listening code might look rather different. The
205−point is to expose the correct interface.
208−#### Github API Connector Code
210−In case the code in `/=home=/app/gh/hoon` drifts out of sync with
214−:: This is a connector for the Github API v3.
216−:: You can interact with this in a few different ways:
218−:: - .^({type} %gx /=gh={/endpoint}) to read data or
219−:: .^(arch %gy /=gh={/endpoint}) to explore the possible
222−:: - subscribe to /listen/{owner}/{repo}/{events...} for
223−:: webhook-powered event notifications. For event list, see
224−:: https://developer.github.com/webhooks/.
226−:: This is written with the standard structure for api
227−:: connectors, as described in lib/connector.hoon.
231−/+ gh-parse, connector
235− +$ move (pair bone card)
73+ +$ step (quip card form)
236− +$ card
74+ +$ card (wind note gift)
237− $% [%diff sub-result]
75+ +$ note
238− [%them wire (unit hiss:eyre)]
76+ $% [%arvo =note-arvo]
239− [%hiss wire [%~ %~] %httr [%hiss hiss:eyre]]
77+ [%agent [=ship name=term] =task]
81+ [%watch-as =mark =path]
84+ [%poke-as =mark =cage]
87+ $% [%fact path=(unit path) =cage]
88+ [%kick path=(unit path) ship=(unit ship)]
89+ [%watch-ack p=(unit tang)]
90+ [%poke-ack p=(unit tang)]
93+ $% [%poke-ack p=(unit tang)]
94+ [%watch-ack p=(unit tang)]
102+ *(quip card _^|(..on-init))
242− :: Types of results we produce to subscribers.
107+ ++ on-load
109+ *(quip card _^|(..on-init))
244− +$ sub-result
111+ ++ on-poke
245− $% [%arch arch]
112+ |~ [mark vase]
246− [%gh-issue issue:gh]
113+ *(quip card _^|(..on-init))
247− [%gh-list-issues (list issue:gh)]
248− [%gh-issues issues:gh]
249− [%gh-issue-comment issue-comment:gh]
254− :: Types of webhooks we expect.
115+ ++ on-watch
117+ *(quip card _^|(..on-init))
256− +$ hook-response
119+ ++ on-leave
257− $% [%gh-issues issues:gh]
120+ |~ path
258− [%gh-issue-comment issue-comment:gh]
121+ *(quip card _^|(..on-init))
129+ *(quip card _^|(..on-init))
133+ *(quip card _^|(..on-init))
137+ *(quip card _^|(..on-init))
261−=+ connector=(connector move sub-result) :: Set up connector library
264− hook=(map @t [id=@t listeners=(set bone)]) :: map events to listeners
266−:: ++ prep _`. :: Clear state when code changes
272− ^- (list place:connector)
273− =+ (helpers:connector ost.hid wir "https://api.github.com")
274− => |% :: gh-specific helpers
275− ++ sigh-list-issues-x
277− %+ bind ((ar:jo issue:gh-parse) jon)
278− |= issues=(list issue:gh)
279− gh-list-issues+issues
281− ++ sigh-list-issues-y
283− %+ bind ((ar:jo issue:gh-parse) jon)
284− |= issues=(list issue:gh)
285− :- `(shax (jam issues))
286− (malt (turn issues |=(issue:gh [(rsh 3 2 (scot %ui number)) ~])))
291− read-y=(read-static %issues ~)
296− :* guard=,[%issues %~]
298− read-y=(read-static %mine %by-repo ~)
302− ^- place :: /issues/mine
303− :* guard=,[%issues %mine %~]
304− read-x=(read-get /issues)
305− read-y=(read-get /issues)
306− sigh-x=sigh-list-issues-x
307− sigh-y=sigh-list-issues-y
309− ^- place :: /issues/by-repo
310− :* guard=,[%issues %by-repo %~]
314− =+ /(scot %p our.hid)/home/(scot %da now.hid)/web/plan
315− =+ .^([* acc=(map knot plan-acct)] %cx -)
317− ((read-static usr:(~(got by acc) %github) ~) pax)
321− ^- place :: /issues/by-repo/<user>
322− :* guard=,[%issues %by-repo @t %~]
324− read-y=|=(pax=path (get /users/[-.+>.pax]/repos))
328− %+ bind ((ar:jo repository:gh-parse) jon)
329− |= repos=(list repository:gh)
330− [~ (malt (turn repos |=(repository:gh [name ~])))]
332− ^- place :: /issues/by-repo/<user>/<repo>
333− :* guard=,[%issues %by-repo @t @t %~]
334− read-x=|=(pax=path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
335− read-y=|=(pax=path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
336− sigh-x=sigh-list-issues-x
337− sigh-y=sigh-list-issues-y
339− ^- place :: /issues/by-repo/<user>/<repo>
340− :* guard=,[%issues %by-repo @t @t @t %~]
342− |=(pax=path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues/[-.+>+>.pax]))
344− read-y=(read-static ~)
347− %+ bind (issue:gh-parse jon)
355−:: When a peek on a path blocks, ford turns it into a peer on
356−:: /scry/{care}/{path}. You can also just peer to this
359−:: We hand control to ++scry.
363− ^- [(list move) _+>.$]
364− ?> ?=([care:clay *] pax)
366− (read:connector ost.hid (places %read pax) i.pax t.pax)
368−:: HTTP response. We make sure the response is good, then
369−:: produce the result (as JSON) to whoever sent the request.
372− |= [way=wire res=httr:eyre]
373− ^- [(list move) _+>.$]
374− ?. ?=([%read care:clay @ *] way)
382− (sigh:connector (places ren style pax) ren pax res)
384−:: HTTP error. We just print it out, though maybe we should
385−:: also produce a result so that the request doesn't hang?
388− |= [way=wire tan=tang]
389− ^- [(list move) _+>.$]
390− %- (slog >%gh-sigh-tang< tan)
391− [[ost.hid %diff null+~]~ +>.$]
393−:: We can't actually give the response to pretty much anything
394−:: without blocking, so we just block unconditionally.
397− |= [ren=@tas tyl=path]
398− ^- (unit (unit (pair mark *)))
399− ~ ::``noun=[ren tyl]
401−::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
402−:: Webhook-powered event streams (/listen) ::
403−::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
405−:: To listen to a webhook-powered stream of events, subscribe
406−:: to /listen/<user>/<repo>/<events...>
408−:: We hand control to ++listen.
412− ^- [(list move) _+>.$]
414− ~& [%bad-listen-path pax]
418−:: This core handles event subscription requests by starting or
419−:: updating the webhook flow for each event.
426− ++ abet :: Resolve core.
427− ^- [(list move) _+>.$]
430− ++ send-hiss :: Send a hiss
433− =+ wir=`wire`[%x %listen pax]
434− +>.$(mow [[ost.hid %hiss wir `~ %httr [%hiss hiz]] mow])
436− :: Create or update a webhook to listen for a set of events.
440− =+ pax=pax :: TMI-proofing
446− ?: (~(has by hook) i.events)
447− $(+>+ (update-hook i.events), events t.events)
448− $(+>+ (create-hook i.events), events t.events)
450− :: Set up a webhook.
456− =+ clean-event=`tape`(turn (trip event) |=(a=@tD ?:(=('_' a) '-' a)))
458− %+ ~(put by hook) (crip clean-event)
460− (~(get by hook) (crip clean-event))
461− *[id=@t listeners=(set bone)]
462− [id (~(put in listeners) ost.hid)]
465− =+ [(trip i.pax) (trip i.t.pax)]
466− "https://api.github.com/repos/{-<}/{->}/hooks"
469− %- as-octs:mimes:html %- crip %- en-json:html %- pairs:enjs:format :~
472− events+a+~[s+event] ::(turn `(list ,@t)`t.t.pax |=(a=@t s=a))
474− %- pairs:enjs:format :~
476− "http://107.170.195.5:8443/~/to/gh/gh-{-}.json?anon&wire=/"
478− [%'content_type' s+%json]
483− :: Add current bone to the list of subscribers for this event.
488− =+ hok=(~(got by hook) event)
491− %+ ~(put by hook) event
492− hok(listeners (~(put in listeners.hok) ost.hid))
496−:: Pokes that aren't caught in more specific arms are handled
497−:: here. These should be only from webhooks firing, so if we
498−:: get any mark that we shouldn't get from a webhook, we reject
499−:: it. Otherwise, we spam out the event to everyone who's
500−:: listening for that event.
503− |= response=hook-response
504− ^- [(list move) _+>.$]
505− =+ hook-data=(~(get by hook) (rsh 3 3 -.response))
507− ~& [%strange-hook hook response]
509− :: ~& response=response
511− %+ turn ~(tap in listeners.u.hook-data)
513− [ost %diff response]
517−## Generators
141+Here's a skeleton example of an implementation:
519−Up until now we've poked apps directly. This requires the user to
520−specify the mark, and it requires the app to accept the arguments in a
521−way that's convenient for users to input. This is the "plumbing" way to
522−interact with apps. Generators are the "porcelain" layer. This is why
523−when you run a command like `+ls` or `|merge`, there are no marks in
526−At that point, we just used the
527−generators to produce values -- we didn't pipe their results into apps.
528−In the dojo cast, the role of a generator is to take a list of arguments
529−and produce a value, which is often, though not always, piped into an
530−app. Generators are pure, stateless functions, and they cannot send
533−`+ls` and `+cat` are commonly-used generators that produce a directory
534−listing and print out a file, respectively. These are useful in
535−themselves, so we generally don't pipe the results into an app.
537−Another generator is `+hood/merge`. Try running `+hood/merge` with the
538−normal arguments for `|merge`:
541−> +hood/merge %home our %they
542−[syd=%home her=~zod sud=%they cas=[%da p=~2018.10.4..21.11.49..901c] gem=%auto]
544−> +hood/merge %home our %they, =gem %this
545−[syd=%home her=~zod sud=%they cas=[%da p=~2018.10.4..21.12.11..ee32] gem=%this]
548−This didn't run any merge, it just constructed the command that, if sent
549−to `:hood`, would run it. Note first that, even though it's not printed,
550−this value has mark `kiln-merge`. Also, the merge strategy `gem=%auto`
551−was added automatically. Optional arguments are straightforward with
554−In general, generators are prepended by either `+` or `|`. The general
555−form is `+generator`, but often generators are created specifically to
556−work with a single app, they're usually placed in a subdirectory of gen
557−and run with `|`. Thus, if `my-generator` generator is made for use with
558−`:my-app`, then `my-generator` is put in
559−`/gen/my-app/my-generator.hoon`, and can be run directly with
560−`+my-app/my-generator <args>`. If you want to pipe the result into
561−`:my-app`, then run `:my-app +my-app/my-generator <args>`. Because this
562−pattern is so common, this can be abbreviated to
563−`:my-app|my-generator <args>`. Because most built-in commands are
564−generators for the `:hood` app, `:hood|generator <args>` can be
565−shortened to `|generator <args>`.
567−Let's write a generator called `:pong` that takes an urbit address, which is of
568−mark `urbit`, and sends that urbit the message `'howdy'`. First--without a
569−generator--let's make `:ping` that does the same, except that
570−it lets the user optionally specify the message as well.
572−We'll need a new mark for our arguments. Let's call it
575−> For app-specific marks, it's good style to prefix the name of the mark with the name of the app. Since many apps have several such marks, subdirectories in `/mar` are rendered as `-`, so that `ping-message` is written in (/mar/ping/message.hoon).
579−:::: /mar/ping/message/hoon
145+=| state=@
582−|_ [to=@p message=@t]
148+++ on-init
585− +$ noun [@p @t]
151+++ on-save
156+ =/ old-state !<(@ old-state-vase)
161+ `this(state u.old-state)
166+ ~& got-poked-with-data=mark]
183+ |= [wire sign:agent:gall]
590−The app can easily be modified to use this (`/app/ping.hoon`):
196+We also supply a `default-agent` library, which is useful if you don't
197+want to handle some of the arms. Most of the above could also be
593−:: Allows one ship to ping another with a string of text
201+/+ default-agent
595−:::: /app/ping/hoon
203+=| state=@
598−|%
206+ default ~(. (default-agent %|) bowl)
599− +$ move [bone term wire *]
207+::
208+++ on-init on-init:default
209+++ on-save on-save:default
210+++ on-load on-load:default
214+ ~& got-poked-with-data=mark
218+++ on-watch on-watch:default
219+++ on-leave on-leave:default
220+++ on-peek on-peek:default
221+++ on-agent on-agent:default
222+++ on-arvo on-arvo:default
223+++ on-fail on-fail:default
602−|_ [bowl:gall state=%~]
605− |= [to=@p message=@t]
607− ^- [(list move) _+>.$]
609− :~ `move`[ost %poke /sending [to dap] %atom message]
616− ^- [(list move) _+>.$]
618− ~& [%receiving (@t arg)]
625−Now we can run this with:
628−~fintud-macrep:dojo> |start %ping
631−~fintud-macrep:dojo> :ping &ping-message [~sampel-sipnym 'heyya']
227+So, an agent is a core with 10 arms. The handlers correspond to
632−>=
228+different sorts of input. We'll discuss each of these in detail, but
633−```
229+first a few general concepts.
635−And on `~sampel-sipnym`, assuming it's running `:ping` as well,
636−we see `[%receiving 'heyya']`.
638−This is an annoying way to invoke the app, though. The `[]` are
639−mandatory, and optional arguments are hard. Let's make a generator,
640−`+send` to make everything nicer. Since it's specific to
641−`:ping`, let's put it in `/gen/ping/send.hoon`:
645−|= [^ [to=@p message=?(%~ [text=@t %~])] %~]
646−[%ping-message to ?~(message 'howdy' text.message)]
649−A couple of new things here. Firstly, `message=?(%~ [text=@t %~])] %~]`
650−should be read as "the message is either null or a pair of text and
651−null". Generator argument lists are always null-terminated, which makes
652−it convenient to accept lists in tail position (which are particularly
653−annoying without generators). `?(a b)` is the irregular form of
654−`$?(a b)`, which is a union type. Thus, `?(a b)` means the type of
655−anything that's in either `a` or `b`. Thus, in our case,
656−`?(~ [text=@t ~])` is either null or a pair of text and null.
658−This is run as follows:
231+## Basic concepts
660−```
233+### Interacting with an agent
661−~fintud-macrep:dojo> :ping|send ~sampel-sipnym
664−~fintud-macrep:dojo> :ping|send ~sampel-sipnym 'how do you do'
668−Which causes `~sampel-sipnym` to print `[%receiving 'howdy']` and
669−`[%receiving 'how do you do']`.
671−(Note: `:ping|send ~sampel-sipnym` is short for `:ping +ping/send ~sampel-sipnym`.)
235+There are two basic ways programs can interact with an agent: you may
236+subscribe to data as described above, or you may "poke" an agent to send
237+it a command. Suppose there is an agent for a calendar service; then,
238+pokes would likely include `%create-event`, `%modify-event`,
239+`%delete-event`, and `%change-time-zone` (along with assocated data).
240+Subscription paths could include `/next-event` and `/all-events`.
673−**Exercises**:
242+Agents generally conform to CQRS -- pokes may change state, but
243+subscriptions generally shouldn't. In practice, there may be state
244+changes required to properly initialize the subscription, but these
245+shouldn't change the essential state of the agent.
675−- Create a generator for `:sum` from [State](#state) so that you can run `:sum|add 5` to add numbers to it.
247+### Cards
677−- Create a generator for `:click` from [Web Apps](#backend) so that you can run `:click|poke` to poke it.
249+Most of the handlers produce a `(quip card agent:gall)`, which is just
250+`[(list card) agent:gall]`. The first allows us to produce effects, and
251+the second allows us to maintain state.
253+A card is one of two things: a `note` or a `gift`. You `%pass` `note`s
254+and you `%give` `gift`s. When you pass a `note`, you expect zero or more
255+responses; when you give a `gift`, you will not get a response. Since you
256+may get a response to a `note`, you tag it with a `wire` so that you can
257+identify the response.
680−## Writing an HTTP Request
259+Thus, we say you "pass a `note` along a `wire`", or you "give a `gift`".
681−
260+These phrases correspond to producing a card that looks like:
682−There's a variety of arvo services we haven't touched on yet.
683−Let's figure out how to make HTTP requests and set timers.
685−Let's build an uptime-monitoring system. It'll ping a web
686−service periodically and print out an error when the response
687−code isn't 2xx. Save the following as `app/up.hoon`:
690−:: Up-ness monitor. Accepts atom url, 'on', or 'off'
263+[%pass /my/wire a-note]
698− $% [%hiss wire unit-iden=$~ mark=%httr cage=[mark=%purl vase=purl:eyre]]
702− $% [%on $~] :: enable polling('on')
703− [%off $~] :: disable polling('off)
704− [%target p=cord] :: set poll target('http://...')
707−|_ [hid=bowl:gall on=_| in-progress=_| target=@t]
709− |= url-or-command=@t ^- (quip move _+>)
711− ?: ?=($on url-or-command) [%on ~]
712− ?: ?=($off url-or-command) [%off ~]
713− [%target url-or-command]
715− $target [~ +>.$(target p.act)]
718− :- ?: |(on in-progress) ~
719− [ost.hid %hiss /request ~ %httr %purl (need (de-purl:html target))]~
720− +>.$(on &, in-progress &)
723− |= [wir=wire code=@ud headers=mess:eyre body=(unit octs)]
725− ^- [(list move) _+>.$]
726− ?: &((gte code 200) (lth code 300))
727− ~& [%all-is-well code]
729− [ost.hid %wait /timer (add ~s10 now.hid)]~
730− ~& [%we-have-a-problem code]
731− ~& [%headers headers]
734− [ost.hid %wait /timer (add ~s10 now.hid)]~
736− |= [wir=wire $~] ^- (quip move _+>)
739− [ost.hid %hiss /request ~ %httr %purl (need (de-purl:html target))]~
740− [~ +>.$(in-progress |)]
742−++ prep ~& target _`. :: computed when the source file changes;
743−-- :: here it prints target then resets our state
746−There's some fancy stuff going on here. We'll go through it line by
266+or
749−There's two kinds of cards we're sending to arvo. A `%hiss` move tells
750−`%eyre` to make an HTTP request. It expects a `(unit iden)`, which will
751−be null here because we aren't doing any authentication; a mark, in this
752−case the http request mark `&httr`; and some data in the form of a
753−`cage`, which is a `[mark vase]`. This is confusing, so the mold in the
754−code quoted above includes a bunch of otherwise unnecessary faces for
755−the purpose of illustration. Compare: `[%hiss p=(unit user) q=mark
756−r=cage]` from `zuse`.
758−> You can grep zuse.hoon and arvo.hoon for most of these definitions. However, beware the ambiguity surrounding "hiss": there's ++hiss in section 3bI of `zuse` ("Arvo structures"), and there's `%hiss` the move that is sent to %eyre to make our HTTP request happen. Here we're talking about the latter. See `++task:able:eyre` in `zuse`.
760−> Recall that `++unit` means "maybe". Formally, "`(unit a)` is either null or a pair of null and a value in `a`". Also recall that to pull a value out of a unit (`u.unit`), you must first verify that the unit is not null (for example with `?~`). See `++need`.
762−A `purl:eyre` is a parsed url structure, which can be created with `de-purl:html`,
763−which is a function that takes a url as text and parses it into a `(unit
764−purl:eyre)`. Thus, the result is null if and only if the url is malformed.
765−'Purl' is also the mark which will be applied to this result.
767−When you send this request, you can expect a `%sigh` with the
768−response, which we handle later on in `++sigh-httr`.
770−For `%wait`, you just pass a `@da` (absolute date), and arvo will
771−produce a `%wake` when the time comes.
773−> A timer is guaranteed to not be triggered before the given time, but it's currently impossible to guarantee the timer will be triggered at exactly the requested time.
775−Let's take a look at our state:
778−|_ [hid=bowl:gall on=_| in-progress=_| target=@t]
269+[%give a-gift]
781−We have three pieces of app-specific state. `target` is the url
272+When you give a `gift`, that's the end of the story, but when you pass a
782−we're monitoring. `on` and `in-progress` are booleans
273+`note`, you may get a response. If you do, then the response will come
783−representing, respectively, whether or not we're supposed to keep
274+tagged with the `wire` that you used to pass the `note`. It's generally
784−monitoring and whether or not we're in the middle of a request.
275+good practice to make your `wire`s fairly unique within your agent, since
785−
276+otherwise you may not be able to distinguish the responses.
786−The type of booleans is usually written as `?`, which is a union
787−of `&` (true) and `|` false. This type defaults to `&` (true). In our
788−case, we want both of these to default to `|` (false). Because
789−of that, we use `_|` as the type. `_value` means "take the type
790−of `value`, but make the default value be `value`". Thus, our
791−type is still a boolean, just like `?`, but the default value is
794−> This is the same `_` used in `_+>.$`, which means "the same type as the value "+>.$". In other words, the same type as our current context and state.
278+#### Notes
796−Let's take a look at `++poke-atom`. When we're poked with an atom, we
280+An agent may pass `note`s to either Arvo or another agent. If the `note` is
797−first check whether the atom is `'off'`. If so, we set our state
281+to another agent, then it should usually be one of these:
798−variable `on` to false.
800−If not, we check whether the atom is `'on'`. If so, we set `on` and
801−`in-progress` to true. If it was already either on or in progress, then
802−we don't take any other immediate action. If it was both off and not in
803−progress, then we send an HTTP request.
805−If the argument is neither `'off'` nor `'on'`, then we assume it's an
806−actual url, so we save it in `target`.
808−Here's the move that sends the HTTP request:
811−[ost.hid %hiss /request ~ %httr %purl (need (de-purl:html target))]
284+[%pass /my/wire %agent our.bowl agent-name %watch /a/path]
285+[%pass /my/wire %agent our.bowl agent-name %leave ~]
286+[%pass /my/wire %agent our.bowl agent-name %poke %foo-mark !>(poke-data)]
814−> Remember, we are expected to produce a **list** of moves. Note the `~` after the move in the full example. This is a convenient shortcut for creating a list of a single element. It's part of a small family of such shortcuts. `~[a b c]` is `[a b c ~]`, `[a b c]~` is `[[a b c] ~]` and `` `[a b c] `` is `[~ a b c]`. These may be mixed and matched to create various convoluted structures and emojis.
289+Note that to unsubscribe to a `path`, you must send the unsubscription on
290+the same `wire` that sent for the original subscription. Don't subscribe
291+to separate `path`s along the same `wire`, because then you can't properly
292+distinguish them for cancelling (besides not being able to distinguish
293+the subscription updates). In other words, besides letting *you*
294+distinguish your `card`s, the `wire` also identifies requests for the system.
816−The correspondence between this move and `[bone card]` can be hard to
296+If the `note` is not to another agent but to Arvo itself, then it is a
817−visualize on one line. Here it is more pedantically:
297+request to one of the vanes, which are kernel modules that provide
298+various system services, including IO. Here are some examples:
820−:* bone=ost.hid :: the move
301+[%pass /my/wire %arvo %b %wait (add now.hid ~s10)]
825− cage=[%purl (need (de-purl:html target))]
829−When the HTTP response comes back, we handle it with
830−`++sigh-httr`, which, along with the wire the request was sent
831−on, takes the status code, the response headers, and the response
834−We check whether the status code is between 200 and 300. If so,
835−all is well, and we print a message saying so. We start the
836−timer for ten seconds after the present time.
838−If we got a bad status code, then we print out the entire
839−response and start the timer again.
841−After ten seconds, arvo will give us a `%wake` event, which will
842−be handled in `++wake-timer`. If we're still supposed to keep
843−monitoring, we send the same HTTP request as before. Otherwise,
844−we set `in-progress` to false.
849−~fintud-macrep:dojo> |start %up
851−~fintud-macrep:dojo> :up &atom 'http://www.google.com'
855−~fintud-macrep:dojo> :up &atom 'http://example.com'
858−~fintud-macrep:dojo> :up &atom 'http://google.com'
860−[%we-have-a-problem 301]
863− [p='X-Frame-Options' q='SAMEORIGIN']
864− [p='X-XSS-Protection' q='1; mode=block']
865− [p='Content-Length' q='219']
866− [p='Server' q='gws']
867− [p='Cache-Control' q='public, max-age=2592000']
868− [p='Expires' q='Sun, 10 Jan 2016 23:45:05 GMT']
869− [p='Date' q='Fri, 11 Dec 2015 23:45:05 GMT']
870− [p='Content-Type' q='text/html; charset=UTF-8']
871− [p='Location' q='http://www.google.com/']
878− \/'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\0a<TITLE>301 Moved</TI\/
879− TLE></HEAD><BODY>\0a<H1>301 Moved</H1>\0aThe document has moved\0a<A HREF="http://www.google.com/">her
880− e</A>.\0d\0a</BODY></HTML>\0d\0a'
885−~fintud-macrep:dojo> :up &atom 'off'
888−## Marks
304+This is a request to the `%b` vane (Behn, the timer vane) to set a timer
889−
305+for 10 seconds in the future. After 10 seconds, Behn will respond with
890−We've used predefined marks already, but we haven't yet created
306+a `%wake` card, which you will recieve in `+on-arvo`. It will come back
891−our own marks. Let's write a sample mark of our own, then chain
307+on `/my/wire`.
892−it together with some preexisting ones to have fun with type
895−Let's make a small "cord" mark. "Cord" is a name we use for `@t`
896−text, but there's no predefined mark for it. Let's put the following code in:
900−/? 314
310+=/ =path /(scot %p our.hid)/home/(scot %da now.hid)/my-file/txt
901−|_ cod=@t
311+=/ contents=cage [%txt !>(~['text file line 1' 'line 2'])]
902−++ grab
312+[%pass /my/wire %arvo %c %info (foal path contents)]
904− ++ atom |=(arg=@ `@t`(scot %ud arg))
913−`/? 314` is the required
914−version number, just like in apps. After that everything's in a
915−`|_`, which is a `|%` core, but with input arguments. In our
916−case, the argument is the marked data.
918−There are three possible top-level arms in the `|_` core,
919−`++grab`, `++grow`, and `++grad` (which is used for revision
920−control, covered elsewhere). `++grab` specifies
921−functions to convert from another mark to the current mark.
922−`++grow` specifies how to convert from the current mark to
925−In our case, we only define one arm in `++grab`, namely `++atom`.
926−This allows us to convert any atom to a cord. `++scot` is a
927−standard library function in `hoon.hoon` which pretty-prints
928−atoms of various types. We want to treat the atom as an
929−unsigned decimal number, so we give `%ud` and the given argument
930−to `++scot`. Thus, the form of an arm in `++grab` is
931−`|=(other-mark-type this-mark-value)`.
933−In `++grow`, we just convert to the `md` mark, which is just a
934−`@t` internally, but it has fancy conversion functions to things
935−like html. `++md` isn't a function because the "argument" is
936−already a `@t`, so we can just produce the new value directly.
938−Let's play around a bit with this mark. First, let's take a
939−marked atom and convert it to our new mark.
942−~fintud-macrep:dojo> &atom 9
945−~fintud-macrep:dojo> &cord &atom 9
948−~fintud-macrep:dojo> &cord &atom &cord &atom 9
952−ASCII `9` is `57`. There's no requirement, implicit or otherwise,
953−that conversions back and forth between marks be inverses of each
954−other. These are semantic conversions, in that they refer to the
955−same concept. They're not isomorphisms.
957−Let's play around a little more:
960−~fintud-macrep:dojo> &md &cord &atom 17
963−~fintud-macrep:dojo> &hymn &md &cord &atom 17
966− [[%body ~] [g=[n=%pre a=~] c=~[[g=[n=%$ a=~[[n=%$ v="17"]]] c=~]]] ~]
971−That was exciting. If you squint at that, it looks an
972−awful lot like html. Our value `17` is still in there, in the
973−body. `urb` is the mark we render most web pages to. It makes
974−sure you have a complete skeleton of a web page. Of course, by
975−the time this gets to the web page, it's plain html. Let's do
976−the final step in the conversion.
979−~fintud-macrep:dojo> &mime &hymn &md &cord &atom 17
980−[[%text %html ~] p=52 q='<html><head></head><body><pre>17</pre></body></html>']
983−This is a mime-typed octet stream with type `/text/html`, length
984−121 bytes, and our lowly number `17` rendered to a web page.
985−`cord` was just one step in the chain.
987−Of course, arvo can infer some parts of this chain:
990−~fintud-macrep:dojo> &cord 17
994−Likewise, `md` is the only way to get from `cord` to `hymn`, so
995−that can be omitted. If we omit the `hymn`, though, we get:
998−~fintud-macrep:dojo> &mime &cord 17
999−[[%text %x-markdown ~] p=2 q='17']
1002−Here, we converted straight from `cord` to `md` to `mime`. Arvo
1003−decided that was the most straightforward conversion, but it
1004−gives a different result than passing it through the `hymn`
1007−The minimal correct input, then, is `&mime &hymn &cord &atom 17`.
1011−In the following we assume that you have some knowledge of Hoon. If you don't, check out chapters 1 and 2 of the Hoon tutorial and check back when you're ready.
1013−Let's get our planets to talk to each other. To listen for and receive messages from other planets, we'll need an app. Let's look at a very simple one, `echo.hoon`:
1016−:: Accepts any noun from dojo and prints it out
1018−:::: /===/app/echo/hoon
1020−!: :: enables debug printfs
1021−|% :: forms a core with subject as payload
1022−+$ move [bone card] :: defines echo app's arvo move mold
1023−+$ card $% $~ :: defines card (data) mold as null tagged union
1024− == :: end of mold definition; see $%
1025−-- :: end of card arm definition
1027−|_ [bow=bowl:gall $~] :: forms a core with bowl (app state) as sample
1028−++ poke-noun :: begin definition of poke-noun arm
1029− |= non=* :: forms a gate which accepts any noun
1030− ^- [(list move) _+>.$] :: casts product to mold of moves and context
1031− ~& echo+noun+non :: debug printf, i.e. '[%echo %noun *]'
1032− [~ +>.$] :: produce a cell of empty list and our state
1033−-- :: end of poke-noun arm definition
1036−Save this as `echo.hoon` in the `/app` directory of your urbit's pier.
1038−This is a very simple app that does only one thing. If you poke it with
1039−a value, it prints that value out. To try this out, you have to start
1040−the app, then you can poke it from the command line with the following
1044−~fintud-macrep:dojo> |start %echo
1047−~fintud-macrep:dojo> :echo 5
1051−~fintud-macrep:dojo> :echo [1 2]
1056−> There is currently a bug where the `[%echo %noun *]` lines are printed **above** the line you entered, so your output may not look exactly like this. `>=` means that a command was successfully received and executed.
1058−Most of the app code should be simple enough to guess its function. The
1059−important part of this code is the definition of `++poke-noun`.
1061−Once an app starts, it's always running in the background and you interact with
1062−it by sending it messages. The most straightforward way to do that is to poke it
1063−from the command line, which we we did with `:echo 5`
1064−(`:[app-name] [argument(s)]`).
1066−In this case, `++poke-noun` takes an argument (**sample**) `arg` and
1067−prints it to dojo with `~&` ([sigpam](/docs/reference/hoon-expressions/rune/sig#sigpam/)).
1068−This is an unusual rune that formally "does nothing", but the interpreter
1069−detects it and printfs the first child, before executing the second as if the
1070−first didn't exist. This is a slightly hacky way of printing to the console,
1071−but we'll get to the correct way later on.
1073−But what does `++poke-noun` produce? Recall that `^-` casts to a type. In this
1074−case, it's declaring that the end result (**product**) of the function
1075−(`++poke-noun`'s **gate**) will be of type `[(list) _+>.$]`. But what does this
1078−The phrase to remember is "a list of moves and our state". Urbit is a message
1079−passing system, so whenever we want to do something that interacts with the rest
1080−of the system, we send a message. Thus, a move is Arvo's equivalent of a
1081−syscall. The first thing that `++poke-noun` produces is a list of messages,
1082−called "moves". In this case, we don't actually want the system to do anything,
1083−so we produce the empty list, `~` (in the `[~ +>.$]` line).
1085−The second thing `++poke-noun` produces is our state. `+>.$` refers to a
1086−particular address in our subject where our formal app state is stored.
1087−It'll become clear why this is later on, but for now pretend that `+>.$`
1088−is a magic invocation that means "app state".
1090−Let's look at another example. Say we want to
1091−only accept a number, and then print out the square of that number.
1094−:: Accepts an atom from the dojo and squares it.
1096−:::: /===/app/square/hoon
1099−|% :: no moves in :square
1100−+$ move [bone card] :: no cards in :square
1105−|_ [bow=bowl:gall $~]
1108− ^- [(list move) _+>.$]
1109− ~& square+(mul tom tom)
1114−Save this as `square.hoon` in the `/app` directory of your urbit's pier.
1116−A few things have changed. Firstly, we no longer accept arbitrary nouns because
1117−we can only square atoms (integers, in this case an unsigned one). Thus, our
1118−argument is now `tom=@`. Secondly, it's `++poke-atom` rather than `++poke-noun`.
1122−Are there other `++poke`s? Yes. In fact, `noun` and `atom` are just two
1123−of arbitrarily many "marks". A mark is fundamentally a type definition,
1124−but accessible at the Arvo level. Each mark is defined in the `/mar`
1125−directory. Some marks have conversion routines to other marks, and some
1126−have diff, patch, and merge algorithms. None of these are required for a
1127−mark to exist, however.
1129−`noun` and `atom` are two predefined marks. In your `/mar` directory,
1130−there are already many more, and you may add more at will. The type
1131−associated with `noun` is `*`, and the type associated with `atom` is
1134−When we poke an app from anywhere, we do so with a mark that searches
1135−for the corresponding (`++poke-[mark]`) arm. Data constructed on the
1136−command line is by default marked with `noun`. In this case, the app is
1137−expecting an atom, so we have to explicitly mark the data with `atom`
1138−using `&[mark]`. Try the following commands:
1141−~fintud-macrep:dojo> |start %square
1144−~fintud-macrep:dojo> :square 6
1145−gall: %square: no poke arm for noun
1147−~fintud-macrep:dojo> :square &atom 6
1152−> Recall the bug where `%square` may get printed above the input line.
1154−Marks are powerful, and they're the backbone of Urbit's data pipeline,
1155−so we'll be getting quite used to them.
1157−**Exercise**:
315+This is a request to the `%c` vane (Clay, the filesystem vane) to write
1158−
316+a file to `/my-file/txt` in the home desk. You will not receive a
1159−- Write an app that computes fizzbuzz on its input.
317+response to this `note`.
1161−#### Sending a message to another urbit
1163−Let's write our first network message! Here's `examples/app/pong.hoon`:
1166−:: Allows one urbit to send the string 'Pong' to
320+=/ =path /my-file/txt
1167−:: another urbit.
321+=/ =moat [da+now.hid da+(add now.hid ~h1) path]
1168−::
322+[%pass /my/wire %arvo %c %warp our.hid %home ~ %many & moat]
1169−:::: /===/app/pong/hoon
1174−+$ card $% [$poke wire dock poke-contents]
1176−+$ poke-contents $% [$atom @]
1179−|_ [bow=bowl:gall $~] :: stateless
1183− ^- [(list move) _+>.$]
1184− ~& pong+'Outgoing pong!'
1186− ~[[ost.bow %poke /sending [to dap.bow] %atom 'Pong']]
1190− ^- [(list move) _+>.$]
1191− ~& pong+'Incoming pong!'
1192− ~& pong+received+`@t`tom
1195−++ coup |=(* [~ +>.$])
1201−Run it with these commands:
1204−~fintud-macrep:dojo> |start %pong
1207−~fintud-macrep:dojo> :pong &urbit ~sampel-sipnym
325+This is a request to the `%c` vane to send us a `%writ` `card` whenever
1208−>=
326+`/my-file/txt` changes in the next hour. There may be many responses to
1209−```
327+this note if the file changes many times.
1211−Replace `~sampel-sipnym` with another urbit. The easiest thing to do is to start
1212−a comet, a free and disposable Urbit identity. If you don't know how to start a
1213−comet, see [the user setup section](@/using/operations/_index.md). Don't forget to start
1214−the `%pong` app on that urbit, too. You should see, on the foreign
1218−[%pong 'Incoming pong!']
330+=/ =request:http
1219−[%pong %received 'Pong']
331+ [%'GET' 'https://example.com' ~ ~]
332+[%pass /my/wire %arvo %i %request request *outbound-config:iris]
1222−Most of the code should be straightforward. In `++poke-atom`, the only new thing
335+This is a request to the `%i` vane (Iris, the HTTP client vane) to make
1223−is the expression `` `@t`tom ``, which is **casting** the argument `tom` to type
336+a GET HTTP request to example.com. The response will come as an
1224−`@t`. As we already know, `@t` is the type of "cord" (text string).
337+`%http-response` card.
1226−The more interesting part is in `++poke-urbit`. The `urbit` mark is an urbit
339+#### Gifts
1227−identity, and the Hoon type associated with it is `ship` or `@p` (the "p"
1228−stands for "phonetic base").
1230−Recall that in a `++poke` arm we produce "a list of moves and our state". Until
341+In contrast to the many possible `note`s, there are only two types of
1231−now, we've left the list of moves empty, since we haven't wanted to tell Arvo to
342+`gift`s that an agent may give:
1232−do anything in particular. Now we want to send a message to another urbit. Thus,
1233−we produce a list with one element:
1236−~[[ost.bow %poke /sending [to dap.bow] %atom 'Pong]]
345+[%give %fact (unit path) =cage]
346+[%give %kick (unit path)]
1239−#### Moves
349+A subscription update is a new piece of subscription content for all
1240−
350+subscribers on a given `path`. If no `path` is given, then the update is
1241−The general form of a move is `[bone term wire *]`.
351+only given to the program that instigated this request. Typical use of
1242−
352+this mode is in `+on-watch` to give an initial update to a new
1243−Or, in pseudo-code:
353+subscriber to get them up to date.
1245−`["cause" sys-call/action tack-new-layer-on-cause action-specific information]`
355+A subscription close closes the subscription for all subscribers on a
1246−
356+given `path`. If no `path` is given, then the update is only given to the
1247−Let's walk through each of these elements step by step.
357+program that instigated this request. Typical use of this mode would be
1248−
358+in `+on-watch` to produce a single update to a subscription then close
1249−##### Bones ("cause")
359+the subscription.
1251−If you look up `++bone` in `hoon.hoon`, you'll see that it's a number (`@ud`),
361+### Vases and cages
1252−and that it's an opaque reference to a duct. `++duct` in `hoon.hoon` is a list
362+
1253−of `wire`s, where `++wire` is an alias for `++path`. `++path` is a list of
363+A `vase` is a piece of dynamic data. Structurally, it's a pair of an
1254−`++knot`s, which are ASCII text. Thus, a duct is a list of paths, and a bone is
364+explicit reification of a type and an untyped noun. This lets us
1255−an opaque reference to it (in the same way that a Unix file descriptor is an
365+represent a value which has a type that isn't known at compile time. A
1256−opaque reference to a file structure). Thus, to truly understand bones, we must
366+vase has three operations:
1257−understand ducts.
367+
1258−
368+- `!>` is a unary rune that lifts a statically typed value to a
1259−A duct is a stack of causes, again, represented as paths, which are called wires.
369+ dynamically-typed `vase`. For example, `!>('hi')` gives `[[%atom %t ~]
1260−At the bottom of every duct is a unix event, such as a keystroke, network
370+ 26.984']`, which has type `[type *]`.
1261−packet, file change, or timer event. When Arvo is given this event, it routes
371+
1262−the event to appropriate kernel module for handling.
372+- `!<` is a binary rune that takes a mold and a dynamically typed `vase`
1263−
373+ and reduces it to a statically typed value. If the `vase` does not in
1264−Sometimes, the module can immediately handle the event and produce any necessary
374+ fact have the type of the mold you give it, then it produces `~`, else
1265−results. For example, when we poked the `++poke-atom` arm above, we poked `%gall`,
375+ it produces `[~ value]`. For example, `!<(@t !>('hi'))` produces `[~
1266−our application server, which was able to respond to our poke directly. When the
376+ 'hi']` while `!<(^ !>('hi'))` produces `~`.
1267−module cannot service the request itself, it sends instructions to another
377+
1268−kernel module or application (through the `%gall` module) to do a specified
378+- The compiler takes text and converts it to a `vase` of the compiled
1269−action, and produces the result from that.
379+ code. Agents shouldn't need this directly, but Gall uses this to
1270−
380+ compile agents to `vase`s, on which it calls `!<(agent:gall
1271−Furthermore, when one module sends a message to another kernel module or
381+ compiled-agent-vase)`.
1272−application, it also sends along the duct it was given with its new wire tacked
382+
1273−onto the top. Now the duct has two entries, with the unix event on the bottom
383+A `cage` is simply a pair of a mark and a vase. A mark is a textual tag
1274−and the kernel module that handled it on top. This process can continue
384+that should correspond to the particular dynamic type in the vase.
1275−indefinitely, pushing more and more wires onto the top of the duct. When an
385+
1276−entity finally produces a result, a wire is popped off the duct, and the result
386+In agents, we use vases to represent types which Gall doesn't know about
1277−is passed all the way back down, repeating the process of wire popping
387+when it was compiled, but which nevertheless need to go outside the
1278−sequentially until the bottom of the duct is reached.
388+agent. In practice, there are three common cases:
1280−In effect, a duct is an Arvo-level call stack. It's worth noting that while in
390+- The data in a "poke" request is of a type that is defined by each
1281−traditional call stacks a function call happens synchronously and returns
391+ agent, so it must by dynamic. This is the input to `++on-poke` as well
1282−exactly once. In Arvo, multiple moves can be sent at once, are evaluated
392+ as the cage in the `%poke` case of the `%agent` note.
1283−asynchronously, and each one may be responded to zero or more times.
393+
1284−
394+- The data in a subscription update is defined by each agent, so it must
1285−The point to take home is that whatever caused `++poke-urbit` to be called is
395+ be dynamic. This is the cage in `%fact`, both when giving the update
1286−also the root cause for the network message we're trying to send. Thus, we say
396+ and in the input to `+on-agent`.
1287−to send the network message along the given bone `ost`.
397+
1288−
398+- The state of an agent is also unique to each agent. Most of the time,
1289−##### Wire (path)
399+ Gall doesn't interact directly with agent's state, but when upgrading
1290−
400+ an agent, it must pass the state of the old version of the agent to
1291−Of course, we have to push a new wire onto our duct before passing it along (or
401+ the new version of the agent. This is the output of `+on-save` and
1292−responding to it directly) anywhere. This wire can have any data we want in it,
402+ the input to `+on-load`.
1293−but we don't need anything specific here, so we just use the wire `/sending`
403+
1294−(`/elem1/elem2/elemN` is one syntax used to create `++path`s and `++wire`s of N
404+### State
1295−elements). If we were expecting a response (which we're not), it would come back
405+
1296−along the `/sending` wire, meaning that the path `/sending` will be passed back
406+In the definition of an agent, the entire core is wrapped with the `^|`
1297−to the response handler as an argument. Although it's not required, it's
407+rune, which corresponds to the "iron" variance mode. This means it's
1298−generally a good idea to make the wire human-readable for bug-handling purposes.
408+"contravariant" in the input, which allows Gall to write to the new bowl
1299−
409+on every event. It also means that the context of the core is opaque to
1300−##### Term (sys-call)
410+Gall, which means you can store state in your context in whatever
1301−
411+structure you want.
1302−Each move also has a `term`, composed of lowercase ASCII and/or `-`. This `term`
412+
1303−has the aura `@tas`. In this case, our `term` is `%poke`, which is the name of
413+This is why most of the handler arms produce not just a list of cards
1304−the particular kind of move we're sending. You can always use `%poke` to message
414+but also a new `agent:gall`. Usually, this will be the "same" agent in
1305−an app. Other common names include `%warp`, to read from the filesystem;
415+the sense that the code will be the same, but you may have changed the
1306−`%wait`, to set a timer; and `%them`, to send an http request.
416+state that's stored in its (opaque to Gall) context.
1308−The move ends with `*` (that is, any noun) since each type of move takes
418+The skeleton example above gives an example of keeping an atom as your
1309−different data. In our case, a `%poke` move takes a target (urbit and app) and
419+state. Define the state in the context of your agent core, then you can
1310−marked data, then pokes the arm of the corresponding mark on that app on that
420+change it with `=.` or `%=`, as long as you produce the new version of
1311−urbit with that data. `[to-urbit-address %pong]` is the target urbit and app,
421+the agent core.
1312−`%atom` is the `mark`, and`'Pong'` is the data.
422+
1313−
423+When you upgrade an agent, you need to extract the state from your
1314−When Arvo receives a `%poke` move, it calls the appropriate `++poke`.
424+opaque context and produce it to Gall as a dynamically typed vase.
1315−The same mechanism is used for sending messages between apps on the same
425+Usually, this will be easy: just call `!>` on whatever state you wish
1316−urbit as for sending messages between apps on different urbits.
426+to preserve. This is `+on-save`.
1318−> We said earlier that we're not expecting a response. This is not entirely true: the `++coup` is called when we receive acknowledgment that the `++poke` was called. We don't do anything with this information right now, but we could.
428+When the new agent is about to be started, Gall will call it with
1319−
429+`+on-load` with the vase just produced above. This allows you to ingest
1320−**Exercises**:
430+your old state and continue right where you left off.
1322−- Extend either of the apps in the first two exercises to accept input over the network in the same way as `pong`.
432+If the type of your state hasn't changed, you can just
1324−- Modify `pong.hoon` to print out a message when it receives acknowledgement.
1326−- Write two apps, `even` and `odd`. When you pass an atom to `even`, check whether it's even. If so, divide it by two and recurse; otherwise, poke `odd` with it. When `odd` receives an atom, check whether it's equal to one. If so, terminate, printing `%success`. Otherwise, check whether it's odd. If so, multiply it by three, add one, and recurse; otherwise, poke `even` with it. When either app receives a number, print it out along with the name of the app. In the end, you should be able to watch Collatz's conjecture play out between the two apps. Sample output:
1329−~fintud-macrep:dojo> :even &atom 18
1353−- Put `even` and `odd` on two separate urbits and pass the messages over the network. Post a link to a working solution in `:talk` to receive a cookie.
1358−A security driver is a file in `/=home=/sec/<tld>/<domain>/hoon`
1359−that handles the authentication for all HTTP requests to
1360−`https://<domain>.<tld>`. When anything in Urbit makes an HTTP
1361−request through `%eyre` (our web server), it checks to see if we
1362−have a security driver for the requested domain, and if so
1363−filters the request through the driver. The security driver will
1364−usually either decorate the request with the needed credentials
1365−if it has them, or else it will guide the user through the
1366−process of authenticating Urbit with the service.
1368−Each web service needs its own security driver, but most of them
1369−are pretty standard. We recommend starting with an existing
1370−security driver based on your needed authentication method (e.g.
1371−basic auth, OAuth1, OAuth2), and changing whatever is specific to
1374−The best strategy for building security drivers is to copy a
1375−similar one and tweak it until it works for you. The best
1376−representatives are Github for Basic Authentication, Twitter for
1377−OAuth1, Slack for OAuth2 when access tokens don't expire, and
1378−Google APIs for OAuth2 when access tokens do expire. In most
1379−cases you'll need to make very few changes to one of these
1382−Still, it's worth seeing one or two built from the ground up.
1383−Here, we'll build a connector for the Github API v3. (It's already in `/sec/com/github.hoon` if you want to see it.)
1386−simplest way to interact with the Github API is to just fetch
1387−https://api.github.com from the dojo. First, run this:
1390−> |init-auth-basic /com/github
1393−Input your Github username and password as directed. Next:
1396−~your-urbit:dojo> +https://api.github.com
1399−> Note: the current version of this API is outdated, so you'll get a 403 message at this point. We need to update this so you can follow along with the examples. We're sorry!
1401−Github exposes a few endpoints to the general web, and the root
1402−endpoint is one of them. This gives you a textual representation
1403−of a JSON object that contains a bunch of urls to other parts of
1404−Github's API. This is exactly the same response you would get
1405−from just running `curl https://api.github.com` on UNIX.
1407−If we don't have a security driver for Github yet, many of the
1408−endpoints won't be accessible, or they will only have publicly
1409−accessible information. Most of what we care about requires us
1410−to be authenticated.
1414−Here's a simple security driver:
1417−:: Test url +https://api.github.com/user
435+=. state !<(state-type old-state-vase)
1419−:::: /hoon/github/com/sec
1423−|_ [bal=(bale keys:basic-auth) $~]
1424−++ aut ~(standard basic-auth bal ~)
1425−++ filter-request out-adding-header:aut
1429−Github supports authentication through either Basic
1430−Authentication or OAuth2. We'll show the basic auth example
1431−first, but in general we'd prefer OAuth2.
1433−Since this driver is for Github, put it in
1434−`/=home=/sec/com/github/hoon`.
1436−To try this out, we first have to initialize our credentials with
1437−`|init-auth-basic`, which prompts for the url of the service
1438−(api.github.com), username, and password. It stores this
1439−information in a manner that `%eyre` knows how to decode and send
1440−put into your `bale`. Note that `|init-auth-basic` is standard for
1441−basic auth, but other auth schemes (like OAuth) are initialized
1444−After you've run `|init-auth-basic`, you should be able to run
1445−`+https://api.github.com/user` and get a response indicating who
1446−you're logged in as.
1448−You can similarly POST:
1451−~your-urbit:dojo> +https://api.github.com/gists &json _(cork de-json:html need) '{"files":{"file1.txt":{"content":"can\'t stop the signal"}}}'
1454−This creates a gist, go to `https://gist.github.com/<username>` to
1457−The `+url` syntax used in "source" position makes a GET request,
438+If it has changed, then it should look more like:
1458−but if you pass it data it will make a POST request with that
1459−data as the body. This request is authenticated seamlessly. You
1460−can send data of any mark as long as it has a conversion path to
1463−Let's take a look at the code and see if we can figure out what's
1467−/+ basic-auth
441+=. state (upgrade-state !<(old-state-type old-state-vase))
1470−First, we load a library called `basic-auth`. This library
444+It's useful to tag your state with a version number so that the
1471−exposes two items. `keys` is the type of the basic auth key,
445+`+upgrade-state` function can take a tagged union of all your old state
1472−which is essentially a base64'd username and password.
446+types and upgrade from any of them to the current state type. For
1473−`standard` accepts a `bale` and produces a core with everything
447+example, it may have the following structure:
1474−we'll need to implement basic auth.
1476−Every security driver needs some amount of state, which is stored
1477−in `bale`. `bale` contains:
1479−- `our`, our urbit name
1480−- `now`, the current time
1481−- `eny`, 256 bits of entropy
1482−- `byk`, the urbit, desk and case which the security driver is running from
1483−- `usr`, the particular identity we're logging in with
1484−- `dom`, the site we're accessing
1485−- `key`, the secrets required to authenticate requests. The type for this entry is supplied by the programmer as the argument to `++bale`. Thus, in our case, `key` in our bale is of type `keys:basic-auth`.
1487−Additionally, a security driver contains at least one out of the
1488−five "special" arms:
1490−- `++filter-request` is a function which takes a hiss (http request) and produces a `sec-move` (defined in zuse), which is one of:
1491− - `[%send hiss]`, which sends the new hiss. This is the case in, for example, basic auth, where all we need to do is add an extra header to the request.
1492− - `[%give httr]`, where httr is an http response. This immediately returns an http response to the sender.
1493− - `[%show purl]`, where a purl is a parsed url. This displays a message asking the user to visit the given url to continue the authentication process.
1494− - `[%redo ~]`, which redoes the request.
1495− - `%eyre` calls `++filter-request` just before sending an HTTP request to the specified domain. This allows the security driver to filter the requests, decorating them with authentication data.
1496−- `++filter-response` is a function which takes an httr and produces a `sec-move`. `%eyre` calls it after receiving an HTTP response from the specified domain. This allows the security driver to handle authentication errors, commonly caused by expired tokens, and retry the request.
1497−- `++receive-auth-query-string` is a function that takes a `quay` (list of query parameters) and produces a `sec-move`. `%eyre` calls it when it receives a request on the callback url. Generally, this happens in OAuth after the user has granted access to the urbit app, and the service makes a request to the callback url with the code. The security driver should then make a request to convert the code into an access token.
1498−- `++receive-auth-response` is a function that takes an httr and produces a `sec-move`. `%eyre` calls it when it gets a response to the request made in `++receive-auth-query-string`.
1499−- `++update` is a function which converts old state to a new format when the security driver gets updated. If you want to just get rid of the old state, define `++discard-state` as `~`.
1501−For basic auth, we only have to worry about `++filter-request`,
1502−since all we need to do is add the correct header to each
1503−request. Our `++aut` core contains a function
1504−`++out-addding-header`, which does exactly what we want.
1506−Where do the keys come from, though? Remember we ran
1507−`|init-auth-basic` to input them. This just prompts you for the
1508−service url, username, and password, and it stores them next to
1509−the driver. In our case, that's
1510−`/=home=/sec/com/gitub/api/atom`. These keys are stored
1511−encrypted, and you don't want to edit them directly. `%eyre`
1512−loads them directly into your `bale`.
1516−Most services are better accessed through some form of OAuth.
1517−Github can be accessed with OAuth2, which is a little more
1518−complicated than basic auth, but not hugely so.
1520−The basic steps for OAuth2 are these:
1522−- create an app and get its client id and client secret (on Github)
1523−- store these in Urbit (with `|init-oauth2`)
1524−- try to make a request that requires authentication. The security driver should prompt you to visit a particular url.
1525−- visit the url and click "authorize", which gives the app access to your account
1526−- the security driver will turn the code it receives into an access token
1527−- on every request, include that access token
1529−Creating an app is easy and
1530−[well-documented](https://github.com/settings/applications/new).
1531−If necessary, set the callback url to
1532−`/~/ac/github.com/_state/in`, and note its client id and client
1535−Run `|init-oauth2`, which will prompt for the hostname
1536−(api.github.com), client id, and client secret. This will store
1537−the information, encrypted, in `/=home=/sec/com/github.atom`,
1538−just as with Basic Authentication.
1540−Now we need a security driver. Use this:
1543−:: Test url +https://api.github.com/user
450++$ old-state-types
1544−::
451+ $% [%0 s=state-0]
1545−:::: /hoon/github/com/sec
452+ [%1 s=state-1]
1546− ::
453+ [%2 s=state-2]
1551−|_ [bal=(bale keys:oauth2) tok=token:oauth2]
1552−++ scopes :: comment out scopes to taste
1553− :~ 'user' 'user:email' 'user:follow' 'public_repo' 'repo'
1554− 'repo_deployment' 'repo:status' 'delete_repo' 'notifications'
1555− 'gist' 'read:repo_hook' 'write:repo_hook' 'admin:repo_hook'
1556− 'admin:org_hook' 'read:org' 'write:org' 'admin:org'
1557− 'read:public_key' 'write:public_key' 'admin:public_key'
1559−:: ++aut is a "standard oauth2" core, which implements the
455+++ upgrade-state
1560−:: most common handling of oauth2 semantics. see lib/oauth2 for more details,
456+ |= old=old-state-types
1561−:: and examples at the bottom of the file.
457+ =? old ?=(%0 -.old)
1562−++ aut (~(standard oauth2 bal tok) . |=(tok/token:oauth2 +>(tok tok)))
458+ (state-0-to-1 s.old)
1563−++ filter-request
459+ =? old ?=(%1 -.old)
1564− %^ out-add-query-param:aut 'access_token'
460+ (state-1-to-2 s.old)
1565− scope=~[%client %admin]
461+ ?> ?=(%2 -.old)
1566− oauth-dialog='https://github.com/login/oauth/authorize'
462+ s.old
1568−++ receive-auth-query-string
1569− %- in-code-to-token:aut
1570− url='https://github.com/login/oauth/api/access_token'
1571−++ receive-auth-response bak-save-token:aut
1575−The oauth2 library provides the main "engine" in `standard`, just
1576−like in basic auth, except that we also have to specify a
1577−function to save the access token when we get it. It's worth
1578−noting that this library is well documented in the source,
1579−including examples: `/=home=/lib/oauth2/hoon`.
1581−Running `+http://api.github.com/user` tries to make a request to
1582−Github with authentication. This loads into the bale the keys,
1583−which are of type `keys:oauth2`, defined in the oauth2 library.
1584−We don't have to worry about the specifics of these keys -- the
1585−library handles them -- but they include the client id and the
1588−Before the request is sent, `%eyre` calls `++filter-request` to
1589−decorate it with authentication headers. Since we don't yet have
1590−an access token, we need to prompt the user to visit the "dialog"
1591−url. `++out-add-query-param` checks whether we have a token, and
1592−if we don't it produces `%show` and a message telling the user to
1593−visit the `oauth-dialog` url we provide, with the extra query
1594−parameters added (client id, scopes, and state).
1596−When the user visits that url, Github will ask them to log in and
1597−authorize the application. When they do so, Github will post a
1598−request to `/~/ac/github.com/_state/in` with a code in the query
1599−string. `%eyre` sends the query string to
1600−`++receive-auth-query-string`, which is a function that takes a
1601−quay and produces a `sec-move`. `++in-code-to-token` from our
1602−`aut` takes the `code` parameter from the quay and uses it to
1603−create a request to the given url. This request includes the
1604−client id, the client secret, and the code we just got.
1606−Github's response to that request includes an access token in its
1607−body. This is handled in `++receive-auth-response`.
1608−`++bak-save-token` extracts that token and gives it to the
1609−function that we passed in the definition of `++aut`. In our
1610−case, we just save that token into our state for later usage.
1612−Now we have the credentials we need, so `++filter-request` is
1613−called with the original request, and `++out-add-query-param`
1614−passes the request through with the addition of the query
1615−parameter `access_token`.
1617−Just like with Basic Authentication, you should be able to run
1618−`+https://api.github.com/user` and get a response indicating who
1619−you're logged in as.
1621−Some drivers are slightly more complicated. For example, Github's
1622−access token's don't expire, but that's not the case for all
1623−services. The googleapis driver shows the flow when the access
1624−token can expire. Twitter uses OAuth1, for which we have another
1627−Remember, of course, that the best strategy for building security
1628−drivers is to copy a similar one and tweak it to taste.
1632−In the last section we built a few small apps that sent moves. These apps were
1633−entirely stateless, however. Most useful apps require some amount of state.
1634−Let's build a trivial stateful app. It'll keep a running the sum of all the
1635−atoms we poke it with. Here's `examples/app/sum.hoon`:
1638−:: Keeps track of the sum of all the atoms it has been
1639−:: poked with and prints the sum out
1641−:::: /===/app/sum/hoon
1649−|_ [bow=bowl:gall sum=@]
1653− ^- [(list move) _+>.$]
1654− ~& sum+(add sum tom)
1655− [~ +>.$(sum (add sum tom))]
1658− |= [wir=wire err=(unit tang)]
1659− ^- [(list move) _+>.$]
1661− ~& sum+success+'Poke succeeded!'
1663− ~& sum+error+'Poke failed. Error:'
1670−We can start it with `|start %sum`, and then run it:
1673−~fintud-macrep:dojo> :sum &atom 5
1677−~fintud-macrep:dojo> :sum &atom 2
465+### Bowl
1681−~fintud-macrep:dojo> :sum &atom 15
467+The core takes as input a `bowl`, which includes useful info like the
1682−[%sum 22]
468+current ship, the current time, and a renewable source of entropy. This
1683−>=
469+information is available to any of the handlers.
1686−We can see that app state is being saved, but when, where, and how?
1688−The state is stored as the second thing in the `|_` line. In our case, it's
471+## Arms
1689−simply an atom named `sum`. We change it by producing our state not with
1690−`+>.$` (as before), but with `+>.$(sum (add sum tom))`. We've seen all these
1691−parts before, but you might not recognize them.
1693−Recall in the first chapter that we recursed with the expression
473+A description of each of the handler arms follows.
1694−`$(b (add 3 b))`. This meant "produce `$` with `b` changed to `(add 3 b)`.
1695−Similarly, `+>.$(sum (add sum tom))` means "produce `+>.$` (i.e. our
1696−context, which contains our state) with `sum` changed to `(add sum tom)`.
1698−At a high level, then, when we handle state, we do it explicitly. It's passed in
475+### +on-init
1699−and produced explicitly. In Unix systems, application state is just a block of
1700−memory, which you need to serialize to disk if you want to keep it around for
1703−In Urbit, app state is a single (usually complex) value. In our example, we have
477+This arm is called once when the agent is started. It has no input and
1704−very simple state, so we defined `sum=@`, meaning that our state is an atom.
478+lets you perform any initial IO.
1705−Of course, `sum` is just a name, and you're free to name your state whatever
1706−you like. But let's clarify a couple other things before we continue.
1708−First, `bowl` is a set of general global states. This set is managed by the
480+### +on-save
1709−system. It includes things like `now` (the current time), `our` (our urbit
1710−identity), and `eny` (512 bits of guaranteed-fresh entropy). For the full list
1711−of things in `++bowl`, search for `++ bowl` (note: two spaces) in
1714−> This is, perhaps, the most common way to learn Hoon. The easiest way to learn about an identifier you see in code is to search in `/arvo/sys/zuse.hoon` and `/arvo/sys/hoon.hoon` for it. Urbit's codebase is less than 30000 lines of code combined, including the hoon parser, the compiler, and the `/arvo` microkernel, so you can usually use the code and its comments as a reference doc. You can also read [zuse.hoon](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon) and [hoon.hoon](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/hoon.hoon) in your browser.
482+This arm is called immediately before the agent is upgraded. It
483+packages the permament state of the agent in a `vase` for the next version
484+of the agent. Unlike most handlers, this cannot produce effects.
1716−The second thing we should clear up is this: Urbit needs no "serialize to disk"
486+### +on-load
1717−step. Everything you produce in the app state is persistent across calls to the
1718−app, restarts of the urbit, and even power failure. If you want to write to the
1719−Unix filesystem, you can, but it's not needed for persistence. Urbit has
1720−transactional events, which makes it an [ACID operating
1721−system](https://en.wikipedia.org/wiki/ACID). Thus, you don't have to worry about
1722−persistence when programming in Urbit, or ever go through the hassle of having
1723−to set up and write to a database.
1725−**Exercises**:
488+This arm is called immediately after the agent is upgraded. It receives
489+a `vase` of the state of the previously-running version of the agent,
490+which allows it to cleanly upgrade from the old agent.
1727−- Modify `:sum` to reset the counter when you poke it with `0`.
492+### +on-poke
1729−- Write an app called `:last` that prints out the previous value you poked it with.
494+This arm is called when the agent is "poked". The input is a `cage`, so
495+it's a pair of a mark and a dynamic `vase`.
1731−Sample output:
497+### +on-watch
1733−```
499+This arm is called when a program wants to subscribe to the agent on a
1734−~fintud-macrep:dojo> :last 7
500+particular `path`. The agent may or may not need to perform setup steps
1735−[%last 0]
501+to intialize the subscription. It may produce a `%give`
1736−>=
502+`%subscription-result` to the subscriber to get it up to date, but after
503+this event is complete, it cannot give further updates to a specific
504+subscriber. It must give all further updates to all subscribers on a
1738−~fintud-macrep:dojo> :last [1 2 3]
507+If this arm crashes, then the subscription is immediately terminated.
1739−[%last 7]
508+More specifcally, it never started -- the subscriber will receive a
1740−>=
509+negative `%watch-ack`. You may also produce an explicit `%kick` to
510+close the subscription without crashing -- for example, you could
511+produce a single update followed by a `%kick`.
1742−~fintud-macrep:dojo> :last 'howdy'
513+### +on-leave
515+This arm is called when a program becomes unsubscribed to you.
516+Subscriptions may close because the subscriber intentionally
517+unsubscribed, but they also could be closed by an intermediary. For
518+example, if a subscription is from another ship which is currently
519+unreachable, Ames may choose to close the subscription to avoid queueing
520+updates indefinitely. If the program crashes while processing an
521+update, this may also generate an unsubscription. You should consider
522+subscriptions to be closable at any time.
1747−## Subscriptions
524+### +on-peek
1749−We've dealt fairly extensively with "poke" messages to an app, but these
526+This arm is called when a program reads from the agent's "scry"
1750−are somewhat limited. A poke is a one-way message, but more often we
527+namespace, which should be referentially transparent. Unlike most
1751−want to subscribe to updates from another app. You could build a
528+handlers, this cannot perform IO, and it cannot change the state. All
1752−subscription model out of one-way pokes, but it's such a common pattern
529+it can do is produce a piece of data to the caller, or not.
1753−that it's built into Arvo.
1755−Let's take a look at two apps, `:source` and `:sink`. First,
531+If this arm produces `[~ ~ data]`, then `data` is the value at the the
1756−`:source`:
532+given `path`. If it produces `[~ ~]`, then there is no data at the given
1757−
533+`path` and never will be. If it produces `~`, then we don't know yet
1758−```hoon
534+whether there is or will be data at the given `path`.
1759−:: Sends subscription updates to sink.hoon
1761−:::: /===/app/source/hoon
1766− +$ move [bone card]
1767− +$ card $% [%diff diff-contents]
1769−+$ diff-contents $% [%noun *]
1773−|_ [bow=bowl:gall $~]
1777− ^- [(list move) _+>.$]
1779− %+ turn (prey:pubsub:userlib /example-path bow)
1780− |=([o=bone *] [o %diff %noun non])
1782−++ peer-example-path
1784− ^- [(list move) _+>.$]
1785− ~& source+peer-notify+'Someone subscribed to you!'
1786− ~& source+[ship+src.bow path+pax]
1790− |= [wir=wire err=(unit tang)]
1791− ^- [(list move) _+>.$]
1793− ~& source+success+'Poke succeeded!'
1795− ~& source+error+'Poke failed. Error:'
1796− ~& source+error+err
1800− |= [wir=wire err=(unit tang)]
1801− ^- [(list move) _+>.$]
1803− ~& source+success+'Peer succeeded!'
1805− ~& source+error+'Peer failed. Error:'
1806− ~& source+error+err
1812−Save this as `source.hoon` in the `/app` directory of your urbit's pier.
1814−And secondly, `:sink`:
1817−:: Sets up a simple subscription to source.hoon
1819−:::: /===/app/sink/hoon
1824−+$ card $% [%peer wire dock path]
1825− [%pull wire dock $~]
1829−|_ [bow=bowl:gall val=?]
1833− ^- [(list move) _+>.$]
1834− ?: &(=(%on non) val)
1843− ?: &(=(%off non) !val)
1845− ~[[ost.bow %pull /subscribe [our.bow %source] ~]]
1847− sink+unsubscribed+'You are now unsubscribed!'
1848− sink+subscribed+'You are now subscribed!'
1852− |= [wir=wire non=*]
1853− ^- [(list move) _+>.$]
1854− ~& sink+received-data+'You got something!'
1859− |= [wir=wire err=(unit tang)]
1860− ^- [(list move) _+>.$]
1862− ~& sink+success+'Poke succeeded!'
1864− ~& sink+error+'Poke failed. Error:'
1869− |= [wir=wire err=(unit tang)]
1870− ^- [(list move) _+>.$]
1872− ~& sink+success+'Peer succeeded!'
1874− ~& sink+error+'Peer failed. Error:'
1881−Save this as `sink.hoon` in the `/app` directory of your urbit's pier (make sure both `source` and `sink` are on the same ship).
1885−- `bowl` is the type of the system state within our app. For example, it includes things like `our`, the name of the host urbit, and `now`, the current time.
1887−- You may have noticed the separate `|%` above the application core `|_`. We usually put our types in another core on top of the application core. We can access these type from our `|_` because in `.hoon` files, all cores are called against each other. (The shorthand for 'called' is `=>`.) Thus, the `|%` with the types is in the context of the `|_`, as it lies above it: `hoon.hoon` `=> |% w types => |_`
1889−Here's some sample output of the two working together:
1892−~fintud-macrep:dojo> |start %source
1895−~fintud-macrep:dojo> |start %sink
1898−~fintud-macrep:dojo> :sink %on
1899−[%source %peer-notify 'Someone subscribed to you!']
1900−[%source [%ship ~fintud-macrep] %path /]
1901−[%sink %success 'Peer succeeded!']
1904−~fintud-macrep:dojo> :source 5
1905−[%sink %received-data 'You got something!']
1909−~fintud-macrep:dojo> :sink %off
1912−~fintud-macrep:dojo> :source 6
1915−~fintud-macrep:dojo> :sink %on
1916−[%source %peer-notify 'Someone subscribed to you!']
1917−[%source [%ship ~fintud-macrep] %path /]
1918−[%sink %success 'Peer succeeded!']
1921−~fintud-macrep:dojo> :source 7
1922−[%sink %received-data 'You got something!']
1929−Hopefully you can get a sense for what's happening here. When we poke
1930−`:sink` with `%on`, `:sink` subscribes to `:source`,
1931−and so whenever we poke `:source`, `:sink` gets the update and
1932−prints it out. Then we unsubscribe by poking `:sink` with `%off`, and
1933−`:sink` stops getting updates. We then resubscribe.
1935−There's a fair bit going on in this code. Let's look at `:source`
1938−Our definition of `move` is fairly specific, since we're only going to
1939−sending one kind of move. The `%diff` move is a subscription update, and
1940−its content is marked data which gall routes to our subscribers.
1942−This is a slightly different kind of move than we've dealt with so
1943−far. It's producing a result rather than calling other
1944−code (i.e. it's a return rather than a function call), so if you recall
1945−the discussion of ducts, a layer gets popped off the duct rather than
1946−added to it. This is why no wire is needed for the move -- we won't
1947−receive anything in response to it.
1949−Anyways, there are four functions (arms) inside the `|_`. We already know when
1950−`++poke-noun` is called. `++peer-example-path` is called when someone tries to
1951−subscribe to our app. Of course, you don't just subscribe to an app; you
1952−subscribe to a path on that app. This path comes in as the argument to `++peer`.
1954−In our case, we don't care what path you subscribed on, and all we do is print
1955−out that you subscribed. Arvo keeps track of your subscriptions, so you don't
1956−have to. You can access your subscribers by looking at `sup` in the bowl that's
1957−passed in. `sup` is of type `(map bone {@p path})`, which associates bones with
1958−the urbit who subscribed, and which path they subscribed on. If you want to
1959−communicate with your subscribers, send them messages along their bone.
1961−`++poke-noun` "spams" the given argument to all our subscribers.
1962−There's a few things we haven't seen before. Firstly, `:_(a b)` is the
1963−same as `[b a]`. It's just a convenient way of formatting things when
1964−the first thing in a cell is much more complicated than the second.
1965−Thus, we're producing our state unchanged.
1967−Our list of moves is the result of a call to `++turn`. `++turn` is what many
1968−languages call "map" -- it runs a function on every item in a list and collects
1969−the results in a list. The list is `(prey:pubsub:userlib /example-path bow)` and the function
1970−is the `|=` line right after it.
1972−`prey:pubsub:userlib` is a standard library function defined in `zuse.hoon`. It takes a path
1973−and a bowl and gives you a list of the subscribers who are subscribed on a path
1974−that begins with the given path. "Prey" is short for "prefix".
1976−Now we have the list of relevant subscribers. This a list of triples,
1977−`[bone @p path]`, where the only thing we really need is the bone, because we
1978−don't need to know their urbit or what exact path they subscribed on. Thus, our
1979−transformer function takes `[o=bone *]` and produces `[o %diff %noun non]`,
1980−which is a move that provides bone `o` with this subscription update:
1981−`[%noun non]`". This is fairly dense code, but what it's doing is
1986−`:source` should now make sense. `:sink` is a little longer, but not much more
1989−In `:sink`, our definition of of `++move` is different. All moves start
1990−with a `bone`, and we conventionally refer to the second half as the "card", so
1991−that we can say a move is an action that sends a card along a bone.
1993−We have two kinds of cards here: we `%peer` to start a subscription, and we
1994−`%pull` to stop it. Both of these are "forward" moves that may receive a
1995−response, so they need a wire to tack onto the duct before they pass it on. They
1996−also need a target, which is a pair of an urbit and an app name. Additionally,
1997−`%peer` needs a path on that app to subscribe too. `%pull` doesn't need this,
1998−because its semantics are to cancel any subscriptions coming over this duct. If
1999−your bone and wire are the same as when you subscribed, then the cancellation
2000−will happen correctly.
2002−The only state we need for `:sink` is a boolean to indicate whether
2003−we're already subscribed to `:source`. We use `val=?`, where `?`
2004−is the type associated with boolean (similar to `*`, `@`), which defaults
2005−to `%.y` (that is, `0`).
2007−In `++poke-noun` we check our input to see both if it's `%on` and
2008−available (`val` is `%.y`). If so, we produce the move to subscribe to
2021−Also, in the preceding lines, we set `val` to `%.n` (`|`) with `+>.$(val |)`.
2022−Remember that the `:_` constructs an inverted cell, with the first child
2023−(`+>.$(val |)` in our case) as the tail and the second child as the head. Here,
2024−the cell we produce when our subscription is `%on` and `val` is `%.y` has a
2025−head with our new state where `val` is set to `%.n` and a tail of our list of
2026−moves, which is shown in the code block above.
2028−Otherwise, if our input is `%off` and we're already subscribed (i.e. `val`
2029−is `%.n`), then we unsubscribe from `:source` and set `val` back to `%.y` (`&`),
2030−again using our handy inverted cell constructor mold `:_`:
2034−~[[ost.bow %pull /subscribe [our.bow %source] ~]]
2037−It's important to send over the same bone and wire (`/subscribe`) as the
2038−one we originally subscribed on.
2040−If neither of these cases are true, then we print our current subscription
2041−state, based on whether `val` is `%.y` or `%.n`, and return a cell containing
2042−a null list of moves and our unchanged app state:
2046− sink+unsubscribed+'You are now unsubscribed!'
2047− sink+subscribed+'You are now subscribed!'
2051−`++diff-noun` is called when we get a `%diff` update along a subscription with a
2052−mark of `noun`. `++diff-noun` is given the wire that we originally passed with
2053−the `%peer` subscription request along and the data we got back. In our case we
2054−just print out the data:
2057−~& sink+received-data+'You got something!'
2061−`++reap` is called when we receive an acknowledgment as to whether the
2062−subscription was handled successfully. You can remember that `++reap` is the
2063−counterpart to `++peer` as it's pronounced like 'peer' backwards. Similarly,
2064−`coup` is similar to 'poke' backwards.
2066−Moving forward, `++reap` is given the wire we attempted to subscribe over,
2067−possibly along with an error message in cases of failure. `(unit type)` means
2068−"either `~` or `[~ type]`, which means it's used like Haskell's "maybe" or C's
2069−nullability. If `err` is `~`, then the subscription was successful and we tell
2070−that to the user. Otherwise, we print out the error message.
2075−We've done purely functional web pages as hook files, and we've
2076−done command line apps, so now let's interact with an app over
2077−the web. The fundamental concepts are no different than from the
2078−command line -- you send "poke" messages to apps and subscribe to
2079−data streams. The only difference is that you do it with
2082−Let's create a small web app that counts the number of times you
2083−poke it. There'll be a button to poke the app, and line of text
2084−below it saying how many times the app has been poked. We can
2085−update this by subscribing our page to the relevant path on its app.
2087−Save the following as `click.hoon` in the `/app` directory of your urbit's pier.
2093−++ move [bone %diff %mark *]
2096−|_ [hid=bowl:gall clicks=@]
2099− ~& [%poked +(clicks)]
2100− :_ +>.$(clicks +(clicks))
2101− %+ turn (prey:pubsub:userlib /the-path hid)
2102− |=([o=bone *] [o %diff %clicks +(clicks)])
2105− [[[ost.hid %diff %clicks clicks] ~] +>.$]
2109−There's nothing really new here, except that we use a couple of
2110−new marks, `click` and `clicks`. When we get poked with a
2111−`click`, we increment the variable `clicks` in our state and tell
2112−all our subscribers the new value. When someone subscribes to us
2113−on the path `/the-path`, we immediately give them the current
2116−Let's take a look at the new marks. Save this as `/mar/click.hoon`:
2122− ++ noun |=(* %click)
2125− ?> =('click' (need (so:dejs-soft:format jon)))
2131−The mark `click` has type `%click`, which means the only
2132−valid value is `%click`. We can convert from `noun` by just
2135−We also convert from json by parsing a json string with `so:dejs-soft:format`,
2136−asserting the parsing succeeded with `need`, and asserting the
2137−result was `'click'` with `?>`, which asserts that its first child
2140−> Note the argument to `++json`is `jon=^json`. Why `^json`? `++json` shadows the type definition, so if we want to refer to the type, we have to prepend a `^`. This extends to multiple levels: `^^^foo` means the fourth most innermost instance of `foo`.
2142−> `dejs-soft:format` in `zuse` is a useful library for parsing complex json into hoon structures. In this case, the `:` between `dejs-soft` and `format` means 'inside of', because `dejs-soft` is an arm contained within the core `format`. Our case is actually simple enough that the `?>` line could have been `?> =([%s 'click'] jon)`.
2144−We can test this mark from the command line (don't forget to start your app with `|start %click`)
2147−~fintud-macrep:dojo> &click %click
2150−~fintud-macrep:dojo> &click &json [%s 'click']
2153−~fintud-macrep:dojo> &click &json &mime [/text/json (as-octs:mimes:html '"click"')]
2156−~fintud-macrep:dojo> &click &json &mime [/text/json (as-octs:mimes:html '"clickety"')]
2157−/~fintud-macrep/home/0/mar/click:<[7 5].[8 11]>
2158−ford: casting %json to %click
2162−And save the following as `/mar/clicks.hoon`:
2173− (frond:enjs:format %clicks (numb:enjs:format clicks))
2178−`clicks` is just an atom. We convert to json by creating an
2179−object with a single field "clicks" with our value.
2181−> Be sure to check out section 3bD, JSON and XML, in `zuse.hoon`. `frond:enjs:format` is just a function that takes a key-value pair and produces a JSON object with one element.
2184−~fintud-macrep:dojo> &json &clicks 6
2185−[%o p={[p='clicks' q=[%n p=~.6]]}]
2188−Now that we know the marks involved, take another look at the app
2189−above. Everything should be pretty straightforward. Let's poke
2190−the app from the command line.
2193−~fintud-macrep:dojo> |start %click
2195−~fintud-macrep:dojo> :click &click %click
2198−~fintud-macrep:dojo> :click &click %click
2205−- Modify `:sink` from the subscriptions chapter to listen to `:click` and print out the subscription updates on the command line.
2209−> Note: the instructions below are outdated and do not work in the current version of Urbit. Feel free to experiment though.
2211−That's all that's needed for the back end. The front end is just
2212−some "sail" html (Hoon markup for XML) and javascript. Here's `/web/pages/click.hoon`:
2217− ;script(type "text/javascript", src "//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.min.js");
2218− ;script(type "text/javascript", src "/~/at/lib/js/urb.js");
2223− ;input##go(type "button", value "Poke!");
2224− ;div##err(class "disabled");
2227− ;script(type "text/javascript", src "/pages/click/click.js");
2232−You should recognize the sail syntax from an earlier chapter.
2233−Aside from jquery, we also include `urb.js`, which is a framework
2234−for interacting with urbit.
2236−To view the frontend, point your browser at `ship-name.urbit.org/~~/pages/click` if you're on the live network. If you're running a fake galaxy, navigate to whatever port it's running on, which is usually: `https://localhost:80/~~/pages/click`.
2238−We have a button labeled "Poke!" and a div with id `clicks` where
2239−we'll put the number of clicks. We also include a small
2240−javascript file where the client-side application logic can be
2241−found. It's in `/web/pages/click.js`:
2245− var clicks, $go, $clicks, $err
2248− $clicks = $('##clicks')
2254− "click", {mark: "click-click"}
2255− ,function(err,res) {
2257− return $err.text("There was an error. Sorry!")
2258− if(res.data !== undefined &&
2259− res.data.ok !== undefined &&
2260− res.data.ok !== true)
2261− $err.text(res.data.res)
2263− console.log("poke succeeded");
2267− window.urb.appl = "click"
2268− window.urb.bind('/the-path',
2269− function(err,dat) {
2270− clicks = dat.data.clicks
2271− $clicks.text(clicks)
2277−We set up two event handlers. When we click the button, we run
536+### +on-agent
2278−`window.urb.send`, which sends a poke to the app specified as the first argument, in this case our `:click` app. The arguments
2279−are data, parameters, and the callback. The data is the specific
2280−data we want to send. The parameters are all optional, but here's a list
2281−of the available ones:
2283−- `ship`: target urbit. Defaults to `window.urb.ship`, which
538+This arm is called to handle responses to `%pass` moves to other agents.
2284− defaults to the urbit which served the page.
539+It will be one of the following types of response:
2285−- `appl`: target app. Defaults to `window.urb.appl`, which
2287−- `mark`: mark of the data. Defaults to `window.urb.send.mark`,
2288− which defaults to "json".
2289−- `wire`: wire of the poke. Defaults to `/`.
2291−In our case, we specify only that the mark is "click".
541+- `%poke-ack`: acknowledgment (positive or negative) of a poke. If the
542+ value is `~`, then the poke succeeded. If the value is `[~ tang]`,
543+ then the poke failed, and a printable explanation (e.g. a stack trace)
544+ is given in the `tang`.
2293−The callback function is called when we receive an
546+- `%watch-ack`: acknowledgment (positive or negative) of a subscription.
2294−acknowledgment. If there was an error, we put it in the `err`
547+ If negative, the subscription is already ended (technically, it never
2295−div that we defined above. Otherwise, we printf to the console a
548+ started).
2296−message saying the poke succeeded. As is common, we gray
2297−out the button when it is clicked. It is only reenabled when positive acknowledgement is received.
2299−Our second event handler is `window.urb.bind`, which is called immediately when the page is loaded, subscribing the page to the specified data stream on the app set with `urb.appl` (which is, in this case, `:click`). It also takes three arguments: path, parameters, and callback. The path is the path on the app to subscribe to. The parameters are all optional (indeed, we omit them entirely here), but are similar to those for `send`:
550+- `%fact`: update from the publisher.
2301−- `ship`: target urbit. Defaults to `window.urb.ship`, which defaults to the urbit which served the page.
552+- `%kick`: notification that the subscription has ended.
2302−- `appl`: target app. Defaults to `window.urb.appl`, which defaults to null.
2303−- `mark`: mark of expected data. Data of other marks is converted to this mark on the server before coming to the web front end. Defaults to `window.urb.bind.mark`, which defaults to "json".
2304−- `wire`: wire of the subscription. Defaults to the given path.
2306−The callback function here just updates the data in the `clicks`
554+### +on-arvo
2307−div. A truly robust app would intelligently handle errors, of
2310−Note that the app doesn't have anything web-specific in itself.
556+This arm is called to handle responses to `%pass` `move`s to vanes. The
2311−As far as it knows, it's just receiving pokes and subscriptions.
557+list of possible responses from the system is statically defined in
2312−The javascript is fairly pure as well, sending and receiving json
558+sys/zuse.hoon (grep for `+ sign-arvo`).
2313−everywhere. The marks are the translation layer, and they're the
2314−only things that need to know how the hoon types map to json.
2316−**Exercise**:
560+### +on-fail
2318−- Open the app in multiple tabs, click the button, and verify that all the tabs stay in sink. Poke it manually from the command line and verify the tabs are updated as well.
562+If an error happens in `+on-poke`, the crash report goes into the
563+`%poke-ack` response. Similarly, if an error happens in
564+`+on-subscription`, the crash report goes into the `%watch-ack`
565+response. If a crash happens in any of the other handlers, the report
566+is passed into this arm.