@@ -4,2315 +4,563 @@ weight = 6
44template = "doc.html"
55aliases = ["/docs/learn/arvo/gall/"]
66+++
7−This is documentation about how to write Gall apps. Some knowledge of Hoon is assumed.
87
9−> Note: the last major revision of these lessons was in 2016. They need to be
10− rewritten; in some ways they are outdated, and in other ways they were not
11− always accessible enough to beginners. We are planning to rewrite these in the
12− near future. There is still useful information to be found in the meantime.
13−
14−## API Connectors
15−
16−> Note: This lesson depends on some code that needs to be updated before the examples will work correctly. This will be addressed in the near future. We apologize!
17−
18−Most people have lots of data stored in online services, many of
19−which have APIs. API connectors allow the user to access this
20−data from within their Urbit.
21−
22−A security driver allows the user to make authenticated requests
23−to the service's API, but the user still needs to know the API
24−and send and receive JSON. An API connector puts a layer of
25−porcelain over this to allow the user easier control over their
26−data.
27−
28−Each connector should perform two basic functions. First, it
29−should expose a tree of the data in the service that's accessible
30−through `.^` and FUSE. Second, it should expose event streams as
31−actions occur in the service. Let's take a look at how the `%gh`
32−app accomplishes both for Github.
33−
34−After starting `%gh` (`|start %gh`), let's look at the root of
35−the tree that `%gh` exposes:
36−
37−```
38−~your-urbit:dojo> .^(arch %gy /=gh=)
39−[fil=~ dir={[p=~.issues q=~]}]
40−```
41−
42−`%gh` is currently a skeleton -- it contains examples of all the
43−functionality necessary for a connector, but many endpoints
44−aren't implemented. As we can see here, only issues are
45−implemented. We can explore this tree:
46−
47−```
48−~your-urbit:dojo> .^(arch %gy /=gh=/issues)
49−[fil=~ dir={[p=~.by-repo q=~] [p=~.mine q=~]}]
50−```
51−
52−And eventually:
53−
54−```
55−/- gh
56−.^(issue:gh %gx /=gh=/issues/by-repo/philipcmonktest/testing/7)
57−<issue...>
58−login.user:.^(issue:gh %gx /=gh=/issues/by-repo/philipcmonktest/testing/7)
59−'philipcmonktest'
60−```
61−
62−The other thing an API connector needs to do is expose event
63−streams. One straightforward way to use this capacity is with
64−the `:pipe` app:
65−
66−```
67−:pipe|connect %gh /listen/philipcmonktest/testing/issues/'issue_comment' %public
68−```
69−
70−Now creating an issue results in the following message in your
71−`%public` `:talk` channel:
72−
73−```
74−~your-urbit[philipcmonktest@github]: opened issue ##11: i found a bug!
75−```
76−
77−You'll also get notifications of issue status changes and
78−comments.
79−
80−As is often the case, the best way to write an API connector is
81−to copy an existing one and modify it. We'll dissect the Github
82−connector here. This code is in `/=home=/app/gh/hoon`, and it's
83−also reproduced at the bottom of this page (in case the code
84−drifts out of sync with this doc).
85−
86−We'll go over the two parts to an API connector (one-time reading vs listening
87−for events) separately.
88−
89−#### Reading
90−
91−A connector exposes a tree of data. Every read request has a
92−`care`, which is generally either `%x` or `%y`. `%x` is a request
93−for a particular piece of data, while `%y` is a request for a
94−directory listing. Roughly, `%x` means Unix `cat` and `%y` means
95−Unix `ls`.
96−
97−> Sometimes you wish to expose a tree where a part of the path can't be enumerated. For example, a `%y` of `/issues/by-repo` "should" produce a list of all Github users, but we don't do that because it's too long. Instead, we just produce our own username (from the `web.plan` file). You can still access repos from other users, you just don't see them in the directory listing.
98−
99−The usual flow for implementing this tree of data makes heavy use
100−of the `connector` library. This library is well documented in
101−the source, so check out `/=home=/lib/connector/hoon`.
102−
103−The `connector` library needs to be initialized with definitions
104−of `move` and `sub-result` (all the types of data that can be
105−returned by a place). This can just be a line at the top of the
106−main core:
107−
108−```hoon
109−=+ connector=(connector move sub-result) :: Set up connector library
110−```
111−
112−Most of the Github-specific logic is in `++places`, which is a
113−list of all the places we can request. A place consists of:
8+This document describes Gall, the Arvo userspace vane. It is accurate as of 2019.12.02.
9+
10+# Agents
11+
12+Urbit code lives in the following basic categories:
13+
14+- Runtime (Nock interpreter, persistence engine, IO drivers, jets)
15+- Kernel vanes (managed by Arvo)
16+- Userspace agents (managed by Gall, permanent state)
17+- Userspace imps (managed by Spider, transient state)
18+
19+This lays out the framework for the third category: Agents.
20+
21+An agent is a piece of software that is primarily focused on maintaining
22+and distributing a piece of state with a defined structure. It exposes
23+an interface that lets programs read, subscribe to, and manipulate the
24+state. Every event happens in an atomic transaction, so the state is
25+never inconsistent. Since the state is permanent, when the agent is
26+upgraded with a change to the structure of the state, the developer
27+provides a migration function from the old state type to the new state
28+type.
29+
30+It's not too far off to think of an agent as simply a database with
31+developer-defined logic. But an agent is significantly less constrained
32+than a database. Databases are usually tightly constrained in one or
33+more ways because they need to provide certain guarantees (like
34+atomicity) or optimizations (like indexes). Urbit is a single-level
35+store, so atomicity comes for free. Many applications don't use
36+databases because they need relational indices; rather, they use them
37+for their guarantees around persistence. Some do need the indices,
38+though, and it's not hard to imagine an agent which provides a SQL-like
39+interface.
40+
41+On the other hand, an agent is also a lot like what many systems call a
42+"service". An agent is permanent and addressable -- a running program
43+can talk to an agent just by naming it. An agent can perform IO, unlike
44+most databases. This is a critical part of an agent: it performs IO
45+along the same transaction boundaries as changes to its state, so if an
46+effect happens, you know that the associated state change has happened.
47+You should be careful, though, to avoid creating implicit non-atomicity
48+-- eg if it comes into an inconsistent state pending a response to an IO
49+action.
50+
51+But the best way to think about an agent is as a state machine. Like a
52+state machine, any input could happen at any time, and it must react
53+coherently to that input. Ouput (effects) and the next state of the
54+machine are both pure functions of the previous state and the input
55+event. Of course, it's important to ensure there isn't an order of
56+events that could cause your agent to enter an inconsistent state.
57+
58+We often think of state machines as finite state machines, but of course
59+in this instance, the state is not actually finite (though it should be
60+definable in a regular recursive data type).
61+
62+## Specification
63+
64+An agent is defined as a core with a set of arms to handle various
65+events. These handlers usually produce a list of effects and the next
66+state of the agent. The interface definition can be found in
67+`sys/zuse.hoon`, which at the time of writing is:
11468
11569```hoon
116−++ place
117− $: guard/mold
118− read-x/$-(path move)
119− read-y/$-(path move)
120− sigh-x/$-(jon/json (unit sub-result))
121− sigh-y/$-(jon/json (unit arch))
122− ==
123−```
124−
125−- `guard`, the type of the paths we should match. For example, to match `/issues/<user>/<repo>` use `{$issues @t @t $~}`.
126−
127−- `read-x`, called when someone tries to read the place with `care` `%x`. Should produce a single move, usually either a `%diff` response if we can immediately answer or a `%hiss` http request if we need to make a request to the api. See the `++read-*` functions in `++helpers` for some common handlers.
128−
129−- `read-y`, same as `read-x` except with care `%y`.
130−
131−- `sigh-x`, called when an http response comes back on this place. You're given the json of the result, and you should produce either a result or null. Null represents an error. If you didn't create an http request in `read-x`, then this should never be called. Use `++sigh-strange` from `++helpers` to unconditionally signal an error.
132−
133−- `sigh-y`, same as `sigh-x` except with care `%y`. Note that a `%y` request must produce an arch, unlike a `%x` request, which may produce data of any mark.
134−
135−Filling out the list of places is a lot of grunt work, but most
136−places are fairly straightforward, and the `++read-*` helper
137−functions are useful. Check out the library source for more
138−information on those.
139−
140−Besides the list of places, we just need to handle the flow of
141−control. `++peek` and `++peer-scry` are the interface we expose
142−to the rest of the system while `++sigh-httr` and `++sigh-tang`
143−are used to handle responses when we make HTTP requests.
144−
145−- `++peek` should usually just produce `~`. If there are cases where we can respond directly to a `.^` request without blocking on anything, we could do it here, but it's generally not worth the hassle since the same logic should be duplicated in `++peer-scry`.
146−
147−- `++peer-scry` is where the actual handling for a read request goes. In general, we just need to call `++read` from the `connector` library with the bone, list of places, care, and path. This will match the path to the appropriate place and run either `++read-x` or `++read-y`, depending on the care.
148−
149−- If `++read-x` or `++read-y` made a successful API request, then the response will come back on `++sigh-httr`. Here, we just need to parse out the wire and call `++sigh` from the `connector` library with the list of places, care, path, and HTTP result. This will match the path to the appropriate place and run either `++sigh-x` or `++sigh-y`, depending on the care.
150−
151−- If `++read-x` or `++read-y` made an API request that failed, then we'll get a stack trace in `++sigh-tang`. Here, we just print it out and move on.
152−
153−That's really all there is to the reading portion of API
154−connectors.
155−
156−One of the most accessible ways to jump into Arvo programming is
157−to just add more places to an existing API connector. It's
158−useful, small in scope, and comes in bite-sized chunks since most
159−places are less than ten lines of code.
160−
161−#### Listening
162−
163−Listening for events is fairly service-specific. In some
164−services, we poll for changes. The Twitter connector has an
165−example of this, but note that it predates the `connector`
166−library and is thus more complicated than it needs to be, and the
167−interface it exposes isn't standard. In Github, we power our
168−event streams with webhooks.
169−
170−For Github, when someone subscribes to
171−`/listen/<user>/<repo>/<events...>`, we want to produce
172−well-typed results when they occur.
173−
174−We only want to create one webhook per event, so in our state we
175−have `hook`, which is a map of event names to the set of bones
176−which are subscribed to that event. We must always keep this
177−up-to-date.
178−
179−Flow of control starts in `++peer-listen`, where we just call
180−`++listen`. In `++listen`, for each event in the list of events,
181−we check to see if we have that hook set up (by checking whether
182−it exists in `hook`). If so, we call `++update-hook` to add the
183−current bone to the set of listeners. Otherwise, we call
184−`++create-hook`, which sends a request to Github to set up the
185−new webhook. We also create an entry in `hook` with the current
186−bone.
187−
188−When we created the webhook, we told Github to send the event to
189−`/~/to/gh/gh-<event>.json?anon&wire=/`. This turns into a poke.
190−Let's parse out this url. The first `gh` is the app name to
191−poke. The next portion, `gh-<event>` is the mark to convert the
192−JSON to. For example, when the "issues" event fires, it'll get
193−converted to mark `gh-issues`. This mark has a conversion
194−routine from `json` to the `issues` type, which is defined in the
195−`gh` structure (`/=home=/sur/gh/hoon`). Once this conversion
196−happens, the `gh` app will get poked with the correctly marked
197−data in `++poke`.
198−
199−In `++poke`, we check `hook` to find the subscribers to the event
200−that was fired, and we update them.
201−
202−That's really all there is to it. Webhook flow isn't well
203−standardized, so even if your chosen service is powered by
204−webhooks, your listening code might look rather different. The
205−point is to expose the correct interface.
206−
207−
208−#### Github API Connector Code
209−
210−In case the code in `/=home=/app/gh/hoon` drifts out of sync with
211−this doc:
212−
213−```
214−:: This is a connector for the Github API v3.
215−::
216−:: You can interact with this in a few different ways:
217−::
218−:: - .^({type} %gx /=gh={/endpoint}) to read data or
219−:: .^(arch %gy /=gh={/endpoint}) to explore the possible
220−:: endpoints.
221−::
222−:: - subscribe to /listen/{owner}/{repo}/{events...} for
223−:: webhook-powered event notifications. For event list, see
224−:: https://developer.github.com/webhooks/.
225−::
226−:: This is written with the standard structure for api
227−:: connectors, as described in lib/connector.hoon.
228−::
229−/? 314
230−/- gh, plan-acct
231−/+ gh-parse, connector
232−::
233−!:
234−=> |%
235− +$ move (pair bone card)
236− +$ card
237− $% [%diff sub-result]
238− [%them wire (unit hiss:eyre)]
239− [%hiss wire [%~ %~] %httr [%hiss hiss:eyre]]
240− ==
70+++ agent
71+ =< form
72+ |%
73+ +$ step (quip card form)
74+ +$ card (wind note gift)
75+ +$ note
76+ $% [%arvo =note-arvo]
77+ [%agent [=ship name=term] =task]
78+ ==
79+ +$ task
80+ $% [%watch =path]
81+ [%watch-as =mark =path]
82+ [%leave ~]
83+ [%poke =cage]
84+ [%poke-as =mark =cage]
85+ ==
86+ +$ gift
87+ $% [%fact path=(unit path) =cage]
88+ [%kick path=(unit path) ship=(unit ship)]
89+ [%watch-ack p=(unit tang)]
90+ [%poke-ack p=(unit tang)]
91+ ==
92+ +$ sign
93+ $% [%poke-ack p=(unit tang)]
94+ [%watch-ack p=(unit tang)]
95+ [%fact =cage]
96+ [%kick ~]
97+ ==
98+ ++ form
99+ $_ ^|
100+ |_ bowl
101+ ++ on-init
102+ *(quip card _^|(..on-init))
103+ ::
104+ ++ on-save
105+ *vase
241106 ::
242− :: Types of results we produce to subscribers.
107+ ++ on-load
108+ |~ old-state=vase
109+ *(quip card _^|(..on-init))
243110 ::
244− +$ sub-result
245− $% [%arch arch]
246− [%gh-issue issue:gh]
247− [%gh-list-issues (list issue:gh)]
248− [%gh-issues issues:gh]
249− [%gh-issue-comment issue-comment:gh]
250− [%json json]
251− [%null %~]
252− ==
111+ ++ on-poke
112+ |~ [mark vase]
113+ *(quip card _^|(..on-init))
253114 ::
254− :: Types of webhooks we expect.
115+ ++ on-watch
116+ |~ path
117+ *(quip card _^|(..on-init))
255118 ::
256− +$ hook-response
257− $% [%gh-issues issues:gh]
258− [%gh-issue-comment issue-comment:gh]
259− ==
119+ ++ on-leave
120+ |~ path
121+ *(quip card _^|(..on-init))
122+ ::
123+ ++ on-peek
124+ |~ path
125+ *(unit (unit cage))
126+ ::
127+ ++ on-agent
128+ |~ [wire sign]
129+ *(quip card _^|(..on-init))
130+ ::
131+ ++ on-arvo
132+ |~ [wire sign-arvo]
133+ *(quip card _^|(..on-init))
134+ ::
135+ ++ on-fail
136+ |~ [term tang]
137+ *(quip card _^|(..on-init))
260138 --
261−=+ connector=(connector move sub-result) :: Set up connector library
262−::
263−|_ $: hid=bowl:gall
264− hook=(map @t [id=@t listeners=(set bone)]) :: map events to listeners
265− ==
266−:: ++ prep _`. :: Clear state when code changes
267−::
268−:: List of endpoints
269−::
270−++ places
271− |= wir=wire
272− ^- (list place:connector)
273− =+ (helpers:connector ost.hid wir "https://api.github.com")
274− => |% :: gh-specific helpers
275− ++ sigh-list-issues-x
276− |= jon=json
277− %+ bind ((ar:jo issue:gh-parse) jon)
278− |= issues=(list issue:gh)
279− gh-list-issues+issues
280− ::
281− ++ sigh-list-issues-y
282− |= jon=json
283− %+ bind ((ar:jo issue:gh-parse) jon)
284− |= issues=(list issue:gh)
285− :- `(shax (jam issues))
286− (malt (turn issues |=(issue:gh [(rsh 3 2 (scot %ui number)) ~])))
287− --
288− :~ ^- place :: /
289− :* guard=,%~
290− read-x=read-null
291− read-y=(read-static %issues ~)
292− sigh-x=sigh-strange
293− sigh-y=sigh-strange
294− ==
295− ^- place :: /issues
296− :* guard=,[%issues %~]
297− read-x=read-null
298− read-y=(read-static %mine %by-repo ~)
299− sigh-x=sigh-strange
300− sigh-y=sigh-strange
301− ==
302− ^- place :: /issues/mine
303− :* guard=,[%issues %mine %~]
304− read-x=(read-get /issues)
305− read-y=(read-get /issues)
306− sigh-x=sigh-list-issues-x
307− sigh-y=sigh-list-issues-y
308− ==
309− ^- place :: /issues/by-repo
310− :* guard=,[%issues %by-repo %~]
311− read-x=read-null
312− ^= read-y
313− |= pax=path
314− =+ /(scot %p our.hid)/home/(scot %da now.hid)/web/plan
315− =+ .^([* acc=(map knot plan-acct)] %cx -)
316− ::
317− ((read-static usr:(~(got by acc) %github) ~) pax)
318− sigh-x=sigh-strange
319− sigh-y=sigh-strange
320− ==
321− ^- place :: /issues/by-repo/<user>
322− :* guard=,[%issues %by-repo @t %~]
323− read-x=read-null
324− read-y=|=(pax=path (get /users/[-.+>.pax]/repos))
325− sigh-x=sigh-strange
326− ^= sigh-y
327− |= jon=json
328− %+ bind ((ar:jo repository:gh-parse) jon)
329− |= repos=(list repository:gh)
330− [~ (malt (turn repos |=(repository:gh [name ~])))]
331− ==
332− ^- place :: /issues/by-repo/<user>/<repo>
333− :* guard=,[%issues %by-repo @t @t %~]
334− read-x=|=(pax=path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
335− read-y=|=(pax=path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
336− sigh-x=sigh-list-issues-x
337− sigh-y=sigh-list-issues-y
338− ==
339− ^- place :: /issues/by-repo/<user>/<repo>
340− :* guard=,[%issues %by-repo @t @t @t %~]
341− ^= read-x
342− |=(pax=path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues/[-.+>+>.pax]))
343− ::
344− read-y=(read-static ~)
345− ^= sigh-x
346− |= jon=json
347− %+ bind (issue:gh-parse jon)
348− |= issue=issue:gh
349− gh-issue+issue
350− ::
351− sigh-y=sigh-strange
352− ==
353− ==
354−::
355−:: When a peek on a path blocks, ford turns it into a peer on
356−:: /scry/{care}/{path}. You can also just peer to this
357−:: directly.
358−::
359−:: We hand control to ++scry.
360−::
361−++ peer-scry
362− |= pax=path
363− ^- [(list move) _+>.$]
364− ?> ?=([care:clay *] pax)
365− :_ +>.$ :_ ~
366− (read:connector ost.hid (places %read pax) i.pax t.pax)
367−::
368−:: HTTP response. We make sure the response is good, then
369−:: produce the result (as JSON) to whoever sent the request.
370−::
371−++ sigh-httr
372− |= [way=wire res=httr:eyre]
373− ^- [(list move) _+>.$]
374− ?. ?=([%read care:clay @ *] way)
375− ~& res=res
376− [~ +>.$]
377− =* style i.way
378− =* ren i.t.way
379− =* pax t.t.way
380− :_ +>.$ :_ ~
381− :+ ost.hid %diff
382− (sigh:connector (places ren style pax) ren pax res)
383−::
384−:: HTTP error. We just print it out, though maybe we should
385−:: also produce a result so that the request doesn't hang?
386−::
387−++ sigh-tang
388− |= [way=wire tan=tang]
389− ^- [(list move) _+>.$]
390− %- (slog >%gh-sigh-tang< tan)
391− [[ost.hid %diff null+~]~ +>.$]
392−::
393−:: We can't actually give the response to pretty much anything
394−:: without blocking, so we just block unconditionally.
395−::
396−++ peek
397− |= [ren=@tas tyl=path]
398− ^- (unit (unit (pair mark *)))
399− ~ ::``noun=[ren tyl]
400−::
401−::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
402−:: Webhook-powered event streams (/listen) ::
403−::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
404−::
405−:: To listen to a webhook-powered stream of events, subscribe
406−:: to /listen/<user>/<repo>/<events...>
407−::
408−:: We hand control to ++listen.
409−::
410−++ peer-listen
411− |= pax=path
412− ^- [(list move) _+>.$]
413− ?. ?=([@ @ *] pax)
414− ~& [%bad-listen-path pax]
415− [~ +>.$]
416− (listen pax)
417−::
418−:: This core handles event subscription requests by starting or
419−:: updating the webhook flow for each event.
420−::
421−++ listen
422− |= pax=path
423− =| mow=(list move)
424− =< abet:listen
425− |%
426− ++ abet :: Resolve core.
427− ^- [(list move) _+>.$]
428− [(flop mow) +>.$]
429− ::
430− ++ send-hiss :: Send a hiss
431− |= hiz=hiss:eyre
432− ^+ +>
433− =+ wir=`wire`[%x %listen pax]
434− +>.$(mow [[ost.hid %hiss wir `~ %httr [%hiss hiz]] mow])
435− ::
436− :: Create or update a webhook to listen for a set of events.
437− ::
438− ++ listen
439− ^+ .
440− =+ pax=pax :: TMI-proofing
441− ?> ?=([@ @ *] pax)
442− =+ events=t.t.pax
443− |- ^+ +>+.$
444− ?~ events
445− +>+.$
446− ?: (~(has by hook) i.events)
447− $(+>+ (update-hook i.events), events t.events)
448− $(+>+ (create-hook i.events), events t.events)
449− ::
450− :: Set up a webhook.
451− ::
452− ++ create-hook
453− |= event=@t
454− ^+ +>
455− ?> ?=([@ @ *] pax)
456− =+ clean-event=`tape`(turn (trip event) |=(a=@tD ?:(=('_' a) '-' a)))
457− =. hook
458− %+ ~(put by hook) (crip clean-event)
459− =+ %+ fall
460− (~(get by hook) (crip clean-event))
461− *[id=@t listeners=(set bone)]
462− [id (~(put in listeners) ost.hid)]
463− %- send-hiss
464− :* %+ scan
465− =+ [(trip i.pax) (trip i.t.pax)]
466− "https://api.github.com/repos/{-<}/{->}/hooks"
467− auri:de-purl:html
468− %post ~ ~
469− %- as-octs:mimes:html %- crip %- en-json:html %- pairs:enjs:format :~
470− name+s+%web
471− active+b+&
472− events+a+~[s+event] ::(turn `(list ,@t)`t.t.pax |=(a=@t s=a))
473− :- %config
474− %- pairs:enjs:format :~
475− =+ =+ clean-event
476− "http://107.170.195.5:8443/~/to/gh/gh-{-}.json?anon&wire=/"
477− [%url s+(crip -)]
478− [%'content_type' s+%json]
479− ==
480− ==
481− ==
482− ::
483− :: Add current bone to the list of subscribers for this event.
484− ::
485− ++ update-hook
486− |= event=@t
487− ^+ +>
488− =+ hok=(~(got by hook) event)
489− %_ +>.$
490− hook
491− %+ ~(put by hook) event
492− hok(listeners (~(put in listeners.hok) ost.hid))
493− ==
494− --
495−::
496−:: Pokes that aren't caught in more specific arms are handled
497−:: here. These should be only from webhooks firing, so if we
498−:: get any mark that we shouldn't get from a webhook, we reject
499−:: it. Otherwise, we spam out the event to everyone who's
500−:: listening for that event.
501−::
502−++ poke
503− |= response=hook-response
504− ^- [(list move) _+>.$]
505− =+ hook-data=(~(get by hook) (rsh 3 3 -.response))
506− ?~ hook-data
507− ~& [%strange-hook hook response]
508− [~ +>.$]
509− :: ~& response=response
510− :_ +>.$
511− %+ turn ~(tap in listeners.u.hook-data)
512− |= ost=bone
513− [ost %diff response]
514−--
515139```
516140
517−## Generators
518−
519−Up until now we've poked apps directly. This requires the user to
520−specify the mark, and it requires the app to accept the arguments in a
521−way that's convenient for users to input. This is the "plumbing" way to
522−interact with apps. Generators are the "porcelain" layer. This is why
523−when you run a command like `+ls` or `|merge`, there are no marks in
524−sight.
525−
526−At that point, we just used the
527−generators to produce values -- we didn't pipe their results into apps.
528−In the dojo cast, the role of a generator is to take a list of arguments
529−and produce a value, which is often, though not always, piped into an
530−app. Generators are pure, stateless functions, and they cannot send
531−moves.
532−
533−`+ls` and `+cat` are commonly-used generators that produce a directory
534−listing and print out a file, respectively. These are useful in
535−themselves, so we generally don't pipe the results into an app.
536−
537−Another generator is `+hood/merge`. Try running `+hood/merge` with the
538−normal arguments for `|merge`:
539−
540−```
541−> +hood/merge %home our %they
542−[syd=%home her=~zod sud=%they cas=[%da p=~2018.10.4..21.11.49..901c] gem=%auto]
543−
544−> +hood/merge %home our %they, =gem %this
545−[syd=%home her=~zod sud=%they cas=[%da p=~2018.10.4..21.12.11..ee32] gem=%this]
546−```
547−
548−This didn't run any merge, it just constructed the command that, if sent
549−to `:hood`, would run it. Note first that, even though it's not printed,
550−this value has mark `kiln-merge`. Also, the merge strategy `gem=%auto`
551−was added automatically. Optional arguments are straightforward with
552−generators.
553−
554−In general, generators are prepended by either `+` or `|`. The general
555−form is `+generator`, but often generators are created specifically to
556−work with a single app, they're usually placed in a subdirectory of gen
557−and run with `|`. Thus, if `my-generator` generator is made for use with
558−`:my-app`, then `my-generator` is put in
559−`/gen/my-app/my-generator.hoon`, and can be run directly with
560−`+my-app/my-generator <args>`. If you want to pipe the result into
561−`:my-app`, then run `:my-app +my-app/my-generator <args>`. Because this
562−pattern is so common, this can be abbreviated to
563−`:my-app|my-generator <args>`. Because most built-in commands are
564−generators for the `:hood` app, `:hood|generator <args>` can be
565−shortened to `|generator <args>`.
566−
567−Let's write a generator called `:pong` that takes an urbit address, which is of
568−mark `urbit`, and sends that urbit the message `'howdy'`. First--without a
569−generator--let's make `:ping` that does the same, except that
570−it lets the user optionally specify the message as well.
571−
572−We'll need a new mark for our arguments. Let's call it
573−`ping-message`.
574−
575−> For app-specific marks, it's good style to prefix the name of the mark with the name of the app. Since many apps have several such marks, subdirectories in `/mar` are rendered as `-`, so that `ping-message` is written in (/mar/ping/message.hoon).
141+Here's a skeleton example of an implementation:
576142
577143```hoon
578−::
579−:::: /mar/ping/message/hoon
580− ::
581−/? 314
582−|_ [to=@p message=@t]
583−++ grab
584− |%
585− +$ noun [@p @t]
586− --
144+^- agent:gall
145+=| state=@
146+|_ bowl:gall
147++* this .
148+++ on-init
149+ `this
150+::
151+++ on-save
152+ !>(state)
153+::
154+++ on-load
155+ |= =old-state=vase
156+ =/ old-state !<(@ old-state-vase)
157+ ?~ old-state
158+ ~& %prep-lost
159+ `this
160+ ~& %prep-found
161+ `this(state u.old-state)
162+::
163+++ on-poke
164+ |= [=mark =vase]
165+ ~& state=state
166+ ~& got-poked-with-data=mark]
167+ =. state +(state)
168+ `this
169+::
170+++ on-watch
171+ |= path
172+ `this
173+::
174+++ on-leave
175+ |= path
176+ `this
177+::
178+++ on-peek
179+ |= path
180+ *(unit (unit cage))
181+::
182+++ on-agent
183+ |= [wire sign:agent:gall]
184+ `this
185+::
186+++ on-arvo
187+ |= [wire sign-arvo]
188+ `this
189+::
190+++ on-fail
191+ |= [term tang]
192+ `this
587193--
588194```
589195
590−The app can easily be modified to use this (`/app/ping.hoon`):
196+We also supply a `default-agent` library, which is useful if you don't
197+want to handle some of the arms. Most of the above could also be
198+implemented as:
591199
592200```hoon
593−:: Allows one ship to ping another with a string of text
594−::
595−:::: /app/ping/hoon
596− ::
597−/? 314
598−|%
599− +$ move [bone term wire *]
201+/+ default-agent
202+^- agent:gall
203+=| state=@
204+|_ =bowl:gall
205++* this .
206+ default ~(. (default-agent %|) bowl)
207+::
208+++ on-init on-init:default
209+++ on-save on-save:default
210+++ on-load on-load:default
211+++ on-poke
212+ |= [=mark =vase]
213+ ~& > state=state
214+ ~& got-poked-with-data=mark
215+ =. state +(state)
216+ `this
217+::
218+++ on-watch on-watch:default
219+++ on-leave on-leave:default
220+++ on-peek on-peek:default
221+++ on-agent on-agent:default
222+++ on-arvo on-arvo:default
223+++ on-fail on-fail:default
600224--
601−!:
602−|_ [bowl:gall state=%~]
603−::
604−++ poke-ping-message
605− |= [to=@p message=@t]
606− ~& 'sent'
607− ^- [(list move) _+>.$]
608− :- ^- (list move)
609− :~ `move`[ost %poke /sending [to dap] %atom message]
610− ==
611− +>.$
612−::
613−++ poke-atom
614− |= arg=@
615− ~& 'received'
616− ^- [(list move) _+>.$]
617− ::
618− ~& [%receiving (@t arg)]
619− [~ +>.$]
620−::
621−++ coup |=(* `+>)
622−--
623−```
624−
625−Now we can run this with:
626−
627225```
628−~fintud-macrep:dojo> |start %ping
629−>=
630226
631−~fintud-macrep:dojo> :ping &ping-message [~sampel-sipnym 'heyya']
632−>=
633−```
634−
635−And on `~sampel-sipnym`, assuming it's running `:ping` as well,
636−we see `[%receiving 'heyya']`.
637−
638−This is an annoying way to invoke the app, though. The `[]` are
639−mandatory, and optional arguments are hard. Let's make a generator,
640−`+send` to make everything nicer. Since it's specific to
641−`:ping`, let's put it in `/gen/ping/send.hoon`:
642−
643−```hoon
644−:- %say
645−|= [^ [to=@p message=?(%~ [text=@t %~])] %~]
646−[%ping-message to ?~(message 'howdy' text.message)]
647−```
648−
649−A couple of new things here. Firstly, `message=?(%~ [text=@t %~])] %~]`
650−should be read as "the message is either null or a pair of text and
651−null". Generator argument lists are always null-terminated, which makes
652−it convenient to accept lists in tail position (which are particularly
653−annoying without generators). `?(a b)` is the irregular form of
654−`$?(a b)`, which is a union type. Thus, `?(a b)` means the type of
655−anything that's in either `a` or `b`. Thus, in our case,
656−`?(~ [text=@t ~])` is either null or a pair of text and null.
227+So, an agent is a core with 10 arms. The handlers correspond to
228+different sorts of input. We'll discuss each of these in detail, but
229+first a few general concepts.
657230
658−This is run as follows:
231+## Basic concepts
659232
660−```
661−~fintud-macrep:dojo> :ping|send ~sampel-sipnym
662−>=
663−
664−~fintud-macrep:dojo> :ping|send ~sampel-sipnym 'how do you do'
665−>=
666−```
667−
668−Which causes `~sampel-sipnym` to print `[%receiving 'howdy']` and
669−`[%receiving 'how do you do']`.
233+### Interacting with an agent
670234
671−(Note: `:ping|send ~sampel-sipnym` is short for `:ping +ping/send ~sampel-sipnym`.)
235+There are two basic ways programs can interact with an agent: you may
236+subscribe to data as described above, or you may "poke" an agent to send
237+it a command. Suppose there is an agent for a calendar service; then,
238+pokes would likely include `%create-event`, `%modify-event`,
239+`%delete-event`, and `%change-time-zone` (along with assocated data).
240+Subscription paths could include `/next-event` and `/all-events`.
672241
673−**Exercises**:
242+Agents generally conform to CQRS -- pokes may change state, but
243+subscriptions generally shouldn't. In practice, there may be state
244+changes required to properly initialize the subscription, but these
245+shouldn't change the essential state of the agent.
674246
675−- Create a generator for `:sum` from [State](#state) so that you can run `:sum|add 5` to add numbers to it.
247+### Cards
676248
677−- Create a generator for `:click` from [Web Apps](#backend) so that you can run `:click|poke` to poke it.
249+Most of the handlers produce a `(quip card agent:gall)`, which is just
250+`[(list card) agent:gall]`. The first allows us to produce effects, and
251+the second allows us to maintain state.
678252
253+A card is one of two things: a `note` or a `gift`. You `%pass` `note`s
254+and you `%give` `gift`s. When you pass a `note`, you expect zero or more
255+responses; when you give a `gift`, you will not get a response. Since you
256+may get a response to a `note`, you tag it with a `wire` so that you can
257+identify the response.
679258
680−## Writing an HTTP Request
681−
682−There's a variety of arvo services we haven't touched on yet.
683−Let's figure out how to make HTTP requests and set timers.
684−
685−Let's build an uptime-monitoring system. It'll ping a web
686−service periodically and print out an error when the response
687−code isn't 2xx. Save the following as `app/up.hoon`:
259+Thus, we say you "pass a `note` along a `wire`", or you "give a `gift`".
260+These phrases correspond to producing a card that looks like:
688261
689262```hoon
690−:: Up-ness monitor. Accepts atom url, 'on', or 'off'
691−::
692−:::: /hoon/up/app
693− ::
694−/? 314
695−|%
696−+$ move [bone card]
697−+$ card
698− $% [%hiss wire unit-iden=$~ mark=%httr cage=[mark=%purl vase=purl:eyre]]
699− [%wait wire @da]
700− ==
701−+$ action
702− $% [%on $~] :: enable polling('on')
703− [%off $~] :: disable polling('off)
704− [%target p=cord] :: set poll target('http://...')
705− ==
706−--
707−|_ [hid=bowl:gall on=_| in-progress=_| target=@t]
708−++ poke-atom
709− |= url-or-command=@t ^- (quip move _+>)
710− =+ ^- act/action
711− ?: ?=($on url-or-command) [%on ~]
712− ?: ?=($off url-or-command) [%off ~]
713− [%target url-or-command]
714− ?- -.act
715− $target [~ +>.$(target p.act)]
716− $off [~ +>.$(on |)]
717− $on
718− :- ?: |(on in-progress) ~
719− [ost.hid %hiss /request ~ %httr %purl (need (de-purl:html target))]~
720− +>.$(on &, in-progress &)
721− ==
722−++ sigh-httr
723− |= [wir=wire code=@ud headers=mess:eyre body=(unit octs)]
724− ~& 'arrive here'
725− ^- [(list move) _+>.$]
726− ?: &((gte code 200) (lth code 300))
727− ~& [%all-is-well code]
728− :_ +>.$
729− [ost.hid %wait /timer (add ~s10 now.hid)]~
730− ~& [%we-have-a-problem code]
731− ~& [%headers headers]
732− ~& [%body body]
733− :_ +>.$
734− [ost.hid %wait /timer (add ~s10 now.hid)]~
735−++ wake-timer
736− |= [wir=wire $~] ^- (quip move _+>)
737− ?: on
738− :_ +>.$
739− [ost.hid %hiss /request ~ %httr %purl (need (de-purl:html target))]~
740− [~ +>.$(in-progress |)]
741−::
742−++ prep ~& target _`. :: computed when the source file changes;
743−-- :: here it prints target then resets our state
263+[%pass /my/wire a-note]
744264```
745265
746−There's some fancy stuff going on here. We'll go through it line by
747−line, though.
748−
749−There's two kinds of cards we're sending to arvo. A `%hiss` move tells
750−`%eyre` to make an HTTP request. It expects a `(unit iden)`, which will
751−be null here because we aren't doing any authentication; a mark, in this
752−case the http request mark `&httr`; and some data in the form of a
753−`cage`, which is a `[mark vase]`. This is confusing, so the mold in the
754−code quoted above includes a bunch of otherwise unnecessary faces for
755−the purpose of illustration. Compare: `[%hiss p=(unit user) q=mark
756−r=cage]` from `zuse`.
757−
758−> You can grep zuse.hoon and arvo.hoon for most of these definitions. However, beware the ambiguity surrounding "hiss": there's ++hiss in section 3bI of `zuse` ("Arvo structures"), and there's `%hiss` the move that is sent to %eyre to make our HTTP request happen. Here we're talking about the latter. See `++task:able:eyre` in `zuse`.
759−
760−> Recall that `++unit` means "maybe". Formally, "`(unit a)` is either null or a pair of null and a value in `a`". Also recall that to pull a value out of a unit (`u.unit`), you must first verify that the unit is not null (for example with `?~`). See `++need`.
761−
762−A `purl:eyre` is a parsed url structure, which can be created with `de-purl:html`,
763−which is a function that takes a url as text and parses it into a `(unit
764−purl:eyre)`. Thus, the result is null if and only if the url is malformed.
765−'Purl' is also the mark which will be applied to this result.
766−
767−When you send this request, you can expect a `%sigh` with the
768−response, which we handle later on in `++sigh-httr`.
769−
770−For `%wait`, you just pass a `@da` (absolute date), and arvo will
771−produce a `%wake` when the time comes.
772−
773−> A timer is guaranteed to not be triggered before the given time, but it's currently impossible to guarantee the timer will be triggered at exactly the requested time.
774−
775−Let's take a look at our state:
266+or
776267
777268```hoon
778−|_ [hid=bowl:gall on=_| in-progress=_| target=@t]
269+[%give a-gift]
779270```
780271
781−We have three pieces of app-specific state. `target` is the url
782−we're monitoring. `on` and `in-progress` are booleans
783−representing, respectively, whether or not we're supposed to keep
784−monitoring and whether or not we're in the middle of a request.
785−
786−The type of booleans is usually written as `?`, which is a union
787−of `&` (true) and `|` false. This type defaults to `&` (true). In our
788−case, we want both of these to default to `|` (false). Because
789−of that, we use `_|` as the type. `_value` means "take the type
790−of `value`, but make the default value be `value`". Thus, our
791−type is still a boolean, just like `?`, but the default value is
792−`|` (false).
272+When you give a `gift`, that's the end of the story, but when you pass a
273+`note`, you may get a response. If you do, then the response will come
274+tagged with the `wire` that you used to pass the `note`. It's generally
275+good practice to make your `wire`s fairly unique within your agent, since
276+otherwise you may not be able to distinguish the responses.
793277
794−> This is the same `_` used in `_+>.$`, which means "the same type as the value "+>.$". In other words, the same type as our current context and state.
278+#### Notes
795279
796−Let's take a look at `++poke-atom`. When we're poked with an atom, we
797−first check whether the atom is `'off'`. If so, we set our state
798−variable `on` to false.
799−
800−If not, we check whether the atom is `'on'`. If so, we set `on` and
801−`in-progress` to true. If it was already either on or in progress, then
802−we don't take any other immediate action. If it was both off and not in
803−progress, then we send an HTTP request.
804−
805−If the argument is neither `'off'` nor `'on'`, then we assume it's an
806−actual url, so we save it in `target`.
807−
808−Here's the move that sends the HTTP request:
280+An agent may pass `note`s to either Arvo or another agent. If the `note` is
281+to another agent, then it should usually be one of these:
809282
810283```hoon
811−[ost.hid %hiss /request ~ %httr %purl (need (de-purl:html target))]
284+[%pass /my/wire %agent our.bowl agent-name %watch /a/path]
285+[%pass /my/wire %agent our.bowl agent-name %leave ~]
286+[%pass /my/wire %agent our.bowl agent-name %poke %foo-mark !>(poke-data)]
812287```
813288
814−> Remember, we are expected to produce a **list** of moves. Note the `~` after the move in the full example. This is a convenient shortcut for creating a list of a single element. It's part of a small family of such shortcuts. `~[a b c]` is `[a b c ~]`, `[a b c]~` is `[[a b c] ~]` and `` `[a b c] `` is `[~ a b c]`. These may be mixed and matched to create various convoluted structures and emojis.
289+Note that to unsubscribe to a `path`, you must send the unsubscription on
290+the same `wire` that sent for the original subscription. Don't subscribe
291+to separate `path`s along the same `wire`, because then you can't properly
292+distinguish them for cancelling (besides not being able to distinguish
293+the subscription updates). In other words, besides letting *you*
294+distinguish your `card`s, the `wire` also identifies requests for the system.
815295
816−The correspondence between this move and `[bone card]` can be hard to
817−visualize on one line. Here it is more pedantically:
296+If the `note` is not to another agent but to Arvo itself, then it is a
297+request to one of the vanes, which are kernel modules that provide
298+various system services, including IO. Here are some examples:
818299
819300```hoon
820−:* bone=ost.hid :: the move
821− term=%hiss
822− wire=/request
823− unit-iden=~
824− mark=%httr
825− cage=[%purl (need (de-purl:html target))]
826−==
827−```
828−
829−When the HTTP response comes back, we handle it with
830−`++sigh-httr`, which, along with the wire the request was sent
831−on, takes the status code, the response headers, and the response
832−body.
833−
834−We check whether the status code is between 200 and 300. If so,
835−all is well, and we print a message saying so. We start the
836−timer for ten seconds after the present time.
837−
838−If we got a bad status code, then we print out the entire
839−response and start the timer again.
840−
841−After ten seconds, arvo will give us a `%wake` event, which will
842−be handled in `++wake-timer`. If we're still supposed to keep
843−monitoring, we send the same HTTP request as before. Otherwise,
844−we set `in-progress` to false.
845−
846−Let's try it out:
847−
848−```
849−~fintud-macrep:dojo> |start %up
850−>=
851−~fintud-macrep:dojo> :up &atom 'http://www.google.com'
852−>=
853−[%all-is-well 200]
854−[%all-is-well 200]
855−~fintud-macrep:dojo> :up &atom 'http://example.com'
856−>=
857−[%all-is-well 200]
858−~fintud-macrep:dojo> :up &atom 'http://google.com'
859−>=
860−[%we-have-a-problem 301]
861−[ %headers
862− ~[
863− [p='X-Frame-Options' q='SAMEORIGIN']
864− [p='X-XSS-Protection' q='1; mode=block']
865− [p='Content-Length' q='219']
866− [p='Server' q='gws']
867− [p='Cache-Control' q='public, max-age=2592000']
868− [p='Expires' q='Sun, 10 Jan 2016 23:45:05 GMT']
869− [p='Date' q='Fri, 11 Dec 2015 23:45:05 GMT']
870− [p='Content-Type' q='text/html; charset=UTF-8']
871− [p='Location' q='http://www.google.com/']
872− ]
873−]
874−[ %body
875− [ ~
876− [ p=219
877− q
878− \/'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\0a<TITLE>301 Moved</TI\/
879− TLE></HEAD><BODY>\0a<H1>301 Moved</H1>\0aThe document has moved\0a<A HREF="http://www.google.com/">her
880− e</A>.\0d\0a</BODY></HTML>\0d\0a'
881− \/ \/
882− ]
883− ]
884−]
885−~fintud-macrep:dojo> :up &atom 'off'
301+[%pass /my/wire %arvo %b %wait (add now.hid ~s10)]
886302```
887303
888−## Marks
889−
890−We've used predefined marks already, but we haven't yet created
891−our own marks. Let's write a sample mark of our own, then chain
892−it together with some preexisting ones to have fun with type
893−conversions.
894−
895−Let's make a small "cord" mark. "Cord" is a name we use for `@t`
896−text, but there's no predefined mark for it. Let's put the following code in:
897−`/mar/cord.hoon`:
304+This is a request to the `%b` vane (Behn, the timer vane) to set a timer
305+for 10 seconds in the future. After 10 seconds, Behn will respond with
306+a `%wake` card, which you will recieve in `+on-arvo`. It will come back
307+on `/my/wire`.
898308
899309```hoon
900−/? 314
901−|_ cod=@t
902−++ grab
903− |%
904− ++ atom |=(arg=@ `@t`(scot %ud arg))
905− --
906−++ grow
907− |%
908− ++ md cod
909− --
910−--
911−```
912−
913−`/? 314` is the required
914−version number, just like in apps. After that everything's in a
915−`|_`, which is a `|%` core, but with input arguments. In our
916−case, the argument is the marked data.
917−
918−There are three possible top-level arms in the `|_` core,
919−`++grab`, `++grow`, and `++grad` (which is used for revision
920−control, covered elsewhere). `++grab` specifies
921−functions to convert from another mark to the current mark.
922−`++grow` specifies how to convert from the current mark to
923−another one.
924−
925−In our case, we only define one arm in `++grab`, namely `++atom`.
926−This allows us to convert any atom to a cord. `++scot` is a
927−standard library function in `hoon.hoon` which pretty-prints
928−atoms of various types. We want to treat the atom as an
929−unsigned decimal number, so we give `%ud` and the given argument
930−to `++scot`. Thus, the form of an arm in `++grab` is
931−`|=(other-mark-type this-mark-value)`.
932−
933−In `++grow`, we just convert to the `md` mark, which is just a
934−`@t` internally, but it has fancy conversion functions to things
935−like html. `++md` isn't a function because the "argument" is
936−already a `@t`, so we can just produce the new value directly.
937−
938−Let's play around a bit with this mark. First, let's take a
939−marked atom and convert it to our new mark.
940−
941−```
942−~fintud-macrep:dojo> &atom 9
943−9
944−
945−~fintud-macrep:dojo> &cord &atom 9
946−'9'
947−
948−~fintud-macrep:dojo> &cord &atom &cord &atom 9
949− '57'
950−```
951−
952−ASCII `9` is `57`. There's no requirement, implicit or otherwise,
953−that conversions back and forth between marks be inverses of each
954−other. These are semantic conversions, in that they refer to the
955−same concept. They're not isomorphisms.
956−
957−Let's play around a little more:
958−
959−```
960−~fintud-macrep:dojo> &md &cord &atom 17
961−'17'
962−
963−~fintud-macrep:dojo> &hymn &md &cord &atom 17
964−[ [%html ~]
965− [[%head ~] ~]
966− [[%body ~] [g=[n=%pre a=~] c=~[[g=[n=%$ a=~[[n=%$ v="17"]]] c=~]]] ~]
967− ~
968−]
969−```
970−
971−That was exciting. If you squint at that, it looks an
972−awful lot like html. Our value `17` is still in there, in the
973−body. `urb` is the mark we render most web pages to. It makes
974−sure you have a complete skeleton of a web page. Of course, by
975−the time this gets to the web page, it's plain html. Let's do
976−the final step in the conversion.
977−
978−```
979−~fintud-macrep:dojo> &mime &hymn &md &cord &atom 17
980−[[%text %html ~] p=52 q='<html><head></head><body><pre>17</pre></body></html>']
981−```
982−
983−This is a mime-typed octet stream with type `/text/html`, length
984−121 bytes, and our lowly number `17` rendered to a web page.
985−`cord` was just one step in the chain.
986−
987−Of course, arvo can infer some parts of this chain:
988−
989−```
990−~fintud-macrep:dojo> &cord 17
991−'17'
992−```
993−
994−Likewise, `md` is the only way to get from `cord` to `hymn`, so
995−that can be omitted. If we omit the `hymn`, though, we get:
996−
997−```
998−~fintud-macrep:dojo> &mime &cord 17
999−[[%text %x-markdown ~] p=2 q='17']
1000−```
1001−
1002−Here, we converted straight from `cord` to `md` to `mime`. Arvo
1003−decided that was the most straightforward conversion, but it
1004−gives a different result than passing it through the `hymn`
1005−mark.
1006−
1007−The minimal correct input, then, is `&mime &hymn &cord &atom 17`.
1008−
1009−## Network Messages
1010−
1011−In the following we assume that you have some knowledge of Hoon. If you don't, check out chapters 1 and 2 of the Hoon tutorial and check back when you're ready.
1012−
1013−Let's get our planets to talk to each other. To listen for and receive messages from other planets, we'll need an app. Let's look at a very simple one, `echo.hoon`:
1014−
1015−```hoon
1016−:: Accepts any noun from dojo and prints it out
1017−::
1018−:::: /===/app/echo/hoon
1019− ::
1020−!: :: enables debug printfs
1021−|% :: forms a core with subject as payload
1022−+$ move [bone card] :: defines echo app's arvo move mold
1023−+$ card $% $~ :: defines card (data) mold as null tagged union
1024− == :: end of mold definition; see $%
1025−-- :: end of card arm definition
1026−:: ::
1027−|_ [bow=bowl:gall $~] :: forms a core with bowl (app state) as sample
1028−++ poke-noun :: begin definition of poke-noun arm
1029− |= non=* :: forms a gate which accepts any noun
1030− ^- [(list move) _+>.$] :: casts product to mold of moves and context
1031− ~& echo+noun+non :: debug printf, i.e. '[%echo %noun *]'
1032− [~ +>.$] :: produce a cell of empty list and our state
1033−-- :: end of poke-noun arm definition
1034−```
1035−
1036−Save this as `echo.hoon` in the `/app` directory of your urbit's pier.
1037−
1038−This is a very simple app that does only one thing. If you poke it with
1039−a value, it prints that value out. To try this out, you have to start
1040−the app, then you can poke it from the command line with the following
1041−commands:
1042−
1043−```
1044−~fintud-macrep:dojo> |start %echo
1045−>=
1046−
1047−~fintud-macrep:dojo> :echo 5
1048−[%echo %noun 5]
1049−>=
1050−
1051−~fintud-macrep:dojo> :echo [1 2]
1052−[%echo %noun [1 2]]
1053−>=
1054−```
1055−
1056−> There is currently a bug where the `[%echo %noun *]` lines are printed **above** the line you entered, so your output may not look exactly like this. `>=` means that a command was successfully received and executed.
1057−
1058−Most of the app code should be simple enough to guess its function. The
1059−important part of this code is the definition of `++poke-noun`.
1060−
1061−Once an app starts, it's always running in the background and you interact with
1062−it by sending it messages. The most straightforward way to do that is to poke it
1063−from the command line, which we we did with `:echo 5`
1064−(`:[app-name] [argument(s)]`).
1065−
1066−In this case, `++poke-noun` takes an argument (**sample**) `arg` and
1067−prints it to dojo with `~&` ([sigpam](/docs/reference/hoon-expressions/rune/sig#sigpam/)).
1068−This is an unusual rune that formally "does nothing", but the interpreter
1069−detects it and printfs the first child, before executing the second as if the
1070−first didn't exist. This is a slightly hacky way of printing to the console,
1071−but we'll get to the correct way later on.
1072−
1073−But what does `++poke-noun` produce? Recall that `^-` casts to a type. In this
1074−case, it's declaring that the end result (**product**) of the function
1075−(`++poke-noun`'s **gate**) will be of type `[(list) _+>.$]`. But what does this
1076−mean?
1077−
1078−The phrase to remember is "a list of moves and our state". Urbit is a message
1079−passing system, so whenever we want to do something that interacts with the rest
1080−of the system, we send a message. Thus, a move is Arvo's equivalent of a
1081−syscall. The first thing that `++poke-noun` produces is a list of messages,
1082−called "moves". In this case, we don't actually want the system to do anything,
1083−so we produce the empty list, `~` (in the `[~ +>.$]` line).
1084−
1085−The second thing `++poke-noun` produces is our state. `+>.$` refers to a
1086−particular address in our subject where our formal app state is stored.
1087−It'll become clear why this is later on, but for now pretend that `+>.$`
1088−is a magic invocation that means "app state".
1089−
1090−Let's look at another example. Say we want to
1091−only accept a number, and then print out the square of that number.
1092−
1093−```hoon
1094−:: Accepts an atom from the dojo and squares it.
1095−::
1096−:::: /===/app/square/hoon
1097− ::
1098−!:
1099−|% :: no moves in :square
1100−+$ move [bone card] :: no cards in :square
1101−+$ card $% $~
1102− ==
1103−--
1104−:: :: stateless
1105−|_ [bow=bowl:gall $~]
1106−++ poke-atom
1107− |= tom=@
1108− ^- [(list move) _+>.$]
1109− ~& square+(mul tom tom)
1110− [~ +>.$]
1111−--
1112−```
1113−
1114−Save this as `square.hoon` in the `/app` directory of your urbit's pier.
1115−
1116−A few things have changed. Firstly, we no longer accept arbitrary nouns because
1117−we can only square atoms (integers, in this case an unsigned one). Thus, our
1118−argument is now `tom=@`. Secondly, it's `++poke-atom` rather than `++poke-noun`.
1119−
1120−### Intro to marks
1121−
1122−Are there other `++poke`s? Yes. In fact, `noun` and `atom` are just two
1123−of arbitrarily many "marks". A mark is fundamentally a type definition,
1124−but accessible at the Arvo level. Each mark is defined in the `/mar`
1125−directory. Some marks have conversion routines to other marks, and some
1126−have diff, patch, and merge algorithms. None of these are required for a
1127−mark to exist, however.
1128−
1129−`noun` and `atom` are two predefined marks. In your `/mar` directory,
1130−there are already many more, and you may add more at will. The type
1131−associated with `noun` is `*`, and the type associated with `atom` is
1132−`@`.
1133−
1134−When we poke an app from anywhere, we do so with a mark that searches
1135−for the corresponding (`++poke-[mark]`) arm. Data constructed on the
1136−command line is by default marked with `noun`. In this case, the app is
1137−expecting an atom, so we have to explicitly mark the data with `atom`
1138−using `&[mark]`. Try the following commands:
1139−
310+=/ =path /(scot %p our.hid)/home/(scot %da now.hid)/my-file/txt
311+=/ contents=cage [%txt !>(~['text file line 1' 'line 2'])]
312+[%pass /my/wire %arvo %c %info (foal path contents)]
1140313```
1141−~fintud-macrep:dojo> |start %square
1142−>=
1143−
1144−~fintud-macrep:dojo> :square 6
1145−gall: %square: no poke arm for noun
1146−
1147−~fintud-macrep:dojo> :square &atom 6
1148−[%square 36]
1149−>=
1150−```
1151−
1152−> Recall the bug where `%square` may get printed above the input line.
1153−
1154−Marks are powerful, and they're the backbone of Urbit's data pipeline,
1155−so we'll be getting quite used to them.
1156314
1157−**Exercise**:
1158−
1159−- Write an app that computes fizzbuzz on its input.
1160−
1161−#### Sending a message to another urbit
1162−
1163−Let's write our first network message! Here's `examples/app/pong.hoon`:
315+This is a request to the `%c` vane (Clay, the filesystem vane) to write
316+a file to `/my-file/txt` in the home desk. You will not receive a
317+response to this `note`.
1164318
1165319```hoon
1166−:: Allows one urbit to send the string 'Pong' to
1167−:: another urbit.
1168−::
1169−:::: /===/app/pong/hoon
1170− ::
1171−!:
1172−|%
1173−+$ move [bone card]
1174−+$ card $% [$poke wire dock poke-contents]
1175− ==
1176−+$ poke-contents $% [$atom @]
1177− ==
1178−--
1179−|_ [bow=bowl:gall $~] :: stateless
1180−::
1181−++ poke-urbit
1182− |= to=ship
1183− ^- [(list move) _+>.$]
1184− ~& pong+'Outgoing pong!'
1185− :_ +>.$
1186− ~[[ost.bow %poke /sending [to dap.bow] %atom 'Pong']]
1187−::
1188−++ poke-atom
1189− |= tom=@
1190− ^- [(list move) _+>.$]
1191− ~& pong+'Incoming pong!'
1192− ~& pong+received+`@t`tom
1193− [~ +>.$]
1194−::
1195−++ coup |=(* [~ +>.$])
1196−::
1197−--
1198−```
1199−
1200−
1201−Run it with these commands:
1202−
320+=/ =path /my-file/txt
321+=/ =moat [da+now.hid da+(add now.hid ~h1) path]
322+[%pass /my/wire %arvo %c %warp our.hid %home ~ %many & moat]
1203323```
1204−~fintud-macrep:dojo> |start %pong
1205−>=
1206324
1207−~fintud-macrep:dojo> :pong &urbit ~sampel-sipnym
1208−>=
1209−```
1210−
1211−Replace `~sampel-sipnym` with another urbit. The easiest thing to do is to start
1212−a comet, a free and disposable Urbit identity. If you don't know how to start a
1213−comet, see [the user setup section](@/using/operations/_index.md). Don't forget to start
1214−the `%pong` app on that urbit, too. You should see, on the foreign
1215−urbit, this output:
325+This is a request to the `%c` vane to send us a `%writ` `card` whenever
326+`/my-file/txt` changes in the next hour. There may be many responses to
327+this note if the file changes many times.
1216328
1217329```hoon
1218−[%pong 'Incoming pong!']
1219−[%pong %received 'Pong']
330+=/ =request:http
331+ [%'GET' 'https://example.com' ~ ~]
332+[%pass /my/wire %arvo %i %request request *outbound-config:iris]
1220333```
1221334
1222−Most of the code should be straightforward. In `++poke-atom`, the only new thing
1223−is the expression `` `@t`tom ``, which is **casting** the argument `tom` to type
1224−`@t`. As we already know, `@t` is the type of "cord" (text string).
335+This is a request to the `%i` vane (Iris, the HTTP client vane) to make
336+a GET HTTP request to example.com. The response will come as an
337+`%http-response` card.
1225338
1226−The more interesting part is in `++poke-urbit`. The `urbit` mark is an urbit
1227−identity, and the Hoon type associated with it is `ship` or `@p` (the "p"
1228−stands for "phonetic base").
339+#### Gifts
1229340
1230−Recall that in a `++poke` arm we produce "a list of moves and our state". Until
1231−now, we've left the list of moves empty, since we haven't wanted to tell Arvo to
1232−do anything in particular. Now we want to send a message to another urbit. Thus,
1233−we produce a list with one element:
341+In contrast to the many possible `note`s, there are only two types of
342+`gift`s that an agent may give:
1234343
1235344```hoon
1236−~[[ost.bow %poke /sending [to dap.bow] %atom 'Pong]]
1237−```
1238−
1239−#### Moves
1240−
1241−The general form of a move is `[bone term wire *]`.
1242−
1243−Or, in pseudo-code:
1244−
1245−`["cause" sys-call/action tack-new-layer-on-cause action-specific information]`
1246−
1247−Let's walk through each of these elements step by step.
1248−
1249−##### Bones ("cause")
1250−
1251−If you look up `++bone` in `hoon.hoon`, you'll see that it's a number (`@ud`),
1252−and that it's an opaque reference to a duct. `++duct` in `hoon.hoon` is a list
1253−of `wire`s, where `++wire` is an alias for `++path`. `++path` is a list of
1254−`++knot`s, which are ASCII text. Thus, a duct is a list of paths, and a bone is
1255−an opaque reference to it (in the same way that a Unix file descriptor is an
1256−opaque reference to a file structure). Thus, to truly understand bones, we must
1257−understand ducts.
1258−
1259−A duct is a stack of causes, again, represented as paths, which are called wires.
1260−At the bottom of every duct is a unix event, such as a keystroke, network
1261−packet, file change, or timer event. When Arvo is given this event, it routes
1262−the event to appropriate kernel module for handling.
1263−
1264−Sometimes, the module can immediately handle the event and produce any necessary
1265−results. For example, when we poked the `++poke-atom` arm above, we poked `%gall`,
1266−our application server, which was able to respond to our poke directly. When the
1267−module cannot service the request itself, it sends instructions to another
1268−kernel module or application (through the `%gall` module) to do a specified
1269−action, and produces the result from that.
1270−
1271−Furthermore, when one module sends a message to another kernel module or
1272−application, it also sends along the duct it was given with its new wire tacked
1273−onto the top. Now the duct has two entries, with the unix event on the bottom
1274−and the kernel module that handled it on top. This process can continue
1275−indefinitely, pushing more and more wires onto the top of the duct. When an
1276−entity finally produces a result, a wire is popped off the duct, and the result
1277−is passed all the way back down, repeating the process of wire popping
1278−sequentially until the bottom of the duct is reached.
1279−
1280−In effect, a duct is an Arvo-level call stack. It's worth noting that while in
1281−traditional call stacks a function call happens synchronously and returns
1282−exactly once. In Arvo, multiple moves can be sent at once, are evaluated
1283−asynchronously, and each one may be responded to zero or more times.
1284−
1285−The point to take home is that whatever caused `++poke-urbit` to be called is
1286−also the root cause for the network message we're trying to send. Thus, we say
1287−to send the network message along the given bone `ost`.
1288−
1289−##### Wire (path)
1290−
1291−Of course, we have to push a new wire onto our duct before passing it along (or
1292−responding to it directly) anywhere. This wire can have any data we want in it,
1293−but we don't need anything specific here, so we just use the wire `/sending`
1294−(`/elem1/elem2/elemN` is one syntax used to create `++path`s and `++wire`s of N
1295−elements). If we were expecting a response (which we're not), it would come back
1296−along the `/sending` wire, meaning that the path `/sending` will be passed back
1297−to the response handler as an argument. Although it's not required, it's
1298−generally a good idea to make the wire human-readable for bug-handling purposes.
1299−
1300−##### Term (sys-call)
1301−
1302−Each move also has a `term`, composed of lowercase ASCII and/or `-`. This `term`
1303−has the aura `@tas`. In this case, our `term` is `%poke`, which is the name of
1304−the particular kind of move we're sending. You can always use `%poke` to message
1305−an app. Other common names include `%warp`, to read from the filesystem;
1306−`%wait`, to set a timer; and `%them`, to send an http request.
1307−
1308−The move ends with `*` (that is, any noun) since each type of move takes
1309−different data. In our case, a `%poke` move takes a target (urbit and app) and
1310−marked data, then pokes the arm of the corresponding mark on that app on that
1311−urbit with that data. `[to-urbit-address %pong]` is the target urbit and app,
1312−`%atom` is the `mark`, and`'Pong'` is the data.
1313−
1314−When Arvo receives a `%poke` move, it calls the appropriate `++poke`.
1315−The same mechanism is used for sending messages between apps on the same
1316−urbit as for sending messages between apps on different urbits.
1317−
1318−> We said earlier that we're not expecting a response. This is not entirely true: the `++coup` is called when we receive acknowledgment that the `++poke` was called. We don't do anything with this information right now, but we could.
1319−
1320−**Exercises**:
1321−
1322−- Extend either of the apps in the first two exercises to accept input over the network in the same way as `pong`.
1323−
1324−- Modify `pong.hoon` to print out a message when it receives acknowledgement.
1325−
1326−- Write two apps, `even` and `odd`. When you pass an atom to `even`, check whether it's even. If so, divide it by two and recurse; otherwise, poke `odd` with it. When `odd` receives an atom, check whether it's equal to one. If so, terminate, printing `%success`. Otherwise, check whether it's odd. If so, multiply it by three, add one, and recurse; otherwise, poke `even` with it. When either app receives a number, print it out along with the name of the app. In the end, you should be able to watch Collatz's conjecture play out between the two apps. Sample output:
1327−
1328−```
1329−~fintud-macrep:dojo> :even &atom 18
1330−[%even 18]
1331−[%odd 9]
1332−[%even 28]
1333−[%even 14]
1334−[%odd 7]
1335−[%even 22]
1336−[%odd 11]
1337−[%even 34]
1338−[%odd 17]
1339−[%even 52]
1340−[%even 26]
1341−[%odd 13]
1342−[%even 40]
1343−[%even 20]
1344−[%even 10]
1345−[%odd 5]
1346−[%even 16]
1347−[%even 8]
1348−[%even 4]
1349−[%even 2]
1350−%success
1351−```
1352−
1353−- Put `even` and `odd` on two separate urbits and pass the messages over the network. Post a link to a working solution in `:talk` to receive a cookie.
1354−
1355−
1356−## Security Drivers
1357−
1358−A security driver is a file in `/=home=/sec/<tld>/<domain>/hoon`
1359−that handles the authentication for all HTTP requests to
1360−`https://<domain>.<tld>`. When anything in Urbit makes an HTTP
1361−request through `%eyre` (our web server), it checks to see if we
1362−have a security driver for the requested domain, and if so
1363−filters the request through the driver. The security driver will
1364−usually either decorate the request with the needed credentials
1365−if it has them, or else it will guide the user through the
1366−process of authenticating Urbit with the service.
1367−
1368−Each web service needs its own security driver, but most of them
1369−are pretty standard. We recommend starting with an existing
1370−security driver based on your needed authentication method (e.g.
1371−basic auth, OAuth1, OAuth2), and changing whatever is specific to
1372−your service.
1373−
1374−The best strategy for building security drivers is to copy a
1375−similar one and tweak it until it works for you. The best
1376−representatives are Github for Basic Authentication, Twitter for
1377−OAuth1, Slack for OAuth2 when access tokens don't expire, and
1378−Google APIs for OAuth2 when access tokens do expire. In most
1379−cases you'll need to make very few changes to one of these
1380−models.
1381−
1382−Still, it's worth seeing one or two built from the ground up.
1383−Here, we'll build a connector for the Github API v3. (It's already in `/sec/com/github.hoon` if you want to see it.)
1384−
1385−The
1386−simplest way to interact with the Github API is to just fetch
1387−https://api.github.com from the dojo. First, run this:
1388−
1389−```
1390−> |init-auth-basic /com/github
1391−```
1392−
1393−Input your Github username and password as directed. Next:
1394−
1395−```
1396−~your-urbit:dojo> +https://api.github.com
1397−```
1398−
1399−> Note: the current version of this API is outdated, so you'll get a 403 message at this point. We need to update this so you can follow along with the examples. We're sorry!
1400−
1401−Github exposes a few endpoints to the general web, and the root
1402−endpoint is one of them. This gives you a textual representation
1403−of a JSON object that contains a bunch of urls to other parts of
1404−Github's API. This is exactly the same response you would get
1405−from just running `curl https://api.github.com` on UNIX.
1406−
1407−If we don't have a security driver for Github yet, many of the
1408−endpoints won't be accessible, or they will only have publicly
1409−accessible information. Most of what we care about requires us
1410−to be authenticated.
1411−
1412−#### Basic auth
1413−
1414−Here's a simple security driver:
345+[%give %fact (unit path) =cage]
346+[%give %kick (unit path)]
347+```
348+
349+A subscription update is a new piece of subscription content for all
350+subscribers on a given `path`. If no `path` is given, then the update is
351+only given to the program that instigated this request. Typical use of
352+this mode is in `+on-watch` to give an initial update to a new
353+subscriber to get them up to date.
354+
355+A subscription close closes the subscription for all subscribers on a
356+given `path`. If no `path` is given, then the update is only given to the
357+program that instigated this request. Typical use of this mode would be
358+in `+on-watch` to produce a single update to a subscription then close
359+the subscription.
360+
361+### Vases and cages
362+
363+A `vase` is a piece of dynamic data. Structurally, it's a pair of an
364+explicit reification of a type and an untyped noun. This lets us
365+represent a value which has a type that isn't known at compile time. A
366+vase has three operations:
367+
368+- `!>` is a unary rune that lifts a statically typed value to a
369+ dynamically-typed `vase`. For example, `!>('hi')` gives `[[%atom %t ~]
370+ 26.984']`, which has type `[type *]`.
371+
372+- `!<` is a binary rune that takes a mold and a dynamically typed `vase`
373+ and reduces it to a statically typed value. If the `vase` does not in
374+ fact have the type of the mold you give it, then it produces `~`, else
375+ it produces `[~ value]`. For example, `!<(@t !>('hi'))` produces `[~
376+ 'hi']` while `!<(^ !>('hi'))` produces `~`.
377+
378+- The compiler takes text and converts it to a `vase` of the compiled
379+ code. Agents shouldn't need this directly, but Gall uses this to
380+ compile agents to `vase`s, on which it calls `!<(agent:gall
381+ compiled-agent-vase)`.
382+
383+A `cage` is simply a pair of a mark and a vase. A mark is a textual tag
384+that should correspond to the particular dynamic type in the vase.
385+
386+In agents, we use vases to represent types which Gall doesn't know about
387+when it was compiled, but which nevertheless need to go outside the
388+agent. In practice, there are three common cases:
389+
390+- The data in a "poke" request is of a type that is defined by each
391+ agent, so it must by dynamic. This is the input to `++on-poke` as well
392+ as the cage in the `%poke` case of the `%agent` note.
393+
394+- The data in a subscription update is defined by each agent, so it must
395+ be dynamic. This is the cage in `%fact`, both when giving the update
396+ and in the input to `+on-agent`.
397+
398+- The state of an agent is also unique to each agent. Most of the time,
399+ Gall doesn't interact directly with agent's state, but when upgrading
400+ an agent, it must pass the state of the old version of the agent to
401+ the new version of the agent. This is the output of `+on-save` and
402+ the input to `+on-load`.
403+
404+### State
405+
406+In the definition of an agent, the entire core is wrapped with the `^|`
407+rune, which corresponds to the "iron" variance mode. This means it's
408+"contravariant" in the input, which allows Gall to write to the new bowl
409+on every event. It also means that the context of the core is opaque to
410+Gall, which means you can store state in your context in whatever
411+structure you want.
412+
413+This is why most of the handler arms produce not just a list of cards
414+but also a new `agent:gall`. Usually, this will be the "same" agent in
415+the sense that the code will be the same, but you may have changed the
416+state that's stored in its (opaque to Gall) context.
417+
418+The skeleton example above gives an example of keeping an atom as your
419+state. Define the state in the context of your agent core, then you can
420+change it with `=.` or `%=`, as long as you produce the new version of
421+the agent core.
422+
423+When you upgrade an agent, you need to extract the state from your
424+opaque context and produce it to Gall as a dynamically typed vase.
425+Usually, this will be easy: just call `!>` on whatever state you wish
426+to preserve. This is `+on-save`.
427+
428+When the new agent is about to be started, Gall will call it with
429+`+on-load` with the vase just produced above. This allows you to ingest
430+your old state and continue right where you left off.
431+
432+If the type of your state hasn't changed, you can just
1415433
1416434```hoon
1417−:: Test url +https://api.github.com/user
1418−::
1419−:::: /hoon/github/com/sec
1420− ::
1421−/+ basic-auth
1422−!:
1423−|_ [bal=(bale keys:basic-auth) $~]
1424−++ aut ~(standard basic-auth bal ~)
1425−++ filter-request out-adding-header:aut
1426−--
1427−```
1428−
1429−Github supports authentication through either Basic
1430−Authentication or OAuth2. We'll show the basic auth example
1431−first, but in general we'd prefer OAuth2.
1432−
1433−Since this driver is for Github, put it in
1434−`/=home=/sec/com/github/hoon`.
1435−
1436−To try this out, we first have to initialize our credentials with
1437−`|init-auth-basic`, which prompts for the url of the service
1438−(api.github.com), username, and password. It stores this
1439−information in a manner that `%eyre` knows how to decode and send
1440−put into your `bale`. Note that `|init-auth-basic` is standard for
1441−basic auth, but other auth schemes (like OAuth) are initialized
1442−in other ways.
1443−
1444−After you've run `|init-auth-basic`, you should be able to run
1445−`+https://api.github.com/user` and get a response indicating who
1446−you're logged in as.
1447−
1448−You can similarly POST:
1449−
435+=. state !<(state-type old-state-vase)
1450436```
1451−~your-urbit:dojo> +https://api.github.com/gists &json _(cork de-json:html need) '{"files":{"file1.txt":{"content":"can\'t stop the signal"}}}'
1452−```
1453−
1454−This creates a gist, go to `https://gist.github.com/<username>` to
1455−see it.
1456437
1457−The `+url` syntax used in "source" position makes a GET request,
1458−but if you pass it data it will make a POST request with that
1459−data as the body. This request is authenticated seamlessly. You
1460−can send data of any mark as long as it has a conversion path to
1461−`%mime`.
1462−
1463−Let's take a look at the code and see if we can figure out what's
1464−happening here:
438+If it has changed, then it should look more like:
1465439
1466440```hoon
1467−/+ basic-auth
441+=. state (upgrade-state !<(old-state-type old-state-vase))
1468442```
1469443
1470−First, we load a library called `basic-auth`. This library
1471−exposes two items. `keys` is the type of the basic auth key,
1472−which is essentially a base64'd username and password.
1473−`standard` accepts a `bale` and produces a core with everything
1474−we'll need to implement basic auth.
1475−
1476−Every security driver needs some amount of state, which is stored
1477−in `bale`. `bale` contains:
1478−
1479−- `our`, our urbit name
1480−- `now`, the current time
1481−- `eny`, 256 bits of entropy
1482−- `byk`, the urbit, desk and case which the security driver is running from
1483−- `usr`, the particular identity we're logging in with
1484−- `dom`, the site we're accessing
1485−- `key`, the secrets required to authenticate requests. The type for this entry is supplied by the programmer as the argument to `++bale`. Thus, in our case, `key` in our bale is of type `keys:basic-auth`.
1486−
1487−Additionally, a security driver contains at least one out of the
1488−five "special" arms:
1489−
1490−- `++filter-request` is a function which takes a hiss (http request) and produces a `sec-move` (defined in zuse), which is one of:
1491− - `[%send hiss]`, which sends the new hiss. This is the case in, for example, basic auth, where all we need to do is add an extra header to the request.
1492− - `[%give httr]`, where httr is an http response. This immediately returns an http response to the sender.
1493− - `[%show purl]`, where a purl is a parsed url. This displays a message asking the user to visit the given url to continue the authentication process.
1494− - `[%redo ~]`, which redoes the request.
1495− - `%eyre` calls `++filter-request` just before sending an HTTP request to the specified domain. This allows the security driver to filter the requests, decorating them with authentication data.
1496−- `++filter-response` is a function which takes an httr and produces a `sec-move`. `%eyre` calls it after receiving an HTTP response from the specified domain. This allows the security driver to handle authentication errors, commonly caused by expired tokens, and retry the request.
1497−- `++receive-auth-query-string` is a function that takes a `quay` (list of query parameters) and produces a `sec-move`. `%eyre` calls it when it receives a request on the callback url. Generally, this happens in OAuth after the user has granted access to the urbit app, and the service makes a request to the callback url with the code. The security driver should then make a request to convert the code into an access token.
1498−- `++receive-auth-response` is a function that takes an httr and produces a `sec-move`. `%eyre` calls it when it gets a response to the request made in `++receive-auth-query-string`.
1499−- `++update` is a function which converts old state to a new format when the security driver gets updated. If you want to just get rid of the old state, define `++discard-state` as `~`.
1500−
1501−For basic auth, we only have to worry about `++filter-request`,
1502−since all we need to do is add the correct header to each
1503−request. Our `++aut` core contains a function
1504−`++out-addding-header`, which does exactly what we want.
1505−
1506−Where do the keys come from, though? Remember we ran
1507−`|init-auth-basic` to input them. This just prompts you for the
1508−service url, username, and password, and it stores them next to
1509−the driver. In our case, that's
1510−`/=home=/sec/com/gitub/api/atom`. These keys are stored
1511−encrypted, and you don't want to edit them directly. `%eyre`
1512−loads them directly into your `bale`.
1513−
1514−#### OAuth2
1515−
1516−Most services are better accessed through some form of OAuth.
1517−Github can be accessed with OAuth2, which is a little more
1518−complicated than basic auth, but not hugely so.
1519−
1520−The basic steps for OAuth2 are these:
1521−
1522−- create an app and get its client id and client secret (on Github)
1523−- store these in Urbit (with `|init-oauth2`)
1524−- try to make a request that requires authentication. The security driver should prompt you to visit a particular url.
1525−- visit the url and click "authorize", which gives the app access to your account
1526−- the security driver will turn the code it receives into an access token
1527−- on every request, include that access token
1528−
1529−Creating an app is easy and
1530−[well-documented](https://github.com/settings/applications/new).
1531−If necessary, set the callback url to
1532−`/~/ac/github.com/_state/in`, and note its client id and client
1533−secret.
1534−
1535−Run `|init-oauth2`, which will prompt for the hostname
1536−(api.github.com), client id, and client secret. This will store
1537−the information, encrypted, in `/=home=/sec/com/github.atom`,
1538−just as with Basic Authentication.
1539−
1540−Now we need a security driver. Use this:
444+It's useful to tag your state with a version number so that the
445+`+upgrade-state` function can take a tagged union of all your old state
446+types and upgrade from any of them to the current state type. For
447+example, it may have the following structure:
1541448
1542449```hoon
1543−:: Test url +https://api.github.com/user
1544−::
1545−:::: /hoon/github/com/sec
1546− ::
1547−/+ oauth2
1548−!:
1549−::::
1550− ::
1551−|_ [bal=(bale keys:oauth2) tok=token:oauth2]
1552−++ scopes :: comment out scopes to taste
1553− :~ 'user' 'user:email' 'user:follow' 'public_repo' 'repo'
1554− 'repo_deployment' 'repo:status' 'delete_repo' 'notifications'
1555− 'gist' 'read:repo_hook' 'write:repo_hook' 'admin:repo_hook'
1556− 'admin:org_hook' 'read:org' 'write:org' 'admin:org'
1557− 'read:public_key' 'write:public_key' 'admin:public_key'
450++$ old-state-types
451+ $% [%0 s=state-0]
452+ [%1 s=state-1]
453+ [%2 s=state-2]
1558454 ==
1559−:: ++aut is a "standard oauth2" core, which implements the
1560−:: most common handling of oauth2 semantics. see lib/oauth2 for more details,
1561−:: and examples at the bottom of the file.
1562−++ aut (~(standard oauth2 bal tok) . |=(tok/token:oauth2 +>(tok tok)))
1563−++ filter-request
1564− %^ out-add-query-param:aut 'access_token'
1565− scope=~[%client %admin]
1566− oauth-dialog='https://github.com/login/oauth/authorize'
1567−::
1568−++ receive-auth-query-string
1569− %- in-code-to-token:aut
1570− url='https://github.com/login/oauth/api/access_token'
1571−++ receive-auth-response bak-save-token:aut
1572−--
1573−```
1574−
1575−The oauth2 library provides the main "engine" in `standard`, just
1576−like in basic auth, except that we also have to specify a
1577−function to save the access token when we get it. It's worth
1578−noting that this library is well documented in the source,
1579−including examples: `/=home=/lib/oauth2/hoon`.
1580−
1581−Running `+http://api.github.com/user` tries to make a request to
1582−Github with authentication. This loads into the bale the keys,
1583−which are of type `keys:oauth2`, defined in the oauth2 library.
1584−We don't have to worry about the specifics of these keys -- the
1585−library handles them -- but they include the client id and the
1586−client secret.
1587−
1588−Before the request is sent, `%eyre` calls `++filter-request` to
1589−decorate it with authentication headers. Since we don't yet have
1590−an access token, we need to prompt the user to visit the "dialog"
1591−url. `++out-add-query-param` checks whether we have a token, and
1592−if we don't it produces `%show` and a message telling the user to
1593−visit the `oauth-dialog` url we provide, with the extra query
1594−parameters added (client id, scopes, and state).
1595−
1596−When the user visits that url, Github will ask them to log in and
1597−authorize the application. When they do so, Github will post a
1598−request to `/~/ac/github.com/_state/in` with a code in the query
1599−string. `%eyre` sends the query string to
1600−`++receive-auth-query-string`, which is a function that takes a
1601−quay and produces a `sec-move`. `++in-code-to-token` from our
1602−`aut` takes the `code` parameter from the quay and uses it to
1603−create a request to the given url. This request includes the
1604−client id, the client secret, and the code we just got.
1605−
1606−Github's response to that request includes an access token in its
1607−body. This is handled in `++receive-auth-response`.
1608−`++bak-save-token` extracts that token and gives it to the
1609−function that we passed in the definition of `++aut`. In our
1610−case, we just save that token into our state for later usage.
1611−
1612−Now we have the credentials we need, so `++filter-request` is
1613−called with the original request, and `++out-add-query-param`
1614−passes the request through with the addition of the query
1615−parameter `access_token`.
1616−
1617−Just like with Basic Authentication, you should be able to run
1618−`+https://api.github.com/user` and get a response indicating who
1619−you're logged in as.
1620−
1621−Some drivers are slightly more complicated. For example, Github's
1622−access token's don't expire, but that's not the case for all
1623−services. The googleapis driver shows the flow when the access
1624−token can expire. Twitter uses OAuth1, for which we have another
1625−library.
1626−
1627−Remember, of course, that the best strategy for building security
1628−drivers is to copy a similar one and tweak it to taste.
1629−
1630−## State
1631−
1632−In the last section we built a few small apps that sent moves. These apps were
1633−entirely stateless, however. Most useful apps require some amount of state.
1634−Let's build a trivial stateful app. It'll keep a running the sum of all the
1635−atoms we poke it with. Here's `examples/app/sum.hoon`:
1636−
1637−```hoon
1638−:: Keeps track of the sum of all the atoms it has been
1639−:: poked with and prints the sum out
1640−::
1641−:::: /===/app/sum/hoon
1642− ::
1643−!:
1644−|%
1645−+$ move [bone card]
1646−+$ card $% $~
1647− ==
1648−--
1649−|_ [bow=bowl:gall sum=@]
1650−::
1651−++ poke-atom
1652− |= tom=@
1653− ^- [(list move) _+>.$]
1654− ~& sum+(add sum tom)
1655− [~ +>.$(sum (add sum tom))]
1656−::
1657−++ coup
1658− |= [wir=wire err=(unit tang)]
1659− ^- [(list move) _+>.$]
1660− ?~ err
1661− ~& sum+success+'Poke succeeded!'
1662− [~ +>.$]
1663− ~& sum+error+'Poke failed. Error:'
1664− ~& sum+error+err
1665− [~ +>.$]
1666−::
1667−--
1668−```
1669−
1670−We can start it with `|start %sum`, and then run it:
1671−
455+++ upgrade-state
456+ |= old=old-state-types
457+ =? old ?=(%0 -.old)
458+ (state-0-to-1 s.old)
459+ =? old ?=(%1 -.old)
460+ (state-1-to-2 s.old)
461+ ?> ?=(%2 -.old)
462+ s.old
1672463```
1673−~fintud-macrep:dojo> :sum &atom 5
1674−[%sum 5]
1675−>=
1676464
1677−~fintud-macrep:dojo> :sum &atom 2
1678−[%sum 7]
1679−>=
465+### Bowl
1680466
1681−~fintud-macrep:dojo> :sum &atom 15
1682−[%sum 22]
1683−>=
1684−```
1685−
1686−We can see that app state is being saved, but when, where, and how?
467+The core takes as input a `bowl`, which includes useful info like the
468+current ship, the current time, and a renewable source of entropy. This
469+information is available to any of the handlers.
1687470
1688−The state is stored as the second thing in the `|_` line. In our case, it's
1689−simply an atom named `sum`. We change it by producing our state not with
1690−`+>.$` (as before), but with `+>.$(sum (add sum tom))`. We've seen all these
1691−parts before, but you might not recognize them.
471+## Arms
1692472
1693−Recall in the first chapter that we recursed with the expression
1694−`$(b (add 3 b))`. This meant "produce `$` with `b` changed to `(add 3 b)`.
1695−Similarly, `+>.$(sum (add sum tom))` means "produce `+>.$` (i.e. our
1696−context, which contains our state) with `sum` changed to `(add sum tom)`.
473+A description of each of the handler arms follows.
1697474
1698−At a high level, then, when we handle state, we do it explicitly. It's passed in
1699−and produced explicitly. In Unix systems, application state is just a block of
1700−memory, which you need to serialize to disk if you want to keep it around for
1701−very long.
475+### +on-init
1702476
1703−In Urbit, app state is a single (usually complex) value. In our example, we have
1704−very simple state, so we defined `sum=@`, meaning that our state is an atom.
1705−Of course, `sum` is just a name, and you're free to name your state whatever
1706−you like. But let's clarify a couple other things before we continue.
477+This arm is called once when the agent is started. It has no input and
478+lets you perform any initial IO.
1707479
1708−First, `bowl` is a set of general global states. This set is managed by the
1709−system. It includes things like `now` (the current time), `our` (our urbit
1710−identity), and `eny` (512 bits of guaranteed-fresh entropy). For the full list
1711−of things in `++bowl`, search for `++ bowl` (note: two spaces) in
1712−`/arvo/zuse.hoon`.
480+### +on-save
1713481
1714−> This is, perhaps, the most common way to learn Hoon. The easiest way to learn about an identifier you see in code is to search in `/arvo/sys/zuse.hoon` and `/arvo/sys/hoon.hoon` for it. Urbit's codebase is less than 30000 lines of code combined, including the hoon parser, the compiler, and the `/arvo` microkernel, so you can usually use the code and its comments as a reference doc. You can also read [zuse.hoon](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon) and [hoon.hoon](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/hoon.hoon) in your browser.
482+This arm is called immediately before the agent is upgraded. It
483+packages the permament state of the agent in a `vase` for the next version
484+of the agent. Unlike most handlers, this cannot produce effects.
1715485
1716−The second thing we should clear up is this: Urbit needs no "serialize to disk"
1717−step. Everything you produce in the app state is persistent across calls to the
1718−app, restarts of the urbit, and even power failure. If you want to write to the
1719−Unix filesystem, you can, but it's not needed for persistence. Urbit has
1720−transactional events, which makes it an [ACID operating
1721−system](https://en.wikipedia.org/wiki/ACID). Thus, you don't have to worry about
1722−persistence when programming in Urbit, or ever go through the hassle of having
1723−to set up and write to a database.
486+### +on-load
1724487
1725−**Exercises**:
488+This arm is called immediately after the agent is upgraded. It receives
489+a `vase` of the state of the previously-running version of the agent,
490+which allows it to cleanly upgrade from the old agent.
1726491
1727−- Modify `:sum` to reset the counter when you poke it with `0`.
492+### +on-poke
1728493
1729−- Write an app called `:last` that prints out the previous value you poked it with.
494+This arm is called when the agent is "poked". The input is a `cage`, so
495+it's a pair of a mark and a dynamic `vase`.
1730496
1731−Sample output:
497+### +on-watch
1732498
1733−```
1734−~fintud-macrep:dojo> :last 7
1735−[%last 0]
1736−>=
499+This arm is called when a program wants to subscribe to the agent on a
500+particular `path`. The agent may or may not need to perform setup steps
501+to intialize the subscription. It may produce a `%give`
502+`%subscription-result` to the subscriber to get it up to date, but after
503+this event is complete, it cannot give further updates to a specific
504+subscriber. It must give all further updates to all subscribers on a
505+specific `path`.
1737506
1738−~fintud-macrep:dojo> :last [1 2 3]
1739−[%last 7]
1740−>=
507+If this arm crashes, then the subscription is immediately terminated.
508+More specifcally, it never started -- the subscriber will receive a
509+negative `%watch-ack`. You may also produce an explicit `%kick` to
510+close the subscription without crashing -- for example, you could
511+produce a single update followed by a `%kick`.
1741512
1742−~fintud-macrep:dojo> :last 'howdy'
1743−[%last [1 2 3]]
1744−```
513+### +on-leave
1745514
515+This arm is called when a program becomes unsubscribed to you.
516+Subscriptions may close because the subscriber intentionally
517+unsubscribed, but they also could be closed by an intermediary. For
518+example, if a subscription is from another ship which is currently
519+unreachable, Ames may choose to close the subscription to avoid queueing
520+updates indefinitely. If the program crashes while processing an
521+update, this may also generate an unsubscription. You should consider
522+subscriptions to be closable at any time.
1746523
1747−## Subscriptions
524+### +on-peek
1748525
1749−We've dealt fairly extensively with "poke" messages to an app, but these
1750−are somewhat limited. A poke is a one-way message, but more often we
1751−want to subscribe to updates from another app. You could build a
1752−subscription model out of one-way pokes, but it's such a common pattern
1753−that it's built into Arvo.
526+This arm is called when a program reads from the agent's "scry"
527+namespace, which should be referentially transparent. Unlike most
528+handlers, this cannot perform IO, and it cannot change the state. All
529+it can do is produce a piece of data to the caller, or not.
1754530
1755−Let's take a look at two apps, `:source` and `:sink`. First,
1756−`:source`:
1757−
1758−```hoon
1759−:: Sends subscription updates to sink.hoon
1760−::
1761−:::: /===/app/source/hoon
1762− ::
1763−!:
1764−::
1765−|%
1766− +$ move [bone card]
1767− +$ card $% [%diff diff-contents]
1768− ==
1769−+$ diff-contents $% [%noun *]
1770− ==
1771−--
1772−::
1773−|_ [bow=bowl:gall $~]
1774−::
1775−++ poke-noun
1776− |= non=*
1777− ^- [(list move) _+>.$]
1778− :_ +>.$
1779− %+ turn (prey:pubsub:userlib /example-path bow)
1780− |=([o=bone *] [o %diff %noun non])
1781−::
1782−++ peer-example-path
1783− |= pax=path
1784− ^- [(list move) _+>.$]
1785− ~& source+peer-notify+'Someone subscribed to you!'
1786− ~& source+[ship+src.bow path+pax]
1787− [~ +>.$]
1788−::
1789−++ coup
1790− |= [wir=wire err=(unit tang)]
1791− ^- [(list move) _+>.$]
1792− ?~ err
1793− ~& source+success+'Poke succeeded!'
1794− [~ +>.$]
1795− ~& source+error+'Poke failed. Error:'
1796− ~& source+error+err
1797− [~ +>.$]
1798−::
1799−++ reap
1800− |= [wir=wire err=(unit tang)]
1801− ^- [(list move) _+>.$]
1802− ?~ err
1803− ~& source+success+'Peer succeeded!'
1804− [~ +>.$]
1805− ~& source+error+'Peer failed. Error:'
1806− ~& source+error+err
1807− [~ +>.$]
1808−::
1809−--
1810−```
1811−
1812−Save this as `source.hoon` in the `/app` directory of your urbit's pier.
1813−
1814−And secondly, `:sink`:
1815−
1816−```hoon
1817−:: Sets up a simple subscription to source.hoon
1818−::
1819−:::: /===/app/sink/hoon
1820− ::
1821−!:
1822−|%
1823−+$ move [bone card]
1824−+$ card $% [%peer wire dock path]
1825− [%pull wire dock $~]
1826− ==
1827−--
1828−::
1829−|_ [bow=bowl:gall val=?]
1830−::
1831−++ poke-noun
1832− |= non=*
1833− ^- [(list move) _+>.$]
1834− ?: &(=(%on non) val)
1835− :_ +>.$(val |)
1836− :~ :* ost.bow
1837− %peer
1838− /subscribe
1839− [our.bow %source]
1840− /example-path
1841− ==
1842− ==
1843− ?: &(=(%off non) !val)
1844− :_ +>.$(val &)
1845− ~[[ost.bow %pull /subscribe [our.bow %source] ~]]
1846− ~& ?: val
1847− sink+unsubscribed+'You are now unsubscribed!'
1848− sink+subscribed+'You are now subscribed!'
1849− [~ +>.$]
1850−::
1851−++ diff-noun
1852− |= [wir=wire non=*]
1853− ^- [(list move) _+>.$]
1854− ~& sink+received-data+'You got something!'
1855− ~& sink+data+non
1856− [~ +>.$]
1857−::
1858−++ coup
1859− |= [wir=wire err=(unit tang)]
1860− ^- [(list move) _+>.$]
1861− ?~ err
1862− ~& sink+success+'Poke succeeded!'
1863− [~ +>.$]
1864− ~& sink+error+'Poke failed. Error:'
1865− ~& sink+error+err
1866− [~ +>.$]
1867−::
1868−++ reap
1869− |= [wir=wire err=(unit tang)]
1870− ^- [(list move) _+>.$]
1871− ?~ err
1872− ~& sink+success+'Peer succeeded!'
1873− [~ +>.$]
1874− ~& sink+error+'Peer failed. Error:'
1875− ~& sink+error+err
1876− [~ +>.$]
1877−::
1878−--
1879−```
1880−
1881−Save this as `sink.hoon` in the `/app` directory of your urbit's pier (make sure both `source` and `sink` are on the same ship).
1882−
1883−Two notes:
1884−
1885−- `bowl` is the type of the system state within our app. For example, it includes things like `our`, the name of the host urbit, and `now`, the current time.
1886−
1887−- You may have noticed the separate `|%` above the application core `|_`. We usually put our types in another core on top of the application core. We can access these type from our `|_` because in `.hoon` files, all cores are called against each other. (The shorthand for 'called' is `=>`.) Thus, the `|%` with the types is in the context of the `|_`, as it lies above it: `hoon.hoon` `=> |% w types => |_`
1888−
1889−Here's some sample output of the two working together:
1890−
1891−```
1892−~fintud-macrep:dojo> |start %source
1893−>=
1894−
1895−~fintud-macrep:dojo> |start %sink
1896−>=
1897−
1898−~fintud-macrep:dojo> :sink %on
1899−[%source %peer-notify 'Someone subscribed to you!']
1900−[%source [%ship ~fintud-macrep] %path /]
1901−[%sink %success 'Peer succeeded!']
1902−>=
1903−
1904−~fintud-macrep:dojo> :source 5
1905−[%sink %received-data 'You got something!']
1906−[%sink %data 5]
1907−>=
1908−
1909−~fintud-macrep:dojo> :sink %off
1910−>=
1911−
1912−~fintud-macrep:dojo> :source 6
1913−>=
1914−
1915−~fintud-macrep:dojo> :sink %on
1916−[%source %peer-notify 'Someone subscribed to you!']
1917−[%source [%ship ~fintud-macrep] %path /]
1918−[%sink %success 'Peer succeeded!']
1919−>=
1920−
1921−~fintud-macrep:dojo> :source 7
1922−[%sink %received-data 'You got something!']
1923−[%sink %data 7]
1924−>=
1925−```
1926−
1927−##### :source
1928−
1929−Hopefully you can get a sense for what's happening here. When we poke
1930−`:sink` with `%on`, `:sink` subscribes to `:source`,
1931−and so whenever we poke `:source`, `:sink` gets the update and
1932−prints it out. Then we unsubscribe by poking `:sink` with `%off`, and
1933−`:sink` stops getting updates. We then resubscribe.
1934−
1935−There's a fair bit going on in this code. Let's look at `:source`
1936−first.
1937−
1938−Our definition of `move` is fairly specific, since we're only going to
1939−sending one kind of move. The `%diff` move is a subscription update, and
1940−its content is marked data which gall routes to our subscribers.
1941−
1942−This is a slightly different kind of move than we've dealt with so
1943−far. It's producing a result rather than calling other
1944−code (i.e. it's a return rather than a function call), so if you recall
1945−the discussion of ducts, a layer gets popped off the duct rather than
1946−added to it. This is why no wire is needed for the move -- we won't
1947−receive anything in response to it.
1948−
1949−Anyways, there are four functions (arms) inside the `|_`. We already know when
1950−`++poke-noun` is called. `++peer-example-path` is called when someone tries to
1951−subscribe to our app. Of course, you don't just subscribe to an app; you
1952−subscribe to a path on that app. This path comes in as the argument to `++peer`.
1953−
1954−In our case, we don't care what path you subscribed on, and all we do is print
1955−out that you subscribed. Arvo keeps track of your subscriptions, so you don't
1956−have to. You can access your subscribers by looking at `sup` in the bowl that's
1957−passed in. `sup` is of type `(map bone {@p path})`, which associates bones with
1958−the urbit who subscribed, and which path they subscribed on. If you want to
1959−communicate with your subscribers, send them messages along their bone.
1960−
1961−`++poke-noun` "spams" the given argument to all our subscribers.
1962−There's a few things we haven't seen before. Firstly, `:_(a b)` is the
1963−same as `[b a]`. It's just a convenient way of formatting things when
1964−the first thing in a cell is much more complicated than the second.
1965−Thus, we're producing our state unchanged.
1966−
1967−Our list of moves is the result of a call to `++turn`. `++turn` is what many
1968−languages call "map" -- it runs a function on every item in a list and collects
1969−the results in a list. The list is `(prey:pubsub:userlib /example-path bow)` and the function
1970−is the `|=` line right after it.
1971−
1972−`prey:pubsub:userlib` is a standard library function defined in `zuse.hoon`. It takes a path
1973−and a bowl and gives you a list of the subscribers who are subscribed on a path
1974−that begins with the given path. "Prey" is short for "prefix".
1975−
1976−Now we have the list of relevant subscribers. This a list of triples,
1977−`[bone @p path]`, where the only thing we really need is the bone, because we
1978−don't need to know their urbit or what exact path they subscribed on. Thus, our
1979−transformer function takes `[o=bone *]` and produces `[o %diff %noun non]`,
1980−which is a move that provides bone `o` with this subscription update:
1981−`[%noun non]`". This is fairly dense code, but what it's doing is
1982−straightforward!
1983−
1984−##### :sink
1985−
1986−`:source` should now make sense. `:sink` is a little longer, but not much more
1987−complicated.
1988−
1989−In `:sink`, our definition of of `++move` is different. All moves start
1990−with a `bone`, and we conventionally refer to the second half as the "card", so
1991−that we can say a move is an action that sends a card along a bone.
1992−
1993−We have two kinds of cards here: we `%peer` to start a subscription, and we
1994−`%pull` to stop it. Both of these are "forward" moves that may receive a
1995−response, so they need a wire to tack onto the duct before they pass it on. They
1996−also need a target, which is a pair of an urbit and an app name. Additionally,
1997−`%peer` needs a path on that app to subscribe too. `%pull` doesn't need this,
1998−because its semantics are to cancel any subscriptions coming over this duct. If
1999−your bone and wire are the same as when you subscribed, then the cancellation
2000−will happen correctly.
2001−
2002−The only state we need for `:sink` is a boolean to indicate whether
2003−we're already subscribed to `:source`. We use `val=?`, where `?`
2004−is the type associated with boolean (similar to `*`, `@`), which defaults
2005−to `%.y` (that is, `0`).
2006−
2007−In `++poke-noun` we check our input to see both if it's `%on` and
2008−available (`val` is `%.y`). If so, we produce the move to subscribe to
2009−`:source`:
2010−
2011−```hoon
2012−:~ :* ost.bow
2013− %peer
2014− /subscribe
2015− [our.bow %source]
2016− /example-path
2017− ==
2018−==
2019−```
2020−
2021−Also, in the preceding lines, we set `val` to `%.n` (`|`) with `+>.$(val |)`.
2022−Remember that the `:_` constructs an inverted cell, with the first child
2023−(`+>.$(val |)` in our case) as the tail and the second child as the head. Here,
2024−the cell we produce when our subscription is `%on` and `val` is `%.y` has a
2025−head with our new state where `val` is set to `%.n` and a tail of our list of
2026−moves, which is shown in the code block above.
2027−
2028−Otherwise, if our input is `%off` and we're already subscribed (i.e. `val`
2029−is `%.n`), then we unsubscribe from `:source` and set `val` back to `%.y` (`&`),
2030−again using our handy inverted cell constructor mold `:_`:
2031−
2032−```hoon
2033−:_ +>.$(val &)
2034−~[[ost.bow %pull /subscribe [our.bow %source] ~]]
2035−```
2036−
2037−It's important to send over the same bone and wire (`/subscribe`) as the
2038−one we originally subscribed on.
2039−
2040−If neither of these cases are true, then we print our current subscription
2041−state, based on whether `val` is `%.y` or `%.n`, and return a cell containing
2042−a null list of moves and our unchanged app state:
2043−
2044−```hoon
2045−~& ?: val
2046− sink+unsubscribed+'You are now unsubscribed!'
2047− sink+subscribed+'You are now subscribed!'
2048−[~ +>.$]
2049−```
2050−
2051−`++diff-noun` is called when we get a `%diff` update along a subscription with a
2052−mark of `noun`. `++diff-noun` is given the wire that we originally passed with
2053−the `%peer` subscription request along and the data we got back. In our case we
2054−just print out the data:
2055−
2056−```hoon
2057−~& sink+received-data+'You got something!'
2058−~& sink+data+non
2059−```
2060−
2061−`++reap` is called when we receive an acknowledgment as to whether the
2062−subscription was handled successfully. You can remember that `++reap` is the
2063−counterpart to `++peer` as it's pronounced like 'peer' backwards. Similarly,
2064−`coup` is similar to 'poke' backwards.
2065−
2066−Moving forward, `++reap` is given the wire we attempted to subscribe over,
2067−possibly along with an error message in cases of failure. `(unit type)` means
2068−"either `~` or `[~ type]`, which means it's used like Haskell's "maybe" or C's
2069−nullability. If `err` is `~`, then the subscription was successful and we tell
2070−that to the user. Otherwise, we print out the error message.
2071−
2072−
2073−## Backend
2074−
2075−We've done purely functional web pages as hook files, and we've
2076−done command line apps, so now let's interact with an app over
2077−the web. The fundamental concepts are no different than from the
2078−command line -- you send "poke" messages to apps and subscribe to
2079−data streams. The only difference is that you do it with
2080−javascript.
2081−
2082−Let's create a small web app that counts the number of times you
2083−poke it. There'll be a button to poke the app, and line of text
2084−below it saying how many times the app has been poked. We can
2085−update this by subscribing our page to the relevant path on its app.
2086−
2087−Save the following as `click.hoon` in the `/app` directory of your urbit's pier.
2088−
2089−```hoon
2090−/? 314
2091−!:
2092−|%
2093−++ move [bone %diff %mark *]
2094−--
2095−!:
2096−|_ [hid=bowl:gall clicks=@]
2097−++ poke-click
2098− |= click=%click
2099− ~& [%poked +(clicks)]
2100− :_ +>.$(clicks +(clicks))
2101− %+ turn (prey:pubsub:userlib /the-path hid)
2102− |=([o=bone *] [o %diff %clicks +(clicks)])
2103−++ peer-the-path
2104− |= pax=path
2105− [[[ost.hid %diff %clicks clicks] ~] +>.$]
2106−--
2107−```
2108−
2109−There's nothing really new here, except that we use a couple of
2110−new marks, `click` and `clicks`. When we get poked with a
2111−`click`, we increment the variable `clicks` in our state and tell
2112−all our subscribers the new value. When someone subscribes to us
2113−on the path `/the-path`, we immediately give them the current
2114−number of clicks.
2115−
2116−Let's take a look at the new marks. Save this as `/mar/click.hoon`:
2117−
2118−```hoon
2119−|_ click=%click
2120−++ grab
2121− |%
2122− ++ noun |=(* %click)
2123− ++ json
2124− |= jon=^json
2125− ?> =('click' (need (so:dejs-soft:format jon)))
2126− %click
2127− --
2128−--
2129−```
2130−
2131−The mark `click` has type `%click`, which means the only
2132−valid value is `%click`. We can convert from `noun` by just
2133−producing `%click`.
2134−
2135−We also convert from json by parsing a json string with `so:dejs-soft:format`,
2136−asserting the parsing succeeded with `need`, and asserting the
2137−result was `'click'` with `?>`, which asserts that its first child
2138−is true.
2139−
2140−> Note the argument to `++json`is `jon=^json`. Why `^json`? `++json` shadows the type definition, so if we want to refer to the type, we have to prepend a `^`. This extends to multiple levels: `^^^foo` means the fourth most innermost instance of `foo`.
2141−
2142−> `dejs-soft:format` in `zuse` is a useful library for parsing complex json into hoon structures. In this case, the `:` between `dejs-soft` and `format` means 'inside of', because `dejs-soft` is an arm contained within the core `format`. Our case is actually simple enough that the `?>` line could have been `?> =([%s 'click'] jon)`.
2143−
2144−We can test this mark from the command line (don't forget to start your app with `|start %click`)
2145−
2146−```
2147−~fintud-macrep:dojo> &click %click
2148−%click
2149−
2150−~fintud-macrep:dojo> &click &json [%s 'click']
2151−%click
2152−
2153−~fintud-macrep:dojo> &click &json &mime [/text/json (as-octs:mimes:html '"click"')]
2154−%click
2155−
2156−~fintud-macrep:dojo> &click &json &mime [/text/json (as-octs:mimes:html '"clickety"')]
2157−/~fintud-macrep/home/0/mar/click:<[7 5].[8 11]>
2158−ford: casting %json to %click
2159−ford: cast %click
2160−```
2161−
2162−And save the following as `/mar/clicks.hoon`:
2163−
2164−```hoon
2165−|_ clicks=@
2166−++ grab
2167− |%
2168− ++ noun @
2169− --
2170−++ grow
2171− |%
2172− ++ json
2173− (frond:enjs:format %clicks (numb:enjs:format clicks))
2174− --
2175−--
2176−```
2177−
2178−`clicks` is just an atom. We convert to json by creating an
2179−object with a single field "clicks" with our value.
2180−
2181−> Be sure to check out section 3bD, JSON and XML, in `zuse.hoon`. `frond:enjs:format` is just a function that takes a key-value pair and produces a JSON object with one element.
2182−
2183−```
2184−~fintud-macrep:dojo> &json &clicks 6
2185−[%o p={[p='clicks' q=[%n p=~.6]]}]
2186−```
2187−
2188−Now that we know the marks involved, take another look at the app
2189−above. Everything should be pretty straightforward. Let's poke
2190−the app from the command line.
2191−
2192−```
2193−~fintud-macrep:dojo> |start %click
2194−>=
2195−~fintud-macrep:dojo> :click &click %click
2196−[%poked 1]
2197−>=
2198−~fintud-macrep:dojo> :click &click %click
2199−[%poked 2]
2200−>=
2201−```
2202−
2203−**Exercise**:
2204−
2205−- Modify `:sink` from the subscriptions chapter to listen to `:click` and print out the subscription updates on the command line.
2206−
2207−## Frontend
2208−
2209−> Note: the instructions below are outdated and do not work in the current version of Urbit. Feel free to experiment though.
2210−
2211−That's all that's needed for the back end. The front end is just
2212−some "sail" html (Hoon markup for XML) and javascript. Here's `/web/pages/click.hoon`:
2213−
2214−```
2215−;html
2216− ;head
2217− ;script(type "text/javascript", src "//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.min.js");
2218− ;script(type "text/javascript", src "/~/at/lib/js/urb.js");
2219− ;title: Clickety!
2220− ==
2221− ;body
2222− ;div##cont
2223− ;input##go(type "button", value "Poke!");
2224− ;div##err(class "disabled");
2225− ;div##clicks;
2226− ==
2227− ;script(type "text/javascript", src "/pages/click/click.js");
2228− ==
2229−==
2230−```
2231−
2232−You should recognize the sail syntax from an earlier chapter.
2233−Aside from jquery, we also include `urb.js`, which is a framework
2234−for interacting with urbit.
2235−
2236−To view the frontend, point your browser at `ship-name.urbit.org/~~/pages/click` if you're on the live network. If you're running a fake galaxy, navigate to whatever port it's running on, which is usually: `https://localhost:80/~~/pages/click`.
2237−
2238−We have a button labeled "Poke!" and a div with id `clicks` where
2239−we'll put the number of clicks. We also include a small
2240−javascript file where the client-side application logic can be
2241−found. It's in `/web/pages/click.js`:
2242−
2243−```js
2244−$(function() {
2245− var clicks, $go, $clicks, $err
2246−
2247− $go = $('##go')
2248− $clicks = $('##clicks')
2249− $err = $('##err')
2250−
2251− $go.on("click",
2252− function() {
2253− window.urb.send(
2254− "click", {mark: "click-click"}
2255− ,function(err,res) {
2256− if(err)
2257− return $err.text("There was an error. Sorry!")
2258− if(res.data !== undefined &&
2259− res.data.ok !== undefined &&
2260− res.data.ok !== true)
2261− $err.text(res.data.res)
2262− else
2263− console.log("poke succeeded");
2264− })
2265− })
2266−
2267− window.urb.appl = "click"
2268− window.urb.bind('/the-path',
2269− function(err,dat) {
2270− clicks = dat.data.clicks
2271− $clicks.text(clicks)
2272− }
2273− )
2274−})
2275−```
531+If this arm produces `[~ ~ data]`, then `data` is the value at the the
532+given `path`. If it produces `[~ ~]`, then there is no data at the given
533+`path` and never will be. If it produces `~`, then we don't know yet
534+whether there is or will be data at the given `path`.
2276535
2277−We set up two event handlers. When we click the button, we run
2278−`window.urb.send`, which sends a poke to the app specified as the first argument, in this case our `:click` app. The arguments
2279−are data, parameters, and the callback. The data is the specific
2280−data we want to send. The parameters are all optional, but here's a list
2281−of the available ones:
536+### +on-agent
2282537
2283−- `ship`: target urbit. Defaults to `window.urb.ship`, which
2284− defaults to the urbit which served the page.
2285−- `appl`: target app. Defaults to `window.urb.appl`, which
2286− defaults to null.
2287−- `mark`: mark of the data. Defaults to `window.urb.send.mark`,
2288− which defaults to "json".
2289−- `wire`: wire of the poke. Defaults to `/`.
538+This arm is called to handle responses to `%pass` moves to other agents.
539+It will be one of the following types of response:
2290540
2291−In our case, we specify only that the mark is "click".
541+- `%poke-ack`: acknowledgment (positive or negative) of a poke. If the
542+ value is `~`, then the poke succeeded. If the value is `[~ tang]`,
543+ then the poke failed, and a printable explanation (e.g. a stack trace)
544+ is given in the `tang`.
2292545
2293−The callback function is called when we receive an
2294−acknowledgment. If there was an error, we put it in the `err`
2295−div that we defined above. Otherwise, we printf to the console a
2296−message saying the poke succeeded. As is common, we gray
2297−out the button when it is clicked. It is only reenabled when positive acknowledgement is received.
546+- `%watch-ack`: acknowledgment (positive or negative) of a subscription.
547+ If negative, the subscription is already ended (technically, it never
548+ started).
2298549
2299−Our second event handler is `window.urb.bind`, which is called immediately when the page is loaded, subscribing the page to the specified data stream on the app set with `urb.appl` (which is, in this case, `:click`). It also takes three arguments: path, parameters, and callback. The path is the path on the app to subscribe to. The parameters are all optional (indeed, we omit them entirely here), but are similar to those for `send`:
550+- `%fact`: update from the publisher.
2300551
2301−- `ship`: target urbit. Defaults to `window.urb.ship`, which defaults to the urbit which served the page.
2302−- `appl`: target app. Defaults to `window.urb.appl`, which defaults to null.
2303−- `mark`: mark of expected data. Data of other marks is converted to this mark on the server before coming to the web front end. Defaults to `window.urb.bind.mark`, which defaults to "json".
2304−- `wire`: wire of the subscription. Defaults to the given path.
552+- `%kick`: notification that the subscription has ended.
2305553
2306−The callback function here just updates the data in the `clicks`
2307−div. A truly robust app would intelligently handle errors, of
2308−course.
554+### +on-arvo
2309555
2310−Note that the app doesn't have anything web-specific in itself.
2311−As far as it knows, it's just receiving pokes and subscriptions.
2312−The javascript is fairly pure as well, sending and receiving json
2313−everywhere. The marks are the translation layer, and they're the
2314−only things that need to know how the hoon types map to json.
556+This arm is called to handle responses to `%pass` `move`s to vanes. The
557+list of possible responses from the system is statically defined in
558+sys/zuse.hoon (grep for `+ sign-arvo`).
2315559
2316−**Exercise**:
560+### +on-fail
2317561
2318−- Open the app in multiple tabs, click the button, and verify that all the tabs stay in sink. Poke it manually from the command line and verify the tabs are updated as well.
562+If an error happens in `+on-poke`, the crash report goes into the
563+`%poke-ack` response. Similarly, if an error happens in
564+`+on-subscription`, the crash report goes into the `%watch-ack`
565+response. If a crash happens in any of the other handlers, the report
566+is passed into this arm.