@@ -1,1938 +1,113 @@
11+++
22title = "Ames"
3−weight = 1
3+weight = 2
44template = "doc.html"
55aliases = ["/docs/learn/arvo/ames/"]
66+++
77Our networking protocol.
88
9−`%ames` is the name of both our network and the vane that communicates
9+Ames is the name of both our network and the vane that communicates
1010over it. When Unix receives a packet over the correct UDP port, it pipes
11−it straight into `%ames` for handling. Also, all packets sent over the
12−`%ames` network are sent by the `%ames` vane. Apps and vanes may use
13−`%ames` to directly send messages to other ships. In general, apps use
14−gall and clay to communicate with other ships rather than using `%ames`
15−directly, but this isn't a requirement. Of course, gall and clay use
16−`%ames` behind the scenes to communicate across the network. These are
17−the only two vanes that use `%ames`.
18−
19−`%ames` includes several significant components. Although the actual
20−crypto algorithms are defined in zuse, they're used extensively in
21−`%ames` for encrypting and decrypting packets. Congestion control and
22−routing is handled entirely in `%ames`. Finally, the actual `%ames`
11+it straight into Ames for handling. Also, all packets sent over the
12+Ames network are sent by the Ames vane. Apps and vanes may use
13+Ames to directly send messages to other ships. In general, apps use
14+[Gall](@/docs/tutorials/arvo/gall.md) and [Clay](@/docs/tutorials/arvo/clay.md)
15+to communicate with other ships rather than using Ames
16+directly, but this isn't a requirement. Of course, Gall and Clay use
17+Ames behind the scenes to communicate across the network. Jael is the only other
18+vane to utilize Ames.
19+
20+Ames includes several significant components. Although the actual
21+crypto algorithms are defined in `zuse`, they're used extensively in
22+Ames for encrypting and decrypting packets. Congestion control and
23+routing is handled entirely in Ames. Finally, the actual Ames
2324protocol itself, including how to route incoming packets to the correct
24−vane or app, is defined in `%ames`.
25−
26−## Commentary
27−
28−`%ames` is our networking protocol.
29−
30−First we give commentary on the code, the algorithms involved, and the
31−protocol. We trace through the code touched when a packet is sent,
32−received, acknowledged, and that acknowledgment applied. This is fairly
33−comprehensive, and contains many implementation details, but if you
34−understand this, then you understand `%ames`.
35−
36−If you've scrolled down this page, you may be intimidated by the amount
37−of Hoon code, especially if you are new to the language. Don't be afraid
38−of it, you don't have to read any of it if you don't want to -- every
39−interesting action the code takes is explained in plain English. In
40−fact, if you are new to the language, this may be a good learning
41−opportunity. Even if you don't understand every line of Hoon code,
42−you'll hopefully be able to follow most lines. By the time you've worked
43−through this, you'll have seen many common patterns and best practices.
44−Hoon, much more than other languages, is best learned by reading and
45−understanding large quantities of existing code. In this way, it is
46−similar to learning a natural language. All of this code is in
47−`arvo/ames.hoon`.
48−
49−After the commentary, we have reference documentation for all the data
50−structures that are specific to `%ames`. If you see a data structure or
51−a variable used that you don't recognize, search for it in the code, and
52−it's very likely defined in one of these data structures. We recommend
53−that another tab is kept open for easy access to the data structure
54−reference documentation. The code for these is split between
55−`arvo/ames.hoon` and `arvo/zuse.hoon`.
56−
57−The Lifecycle of a Packet (or, How a Packet Becomes Law)
58−
59−---
60−
61−Here, we will trace a packet as it makes its way through ames. There are
62−actually two pathways through ames: the legacy path through `%want`, and
63−the modern way, entered through `%wont`, with full end-to-end
64−acknowledgments. Here we will only trace the modern way, though much of
65−the path is the same for both.
66−
67−When an app (or a vane) wishes to send a packet to another ship, it must
68−send a `%wont` card:
69−
70−```hoon
71− [%wont p=sock q=path r=*] :: e2e send message
72−```
73−
74−This card takes three arguments. The `p` is a `sock`, that is, a pair of
75−two ships, the first of which is the sender and the second is the
76−receiver. But wait, you ask, why do I get to decide who is the sender?
77−Can I fake like I'm someone else? The reason is that there are
78−potentially multiple ships on the same pier, and the kernel can send a
79−message from any of them. If you attempt to send a message from a ship
80−not on your pier, then ames will refuse to send it. If you hack around
81−in your own copy of ames to go ahead and send it anyway, then the other
82−ship will reject it because your key is bad. Only send messages from
83−yourself.
84−
85−The `q` is a path, representing the place on the other side that you
86−want to receive your message. It is approximately equivalent to a port
87−number. Messages on the same path are guaranteed to arrive in the same
88−order as they were sent. No such guarantees are made across paths.
89−
90−The `r` is the actual data that you are sending. As the type implies,
91−this can be an arbitrary noun, and it will be transferred to the
92−receiver exactly as-is, in a well-typed way. Of course, this is data
93−that is sent over the wire, so be careful not to send anything too
94−massive unless you're willing to wait.
95−
96−But enough about the interface. Grepping in ames.hoon for `%wont`, we
97−find that it appears in `++knob`. We see that we go
98−directly into `++wise:am`.
99−
100−```hoon
101− ++ wise :: wise:am
102− |= [soq=sock hen=duct cha=path val=* ete=?] :: send a statement
103− ^- [p=(list boon) q=fort]
104− zork:zank:(wool:(ho:(um p.soq) q.soq) hen cha val ete)
105−```
106−
107−The inputs to this gate are exactly the sort of thing you'd expect. In
108−particular, everything in the `%wont` gate is here plus the calling duct
109−so that we know where to send the acknowledgment and `ete` to determine
110−if we're going to do the modern end-to-end acknowledgments.
111−
112−The actual line of code looks intimidating, but it's really not all that
113−bad. Working from the inside out, the call to `++um` sets up our
114−domestic server, and the call to `++ho` sets up our knowledge about the
115−neighbor we're sending to. From the outside, `++zork` and `++zank` just
116−apply the changes made to our `++um` and `++am` cores, respectively. If
117−you're familiar with the common idiom of `++abet`, that's all this is.
118−The code predates the widespread usage of that name.
119−
120−The interesting part, then, is in `++wool:ho:um:am`. Let's look at the
121−code.
122−
123−```hoon
124− ++ wool :: wool:ho:um:am
125− |= [hen=duct cha=path val=* ete=?] :: send a statement
126− ^+ +>
127− =+ ^= rol ^- rill
128− =+ rol=(~(get by ryl.bah) cha)
129− ?~(rol *rill u.rol)
130− =+ sex=sed.rol
131− :: ~& [%tx [our her] cha sex]
132− =. ryl.bah
133− %+ ~(put by ryl.bah) cha
134− rol(sed +(sed.rol), san (~(put by san.rol) sex hen))
135− =+ cov=[p=p:sen:gus q=clon:diz]
136− %+ wind [cha sex]
137− ?: ete
138− [%bund q.cov cha sex val]
139− [%bond q.cov cha sex val]
140−```
141−
142−This is slightly more complicated, but it's still not all that bad. Our
143−inputs, at least, are fairly obvious.
144−
145−If you glance at the code for a second, you'll see that
146−`++wind:ho:um:am` seems to be able to send a message, or `++meal:ames`, given
147−a `++soup`. This gate, then, just sets up the things we need to for
148−`++wind` to do its job.
149−
150−We first get `rol`, which is a `++rill:ames`, that is, a particular outbound
151−stream. This stream is specific to the path on which we're sending. If
152−the path hasn't been used before, then we create it. We let `sex` be the
153−number of messages we've already sent on this path.
154−
155−Then, we update the outbound stream by incrementing the number of
156−messages sent and placing an entry in `san.rol` that associates the
157−message number with the `duct` that sent the message. This allows us to
158−give the acknowledgment to the one who sent the message.
159−
160−We let `cov` be the current life of our crypto and our neighbor's
161−crypto. At the moment, we only need our neighbor's life, which we put
162−into the meal.
163−
164−Finally, we call `++wind:ho:um:am` with the `++soup` of the path and
165−message number and the `++meal:ames` of the payload itself. For end-to-end
166−acknowledged messages, we use `%bund`.
167−
168−```hoon
169− [%bund p=life q=path r=@ud s=*] :: e2e message
170−```
171−
172−Looking at how we create the `%bund`, we can easily see what each field
173−is for.
174−
175−Following the trail a little further, we go to `++wind:ho:um:am`.
176−
177−```hoon
178− ++ wind :: wind:ho:um:am
179− |= [gom=soup ham=meal]
180− :: ~& [%wind her gom]
181− ^+ +>
182− =^ wyv diz (zuul:diz now ham)
183− =^ feh puz (whap:puz now gom wyv)
184− (busk xong:diz feh)
185−```
186−
187−`++wind` does three things: it (1) encodes the message into a list of
188−possibly-encrypted packets, (2) puts the message into the packet pump,
189−and (3) sends any packets that are ready to be sent. Yes, our nice
190−little linear run of each gate calling exactly one other interesting
191−gate is over. We'll go in order here.
192−
193−`++zuul:lax:as:go` is the what converts a `++meal:ames` into a list of
194−actual, 1KB packets.
195−
196−```hoon
197− ++ zuul :: zuul:lax:as:go
198− |= [now=@da ham=meal] :: encode message
199− ^- [p=(list rock) q=_+>]
200− =< weft
201− ++ wasp :: null security
202− ++ weft :: fragment message
203− ++ wisp :: generate message
204−```
205−
206−For organizational purposes, `++zuul` constructs an internal core with
207−three arms. `++wasp` encodes the meal into an atom with no encryption.
208−`++wisp` encodes a meal with possible encryption (else it simply calls
209−`++wasp`). `++weft` takes the result of `++wisp` and splits it into
210−actual packets.
211−
212−```hoon
213− ++ wasp :: null security
214− ^-([p=skin q=@] [%none (jam ham)])
215−```
216−
217−This simply jams the meal, wrapping it with the `skin` of `%none`,
218−meaning no encryption.
219−
220−Since `++wisp` is a little long, we'll go through it line-by-line.
221−
222−```hoon
223− ++ wisp :: generate message
224− ^- [[p=skin q=@] q=_..wisp]
225−```
226−
227−`++wisp` produces a pair of a `skin` and an atom, which is the meal
228−encoded as a single atom and possibly encrypted.
229−
230−```hoon
231− ?: =(%carp -.ham)
232− [wasp ..wisp]
233−```
234−
235−If the meal that we're encoding is a `%carp`, then we don't encrypt it.
236−A `%carp` meal is a partial meal, used when a message is more than 1KB.
237−Since the entire message is already encrypted, we don't need to encrypt
238−each packet individually again.
239−
240−```hoon
241− ?: !=(~ yed.caq.dur)
242− ?> ?=(^ yed.caq.dur)
243− :_ ..wisp
244− :- %fast
245− %^ cat 7
246− p.u.yed.caq.dur
247− (en:r:cluy q.u.yed.caq.dur (jam ham))
248−```
249−
250−If we have a symmetric key set up with this neighbor, then we simply use
251−it. The skin `%fast` is used to indicate a symmetric key.
252−
253−```hoon
254− ?: &(=(~ lew.wod.dur) |(=(%back -.ham) =(%buck -.ham)))
255− [wasp ..wisp]
256−```
257−
258−If we do not yet have our neighbor's will, then there is no way that we
259−can seal the message so that only they may read it. If what we're
260−sending is an acknowledgment, then we go ahead and just send it in the
261−clear.
262−
263−```hoon
264− =^ tuy +>.$
265− ?:(=(~ lew.wod.dur) [*code +>.$] (griz now))
266−```
267−
268−If we don't have our neighbor's will, then we "encrypt" with a key of 0.
269−If we do have their will, then we generate a new symmetric key that we
270−will propose.
271−
272−```hoon
273− :_ ..wisp
274− =+ yig=sen
275− =+ bil=law.saf :: XX send whole will
276− =+ hom=(jam ham)
277−```
278−
279−`yig` will be the life and engine for our current crypto. `bil` is our
280−will. `hom` is the meal encoded as a single atom.
281−
282−```hoon
283− ?: =(~ lew.wod.dur)
284− :- %open
285− %^ jam
286− [~ `life`p.yig]
287− bil
288− (sign:as:q.yig tuy hom)
289−```
290−
291−If we do not have our neighbor's will, then we send our current life
292−along with our will and the message. The message itself is "signed" with
293−a key of 0.
294−
295−```hoon
296− :- %full
297− =+ cay=cluy
298− %^ jam
299− [`life`p.cay `life`p.yig]
300− bil
301− (seal:as:q.yig pub:ex:r.cay tuy hom)
302− -- :: --zuul:lax:as:go
303−```
304−
305−If we do have our neighbor's will, then we send our perception of their
306−current life, our current life, our will, and the message. The message
307−is sealed with their public key so that only they can read our message.
308−
309−Once we have the message encoded as an atom, `++weft` goes to work.
310−
311−```hoon
312− ++ weft :: fragment message
313− ^- [p=(list rock) q=_+>.$]
314− =^ gim ..weft wisp
315− :_ +>.$
316− ^- (list rock)
317−```
318−
319−We're going to produce a list of the packets to send. First, we use the
320−aforementioned `++wisp` to get the message as an atom.
321−
322−```hoon
323− =+ wit=(met 13 q.gim)
324− ?< =(0 wit)
325−```
326−
327−`wit` is the number of 1KB (2\^13 bit) blocks in the message. We assert
328−that there is at least one block.
329−
330−```hoon
331− ?: =(1 wit)
332− =+ yup=(spit [our her] p.gim q.gim)
333− [yup ~]
334−```
335−
336−If there is exactly one block, then we just call `++spit` to turn the
337−message into a packet. We'll explain what `++spit` does momentarily.
338−
339−```hoon
340− =+ ruv=(rip 13 q.gim)
341− =+ gom=(shaf %thug q.gim)
342− =+ inx=0
343−```
344−
345−If there is more than one block, then we rip it into blocks in `ruv`.
346−`gom` is a hash of the message, used as an id. `inx` is the number of
347−packets we've already made.
348−
349−```hoon
350− |- ^- (list rock)
351− ?~ ruv ~
352− =+ ^= vie
353− %+ spit
354− [our her]
355− wasp(ham [%carp (ksin p.gim) inx wit gom i.ruv])
356− :- vie
357− $(ruv t.ruv, inx +(inx))
358−```
359−
360−Here we package each block into a packet with `++spit` and produce the
361−list of packets.
362−
363−```hoon
364− ++ spit :: cake to packet
365− |= kec=cake ^- @
366− =+ wim=(met 3 p.p.kec)
367− =+ dum=(met 3 q.p.kec)
368− =+ yax=?:((lte wim 2) 0 ?:((lte wim 4) 1 ?:((lte wim 8) 2 3)))
369− =+ qax=?:((lte dum 2) 0 ?:((lte dum 4) 1 ?:((lte dum 8) 2 3)))
370− =+ wix=(bex +(yax))
371− =+ vix=(bex +(qax))
372− =+ bod=:(mix p.p.kec (lsh 3 wix q.p.kec) (lsh 3 (add wix vix) r.kec))
373− =+ tay=(ksin q.kec)
374− %+ mix
375− %+ can 0
376− :~ [3 1]
377− [20 (mug bod)]
378− [2 yax]
379− [2 qax]
380− [5 tay]
381− ==
382− (lsh 5 1 bod)
383−```
384−
385−This is how we turn a message into a real packet. This has the
386−definition of the packet format.
387−
388−`wim` is the length of the sending ship, and `dum` is the length of the
389−receiving ship. There are only five possibilities for each of those,
390−corresponding to carriers, cruisers, destroyers, yachts, and submarines.
391−These are encoded in `yax` and `qax` as 0, 0, 1, 2, and 3, respectively.
392−Thus, `wix` and `vix` are the number of bytes that must be reserved for
393−the ship names in a packet.
394−
395−Next, we construct `bod` by simply concatenating the sending ship, the
396−receiving ship, and the body of the message. Then, we get the encryption
397−mechanism from `++skin:ames`, which may be a 0, 1, 2, or 3, and put it in
398−`tay`.
399−
400−Next, we concatenate together, bit by bit, some final metadata. We use
401−three bits for our protocol number, which is incremented modulo eight
402−when there is a continuity breach or the protocol changes. We use the
403−final twenty bits of a hash of the body (which, we suppose, makes it a
404−twenty bit hash) for error-checking. We use two bits to tell how much
405−room is used in the body for the sending ship, and another two bits for
406−the receiving ship. Finally, we use five bits to store the encryption
407−type. Note that since there are only two bits worth of encryption types,
408−there are three unused bits here. This adds up to 32 bits of header
409−data. Finally, we concatenate this onto the front of the packet. Thus,
410−we can summarize the packet header format as follows.
411−
412−```
413− 0 1 2 3
414− 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
415− +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
416− |Proto| Hash of Body |yax|qax| Crypto |
417− +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
418−```
419−
420−After this, there are `yax` bits of the sender name, `qax` bits of the
421−receiver name, and up to 8192 bits of data. Thus, the maximum size of a
422−packet is achieved in a message between two submarines with 8192 bits of
423−data. This will require 32+128+128+8192 = 8480 bits, or 1060 bytes.
424−
425−This concludes our discussion of `++zuul:lax:as:go`. If you recall from
426−`++wind:ho:um:am`, the list of packets from `++zuul` is passed into
427−`++whap:pu` to update the packet pump and get any packets that can be
428−sent immediately.
429−
430−```hoon
431− ++ whap :: whap:pu
432− |= [now=@da gom=soup wyv=(list rock)] :: send a message
433− ^- [(list rock) _+>]
434− =. pyz (~(put by pyz) gom (lent wyv))
435− =. +>
436− |- ^+ +>.^$
437− ?~ wyv +>.^$
438− %= $
439− wyv t.wyv
440− nus +(nus)
441− diq (~(put by diq) (shaf %flap i.wyv) nus)
442− puq (~(put to puq) [nus `soul`[gom 0 | ~2000.1.1 i.wyv]])
443− ==
444− (harv now)
445−```
446−
447−First, we put into `pyz` the id for this message and the number of its
448−packets that have not yet been acknowledged, which is of course the
449−total number of packets since we haven't even sent the packets.
450−
451−For every packet, we change three things in the state (`++shed:ames`) of our
452−packet pump: (1) we increment `nus`, the number of packets sent; (2) we
453−put the packet number into `diq` keyed by a hash of the packet; and (3)
454−we put the packet into the packet queue, with the basic metadata of its
455−id `gom`, 0 transmissions, not live yet, last sent in the year 2000, and
456−the packet itself.
457−
458−Finally, we harvest the packet pump.
459−
460−```hoon
461− ++ harv :: harv:pu
462− |= now=@da :: harvest queue
463− ^- [(list rock) _+>]
464− ?: =(~ puq) [~ +>(rtn ~)]
465− ?. (gth caw nif) [~ +>]
466− =+ wid=(sub caw nif)
467− =| rub=(list rock)
468− =< abet =< apse
469− |%
470−```
471−
472−`++harv` contains a core for most of its work. The meat is in `++apse`.
473−First, though, it sets itself up. If there aren't any packets in the
474−queue, then we simply do nothing except set `rtn`, our next timeout, to
475−nil because we don't have any packets that may need to be retransmitted.
476−If we have more live (that is, sent and unacknowledged) packets than our
477−window size, then we don't do anything.
478−
479−Otherwise, we let `wid` be the width of our remaining packet window, and
480−we initialize `rub` to nil. `rub` will be the list of packets that are
481−ready to be sent. We then call `++apse` and pass the result to `++abet`.
482−`++apse` decides which packets are ready to be sent.
483−
484−```hoon
485− ++ apse
486− ^+ .
487− ?~ puq .
488− ?: =(0 wid) .
489− => rigt =< left
490− ?> ?=(^ puq)
491− ?: =(0 wid) .
492− ?. =(| liv.q.n.puq) .
493− :: ~& [%harv nux.q.n.puq p.n.puq]
494− %_ .
495− wid (dec wid)
496− rub [pac.q.n.puq rub]
497− nif +(nif)
498− liv.q.n.puq &
499− nux.q.n.puq +(nux.q.n.puq)
500− lys.q.n.puq now
501− ==
502−```
503−
504−If there are no remaining packets to send, or if we've filled the packet
505−window, do nothing. We call `++rigt` and `++left` to process the left
506−and right branches of the packet queue.
507−
508−Now we assert that the queue is not empty, and we again check that we
509−haven't filled the packet window. We will operate on the head of the
510−queue. If the packet is live, then do nothing. Otherwise, we go ahead
511−and send it.
512−
513−To send, we (1) decrement `wid`, our packet window width; (2) cons the
514−packet onto the `rub`, which will be returned as the list of packets to
515−send; (3) increment `nif`, the number of live packets; (4) set the
516−packet to be live; (5) increment the number of transmissions of the
517−packet; and (6) set the last sent time of the packet to now.
518−
519−```hoon
520− ++ left
521− ?> ?=(^ puq)
522− ^+(. =+(lef=apse(puq l.puq) lef(puq [n.puq puq.lef r.puq])))
523− ++ rigt
524− ?> ?=(^ puq)
525− ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
526−```
527−
528−These do exactly what you would expect: they traverse the packet queue
529−so that `++apse` gets called recursively through it.
530−
531−Finally, `++abet` gets called, which resolves the changes.
532−
533−```hoon
534− ++ abet
535− ?~ rub [~ +>.$]
536− [(flop rub) +>.$(rtn [~ (add rto now)])]
537−```
538−
539−This returns the packets that we wish to send, and it updates the
540−timeout so that we know when to try resending unacknowledged packets.
541−
542−This concludes our discussion of `++whap:pu`. To finish
543−`++wind:ho:um:am`, we just need to delve into `++busk:ho:um:am`. But
544−wait, in the call to `++busk`, the first argument is `xong:diz`. What is
545−this? This, my dear reader, is one more detour, this time into
546−`++xong:lax:as:go`.
547−
548−```hoon
549− ++ xong :: xong:lax:as:go
550− ^- (list ship) :: route unto
551− =+ [fro=xen too=xeno]
552− =+ ^= oot ^- (list ship)
553− =| oot=(list ship)
554− |- ^+ oot
555− ?~ too ~
556− ?: (lien fro |=(a=ship =(a i.too))) ~
557− [i.too $(too t.too)]
558− :: ~& [%xong-to [our her] (weld oot ?>(?=(^ fro) t.fro))]
559− (weld oot ?>(?=(^ fro) t.fro))
560−```
561−
562−This gets the list of intermediate ships needed to get a packet from us
563−to our neighbor. First, we get `fro` and `too`, the "canons" of ourself
564−and our neighbor, respectively.
565−
566−What is this "canon", you ask? A canon is simply a ship plus its
567−"ancestors", as defined by `++sein:title`. For example, the canon of
568−`~hoclur-bicrel` is:
569−
570−```
571− ~hoclur-bicrel/try=> (saxo:title ~hoclur-bicrel)
572− ~[~hoclur-bicrel ~tasruc ~tug]
573−```
574−
575−If we follow the algorithm in `++xong`, we see that we are simply
576−creating a list of ships that form a path from our neighbor to ourself.
577−Essentially, we look through the canon of our neighbor until we find
578−something in our own cannon -- a common ancestor. Or, if we are from
579−different carriers, then there is no common ancestor. We then weld this
580−onto the tail of our own canon. In the end, this is simply a list of
581−possible ships to try to route via to get to our neighbor, ordered by
582−preferability (that is, closeness to our neighbor). We will end up
583−trying, in order, to find a lane to these.
584−
585−Now, we can finally get to `++busk:ho:um:am`.
586−
587−```hoon
588− ++ busk :: busk:ho:um:am
589− |= [waz=(list ship) pax=(list rock)] :: send packets
590− %_ +>
591− bin
592− |- ^+ bin
593− ?~ pax bin
594− $(pax t.pax, bin (weld (flop (wist:diz now waz ~ i.pax)) bin))
595− ==
596−```
597−
598−Thankfully, `++busk` is fairly simple. We go through the list of packets
599−and convert them to `++boon:ames`s with `++wist:lax:as:go`. These boons are
600−placed into `bin`, and they end up getting processed by `++clop` (this
601−happens in `++knob`).
602−
603−```hoon
604− ++ wist :: wist:lax:as:go
605− |= $: now=@da :: route via
606− waz=(list ,@p)
607− ryn=(unit lane)
608− pac=rock
609− ==
610− ^- (list boon)
611− ?: =(our her) [[%ouzo *lane pac] ~]
612− ?~ waz ~
613− =+ dyr=?:(=(her i.waz) dur (gur i.waz))
614− ?. ?& !=(our i.waz)
615− ?=(^ lun.wod.dyr)
616− ==
617− $(waz t.waz)
618− :_ ?: ?=(%ix -.u.lun.wod.dyr)
619− $(waz t.waz)
620− ~
621− :+ %ouzo u.lun.wod.dyr
622− ?: &(=(i.waz her) =(~ ryn)) pac
623− =+ mal=(jam `meal`[%fore her ryn pac])
624− %- spit
625− ^- cake
626− :* [our i.waz]
627− ?~ yed.caq.dyr [%none mal]
628− :- %fast
629− %^ cat 7
630− p.u.yed.caq.dyr
631− (en:crub:crypto q.u.yed.caq.dyr mal)
632− ==
633−```
634−
635−This takes a sample of the current time, the list of ships that we just
636−generated, a lane if we already know it, and the packet itself.
637−
638−First, if we are sending a message to ourself, then we simply create a
639−`%ouzo` boon with a bunted lane. Otherwise, if there are no routing
640−candidates, there is nothing we can do, so we return nil.
641−
642−Next, we get the `++dore:ames` of the first routing candidate. If we're looking
643−at the neighbor to whom we're trying to send the message, then we simply
644−use the `++dore:ames` that we already have. Otherwise, we get a default `++dore:ames`.
645−
646−If we're the first routing candidate, or if we have don't have a lane to
647−this candidate, then we skip this candidate and move on to the next one.
648−
649−If we have only a provisional ip address, then we try to send on it, but
650−we also try to send on later routing candidates as well. Otherwise, we
651−only send on this one candidate.
652−
653−Finally, we create the actual `%ouzo` boon. The lane is the one from our
654−`++dore:ames`. If we're sending it directly to our intended recipient, and we
655−haven't been told to use a specific lane, then we just send the packet
656−directly. Otherwise, we wrap it in a little `%fore` meal, telling the
657−intermediary to whom we wish it to be sent. If we have already set up a
658−symmetric key with the intermediary, then we encrypt it with that.
659−Otherwise, we send it in the clear.
660−
661−Now, if you recall, we have traced all the way through from the
662−beginning when, in `++knob`, the `%wont` card was handled by a call to
663−`++wise`. There is only one more step before the packet is finally sent.
664−Looking in `++knob`, we see that the resultant list of boons is passed
665−into `++clop`, which will execute the correct actions and return a list
666−of moves. In `++clop`, we see the handling of each specific boon. The
667−one we are interested in is `%ouzo`, since that is the only one we have
668−sent thus far.
669−
670−```hoon
671− %ouzo
672− :: ~& [%send now p.bon `@p`(mug (shaf %flap q.bon))]
673− :_ fox
674− [[gad.fox [%give %send p.bon q.bon]] ~]
675−```
676−
677−Very simply, we give a `%send` gift along the special duct that goes
678−straight into the bowels of unix. This is the last stop before we drop
679−into vere, and later libuv. And then... the world.
680−
681−The packet, after its creation, embarks on a journey across physical
682−time and space into the great unknown. Hurtling through fiber-optic
683−cables at hundreds of thousands of kilometers per second, it finally
684−arrives at our neighbor's network adapter. The adapter tells unix, unix
685−tells libuv, libuv tells vere, and vere sends a `%hear` task to ames.
686−And now we reenter the kernel.
687−
688−The `%hear` task goes straight to `++knob`, just as did the `%wont` task
689−earlier.
690−
691−```hoon
692− %hear
693− (~(gnaw am [now fox]) %good p.kyz q.kyz)
694−```
695−
696−Here, though, we call `++gnaw:am` to process the packet. The arguments
697−to `++gnaw` are the same as those to the `%hear` task: the lane on which
698−the packet was received and the packet itself. The other argument is
699−just `%good`, which is a `++cape:ames` saying that we expect the packet to
700−succeed. If a formal error occurs, then since we have a transactional
701−event system, the `%hear` event will never be considered to have
702−actually happened, and unix will send a `%hole` task so that we may send
703−a negative acknowledgment.
704−
705−```hoon
706− ++ gnaw :: gnaw:am
707− |= [kay=cape ryn=lane pac=rock] :: process packet
708− ^- [p=(list boon) q=fort]
709− ?. =(2 (end 0 3 pac)) [~ fox]
710− =+ kec=(bite pac)
711− ?: (goop p.p.kec) [~ fox]
712− ?. (~(has by urb.ton.fox) q.p.kec)
713− [~ fox]
714− =< zork
715− =< zank
716− %- ~(chew la:(ho:(um q.p.kec) p.p.kec) kay ryn %none (shaf %flap pac))
717− [q.kec r.kec]
718−```
719−
720−First, we check the protocol number. If it is not correct, then we
721−simply ignore the packet entirely. Otherwise, we parse the packet with
722−`++bite`, which converts a packet atom into a `++cake:ames`, that is, a triple
723−of the `sock` (pair of sender and receiver), the `skin` (encryption
724−type), and the data.
725−
726−```hoon
727− ++ bite :: packet to cake
728− |= pac=rock ^- cake
729− =+ [mag=(end 5 1 pac) bod=(rsh 5 1 pac)]
730− =+ :* vez=(end 0 3 mag) :: protocol version
731− chk=(cut 0 [3 20] mag) :: checksum
732− wix=(bex +((cut 0 [23 2] mag))) :: width of receiver
733− vix=(bex +((cut 0 [25 2] mag))) :: width of sender
734− tay=(cut 0 [27 5] mag) :: message type
735− ==
736− ?> =(2 vez)
737− ?> =(chk (end 0 20 (mug bod)))
738− :+ [(end 3 wix bod) (cut 3 [wix vix] bod)]
739− (kins tay)
740− (rsh 3 (add wix vix) bod)
741−```
742−
743−This is exactly the inverse of `++spit`. Note that here we check both
744−the protocol number and the hash, crashing on error. Remember that a
745−crash will result in a negative acknowledgment being sent.
746−
747−Continuing in `++gnaw`, we see that if the intended recipient is not on
748−our pier, then we drop the packet.
749−
750−If we've gotten this far, then we wish to process the packet. Recall
751−that `++ho` and `++um` set up the domestic server and foreign client
752−cores, respectively, and that `++zork` and `++zank` resolve any changes
753−to these cores.
754−
755−The new stuff here, then, is the `++la` core and the `++chew` arm. The
756−`++la` sets up a core for this particular packet, containing the current
757−success/failure `++cape:ames`, the lane it was sent on, the encryption type,
758−and a hash of the packet, used as an id.
759−
760−`++chew` is called with the encryption type and the message itself. It
761−contains a little helper core inside of it, which starts immediately
762−with `++apse`.
763−
764−```hoon
765− ++ apse
766− ^+ +>.$
767− =+ oub=bust:puz
768− =+ neg==(~ yed.caq.dur.diz)
769− =. +>.$ east
770− =+ eng==(~ yed.caq.dur.diz)
771− =+ bou=bust:puz
772− =. bin
773− ?. &(oub !bou) bin
774− :_(bin [%wine [our her] " is ok"])
775− =. bin
776− ?. &(neg !eng) bin
777− :_(bin [%wine [our her] " is your neighbor"])
778− +>.$
779−```
780−
781−First, we let `oub` be true if our neighbor hasn't been responding to us
782−for more than sixteen seconds. Let `neg` be true if we haven't yet
783−proposed a symmetric key, meaning that we haven't yet corresponded with
784−this ship, so they are not our neighbor. Next, we run `++east`, which
785−we'll go into in just a minute.
786−
787−We now do the same two checks and store the results in `eng` and `bou`.
788−If our neighbor has, like the prodigal son, returned after an extended
789−absence, then we send a `%wine` boon as the proverbial fatted calf,
790−which is simply printed out to the console. Likewise, if we are meeting
791−one with whom we have never had the pleasure of acquainting ourselves,
792−we send a message to the console to that effect.
793−
794−We skipped over `++east`, which contains the meat of the processing. It
795−first decrypts the message, then calls `++chow:la:ho:um:am` with the
796−resultant meal. We'll go through each of the four cases in turn, but
797−first since each one calls `++bilk:pu`, we'll take a brief detour.
798−
799−```hoon
800− ++ bilk :: bilk:pu
801− |= now=@da :: inbound packet
802− ^+ +>
803− =+ trt=(mul 2 rtt)
804− %= +>.$
805− rue [~ now]
806− rto trt
807− rtn ?~(puq ~ [~ (add now trt)])
808− ==
809−```
810−
811−This updates the timing information in our packet pump. `rue`, the last
812−time we have heard from this neighbor, is set to now. `rto`, the
813−retransmit timeout is set to twice the current ping time, and if there
814−is anything in the packet queue, then we reset the next timeout, since
815−we've just heard a message.
816−
817−Back to `++east`.
818−
819−```hoon
820− %none
821− =. puz (bilk:puz now)
822− (chow ((hard meal) (cue msg)))
823−```
824−
825−The simplest case is when the encryption type is `%none`. We first call
826−`++bilk` to update the packet pump, then we cue (unjam) the message into
827−a meal. We hard cast it into a meal -- if the cast fails, then we do
828−want to crash since someone is sending us malformed data. Finally, we
829−send the result to `++chow` for interpretation and handling.
830−
831−```hoon
832− %fast
833− =+ [mag=`hand`(end 7 1 msg) bod=(rsh 7 1 msg)]
834− =+ dey=(kuch:diz mag)
835− ?~ dey
836− ~& [%bad-key her mag]
837− +>.$ :: ignore unknown key
838− =. puz (bilk:puz now)
839− =^ key diz u.dey
840− (chow(aut sin) ((hard meal) (cue (dy:q:sen:gus key bod))))
841−```
842−
843−For symmetric encryption, we first get the `++hand`, which is the hash of
844−the symmetric key. We pass it to `++kuch:lax:as:go`, which returns the
845−key if we either have used it before or we have proposed it. If we have
846−proposed it, then we change its status from proposed to real. If
847−`++kuch` fails, then we drop the packet and print out a `%bad-key`
848−message.
849−
850−Otherwise, we call `++bilk` as before to update the packet pump and pass
851−into `++chow` the decrypted data.
852−
853−```hoon
854− %full
855− =+ mex=((hard ,[p=[p=life q=life] q=will r=@]) (cue msg))
856− =. diz (deng:diz q.mex)
857− =+ wug=cluy:diz
858− ?> =(q.p.mex p.wug)
859− =+ gey=(sev:gus p.p.mex)
860− =+ mes=(need (tear:as:q.gey pub:ex:r.wug r.mex))
861− =. diz (wasc:diz p.mes)
862− =. puz (bilk:puz now)
863− (west(msg q.mes))
864−```
865−
866−For sealed asymmetric encryption, we first take off the the layer of
867−data that gives us the life and will of our neighbor, and we apply try
868−to extend their former will with the new data. `++deng` will fail if
869−this is impossible.
870−
871−Next, we get our most current understanding of our neighbor's crypto,
872−and we verify that it's the same life as what they're sending. Then, we
873−get our own crypto from `++sev` and decrypt the message with the public
874−key from our neighbor's crypto. We register the proposed symmetric key,
875−update the packet pump, and call `++west`, which simply casts the
876−message to a meal and calls `++chow`, reporting any error.
877−
878−```hoon
879− %open
880− =+ mex=((hard ,[p=[~ q=life] q=will r=@]) (cue msg))
881− =. diz (deng:diz q.mex)
882− =+ wug=cluy:diz
883− ?> =(q.p.mex p.wug)
884− =+ mes=(need (sure:as:r.wug *code r.mex))
885− =. puz (bilk:puz now)
886− (west(msg mes))
887−```
888−
889−Finally, for signed asymmetric encryption, we, as before, take off the
890−layer of data that gives us the life and will of our neighbor. This
891−time, of course, we do not get our own crypto -- only that of our
892−neighbor.
893−
894−The rest you have seen. We call `++deng` to extend the will, we verify
895−that their crypto life is what we think it ought to be, we "decrypt" the
896−data, we update the packet pump, and we call `++west` to call `++chow`.
897−
898−```hoon
899− ++ chow :: chow:la:ho:um:am
900− |= fud=meal :: interpret meal
901− ^+ +>
902− =. diz ?:(=(%none aut) diz (wast:diz ryn))
903− (dine fud)
904−```
905−
906−Here, if the message was encrypted at all, then we call
907−`++wast:lax:as:go`, which simply updates the lane (route) to our
908−neighbor (unless we're given a provisional route). This ensures that we
909−always have the most direct possible path to them.
910−
911−We've been handling this meal for so long, we've almost forgotten what
912−we want to do with it. The telos is of any meal to be dined on. We will
913−choose out the cases here that are important to our current
914−investigation.
915−
916−```hoon
917− %fore
918− =+ ^= lyn ^- lane
919− ?~ q.fud ryn
920− ?. ?=(%if -.u.q.fud) u.q.fud
921− [%ix +.u.q.fud]
922− :: u.q.fud
923− ?: =(our p.fud)
924− (emit %mead lyn r.fud)
925− =+ zid=(myx:gus p.fud)
926− (emir (wist:zid now xong:zid [~ lyn] r.fud))
927−```
928−
929−Forwarding is the simplest case, since we've seen all the arms before,
930−except perhaps `++emit` and `++emir`, which simply take a boon or list
931−of boons respectively and queue them up to be handled when the core
932−resolves. If we're told to forward a packet to ourselves, then we emit a
933−`%mead` boon which simply sends another `%hear` task to ourselves with
934−the data. Otherwise, we try to find a route to the recipient, as before.
935−
936−```hoon
937− %carp
938− =+ zol=(~(get by olz.weg) s.fud)
939− ?^ zol cock(kay u.zol)
940− =^ neb nys.weg
941− =+ neb=(~(get by nys.weg) s.fud)
942− ?^ neb [u.neb nys.weg]
943− =+ neb=`bait`[(kins p.fud) 0 r.fud ~]
944− [neb (~(put by nys.weg) s.fud neb)]
945− ?> (lth q.fud p.r.neb)
946− ?> =((kins p.fud) p.neb)
947− ?> =(r.fud p.r.neb)
948− =+ doy=`(unit ,@)`(~(get by q.r.neb) q.fud)
949− ?^ doy cock
950− => ^+ . %= .
951− q.r.neb (~(put by q.r.neb) q.fud t.fud)
952− q.neb +(q.neb)
953− ==
954− :: ~& [%carp q.fud s.fud q.neb p.r.neb]
955− ?: =(q.neb p.r.neb)
956− =: nys.weg (~(del by nys.weg) s.fud)
957− olz.weg (~(put by olz.weg) s.fud kay)
958− ==
959− (golf p.neb r.neb)
960− =. +>.$ cock
961− +>.$(nys.weg (~(put by nys.weg) s.fud neb))
962−```
963−
964−Here, we have received a partial message, and we're just assembling the
965−individual packets into a message. Most of this code is fairly
966−algorithmic, so we'll just hit the high points. In the beginning, we
967−check if we've already received this message, and if so, we resend the
968−acknowledgment. Remember, "always ack a dupe, never ack an ack".
969−
970−In `nys.weg` we keep track of an incoming set of partial packets,
971−indexed by the `flap` hash that comes with every packet. We check to see
972−if we have already received this partial message, and if so we
973−acknowledge it. Otherwise, we put it in `nys.weg` unless this is the
974−last message, in which case we ack the last partial message, move the
975−complete message into `olz.weg`, and call `++golf`, which assembles the
976−message and calls `++chew`, to start the dance again with the complete
977−message.
978−
979−```hoon
980− %bund
981− :: ~& [%bund q.fud r.fud]
982− ?> =(p:sen:gus p.fud)
983− (deer q.fud r.fud ?-(kay %dead ~, %good [~ s.fud]))
984−```
985−
986−What if we're just receiving a regular old, garden variety message? We
987−call `++deer` with the data from the message. If we already know that
988−the message processing will fail (that is, if we got a `%hole` card from
989−unix rather than a `%hear` card), then we don't even send the data at
990−all. Remember, if a packet fails to process, it's as if it never even
991−arrived, except that we send a negative acknowledgment.
992−
993−```hoon
994− ++ deer :: deer:la:ho:um:am
995− |= [cha=path num=@ud dut=(unit)] :: interpret message
996− ^+ +>
997− =+ rum=(fall (~(get by raz.bah) cha) *race)
998− %= +>.$
999− +>
1000− ?. (gte num did.rum) :: always ack a dup
1001− (cook (~(get by bum.rum) num) cha ~ ryn dam)
1002− ?: dod.rum
1003− (coat cha rum(mis (~(put by mis.rum) num [kay ryn dam dut])))
1004− %= +>.+>.$
1005− raz.bah
1006− %+ ~(put by raz.bah) cha
1007− rum(mis (~(put by mis.rum) num [kay ryn dam dut]))
1008− ==
1009− ==
1010−```
1011−
1012−First, we get the race for this particular triple of sender, receiver,
1013−and path, creating it if it doesn't exist. If we've already acked the
1014−message, then we resend the ack. Note that `did.rum` is the number of
1015−packets we acknowledged, positively or negatively while `bum.rum` is a
1016−map of message numbers to negative acknowledgments. Thus, if a message
1017−number is less than `did.rum`, then if it's in `bum.rum` then it was
1018−negatively acknowledged, otherwise it's positively acknowledged. Thus, we
1019−are constant in space with the number of successful messages and linear
1020−in the number of failed messages. We'll document `++cook` later on, but
1021−suffice it to say that it sends an acknowledgment. It is to end-to-end
1022−acknowledgments what `++cock` is to packet-level acknowledgments.
1023−
1024−If we are still processing a message (that is, `dod.rum` is false), then
1025−we simply put this message in the map of misordered packets to be
1026−processed when their time comes. "Processing a message" in this case
1027−means that we've received the message and notified the correct
1028−application, but we're still waiting for the application-level
1029−acknowledgment.
1030−
1031−Otherwise, we're ready for a packet, so we process it.
1032−
1033−```hoon
1034− ++ coat :: coat:ho:um:am
1035− |= [cha=path rum=race] :: update input race
1036− ^+ +>
1037− =+ cun=(~(get by mis.rum) did.rum)
1038− ?~ cun
1039− +>.$(raz.bah (~(put by raz.bah) cha rum))
1040− ?. =(%good p.u.cun) +>.$
1041− ?> ?=(^ s.u.cun)
1042− %= +>.$
1043− raz.bah (~(put by raz.bah) cha rum(dod |))
1044− bin
1045− :_ bin
1046− :^ %mulk
1047− [our her]
1048− `soap:ames`[[p:sen:gus clon:diz] cha did.rum]
1049− u.s.u.cun
1050− ==
1051−```
1052−
1053−First, we grab the message we want to process and store it in `cun`. If
1054−it's a good packet, then we change `dod.rum` to false, meaning that
1055−we're in the middle of processing a packet and should not start
1056−processing another one. We also put a `%mulk` boon into the queue so
1057−that, when it all resolves, we send a mesage to the intended recipient
1058−application. The boon contains the sender, the receiver, the identity of
1059−the message, and the message itself.
1060−
1061−This bubbles up all the way back to `++knob`, where we were handling the
1062−`%hear` card. Following the logic in `++knob`, we can see that the boons
1063−get sent into `++clop` to be turned into actual arvo-level moves. We've
1064−been here before, if you recall, when we handled the `%cake` boon to
1065−send a message. Now, we're handling the `%mulk` boon, which is
1066−unfortunately slightly more complicated.
1067−
1068−```hoon
1069− %mulk
1070− :: ~& [%mulk p.bon q.bon]
1071− ?> ?=([@ @ *] q.q.bon)
1072− ?> ?=(%q i.q.q.bon)
1073− ?+ i.t.q.q.bon
1074− ~& %mulk-bad
1075− :_ fox
1076− :~ :- (claw p.p.bon)
1077− [%sick %wart p.bon i.t.q.q.bon t.t.q.q.bon r.bon]
1078− ==
1079− %ge :: %gall request
1080− ?> ?=([@ ~] t.t.q.q.bon)
1081− =+ app=`term`(need ((sand %tas) i.t.t.q.q.bon))
1082− =+ ^= pax
1083− :+ (scot %p p.p.bon)
1084− (scot %p q.p.bon)
1085− q.q.bon
1086− :_ fox [hen %pass pax %g %rote p.bon app r.bon]~
1087− %gh :: %gall response
1088− ?> ?=([@ ~] t.t.q.q.bon)
1089− =+ app=`term`(need ((sand %tas) i.t.t.q.q.bon))
1090− =+ ^= pax
1091− :+ (scot %p p.p.bon)
1092− (scot %p q.p.bon)
1093− q.q.bon
1094− :_ fox [hen %pass pax %g %roth p.bon app r.bon]~
1095− ==
1096−```
1097−
1098−We're dispatching messages based on the prefix of their path. Since only
1099−`%gall` apps use end-to-end acknowledgments at the moment, every path
1100−must have at least two elements, and the first one must be `%q`. Beyond
1101−that, we handle the `/q/ge` and `/q/gh` cases for gall requests and
1102−responses, respectively.
1103−
1104−In both cases, we require the next term in the path to be the name of
1105−the intended recipient `%gall` app. Thus, a message to `/q/ge/talk` for
1106−example, will send a message to the talk app.
1107−
1108−We then send a message to the app itself. The message is either a
1109−`%rote` or a `%roth` for a request and a response, respectively. The
1110−content is the `rook` or `roon` that was sent (stored in `r.bon`), but
1111−we don't actually handle that at all here. That's completely a
1112−`%gall`-level thing. We're just the messenger.
1113−
1114−Notice the path we send this over. We encode the sender, the receiver,
1115−and the path over which it was sent. This fully specifies the `race` so
1116−that when the app gives us the acknowledgment we know where to send it.
1117−
1118−We now have another interlude. We have entrusted our precious data, so
1119−carefully guarded and guided from the app on that far-away ship, to our
1120−local app. It has the ability to do whatever it pleases with it. It may
1121−take a significant amount of time to process. When the message has been
1122−handled by this app, though, it must produce an acknowledgment. Our
1123−final task is to deliver this acknowledgment to the sending app.
1124−
1125−We should describe here what exactly these oft-mentioned acknowledgments
1126−actually consist of. There are two kinds of acknowledgments: positive
1127−and negative. A positive acknowledgment contains no data other than its
1128−existence. A negative acknowledgment may optionally include a reason for
1129−said negativity. Formally, a negative acknowledgment is an `ares`, which
1130−is a unit pair of a term and a list of tanks. If this is null, this is
1131−simply a failure with no associated information. If the pair exists, the
1132−term is a short error code that is usually both human and computer
1133−readable. For example, if you try to send a message to a valid `%gall`
1134−app that doesn't have any `++poke` to handle it, then `%gall` will give
1135−a negative acknowledgment with error term `%poke-find-fail`. The list of
1136−tanks is a human-readable description of the error. This often contains
1137−a stack trace. At any rate, all this information is returned to the
1138−sending app on the other end of the wire.
1139−
1140−After this brief interlude, our story resumes in `++knap`, where we
1141−receive responses. In particular, a `%mean` indicates a negative
1142−acknowledgment while a `%nice` indicates a positive acknowledgment.
1143−
1144−```hoon
1145− ?(%mean %nice)
1146− ?> ?=([@ @ @ *] tea)
1147− =+ soq=[(slav %p i.tea) (slav %p i.t.tea)]
1148− =+ pax=t.t.tea
1149− =+ ^= fuy
1150− =< zork =< zank
1151− %^ ~(rack am [now fox]) soq pax
1152− ?-(+<.sih %mean `p.+.sih, %nice ~)
1153− => %_(. fox q.fuy)
1154− =| out=(list move)
1155− |- ^- [p=(list move) q=_+>.^$]
1156− ?~ p.fuy
1157− [(flop out) +>.^$]
1158− =^ toe fox (clop now hen i.p.fuy)
1159− $(p.fuy t.p.fuy, out (weld (flop toe) out))
1160−```
1161−
1162−Recall the format of the path we sent the message on, and you'll
1163−understand why `soq` and `pax` are the sender/receiver pair and path on
1164−which the message was sent. The rest of this is structured much like
1165−`++knob`, so we call `++rack:am` and send the resulting boons to
1166−`++clop`. Business as usual.
1167−
1168−```hoon
1169− ++ rack :: rack:am
1170− |= [soq=sock cha=path cop=coop] :: e2e ack
1171− =+ oh=(ho:(um p.soq) q.soq)
1172− =. oh (cook:oh cop cha ~)
1173− (cans:oh cha)
1174−```
1175−
1176−First, we set up `++um` and `++ho`, as we've done twice before, for our
1177−domestic and foreign servers, respectively. The other two things are
1178−new, though. Well, `++cook` is not actually new, but we delayed the
1179−explanation saying only that it sends an acknowledgment. The time has
1180−come.
1181−
1182−```hoon
1183− ++ cook :: cook:ho:um:am
1184− |= [cop=coop cha=path ram=(unit ,[ryn=lane dam=flap])]
1185− ^+ +> :: acknowledgment
1186− =+ rum=(need (~(get by raz.bah) cha))
1187− =+ lat=(~(get by mis.rum) did.rum)
1188− ?: &(?=(~ lat) ?=(~ ram)) ~&(%ack-late-or-redundant +>.$)
1189− =+ ^- [ryn=lane dam=flap]
1190− ?^ ram [ryn.u.ram dam.u.ram]
1191− ?< ?=(~ lat)
1192− [q r]:u.lat
1193− =. raz.bah
1194− ?^ ram raz.bah
1195− %+ ~(put by raz.bah) cha
1196− rum(dod &, bum ?~(cop bum.rum (~(put by bum.rum) did.rum u.cop)))
1197− =^ roc diz (zuul:diz now [%buck cop dam ~s0])
1198− (busk(diz (wast:diz ryn)) xong:diz roc)
1199−```
1200−
1201−If we are acknowledging a message that we have already acked, the `ram`
1202−will contain the new lane and flap to send the duplicate ack to. This
1203−happens if we call `++cook` in `++deer`, but it doesn't happen from
1204−`++rack`. If there is no message waiting to be acknowledged and we're
1205−not given an explicit lane and flap (that is, we're not sending a
1206−duplicate ack), then the app must have sent us multiple acknowledgments.
1207−We do the only sensible thing we can do and drop all acknowledgments
1208−after the first, printing a message. This is, in fact, an error, so it
1209−could be argued that we ought to crash. Whatever you do, don't depend on
1210−this not crashing.
1211−
1212−First, we grab the race specified by the given path, and we get the most
1213−recent in-order message, which must be the one which is being
1214−acknowledged.
1215−
1216−Then, we decide which lane/flap to respond on/to. Basically, in the
1217−usual case we respond on the lane through which the initial message was
1218−sent, which is stored along with the other packet information in
1219−`mis.rum`, since it has to be remembered across calls to ames. However,
1220−if we receive a duplicate message, then we must respond to the new
1221−message. It's quite possible the reason the other acknowledgment didn't
1222−get returned was that the lane between the ships was broken.
1223−
1224−At any rate, we update the race by saying that we've finished processing
1225−this packet (unless we're sending a duplicate ack) and, if we're sending
1226−a negative acknowledgment, putting the negative ack into `bum.rum` so
1227−that we can resend it if necessary.
1228−
1229−We encode our new message, updating the packet pump, with `++zuul`, as
1230−before, and we send it off with `++busk`, routed via `++wast` to one of
1231−the ships in `++xong`. Of course, in practice, we don't even look at the
1232−ships in `++xong` because we already have a lane directly to our
1233−neighbor (the one over which they sent their message to us).
1234−
1235−We glossed over the actual message we're sending back. We're sending a
1236−`%buck` meal, which is an acknowledgment. The `cop` specifies whether
1237−this is a positive or a negative ack, `dam` specifies the message we're
1238−acknowledging, and the `~s0` is a placeholder for the processing time
1239−required. This time is neither calculated (though it is hopefully
1240−obvious how to do so) nor used at present, but this information may be
1241−used in the future for improved congestion control. Since the round-trip
1242−time for an end-to-end acknowledged packet includes the processing time
1243−on the other end, most common congestion control algorithms will stumble
1244−when some messages take much longer to process than others. As noted,
1245−though, this is simply an opportunity for improvement -- our congestion
1246−control algorithms are relatively naive at the moment.
1247−
1248−Recall that `++busk` calls `++wist` to put the actual `%ouzo` boon in
1249−the queue, which gets handled by `++clop` to actually send the message.
1250−This is the same pipeline as sending any other message, so we'll refer
1251−you to the explanation above if you've forgotten it.
1252−
1253−The last thing we need to do on this ship is move on to the next packet
1254−in the queue if there is one. If you recall, in `++rack` after the call
1255−to `++cook` there was a call to `++cans:ho:um:am`.
1256−
1257−```hoon
1258− ++ cans :: cans:ho:um:am
1259− |= cha=path
1260− =+ rum=(need (~(get by raz.bah) cha))
1261− =. rum
1262− %= rum
1263− did +(did.rum)
1264− mis (~(del by mis.rum) did.rum)
1265− ==
1266− (coat cha rum)
1267−```
1268−
1269−This is very simple. We increment the number of packets that we've
1270−acknowledged on this race and we delete the packet that we just
1271−acknowledged from the set of misordered packets.
1272−
1273−Then, we call `++coat` again to process the next packet if we've already
1274−received it. And that's it for this.
1275−
1276−The acknowledgment now travels the same path that its forebearer, the
1277−original message, once tread, but this time not into the great unknown.
1278−The weary traveler is seeking out its familial roots, finding the app
1279−from whom sprung forth the original message way back in paragraph three.
1280−When it arrives at the network adapter of its ancestors, the adapter
1281−tells unix, unix tells libuv, libuv tells vere, and vere sends a `%hear`
1282−task to ames. Once more into the kernel.
1283−
1284−The `%hear` task is handled in `++knob` as before, leading to `++gnaw`,
1285−going over to `++chew`, `++apse`, `++chow`, and eventualy to `++dine`.
1286−We've seen most of the cases in `++dine`, but we haven't yet looked at
1287−the handling of this `%buck` meal.
1288−
1289−```hoon
1290− %buck
1291− =. +> ?.(=(%full aut) +> cock) :: finish key exch
1292− +>(..la (tock p.fud q.fud r.fud))
1293−```
1294−
1295−We send a packet level acknowledgment if we're finishing a key exchange,
1296−else we call `++tock` to process the acknowledgment.
1297−
1298−This will get a little involved, so if you don't much care about how
1299−exactly an acknowledgment happens, just know that the result gets gifted
1300−as a `%woot` card back to the app who sent it. For those brave souls who
1301−wish to see this thing through to the end, it's once more into the
1302−breach.
1303−
1304−```hoon
1305− ++ tock :: tock:ho:um:am
1306− |= [cop=coop fap=flap cot=@dr] :: e2e ack by hash
1307− ^+ +>
1308− =^ yoh puz (bick:puz now fap)
1309− =. +>.$
1310− ?~ p.yoh +>.$
1311− =^ hud +>.$
1312− (done p.u.p.yoh q.u.p.yoh)
1313− ?~ hud +>.$
1314− %= +>.$
1315− bin
1316− :_ bin
1317− `boon`[%cake [our her] [[p:sen:gus clon:diz] u.p.yoh] cop u.hud]
1318− ==
1319− (busk xong:diz q.yoh)
1320−```
1321−
1322−We're going to work through this one a little backwards since it's
1323−mostly fairly simple except the call to `++bick:pu`. In fact, we'll just
1324−skip `++bick` for the moment and finish the rest.
1325−
1326−If `++bick` succesfully acks the message, then we call `++done`.
1327−
1328−```hoon
1329− ++ done :: done:ho:um:am
1330− |= [cha=path num=@ud] :: complete outgoing
1331− ^- [(unit duct) _+>]
1332− =+ rol=(need (~(get by ryl.bah) cha))
1333− =+ rix=(~(get by san.rol) num)
1334− ?~ rix [~ +>.$]
1335− :- rix
1336− %_ +>.$
1337− ryl.bah
1338− (~(put by ryl.bah) cha rol(san (~(del by san.rol) num)))
1339− ==
1340−```
1341−
1342−This very simply gets the rill (the outgoing counterpart to a race, if
1343−you recall), pulls out of the map of outstanding messages the duct over
1344−which the original message was sent, and produces this duct while
1345−deleting that entry from the map of outstanding messages.
1346−
1347−Going back to `++tock`, we now have the duct we need to return the
1348−result over. We do the very sensible thing and put a `%cake` boon in the
1349−queue to be processed later by `++clop`.
1350−
1351−In `q.yoh` we have a list of messages that may need to be sent, which we
1352−pass to `++busk` to send, as usual. When an acknowledgment arrives, that
1353−may trigger other messages immediately. This often happens when sending
1354−more messages than the width of the logical window since for congestion
1355−control reasons another message cannot be sent until some of the earlier
1356−ones have been acknowledged.
1357−
1358−We'll look at the processing of the `%cake` boon in `++clop` before we
1359−get back to talking about `++bick`.
1360−
1361−```hoon
1362− %cake
1363− :_ fox
1364− :~ [s.bon %give %woot q.p.bon r.bon]
1365− ==
1366−```
1367−
1368−We very simply give, along the duct we found above, a `%woot` card with
1369−the ship who sent us the ack and the ack itself. This allows the
1370−application to decide what to do about the result. In case of a failure,
1371−we usually either resend the message or display it to the user.
1372−Sometimes, we recognize the error term and handle it internally. In any
1373−case, the decision of how to handle the acknowledgment is entirely up to
1374−the application. Our job is done.
1375−
1376−Well, except that we skipped `++bick:pu`. Let's go back to that.
1377−
1378−```hoon
1379− ++ bick :: bick:pu
1380− |= [now=@da fap=flap] :: ack by hash
1381− ^- [[p=(unit soup) q=(list rock)] _+>]
1382− =+ sun=(~(get by diq) fap)
1383− ?~ sun
1384− [[~ ~] +>.$]
1385− =. diq (~(del by diq) fap)
1386− =^ gub +>.$ (bock now u.sun)
1387− =^ yop +>.$ (harv now)
1388− [[gub yop] +>.$]
1389−```
1390−
1391−If you recall, in `++whap:pu` we created the packet pump's
1392−representation of the message, which included putting the message into
1393−`diq`, which maps from packet hashes to packet sequence numbers. Thus,
1394−`u.sun` is the sequence number of this particular message.
1395−
1396−We delete this message from `diq` since we have now received an ack for
1397−it. We call `++bock` to perform the ack by sequence number. We call
1398−`++harv` to harvest the packet queue, sending any messages that are now
1399−able to be sent.
1400−
1401−In `++bock`, there are three arms we haven't seen before: `++bine`,
1402−`+wept`, and `++beet`. We'll describe each of these before we get to
1403−`++bock`. `++bine` looks scariest.
1404−
1405−```hoon
1406− ++ bine :: bine:pu
1407− |= [now=@da num=@ud] :: apply ack
1408− ^- [(unit soup) _+>]
1409− ?~ puq !!
1410− ?. =(num p.n.puq)
1411− ?: (gth num p.n.puq)
1412− =+ lef=$(puq l.puq)
1413− [-.lef +.lef(puq [n.puq puq.lef r.puq])]
1414− =+ rig=$(puq r.puq)
1415− [-.rig +.rig(puq [n.puq l.puq puq.rig])]
1416− =: rtt ?. &(liv.q.n.puq =(1 nux.q.n.puq)) rtt
1417− =+ gap=(sub now lys.q.n.puq)
1418− :: ~& [%bock-trip num (div gap (div ~s1 1.000))]
1419− (div (add (mul 2 rtt) gap) 3)
1420− nif (sub nif !liv.q.n.puq)
1421− ==
1422− =+ lez=(dec (need (~(get by pyz) gom.q.n.puq)))
1423− =^ gub pyz
1424− ?: =(0 lez)
1425− [[~ gom.q.n.puq] (~(del by pyz) gom.q.n.puq)]
1426− [~ (~(put by pyz) gom.q.n.puq lez)]
1427− :- gub
1428− +>.$(puq ~(nap to puq))
1429−```
1430−
1431−The first few lines are simply looking through the packet queue until we
1432−find the correct packet to ack. This is basic queue manipulation that
1433−operates directly on the treap structure of the queue. If you understand
1434−treap queues, the logic is easy to follow. Otherwise, just trust us that
1435−by the time we get to the `=:`, the packet with sequence number `num` is
1436−on the top of the packet queue (that is, at `n.puq`).
1437−
1438−We first update the round-trip time. If the packet is either not alive
1439−or had to be transmitted more than once, then we don't have any reliable
1440−way of calculating the round-trip time since we're unsure of exactly
1441−which transmission was acknowledged. Otherwise, the round-trip time is
1442−the difference between now and when the packet was last sent. We set
1443−`rtt` by a little weighted average where the previous smoothed RTT is
1444−weighted twice as much as the RTT of the current packet. Thus,
1445−`(2*rtt+gap)/3`. This gives us a nice smooth RTT that is somewhat
1446−resilient to outlier data while still being responsive to our
1447−ever-changing world.
1448−
1449−If the packet wasn't already dead, then we decrement the number of live
1450−packets, which may allow more packets to be sent.
1451−
1452−We decrement the number of unacknowledged packets in our `pyz` for this
1453−particular message. If you recall, this was set in `++whap` to the
1454−number of packets required to send a message.
1455−
1456−If that was the last packet in the messge that needed to be acked, then
1457−we delete the messgae reference from `pyz` and produce the id of the
1458−message. Otherwise, we simply update `pyz` with the new number of
1459−unacked messages. In either case, we remove the packet from the packet
1460−queue.
1461−
1462−```hoon
1463− ++ wept :: wept:pu
1464− |= [fip=@ud lap=@ud] :: fip thru lap-1
1465− =< abet =< apse
1466− |%
1467− ++ abet +>.$
1468− ++ apse
1469− ^+ .
1470− ?~ puq .
1471− ?: (lth p.n.puq fip) ?~(l.puq . left)
1472− ?: (gte p.n.puq lap) ?~(r.puq . rigt)
1473− => rigt =< left
1474− ?> ?=(^ puq)
1475− ?.(liv.q.n.puq . .(nif (dec nif), liv.q.n.puq |))
1476− ::
1477− ++ left
1478− ?> ?=(^ puq)
1479− ^+(. =+(lef=apse(puq l.puq) lef(puq [n.puq puq.lef r.puq])))
1480− ++ rigt
1481− ?> ?=(^ puq)
1482− ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
1483− --
1484−```
1485−
1486−The algorithm is a simple case of traversing the packet queue.
1487−Essentialy, we mark as dead all packets in the queue between `fip` and
1488−`(dec lap)`. We also update `nif`, the number of live packets. Lest you
1489−mourn too much the passing of these packets, know that they shall soon
1490−rise again. Recall that in `++bick` after the call to `++bock` we call
1491−`++harv`. This will resend the packets that have just been labeled dead.
1492−
1493−```hoon
1494− ++ beet :: beet:pu
1495− ^+ . :: advance unacked
1496− =- +(nep ?~(foh nus u.foh))
1497− ^= foh
1498− |- ^- (unit ,@ud)
1499− ?~ puq ~
1500− ?: (lte p.n.puq nep) $(puq l.puq)
1501− =+ rig=$(puq r.puq)
1502− ?^(rig rig [~ p.n.puq])
1503−```
1504−
1505−Here we search for the next expected packet number. Basically, we search
1506−the queue for the leftmost packet whose number is greater than the
1507−current `nep`. If we don't find any such packet, we just use the total
1508−number of packets sent.
1509−
1510−We can now dive into `++bock`, our last arm.
1511−
1512−```hoon
1513− ++ bock :: bock:pu
1514− |= [now=@da num=@ud] :: ack by sequence
1515− ^- [(unit soup) _+>]
1516− =^ gym +> (bine now num)
1517− :- gym
1518− ?: (gth num nep)
1519− =+ cam=(max 2 (div caw 2))
1520− :: ~& [%bock-hole num nep cam]
1521− beet:(wept(nep num, cag cam, caw cam) nep num)
1522− =. caw ?: (lth caw cag) +(caw)
1523− (add caw !=(0 (mod (mug now) caw)))
1524− ?: =(num nep)
1525− :: ~& [%bock-fine num nif caw cag]
1526− beet
1527− :: ~& [%bock-fill num nif caw cag]
1528− +>.$
1529−```
1530−
1531−First, we call `++bine` to apply the ack to the packet pump information.
1532−We produce `gym`, which, if it exists, is the id of the packet that was
1533−acked. If we received an ack for a packet later than the one we
1534−expected, then we halve the logical packet window and kill all the
1535−earlier packets so that they may be resent.
1536−
1537−Otherwise, we possibly increase the congestion window. If the window is
1538−less than the congestion threshold, then we increment the size of the
1539−window. Otherwise, we only increment one out of every `caw` times.
1540−
1541−If we received an ack for the packet we expected, then we simply advance
1542−`nep` with `++beet`. If we received an ack for a packet earlier than we
1543−expected, we do nothing.
1544−
1545−It may be hard to believe, but we are, in fact, done. The message has
1546−been sent, received, acknowledged, and the acknowledgment has been
1547−returned to the original sender. We hope it's clear that, while the
1548−process has been somewhat involved, the algorithms are not all that
1549−complicated. If you've read this far, you know `%ames`. The only other
1550−code involves initialization, timeouts, and the like.
1551−
1552−Below, we give detailed reference documentation for the data models
1553−involved.
1554−
1555−## Data Models
1556−
1557−### `++sufi:ames`, domestic host
1558−
1559−```hoon
1560− ++ sufi :: domestic host
1561− $: hoy=(list ship) :: hierarchy
1562− val=wund :: private keys
1563− law=will :: server will
1564− seh=(map hand ,[p=ship q=@da]) :: key cache
1565− hoc=(map ship dore) :: neighborhood
1566− == ::
1567−```
1568−
1569−This is the security state of a domestic server.
1570−
1571−`hoy` is a list of the ships directly above us in the hierarchy of
1572−ships. For example, for `~hoclur-bicrel`, this would be `~tasruc` and
1573−`~tug`. See `++sein:title`.
1574−
1575−`val` is a list of our private keys.
1576−
1577−`law` is our certificate, which is a list of the XXX
1578−
1579−`seh`
1580−
1581−`hoc` is a map of ships to `++dore:ames`. The stores all the security
1582−information about foreign ships. The keys to this map are the neighbors
1583−(ships we have been in contact with) of this domestic server.
1584−
1585−### `++wund:ames`, private keys
1586−
1587−```hoon
1588− ++ wund (list ,[p=life q=ring r=acru]) :: mace in action
1589−```
1590−
1591−This is a list of our own private keys, indexed by life. The key itself
1592−is the `++ring`, and the `++acru:ames` is the encryption engine. We generate
1593−the `++acru:ames` from the private key by calling `++weur`. Thus, we can at
1594−any time regenerate our `++wund:ames` from a `++mace:ames`. The current crypto is
1595−at the head of the list and can be accessed with `++sen:as:go`.
1596−
1597−### `++ring`, private key
1598−
1599−```hoon
1600− ++ ring ,@ :: private key
1601−```
1602−
1603−This is a private key. The first byte is reserved to identify the type
1604−of cryptography. Lower-case means public key, upper-case means public
1605−key, and the letter identifies which `++acru:ames` to use.
1606−
1607−### `++pass`, public key
1608−
1609−```hoon
1610− ++ pass ,@ :: public key
1611−```
1612−
1613−This is a public key. The first byte is reserved to identify the type of
1614−cryptography. Lower-case means public key, upper-case means public key,
1615−and the letter identifies which `++acru:ames` to use.
1616−
1617−### `++mace:ames`, private secrets
1618−
1619−```hoon
1620− ++ mace (list ,[p=life q=ring]) :: private secrets
1621−```
1622−
1623−This is a list of the our private keys, indexed by life. From this we
1624−can generate a `++wund:ames` for actual use.
1625−
1626−### `++skin:ames`, encoding stem
1627−
1628−```hoon
1629− ++ skin ?(%none %open %fast %full) :: encoding stem
1630−```
1631−
1632−This defines the type of encryption used for each message. `%none`
1633−refers to messages sent in the clear, `%open` refers to signed messages,
1634−`%full` refers to sealed messages, and `%fast` refers to symmetrically
1635−encrypted messages. See `++acru:ames` for details.
1636−
1637−### `++acru:ames`, asymmetric cryptosuite
1638−
1639−```hoon
1640− ++ acru :: asym cryptosuite
1641− $_ ^? |% :: opaque object
1642− ++ as ^? :: asym ops
1643− |% ++ seal |=([a=pass b=@ c=@] _@) :: encrypt to a
1644− ++ sign |=([a=@ b=@] _@) :: certify as us
1645− ++ sure |=([a=@ b=@] *(unit ,@)) :: authenticate from us
1646− ++ tear |= [a=pass b=@] :: accept from a
1647− *(unit ,[p=@ q=@]) ::
1648− -- ::
1649− ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1650− ++ dy |+([a=@ b=@] _@) :: symmetric de, hard
1651− ++ en |+([a=@ b=@] _@) :: symmetric en
1652− ++ ex ^? :: export
1653− |% ++ fig _@uvH :: fingerprint
1654− ++ pac _@uvG :: default passcode
1655− ++ pub *pass :: public key
1656− ++ sec *ring :: private key
1657− --
1658− ++ nu ^? :: reconstructors
1659− |% ++ pit |=([a=@ b=@] ^?(..nu)) :: from [width seed]
1660− ++ nol |=(a=@ ^?(..nu)) :: from naked ring
1661− ++ com |=(a=@ ^?(..nu)) :: from naked pass
1662− --
1663− --
1664−```
1665−
1666−This is an opaque interface for a general asymmetric cryptosuite. Any
1667−form of asymmetric cryptography can be dropped in to be used instead of
1668−the default. Right now, there is one cryptosuite: `++crub:crypto:crypto`,
1669−which is elliptic-curve cryptography.
1670−
1671−#### `++as:acru:ames`, asymmetric operations
1672−
1673−```hoon
1674− ++ as ^? :: asym ops
1675− |% ++ seal |=([a=pass b=@ c=@] _@) :: encrypt to a
1676− ++ sign |=([a=@ b=@] _@) :: certify as us
1677− ++ sure |=([a=@ b=@] *(unit ,@)) :: authenticate from us
1678− ++ tear |= [a=pass b=@] :: accept from a
1679− *(unit ,[p=@ q=@]) ::
1680− -- ::
1681−```
1682−
1683−This is the core that defines the standard asymmetric cryptography
1684−operations.
1685−
1686−`++seal:as:acru:ames` allows us to send a message encrypted with someone's
1687−public key so that only they may read it. If Alice seals a message with
1688−Bob's public key, then she can be sure that Bob is the only one who can
1689−read it. This is associated with the `++skin:ames` `%full`.
1690−
1691−`++sign:as:acru:ames` allows us to sign a message with our private key so
1692−that others can verify that we sent the message. If Alice signs a
1693−message with her private key, then Bob can verify with her public key
1694−that it was indeed Alice who sent it. This is associated with the
1695−`++skin:ames` `%open`.
1696−
1697−`++sure:as:acru:ames` is the dual to `++sign:as:acru:ames`. It allows us to verify
1698−that a message we have received is indeed from the claimed sender. If
1699−Alice sends a message with her private key, then Bob can use this arm to
1700−verify that it was indeed Alice who sent it. This is associated with the
1701−`++skin:ames` `%open`.
1702−
1703−`++tear:as:acru:ames` is the dual to `++seal:as:acru:ames`. It allows us to read a
1704−message that we can be sure is only read by us. If Alice seals a message
1705−with Bob's public key, then Bob can use this arm to read it. This is
1706−associated with the `++skin:ames` `%full`.
1707−
1708−#### `++de:acru:ames`, `++dy:acru:ames`, and `++en:acru:ames`, symmetric encryption/decryption
1709−
1710−```hoon
1711− ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1712− ++ dy |+([a=@ b=@] _@) :: symmetric de, hard
1713− ++ en |+([a=@ b=@] _@) :: symmetric en
1714−```
1715−
1716−Symmetric encryption is associated with the `++skin:ames` `%fast`.
1717−
1718−`++de:acru:ames` decrypts a message with a symmetric key, returning `~` on
1719−failure and `[~ u=data]` on success.
1720−
1721−`++dy:acru:ames` decrypts a message with a symmetric key, crashing on
1722−failure. This should almost always be defined as, and should always be
1723−semantically equivalent to, `(need (de a b))`.
1724−
1725−`++en:acru:ames` encrypts a message with a symmetric key.
1726−
1727−#### `++ex:acru:ames`, exporting data
1728−
1729−```hoon
1730− ++ ex ^? :: export
1731− |% ++ fig _@uvH :: fingerprint
1732− ++ pac _@uvG :: default passcode
1733− ++ pub *pass :: public key
1734− ++ sec *ring :: private key
1735− --
1736−```
1737−
1738−`++fig:ex:acru:ames` is our fingerprint, usually a hash of our public key.
1739−This is used, for example, in `++zeno`, where every carrier owner's
1740−fingerprint is stored so that we can ensure that carriers are indeed
1741−owned by their owners
1742−
1743−`++pac:ex:acru:ames` is our default passcode, which is unused at present.
1744−
1745−`++pub:ex:acru:ames` is the `++pass` form of our public key.
1746−
1747−`++sec:ex:acru:ames` is the `++ring` form of our private key.
1748−
1749−#### `++nu:acru:ames`, reconstructors
1750−
1751−```hoon
1752− ++ nu ^? :: reconstructors
1753− |% ++ pit |=([a=@ b=@] ^?(..nu)) :: from [width seed]
1754− ++ nol |=(a=@ ^?(..nu)) :: from naked ring
1755− ++ com |=(a=@ ^?(..nu)) :: from naked pass
1756− --
1757−```
1758−
1759−These arms allow us to reconstruct a `++acru:ames` from basic data.
1760−
1761−`++pit:nu:acru:ames` constructs a `++acru:ames` from the width of our intended key
1762−and seed entropy. This is usually used in the initial construction of
1763−the `++acru:ames`.
1764−
1765−`++nol:nu:acru:ames` constructs a `++acru:ames` from a "naked ring", meaning a
1766−`++ring` without the initial byte identifying the type of crypto.
1767−
1768−`++com:nu:acru:ames` constructs a `++acru:ames` from a "naked pass", meaning a
1769−`++ring` without the initial byte identifying the type of crypto.
1770−
1771−### `++will`, certificate
1772−
1773−```hoon
1774− ++ will (list deed) :: certificate
1775−```
1776−
1777−This is a list of deeds associated with the current ship. There should
1778−be an item in this list for every ship from this point up in the
1779−hierarchy times the number of lives that each ship has had. For example,
1780−\~hoclur-bicrel may have a will with three items: one for itself, one
1781−for \~tasruc (who issued \~hoclur-bicrel's deed) and one for \~tug (who
1782−issued \~tasruc's deed).
1783−
1784−### `++deed`, identity
1785−
1786−```hoon
1787− ++ deed ,[p=@ q=step r=?] :: sig, stage, fake?
1788−```
1789−
1790−`p` is the signature of a particular deed, which is a signed copy of
1791−`q`.
1792−
1793−`q` is the stage in the identity.
1794−
1795−`r` is true if we're working on a fake network, where we don't check
1796−that the carrier fingerprints are correct. This allows us to create fake
1797−networks for development without interfering with the real network.
1798−
1799−### `++step:ames`, identity stage
1800−
1801−```hoon
1802− ++ step ,[p=bray q=gens r=pass] :: identity stage
1803−```
1804−
1805−This is a single stage in our identity. Thus, this is specific to a
1806−single life in a single ship. Everything in here may change between
1807−lives.
1808−
1809−`p`
1810−
1811−`q`
1812−
1813−`r` is the public key for this stage in the identity.
1814−
1815−### `++bray:ames`
1816−
1817−```hoon
1818− ++ bray ,[p=life q=(unit life) r=ship s=@da] :: our parent us now
1819−```
1820−
1821−XXX
1822−
1823−### `++gens:ames`, general identity
1824−
1825−```hoon
1826− ++ gens ,[p=lang q=gcos] :: general identity
1827−```
1828−
1829−`p` is the IETF language code for the preferred language of this
1830−identity. This is unused at the moment, but in the future text should be
1831−localized based on this.
1832−
1833−`q` is the description of the ship.
1834−
1835−### `++gcos:ames`, identity description
1836−
1837−```hoon
1838− ++ gcos :: id description
1839− $% [%czar ~] :: 8-bit ship
1840− [%duke p=what] :: 32-bit ship
1841− [%earl p=@t] :: 64-bit ship
1842− [%king p=@t] :: 16-bit ship
1843− [%pawn p=(unit ,@t)] :: 128-bit ship
1844− == ::
1845−```
1846−
1847−This is the description of the identity of a ship. Most types of
1848−identity have a `@t` field, which is their human-readable name. The
1849−identity of a `%duke` is more involved.
1850−
1851−A `%czar`, a carrier, is a ship with an 8-bit address. Thus, there are
1852−only 256 carriers. These are at the top of the namespace hierarchy, and
1853−the fingerprint of each carrier is stored in `++zeno`. These are the
1854−"senators" of Urbit.
1855−
1856−A `%king`, a cruiser, is a ship with a 16-bit address. Thus, there are
1857−65,536 cruisers. Each carrier may issue 256 cruisers. These are the
1858−infrastructure of Urbit.
1859−
1860−A `%duke`, a destroyer, is a ship with a 32-bit address. Thus, there are
1861−4,294,967,296 destroyers. Each cruiser may issue 65,536 cruisers. These
1862−are the individuals of Urbit.
1863−
1864−A `%earl`, a yacht, is a ship with a 64-bit address. Thus, there are
1865−18,446,744,073,709,551,616 yachts. Each destroyer may issue
1866−4,294,967,296 yachts. These are the devices of Urbit.
1867−
1868−A `%pawn`, a submarine, is a ship with a 128-bit address. Thus, there
1869−are a lot of submarines. The chance of random name collision is
1870−negligible, so submarines are not issued by any ship. They must simply
1871−assert their presence, and they are all considered children of \~zod.
1872−This is the underworld of Urbit, where anonymity reigns supreme.
1873−
1874−### `++what:ames`, logical destroyer identity
1875−
1876−```hoon
1877− ++ what :: logical identity
1878− $% [%anon ~] :: anonymous
1879− [%lady p=whom] :: female person ()
1880− [%lord p=whom] :: male person []
1881− [%punk p=sect q=@t] :: opaque handle ""
1882− == ::
1883−```
1884−
1885−This is the logical identity of a destroyer.
1886−
1887−A `%anon` is a completely anonymous destroyer. The difference between
1888−this and a submarine is that a submarine is ephemeral while a `%anon`
1889−destroyer is not. Thus, we may not know who \~hoclur-bicrel is, but we
1890−do know that it's always the same person.
1891−
1892−A `%lady` is a female person. The name used here should be a real name.
1893−
1894−A `%lord` is a male person. The name used here should be a real name.
1895−
1896−A `%punk` is a person who is identified only by a handle.
1897−
1898−### `++whom:ames`, real person
1899−
1900−```hoon
1901− ++ whom ,[p=@ud q=govt r=sect s=name] :: year/govt/id
1902−```
1903−
1904−Ths is the information associated with a real person. It is mostly
1905−information that could be observed with the briefest of interactions.
1906−
1907−`p` is the birth year.
1908−
1909−`q` is the location of a user, usually of the form "country/zip".
1910−
1911−`r` is the sect of the user.
1912−
1913−`s` is the real name of the person.
1914−
1915−### `++govt:ames`
1916−
1917−```hoon
1918− ++ govt path :: country/postcode
1919−```
1920−
1921−This is the location of the user, usually of the form "country/zip".
1922−
1923−### `++sect:ames`
1924−
1925−```hoon
1926− ++ sect ?(%black %blue %red %orange %white) :: banner
1927−```
1928−
1929−XXX
1930−
1931−### `++name:ames`
1932−
1933−```hoon
1934− ++ name ,[p=@t q=(unit ,@t) r=(unit ,@t) s=@t] :: first mid/nick last
1935−```
1936−
1937−This is the given name, possible middle name/initial, possible nickname,
1938−and surname of a user.
25+vane or app, is defined in Ames.
26+
27+## Technical Overview
28+Ames extends [Arvo's](@/docs/tutorials/arvo/arvo.md) `%pass`/`%give` `move` semantics across the network.
29+
30+Ames receives packets as Arvo events and emits packets as Arvo
31+effects. The runtime is responsible for transferring the bytes in
32+an Ames packet across a physical network to another ship.
33+
34+The runtime tells Ames which physical address a packet came from,
35+represented as an opaque atom. Ames can emit a packet effect to
36+one of those opaque atoms or to the Urbit address of a galaxy
37+(root node), which the runtime is responsible for translating to a
38+physical address. One runtime implementation sends UDP packets
39+using IPv4 addresses for ships and DNS lookups for galaxies, but
40+other implementations may overlay over other kinds of networks.
41+
42+A local vane can pass Ames a `%plea` request message. Ames
43+transmits the message over the wire to the peer ship's Ames, which
44+passes the message to the destination vane.
45+
46+Once the peer has processed the `%plea` message, it sends a
47+message-acknowledgment packet over the wire back to the local
48+Ames. This "ack" can either be positive to indicate the request was
49+processed, or negative to indicate the request failed, in which
50+case it's called a "nack". (Don't confuse Ames nacks with TCP
51+nacks, which are a different concept).
52+
53+When the local Ames receives either a positive message-ack or a
54+combination of a nack and "naxplanation" (explained in more detail
55+below), it gives an `%done` `move` to the local vane that had
56+requested the original `%plea` message be sent.
57+
58+A local vane can give Ames zero or more `%boon` response messages in
59+response to a `%plea`, on the same duct that Ames used to pass the
60+`%plea` to the vane. Ames transmits a `%boon` over the wire to the
61+peer's Ames, which gives it to the destination vane on the same
62+duct the vane had used to pass the original `%plea` to Ames.
63+
64+`%boon` messages are acked automatically by the receiver Ames. They
65+cannot be nacked, and Ames only uses the ack internally, without
66+notifying the client vane that gave Ames the `%boon`.
67+
68+If the Arvo event that completed receipt of a `%boon` message
69+crashes, Ames instead sends the client vane a `%lost` message
70+indicating the `%boon` was missed.
71+
72+`%plea` messages can be nacked, in which case the peer will send
73+both a message-nack packet and a naxplanation message, which is
74+sent in a way that does not interfere with normal operation. The
75+naxplanation is sent as a full Ames message, instead of just a
76+packet, because the contained error information can be arbitrarily
77+large. A naxplanation can only give rise to a positive ack --
78+never ack an ack, and never nack a naxplanation.
79+
80+Ames guarantees a total ordering of messages within a "flow",
81+identified in other vanes by a duct and over the wire by a "bone":
82+an opaque number. Each flow has a FIFO (first-in-first-out) queue of `%plea` requests
83+from the requesting ship to the responding ship and a FIFO queue
84+of `%boon`'s in the other direction.
85+
86+Message order across flows is not specified and may vary based on
87+network conditions.
88+
89+Ames guarantees that a message will only be delivered once to the
90+destination vane.
91+
92+Ames encrypts every message using symmetric-key encryption by
93+performing an elliptic curve Diffie-Hellman using our private key
94+and the public key of the peer. For ships in the Jael PKI
95+(public-key infrastructure), Ames looks up the peer's public key
96+from Jael. Comets (128-bit ephemeral addresses) are not
97+cryptographic assets and must self-attest over Ames by sending a
98+single self-signed packet containing their public key.
99+
100+When a peer suffers a continuity breach, Ames removes all
101+messaging state related to it. Ames does not guarantee that all
102+messages will be fully delivered to the now-stale peer. From
103+Ames's perspective, the newly restarted peer is a new ship.
104+Ames's guarantees are not maintained across a breach.
105+
106+A vane can pass Ames a `%heed` `$task` to request Ames track a peer's
107+responsiveness. If our `%boon`'s to it start backing up locally,
108+Ames will give a `%clog` back to the requesting vane containing the
109+unresponsive peer's urbit address. This interaction does not use
110+ducts as unique keys. Stop tracking a peer by sending Ames a
111+`%jilt` `$task`.
112+
113+Debug output can be adjusted using `%sift` and `%spew` `$task`'s.