4`%ames` is our networking protocol.
6First we give commentary on the code, the algorithms involved, and the
7protocol. We trace through the code touched when a packet is sent,
8received, acknowledged, and that acknowledgment applied. This is fairly
9comprehensive, and contains many implementation details, but if you
10understand this, then you understand `%ames`.
12If you've scrolled down this page, you may be intimidated by the amount
13of Hoon code, especially if you are new to the language. Don't be afraid
14of it, you don't have to read any of it if you don't want to -- every
15interesting action the code takes is explained in plain English. In
16fact, if you are new to the language, this may be a good learning
17opportunity. Even if you don't understand every line of Hoon code,
18you'll hopefully be able to follow most lines. By the time you've worked
19through this, you'll have seen many common patterns and best practices.
20Hoon, much more than other languages, is best learned by reading and
21understanding large quantities of existing code. In this way, it is
22similar to learning a natural language. All of this code is in
25After the commentary, we have reference documentation for all the data
26structures that are specific to `%ames`. If you see a data structure or
27a variable used that you don't recognize, search for it in the code, and
28it's very likely defined in one of these data structures. We recommend
29that another tab is kept open for easy access to the data structure
30reference documentation. The code for these is split between
31`arvo/ames.hoon` and `arvo/zuse.hoon`.
33The Lifecycle of a Packet (or, How a Packet Becomes Law)
34--------------------------------------------------------
36Here, we will trace a packet as it makes its way through ames. There are
37actually two pathways through ames: the legacy path through `%want`, and
38the modern way, entered through `%wont`, with full end-to-end
39acknowledgments. Here we will only trace the modern way, though much of
40the path is the same for both.
42When an app (or a vane) wishes to send a packet to another ship, it must
45 [%wont p=sock q=path r=*] :: e2e send message
47This card takes three arguments. The `p` is a `sock`, that is, a pair of
48two ships, the first of which is the sender and the second is the
49receiver. But wait, you ask, why do I get to decide who is the sender?
50Can I fake like I'm someone else? The reason is that there are
51potentially multiple ships on the same pier, and the kernel can send a
52message from any of them. If you attempt to send a message from a ship
53not on your pier, then ames will refuse to send it. If you hack around
54in your own copy of ames to go ahead and send it anyway, then the other
55ship will reject it because your key is bad. Only send messages from
58The `q` is a path, representing the place on the other side that you
59want to receive your message. It is approximately equivalent to a port
60number. Messages on the same path are guaranteed to arrive in the same
61order as they were sent. No such guarantees are made across paths.
63The `r` is the actual data that you are sending. As the type implies,
64this can be an arbitrary noun, and it will be transferred to the
65receiver exactly as-is, in a well-typed way. Of course, this is data
66that is sent over the wire, so be careful not to send anything too
67massive unless you're willing to wait.
69But enough about the interface. Grepping in ames.hoon for `%wont`, we
70find that it appears in exactly two places: at its definition in
71`++kiss`, and in `++knob`, where it is handled. We see that we go
72directly into `++wise:am`.
75 |= [soq=sock hen=duct cha=path val=* ete=?] :: send a statement
76 ^- [p=(list boon) q=fort]
77 zork:zank:(wool:(ho:(um p.soq) q.soq) hen cha val ete)
79The inputs to this gate are exactly the sort of thing you'd expect. In
80particular, everything in the `%wont` gate is here plus the calling duct
81so that we know where to send the acknowledgment and `ete` to determine
82if we're going to do the modern end-to-end acknowledgments.
84The actual line of code looks intimidating, but it's really not all that
85bad. Working from the inside out, the call to `++um` sets up our
86domestic server, and the call to `++ho` sets up our knowledge about the
87neighbor we're sending to. From the outside, `++zork` and `++zank` just
88apply the changes made to our `++um` and `++am` cores, respectively. If
89you're familiar with the common idiom of `++abet`, that's all this is.
90The code predates the widespread usage of that name.
92The interesting part, then, is in `++wool:ho:um:am`. Let's look at the
95 ++ wool :: wool:ho:um:am
96 |= [hen=duct cha=path val=* ete=?] :: send a statement
99 =+ rol=(~(get by ryl.bah) cha)
102 :: ~& [%tx [our her] cha sex]
104 %+ ~(put by ryl.bah) cha
105 rol(sed +(sed.rol), san (~(put by san.rol) sex hen))
106 =+ cov=[p=p:sen:gus q=clon:diz]
109 [%bund q.cov cha sex val]
110 [%bond q.cov cha sex val]
112This is slightly more complicated, but it's still not all that bad. Our
113inputs, at least, are fairly obvious.
115If you glance at the code for a second, you'll see that
116`++wind:ho:um:am` seems to be able to send a message, or `++meal`, given
117a `++soup`. This gate, then, just sets up the things we need to for
118`++wind` to do its job.
120We first get `rol`, which is a `++rill`, that is, a particular outbound
121stream. This stream is specific to the path on which we're sending. If
122the path hasn't been used before, then we create it. We let `sex` be the
123number of messages we've already sent on this path.
125Then, we update the outbound stream by incrementing the number of
126messages sent and placing an entry in `san.rol` that associates the
127message number with the `duct` that sent the message. This allows us to
128give the acknowledgment to the one who sent the message.
130We let `cov` be the current life of our crypto and our neighbor's
131crypto. At the moment, we only need our neighbor's life, which we put
134Finally, we call `++wind:ho:um:am` with the `++soup` of the path and
135message number and the `++meal` of the payload itself. For end-to-end
136acknowledged messages, we use `%bund`.
138 [%bund p=life q=path r=@ud s=*] :: e2e message
140Looking at how we create the `%bund`, we can easily see what each field
143Following the trail a little further, we go to `++wind:ho:um:am`.
145 ++ wind :: wind:ho:um:am
146 |= [gom=soup ham=meal]
147 :: ~& [%wind her gom]
149 =^ wyv diz (zuul:diz now ham)
150 =^ feh puz (whap:puz now gom wyv)
153`++wind` does three things: it (1) encodes the message into a list of
154possibly-encrypted packets, (2) puts the message into the packet pump,
155and (3) sends any packets that are ready to be sent. Yes, our nice
156little linear run of each gate calling exactly one other interesting
157gate is over. We'll go in order here.
159`++zuul:lax:as:go` is the what converts a `++meal` into a list of
162 ++ zuul :: zuul:lax:as:go
163 |= [now=@da ham=meal] :: encode message
164 ^- [p=(list rock) q=_+>]
166 ++ wasp :: null security
167 ++ weft :: fragment message
168 ++ wisp :: generate message
170For organizational purposes, `++zuul` constructs an internal core with
171three arms. `++wasp` encodes the meal into an atom with no encryption.
172`++wisp` encodes a meal with possible encryption (else it simply calls
173`++wasp`). `++weft` takes the result of `++wisp` and splits it into
176 ++ wasp :: null security
177 ^-([p=skin q=@] [%none (jam ham)])
179This simply jams the meal, wrapping it with the `skin` of `%none`,
180meaning no encryption.
182Since `++wisp` is a little long, we'll go through it line-by-line.
184 ++ wisp :: generate message
185 ^- [[p=skin q=@] q=_..wisp]
187`++wisp` produces a pair of a `skin` and an atom, which is the meal
188encoded as a single atom and possibly encrypted.
193If the meal that we're encoding is a `%carp`, then we don't encrypt it.
194A `%carp` meal is a partial meal, used when a message is more than 1KB.
195Since the entire message is already encrypted, we don't need to encrypt
196each packet individually again.
204 (en:r:cluy q.u.yed.caq.dur (jam ham))
206If we have a symmetric key set up with this neighbor, then we simply use
207it. The skin `%fast` is used to indicate a symmetric key.
209 ?: &(=(~ lew.wod.dur) |(=(%back -.ham) =(%buck -.ham)))
212If we do not yet have our neighbor's will, then there is no way that we
213can seal the message so that only they may read it. If what we're
214sending is an acknowledgment, then we go ahead and just send it in the
218 ?:(=(~ lew.wod.dur) [*code +>.$] (griz now))
220If we don't have our neighbor's will, then we "encrypt" with a key of 0.
221If we do have their will, then we generate a new symmetric key that we
226 =+ bil=law.saf :: XX send whole will
229`yig` will be the life and engine for our current crypto. `bil` is our
230will. `hom` is the meal encoded as a single atom.
237 (sign:as:q.yig tuy hom)
239If we do not have our neighbor's will, then we send our current life
240along with our will and the message. The message itself is "signed" with
246 [`life`p.cay `life`p.yig]
248 (seal:as:q.yig pub:ex:r.cay tuy hom)
249 -- :: --zuul:lax:as:go
251If we do have our neighbor's will, then we send our perception of their
252current life, our current life, our will, and the message. The message
253is sealed with their public key so that only they can read our message.
255Once we have the message encoded as an atom, `++weft` goes to work.
257 ++ weft :: fragment message
258 ^- [p=(list rock) q=_+>.$]
263We're going to produce a list of the packets to send. First, we use the
264aforementioned `++wisp` to get the message as an atom.
266 =+ wit=(met 13 q.gim)
269`wit` is the number of 1KB (2\^13 bit) blocks in the message. We assert
270that there is at least one block.
273 =+ yup=(spit [our her] p.gim q.gim)
276If there is exactly one block, then we just call `++spit` to turn the
277message into a packet. We'll explain what `++spit` does momentarily.
279 =+ ruv=(rip 13 q.gim)
280 =+ gom=(shaf %thug q.gim)
283If there is more than one block, then we rip it into blocks in `ruv`.
284`gom` is a hash of the message, used as an id. `inx` is the number of
285packets we've already made.
292 wasp(ham [%carp (ksin p.gim) inx wit gom i.ruv])
294 $(ruv t.ruv, inx +(inx))
296Here we package each block into a packet with `++spit` and produce the
299 ++ spit :: cake to packet
301 =+ wim=(met 3 p.p.kec)
302 =+ dum=(met 3 q.p.kec)
303 =+ yax=?:((lte wim 2) 0 ?:((lte wim 4) 1 ?:((lte wim 8) 2 3)))
304 =+ qax=?:((lte dum 2) 0 ?:((lte dum 4) 1 ?:((lte dum 8) 2 3)))
307 =+ bod=:(mix p.p.kec (lsh 3 wix q.p.kec) (lsh 3 (add wix vix) r.kec))
319This is how we turn a message into a real packet. This has the
320definition of the packet format.
322`wim` is the length of the sending ship, and `dum` is the length of the
323receiving ship. There are only five possibilities for each of those,
324corresponding to carriers, cruisers, destroyers, yachts, and submarines.
325These are encoded in `yax` and `qax` as 0, 0, 1, 2, and 3, respectively.
326Thus, `wix` and `vix` are the number of bytes that must be reserved for
327the ship names in a packet.
329Next, we construct `bod` by simply concatenating the sending ship, the
330receiving ship, and the body of the message. Then, we get the encryption
331mechanism from `++skin`, which may be a 0, 1, 2, or 3, and put it in
334Next, we concatenate together, bit by bit, some final metadata. We use
335three bits for our protocol number, which is incremented modulo eight
336when there is a continuity breach or the protocol changes. We use the
337final twenty bits of a hash of the body (which, we suppose, makes it a
338twenty bit hash) for error-checking. We use two bits to tell how much
339room is used in the body for the sending ship, and another two bits for
340the receiving ship. Finally, we use five bits to store the encryption
341type. Note that since there are only two bits worth of encryption types,
342there are three unused bits here. This adds up to 32 bits of header
343data. Finally, we concatenate this onto the front of the packet. Thus,
344we can summarize the packet header format as follows.
347 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
348 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
349 |Proto| Hash of Body |yax|qax| Crypto |
350 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
352After this, there are `yax` bits of the sender name, `qax` bits of the
353receiver name, and up to 8192 bits of data. Thus, the maximum size of a
354packet is achieved in a message between two submarines with 8192 bits of
355data. This will require 32+128+128+8192 = 8480 bits, or 1060 bytes.
357This concludes our discussion of `++zuul:lax:as:go`. If you recall from
358`++wind:ho:um:am`, the list of packets from `++zuul` is passed into
359`++whap:pu` to update the packet pump and get any packets that can be
363 |= [now=@da gom=soup wyv=(list rock)] :: send a message
365 =. pyz (~(put by pyz) gom (lent wyv))
372 diq (~(put by diq) (shaf %flap i.wyv) nus)
373 puq (~(put to puq) [nus `soul`[gom 0 | ~2000.1.1 i.wyv]])
377First, we put into `pyz` the id for this message and the number of its
378packets that have not yet been acknowledged, which is of course the
379total number of packets since we haven't even sent the packets.
381For every packet, we change three things in the state (`++shed`) of our
382packet pump: (1) we increment `nus`, the number of packets sent; (2) we
383put the packet number into `diq` keyed by a hash of the packet; and (3)
384we put the packet into the packet queue, with the basic metadata of its
385id `gom`, 0 transmissions, not live yet, last sent in the year 2000, and
388Finally, we harvest the packet pump.
391 |= now=@da :: harvest queue
393 ?: =(~ puq) [~ +>(rtn ~)]
394 ?. (gth caw nif) [~ +>]
400`++harv` contains a core for most of its work. The meat is in `++apse`.
401First, though, it sets itself up. If there aren't any packets in the
402queue, then we simply do nothing except set `rtn`, our next timeout, to
403nil because we don't have any packets that may need to be retransmitted.
404If we have more live (that is, sent and unacknowledged) packets than our
405window size, then we don't do anything.
407Otherwise, we let `wid` be the width of our remaining packet window, and
408we initialize `rub` to nil. `rub` will be the list of packets that are
409ready to be sent. We then call `++apse` and pass the result to `++abet`.
410`++apse` decides which packets are ready to be sent.
419 ?. =(| liv.q.n.puq) .
420 :: ~& [%harv nux.q.n.puq p.n.puq]
423 rub [pac.q.n.puq rub]
426 nux.q.n.puq +(nux.q.n.puq)
430If there are no remaining packets to send, or if we've filled the packet
431window, do nothing. We call `++rigt` and `++left` to process the left
432and right branches of the packet queue.
434Now we assert that the queue is not empty, and we again check that we
435haven't filled the packet window. We will operate on the head of the
436queue. If the packet is live, then do nothing. Otherwise, we go ahead
439To send, we (1) decrement `wid`, our packet window width; (2) cons the
440packet onto the `rub`, which will be returned as the list of packets to
441send; (3) increment `nif`, the number of live packets; (4) set the
442packet to be live; (5) increment the number of transmissions of the
443packet; and (6) set the last sent time of the packet to now.
447 ^+(. =+(lef=apse(puq l.puq) lef(puq [n.puq puq.lef r.puq])))
450 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
452These do exactly what you would expect: they traverse the packet queue
453so that `++apse` gets called recursively through it.
455Finally, `++abet` gets called, which resolves the changes.
459 [(flop rub) +>.$(rtn [~ (add rto now)])]
461This returns the packets that we wish to send, and it updates the
462timeout so that we know when to try resending unacknowledged packets.
464This concludes our discussion of `++whap:pu`. To finish
465`++wind:ho:um:am`, we just need to delve into `++busk:ho:um:am`. But
466wait, in the call to `++busk`, the first argument is `xong:diz`. What is
467this? This, my dear reader, is one more detour, this time into
470 ++ xong :: xong:lax:as:go
471 ^- (list ship) :: route unto
472 =+ [fro=xen too=xeno]
473 =+ ^= oot ^- (list ship)
477 ?: (lien fro |=(a=ship =(a i.too))) ~
479 :: ~& [%xong-to [our her] (weld oot ?>(?=(^ fro) t.fro))]
480 (weld oot ?>(?=(^ fro) t.fro))
482This gets the list of intermediate ships needed to get a packet from us
483to our neighbor. First, we get `fro` and `too`, the "canons" of ourself
484and our neighbor, respectively.
486What is this "canon", you ask? A canon is simply a ship plus its
487"ancestors", as defined by `++sein`. For example, the canon of
490 ~hoclur-bicrel/try=> (saxo ~hoclur-bicrel)
491 ~[~hoclur-bicrel ~tasruc ~tug]
493If we follow the algorithm in `++xong`, we see that we are simply
494creating a list of ships that form a path from our neighbor to ourself.
495Essentially, we look through the canon of our neighbor until we find
496something in our own cannon -- a common ancestor. Or, if we are from
497different carriers, then there is no common ancestor. We then weld this
498onto the tail of our own canon. In the end, this is simply a list of
499possible ships to try to route via to get to our neighbor, ordered by
500preferability (that is, closeness to our neighbor). We will end up
501trying, in order, to find a lane to these.
503Now, we can finally get to `++busk:ho:um:am`.
505 ++ busk :: busk:ho:um:am
506 |= [waz=(list ship) pax=(list rock)] :: send packets
511 $(pax t.pax, bin (weld (flop (wist:diz now waz ~ i.pax)) bin))
514Thankfully, `++busk` is fairly simple. We go through the list of packets
515and convert them to `++boon`s with `++wist:lax:as:go`. These boons are
516placed into `bin`, and they end up getting processed by `++clop` (this
519 ++ wist :: wist:lax:as:go
520 |= $: now=@da :: route via
526 ?: =(our her) [[%ouzo *lane pac] ~]
528 =+ dyr=?:(=(her i.waz) dur (gur i.waz))
533 :_ ?: ?=(%ix -.u.lun.wod.dyr)
536 :+ %ouzo u.lun.wod.dyr
537 ?: &(=(i.waz her) =(~ ryn)) pac
538 =+ mal=(jam `meal`[%fore her ryn pac])
542 ?~ yed.caq.dyr [%none mal]
546 (en:crua q.u.yed.caq.dyr mal)
549This takes a sample of the current time, the list of ships that we just
550generated, a lane if we already know it, and the packet itself.
552First, if we are sending a message to ourself, then we simply create a
553`%ouzo` boon with a bunted lane. Otherwise, if there are no routing
554candidates, there is nothing we can do, so we return nil.
556Next, we get the `dore` of the first routing candidate. If we're looking
557at the neighbor to whom we're trying to send the message, then we simply
558use the `dore` that we already have. Otherwise, we get a default `dore`.
560If we're the first routing candidate, or if we have don't have a lane to
561this candidate, then we skip this candidate and move on to the next one.
563If we have only a provisional ip address, then we try to send on it, but
564we also try to send on later routing candidates as well. Otherwise, we
565only send on this one candidate.
567Finally, we create the actual `%ouzo` boon. The lane is the one from our
568`dore`. If we're sending it directly to our intended recipient, and we
569haven't been told to use a specific lane, then we just send the packet
570directly. Otherwise, we wrap it in a little `%fore` meal, telling the
571intermediary to whom we wish it to be sent. If we have already set up a
572symmetric key with the intermediary, then we encrypt it with that.
573Otherwise, we send it in the clear.
575Now, if you recall, we have traced all the way through from the
576beginning when, in `++knob`, the `%wont` card was handled by a call to
577`++wise`. There is only one more step before the packet is finally sent.
578Looking in `++knob`, we see that the resultant list of boons is passed
579into `++clop`, which will execute the correct actions and return a list
580of moves. In `++clop`, we see the handling of each specific boon. The
581one we are interested in is `%ouzo`, since that is the only one we have
585 :: ~& [%send now p.bon `@p`(mug (shaf %flap q.bon))]
587 [[gad.fox [%give %send p.bon q.bon]] ~]
589Very simply, we give a `%send` gift along the special duct that goes
590straight into the bowels of unix. This is the last stop before we drop
591into vere, and later libuv. And then... the world.
593The packet, after its creation, embarks on a journey across physical
594time and space into the great unknown. Hurtling through fiber-optic
595cables at hundreds of thousands of kilometers per second, it finally
596arrives at our neighbor's network adapter. The adapter tells unix, unix
597tells libuv, libuv tells vere, and vere sends a `%hear` kiss to ames.
598And now we reenter the kernel.
600The `%hear` kiss goes straight to `++knob`, just as did the `%wont` kiss
604 (~(gnaw am [now fox]) %good p.kyz q.kyz)
606Here, though, we call `++gnaw:am` to process the packet. The arguments
607to `++gnaw` are the same as those to the `%hear` kiss: the lane on which
608the packet was received and the packet itself. The other argument is
609just `%good`, which is a `++cape` saying that we expect the packet to
610succeed. If a formal error occurs, then since we have a transactional
611event system, the `%hear` event will never be considered to have
612actually happened, and unix will send a `%hole` kiss so that we may send
613a negative acknowledgment.
616 |= [kay=cape ryn=lane pac=rock] :: process packet
617 ^- [p=(list boon) q=fort]
618 ?. =(2 (end 0 3 pac)) [~ fox]
620 ?: (goop p.p.kec) [~ fox]
621 ?. (~(has by urb.ton.fox) q.p.kec)
625 %- ~(chew la:(ho:(um q.p.kec) p.p.kec) kay ryn %none (shaf %flap pac))
628First, we check the protocol number. If it is not correct, then we
629simply ignore the packet entirely. Otherwise, we parse the packet with
630`++bite`, which converts a packet atom into a `cake`, that is, a triple
631of the `sock` (pair of sender and receiver), the `skin` (encryption
634 ++ bite :: packet to cake
636 =+ [mag=(end 5 1 pac) bod=(rsh 5 1 pac)]
637 =+ :* vez=(end 0 3 mag) :: protocol version
638 chk=(cut 0 [3 20] mag) :: checksum
639 wix=(bex +((cut 0 [23 2] mag))) :: width of receiver
640 vix=(bex +((cut 0 [25 2] mag))) :: width of sender
641 tay=(cut 0 [27 5] mag) :: message type
644 ?> =(chk (end 0 20 (mug bod)))
645 :+ [(end 3 wix bod) (cut 3 [wix vix] bod)]
647 (rsh 3 (add wix vix) bod)
649This is exactly the inverse of `++spit`. Note that here we check both
650the protocol number and the hash, crashing on error. Remember that a
651crash will result in a negative acknowledgment being sent.
653Continuing in `++gnaw`, we see that if the intended recipient is not on
654our pier, then we drop the packet.
656If we've gotten this far, then we wish to process the packet. Recall
657that `++ho` and `++um` set up the domestic server and foreign client
658cores, respectively, and that `++zork` and `++zank` resolve any changes
661The new stuff here, then, is the `++la` core and the `++chew` arm. The
662`++la` sets up a core for this particular packet, containing the current
663success/failure `cape`, the lane it was sent on, the encryption type,
664and a hash of the packet, used as an id.
666`++chew` is called with the encryption type and the message itself. It
667contains a little helper core inside of it, which starts immediately
673 =+ neg==(~ yed.caq.dur.diz)
675 =+ eng==(~ yed.caq.dur.diz)
679 :_(bin [%wine [our her] " is ok"])
682 :_(bin [%wine [our her] " is your neighbor"])
685First, we let `oub` be true if our neighbor hasn't been responding to us
686for more than sixteen seconds. Let `neg` be true if we haven't yet
687proposed a symmetric key, meaning that we haven't yet corresponded with
688this ship, so they are not our neighbor. Next, we run `++east`, which
689we'll go into in just a minute.
691We now do the same two checks and store the results in `eng` and `bou`.
692If our neighbor has, like the prodigal son, returned after an extended
693absense, then we send a `%wine` boon as the proverbial fatted calf,
694which is simply printed out to the console. Likewise, if we are meeting
695one with whom we have never had the pleasure of acquainting ourselves,
696we send a message to the console to that effect.
698We skipped over `++east`, which contains the meat of the processing. It
699first decrypts the message, then calls `++chow:la:ho:um:am` with the
700resultant meal. We'll go through each of the four cases in turn, but
701first since each one calls `++bilk:pu`, we'll take a brief detour.
704 |= now=@da :: inbound packet
710 rtn ?~(puq ~ [~ (add now trt)])
713This updates the timing information in our packet pump. `rue`, the last
714time we have heard from this neighbor, is set to now. `rto`, the
715retransmit timeout is set to twice the current ping time, and if there
716is anything in the packet queue, then we reset the next timeout, since
717we've just heard a message.
722 =. puz (bilk:puz now)
723 (chow ((hard meal) (cue msg)))
725The simplest case is when the encryption type is `%none`. We first call
726`++bilk` to update the packet pump, then we cue (unjam) the message into
727a meal. We hard cast it into a meal -- if the cast fails, then we do
728want to crash since someone is sending us malformed data. Finally, we
729send the result to `++chow` for interpretation and handling.
732 =+ [mag=`hand`(end 7 1 msg) bod=(rsh 7 1 msg)]
733 =+ dey=(kuch:diz mag)
735 ~& [%bad-key her mag]
736 +>.$ :: ignore unknown key
737 =. puz (bilk:puz now)
739 (chow(aut sin) ((hard meal) (cue (dy:q:sen:gus key bod))))
741For symmetric encryption, we first get the `hand`, which is the hash of
742the symmetric key. We pass it to `++kuch:lax:as:go`, which returns the
743key if we either have used it before or we have proposed it. If we have
744proposed it, then we change its status from proposed to real. If
745`++kuch` fails, then we drop the packet and print out a `%bad-key`
748Otherwise, we call `++bilk` as before to update the packet pump and pass
749into `++chow` the decrypted data.
752 =+ mex=((hard ,[p=[p=life q=life] q=will r=@]) (cue msg))
753 =. diz (deng:diz q.mex)
756 =+ gey=(sev:gus p.p.mex)
757 =+ mes=(need (tear:as:q.gey pub:ex:r.wug r.mex))
758 =. diz (wasc:diz p.mes)
759 =. puz (bilk:puz now)
762For sealed asymmetric encryption, we first take off the the layer of
763data that gives us the life and will of our neighbor, and we apply try
764to extend their former will with the new data. `++deng` will fail if
767Next, we get our most current understanding of our neighbor's crypto,
768and we verify that it's the same life as what they're sending. Then, we
769get our own crypto from `++sev` and decrypt the message with the public
770key from our neighbor's crypto. We register the proposed symmetric key,
771update the packet pump, and call `++west`, which simply casts the
772message to a meal and calls `++chow`, reporting any error.
775 =+ mex=((hard ,[p=[~ q=life] q=will r=@]) (cue msg))
776 =. diz (deng:diz q.mex)
779 =+ mes=(need (sure:as:r.wug *code r.mex))
780 =. puz (bilk:puz now)
783Finally, for signed asymmetric encryption, we, as before, take off the
784layer of data that gives us the life and will of our neighbor. This
785time, of course, we do not get our own crypto -- only that of our
788The rest you have seen. We call `++deng` to extend the will, we verify
789that their crypto life is what we think it ought to be, we "decrypt" the
790data, we update the packet pump, and we call `++west` to call `++chow`.
792 ++ chow :: chow:la:ho:um:am
793 |= fud=meal :: interpret meal
795 =. diz ?:(=(%none aut) diz (wast:diz ryn))
798Here, if the message was encrypted at all, then we call
799`++wast:lax:as:go`, which simply updates the lane (route) to our
800neighbor (unless we're given a provisional route). This ensures that we
801always have the most direct possible path to them.
803We've been handling this meal for so long, we've almost forgotten what
804we want to do with it. The telos is of any meal to be dined on. We will
805choose out the cases here that are important to our current
811 ?. ?=(%if -.u.q.fud) u.q.fud
815 (emit %mead lyn r.fud)
816 =+ zid=(myx:gus p.fud)
817 (emir (wist:zid now xong:zid [~ lyn] r.fud))
819Forwarding is the simplest case, since we've seen all the arms before,
820except perhaps `++emit` and `++emir`, which simply take a boon or list
821of boons respectively and queue them up to be handled when the core
822resolves. If we're told to forward a packet to ourselves, then we emit a
823`%mead` boon which simply sends another `%hear` kiss to ourselves with
824the data. Otherwise, we try to find a route to the recipient, as before.
827 =+ zol=(~(get by olz.weg) s.fud)
828 ?^ zol cock(kay u.zol)
830 =+ neb=(~(get by nys.weg) s.fud)
831 ?^ neb [u.neb nys.weg]
832 =+ neb=`bait`[(kins p.fud) 0 r.fud ~]
833 [neb (~(put by nys.weg) s.fud neb)]
834 ?> (lth q.fud p.r.neb)
835 ?> =((kins p.fud) p.neb)
837 =+ doy=`(unit ,@)`(~(get by q.r.neb) q.fud)
840 q.r.neb (~(put by q.r.neb) q.fud t.fud)
843 :: ~& [%carp q.fud s.fud q.neb p.r.neb]
845 =: nys.weg (~(del by nys.weg) s.fud)
846 olz.weg (~(put by olz.weg) s.fud kay)
850 +>.$(nys.weg (~(put by nys.weg) s.fud neb))
852Here, we have received a partial message, and we're just assembling the
853individual packets into a message. Most of this code is fairly
854algorithmic, so we'll just hit the high points. In the beginning, we
855check if we've already received this message, and if so, we resend the
856acknowledgment. Remember, "always ack a dupe, never ack an ack".
858In `nys.weg` we keep track of an incoming set of partial packets,
859indexed by the `flap` hash that comes with every packet. We check to see
860if we have already received this partial message, and if so we
861acknowledge it. Otherwise, we put it in `nys.weg` unless this is the
862last message, in which case we ack the last partial message, move the
863complete message into `olz.weg`, and call `++golf`, which assembles the
864message and calls `++chew`, to start the dance again with the complete
868 :: ~& [%bund q.fud r.fud]
869 ?> =(p:sen:gus p.fud)
870 (deer q.fud r.fud ?-(kay %dead ~, %good [~ s.fud]))
872What if we're just receiving a regular old, garden variety message? We
873call `++deer` with the data from the message. If we already know that
874the message processing will fail (that is, if we got a `%hole` card from
875unix rather than a `%hear` card), then we don't even send the data at
876all. Remember, if a packet fails to process, it's as if it never even
877arrived, except that we send a negative acknowledgment.
879 ++ deer :: deer:la:ho:um:am
880 |= [cha=path num=@ud dut=(unit)] :: interpret message
882 =+ rum=(fall (~(get by raz.bah) cha) *race)
885 ?. (gte num did.rum) :: always ack a dup
886 (cook (~(get by bum.rum) num) cha ~ ryn dam)
888 (coat cha rum(mis (~(put by mis.rum) num [kay ryn dam dut])))
891 %+ ~(put by raz.bah) cha
892 rum(mis (~(put by mis.rum) num [kay ryn dam dut]))
896First, we get the race for this particular triple of sender, receiver,
897and path, creating it if it doesn't exist. If we've already acked the
898message, then we resend the ack. Note that `did.rum` is the number of
899packets we acknowledged, positively or negatively while `bum.rum` is a
900map of message numbers to negative acknowledgments. Thus, if a message
901number is less than `did.rum`, then if it's in `bum.rum` then it was
902negatively acknowledged, otherwise it's postively acknowledged. Thus, we
903are constant in space with the number of successful messages and linear
904in the number of failed messages. We'll document `++cook` later on, but
905suffice it to say that it sends an acknowledgment. It is to end-to-end
906acknowledgments what `++cock` is to packet-level acknowledgments.
908If we are still processing a message (that is, `dod.rum` is false), then
909we simply put this message in the map of misordered packets to be
910processed when their time comes. "Processing a message" in this case
911means that we've received the message and notified the correct
912application, but we're still waiting for the application-level
915Otherwise, we're ready for a packet, so we process it.
917 ++ coat :: coat:ho:um:am
918 |= [cha=path rum=race] :: update input race
920 =+ cun=(~(get by mis.rum) did.rum)
922 +>.$(raz.bah (~(put by raz.bah) cha rum))
923 ?. =(%good p.u.cun) +>.$
926 raz.bah (~(put by raz.bah) cha rum(dod |))
931 `soap`[[p:sen:gus clon:diz] cha did.rum]
935First, we grab the message we want to process and store it in `cun`. If
936it's a good packet, then we change `dod.rum` to false, meaning that
937we're in the middle of processing a packet and should not start
938processing another one. We also put a `%mulk` boon into the queue so
939that, when it all resolves, we send a mesage to the intended recipient
940application. The boon contains the sender, the receiver, the identity of
941the message, and the message itself.
943This bubbles up all the way back to `++knob`, where we were handling the
944`%hear` card. Following the logic in `++knob`, we can see that the boons
945get sent into `++clop` to be turned into actual arvo-level moves. We've
946been here before, if you recall, when we handled the `%cake` boon to
947send a message. Now, we're handling the `%mulk` boon, which is
948unfortunately slightly more complicated.
951 :: ~& [%mulk p.bon q.bon]
952 ?> ?=([@ @ *] q.q.bon)
958 [%sick %wart p.bon i.t.q.q.bon t.t.q.q.bon r.bon]
961 ?> ?=([@ ~] t.t.q.q.bon)
962 =+ app=`term`(need ((sand %tas) i.t.t.q.q.bon))
967 :_ fox [hen %pass pax %g %rote p.bon app r.bon]~
968 %gh :: %gall response
969 ?> ?=([@ ~] t.t.q.q.bon)
970 =+ app=`term`(need ((sand %tas) i.t.t.q.q.bon))
975 :_ fox [hen %pass pax %g %roth p.bon app r.bon]~
978We're dispatching messages based on the prefix of their path. Since only
979`%gall` apps use end-to-end acknowledgments at the moment, every path
980must have at least two elements, and the first one must be `%q`. Beyond
981that, we handle the `/q/ge` and `/q/gh` cases for gall requests and
982responses, respectively.
984In both cases, we require the next term in the path to be the name of
985the intended recipient `%gall` app. Thus, a message to `/q/ge/talk` for
986example, will send a message to the talk app.
988We then send a message to the app itself. The message is either a
989`%rote` or a `%roth` for a request and a response, respectively. The
990content is the `rook` or `roon` that was sent (stored in `r.bon`), but
991we don't actually handle that at all here. That's completely a
992`%gall`-level thing. We're just the messenger.
994Notice the path we send this over. We encode the sender, the receiver,
995and the path over which it was sent. This fully specifies the `race` so
996that when the app gives us the acknowledgment we know where to send it.
998We now have another interlude. We have entrusted our precious data, so
999carefully guarded and guided from the app on that far-away ship, to our
1000local app. It has the ability to do whatever it pleases with it. It may
1001take a significant amount of time to process. When the message has been
1002handled by this app, though, it must produce an acknowledgment. Our
1003final task is to deliver this acknowledgment to the sending app.
1005We should describe here what exactly these oft-mentioned acknowledgments
1006actually consist of. There are two kinds of acknowledgments: positive
1007and negative. A positive acknowledgment contains no data other than its
1008existence. A negative acknowledgment may optionally include a reason for
1009said negativity. Formally, a negative acknowledgment is an `ares`, which
1010is a unit pair of a term and a list of tanks. If this is null, this is
1011simply a failure with no associated information. If the pair exists, the
1012term is a short error code that is usually both human and computer
1013readable. For example, if you try to send a message to a valid `%gall`
1014app that doesn't have any `++poke` to handle it, then `%gall` will give
1015a negative acknowledgment with error term `%poke-find-fail`. The list of
1016tanks is a human-readable description of the error. This often contains
1017a stack trace. At any rate, all this information is returned to the
1018sending app on the other end of the wire.
1020After this brief interlude, our story resumes in `++knap`, where we
1021receive responses. In particular, a `%mean` indicates a negative
1022acknowledgment while a `%nice` indicates a positive acknowledgment.
1025 ?> ?=([@ @ @ *] tea)
1026 =+ soq=[(slav %p i.tea) (slav %p i.t.tea)]
1030 %^ ~(rack am [now fox]) soq pax
1031 ?-(+<.sih %mean `p.+.sih, %nice ~)
1034 |- ^- [p=(list move) q=_+>.^$]
1037 =^ toe fox (clop now hen i.p.fuy)
1038 $(p.fuy t.p.fuy, out (weld (flop toe) out))
1040Recall the format of the path we sent the message on, and you'll
1041understand why `soq` and `pax` are the sender/receiver pair and path on
1042which the message was sent. The rest of this is structured much like
1043`++knob`, so we call `++rack:am` and send the resulting boons to
1044`++clop`. Business as usual.
1047 |= [soq=sock cha=path cop=coop] :: e2e ack
1048 =+ oh=(ho:(um p.soq) q.soq)
1049 =. oh (cook:oh cop cha ~)
1052First, we set up `++um` and `++ho`, as we've done twice before, for our
1053domestic and foreign servers, respectively. The other two things are
1054new, though. Well, `++cook` is not actually new, but we delayed the
1055explanation saying only that it sends an acknowledgment. The time has
1058 ++ cook :: cook:ho:um:am
1059 |= [cop=coop cha=path ram=(unit ,[ryn=lane dam=flap])]
1060 ^+ +> :: acknowledgment
1061 =+ rum=(need (~(get by raz.bah) cha))
1062 =+ lat=(~(get by mis.rum) did.rum)
1063 ?: &(?=(~ lat) ?=(~ ram)) ~&(%ack-late-or-redundant +>.$)
1064 =+ ^- [ryn=lane dam=flap]
1065 ?^ ram [ryn.u.ram dam.u.ram]
1070 %+ ~(put by raz.bah) cha
1071 rum(dod &, bum ?~(cop bum.rum (~(put by bum.rum) did.rum u.cop)))
1072 =^ roc diz (zuul:diz now [%buck cop dam ~s0])
1073 (busk(diz (wast:diz ryn)) xong:diz roc)
1075If we are acknowledging a message that we have already acked, the `ram`
1076will contain the new lane and flap to send the duplicate ack to. This
1077happens if we call `++cook` in `++deer`, but it doesn't happen from
1078`++rack`. If there is no message waiting to be acknowledged and we're
1079not given an explicit lane and flap (that is, we're not sending a
1080duplicate ack), then the app must have sent us multiple acknowledgments.
1081We do the only sensible thing we can do and drop all acknowledgments
1082after the first, printing a message. This is, in fact, an error, so it
1083could be argued that we ought to crash. Whatever you do, don't depend on
1086First, we grab the race specified by the given path, and we get the most
1087recent in-order message, which must be the one which is being
1090Then, we decide which lane/flap to respond on/to. Basically, in the
1091usual case we respond on the lane through which the initial message was
1092sent, which is stored along with the other packet information in
1093`mis.rum`, since it has to be remembered across calls to ames. However,
1094if we receive a duplicate message, then we must respond to the new
1095message. It's quite possible the reason the other acknowledgment didn't
1096get returned was that the lane between the ships was broken.
1098At any rate, we update the race by saying that we've finished processing
1099this packet (unless we're sending a duplicate ack) and, if we're sending
1100a negative acknowledgment, putting the negative ack into `bum.rum` so
1101that we can resend it if necessary.
1103We encode our new message, updating the packet pump, with `++zuul`, as
1104before, and we send it off with `++busk`, routed via `++wast` to one of
1105the ships in `++xong`. Of course, in practice, we don't even look at the
1106ships in `++xong` because we already have a lane directly to our
1107neighbor (the one over which they sent their message to us).
1109We glossed over the actual message we're sending back. We're sending a
1110`%buck` meal, which is an acknowledgment. The `cop` specifies whether
1111this is a positive or a negative ack, `dam` specifies the message we're
1112acknowledging, and the `~s0` is a placeholder for the processing time
1113required. This time is neither calculated (though it is hopefully
1114obvious how to do so) nor used at present, but this information may be
1115used in the future for improved congestion control. Since the round-trip
1116time for an end-to-end acknowledged packet includes the processing time
1117on the other end, most common congestion control algorithms will stumble
1118when some messages take much longer to process than others. As noted,
1119though, this is simply an opportunity for improvement -- our congestion
1120control algorithms are relatively naive at the moment.
1122Recall that `++busk` calls `++wist` to put the actual `%ouzo` boon in
1123the queue, which gets handled by `++clop` to actually send the message.
1124This is the same pipeline as sending any other message, so we'll refer
1125you to the explanation above if you've forgotten it.
1127The last thing we need to do on this ship is move on to the next packet
1128in the queue if there is one. If you recall, in `++rack` after the call
1129to `++cook` there was a call to `++cans:ho:um:am`.
1131 ++ cans :: cans:ho:um:am
1133 =+ rum=(need (~(get by raz.bah) cha))
1137 mis (~(del by mis.rum) did.rum)
1141This is very simple. We increment the number of packets that we've
1142acknowledged on this race and we delete the packet that we just
1143acknowledged from the set of misordered packets.
1145Then, we call `++coat` again to process the next packet if we've already
1146received it. And that's it for this.
1148The acknowledgment now travels the same path that its forebearer, the
1149original message, once tread, but this time not into the great unknown.
1150The weary traveler is seeking out its familial roots, finding the app
1151from whom sprung forth the original message way back in paragraph three.
1152When it arrives at the network adapter of its ancestors, the adapter
1153tells unix, unix tells libuv, libuv tells vere, and vere sends a `%hear`
1154kiss to ames. Once more into the kernel.
1156The `%hear` kiss is handled in `++knob` as before, leading to `++gnaw`,
1157going over to `++chew`, `++apse`, `++chow`, and eventualy to `++dine`.
1158We've seen most of the cases in `++dine`, but we haven't yet looked at
1159the handling of this `%buck` meal.
1162 =. +> ?.(=(%full aut) +> cock) :: finish key exch
1163 +>(..la (tock p.fud q.fud r.fud))
1165We send a packet level acknowledgment if we're finishing a key exchange,
1166else we call `++tock` to process the acknowledgment.
1168This will get a little involved, so if you don't much care about how
1169exactly an acknowledgment happens, just know that the result gets gifted
1170as a `%woot` card back to the app who sent it. For those brave souls who
1171wish to see this thing through to the end, it's once more into the
1174 ++ tock :: tock:ho:um:am
1175 |= [cop=coop fap=flap cot=@dr] :: e2e ack by hash
1177 =^ yoh puz (bick:puz now fap)
1181 (done p.u.p.yoh q.u.p.yoh)
1186 `boon`[%cake [our her] [[p:sen:gus clon:diz] u.p.yoh] cop u.hud]
1188 (busk xong:diz q.yoh)
1190We're going to work through this one a little backwards since it's
1191mostly fairly simple except the call to `++bick:pu`. In fact, we'll just
1192skip `++bick` for the moment and finish the rest.
1194If `++bick` succesfully acks the message, then we call `++done`.
1196 ++ done :: done:ho:um:am
1197 |= [cha=path num=@ud] :: complete outgoing
1198 ^- [(unit duct) _+>]
1199 =+ rol=(need (~(get by ryl.bah) cha))
1200 =+ rix=(~(get by san.rol) num)
1205 (~(put by ryl.bah) cha rol(san (~(del by san.rol) num)))
1208This very simply gets the rill (the outgoing counterpart to a race, if
1209you recall), pulls out of the map of outstanding messages the duct over
1210which the original message was sent, and produces this duct while
1211deleting that entry from the map of outstanding messages.
1213Going back to `++tock`, we now have the duct we need to return the
1214result over. We do the very sensible thing and put a `%cake` boon in the
1215queue to be processed later by `++clop`.
1217In `q.yoh` we have a list of messages that may need to be sent, which we
1218pass to `++busk` to send, as usual. When an acknowledgment arrives, that
1219may trigger other messages immediately. This often happens when sending
1220more messages than the width of the logical window since for congestion
1221control reasons another message cannot be sent until some of the earlier
1222ones have been acknowledged.
1224We'll look at the processing of the `%cake` boon in `++clop` before we
1225get back to talking about `++bick`.
1229 :~ [s.bon %give %woot q.p.bon r.bon]
1232We very simply give, along the duct we found above, a `%woot` card with
1233the ship who sent us the ack and the ack itself. This allows the
1234application to decide what to do about the result. In case of a failure,
1235we usually either resend the message or display it to the user.
1236Sometimes, we recognize the error term and handle it internally. In any
1237case, the decision of how to handle the acknowledgment is entirely up to
1238the application. Our job is done.
1240Well, except that we skipped `++bick:pu`. Let's go back to that.
1243 |= [now=@da fap=flap] :: ack by hash
1244 ^- [[p=(unit soup) q=(list rock)] _+>]
1245 =+ sun=(~(get by diq) fap)
1248 =. diq (~(del by diq) fap)
1249 =^ gub +>.$ (bock now u.sun)
1250 =^ yop +>.$ (harv now)
1253If you recall, in `++whap:pu` we created the packet pump's
1254representation of the message, which included putting the message into
1255`diq`, which maps from packet hashes to packet sequence numbers. Thus,
1256`u.sun` is the sequence number of this particular message.
1258We delete this message from `diq` since we have now received an ack for
1259it. We call `++bock` to perform the ack by sequence number. We call
1260`++harv` to harvest the packet queue, sending any messages that are now
1263In `++bock`, there are three arms we haven't seen before: `++bine`,
1264`+wept`, and `++beet`. We'll describe each of these before we get to
1265`++bock`. `++bine` looks scariest.
1268 |= [now=@da num=@ud] :: apply ack
1269 ^- [(unit soup) _+>]
1272 ?: (gth num p.n.puq)
1274 [-.lef +.lef(puq [n.puq puq.lef r.puq])]
1276 [-.rig +.rig(puq [n.puq l.puq puq.rig])]
1277 =: rtt ?. &(liv.q.n.puq =(1 nux.q.n.puq)) rtt
1278 =+ gap=(sub now lys.q.n.puq)
1279 :: ~& [%bock-trip num (div gap (div ~s1 1.000))]
1280 (div (add (mul 2 rtt) gap) 3)
1281 nif (sub nif !liv.q.n.puq)
1283 =+ lez=(dec (need (~(get by pyz) gom.q.n.puq)))
1286 [[~ gom.q.n.puq] (~(del by pyz) gom.q.n.puq)]
1287 [~ (~(put by pyz) gom.q.n.puq lez)]
1289 +>.$(puq ~(nap to puq))
1291The first few lines are simply looking through the packet queue until we
1292find the correct packet to ack. This is basic queue manipulation that
1293operates directly on the treap structure of the queue. If you understand
1294treap queues, the logic is easy to follow. Otherwise, just trust us that
1295by the time we get to the `=:`, the packet with sequence number `num` is
1296on the top of the packet queue (that is, at `n.puq`).
1298We first update the round-trip time. If the packet is either not alive
1299or had to be transmitted more than once, then we don't have any reliable
1300way of calculating the round-trip time since we're unsure of exactly
1301which transmission was acknowledged. Otherwise, the round-trip time is
1302the difference between now and when the packet was last sent. We set
1303`rtt` by a little weighted average where the previous smoothed RTT is
1304weighted twice as much as the RTT of the current packet. Thus,
1305`(2*rtt+gap)/3`. This gives us a nice smooth RTT that is somewhat
1306resilient to outlier data while still being responsive to our
1309If the packet wasn't already dead, then we decrement the number of live
1310packets, which may allow more packets to be sent.
1312We decrement the number of unacknowledged packets in our `pyz` for this
1313particular message. If you recall, this was set in `++whap` to the
1314number of packets required to send a message.
1316If that was the last packet in the messge that needed to be acked, then
1317we delete the messgae reference from `pyz` and produce the id of the
1318message. Otherwise, we simply update `pyz` with the new number of
1319unacked messages. In either case, we remove the packet from the packet
1323 |= [fip=@ud lap=@ud] :: fip thru lap-1
1330 ?: (lth p.n.puq fip) ?~(l.puq . left)
1331 ?: (gte p.n.puq lap) ?~(r.puq . rigt)
1334 ?.(liv.q.n.puq . .(nif (dec nif), liv.q.n.puq |))
1338 ^+(. =+(lef=apse(puq l.puq) lef(puq [n.puq puq.lef r.puq])))
1341 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
1344The algorithm is a simple case of traversing the packet queue.
1345Essentialy, we mark as dead all packets in the queue between `fip` and
1346`(dec lap)`. We also update `nif`, the number of live packets. Lest you
1347mourn too much the passing of these packets, know that they shall soon
1348rise again. Recall that in `++bick` after the call to `++bock` we call
1349`++harv`. This will resend the packets that have just been labeled dead.
1352 ^+ . :: advance unacked
1353 =- +(nep ?~(foh nus u.foh))
1357 ?: (lte p.n.puq nep) $(puq l.puq)
1359 ?^(rig rig [~ p.n.puq])
1361Here we search for the next expected packet number. Basically, we search
1362the queue for the leftmost packet whose number is greater than the
1363current `nep`. If we don't find any such packet, we just use the total
1364number of packets sent.
1366We can now dive into `++bock`, our last arm.
1369 |= [now=@da num=@ud] :: ack by sequence
1370 ^- [(unit soup) _+>]
1371 =^ gym +> (bine now num)
1374 =+ cam=(max 2 (div caw 2))
1375 :: ~& [%bock-hole num nep cam]
1376 beet:(wept(nep num, cag cam, caw cam) nep num)
1377 =. caw ?: (lth caw cag) +(caw)
1378 (add caw !=(0 (mod (mug now) caw)))
1380 :: ~& [%bock-fine num nif caw cag]
1382 :: ~& [%bock-fill num nif caw cag]
1385First, we call `++bine` to apply the ack to the packet pump information.
1386We produce `gym`, which, if it exists, is the id of the packet that was
1387acked. If we received an ack for a packet later than the one we
1388expected, then we halve the logical packet window and kill all the
1389earlier packets so that they may be resent.
1391Otherwise, we possibly increase the congestion window. If the window is
1392less than the congestion threshold, then we increment the size of the
1393window. Otherwise, we only increment one out of every `caw` times.
1395If we received an ack for the packet we expected, then we simply advance
1396`nep` with `++beet`. If we received an ack for a packet earlier than we
1397expected, we do nothing.
1399It may be hard to believe, but we are, in fact, done. The message has
1400been sent, received, acknowledged, and the acknowledgment has been
1401returned to the original sender. We hope it's clear that, while the
1402process has been somewhat involved, the algorithms are not all that
1403complicated. If you've read this far, you know `%ames`. The only other
1404code involves initialization, timeouts, and the like.
1406Below, we give detailed reference documentation for the data models
1412### `++fort`, formal state
1414 ++ fort :: formal state
1416 gad=duct :: client interface
1417 hop=@da :: network boot date
1418 ton=town :: security
1419 zac=(map ship corn) :: flows by server
1422This is the state of our vane. Anything that must be remembered between
1423calls to ames must be stored in this state.
1425`%0` is the version of the ames state model itself. If the data model
1426`++fort` changes, then this number needs to be incremented, and an
1427adapter must be written to upgrade the old state into the new state.
1428Note that this is the version number of the model itself, not the
1429contents. When the data changes, there is of course no need to change
1432`gad` is a `duct` over which we send `%send` cards to unix. This card is
1433initialized when unix sends a `%barn` card as vere starts up. Vere
1434treats this duct specially -- don't send anything weird over it.
1436`hop` is the network boot date. This is set when the `%kick` card is
1437sent by vere on start up.
1439`ton` is a `++town`, where we store all of our security/encryption
1440state. Note that this is shared across all ships on a pier.
1442`zac` is a map of ships to `++corn`. This stores all the per-ship state.
1443The keys to this map are the ships on the current pier.
1445### `++town`, all security state
1447 ++ town :: all security state
1448 $: lit=@ud :: imperial modulus
1450 urb=(map ship sufi) :: all keys and routes
1454This is the security state of our pier.
1458`any` is 256 bits of entropy. This entropy is used and updated in
1459exactly two places: when we send a `%junk` card, and when we generate a
1460new symmetric key in `++griz:lax:as:go`. When it is updated, it is
1461updated by a SHA-256 hash of the current time and the old value of the
1464`urb` is a map of ships to `++sufi`. This is where we store all the
1465per-ship state for the pier. The keys to this map are the ships on the
1468`fak` is true if we are on a fake network. This disables certain
1469security checks so that anyone may run a fake `~zod`. This is used only
1470for development. To use, run vere with the `-F` option (and the
1471`-I ~zod` option for a fake `~zod`).
1473### `++sufi`, domestic host
1475 ++ sufi :: domestic host
1476 $: hoy=(list ship) :: hierarchy
1477 val=wund :: private keys
1478 law=will :: server will
1479 seh=(map hand ,[p=ship q=@da]) :: key cache
1480 hoc=(map ship dore) :: neighborhood
1483This is the security state of a domestic server.
1485`hoy` is a list of the ships directly above us in the hierarchy of
1486ships. For example, for `~hoclur-bicrel`, this would be `~tasruc` and
1487`~tug`. See `++sein`.
1489`val` is a list of our private keys.
1491`law` is our certificate, which is a list of the XXX
1495`hoc` is a map of ships to `++dore`. The stores all the security
1496informatoin about foreign ships. The keys to this map are the neighbors
1497(ships we have been in contact with) of this domestic server.
1499### `++wund`, private keys
1501 ++ wund (list ,[p=life q=ring r=acru]) :: mace in action
1503This is a list of our own private keys, indexed by life. The key itself
1504is the `++ring`, and the `++acru` is the encryption engine. We generate
1505the `++acru` from the private key by calling `++weur`. Thus, we can at
1506any time regenerate our `++wund` from a `++mace`. The current crypto is
1507at the head of the list and can be accessed with `++sen:as:go`.
1509### `++ring`, private key
1511 ++ ring ,@ :: private key
1513This is a private key. The first byte is reserved to identify the type
1514of cryptography. Lower-case means public key, upper-case means public
1515key, and the letter identifies which `++acru` to use.
1517### `++pass`, public key
1519 ++ pass ,@ :: public key
1521This is a public key. The first byte is reserved to identify the type of
1522cryptography. Lower-case means public key, upper-case means public key,
1523and the letter identifies which `++acru` to use.
1525### `++mace`, private secrets
1527 ++ mace (list ,[p=life q=ring]) :: private secrets
1529This is a list of the our private keys, indexed by life. From this we
1530can generate a `++wund` for actual use.
1532### `++skin`, encoding stem
1534 ++ skin ?(%none %open %fast %full) :: encoding stem
1536This defines the type of encryption used for each message. `%none`
1537refers to messages sent in the clear, `%open` refers to signed messages,
1538`%full` refers to sealed messages, and `%fast` refers to symmetrically
1539encrypted messages. See `++acru` for details.
1541### `++acru`, asymmetric cryptosuite
1543 ++ acru :: asym cryptosuite
1544 $_ ^? |% :: opaque object
1545 ++ as ^? :: asym ops
1546 |% ++ seal |=([a=pass b=@ c=@] _@) :: encrypt to a
1547 ++ sign |=([a=@ b=@] _@) :: certify as us
1548 ++ sure |=([a=@ b=@] *(unit ,@)) :: authenticate from us
1549 ++ tear |= [a=pass b=@] :: accept from a
1550 *(unit ,[p=@ q=@]) ::
1552 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1553 ++ dy |+([a=@ b=@] _@) :: symmetric de, hard
1554 ++ en |+([a=@ b=@] _@) :: symmetric en
1556 |% ++ fig _@uvH :: fingerprint
1557 ++ pac _@uvG :: default passcode
1558 ++ pub *pass :: public key
1559 ++ sec *ring :: private key
1561 ++ nu ^? :: reconstructors
1562 |% ++ pit |=([a=@ b=@] ^?(..nu)) :: from [width seed]
1563 ++ nol |=(a=@ ^?(..nu)) :: from naked ring
1564 ++ com |=(a=@ ^?(..nu)) :: from naked pass
1568This is an opaque interface for a general asymmetric cryptosuite. Any
1569form of asymmetric cryptography can be dropped in to be used instead of
1570the default. Right now, there are two cryptosuites, `++crua`, which is
1571your standard RSA, and `++crub`, which is elliptic curve crypto but is
1572mostly stubbed out at the moment.
1574#### `++as:acru`, asymmetric operations
1576 ++ as ^? :: asym ops
1577 |% ++ seal |=([a=pass b=@ c=@] _@) :: encrypt to a
1578 ++ sign |=([a=@ b=@] _@) :: certify as us
1579 ++ sure |=([a=@ b=@] *(unit ,@)) :: authenticate from us
1580 ++ tear |= [a=pass b=@] :: accept from a
1581 *(unit ,[p=@ q=@]) ::
1584This is the core that defines the standard asymmetric cryptography
1587`++seal:as:acru` allows us to send a message encrypted with someone's
1588public key so that only they may read it. If Alice seals a message with
1589Bob's public key, then she can be sure that Bob is the only one who can
1590read it. This is associated with the `++skin` `%full`.
1592`++sign:as:acru` allows us to sign a message with our private key so
1593that others can verify that we sent the message. If Alice signs a
1594message with her private key, then Bob can verify with her public key
1595that it was indeed Alice who sent it. This is associated with the
1598`++sure:as:acru` is the dual to `++sign:as:acru`. It allows us to verify
1599that a message we have received is indeed from the claimed sender. If
1600Alice sends a message with her private key, then Bob can use this arm to
1601verify that it was indeed Alice who sent it. This is associated with the
1604`++tear:as:acru` is the dual to `++seal:as:acru`. It allows us to read a
1605message that we can be sure is only read by us. If Alice seals a message
1606with Bob's public key, then Bob can use this arm to read it. This is
1607associated with the `++skin` `%full`.
1609#### `++de:acru`, `++dy:acru`, and `++en:acru`, symmetric encryption/decryption
1611 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1612 ++ dy |+([a=@ b=@] _@) :: symmetric de, hard
1613 ++ en |+([a=@ b=@] _@) :: symmetric en
1615Symmetric encryption is associated with the `++skin` `%fast`.
1617`++de:acru` decrypts a message with a symmetric key, returning `~` on
1618failure and `[~ u=data]` on success.
1620`++dy:acru` decrypts a message with a symmetric key, crashing on
1621failure. This should almost always be defined as, and should always be
1622semantically equivalent to, `(need (de a b))`.
1624`++en:acru` encrypts a message with a symmetric key.
1626#### `++ex:acru`, exporting data
1629 |% ++ fig _@uvH :: fingerprint
1630 ++ pac _@uvG :: default passcode
1631 ++ pub *pass :: public key
1632 ++ sec *ring :: private key
1635`++fig:ex:acru` is our fingerprint, usually a hash of our public key.
1636This is used, for example, in `++zeno`, where every carrier owner's
1637fingerprint is stored so that we can ensure that carriers are indeed
1638owned by their owners
1640`++pac:ex:acru` is our default passcode, which is unused at present.
1642`++pub:ex:acru` is the `++pass` form of our public key.
1644`++sec:ex:acru` is the `++ring` form of our private key.
1646#### `++nu:acru`, reconstructors
1648 ++ nu ^? :: reconstructors
1649 |% ++ pit |=([a=@ b=@] ^?(..nu)) :: from [width seed]
1650 ++ nol |=(a=@ ^?(..nu)) :: from naked ring
1651 ++ com |=(a=@ ^?(..nu)) :: from naked pass
1654These arms allow us to reconstruct a `++acru` from basic data.
1656`++pit:nu:acru` constructs a `++acru` from the width of our intended key
1657and seed entropy. This is usually used in the initial construction of
1660`++nol:nu:acru` constructs a `++acru` from a "naked ring", meaning a
1661`++ring` without the initial byte identifying the type of crypto. There
1662is often a helper arm that that wraps this; see `++weur` for `++crua`
1663and `++wear` for `++crub`.
1665`++com:nu:acru` constructs a `++acru` from a "naked pass", meaning a
1666`++ring` without the initial byte identifying the type of crypto. There
1667is often a helper arm that that wraps this; see `++haul` for `++crua`
1668and `++hail` for `++crub`.
1670### `++will`, certificate
1672 ++ will (list deed) :: certificate
1674This is a list of deeds associated with the current ship. There should
1675be an item in this list for every ship from this point up in the
1676hierarchy times the number of lives that each ship has had. For example,
1677\~hoclur-bicrel may have a will with three items: one for itself, one
1678for \~tasruc (who issued \~hoclur-bicrel's deed) and one for \~tug (who
1679issued \~tasruc's deed).
1681### `++deed`, identity
1683 ++ deed ,[p=@ q=step r=?] :: sig, stage, fake?
1685`p` is the signature of a particular deed, which is a signed copy of
1688`q` is the stage in the identity.
1690`r` is true if we're working on a fake network, where we don't check
1691that the carrier fingerprints are correct. This allows us to create fake
1692networks for development without interfering with the real network.
1694### `++step`, identity stage
1696 ++ step ,[p=bray q=gens r=pass] :: identity stage
1698This is a single stage in our identity. Thus, this is specific to a
1699single life in a single ship. Everything in here may change between
1706`r` is the public key for this stage in the identity.
1710 ++ bray ,[p=life q=(unit life) r=ship s=@da] :: our parent us now
1714### `++gens`, general identity
1716 ++ gens ,[p=lang q=gcos] :: general identity
1718`p` is the IETF language code for the preferred language of this
1719identity. This is unused at the moment, but in the future text should be
1720localized based on this.
1722`q` is the description of the ship.
1724### `++gcos`, identity description
1726 ++ gcos :: id description
1727 $% [%czar ~] :: 8-bit ship
1728 [%duke p=what] :: 32-bit ship
1729 [%earl p=@t] :: 64-bit ship
1730 [%king p=@t] :: 16-bit ship
1731 [%pawn p=(unit ,@t)] :: 128-bit ship
1734This is the description of the identity of a ship. Most types of
1735identity have a `@t` field, which is their human-readable name. The
1736identity of a `%duke` is more involved.
1738A `%czar`, a carrier, is a ship with an 8-bit address. Thus, there are
1739only 256 carriers. These are at the top of the namespace hierarchy, and
1740the fingerprint of each carrier is stored in `++zeno`. These are the
1743A `%king`, a cruiser, is a ship with a 16-bit address. Thus, there are
174465,536 cruisers. Each carrier may issue 256 cruisers. These are the
1745infrastructure of Urbit.
1747A `%duke`, a destroyer, is a ship with a 32-bit address. Thus, there are
17484,294,967,296 destroyers. Each cruiser may issue 65,536 cruisers. These
1749are the individuals of Urbit.
1751A `%earl`, a yacht, is a ship with a 64-bit address. Thus, there are
175218,446,744,073,709,551,616 yachts. Each destroyer may issue
17534,294,967,296 yachts. These are the devices of Urbit.
1755A `%pawn`, a submarine, is a ship with a 128-bit address. Thus, there
1756are a lot of submarines. The chance of random name collision is
1757negligible, so submarines are not issued by any ship. They must simply
1758assert their presence, and they are all considered children of \~zod.
1759This is the underworld of Urbit, where anonymity reigns supreme.
1761### `++what`, logical destroyer identity
1763 ++ what :: logical identity
1764 $% [%anon ~] :: anonymous
1765 [%lady p=whom] :: female person ()
1766 [%lord p=whom] :: male person []
1767 [%punk p=sect q=@t] :: opaque handle ""
1770This is the logical identity of a destroyer.
1772A `%anon` is a completely anonymous destroyer. The difference between
1773this and a submarine is that a submarine is ephemeral while a `%anon`
1774destroyer is not. Thus, we may not know who \~hoclur-bicrel is, but we
1775do know that it's always the same person.
1777A `%lady` is a female person. The name used here should be a real name.
1779A `%lord` is a male person. The name used here should be a real name.
1781A `%punk` is a person who is identified only by a handle.
1783### `++whom`, real person
1785 ++ whom ,[p=@ud q=govt r=sect s=name] :: year/govt/id
1787Ths is the information associated with a real person. It is mostly
1788information that could be observed with the briefest of interactions.
1790`p` is the birth year.
1792`q` is the location of a user, usually of the form "country/zip".
1794`r` is the sect of the user.
1796`s` is the real name of the person.
1800 ++ govt path :: country/postcode
1802This is the location of the user, usually of the form "country/zip".
1806 ++ sect ?(%black %blue %red %orange %white) :: banner
1812 ++ name ,[p=@t q=(unit ,@t) r=(unit ,@t) s=@t] :: first mid/nick last
1814This is the given name, possible middle name/initial, possible nickname,
1815and surname of a user.
keysesc to close
- j k
- move down, up
- enter
- open
- esc
- back, or up a level
- 1 2 3 4
- readme, files, commits, refs
- /
- filter
- y
- copy the clone command
- c
- compare the chosen commit with another (or shift-click both)
- ?
- show or hide this list