9`%ames` is our networking protocol.
11First we give commentary on the code, the algorithms involved, and the
12protocol. We trace through the code touched when a packet is sent,
13received, acknowledged, and that acknowledgment applied. This is fairly
14comprehensive, and contains many implementation details, but if you
15understand this, then you understand `%ames`.
17If you've scrolled down this page, you may be intimidated by the amount
18of Hoon code, especially if you are new to the language. Don't be afraid
19of it, you don't have to read any of it if you don't want to -- every
20interesting action the code takes is explained in plain English. In
21fact, if you are new to the language, this may be a good learning
22opportunity. Even if you don't understand every line of Hoon code,
23you'll hopefully be able to follow most lines. By the time you've worked
24through this, you'll have seen many common patterns and best practices.
25Hoon, much more than other languages, is best learned by reading and
26understanding large quantities of existing code. In this way, it is
27similar to learning a natural language. All of this code is in
30After the commentary, we have reference documentation for all the data
31structures that are specific to `%ames`. If you see a data structure or
32a variable used that you don't recognize, search for it in the code, and
33it's very likely defined in one of these data structures. We recommend
34that another tab is kept open for easy access to the data structure
35reference documentation. The code for these is split between
36`arvo/ames.hoon` and `arvo/zuse.hoon`.
38The Lifecycle of a Packet (or, How a Packet Becomes Law)
39--------------------------------------------------------
41Here, we will trace a packet as it makes its way through ames. There are
42actually two pathways through ames: the legacy path through `%want`, and
43the modern way, entered through `%wont`, with full end-to-end
44acknowledgments. Here we will only trace the modern way, though much of
45the path is the same for both.
47When an app (or a vane) wishes to send a packet to another ship, it must
50 [%wont p=sock q=path r=*] :: e2e send message
52This card takes three arguments. The `p` is a `sock`, that is, a pair of
53two ships, the first of which is the sender and the second is the
54receiver. But wait, you ask, why do I get to decide who is the sender?
55Can I fake like I'm someone else? The reason is that there are
56potentially multiple ships on the same pier, and the kernel can send a
57message from any of them. If you attempt to send a message from a ship
58not on your pier, then ames will refuse to send it. If you hack around
59in your own copy of ames to go ahead and send it anyway, then the other
60ship will reject it because your key is bad. Only send messages from
63The `q` is a path, representing the place on the other side that you
64want to receive your message. It is approximately equivalent to a port
65number. Messages on the same path are guaranteed to arrive in the same
66order as they were sent. No such guarantees are made across paths.
68The `r` is the actual data that you are sending. As the type implies,
69this can be an arbitrary noun, and it will be transferred to the
70receiver exactly as-is, in a well-typed way. Of course, this is data
71that is sent over the wire, so be careful not to send anything too
72massive unless you're willing to wait.
74But enough about the interface. Grepping in ames.hoon for `%wont`, we
75find that it appears in exactly two places: at its definition in
76`++kiss`, and in `++knob`, where it is handled. We see that we go
77directly into `++wise:am`.
80 |= [soq=sock hen=duct cha=path val=* ete=?] :: send a statement
81 ^- [p=(list boon) q=fort]
82 zork:zank:(wool:(ho:(um p.soq) q.soq) hen cha val ete)
84The inputs to this gate are exactly the sort of thing you'd expect. In
85particular, everything in the `%wont` gate is here plus the calling duct
86so that we know where to send the acknowledgment and `ete` to determine
87if we're going to do the modern end-to-end acknowledgments.
89The actual line of code looks intimidating, but it's really not all that
90bad. Working from the inside out, the call to `++um` sets up our
91domestic server, and the call to `++ho` sets up our knowledge about the
92neighbor we're sending to. From the outside, `++zork` and `++zank` just
93apply the changes made to our `++um` and `++am` cores, respectively. If
94you're familiar with the common idiom of `++abet`, that's all this is.
95The code predates the widespread usage of that name.
97The interesting part, then, is in `++wool:ho:um:am`. Let's look at the
100 ++ wool :: wool:ho:um:am
101 |= [hen=duct cha=path val=* ete=?] :: send a statement
104 =+ rol=(~(get by ryl.bah) cha)
107 :: ~& [%tx [our her] cha sex]
109 %+ ~(put by ryl.bah) cha
110 rol(sed +(sed.rol), san (~(put by san.rol) sex hen))
111 =+ cov=[p=p:sen:gus q=clon:diz]
114 [%bund q.cov cha sex val]
115 [%bond q.cov cha sex val]
117This is slightly more complicated, but it's still not all that bad. Our
118inputs, at least, are fairly obvious.
120If you glance at the code for a second, you'll see that
121`++wind:ho:um:am` seems to be able to send a message, or `++meal`, given
122a `++soup`. This gate, then, just sets up the things we need to for
123`++wind` to do its job.
125We first get `rol`, which is a `++rill`, that is, a particular outbound
126stream. This stream is specific to the path on which we're sending. If
127the path hasn't been used before, then we create it. We let `sex` be the
128number of messages we've already sent on this path.
130Then, we update the outbound stream by incrementing the number of
131messages sent and placing an entry in `san.rol` that associates the
132message number with the `duct` that sent the message. This allows us to
133give the acknowledgment to the one who sent the message.
135We let `cov` be the current life of our crypto and our neighbor's
136crypto. At the moment, we only need our neighbor's life, which we put
139Finally, we call `++wind:ho:um:am` with the `++soup` of the path and
140message number and the `++meal` of the payload itself. For end-to-end
141acknowledged messages, we use `%bund`.
143 [%bund p=life q=path r=@ud s=*] :: e2e message
145Looking at how we create the `%bund`, we can easily see what each field
148Following the trail a little further, we go to `++wind:ho:um:am`.
150 ++ wind :: wind:ho:um:am
151 |= [gom=soup ham=meal]
152 :: ~& [%wind her gom]
154 =^ wyv diz (zuul:diz now ham)
155 =^ feh puz (whap:puz now gom wyv)
158`++wind` does three things: it (1) encodes the message into a list of
159possibly-encrypted packets, (2) puts the message into the packet pump,
160and (3) sends any packets that are ready to be sent. Yes, our nice
161little linear run of each gate calling exactly one other interesting
162gate is over. We'll go in order here.
164`++zuul:lax:as:go` is the what converts a `++meal` into a list of
167 ++ zuul :: zuul:lax:as:go
168 |= [now=@da ham=meal] :: encode message
169 ^- [p=(list rock) q=_+>]
171 ++ wasp :: null security
172 ++ weft :: fragment message
173 ++ wisp :: generate message
175For organizational purposes, `++zuul` constructs an internal core with
176three arms. `++wasp` encodes the meal into an atom with no encryption.
177`++wisp` encodes a meal with possible encryption (else it simply calls
178`++wasp`). `++weft` takes the result of `++wisp` and splits it into
181 ++ wasp :: null security
182 ^-([p=skin q=@] [%none (jam ham)])
184This simply jams the meal, wrapping it with the `skin` of `%none`,
185meaning no encryption.
187Since `++wisp` is a little long, we'll go through it line-by-line.
189 ++ wisp :: generate message
190 ^- [[p=skin q=@] q=_..wisp]
192`++wisp` produces a pair of a `skin` and an atom, which is the meal
193encoded as a single atom and possibly encrypted.
198If the meal that we're encoding is a `%carp`, then we don't encrypt it.
199A `%carp` meal is a partial meal, used when a message is more than 1KB.
200Since the entire message is already encrypted, we don't need to encrypt
201each packet individually again.
209 (en:r:cluy q.u.yed.caq.dur (jam ham))
211If we have a symmetric key set up with this neighbor, then we simply use
212it. The skin `%fast` is used to indicate a symmetric key.
214 ?: &(=(~ lew.wod.dur) |(=(%back -.ham) =(%buck -.ham)))
217If we do not yet have our neighbor's will, then there is no way that we
218can seal the message so that only they may read it. If what we're
219sending is an acknowledgment, then we go ahead and just send it in the
223 ?:(=(~ lew.wod.dur) [*code +>.$] (griz now))
225If we don't have our neighbor's will, then we "encrypt" with a key of 0.
226If we do have their will, then we generate a new symmetric key that we
231 =+ bil=law.saf :: XX send whole will
234`yig` will be the life and engine for our current crypto. `bil` is our
235will. `hom` is the meal encoded as a single atom.
242 (sign:as:q.yig tuy hom)
244If we do not have our neighbor's will, then we send our current life
245along with our will and the message. The message itself is "signed" with
251 [`life`p.cay `life`p.yig]
253 (seal:as:q.yig pub:ex:r.cay tuy hom)
254 -- :: --zuul:lax:as:go
256If we do have our neighbor's will, then we send our perception of their
257current life, our current life, our will, and the message. The message
258is sealed with their public key so that only they can read our message.
260Once we have the message encoded as an atom, `++weft` goes to work.
262 ++ weft :: fragment message
263 ^- [p=(list rock) q=_+>.$]
268We're going to produce a list of the packets to send. First, we use the
269aforementioned `++wisp` to get the message as an atom.
271 =+ wit=(met 13 q.gim)
274`wit` is the number of 1KB (2\^13 bit) blocks in the message. We assert
275that there is at least one block.
278 =+ yup=(spit [our her] p.gim q.gim)
281If there is exactly one block, then we just call `++spit` to turn the
282message into a packet. We'll explain what `++spit` does momentarily.
284 =+ ruv=(rip 13 q.gim)
285 =+ gom=(shaf %thug q.gim)
288If there is more than one block, then we rip it into blocks in `ruv`.
289`gom` is a hash of the message, used as an id. `inx` is the number of
290packets we've already made.
297 wasp(ham [%carp (ksin p.gim) inx wit gom i.ruv])
299 $(ruv t.ruv, inx +(inx))
301Here we package each block into a packet with `++spit` and produce the
304 ++ spit :: cake to packet
306 =+ wim=(met 3 p.p.kec)
307 =+ dum=(met 3 q.p.kec)
308 =+ yax=?:((lte wim 2) 0 ?:((lte wim 4) 1 ?:((lte wim 8) 2 3)))
309 =+ qax=?:((lte dum 2) 0 ?:((lte dum 4) 1 ?:((lte dum 8) 2 3)))
312 =+ bod=:(mix p.p.kec (lsh 3 wix q.p.kec) (lsh 3 (add wix vix) r.kec))
324This is how we turn a message into a real packet. This has the
325definition of the packet format.
327`wim` is the length of the sending ship, and `dum` is the length of the
328receiving ship. There are only five possibilities for each of those,
329corresponding to carriers, cruisers, destroyers, yachts, and submarines.
330These are encoded in `yax` and `qax` as 0, 0, 1, 2, and 3, respectively.
331Thus, `wix` and `vix` are the number of bytes that must be reserved for
332the ship names in a packet.
334Next, we construct `bod` by simply concatenating the sending ship, the
335receiving ship, and the body of the message. Then, we get the encryption
336mechanism from `++skin`, which may be a 0, 1, 2, or 3, and put it in
339Next, we concatenate together, bit by bit, some final metadata. We use
340three bits for our protocol number, which is incremented modulo eight
341when there is a continuity breach or the protocol changes. We use the
342final twenty bits of a hash of the body (which, we suppose, makes it a
343twenty bit hash) for error-checking. We use two bits to tell how much
344room is used in the body for the sending ship, and another two bits for
345the receiving ship. Finally, we use five bits to store the encryption
346type. Note that since there are only two bits worth of encryption types,
347there are three unused bits here. This adds up to 32 bits of header
348data. Finally, we concatenate this onto the front of the packet. Thus,
349we can summarize the packet header format as follows.
352 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
353 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
354 |Proto| Hash of Body |yax|qax| Crypto |
355 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
357After this, there are `yax` bits of the sender name, `qax` bits of the
358receiver name, and up to 8192 bits of data. Thus, the maximum size of a
359packet is achieved in a message between two submarines with 8192 bits of
360data. This will require 32+128+128+8192 = 8480 bits, or 1060 bytes.
362This concludes our discussion of `++zuul:lax:as:go`. If you recall from
363`++wind:ho:um:am`, the list of packets from `++zuul` is passed into
364`++whap:pu` to update the packet pump and get any packets that can be
368 |= [now=@da gom=soup wyv=(list rock)] :: send a message
370 =. pyz (~(put by pyz) gom (lent wyv))
377 diq (~(put by diq) (shaf %flap i.wyv) nus)
378 puq (~(put to puq) [nus `soul`[gom 0 | ~2000.1.1 i.wyv]])
382First, we put into `pyz` the id for this message and the number of its
383packets that have not yet been acknowledged, which is of course the
384total number of packets since we haven't even sent the packets.
386For every packet, we change three things in the state (`++shed`) of our
387packet pump: (1) we increment `nus`, the number of packets sent; (2) we
388put the packet number into `diq` keyed by a hash of the packet; and (3)
389we put the packet into the packet queue, with the basic metadata of its
390id `gom`, 0 transmissions, not live yet, last sent in the year 2000, and
393Finally, we harvest the packet pump.
396 |= now=@da :: harvest queue
398 ?: =(~ puq) [~ +>(rtn ~)]
399 ?. (gth caw nif) [~ +>]
405`++harv` contains a core for most of its work. The meat is in `++apse`.
406First, though, it sets itself up. If there aren't any packets in the
407queue, then we simply do nothing except set `rtn`, our next timeout, to
408nil because we don't have any packets that may need to be retransmitted.
409If we have more live (that is, sent and unacknowledged) packets than our
410window size, then we don't do anything.
412Otherwise, we let `wid` be the width of our remaining packet window, and
413we initialize `rub` to nil. `rub` will be the list of packets that are
414ready to be sent. We then call `++apse` and pass the result to `++abet`.
415`++apse` decides which packets are ready to be sent.
424 ?. =(| liv.q.n.puq) .
425 :: ~& [%harv nux.q.n.puq p.n.puq]
428 rub [pac.q.n.puq rub]
431 nux.q.n.puq +(nux.q.n.puq)
435If there are no remaining packets to send, or if we've filled the packet
436window, do nothing. We call `++rigt` and `++left` to process the left
437and right branches of the packet queue.
439Now we assert that the queue is not empty, and we again check that we
440haven't filled the packet window. We will operate on the head of the
441queue. If the packet is live, then do nothing. Otherwise, we go ahead
444To send, we (1) decrement `wid`, our packet window width; (2) cons the
445packet onto the `rub`, which will be returned as the list of packets to
446send; (3) increment `nif`, the number of live packets; (4) set the
447packet to be live; (5) increment the number of transmissions of the
448packet; and (6) set the last sent time of the packet to now.
452 ^+(. =+(lef=apse(puq l.puq) lef(puq [n.puq puq.lef r.puq])))
455 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
457These do exactly what you would expect: they traverse the packet queue
458so that `++apse` gets called recursively through it.
460Finally, `++abet` gets called, which resolves the changes.
464 [(flop rub) +>.$(rtn [~ (add rto now)])]
466This returns the packets that we wish to send, and it updates the
467timeout so that we know when to try resending unacknowledged packets.
469This concludes our discussion of `++whap:pu`. To finish
470`++wind:ho:um:am`, we just need to delve into `++busk:ho:um:am`. But
471wait, in the call to `++busk`, the first argument is `xong:diz`. What is
472this? This, my dear reader, is one more detour, this time into
475 ++ xong :: xong:lax:as:go
476 ^- (list ship) :: route unto
477 =+ [fro=xen too=xeno]
478 =+ ^= oot ^- (list ship)
482 ?: (lien fro |=(a=ship =(a i.too))) ~
484 :: ~& [%xong-to [our her] (weld oot ?>(?=(^ fro) t.fro))]
485 (weld oot ?>(?=(^ fro) t.fro))
487This gets the list of intermediate ships needed to get a packet from us
488to our neighbor. First, we get `fro` and `too`, the "canons" of ourself
489and our neighbor, respectively.
491What is this "canon", you ask? A canon is simply a ship plus its
492"ancestors", as defined by `++sein`. For example, the canon of
495 ~hoclur-bicrel/try=> (saxo ~hoclur-bicrel)
496 ~[~hoclur-bicrel ~tasruc ~tug]
498If we follow the algorithm in `++xong`, we see that we are simply
499creating a list of ships that form a path from our neighbor to ourself.
500Essentially, we look through the canon of our neighbor until we find
501something in our own cannon -- a common ancestor. Or, if we are from
502different carriers, then there is no common ancestor. We then weld this
503onto the tail of our own canon. In the end, this is simply a list of
504possible ships to try to route via to get to our neighbor, ordered by
505preferability (that is, closeness to our neighbor). We will end up
506trying, in order, to find a lane to these.
508Now, we can finally get to `++busk:ho:um:am`.
510 ++ busk :: busk:ho:um:am
511 |= [waz=(list ship) pax=(list rock)] :: send packets
516 $(pax t.pax, bin (weld (flop (wist:diz now waz ~ i.pax)) bin))
519Thankfully, `++busk` is fairly simple. We go through the list of packets
520and convert them to `++boon`s with `++wist:lax:as:go`. These boons are
521placed into `bin`, and they end up getting processed by `++clop` (this
524 ++ wist :: wist:lax:as:go
525 |= $: now=@da :: route via
531 ?: =(our her) [[%ouzo *lane pac] ~]
533 =+ dyr=?:(=(her i.waz) dur (gur i.waz))
538 :_ ?: ?=(%ix -.u.lun.wod.dyr)
541 :+ %ouzo u.lun.wod.dyr
542 ?: &(=(i.waz her) =(~ ryn)) pac
543 =+ mal=(jam `meal`[%fore her ryn pac])
547 ?~ yed.caq.dyr [%none mal]
551 (en:crua q.u.yed.caq.dyr mal)
554This takes a sample of the current time, the list of ships that we just
555generated, a lane if we already know it, and the packet itself.
557First, if we are sending a message to ourself, then we simply create a
558`%ouzo` boon with a bunted lane. Otherwise, if there are no routing
559candidates, there is nothing we can do, so we return nil.
561Next, we get the `dore` of the first routing candidate. If we're looking
562at the neighbor to whom we're trying to send the message, then we simply
563use the `dore` that we already have. Otherwise, we get a default `dore`.
565If we're the first routing candidate, or if we have don't have a lane to
566this candidate, then we skip this candidate and move on to the next one.
568If we have only a provisional ip address, then we try to send on it, but
569we also try to send on later routing candidates as well. Otherwise, we
570only send on this one candidate.
572Finally, we create the actual `%ouzo` boon. The lane is the one from our
573`dore`. If we're sending it directly to our intended recipient, and we
574haven't been told to use a specific lane, then we just send the packet
575directly. Otherwise, we wrap it in a little `%fore` meal, telling the
576intermediary to whom we wish it to be sent. If we have already set up a
577symmetric key with the intermediary, then we encrypt it with that.
578Otherwise, we send it in the clear.
580Now, if you recall, we have traced all the way through from the
581beginning when, in `++knob`, the `%wont` card was handled by a call to
582`++wise`. There is only one more step before the packet is finally sent.
583Looking in `++knob`, we see that the resultant list of boons is passed
584into `++clop`, which will execute the correct actions and return a list
585of moves. In `++clop`, we see the handling of each specific boon. The
586one we are interested in is `%ouzo`, since that is the only one we have
590 :: ~& [%send now p.bon `@p`(mug (shaf %flap q.bon))]
592 [[gad.fox [%give %send p.bon q.bon]] ~]
594Very simply, we give a `%send` gift along the special duct that goes
595straight into the bowels of unix. This is the last stop before we drop
596into vere, and later libuv. And then... the world.
598The packet, after its creation, embarks on a journey across physical
599time and space into the great unknown. Hurtling through fiber-optic
600cables at hundreds of thousands of kilometers per second, it finally
601arrives at our neighbor's network adapter. The adapter tells unix, unix
602tells libuv, libuv tells vere, and vere sends a `%hear` kiss to ames.
603And now we reenter the kernel.
605The `%hear` kiss goes straight to `++knob`, just as did the `%wont` kiss
609 (~(gnaw am [now fox]) %good p.kyz q.kyz)
611Here, though, we call `++gnaw:am` to process the packet. The arguments
612to `++gnaw` are the same as those to the `%hear` kiss: the lane on which
613the packet was received and the packet itself. The other argument is
614just `%good`, which is a `++cape` saying that we expect the packet to
615succeed. If a formal error occurs, then since we have a transactional
616event system, the `%hear` event will never be considered to have
617actually happened, and unix will send a `%hole` kiss so that we may send
618a negative acknowledgment.
621 |= [kay=cape ryn=lane pac=rock] :: process packet
622 ^- [p=(list boon) q=fort]
623 ?. =(2 (end 0 3 pac)) [~ fox]
625 ?: (goop p.p.kec) [~ fox]
626 ?. (~(has by urb.ton.fox) q.p.kec)
630 %- ~(chew la:(ho:(um q.p.kec) p.p.kec) kay ryn %none (shaf %flap pac))
633First, we check the protocol number. If it is not correct, then we
634simply ignore the packet entirely. Otherwise, we parse the packet with
635`++bite`, which converts a packet atom into a `cake`, that is, a triple
636of the `sock` (pair of sender and receiver), the `skin` (encryption
639 ++ bite :: packet to cake
641 =+ [mag=(end 5 1 pac) bod=(rsh 5 1 pac)]
642 =+ :* vez=(end 0 3 mag) :: protocol version
643 chk=(cut 0 [3 20] mag) :: checksum
644 wix=(bex +((cut 0 [23 2] mag))) :: width of receiver
645 vix=(bex +((cut 0 [25 2] mag))) :: width of sender
646 tay=(cut 0 [27 5] mag) :: message type
649 ?> =(chk (end 0 20 (mug bod)))
650 :+ [(end 3 wix bod) (cut 3 [wix vix] bod)]
652 (rsh 3 (add wix vix) bod)
654This is exactly the inverse of `++spit`. Note that here we check both
655the protocol number and the hash, crashing on error. Remember that a
656crash will result in a negative acknowledgment being sent.
658Continuing in `++gnaw`, we see that if the intended recipient is not on
659our pier, then we drop the packet.
661If we've gotten this far, then we wish to process the packet. Recall
662that `++ho` and `++um` set up the domestic server and foreign client
663cores, respectively, and that `++zork` and `++zank` resolve any changes
666The new stuff here, then, is the `++la` core and the `++chew` arm. The
667`++la` sets up a core for this particular packet, containing the current
668success/failure `cape`, the lane it was sent on, the encryption type,
669and a hash of the packet, used as an id.
671`++chew` is called with the encryption type and the message itself. It
672contains a little helper core inside of it, which starts immediately
678 =+ neg==(~ yed.caq.dur.diz)
680 =+ eng==(~ yed.caq.dur.diz)
684 :_(bin [%wine [our her] " is ok"])
687 :_(bin [%wine [our her] " is your neighbor"])
690First, we let `oub` be true if our neighbor hasn't been responding to us
691for more than sixteen seconds. Let `neg` be true if we haven't yet
692proposed a symmetric key, meaning that we haven't yet corresponded with
693this ship, so they are not our neighbor. Next, we run `++east`, which
694we'll go into in just a minute.
696We now do the same two checks and store the results in `eng` and `bou`.
697If our neighbor has, like the prodigal son, returned after an extended
698absense, then we send a `%wine` boon as the proverbial fatted calf,
699which is simply printed out to the console. Likewise, if we are meeting
700one with whom we have never had the pleasure of acquainting ourselves,
701we send a message to the console to that effect.
703We skipped over `++east`, which contains the meat of the processing. It
704first decrypts the message, then calls `++chow:la:ho:um:am` with the
705resultant meal. We'll go through each of the four cases in turn, but
706first since each one calls `++bilk:pu`, we'll take a brief detour.
709 |= now=@da :: inbound packet
715 rtn ?~(puq ~ [~ (add now trt)])
718This updates the timing information in our packet pump. `rue`, the last
719time we have heard from this neighbor, is set to now. `rto`, the
720retransmit timeout is set to twice the current ping time, and if there
721is anything in the packet queue, then we reset the next timeout, since
722we've just heard a message.
727 =. puz (bilk:puz now)
728 (chow ((hard meal) (cue msg)))
730The simplest case is when the encryption type is `%none`. We first call
731`++bilk` to update the packet pump, then we cue (unjam) the message into
732a meal. We hard cast it into a meal -- if the cast fails, then we do
733want to crash since someone is sending us malformed data. Finally, we
734send the result to `++chow` for interpretation and handling.
737 =+ [mag=`hand`(end 7 1 msg) bod=(rsh 7 1 msg)]
738 =+ dey=(kuch:diz mag)
740 ~& [%bad-key her mag]
741 +>.$ :: ignore unknown key
742 =. puz (bilk:puz now)
744 (chow(aut sin) ((hard meal) (cue (dy:q:sen:gus key bod))))
746For symmetric encryption, we first get the `hand`, which is the hash of
747the symmetric key. We pass it to `++kuch:lax:as:go`, which returns the
748key if we either have used it before or we have proposed it. If we have
749proposed it, then we change its status from proposed to real. If
750`++kuch` fails, then we drop the packet and print out a `%bad-key`
753Otherwise, we call `++bilk` as before to update the packet pump and pass
754into `++chow` the decrypted data.
757 =+ mex=((hard ,[p=[p=life q=life] q=will r=@]) (cue msg))
758 =. diz (deng:diz q.mex)
761 =+ gey=(sev:gus p.p.mex)
762 =+ mes=(need (tear:as:q.gey pub:ex:r.wug r.mex))
763 =. diz (wasc:diz p.mes)
764 =. puz (bilk:puz now)
767For sealed asymmetric encryption, we first take off the the layer of
768data that gives us the life and will of our neighbor, and we apply try
769to extend their former will with the new data. `++deng` will fail if
772Next, we get our most current understanding of our neighbor's crypto,
773and we verify that it's the same life as what they're sending. Then, we
774get our own crypto from `++sev` and decrypt the message with the public
775key from our neighbor's crypto. We register the proposed symmetric key,
776update the packet pump, and call `++west`, which simply casts the
777message to a meal and calls `++chow`, reporting any error.
780 =+ mex=((hard ,[p=[~ q=life] q=will r=@]) (cue msg))
781 =. diz (deng:diz q.mex)
784 =+ mes=(need (sure:as:r.wug *code r.mex))
785 =. puz (bilk:puz now)
788Finally, for signed asymmetric encryption, we, as before, take off the
789layer of data that gives us the life and will of our neighbor. This
790time, of course, we do not get our own crypto -- only that of our
793The rest you have seen. We call `++deng` to extend the will, we verify
794that their crypto life is what we think it ought to be, we "decrypt" the
795data, we update the packet pump, and we call `++west` to call `++chow`.
797 ++ chow :: chow:la:ho:um:am
798 |= fud=meal :: interpret meal
800 =. diz ?:(=(%none aut) diz (wast:diz ryn))
803Here, if the message was encrypted at all, then we call
804`++wast:lax:as:go`, which simply updates the lane (route) to our
805neighbor (unless we're given a provisional route). This ensures that we
806always have the most direct possible path to them.
808We've been handling this meal for so long, we've almost forgotten what
809we want to do with it. The telos is of any meal to be dined on. We will
810choose out the cases here that are important to our current
816 ?. ?=(%if -.u.q.fud) u.q.fud
820 (emit %mead lyn r.fud)
821 =+ zid=(myx:gus p.fud)
822 (emir (wist:zid now xong:zid [~ lyn] r.fud))
824Forwarding is the simplest case, since we've seen all the arms before,
825except perhaps `++emit` and `++emir`, which simply take a boon or list
826of boons respectively and queue them up to be handled when the core
827resolves. If we're told to forward a packet to ourselves, then we emit a
828`%mead` boon which simply sends another `%hear` kiss to ourselves with
829the data. Otherwise, we try to find a route to the recipient, as before.
832 =+ zol=(~(get by olz.weg) s.fud)
833 ?^ zol cock(kay u.zol)
835 =+ neb=(~(get by nys.weg) s.fud)
836 ?^ neb [u.neb nys.weg]
837 =+ neb=`bait`[(kins p.fud) 0 r.fud ~]
838 [neb (~(put by nys.weg) s.fud neb)]
839 ?> (lth q.fud p.r.neb)
840 ?> =((kins p.fud) p.neb)
842 =+ doy=`(unit ,@)`(~(get by q.r.neb) q.fud)
845 q.r.neb (~(put by q.r.neb) q.fud t.fud)
848 :: ~& [%carp q.fud s.fud q.neb p.r.neb]
850 =: nys.weg (~(del by nys.weg) s.fud)
851 olz.weg (~(put by olz.weg) s.fud kay)
855 +>.$(nys.weg (~(put by nys.weg) s.fud neb))
857Here, we have received a partial message, and we're just assembling the
858individual packets into a message. Most of this code is fairly
859algorithmic, so we'll just hit the high points. In the beginning, we
860check if we've already received this message, and if so, we resend the
861acknowledgment. Remember, "always ack a dupe, never ack an ack".
863In `nys.weg` we keep track of an incoming set of partial packets,
864indexed by the `flap` hash that comes with every packet. We check to see
865if we have already received this partial message, and if so we
866acknowledge it. Otherwise, we put it in `nys.weg` unless this is the
867last message, in which case we ack the last partial message, move the
868complete message into `olz.weg`, and call `++golf`, which assembles the
869message and calls `++chew`, to start the dance again with the complete
873 :: ~& [%bund q.fud r.fud]
874 ?> =(p:sen:gus p.fud)
875 (deer q.fud r.fud ?-(kay %dead ~, %good [~ s.fud]))
877What if we're just receiving a regular old, garden variety message? We
878call `++deer` with the data from the message. If we already know that
879the message processing will fail (that is, if we got a `%hole` card from
880unix rather than a `%hear` card), then we don't even send the data at
881all. Remember, if a packet fails to process, it's as if it never even
882arrived, except that we send a negative acknowledgment.
884 ++ deer :: deer:la:ho:um:am
885 |= [cha=path num=@ud dut=(unit)] :: interpret message
887 =+ rum=(fall (~(get by raz.bah) cha) *race)
890 ?. (gte num did.rum) :: always ack a dup
891 (cook (~(get by bum.rum) num) cha ~ ryn dam)
893 (coat cha rum(mis (~(put by mis.rum) num [kay ryn dam dut])))
896 %+ ~(put by raz.bah) cha
897 rum(mis (~(put by mis.rum) num [kay ryn dam dut]))
901First, we get the race for this particular triple of sender, receiver,
902and path, creating it if it doesn't exist. If we've already acked the
903message, then we resend the ack. Note that `did.rum` is the number of
904packets we acknowledged, positively or negatively while `bum.rum` is a
905map of message numbers to negative acknowledgments. Thus, if a message
906number is less than `did.rum`, then if it's in `bum.rum` then it was
907negatively acknowledged, otherwise it's postively acknowledged. Thus, we
908are constant in space with the number of successful messages and linear
909in the number of failed messages. We'll document `++cook` later on, but
910suffice it to say that it sends an acknowledgment. It is to end-to-end
911acknowledgments what `++cock` is to packet-level acknowledgments.
913If we are still processing a message (that is, `dod.rum` is false), then
914we simply put this message in the map of misordered packets to be
915processed when their time comes. "Processing a message" in this case
916means that we've received the message and notified the correct
917application, but we're still waiting for the application-level
920Otherwise, we're ready for a packet, so we process it.
922 ++ coat :: coat:ho:um:am
923 |= [cha=path rum=race] :: update input race
925 =+ cun=(~(get by mis.rum) did.rum)
927 +>.$(raz.bah (~(put by raz.bah) cha rum))
928 ?. =(%good p.u.cun) +>.$
931 raz.bah (~(put by raz.bah) cha rum(dod |))
936 `soap`[[p:sen:gus clon:diz] cha did.rum]
940First, we grab the message we want to process and store it in `cun`. If
941it's a good packet, then we change `dod.rum` to false, meaning that
942we're in the middle of processing a packet and should not start
943processing another one. We also put a `%mulk` boon into the queue so
944that, when it all resolves, we send a mesage to the intended recipient
945application. The boon contains the sender, the receiver, the identity of
946the message, and the message itself.
948This bubbles up all the way back to `++knob`, where we were handling the
949`%hear` card. Following the logic in `++knob`, we can see that the boons
950get sent into `++clop` to be turned into actual arvo-level moves. We've
951been here before, if you recall, when we handled the `%cake` boon to
952send a message. Now, we're handling the `%mulk` boon, which is
953unfortunately slightly more complicated.
956 :: ~& [%mulk p.bon q.bon]
957 ?> ?=([@ @ *] q.q.bon)
963 [%sick %wart p.bon i.t.q.q.bon t.t.q.q.bon r.bon]
966 ?> ?=([@ ~] t.t.q.q.bon)
967 =+ app=`term`(need ((sand %tas) i.t.t.q.q.bon))
972 :_ fox [hen %pass pax %g %rote p.bon app r.bon]~
973 %gh :: %gall response
974 ?> ?=([@ ~] t.t.q.q.bon)
975 =+ app=`term`(need ((sand %tas) i.t.t.q.q.bon))
980 :_ fox [hen %pass pax %g %roth p.bon app r.bon]~
983We're dispatching messages based on the prefix of their path. Since only
984`%gall` apps use end-to-end acknowledgments at the moment, every path
985must have at least two elements, and the first one must be `%q`. Beyond
986that, we handle the `/q/ge` and `/q/gh` cases for gall requests and
987responses, respectively.
989In both cases, we require the next term in the path to be the name of
990the intended recipient `%gall` app. Thus, a message to `/q/ge/talk` for
991example, will send a message to the talk app.
993We then send a message to the app itself. The message is either a
994`%rote` or a `%roth` for a request and a response, respectively. The
995content is the `rook` or `roon` that was sent (stored in `r.bon`), but
996we don't actually handle that at all here. That's completely a
997`%gall`-level thing. We're just the messenger.
999Notice the path we send this over. We encode the sender, the receiver,
1000and the path over which it was sent. This fully specifies the `race` so
1001that when the app gives us the acknowledgment we know where to send it.
1003We now have another interlude. We have entrusted our precious data, so
1004carefully guarded and guided from the app on that far-away ship, to our
1005local app. It has the ability to do whatever it pleases with it. It may
1006take a significant amount of time to process. When the message has been
1007handled by this app, though, it must produce an acknowledgment. Our
1008final task is to deliver this acknowledgment to the sending app.
1010We should describe here what exactly these oft-mentioned acknowledgments
1011actually consist of. There are two kinds of acknowledgments: positive
1012and negative. A positive acknowledgment contains no data other than its
1013existence. A negative acknowledgment may optionally include a reason for
1014said negativity. Formally, a negative acknowledgment is an `ares`, which
1015is a unit pair of a term and a list of tanks. If this is null, this is
1016simply a failure with no associated information. If the pair exists, the
1017term is a short error code that is usually both human and computer
1018readable. For example, if you try to send a message to a valid `%gall`
1019app that doesn't have any `++poke` to handle it, then `%gall` will give
1020a negative acknowledgment with error term `%poke-find-fail`. The list of
1021tanks is a human-readable description of the error. This often contains
1022a stack trace. At any rate, all this information is returned to the
1023sending app on the other end of the wire.
1025After this brief interlude, our story resumes in `++knap`, where we
1026receive responses. In particular, a `%mean` indicates a negative
1027acknowledgment while a `%nice` indicates a positive acknowledgment.
1030 ?> ?=([@ @ @ *] tea)
1031 =+ soq=[(slav %p i.tea) (slav %p i.t.tea)]
1035 %^ ~(rack am [now fox]) soq pax
1036 ?-(+<.sih %mean `p.+.sih, %nice ~)
1039 |- ^- [p=(list move) q=_+>.^$]
1042 =^ toe fox (clop now hen i.p.fuy)
1043 $(p.fuy t.p.fuy, out (weld (flop toe) out))
1045Recall the format of the path we sent the message on, and you'll
1046understand why `soq` and `pax` are the sender/receiver pair and path on
1047which the message was sent. The rest of this is structured much like
1048`++knob`, so we call `++rack:am` and send the resulting boons to
1049`++clop`. Business as usual.
1052 |= [soq=sock cha=path cop=coop] :: e2e ack
1053 =+ oh=(ho:(um p.soq) q.soq)
1054 =. oh (cook:oh cop cha ~)
1057First, we set up `++um` and `++ho`, as we've done twice before, for our
1058domestic and foreign servers, respectively. The other two things are
1059new, though. Well, `++cook` is not actually new, but we delayed the
1060explanation saying only that it sends an acknowledgment. The time has
1063 ++ cook :: cook:ho:um:am
1064 |= [cop=coop cha=path ram=(unit ,[ryn=lane dam=flap])]
1065 ^+ +> :: acknowledgment
1066 =+ rum=(need (~(get by raz.bah) cha))
1067 =+ lat=(~(get by mis.rum) did.rum)
1068 ?: &(?=(~ lat) ?=(~ ram)) ~&(%ack-late-or-redundant +>.$)
1069 =+ ^- [ryn=lane dam=flap]
1070 ?^ ram [ryn.u.ram dam.u.ram]
1075 %+ ~(put by raz.bah) cha
1076 rum(dod &, bum ?~(cop bum.rum (~(put by bum.rum) did.rum u.cop)))
1077 =^ roc diz (zuul:diz now [%buck cop dam ~s0])
1078 (busk(diz (wast:diz ryn)) xong:diz roc)
1080If we are acknowledging a message that we have already acked, the `ram`
1081will contain the new lane and flap to send the duplicate ack to. This
1082happens if we call `++cook` in `++deer`, but it doesn't happen from
1083`++rack`. If there is no message waiting to be acknowledged and we're
1084not given an explicit lane and flap (that is, we're not sending a
1085duplicate ack), then the app must have sent us multiple acknowledgments.
1086We do the only sensible thing we can do and drop all acknowledgments
1087after the first, printing a message. This is, in fact, an error, so it
1088could be argued that we ought to crash. Whatever you do, don't depend on
1091First, we grab the race specified by the given path, and we get the most
1092recent in-order message, which must be the one which is being
1095Then, we decide which lane/flap to respond on/to. Basically, in the
1096usual case we respond on the lane through which the initial message was
1097sent, which is stored along with the other packet information in
1098`mis.rum`, since it has to be remembered across calls to ames. However,
1099if we receive a duplicate message, then we must respond to the new
1100message. It's quite possible the reason the other acknowledgment didn't
1101get returned was that the lane between the ships was broken.
1103At any rate, we update the race by saying that we've finished processing
1104this packet (unless we're sending a duplicate ack) and, if we're sending
1105a negative acknowledgment, putting the negative ack into `bum.rum` so
1106that we can resend it if necessary.
1108We encode our new message, updating the packet pump, with `++zuul`, as
1109before, and we send it off with `++busk`, routed via `++wast` to one of
1110the ships in `++xong`. Of course, in practice, we don't even look at the
1111ships in `++xong` because we already have a lane directly to our
1112neighbor (the one over which they sent their message to us).
1114We glossed over the actual message we're sending back. We're sending a
1115`%buck` meal, which is an acknowledgment. The `cop` specifies whether
1116this is a positive or a negative ack, `dam` specifies the message we're
1117acknowledging, and the `~s0` is a placeholder for the processing time
1118required. This time is neither calculated (though it is hopefully
1119obvious how to do so) nor used at present, but this information may be
1120used in the future for improved congestion control. Since the round-trip
1121time for an end-to-end acknowledged packet includes the processing time
1122on the other end, most common congestion control algorithms will stumble
1123when some messages take much longer to process than others. As noted,
1124though, this is simply an opportunity for improvement -- our congestion
1125control algorithms are relatively naive at the moment.
1127Recall that `++busk` calls `++wist` to put the actual `%ouzo` boon in
1128the queue, which gets handled by `++clop` to actually send the message.
1129This is the same pipeline as sending any other message, so we'll refer
1130you to the explanation above if you've forgotten it.
1132The last thing we need to do on this ship is move on to the next packet
1133in the queue if there is one. If you recall, in `++rack` after the call
1134to `++cook` there was a call to `++cans:ho:um:am`.
1136 ++ cans :: cans:ho:um:am
1138 =+ rum=(need (~(get by raz.bah) cha))
1142 mis (~(del by mis.rum) did.rum)
1146This is very simple. We increment the number of packets that we've
1147acknowledged on this race and we delete the packet that we just
1148acknowledged from the set of misordered packets.
1150Then, we call `++coat` again to process the next packet if we've already
1151received it. And that's it for this.
1153The acknowledgment now travels the same path that its forebearer, the
1154original message, once tread, but this time not into the great unknown.
1155The weary traveler is seeking out its familial roots, finding the app
1156from whom sprung forth the original message way back in paragraph three.
1157When it arrives at the network adapter of its ancestors, the adapter
1158tells unix, unix tells libuv, libuv tells vere, and vere sends a `%hear`
1159kiss to ames. Once more into the kernel.
1161The `%hear` kiss is handled in `++knob` as before, leading to `++gnaw`,
1162going over to `++chew`, `++apse`, `++chow`, and eventualy to `++dine`.
1163We've seen most of the cases in `++dine`, but we haven't yet looked at
1164the handling of this `%buck` meal.
1167 =. +> ?.(=(%full aut) +> cock) :: finish key exch
1168 +>(..la (tock p.fud q.fud r.fud))
1170We send a packet level acknowledgment if we're finishing a key exchange,
1171else we call `++tock` to process the acknowledgment.
1173This will get a little involved, so if you don't much care about how
1174exactly an acknowledgment happens, just know that the result gets gifted
1175as a `%woot` card back to the app who sent it. For those brave souls who
1176wish to see this thing through to the end, it's once more into the
1179 ++ tock :: tock:ho:um:am
1180 |= [cop=coop fap=flap cot=@dr] :: e2e ack by hash
1182 =^ yoh puz (bick:puz now fap)
1186 (done p.u.p.yoh q.u.p.yoh)
1191 `boon`[%cake [our her] [[p:sen:gus clon:diz] u.p.yoh] cop u.hud]
1193 (busk xong:diz q.yoh)
1195We're going to work through this one a little backwards since it's
1196mostly fairly simple except the call to `++bick:pu`. In fact, we'll just
1197skip `++bick` for the moment and finish the rest.
1199If `++bick` succesfully acks the message, then we call `++done`.
1201 ++ done :: done:ho:um:am
1202 |= [cha=path num=@ud] :: complete outgoing
1203 ^- [(unit duct) _+>]
1204 =+ rol=(need (~(get by ryl.bah) cha))
1205 =+ rix=(~(get by san.rol) num)
1210 (~(put by ryl.bah) cha rol(san (~(del by san.rol) num)))
1213This very simply gets the rill (the outgoing counterpart to a race, if
1214you recall), pulls out of the map of outstanding messages the duct over
1215which the original message was sent, and produces this duct while
1216deleting that entry from the map of outstanding messages.
1218Going back to `++tock`, we now have the duct we need to return the
1219result over. We do the very sensible thing and put a `%cake` boon in the
1220queue to be processed later by `++clop`.
1222In `q.yoh` we have a list of messages that may need to be sent, which we
1223pass to `++busk` to send, as usual. When an acknowledgment arrives, that
1224may trigger other messages immediately. This often happens when sending
1225more messages than the width of the logical window since for congestion
1226control reasons another message cannot be sent until some of the earlier
1227ones have been acknowledged.
1229We'll look at the processing of the `%cake` boon in `++clop` before we
1230get back to talking about `++bick`.
1234 :~ [s.bon %give %woot q.p.bon r.bon]
1237We very simply give, along the duct we found above, a `%woot` card with
1238the ship who sent us the ack and the ack itself. This allows the
1239application to decide what to do about the result. In case of a failure,
1240we usually either resend the message or display it to the user.
1241Sometimes, we recognize the error term and handle it internally. In any
1242case, the decision of how to handle the acknowledgment is entirely up to
1243the application. Our job is done.
1245Well, except that we skipped `++bick:pu`. Let's go back to that.
1248 |= [now=@da fap=flap] :: ack by hash
1249 ^- [[p=(unit soup) q=(list rock)] _+>]
1250 =+ sun=(~(get by diq) fap)
1253 =. diq (~(del by diq) fap)
1254 =^ gub +>.$ (bock now u.sun)
1255 =^ yop +>.$ (harv now)
1258If you recall, in `++whap:pu` we created the packet pump's
1259representation of the message, which included putting the message into
1260`diq`, which maps from packet hashes to packet sequence numbers. Thus,
1261`u.sun` is the sequence number of this particular message.
1263We delete this message from `diq` since we have now received an ack for
1264it. We call `++bock` to perform the ack by sequence number. We call
1265`++harv` to harvest the packet queue, sending any messages that are now
1268In `++bock`, there are three arms we haven't seen before: `++bine`,
1269`+wept`, and `++beet`. We'll describe each of these before we get to
1270`++bock`. `++bine` looks scariest.
1273 |= [now=@da num=@ud] :: apply ack
1274 ^- [(unit soup) _+>]
1277 ?: (gth num p.n.puq)
1279 [-.lef +.lef(puq [n.puq puq.lef r.puq])]
1281 [-.rig +.rig(puq [n.puq l.puq puq.rig])]
1282 =: rtt ?. &(liv.q.n.puq =(1 nux.q.n.puq)) rtt
1283 =+ gap=(sub now lys.q.n.puq)
1284 :: ~& [%bock-trip num (div gap (div ~s1 1.000))]
1285 (div (add (mul 2 rtt) gap) 3)
1286 nif (sub nif !liv.q.n.puq)
1288 =+ lez=(dec (need (~(get by pyz) gom.q.n.puq)))
1291 [[~ gom.q.n.puq] (~(del by pyz) gom.q.n.puq)]
1292 [~ (~(put by pyz) gom.q.n.puq lez)]
1294 +>.$(puq ~(nap to puq))
1296The first few lines are simply looking through the packet queue until we
1297find the correct packet to ack. This is basic queue manipulation that
1298operates directly on the treap structure of the queue. If you understand
1299treap queues, the logic is easy to follow. Otherwise, just trust us that
1300by the time we get to the `=:`, the packet with sequence number `num` is
1301on the top of the packet queue (that is, at `n.puq`).
1303We first update the round-trip time. If the packet is either not alive
1304or had to be transmitted more than once, then we don't have any reliable
1305way of calculating the round-trip time since we're unsure of exactly
1306which transmission was acknowledged. Otherwise, the round-trip time is
1307the difference between now and when the packet was last sent. We set
1308`rtt` by a little weighted average where the previous smoothed RTT is
1309weighted twice as much as the RTT of the current packet. Thus,
1310`(2*rtt+gap)/3`. This gives us a nice smooth RTT that is somewhat
1311resilient to outlier data while still being responsive to our
1314If the packet wasn't already dead, then we decrement the number of live
1315packets, which may allow more packets to be sent.
1317We decrement the number of unacknowledged packets in our `pyz` for this
1318particular message. If you recall, this was set in `++whap` to the
1319number of packets required to send a message.
1321If that was the last packet in the messge that needed to be acked, then
1322we delete the messgae reference from `pyz` and produce the id of the
1323message. Otherwise, we simply update `pyz` with the new number of
1324unacked messages. In either case, we remove the packet from the packet
1328 |= [fip=@ud lap=@ud] :: fip thru lap-1
1335 ?: (lth p.n.puq fip) ?~(l.puq . left)
1336 ?: (gte p.n.puq lap) ?~(r.puq . rigt)
1339 ?.(liv.q.n.puq . .(nif (dec nif), liv.q.n.puq |))
1343 ^+(. =+(lef=apse(puq l.puq) lef(puq [n.puq puq.lef r.puq])))
1346 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
1349The algorithm is a simple case of traversing the packet queue.
1350Essentialy, we mark as dead all packets in the queue between `fip` and
1351`(dec lap)`. We also update `nif`, the number of live packets. Lest you
1352mourn too much the passing of these packets, know that they shall soon
1353rise again. Recall that in `++bick` after the call to `++bock` we call
1354`++harv`. This will resend the packets that have just been labeled dead.
1357 ^+ . :: advance unacked
1358 =- +(nep ?~(foh nus u.foh))
1362 ?: (lte p.n.puq nep) $(puq l.puq)
1364 ?^(rig rig [~ p.n.puq])
1366Here we search for the next expected packet number. Basically, we search
1367the queue for the leftmost packet whose number is greater than the
1368current `nep`. If we don't find any such packet, we just use the total
1369number of packets sent.
1371We can now dive into `++bock`, our last arm.
1374 |= [now=@da num=@ud] :: ack by sequence
1375 ^- [(unit soup) _+>]
1376 =^ gym +> (bine now num)
1379 =+ cam=(max 2 (div caw 2))
1380 :: ~& [%bock-hole num nep cam]
1381 beet:(wept(nep num, cag cam, caw cam) nep num)
1382 =. caw ?: (lth caw cag) +(caw)
1383 (add caw !=(0 (mod (mug now) caw)))
1385 :: ~& [%bock-fine num nif caw cag]
1387 :: ~& [%bock-fill num nif caw cag]
1390First, we call `++bine` to apply the ack to the packet pump information.
1391We produce `gym`, which, if it exists, is the id of the packet that was
1392acked. If we received an ack for a packet later than the one we
1393expected, then we halve the logical packet window and kill all the
1394earlier packets so that they may be resent.
1396Otherwise, we possibly increase the congestion window. If the window is
1397less than the congestion threshold, then we increment the size of the
1398window. Otherwise, we only increment one out of every `caw` times.
1400If we received an ack for the packet we expected, then we simply advance
1401`nep` with `++beet`. If we received an ack for a packet earlier than we
1402expected, we do nothing.
1404It may be hard to believe, but we are, in fact, done. The message has
1405been sent, received, acknowledged, and the acknowledgment has been
1406returned to the original sender. We hope it's clear that, while the
1407process has been somewhat involved, the algorithms are not all that
1408complicated. If you've read this far, you know `%ames`. The only other
1409code involves initialization, timeouts, and the like.
1411Below, we give detailed reference documentation for the data models
1417### `++fort`, formal state
1419 ++ fort :: formal state
1421 gad=duct :: client interface
1422 hop=@da :: network boot date
1423 ton=town :: security
1424 zac=(map ship corn) :: flows by server
1427This is the state of our vane. Anything that must be remembered between
1428calls to ames must be stored in this state.
1430`%0` is the version of the ames state model itself. If the data model
1431`++fort` changes, then this number needs to be incremented, and an
1432adapter must be written to upgrade the old state into the new state.
1433Note that this is the version number of the model itself, not the
1434contents. When the data changes, there is of course no need to change
1437`gad` is a `duct` over which we send `%send` cards to unix. This card is
1438initialized when unix sends a `%barn` card as vere starts up. Vere
1439treats this duct specially -- don't send anything weird over it.
1441`hop` is the network boot date. This is set when the `%kick` card is
1442sent by vere on start up.
1444`ton` is a `++town`, where we store all of our security/encryption
1445state. Note that this is shared across all ships on a pier.
1447`zac` is a map of ships to `++corn`. This stores all the per-ship state.
1448The keys to this map are the ships on the current pier.
1450### `++town`, all security state
1452 ++ town :: all security state
1453 $: lit=@ud :: imperial modulus
1455 urb=(map ship sufi) :: all keys and routes
1459This is the security state of our pier.
1463`any` is 256 bits of entropy. This entropy is used and updated in
1464exactly two places: when we send a `%junk` card, and when we generate a
1465new symmetric key in `++griz:lax:as:go`. When it is updated, it is
1466updated by a SHA-256 hash of the current time and the old value of the
1469`urb` is a map of ships to `++sufi`. This is where we store all the
1470per-ship state for the pier. The keys to this map are the ships on the
1473`fak` is true if we are on a fake network. This disables certain
1474security checks so that anyone may run a fake `~zod`. This is used only
1475for development. To use, run vere with the `-F` option (and the
1476`-I ~zod` option for a fake `~zod`).
1478### `++sufi`, domestic host
1480 ++ sufi :: domestic host
1481 $: hoy=(list ship) :: hierarchy
1482 val=wund :: private keys
1483 law=will :: server will
1484 seh=(map hand ,[p=ship q=@da]) :: key cache
1485 hoc=(map ship dore) :: neighborhood
1488This is the security state of a domestic server.
1490`hoy` is a list of the ships directly above us in the hierarchy of
1491ships. For example, for `~hoclur-bicrel`, this would be `~tasruc` and
1492`~tug`. See `++sein`.
1494`val` is a list of our private keys.
1496`law` is our certificate, which is a list of the XXX
1500`hoc` is a map of ships to `++dore`. The stores all the security
1501informatoin about foreign ships. The keys to this map are the neighbors
1502(ships we have been in contact with) of this domestic server.
1504### `++wund`, private keys
1506 ++ wund (list ,[p=life q=ring r=acru]) :: mace in action
1508This is a list of our own private keys, indexed by life. The key itself
1509is the `++ring`, and the `++acru` is the encryption engine. We generate
1510the `++acru` from the private key by calling `++weur`. Thus, we can at
1511any time regenerate our `++wund` from a `++mace`. The current crypto is
1512at the head of the list and can be accessed with `++sen:as:go`.
1514### `++ring`, private key
1516 ++ ring ,@ :: private key
1518This is a private key. The first byte is reserved to identify the type
1519of cryptography. Lower-case means public key, upper-case means public
1520key, and the letter identifies which `++acru` to use.
1522### `++pass`, public key
1524 ++ pass ,@ :: public key
1526This is a public key. The first byte is reserved to identify the type of
1527cryptography. Lower-case means public key, upper-case means public key,
1528and the letter identifies which `++acru` to use.
1530### `++mace`, private secrets
1532 ++ mace (list ,[p=life q=ring]) :: private secrets
1534This is a list of the our private keys, indexed by life. From this we
1535can generate a `++wund` for actual use.
1537### `++skin`, encoding stem
1539 ++ skin ?(%none %open %fast %full) :: encoding stem
1541This defines the type of encryption used for each message. `%none`
1542refers to messages sent in the clear, `%open` refers to signed messages,
1543`%full` refers to sealed messages, and `%fast` refers to symmetrically
1544encrypted messages. See `++acru` for details.
1546### `++acru`, asymmetric cryptosuite
1548 ++ acru :: asym cryptosuite
1549 $_ ^? |% :: opaque object
1550 ++ as ^? :: asym ops
1551 |% ++ seal |=([a=pass b=@ c=@] _@) :: encrypt to a
1552 ++ sign |=([a=@ b=@] _@) :: certify as us
1553 ++ sure |=([a=@ b=@] *(unit ,@)) :: authenticate from us
1554 ++ tear |= [a=pass b=@] :: accept from a
1555 *(unit ,[p=@ q=@]) ::
1557 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1558 ++ dy |+([a=@ b=@] _@) :: symmetric de, hard
1559 ++ en |+([a=@ b=@] _@) :: symmetric en
1561 |% ++ fig _@uvH :: fingerprint
1562 ++ pac _@uvG :: default passcode
1563 ++ pub *pass :: public key
1564 ++ sec *ring :: private key
1566 ++ nu ^? :: reconstructors
1567 |% ++ pit |=([a=@ b=@] ^?(..nu)) :: from [width seed]
1568 ++ nol |=(a=@ ^?(..nu)) :: from naked ring
1569 ++ com |=(a=@ ^?(..nu)) :: from naked pass
1573This is an opaque interface for a general asymmetric cryptosuite. Any
1574form of asymmetric cryptography can be dropped in to be used instead of
1575the default. Right now, there are two cryptosuites, `++crua`, which is
1576your standard RSA, and `++crub`, which is elliptic curve crypto but is
1577mostly stubbed out at the moment.
1579#### `++as:acru`, asymmetric operations
1581 ++ as ^? :: asym ops
1582 |% ++ seal |=([a=pass b=@ c=@] _@) :: encrypt to a
1583 ++ sign |=([a=@ b=@] _@) :: certify as us
1584 ++ sure |=([a=@ b=@] *(unit ,@)) :: authenticate from us
1585 ++ tear |= [a=pass b=@] :: accept from a
1586 *(unit ,[p=@ q=@]) ::
1589This is the core that defines the standard asymmetric cryptography
1592`++seal:as:acru` allows us to send a message encrypted with someone's
1593public key so that only they may read it. If Alice seals a message with
1594Bob's public key, then she can be sure that Bob is the only one who can
1595read it. This is associated with the `++skin` `%full`.
1597`++sign:as:acru` allows us to sign a message with our private key so
1598that others can verify that we sent the message. If Alice signs a
1599message with her private key, then Bob can verify with her public key
1600that it was indeed Alice who sent it. This is associated with the
1603`++sure:as:acru` is the dual to `++sign:as:acru`. It allows us to verify
1604that a message we have received is indeed from the claimed sender. If
1605Alice sends a message with her private key, then Bob can use this arm to
1606verify that it was indeed Alice who sent it. This is associated with the
1609`++tear:as:acru` is the dual to `++seal:as:acru`. It allows us to read a
1610message that we can be sure is only read by us. If Alice seals a message
1611with Bob's public key, then Bob can use this arm to read it. This is
1612associated with the `++skin` `%full`.
1614#### `++de:acru`, `++dy:acru`, and `++en:acru`, symmetric encryption/decryption
1616 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1617 ++ dy |+([a=@ b=@] _@) :: symmetric de, hard
1618 ++ en |+([a=@ b=@] _@) :: symmetric en
1620Symmetric encryption is associated with the `++skin` `%fast`.
1622`++de:acru` decrypts a message with a symmetric key, returning `~` on
1623failure and `[~ u=data]` on success.
1625`++dy:acru` decrypts a message with a symmetric key, crashing on
1626failure. This should almost always be defined as, and should always be
1627semantically equivalent to, `(need (de a b))`.
1629`++en:acru` encrypts a message with a symmetric key.
1631#### `++ex:acru`, exporting data
1634 |% ++ fig _@uvH :: fingerprint
1635 ++ pac _@uvG :: default passcode
1636 ++ pub *pass :: public key
1637 ++ sec *ring :: private key
1640`++fig:ex:acru` is our fingerprint, usually a hash of our public key.
1641This is used, for example, in `++zeno`, where every carrier owner's
1642fingerprint is stored so that we can ensure that carriers are indeed
1643owned by their owners
1645`++pac:ex:acru` is our default passcode, which is unused at present.
1647`++pub:ex:acru` is the `++pass` form of our public key.
1649`++sec:ex:acru` is the `++ring` form of our private key.
1651#### `++nu:acru`, reconstructors
1653 ++ nu ^? :: reconstructors
1654 |% ++ pit |=([a=@ b=@] ^?(..nu)) :: from [width seed]
1655 ++ nol |=(a=@ ^?(..nu)) :: from naked ring
1656 ++ com |=(a=@ ^?(..nu)) :: from naked pass
1659These arms allow us to reconstruct a `++acru` from basic data.
1661`++pit:nu:acru` constructs a `++acru` from the width of our intended key
1662and seed entropy. This is usually used in the initial construction of
1665`++nol:nu:acru` constructs a `++acru` from a "naked ring", meaning a
1666`++ring` without the initial byte identifying the type of crypto. There
1667is often a helper arm that that wraps this; see `++weur` for `++crua`
1668and `++wear` for `++crub`.
1670`++com:nu:acru` constructs a `++acru` from a "naked pass", meaning a
1671`++ring` without the initial byte identifying the type of crypto. There
1672is often a helper arm that that wraps this; see `++haul` for `++crua`
1673and `++hail` for `++crub`.
1675### `++will`, certificate
1677 ++ will (list deed) :: certificate
1679This is a list of deeds associated with the current ship. There should
1680be an item in this list for every ship from this point up in the
1681hierarchy times the number of lives that each ship has had. For example,
1682\~hoclur-bicrel may have a will with three items: one for itself, one
1683for \~tasruc (who issued \~hoclur-bicrel's deed) and one for \~tug (who
1684issued \~tasruc's deed).
1686### `++deed`, identity
1688 ++ deed ,[p=@ q=step r=?] :: sig, stage, fake?
1690`p` is the signature of a particular deed, which is a signed copy of
1693`q` is the stage in the identity.
1695`r` is true if we're working on a fake network, where we don't check
1696that the carrier fingerprints are correct. This allows us to create fake
1697networks for development without interfering with the real network.
1699### `++step`, identity stage
1701 ++ step ,[p=bray q=gens r=pass] :: identity stage
1703This is a single stage in our identity. Thus, this is specific to a
1704single life in a single ship. Everything in here may change between
1711`r` is the public key for this stage in the identity.
1715 ++ bray ,[p=life q=(unit life) r=ship s=@da] :: our parent us now
1719### `++gens`, general identity
1721 ++ gens ,[p=lang q=gcos] :: general identity
1723`p` is the IETF language code for the preferred language of this
1724identity. This is unused at the moment, but in the future text should be
1725localized based on this.
1727`q` is the description of the ship.
1729### `++gcos`, identity description
1731 ++ gcos :: id description
1732 $% [%czar ~] :: 8-bit ship
1733 [%duke p=what] :: 32-bit ship
1734 [%earl p=@t] :: 64-bit ship
1735 [%king p=@t] :: 16-bit ship
1736 [%pawn p=(unit ,@t)] :: 128-bit ship
1739This is the description of the identity of a ship. Most types of
1740identity have a `@t` field, which is their human-readable name. The
1741identity of a `%duke` is more involved.
1743A `%czar`, a carrier, is a ship with an 8-bit address. Thus, there are
1744only 256 carriers. These are at the top of the namespace hierarchy, and
1745the fingerprint of each carrier is stored in `++zeno`. These are the
1748A `%king`, a cruiser, is a ship with a 16-bit address. Thus, there are
174965,536 cruisers. Each carrier may issue 256 cruisers. These are the
1750infrastructure of Urbit.
1752A `%duke`, a destroyer, is a ship with a 32-bit address. Thus, there are
17534,294,967,296 destroyers. Each cruiser may issue 65,536 cruisers. These
1754are the individuals of Urbit.
1756A `%earl`, a yacht, is a ship with a 64-bit address. Thus, there are
175718,446,744,073,709,551,616 yachts. Each destroyer may issue
17584,294,967,296 yachts. These are the devices of Urbit.
1760A `%pawn`, a submarine, is a ship with a 128-bit address. Thus, there
1761are a lot of submarines. The chance of random name collision is
1762negligible, so submarines are not issued by any ship. They must simply
1763assert their presence, and they are all considered children of \~zod.
1764This is the underworld of Urbit, where anonymity reigns supreme.
1766### `++what`, logical destroyer identity
1768 ++ what :: logical identity
1769 $% [%anon ~] :: anonymous
1770 [%lady p=whom] :: female person ()
1771 [%lord p=whom] :: male person []
1772 [%punk p=sect q=@t] :: opaque handle ""
1775This is the logical identity of a destroyer.
1777A `%anon` is a completely anonymous destroyer. The difference between
1778this and a submarine is that a submarine is ephemeral while a `%anon`
1779destroyer is not. Thus, we may not know who \~hoclur-bicrel is, but we
1780do know that it's always the same person.
1782A `%lady` is a female person. The name used here should be a real name.
1784A `%lord` is a male person. The name used here should be a real name.
1786A `%punk` is a person who is identified only by a handle.
1788### `++whom`, real person
1790 ++ whom ,[p=@ud q=govt r=sect s=name] :: year/govt/id
1792Ths is the information associated with a real person. It is mostly
1793information that could be observed with the briefest of interactions.
1795`p` is the birth year.
1797`q` is the location of a user, usually of the form "country/zip".
1799`r` is the sect of the user.
1801`s` is the real name of the person.
1805 ++ govt path :: country/postcode
1807This is the location of the user, usually of the form "country/zip".
1811 ++ sect ?(%black %blue %red %orange %white) :: banner
1817 ++ name ,[p=@t q=(unit ,@t) r=(unit ,@t) s=@t] :: first mid/nick last
1819This is the given name, possible middle name/initial, possible nickname,
1820and surname of a user.
keysesc to close
- j k
- move down, up
- enter
- open
- esc
- back, or up a level
- 1 2 3 4
- readme, files, commits, refs
- /
- filter
- y
- copy the clone command
- c
- compare the chosen commit with another (or shift-click both)
- ?
- show or hide this list