5aliases = ["/docs/learn/arvo/ames/"]
7Our networking protocol.
9`%ames` is the name of both our network and the vane that communicates
10over it. When Unix receives a packet over the correct UDP port, it pipes
11it straight into `%ames` for handling. Also, all packets sent over the
12`%ames` network are sent by the `%ames` vane. Apps and vanes may use
13`%ames` to directly send messages to other ships. In general, apps use
14gall and clay to communicate with other ships rather than using `%ames`
15directly, but this isn't a requirement. Of course, gall and clay use
16`%ames` behind the scenes to communicate across the network. These are
17the only two vanes that use `%ames`.
19`%ames` includes several significant components. Although the actual
20crypto algorithms are defined in zuse, they're used extensively in
21`%ames` for encrypting and decrypting packets. Congestion control and
22routing is handled entirely in `%ames`. Finally, the actual `%ames`
23protocol itself, including how to route incoming packets to the correct
24vane or app, is defined in `%ames`.
28`%ames` is our networking protocol.
30First we give commentary on the code, the algorithms involved, and the
31protocol. We trace through the code touched when a packet is sent,
32received, acknowledged, and that acknowledgment applied. This is fairly
33comprehensive, and contains many implementation details, but if you
34understand this, then you understand `%ames`.
36If you've scrolled down this page, you may be intimidated by the amount
37of Hoon code, especially if you are new to the language. Don't be afraid
38of it, you don't have to read any of it if you don't want to -- every
39interesting action the code takes is explained in plain English. In
40fact, if you are new to the language, this may be a good learning
41opportunity. Even if you don't understand every line of Hoon code,
42you'll hopefully be able to follow most lines. By the time you've worked
43through this, you'll have seen many common patterns and best practices.
44Hoon, much more than other languages, is best learned by reading and
45understanding large quantities of existing code. In this way, it is
46similar to learning a natural language. All of this code is in
49After the commentary, we have reference documentation for all the data
50structures that are specific to `%ames`. If you see a data structure or
51a variable used that you don't recognize, search for it in the code, and
52it's very likely defined in one of these data structures. We recommend
53that another tab is kept open for easy access to the data structure
54reference documentation. The code for these is split between
55`arvo/ames.hoon` and `arvo/zuse.hoon`.
57The Lifecycle of a Packet (or, How a Packet Becomes Law)
61Here, we will trace a packet as it makes its way through ames. There are
62actually two pathways through ames: the legacy path through `%want`, and
63the modern way, entered through `%wont`, with full end-to-end
64acknowledgments. Here we will only trace the modern way, though much of
65the path is the same for both.
67When an app (or a vane) wishes to send a packet to another ship, it must
71 [%wont p=sock q=path r=*] :: e2e send message
74This card takes three arguments. The `p` is a `sock`, that is, a pair of
75two ships, the first of which is the sender and the second is the
76receiver. But wait, you ask, why do I get to decide who is the sender?
77Can I fake like I'm someone else? The reason is that there are
78potentially multiple ships on the same pier, and the kernel can send a
79message from any of them. If you attempt to send a message from a ship
80not on your pier, then ames will refuse to send it. If you hack around
81in your own copy of ames to go ahead and send it anyway, then the other
82ship will reject it because your key is bad. Only send messages from
85The `q` is a path, representing the place on the other side that you
86want to receive your message. It is approximately equivalent to a port
87number. Messages on the same path are guaranteed to arrive in the same
88order as they were sent. No such guarantees are made across paths.
90The `r` is the actual data that you are sending. As the type implies,
91this can be an arbitrary noun, and it will be transferred to the
92receiver exactly as-is, in a well-typed way. Of course, this is data
93that is sent over the wire, so be careful not to send anything too
94massive unless you're willing to wait.
96But enough about the interface. Grepping in ames.hoon for `%wont`, we
97find that it appears in `++knob`. We see that we go
98directly into `++wise:am`.
102 |= [soq=sock hen=duct cha=path val=* ete=?] :: send a statement
103 ^- [p=(list boon) q=fort]
104 zork:zank:(wool:(ho:(um p.soq) q.soq) hen cha val ete)
107The inputs to this gate are exactly the sort of thing you'd expect. In
108particular, everything in the `%wont` gate is here plus the calling duct
109so that we know where to send the acknowledgment and `ete` to determine
110if we're going to do the modern end-to-end acknowledgments.
112The actual line of code looks intimidating, but it's really not all that
113bad. Working from the inside out, the call to `++um` sets up our
114domestic server, and the call to `++ho` sets up our knowledge about the
115neighbor we're sending to. From the outside, `++zork` and `++zank` just
116apply the changes made to our `++um` and `++am` cores, respectively. If
117you're familiar with the common idiom of `++abet`, that's all this is.
118The code predates the widespread usage of that name.
120The interesting part, then, is in `++wool:ho:um:am`. Let's look at the
124 ++ wool :: wool:ho:um:am
125 |= [hen=duct cha=path val=* ete=?] :: send a statement
128 =+ rol=(~(get by ryl.bah) cha)
131 :: ~& [%tx [our her] cha sex]
133 %+ ~(put by ryl.bah) cha
134 rol(sed +(sed.rol), san (~(put by san.rol) sex hen))
135 =+ cov=[p=p:sen:gus q=clon:diz]
138 [%bund q.cov cha sex val]
139 [%bond q.cov cha sex val]
142This is slightly more complicated, but it's still not all that bad. Our
143inputs, at least, are fairly obvious.
145If you glance at the code for a second, you'll see that
146`++wind:ho:um:am` seems to be able to send a message, or `++meal:ames`, given
147a `++soup`. This gate, then, just sets up the things we need to for
148`++wind` to do its job.
150We first get `rol`, which is a `++rill:ames`, that is, a particular outbound
151stream. This stream is specific to the path on which we're sending. If
152the path hasn't been used before, then we create it. We let `sex` be the
153number of messages we've already sent on this path.
155Then, we update the outbound stream by incrementing the number of
156messages sent and placing an entry in `san.rol` that associates the
157message number with the `duct` that sent the message. This allows us to
158give the acknowledgment to the one who sent the message.
160We let `cov` be the current life of our crypto and our neighbor's
161crypto. At the moment, we only need our neighbor's life, which we put
164Finally, we call `++wind:ho:um:am` with the `++soup` of the path and
165message number and the `++meal:ames` of the payload itself. For end-to-end
166acknowledged messages, we use `%bund`.
169 [%bund p=life q=path r=@ud s=*] :: e2e message
172Looking at how we create the `%bund`, we can easily see what each field
175Following the trail a little further, we go to `++wind:ho:um:am`.
178 ++ wind :: wind:ho:um:am
179 |= [gom=soup ham=meal]
180 :: ~& [%wind her gom]
182 =^ wyv diz (zuul:diz now ham)
183 =^ feh puz (whap:puz now gom wyv)
187`++wind` does three things: it (1) encodes the message into a list of
188possibly-encrypted packets, (2) puts the message into the packet pump,
189and (3) sends any packets that are ready to be sent. Yes, our nice
190little linear run of each gate calling exactly one other interesting
191gate is over. We'll go in order here.
193`++zuul:lax:as:go` is the what converts a `++meal:ames` into a list of
197 ++ zuul :: zuul:lax:as:go
198 |= [now=@da ham=meal] :: encode message
199 ^- [p=(list rock) q=_+>]
201 ++ wasp :: null security
202 ++ weft :: fragment message
203 ++ wisp :: generate message
206For organizational purposes, `++zuul` constructs an internal core with
207three arms. `++wasp` encodes the meal into an atom with no encryption.
208`++wisp` encodes a meal with possible encryption (else it simply calls
209`++wasp`). `++weft` takes the result of `++wisp` and splits it into
213 ++ wasp :: null security
214 ^-([p=skin q=@] [%none (jam ham)])
217This simply jams the meal, wrapping it with the `skin` of `%none`,
218meaning no encryption.
220Since `++wisp` is a little long, we'll go through it line-by-line.
223 ++ wisp :: generate message
224 ^- [[p=skin q=@] q=_..wisp]
227`++wisp` produces a pair of a `skin` and an atom, which is the meal
228encoded as a single atom and possibly encrypted.
235If the meal that we're encoding is a `%carp`, then we don't encrypt it.
236A `%carp` meal is a partial meal, used when a message is more than 1KB.
237Since the entire message is already encrypted, we don't need to encrypt
238each packet individually again.
247 (en:r:cluy q.u.yed.caq.dur (jam ham))
250If we have a symmetric key set up with this neighbor, then we simply use
251it. The skin `%fast` is used to indicate a symmetric key.
254 ?: &(=(~ lew.wod.dur) |(=(%back -.ham) =(%buck -.ham)))
258If we do not yet have our neighbor's will, then there is no way that we
259can seal the message so that only they may read it. If what we're
260sending is an acknowledgment, then we go ahead and just send it in the
265 ?:(=(~ lew.wod.dur) [*code +>.$] (griz now))
268If we don't have our neighbor's will, then we "encrypt" with a key of 0.
269If we do have their will, then we generate a new symmetric key that we
275 =+ bil=law.saf :: XX send whole will
279`yig` will be the life and engine for our current crypto. `bil` is our
280will. `hom` is the meal encoded as a single atom.
288 (sign:as:q.yig tuy hom)
291If we do not have our neighbor's will, then we send our current life
292along with our will and the message. The message itself is "signed" with
299 [`life`p.cay `life`p.yig]
301 (seal:as:q.yig pub:ex:r.cay tuy hom)
302 -- :: --zuul:lax:as:go
305If we do have our neighbor's will, then we send our perception of their
306current life, our current life, our will, and the message. The message
307is sealed with their public key so that only they can read our message.
309Once we have the message encoded as an atom, `++weft` goes to work.
312 ++ weft :: fragment message
313 ^- [p=(list rock) q=_+>.$]
319We're going to produce a list of the packets to send. First, we use the
320aforementioned `++wisp` to get the message as an atom.
323 =+ wit=(met 13 q.gim)
327`wit` is the number of 1KB (2\^13 bit) blocks in the message. We assert
328that there is at least one block.
332 =+ yup=(spit [our her] p.gim q.gim)
336If there is exactly one block, then we just call `++spit` to turn the
337message into a packet. We'll explain what `++spit` does momentarily.
340 =+ ruv=(rip 13 q.gim)
341 =+ gom=(shaf %thug q.gim)
345If there is more than one block, then we rip it into blocks in `ruv`.
346`gom` is a hash of the message, used as an id. `inx` is the number of
347packets we've already made.
355 wasp(ham [%carp (ksin p.gim) inx wit gom i.ruv])
357 $(ruv t.ruv, inx +(inx))
360Here we package each block into a packet with `++spit` and produce the
364 ++ spit :: cake to packet
366 =+ wim=(met 3 p.p.kec)
367 =+ dum=(met 3 q.p.kec)
368 =+ yax=?:((lte wim 2) 0 ?:((lte wim 4) 1 ?:((lte wim 8) 2 3)))
369 =+ qax=?:((lte dum 2) 0 ?:((lte dum 4) 1 ?:((lte dum 8) 2 3)))
372 =+ bod=:(mix p.p.kec (lsh 3 wix q.p.kec) (lsh 3 (add wix vix) r.kec))
385This is how we turn a message into a real packet. This has the
386definition of the packet format.
388`wim` is the length of the sending ship, and `dum` is the length of the
389receiving ship. There are only five possibilities for each of those,
390corresponding to carriers, cruisers, destroyers, yachts, and submarines.
391These are encoded in `yax` and `qax` as 0, 0, 1, 2, and 3, respectively.
392Thus, `wix` and `vix` are the number of bytes that must be reserved for
393the ship names in a packet.
395Next, we construct `bod` by simply concatenating the sending ship, the
396receiving ship, and the body of the message. Then, we get the encryption
397mechanism from `++skin:ames`, which may be a 0, 1, 2, or 3, and put it in
400Next, we concatenate together, bit by bit, some final metadata. We use
401three bits for our protocol number, which is incremented modulo eight
402when there is a continuity breach or the protocol changes. We use the
403final twenty bits of a hash of the body (which, we suppose, makes it a
404twenty bit hash) for error-checking. We use two bits to tell how much
405room is used in the body for the sending ship, and another two bits for
406the receiving ship. Finally, we use five bits to store the encryption
407type. Note that since there are only two bits worth of encryption types,
408there are three unused bits here. This adds up to 32 bits of header
409data. Finally, we concatenate this onto the front of the packet. Thus,
410we can summarize the packet header format as follows.
414 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
415 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
416 |Proto| Hash of Body |yax|qax| Crypto |
417 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
420After this, there are `yax` bits of the sender name, `qax` bits of the
421receiver name, and up to 8192 bits of data. Thus, the maximum size of a
422packet is achieved in a message between two submarines with 8192 bits of
423data. This will require 32+128+128+8192 = 8480 bits, or 1060 bytes.
425This concludes our discussion of `++zuul:lax:as:go`. If you recall from
426`++wind:ho:um:am`, the list of packets from `++zuul` is passed into
427`++whap:pu` to update the packet pump and get any packets that can be
432 |= [now=@da gom=soup wyv=(list rock)] :: send a message
434 =. pyz (~(put by pyz) gom (lent wyv))
441 diq (~(put by diq) (shaf %flap i.wyv) nus)
442 puq (~(put to puq) [nus `soul`[gom 0 | ~2000.1.1 i.wyv]])
447First, we put into `pyz` the id for this message and the number of its
448packets that have not yet been acknowledged, which is of course the
449total number of packets since we haven't even sent the packets.
451For every packet, we change three things in the state (`++shed:ames`) of our
452packet pump: (1) we increment `nus`, the number of packets sent; (2) we
453put the packet number into `diq` keyed by a hash of the packet; and (3)
454we put the packet into the packet queue, with the basic metadata of its
455id `gom`, 0 transmissions, not live yet, last sent in the year 2000, and
458Finally, we harvest the packet pump.
462 |= now=@da :: harvest queue
464 ?: =(~ puq) [~ +>(rtn ~)]
465 ?. (gth caw nif) [~ +>]
472`++harv` contains a core for most of its work. The meat is in `++apse`.
473First, though, it sets itself up. If there aren't any packets in the
474queue, then we simply do nothing except set `rtn`, our next timeout, to
475nil because we don't have any packets that may need to be retransmitted.
476If we have more live (that is, sent and unacknowledged) packets than our
477window size, then we don't do anything.
479Otherwise, we let `wid` be the width of our remaining packet window, and
480we initialize `rub` to nil. `rub` will be the list of packets that are
481ready to be sent. We then call `++apse` and pass the result to `++abet`.
482`++apse` decides which packets are ready to be sent.
492 ?. =(| liv.q.n.puq) .
493 :: ~& [%harv nux.q.n.puq p.n.puq]
496 rub [pac.q.n.puq rub]
499 nux.q.n.puq +(nux.q.n.puq)
504If there are no remaining packets to send, or if we've filled the packet
505window, do nothing. We call `++rigt` and `++left` to process the left
506and right branches of the packet queue.
508Now we assert that the queue is not empty, and we again check that we
509haven't filled the packet window. We will operate on the head of the
510queue. If the packet is live, then do nothing. Otherwise, we go ahead
513To send, we (1) decrement `wid`, our packet window width; (2) cons the
514packet onto the `rub`, which will be returned as the list of packets to
515send; (3) increment `nif`, the number of live packets; (4) set the
516packet to be live; (5) increment the number of transmissions of the
517packet; and (6) set the last sent time of the packet to now.
522 ^+(. =+(lef=apse(puq l.puq) lef(puq [n.puq puq.lef r.puq])))
525 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
528These do exactly what you would expect: they traverse the packet queue
529so that `++apse` gets called recursively through it.
531Finally, `++abet` gets called, which resolves the changes.
536 [(flop rub) +>.$(rtn [~ (add rto now)])]
539This returns the packets that we wish to send, and it updates the
540timeout so that we know when to try resending unacknowledged packets.
542This concludes our discussion of `++whap:pu`. To finish
543`++wind:ho:um:am`, we just need to delve into `++busk:ho:um:am`. But
544wait, in the call to `++busk`, the first argument is `xong:diz`. What is
545this? This, my dear reader, is one more detour, this time into
549 ++ xong :: xong:lax:as:go
550 ^- (list ship) :: route unto
551 =+ [fro=xen too=xeno]
552 =+ ^= oot ^- (list ship)
556 ?: (lien fro |=(a=ship =(a i.too))) ~
558 :: ~& [%xong-to [our her] (weld oot ?>(?=(^ fro) t.fro))]
559 (weld oot ?>(?=(^ fro) t.fro))
562This gets the list of intermediate ships needed to get a packet from us
563to our neighbor. First, we get `fro` and `too`, the "canons" of ourself
564and our neighbor, respectively.
566What is this "canon", you ask? A canon is simply a ship plus its
567"ancestors", as defined by `++sein:title`. For example, the canon of
571 ~hoclur-bicrel/try=> (saxo:title ~hoclur-bicrel)
572 ~[~hoclur-bicrel ~tasruc ~tug]
575If we follow the algorithm in `++xong`, we see that we are simply
576creating a list of ships that form a path from our neighbor to ourself.
577Essentially, we look through the canon of our neighbor until we find
578something in our own cannon -- a common ancestor. Or, if we are from
579different carriers, then there is no common ancestor. We then weld this
580onto the tail of our own canon. In the end, this is simply a list of
581possible ships to try to route via to get to our neighbor, ordered by
582preferability (that is, closeness to our neighbor). We will end up
583trying, in order, to find a lane to these.
585Now, we can finally get to `++busk:ho:um:am`.
588 ++ busk :: busk:ho:um:am
589 |= [waz=(list ship) pax=(list rock)] :: send packets
594 $(pax t.pax, bin (weld (flop (wist:diz now waz ~ i.pax)) bin))
598Thankfully, `++busk` is fairly simple. We go through the list of packets
599and convert them to `++boon:ames`s with `++wist:lax:as:go`. These boons are
600placed into `bin`, and they end up getting processed by `++clop` (this
604 ++ wist :: wist:lax:as:go
605 |= $: now=@da :: route via
611 ?: =(our her) [[%ouzo *lane pac] ~]
613 =+ dyr=?:(=(her i.waz) dur (gur i.waz))
618 :_ ?: ?=(%ix -.u.lun.wod.dyr)
621 :+ %ouzo u.lun.wod.dyr
622 ?: &(=(i.waz her) =(~ ryn)) pac
623 =+ mal=(jam `meal`[%fore her ryn pac])
627 ?~ yed.caq.dyr [%none mal]
631 (en:crub:crypto q.u.yed.caq.dyr mal)
635This takes a sample of the current time, the list of ships that we just
636generated, a lane if we already know it, and the packet itself.
638First, if we are sending a message to ourself, then we simply create a
639`%ouzo` boon with a bunted lane. Otherwise, if there are no routing
640candidates, there is nothing we can do, so we return nil.
642Next, we get the `++dore:ames` of the first routing candidate. If we're looking
643at the neighbor to whom we're trying to send the message, then we simply
644use the `++dore:ames` that we already have. Otherwise, we get a default `++dore:ames`.
646If we're the first routing candidate, or if we have don't have a lane to
647this candidate, then we skip this candidate and move on to the next one.
649If we have only a provisional ip address, then we try to send on it, but
650we also try to send on later routing candidates as well. Otherwise, we
651only send on this one candidate.
653Finally, we create the actual `%ouzo` boon. The lane is the one from our
654`++dore:ames`. If we're sending it directly to our intended recipient, and we
655haven't been told to use a specific lane, then we just send the packet
656directly. Otherwise, we wrap it in a little `%fore` meal, telling the
657intermediary to whom we wish it to be sent. If we have already set up a
658symmetric key with the intermediary, then we encrypt it with that.
659Otherwise, we send it in the clear.
661Now, if you recall, we have traced all the way through from the
662beginning when, in `++knob`, the `%wont` card was handled by a call to
663`++wise`. There is only one more step before the packet is finally sent.
664Looking in `++knob`, we see that the resultant list of boons is passed
665into `++clop`, which will execute the correct actions and return a list
666of moves. In `++clop`, we see the handling of each specific boon. The
667one we are interested in is `%ouzo`, since that is the only one we have
672 :: ~& [%send now p.bon `@p`(mug (shaf %flap q.bon))]
674 [[gad.fox [%give %send p.bon q.bon]] ~]
677Very simply, we give a `%send` gift along the special duct that goes
678straight into the bowels of unix. This is the last stop before we drop
679into vere, and later libuv. And then... the world.
681The packet, after its creation, embarks on a journey across physical
682time and space into the great unknown. Hurtling through fiber-optic
683cables at hundreds of thousands of kilometers per second, it finally
684arrives at our neighbor's network adapter. The adapter tells unix, unix
685tells libuv, libuv tells vere, and vere sends a `%hear` task to ames.
686And now we reenter the kernel.
688The `%hear` task goes straight to `++knob`, just as did the `%wont` task
693 (~(gnaw am [now fox]) %good p.kyz q.kyz)
696Here, though, we call `++gnaw:am` to process the packet. The arguments
697to `++gnaw` are the same as those to the `%hear` task: the lane on which
698the packet was received and the packet itself. The other argument is
699just `%good`, which is a `++cape:ames` saying that we expect the packet to
700succeed. If a formal error occurs, then since we have a transactional
701event system, the `%hear` event will never be considered to have
702actually happened, and unix will send a `%hole` task so that we may send
703a negative acknowledgment.
707 |= [kay=cape ryn=lane pac=rock] :: process packet
708 ^- [p=(list boon) q=fort]
709 ?. =(2 (end 0 3 pac)) [~ fox]
711 ?: (goop p.p.kec) [~ fox]
712 ?. (~(has by urb.ton.fox) q.p.kec)
716 %- ~(chew la:(ho:(um q.p.kec) p.p.kec) kay ryn %none (shaf %flap pac))
720First, we check the protocol number. If it is not correct, then we
721simply ignore the packet entirely. Otherwise, we parse the packet with
722`++bite`, which converts a packet atom into a `++cake:ames`, that is, a triple
723of the `sock` (pair of sender and receiver), the `skin` (encryption
727 ++ bite :: packet to cake
729 =+ [mag=(end 5 1 pac) bod=(rsh 5 1 pac)]
730 =+ :* vez=(end 0 3 mag) :: protocol version
731 chk=(cut 0 [3 20] mag) :: checksum
732 wix=(bex +((cut 0 [23 2] mag))) :: width of receiver
733 vix=(bex +((cut 0 [25 2] mag))) :: width of sender
734 tay=(cut 0 [27 5] mag) :: message type
737 ?> =(chk (end 0 20 (mug bod)))
738 :+ [(end 3 wix bod) (cut 3 [wix vix] bod)]
740 (rsh 3 (add wix vix) bod)
743This is exactly the inverse of `++spit`. Note that here we check both
744the protocol number and the hash, crashing on error. Remember that a
745crash will result in a negative acknowledgment being sent.
747Continuing in `++gnaw`, we see that if the intended recipient is not on
748our pier, then we drop the packet.
750If we've gotten this far, then we wish to process the packet. Recall
751that `++ho` and `++um` set up the domestic server and foreign client
752cores, respectively, and that `++zork` and `++zank` resolve any changes
755The new stuff here, then, is the `++la` core and the `++chew` arm. The
756`++la` sets up a core for this particular packet, containing the current
757success/failure `++cape:ames`, the lane it was sent on, the encryption type,
758and a hash of the packet, used as an id.
760`++chew` is called with the encryption type and the message itself. It
761contains a little helper core inside of it, which starts immediately
768 =+ neg==(~ yed.caq.dur.diz)
770 =+ eng==(~ yed.caq.dur.diz)
774 :_(bin [%wine [our her] " is ok"])
777 :_(bin [%wine [our her] " is your neighbor"])
781First, we let `oub` be true if our neighbor hasn't been responding to us
782for more than sixteen seconds. Let `neg` be true if we haven't yet
783proposed a symmetric key, meaning that we haven't yet corresponded with
784this ship, so they are not our neighbor. Next, we run `++east`, which
785we'll go into in just a minute.
787We now do the same two checks and store the results in `eng` and `bou`.
788If our neighbor has, like the prodigal son, returned after an extended
789absence, then we send a `%wine` boon as the proverbial fatted calf,
790which is simply printed out to the console. Likewise, if we are meeting
791one with whom we have never had the pleasure of acquainting ourselves,
792we send a message to the console to that effect.
794We skipped over `++east`, which contains the meat of the processing. It
795first decrypts the message, then calls `++chow:la:ho:um:am` with the
796resultant meal. We'll go through each of the four cases in turn, but
797first since each one calls `++bilk:pu`, we'll take a brief detour.
801 |= now=@da :: inbound packet
807 rtn ?~(puq ~ [~ (add now trt)])
811This updates the timing information in our packet pump. `rue`, the last
812time we have heard from this neighbor, is set to now. `rto`, the
813retransmit timeout is set to twice the current ping time, and if there
814is anything in the packet queue, then we reset the next timeout, since
815we've just heard a message.
821 =. puz (bilk:puz now)
822 (chow ((hard meal) (cue msg)))
825The simplest case is when the encryption type is `%none`. We first call
826`++bilk` to update the packet pump, then we cue (unjam) the message into
827a meal. We hard cast it into a meal -- if the cast fails, then we do
828want to crash since someone is sending us malformed data. Finally, we
829send the result to `++chow` for interpretation and handling.
833 =+ [mag=`hand`(end 7 1 msg) bod=(rsh 7 1 msg)]
834 =+ dey=(kuch:diz mag)
836 ~& [%bad-key her mag]
837 +>.$ :: ignore unknown key
838 =. puz (bilk:puz now)
840 (chow(aut sin) ((hard meal) (cue (dy:q:sen:gus key bod))))
843For symmetric encryption, we first get the `++hand`, which is the hash of
844the symmetric key. We pass it to `++kuch:lax:as:go`, which returns the
845key if we either have used it before or we have proposed it. If we have
846proposed it, then we change its status from proposed to real. If
847`++kuch` fails, then we drop the packet and print out a `%bad-key`
850Otherwise, we call `++bilk` as before to update the packet pump and pass
851into `++chow` the decrypted data.
855 =+ mex=((hard ,[p=[p=life q=life] q=will r=@]) (cue msg))
856 =. diz (deng:diz q.mex)
859 =+ gey=(sev:gus p.p.mex)
860 =+ mes=(need (tear:as:q.gey pub:ex:r.wug r.mex))
861 =. diz (wasc:diz p.mes)
862 =. puz (bilk:puz now)
866For sealed asymmetric encryption, we first take off the the layer of
867data that gives us the life and will of our neighbor, and we apply try
868to extend their former will with the new data. `++deng` will fail if
871Next, we get our most current understanding of our neighbor's crypto,
872and we verify that it's the same life as what they're sending. Then, we
873get our own crypto from `++sev` and decrypt the message with the public
874key from our neighbor's crypto. We register the proposed symmetric key,
875update the packet pump, and call `++west`, which simply casts the
876message to a meal and calls `++chow`, reporting any error.
880 =+ mex=((hard ,[p=[~ q=life] q=will r=@]) (cue msg))
881 =. diz (deng:diz q.mex)
884 =+ mes=(need (sure:as:r.wug *code r.mex))
885 =. puz (bilk:puz now)
889Finally, for signed asymmetric encryption, we, as before, take off the
890layer of data that gives us the life and will of our neighbor. This
891time, of course, we do not get our own crypto -- only that of our
894The rest you have seen. We call `++deng` to extend the will, we verify
895that their crypto life is what we think it ought to be, we "decrypt" the
896data, we update the packet pump, and we call `++west` to call `++chow`.
899 ++ chow :: chow:la:ho:um:am
900 |= fud=meal :: interpret meal
902 =. diz ?:(=(%none aut) diz (wast:diz ryn))
906Here, if the message was encrypted at all, then we call
907`++wast:lax:as:go`, which simply updates the lane (route) to our
908neighbor (unless we're given a provisional route). This ensures that we
909always have the most direct possible path to them.
911We've been handling this meal for so long, we've almost forgotten what
912we want to do with it. The telos is of any meal to be dined on. We will
913choose out the cases here that are important to our current
920 ?. ?=(%if -.u.q.fud) u.q.fud
924 (emit %mead lyn r.fud)
925 =+ zid=(myx:gus p.fud)
926 (emir (wist:zid now xong:zid [~ lyn] r.fud))
929Forwarding is the simplest case, since we've seen all the arms before,
930except perhaps `++emit` and `++emir`, which simply take a boon or list
931of boons respectively and queue them up to be handled when the core
932resolves. If we're told to forward a packet to ourselves, then we emit a
933`%mead` boon which simply sends another `%hear` task to ourselves with
934the data. Otherwise, we try to find a route to the recipient, as before.
938 =+ zol=(~(get by olz.weg) s.fud)
939 ?^ zol cock(kay u.zol)
941 =+ neb=(~(get by nys.weg) s.fud)
942 ?^ neb [u.neb nys.weg]
943 =+ neb=`bait`[(kins p.fud) 0 r.fud ~]
944 [neb (~(put by nys.weg) s.fud neb)]
945 ?> (lth q.fud p.r.neb)
946 ?> =((kins p.fud) p.neb)
948 =+ doy=`(unit ,@)`(~(get by q.r.neb) q.fud)
951 q.r.neb (~(put by q.r.neb) q.fud t.fud)
954 :: ~& [%carp q.fud s.fud q.neb p.r.neb]
956 =: nys.weg (~(del by nys.weg) s.fud)
957 olz.weg (~(put by olz.weg) s.fud kay)
961 +>.$(nys.weg (~(put by nys.weg) s.fud neb))
964Here, we have received a partial message, and we're just assembling the
965individual packets into a message. Most of this code is fairly
966algorithmic, so we'll just hit the high points. In the beginning, we
967check if we've already received this message, and if so, we resend the
968acknowledgment. Remember, "always ack a dupe, never ack an ack".
970In `nys.weg` we keep track of an incoming set of partial packets,
971indexed by the `flap` hash that comes with every packet. We check to see
972if we have already received this partial message, and if so we
973acknowledge it. Otherwise, we put it in `nys.weg` unless this is the
974last message, in which case we ack the last partial message, move the
975complete message into `olz.weg`, and call `++golf`, which assembles the
976message and calls `++chew`, to start the dance again with the complete
981 :: ~& [%bund q.fud r.fud]
982 ?> =(p:sen:gus p.fud)
983 (deer q.fud r.fud ?-(kay %dead ~, %good [~ s.fud]))
986What if we're just receiving a regular old, garden variety message? We
987call `++deer` with the data from the message. If we already know that
988the message processing will fail (that is, if we got a `%hole` card from
989unix rather than a `%hear` card), then we don't even send the data at
990all. Remember, if a packet fails to process, it's as if it never even
991arrived, except that we send a negative acknowledgment.
994 ++ deer :: deer:la:ho:um:am
995 |= [cha=path num=@ud dut=(unit)] :: interpret message
997 =+ rum=(fall (~(get by raz.bah) cha) *race)
1000 ?. (gte num did.rum) :: always ack a dup
1001 (cook (~(get by bum.rum) num) cha ~ ryn dam)
1003 (coat cha rum(mis (~(put by mis.rum) num [kay ryn dam dut])))
1006 %+ ~(put by raz.bah) cha
1007 rum(mis (~(put by mis.rum) num [kay ryn dam dut]))
1012First, we get the race for this particular triple of sender, receiver,
1013and path, creating it if it doesn't exist. If we've already acked the
1014message, then we resend the ack. Note that `did.rum` is the number of
1015packets we acknowledged, positively or negatively while `bum.rum` is a
1016map of message numbers to negative acknowledgments. Thus, if a message
1017number is less than `did.rum`, then if it's in `bum.rum` then it was
1018negatively acknowledged, otherwise it's positively acknowledged. Thus, we
1019are constant in space with the number of successful messages and linear
1020in the number of failed messages. We'll document `++cook` later on, but
1021suffice it to say that it sends an acknowledgment. It is to end-to-end
1022acknowledgments what `++cock` is to packet-level acknowledgments.
1024If we are still processing a message (that is, `dod.rum` is false), then
1025we simply put this message in the map of misordered packets to be
1026processed when their time comes. "Processing a message" in this case
1027means that we've received the message and notified the correct
1028application, but we're still waiting for the application-level
1031Otherwise, we're ready for a packet, so we process it.
1034 ++ coat :: coat:ho:um:am
1035 |= [cha=path rum=race] :: update input race
1037 =+ cun=(~(get by mis.rum) did.rum)
1039 +>.$(raz.bah (~(put by raz.bah) cha rum))
1040 ?. =(%good p.u.cun) +>.$
1043 raz.bah (~(put by raz.bah) cha rum(dod |))
1048 `soap:ames`[[p:sen:gus clon:diz] cha did.rum]
1053First, we grab the message we want to process and store it in `cun`. If
1054it's a good packet, then we change `dod.rum` to false, meaning that
1055we're in the middle of processing a packet and should not start
1056processing another one. We also put a `%mulk` boon into the queue so
1057that, when it all resolves, we send a mesage to the intended recipient
1058application. The boon contains the sender, the receiver, the identity of
1059the message, and the message itself.
1061This bubbles up all the way back to `++knob`, where we were handling the
1062`%hear` card. Following the logic in `++knob`, we can see that the boons
1063get sent into `++clop` to be turned into actual arvo-level moves. We've
1064been here before, if you recall, when we handled the `%cake` boon to
1065send a message. Now, we're handling the `%mulk` boon, which is
1066unfortunately slightly more complicated.
1070 :: ~& [%mulk p.bon q.bon]
1071 ?> ?=([@ @ *] q.q.bon)
1076 :~ :- (claw p.p.bon)
1077 [%sick %wart p.bon i.t.q.q.bon t.t.q.q.bon r.bon]
1079 %ge :: %gall request
1080 ?> ?=([@ ~] t.t.q.q.bon)
1081 =+ app=`term`(need ((sand %tas) i.t.t.q.q.bon))
1083 :+ (scot %p p.p.bon)
1086 :_ fox [hen %pass pax %g %rote p.bon app r.bon]~
1087 %gh :: %gall response
1088 ?> ?=([@ ~] t.t.q.q.bon)
1089 =+ app=`term`(need ((sand %tas) i.t.t.q.q.bon))
1091 :+ (scot %p p.p.bon)
1094 :_ fox [hen %pass pax %g %roth p.bon app r.bon]~
1098We're dispatching messages based on the prefix of their path. Since only
1099`%gall` apps use end-to-end acknowledgments at the moment, every path
1100must have at least two elements, and the first one must be `%q`. Beyond
1101that, we handle the `/q/ge` and `/q/gh` cases for gall requests and
1102responses, respectively.
1104In both cases, we require the next term in the path to be the name of
1105the intended recipient `%gall` app. Thus, a message to `/q/ge/talk` for
1106example, will send a message to the talk app.
1108We then send a message to the app itself. The message is either a
1109`%rote` or a `%roth` for a request and a response, respectively. The
1110content is the `rook` or `roon` that was sent (stored in `r.bon`), but
1111we don't actually handle that at all here. That's completely a
1112`%gall`-level thing. We're just the messenger.
1114Notice the path we send this over. We encode the sender, the receiver,
1115and the path over which it was sent. This fully specifies the `race` so
1116that when the app gives us the acknowledgment we know where to send it.
1118We now have another interlude. We have entrusted our precious data, so
1119carefully guarded and guided from the app on that far-away ship, to our
1120local app. It has the ability to do whatever it pleases with it. It may
1121take a significant amount of time to process. When the message has been
1122handled by this app, though, it must produce an acknowledgment. Our
1123final task is to deliver this acknowledgment to the sending app.
1125We should describe here what exactly these oft-mentioned acknowledgments
1126actually consist of. There are two kinds of acknowledgments: positive
1127and negative. A positive acknowledgment contains no data other than its
1128existence. A negative acknowledgment may optionally include a reason for
1129said negativity. Formally, a negative acknowledgment is an `ares`, which
1130is a unit pair of a term and a list of tanks. If this is null, this is
1131simply a failure with no associated information. If the pair exists, the
1132term is a short error code that is usually both human and computer
1133readable. For example, if you try to send a message to a valid `%gall`
1134app that doesn't have any `++poke` to handle it, then `%gall` will give
1135a negative acknowledgment with error term `%poke-find-fail`. The list of
1136tanks is a human-readable description of the error. This often contains
1137a stack trace. At any rate, all this information is returned to the
1138sending app on the other end of the wire.
1140After this brief interlude, our story resumes in `++knap`, where we
1141receive responses. In particular, a `%mean` indicates a negative
1142acknowledgment while a `%nice` indicates a positive acknowledgment.
1146 ?> ?=([@ @ @ *] tea)
1147 =+ soq=[(slav %p i.tea) (slav %p i.t.tea)]
1151 %^ ~(rack am [now fox]) soq pax
1152 ?-(+<.sih %mean `p.+.sih, %nice ~)
1155 |- ^- [p=(list move) q=_+>.^$]
1158 =^ toe fox (clop now hen i.p.fuy)
1159 $(p.fuy t.p.fuy, out (weld (flop toe) out))
1162Recall the format of the path we sent the message on, and you'll
1163understand why `soq` and `pax` are the sender/receiver pair and path on
1164which the message was sent. The rest of this is structured much like
1165`++knob`, so we call `++rack:am` and send the resulting boons to
1166`++clop`. Business as usual.
1170 |= [soq=sock cha=path cop=coop] :: e2e ack
1171 =+ oh=(ho:(um p.soq) q.soq)
1172 =. oh (cook:oh cop cha ~)
1176First, we set up `++um` and `++ho`, as we've done twice before, for our
1177domestic and foreign servers, respectively. The other two things are
1178new, though. Well, `++cook` is not actually new, but we delayed the
1179explanation saying only that it sends an acknowledgment. The time has
1183 ++ cook :: cook:ho:um:am
1184 |= [cop=coop cha=path ram=(unit ,[ryn=lane dam=flap])]
1185 ^+ +> :: acknowledgment
1186 =+ rum=(need (~(get by raz.bah) cha))
1187 =+ lat=(~(get by mis.rum) did.rum)
1188 ?: &(?=(~ lat) ?=(~ ram)) ~&(%ack-late-or-redundant +>.$)
1189 =+ ^- [ryn=lane dam=flap]
1190 ?^ ram [ryn.u.ram dam.u.ram]
1195 %+ ~(put by raz.bah) cha
1196 rum(dod &, bum ?~(cop bum.rum (~(put by bum.rum) did.rum u.cop)))
1197 =^ roc diz (zuul:diz now [%buck cop dam ~s0])
1198 (busk(diz (wast:diz ryn)) xong:diz roc)
1201If we are acknowledging a message that we have already acked, the `ram`
1202will contain the new lane and flap to send the duplicate ack to. This
1203happens if we call `++cook` in `++deer`, but it doesn't happen from
1204`++rack`. If there is no message waiting to be acknowledged and we're
1205not given an explicit lane and flap (that is, we're not sending a
1206duplicate ack), then the app must have sent us multiple acknowledgments.
1207We do the only sensible thing we can do and drop all acknowledgments
1208after the first, printing a message. This is, in fact, an error, so it
1209could be argued that we ought to crash. Whatever you do, don't depend on
1212First, we grab the race specified by the given path, and we get the most
1213recent in-order message, which must be the one which is being
1216Then, we decide which lane/flap to respond on/to. Basically, in the
1217usual case we respond on the lane through which the initial message was
1218sent, which is stored along with the other packet information in
1219`mis.rum`, since it has to be remembered across calls to ames. However,
1220if we receive a duplicate message, then we must respond to the new
1221message. It's quite possible the reason the other acknowledgment didn't
1222get returned was that the lane between the ships was broken.
1224At any rate, we update the race by saying that we've finished processing
1225this packet (unless we're sending a duplicate ack) and, if we're sending
1226a negative acknowledgment, putting the negative ack into `bum.rum` so
1227that we can resend it if necessary.
1229We encode our new message, updating the packet pump, with `++zuul`, as
1230before, and we send it off with `++busk`, routed via `++wast` to one of
1231the ships in `++xong`. Of course, in practice, we don't even look at the
1232ships in `++xong` because we already have a lane directly to our
1233neighbor (the one over which they sent their message to us).
1235We glossed over the actual message we're sending back. We're sending a
1236`%buck` meal, which is an acknowledgment. The `cop` specifies whether
1237this is a positive or a negative ack, `dam` specifies the message we're
1238acknowledging, and the `~s0` is a placeholder for the processing time
1239required. This time is neither calculated (though it is hopefully
1240obvious how to do so) nor used at present, but this information may be
1241used in the future for improved congestion control. Since the round-trip
1242time for an end-to-end acknowledged packet includes the processing time
1243on the other end, most common congestion control algorithms will stumble
1244when some messages take much longer to process than others. As noted,
1245though, this is simply an opportunity for improvement -- our congestion
1246control algorithms are relatively naive at the moment.
1248Recall that `++busk` calls `++wist` to put the actual `%ouzo` boon in
1249the queue, which gets handled by `++clop` to actually send the message.
1250This is the same pipeline as sending any other message, so we'll refer
1251you to the explanation above if you've forgotten it.
1253The last thing we need to do on this ship is move on to the next packet
1254in the queue if there is one. If you recall, in `++rack` after the call
1255to `++cook` there was a call to `++cans:ho:um:am`.
1258 ++ cans :: cans:ho:um:am
1260 =+ rum=(need (~(get by raz.bah) cha))
1264 mis (~(del by mis.rum) did.rum)
1269This is very simple. We increment the number of packets that we've
1270acknowledged on this race and we delete the packet that we just
1271acknowledged from the set of misordered packets.
1273Then, we call `++coat` again to process the next packet if we've already
1274received it. And that's it for this.
1276The acknowledgment now travels the same path that its forebearer, the
1277original message, once tread, but this time not into the great unknown.
1278The weary traveler is seeking out its familial roots, finding the app
1279from whom sprung forth the original message way back in paragraph three.
1280When it arrives at the network adapter of its ancestors, the adapter
1281tells unix, unix tells libuv, libuv tells vere, and vere sends a `%hear`
1282task to ames. Once more into the kernel.
1284The `%hear` task is handled in `++knob` as before, leading to `++gnaw`,
1285going over to `++chew`, `++apse`, `++chow`, and eventualy to `++dine`.
1286We've seen most of the cases in `++dine`, but we haven't yet looked at
1287the handling of this `%buck` meal.
1291 =. +> ?.(=(%full aut) +> cock) :: finish key exch
1292 +>(..la (tock p.fud q.fud r.fud))
1295We send a packet level acknowledgment if we're finishing a key exchange,
1296else we call `++tock` to process the acknowledgment.
1298This will get a little involved, so if you don't much care about how
1299exactly an acknowledgment happens, just know that the result gets gifted
1300as a `%woot` card back to the app who sent it. For those brave souls who
1301wish to see this thing through to the end, it's once more into the
1305 ++ tock :: tock:ho:um:am
1306 |= [cop=coop fap=flap cot=@dr] :: e2e ack by hash
1308 =^ yoh puz (bick:puz now fap)
1312 (done p.u.p.yoh q.u.p.yoh)
1317 `boon`[%cake [our her] [[p:sen:gus clon:diz] u.p.yoh] cop u.hud]
1319 (busk xong:diz q.yoh)
1322We're going to work through this one a little backwards since it's
1323mostly fairly simple except the call to `++bick:pu`. In fact, we'll just
1324skip `++bick` for the moment and finish the rest.
1326If `++bick` succesfully acks the message, then we call `++done`.
1329 ++ done :: done:ho:um:am
1330 |= [cha=path num=@ud] :: complete outgoing
1331 ^- [(unit duct) _+>]
1332 =+ rol=(need (~(get by ryl.bah) cha))
1333 =+ rix=(~(get by san.rol) num)
1338 (~(put by ryl.bah) cha rol(san (~(del by san.rol) num)))
1342This very simply gets the rill (the outgoing counterpart to a race, if
1343you recall), pulls out of the map of outstanding messages the duct over
1344which the original message was sent, and produces this duct while
1345deleting that entry from the map of outstanding messages.
1347Going back to `++tock`, we now have the duct we need to return the
1348result over. We do the very sensible thing and put a `%cake` boon in the
1349queue to be processed later by `++clop`.
1351In `q.yoh` we have a list of messages that may need to be sent, which we
1352pass to `++busk` to send, as usual. When an acknowledgment arrives, that
1353may trigger other messages immediately. This often happens when sending
1354more messages than the width of the logical window since for congestion
1355control reasons another message cannot be sent until some of the earlier
1356ones have been acknowledged.
1358We'll look at the processing of the `%cake` boon in `++clop` before we
1359get back to talking about `++bick`.
1364 :~ [s.bon %give %woot q.p.bon r.bon]
1368We very simply give, along the duct we found above, a `%woot` card with
1369the ship who sent us the ack and the ack itself. This allows the
1370application to decide what to do about the result. In case of a failure,
1371we usually either resend the message or display it to the user.
1372Sometimes, we recognize the error term and handle it internally. In any
1373case, the decision of how to handle the acknowledgment is entirely up to
1374the application. Our job is done.
1376Well, except that we skipped `++bick:pu`. Let's go back to that.
1380 |= [now=@da fap=flap] :: ack by hash
1381 ^- [[p=(unit soup) q=(list rock)] _+>]
1382 =+ sun=(~(get by diq) fap)
1385 =. diq (~(del by diq) fap)
1386 =^ gub +>.$ (bock now u.sun)
1387 =^ yop +>.$ (harv now)
1391If you recall, in `++whap:pu` we created the packet pump's
1392representation of the message, which included putting the message into
1393`diq`, which maps from packet hashes to packet sequence numbers. Thus,
1394`u.sun` is the sequence number of this particular message.
1396We delete this message from `diq` since we have now received an ack for
1397it. We call `++bock` to perform the ack by sequence number. We call
1398`++harv` to harvest the packet queue, sending any messages that are now
1401In `++bock`, there are three arms we haven't seen before: `++bine`,
1402`+wept`, and `++beet`. We'll describe each of these before we get to
1403`++bock`. `++bine` looks scariest.
1407 |= [now=@da num=@ud] :: apply ack
1408 ^- [(unit soup) _+>]
1411 ?: (gth num p.n.puq)
1413 [-.lef +.lef(puq [n.puq puq.lef r.puq])]
1415 [-.rig +.rig(puq [n.puq l.puq puq.rig])]
1416 =: rtt ?. &(liv.q.n.puq =(1 nux.q.n.puq)) rtt
1417 =+ gap=(sub now lys.q.n.puq)
1418 :: ~& [%bock-trip num (div gap (div ~s1 1.000))]
1419 (div (add (mul 2 rtt) gap) 3)
1420 nif (sub nif !liv.q.n.puq)
1422 =+ lez=(dec (need (~(get by pyz) gom.q.n.puq)))
1425 [[~ gom.q.n.puq] (~(del by pyz) gom.q.n.puq)]
1426 [~ (~(put by pyz) gom.q.n.puq lez)]
1428 +>.$(puq ~(nap to puq))
1431The first few lines are simply looking through the packet queue until we
1432find the correct packet to ack. This is basic queue manipulation that
1433operates directly on the treap structure of the queue. If you understand
1434treap queues, the logic is easy to follow. Otherwise, just trust us that
1435by the time we get to the `=:`, the packet with sequence number `num` is
1436on the top of the packet queue (that is, at `n.puq`).
1438We first update the round-trip time. If the packet is either not alive
1439or had to be transmitted more than once, then we don't have any reliable
1440way of calculating the round-trip time since we're unsure of exactly
1441which transmission was acknowledged. Otherwise, the round-trip time is
1442the difference between now and when the packet was last sent. We set
1443`rtt` by a little weighted average where the previous smoothed RTT is
1444weighted twice as much as the RTT of the current packet. Thus,
1445`(2*rtt+gap)/3`. This gives us a nice smooth RTT that is somewhat
1446resilient to outlier data while still being responsive to our
1449If the packet wasn't already dead, then we decrement the number of live
1450packets, which may allow more packets to be sent.
1452We decrement the number of unacknowledged packets in our `pyz` for this
1453particular message. If you recall, this was set in `++whap` to the
1454number of packets required to send a message.
1456If that was the last packet in the messge that needed to be acked, then
1457we delete the messgae reference from `pyz` and produce the id of the
1458message. Otherwise, we simply update `pyz` with the new number of
1459unacked messages. In either case, we remove the packet from the packet
1464 |= [fip=@ud lap=@ud] :: fip thru lap-1
1471 ?: (lth p.n.puq fip) ?~(l.puq . left)
1472 ?: (gte p.n.puq lap) ?~(r.puq . rigt)
1475 ?.(liv.q.n.puq . .(nif (dec nif), liv.q.n.puq |))
1479 ^+(. =+(lef=apse(puq l.puq) lef(puq [n.puq puq.lef r.puq])))
1482 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
1486The algorithm is a simple case of traversing the packet queue.
1487Essentialy, we mark as dead all packets in the queue between `fip` and
1488`(dec lap)`. We also update `nif`, the number of live packets. Lest you
1489mourn too much the passing of these packets, know that they shall soon
1490rise again. Recall that in `++bick` after the call to `++bock` we call
1491`++harv`. This will resend the packets that have just been labeled dead.
1495 ^+ . :: advance unacked
1496 =- +(nep ?~(foh nus u.foh))
1500 ?: (lte p.n.puq nep) $(puq l.puq)
1502 ?^(rig rig [~ p.n.puq])
1505Here we search for the next expected packet number. Basically, we search
1506the queue for the leftmost packet whose number is greater than the
1507current `nep`. If we don't find any such packet, we just use the total
1508number of packets sent.
1510We can now dive into `++bock`, our last arm.
1514 |= [now=@da num=@ud] :: ack by sequence
1515 ^- [(unit soup) _+>]
1516 =^ gym +> (bine now num)
1519 =+ cam=(max 2 (div caw 2))
1520 :: ~& [%bock-hole num nep cam]
1521 beet:(wept(nep num, cag cam, caw cam) nep num)
1522 =. caw ?: (lth caw cag) +(caw)
1523 (add caw !=(0 (mod (mug now) caw)))
1525 :: ~& [%bock-fine num nif caw cag]
1527 :: ~& [%bock-fill num nif caw cag]
1531First, we call `++bine` to apply the ack to the packet pump information.
1532We produce `gym`, which, if it exists, is the id of the packet that was
1533acked. If we received an ack for a packet later than the one we
1534expected, then we halve the logical packet window and kill all the
1535earlier packets so that they may be resent.
1537Otherwise, we possibly increase the congestion window. If the window is
1538less than the congestion threshold, then we increment the size of the
1539window. Otherwise, we only increment one out of every `caw` times.
1541If we received an ack for the packet we expected, then we simply advance
1542`nep` with `++beet`. If we received an ack for a packet earlier than we
1543expected, we do nothing.
1545It may be hard to believe, but we are, in fact, done. The message has
1546been sent, received, acknowledged, and the acknowledgment has been
1547returned to the original sender. We hope it's clear that, while the
1548process has been somewhat involved, the algorithms are not all that
1549complicated. If you've read this far, you know `%ames`. The only other
1550code involves initialization, timeouts, and the like.
1552Below, we give detailed reference documentation for the data models
1557### `++sufi:ames`, domestic host
1560 ++ sufi :: domestic host
1561 $: hoy=(list ship) :: hierarchy
1562 val=wund :: private keys
1563 law=will :: server will
1564 seh=(map hand ,[p=ship q=@da]) :: key cache
1565 hoc=(map ship dore) :: neighborhood
1569This is the security state of a domestic server.
1571`hoy` is a list of the ships directly above us in the hierarchy of
1572ships. For example, for `~hoclur-bicrel`, this would be `~tasruc` and
1573`~tug`. See `++sein:title`.
1575`val` is a list of our private keys.
1577`law` is our certificate, which is a list of the XXX
1581`hoc` is a map of ships to `++dore:ames`. The stores all the security
1582information about foreign ships. The keys to this map are the neighbors
1583(ships we have been in contact with) of this domestic server.
1585### `++wund:ames`, private keys
1588 ++ wund (list ,[p=life q=ring r=acru]) :: mace in action
1591This is a list of our own private keys, indexed by life. The key itself
1592is the `++ring`, and the `++acru:ames` is the encryption engine. We generate
1593the `++acru:ames` from the private key by calling `++weur`. Thus, we can at
1594any time regenerate our `++wund:ames` from a `++mace:ames`. The current crypto is
1595at the head of the list and can be accessed with `++sen:as:go`.
1597### `++ring`, private key
1600 ++ ring ,@ :: private key
1603This is a private key. The first byte is reserved to identify the type
1604of cryptography. Lower-case means public key, upper-case means public
1605key, and the letter identifies which `++acru:ames` to use.
1607### `++pass`, public key
1610 ++ pass ,@ :: public key
1613This is a public key. The first byte is reserved to identify the type of
1614cryptography. Lower-case means public key, upper-case means public key,
1615and the letter identifies which `++acru:ames` to use.
1617### `++mace:ames`, private secrets
1620 ++ mace (list ,[p=life q=ring]) :: private secrets
1623This is a list of the our private keys, indexed by life. From this we
1624can generate a `++wund:ames` for actual use.
1626### `++skin:ames`, encoding stem
1629 ++ skin ?(%none %open %fast %full) :: encoding stem
1632This defines the type of encryption used for each message. `%none`
1633refers to messages sent in the clear, `%open` refers to signed messages,
1634`%full` refers to sealed messages, and `%fast` refers to symmetrically
1635encrypted messages. See `++acru:ames` for details.
1637### `++acru:ames`, asymmetric cryptosuite
1640 ++ acru :: asym cryptosuite
1641 $_ ^? |% :: opaque object
1642 ++ as ^? :: asym ops
1643 |% ++ seal |=([a=pass b=@ c=@] _@) :: encrypt to a
1644 ++ sign |=([a=@ b=@] _@) :: certify as us
1645 ++ sure |=([a=@ b=@] *(unit ,@)) :: authenticate from us
1646 ++ tear |= [a=pass b=@] :: accept from a
1647 *(unit ,[p=@ q=@]) ::
1649 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1650 ++ dy |+([a=@ b=@] _@) :: symmetric de, hard
1651 ++ en |+([a=@ b=@] _@) :: symmetric en
1653 |% ++ fig _@uvH :: fingerprint
1654 ++ pac _@uvG :: default passcode
1655 ++ pub *pass :: public key
1656 ++ sec *ring :: private key
1658 ++ nu ^? :: reconstructors
1659 |% ++ pit |=([a=@ b=@] ^?(..nu)) :: from [width seed]
1660 ++ nol |=(a=@ ^?(..nu)) :: from naked ring
1661 ++ com |=(a=@ ^?(..nu)) :: from naked pass
1666This is an opaque interface for a general asymmetric cryptosuite. Any
1667form of asymmetric cryptography can be dropped in to be used instead of
1668the default. Right now, there is one cryptosuite: `++crub:crypto:crypto`,
1669which is elliptic-curve cryptography.
1671#### `++as:acru:ames`, asymmetric operations
1674 ++ as ^? :: asym ops
1675 |% ++ seal |=([a=pass b=@ c=@] _@) :: encrypt to a
1676 ++ sign |=([a=@ b=@] _@) :: certify as us
1677 ++ sure |=([a=@ b=@] *(unit ,@)) :: authenticate from us
1678 ++ tear |= [a=pass b=@] :: accept from a
1679 *(unit ,[p=@ q=@]) ::
1683This is the core that defines the standard asymmetric cryptography
1686`++seal:as:acru:ames` allows us to send a message encrypted with someone's
1687public key so that only they may read it. If Alice seals a message with
1688Bob's public key, then she can be sure that Bob is the only one who can
1689read it. This is associated with the `++skin:ames` `%full`.
1691`++sign:as:acru:ames` allows us to sign a message with our private key so
1692that others can verify that we sent the message. If Alice signs a
1693message with her private key, then Bob can verify with her public key
1694that it was indeed Alice who sent it. This is associated with the
1695`++skin:ames` `%open`.
1697`++sure:as:acru:ames` is the dual to `++sign:as:acru:ames`. It allows us to verify
1698that a message we have received is indeed from the claimed sender. If
1699Alice sends a message with her private key, then Bob can use this arm to
1700verify that it was indeed Alice who sent it. This is associated with the
1701`++skin:ames` `%open`.
1703`++tear:as:acru:ames` is the dual to `++seal:as:acru:ames`. It allows us to read a
1704message that we can be sure is only read by us. If Alice seals a message
1705with Bob's public key, then Bob can use this arm to read it. This is
1706associated with the `++skin:ames` `%full`.
1708#### `++de:acru:ames`, `++dy:acru:ames`, and `++en:acru:ames`, symmetric encryption/decryption
1711 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1712 ++ dy |+([a=@ b=@] _@) :: symmetric de, hard
1713 ++ en |+([a=@ b=@] _@) :: symmetric en
1716Symmetric encryption is associated with the `++skin:ames` `%fast`.
1718`++de:acru:ames` decrypts a message with a symmetric key, returning `~` on
1719failure and `[~ u=data]` on success.
1721`++dy:acru:ames` decrypts a message with a symmetric key, crashing on
1722failure. This should almost always be defined as, and should always be
1723semantically equivalent to, `(need (de a b))`.
1725`++en:acru:ames` encrypts a message with a symmetric key.
1727#### `++ex:acru:ames`, exporting data
1731 |% ++ fig _@uvH :: fingerprint
1732 ++ pac _@uvG :: default passcode
1733 ++ pub *pass :: public key
1734 ++ sec *ring :: private key
1738`++fig:ex:acru:ames` is our fingerprint, usually a hash of our public key.
1739This is used, for example, in `++zeno`, where every carrier owner's
1740fingerprint is stored so that we can ensure that carriers are indeed
1741owned by their owners
1743`++pac:ex:acru:ames` is our default passcode, which is unused at present.
1745`++pub:ex:acru:ames` is the `++pass` form of our public key.
1747`++sec:ex:acru:ames` is the `++ring` form of our private key.
1749#### `++nu:acru:ames`, reconstructors
1752 ++ nu ^? :: reconstructors
1753 |% ++ pit |=([a=@ b=@] ^?(..nu)) :: from [width seed]
1754 ++ nol |=(a=@ ^?(..nu)) :: from naked ring
1755 ++ com |=(a=@ ^?(..nu)) :: from naked pass
1759These arms allow us to reconstruct a `++acru:ames` from basic data.
1761`++pit:nu:acru:ames` constructs a `++acru:ames` from the width of our intended key
1762and seed entropy. This is usually used in the initial construction of
1765`++nol:nu:acru:ames` constructs a `++acru:ames` from a "naked ring", meaning a
1766`++ring` without the initial byte identifying the type of crypto.
1768`++com:nu:acru:ames` constructs a `++acru:ames` from a "naked pass", meaning a
1769`++ring` without the initial byte identifying the type of crypto.
1771### `++will`, certificate
1774 ++ will (list deed) :: certificate
1777This is a list of deeds associated with the current ship. There should
1778be an item in this list for every ship from this point up in the
1779hierarchy times the number of lives that each ship has had. For example,
1780\~hoclur-bicrel may have a will with three items: one for itself, one
1781for \~tasruc (who issued \~hoclur-bicrel's deed) and one for \~tug (who
1782issued \~tasruc's deed).
1784### `++deed`, identity
1787 ++ deed ,[p=@ q=step r=?] :: sig, stage, fake?
1790`p` is the signature of a particular deed, which is a signed copy of
1793`q` is the stage in the identity.
1795`r` is true if we're working on a fake network, where we don't check
1796that the carrier fingerprints are correct. This allows us to create fake
1797networks for development without interfering with the real network.
1799### `++step:ames`, identity stage
1802 ++ step ,[p=bray q=gens r=pass] :: identity stage
1805This is a single stage in our identity. Thus, this is specific to a
1806single life in a single ship. Everything in here may change between
1813`r` is the public key for this stage in the identity.
1818 ++ bray ,[p=life q=(unit life) r=ship s=@da] :: our parent us now
1823### `++gens:ames`, general identity
1826 ++ gens ,[p=lang q=gcos] :: general identity
1829`p` is the IETF language code for the preferred language of this
1830identity. This is unused at the moment, but in the future text should be
1831localized based on this.
1833`q` is the description of the ship.
1835### `++gcos:ames`, identity description
1838 ++ gcos :: id description
1839 $% [%czar ~] :: 8-bit ship
1840 [%duke p=what] :: 32-bit ship
1841 [%earl p=@t] :: 64-bit ship
1842 [%king p=@t] :: 16-bit ship
1843 [%pawn p=(unit ,@t)] :: 128-bit ship
1847This is the description of the identity of a ship. Most types of
1848identity have a `@t` field, which is their human-readable name. The
1849identity of a `%duke` is more involved.
1851A `%czar`, a carrier, is a ship with an 8-bit address. Thus, there are
1852only 256 carriers. These are at the top of the namespace hierarchy, and
1853the fingerprint of each carrier is stored in `++zeno`. These are the
1856A `%king`, a cruiser, is a ship with a 16-bit address. Thus, there are
185765,536 cruisers. Each carrier may issue 256 cruisers. These are the
1858infrastructure of Urbit.
1860A `%duke`, a destroyer, is a ship with a 32-bit address. Thus, there are
18614,294,967,296 destroyers. Each cruiser may issue 65,536 cruisers. These
1862are the individuals of Urbit.
1864A `%earl`, a yacht, is a ship with a 64-bit address. Thus, there are
186518,446,744,073,709,551,616 yachts. Each destroyer may issue
18664,294,967,296 yachts. These are the devices of Urbit.
1868A `%pawn`, a submarine, is a ship with a 128-bit address. Thus, there
1869are a lot of submarines. The chance of random name collision is
1870negligible, so submarines are not issued by any ship. They must simply
1871assert their presence, and they are all considered children of \~zod.
1872This is the underworld of Urbit, where anonymity reigns supreme.
1874### `++what:ames`, logical destroyer identity
1877 ++ what :: logical identity
1878 $% [%anon ~] :: anonymous
1879 [%lady p=whom] :: female person ()
1880 [%lord p=whom] :: male person []
1881 [%punk p=sect q=@t] :: opaque handle ""
1885This is the logical identity of a destroyer.
1887A `%anon` is a completely anonymous destroyer. The difference between
1888this and a submarine is that a submarine is ephemeral while a `%anon`
1889destroyer is not. Thus, we may not know who \~hoclur-bicrel is, but we
1890do know that it's always the same person.
1892A `%lady` is a female person. The name used here should be a real name.
1894A `%lord` is a male person. The name used here should be a real name.
1896A `%punk` is a person who is identified only by a handle.
1898### `++whom:ames`, real person
1901 ++ whom ,[p=@ud q=govt r=sect s=name] :: year/govt/id
1904Ths is the information associated with a real person. It is mostly
1905information that could be observed with the briefest of interactions.
1907`p` is the birth year.
1909`q` is the location of a user, usually of the form "country/zip".
1911`r` is the sect of the user.
1913`s` is the real name of the person.
1918 ++ govt path :: country/postcode
1921This is the location of the user, usually of the form "country/zip".
1926 ++ sect ?(%black %blue %red %orange %white) :: banner
1934 ++ name ,[p=@t q=(unit ,@t) r=(unit ,@t) s=@t] :: first mid/nick last
1937This is the given name, possible middle name/initial, possible nickname,
1938and surname of a user.
keysesc to close
- j k
- move down, up
- enter
- open
- esc
- back, or up a level
- 1 2 3 4
- readme, files, commits, refs
- /
- filter
- y
- copy the clone command
- c
- compare the chosen commit with another (or shift-click both)
- ?
- show or hide this list