@@ -175,16 +175,19 @@ works in the same fashion as TCP, namely exponential backoff.
176Once all packets are received, the receiving Ames decrypts them, deserializes
176Once all packets are received, the receiving Ames decrypts them, deserializes
177them, and puts them back together to form a message.
177them, and puts them back together to form a message.
179−### Acks
181+### Acks and Nacks
181−In this section we discuss acks, short for acknowledgements, which are small
183+In this section we discuss acks and nacks. Ack is short for acknowledgement, which are small
182packets attesting that packets were successfully received. Ames makes use of
184packets attesting that packets were successfully received. Ames makes use of
183−acks maintain synchrony between two communicating parties. Nacks are 'negative
185+acks to maintain synchrony between two communicating parties. Nacks are 'negative
184−acknowledgments' and are used when something goes wrong. This topic is more
186+acknowledgments' and are used when something goes wrong.
185−complicated and so discussion of them is deferred.
187+
187−Every message (i.e. a `%plea` or a `%boon`) is split up by Ames into some number
190+Every message (i.e. a `%plea`, `%boon`, or naxplanation) is split up by Ames into some number
188of _fragments_ that are 1kB in size or less. The fragments are then encrypted
191of _fragments_ that are 1kB in size or less. The fragments are then encrypted
189and encapsulated into
192and encapsulated into
190packets and sent along a flow. The message will be considered successfully
193packets and sent along a flow. The message will be considered successfully
@@ -203,11 +206,31 @@ Acks are considered to be part of the flow in which that `%plea` or
203`%boon` lives, as the packets containing their fragments and packets acking the
206`%boon` lives, as the packets containing their fragments and packets acking the
204receipt of those packets are considered to be what makes up a given message.
207receipt of those packets are considered to be what makes up a given message.
205Thus a message-level ack must be received before the next message on the flow
208Thus a message-level ack must be received before the next message on the flow
206−can begin.
209+can begin. The full story is more complicated than this, see the section on
208−#### Ack packets
215+A nack indicates a negative acknowledgement to a `%plea`, meaning that the
216+requested action was not performed.
218+`%boon`s and naxplanations are never nacked.
219+Individual packets are never nacked, only complete `%plea` messages. This works
220+because only complete packets ever make it to Ames, as malformed
221+packets will be discarded by the Ames I/O module in the
222+[King](#the-serf-and-the-king). Therefore there is no need for Ames to do
223+something like nack a packet whose checksum does not match the `+mug` of the
226+A nack will be accompanied by a naxplanation, which is a third type of message
227+separate from `%plea`s and `%boon`s. Ames won't give the vane that requested the
228+initial `%plea` a nack until it also receives the naxplanation, which it will send to
229+the vane as a `%done` gift. Naxplanations may only be acked, never nacked.
210−The contents of a given ack packet are deterministic. The `content` (i.e. the
233+The contents of a given (n)ack packet are deterministic. The `content` (i.e. the
211encrypted portion of the packet) of a fragment ack is obtained by encrypting the `+jam` of the
234encrypted portion of the packet) of a fragment ack is obtained by encrypting the `+jam` of the
212following noun:
235following noun:
@@ -224,16 +247,69 @@ message in the flow identified by `bone`, and
224`fragment-num` denotes the number of the fragment of the message identified by
247`fragment-num` denotes the number of the fragment of the message identified by
225`message-num` being acked.
248`message-num` being acked.
227−A message ack is an extended fragment ack that is obtained by encrypting the
250+A message (n)ack is an extended fragment ack that is obtained by encrypting the
228`+jam` of the following noun:
251`+jam` of the following noun:
231[our-life her-life bone message-num fragment-num ok=? lag=@dr]
254[our-life her-life bone message-num fragment-num ok=? lag=@dr]
233`ok` is a flag that is set to `%.y` for a message ack and `%.n` for a message
256`ok` is a flag that is set to `%.y` for a message ack and `%.n` for a message
234−nack. We defer discussion of nacks. In the future, `lag` will be used to
257+nack. In the future, `lag` will be used to
235describe the time it took to process a message, but for now it is always zero.
258describe the time it took to process a message, but for now it is always zero.
263+Flows are asymmetric communication channels along which two ships send and
264+receive packets, and all Ames packets are part of some flow.
266+Flows are asymmetric in that one ship is always the sender of
267+`%plea`s and the other is always the sender of `%boon`s on a given flow. Thus ships will
268+typically have multiple flows between them.
270+The sequence of `%plea`s in a flow is totally ordered, though
271+the packets which make them up need not be. `%boon`s are totally ordered in the
272+same fashion, but there is not a coordinated ordering between `%plea`s and
273+`%boon`s in a given flow beyond the implicit one arising from the fact that a
274+`%boon` is always a response to a `%plea`.
276+Inside of Ames, each flow has four sequential opaque `@ud`s
277+called bones that are unique to that flow. Thus the flow itself is often
278+referred to by its first bone. The set of types of packets is partitioned into
279+four subsets, each of which is assigned a bone. Each bone is a one-way street
280+for packets to travel along, so e.g. acks to packets making up a `%plea` are
281+sent along different bones.
283+We give an example of such a partition. Let flow 12 be a flow between `~bacbel-tagfeg` and `~worwel-sipnum` with
284+`~bacbel-tagfeg` as the `%plea` sender. Then bones 12-15 are associated with the
285+flow, and the types of packets for each bone are:
287+ - bone 12, `%plea`s and acks to `~worwel-sipnum`,
288+ - bone 13, `%boon`s and (n)acks to `~bacbel-tagfeg`,
289+ - bone 14, acks of naxplanations to `~worwel-sipnum`,
290+ - bone 15, naxplanations to `~bacbel-tagfeg`.
292+Each bone is handled separately by congestion control, and this is the reason
293+for their segregation. For example, say a two packet `%plea` is sent on bone 12,
294+with `~bacbel-tagfeg` requesting to join a group on `~worwel-sipnum`.
295+Then `~worwel-sipnum` can ack the first packet on bone 13, and then send a nack
296+on bone 13 as well. A nack by itself contains no information as to why the nack
297+happened. Then `~worwel-sipnum` can also send a naxplanation on bone 15 saying
298+to `~bacbel-tagfeg` that they cannot join the group, to which an ack is received
301+`%plea`s and `%boon`s are handled on separate bones so that e.g. sending a large
302+`%boon` doesn't stop an additional `%plea` from being received. It is also
303+simply cleaner to make each bone one-way. Acks and nacks are very small packets
304+as well as integral to controlling the total orderings on `%plea`s and `%boon`s
305+and so they are included on these bones as well.
307+Naxplanations can potentially be very large, as they often contain things like stack
308+traces or other crash reports. Thus it is undesirable to have them share a bone
309+with `%boon`s and create congestion there. So naxplanations have their own bone,
310+and so do acks to packets that make up a naxplanation, as well as the
311+message-level naxplanation ack.
237### Packet relaying
313### Packet relaying
239Here we describes how the Ames network relays packets. We ignore all details
315Here we describes how the Ames network relays packets. We ignore all details