@@ -175,16 +175,19 @@ works in the same fashion as TCP, namely exponential backoff.
175175
176176Once all packets are received, the receiving Ames decrypts them, deserializes
177177them, and puts them back together to form a message.
178+
179+
178180
179−### Acks
181+### Acks and Nacks
180182
181−In this section we discuss acks, short for acknowledgements, which are small
183+In this section we discuss acks and nacks. Ack is short for acknowledgement, which are small
182184packets attesting that packets were successfully received. Ames makes use of
183−acks maintain synchrony between two communicating parties. Nacks are 'negative
184−acknowledgments' and are used when something goes wrong. This topic is more
185−complicated and so discussion of them is deferred.
185+acks to maintain synchrony between two communicating parties. Nacks are 'negative
186+acknowledgments' and are used when something goes wrong.
187+
188+#### Acks
186189
187−Every message (i.e. a `%plea` or a `%boon`) is split up by Ames into some number
190+Every message (i.e. a `%plea`, `%boon`, or naxplanation) is split up by Ames into some number
188191of _fragments_ that are 1kB in size or less. The fragments are then encrypted
189192and encapsulated into
190193packets and sent along a flow. The message will be considered successfully
@@ -203,11 +206,31 @@ Acks are considered to be part of the flow in which that `%plea` or
203206`%boon` lives, as the packets containing their fragments and packets acking the
204207receipt of those packets are considered to be what makes up a given message.
205208Thus a message-level ack must be received before the next message on the flow
206−can begin.
209+can begin. The full story is more complicated than this, see the section on
210+[flows](#flows).
211+
212+
213+#### Nacks
207214
208−#### Ack packets
215+A nack indicates a negative acknowledgement to a `%plea`, meaning that the
216+requested action was not performed.
209217
210−The contents of a given ack packet are deterministic. The `content` (i.e. the
218+`%boon`s and naxplanations are never nacked.
219+Individual packets are never nacked, only complete `%plea` messages. This works
220+because only complete packets ever make it to Ames, as malformed
221+packets will be discarded by the Ames I/O module in the
222+[King](#the-serf-and-the-king). Therefore there is no need for Ames to do
223+something like nack a packet whose checksum does not match the `+mug` of the
224+body.
225+
226+A nack will be accompanied by a naxplanation, which is a third type of message
227+separate from `%plea`s and `%boon`s. Ames won't give the vane that requested the
228+initial `%plea` a nack until it also receives the naxplanation, which it will send to
229+the vane as a `%done` gift. Naxplanations may only be acked, never nacked.
230+
231+#### (N)ack packets
232+
233+The contents of a given (n)ack packet are deterministic. The `content` (i.e. the
211234encrypted portion of the packet) of a fragment ack is obtained by encrypting the `+jam` of the
212235following noun:
213236
@@ -224,16 +247,69 @@ message in the flow identified by `bone`, and
224247`fragment-num` denotes the number of the fragment of the message identified by
225248`message-num` being acked.
226249
227−A message ack is an extended fragment ack that is obtained by encrypting the
250+A message (n)ack is an extended fragment ack that is obtained by encrypting the
228251`+jam` of the following noun:
229252
230253```hoon
231254[our-life her-life bone message-num fragment-num ok=? lag=@dr]
232255```
233256`ok` is a flag that is set to `%.y` for a message ack and `%.n` for a message
234−nack. We defer discussion of nacks. In the future, `lag` will be used to
257+nack. In the future, `lag` will be used to
235258describe the time it took to process a message, but for now it is always zero.
236259
260+
261+### Flows
262+
263+Flows are asymmetric communication channels along which two ships send and
264+receive packets, and all Ames packets are part of some flow.
265+
266+Flows are asymmetric in that one ship is always the sender of
267+`%plea`s and the other is always the sender of `%boon`s on a given flow. Thus ships will
268+typically have multiple flows between them.
269+
270+The sequence of `%plea`s in a flow is totally ordered, though
271+the packets which make them up need not be. `%boon`s are totally ordered in the
272+same fashion, but there is not a coordinated ordering between `%plea`s and
273+`%boon`s in a given flow beyond the implicit one arising from the fact that a
274+`%boon` is always a response to a `%plea`.
275+
276+Inside of Ames, each flow has four sequential opaque `@ud`s
277+called bones that are unique to that flow. Thus the flow itself is often
278+referred to by its first bone. The set of types of packets is partitioned into
279+four subsets, each of which is assigned a bone. Each bone is a one-way street
280+for packets to travel along, so e.g. acks to packets making up a `%plea` are
281+sent along different bones.
282+
283+We give an example of such a partition. Let flow 12 be a flow between `~bacbel-tagfeg` and `~worwel-sipnum` with
284+`~bacbel-tagfeg` as the `%plea` sender. Then bones 12-15 are associated with the
285+flow, and the types of packets for each bone are:
286+
287+ - bone 12, `%plea`s and acks to `~worwel-sipnum`,
288+ - bone 13, `%boon`s and (n)acks to `~bacbel-tagfeg`,
289+ - bone 14, acks of naxplanations to `~worwel-sipnum`,
290+ - bone 15, naxplanations to `~bacbel-tagfeg`.
291+
292+Each bone is handled separately by congestion control, and this is the reason
293+for their segregation. For example, say a two packet `%plea` is sent on bone 12,
294+with `~bacbel-tagfeg` requesting to join a group on `~worwel-sipnum`.
295+Then `~worwel-sipnum` can ack the first packet on bone 13, and then send a nack
296+on bone 13 as well. A nack by itself contains no information as to why the nack
297+happened. Then `~worwel-sipnum` can also send a naxplanation on bone 15 saying
298+to `~bacbel-tagfeg` that they cannot join the group, to which an ack is received
299+on bone 14.
300+
301+`%plea`s and `%boon`s are handled on separate bones so that e.g. sending a large
302+`%boon` doesn't stop an additional `%plea` from being received. It is also
303+simply cleaner to make each bone one-way. Acks and nacks are very small packets
304+as well as integral to controlling the total orderings on `%plea`s and `%boon`s
305+and so they are included on these bones as well.
306+
307+Naxplanations can potentially be very large, as they often contain things like stack
308+traces or other crash reports. Thus it is undesirable to have them share a bone
309+with `%boon`s and create congestion there. So naxplanations have their own bone,
310+and so do acks to packets that make up a naxplanation, as well as the
311+message-level naxplanation ack.
312+
237313### Packet relaying
238314
239315Here we describes how the Ames network relays packets. We ignore all details