@@ -1,2749 +0,0 @@
1−+++
2−title = "Clay"
3−weight = 4
4−template = "doc.html"
5−aliases = ["/docs/learn/arvo/clay/"]
6−+++
7−
8−
9−Our filesystem.
10−
11−`%clay` is version-controlled, referentially-transparent, and global.
12−While this filesystem is stored in `%clay`, it is mirrored to Unix for
13−convenience. Unix tells `%clay`s whenever a file changes in the Unix
14−copy of the filesystem so that the change may be applied. `%clay` tells
15−unix whenever an app or vane changes the filesystem so that the change
16−can be effected in Unix. Apps and vanes may use `%clay` to write to the
17−filesystem, query it, and subscribe to changes in it. Ford and gall use
18−`%clay` to serve up apps and web pages.
19−
20−`%clay` includes three components. First is the filesystem/version
21−control algorithms, which are mostly defined in `++ze` and `++zu` in
22−zuse. Second is the write, query, and subscription logic. Finally, there
23−is the logic for communicating requests to, and receiving requests from,
24−foreign ships.
25−
26−## Merge strategies
27−
28−*This section is up to date as of November 2020*
29−
30−Usage:
31−```
32−|merge %destination-desk ~source-ship %source-desk
33−|merge %destination-desk ~source-ship %source-desk, =gem %strategy
34−|merge %destination-desk ~source-ship %source-desk, =cas ud+5
35−```
36−
37−Clay supports various merge strategies. A "commit" is a snapshot of
38−the files with a list of parents plus a date. Most commits have
39−one parent; a "merge" commit is a commit with two parents. The
40−%home desk starts with an initial commit with no parents; commits
41−with several parents ("octopus merges") are possible but we don't
42−generate them right now.
43−
44−Unless otherwise specified, all of the following create a new commit
45−with the source and destination commits as parents.
46−
47−Several strategies need a "merge-base". They find it by identifying
48−the most recent common ancestor of the two desks. If none, fail
49−with `%merge-no-merge-base`; if there are two or more, pick one.
50−
51−`%init`: the only way to create a desk. Not a true merge, since it
52−simply assigns the source commit to the destination.
53−
54−`%fine`: if source or destination are in the ancestry of each other,
55−use the newer one; else abort. If the destination is ahead of the
56−source, succeed but do nothing. If the source is ahead of the
57−destination, assign the next revision number to the source commit.
58−Some call this "fast-forward".
59−
60−`%meet`: combine changes, failing if both sides changed the same file.
61−Specifically, take diff(merge-base,source) and
62−diff(merge-base,destination) and combine them as long as those diffs
63−touch different files.
64−
65−`%mate`: combine changes, failing if both sides changed the same part
66−of a file. Identical to `%meet`, except that some marks, like `%hoon`,
67−allow intelligent merge of changes to different parts of a file.
68−
69−`%meld`: combine changes; if both sides changed the same part of a
70−file, use the version of the file in the merge-base.
71−
72−`%only-this`: create a merge commit with exactly the contents of the
73−destination desk.
74−
75−`%only-that`: create a merge commit with exactly the contents of the
76−source commit.
77−
78−`%take-this`: create a merge commit with exactly the contents of the
79−destination desk except take any files from the source commit which
80−are not in the destination desk.
81−
82−`%take-that`: create a merge commit with exactly the contents of the
83−source commit except preserve any files from the destination desk
84−which are not in the source commit.
85−
86−`%meet-this`: merge as in `%meet`, except if both sides changed the same
87−file, use the version in the destination desk.
88−
89−`%meet-that`: merge as in `%meet`, except if both sides changed the same
90−file, use the version in the source commit.
91−
92−### Examples and notes:
93−
94−The most common merge strategy is %mate, which is a normal 3-way
95−merge which aborts on conflict.
96−
97−`%take-that` is useful to "force" an OTA. After running `%take-that`,
98−you're guaranteed to have exactly the files in the source commit plus
99−any files you separately added.
100−
101−We speak of merging into a destination *desk* from a source *commit*
102−because while you can only merge on top of a desk, you can merge from
103−historical commits. For example,
104−```
105−|merge %old our %home, =cas ud+5, =gem %init
106−```
107−will create a new desk called `%old` with the 5th commit in `%home`.
108−You can revert the contents of a desk to what they were yesterday
109−with
110−```
111−|merge %home our %home, =cas da+(sub now ~d1), =gem %only-that
112−```
113−
114−Note this is a normal `%only-that merge`, which means you're creating a
115−*new* commit with the old *contents*.
116−
117−`%meld` is rarely used on its own, however if you specify `%auto` or
118−omit the merge strategy, `%kiln` will run a `%meld` merge into a scratch
119−desk and then annotate the conflicts there.
120−
121−If you resolve merge conflicts manually, for example by mounting the
122−desks, copying the files in Unix and then running `|commit`, you
123−should usually run an `%only-this merge`. This will not change the
124−newly-fixed contents of your desk, but it will record that the merge
125−happened so that those conflicts don't reappear in later merges.
126−
127−If you get a `%merge-no-merge-base` error, this means you're trying to
128−merge two desks which have no common ancestors. You need to give
129−them a common ancestor by choosing a merge strategy which doesn't
130−need a merge-base, like `%only-this`, `%only-that`, `%take-this`, or
131−`%take-that`.
132−
133−`%take-this` could be useful to install 3rd party software, but you
134−couldn't get subsequent updates this way, since the files would
135−already exist in the destination desk. Something like "take only
136−the files which aren't in my OTA source or any other 3rd party app"
137−would be basically correct. This would require a parameter listing
138−the desks to not conflict with.
139−
140−`%meet-this` and `%meet-that` imply the existence of `%mate-this` and
141−`%mate-that`, but those don't exist yet.
142−
143−
144−*The remainder of this document is out of date as of November 2020, but certain parts are still
145−accurate. Please independently confirm information here before acting on it.*
146−
147−## Architecture
148−
149−Clay is the primary filesystem for the arvo operating system,
150−which is the [core](/docs/glossary/core/) of an urbit. The architecture of clay is
151−intrinsically connected with arvo, but we assume no knowledge of
152−either arvo or urbit. We will point out only those features of
153−arvo that are necessary for an understanding of clay, and we will
154−do so only when they arise.
155−
156−The first relevant feature of arvo is that it is a deterministic
157−system where input and output are defined as a series of events
158−and effects. The state of arvo is simply a function of its event
159−log. None of the effects from an event are emitted until the
160−event is entered in the log and persisted, either to disk or
161−another trusted source of persistence, such as a Kafka cluster.
162−Consequently, arvo is a single-level store: everything in its
163−state is persistent.
164−
165−In a more traditional OS, everything in RAM can be erased at any
166−time by power failure, and is always erased on reboot. Thus, a
167−primary purpose of a filesystem is to ensure files persist across
168−power failures and reboots. In arvo, both power failures and
169−reboots are special cases of suspending computation, which is
170−done safely since our event log is already persistent. Therefore,
171−clay is not needed in arvo for persistence. Why, then, do we have a
172−filesystem? There are two answers to this question.
173−
174−First, clay provides a filesystem tree, which is a convenient
175−user interface for some applications. Unix has the useful concept
176−of virtual filesystems, which are used for everything from direct
177−access to devices, to random number generators, to the /proc
178−tree. It is easy and intuitive to read from and write to a
179−filesystem tree.
180−
181−Second, clay has a distributed revision-control system baked into
182−it. Traditional filesystems are not revision controlled, so
183−userspace software -- such as git -- is written on top of them to
184−do so. clay natively provides the same functionality as modern
185−DVCSes, and more.
186−
187−clay has two other unique properties that we'll cover later on:
188−it supports typed data and is referentially transparent.
189−
190−### Revision Control
191−
192−Every urbit has one or more "desks", which are independently
193−revision-controlled branches. Each desk contains its own mark
194−definitions, apps, doc, and so forth.
195−
196−Traditionally, an urbit has at least a base and a home desk. The
197−base desk has all the system software from the distribution. the
198−home desk is a fork of base with all the stuff specific to the
199−user of the urbit.
200−
201−A desk is a series of numbered commits, the most recent of which
202−represents the current state of the desk. A commit is composed of
203−(1) an absolute time when it was created, (2) a list of zero or
204−more parents, and (3) a map from paths to data.
205−
206−Most commits have exactly one parent, but the initial commit on a
207−desk may have zero parents, and merge commits have more than one
208−parent.
209−
210−The non-meta data is stored in the map of paths to data. It's
211−worth noting that no constraints are put on this map, so, for
212−example, both /a/b and /a/b/c could have data. This is impossible
213−in a traditional Unix filesystem since it means that /a/b is both
214−a file and a directory. Conventionally, the final element in the
215−path is its mark -- much like a filename extension in Unix. Thus,
216−/doc/readme.md in Unix is stored as /doc/readme/md in urbit.
217−
218−The data is not stored directly in the map; rather, a hash of the
219−data is stored, and we maintain a master blob store. Thus, if the
220−same data is referred to in multiple commits (as, for example,
221−when a file doesn't change between commits), only the hash is
222−duplicated.
223−
224−In the master blob store, we either store the data directly, or
225−else we store a diff against another blob. The hash is dependent
226−only on the data within and not on whether or not it's stored
227−directly, so we may on occasion rearrange the contents of the
228−blob store for performance reasons.
229−
230−Recall that a desk is a series of numbered commits. Not every
231−commit in a desk must be numbered. For example, if the base desk
232−has had 50 commits since home was forked from it, then a merge
233−from base to home will only add a single revision number to home,
234−although the full commit history will be accessible by traversing
235−the parentage of the individual commits.
236−
237−We do guarantee that the first commit is numbered 1, commits are
238−numbered consecutively after that (i.e. there are no "holes"),
239−the topmost commit is always numbered, and every numbered commit
240−is an ancestor of every later numbered commit.
241−
242−There are three ways to refer to particular commits in the
243−revision history. Firstly, one can use the revision number.
244−Secondly, one can use any absolute time between the one numbered
245−commit and the next (inclusive of the first, exclusive of the
246−second). Thirdly, every desk has a map of labels to revision
247−numbers. These labels may be used to refer to specific commits.
248−
249−Additionally, clay is a global filesystem, so data on other urbit
250−is easily accessible the same way as data on our local urbit. In
251−general, the path to a particular revision of a desk is
252−/~urbit-name/desk-name/revision. Thus, to get /try/readme/md
253−from revision 5 of the home desk on ~sampel-sipnym, we refer to
254−/~sampel-sipnym/home/5/try/readme/md. Clay's namespace is thus
255−global and referentially transparent.
256−
257−XXX reactivity here?
258−
259−### A Typed Filesystem
260−
261−Since clay is a general filesystem for storing data of arbitrary
262−types, in order to revision control correctly it needs to be
263−aware of types all the way through. Traditional revision control
264−does an excellent job of handling source code, so for source code
265−we act very similar to traditional revision control. The
266−challenge is to handle other data similarly well.
267−
268−For example, modern VCSs generally support "binary files", which
269−are files for which the standard textual diffing, patching, and
270−merging algorithms are not helpful. A "diff" of two binary files
271−is just a pair of the files, "patching" this diff is just
272−replacing the old file with the new one, and "merging"
273−non-identical diffs is always a conflict, which can't even be
274−helpfully annotated. Without knowing anything about the structure
275−of a blob of data, this is the best we can do.
276−
277−Often, though, "binary" files have some internal structure, and
278−it is possible to create diff, patch, and merge algorithms that
279−take advantage of this structure. An image may be the result of a
280−base image with some set of operations applied. With algorithms
281−aware of this set of operations, not only can revision control
282−software save space by not having to save every revision of the
283−image individually, these transformations can be made on parallel
284−branches and merged at will.
285−
286−Suppose Alice is tasked with touching up a picture, improving the
287−color balance, adjusting the contrast, and so forth, while Bob
288−has the job of cropping the picture to fit where it's needed and
289−adding textual overlay. Without type-aware revision control,
290−these changes must be made serially, requiring Alice and Bob to
291−explicitly coordinate their efforts. With type-aware revision
292−control, these operations may be performed in parallel, and then
293−the two changesets can be merged programmatically.
294−
295−Of course, even some kinds of text files may be better served by
296−diff, patch, and merge algorithms aware of the structure of the
297−files. Consider a file containing a pretty-printed JSON object.
298−Small changes in the JSON object may result in rather significant
299−changes in how the object is pretty-printed (for example, by
300−addding an indentation level, splitting a single line into
301−multiple lines).
302−
303−A text file wrapped at 80 columns also reacts suboptimally with
304−unadorned Hunt-McIlroy diffs. A single word inserted in a
305−paragraph may push the final word or two of the line onto the
306−next line, and the entire rest of the paragraph may be flagged as
307−a change. Two diffs consisting of a single added word to
308−different sentences may be flagged as a conflict. In general,
309−prose should be diffed by sentence, not by line.
310−
311−As far as the author is aware, clay is the first generalized,
312−type-aware revision control system. We'll go into the workings
313−of this system in some detail.
314−
315−### Marks
316−
317−Central to a typed filesystem is the idea of types. In clay, we
318−call these "marks". A mark is a file that defines a type,
319−conversion routines to and from the mark, and diff, patch, and
320−merge routines.
321−
322−For example, a `%txt` mark may be a list of lines of text, and it
323−may include conversions to `%mime` to allow it to be serialized
324−and sent to a browswer or to the unix filesystem. It will also
325−include Hunt-McIlroy diff, patch, and merge algorithms.
326−
327−A `%json` mark would be defined as a json object in the code, and
328−it would have a parser to convert from `%txt` and a printer to
329−convert back to `%txt`. The diff, patch, and merge algorithms are
330−fairly straightforward for json, though they're very different
331−from the text ones.
332−
333−More formally, a mark is a core with three [arms](/docs/glossary/arm/), `++grab`,
334−`++grow`, and `++grad`. In `++grab` is a series of functions to
335−convert from other marks to the given mark. In `++grow` is a
336−series of functions to convert from the given mark to other
337−marks. In `++grad` is `++diff`, `++pact`, `++join`, and `++mash`.
338−
339−The types are as follows, in an informal pseudocode:
340−
341−```
342− ++ grab:
343− ++ mime: <mime> -> <mark-type>
344− ++ txt: <txt> -> <mark-type>
345− ...
346− ++ grow:
347− ++ mime: <mark-type> -> <mime>
348− ++ txt: <mark-type> -> <txt>
349− ...
350− ++ grad
351− ++ diff: (<mark-type>, <mark-type>) -> <diff-type>
352− ++ pact: (<mark-type>, <diff-type>) -> <mark-type>
353− ++ join: (<diff-type>, <diff-type>) -> <diff-type> or NULL
354− ++ mash: (<diff-type>, <diff-type>) -> <diff-type>
355−```
356−
357−These types are basically what you would expect. Not every mark
358−has each of these functions defined -- all of them are optional
359−in the general case.
360−
361−In general, for a particular mark, the `++grab` and `++grow` entries
362−(if they exist) should be inverses of each other.
363−
364−In `++grad`, `++diff` takes two instances of a mark and produces
365−a diff of them. `++pact` takes an instance of a mark and patches
366−it with the given diff. `++join` takes two diffs and attempts to
367−merge them into a single diff. If there are conflicts, it
368−produces null. `++mash` takes two diffs and forces a merge,
369−annotating any conflicts.
370−
371−In general, if `++diff` called with A and B produces diff D, then
372−`++pact` called with A and D should produce B. Also, if `++join`
373−of two diffs does not produce null, then `++mash` of the same
374−diffs should produce the same result.
375−
376−Alternately, instead of `++diff`, `++pact`, `++join`, and
377−`++mash`, a mark can provide the same functionality by defining
378−`++sted` to be the name of another mark to which we wish to
379−delegate the revision control responsibilities. Then, before
380−running any of those functions, clay will convert to the other
381−mark, and convert back afterward. For example, the `%hoon` mark
382−is revision-controlled in the same way as `%txt`, so its `++grad`
383−is simply `++sted %txt`. Of course, `++txt` must be defined in
384−`++grow` and `++grab` as well.
385−
386−Every file in clay has a mark, and that mark must have a
387−fully-functioning `++grad`. Marks are used for more than just
388−clay, and other marks don't need a `++grad`, but if a piece of
389−data is to be saved to clay, we must know how to revision-control
390−it.
391−
392−Additionally, if a file is to be synced out to unix, then it must
393−have conversion routines to and from the `%mime` mark.
394−
395−## Using clay
396−
397−### Reading and Subscribing
398−
399−When reading from Clay, there are three types of requests. A
400−`%sing` request asks for data at single revsion. A `%next`
401−request asks to be notified the next time there's a change to
402−given file. A `%many` request asks to be notified on every
403−change in a desk for a range of changes.
404−
405−For `%sing` and `%next`, there are generally three things to be
406−queried. A `%u` request simply checks for the existence of a
407−file at a path. A `%x` request gets the data in the file at a
408−path. A `%y` request gets a hash of the data in the file at the
409−path combined with all its children and their data. Thus, `%y`
410−of a node changes if it or any of its children change.
411−
412−A `%sing` request is fulfilled immediately if possible. If the
413−requested revision is in the future, or is on another ship for
414−which we don't have the result cached, we don't respond
415−immediately. If the requested revision is in the future, we wait
416−until the revision happens before we respond to the request. If
417−the request is for data on another ship, we pass on the request
418−to the other ship. In general, Clay subscriptions, like most
419−things in Urbit, aren't guaranteed to return immediately.
420−They'll return when they can, and they'll do so in a
421−referentially transparent manner.
422−
423−A `%next` request checks query at the given revision, and it
424−produces the result of the query the next time it changes, along
425−with the revsion number when it changes. Thus, a `%next` of a
426−`%u` is triggered when a file is added or deleted, a `%next` of a
427−`%x` is triggered when a file is added, deleted, or changed, and
428−a `%next` of a `%y` is triggered when a file or any of its
429−children is added, deleted, or changed.
430−
431−A `%many` request is triggered every time the given desk has a
432−new revision. Unlike a `%next`, a `%many` has both a start and
433−an end revsion, after which it stops returning. For `%next`, a
434−single change is reported, and if the caller wishes to hear of
435−the next change, it must resubscribe. For `%many`, every revsion
436−from the start to the end triggers a response. Since a `%many`
437−request doesn't ask for any particular data, there aren't `%u`,
438−`%x`, and `%y` versions for it.
439−
440−### Unix sync
441−
442−One of the primary functions of clay is as a convenient user
443−interface. While tools exist to use clay from within urbit, it's
444−often useful to be able to treat clay like any other filesystem
445−from the Unix perspective -- to "mount" it, as it were.
446−
447−From urbit, you can run `|mount /path/to/directory %mount-point`,
448−and this will mount the given clay directory to the mount-point
449−directory in Unix. Every file is converted to `%mime` before it's
450−written to Unix, and converted back when read from Unix. The
451−entire directory is watched (a la Dropbox), and every change is
452−committed once you run `|commit %mount-point`.
453−
454−### Merging
455−
456−Merging is a fundamental operation for a distributed revision
457−control system. At their root, clay's merges are similar to
458−git's, but with some additions to accomodate typed data. There
459−are seven different merge strategies.
460−
461−Throughout our discussion, we'll say that the merge is from
462−Alice's desk to Bob's. Recall that a commit is a date (for all
463−new commits this will be the current date), a list of parents,
464−and the data itself.
465−
466−A `%init` merge should be used iff it's the first commit to a
467−desk. The head of Alice's desk is used as the number 1 commit to
468−Bob's desk. Obviously, the ancestry remains intact through
469−traversing the parentage of the commit even though previous
470−commits are not numbered for Bob's desk.
471−
472−A `%this` merge means to keep what's in Bob's desk, but join the
473−ancestry. Thus, the new commit has the head of each desk as
474−parents, but the data is exactly what's in Bob's desk. For those
475−following along in git, this is the 'ours' merge strategy, not
476−the '--ours' option to the 'recursive' merge strategy. In other
477−words, even if Alice makes a change that does not conflict with
478−Bob, we throw it away. It's Bob's way or the highway.
479−
480−A `%that` merge means to take what's in Alice's desk, but join
481−the ancestry. This is the reverse of `%this`.
482−
483−A `%fine` merge is a "fast-forward" merge. This succeeds iff one
484−head is in the ancestry of the other. In this case, we use the
485−descendant as our new head.
486−
487−For `%meet`, `%mate`, and `%meld` merges, we first find the most
488−recent common ancestor to use as our merge base. If we have no
489−common ancestors, then we fail. If we have more than one most
490−recent common ancestor, then we have a criss-cross situation,
491−which should be handled delicately. At present, we delicately
492−throw up our hands and give up, but something akin to git's
493−'recursive' strategy should be implemented in the future.
494−
495−There's a functional inclusion ordering on `%fine`, `%meet`,
496−`%mate`, and `%meld` such that if an earlier strategy would have
497−succeeded, then every later strategy will produce the same
498−result. Put another way, every earlier strategy is the same as
499−every later strategy except with a restricted domain.
500−
501−A `%meet` merge only succeeds if the changes from the merge base
502−to Alice's head (hereafter, "Alice's changes") are in different
503−files than Bob's changes. In this case, the parents are both
504−Alice's and Bob's heads, and the data is the merge base plus
505−Alice's changed files plus Bob's changed files.
506−
507−A `%mate` merge attempts to merge changes to the same file when
508−both Alice and bob change it. If the merge is clean, we use it;
509−otherwise, we fail. A merge between different types of changes --
510−for example, deleting a file vs changing it -- is always a
511−conflict. If we succeed, the parents are both Alice's and Bob's
512−heads, and the data is the merge base plus Alice's changed files
513−plus Bob's changed files plus the merged files.
514−
515−A `%meld` merge will succeed even if there are conflicts. If
516−there are conflicts in a file, then we use the merge base's
517−version of that file, and we produce a set of files with
518−conflicts. The parents are both Alice's and Bob's heads, and the
519−data is the merge base plus Alice's changed files plus Bob's
520−changed files plus the successfully merged files plus the merge
521−base's version of the conflicting files.
522−
523−That's the extent of the merge options in clay proper. In
524−userspace there's a final option `%auto`, which is the most
525−common. `%auto` checks to see if Bob's desk exists, and if it
526−doesn't we use a `%init` merge. Otherwise, we progressively try
527−`%fine`, `%meet`, and `%mate` until one succeeds.
528−
529−If none succeed, we merge Bob's desk into a scratch desk. Then,
530−we merge Alice's desk into the scratch desk with the `%meld`
531−option to force the merge. For each file in the produced set of
532−conflicting files, we call the `++mash` function for the
533−appropriate mark, which annotates the conflicts if we know how.
534−
535−Finally, we display a message to the user informing them of the
536−scratch desk's existence, which files have annotated conflicts,
537−and which files have unannotated conflicts. When the user has
538−resolved the conflicts, they can merge the scratch desk back into
539−Bob's desk. This will be a `%fine` merge since Bob's head is in
540−the ancestry of the scratch desk.
541−
542−### Autosync
543−
544−Tracking and staying in sync with another desk is another
545−fundamental operation. We call this "autosync". This doesn't mean
546−simply mirroring a desk, since that wouldn't allow local changes.
547−We simply want to apply changes as they are made upstream, as
548−long as there are no conflicts with local changes.
549−
550−This is implemented by watching the other desk, and, when it has
551−changes, merging these changes into our desk with the usual merge
552−strategies.
553−
554−Note that it's quite reasonable for two desks to be autosynced to
555−each other. This results in any change on one desk being mirrored
556−to the other and vice versa.
557−
558−Additionally, it's fine to set up an autosync even if one desk,
559−the other desk, or both desks do not exist. The sync will be
560−activated when the upstream desk comes into existence and will
561−create the downstream desk if needed.
562−
563−## Commentary
564−
565−The first part of this section will be reference documentation for the data
566−types used by our filesystem. In fact, as a general guide, we recommend
567−reading and attempting to understand the data structures used in any
568−Hoon code before you try to read the code itself. Although complete
569−understanding of the data structures is impossible without seeing them
570−used in the code, an 80% understanding greatly clarifies the code. As
571−another general guide, when reading Hoon, it rarely pays off to
572−understand every line of code when it appears. Try to get the gist of
573−it, and then move on. The next time you come back to it, it'll likely
574−make a lot more sense.
575−
576−After a description of the data models, we'll give an overview of the
577−interface that vanes and applications can use to interact with the
578−filesystem.
579−
580−Finally, we'll dive into the code and the algorithms themselves. You
581−know, the fun part.
582−
583−### Data Models
584−
585−As you're reading through this section, remember you can always come
586−back to this when you run into these types later on. You're not going to
587−remember everything the first time through, but it is worth reading, or
588−at least skimming, this so that you get a rough idea of how our state is
589−organized.
590−
591−The types that are certainly worth reading are `++raft`, `++room`,
592−`++dome:clay`, `++ankh:clay`, `++rung:clay`, `++rang:clay`, `++blob:clay`, `++yaki:clay`, and `++nori:clay`
593−(possibly in that order). All in all, though, this section isn't too
594−long, so many readers may wish to quickly read through all of it. If you
595−get bored, though, just skip to the next section. You can always come
596−back when you need to.
597−
598−### `++raft`, formal state
599−
600−```hoon
601− ++ raft :: filesystem
602− $: fat=(map ship room) :: domestic
603− hoy=(map ship rung) :: foreign
604− ran=rang :: hashes
605− == ::
606−```
607−
608−This is the state of our vane. Anything that must be remembered between
609−calls to clay is stored in this state.
610−
611−`fat` is the set of domestic servers. This stores all the information
612−that is specfic to a particular ship on this pier. The keys to this map
613−are the ships on the current pier. all the information that is specific
614−to a particular foreign ship. The keys to this map are all the ships
615−whose filesystems we have attempted to access through clay.
616−
617−`ran` is the store of all commits and deltas, keyed by hash. The is
618−where all the "real" data we know is stored; the rest is "just
619−bookkeeping".
620−
621−#### `++room`, filesystem per domestic ship
622−
623−```hoon
624− ++ room :: fs per ship
625− $: hun=duct :: terminal duct
626− hez=(unit duct) :: sync duch
627− dos=(map desk dojo) :: native desk
628− == ::
629−```
630−
631−This is the representation of the filesystem of a ship on our pier.
632−
633−`hun` is the duct we use to send messages to dill to display
634−notifications of filesystem changes. Only `%note` gifts should be
635−produced along this duct. This is set by the `%init` kiss.
636−
637−`hez`, if present, is the duct we use to send sync messages to unix so
638−that they end up in the pier unix directory. Only `%ergo` gifts should
639−be producd along this duct. This is set by `%into` and `%invo` kisses.
640−
641−`dos` is a well-known operating system released in 1981. It is also the
642−set of desks on this ship, mapped to their data.
643−
644−#### `++desk`, filesystem branch
645−
646−```hoon
647− ++ desk ,@tas :: ship desk case spur
648−```
649−
650−This is the name of a branch of the filesystem. The default desks are
651−"arvo", "main", and "try". More may be created by simply referencing
652−them. Desks have independent histories and states, and they may be
653−merged into each other.
654−
655−### `++dojo`, domestic desk state
656−
657−```hoon
658− ++ dojo ,[p=cult q=dome] :: domestic desk state
659−```
660−
661−This is the all the data that is specific to a particular desk on a
662−domestic ship. `p` is the set of subscribers to this desk and `q` is the
663−data in the desk.
664−
665−#### `++cult`, subscriptions
666−
667−```hoon
668− ++ cult (map duct rave) :: subscriptions
669−```
670−
671−This is the set of subscriptions to a particular desk. The keys are the
672−ducts from where the subscriptions requests came. The results will be
673−produced along these ducts. The values are a description of the
674−requested information.
675−
676−#### `++rave:clay`, general subscription request
677−
678−```hoon
679− ++ rave :: general request
680− $% [& p=mood] :: single request
681− [| p=moat] :: change range
682− == ::
683−```
684−
685−This represents a subscription request for a desk. The request can be
686−for either a single item in the desk or else for a range of changes on
687−the desk.
688−
689−### `++rove`, stored general subscription request
690−
691−```hoon
692− ++ rove (each mood moot) :: stored request
693−```
694−
695−When we store a request, we store subscriptions with a little extra
696−information so that we can determine whether new versions actually
697−affect the path we're subscribed to.
698−
699−#### `++mood:clay`, single subscription request
700−
701−```hoon
702− ++ mood ,[p=care q=case r=path] :: request in desk
703−```
704−
705−This represents a request for the state of the desk at a particular
706−commit, specfied by `q`. `p` specifies what kind of information is
707−desired, and `r` specifies the path we are requesting.
708−
709−#### `++moat:clay`, range subscription request
710−
711−```hoon
712− ++ moat ,[p=case q=case r=path] :: change range
713−```
714−
715−This represents a request for all changes between `p` and `q` on path
716−`r`. You will be notified when a change is made to the node referenced
717−by the path or to any of its children.
718−
719−### `++moot`, stored range subscription request
720−
721−```hoon
722− ++ moot ,[p=case q=case r=path s=(map path lobe)] ::
723−```
724−
725−This is just a `++moat:clay` plus a map of paths to lobes. This map
726−represents the data at the node referenced by the path at case `p`, if
727−we've gotten to that case (else null). We only send a notification along
728−the subscription if the data at a new revision is different than it was.
729−
730−### `++care:clay`, clay submode
731−
732−```hoon
733− ++ care ?(%u %v %w %x %y %z) :: clay submode
734−```
735−
736−This specifies what type of information is requested in a subscription
737−or a scry.
738−
739−`%u` requests the `++rang:clay` at the current moment. Because this
740−information is not stored for any moment other than the present, we
741−crash if the `++case:clay` is not a `%da` for now.
742−
743−`%v` requests the `++dome:clay` at the specified commit.
744−
745−`%w` requests the revsion number of the desk.
746−
747−`%x` requests the file at a specified path at the specified commit. If
748−there is no node at that path or if the node has no contents (that is,
749−if `q:ankh` is null), then this produces null.
750−
751−`%y` requests a `++arch` of the specfied commit at the specified path.
752−
753−`%z` requests the `++ankh` of the specified commit at the specfied path.
754−
755−### `++arch`, shallow filesystem node
756−
757−```hoon
758− ++ arch ,[p=@uvI q=(unit ,@uvI) r=(map ,@ta ,~)] :: fundamental node
759−```
760−
761−This is analogous to `++ankh:clay` except that the we have neither our
762−contents nor the ankhs of our children. The other fields are exactly the
763−same, so `p` is a hash of the associated ankh, `u.q`, if it exists, is a
764−hash of the contents of this node, and the keys of `r` are the names of
765−our children. `r` is a map to null rather than a set so that the
766−ordering of the map will be equivalent to that of `r:ankh`, allowing
767−efficient conversion.
768−
769−#### `++case:clay`, specifying a commit
770−
771−```hoon
772− ++ case :: ship desk case spur
773− $% [%da p=@da] :: date
774− [%tas p=@tas] :: label
775− [%ud p=@ud] :: number
776− == ::
777−```
778−
779−A commit can be referred to in three ways: `%da` refers to the commit
780−that was at the head on date `p`, `%tas` refers to the commit labeled
781−`p`, and `%ud` refers to the commit numbered `p`. Note that since these
782−all can be reduced down to a `%ud`, only numbered commits may be
783−referenced with a `++case:clay`.
784−
785−#### `++dome:clay`, desk data
786−
787−```hoon
788− ++ dome :: project state
789− $: ang=agon :: pedigree
790− ank=ankh :: state
791− let=@ud :: top id
792− hit=(map ,@ud tako) :: changes by id
793− lab=(map ,@tas ,@ud) :: labels
794− == ::
795−```
796−
797−This is the data that is actually stored in a desk.
798−
799−`ang` is unused and should be removed.
800−
801−`ank` is the current state of the desk. Thus, it is the state of the
802−filesystem at revison `let`. The head of a desk is always a numbered
803−commit.
804−
805−`let` is the number of the most recently numbered commit. This is also
806−the total number of numbered commits.
807−
808−`hit` is a map of numerical ids to hashes of commits. These hashes are
809−mapped into their associated commits in `hut:rang:clay`. In general, the keys
810−of this map are exactly the numbers from 1 to `let`, with no gaps. Of
811−course, when there are no numbered commits, `let` is 0, so `hit` is
812−null. Additionally, each of the commits is an ancestor of every commit
813−numbered greater than this one. Thus, each is a descendant of every
814−commit numbered less than this one. Since it is true that the date in
815−each commit (`t:yaki`) is no earlier than that of each of its parents,
816−the numbered commits are totally ordered in the same way by both
817−pedigree and date. Of course, not every commit is numbered. If that
818−sounds too complicated to you, don't worry about it. It basically
819−behaves exactly as you would expect.
820−
821−`lab` is a map of textual labels to numbered commits. Note that labels
822−can only be applied to numbered commits. Labels must be unique across a
823−desk.
824−
825−#### `++ankh`, filesystem node
826−
827−```hoon
828− ++ ankh :: fs node (new)
829− $: p=cash :: recursive hash
830− q=(unit ,[p=cash q=*]) :: file
831− r=(map ,@ta ankh) :: folders
832− == ::
833−```
834−
835−This is a single node in the filesystem. This may be file or a directory
836−or both. In earth filesystems, a node is a file xor a directory. On
837−mars, we're inclusive, so a node is a file ior a directory.
838−
839−`p` is a recursive hash that depends on the contents of the this file or
840−directory and on any children.
841−
842−`q` is the contents of this file, if any. `p.q` is a hash of the
843−contents while `q.q` is the data itself.
844−
845−`r` is the set of children of this node. In the case of a pure file,
846−this is empty. The keys are the names of the children and the values
847−are, recursively, the nodes themselves.
848−
849−#### `++cash`, ankh hash
850−
851−```hoon
852− ++ cash ,@uvH :: ankh hash
853−```
854−
855−This is a 128-bit hash of an ankh. These are mostly stored within ankhs
856−themselves, and they are used to check for changes in possibly-deep
857−hierarchies.
858−
859−#### `++rung`, filesystem per neighbor ship
860−
861−```hoon
862− ++ rung $: rus=(map desk rede) :: neighbor desks
863− == ::
864−```
865−
866−This is the filesystem of a neighbor ship. The keys to this map are all
867−the desks we know about on their ship.
868−
869−#### `++rede`, desk state
870−
871−```hoon
872− ++ rede :: universal project
873− $: lim=@da :: complete to
874− qyx=cult :: subscribers
875− ref=(unit rind) :: outgoing requests
876− dom=dome :: revision state
877− == ::
878−```
879−
880−This is our knowledge of the state of a desk, either foreign or
881−domestic.
882−
883−`lim` is the date of the last full update. We only respond to requests
884−for stuff before this time.
885−
886−`qyx` is the list of subscribers to this desk. For domestic desks, this
887−is simply `p:dojo`, all subscribers to the desk, while in foreign desks
888−this is all the subscribers from our ship to the foreign desk.
889−
890−`ref` is the request manager for the desk. For domestic desks, this is
891−null since we handle requests ourselves.
892−
893−`dom` is the actual data in the desk.
894−
895−#### `++rind`, request manager
896−
897−```hoon
898− ++ rind :: request manager
899− $: nix=@ud :: request index
900− bom=(map ,@ud ,[p=duct q=rave]) :: outstanding
901− fod=(map duct ,@ud) :: current requests
902− haw=(map mood (unit)) :: simple cache
903− == ::
904−```
905−
906−This is the request manager for a foreign desk.
907−
908−`nix` is one more than the index of the most recent request. Thus, it is
909−the next available request number.
910−
911−`bom` is the set of outstanding requests. The keys of this map are some
912−subset of the numbers between 0 and one less than `nix`. The members of
913−the map are exactly those requests that have not yet been fully
914−satisfied.
915−
916−`fod` is the same set as `bom`, but from a different perspective. In
917−particular, the values of `fod` are the same as the values of `bom`, and
918−the `p` out of the values of `bom` are the same as the keys of `fod`.
919−Thus, we can map ducts to their associated request number and `++rave:clay`,
920−and we can map numbers to their associated duct and `++rave:clay`.
921−
922−`haw` is a map from simple requests to their values. This acts as a
923−cache for requests that have already been made. Thus, the second request
924−for a particular `++mood:clay` is nearly instantaneous.
925−
926−#### `++rang:clay`, data store
927−
928−```hoon
929− ++ rang $: hut=(map tako yaki) ::
930− lat=(map lobe blob) ::
931− == ::
932−```
933−
934−This is a set of data keyed by hash. Thus, this is where the "real" data
935−is stored, but it is only meaningful if we know the hash of what we're
936−looking for.
937−
938−`hut` is a map from hashes to commits. We often get the hashes from
939−`hit:dome:clay`, which keys them by logical id. Not every commit has an id.
940−
941−`lat` is a map from hashes to the actual data. We often get the hashes
942−from a `++yaki`, a commit, which references this map to get the data.
943−There is no `++blob:clay` in any `++yaki:clay`. They are only accessible through
944−this map.
945−
946−#### `++tako:clay`, commit reference
947−
948−```hoon
949− ++ tako ,@ :: yaki ref
950−```
951−
952−This is a hash of a `++yaki:clay`, a commit. These are most notably used as
953−the keys in `hut:rang:clay`, where they are associated with the actual
954−`++yaki:clay`, and as the values in `hit:dome:clay`, where sequential ids are
955−associated with these.
956−
957−#### `++yaki:clay`, commit
958−
959−```hoon
960− ++ yaki ,[p=(list tako) q=(map path lobe) r=tako t=@da] :: commit
961−```
962−
963−This is a single commit.
964−
965−`p` is a list of the hashes of the parents of this commit. In most
966−cases, this will be a single commit, but in a merge there may be more
967−parents. In theory, there may be an arbitrary number of parents, but in
968−practice merges have exactly two parents. This may change in the future.
969−For commit 1, there is no parent.
970−
971−`q` is a map of the paths on a desk to the data at that location. If you
972−understand what a `++lobe:clay` and a `++blob:clay` is, then the type signature
973−here tells the whole story.
974−
975−`r` is the hash associated with this commit.
976−
977−`t` is the date at which this commit was made.
978−
979−#### `++lobe:clay`, data reference
980−
981−```hoon
982− ++ lobe ,@ :: blob ref
983−```
984−
985−This is a hash of a `++blob:clay`. These are most notably used in `lat:rang:clay`,
986−where they are associated with the actual `++blob:clay`, and as the values in
987−`q:yaki:clay`, where paths are associated with their data in a commit.
988−
989−#### `++blob:clay`, data
990−
991−```hoon
992− ++ blob $% [%delta p=lobe q=lobe r=udon] :: delta on q
993− [%direct p=lobe q=* r=umph] ::
994− [%indirect p=lobe q=* r=udon s=lobe] ::
995− == ::
996−```
997−
998−This is a node of data. In every case, `p` is the hash of the blob.
999−
1000−`%delta` is the case where we define the data by a delta on other data.
1001−In practice, the other data is always the previous commit, but nothing
1002−depends on this. `q` is the hash of the parent blob, and `r` is the
1003−delta.
1004−
1005−`%direct` is the case where we simply have the data directly. `q` is the
1006−data itself, and `r` is any preprocessing instructions. These almost
1007−always come from the creation of a file.
1008−
1009−`%indirect` is both of the preceding cases at once. `q` is the direct
1010−data, `r` is the delta, and `s` is the parent blob. It should always be
1011−the case that applying `r` to `s` gives the same data as `q` directly
1012−(with the prepreprocessor instructions in `p.r`). This exists purely for
1013−performance reasons. This is unused, at the moment, but in general these
1014−should be created when there are a long line of changes so that we do
1015−not have to traverse the delta chain back to the creation of the file.
1016−
1017−#### `++udon`, abstract delta
1018−
1019−```hoon
1020− ++ udon :: abstract delta
1021− $: p=umph :: preprocessor
1022− $= q :: patch
1023− $% [%a p=* q=*] :: trivial replace
1024− [%b p=udal] :: atomic indel
1025− [%c p=(urge)] :: list indel
1026− [%d p=upas q=upas] :: tree edit
1027− == ::
1028− == ::
1029−```
1030−
1031−This is an abstract change to a file. This is a superset of what would
1032−normally be called diffs. Diffs usually refer to changes in lines of
1033−text while we have the ability to do more interesting deltas on
1034−arbitrary data structures.
1035−
1036−`p` is any preprocessor instructions.
1037−
1038−`%a` refers to the trival delta of a complete replace of old data with
1039−new data.
1040−
1041−`%b` refers to changes in an opaque atom on the block level. This has
1042−very limited usefulness, and is not used at the moment.
1043−
1044−`%c` refers to changes in a list of data. This is often lines of text,
1045−which is your classic diff. We, however, will work on any list of data.
1046−
1047−`%d` refers to changes in a tree of data. This is general enough to
1048−describe changes to any hoon [noun](/docs/glossary/noun/), but often more special-purpose delta
1049−should be created for different content types. This is not used at the
1050−moment, and may in fact be unimplemented.
1051−
1052−#### `++urge`, list change
1053−
1054−```hoon
1055− ++ urge |*(a=_,* (list (unce a))) :: list change
1056−```
1057−
1058−This is a parametrized type for list changes. For example, `(urge ,@t)`
1059−is a list change for lines of text.
1060−
1061−#### `++unce`, change part of a list.
1062−
1063−```hoon
1064− ++ unce |* a=_,* :: change part
1065− $% [%& p=@ud] :: skip[copy]
1066− [%| p=(list a) q=(list a)] :: p -> q[chunk]
1067− == ::
1068−```
1069−
1070−This is a single change in a list of elements of type `a`. For example,
1071−`(unce ,@t)` is a single change in a lines of text.
1072−
1073−`%&` means the next `p` lines are unchanged.
1074−
1075−`%|` means the lines `p` have changed to `q`.
1076−
1077−#### `++umph`, preprocessing information
1078−
1079−```hoon
1080− ++ umph :: change filter
1081− $| $? %a :: no filter
1082− %b :: jamfile
1083− %c :: LF text
1084− == ::
1085− $% [%d p=@ud] :: blocklist
1086− == ::
1087−```
1088−
1089−This space intentionally left undocumented. This stuff will change once
1090−we get a well-typed clay.
1091−
1092−#### `++upas`, tree change
1093−
1094−```hoon
1095− ++ upas :: tree change (%d)
1096− $& [p=upas q=upas] :: cell
1097− $% [%0 p=axis] :: copy old
1098− [%1 p=*] :: insert new
1099− [%2 p=axis q=udon] :: mutate!
1100− == ::
1101−```
1102−
1103−This space intentionally left undocumented. This stuff is not known to
1104−work, and will likely change when we get a well-typed clay. Also, this
1105−is not a complicated type; it is not difficult to work out the meaning.
1106−
1107−#### `++nori:clay`, repository action
1108−
1109−```hoon
1110− ++ nori :: repository action
1111− $% [& q=soba] :: delta
1112− [| p=@tas] :: label
1113− == ::
1114−```
1115−
1116−This describes a change that we are asking clay to make to the desk.
1117−There are two kinds of changes that may be made: we can modify files or
1118−we can apply a label to a commit.
1119−
1120−In the `|` case, we will simply label the current commit with the given
1121−label. In the `&` case, we will apply the given changes.
1122−
1123−#### `++soba:clay`, delta
1124−
1125−```hoon
1126− ++ soba ,[p=cart q=(list ,[p=path q=miso])] :: delta
1127−```
1128−
1129−This describes a set of changes to make to a desk. The `cart` is simply
1130−a pair of the old hash and the new hash of the desk. The list is a list
1131−of changes keyed by the file they're changing. Thus, the paths are paths
1132−to files to be changed while `miso` is a description of the change
1133−itself.
1134−
1135−#### `++miso:clay`, ankh delta
1136−
1137−```hoon
1138− ++ miso :: ankh delta
1139− $% [%del p=*] :: delete
1140− [%ins p=*] :: insert
1141− [%mut p=udon] :: mutate
1142− == ::
1143−```
1144−
1145−There are three kinds of changes that may be made to a node in a desk.
1146−We can insert a file, in which case `p` is the contents of the new file.
1147−We can delete a file, in which case `p` is the contents of the old file.
1148−Finally, we can mutate that file, in which case the `udon` describes the
1149−changes we are applying to the file.
1150−
1151−#### `++mizu:clay`, merged state
1152−
1153−```hoon
1154− ++ mizu ,[p=@u q=(map ,@ud tako) r=rang] :: new state
1155−```
1156−
1157−This is the input to the `%merg` kiss, which allows us to perform a
1158−merge. The `p` is the number of the new head commit. The `q` is a map
1159−from numbers to commit hashes. This is all the new numbered commits that
1160−are to be inserted. The keys to this should always be the numbers from
1161−`let.dom` plus one to `p`, inclusive. The `r` is the maps of all the new
1162−commits and data. Since these are merged into the current state, no old
1163−commits or data need be here.
1164−
1165−#### `++riff:clay`, request/desist
1166−
1167−```hoon
1168− ++ riff ,[p=desk q=(unit rave)] :: request/desist
1169−```
1170−
1171−This represents a request for data about a particular desk. If `q`
1172−contains a `rave`, then this opens a subscription to the desk for that
1173−data. If `q` is null, then this tells clay to cancel the subscription
1174−along this duct.
1175−
1176−#### `++riot:clay`, response
1177−
1178−```hoon
1179− ++ riot (unit rant) :: response/complete
1180−```
1181−
1182−A riot is a response to a subscription. If null, the subscription has
1183−been completed, and no more responses will be sent. Otherwise, the
1184−`rant` is the produced data.
1185−
1186−#### `++rant:clay`, response data
1187−
1188−```hoon
1189− ++ rant :: namespace binding
1190− $: p=[p=care q=case r=@tas] :: clade release book
1191− q=path :: spur
1192− r=* :: data
1193− == ::
1194−```
1195−
1196−This is the data at a particular node in the filesystem. `p.p` specifies
1197−the type of data that was requested (and is produced). `q.p` gives the
1198−specific version reported (since a range of versions may be requested in
1199−a subscription). `r.p` is the desk. `q` is the path to the filesystem
1200−node. `r` is the data itself (in the format specified by `p.p`).
1201−
1202−### `++nako`, subscription response data
1203−
1204−```hoon
1205− ++ nako $: gar=(map ,@ud tako) :: new ids
1206− let=@ud :: next id
1207− lar=(set yaki) :: new commits
1208− bar=(set blob) :: new content
1209− == ::
1210−```
1211−
1212−This is the data that is produced by a request for a range of revisions
1213−of a desk. This allows us to easily keep track of a remote repository --
1214−all the new information we need is contained in the `nako`.
1215−
1216−`gar` is a map of the revisions in the range to the hash of the commit
1217−at that revision. These hashes can be used with `hut:rang:clay` to find the
1218−commit itself.
1219−
1220−`let` is either the last revision number in the range or the most recent
1221−revision number, whichever is smaller.
1222−
1223−`lar` is the set of new commits, and `bar` is the set of new content.
1224−
1225−## Public Interface
1226−
1227−As with all vanes, there are exactly two ways to interact with clay.
1228−`%clay` exports a namespace accessible through `.^`, which is described
1229−above under `++care:clay`. The primary way of interacting with clay, though,
1230−is by sending kisses and receiving gifts.
1231−
1232−```hoon
1233− ++ gift :: out result <-$
1234− $% [%ergo p=@p q=@tas r=@ud] :: version update
1235− [%note p=@tD q=tank] :: debug message
1236− [%writ p=riot] :: response
1237− == ::
1238− ++ kiss :: in request ->$
1239− $% [%info p=@p q=@tas r=nori] :: internal edit
1240− [%ingo p=@p q=@tas r=nori] :: internal noun edit
1241− [%init p=@p] :: report install
1242− [%into p=@p q=@tas r=nori] :: external edit
1243− [%invo p=@p q=@tas r=nori] :: external noun edit
1244− [%merg p=@p q=@tas r=mizu] :: internal change
1245− [%wart p=sock q=@tas r=path s=*] :: network request
1246− [%warp p=sock q=riff] :: file request
1247− == ::
1248−```
1249−
1250−There are only a small number of possible kisses, so it behooves us to
1251−describe each in detail.
1252−
1253−```hoon
1254− $% [%info p=@p q=@tas r=nori] :: internal edit
1255−
1256− [%into p=@p q=@tas r=nori] :: external edit
1257−```
1258−
1259−These two kisses are nearly identical. At a high level, they apply
1260−changes to the filesystem. Whenever we add, remove, or edit a file, one
1261−of these cards is sent. The `p` is the ship whose filesystem we're
1262−trying to change, the `q` is the desk we're changing, and the `r` is the
1263−request change. For the format of the requested change, see the
1264−documentation for `++nori:clay` above.
1265−
1266−When a file is changed in the unix filesystem, vere will send a `%into`
1267−kiss. This tells clay that the duct over which the kiss was sent is the
1268−duct that unix is listening on for changes. From within Arvo, though, we
1269−should never send a `%into` kiss. The `%info` kiss is exactly identical
1270−except it does not reset the duct.
1271−
1272−```hoon
1273− [%ingo p=@p q=@tas r=nori] :: internal noun edit
1274−
1275− [%invo p=@p q=@tas r=nori] :: external noun edit
1276−```
1277−
1278−These kisses are currently identical to `%info` and `%into`, though this
1279−will not always be the case. The intent is for these kisses to allow
1280−typed changes to clay so that we may store typed data. This is currently
1281−unimplemented.
1282−
1283−```hoon
1284− [%init p=@p] :: report install
1285−```
1286−
1287−Init is called when a ship is started on our pier. This simply creates a
1288−default `room` to go into our `raft`. Essentially, this initializes the
1289−filesystem for a ship.
1290−
1291−```hoon
1292− [%merg p=@p q=@tas r=mizu] :: internal change
1293−```
1294−
1295−This is called to perform a merge. This is most visibly called by
1296−:update to update the filesystem of the current ship to that of its
1297−sein. The `p` and `q` are as in `%info`, and the `r` is the description
1298−of the merge. See `++mizu:clay` above.
1299−
1300−XX
1301−`XX [%wake ~] :: timer activate XX`
1302−XX\
1303−XX This card is sent by unix at the time specified by `++doze`. This
1304−time is XX usually the closest time specified in a subscription request.
1305−When `%wake` is XX called, we update our subscribers if there have been
1306−any changes.
1307−
1308−```hoon
1309− [%wart p=sock q=@tas r=path s=*] :: network request
1310−```
1311−
1312−This is a request that has come across the network for a particular
1313−file. When another ship asks for a file from us, that request comes to
1314−us in the form of a `%wart` kiss. This is handled by trivially turning
1315−it into a `%warp`.
1316−
1317−```hoon
1318− [%warp p=sock q=riff] :: file request
1319−```
1320−
1321−This is a request for information about a particular desk. This is, in
1322−its most general form, a subscription, though in many cases it is the
1323−trivial case of a subscription -- a read. See `++riff:clay` for the format of
1324−the request.
1325−
1326−## Lifecycle of a Local Read
1327−
1328−There are two real types of interaction with a filesystem: you can read,
1329−and you can write. We'll describe each process, detailing both the flow
1330−of control followed by the kernel and the algorithms involved. The
1331−simpler case is that of the read, so we'll begin with that.
1332−
1333−When a vane or an application wishes to read a file from the filesystem,
1334−it sends a `%warp` kiss, as described above. Of course, you may request
1335−a file on another ship and, being a global filesystem, clay will happily
1336−produce it for you. That code pathway will be described in another
1337−section; here, we will restrict ourselves to examining the case of a
1338−read from a ship on our own pier.
1339−
1340−The kiss can request either a single version of a file node or a range
1341−of versions of a desk. Here, we'll deal only with a request for a single
1342−version.
1343−
1344−As in all vanes, a kiss enters clay via a call to `++call`. Scanning
1345−through the arm, we quickly see where `%warp` is handled.
1346−
1347−```hoon
1348− ?: =(p.p.q.hic q.p.q.hic)
1349− =+ une=(un p.p.q.hic now ruf)
1350− =+ wex=(di:une p.q.q.hic)
1351− =+ ^= wao
1352− ?~ q.q.q.hic
1353− (ease:wex hen)
1354− (eave:wex hen u.q.q.q.hic)
1355− =+ ^= woo
1356− abet:wao
1357− [-.woo abet:(pish:une p.q.q.hic +.woo ran.wao)]
1358−```
1359−
1360−We're following the familar patern of producing a list of moves and an
1361−updated state. In this case, the state is `++raft`.
1362−
1363−We first check to see if the sending and receiving ships are the same.
1364−If they're not, then this is a request for data on another ship. We
1365−describe that process later. Here, we discuss only the case of a local
1366−read.
1367−
1368−At a high level, the call to `++un` sets up the core for the domestic
1369−ship that contains the files we're looking for. The call to `++di` sets
1370−up the core for the particular desk we're referring to.
1371−
1372−After this, we perform the actual request. If there is no rave in the
1373−riff, then that means we are cancelling a request, so we call
1374−`++ease:de`. Otherwise, we start a subscription with `++eave:de`. We
1375−call `++abet:de` to resolve our various types of output into actual
1376−moves. We produce the moves we found above and the `++un` core resolved
1377−with `++pish:un` (putting the modified desk in the room) and `++abet:un`
1378−(putting the modified room in the raft).
1379−
1380−Much of this is fairly straightforward, so we'll only describe `++ease`,
1381−`++eave`, and `++abet:de`. Feel free to look up the code to the other
1382−steps -- it should be easy to follow.
1383−
1384−Although it's called last, it's usually worth examining `++abet` first,
1385−since it defines in what ways we can cause side effects. Let's do that,
1386−and also a few of the lines at the beginning of `++de`.
1387−
1388−```hoon
1389− =| yel=(list ,[p=duct q=gift])
1390− =| byn=(list ,[p=duct q=riot])
1391− =| vag=(list ,[p=duct q=gift])
1392− =| say=(list ,[p=duct q=path r=ship s=[p=@ud q=riff]])
1393− =| tag=(list ,[p=duct q=path c=note])
1394− |%
1395− ++ abet
1396− ^- [(list move) rede]
1397− :_ red
1398− ;: weld
1399− %+ turn (flop yel)
1400− |=([a=duct b=gift] [hun %give b])
1401− ::
1402− %+ turn (flop byn)
1403− |=([a=duct b=riot] [a %give [%writ b]])
1404− ::
1405− %+ turn (flop vag)
1406− |=([a=duct b=gift] [a %give b])
1407− ::
1408− %+ turn (flop say)
1409− |= [a=duct b=path c=ship d=[p=@ud q=riff]]
1410− :- a
1411− [%pass b %a %want [who c] [%q %re p.q.d (scot %ud p.d) ~] q.d]
1412− ::
1413− %+ turn (flop tag)
1414− |=([a=duct b=path c=note] [a %pass b c])
1415− ==
1416−```
1417−
1418−This is very simple code. We see there are exactly five different kinds
1419−of side effects we can generate.
1420−
1421−In `yel` we put gifts that we wish to be sent along the `hun:room` duct
1422−to dill. See the documentation for `++room` above. This is how we
1423−display messages to the terminal.
1424−
1425−In `byn` we put riots that we wish returned to subscribers. Recall that
1426−a riot is a response to a subscription. These are returned to our
1427−subscribers in the form of a `%writ` gift.
1428−
1429−In `vag` we put gifts along with the ducts on which to send them. This
1430−allows us to produce arbitrary gifts, but in practice this is only used
1431−to produce `%ergo` gifts.
1432−
1433−In `say` we put messages we wish to pass to ames. These messages are
1434−used to request information from clay on other piers. We must provide
1435−not only the duct and the request (the riff), but also the return path,
1436−the other ship to talk to, and the sequence number of the request.
1437−
1438−In `tag` we put arbitrary notes we wish to pass to other vanes. For now,
1439−the only notes we pass here are `%wait` and `%rest` to the timer vane.
1440−
1441−Now that we know what kinds of side effects we may have, we can jump
1442−into the handling of requests.
1443−
1444−```hoon
1445− ++ ease :: release request
1446− |= hen=duct
1447− ^+ +>
1448− ?~ ref +>
1449− =+ rov=(~(got by qyx) hen)
1450− =. qyx (~(del by qyx) hen)
1451− (mabe rov (cury best hen))
1452− =. qyx (~(del by qyx) hen)
1453− |- ^+ +>+.$
1454− =+ nux=(~(get by fod.u.ref) hen)
1455− ?~ nux +>+.$
1456− %= +>+.$
1457− say [[hen [(scot %ud u.nux) ~] for [u.nux syd ~]] say]
1458− fod.u.ref (~(del by fod.u.ref) hen)
1459− bom.u.ref (~(del by bom.u.ref) u.nux)
1460− ==
1461−```
1462−
1463−This is called when we're cancelling a subscription. For domestic desks,
1464−`ref` is null, so we're going to cancel any timer we might have created.
1465−We first delete the duct from our map of requests, and then we call
1466−`++mabe` with `++best` to send a `%rest` kiss to the timer vane if we
1467−have started a timer. We'll describe `++best` and `++mabe` momentarily.
1468−
1469−Although we said we're not going to talk about foreign requests yet,
1470−it's easy to see that for foreign desks, we cancel any outstanding
1471−requests for this duct and send a message over ames to the other ship
1472−telling them to cancel the subscription.
1473−
1474−```hoon
1475− ++ best
1476− |= [hen=duct tym=@da]
1477− %_(+> tag :_(tag [hen /tyme %t %rest tym]))
1478−```
1479−
1480−This simply pushes a `%rest` note onto `tag`, from where it will be
1481−passed back to arvo to be handled. This cancels the timer at the given
1482−duct (with the given time).
1483−
1484−```hoon
1485− ++ mabe :: maybe fire function
1486− |* [rov=rove fun=$+(@da _+>.^$)]
1487− ^+ +>.$
1488− %- fall :_ +>.$
1489− %- bind :_ fun
1490− ^- (unit ,@da)
1491− ?- -.rov
1492− %&
1493− ?. ?=(%da -.q.p.rov) ~
1494− `p.q.p.rov
1495− %|
1496− =* mot p.rov
1497− %+ hunt
1498− ?. ?=(%da -.p.mot) ~
1499− ?.((lth now p.p.mot) ~ [~ p.p.mot])
1500− ?. ?=(%da -.q.mot) ~
1501− ?.((lth now p.q.mot) [~ now] [~ p.q.mot])
1502− ==
1503−```
1504−
1505−This decides whether the given request can only be satsified in the
1506−future. In that case, we call the given function with the time in the
1507−future when we expect to have an update to give to this request. This is
1508−called with `++best` to cancel timers and with `++bait` to start them.
1509−
1510−For single requests, we have a time if the request is for a particular
1511−time (which is assumed to be in the future). For ranges of requests, we
1512−check both the start and end cases to see if they are time cases. If so,
1513−we choose the earlier time.
1514−
1515−If any of those give us a time, then we call the given funciton with the
1516−smallest time.
1517−
1518−The more interesting case is, of course, when we're not cancelling a
1519−subscription but starting one.
1520−
1521−```hoon
1522− ++ eave :: subscribe
1523− |= [hen=duct rav=rave]
1524− ^+ +>
1525− ?- -.rav
1526− &
1527− ?: &(=(p.p.rav %u) !=(p.q.p.rav now))
1528− ~& [%clay-fail p.q.p.rav %now now]
1529− !!
1530− =+ ver=(aver p.rav)
1531− ?~ ver
1532− (duce hen rav)
1533− ?~ u.ver
1534− (blub hen)
1535− (blab hen p.rav u.u.ver)
1536−```
1537−
1538−There are two types of subscriptions -- either we're requesting a single
1539−file or we're requesting a range of versions of a desk. We'll dicuss the
1540−simpler case first.
1541−
1542−First, we check that we're not requesting the `rang` from any time other
1543−than the present. Since we don't store that information for any other
1544−time, we can't produce it in a referentially transparent manner for any
1545−time other than the present.
1546−
1547−Then, we try to read the requested `mood:clay` `p.rav`. If we can't access
1548−the request data right now, we call `++duce` to put the request in our
1549−queue to be satisfied when the information becomes available.
1550−
1551−This case occurs when we make a request for a case whose (1) date is
1552−after the current date, (2) number is after the current number, or (3)
1553−label is not yet used.
1554−
1555−```hoon
1556− ++ duce :: produce request
1557− |= [hen=duct rov=rove]
1558− ^+ +>
1559− =. qyx (~(put by qyx) hen rov)
1560− ?~ ref
1561− (mabe rov (cury bait hen))
1562− |- ^+ +>+.$ :: XX why?
1563− =+ rav=(reve rov)
1564− =+ ^= vaw ^- rave
1565− ?. ?=([%& %v *] rav) rav
1566− [%| [%ud let.dom] `case`q.p.rav r.p.rav]
1567− =+ inx=nix.u.ref
1568− %= +>+.$
1569− say [[hen [(scot %ud inx) ~] for [inx syd ~ vaw]] say]
1570− nix.u.ref +(nix.u.ref)
1571− bom.u.ref (~(put by bom.u.ref) inx [hen vaw])
1572− fod.u.ref (~(put by fod.u.ref) hen inx)
1573− ==
1574−```
1575−
1576−The code for `++duce` is nearly the exact inverse of `++ease`, which in
1577−the case of a domestic desk is very simple -- we simply put the duct and
1578−rave into `qyx` and possibly start a timer with `++mabe` and `++bait`.
1579−Recall that `ref` is null for domestic desks and that `++mabe` fires the
1580−given function with the time we need to be woken up at, if we need to be
1581−woken up at a particular time.
1582−
1583−```hoon
1584− ++ bait
1585− |= [hen=duct tym=@da]
1586− %_(+> tag :_(tag [hen /tyme %t %wait tym]))
1587−```
1588−
1589−This sets an alarm by sending a `%wait` card with the given time to the
1590−timer vane.
1591−
1592−Back in `++eave`, if `++aver` returned `[~ ~]`, then we cancel the
1593−subscription. This occurs when we make (1) a `%x` request for a file
1594−that does not exist, (2) a `%w` request with a case that is not a
1595−number, or (3) a `%w` request with a nonempty path. The `++blub` is
1596−exactly what you would expect it to be.
1597−
1598−```hoon
1599− ++ blub :: ship stop
1600− |= hen=duct
1601− %_(+> byn [[hen ~] byn])
1602−```
1603−
1604−We notify the duct that we're cancelling their subscription since it
1605−isn't satisfiable.
1606−
1607−Otherwise, we have received the desired information, so we send it on to
1608−the subscriber with `++blab`.
1609−
1610−```hoon
1611− ++ blab :: ship result
1612− |= [hen=duct mun=mood dat=*]
1613− ^+ +>
1614− +>(byn [[hen ~ [p.mun q.mun syd] r.mun dat] byn])
1615−```
1616−
1617−The most interesting arm called in `++eave` is, of course, `++aver`,
1618−where we actually try to read the data.
1619−
1620−```hoon
1621− ++ aver :: read
1622− |= mun=mood
1623− ^- (unit (unit ,*))
1624− ?: &(=(p.mun %u) !=(p.q.mun now)) :: prevent bad things
1625− ~& [%clay-fail p.q.mun %now now]
1626− !!
1627− =+ ezy=?~(ref ~ (~(get by haw.u.ref) mun))
1628− ?^ ezy ezy
1629− =+ nao=(~(case-to-aeon ze lim dom ran) q.mun)
1630− ?~(nao ~ [~ (~(read-at-aeon ze lim dom ran) u.nao mun)])
1631−```
1632−
1633−We check immediately that we're not requesting the `rang` for any time
1634−other than the present.
1635−
1636−If this is a foreign desk, then we check our cache for the specific
1637−request. If either this is a domestic desk or we don't have the request
1638−in our cache, then we have to actually go read the data from our dome.
1639−
1640−We need to do two things. First, we try to find the number of the commit
1641−specified by the given case, and then we try to get the data there.
1642−
1643−Here, we jump into `arvo/zuse.hoon`, which is where much of the
1644−algorithmic code is stored, as opposed to the clay interface, which is
1645−stored in `arvo/clay.hoon`. We examine `++case-to-aeon:ze`.
1646−
1647−```hoon
1648− ++ case-to-aeon :: case-to-aeon:ze
1649− |= lok=case :: act count through
1650− ^- (unit aeon)
1651− ?- -.lok
1652− %da
1653− ?: (gth p.lok lim) ~
1654− |- ^- (unit aeon)
1655− ?: =(0 let) [~ 0] :: avoid underflow
1656− ?: %+ gte p.lok
1657− =< t
1658− %- tako-to-yaki
1659− %- aeon-to-tako
1660− let
1661− [~ let]
1662− $(let (dec let))
1663− ::
1664− %tas (~(get by lab) p.lok)
1665− %ud ?:((gth p.lok let) ~ [~ p.lok])
1666− ==
1667−```
1668−
1669−We handle each type of `case:clay` differently. The latter two types are
1670−easy.
1671−
1672−If we're requesting a revision by label, then we simply look up the
1673−requested label in `lab` from the given dome. If it exists, that is our
1674−aeon; else we produce null, indicating the requested revision does not
1675−yet exist.
1676−
1677−If we're requesting a revision by number, we check if we've yet reached
1678−that number. If so, we produce the number; else we produce null.
1679−
1680−If we're requesting a revision by date, we check first if the date is in
1681−the future, returning null if so. Else we start from the most recent
1682−revision and scan backwards until we find the first revision committed
1683−before that date, and we produce that. If we requested a date before any
1684−revisions were committed, we produce `0`.
1685−
1686−The definitions of `++aeon-to-tako` and `++tako-to-yaki` are trivial.
1687−
1688−```hoon
1689− ++ aeon-to-tako ~(got by hit)
1690−
1691− ++ tako-to-yaki ~(got by hut) :: grab yaki
1692−```
1693−
1694−We simply look up the aeon or tako in their respective maps (`hit` and
1695−`hut`).
1696−
1697−Assuming we got a valid version number, `++aver` calls
1698−`++read-at-aeon:ze`, which reads the requested data at the given
1699−revision.
1700−
1701−```hoon
1702− ++ read-at-aeon :: read-at-aeon:ze
1703− |= [oan=aeon mun=mood] :: seek and read
1704− ^- (unit)
1705− ?: &(?=(%w p.mun) !?=(%ud -.q.mun)) :: NB only for speed
1706− ?^(r.mun ~ [~ oan])
1707− (read:(rewind oan) mun)
1708−```
1709−
1710−If we're requesting the revision number with a case other than by
1711−number, then we go ahead and just produce the number we were given.
1712−Otherwise, we call `++rewind` to rewind our state to the given revision,
1713−and then we call `++read` to get the requested information.
1714−
1715−```hoon
1716− ++ rewind :: rewind:ze
1717− |= oan=aeon :: rewind to aeon
1718− ^+ +>
1719− ?: =(let oan) +>
1720− ?: (gth oan let) !! :: don't have version
1721− +>(ank (checkout-ankh q:(tako-to-yaki (aeon-to-tako oan))), let oan)
1722−```
1723−
1724−If we're already at the requested version, we do nothing. If we're
1725−requesting a version later than our head, we are unable to comply.
1726−
1727−Otherwise, we get the hash of the commit at the number, and from that we
1728−get the commit itself (the yaki), which has the map of path to lobe that
1729−represents a version of the filesystem. We call `++checkout-ankh` to
1730−checkout the commit, and we replace `ank` in our context with the
1731−result.
1732−
1733−```hoon
1734− ++ checkout-ankh :: checkout-ankh:ze
1735− |= hat=(map path lobe) :: checkout commit
1736− ^- ankh
1737− %- cosh
1738− %+ roll (~(tap by hat) ~)
1739− |= [[pat=path bar=lobe] ank=ankh]
1740− ^- ankh
1741− %- cosh
1742− ?~ pat
1743− =+ zar=(lobe-to-noun bar)
1744− ank(q [~ (sham zar) zar])
1745− =+ nak=(~(get by r.ank) i.pat)
1746− %= ank
1747− r %+ ~(put by r.ank) i.pat
1748− $(pat t.pat, ank (fall nak _ankh))
1749− ==
1750−```
1751−
1752−Twice we call `++cosh`, which hashes a commit, updating `p` in an
1753−`ankh`. Let's jump into that algorithm before we describe
1754−`++checkout-ankh`.
1755−
1756−```hoon
1757− ++ cosh :: locally rehash
1758− |= ank=ankh :: NB v/unix.c
1759− ank(p rehash:(zu ank))
1760−```
1761−
1762−We simply replace `p` in the hash with the `cash` we get from a call to
1763−`++rehash:zu`.
1764−
1765−```hoon
1766− ++ zu !: :: filesystem
1767− |= ank=ankh :: filesystem state
1768− =| myz=(list ,[p=path q=miso]) :: changes in reverse
1769− =| ram=path :: reverse path into
1770− |%
1771− ++ rehash :: local rehash
1772− ^- cash
1773− %+ mix ?~(q.ank 0 p.u.q.ank)
1774− =+ axe=1
1775− |- ^- cash
1776− ?~ r.ank _@
1777− ;: mix
1778− (shaf %dash (mix axe (shaf %dush (mix p.n.r.ank p.q.n.r.ank))))
1779− $(r.ank l.r.ank, axe (peg axe 2))
1780− $(r.ank r.r.ank, axe (peg axe 3))
1781− ==
1782−```
1783−
1784−`++zu` is a core we set up with a particular filesystem node to traverse
1785−a checkout of the filesystem and access the actual data inside it. One
1786−of the things we can do with it is to create a recursive hash of the
1787−node.
1788−
1789−In `++rehash`, if this node is a file, then we xor the remainder of the
1790−hash with the hash of the contents of the file. The remainder of the
1791−hash is `0` if we have no children, else we descend into our children.
1792−Basically, we do a half SHA-256 of the xor of the axis of this child and
1793−the half SHA-256 of the xor of the name of the child and the hash of the
1794−child. This is done for each child and all the results are xored
1795−together.
1796−
1797−Now we return to our discussion of `++checkout-ankh`.
1798−
1799−We fold over every path in this version of the filesystem and create a
1800−great ankh out of them. First, we call `++lobe-to-noun` to get the raw
1801−data referred to be each lobe.
1802−
1803−```hoon
1804− ++ lobe-to-noun :: grab blob
1805− |= p=lobe :: ^- *
1806− %- blob-to-noun
1807− (lobe-to-blob p)
1808−```
1809−
1810−This converts a lobe into the raw data it refers to by first getting the
1811−blob with `++lobe-to-blob` and converting that into data with
1812−`++blob-to-noun`.
1813−
1814−```hoon
1815− ++ lobe-to-blob ~(got by lat) :: grab blob
1816−```
1817−
1818−This just grabs the blob that the lobe refers to.
1819−
1820−```hoon
1821− ++ blob-to-noun :: grab blob
1822− |= p=blob
1823− ?- -.p
1824− %delta (lump r.p (lobe-to-noun q.p))
1825− %direct q.p
1826− %indirect q.p
1827− ==
1828−```
1829−
1830−If we have either a direct or an indirect blob, then the data is stored
1831−right in the blob. Otherwise, we have to reconstruct it from the diffs.
1832−We do this by calling `++lump` on the diff in the blob with the data
1833−obtained by recursively calling the parent of this blob.
1834−
1835−```hoon
1836− ++ lump :: apply patch
1837− |= [don=udon src=*]
1838− ^- *
1839− ?+ p.don ~|(%unsupported !!)
1840− %a
1841− ?+ -.q.don ~|(%unsupported !!)
1842− %a q.q.don
1843− %c (lurk ((hard (list)) src) p.q.don)
1844− %d (lure src p.q.don)
1845− ==
1846− ::
1847− %c
1848− =+ dst=(lore ((hard ,@) src))
1849− %- roly
1850− ?+ -.q.don ~|(%unsupported !!)
1851− %a ((hard (list ,@t)) q.q.don)
1852− %c (lurk dst p.q.don)
1853− ==
1854− ==
1855−```
1856−
1857−This is defined in `arvo/hoon.hoon` for historical reasons which are
1858−likely no longer applicable. Since the `++umph` structure will likely
1859−change we convert clay to be a typed filesystem, we'll only give a
1860−high-level description of this process. If we have a `%a` udon, then
1861−we're performing a trivial replace, so we produce simply `q.q.don`. If
1862−we have a `%c` udon, then we're performing a list merge (as in, for
1863−example, lines of text). The merge is performed by `++lurk`.
1864−
1865−```hoon
1866− ++ lurk :: apply list patch
1867− |* [hel=(list) rug=(urge)]
1868− ^+ hel
1869− =+ war=`_hel`~
1870− |- ^+ hel
1871− ?~ rug (flop war)
1872− ?- -.i.rug
1873− &
1874− %= $
1875− rug t.rug
1876− hel (slag p.i.rug hel)
1877− war (weld (flop (scag p.i.rug hel)) war)
1878− ==
1879− ::
1880− |
1881− %= $
1882− rug t.rug
1883− hel =+ gur=(flop p.i.rug)
1884− |- ^+ hel
1885− ?~ gur hel
1886− ?>(&(?=(^ hel) =(i.gur i.hel)) $(hel t.hel, gur t.gur))
1887− war (weld q.i.rug war)
1888− ==
1889− ==
1890−```
1891−
1892−We accumulate our final result in `war`. If there's nothing more in our
1893−list of merge instructions (unces), we just reverse `war` and produce
1894−it. Otherwise, we process another unce. If the unce is of type `&`, then
1895−we have `p.i.rug` lines of no changes, so we just copy them over from
1896−`hel` to `war`. If the unice is of type `|`, then we verify that the
1897−source lines (in `hel`) are what we expect them to be (`p.i.rug`),
1898−crashing on failure. If they're good, then we append the new lines in
1899−`q.i.rug` onto `war`.
1900−
1901−And that's really it. List merges are pretty easy. Anyway, if you
1902−recall, we were discussing `++checkout-ankh`.
1903−
1904−```hoon
1905− ++ checkout-ankh :: checkout-ankh:ze
1906− |= hat=(map path lobe) :: checkout commit
1907− ^- ankh
1908− %- cosh
1909− %+ roll (~(tap by hat) ~)
1910− |= [[pat=path bar=lobe] ank=ankh]
1911− ^- ankh
1912− %- cosh
1913− ?~ pat
1914− =+ zar=(lobe-to-noun bar)
1915− ank(q [~ (sham zar) zar])
1916− =+ nak=(~(get by r.ank) i.pat)
1917− %= ank
1918− r %+ ~(put by r.ank) i.pat
1919− $(pat t.pat, ank (fall nak _ankh))
1920− ==
1921−```
1922−
1923−If the path is null, then we calculate `zar`, the raw data at the path
1924−`pat` in this version. We produce the given ankh with the correct data.
1925−
1926−Otherwise, we try to get the child we're looking at from our parent
1927−ankh. If it's already been created, this succeeds; otherwise, we simply
1928−create a default blank ankh. We place ourselves in our parent after
1929−recursively computing our children.
1930−
1931−This algorithm really isn't that complicated, but it may not be
1932−immediately obvious why it works. An example should clear everything up.
1933−
1934−Suppose `hat` is a map of the following information.
1935−
1936−```
1937− /greeting --> "customary upon meeting"
1938− /greeting/english --> "hello"
1939− /greeting/spanish --> "hola"
1940− /greeting/russian/short --> "привет"
1941− /greeting/russian/long --> "Здравствуйте"
1942− /farewell/russian --> "до свидания"
1943−```
1944−
1945−Furthermore, let's say that we process them in this order:
1946−
1947−```
1948− /greeting/english
1949− /greeting/russian/short
1950− /greeting/russian/long
1951− /greeting
1952− /greeting/spanish
1953− /farewell/russian
1954−```
1955−
1956−Then, the first path we process is `/greeting/english` . Since our path
1957−is not null, we try to get `nak`, but because our ankh is blank at this
1958−point it doesn't find anything. Thus, update our blank top-level ankh
1959−with a child `%greeting`. and recurse with the blank `nak` to create the
1960−ankh of the new child.
1961−
1962−In the recursion, we our path is `/english` and our ankh is again blank.
1963−We try to get the `english` child of our ankh, but this of course fails.
1964−Thus, we update our blank `/greeting` ankh with a child `english`
1965−produced by recursing.
1966−
1967−Now our path is null, so we call `++lobe-to-noun` to get the actual
1968−data, and we place it in the brand-new ankh.
1969−
1970−Next, we process `/greeting/russian/short`. Since our path is not null,
1971−we try to get the child named `%greeting`, which does exist since we
1972−created it earlier. We put modify this child by recursing on it. Our
1973−path is now `/russian/short`, so we look for a `%russian` child in our
1974−`/greeting` ankh. This doesn't exist, so we add it by recursing. Our
1975−path is now `/short`, so we look for a `%short` child in our
1976−`/greeting/russian` ankh. This doesn't exist, so we add it by recursing.
1977−Now our path is null, so we set the contents of this node to `"привет"`,
1978−and we're done processing this path.
1979−
1980−Next, we process `/greeting/russian/long`. This proceeds similarly to
1981−the previous except that now the ankh for `/greeting/russian` already
1982−exists, so we simply reuse it rather than creating a new one. Of course,
1983−we still must create a new `/greeting/russian/long` ankh.
1984−
1985−Next, we process `/greeting`. This ankh already exists, so after we've
1986−recursed once, our path is null, and our ankh is not blank -- it already
1987−has two children (and two grandchildren). We don't touch those, though,
1988−since a node may be both a file and a directory. We just add the
1989−contents of the file -- "customary upon meeting" -- to the existing
1990−ankh.
1991−
1992−Next, we process `/greeting/spanish`. Of course, the `/greeting` ankh
1993−already exists, but it doesn't have a `%spanish` child, so we create
1994−that, taking care not to disturb the contents of the `/greeting` file.
1995−We put "hola" into the ankh and call it good.
1996−
1997−Finally, we process `/farewell/russian`. Here, the `/farewell` ankh
1998−doesn't exist, so we create it. Clearly the newly-created ankh doesn't
1999−have any children, so we have to add a `%russian` child, and in this
2000−child we put our last content -- "до свидания".
2001−
2002−We hope it's fairly obvious that the order we process the paths doesn't
2003−affect the final ankh tree. The tree will be constructed in a very
2004−different order depending on what order the paths come in, but the
2005−resulting tree is independent of order.
2006−
2007−At any rate, we were talking about something important, weren't we? If
2008−you recall, that concludes our discussion of `++rewind`, which was
2009−called from `++read-at-aeon`. In summary, `++rewind` returns a context
2010−in which our current state is (very nearly) as it was when the specified
2011−version of the desk was the head. This allows `++read-at-aeon` to call
2012−`++read` to read the requested information.
2013−
2014−```hoon
2015− ++ read :: read:ze
2016− |= mun=mood :: read at point
2017− ^- (unit)
2018− ?: ?=(%v p.mun)
2019− [~ `dome`+<+<.read]
2020− ?: &(?=(%w p.mun) !?=(%ud -.q.mun))
2021− ?^(r.mun ~ [~ let])
2022− ?: ?=(%w p.mun)
2023− =+ ^= yak
2024− %- tako-to-yaki
2025− %- aeon-to-tako
2026− let
2027− ?^(r.mun ~ [~ [t.yak (forge-nori yak)]])
2028− ::?> ?=(^ hit) ?^(r.mun ~ [~ i.hit]) :: what do?? need [@da nori]
2029− (query(ank ank:(descend-path:(zu ank) r.mun)) p.mun)
2030−```
2031−
2032−If we're requesting the dome, then we just return that immediately.
2033−
2034−If we're requesting the revision number of the desk and we're not
2035−requesting it by number, then we just return the current number of this
2036−desk. Note of course that this was really already handled in
2037−`++read-at-aeon`.
2038−
2039−If we're requesting a `%w` with a specific revision number, then we do
2040−something or other with the commit there. It's kind of weird, and it
2041−doesn't seem to work, so we'll ignore this case.
2042−
2043−Otherwise, we descend into the ankh tree to the given path with
2044−`++descend-path:zu`, and then we handle specific request in `++query`.
2045−
2046−```hoon
2047− ++ descend-path :: descend recursively
2048− |= way=path
2049− ^+ +>
2050− ?~(way +> $(way t.way, +> (descend i.way)))
2051−```
2052−
2053−This is simple recursion down into the ankh tree. `++descend` descends
2054−one level, so this will eventually get us down to the path we want.
2055−
2056−```hoon
2057− ++ descend :: descend
2058− |= lol=@ta
2059− ^+ +>
2060− =+ you=(~(get by r.ank) lol)
2061− +>.$(ram [lol ram], ank ?~(you [*cash ~ ~] u.you))
2062−```
2063−
2064−`ram` is the path that we're at, so to descend one level we push the
2065−name of this level onto that path. We update the ankh with the correct
2066−one at that path if it exists; else we create a blank one.
2067−
2068−Once we've decscended to the correct level, we need to actually deal
2069−with the request.
2070−
2071−```hoon
2072− ++ query :: query:ze
2073− |= ren=?(%u %v %x %y %z) :: endpoint query
2074− ^- (unit ,*)
2075− ?- ren
2076− %u [~ `rang`+<+>.query]
2077− %v [~ `dome`+<+<.query]
2078− %x ?~(q.ank ~ [~ q.u.q.ank])
2079− %y [~ as-arch]
2080− %z [~ ank]
2081− ==
2082−```
2083−
2084−Now that everything's set up, it's really easy. If they're requesting
2085−the rang, dome, or ankh, we give it to them. If the contents of a file,
2086−we give it to them if it is in fact a file. If the `arch`, then we
2087−calculate it with `++as-arch`.
2088−
2089−```hoon
2090− ++ as-arch :: as-arch:ze
2091− ^- arch :: arch report
2092− :+ p.ank
2093− ?~(q.ank ~ [~ p.u.q.ank])
2094− |- ^- (map ,@ta ,~)
2095− ?~ r.ank ~
2096− [[p.n.r.ank ~] $(r.ank l.r.ank) $(r.ank r.r.ank)]
2097−```
2098−
2099−This very simply strips out all the "real" data and returns just our own
2100−hash, the hash of the file contents (if we're a file), and a map of the
2101−names of our immediate children.
2102−
2103−## Lifecycle of a Local Subscription
2104−
2105−A subscription to a range of revisions of a desk initially follows the
2106−same path that a single read does. In `++aver`, we checked the head of
2107−the given rave. If the head was `&`, then it was a single request, so we
2108−handled it above. If `|`, then we handle it with the following code.
2109−
2110−```hoon
2111− =+ nab=(~(case-to-aeon ze lim dom ran) p.p.rav)
2112− ?~ nab
2113− ?> =(~ (~(case-to-aeon ze lim dom ran) q.p.rav))
2114− (duce hen (rive rav))
2115− =+ huy=(~(case-to-aeon ze lim dom ran) q.p.rav)
2116− ?: &(?=(^ huy) |((lth u.huy u.nab) &(=(0 u.huy) =(0 u.nab))))
2117− (blub hen)
2118− =+ top=?~(huy let.dom u.huy)
2119− =+ sar=(~(lobes-at-path ze lim dom ran) u.nab r.p.rav)
2120− =+ ear=(~(lobes-at-path ze lim dom ran) top r.p.rav)
2121− =. +>.$
2122− ?: =(sar ear) +>.$
2123− =+ fud=(~(make-nako ze lim dom ran) u.nab top)
2124− (bleb hen u.nab fud)
2125− ?^ huy
2126− (blub hen)
2127− =+ ^= ptr ^- case
2128− [%ud +(let.dom)]
2129− (duce hen `rove`[%| ptr q.p.rav r.p.rav ear])
2130− ==
2131−```
2132−
2133−Recall that `++case-to-aeon:ze` produces the revision number that a case
2134−corresponds to, if it corresponds to any. If it doesn't yet correspond
2135−to a revision, then it produces null.
2136−
2137−Thus, we first check to see if we've even gotten to the beginning of the
2138−range of revisions requested. If not, then we assert that we haven't yet
2139−gotten to the end of the range either, because that would be really
2140−strange. If not, then we immediately call `++duce`, which, if you
2141−recall, for a local request, simply puts this duct and rove into our
2142−cult `qyx`, so that we know who to respond to when the revision does
2143−appear.
2144−
2145−If we've already gotten to the first revision, then we can produce some
2146−content immediately. If we've also gotten to the final revision, and
2147−that revision is earlier than the start revision, then it's a bad
2148−request and we call `++blub`, which tells the subscriber that his
2149−subscription will not be satisfied.
2150−
2151−Otherwise, we find the data at the given path at the beginning of the
2152−subscription and at the last available revision in the subscription. If
2153−they're the same, then we don't send a notification. Otherwise, we call
2154−`++gack`, which creates the `++nako` we need to produce. We call
2155−`++bleb` to actually produce the information.
2156−
2157−If we already have the last requested revision, then we also tell the
2158−subscriber with `++blub` that the subscription will receive no further
2159−updates.
2160−
2161−If there will be more revisions in the subscription, then we call
2162−`++duce`, adding the duct to our subscribers. We modify the rove to
2163−start at the next revision since we've already handled all the revisions
2164−up to the present.
2165−
2166−We glossed over the calls to `++lobes-at-path`, `++make-nako`, and
2167−`++bleb`, so we'll get back to those right now. `++bleb` is simple, so
2168−we'll start with that.
2169−
2170−```hoon
2171− ++ bleb :: ship sequence
2172− |= [hen=duct ins=@ud hip=nako]
2173− ^+ +>
2174− (blab hen [%w [%ud ins] ~] hip)
2175−```
2176−
2177−We're given a duct, the beginning revision number, and the nako that
2178−contains the updates since that revision. We use `++blab` to produce
2179−this result to the subscriber. The case is `%w` with a revision number
2180−of the beginning of the subscription, and the data is the nako itself.
2181−
2182−We call `++lobes-at-path:ze` to get the data at the particular path.
2183−
2184−```hoon
2185− ++ lobes-at-path :: lobes-at-path:ze
2186− |= [oan=aeon pax=path] :: data at path
2187− ^- (map path lobe)
2188− ?: =(0 oan) ~
2189− %- mo
2190− %+ skim
2191− %. ~
2192− %~ tap by
2193− =< q
2194− %- tako-to-yaki
2195− %- aeon-to-tako
2196− oan
2197− |= [p=path q=lobe]
2198− ?| ?=(~ pax)
2199− ?& !?=(~ p)
2200− =(-.pax -.p)
2201− $(p +.p, pax +.pax)
2202− == ==
2203−```
2204−
2205−At revision zero, the theoretical common revision between all
2206−repositories, there is no data, so we produce null.
2207−
2208−We get the list of paths (paired with their lobe) in the revision
2209−referred to by the given number and we keep only those paths which begin
2210−with `pax`. Converting to a map, we now have a map from the subpaths at
2211−the given path to the hash of their data. This is simple and efficient
2212−to calculate and compare to later revisions. This allows us to easily
2213−tell if a node or its children have changed.
2214−
2215−Finally, we will describe `++make-nako:ze`.
2216−
2217−```hoon
2218− ++ make-nako :: gack a through b
2219− |= [a=aeon b=aeon]
2220− ^- [(map aeon tako) aeon (set yaki) (set blob)]
2221− :_ :- b
2222− =- [(takos-to-yakis -<) (lobes-to-blobs ->)]
2223− %+ reachable-between-takos
2224− (~(get by hit) a) :: if a not found, a=0
2225− (aeon-to-tako b)
2226− ^- (map aeon tako)
2227− %- mo %+ skim (~(tap by hit) ~)
2228− |= [p=aeon *]
2229− &((gth p a) (lte p b))
2230−```
2231−
2232−We need to produce four things -- the numbers of the new commits, the
2233−number of the latest commit, the new commits themselves, and the new
2234−data itself.
2235−
2236−The first is fairly easy to produce. We simply go over our map of
2237−numbered commits and produce all those numbered greater than `a` and not
2238−greater than `b`.
2239−
2240−The second is even easier to produce -- `b` is clearly our most recent
2241−commit.
2242−
2243−The third and fourth are slightly more interesting, though not too
2244−terribly difficult. First, we call `++reachable-between-takos`.
2245−
2246−```hoon
2247− ++ reachable-between-takos
2248− |= [a=(unit tako) b=tako] :: pack a through b
2249− ^- [(set tako) (set lobe)]
2250− =+ ^= sar
2251− ?~ a ~
2252− (reachable-takos r:(tako-to-yaki u.a))
2253− =+ yak=`yaki`(tako-to-yaki b)
2254− %+ new-lobes-takos (new-lobes ~ sar) :: get lobes
2255− |- ^- (set tako) :: walk onto sar
2256− ?: (~(has in sar) r.yak)
2257− ~
2258− =+ ber=`(set tako)`(~(put in `(set tako)`~) `tako`r.yak)
2259− %- ~(uni in ber)
2260− ^- (set tako)
2261− %+ roll p.yak
2262− |= [yek=tako bar=(set tako)]
2263− ^- (set tako)
2264− ?: (~(has in bar) yek) :: save some time
2265− bar
2266− %- ~(uni in bar)
2267− ^$(yak (tako-to-yaki yek))
2268−```
2269−
2270−We take a possible starting commit and a definite ending commit, and we
2271−produce the set of commits and the set of data between them.
2272−
2273−We let `sar` be the set of commits reachable from `a`. If `a` is null,
2274−then obviously no commits are reachable. Otherwise, we call
2275−`++reachable-takos` to calculate this.
2276−
2277−```hoon
2278− ++ reachable-takos :: reachable
2279− |= p=tako :: XX slow
2280− ^- (set tako)
2281− =+ y=(tako-to-yaki p)
2282− =+ t=(~(put in _(set tako)) p)
2283− %+ roll p.y
2284− |= [q=tako s=_t]
2285− ?: (~(has in s) q) :: already done
2286− s :: hence skip
2287− (~(uni in s) ^$(p q)) :: otherwise traverse
2288−```
2289−
2290−We very simply produce the set of the given tako plus its parents,
2291−recursively.
2292−
2293−Back in `++reachable-between-takos`, we let `yak` be the yaki of `b`,
2294−the ending commit. With this, we create a set that is the union of `sar`
2295−and all takos reachable from `b`.
2296−
2297−We pass `sar` into `++new-lobes` to get all the lobes referenced by any
2298−tako referenced by `a`. The result is passed into `++new-lobes-takos` to
2299−do the same, but not recomputing those in already calculated last
2300−sentence. This produces the sets of takos and lobes we need.
2301−
2302−```hoon
2303− ++ new-lobes :: object hash set
2304− |= [b=(set lobe) a=(set tako)] :: that aren't in b
2305− ^- (set lobe)
2306− %+ roll (~(tap in a) ~)
2307− |= [tak=tako bar=(set lobe)]
2308− ^- (set lobe)
2309− =+ yak=(tako-to-yaki tak)
2310− %+ roll (~(tap by q.yak) ~)
2311− |= [[path lob=lobe] far=_bar]
2312− ^- (set lobe)
2313− ?~ (~(has in b) lob) :: don't need
2314− far
2315− =+ gar=(lobe-to-blob lob)
2316− ?- -.gar
2317− %direct (~(put in far) lob)
2318− %delta (~(put in $(lob q.gar)) lob)
2319− %indirect (~(put in $(lob s.gar)) lob)
2320− ==
2321−```
2322−
2323−Here, we're creating a set of lobes referenced in a commit in `a`. We
2324−start out with `b` as the initial set of lobes, so we don't need to
2325−recompute any of the lobes referenced in there.
2326−
2327−The algorithm is pretty simple, so we won't bore you with the details.
2328−We simply traverse every commit in `a`, looking at every blob referenced
2329−there, and, if it's not already in `b`, we add it to `b`. In the case of
2330−a direct blob, we're done. For a delta or an indirect blob, we
2331−recursively add every blob referenced within the blob.
2332−
2333−```hoon
2334− ++ new-lobes-takos :: garg & repack
2335− |= [b=(set lobe) a=(set tako)]
2336− ^- [(set tako) (set lobe)]
2337− [a (new-lobes b a)]
2338−```
2339−
2340−Here, we just update the set of lobes we're given with the commits we're
2341−given and produce both sets.
2342−
2343−This concludes our discussion of a local subscription.
2344−
2345−## Lifecycle of a Foreign Read or Subscription
2346−
2347−Foreign reads and subscriptions are handled in much the same way as
2348−local ones. The interface is the same -- a vane or app sends a `%warp`
2349−kiss with the request. The difference is simply that the `sock` refers
2350−to the foreign ship.
2351−
2352−Thus, we start in the same place -- in `++call`, handling `%warp`.
2353−However, since the two side of the `sock` are different, we follow a
2354−different path.
2355−
2356−```hoon
2357− =+ wex=(do now p.q.hic p.q.q.hic ruf)
2358− =+ ^= woo
2359− ?~ q.q.q.hic
2360− abet:(ease:wex hen)
2361− abet:(eave:wex hen u.q.q.q.hic)
2362− [-.woo (posh q.p.q.hic p.q.q.hic +.woo ruf)]
2363−```
2364−
2365−If we compare this to how the local case was handled, we see that it's
2366−not all that different. We use `++do` rather than `++un` and `++de` to
2367−set up the core for the foreign ship. This gives us a `++de` core, so we
2368−either cancel or begin the request by calling `++ease` or `++eave`,
2369−exactly as in the local case. In either case, we call `++abet:de` to
2370−resolve our various types of output into actual moves, as described in
2371−the local case. Finally, we call `++posh` to update our raft, putting
2372−the modified rung into the raft.
2373−
2374−We'll first trace through `++do`.
2375−
2376−```hoon
2377− ++ do
2378− |= [now=@da [who=ship him=ship] syd=@tas ruf=raft]
2379− =+ ^= rug ^- rung
2380− =+ rug=(~(get by hoy.ruf) him)
2381− ?^(rug u.rug *rung)
2382− =+ ^= red ^- rede
2383− =+ yit=(~(get by rus.rug) syd)
2384− ?^(yit u.yit `rede`[~2000.1.1 ~ [~ *rind] *dome])
2385− ((de now ~ ~) [who him] syd red ran.ruf)
2386−```
2387−
2388−If we already have a rung for this foreign ship, then we use that.
2389−Otherwise, we create a new blank one. If we already have a rede in this
2390−rung, then we use that, otherwise we create a blank one. An important
2391−point to note here is that we let `ref` in the rede be `[~ *rind]`.
2392−Recall, for domestic desks `ref` is null. We use this to distinguish
2393−between foreign and domestic desks in `++de`.
2394−
2395−With this information, we create a `++de` core as usual.
2396−
2397−Although we've already covered `++ease` and `++eave`, we'll go through
2398−them quickly again, highlighting the case of foreign request.
2399−
2400−```hoon
2401− ++ ease :: release request
2402− |= hen=duct
2403− ^+ +>
2404− ?~ ref +>
2405− =+ rov=(~(got by qyx) hen)
2406− =. qyx (~(del by qyx) hen)
2407− (mabe rov (cury best hen))
2408− =. qyx (~(del by qyx) hen)
2409− |- ^+ +>+.$
2410− =+ nux=(~(get by fod.u.ref) hen)
2411− ?~ nux +>+.$
2412− %= +>+.$
2413− say [[hen [(scot %ud u.nux) ~] for [u.nux syd ~]] say]
2414− fod.u.ref (~(del by fod.u.ref) hen)
2415− bom.u.ref (~(del by bom.u.ref) u.nux)
2416− ==
2417−```
2418−
2419−Here, we still remove the duct from our cult (we maintain a cult even
2420−for foreign desks), but we also need to tell the foreign desk to cancel
2421−our subscription. We do this by sending a request (by appending to
2422−`say`, which gets resolved in `++abet:de` to a `%want` kiss to ames) to
2423−the foreign ship to cancel the subscription. Since we don't anymore
2424−expect a response on this duct, we remove it from `fod` and `bom`, which
2425−are the maps between ducts, raves, and request sequence numbers.
2426−Basically, we remove every trace of the subscription from our request
2427−manager.
2428−
2429−In the case of `++eave`, where we're creating a new request, everything
2430−is exactly identical to the case of the local request except `++duce`.
2431−We said that `++duce` simply puts the request into our cult. This is
2432−true for a domestic request, but distinctly untrue for foreign requests.
2433−
2434−```hoon
2435− ++ duce :: produce request
2436− |= [hen=duct rov=rove]
2437− ^+ +>
2438− =. qyx (~(put by qyx) hen rov)
2439− ?~ ref +>.$
2440− |- ^+ +>+.$ :: XX why?
2441− =+ rav=(reve rov)
2442− =+ ^= vaw ^- rave
2443− ?. ?=([%& %v *] rav) rav
2444− [%| [%ud let.dom] `case`q.p.rav r.p.rav]
2445− =+ inx=nix.u.ref
2446− %= +>+.$
2447− say [[hen [(scot %ud inx) ~] for [inx syd ~ vaw]] say]
2448− nix.u.ref +(nix.u.ref)
2449− bom.u.ref (~(put by bom.u.ref) inx [hen vaw])
2450− fod.u.ref (~(put by fod.u.ref) hen inx)
2451− ==
2452−```
2453−
2454−If we have a request manager (i.e. this is a foreign desk), then we do
2455−the approximate inverse of `++ease`. We create a rave out of the given
2456−request and send it off to the foreign desk by putting it in `say`. Note
2457−that the rave is created to request the information starting at the next
2458−revision number. Since this is a new request, we put it into `fod` and
2459−`bom` to associate the request with its duct and its sequence number.
2460−Since we're using another sequence number, we must increment `nix`,
2461−which represents the next available sequence number.
2462−
2463−And that's really it for this side of the request. Requesting foreign
2464−information isn't that hard. Let's see what it looks like on the other
2465−side. When we get a request from another ship for information on our
2466−ship, that comes to us in the form of a `%wart` from ames.
2467−
2468−We handle a `%wart` in `++call`, right next to where we handle the
2469−`%warp` case.
2470−
2471−```hoon
2472− %wart
2473− ?> ?=(%re q.q.hic)
2474− =+ ryf=((hard riff) s.q.hic)
2475− :_ ..^$
2476− :~ :- hen
2477− :^ %pass [(scot %p p.p.q.hic) (scot %p q.p.q.hic) r.q.hic]
2478− %c
2479− [%warp [p.p.q.hic p.p.q.hic] ryf]
2480− ==
2481−```
2482−
2483−Every request we receive should be of type `riff`, so we coerce it into
2484−that type. We just convert this into a new `%warp` kiss that we pass to
2485−ourself. This gets handled like normal, as a local request. When the
2486−request produces a value, it does so like normal as a `%writ`, which is
2487−returned to `++take` along the path we just sent it on.
2488−
2489−```hoon
2490− %writ
2491− ?> ?=([@ @ *] tea)
2492− =+ our=(need (slaw %p i.tea))
2493− =+ him=(need (slaw %p i.t.tea))
2494− :_ ..^$
2495− :~ :- hen
2496− [%pass ~ %a [%want [our him] [%r %re %c t.t.tea] p.+.q.hin]]
2497− ==
2498−```
2499−
2500−Since we encoded the ship we need to respond to in the path, we can just
2501−pass our `%want` back to ames, so that we tell the requesting ship about
2502−the new data.
2503−
2504−This comes back to the original ship as a `%waft` from ames, which comes
2505−into `++take`, right next to where we handled `%writ`.
2506−
2507−```hoon
2508− %waft
2509− ?> ?=([@ @ ~] tea)
2510− =+ syd=(need (slaw %tas i.tea))
2511− =+ inx=(need (slaw %ud i.t.tea))
2512− =+ ^= zat
2513− =< wake
2514− (knit:(do now p.+.q.hin syd ruf) [inx ((hard riot) q.+.q.hin)])
2515− =^ mos ruf
2516− =+ zot=abet.zat
2517− [-.zot (posh q.p.+.q.hin syd +.zot ruf)]
2518− [mos ..^$(ran.ruf ran.zat)] :: merge in new obj
2519−```
2520−
2521−This gets the desk and sequence number from the path the request was
2522−sent over. This determines exactly which request is being responded to.
2523−We call `++knit:de` to apply the changes to our local desk, and we call
2524−`++wake` to update our subscribers. Then we call `++abet:de` and
2525−`++posh` as normal (like in `++eave`).
2526−
2527−We'll examine `++knit` and `++wake`, in that order.
2528−
2529−```hoon
2530− ++ knit :: external change
2531− |= [inx=@ud rot=riot]
2532− ^+ +>
2533− ?> ?=(^ ref)
2534− |- ^+ +>+.$
2535− =+ ruv=(~(get by bom.u.ref) inx)
2536− ?~ ruv +>+.$
2537− => ?. |(?=(~ rot) ?=(& -.q.u.ruv)) .
2538− %_ .
2539− bom.u.ref (~(del by bom.u.ref) inx)
2540− fod.u.ref (~(del by fod.u.ref) p.u.ruv)
2541− ==
2542− ?~ rot
2543− =+ rav=`rave`q.u.ruv
2544− %= +>+.$
2545− lim
2546− ?.(&(?=(| -.rav) ?=(%da -.q.p.rav)) lim `@da`p.q.p.rav)
2547− ::
2548− haw.u.ref
2549− ?. ?=(& -.rav) haw.u.ref
2550− (~(put by haw.u.ref) p.rav ~)
2551− ==
2552− ?< ?=(%v p.p.u.rot)
2553− =. haw.u.ref
2554− (~(put by haw.u.ref) [p.p.u.rot q.p.u.rot q.u.rot] ~ r.u.rot)
2555− ?. ?=(%w p.p.u.rot) +>+.$
2556− |- ^+ +>+.^$
2557− =+ nez=[%w [%ud let.dom] ~]
2558− =+ nex=(~(get by haw.u.ref) nez)
2559− ?~ nex +>+.^$
2560− ?~ u.nex +>+.^$ :: should never happen
2561− =. +>+.^$ =+ roo=(edis ((hard nako) u.u.nex))
2562− ?>(?=(^ ref.roo) roo)
2563− %= $
2564− haw.u.ref (~(del by haw.u.ref) nez)
2565− ==
2566−```
2567−
2568−This is kind of a long [gate](/docs/glossary/gate/), but don't worry, it's not bad at all.
2569−
2570−First, we assert that we're not a domestic desk. That wouldn't make any
2571−sense at all.
2572−
2573−Since we have the sequence number of the request, we can get the duct
2574−and rave from `bom` in our request manager. If we didn't actually
2575−request this data (or the request was canceled before we got it), then
2576−we do nothing.
2577−
2578−Else, we remove the request from `bom` and `fod` unless this was a
2579−subscription request and we didn't receive a null riot (which would
2580−indicate the last message on the subscription).
2581−
2582−Now, if we received a null riot, then if this was a subscription request
2583−by date, then we update `lim` in our request manager (representing the
2584−latest time at which we have complete information for this desk) to the
2585−date that was requested. If this was a single read request, then we put
2586−the result in our simple cache `haw` to make future requests faster.
2587−Then we're done.
2588−
2589−If we received actual data, then we put it into our cache `haw`. Unless
2590−it's a `%w` request, we're done.
2591−
2592−If it is a `%w` request, then we try to get the `%w` at our current head
2593−from the cache. In general, that should be the thing we just put in a
2594−moment ago, but that is not guaranteed. The most common case where this
2595−is different is when we receive desk updates out of order. At any rate,
2596−since we now have new information, we need to apply it to our local copy
2597−of the desk. We do so in `++edis`, and then we remove the stuff we just
2598−applied from the cache, since it's not really a true "single read", like
2599−what should really be in the cache.
2600−
2601−```hoon
2602− ++ edis :: apply subscription
2603− |= nak=nako
2604− ^+ +>
2605− %= +>
2606− hit.dom (~(uni by hit.dom) gar.nak)
2607− let.dom let.nak
2608− lat.ran %+ roll (~(tap in bar.nak) ~)
2609− =< .(yeb lat.ran)
2610− |= [sar=blob yeb=(map lobe blob)]
2611− =+ zax=(blob-to-lobe sar)
2612− %+ ~(put by yeb) zax sar
2613− hut.ran %+ roll (~(tap in lar.nak) ~)
2614− =< .(yeb hut.ran)
2615− |= [sar=yaki yeb=(map tako yaki)]
2616− %+ ~(put by yeb) r.sar sar
2617− ==
2618−```
2619−
2620−This shows, of course, exactly why nako is defined the way it is. To
2621−become completely up to date with the foreign desk, we need to merge
2622−`hit` with the foreign one so that we have all the revision numbers. We
2623−update `let` so that we know which revision is the head.
2624−
2625−We merge the new blobs in `lat`, keying them by their hash, which we get
2626−from a call to `++blob-to-lobe`. Recall that the hash is stored in the
2627−actual blob itself. We do the same thing to the new yakis, putting them
2628−in `hut`, keyed by their hash.
2629−
2630−Now, our local dome should be exactly the same as the foreign one.
2631−
2632−This concludes our discussion of `++knit`. Now the changes have been
2633−applied to our local copy of the desk, and we just need to update our
2634−subscribers. We do so in `++wake:de`.
2635−
2636−```hoon
2637− ++ wake :: update subscribers
2638− ^+ .
2639− =+ xiq=(~(tap by qyx) ~)
2640− =| xaq=(list ,[p=duct q=rove])
2641− |- ^+ ..wake
2642− ?~ xiq
2643− ..wake(qyx (~(gas by *cult) xaq))
2644− ?- -.q.i.xiq
2645− &
2646− =+ cas=?~(ref ~ (~(get by haw.u.ref) `mood`p.q.i.xiq))
2647− ?^ cas
2648− %= $
2649− xiq t.xiq
2650− ..wake ?~ u.cas (blub p.i.xiq)
2651− (blab p.i.xiq p.q.i.xiq u.u.cas)
2652− ==
2653− =+ nao=(~(case-to-aeon ze lim dom ran) q.p.q.i.xiq)
2654− ?~ nao $(xiq t.xiq, xaq [i.xiq xaq])
2655− $(xiq t.xiq, ..wake (balk p.i.xiq u.nao p.q.i.xiq))
2656− ::
2657− |
2658− =+ mot=`moot`p.q.i.xiq
2659− =+ nab=(~(case-to-aeon ze lim dom ran) p.mot)
2660− ?~ nab
2661− $(xiq t.xiq, xaq [i.xiq xaq])
2662− =+ huy=(~(case-to-aeon ze lim dom ran) q.mot)
2663− ?~ huy
2664− =+ ptr=[%ud +(let.dom)]
2665− %= $
2666− xiq t.xiq
2667− xaq [[p.i.xiq [%| ptr q.mot r.mot s.mot]] xaq]
2668− ..wake =+ ^= ear
2669− (~(lobes-at-path ze lim dom ran) let.dom r.p.q.i.xiq)
2670− ?: =(s.p.q.i.xiq ear) ..wake
2671− =+ fud=(~(make-nako ze lim dom ran) u.nab let.dom)
2672− (bleb p.i.xiq let.dom fud)
2673− ==
2674− %= $
2675− xiq t.xiq
2676− ..wake =- (blub:- p.i.xiq)
2677− =+ ^= ear
2678− (~(lobes-at-path ze lim dom ran) u.huy r.p.q.i.xiq)
2679− ?: =(s.p.q.i.xiq ear) (blub p.i.xiq)
2680− =+ fud=(~(make-nako ze lim dom ran) u.nab u.huy)
2681− (bleb p.i.xiq +(u.nab) fud)
2682− ==
2683− ==
2684− --
2685−```
2686−
2687−This is even longer than `++knit`, but it's pretty similar to `++eave`.
2688−We loop through each of our subscribers `xiq`, processing each in turn.
2689−When we're done, we just put the remaining subscribers back in our
2690−subscriber list.
2691−
2692−If the subscriber is a single read, then, if this is a foreign desk
2693−(note that `++wake` is called from other arms, and not only on foreign
2694−desks). Obviously, if we find an identical request there, then we can
2695−produce the result immediately. Referential transparency for the win. We
2696−produce the result with a call to `++blab`. If this is a foreign desk
2697−but the result is not in the cache, then we produce `++blub` (canceled
2698−subscription with no data) for reasons entirely opaque to me. Seriously,
2699−it seems like we should wait until we get an actual response to the
2700−request. If someone figures out why this is, let me know. At any rate,
2701−it seems to work.
2702−
2703−If this is a domestic desk, then we check to see if the case exists yet.
2704−If it doesn't, then we simply move on to the next subscriber, consing
2705−this one onto `xaq` so that we can check again the next time around. If
2706−it does exist, then we call `++balk` to fulfill the request and produce
2707−it.
2708−
2709−`++balk` is very simple, so we'll describe it here before we get to the
2710−subscription case.
2711−
2712−```hoon
2713− ++ balk :: read and send
2714− |= [hen=duct oan=@ud mun=mood]
2715− ^+ +>
2716− =+ vid=(~(read-at-aeon ze lim dom ran) oan mun)
2717− ?~ vid (blub hen) (blab hen mun u.vid)
2718−```
2719−
2720−We call `++read-at-aeon` on the given request and aeon. If you recall,
2721−this processes a `mood` at a particular aeon and produces the result, if
2722−there is one. If there is data at the requested location, then we
2723−produce it with `++blab`. Else, we call `++blub` to notify the
2724−subscriber that no data can ever come over this subscriptioin since it
2725−is now impossible for there to ever be data for the given request.
2726−Because referential transparency.
2727−
2728−At any rate, back to `++wake`. If the given rave is a subscription
2729−request, then we proceed similarly to how we do in `++eave`. We first
2730−try to get the aeon referred to by the starting case. If it doesn't
2731−exist yet, then we can't do anything interesting with this subscription,
2732−so we move on to the next one.
2733−
2734−Otherwise, we try to get the aeon referred to by the ending case. If it
2735−doesn't exist yet, then we produce all the information we can. We call
2736−`++lobes-at-path` at the given aeon and path to see if the requested
2737−path has actually changed. If it hasn't, then we don't produce anything;
2738−else, we produce the correct nako by calling `++bleb` on the result of
2739−`++make-nako`, as in `++eave`. At any rate, we move on to the next
2740−subscription, putting back into our cult the current subscription with a
2741−new start case of the next aeon after the present.
2742−
2743−If the aeon referred to by the ending case does exist, then we drop this
2744−subscriber from our cult and satisfy its request immediately. This is
2745−the same as before -- we check to see if the data at the path has
2746−actually changed, producing it if it has; else, we call `++blub` since
2747−no more data can be produced over this subscription.
2748−
2749−This concludes our discussion of foreign requests.