@@ -5,14 +5,15 @@ template = "doc.html"
55+++
66
77This section will be reference documentation for the data types used by our
8−filesystem. Parts of it may be inaccurate as of March 2021. As a general guide,
9−we recommend reading and attempting to understand the data structures used in
10−any Hoon code before you try to read the code itself. Although complete
11−understanding of the data structures is impossible without seeing them used in
12−the code, an 80% understanding greatly clarifies the code. As another general
13−guide, when reading Hoon, it rarely pays off to understand every line of code
14−when it appears. Try to get the gist of it, and then move on. The next time you
15−come back to it, it'll likely make a lot more sense.
8+filesystem. It is up to date as of April 2021.
9+
10+As a general guide, we recommend reading and attempting to understand the data
11+structures used in any Hoon code before you try to read the code itself.
12+Although complete understanding of the data structures is impossible without
13+seeing them used in the code, an 80% understanding greatly clarifies the code.
14+As another general guide, when reading Hoon, it rarely pays off to understand
15+every line of code when it appears. Try to get the gist of it, and then move on.
16+The next time you come back to it, it'll likely make a lot more sense.
1617
1718### Data Models
1819
@@ -23,191 +24,275 @@ at least skimming, this so that you get a rough idea of how our state is
2324organized.
2425
2526The types that are certainly worth reading are `++raft`, `++room`,
26−`++dome:clay`, `++ankh:clay`, `++rung:clay`, `++rang:clay`, `++blob:clay`, `++yaki:clay`, and `++nori:clay`
27−(possibly in that order). All in all, though, this section isn't too
28−long, so many readers may wish to quickly read through all of it. If you
29−get bored, though, just skip to the next section. You can always come
30−back when you need to.
27+`++dome:clay`, `++ankh:clay`, `++rung:clay`, `++rang:clay`, `++blob:clay`,
28+`++yaki:clay`, and `++nori:clay` (possibly in that order). All in all, though,
29+this section isn't too long, so many readers may wish to quickly read through
30+all of it. If you get bored, though, just skip to the next section. You can
31+always come back when you need to.
3132
32−### `++raft`, formal state
33+### `$raft`, formal state
3334
3435```hoon
35− ++ raft :: filesystem
36− $: fat=(map ship room) :: domestic
37− hoy=(map ship rung) :: foreign
38− ran=rang :: hashes
39− == ::
36++$ raft :: filesystem
37+ $: rom=room :: domestic
38+ hoy=(map ship rung) :: foreign
39+ ran=rang :: hashes
40+ mon=(map term beam) :: mount points
41+ hez=(unit duct) :: sync duct
42+ cez=(map @ta crew) :: permission groups
43+ pud=(unit [=desk =yoki]) :: pending update
44+ == ::
4045```
4146
42−This is the state of our vane. Anything that must be remembered between
43−calls to clay is stored in this state.
44−
45−`fat` is the set of domestic servers. This stores all the information
46−that is specfic to a particular ship on this pier. The keys to this map
47−are the ships on the current pier. all the information that is specific
48−to a particular foreign ship. The keys to this map are all the ships
49−whose filesystems we have attempted to access through clay.
47+This is the state of the vane. Anything that must be remembered between
48+calls to Clay is stored in this state.
5049
5150`ran` is the store of all commits and deltas, keyed by hash. The is
5251where all the "real" data we know is stored; the rest is "just
5352bookkeeping".
5453
55−#### `++room`, filesystem per domestic ship
54+`rom` is the state for all local desks. It consists of a `duct` to
55+[Dill](@/docs/arvo/dill/dill.md) and a collection of `desk`s.
56+
57+`hoy` is the state for all foreign desks.
58+
59+`ran` is the global, hash-addressed object store. It has maps of commit hashes
60+to commits and content hashes to content.
61+
62+`mon` is a collection of Unix mount points. `term` is the mount point (relative
63+to th pier) and `beam` is a domestic Clay directory.
64+
65+`hez` is the duct used to sync with Unix.
66+
67+`cez` is a collection of named permission groups.
68+
69+`pud` is an update that's waiting on a kernel upgrade.
70+
71+
72+#### `$room`, filesystem per domestic ship
5673
5774```hoon
58− ++ room :: fs per ship
59− $: hun=duct :: terminal duct
60− hez=(unit duct) :: sync duch
61− dos=(map desk dojo) :: native desk
62− == ::
75++$ room :: fs per ship
76+ $: hun=duct :: terminal duct
77+ dos=(map desk dojo) :: native desk
78+ == ::
6379```
6480
6581This is the representation of the filesystem of a ship on our pier.
6682
67−`hun` is the duct we use to send messages to dill to display
68−notifications of filesystem changes. Only `%note` gifts should be
69−produced along this duct. This is set by the `%init` kiss.
70−
71−`hez`, if present, is the duct we use to send sync messages to unix so
72−that they end up in the pier unix directory. Only `%ergo` gifts should
73−be producd along this duct. This is set by `%into` and `%invo` kisses.
83+`hun` is the duct we use to send messages to [Dill](@/docs/arvo/dill/dill.md) to
84+display notifications of filesystem changes. Only `%note` `%gift`s should be
85+produced along this `duct`. This is set by the `%init` `move`.
7486
7587`dos` is a well-known operating system released in 1981. It is also the
76−set of desks on this ship, mapped to their data.
88+set of `desk`s on this ship, mapped to their `desk` state.
7789
78−#### `++desk`, filesystem branch
90+#### `$desk`, filesystem branch
7991
8092```hoon
81− ++ desk ,@tas :: ship desk case spur
93++$ desk @tas
8294```
8395
84−This is the name of a branch of the filesystem. The default desks are
85−"arvo", "main", and "try". More may be created by simply referencing
86−them. Desks have independent histories and states, and they may be
87−merged into each other.
96+This is the name of a branch of the filesystem. The default `desk`s are `%home`
97+and `%kids`. More may be created by simply referencing them. `desk`s have
98+independent histories and states, and they may be
99+[merged](@/docs/arvo/clay/using.md#merging) into each other.
88100
89−### `++dojo`, domestic desk state
101+### `$dojo`, domestic desk state
90102
91103```hoon
92− ++ dojo ,[p=cult q=dome] :: domestic desk state
104++$ dojo
105+ $: qyx=cult :: subscribers
106+ dom=dome :: desk state
107+ per=regs :: read perms per path
108+ pew=regs :: write perms per path
109+ ==
93110```
94111
95−This is the all the data that is specific to a particular desk on a
96−domestic ship. `p` is the set of subscribers to this desk and `q` is the
97−data in the desk.
112+This is the all the data that is specific to a particular `desk` on a domestic
113+ship. `qyx` is the set of subscribers to this `desk`, and `dom` is the data in
114+the `desk`. `regs` are `(map path rule)`, and so `per` is a `map` of read
115+permissions by `path` and `pew` is a `map` of write permissions by `path`.
98116
99−#### `++cult`, subscriptions
117+#### `$cult`, subscriptions
100118
101119```hoon
102− ++ cult (map duct rave) :: subscriptions
120++$ cult (jug wove duct)
103121```
104122
105−This is the set of subscriptions to a particular desk. The keys are the
106−ducts from where the subscriptions requests came. The results will be
107−produced along these ducts. The values are a description of the
108−requested information.
123+`cult`s keep track of subscribers. `wove`s are associated to requests, and each
124+`wove` is mapped to a set of `duct`s associated to subscribers who should be
125+notified when the request is filled/updated.
109126
110−#### `++rave:clay`, general subscription request
127+#### `$rave:clay`, general subscription request
111128
112129```hoon
113− ++ rave :: general request
114− $% [& p=mood] :: single request
115− [| p=moat] :: change range
116− == ::
130+ +$ rave :: general request
131+ $% [%sing =mood] :: single request
132+ [%next =mood] :: await next version
133+ [%mult =mool] :: next version of any
134+ [%many track=? =moat] :: track range
135+ == ::
117136```
118137
119−This represents a subscription request for a desk. The request can be
120−for either a single item in the desk or else for a range of changes on
121−the desk.
138+This represents a subscription request for a `desk`.
122139
123−### `++rove`, stored general subscription request
140+A `%sing` request asks for data at single revision.
124141
125−```hoon
126− ++ rove (each mood moot) :: stored request
127−```
142+A `%next` request asks to be notified the next time there’s a change to the
143+specified file.
128144
129−When we store a request, we store subscriptions with a little extra
130−information so that we can determine whether new versions actually
131−affect the path we're subscribed to.
145+A `%mult` request asks to be notified the next time there's a change to a
146+specified set of files.
132147
133−#### `++mood:clay`, single subscription request
148+A `%many` request asks to be notified on every change in a `desk` for a range of changes (including into the future).
149+
150+### `$rove`, stored general subscription request
134151
135152```hoon
136− ++ mood ,[p=care q=case r=path] :: request in desk
153+ +$ rove :: stored request
154+ $% [%sing =mood] :: single request
155+ [%next =mood aeon=(unit aeon) =cach] :: next version of one
156+ $: %mult :: next version of any
157+ =mool :: original request
158+ aeon=(unit aeon) :: checking for change
159+ old-cach=(map [=care =path] cach) :: old version
160+ new-cach=(map [=care =path] cach) :: new version
161+ == ::
162+ [%many track=? =moat lobes=(map path lobe)] :: change range
163+ == ::
137164```
138165
139−This represents a request for the state of the desk at a particular
140−commit, specfied by `q`. `p` specifies what kind of information is
141−desired, and `r` specifies the path we are requesting.
166+Like a `rave` but with provisions to store current versions for `%next` and `%many`.
167+Generally used when we store a request in our state somewhere. This is so that
168+we can determine whether new versions actually affect the path we're subscribed to.
169+
142170
143−#### `++moat:clay`, range subscription request
171+#### `$mood:clay`, single subscription request
144172
145173```hoon
146− ++ moat ,[p=case q=case r=path] :: change range
174+ +$ mood [=care =case =path] :: request in desk
147175```
148176
149−This represents a request for all changes between `p` and `q` on path
150−`r`. You will be notified when a change is made to the node referenced
151−by the path or to any of its children.
177+This represents a request for data related to the state of the `desk` at a
178+particular commit, specfied by `case`. `care` specifies what kind of information
179+is desired, and `path` specifies the path we are requesting.
152180
153−### `++moot`, stored range subscription request
181+#### `$moat:clay`, range subscription request
154182
155183```hoon
156− ++ moot ,[p=case q=case r=path s=(map path lobe)] ::
184+ +$ moat [from=case to=case =path] :: change range
157185```
158186
159−This is just a `++moat:clay` plus a map of paths to lobes. This map
160−represents the data at the node referenced by the path at case `p`, if
161−we've gotten to that case (else null). We only send a notification along
162−the subscription if the data at a new revision is different than it was.
187+This represents a request for all changes between `from` and `to` on `path`. You
188+will be notified when a change is made to the node referenced by the `path` or to
189+any of its children.
163190
164−### `++care:clay`, clay submode
191+### `$care:clay`, Clay submode
165192
166193```hoon
167− ++ care ?(%u %v %w %x %y %z) :: clay submode
194+ +$ care ?(%a %b %c %d %e %f %p %r %s %t %u %v %w %x %y %z) :: clay submode
168195```
169196
197+
170198This specifies what type of information is requested in a subscription
171199or a scry.
172200
173−`%u` requests the `++rang:clay` at the current moment. Because this
174−information is not stored for any moment other than the present, we
175−crash if the `++case:clay` is not a `%da` for now.
201+`%a` build a Hoon file at a `path`.
202+
203+`%b` build a dynamically typed `mark` by name (a `$dais` mark-interface core).
204+
205+`%c` build a dynamically typed `mark` conversion gate (a `$tube`) by "from" and
206+"to" `mark` names.
207+
208+`%d` returns a `(set desk)` of the `desk`s that exist on your ship.
209+
210+`%e` builds a statically typed `mark` by name (a `$nave` mark-interface core).
211+
212+`%f` builds a statically typed mark converstion gate.
213+
214+`%p` produces the permissions for a directory, returned as a `[dict:clay dict:clay]`.
215+
216+`%r` requests the file in the same fashion as `%x`, but wraps the result in a `vase`.
217+
218+`%s` has miscellaneous debug endpoints.
176219
177−`%v` requests the `++dome:clay` at the specified commit.
220+`%t` produces a `(list path)` of descendent `path`s for a directory within a `yaki`.
178221
179−`%w` requests the revsion number of the desk.
222+`%u` produces a `?` depending on whether or not the specified file exists. It
223+does not check any of its children.
180224
181−`%x` requests the file at a specified path at the specified commit. If
182−there is no node at that path or if the node has no contents (that is,
183−if `q:ankh` is null), then this produces null.
225+`%v` requests the entire `dome` for a specified `desk` at a particular `aeon`.
226+When used on a foreign `desk`, this get us up-to-date to the requested version.
184227
185−`%y` requests a `++arch` of the specfied commit at the specified path.
228+`%w` requests the revision number and date of the specified path, returned as a `cass:clay`.
186229
187−`%z` requests the `++ankh` of the specified commit at the specfied path.
230+`%x` requests the file at a specified path at the specified commit, returned as
231+an `@`. If there is no node at that path or if the node has no contents (that
232+is, if `fil:ankh` is null), then this crashes.
188233
189−### `++arch`, shallow filesystem node
234+`%y` requests an `arch` of the specfied commit at the specified path. It will
235+return the bunt of an `arch` if the file or directory is not found.
236+
237+`%z` requests a recursive hash of a node and all its children, returned as a `@uxI`.
238+
239+#### `$ankh`, filesystem node
190240
191241```hoon
192− ++ arch ,[p=@uvI q=(unit ,@uvI) r=(map ,@ta ,~)] :: fundamental node
242++$ ankh :: expanded node
243+ $~ [~ ~]
244+ $: fil=(unit [p=lobe q=cage]) :: file
245+ dir=(map @ta ankh) :: folders
246+ == ::
193247```
194248
195−This is analogous to `++ankh:clay` except that the we have neither our
196−contents nor the ankhs of our children. The other fields are exactly the
197−same, so `p` is a hash of the associated ankh, `u.q`, if it exists, is a
198−hash of the contents of this node, and the keys of `r` are the names of
199−our children. `r` is a map to null rather than a set so that the
200−ordering of the map will be equivalent to that of `r:ankh`, allowing
201−efficient conversion.
249+This is a recursive filesystem node type that can describe a whole tree of
250+folders and files, with subtrees organized by path prefix. In Earth filesystems,
251+a node is a file xor a directory. On Mars, we're inclusive, so a node is a file
252+ior a directory.
253+
254+`fil` is the contents of this file, if any. `p.fil` is a hash of the
255+contents while `q.fil` is the data itself.
202256
203−#### `++case:clay`, specifying a commit
257+`dir` is the set of children of this node. In the case of a pure file,
258+this is empty. The keys are the names of the children and the values
259+are, recursively, the nodes themselves.
260+
261+
262+### `$arch`, shallow filesystem node
204263
205264```hoon
206− ++ case :: ship desk case spur
207− $% [%da p=@da] :: date
208− [%tas p=@tas] :: label
209− [%ud p=@ud] :: number
210− == ::
265++$ arch (axil @uvI)
266+++ axil
267+ |$ [item]
268+ [fil=(unit item) dir=(map @ta ~)]
269+```
270+
271+An `arch` is a lightweight version of an `ankh` that only contains the hash
272+of the associated file and a `map` of child directories, but not the children of
273+those directories.
274+
275+The child directories are given by a `map` to null rather than a `set` so that the
276+ordering of the `map` will be the same as it is for an `axal`, allowing
277+efficient conversion for when the heavier node is needed.
278+```hoon
279+++ axal
280+ |$ [item]
281+ [fil=(unit item) dir=(map @ta $)]
282+```
283+
284+#### `$case`, specifying a commit
285+
286+```hoon
287++$ case
288+ $% :: %da: date
289+ :: %tas: label
290+ :: %ud: sequence
291+ ::
292+ [%da p=@da]
293+ [%tas p=@tas]
294+ [%ud p=@ud]
295+ ==
211296```
212297
213298A commit can be referred to in three ways: `%da` refers to the commit
@@ -216,91 +301,66 @@ that was at the head on date `p`, `%tas` refers to the commit labeled
216301all can be reduced down to a `%ud`, only numbered commits may be
217302referenced with a `++case:clay`.
218303
219−#### `++dome:clay`, desk data
304+#### `$dome`, desk data
220305
221306```hoon
222− ++ dome :: project state
223− $: ang=agon :: pedigree
224− ank=ankh :: state
225− let=@ud :: top id
226− hit=(map ,@ud tako) :: changes by id
227− lab=(map ,@tas ,@ud) :: labels
228− == ::
307++$ dome
308+ $: ank=ankh :: state
309+ let=aeon :: top id
310+ hit=(map aeon tako) :: versions by id
311+ lab=(map @tas aeon) :: labels
312+ mim=(map path mime) :: mime cache
313+ fod=ford-cache :: ford cache
314+ fer=(unit reef-cache) :: reef cache
315+ == ::
229316```
230317
231−This is the data that is actually stored in a desk.
232−
233−`ang` is unused and should be removed.
318+A `dome` is the state of a `desk` and associated data.
234319
235320`ank` is the current state of the desk. Thus, it is the state of the
236−filesystem at revison `let`. The head of a desk is always a numbered
321+filesystem at revison `let`. The head of a `desk` is always a numbered
237322commit.
238323
239324`let` is the number of the most recently numbered commit. This is also
240325the total number of numbered commits.
241326
242−`hit` is a map of numerical ids to hashes of commits. These hashes are
243−mapped into their associated commits in `hut:rang:clay`. In general, the keys
244−of this map are exactly the numbers from 1 to `let`, with no gaps. Of
245−course, when there are no numbered commits, `let` is 0, so `hit` is
246−null. Additionally, each of the commits is an ancestor of every commit
247−numbered greater than this one. Thus, each is a descendant of every
248−commit numbered less than this one. Since it is true that the date in
249−each commit (`t:yaki`) is no earlier than that of each of its parents,
250−the numbered commits are totally ordered in the same way by both
251−pedigree and date. Of course, not every commit is numbered. If that
252−sounds too complicated to you, don't worry about it. It basically
253−behaves exactly as you would expect.
327+`hit` is a map of numerical ids to commit hashes. These hashes are mapped into
328+their associated commits in `hut.rang` in the `raft` of Clay. In general, the
329+keys of this map are exactly the numbers from 1 to `let`, with no gaps. Of
330+course, when there are no numbered commits, `let` is 0, so `hit` is null.
331+Additionally, each of the commits is an ancestor of every commit numbered
332+greater than this one. Thus, each is a descendant of every commit numbered less
333+than this one. Since it is true that the date in each commit (`t:yaki`) is no
334+earlier than that of each of its parents, the numbered commits are totally
335+ordered in the same way by both pedigree and date. Of course, not every commit
336+is numbered. If that sounds too complicated to you, don't worry about it. It
337+basically behaves exactly as you would expect.
254338
255339`lab` is a map of textual labels to numbered commits. Note that labels
256340can only be applied to numbered commits. Labels must be unique across a
257341desk.
258342
259−#### `++ankh`, filesystem node
343+`mim` is a cache of the content in the directories that are mounted to Unix.
260344
261−```hoon
262− ++ ankh :: fs node (new)
263− $: p=cash :: recursive hash
264− q=(unit ,[p=cash q=*]) :: file
265− r=(map ,@ta ankh) :: folders
266− == ::
267−```
268−
269−This is a single node in the filesystem. This may be file or a directory
270−or both. In earth filesystems, a node is a file xor a directory. On
271−mars, we're inclusive, so a node is a file ior a directory.
272−
273−`p` is a recursive hash that depends on the contents of the this file or
274−directory and on any children.
275−
276−`q` is the contents of this file, if any. `p.q` is a hash of the
277−contents while `q.q` is the data itself.
278−
279−`r` is the set of children of this node. In the case of a pure file,
280−this is empty. The keys are the names of the children and the values
281−are, recursively, the nodes themselves.
345+`fod` is the Ford cache, which keeps a cache of the results of builds performed
346+at this `desk`'s current revision, including a full transitive closure of
347+dependencies for each completed build.
282348
283−#### `++cash`, ankh hash
284−
285−```hoon
286− ++ cash ,@uvH :: ankh hash
287−```
288−
289−This is a 128-bit hash of an ankh. These are mostly stored within ankhs
290−themselves, and they are used to check for changes in possibly-deep
291−hierarchies.
349+`fer` is the system file cache, which consists of `vase`s for `hoon.hoon`,
350+`arvo.hoon`, `lull.hoon`, and `zuse.hoon`.
292351
293352#### `++rung`, filesystem per neighbor ship
294353
295354```hoon
296− ++ rung $: rus=(map desk rede) :: neighbor desks
297− == ::
355++$ rung
356+ $: rus=(map desk rede) :: neighbor desks
357+ ==
298358```
299359
300−This is the filesystem of a neighbor ship. The keys to this map are all
301−the desks we know about on their ship.
360+This is the filesystem of a neighbor ship. The keys to this `map` are all
361+the `desk`s we know about on their ship.
302362
303−#### `++rede`, desk state
363+#### `++rede`, generic desk state
304364
305365```hoon
306366 ++ rede :: universal project
@@ -309,342 +369,312 @@ the desks we know about on their ship.
309369 ref=(unit rind) :: outgoing requests
310370 dom=dome :: revision state
311371 == ::
372++$ rede :: universal project
373+ $: lim=@da :: complete to
374+ ref=(unit rind) :: outgoing requests
375+ qyx=cult :: subscribers
376+ dom=dome :: revision state
377+ per=regs :: read perms per path
378+ pew=regs :: write perms per path
379+ == ::
312380```
313381
314382This is our knowledge of the state of a desk, either foreign or
315383domestic.
316384
317−`lim` is the date of the last full update. We only respond to requests
318−for stuff before this time.
385+`lim` is the most recent `@da` for which we're confident we have all the
386+information for. For local `desk`s, this is always `now`. For foriegn `desk`s,
387+this is the last time we got a full update from the foreign ship.
319388
320−`qyx` is the list of subscribers to this desk. For domestic desks, this
321−is simply `p:dojo`, all subscribers to the desk, while in foreign desks
322−this is all the subscribers from our ship to the foreign desk.
389+`ref` is the request manager for the desk. For domestic `desk`s, this is
390+null since we handle requests ourselves. For foreign `desk`s, this keeps track
391+of all pending foriegn requests plus a cache of the responses to previous requests.
323392
324−`ref` is the request manager for the desk. For domestic desks, this is
325−null since we handle requests ourselves.
393+`qyx` is the `set` of subscriptions to this desk, with listening `duct`s. These
394+subscriptions exist only until they've been filled. For domestic `desk`s, this
395+is simply `qyx:dojo` - all subscribers to the `desk`. For foreign `desk`s this
396+is all the subscribers from our ship to the foreign `desk`.
326397
327−`dom` is the actual data in the desk.
398+`dom` is the data in the `desk`.
328399
329−#### `++rind`, request manager
400+`regs` are `(map path rule)`, and so `per` is a `map` of read permissions by
401+`path` and `pew` is a `map` of write permissions by `path`.
402+
403+
404+#### `$rind`, foreign request manager
330405
331406```hoon
332− ++ rind :: request manager
333− $: nix=@ud :: request index
334− bom=(map ,@ud ,[p=duct q=rave]) :: outstanding
335− fod=(map duct ,@ud) :: current requests
336− haw=(map mood (unit)) :: simple cache
337− == ::
407++$ rind :: request manager
408+ $: nix=@ud :: request index
409+ bom=(map @ud update-state) :: outstanding
410+ fod=(map duct @ud) :: current requests
411+ haw=(map mood (unit cage)) :: simple cache
412+ == ::
338413```
339414
340−This is the request manager for a foreign desk.
415+This is the request manager for a foreign `desk`. When we send a request to a
416+foreign ship, we keep track of it in here.
341417
342−`nix` is one more than the index of the most recent request. Thus, it is
343−the next available request number.
418+`nix` is one more than the index of the most recent request. Thus, it is the
419+next available request number.
344420
345−`bom` is the set of outstanding requests. The keys of this map are some
346−subset of the numbers between 0 and one less than `nix`. The members of
347−the map are exactly those requests that have not yet been fully
348−satisfied.
421+`bom` is the set of outstanding requests. The keys of this `map` are some subset
422+of the numbers between 0 and one less than `nix`. The members of the `map` are
423+exactly those requests that have not yet been fully satisfied.
349424
350−`fod` is the same set as `bom`, but from a different perspective. In
351−particular, the values of `fod` are the same as the values of `bom`, and
352−the `p` out of the values of `bom` are the same as the keys of `fod`.
353−Thus, we can map ducts to their associated request number and `++rave:clay`,
354−and we can map numbers to their associated duct and `++rave:clay`.
425+`fod` is the same set as `bom`, but from a different perspective. In particular,
426+the values of `fod` are the same as the keys of `bom`, and the `duct` of the
427+values of `bom` are the same as the keys of `fod`. Thus, we can map `duct`s to
428+their associated request number and `update-state`, and we can map request
429+numbers to their associated `duct`.
355430
356−`haw` is a map from simple requests to their values. This acts as a
357−cache for requests that have already been made. Thus, the second request
358−for a particular `++mood:clay` is nearly instantaneous.
431+`haw` is a map from `%sing` requests to their values. This acts as a cache for
432+requests that have already been filled.
359433
360−#### `++rang:clay`, data store
434+#### `$update-state`, status of outstanding foreign request
361435
362436```hoon
363− ++ rang $: hut=(map tako yaki) ::
364− lat=(map lobe blob) ::
365− == ::
437++$ update-state
438+ $: =duct
439+ =rave
440+ have=(map lobe blob)
441+ need=(list lobe)
442+ nako=(qeu (unit nako))
443+ busy=_|
444+ ==
445+```
446+
447+An `update-state` is used to represent the status of an outstanding request to a
448+foreign `desk`.
449+
450+`duct` and `rave` are the `duct` along which the request was made and the
451+request itself.
452+
453+`have` is a map of hashes thus far acquired in the request to the data
454+associated with those hashes.
455+
456+`need` is a list of hashes yet to be acquired.
457+
458+`nako` is a queue of data yet to be validated.
459+
460+`busy` tracks whether or not the request is currently being fulfilled.
461+
462+#### `$rang:clay`, data repository
463+
464+```hoon
465+ +$ rang :: repository
466+ $: hut=(map tako yaki) :: changes
467+ lat=(map lobe blob) :: data
468+ == ::
366469```
367470
368−This is a set of data keyed by hash. Thus, this is where the "real" data
471+This is a data repository keyed by hash. Thus, this is where the "real" data
369472is stored, but it is only meaningful if we know the hash of what we're
370473looking for.
371474
372−`hut` is a map from hashes to commits. We often get the hashes from
373−`hit:dome:clay`, which keys them by logical id. Not every commit has an id.
475+`hut` is a `map` from commit hashes (`tako`s) to commits (`yaki`s). We often get
476+the hashes from `hit:dome`, which keys them by numerical id. Not every commit
477+has an numerical id.
374478
375−`lat` is a map from hashes to the actual data. We often get the hashes
376−from a `++yaki`, a commit, which references this map to get the data.
377−There is no `++blob:clay` in any `++yaki:clay`. They are only accessible through
378−this map.
479+`lat` is a `map` from content hashes (`lobe`s) to the actual content (`blob`s).
480+We often get the hashes from a `yaki`, which references this `map` to get the
481+data. There is no `blob` in `yaki:clay`. They are only accessible through
482+`lat`.
379483
380−#### `++tako:clay`, commit reference
484+#### `$tako:clay`, commit reference
381485
382486```hoon
383− ++ tako ,@ :: yaki ref
487+ +$ tako @ :: yaki ref
384488```
385489
386−This is a hash of a `++yaki:clay`, a commit. These are most notably used as
387−the keys in `hut:rang:clay`, where they are associated with the actual
388−`++yaki:clay`, and as the values in `hit:dome:clay`, where sequential ids are
490+This is a hash of a `yaki:clay`, a commit. These are most notably used as the
491+keys in `hut:rang:clay`, where they are associated with the actual `yaki:clay`,
492+and as the values in `hit:dome:clay`, where sequential numerical ids are
389493associated with these.
390494
391−#### `++yaki:clay`, commit
495+#### `$yaki:clay`, commit
392496
393497```hoon
394− ++ yaki ,[p=(list tako) q=(map path lobe) r=tako t=@da] :: commit
498+ +$ yaki :: commit
499+ $: p=(list tako) :: parents
500+ q=(map path lobe) :: namespace
501+ r=tako :: self-reference
502+ t=@da :: date
503+ == ::
395504```
396505
397506This is a single commit.
398507
399−`p` is a list of the hashes of the parents of this commit. In most
508+`p` is a `list` of the hashes of the parents of this commit. In most
400509cases, this will be a single commit, but in a merge there may be more
401510parents. In theory, there may be an arbitrary number of parents, but in
402511practice merges have exactly two parents. This may change in the future.
403512For commit 1, there is no parent.
404513
405−`q` is a map of the paths on a desk to the data at that location. If you
406−understand what a `++lobe:clay` and a `++blob:clay` is, then the type signature
407−here tells the whole story.
514+`q` is a `map` of the `path`s on a desk to the content hashes at that location.
515+If you understand what a `lobe:clay` and a `blob:clay` is, then the type
516+signature here tells the whole story.
408517
409518`r` is the hash associated with this commit.
410519
411520`t` is the date at which this commit was made.
412521
413−#### `++lobe:clay`, data reference
522+#### `$lobe:clay`, data reference
414523
415524```hoon
416− ++ lobe ,@ :: blob ref
525+ +$ lobe @uvI :: blob ref
417526```
418527
419−This is a hash of a `++blob:clay`. These are most notably used in `lat:rang:clay`,
420−where they are associated with the actual `++blob:clay`, and as the values in
421−`q:yaki:clay`, where paths are associated with their data in a commit.
422−
423−#### `++blob:clay`, data
424−
425−```hoon
426− ++ blob $% [%delta p=lobe q=lobe r=udon] :: delta on q
427− [%direct p=lobe q=* r=umph] ::
428− [%indirect p=lobe q=* r=udon s=lobe] ::
429− == ::
430−```
431−
432−This is a node of data. In every case, `p` is the hash of the blob.
433−
434−`%delta` is the case where we define the data by a delta on other data.
435−In practice, the other data is always the previous commit, but nothing
436−depends on this. `q` is the hash of the parent blob, and `r` is the
437−delta.
528+This is a hash of a `blob:clay`. These are most notably used in `lat:rang:clay`,
529+where they are associated with the actual `blob:clay`, and as the values in
530+`q:yaki:clay`, where `path`s are associated with their content hashes in a commit.
438531
439−`%direct` is the case where we simply have the data directly. `q` is the
440−data itself, and `r` is any preprocessing instructions. These almost
441−always come from the creation of a file.
442−
443−`%indirect` is both of the preceding cases at once. `q` is the direct
444−data, `r` is the delta, and `s` is the parent blob. It should always be
445−the case that applying `r` to `s` gives the same data as `q` directly
446−(with the prepreprocessor instructions in `p.r`). This exists purely for
447−performance reasons. This is unused, at the moment, but in general these
448−should be created when there are a long line of changes so that we do
449−not have to traverse the delta chain back to the creation of the file.
450−
451−#### `++udon`, abstract delta
532+#### `$blob:clay`, data
452533
453534```hoon
454− ++ udon :: abstract delta
455− $: p=umph :: preprocessor
456− $= q :: patch
457− $% [%a p=* q=*] :: trivial replace
458− [%b p=udal] :: atomic indel
459− [%c p=(urge)] :: list indel
460− [%d p=upas q=upas] :: tree edit
461− == ::
462− == ::
535+ +$ blob :: fs blob
536+ $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q
537+ [%direct p=lobe q=page] :: immediate
463538```
464539
465−This is an abstract change to a file. This is a superset of what would
466−normally be called diffs. Diffs usually refer to changes in lines of
467−text while we have the ability to do more interesting deltas on
468−arbitrary data structures.
469−
470−`p` is any preprocessor instructions.
471−
472−`%a` refers to the trival delta of a complete replace of old data with
473−new data.
540+This is a node of data. In both cases, `p` is the hash of the blob.
474541
475−`%b` refers to changes in an opaque atom on the block level. This has
476−very limited usefulness, and is not used at the moment.
542+`%delta` is the case where we define the data by a delta on other data. In
543+practice, the other data is always the previous commit, but nothing depends on
544+this. `p.q` is the `mark` of the parent blob, `q.q` is the hash of the parent
545+blob, and `r` is the delta.
477546
478−`%c` refers to changes in a list of data. This is often lines of text,
479−which is your classic diff. We, however, will work on any list of data.
547+`%direct` is the case where we simply have the data directly. `q` is the data.
548+These almost always come from the creation of a file.
480549
481−`%d` refers to changes in a tree of data. This is general enough to
482−describe changes to any hoon [noun](/docs/glossary/noun/), but often more special-purpose delta
483−should be created for different content types. This is not used at the
484−moment, and may in fact be unimplemented.
485550
486−#### `++urge`, list change
551+#### `+urge`, list change
487552
488553```hoon
489− ++ urge |*(a=_,* (list (unce a))) :: list change
554+ ++ urge |*(a=mold (list (unce a))) :: list change
490555```
491556
492−This is a parametrized type for list changes. For example, `(urge ,@t)`
557+This is a parametrized type for list changes. For example, `(urge @t)`
493558is a list change for lines of text.
494559
495−#### `++unce`, change part of a list.
560+#### `+unce`, change part of a list.
496561
497562```hoon
498− ++ unce |* a=_,* :: change part
499− $% [%& p=@ud] :: skip[copy]
500− [%| p=(list a) q=(list a)] :: p -> q[chunk]
501− == ::
563+ ++ unce :: change part
564+ |* a=mold ::
565+ $% [%& p=@ud] :: skip[copy]
566+ [%| p=(list a) q=(list a)] :: p -> q[chunk]
567+ == ::
502568```
503569
504570This is a single change in a list of elements of type `a`. For example,
505−`(unce ,@t)` is a single change in a lines of text.
571+`(unce @t)` is a single change in lines of text.
506572
507573`%&` means the next `p` lines are unchanged.
508574
509575`%|` means the lines `p` have changed to `q`.
510576
511−#### `++umph`, preprocessing information
512−
513−```hoon
514− ++ umph :: change filter
515− $| $? %a :: no filter
516− %b :: jamfile
517− %c :: LF text
518− == ::
519− $% [%d p=@ud] :: blocklist
520− == ::
521−```
522−
523−This space intentionally left undocumented. This stuff will change once
524−we get a well-typed clay.
525−
526−#### `++upas`, tree change
527−
528−```hoon
529− ++ upas :: tree change (%d)
530− $& [p=upas q=upas] :: cell
531− $% [%0 p=axis] :: copy old
532− [%1 p=*] :: insert new
533− [%2 p=axis q=udon] :: mutate!
534− == ::
535−```
536−
537−This space intentionally left undocumented. This stuff is not known to
538−work, and will likely change when we get a well-typed clay. Also, this
539−is not a complicated type; it is not difficult to work out the meaning.
540−
541−#### `++nori:clay`, repository action
577+#### `$nori:clay`, repository action
542578
543579```hoon
544− ++ nori :: repository action
545− $% [& q=soba] :: delta
546− [| p=@tas] :: label
547− == ::
580+ +$ nori :: repository action
581+ $% [%& p=soba] :: delta
582+ [%| p=@tas] :: label
583+ == ::
548584```
549585
550−This describes a change that we are asking clay to make to the desk.
586+This describes a change that we are asking Clay to make to the `desk`.
551587There are two kinds of changes that may be made: we can modify files or
552588we can apply a label to a commit.
553589
554590In the `|` case, we will simply label the current commit with the given
555591label. In the `&` case, we will apply the given changes.
556592
557−#### `++soba:clay`, delta
593+#### `$soba:clay`, delta
558594
559595```hoon
560− ++ soba ,[p=cart q=(list ,[p=path q=miso])] :: delta
596+ +$ soba (list [p=path q=miso]) :: delta
561597```
562598
563−This describes a set of changes to make to a desk. The `cart` is simply
564−a pair of the old hash and the new hash of the desk. The list is a list
565−of changes keyed by the file they're changing. Thus, the paths are paths
566−to files to be changed while `miso` is a description of the change
567−itself.
599+This describes a `list` of changes to make to a `desk`. The `path`s are `path`s
600+to files to be changed, and the corresponding `miso` value is a description of
601+the change itself.
568602
569−#### `++miso:clay`, ankh delta
603+#### `$miso:clay`, ankh delta
570604
571605```hoon
572− ++ miso :: ankh delta
573− $% [%del p=*] :: delete
574− [%ins p=*] :: insert
575− [%mut p=udon] :: mutate
576− == ::
606+ +$ miso :: ankh delta
607+ $% [%del ~] :: delete
608+ [%ins p=cage] :: insert
609+ [%dif p=cage] :: mutate from diff
610+ [%mut p=cage] :: mutate from raw
611+ == ::
577612```
578613
579−There are three kinds of changes that may be made to a node in a desk.
580−We can insert a file, in which case `p` is the contents of the new file.
581−We can delete a file, in which case `p` is the contents of the old file.
582−Finally, we can mutate that file, in which case the `udon` describes the
583−changes we are applying to the file.
614+There are four kinds of changes that may be made to a node in a `desk`.
584615
585−#### `++mizu:clay`, merged state
616+`%del` deletes the node.
586617
587−```hoon
588− ++ mizu ,[p=@u q=(map ,@ud tako) r=rang] :: new state
589−```
618+`%ins` inserts a file given by `p`.
619+
620+`%dif` is currently unimplemented. This may seem strange, so we remark that
621+diffs for individual files are implemented using `+diff` and `+pact` in `mark`s.
622+So for an `ankh`, which may include both files and directories, `%dif` being
623+unimplemented really just means that we do not yet have a formal concept of
624+changes in directory structure.
590625
591−This is the input to the `%merg` kiss, which allows us to perform a
592−merge. The `p` is the number of the new head commit. The `q` is a map
593−from numbers to commit hashes. This is all the new numbered commits that
594−are to be inserted. The keys to this should always be the numbers from
595−`let.dom` plus one to `p`, inclusive. The `r` is the maps of all the new
596−commits and data. Since these are merged into the current state, no old
597−commits or data need be here.
626+`%mut` mutates the file using raw data given by `p`.
598627
599−#### `++riff:clay`, request/desist
628+#### `$riff:clay`, request/desist
600629
601630```hoon
602− ++ riff ,[p=desk q=(unit rave)] :: request/desist
631+ +$ riff [p=desk q=(unit rave)] :: request+desist
603632```
604633
605−This represents a request for data about a particular desk. If `q`
606−contains a `rave`, then this opens a subscription to the desk for that
607−data. If `q` is null, then this tells clay to cancel the subscription
634+This represents a request for data about a particular `desk`. If `q`
635+contains a `rave`, then this opens a subscription to the `desk` for that
636+data. If `q` is null, then this tells Clay to cancel the subscription
608637along this duct.
609638
610−#### `++riot:clay`, response
639+#### `$riot:clay`, response
611640
612641```hoon
613− ++ riot (unit rant) :: response/complete
642+ +$ riot (unit rant) :: response+complete
614643```
615644
616−A riot is a response to a subscription. If null, the subscription has
645+A `riot` is a response to a subscription. If null, the subscription has
617646been completed, and no more responses will be sent. Otherwise, the
618647`rant` is the produced data.
619648
620−#### `++rant:clay`, response data
649+#### `$rant:clay`, response data
621650
622651```hoon
623− ++ rant :: namespace binding
624− $: p=[p=care q=case r=@tas] :: clade release book
625− q=path :: spur
626− r=* :: data
627− == ::
652+ +$ rant :: response to request
653+ $: p=[p=care q=case r=desk] :: clade release book
654+ q=path :: spur
655+ r=cage :: data
656+ == ::
628657```
629658
630−This is the data at a particular node in the filesystem. `p.p` specifies
659+This is the data associated to the response to a request. `p.p` specifies
631660the type of data that was requested (and is produced). `q.p` gives the
632661specific version reported (since a range of versions may be requested in
633−a subscription). `r.p` is the desk. `q` is the path to the filesystem
662+a subscription). `r.p` is the `desk`. `q` is the path to the filesystem
634663node. `r` is the data itself (in the format specified by `p.p`).
635664
636−### `++nako`, subscription response data
665+### `$nako`, subscription response data
637666
638667```hoon
639− ++ nako $: gar=(map ,@ud tako) :: new ids
640− let=@ud :: next id
641− lar=(set yaki) :: new commits
642− bar=(set blob) :: new content
643− == ::
668++$ nako :: subscription state
669+ $: gar=(map aeon tako) :: new ids
670+ let=aeon :: next id
671+ lar=(set yaki) :: new commits
672+ bar=(set plop) :: new content
673+ == ::
644674```
645675
646676This is the data that is produced by a request for a range of revisions
647−of a desk. This allows us to easily keep track of a remote repository --
677+of a `desk`. This allows us to easily keep track of a remote repository --
648678all the new information we need is contained in the `nako`.
649679
650680`gar` is a map of the revisions in the range to the hash of the commit
@@ -656,105 +686,3 @@ revision number, whichever is smaller.
656686
657687`lar` is the set of new commits, and `bar` is the set of new content.
658688
659−## Public Interface
660−
661−As with all vanes, there are exactly two ways to interact with clay.
662−`%clay` exports a namespace accessible through `.^`, which is described
663−above under `++care:clay`. The primary way of interacting with clay, though,
664−is by sending kisses and receiving gifts.
665−
666−```hoon
667− ++ gift :: out result <-$
668− $% [%ergo p=@p q=@tas r=@ud] :: version update
669− [%note p=@tD q=tank] :: debug message
670− [%writ p=riot] :: response
671− == ::
672− ++ kiss :: in request ->$
673− $% [%info p=@p q=@tas r=nori] :: internal edit
674− [%ingo p=@p q=@tas r=nori] :: internal noun edit
675− [%init p=@p] :: report install
676− [%into p=@p q=@tas r=nori] :: external edit
677− [%invo p=@p q=@tas r=nori] :: external noun edit
678− [%merg p=@p q=@tas r=mizu] :: internal change
679− [%wart p=sock q=@tas r=path s=*] :: network request
680− [%warp p=sock q=riff] :: file request
681− == ::
682−```
683−
684−There are only a small number of possible kisses, so it behooves us to
685−describe each in detail.
686−
687−```hoon
688− $% [%info p=@p q=@tas r=nori] :: internal edit
689−
690− [%into p=@p q=@tas r=nori] :: external edit
691−```
692−
693−These two kisses are nearly identical. At a high level, they apply
694−changes to the filesystem. Whenever we add, remove, or edit a file, one
695−of these cards is sent. The `p` is the ship whose filesystem we're
696−trying to change, the `q` is the desk we're changing, and the `r` is the
697−request change. For the format of the requested change, see the
698−documentation for `++nori:clay` above.
699−
700−When a file is changed in the unix filesystem, vere will send a `%into`
701−kiss. This tells clay that the duct over which the kiss was sent is the
702−duct that unix is listening on for changes. From within Arvo, though, we
703−should never send a `%into` kiss. The `%info` kiss is exactly identical
704−except it does not reset the duct.
705−
706−```hoon
707− [%ingo p=@p q=@tas r=nori] :: internal noun edit
708−
709− [%invo p=@p q=@tas r=nori] :: external noun edit
710−```
711−
712−These kisses are currently identical to `%info` and `%into`, though this
713−will not always be the case. The intent is for these kisses to allow
714−typed changes to clay so that we may store typed data. This is currently
715−unimplemented.
716−
717−```hoon
718− [%init p=@p] :: report install
719−```
720−
721−Init is called when a ship is started on our pier. This simply creates a
722−default `room` to go into our `raft`. Essentially, this initializes the
723−filesystem for a ship.
724−
725−```hoon
726− [%merg p=@p q=@tas r=mizu] :: internal change
727−```
728−
729−This is called to perform a merge. This is most visibly called by
730−:update to update the filesystem of the current ship to that of its
731−sein. The `p` and `q` are as in `%info`, and the `r` is the description
732−of the merge. See `++mizu:clay` above.
733−
734−XX
735−`XX [%wake ~] :: timer activate XX`
736−XX\
737−XX This card is sent by unix at the time specified by `++doze`. This
738−time is XX usually the closest time specified in a subscription request.
739−When `%wake` is XX called, we update our subscribers if there have been
740−any changes.
741−
742−```hoon
743− [%wart p=sock q=@tas r=path s=*] :: network request
744−```
745−
746−This is a request that has come across the network for a particular
747−file. When another ship asks for a file from us, that request comes to
748−us in the form of a `%wart` kiss. This is handled by trivially turning
749−it into a `%warp`.
750−
751−```hoon
752− [%warp p=sock q=riff] :: file request
753−```
754−
755−This is a request for information about a particular desk. This is, in
756−its most general form, a subscription, though in many cases it is the
757−trivial case of a subscription -- a read. See `++riff:clay` for the format of
758−the request.
759−
760−