@@ -1,79 +1,82 @@
1−+++
1+++
22title = "Architecture"
33weight = 2
44template = "doc.html"
5−+++
5+++
66
7−Clay is the primary filesystem for the arvo operating system,
8−which is the [core](/docs/glossary/core/) of an urbit. The architecture of clay is
9−intrinsically connected with arvo, but we assume no knowledge of
10−either arvo or urbit. We will point out only those features of
11−arvo that are necessary for an understanding of clay, and we will
7+Clay is the primary filesystem for the [Arvo](@/docs/Arvo/overview.md) operating system,
8+which is the [core](/docs/glossary/core/) of an urbit. The architecture of Clay is
9+intrinsically connected with Arvo, but for this section we assume no knowledge of
10+either Arvo or Urbit. We will point out only those features of
11+Arvo that are necessary for an understanding of Clay, and we will
1212do so only when they arise.
1313
14−The first relevant feature of arvo is that it is a deterministic
14+The first relevant feature of Arvo is that it is a deterministic
1515system where input and output are defined as a series of events
16−and effects. The state of arvo is simply a function of its event
16+and effects. The state of Arvo is simply a pure function of its event
1717log. None of the effects from an event are emitted until the
1818event is entered in the log and persisted, either to disk or
1919another trusted source of persistence, such as a Kafka cluster.
20−Consequently, arvo is a single-level store: everything in its
20+Consequently, Arvo is a single-level store: everything in its
2121state is persistent.
2222
2323In a more traditional OS, everything in RAM can be erased at any
2424time by power failure, and is always erased on reboot. Thus, a
2525primary purpose of a filesystem is to ensure files persist across
26−power failures and reboots. In arvo, both power failures and
26+power failures and reboots. In Arvo, both power failures and
2727reboots are special cases of suspending computation, which is
2828done safely since our event log is already persistent. Therefore,
29−clay is not needed in arvo for persistence. Why, then, do we have a
29+Clay is not needed in Arvo for persistence. Why, then, do we have a
3030filesystem? There are two answers to this question.
3131
32−First, clay provides a filesystem tree, which is a convenient
32+First, Clay provides a filesystem tree, which is a convenient
3333user interface for some applications. Unix has the useful concept
3434of virtual filesystems, which are used for everything from direct
3535access to devices, to random number generators, to the /proc
3636tree. It is easy and intuitive to read from and write to a
3737filesystem tree.
3838
39−Second, clay has a distributed revision-control system baked into
39+Second, Clay has a distributed revision-control system baked into
4040it. Traditional filesystems are not revision controlled, so
4141userspace software -- such as git -- is written on top of them to
42−do so. clay natively provides the same functionality as modern
42+do so. Clay natively provides the same functionality as modern
4343DVCSes, and more.
4444
45−clay has two other unique properties that we'll cover later on:
45+Clay has two other unique properties that we'll cover later on:
4646it supports typed data and is referentially transparent.
4747
4848### Revision Control
4949
50−Every urbit has one or more "desks", which are independently
51−revision-controlled branches. Each desk contains its own mark
52−definitions, apps, doc, and so forth.
50+Every urbit has one or more `desk`s, which are independently
51+revision-controlled branches. Each `desk` contains its own `mark`
52+definitions, apps, and so forth.
5353
54−Traditionally, an urbit has at least a base and a home desk. The
55−base desk has all the system software from the distribution. the
56−home desk is a fork of base with all the stuff specific to the
57−user of the urbit.
54+Traditionally, an urbit has at least a `%home` `desk` and `%kids` `desk`. The
55+`%home` `desk` has all the system software from the distribution, along with your
56+personal files. The `%home` `desk` is a fork of the `%kids` desk of whichever ship
57+you download [OTAs](@/docs/glossary/ota-updates.md) from - typically your sponsor,
58+but theoretically may be any ship. You will also find references to a `%base`
59+`desk` - which is not a real `desk`, rather it just a hash associated to the
60+`desk` from which your `%home` `desk` was forked.
5861
59−A desk is a series of numbered commits, the most recent of which
60−represents the current state of the desk. A commit is composed of
62+A `desk` is a series of numbered commits, the most recent of which
63+represents the current state of the `desk`. A commit is composed of
6164(1) an absolute time when it was created, (2) a list of zero or
6265more parents, and (3) a map from paths to data.
6366
6467Most commits have exactly one parent, but the initial commit on a
65−desk may have zero parents, and merge commits have more than one
68+`desk` may have zero parents, and merge commits have more than one
6669parent.
6770
68−The non-meta data is stored in the map of paths to data. It's
69−worth noting that no constraints are put on this map, so, for
70−example, both /a/b and /a/b/c could have data. This is impossible
71−in a traditional Unix filesystem since it means that /a/b is both
71+The non-metadata is stored as a `map` of `path`s to data. It's
72+worth noting that no constraints are put on this `map`, so, for
73+example, both `/a/b` and `/a/b/c` could have data. This is impossible
74+in a traditional Unix filesystem since it means that `/a/b` is both
7275a file and a directory. Conventionally, the final element in the
73−path is its mark -- much like a filename extension in Unix. Thus,
74−/doc/readme.md in Unix is stored as /doc/readme/md in urbit.
76+path is its `mark` -- much like a filename extension in Unix. Thus,
77+`/doc/readme.md` in Unix is stored as `/doc/readme/md` in urbit.
7578
76−The data is not stored directly in the map; rather, a hash of the
79+The data is not stored directly in the `map`; rather, a hash of the
7780data is stored, and we maintain a master blob store. Thus, if the
7881same data is referred to in multiple commits (as, for example,
7982when a file doesn't change between commits), only the hash is
@@ -85,8 +88,8 @@ only on the data within and not on whether or not it's stored
8588directly, so we may on occasion rearrange the contents of the
8689blob store for performance reasons.
8790
88−Recall that a desk is a series of numbered commits. Not every
89−commit in a desk must be numbered. For example, if the base desk
91+Recall that a `desk` is a series of numbered commits. Not every
92+commit in a `desk` must be numbered. For example, if the base `desk`
9093has had 50 commits since home was forked from it, then a merge
9194from base to home will only add a single revision number to home,
9295although the full commit history will be accessible by traversing
@@ -101,20 +104,20 @@ There are three ways to refer to particular commits in the
101104revision history. Firstly, one can use the revision number.
102105Secondly, one can use any absolute time between the one numbered
103106commit and the next (inclusive of the first, exclusive of the
104−second). Thirdly, every desk has a map of labels to revision
107+second). Thirdly, every `desk` has a `map` of labels to revision
105108numbers. These labels may be used to refer to specific commits.
106109
107−Additionally, clay is a global filesystem, so data on other urbit
110+Additionally, Clay is a global filesystem, so data on other urbits
108111is easily accessible the same way as data on our local urbit. In
109−general, the path to a particular revision of a desk is
110−/~urbit-name/desk-name/revision. Thus, to get /try/readme/md
111−from revision 5 of the home desk on ~sampel-sipnym, we refer to
112−/~sampel-sipnym/home/5/try/readme/md. Clay's namespace is thus
112+general, the path to a particular revision of a `desk` is
113+`/~urbit-name/desk-name/revision`. Thus, to get `/try/readme/md`
114+from revision 5 of the home `desk` on `~sampel-sipnym`, we refer to
115+`/~sampel-sipnym/home/5/try/readme/md`. Clay's namespace is thus
113116global and referentially transparent.
114117
115118### A Typed Filesystem
116119
117−Since clay is a general filesystem for storing data of arbitrary
120+Since Clay is a general filesystem for storing data of arbitrary
118121types, in order to revision control correctly it needs to be
119122aware of types all the way through. Traditional revision control
120123does an excellent job of handling source code, so for source code
@@ -164,33 +167,34 @@ a change. Two diffs consisting of a single added word to
164167different sentences may be flagged as a conflict. In general,
165168prose should be diffed by sentence, not by line.
166169
167−As far as the author is aware, clay is the first generalized,
170+As far as we are aware, Clay is the first generalized,
168171type-aware revision control system. We'll go into the workings
169172of this system in some detail.
170173
171174### Marks
172175
173−Central to a typed filesystem is the idea of types. In clay, we
174−call these "marks". A mark is a file that defines a type,
175−conversion routines to and from the mark, and diff, patch, and
176+Central to a typed filesystem is the idea of types. In Clay, we
177+call these `mark`s. A `mark` is a file that defines a type,
178+conversion routines to and from the `mark`, and diff, patch, and
176179merge routines.
177180
178181For example, a `%txt` mark may be a list of lines of text, and it
179182may include conversions to `%mime` to allow it to be serialized
180−and sent to a browswer or to the unix filesystem. It will also
183+and sent to a browser or to the Unix filesystem. It will also
181184include Hunt-McIlroy diff, patch, and merge algorithms.
182185
183−A `%json` mark would be defined as a json object in the code, and
186+A `%json` `mark` would be defined as a JSON object in the code, and
184187it would have a parser to convert from `%txt` and a printer to
185188convert back to `%txt`. The diff, patch, and merge algorithms are
186−fairly straightforward for json, though they're very different
189+fairly straightforward for JSON, though they're very different
187190from the text ones.
188191
189−More formally, a mark is a core with three [arms](/docs/glossary/arm/), `++grab`,
190−`++grow`, and `++grad`. In `++grab` is a series of functions to
191−convert from other marks to the given mark. In `++grow` is a
192−series of functions to convert from the given mark to other
193−marks. In `++grad` is `++diff`, `++pact`, `++join`, and `++mash`.
192+More formally, a `mark` is a core with three [arms](/docs/glossary/arm/): `+grab`,
193+`+grow`, and `+grad`. In `+grab` is a series of functions to
194+convert from other `mark`s to the given `mark`. In `+grow` is a
195+series of functions to convert from the given `mark` to other
196+`mark`s. In `+grad` is `+diff`, `+pact`, `+join`, and `+mash`, and
197+`+form`.
194198
195199The types are as follows, in an informal pseudocode:
196200
@@ -204,47 +208,47 @@ The types are as follows, in an informal pseudocode:
204208 ++ txt: <mark-type> -> <txt>
205209 ...
206210 ++ grad
211+ ++ form: <mark-type>
207212 ++ diff: (<mark-type>, <mark-type>) -> <diff-type>
208213 ++ pact: (<mark-type>, <diff-type>) -> <mark-type>
209214 ++ join: (<diff-type>, <diff-type>) -> <diff-type> or NULL
210215 ++ mash: (<diff-type>, <diff-type>) -> <diff-type>
211216```
212217
213−These types are basically what you would expect. Not every mark
218+These types are basically what you would expect. Not every `mark`
214219has each of these functions defined -- all of them are optional
215220in the general case.
216221
217−In general, for a particular mark, the `++grab` and `++grow` entries
222+In general, for a particular `mark`, the `+grab` and `+grow` entries
218223(if they exist) should be inverses of each other.
219224
220−In `++grad`, `++diff` takes two instances of a mark and produces
221−a diff of them. `++pact` takes an instance of a mark and patches
222−it with the given diff. `++join` takes two diffs and attempts to
223−merge them into a single diff. If there are conflicts, it
224−produces null. `++mash` takes two diffs and forces a merge,
225−annotating any conflicts.
225+In `+grad`, `+diff` takes two instances of a `mark` and produces a diff of them
226+whose `mark` is given by `+form`. `+pact` takes an instance of a `mark` and
227+patches it with the given diff. `+join` takes two diffs and attempts to merge
228+them into a single diff. If there are conflicts, it produces null. `+mash` takes
229+two diffs and forces a merge, annotating any conflicts.
226230
227−In general, if `++diff` called with A and B produces diff D, then
228−`++pact` called with A and D should produce B. Also, if `++join`
229−of two diffs does not produce null, then `++mash` of the same
231+In general, if `+diff` called with A and B produces diff D, then
232+`+pact` called with A and D should produce B. Also, if `+join`
233+of two diffs does not produce null, then `+mash` of the same
230234diffs should produce the same result.
231235
232−Alternately, instead of `++diff`, `++pact`, `++join`, and
233−`++mash`, a mark can provide the same functionality by defining
234−`++sted` to be the name of another mark to which we wish to
236+Alternately, instead of `+diff`, `+pact`, `+join`, and
237+`+mash`, a `mark` can provide the same functionality by defining
238+`+grad` to be the name of another `mark` to which we wish to
235239delegate the revision control responsibilities. Then, before
236−running any of those functions, clay will convert to the other
237−mark, and convert back afterward. For example, the `%hoon` mark
238−is revision-controlled in the same way as `%txt`, so its `++grad`
239−is simply `++sted %txt`. Of course, `++txt` must be defined in
240−`++grow` and `++grab` as well.
241−
242−Every file in clay has a mark, and that mark must have a
243−fully-functioning `++grad`. Marks are used for more than just
244−clay, and other marks don't need a `++grad`, but if a piece of
245−data is to be saved to clay, we must know how to revision-control
240+running any of those functions, Clay will convert to the other
241+`mark`, and convert back afterward. For example, the `%hoon` `mark`
242+is revision-controlled in the same way as `%txt`, so its `+grad`
243+is simply `++ grad %txt`. Of course, `+txt` must be defined in
244+`+grow` and `+grab` as well.
245+
246+Every file in Clay has a `mark`, and that `mark` must have a
247+fully-functioning `+grad`. `Mark`s are used for more than just
248+Clay, and other `mark`s don't need a `+grad`, but if a piece of
249+data is to be saved to Clay, we must know how to revision-control
246250it.
247251
248−Additionally, if a file is to be synced out to unix, then it must
249−have conversion routines to and from the `%mime` mark.
252+Additionally, if a file is to be synced out to Unix, then it must
253+have conversion routines to and from the `%mime` `mark`.
250254