on master help sign in
Run pandoc recursively over docs
105bd25Keaton Dunsford
docs.md+55−34
@@ -1,67 +1,88 @@
1---
1---
2anchor: none
2anchor: none
3−footer: true
3+footer: True
4−title: Docs
5sort: 4
4sort: 4
5+title: Docs
6---
6---
7
7
8<div class="lead">
8<div class="lead">
9 <div class="logo inverse"></div>
9 <div class="logo inverse"></div>
10 <h1 class="blue i-b">Urbit docs</h1>
10 <h1 class="blue i-b">Urbit docs</h1>
11−</div>
12
11
12+</div>
13<div class="col-md-12 head">
13<div class="col-md-12 head">
14−
15<div class="col-md-10">
14<div class="col-md-10">
15+**Urbit** is a secure peer-to-peer network of personal servers, built on a
16+clean-slate system software stack.
16
17
17−**Urbit** is a secure peer-to-peer network of personal servers, built on a clean-slate system software stack.
18+Check out the [technical overview](about/overview) for a high-level architecture
19+description. To install and run Urbit, head to the Using section, starting with
20+the [installation guide](using/install) and continuing forward.
18
21
19−Check out the [technical overview](about/overview) for a high-level
22+To begin developing for Urbit, start by walking through
20−architecture description. To install and run Urbit, head to the Using section, starting with the [installation guide](using/install) and continuing forward.
23+[Urbytes](https://urbit.org/docs/byte/0). After that, delve into Hoon
24+[syntax](hoon/syntax) directly, then walk through the [Arvo tutorials](arvo).
25+The examples and exercises in Arvo will teach you to build high-level programs
26+and interact with the network.
21
27
22−To begin developing for Urbit, start by walking through [Urbytes](https://urbit.org/docs/byte/0). After that, delve into Hoon [syntax](hoon/syntax) directly, then walk through the [Arvo tutorials](arvo). The examples and exercises in Arvo will teach you to build high-level programs and interact with the network.
28+Dig into the mechanics of Urbit with [Core Hoon](hoon). Refer back there for
29+pure-code exercises and functions in the [standard library](hoon/library).
23
30
24−Dig into the mechanics of Urbit with [Core Hoon](hoon). Refer back
25−there for pure-code exercises and functions in the [standard library](hoon/library).
26−
27−Anyone obsessed with math, theory, fundamentals of computation,
31+Anyone obsessed with math, theory, fundamentals of computation, etc, should
28−etc, should start with the [Nock definition](nock/definition) and work upward from there.
32+start with the [Nock definition](nock/definition) and work upward from there.
29
33
30</div>
34</div>
31−
32</div>
35</div>
33−
34<div class="col-md-12 head">
36<div class="col-md-12 head">
35−
36<div class="col-md-4">
37<div class="col-md-4">
37−
38<b>Community & contact</b>
38<b>Community & contact</b>
39<ul>
39<ul>
40+<li>
40−<li>Chat with us on <a href="https://urbit.org/stream"><code>:talk</code></a></li>
41+Chat with us on <a href="https://urbit.org/stream"><code>:talk</code></a>
42+</li>
43+<li>
41−<li>Our discussion forum <a href="http://urbit.org/fora"><code>fora</code></a></li>
44+Our discussion forum <a href="http://urbit.org/fora"><code>fora</code></a>
45+</li>
46+<li>
42−<li>On Twitter <a href="https://twitter.com/urbit">@urbit</a></li>
47+On Twitter <a href="https://twitter.com/urbit">@urbit</a>
48+</li>
49+<li>
43−<li>Or email us at <a href="mailto:urbit@urbit.org">urbit@urbit.org</a></li>
50+Or email us at <a href="mailto:urbit@urbit.org">urbit@urbit.org</a>
51+</li>
44</ul>
52</ul>
45−
46</div>
53</div>
47−
48<div class="col-md-4">
54<div class="col-md-4">
49−
50<b>Source code on <a href="https://github.com/urbit">GitHub</a></b>
55<b>Source code on <a href="https://github.com/urbit">GitHub</a></b>
51<ul>
56<ul>
57+<li>
52−<li><a href="https://github.com/urbit/urbit"><code>vere</code></a> - Urbit's VM</li>
58+<a href="https://github.com/urbit/urbit"><code>vere</code></a> - Urbit's VM
59+</li>
60+<li>
53−<li><a href="https://github.com/urbit/arvo"><code>arvo</code></a> - Urbit's OS</li>
61+<a href="https://github.com/urbit/arvo"><code>arvo</code></a> - Urbit's OS
62+</li>
63+<li>
54−<li><a href="https://github.com/urbit/tree"><code>tree</code></a> - Urbit's web publishing UI</li>
64+<a href="https://github.com/urbit/tree"><code>tree</code></a> - Urbit's web
65+publishing UI
66+</li>
67+<li>
55−<li><a href="https://github.com/urbit/talk"><code>talk</code></a> - Urbit's web chat</li>
68+<a href="https://github.com/urbit/talk"><code>talk</code></a> - Urbit's web chat
69+</li>
70+<li>
56−<li><a href="https://github.com/urbit/sole"><code>sole</code></a> - Urbit's web console</li>
71+<a href="https://github.com/urbit/sole"><code>sole</code></a> - Urbit's web
72+console
73+</li>
74+<li>
57−<li><a href="https://github.com/urbit/examples"><code>examples</code></a> - Hoon, Arvo and Tree</li>
75+<a href="https://github.com/urbit/examples"><code>examples</code></a> - Hoon,
76+Arvo and Tree
77+</li>
78+<li>
58−<li><a href="https://github.com/urbit/docs"><code>docs</code></a> - The repo for these docs</li>
79+<a href="https://github.com/urbit/docs"><code>docs</code></a> - The repo for
80+these docs
81+</li>
59</ul>
82</ul>
60−
61</div>
83</div>
62−
63</div>
84</div>
64−
65<div class="sections">
85<div class="sections">
66 <kids grid="true"></kids>
86 <kids grid="true"></kids>
87+
67</div>
88</div>
docs/about.md+1−1
@@ -1,5 +1,5 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 3
3sort: 3
4---
4---
5
5
docs/about/contributing.md+15−14
@@ -7,24 +7,25 @@ title: Contributing to Urbit
7
7
8# Contributing to Urbit
8# Contributing to Urbit
9
9
10−It's always a good day for pull requests! We gladly accept submissions
10+It's always a good day for pull requests! We gladly accept submissions for
11−for improving our [code](https://github.com/urbit/urbit/) and
11+improving our [code](https://github.com/urbit/urbit/) and
12[documentation](https://github.com/urbit/docs/) alike.
12[documentation](https://github.com/urbit/docs/) alike.
13
13
14## Code guidelines
14## Code guidelines
15
15
16Our official guide to contributing can be found
16Our official guide to contributing can be found
17−[here](https://github.com/urbit/urbit/blob/master/CONTRIBUTING.md),
17+[here](https://github.com/urbit/urbit/blob/master/CONTRIBUTING.md), which is
18−which is most useful for large and low-level changes. For simpler
18+most useful for large and low-level changes. For simpler revisions, refer to
19−revisions, refer to [Hoon Syntax](../../hoon/syntax) for an in-depth guide to
19+[Hoon Syntax](../../hoon/syntax) for an in-depth guide to good Hoon.
20−good Hoon. Alternatively, [browse the
20+Alternatively, [browse the
21source](https://github.com/urbit/arvo/blob/master/arvo/hoon.hoon).
21source](https://github.com/urbit/arvo/blob/master/arvo/hoon.hoon).
22
22
23## Docs guidelines
23## Docs guidelines
24
24
25For documentation pull requests, if you are making a brand new page, you'll want
25For documentation pull requests, if you are making a brand new page, you'll want
26−it to conform to our general documentation standards. Acquire [pandoc](http://pandoc.org)
26+it to conform to our general documentation standards. Acquire
27−and make sure to format your edited files with the following flags:
27+[pandoc](http://pandoc.org) and make sure to format your edited files with the
28+following flags:
28
29
29 $ pandoc --standalone \
30 $ pandoc --standalone \
30 --from=markdown+yaml_metadata_block \
31 --from=markdown+yaml_metadata_block \
@@ -33,13 +34,13 @@ and make sure to format your edited files with the following flags:
33 input.md
34 input.md
34
35
35If you are editing an existing page, reformatting the entire page with pandoc
36If you are editing an existing page, reformatting the entire page with pandoc
36−will make the diff hard to read. Instead, just make sure that your edits generally
37+will make the diff hard to read. Instead, just make sure that your edits
37−conform to the style of the existing docs around it. For example, you'll notice
38+generally conform to the style of the existing docs around it. For example,
38−that the paragraphs mostly wrap at column 80.
39+you'll notice that the paragraphs mostly wrap at column 80.
39
40
40−**Note:** If you have an old version of Pandoc, it may convert the
41+**Note:** If you have an old version of Pandoc, it may convert the terminating
41−terminating `---` in YAML metadata to `...`, which is grounds for termination.
42+`---` in YAML metadata to `...`, which is grounds for termination. Run the
42−Run the following one-liner to atone.
43+following one-liner to atone.
43
44
44 $ sed '1,/^$/ s/^\.\.\./---/g' input.md > output.md
45 $ sed '1,/^$/ s/^\.\.\./---/g' input.md > output.md
45
46
docs/about/glossary.md+93−109
@@ -7,17 +7,17 @@ title: Urbit glossary
7
7
8# Urbit Glossary
8# Urbit Glossary
9
9
10−Urbit is renowned for its exotic terminology. Here's a simple overview
10+Urbit is renowned for its exotic terminology. Here's a simple overview from the
11−from the strange words in.
11+strange words in.
12
12
13−As Dijkstra put it: "The purpose of abstraction is not to be vague, but
13+As Dijkstra put it: "The purpose of abstraction is not to be vague, but to
14−to create a new semantic level in which one can be absolutely precise."
14+create a new semantic level in which one can be absolutely precise."
15
15
16## Ships
16## Ships
17
17
18−An Urbit *ship* is a cryptographic title on a *will* signed by private
18+An Urbit *ship* is a cryptographic title on a *will* signed by private key, a
19−key, a human-memorable name, and a packet routing address. Ships are
19+human-memorable name, and a packet routing address. Ships are classed by the
20−classed by the number of bits in their address:
20+number of bits in their address:
21
21
22 Size Name Parent Object Example
22 Size Name Parent Object Example
23 ----- ------ ------ ------ -------
23 ----- ------ ------ ------ -------
@@ -31,31 +31,29 @@ Any ship can be called an "urbit."
31
31
32> A summary [here](http://urbit.org/posts/address-space/).
32> A summary [here](http://urbit.org/posts/address-space/).
33
33
34−An Urbit identity is a string like `~firbyr-napbes`. It means nothing,
34+An Urbit identity is a string like `~firbyr-napbes`. It means nothing, but it's
35−but it's easy to remember and say out loud. `~firbyr-napbes` is actually
35+easy to remember and say out loud. `~firbyr-napbes` is actually just a 32-bit
36−just a 32-bit number, like an IP address, that we turn into a
36+number, like an IP address, that we turn into a human-memorable string.
37−human-memorable string.
38
37
39−Technically, an urbit is a secure digital identity that you own and
38+Technically, an urbit is a secure digital identity that you own and control with
40−control with a cryptographic key, like a Bitcoin wallet. As in Bitcoin,
39+a cryptographic key, like a Bitcoin wallet. As in Bitcoin, the supply of urbits
41−the supply of urbits is mathematically limited. This keeps the network
40+is mathematically limited. This keeps the network friendly, by making spam and
42−friendly, by making spam and abuse expensive.
41+abuse expensive.
43
42
44−An urbit name is just a number; smaller numbers make shorter names.
43+An urbit name is just a number; smaller numbers make shorter names. Shorter
45−Shorter names are easier to remember, so they're more valuable. So
44+names are easier to remember, so they're more valuable. So urbits are classified
46−urbits are classified by the number of bits in their name. (A ship name
45+by the number of bits in their name. (A ship name is just a scrambled base-256
47−is just a scrambled base-256 representation of the number.)
46+representation of the number.)
48
47
49−A 32-bit urbit (like `~firbyr-napbes`) is called a *planet.* A 16-bit
48+A 32-bit urbit (like `~firbyr-napbes`) is called a *planet.* A 16-bit urbit
50−urbit (like `~pollev)` is a *star.* An 8-bit ship (like `~mun`) is a
49+(like `~pollev)` is a *star.* An 8-bit ship (like `~mun`) is a *galaxy.* A
51−*galaxy.* A planet is an identity for an independent, adult human. Stars
50+planet is an identity for an independent, adult human. Stars and galaxies are
52−and galaxies are network infrastructure.
51+network infrastructure.
53
52
54−Each planet or star is launched by its *parent,* the star or galaxy
53+Each planet or star is launched by its *parent,* the star or galaxy whose number
55−whose number is its bottom half. So the planet `~firbyr-napbes`,
54+is its bottom half. So the planet `~firbyr-napbes`, `0xdead.beef` or
56−`0xdead.beef` or `3.735.928.559`, is the child of `~pollev`, `0xbeef` or
55+`3.735.928.559`, is the child of `~pollev`, `0xbeef` or `48.879`, whose parent
57−`48.879`, whose parent is `~mun`, `0xef`, `239`. The parent of `~mun`
58−and all galaxies is `~zod`, `0`.
56+is `~mun`, `0xef`, `239`. The parent of `~mun` and all galaxies is `~zod`, `0`.
59
57
60### Nock
58### Nock
61
59
@@ -64,32 +62,30 @@ and all galaxies is `~zod`, `0`.
64- *noun*: an *atom* or a *cell*
62- *noun*: an *atom* or a *cell*
65- *atom*: any natural number
63- *atom*: any natural number
66- *cell*: any ordered pair of nouns
64- *cell*: any ordered pair of nouns
67−- *subject*: a noun - the data against which a `formula` is
65+- *subject*: a noun - the data against which a `formula` is evaluated
68− evaluated
69- *formula*: a noun - a function at the Nock level
66- *formula*: a noun - a function at the Nock level
70−- *product*: a noun - the result of evaluating a formula against a
67+- *product*: a noun - the result of evaluating a formula against a subject
71− subject
72
68
73*See [Nock definition](../../nock/definition/).*
69*See [Nock definition](../../nock/definition/).*
74
70
75### Hoon
71### Hoon
76
72
77−> Hoon is a strict, higher-order typed functional language that compiles
73+> Hoon is a strict, higher-order typed functional language that compiles itself
78−> itself to Nock.
74+> to Nock.
79
75
80###### span: an inferred type
76###### span: an inferred type
81
77
82−A *span* defines a set (finite or infinite) of nouns and ascribes some
78+A *span* defines a set (finite or infinite) of nouns and ascribes some semantics
83−semantics to it. There is no Hoon syntax for a span; it is always
79+to it. There is no Hoon syntax for a span; it is always produced as the inferred
84−produced as the inferred range of an expression (*twig*).
80+range of an expression (*twig*).
85
81
86*See [basic types](../../hoon/basic/#-type-span-and-mold).*
82*See [basic types](../../hoon/basic/#-type-span-and-mold).*
87
83
88###### core: a code-data cell
84###### core: a code-data cell
89
85
90The code (*battery*) is the head, the data (*payload*) is the tail. All
86The code (*battery*) is the head, the data (*payload*) is the tail. All
91−code-data structures in normal languages (functions, objects, modules,
87+code-data structures in normal languages (functions, objects, modules, etc)
92−etc) become *core*s in Hoon.
88+become *core*s in Hoon.
93
89
94- *battery*: the code of a core, a tree of *arms*
90- *battery*: the code of a core, a tree of *arms*
95- *payload*: the data in a core
91- *payload*: the data in a core
@@ -98,29 +94,27 @@ etc) become *core*s in Hoon.
98
94
99###### arm: a named, functionally-computed attribute of a core
95###### arm: a named, functionally-computed attribute of a core
100
96
101−The *twig* of each arm is compiled to a Nock formula, with the
97+The *twig* of each arm is compiled to a Nock formula, with the enclosing `core`
102−enclosing `core` itself as the subject.
98+itself as the subject.
103
99
104There are two kinds of arms: *dry* and *wet*. Most arms are dry.
100There are two kinds of arms: *dry* and *wet*. Most arms are dry.
105
101
106- *dry*: normal, `++`, polymorphic by means of *variance*
102- *dry*: normal, `++`, polymorphic by means of *variance*
107
103
108−For a dry arm, we ask, is the new payload compatible with the old
104+For a dry arm, we ask, is the new payload compatible with the old payload
109−payload (against which the core was compiled)?
105+(against which the core was compiled)?
110
106
111- *wet*: unusual, `+-`, polymorphic by means of *genericity*
107- *wet*: unusual, `+-`, polymorphic by means of *genericity*
112
108
113−A wet arm uses the twig as a macro. We create a new type analysis
109+A wet arm uses the twig as a macro. We create a new type analysis path, which
114−path, which works as if we expanded the callee with the caller's
110+works as if we expanded the callee with the caller's context.
115−context.
116
111
117*See [advanced types](/hoon/advanced)*.
112*See [advanced types](/hoon/advanced)*.
118
113
119###### metal: a variance model
114###### metal: a variance model
120
115
121−Every core has a *metal* which defines its variance model (ie, the
116+Every core has a *metal* which defines its variance model (ie, the properties of
122−properties of the span of a compatible core). The default is `gold`
117+the span of a compatible core). The default is `gold` (invariant).
123−(invariant).
124
118
125- `gold`: *invariant*
119- `gold`: *invariant*
126- `lead`: *bivariant*
120- `lead`: *bivariant*
@@ -131,9 +125,8 @@ properties of the span of a compatible core). The default is `gold`
131
125
132###### gate: a function/lambda/closure
126###### gate: a function/lambda/closure
133
127
134−A *gate* is a core with one arm. To call a gate on an argument,
128+A *gate* is a core with one arm. To call a gate on an argument, replace the
135−replace the sample (at [tree
136−address](../../hoon/twig/limb/limb/) `+6` in the core) with the
129+sample (at [tree address](../../hoon/twig/limb/limb/) `+6` in the core) with the
137argument, and then compute the arm.
130argument, and then compute the arm.
138
131
139The payload of a gate has a shape of `{sample context}`.
132The payload of a gate has a shape of `{sample context}`.
@@ -141,51 +134,46 @@ The payload of a gate has a shape of `{sample context}`.
141- *sample*: the argument tuple
134- *sample*: the argument tuple
142- *context*: the subject in which the gate was defined
135- *context*: the subject in which the gate was defined
143
136
144−*See [basic types](../../hoon/basic/#-core-p-span-q-map-term-span),
137+*See [basic types](../../hoon/basic/#-core-p-span-q-map-term-span), [`%-` or
145−[`%-` or `:call`](../../hoon/twig/cen-call/hep-call/) (the `twig` for
138+`:call`](../../hoon/twig/cen-call/hep-call/) (the `twig` for calling a `gate`).*
146−calling a `gate`).*
147
139
148###### mold: a type constructor / validator
140###### mold: a type constructor / validator
149
141
150−A *mold* is an idempotent gate (function), accepting any noun, and
142+A *mold* is an idempotent gate (function), accepting any noun, and producing a
151−producing a range with a useful span.
143+range with a useful span.
152
144
153−- *bunt*: the value a mold produces when normalizing its default
145+- *bunt*: the value a mold produces when normalizing its default sample
154− sample
155- *icon*: the span of the mold's range
146- *icon*: the span of the mold's range
156
147
157*See [mold twigs](/hoon/twig/buc-mold/).*
148*See [mold twigs](/hoon/twig/buc-mold/).*
158
149
159###### atom
150###### atom
160
151
161−A Hoon *atom* span describes a Nock atom, with two additional pieces
152+A Hoon *atom* span describes a Nock atom, with two additional pieces of
162−of metadata: an *aura*, and an optional constant.
153+metadata: an *aura*, and an optional constant.
163
154
164An atom span is *warm* or *cold* based on whether the constant exists.
155An atom span is *warm* or *cold* based on whether the constant exists.
165
156
166- *warm*: if the constant is `~` (null), any atom is in the span
157- *warm*: if the constant is `~` (null), any atom is in the span
167−- *cold*: if the constant is `[~ atom]`, its only legal value is
158+- *cold*: if the constant is `[~ atom]`, its only legal value is `atom`.
168− `atom`.
169
159
170*See [basic types](../../hoon/basic/#-atom-p-term-q-unit-atom)*
160*See [basic types](../../hoon/basic/#-atom-p-term-q-unit-atom)*
171
161
172###### aura: a soft atom type
162###### aura: a soft atom type
173
163
174−*aura* is a name for an atom type. It represents the structure of an
164+*aura* is a name for an atom type. It represents the structure of an atom in a
175−atom in a string beginning with `@`. An aura may represent units,
165+string beginning with `@`. An aura may represent units, print format, or other
176−print format, or other semantics. Its constraints on the value of an
166+semantics. Its constraints on the value of an atom aren't enforced in any way.
177−atom aren't enforced in any way.
178
167
179−Two auras are compatible if one is a prefix of the other. You can
168+Two auras are compatible if one is a prefix of the other. You can change any
180−change any aura into any other aura by casting through the empty
169+aura into any other aura by casting through the empty aura, `@`. The standard
181−aura, `@`. The standard library has molds which alias many common
170+library has molds which alias many common auras.
182−auras.
183
171
184Some common auras and their aliases:
172Some common auras and their aliases:
185
173
186−- *term* (`@tas`): a symbol - an atomic ASCII string which obeys
174+- *term* (`@tas`): a symbol - an atomic ASCII string which obeys symbol rules:
187− symbol rules: lowercase and digit only, infix hyphen ("kebab-case"),
175+ lowercase and digit only, infix hyphen ("kebab-case"), first character must
188− first character must be lowercase alphabetic.
176+ be lowercase alphabetic.
189- *cord* (`@t`): UTF-8 text, least-significant-byte first
177- *cord* (`@t`): UTF-8 text, least-significant-byte first
190- *char* (`@tD`): a character, a single unicode byte (for multi-byte
178- *char* (`@tD`): a character, a single unicode byte (for multi-byte
191 characters and codepoints, see `@c`)
179 characters and codepoints, see `@c`)
@@ -194,70 +182,66 @@ Some common auras and their aliases:
194
182
195###### twig: a Hoon expression
183###### twig: a Hoon expression
196
184
197−A *twig* is a Hoon expression. Specifically, twig is the mold for the
185+A *twig* is a Hoon expression. Specifically, twig is the mold for the noun that
198−noun that a Hoon source expression compiles to. A twig is always a cell.
186+a Hoon source expression compiles to. A twig is always a cell.
199
187
200−A twig has one of two forms: `{twig twig}`, which produces the ordered
188+A twig has one of two forms: `{twig twig}`, which produces the ordered pair of
201−pair of the two twigs, or a cell `{stem bulb}`, where `stem` is an
189+the two twigs, or a cell `{stem bulb}`, where `stem` is an atomic symbol.
202−atomic symbol.
203
190
204- *stem*: an atomic symbol (`@tas`) - the name of a twig.
191- *stem*: an atomic symbol (`@tas`) - the name of a twig.
205- *bulb*: The mold of the twig's contents.
192- *bulb*: The mold of the twig's contents.
206
193
207−Most twigs have a *regular form*, beginning with a *sigil* which is
194+Most twigs have a *regular form*, beginning with a *sigil* which is either a
208−either a *keyword* or a *rune*. Some twigs also have a syntactic
195+*keyword* or a *rune*. Some twigs also have a syntactic *irregular form*; a few
209−*irregular form*; a few have *only* an *irregular form*.
196+have *only* an *irregular form*.
210
197
211- *sigil*: a *keyword* or *rune* used to begin a twig
198- *sigil*: a *keyword* or *rune* used to begin a twig
212- *keyword*: the value of a stem, prefixed with `:`.
199- *keyword*: the value of a stem, prefixed with `:`.
213- *rune*: a pair of ASCII symbols used to begin a twig.
200- *rune*: a pair of ASCII symbols used to begin a twig.
214
201
215For example, the stem `%if` becomes the keyword
202For example, the stem `%if` becomes the keyword
216−[`:if`](../../hoon/twig/wut-test/col-if/), with the rune `?:`. The first
203+[`:if`](../../hoon/twig/wut-test/col-if/), with the rune `?:`. The first symbol
217−symbol in a rune represents a family of related `twig`s. For example,
204+in a rune represents a family of related `twig`s. For example, the [`?`
218−the [`?` family](../../hoon/twig/wut-test/) are all conditionals.
205+family](../../hoon/twig/wut-test/) are all conditionals.
219
206
220−The keyword or rune forms are up to the programmer's choice. Most
207+The keyword or rune forms are up to the programmer's choice. Most existing code
221−existing code uses runes, but the keyword form makes the learning path
208+uses runes, but the keyword form makes the learning path easier.
222−easier.
223
209
224A regular twig has two syntactic forms, *tall* and *flat*:
210A regular twig has two syntactic forms, *tall* and *flat*:
225
211
226−- *tall*: multiple lines, no parentheses, two or more spaces between
212+- *tall*: multiple lines, no parentheses, two or more spaces between tokens
227− tokens
228- *flat*: one line, parentheses, one space between tokens
213- *flat*: one line, parentheses, one space between tokens
229
214
230−Tall twigs can contain flat ones, but not vice versa. All irregular
215+Tall twigs can contain flat ones, but not vice versa. All irregular forms are
231−forms are flat.
216+flat.
232
217
233*See [twig concept](../../hoon/concepts/#-twig-expression),
218*See [twig concept](../../hoon/concepts/#-twig-expression),
234[expressions](../../hoon/twig/), and [syntax](../../hoon/syntax/)*.
219[expressions](../../hoon/twig/), and [syntax](../../hoon/syntax/)*.
235
220
236###### limb: attribute or variable reference
221###### limb: attribute or variable reference
237
222
238−To resolve a *limb* named "foo", the subject is searched depth-first,
223+To resolve a *limb* named "foo", the subject is searched depth-first, head-first
239−head-first for either a face named "foo" or a core with an arm of
224+for either a face named "foo" or a core with an arm of "foo". If a face is
240−"foo". If a face is found, the result is a leg, if a core is
241−found, the result is the product of the arm.
225+found, the result is a leg, if a core is found, the result is the product of the
226+arm.
242
227
243- *leg*: a subexpression, or subtree of the subject.
228- *leg*: a subexpression, or subtree of the subject.
244−- *wing*: a list of limbs, searched from right to left (`a.b` means
229+- *wing*: a list of limbs, searched from right to left (`a.b` means `b` within
245− `b` within `a`).
230+ `a`).
246
231
247*See [Limbs and wings](../../hoon/twig/limb)*
232*See [Limbs and wings](../../hoon/twig/limb)*
248
233
249###### slot: a tree addressing scheme
234###### slot: a tree addressing scheme
250
235
251−Every cell has a head and a tail, each of which may be either an atom or
236+Every cell has a head and a tail, each of which may be either an atom or a cell.
252−a cell. Therefore every noun is a binary tree. `+1` or `.` resolves to
237+Therefore every noun is a binary tree. `+1` or `.` resolves to the whole cell
253−the whole cell (technically this would work against an atom as well).
238+(technically this would work against an atom as well). The head of `+n` is
254−The head of `+n` is `+2n`, the tail is `+(2n+1)`.
239+`+2n`, the tail is `+(2n+1)`.
255
240
256###### face: a labeled subtree
241###### face: a labeled subtree
257
242
258−Hoon has no scope or symbol-table; there is only the subject. To
243+Hoon has no scope or symbol-table; there is only the subject. To "declare" a
259−"declare" a "variable" is to construct a new subject:
244+"variable" is to construct a new subject: `[name=value old-subject]`.
260−`[name=value old-subject]`.
261
245
262A *face* is a span that wraps a name (`@tas`) around another span.
246A *face* is a span that wraps a name (`@tas`) around another span.
263
247
@@ -265,8 +249,8 @@ A *face* is a span that wraps a name (`@tas`) around another span.
265
249
266###### nest: type compatibility test
250###### nest: type compatibility test
267
251
268−*nest* is an internal function on two spans, which produces yes if the
252+*nest* is an internal function on two spans, which produces yes if the set of
269−set of nouns in the second span is provably a subset of the first.
253+nouns in the second span is provably a subset of the first.
270
254
271*See [advanced types](../../hoon/advanced/)*.
255*See [advanced types](../../hoon/advanced/)*.
272
256
@@ -274,5 +258,5 @@ set of nouns in the second span is provably a subset of the first.
274
258
275`0` (`%.y`) is *yes*, `1` (`%.n`) is *no*.
259`0` (`%.y`) is *yes*, `1` (`%.n`) is *no*.
276
260
277−> Why? It's fresh, it's different, it's new. And it's annoying. And it
261+> Why? It's fresh, it's different, it's new. And it's annoying. And it keeps you
278−> keeps you on your toes. And it's also just intuitively right.
262+> on your toes. And it's also just intuitively right.
docs/about/objections.md+453−170
@@ -7,262 +7,545 @@ title: Common objections to Urbit
7
7
8# Common objections to Urbit
8# Common objections to Urbit
9
9
10−"We choose to go to the moon, not because it is easy, but because it is hard." -- John F. Kennedy
10+"We choose to go to the moon, not because it is easy, but because it is hard."
11−
11+-- John F. Kennedy
12+
12−We're strong believers in crowdsourced criticism. We've heard a lot of great objections from all kinds of directions. Surely there's more out there -- if you think of a reason Urbit can't work, and it's not listed here, please add it in a comment at the bottom, post it somewhere and send us a link, etc, etc.
13+We're strong believers in crowdsourced criticism. We've heard a lot of great
13−
14+objections from all kinds of directions. Surely there's more out there -- if you
15+think of a reason Urbit can't work, and it's not listed here, please add it in a
16+comment at the bottom, post it somewhere and send us a link, etc, etc.
17+
14−## Urbit is a total scamcoin. It's 100% premined.
18+## Urbit is a total scamcoin. It's 100% premined.
15−
19+
16−Like a cryptocurrency, Urbit is a digital title. But Urbit is digital land, not digital money. If you assess it as a currency, it's a lousy currency. If you assess a cat as a dog, it's a lousy dog.
20+Like a cryptocurrency, Urbit is a digital title. But Urbit is digital land, not
17−
21+digital money. If you assess it as a currency, it's a lousy currency. If you
22+assess a cat as a dog, it's a lousy dog.
23+
18−In real-world economics, the crucial distinction between land and money is that land (a) has intrinsic utility and (b) is non-fungible.
24+In real-world economics, the crucial distinction between land and money is that
19−
25+land (a) has intrinsic utility and (b) is non-fungible.
26+
27+Because of (b) alone, frictionless transactions in land are impossible. A
20−Because of (b) alone, frictionless transactions in land are impossible. A blockchain is only needed to ensure zero-cost high-frequency transactions. So digital land doesn't need a blockchain. Which means it doesn't need mining.
28+blockchain is only needed to ensure zero-cost high-frequency transactions. So
21−
29+digital land doesn't need a blockchain. Which means it doesn't need mining.
30+
22−Of course, just because a digital title system doesn't need mining, doesn't mean it can't distribute an initial allocation by proof-of-work -- or by any other arbitrary method which is careful not to benefit the system's own developers. Could we give away the Urbit address space randomly? Sure. But why?
31+Of course, just because a digital title system doesn't need mining, doesn't mean
23−
32+it can't distribute an initial allocation by proof-of-work -- or by any other
33+arbitrary method which is careful not to benefit the system's own developers.
34+Could we give away the Urbit address space randomly? Sure. But why?
35+
36+Probably the purest design is the "proof-of-burn" initial allocation used by
24−Probably the purest design is the "proof-of-burn" initial allocation used by some coins. By any standard, proof-of-burn is technically beautiful and results in an extraordinary level of ethical independence. We can't help but admire it.
37+some coins. By any standard, proof-of-burn is technically beautiful and results
25−
38+in an extraordinary level of ethical independence. We can't help but admire it.
39+
26−But any altcoin or metacoin is just way less work than Urbit. An altcoin can afford to burn even its creation energy, capturing no value at all from its seigniorage. Your altcoin doesn't have a clean-slate OS to fund, and no one started building it in 2002.
40+But any altcoin or metacoin is just way less work than Urbit. An altcoin can
27−
41+afford to burn even its creation energy, capturing no value at all from its
42+seigniorage. Your altcoin doesn't have a clean-slate OS to fund, and no one
43+started building it in 2002.
44+
45+Ethically, what makes "premining" feel wrong is that the creators of the
46+currency are rewarded disproportionately for their effort. The scamcoin is a
28−Ethically, what makes "premining" feel wrong is that the creators of the currency are rewarded disproportionately for their effort. The scamcoin is a get-rich-quick scheme, vaguely pyramid-shaped. Nothing about Urbit matches this pattern at all.
47+get-rich-quick scheme, vaguely pyramid-shaped. Nothing about Urbit matches this
29−
48+pattern at all.
49+
30−To put it bluntly, Galen drives a 2004 Saab and Curtis drives a moldy '99 Cougar covered with raven shit. And we both love cars. We also have kids. And our spouses are more skeptical of Urbit than anyone else could ever be. (Our next startup definitely *will* be a get-rich-quick scheme, though.)
50+To put it bluntly, Galen drives a 2004 Saab and Curtis drives a moldy '99 Cougar
31−
51+covered with raven shit. And we both love cars. We also have kids. And our
32−Curtis created Urbit. So by the [homesteading principle](https://en.wikipedia.org/wiki/Homestead_principle), he started out owning all of it. But by now, Urbit's 8-bit galaxies have already been distributed among [57 separate individuals](). Curtis owns 4 -- or 24, counting his whole family. (Some of whom he still owes money.)
52+spouses are more skeptical of Urbit than anyone else could ever be. (Our next
33−
53+startup definitely *will* be a get-rich-quick scheme, though.)
54+
55+Curtis created Urbit. So by the [homesteading
56+principle](https://en.wikipedia.org/wiki/Homestead_principle), he started out
57+owning all of it. But by now, Urbit's 8-bit galaxies have already been
58+distributed among [57 separate individuals](). Curtis owns 4 -- or 24, counting
59+his whole family. (Some of whom he still owes money.)
60+
34−Tlon, the corporation which still runs Urbit development, has 95. (Without Tlon, there is basically no Arvo.) 50 are reserved for a hypothetical Urbit Foundation. Tlon employees have another 16.
61+Tlon, the corporation which still runs Urbit development, has 95. (Without Tlon,
35−
62+there is basically no Arvo.) 50 are reserved for a hypothetical Urbit
36−The rest of the galaxies belong to donors, investors, contributors, contest winners, and just random people who were in the right place at the right time. One is a Silicon Valley tycoon. Another is an Orthodox nun in a monastery in Greece. A couple are ex-Tlon employees who might totally hate us. There is nothing they all have in common, which is sort of the point.
63+Foundation. Tlon employees have another 16.
37−
64+
38−This property distribution looks a lot like land ownership in the real world -- complex and asymmetrical. That's no accident. If you have a problem living in New York because New York's real estate is not distributed equally among its residents, you may have a similar problem with Urbit. The reality is that Urbit, like New York, wasn't born yesterday and comes with a complex ownership history which we couldn't possibly change, even if we wanted to.
65+The rest of the galaxies belong to donors, investors, contributors, contest
39−
66+winners, and just random people who were in the right place at the right time.
67+One is a Silicon Valley tycoon. Another is an Orthodox nun in a monastery in
68+Greece. A couple are ex-Tlon employees who might totally hate us. There is
69+nothing they all have in common, which is sort of the point.
70+
71+This property distribution looks a lot like land ownership in the real world --
72+complex and asymmetrical. That's no accident. If you have a problem living in
73+New York because New York's real estate is not distributed equally among its
74+residents, you may have a similar problem with Urbit. The reality is that Urbit,
75+like New York, wasn't born yesterday and comes with a complex ownership history
76+which we couldn't possibly change, even if we wanted to.
77+
78+Also, even in Manhattan you can just sleep in your van, and in Urbit you can
40−Also, even in Manhattan you can just sleep in your van, and in Urbit you can always make your own comet. But people can't be prevented from thinking this means you're a weirdo.
79+always make your own comet. But people can't be prevented from thinking this
41−
80+means you're a weirdo.
81+
42−## Urbit isn't even really decentralized! It has a government!
82+## Urbit isn't even really decentralized! It has a government!
43−
83+
44−TLDR: yeah. But at least it's not the DAO.
84+TLDR: yeah. But at least it's not the DAO.
45−
85+
46−Real TLDR: it's technically impossible for Urbit to be decentralized at this stage of its life. Urbit is designed to achieve decentralization, not be born with it.
86+Real TLDR: it's technically impossible for Urbit to be decentralized at this
47−
87+stage of its life. Urbit is designed to achieve decentralization, not be born
88+with it.
89+
90+A young network can't afford to be decentralized. It has to act as a unit.
48−A young network can't afford to be decentralized. It has to act as a unit. Because it has no network effect, it needs every scrap of efficiency it can get. Right now, Urbit needs a central government about ten times over.
91+Because it has no network effect, it needs every scrap of efficiency it can get.
49−
92+Right now, Urbit needs a central government about ten times over.
50−As we found out with the DAO, if a network isn't ready to be decentralized, its developers shouldn't even try. If Ethereum is hard-forked to roll back the DAO (which looks like the most probable result), it acknowledges that it both has a central government, and needs a central government. Decentralization is good. Pretend decentralization -- "decentralization theater" -- is harmful, and not a effective path to actual decentralization.
93+
51−
94+As we found out with the DAO, if a network isn't ready to be decentralized, its
52−Urbit did not have to get hacked to realize that a decentralized network can only be bootstrapped by a central government. It's part of Urbit's basic design. For example, by default your urbit upgrades its OS automatically (like an "evergreen" browser) with signed network updates. Whoever signs these updates is a central government by definition.
95+developers shouldn't even try. If Ethereum is hard-forked to roll back the DAO
53−
96+(which looks like the most probable result), it acknowledges that it both has a
97+central government, and needs a central government. Decentralization is good.
98+Pretend decentralization -- "decentralization theater" -- is harmful, and not a
99+effective path to actual decentralization.
100+
101+Urbit did not have to get hacked to realize that a decentralized network can
102+only be bootstrapped by a central government. It's part of Urbit's basic design.
103+For example, by default your urbit upgrades its OS automatically (like an
104+"evergreen" browser) with signed network updates. Whoever signs these updates is
105+a central government by definition.
106+
107+But the test of a free software project is whether it can fork. So long as the
108+unity of the PKI and the protocol is maintained, Urbit development can fork
54−But the test of a free software project is whether it can fork. So long as the unity of the PKI and the protocol is maintained, Urbit development can fork easily without disrupting the network. Any stress will fragment this central government, and as it fragments it grows more limited and stable.
109+easily without disrupting the network. Any stress will fragment this central
55−
110+government, and as it fragments it grows more limited and stable.
56−Urbit is designed to govern itself collectively by address space ownership -- effectively, "proof of stake." Any arbitrary power, will tend to fracture along these property lines. For instance, our hot updates are sent through the galaxy hierarchy; in a development fork, active galaxies would pick sides. This would actually be awesome for Urbit.
111+
57−
112+Urbit is designed to govern itself collectively by address space ownership --
113+effectively, "proof of stake." Any arbitrary power, will tend to fracture along
114+these property lines. For instance, our hot updates are sent through the galaxy
115+hierarchy; in a development fork, active galaxies would pick sides. This would
116+actually be awesome for Urbit.
117+
118+By fragmenting the galaxy space into 57 separate partitions, we've created a
58−By fragmenting the galaxy space into 57 separate partitions, we've created a global property map which is much more likely to splinter further, than reunite.
119+global property map which is much more likely to splinter further, than reunite.
59−
120+
121+In the long run, every galaxy should have a separate owner, who is a human being
60−In the long run, every galaxy should have a separate owner, who is a human being rather than a corporation. Combining galaxy holdings is regarded as creepy, and damages the value of both galaxies. Splitting up any remaining clumps is meritorious.
122+rather than a corporation. Combining galaxy holdings is regarded as creepy, and
61−
123+damages the value of both galaxies. Splitting up any remaining clumps is
124+meritorious.
125+
126+The technical goal of Urbit is to create a network which, *if it actually takes
62−The technical goal of Urbit is to create a network which, *if it actually takes over the world*, inevitably becomes decentralized. Even if Urbit takes over the world, nobody takes over the world. World domination must be technically implausible.
127+over the world*, inevitably becomes decentralized. Even if Urbit takes over the
63−
128+world, nobody takes over the world. World domination must be technically
129+implausible.
130+
64−This is the goal of all the new "web3" decentralized systems. If IPFS, Sandstorm or Ethereum takes over the world, nobody takes over the world. (The same is almost true of the Internet -- though it wouldn't be that surprising if a time traveler from 2116 revealed that ICANN had become a planetary government.)
131+This is the goal of all the new "web3" decentralized systems. If IPFS, Sandstorm
65−
132+or Ethereum takes over the world, nobody takes over the world. (The same is
133+almost true of the Internet -- though it wouldn't be that surprising if a time
134+traveler from 2116 revealed that ICANN had become a planetary government.)
135+
66−Urbit, by starting out as a centralized network, just reduces the technical problem to match the actual goal. Only long-term decentralization matters.
136+Urbit, by starting out as a centralized network, just reduces the technical
67−
137+problem to match the actual goal. Only long-term decentralization matters.
138+
68−It's very important that Urbit in 2116 be fully decentralized. For Urbit in 2016, pretending to be decentralized would be a joke at best, and a hoax at worst. And of course, no one can DAO us -- since the property system is still just a central registry.
139+It's very important that Urbit in 2116 be fully decentralized. For Urbit in
140+2016, pretending to be decentralized would be a joke at best, and a hoax at
141+worst. And of course, no one can DAO us -- since the property system is still
142+just a central registry.
69
143
70## {Sandstorm, IPFS, Ethereum, ...} is way ahead of Urbit.
144## {Sandstorm, IPFS, Ethereum, ...} is way ahead of Urbit.
71
145
72−In general, web3 (next distributed web) solutions don't compete with each other. They support each other. Concretely, they are all part of each others' network effect.
146+In general, web3 (next distributed web) solutions don't compete with each other.
147+They support each other. Concretely, they are all part of each others' network
148+effect.
73
149
150+IPFS, Sandstorm, Ethereum, ZeroTier, and Urbit can all take over the world at
151+the same time. None of them competes with the others. They enable each other.
74−IPFS, Sandstorm, Ethereum, ZeroTier, and Urbit can all take over the world at the same time. None of them competes with the others. They enable each other. For instance, IPFS is not a computer; Urbit will be a great computer to use IPFS from; IPFS is a great CDN for Urbit. Moar IPFS now!
152+For instance, IPFS is not a computer; Urbit will be a great computer to use IPFS
153+from; IPFS is a great CDN for Urbit. Moar IPFS now!
75
154
76−The closest thing to a competitor that Urbit has is Sandstorm. Sandstorm, which by the way rocks, is also a personal server.
155+The closest thing to a competitor that Urbit has is Sandstorm. Sandstorm, which
156+by the way rocks, is also a personal server.
77
157
158+But Sandstorm and Urbit compete in the way that, on the PC, the OS and the
78−But Sandstorm and Urbit compete in the way that, on the PC, the OS and the browser compete. In theory, the OS could solve the problems that the browser solves; or the browser could be the OS. In practice, Windows is a great OS to run a browser on.
159+browser compete. In theory, the OS could solve the problems that the browser
160+solves; or the browser could be the OS. In practice, Windows is a great OS to
161+run a browser on.
79
162
163+Similarly, a world in which Sandstorm takes off is an awesome world for Urbit,
80−Similarly, a world in which Sandstorm takes off is an awesome world for Urbit, because everyone who has a Sandstorm instance has a great platform for running Urbit on. (Although Sandstorm could still work on its UDP capabilities.)
164+because everyone who has a Sandstorm instance has a great platform for running
165+Urbit on. (Although Sandstorm could still work on its UDP capabilities.)
81
166
82−Urbit on Sandstorm will be a sort of "double-hulled" personal server, even safer and more friendly than either system alone. And Sandstorm is not its own P2P network, like Urbit.
167+Urbit on Sandstorm will be a sort of "double-hulled" personal server, even safer
168+and more friendly than either system alone. And Sandstorm is not its own P2P
169+network, like Urbit.
83
170
84## Urbit doesn't have enough planets for every human being.
171## Urbit doesn't have enough planets for every human being.
85
172
86−A planet is a tool, not a toy. Like a car, it's a device for a responsible and civilized adult. There aren't 4 billion cars in the world, nor 4 billion responsible adults.
173+A planet is a tool, not a toy. Like a car, it's a device for a responsible and
174+civilized adult. There aren't 4 billion cars in the world, nor 4 billion
175+responsible adults.
87
176
88−And limiting the supply of anything doesn't create a shortage. It sets a price. Ideally, in the long run, we think a planet should cost about $10. This could be off by an order of magnitude either way, and it wouldn't be the end of the world.
177+And limiting the supply of anything doesn't create a shortage. It sets a price.
178+Ideally, in the long run, we think a planet should cost about $10. This could be
179+off by an order of magnitude either way, and it wouldn't be the end of the
180+world.
89
181
182+If a planet costs $10, that means you paid $10 to get a planet. A planet with no
183+reputation has a positive reputation, because someone paid $10 for it and hasn't
90−If a planet costs $10, that means you paid $10 to get a planet. A planet with no reputation has a positive reputation, because someone paid $10 for it and hasn't yet done anything wrong. Positive default reputation is the single quality that makes a social network social.
184+yet done anything wrong. Positive default reputation is the single quality that
185+makes a social network social.
91
186
92−There is always a tradeoff between free speech and free beer. Urbit will always choose free speech.
187+There is always a tradeoff between free speech and free beer. Urbit will always
188+choose free speech.
93
189
94## Cloud computing is inherently non-private.
190## Cloud computing is inherently non-private.
95
191
192+TLDR: a general-purpose cloud computer is much, much more private than a
96−TLDR: a general-purpose cloud computer is much, much more private than a special-purpose cloud computer. In general, a VM is secure against corporations but not governments.
193+special-purpose cloud computer. In general, a VM is secure against corporations
97−
194+but not governments.
195+
98−One reason we want the decentralized Internet is that we're tired of giant corporations spying on us. Think about it this way: Amazon is Netflix's main competitor. And yet, Netflix runs its whole operation on AWS. Why is Netflix sure that Amazon won't spy on it?
196+One reason we want the decentralized Internet is that we're tired of giant
99−
197+corporations spying on us. Think about it this way: Amazon is Netflix's main
100−A general-purpose virtual machine is special from a privacy standpoint, just because it's commercially realistic to design terms of service that prevent the host from peeking and poking at the customer's machine. Crossing this moat is technically nontrivial and can't possibly happen by any reasonable accident. No hosting contract could contemplate it.
198+competitor. And yet, Netflix runs its whole operation on AWS. Why is Netflix
101−
199+sure that Amazon won't spy on it?
200+
201+A general-purpose virtual machine is special from a privacy standpoint, just
202+because it's commercially realistic to design terms of service that prevent the
203+host from peeking and poking at the customer's machine. Crossing this moat is
204+technically nontrivial and can't possibly happen by any reasonable accident. No
205+hosting contract could contemplate it.
206+
102−Whereas when your cloud computer is a special-purpose machine -- just a database row on some corporation's mainframe -- there is no way these services can promise not to handle your data. Handling your data is *what they do*. On a special-purpose server, the concept of privacy is not even defined.
207+Whereas when your cloud computer is a special-purpose machine -- just a database
103−
208+row on some corporation's mainframe -- there is no way these services can
104−Where even a general-purpose server falls down is where the user has a sovereign adversary, which can secretly violate explicit or implicit hosting contracts. Users with a sovereign adversary must rely only on their own physical security. However, most users don't have a sovereign adversary. And those who do maximize their "herd immunity" if they can use the same protocols as those who don't.
209+promise not to handle your data. Handling your data is *what they do*. On a
105−
210+special-purpose server, the concept of privacy is not even defined.
211+
212+Where even a general-purpose server falls down is where the user has a sovereign
213+adversary, which can secretly violate explicit or implicit hosting contracts.
214+Users with a sovereign adversary must rely only on their own physical security.
215+However, most users don't have a sovereign adversary. And those who do maximize
216+their "herd immunity" if they can use the same protocols as those who don't.
217+
218+Also, the portability of a cloud server image is critical. Of course, since
219+Urbit is formally specified, it's trivially portable. In fact, one strategy
106−Also, the portability of a cloud server image is critical. Of course, since Urbit is formally specified, it's trivially portable. In fact, one strategy against sovereign adversaries may be continuous rotation between hosts, like an MX missile.
220+against sovereign adversaries may be continuous rotation between hosts, like an
221+MX missile.
107
222
108## "Jets" are, like, a total hoax.
223## "Jets" are, like, a total hoax.
109
224
110−TLDR: like, all of computing is a hoax. You can't compare a hoax to nothing. You can only compare it to all the other hoaxes.
225+TLDR: like, all of computing is a hoax. You can't compare a hoax to nothing. You
111−
226+can only compare it to all the other hoaxes.
112−A "jet" is a sort of software device driver in Urbit. Urbit is precisely defined in terms of an interpreter that fits on a T-shirt, Nock. But a naive implementation of Nock is nowhere near practical, because Nock has only one integer operation: increment. For example, to decrement a number in Nock, you increment a counter up to the number under it.
227+
113−
228+A "jet" is a sort of software device driver in Urbit. Urbit is precisely defined
114−The only way an efficient Nock interpreter can work is to recognize algorithms like decrement, and implement them non-naively. By far the easiest way to do this is not to analyze arbitrary code as it's being executed, but to register and match well-known implementations in the standard library. No one should be writing their own decrement.
229+in terms of an interpreter that fits on a T-shirt, Nock. But a naive
115−
230+implementation of Nock is nowhere near practical, because Nock has only one
231+integer operation: increment. For example, to decrement a number in Nock, you
232+increment a counter up to the number under it.
233+
234+The only way an efficient Nock interpreter can work is to recognize algorithms
235+like decrement, and implement them non-naively. By far the easiest way to do
236+this is not to analyze arbitrary code as it's being executed, but to register
237+and match well-known implementations in the standard library. No one should be
238+writing their own decrement.
239+
116−But the semantics of the interpreter, *so long as the jet and the jet harness are defined correctly*, are exactly those of Nock. We are not extending or changing Nock in any way. We are just taking a shortcut in certain cases we understand perfectly.
240+But the semantics of the interpreter, *so long as the jet and the jet harness
117−
241+are defined correctly*, are exactly those of Nock. We are not extending or
118−It's not at all unprecedented to have implementations much more complex than the corresponding specification. The x86 instruction set is relatively simple; the complexity of its decoding and execution path in a modern CPU is enormous. This path may well have "peephole optimizations" that replace matched algorithms, and if it doesn't the modern C compiler does.
242+changing Nock in any way. We are just taking a shortcut in certain cases we
119−
243+understand perfectly.
244+
245+It's not at all unprecedented to have implementations much more complex than the
246+corresponding specification. The x86 instruction set is relatively simple; the
247+complexity of its decoding and execution path in a modern CPU is enormous. This
248+path may well have "peephole optimizations" that replace matched algorithms, and
249+if it doesn't the modern C compiler does.
250+
251+So jets are *morally* defensible. The fact remains, however, that a Nock
252+interpreter without jets is completely useless. And there is not even an
120−So jets are *morally* defensible. The fact remains, however, that a Nock interpreter without jets is completely useless. And there is not even an informal document that tells you what jets you need for a reasonably efficient Urbit. (This last, we can agree, should be corrected.)
253+informal document that tells you what jets you need for a reasonably efficient
121−
254+Urbit. (This last, we can agree, should be corrected.)
255+
256+The correct way to think about jets is not to compare them to a perfect
257+interpreter implemented by unicorns. Jets are a combined replacement for two
122−The correct way to think about jets is not to compare them to a perfect interpreter implemented by unicorns. Jets are a combined replacement for two other techniques in language implementation: intrinsics and a foreign-function interface.
258+other techniques in language implementation: intrinsics and a foreign-function
123−
259+interface.
124−Compared to both intrinsics and an FFI, your jets have one key disadvantage: they need to be specified in pure code. You may not have this executable specification, which means you need to write it. Is this really so bad for your development process? An executable specification? Which can be trivially tested against the efficient implementation?
260+
125−
261+Compared to both intrinsics and an FFI, your jets have one key disadvantage:
262+they need to be specified in pure code. You may not have this executable
263+specification, which means you need to write it. Is this really so bad for your
264+development process? An executable specification? Which can be trivially tested
265+against the efficient implementation?
266+
126−Compared to intrinsics, jets give you true separation between mechanism and policy; only the interpreter, not the programmer and not the compiler, needs to know what's accelerated.
267+Compared to intrinsics, jets give you true separation between mechanism and
127−
268+policy; only the interpreter, not the programmer and not the compiler, needs to
269+know what's accelerated.
270+
271+Compared to an FFI, jets preclude I/O, so can be trivially sandboxed. Except
128−Compared to an FFI, jets preclude I/O, so can be trivially sandboxed. Except that the programmer needs to mark routines which may benefit from optimization with a hint, here too we separate mechanism (efficient implementation) from policy.
272+that the programmer needs to mark routines which may benefit from optimization
129−
273+with a hint, here too we separate mechanism (efficient implementation) from
274+policy.
275+
130−Yes, formal verification of jet equivalence is desirable. However, in the real world, many specifications, including quite complex ones, are tested into equivalence by brute force.
276+Yes, formal verification of jet equivalence is desirable. However, in the real
131−
277+world, many specifications, including quite complex ones, are tested into
278+equivalence by brute force.
279+
132−Urbit has some jet abuses. The equivalence between our markdown parser in Hoon and the CommonMark C library is super dubious. It should be taken out and shot.
280+Urbit has some jet abuses. The equivalence between our markdown parser in Hoon
133−
281+and the CommonMark C library is super dubious. It should be taken out and shot.
134−However, the miscomputation resulting from a minor jet discrepancy is not a security hole or even fatal. It effectively means the event history does not match the current state. The way to resolve this conflict is for the current state to win, and the event history to be discarded -- not abnormal anyway. See the next question.
282+
283+However, the miscomputation resulting from a minor jet discrepancy is not a
284+security hole or even fatal. It effectively means the event history does not
285+match the current state. The way to resolve this conflict is for the current
286+state to win, and the event history to be discarded -- not abnormal anyway. See
287+the next question.
135
288
136## Urbit needs an infinitely extending packet/event log.
289## Urbit needs an infinitely extending packet/event log.
137
290
138TLDR: no, it doesn't.
291TLDR: no, it doesn't.
139
292
140−We can prune the event log because the Urbit transition function doesn't have any access to the event history -- just the current state and the current event.
293+We can prune the event log because the Urbit transition function doesn't have
294+any access to the event history -- just the current state and the current event.
141
295
296+Urbit defines the semantics of a complete event log. It doesn't and can't
297+require an implementation to maintain the whole log. It can maintain a
142−Urbit defines the semantics of a complete event log. It doesn't and can't require an implementation to maintain the whole log. It can maintain a checkpoint and the log since that checkpoint. The current implementation of Urbit keeps the whole log, but the next release will optionally prune it.
298+checkpoint and the log since that checkpoint. The current implementation of
299+Urbit keeps the whole log, but the next release will optionally prune it.
143
300
144## Urbit will never be able to call any existing code.
301## Urbit will never be able to call any existing code.
145
302
146−TLDR: neither will Javascript. And dear Unix, the browser does not miss interacting with your `$LD_LIBRARY_PATH` at all.
303+TLDR: neither will Javascript. And dear Unix, the browser does not miss
304+interacting with your `$LD_LIBRARY_PATH` at all.
147
305
306+You can't talk to the OS or use system libraries from Javascript. Therefore,
148−You can't talk to the OS or use system libraries from Javascript. Therefore, there is no point in having any kind of gateway, compatibility bridge, native methods, or other interface between Javascript and C.
307+there is no point in having any kind of gateway, compatibility bridge, native
308+methods, or other interface between Javascript and C.
149
309
150−When in doubt, remember that Urbit is "the browser for the server side." It plays by the same general rules. Isolation means isolation.
310+When in doubt, remember that Urbit is "the browser for the server side." It
311+plays by the same general rules. Isolation means isolation.
151
312
152−## There's no way Urbit will ever be adopted. It's too weird.
313+## There's no way Urbit will ever be adopted. It's too weird.
153
314
154−TLDR: Urbit adoption is designed to be driven by ordinary end users. Urbit isn't ready for ordinary end users yet. We're working on it -- come help.
315+TLDR: Urbit adoption is designed to be driven by ordinary end users. Urbit isn't
316+ready for ordinary end users yet. We're working on it -- come help.
155
317
318+Right now, Urbit is a developer tool. Or rather, a developer toy -- since the
156−Right now, Urbit is a developer tool. Or rather, a developer toy -- since the only thing it's useful for is developing Urbit. It also could be a much better toy. We're working on that too.
319+only thing it's useful for is developing Urbit. It also could be a much better
320+toy. We're working on that too.
157
321
158−But it's absolutely wrong to think of Urbit being adopted as a developer tool in the conventional sense. If you think this is almost impossible, you're right.
322+But it's absolutely wrong to think of Urbit being adopted as a developer tool in
323+the conventional sense. If you think this is almost impossible, you're right.
159
324
160−If we look at the way most successful platforms are adopted, they're not adopted for their own virtues. They're adopted because they're part of a larger solution. It's very hard to displace a native component of an existing platform. Urbit is not trying to do this.
325+If we look at the way most successful platforms are adopted, they're not adopted
326+for their own virtues. They're adopted because they're part of a larger
327+solution. It's very hard to displace a native component of an existing platform.
328+Urbit is not trying to do this.
161
329
162−C is a better language than Pascal, but it didn't beat Pascal because it's a better language. C beat Pascal because C was the native language of Unix, and people wanted to program in Unix. People don't use Javascript because Javascript is a great language, but because it's the native language of the browser. Replacing the native language of a platform may be doable -- but it's incredibly hard.
330+C is a better language than Pascal, but it didn't beat Pascal because it's a
331+better language. C beat Pascal because C was the native language of Unix, and
332+people wanted to program in Unix. People don't use Javascript because Javascript
333+is a great language, but because it's the native language of the browser.
334+Replacing the native language of a platform may be doable -- but it's incredibly
335+hard.
163
336
164−Hoon is not competing with C, or even Haskell, as a Unix programming language. The only reason to program in Hoon is that it's the native language of Urbit. Urbit is not competing with node.js or Rails as a Web backend. The only reason to run Urbit is to run Urbit.
337+Hoon is not competing with C, or even Haskell, as a Unix programming language.
338+The only reason to program in Hoon is that it's the native language of Urbit.
339+Urbit is not competing with node.js or Rails as a Web backend. The only reason
340+to run Urbit is to run Urbit.
165
341
166−Nock is cool, but it only gets adopted if Hoon gets adopted. Hoon is cool, but it only gets adopted if Urbit gets adopted. Urbit only gets adopted if it's useful as a personal server for human beings -- most of whom won't be Hoon programmers, or programmers at all.
342+Nock is cool, but it only gets adopted if Hoon gets adopted. Hoon is cool, but
343+it only gets adopted if Urbit gets adopted. Urbit only gets adopted if it's
344+useful as a personal server for human beings -- most of whom won't be Hoon
345+programmers, or programmers at all.
167
346
168−## Hoon is too weird. No one will be able to program in it.
347+## Hoon is too weird. No one will be able to program in it.
169
348
170−TLDR: Hoon is ridiculously simple. Our documentation and tooling are just immature.
349+TLDR: Hoon is ridiculously simple. Our documentation and tooling are just
350+immature.
171
351
172−It's pretty fair to assume that (a) functional programming is hard and weird, (b) Hoon is weird even for a functional language, so (c) Hoon must be hard-squared.
352+It's pretty fair to assume that (a) functional programming is hard and weird,
353+(b) Hoon is weird even for a functional language, so (c) Hoon must be
354+hard-squared.
173
355
356+Actually, Hoon is weird even for a functional language because it's an attempt
174−Actually, Hoon is weird even for a functional language because it's an attempt to remove the hardest things about functional programming: category theory, lazy evaluation, and gratuitous abstraction. To know a language is to know what the compiler is doing; all the semantics of the Hoon compiler fit in 2000 lines of code.
357+to remove the hardest things about functional programming: category theory, lazy
358+evaluation, and gratuitous abstraction. To know a language is to know what the
359+compiler is doing; all the semantics of the Hoon compiler fit in 2000 lines of
360+code.
175
361
362+It's too easy for us to wind up living in Isaac Asimov's Trantor, a world in
363+which everything got invented thousands of years ago and the idea of new
176−It's too easy for us to wind up living in Isaac Asimov's Trantor, a world in which everything got invented thousands of years ago and the idea of new technology is just weird and scary. History isn't over yet, and nor is technical history.
364+technology is just weird and scary. History isn't over yet, and nor is technical
365+history.
177
366
367+New things are harder to learn than old things. We ask smart high-school
178−New things are harder to learn than old things. We ask smart high-school students to learn calculus. Think about how much harder it was for Newton and Leibniz to invent calculus, or for their first students to pick it up.
368+students to learn calculus. Think about how much harder it was for Newton and
369+Leibniz to invent calculus, or for their first students to pick it up.
179
370
180−Most of the difference is just the collective certainty that our society conveys to a high-school student that calculus is learnable. When you're learning Urbit, you don't have this collective knowledge; so you don't have the confidence that the problem is solvable. Confidence makes a huge difference. Moreover, any bug or glitch in the developer experience destroys this confidence and takes a long time to heal.
371+Most of the difference is just the collective certainty that our society conveys
372+to a high-school student that calculus is learnable. When you're learning Urbit,
373+you don't have this collective knowledge; so you don't have the confidence that
374+the problem is solvable. Confidence makes a huge difference. Moreover, any bug
375+or glitch in the developer experience destroys this confidence and takes a long
376+time to heal.
181
377
378+Yes, we know: Hoon remains quite hard to learn. Again, we have no reason to
182−Yes, we know: Hoon remains quite hard to learn. Again, we have no reason to think this problem isn't solvable. Public enlightenment remains our main priority.
379+think this problem isn't solvable. Public enlightenment remains our main
380+priority.
183
381
184## Urbit programmers are stuck with Hoon.
382## Urbit programmers are stuck with Hoon.
185
383
186TLDR: patience, grasshopper.
384TLDR: patience, grasshopper.
187
385
386+There's quite a straightforward way to implement any perfectly specified
188−There's quite a straightforward way to implement any perfectly specified language with a reliable interpreter in Hoon. First, write an interpreter for that language in Hoon. Second, jet that intepreter function with the existing implementation.
387+language with a reliable interpreter in Hoon. First, write an interpreter for
388+that language in Hoon. Second, jet that intepreter function with the existing
389+implementation.
189
390
190−It is not clear that Javascript is *quite* well-defined enough for this treatment. But Lua probably is.
391+It is not clear that Javascript is *quite* well-defined enough for this
392+treatment. But Lua probably is.
191
393
192## Ordinary users will never be able to manage an Urbit server.
394## Ordinary users will never be able to manage an Urbit server.
193
395
194−TLDR: Urbit's technical simplicity has always been designed to drive human usability. But the polish is the last thing that goes on. So we can't evaluate this claim right now.
396+TLDR: Urbit's technical simplicity has always been designed to drive human
397+usability. But the polish is the last thing that goes on. So we can't evaluate
398+this claim right now.
195
399
196−Right now, Urbit is only for geeks. You already need to be able to wrangle a Unix machine to run it. What's the point?
400+Right now, Urbit is only for geeks. You already need to be able to wrangle a
401+Unix machine to run it. What's the point?
197
402
198−This Urbit isn't the real Urbit. It's an early milestone on the way to the real Urbit.
403+This Urbit isn't the real Urbit. It's an early milestone on the way to the real
404+Urbit.
199
405
406+The real Urbit -- for normal human beings -- is a hosted service. Getting a
200−The real Urbit -- for normal human beings -- is a hosted service. Getting a planet is as easy as signing up for Facebook. Probably many users will think of it as another web account. They might not even know it can run more apps than the ones they use.
407+planet is as easy as signing up for Facebook. Probably many users will think of
408+it as another web account. They might not even know it can run more apps than
409+the ones they use.
201
410
411+A deterministic, persistent cloud computer is something different from a
202−A deterministic, persistent cloud computer is something different from a traditional server. Normally when we think of a server in the cloud, we think of it like a live animal -- a creature that needs to be cared for, fed, and herded. A file in the cloud is more like a rock -- you can forget about it for ten years.
412+traditional server. Normally when we think of a server in the cloud, we think of
413+it like a live animal -- a creature that needs to be cared for, fed, and herded.
414+A file in the cloud is more like a rock -- you can forget about it for ten
415+years.
203
416
204−An urbit is a general-purpose computer, but it doesn't even have the concept of a reboot. It upgrades itself. If you don't feed it any packets for ten years, then turn it back on, it will spend a day or two upgrading itself, then be back to normal. Maybe it's something in between a rock and an animal -- a cactus?
417+An urbit is a general-purpose computer, but it doesn't even have the concept of
418+a reboot. It upgrades itself. If you don't feed it any packets for ten years,
419+then turn it back on, it will spend a day or two upgrading itself, then be back
420+to normal. Maybe it's something in between a rock and an animal -- a cactus?
205
421
206−Another huge difference versus an Internet server is that we're confident that Urbit's limited supply of identities will make it easy to keep abusers off the network. So far, so good -- but of course, no one abuses a young network.
422+Another huge difference versus an Internet server is that we're confident that
423+Urbit's limited supply of identities will make it easy to keep abusers off the
424+network. So far, so good -- but of course, no one abuses a young network.
207
425
208−The bottom line is that managing your urbit has to be as easy as managing your iPhone. Normally, it consists of deciding what apps you want to be running, and how much you trust them. Resources can't be free, so sometimes you have to play memory police. Humans have shown that they can handle these tasks.
426+The bottom line is that managing your urbit has to be as easy as managing your
427+iPhone. Normally, it consists of deciding what apps you want to be running, and
428+how much you trust them. Resources can't be free, so sometimes you have to play
429+memory police. Humans have shown that they can handle these tasks.
209
430
210−Our '70s OSes and networks, which were great designs in their day and have matured into excellent industrial tools, give us a totally misleading impression of how hard it should be to manage a general-purpose computer. Computing isn't scary or dangerous. Unix and the Internet are scary and dangerous. That's fine for an industrial tool, but not a personal tool.
431+Our '70s OSes and networks, which were great designs in their day and have
432+matured into excellent industrial tools, give us a totally misleading impression
433+of how hard it should be to manage a general-purpose computer. Computing isn't
434+scary or dangerous. Unix and the Internet are scary and dangerous. That's fine
435+for an industrial tool, but not a personal tool.
211
436
212## Urbit has no killer app.
437## Urbit has no killer app.
213
438
214TLDR: the killer app is personal API aggregation.
439TLDR: the killer app is personal API aggregation.
215
440
216−It's a truism that any new platform has to be fundamentally much better at solving some problem than the mature platform it seeks to replace.
441+It's a truism that any new platform has to be fundamentally much better at
217−
442+solving some problem than the mature platform it seeks to replace.
443+
218−We can look at Urbit's killer app from two perspectives: as a developer, and as a user.
444+We can look at Urbit's killer app from two perspectives: as a developer, and as
219−
445+a user.
220−From the developer's perspective, Urbit's advantage is that it's a network stack which provides exactly-once delivery of typed, validated, transactional, encrypted and authenticated messages. The difference between an Urbit API and a REST API is like the difference between a paved road and a dirt road. On the other hand, you do need to learn a new functional language.
446+
221−
447+From the developer's perspective, Urbit's advantage is that it's a network stack
448+which provides exactly-once delivery of typed, validated, transactional,
449+encrypted and authenticated messages. The difference between an Urbit API and a
450+REST API is like the difference between a paved road and a dirt road. On the
451+other hand, you do need to learn a new functional language.
452+
222−From the user's perspective, there's one task that any personal server can perform, that can't be performed by any Web service: unrestricted API aggregation.
453+From the user's perspective, there's one task that any personal server can
223−
454+perform, that can't be performed by any Web service: unrestricted API
455+aggregation.
456+
224−Well before Urbit is ready to *replace* any conventional web services, it should be ready to *command* them in place by API. (Or if needed, by scraping.) If all you care about is being in control of your own data, it's fine to leave it in the hands of these services, as you have the API secrets to get it out.
457+Well before Urbit is ready to *replace* any conventional web services, it should
225−
458+be ready to *command* them in place by API. (Or if needed, by scraping.) If all
459+you care about is being in control of your own data, it's fine to leave it in
460+the hands of these services, as you have the API secrets to get it out.
461+
462+Managing a set of security connections to your existing services is one of the
463+first important tasks of a modern personal server. This is a stateful task that
464+typically requires both a durable API key and a temporary OAuth token. APIs
226−Managing a set of security connections to your existing services is one of the first important tasks of a modern personal server. This is a stateful task that typically requires both a durable API key and a temporary OAuth token. APIs change all the time, so connection maintenance also requires evergreen updates.
465+change all the time, so connection maintenance also requires evergreen updates.
227−
466+
228−The origin of this API key will vary, but usually the best way to set it is BYOK: the user registers their own urbit as an "app." And the neat thing about having your own API keys, for multiple services, on a single general-purpose computer, is that none of these services really has any practical way to control what you do with your own data.
467+The origin of this API key will vary, but usually the best way to set it is
229−
468+BYOK: the user registers their own urbit as an "app." And the neat thing about
469+having your own API keys, for multiple services, on a single general-purpose
470+computer, is that none of these services really has any practical way to control
471+what you do with your own data.
472+
473+Terms of service which constrain the use of data (as opposed to the abuse of the
474+service) simply aren't realistic and will no longer apply. This enables a
230−Terms of service which constrain the use of data (as opposed to the abuse of the service) simply aren't realistic and will no longer apply. This enables a variety of novel user experiences which aren't implementable as traditional web services, and also aren't available via conventional third-party API aggregators.
475+variety of novel user experiences which aren't implementable as traditional web
231−
476+services, and also aren't available via conventional third-party API
232−Also, since Urbit is after all quite immature, it doesn't make much sense to restrict this security state to code within Urbit. The Urbit interpreter `vere` runs as a Unix daemon that implements the FUSE user-level filesystem protocol, and maps the data on your services into the Unix filesystem. For example, one of our demos lets you `ls` your Github issues.
477+aggregators.
233−
478+
479+Also, since Urbit is after all quite immature, it doesn't make much sense to
480+restrict this security state to code within Urbit. The Urbit interpreter `vere`
481+runs as a Unix daemon that implements the FUSE user-level filesystem protocol,
482+and maps the data on your services into the Unix filesystem. For example, one of
483+our demos lets you `ls` your Github issues.
484+
234−So it's straightforward to build, for instance, a Python app that uses Urbit as a gateway to mash up a bunch of web services. As a Unix programmer, you might even just regard Urbit as a generic Web API gateway daemon, without worrying about its internals.
485+So it's straightforward to build, for instance, a Python app that uses Urbit as
235−
486+a gateway to mash up a bunch of web services. As a Unix programmer, you might
487+even just regard Urbit as a generic Web API gateway daemon, without worrying
488+about its internals.
489+
490+We have other ideas for initial killer apps. But this, we feel, is the best one.
491+API libraries for `npm` et al abound. Stateful, self-updating, generic API
236−We have other ideas for initial killer apps. But this, we feel, is the best one. API libraries for `npm` et al abound. Stateful, self-updating, generic API aggregators with FUSE integration, also integrated with a reactive web server, are more rare. So perhaps there's a small niche here.
492+aggregators with FUSE integration, also integrated with a reactive web server,
493+are more rare. So perhaps there's a small niche here.
237
494
238## Urbit is sneered at by brilliant talking horses.
495## Urbit is sneered at by brilliant talking horses.
239
496
497+It's difficult to categorize this [brilliant essay of
240−It's difficult to categorize this [brilliant essay of Fare's](http://ngnghm.github.io/blog/2016/06/11/chapter-10-houyhnhnms-vs-martians/) in a single bullet point. So I've saved it for last.
498+Fare's](http://ngnghm.github.io/blog/2016/06/11/chapter-10-houyhnhnms-vs-martians/)
499+in a single bullet point. So I've saved it for last.
241
500
242−We've already addressed the point most commonly made by others: that jets are a sham. It's not always trivial to decode this critique, which is quite abstract. But Fare' makes a few other points.
501+We've already addressed the point most commonly made by others: that jets are a
502+sham. It's not always trivial to decode this critique, which is quite abstract.
503+But Fare' makes a few other points.
243
504
244### Event-sourcing and/or caching is a pattern, not a layer
505### Event-sourcing and/or caching is a pattern, not a layer
245
506
246−Fare' is perfectly right that "one size does not fit all" in persistence. Generally, Urbit uses event-sourced persistence.
507+Fare' is perfectly right that "one size does not fit all" in persistence.
508+Generally, Urbit uses event-sourced persistence.
247
509
248−But it has multiple layers of event-sourcing: in the definition of Urbit itself; in the application sandbox vane (Gall), and in the application itself. Urbit also has a completely orthogonal revision control system (Clay).
510+But it has multiple layers of event-sourcing: in the definition of Urbit itself;
511+in the application sandbox vane (Gall), and in the application itself. Urbit
512+also has a completely orthogonal revision control system (Clay).
249
513
250−And there is not a single cache across the whole system, though arguably there should be; for instance, the functional build system (Ford, which Fare' the master of build systems might appreciate) has its own internal cache.
514+And there is not a single cache across the whole system, though arguably there
515+should be; for instance, the functional build system (Ford, which Fare' the
516+master of build systems might appreciate) has its own internal cache.
251
517
252−It is a common mistake to confuse a design pattern with a layer or feature, and equally fatal whichever direction you fail.
518+It is a common mistake to confuse a design pattern with a layer or feature, and
519+equally fatal whichever direction you fail.
253
520
254### Let's be clear about why DSLs suck
521### Let's be clear about why DSLs suck
255
522
256−DSLs suck not because they are difficult to newbies in the base language. DSLs suck because every time a programmer, even one *experienced* in the base language, comes into a new codebase or project, she has to learn a new custom language.
523+DSLs suck not because they are difficult to newbies in the base language. DSLs
524+suck because every time a programmer, even one *experienced* in the base
525+language, comes into a new codebase or project, she has to learn a new custom
526+language.
257
527
258−You need enormous wins to justify this cost. It's the goal of the base language designer to preempt these wins, creating a language which is almost never so tiresome and full of boilerplate that it requires a DSL or template layer.
528+You need enormous wins to justify this cost. It's the goal of the base language
529+designer to preempt these wins, creating a language which is almost never so
530+tiresome and full of boilerplate that it requires a DSL or template layer.
259
531
260−Hoon is almost entirely made of macros -- but they are internal, fixed, built-in macros (with no gensyms). This fixed set of macros is essentially enough. Occasionally we want to add one or two. But we never, ever pine for context-dependent macros.
532+Hoon is almost entirely made of macros -- but they are internal, fixed, built-in
533+macros (with no gensyms). This fixed set of macros is essentially enough.
534+Occasionally we want to add one or two. But we never, ever pine for
535+context-dependent macros.
261
536
262### Determinism is not consensus, and is not constraining
537### Determinism is not consensus, and is not constraining
263
538
539+Fare' is totally right that an urbit's data is meant for itself alone. In fact,
264−Fare' is totally right that an urbit's data is meant for itself alone. In fact, an urbit has the right to compute incorrectly if it wants. We go to considerable lengths to ensure that you can't gain any particular advantage by incorrect computing.
540+an urbit has the right to compute incorrectly if it wants. We go to considerable
541+lengths to ensure that you can't gain any particular advantage by incorrect
542+computing.
265
543
544+The whole point of Nock is that it's so low-level that there is no conceivable
266−The whole point of Nock is that it's so low-level that there is no conceivable advantage in upgrading it. Divergence is just pointless. This is not a question of network integrity, just of practical standardization.
545+advantage in upgrading it. Divergence is just pointless. This is not a question
546+of network integrity, just of practical standardization.
267
547
548+This consistency stops at Nock. Without anyone violating the correctness of
268−This consistency stops at Nock. Without anyone violating the correctness of their lifecycle functions, anyone or everyone in Urbit could replace Hoon with Javascript. Determinism does not cramp Urbit's semantic style in any way, shape or form.
549+their lifecycle functions, anyone or everyone in Urbit could replace Hoon with
550+Javascript. Determinism does not cramp Urbit's semantic style in any way, shape
551+or form.
docs/about/overview.md+47−58
@@ -1,88 +1,77 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: true
3+next: True
4sort: 1
4sort: 1
5title: Technical overview
5title: Technical overview
6---
6---
7
7
8# Technical overview
8# Technical overview
9
9
10−Urbit is a clean-slate system software stack defined as a
10+Urbit is a clean-slate system software stack defined as a deterministic
11−deterministic computer. An encrypted P2P network, `%ames`, runs
11+computer. An encrypted P2P network, `%ames`, runs on a functional operating
12−on a functional operating system, Arvo, written in a strict,
13−typed functional language, Hoon, which compiles itself to a
12+system, Arvo, written in a strict, typed functional language, Hoon, which
14−combinator interpreter, Nock, whose spec gzips to 340 bytes.
13+compiles itself to a combinator interpreter, Nock, whose spec gzips to 340
14+bytes.
15
15
16## Architectural overview
16## Architectural overview
17
17
18−A deterministic computer? Urbit's state is a pure function of
18+A deterministic computer? Urbit's state is a pure function of its event history.
19−its event history. In practice it uses a memory checkpoint and
19+In practice it uses a memory checkpoint and an append-only log. Every event is a
20−an append-only log. Every event is a transaction; Urbit is an
21−ACID database and a single-level store. Urbit runs on Unix now,
20+transaction; Urbit is an ACID database and a single-level store. Urbit runs on
22−but it's easy to imagine on a hypervisor or even bare metal.
21+Unix now, but it's easy to imagine on a hypervisor or even bare metal.
23
22
24−A purely functional OS? Urbit is pure -- no code inside it can
23+A purely functional OS? Urbit is pure -- no code inside it can make system calls
25−make system calls or otherwise affect the underlying platform.
24+or otherwise affect the underlying platform. Instead, the top-level event
26−Instead, the top-level event function defines an I/O protocol.
27−It maps an input event and the current state to a list of output
25+function defines an I/O protocol. It maps an input event and the current state
28−actions and the subsequent state. In Hoon:
26+to a list of output actions and the subsequent state. In Hoon:
29
27
30−```
31−$-([event state] [(list action) state])
28+ $-([event state] [(list action) state])
32−```
33
29
34### Nock
30### Nock
35
31
36−Nock is a sort of nano-Lisp without syntax, symbols or lambdas.
32+Nock is a sort of nano-Lisp without syntax, symbols or lambdas. Most Lisps are
37−Most Lisps are one-layer: they create a practical language by
33+one-layer: they create a practical language by extending a theoretically simple
38−extending a theoretically simple interpreter. The abstraction is
39−simple and the implementation is practical; there is no actual
34+interpreter. The abstraction is simple and the implementation is practical;
40−codebase both simple and practical. Hoon and Nock are two
35+there is no actual codebase both simple and practical. Hoon and Nock are two
41−layers: Hoon compiles itself to pure Nock. Since Urbit is
36+layers: Hoon compiles itself to pure Nock. Since Urbit is defined in Nock, not
42−defined in Nock, not Hoon, we can upgrade Hoon over the air.
37+Hoon, we can upgrade Hoon over the air.
43
38
44−The Nock data model is especially trivial. A *noun* is an atom
39+The Nock data model is especially trivial. A *noun* is an atom or a cell. An
45−or a cell. An atom is any unsigned integer. A cell is an
40+atom is any unsigned integer. A cell is an ordered pair of nouns. Nouns are
46−ordered pair of nouns. Nouns are acyclic and expose no pointer
41+acyclic and expose no pointer equality test.
47−equality test.
48
42
49### Hoon
43### Hoon
50
44
51−Hoon is a strict combinator language that avoids mathematical
45+Hoon is a strict combinator language that avoids mathematical theory and
52−theory and notation. It aims at a mechanical, imperative feel.
46+notation. It aims at a mechanical, imperative feel. Hoon uses ASCII digraphs
53−Hoon uses ASCII digraphs instead of keywords; there are no
54−user-level macros. The type system infers only forward and does
47+instead of keywords; there are no user-level macros. The type system infers only
55−not use unification, but is not much weaker than Haskell's. The
48+forward and does not use unification, but is not much weaker than Haskell's. The
56compiler and inference engine is about 2000 lines of Hoon.
49compiler and inference engine is about 2000 lines of Hoon.
57
50
58### Arvo
51### Arvo
59
52
60−Arvo is an event-driven OS written in Hoon. It can upgrade
53+Arvo is an event-driven OS written in Hoon. It can upgrade itself and everything
61−itself and everything inside it over the network. The Arvo
62−kernel proper is 500 lines of Hoon, which implements a typed
54+inside it over the network. The Arvo kernel proper is 500 lines of Hoon, which
63−event system with explicit call-stack structure. Arvo ships
55+implements a typed event system with explicit call-stack structure. Arvo ships
64−with modules that provide P2P networking (`%ames`), a revision-
56+with modules that provide P2P networking (`%ames`), a revision- control system
65−control system (`%clay`), a web client/server (`%eyre`), a
57+(`%clay`), a web client/server (`%eyre`), a functional build system (`%ford`),
66−functional build system (`%ford`), and an application engine
58+and an application engine `(%gall)`.
67−`(%gall)`.
68
59
69### Ames
60### Ames
70
61
71−Ames, the Urbit network, is an encrypted P2P protocol over UDP.
62+Ames, the Urbit network, is an encrypted P2P protocol over UDP. Its address
72−Its address space is semi-decentralized; 64-bit addresses are
63+space is semi-decentralized; 64-bit addresses are hierarchically distributed,
73−hierarchically distributed, 128-bit addresses are self-created.
74−Addresses (or *plots*) are rendered in a phonemic syntax for
64+128-bit addresses are self-created. Addresses (or *plots*) are rendered in a
75−memorability. The scarcity of short plots helps control spam and
65+phonemic syntax for memorability. The scarcity of short plots helps control spam
76−other Sybil attacks. The short plot hierarchy is also reused as
66+and other Sybil attacks. The short plot hierarchy is also reused as a supernode
77−a supernode routing system for NAT traversal.
67+routing system for NAT traversal.
78
68
79### Dojo and talk
69### Dojo and talk
80
70
81−Urbit ships with two default applications: a REPL or shell
71+Urbit ships with two default applications: a REPL or shell `:dojo`, and a
82−`:dojo`, and a distributed user-level message-bus `:talk`.
72+distributed user-level message-bus `:talk`. `:talk` under the hood resembles
83−`:talk` under the hood resembles NNTP; to the user, it looks like
84−a self-hosted Slack or persistent IRC.
73+NNTP; to the user, it looks like a self-hosted Slack or persistent IRC.
85
74
86−The full Urbit stack (compiler, standard library, kernel,
75+The full Urbit stack (compiler, standard library, kernel, modules, and
87−modules, and applications) is about 25,000 lines of Hoon.
76+applications) is about 25,000 lines of Hoon. Urbit is patent-free and MIT
88−Urbit is patent-free and MIT licensed.
77+licensed.
docs/about/runtime.md+794−923
@@ -1,67 +1,60 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 2
3sort: 2
4title: C runtime system
4title: C runtime system
5---
5---
6
6
7# C runtime system
7# C runtime system
8
8
9−The Urbit interpreter is built on a Nock runtime system written
9+The Urbit interpreter is built on a Nock runtime system written in C, `u3`. This
10−in C, `u3`. This document is a relatively complete description
10+document is a relatively complete description
11
11
12−You should keep reading if (a) you're planning to work on the
12+You should keep reading if (a) you're planning to work on the Urbit interpreter;
13−Urbit interpreter; (b) you're a language implementation geek; or
13+(b) you're a language implementation geek; or (c) you don't really understand
14−(c) you don't really understand anything until you've seen the
14+anything until you've seen the actual structs.
15−actual structs.
16
15
17## u3: noun processing in C
16## u3: noun processing in C
18
17
19−`u3` is the C library that makes Urbit work. If it wasn't called
18+`u3` is the C library that makes Urbit work. If it wasn't called `u3`, it might
20−`u3`, it might be called `libnoun` - it's a library for making
19+be called `libnoun` - it's a library for making and storing nouns.
21−and storing nouns.
22
20
23−What's a noun? A noun is either a cell or an atom. A cell is an
21+What's a noun? A noun is either a cell or an atom. A cell is an ordered pair of
24−ordered pair of any two nouns. An atom is an unsigned integer of
22+any two nouns. An atom is an unsigned integer of any size.
25−any size.
26
23
27−To the C programmer, this is not a terribly complicated data
24+To the C programmer, this is not a terribly complicated data structure, so why
28−structure, so why do you need a library for it?
25+do you need a library for it?
29
26
30−One: nouns have a well-defined computation kernel, Nock, whose
27+One: nouns have a well-defined computation kernel, Nock, whose spec fits on a
31−spec fits on a page and gzips to 340 bytes. But the only
28+page and gzips to 340 bytes. But the only arithmetic operation in Nock is
32−arithmetic operation in Nock is increment. So it's nontrivial
33−to compute both efficiently and correctly.
29+increment. So it's nontrivial to compute both efficiently and correctly.
34
30
35−Two: `u3` is designed to be a "solid-state interpreter," ie, a
31+Two: `u3` is designed to be a "solid-state interpreter," ie, a single-level
36−single-level store which is transparently snapshotted. This
32+store which is transparently snapshotted. This implies a specialized
37−implies a specialized memory-management model, etc, etc.
33+memory-management model, etc, etc.
38
34
39−(Does `u3` depend on the higher levels of Urbit, Arvo and Hoon?
35+(Does `u3` depend on the higher levels of Urbit, Arvo and Hoon? Yes and no. `u3`
40−Yes and no. `u3` expects you to load something shaped like an
36+expects you to load something shaped like an Arvo kernel, and use it as an
41−Arvo kernel, and use it as an event-processing function. But you
42−don't need to use this feature if you don't want, and your kernel
37+event-processing function. But you don't need to use this feature if you don't
43−doesn't have to be Arvo proper - just Arvo-compatible. Think of
38+want, and your kernel doesn't have to be Arvo proper - just Arvo-compatible.
44−`u3` as the BIOS and Arvo as the boot kernel. And there are no
39+Think of `u3` as the BIOS and Arvo as the boot kernel. And there are no
45dependencies at all between Hoon the language and `u3`.)
40dependencies at all between Hoon the language and `u3`.)
46
41
47## c3: C in Urbit
42## c3: C in Urbit
48
43
49−Under `u3` is the simple `c3` layer, which is just how we write C
44+Under `u3` is the simple `c3` layer, which is just how we write C in Urbit.
50−in Urbit.
51
45
52−When writing C in u3, please of course follow the conventions of
46+When writing C in u3, please of course follow the conventions of the code around
53−the code around you as regards indentation, etc. It's especially
47+you as regards indentation, etc. It's especially important that every function
54−important that every function have a header comment, even if it
55−says nothing interesting.
48+have a header comment, even if it says nothing interesting.
56
49
57−But some of our idiosyncrasies go beyond convention. Yes, we've
50+But some of our idiosyncrasies go beyond convention. Yes, we've done awful
58−done awful things to C. Here's what we did and why we did.
51+things to C. Here's what we did and why we did.
59
52
60### c3: integer types
53### c3: integer types
61
54
62−First, it's generally acknowledged that underspecified integer
55+First, it's generally acknowledged that underspecified integer types are C's
63−types are C's worst disaster. C99 fixed this, but the `stdint`
56+worst disaster. C99 fixed this, but the `stdint` types are wordy and annoying.
64−types are wordy and annoying. We've replaced them with:
57+We've replaced them with:
65
58
66 /* Good integers.
59 /* Good integers.
67 */
60 */
@@ -88,50 +81,47 @@ types are wordy and annoying. We've replaced them with:
88 typedef uintptr_t c3_p; // pointer-length uint - really really bad
81 typedef uintptr_t c3_p; // pointer-length uint - really really bad
89 typedef intptr_t c3_ps; // pointer-length int - really really bad
82 typedef intptr_t c3_ps; // pointer-length int - really really bad
90
83
91−Some of these need explanation. A loobean is a Nock boolean -
84+Some of these need explanation. A loobean is a Nock boolean - Nock, for
92−Nock, for mysterious reasons, uses 0 as true (always say "yes")
85+mysterious reasons, uses 0 as true (always say "yes") and 1 as false (always say
93−and 1 as false (always say "no").
86+"no").
94
87
95−Nock and/or Hoon cannot tell the difference between a short atom
88+Nock and/or Hoon cannot tell the difference between a short atom and a long one,
96−and a long one, but at the `u3` level every atom under `2^31` is
89+but at the `u3` level every atom under `2^31` is direct. The `c3_l` type is
97−direct. The `c3_l` type is useful to annotate this. A `c3_m` is
98−a *mote* - a string of up to 4 characters in a `c3_l`, least
90+useful to annotate this. A `c3_m` is a *mote* - a string of up to 4 characters
99−significant byte first. A `c3_g` should be a 5-bit atom. Of
91+in a `c3_l`, least significant byte first. A `c3_g` should be a 5-bit atom. Of
100course, C cannot enforce these constraints, only document them.
92course, C cannot enforce these constraints, only document them.
101
93
102−Use the "bad" - ie, poorly specified - integer types only when
94+Use the "bad" - ie, poorly specified - integer types only when interfacing with
103−interfacing with external code that expects them.
95+external code that expects them.
104
96
105−An enormous number of motes are defined in `i/c/motes.h`. There
97+An enormous number of motes are defined in `i/c/motes.h`. There is no reason to
106−is no reason to delete motes that aren't being used, or even to
98+delete motes that aren't being used, or even to modularize the definitions. Keep
107−modularize the definitions. Keep them alphabetical, though.
99+them alphabetical, though.
108
100
109### c3: variables and variable naming
101### c3: variables and variable naming
110
102
111−The C3 style uses Hoon style TLV variable names, with a quasi
103+The C3 style uses Hoon style TLV variable names, with a quasi Hungarian syntax.
112−Hungarian syntax. This is weird, but works really well, as long
104+This is weird, but works really well, as long as what you're doing isn't
113−as what you're doing isn't hideously complicated. (Then it works
114−badly, but we shouldn't need anything hideous in u3.)
105+hideously complicated. (Then it works badly, but we shouldn't need anything
106+hideous in u3.)
115
107
116−A TLV variable name is a random pronounceable three-letter
108+A TLV variable name is a random pronounceable three-letter string, sometimes
117−string, sometimes with some vague relationship to its meaning,
109+with some vague relationship to its meaning, but usually not. Usually CVC
118−but usually not. Usually CVC (consonant-vowel-consonant) is a
110+(consonant-vowel-consonant) is a good choice.
119−good choice.
120
111
121−You should use TLVs much the way math people use Greek letters.
112+You should use TLVs much the way math people use Greek letters. The same concept
122−The same concept should in general get the same name across
113+should in general get the same name across different contexts. When you're
123−different contexts. When you're working in a given area, you'll
124−tend to remember the binding from TLV to concept by sheer power
114+working in a given area, you'll tend to remember the binding from TLV to concept
125−of associative memory. When you come back to it, it's not that
115+by sheer power of associative memory. When you come back to it, it's not that
126−hard to relearn. And of course, when in doubt, comment it.
116+hard to relearn. And of course, when in doubt, comment it.
127
117
128−Variables take pseudo-Hungarian suffixes, matching in general the
118+Variables take pseudo-Hungarian suffixes, matching in general the suffix of the
129−suffix of the integer type:
119+integer type:
130
120
131 c3_w wor_w; // 32-bit word
121 c3_w wor_w; // 32-bit word
132
122
133−Unlike in standard Hungarian, there is no change for pointer
123+Unlike in standard Hungarian, there is no change for pointer variables. C
134−variables. C structure variables take a `_u` suffix.
124+structure variables take a `_u` suffix.
135
125
136### c3: loobeans
126### c3: loobeans
137
127
@@ -145,20 +135,19 @@ The code (from `defs.h`) tells the story:
145 # define c3a(x, y) __(_(x) && _(y))
135 # define c3a(x, y) __(_(x) && _(y))
146 # define c3o(x, y) __(_(x) || _(y))
136 # define c3o(x, y) __(_(x) || _(y))
147
137
148−In short, use `_()` to turn a loobean into a boolean, `__` to go
138+In short, use `_()` to turn a loobean into a boolean, `__` to go the other way.
149−the other way. Use `!` as usual, `c3y` for yes and `c3n` for no,
139+Use `!` as usual, `c3y` for yes and `c3n` for no, `c3a` for and and `c3o` for
150−`c3a` for and and `c3o` for or.
140+or.
151
141
152## u3: land of nouns
142## u3: land of nouns
153
143
154−The division between `c3` and `u3` is that you could theoretically
144+The division between `c3` and `u3` is that you could theoretically imagine using
155−imagine using `c3` as just a generic C environment. Anything to do
145+`c3` as just a generic C environment. Anything to do with nouns is in `u3`.
156−with nouns is in `u3`.
157
146
158### u3: a map of the system
147### u3: a map of the system
159
148
160−There are two kinds of symbols in `u3`: regular and irregular.
149+There are two kinds of symbols in `u3`: regular and irregular. Regular symbols
161−Regular symbols follow this pattern:
150+follow this pattern:
162
151
163 prefix purpose .h .c
152 prefix purpose .h .c
164 -------------------------------------------------------
153 -------------------------------------------------------
@@ -178,23 +167,20 @@ Regular symbols follow this pattern:
178 u3q[a-g] jets (retain, C args) i/j/q.h j/[a-g]/*.c
167 u3q[a-g] jets (retain, C args) i/j/q.h j/[a-g]/*.c
179 u3w[a-g] jets (retain, nock core) i/j/w.h j/[a-g]/*.c
168 u3w[a-g] jets (retain, nock core) i/j/w.h j/[a-g]/*.c
180
169
181−Irregular symbols always start with `u3` and obey no other rules.
170+Irregular symbols always start with `u3` and obey no other rules. They're
182−They're defined in `i/n/u.h`. Finally, `i/all.h` includes all
171+defined in `i/n/u.h`. Finally, `i/all.h` includes all these headers (fast
183−these headers (fast compilers, yay) and is all you need to
172+compilers, yay) and is all you need to program in `u3`.
184−program in `u3`.
185
173
186### u3: noun internals
174### u3: noun internals
187
175
188A noun is a `u3_noun` - currently defined as a 32-bit `c3_w`.
176A noun is a `u3_noun` - currently defined as a 32-bit `c3_w`.
189
177
190−If your `u3_noun` is less than `(1 << 31)`, it's a direct atom.
178+If your `u3_noun` is less than `(1 << 31)`, it's a direct atom. Every unsigned
191−Every unsigned integer between `0` and `0x7fffffff` inclusive is
179+integer between `0` and `0x7fffffff` inclusive is its own noun.
192−its own noun.
193
180
194−If bit `31` is set in a `u3_noun`, bit `30` is always set - this
181+If bit `31` is set in a `u3_noun`, bit `30` is always set - this bit is
195−bit is reserved. Bit `29` is `1` if the noun is a cell, `0` if
182+reserved. Bit `29` is `1` if the noun is a cell, `0` if it's an atom. Bits `28`
196−it's an atom. Bits `28` through `0` are a word pointer into the
197−loom - see below. The structures are:
183+through `0` are a word pointer into the loom - see below. The structures are:
198
184
199 typedef struct {
185 typedef struct {
200 c3_w mug_w;
186 c3_w mug_w;
@@ -208,57 +194,53 @@ loom - see below. The structures are:
208 u3_noun tel;
194 u3_noun tel;
209 } u3a_cell;
195 } u3a_cell;
210
196
211−The only thing that should be mysterious here is `mug_w`, which
197+The only thing that should be mysterious here is `mug_w`, which is a 31-bit
212−is a 31-bit lazily computed nonzero short hash (FNV currently,
198+lazily computed nonzero short hash (FNV currently, soon Murmur3). If `mug_w` is
213−soon Murmur3). If `mug_w` is 0, the hash is not yet computed.
214−We also hijack this field for various hacks, such as saving the
199+0, the hash is not yet computed. We also hijack this field for various hacks,
215−new address of a noun when copying over.
200+such as saving the new address of a noun when copying over.
216
201
217−Also, the value `0xffffffff` is `u3_none`, which is never a valid
202+Also, the value `0xffffffff` is `u3_none`, which is never a valid noun. Use the
218−noun. Use the type `u3_weak` to express that a noun variable may
203+type `u3_weak` to express that a noun variable may be `u3_none`.
219−be `u3_none`.
220
204
221### u3: reference counts
205### u3: reference counts
222
206
223−The only really essential thing you need to know about `u3` is
207+The only really essential thing you need to know about `u3` is how to handle
224−how to handle reference counts. Everything else, you can skip
208+reference counts. Everything else, you can skip and just get to work.
225−and just get to work.
226
209
227−u3 deals with reference-counted, immutable, acyclic nouns.
210+u3 deals with reference-counted, immutable, acyclic nouns. Unfortunately, we are
228−Unfortunately, we are not Apple and can't build reference
229−counting into your C compiler, so you need to count by hand.
211+not Apple and can't build reference counting into your C compiler, so you need
212+to count by hand.
230
213
231−Every allocated noun (or any allocation object, because our
214+Every allocated noun (or any allocation object, because our allocator is
232−allocator is general-purpose) contains a counter which counts the
215+general-purpose) contains a counter which counts the number of references to it
233−number of references to it - typically variables with type
234−`u3_noun`. When this counter goes to 0, the noun is freed.
216+- typically variables with type `u3_noun`. When this counter goes to 0, the noun
217+is freed.
235
218
236−To tell `u3` that you've added a reference to a noun, call the
219+To tell `u3` that you've added a reference to a noun, call the function
237−function `u3a_gain()` or its shorthand `u3k()`. (For your
220+`u3a_gain()` or its shorthand `u3k()`. (For your convenience, this function
238−convenience, this function returns its argument.) To tell `u3`
239−that you've destroyed a reference, call `u3a_lose()` or `u3z()`.
221+returns its argument.) To tell `u3` that you've destroyed a reference, call
222+`u3a_lose()` or `u3z()`.
240
223
241−(If you screw up by decrementing the counter too much, `u3` will
224+(If you screw up by decrementing the counter too much, `u3` will dump core in
242−dump core in horrible ways. If you screw up by incrementing it
225+horrible ways. If you screw up by incrementing it too much, `u3` will leak
243−too much, `u3` will leak memory. To check for memory leaks,
244−set the `bug_o` flag in `u3e_boot()` - eg, run `vere` with `-g`.
226+memory. To check for memory leaks, set the `bug_o` flag in `u3e_boot()` - eg,
245−Memory leaks are difficult to debug - the best way to handle
227+run `vere` with `-g`. Memory leaks are difficult to debug - the best way to
246−leaks is just to revert to a version that didn't have them, and
228+handle leaks is just to revert to a version that didn't have them, and look over
247−look over your code again.)
229+your code again.)
248
230
249−(You can gain or lose a direct atom. It does nothing.)
231+(You can gain or lose a direct atom. It does nothing.)
250
232
251### u3: reference protocols
233### u3: reference protocols
252
234
253*THIS IS THE MOST CRITICAL SECTION IN THE `u3` DOCUMENTATION.*
235*THIS IS THE MOST CRITICAL SECTION IN THE `u3` DOCUMENTATION.*
254
236
255−The key question when calling a C function in a refcounted world
237+The key question when calling a C function in a refcounted world is what the
256−is what the function will do to the noun refcounts - and, if the
238+function will do to the noun refcounts - and, if the function returns a noun,
257−function returns a noun, what it does to the return.
239+what it does to the return.
258
240
259−There are two semantic patterns, `transfer` and `retain`. In
241+There are two semantic patterns, `transfer` and `retain`. In `transfer`
260−`transfer` semantics, the caller "gives" a use count to the
242+semantics, the caller "gives" a use count to the callee, which "gives back" any
261−callee, which "gives back" any return. For instance, if I have
243+return. For instance, if I have
262
244
263 {
245 {
264 u3_noun foo = u3i_string("foobar");
246 u3_noun foo = u3i_string("foobar");
@@ -269,14 +251,12 @@ callee, which "gives back" any return. For instance, if I have
269 u3z(bar);
251 u3z(bar);
270 }
252 }
271
253
272−Suppose `u3f_futz()` has `transfer` semantics. At `[...]`, my
254+Suppose `u3f_futz()` has `transfer` semantics. At `[...]`, my code holds one
273−code holds one reference to `bar` and zero references to `foo` -
255+reference to `bar` and zero references to `foo` - which has been freed, unless
274−which has been freed, unless it's part of `bar`. My code now
275−owns `bar` and gets to work with it until it's done, at which
256+it's part of `bar`. My code now owns `bar` and gets to work with it until it's
276−point a `u3z()` is required.
257+done, at which point a `u3z()` is required.
277
258
278−On the other hand, if `u3f_futz()` has `retain` semantics, we
259+On the other hand, if `u3f_futz()` has `retain` semantics, we need to write
279−need to write
280
260
281 {
261 {
282 u3_noun foo = u3i_string("foobar");
262 u3_noun foo = u3i_string("foobar");
@@ -287,75 +267,71 @@ need to write
287 u3z(foo);
267 u3z(foo);
288 }
268 }
289
269
290−because calling `u3f_futz()` does not release our ownership of
270+because calling `u3f_futz()` does not release our ownership of `foo`, which we
291−`foo`, which we have to free ourselves.
271+have to free ourselves.
292
272
293−But if we free `bar`, we are making a great mistake, because our
273+But if we free `bar`, we are making a great mistake, because our reference to it
294−reference to it is not in any way registered in the memory
274+is not in any way registered in the memory manager (which cannot track
295−manager (which cannot track references in C variables, of
296−course). It is normal and healthy to have these uncounted
275+references in C variables, of course). It is normal and healthy to have these
297−C references, but they must be treated with care.
276+uncounted C references, but they must be treated with care.
298
277
299−The bottom line is that it's essential for the caller to know
278+The bottom line is that it's essential for the caller to know the refcount
300−the refcount semantics of any function which takes or returns a
279+semantics of any function which takes or returns a noun. (In some unusual
301−noun. (In some unusual circumstances, different arguments or
302−returns in one function may be handled differently.)
280+circumstances, different arguments or returns in one function may be handled
281+differently.)
303
282
304−Broadly speaking, as a design question, retain semantics are more
283+Broadly speaking, as a design question, retain semantics are more appropriate
305−appropriate for functions which inspect or query nouns. For
284+for functions which inspect or query nouns. For instance, `u3h()` (which takes
306−instance, `u3h()` (which takes the head of a noun) retains, so
307−that we can traverse a noun tree without constantly incrementing
285+the head of a noun) retains, so that we can traverse a noun tree without
308−and decrementing.
286+constantly incrementing and decrementing.
309
287
310−Transfer semantics are more appropriate for functions which make
288+Transfer semantics are more appropriate for functions which make nouns, which is
311−nouns, which is obviously what most functions do.
289+obviously what most functions do.
312
290
313−In general, though, in most places it's not worth thinking about
291+In general, though, in most places it's not worth thinking about what your
314−what your function does. There is a convention for it, which
292+function does. There is a convention for it, which depends on where it is, not
315−depends on where it is, not what it does. Follow the convention.
293+what it does. Follow the convention.
316
294
317### u3: reference conventions
295### u3: reference conventions
318−
319−The `u3` convention is that, unless otherwise specified, *all
320−functions have transfer semantics* - with the exception of the
321−prefixes: `u3r`, `u3x`, `u3z`, `u3q` and `u3w`. Also, within
322−jet directories `a` through `f` (but not `g`), internal functions
323−retain (for historical reasons).
324
296
297+The `u3` convention is that, unless otherwise specified, *all functions have
298+transfer semantics* - with the exception of the prefixes: `u3r`, `u3x`, `u3z`,
299+`u3q` and `u3w`. Also, within jet directories `a` through `f` (but not `g`),
300+internal functions retain (for historical reasons).
301+
325−If functions outside this set have retain semantics, they need to
302+If functions outside this set have retain semantics, they need to be commented,
326−be commented, both in the `.h` and `.c` file, with `RETAIN` in
303+both in the `.h` and `.c` file, with `RETAIN` in all caps. Yes, it's this
327−all caps. Yes, it's this important.
304+important.
328
305
329### u3: system architecture
306### u3: system architecture
330
307
331−If you just want to tinker with some existing code, it might be
308+If you just want to tinker with some existing code, it might be enough to
332−enough to understand the above. If not, it's probably worth
309+understand the above. If not, it's probably worth taking the time to look at
333−taking the time to look at `u3` as a whole.
310+`u3` as a whole.
334
311
335−`u3` is designed to work as a persistent event processor.
312+`u3` is designed to work as a persistent event processor. Logically, it computes
336−Logically, it computes a function of the form
313+a function of the form
337
314
338 f(event, old state) -> (actions, new state)
315 f(event, old state) -> (actions, new state)
339
316
340−Obviously almost any computing model - including, but not limited
317+Obviously almost any computing model - including, but not limited to, Urbit -
341−to, Urbit - can be defined in this form. To create the illusion
318+can be defined in this form. To create the illusion of a computer that never
342−of a computer that never loses state and never fails, we:
319+loses state and never fails, we:
343
320
344−- log every event externally before it goes into u3
321+- log every event externally before it goes into u3
345−- keep a single reference to a permanent state noun.
322+- keep a single reference to a permanent state noun.
346−- can abort any event without damaging the permanent state.
323+- can abort any event without damaging the permanent state.
347−- snapshot the permanent state periodically, and/or prune logs.
324+- snapshot the permanent state periodically, and/or prune logs.
348
325
349### u3: the road model
326### u3: the road model
350
327
351−`u3` uses a memory design which I'm sure someone has invented
328+`u3` uses a memory design which I'm sure someone has invented somewhere before,
352−somewhere before, because it's not very clever, but I've never
329+because it's not very clever, but I've never seen it anywhere in particular.
353−seen it anywhere in particular.
354
330
355−Every allocation starts with a solid block of memory, which `u3`
331+Every allocation starts with a solid block of memory, which `u3` calls the
356−calls the `loom`. How do we allocate on the loom? You're
332+`loom`. How do we allocate on the loom? You're probably familiar with the Unix
357−probably familiar with the Unix heap-stack design, in which the
358−stack grows downward and the heap (malloc arena) grows upward:
333+heap-stack design, in which the stack grows downward and the heap (malloc arena)
334+grows upward:
359
335
360 0 brk ffff
336 0 brk ffff
361 | heap | stack |
337 | heap | stack |
@@ -363,40 +339,35 @@ stack grows downward and the heap (malloc arena) grows upward:
363 | | |
339 | | |
364 0 sp ffff
340 0 sp ffff
365
341
366−A road is a normal heap-stack system, except that the heap
342+A road is a normal heap-stack system, except that the heap and stack can point
367−and stack can point in *either direction*. Therefore, inside
343+in *either direction*. Therefore, inside a road, we can nest another road in the
368−a road, we can nest another road in the *opposite direction*.
344+*opposite direction*.
369
345
370−When the opposite road completes, its heap is left on top of
346+When the opposite road completes, its heap is left on top of the opposite heap's
371−the opposite heap's stack. It's no more than the normal
347+stack. It's no more than the normal behavior of a stack machine for all
372−behavior of a stack machine for all subcomputations to push
373−their results on the stack.
348+subcomputations to push their results on the stack.
374
349
375−The performance tradeoff of "leaping" - reversing directions in
350+The performance tradeoff of "leaping" - reversing directions in the road - is
376−the road - is that if the outer computation wants to preserve the
351+that if the outer computation wants to preserve the results of the inner one,
377−results of the inner one, not just use them for temporary
378−purposes, it has to *copy them*.
352+not just use them for temporary purposes, it has to *copy them*.
379
353
380−This is a trivial cost in some cases, a prohibitive cost in
354+This is a trivial cost in some cases, a prohibitive cost in others. The upside,
381−others. The upside, of course, is that all garbage accrued
382−in the inner computation is discarded at zero cost.
355+of course, is that all garbage accrued in the inner computation is discarded at
356+zero cost.
383
357
384−The goal of the road system is the ability to *layer* memory
358+The goal of the road system is the ability to *layer* memory models. If you are
385−models. If you are allocating on a road, you have no idea
386−how deep within a nested road system you are - in other words,
359+allocating on a road, you have no idea how deep within a nested road system you
387−you have no idea exactly how durable your result may be.
360+are - in other words, you have no idea exactly how durable your result may be.
388But free space is never fragmented within a road.
361But free space is never fragmented within a road.
389
362
390−Roads do not reduce the generality or performance of a memory
363+Roads do not reduce the generality or performance of a memory system, since even
391−system, since even the most complex GC system can be nested
364+the most complex GC system can be nested within a road at no particular loss of
392−within a road at no particular loss of performance - a road
393−is just a block of memory.
365+performance - a road is just a block of memory.
394
366
395−Each road (`u3a_road` to be exact) uses four pointers: `rut` is
367+Each road (`u3a_road` to be exact) uses four pointers: `rut` is the bottom of
396−the bottom of the arena, `hat` the top of the arena, `mat` the
368+the arena, `hat` the top of the arena, `mat` the bottom of the stack, `cap` the
397−bottom of the stack, `cap` the top of the stack. (Bear in mind
398−that the road "stack" is not actually used as the C function-call
369+top of the stack. (Bear in mind that the road "stack" is not actually used as
399−stack, though it probably should be.)
370+the C function-call stack, though it probably should be.)
400
371
401A "north" road has the stack high and the heap low:
372A "north" road has the stack high and the heap low:
402
373
@@ -414,125 +385,112 @@ A "south" road is the other way around:
414 | | | |
385 | | | |
415 0 hat rut ffff
386 0 hat rut ffff
416
387
417−Legend: `-` is durable storage (heap); `+` is temporary storage
388+Legend: `-` is durable storage (heap); `+` is temporary storage (stack); `~` is
418−(stack); `~` is deep storage (immutable); `$` is the allocation
389+deep storage (immutable); `$` is the allocation frame `#` is free memory.
419−frame `#` is free memory.
420
390
421−Pointer restrictions: pointers stored in `+` can point anywhere.
391+Pointer restrictions: pointers stored in `+` can point anywhere. Pointers in `-`
422−Pointers in `-` can only point to `-` or `~`; pointers in `~`
392+can only point to `-` or `~`; pointers in `~` only point to `~`.
423−only point to `~`.
424
393
425−To "leap" is to create a new inner road in the `###` free space.
394+To "leap" is to create a new inner road in the `###` free space. but in the
426−but in the reverse direction, so that when the inner road
395+reverse direction, so that when the inner road "falls" (terminates), its durable
427−"falls" (terminates), its durable storage is left on the
428−temporary storage of the outer road.
396+storage is left on the temporary storage of the outer road.
429
397
430−`u3` keeps a global variable, `u3_Road` or its alias `u3R`, which
398+`u3` keeps a global variable, `u3_Road` or its alias `u3R`, which points to the
431−points to the current road. (If we ever run threads in inner
399+current road. (If we ever run threads in inner roads - see below - this will
432−roads - see below - this will become a thread-local variable.)
433−Relative to `u3R`, `+` memory is called `junior` memory; `-`
400+become a thread-local variable.) Relative to `u3R`, `+` memory is called
434−memory is `normal` memory; `~` is `senior` memory.
401+`junior` memory; `-` memory is `normal` memory; `~` is `senior` memory.
435
402
436### u3: explaining the road model
403### u3: explaining the road model
437
404
438But... why?
405But... why?
439
406
440−We're now ready to understand why the road system works so
407+We're now ready to understand why the road system works so logically with the
441−logically with the event and persistence model.
408+event and persistence model.
442−
409+
443−The key is that *we don't update refcounts in senior memory.*
410+The key is that *we don't update refcounts in senior memory.* A pointer from an
444−A pointer from an inner road to an outer road is not counted.
411+inner road to an outer road is not counted. Also, the outmost, or `surface`
445−Also, the outmost, or `surface` road, is the only part of the
446−image that gets checkpointed.
412+road, is the only part of the image that gets checkpointed.
447−
413+
448−So the surface road contains the entire durable state of `u3`.
414+So the surface road contains the entire durable state of `u3`. When we process
449−When we process an event, or perform any kind of complicated or
415+an event, or perform any kind of complicated or interesting calculation, *we
450−interesting calculation, *we process it in an inner road*. If
451−its results are saved, they need to be copied.
416+process it in an inner road*. If its results are saved, they need to be copied.
452−
417+
453−Since processing in an inner road does not touch surface memory,
418+Since processing in an inner road does not touch surface memory, (a) we can
454−(a) we can leave the surface road in a read-only state and not
419+leave the surface road in a read-only state and not mark its pages dirty; (b) we
455−mark its pages dirty; (b) we can abort an inner calculation
456−without screwing up the surface; and (c) because inner results
420+can abort an inner calculation without screwing up the surface; and (c) because
457−are copied onto the surface, the surface doesn't get fragmented.
421+inner results are copied onto the surface, the surface doesn't get fragmented.
458−
422+
459−All of (a), (b) and (c) are needed for checkpointing to be easy.
423+All of (a), (b) and (c) are needed for checkpointing to be easy. It might be
460−It might be tractable otherwise, but easy is even better.
424+tractable otherwise, but easy is even better.
461−
425+
462−Moreover, while the surface is most definitely single-threaded,
426+Moreover, while the surface is most definitely single-threaded, we could easily
463−we could easily run multiple threads in multiple inner roads
427+run multiple threads in multiple inner roads (as long as the threads don't have
464−(as long as the threads don't have pointers into each others'
465−memory, which they obviously shouldn't).
428+pointers into each others' memory, which they obviously shouldn't).
466−
429+
467−Moreover, in future, we'll experiment more with adding road
430+Moreover, in future, we'll experiment more with adding road control hints to the
468−control hints to the programmer's toolbox. Reference counting is
431+programmer's toolbox. Reference counting is expensive. We hypothesize that in
469−expensive. We hypothesize that in many - if not most - cases,
470−the programmer can identify procedural structures whose garbage
432+many - if not most - cases, the programmer can identify procedural structures
471−should be discarded in one step by copying the results. Then,
433+whose garbage should be discarded in one step by copying the results. Then,
472−within the procedure, we can switch the allocator into `sand`
434+within the procedure, we can switch the allocator into `sand` mode, and stop
473−mode, and stop tracking references at all.
435+tracking references at all.
474
436
475### u3: rules for C programming
437### u3: rules for C programming
476
438
477−There are two levels at which we program in C: (1) above the
439+There are two levels at which we program in C: (1) above the interpreter; (2)
478−interpreter; (2) within the interpreter or jets. These have
440+within the interpreter or jets. These have separate rules which need to be
479−separate rules which need to be respected.
441+respected.
480
442
481### u3: rules above the interpreter
443### u3: rules above the interpreter
482
444
483−In its relations with Unix, Urbit follows a strict rule of "call
445+In its relations with Unix, Urbit follows a strict rule of "call me, I won't
484−me, I won't call you." We do of course call Unix system calls,
446+call you." We do of course call Unix system calls, but only for the purpose of
485−but only for the purpose of actually computing.
447+actually computing.
486
448
487−Above Urbit, you are in a normal C/Unix programming environment
449+Above Urbit, you are in a normal C/Unix programming environment and can call
488−and can call anything in or out of Urbit. Note that when using
450+anything in or out of Urbit. Note that when using `u3`, you're always on the
489−`u3`, you're always on the surface road, which is not thread-safe
451+surface road, which is not thread-safe by default. Generally speaking, `u3` is
490−by default. Generally speaking, `u3` is designed to support
491−event-oriented, single-threaded programming.
452+designed to support event-oriented, single-threaded programming.
492
453
493−If you need threads which create nouns, you could use
454+If you need threads which create nouns, you could use `u3m_hate()` and
494−`u3m_hate()` and `u3m_love()` to run these threads in subroads.
455+`u3m_love()` to run these threads in subroads. You'd need to make the global
495−You'd need to make the global road pointer, `u3R`, a thread-local
496−variable instead. This seems perfectly practical, but we haven't
456+road pointer, `u3R`, a thread-local variable instead. This seems perfectly
497−done it because we haven't needed to.
457+practical, but we haven't done it because we haven't needed to.
498
458
499### u3: rules within the interpreter
459### u3: rules within the interpreter
500
460
501−Within the interpreter, your code can run either in the surface
461+Within the interpreter, your code can run either in the surface road or in a
502−road or in a deep road. You can test this by testing
462+deep road. You can test this by testing
503
463
504 (&u3H->rod_u == u3R)
464 (&u3H->rod_u == u3R)
505
465
506ie: does the pier's home road equal the current road pointer?
466ie: does the pier's home road equal the current road pointer?
507
467
508−Normally in this context you assume you're obeying the rules of
468+Normally in this context you assume you're obeying the rules of running on an
509−running on an inner road, ie, "deep memory." Remember, however,
469+inner road, ie, "deep memory." Remember, however, that the interpreter *can* run
510−that the interpreter *can* run on surface memory - but anything
511−you can do deep, you can do on the surface. The converse is by
470+on surface memory - but anything you can do deep, you can do on the surface. The
512−no means the case.
471+converse is by no means the case.
513
472
514−In deep memory, think of yourself as if in a signal handler.
473+In deep memory, think of yourself as if in a signal handler. Your execution
515−Your execution context is extremely fragile and may be terminated
474+context is extremely fragile and may be terminated without warning or cleanup at
516−without warning or cleanup at any time (for instance, by `ctrl-c`).
475+any time (for instance, by `ctrl-c`).
517
476
518−For instance, you can't call `malloc` (or C++ `new`) in your C
477+For instance, you can't call `malloc` (or C++ `new`) in your C code, because you
519−code, because you don't have the right to modify data structures
478+don't have the right to modify data structures at the global level, and will
520−at the global level, and will leave them in an inconsistent state
521−if your inner road gets terminated. (Instead, use our drop-in
479+leave them in an inconsistent state if your inner road gets terminated.
522−replacements, `u3a_malloc()`, `u3a_free()`, `u3a_realloc()`.)
480+(Instead, use our drop-in replacements, `u3a_malloc()`, `u3a_free()`,
481+`u3a_realloc()`.)
523
482
524−A good example is the different meaning of `c3_assert()` inside
483+A good example is the different meaning of `c3_assert()` inside and outside the
525−and outside the interpreter. At either layer, you can use
484+interpreter. At either layer, you can use regular assert(), which will just kill
526−regular assert(), which will just kill your process. On the
527−surface, `c3_assert()` will just... kill your process.
485+your process. On the surface, `c3_assert()` will just... kill your process.
528
486
529−In deep execution, `c3_assert()` will issue an exception that
487+In deep execution, `c3_assert()` will issue an exception that queues an error
530−queues an error event, complete with trace stack, on the Arvo
488+event, complete with trace stack, on the Arvo event queue. Let's see how this
531−event queue. Let's see how this happens.
489+happens.
532
490
533### u3: exceptions
491### u3: exceptions
534
492
535−You produce an exception with
493+You produce an exception with
536
494
537 /* u3m_bail(): bail out. Does not return.
495 /* u3m_bail(): bail out. Does not return.
538 **
496 **
@@ -551,82 +509,75 @@ You produce an exception with
551 c3_i
509 c3_i
552 u3m_bail(c3_m how_m);
510 u3m_bail(c3_m how_m);
553
511
554−Broadly speaking, there are two classes of exception: internal
512+Broadly speaking, there are two classes of exception: internal and external. An
555−and external. An external exception begins in a Unix signal
513+external exception begins in a Unix signal handler. An internal exception begins
556−handler. An internal exception begins with a call to longjmp()
557−on the main thread.
514+with a call to longjmp() on the main thread.
558
515
559−There are also two kinds of exception: mild and severe. An
516+There are also two kinds of exception: mild and severe. An external exception is
560−external exception is always severe. An internal exception is
517+always severe. An internal exception is normally mild, but some (like
561−normally mild, but some (like `c3__oops`, generated by
562−`c3_assert()`) are severe.
518+`c3__oops`, generated by `c3_assert()`) are severe.
563
519
564−Either way, exceptions come with a stack trace. The `u3` nock
520+Either way, exceptions come with a stack trace. The `u3` nock interpreter is
565−interpreter is instrumented to retain stack trace hints and
521+instrumented to retain stack trace hints and produce them as a printable
566−produce them as a printable `(list tank)`.
522+`(list tank)`.
567
523
568−Mild exceptions are caught by the first virtualization layer and
524+Mild exceptions are caught by the first virtualization layer and returned to the
569−returned to the caller, following the behavior of the Nock
525+caller, following the behavior of the Nock virtualizer `++mock` (in `hoon.hoon`)
570−virtualizer `++mock` (in `hoon.hoon`)
571
526
572−Severe exceptions, or mild exceptions at the surface, terminate
527+Severe exceptions, or mild exceptions at the surface, terminate the entire
573−the entire execution stack at any depth and send the cumulative
528+execution stack at any depth and send the cumulative trace back to the `u3`
574−trace back to the `u3` caller.
529+caller.
575
530
576−For instance, `vere` uses this trace to construct a `%crud`
531+For instance, `vere` uses this trace to construct a `%crud` event, which conveys
577−event, which conveys our trace back toward the Arvo context where
532+our trace back toward the Arvo context where it crashed. This lets any UI
578−it crashed. This lets any UI component anywhere, even on a
579−remote node, render the stacktrace as a consequence of the user's
533+component anywhere, even on a remote node, render the stacktrace as a
580−action - even if its its direct cause was (for instance) a Unix
534+consequence of the user's action - even if its its direct cause was (for
581−SIGINT or SIGALRM.
535+instance) a Unix SIGINT or SIGALRM.
582
536
583### u3: C structures on the loom
537### u3: C structures on the loom
584
538
585−Normally, all data on the loom is nouns. Sometimes we break this
539+Normally, all data on the loom is nouns. Sometimes we break this rule just a
586−rule just a little, though - eg, in the `u3h` hashtables.
540+little, though - eg, in the `u3h` hashtables.
587
541
588−To point to non-noun C structs on the loom, we use a `u3_post`,
542+To point to non-noun C structs on the loom, we use a `u3_post`, which is just a
589−which is just a loom word offset. A macro lets us declare this
543+loom word offset. A macro lets us declare this as if it was a pointer:
590−as if it was a pointer:
591
544
592 typedef c3_w u3_post;
545 typedef c3_w u3_post;
593 #define u3p(type) u3_post
546 #define u3p(type) u3_post
594
547
595−Some may regard this as clever, others as pointless. Anyway, use
548+Some may regard this as clever, others as pointless. Anyway, use `u3to()` and
596−`u3to()` and `u3of()` to convert to and from pointers.
549+`u3of()` to convert to and from pointers.
597
550
598−When using C structs on the loom - generally a bad idea - make
551+When using C structs on the loom - generally a bad idea - make sure anything
599−sure anything which could be on the surface road is structurally
552+which could be on the surface road is structurally portable, eg, won't change
600−portable, eg, won't change size when the pointer size changes.
601−(Note also: we consider little-endian, rightly or wrongly, to
553+size when the pointer size changes. (Note also: we consider little-endian,
602−have won the endian wars.)
554+rightly or wrongly, to have won the endian wars.)
603
555
604## u3: API overview by prefix
556## u3: API overview by prefix
605
557
606−Let's run through the `u3` modules one by one. All public
558+Let's run through the `u3` modules one by one. All public functions are
607−functions are commented, but the comments may be cryptic.
559+commented, but the comments may be cryptic.
608
560
609### u3m: main control
561### u3m: main control
610
562
611−To start `u3`, run
563+To start `u3`, run
612
564
613 /* u3m_boot(): start the u3 system.
565 /* u3m_boot(): start the u3 system.
614 */
566 */
615 void
567 void
616 u3m_boot(c3_o nuu_o, c3_o bug_o, c3_c* dir_c);
568 u3m_boot(c3_o nuu_o, c3_o bug_o, c3_c* dir_c);
617
569
618−`nuu_o` is `c3y` (yes, `0`) if you're creating a new pier,
570+`nuu_o` is `c3y` (yes, `0`) if you're creating a new pier, `c3n` (no, `1`) if
619−`c3n` (no, `1`) if you're loading an existing one. `bug_o`
571+you're loading an existing one. `bug_o` is `c3y` if you want to test the
620−is `c3y` if you want to test the garbage-collector, `c3n`
621−otherwise. `dir_c` is the directory for the pier files.
572+garbage-collector, `c3n` otherwise. `dir_c` is the directory for the pier files.
622
573
623−`u3m_boot()` expects an `urbit.pill` file to load the kernel
574+`u3m_boot()` expects an `urbit.pill` file to load the kernel from. It will try
624−from. It will try first `$dir/.urb.urbit.pill`, then `U3_LIB`.
575+first `$dir/.urb.urbit.pill`, then `U3_LIB`.
625
576
626−Any significant computation with nouns, certainly anything Turing
577+Any significant computation with nouns, certainly anything Turing complete,
627−complete, should be run (a) virtualized and (b) in an inner road.
578+should be run (a) virtualized and (b) in an inner road. These are slightly
628−These are slightly different things, but at the highest level we
579+different things, but at the highest level we bundle them together for your
629−bundle them together for your convenience, in `u3m_soft()`:
580+convenience, in `u3m_soft()`:
630
581
631 /* u3m_soft(): system soft wrapper. unifies unix and nock errors.
582 /* u3m_soft(): system soft wrapper. unifies unix and nock errors.
632 **
583 **
@@ -635,175 +586,160 @@ bundle them together for your convenience, in `u3m_soft()`:
635 u3_noun
586 u3_noun
636 u3m_soft(c3_w sec_w, u3_funk fun_f, u3_noun arg);
587 u3m_soft(c3_w sec_w, u3_funk fun_f, u3_noun arg);
637
588
638−`sec_w` is the number of seconds to time out the computation.
589+`sec_w` is the number of seconds to time out the computation. `fun_f` is a C
639−`fun_f` is a C function accepting `arg`.
590+function accepting `arg`.
640−
591+
641−The result of `u3m_soft()` is a cell whose head is an atom. If
592+The result of `u3m_soft()` is a cell whose head is an atom. If the head is `%$`
642−the head is `%$` - ie, `0` - the tail is the result of
643−`fun_f(arg)`. Otherwise, the head is a `term` (an atom which is
593+- ie, `0` - the tail is the result of `fun_f(arg)`. Otherwise, the head is a
644−an LSB first string), and the tail is a `(list tank)` (a list of
594+`term` (an atom which is an LSB first string), and the tail is a `(list tank)`
645−`tank` printables - see `++tank` in `hoon.hoon`). Error terms
595+(a list of `tank` printables - see `++tank` in `hoon.hoon`). Error terms should
646−should be the same as the exception terms above.
596+be the same as the exception terms above.
647−
597+
648−If you're confident that your computation won't fail, you can
598+If you're confident that your computation won't fail, you can use
649−use `u3m_soft_sure()`, `u3m_soft_slam()`, or `u3m_soft_nock()`
599+`u3m_soft_sure()`, `u3m_soft_slam()`, or `u3m_soft_nock()` for C functions, Hoon
650−for C functions, Hoon function calls, and Nock invocations.
600+function calls, and Nock invocations. Caution - this returns just the result,
651−Caution - this returns just the result, and asserts globally.
601+and asserts globally.
652−
602+
653−All the `u3m_soft` functions above work *only on the surface*.
603+All the `u3m_soft` functions above work *only on the surface*. Within the
654−Within the surface, virtualize with `u3m_soft_run()`. Note that
604+surface, virtualize with `u3m_soft_run()`. Note that this takes a `fly` (a
655−this takes a `fly` (a namespace gate), thus activating the `11`
605+namespace gate), thus activating the `11` super-operator in the nock
656−super-operator in the nock virtualizer, `++mock`. When actually
657−using the `fly`, call `u3m_soft_esc()`. Don't do either unless
606+virtualizer, `++mock`. When actually using the `fly`, call `u3m_soft_esc()`.
658−you know what you're doing!
607+Don't do either unless you know what you're doing!
659−
608+
660−For descending into a subroad *without* Nock virtualization,
609+For descending into a subroad *without* Nock virtualization, use `u3m_hate()`
661−use `u3m_hate()` and `u3m_love` respectively. Hating enters
610+and `u3m_love` respectively. Hating enters a subroad; loving leaves it, copying
662−a subroad; loving leaves it, copying out a product noun.
611+out a product noun.
663−
612+
664−Other miscellaneous tools in `u3m`: `u3m_file()` loads a Unix
613+Other miscellaneous tools in `u3m`: `u3m_file()` loads a Unix file as a Nock
665−file as a Nock atom; `u3m_water()` measures the boundaries of
614+atom; `u3m_water()` measures the boundaries of the loom in current use (ie,
666−the loom in current use (ie, watermarks); and a variety of
667−prettyprinting routines, none perfect, are available, mainly for
615+watermarks); and a variety of prettyprinting routines, none perfect, are
668−debugging printfs: `u3m_pretty()`, `u3m_p()`, `u3m_tape()` and
616+available, mainly for debugging printfs: `u3m_pretty()`, `u3m_p()`, `u3m_tape()`
669−`u3m_wall()`.
617+and `u3m_wall()`.
670−
618+
671−It's sometimes nice to run a mark-and-sweep garbage collector,
619+It's sometimes nice to run a mark-and-sweep garbage collector, `u3m_grab()`,
672−`u3m_grab()`, which collects the world from a list of roots,
620+which collects the world from a list of roots, and asserts if it finds any leaks
673−and asserts if it finds any leaks or incorrect refcounts. This
674−tool is for debugging and long-term maintenance only; refcounts
621+or incorrect refcounts. This tool is for debugging and long-term maintenance
675−should never err.
622+only; refcounts should never err.
676
623
677### u3j: jets
624### u3j: jets
678
625
679−The jet system, `u3j`, is what makes `u3` and `nock` in any sense
626+The jet system, `u3j`, is what makes `u3` and `nock` in any sense a useful
680−a useful computing environment. Except perhaps `u3a` (there is
627+computing environment. Except perhaps `u3a` (there is really no such thing as a
681−really no such thing as a trivial allocator, though `u3a` is
628+trivial allocator, though `u3a` is dumber than most) - `u3j` is the most
682−dumber than most) - `u3j` is the most interesting code in `u3`.
629+interesting code in `u3`.
683
630
684−Let's consider the minor miracle of driver-to-battery binding
631+Let's consider the minor miracle of driver-to-battery binding which lets `u3j`
685−which lets `u3j` work - and decrement not be `O(n)` - without
632+work - and decrement not be `O(n)` - without violating the precisely defined
686−violating the precisely defined semantics of pure Nock, *ever*.
633+semantics of pure Nock, *ever*.
687
634
688−It's easy to assume that jets represent an architectural coupling
635+It's easy to assume that jets represent an architectural coupling between Hoon
689−between Hoon language semantics and Nock interpreter internals.
636+language semantics and Nock interpreter internals. Indeed such a coupling would
690−Indeed such a coupling would be wholly wrongtious and un-Urbit.
691−But the jet system is not Hoon-specific. It is specific to nock
637+be wholly wrongtious and un-Urbit. But the jet system is not Hoon-specific. It
692−runtime systems that use a design pattern we call a `core`.
638+is specific to nock runtime systems that use a design pattern we call a `core`.
693
639
694#### u3j: core structure
640#### u3j: core structure
695
641
696−A core is no more than a cell `[code data]`, in which a `code` is
642+A core is no more than a cell `[code data]`, in which a `code` is either a Nock
697−either a Nock formula or a cell of `code`s, and `data` is anything.
643+formula or a cell of `code`s, and `data` is anything. In a proper core, the
698−In a proper core, the subject each formula expects is the core
644+subject each formula expects is the core itself.
699−itself.
645+
700−
701−Except for the arbitrary decision to make a core `[code data]`,
646+Except for the arbitrary decision to make a core `[code data]`, (or as we
702−(or as we sometimes say, `[battery payload]`), instead of `[data
647+sometimes say, `[battery payload]`), instead of `[data code]`, any high-level
703−code]`, any high-level language transforming itself to Nock would
648+language transforming itself to Nock would use this design.
704−use this design.
649+
705−
706−So jets are in fact fully general. Broadly speaking, the jet
650+So jets are in fact fully general. Broadly speaking, the jet system works by
707−system works by matching a C *driver* to a battery. When the
651+matching a C *driver* to a battery. When the battery is invoked with Nock
708−battery is invoked with Nock operator `9`, it must be found in
709−associative memory and linked to its driver. Then we link the
652+operator `9`, it must be found in associative memory and linked to its driver.
710−formula axis of the operation (`a` in `[9 a b]`) to a specific
653+Then we link the formula axis of the operation (`a` in `[9 a b]`) to a specific
711function in the driver.
654function in the driver.
712
655
713−To validate this jet binding, we need to know two things. One,
656+To validate this jet binding, we need to know two things. One, we need to know
714−we need to know the C function actually is a perfect semantic
657+the C function actually is a perfect semantic match for the Nock formula. This
715−match for the Nock formula. This can be developed with driver
716−test flags, which work, and locked down with a secure formula
658+can be developed with driver test flags, which work, and locked down with a
717−hash in the driver, which we haven't bothered with just yet.
659+secure formula hash in the driver, which we haven't bothered with just yet. (You
718−(You could also try to develop a formal method for verifying
660+could also try to develop a formal method for verifying that C functions and
719−that C functions and Nock formulas are equivalent, but this is
720−a research problem for the future.)
661+Nock formulas are equivalent, but this is a research problem for the future.)
721−
662+
722−Two, we need to validate that the payload is appropriate for the
663+Two, we need to validate that the payload is appropriate for the battery. We
723−battery. We should note that jets are a Nock feature and have no
664+should note that jets are a Nock feature and have no reference to Hoon. A driver
724−reference to Hoon. A driver which relies on the Hoon type system
725−to only pair it with valid payloads is a broken driver, and
665+which relies on the Hoon type system to only pair it with valid payloads is a
726−breaks the Nock compliance of the system as a whole. So don't.
666+broken driver, and breaks the Nock compliance of the system as a whole. So
727−
667+don't.
668+
728−Now, a casual observer might look at `[battery payload]` and
669+Now, a casual observer might look at `[battery payload]` and expect the simplest
729−expect the simplest case of it to be `[formula subject]`. That
670+case of it to be `[formula subject]`. That is: to execute a simple core whose
730−is: to execute a simple core whose battery is a single formula,
671+battery is a single formula, we compute
731−we compute
732
672
733 nock(+.a -.a)
673 nock(+.a -.a)
734
674
735−Then, naturally, when we go from Hoon or a high-level language
675+Then, naturally, when we go from Hoon or a high-level language containing
736−containing functions down to Nock, `[function arguments]` turns
676+functions down to Nock, `[function arguments]` turns into `[formula subject]`.
737−into `[formula subject]`. This seems like an obvious design, and
677+This seems like an obvious design, and we mention it only because it is
738−we mention it only because it is *completely wrong*.
678+*completely wrong*.
739
679
740Rather, to execute a one-armed core like the above, we run
680Rather, to execute a one-armed core like the above, we run
741
681
742 nock(a -.a)
682 nock(a -.a)
743
683
744−and the normal structure of a `gate`, which is simply Urbitese
684+and the normal structure of a `gate`, which is simply Urbitese for "function,"
745−for "function," is:
685+is:
746
686
747 [formula [sample context]]
687 [formula [sample context]]
748
688
749−where `sample` is Urbitese for "arguments" - and `context`, any
689+where `sample` is Urbitese for "arguments" - and `context`, any Lisper will at
750−Lisper will at once recognize, is Urbitese for "environment."
690+once recognize, is Urbitese for "environment."
751
691
752−To `slam` or call the gate, we simply replace the default sample
692+To `slam` or call the gate, we simply replace the default sample with the
753−with the caller's data, then nock the formula on the entire gate.
693+caller's data, then nock the formula on the entire gate.
754
694
755−What's in the context? Unlike in most dynamic languages, it is
695+What's in the context? Unlike in most dynamic languages, it is not some secret
756−not some secret system-level bag of tricks. Almost always it is
696+system-level bag of tricks. Almost always it is another core. This onion
757−another core. This onion continues until at the bottom, there is
697+continues until at the bottom, there is an atomic constant, conventionally is
758−an atomic constant, conventionally is the kernel version number.
698+the kernel version number.
759
699
760Thus a (highly desirable) `static` core is one of the form
700Thus a (highly desirable) `static` core is one of the form
761
701
762 [battery constant]
702 [battery constant]
763 [battery static-core]
703 [battery static-core]
764
704
765−ie, a solid stack of nested libraries without any dynamic data.
705+ie, a solid stack of nested libraries without any dynamic data. The typical gate
766−The typical gate will thus be, for example,
706+will thus be, for example,
767
707
768 [formula [sample [battery battery battery constant]]]
708 [formula [sample [battery battery battery constant]]]
769
709
770−but we would be most foolish to restrict the jet mechanism to
710+but we would be most foolish to restrict the jet mechanism to cores of this
771−cores of this particular structure. We cannot constrain a
711+particular structure. We cannot constrain a payload to be
772−payload to be `[sample static-core]`, or even `[sample core]`.
712+`[sample static-core]`, or even `[sample core]`. Any such constraint would not
773−Any such constraint would not be rich enough to handle Hoon,
774−let alone other languages.
713+be rich enough to handle Hoon, let alone other languages.
775
714
776#### u3j: jet state
715#### u3j: jet state
777
716
778−There are two fundamental rules of computer science: (1) every
717+There are two fundamental rules of computer science: (1) every system is best
779−system is best understood through its state; (2) less state is
718+understood through its state; (2) less state is better than more state. Sadly, a
780−better than more state. Sadly, a pier has three different jet
781−state systems: `cold`, `warm` and `hot`. It needs all of them.
719+pier has three different jet state systems: `cold`, `warm` and `hot`. It needs
720+all of them.
782
721
783−Hot state is associated with this particular Unix process. The
722+Hot state is associated with this particular Unix process. The persistent pier
784−persistent pier is portable not just between process and process,
723+is portable not just between process and process, but machine and machine or OS
785−but machine and machine or OS and OS. The set of jets loaded
786−into a pier may itself change (in theory, though not in the
724+and OS. The set of jets loaded into a pier may itself change (in theory, though
787−present implementation) during the lifetime of the process. Hot
725+not in the present implementation) during the lifetime of the process. Hot state
788−state is a pure C data structure.
726+is a pure C data structure.
789
727
790−Cold state is associated with the logical execution history of
728+Cold state is associated with the logical execution history of the pier. It
791−the pier. It consists entirely of nouns and ignores restarts.
729+consists entirely of nouns and ignores restarts.
792
730
793−Warm state contains all dependencies between cold and hot
731+Warm state contains all dependencies between cold and hot state. It consists of
794−state. It consists of C structures allocated on the loom.
732+C structures allocated on the loom.
795
733
796−Warm state is purely a function of cold and hot states, and
734+Warm state is purely a function of cold and hot states, and we can wipe and
797−we can wipe and regenerate it at any time. On any restart where
735+regenerate it at any time. On any restart where the hot state might have
798−the hot state might have changed, we clear the warm state
736+changed, we clear the warm state with `u3j_ream()`.
799−with `u3j_ream()`.
800
737
801−There is only one hot state, the global jet dashboard
738+There is only one hot state, the global jet dashboard `u3j_Dash` or `u3D` for
802−`u3j_Dash` or `u3D` for short. In the present implementation,
803−u3D is a static structure not modified at runtime, except for
739+short. In the present implementation, u3D is a static structure not modified at
804−numbering itself on process initialization. This structure -
740+runtime, except for numbering itself on process initialization. This structure -
805−which embeds function pointers to all the jets - is defined
741+which embeds function pointers to all the jets - is defined in `j/tree.c`. The
806−in `j/tree.c`. The data structures:
742+data structures:
807
743
808 /* u3j_harm: driver arm.
744 /* u3j_harm: driver arm.
809 */
745 */
@@ -834,16 +770,16 @@ in `j/tree.c`. The data structures:
834 u3j_core* ray_u; // dynamic driver array
770 u3j_core* ray_u; // dynamic driver array
835 } u3j_dash;
771 } u3j_dash;
836
772
837−Warm and cold state is *per road*. In other words, as we nest
773+Warm and cold state is *per road*. In other words, as we nest roads, we also
838−roads, we also nest jet state. The jet state in the road is:
774+nest jet state. The jet state in the road is:
839
775
840 struct { // jet dashboard
776 struct { // jet dashboard
841 u3p(u3h_root) har_p; // warm state
777 u3p(u3h_root) har_p; // warm state
842 u3_noun das; // cold state
778 u3_noun das; // cold state
843 } jed;
779 } jed;
844
780
845−In case you understand Hoon, `das` (cold state) is a `++dash`,
781+In case you understand Hoon, `das` (cold state) is a `++dash`, and `har_p` (warm
846−and `har_p` (warm state) is a map from battery to `++calx`:
782+state) is a map from battery to `++calx`:
847
783
848 ++ bane ,@tas :: battery name
784 ++ bane ,@tas :: battery name
849 ++ bash ,@uvH :: label hash
785 ++ bash ,@uvH :: label hash
@@ -863,323 +799,288 @@ and `har_p` (warm state) is a map from battery to `++calx`:
863 ++ corp (each core batt) :: parent or static
799 ++ corp (each core batt) :: parent or static
864 ++ dash (map bash clog) :: jet system
800 ++ dash (map bash clog) :: jet system
865
801
866−The driver index `jax` in a `++calx` is an index into `ray_u` in the
802+The driver index `jax` in a `++calx` is an index into `ray_u` in the dashboard -
867−dashboard - ie, a pointer into hot state. This is why the warm
803+ie, a pointer into hot state. This is why the warm state has to be reset when we
868−state has to be reset when we reload the pier in a new process.
804+reload the pier in a new process.
869
805
870−Why is jet state nested? Nock of course is a functional system,
806+Why is jet state nested? Nock of course is a functional system, so as we compute
871−so as we compute we don't explicitly create state. Jet state is
807+we don't explicitly create state. Jet state is an exception to this principle
872−an exception to this principle (which works only because it can't
873−be semantically detected from Nock/Hoon) - but it can't violate
808+(which works only because it can't be semantically detected from Nock/Hoon) -
874−the fundamental rules of the allocation system.
809+but it can't violate the fundamental rules of the allocation system.
875
810
876−For instance, when we're on an inner road, we can't allocate on
811+For instance, when we're on an inner road, we can't allocate on an outer road,
877−an outer road, or point from an outer road to an inner. So if we
812+or point from an outer road to an inner. So if we learn something - like a
878−learn something - like a mapping from battery to jet - in the
879−inner road, we have to keep it in the inner road.
813+mapping from battery to jet - in the inner road, we have to keep it in the inner
814+road.
880
815
881−Mitigating this problem, when we leave an inner road (with
816+Mitigating this problem, when we leave an inner road (with `u3m_love()`), we
882−`u3m_love()`), we call `u3j_reap()` to promote jet information in
817+call `u3j_reap()` to promote jet information in the dying road. Reaping promotes
883−the dying road. Reaping promotes anything we've learned about
884−any battery that either (a) already existed in the outer road, or
818+anything we've learned about any battery that either (a) already existed in the
885−(b) is being saved to the outer road.
819+outer road, or (b) is being saved to the outer road.
886
820
887#### u3j: jet binding
821#### u3j: jet binding
888
822
889−Jet binding starts with a `%fast` hint. (In Hoon, this is
823+Jet binding starts with a `%fast` hint. (In Hoon, this is produced by the runes
890−produced by the runes `~%`, for the general case, or `~/`
891−for simple functions.) To bind a jet, execute a formula of the
824+`~%`, for the general case, or `~/` for simple functions.) To bind a jet,
892−form:
825+execute a formula of the form:
893
826
894 [10 [%fast clue-formula] core-formula]
827 [10 [%fast clue-formula] core-formula]
895
828
896−`core-formula` assembles the core to be jet-propelled.
829+`core-formula` assembles the core to be jet-propelled. `clue-formula` produces
897−`clue-formula` produces the hint information, or `++clue`
898−above, which we want to annotate it with.
830+the hint information, or `++clue` above, which we want to annotate it with.
899
831
900A clue is a triple of name, parent, and hooks:
832A clue is a triple of name, parent, and hooks:
901
833
902 ++ clue (trel chum nock (list (pair term nock)))
834 ++ clue (trel chum nock (list (pair term nock)))
903
835
904−The name, or `++chum`, has a bunch of historical structure which
836+The name, or `++chum`, has a bunch of historical structure which we don't need
905−we don't need (cleaning these things up is tricky), but just gets
837+(cleaning these things up is tricky), but just gets flattened into a term.
906−flattened into a term.
907
838
908−The parent axis is a nock formula, but always reduces to a simple
839+The parent axis is a nock formula, but always reduces to a simple axis, which is
909−axis, which is the address of this core's *parent*. Consider
840+the address of this core's *parent*. Consider again an ordinary gate
910−again an ordinary gate
911
841
912 [formula [sample context]]
842 [formula [sample context]]
913
843
914−Typically the `context` is itself a library core, which itself
844+Typically the `context` is itself a library core, which itself has a jet
915−has a jet binding. If so, the parent axis of this gate is `7`.
845+binding. If so, the parent axis of this gate is `7`.
916
846
917−If the parent is already bound - and the parent *must* be already
847+If the parent is already bound - and the parent *must* be already bound, in this
918−bound, in this road or a road containing it - we can hook this core
848+road or a road containing it - we can hook this core bottom-up into a tree
919−bottom-up into a tree hierarchy. Normally the child core is
920−produced by an arm of the parent core, so this is not a problem -
849+hierarchy. Normally the child core is produced by an arm of the parent core, so
921−we wouldn't have the child if we hadn't already made the parent.
850+this is not a problem - we wouldn't have the child if we hadn't already made the
851+parent.
922
852
923−The clue also contains a list of *hooks*, named nock formulas on
853+The clue also contains a list of *hooks*, named nock formulas on the core.
924−the core. Usually these are arms, but they need not be. The
854+Usually these are arms, but they need not be. The point is that we often want to
925−point is that we often want to call a core from C, in a situation
926−where we have no type or other source information. A common case
855+call a core from C, in a situation where we have no type or other source
927−of this is a complex system in which we're mixing functions which
856+information. A common case of this is a complex system in which we're mixing
928−are jet-propelled with functions that aren't.
857+functions which are jet-propelled with functions that aren't.
929
858
930−In any case, all the information in the `%fast` hint goes to
859+In any case, all the information in the `%fast` hint goes to `u3j_mine()`, which
931−`u3j_mine()`, which registers the battery in cold state (`das` in
860+registers the battery in cold state (`das` in `jed` in `u3R`), then warm state
932−`jed` in `u3R`), then warm state (`har_p` in `jed`).
861+(`har_p` in `jed`).
933
862
934−It's essential to understand that the `%fast` hint has to be,
863+It's essential to understand that the `%fast` hint has to be, well, fast -
935−well, fast - because we apply it whenever we build a core. For
864+because we apply it whenever we build a core. For instance, if the core is a
936−instance, if the core is a Hoon gate - a function - we will call
937−`u3j_mine` every time the function is called.
865+Hoon gate - a function - we will call `u3j_mine` every time the function is
866+called.
938
867
939### u3j: the cold jet dashboard
868### u3j: the cold jet dashboard
940
869
941−For even more fun, the jet tree is not actually a tree of
870+For even more fun, the jet tree is not actually a tree of batteries. It's a tree
942−batteries. It's a tree of battery *labels*, where a label is
943−an [axis term] path from the root of the tree. (At the root,
871+of battery *labels*, where a label is an \[axis term\] path from the root of the
944−if the core pattern is always followed properly, is a core whose
872+tree. (At the root, if the core pattern is always followed properly, is a core
945−payload is an atomic constant, conventionally the Hoon version.)
873+whose payload is an atomic constant, conventionally the Hoon version.)
946
874
947−Under each of these labels, it's normal to have an arbitrary
875+Under each of these labels, it's normal to have an arbitrary number of different
948−number of different Nock batteries (not just multiple copies
876+Nock batteries (not just multiple copies of the same noun, a situation we *do*
949−of the same noun, a situation we *do* strive to avoid). For
950−instance, one might be compiled with debugging hints, one not.
877+strive to avoid). For instance, one might be compiled with debugging hints, one
878+not.
951
879
952−We might even have changed the semantics of the battery without
880+We might even have changed the semantics of the battery without changing the
953−changing the label - so long as those semantics don't invalidate
881+label - so long as those semantics don't invalidate any attached driver.
954−any attached driver.
955
882
956−For instance, it's normal to have two equivalent Nock batteries
883+For instance, it's normal to have two equivalent Nock batteries at the same time
957−at the same time in one pier: one battery compiled with
884+in one pier: one battery compiled with debugging hints, one not.
958−debugging hints, one not.
959
885
960−Rather, the jet tree is a semantic hierarchy. The root of the
886+Rather, the jet tree is a semantic hierarchy. The root of the hierarchy is a
961−hierarchy is a constant, by convention the Hoon kernel version
887+constant, by convention the Hoon kernel version because any normal jet-propelled
962−because any normal jet-propelled core has, at the bottom of its
963−onion of libraries, the standard kernel. Thus if the core is
888+core has, at the bottom of its onion of libraries, the standard kernel. Thus if
889+the core is
964
890
965 [foo-battery [bar-battery [moo-battery 164]]]
891 [foo-battery [bar-battery [moo-battery 164]]]
966
892
967−we can reverse the nesting to construct a hierarchical core
893+we can reverse the nesting to construct a hierarchical core path. The static
968−path. The static core
894+core
969
895
970 164/moo/bar/foo
896 164/moo/bar/foo
971
897
972−extends the static core `164/moo/bar` by wrapping the `foo`
898+extends the static core `164/moo/bar` by wrapping the `foo` battery (ie, in
973−battery (ie, in Hoon, `|%`) around it. With the core above,
899+Hoon, `|%`) around it. With the core above, you can compute `foo` stuff, `bar`
974−you can compute `foo` stuff, `bar` stuff, and `moo` stuff.
975−Rocket science, not.
900+stuff, and `moo` stuff. Rocket science, not.
976−
901+
977−Not all cores are static, of course - they may contain live data,
902+Not all cores are static, of course - they may contain live data, like the
978−like the sample in a gate (ie, argument to a function). Once
903+sample in a gate (ie, argument to a function). Once again, it's important to
979−again, it's important to remember that we track jet bindings not
904+remember that we track jet bindings not by the core, which may not be static,
980−by the core, which may not be static, but by the battery, which
981−is always static.
905+but by the battery, which is always static.
982−
906+
983−(And if you're wondering how we can use a deep noun like a Nock
907+(And if you're wondering how we can use a deep noun like a Nock formula or
984−formula or battery as a key in a key-value table, remember
908+battery as a key in a key-value table, remember `mug_w`, the lazily computed
985−`mug_w`, the lazily computed short hash, in all boxed nouns.)
909+short hash, in all boxed nouns.)
986−
910+
987−In any case, `das`, the dashboard, is a map from `bash` to jet
911+In any case, `das`, the dashboard, is a map from `bash` to jet location record
988−location record (`++clog`). A `clog` in turn contains two kinds
912+(`++clog`). A `clog` in turn contains two kinds of information: the `++cope`, or
989−of information: the `++cope`, or per-location noun; and a map of
990−batteries to a per-battery `++club`.
913+per-location noun; and a map of batteries to a per-battery `++club`.
991−
914+
992−The `cope` is a triple of `++bane` (battery name, right now just
915+The `cope` is a triple of `++bane` (battery name, right now just a `term`);
993−a `term`); `++axis`, the axis, within *this* core, of the parent;
916+`++axis`, the axis, within *this* core, of the parent; and `(each bash noun)`,
994−and `(each bash noun)`, which is either `[0 bash]` if the parent
917+which is either `[0 bash]` if the parent is another core, or `[1 noun]`, for the
995−is another core, or `[1 noun]`, for the constant noun (like
996−`164`) if there is no parent core.
918+constant noun (like `164`) if there is no parent core.
997−
919+
998−A `bash` is just the noun hash (`++sham`) of a `cope`, which
920+A `bash` is just the noun hash (`++sham`) of a `cope`, which uniquely expresses
999−uniquely expresses the battery's hierarchical location without
921+the battery's hierarchical location without depending on the actual formulas.
1000−depending on the actual formulas.
922+
1001−
1002−The `club` contains a `++corp`, which we use to actually validate
923+The `club` contains a `++corp`, which we use to actually validate the core.
1003−the core. Obviously jet execution has to be perfectly compatible
924+Obviously jet execution has to be perfectly compatible with Nock. We search on
1004−with Nock. We search on the battery, but getting the battery
1005−right is not enough - a typical battery is dependent on its
925+the battery, but getting the battery right is not enough - a typical battery is
1006−context. For example, your jet-propelled library function is
926+dependent on its context. For example, your jet-propelled library function is
1007−very likely to call `++dec` or other advanced kernel technology.
927+very likely to call `++dec` or other advanced kernel technology. If you've
1008−If you've replaced the kernel in your context with something
928+replaced the kernel in your context with something else, we need to detect this
1009−else, we need to detect this and not run the jet.
929+and not run the jet.
1010−
930+
1011−There are two cases for a jet-propelled core - either the entire
931+There are two cases for a jet-propelled core - either the entire core is a
1012−core is a static constant, or it isn't. Hence the definition
932+static constant, or it isn't. Hence the definition of `corp`:
1013−of `corp`:
1014
933
1015 ++ corp (each core batt) :: parent or static
934 ++ corp (each core batt) :: parent or static
1016
935
1017−Ie, a `corp` is `[0 core]` or `[1 batt]`. If it's static -
936+Ie, a `corp` is `[0 core]` or `[1 batt]`. If it's static - meaning that the jet
1018−meaning that the jet only works with one specific core, ie, the
937+only works with one specific core, ie, the parent axis of each location in the
1019−parent axis of each location in the hierarchy is `3` - we can
1020−validate with a single comparison. Otherwise, we have to recurse
938+hierarchy is `3` - we can validate with a single comparison. Otherwise, we have
1021−upward by checking the parent.
939+to recurse upward by checking the parent.
1022
940
1023−Note that there is at present no way to force a jet to depend on
941+Note that there is at present no way to force a jet to depend on static *data*.
1024−static *data*.
1025
942
1026### u3j: the warm jet dashboard
943### u3j: the warm jet dashboard
1027
944
1028−We don't use the cold state to match jets as we call them. We
945+We don't use the cold state to match jets as we call them. We use the cold state
1029−use the cold state to register jets as we find them, and also to
946+to register jets as we find them, and also to rebuild the warm state after the
1030−rebuild the warm state after the hot state is reset.
947+hot state is reset.
1031
948
1032−What we actually use at runtime is the warm state, `jed->har_p`,
949+What we actually use at runtime is the warm state, `jed->har_p`, which is a
1033−which is a `u3h` (built-in hashtable), allocated on the loom,
950+`u3h` (built-in hashtable), allocated on the loom, from battery to `++calx`.
1034−from battery to `++calx`.
1035
951
1036−A `calx` is a triple of a `++calf`, a `[bash cope]` cell, and a
952+A `calx` is a triple of a `++calf`, a `[bash cope]` cell, and a `club`. The
1037−`club`. The latter two are all straight from cold state.
953+latter two are all straight from cold state.
1038
954
1039−The `calf` contains warm data dependent on hot state. It's a
955+The `calf` contains warm data dependent on hot state. It's a quadruple: of
1040−quadruple: of `jax`, the hot driver index (in `ray_u` in
956+`jax`, the hot driver index (in `ray_u` in `u3j_dash`); `hap`, a table from arm
1041−`u3j_dash`); `hap`, a table from arm axis (ie, the axis of each
1042−formula within the battery) to driver arm index (into `arm_u` in
957+axis (ie, the axis of each formula within the battery) to driver arm index (into
1043−`u3j_core`); `lab`, the complete label path; and `jit`, any
958+`arm_u` in `u3j_core`); `lab`, the complete label path; and `jit`, any other
1044−other dynamic data that may speed up execution.
959+dynamic data that may speed up execution.
1045
960
1046−We construct `hap`, when we create the calx, by iterating through
961+We construct `hap`, when we create the calx, by iterating through the arms
1047−the arms registered in the `u3j_core`. Note the way a `u3j_harm`
962+registered in the `u3j_core`. Note the way a `u3j_harm` declares itself, with
1048−declares itself, with the string `fcs_c` which can contain either
963+the string `fcs_c` which can contain either an axis or a name. Most jetted cores
1049−an axis or a name. Most jetted cores are of course gates, which
1050−have one formula at one axis within the core: `fcs_c` is `".3"`.
964+are of course gates, which have one formula at one axis within the core: `fcs_c`
965+is `".3"`.
1051
966
1052−But we do often have fast cores with more complex arm structure,
967+But we do often have fast cores with more complex arm structure, and it would be
1053−and it would be sad to have to manage their axes by hand. To use
968+sad to have to manage their axes by hand. To use an `fcs_c` with a named arm,
1054−an `fcs_c` with a named arm, it's sufficient to make sure the
1055−name is bound to a formula `[0 axis]` in the hook table.
969+it's sufficient to make sure the name is bound to a formula `[0 axis]` in the
970+hook table.
1056
971
1057−`jit`, as its name suggests, is a stub where any sort of
972+`jit`, as its name suggests, is a stub where any sort of optimization data
1058−optimization data computed on battery registration might go. To
973+computed on battery registration might go. To use it, fill in the `_cj_jit()`
1059−use it, fill in the `_cj_jit()` function.
974+function.
1060
975
1061### u3j: the hot dashboard
976### u3j: the hot dashboard
1062
977
1063−Now it should be easy to see how we actually invoke jets. Every
978+Now it should be easy to see how we actually invoke jets. Every time we run a
1064−time we run a nock `9` instruction (pretty often, obviously), we
979+nock `9` instruction (pretty often, obviously), we have a core and an axis. We
1065−have a core and an axis. We pass these to `u3j_kick()`, which
1066−will try to execute them.
980+pass these to `u3j_kick()`, which will try to execute them.
1067
981
1068−Because nouns with a reference count of 1 are precious,
982+Because nouns with a reference count of 1 are precious, `u3j_kick()` has a
1069−`u3j_kick()` has a tricky reference control definition. It
983+tricky reference control definition. It reserves the right to return `u3_none`
1070−reserves the right to return `u3_none` in the case where there is
1071−no driver, or the driver does not apply for this case; in this
984+in the case where there is no driver, or the driver does not apply for this
1072−case, it retains argument `cor`. If it succeeds, though, it
985+case; in this case, it retains argument `cor`. If it succeeds, though, it
1073transfers `cor`.
986transfers `cor`.
1074
987
1075−`u3j_kick()` searches for the battery (always the head of the
988+`u3j_kick()` searches for the battery (always the head of the core, of course)
1076−core, of course) in the hot dashboard. If the battery is
1077−registered, it searches for the axis in `hap` in the `calx`.
989+in the hot dashboard. If the battery is registered, it searches for the axis in
1078−If it exists, the core matches a driver and the driver jets this
990+`hap` in the `calx`. If it exists, the core matches a driver and the driver jets
1079−arm. If not, we return `u3_none`.
991+this arm. If not, we return `u3_none`.
1080
992
1081−Otherwise, we call `fun_f` in our `u3j_harm`. This obeys the
993+Otherwise, we call `fun_f` in our `u3j_harm`. This obeys the same protocol as
1082−same protocol as `u3j_kick()`; it can refuse to function by
1083−returning `u3_none`, or consume the noun.
994+`u3j_kick()`; it can refuse to function by returning `u3_none`, or consume the
995+noun.
1084
996
1085−Besides the actual function pointer `fun_f`, we have some flags
997+Besides the actual function pointer `fun_f`, we have some flags in the
1086−in the `u3j_harm` which tell us how to call the arm function.
998+`u3j_harm` which tell us how to call the arm function.
1087
999
1088−If `ice` is yes (`&`, `0`), the jet is known to be perfect and we
1000+If `ice` is yes (`&`, `0`), the jet is known to be perfect and we can just trust
1089−can just trust the product of `fun_f`. Otherwise, we need to run
1001+the product of `fun_f`. Otherwise, we need to run *both* the Nock arm and
1090−*both* the Nock arm and `fun_f`, and compare their results.
1002+`fun_f`, and compare their results.
1091
1003
1092−(Note that while executing the C side of this test, we have to
1004+(Note that while executing the C side of this test, we have to set `ice` to yes;
1093−set `ice` to yes; on the Nock side, we have to set `liv` to no.
1005+on the Nock side, we have to set `liv` to no. Otherwise, many non-exponential
1094−Otherwise, many non-exponential functions become exponential.
1095−When auto-testing jets in this way, the principle is that the
1006+functions become exponential. When auto-testing jets in this way, the principle
1096−test is on the outermost layer of recursion.)
1007+is that the test is on the outermost layer of recursion.)
1097
1008
1098−(Note also that anyone who multi-threads this execution
1009+(Note also that anyone who multi-threads this execution environment has a slight
1099−environment has a slight locking problem with these flags if arm
1010+locking problem with these flags if arm testing is multi-threaded.)
1100−testing is multi-threaded.)
1101
1011
1102−If `tot` is yes, (`&`, `0`), the arm function is *total* and has
1012+If `tot` is yes, (`&`, `0`), the arm function is *total* and has to return
1103−to return properly (though it can still return *u3_none*).
1013+properly (though it can still return \*u3\_none*). Otherwise, it is*partial\*
1104−Otherwise, it is *partial* and can `u3_cm_bail()` out with
1105−c3__punt. This feature has a cost: the jet runs in a subroad.
1014+and can `u3_cm_bail()` out with c3\_\_punt. This feature has a cost: the jet
1015+runs in a subroad.
1106
1016
1107Finally, if `liv` is no (`|`, 1), the jet is off and doesn't run.
1017Finally, if `liv` is no (`|`, 1), the jet is off and doesn't run.
1108
1018
1109−It should be easy to see how the tree of cores gets declared -
1019+It should be easy to see how the tree of cores gets declared - precisely, in
1110−precisely, in `j/dash.c`. We declare the hierarchy as a tree
1020+`j/dash.c`. We declare the hierarchy as a tree of `u3j_core` structures, each of
1111−of `u3j_core` structures, each of which comes with a static list
1112−of arms `arm_u` and sub-cores `dev_u`.
1021+which comes with a static list of arms `arm_u` and sub-cores `dev_u`.
1113
1022
1114−In `u3j_boot()`, we traverse the hierarchy, fill in parent
1023+In `u3j_boot()`, we traverse the hierarchy, fill in parent pointers `par_u`, and
1115−pointers `par_u`, and enumerate all `u3j_core` structures
1116−into a single flat array `u3j_dash.ray_u`. Our hot state
1024+enumerate all `u3j_core` structures into a single flat array `u3j_dash.ray_u`.
1117−then appears ready for action.
1025+Our hot state then appears ready for action.
1118
1026
1119### u3j: jet functions
1027### u3j: jet functions
1120
1028
1121−At present, all drivers are compiled statically into `u3`. This is
1029+At present, all drivers are compiled statically into `u3`. This is not a
1122−not a long-term permanent solution or anything. However, it will
1030+long-term permanent solution or anything. However, it will always be the case
1123−always be the case with a certain amount of core functionality.
1031+with a certain amount of core functionality.
1124−
1032+
1125−For instance, there are some jet functions that we need to call
1033+For instance, there are some jet functions that we need to call as part of
1126−as part of loading the Arvo kernel - like `++cue` to unpack a
1034+loading the Arvo kernel - like `++cue` to unpack a noun from an atom. And
1127−noun from an atom. And obviously it makes sense, when jets are
1035+obviously it makes sense, when jets are significant enough to compile into `u3`,
1128−significant enough to compile into `u3`, to export their symbols
1129−in headers and the linker.
1036+to export their symbols in headers and the linker.
1130−
1037+
1131−There are three interface prefixes for standard jet functions:
1038+There are three interface prefixes for standard jet functions: `u3k`, `u3q`, and
1132−`u3k`, `u3q`, and `u3w`. All jets have `u3w` interfaces; most
1039+`u3w`. All jets have `u3w` interfaces; most have `u3q`; some have `u3k`. Of
1133−have `u3q`; some have `u3k`. Of course the actual logic is
1040+course the actual logic is shared.
1134−shared.
1041+
1135−
1136−`u3w` interfaces use the same protocol as `fun_f` above: the
1042+`u3w` interfaces use the same protocol as `fun_f` above: the caller passes the
1137−caller passes the entire core, which is retained if the function
1043+entire core, which is retained if the function returns `u3_none`, transferred
1138−returns `u3_none`, transferred otherwise. Why? Again, use
1139−counts of 1 are special and precious for performance hackers.
1044+otherwise. Why? Again, use counts of 1 are special and precious for performance
1140−
1045+hackers.
1046+
1141−`u3q` interfaces break the core into C arguments, *retain* noun
1047+`u3q` interfaces break the core into C arguments, *retain* noun arguments, and
1142−arguments, and *transfer* noun returns. `u3k` interfaces are the
1048+*transfer* noun returns. `u3k` interfaces are the same, except with more use of
1143−same, except with more use of `u3_none` and other simple C
1144−variations on the Hoon original, but *transfer* both arguments
1049+`u3_none` and other simple C variations on the Hoon original, but *transfer*
1145−and returns. Generally, `u3k` are most convenient for new code.
1050+both arguments and returns. Generally, `u3k` are most convenient for new code.
1146−
1051+
1147−Following `u3k/q/w` is `[a-f]`, corresponding to the 6 logical
1052+Following `u3k/q/w` is `[a-f]`, corresponding to the 6 logical tiers of the
1148−tiers of the kernel, or `g` for user-level jets. Another letter
1053+kernel, or `g` for user-level jets. Another letter is added for functions within
1149−is added for functions within subcores. The filename, under
1150−`j/`, follows the tier and the function name.
1054+subcores. The filename, under `j/`, follows the tier and the function name.
1151−
1055+
1152−For instance, `++add` is `u3wa_add(cor)`, `u3qa_add(a, b)`, or
1056+For instance, `++add` is `u3wa_add(cor)`, `u3qa_add(a, b)`, or `u3ka_add(a, b)`,
1153−`u3ka_add(a, b)`, in `j/a/add.c`. `++get` in `++by` is
1154−`u3wdb_get(cor)`, `u3kdb_get(a, b)`, etc, in `j/d/by_get.c`.
1057+in `j/a/add.c`. `++get` in `++by` is `u3wdb_get(cor)`, `u3kdb_get(a, b)`, etc,
1155−
1058+in `j/d/by_get.c`.
1059+
1156−For historical reasons, all internal jet code in `j/[a-f]`
1060+For historical reasons, all internal jet code in `j/[a-f]` *retains* noun
1157−*retains* noun arguments, and *transfers* noun results. Please
1061+arguments, and *transfers* noun results. Please do not do this in new `g` jets!
1158−do not do this in new `g` jets! The new standard protocol is to
1159−transfer both arguments and results.
1062+The new standard protocol is to transfer both arguments and results.
1160
1063
1161### u3a: allocation functions
1064### u3a: allocation functions
1162
1065
1163−`u3a` allocates on the current road (u3R). Its internal
1066+`u3a` allocates on the current road (u3R). Its internal structures are
1164−structures are uninteresting and typical of a naive allocator.
1067+uninteresting and typical of a naive allocator.
1165
1068
1166−The two most-used `u3a` functions are `u3a_gain()` to add a
1069+The two most-used `u3a` functions are `u3a_gain()` to add a reference count, and
1167−reference count, and `u3a_lose()` to release one (and free the
1070+`u3a_lose()` to release one (and free the noun, if the use count is zero). For
1168−noun, if the use count is zero). For convenience, `u3a_gain()`
1169−returns its argument. The pair are generally abbreviated with
1071+convenience, `u3a_gain()` returns its argument. The pair are generally
1170−the macros `u3k()` and `u3z()` respectively.
1072+abbreviated with the macros `u3k()` and `u3z()` respectively.
1171
1073
1172−Normally we create nouns through `u3i` functions, and don't call
1074+Normally we create nouns through `u3i` functions, and don't call the `u3a`
1173−the `u3a` allocators directly. But if you do:
1075+allocators directly. But if you do:
1174
1076
1175−One, there are *two* sets of allocators: the word-aligned
1077+One, there are *two* sets of allocators: the word-aligned allocators and the
1176−allocators and the fully-aligned (ie, malloc compatible)
1177−allocators. For instance, on a typical OS X setup, malloc
1078+fully-aligned (ie, malloc compatible) allocators. For instance, on a typical OS
1178−produces 16-byte aligned results - needed for some SSE
1079+X setup, malloc produces 16-byte aligned results - needed for some SSE
1179instructions.
1080instructions.
1180
1081
1181−These allocators are *not compatible*. For 32-bit alignment
1082+These allocators are *not compatible*. For 32-bit alignment as used in nouns,
1182−as used in nouns, call
1083+call
1183
1084
1184 /* u3a_walloc(): allocate storage measured in words.
1085 /* u3a_walloc(): allocate storage measured in words.
1185 */
1086 */
@@ -1223,48 +1124,41 @@ For full alignment, call:
1223 void
1124 void
1224 u3a_free2(void* tox_v, size_t siz_i);
1125 u3a_free2(void* tox_v, size_t siz_i);
1225
1126
1226−There are also a set of special-purpose allocators for building
1127+There are also a set of special-purpose allocators for building atoms. When
1227−atoms. When building atoms, please remember that it's incorrect
1128+building atoms, please remember that it's incorrect to have a high 0 word - the
1228−to have a high 0 word - the word length in the atom structure
1229−must be strictly correct.
1129+word length in the atom structure must be strictly correct.
1230−
1130+
1231−Of course, we don't always know how large our atom will be.
1131+Of course, we don't always know how large our atom will be. Therefore, the
1232−Therefore, the standard way of building large atoms is to
1132+standard way of building large atoms is to allocate a block of raw space with
1233−allocate a block of raw space with `u3a_slab()`, then chop off
1234−the end with `u3a_malt()` (which does the measuring itself)
1133+`u3a_slab()`, then chop off the end with `u3a_malt()` (which does the measuring
1235−or `u3a_mint()` in case you've measured it yourself.
1134+itself) or `u3a_mint()` in case you've measured it yourself.
1236−
1135+
1237−Once again, *do not call `malloc()`* (or C++ `new`) within any
1136+Once again, *do not call `malloc()`* (or C++ `new`) within any code that may be
1238−code that may be run within a jet. This will cause rare sporadic
1137+run within a jet. This will cause rare sporadic corruption when we interrupt
1239−corruption when we interrupt execution within a `malloc()`. We'd
1240−just override the symbol, but `libuv` uses `malloc()` across
1138+execution within a `malloc()`. We'd just override the symbol, but `libuv` uses
1241−threads within its own synchronization primitives - for this to
1139+`malloc()` across threads within its own synchronization primitives - for this
1242−work with `u3a_malloc()`, we'd have to introduce our own locks on
1140+to work with `u3a_malloc()`, we'd have to introduce our own locks on the
1243−the surface-level road (which might be a viable solution).
1141+surface-level road (which might be a viable solution).
1244
1142
1245### u3n: nock execution
1143### u3n: nock execution
1246
1144
1247−The `u3n` routines execute Nock itself. On the inside, they have
1145+The `u3n` routines execute Nock itself. On the inside, they have a surprising
1248−a surprising resemblance to the spec proper (the only interesting
1146+resemblance to the spec proper (the only interesting detail is how we handle
1249−detail is how we handle tail-call elimination) and are, as one
1147+tail-call elimination) and are, as one would expect, quite slow. (There is no
1250−would expect, quite slow. (There is no such thing as a fast tree
1251−interpreter.)
1148+such thing as a fast tree interpreter.)
1252−
1149+
1253−There is only one Nock, but there are lots of ways to call it.
1150+There is only one Nock, but there are lots of ways to call it. (Remember that
1254−(Remember that all `u3n` functions *transfer* C arguments and
1151+all `u3n` functions *transfer* C arguments and returns.)
1255−returns.)
1152+
1256−
1257−The simplest interpreter, `u3n_nock_on(u3_noun bus, u3_noun fol)`
1153+The simplest interpreter, `u3n_nock_on(u3_noun bus, u3_noun fol)` invokes Nock
1258−invokes Nock on `bus` (the subject) and `fol` (the formula).
1154+on `bus` (the subject) and `fol` (the formula). (Why is it`[subject formula]`,
1259−(Why is it`[subject formula]`, not `[formula subject]`? The same
1260−reason `0` is true and `1` is false.)
1155+not `[formula subject]`? The same reason `0` is true and `1` is false.)
1261−
1156+
1262−A close relative is `u3n_slam_on(u3_noun gat, u3_noun sam)`,
1157+A close relative is `u3n_slam_on(u3_noun gat, u3_noun sam)`, which slams a
1263−which slams a *gate* (`gat`) on a sample (`sam`). (In a normal
1158+*gate* (`gat`) on a sample (`sam`). (In a normal programming language which
1264−programming language which didn't talk funny and was retarded,
1265−`u3n_slam_on()` would call a function on an argument.) We could
1159+didn't talk funny and was retarded, `u3n_slam_on()` would call a function on an
1266−write it most simply as:
1160+argument.) We could write it most simply as:
1267−
1161+
1268 u3_noun
1162 u3_noun
1269 u3n_slam_on(u3_noun gat, u3_noun sam)
1163 u3n_slam_on(u3_noun gat, u3_noun sam)
1270 {
1164 {
@@ -1276,9 +1170,9 @@ write it most simply as:
1276 return pro;
1170 return pro;
1277 }
1171 }
1278
1172
1279−Simpler is `u3n_kick_on(u3_noun gat)`, which slams a gate (or,
1173+Simpler is `u3n_kick_on(u3_noun gat)`, which slams a gate (or, more generally, a
1280−more generally, a *trap* - because sample structure is not even
1174+*trap* - because sample structure is not even needed here) without changing its
1281−needed here) without changing its sample:
1175+sample:
1282
1176
1283 u3_noun
1177 u3_noun
1284 u3n_kick_on(u3_noun gat, u3_noun sam)
1178 u3n_kick_on(u3_noun gat, u3_noun sam)
@@ -1286,37 +1180,34 @@ needed here) without changing its sample:
1286 return u3n_nock_on(gat, u3k(u3h(gat)));
1180 return u3n_nock_on(gat, u3k(u3h(gat)));
1287 }
1181 }
1288
1182
1289−The `_on` functions in `u3n` are all defined as pure Nock. But
1183+The `_on` functions in `u3n` are all defined as pure Nock. But actually, even
1290−actually, even though we say we don't extend Nock, we do. But we
1184+though we say we don't extend Nock, we do. But we don't. But we do.
1291−don't. But we do.
1292
1185
1293−Note that `u3` has a well-developed error handling system -
1186+Note that `u3` has a well-developed error handling system - `u3m_bail()` to
1294−`u3m_bail()` to throw an exception, `u3m_soft_*` to catch one.
1187+throw an exception, `u3m_soft_*` to catch one. But Nock has no exception model
1295−But Nock has no exception model at all. That's okay - all it
1296−means if that if an `_on` function bails, the exception is an
1188+at all. That's okay - all it means if that if an `_on` function bails, the
1297−exception in the caller.
1189+exception is an exception in the caller.
1298
1190
1299−However, `u3`'s exception handling happens to match a convenient
1191+However, `u3`'s exception handling happens to match a convenient virtual
1300−virtual super-Nock in `hoon.hoon`, the infamous `++mock`. Of
1192+super-Nock in `hoon.hoon`, the infamous `++mock`. Of course, Nock is slow, and
1301−course, Nock is slow, and `mock` is Nock in Nock, so it is
1302−(logically) super-slow. Then again, so is decrement.
1193+`mock` is Nock in Nock, so it is (logically) super-slow. Then again, so is
1194+decrement.
1303
1195
1304−With the power of `u3`, we nest arbitrary layers of `mock`
1196+With the power of `u3`, we nest arbitrary layers of `mock` without any
1305−without any particular performance cost. Moreover, we simply
1197+particular performance cost. Moreover, we simply treat Nock proper as a special
1306−treat Nock proper as a special case of `mock`. (More precisely,
1307−the internal VM loop is `++mink` and the error compiler is
1198+case of `mock`. (More precisely, the internal VM loop is `++mink` and the error
1308−`++mook`. But we call the whole sandbox system `mock`.)
1199+compiler is `++mook`. But we call the whole sandbox system `mock`.)
1309
1200
1310−The nice thing about `mock` functions is that (by executing
1201+The nice thing about `mock` functions is that (by executing within
1311−within `u3m_soft_run()`, which as you may recall uses a nested
1202+`u3m_soft_run()`, which as you may recall uses a nested road) they provide both
1312−road) they provide both exceptions and the namespace operator -
1203+exceptions and the namespace operator - `.^` in Hoon, which becomes operator
1313−`.^` in Hoon, which becomes operator `11` in `mock`.
1204+`11` in `mock`.
1314
1205
1315−`11` requires a namespace function, or `fly`, which produces a
1206+`11` requires a namespace function, or `fly`, which produces a `++unit` - `~`
1316−`++unit` - `~` (`0`) for no binding, or `[0 value]`. The sample
1207+(`0`) for no binding, or `[0 value]`. The sample to a `fly` is a `++path`, just
1317−to a `fly` is a `++path`, just a list of text `span`.
1208+a list of text `span`.
1318
1209
1319−`mock` functions produce a `++toon`. Fully elaborated:
1210+`mock` functions produce a `++toon`. Fully elaborated:
1320
1211
1321 ++ noun ,* :: any noun
1212 ++ noun ,* :: any noun
1322 ++ path (list ,@ta) :: namespace path
1213 ++ path (list ,@ta) :: namespace path
@@ -1339,20 +1230,18 @@ to a `fly` is a `++path`, just a list of text `span`.
1339
1230
1340(Note that `tank` is overdesigned and due for replacement.)
1231(Note that `tank` is overdesigned and due for replacement.)
1341
1232
1342−What does a `toon` mean? Either your computation succeded (`[0
1233+What does a `toon` mean? Either your computation succeded (`[0 noun]`, or could
1343−noun]`, or could not finish because it blocked on one or more
1234+not finish because it blocked on one or more global paths (`[1 (list path)]`),
1344−global paths (`[1 (list path)]`), or it exited with a stack trace
1235+or it exited with a stack trace (`[2 (list tank)]`).
1345−(`[2 (list tank)]`).
1346
1236
1347−Note that of all the `u3` exceptions, only `%exit` is produced
1237+Note that of all the `u3` exceptions, only `%exit` is produced deterministically
1348−deterministically by the Nock definition. Therefore, only
1238+by the Nock definition. Therefore, only `%exit` produces a `2` result. Any other
1349−`%exit` produces a `2` result. Any other argument to
1350−`u3m_bail()` will unwind the virtualization stack all the way to
1239+argument to `u3m_bail()` will unwind the virtualization stack all the way to the
1351−the top - or to be more exact, to `u3m_soft_top()`.
1240+top - or to be more exact, to `u3m_soft_top()`.
1352
1241
1353−In any case, the simplest `mock` functions are `u3n_nock_un()`
1242+In any case, the simplest `mock` functions are `u3n_nock_un()` and
1354−and `u3n_slam_un()`. These provide exception control without
1243+`u3n_slam_un()`. These provide exception control without any namespace change,
1355−any namespace change, as you can see by the code:
1244+as you can see by the code:
1356
1245
1357 /* u3n_nock_un(): produce .*(bus fol), as ++toon.
1246 /* u3n_nock_un(): produce .*(bus fol), as ++toon.
1358 */
1247 */
@@ -1374,104 +1263,92 @@ any namespace change, as you can see by the code:
1374 return u3n_slam_in(fly, gat, sam);
1263 return u3n_slam_in(fly, gat, sam);
1375 }
1264 }
1376
1265
1377−The `fly` is added as the first argument to `u3n_nock_in()` and
1266+The `fly` is added as the first argument to `u3n_nock_in()` and `u3n_slam_in()`.
1378−`u3n_slam_in()`. Of course, logically, `fly` executes in the
1267+Of course, logically, `fly` executes in the caller's exception layer.
1379−caller's exception layer. (Maintaining this illusion is slightly
1268+(Maintaining this illusion is slightly nontrivial.) Finally, `u3n_nock_an()` is
1380−nontrivial.) Finally, `u3n_nock_an()` is a sandbox with a null
1381−namespace.
1269+a sandbox with a null namespace.
1382
1270
1383### u3e: persistence
1271### u3e: persistence
1384
1272
1385−The only `u3e` function you should need to call is `u3e_save()`,
1273+The only `u3e` function you should need to call is `u3e_save()`, which saves the
1386−which saves the loom. As it can be restored on any platform,
1274+loom. As it can be restored on any platform, please make sure you don't have any
1387−please make sure you don't have any state in the loom that is
1388−bound to your process or architecture - except for exceptions
1275+state in the loom that is bound to your process or architecture - except for
1389−like the warm jet state, which is actively purged on reboot.
1276+exceptions like the warm jet state, which is actively purged on reboot.
1390
1277
1391### u3r: reading nouns (weak)
1278### u3r: reading nouns (weak)
1392
1279
1393−As befits accessors they don't make anything, `u3r` noun reading
1280+As befits accessors they don't make anything, `u3r` noun reading functions
1394−functions always retain their arguments and their returns. They
1281+always retain their arguments and their returns. They never bail; rather, when
1395−never bail; rather, when they don't work, they return a `u3_weak`
1282+they don't work, they return a `u3_weak` result.
1396−result.
1283+
1397−
1398−Most of these functions are straightforward and do only what
1284+Most of these functions are straightforward and do only what their comments say.
1399−their comments say. A few are interesting enough to discuss.
1285+A few are interesting enough to discuss.
1400−
1286+
1401−`u3r_at()` is the familiar tree fragment function, `/` from the
1287+`u3r_at()` is the familiar tree fragment function, `/` from the Nock spec. For
1402−Nock spec. For taking complex nouns apart, `u3r_mean()` is a
1288+taking complex nouns apart, `u3r_mean()` is a relatively funky way of
1403−relatively funky way of deconstructing nouns with a varargs list
1289+deconstructing nouns with a varargs list of `axis`, `u3_noun *`. For cells,
1404−of `axis`, `u3_noun *`. For cells, triples, etc, decompose with
1405−`u3r_cell()`, `u3r_trel()`, etc. For the tagged equivalents, use
1290+triples, etc, decompose with `u3r_cell()`, `u3r_trel()`, etc. For the tagged
1406−`u3r_pq()` and friends.
1291+equivalents, use `u3r_pq()` and friends.
1407−
1292+
1408−`u3r_sing(u3_noun a, u3_noun b)` (true if `a` and `b` are a
1293+`u3r_sing(u3_noun a, u3_noun b)` (true if `a` and `b` are a *single* noun) are
1409−*single* noun) are interesting because it uses mugs to help it
1294+interesting because it uses mugs to help it out. Clearly, different nouns may
1410−out. Clearly, different nouns may have the same mug, but the
1411−same nouns cannot have a different mug. It's important to
1295+have the same mug, but the same nouns cannot have a different mug. It's
1412−understand the performance characteristics of `u3r_sing()`:
1296+important to understand the performance characteristics of `u3r_sing()`: the
1413−the worst possible case is a comparison of duplicate nouns,
1297+worst possible case is a comparison of duplicate nouns, which have the same
1414−which have the same value but were created separately. In this
1298+value but were created separately. In this case, the tree is traversed
1415−case, the tree is traversed
1299+
1416−
1417−`u3r_sung()` is a deeply funky and frightening version of
1300+`u3r_sung()` is a deeply funky and frightening version of `u3r_sing()` that
1418−`u3r_sing()` that unifies pointers to the duplicate nouns it
1301+unifies pointers to the duplicate nouns it finds, freeing the second copy.
1419−finds, freeing the second copy. Obviously, do not use
1420−`u3r_sung()` when you have live, but not reference counted, noun
1302+Obviously, do not use `u3r_sung()` when you have live, but not reference
1421−references from C - if they match a noun with a refcount of 1
1303+counted, noun references from C - if they match a noun with a refcount of 1 that
1422−that gets freed, bad things happen.
1304+gets freed, bad things happen.
1423−
1305+
1424−It's important to remember that `u3r_mug()`, which produces a
1306+It's important to remember that `u3r_mug()`, which produces a 31-bit, nonzero
1425−31-bit, nonzero insecure hash, uses the `mug_w` slot in any boxed
1307+insecure hash, uses the `mug_w` slot in any boxed noun as a lazy cache. There
1426−noun as a lazy cache. There are a number of variants of
1427−`u3r_mug()` that can get you out of building unneeded nouns.
1308+are a number of variants of `u3r_mug()` that can get you out of building
1309+unneeded nouns.
1428
1310
1429### u3x: reading nouns (bail)
1311### u3x: reading nouns (bail)
1430
1312
1431−`u3x` functions are like `u3r` functions, but instead of
1313+`u3x` functions are like `u3r` functions, but instead of returning `u3_none`
1432−returning `u3_none` when (for instance) we try to take the head
1314+when (for instance) we try to take the head of an atom, they bail with `%exit`.
1433−of an atom, they bail with `%exit`. In other words, they do what
1434−the same operation would do in Nock.
1315+In other words, they do what the same operation would do in Nock.
1435
1316
1436### u3h: hash tables.
1317### u3h: hash tables.
1437
1318
1438−We can of course use the Hoon `map` structure as an associative
1319+We can of course use the Hoon `map` structure as an associative array. This is a
1439−array. This is a balanced treap and reasonably fast. However,
1320+balanced treap and reasonably fast. However, it's considerably inferior to a
1440−it's considerably inferior to a custom structure like an HAMT
1441−(hash array-mapped trie). We use `u3_post` to allocate HAMT
1321+custom structure like an HAMT (hash array-mapped trie). We use `u3_post` to
1442−structures on the loom.
1322+allocate HAMT structures on the loom.
1443
1323
1444−(Our HAMT implements the classic Bagwell algorithm which depends
1324+(Our HAMT implements the classic Bagwell algorithm which depends on the `gcc`
1445−on the `gcc` standard directive `__builtin_popcount()`. On a CPU
1325+standard directive `__builtin_popcount()`. On a CPU which doesn't support
1446−which doesn't support popcount or an equivalent instruction, some
1326+popcount or an equivalent instruction, some other design would probably be
1447−other design would probably be preferable.)
1327+preferable.)
1448
1328
1449−There's no particular rocket science in the API. `u3h_new()`
1329+There's no particular rocket science in the API. `u3h_new()` creates a
1450−creates a hashtable; `u3h_free()` destroys one; `u3h_put()`
1330+hashtable; `u3h_free()` destroys one; `u3h_put()` inserts, `u3h_get()`
1451−inserts, `u3h_get()` retrieves. You can transform values in a
1331+retrieves. You can transform values in a hashtable with `u3h_walk()`.
1452−hashtable with `u3h_walk()`.
1453
1332
1454−The only funky function is `u3h_gut()`, which unifies keys with
1333+The only funky function is `u3h_gut()`, which unifies keys with `u3r_sung()`. As
1455−`u3r_sung()`. As with all cases of `u3r_sung()`, this must be
1334+with all cases of `u3r_sung()`, this must be used with extreme caution.
1456−used with extreme caution.
1457
1335
1458### u3z: memoization
1336### u3z: memoization
1459
1337
1460−Connected to the `~+` rune in Hoon, via the Nock `%memo` hint,
1338+Connected to the `~+` rune in Hoon, via the Nock `%memo` hint, the memoization
1461−the memoization facility is a general-purpose cache.
1339+facility is a general-purpose cache.
1462
1340
1463−(It's also used for partial memoization - a feature that'll
1341+(It's also used for partial memoization - a feature that'll probably be removed,
1464−probably be removed, in which conservative worklist algorithms
1342+in which conservative worklist algorithms (which would otherwise be exponential)
1465−(which would otherwise be exponential) memoize everything in the
1466−subject *except* the worklist. This is used heavily in the Hoon
1343+memoize everything in the subject *except* the worklist. This is used heavily in
1467−compiler jets (j/f/*.c). Unfortunately, it's probably not
1344+the Hoon compiler jets (j/f/\*.c). Unfortunately, it's probably not possible to
1468−possible to make this work perfectly in that it can't be abused
1345+make this work perfectly in that it can't be abused to violate Nock, so we'll
1469−to violate Nock, so we'll probably remove it at a later date,
1470−instead making `++ut` keep its own monadic cache.)
1346+probably remove it at a later date, instead making `++ut` keep its own monadic
1347+cache.)
1471
1348
1472−Each `u3z` function comes with a `c3_m` mote which disambiguates
1349+Each `u3z` function comes with a `c3_m` mote which disambiguates the function
1473−the function mapping key to value. For Nock itself, use 0. For
1350+mapping key to value. For Nock itself, use 0. For extra speed, small tuples are
1474−extra speed, small tuples are split out in C; thus, find with
1351+split out in C; thus, find with
1475
1352
1476 u3_weak u3z_find(c3_m, u3_noun);
1353 u3_weak u3z_find(c3_m, u3_noun);
1477 u3_weak u3z_find_2(c3_m, u3_noun, u3_noun);
1354 u3_weak u3z_find_2(c3_m, u3_noun, u3_noun);
@@ -1485,18 +1362,18 @@ and save with
1485 u3_noun u3z_save_3(c3_m, u3_noun, u3_noun, u3_noun, u3_noun);
1362 u3_noun u3z_save_3(c3_m, u3_noun, u3_noun, u3_noun, u3_noun);
1486 u3_noun u3z_save_4(c3_m, u3_noun, u3_noun, u3_noun, u3_noun, u3_noun);
1363 u3_noun u3z_save_4(c3_m, u3_noun, u3_noun, u3_noun, u3_noun, u3_noun);
1487
1364
1488−where the value is the last argument. To eliminate duplicate
1365+where the value is the last argument. To eliminate duplicate nouns, there is
1489−nouns, there is also
1366+also
1490
1367
1491 u3_noun
1368 u3_noun
1492 u3z_uniq(u3_noun);
1369 u3z_uniq(u3_noun);
1493
1370
1494`u3z` functions retain keys and transfer values.
1371`u3z` functions retain keys and transfer values.
1495−
1372+
1496−The `u3z` cache, built on `u3h` hashes, is part of the current
1373+The `u3z` cache, built on `u3h` hashes, is part of the current road, and goes
1497−road, and goes away when it goes away. (In future, we may wish
1374+away when it goes away. (In future, we may wish to promote keys/values which
1498−to promote keys/values which outlive the road, as we do with jet
1499−state.) There is no cache reclamation at present, so be careful.
1375+outlive the road, as we do with jet state.) There is no cache reclamation at
1376+present, so be careful.
1500
1377
1501### u3t: tracing and profiling.
1378### u3t: tracing and profiling.
1502
1379
@@ -1504,9 +1381,8 @@ TBD.
1504
1381
1505### u3v: the Arvo kernel
1382### u3v: the Arvo kernel
1506
1383
1507−An Arvo kernel - or at least, a core that compiles with the Arvo
1384+An Arvo kernel - or at least, a core that compiles with the Arvo interface - is
1508−interface - is part of the global `u3` state. What is an Arvo
1385+part of the global `u3` state. What is an Arvo core? Slightly pseudocoded:
1509−core? Slightly pseudocoded:
1510
1386
1511 ++ arvo
1387 ++ arvo
1512 |%
1388 |%
@@ -1533,54 +1409,49 @@ core? Slightly pseudocoded:
1533 ++ ovum {p/wire q/card} :: Arvo event
1409 ++ ovum {p/wire q/card} :: Arvo event
1534 ++ wire path :: event cause
1410 ++ wire path :: event cause
1535
1411
1536−This is the Arvo ABI in a very real sense. Arvo is a core with
1412+This is the Arvo ABI in a very real sense. Arvo is a core with these six arms.
1537−these six arms. To use these arms, we hardcode the axis of the
1413+To use these arms, we hardcode the axis of the formula (`11`, `4`, `86`, etc)
1538−formula (`11`, `4`, `86`, etc) into the C code that calls Arvo,
1539−because otherwise we'd need type metadata - which we can get, by
1414+into the C code that calls Arvo, because otherwise we'd need type metadata -
1540−calling Arvo.
1415+which we can get, by calling Arvo.
1541
1416
1542−It's important to understand the Arvo event/action structure, or
1417+It's important to understand the Arvo event/action structure, or `++ovum`. An
1543−`++ovum`. An `ovum` is a `card`, which is any `[term noun]`
1418+`ovum` is a `card`, which is any `[term noun]` cell, and a `++wire`, a `path`
1544−cell, and a `++wire`, a `path` which indicates the location of
1545−the event. At the Unix level, the `wire` corresponds to a system
1419+which indicates the location of the event. At the Unix level, the `wire`
1546−module or context. For input events, this is the module that
1420+corresponds to a system module or context. For input events, this is the module
1547−caused the event; for output actions, it's the module that
1421+that caused the event; for output actions, it's the module that performs the
1548−performs the action.
1422+action.
1549
1423
1550−`++poke` sends Arvo an event `ovum`, producing a cell of action
1424+`++poke` sends Arvo an event `ovum`, producing a cell of action ova and a new
1551−ova and a new Arvo core.
1425+Arvo core.
1552
1426
1553−`++peek` dereferences the Arvo namespace. It takes a date and a
1427+`++peek` dereferences the Arvo namespace. It takes a date and a key, and
1554−key, and produces `~` (`0`) or `[~ value]`.
1428+produces `~` (`0`) or `[~ value]`.
1555
1429
1556−`++keep` asks Arvo the next time it wants to be woken up, for the
1430+`++keep` asks Arvo the next time it wants to be woken up, for the given `wire`.
1557−given `wire`. (This input will probably be eliminated in favor
1431+(This input will probably be eliminated in favor of a single global timer.)
1558−of a single global timer.)
1559
1432
1560−`++wish` compiles a string of Hoon source. While just a
1433+`++wish` compiles a string of Hoon source. While just a convenience, it's a very
1561−convenience, it's a very convenient convenience.
1434+convenient convenience.
1562
1435
1563−`++come` and `++load` are used by Arvo to reset itself (more
1436+`++come` and `++load` are used by Arvo to reset itself (more precisely, to shift
1564−precisely, to shift the Arvo state from an old kernel to a new
1437+the Arvo state from an old kernel to a new one); there is no need to call them
1565−one); there is no need to call them from C.
1438+from C.
1566
1439
1567−Now that we understand the Arvo kernel interface, let's look at
1440+Now that we understand the Arvo kernel interface, let's look at the `u3v` API.
1568−the `u3v` API. As usual, all the functions in `u3v` are
1441+As usual, all the functions in `u3v` are commented, but unfortunately it's hard
1569−commented, but unfortunately it's hard to describe this API as
1570−clean at present. The problem is that `u3v` remains design
1442+to describe this API as clean at present. The problem is that `u3v` remains
1571−coupled to the old `vere` event handling code written for `u2`.
1443+design coupled to the old `vere` event handling code written for `u2`. But let's
1572−But let's describe the functions you should be calling, assuming
1444+describe the functions you should be calling, assuming you're not writing the
1573−you're not writing the next event system. There are only two.
1445+next event system. There are only two.
1574
1446
1575−`u3v_wish(str_c)` wraps the `++wish` functionality in a cache
1447+`u3v_wish(str_c)` wraps the `++wish` functionality in a cache (which is
1576−(which is read-only unless you're on the surface road).
1448+read-only unless you're on the surface road).
1577
1449
1578−`u3v_do()` uses `wish` to provide a convenient interface for
1450+`u3v_do()` uses `wish` to provide a convenient interface for calling Hoon kernel
1579−calling Hoon kernel functions by name. Even more conveniently,
1451+functions by name. Even more conveniently, we tend to call `u3v_do()` with these
1580−we tend to call `u3v_do()` with these convenient aliases:
1452+convenient aliases:
1581
1453
1582 #define u3do(txt_c, arg) u3v_do(txt_c, arg)
1454 #define u3do(txt_c, arg) u3v_do(txt_c, arg)
1583 #define u3dc(txt_c, a, b) u3v_do(txt_c, u3nc(a, b))
1455 #define u3dc(txt_c, a, b) u3v_do(txt_c, u3nc(a, b))
1584 #define u3dt(txt_c, a, b, c) u3v_do(txt_c, u3nt(a, b, c))
1456 #define u3dt(txt_c, a, b, c) u3v_do(txt_c, u3nt(a, b, c))
1585 #define u3dq(txt_c, a, b, c, d) u3v_do(txt_c, u3nt(a, b, c, d))
1457 #define u3dq(txt_c, a, b, c, d) u3v_do(txt_c, u3nt(a, b, c, d))
1586−
docs/arvo.md+3−2
@@ -1,11 +1,12 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 4
3sort: 4
4---
4---
5
5
6# Arvo
6# Arvo
7
7
8−Fast, pragmatic tutorials which show you how to build simple programs in Hoon and Arvo.
8+Fast, pragmatic tutorials which show you how to build simple programs in Hoon
9+and Arvo.
9
10
10If you learn by getting things done, not by exploring fundamentals, start here.
11If you learn by getting things done, not by exploring fundamentals, start here.
11
12
docs/arvo/api.md+425−456
@@ -1,38 +1,34 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: true
3+next: True
4sort: 21
4sort: 21
5title: API Connectors
5title: API Connectors
6---
6---
7
7
8# API Connectors
8# API Connectors
9
9
10−Most people have lots of data stored in online services, many of
10+Most people have lots of data stored in online services, many of which have
11−which have APIs. API connectors allow the user to access this
11+APIs. API connectors allow the user to access this data from within their Urbit.
12−data from within their Urbit.
13
12
14−A security driver allows the user to make authenticated requests
13+A security driver allows the user to make authenticated requests to the
15−to the service's API, but the user still needs to know the API
14+service's API, but the user still needs to know the API and send and receive
16−and send and receive JSON. An API connector puts a layer of
15+JSON. An API connector puts a layer of porcelain over this to allow the user
17−porcelain over this to allow the user easier control over their
16+easier control over their data.
18−data.
19
17
20−Each connector should perform two basic functions. First, it
18+Each connector should perform two basic functions. First, it should expose a
21−should expose a tree of the data in the service that's accessible
19+tree of the data in the service that's accessible through `.^` and FUSE. Second,
22−through `.^` and FUSE. Second, it should expose event streams as
23−actions occur in the service. Let's take a look at how the `%gh`
20+it should expose event streams as actions occur in the service. Let's take a
24−app accomplishes both for Github.
21+look at how the `%gh` app accomplishes both for Github.
25
22
26−After starting `%gh` (`|start %gh`), let's look at the root of
23+After starting `%gh` (`|start %gh`), let's look at the root of the tree that
27−the tree that `%gh` exposes:
24+`%gh` exposes:
28
25
29 ~your-urbit:dojo> .^(arch %gy /=gh=)
26 ~your-urbit:dojo> .^(arch %gy /=gh=)
30 [fil=~ dir={[p=~.issues q=~]}]
27 [fil=~ dir={[p=~.issues q=~]}]
31
28
32−`%gh` is currently a skeleton -- it contains examples of all the
29+`%gh` is currently a skeleton -- it contains examples of all the functionality
33−functionality necessary for a connector, but many endpoints
30+necessary for a connector, but many endpoints aren't implemented. As we can see
34−aren't implemented. As we can see here, only issues are
35−implemented. We can explore this tree:
31+here, only issues are implemented. We can explore this tree:
36
32
37 ~your-urbit:dojo> .^(arch %gy /=gh=/issues)
33 ~your-urbit:dojo> .^(arch %gy /=gh=/issues)
38 [fil=~ dir={[p=~.by-repo q=~] [p=~.mine q=~]}]
34 [fil=~ dir={[p=~.by-repo q=~] [p=~.mine q=~]}]
@@ -45,58 +41,50 @@ And eventually:
45 login.user:.^(issue:gh %gx /=gh=/issues/by-repo/philipcmonktest/testing/7)
41 login.user:.^(issue:gh %gx /=gh=/issues/by-repo/philipcmonktest/testing/7)
46 'philipcmonktest'
42 'philipcmonktest'
47
43
48−The other thing an API connector needs to do is expose event
44+The other thing an API connector needs to do is expose event streams. One
49−streams. One straightforward way to use this capacity is with
45+straightforward way to use this capacity is with the `:pipe` app:
50−the `:pipe` app:
51
46
52 :pipe|connect %gh /listen/philipcmonktest/testing/issues/'issue_comment' %public
47 :pipe|connect %gh /listen/philipcmonktest/testing/issues/'issue_comment' %public
53
48
54−Now creating an issue results in the following message in your
49+Now creating an issue results in the following message in your `%public` `:talk`
55−`%public` `:talk` channel:
50+channel:
56
51
57 ~your-urbit[philipcmonktest@github]: opened issue #11: i found a bug!
52 ~your-urbit[philipcmonktest@github]: opened issue #11: i found a bug!
58
53
59−You'll also get notifications of issue status changes and
54+You'll also get notifications of issue status changes and comments.
60−comments.
61
55
62−As is often the case, the best way to write an API connector is
56+As is often the case, the best way to write an API connector is to copy an
63−to copy an existing one and modify it. We'll dissect the Github
57+existing one and modify it. We'll dissect the Github connector here. This code
64−connector here. This code is in `/=home=/app/gh/hoon`, and it's
65−also reproduced at the bottom of this page (in case the code
58+is in `/=home=/app/gh/hoon`, and it's also reproduced at the bottom of this page
66−drifts out of sync with this doc).
59+(in case the code drifts out of sync with this doc).
67
60
68We'll go over the two parts to an API connector (one-time reading vs listening
61We'll go over the two parts to an API connector (one-time reading vs listening
69for events) separately.
62for events) separately.
70
63
71## Reading
64## Reading
72
65
73−A connector exposes a tree of data. Every read request has a
66+A connector exposes a tree of data. Every read request has a `care`, which is
74−`care`, which is generally either `%x` or `%y. `%x` is a request
67+generally either `%x` or
75−for a particular piece of data, while `%y` is a request for a
68+`%y.`%x`is a request for a particular piece of data, while`%y`is a request for a directory listing. Roughly,`%x`means Unix`cat`and`%y`means Unix`ls\`.
76−directory listing. Roughly, `%x` means Unix `cat` and `%y` means
69+
77−Unix `ls`.
78−
79−> Sometimes you wish to expose a tree where a part of the path
70+> Sometimes you wish to expose a tree where a part of the path can't be
80−> can't be enumerated. For example, a `%y` of `/issues/by-repo`
71+> enumerated. For example, a `%y` of `/issues/by-repo` "should" produce a list
81−> "should" produce a list of all Github users, but we don't do
82−> that because it's too long. Instead, we just produce our own
72+> of all Github users, but we don't do that because it's too long. Instead, we
83−> username (from the `web.plan` file). You can still access
73+> just produce our own username (from the `web.plan` file). You can still access
84−> repos from other users, you just don't see them in the
74+> repos from other users, you just don't see them in the directory listing.
85−> directory listing.
75+
86−
87−The usual flow for implementing this tree of data makes heavy use
76+The usual flow for implementing this tree of data makes heavy use of the
88−of the `connector` library. This library is well documented in
77+`connector` library. This library is well documented in the source, so check out
89−the source, so check out `/=home=/lib/connector/hoon`.
78+`/=home=/lib/connector/hoon`.
90−
79+
91−The `connector` library needs to be initialized with definitions
80+The `connector` library needs to be initialized with definitions of `move` and
92−of `move` and `sub-result` (all the types of data that can be
81+`sub-result` (all the types of data that can be returned by a place). This can
93−returned by a place). This can just be a line at the top of the
82+just be a line at the top of the main core:
94−main core:
95
83
96 =+ connector=(connector move sub-result) :: Set up connector library
84 =+ connector=(connector move sub-result) :: Set up connector library
97
85
98−Most of the Github-specific logic is in `++places`, which is a
86+Most of the Github-specific logic is in `++places`, which is a list of all the
99−list of all the places we can request. A place consists of:
87+places we can request. A place consists of:
100
88
101 ++ place
89 ++ place
102 $: guard/mold
90 $: guard/mold
@@ -106,421 +94,402 @@ list of all the places we can request. A place consists of:
106 sigh-y/$-(jon/json (unit arch))
94 sigh-y/$-(jon/json (unit arch))
107 ==
95 ==
108
96
109−- `guard`, the type of the paths we should match. For example,
97+- `guard`, the type of the paths we should match. For example, to match
110− to match `/issues/<user>/<repo>` use `{$issues @t @t $~}`.
98+ `/issues/<user>/<repo>` use `{$issues @t @t $~}`.
111−
99+
112−- `read-x`, called when someone tries to read the place with
100+- `read-x`, called when someone tries to read the place with `care` `%x`.
113− `care` `%x`. Should produce a single move, usually either a
101+ Should produce a single move, usually either a `%diff` response if we can
114− `%diff` response if we can immediately answer or a `%hiss`
115− http request if we need to make a request to the api. See the
102+ immediately answer or a `%hiss` http request if we need to make a request to
116− `++read-*` functions in `++helpers` for some common handlers.
103+ the api. See the `++read-*` functions in `++helpers` for some common
117−
104+ handlers.
105+
118−- `read-y`, same as `read-x` except with care `%y`.
106+- `read-y`, same as `read-x` except with care `%y`.
119−
107+
120−- `sigh-x`, called when an http response comes back on this
108+- `sigh-x`, called when an http response comes back on this place. You're
121− place. You're given the json of the result, and you should
109+ given the json of the result, and you should produce either a result or
122− produce either a result or null. Null represents an error.
123− If you didn't create an http request in `read-x`, then this
110+ null. Null represents an error. If you didn't create an http request in
124− should never be called. Use `++sigh-strange` from `++helpers`
111+ `read-x`, then this should never be called. Use `++sigh-strange` from
125− to unconditionally signal an error.
112+ `++helpers` to unconditionally signal an error.
126−
113+
127−- `sigh-y`, same as `sigh-x` except with care `%y`. Note that a
114+- `sigh-y`, same as `sigh-x` except with care `%y`. Note that a `%y` request
128− `%y` request must produce an arch, unlike a `%x` request,
115+ must produce an arch, unlike a `%x` request, which may produce data of any
129− which may produce data of any mark.
116+ mark.
130−
117+
131−Filling out the list of places is a lot of grunt work, but most
118+Filling out the list of places is a lot of grunt work, but most places are
132−places are fairly straightforward, and the `++read-*` helper
119+fairly straightforward, and the `++read-*` helper functions are useful. Check
133−functions are useful. Check out the library source for more
134−information on those.
120+out the library source for more information on those.
135−
121+
136−Besides the list of places, we just need to handle the flow of
122+Besides the list of places, we just need to handle the flow of control. `++peek`
137−control. `++peek` and `++peer-scry` are the interface we expose
123+and `++peer-scry` are the interface we expose to the rest of the system while
138−to the rest of the system while `++sigh-httr` and `++sigh-tang`
139−are used to handle responses when we make HTTP requests.
124+`++sigh-httr` and `++sigh-tang` are used to handle responses when we make HTTP
140−
125+requests.
126+
141−- `++peek` should usually just produce `~`. If there are cases
127+- `++peek` should usually just produce `~`. If there are cases where we can
142− where we can respond directly to a `.^` request without
128+ respond directly to a `.^` request without blocking on anything, we could do
143− blocking on anything, we could do it here, but it's generally
144− not worth the hassle since the same logic should be duplicated
129+ it here, but it's generally not worth the hassle since the same logic should
145− in `++peer-scry`.
130+ be duplicated in `++peer-scry`.
146−
131+
147−- `++peer-scry` is where the actual handling for a read request
132+- `++peer-scry` is where the actual handling for a read request goes. In
148− goes. In general, we just need to call `++read` from the
133+ general, we just need to call `++read` from the `connector` library with the
149− `connector` library with the bone, list of places, care, and
150− path. This will match the path to the appropriate place and
134+ bone, list of places, care, and path. This will match the path to the
151− run either `++read-x` or `++read-y`, depending on the care.
135+ appropriate place and run either `++read-x` or `++read-y`, depending on the
152−
136+ care.
137+
153−- If `++read-x` or `++read-y` made a successful API request, then
138+- If `++read-x` or `++read-y` made a successful API request, then the response
154− the response will come back on `++sigh-httr`. Here, we just
139+ will come back on `++sigh-httr`. Here, we just need to parse out the wire
155− need to parse out the wire and call `++sigh` from the
156− `connector` library with the list of places, care, path, and
140+ and call `++sigh` from the `connector` library with the list of places,
157− HTTP result. This will match the path to the appropriate place
141+ care, path, and HTTP result. This will match the path to the appropriate
158− and run either `++sigh-x` or `++sigh-y`, depending on the care.
142+ place and run either `++sigh-x` or `++sigh-y`, depending on the care.
159−
143+
160−- If `++read-x` or `++read-y` made an API request that failed,
144+- If `++read-x` or `++read-y` made an API request that failed, then we'll get
161− then we'll get a stack trace in `++sigh-tang`. Here, we just
145+ a stack trace in `++sigh-tang`. Here, we just print it out and move on.
162− print it out and move on.
146+
163−
164−That's really all there is to the reading portion of API
147+That's really all there is to the reading portion of API connectors.
165−connectors.
148+
166−
167−One of the most accessbile ways to jump into Arvo programming is
149+One of the most accessbile ways to jump into Arvo programming is to just add
168−to just add more places to an existing API connector. It's
150+more places to an existing API connector. It's useful, small in scope, and comes
169−useful, small in scope, and comes in bite-sized chunks since most
170−places are less than ten lines of code.
151+in bite-sized chunks since most places are less than ten lines of code.
171
152
172## Listening
153## Listening
173
154
174−Listening for events is fairly service-specific. In some
155+Listening for events is fairly service-specific. In some services, we poll for
175−services, we poll for changes. The Twitter connector has an
176−example of this, but note that it predates the `connector`
156+changes. The Twitter connector has an example of this, but note that it predates
177−library and is thus more complicated than it needs to be, and the
157+the `connector` library and is thus more complicated than it needs to be, and
178−interface it exposes isn't standard. In Github, we power our
158+the interface it exposes isn't standard. In Github, we power our event streams
179−event streams with webhooks.
159+with webhooks.
180−
181−For Github, when someone subscribes to
182−`/listen/<user>/<repo>/<events...>`, we want to produce
183−well-typed results when they occur.
184−
185−We only want to create one webhook per event, so in our state we
186−have `hook`, which is a map of event names to the set of bones
187−which are subscribed to that event. We must always keep this
188−up-to-date.
189−
190−Flow of control starts in `++peer-listen`, where we just call
191−`++listen`. In `++listen`, for each event in the list of events,
192−we check to see if we have that hook set up (by checking whether
193−it exists in `hook`). If so, we call `++update-hook` to add the
194−current bone to the set of listeners. Otherwise, we call
195−`++create-hook`, which sends a request to Github to set up the
196−new webhook. We also create an entry in `hook` with the current
197−bone.
198
160
199−When we created the webhook, we told Github to send the event to
161+For Github, when someone subscribes to `/listen/<user>/<repo>/<events...>`, we
200−`/~/to/gh/gh-<event>.json?anon&wire=/`. This turns into a poke.
162+want to produce well-typed results when they occur.
201−Let's parse out this url. The first `gh` is the app name to
163+
202−poke. The next portion, `gh-<event>` is the mark to convert the
164+We only want to create one webhook per event, so in our state we have `hook`,
203−JSON to. For example, when the "issues" event fires, it'll get
165+which is a map of event names to the set of bones which are subscribed to that
204−converted to mark `gh-issues`. This mark has a conversion
166+event. We must always keep this up-to-date.
205−routine from `json` to the `issues` type, which is defined in the
206−`gh` structure (`/=home=/sur/gh/hoon`). Once this conversion
207−happens, the `gh` app will get poked with the correctly marked
208−data in `++poke`.
209
167
210−In `++poke`, we check `hook` to find the subscribers to the event
168+Flow of control starts in `++peer-listen`, where we just call `++listen`. In
211−that was fired, and we update them.
169+`++listen`, for each event in the list of events, we check to see if we have
170+that hook set up (by checking whether it exists in `hook`). If so, we call
171+`++update-hook` to add the current bone to the set of listeners. Otherwise, we
172+call `++create-hook`, which sends a request to Github to set up the new webhook.
173+We also create an entry in `hook` with the current bone.
212
174
213−That's really all there is to it. Webhook flow isn't well
175+When we created the webhook, we told Github to send the event to
214−standardized, so even if your chosen service is powered by
176+`/~/to/gh/gh-<event>.json?anon&wire=/`. This turns into a poke. Let's parse out
215−webhooks, your listening code might look rather different. The
177+this url. The first `gh` is the app name to poke. The next portion, `gh-<event>`
216−point is to expose the correct interface.
178+is the mark to convert the JSON to. For example, when the "issues" event fires,
179+it'll get converted to mark `gh-issues`. This mark has a conversion routine from
180+`json` to the `issues` type, which is defined in the `gh` structure
181+(`/=home=/sur/gh/hoon`). Once this conversion happens, the `gh` app will get
182+poked with the correctly marked data in `++poke`.
217
183
184+In `++poke`, we check `hook` to find the subscribers to the event that was
185+fired, and we update them.
186+
187+That's really all there is to it. Webhook flow isn't well standardized, so even
188+if your chosen service is powered by webhooks, your listening code might look
189+rather different. The point is to expose the correct interface.
218
190
219## Github API Connector Code
191## Github API Connector Code
220
192
221−In case the code in `/=home=/app/gh/hoon` drifts out of sync with
193+In case the code in `/=home=/app/gh/hoon` drifts out of sync with this doc:
222−this doc:
194+
223−
224−```
225−:: This is a connector for the Github API v3.
195+ :: This is a connector for the Github API v3.
226−::
227−:: You can interact with this in a few different ways:
228−::
229−:: - .^({type} %gx /=gh={/endpoint}) to read data or
230−:: .^(arch %gy /=gh={/endpoint}) to explore the possible
231−:: endpoints.
232−::
233−:: - subscribe to /listen/{owner}/{repo}/{events...} for
234−:: webhook-powered event notifications. For event list, see
235−:: https://developer.github.com/webhooks/.
236−::
237−:: This is written with the standard structure for api
238−:: connectors, as described in lib/connector.hoon.
239−::
240−/? 314
241−/- gh, plan-acct
242−/+ gh-parse, connector
243−::
244−!:
245−=> |%
246− ++ move (pair bone card)
247− ++ card
248− $% {$diff sub-result}
249− {$them wire (unit hiss)}
250− {$hiss wire {$~ $~} $httr {$hiss hiss}}
251− ==
252 ::
196 ::
253− :: Types of results we produce to subscribers.
197+ :: You can interact with this in a few different ways:
254 ::
198 ::
255− ++ sub-result
199+ :: - .^({type} %gx /=gh={/endpoint}) to read data or
256− $% {$arch arch}
200+ :: .^(arch %gy /=gh={/endpoint}) to explore the possible
257− {$gh-issue issue:gh}
201+ :: endpoints.
258− {$gh-list-issues (list issue:gh)}
259− {$gh-issues issues:gh}
260− {$gh-issue-comment issue-comment:gh}
261− {$json json}
262− {$null $~}
263− ==
264 ::
202 ::
265− :: Types of webhooks we expect.
203+ :: - subscribe to /listen/{owner}/{repo}/{events...} for
204+ :: webhook-powered event notifications. For event list, see
205+ :: https://developer.github.com/webhooks/.
266 ::
206 ::
267− ++ hook-response
207+ :: This is written with the standard structure for api
268− $% {$gh-issues issues:gh}
208+ :: connectors, as described in lib/connector.hoon.
269− {$gh-issue-comment issue-comment:gh}
209+ ::
270− ==
210+ /? 314
271− --
211+ /- gh, plan-acct
272−=+ connector=(connector move sub-result) :: Set up connector library
212+ /+ gh-parse, connector
273−::
213+ ::
274−|_ $: hid/bowl
214+ !:
275− hook/(map @t {id/@t listeners/(set bone)}) :: map events to listeners
215+ => |%
276− ==
216+ ++ move (pair bone card)
277−:: ++ prep _`. :: Clear state when code changes
217+ ++ card
278−::
218+ $% {$diff sub-result}
279−:: List of endpoints
219+ {$them wire (unit hiss)}
280−::
220+ {$hiss wire {$~ $~} $httr {$hiss hiss}}
281−++ places
221+ ==
282− |= wir/wire
283− ^- (list place:connector)
284− =+ (helpers:connector ost.hid wir "https://api.github.com")
285− => |% :: gh-specific helpers
286− ++ sigh-list-issues-x
287− |= jon/json
288− %+ bind ((ar:jo issue:gh-parse) jon)
289− |= issues/(list issue:gh)
290− gh-list-issues+issues
291− ::
292− ++ sigh-list-issues-y
293− |= jon/json
294− %+ bind ((ar:jo issue:gh-parse) jon)
295− |= issues/(list issue:gh)
296− :- `(shax (jam issues))
297− (malt (turn issues |=(issue:gh [(rsh 3 2 (scot %ui number)) ~])))
298− --
299− :~ ^- place :: /
300− :* guard=$~
301− read-x=read-null
302− read-y=(read-static %issues ~)
303− sigh-x=sigh-strange
304− sigh-y=sigh-strange
305− ==
306− ^- place :: /issues
307− :* guard={$issues $~}
308− read-x=read-null
309− read-y=(read-static %mine %by-repo ~)
310− sigh-x=sigh-strange
311− sigh-y=sigh-strange
312− ==
313− ^- place :: /issues/mine
314− :* guard={$issues $mine $~}
315− read-x=(read-get /issues)
316− read-y=(read-get /issues)
317− sigh-x=sigh-list-issues-x
318− sigh-y=sigh-list-issues-y
319− ==
320− ^- place :: /issues/by-repo
321− :* guard={$issues $by-repo $~}
322− read-x=read-null
323− ^= read-y
324− |= pax/path
325− =+ /(scot %p our.hid)/home/(scot %da now.hid)/web/plan
326− =+ .^({* acc/(map knot plan-acct)} %cx -)
327 ::
222 ::
328− ((read-static usr:(~(got by acc) %github) ~) pax)
223+ :: Types of results we produce to subscribers.
329− sigh-x=sigh-strange
330− sigh-y=sigh-strange
331− ==
332− ^- place :: /issues/by-repo/<user>
333− :* guard={$issues $by-repo @t $~}
334− read-x=read-null
335− read-y=|=(pax/path (get /users/[-.+>.pax]/repos))
336− sigh-x=sigh-strange
337− ^= sigh-y
338− |= jon/json
339− %+ bind ((ar:jo repository:gh-parse) jon)
340− |= repos/(list repository:gh)
341− [~ (malt (turn repos |=(repository:gh [name ~])))]
342− ==
343− ^- place :: /issues/by-repo/<user>/<repo>
344− :* guard={$issues $by-repo @t @t $~}
345− read-x=|=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
346− read-y=|=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
347− sigh-x=sigh-list-issues-x
348− sigh-y=sigh-list-issues-y
349− ==
350− ^- place :: /issues/by-repo/<user>/<repo>
351− :* guard={$issues $by-repo @t @t @t $~}
352− ^= read-x
353− |=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues/[-.+>+>.pax]))
354 ::
224 ::
355− read-y=(read-static ~)
225+ ++ sub-result
356− ^= sigh-x
226+ $% {$arch arch}
357− |= jon/json
227+ {$gh-issue issue:gh}
358− %+ bind (issue:gh-parse jon)
228+ {$gh-list-issues (list issue:gh)}
359− |= issue/issue:gh
229+ {$gh-issues issues:gh}
360− gh-issue+issue
230+ {$gh-issue-comment issue-comment:gh}
231+ {$json json}
232+ {$null $~}
233+ ==
361 ::
234 ::
362− sigh-y=sigh-strange
235+ :: Types of webhooks we expect.
363− ==
364− ==
365−::
366−:: When a peek on a path blocks, ford turns it into a peer on
367−:: /scry/{care}/{path}. You can also just peer to this
368−:: directly.
369−::
236+ ::
370−:: We hand control to ++scry.
237+ ++ hook-response
371−::
238+ $% {$gh-issues issues:gh}
372−++ peer-scry
239+ {$gh-issue-comment issue-comment:gh}
373− |= pax/path
240+ ==
374− ^- {(list move) _+>.$}
241+ --
375− ?> ?=({care *} pax)
242+ =+ connector=(connector move sub-result) :: Set up connector library
376− :_ +>.$ :_ ~
243+ ::
377− (read:connector ost.hid (places %read pax) i.pax t.pax)
244+ |_ $: hid/bowl
378−::
379−:: HTTP response. We make sure the response is good, then
380−:: produce the result (as JSON) to whoever sent the request.
381−::
382−++ sigh-httr
383− |= {way/wire res/httr}
384− ^- {(list move) _+>.$}
385− ?. ?=({$read care @ *} way)
386− ~& res=res
387− [~ +>.$]
388− =* style i.way
389− =* ren i.t.way
390− =* pax t.t.way
391− :_ +>.$ :_ ~
392− :+ ost.hid %diff
393− (sigh:connector (places ren style pax) ren pax res)
394−::
395−:: HTTP error. We just print it out, though maybe we should
396−:: also produce a result so that the request doesn't hang?
397−::
398−++ sigh-tang
399− |= {way/wire tan/tang}
400− ^- {(list move) _+>.$}
401− %- (slog >%gh-sigh-tang< tan)
402− [[ost.hid %diff null+~]~ +>.$]
403−::
404−:: We can't actually give the response to pretty much anything
405−:: without blocking, so we just block unconditionally.
406−::
407−++ peek
408− |= {ren/@tas tyl/path}
409− ^- (unit (unit (pair mark *)))
410− ~ ::``noun/[ren tyl]
411−::
412−::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
413−:: Webhook-powered event streams (/listen) ::
414−::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
415−::
416−:: To listen to a webhook-powered stream of events, subscribe
417−:: to /listen/<user>/<repo>/<events...>
418−::
419−:: We hand control to ++listen.
420−::
421−++ peer-listen
422− |= pax/path
423− ^- {(list move) _+>.$}
424− ?. ?=({@ @ *} pax)
425− ~& [%bad-listen-path pax]
426− [~ +>.$]
427− (listen pax)
428−::
429−:: This core handles event subscription requests by starting or
430−:: updating the webhook flow for each event.
431−::
432−++ listen
433− |= pax/path
434− =| mow/(list move)
435− =< abet:listen
436− |%
437− ++ abet :: Resolve core.
438− ^- {(list move) _+>.$}
439− [(flop mow) +>.$]
440− ::
441− ++ send-hiss :: Send a hiss
442− |= hiz/hiss
443− ^+ +>
444− =+ wir=`wire`[%x %listen pax]
445− +>.$(mow [[ost.hid %hiss wir `~ %httr [%hiss hiz]] mow])
446− ::
447− :: Create or update a webhook to listen for a set of events.
448− ::
449− ++ listen
450− ^+ .
451− =+ pax=pax :: TMI-proofing
452− ?> ?=({@ @ *} pax)
453− =+ events=t.t.pax
454− |- ^+ +>+.$
455− ?~ events
456− +>+.$
457− ?: (~(has by hook) i.events)
458− $(+>+ (update-hook i.events), events t.events)
459− $(+>+ (create-hook i.events), events t.events)
460− ::
461− :: Set up a webhook.
462− ::
463− ++ create-hook
464− |= event/@t
465− ^+ +>
466− ?> ?=({@ @ *} pax)
467− =+ clean-event=`tape`(turn (trip event) |=(a/@tD ?:(=('_' a) '-' a)))
468− =. hook
469− %+ ~(put by hook) (crip clean-event)
470− =+ %+ fall
471− (~(get by hook) (crip clean-event))
472− *{id/@t listeners/(set bone)}
245+ hook/(map @t {id/@t listeners/(set bone)}) :: map events to listeners
473− [id (~(put in listeners) ost.hid)]
246+ ==
474− %- send-hiss
247+ :: ++ prep _`. :: Clear state when code changes
475− :* %+ scan
248+ ::
476− =+ [(trip i.pax) (trip i.t.pax)]
249+ :: List of endpoints
250+ ::
251+ ++ places
252+ |= wir/wire
253+ ^- (list place:connector)
477− "https://api.github.com/repos/{-<}/{->}/hooks"
254+ =+ (helpers:connector ost.hid wir "https://api.github.com")
478− auri:epur
255+ => |% :: gh-specific helpers
479− %post ~ ~
256+ ++ sigh-list-issues-x
480− %- taco %- crip %- pojo %- jobe :~
257+ |= jon/json
481− name+s+%web
258+ %+ bind ((ar:jo issue:gh-parse) jon)
482− active+b+&
259+ |= issues/(list issue:gh)
483− events+a+~[s+event] ::(turn `(list ,@t)`t.t.pax |=(a=@t s/a))
260+ gh-list-issues+issues
484− :- %config
261+ ::
485− %- jobe :~
262+ ++ sigh-list-issues-y
486− =+ =+ clean-event
263+ |= jon/json
487− "http://107.170.195.5:8443/~/to/gh/gh-{-}.json?anon&wire=/"
264+ %+ bind ((ar:jo issue:gh-parse) jon)
488− [%url s+(crip -)]
265+ |= issues/(list issue:gh)
489− [%'content_type' s+%json]
266+ :- `(shax (jam issues))
267+ (malt (turn issues |=(issue:gh [(rsh 3 2 (scot %ui number)) ~])))
268+ --
269+ :~ ^- place :: /
270+ :* guard=$~
271+ read-x=read-null
272+ read-y=(read-static %issues ~)
273+ sigh-x=sigh-strange
274+ sigh-y=sigh-strange
275+ ==
276+ ^- place :: /issues
277+ :* guard={$issues $~}
278+ read-x=read-null
279+ read-y=(read-static %mine %by-repo ~)
280+ sigh-x=sigh-strange
281+ sigh-y=sigh-strange
282+ ==
283+ ^- place :: /issues/mine
284+ :* guard={$issues $mine $~}
285+ read-x=(read-get /issues)
286+ read-y=(read-get /issues)
287+ sigh-x=sigh-list-issues-x
288+ sigh-y=sigh-list-issues-y
289+ ==
290+ ^- place :: /issues/by-repo
291+ :* guard={$issues $by-repo $~}
292+ read-x=read-null
293+ ^= read-y
294+ |= pax/path
295+ =+ /(scot %p our.hid)/home/(scot %da now.hid)/web/plan
296+ =+ .^({* acc/(map knot plan-acct)} %cx -)
297+ ::
298+ ((read-static usr:(~(got by acc) %github) ~) pax)
299+ sigh-x=sigh-strange
300+ sigh-y=sigh-strange
301+ ==
302+ ^- place :: /issues/by-repo/<user>
303+ :* guard={$issues $by-repo @t $~}
304+ read-x=read-null
305+ read-y=|=(pax/path (get /users/[-.+>.pax]/repos))
306+ sigh-x=sigh-strange
307+ ^= sigh-y
308+ |= jon/json
309+ %+ bind ((ar:jo repository:gh-parse) jon)
310+ |= repos/(list repository:gh)
311+ [~ (malt (turn repos |=(repository:gh [name ~])))]
312+ ==
313+ ^- place :: /issues/by-repo/<user>/<repo>
314+ :* guard={$issues $by-repo @t @t $~}
315+ read-x=|=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
316+ read-y=|=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
317+ sigh-x=sigh-list-issues-x
318+ sigh-y=sigh-list-issues-y
490 ==
319 ==
320+ ^- place :: /issues/by-repo/<user>/<repo>
321+ :* guard={$issues $by-repo @t @t @t $~}
322+ ^= read-x
323+ |=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues/[-.+>+>.pax]))
324+ ::
325+ read-y=(read-static ~)
326+ ^= sigh-x
327+ |= jon/json
328+ %+ bind (issue:gh-parse jon)
329+ |= issue/issue:gh
330+ gh-issue+issue
331+ ::
332+ sigh-y=sigh-strange
333+ ==
334+ ==
335+ ::
336+ :: When a peek on a path blocks, ford turns it into a peer on
337+ :: /scry/{care}/{path}. You can also just peer to this
338+ :: directly.
339+ ::
340+ :: We hand control to ++scry.
341+ ::
342+ ++ peer-scry
343+ |= pax/path
344+ ^- {(list move) _+>.$}
345+ ?> ?=({care *} pax)
346+ :_ +>.$ :_ ~
347+ (read:connector ost.hid (places %read pax) i.pax t.pax)
348+ ::
349+ :: HTTP response. We make sure the response is good, then
350+ :: produce the result (as JSON) to whoever sent the request.
351+ ::
352+ ++ sigh-httr
353+ |= {way/wire res/httr}
354+ ^- {(list move) _+>.$}
355+ ?. ?=({$read care @ *} way)
356+ ~& res=res
357+ [~ +>.$]
358+ =* style i.way
359+ =* ren i.t.way
360+ =* pax t.t.way
361+ :_ +>.$ :_ ~
362+ :+ ost.hid %diff
363+ (sigh:connector (places ren style pax) ren pax res)
364+ ::
365+ :: HTTP error. We just print it out, though maybe we should
366+ :: also produce a result so that the request doesn't hang?
367+ ::
368+ ++ sigh-tang
369+ |= {way/wire tan/tang}
370+ ^- {(list move) _+>.$}
371+ %- (slog >%gh-sigh-tang< tan)
372+ [[ost.hid %diff null+~]~ +>.$]
373+ ::
374+ :: We can't actually give the response to pretty much anything
375+ :: without blocking, so we just block unconditionally.
376+ ::
377+ ++ peek
378+ |= {ren/@tas tyl/path}
379+ ^- (unit (unit (pair mark *)))
380+ ~ ::``noun/[ren tyl]
381+ ::
382+ ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
383+ :: Webhook-powered event streams (/listen) ::
384+ ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
385+ ::
386+ :: To listen to a webhook-powered stream of events, subscribe
387+ :: to /listen/<user>/<repo>/<events...>
388+ ::
389+ :: We hand control to ++listen.
390+ ::
391+ ++ peer-listen
392+ |= pax/path
393+ ^- {(list move) _+>.$}
394+ ?. ?=({@ @ *} pax)
395+ ~& [%bad-listen-path pax]
396+ [~ +>.$]
397+ (listen pax)
398+ ::
399+ :: This core handles event subscription requests by starting or
400+ :: updating the webhook flow for each event.
401+ ::
402+ ++ listen
403+ |= pax/path
404+ =| mow/(list move)
405+ =< abet:listen
406+ |%
407+ ++ abet :: Resolve core.
408+ ^- {(list move) _+>.$}
409+ [(flop mow) +>.$]
410+ ::
411+ ++ send-hiss :: Send a hiss
412+ |= hiz/hiss
413+ ^+ +>
414+ =+ wir=`wire`[%x %listen pax]
415+ +>.$(mow [[ost.hid %hiss wir `~ %httr [%hiss hiz]] mow])
416+ ::
417+ :: Create or update a webhook to listen for a set of events.
418+ ::
419+ ++ listen
420+ ^+ .
421+ =+ pax=pax :: TMI-proofing
422+ ?> ?=({@ @ *} pax)
423+ =+ events=t.t.pax
424+ |- ^+ +>+.$
425+ ?~ events
426+ +>+.$
427+ ?: (~(has by hook) i.events)
428+ $(+>+ (update-hook i.events), events t.events)
429+ $(+>+ (create-hook i.events), events t.events)
430+ ::
431+ :: Set up a webhook.
432+ ::
433+ ++ create-hook
434+ |= event/@t
435+ ^+ +>
436+ ?> ?=({@ @ *} pax)
437+ =+ clean-event=`tape`(turn (trip event) |=(a/@tD ?:(=('_' a) '-' a)))
438+ =. hook
439+ %+ ~(put by hook) (crip clean-event)
440+ =+ %+ fall
441+ (~(get by hook) (crip clean-event))
442+ *{id/@t listeners/(set bone)}
443+ [id (~(put in listeners) ost.hid)]
444+ %- send-hiss
445+ :* %+ scan
446+ =+ [(trip i.pax) (trip i.t.pax)]
447+ "https://api.github.com/repos/{-<}/{->}/hooks"
448+ auri:epur
449+ %post ~ ~
450+ %- taco %- crip %- pojo %- jobe :~
451+ name+s+%web
452+ active+b+&
453+ events+a+~[s+event] ::(turn `(list ,@t)`t.t.pax |=(a=@t s/a))
454+ :- %config
455+ %- jobe :~
456+ =+ =+ clean-event
457+ "http://107.170.195.5:8443/~/to/gh/gh-{-}.json?anon&wire=/"
458+ [%url s+(crip -)]
459+ [%'content_type' s+%json]
460+ ==
461+ ==
462+ ==
463+ ::
464+ :: Add current bone to the list of subscribers for this event.
465+ ::
466+ ++ update-hook
467+ |= event/@t
468+ ^+ +>
469+ =+ hok=(~(got by hook) event)
470+ %_ +>.$
471+ hook
472+ %+ ~(put by hook) event
473+ hok(listeners (~(put in listeners.hok) ost.hid))
491 ==
474 ==
492− ==
493− ::
494− :: Add current bone to the list of subscribers for this event.
495− ::
496− ++ update-hook
497− |= event/@t
498− ^+ +>
499− =+ hok=(~(got by hook) event)
500− %_ +>.$
501− hook
502− %+ ~(put by hook) event
503− hok(listeners (~(put in listeners.hok) ost.hid))
504− ==
505− --
475+ --
506−::
476+ ::
507−:: Pokes that aren't caught in more specific arms are handled
477+ :: Pokes that aren't caught in more specific arms are handled
508−:: here. These should be only from webhooks firing, so if we
478+ :: here. These should be only from webhooks firing, so if we
509−:: get any mark that we shouldn't get from a webhook, we reject
479+ :: get any mark that we shouldn't get from a webhook, we reject
510−:: it. Otherwise, we spam out the event to everyone who's
480+ :: it. Otherwise, we spam out the event to everyone who's
511−:: listening for that event.
481+ :: listening for that event.
512−::
482+ ::
513−++ poke
483+ ++ poke
514− |= response/hook-response
484+ |= response/hook-response
515− ^- {(list move) _+>.$}
485+ ^- {(list move) _+>.$}
516− =+ hook-data=(~(get by hook) (rsh 3 3 -.response))
486+ =+ hook-data=(~(get by hook) (rsh 3 3 -.response))
517− ?~ hook-data
487+ ?~ hook-data
518− ~& [%strange-hook hook response]
488+ ~& [%strange-hook hook response]
519− [~ +>.$]
489+ [~ +>.$]
520− :: ~& response=response
490+ :: ~& response=response
521− :_ +>.$
491+ :_ +>.$
522− %+ turn (~(tap in listeners.u.hook-data))
492+ %+ turn (~(tap in listeners.u.hook-data))
523− |= ost/bone
493+ |= ost/bone
524− [ost %diff response]
494+ [ost %diff response]
525−--
495+ --
526−```
docs/arvo/basic.md+103−113
@@ -1,49 +1,43 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3+next: True
3sort: 11
4sort: 11
4−next: true
5title: Basic Hoon
5title: Basic Hoon
6---
6---
7
7
8−Basic Hoon
8+# Basic Hoon
9−==========
10
9
11−Our goal is to get you programming interesting and useful things as soon
10+Our goal is to get you programming interesting and useful things as soon as
12−as possible. To get there, we have to quickly cover some of the
11+possible. To get there, we have to quickly cover some of the fundamentals of
13−fundamentals of [Hoon](/docs/hoon/). To do this, we'll walk through two simple
12+[Hoon](/docs/hoon/). To do this, we'll walk through two simple programs: the
14−programs: the first [Project Euler](https://projecteuler.net/) problem
13+first [Project Euler](https://projecteuler.net/) problem and
15−and [fizzbuzz](https://en.wikipedia.org/wiki/Fizz_buzz).
14+[fizzbuzz](https://en.wikipedia.org/wiki/Fizz_buzz).
16
15
17−To run this code, you'll need an urbit, and will have to copy the
16+To run this code, you'll need an urbit, and will have to copy the [examples
18−[examples repo](https://github.com/urbit/examples) into it.
17+repo](https://github.com/urbit/examples) into it.
19
18
20If you haven't installed urbit yet, check out the [installation
19If you haven't installed urbit yet, check out the [installation
21−instructions](/docs/using/install). Once urbit is installed, take a look
20+instructions](/docs/using/install). Once urbit is installed, take a look at the
22−at the [basic operation](/docs/using/admin) of your urbit.
21+[basic operation](/docs/using/admin) of your urbit.
23
22
24−If you haven't copied in the examples repo, running the following
23+If you haven't copied in the examples repo, running the following commands from
25−commands from your urbit directory should do the trick:
24+your urbit directory should do the trick:
26
25
27Make sure you've mounted your `%home` desk:
26Make sure you've mounted your `%home` desk:
28
27
29−```
30−~fintud-macrep:dojo> |mount %
28+ ~fintud-macrep:dojo> |mount %
31−```
32
29
33Then, copy the example into it
30Then, copy the example into it
34
31
35−```
36−$ cp -r {urbit-examples}/gall/*/* {your-pier}/home/
32+ $ cp -r {urbit-examples}/gall/*/* {your-pier}/home/
37−$ cp -r {urbit-examples}/dojo/*/* {your-pier}/home/
33+ $ cp -r {urbit-examples}/dojo/*/* {your-pier}/home/
38−```
39
34
40Run an example to ensure it worked:
35Run an example to ensure it worked:
41
36
42 ~fintud-macrep:dojo> +examples-euler1
37 ~fintud-macrep:dojo> +examples-euler1
43 233.168
38 233.168
44
39
45−Euler 1
40+## Euler 1
46−-------
47
41
48Let's check out the code for Euler 1. First, the problem:
42Let's check out the code for Euler 1. First, the problem:
49
43
@@ -91,11 +85,11 @@ Here is the hoon solution (which should be in your pier directory under
91 (add (five a) (three a))
85 (add (five a) (three a))
92 --
86 --
93
87
94−> Hoon is not generally whitespace sensitive, but we do have two
88+> Hoon is not generally whitespace sensitive, but we do have two different kinds
95−> different kinds of whitespace: a single space and a gap, which is two
89+> of whitespace: a single space and a gap, which is two or more spaces or a
96−> or more spaces or a linebreak. Tabs are taboo. Do not use them.
90+> linebreak. Tabs are taboo. Do not use them. Really. For a more detailed
97−> Really. For a more detailed explanation of when to use spaces vs.
98−> gaps, see the syntax section before the first exercises.
91+> explanation of when to use spaces vs. gaps, see the syntax section before the
92+> first exercises.
99
93
100### Lines 1-11:
94### Lines 1-11:
101
95
@@ -106,31 +100,29 @@ Any line that begins with `::` is a comment.
106 =< (sum 1.000)
100 =< (sum 1.000)
107
101
108All you need to know about the lines above is that they call the `++sum`
102All you need to know about the lines above is that they call the `++sum`
109−function with an argument of `1.000`. We'll cover them in more detail
103+function with an argument of `1.000`. We'll cover them in more detail later.
110−later.
111
104
112### How to form expressions
105### How to form expressions
113
106
114Hoon does not typically use reserved words to form expressions. Instead,
107Hoon does not typically use reserved words to form expressions. Instead,
115−expressions are formed with *runes*: digraphs of two ASCII symbols. Each
108+expressions are formed with *runes*: digraphs of two ASCII symbols. Each rune
116−rune takes a specific number of children, either expressions formed by
109+takes a specific number of children, either expressions formed by other runes or
117−other runes or literals that produce their own value (some runes take
110+literals that produce their own value (some runes take *N* children, and are
118−*N* children, and are usually closed with `==`).
111+usually closed with `==`).
119
112
120−For example, the rune `?:` from line 18 is the classic 'if-then-else'
113+For example, the rune `?:` from line 18 is the classic 'if-then-else' statement,
121−statement, and thus takes three children:
114+and thus takes three children:
122
115
123 ?: (lth a b) :: if first child evals to true
116 ?: (lth a b) :: if first child evals to true
124 0 :: then produce result of second
117 0 :: then produce result of second
125 (add b $(b (add 3 b))) :: else, produce result of third
118 (add b $(b (add 3 b))) :: else, produce result of third
126
119
127Since runes are such a fundamental structure in Hoon, we found ourselves
120Since runes are such a fundamental structure in Hoon, we found ourselves
128−speaking them out loud frequently. It quickly grew cumbersome to have to
121+speaking them out loud frequently. It quickly grew cumbersome to have to say
129−say "question mark, colon" to describe `?:`. To alleviate this problem,
122+"question mark, colon" to describe `?:`. To alleviate this problem, we came up
130−we came up with our own naming scheme: each ASCII glyph has a
123+with our own naming scheme: each ASCII glyph has a single-syllable pronunciation
131−single-syllable pronunciation phonetically designed to be both easily
124+phonetically designed to be both easily remembered and easily pronounced in
132−remembered and easily pronounced in conjunction with the other glyphs
125+conjunction with the other glyphs (when forming a rune).
133−(when forming a rune).
134
126
135See the entire naming scheme below.
127See the entire naming scheme below.
136
128
@@ -150,9 +142,8 @@ See the entire naming scheme below.
150Using this scheme, we would pronounce `?:` as 'wutcol'.
142Using this scheme, we would pronounce `?:` as 'wutcol'.
151
143
152For those who prefer to use reserved words to form expressions, you can
144For those who prefer to use reserved words to form expressions, you can
153−substitute any rune with that rune's keyword, which you can find in
145+substitute any rune with that rune's keyword, which you can find in `++twig` in
154−`++twig` in `/arvo/hoon.hoon`. For example, here's the code for `?:` in
146+`/arvo/hoon.hoon`. For example, here's the code for `?:` in `++twig`:
155−`++twig`:
156
147
157`{$if p/twig q/twig r/twig} :: ?: if/then/else`
148`{$if p/twig q/twig r/twig} :: ?: if/then/else`
158
149
@@ -168,16 +159,16 @@ You can write:
168
159
169### Lines 13-35
160### Lines 13-35
170
161
171−Now let's quickly walk through this code line-by-line. Lines 13-35 are
162+Now let's quickly walk through this code line-by-line. Lines 13-35 are wrapped
172−wrapped in a `|%` ('[barcen](../../hoon/twig/bar-core/cen-core/)'), which produces a core. Cores are a
163+in a `|%` ('[barcen](../../hoon/twig/bar-core/cen-core/)'), which produces a
173−fundamental datatype in Hoon, similar to a struct, class, or object. A
164+core. Cores are a fundamental datatype in Hoon, similar to a struct, class, or
174−core is just a map of names to any kind of code, whether it be functions
165+object. A core is just a map of names to any kind of code, whether it be
175−or data. Each element in this map begins with a `++` followed by the
166+functions or data. Each element in this map begins with a `++` followed by the
176name and the corresponding code. Since `|%` takes an arbitrary number of
167name and the corresponding code. Since `|%` takes an arbitrary number of
177children, it needs to be closed with a `--`.
168children, it needs to be closed with a `--`.
178
169
179−> `++` ('luslus') is not technically a rune, since it is only used in
170+> `++` ('luslus') is not technically a rune, since it is only used in core
180−> core syntax, as shown above.
171+> syntax, as shown above.
181
172
182Let's step into each of the three arms within our core.
173Let's step into each of the three arms within our core.
183
174
@@ -188,17 +179,16 @@ Let's step into each of the three arms within our core.
188 (add (five a) (three a))
179 (add (five a) (three a))
189 --
180 --
190
181
191−`|=` ('[bartis](../../hoon/twig/bar-core/tis-gate/)') produces a function, much like a lambda in lisp. It
182+`|=` ('[bartis](../../hoon/twig/bar-core/tis-gate/)') produces a function, much
192−takes two children:
183+like a lambda in lisp. It takes two children:
193
184
194−1. A set of argument(s). In this case our argument set only contains
185+1. A set of argument(s). In this case our argument set only contains one: `a`,
195− one: `a`, which is required to be an atom or natural number, denoted
186+ which is required to be an atom or natural number, denoted by `@`.
196− by `@`.
197
187
198−2. The body of the function itself, which is executed when the function
188+2. The body of the function itself, which is executed when the function is
199− is called (in this case, with `(sum 1.000)`). This particular
189+ called (in this case, with `(sum 1.000)`). This particular function adds the
200− function adds the results of evaluating the gates `++ five` and
190+ results of evaluating the gates `++ five` and `++three` with each of their
201− `++three` with each of their respective input parameters set to `a`.
191+ respective input parameters set to `a`.
202
192
203### ++ three
193### ++ three
204
194
@@ -210,17 +200,17 @@ takes two children:
210 0
200 0
211 (add b $(b (add 3 b)))
201 (add b $(b (add 3 b)))
212
202
213−As above, `++three` takes an integer argument, `a`, and then executes
203+As above, `++three` takes an integer argument, `a`, and then executes the
214−the remainder of the code with `a` set to the actual arguments.
204+remainder of the code with `a` set to the actual arguments.
215
205
216−Similarly, `=|` ('[tisbar](../../hoon/twig/tis-flow/bar-new/)') pushes its first child, `b` into our context
206+Similarly, `=|` ('[tisbar](../../hoon/twig/tis-flow/bar-new/)') pushes its first
217−(in other words, it declares a variable `b`) and executes the remainder
207+child, `b` into our context (in other words, it declares a variable `b`) and
218−of the code. However, `b` is not an argument; `=|` sets `b` to the
208+executes the remainder of the code. However, `b` is not an argument; `=|` sets
219−default value of whatever type it is declared as. Since the default
209+`b` to the default value of whatever type it is declared as. Since the default
220value of an atom is `0`, b is set to `0`.
210value of an atom is `0`, b is set to `0`.
221
211
222−> To produce the default value of any given type, use `*` ('tar')
212+> To produce the default value of any given type, use `*` ('tar') followed by
223−> followed by the type. This operaction is called *bunt*.
213+> the type. This operaction is called *bunt*.
224>
214>
225> > *@ :: produce the default value of atom @
215> > *@ :: produce the default value of atom @
226> 0
216> 0
@@ -228,25 +218,24 @@ value of an atom is `0`, b is set to `0`.
228> > *@p :: produce the phonemic base of atom @
218> > *@p :: produce the phonemic base of atom @
229> ~zod
219> ~zod
230
220
231−So now we have two variables: `a` is set to our input, and `b` is
221+So now we have two variables: `a` is set to our input, and `b` is initialized to
232−initialized to `0`.
222+`0`.
233
223
234−One way to think about `|-` ('[barhep](../../hoon/twig/bar-core/hep-loop/)') is that it lays down a recursion
224+One way to think about `|-` ('[barhep](../../hoon/twig/bar-core/hep-loop/)') is
235−point. More on this later.
225+that it lays down a recursion point. More on this later.
236
226
237−`^-` ('[kethep](../../hoon/core/ket-cast/hep-cast/)') is just a cast that sets the result of the remainder of
227+`^-` ('[kethep](../../hoon/core/ket-cast/hep-cast/)') is just a cast that sets
238−the code to an unsigned integer, `@u`.
228+the result of the remainder of the code to an unsigned integer, `@u`.
239
229
240−In pseudocode, the last three lines read like this: if `a` is less than
230+In pseudocode, the last three lines read like this: if `a` is less than `b`,
241−`b`, produce zero. Else, add `b` to the result of rerunning the segment
231+produce zero. Else, add `b` to the result of rerunning the segment of the
242−of the function following the `|-` with the value of `b` changed to `b`
232+function following the `|-` with the value of `b` changed to `b` plus three.
243−plus three.
244
233
245The only thing that should look completely unfamiliar to you here is the
234The only thing that should look completely unfamiliar to you here is the
246−`$(b (add 3 b))`, which causes us to recurse back to our last recursion
235+`$(b (add 3 b))`, which causes us to recurse back to our last recursion point
247−point with the value of `b` set to `(add 3 b)`. Note that we only
236+with the value of `b` set to `(add 3 b)`. Note that we only specify what changes
248−specify what changes (`b` in this case). If you recurse by an actual
237+(`b` in this case). If you recurse by an actual function call, then you have to
249−function call, then you have to specify every argument.
238+specify every argument.
250
239
251> If you're familiar with Clojure, `|-` is `loop` and `$()` is `recur`.
240> If you're familiar with Clojure, `|-` is `loop` and `$()` is `recur`.
252
241
@@ -254,18 +243,18 @@ function call, then you have to specify every argument.
254
243
255Tweak your code to complete the following exercises.
244Tweak your code to complete the following exercises.
256
245
257−There are a few runes and some syntax that we have yet to cover that you
246+There are a few runes and some syntax that we have yet to cover that you will
258−will need to complete the exercises below. For these, please refer to
247+need to complete the exercises below. For these, please refer to our cheatsheat
259−our cheatsheat at the bottom.
248+at the bottom.
260
249
2611. Read and understand `++five` line by line.
2501. Read and understand `++five` line by line.
262
251
263−2. Change `++sum` to accept two variables, `a` and `b`. Pass `a` to
252+2. Change `++sum` to accept two variables, `a` and `b`. Pass `a` to `three` and
264− `three` and `b` to `five`. Then run the code with `a` set to `1.000`
253+ `b` to `five`. Then run the code with `a` set to `1.000` and `b` set to
265− and `b` set to `2.000`.
254+ `2.000`.
266
255
267−3. Check if this new result is under one thousand. If it is, return the
256+3. Check if this new result is under one thousand. If it is, return the string
268− string `'result < 1.000'`. If not, return `'result >= 1.000'`.
257+ `'result < 1.000'`. If not, return `'result >= 1.000'`.
269
258
270### Review
259### Review
271
260
@@ -278,39 +267,40 @@ our cheatsheat at the bottom.
278 =(a b) test equality
267 =(a b) test equality
279 (function args ...) call function with args
268 (function args ...) call function with args
280
269
281−Lookup each of these expressions (and all others!) in the [Twig Expressions](../../hoon/twig/) definition.
270+Lookup each of these expressions (and all others!) in the [Twig
271+Expressions](../../hoon/twig/) definition.
282
272
283### New material
273### New material
284
274
2851. There are two syntaxes for writing Hoon: tallform and wideform.
2751. There are two syntaxes for writing Hoon: tallform and wideform.
286
276
287− In tallform, expressions are formed with either two spaces or a line
277+ In tallform, expressions are formed with either two spaces or a line break
288− break separating both a rune from its children and each of its children
278+ separating both a rune from its children and each of its children from one
289− from one another. We use tallform when writing multiline expressions.
279+ another. We use tallform when writing multiline expressions.
290
280
291 For more concise expressions, we use wideform, which is always a single
281 For more concise expressions, we use wideform, which is always a single
292− line. Wideform can be used inside tallform expressions, but not vice
282+ line. Wideform can be used inside tallform expressions, but not vice versa.
293− versa.
294
283
295− Wideform expressions are formed with a rune followed by `()` containing
284+ Wideform expressions are formed with a rune followed by `()` containing its
296− its children, all of which are separated by a single space. For example,
285+ children, all of which are separated by a single space. For example, to make
297− to make a cell of two elements, use `:-(a b)`.
286+ a cell of two elements, use `:-(a b)`.
298
287
299− We've already seen wideform in action, for example with
288+ We've already seen wideform in action, for example with `=((mod b 3) 0)`. In
300− `=((mod b 3) 0)`. In this case, `=` is actually an irregular form of
289+ this case, `=` is actually an irregular form of `.=`, which tests its two
301− `.=`, which tests its two children for equality.
290+ children for equality.
302
291
303 Surrounding a function with `()` is an irregular wide form syntax for
292 Surrounding a function with `()` is an irregular wide form syntax for
304 calling a function with *N* arguments. More on this later.
293 calling a function with *N* arguments. More on this later.
305
294
295+2. For a set of multiple arguments following `|=`
306−2. For a set of multiple arguments following `|=` ('[bartis](../../hoon/twig/bar-core/tis-gate/)'), use `{` 'kel' and `}` 'ker.' For example: `{a/@u b/@u}`.
296+ ('[bartis](../../hoon/twig/bar-core/tis-gate/)'), use `{` 'kel' and `}`
297+ 'ker.' For example: `{a/@u b/@u}`.
307
298
308−3. `:-` makes a cell of values. The irregular wide form of this is
299+3. `:-` makes a cell of values. The irregular wide form of this is `[a b]`,
309− `[a b]`, with two expressions separated by a single space. While the
300+ with two expressions separated by a single space. While the regular form of
310− regular form of this rune takes a fixed number of children (two),
301+ this rune takes a fixed number of children (two), its irregular wide form
311− its irregular wide form can accept *N* expressions: `[a1..an]`.
302+ can accept *N* expressions: `[a1..an]`.
312
303
313−4. Cords are one datatype for text in Hoon. They're just a big atom
304+4. Cords are one datatype for text in Hoon. They're just a big atom formed from
314− formed from adjacent unicode bytes -- a "c string". To produce a
305+ adjacent unicode bytes -- a "c string". To produce a cord, enclose text
315− cord, enclose text within single quotes. To set the type of an
306+ within single quotes. To set the type of an argument to a cord, use `@t`.
316− argument to a cord, use `@t`.
docs/arvo/generators.md+76−86
@@ -7,66 +7,60 @@ title: Using generators with apps
7
7
8# Generators
8# Generators
9
9
10−Up until now we've poked apps directly. This requires the user to
10+Up until now we've poked apps directly. This requires the user to specify the
11−specify the mark, and it requires the app to accept the arguments in a
11+mark, and it requires the app to accept the arguments in a way that's convenient
12−way that's convenient for users to input. This is the "plumbing" way to
12+for users to input. This is the "plumbing" way to interact with apps. Generators
13−interact with apps. Generators are the "porcelain" layer. This is why
13+are the "porcelain" layer. This is why when you run a command like `+ls` or
14−when you run a command like `+ls` or `|merge`, there are no marks in
14+`|merge`, there are no marks in sight.
15−sight.
15+
16−
17−We've used generators before, back in [Basic
16+We've used generators before, back in [Basic Operation](/docs/using/admin). At
18−Operation](/docs/using/admin). At that point, we just used the
19−generators to produce values -- we didn't pipe their results into apps.
17+that point, we just used the generators to produce values -- we didn't pipe
20−In the dojo cast, the role of a generator is to take a list of arguments
18+their results into apps. In the dojo cast, the role of a generator is to take a
21−and produce a value, which is often, though not always, piped into an
19+list of arguments and produce a value, which is often, though not always, piped
22−app. Generators are pure, stateless functions, and they cannot send
20+into an app. Generators are pure, stateless functions, and they cannot send
23moves.
21moves.
24
22
25−`+ls` and `+cat` are commonly-used generators that produce a directory
23+`+ls` and `+cat` are commonly-used generators that produce a directory listing
26−listing and print out a file, respectively. These are useful in
24+and print out a file, respectively. These are useful in themselves, so we
27−themselves, so we generally don't pipe the results into an app.
25+generally don't pipe the results into an app.
28
26
29−Another generator is `+hood/merge`. Try running `+hood/merge` with the
27+Another generator is `+hood/merge`. Try running `+hood/merge` with the normal
30−normal arguments for `|merge`:
28+arguments for `|merge`:
31
29
32 ~fintud-macrep:dojo> +hood/merge %home our %they
30 ~fintud-macrep:dojo> +hood/merge %home our %they
33 [syd=%home her=~fintud-macrep sud=%they gem=%auto]
31 [syd=%home her=~fintud-macrep sud=%they gem=%auto]
34 ~fintud-macrep:dojo> +hood/merge %home our %they, =gem %this
32 ~fintud-macrep:dojo> +hood/merge %home our %they, =gem %this
35 [syd=%home her=~fintud-macrep sud=%they gem=%auto]
33 [syd=%home her=~fintud-macrep sud=%they gem=%auto]
36
34
37−This didn't run any merge, it just constructed the command that, if sent
35+This didn't run any merge, it just constructed the command that, if sent to
38−to `:hood`, would run it. Note first that, even though it's not printed,
36+`:hood`, would run it. Note first that, even though it's not printed, this value
39−this value has mark `kiln-merge`. Also, the merge strategy `gem=%auto`
37+has mark `kiln-merge`. Also, the merge strategy `gem=%auto` was added
40−was added automatically. Optional arguments are straightforward with
38+automatically. Optional arguments are straightforward with generators.
41−generators.
39+
42−
43−In general, generators are prepended by either `+` or `|`. The general
40+In general, generators are prepended by either `+` or `|`. The general form is
44−form is `+generator`, but often generators are created specifically to
41+`+generator`, but often generators are created specifically to work with a
45−work with a single app, they're usually placed in a subdirectory of gen
42+single app, they're usually placed in a subdirectory of gen and run with `|`.
46−and run with `|`. Thus, if `my-generator` generator is made for use with
43+Thus, if `my-generator` generator is made for use with `:my-app`, then
47−`:my-app`, then `my-generator` is put in
48−`/gen/my-app/my-generator.hoon`, and can be run directly with
44+`my-generator` is put in `/gen/my-app/my-generator.hoon`, and can be run
49−`+my-app/my-generator <args>`. If you want to pipe the result into
45+directly with `+my-app/my-generator <args>`. If you want to pipe the result into
50−`:my-app`, then run `:my-app +my-app/my-generator <args>`. Because this
46+`:my-app`, then run `:my-app +my-app/my-generator <args>`. Because this pattern
51−pattern is so common, this can be abbreviated to
47+is so common, this can be abbreviated to `:my-app|my-generator <args>`. Because
52−`:my-app|my-generator <args>`. Because most built-in commands are
53−generators for the `:hood` app, `:hood|generator <args>` can be
48+most built-in commands are generators for the `:hood` app,
54−shortened to `|generator <args>`.
49+`:hood|generator <args>` can be shortened to `|generator <args>`.
55−
50+
56−Let's write a generator for a modified version of `:examples-pong` from
51+Let's write a generator for a modified version of `:examples-pong` from the
57−the chapter on [Network Messages](/docs/arvo/system/network). Recall
52+chapter on [Network Messages](/docs/arvo/system/network). Recall that
58−that `:examples-pong` takes an urbit address, which is of mark `urbit`,
53+`:examples-pong` takes an urbit address, which is of mark `urbit`, and sends
59−and sends that urbit the message `'howdy'`. First--without a
54+that urbit the message `'howdy'`. First--without a generator--let's make
60−generator--let's make `:examples-ping` that does the same, except that
55+`:examples-ping` that does the same, except that it lets the user optionally
61−it lets the user optionally specify the message as well.
56+specify the message as well.
62−
57+
63−We'll need a new mark for our arguments. Let's call it
58+We'll need a new mark for our arguments. Let's call it `examples-ping-message`.
64−`examples-ping-message`.
59+
65−
66−> For app-specific marks, it's good style to prefix the name of the mark
60+> For app-specific marks, it's good style to prefix the name of the mark with
67−> with the name of the app. Since many apps have several such marks,
61+> the name of the app. Since many apps have several such marks, subdirectories
68−> subdirectories in `/mar` are rendered as `-`, so that `ping-message`
62+> in `/mar` are rendered as `-`, so that `ping-message` is written in
69−> is written in (/mar/examples/ping/message.hoon).
63+> (/mar/examples/ping/message.hoon).
70
64
71 ::
65 ::
72 :::: /hoon/message/ping/examples/app
66 :::: /hoon/message/ping/examples/app
@@ -119,43 +113,40 @@ Now we can run this with:
119 ~fintud-macrep:dojo> :examples-ping &examples-ping-message [~sampel-sipnym 'heyya']
113 ~fintud-macrep:dojo> :examples-ping &examples-ping-message [~sampel-sipnym 'heyya']
120 >=
114 >=
121
115
122−And on `~sampel-sipnym`, assuming it's running `:examples-ping` as well,
116+And on `~sampel-sipnym`, assuming it's running `:examples-ping` as well, we see
123−we see `[%receiving 'heyya']`.
117+`[%receiving 'heyya']`.
124
118
125−This is an annoying way to invoke the app, though. The `[]` are
119+This is an annoying way to invoke the app, though. The `[]` are mandatory, and
126−mandatory, and optional arguments are hard. Let's make a generator,
120+optional arguments are hard. Let's make a generator, `+examples-send` to make
127−`+examples-send` to make everything nicer. Since it's specific to
121+everything nicer. Since it's specific to `:examples-ping`, let's put it in
128−`:examples-ping`, let's put it in `/gen/examples/ping/send.hoon`:
122+`/gen/examples/ping/send.hoon`:
129
123
130 :- %say
124 :- %say
131 |= {^ {to/@p message/?($~ {text/@t $~})} $~}
125 |= {^ {to/@p message/?($~ {text/@t $~})} $~}
132 [%examples-ping-message to ?~(message 'howdy' text.message)]
126 [%examples-ping-message to ?~(message 'howdy' text.message)]
133
127
134−A couple of new things here. Firstly, `message/?($~ {text/@t $~})`
128+A couple of new things here. Firstly, `message/?($~ {text/@t $~})` should be
135−should be read as "the message is either null or a pair of text and
129+read as "the message is either null or a pair of text and null". Generator
136−null". Generator argument lists are always null-terminated, which makes
130+argument lists are always null-terminated, which makes it convenient to accept
137−it convenient to accept lists in tail position (which are particularly
131+lists in tail position (which are particularly annoying without generators).
138−annoying without generators). `?(a b)` is the irregular form of
139−`$?(a b)`, which is a union type. Thus, `?(a b)` means the type of
132+`?(a b)` is the irregular form of `$?(a b)`, which is a union type. Thus,
140−anything that's in either `a` or `b`. Thus, in our case,
133+`?(a b)` means the type of anything that's in either `a` or `b`. Thus, in our
141−`?(~ [text=@t ~])` is either null or a pair of text and null.
134+case, `?(~ [text=@t ~])` is either null or a pair of text and null.
142−
135+
143−Secondly, `?~(a b c)` is a rune which means "if the `a` is null, do `b`,
136+Secondly, `?~(a b c)` is a rune which means "if the `a` is null, do `b`, else
144−else `c`". It is roughly equivalent to `?:(=(~ a) b c)`.
137+`c`". It is roughly equivalent to `?:(=(~ a) b c)`.
145−
138+
146−> `?~(a b c)` is actually equivalent to `?:=(?=(~ a) b c)`, which,
139+> `?~(a b c)` is actually equivalent to `?:=(?=(~ a) b c)`, which, although
147−> although identical at run time, is subtly different at compile time.
140+> identical at run time, is subtly different at compile time. Specifically,
148−> Specifically, using `?=` rather than `=` means that we're checking
141+> using `?=` rather than `=` means that we're checking whether `a` is in the
149−> whether `a` is in the *type* of `~`, and so the compiler knows that in
142+> *type* of `~`, and so the compiler knows that in the `b` case `a` is null, and
150−> the `b` case `a` is null, and in the `c` case `a` is not null. Since
143+> in the `c` case `a` is not null. Since `=` is purely a runtime value check
151−> `=` is purely a runtime value check with no type implications, the
152−> compiler doesn't gain any information.
144+> with no type implications, the compiler doesn't gain any information.
153>
145>
154−> At any rate, this is the reason why we can refer to `text.message` in
146+> At any rate, this is the reason why we can refer to `text.message` in the `c`
155−> the `c` clause. The compiler knows that `message` is not null, so it
147+> clause. The compiler knows that `message` is not null, so it must have `text`
156−> must have `text` within it. If you used
157−> `?:(=(~ message) 'howdy' text.message)` the compiler would complain
148+> within it. If you used `?:(=(~ message) 'howdy' text.message)` the compiler
158−> that it doesn't know whether `message` has `text` within it.
149+> would complain that it doesn't know whether `message` has `text` within it.
159
150
160This is run as follows:
151This is run as follows:
161
152
@@ -169,10 +160,9 @@ Which causes `~sampel-sipnym` to print `[%receiving 'howdy']` and
169
160
170**Exercises**:
161**Exercises**:
171
162
172−- Create a generator for `:examples-sum` from
163+- Create a generator for `:examples-sum` from [State](/docs/arvo/state) so
173− [State](/docs/arvo/state) so that you can run `:examples-sum|add 5`
164+ that you can run `:examples-sum|add 5` to add numbers to it.
174− to add numbers to it.
175
165
176- Create a generator for `:examples-click` from [Web
166- Create a generator for `:examples-click` from [Web
177− Apps](/docs/arvo/web-apps) so that you can run
167+ Apps](/docs/arvo/web-apps) so that you can run `:examples-click|poke` to
178− `:examples-click|poke` to poke it.
168+ poke it.
docs/arvo/http.md+192−208
@@ -1,230 +1,214 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: false
3+next: False
4sort: 19
4sort: 19
5title: HTTP requests and timers
5title: HTTP requests and timers
6---
6---
7
7
8# Writing an HTTP request
8# Writing an HTTP request
9
9
10−There's a variety of arvo services we haven't touched on yet.
10+There's a variety of arvo services we haven't touched on yet. Let's figure out
11−Let's figure out how to make HTTP requests and set timers.
11+how to make HTTP requests and set timers.
12−
12+
13−Let's build an uptime-monitoring system. It'll ping a web
13+Let's build an uptime-monitoring system. It'll ping a web service periodically
14−service periodically and print out an error when the response
14+and print out an error when the response code isn't 2xx. Here's `app/up.hoon`:
15−code isn't 2xx. Here's `app/up.hoon`:
15+
16−
17−```
18−:: Up-ness monitor. Accepts atom url, 'on', or 'off'
16+ :: Up-ness monitor. Accepts atom url, 'on', or 'off'
19−::
17+ ::
20−:::: /hoon/up/examples/app
18+ :::: /hoon/up/examples/app
21− ::
19+ ::
22−/? 314
20+ /? 314
23−|%
21+ |%
24−++ move {bone card}
22+ ++ move {bone card}
25−++ card
23+ ++ card
26− $% {$hiss wire unit-iden/$~ mark/$httr cage/{mark/$purl vase/purl}}
24+ $% {$hiss wire unit-iden/$~ mark/$httr cage/{mark/$purl vase/purl}}
27− {$wait wire @da}
25+ {$wait wire @da}
28− ==
26+ ==
29−++ action
27+ ++ action
30− $% {$on $~} :: enable polling('on')
28+ $% {$on $~} :: enable polling('on')
31− {$off $~} :: disable polling('off)
29+ {$off $~} :: disable polling('off)
32− {$target p/cord} :: set poll target('http://...')
30+ {$target p/cord} :: set poll target('http://...')
33− ==
31+ ==
34−--
32+ --
35−|_ {hid/bowl on/_| in-progress/_| target/@t}
33+ |_ {hid/bowl on/_| in-progress/_| target/@t}
36−++ poke-atom
34+ ++ poke-atom
37− |= url-or-command/@t ^- (quip move +>)
35+ |= url-or-command/@t ^- (quip move +>)
38− =+ ^- act/action
36+ =+ ^- act/action
39− ?: ?=($on url-or-command) [%on ~]
37+ ?: ?=($on url-or-command) [%on ~]
40− ?: ?=($off url-or-command) [%off ~]
38+ ?: ?=($off url-or-command) [%off ~]
41− [%target url-or-command]
39+ [%target url-or-command]
42− ?- -.act
40+ ?- -.act
43− $target [~ +>.$(target p.act)]
41+ $target [~ +>.$(target p.act)]
44− $off [~ +>.$(on |)]
42+ $off [~ +>.$(on |)]
45− $on
43+ $on
46− :- ?: |(on in-progress) ~
44+ :- ?: |(on in-progress) ~
47− [ost.hid %hiss /request ~ %httr %purl (need (epur target))]~
45+ [ost.hid %hiss /request ~ %httr %purl (need (epur target))]~
48− +>.$(on &, in-progress &)
46+ +>.$(on &, in-progress &)
49− ==
47+ ==
50−++ sigh-httr
48+ ++ sigh-httr
51− |= {wir/wire code/@ud headers/mess body/(unit octs)}
49+ |= {wir/wire code/@ud headers/mess body/(unit octs)}
52− ~& 'arrive here'
50+ ~& 'arrive here'
53− ^- {(list move) _+>.$}
51+ ^- {(list move) _+>.$}
54− ?: &((gte code 200) (lth code 300))
52+ ?: &((gte code 200) (lth code 300))
55− ~& [%all-is-well code]
53+ ~& [%all-is-well code]
56− :_ +>.$
54+ :_ +>.$
57− [ost.hid %wait /timer (add ~s10 now.hid)]~
55+ [ost.hid %wait /timer (add ~s10 now.hid)]~
58− ~& [%we-have-a-problem code]
56+ ~& [%we-have-a-problem code]
59− ~& [%headers headers]
57+ ~& [%headers headers]
60− ~& [%body body]
58+ ~& [%body body]
61− :_ +>.$
59+ :_ +>.$
62− [ost.hid %wait /timer (add ~s10 now.hid)]~
60+ [ost.hid %wait /timer (add ~s10 now.hid)]~
63−++ wake-timer
61+ ++ wake-timer
64− |= {wir/wire $~} ^- (quip move +>)
62+ |= {wir/wire $~} ^- (quip move +>)
65− ?: on
63+ ?: on
66− :_ +>.$
64+ :_ +>.$
67− [ost.hid %hiss /request ~ %httr %purl (need (epur target))]~
65+ [ost.hid %hiss /request ~ %httr %purl (need (epur target))]~
68− [~ +>.$(in-progress |)]
66+ [~ +>.$(in-progress |)]
69−::
67+ ::
70−++ prep ~& target _`. :: computed when the source file changes;
68+ ++ prep ~& target _`. :: computed when the source file changes;
71−-- :: here it prints target then resets our state
69+ -- :: here it prints target then resets our state
72−```
70+
73−
74−There's some fancy stuff going on here. We'll go through it line by
71+There's some fancy stuff going on here. We'll go through it line by line,
75−line, though.
72+though.
76−
73+
77−There's two kinds of cards we're sending to arvo. A `%hiss` move tells
74+There's two kinds of cards we're sending to arvo. A `%hiss` move tells `%eyre`
78−`%eyre` to make an HTTP request. It expects a `(unit iden)`, which will
75+to make an HTTP request. It expects a `(unit iden)`, which will be null here
79−be null here because we aren't doing any authentication; a mark, in this
76+because we aren't doing any authentication; a mark, in this case the http
80−case the http request mark `&httr`; and some data in the form of a
77+request mark `&httr`; and some data in the form of a `cage`, which is a
81−`cage`, which is a `{mark vase}`. This is confusing, so the mold in the
78+`{mark vase}`. This is confusing, so the mold in the code quoted above includes
82−code quoted above includes a bunch of otherwise unnecessary faces for
79+a bunch of otherwise unnecessary faces for the purpose of illustration. Compare:
83−the purpose of illustration. Compare: `{$hiss p/(unit iden) q/mark
84−r/cage}` from `zuse`.
80+`{$hiss p/(unit iden) q/mark r/cage}` from `zuse`.
85−
81+
86−> You can grep zuse.hoon and arvo.hoon for most of these definitions.
82+> You can grep zuse.hoon and arvo.hoon for most of these definitions. However,
87−> However, beware the ambiguity surrounding "hiss": there's ++hiss in
83+> beware the ambiguity surrounding "hiss": there's ++hiss in section 3bI of
88−> section 3bI of `zuse` ("Arvo structures"), and there's `%hiss` the
84+> `zuse` ("Arvo structures"), and there's `%hiss` the move that is sent to %eyre
89−> move that is sent to %eyre to make our HTTP request happen. Here we're
85+> to make our HTTP request happen. Here we're talking about the latter. See
90−> talking about the latter. See `++kiss-eyre` in `zuse`.
86+> `++kiss-eyre` in `zuse`.
91−
87+
92−> Recall that `++unit` means "maybe". Formally, "`(unit a)` is
88+> Recall that `++unit` means "maybe". Formally, "`(unit a)` is either null or a
93−> either null or a pair of null and a value in `a`". Also recall
89+> pair of null and a value in `a`". Also recall that to pull a value out of a
94−> that to pull a value out of a unit (`u.unit`), you must first
95−> verify that the unit is not null (for example with `?~`). See
90+> unit (`u.unit`), you must first verify that the unit is not null (for example
96−> `++need`.
91+> with `?~`). See `++need`.
97−
92+
98−A `purl` is a parsed url structure, which can be created with `++epur`,
93+A `purl` is a parsed url structure, which can be created with `++epur`, which is
99−which is a function that takes a url as text and parses it into a `(unit
94+a function that takes a url as text and parses it into a `(unit purl)`. Thus,
100−purl)`. Thus, the result is null if and only if the url is malformed.
95+the result is null if and only if the url is malformed. 'Purl' is also the mark
101−'Purl' is also the mark which will be applied to this result.
96+which will be applied to this result.
102−
97+
103−When you send this request, you can expect a `%sigh` with the
98+When you send this request, you can expect a `%sigh` with the response, which we
104−response, which we handle later on in `++sigh-httr`.
99+handle later on in `++sigh-httr`.
105−
100+
106−For `%wait`, you just pass a [`@da`]() (absolute date), and arvo will
101+For `%wait`, you just pass a [`@da`]() (absolute date), and arvo will produce a
107−produce a `%wake` when the time comes.
102+`%wake` when the time comes.
108−
103+
109−> A timer is guaranteed to not be triggered before the given
104+> A timer is guaranteed to not be triggered before the given time, but it's
110−> time, but it's currently impossible to guarantee the timer will be
105+> currently impossible to guarantee the timer will be triggered at exactly the
111−> triggered at exactly the requested time.
106+> requested time.
112
107
113Let's take a look at our state:
108Let's take a look at our state:
114
109
115−```
116−|_ {hid/bowl on/_| in-progress=_| target/@t}
110+ |_ {hid/bowl on/_| in-progress=_| target/@t}
117−```
118
111
119−We have three pieces of app-specific state. `target` is the url
112+We have three pieces of app-specific state. `target` is the url we're
120−we're monitoring. `on` and `in-progress` are booleans
113+monitoring. `on` and `in-progress` are booleans representing, respectively,
121−representing, respectively, whether or not we're supposed to keep
122−monitoring and whether or not we're in the middle of a request.
114+whether or not we're supposed to keep monitoring and whether or not we're in the
115+middle of a request.
123
116
124−The type of booleans is usually written as `?`, which is a union
117+The type of booleans is usually written as `?`, which is a union of `&` (true)
125−of `&` (true) and `|` false. This type defaults to `&` (true). In our
118+and `|` false. This type defaults to `&` (true). In our case, we want both of
126−case, we want both of these to default to `|` (false). Because
119+these to default to `|` (false). Because of that, we use `_|` as the type.
127−of that, we use `_|` as the type. `_value` means "take the type
128−of `value`, but make the default value be `value`". Thus, our
120+`_value` means "take the type of `value`, but make the default value be
129−type is still a boolean, just like `?`, but the default value is
121+`value`". Thus, our type is still a boolean, just like `?`, but the default
130−`|` (false).
122+value is `|` (false).
131
123
132−> This is the same `_` used in `_+>.$`, which means "the same
124+> This is the same `_` used in `_+>.$`, which means "the same type as the value
133−> type as the value "+>.$". In other words, the same type as our
125+> "+&gt;.$". In other words, the same type as our current context and state.
134−> current context and state.
135
126
136−Let's take a look at `++poke-atom`. When we're poked with an atom, we
127+Let's take a look at `++poke-atom`. When we're poked with an atom, we first
137−first check whether the atom is `'off'`. If so, we set our state
128+check whether the atom is `'off'`. If so, we set our state variable `on` to
138−variable `on` to false.
129+false.
139
130
140−If not, we check whether the atom is `'on'`. If so, we set `on` and
131+If not, we check whether the atom is `'on'`. If so, we set `on` and
141−`in-progress` to true. If it was already either on or in progress, then
132+`in-progress` to true. If it was already either on or in progress, then we don't
142−we don't take any other immediate action. If it was both off and not in
133+take any other immediate action. If it was both off and not in progress, then we
143−progress, then we send an HTTP request.
134+send an HTTP request.
144
135
145−If the argument is neither 'off' nor 'on', then we assume it's an
136+If the argument is neither 'off' nor 'on', then we assume it's an actual url, so
146−actual url, so we save it in `target`.
137+we save it in `target`.
147
138
148Here's the move that sends the HTTP request:
139Here's the move that sends the HTTP request:
149−```
140+
150−[ost.hid %hiss /request ~ %httr %purl (need (epur target))]
141+ [ost.hid %hiss /request ~ %httr %purl (need (epur target))]
151−```
142+
152−
153−> Remember, we are expected to produce a *list* of moves. Note the `~`
143+> Remember, we are expected to produce a *list* of moves. Note the `~` after the
154−> after the move in the full example. This is a convenient shortcut for
144+> move in the full example. This is a convenient shortcut for creating a list of
155−> creating a list of a single element. It's part of a small family of
145+> a single element. It's part of a small family of such shortcuts. `~[a b c]` is
156−> such shortcuts. `~[a b c]` is `[a b c ~]`, `[a b c]~` is `[[a b c]
146+> `[a b c ~]`, `[a b c]~` is `[[a b c] ~]` and `\`\[a b c\]`is`\[~ a b c\]\`.
157−> ~]` and `\`[a b c]` is `[~ a b c]`. These may be mixed and matched to
158−> create various convoluted structures and emojis.
147+> These may be mixed and matched to create various convoluted structures and
159−
148+> emojis.
149+
160−The correspondence between this move and `{bone card}` can be hard to
150+The correspondence between this move and `{bone card}` can be hard to visualize
161−visualize on one line. Here it is more pedantically:
151+on one line. Here it is more pedantically:
162−
152+
163−```
164−:* bone=ost.hid :: the move
153+ :* bone=ost.hid :: the move
165− term=%hiss
154+ term=%hiss
166− wire=/request
155+ wire=/request
167− unit-iden=~
156+ unit-iden=~
168− mark=%httr
157+ mark=%httr
169− cage=[%purl (need (epur target))]
158+ cage=[%purl (need (epur target))]
170−==
159+ ==
171−```
160+
172−
173−When the HTTP response comes back, we handle it with
161+When the HTTP response comes back, we handle it with `++sigh-httr`, which, along
174−`++sigh-httr`, which, along with the wire the request was sent
175−on, takes the status code, the response headers, and the response
162+with the wire the request was sent on, takes the status code, the response
176−body.
163+headers, and the response body.
177−
164+
178−We check whether the status code is between 200 and 300. If so,
165+We check whether the status code is between 200 and 300. If so, all is well, and
179−all is well, and we print a message saying so. We start the
166+we print a message saying so. We start the timer for ten seconds after the
180−timer for ten seconds after the present time.
167+present time.
181−
168+
182−If we got a bad status code, then we print out the entire
169+If we got a bad status code, then we print out the entire response and start the
183−response and start the timer again.
170+timer again.
184−
171+
185−After ten seconds, arvo will give us a `%wake` event, which will
172+After ten seconds, arvo will give us a `%wake` event, which will be handled in
186−be handled in `++wake-timer`. If we're still supposed to keep
173+`++wake-timer`. If we're still supposed to keep monitoring, we send the same
187−monitoring, we send the same HTTP request as before. Otherwise,
188−we set `in-progress` to false.
174+HTTP request as before. Otherwise, we set `in-progress` to false.
189
175
190Let's try it out:
176Let's try it out:
191
177
192−```
193−~fintud-macrep:dojo> |start %up
178+ ~fintud-macrep:dojo> |start %up
194−>=
179+ >=
195−~fintud-macrep:dojo> :examples-up &atom 'http://www.google.com'
180+ ~fintud-macrep:dojo> :examples-up &atom 'http://www.google.com'
196−>=
181+ >=
197−[%all-is-well 200]
182+ [%all-is-well 200]
198−[%all-is-well 200]
183+ [%all-is-well 200]
199−~fintud-macrep:dojo> :examples-up &atom 'http://example.com'
184+ ~fintud-macrep:dojo> :examples-up &atom 'http://example.com'
200−>=
185+ >=
201−[%all-is-well 200]
186+ [%all-is-well 200]
202−~fintud-macrep:dojo> :examples-up &atom 'http://google.com'
187+ ~fintud-macrep:dojo> :examples-up &atom 'http://google.com'
203−>=
188+ >=
204−[%we-have-a-problem 301]
189+ [%we-have-a-problem 301]
205−[ %headers
190+ [ %headers
206− ~[
191+ ~[
207− [p='X-Frame-Options' q='SAMEORIGIN']
192+ [p='X-Frame-Options' q='SAMEORIGIN']
208− [p='X-XSS-Protection' q='1; mode=block']
193+ [p='X-XSS-Protection' q='1; mode=block']
209− [p='Content-Length' q='219']
194+ [p='Content-Length' q='219']
210− [p='Server' q='gws']
195+ [p='Server' q='gws']
211− [p='Cache-Control' q='public, max-age=2592000']
196+ [p='Cache-Control' q='public, max-age=2592000']
212− [p='Expires' q='Sun, 10 Jan 2016 23:45:05 GMT']
197+ [p='Expires' q='Sun, 10 Jan 2016 23:45:05 GMT']
213− [p='Date' q='Fri, 11 Dec 2015 23:45:05 GMT']
198+ [p='Date' q='Fri, 11 Dec 2015 23:45:05 GMT']
214− [p='Content-Type' q='text/html; charset=UTF-8']
199+ [p='Content-Type' q='text/html; charset=UTF-8']
215− [p='Location' q='http://www.google.com/']
200+ [p='Location' q='http://www.google.com/']
216− ]
201+ ]
217−]
202+ ]
218−[ %body
203+ [ %body
219− [ ~
204+ [ ~
220− [ p=219
205+ [ p=219
221− q
206+ q
222− \/'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\0a<TITLE>301 Moved</TI\/
207+ \/'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\0a<TITLE>301 Moved</TI\/
223− TLE></HEAD><BODY>\0a<H1>301 Moved</H1>\0aThe document has moved\0a<A HREF="http://www.google.com/">her
208+ TLE></HEAD><BODY>\0a<H1>301 Moved</H1>\0aThe document has moved\0a<A HREF="http://www.google.com/">her
224− e</A>.\0d\0a</BODY></HTML>\0d\0a'
209+ e</A>.\0d\0a</BODY></HTML>\0d\0a'
225− \/ \/
210+ \/ \/
211+ ]
212+ ]
226 ]
213 ]
227− ]
228−]
229−~fintud-macrep:dojo> :up &atom 'off'
214+ ~fintud-macrep:dojo> :up &atom 'off'
230−```
docs/arvo/internals.md+90−102
@@ -1,5 +1,5 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 22
3sort: 22
4---
4---
5
5
@@ -9,27 +9,25 @@ Arvo is composed of modules called vanes:
9
9
10<list dataPreview="true"></list>
10<list dataPreview="true"></list>
11
11
12−<hr></hr>
12+<hr>
13+</hr>
14+At a high level `%arvo` takes a mess of unix io events and turns them into
15+something clean and structured for the programmer.
13
16
14−At a high level `%arvo` takes a mess of unix io events and turns them
17+`%arvo` is designed to avoid the usual state of complex event networks: event
15−into something clean and structured for the programmer.
18+spaghetti. We keep track of every event's cause so that we have a clear causal
19+chain for every computation. At the bottom of every chain is a unix io event,
20+such as a network request, terminal input, file sync, or timer event. We push
21+every step in the path the request takes onto the chain until we get to the
22+terminal cause of the computation. Then we use this causal stack to route
23+results back to the caller.
16
24
17−`%arvo` is designed to avoid the usual state of complex event networks:
25+## `++ducts`
18−event spaghetti. We keep track of every event's cause so that we have a
19−clear causal chain for every computation. At the bottom of every chain
20−is a unix io event, such as a network request, terminal input, file
21−sync, or timer event. We push every step in the path the request takes
22−onto the chain until we get to the terminal cause of the computation.
23−Then we use this causal stack to route results back to the caller.
24
26
25−`++ducts`
26−---------
27−
28−The `%arvo` causal stack is called a `++duct`. This is represented
27+The `%arvo` causal stack is called a `++duct`. This is represented simply as a
29−simply as a list of paths, where each path represents a step in the
28+list of paths, where each path represents a step in the causal chain. The first
30−causal chain. The first element in the path is the first letter of
29+element in the path is the first letter of whichever vane handled that step in
31−whichever vane handled that step in the computation, or the empty span
30+the computation, or the empty span for unix.
32−for unix.
33
31
34Here's a duct that was recently observed in the wild:
32Here's a duct that was recently observed in the wild:
35
33
@@ -41,108 +39,98 @@ Here's a duct that was recently observed in the wild:
41 //term/1
39 //term/1
42 ]
40 ]
43
41
44−This is the duct the timer vane receives when "timer" sample app asks
42+This is the duct the timer vane receives when "timer" sample app asks the timer
45−the timer vane to set a timer. This is also the duct over which the
43+vane to set a timer. This is also the duct over which the response is produced
46−response is produced at the specified time. Unix sent a terminal
44+at the specified time. Unix sent a terminal keystroke event (enter), and arvo
47−keystroke event (enter), and arvo routed it to %dill(our terminal),
48−which passed it on to the %gall app terminal, which sent it to shell,
45+routed it to %dill(our terminal), which passed it on to the %gall app terminal,
49−its child, which created a new child (with process id 4), which on
46+which sent it to shell, its child, which created a new child (with process id
50−startup asked the timer vane to set a timer.
47+4), which on startup asked the timer vane to set a timer.
51−
48+
52−The timer vane saves this duct, so that when the specified time arrives
49+The timer vane saves this duct, so that when the specified time arrives and unix
53−and unix sends a wakeup event to the timer vane, it can produce the
50+sends a wakeup event to the timer vane, it can produce the response on the same
54−response on the same duct. This response is routed to the place we
51+duct. This response is routed to the place we popped off the top of the duct,
55−popped off the top of the duct, i.e. the time app. This app produces the
52+i.e. the time app. This app produces the text "ding", which falls down to the
56−text "ding", which falls down to the shell, which drops it through to
57−the terminal. Terminal drops this down to dill, which converts it into
53+shell, which drops it through to the terminal. Terminal drops this down to dill,
58−an effect that unix will recognize as a request to print "ding" to the
54+which converts it into an effect that unix will recognize as a request to print
59−screen. When dill produces this, the last path in the duct has an
55+"ding" to the screen. When dill produces this, the last path in the duct has an
60−initial element of the empty span, so this is routed to unix, which
56+initial element of the empty span, so this is routed to unix, which applies the
61−applies the effects.
57+effects.
62
58
63This is a call stack, with a crucial feature: the stack is a first-class
59This is a call stack, with a crucial feature: the stack is a first-class
64−citizen. You can respond over a duct zero, one, or many times. You can
60+citizen. You can respond over a duct zero, one, or many times. You can save
65−save ducts for later use. There are definitely parallels to Scheme-style
61+ducts for later use. There are definitely parallels to Scheme-style
66continuations, but simpler and with more structure.
62continuations, but simpler and with more structure.
67
63
68−Making Moves
64+## Making Moves
69−------------
70
65
71−If ducts are a call stack, then how do we make calls and produce
66+If ducts are a call stack, then how do we make calls and produce results? Arvo
72−results? Arvo processes "moves" which are a combination of message data
67+processes "moves" which are a combination of message data and metadata. There
73−and metadata. There are two types of moves. A `%pass` move is analogous
68+are two types of moves. A `%pass` move is analogous to a call:
74−to a call:
75
69
76 [duct %pass return-path=path vane-name=@tD data=card]
70 [duct %pass return-path=path vane-name=@tD data=card]
77
71
78−Arvo pushes the return path (preceded by the first letter of the vane
72+Arvo pushes the return path (preceded by the first letter of the vane name) onto
79−name) onto the duct and sends the given data, a card, to the vane we
73+the duct and sends the given data, a card, to the vane we specified. Any
80−specified. Any response will come along the same duct with the path
74+response will come along the same duct with the path `return-path`.
81−`return-path`.
82
75
83A `%give` move is analogous to a return:
76A `%give` move is analogous to a return:
84
77
85 [duct %give data=card]
78 [duct %give data=card]
86
79
87−Arvo pops the top path off the duct and sends the given card back to the
80+Arvo pops the top path off the duct and sends the given card back to the caller.
88−caller.
89
81
90−Vanes
82+## Vanes
91−-----
92
83
93−As shown above, we use arvo proper to route and control the flow of
84+As shown above, we use arvo proper to route and control the flow of moves.
94−moves. However, arvo proper is rarely directly responsible for
85+However, arvo proper is rarely directly responsible for processing the event
95−processing the event data that directly causes the desired outcome of a
86+data that directly causes the desired outcome of a move. This event data is
96−move. This event data is contained within a card, which is simply a
87+contained within a card, which is simply a `(pair term noun)`. Instead, arvo
97−`(pair term noun)`. Instead, arvo proper passes the card off to one of
98−its vanes, which each present an interface to clients for a particular
88+proper passes the card off to one of its vanes, which each present an interface
99−well-defined, stable, and general-purpose piece of functionality.
89+to clients for a particular well-defined, stable, and general-purpose piece of
90+functionality.
100
91
101−As of this writing, we have seven vanes, which each provide the
92+As of this writing, we have seven vanes, which each provide the following
102−following services:
93+services:
103
94
104−- `%ames` name of both our network and the vane that communicates over
95+- `%ames` name of both our network and the vane that communicates over it
105− it
106- `%clay` version-controlled, referentially- transparent, and global
96- `%clay` version-controlled, referentially- transparent, and global
107 filesystem
97 filesystem
108−- `%dill` terminal driver. Unix sends keyboard events to `%dill` from
98+- `%dill` terminal driver. Unix sends keyboard events to `%dill` from either
109− either the console or telnet, and `%dill` produces terminal output.
99+ the console or telnet, and `%dill` produces terminal output.
110−- `%eyre` http server. Unix sends http messages to `%eyre`, and
100+- `%eyre` http server. Unix sends http messages to `%eyre`, and `%eyre`
111− `%eyre` produces http messages in response
101+ produces http messages in response
112- `%ford` handles resources and publishing
102- `%ford` handles resources and publishing
113−- `%gall` manages our userspace applications.. `%gall` keeps state and
103+- `%gall` manages our userspace applications.. `%gall` keeps state and manages
114− manages subscribers
104+ subscribers
115- `%time` a simple timer
105- `%time` a simple timer
116
106
117−Cards
107+## Cards
118−-----
108+
119−
120−Cards are the vane-specific portion of a move. Each vane defines a
109+Cards are the vane-specific portion of a move. Each vane defines a protocol for
121−protocol for interacting with other vanes (via arvo) by defining four
110+interacting with other vanes (via arvo) by defining four types of cards: kisses,
122−types of cards: kisses, gifts, notes, and signs.
111+gifts, notes, and signs.
123−
112+
124−When one vane is `%pass`ed a card in its `++kiss`, arvo activates the
113+When one vane is `%pass`ed a card in its `++kiss`, arvo activates the `++call`
125−`++call` gate with the card as its argument. To produce a result, the
114+gate with the card as its argument. To produce a result, the vane `%give`s one
126−vane `%give`s one of the cards defined in its `++gift`. If the vane
115+of the cards defined in its `++gift`. If the vane needs to request something of
127−needs to request something of another vane, it `%pass`es it a `++note`
128−card. When that other vane returns a result, arvo activates the `++take`
116+another vane, it `%pass`es it a `++note` card. When that other vane returns a
129−gate of the initial vane with one of the cards defined in its `++sign`.
117+result, arvo activates the `++take` gate of the initial vane with one of the
130−
118+cards defined in its `++sign`.
119+
131−In other words, there are only four ways of seeing a move: (1) as a
120+In other words, there are only four ways of seeing a move: (1) as a request seen
132−request seen by the caller, which is a ++note. (2) that same request as
121+by the caller, which is a ++note. (2) that same request as seen by the callee, a
133−seen by the callee, a `++kiss`. (3) the response to that first request
122+`++kiss`. (3) the response to that first request as seen by the callee, a
134−as seen by the callee, a `++gift`. (4) the response to the first request
123+`++gift`. (4) the response to the first request as seen by the caller, a
135−as seen by the caller, a `++sign`.
124+`++sign`.
136−
125+
137−When a `++kiss` card is passed to a vane, arvo calls its `++call` gate,
126+When a `++kiss` card is passed to a vane, arvo calls its `++call` gate, passing
138−passing it both the card and its duct. This gate must be defined in
127+it both the card and its duct. This gate must be defined in every vane. It
139−every vane. It produces two things in the following order: a list of
128+produces two things in the following order: a list of moves and a
140−moves and a possibly-modified copy of its context. The moves are used to
129+possibly-modified copy of its context. The moves are used to interact with other
141−interact with other vanes, while the new context allows the vane to save
130+vanes, while the new context allows the vane to save its state. The next time
142−its state. The next time arvo activates the vane it will have this
131+arvo activates the vane it will have this context as its subject.
143−context as its subject.
132+
144−
145−This overview has detailed how to pass a card to a particular vane. To
133+This overview has detailed how to pass a card to a particular vane. To see the
146−see the cards each vane can be `%pass`ed as a `++kiss` or return as a
134+cards each vane can be `%pass`ed as a `++kiss` or return as a `++gift` (as well
147−`++gift` (as well as the semantics tied to them), each vane's public
135+as the semantics tied to them), each vane's public interface is explained in
148−interface is explained in detail in its respective overview.
136+detail in its respective overview.
docs/arvo/internals/ames.md+15−18
@@ -1,28 +1,25 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 1
3sort: 1
4---
4---
5
5
6−`%ames`
6+# `%ames`
7−=======
8
7
9Our networking protocol.
8Our networking protocol.
10
9
11−`%ames` is the name of both our network and the vane that communicates
10+`%ames` is the name of both our network and the vane that communicates over it.
12−over it. When Unix receives a packet over the correct UDP port, it pipes
11+When Unix receives a packet over the correct UDP port, it pipes it straight into
13−it straight into `%ames` for handling. Also, all packets sent over the
12+`%ames` for handling. Also, all packets sent over the `%ames` network are sent
14−`%ames` network are sent by the `%ames` vane. Apps and vanes may use
13+by the `%ames` vane. Apps and vanes may use `%ames` to directly send messages to
15−`%ames` to directly send messages to other ships. In general, apps use
16−gall and clay to communicate with other ships rather than using `%ames`
14+other ships. In general, apps use gall and clay to communicate with other ships
17−directly, but this isn't a requirement. Of course, gall and clay use
15+rather than using `%ames` directly, but this isn't a requirement. Of course,
18−`%ames` behind the scenes to communicate across the network. These are
16+gall and clay use `%ames` behind the scenes to communicate across the network.
19−the only two vanes that use `%ames`.
17+These are the only two vanes that use `%ames`.
20
18
21−`%ames` includes several significant components. Although the actual
19+`%ames` includes several significant components. Although the actual crypto
22−crypto algorithms are defined in zuse, they're used extensively in
20+algorithms are defined in zuse, they're used extensively in `%ames` for
23−`%ames` for encrypting and decrypting packets. Congestion control and
21+encrypting and decrypting packets. Congestion control and routing is handled
24−routing is handled entirely in `%ames`. Finally, the actual `%ames`
22+entirely in `%ames`. Finally, the actual `%ames` protocol itself, including how
25−protocol itself, including how to route incoming packets to the correct
26−vane or app, is defined in `%ames`.
23+to route incoming packets to the correct vane or app, is defined in `%ames`.
27
24
28<list></list>
25<list></list>
docs/arvo/internals/ames/commentary.md+690−769
@@ -1,80 +1,69 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3---
3---
4
4
5−
6−`%ames` commentary
5+# `%ames` commentary
7−==================
8
6
9`%ames` is our networking protocol.
7`%ames` is our networking protocol.
10
8
11−First we give commentary on the code, the algorithms involved, and the
9+First we give commentary on the code, the algorithms involved, and the protocol.
12−protocol. We trace through the code touched when a packet is sent,
10+We trace through the code touched when a packet is sent, received, acknowledged,
13−received, acknowledged, and that acknowledgment applied. This is fairly
11+and that acknowledgment applied. This is fairly comprehensive, and contains many
14−comprehensive, and contains many implementation details, but if you
15−understand this, then you understand `%ames`.
12+implementation details, but if you understand this, then you understand `%ames`.
16−
13+
17−If you've scrolled down this page, you may be intimidated by the amount
14+If you've scrolled down this page, you may be intimidated by the amount of Hoon
18−of Hoon code, especially if you are new to the language. Don't be afraid
15+code, especially if you are new to the language. Don't be afraid of it, you
19−of it, you don't have to read any of it if you don't want to -- every
16+don't have to read any of it if you don't want to -- every interesting action
20−interesting action the code takes is explained in plain English. In
17+the code takes is explained in plain English. In fact, if you are new to the
21−fact, if you are new to the language, this may be a good learning
22−opportunity. Even if you don't understand every line of Hoon code,
18+language, this may be a good learning opportunity. Even if you don't understand
23−you'll hopefully be able to follow most lines. By the time you've worked
19+every line of Hoon code, you'll hopefully be able to follow most lines. By the
24−through this, you'll have seen many common patterns and best practices.
20+time you've worked through this, you'll have seen many common patterns and best
25−Hoon, much more than other languages, is best learned by reading and
21+practices. Hoon, much more than other languages, is best learned by reading and
26−understanding large quantities of existing code. In this way, it is
22+understanding large quantities of existing code. In this way, it is similar to
27−similar to learning a natural language. All of this code is in
23+learning a natural language. All of this code is in `arvo/ames.hoon`.
28−`arvo/ames.hoon`.
29
24
30After the commentary, we have reference documentation for all the data
25After the commentary, we have reference documentation for all the data
31−structures that are specific to `%ames`. If you see a data structure or
26+structures that are specific to `%ames`. If you see a data structure or a
32−a variable used that you don't recognize, search for it in the code, and
27+variable used that you don't recognize, search for it in the code, and it's very
33−it's very likely defined in one of these data structures. We recommend
28+likely defined in one of these data structures. We recommend that another tab is
34−that another tab is kept open for easy access to the data structure
29+kept open for easy access to the data structure reference documentation. The
35−reference documentation. The code for these is split between
36−`arvo/ames.hoon` and `arvo/zuse.hoon`.
30+code for these is split between `arvo/ames.hoon` and `arvo/zuse.hoon`.
37
31
38−The Lifecycle of a Packet (or, How a Packet Becomes Law)
32+## The Lifecycle of a Packet (or, How a Packet Becomes Law)
39−--------------------------------------------------------
40
33
41Here, we will trace a packet as it makes its way through ames. There are
34Here, we will trace a packet as it makes its way through ames. There are
42−actually two pathways through ames: the legacy path through `%want`, and
35+actually two pathways through ames: the legacy path through `%want`, and the
43−the modern way, entered through `%wont`, with full end-to-end
36+modern way, entered through `%wont`, with full end-to-end acknowledgments. Here
44−acknowledgments. Here we will only trace the modern way, though much of
37+we will only trace the modern way, though much of the path is the same for both.
45−the path is the same for both.
46
38
47−When an app (or a vane) wishes to send a packet to another ship, it must
39+When an app (or a vane) wishes to send a packet to another ship, it must send a
48−send a `%wont` card:
40+`%wont` card:
49
41
50 [%wont p=sock q=path r=*] :: e2e send message
42 [%wont p=sock q=path r=*] :: e2e send message
51
43
52−This card takes three arguments. The `p` is a `sock`, that is, a pair of
44+This card takes three arguments. The `p` is a `sock`, that is, a pair of two
53−two ships, the first of which is the sender and the second is the
45+ships, the first of which is the sender and the second is the receiver. But
54−receiver. But wait, you ask, why do I get to decide who is the sender?
46+wait, you ask, why do I get to decide who is the sender? Can I fake like I'm
55−Can I fake like I'm someone else? The reason is that there are
47+someone else? The reason is that there are potentially multiple ships on the
56−potentially multiple ships on the same pier, and the kernel can send a
57−message from any of them. If you attempt to send a message from a ship
48+same pier, and the kernel can send a message from any of them. If you attempt to
58−not on your pier, then ames will refuse to send it. If you hack around
49+send a message from a ship not on your pier, then ames will refuse to send it.
59−in your own copy of ames to go ahead and send it anyway, then the other
50+If you hack around in your own copy of ames to go ahead and send it anyway, then
60−ship will reject it because your key is bad. Only send messages from
51+the other ship will reject it because your key is bad. Only send messages from
61yourself.
52yourself.
62
53
63−The `q` is a path, representing the place on the other side that you
54+The `q` is a path, representing the place on the other side that you want to
64−want to receive your message. It is approximately equivalent to a port
55+receive your message. It is approximately equivalent to a port number. Messages
65−number. Messages on the same path are guaranteed to arrive in the same
56+on the same path are guaranteed to arrive in the same order as they were sent.
66−order as they were sent. No such guarantees are made across paths.
57+No such guarantees are made across paths.
67
58
68−The `r` is the actual data that you are sending. As the type implies,
59+The `r` is the actual data that you are sending. As the type implies, this can
69−this can be an arbitrary noun, and it will be transferred to the
60+be an arbitrary noun, and it will be transferred to the receiver exactly as-is,
70−receiver exactly as-is, in a well-typed way. Of course, this is data
61+in a well-typed way. Of course, this is data that is sent over the wire, so be
71−that is sent over the wire, so be careful not to send anything too
72−massive unless you're willing to wait.
62+careful not to send anything too massive unless you're willing to wait.
73
63
74−But enough about the interface. Grepping in ames.hoon for `%wont`, we
64+But enough about the interface. Grepping in ames.hoon for `%wont`, we find that
75−find that it appears in exactly two places: at its definition in
65+it appears in exactly two places: at its definition in `++kiss`, and in
76−`++kiss`, and in `++knob`, where it is handled. We see that we go
66+`++knob`, where it is handled. We see that we go directly into `++wise:am`.
77−directly into `++wise:am`.
78
67
79 ++ wise :: wise:am
68 ++ wise :: wise:am
80 |= [soq=sock hen=duct cha=path val=* ete=?] :: send a statement
69 |= [soq=sock hen=duct cha=path val=* ete=?] :: send a statement
@@ -82,20 +71,19 @@ directly into `++wise:am`.
82 zork:zank:(wool:(ho:(um p.soq) q.soq) hen cha val ete)
71 zork:zank:(wool:(ho:(um p.soq) q.soq) hen cha val ete)
83
72
84The inputs to this gate are exactly the sort of thing you'd expect. In
73The inputs to this gate are exactly the sort of thing you'd expect. In
85−particular, everything in the `%wont` gate is here plus the calling duct
74+particular, everything in the `%wont` gate is here plus the calling duct so that
86−so that we know where to send the acknowledgment and `ete` to determine
75+we know where to send the acknowledgment and `ete` to determine if we're going
87−if we're going to do the modern end-to-end acknowledgments.
76+to do the modern end-to-end acknowledgments.
88
77
89−The actual line of code looks intimidating, but it's really not all that
78+The actual line of code looks intimidating, but it's really not all that bad.
90−bad. Working from the inside out, the call to `++um` sets up our
79+Working from the inside out, the call to `++um` sets up our domestic server, and
91−domestic server, and the call to `++ho` sets up our knowledge about the
80+the call to `++ho` sets up our knowledge about the neighbor we're sending to.
92−neighbor we're sending to. From the outside, `++zork` and `++zank` just
81+From the outside, `++zork` and `++zank` just apply the changes made to our
93−apply the changes made to our `++um` and `++am` cores, respectively. If
94−you're familiar with the common idiom of `++abet`, that's all this is.
82+`++um` and `++am` cores, respectively. If you're familiar with the common idiom
95−The code predates the widespread usage of that name.
83+of `++abet`, that's all this is. The code predates the widespread usage of that
84+name.
96
85
97−The interesting part, then, is in `++wool:ho:um:am`. Let's look at the
86+The interesting part, then, is in `++wool:ho:um:am`. Let's look at the code.
98−code.
99
87
100 ++ wool :: wool:ho:um:am
88 ++ wool :: wool:ho:um:am
101 |= [hen=duct cha=path val=* ete=?] :: send a statement
89 |= [hen=duct cha=path val=* ete=?] :: send a statement
@@ -114,36 +102,33 @@ code.
114 [%bund q.cov cha sex val]
102 [%bund q.cov cha sex val]
115 [%bond q.cov cha sex val]
103 [%bond q.cov cha sex val]
116
104
117−This is slightly more complicated, but it's still not all that bad. Our
105+This is slightly more complicated, but it's still not all that bad. Our inputs,
118−inputs, at least, are fairly obvious.
106+at least, are fairly obvious.
119
107
120−If you glance at the code for a second, you'll see that
108+If you glance at the code for a second, you'll see that `++wind:ho:um:am` seems
121−`++wind:ho:um:am` seems to be able to send a message, or `++meal`, given
109+to be able to send a message, or `++meal`, given a `++soup`. This gate, then,
122−a `++soup`. This gate, then, just sets up the things we need to for
110+just sets up the things we need to for `++wind` to do its job.
123−`++wind` to do its job.
124
111
125−We first get `rol`, which is a `++rill`, that is, a particular outbound
112+We first get `rol`, which is a `++rill`, that is, a particular outbound stream.
126−stream. This stream is specific to the path on which we're sending. If
113+This stream is specific to the path on which we're sending. If the path hasn't
127−the path hasn't been used before, then we create it. We let `sex` be the
114+been used before, then we create it. We let `sex` be the number of messages
128−number of messages we've already sent on this path.
115+we've already sent on this path.
129
116
130−Then, we update the outbound stream by incrementing the number of
117+Then, we update the outbound stream by incrementing the number of messages sent
131−messages sent and placing an entry in `san.rol` that associates the
118+and placing an entry in `san.rol` that associates the message number with the
132−message number with the `duct` that sent the message. This allows us to
119+`duct` that sent the message. This allows us to give the acknowledgment to the
133−give the acknowledgment to the one who sent the message.
120+one who sent the message.
134
121
135−We let `cov` be the current life of our crypto and our neighbor's
122+We let `cov` be the current life of our crypto and our neighbor's crypto. At the
136−crypto. At the moment, we only need our neighbor's life, which we put
123+moment, we only need our neighbor's life, which we put into the meal.
137−into the meal.
138
124
139−Finally, we call `++wind:ho:um:am` with the `++soup` of the path and
125+Finally, we call `++wind:ho:um:am` with the `++soup` of the path and message
140−message number and the `++meal` of the payload itself. For end-to-end
126+number and the `++meal` of the payload itself. For end-to-end acknowledged
141−acknowledged messages, we use `%bund`.
127+messages, we use `%bund`.
142
128
143 [%bund p=life q=path r=@ud s=*] :: e2e message
129 [%bund p=life q=path r=@ud s=*] :: e2e message
144
130
145−Looking at how we create the `%bund`, we can easily see what each field
131+Looking at how we create the `%bund`, we can easily see what each field is for.
146−is for.
147
132
148Following the trail a little further, we go to `++wind:ho:um:am`.
133Following the trail a little further, we go to `++wind:ho:um:am`.
149
134
@@ -156,13 +141,13 @@ Following the trail a little further, we go to `++wind:ho:um:am`.
156 (busk xong:diz feh)
141 (busk xong:diz feh)
157
142
158`++wind` does three things: it (1) encodes the message into a list of
143`++wind` does three things: it (1) encodes the message into a list of
159−possibly-encrypted packets, (2) puts the message into the packet pump,
144+possibly-encrypted packets, (2) puts the message into the packet pump, and (3)
160−and (3) sends any packets that are ready to be sent. Yes, our nice
145+sends any packets that are ready to be sent. Yes, our nice little linear run of
161−little linear run of each gate calling exactly one other interesting
146+each gate calling exactly one other interesting gate is over. We'll go in order
162−gate is over. We'll go in order here.
147+here.
163
148
164−`++zuul:lax:as:go` is the what converts a `++meal` into a list of
149+`++zuul:lax:as:go` is the what converts a `++meal` into a list of actual, 1KB
165−actual, 1KB packets.
150+packets.
166
151
167 ++ zuul :: zuul:lax:as:go
152 ++ zuul :: zuul:lax:as:go
168 |= [now=@da ham=meal] :: encode message
153 |= [now=@da ham=meal] :: encode message
@@ -172,33 +157,32 @@ actual, 1KB packets.
172 ++ weft :: fragment message
157 ++ weft :: fragment message
173 ++ wisp :: generate message
158 ++ wisp :: generate message
174
159
175−For organizational purposes, `++zuul` constructs an internal core with
160+For organizational purposes, `++zuul` constructs an internal core with three
176−three arms. `++wasp` encodes the meal into an atom with no encryption.
161+arms. `++wasp` encodes the meal into an atom with no encryption. `++wisp`
177−`++wisp` encodes a meal with possible encryption (else it simply calls
162+encodes a meal with possible encryption (else it simply calls `++wasp`).
178−`++wasp`). `++weft` takes the result of `++wisp` and splits it into
163+`++weft` takes the result of `++wisp` and splits it into actual packets.
179−actual packets.
180
164
181 ++ wasp :: null security
165 ++ wasp :: null security
182 ^-([p=skin q=@] [%none (jam ham)])
166 ^-([p=skin q=@] [%none (jam ham)])
183
167
184−This simply jams the meal, wrapping it with the `skin` of `%none`,
168+This simply jams the meal, wrapping it with the `skin` of `%none`, meaning no
185−meaning no encryption.
169+encryption.
186
170
187Since `++wisp` is a little long, we'll go through it line-by-line.
171Since `++wisp` is a little long, we'll go through it line-by-line.
188
172
189 ++ wisp :: generate message
173 ++ wisp :: generate message
190 ^- [[p=skin q=@] q=_..wisp]
174 ^- [[p=skin q=@] q=_..wisp]
191
175
192−`++wisp` produces a pair of a `skin` and an atom, which is the meal
176+`++wisp` produces a pair of a `skin` and an atom, which is the meal encoded as a
193−encoded as a single atom and possibly encrypted.
177+single atom and possibly encrypted.
194
178
195 ?: =(%carp -.ham)
179 ?: =(%carp -.ham)
196 [wasp ..wisp]
180 [wasp ..wisp]
197
181
198−If the meal that we're encoding is a `%carp`, then we don't encrypt it.
182+If the meal that we're encoding is a `%carp`, then we don't encrypt it. A
199−A `%carp` meal is a partial meal, used when a message is more than 1KB.
183+`%carp` meal is a partial meal, used when a message is more than 1KB. Since the
200−Since the entire message is already encrypted, we don't need to encrypt
184+entire message is already encrypted, we don't need to encrypt each packet
201−each packet individually again.
185+individually again.
202
186
203 ?: !=(~ yed.caq.dur)
187 ?: !=(~ yed.caq.dur)
204 ?> ?=(^ yed.caq.dur)
188 ?> ?=(^ yed.caq.dur)
@@ -208,31 +192,29 @@ each packet individually again.
208 p.u.yed.caq.dur
192 p.u.yed.caq.dur
209 (en:r:cluy q.u.yed.caq.dur (jam ham))
193 (en:r:cluy q.u.yed.caq.dur (jam ham))
210
194
211−If we have a symmetric key set up with this neighbor, then we simply use
195+If we have a symmetric key set up with this neighbor, then we simply use it. The
212−it. The skin `%fast` is used to indicate a symmetric key.
196+skin `%fast` is used to indicate a symmetric key.
213
197
214 ?: &(=(~ lew.wod.dur) |(=(%back -.ham) =(%buck -.ham)))
198 ?: &(=(~ lew.wod.dur) |(=(%back -.ham) =(%buck -.ham)))
215 [wasp ..wisp]
199 [wasp ..wisp]
216
200
217−If we do not yet have our neighbor's will, then there is no way that we
201+If we do not yet have our neighbor's will, then there is no way that we can seal
218−can seal the message so that only they may read it. If what we're
202+the message so that only they may read it. If what we're sending is an
219−sending is an acknowledgment, then we go ahead and just send it in the
203+acknowledgment, then we go ahead and just send it in the clear.
220−clear.
221
204
222 =^ tuy +>.$
205 =^ tuy +>.$
223 ?:(=(~ lew.wod.dur) [*code +>.$] (griz now))
206 ?:(=(~ lew.wod.dur) [*code +>.$] (griz now))
224
207
225−If we don't have our neighbor's will, then we "encrypt" with a key of 0.
208+If we don't have our neighbor's will, then we "encrypt" with a key of 0. If we
226−If we do have their will, then we generate a new symmetric key that we
209+do have their will, then we generate a new symmetric key that we will propose.
227−will propose.
228
210
229 :_ ..wisp
211 :_ ..wisp
230 =+ yig=sen
212 =+ yig=sen
231 =+ bil=law.saf :: XX send whole will
213 =+ bil=law.saf :: XX send whole will
232 =+ hom=(jam ham)
214 =+ hom=(jam ham)
233
215
234−`yig` will be the life and engine for our current crypto. `bil` is our
216+`yig` will be the life and engine for our current crypto. `bil` is our will.
235−will. `hom` is the meal encoded as a single atom.
217+`hom` is the meal encoded as a single atom.
236
218
237 ?: =(~ lew.wod.dur)
219 ?: =(~ lew.wod.dur)
238 :- %open
220 :- %open
@@ -241,9 +223,8 @@ will. `hom` is the meal encoded as a single atom.
241 bil
223 bil
242 (sign:as:q.yig tuy hom)
224 (sign:as:q.yig tuy hom)
243
225
244−If we do not have our neighbor's will, then we send our current life
226+If we do not have our neighbor's will, then we send our current life along with
245−along with our will and the message. The message itself is "signed" with
227+our will and the message. The message itself is "signed" with a key of 0.
246−a key of 0.
247
228
248 :- %full
229 :- %full
249 =+ cay=cluy
230 =+ cay=cluy
@@ -253,9 +234,9 @@ a key of 0.
253 (seal:as:q.yig pub:ex:r.cay tuy hom)
234 (seal:as:q.yig pub:ex:r.cay tuy hom)
254 -- :: --zuul:lax:as:go
235 -- :: --zuul:lax:as:go
255
236
256−If we do have our neighbor's will, then we send our perception of their
237+If we do have our neighbor's will, then we send our perception of their current
257−current life, our current life, our will, and the message. The message
238+life, our current life, our will, and the message. The message is sealed with
258−is sealed with their public key so that only they can read our message.
239+their public key so that only they can read our message.
259
240
260Once we have the message encoded as an atom, `++weft` goes to work.
241Once we have the message encoded as an atom, `++weft` goes to work.
261
242
@@ -271,23 +252,23 @@ aforementioned `++wisp` to get the message as an atom.
271 =+ wit=(met 13 q.gim)
252 =+ wit=(met 13 q.gim)
272 ?< =(0 wit)
253 ?< =(0 wit)
273
254
274−`wit` is the number of 1KB (2\^13 bit) blocks in the message. We assert
255+`wit` is the number of 1KB (2\\^13 bit) blocks in the message. We assert that
275−that there is at least one block.
256+there is at least one block.
276
257
277 ?: =(1 wit)
258 ?: =(1 wit)
278 =+ yup=(spit [our her] p.gim q.gim)
259 =+ yup=(spit [our her] p.gim q.gim)
279 [yup ~]
260 [yup ~]
280
261
281−If there is exactly one block, then we just call `++spit` to turn the
262+If there is exactly one block, then we just call `++spit` to turn the message
282−message into a packet. We'll explain what `++spit` does momentarily.
263+into a packet. We'll explain what `++spit` does momentarily.
283
264
284 =+ ruv=(rip 13 q.gim)
265 =+ ruv=(rip 13 q.gim)
285 =+ gom=(shaf %thug q.gim)
266 =+ gom=(shaf %thug q.gim)
286 =+ inx=0
267 =+ inx=0
287
268
288−If there is more than one block, then we rip it into blocks in `ruv`.
269+If there is more than one block, then we rip it into blocks in `ruv`. `gom` is a
289−`gom` is a hash of the message, used as an id. `inx` is the number of
270+hash of the message, used as an id. `inx` is the number of packets we've already
290−packets we've already made.
271+made.
291
272
292 |- ^- (list rock)
273 |- ^- (list rock)
293 ?~ ruv ~
274 ?~ ruv ~
@@ -298,8 +279,8 @@ packets we've already made.
298 :- vie
279 :- vie
299 $(ruv t.ruv, inx +(inx))
280 $(ruv t.ruv, inx +(inx))
300
281
301−Here we package each block into a packet with `++spit` and produce the
282+Here we package each block into a packet with `++spit` and produce the list of
302−list of packets.
283+packets.
303
284
304 ++ spit :: cake to packet
285 ++ spit :: cake to packet
305 |= kec=cake ^- @
286 |= kec=cake ^- @
@@ -321,32 +302,30 @@ list of packets.
321 ==
302 ==
322 (lsh 5 1 bod)
303 (lsh 5 1 bod)
323
304
324−This is how we turn a message into a real packet. This has the
305+This is how we turn a message into a real packet. This has the definition of the
325−definition of the packet format.
306+packet format.
326
307
327`wim` is the length of the sending ship, and `dum` is the length of the
308`wim` is the length of the sending ship, and `dum` is the length of the
328receiving ship. There are only five possibilities for each of those,
309receiving ship. There are only five possibilities for each of those,
329−corresponding to carriers, cruisers, destroyers, yachts, and submarines.
310+corresponding to carriers, cruisers, destroyers, yachts, and submarines. These
330−These are encoded in `yax` and `qax` as 0, 0, 1, 2, and 3, respectively.
311+are encoded in `yax` and `qax` as 0, 0, 1, 2, and 3, respectively. Thus, `wix`
331−Thus, `wix` and `vix` are the number of bytes that must be reserved for
312+and `vix` are the number of bytes that must be reserved for the ship names in a
332−the ship names in a packet.
313+packet.
333−
314+
334−Next, we construct `bod` by simply concatenating the sending ship, the
315+Next, we construct `bod` by simply concatenating the sending ship, the receiving
335−receiving ship, and the body of the message. Then, we get the encryption
316+ship, and the body of the message. Then, we get the encryption mechanism from
336−mechanism from `++skin`, which may be a 0, 1, 2, or 3, and put it in
317+`++skin`, which may be a 0, 1, 2, or 3, and put it in `tay`.
337−`tay`.
318+
338−
339−Next, we concatenate together, bit by bit, some final metadata. We use
319+Next, we concatenate together, bit by bit, some final metadata. We use three
340−three bits for our protocol number, which is incremented modulo eight
320+bits for our protocol number, which is incremented modulo eight when there is a
341−when there is a continuity breach or the protocol changes. We use the
321+continuity breach or the protocol changes. We use the final twenty bits of a
342−final twenty bits of a hash of the body (which, we suppose, makes it a
322+hash of the body (which, we suppose, makes it a twenty bit hash) for
343−twenty bit hash) for error-checking. We use two bits to tell how much
323+error-checking. We use two bits to tell how much room is used in the body for
344−room is used in the body for the sending ship, and another two bits for
324+the sending ship, and another two bits for the receiving ship. Finally, we use
345−the receiving ship. Finally, we use five bits to store the encryption
346−type. Note that since there are only two bits worth of encryption types,
325+five bits to store the encryption type. Note that since there are only two bits
347−there are three unused bits here. This adds up to 32 bits of header
326+worth of encryption types, there are three unused bits here. This adds up to 32
348−data. Finally, we concatenate this onto the front of the packet. Thus,
327+bits of header data. Finally, we concatenate this onto the front of the packet.
349−we can summarize the packet header format as follows.
328+Thus, we can summarize the packet header format as follows.
350
329
351 0 1 2 3
330 0 1 2 3
352 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
331 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
@@ -354,15 +333,14 @@ we can summarize the packet header format as follows.
354 |Proto| Hash of Body |yax|qax| Crypto |
333 |Proto| Hash of Body |yax|qax| Crypto |
355 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
334 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
356
335
357−After this, there are `yax` bits of the sender name, `qax` bits of the
336+After this, there are `yax` bits of the sender name, `qax` bits of the receiver
358−receiver name, and up to 8192 bits of data. Thus, the maximum size of a
337+name, and up to 8192 bits of data. Thus, the maximum size of a packet is
359−packet is achieved in a message between two submarines with 8192 bits of
338+achieved in a message between two submarines with 8192 bits of data. This will
360−data. This will require 32+128+128+8192 = 8480 bits, or 1060 bytes.
339+require 32+128+128+8192 = 8480 bits, or 1060 bytes.
361
340
362This concludes our discussion of `++zuul:lax:as:go`. If you recall from
341This concludes our discussion of `++zuul:lax:as:go`. If you recall from
363−`++wind:ho:um:am`, the list of packets from `++zuul` is passed into
342+`++wind:ho:um:am`, the list of packets from `++zuul` is passed into `++whap:pu`
364−`++whap:pu` to update the packet pump and get any packets that can be
343+to update the packet pump and get any packets that can be sent immediately.
365−sent immediately.
366
344
367 ++ whap :: whap:pu
345 ++ whap :: whap:pu
368 |= [now=@da gom=soup wyv=(list rock)] :: send a message
346 |= [now=@da gom=soup wyv=(list rock)] :: send a message
@@ -379,16 +357,15 @@ sent immediately.
379 ==
357 ==
380 (harv now)
358 (harv now)
381
359
382−First, we put into `pyz` the id for this message and the number of its
360+First, we put into `pyz` the id for this message and the number of its packets
383−packets that have not yet been acknowledged, which is of course the
361+that have not yet been acknowledged, which is of course the total number of
384−total number of packets since we haven't even sent the packets.
362+packets since we haven't even sent the packets.
385
363
386−For every packet, we change three things in the state (`++shed`) of our
364+For every packet, we change three things in the state (`++shed`) of our packet
387−packet pump: (1) we increment `nus`, the number of packets sent; (2) we
365+pump: (1) we increment `nus`, the number of packets sent; (2) we put the packet
388−put the packet number into `diq` keyed by a hash of the packet; and (3)
366+number into `diq` keyed by a hash of the packet; and (3) we put the packet into
389−we put the packet into the packet queue, with the basic metadata of its
367+the packet queue, with the basic metadata of its id `gom`, 0 transmissions, not
390−id `gom`, 0 transmissions, not live yet, last sent in the year 2000, and
368+live yet, last sent in the year 2000, and the packet itself.
391−the packet itself.
392
369
393Finally, we harvest the packet pump.
370Finally, we harvest the packet pump.
394
371
@@ -402,17 +379,17 @@ Finally, we harvest the packet pump.
402 =< abet =< apse
379 =< abet =< apse
403 |%
380 |%
404
381
405−`++harv` contains a core for most of its work. The meat is in `++apse`.
382+`++harv` contains a core for most of its work. The meat is in `++apse`. First,
406−First, though, it sets itself up. If there aren't any packets in the
383+though, it sets itself up. If there aren't any packets in the queue, then we
407−queue, then we simply do nothing except set `rtn`, our next timeout, to
384+simply do nothing except set `rtn`, our next timeout, to nil because we don't
408−nil because we don't have any packets that may need to be retransmitted.
385+have any packets that may need to be retransmitted. If we have more live (that
409−If we have more live (that is, sent and unacknowledged) packets than our
386+is, sent and unacknowledged) packets than our window size, then we don't do
410−window size, then we don't do anything.
387+anything.
411
388
412−Otherwise, we let `wid` be the width of our remaining packet window, and
389+Otherwise, we let `wid` be the width of our remaining packet window, and we
413−we initialize `rub` to nil. `rub` will be the list of packets that are
390+initialize `rub` to nil. `rub` will be the list of packets that are ready to be
414−ready to be sent. We then call `++apse` and pass the result to `++abet`.
391+sent. We then call `++apse` and pass the result to `++abet`. `++apse` decides
415−`++apse` decides which packets are ready to be sent.
392+which packets are ready to be sent.
416
393
417 ++ apse
394 ++ apse
418 ^+ .
395 ^+ .
@@ -432,20 +409,19 @@ ready to be sent. We then call `++apse` and pass the result to `++abet`.
432 lys.q.n.puq now
409 lys.q.n.puq now
433 ==
410 ==
434
411
435−If there are no remaining packets to send, or if we've filled the packet
412+If there are no remaining packets to send, or if we've filled the packet window,
436−window, do nothing. We call `++rigt` and `++left` to process the left
413+do nothing. We call `++rigt` and `++left` to process the left and right branches
437−and right branches of the packet queue.
414+of the packet queue.
438
415
439−Now we assert that the queue is not empty, and we again check that we
416+Now we assert that the queue is not empty, and we again check that we haven't
440−haven't filled the packet window. We will operate on the head of the
417+filled the packet window. We will operate on the head of the queue. If the
441−queue. If the packet is live, then do nothing. Otherwise, we go ahead
418+packet is live, then do nothing. Otherwise, we go ahead and send it.
442−and send it.
443
419
444−To send, we (1) decrement `wid`, our packet window width; (2) cons the
420+To send, we (1) decrement `wid`, our packet window width; (2) cons the packet
445−packet onto the `rub`, which will be returned as the list of packets to
421+onto the `rub`, which will be returned as the list of packets to send; (3)
446−send; (3) increment `nif`, the number of live packets; (4) set the
422+increment `nif`, the number of live packets; (4) set the packet to be live; (5)
447−packet to be live; (5) increment the number of transmissions of the
423+increment the number of transmissions of the packet; and (6) set the last sent
448−packet; and (6) set the last sent time of the packet to now.
424+time of the packet to now.
449
425
450 ++ left
426 ++ left
451 ?> ?=(^ puq)
427 ?> ?=(^ puq)
@@ -454,8 +430,8 @@ packet; and (6) set the last sent time of the packet to now.
454 ?> ?=(^ puq)
430 ?> ?=(^ puq)
455 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
431 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
456
432
457−These do exactly what you would expect: they traverse the packet queue
433+These do exactly what you would expect: they traverse the packet queue so that
458−so that `++apse` gets called recursively through it.
434+`++apse` gets called recursively through it.
459
435
460Finally, `++abet` gets called, which resolves the changes.
436Finally, `++abet` gets called, which resolves the changes.
461
437
@@ -463,14 +439,13 @@ Finally, `++abet` gets called, which resolves the changes.
463 ?~ rub [~ +>.$]
439 ?~ rub [~ +>.$]
464 [(flop rub) +>.$(rtn [~ (add rto now)])]
440 [(flop rub) +>.$(rtn [~ (add rto now)])]
465
441
466−This returns the packets that we wish to send, and it updates the
442+This returns the packets that we wish to send, and it updates the timeout so
467−timeout so that we know when to try resending unacknowledged packets.
443+that we know when to try resending unacknowledged packets.
468
444
469−This concludes our discussion of `++whap:pu`. To finish
445+This concludes our discussion of `++whap:pu`. To finish `++wind:ho:um:am`, we
470−`++wind:ho:um:am`, we just need to delve into `++busk:ho:um:am`. But
446+just need to delve into `++busk:ho:um:am`. But wait, in the call to `++busk`,
471−wait, in the call to `++busk`, the first argument is `xong:diz`. What is
447+the first argument is `xong:diz`. What is this? This, my dear reader, is one
472−this? This, my dear reader, is one more detour, this time into
448+more detour, this time into `++xong:lax:as:go`.
473−`++xong:lax:as:go`.
474
449
475 ++ xong :: xong:lax:as:go
450 ++ xong :: xong:lax:as:go
476 ^- (list ship) :: route unto
451 ^- (list ship) :: route unto
@@ -484,26 +459,24 @@ this? This, my dear reader, is one more detour, this time into
484 :: ~& [%xong-to [our her] (weld oot ?>(?=(^ fro) t.fro))]
459 :: ~& [%xong-to [our her] (weld oot ?>(?=(^ fro) t.fro))]
485 (weld oot ?>(?=(^ fro) t.fro))
460 (weld oot ?>(?=(^ fro) t.fro))
486
461
487−This gets the list of intermediate ships needed to get a packet from us
462+This gets the list of intermediate ships needed to get a packet from us to our
488−to our neighbor. First, we get `fro` and `too`, the "canons" of ourself
463+neighbor. First, we get `fro` and `too`, the "canons" of ourself and our
489−and our neighbor, respectively.
464+neighbor, respectively.
490
465
491−What is this "canon", you ask? A canon is simply a ship plus its
466+What is this "canon", you ask? A canon is simply a ship plus its "ancestors", as
492−"ancestors", as defined by `++sein`. For example, the canon of
467+defined by `++sein`. For example, the canon of `~hoclur-bicrel` is:
493−`~hoclur-bicrel` is:
494
468
495 ~hoclur-bicrel/try=> (saxo ~hoclur-bicrel)
469 ~hoclur-bicrel/try=> (saxo ~hoclur-bicrel)
496 ~[~hoclur-bicrel ~tasruc ~tug]
470 ~[~hoclur-bicrel ~tasruc ~tug]
497
471
498−If we follow the algorithm in `++xong`, we see that we are simply
472+If we follow the algorithm in `++xong`, we see that we are simply creating a
499−creating a list of ships that form a path from our neighbor to ourself.
473+list of ships that form a path from our neighbor to ourself. Essentially, we
500−Essentially, we look through the canon of our neighbor until we find
474+look through the canon of our neighbor until we find something in our own cannon
501−something in our own cannon -- a common ancestor. Or, if we are from
475+-- a common ancestor. Or, if we are from different carriers, then there is no
502−different carriers, then there is no common ancestor. We then weld this
503−onto the tail of our own canon. In the end, this is simply a list of
476+common ancestor. We then weld this onto the tail of our own canon. In the end,
504−possible ships to try to route via to get to our neighbor, ordered by
477+this is simply a list of possible ships to try to route via to get to our
505−preferability (that is, closeness to our neighbor). We will end up
478+neighbor, ordered by preferability (that is, closeness to our neighbor). We will
506−trying, in order, to find a lane to these.
479+end up trying, in order, to find a lane to these.
507
480
508Now, we can finally get to `++busk:ho:um:am`.
481Now, we can finally get to `++busk:ho:um:am`.
509
482
@@ -516,10 +489,9 @@ Now, we can finally get to `++busk:ho:um:am`.
516 $(pax t.pax, bin (weld (flop (wist:diz now waz ~ i.pax)) bin))
489 $(pax t.pax, bin (weld (flop (wist:diz now waz ~ i.pax)) bin))
517 ==
490 ==
518
491
519−Thankfully, `++busk` is fairly simple. We go through the list of packets
492+Thankfully, `++busk` is fairly simple. We go through the list of packets and
520−and convert them to `++boon`s with `++wist:lax:as:go`. These boons are
493+convert them to `++boon`s with `++wist:lax:as:go`. These boons are placed into
521−placed into `bin`, and they end up getting processed by `++clop` (this
494+`bin`, and they end up getting processed by `++clop` (this happens in `++knob`).
522−happens in `++knob`).
523
495
524 ++ wist :: wist:lax:as:go
496 ++ wist :: wist:lax:as:go
525 |= $: now=@da :: route via
497 |= $: now=@da :: route via
@@ -554,53 +526,50 @@ happens in `++knob`).
554This takes a sample of the current time, the list of ships that we just
526This takes a sample of the current time, the list of ships that we just
555generated, a lane if we already know it, and the packet itself.
527generated, a lane if we already know it, and the packet itself.
556
528
557−First, if we are sending a message to ourself, then we simply create a
529+First, if we are sending a message to ourself, then we simply create a `%ouzo`
558−`%ouzo` boon with a bunted lane. Otherwise, if there are no routing
530+boon with a bunted lane. Otherwise, if there are no routing candidates, there is
559−candidates, there is nothing we can do, so we return nil.
531+nothing we can do, so we return nil.
560−
532+
561−Next, we get the `dore` of the first routing candidate. If we're looking
533+Next, we get the `dore` of the first routing candidate. If we're looking at the
562−at the neighbor to whom we're trying to send the message, then we simply
534+neighbor to whom we're trying to send the message, then we simply use the `dore`
563−use the `dore` that we already have. Otherwise, we get a default `dore`.
535+that we already have. Otherwise, we get a default `dore`.
564−
536+
565−If we're the first routing candidate, or if we have don't have a lane to
537+If we're the first routing candidate, or if we have don't have a lane to this
566−this candidate, then we skip this candidate and move on to the next one.
538+candidate, then we skip this candidate and move on to the next one.
567−
539+
568−If we have only a provisional ip address, then we try to send on it, but
540+If we have only a provisional ip address, then we try to send on it, but we also
569−we also try to send on later routing candidates as well. Otherwise, we
541+try to send on later routing candidates as well. Otherwise, we only send on this
570−only send on this one candidate.
542+one candidate.
571−
543+
572−Finally, we create the actual `%ouzo` boon. The lane is the one from our
544+Finally, we create the actual `%ouzo` boon. The lane is the one from our `dore`.
573−`dore`. If we're sending it directly to our intended recipient, and we
545+If we're sending it directly to our intended recipient, and we haven't been told
574−haven't been told to use a specific lane, then we just send the packet
546+to use a specific lane, then we just send the packet directly. Otherwise, we
575−directly. Otherwise, we wrap it in a little `%fore` meal, telling the
547+wrap it in a little `%fore` meal, telling the intermediary to whom we wish it to
576−intermediary to whom we wish it to be sent. If we have already set up a
577−symmetric key with the intermediary, then we encrypt it with that.
548+be sent. If we have already set up a symmetric key with the intermediary, then
578−Otherwise, we send it in the clear.
549+we encrypt it with that. Otherwise, we send it in the clear.
579−
550+
580−Now, if you recall, we have traced all the way through from the
551+Now, if you recall, we have traced all the way through from the beginning when,
581−beginning when, in `++knob`, the `%wont` card was handled by a call to
552+in `++knob`, the `%wont` card was handled by a call to `++wise`. There is only
582−`++wise`. There is only one more step before the packet is finally sent.
553+one more step before the packet is finally sent. Looking in `++knob`, we see
583−Looking in `++knob`, we see that the resultant list of boons is passed
554+that the resultant list of boons is passed into `++clop`, which will execute the
584−into `++clop`, which will execute the correct actions and return a list
585−of moves. In `++clop`, we see the handling of each specific boon. The
555+correct actions and return a list of moves. In `++clop`, we see the handling of
586−one we are interested in is `%ouzo`, since that is the only one we have
556+each specific boon. The one we are interested in is `%ouzo`, since that is the
587−sent thus far.
557+only one we have sent thus far.
588
558
589 %ouzo
559 %ouzo
590 :: ~& [%send now p.bon `@p`(mug (shaf %flap q.bon))]
560 :: ~& [%send now p.bon `@p`(mug (shaf %flap q.bon))]
591 :_ fox
561 :_ fox
592 [[gad.fox [%give %send p.bon q.bon]] ~]
562 [[gad.fox [%give %send p.bon q.bon]] ~]
593
563
594−Very simply, we give a `%send` gift along the special duct that goes
564+Very simply, we give a `%send` gift along the special duct that goes straight
595−straight into the bowels of unix. This is the last stop before we drop
565+into the bowels of unix. This is the last stop before we drop into vere, and
596−into vere, and later libuv. And then... the world.
566+later libuv. And then... the world.
597
567
598−The packet, after its creation, embarks on a journey across physical
568+The packet, after its creation, embarks on a journey across physical time and
599−time and space into the great unknown. Hurtling through fiber-optic
569+space into the great unknown. Hurtling through fiber-optic cables at hundreds of
600−cables at hundreds of thousands of kilometers per second, it finally
570+thousands of kilometers per second, it finally arrives at our neighbor's network
601−arrives at our neighbor's network adapter. The adapter tells unix, unix
602−tells libuv, libuv tells vere, and vere sends a `%hear` kiss to ames.
571+adapter. The adapter tells unix, unix tells libuv, libuv tells vere, and vere
603−And now we reenter the kernel.
572+sends a `%hear` kiss to ames. And now we reenter the kernel.
604
573
605The `%hear` kiss goes straight to `++knob`, just as did the `%wont` kiss
574The `%hear` kiss goes straight to `++knob`, just as did the `%wont` kiss
606earlier.
575earlier.
@@ -608,14 +577,13 @@ earlier.
608 %hear
577 %hear
609 (~(gnaw am [now fox]) %good p.kyz q.kyz)
578 (~(gnaw am [now fox]) %good p.kyz q.kyz)
610
579
611−Here, though, we call `++gnaw:am` to process the packet. The arguments
580+Here, though, we call `++gnaw:am` to process the packet. The arguments to
612−to `++gnaw` are the same as those to the `%hear` kiss: the lane on which
581+`++gnaw` are the same as those to the `%hear` kiss: the lane on which the packet
613−the packet was received and the packet itself. The other argument is
582+was received and the packet itself. The other argument is just `%good`, which is
614−just `%good`, which is a `++cape` saying that we expect the packet to
583+a `++cape` saying that we expect the packet to succeed. If a formal error
615−succeed. If a formal error occurs, then since we have a transactional
584+occurs, then since we have a transactional event system, the `%hear` event will
616−event system, the `%hear` event will never be considered to have
617−actually happened, and unix will send a `%hole` kiss so that we may send
585+never be considered to have actually happened, and unix will send a `%hole` kiss
618−a negative acknowledgment.
586+so that we may send a negative acknowledgment.
619
587
620 ++ gnaw :: gnaw:am
588 ++ gnaw :: gnaw:am
621 |= [kay=cape ryn=lane pac=rock] :: process packet
589 |= [kay=cape ryn=lane pac=rock] :: process packet
@@ -630,11 +598,10 @@ a negative acknowledgment.
630 %- ~(chew la:(ho:(um q.p.kec) p.p.kec) kay ryn %none (shaf %flap pac))
598 %- ~(chew la:(ho:(um q.p.kec) p.p.kec) kay ryn %none (shaf %flap pac))
631 [q.kec r.kec]
599 [q.kec r.kec]
632
600
633−First, we check the protocol number. If it is not correct, then we
601+First, we check the protocol number. If it is not correct, then we simply ignore
634−simply ignore the packet entirely. Otherwise, we parse the packet with
602+the packet entirely. Otherwise, we parse the packet with `++bite`, which
635−`++bite`, which converts a packet atom into a `cake`, that is, a triple
603+converts a packet atom into a `cake`, that is, a triple of the `sock` (pair of
636−of the `sock` (pair of sender and receiver), the `skin` (encryption
604+sender and receiver), the `skin` (encryption type), and the data.
637−type), and the data.
638
605
639 ++ bite :: packet to cake
606 ++ bite :: packet to cake
640 |= pac=rock ^- cake
607 |= pac=rock ^- cake
@@ -651,26 +618,24 @@ type), and the data.
651 (kins tay)
618 (kins tay)
652 (rsh 3 (add wix vix) bod)
619 (rsh 3 (add wix vix) bod)
653
620
654−This is exactly the inverse of `++spit`. Note that here we check both
621+This is exactly the inverse of `++spit`. Note that here we check both the
655−the protocol number and the hash, crashing on error. Remember that a
622+protocol number and the hash, crashing on error. Remember that a crash will
656−crash will result in a negative acknowledgment being sent.
623+result in a negative acknowledgment being sent.
657
624
658−Continuing in `++gnaw`, we see that if the intended recipient is not on
625+Continuing in `++gnaw`, we see that if the intended recipient is not on our
659−our pier, then we drop the packet.
626+pier, then we drop the packet.
660
627
661−If we've gotten this far, then we wish to process the packet. Recall
628+If we've gotten this far, then we wish to process the packet. Recall that `++ho`
662−that `++ho` and `++um` set up the domestic server and foreign client
629+and `++um` set up the domestic server and foreign client cores, respectively,
663−cores, respectively, and that `++zork` and `++zank` resolve any changes
630+and that `++zork` and `++zank` resolve any changes to these cores.
664−to these cores.
665
631
666−The new stuff here, then, is the `++la` core and the `++chew` arm. The
632+The new stuff here, then, is the `++la` core and the `++chew` arm. The `++la`
667−`++la` sets up a core for this particular packet, containing the current
633+sets up a core for this particular packet, containing the current
668−success/failure `cape`, the lane it was sent on, the encryption type,
634+success/failure `cape`, the lane it was sent on, the encryption type, and a hash
669−and a hash of the packet, used as an id.
635+of the packet, used as an id.
670
636
671−`++chew` is called with the encryption type and the message itself. It
637+`++chew` is called with the encryption type and the message itself. It contains
672−contains a little helper core inside of it, which starts immediately
638+a little helper core inside of it, which starts immediately with `++apse`.
673−with `++apse`.
674
639
675 ++ apse
640 ++ apse
676 ^+ +>.$
641 ^+ +>.$
@@ -687,23 +652,23 @@ with `++apse`.
687 :_(bin [%wine [our her] " is your neighbor"])
652 :_(bin [%wine [our her] " is your neighbor"])
688 +>.$
653 +>.$
689
654
690−First, we let `oub` be true if our neighbor hasn't been responding to us
655+First, we let `oub` be true if our neighbor hasn't been responding to us for
691−for more than sixteen seconds. Let `neg` be true if we haven't yet
656+more than sixteen seconds. Let `neg` be true if we haven't yet proposed a
692−proposed a symmetric key, meaning that we haven't yet corresponded with
657+symmetric key, meaning that we haven't yet corresponded with this ship, so they
693−this ship, so they are not our neighbor. Next, we run `++east`, which
658+are not our neighbor. Next, we run `++east`, which we'll go into in just a
694−we'll go into in just a minute.
659+minute.
695
660
696−We now do the same two checks and store the results in `eng` and `bou`.
661+We now do the same two checks and store the results in `eng` and `bou`. If our
697−If our neighbor has, like the prodigal son, returned after an extended
662+neighbor has, like the prodigal son, returned after an extended absense, then we
698−absense, then we send a `%wine` boon as the proverbial fatted calf,
663+send a `%wine` boon as the proverbial fatted calf, which is simply printed out
699−which is simply printed out to the console. Likewise, if we are meeting
664+to the console. Likewise, if we are meeting one with whom we have never had the
700−one with whom we have never had the pleasure of acquainting ourselves,
701−we send a message to the console to that effect.
665+pleasure of acquainting ourselves, we send a message to the console to that
666+effect.
702
667
703−We skipped over `++east`, which contains the meat of the processing. It
668+We skipped over `++east`, which contains the meat of the processing. It first
704−first decrypts the message, then calls `++chow:la:ho:um:am` with the
669+decrypts the message, then calls `++chow:la:ho:um:am` with the resultant meal.
705−resultant meal. We'll go through each of the four cases in turn, but
670+We'll go through each of the four cases in turn, but first since each one calls
706−first since each one calls `++bilk:pu`, we'll take a brief detour.
671+`++bilk:pu`, we'll take a brief detour.
707
672
708 ++ bilk :: bilk:pu
673 ++ bilk :: bilk:pu
709 |= now=@da :: inbound packet
674 |= now=@da :: inbound packet
@@ -715,11 +680,10 @@ first since each one calls `++bilk:pu`, we'll take a brief detour.
715 rtn ?~(puq ~ [~ (add now trt)])
680 rtn ?~(puq ~ [~ (add now trt)])
716 ==
681 ==
717
682
718−This updates the timing information in our packet pump. `rue`, the last
683+This updates the timing information in our packet pump. `rue`, the last time we
719−time we have heard from this neighbor, is set to now. `rto`, the
684+have heard from this neighbor, is set to now. `rto`, the retransmit timeout is
720−retransmit timeout is set to twice the current ping time, and if there
685+set to twice the current ping time, and if there is anything in the packet
721−is anything in the packet queue, then we reset the next timeout, since
686+queue, then we reset the next timeout, since we've just heard a message.
722−we've just heard a message.
723
687
724Back to `++east`.
688Back to `++east`.
725
689
@@ -727,11 +691,11 @@ Back to `++east`.
727 =. puz (bilk:puz now)
691 =. puz (bilk:puz now)
728 (chow ((hard meal) (cue msg)))
692 (chow ((hard meal) (cue msg)))
729
693
730−The simplest case is when the encryption type is `%none`. We first call
694+The simplest case is when the encryption type is `%none`. We first call `++bilk`
731−`++bilk` to update the packet pump, then we cue (unjam) the message into
695+to update the packet pump, then we cue (unjam) the message into a meal. We hard
732−a meal. We hard cast it into a meal -- if the cast fails, then we do
696+cast it into a meal -- if the cast fails, then we do want to crash since someone
733−want to crash since someone is sending us malformed data. Finally, we
697+is sending us malformed data. Finally, we send the result to `++chow` for
734−send the result to `++chow` for interpretation and handling.
698+interpretation and handling.
735
699
736 %fast
700 %fast
737 =+ [mag=`hand`(end 7 1 msg) bod=(rsh 7 1 msg)]
701 =+ [mag=`hand`(end 7 1 msg) bod=(rsh 7 1 msg)]
@@ -743,15 +707,14 @@ send the result to `++chow` for interpretation and handling.
743 =^ key diz u.dey
707 =^ key diz u.dey
744 (chow(aut sin) ((hard meal) (cue (dy:q:sen:gus key bod))))
708 (chow(aut sin) ((hard meal) (cue (dy:q:sen:gus key bod))))
745
709
746−For symmetric encryption, we first get the `hand`, which is the hash of
710+For symmetric encryption, we first get the `hand`, which is the hash of the
747−the symmetric key. We pass it to `++kuch:lax:as:go`, which returns the
711+symmetric key. We pass it to `++kuch:lax:as:go`, which returns the key if we
748−key if we either have used it before or we have proposed it. If we have
712+either have used it before or we have proposed it. If we have proposed it, then
749−proposed it, then we change its status from proposed to real. If
713+we change its status from proposed to real. If `++kuch` fails, then we drop the
750−`++kuch` fails, then we drop the packet and print out a `%bad-key`
714+packet and print out a `%bad-key` message.
751−message.
752
715
753−Otherwise, we call `++bilk` as before to update the packet pump and pass
716+Otherwise, we call `++bilk` as before to update the packet pump and pass into
754−into `++chow` the decrypted data.
717+`++chow` the decrypted data.
755
718
756 %full
719 %full
757 =+ mex=((hard ,[p=[p=life q=life] q=will r=@]) (cue msg))
720 =+ mex=((hard ,[p=[p=life q=life] q=will r=@]) (cue msg))
@@ -764,17 +727,16 @@ into `++chow` the decrypted data.
764 =. puz (bilk:puz now)
727 =. puz (bilk:puz now)
765 (west(msg q.mes))
728 (west(msg q.mes))
766
729
767−For sealed asymmetric encryption, we first take off the the layer of
730+For sealed asymmetric encryption, we first take off the the layer of data that
768−data that gives us the life and will of our neighbor, and we apply try
731+gives us the life and will of our neighbor, and we apply try to extend their
769−to extend their former will with the new data. `++deng` will fail if
732+former will with the new data. `++deng` will fail if this is impossible.
770−this is impossible.
771
733
772−Next, we get our most current understanding of our neighbor's crypto,
734+Next, we get our most current understanding of our neighbor's crypto, and we
773−and we verify that it's the same life as what they're sending. Then, we
735+verify that it's the same life as what they're sending. Then, we get our own
774−get our own crypto from `++sev` and decrypt the message with the public
736+crypto from `++sev` and decrypt the message with the public key from our
775−key from our neighbor's crypto. We register the proposed symmetric key,
737+neighbor's crypto. We register the proposed symmetric key, update the packet
776−update the packet pump, and call `++west`, which simply casts the
738+pump, and call `++west`, which simply casts the message to a meal and calls
777−message to a meal and calls `++chow`, reporting any error.
739+`++chow`, reporting any error.
778
740
779 %open
741 %open
780 =+ mex=((hard ,[p=[~ q=life] q=will r=@]) (cue msg))
742 =+ mex=((hard ,[p=[~ q=life] q=will r=@]) (cue msg))
@@ -785,14 +747,13 @@ message to a meal and calls `++chow`, reporting any error.
785 =. puz (bilk:puz now)
747 =. puz (bilk:puz now)
786 (west(msg mes))
748 (west(msg mes))
787
749
788−Finally, for signed asymmetric encryption, we, as before, take off the
750+Finally, for signed asymmetric encryption, we, as before, take off the layer of
789−layer of data that gives us the life and will of our neighbor. This
751+data that gives us the life and will of our neighbor. This time, of course, we
790−time, of course, we do not get our own crypto -- only that of our
752+do not get our own crypto -- only that of our neighbor.
791−neighbor.
792
753
793−The rest you have seen. We call `++deng` to extend the will, we verify
754+The rest you have seen. We call `++deng` to extend the will, we verify that
794−that their crypto life is what we think it ought to be, we "decrypt" the
755+their crypto life is what we think it ought to be, we "decrypt" the data, we
795−data, we update the packet pump, and we call `++west` to call `++chow`.
756+update the packet pump, and we call `++west` to call `++chow`.
796
757
797 ++ chow :: chow:la:ho:um:am
758 ++ chow :: chow:la:ho:um:am
798 |= fud=meal :: interpret meal
759 |= fud=meal :: interpret meal
@@ -800,15 +761,14 @@ data, we update the packet pump, and we call `++west` to call `++chow`.
800 =. diz ?:(=(%none aut) diz (wast:diz ryn))
761 =. diz ?:(=(%none aut) diz (wast:diz ryn))
801 (dine fud)
762 (dine fud)
802
763
803−Here, if the message was encrypted at all, then we call
764+Here, if the message was encrypted at all, then we call `++wast:lax:as:go`,
804−`++wast:lax:as:go`, which simply updates the lane (route) to our
765+which simply updates the lane (route) to our neighbor (unless we're given a
805−neighbor (unless we're given a provisional route). This ensures that we
806−always have the most direct possible path to them.
766+provisional route). This ensures that we always have the most direct possible
767+path to them.
807
768
808−We've been handling this meal for so long, we've almost forgotten what
769+We've been handling this meal for so long, we've almost forgotten what we want
809−we want to do with it. The telos is of any meal to be dined on. We will
770+to do with it. The telos is of any meal to be dined on. We will choose out the
810−choose out the cases here that are important to our current
771+cases here that are important to our current investigation.
811−investigation.
812
772
813 %fore
773 %fore
814 =+ ^= lyn ^- lane
774 =+ ^= lyn ^- lane
@@ -821,12 +781,12 @@ investigation.
821 =+ zid=(myx:gus p.fud)
781 =+ zid=(myx:gus p.fud)
822 (emir (wist:zid now xong:zid [~ lyn] r.fud))
782 (emir (wist:zid now xong:zid [~ lyn] r.fud))
823
783
824−Forwarding is the simplest case, since we've seen all the arms before,
784+Forwarding is the simplest case, since we've seen all the arms before, except
825−except perhaps `++emit` and `++emir`, which simply take a boon or list
785+perhaps `++emit` and `++emir`, which simply take a boon or list of boons
826−of boons respectively and queue them up to be handled when the core
786+respectively and queue them up to be handled when the core resolves. If we're
827−resolves. If we're told to forward a packet to ourselves, then we emit a
787+told to forward a packet to ourselves, then we emit a `%mead` boon which simply
828−`%mead` boon which simply sends another `%hear` kiss to ourselves with
788+sends another `%hear` kiss to ourselves with the data. Otherwise, we try to find
829−the data. Otherwise, we try to find a route to the recipient, as before.
789+a route to the recipient, as before.
830
790
831 %carp
791 %carp
832 =+ zol=(~(get by olz.weg) s.fud)
792 =+ zol=(~(get by olz.weg) s.fud)
@@ -855,31 +815,30 @@ the data. Otherwise, we try to find a route to the recipient, as before.
855 +>.$(nys.weg (~(put by nys.weg) s.fud neb))
815 +>.$(nys.weg (~(put by nys.weg) s.fud neb))
856
816
857Here, we have received a partial message, and we're just assembling the
817Here, we have received a partial message, and we're just assembling the
858−individual packets into a message. Most of this code is fairly
818+individual packets into a message. Most of this code is fairly algorithmic, so
859−algorithmic, so we'll just hit the high points. In the beginning, we
819+we'll just hit the high points. In the beginning, we check if we've already
860−check if we've already received this message, and if so, we resend the
820+received this message, and if so, we resend the acknowledgment. Remember,
861−acknowledgment. Remember, "always ack a dupe, never ack an ack".
821+"always ack a dupe, never ack an ack".
862−
822+
863−In `nys.weg` we keep track of an incoming set of partial packets,
823+In `nys.weg` we keep track of an incoming set of partial packets, indexed by the
864−indexed by the `flap` hash that comes with every packet. We check to see
824+`flap` hash that comes with every packet. We check to see if we have already
865−if we have already received this partial message, and if so we
825+received this partial message, and if so we acknowledge it. Otherwise, we put it
866−acknowledge it. Otherwise, we put it in `nys.weg` unless this is the
867−last message, in which case we ack the last partial message, move the
826+in `nys.weg` unless this is the last message, in which case we ack the last
868−complete message into `olz.weg`, and call `++golf`, which assembles the
827+partial message, move the complete message into `olz.weg`, and call `++golf`,
869−message and calls `++chew`, to start the dance again with the complete
828+which assembles the message and calls `++chew`, to start the dance again with
870−message.
829+the complete message.
871
830
872 %bund
831 %bund
873 :: ~& [%bund q.fud r.fud]
832 :: ~& [%bund q.fud r.fud]
874 ?> =(p:sen:gus p.fud)
833 ?> =(p:sen:gus p.fud)
875 (deer q.fud r.fud ?-(kay %dead ~, %good [~ s.fud]))
834 (deer q.fud r.fud ?-(kay %dead ~, %good [~ s.fud]))
876
835
877−What if we're just receiving a regular old, garden variety message? We
836+What if we're just receiving a regular old, garden variety message? We call
878−call `++deer` with the data from the message. If we already know that
837+`++deer` with the data from the message. If we already know that the message
879−the message processing will fail (that is, if we got a `%hole` card from
838+processing will fail (that is, if we got a `%hole` card from unix rather than a
880−unix rather than a `%hear` card), then we don't even send the data at
839+`%hear` card), then we don't even send the data at all. Remember, if a packet
881−all. Remember, if a packet fails to process, it's as if it never even
840+fails to process, it's as if it never even arrived, except that we send a
882−arrived, except that we send a negative acknowledgment.
841+negative acknowledgment.
883
842
884 ++ deer :: deer:la:ho:um:am
843 ++ deer :: deer:la:ho:um:am
885 |= [cha=path num=@ud dut=(unit)] :: interpret message
844 |= [cha=path num=@ud dut=(unit)] :: interpret message
@@ -898,24 +857,22 @@ arrived, except that we send a negative acknowledgment.
898 ==
857 ==
899 ==
858 ==
900
859
901−First, we get the race for this particular triple of sender, receiver,
860+First, we get the race for this particular triple of sender, receiver, and path,
902−and path, creating it if it doesn't exist. If we've already acked the
861+creating it if it doesn't exist. If we've already acked the message, then we
903−message, then we resend the ack. Note that `did.rum` is the number of
862+resend the ack. Note that `did.rum` is the number of packets we acknowledged,
904−packets we acknowledged, positively or negatively while `bum.rum` is a
863+positively or negatively while `bum.rum` is a map of message numbers to negative
905−map of message numbers to negative acknowledgments. Thus, if a message
906−number is less than `did.rum`, then if it's in `bum.rum` then it was
864+acknowledgments. Thus, if a message number is less than `did.rum`, then if it's
907−negatively acknowledged, otherwise it's postively acknowledged. Thus, we
865+in `bum.rum` then it was negatively acknowledged, otherwise it's postively
908−are constant in space with the number of successful messages and linear
866+acknowledged. Thus, we are constant in space with the number of successful
909−in the number of failed messages. We'll document `++cook` later on, but
867+messages and linear in the number of failed messages. We'll document `++cook`
910−suffice it to say that it sends an acknowledgment. It is to end-to-end
868+later on, but suffice it to say that it sends an acknowledgment. It is to
911−acknowledgments what `++cock` is to packet-level acknowledgments.
869+end-to-end acknowledgments what `++cock` is to packet-level acknowledgments.
912−
870+
913−If we are still processing a message (that is, `dod.rum` is false), then
871+If we are still processing a message (that is, `dod.rum` is false), then we
914−we simply put this message in the map of misordered packets to be
872+simply put this message in the map of misordered packets to be processed when
915−processed when their time comes. "Processing a message" in this case
873+their time comes. "Processing a message" in this case means that we've received
916−means that we've received the message and notified the correct
917−application, but we're still waiting for the application-level
874+the message and notified the correct application, but we're still waiting for
918−acknowledgment.
875+the application-level acknowledgment.
919
876
920Otherwise, we're ready for a packet, so we process it.
877Otherwise, we're ready for a packet, so we process it.
921
878
@@ -937,20 +894,18 @@ Otherwise, we're ready for a packet, so we process it.
937 u.s.u.cun
894 u.s.u.cun
938 ==
895 ==
939
896
940−First, we grab the message we want to process and store it in `cun`. If
897+First, we grab the message we want to process and store it in `cun`. If it's a
941−it's a good packet, then we change `dod.rum` to false, meaning that
898+good packet, then we change `dod.rum` to false, meaning that we're in the middle
942−we're in the middle of processing a packet and should not start
899+of processing a packet and should not start processing another one. We also put
943−processing another one. We also put a `%mulk` boon into the queue so
944−that, when it all resolves, we send a mesage to the intended recipient
900+a `%mulk` boon into the queue so that, when it all resolves, we send a mesage to
945−application. The boon contains the sender, the receiver, the identity of
901+the intended recipient application. The boon contains the sender, the receiver,
946−the message, and the message itself.
902+the identity of the message, and the message itself.
947−
903+
948−This bubbles up all the way back to `++knob`, where we were handling the
904+This bubbles up all the way back to `++knob`, where we were handling the `%hear`
949−`%hear` card. Following the logic in `++knob`, we can see that the boons
905+card. Following the logic in `++knob`, we can see that the boons get sent into
950−get sent into `++clop` to be turned into actual arvo-level moves. We've
906+`++clop` to be turned into actual arvo-level moves. We've been here before, if
951−been here before, if you recall, when we handled the `%cake` boon to
907+you recall, when we handled the `%cake` boon to send a message. Now, we're
952−send a message. Now, we're handling the `%mulk` boon, which is
953−unfortunately slightly more complicated.
908+handling the `%mulk` boon, which is unfortunately slightly more complicated.
954
909
955 %mulk
910 %mulk
956 :: ~& [%mulk p.bon q.bon]
911 :: ~& [%mulk p.bon q.bon]
@@ -980,51 +935,48 @@ unfortunately slightly more complicated.
980 :_ fox [hen %pass pax %g %roth p.bon app r.bon]~
935 :_ fox [hen %pass pax %g %roth p.bon app r.bon]~
981 ==
936 ==
982
937
983−We're dispatching messages based on the prefix of their path. Since only
938+We're dispatching messages based on the prefix of their path. Since only `%gall`
984−`%gall` apps use end-to-end acknowledgments at the moment, every path
939+apps use end-to-end acknowledgments at the moment, every path must have at least
985−must have at least two elements, and the first one must be `%q`. Beyond
940+two elements, and the first one must be `%q`. Beyond that, we handle the `/q/ge`
986−that, we handle the `/q/ge` and `/q/gh` cases for gall requests and
941+and `/q/gh` cases for gall requests and responses, respectively.
987−responses, respectively.
942+
988−
989−In both cases, we require the next term in the path to be the name of
943+In both cases, we require the next term in the path to be the name of the
990−the intended recipient `%gall` app. Thus, a message to `/q/ge/talk` for
944+intended recipient `%gall` app. Thus, a message to `/q/ge/talk` for example,
991−example, will send a message to the talk app.
945+will send a message to the talk app.
992−
946+
993−We then send a message to the app itself. The message is either a
947+We then send a message to the app itself. The message is either a `%rote` or a
994−`%rote` or a `%roth` for a request and a response, respectively. The
948+`%roth` for a request and a response, respectively. The content is the `rook` or
995−content is the `rook` or `roon` that was sent (stored in `r.bon`), but
949+`roon` that was sent (stored in `r.bon`), but we don't actually handle that at
996−we don't actually handle that at all here. That's completely a
997−`%gall`-level thing. We're just the messenger.
950+all here. That's completely a `%gall`-level thing. We're just the messenger.
998−
951+
999−Notice the path we send this over. We encode the sender, the receiver,
952+Notice the path we send this over. We encode the sender, the receiver, and the
1000−and the path over which it was sent. This fully specifies the `race` so
953+path over which it was sent. This fully specifies the `race` so that when the
1001−that when the app gives us the acknowledgment we know where to send it.
954+app gives us the acknowledgment we know where to send it.
1002−
955+
1003−We now have another interlude. We have entrusted our precious data, so
956+We now have another interlude. We have entrusted our precious data, so carefully
1004−carefully guarded and guided from the app on that far-away ship, to our
957+guarded and guided from the app on that far-away ship, to our local app. It has
1005−local app. It has the ability to do whatever it pleases with it. It may
958+the ability to do whatever it pleases with it. It may take a significant amount
1006−take a significant amount of time to process. When the message has been
959+of time to process. When the message has been handled by this app, though, it
1007−handled by this app, though, it must produce an acknowledgment. Our
1008−final task is to deliver this acknowledgment to the sending app.
960+must produce an acknowledgment. Our final task is to deliver this acknowledgment
961+to the sending app.
1009
962
1010We should describe here what exactly these oft-mentioned acknowledgments
963We should describe here what exactly these oft-mentioned acknowledgments
1011−actually consist of. There are two kinds of acknowledgments: positive
964+actually consist of. There are two kinds of acknowledgments: positive and
1012−and negative. A positive acknowledgment contains no data other than its
965+negative. A positive acknowledgment contains no data other than its existence. A
1013−existence. A negative acknowledgment may optionally include a reason for
966+negative acknowledgment may optionally include a reason for said negativity.
1014−said negativity. Formally, a negative acknowledgment is an `ares`, which
967+Formally, a negative acknowledgment is an `ares`, which is a unit pair of a term
1015−is a unit pair of a term and a list of tanks. If this is null, this is
968+and a list of tanks. If this is null, this is simply a failure with no
1016−simply a failure with no associated information. If the pair exists, the
969+associated information. If the pair exists, the term is a short error code that
1017−term is a short error code that is usually both human and computer
1018−readable. For example, if you try to send a message to a valid `%gall`
970+is usually both human and computer readable. For example, if you try to send a
1019−app that doesn't have any `++poke` to handle it, then `%gall` will give
971+message to a valid `%gall` app that doesn't have any `++poke` to handle it, then
1020−a negative acknowledgment with error term `%poke-find-fail`. The list of
972+`%gall` will give a negative acknowledgment with error term `%poke-find-fail`.
1021−tanks is a human-readable description of the error. This often contains
973+The list of tanks is a human-readable description of the error. This often
1022−a stack trace. At any rate, all this information is returned to the
974+contains a stack trace. At any rate, all this information is returned to the
1023sending app on the other end of the wire.
975sending app on the other end of the wire.
1024
976
1025−After this brief interlude, our story resumes in `++knap`, where we
977+After this brief interlude, our story resumes in `++knap`, where we receive
1026−receive responses. In particular, a `%mean` indicates a negative
978+responses. In particular, a `%mean` indicates a negative acknowledgment while a
1027−acknowledgment while a `%nice` indicates a positive acknowledgment.
979+`%nice` indicates a positive acknowledgment.
1028
980
1029 ?(%mean %nice)
981 ?(%mean %nice)
1030 ?> ?=([@ @ @ *] tea)
982 ?> ?=([@ @ @ *] tea)
@@ -1042,11 +994,10 @@ acknowledgment while a `%nice` indicates a positive acknowledgment.
1042 =^ toe fox (clop now hen i.p.fuy)
994 =^ toe fox (clop now hen i.p.fuy)
1043 $(p.fuy t.p.fuy, out (weld (flop toe) out))
995 $(p.fuy t.p.fuy, out (weld (flop toe) out))
1044
996
1045−Recall the format of the path we sent the message on, and you'll
997+Recall the format of the path we sent the message on, and you'll understand why
1046−understand why `soq` and `pax` are the sender/receiver pair and path on
998+`soq` and `pax` are the sender/receiver pair and path on which the message was
1047−which the message was sent. The rest of this is structured much like
999+sent. The rest of this is structured much like `++knob`, so we call `++rack:am`
1048−`++knob`, so we call `++rack:am` and send the resulting boons to
1049−`++clop`. Business as usual.
1000+and send the resulting boons to `++clop`. Business as usual.
1050
1001
1051 ++ rack :: rack:am
1002 ++ rack :: rack:am
1052 |= [soq=sock cha=path cop=coop] :: e2e ack
1003 |= [soq=sock cha=path cop=coop] :: e2e ack
@@ -1054,11 +1005,10 @@ which the message was sent. The rest of this is structured much like
1054 =. oh (cook:oh cop cha ~)
1005 =. oh (cook:oh cop cha ~)
1055 (cans:oh cha)
1006 (cans:oh cha)
1056
1007
1057−First, we set up `++um` and `++ho`, as we've done twice before, for our
1008+First, we set up `++um` and `++ho`, as we've done twice before, for our domestic
1058−domestic and foreign servers, respectively. The other two things are
1009+and foreign servers, respectively. The other two things are new, though. Well,
1059−new, though. Well, `++cook` is not actually new, but we delayed the
1010+`++cook` is not actually new, but we delayed the explanation saying only that it
1060−explanation saying only that it sends an acknowledgment. The time has
1011+sends an acknowledgment. The time has come.
1061−come.
1062
1012
1063 ++ cook :: cook:ho:um:am
1013 ++ cook :: cook:ho:um:am
1064 |= [cop=coop cha=path ram=(unit ,[ryn=lane dam=flap])]
1014 |= [cop=coop cha=path ram=(unit ,[ryn=lane dam=flap])]
@@ -1077,61 +1027,58 @@ come.
1077 =^ roc diz (zuul:diz now [%buck cop dam ~s0])
1027 =^ roc diz (zuul:diz now [%buck cop dam ~s0])
1078 (busk(diz (wast:diz ryn)) xong:diz roc)
1028 (busk(diz (wast:diz ryn)) xong:diz roc)
1079
1029
1080−If we are acknowledging a message that we have already acked, the `ram`
1030+If we are acknowledging a message that we have already acked, the `ram` will
1081−will contain the new lane and flap to send the duplicate ack to. This
1031+contain the new lane and flap to send the duplicate ack to. This happens if we
1082−happens if we call `++cook` in `++deer`, but it doesn't happen from
1032+call `++cook` in `++deer`, but it doesn't happen from `++rack`. If there is no
1083−`++rack`. If there is no message waiting to be acknowledged and we're
1033+message waiting to be acknowledged and we're not given an explicit lane and flap
1084−not given an explicit lane and flap (that is, we're not sending a
1085−duplicate ack), then the app must have sent us multiple acknowledgments.
1034+(that is, we're not sending a duplicate ack), then the app must have sent us
1086−We do the only sensible thing we can do and drop all acknowledgments
1035+multiple acknowledgments. We do the only sensible thing we can do and drop all
1087−after the first, printing a message. This is, in fact, an error, so it
1036+acknowledgments after the first, printing a message. This is, in fact, an error,
1088−could be argued that we ought to crash. Whatever you do, don't depend on
1037+so it could be argued that we ought to crash. Whatever you do, don't depend on
1089this not crashing.
1038this not crashing.
1090
1039
1091−First, we grab the race specified by the given path, and we get the most
1040+First, we grab the race specified by the given path, and we get the most recent
1092−recent in-order message, which must be the one which is being
1041+in-order message, which must be the one which is being acknowledged.
1093−acknowledged.
1042+
1094−
1095−Then, we decide which lane/flap to respond on/to. Basically, in the
1043+Then, we decide which lane/flap to respond on/to. Basically, in the usual case
1096−usual case we respond on the lane through which the initial message was
1044+we respond on the lane through which the initial message was sent, which is
1097−sent, which is stored along with the other packet information in
1045+stored along with the other packet information in `mis.rum`, since it has to be
1098−`mis.rum`, since it has to be remembered across calls to ames. However,
1046+remembered across calls to ames. However, if we receive a duplicate message,
1099−if we receive a duplicate message, then we must respond to the new
1100−message. It's quite possible the reason the other acknowledgment didn't
1047+then we must respond to the new message. It's quite possible the reason the
1101−get returned was that the lane between the ships was broken.
1048+other acknowledgment didn't get returned was that the lane between the ships was
1102−
1049+broken.
1050+
1103−At any rate, we update the race by saying that we've finished processing
1051+At any rate, we update the race by saying that we've finished processing this
1104−this packet (unless we're sending a duplicate ack) and, if we're sending
1052+packet (unless we're sending a duplicate ack) and, if we're sending a negative
1105−a negative acknowledgment, putting the negative ack into `bum.rum` so
1053+acknowledgment, putting the negative ack into `bum.rum` so that we can resend it
1106−that we can resend it if necessary.
1054+if necessary.
1107−
1055+
1108−We encode our new message, updating the packet pump, with `++zuul`, as
1056+We encode our new message, updating the packet pump, with `++zuul`, as before,
1109−before, and we send it off with `++busk`, routed via `++wast` to one of
1057+and we send it off with `++busk`, routed via `++wast` to one of the ships in
1110−the ships in `++xong`. Of course, in practice, we don't even look at the
1058+`++xong`. Of course, in practice, we don't even look at the ships in `++xong`
1111−ships in `++xong` because we already have a lane directly to our
1059+because we already have a lane directly to our neighbor (the one over which they
1112−neighbor (the one over which they sent their message to us).
1060+sent their message to us).
1113−
1061+
1114−We glossed over the actual message we're sending back. We're sending a
1062+We glossed over the actual message we're sending back. We're sending a `%buck`
1115−`%buck` meal, which is an acknowledgment. The `cop` specifies whether
1063+meal, which is an acknowledgment. The `cop` specifies whether this is a positive
1116−this is a positive or a negative ack, `dam` specifies the message we're
1064+or a negative ack, `dam` specifies the message we're acknowledging, and the
1117−acknowledging, and the `~s0` is a placeholder for the processing time
1065+`~s0` is a placeholder for the processing time required. This time is neither
1118−required. This time is neither calculated (though it is hopefully
1119−obvious how to do so) nor used at present, but this information may be
1066+calculated (though it is hopefully obvious how to do so) nor used at present,
1120−used in the future for improved congestion control. Since the round-trip
1067+but this information may be used in the future for improved congestion control.
1121−time for an end-to-end acknowledged packet includes the processing time
1068+Since the round-trip time for an end-to-end acknowledged packet includes the
1122−on the other end, most common congestion control algorithms will stumble
1069+processing time on the other end, most common congestion control algorithms will
1123−when some messages take much longer to process than others. As noted,
1070+stumble when some messages take much longer to process than others. As noted,
1124−though, this is simply an opportunity for improvement -- our congestion
1071+though, this is simply an opportunity for improvement -- our congestion control
1125−control algorithms are relatively naive at the moment.
1072+algorithms are relatively naive at the moment.
1126−
1073+
1127−Recall that `++busk` calls `++wist` to put the actual `%ouzo` boon in
1074+Recall that `++busk` calls `++wist` to put the actual `%ouzo` boon in the queue,
1128−the queue, which gets handled by `++clop` to actually send the message.
1075+which gets handled by `++clop` to actually send the message. This is the same
1129−This is the same pipeline as sending any other message, so we'll refer
1076+pipeline as sending any other message, so we'll refer you to the explanation
1130−you to the explanation above if you've forgotten it.
1077+above if you've forgotten it.
1131−
1078+
1132−The last thing we need to do on this ship is move on to the next packet
1079+The last thing we need to do on this ship is move on to the next packet in the
1133−in the queue if there is one. If you recall, in `++rack` after the call
1080+queue if there is one. If you recall, in `++rack` after the call to `++cook`
1134−to `++cook` there was a call to `++cans:ho:um:am`.
1081+there was a call to `++cans:ho:um:am`.
1135
1082
1136 ++ cans :: cans:ho:um:am
1083 ++ cans :: cans:ho:um:am
1137 |= cha=path
1084 |= cha=path
@@ -1143,38 +1090,37 @@ to `++cook` there was a call to `++cans:ho:um:am`.
1143 ==
1090 ==
1144 (coat cha rum)
1091 (coat cha rum)
1145
1092
1146−This is very simple. We increment the number of packets that we've
1093+This is very simple. We increment the number of packets that we've acknowledged
1147−acknowledged on this race and we delete the packet that we just
1094+on this race and we delete the packet that we just acknowledged from the set of
1148−acknowledged from the set of misordered packets.
1095+misordered packets.
1149
1096
1150Then, we call `++coat` again to process the next packet if we've already
1097Then, we call `++coat` again to process the next packet if we've already
1151received it. And that's it for this.
1098received it. And that's it for this.
1152
1099
1153−The acknowledgment now travels the same path that its forebearer, the
1100+The acknowledgment now travels the same path that its forebearer, the original
1154−original message, once tread, but this time not into the great unknown.
1101+message, once tread, but this time not into the great unknown. The weary
1155−The weary traveler is seeking out its familial roots, finding the app
1102+traveler is seeking out its familial roots, finding the app from whom sprung
1156−from whom sprung forth the original message way back in paragraph three.
1103+forth the original message way back in paragraph three. When it arrives at the
1157−When it arrives at the network adapter of its ancestors, the adapter
1104+network adapter of its ancestors, the adapter tells unix, unix tells libuv,
1158−tells unix, unix tells libuv, libuv tells vere, and vere sends a `%hear`
1159−kiss to ames. Once more into the kernel.
1105+libuv tells vere, and vere sends a `%hear` kiss to ames. Once more into the
1106+kernel.
1160
1107
1161−The `%hear` kiss is handled in `++knob` as before, leading to `++gnaw`,
1108+The `%hear` kiss is handled in `++knob` as before, leading to `++gnaw`, going
1162−going over to `++chew`, `++apse`, `++chow`, and eventualy to `++dine`.
1109+over to `++chew`, `++apse`, `++chow`, and eventualy to `++dine`. We've seen most
1163−We've seen most of the cases in `++dine`, but we haven't yet looked at
1110+of the cases in `++dine`, but we haven't yet looked at the handling of this
1164−the handling of this `%buck` meal.
1111+`%buck` meal.
1165
1112
1166 %buck
1113 %buck
1167 =. +> ?.(=(%full aut) +> cock) :: finish key exch
1114 =. +> ?.(=(%full aut) +> cock) :: finish key exch
1168 +>(..la (tock p.fud q.fud r.fud))
1115 +>(..la (tock p.fud q.fud r.fud))
1169
1116
1170−We send a packet level acknowledgment if we're finishing a key exchange,
1117+We send a packet level acknowledgment if we're finishing a key exchange, else we
1171−else we call `++tock` to process the acknowledgment.
1118+call `++tock` to process the acknowledgment.
1172
1119
1173−This will get a little involved, so if you don't much care about how
1120+This will get a little involved, so if you don't much care about how exactly an
1174−exactly an acknowledgment happens, just know that the result gets gifted
1121+acknowledgment happens, just know that the result gets gifted as a `%woot` card
1175−as a `%woot` card back to the app who sent it. For those brave souls who
1122+back to the app who sent it. For those brave souls who wish to see this thing
1176−wish to see this thing through to the end, it's once more into the
1123+through to the end, it's once more into the breach.
1177−breach.
1178
1124
1179 ++ tock :: tock:ho:um:am
1125 ++ tock :: tock:ho:um:am
1180 |= [cop=coop fap=flap cot=@dr] :: e2e ack by hash
1126 |= [cop=coop fap=flap cot=@dr] :: e2e ack by hash
@@ -1192,9 +1138,9 @@ breach.
1192 ==
1138 ==
1193 (busk xong:diz q.yoh)
1139 (busk xong:diz q.yoh)
1194
1140
1195−We're going to work through this one a little backwards since it's
1141+We're going to work through this one a little backwards since it's mostly fairly
1196−mostly fairly simple except the call to `++bick:pu`. In fact, we'll just
1142+simple except the call to `++bick:pu`. In fact, we'll just skip `++bick` for the
1197−skip `++bick` for the moment and finish the rest.
1143+moment and finish the rest.
1198
1144
1199If `++bick` succesfully acks the message, then we call `++done`.
1145If `++bick` succesfully acks the message, then we call `++done`.
1200
1146
@@ -1210,37 +1156,35 @@ If `++bick` succesfully acks the message, then we call `++done`.
1210 (~(put by ryl.bah) cha rol(san (~(del by san.rol) num)))
1156 (~(put by ryl.bah) cha rol(san (~(del by san.rol) num)))
1211 ==
1157 ==
1212
1158
1213−This very simply gets the rill (the outgoing counterpart to a race, if
1159+This very simply gets the rill (the outgoing counterpart to a race, if you
1214−you recall), pulls out of the map of outstanding messages the duct over
1160+recall), pulls out of the map of outstanding messages the duct over which the
1215−which the original message was sent, and produces this duct while
1161+original message was sent, and produces this duct while deleting that entry from
1216−deleting that entry from the map of outstanding messages.
1162+the map of outstanding messages.
1217
1163
1218−Going back to `++tock`, we now have the duct we need to return the
1164+Going back to `++tock`, we now have the duct we need to return the result over.
1219−result over. We do the very sensible thing and put a `%cake` boon in the
1165+We do the very sensible thing and put a `%cake` boon in the queue to be
1220−queue to be processed later by `++clop`.
1166+processed later by `++clop`.
1221
1167
1222−In `q.yoh` we have a list of messages that may need to be sent, which we
1168+In `q.yoh` we have a list of messages that may need to be sent, which we pass to
1223−pass to `++busk` to send, as usual. When an acknowledgment arrives, that
1169+`++busk` to send, as usual. When an acknowledgment arrives, that may trigger
1224−may trigger other messages immediately. This often happens when sending
1170+other messages immediately. This often happens when sending more messages than
1225−more messages than the width of the logical window since for congestion
1171+the width of the logical window since for congestion control reasons another
1226−control reasons another message cannot be sent until some of the earlier
1172+message cannot be sent until some of the earlier ones have been acknowledged.
1227−ones have been acknowledged.
1228
1173
1229−We'll look at the processing of the `%cake` boon in `++clop` before we
1174+We'll look at the processing of the `%cake` boon in `++clop` before we get back
1230−get back to talking about `++bick`.
1175+to talking about `++bick`.
1231
1176
1232 %cake
1177 %cake
1233 :_ fox
1178 :_ fox
1234 :~ [s.bon %give %woot q.p.bon r.bon]
1179 :~ [s.bon %give %woot q.p.bon r.bon]
1235 ==
1180 ==
1236
1181
1237−We very simply give, along the duct we found above, a `%woot` card with
1182+We very simply give, along the duct we found above, a `%woot` card with the ship
1238−the ship who sent us the ack and the ack itself. This allows the
1183+who sent us the ack and the ack itself. This allows the application to decide
1239−application to decide what to do about the result. In case of a failure,
1184+what to do about the result. In case of a failure, we usually either resend the
1240−we usually either resend the message or display it to the user.
1241−Sometimes, we recognize the error term and handle it internally. In any
1185+message or display it to the user. Sometimes, we recognize the error term and
1242−case, the decision of how to handle the acknowledgment is entirely up to
1186+handle it internally. In any case, the decision of how to handle the
1243−the application. Our job is done.
1187+acknowledgment is entirely up to the application. Our job is done.
1244
1188
1245Well, except that we skipped `++bick:pu`. Let's go back to that.
1189Well, except that we skipped `++bick:pu`. Let's go back to that.
1246
1190
@@ -1255,19 +1199,18 @@ Well, except that we skipped `++bick:pu`. Let's go back to that.
1255 =^ yop +>.$ (harv now)
1199 =^ yop +>.$ (harv now)
1256 [[gub yop] +>.$]
1200 [[gub yop] +>.$]
1257
1201
1258−If you recall, in `++whap:pu` we created the packet pump's
1202+If you recall, in `++whap:pu` we created the packet pump's representation of the
1259−representation of the message, which included putting the message into
1203+message, which included putting the message into `diq`, which maps from packet
1260−`diq`, which maps from packet hashes to packet sequence numbers. Thus,
1204+hashes to packet sequence numbers. Thus, `u.sun` is the sequence number of this
1261−`u.sun` is the sequence number of this particular message.
1205+particular message.
1262
1206
1263−We delete this message from `diq` since we have now received an ack for
1207+We delete this message from `diq` since we have now received an ack for it. We
1264−it. We call `++bock` to perform the ack by sequence number. We call
1208+call `++bock` to perform the ack by sequence number. We call `++harv` to harvest
1265−`++harv` to harvest the packet queue, sending any messages that are now
1209+the packet queue, sending any messages that are now able to be sent.
1266−able to be sent.
1267
1210
1268−In `++bock`, there are three arms we haven't seen before: `++bine`,
1211+In `++bock`, there are three arms we haven't seen before: `++bine`, `+wept`, and
1269−`+wept`, and `++beet`. We'll describe each of these before we get to
1212+`++beet`. We'll describe each of these before we get to `++bock`. `++bine` looks
1270−`++bock`. `++bine` looks scariest.
1213+scariest.
1271
1214
1272 ++ bine :: bine:pu
1215 ++ bine :: bine:pu
1273 |= [now=@da num=@ud] :: apply ack
1216 |= [now=@da num=@ud] :: apply ack
@@ -1293,36 +1236,34 @@ In `++bock`, there are three arms we haven't seen before: `++bine`,
1293 :- gub
1236 :- gub
1294 +>.$(puq ~(nap to puq))
1237 +>.$(puq ~(nap to puq))
1295
1238
1296−The first few lines are simply looking through the packet queue until we
1239+The first few lines are simply looking through the packet queue until we find
1297−find the correct packet to ack. This is basic queue manipulation that
1240+the correct packet to ack. This is basic queue manipulation that operates
1298−operates directly on the treap structure of the queue. If you understand
1241+directly on the treap structure of the queue. If you understand treap queues,
1299−treap queues, the logic is easy to follow. Otherwise, just trust us that
1242+the logic is easy to follow. Otherwise, just trust us that by the time we get to
1300−by the time we get to the `=:`, the packet with sequence number `num` is
1243+the `=:`, the packet with sequence number `num` is on the top of the packet
1301−on the top of the packet queue (that is, at `n.puq`).
1244+queue (that is, at `n.puq`).
1302−
1245+
1303−We first update the round-trip time. If the packet is either not alive
1246+We first update the round-trip time. If the packet is either not alive or had to
1304−or had to be transmitted more than once, then we don't have any reliable
1247+be transmitted more than once, then we don't have any reliable way of
1305−way of calculating the round-trip time since we're unsure of exactly
1248+calculating the round-trip time since we're unsure of exactly which transmission
1306−which transmission was acknowledged. Otherwise, the round-trip time is
1249+was acknowledged. Otherwise, the round-trip time is the difference between now
1307−the difference between now and when the packet was last sent. We set
1250+and when the packet was last sent. We set `rtt` by a little weighted average
1308−`rtt` by a little weighted average where the previous smoothed RTT is
1309−weighted twice as much as the RTT of the current packet. Thus,
1251+where the previous smoothed RTT is weighted twice as much as the RTT of the
1310−`(2*rtt+gap)/3`. This gives us a nice smooth RTT that is somewhat
1252+current packet. Thus, `(2*rtt+gap)/3`. This gives us a nice smooth RTT that is
1311−resilient to outlier data while still being responsive to our
1253+somewhat resilient to outlier data while still being responsive to our
1312ever-changing world.
1254ever-changing world.
1313
1255
1314−If the packet wasn't already dead, then we decrement the number of live
1256+If the packet wasn't already dead, then we decrement the number of live packets,
1315−packets, which may allow more packets to be sent.
1257+which may allow more packets to be sent.
1316
1258
1317We decrement the number of unacknowledged packets in our `pyz` for this
1259We decrement the number of unacknowledged packets in our `pyz` for this
1318−particular message. If you recall, this was set in `++whap` to the
1260+particular message. If you recall, this was set in `++whap` to the number of
1319−number of packets required to send a message.
1261+packets required to send a message.
1320
1262
1321−If that was the last packet in the messge that needed to be acked, then
1263+If that was the last packet in the messge that needed to be acked, then we
1322−we delete the messgae reference from `pyz` and produce the id of the
1264+delete the messgae reference from `pyz` and produce the id of the message.
1323−message. Otherwise, we simply update `pyz` with the new number of
1265+Otherwise, we simply update `pyz` with the new number of unacked messages. In
1324−unacked messages. In either case, we remove the packet from the packet
1266+either case, we remove the packet from the packet queue.
1325−queue.
1326
1267
1327 ++ wept :: wept:pu
1268 ++ wept :: wept:pu
1328 |= [fip=@ud lap=@ud] :: fip thru lap-1
1269 |= [fip=@ud lap=@ud] :: fip thru lap-1
@@ -1346,12 +1287,12 @@ queue.
1346 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
1287 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
1347 --
1288 --
1348
1289
1349−The algorithm is a simple case of traversing the packet queue.
1290+The algorithm is a simple case of traversing the packet queue. Essentialy, we
1350−Essentialy, we mark as dead all packets in the queue between `fip` and
1291+mark as dead all packets in the queue between `fip` and `(dec lap)`. We also
1351−`(dec lap)`. We also update `nif`, the number of live packets. Lest you
1292+update `nif`, the number of live packets. Lest you mourn too much the passing of
1352−mourn too much the passing of these packets, know that they shall soon
1293+these packets, know that they shall soon rise again. Recall that in `++bick`
1353−rise again. Recall that in `++bick` after the call to `++bock` we call
1294+after the call to `++bock` we call `++harv`. This will resend the packets that
1354−`++harv`. This will resend the packets that have just been labeled dead.
1295+have just been labeled dead.
1355
1296
1356 ++ beet :: beet:pu
1297 ++ beet :: beet:pu
1357 ^+ . :: advance unacked
1298 ^+ . :: advance unacked
@@ -1363,10 +1304,9 @@ rise again. Recall that in `++bick` after the call to `++bock` we call
1363 =+ rig=$(puq r.puq)
1304 =+ rig=$(puq r.puq)
1364 ?^(rig rig [~ p.n.puq])
1305 ?^(rig rig [~ p.n.puq])
1365
1306
1366−Here we search for the next expected packet number. Basically, we search
1307+Here we search for the next expected packet number. Basically, we search the
1367−the queue for the leftmost packet whose number is greater than the
1308+queue for the leftmost packet whose number is greater than the current `nep`. If
1368−current `nep`. If we don't find any such packet, we just use the total
1309+we don't find any such packet, we just use the total number of packets sent.
1369−number of packets sent.
1370
1310
1371We can now dive into `++bock`, our last arm.
1311We can now dive into `++bock`, our last arm.
1372
1312
@@ -1387,32 +1327,29 @@ We can now dive into `++bock`, our last arm.
1387 :: ~& [%bock-fill num nif caw cag]
1327 :: ~& [%bock-fill num nif caw cag]
1388 +>.$
1328 +>.$
1389
1329
1390−First, we call `++bine` to apply the ack to the packet pump information.
1330+First, we call `++bine` to apply the ack to the packet pump information. We
1391−We produce `gym`, which, if it exists, is the id of the packet that was
1331+produce `gym`, which, if it exists, is the id of the packet that was acked. If
1392−acked. If we received an ack for a packet later than the one we
1332+we received an ack for a packet later than the one we expected, then we halve
1393−expected, then we halve the logical packet window and kill all the
1333+the logical packet window and kill all the earlier packets so that they may be
1394−earlier packets so that they may be resent.
1334+resent.
1395
1335
1396−Otherwise, we possibly increase the congestion window. If the window is
1336+Otherwise, we possibly increase the congestion window. If the window is less
1397−less than the congestion threshold, then we increment the size of the
1337+than the congestion threshold, then we increment the size of the window.
1398−window. Otherwise, we only increment one out of every `caw` times.
1338+Otherwise, we only increment one out of every `caw` times.
1399
1339
1400−If we received an ack for the packet we expected, then we simply advance
1340+If we received an ack for the packet we expected, then we simply advance `nep`
1401−`nep` with `++beet`. If we received an ack for a packet earlier than we
1341+with `++beet`. If we received an ack for a packet earlier than we expected, we
1402−expected, we do nothing.
1342+do nothing.
1403
1343
1404−It may be hard to believe, but we are, in fact, done. The message has
1344+It may be hard to believe, but we are, in fact, done. The message has been sent,
1405−been sent, received, acknowledged, and the acknowledgment has been
1345+received, acknowledged, and the acknowledgment has been returned to the original
1406−returned to the original sender. We hope it's clear that, while the
1346+sender. We hope it's clear that, while the process has been somewhat involved,
1407−process has been somewhat involved, the algorithms are not all that
1408−complicated. If you've read this far, you know `%ames`. The only other
1347+the algorithms are not all that complicated. If you've read this far, you know
1409−code involves initialization, timeouts, and the like.
1348+`%ames`. The only other code involves initialization, timeouts, and the like.
1410
1349
1411−Below, we give detailed reference documentation for the data models
1350+Below, we give detailed reference documentation for the data models involved.
1412−involved.
1413
1351
1414−Data Models
1352+## Data Models
1415−-----------
1416
1353
1417### `++fort`, formal state
1354### `++fort`, formal state
1418
1355
@@ -1424,28 +1361,27 @@ Data Models
1424 zac=(map ship corn) :: flows by server
1361 zac=(map ship corn) :: flows by server
1425 == ::
1362 == ::
1426
1363
1427−This is the state of our vane. Anything that must be remembered between
1364+This is the state of our vane. Anything that must be remembered between calls to
1428−calls to ames must be stored in this state.
1365+ames must be stored in this state.
1429
1366
1430−`%0` is the version of the ames state model itself. If the data model
1367+`%0` is the version of the ames state model itself. If the data model `++fort`
1431−`++fort` changes, then this number needs to be incremented, and an
1368+changes, then this number needs to be incremented, and an adapter must be
1432−adapter must be written to upgrade the old state into the new state.
1369+written to upgrade the old state into the new state. Note that this is the
1433−Note that this is the version number of the model itself, not the
1370+version number of the model itself, not the contents. When the data changes,
1434−contents. When the data changes, there is of course no need to change
1371+there is of course no need to change this.
1435−this.
1436
1372
1437`gad` is a `duct` over which we send `%send` cards to unix. This card is
1373`gad` is a `duct` over which we send `%send` cards to unix. This card is
1438−initialized when unix sends a `%barn` card as vere starts up. Vere
1374+initialized when unix sends a `%barn` card as vere starts up. Vere treats this
1439−treats this duct specially -- don't send anything weird over it.
1375+duct specially -- don't send anything weird over it.
1440
1376
1441−`hop` is the network boot date. This is set when the `%kick` card is
1377+`hop` is the network boot date. This is set when the `%kick` card is sent by
1442−sent by vere on start up.
1378+vere on start up.
1443
1379
1444−`ton` is a `++town`, where we store all of our security/encryption
1380+`ton` is a `++town`, where we store all of our security/encryption state. Note
1445−state. Note that this is shared across all ships on a pier.
1381+that this is shared across all ships on a pier.
1446
1382
1447−`zac` is a map of ships to `++corn`. This stores all the per-ship state.
1383+`zac` is a map of ships to `++corn`. This stores all the per-ship state. The
1448−The keys to this map are the ships on the current pier.
1384+keys to this map are the ships on the current pier.
1449
1385
1450### `++town`, all security state
1386### `++town`, all security state
1451
1387
@@ -1460,20 +1396,17 @@ This is the security state of our pier.
1460
1396
1461`lit` is unused.
1397`lit` is unused.
1462
1398
1463−`any` is 256 bits of entropy. This entropy is used and updated in
1399+`any` is 256 bits of entropy. This entropy is used and updated in exactly two
1464−exactly two places: when we send a `%junk` card, and when we generate a
1400+places: when we send a `%junk` card, and when we generate a new symmetric key in
1465−new symmetric key in `++griz:lax:as:go`. When it is updated, it is
1401+`++griz:lax:as:go`. When it is updated, it is updated by a SHA-256 hash of the
1466−updated by a SHA-256 hash of the current time and the old value of the
1402+current time and the old value of the entropy.
1467−entropy.
1468
1403
1469−`urb` is a map of ships to `++sufi`. This is where we store all the
1404+`urb` is a map of ships to `++sufi`. This is where we store all the per-ship
1470−per-ship state for the pier. The keys to this map are the ships on the
1405+state for the pier. The keys to this map are the ships on the current pier.
1471−current pier.
1472
1406
1473−`fak` is true if we are on a fake network. This disables certain
1407+`fak` is true if we are on a fake network. This disables certain security checks
1474−security checks so that anyone may run a fake `~zod`. This is used only
1408+so that anyone may run a fake `~zod`. This is used only for development. To use,
1475−for development. To use, run vere with the `-F` option (and the
1476−`-I ~zod` option for a fake `~zod`).
1409+run vere with the `-F` option (and the `-I ~zod` option for a fake `~zod`).
1477
1410
1478### `++sufi`, domestic host
1411### `++sufi`, domestic host
1479
1412
@@ -1487,9 +1420,8 @@ for development. To use, run vere with the `-F` option (and the
1487
1420
1488This is the security state of a domestic server.
1421This is the security state of a domestic server.
1489
1422
1490−`hoy` is a list of the ships directly above us in the hierarchy of
1423+`hoy` is a list of the ships directly above us in the hierarchy of ships. For
1491−ships. For example, for `~hoclur-bicrel`, this would be `~tasruc` and
1424+example, for `~hoclur-bicrel`, this would be `~tasruc` and `~tug`. See `++sein`.
1492−`~tug`. See `++sein`.
1493
1425
1494`val` is a list of our private keys.
1426`val` is a list of our private keys.
1495
1427
@@ -1497,51 +1429,51 @@ ships. For example, for `~hoclur-bicrel`, this would be `~tasruc` and
1497
1429
1498`seh`
1430`seh`
1499
1431
1500−`hoc` is a map of ships to `++dore`. The stores all the security
1432+`hoc` is a map of ships to `++dore`. The stores all the security informatoin
1501−informatoin about foreign ships. The keys to this map are the neighbors
1433+about foreign ships. The keys to this map are the neighbors (ships we have been
1502−(ships we have been in contact with) of this domestic server.
1434+in contact with) of this domestic server.
1503
1435
1504### `++wund`, private keys
1436### `++wund`, private keys
1505
1437
1506 ++ wund (list ,[p=life q=ring r=acru]) :: mace in action
1438 ++ wund (list ,[p=life q=ring r=acru]) :: mace in action
1507
1439
1508−This is a list of our own private keys, indexed by life. The key itself
1440+This is a list of our own private keys, indexed by life. The key itself is the
1509−is the `++ring`, and the `++acru` is the encryption engine. We generate
1441+`++ring`, and the `++acru` is the encryption engine. We generate the `++acru`
1510−the `++acru` from the private key by calling `++weur`. Thus, we can at
1442+from the private key by calling `++weur`. Thus, we can at any time regenerate
1511−any time regenerate our `++wund` from a `++mace`. The current crypto is
1443+our `++wund` from a `++mace`. The current crypto is at the head of the list and
1512−at the head of the list and can be accessed with `++sen:as:go`.
1444+can be accessed with `++sen:as:go`.
1513
1445
1514### `++ring`, private key
1446### `++ring`, private key
1515
1447
1516 ++ ring ,@ :: private key
1448 ++ ring ,@ :: private key
1517
1449
1518−This is a private key. The first byte is reserved to identify the type
1450+This is a private key. The first byte is reserved to identify the type of
1519−of cryptography. Lower-case means public key, upper-case means public
1451+cryptography. Lower-case means public key, upper-case means public key, and the
1520−key, and the letter identifies which `++acru` to use.
1452+letter identifies which `++acru` to use.
1521
1453
1522### `++pass`, public key
1454### `++pass`, public key
1523
1455
1524 ++ pass ,@ :: public key
1456 ++ pass ,@ :: public key
1525
1457
1526This is a public key. The first byte is reserved to identify the type of
1458This is a public key. The first byte is reserved to identify the type of
1527−cryptography. Lower-case means public key, upper-case means public key,
1459+cryptography. Lower-case means public key, upper-case means public key, and the
1528−and the letter identifies which `++acru` to use.
1460+letter identifies which `++acru` to use.
1529
1461
1530### `++mace`, private secrets
1462### `++mace`, private secrets
1531
1463
1532 ++ mace (list ,[p=life q=ring]) :: private secrets
1464 ++ mace (list ,[p=life q=ring]) :: private secrets
1533
1465
1534−This is a list of the our private keys, indexed by life. From this we
1466+This is a list of the our private keys, indexed by life. From this we can
1535−can generate a `++wund` for actual use.
1467+generate a `++wund` for actual use.
1536
1468
1537### `++skin`, encoding stem
1469### `++skin`, encoding stem
1538
1470
1539 ++ skin ?(%none %open %fast %full) :: encoding stem
1471 ++ skin ?(%none %open %fast %full) :: encoding stem
1540
1472
1541−This defines the type of encryption used for each message. `%none`
1473+This defines the type of encryption used for each message. `%none` refers to
1542−refers to messages sent in the clear, `%open` refers to signed messages,
1474+messages sent in the clear, `%open` refers to signed messages, `%full` refers to
1543−`%full` refers to sealed messages, and `%fast` refers to symmetrically
1475+sealed messages, and `%fast` refers to symmetrically encrypted messages. See
1544−encrypted messages. See `++acru` for details.
1476+`++acru` for details.
1545
1477
1546### `++acru`, asymmetric cryptosuite
1478### `++acru`, asymmetric cryptosuite
1547
1479
@@ -1570,11 +1502,11 @@ encrypted messages. See `++acru` for details.
1570 --
1502 --
1571 --
1503 --
1572
1504
1573−This is an opaque interface for a general asymmetric cryptosuite. Any
1505+This is an opaque interface for a general asymmetric cryptosuite. Any form of
1574−form of asymmetric cryptography can be dropped in to be used instead of
1506+asymmetric cryptography can be dropped in to be used instead of the default.
1575−the default. Right now, there are two cryptosuites, `++crua`, which is
1507+Right now, there are two cryptosuites, `++crua`, which is your standard RSA, and
1576−your standard RSA, and `++crub`, which is elliptic curve crypto but is
1508+`++crub`, which is elliptic curve crypto but is mostly stubbed out at the
1577−mostly stubbed out at the moment.
1509+moment.
1578
1510
1579#### `++as:acru`, asymmetric operations
1511#### `++as:acru`, asymmetric operations
1580
1512
@@ -1586,31 +1518,28 @@ mostly stubbed out at the moment.
1586 *(unit ,[p=@ q=@]) ::
1518 *(unit ,[p=@ q=@]) ::
1587 -- ::
1519 -- ::
1588
1520
1589−This is the core that defines the standard asymmetric cryptography
1521+This is the core that defines the standard asymmetric cryptography operations.
1590−operations.
1522+
1591−
1592−`++seal:as:acru` allows us to send a message encrypted with someone's
1523+`++seal:as:acru` allows us to send a message encrypted with someone's public key
1593−public key so that only they may read it. If Alice seals a message with
1524+so that only they may read it. If Alice seals a message with Bob's public key,
1594−Bob's public key, then she can be sure that Bob is the only one who can
1525+then she can be sure that Bob is the only one who can read it. This is
1595−read it. This is associated with the `++skin` `%full`.
1596−
1597−`++sign:as:acru` allows us to sign a message with our private key so
1598−that others can verify that we sent the message. If Alice signs a
1599−message with her private key, then Bob can verify with her public key
1600−that it was indeed Alice who sent it. This is associated with the
1601−`++skin` `%open`.
1602−
1603−`++sure:as:acru` is the dual to `++sign:as:acru`. It allows us to verify
1604−that a message we have received is indeed from the claimed sender. If
1605−Alice sends a message with her private key, then Bob can use this arm to
1606−verify that it was indeed Alice who sent it. This is associated with the
1607−`++skin` `%open`.
1608−
1609−`++tear:as:acru` is the dual to `++seal:as:acru`. It allows us to read a
1610−message that we can be sure is only read by us. If Alice seals a message
1611−with Bob's public key, then Bob can use this arm to read it. This is
1612associated with the `++skin` `%full`.
1526associated with the `++skin` `%full`.
1613
1527
1528+`++sign:as:acru` allows us to sign a message with our private key so that others
1529+can verify that we sent the message. If Alice signs a message with her private
1530+key, then Bob can verify with her public key that it was indeed Alice who sent
1531+it. This is associated with the `++skin` `%open`.
1532+
1533+`++sure:as:acru` is the dual to `++sign:as:acru`. It allows us to verify that a
1534+message we have received is indeed from the claimed sender. If Alice sends a
1535+message with her private key, then Bob can use this arm to verify that it was
1536+indeed Alice who sent it. This is associated with the `++skin` `%open`.
1537+
1538+`++tear:as:acru` is the dual to `++seal:as:acru`. It allows us to read a message
1539+that we can be sure is only read by us. If Alice seals a message with Bob's
1540+public key, then Bob can use this arm to read it. This is associated with the
1541+`++skin` `%full`.
1542+
1614#### `++de:acru`, `++dy:acru`, and `++en:acru`, symmetric encryption/decryption
1543#### `++de:acru`, `++dy:acru`, and `++en:acru`, symmetric encryption/decryption
1615
1544
1616 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1545 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
@@ -1619,12 +1548,12 @@ associated with the `++skin` `%full`.
1619
1548
1620Symmetric encryption is associated with the `++skin` `%fast`.
1549Symmetric encryption is associated with the `++skin` `%fast`.
1621
1550
1622−`++de:acru` decrypts a message with a symmetric key, returning `~` on
1551+`++de:acru` decrypts a message with a symmetric key, returning `~` on failure
1623−failure and `[~ u=data]` on success.
1552+and `[~ u=data]` on success.
1624
1553
1625−`++dy:acru` decrypts a message with a symmetric key, crashing on
1554+`++dy:acru` decrypts a message with a symmetric key, crashing on failure. This
1626−failure. This should almost always be defined as, and should always be
1555+should almost always be defined as, and should always be semantically equivalent
1627−semantically equivalent to, `(need (de a b))`.
1556+to, `(need (de a b))`.
1628
1557
1629`++en:acru` encrypts a message with a symmetric key.
1558`++en:acru` encrypts a message with a symmetric key.
1630
1559
@@ -1637,10 +1566,9 @@ semantically equivalent to, `(need (de a b))`.
1637 ++ sec *ring :: private key
1566 ++ sec *ring :: private key
1638 --
1567 --
1639
1568
1640−`++fig:ex:acru` is our fingerprint, usually a hash of our public key.
1569+`++fig:ex:acru` is our fingerprint, usually a hash of our public key. This is
1641−This is used, for example, in `++zeno`, where every carrier owner's
1570+used, for example, in `++zeno`, where every carrier owner's fingerprint is
1642−fingerprint is stored so that we can ensure that carriers are indeed
1571+stored so that we can ensure that carriers are indeed owned by their owners
1643−owned by their owners
1644
1572
1645`++pac:ex:acru` is our default passcode, which is unused at present.
1573`++pac:ex:acru` is our default passcode, which is unused at present.
1646
1574
@@ -1658,51 +1586,45 @@ owned by their owners
1658
1586
1659These arms allow us to reconstruct a `++acru` from basic data.
1587These arms allow us to reconstruct a `++acru` from basic data.
1660
1588
1661−`++pit:nu:acru` constructs a `++acru` from the width of our intended key
1589+`++pit:nu:acru` constructs a `++acru` from the width of our intended key and
1662−and seed entropy. This is usually used in the initial construction of
1590+seed entropy. This is usually used in the initial construction of the `++acru`.
1663−the `++acru`.
1664
1591
1665−`++nol:nu:acru` constructs a `++acru` from a "naked ring", meaning a
1592+`++nol:nu:acru` constructs a `++acru` from a "naked ring", meaning a `++ring`
1666−`++ring` without the initial byte identifying the type of crypto. There
1593+without the initial byte identifying the type of crypto. There is often a helper
1667−is often a helper arm that that wraps this; see `++weur` for `++crua`
1594+arm that that wraps this; see `++weur` for `++crua` and `++wear` for `++crub`.
1668−and `++wear` for `++crub`.
1669
1595
1670−`++com:nu:acru` constructs a `++acru` from a "naked pass", meaning a
1596+`++com:nu:acru` constructs a `++acru` from a "naked pass", meaning a `++ring`
1671−`++ring` without the initial byte identifying the type of crypto. There
1597+without the initial byte identifying the type of crypto. There is often a helper
1672−is often a helper arm that that wraps this; see `++haul` for `++crua`
1598+arm that that wraps this; see `++haul` for `++crua` and `++hail` for `++crub`.
1673−and `++hail` for `++crub`.
1674
1599
1675### `++will`, certificate
1600### `++will`, certificate
1676
1601
1677 ++ will (list deed) :: certificate
1602 ++ will (list deed) :: certificate
1678
1603
1679−This is a list of deeds associated with the current ship. There should
1604+This is a list of deeds associated with the current ship. There should be an
1680−be an item in this list for every ship from this point up in the
1605+item in this list for every ship from this point up in the hierarchy times the
1681−hierarchy times the number of lives that each ship has had. For example,
1606+number of lives that each ship has had. For example, ~hoclur-bicrel may have a
1682−\~hoclur-bicrel may have a will with three items: one for itself, one
1607+will with three items: one for itself, one for ~tasruc (who issued
1683−for \~tasruc (who issued \~hoclur-bicrel's deed) and one for \~tug (who
1608+~hoclur-bicrel's deed) and one for ~tug (who issued ~tasruc's deed).
1684−issued \~tasruc's deed).
1685
1609
1686### `++deed`, identity
1610### `++deed`, identity
1687
1611
1688 ++ deed ,[p=@ q=step r=?] :: sig, stage, fake?
1612 ++ deed ,[p=@ q=step r=?] :: sig, stage, fake?
1689
1613
1690−`p` is the signature of a particular deed, which is a signed copy of
1614+`p` is the signature of a particular deed, which is a signed copy of `q`.
1691−`q`.
1692
1615
1693`q` is the stage in the identity.
1616`q` is the stage in the identity.
1694
1617
1695−`r` is true if we're working on a fake network, where we don't check
1618+`r` is true if we're working on a fake network, where we don't check that the
1696−that the carrier fingerprints are correct. This allows us to create fake
1619+carrier fingerprints are correct. This allows us to create fake networks for
1697−networks for development without interfering with the real network.
1620+development without interfering with the real network.
1698
1621
1699### `++step`, identity stage
1622### `++step`, identity stage
1700
1623
1701 ++ step ,[p=bray q=gens r=pass] :: identity stage
1624 ++ step ,[p=bray q=gens r=pass] :: identity stage
1702
1625
1703−This is a single stage in our identity. Thus, this is specific to a
1626+This is a single stage in our identity. Thus, this is specific to a single life
1704−single life in a single ship. Everything in here may change between
1627+in a single ship. Everything in here may change between lives.
1705−lives.
1706
1628
1707`p`
1629`p`
1708
1630
@@ -1720,9 +1642,9 @@ XXX
1720
1642
1721 ++ gens ,[p=lang q=gcos] :: general identity
1643 ++ gens ,[p=lang q=gcos] :: general identity
1722
1644
1723−`p` is the IETF language code for the preferred language of this
1645+`p` is the IETF language code for the preferred language of this identity. This
1724−identity. This is unused at the moment, but in the future text should be
1646+is unused at the moment, but in the future text should be localized based on
1725−localized based on this.
1647+this.
1726
1648
1727`q` is the description of the ship.
1649`q` is the description of the ship.
1728
1650
@@ -1736,32 +1658,31 @@ localized based on this.
1736 [%pawn p=(unit ,@t)] :: 128-bit ship
1658 [%pawn p=(unit ,@t)] :: 128-bit ship
1737 == ::
1659 == ::
1738
1660
1739−This is the description of the identity of a ship. Most types of
1661+This is the description of the identity of a ship. Most types of identity have a
1740−identity have a `@t` field, which is their human-readable name. The
1662+`@t` field, which is their human-readable name. The identity of a `%duke` is
1741−identity of a `%duke` is more involved.
1663+more involved.
1742
1664
1743−A `%czar`, a carrier, is a ship with an 8-bit address. Thus, there are
1665+A `%czar`, a carrier, is a ship with an 8-bit address. Thus, there are only 256
1744−only 256 carriers. These are at the top of the namespace hierarchy, and
1666+carriers. These are at the top of the namespace hierarchy, and the fingerprint
1745−the fingerprint of each carrier is stored in `++zeno`. These are the
1667+of each carrier is stored in `++zeno`. These are the "senators" of Urbit.
1746−"senators" of Urbit.
1747
1668
1748−A `%king`, a cruiser, is a ship with a 16-bit address. Thus, there are
1669+A `%king`, a cruiser, is a ship with a 16-bit address. Thus, there are 65,536
1749−65,536 cruisers. Each carrier may issue 256 cruisers. These are the
1670+cruisers. Each carrier may issue 256 cruisers. These are the infrastructure of
1750−infrastructure of Urbit.
1671+Urbit.
1751
1672
1752A `%duke`, a destroyer, is a ship with a 32-bit address. Thus, there are
1673A `%duke`, a destroyer, is a ship with a 32-bit address. Thus, there are
1753−4,294,967,296 destroyers. Each cruiser may issue 65,536 cruisers. These
1674+4,294,967,296 destroyers. Each cruiser may issue 65,536 cruisers. These are the
1754−are the individuals of Urbit.
1675+individuals of Urbit.
1755
1676
1756A `%earl`, a yacht, is a ship with a 64-bit address. Thus, there are
1677A `%earl`, a yacht, is a ship with a 64-bit address. Thus, there are
1757−18,446,744,073,709,551,616 yachts. Each destroyer may issue
1678+18,446,744,073,709,551,616 yachts. Each destroyer may issue 4,294,967,296
1758−4,294,967,296 yachts. These are the devices of Urbit.
1679+yachts. These are the devices of Urbit.
1759
1680
1760−A `%pawn`, a submarine, is a ship with a 128-bit address. Thus, there
1681+A `%pawn`, a submarine, is a ship with a 128-bit address. Thus, there are a lot
1761−are a lot of submarines. The chance of random name collision is
1682+of submarines. The chance of random name collision is negligible, so submarines
1762−negligible, so submarines are not issued by any ship. They must simply
1683+are not issued by any ship. They must simply assert their presence, and they are
1763−assert their presence, and they are all considered children of \~zod.
1684+all considered children of ~zod. This is the underworld of Urbit, where
1764−This is the underworld of Urbit, where anonymity reigns supreme.
1685+anonymity reigns supreme.
1765
1686
1766### `++what`, logical destroyer identity
1687### `++what`, logical destroyer identity
1767
1688
@@ -1774,10 +1695,10 @@ This is the underworld of Urbit, where anonymity reigns supreme.
1774
1695
1775This is the logical identity of a destroyer.
1696This is the logical identity of a destroyer.
1776
1697
1777−A `%anon` is a completely anonymous destroyer. The difference between
1698+A `%anon` is a completely anonymous destroyer. The difference between this and a
1778−this and a submarine is that a submarine is ephemeral while a `%anon`
1699+submarine is that a submarine is ephemeral while a `%anon` destroyer is not.
1779−destroyer is not. Thus, we may not know who \~hoclur-bicrel is, but we
1700+Thus, we may not know who ~hoclur-bicrel is, but we do know that it's always the
1780−do know that it's always the same person.
1701+same person.
1781
1702
1782A `%lady` is a female person. The name used here should be a real name.
1703A `%lady` is a female person. The name used here should be a real name.
1783
1704
@@ -1789,8 +1710,8 @@ A `%punk` is a person who is identified only by a handle.
1789
1710
1790 ++ whom ,[p=@ud q=govt r=sect s=name] :: year/govt/id
1711 ++ whom ,[p=@ud q=govt r=sect s=name] :: year/govt/id
1791
1712
1792−Ths is the information associated with a real person. It is mostly
1713+Ths is the information associated with a real person. It is mostly information
1793−information that could be observed with the briefest of interactions.
1714+that could be observed with the briefest of interactions.
1794
1715
1795`p` is the birth year.
1716`p` is the birth year.
1796
1717
@@ -1816,5 +1737,5 @@ XXX
1816
1737
1817 ++ name ,[p=@t q=(unit ,@t) r=(unit ,@t) s=@t] :: first mid/nick last
1738 ++ name ,[p=@t q=(unit ,@t) r=(unit ,@t) s=@t] :: first mid/nick last
1818
1739
1819−This is the given name, possible middle name/initial, possible nickname,
1740+This is the given name, possible middle name/initial, possible nickname, and
1820−and surname of a user.
1741+surname of a user.
docs/arvo/internals/behn.md+12−14
@@ -1,22 +1,20 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 2
3sort: 2
4---
4---
5
5
6−`%behn`
6+# `%behn`
7−=======
8
7
9Our simple timer.
8Our simple timer.
10
9
11−It allows vanes and applications to set and timer events, which are
10+It allows vanes and applications to set and timer events, which are managed in a
12−managed in a simple priority queue. `%behn` produces effects to start
11+simple priority queue. `%behn` produces effects to start the unix timer, and
13−the unix timer, and when the requested `%behn` passes, unix sends wake
12+when the requested `%behn` passes, unix sends wake events to `%behn`, which time
14−events to `%behn`, which time routes back to original sender. We don't
13+routes back to original sender. We don't guarantee that a timer event will
15−guarantee that a timer event will happen at exactly the `%behn` it was
16−set for, or even that it'll be particularly close. A timer event is a
14+happen at exactly the `%behn` it was set for, or even that it'll be particularly
17−request to not be woken until after the given time.
15+close. A timer event is a request to not be woken until after the given time.
18
16
19−`%eyre` uses `%behn` for timing out sessions, and `%clay` uses `%behn`
17+`%eyre` uses `%behn` for timing out sessions, and `%clay` uses `%behn` for
20−for keeping track of time-specified file requests. `%ames` should
18+keeping track of time-specified file requests. `%ames` should probably use
21−probably use `%behn` to keep track of things like network timeouts and
19+`%behn` to keep track of things like network timeouts and retry timing, but it
22−retry timing, but it currently uses its own alarm system.
20+currently uses its own alarm system.
docs/arvo/internals/clay.md+13−16
@@ -1,26 +1,23 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 3
3sort: 3
4---
4---
5
5
6−`%clay`
6+# `%clay`
7−=======
8
7
9Our filesystem.
8Our filesystem.
10
9
11−`%clay` is version-controlled, referentially-transparent, and global.
10+`%clay` is version-controlled, referentially-transparent, and global. While this
12−While this filesystem is stored in `%clay`, it is mirrored to Unix for
11+filesystem is stored in `%clay`, it is mirrored to Unix for convenience. Unix
13−convenience. Unix tells `%clay`s whenever a file changes in the Unix
12+tells `%clay`s whenever a file changes in the Unix copy of the filesystem so
14−copy of the filesystem so that the change may be applied. `%clay` tells
13+that the change may be applied. `%clay` tells unix whenever an app or vane
15−unix whenever an app or vane changes the filesystem so that the change
14+changes the filesystem so that the change can be effected in Unix. Apps and
16−can be effected in Unix. Apps and vanes may use `%clay` to write to the
17−filesystem, query it, and subscribe to changes in it. Ford and gall use
15+vanes may use `%clay` to write to the filesystem, query it, and subscribe to
18−`%clay` to serve up apps and web pages.
16+changes in it. Ford and gall use `%clay` to serve up apps and web pages.
19
17
20−`%clay` includes three components. First is the filesystem/version
18+`%clay` includes three components. First is the filesystem/version control
21−control algorithms, which are mostly defined in `++ze` and `++zu` in
19+algorithms, which are mostly defined in `++ze` and `++zu` in zuse. Second is the
22−zuse. Second is the write, query, and subscription logic. Finally, there
20+write, query, and subscription logic. Finally, there is the logic for
23−is the logic for communicating requests to, and receiving requests from,
21+communicating requests to, and receiving requests from, foreign ships.
24−foreign ships.
25
22
26<list></list>
23<list></list>
docs/arvo/internals/clay/architecture.md+295−349
@@ -1,201 +1,176 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3---
3---
4
4
5−
6# clay
5# clay
7
6
8## high-level
7## high-level
9
8
10−clay is the primary filesystem for the arvo operating system,
9+clay is the primary filesystem for the arvo operating system, which is the core
11−which is the core of an urbit. The architecture of clay is
10+of an urbit. The architecture of clay is intrinsically connected with arvo, but
12−intrinsically connected with arvo, but we assume no knowledge of
13−either arvo or urbit. We will point out only those features of
11+we assume no knowledge of either arvo or urbit. We will point out only those
14−arvo that are necessary for an understanding of clay, and we will
12+features of arvo that are necessary for an understanding of clay, and we will do
15−do so only when they arise.
13+so only when they arise.
16−
14+
17−The first relevant feature of arvo is that it is a deterministic
15+The first relevant feature of arvo is that it is a deterministic system where
18−system where input and output are defined as a series of events
16+input and output are defined as a series of events and effects. The state of
19−and effects. The state of arvo is simply a function of its event
17+arvo is simply a function of its event log. None of the effects from an event
20−log. None of the effects from an event are emitted until the
21−event is entered in the log and persisted, either to disk or
18+are emitted until the event is entered in the log and persisted, either to disk
22−another trusted source of persistence, such as a Kafka cluster.
19+or another trusted source of persistence, such as a Kafka cluster. Consequently,
23−Consequently, arvo is a single-level store: everything in its
20+arvo is a single-level store: everything in its state is persistent.
24−state is persistent.
21+
25−
26−In a more traditional OS, everything in RAM can be erased at any
22+In a more traditional OS, everything in RAM can be erased at any time by power
27−time by power failure, and is always erased on reboot. Thus, a
23+failure, and is always erased on reboot. Thus, a primary purpose of a filesystem
28−primary purpose of a filesystem is to ensure files persist across
29−power failures and reboots. In arvo, both power failures and
24+is to ensure files persist across power failures and reboots. In arvo, both
30−reboots are special cases of suspending computation, which is
25+power failures and reboots are special cases of suspending computation, which is
31−done safely since our event log is already persistent. Therefore,
26+done safely since our event log is already persistent. Therefore, clay is not
32−clay is not needed in arvo for persistence. Why, then, do we have a
27+needed in arvo for persistence. Why, then, do we have a filesystem? There are
33−filesystem? There are two answers to this question.
28+two answers to this question.
34−
29+
35−First, clay provides a filesystem tree, which is a convenient
30+First, clay provides a filesystem tree, which is a convenient user interface for
36−user interface for some applications. Unix has the useful concept
31+some applications. Unix has the useful concept of virtual filesystems, which are
37−of virtual filesystems, which are used for everything from direct
38−access to devices, to random number generators, to the /proc
32+used for everything from direct access to devices, to random number generators,
39−tree. It is easy and intuitive to read from and write to a
33+to the /proc tree. It is easy and intuitive to read from and write to a
40filesystem tree.
34filesystem tree.
41
35
42−Second, clay has a distributed revision control system baked into
36+Second, clay has a distributed revision control system baked into it.
43−it. Traditional filesystems are not revision controlled, so
37+Traditional filesystems are not revision controlled, so userspace software --
44−userspace software -- such as git -- is written on top of them to
38+such as git -- is written on top of them to do so. clay natively provides the
45−do so. clay natively provides the same functionality as modern
39+same functionality as modern DVCSes, and more.
46−DVCSes, and more.
47
40
48−clay has two other unique properties that we'll cover later on:
41+clay has two other unique properties that we'll cover later on: it supports
49−it supports typed data and is referentially transparent.
42+typed data and is referentially transparent.
50
43
51### Revision Control
44### Revision Control
52
45
53−Every urbit has one or more "desks", which are independently
46+Every urbit has one or more "desks", which are independently revision-controlled
54−revision-controlled branches. Each desk contains its own mark
55−definitions, apps, doc, and so forth.
47+branches. Each desk contains its own mark definitions, apps, doc, and so forth.
56−
48+
57−Traditionally, an urbit has at least a base and a home desk. The
49+Traditionally, an urbit has at least a base and a home desk. The base desk has
58−base desk has all the system software from the distribution. the
50+all the system software from the distribution. the home desk is a fork of base
59−home desk is a fork of base with all the stuff specific to the
51+with all the stuff specific to the user of the urbit.
60−user of the urbit.
52+
61−
62−A desk is a series of numbered commits, the most recent of which
53+A desk is a series of numbered commits, the most recent of which represents the
63−represents the current state of the desk. A commit is composed of
54+current state of the desk. A commit is composed of (1) an absolute time when it
64−(1) an absolute time when it was created, (2) a list of zero or
65−more parents, and (3) a map from paths to data.
55+was created, (2) a list of zero or more parents, and (3) a map from paths to
66−
56+data.
57+
67−Most commits have exactly one parent, but the initial commit on a
58+Most commits have exactly one parent, but the initial commit on a desk may have
68−desk may have zero parents, and merge commits have more than one
59+zero parents, and merge commits have more than one parent.
69−parent.
60+
70−
71−The non-meta data is stored in the map of paths to data. It's
61+The non-meta data is stored in the map of paths to data. It's worth noting that
72−worth noting that no constraints are put on this map, so, for
62+no constraints are put on this map, so, for example, both /a/b and /a/b/c could
73−example, both /a/b and /a/b/c could have data. This is impossible
74−in a traditional Unix filesystem since it means that /a/b is both
63+have data. This is impossible in a traditional Unix filesystem since it means
75−a file and a directory. Conventionally, the final element in the
64+that /a/b is both a file and a directory. Conventionally, the final element in
76−path is its mark -- much like a filename extension in Unix. Thus,
65+the path is its mark -- much like a filename extension in Unix. Thus,
77/doc/readme.md in Unix is stored as /doc/readme/md in urbit.
66/doc/readme.md in Unix is stored as /doc/readme/md in urbit.
78
67
79−The data is not stored directly in the map; rather, a hash of the
68+The data is not stored directly in the map; rather, a hash of the data is
80−data is stored, and we maintain a master blob store. Thus, if the
69+stored, and we maintain a master blob store. Thus, if the same data is referred
81−same data is referred to in multiple commits (as, for example,
70+to in multiple commits (as, for example, when a file doesn't change between
82−when a file doesn't change between commits), only the hash is
71+commits), only the hash is duplicated.
83−duplicated.
72+
84−
85−In the master blob store, we either store the data directly, or
73+In the master blob store, we either store the data directly, or else we store a
86−else we store a diff against another blob. The hash is dependent
74+diff against another blob. The hash is dependent only on the data within and not
87−only on the data within and not on whether or not it's stored
88−directly, so we may on occasion rearrange the contents of the
75+on whether or not it's stored directly, so we may on occasion rearrange the
89−blob store for performance reasons.
76+contents of the blob store for performance reasons.
90−
77+
91−Recall that a desk is a series of numbered commits. Not every
78+Recall that a desk is a series of numbered commits. Not every commit in a desk
92−commit in a desk must be numbered. For example, if the base desk
79+must be numbered. For example, if the base desk has had 50 commits since home
93−has had 50 commits since home was forked from it, then a merge
94−from base to home will only add a single revision number to home,
80+was forked from it, then a merge from base to home will only add a single
95−although the full commit history will be accessible by traversing
81+revision number to home, although the full commit history will be accessible by
96−the parentage of the individual commits.
82+traversing the parentage of the individual commits.
97−
83+
98−We do guarantee that the first commit is numbered 1, commits are
84+We do guarantee that the first commit is numbered 1, commits are numbered
99−numbered consecutively after that (i.e. there are no "holes"),
85+consecutively after that (i.e. there are no "holes"), the topmost commit is
100−the topmost commit is always numbered, and every numbered commit
86+always numbered, and every numbered commit is an ancestor of every later
101−is an ancestor of every later numbered commit.
87+numbered commit.
102−
88+
103−There are three ways to refer to particular commits in the
89+There are three ways to refer to particular commits in the revision history.
104−revision history. Firstly, one can use the revision number.
90+Firstly, one can use the revision number. Secondly, one can use any absolute
105−Secondly, one can use any absolute time between the one numbered
106−commit and the next (inclusive of the first, exclusive of the
91+time between the one numbered commit and the next (inclusive of the first,
107−second). Thirdly, every desk has a map of labels to revision
92+exclusive of the second). Thirdly, every desk has a map of labels to revision
108numbers. These labels may be used to refer to specific commits.
93numbers. These labels may be used to refer to specific commits.
109
94
110−Additionally, clay is a global filesystem, so data on other urbit
95+Additionally, clay is a global filesystem, so data on other urbit is easily
111−is easily accessible the same way as data on our local urbit. In
96+accessible the same way as data on our local urbit. In general, the path to a
112−general, the path to a particular revision of a desk is
113−/~urbit-name/desk-name/revision. Thus, to get /try/readme/md
97+particular revision of a desk is /~urbit-name/desk-name/revision. Thus, to get
114−from revision 5 of the home desk on ~sampel-sipnym, we refer to
98+/try/readme/md from revision 5 of the home desk on ~sampel-sipnym, we refer to
115−/~sampel-sipnym/home/5/try/readme/md. Clay's namespace is thus
99+/~sampel-sipnym/home/5/try/readme/md. Clay's namespace is thus global and
116−global and referentially transparent.
100+referentially transparent.
117
101
118XXX reactivity here?
102XXX reactivity here?
119
103
120### A Typed Filesystem
104### A Typed Filesystem
121
105
122−Since clay is a general filesystem for storing data of arbitrary
106+Since clay is a general filesystem for storing data of arbitrary types, in order
123−types, in order to revision control correctly it needs to be
107+to revision control correctly it needs to be aware of types all the way through.
124−aware of types all the way through. Traditional revision control
125−does an excellent job of handling source code, so for source code
108+Traditional revision control does an excellent job of handling source code, so
126−we act very similar to traditional revision control. The
109+for source code we act very similar to traditional revision control. The
127challenge is to handle other data similarly well.
110challenge is to handle other data similarly well.
128
111
129−For example, modern VCSs generally support "binary files", which
112+For example, modern VCSs generally support "binary files", which are files for
130−are files for which the standard textual diffing, patching, and
113+which the standard textual diffing, patching, and merging algorithms are not
131−merging algorithms are not helpful. A "diff" of two binary files
132−is just a pair of the files, "patching" this diff is just
114+helpful. A "diff" of two binary files is just a pair of the files, "patching"
133−replacing the old file with the new one, and "merging"
115+this diff is just replacing the old file with the new one, and "merging"
134−non-identical diffs is always a conflict, which can't even be
116+non-identical diffs is always a conflict, which can't even be helpfully
135−helpfully annotated. Without knowing anything about the structure
117+annotated. Without knowing anything about the structure of a blob of data, this
136−of a blob of data, this is the best we can do.
118+is the best we can do.
137−
119+
138−Often, though, "binary" files have some internal structure, and
120+Often, though, "binary" files have some internal structure, and it is possible
139−it is possible to create diff, patch, and merge algorithms that
121+to create diff, patch, and merge algorithms that take advantage of this
140−take advantage of this structure. An image may be the result of a
122+structure. An image may be the result of a base image with some set of
141−base image with some set of operations applied. With algorithms
142−aware of this set of operations, not only can revision control
123+operations applied. With algorithms aware of this set of operations, not only
143−software save space by not having to save every revision of the
124+can revision control software save space by not having to save every revision of
144−image individually, these transformations can be made on parallel
125+the image individually, these transformations can be made on parallel branches
145−branches and merged at will.
126+and merged at will.
146−
127+
147−Suppose Alice is tasked with touching up a picture, improving the
128+Suppose Alice is tasked with touching up a picture, improving the color balance,
148−color balance, adjusting the contrast, and so forth, while Bob
129+adjusting the contrast, and so forth, while Bob has the job of cropping the
149−has the job of cropping the picture to fit where it's needed and
150−adding textual overlay. Without type-aware revision control,
130+picture to fit where it's needed and adding textual overlay. Without type-aware
151−these changes must be made serially, requiring Alice and Bob to
131+revision control, these changes must be made serially, requiring Alice and Bob
152−explicitly coordinate their efforts. With type-aware revision
132+to explicitly coordinate their efforts. With type-aware revision control, these
153−control, these operations may be performed in parallel, and then
133+operations may be performed in parallel, and then the two changesets can be
154−the two changesets can be merged programmatically.
134+merged programmatically.
155−
135+
156−Of course, even some kinds of text files may be better served by
136+Of course, even some kinds of text files may be better served by diff, patch,
157−diff, patch, and merge algorithms aware of the structure of the
137+and merge algorithms aware of the structure of the files. Consider a file
158−files. Consider a file containing a pretty-printed JSON object.
138+containing a pretty-printed JSON object. Small changes in the JSON object may
159−Small changes in the JSON object may result in rather significant
160−changes in how the object is pretty-printed (for example, by
139+result in rather significant changes in how the object is pretty-printed (for
161−addding an indentation level, splitting a single line into
140+example, by addding an indentation level, splitting a single line into multiple
162−multiple lines).
141+lines).
163−
142+
164−A text file wrapped at 80 columns also reacts suboptimally with
143+A text file wrapped at 80 columns also reacts suboptimally with unadorned
165−unadorned Hunt-McIlroy diffs. A single word inserted in a
144+Hunt-McIlroy diffs. A single word inserted in a paragraph may push the final
166−paragraph may push the final word or two of the line onto the
167−next line, and the entire rest of the paragraph may be flagged as
145+word or two of the line onto the next line, and the entire rest of the paragraph
168−a change. Two diffs consisting of a single added word to
146+may be flagged as a change. Two diffs consisting of a single added word to
169−different sentences may be flagged as a conflict. In general,
147+different sentences may be flagged as a conflict. In general, prose should be
170−prose should be diffed by sentence, not by line.
148+diffed by sentence, not by line.
171−
149+
172−As far as the author is aware, clay is the first generalized,
150+As far as the author is aware, clay is the first generalized, type-aware
173−type-aware revision control system. We'll go into the workings
151+revision control system. We'll go into the workings of this system in some
174−of this system in some detail.
152+detail.
175
153
176### Marks
154### Marks
177
155
178−Central to a typed filesystem is the idea of types. In clay, we
156+Central to a typed filesystem is the idea of types. In clay, we call these
179−call these "marks". A mark is a file that defines a type,
157+"marks". A mark is a file that defines a type, conversion routines to and from
180−conversion routines to and from the mark, and diff, patch, and
181−merge routines.
158+the mark, and diff, patch, and merge routines.
182
159
183−For example, a `%txt` mark may be a list of lines of text, and it
160+For example, a `%txt` mark may be a list of lines of text, and it may include
184−may include conversions to `%mime` to allow it to be serialized
161+conversions to `%mime` to allow it to be serialized and sent to a browswer or to
185−and sent to a browswer or to the unix filesystem. It will also
186−include Hunt-McIlroy diff, patch, and merge algorithms.
162+the unix filesystem. It will also include Hunt-McIlroy diff, patch, and merge
163+algorithms.
187
164
188−A `%json` mark would be defined as a json object in the code, and
165+A `%json` mark would be defined as a json object in the code, and it would have
189−it would have a parser to convert from `%txt` and a printer to
166+a parser to convert from `%txt` and a printer to convert back to `%txt`. The
190−convert back to `%txt`. The diff, patch, and merge algorithms are
191−fairly straightforward for json, though they're very different
167+diff, patch, and merge algorithms are fairly straightforward for json, though
192−from the text ones.
168+they're very different from the text ones.
193
169
194−More formally, a mark is a core with three arms, `++grab`,
170+More formally, a mark is a core with three arms, `++grab`, `++grow`, and
195−`++grow`, and `++grad`. In `++grab` is a series of functions to
171+`++grad`. In `++grab` is a series of functions to convert from other marks to
196−convert from other marks to the given mark. In `++grow` is a
197−series of functions to convert from the given mark to other
172+the given mark. In `++grow` is a series of functions to convert from the given
198−marks. In `++grad` is `++diff`, `++pact`, `++join`, and `++mash`.
173+mark to other marks. In `++grad` is `++diff`, `++pact`, `++join`, and `++mash`.
199
174
200The types are as follows, in an informal pseudocode:
175The types are as follows, in an informal pseudocode:
201
176
@@ -213,208 +188,179 @@ The types are as follows, in an informal pseudocode:
213 ++ join: (<diff-type>, <diff-type>) -> <diff-type> or NULL
188 ++ join: (<diff-type>, <diff-type>) -> <diff-type> or NULL
214 ++ mash: (<diff-type>, <diff-type>) -> <diff-type>
189 ++ mash: (<diff-type>, <diff-type>) -> <diff-type>
215
190
216−These types are basically what you would expect. Not every mark
191+These types are basically what you would expect. Not every mark has each of
217−has each of these functions defined -- all of them are optional
192+these functions defined -- all of them are optional in the general case.
218−in the general case.
219
193
220−In general, for a particular mark, the `++grab` and `++grow` entries
194+In general, for a particular mark, the `++grab` and `++grow` entries (if they
221−(if they exist) should be inverses of each other.
195+exist) should be inverses of each other.
222
196
223−In `++grad`, `++diff` takes two instances of a mark and produces
197+In `++grad`, `++diff` takes two instances of a mark and produces a diff of them.
224−a diff of them. `++pact` takes an instance of a mark and patches
198+`++pact` takes an instance of a mark and patches it with the given diff.
225−it with the given diff. `++join` takes two diffs and attempts to
226−merge them into a single diff. If there are conflicts, it
199+`++join` takes two diffs and attempts to merge them into a single diff. If there
227−produces null. `++mash` takes two diffs and forces a merge,
200+are conflicts, it produces null. `++mash` takes two diffs and forces a merge,
228annotating any conflicts.
201annotating any conflicts.
229
202
230−In general, if `++diff` called with A and B produces diff D, then
203+In general, if `++diff` called with A and B produces diff D, then `++pact`
231−`++pact` called with A and D should produce B. Also, if `++join`
204+called with A and D should produce B. Also, if `++join` of two diffs does not
232−of two diffs does not produce null, then `++mash` of the same
233−diffs should produce the same result.
205+produce null, then `++mash` of the same diffs should produce the same result.
234
206
235−Alternately, instead of `++diff`, `++pact`, `++join`, and
207+Alternately, instead of `++diff`, `++pact`, `++join`, and `++mash`, a mark can
236−`++mash`, a mark can provide the same functionality by defining
208+provide the same functionality by defining `++sted` to be the name of another
237−`++sted` to be the name of another mark to which we wish to
238−delegate the revision control responsibilities. Then, before
209+mark to which we wish to delegate the revision control responsibilities. Then,
239−running any of those functions, clay will convert to the other
210+before running any of those functions, clay will convert to the other mark, and
240−mark, and convert back afterward. For example, the `%hoon` mark
211+convert back afterward. For example, the `%hoon` mark is revision-controlled in
241−is revision-controlled in the same way as `%txt`, so its `++grad`
242−is simply `++sted %txt`. Of course, `++txt` must be defined in
212+the same way as `%txt`, so its `++grad` is simply `++sted %txt`. Of course,
243−`++grow` and `++grab` as well.
213+`++txt` must be defined in `++grow` and `++grab` as well.
244
214
245−Every file in clay has a mark, and that mark must have a
215+Every file in clay has a mark, and that mark must have a fully-functioning
246−fully-functioning `++grad`. Marks are used for more than just
216+`++grad`. Marks are used for more than just clay, and other marks don't need a
247−clay, and other marks don't need a `++grad`, but if a piece of
248−data is to be saved to clay, we must know how to revision-control
217+`++grad`, but if a piece of data is to be saved to clay, we must know how to
249−it.
218+revision-control it.
250
219
251−Additionally, if a file is to be synced out to unix, then it must
220+Additionally, if a file is to be synced out to unix, then it must have
252−have conversion routines to and from the `%mime` mark.
221+conversion routines to and from the `%mime` mark.
253
222
254−##Using clay
223+## Using clay
255
224
256### Reading and Subscribing
225### Reading and Subscribing
257
226
258−When reading from Clay, there are three types of requests. A
227+When reading from Clay, there are three types of requests. A `%sing` request
259−`%sing` request asks for data at single revsion. A `%next`
228+asks for data at single revsion. A `%next` request asks to be notified the next
260−request asks to be notified the next time there's a change to
261−given file. A `%many` request asks to be notified on every
229+time there's a change to given file. A `%many` request asks to be notified on
262−change in a desk for a range of changes.
230+every change in a desk for a range of changes.
263−
231+
264−For `%sing` and `%next`, there are generally three things to be
232+For `%sing` and `%next`, there are generally three things to be queried. A `%u`
265−queried. A `%u` request simply checks for the existence of a
233+request simply checks for the existence of a file at a path. A `%x` request gets
266−file at a path. A `%x` request gets the data in the file at a
267−path. A `%y` request gets a hash of the data in the file at the
234+the data in the file at a path. A `%y` request gets a hash of the data in the
268−path combined with all its children and their data. Thus, `%y`
235+file at the path combined with all its children and their data. Thus, `%y` of a
269−of a node changes if it or any of its children change.
236+node changes if it or any of its children change.
270−
237+
271−A `%sing` request is fulfilled immediately if possible. If the
238+A `%sing` request is fulfilled immediately if possible. If the requested
272−requested revision is in the future, or is on another ship for
239+revision is in the future, or is on another ship for which we don't have the
273−which we don't have the result cached, we don't respond
274−immediately. If the requested revision is in the future, we wait
240+result cached, we don't respond immediately. If the requested revision is in the
275−until the revision happens before we respond to the request. If
241+future, we wait until the revision happens before we respond to the request. If
276−the request is for data on another ship, we pass on the request
242+the request is for data on another ship, we pass on the request to the other
277−to the other ship. In general, Clay subscriptions, like most
243+ship. In general, Clay subscriptions, like most things in Urbit, aren't
278−things in Urbit, aren't guaranteed to return immediately.
279−They'll return when they can, and they'll do so in a
244+guaranteed to return immediately. They'll return when they can, and they'll do
280−referentially transparent manner.
245+so in a referentially transparent manner.
281−
246+
282−A `%next` request checks query at the given revision, and it
247+A `%next` request checks query at the given revision, and it produces the result
283−produces the result of the query the next time it changes, along
248+of the query the next time it changes, along with the revsion number when it
284−with the revsion number when it changes. Thus, a `%next` of a
285−`%u` is triggered when a file is added or deleted, a `%next` of a
249+changes. Thus, a `%next` of a `%u` is triggered when a file is added or deleted,
286−`%x` is triggered when a file is added, deleted, or changed, and
250+a `%next` of a `%x` is triggered when a file is added, deleted, or changed, and
287−a `%next` of a `%y` is triggered when a file or any of its
251+a `%next` of a `%y` is triggered when a file or any of its children is added,
288−children is added, deleted, or changed.
252+deleted, or changed.
289−
253+
290−A `%many` request is triggered every time the given desk has a
254+A `%many` request is triggered every time the given desk has a new revision.
291−new revision. Unlike a `%next`, a `%many` has both a start and
255+Unlike a `%next`, a `%many` has both a start and an end revsion, after which it
292−an end revsion, after which it stops returning. For `%next`, a
293−single change is reported, and if the caller wishes to hear of
256+stops returning. For `%next`, a single change is reported, and if the caller
294−the next change, it must resubscribe. For `%many`, every revsion
257+wishes to hear of the next change, it must resubscribe. For `%many`, every
295−from the start to the end triggers a response. Since a `%many`
258+revsion from the start to the end triggers a response. Since a `%many` request
296−request doesn't ask for any particular data, there aren't `%u`,
259+doesn't ask for any particular data, there aren't `%u`, `%x`, and `%y` versions
297−`%x`, and `%y` versions for it.
260+for it.
298
261
299### Unix sync
262### Unix sync
300
263
301−One of the primary functions of clay is as a convenient user
264+One of the primary functions of clay is as a convenient user interface. While
302−interface. While tools exist to use clay from within urbit, it's
265+tools exist to use clay from within urbit, it's often useful to be able to treat
303−often useful to be able to treat clay like any other filesystem
304−from the Unix perspective -- to "mount" it, as it were.
266+clay like any other filesystem from the Unix perspective -- to "mount" it, as it
267+were.
305
268
306−From urbit, you can run `|mount /path/to/directory %mount-point`,
269+From urbit, you can run `|mount /path/to/directory %mount-point`, and this will
307−and this will mount the given clay directory to the mount-point
270+mount the given clay directory to the mount-point directory in Unix. Every file
308−directory in Unix. Every file is converted to `%mime` before it's
309−written to Unix, and converted back when read from Unix. The
271+is converted to `%mime` before it's written to Unix, and converted back when
310−entire directory is watched (a la Dropbox), and every change is
272+read from Unix. The entire directory is watched (a la Dropbox), and every change
311−auto-committed to clay.
273+is auto-committed to clay.
312
274
313### Merging
275### Merging
314
276
315−Merging is a fundamental operation for a distributed revision
277+Merging is a fundamental operation for a distributed revision control system. At
316−control system. At their root, clay's merges are similar to
278+their root, clay's merges are similar to git's, but with some additions to
317−git's, but with some additions to accomodate typed data. There
318−are seven different merge strategies.
279+accomodate typed data. There are seven different merge strategies.
319−
280+
320−Throughout our discussion, we'll say that the merge is from
281+Throughout our discussion, we'll say that the merge is from Alice's desk to
321−Alice's desk to Bob's. Recall that a commit is a date (for all
282+Bob's. Recall that a commit is a date (for all new commits this will be the
322−new commits this will be the current date), a list of parents,
283+current date), a list of parents, and the data itself.
323−and the data itself.
284+
324−
325−A `%init` merge should be used iff it's the first commit to a
285+A `%init` merge should be used iff it's the first commit to a desk. The head of
326−desk. The head of Alice's desk is used as the number 1 commit to
286+Alice's desk is used as the number 1 commit to Bob's desk. Obviously, the
327−Bob's desk. Obviously, the ancestry remains intact through
328−traversing the parentage of the commit even though previous
287+ancestry remains intact through traversing the parentage of the commit even
329−commits are not numbered for Bob's desk.
288+though previous commits are not numbered for Bob's desk.
330−
289+
331−A `%this` merge means to keep what's in Bob's desk, but join the
290+A `%this` merge means to keep what's in Bob's desk, but join the ancestry. Thus,
332−ancestry. Thus, the new commit has the head of each desk as
291+the new commit has the head of each desk as parents, but the data is exactly
333−parents, but the data is exactly what's in Bob's desk. For those
334−following along in git, this is the 'ours' merge strategy, not
292+what's in Bob's desk. For those following along in git, this is the 'ours' merge
335−the '--ours' option to the 'recursive' merge strategy. In other
293+strategy, not the '--ours' option to the 'recursive' merge strategy. In other
336−words, even if Alice makes a change that does not conflict with
294+words, even if Alice makes a change that does not conflict with Bob, we throw it
337−Bob, we throw it away. It's Bob's way or the highway.
295+away. It's Bob's way or the highway.
338−
296+
339−A `%that` merge means to take what's in Alice's desk, but join
297+A `%that` merge means to take what's in Alice's desk, but join the ancestry.
340−the ancestry. This is the reverse of `%this`.
298+This is the reverse of `%this`.
341−
299+
342−A `%fine` merge is a "fast-forward" merge. This succeeds iff one
300+A `%fine` merge is a "fast-forward" merge. This succeeds iff one head is in the
343−head is in the ancestry of the other. In this case, we use the
301+ancestry of the other. In this case, we use the descendant as our new head.
344−descendant as our new head.
302+
345−
346−For `%meet`, `%mate`, and `%meld` merges, we first find the most
303+For `%meet`, `%mate`, and `%meld` merges, we first find the most recent common
347−recent common ancestor to use as our merge base. If we have no
304+ancestor to use as our merge base. If we have no common ancestors, then we fail.
348−common ancestors, then we fail. If we have more than one most
349−recent common ancestor, then we have a criss-cross situation,
305+If we have more than one most recent common ancestor, then we have a criss-cross
350−which should be handled delicately. At present, we delicately
306+situation, which should be handled delicately. At present, we delicately throw
351−throw up our hands and give up, but something akin to git's
307+up our hands and give up, but something akin to git's 'recursive' strategy
352−'recursive' strategy should be implemented in the future.
308+should be implemented in the future.
353−
309+
354−There's a functional inclusion ordering on `%fine`, `%meet`,
310+There's a functional inclusion ordering on `%fine`, `%meet`, `%mate`, and
355−`%mate`, and `%meld` such that if an earlier strategy would have
311+`%meld` such that if an earlier strategy would have succeeded, then every later
356−succeeded, then every later strategy will produce the same
357−result. Put another way, every earlier strategy is the same as
312+strategy will produce the same result. Put another way, every earlier strategy
358−every later strategy except with a restricted domain.
313+is the same as every later strategy except with a restricted domain.
359−
314+
360−A `%meet` merge only succeeds if the changes from the merge base
315+A `%meet` merge only succeeds if the changes from the merge base to Alice's head
361−to Alice's head (hereafter, "Alice's changes") are in different
316+(hereafter, "Alice's changes") are in different files than Bob's changes. In
362−files than Bob's changes. In this case, the parents are both
363−Alice's and Bob's heads, and the data is the merge base plus
317+this case, the parents are both Alice's and Bob's heads, and the data is the
364−Alice's changed files plus Bob's changed files.
318+merge base plus Alice's changed files plus Bob's changed files.
365−
319+
366−A `%mate` merge attempts to merge changes to the same file when
320+A `%mate` merge attempts to merge changes to the same file when both Alice and
367−both Alice and bob change it. If the merge is clean, we use it;
321+bob change it. If the merge is clean, we use it; otherwise, we fail. A merge
368−otherwise, we fail. A merge between different types of changes --
369−for example, deleting a file vs changing it -- is always a
322+between different types of changes -- for example, deleting a file vs changing
370−conflict. If we succeed, the parents are both Alice's and Bob's
323+it -- is always a conflict. If we succeed, the parents are both Alice's and
371−heads, and the data is the merge base plus Alice's changed files
324+Bob's heads, and the data is the merge base plus Alice's changed files plus
372−plus Bob's changed files plus the merged files.
325+Bob's changed files plus the merged files.
373−
326+
374−A `%meld` merge will succeed even if there are conflicts. If
327+A `%meld` merge will succeed even if there are conflicts. If there are conflicts
375−there are conflicts in a file, then we use the merge base's
376−version of that file, and we produce a set of files with
328+in a file, then we use the merge base's version of that file, and we produce a
377−conflicts. The parents are both Alice's and Bob's heads, and the
329+set of files with conflicts. The parents are both Alice's and Bob's heads, and
378−data is the merge base plus Alice's changed files plus Bob's
330+the data is the merge base plus Alice's changed files plus Bob's changed files
379−changed files plus the successfully merged files plus the merge
331+plus the successfully merged files plus the merge base's version of the
380−base's version of the conflicting files.
332+conflicting files.
381−
333+
382−That's the extent of the merge options in clay proper. In
334+That's the extent of the merge options in clay proper. In userspace there's a
383−userspace there's a final option `%auto`, which is the most
335+final option `%auto`, which is the most common. `%auto` checks to see if Bob's
384−common. `%auto` checks to see if Bob's desk exists, and if it
385−doesn't we use a `%init` merge. Otherwise, we progressively try
336+desk exists, and if it doesn't we use a `%init` merge. Otherwise, we
386−`%fine`, `%meet`, and `%mate` until one succeeds.
337+progressively try `%fine`, `%meet`, and `%mate` until one succeeds.
387−
338+
388−If none succeed, we merge Bob's desk into a scratch desk. Then,
339+If none succeed, we merge Bob's desk into a scratch desk. Then, we merge Alice's
389−we merge Alice's desk into the scratch desk with the `%meld`
340+desk into the scratch desk with the `%meld` option to force the merge. For each
390−option to force the merge. For each file in the produced set of
391−conflicting files, we call the `++mash` function for the
341+file in the produced set of conflicting files, we call the `++mash` function for
392−appropriate mark, which annotates the conflicts if we know how.
342+the appropriate mark, which annotates the conflicts if we know how.
393−
343+
394−Finally, we display a message to the user informing them of the
344+Finally, we display a message to the user informing them of the scratch desk's
395−scratch desk's existence, which files have annotated conflicts,
345+existence, which files have annotated conflicts, and which files have
396−and which files have unannotated conflicts. When the user has
346+unannotated conflicts. When the user has resolved the conflicts, they can merge
397−resolved the conflicts, they can merge the scratch desk back into
398−Bob's desk. This will be a `%fine` merge since Bob's head is in
347+the scratch desk back into Bob's desk. This will be a `%fine` merge since Bob's
399−the ancestry of the scratch desk.
348+head is in the ancestry of the scratch desk.
400
349
401### Autosync
350### Autosync
402
351
403−Tracking and staying in sync with another desk is another
352+Tracking and staying in sync with another desk is another fundamental operation.
404−fundamental operation. We call this "autosync". This doesn't mean
353+We call this "autosync". This doesn't mean simply mirroring a desk, since that
405−simply mirroring a desk, since that wouldn't allow local changes.
406−We simply want to apply changes as they are made upstream, as
354+wouldn't allow local changes. We simply want to apply changes as they are made
407−long as there are no conflicts with local changes.
355+upstream, as long as there are no conflicts with local changes.
408
356
409−This is implemented by watching the other desk, and, when it has
357+This is implemented by watching the other desk, and, when it has changes,
410−changes, merging these changes into our desk with the usual merge
358+merging these changes into our desk with the usual merge strategies.
411−strategies.
412
359
413−Note that it's quite reasonable for two desks to be autosynced to
360+Note that it's quite reasonable for two desks to be autosynced to each other.
414−each other. This results in any change on one desk being mirrored
361+This results in any change on one desk being mirrored to the other and vice
415−to the other and vice versa.
362+versa.
416
363
417−Additionally, it's fine to set up an autosync even if one desk,
364+Additionally, it's fine to set up an autosync even if one desk, the other desk,
418−the other desk, or both desks do not exist. The sync will be
365+or both desks do not exist. The sync will be activated when the upstream desk
419−activated when the upstream desk comes into existence and will
420−create the downstream desk if needed.
366+comes into existence and will create the downstream desk if needed.
docs/arvo/internals/clay/commentary.md+724−811
@@ -1,46 +1,39 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3---
3---
4
4
5−
6−`%clay` commentary
5+# `%clay` commentary
7−==================
8
6
9`%clay` is our filesystem.
7`%clay` is our filesystem.
10
8
11−The first part of this will be reference documentation for the data
9+The first part of this will be reference documentation for the data types used
12−types used by our filesystem. In fact, as a general guide, we recommend
10+by our filesystem. In fact, as a general guide, we recommend reading and
13−reading and attempting to understand the data structures used in any
11+attempting to understand the data structures used in any Hoon code before you
14−Hoon code before you try to read the code itself. Although complete
12+try to read the code itself. Although complete understanding of the data
15−understanding of the data structures is impossible without seeing them
13+structures is impossible without seeing them used in the code, an 80%
16−used in the code, an 80% understanding greatly clarifies the code. As
14+understanding greatly clarifies the code. As another general guide, when reading
17−another general guide, when reading Hoon, it rarely pays off to
18−understand every line of code when it appears. Try to get the gist of
15+Hoon, it rarely pays off to understand every line of code when it appears. Try
19−it, and then move on. The next time you come back to it, it'll likely
16+to get the gist of it, and then move on. The next time you come back to it,
20−make a lot more sense.
17+it'll likely make a lot more sense.
21−
18+
22−After a description of the data models, we'll give an overview of the
19+After a description of the data models, we'll give an overview of the interface
23−interface that vanes and applications can use to interact with the
20+that vanes and applications can use to interact with the filesystem.
24−filesystem.
21+
25−
26−Finally, we'll dive into the code and the algorithms themselves. You
22+Finally, we'll dive into the code and the algorithms themselves. You know, the
27−know, the fun part.
23+fun part.
28−
24+
29−Data Models
25+## Data Models
30−-----------
26+
31−
32−As you're reading through this section, remember you can always come
27+As you're reading through this section, remember you can always come back to
33−back to this when you run into these types later on. You're not going to
28+this when you run into these types later on. You're not going to remember
34−remember everything the first time through, but it is worth reading, or
29+everything the first time through, but it is worth reading, or at least
35−at least skimming, this so that you get a rough idea of how our state is
30+skimming, this so that you get a rough idea of how our state is organized.
36−organized.
31+
37−
38−The types that are certainly worth reading are `++raft`, `++room`,
32+The types that are certainly worth reading are `++raft`, `++room`, `++dome`,
39−`++dome`, `++ankh`, `++rung`, `++rang`, `++blob`, `++yaki`, and `++nori`
33+`++ankh`, `++rung`, `++rang`, `++blob`, `++yaki`, and `++nori` (possibly in that
40−(possibly in that order). All in all, though, this section isn't too
34+order). All in all, though, this section isn't too long, so many readers may
41−long, so many readers may wish to quickly read through all of it. If you
35+wish to quickly read through all of it. If you get bored, though, just skip to
42−get bored, though, just skip to the next section. You can always come
36+the next section. You can always come back when you need to.
43−back when you need to.
44
37
45### `++raft`, formal state
38### `++raft`, formal state
46
39
@@ -50,18 +43,17 @@ back when you need to.
50 ran=rang :: hashes
43 ran=rang :: hashes
51 == ::
44 == ::
52
45
53−This is the state of our vane. Anything that must be remembered between
46+This is the state of our vane. Anything that must be remembered between calls to
54−calls to clay is stored in this state.
47+clay is stored in this state.
55
48
56−`fat` is the set of domestic servers. This stores all the information
49+`fat` is the set of domestic servers. This stores all the information that is
57−that is specfic to a particular ship on this pier. The keys to this map
50+specfic to a particular ship on this pier. The keys to this map are the ships on
58−are the ships on the current pier. all the information that is specific
51+the current pier. all the information that is specific to a particular foreign
59−to a particular foreign ship. The keys to this map are all the ships
52+ship. The keys to this map are all the ships whose filesystems we have attempted
60−whose filesystems we have attempted to access through clay.
53+to access through clay.
61
54
62−`ran` is the store of all commits and deltas, keyed by hash. The is
55+`ran` is the store of all commits and deltas, keyed by hash. The is where all
63−where all the "real" data we know is stored; the rest is "just
56+the "real" data we know is stored; the rest is "just bookkeeping".
64−bookkeeping".
65
57
66### `++room`, filesystem per domestic ship
58### `++room`, filesystem per domestic ship
67
59
@@ -73,42 +65,40 @@ bookkeeping".
73
65
74This is the representation of the filesystem of a ship on our pier.
66This is the representation of the filesystem of a ship on our pier.
75
67
76−`hun` is the duct we use to send messages to dill to display
68+`hun` is the duct we use to send messages to dill to display notifications of
77−notifications of filesystem changes. Only `%note` gifts should be
69+filesystem changes. Only `%note` gifts should be produced along this duct. This
78−produced along this duct. This is set by the `%init` kiss.
70+is set by the `%init` kiss.
79
71
80−`hez`, if present, is the duct we use to send sync messages to unix so
72+`hez`, if present, is the duct we use to send sync messages to unix so that they
81−that they end up in the pier unix directory. Only `%ergo` gifts should
73+end up in the pier unix directory. Only `%ergo` gifts should be producd along
82−be producd along this duct. This is set by `%into` and `%invo` kisses.
74+this duct. This is set by `%into` and `%invo` kisses.
83
75
84−`dos` is a well-known operating system released in 1981. It is also the
76+`dos` is a well-known operating system released in 1981. It is also the set of
85−set of desks on this ship, mapped to their data.
77+desks on this ship, mapped to their data.
86
78
87### `++desk`, filesystem branch
79### `++desk`, filesystem branch
88
80
89 ++ desk ,@tas :: ship desk case spur
81 ++ desk ,@tas :: ship desk case spur
90
82
91−This is the name of a branch of the filesystem. The default desks are
83+This is the name of a branch of the filesystem. The default desks are "arvo",
92−"arvo", "main", and "try". More may be created by simply referencing
84+"main", and "try". More may be created by simply referencing them. Desks have
93−them. Desks have independent histories and states, and they may be
85+independent histories and states, and they may be merged into each other.
94−merged into each other.
95
86
96### `++dojo`, domestic desk state
87### `++dojo`, domestic desk state
97
88
98 ++ dojo ,[p=cult q=dome] :: domestic desk state
89 ++ dojo ,[p=cult q=dome] :: domestic desk state
99
90
100−This is the all the data that is specific to a particular desk on a
91+This is the all the data that is specific to a particular desk on a domestic
101−domestic ship. `p` is the set of subscribers to this desk and `q` is the
92+ship. `p` is the set of subscribers to this desk and `q` is the data in the
102−data in the desk.
93+desk.
103
94
104### `++cult`, subscriptions
95### `++cult`, subscriptions
105
96
106 ++ cult (map duct rave) :: subscriptions
97 ++ cult (map duct rave) :: subscriptions
107
98
108−This is the set of subscriptions to a particular desk. The keys are the
99+This is the set of subscriptions to a particular desk. The keys are the ducts
109−ducts from where the subscriptions requests came. The results will be
100+from where the subscriptions requests came. The results will be produced along
110−produced along these ducts. The values are a description of the
101+these ducts. The values are a description of the requested information.
111−requested information.
112
102
113### `++rave`, general subscription request
103### `++rave`, general subscription request
114
104
@@ -117,61 +107,60 @@ requested information.
117 [| p=moat] :: change range
107 [| p=moat] :: change range
118 == ::
108 == ::
119
109
120−This represents a subscription request for a desk. The request can be
110+This represents a subscription request for a desk. The request can be for either
121−for either a single item in the desk or else for a range of changes on
111+a single item in the desk or else for a range of changes on the desk.
122−the desk.
123
112
124### `++rove`, stored general subscription request
113### `++rove`, stored general subscription request
125
114
126 ++ rove (each mood moot) :: stored request
115 ++ rove (each mood moot) :: stored request
127
116
128−When we store a request, we store subscriptions with a little extra
117+When we store a request, we store subscriptions with a little extra information
129−information so that we can determine whether new versions actually
118+so that we can determine whether new versions actually affect the path we're
130−affect the path we're subscribed to.
119+subscribed to.
131
120
132### `++mood`, single subscription request
121### `++mood`, single subscription request
133
122
134 ++ mood ,[p=care q=case r=path] :: request in desk
123 ++ mood ,[p=care q=case r=path] :: request in desk
135
124
136−This represents a request for the state of the desk at a particular
125+This represents a request for the state of the desk at a particular commit,
137−commit, specfied by `q`. `p` specifies what kind of information is
126+specfied by `q`. `p` specifies what kind of information is desired, and `r`
138−desired, and `r` specifies the path we are requesting.
127+specifies the path we are requesting.
139
128
140### `++moat`, range subscription request
129### `++moat`, range subscription request
141
130
142 ++ moat ,[p=case q=case r=path] :: change range
131 ++ moat ,[p=case q=case r=path] :: change range
143
132
144−This represents a request for all changes between `p` and `q` on path
133+This represents a request for all changes between `p` and `q` on path `r`. You
145−`r`. You will be notified when a change is made to the node referenced
134+will be notified when a change is made to the node referenced by the path or to
146−by the path or to any of its children.
135+any of its children.
147
136
148### `++moot`, stored range subscription request
137### `++moot`, stored range subscription request
149
138
150 ++ moot ,[p=case q=case r=path s=(map path lobe)] ::
139 ++ moot ,[p=case q=case r=path s=(map path lobe)] ::
151
140
152−This is just a `++moat` plus a map of paths to lobes. This map
141+This is just a `++moat` plus a map of paths to lobes. This map represents the
153−represents the data at the node referenced by the path at case `p`, if
142+data at the node referenced by the path at case `p`, if we've gotten to that
154−we've gotten to that case (else null). We only send a notification along
143+case (else null). We only send a notification along the subscription if the data
155−the subscription if the data at a new revision is different than it was.
144+at a new revision is different than it was.
156
145
157### `++care`, clay submode
146### `++care`, clay submode
158
147
159 ++ care ?(%u %v %w %x %y %z) :: clay submode
148 ++ care ?(%u %v %w %x %y %z) :: clay submode
160
149
161−This specifies what type of information is requested in a subscription
150+This specifies what type of information is requested in a subscription or a
162−or a scry.
151+scry.
163
152
164−`%u` requests the `++rang` at the current moment. Because this
153+`%u` requests the `++rang` at the current moment. Because this information is
165−information is not stored for any moment other than the present, we
154+not stored for any moment other than the present, we crash if the `++case` is
166−crash if the `++case` is not a `%da` for now.
155+not a `%da` for now.
167
156
168`%v` requests the `++dome` at the specified commit.
157`%v` requests the `++dome` at the specified commit.
169
158
170`%w` requests the revsion number of the desk.
159`%w` requests the revsion number of the desk.
171
160
172−`%x` requests the file at a specified path at the specified commit. If
161+`%x` requests the file at a specified path at the specified commit. If there is
173−there is no node at that path or if the node has no contents (that is,
162+no node at that path or if the node has no contents (that is, if `q:ankh` is
174−if `q:ankh` is null), then this produces null.
163+null), then this produces null.
175
164
176`%y` requests a `++arch` of the specfied commit at the specified path.
165`%y` requests a `++arch` of the specfied commit at the specified path.
177
166
@@ -181,13 +170,12 @@ if `q:ankh` is null), then this produces null.
181
170
182 ++ arch ,[p=@uvI q=(unit ,@uvI) r=(map ,@ta ,~)] :: fundamental node
171 ++ arch ,[p=@uvI q=(unit ,@uvI) r=(map ,@ta ,~)] :: fundamental node
183
172
184−This is analogous to `++ankh` except that the we have neither our
173+This is analogous to `++ankh` except that the we have neither our contents nor
185−contents nor the ankhs of our children. The other fields are exactly the
174+the ankhs of our children. The other fields are exactly the same, so `p` is a
186−same, so `p` is a hash of the associated ankh, `u.q`, if it exists, is a
175+hash of the associated ankh, `u.q`, if it exists, is a hash of the contents of
187−hash of the contents of this node, and the keys of `r` are the names of
176+this node, and the keys of `r` are the names of our children. `r` is a map to
188−our children. `r` is a map to null rather than a set so that the
189−ordering of the map will be equivalent to that of `r:ankh`, allowing
177+null rather than a set so that the ordering of the map will be equivalent to
190−efficient conversion.
178+that of `r:ankh`, allowing efficient conversion.
191
179
192### `++case`, specifying a commit
180### `++case`, specifying a commit
193
181
@@ -197,11 +185,10 @@ efficient conversion.
197 [%ud p=@ud] :: number
185 [%ud p=@ud] :: number
198 == ::
186 == ::
199
187
200−A commit can be referred to in three ways: `%da` refers to the commit
188+A commit can be referred to in three ways: `%da` refers to the commit that was
201−that was at the head on date `p`, `%tas` refers to the commit labeled
189+at the head on date `p`, `%tas` refers to the commit labeled `p`, and `%ud`
202−`p`, and `%ud` refers to the commit numbered `p`. Note that since these
190+refers to the commit numbered `p`. Note that since these all can be reduced down
203−all can be reduced down to a `%ud`, only numbered commits may be
191+to a `%ud`, only numbered commits may be referenced with a `++case`.
204−referenced with a `++case`.
205
192
206### `++dome`, desk data
193### `++dome`, desk data
207
194
@@ -217,29 +204,26 @@ This is the data that is actually stored in a desk.
217
204
218`ang` is unused and should be removed.
205`ang` is unused and should be removed.
219
206
220−`ank` is the current state of the desk. Thus, it is the state of the
207+`ank` is the current state of the desk. Thus, it is the state of the filesystem
221−filesystem at revison `let`. The head of a desk is always a numbered
208+at revison `let`. The head of a desk is always a numbered commit.
222−commit.
209+
223−
224−`let` is the number of the most recently numbered commit. This is also
210+`let` is the number of the most recently numbered commit. This is also the total
225−the total number of numbered commits.
211+number of numbered commits.
226−
212+
227−`hit` is a map of numerical ids to hashes of commits. These hashes are
213+`hit` is a map of numerical ids to hashes of commits. These hashes are mapped
228−mapped into their associated commits in `hut:rang`. In general, the keys
214+into their associated commits in `hut:rang`. In general, the keys of this map
229−of this map are exactly the numbers from 1 to `let`, with no gaps. Of
215+are exactly the numbers from 1 to `let`, with no gaps. Of course, when there are
230−course, when there are no numbered commits, `let` is 0, so `hit` is
216+no numbered commits, `let` is 0, so `hit` is null. Additionally, each of the
231−null. Additionally, each of the commits is an ancestor of every commit
217+commits is an ancestor of every commit numbered greater than this one. Thus,
232−numbered greater than this one. Thus, each is a descendant of every
233−commit numbered less than this one. Since it is true that the date in
218+each is a descendant of every commit numbered less than this one. Since it is
234−each commit (`t:yaki`) is no earlier than that of each of its parents,
219+true that the date in each commit (`t:yaki`) is no earlier than that of each of
235−the numbered commits are totally ordered in the same way by both
220+its parents, the numbered commits are totally ordered in the same way by both
236−pedigree and date. Of course, not every commit is numbered. If that
221+pedigree and date. Of course, not every commit is numbered. If that sounds too
237−sounds too complicated to you, don't worry about it. It basically
222+complicated to you, don't worry about it. It basically behaves exactly as you
238−behaves exactly as you would expect.
223+would expect.
239−
224+
240−`lab` is a map of textual labels to numbered commits. Note that labels
225+`lab` is a map of textual labels to numbered commits. Note that labels can only
241−can only be applied to numbered commits. Labels must be unique across a
226+be applied to numbered commits. Labels must be unique across a desk.
242−desk.
243
227
244### `++ankh`, filesystem node
228### `++ankh`, filesystem node
245
229
@@ -249,35 +233,34 @@ desk.
249 r=(map ,@ta ankh) :: folders
233 r=(map ,@ta ankh) :: folders
250 == ::
234 == ::
251
235
252−This is a single node in the filesystem. This may be file or a directory
236+This is a single node in the filesystem. This may be file or a directory or
253−or both. In earth filesystems, a node is a file xor a directory. On
237+both. In earth filesystems, a node is a file xor a directory. On mars, we're
254−mars, we're inclusive, so a node is a file ior a directory.
238+inclusive, so a node is a file ior a directory.
255
239
256`p` is a recursive hash that depends on the contents of the this file or
240`p` is a recursive hash that depends on the contents of the this file or
257directory and on any children.
241directory and on any children.
258
242
259−`q` is the contents of this file, if any. `p.q` is a hash of the
243+`q` is the contents of this file, if any. `p.q` is a hash of the contents while
260−contents while `q.q` is the data itself.
244+`q.q` is the data itself.
261
245
262−`r` is the set of children of this node. In the case of a pure file,
246+`r` is the set of children of this node. In the case of a pure file, this is
263−this is empty. The keys are the names of the children and the values
247+empty. The keys are the names of the children and the values are, recursively,
264−are, recursively, the nodes themselves.
248+the nodes themselves.
265
249
266### `++cash`, ankh hash
250### `++cash`, ankh hash
267
251
268 ++ cash ,@uvH :: ankh hash
252 ++ cash ,@uvH :: ankh hash
269
253
270This is a 128-bit hash of an ankh. These are mostly stored within ankhs
254This is a 128-bit hash of an ankh. These are mostly stored within ankhs
271−themselves, and they are used to check for changes in possibly-deep
255+themselves, and they are used to check for changes in possibly-deep hierarchies.
272−hierarchies.
273
256
274### `++rung`, filesystem per neighbor ship
257### `++rung`, filesystem per neighbor ship
275
258
276 ++ rung $: rus=(map desk rede) :: neighbor desks
259 ++ rung $: rus=(map desk rede) :: neighbor desks
277 == ::
260 == ::
278
261
279−This is the filesystem of a neighbor ship. The keys to this map are all
262+This is the filesystem of a neighbor ship. The keys to this map are all the
280−the desks we know about on their ship.
263+desks we know about on their ship.
281
264
282### `++rede`, desk state
265### `++rede`, desk state
283
266
@@ -288,18 +271,17 @@ the desks we know about on their ship.
288 dom=dome :: revision state
271 dom=dome :: revision state
289 == ::
272 == ::
290
273
291−This is our knowledge of the state of a desk, either foreign or
274+This is our knowledge of the state of a desk, either foreign or domestic.
292−domestic.
293
275
294−`lim` is the date of the last full update. We only respond to requests
276+`lim` is the date of the last full update. We only respond to requests for stuff
295−for stuff before this time.
277+before this time.
296
278
297−`qyx` is the list of subscribers to this desk. For domestic desks, this
279+`qyx` is the list of subscribers to this desk. For domestic desks, this is
298−is simply `p:dojo`, all subscribers to the desk, while in foreign desks
280+simply `p:dojo`, all subscribers to the desk, while in foreign desks this is all
299−this is all the subscribers from our ship to the foreign desk.
281+the subscribers from our ship to the foreign desk.
300
282
301−`ref` is the request manager for the desk. For domestic desks, this is
283+`ref` is the request manager for the desk. For domestic desks, this is null
302−null since we handle requests ourselves.
284+since we handle requests ourselves.
303
285
304`dom` is the actual data in the desk.
286`dom` is the actual data in the desk.
305
287
@@ -314,23 +296,22 @@ null since we handle requests ourselves.
314
296
315This is the request manager for a foreign desk.
297This is the request manager for a foreign desk.
316
298
317−`nix` is one more than the index of the most recent request. Thus, it is
299+`nix` is one more than the index of the most recent request. Thus, it is the
318−the next available request number.
300+next available request number.
319
301
320−`bom` is the set of outstanding requests. The keys of this map are some
302+`bom` is the set of outstanding requests. The keys of this map are some subset
321−subset of the numbers between 0 and one less than `nix`. The members of
303+of the numbers between 0 and one less than `nix`. The members of the map are
322−the map are exactly those requests that have not yet been fully
304+exactly those requests that have not yet been fully satisfied.
323−satisfied.
324
305
325−`fod` is the same set as `bom`, but from a different perspective. In
306+`fod` is the same set as `bom`, but from a different perspective. In particular,
326−particular, the values of `fod` are the same as the values of `bom`, and
307+the values of `fod` are the same as the values of `bom`, and the `p` out of the
327−the `p` out of the values of `bom` are the same as the keys of `fod`.
308+values of `bom` are the same as the keys of `fod`. Thus, we can map ducts to
328−Thus, we can map ducts to their associated request number and `++rave`,
309+their associated request number and `++rave`, and we can map numbers to their
329−and we can map numbers to their associated duct and `++rave`.
310+associated duct and `++rave`.
330
311
331−`haw` is a map from simple requests to their values. This acts as a
312+`haw` is a map from simple requests to their values. This acts as a cache for
332−cache for requests that have already been made. Thus, the second request
313+requests that have already been made. Thus, the second request for a particular
333−for a particular `++mood` is nearly instantaneous.
314+`++mood` is nearly instantaneous.
334
315
335### `++rang`, data store
316### `++rang`, data store
336
317
@@ -338,26 +319,23 @@ for a particular `++mood` is nearly instantaneous.
338 lat=(map lobe blob) ::
319 lat=(map lobe blob) ::
339 == ::
320 == ::
340
321
341−This is a set of data keyed by hash. Thus, this is where the "real" data
322+This is a set of data keyed by hash. Thus, this is where the "real" data is
342−is stored, but it is only meaningful if we know the hash of what we're
323+stored, but it is only meaningful if we know the hash of what we're looking for.
343−looking for.
344
324
345−`hut` is a map from hashes to commits. We often get the hashes from
325+`hut` is a map from hashes to commits. We often get the hashes from `hit:dome`,
346−`hit:dome`, which keys them by logical id. Not every commit has an id.
326+which keys them by logical id. Not every commit has an id.
347
327
348−`lat` is a map from hashes to the actual data. We often get the hashes
328+`lat` is a map from hashes to the actual data. We often get the hashes from a
349−from a `++yaki`, a commit, which references this map to get the data.
329+`++yaki`, a commit, which references this map to get the data. There is no
350−There is no `++blob` in any `++yaki`. They are only accessible through
330+`++blob` in any `++yaki`. They are only accessible through this map.
351−this map.
352
331
353### `++tako`, commit reference
332### `++tako`, commit reference
354
333
355 ++ tako ,@ :: yaki ref
334 ++ tako ,@ :: yaki ref
356
335
357−This is a hash of a `++yaki`, a commit. These are most notably used as
336+This is a hash of a `++yaki`, a commit. These are most notably used as the keys
358−the keys in `hut:rang`, where they are associated with the actual
337+in `hut:rang`, where they are associated with the actual `++yaki`, and as the
359−`++yaki`, and as the values in `hit:dome`, where sequential ids are
338+values in `hit:dome`, where sequential ids are associated with these.
360−associated with these.
361
339
362### `++yaki`, commit
340### `++yaki`, commit
363
341
@@ -365,15 +343,14 @@ associated with these.
365
343
366This is a single commit.
344This is a single commit.
367
345
368−`p` is a list of the hashes of the parents of this commit. In most
346+`p` is a list of the hashes of the parents of this commit. In most cases, this
369−cases, this will be a single commit, but in a merge there may be more
347+will be a single commit, but in a merge there may be more parents. In theory,
370−parents. In theory, there may be an arbitrary number of parents, but in
348+there may be an arbitrary number of parents, but in practice merges have exactly
371−practice merges have exactly two parents. This may change in the future.
372−For commit 1, there is no parent.
349+two parents. This may change in the future. For commit 1, there is no parent.
373
350
374`q` is a map of the paths on a desk to the data at that location. If you
351`q` is a map of the paths on a desk to the data at that location. If you
375−understand what a `++lobe` and a `++blob` is, then the type signature
352+understand what a `++lobe` and a `++blob` is, then the type signature here tells
376−here tells the whole story.
353+the whole story.
377
354
378`r` is the hash associated with this commit.
355`r` is the hash associated with this commit.
379
356
@@ -383,9 +360,9 @@ here tells the whole story.
383
360
384 ++ lobe ,@ :: blob ref
361 ++ lobe ,@ :: blob ref
385
362
386−This is a hash of a `++blob`. These are most notably used in `lat:rang`,
363+This is a hash of a `++blob`. These are most notably used in `lat:rang`, where
387−where they are associated with the actual `++blob`, and as the values in
364+they are associated with the actual `++blob`, and as the values in `q:yaki`,
388−`q:yaki`, where paths are associated with their data in a commit.
365+where paths are associated with their data in a commit.
389
366
390### `++blob`, data
367### `++blob`, data
391
368
@@ -396,22 +373,21 @@ where they are associated with the actual `++blob`, and as the values in
396
373
397This is a node of data. In every case, `p` is the hash of the blob.
374This is a node of data. In every case, `p` is the hash of the blob.
398
375
399−`%delta` is the case where we define the data by a delta on other data.
376+`%delta` is the case where we define the data by a delta on other data. In
400−In practice, the other data is always the previous commit, but nothing
377+practice, the other data is always the previous commit, but nothing depends on
401−depends on this. `q` is the hash of the parent blob, and `r` is the
378+this. `q` is the hash of the parent blob, and `r` is the delta.
402−delta.
403
379
404−`%direct` is the case where we simply have the data directly. `q` is the
380+`%direct` is the case where we simply have the data directly. `q` is the data
405−data itself, and `r` is any preprocessing instructions. These almost
381+itself, and `r` is any preprocessing instructions. These almost always come from
406−always come from the creation of a file.
382+the creation of a file.
407
383
408−`%indirect` is both of the preceding cases at once. `q` is the direct
384+`%indirect` is both of the preceding cases at once. `q` is the direct data, `r`
409−data, `r` is the delta, and `s` is the parent blob. It should always be
385+is the delta, and `s` is the parent blob. It should always be the case that
410−the case that applying `r` to `s` gives the same data as `q` directly
386+applying `r` to `s` gives the same data as `q` directly (with the
411−(with the prepreprocessor instructions in `p.r`). This exists purely for
387+prepreprocessor instructions in `p.r`). This exists purely for performance
412−performance reasons. This is unused, at the moment, but in general these
388+reasons. This is unused, at the moment, but in general these should be created
413−should be created when there are a long line of changes so that we do
389+when there are a long line of changes so that we do not have to traverse the
414−not have to traverse the delta chain back to the creation of the file.
390+delta chain back to the creation of the file.
415
391
416### `++udon`, abstract delta
392### `++udon`, abstract delta
417
393
@@ -425,33 +401,31 @@ not have to traverse the delta chain back to the creation of the file.
425 == ::
401 == ::
426 == ::
402 == ::
427
403
428−This is an abstract change to a file. This is a superset of what would
404+This is an abstract change to a file. This is a superset of what would normally
429−normally be called diffs. Diffs usually refer to changes in lines of
405+be called diffs. Diffs usually refer to changes in lines of text while we have
430−text while we have the ability to do more interesting deltas on
406+the ability to do more interesting deltas on arbitrary data structures.
431−arbitrary data structures.
432
407
433`p` is any preprocessor instructions.
408`p` is any preprocessor instructions.
434
409
435−`%a` refers to the trival delta of a complete replace of old data with
410+`%a` refers to the trival delta of a complete replace of old data with new data.
436−new data.
437
411
438−`%b` refers to changes in an opaque atom on the block level. This has
412+`%b` refers to changes in an opaque atom on the block level. This has very
439−very limited usefulness, and is not used at the moment.
413+limited usefulness, and is not used at the moment.
440
414
441−`%c` refers to changes in a list of data. This is often lines of text,
415+`%c` refers to changes in a list of data. This is often lines of text, which is
442−which is your classic diff. We, however, will work on any list of data.
416+your classic diff. We, however, will work on any list of data.
443
417
444−`%d` refers to changes in a tree of data. This is general enough to
418+`%d` refers to changes in a tree of data. This is general enough to describe
445−describe changes to any hoon noun, but often more special-purpose delta
419+changes to any hoon noun, but often more special-purpose delta should be created
446−should be created for different content types. This is not used at the
420+for different content types. This is not used at the moment, and may in fact be
447−moment, and may in fact be unimplemented.
421+unimplemented.
448
422
449### `++urge`, list change
423### `++urge`, list change
450
424
451 ++ urge |*(a=_,* (list (unce a))) :: list change
425 ++ urge |*(a=_,* (list (unce a))) :: list change
452
426
453−This is a parametrized type for list changes. For example, `(urge ,@t)`
427+This is a parametrized type for list changes. For example, `(urge ,@t)` is a
454−is a list change for lines of text.
428+list change for lines of text.
455
429
456### `++unce`, change part of a list.
430### `++unce`, change part of a list.
457
431
@@ -477,8 +451,8 @@ This is a single change in a list of elements of type `a`. For example,
477 $% [%d p=@ud] :: blocklist
451 $% [%d p=@ud] :: blocklist
478 == ::
452 == ::
479
453
480−This space intentionally left undocumented. This stuff will change once
454+This space intentionally left undocumented. This stuff will change once we get a
481−we get a well-typed clay.
455+well-typed clay.
482
456
483### `++upas`, tree change
457### `++upas`, tree change
484
458
@@ -489,9 +463,9 @@ we get a well-typed clay.
489 [%2 p=axis q=udon] :: mutate!
463 [%2 p=axis q=udon] :: mutate!
490 == ::
464 == ::
491
465
492−This space intentionally left undocumented. This stuff is not known to
466+This space intentionally left undocumented. This stuff is not known to work, and
493−work, and will likely change when we get a well-typed clay. Also, this
467+will likely change when we get a well-typed clay. Also, this is not a
494−is not a complicated type; it is not difficult to work out the meaning.
468+complicated type; it is not difficult to work out the meaning.
495
469
496### `++nori`, repository action
470### `++nori`, repository action
497
471
@@ -500,22 +474,21 @@ is not a complicated type; it is not difficult to work out the meaning.
500 [| p=@tas] :: label
474 [| p=@tas] :: label
501 == ::
475 == ::
502
476
503−This describes a change that we are asking clay to make to the desk.
477+This describes a change that we are asking clay to make to the desk. There are
504−There are two kinds of changes that may be made: we can modify files or
478+two kinds of changes that may be made: we can modify files or we can apply a
505−we can apply a label to a commit.
479+label to a commit.
506
480
507−In the `|` case, we will simply label the current commit with the given
481+In the `|` case, we will simply label the current commit with the given label.
508−label. In the `&` case, we will apply the given changes.
482+In the `&` case, we will apply the given changes.
509
483
510### `++soba`, delta
484### `++soba`, delta
511
485
512 ++ soba ,[p=cart q=(list ,[p=path q=miso])] :: delta
486 ++ soba ,[p=cart q=(list ,[p=path q=miso])] :: delta
513
487
514−This describes a set of changes to make to a desk. The `cart` is simply
488+This describes a set of changes to make to a desk. The `cart` is simply a pair
515−a pair of the old hash and the new hash of the desk. The list is a list
489+of the old hash and the new hash of the desk. The list is a list of changes
516−of changes keyed by the file they're changing. Thus, the paths are paths
490+keyed by the file they're changing. Thus, the paths are paths to files to be
517−to files to be changed while `miso` is a description of the change
491+changed while `miso` is a description of the change itself.
518−itself.
519
492
520### `++miso`, ankh delta
493### `++miso`, ankh delta
521
494
@@ -525,40 +498,38 @@ itself.
525 [%mut p=udon] :: mutate
498 [%mut p=udon] :: mutate
526 == ::
499 == ::
527
500
528−There are three kinds of changes that may be made to a node in a desk.
501+There are three kinds of changes that may be made to a node in a desk. We can
529−We can insert a file, in which case `p` is the contents of the new file.
502+insert a file, in which case `p` is the contents of the new file. We can delete
530−We can delete a file, in which case `p` is the contents of the old file.
503+a file, in which case `p` is the contents of the old file. Finally, we can
531−Finally, we can mutate that file, in which case the `udon` describes the
504+mutate that file, in which case the `udon` describes the changes we are applying
532−changes we are applying to the file.
505+to the file.
533
506
534### `++mizu`, merged state
507### `++mizu`, merged state
535
508
536 ++ mizu ,[p=@u q=(map ,@ud tako) r=rang] :: new state
509 ++ mizu ,[p=@u q=(map ,@ud tako) r=rang] :: new state
537
510
538−This is the input to the `%merg` kiss, which allows us to perform a
511+This is the input to the `%merg` kiss, which allows us to perform a merge. The
539−merge. The `p` is the number of the new head commit. The `q` is a map
512+`p` is the number of the new head commit. The `q` is a map from numbers to
540−from numbers to commit hashes. This is all the new numbered commits that
513+commit hashes. This is all the new numbered commits that are to be inserted. The
541−are to be inserted. The keys to this should always be the numbers from
514+keys to this should always be the numbers from `let.dom` plus one to `p`,
542−`let.dom` plus one to `p`, inclusive. The `r` is the maps of all the new
515+inclusive. The `r` is the maps of all the new commits and data. Since these are
543−commits and data. Since these are merged into the current state, no old
544−commits or data need be here.
516+merged into the current state, no old commits or data need be here.
545
517
546### `++riff`, request/desist
518### `++riff`, request/desist
547
519
548 ++ riff ,[p=desk q=(unit rave)] :: request/desist
520 ++ riff ,[p=desk q=(unit rave)] :: request/desist
549
521
550−This represents a request for data about a particular desk. If `q`
522+This represents a request for data about a particular desk. If `q` contains a
551−contains a `rave`, then this opens a subscription to the desk for that
523+`rave`, then this opens a subscription to the desk for that data. If `q` is
552−data. If `q` is null, then this tells clay to cancel the subscription
524+null, then this tells clay to cancel the subscription along this duct.
553−along this duct.
554
525
555### `++riot`, response
526### `++riot`, response
556
527
557 ++ riot (unit rant) :: response/complete
528 ++ riot (unit rant) :: response/complete
558
529
559−A riot is a response to a subscription. If null, the subscription has
530+A riot is a response to a subscription. If null, the subscription has been
560−been completed, and no more responses will be sent. Otherwise, the
531+completed, and no more responses will be sent. Otherwise, the `rant` is the
561−`rant` is the produced data.
532+produced data.
562
533
563### `++rant`, response data
534### `++rant`, response data
564
535
@@ -568,11 +539,11 @@ been completed, and no more responses will be sent. Otherwise, the
568 r=* :: data
539 r=* :: data
569 == ::
540 == ::
570
541
571−This is the data at a particular node in the filesystem. `p.p` specifies
542+This is the data at a particular node in the filesystem. `p.p` specifies the
572−the type of data that was requested (and is produced). `q.p` gives the
543+type of data that was requested (and is produced). `q.p` gives the specific
573−specific version reported (since a range of versions may be requested in
544+version reported (since a range of versions may be requested in a subscription).
574−a subscription). `r.p` is the desk. `q` is the path to the filesystem
545+`r.p` is the desk. `q` is the path to the filesystem node. `r` is the data
575−node. `r` is the data itself (in the format specified by `p.p`).
546+itself (in the format specified by `p.p`).
576
547
577### `++nako`, subscription response data
548### `++nako`, subscription response data
578
549
@@ -582,26 +553,24 @@ node. `r` is the data itself (in the format specified by `p.p`).
582 bar=(set blob) :: new content
553 bar=(set blob) :: new content
583 == ::
554 == ::
584
555
585−This is the data that is produced by a request for a range of revisions
556+This is the data that is produced by a request for a range of revisions of a
586−of a desk. This allows us to easily keep track of a remote repository --
557+desk. This allows us to easily keep track of a remote repository -- all the new
587−all the new information we need is contained in the `nako`.
558+information we need is contained in the `nako`.
588
559
589−`gar` is a map of the revisions in the range to the hash of the commit
560+`gar` is a map of the revisions in the range to the hash of the commit at that
590−at that revision. These hashes can be used with `hut:rang` to find the
561+revision. These hashes can be used with `hut:rang` to find the commit itself.
591−commit itself.
592
562
593`let` is either the last revision number in the range or the most recent
563`let` is either the last revision number in the range or the most recent
594revision number, whichever is smaller.
564revision number, whichever is smaller.
595
565
596`lar` is the set of new commits, and `bar` is the set of new content.
566`lar` is the set of new commits, and `bar` is the set of new content.
597
567
598−Public Interface
568+## Public Interface
599−----------------
600
569
601−As with all vanes, there are exactly two ways to interact with clay.
570+As with all vanes, there are exactly two ways to interact with clay. `%clay`
602−`%clay` exports a namespace accessible through `.^`, which is described
571+exports a namespace accessible through `.^`, which is described above under
603−above under `++care`. The primary way of interacting with clay, though,
572+`++care`. The primary way of interacting with clay, though, is by sending kisses
604−is by sending kisses and receiving gifts.
573+and receiving gifts.
605
574
606 ++ gift :: out result <-$
575 ++ gift :: out result <-$
607 $% [%ergo p=@p q=@tas r=@ud] :: version update
576 $% [%ergo p=@p q=@tas r=@ud] :: version update
@@ -619,91 +588,82 @@ is by sending kisses and receiving gifts.
619 [%warp p=sock q=riff] :: file request
588 [%warp p=sock q=riff] :: file request
620 == ::
589 == ::
621
590
622−There are only a small number of possible kisses, so it behooves us to
591+There are only a small number of possible kisses, so it behooves us to describe
623−describe each in detail.
592+each in detail.
624
593
625 $% [%info p=@p q=@tas r=nori] :: internal edit
594 $% [%info p=@p q=@tas r=nori] :: internal edit
626
595
627 [%into p=@p q=@tas r=nori] :: external edit
596 [%into p=@p q=@tas r=nori] :: external edit
628
597
629−These two kisses are nearly identical. At a high level, they apply
598+These two kisses are nearly identical. At a high level, they apply changes to
630−changes to the filesystem. Whenever we add, remove, or edit a file, one
599+the filesystem. Whenever we add, remove, or edit a file, one of these cards is
631−of these cards is sent. The `p` is the ship whose filesystem we're
600+sent. The `p` is the ship whose filesystem we're trying to change, the `q` is
632−trying to change, the `q` is the desk we're changing, and the `r` is the
601+the desk we're changing, and the `r` is the request change. For the format of
633−request change. For the format of the requested change, see the
634−documentation for `++nori` above.
602+the requested change, see the documentation for `++nori` above.
635
603
636−When a file is changed in the unix filesystem, vere will send a `%into`
604+When a file is changed in the unix filesystem, vere will send a `%into` kiss.
637−kiss. This tells clay that the duct over which the kiss was sent is the
605+This tells clay that the duct over which the kiss was sent is the duct that unix
638−duct that unix is listening on for changes. From within Arvo, though, we
606+is listening on for changes. From within Arvo, though, we should never send a
639−should never send a `%into` kiss. The `%info` kiss is exactly identical
607+`%into` kiss. The `%info` kiss is exactly identical except it does not reset the
640−except it does not reset the duct.
608+duct.
641
609
642 [%ingo p=@p q=@tas r=nori] :: internal noun edit
610 [%ingo p=@p q=@tas r=nori] :: internal noun edit
643
611
644 [%invo p=@p q=@tas r=nori] :: external noun edit
612 [%invo p=@p q=@tas r=nori] :: external noun edit
645
613
646−These kisses are currently identical to `%info` and `%into`, though this
614+These kisses are currently identical to `%info` and `%into`, though this will
647−will not always be the case. The intent is for these kisses to allow
615+not always be the case. The intent is for these kisses to allow typed changes to
648−typed changes to clay so that we may store typed data. This is currently
616+clay so that we may store typed data. This is currently unimplemented.
649−unimplemented.
650
617
651 [%init p=@p] :: report install
618 [%init p=@p] :: report install
652
619
653−Init is called when a ship is started on our pier. This simply creates a
620+Init is called when a ship is started on our pier. This simply creates a default
654−default `room` to go into our `raft`. Essentially, this initializes the
621+`room` to go into our `raft`. Essentially, this initializes the filesystem for a
655−filesystem for a ship.
622+ship.
656
623
657 [%merg p=@p q=@tas r=mizu] :: internal change
624 [%merg p=@p q=@tas r=mizu] :: internal change
658
625
659−This is called to perform a merge. This is most visibly called by
626+This is called to perform a merge. This is most visibly called by :update to
660−:update to update the filesystem of the current ship to that of its
627+update the filesystem of the current ship to that of its sein. The `p` and `q`
661−sein. The `p` and `q` are as in `%info`, and the `r` is the description
628+are as in `%info`, and the `r` is the description of the merge. See `++mizu`
662−of the merge. See `++mizu` above.
629+above.
663
630
664XX
631XX
665`XX [%wake ~] :: timer activate XX`
632`XX [%wake ~] :: timer activate XX`
666−XX\
667−XX This card is sent by unix at the time specified by `++doze`. This
633+XX\\ XX This card is sent by unix at the time specified by `++doze`. This time
668−time is XX usually the closest time specified in a subscription request.
634+is XX usually the closest time specified in a subscription request. When `%wake`
669−When `%wake` is XX called, we update our subscribers if there have been
635+is XX called, we update our subscribers if there have been any changes.
670−any changes.
671
636
672 [%wart p=sock q=@tas r=path s=*] :: network request
637 [%wart p=sock q=@tas r=path s=*] :: network request
673
638
674−This is a request that has come across the network for a particular
639+This is a request that has come across the network for a particular file. When
675−file. When another ship asks for a file from us, that request comes to
640+another ship asks for a file from us, that request comes to us in the form of a
676−us in the form of a `%wart` kiss. This is handled by trivially turning
641+`%wart` kiss. This is handled by trivially turning it into a `%warp`.
677−it into a `%warp`.
678
642
679 [%warp p=sock q=riff] :: file request
643 [%warp p=sock q=riff] :: file request
680
644
681−This is a request for information about a particular desk. This is, in
645+This is a request for information about a particular desk. This is, in its most
682−its most general form, a subscription, though in many cases it is the
646+general form, a subscription, though in many cases it is the trivial case of a
683−trivial case of a subscription -- a read. See `++riff` for the format of
647+subscription -- a read. See `++riff` for the format of the request.
684−the request.
685
648
686−Lifecycle of a Local Read
649+## Lifecycle of a Local Read
687−-------------------------
688
650
689−There are two real types of interaction with a filesystem: you can read,
651+There are two real types of interaction with a filesystem: you can read, and you
690−and you can write. We'll describe each process, detailing both the flow
652+can write. We'll describe each process, detailing both the flow of control
691−of control followed by the kernel and the algorithms involved. The
653+followed by the kernel and the algorithms involved. The simpler case is that of
692−simpler case is that of the read, so we'll begin with that.
654+the read, so we'll begin with that.
693
655
694−When a vane or an application wishes to read a file from the filesystem,
656+When a vane or an application wishes to read a file from the filesystem, it
695−it sends a `%warp` kiss, as described above. Of course, you may request
657+sends a `%warp` kiss, as described above. Of course, you may request a file on
696−a file on another ship and, being a global filesystem, clay will happily
658+another ship and, being a global filesystem, clay will happily produce it for
697−produce it for you. That code pathway will be described in another
659+you. That code pathway will be described in another section; here, we will
698−section; here, we will restrict ourselves to examining the case of a
660+restrict ourselves to examining the case of a read from a ship on our own pier.
699−read from a ship on our own pier.
700
661
701−The kiss can request either a single version of a file node or a range
662+The kiss can request either a single version of a file node or a range of
702−of versions of a desk. Here, we'll deal only with a request for a single
663+versions of a desk. Here, we'll deal only with a request for a single version.
703−version.
704
664
705−As in all vanes, a kiss enters clay via a call to `++call`. Scanning
665+As in all vanes, a kiss enters clay via a call to `++call`. Scanning through the
706−through the arm, we quickly see where `%warp` is handled.
666+arm, we quickly see where `%warp` is handled.
707
667
708 ?: =(p.p.q.hic q.p.q.hic)
668 ?: =(p.p.q.hic q.p.q.hic)
709 =+ une=(un p.p.q.hic now ruf)
669 =+ une=(un p.p.q.hic now ruf)
@@ -716,33 +676,31 @@ through the arm, we quickly see where `%warp` is handled.
716 abet:wao
676 abet:wao
717 [-.woo abet:(pish:une p.q.q.hic +.woo ran.wao)]
677 [-.woo abet:(pish:une p.q.q.hic +.woo ran.wao)]
718
678
719−We're following the familar patern of producing a list of moves and an
679+We're following the familar patern of producing a list of moves and an updated
720−updated state. In this case, the state is `++raft`.
680+state. In this case, the state is `++raft`.
721
681
722−We first check to see if the sending and receiving ships are the same.
682+We first check to see if the sending and receiving ships are the same. If
723−If they're not, then this is a request for data on another ship. We
683+they're not, then this is a request for data on another ship. We describe that
724−describe that process later. Here, we discuss only the case of a local
684+process later. Here, we discuss only the case of a local read.
725−read.
726
685
727−At a high level, the call to `++un` sets up the core for the domestic
686+At a high level, the call to `++un` sets up the core for the domestic ship that
728−ship that contains the files we're looking for. The call to `++di` sets
687+contains the files we're looking for. The call to `++di` sets up the core for
729−up the core for the particular desk we're referring to.
688+the particular desk we're referring to.
730
689
731−After this, we perform the actual request. If there is no rave in the
690+After this, we perform the actual request. If there is no rave in the riff, then
732−riff, then that means we are cancelling a request, so we call
691+that means we are cancelling a request, so we call `++ease:de`. Otherwise, we
733−`++ease:de`. Otherwise, we start a subscription with `++eave:de`. We
692+start a subscription with `++eave:de`. We call `++abet:de` to resolve our
734−call `++abet:de` to resolve our various types of output into actual
735−moves. We produce the moves we found above and the `++un` core resolved
693+various types of output into actual moves. We produce the moves we found above
736−with `++pish:un` (putting the modified desk in the room) and `++abet:un`
694+and the `++un` core resolved with `++pish:un` (putting the modified desk in the
737−(putting the modified room in the raft).
695+room) and `++abet:un` (putting the modified room in the raft).
738
696
739Much of this is fairly straightforward, so we'll only describe `++ease`,
697Much of this is fairly straightforward, so we'll only describe `++ease`,
740−`++eave`, and `++abet:de`. Feel free to look up the code to the other
698+`++eave`, and `++abet:de`. Feel free to look up the code to the other steps --
741−steps -- it should be easy to follow.
699+it should be easy to follow.
742
700
743−Although it's called last, it's usually worth examining `++abet` first,
701+Although it's called last, it's usually worth examining `++abet` first, since it
744−since it defines in what ways we can cause side effects. Let's do that,
702+defines in what ways we can cause side effects. Let's do that, and also a few of
745−and also a few of the lines at the beginning of `++de`.
703+the lines at the beginning of `++de`.
746
704
747 =| yel=(list ,[p=duct q=gift])
705 =| yel=(list ,[p=duct q=gift])
748 =| byn=(list ,[p=duct q=riot])
706 =| byn=(list ,[p=duct q=riot])
@@ -772,31 +730,31 @@ and also a few of the lines at the beginning of `++de`.
772 |=([a=duct b=path c=note] [a %pass b c])
730 |=([a=duct b=path c=note] [a %pass b c])
773 ==
731 ==
774
732
775−This is very simple code. We see there are exactly five different kinds
733+This is very simple code. We see there are exactly five different kinds of side
776−of side effects we can generate.
734+effects we can generate.
777
735
778−In `yel` we put gifts that we wish to be sent along the `hun:room` duct
736+In `yel` we put gifts that we wish to be sent along the `hun:room` duct to dill.
779−to dill. See the documentation for `++room` above. This is how we
737+See the documentation for `++room` above. This is how we display messages to the
780−display messages to the terminal.
738+terminal.
781
739
782−In `byn` we put riots that we wish returned to subscribers. Recall that
740+In `byn` we put riots that we wish returned to subscribers. Recall that a riot
783−a riot is a response to a subscription. These are returned to our
741+is a response to a subscription. These are returned to our subscribers in the
784−subscribers in the form of a `%writ` gift.
742+form of a `%writ` gift.
785
743
786−In `vag` we put gifts along with the ducts on which to send them. This
744+In `vag` we put gifts along with the ducts on which to send them. This allows us
787−allows us to produce arbitrary gifts, but in practice this is only used
745+to produce arbitrary gifts, but in practice this is only used to produce `%ergo`
788−to produce `%ergo` gifts.
746+gifts.
789
747
790−In `say` we put messages we wish to pass to ames. These messages are
748+In `say` we put messages we wish to pass to ames. These messages are used to
791−used to request information from clay on other piers. We must provide
749+request information from clay on other piers. We must provide not only the duct
792−not only the duct and the request (the riff), but also the return path,
750+and the request (the riff), but also the return path, the other ship to talk to,
793−the other ship to talk to, and the sequence number of the request.
751+and the sequence number of the request.
794
752
795−In `tag` we put arbitrary notes we wish to pass to other vanes. For now,
753+In `tag` we put arbitrary notes we wish to pass to other vanes. For now, the
796−the only notes we pass here are `%wait` and `%rest` to the timer vane.
754+only notes we pass here are `%wait` and `%rest` to the timer vane.
797
755
798−Now that we know what kinds of side effects we may have, we can jump
756+Now that we know what kinds of side effects we may have, we can jump into the
799−into the handling of requests.
757+handling of requests.
800
758
801 ++ ease :: release request
759 ++ ease :: release request
802 |= hen=duct
760 |= hen=duct
@@ -815,24 +773,24 @@ into the handling of requests.
815 bom.u.ref (~(del by bom.u.ref) u.nux)
773 bom.u.ref (~(del by bom.u.ref) u.nux)
816 ==
774 ==
817
775
818−This is called when we're cancelling a subscription. For domestic desks,
776+This is called when we're cancelling a subscription. For domestic desks, `ref`
819−`ref` is null, so we're going to cancel any timer we might have created.
777+is null, so we're going to cancel any timer we might have created. We first
820−We first delete the duct from our map of requests, and then we call
778+delete the duct from our map of requests, and then we call `++mabe` with
821−`++mabe` with `++best` to send a `%rest` kiss to the timer vane if we
779+`++best` to send a `%rest` kiss to the timer vane if we have started a timer.
822−have started a timer. We'll describe `++best` and `++mabe` momentarily.
780+We'll describe `++best` and `++mabe` momentarily.
823
781
824−Although we said we're not going to talk about foreign requests yet,
782+Although we said we're not going to talk about foreign requests yet, it's easy
825−it's easy to see that for foreign desks, we cancel any outstanding
783+to see that for foreign desks, we cancel any outstanding requests for this duct
826−requests for this duct and send a message over ames to the other ship
784+and send a message over ames to the other ship telling them to cancel the
827−telling them to cancel the subscription.
785+subscription.
828
786
829 ++ best
787 ++ best
830 |= [hen=duct tym=@da]
788 |= [hen=duct tym=@da]
831 %_(+> tag :_(tag [hen /tyme %t %rest tym]))
789 %_(+> tag :_(tag [hen /tyme %t %rest tym]))
832
790
833−This simply pushes a `%rest` note onto `tag`, from where it will be
791+This simply pushes a `%rest` note onto `tag`, from where it will be passed back
834−passed back to arvo to be handled. This cancels the timer at the given
792+to arvo to be handled. This cancels the timer at the given duct (with the given
835−duct (with the given time).
793+time).
836
794
837 ++ mabe :: maybe fire function
795 ++ mabe :: maybe fire function
838 |* [rov=rove fun=$+(@da _+>.^$)]
796 |* [rov=rove fun=$+(@da _+>.^$)]
@@ -853,15 +811,15 @@ duct (with the given time).
853 ?.((lth now p.q.mot) [~ now] [~ p.q.mot])
811 ?.((lth now p.q.mot) [~ now] [~ p.q.mot])
854 ==
812 ==
855
813
856−This decides whether the given request can only be satsified in the
814+This decides whether the given request can only be satsified in the future. In
857−future. In that case, we call the given function with the time in the
815+that case, we call the given function with the time in the future when we expect
858−future when we expect to have an update to give to this request. This is
816+to have an update to give to this request. This is called with `++best` to
859−called with `++best` to cancel timers and with `++bait` to start them.
817+cancel timers and with `++bait` to start them.
860
818
861−For single requests, we have a time if the request is for a particular
819+For single requests, we have a time if the request is for a particular time
862−time (which is assumed to be in the future). For ranges of requests, we
820+(which is assumed to be in the future). For ranges of requests, we check both
863−check both the start and end cases to see if they are time cases. If so,
821+the start and end cases to see if they are time cases. If so, we choose the
864−we choose the earlier time.
822+earlier time.
865
823
866If any of those give us a time, then we call the given funciton with the
824If any of those give us a time, then we call the given funciton with the
867smallest time.
825smallest time.
@@ -884,22 +842,22 @@ subscription but starting one.
884 (blub hen)
842 (blub hen)
885 (blab hen p.rav u.u.ver)
843 (blab hen p.rav u.u.ver)
886
844
887−There are two types of subscriptions -- either we're requesting a single
845+There are two types of subscriptions -- either we're requesting a single file or
888−file or we're requesting a range of versions of a desk. We'll dicuss the
846+we're requesting a range of versions of a desk. We'll dicuss the simpler case
889−simpler case first.
847+first.
890
848
891−First, we check that we're not requesting the `rang` from any time other
849+First, we check that we're not requesting the `rang` from any time other than
892−than the present. Since we don't store that information for any other
850+the present. Since we don't store that information for any other time, we can't
893−time, we can't produce it in a referentially transparent manner for any
851+produce it in a referentially transparent manner for any time other than the
894−time other than the present.
852+present.
895
853
896−Then, we try to read the requested `mood` `p.rav`. If we can't access
854+Then, we try to read the requested `mood` `p.rav`. If we can't access the
897−the request data right now, we call `++duce` to put the request in our
855+request data right now, we call `++duce` to put the request in our queue to be
898−queue to be satisfied when the information becomes available.
856+satisfied when the information becomes available.
899
857
900−This case occurs when we make a request for a case whose (1) date is
858+This case occurs when we make a request for a case whose (1) date is after the
901−after the current date, (2) number is after the current number, or (3)
859+current date, (2) number is after the current number, or (3) label is not yet
902−label is not yet used.
860+used.
903
861
904 ++ duce :: produce request
862 ++ duce :: produce request
905 |= [hen=duct rov=rove]
863 |= [hen=duct rov=rove]
@@ -920,43 +878,41 @@ label is not yet used.
920 fod.u.ref (~(put by fod.u.ref) hen inx)
878 fod.u.ref (~(put by fod.u.ref) hen inx)
921 ==
879 ==
922
880
923−The code for `++duce` is nearly the exact inverse of `++ease`, which in
881+The code for `++duce` is nearly the exact inverse of `++ease`, which in the case
924−the case of a domestic desk is very simple -- we simply put the duct and
882+of a domestic desk is very simple -- we simply put the duct and rave into `qyx`
925−rave into `qyx` and possibly start a timer with `++mabe` and `++bait`.
883+and possibly start a timer with `++mabe` and `++bait`. Recall that `ref` is null
926−Recall that `ref` is null for domestic desks and that `++mabe` fires the
884+for domestic desks and that `++mabe` fires the given function with the time we
927−given function with the time we need to be woken up at, if we need to be
928−woken up at a particular time.
885+need to be woken up at, if we need to be woken up at a particular time.
929
886
930 ++ bait
887 ++ bait
931 |= [hen=duct tym=@da]
888 |= [hen=duct tym=@da]
932 %_(+> tag :_(tag [hen /tyme %t %wait tym]))
889 %_(+> tag :_(tag [hen /tyme %t %wait tym]))
933
890
934−This sets an alarm by sending a `%wait` card with the given time to the
891+This sets an alarm by sending a `%wait` card with the given time to the timer
935−timer vane.
892+vane.
936
893
937−Back in `++eave`, if `++aver` returned `[~ ~]`, then we cancel the
894+Back in `++eave`, if `++aver` returned `[~ ~]`, then we cancel the subscription.
938−subscription. This occurs when we make (1) a `%x` request for a file
895+This occurs when we make (1) a `%x` request for a file that does not exist, (2)
939−that does not exist, (2) a `%w` request with a case that is not a
896+a `%w` request with a case that is not a number, or (3) a `%w` request with a
940−number, or (3) a `%w` request with a nonempty path. The `++blub` is
941−exactly what you would expect it to be.
897+nonempty path. The `++blub` is exactly what you would expect it to be.
942
898
943 ++ blub :: ship stop
899 ++ blub :: ship stop
944 |= hen=duct
900 |= hen=duct
945 %_(+> byn [[hen ~] byn])
901 %_(+> byn [[hen ~] byn])
946
902
947−We notify the duct that we're cancelling their subscription since it
903+We notify the duct that we're cancelling their subscription since it isn't
948−isn't satisfiable.
904+satisfiable.
949
905
950−Otherwise, we have received the desired information, so we send it on to
906+Otherwise, we have received the desired information, so we send it on to the
951−the subscriber with `++blab`.
907+subscriber with `++blab`.
952
908
953 ++ blab :: ship result
909 ++ blab :: ship result
954 |= [hen=duct mun=mood dat=*]
910 |= [hen=duct mun=mood dat=*]
955 ^+ +>
911 ^+ +>
956 +>(byn [[hen ~ [p.mun q.mun syd] r.mun dat] byn])
912 +>(byn [[hen ~ [p.mun q.mun syd] r.mun dat] byn])
957
913
958−The most interesting arm called in `++eave` is, of course, `++aver`,
914+The most interesting arm called in `++eave` is, of course, `++aver`, where we
959−where we actually try to read the data.
915+actually try to read the data.
960
916
961 ++ aver :: read
917 ++ aver :: read
962 |= mun=mood
918 |= mun=mood
@@ -969,19 +925,19 @@ where we actually try to read the data.
969 =+ nao=(~(case-to-aeon ze lim dom ran) q.mun)
925 =+ nao=(~(case-to-aeon ze lim dom ran) q.mun)
970 ?~(nao ~ [~ (~(read-at-aeon ze lim dom ran) u.nao mun)])
926 ?~(nao ~ [~ (~(read-at-aeon ze lim dom ran) u.nao mun)])
971
927
972−We check immediately that we're not requesting the `rang` for any time
928+We check immediately that we're not requesting the `rang` for any time other
973−other than the present.
929+than the present.
974
930
975−If this is a foreign desk, then we check our cache for the specific
931+If this is a foreign desk, then we check our cache for the specific request. If
976−request. If either this is a domestic desk or we don't have the request
932+either this is a domestic desk or we don't have the request in our cache, then
977−in our cache, then we have to actually go read the data from our dome.
933+we have to actually go read the data from our dome.
978
934
979We need to do two things. First, we try to find the number of the commit
935We need to do two things. First, we try to find the number of the commit
980specified by the given case, and then we try to get the data there.
936specified by the given case, and then we try to get the data there.
981
937
982−Here, we jump into `arvo/zuse.hoon`, which is where much of the
938+Here, we jump into `arvo/zuse.hoon`, which is where much of the algorithmic code
983−algorithmic code is stored, as opposed to the clay interface, which is
939+is stored, as opposed to the clay interface, which is stored in
984−stored in `arvo/clay.hoon`. We examine `++case-to-aeon:ze`.
940+`arvo/clay.hoon`. We examine `++case-to-aeon:ze`.
985
941
986 ++ case-to-aeon :: case-to-aeon:ze
942 ++ case-to-aeon :: case-to-aeon:ze
987 |= lok=case :: act count through
943 |= lok=case :: act count through
@@ -1003,22 +959,20 @@ stored in `arvo/clay.hoon`. We examine `++case-to-aeon:ze`.
1003 %ud ?:((gth p.lok let) ~ [~ p.lok])
959 %ud ?:((gth p.lok let) ~ [~ p.lok])
1004 ==
960 ==
1005
961
1006−We handle each type of `case` differently. The latter two types are
962+We handle each type of `case` differently. The latter two types are easy.
1007−easy.
1008
963
1009−If we're requesting a revision by label, then we simply look up the
964+If we're requesting a revision by label, then we simply look up the requested
1010−requested label in `lab` from the given dome. If it exists, that is our
965+label in `lab` from the given dome. If it exists, that is our aeon; else we
1011−aeon; else we produce null, indicating the requested revision does not
966+produce null, indicating the requested revision does not yet exist.
1012−yet exist.
1013
967
1014−If we're requesting a revision by number, we check if we've yet reached
968+If we're requesting a revision by number, we check if we've yet reached that
1015−that number. If so, we produce the number; else we produce null.
969+number. If so, we produce the number; else we produce null.
1016
970
1017−If we're requesting a revision by date, we check first if the date is in
971+If we're requesting a revision by date, we check first if the date is in the
1018−the future, returning null if so. Else we start from the most recent
972+future, returning null if so. Else we start from the most recent revision and
1019−revision and scan backwards until we find the first revision committed
973+scan backwards until we find the first revision committed before that date, and
1020−before that date, and we produce that. If we requested a date before any
974+we produce that. If we requested a date before any revisions were committed, we
1021−revisions were committed, we produce `0`.
975+produce `0`.
1022
976
1023The definitions of `++aeon-to-tako` and `++tako-to-yaki` are trivial.
977The definitions of `++aeon-to-tako` and `++tako-to-yaki` are trivial.
1024
978
@@ -1026,12 +980,10 @@ The definitions of `++aeon-to-tako` and `++tako-to-yaki` are trivial.
1026
980
1027 ++ tako-to-yaki ~(got by hut) :: grab yaki
981 ++ tako-to-yaki ~(got by hut) :: grab yaki
1028
982
1029−We simply look up the aeon or tako in their respective maps (`hit` and
983+We simply look up the aeon or tako in their respective maps (`hit` and `hut`).
1030−`hut`).
1031
984
1032−Assuming we got a valid version number, `++aver` calls
985+Assuming we got a valid version number, `++aver` calls `++read-at-aeon:ze`,
1033−`++read-at-aeon:ze`, which reads the requested data at the given
986+which reads the requested data at the given revision.
1034−revision.
1035
987
1036 ++ read-at-aeon :: read-at-aeon:ze
988 ++ read-at-aeon :: read-at-aeon:ze
1037 |= [oan=aeon mun=mood] :: seek and read
989 |= [oan=aeon mun=mood] :: seek and read
@@ -1040,10 +992,10 @@ revision.
1040 ?^(r.mun ~ [~ oan])
992 ?^(r.mun ~ [~ oan])
1041 (read:(rewind oan) mun)
993 (read:(rewind oan) mun)
1042
994
1043−If we're requesting the revision number with a case other than by
995+If we're requesting the revision number with a case other than by number, then
1044−number, then we go ahead and just produce the number we were given.
996+we go ahead and just produce the number we were given. Otherwise, we call
1045−Otherwise, we call `++rewind` to rewind our state to the given revision,
997+`++rewind` to rewind our state to the given revision, and then we call `++read`
1046−and then we call `++read` to get the requested information.
998+to get the requested information.
1047
999
1048 ++ rewind :: rewind:ze
1000 ++ rewind :: rewind:ze
1049 |= oan=aeon :: rewind to aeon
1001 |= oan=aeon :: rewind to aeon
@@ -1052,14 +1004,13 @@ and then we call `++read` to get the requested information.
1052 ?: (gth oan let) !! :: don't have version
1004 ?: (gth oan let) !! :: don't have version
1053 +>(ank (checkout-ankh q:(tako-to-yaki (aeon-to-tako oan))), let oan)
1005 +>(ank (checkout-ankh q:(tako-to-yaki (aeon-to-tako oan))), let oan)
1054
1006
1055−If we're already at the requested version, we do nothing. If we're
1007+If we're already at the requested version, we do nothing. If we're requesting a
1056−requesting a version later than our head, we are unable to comply.
1008+version later than our head, we are unable to comply.
1057
1009
1058−Otherwise, we get the hash of the commit at the number, and from that we
1010+Otherwise, we get the hash of the commit at the number, and from that we get the
1059−get the commit itself (the yaki), which has the map of path to lobe that
1011+commit itself (the yaki), which has the map of path to lobe that represents a
1060−represents a version of the filesystem. We call `++checkout-ankh` to
1012+version of the filesystem. We call `++checkout-ankh` to checkout the commit, and
1061−checkout the commit, and we replace `ank` in our context with the
1013+we replace `ank` in our context with the result.
1062−result.
1063
1014
1064 ++ checkout-ankh :: checkout-ankh:ze
1015 ++ checkout-ankh :: checkout-ankh:ze
1065 |= hat=(map path lobe) :: checkout commit
1016 |= hat=(map path lobe) :: checkout commit
@@ -1078,9 +1029,8 @@ result.
1078 $(pat t.pat, ank (fall nak _ankh))
1029 $(pat t.pat, ank (fall nak _ankh))
1079 ==
1030 ==
1080
1031
1081−Twice we call `++cosh`, which hashes a commit, updating `p` in an
1032+Twice we call `++cosh`, which hashes a commit, updating `p` in an `ankh`. Let's
1082−`ankh`. Let's jump into that algorithm before we describe
1033+jump into that algorithm before we describe `++checkout-ankh`.
1083−`++checkout-ankh`.
1084
1034
1085 ++ cosh :: locally rehash
1035 ++ cosh :: locally rehash
1086 |= ank=ankh :: NB v/unix.c
1036 |= ank=ankh :: NB v/unix.c
@@ -1106,33 +1056,30 @@ We simply replace `p` in the hash with the `cash` we get from a call to
1106 $(r.ank r.r.ank, axe (peg axe 3))
1056 $(r.ank r.r.ank, axe (peg axe 3))
1107 ==
1057 ==
1108
1058
1109−`++zu` is a core we set up with a particular filesystem node to traverse
1059+`++zu` is a core we set up with a particular filesystem node to traverse a
1110−a checkout of the filesystem and access the actual data inside it. One
1060+checkout of the filesystem and access the actual data inside it. One of the
1111−of the things we can do with it is to create a recursive hash of the
1061+things we can do with it is to create a recursive hash of the node.
1112−node.
1113
1062
1114−In `++rehash`, if this node is a file, then we xor the remainder of the
1063+In `++rehash`, if this node is a file, then we xor the remainder of the hash
1115−hash with the hash of the contents of the file. The remainder of the
1064+with the hash of the contents of the file. The remainder of the hash is `0` if
1116−hash is `0` if we have no children, else we descend into our children.
1065+we have no children, else we descend into our children. Basically, we do a half
1117−Basically, we do a half SHA-256 of the xor of the axis of this child and
1066+SHA-256 of the xor of the axis of this child and the half SHA-256 of the xor of
1118−the half SHA-256 of the xor of the name of the child and the hash of the
1067+the name of the child and the hash of the child. This is done for each child and
1119−child. This is done for each child and all the results are xored
1068+all the results are xored together.
1120−together.
1121
1069
1122Now we return to our discussion of `++checkout-ankh`.
1070Now we return to our discussion of `++checkout-ankh`.
1123
1071
1124−We fold over every path in this version of the filesystem and create a
1072+We fold over every path in this version of the filesystem and create a great
1125−great ankh out of them. First, we call `++lobe-to-noun` to get the raw
1073+ankh out of them. First, we call `++lobe-to-noun` to get the raw data referred
1126−data referred to be each lobe.
1074+to be each lobe.
1127
1075
1128 ++ lobe-to-noun :: grab blob
1076 ++ lobe-to-noun :: grab blob
1129 |= p=lobe :: ^- *
1077 |= p=lobe :: ^- *
1130 %- blob-to-noun
1078 %- blob-to-noun
1131 (lobe-to-blob p)
1079 (lobe-to-blob p)
1132
1080
1133−This converts a lobe into the raw data it refers to by first getting the
1081+This converts a lobe into the raw data it refers to by first getting the blob
1134−blob with `++lobe-to-blob` and converting that into data with
1082+with `++lobe-to-blob` and converting that into data with `++blob-to-noun`.
1135−`++blob-to-noun`.
1136
1083
1137 ++ lobe-to-blob ~(got by lat) :: grab blob
1084 ++ lobe-to-blob ~(got by lat) :: grab blob
1138
1085
@@ -1146,10 +1093,10 @@ This just grabs the blob that the lobe refers to.
1146 %indirect q.p
1093 %indirect q.p
1147 ==
1094 ==
1148
1095
1149−If we have either a direct or an indirect blob, then the data is stored
1096+If we have either a direct or an indirect blob, then the data is stored right in
1150−right in the blob. Otherwise, we have to reconstruct it from the diffs.
1097+the blob. Otherwise, we have to reconstruct it from the diffs. We do this by
1151−We do this by calling `++lump` on the diff in the blob with the data
1098+calling `++lump` on the diff in the blob with the data obtained by recursively
1152−obtained by recursively calling the parent of this blob.
1099+calling the parent of this blob.
1153
1100
1154 ++ lump :: apply patch
1101 ++ lump :: apply patch
1155 |= [don=udon src=*]
1102 |= [don=udon src=*]
@@ -1171,13 +1118,12 @@ obtained by recursively calling the parent of this blob.
1171 ==
1118 ==
1172 ==
1119 ==
1173
1120
1174−This is defined in `arvo/hoon.hoon` for historical reasons which are
1121+This is defined in `arvo/hoon.hoon` for historical reasons which are likely no
1175−likely no longer applicable. Since the `++umph` structure will likely
1122+longer applicable. Since the `++umph` structure will likely change we convert
1176−change we convert clay to be a typed filesystem, we'll only give a
1123+clay to be a typed filesystem, we'll only give a high-level description of this
1177−high-level description of this process. If we have a `%a` udon, then
1178−we're performing a trivial replace, so we produce simply `q.q.don`. If
1124+process. If we have a `%a` udon, then we're performing a trivial replace, so we
1179−we have a `%c` udon, then we're performing a list merge (as in, for
1125+produce simply `q.q.don`. If we have a `%c` udon, then we're performing a list
1180−example, lines of text). The merge is performed by `++lurk`.
1126+merge (as in, for example, lines of text). The merge is performed by `++lurk`.
1181
1127
1182 ++ lurk :: apply list patch
1128 ++ lurk :: apply list patch
1183 |* [hel=(list) rug=(urge)]
1129 |* [hel=(list) rug=(urge)]
@@ -1204,17 +1150,16 @@ example, lines of text). The merge is performed by `++lurk`.
1204 ==
1150 ==
1205 ==
1151 ==
1206
1152
1207−We accumulate our final result in `war`. If there's nothing more in our
1153+We accumulate our final result in `war`. If there's nothing more in our list of
1208−list of merge instructions (unces), we just reverse `war` and produce
1154+merge instructions (unces), we just reverse `war` and produce it. Otherwise, we
1209−it. Otherwise, we process another unce. If the unce is of type `&`, then
1155+process another unce. If the unce is of type `&`, then we have `p.i.rug` lines
1210−we have `p.i.rug` lines of no changes, so we just copy them over from
1156+of no changes, so we just copy them over from `hel` to `war`. If the unice is of
1211−`hel` to `war`. If the unice is of type `|`, then we verify that the
1212−source lines (in `hel`) are what we expect them to be (`p.i.rug`),
1157+type `|`, then we verify that the source lines (in `hel`) are what we expect
1213−crashing on failure. If they're good, then we append the new lines in
1158+them to be (`p.i.rug`), crashing on failure. If they're good, then we append the
1214−`q.i.rug` onto `war`.
1159+new lines in `q.i.rug` onto `war`.
1215
1160
1216−And that's really it. List merges are pretty easy. Anyway, if you
1161+And that's really it. List merges are pretty easy. Anyway, if you recall, we
1217−recall, we were discussing `++checkout-ankh`.
1162+were discussing `++checkout-ankh`.
1218
1163
1219 ++ checkout-ankh :: checkout-ankh:ze
1164 ++ checkout-ankh :: checkout-ankh:ze
1220 |= hat=(map path lobe) :: checkout commit
1165 |= hat=(map path lobe) :: checkout commit
@@ -1233,16 +1178,16 @@ recall, we were discussing `++checkout-ankh`.
1233 $(pat t.pat, ank (fall nak _ankh))
1178 $(pat t.pat, ank (fall nak _ankh))
1234 ==
1179 ==
1235
1180
1236−If the path is null, then we calculate `zar`, the raw data at the path
1181+If the path is null, then we calculate `zar`, the raw data at the path `pat` in
1237−`pat` in this version. We produce the given ankh with the correct data.
1182+this version. We produce the given ankh with the correct data.
1238
1183
1239−Otherwise, we try to get the child we're looking at from our parent
1184+Otherwise, we try to get the child we're looking at from our parent ankh. If
1240−ankh. If it's already been created, this succeeds; otherwise, we simply
1185+it's already been created, this succeeds; otherwise, we simply create a default
1241−create a default blank ankh. We place ourselves in our parent after
1186+blank ankh. We place ourselves in our parent after recursively computing our
1242−recursively computing our children.
1187+children.
1243
1188
1244−This algorithm really isn't that complicated, but it may not be
1189+This algorithm really isn't that complicated, but it may not be immediately
1245−immediately obvious why it works. An example should clear everything up.
1190+obvious why it works. An example should clear everything up.
1246
1191
1247Suppose `hat` is a map of the following information.
1192Suppose `hat` is a map of the following information.
1248
1193
@@ -1262,63 +1207,60 @@ Furthermore, let's say that we process them in this order:
1262 /greeting/spanish
1207 /greeting/spanish
1263 /farewell/russian
1208 /farewell/russian
1264
1209
1265−Then, the first path we process is `/greeting/english` . Since our path
1210+Then, the first path we process is `/greeting/english` . Since our path is not
1266−is not null, we try to get `nak`, but because our ankh is blank at this
1211+null, we try to get `nak`, but because our ankh is blank at this point it
1267−point it doesn't find anything. Thus, update our blank top-level ankh
1212+doesn't find anything. Thus, update our blank top-level ankh with a child
1268−with a child `%greeting`. and recurse with the blank `nak` to create the
1213+`%greeting`. and recurse with the blank `nak` to create the ankh of the new
1269−ankh of the new child.
1214+child.
1270−
1215+
1271−In the recursion, we our path is `/english` and our ankh is again blank.
1216+In the recursion, we our path is `/english` and our ankh is again blank. We try
1272−We try to get the `english` child of our ankh, but this of course fails.
1217+to get the `english` child of our ankh, but this of course fails. Thus, we
1273−Thus, we update our blank `/greeting` ankh with a child `english`
1218+update our blank `/greeting` ankh with a child `english` produced by recursing.
1274−produced by recursing.
1219+
1275−
1276−Now our path is null, so we call `++lobe-to-noun` to get the actual
1220+Now our path is null, so we call `++lobe-to-noun` to get the actual data, and we
1277−data, and we place it in the brand-new ankh.
1221+place it in the brand-new ankh.
1278−
1222+
1279−Next, we process `/greeting/russian/short`. Since our path is not null,
1223+Next, we process `/greeting/russian/short`. Since our path is not null, we try
1280−we try to get the child named `%greeting`, which does exist since we
1224+to get the child named `%greeting`, which does exist since we created it
1281−created it earlier. We put modify this child by recursing on it. Our
1225+earlier. We put modify this child by recursing on it. Our path is now
1282−path is now `/russian/short`, so we look for a `%russian` child in our
1226+`/russian/short`, so we look for a `%russian` child in our `/greeting` ankh.
1283−`/greeting` ankh. This doesn't exist, so we add it by recursing. Our
1227+This doesn't exist, so we add it by recursing. Our path is now `/short`, so we
1284−path is now `/short`, so we look for a `%short` child in our
1285−`/greeting/russian` ankh. This doesn't exist, so we add it by recursing.
1228+look for a `%short` child in our `/greeting/russian` ankh. This doesn't exist,
1286−Now our path is null, so we set the contents of this node to `"привет"`,
1229+so we add it by recursing. Now our path is null, so we set the contents of this
1287−and we're done processing this path.
1230+node to `"привет"`, and we're done processing this path.
1288−
1231+
1289−Next, we process `/greeting/russian/long`. This proceeds similarly to
1232+Next, we process `/greeting/russian/long`. This proceeds similarly to the
1290−the previous except that now the ankh for `/greeting/russian` already
1233+previous except that now the ankh for `/greeting/russian` already exists, so we
1291−exists, so we simply reuse it rather than creating a new one. Of course,
1234+simply reuse it rather than creating a new one. Of course, we still must create
1292−we still must create a new `/greeting/russian/long` ankh.
1235+a new `/greeting/russian/long` ankh.
1293−
1236+
1294−Next, we process `/greeting`. This ankh already exists, so after we've
1237+Next, we process `/greeting`. This ankh already exists, so after we've recursed
1295−recursed once, our path is null, and our ankh is not blank -- it already
1238+once, our path is null, and our ankh is not blank -- it already has two children
1296−has two children (and two grandchildren). We don't touch those, though,
1239+(and two grandchildren). We don't touch those, though, since a node may be both
1297−since a node may be both a file and a directory. We just add the
1240+a file and a directory. We just add the contents of the file -- "customary upon
1298−contents of the file -- "customary upon meeting" -- to the existing
1241+meeting" -- to the existing ankh.
1299−ankh.
1242+
1300−
1301−Next, we process `/greeting/spanish`. Of course, the `/greeting` ankh
1243+Next, we process `/greeting/spanish`. Of course, the `/greeting` ankh already
1302−already exists, but it doesn't have a `%spanish` child, so we create
1244+exists, but it doesn't have a `%spanish` child, so we create that, taking care
1303−that, taking care not to disturb the contents of the `/greeting` file.
1245+not to disturb the contents of the `/greeting` file. We put "hola" into the ankh
1304−We put "hola" into the ankh and call it good.
1246+and call it good.
1305−
1247+
1306−Finally, we process `/farewell/russian`. Here, the `/farewell` ankh
1248+Finally, we process `/farewell/russian`. Here, the `/farewell` ankh doesn't
1307−doesn't exist, so we create it. Clearly the newly-created ankh doesn't
1249+exist, so we create it. Clearly the newly-created ankh doesn't have any
1308−have any children, so we have to add a `%russian` child, and in this
1250+children, so we have to add a `%russian` child, and in this child we put our
1309−child we put our last content -- "до свидания".
1251+last content -- "до свидания".
1310−
1252+
1311−We hope it's fairly obvious that the order we process the paths doesn't
1253+We hope it's fairly obvious that the order we process the paths doesn't affect
1312−affect the final ankh tree. The tree will be constructed in a very
1254+the final ankh tree. The tree will be constructed in a very different order
1313−different order depending on what order the paths come in, but the
1255+depending on what order the paths come in, but the resulting tree is independent
1314−resulting tree is independent of order.
1256+of order.
1315−
1257+
1316−At any rate, we were talking about something important, weren't we? If
1258+At any rate, we were talking about something important, weren't we? If you
1317−you recall, that concludes our discussion of `++rewind`, which was
1259+recall, that concludes our discussion of `++rewind`, which was called from
1318−called from `++read-at-aeon`. In summary, `++rewind` returns a context
1260+`++read-at-aeon`. In summary, `++rewind` returns a context in which our current
1319−in which our current state is (very nearly) as it was when the specified
1261+state is (very nearly) as it was when the specified version of the desk was the
1320−version of the desk was the head. This allows `++read-at-aeon` to call
1262+head. This allows `++read-at-aeon` to call `++read` to read the requested
1321−`++read` to read the requested information.
1263+information.
1322
1264
1323 ++ read :: read:ze
1265 ++ read :: read:ze
1324 |= mun=mood :: read at point
1266 |= mun=mood :: read at point
@@ -1338,14 +1280,13 @@ version of the desk was the head. This allows `++read-at-aeon` to call
1338
1280
1339If we're requesting the dome, then we just return that immediately.
1281If we're requesting the dome, then we just return that immediately.
1340
1282
1341−If we're requesting the revision number of the desk and we're not
1283+If we're requesting the revision number of the desk and we're not requesting it
1342−requesting it by number, then we just return the current number of this
1284+by number, then we just return the current number of this desk. Note of course
1343−desk. Note of course that this was really already handled in
1285+that this was really already handled in `++read-at-aeon`.
1344−`++read-at-aeon`.
1345
1286
1346−If we're requesting a `%w` with a specific revision number, then we do
1287+If we're requesting a `%w` with a specific revision number, then we do something
1347−something or other with the commit there. It's kind of weird, and it
1288+or other with the commit there. It's kind of weird, and it doesn't seem to work,
1348−doesn't seem to work, so we'll ignore this case.
1289+so we'll ignore this case.
1349
1290
1350Otherwise, we descend into the ankh tree to the given path with
1291Otherwise, we descend into the ankh tree to the given path with
1351`++descend-path:zu`, and then we handle specific request in `++query`.
1292`++descend-path:zu`, and then we handle specific request in `++query`.
@@ -1355,8 +1296,8 @@ Otherwise, we descend into the ankh tree to the given path with
1355 ^+ +>
1296 ^+ +>
1356 ?~(way +> $(way t.way, +> (descend i.way)))
1297 ?~(way +> $(way t.way, +> (descend i.way)))
1357
1298
1358−This is simple recursion down into the ankh tree. `++descend` descends
1299+This is simple recursion down into the ankh tree. `++descend` descends one
1359−one level, so this will eventually get us down to the path we want.
1300+level, so this will eventually get us down to the path we want.
1360
1301
1361 ++ descend :: descend
1302 ++ descend :: descend
1362 |= lol=@ta
1303 |= lol=@ta
@@ -1364,12 +1305,12 @@ one level, so this will eventually get us down to the path we want.
1364 =+ you=(~(get by r.ank) lol)
1305 =+ you=(~(get by r.ank) lol)
1365 +>.$(ram [lol ram], ank ?~(you [*cash ~ ~] u.you))
1306 +>.$(ram [lol ram], ank ?~(you [*cash ~ ~] u.you))
1366
1307
1367−`ram` is the path that we're at, so to descend one level we push the
1308+`ram` is the path that we're at, so to descend one level we push the name of
1368−name of this level onto that path. We update the ankh with the correct
1309+this level onto that path. We update the ankh with the correct one at that path
1369−one at that path if it exists; else we create a blank one.
1310+if it exists; else we create a blank one.
1370
1311
1371−Once we've decscended to the correct level, we need to actually deal
1312+Once we've decscended to the correct level, we need to actually deal with the
1372−with the request.
1313+request.
1373
1314
1374 ++ query :: query:ze
1315 ++ query :: query:ze
1375 |= ren=?(%u %v %x %y %z) :: endpoint query
1316 |= ren=?(%u %v %x %y %z) :: endpoint query
@@ -1382,10 +1323,9 @@ with the request.
1382 %z [~ ank]
1323 %z [~ ank]
1383 ==
1324 ==
1384
1325
1385−Now that everything's set up, it's really easy. If they're requesting
1326+Now that everything's set up, it's really easy. If they're requesting the rang,
1386−the rang, dome, or ankh, we give it to them. If the contents of a file,
1327+dome, or ankh, we give it to them. If the contents of a file, we give it to them
1387−we give it to them if it is in fact a file. If the `arch`, then we
1328+if it is in fact a file. If the `arch`, then we calculate it with `++as-arch`.
1388−calculate it with `++as-arch`.
1389
1329
1390 ++ as-arch :: as-arch:ze
1330 ++ as-arch :: as-arch:ze
1391 ^- arch :: arch report
1331 ^- arch :: arch report
@@ -1395,17 +1335,16 @@ calculate it with `++as-arch`.
1395 ?~ r.ank ~
1335 ?~ r.ank ~
1396 [[p.n.r.ank ~] $(r.ank l.r.ank) $(r.ank r.r.ank)]
1336 [[p.n.r.ank ~] $(r.ank l.r.ank) $(r.ank r.r.ank)]
1397
1337
1398−This very simply strips out all the "real" data and returns just our own
1338+This very simply strips out all the "real" data and returns just our own hash,
1399−hash, the hash of the file contents (if we're a file), and a map of the
1339+the hash of the file contents (if we're a file), and a map of the names of our
1400−names of our immediate children.
1340+immediate children.
1401
1341
1402−Lifecycle of a Local Subscription
1342+## Lifecycle of a Local Subscription
1403−---------------------------------
1404
1343
1405−A subscription to a range of revisions of a desk initially follows the
1344+A subscription to a range of revisions of a desk initially follows the same path
1406−same path that a single read does. In `++aver`, we checked the head of
1345+that a single read does. In `++aver`, we checked the head of the given rave. If
1407−the given rave. If the head was `&`, then it was a single request, so we
1346+the head was `&`, then it was a single request, so we handled it above. If `|`,
1408−handled it above. If `|`, then we handle it with the following code.
1347+then we handle it with the following code.
1409
1348
1410 =+ nab=(~(case-to-aeon ze lim dom ran) p.p.rav)
1349 =+ nab=(~(case-to-aeon ze lim dom ran) p.p.rav)
1411 ?~ nab
1350 ?~ nab
@@ -1429,51 +1368,46 @@ handled it above. If `|`, then we handle it with the following code.
1429 ==
1368 ==
1430
1369
1431Recall that `++case-to-aeon:ze` produces the revision number that a case
1370Recall that `++case-to-aeon:ze` produces the revision number that a case
1432−corresponds to, if it corresponds to any. If it doesn't yet correspond
1371+corresponds to, if it corresponds to any. If it doesn't yet correspond to a
1433−to a revision, then it produces null.
1372+revision, then it produces null.
1434−
1373+
1435−Thus, we first check to see if we've even gotten to the beginning of the
1374+Thus, we first check to see if we've even gotten to the beginning of the range
1436−range of revisions requested. If not, then we assert that we haven't yet
1375+of revisions requested. If not, then we assert that we haven't yet gotten to the
1437−gotten to the end of the range either, because that would be really
1376+end of the range either, because that would be really strange. If not, then we
1438−strange. If not, then we immediately call `++duce`, which, if you
1377+immediately call `++duce`, which, if you recall, for a local request, simply
1439−recall, for a local request, simply puts this duct and rove into our
1440−cult `qyx`, so that we know who to respond to when the revision does
1378+puts this duct and rove into our cult `qyx`, so that we know who to respond to
1441−appear.
1379+when the revision does appear.
1442−
1380+
1443−If we've already gotten to the first revision, then we can produce some
1381+If we've already gotten to the first revision, then we can produce some content
1444−content immediately. If we've also gotten to the final revision, and
1382+immediately. If we've also gotten to the final revision, and that revision is
1445−that revision is earlier than the start revision, then it's a bad
1383+earlier than the start revision, then it's a bad request and we call `++blub`,
1446−request and we call `++blub`, which tells the subscriber that his
1447−subscription will not be satisfied.
1384+which tells the subscriber that his subscription will not be satisfied.
1448
1385
1449Otherwise, we find the data at the given path at the beginning of the
1386Otherwise, we find the data at the given path at the beginning of the
1450−subscription and at the last available revision in the subscription. If
1387+subscription and at the last available revision in the subscription. If they're
1451−they're the same, then we don't send a notification. Otherwise, we call
1388+the same, then we don't send a notification. Otherwise, we call `++gack`, which
1452−`++gack`, which creates the `++nako` we need to produce. We call
1389+creates the `++nako` we need to produce. We call `++bleb` to actually produce
1453−`++bleb` to actually produce the information.
1390+the information.
1454
1391
1455−If we already have the last requested revision, then we also tell the
1392+If we already have the last requested revision, then we also tell the subscriber
1456−subscriber with `++blub` that the subscription will receive no further
1393+with `++blub` that the subscription will receive no further updates.
1457−updates.
1458
1394
1459−If there will be more revisions in the subscription, then we call
1395+If there will be more revisions in the subscription, then we call `++duce`,
1460−`++duce`, adding the duct to our subscribers. We modify the rove to
1396+adding the duct to our subscribers. We modify the rove to start at the next
1461−start at the next revision since we've already handled all the revisions
1397+revision since we've already handled all the revisions up to the present.
1462−up to the present.
1463
1398
1464−We glossed over the calls to `++lobes-at-path`, `++make-nako`, and
1399+We glossed over the calls to `++lobes-at-path`, `++make-nako`, and `++bleb`, so
1465−`++bleb`, so we'll get back to those right now. `++bleb` is simple, so
1400+we'll get back to those right now. `++bleb` is simple, so we'll start with that.
1466−we'll start with that.
1467
1401
1468 ++ bleb :: ship sequence
1402 ++ bleb :: ship sequence
1469 |= [hen=duct ins=@ud hip=nako]
1403 |= [hen=duct ins=@ud hip=nako]
1470 ^+ +>
1404 ^+ +>
1471 (blab hen [%w [%ud ins] ~] hip)
1405 (blab hen [%w [%ud ins] ~] hip)
1472
1406
1473−We're given a duct, the beginning revision number, and the nako that
1407+We're given a duct, the beginning revision number, and the nako that contains
1474−contains the updates since that revision. We use `++blab` to produce
1408+the updates since that revision. We use `++blab` to produce this result to the
1475−this result to the subscriber. The case is `%w` with a revision number
1409+subscriber. The case is `%w` with a revision number of the beginning of the
1476−of the beginning of the subscription, and the data is the nako itself.
1410+subscription, and the data is the nako itself.
1477
1411
1478We call `++lobes-at-path:ze` to get the data at the particular path.
1412We call `++lobes-at-path:ze` to get the data at the particular path.
1479
1413
@@ -1496,15 +1430,14 @@ We call `++lobes-at-path:ze` to get the data at the particular path.
1496 $(p +.p, pax +.pax)
1430 $(p +.p, pax +.pax)
1497 == ==
1431 == ==
1498
1432
1499−At revision zero, the theoretical common revision between all
1433+At revision zero, the theoretical common revision between all repositories,
1500−repositories, there is no data, so we produce null.
1434+there is no data, so we produce null.
1501
1435
1502−We get the list of paths (paired with their lobe) in the revision
1436+We get the list of paths (paired with their lobe) in the revision referred to by
1503−referred to by the given number and we keep only those paths which begin
1437+the given number and we keep only those paths which begin with `pax`. Converting
1504−with `pax`. Converting to a map, we now have a map from the subpaths at
1438+to a map, we now have a map from the subpaths at the given path to the hash of
1505−the given path to the hash of their data. This is simple and efficient
1439+their data. This is simple and efficient to calculate and compare to later
1506−to calculate and compare to later revisions. This allows us to easily
1507−tell if a node or its children have changed.
1440+revisions. This allows us to easily tell if a node or its children have changed.
1508
1441
1509Finally, we will describe `++make-nako:ze`.
1442Finally, we will describe `++make-nako:ze`.
1510
1443
@@ -1521,19 +1454,17 @@ Finally, we will describe `++make-nako:ze`.
1521 |= [p=aeon *]
1454 |= [p=aeon *]
1522 &((gth p a) (lte p b))
1455 &((gth p a) (lte p b))
1523
1456
1524−We need to produce four things -- the numbers of the new commits, the
1457+We need to produce four things -- the numbers of the new commits, the number of
1525−number of the latest commit, the new commits themselves, and the new
1458+the latest commit, the new commits themselves, and the new data itself.
1526−data itself.
1527
1459
1528−The first is fairly easy to produce. We simply go over our map of
1460+The first is fairly easy to produce. We simply go over our map of numbered
1529−numbered commits and produce all those numbered greater than `a` and not
1461+commits and produce all those numbered greater than `a` and not greater than
1530−greater than `b`.
1462+`b`.
1531
1463
1532−The second is even easier to produce -- `b` is clearly our most recent
1464+The second is even easier to produce -- `b` is clearly our most recent commit.
1533−commit.
1534
1465
1535−The third and fourth are slightly more interesting, though not too
1466+The third and fourth are slightly more interesting, though not too terribly
1536−terribly difficult. First, we call `++reachable-between-takos`.
1467+difficult. First, we call `++reachable-between-takos`.
1537
1468
1538 ++ reachable-between-takos
1469 ++ reachable-between-takos
1539 |= [a=(unit tako) b=tako] :: pack a through b
1470 |= [a=(unit tako) b=tako] :: pack a through b
@@ -1557,12 +1488,12 @@ terribly difficult. First, we call `++reachable-between-takos`.
1557 %- ~(uni in bar)
1488 %- ~(uni in bar)
1558 ^$(yak (tako-to-yaki yek))
1489 ^$(yak (tako-to-yaki yek))
1559
1490
1560−We take a possible starting commit and a definite ending commit, and we
1491+We take a possible starting commit and a definite ending commit, and we produce
1561−produce the set of commits and the set of data between them.
1492+the set of commits and the set of data between them.
1562
1493
1563−We let `sar` be the set of commits reachable from `a`. If `a` is null,
1494+We let `sar` be the set of commits reachable from `a`. If `a` is null, then
1564−then obviously no commits are reachable. Otherwise, we call
1495+obviously no commits are reachable. Otherwise, we call `++reachable-takos` to
1565−`++reachable-takos` to calculate this.
1496+calculate this.
1566
1497
1567 ++ reachable-takos :: reachable
1498 ++ reachable-takos :: reachable
1568 |= p=tako :: XX slow
1499 |= p=tako :: XX slow
@@ -1575,17 +1506,16 @@ then obviously no commits are reachable. Otherwise, we call
1575 s :: hence skip
1506 s :: hence skip
1576 (~(uni in s) ^$(p q)) :: otherwise traverse
1507 (~(uni in s) ^$(p q)) :: otherwise traverse
1577
1508
1578−We very simply produce the set of the given tako plus its parents,
1509+We very simply produce the set of the given tako plus its parents, recursively.
1579−recursively.
1580
1510
1581−Back in `++reachable-between-takos`, we let `yak` be the yaki of `b`,
1511+Back in `++reachable-between-takos`, we let `yak` be the yaki of `b`, the ending
1582−the ending commit. With this, we create a set that is the union of `sar`
1512+commit. With this, we create a set that is the union of `sar` and all takos
1583−and all takos reachable from `b`.
1513+reachable from `b`.
1584
1514
1585−We pass `sar` into `++new-lobes` to get all the lobes referenced by any
1515+We pass `sar` into `++new-lobes` to get all the lobes referenced by any tako
1586−tako referenced by `a`. The result is passed into `++new-lobes-takos` to
1516+referenced by `a`. The result is passed into `++new-lobes-takos` to do the same,
1587−do the same, but not recomputing those in already calculated last
1517+but not recomputing those in already calculated last sentence. This produces the
1588−sentence. This produces the sets of takos and lobes we need.
1518+sets of takos and lobes we need.
1589
1519
1590 ++ new-lobes :: object hash set
1520 ++ new-lobes :: object hash set
1591 |= [b=(set lobe) a=(set tako)] :: that aren't in b
1521 |= [b=(set lobe) a=(set tako)] :: that aren't in b
@@ -1606,37 +1536,34 @@ sentence. This produces the sets of takos and lobes we need.
1606 %indirect (~(put in $(lob s.gar)) lob)
1536 %indirect (~(put in $(lob s.gar)) lob)
1607 ==
1537 ==
1608
1538
1609−Here, we're creating a set of lobes referenced in a commit in `a`. We
1539+Here, we're creating a set of lobes referenced in a commit in `a`. We start out
1610−start out with `b` as the initial set of lobes, so we don't need to
1540+with `b` as the initial set of lobes, so we don't need to recompute any of the
1611−recompute any of the lobes referenced in there.
1541+lobes referenced in there.
1612
1542
1613−The algorithm is pretty simple, so we won't bore you with the details.
1543+The algorithm is pretty simple, so we won't bore you with the details. We simply
1614−We simply traverse every commit in `a`, looking at every blob referenced
1544+traverse every commit in `a`, looking at every blob referenced there, and, if
1615−there, and, if it's not already in `b`, we add it to `b`. In the case of
1545+it's not already in `b`, we add it to `b`. In the case of a direct blob, we're
1616−a direct blob, we're done. For a delta or an indirect blob, we
1617−recursively add every blob referenced within the blob.
1546+done. For a delta or an indirect blob, we recursively add every blob referenced
1547+within the blob.
1618
1548
1619 ++ new-lobes-takos :: garg & repack
1549 ++ new-lobes-takos :: garg & repack
1620 |= [b=(set lobe) a=(set tako)]
1550 |= [b=(set lobe) a=(set tako)]
1621 ^- [(set tako) (set lobe)]
1551 ^- [(set tako) (set lobe)]
1622 [a (new-lobes b a)]
1552 [a (new-lobes b a)]
1623
1553
1624−Here, we just update the set of lobes we're given with the commits we're
1554+Here, we just update the set of lobes we're given with the commits we're given
1625−given and produce both sets.
1555+and produce both sets.
1626
1556
1627This concludes our discussion of a local subscription.
1557This concludes our discussion of a local subscription.
1628
1558
1629−Lifecycle of a Foreign Read or Subscription
1559+## Lifecycle of a Foreign Read or Subscription
1630−-------------------------------------------
1631
1560
1632−Foreign reads and subscriptions are handled in much the same way as
1561+Foreign reads and subscriptions are handled in much the same way as local ones.
1633−local ones. The interface is the same -- a vane or app sends a `%warp`
1562+The interface is the same -- a vane or app sends a `%warp` kiss with the
1634−kiss with the request. The difference is simply that the `sock` refers
1563+request. The difference is simply that the `sock` refers to the foreign ship.
1635−to the foreign ship.
1636
1564
1637−Thus, we start in the same place -- in `++call`, handling `%warp`.
1565+Thus, we start in the same place -- in `++call`, handling `%warp`. However,
1638−However, since the two side of the `sock` are different, we follow a
1566+since the two side of the `sock` are different, we follow a different path.
1639−different path.
1640
1567
1641 =+ wex=(do now p.q.hic p.q.q.hic ruf)
1568 =+ wex=(do now p.q.hic p.q.q.hic ruf)
1642 =+ ^= woo
1569 =+ ^= woo
@@ -1645,14 +1572,13 @@ different path.
1645 abet:(eave:wex hen u.q.q.q.hic)
1572 abet:(eave:wex hen u.q.q.q.hic)
1646 [-.woo (posh q.p.q.hic p.q.q.hic +.woo ruf)]
1573 [-.woo (posh q.p.q.hic p.q.q.hic +.woo ruf)]
1647
1574
1648−If we compare this to how the local case was handled, we see that it's
1575+If we compare this to how the local case was handled, we see that it's not all
1649−not all that different. We use `++do` rather than `++un` and `++de` to
1576+that different. We use `++do` rather than `++un` and `++de` to set up the core
1650−set up the core for the foreign ship. This gives us a `++de` core, so we
1577+for the foreign ship. This gives us a `++de` core, so we either cancel or begin
1651−either cancel or begin the request by calling `++ease` or `++eave`,
1578+the request by calling `++ease` or `++eave`, exactly as in the local case. In
1652−exactly as in the local case. In either case, we call `++abet:de` to
1653−resolve our various types of output into actual moves, as described in
1579+either case, we call `++abet:de` to resolve our various types of output into
1654−the local case. Finally, we call `++posh` to update our raft, putting
1580+actual moves, as described in the local case. Finally, we call `++posh` to
1655−the modified rung into the raft.
1581+update our raft, putting the modified rung into the raft.
1656
1582
1657We'll first trace through `++do`.
1583We'll first trace through `++do`.
1658
1584
@@ -1666,17 +1592,16 @@ We'll first trace through `++do`.
1666 ?^(yit u.yit `rede`[~2000.1.1 ~ [~ *rind] *dome])
1592 ?^(yit u.yit `rede`[~2000.1.1 ~ [~ *rind] *dome])
1667 ((de now ~ ~) [who him] syd red ran.ruf)
1593 ((de now ~ ~) [who him] syd red ran.ruf)
1668
1594
1669−If we already have a rung for this foreign ship, then we use that.
1595+If we already have a rung for this foreign ship, then we use that. Otherwise, we
1670−Otherwise, we create a new blank one. If we already have a rede in this
1596+create a new blank one. If we already have a rede in this rung, then we use
1671−rung, then we use that, otherwise we create a blank one. An important
1597+that, otherwise we create a blank one. An important point to note here is that
1672−point to note here is that we let `ref` in the rede be `[~ *rind]`.
1673−Recall, for domestic desks `ref` is null. We use this to distinguish
1598+we let `ref` in the rede be `[~ *rind]`. Recall, for domestic desks `ref` is
1674−between foreign and domestic desks in `++de`.
1599+null. We use this to distinguish between foreign and domestic desks in `++de`.
1675
1600
1676With this information, we create a `++de` core as usual.
1601With this information, we create a `++de` core as usual.
1677
1602
1678−Although we've already covered `++ease` and `++eave`, we'll go through
1603+Although we've already covered `++ease` and `++eave`, we'll go through them
1679−them quickly again, highlighting the case of foreign request.
1604+quickly again, highlighting the case of foreign request.
1680
1605
1681 ++ ease :: release request
1606 ++ ease :: release request
1682 |= hen=duct
1607 |= hen=duct
@@ -1695,20 +1620,19 @@ them quickly again, highlighting the case of foreign request.
1695 bom.u.ref (~(del by bom.u.ref) u.nux)
1620 bom.u.ref (~(del by bom.u.ref) u.nux)
1696 ==
1621 ==
1697
1622
1698−Here, we still remove the duct from our cult (we maintain a cult even
1623+Here, we still remove the duct from our cult (we maintain a cult even for
1699−for foreign desks), but we also need to tell the foreign desk to cancel
1624+foreign desks), but we also need to tell the foreign desk to cancel our
1700−our subscription. We do this by sending a request (by appending to
1625+subscription. We do this by sending a request (by appending to `say`, which gets
1701−`say`, which gets resolved in `++abet:de` to a `%want` kiss to ames) to
1626+resolved in `++abet:de` to a `%want` kiss to ames) to the foreign ship to cancel
1702−the foreign ship to cancel the subscription. Since we don't anymore
1627+the subscription. Since we don't anymore expect a response on this duct, we
1703−expect a response on this duct, we remove it from `fod` and `bom`, which
1704−are the maps between ducts, raves, and request sequence numbers.
1628+remove it from `fod` and `bom`, which are the maps between ducts, raves, and
1705−Basically, we remove every trace of the subscription from our request
1629+request sequence numbers. Basically, we remove every trace of the subscription
1706−manager.
1630+from our request manager.
1707−
1631+
1708−In the case of `++eave`, where we're creating a new request, everything
1632+In the case of `++eave`, where we're creating a new request, everything is
1709−is exactly identical to the case of the local request except `++duce`.
1633+exactly identical to the case of the local request except `++duce`. We said that
1710−We said that `++duce` simply puts the request into our cult. This is
1634+`++duce` simply puts the request into our cult. This is true for a domestic
1711−true for a domestic request, but distinctly untrue for foreign requests.
1635+request, but distinctly untrue for foreign requests.
1712
1636
1713 ++ duce :: produce request
1637 ++ duce :: produce request
1714 |= [hen=duct rov=rove]
1638 |= [hen=duct rov=rove]
@@ -1728,22 +1652,21 @@ true for a domestic request, but distinctly untrue for foreign requests.
1728 fod.u.ref (~(put by fod.u.ref) hen inx)
1652 fod.u.ref (~(put by fod.u.ref) hen inx)
1729 ==
1653 ==
1730
1654
1731−If we have a request manager (i.e. this is a foreign desk), then we do
1655+If we have a request manager (i.e. this is a foreign desk), then we do the
1732−the approximate inverse of `++ease`. We create a rave out of the given
1656+approximate inverse of `++ease`. We create a rave out of the given request and
1733−request and send it off to the foreign desk by putting it in `say`. Note
1657+send it off to the foreign desk by putting it in `say`. Note that the rave is
1734−that the rave is created to request the information starting at the next
1658+created to request the information starting at the next revision number. Since
1735−revision number. Since this is a new request, we put it into `fod` and
1659+this is a new request, we put it into `fod` and `bom` to associate the request
1736−`bom` to associate the request with its duct and its sequence number.
1660+with its duct and its sequence number. Since we're using another sequence
1737−Since we're using another sequence number, we must increment `nix`,
1738−which represents the next available sequence number.
1661+number, we must increment `nix`, which represents the next available sequence
1662+number.
1739
1663
1740And that's really it for this side of the request. Requesting foreign
1664And that's really it for this side of the request. Requesting foreign
1741−information isn't that hard. Let's see what it looks like on the other
1665+information isn't that hard. Let's see what it looks like on the other side.
1742−side. When we get a request from another ship for information on our
1666+When we get a request from another ship for information on our ship, that comes
1743−ship, that comes to us in the form of a `%wart` from ames.
1667+to us in the form of a `%wart` from ames.
1744
1668
1745−We handle a `%wart` in `++call`, right next to where we handle the
1669+We handle a `%wart` in `++call`, right next to where we handle the `%warp` case.
1746−`%warp` case.
1747
1670
1748 %wart
1671 %wart
1749 ?> ?=(%re q.q.hic)
1672 ?> ?=(%re q.q.hic)
@@ -1755,11 +1678,11 @@ We handle a `%wart` in `++call`, right next to where we handle the
1755 [%warp [p.p.q.hic p.p.q.hic] ryf]
1678 [%warp [p.p.q.hic p.p.q.hic] ryf]
1756 ==
1679 ==
1757
1680
1758−Every request we receive should be of type `riff`, so we coerce it into
1681+Every request we receive should be of type `riff`, so we coerce it into that
1759−that type. We just convert this into a new `%warp` kiss that we pass to
1682+type. We just convert this into a new `%warp` kiss that we pass to ourself. This
1760−ourself. This gets handled like normal, as a local request. When the
1683+gets handled like normal, as a local request. When the request produces a value,
1761−request produces a value, it does so like normal as a `%writ`, which is
1684+it does so like normal as a `%writ`, which is returned to `++take` along the
1762−returned to `++take` along the path we just sent it on.
1685+path we just sent it on.
1763
1686
1764 %writ
1687 %writ
1765 ?> ?=([@ @ *] tea)
1688 ?> ?=([@ @ *] tea)
@@ -1770,12 +1693,12 @@ returned to `++take` along the path we just sent it on.
1770 [%pass ~ %a [%want [our him] [%r %re %c t.t.tea] p.+.q.hin]]
1693 [%pass ~ %a [%want [our him] [%r %re %c t.t.tea] p.+.q.hin]]
1771 ==
1694 ==
1772
1695
1773−Since we encoded the ship we need to respond to in the path, we can just
1696+Since we encoded the ship we need to respond to in the path, we can just pass
1774−pass our `%want` back to ames, so that we tell the requesting ship about
1697+our `%want` back to ames, so that we tell the requesting ship about the new
1775−the new data.
1698+data.
1776
1699
1777−This comes back to the original ship as a `%waft` from ames, which comes
1700+This comes back to the original ship as a `%waft` from ames, which comes into
1778−into `++take`, right next to where we handled `%writ`.
1701+`++take`, right next to where we handled `%writ`.
1779
1702
1780 %waft
1703 %waft
1781 ?> ?=([@ @ ~] tea)
1704 ?> ?=([@ @ ~] tea)
@@ -1789,11 +1712,10 @@ into `++take`, right next to where we handled `%writ`.
1789 [-.zot (posh q.p.+.q.hin syd +.zot ruf)]
1712 [-.zot (posh q.p.+.q.hin syd +.zot ruf)]
1790 [mos ..^$(ran.ruf ran.zat)] :: merge in new obj
1713 [mos ..^$(ran.ruf ran.zat)] :: merge in new obj
1791
1714
1792−This gets the desk and sequence number from the path the request was
1715+This gets the desk and sequence number from the path the request was sent over.
1793−sent over. This determines exactly which request is being responded to.
1716+This determines exactly which request is being responded to. We call `++knit:de`
1794−We call `++knit:de` to apply the changes to our local desk, and we call
1717+to apply the changes to our local desk, and we call `++wake` to update our
1795−`++wake` to update our subscribers. Then we call `++abet:de` and
1718+subscribers. Then we call `++abet:de` and `++posh` as normal (like in `++eave`).
1796−`++posh` as normal (like in `++eave`).
1797
1719
1798We'll examine `++knit` and `++wake`, in that order.
1720We'll examine `++knit` and `++wake`, in that order.
1799
1721
@@ -1836,36 +1758,33 @@ We'll examine `++knit` and `++wake`, in that order.
1836
1758
1837This is kind of a long gate, but don't worry, it's not bad at all.
1759This is kind of a long gate, but don't worry, it's not bad at all.
1838
1760
1839−First, we assert that we're not a domestic desk. That wouldn't make any
1761+First, we assert that we're not a domestic desk. That wouldn't make any sense at
1840−sense at all.
1762+all.
1841−
1763+
1842−Since we have the sequence number of the request, we can get the duct
1764+Since we have the sequence number of the request, we can get the duct and rave
1843−and rave from `bom` in our request manager. If we didn't actually
1765+from `bom` in our request manager. If we didn't actually request this data (or
1844−request this data (or the request was canceled before we got it), then
1766+the request was canceled before we got it), then we do nothing.
1845−we do nothing.
1767+
1846−
1768+Else, we remove the request from `bom` and `fod` unless this was a subscription
1847−Else, we remove the request from `bom` and `fod` unless this was a
1769+request and we didn't receive a null riot (which would indicate the last message
1848−subscription request and we didn't receive a null riot (which would
1770+on the subscription).
1849−indicate the last message on the subscription).
1771+
1850−
1772+Now, if we received a null riot, then if this was a subscription request by
1851−Now, if we received a null riot, then if this was a subscription request
1773+date, then we update `lim` in our request manager (representing the latest time
1852−by date, then we update `lim` in our request manager (representing the
1774+at which we have complete information for this desk) to the date that was
1853−latest time at which we have complete information for this desk) to the
1775+requested. If this was a single read request, then we put the result in our
1854−date that was requested. If this was a single read request, then we put
1776+simple cache `haw` to make future requests faster. Then we're done.
1855−the result in our simple cache `haw` to make future requests faster.
1777+
1856−Then we're done.
1778+If we received actual data, then we put it into our cache `haw`. Unless it's a
1857−
1779+`%w` request, we're done.
1858−If we received actual data, then we put it into our cache `haw`. Unless
1780+
1859−it's a `%w` request, we're done.
1781+If it is a `%w` request, then we try to get the `%w` at our current head from
1860−
1782+the cache. In general, that should be the thing we just put in a moment ago, but
1861−If it is a `%w` request, then we try to get the `%w` at our current head
1783+that is not guaranteed. The most common case where this is different is when we
1862−from the cache. In general, that should be the thing we just put in a
1784+receive desk updates out of order. At any rate, since we now have new
1863−moment ago, but that is not guaranteed. The most common case where this
1785+information, we need to apply it to our local copy of the desk. We do so in
1864−is different is when we receive desk updates out of order. At any rate,
1786+`++edis`, and then we remove the stuff we just applied from the cache, since
1865−since we now have new information, we need to apply it to our local copy
1787+it's not really a true "single read", like what should really be in the cache.
1866−of the desk. We do so in `++edis`, and then we remove the stuff we just
1867−applied from the cache, since it's not really a true "single read", like
1868−what should really be in the cache.
1869
1788
1870 ++ edis :: apply subscription
1789 ++ edis :: apply subscription
1871 |= nak=nako
1790 |= nak=nako
@@ -1884,21 +1803,21 @@ what should really be in the cache.
1884 %+ ~(put by yeb) r.sar sar
1803 %+ ~(put by yeb) r.sar sar
1885 ==
1804 ==
1886
1805
1887−This shows, of course, exactly why nako is defined the way it is. To
1806+This shows, of course, exactly why nako is defined the way it is. To become
1888−become completely up to date with the foreign desk, we need to merge
1807+completely up to date with the foreign desk, we need to merge `hit` with the
1889−`hit` with the foreign one so that we have all the revision numbers. We
1808+foreign one so that we have all the revision numbers. We update `let` so that we
1890−update `let` so that we know which revision is the head.
1809+know which revision is the head.
1891
1810
1892−We merge the new blobs in `lat`, keying them by their hash, which we get
1811+We merge the new blobs in `lat`, keying them by their hash, which we get from a
1893−from a call to `++blob-to-lobe`. Recall that the hash is stored in the
1812+call to `++blob-to-lobe`. Recall that the hash is stored in the actual blob
1894−actual blob itself. We do the same thing to the new yakis, putting them
1813+itself. We do the same thing to the new yakis, putting them in `hut`, keyed by
1895−in `hut`, keyed by their hash.
1814+their hash.
1896
1815
1897Now, our local dome should be exactly the same as the foreign one.
1816Now, our local dome should be exactly the same as the foreign one.
1898
1817
1899−This concludes our discussion of `++knit`. Now the changes have been
1818+This concludes our discussion of `++knit`. Now the changes have been applied to
1900−applied to our local copy of the desk, and we just need to update our
1819+our local copy of the desk, and we just need to update our subscribers. We do so
1901−subscribers. We do so in `++wake:de`.
1820+in `++wake:de`.
1902
1821
1903 ++ wake :: update subscribers
1822 ++ wake :: update subscribers
1904 ^+ .
1823 ^+ .
@@ -1949,27 +1868,24 @@ subscribers. We do so in `++wake:de`.
1949 ==
1868 ==
1950 --
1869 --
1951
1870
1952−This is even longer than `++knit`, but it's pretty similar to `++eave`.
1871+This is even longer than `++knit`, but it's pretty similar to `++eave`. We loop
1953−We loop through each of our subscribers `xiq`, processing each in turn.
1872+through each of our subscribers `xiq`, processing each in turn. When we're done,
1954−When we're done, we just put the remaining subscribers back in our
1873+we just put the remaining subscribers back in our subscriber list.
1955−subscriber list.
1874+
1956−
1875+If the subscriber is a single read, then, if this is a foreign desk (note that
1957−If the subscriber is a single read, then, if this is a foreign desk
1876+`++wake` is called from other arms, and not only on foreign desks). Obviously,
1958−(note that `++wake` is called from other arms, and not only on foreign
1877+if we find an identical request there, then we can produce the result
1959−desks). Obviously, if we find an identical request there, then we can
1878+immediately. Referential transparency for the win. We produce the result with a
1960−produce the result immediately. Referential transparency for the win. We
1879+call to `++blab`. If this is a foreign desk but the result is not in the cache,
1961−produce the result with a call to `++blab`. If this is a foreign desk
1880+then we produce `++blub` (canceled subscription with no data) for reasons
1962−but the result is not in the cache, then we produce `++blub` (canceled
1881+entirely opaque to me. Seriously, it seems like we should wait until we get an
1963−subscription with no data) for reasons entirely opaque to me. Seriously,
1882+actual response to the request. If someone figures out why this is, let me know.
1964−it seems like we should wait until we get an actual response to the
1883+At any rate, it seems to work.
1965−request. If someone figures out why this is, let me know. At any rate,
1884+
1966−it seems to work.
1885+If this is a domestic desk, then we check to see if the case exists yet. If it
1967−
1886+doesn't, then we simply move on to the next subscriber, consing this one onto
1968−If this is a domestic desk, then we check to see if the case exists yet.
1887+`xaq` so that we can check again the next time around. If it does exist, then we
1969−If it doesn't, then we simply move on to the next subscriber, consing
1888+call `++balk` to fulfill the request and produce it.
1970−this one onto `xaq` so that we can check again the next time around. If
1971−it does exist, then we call `++balk` to fulfill the request and produce
1972−it.
1973
1889
1974`++balk` is very simple, so we'll describe it here before we get to the
1890`++balk` is very simple, so we'll describe it here before we get to the
1975subscription case.
1891subscription case.
@@ -1980,33 +1896,30 @@ subscription case.
1980 =+ vid=(~(read-at-aeon ze lim dom ran) oan mun)
1896 =+ vid=(~(read-at-aeon ze lim dom ran) oan mun)
1981 ?~ vid (blub hen) (blab hen mun u.vid)
1897 ?~ vid (blub hen) (blab hen mun u.vid)
1982
1898
1983−We call `++read-at-aeon` on the given request and aeon. If you recall,
1899+We call `++read-at-aeon` on the given request and aeon. If you recall, this
1984−this processes a `mood` at a particular aeon and produces the result, if
1900+processes a `mood` at a particular aeon and produces the result, if there is
1985−there is one. If there is data at the requested location, then we
1901+one. If there is data at the requested location, then we produce it with
1986−produce it with `++blab`. Else, we call `++blub` to notify the
1902+`++blab`. Else, we call `++blub` to notify the subscriber that no data can ever
1987−subscriber that no data can ever come over this subscriptioin since it
1903+come over this subscriptioin since it is now impossible for there to ever be
1988−is now impossible for there to ever be data for the given request.
1904+data for the given request. Because referential transparency.
1989−Because referential transparency.
1905+
1990−
1906+At any rate, back to `++wake`. If the given rave is a subscription request, then
1991−At any rate, back to `++wake`. If the given rave is a subscription
1907+we proceed similarly to how we do in `++eave`. We first try to get the aeon
1992−request, then we proceed similarly to how we do in `++eave`. We first
1908+referred to by the starting case. If it doesn't exist yet, then we can't do
1993−try to get the aeon referred to by the starting case. If it doesn't
1909+anything interesting with this subscription, so we move on to the next one.
1994−exist yet, then we can't do anything interesting with this subscription,
1910+
1995−so we move on to the next one.
1911+Otherwise, we try to get the aeon referred to by the ending case. If it doesn't
1996−
1912+exist yet, then we produce all the information we can. We call `++lobes-at-path`
1997−Otherwise, we try to get the aeon referred to by the ending case. If it
1913+at the given aeon and path to see if the requested path has actually changed. If
1998−doesn't exist yet, then we produce all the information we can. We call
1914+it hasn't, then we don't produce anything; else, we produce the correct nako by
1999−`++lobes-at-path` at the given aeon and path to see if the requested
1915+calling `++bleb` on the result of `++make-nako`, as in `++eave`. At any rate, we
2000−path has actually changed. If it hasn't, then we don't produce anything;
1916+move on to the next subscription, putting back into our cult the current
2001−else, we produce the correct nako by calling `++bleb` on the result of
1917+subscription with a new start case of the next aeon after the present.
2002−`++make-nako`, as in `++eave`. At any rate, we move on to the next
2003−subscription, putting back into our cult the current subscription with a
2004−new start case of the next aeon after the present.
2005
1918
2006If the aeon referred to by the ending case does exist, then we drop this
1919If the aeon referred to by the ending case does exist, then we drop this
2007−subscriber from our cult and satisfy its request immediately. This is
1920+subscriber from our cult and satisfy its request immediately. This is the same
2008−the same as before -- we check to see if the data at the path has
1921+as before -- we check to see if the data at the path has actually changed,
2009−actually changed, producing it if it has; else, we call `++blub` since
1922+producing it if it has; else, we call `++blub` since no more data can be
2010−no more data can be produced over this subscription.
1923+produced over this subscription.
2011
1924
2012This concludes our discussion of foreign requests.
1925This concludes our discussion of foreign requests.
docs/arvo/internals/dill.md+16−18
@@ -1,26 +1,24 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 4
3sort: 4
4---
4---
5
5
6−`%dill`
6+# `%dill`
7−=======
8
7
9Our terminal driver.
8Our terminal driver.
10
9
11−Unix sends keyboard events to `%dill` from either the console or telnet,
10+Unix sends keyboard events to `%dill` from either the console or telnet, and
12−and `%dill` produces terminal output. The only app that should directly
11+`%dill` produces terminal output. The only app that should directly talk to
13−talk to `%dill` is the terminal app. Command-line apps are run by,
12+`%dill` is the terminal app. Command-line apps are run by, receive input from,
14−receive input from, and produce output to, the `%shell` app, which is
13+and produce output to, the `%shell` app, which is controlled by `%terminal`,
15−controlled by `%terminal`, which talks to `%dill`, which talks to unix.
14+which talks to `%dill`, which talks to unix. Clay also uses `%dill` directly to
16−Clay also uses `%dill` directly to print out the filesystem change
15+print out the filesystem change events, but this is questionable behavior.
17−events, but this is questionable behavior.
18
16
19−`%dill` has two main components. First, it controls the terminal on a
17+`%dill` has two main components. First, it controls the terminal on a very
20−very basic, event-by-event level. This includes keeping track of things
18+basic, event-by-event level. This includes keeping track of things like the
21−like the dimensions of the terminal, the prompt type (plain text or
19+dimensions of the terminal, the prompt type (plain text or password), which duct
22−password), which duct to produce effects on, and so forth. Second, it
20+to produce effects on, and so forth. Second, it handles terminal events,
23−handles terminal events, keystroke by keystroke. Most characters are
21+keystroke by keystroke. Most characters are simply pushed onto the buffer and
24−simply pushed onto the buffer and blitted to the screen, but some
22+blitted to the screen, but some characters, including control-modified keys,
25−characters, including control-modified keys, arrow keys, etc. require
23+arrow keys, etc. require special handling. Most of the readline functionality is
26−special handling. Most of the readline functionality is in `%dill`.
24+in `%dill`.
docs/arvo/internals/eyre.md+14−13
@@ -1,22 +1,23 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 5
3sort: 5
4---
4---
5
5
6−`%eyre`
6+# `%eyre`
7−=======
8
7
9Our http server.
8Our http server.
10
9
11−Unix sends http messages to `%eyre`, and `%eyre` produces http messages
10+Unix sends http messages to `%eyre`, and `%eyre` produces http messages in
12−in response. In general, apps and vanes do not call `%eyre`; rather,
11+response. In general, apps and vanes do not call `%eyre`; rather, `%eyre` calls
13−`%eyre` calls apps and vanes. `%eyre` uses `%ford` and `%gall` to
12+apps and vanes. `%eyre` uses `%ford` and `%gall` to functionally publish pages
14−functionally publish pages and facilitate communication with apps.
13+and facilitate communication with apps.
15
14
16−`%eyre` primarily parses web requests and handles them in a variety of
15+`%eyre` primarily parses web requests and handles them in a variety of ways,
17−ways, depending on the control string. Nearly all of these are
16+depending on the control string. Nearly all of these are essentially stateless,
18−essentially stateless, like functional publishing with `%ford`.
17+like functional publishing with `%ford`. Additionally, there's a fairly
19−Additionally, there's a fairly significant component that handles
18+significant component that handles `%gall` messaging and subscriptions, which
20−`%gall` messaging and subscriptions, which must be stateful.
19+must be stateful.
21
20
22−<div><list/></div>
21+<div>
22+<list/>
23+</div>
docs/arvo/internals/eyre/commentary.md+50−45
@@ -1,43 +1,42 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3---
3---
4
4
5−Eyre: Reference
5+# Eyre: Reference
6−===============
7
6
8## State
7## State
9
8
10Stored
9Stored
11
10
12−- `++bolo`
11+- `++bolo`
13−- `++cyst`
12+- `++cyst`
14−- `++stem`
13+- `++stem`
15
14
16Runtime
15Runtime
17
16
18−- `perk`
17+- `perk`
19− + `perk-auth`
18+- `perk-auth`
20−- `pest`
19+- `pest`
21−- `even`
20+- `even`
22
21
23## Cores
22## Cores
24
23
25−- `++handle`
24+- `++handle`
26− + `++parse`
25+- `++parse`
27− - `++as...`
26+ - `++as...`
28−- `++ya`
27+- `++ya`
29−- `++ix`
28+- `++ix`
30
29
31−Eyre: Commentary
30+# Eyre: Commentary
32−================
33
31
34−Let us follow the loading of a simple cli app, as it bounces from
32+Let us follow the loading of a simple cli app, as it bounces from browser to
35−browser to server to browser and back.
33+server to browser and back.
36
34
37## Initial request<a id="init"/>
35## Initial request<a id="init"/>
38
36
39−An http request for `http://sampel-sipnym.urbit.org/cli` will be [redirected](dns)
37+An http request for `http://sampel-sipnym.urbit.org/cli` will be
40−to the `%eyre` on ~sampel-sipnym, and come in as a `%this` kiss.
38+[redirected](dns) to the `%eyre` on ~sampel-sipnym, and come in as a `%this`
39+kiss.
41
40
42From arvo, requests enter `++call`, which after some type reification are passed
41From arvo, requests enter `++call`, which after some type reification are passed
43along to `++apex:ye`. In the case of a `%this` kiss, its components are
42along to `++apex:ye`. In the case of a `%this` kiss, its components are
@@ -46,7 +45,8 @@ in `++emule` to produce a `++fail` page in case of error. `++apex:handle` will
46`++process` the request to a `pest` or a `++done` core, and in the former case
45`++process` the request to a `pest` or a `++done` core, and in the former case
47`++resolve` the pest into an outgoing card.
46`++resolve` the pest into an outgoing card.
48
47
49−XX it also seems to affect the current ship, test that serving ship name is consistently correct
48+XX it also seems to affect the current ship, test that serving ship name is
49+consistently correct
50
50
51The pest is produced by `++process`, which will first further `++parse` the
51The pest is produced by `++process`, which will first further `++parse` the
52request, and if this does not make the response immediately obvious,
52request, and if this does not make the response immediately obvious,
@@ -68,8 +68,8 @@ the perk it into a `%boil` `++ford-req`, adding an `++fcgi` path-segment
68containing query string and `++fcgi-cred:for-client` auth information.
68containing query string and `++fcgi-cred:for-client` auth information.
69
69
70`%ford`s translation of `/=cli=/hymn/hook` to a self-refreshing `%urb` html page
70`%ford`s translation of `/=cli=/hymn/hook` to a self-refreshing `%urb` html page
71−[deserves its own commentary](../ford/commentary), but we resume in `%eyre`
71+[deserves its own commentary](../ford/commentary), but we resume in `%eyre` when
72−when the `%made` sign arrives in `++take`, and soon after `++axon:ye`. There the
72+the `%made` sign arrives in `++take`, and soon after `++axon:ye`. There the
73`wire`, or rather the `whir` it has been verified to be, determines that the
73`wire`, or rather the `whir` it has been verified to be, determines that the
74response should be served immediately. However, as the mark is not `%mime`,
74response should be served immediately. However, as the mark is not `%mime`,
75another trip to `%ford` is required to encode it, on the same wire; afterwards,
75another trip to `%ford` is required to encode it, on the same wire; afterwards,
@@ -145,7 +145,7 @@ To understand `/~/at`, there will first be a brief diversion to `~/auth.json`.
145`auth.json`, perk `[%auth %json]`, in `++process-auth` serves `++stat-json:ya`,
145`auth.json`, perk `[%auth %json]`, in `++process-auth` serves `++stat-json:ya`,
146containing such information as the serving ship, which identities are associated
146containing such information as the serving ship, which identities are associated
147with this session, and `oryx`, a CSRF token. An oryx must be present on all
147with this session, and `oryx`, a CSRF token. An oryx must be present on all
148−stateful requests, in this case executing a log in. It also saves the new/old
148+stateful requests, in this case executing a log in. It also saves the new/old
149session using `abet`.
149session using `abet`.
150
150
151XX explain `ixor` here and not [later](#ixor)?
151XX explain `ixor` here and not [later](#ixor)?
@@ -153,21 +153,22 @@ XX explain `ixor` here and not [later](#ixor)?
153`/~/at` is an alternate interface, which injects `auth.json` data into the
153`/~/at` is an alternate interface, which injects `auth.json` data into the
154requested file. `/~/at/auth.js`, then, is a request for the built-in `auth:js`
154requested file. `/~/at/auth.js`, then, is a request for the built-in `auth:js`
155(parsed to and processed from an `[%auth %js ~]` perk), with session data added
155(parsed to and processed from an `[%auth %js ~]` perk), with session data added
156−as `window.urb`. And indeed, ``[`%js /~/at/auth]`` is parsed to
156+as `window.urb`. And indeed, `` [`%js /~/at/auth] `` is parsed to
157−``[%auth at [`%js /auth]``, which in `++process-auth` is re-`process`ed to
157+`` [%auth at [`%js /auth] ``, which in `++process-auth` is re-`process`ed to
158−`[%js ++auth:js]`, which is `++resolve`d after an `++add-json` of
158+`[%js ++auth:js]`, which is `++resolve`d after an `++add-json` of the relevant
159−the relevant data. The yac cookies are also passed to `resolve`, which
159+data. The yac cookies are also passed to `resolve`, which `++add-cookies`
160−`++add-cookies` injects into the `httr`.
160+injects into the `httr`.
161
161
162−It is at this point that there is first occasion for user input, namely the password.
162+It is at this point that there is first occasion for user input, namely the
163+password.
163
164
164The `auth:js` script sends a `PUT` request, also to `/~/auth.json`. In `parse`,
165The `auth:js` script sends a `PUT` request, also to `/~/auth.json`. In `parse`,
165the first nontrivial `++check-oryx` occurs, `++grab-body` the request oryx and
166the first nontrivial `++check-oryx` occurs, `++grab-body` the request oryx and
166ensuring it is recorded for the session. The request parsed with `++need-body`
167ensuring it is recorded for the session. The request parsed with `++need-body`
167to a `[%auth %try {password}]` perk. `%get:process-auth` checks it against
168to a `[%auth %try {password}]` perk. `%get:process-auth` checks it against
168−`++load-secret`, upon success updates the session with `++logon:ya`, and
169+`++load-secret`, upon success updates the session with `++logon:ya`, and serves
169−serves a fresh `auth.json` which reflects the changed `user`. Upon receiving
170+a fresh `auth.json` which reflects the changed `user`. Upon receiving this, the
170−this, the page is refreshed to retry the original request.
171+page is refreshed to retry the original request.
171
172
172## Post-authentication: app communication. <a id="auth-ok"/>
173## Post-authentication: app communication. <a id="auth-ok"/>
173
174
@@ -182,7 +183,7 @@ know to serve the window.urb necessary to make requests, and the `urb.js`
182standard library which extends it with a number of wrappers to them and other
183standard library which extends it with a number of wrappers to them and other
183useful functions.
184useful functions.
184
185
185−---
186+--------------------------------------------------------------------------------
186
187
187One of those functions is `urb.bind`, which is used to subscribe to application
188One of those functions is `urb.bind`, which is used to subscribe to application
188data. Userspace javascript sets `urb.appl` to `/tic`, and binds `lines` to a
189data. Userspace javascript sets `urb.appl` to `/tic`, and binds `lines` to a
@@ -212,18 +213,19 @@ is started using `urb.poll`. At `/~/of/{ixor}`, perk
212`[%view ixor ~ {sequence-number}]`, it is `process`ed by `++poll:ix` (the cyst
213`[%view ixor ~ {sequence-number}]`, it is `process`ed by `++poll:ix` (the cyst
213is retrieved by `++ire-ix` form global state, using the perk `ixor`): the
214is retrieved by `++ire-ix` form global state, using the perk `ixor`): the
214sequence number is in the past, so the previously received `%rush` is
215sequence number is in the past, so the previously received `%rush` is
215−`++give-even`. After deleting the previous message in the queue and invoking
216+`++give-even`. After deleting the previous message in the queue and invoking
216`++pass-took` to signal `%gall` of this occurrence, the data is annotated with
217`++pass-took` to signal `%gall` of this occurrence, the data is annotated with
217the source app+path `++subs-to-json`, and returned to the polling duct.
218the source app+path `++subs-to-json`, and returned to the polling duct.
218
219
219On the client, the user callback receives the `/cli` history, and displays it on
220On the client, the user callback receives the `/cli` history, and displays it on
220the page. The `/~/of` long poll is continued, this time reaching `++poll:ix`
221the page. The `/~/of` long poll is continued, this time reaching `++poll:ix`
221−with the "pending" sequence number, and being stored in the `cyst` for its troubles.
222+with the "pending" sequence number, and being stored in the `cyst` for its
223+troubles.
222
224
223−---
225+--------------------------------------------------------------------------------
224
226
225Its next update proceeds idenitcally, but first it must be triggered, which
227Its next update proceeds idenitcally, but first it must be triggered, which
226−happens when the user enters "(add 2 2)\n", firing an `urb.send` from the event
228+happens when the user enters "(add 2 2)\\n", firing an `urb.send` from the event
227handler. This sends a `POST` request to `/~/to/cli/json.json`, perk `%mess`,
229handler. This sends a `POST` request to `/~/to/cli/json.json`, perk `%mess`,
228`++process`ed to a `%g %mess`. Were the mark not `%json`, a `%ford` conversion
230`++process`ed to a `%g %mess`. Were the mark not `%json`, a `%ford` conversion
229would occur first, and `%made:axon` would send the gall message proper. In
231would occur first, and `%made:axon` would send the gall message proper. In
@@ -233,7 +235,7 @@ sent to the client callback.
233## A path not taken: magic filenames <a id="mage"/>
235## A path not taken: magic filenames <a id="mage"/>
234
236
235The `/robots.txt` and `/favicon.(ico|png)` files are static, and served
237The `/robots.txt` and `/favicon.(ico|png)` files are static, and served
236−immediately when caught by a `++parse`.
238+immediately when caught by a `++parse`.
237
239
238XX index.html?
240XX index.html?
239
241
@@ -250,7 +252,6 @@ normal login page but later sends the client back. XX expand, basically the
250status quo is you're logged in and `/~/as/foo` is ignored, just setting your
252status quo is you're logged in and `/~/as/foo` is ignored, just setting your
251`urb.user` XX
253`urb.user` XX
252
254
253−
254## A path not taken: deauthentication
255## A path not taken: deauthentication
255
256
256`/~/away`, perk `[%away ~]`, produces a static `++logout-page:xml`, which also
257`/~/away`, perk `[%away ~]`, produces a static `++logout-page:xml`, which also
@@ -264,8 +265,12 @@ from memory.
264`++del-subs:ix` acting as reverse of `++add-subs` by deleting the duct binding
265`++del-subs:ix` acting as reverse of `++add-subs` by deleting the duct binding
265and sending `%g %nuke`.
266and sending `%g %nuke`.
266
267
267−
268+XX unmentioned arms: abet, add-poll, adit, ames-gram, anon, ares-to-json, bolo,
268−XX unmentioned arms: abet, add-poll, adit, ames-gram, anon, ares-to-json, bolo, cyst, doze, even, ford-kill, get-mean, gift, give-json, give-thou, gram, hapt, hasp, host-to-ship, ix, ixor, js, kiss, load, mean-json, move, note, pass-note, perk, perk-auth, pest, poke-test, print-subs, render-tang, resp, root-beak, scry, ses-authed, ses-ya, sign, silk, sine, stay, stem, teba, titl, to-oryx, urb, wait-era, wake, whir, wush, xml, ya, ye
269+cyst, doze, even, ford-kill, get-mean, gift, give-json, give-thou, gram, hapt,
270+hasp, host-to-ship, ix, ixor, js, kiss, load, mean-json, move, note, pass-note,
271+perk, perk-auth, pest, poke-test, print-subs, render-tang, resp, root-beak,
272+scry, ses-authed, ses-ya, sign, silk, sine, stay, stem, teba, titl, to-oryx,
273+urb, wait-era, wake, whir, wush, xml, ya, ye
269
274
270## Appendix A: DNS <a id="dns"/>
275## Appendix A: DNS <a id="dns"/>
271
276
@@ -273,5 +278,5 @@ The `*.urbit.org` domain can be used to access destroyers and cruisers. In the
273common case oh hosted ships, this is done by dynamic DNS directly to the hosting
278common case oh hosted ships, this is done by dynamic DNS directly to the hosting
274instance. We do not speak of the uncommon case. When ports are blocked and
279instance. We do not speak of the uncommon case. When ports are blocked and
275infrastructure crumbles around you, only imported martian networking can be
280infrastructure crumbles around you, only imported martian networking can be
276−trusted: the `%get` and `%got` [gram]()s are used to proxy [`%this` requests]() and
281+trusted: the `%get` and `%got` [gram]()s are used to proxy [`%this` requests]()
277−[`%thou` responses]() respectively.
282+and [`%thou` responses]() respectively.
docs/arvo/internals/eyre/specification.md+355−189
@@ -1,135 +1,186 @@
1# Specification
1# Specification
2
2
3−The `%eyre` vane serves one purpose: receiving inbound HTTP requests from browsers and clients pretending to be browsers. To facilitate this communication, `%eyre` does two things: 1. provide an API for traditional clients to talk to; and 2. provide a mechanism for saving state that allows the server to keep track of identities/sessions, subscription data, and active connections.
3+The `%eyre` vane serves one purpose: receiving inbound HTTP requests from
4+browsers and clients pretending to be browsers. To facilitate this
5+communication, `%eyre` does two things: 1. provide an API for traditional
6+clients to talk to; and 2. provide a mechanism for saving state that allows the
7+server to keep track of identities/sessions, subscription data, and active
8+connections.
4
9
5This document outlines the following:
10This document outlines the following:
6
11
7−1. [`perk`](#perk), a `pork` subset that defines what URLs the client can send to `%eyre`.
12+1. [`perk`](#perk), a `pork` subset that defines what URLs the client can send
8−2. [the values stored on the client](#client), both statically in cookies and dynamically as the `window.urb` object, which facilitate this transaction.
13+ to `%eyre`.
14+2. [the values stored on the client](#client), both statically in cookies and
15+ dynamically as the `window.urb` object, which facilitate this transaction.
9
16
10−3. [`kiss`](#kiss), the requests `%eyre` will accept.
17+3. [`kiss`](#kiss), the requests `%eyre` will accept.
11−4. [`gift`](#gift), the responses it will give to those requests.
18+4. [`gift`](#gift), the responses it will give to those requests.
12
19
13−5. [`note`](#note), the set of arvo requests that %eyre makes to the other vanes.
20+5. [`note`](#note), the set of arvo requests that %eyre makes to the other
14−6. [`whir`](#whir), a `wire` subset, the associated dynamic state
21+ vanes.
15−7. [`sign`](#sign), the responses `%eyre` accepts
22+6. [`whir`](#whir), a `wire` subset, the associated dynamic state
16−8. [`bolo`](#bolo), the state stored between separate events.
23+7. [`sign`](#sign), the responses `%eyre` accepts
24+8. [`bolo`](#bolo), the state stored between separate events.
17
25
18# 1. URL structure <a id="perk"/>
26# 1. URL structure <a id="perk"/>
19−There exist several URL forms accepted by `%eyre`.
27+
28+There exist several URL forms accepted by `%eyre`.
20
29
21## 1.1 Functional request
30## 1.1 Functional request
31+
22The simplest of them is a static `%f` functional publish request:
32The simplest of them is a static `%f` functional publish request:
23
33
24− ```hoon
34+`hoon GET /[ship]/[desk]/[case]/path/to/file.[mark]` e.g.
25− GET /[ship]/[desk]/[case]/path/to/file.[mark]
26− ```
27−e.g.
28
35
29− ```hoon
36+`hoon /~pittyp-pittyp/main/5/lib/urb.js` A `case` of 0 signifies the most
30− /~pittyp-pittyp/main/5/lib/urb.js
37+recent version.
31− ```
32−A `case` of 0 signifies the most recent version.
33
38
34−If no file extension is provided, the default `mark` is `%urb`, which consists of `%html` injected with an auto-refresh script.
39+If no file extension is provided, the default `mark` is `%urb`, which consists
40+of `%html` injected with an auto-refresh script.
35
41
36A shorter form:
42A shorter form:
37
43
38− ```hoon
44+`hoon /path/to.[mark]` is prepended with the [default request path](#prefix).
39− /path/to.[mark]
45+For example, if `%eyre` is configured with a root of `/<our>/main/0/pub`, then
40− ```
46+`/doc/arvo` is interpreted as a request for `/=main=/pub/doc/arvo.urb`.
41−is prepended with the [default request path](#prefix). For example, if `%eyre`
42−is configured with a root of `/<our>/main/0/pub`, then `/doc/arvo` is
43−interpreted as a request for `/=main=/pub/doc/arvo.urb`.
44
47
45Auxiliary paths start with `/~/`, and provide other functionality, such as:
48Auxiliary paths start with `/~/`, and provide other functionality, such as:
46
49
47## 1.2 Auto-reload
50## 1.2 Auto-reload
48−When a new version of a page becomes available, it is useful to propagate it to the client. The `%f` vane manages the particulars, and provides an update token along with all published pages. The `/~/on` long-polling endpoint makes use of that token, which is a hash of the server-stored dependency set.
51+
49−- `/~/on/[hash].json` retuns `true` when a page may require updating.
52+When a new version of a page becomes available, it is useful to propagate it to
50−- `/~/on/[hash].js` is a lightweight script that polls for the above. It is primarily used in error messages.
53+the client. The `%f` vane manages the particulars, and provides an update token
54+along with all published pages. The `/~/on` long-polling endpoint makes use of
55+that token, which is a hash of the server-stored dependency set. -
56+`/~/on/[hash].json` retuns `true` when a page may require updating. -
57+`/~/on/[hash].js` is a lightweight script that polls for the above. It is
58+primarily used in error messages.
51
59
52## 1.3 Authentication <a id="auth"/>
60## 1.3 Authentication <a id="auth"/>
53−Authenticated requests are accomplished through sessions, tracked with cookies which are set on the first such request.
61+
54−- `/~/as/[user]/...` is an authenticated request to `...`, where `user` can be a `@p` ship, `'own'`, `'any'`, or `'anon'`.
62+Authenticated requests are accomplished through sessions, tracked with cookies
55− + `'anon'` is a randomly-generated per-session submarine, and requires no further authentication.
63+which are set on the first such request. - `/~/as/[user]/...` is an
56− + `'own'` represents the serving ship. The short form of `/~/as/own/` is `/~~/`
64+authenticated request to `...`, where `user` can be a `@p` ship, `'own'`,
57− + `'any'` requires a network ship, chosen with a prompt if unknown. The short form of `/~/as/any/` is `/_/`
65+`'any'`, or `'anon'`. + `'anon'` is a randomly-generated per-session submarine,
58− + Auth requests are handled in three ways:
66+and requires no further authentication. + `'own'` represents the serving ship.
59− + I. If the session is associated with the `ship`, serve `...`, storing the ship as the session's default ship.
67+The short form of `/~/as/own/` is `/~~/` + `'any'` requires a network ship,
60− For example,
68+chosen with a prompt if unknown. The short form of `/~/as/any/` is `/_/` + Auth
61− /~/as/sivtyv-barnel/main/pub/test.html
69+requests are handled in three ways: + I. If the session is associated with the
62− is an authenticated request of
70+`ship`, serve `...`, storing the ship as the session's default ship. For
63− /main/pub/test.html
71+example, /~/as/sivtyv-barnel/main/pub/test.html is an authenticated request of
64− with the identity of `~sivtyv-barnel`.
72+/main/pub/test.html with the identity of `~sivtyv-barnel`. + II. If the session
65− + II. If the session is not associated with the `ship` in the path, the client is redirected to a login page, where a password can be entered. Successful authentication results in a page refresh to the requested path.
73+is not associated with the `ship` in the path, the client is redirected to a
66− + III. otherwise, [send a message](#foreign) to the claimed ship, and redirect the user to `https://[foreign-host]/~/am/[ses]/...#[current-host]&?code=[code]`
74+login page, where a password can be entered. Successful authentication results
67−
75+in a page refresh to the requested path. + III. otherwise, [send a
68−
76+message](#foreign) to the claimed ship, and redirect the user to
69−- `/~/am/[ses]/...#original-host` similarly serves a login page. Upon success, inform the `ship` of the success and redirect it back to `http://[original-host]/~/as/<this-ship>/...`
77+`https://[foreign-host]/~/am/[ses]/...#[current-host]&?code=[code]`
70−- `/~/away.html` is a log-out page.
78+
71−- `/~/auth.json` returns `{ship,oryx,user,auth}`: the serving ship, a CSRF token, active user, and all allowed users respectively.
79+- `/~/am/[ses]/...#original-host` similarly serves a login page. Upon success,
72−- `/~/at/{...}.js` is an alternate form of `/~/auth.json`, which injects a line into the served js file that sets `{ship, oryx, user, auth}` on the `window.urb` object.
80+ inform the `ship` of the success and redirect it back to
73−
81+ `http://[original-host]/~/as/<this-ship>/...`
74−So far, all the paths specified have been GET requests. Authentication, however, requires sending data. This is also done at the `/~/auth.json` endpoint.
82+- `/~/away.html` is a log-out page.
75−- `POST {oryx,ship,code} /~/auth.json?PUT` is used by the login script to authorize a user, and returns the same result as `GET`, additionally containing either`{ok:true}` or `{fail:'type',mess:"Error message"}`
83+- `/~/auth.json` returns `{ship,oryx,user,auth}`: the serving ship, a CSRF
76−- `POST {oryx,ship?} /~/auth.json?DELETE` revokes authorization. Response as above.
84+ token, active user, and all allowed users respectively.
85+- `/~/at/{...}.js` is an alternate form of `/~/auth.json`, which injects a
86+ line into the served js file that sets `{ship, oryx, user, auth}` on the
87+ `window.urb` object.
88+
89+So far, all the paths specified have been GET requests. Authentication, however,
90+requires sending data. This is also done at the `/~/auth.json` endpoint. -
91+`POST {oryx,ship,code} /~/auth.json?PUT` is used by the login script to
92+authorize a user, and returns the same result as `GET`, additionally containing
93+either`{ok:true}` or `{fail:'type',mess:"Error message"}` -
94+`POST {oryx,ship?} /~/auth.json?DELETE` revokes authorization. Response as
95+above.
77
96
78## 1.4 Messaging
97## 1.4 Messaging
79−- `POST {oryx, wire, xyro} /~/to/[app]/[mark].json`, where `xyro` is data that will be converted to the `mark`. In the simplest case, `/~/to/hello/json.json` will pass `xyro` through verbatim, sending `[%json xyro]` to the `++poke` arm of app `%hello`.
98+
80−- `POST {oryx, wire, xyro} /~/to/<ship>/[app]/[mark].json` is a foreign message send, as above.
99+- `POST {oryx, wire, xyro} /~/to/[app]/[mark].json`, where `xyro` is data that
100+ will be converted to the `mark`. In the simplest case,
101+ `/~/to/hello/json.json` will pass `xyro` through verbatim, sending
102+ `[%json xyro]` to the `++poke` arm of app `%hello`.
103+- `POST {oryx, wire, xyro} /~/to/<ship>/[app]/[mark].json` is a foreign
104+ message send, as above.
81
105
82## 1.5 Subscriptions <a id="subs"/>
106## 1.5 Subscriptions <a id="subs"/>
83−- `/~/of/[ixor]`<a id="of-ixor"/> is an `EventSource`: a conceptually infinite file containing a stream of events. By default, it sends a newline every 30 seconds serving to signal that the connection is alive to both the server and IP middleware.
84− + `/~/of/[ixor]?poll={n}` is a long-polling fallback interface. Produces event number {n}, blocking until it occurs.
85
107
86−Its contexts can be affected by `POST` requests with a body of `{oryx, wire}`, and a query string of `PUT` or `DELETE`. They return either `{mark}`(which may be `null`), or [`{fail, mess}`](#mean-json)
108+- `/~/of/[ixor]`<a id="of-ixor"/> is an `EventSource`: a conceptually infinite
109+ file containing a stream of events. By default, it sends a newline every 30
110+ seconds serving to signal that the connection is alive to both the server
111+ and IP middleware.
112+- `/~/of/[ixor]?poll={n}` is a long-polling fallback interface. Produces event
113+ number {n}, blocking until it occurs.
114+
115+Its contexts can be affected by `POST` requests with a body of `{oryx, wire}`,
116+and a query string of `PUT` or `DELETE`. They return either `{mark}`(which may
117+be `null`), or [`{fail, mess}`](#mean-json)
87
118
88−- `/~/in/[hash].json` is a dual to [`/~/on`](#on-change). It binds `mod` events, which echo the requesting token.
119+- `/~/in/[hash].json` is a dual to [`/~/on`](#on-change). It binds `mod`
89−- `/~/is/[app]/path/to.[mark]` binds `rush` events, formatted with a first line of `[app] /path/to`, and the rest containing subscription data. Additionally, `mean` events formatted similarly may arrive, also converted to the relevant mark.
120+ events, which echo the requesting token.
121+- `/~/is/[app]/path/to.[mark]` binds `rush` events, formatted with a first
122+ line of `[app] /path/to`, and the rest containing subscription data.
123+ Additionally, `mean` events formatted similarly may arrive, also converted
124+ to the relevant mark.
90
125
91## 1.6 Ablative <a id="temp"/>
126## 1.6 Ablative <a id="temp"/>
92−These interfaces will temporarily exist to aid development, and are to be considered unstable.
127+
93−- `/~/debug/...` access normally inaccessible pages. For example, `/~/debug/as.html` will present the login page, regardless of current session status.
128+These interfaces will temporarily exist to aid development, and are to be
129+considered unstable. - `/~/debug/...` access normally inaccessible pages. For
130+example, `/~/debug/as.html` will present the login page, regardless of current
131+session status.
94
132
95# 2. Client state <a id="client"/>
133# 2. Client state <a id="client"/>
134+
96Some information is stored on, and provided to, browser clients
135Some information is stored on, and provided to, browser clients
97
136
98## 2.1 Cookies
137## 2.1 Cookies
99−Authenticated users receive a cookie on the domain of `*.urbit.org`. The cookie contains a client session token, keyed by the serving ship.
138+
139+Authenticated users receive a cookie on the domain of `*.urbit.org`. The cookie
140+contains a client session token, keyed by the serving ship.
100
141
101## 2.2 Authentication <a id="auth-json"/>
142## 2.2 Authentication <a id="auth-json"/>
102−It is common(e.g. by the `%urb` mark) to set window.urb to the contents of `/~/auth.json`:
143+
103−- `ship` is the serving ship
144+It is common(e.g. by the `%urb` mark) to set window.urb to the contents of
104−- `oryx` is a unique CSRF token that identifies this view, bound to the session cookie
145+`/~/auth.json`: - `ship` is the serving ship - `oryx` is a unique CSRF token
105−- `ixor` is a hash of `oryx`, used as an insecure view identifier
146+that identifies this view, bound to the session cookie - `ixor` is a hash of
106−- `user` is the authenticated/generated ship name
147+`oryx`, used as an insecure view identifier - `user` is the
107−- `auth` is a list of identities usable by this session
148+authenticated/generated ship name - `auth` is a list of identities usable by
149+this session
108
150
109## 2.3 Client library
151## 2.3 Client library
110−To this object, `/main/lib/urb.js` adds helpers:
152+
111−- `app` is an optional app name to send messages to by default
153+To this object, `/main/lib/urb.js` adds helpers: - `app` is an optional app name
112−- `send({data,mark?='json',app?=urb.app,ship?}, cb?)` delivers messages
154+to send messages to by default -
113−- `bind({path,app?=urb.app}, cb)` subscribes by path
155+`send({data,mark?='json',app?=urb.app,ship?}, cb?)` delivers messages -
114−- `drop({path,app?=urb.app}, cb?)` pulls the subscription
156+`bind({path,app?=urb.app}, cb)` subscribes by path -
115−- `util` is an object containing methods for converting between JavaScript types and Hoon atom odors.
157+`drop({path,app?=urb.app}, cb?)` pulls the subscription - `util` is an object
158+containing methods for converting between JavaScript types and Hoon atom odors.
116
159
117# 3. Requests <a id="kiss"/>
160# 3. Requests <a id="kiss"/>
118
161
119## `[%born port=@ud]`, unix init
162## `[%born port=@ud]`, unix init
120−When the `vere` process is re/started, a TCP port is bound, HTTP requests on which are the scope of this document. The `%born` gift informs eyre of this occurrence, which in turn updates [its state.](#bolo)
163+
164+When the `vere` process is re/started, a TCP port is bound, HTTP requests on
165+which are the scope of this document. The `%born` gift informs eyre of this
166+occurrence, which in turn updates [its state.](#bolo)
121
167
122**Gifts given in response:** none.
168**Gifts given in response:** none.
123
169
124## `[%hiss p=mark q=cage]`, outbound http
170## `[%hiss p=mark q=cage]`, outbound http
171+
125Userspace http request, converted by `%ford` into an actual `hiss`.
172Userspace http request, converted by `%ford` into an actual `hiss`.
126
173
127**Gifts given in response:** a [`%sigh`]()
174**Gifts given in response:** a [`%sigh`]()
128
175
129## 3.1 `[%this httq]`, inbound http
176## 3.1 `[%this httq]`, inbound http
130−The primary `%eyre` request type comes from the host system, and constitutes an inbound http request. It consists of five parts:
131
177
132−I. `p=[? @if]`: a loobean determining whether the request is being made over https, and the client source, an IPv6 address possibly encoding an IPv4 one as `.0.0.0.0.0.ffff.wwxx.yyzz`
178+The primary `%eyre` request type comes from the host system, and constitutes an
179+inbound http request. It consists of five parts:
180+
181+I. `p=[? @if]`: a loobean determining whether the request is being made over
182+https, and the client source, an IPv6 address possibly encoding an IPv4 one as
183+`.0.0.0.0.0.ffff.wwxx.yyzz`
133
184
134II. `q=meth`, an http method
185II. `q=meth`, an http method
135
186
@@ -139,225 +190,342 @@ IV. `s=mess`, an associative list of header keys/values
139
190
140V. `t=(unit octs)`, the request body, if any.
191V. `t=(unit octs)`, the request body, if any.
141
192
142−After `r` is parsed to a `purl`, the `pork`(relative path) determines the resource being served. The first step in handling an http request is [classifying its intent](#perk).
193+After `r` is parsed to a `purl`, the `pork`(relative path) determines the
143−
194+resource being served. The first step in handling an http request is
144−- A ford request results in a [`%boil` note](#boil).
195+[classifying its intent](#perk).
145−- `/~/[name].js` is generated from [session](#sink) or [view](#stem) state, and [served](#mime) as `%js`.
196+
146−- `/~/am` generates a login page.
197+- A ford request results in a [`%boil` note](#boil).
147−- `/~/as` is resolved further if authenticated, generates a similar login page if the claimed ship is local, and otherwise is saved to [session state](#sink) and sends a ["login" `%a` message](#lon) to the foreign ship.
198+- `/~/[name].js` is generated from [session](#sink) or [view](#stem) state,
148−- `/~/at` and `/~/auth.json` generate a new `oryx` and store it in the [session state](#sink).
199+ and [served](#mime) as `%js`.
149−- `/~/in` and `/~/is` affect [view state](#stem), possibly sending subscriptions or unsubscriptions to `%f` and `%g` respectively, and are generated from view state.
200+- `/~/am` generates a login page.
150−- `/~/on` is saved to the `bolo` by `hash`, and translated to a [`%wasp` note](#wasp).
201+- `/~/as` is resolved further if authenticated, generates a similar login page
151−- `/~/of` fails if the `ixor` is unknown, and otherwise is stored in the `ixor`, receiving a partial response and all events after its [last-seen header](#). If it has a `poll` parameter, the response is full, and only occurs if the body is non-empty. Otherwise the oryx is saved as a [live](#live) request in case of cancellation.
202+ if the claimed ship is local, and otherwise is saved to [session
152−- `/~/to` transforms into a [`%g` message](#mess).
203+ state](#sink) and sends a ["login" `%a` message](#lon) to the foreign ship.
153−
204+- `/~/at` and `/~/auth.json` generate a new `oryx` and store it in the
154−**Gifts given in response:** a [`%thou`](), or a [`%that`](#that) followed by multiple [`%thar`]()s.
205+ [session state](#sink).
206+- `/~/in` and `/~/is` affect [view state](#stem), possibly sending
207+ subscriptions or unsubscriptions to `%f` and `%g` respectively, and are
208+ generated from view state.
209+- `/~/on` is saved to the `bolo` by `hash`, and translated to a [`%wasp`
210+ note](#wasp).
211+- `/~/of` fails if the `ixor` is unknown, and otherwise is stored in the
212+ `ixor`, receiving a partial response and all events after its [last-seen
213+ header](#). If it has a `poll` parameter, the response is full, and only
214+ occurs if the body is non-empty. Otherwise the oryx is saved as a
215+ [live](#live) request in case of cancellation.
216+- `/~/to` transforms into a [`%g` message](#mess).
217+
218+**Gifts given in response:** a [`%thou`](), or a [`%that`](#that) followed by
219+multiple [`%thar`]()s.
155
220
156## 3.2 `[%thud ~]`, inbound close
221## 3.2 `[%thud ~]`, inbound close
157−Sent when an unresponded-to request is cancelled. The [live](#live) effect of the request is looked up, and
222+
158−- `%exec` ford boil requests are cancelled with `[%exec ~]`
223+Sent when an unresponded-to request is cancelled. The [live](#live) effect of
159−- `%wasp` ford deps requests are cancelled with `[%wasp |]`
224+the request is looked up, and - `%exec` ford boil requests are cancelled with
160−- `%poll` session polls `%rest` the session heartbeat timeout
225+`[%exec ~]` - `%wasp` ford deps requests are cancelled with `[%wasp |]` -
226+`%poll` session polls `%rest` the session heartbeat timeout
161
227
162## 3.3 `[%thin p=@ud]`, delivery failure
228## 3.3 `[%thin p=@ud]`, delivery failure
163−Sent when a partial [response](#that) or [body](#thar) fails to arrive. Its `duct` is the same as that of the original request. This is handled by [waiting](#wait) for the client to reconnect, and if this not occur, unwinding all subscription state.
229+
230+Sent when a partial [response](#that) or [body](#thar) fails to arrive. Its
231+`duct` is the same as that of the original request. This is handled by
232+[waiting](#wait) for the client to reconnect, and if this not occur, unwinding
233+all subscription state.
164
234
165**Gifts given in response:** None.
235**Gifts given in response:** None.
166
236
167−## 3.4 ``[%wart sock [path *]]``, inbound message
237+## 3.4 `[%wart sock [path *]]`, inbound message
168−Sometimes, messages are received from other ships. They are expected to take the from of a [`gram`]().
238+
239+Sometimes, messages are received from other ships. They are expected to take the
240+from of a [`gram`]().
169
241
170**Gifts given in response:** [`%nice` or `%mean`](#ack).
242**Gifts given in response:** [`%nice` or `%mean`](#ack).
171
243
172### 3.4.1 `gram` <a id="gram"/>
244### 3.4.1 `gram` <a id="gram"/>
173
245
174−There are three messages of note, all concerning authentication <a id="foreign"/>
246+There are three messages of note, all concerning authentication
175−- `[/lon ses]` <a id="lon"/> is a login request, which contains the session wishing to authenticate.
247+<a id="foreign"/> - `[/lon ses]` <a id="lon"/> is a login request, which
176−- if the session is already authorized, a reply of `[/aut ses]` from (%eyre on) a foreign ship prescribes that the session is henceforth allowed to act on its behalf.
248+contains the session wishing to authenticate. - if the session is already
177−All waiting `/~/as` are resolved as succesful.
249+authorized, a reply of `[/aut ses]` from (%eyre on) a foreign ship prescribes
178−- otherwise, `[/hat ses hart]` contains the ship's preferred hostname. This can then be used to redirect the client.
250+that the session is henceforth allowed to act on its behalf. All waiting `/~/as`
179−All waiting `/~/as` are given `307 Redirect`s to `/~/am` on the provided host.
251+are resolved as succesful. - otherwise, `[/hat ses hart]` contains the ship's
252+preferred hostname. This can then be used to redirect the client. All waiting
253+`/~/as` are given `307 Redirect`s to `/~/am` on the provided host.
180
254
181**Gifts given in response:** nice?
255**Gifts given in response:** nice?
182
256
183# 4. Responses <a id="gift"/>
257# 4. Responses <a id="gift"/>
258+
184## 4.1 `[%nice ~]`, `[%mean ares]`, network acknowledgement <a id="ack"/>
259## 4.1 `[%nice ~]`, `[%mean ares]`, network acknowledgement <a id="ack"/>
185−A `%nice` is given upon receiving an [ames message](#gram), indicating succesful receipt. `%mean` is currently unused directly, but reserved for error conditions.
260+
261+A `%nice` is given upon receiving an [ames message](#gram), indicating succesful
262+receipt. `%mean` is currently unused directly, but reserved for error
263+conditions.
186
264
187## `[%sigh p=cage]`, inbound http response
265## `[%sigh p=cage]`, inbound http response
188−An HTTP response gets converted to the requested mark, or `%tang` in case of error.
266+
267+An HTTP response gets converted to the requested mark, or `%tang` in case of
268+error.
189
269
190## 4.2 `[%thou httr]`, full HTTP response
270## 4.2 `[%thou httr]`, full HTTP response
191−Most requests are served with one coherent response, consisting of
271+
192−- `p=@ud`, an HTTP status code
272+Most requests are served with one coherent response, consisting of - `p=@ud`, an
193−- `q=mess`, response headers
273+HTTP status code - `q=mess`, response headers - `r=(unit octs)`, an optional
194−- `r=(unit octs)`, an optional body
274+body
195
275
196## 4.3 `[%that httr]`, partial HTTP response <a id="that"/>
276## 4.3 `[%that httr]`, partial HTTP response <a id="that"/>
197−[EventStream responses](#of-ixor) consist of multiple sequential chunks. Treated as `%thou`, except the request is kept open.
277+
278+[EventStream responses](#of-ixor) consist of multiple sequential chunks. Treated
279+as `%thou`, except the request is kept open.
198
280
199## 4.4 `[%thar (unit octs)]`, partial HTTP body <a id="thar"/>
281## 4.4 `[%thar (unit octs)]`, partial HTTP body <a id="thar"/>
200−Complementing `%that`, `%thar` is a body chunk, containing an event. An empty `%thar` signals for the connection to be closed.
201
282
202−Body chunks, besides `[1 '\0a']`(a heartbeat newline), are encoded from `even` events. The stem becomes the [`event` field](#eventsource), and the content, `data` lines.
283+Complementing `%that`, `%thar` is a body chunk, containing an event. An empty
284+`%thar` signals for the connection to be closed.
285+
286+Body chunks, besides `[1 '\0a']`(a heartbeat newline), are encoded from `even`
287+events. The stem becomes the [`event` field](#eventsource), and the content,
288+`data` lines.
203
289
204### 4.4.1 `even`, event types <a id="even"/>
290### 4.4.1 `even`, event types <a id="even"/>
291+
205There are three events that a [client subscription](#subs) will be given.
292There are three events that a [client subscription](#subs) will be given.
206
293
207−- `[%news hash]`<a id="even-news"/> is a `%f` update, and contains the relevant dependency token.
294+- `[%news hash]`<a id="even-news"/> is a `%f` update, and contains the
208−- `[%rush [term path] wain]`<a id="even-rush"/> is sourced subscription data.
295+ relevant dependency token.
209−- `[%mean [term path] ares]` is a sourced subscription error.
296+- `[%rush [term path] wain]`<a id="even-rush"/> is sourced subscription data.
297+- `[%mean [term path] ares]` is a sourced subscription error.
210
298
211# 5. Vane requests <a id="note"/>
299# 5. Vane requests <a id="note"/>
212−## 5.1 ``[%a %wont sock `[path *]`gram]``, outbound message
300+
301+## 5.1 `` [%a %wont sock `[path *]`gram] ``, outbound message
302+
213The `%a` interface provides conveyance of [messages](#gram) over UDP.
303The `%a` interface provides conveyance of [messages](#gram) over UDP.
214
304
215Signs a `%woot` upon message arrival.
305Signs a `%woot` upon message arrival.
216
306
217## 5.2 `[%b ?(%wait %rest) time]`, timeout set/unset <a id="wait"/>
307## 5.2 `[%b ?(%wait %rest) time]`, timeout set/unset <a id="wait"/>
218−All open [subscriptions](#of-ixor) require a "heartbeat" newline every `~s30`. When this fails to arrive, a complementary timer is set for `~m1`, after which the client is considered to have departed.
308+
219−The `%b` timer interface is used to schedule the next such event, or cancel past scheduled ones when a connection closes.
309+All open [subscriptions](#of-ixor) require a "heartbeat" newline every `~s30`.
310+When this fails to arrive, a complementary timer is set for `~m1`, after which
311+the client is considered to have departed. The `%b` timer interface is used to
312+schedule the next such event, or cancel past scheduled ones when a connection
313+closes.
220
314
221Signs a `%wake` upon timer activation.
315Signs a `%wake` upon timer activation.
222
316
223## 5.3 `[%f %wasp @uvI]`, dependency listen <a id="wasp"/>
317## 5.3 `[%f %wasp @uvI]`, dependency listen <a id="wasp"/>
224−Knowledge of filesystem changes, requested by `/~/on` and `/~/in`, is requested the `%wasp` note. It contains the hash token which idenitifes a set of dependencies to query. Saved as a (live[live](#live)#live] request in case of cancellation when caused by `/~/on`.
225
318
226−Signs a `%writ` upon change.
319+Knowledge of filesystem changes, requested by `/~/on` and `/~/in`, is requested
320+the `%wasp` note. It contains the hash token which idenitifes a set of
321+dependencies to query. Saved as a (live[live](#live)\#live\] request in case of
322+cancellation when caused by `/~/on`.
323+
324+Signs a `%writ` upon change.
227
325
228## 5.4 `[%f %exec (unit silk)]`, functional transformations
326## 5.4 `[%f %exec (unit silk)]`, functional transformations
229−The `%f` interface converts nouns from one form to another. An empty unit represents cancellation.
327+
328+The `%f` interface converts nouns from one form to another. An empty unit
329+represents cancellation.
230
330
231Signs a `%made` containing the computation result.
331Signs a `%made` containing the computation result.
232
332
233−The `silk`s used are as follows:
333+The `silk`s used are as follows:
234
334
235### `[%cast %mime %boil mark beam [%web span ~]]`, file load
335### `[%cast %mime %boil mark beam [%web span ~]]`, file load
236−The simplest functional request is the construction of a page. Saved as a [live](#live) request in case of cancellation.
336+
237−- The `mark` is the request extension, defaulting to `%html`.
337+The simplest functional request is the construction of a page. Saved as a
238−- A `beam`(path in `%clay`) is decoded from the request path.
338+[live](#live) request in case of cancellation. - The `mark` is the request
239−- A `/web/<nyp ced quy>` virtual path is appended, span-encoding method, auth, and query string.
339+extension, defaulting to `%html`. - A `beam`(path in `%clay`) is decoded from
340+the request path. - A `/web/<nyp ced quy>` virtual path is appended,
341+span-encoding method, auth, and query string.
240
342
241### `[%cast mark %done ~ cage]`, convert <a id="done"/>
343### `[%cast mark %done ~ cage]`, convert <a id="done"/>
242−This is used when communicating with apps, in both directions.
344+
243−- Client messages are encoded as JSON objects, but can contain many different marks. Ones sent to non-`json.json` endpoints are `%cast` to he relevant mark, with a `cage` of `[%json p:!>(*json) jon]`, prior to being sent onwards to applications with `[%g %mess]`.
345+This is used when communicating with apps, in both directions. - Client messages
244−- Subscription responses also come in arbitrary marks, but are required to be sent as whichever mark an endpoint is subscribed to. They get `%cast` prior to being sent back as a `%that` EventSource segment.
346+are encoded as JSON objects, but can contain many different marks. Ones sent to
347+non-`json.json` endpoints are `%cast` to he relevant mark, with a `cage` of
348+`[%json p:!>(*json) jon]`, prior to being sent onwards to applications with
349+`[%g %mess]`. - Subscription responses also come in arbitrary marks, but are
350+required to be sent as whichever mark an endpoint is subscribed to. They get
351+`%cast` prior to being sent back as a `%that` EventSource segment.
245
352
246## 5.5 `%g`, app actions
353## 5.5 `%g`, app actions
247−The end goal of many a userspace `hymn.hook` is to provide UI for a `%gall` app. To accommodate this, various functionality needs to be interfaced with.
354+
355+The end goal of many a userspace `hymn.hook` is to provide UI for a `%gall` app.
356+To accommodate this, various functionality needs to be interfaced with.
248
357
249### `[%mess hapt ship cage]`, app message <a id="mess"/>
358### `[%mess hapt ship cage]`, app message <a id="mess"/>
250−After a `/~/to` POST has been received, and possibly converted to the correct mark, it is sent to `%g` for processing. The `hapt` is the destination, the `ship` is the source, the `cage` is the marked message data.
359+
360+After a `/~/to` POST has been received, and possibly converted to the correct
361+mark, it is sent to `%g` for processing. The `hapt` is the destination, the
362+`ship` is the source, the `cage` is the marked message data.
251
363
252Signs a `%nice` or `%mean`, in userspace or upon crashing.
364Signs a `%nice` or `%mean`, in userspace or upon crashing.
253
365
254### `[%show hapt ship path]`, app subscription<a id="show"/>
366### `[%show hapt ship path]`, app subscription<a id="show"/>
255−An `/~/is` PUT, if not already registered, results in a new subscription. A request for such contains the destination `hapt`, requesting `ship`, and app-internal `path`.
256
367
257−Signs a `%nice` on init, `%rush` on data, and a `%mean` in subscription termination.
368+An `/~/is` PUT, if not already registered, results in a new subscription. A
369+request for such contains the destination `hapt`, requesting `ship`, and
370+app-internal `path`.
371+
372+Signs a `%nice` on init, `%rush` on data, and a `%mean` in subscription
373+termination.
258
374
259### `[%took hapt ship]`, %rush confirmation
375### `[%took hapt ship]`, %rush confirmation
260−A `%took` must be sent to acknowledge the receipt of subscription data, which in this case occurs automatically.
376+
377+A `%took` must be sent to acknowledge the receipt of subscription data, which in
378+this case occurs automatically.
261
379
262Causes no signs.
380Causes no signs.
263
381
264### `[%nuke hapt $|(ship [ship path])]`, subscription cancel
382### `[%nuke hapt $|(ship [ship path])]`, subscription cancel
265−An `/~/is` DELETE is used to remove a subscription, and is converted straightforwardly to a `%nuke`. Additionally, when a client disconnects, all of its subscriptions are `%nuke`d to reflect their experiation.
383+
384+An `/~/is` DELETE is used to remove a subscription, and is converted
385+straightforwardly to a `%nuke`. Additionally, when a client disconnects, all of
386+its subscriptions are `%nuke`d to reflect their experiation.
266
387
267Signs an empty `%mean` for each open subscription that is closed.
388Signs an empty `%mean` for each open subscription that is closed.
268
389
269# 6. In-flight metadata <a id="whir"/>
390# 6. In-flight metadata <a id="whir"/>
270−Various state can be associated with requests, but not necessarily be returned in responses to them.
391+
392+Various state can be associated with requests, but not necessarily be returned
393+in responses to them.
271
394
272## 6.1 `~`, dropthrough <a id="wire-drop"/>
395## 6.1 `~`, dropthrough <a id="wire-drop"/>
273−If the response should be sent statelessly further up the duct, the `wire` is empty. This is used in `%f` pure functional page generation.
396+
397+If the response should be sent statelessly further up the duct, the `wire` is
398+empty. This is used in `%f` pure functional page generation.
274
399
275## 6.2 `[?(%y %n) ...]`, stability <a id="wire-live"/>
400## 6.2 `[?(%y %n) ...]`, stability <a id="wire-live"/>
276−The first element of a [`%b` timer](#wait) path distinguishes between live(%y) and dying(%n) channels.
401+
402+The first element of a [`%b` timer](#wait) path distinguishes between live(%y)
403+and dying(%n) channels.
277
404
278## 6.3 `/of/[ixor]`, view
405## 6.3 `/of/[ixor]`, view
406+
279Designates which stream to act upon. Present on `%b` timer cards.
407Designates which stream to act upon. Present on `%b` timer cards.
280
408
281## 6.4 `/on/<deps>`
409## 6.4 `/on/<deps>`
282−Present on `%f` dependency news. Looked up in [the state](#bolo) to see if any response is necessary.
410+
411+Present on `%f` dependency news. Looked up in [the state](#bolo) to see if any
412+response is necessary.
283
413
284## 6.5 `/to/<hasp>/<ship>`, app <a id="wire-to"/>
414## 6.5 `/to/<hasp>/<ship>`, app <a id="wire-to"/>
415+
285Present on `%f` translations of message marks
416Present on `%f` translations of message marks
286
417
287## 6.6 `/is/[ixor]/{hasp path}`, subscription <a id="wire-is"/>
418## 6.6 `/is/[ixor]/{hasp path}`, subscription <a id="wire-is"/>
288−Subscription requests. Present on `%g` subscription requests, and `%f` translations of [`%rush` subscription data](#show).
419+
420+Subscription requests. Present on `%g` subscription requests, and `%f`
421+translations of [`%rush` subscription data](#show).
289
422
290# 7. Vane responses <a id="sign"/>
423# 7. Vane responses <a id="sign"/>
291−The goal of requests is to cause some manner of result. Specifically,
424+
425+The goal of requests is to cause some manner of result. Specifically,
292
426
293## 7.1 `[%a %woot ship coop]`, acknowledgement
427## 7.1 `[%a %woot ship coop]`, acknowledgement
428+
294A [message](#mess) which has been acknowledged remotely.
429A [message](#mess) which has been acknowledged remotely.
295
430
296## 7.2 `[%b %wake ~]`, timer
431## 7.2 `[%b %wake ~]`, timer
297−Timer activate. If this is a `%y` [timer](#wire-live), send a newline to each `/~/of/[oryx]`; if `%n` and the `oryx` still has no listeners, wipe its state/subscriptions.
432+
433+Timer activate. If this is a `%y` [timer](#wire-live), send a newline to each
434+`/~/of/[oryx]`; if `%n` and the `oryx` still has no listeners, wipe its
435+state/subscriptions.
298
436
299## 7.3 `[%f %news ~]`, dependency update
437## 7.3 `[%f %news ~]`, dependency update
300−This message signifies a `hash` has changed, and listeners to it should be updated. Results in [`%news` events](#even-news) to concerned streams, and [full responses](#thou) to relevant long-polls.
438+
439+This message signifies a `hash` has changed, and listeners to it should be
440+updated. Results in [`%news` events](#even-news) to concerned streams, and [full
441+responses](#thou) to relevant long-polls.
301
442
302## 7.4 `[%f %made (each bead (list tank))]`, compute result
443## 7.4 `[%f %made (each bead (list tank))]`, compute result
303−Further action depends on the [`wire`](#whir). It can:
444+
304−- Be empty, designating direct request data, which is [served](#mime) back up the `duct`.
445+Further action depends on the [`wire`](#whir). It can: - Be empty, designating
305−- Contain [app and requester identities](#wire-to), which designates an app message has been converted to the correct `mark`, and can now be [sent to the app](#mess) with an empty `wire`.
446+direct request data, which is [served](#mime) back up the `duct`. - Contain [app
306−- Contain an `oryx`, app, and `path`, designating subscription data. If the subscription is active in the relevant `stem`, it is acknowledged with a `%took` and given as a [`%rush` event](#even-rush) to all listeners. Otherwise, the subscription is `%nuke`d.
447+and requester identities](#wire-to), which designates an app message has been
448+converted to the correct `mark`, and can now be [sent to the app](#mess) with an
449+empty `wire`. - Contain an `oryx`, app, and `path`, designating subscription
450+data. If the subscription is active in the relevant `stem`, it is acknowledged
451+with a `%took` and given as a [`%rush` event](#even-rush) to all listeners.
452+Otherwise, the subscription is `%nuke`d.
307
453
308## 7.5 `%g`, applications
454## 7.5 `%g`, applications
455+
309The `%g` interface sends several responses.
456The `%g` interface sends several responses.
310
457
311### `[%nice ~]` and `[%mean ares]`, acknowledgment
458### `[%nice ~]` and `[%mean ares]`, acknowledgment
312−These are given as a response to `%mess`, with an empty wire, and `%show`s, with a [subscription wire](#wire-is).
459+
313−The former is [served](#mime) back to the requester(i.e. the remaining duct), and the latter returned to the client.
460+These are given as a response to `%mess`, with an empty wire, and `%show`s, with
461+a [subscription wire](#wire-is). The former is [served](#mime) back to the
462+requester(i.e. the remaining duct), and the latter returned to the client.
314
463
315### 7.5 `[%rush cage]`, subscription data
464### 7.5 `[%rush cage]`, subscription data
316−A `%rush` arrives on a [subscription wire](#wire-is), and is then sent [for conversion](#done) to the correct mark. An unexpected `%rush` is `%nuke`d.
465+
466+A `%rush` arrives on a [subscription wire](#wire-is), and is then sent [for
467+conversion](#done) to the correct mark. An unexpected `%rush` is `%nuke`d.
317
468
318# 8. Server state <a id="bolo"/>
469# 8. Server state <a id="bolo"/>
470+
319There is a quantity of data that persists between events.
471There is a quantity of data that persists between events.
320
472
321## 8.1 Global
473## 8.1 Global
322−- `path` <a id="prefix"/> the default path which is prepended to relative plain requests. Initialized to `/<our>`, the serving ship; this interprets the first element of requests as a desk, and injects a `case` of `0`.
474+
323−- `(jar ship hart)` remembered hostnames in order of preference, made accessible through `.^`
475+- `path` <a id="prefix"/> the default path which is prepended to relative
324− + The default values(for self) are `http://<ship>.urbit.org:80` if not on a fake network, followed by `https://0.0.0.0:[port]`.
476+ plain requests. Initialized to `/<our>`, the serving ship; this interprets
325−- `(jug deps (each duct oryx))`, `/~/on` and `/~/in` endpoints listening to `%c` updates.
477+ the first element of requests as a desk, and injects a `case` of `0`.
326−- `(map duct live)`, inbound request state
478+- `(jar ship hart)` remembered hostnames in order of preference, made
327−- `(map ixor oryx)`, open `/~/of` streams.
479+ accessible through `.^`
328−- `(map oryx stem)`, open view state.
480+- The default values(for self) are `http://<ship>.urbit.org:80` if not on a
329−- `(map hole sink)`, session state.
481+ fake network, followed by `https://0.0.0.0:[port]`.
330−- `(map hole ,[ship ?])`, foreign session names, along with their origin ships and whether they are authorized to act on our behalf.
482+- `(jug deps (each duct oryx))`, `/~/on` and `/~/in` endpoints listening to
483+ `%c` updates.
484+- `(map duct live)`, inbound request state
485+- `(map ixor oryx)`, open `/~/of` streams.
486+- `(map oryx stem)`, open view state.
487+- `(map hole sink)`, session state.
488+- `(map hole ,[ship ?])`, foreign session names, along with their origin ships
489+ and whether they are authorized to act on our behalf.
331
490
332## 8.2 `live`, per-request state <a id="live"/>
491## 8.2 `live`, per-request state <a id="live"/>
333−To honor request cancellations, each unserved request must track which effect it is causing.
492+
334−+ `[%exec whir]` if it is a ford data request
493+To honor request cancellations, each unserved request must track which effect it
335−+ `[%wasp (list @uvH)]` if it is a ford dependency poll
494+is causing. + `[%exec whir]` if it is a ford data request +
336−+ `[%xeno ship]` if this is a proxied request
495+`[%wasp (list @uvH)]` if it is a ford dependency poll + `[%xeno ship]` if this
337−+ `[%poll ixor]` if this is a session long-poll
496+is a proxied request + `[%poll ixor]` if this is a session long-poll
338
497
339## 8.3 `sink`, per-session state <a id="sink"/>
498## 8.3 `sink`, per-session state <a id="sink"/>
340−Each `hole` has associated authentication state.
499+
341−- `priv`, a random secret that verifies the session, stored in cookie.
500+Each `hole` has associated authentication state. - `priv`, a random secret that
342−- `[ship (set ship)]`, the `ship` a session is acting as and any others it has been authorized with.
501+verifies the session, stored in cookie. - `[ship (set ship)]`, the `ship` a
343−- `(jug ship ,[path duct])`, authentication `/~/as` requests waiting on foreign ships.
502+session is acting as and any others it has been authorized with. -
344−- `(set oryx)`, views associated with this session. Dual of `hole` in `stem`.
503+`(jug ship ,[path duct])`, authentication `/~/as` requests waiting on foreign
504+ships. - `(set oryx)`, views associated with this session. Dual of `hole` in
505+`stem`.
345
506
346## 8.4 `stem`, per-view state <a id="stem"/>
507## 8.4 `stem`, per-view state <a id="stem"/>
347−Each `oryx` has active subscription state.
508+
348−- `hole`, session in which this view resides. Dual of `(set oryx)` in `sink`.
509+Each `oryx` has active subscription state. - `hole`, session in which this view
349−- `ixor`<a id="ixor"/>, a cached hash of the `oryx` that is used as a subscription id, by the `/~/of` EventStream.
510+resides. Dual of `(set oryx)` in `sink`. - `ixor`<a id="ixor"/>, a cached hash
350−- `[,@u (map ,@u even)]`, queued [events](#even). Has a maximum size.
511+of the `oryx` that is used as a subscription id, by the `/~/of` EventStream. -
351−- `(unit ,[duct @u ?])`, http connection, its last received event, and whether it is a long-poll request.
512+`[,@u (map ,@u even)]`, queued [events](#even). Has a maximum size. -
352−- `(set deps)`, subscribed `/~/in` points. Mirrored in `bolo`.
513+`(unit ,[duct @u ?])`, http connection, its last received event, and whether it
353−- `(map ,[hasp path] mark)`, subscribed `/~/is` points
514+is a long-poll request. - `(set deps)`, subscribed `/~/in` points. Mirrored in
354−- `(que duct)`, where any `%nice`/`%mean
515+`bolo`. - `(map ,[hasp path] mark)`, subscribed `/~/is` points - `(que duct)`,
516+where any `%nice`/\`%mean
355
517
356# Appendix A: Glossary
518# Appendix A: Glossary
357−- An `oryx` is a CSRF token used for [authenticated requests](#auth)
519+
358−- `mean.json`<a id="mean-json"/> is a rendering of hoon `ares`: an error message formatted as `{fail:'type',mess:"Error message"}`
520+- An `oryx` is a CSRF token used for [authenticated requests](#auth)
359−- `urb.js`<a id="urb-js"/> is the standard client-side implementation of the `%eyre` protocols, normally found in `/=main=/lib`
521+- `mean.json`<a id="mean-json"/> is a rendering of hoon `ares`: an error
360−- The act of "serving"<a id="mime"/>, refers to the wrapping of a `%mime` cage in an HTTP `200 Success` response, and errors or other marks being [sent to ford](#done) for conversion. Inside a `sign`, this conversion occurs along the same [`wire`](#wire); otherwise, the `wire` is [empty](#wire-drop).
522+ message formatted as `{fail:'type',mess:"Error message"}`
523+- `urb.js`<a id="urb-js"/> is the standard client-side implementation of the
524+ `%eyre` protocols, normally found in `/=main=/lib`
525+- The act of "serving"<a id="mime"/>, refers to the wrapping of a `%mime` cage
526+ in an HTTP `200 Success` response, and errors or other marks being [sent to
527+ ford](#done) for conversion. Inside a `sign`, this conversion occurs along
528+ the same [`wire`](#wire); otherwise, the `wire` is [empty](#wire-drop).
361
529
362# Appendix B: EventSource <a id="eventsource"/>
530# Appendix B: EventSource <a id="eventsource"/>
363
531
@@ -365,12 +533,10 @@ UNIMPLEMENTED
365
533
366For more information, see <http://dev.w3.org/html5/eventsource/>
534For more information, see <http://dev.w3.org/html5/eventsource/>
367
535
368−An EventSource is a stream of events, encoded in a conceptually infinite `GET` request. Each event is formatted as
536+An EventSource is a stream of events, encoded in a conceptually infinite `GET`
369− ```http
537+request. Each event is formatted as
370− id: {number}
538+`http id: {number} event: {type} data: {contents} data: {more-contents}`
371− event: {type}
372− data: {contents}
373− data: {more-contents}
374− ```
375
539
376−Data lines are joined by ASCII `10` newline characters; multiple newlines signify separate events. The `id` is used in a `Last-Event-ID` HTTP request header upon reconnection.
540+Data lines are joined by ASCII `10` newline characters; multiple newlines
541+signify separate events. The `id` is used in a `Last-Event-ID` HTTP request
542+header upon reconnection.
docs/arvo/internals/ford.md+47−55
@@ -1,53 +1,47 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 6
3sort: 6
4---
4---
5
5
6−
6+# `%ford`
7−`%ford`
8−=======
9
7
10Our typed and marked computation engine.
8Our typed and marked computation engine.
11
9
12−A variety of different services are provided by `%ford`, but they mostly
10+A variety of different services are provided by `%ford`, but they mostly involve
13−involve compiling hook files, slapping/slammming code with marked data,
11+compiling hook files, slapping/slammming code with marked data, and converting
14−and converting data between marks, including validating data to a mark.
12+data between marks, including validating data to a mark. Throughout every
15−Throughout every computation, `%ford` keeps track of which resources are
13+computation, `%ford` keeps track of which resources are dependencies so that the
16−dependencies so that the client may be aware when one or more
14+client may be aware when one or more dependencies are updated.
17−dependencies are updated.
18
15
19−`%ford` neither accepts unix events nor produces effects. It exists
16+`%ford` neither accepts unix events nor produces effects. It exists entirely for
20−entirely for the benefit of applications and other vanes, in particular
17+the benefit of applications and other vanes, in particular `%gall`. `%eyre`
21−`%gall`. `%eyre` exposes the functional publishing aspects of `%ford`
18+exposes the functional publishing aspects of `%ford` while `%gall` uses `%ford`
22−while `%gall` uses `%ford` to control the execution of applications.
19+to control the execution of applications. `%clay` is intended to use `%ford` to
23−`%clay` is intended to use `%ford` to managed marked data, but this is
20+managed marked data, but this is not yet reality.
24−not yet reality.
25
21
26<list></list>
22<list></list>
27
23
28−Cards
24+## Cards
29−-----
30
25
31−`%ford` accepts just one card, `%exec`. This is misleading, however,
26+`%ford` accepts just one card, `%exec`. This is misleading, however, since there
32−since there are fourteen different `silk`s that may be used with it. In
27+are fourteen different `silk`s that may be used with it. In every case, the
33−every case, the expected response to a `%exec` card is a `%made` gift
28+expected response to a `%exec` card is a `%made` gift with either an error or
34−with either an error or the produced result along with its set of
29+the produced result along with its set of dependencies.
35−dependencies.
36
30
37−Silks may autocons, so that the product of a cell of silks is a cell of
31+Silks may autocons, so that the product of a cell of silks is a cell of the
38−the product of the two silks.
32+product of the two silks.
39
33
40### `%bake`
34### `%bake`
41
35
42−Tries to functionally produce the file at a given beam with the given
36+Tries to functionally produce the file at a given beam with the given mark and
43−mark and heel. It fails if there is no way to translate at this level.
37+heel. It fails if there is no way to translate at this level.
44
38
45### `%boil`
39### `%boil`
46
40
47−Functionally produces the file at a given beam with the given mark and
41+Functionally produces the file at a given beam with the given mark and heel. If
48−heel. If there is no way to translate at this beam, we pop levels off
42+there is no way to translate at this beam, we pop levels off the stack and
49−the stack and attempt to bake there until we find a level we can bake.
43+attempt to bake there until we find a level we can bake. This should almost
50−This should almost always be called instead of `%bake`.
44+always be called instead of `%bake`.
51
45
52### `%call`
46### `%call`
53
47
@@ -55,52 +49,50 @@ Slams the result of one silk against the result of another.
55
49
56### `%cast`
50### `%cast`
57
51
58−Translates the given silk to the given mark, if possible. This is one of
52+Translates the given silk to the given mark, if possible. This is one of the
59−the critical and fundamental operations of ford.
53+critical and fundamental operations of ford.
60
54
61### `%diff`
55### `%diff`
62
56
63−Diffs the two given silks (which must be of the same mark), producing a
57+Diffs the two given silks (which must be of the same mark), producing a cage of
64−cage of the mark specified in `++mark` in `++grad` for the mark of the
58+the mark specified in `++mark` in `++grad` for the mark of the two silks.
65−two silks.
66
59
67### `%done`
60### `%done`
68
61
69−Produces exactly its input. This is rarely used on its own, but many
62+Produces exactly its input. This is rarely used on its own, but many silks are
70−silks are recursively defined in terms of other silks, so we often need
63+recursively defined in terms of other silks, so we often need a silk that simply
71−a silk that simply produces its input. A monadic return, if you will.
64+produces its input. A monadic return, if you will.
72
65
73### `%dude`
66### `%dude`
74
67
75−Computes the given silk with the given tank as part of the stack trace
68+Computes the given silk with the given tank as part of the stack trace if there
76−if there is an error.
69+is an error.
77
70
78### `%dune`
71### `%dune`
79
72
80−Produces an error if the cage is empty. Otherwise, it produces the value
73+Produces an error if the cage is empty. Otherwise, it produces the value in the
81−in the unit.
74+unit.
82
75
83### `%mute`
76### `%mute`
84
77
85−Takes a silk and a list of changes to make to the silk. At each wing in
78+Takes a silk and a list of changes to make to the silk. At each wing in the list
86−the list we put the value of the associated silk.
79+we put the value of the associated silk.
87
80
88### `%pact`
81### `%pact`
89
82
90−Applies the second silk as a patch to the first silk. The second silk
83+Applies the second silk as a patch to the first silk. The second silk must be of
91−must be of the mark specified in `++mark` in `++grad` for the mark of
84+the mark specified in `++mark` in `++grad` for the mark of the first silk.
92−the first silk.
93
85
94### `%plan`
86### `%plan`
95
87
96−Performs a structured assembly directly. This is not generally directly
88+Performs a structured assembly directly. This is not generally directly useful
97−useful because several other silks perform supersets of this
89+because several other silks perform supersets of this functionality. We don't
98−functionality. We don't usually have naked hoods outside ford.
90+usually have naked hoods outside ford.
99
91
100### `%reef`
92### `%reef`
101
93
102−Produces a core containing the entirety of zuse and hoon, suitable for
94+Produces a core containing the entirety of zuse and hoon, suitable for running
103−running arbitrary code against. The mark is `%noun`.
95+arbitrary code against. The mark is `%noun`.
104
96
105### `%ride`
97### `%ride`
106
98
@@ -108,5 +100,5 @@ Slaps a twig against a subject silk. The mark of the result is `%noun`.
108
100
109### `%vale`
101### `%vale`
110
102
111−Validates untyped data from a ship against a given mark. This is an
103+Validates untyped data from a ship against a given mark. This is an extremely
112−extremely useful function.
104+useful function.
docs/arvo/internals/ford/commentary.md+852−956
@@ -1,13 +1,10 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3---
3---
4
4
5+# Reference
5
6
6−Reference
7+## Data Models
7−=========
8−
9−Data Models
10−-----------
11
8
12### `++axle`, formal state
9### `++axle`, formal state
13
10
@@ -16,13 +13,13 @@ Data Models
16 pol=(map ship baby) ::
13 pol=(map ship baby) ::
17 == ::
14 == ::
18
15
19−This is the formal state of our vane. Anything that must be remembered
16+This is the formal state of our vane. Anything that must be remembered between
20−between calls to ford must be stored here. The number `%1` is a version
17+calls to ford must be stored here. The number `%1` is a version number for our
21−number for our state that allows us to upgrade the structure of our
18+state that allows us to upgrade the structure of our state in the future if we
22−state in the future if we wish.
19+wish.
23
20
24−`pol` is the a map from every ship on our pier to their individual ford
21+`pol` is the a map from every ship on our pier to their individual ford state.
25−state. There is no shared ford state -- every ship is entirely separate.
22+There is no shared ford state -- every ship is entirely separate.
26
23
27### `++baby`, state by ship
24### `++baby`, state by ship
28
25
@@ -34,19 +31,18 @@ state. There is no shared ford state -- every ship is entirely separate.
34
31
35This is the state specific to each ship.
32This is the state specific to each ship.
36
33
37−`tad` and `dym` keep track of the tasks we're currently working on.
34+`tad` and `dym` keep track of the tasks we're currently working on. `dym` is a
38−`dym` is a map from ducts to task numbers, and `q.tad` is a map from
35+map from ducts to task numbers, and `q.tad` is a map from task number to the
39−task number to the task itself. `p.tad` is the next available task
36+task itself. `p.tad` is the next available task number. Thus, the keys of
40−number. Thus, the keys of `q.tad` are a subset of the numbers less than
37+`q.tad` are a subset of the numbers less than `p.tad`, and ford has attempted
41−`p.tad`, and ford has attempted exactly `p.tad` tasks so far.
38+exactly `p.tad` tasks so far.
42
39
43−`jav` is the cache of previously-solved problems. The keys are a pair of
40+`jav` is the cache of previously-solved problems. The keys are a pair of a term
44−a term (either `%hood`, `%slap`, or `%slam`) and a noun that represents
41+(either `%hood`, `%slap`, or `%slam`) and a noun that represents the exact
45−the exact problem solved. In the case of a `%hood`, then, the key is of
42+problem solved. In the case of a `%hood`, then, the key is of the form
46−the form `[%hood beam cage]`. For `%slap`, there is `[%slap vase twig]`.
43+`[%hood beam cage]`. For `%slap`, there is `[%slap vase twig]`. For `%slam`,
47−For `%slam`, there is `[%slam vase vase]`. The values are the result of
44+there is `[%slam vase vase]`. The values are the result of solving the problem.
48−solving the problem. Note that this cache is wiped in `++stay` when ford
45+Note that this cache is wiped in `++stay` when ford is reloaded.
49−is reloaded.
50
46
51### `++task`, problem in progress
47### `++task`, problem in progress
52
48
@@ -56,12 +52,12 @@ is reloaded.
56 kig=[p=@ud q=(map ,@ud beam)] :: blocks
52 kig=[p=@ud q=(map ,@ud beam)] :: blocks
57 == ::
53 == ::
58
54
59−This is all the state we keep regarding a particular task. `nah` is the
55+This is all the state we keep regarding a particular task. `nah` is the duct
60−duct which asked us to solve the problem, and `kas` is the most recent
56+which asked us to solve the problem, and `kas` is the most recent statement of
61−statement of the problem.
57+the problem.
62
58
63−`kig` keeps track of which resources we are blocked on. Our blocks are
59+`kig` keeps track of which resources we are blocked on. Our blocks are stored by
64−stored by index in `q.kig`, and the next available index is `p.kig`.
60+index in `q.kig`, and the next available index is `p.kig`.
65
61
66### `++silk`, problem
62### `++silk`, problem
67
63
@@ -83,13 +79,12 @@ stored by index in `q.kig`, and the next available index is `p.kig`.
83 [%vale p=mark q=ship r=*] :: validate [our his]
79 [%vale p=mark q=ship r=*] :: validate [our his]
84 == ::
80 == ::
85
81
86−This is the every type of problem we can solve. Every `%exec` kiss that
82+This is the every type of problem we can solve. Every `%exec` kiss that requests
87−requests us to solve a problem must choose one of these problems to
83+us to solve a problem must choose one of these problems to solve.
88−solve.
89
84
90−Because this is both an internal structure used in ford and the public
85+Because this is both an internal structure used in ford and the public interface
91−interface to ford, we choose to document this structure in our
86+to ford, we choose to document this structure in our discussion of the public
92−discussion of the public interface to ford below.
87+interface to ford below.
93
88
94### `++calx`, cache line
89### `++calx`, cache line
95
90
@@ -99,20 +94,19 @@ discussion of the public interface to ford below.
99 [%slam p=calm q=[p=vase q=vase] r=vase] :: compute
94 [%slam p=calm q=[p=vase q=vase] r=vase] :: compute
100 == ::
95 == ::
101
96
102−There are three kinds of cache entries. Every entry includes some
97+There are three kinds of cache entries. Every entry includes some metadata in
103−metadata in `p` and is the combination of an input and its output.
98+`p` and is the combination of an input and its output.
104
99
105−The input to a `%hood` is the location of the resource and a cage
100+The input to a `%hood` is the location of the resource and a cage representing
106−representing the data at that location. The output is the hood found by
101+the data at that location. The output is the hood found by compiling the given
107−compiling the given cage at the given location.
102+cage at the given location.
108
103
109−The input to a `%slap` is the vase of the subject and the twig of the
104+The input to a `%slap` is the vase of the subject and the twig of the formula
110−formula against which we are slapping the subject. The output is the
105+against which we are slapping the subject. The output is the vase produced by
111−vase produced by slapping them.
106+slapping them.
112
107
113−The input to a `%slam` is the vase of the subject and the vase of the
108+The input to a `%slam` is the vase of the subject and the vase of the gate which
114−gate which we are slapping. The output is the vase produced by slamming
109+we are slapping. The output is the vase produced by slamming them.
115−them.
116
110
117### `++calm`, cache line metadata
111### `++calm`, cache line metadata
118
112
@@ -121,9 +115,9 @@ them.
121 dep=(set beam) :: dependencies
115 dep=(set beam) :: dependencies
122 == ::
116 == ::
123
117
124−Every line in the cache needs to have two pieces of metadata. We must
118+Every line in the cache needs to have two pieces of metadata. We must know the
125−know the last time this line in the cache was accessed, and we must know
119+last time this line in the cache was accessed, and we must know what are the
126−what are the dependencies of this line.
120+dependencies of this line.
127
121
128### `++hood`, assembly components
122### `++hood`, assembly components
129
123
@@ -137,46 +131,43 @@ what are the dependencies of this line.
137
131
138When assembling a hook file, we split it into several sections.
132When assembling a hook file, we split it into several sections.
139
133
140−`zus` is the kelvin version of the required zuse. In general, we assume
134+`zus` is the kelvin version of the required zuse. In general, we assume that any
141−that any newer (lower-numbered) zuse will retain backward compatibility,
135+newer (lower-numbered) zuse will retain backward compatibility, so any newer
142−so any newer zuse is also permissible. This number is set with a `/?` at
136+zuse is also permissible. This number is set with a `/?` at the beginning of the
143−the beginning of the file.
137+file.
144
138
145−`sur` is the set of structures included. These are included with the
139+`sur` is the set of structures included. These are included with the `/-` rune.
146−`/-` rune. When a structure is included, we look in `/=main=/sur` for
140+When a structure is included, we look in `/=main=/sur` for the given structure
147−the given structure and we load the gate there. When compiling, all the
141+and we load the gate there. When compiling, all the included structures are
148−included structures are collected into a single core placed in the
142+collected into a single core placed in the subject of the body with a `=>`.
149−subject of the body with a `=>`.
150
143
151−`lib` is the set of librarires included. These are included with the
144+`lib` is the set of librarires included. These are included with the `/+` rune.
152−`/+` rune. When a library is included, we look in `/=main=/lib` for the
145+When a library is included, we look in `/=main=/lib` for the given library and
153−given library and we load the library there. As with structures, all the
146+we load the library there. As with structures, all the included libraries are
154−included libraries are collected into a single core placed in the
147+collected into a single core placed in the subject of the body with a `=>`.
155−subject of the body with a `=>`.
156
148
157−`fan` is the set of resources included. These are loaded in many
149+`fan` is the set of resources included. These are loaded in many different ways
158−different ways and may load resources from any location. These are
150+and may load resources from any location. These are placed in the subject of the
159−placed in the subject of the body with a `=~`.
151+body with a `=~`.
160
152
161−`src` is the set of twigs or references to twigs in the body of the
153+`src` is the set of twigs or references to twigs in the body of the program.
162−program. Generally, each of these will represent a core, but this is not
154+Generally, each of these will represent a core, but this is not required. When
163−required. When compiling, these are strung together in a `=~`.
155+compiling, these are strung together in a `=~`.
164
156
165### `++hoot`
157### `++hoot`
166
158
167 ++ hoot (pair bean hoof) :: structure gate/core
159 ++ hoot (pair bean hoof) :: structure gate/core
168
160
169−A structures may be either a direct gate or a core. These are
161+A structures may be either a direct gate or a core. These are syntactically
170−syntactically distinguished by adding a `*` to the beginning of the
162+distinguished by adding a `*` to the beginning of the structure name for a core.
171−structure name for a core. The structure itself is a `hoof`.
163+The structure itself is a `hoof`.
172
164
173### `++hoof`
165### `++hoof`
174
166
175 ++ hoof (pair term (unit (pair case ship))) :: resource reference
167 ++ hoof (pair term (unit (pair case ship))) :: resource reference
176
168
177−A hoof, which is either a structure or a library, has a name, and it may
169+A hoof, which is either a structure or a library, has a name, and it may also
178−also specify which version of the resource to use and which ship to
170+specify which version of the resource to use and which ship to retrieve it from.
179−retrieve it from.
180
171
181### `++horn`
172### `++horn`
182
173
@@ -202,44 +193,42 @@ discussion of their use from a user's perspective is documented in the [runes
202documentation](https://urbit.org/docs/arvo/internals/runes), so we will here
193documentation](https://urbit.org/docs/arvo/internals/runes), so we will here
203only give a description of the data structure itself.
194only give a description of the data structure itself.
204
195
205−A `%ape` horn is simply a twig that gets evaluated and placed in the
196+A `%ape` horn is simply a twig that gets evaluated and placed in the subject.
206−subject.
207
197
208−A `%arg` is a gate that gets evaluated with a sample of our location and
198+A `%arg` is a gate that gets evaluated with a sample of our location and our
209−our heel.
199+heel.
210
200
211−A `%day` is a horn that applies to each of a list of `@dr`-named files
201+A `%day` is a horn that applies to each of a list of `@dr`-named files in the
212−in the directory.
202+directory.
213
203
214−A `%dub` is a term and horn, where the result of a horn is given the
204+A `%dub` is a term and horn, where the result of a horn is given the face of the
215−face of the term.
205+term.
216
206
217A `%fan` is a list of horns, all at the current directory level.
207A `%fan` is a list of horns, all at the current directory level.
218
208
219−A `%for` is a path and a horn, where the horn is evaluated relative to
209+A `%for` is a path and a horn, where the horn is evaluated relative to the given
220−the given path, where the given path is relative to the current
210+path, where the given path is relative to the current location.
221−location.
222
211
223−A `%hub` is a horn that applies to each of a list of `@ud`-named files
212+A `%hub` is a horn that applies to each of a list of `@ud`-named files in the
224−in the directory.
213+directory.
225
214
226−A `%man` is a map of spans to horns where the result is a set of each
215+A `%man` is a map of spans to horns where the result is a set of each horn
227−horn applied to the current directory given the associated face.
216+applied to the current directory given the associated face.
228
217
229−A `%nap` is a homogenous map where each entry in a directory is handled
218+A `%nap` is a homogenous map where each entry in a directory is handled with the
230−with the same horn and is given a face according to its name.
219+same horn and is given a face according to its name.
231
220
232−A `%now` is a horn that applies to each of a list of `@da`-named files
221+A `%now` is a horn that applies to each of a list of `@da`-named files in the
233−in the directory.
222+directory.
234
223
235−A `%saw` is a twig and a horn, where the twig operates on the result of
224+A `%saw` is a twig and a horn, where the twig operates on the result of the
236−the horn.
225+horn.
237
226
238−A `%see` is a beam and a horn, where the horn is evaluated at a location
227+A `%see` is a beam and a horn, where the horn is evaluated at a location of the
239−of the given beam.
228+given beam.
240
229
241−A `%sic` is a tile and a horn, where the horn is evaluated and cast to
230+A `%sic` is a tile and a horn, where the horn is evaluated and cast to the type
242−the type associated with the tile.
231+associated with the tile.
243
232
244A `%toy` is simply a mark to be baked.
233A `%toy` is simply a mark to be baked.
245
234
@@ -250,9 +239,9 @@ A `%toy` is simply a mark to be baked.
250 [%| p=beam] :: resource location
239 [%| p=beam] :: resource location
251 == ::
240 == ::
252
241
253−This is an entry in the body of the hook file. The hoop can either be
242+This is an entry in the body of the hook file. The hoop can either be defined
254−defined directly in the given file or it can be a reference to another
243+directly in the given file or it can be a reference to another file. The second
255−file. The second is specified with a `//` rune.
244+is specified with a `//` rune.
256
245
257### `++bolt`, monadic edge
246### `++bolt`, monadic edge
258
247
@@ -266,28 +255,27 @@ file. The second is specified with a `//` rune.
266 == ::
255 == ::
267 == ::
256 == ::
268
257
269−Throughout our computation, we let our result flow through with the set
258+Throughout our computation, we let our result flow through with the set of
270−of dependencies of the value. At various times, we may wish to either
259+dependencies of the value. At various times, we may wish to either throw an
271−throw an error or declare that the actual result cannot be found until a
260+error or declare that the actual result cannot be found until a particular
272−particular resource is retrieved. This is a perfect case for a monad, so
261+resource is retrieved. This is a perfect case for a monad, so here we define a
273−here we define a data structure for it.
262+data structure for it.
274
263
275−At every step, we have a cache, so we store that in `p`. In `q` we store
264+At every step, we have a cache, so we store that in `p`. In `q` we store the
276−the data.
265+data.
277
266
278−In the case of `%0`, we have the result in `q` and the set of
267+In the case of `%0`, we have the result in `q` and the set of dependencies in
279−dependencies in `p`.
268+`p`.
280
269
281−In the case of `%1`, we have a set of dependencies on which we are
270+In the case of `%1`, we have a set of dependencies on which we are blocking.
282−blocking. When this happens, we make a call to clay to get the
271+When this happens, we make a call to clay to get the dependencies, and we
283−dependencies, and we proceed with the computation when we receive them.
272+proceed with the computation when we receive them. Technically, we restart the
284−Technically, we restart the computation, but since every expensive step
273+computation, but since every expensive step is cached, there is no significant
285−is cached, there is no significant performance penalty to doing this.
274+performance penalty to doing this. Referential transparency has its uses.
286−Referential transparency has its uses.
287
275
288−In the case of `%2`, we have a hit an error. This gets passed all the
276+In the case of `%2`, we have a hit an error. This gets passed all the way
289−way through to the calling duct. The list of tanks is some description
277+through to the calling duct. The list of tanks is some description of what went
290−of what went wrong, often including a stack trace.
278+wrong, often including a stack trace.
291
279
292### `++burg`, monadic rule
280### `++burg`, monadic rule
293
281
@@ -296,53 +284,49 @@ of what went wrong, often including a stack trace.
296 $+([c=cafe d=a] (bolt b)) ::
284 $+([c=cafe d=a] (bolt b)) ::
297 :: ::
285 :: ::
298
286
299−To operate on bolts, we use `++cope` as our bind operator, and the
287+To operate on bolts, we use `++cope` as our bind operator, and the functions it
300−functions it works on are of type `burg`. Our functions that operate on
288+works on are of type `burg`. Our functions that operate on bolts should have a
301−bolts should have a sample of the cache and a value. Their output should
289+sample of the cache and a value. Their output should be a bolt of the output
302−be a bolt of the output value. Then, `++cope` will only call the
290+value. Then, `++cope` will only call the function when necessary (in the `%0`
303−function when necessary (in the `%0` case), and it will do so without
291+case), and it will do so without the wrapping of a bolt.
304−the wrapping of a bolt.
305
292
306−If you understand monads, this is probably fairly obvious. Otherwise,
293+If you understand monads, this is probably fairly obvious. Otherwise, see the
307−see the discussion on `++cope` (link).
294+discussion on `++cope` (link).
308
295
309−Public Interface
296+## Public Interface
310−----------------
311
297
312−Ford does not export a scry interface, so the only way to interact with
298+Ford does not export a scry interface, so the only way to interact with ford is
313−ford is by sending kisses and receiving gifts. In fact, ford only sends
299+by sending kisses and receiving gifts. In fact, ford only sends accepts one kiss
314−accepts one kiss and gives one gift. This is, of course, misleading
300+and gives one gift. This is, of course, misleading because ford actually does
315−because ford actually does many different things. It does, however, only
301+many different things. It does, however, only produce one type of thing -- a
316−produce one type of thing -- a result of a computation, which is either
302+result of a computation, which is either an error or the value produced along
317−an error or the value produced along with the set of dependencies
303+with the set of dependencies referenced by it.
318−referenced by it.
319
304
320 ++ kiss :: in request ->$
305 ++ kiss :: in request ->$
321 $% [%exec p=@p q=(unit silk)] :: make / kill
306 $% [%exec p=@p q=(unit silk)] :: make / kill
322 == ::
307 == ::
323
308
324The `%exec` gift requests ford to perform a computation on behalf of a
309The `%exec` gift requests ford to perform a computation on behalf of a
325−particular ship. `p` is the ship, and `q` is the computation. If `q` is
310+particular ship. `p` is the ship, and `q` is the computation. If `q` is null,
326−null, then we are requesting that ford cancel the computation that it is
311+then we are requesting that ford cancel the computation that it is currently
327−currently being run along this duct. Thus, if you wish to cancel a
312+being run along this duct. Thus, if you wish to cancel a computation, you must
328−computation, you must send the kiss along the same duct as the original
313+send the kiss along the same duct as the original request.
329−request.
330
314
331−Otherwise, we ask ford to perform a certain computation, as defined in
315+Otherwise, we ask ford to perform a certain computation, as defined in `++silk`.
332−`++silk`. Since all computations produce the same type of result, we
316+Since all computations produce the same type of result, we will discuss that
333−will discuss that result before we jump into `++silk`.
317+result before we jump into `++silk`.
334
318
335 ++ gift :: out result <-$
319 ++ gift :: out result <-$
336 $% [%made p=(each bead (list tank))] :: computed result
320 $% [%made p=(each bead (list tank))] :: computed result
337 == ::
321 == ::
338
322
339−We give either a `bead`, which is a result, or a list of tanks, which is
323+We give either a `bead`, which is a result, or a list of tanks, which is an
340−an error messge, often including a stack trace.
324+error messge, often including a stack trace.
341
325
342 ++ bead ,[p=(set beam) q=cage] :: computed result
326 ++ bead ,[p=(set beam) q=cage] :: computed result
343
327
344−This is a set of dependencies required to compute this value and a cage
328+This is a set of dependencies required to compute this value and a cage of the
345−of the result with its associated mark.
329+result with its associated mark.
346
330
347There are twelve possible computations defined in `++silk`.
331There are twelve possible computations defined in `++silk`.
348
332
@@ -362,83 +346,77 @@ There are twelve possible computations defined in `++silk`.
362 [%vale p=mark q=ship r=*] :: validate [our his]
346 [%vale p=mark q=ship r=*] :: validate [our his]
363 == ::
347 == ::
364
348
365−First, we allow silks to autocons. A cell of silks is also a silk, and
349+First, we allow silks to autocons. A cell of silks is also a silk, and the
366−the product vase is a cell of the two silks. This obviously extends to
350+product vase is a cell of the two silks. This obviously extends to an arbitrary
367−an arbitrary number of silks.
351+number of silks.
368
352
369−`%bake` tries to functionally produce the file at a given beam with the
353+`%bake` tries to functionally produce the file at a given beam with the given
370−given mark and heel. It fails if there is no way to translate at this
354+mark and heel. It fails if there is no way to translate at this level.
371−level.
372
355
373−`%boil` functionally produces the file at a given beam with the given
356+`%boil` functionally produces the file at a given beam with the given mark and
374−mark and heel. If there is no way to translate at this beam, we pop
357+heel. If there is no way to translate at this beam, we pop levels off the stack
375−levels off the stack and attempt to bake there until we find a level we
358+and attempt to bake there until we find a level we can bake. This should almost
376−can bake. This should almost always be called instead of `%bake`.
359+always be called instead of `%bake`.
377
360
378`%call` slams the result of one silk against the result of another.
361`%call` slams the result of one silk against the result of another.
379
362
380−`%cast` translates the given silk to the given mark, if possible. This
363+`%cast` translates the given silk to the given mark, if possible. This is one of
381−is one of the critical and fundamental operations of ford.
364+the critical and fundamental operations of ford.
382
365
383−`%done` produces exactly its input. This is rarely used on its own, but
366+`%done` produces exactly its input. This is rarely used on its own, but many
384−many silks are recursively defined in terms of other silks, so we often
367+silks are recursively defined in terms of other silks, so we often need a silk
385−need a silk that simply produces its input. A monadic return, if you
368+that simply produces its input. A monadic return, if you will.
386−will.
387
369
388−`%diff` diffs the two given silks (which must be of the same mark),
370+`%diff` diffs the two given silks (which must be of the same mark), producing a
389−producing a cage of the mark specified in `++mark` in `++grad` for the
371+cage of the mark specified in `++mark` in `++grad` for the mark of the two
390−mark of the two silks.
372+silks.
391
373
392−`%dude` computes the given silk with the given tank as part of the stack
374+`%dude` computes the given silk with the given tank as part of the stack trace
393−trace if there is an error.
375+if there is an error.
394
376
395−`%dune` produces an error if the cage is empty. Otherwise, it produces
377+`%dune` produces an error if the cage is empty. Otherwise, it produces the value
396−the value in the unit.
378+in the unit.
397
379
398−`%mute` takes a silk and a list of changes to make to the silk. At each
380+`%mute` takes a silk and a list of changes to make to the silk. At each wing in
399−wing in the list we put the value of the associated silk.
381+the list we put the value of the associated silk.
400
382
401−`%pact` applies the second silk as a patch to the first silk. The second
383+`%pact` applies the second silk as a patch to the first silk. The second silk
402−silk must be of the mark specified in `++mark` in `++grad` for the mark
384+must be of the mark specified in `++mark` in `++grad` for the mark of the first
403−of the first silk.
385+silk.
404
386
405−`%plan` performs a structured assembly directly. This is not generally
387+`%plan` performs a structured assembly directly. This is not generally directly
406−directly useful because several other silks perform supersets of this
388+useful because several other silks perform supersets of this functionality. We
407−functionality. We don't usually have naked hoods outside ford.
389+don't usually have naked hoods outside ford.
408
390
409−`%reef` produces a core containing the entirety of zuse and hoon,
391+`%reef` produces a core containing the entirety of zuse and hoon, suitable for
410−suitable for running arbitrary code against. The mark is `%noun`.
392+running arbitrary code against. The mark is `%noun`.
411
393
412−`%ride` slaps a twig against a subject silk. The mark of the result is
394+`%ride` slaps a twig against a subject silk. The mark of the result is `%noun`.
413−`%noun`.
414
395
415−`%vale` validates untyped data from a ship against a given mark. This is
396+`%vale` validates untyped data from a ship against a given mark. This is an
416−an extremely useful function.
397+extremely useful function.
417
398
418−Commentary
399+# Commentary
419−==========
420
400
421−Parsing Hook Files
401+## Parsing Hook Files
422−------------------
423
402
424−In the commentary on other vanes, we have traced through the lifecycle
403+In the commentary on other vanes, we have traced through the lifecycle of
425−of various external requests. This is generally a very reasonable order
404+various external requests. This is generally a very reasonable order to examine
426−to examine vanes since it will eventually cover the entire vane, and we
405+vanes since it will eventually cover the entire vane, and we are never left
427−are never left wondering why we are doing something.
406+wondering why we are doing something.
428
407
429−For ford, however, it makes more sense to begin by discussing the
408+For ford, however, it makes more sense to begin by discussing the parsing and
430−parsing and assembliing of hook files. Many of the possible requests
409+assembliing of hook files. Many of the possible requests require us to assemble
431−require us to assemble hook files, so we may as well examine this
410+hook files, so we may as well examine this immediately.
432−immediately.
433
411
434−First, we will examine the parsing. We parse a file at a beam to a hood
412+First, we will examine the parsing. We parse a file at a beam to a hood in
435−in `++fade:zo:za`. The top-level parsing rule is `++fair`, which takes a
413+`++fade:zo:za`. The top-level parsing rule is `++fair`, which takes a beam and
436−beam and produces a rule to parse an entire hood file.
414+produces a rule to parse an entire hood file.
437
415
438−A note on the naming scheme: the parsing the combinators that parse into
416+A note on the naming scheme: the parsing the combinators that parse into a
439−a particular structure are conventionally given the same name as the
417+particular structure are conventionally given the same name as the structure.
440−structure. Although this locally clobbers the type names, this pattern
418+Although this locally clobbers the type names, this pattern makes obvious the
441−makes obvious the intent of the parsing combinators.
419+intent of the parsing combinators.
442
420
443We kick off with `++hood:fair`.
421We kick off with `++hood:fair`.
444
422
@@ -464,24 +442,23 @@ We kick off with `++hood:fair`.
464 (most gap hoop)
442 (most gap hoop)
465 ==
443 ==
466
444
467−There are five sections to a hood: system version, structures,
445+There are five sections to a hood: system version, structures, libraries,
468−libraries, resources, and body.
446+resources, and body.
469
447
470−First, we parse the requested version number of the system. This is
448+First, we parse the requested version number of the system. This is specified
471−specified with a unary `/?` rune. If not present, then we default to the
449+with a unary `/?` rune. If not present, then we default to the current version.
472−current version.
473
450
474−Second, we may have zero or more `/-` runes followed by a parsing of a
451+Second, we may have zero or more `/-` runes followed by a parsing of a `++hoot`,
475−`++hoot`, which represents a shared structure.
452+which represents a shared structure.
476
453
477−Third, we may have zero or more `/+` runes followed by a parsing of a
454+Third, we may have zero or more `/+` runes followed by a parsing of a `++hoof`,
478−`++hoof`, which represents a shared library.
455+which represents a shared library.
479
456
480−Fourth, we may have zero or more other `/` runes (as described in
457+Fourth, we may have zero or more other `/` runes (as described in `++horn`),
481−`++horn`), which represent program-specific resources to be loaded.
458+which represent program-specific resources to be loaded.
482
459
483−Fifth and finally, we must have one or more body statements (hoops),
460+Fifth and finally, we must have one or more body statements (hoops), which are
484−which are either direct twigs or `//` runes.
461+either direct twigs or `//` runes.
485
462
486 ++ hoot
463 ++ hoot
487 ;~ pose
464 ;~ pose
@@ -489,9 +466,9 @@ which are either direct twigs or `//` runes.
489 (stag %& hoof)
466 (stag %& hoof)
490 ==
467 ==
491
468
492−A structure can either be a direct gate, or it can be a simple core.
469+A structure can either be a direct gate, or it can be a simple core. Either one
493−Either one is parsed with `++hoof`, so we distinguish the two cases by
470+is parsed with `++hoof`, so we distinguish the two cases by requireing core
494−requireing core references to be prefixed by a `*`.
471+references to be prefixed by a `*`.
495
472
496 ++ hoof
473 ++ hoof
497 %+ cook |=(a=^hoof a)
474 %+ cook |=(a=^hoof a)
@@ -504,11 +481,11 @@ requireing core references to be prefixed by a `*`.
504 ==
481 ==
505 ==
482 ==
506
483
507−A hoof must have a name, which is a term. Optionally, we also include a
484+A hoof must have a name, which is a term. Optionally, we also include a case and
508−case and a ship. This is marked by appending a `/` followed by a case to
485+a ship. This is marked by appending a `/` followed by a case to denote the
509−denote the requested version of the resource and a `/` followed by a
486+requested version of the resource and a `/` followed by a ship name to denote
510−ship name to denote the requested source of the resource. For example,
487+the requested source of the resource. For example, `resource/1/~zod` requests
511−`resource/1/~zod` requests the first version of `resource` on `~zod`.
488+the first version of `resource` on `~zod`.
512
489
513 ++ case
490 ++ case
514 %- sear
491 %- sear
@@ -517,11 +494,11 @@ ship name to denote the requested source of the resource. For example,
517 ?. ?=([%$ ?(%da %ud %tas) *] a) ~
494 ?. ?=([%$ ?(%da %ud %tas) *] a) ~
518 [~ u=(^case a)]
495 [~ u=(^case a)]
519
496
520−Here, we parse a literal with `++nuck:so`, and we accept the input if it
497+Here, we parse a literal with `++nuck:so`, and we accept the input if it is
521−is either an absolute date, an unsigned decimal, or a label.
498+either an absolute date, an unsigned decimal, or a label.
522
499
523−This leaves only horns and hoops to parse. Hoops are much simple to
500+This leaves only horns and hoops to parse. Hoops are much simple to parse, so
524−parse, so we'll discuss those first.
501+we'll discuss those first.
525
502
526 ++ hoop
503 ++ hoop
527 ;~ pose
504 ;~ pose
@@ -529,20 +506,20 @@ parse, so we'll discuss those first.
529 (stag %& tall:vez)
506 (stag %& tall:vez)
530 ==
507 ==
531
508
532−There are two types of hoops. Direct twigs are parsed with
509+There are two types of hoops. Direct twigs are parsed with `++tall:vast`, which
533−`++tall:vast`, which is the just the hoon parser for a tall-form twig.
510+is the just the hoon parser for a tall-form twig.
534
511
535−References to external twigs are marked with a `//` rune followed by a
512+References to external twigs are marked with a `//` rune followed by a beam,
536−beam, which is parsed with `++have`.
513+which is parsed with `++have`.
537
514
538 ++ hath (sear plex:voz (stag %clsg poor:voz)) :: hood path
515 ++ hath (sear plex:voz (stag %clsg poor:voz)) :: hood path
539 ++ have (sear tome ;~(pfix fas hath)) :: hood beam
516 ++ have (sear tome ;~(pfix fas hath)) :: hood beam
540
517
541−`++have` parses a path with `++hath`, and then it converts the path into
518+`++have` parses a path with `++hath`, and then it converts the path into a beam
542−a beam with `++tome`.
519+with `++tome`.
543
520
544−`++hath` parses a `/`-separated list with `++poor:vast`, then converts
521+`++hath` parses a `/`-separated list with `++poor:vast`, then converts it to an
545−it to an actual path with `++plex:vast`.
522+actual path with `++plex:vast`.
546
523
547This leaves only horns to parse.
524This leaves only horns to parse.
548
525
@@ -571,18 +548,17 @@ This leaves only horns to parse.
571 ==
548 ==
572 ==
549 ==
573
550
574−Horn parsing is slightly complex, so we create an internal core to
551+Horn parsing is slightly complex, so we create an internal core to organize our
575−organize our code. Our core has a global variable of `tol`, which is
552+code. Our core has a global variable of `tol`, which is true if tall form is
576−true if tall form is permissible and false if we're already in wide
553+permissible and false if we're already in wide form. We kick off the parsing
577−form. We kick off the parsing with `++apex`.
554+with `++apex`.
578
555
579−`++apex` specifies how each rune is parsed. This allows us to offload
556+`++apex` specifies how each rune is parsed. This allows us to offload the
580−the different ways of parsing the arguments to these runes into separate
557+different ways of parsing the arguments to these runes into separate arms. The
581−arms. The exception here is that the `%toy` horn is simply of the form
558+exception here is that the `%toy` horn is simply of the form `/mark/`.
582−`/mark/`.
583
559
584−We'll examine each of the horn parsing arms right after we discuss
560+We'll examine each of the horn parsing arms right after we discuss `++rail`,
585−`++rail`, which is used in each one.
561+which is used in each one.
586
562
587 ++ rail
563 ++ rail
588 |* [wid=_rule tal=_rule]
564 |* [wid=_rule tal=_rule]
@@ -590,8 +566,8 @@ We'll examine each of the horn parsing arms right after we discuss
590 ;~(pose wid tal)
566 ;~(pose wid tal)
591
567
592This takes a wide-form and a tall-form parsing rule. If tall form is
568This takes a wide-form and a tall-form parsing rule. If tall form is
593−permissible, then it allows either rule to match; else, it allows only
569+permissible, then it allows either rule to match; else, it allows only the wide
594−the wide form rule.
570+form rule.
595
571
596 ++ read
572 ++ read
597 |% ++ ape
573 |% ++ ape
@@ -599,49 +575,47 @@ the wide form rule.
599 (ifix [sel ser] (stag %cltr (most ace wide:vez)))
575 (ifix [sel ser] (stag %cltr (most ace wide:vez)))
600 ;~(pfix gap tall:vez)
576 ;~(pfix gap tall:vez)
601
577
602−`++ape:read` parses for both the `/~` and the `/$` runes. It produces a
578+`++ape:read` parses for both the `/~` and the `/$` runes. It produces a twig.
603−twig. The wide form is a tuple of one or more ace-separated wide-form
579+The wide form is a tuple of one or more ace-separated wide-form twigs parsed
604−twigs parsed with `++wide:vast` and surrounded by `[` and `]`. The tall
580+with `++wide:vast` and surrounded by `[` and `]`. The tall form is a single tall
605−form is a single tall form twig parsed by `++tall:vast`
581+form twig parsed by `++tall:vast`
606
582
607 ++ day
583 ++ day
608 %+ rail
584 %+ rail
609 apex(tol |)
585 apex(tol |)
610 ;~(pfix gap apex)
586 ;~(pfix gap apex)
611
587
612−This parses for the `/|`, `/@`, `/%`, and `/&` runes. It produces a
588+This parses for the `/|`, `/@`, `/%`, and `/&` runes. It produces a horn. The
613−horn. The wide form is, recursively, the entire horn parser with tall
589+wide form is, recursively, the entire horn parser with tall form disabled. The
614−form disabled. The tall form is a gap followed by, recursively, the
590+tall form is a gap followed by, recursively, the entire horn parser.
615−entire horn parser.
616
591
617 ++ dub
592 ++ dub
618 %+ rail
593 %+ rail
619 ;~(plug sym ;~(pfix tis apex(tol |)))
594 ;~(plug sym ;~(pfix tis apex(tol |)))
620 ;~(pfix gap ;~(plug sym ;~(pfix gap apex)))
595 ;~(pfix gap ;~(plug sym ;~(pfix gap apex)))
621
596
622−This parses for the `/=` rune. It produces a term followed by a horn.
597+This parses for the `/=` rune. It produces a term followed by a horn. The wide
623−The wide form is a symbol name followed by a `=` and, recursively, the
598+form is a symbol name followed by a `=` and, recursively, the entire horn parser
624−entire horn parser with tall form disabled. The tall form is a gap
599+with tall form disabled. The tall form is a gap followed by a symbol name,
625−followed by a symbol name, another gap, and, recursively, the entire
600+another gap, and, recursively, the entire horn parser.
626−horn parser.
627
601
628 ++ fan
602 ++ fan
629 %+ rail fail
603 %+ rail fail
630 ;~(sfix (star ;~(pfix gap apex)) ;~(plug gap duz))
604 ;~(sfix (star ;~(pfix gap apex)) ;~(plug gap duz))
631
605
632−This parses for the `/.` rune. It produces a list of horns. There is no
606+This parses for the `/.` rune. It produces a list of horns. There is no wide
633−wide form. The tall form is a stet-terminated series of gap-separated
607+form. The tall form is a stet-terminated series of gap-separated recursive calls
634−recursive calls to the entire horn parser.
608+to the entire horn parser.
635
609
636 ++ for
610 ++ for
637 %+ rail
611 %+ rail
638 ;~(plug (ifix [sel ser] hath) apex(tol |))
612 ;~(plug (ifix [sel ser] hath) apex(tol |))
639 ;~(pfix gap ;~(plug hath ;~(pfix gap apex)))
613 ;~(pfix gap ;~(plug hath ;~(pfix gap apex)))
640
614
641−This parses for the `/,` rune. It produces a path and a horn. The wide
615+This parses for the `/,` rune. It produces a path and a horn. The wide form is a
642−form is a `[`-`]`-surrounded path followed by, recursively, the entire
616+`[`-`]`-surrounded path followed by, recursively, the entire horn parser with
643−horn parser with tall form disabled. The tall form is a gap followed by
617+tall form disabled. The tall form is a gap followed by a path, another gap, and,
644−a path, another gap, and, recursively, the entire horn parser.
618+recursively, the entire horn parser.
645
619
646 ++ man
620 ++ man
647 %+ rail fail
621 %+ rail fail
@@ -654,31 +628,31 @@ a path, another gap, and, recursively, the entire horn parser.
654 ?. ?=(%dub -.i.fan) ~
628 ?. ?=(%dub -.i.fan) ~
655 $(fan t.fan, naf [[p.i.fan q.i.fan] naf])
629 $(fan t.fan, naf [[p.i.fan q.i.fan] naf])
656
630
657−This parses for the `/*` rune. It produces a map of spans to horns.
631+This parses for the `/*` rune. It produces a map of spans to horns. There is no
658−There is no wide form. The tall form is a stet-terminated series of
632+wide form. The tall form is a stet-terminated series of gap-separated recursive
659−gap-separated recursive calls to the entire horn parser. All produced
633+calls to the entire horn parser. All produced horns are expected to be from `/=`
660−horns are expected to be from `/=` runes. The term and horn in each `/=`
634+runes. The term and horn in each `/=` horn is inserted into the produced map as
661−horn is inserted into the produced map as a key-value pair.
635+a key-value pair.
662
636
663 ++ saw
637 ++ saw
664 %+ rail
638 %+ rail
665 ;~(plug ;~(sfix wide:vez sem) apex(tol |))
639 ;~(plug ;~(sfix wide:vez sem) apex(tol |))
666 ;~(pfix gap ;~(plug tall:vez ;~(pfix gap apex)))
640 ;~(pfix gap ;~(plug tall:vez ;~(pfix gap apex)))
667
641
668−This parses for the `/;` rune. It produces a twig and a horn. The wide
642+This parses for the `/;` rune. It produces a twig and a horn. The wide form is a
669−form is a wide-form twig followed by a `;` and, recursively, the entire
643+wide-form twig followed by a `;` and, recursively, the entire horn parser with
670−horn parser with tall form disabled. The tall form is a gap followed by
644+tall form disabled. The tall form is a gap followed by a tall-form twig, another
671−a tall-form twig, another gap, and, recursively, the entire horn parser.
645+gap, and, recursively, the entire horn parser.
672
646
673 ++ see
647 ++ see
674 %+ rail
648 %+ rail
675 ;~(plug ;~(sfix have col) apex(tol |))
649 ;~(plug ;~(sfix have col) apex(tol |))
676 ;~(pfix gap ;~(plug have ;~(pfix gap apex)))
650 ;~(pfix gap ;~(plug have ;~(pfix gap apex)))
677
651
678−This parses for the `/:` rune. It produces a beam and a horn. The wide
652+This parses for the `/:` rune. It produces a beam and a horn. The wide form is a
679−form is a beam followed by a `;` and, recursively, the entire horn
653+beam followed by a `;` and, recursively, the entire horn parser with tall form
680−parser with tall form disabled. The tall form is a gap followed by a
654+disabled. The tall form is a gap followed by a beam, another gap, and,
681−beam, another gap, and, recursively, the entire horn parser.
655+recursively, the entire horn parser.
682
656
683 ++ sic
657 ++ sic
684 %+ rail
658 %+ rail
@@ -686,18 +660,17 @@ beam, another gap, and, recursively, the entire horn parser.
686 ;~(pfix gap ;~(plug howl:vez ;~(pfix gap apex)))
660 ;~(pfix gap ;~(plug howl:vez ;~(pfix gap apex)))
687 --
661 --
688
662
689−This parses for the `/^` rune. It produces a tile and a horn. The wide
663+This parses for the `/^` rune. It produces a tile and a horn. The wide form is a
690−form is a wide-form tile, parsed with `++toil:vast`, followed by a `^`
664+wide-form tile, parsed with `++toil:vast`, followed by a `^` and, recursively,
691−and, recursively, the entire horn parser with tall form disabled. The
665+the entire horn parser with tall form disabled. The tall form is a gap followed
692−tall form is a gap followed by a tall-form tile, parsed with
666+by a tall-form tile, parsed with `++howl:vast`, another gap, and, recursively,
693−`++howl:vast`, another gap, and, recursively, the entire horn parser.
667+the entire horn parser.
694
668
695−Assembling Hook Files
669+## Assembling Hook Files
696−---------------------
697
670
698−At this point, we've parsed a hook file into a hood. We will now
671+At this point, we've parsed a hook file into a hood. We will now describe
699−describe exactly how this hood is assembled into a vase. The problem of
672+exactly how this hood is assembled into a vase. The problem of assembling is
700−assembling is handled entirely within the `++meow:zo:za` core.
673+handled entirely within the `++meow:zo:za` core.
701
674
702 ++ meow :: assemble
675 ++ meow :: assemble
703 |= [how=beam arg=heel]
676 |= [how=beam arg=heel]
@@ -711,47 +684,45 @@ assembling is handled entirely within the `++meow:zo:za` core.
711 ==
684 ==
712 |%
685 |%
713
686
714−We take two arguments and keep seven pieces of state. `how` is the
687+We take two arguments and keep seven pieces of state. `how` is the location of
715−location of the hook file we're assembling, and `arg` is the heel, or
688+the hook file we're assembling, and `arg` is the heel, or virtual path
716−virtual path extension, of the file.
689+extension, of the file.
717
690
718−In `rop`, we maintain a map of terms to pairs of hooves and twigs to
691+In `rop`, we maintain a map of terms to pairs of hooves and twigs to represent
719−represent the structures we've encountered that we will put together in
692+the structures we've encountered that we will put together in a core at the top
720−a core at the top of the file.
693+of the file.
721
694
722−In `zog`, we maintain the set of structures we're in the middle of
695+In `zog`, we maintain the set of structures we're in the middle of loading. If
723−loading. If we try to load a structure already in our dependency
696+we try to load a structure already in our dependency ancestry, then we fail
724−ancestry, then we fail because we do not allow circular dependencies.
697+because we do not allow circular dependencies. This enforces that our structure
725−This enforces that our structure dependency graph is a DAG.
698+dependency graph is a DAG.
726
699
727−In `bil`, we maintain a map of terms to pairs of hooves and twigs to
700+In `bil`, we maintain a map of terms to pairs of hooves and twigs to represent
728−represent the libraries we've encountered that we will put together in a
701+the libraries we've encountered that we will put together in a series of cores
729−series of cores after the structure core.
702+after the structure core.
730
703
731−In `lot`, we maintain a stack of library names as they are encountered
704+In `lot`, we maintain a stack of library names as they are encountered during a
732−during a depth-first search. More precisely, we push a library onto the
705+depth-first search. More precisely, we push a library onto the stack after we've
733−stack after we've processed all its children. Thus, every library
706+processed all its children. Thus, every library depends only on things deeper in
734−depends only on things deeper in the list. The libraries must be loaded
707+the list. The libraries must be loaded in the reverse of this order. Concisely,
735−in the reverse of this order. Concisely, this is a topological sort of
708+this is a topological sort of the library dependency partial ordering.
736−the library dependency partial ordering.
737
709
738−In `zeg`, we maintain the set of libraries we're in the middle of
710+In `zeg`, we maintain the set of libraries we're in the middle of loading. If we
739−loading. If we try to load a library already in our dependency ancestry,
711+try to load a library already in our dependency ancestry, then we fail because
740−then we fail because we do not allow circular dependencies. This
712+we do not allow circular dependencies. This enforces that our library dependency
741−enforces that our library dependency graph is a DAG.
713+graph is a DAG.
742
714
743−In `boy`, we maintain a stack of body twigs, which we'll put together in
715+In `boy`, we maintain a stack of body twigs, which we'll put together in a
744−a series of cores at the end of the file.
716+series of cores at the end of the file.
745
717
746−In `hol`, we decide if we're allowed to contain horns. Libraries and
718+In `hol`, we decide if we're allowed to contain horns. Libraries and structures
747−structures are not allowed to contain horns.
719+are not allowed to contain horns.
748
720
749−We in every case enter `++meow` through `++abut`. You'll notice that
721+We in every case enter `++meow` through `++abut`. You'll notice that there are
750−there are four (count 'em, four!) calls to `++cope` in `++abut`. If
722+four (count 'em, four!) calls to `++cope` in `++abut`. If you've glanced at the
751−you've glanced at the ford code in general, you've probably seen cope
723+ford code in general, you've probably seen cope over and over. It is called in
752−over and over. It is called in 79 different places. We need to discuss
724+79 different places. We need to discuss the use of this critical function in
753−the use of this critical function in detail, so we may as well do it
725+detail, so we may as well do it here.
754−here.
755
726
756 ++ cope :: bolt along
727 ++ cope :: bolt along
757 |* [hoc=(bolt) fun=(burg)]
728 |* [hoc=(bolt) fun=(burg)]
@@ -767,40 +738,36 @@ here.
767 %0 [%0 p=(grom `_p.q.nuf`p.q.hoc p.q.nuf) q=q.q.nuf]
738 %0 [%0 p=(grom `_p.q.nuf`p.q.hoc p.q.nuf) q=q.q.nuf]
768 == ==
739 == ==
769
740
770−In monad-speak, this is the bind operator for the bolt monad. If monads
741+In monad-speak, this is the bind operator for the bolt monad. If monads aren't
771−aren't your thing, don't worry, we're going to explain the use of cope
742+your thing, don't worry, we're going to explain the use of cope without further
772−without further reference to them.
743+reference to them.
773−
744+
774−Recall that there are three different types of bolt. A `%2` error bolt
745+Recall that there are three different types of bolt. A `%2` error bolt contains
775−contains a list of tanks describing the error, a `%1` block bolt
746+a list of tanks describing the error, a `%1` block bolt contains a set of
776−contains a set of resources we're blocked on, and a `%0` value bolt
747+resources we're blocked on, and a `%0` value bolt contains an actual value and
777−contains an actual value and the set of its dependencies.
748+the set of its dependencies.
778−
749+
779−We most commonly want to perform an operation on the value in a bolt if
750+We most commonly want to perform an operation on the value in a bolt if it is a
780−it is a `%0` bolt. If it's not a `%0` bolt, we want to leave it alone.
751+`%0` bolt. If it's not a `%0` bolt, we want to leave it alone. This requires us
781−This requires us to write a certain amount of boilerplate between each
752+to write a certain amount of boilerplate between each of our operations to see
782−of our operations to see if any of them produced a `%1` or a `%2` bolt.
753+if any of them produced a `%1` or a `%2` bolt. This gets tiresome, so we pull it
783−This gets tiresome, so we pull it out into a separate arm and call it
754+out into a separate arm and call it `++cope`.
784−`++cope`.
755+
785−
756+Intuitively, we're calling the function `fun` with the value in `hoc`, where
786−Intuitively, we're calling the function `fun` with the value in `hoc`,
757+`fun` takes an argument of type whatever is the value in a `%0` case of `hoc`,
787−where `fun` takes an argument of type whatever is the value in a `%0`
758+and it produces a bolt of some (possibly different) type. For brevity, we will
788−case of `hoc`, and it produces a bolt of some (possibly different) type.
759+refer to the type of the of the value in the `%0` case of a bolt as the "type of
789−For brevity, we will refer to the type of the of the value in the `%0`
760+the bolt".
790−case of a bolt as the "type of the bolt".
761+
791−
762+If the `hoc` bolt we're given as input to `fun` is already a `%1` or a `%2`
792−If the `hoc` bolt we're given as input to `fun` is already a `%1` or a
763+bolt, then we simply produce that. We don't even try to run `fun` on it.
793−`%2` bolt, then we simply produce that. We don't even try to run `fun`
764+
794−on it.
765+Otherwise, we run `fun` with the arguments from the bolt and, if it produces a
795−
766+`%1` or a `%2` bolt, we simply produce that. If it produces a `%0` bolt, then we
796−Otherwise, we run `fun` with the arguments from the bolt and, if it
767+produce that with the old set of dependencies merged in with the new set.
797−produces a `%1` or a `%2` bolt, we simply produce that. If it produces a
768+
798−`%0` bolt, then we produce that with the old set of dependencies merged
769+We'll see more about how the bolt monad works as we run into more interesting
799−in with the new set.
770+uses of it. For now, this is sufficient to move on with `++abut`.
800−
801−We'll see more about how the bolt monad works as we run into more
802−interesting uses of it. For now, this is sufficient to move on with
803−`++abut`.
804
771
805 ++ abut :: generate
772 ++ abut :: generate
806 |= [cof=cafe hyd=hood]
773 |= [cof=cafe hyd=hood]
@@ -816,18 +783,17 @@ interesting uses of it. For now, this is sufficient to move on with
816 |= [cof=cafe fin=vase]
783 |= [cof=cafe fin=vase]
817 (fine cof fin)
784 (fine cof fin)
818
785
819−Our job is simple: we must assemble a hood file into a vase. Hopefully,
786+Our job is simple: we must assemble a hood file into a vase. Hopefully, the
820−the usage of `++cope` is fairly understandable. The correct way to read
787+usage of `++cope` is fairly understandable. The correct way to read this is that
821−this is that it does essentially five things.
788+it does essentially five things.
822
789
823−First, we call `++apex` to process the structures, libraries, and body.
790+First, we call `++apex` to process the structures, libraries, and body. This
824−This changes our state, so we set our context to the produced context.
791+changes our state, so we set our context to the produced context. Second, we
825−Second, we call `++able` to assemble the strucutres and libraries into a
792+call `++able` to assemble the strucutres and libraries into a twig, which we
826−twig, which we slap against zuse with `++maim`. Third, we call `++chap`
793+slap against zuse with `++maim`. Third, we call `++chap` to process the
827−to process the resources in the context of the already-loaded structures
794+resources in the context of the already-loaded structures and libraries. Fourth,
828−and libraries. Fourth, we slap the body against the structures,
795+we slap the body against the structures, libraries, and resources. Fifth and
829−libraries, and resources. Fifth and finally, we produce the resultant
796+finally, we produce the resultant vase.
830−vase.
831
797
832 ++ apex :: build to body
798 ++ apex :: build to body
833 |= [cof=cafe hyd=hood]
799 |= [cof=cafe hyd=hood]
@@ -852,9 +818,9 @@ Otherwise, we produce and error with `++flaw`.
852
818
853This produces a `%2` error bolt from a list of tanks. Fairly trivial.
819This produces a `%2` error bolt from a list of tanks. Fairly trivial.
854
820
855−We should be starting to get used to the cope syntax, so we can see that
821+We should be starting to get used to the cope syntax, so we can see that we
856−we really only do three things here. We process the body with `++body`,
822+really only do three things here. We process the body with `++body`, the
857−the libraries with `++neck`, and the structures with `++head`.
823+libraries with `++neck`, and the structures with `++head`.
858
824
859 ++ body :: produce functions
825 ++ body :: produce functions
860 |= [cof=cafe src=(list hoop)]
826 |= [cof=cafe src=(list hoop)]
@@ -864,19 +830,19 @@ the libraries with `++neck`, and the structures with `++head`.
864 |= [cof=cafe sel=_..body]
830 |= [cof=cafe sel=_..body]
865 ^$(cof cof, src t.src, ..body sel)
831 ^$(cof cof, src t.src, ..body sel)
866
832
867−We must process a list of hoops that represent our body. If there are no
833+We must process a list of hoops that represent our body. If there are no more
868−more hoops, we just produce our context in a `%0` bolt with `++fine`.
834+hoops, we just produce our context in a `%0` bolt with `++fine`.
869
835
870 ++ fine |* [a=cafe b=*] :: bolt from data
836 ++ fine |* [a=cafe b=*] :: bolt from data
871 [p=`cafe`a q=[%0 p=*(set beam) q=b]] ::
837 [p=`cafe`a q=[%0 p=*(set beam) q=b]] ::
872
838
873−In monad-speak, this is the return operator. For us, this just means
839+In monad-speak, this is the return operator. For us, this just means that we're
874−that we're producing a `%0` bolt, which contains a path and a set of
840+producing a `%0` bolt, which contains a path and a set of dependencies. We
875−dependencies. We assume there are no dependencies for the given data, or
841+assume there are no dependencies for the given data, or that they will be added
876−that they will be added later.
842+later.
877
843
878−If there are more hoops in `++body`, we call `++wilt` to process an
844+If there are more hoops in `++body`, we call `++wilt` to process an individual
879−individual hoop and recurse.
845+hoop and recurse.
880
846
881 ++ wilt :: process body entry
847 ++ wilt :: process body entry
882 |= [cof=cafe hop=hoop]
848 |= [cof=cafe hop=hoop]
@@ -916,24 +882,22 @@ individual hoop and recurse.
916 (fine cof sel(boy [[%brcn mav] boy]))
882 (fine cof sel(boy [[%brcn mav] boy]))
917 ==
883 ==
918
884
919−In the case of a direct twig hoop, we just push it onto `boy` and we're
885+In the case of a direct twig hoop, we just push it onto `boy` and we're done. In
920−done. In the case of an indirect hoop, we must compile the referenced
886+the case of an indirect hoop, we must compile the referenced file.
921−file.
922
887
923−First, we push onto the stack trace a message indicating which file
888+First, we push onto the stack trace a message indicating which file exactly
924−exactly we're compiling at the moment with `++cool`.
889+we're compiling at the moment with `++cool`.
925
890
926 ++ cool :: error caption
891 ++ cool :: error caption
927 |* [cyt=trap hoc=(bolt)]
892 |* [cyt=trap hoc=(bolt)]
928 ?. ?=(%2 -.q.hoc) hoc
893 ?. ?=(%2 -.q.hoc) hoc
929 [p.hoc [%2 *cyt p.q.hoc]]
894 [p.hoc [%2 *cyt p.q.hoc]]
930
895
931−If an error occurred in computing `hoc`, we put the bunt of `cyt` onto
896+If an error occurred in computing `hoc`, we put the bunt of `cyt` onto the stack
932−the stack trace. Thus, `cyt` is not evaluated at all unless an error
897+trace. Thus, `cyt` is not evaluated at all unless an error occurred.
933−occurred.
934
898
935−Next in `++wilt`, we load the information about the filesystem node
899+Next in `++wilt`, we load the information about the filesystem node referenced
936−referenced by the hoop with `++lend`.
900+by the hoop with `++lend`.
937
901
938 ++ lend :: load arch
902 ++ lend :: load arch
939 |= [cof=cafe bem=beam]
903 |= [cof=cafe bem=beam]
@@ -942,28 +906,27 @@ referenced by the hoop with `++lend`.
942 ?~ von [p=cof q=[%1 [bem ~] ~ ~]]
906 ?~ von [p=cof q=[%1 [bem ~] ~ ~]]
943 (fine cof ((hard arch) (need u.von)))
907 (fine cof ((hard arch) (need u.von)))
944
908
945−This is a simple call to the namespace. If the resource does not yet
909+This is a simple call to the namespace. If the resource does not yet exist, we
946−exist, we block on it by producing a `%1` bolt. Otherwise, we cast it to
910+block on it by producing a `%1` bolt. Otherwise, we cast it to an arch and
947−an arch and produce this.
911+produce this.
948
912
949Continuing in `++wilt`, we examine the produced arch. If the referenced
913Continuing in `++wilt`, we examine the produced arch. If the referenced
950−filesystem node has a `hoon` child node, then we've found the required
914+filesystem node has a `hoon` child node, then we've found the required source,
951−source, so we parse it with `++fade`. Recall that we referred earlier to
915+so we parse it with `++fade`. Recall that we referred earlier to `++fade`. The
952−`++fade`. The salient point there is that it takes a beam, reads in the
916+salient point there is that it takes a beam, reads in the hook file there, and
953−hook file there, and parses it into a hood file with `++fair`.
917+parses it into a hood file with `++fair`.
954
918
955−Now, we simply recurse on `++apex` to compile the new hood. Note that,
919+Now, we simply recurse on `++apex` to compile the new hood. Note that, while we
956−while we do clear the `boy` list, we do not clear the other lists. Thus,
920+do clear the `boy` list, we do not clear the other lists. Thus, we are
957−we are accumulating all the structures and libraries referenced in all
921+accumulating all the structures and libraries referenced in all the referenced
958−the referenced hook files in one group, which we will put at the top of
922+hook files in one group, which we will put at the top of the product.
959−the product.
960
923
961−After this, we put the new list of body twigs into a `=~`, push this
924+After this, we put the new list of body twigs into a `=~`, push this onto our
962−onto our old list of body twigs, and produce the result.
925+old list of body twigs, and produce the result.
963
926
964−If there is no hoon file here, then we descend into each of our children
927+If there is no hoon file here, then we descend into each of our children until
965−until we find a hoon file. First, we produce a list of all our children
928+we find a hoon file. First, we produce a list of all our children whose names
966−whose names are terms with `++lark`.
929+are terms with `++lark`.
967
930
968 ++ lark :: filter arch names
931 ++ lark :: filter arch names
969 |= [wox=$+(span (unit ,@)) arc=arch]
932 |= [wox=$+(span (unit ,@)) arc=arch]
@@ -976,43 +939,39 @@ whose names are terms with `++lark`.
976 =+ gib=(wox p.n.r.arc)
939 =+ gib=(wox p.n.r.arc)
977 ?~(gib rac [[u.gib p.n.r.arc] rac])
940 ?~(gib rac [[u.gib p.n.r.arc] rac])
978
941
979−We traverse the children map of `arc` to filter out those children whose
942+We traverse the children map of `arc` to filter out those children whose names
980−names aren't accepted by `wox` and produce a map from the product of
943+aren't accepted by `wox` and produce a map from the product of `wox` to the
981−`wox` to the original name. `++lark` is used in many cases to parse
944+original name. `++lark` is used in many cases to parse names into other types,
982−names into other types, like numbers or dates, ignoring those which do
945+like numbers or dates, ignoring those which do not fit the format. In `++wilt`,
983−not fit the format. In `++wilt`, though, we simply want to filter out
946+though, we simply want to filter out those children whose names are not terms.
984−those children whose names are not terms.
985
947
986−Next, we will produce a map from terms to feet. Each of these feet will
948+Next, we will produce a map from terms to feet. Each of these feet will be
987−be placed in a core named by the child name, and it will contain arms
949+placed in a core named by the child name, and it will contain arms according to
988−according to its children. Thus, if the indirect hoop references
950+its children. Thus, if the indirect hoop references `/path`, then to access the
989−`/path`, then to access the twig defined in `/path/to/twig/hoon`, our
951+twig defined in `/path/to/twig/hoon`, our body must refer to `twig:to`.
990−body must refer to `twig:to`.
991
952
992−If there are no more children, then we are done, so we produce our
953+If there are no more children, then we are done, so we produce our current
993−current context.
954+context.
994
955
995−Else, we recurse into the left and right sides of our map. Finally, we
956+Else, we recurse into the left and right sides of our map. Finally, we process
996−process our current entry in the map. We first recurse by calling
957+our current entry in the map. We first recurse by calling `++wilt` one level
997−`++wilt` one level down. Thus, in the previous example, the first time
958+down. Thus, in the previous example, the first time we get to this point we are
998−we get to this point we are processing `/path`, so we recurse on
959+processing `/path`, so we recurse on `++wilt` with path `/path/to`. We also
999−`++wilt` with path `/path/to`. We also remove our current body from the
960+remove our current body from the recursion, so that we may add it back in later
1000−recursion, so that we may add it back in later the way we want to.
961+the way we want to.
1001
962
1002−After recursing, we push the new body onto our map, keyed by its name.
963+After recursing, we push the new body onto our map, keyed by its name. We also
1003−We also produce the new context so that all external structures,
964+produce the new context so that all external structures, libraries, and
1004−libraries, and resources are collected into the same place.
965+resources are collected into the same place.
1005
966
1006−Finally, we have a map of names to feet. If this map is empty, then
967+Finally, we have a map of names to feet. If this map is empty, then there were
1007−there were no twigs at the requested path, so we give an error with
968+no twigs at the requested path, so we give an error with `++flaw`.
1008−`++flaw`.
1009
969
1010−If the map is nonempty, then we finally produce our context with with
970+If the map is nonempty, then we finally produce our context with with one thing
1011−one thing pushed onto the front: a core made out of the map we just
971+pushed onto the front: a core made out of the map we just produced.
1012−produced.
1013
972
1014−This concludes our discussion of `++wilt` and `++body`. Thus, it remains
973+This concludes our discussion of `++wilt` and `++body`. Thus, it remains in
1015−in `++apex` to discuss `++neck` and `++head`.
974+`++apex` to discuss `++neck` and `++head`.
1016
975
1017 ++ neck :: consume libraries
976 ++ neck :: consume libraries
1018 |= [cof=cafe bir=(list hoof)]
977 |= [cof=cafe bir=(list hoof)]
@@ -1040,26 +999,24 @@ in `++apex` to discuss `++neck` and `++head`.
1040 ==
999 ==
1041 ^^$(cof cof, bir t.bir)
1000 ^^$(cof cof, bir t.bir)
1042
1001
1043−Here, we're going to consume the list of libraries and place them in
1002+Here, we're going to consume the list of libraries and place them in `bil`. If
1044−`bil`. If there are no more libraries, we're done, so we just produce
1003+there are no more libraries, we're done, so we just produce our current context.
1045−our current context.
1046
1004
1047−Otherwise, we check to see if the next library in the list is in `zeg`.
1005+Otherwise, we check to see if the next library in the list is in `zeg`. If so,
1048−If so, then this library is one of the libraries that we're already in
1006+then this library is one of the libraries that we're already in the middle of
1049−the middle of compiling. There is a circular dependency, so we fail.
1007+compiling. There is a circular dependency, so we fail.
1050
1008
1051Otherwise, we let `gez` be `zeg` plus the current library so that while
1009Otherwise, we let `gez` be `zeg` plus the current library so that while
1052−compiling the dependencies of this library we don't later create a
1010+compiling the dependencies of this library we don't later create a circular
1053−circular dependency. We check next to see if this library is alredy in
1011+dependency. We check next to see if this library is alredy in `bil`. If so, then
1054−`bil`. If so, then we have already included this library earlier, so we
1012+we have already included this library earlier, so we check to see if this is the
1055−check to see if this is the same version of the library as we included
1013+same version of the library as we included earlier. If so, we skip it. Else, we
1056−earlier. If so, we skip it. Else, we fail since we can't include two
1014+fail since we can't include two different versions of a library. We really
1057−different versions of a library. We really should allow for newer
1015+should allow for newer versions of a library since in kelvin versioning we
1058−versions of a library since in kelvin versioning we assume backwards
1016+assume backwards compatibility, but for now we require an exact match.
1059−compatibility, but for now we require an exact match.
1017+
1060−
1018+If we haven't already included this library, then we're going to do that. First,
1061−If we haven't already included this library, then we're going to do
1019+we get the location of the library with `++hone`.
1062−that. First, we get the location of the library with `++hone`.
1063
1020
1064 ++ hone :: plant hoof
1021 ++ hone :: plant hoof
1065 |= [for=@tas way=@tas huf=hoof]
1022 |= [for=@tas way=@tas huf=hoof]
@@ -1068,28 +1025,27 @@ that. First, we get the location of the library with `++hone`.
1068 how(s ~[for p.huf way])
1025 how(s ~[for p.huf way])
1069 [[q.u.q.huf %main p.u.q.huf] ~[for p.huf way]]
1026 [[q.u.q.huf %main p.u.q.huf] ~[for p.huf way]]
1070
1027
1071−If we haven't specified the version of the library, we use the current
1028+If we haven't specified the version of the library, we use the current ship,
1072−ship, desk, and case. Otherwise, we use the given ship and case on desk
1029+desk, and case. Otherwise, we use the given ship and case on desk `%main`. In
1073−`%main`. In either case, the path is `/way/p.huf/for`. In the case of
1030+either case, the path is `/way/p.huf/for`. In the case of `++neck`, this means
1074−`++neck`, this means `/lib/core/[library name]`.
1031+`/lib/core/[library name]`.
1075−
1032+
1076−In `++neck`, we next compile the hook file at that location with
1033+In `++neck`, we next compile the hook file at that location with `++fade`.
1077−`++fade`. Again, we will delay the discussion of `++fade`, noting only
1034+Again, we will delay the discussion of `++fade`, noting only that it takes a
1078−that it takes a beam and parses the hook file there into a hood.
1035+beam and parses the hook file there into a hood.
1079−
1036+
1080−We recurse on this to compile the library. During the compilation, we
1037+We recurse on this to compile the library. During the compilation, we let `zeg`
1081−let `zeg` be `gez` to avoid circular dependencies, we let `hol` be false
1038+be `gez` to avoid circular dependencies, we let `hol` be false since we don't
1082−since we don't allow horns in libraries, and we let `boy` be null so
1039+allow horns in libraries, and we let `boy` be null so that we can isolate the
1083−that we can isolate the new body twigs.
1040+new body twigs.
1084−
1041+
1085−Next, we reintegrate the new data into our context. We use the context
1042+Next, we reintegrate the new data into our context. We use the context created
1086−created by the recursion with four changes. First, we reset `zeg` to our
1043+by the recursion with four changes. First, we reset `zeg` to our old `zeg`.
1087−old `zeg`. Second, we reset `hol` to our old `hol`. Third, we put the
1044+Second, we reset `hol` to our old `hol`. Third, we put the name of our library
1088−name of our library onto the stack of libraries. This means all of a
1045+onto the stack of libraries. This means all of a libraries dependencies will be
1089−libraries dependencies will be earlier in `lot` than the library itself,
1046+earlier in `lot` than the library itself, making `lot` a topological ordering on
1090−making `lot` a topological ordering on the dependency graph. Fourth, we
1047+the dependency graph. Fourth, we put in `bil` the library hoof and body (with
1091−put in `bil` the library hoof and body (with all body twigs collected in
1048+all body twigs collected in a `=~`), keyed by the library name.
1092−a `=~`), keyed by the library name.
1093
1049
1094Finally, we recurse, processing the next library in our list.
1050Finally, we recurse, processing the next library in our list.
1095
1051
@@ -1128,38 +1084,36 @@ To complete our disucssion of `++apex`, we must process our structures.
1128 ==
1084 ==
1129 ^^$(cof cof, bir t.bir)
1085 ^^$(cof cof, bir t.bir)
1130
1086
1131−The processing of our structures is very similar to that of our
1087+The processing of our structures is very similar to that of our libraries. For
1132−libraries. For clarity, we'll use many of the same phrases in describing
1088+clarity, we'll use many of the same phrases in describing the parallel natures.
1133−the parallel natures. First, we check to see if there are more
1089+First, we check to see if there are more structures to process. If not, we're
1134−structures to process. If not, we're done, so we produce our context.
1090+done, so we produce our context.
1135−
1091+
1136−Otherwise, we let `goz` be `zog` plus the current structure so that
1092+Otherwise, we let `goz` be `zog` plus the current structure so that while
1137−while compiling the dependencies of this structure we don't later create
1093+compiling the dependencies of this structure we don't later create a circular
1138−a circular dependency. We check next to see if this structure is alredy
1094+dependency. We check next to see if this structure is alredy in `rop`. If so,
1139−in `rop`. If so, then we have already included this structure earlier,
1095+then we have already included this structure earlier, so we check to see if this
1140−so we check to see if this is the same version of the structure as we
1096+is the same version of the structure as we included earlier. If so, we skip it.
1141−included earlier. If so, we skip it. Else, we fail since we can't
1097+Else, we fail since we can't include two different versions of a structure.
1142−include two different versions of a structure.
1098+
1143−
1099+If we haven't loaded this structure, then we call `++hone` to get the beam where
1144−If we haven't loaded this structure, then we call `++hone` to get the
1100+the file structure should be. If the loobean in the hoot is true, then we're
1145−beam where the file structure should be. If the loobean in the hoot is
1101+looking for a gate; otherwise, we're looking for a core. We parse this file with
1146−true, then we're looking for a gate; otherwise, we're looking for a
1102+`++fade`.
1147−core. We parse this file with `++fade`.
1103+
1148−
1104+Now, we recurse on this to compile the structure. During the recursion, there we
1149−Now, we recurse on this to compile the structure. During the recursion,
1105+have threee changes. Frist, we let `zog` be `goz` so that we don't create a
1150−there we have threee changes. Frist, we let `zog` be `goz` so that we
1106+circular dependency. Second, we let `hol` be false since we do not allow horns
1151−don't create a circular dependency. Second, we let `hol` be false since
1107+in structures. Third, we let `boy` be null so that we can isolate the new body
1152−we do not allow horns in structures. Third, we let `boy` be null so that
1108+twigs.
1153−we can isolate the new body twigs.
1109+
1154−
1110+Next, we reintegrate the new data into our context. We use the context cretaed
1155−Next, we reintegrate the new data into our context. We use the context
1111+by the recursion with four changes. First, if we're including a gate structure,
1156−cretaed by the recursion with four changes. First, if we're including a
1112+then we reset the body to its original body. Else we put on the top of our list
1157−gate structure, then we reset the body to its original body. Else we put
1113+of body twigs what is essentially a `=+ structure-name` to take off the face of
1158−on the top of our list of body twigs what is essentially a
1114+the structure. Second, we reset `zog` to our old `zog`. Third, we reset `hol` to
1159−`=+ structure-name` to take off the face of the structure. Second, we
1115+our old `hol`. Finally, we put in `rop` the structure hoof and body (with all
1160−reset `zog` to our old `zog`. Third, we reset `hol` to our old `hol`.
1116+body twiggs collected in a `=~`), keyed by the structure name.
1161−Finally, we put in `rop` the structure hoof and body (with all body
1162−twiggs collected in a `=~`), keyed by the structure name.
1163
1117
1164Finally, we recurse, processing the next structure in our list.
1118Finally, we recurse, processing the next structure in our list.
1165
1119
@@ -1179,9 +1133,9 @@ This concludes our discussion of `++apex`.
1179 |= [cof=cafe fin=vase]
1133 |= [cof=cafe fin=vase]
1180 (fine cof fin)
1134 (fine cof fin)
1181
1135
1182−Returning to `++abut`, we have now processed the structures, libraries
1136+Returning to `++abut`, we have now processed the structures, libraries and body
1183−and body twigs. Next, we slap our preamble (structures and libraries)
1137+twigs. Next, we slap our preamble (structures and libraries) against zuse.
1184−against zuse. First, we construct our preamble in `++able`.
1138+First, we construct our preamble in `++able`.
1185
1139
1186 ++ able :: assemble preamble
1140 ++ able :: assemble preamble
1187 ^- twig
1141 ^- twig
@@ -1189,10 +1143,10 @@ against zuse. First, we construct our preamble in `++able`.
1189 ?:(=(~ rop) [%$ 1] [%brcn (~(run by rop) |=([* a=twig] [%ash a]))])
1143 ?:(=(~ rop) [%$ 1] [%brcn (~(run by rop) |=([* a=twig] [%ash a]))])
1190 [%tssg (turn (flop lot) |=(a=term q:(need (~(get by bil) a))))]
1144 [%tssg (turn (flop lot) |=(a=term q:(need (~(get by bil) a))))]
1191
1145
1192−We first put the structures in `rop` into a single `|%` at the top and
1146+We first put the structures in `rop` into a single `|%` at the top and `=>` it
1193−`=>` it onto a `=~` of our libraries, in the reverse order that they
1147+onto a `=~` of our libraries, in the reverse order that they appear in `lot`.
1194−appear in `lot`. Thus, the structures are in a single core while the
1148+Thus, the structures are in a single core while the libraries are in consecutive
1195−libraries are in consecutive cores.
1149+cores.
1196
1150
1197We slap the preamble against zuse with `++maim`.
1151We slap the preamble against zuse with `++maim`.
1198
1152
@@ -1209,49 +1163,47 @@ We slap the preamble against zuse with `++maim`.
1209 `vase`[p.p.puz val]
1163 `vase`[p.p.puz val]
1210 ==
1164 ==
1211
1165
1212−Here we start to get into ford's caching system. We wrap our computation
1166+Here we start to get into ford's caching system. We wrap our computation in a
1213−in a call to `++clef` so that we only actually compute it if the result
1167+call to `++clef` so that we only actually compute it if the result is not
1214−is not already in our cache. First we'll discuss the computation, then
1168+already in our cache. First we'll discuss the computation, then we'll discuss
1215−we'll discuss the caching system.
1169+the caching system.
1216
1170
1217−We call `++mule` with a call to `++mint:ut` on the type of our subject
1171+We call `++mule` with a call to `++mint:ut` on the type of our subject vase
1218−vase against the given twig. In other words, we're compiling the twig
1172+against the given twig. In other words, we're compiling the twig with against
1219−with against the subject type in the given subject vase.
1173+the subject type in the given subject vase.
1220
1174
1221−If compilation fails, then we produce an error bolt with the produced
1175+If compilation fails, then we produce an error bolt with the produced stack
1222−stack trace. Otherwise, we run the produced nock with `++mock` and our
1176+trace. Otherwise, we run the produced nock with `++mock` and our sky function.
1223−sky function. We convert the produced toon to a bolt with `++coup` and
1177+We convert the produced toon to a bolt with `++coup` and use the type from `puz`
1224−use the type from `puz` combined with the value from `mock` to produce
1178+combined with the value from `mock` to produce our vase.
1225−our vase.
1226
1179
1227−If this process seems harder than just calling `++slap`, it's because it
1180+If this process seems harder than just calling `++slap`, it's because it is. We
1228−is. We have two requirements that `++slap` doesn't satisfy. First, we
1181+have two requirements that `++slap` doesn't satisfy. First, we want the to use
1229−want the to use an explicit sky function for use with `.^`. With
1182+an explicit sky function for use with `.^`. With `++slap`, you get whatever sky
1230−`++slap`, you get whatever sky function is available in the calling
1183+function is available in the calling context, which in ford is none. Second, we
1231−context, which in ford is none. Second, we want to explicitly handle the
1184+want to explicitly handle the stack trace on failure. `++slap` would cause crash
1232−stack trace on failure. `++slap` would cause crash on failure.
1185+on failure.
1233
1186
1234−We haven't yet discussed either `++clef` or `++coup`. We'll start with
1187+We haven't yet discussed either `++clef` or `++coup`. We'll start with `++coup`
1235−`++coup` to finish the discussion of the computation.
1188+to finish the discussion of the computation.
1236
1189
1237\``++ coup :: toon to bolt |= cof=cafe |* [ton=toon fun=$+(* *)] :- p=cof ^= q ?- -.ton %2 [%2 p=p.ton] %0 [%0 p=*(set beam) q=(fun p.ton)] %1 ~& [%coup-need ((list path) p.ton)] =- ?- -.faw & [%1 p=(sa (turn p.faw |=(a=beam [a *(list tank)])))] | [%2 p=p.faw] == ^= faw |- ^- (each (list beam) (list tank)) ?~ p.ton [%& ~] =+ nex=$(p.ton t.p.ton) =+ pax=(path i.p.ton) ?~ pax [%| (smyt pax) ?:(?=(& -.nex) ~ p.nex)] =+ zis=(tome t.pax) ?~ zis [%| (smyt pax) ?:(?=(& -.nex) ~ p.nex)] ?- -.nex & [%& u.zis p.nex] | nex == ==`
1190\``++ coup :: toon to bolt |= cof=cafe |* [ton=toon fun=$+(* *)] :- p=cof ^= q ?- -.ton %2 [%2 p=p.ton] %0 [%0 p=*(set beam) q=(fun p.ton)] %1 ~& [%coup-need ((list path) p.ton)] =- ?- -.faw & [%1 p=(sa (turn p.faw |=(a=beam [a *(list tank)])))] | [%2 p=p.faw] == ^= faw |- ^- (each (list beam) (list tank)) ?~ p.ton [%& ~] =+ nex=$(p.ton t.p.ton) =+ pax=(path i.p.ton) ?~ pax [%| (smyt pax) ?:(?=(& -.nex) ~ p.nex)] =+ zis=(tome t.pax) ?~ zis [%| (smyt pax) ?:(?=(& -.nex) ~ p.nex)] ?- -.nex & [%& u.zis p.nex] | nex == ==`
1238
1191
1239−Recall that a toon is either a `%0` value, a `%1` block, or a `%2`
1192+Recall that a toon is either a `%0` value, a `%1` block, or a `%2` failure.
1240−failure. Converting a `%2` toon failure into a `%2` bolt failure is
1193+Converting a `%2` toon failure into a `%2` bolt failure is trivial. Converting a
1241−trivial. Converting a `%0` toon value into a `%0` bolt value is easy
1194+`%0` toon value into a `%0` bolt value is easy since we assume there were no
1242−since we assume there were no dependencies. Converting the blocks is
1195+dependencies. Converting the blocks is rather more difficult.
1243−rather more difficult.
1244
1196
1245−To compute `faw`, we recurse through the list of paths in the `%1` toon.
1197+To compute `faw`, we recurse through the list of paths in the `%1` toon. At each
1246−At each one, we make sure with `++tome` that it is, in fact, a beam. If
1198+one, we make sure with `++tome` that it is, in fact, a beam. If so, then we
1247−so, then we check to see if the later paths succeed as well. If so, we
1199+check to see if the later paths succeed as well. If so, we append the current
1248−append the current path to the list of other paths. If not, we produce
1200+path to the list of other paths. If not, we produce the error message we got
1249−the error message we got from processing the rest of the paths. If this
1201+from processing the rest of the paths. If this path is not a beam, then we fail,
1250−path is not a beam, then we fail, producing a list of tanks including
1202+producing a list of tanks including this path and, if later paths fail too,
1251−this path and, if later paths fail too, those paths as well.
1203+those paths as well.
1252
1204
1253−If some paths were not beams, then we produce a `%2` error bolt. If all
1205+If some paths were not beams, then we produce a `%2` error bolt. If all paths
1254−paths were correct, then we produce a `%1` blocking bolt.
1206+were correct, then we produce a `%1` blocking bolt.
1255
1207
1256We will now discuss `++clef`. This is where the cache magic happens.
1208We will now discuss `++clef`. This is where the cache magic happens.
1257
1209
@@ -1286,29 +1238,28 @@ Otherwise, we look up the request in the cache with `++calk`.
1286 ?~ d [~ a] ::
1238 ?~ d [~ a] ::
1287 [d a(p (~(put in p.a) u.d))] ::
1239 [d a(p (~(put in p.a) u.d))] ::
1288
1240
1289−When looking up something in the cache, we mark it if we find it. This
1241+When looking up something in the cache, we mark it if we find it. This way, we
1290−way, we have in our cache the set of all cache entries that have been
1242+have in our cache the set of all cache entries that have been referenced. While
1291−referenced. While we do not at present do anything with this data, it
1243+we do not at present do anything with this data, it should be used to clear out
1292−should be used to clear out old and unused entries in the cache.
1244+old and unused entries in the cache.
1293
1245
1294−Moving on in `++clef`, we check to see if we actually found anything. If
1246+Moving on in `++clef`, we check to see if we actually found anything. If we
1295−we didn't find a cache entry, then we run the computation in `fun`, and
1247+didn't find a cache entry, then we run the computation in `fun`, and examine its
1296−examine its result. If it produced a `%2` error or `%1` block bolt, we
1248+result. If it produced a `%2` error or `%1` block bolt, we just pass that
1297−just pass that through. Otherwise, we produce both the value and an
1249+through. Otherwise, we produce both the value and an updated cache with this new
1298−updated cache with this new entry. We add the entry with `++came`.
1250+entry. We add the entry with `++came`.
1299
1251
1300 ++ came ::
1252 ++ came ::
1301 |= [a=cafe b=calx] :: cache install
1253 |= [a=cafe b=calx] :: cache install
1302 ^- cafe ::
1254 ^- cafe ::
1303 a(q (~(put by q.a) [-.b q.b] b)) ::
1255 a(q (~(put by q.a) [-.b q.b] b)) ::
1304
1256
1305−We key cache entries by the type of computation (`-:calx`) and the
1257+We key cache entries by the type of computation (`-:calx`) and the inputs to the
1306−inputs to the computation (`q:calc`). This just puts the cache line in
1258+computation (`q:calc`). This just puts the cache line in the cache at the
1307−the cache at the correct key.
1259+correct key.
1308
1260
1309−Back in `++clef`, if we did find a cache entry, then we just produce the
1261+Back in `++clef`, if we did find a cache entry, then we just produce the value
1310−value at that cache line. We convert the cache line into a value with
1262+at that cache line. We convert the cache line into a value with `++calf`.
1311−`++calf`.
1312
1263
1313 ++ calf :: reduce calx
1264 ++ calf :: reduce calx
1314 |* sem=* :: a typesystem hack
1265 |* sem=* :: a typesystem hack
@@ -1319,23 +1270,22 @@ value at that cache line. We convert the cache line into a value with
1319 %slam ?>(?=(%slam -.cax) r.cax)
1270 %slam ?>(?=(%slam -.cax) r.cax)
1320 ==
1271 ==
1321
1272
1322−This is simply a typesystem hack. Because the `sem` is passed in through
1273+This is simply a typesystem hack. Because the `sem` is passed in through a wet
1323−a wet gate, we know at type time which of the three cases will be
1274+gate, we know at type time which of the three cases will be chosen. Thus, the
1324−chosen. Thus, the correct type of the value in the cache line gets
1275+correct type of the value in the cache line gets passed through to the caller.
1325−passed through to the caller. This also depends on the fact that
1276+This also depends on the fact that `++clef` is wet. The type stuff here is
1326−`++clef` is wet. The type stuff here is mathematically interesting, but
1277+mathematically interesting, but the action is simple: we get the value from the
1327−the action is simple: we get the value from the cache line.
1278+cache line.
1328
1279
1329This concludes our discussion of `++clef` and `++maim`.
1280This concludes our discussion of `++clef` and `++maim`.
1330
1281
1331−Back in `++abut`, recall that we processed the structures, libraries,
1282+Back in `++abut`, recall that we processed the structures, libraries, and body
1332−and body with `++apex`. Then, we slapped our preamble (structures and
1283+with `++apex`. Then, we slapped our preamble (structures and libraries) against
1333−libraries) against zuse with `++maim`. Next, we process our resources
1284+zuse with `++maim`. Next, we process our resources with `++chap`. Note that we
1334−with `++chap`. Note that we pass in the preamble so that we may refer to
1285+pass in the preamble so that we may refer to anything in there in our resources.
1335−anything in there in our resources.
1336
1286
1337−`++chap` is broken up into a different case for each horn. We'll go
1287+`++chap` is broken up into a different case for each horn. We'll go through them
1338−through them one by one.
1288+one by one.
1339
1289
1340 ++ chap :: produce resources
1290 ++ chap :: produce resources
1341 |= [cof=cafe bax=vase hon=horn]
1291 |= [cof=cafe bax=vase hon=horn]
@@ -1350,9 +1300,9 @@ This is `/~`. We slap the twig against our context.
1350 |= [cof=cafe gat=vase]
1300 |= [cof=cafe gat=vase]
1351 (maul cof gat !>([how arg]))
1301 (maul cof gat !>([how arg]))
1352
1302
1353−This is `/$`. We slap the twig against our context, which we expect to
1303+This is `/$`. We slap the twig against our context, which we expect to produce a
1354−produce a gate. We slam this gate with a sample of `how` and `arg`,
1304+gate. We slam this gate with a sample of `how` and `arg`, which is our location
1355−which is our location and the heel (virtual path extension).
1305+and the heel (virtual path extension).
1356
1306
1357`++maul` is similar to `++maim`, but it slams instead of slaps.
1307`++maul` is similar to `++maim`, but it slams instead of slaps.
1358
1308
@@ -1369,20 +1319,19 @@ which is our location and the heel (virtual path extension).
1369 `vase`[p.top val]
1319 `vase`[p.top val]
1370 ==
1320 ==
1371
1321
1372−We cache slams exactly as we cache slaps. We use `++slit` to find the
1322+We cache slams exactly as we cache slaps. We use `++slit` to find the type of
1373−type of the product of the slam given the types of the gate and the
1323+the product of the slam given the types of the gate and the sample.
1374−sample.
1375
1324
1376−If this type fails, we produce the given stack trace as a `%2` error
1325+If this type fails, we produce the given stack trace as a `%2` error bolt.
1377−bolt. Otherwise, we produce the top produced above combined with the
1326+Otherwise, we produce the top produced above combined with the value we get from
1378−value we get from slamming the values in the vases with `++mong`.
1327+slamming the values in the vases with `++mong`.
1379
1328
1380Back to `++chap`.
1329Back to `++chap`.
1381
1330
1382 %day (chad cof bax %dr p.hon)
1331 %day (chad cof bax %dr p.hon)
1383
1332
1384−This is `/|`. We call `++chad` to convert textual names to relative
1333+This is `/|`. We call `++chad` to convert textual names to relative dates and
1385−dates and process the next horn against each of the discovered paths.
1334+process the next horn against each of the discovered paths.
1386
1335
1387 ++ chad :: atomic list
1336 ++ chad :: atomic list
1388 |= [cof=cafe bax=vase doe=term hon=horn]
1337 |= [cof=cafe bax=vase doe=term hon=horn]
@@ -1406,8 +1355,8 @@ dates and process the next horn against each of the discovered paths.
1406 ?~ yal [[%cube 0 [%atom %n]] 0]
1355 ?~ yal [[%cube 0 [%atom %n]] 0]
1407 (slop (slop [[%atom doe] p.i.yal] q.i.yal) $(yal t.yal))
1356 (slop (slop [[%atom doe] p.i.yal] q.i.yal) $(yal t.yal))
1408
1357
1409−First, we call `++lash` to parse the children of the current beam and
1358+First, we call `++lash` to parse the children of the current beam and pick out
1410−pick out those ones that are of the requested format.
1359+those ones that are of the requested format.
1411
1360
1412 ++ lash :: atomic sequence
1361 ++ lash :: atomic sequence
1413 |= wox=$+(span (unit ,@))
1362 |= wox=$+(span (unit ,@))
@@ -1417,26 +1366,22 @@ pick out those ones that are of the requested format.
1417 |= [cof=cafe arc=arch]
1366 |= [cof=cafe arc=arch]
1418 (fine cof (lark wox arc))
1367 (fine cof (lark wox arc))
1419
1368
1420−First, we get the arch with `++lend`, as described above. We filter and
1369+First, we get the arch with `++lend`, as described above. We filter and parse
1421−parse the child names with `++lark` according to the given parser
1370+the child names with `++lark` according to the given parser function.
1422−function.
1423
1371
1424−In `++chad`, this parser function is `(slat doe)`, which will parse a
1372+In `++chad`, this parser function is `(slat doe)`, which will parse a cord into
1425−cord into an atom of the requested odor. For `%day` the odor is for
1373+an atom of the requested odor. For `%day` the odor is for relative dates.
1426−relative dates.
1427
1374
1428−Thus, we now have a map from atoms of the given odor to the actual child
1375+Thus, we now have a map from atoms of the given odor to the actual child names.
1429−names. We next turn this map into a list and sort it in increasing order
1376+We next turn this map into a list and sort it in increasing order by the atom.
1430−by the atom.
1431
1377
1432−We next convert this list of pairs of atoms and spans to a list of pairs
1378+We next convert this list of pairs of atoms and spans to a list of pairs of
1433−of atoms and vases. We process the given horn once at every child beam,
1379+atoms and vases. We process the given horn once at every child beam, producing
1434−producing the resource at that location.
1380+the resource at that location.
1435
1381
1436−Finally, we convert this list of pairs of atoms and vases to a vase of a
1382+Finally, we convert this list of pairs of atoms and vases to a vase of a list of
1437−list of pairs of atoms to (well-typed) values. Each entry in the list is
1383+pairs of atoms to (well-typed) values. Each entry in the list is of type atom
1438−of type atom with the given odor combined with the type of the produced
1384+with the given odor combined with the type of the produced vase.
1439−vase.
1440
1385
1441Back in `++chap`, we continue parsing resources.
1386Back in `++chap`, we continue parsing resources.
1442
1387
@@ -1445,8 +1390,8 @@ Back in `++chap`, we continue parsing resources.
1445 |= [cof=cafe vax=vase]
1390 |= [cof=cafe vax=vase]
1446 (fine cof [[%face p.hon p.vax] q.vax])
1391 (fine cof [[%face p.hon p.vax] q.vax])
1447
1392
1448−This is `/=`. We process the given horn, giving us a vase. We put as a
1393+This is `/=`. We process the given horn, giving us a vase. We put as a face on
1449−face on the vase so that it may be referred to later by name.
1394+the vase so that it may be referred to later by name.
1450
1395
1451 %fan
1396 %fan
1452 %+ cope
1397 %+ cope
@@ -1463,21 +1408,21 @@ face on the vase so that it may be referred to later by name.
1463 ?~ tev [[%cube 0 [%atom %n]] 0]
1408 ?~ tev [[%cube 0 [%atom %n]] 0]
1464 (slop i.tev $(tev t.tev))
1409 (slop i.tev $(tev t.tev))
1465
1410
1466−This is `/.`. We first process each of the child horns, producing a list
1411+This is `/.`. We first process each of the child horns, producing a list of
1467−of vases. This is done by just recursing on `++chap`. Then, we simply
1412+vases. This is done by just recursing on `++chap`. Then, we simply fold over
1468−fold over this list to create a vase of the list of values.
1413+this list to create a vase of the list of values.
1469
1414
1470 %for $(hon q.hon, s.how (weld (flop p.hon) s.how))
1415 %for $(hon q.hon, s.how (weld (flop p.hon) s.how))
1471
1416
1472−This is `/,`. We simply recurse on the horn with the given path welded
1417+This is `/,`. We simply recurse on the horn with the given path welded onto our
1473−onto our current beam.
1418+current beam.
1474
1419
1475 %hub (chad cof bax %ud p.hon)
1420 %hub (chad cof bax %ud p.hon)
1476
1421
1477−This is `/@`. This is exactly like the processing of `%day` except we
1422+This is `/@`. This is exactly like the processing of `%day` except we expect the
1478−expect the children to be named as unsigned integers rather than
1423+children to be named as unsigned integers rather than relative dates. We process
1479−relative dates. We process the horn at each of the children's locations
1424+the horn at each of the children's locations and produce a list of pairs of
1480−and produce a list of pairs of absolute dates and values.
1425+absolute dates and values.
1481
1426
1482 %man
1427 %man
1483 |- ^- (bolt vase)
1428 |- ^- (bolt vase)
@@ -1493,21 +1438,20 @@ and produce a list of pairs of absolute dates and values.
1493 (slop [[%atom %tas] p.n.p.hon] vax)
1438 (slop [[%atom %tas] p.n.p.hon] vax)
1494 (slop lef rig)
1439 (slop lef rig)
1495
1440
1496−This is `/*`. We process each of the horns in the given map by recursion
1441+This is `/*`. We process each of the horns in the given map by recursion through
1497−through `++chap`. Once we have these vases, we create a vase of a map
1442+`++chap`. Once we have these vases, we create a vase of a map from the given
1498−from the given textual names to the produced values.
1443+textual names to the produced values.
1499
1444
1500 %now (chad cof bax %da p.hon)
1445 %now (chad cof bax %da p.hon)
1501
1446
1502−This is `/&`. This is exactly like the processing of `%now` except we
1447+This is `/&`. This is exactly like the processing of `%now` except we expect the
1503−expect the children to be names as absolute dates rather than relative
1448+children to be names as absolute dates rather than relative dates. We process
1504−dates. We process the horn at each of the children's locations and
1449+the horn at each of the children's locations and produce a list of pairs of
1505−produce a list of pairs of absolute dates and values.
1450+absolute dates and values.
1506
1451
1507 %nap (chai cof bax p.hon)
1452 %nap (chai cof bax p.hon)
1508
1453
1509−This is `/%`. Here, we process the horn at each of our children with
1454+This is `/%`. Here, we process the horn at each of our children with `++chai`.
1510−`++chai`.
1511
1455
1512 ++ chai :: atomic map
1456 ++ chai :: atomic map
1513 |= [cof=cafe bax=vase hon=horn]
1457 |= [cof=cafe bax=vase hon=horn]
@@ -1532,10 +1476,10 @@ This is `/%`. Here, we process the horn at each of our children with
1532 (slop [[%atom %a] p.n.doy] q.n.doy)
1476 (slop [[%atom %a] p.n.doy] q.n.doy)
1533 (slop $(doy l.doy) $(doy r.doy))
1477 (slop $(doy l.doy) $(doy r.doy))
1534
1478
1535−We get the arch at our current beam with `++lend`. Then, we process the
1479+We get the arch at our current beam with `++lend`. Then, we process the horn at
1536−horn at each of our children to give us a map of atoms to vases.
1480+each of our children to give us a map of atoms to vases. Finally, we convert
1537−Finally, we convert that into a vase of a map of these atoms to the
1481+that into a vase of a map of these atoms to the values. This is very similar to
1538−values. This is very similar to `++chad` and the handling of `%man`.
1482+`++chad` and the handling of `%man`.
1539
1483
1540 %see $(hon q.hon, how p.hon)
1484 %see $(hon q.hon, how p.hon)
1541
1485
@@ -1548,9 +1492,9 @@ This is `/:`. We process the given horn at the given beam.
1548 |= [cof=cafe gat=vase]
1492 |= [cof=cafe gat=vase]
1549 (maul cof gat sam)
1493 (maul cof gat sam)
1550
1494
1551−This is `/;`. First, we process the given horn. Then, we slap the given
1495+This is `/;`. First, we process the given horn. Then, we slap the given twig
1552−twig against our context to produce (hopefully) a gate. Finally, we slam
1496+against our context to produce (hopefully) a gate. Finally, we slam the vase we
1553−the vase we got from processing the horn against the gate.
1497+got from processing the horn against the gate.
1554
1498
1555 %sic
1499 %sic
1556 %+ cope $(hon q.hon)
1500 %+ cope $(hon q.hon)
@@ -1561,27 +1505,26 @@ the vase we got from processing the horn against the gate.
1561 (flaw cof [%leaf "type error: {<p.hon>} {<q.hon>}"]~)
1505 (flaw cof [%leaf "type error: {<p.hon>} {<q.hon>}"]~)
1562 (fine cof [p.tug q.vax])
1506 (fine cof [p.tug q.vax])
1563
1507
1564−This is `/^`. First, we process the given horn. Then, we slap the the
1508+This is `/^`. First, we process the given horn. Then, we slap the the bunt of
1565−bunt of the given tile against our context. This will produce a vase
1509+the given tile against our context. This will produce a vase with the correct
1566−with the correct type. We test to see if this type nests within the type
1510+type. We test to see if this type nests within the type of the vase we got from
1567−of the vase we got from processing the horn. If so, we produce the value
1511+processing the horn. If so, we produce the value from the horn along with the
1568−from the horn along with the type from the tile. Otherwise, we produce a
1512+type from the tile. Otherwise, we produce a `%2` error bolt.
1569−`%2` error bolt.
1570
1513
1571 %toy (cope (make cof %bake p.hon how ~) feel)
1514 %toy (cope (make cof %bake p.hon how ~) feel)
1572 ==
1515 ==
1573
1516
1574−This is `/mark/`. Here, we simply run the `%bake` silk on the given
1517+This is `/mark/`. Here, we simply run the `%bake` silk on the given mark,
1575−mark, producing a cage. We convert this cage into a vase with `++feel`,
1518+producing a cage. We convert this cage into a vase with `++feel`, which is
1576−which is exactly as simple as it sounds like it should be.
1519+exactly as simple as it sounds like it should be.
1577
1520
1578 ++ feel |=([a=cafe b=cage] (fine a q.b)) :: cage to vase
1521 ++ feel |=([a=cafe b=cage] (fine a q.b)) :: cage to vase
1579
1522
1580This is trivial.
1523This is trivial.
1581
1524
1582−We will discuss later `++make` and how `%bake` is processed. Suffice it
1525+We will discuss later `++make` and how `%bake` is processed. Suffice it to say
1583−to say that baking a resource with a given mark gets the resource and
1526+that baking a resource with a given mark gets the resource and converts it, if
1584−converts it, if necessary, to the requested mark.
1527+necessary, to the requested mark.
1585
1528
1586This concludes our discussion of `++chap`.
1529This concludes our discussion of `++chap`.
1587
1530
@@ -1601,26 +1544,24 @@ We return once more to `++abut`.
1601 |= [cof=cafe fin=vase]
1544 |= [cof=cafe fin=vase]
1602 (fine cof fin)
1545 (fine cof fin)
1603
1546
1604−Recall that we processed our structures, libraries and body with
1547+Recall that we processed our structures, libraries and body with `++apex`. We
1605−`++apex`. We slapped our structures and libraries against zuse with
1548+slapped our structures and libraries against zuse with `++maim`. We processed
1606−`++maim`. We processed our resources with `++chap`. Now, all our body
1549+our resources with `++chap`. Now, all our body twigs are collected in a `=~` and
1607−twigs are collected in a `=~` and slapped against our structures,
1550+slapped against our structures, libraries, and resources. This produces our
1608−libraries, and resources. This produces our final result.
1551+final result.
1609
1552
1610The hook file has been assembled. And there was great rejoicing.
1553The hook file has been assembled. And there was great rejoicing.
1611
1554
1612−Lifecycle of a Kiss
1555+## Lifecycle of a Kiss
1613−-------------------
1614
1556
1615−We're now going to go through a series of lifecycle descriptions. When a
1557+We're now going to go through a series of lifecycle descriptions. When a user of
1616−user of ford sends a kiss, it is one of a dozen different types of silk.
1558+ford sends a kiss, it is one of a dozen different types of silk. We'll go
1617−We'll go through each one, tracing through the flow of control of each
1559+through each one, tracing through the flow of control of each of these.
1618−of these.
1619
1560
1620First, though, we'll describe the common handling to all kisses.
1561First, though, we'll describe the common handling to all kisses.
1621
1562
1622−The silk in a `%exec` kiss to ford ends up in `++apex`, so we'll enter
1563+The silk in a `%exec` kiss to ford ends up in `++apex`, so we'll enter the
1623−the narrative here.
1564+narrative here.
1624
1565
1625 ++ apex :: call
1566 ++ apex :: call
1626 |= kus=(unit silk)
1567 |= kus=(unit silk)
@@ -1639,13 +1580,13 @@ the narrative here.
1639 ==
1580 ==
1640 ~(exec zo [num `task`[hen u.kus 0 ~]])
1581 ~(exec zo [num `task`[hen u.kus 0 ~]])
1641
1582
1642−Recall that a `%exec` kiss actually sends a unit silk. If it's null,
1583+Recall that a `%exec` kiss actually sends a unit silk. If it's null, we're
1643−we're trying to cancel the request. We first look up the task number
1584+trying to cancel the request. We first look up the task number keyed by duct. If
1644−keyed by duct. If we don't find it, then we're trying to cancel a
1585+we don't find it, then we're trying to cancel a request that either was never
1645−request that either was never started or has already completed. We print
1586+started or has already completed. We print out `%ford-mystery` and do nothing.
1646−out `%ford-mystery` and do nothing. If we do find the task number, then
1587+If we do find the task number, then we look up the task from it, call
1647−we look up the task from it, call `++camo:zo` to cancel pending
1588+`++camo:zo` to cancel pending requests, and call `++amok:zo` to remove the task
1648−requests, and call `++amok:zo` to remove the task from our task lists.
1589+from our task lists.
1649
1590
1650 ++ camo :: stop requests
1591 ++ camo :: stop requests
1651 ^+ .
1592 ^+ .
@@ -1661,9 +1602,9 @@ requests, and call `++amok:zo` to remove the task from our task lists.
1661 [%warp [our p.q.i.kiz] q.q.i.kiz ~]
1602 [%warp [our p.q.i.kiz] q.q.i.kiz ~]
1662 ==
1603 ==
1663
1604
1664−Our list of blocks is in `q.kig`, so we iterate over it, cancelling our
1605+Our list of blocks is in `q.kig`, so we iterate over it, cancelling our pending
1665−pending requests for each block. Our requests are all to clay, so we
1606+requests for each block. Our requests are all to clay, so we need only to send
1666−need only to send `%warp` kisses with a null instead of a rave.
1607+`%warp` kisses with a null instead of a rave.
1667
1608
1668 ++ amok
1609 ++ amok
1669 %_ ..zo
1610 %_ ..zo
@@ -1671,12 +1612,12 @@ need only to send `%warp` kisses with a null instead of a rave.
1671 dym.bay (~(del by dym.bay) nah)
1612 dym.bay (~(del by dym.bay) nah)
1672 ==
1613 ==
1673
1614
1674−We remove the task number from the map of numbers to tasks and the duct
1615+We remove the task number from the map of numbers to tasks and the duct from the
1675−from the map of ducts to task numbers.
1616+map of ducts to task numbers.
1676
1617
1677−Back in `++apex`, if we were given a silk, we need to process it. We add
1618+Back in `++apex`, if we were given a silk, we need to process it. We add the
1678−the task to our maps, increment the next task number, and call
1619+task to our maps, increment the next task number, and call `++exec:zo` on the
1679−`++exec:zo` on the new task.
1620+new task.
1680
1621
1681 ++ exec :: execute app
1622 ++ exec :: execute app
1682 ^+ ..zo
1623 ^+ ..zo
@@ -1697,9 +1638,9 @@ the task to our maps, increment the next task number, and call
1697
1638
1698If we're still blocked on something in `q.kig`, we don't do anything.
1639If we're still blocked on something in `q.kig`, we don't do anything.
1699
1640
1700−Otherwise, we try to process the silk with `++make`. `++make` handles
1641+Otherwise, we try to process the silk with `++make`. `++make` handles each
1701−each individual request and will be the entire focus of the remainder of
1642+individual request and will be the entire focus of the remainder of this doc
1702−this doc after this section. It produces a bolt of a cage.
1643+after this section. It produces a bolt of a cage.
1703
1644
1704We put the new cache in our state with `++dash`.
1645We put the new cache in our state with `++dash`.
1705
1646
@@ -1708,13 +1649,12 @@ We put the new cache in our state with `++dash`.
1708 ^+ +>
1649 ^+ +>
1709 %_(+> jav.bay q.cof)
1650 %_(+> jav.bay q.cof)
1710
1651
1711−The cache is put in the baby so that it gets stored across calls to
1652+The cache is put in the baby so that it gets stored across calls to ford.
1712−ford.
1713
1653
1714−In `++exec`, we process the bolt in three different ways according to
1654+In `++exec`, we process the bolt in three different ways according to the type
1715−the type of bolt produced. If we produced a `%0` value bolt, we use
1655+of bolt produced. If we produced a `%0` value bolt, we use `++expo` to give the
1716−`++expo` to give the produced value and set of dependencies as a `%made`
1656+produced value and set of dependencies as a `%made` gift, and we remove
1717−gift, and we remove ourselves from the task list with `++amok`.
1657+ourselves from the task list with `++amok`.
1718
1658
1719 ++ expo :: return gift
1659 ++ expo :: return gift
1720 |= gef=gift
1660 |= gef=gift
@@ -1722,11 +1662,11 @@ gift, and we remove ourselves from the task list with `++amok`.
1722
1662
1723We simply push the gift onto our list of moves.
1663We simply push the gift onto our list of moves.
1724
1664
1725−In `++exec`, if we produced a `%2` error bolt, we produce a `%made` gift
1665+In `++exec`, if we produced a `%2` error bolt, we produce a `%made` gift with
1726−with the stack trace.
1666+the stack trace.
1727
1667
1728−If we produced a `%1` block bolt, we iterate through each of the blocks
1668+If we produced a `%1` block bolt, we iterate through each of the blocks and call
1729−and call `++camp` to produce a clay request for the resource.
1669+`++camp` to produce a clay request for the resource.
1730
1670
1731 ++ camp :: request a file
1671 ++ camp :: request a file
1732 |= [ren=care bem=beam]
1672 |= [ren=care bem=beam]
@@ -1740,16 +1680,13 @@ and call `++camp` to produce a clay request for the resource.
1740 [%warp [our p.bem] q.bem [~ %& %x r.bem s.bem]]
1680 [%warp [our p.bem] q.bem [~ %& %x r.bem s.bem]]
1741 ==
1681 ==
1742
1682
1743−We put the resource in our block list in `q.kig` so that we save the
1683+We put the resource in our block list in `q.kig` so that we save the fact that
1744−fact that we're blocked. We then produce the `%warp` request to clay for
1684+we're blocked. We then produce the `%warp` request to clay for the resource. Our
1745−the resource. Our request path has the format
1685+request path has the format \`/\[our-ship\]/\[task-number\]/\[block-number\]'.
1746−\`/[our-ship]/[task-number]/[block-number]'.
1747
1686
1748−We'll now describe how each of the individual silks are processed in
1687+We'll now describe how each of the individual silks are processed in `++make`.
1749−`++make`.
1750
1688
1751−Lifecycle of a Cell
1689+## Lifecycle of a Cell
1752−-------------------
1753
1690
1754 ^
1691 ^
1755 %. [cof p.kas q.kas]
1692 %. [cof p.kas q.kas]
@@ -1763,10 +1700,10 @@ Lifecycle of a Cell
1763 [p=cof q=[%0 ~ [%$ (slop q.bor q.heg)]]]
1700 [p=cof q=[%0 ~ [%$ (slop q.bor q.heg)]]]
1764 ==
1701 ==
1765
1702
1766−Silks autocons. The product of a cell of silks is a cell of the products
1703+Silks autocons. The product of a cell of silks is a cell of the products of the
1767−of the silks, so we evaluate the two silks in parallel with `++coax` and
1704+silks, so we evaluate the two silks in parallel with `++coax` and slop together
1768−slop together the results in a cell vase. We mark the product with `%$`,
1705+the results in a cell vase. We mark the product with `%$`, which means we know
1769−which means we know no more mark information than that it is a noun.
1706+no more mark information than that it is a noun.
1770
1707
1771 ++ coax :: bolt across
1708 ++ coax :: bolt across
1772 |* [hoc=(bolt) fun=(burg)]
1709 |* [hoc=(bolt) fun=(burg)]
@@ -1790,23 +1727,19 @@ which means we know no more mark information than that it is a noun.
1790 %2 hoc
1727 %2 hoc
1791 ==
1728 ==
1792
1729
1793−If the first bolt is a value, we evaluate the burg to get the next bolt.
1730+If the first bolt is a value, we evaluate the burg to get the next bolt. If that
1794−If that also produces a value, we merge the dependency sets and produce
1731+also produces a value, we merge the dependency sets and produce a cell of the
1795−a cell of the two values. Otherwise, we produce the block or error of
1732+two values. Otherwise, we produce the block or error of the second bolt.
1796−the second bolt.
1797
1733
1798−If the first bolt is a block, we evaluate the burg to get the next bolt.
1734+If the first bolt is a block, we evaluate the burg to get the next bolt. If that
1799−If that produces a value, we just produce the block. If it produces a
1735+produces a value, we just produce the block. If it produces a block, we merge
1800−block, we merge the two block sets. If it produces an error, we produce
1736+the two block sets. If it produces an error, we produce that error.
1801−that error.
1802
1737
1803If the first bolt is already an error, we just pass that through.
1738If the first bolt is already an error, we just pass that through.
1804
1739
1805−Note that `++coax` (and, indeed, `++cope`) is reasonable to use with
1740+Note that `++coax` (and, indeed, `++cope`) is reasonable to use with `;~`.
1806−`;~`.
1807
1741
1808−Lifecycle of a `%bake`
1742+## Lifecycle of a `%bake`
1809−----------------------
1810
1743
1811 %bake
1744 %bake
1812 %+ cool |.(leaf/"ford: bake {<p.kas>} {<(tope q.kas)>}")
1745 %+ cool |.(leaf/"ford: bake {<p.kas>} {<(tope q.kas)>}")
@@ -1816,12 +1749,11 @@ Lifecycle of a `%bake`
1816 (flaw cof (smyt (tope q.kas)) ~)
1749 (flaw cof (smyt (tope q.kas)) ~)
1817 (fine cof [p.kas u.vux])
1750 (fine cof [p.kas u.vux])
1818
1751
1819−This is one of the most critical silks. We are going to functionally
1752+This is one of the most critical silks. We are going to functionally produce the
1820−produce the hook file at the given beam with the given heel. The result
1753+hook file at the given beam with the given heel. The result will be of the
1821−will be of the correct mark, even if we need to run conversion
1754+correct mark, even if we need to run conversion functions. The functionality is
1822−functions. The functionality is encapsulated in `++lime`. If it produces
1755+encapsulated in `++lime`. If it produces null, then we produce an error.
1823−null, then we produce an error. Otherwise, we take the vase produced and
1756+Otherwise, we take the vase produced and give it the correct mark.
1824−give it the correct mark.
1825
1757
1826 ++ lima :: load at depth
1758 ++ lima :: load at depth
1827 |= [cof=cafe for=mark bem=beam arg=heel]
1759 |= [cof=cafe for=mark bem=beam arg=heel]
@@ -1843,19 +1775,19 @@ give it the correct mark.
1843 |= [cof=cafe vax=vase]
1775 |= [cof=cafe vax=vase]
1844 (fine cof ~ vax)
1776 (fine cof ~ vax)
1845
1777
1846−First, we load the arch at the given beam with `++lend`. If we have a
1778+First, we load the arch at the given beam with `++lend`. If we have a child
1847−child named the mark, our job is straightforward, so we go ahead and
1779+named the mark, our job is straightforward, so we go ahead and load that with
1848−load that with `++lace`.
1780+`++lace`.
1849
1781
1850−Otherwise, we iterate through our children. If we have no children, we
1782+Otherwise, we iterate through our children. If we have no children, we produce
1851−produce null, signifying that we didn't find any way to convert to the
1783+null, signifying that we didn't find any way to convert to the requested mark.
1852−requested mark. Otherwise, we call `++lion` to find a translation path
1784+Otherwise, we call `++lion` to find a translation path from one of the available
1853−from one of the available marks into the target mark. We recursively
1785+marks into the target mark. We recursively bake the child that has a path to the
1854−bake the child that has a path to the target mark, and then we call
1786+target mark, and then we call `++lope` to translate this mark into the target
1855−`++lope` to translate this mark into the target mark.
1787+mark.
1856
1788
1857−We'll first discuss the direct case of when one of our children is of
1789+We'll first discuss the direct case of when one of our children is of the
1858−the correct mark.
1790+correct mark.
1859
1791
1860 ++ lace :: load and check
1792 ++ lace :: load and check
1861 |= [cof=cafe for=mark bem=beam arg=heel]
1793 |= [cof=cafe for=mark bem=beam arg=heel]
@@ -1871,11 +1803,10 @@ the correct mark.
1871 (cope (cope (abut:(meow bem arg) cof hyd) (lake for bek)) fest)
1803 (cope (cope (abut:(meow bem arg) cof hyd) (lake for bek)) fest)
1872 (fine cof ~)
1804 (fine cof ~)
1873
1805
1874−First, we get the arch at the given beam with `++lend`. If this is a
1806+First, we get the arch at the given beam with `++lend`. If this is a file, we
1875−file, we load the file with `++liar` and coerce the type with `++lake`.
1807+load the file with `++liar` and coerce the type with `++lake`. Otherwise, we
1876−Otherwise, we check to see if we have a hook file here. If so, we parse
1808+check to see if we have a hook file here. If so, we parse it with `++fade`,
1877−it with `++fade`, compile it with `++abut:meow`, and coerce the type
1809+compile it with `++abut:meow`, and coerce the type with `++lake`.
1878−with `++lake`.
1879
1810
1880Otherwise, there is no way to translate this, so we produce null.
1811Otherwise, there is no way to translate this, so we produce null.
1881
1812
@@ -1886,9 +1817,8 @@ Otherwise, there is no way to translate this, so we produce null.
1886This is just `++some` for bolts.
1817This is just `++some` for bolts.
1887
1818
1888We've delayed the discussion of `++fade` far too many times. It's not
1819We've delayed the discussion of `++fade` far too many times. It's not
1889−complicated, we just wanted to spare a premature discussion of `++make`
1820+complicated, we just wanted to spare a premature discussion of `++make` and the
1890−and the `%bake` silk. We 're now able to discuss everything in `++fade`
1821+`%bake` silk. We 're now able to discuss everything in `++fade` with ease.
1891−with ease.
1892
1822
1893 ++ fade :: compile to hood
1823 ++ fade :: compile to hood
1894 |= [cof=cafe for=mark bem=beam]
1824 |= [cof=cafe for=mark bem=beam]
@@ -1907,18 +1837,17 @@ with ease.
1907 (flaw cof [%leaf "syntax error: {<p.p.vex>} {<q.p.vex>}"] ~)
1837 (flaw cof [%leaf "syntax error: {<p.p.vex>} {<q.p.vex>}"] ~)
1908 (fine cof p.u.q.vex)
1838 (fine cof p.u.q.vex)
1909
1839
1910−We first push a line onto a stack trace to say that we're parsing into a
1840+We first push a line onto a stack trace to say that we're parsing into a hood
1911−hood file.
1841+file.
1912
1842
1913−We bake the given beam with the given mark and no heel. Recall that
1843+We bake the given beam with the given mark and no heel. Recall that baking gate,
1914−baking gate, core, door, hoon, and hook files produces simply an atom of
1844+core, door, hoon, and hook files produces simply an atom of the text. We check
1915−the text. We check to make sure that our value is an atom, failing
1845+to make sure that our value is an atom, failing otherwise.
1916−otherwise.
1917
1846
1918−The parsing step is run within `++clef` so that the result is cached. We
1847+The parsing step is run within `++clef` so that the result is cached. We call
1919−call `++fair` with the current beam to generate the parsing rule, and we
1848+`++fair` with the current beam to generate the parsing rule, and we parse the
1920−parse the file. If parsing fails, we fail giving a syntax error with the
1849+file. If parsing fails, we fail giving a syntax error with the line and column
1921−line and column number. Otherwise, we produce the value.
1850+number. Otherwise, we produce the value.
1922
1851
1923 ++ liar :: load vase
1852 ++ liar :: load vase
1924 |= [cof=cafe bem=beam]
1853 |= [cof=cafe bem=beam]
@@ -1930,15 +1859,14 @@ line and column number. Otherwise, we produce the value.
1930 (flaw cof (smyt (tope bem)) ~)
1859 (flaw cof (smyt (tope bem)) ~)
1931 (fine cof ?^(u.u.von [%cell %noun %noun] [%atom %$]) u.u.von)
1860 (fine cof ?^(u.u.von [%cell %noun %noun] [%atom %$]) u.u.von)
1932
1861
1933−This takes a beam and loads the file at that location. If our sky
1862+This takes a beam and loads the file at that location. If our sky function
1934−function produces null, that means the resource is currently
1863+produces null, that means the resource is currently unavailable, so we block on
1935−unavailable, so we block on it. If it produces `[~ ~]`, that means our
1864+it. If it produces `[~ ~]`, that means our resource is permanently unavailable,
1936−resource is permanently unavailable, so we produce an error. Otherwise,
1865+so we produce an error. Otherwise, we produce the value there with a type of
1937−we produce the value there with a type of either a cell of two nouns or
1866+either a cell of two nouns or an atom, depending on whether the value is a cell
1938−an atom, depending on whether the value is a cell or not.
1867+or not.
1939
1868
1940−Back in `++lima`, recall that we call `++lion` to find a translation
1869+Back in `++lima`, recall that we call `++lion` to find a translation path.
1941−path.
1942
1870
1943 ++ lion :: translation search
1871 ++ lion :: translation search
1944 |= [cof=cafe too=@tas bek=beak fro=(list ,@tas)]
1872 |= [cof=cafe too=@tas bek=beak fro=(list ,@tas)]
@@ -1973,41 +1901,38 @@ path.
1973 (fine cof ?~(ked ~ [~ for u.ked]))
1901 (fine cof ?~(ked ~ [~ for u.ked]))
1974 --
1902 --
1975
1903
1976−At a high level, we have `++apex` and `++apse`. `++apex` takes a list of
1904+At a high level, we have `++apex` and `++apse`. `++apex` takes a list of marks
1977−marks to try in succession until we find one that can be translated into
1905+to try in succession until we find one that can be translated into the target
1978−the target mark. On each one, it calls `++apse`, which takes a single
1906+mark. On each one, it calls `++apse`, which takes a single mark and tries to
1979−mark and tries to find a translation path from this mark to the target.
1907+find a translation path from this mark to the target. To do this, it sees which
1980−To do this, it sees which marks we know how to directly translate to,
1908+marks we know how to directly translate to, and calls `++apex` on this list. The
1981−and calls `++apex` on this list. The result of this mututal recursion is
1909+result of this mututal recursion is a depth-first search of the translation
1982−a depth-first search of the translation graph to find the target mark.
1910+graph to find the target mark. Since the translation graph is not necessarily
1983−Since the translation graph is not necessarily acyclic, we maintain a
1911+acyclic, we maintain a set of marks that we've already tried.
1984−set of marks that we've already tried.
1985
1912
1986−We kick off our search in `++apex`, starting with the given initial list
1913+We kick off our search in `++apex`, starting with the given initial list of
1987−of marks that we know how to get to.
1914+marks that we know how to get to.
1988
1915
1989−If `++apex` is called with a bolt other than a `%0` value bolt, we
1916+If `++apex` is called with a bolt other than a `%0` value bolt, we simply
1990−simply produce it. Otherwise, we check to see if the list of available
1917+produce it. Otherwise, we check to see if the list of available marks to
1991−marks to investigate is null. If so, then we're done, so we produce a
1918+investigate is null. If so, then we're done, so we produce a `%0` bolt with a
1992−`%0` bolt with a null list of accessible marks.
1919+null list of accessible marks.
1993
1920
1994Otherwise, we process this next mark with `++apse`, which will produce a
1921Otherwise, we process this next mark with `++apse`, which will produce a
1995−possible list of marks from this one to the target mark. If it fails to
1922+possible list of marks from this one to the target mark. If it fails to produce
1996−produce a `%0` bolt, we just produce that. Otherwise, if it produces
1923+a `%0` bolt, we just produce that. Otherwise, if it produces null, we can't get
1997−null, we can't get to our target through this mark, so we move on to the
1924+to our target through this mark, so we move on to the next one.
1998−next one.
1999
1925
2000−If it doesn't produce null, then we have successfully found a
1926+If it doesn't produce null, then we have successfully found a translation path,
2001−translation path, so we produce it.
1927+so we produce it.
2002
1928
2003−In `++apse`, we first test to see if we've arrived at the target path.
1929+In `++apse`, we first test to see if we've arrived at the target path. If so,
2004−If so, we're done, so we produce a list including just ourself.
1930+we're done, so we produce a list including just ourself. Otherwise, we check to
2005−Otherwise, we check to see if we've already tried this mark. If so, we
1931+see if we've already tried this mark. If so, we know we can't succeed here, so
2006−know we can't succeed here, so we produce null. Otherwise, we put
1932+we produce null. Otherwise, we put ourselves in the set of already-tried marks,
2007−ourselves in the set of already-tried marks, and we move on.
1933+and we move on.
2008
1934
2009−We call `++lily` to get the list of marks we can translate this one
1935+We call `++lily` to get the list of marks we can translate this one into.
2010−into.
2011
1936
2012 ++ lily :: translation targets
1937 ++ lily :: translation targets
2013 |= [cof=cafe for=mark bek=beak]
1938 |= [cof=cafe for=mark bek=beak]
@@ -2026,29 +1951,27 @@ into.
2026 =+ gow=(slap vax [%cnzy %grow])
1951 =+ gow=(slap vax [%cnzy %grow])
2027 (sloe p.gow)
1952 (sloe p.gow)
2028
1953
2029−We call `++fang` to get the mark definition door. This is documented
1954+We call `++fang` to get the mark definition door. This is documented under
2030−under `%vale`. If getting the mark fails, we produce null because we
1955+`%vale`. If getting the mark fails, we produce null because we can't translate a
2031−can't translate a non-existent mark into anything.
1956+non-existent mark into anything.
2032
1957
2033−Otherwise, we examine the door. The door may have a `++garb`, which is
1958+Otherwise, we examine the door. The door may have a `++garb`, which is simply a
2034−simply a list of marks which know how to translate from the current one.
1959+list of marks which know how to translate from the current one. There must be a
2035−There must be a corresponding `++grab` in the definition of the other
1960+corresponding `++grab` in the definition of the other mark, though we don't
2036−mark, though we don't check that here.
1961+check that here.
2037
1962
2038−The door may also have a `++grow`, which defines how to translate this
1963+The door may also have a `++grow`, which defines how to translate this mark into
2039−mark into another one. Each arm in `++grow` is the name of a mark we can
1964+another one. Each arm in `++grow` is the name of a mark we can translate into.
2040−translate into. The call to `++sloe` simply produces a list of arm names
1965+The call to `++sloe` simply produces a list of arm names in `++grow`.
2041−in `++grow`.
2042
1966
2043−Back in `++apse:lion`, we take the list of translation targets we just
1967+Back in `++apse:lion`, we take the list of translation targets we just found and
2044−found and call `++apex` on it. If we got back a null, we produce a null;
1968+call `++apex` on it. If we got back a null, we produce a null; otherwise, we
2045−otherwise, we produce the list of marks we got back plus the current
1969+produce the list of marks we got back plus the current mark.
2046−mark.
2047
1970
2048This concludes our discussion of `++lion`.
1971This concludes our discussion of `++lion`.
2049
1972
2050−The final piece of `++lima` is `++lope`, which performs the actual
1973+The final piece of `++lima` is `++lope`, which performs the actual translation
2051−translation along the path we just computed.
1974+along the path we just computed.
2052
1975
2053 ++ lope :: translation pipe
1976 ++ lope :: translation pipe
2054 |= [cof=cafe for=mark yaw=(list mark) bek=beak vax=vase]
1977 |= [cof=cafe for=mark yaw=(list mark) bek=beak vax=vase]
@@ -2058,14 +1981,12 @@ translation along the path we just computed.
2058 |= [cof=cafe yed=vase]
1981 |= [cof=cafe yed=vase]
2059 ^$(cof cof, for i.yaw, yaw t.yaw, vax yed)
1982 ^$(cof cof, for i.yaw, yaw t.yaw, vax yed)
2060
1983
2061−We iterate through our list, calling `++link` on every adjacent pair of
1984+We iterate through our list, calling `++link` on every adjacent pair of marks,
2062−marks, translating from one mark to the next until we finish the list of
1985+translating from one mark to the next until we finish the list of marks. A call
2063−marks. A call to `++link` is equivalent to a `%cast` silk, so we
1986+to `++link` is equivalent to a `%cast` silk, so we document it there. After
2064−document it there. After we've called performed every step in the
1987+we've called performed every step in the translation pipeline, we're done.
2065−translation pipeline, we're done.
2066
1988
2067−Lifecycle of a `%boil`
1989+## Lifecycle of a `%boil`
2068−----------------------
2069
1990
2070 %boil
1991 %boil
2071 %+ cool |.(leaf/"ford: boil {<p.kas>} {<(tope q.kas)>} {<r.kas>}")
1992 %+ cool |.(leaf/"ford: boil {<p.kas>} {<(tope q.kas)>} {<r.kas>}")
@@ -2075,13 +1996,13 @@ Lifecycle of a `%boil`
2075 |= [cof=cafe vax=vase]
1996 |= [cof=cafe vax=vase]
2076 (fine cof `cage`[p.kas vax])
1997 (fine cof `cage`[p.kas vax])
2077
1998
2078−At a high level, we try to bake at the given beam, and if it fails, we
1999+At a high level, we try to bake at the given beam, and if it fails, we go up a
2079−go up a level and try again. This is the usual semantics of ford, and
2000+level and try again. This is the usual semantics of ford, and this should nearly
2080−this should nearly always be preferred over directly baking.
2001+always be preferred over directly baking.
2081
2002
2082−First, we normalize the version case to a number with `++lamp`. This
2003+First, we normalize the version case to a number with `++lamp`. This allows
2083−allows caching to be based on revision number rather than something more
2004+caching to be based on revision number rather than something more ephemeral like
2084−ephemeral like a particular time.
2005+a particular time.
2085
2006
2086 ++ lamp :: normalize version
2007 ++ lamp :: normalize version
2087 |= [cof=cafe bem=beam]
2008 |= [cof=cafe bem=beam]
@@ -2090,11 +2011,10 @@ ephemeral like a particular time.
2090 ?~ von [p=cof q=[%1 [bem ~] ~ ~]]
2011 ?~ von [p=cof q=[%1 [bem ~] ~ ~]]
2091 (fine cof bem(r [%ud ((hard ,@) (need u.von))]))
2012 (fine cof bem(r [%ud ((hard ,@) (need u.von))]))
2092
2013
2093−We call the sky function with `%cw`, asking clay for the revision number
2014+We call the sky function with `%cw`, asking clay for the revision number at this
2094−at this case. If the case refers to a revision that isn't there yet, we
2015+case. If the case refers to a revision that isn't there yet, we produce a `%1`
2095−produce a `%1` blocking bolt. Otherwise, we require that the value exist
2016+blocking bolt. Otherwise, we require that the value exist and that it's a
2096−and that it's a number, both of which are guaranteed by clay. We produce
2017+number, both of which are guaranteed by clay. We produce this number.
2097−this number.
2098
2018
2099Next for `%boil` we call `++lime` to try to load the beam.
2019Next for `%boil` we call `++lime` to try to load the beam.
2100
2020
@@ -2109,14 +2029,13 @@ Next for `%boil` we call `++lime` to try to load the beam.
2109 (flaw cof (smyt (tope bem)) ~)
2029 (flaw cof (smyt (tope bem)) ~)
2110 ^$(s.mob t.s.mob, mer [i.s.mob mer])
2030 ^$(s.mob t.s.mob, mer [i.s.mob mer])
2111
2031
2112−We start at the given beam and try to bake it. If it succeeds, we're
2032+We start at the given beam and try to bake it. If it succeeds, we're good.
2113−good. Otherwise, we pop off the top level of the path and put it in our
2033+Otherwise, we pop off the top level of the path and put it in our heel (virtual
2114−heel (virtual path extension). We do this recursively until either we
2034+path extension). We do this recursively until either we find something we can
2115−find something we can bake or we've gone all the way up to the root path
2035+bake or we've gone all the way up to the root path of the desk, in which case we
2116−of the desk, in which case we fail.
2036+fail.
2117
2037
2118−Lifecycle of a `%call`
2038+## Lifecycle of a `%call`
2119−----------------------
2120
2039
2121 %call
2040 %call
2122 %+ cool |.(leaf/"ford: call {<`@p`(mug kas)>}")
2041 %+ cool |.(leaf/"ford: call {<`@p`(mug kas)>}")
@@ -2134,15 +2053,13 @@ Lifecycle of a `%call`
2134 (fine cof %noun vax)
2053 (fine cof %noun vax)
2135 ==
2054 ==
2136
2055
2137−This is slam for silks. We process both of the given silks in parallel
2056+This is slam for silks. We process both of the given silks in parallel with
2138−with `++coax`. We then slam the two produced vases together with
2057+`++coax`. We then slam the two produced vases together with `++maul` and mark
2139−`++maul` and mark the produced vase with `%noun` since we don't know any
2058+the produced vase with `%noun` since we don't know any more specific mark.
2140−more specific mark.
2141
2059
2142`++coax` is documented under Lifecycle of a Cell.
2060`++coax` is documented under Lifecycle of a Cell.
2143
2061
2144−Lifecycle of a `%cast`
2062+## Lifecycle of a `%cast`
2145−----------------------
2146
2063
2147 %cast
2064 %cast
2148 %+ cool |.(leaf/"ford: cast {<p.kas>}")
2065 %+ cool |.(leaf/"ford: cast {<p.kas>}")
@@ -2152,11 +2069,11 @@ Lifecycle of a `%cast`
2152 |= [cof=cafe vax=vase]
2069 |= [cof=cafe vax=vase]
2153 (fine cof [p.kas vax])
2070 (fine cof [p.kas vax])
2154
2071
2155−This is a request to convert data of one mark to another mark directly.
2072+This is a request to convert data of one mark to another mark directly. We
2156−We evaluate the given silk and pass the result into `++link`, which
2073+evaluate the given silk and pass the result into `++link`, which performs the
2157−performs the actual translation. Note that this will not search for
2074+actual translation. Note that this will not search for indirect conversion
2158−indirect conversion paths, so the conversion must be defined either in
2075+paths, so the conversion must be defined either in the `++grow` of the given
2159−the `++grow` of the given mark or the `++grab` of the target mark.
2076+mark or the `++grab` of the target mark.
2160
2077
2161 ++ link :: translate
2078 ++ link :: translate
2162 |= [cof=cafe too=mark for=mark bek=beak vax=vase]
2079 |= [cof=cafe too=mark for=mark bek=beak vax=vase]
@@ -2181,29 +2098,28 @@ the `++grow` of the given mark or the `++grab` of the target mark.
2181 (flaw cof [%leaf "ford: no link: {<[for too]>}"]~)
2098 (flaw cof [%leaf "ford: no link: {<[for too]>}"]~)
2182 (maul cof u.zat vax)
2099 (maul cof u.zat vax)
2183
2100
2184−This performs one step in the translation pipeline. If the given and
2101+This performs one step in the translation pipeline. If the given and target
2185−target marks are the same, we're done. If we're translating from a noun
2102+marks are the same, we're done. If we're translating from a noun or the empty
2186−or the empty mark, we coerce with `++lake` (documented in `%vale`).
2103+mark, we coerce with `++lake` (documented in `%vale`). Otherwise, we're
2187−Otherwise, we're translating from a user-defined mark.
2104+translating from a user-defined mark.
2188
2105
2189−We load the definition of the given mark with `++fang`, and we check to
2106+We load the definition of the given mark with `++fang`, and we check to see if
2190−see if it has an arm in `++grow` named the target mark. If so, we place
2107+it has an arm in `++grow` named the target mark. If so, we place our data in the
2191−our data in the sample of the door with `++keel` and slap the arm.
2108+sample of the door with `++keel` and slap the arm. `++keel` is equivalent to a
2192−`++keel` is equivalent to a `%mute` silk, so we document it there.
2109+`%mute` silk, so we document it there.
2193
2110
2194−If there is no arm in `++grow` of the given mark named the target mark,
2111+If there is no arm in `++grow` of the given mark named the target mark, we
2195−we suppose there must be an arm in `++grab` of the target mark named the
2112+suppose there must be an arm in `++grab` of the target mark named the given
2196−given mark. We get the definition of the target mark and check to see if
2113+mark. We get the definition of the target mark and check to see if it has the
2197−it has the required arm, failing if it doesn't. Finally, we slam the
2114+required arm, failing if it doesn't. Finally, we slam the data against the
2198−data against the correct arm, producing the translated data.
2115+correct arm, producing the translated data.
2199
2116
2200If you're confused as to why the handling of `++grow` and `++grab` look
2117If you're confused as to why the handling of `++grow` and `++grab` look
2201−superficially so different, remember that the correct arm in `++grow`
2118+superficially so different, remember that the correct arm in `++grow` does not
2202−does not have a sample while the one in `++grab` does. This means they
2119+have a sample while the one in `++grab` does. This means they must be called
2203−must be called rather differently.
2120+rather differently.
2204
2121
2205−Lifecycle of a `%diff`
2122+## Lifecycle of a `%diff`
2206−----------------------
2207
2123
2208 %diff
2124 %diff
2209 %+ cool |.(leaf/"ford: diff {<`@p`(mug p.kas)>} {<`@p`(mug q.kas)>}")
2125 %+ cool |.(leaf/"ford: diff {<`@p`(mug p.kas)>} {<`@p`(mug q.kas)>}")
@@ -2228,8 +2144,7 @@ First, we process the two given silks to get our arguments.
2228 %+ flaw cof :_ ~
2144 %+ flaw cof :_ ~
2229 leaf/"diff on data of different marks: {(trip p.cay)} {(trip p.coy)}"
2145 leaf/"diff on data of different marks: {(trip p.cay)} {(trip p.coy)}"
2230
2146
2231−If the two cages have different marks, then we can't diff them, so we
2147+If the two cages have different marks, then we can't diff them, so we complain.
2232−complain.
2233
2148
2234 %+ cope (fang cof p.cay [our %main %da now])
2149 %+ cope (fang cof p.cay [our %main %da now])
2235 |= [cof=cafe pro=vase]
2150 |= [cof=cafe pro=vase]
@@ -2247,12 +2162,11 @@ We pull in the relevant mark's definition.
2247 (flaw cof leaf/"bad ++sted:grad" ~)
2162 (flaw cof leaf/"bad ++sted:grad" ~)
2248 (make cof %diff [%cast u.for kas] [%cast u.for kos])
2163 (make cof %diff [%cast u.for kas] [%cast u.for kos])
2249
2164
2250−If there's no `++grad`, we complain. If there's no `++form:grad`, then
2165+If there's no `++grad`, we complain. If there's no `++form:grad`, then we look
2251−we look for a `++sted:grad`. If we can't find either, or if
2166+for a `++sted:grad`. If we can't find either, or if `++sted:grad` isn't a term,
2252−`++sted:grad` isn't a term, then we complain. If `++sted:grad` exists
2167+then we complain. If `++sted:grad` exists and is a term, then it represents the
2253−and is a term, then it represents the mark we should use as a proxy to
2168+mark we should use as a proxy to get our diff. So, we cast both our given cages
2254−get our diff. So, we cast both our given cages to the new mark and start
2169+to the new mark and start the dance again.
2255−the dance again.
2256
2170
2257 ?. (slab %diff p.gar)
2171 ?. (slab %diff p.gar)
2258 (flaw cof leaf/"no ++diff:grad" ~)
2172 (flaw cof leaf/"no ++diff:grad" ~)
@@ -2262,8 +2176,7 @@ Otherwise, we expect a `++diff:grad`.
2262 %+ cope (keel cof pro [[%& 6]~ q.cay]~)
2176 %+ cope (keel cof pro [[%& 6]~ q.cay]~)
2263 |= [cof=cafe pox=vase]
2177 |= [cof=cafe pox=vase]
2264
2178
2265−We put the first cage's data into the sample of the given mark's
2179+We put the first cage's data into the sample of the given mark's definition.
2266−definition.
2267
2180
2268 %+ cope
2181 %+ cope
2269 %^ maul cof
2182 %^ maul cof
@@ -2279,46 +2192,41 @@ We run `++diff:grad` with a sample of the second cage's data.
2279 (fine cof u.for dif)
2192 (fine cof u.for dif)
2280 ==
2193 ==
2281
2194
2282−We check that `++form:grad` exists, and we tag the result with it to
2195+We check that `++form:grad` exists, and we tag the result with it to give the
2283−give the final cage.
2196+final cage.
2284
2197
2285−Lifecycle of a `%done`
2198+## Lifecycle of a `%done`
2286−----------------------
2287
2199
2288 %done [cof %0 p.kas q.kas]
2200 %done [cof %0 p.kas q.kas]
2289
2201
2290This is trivial. We simply produce the given cage with the given set of
2202This is trivial. We simply produce the given cage with the given set of
2291−dependencies. This is used when we already have a cage that we want to
2203+dependencies. This is used when we already have a cage that we want to insert
2292−insert into another silk that requires a silk argument. It's analogous
2204+into another silk that requires a silk argument. It's analogous to the return
2293−to the return operator in a monad -- which makes it sound way more
2205+operator in a monad -- which makes it sound way more complicated than it is.
2294−complicated than it is.
2295
2206
2296−Lifecycle of a `%dude`
2207+## Lifecycle of a `%dude`
2297−----------------------
2298
2208
2299 %dude (cool |.(p.kas) $(kas q.kas))
2209 %dude (cool |.(p.kas) $(kas q.kas))
2300
2210
2301−This simply puts a given tank on the stack trace if the given silk
2211+This simply puts a given tank on the stack trace if the given silk produces an
2302−produces an error. This is implemented as a simple call to `++cool`.
2212+error. This is implemented as a simple call to `++cool`.
2303
2213
2304−Lifecycle of a `%dune`
2214+## Lifecycle of a `%dune`
2305−----------------------
2306
2215
2307 %dune
2216 %dune
2308 ?~ q.kas [cof [%2 [%leaf "no data"]~]]
2217 ?~ q.kas [cof [%2 [%leaf "no data"]~]]
2309 $(kas [%done p.kas u.q.kas])
2218 $(kas [%done p.kas u.q.kas])
2310
2219
2311−This is a sort of a `++need` for silks. If there is no data in the unit
2220+This is a sort of a `++need` for silks. If there is no data in the unit cage, we
2312−cage, we produce an error. Else, we simply produce the data in the cage.
2221+produce an error. Else, we simply produce the data in the cage.
2313
2222
2314−Lifcycle of a `%mute`
2223+## Lifcycle of a `%mute`
2315−---------------------
2316
2224
2317 %mute (kale cof p.kas q.kas)
2225 %mute (kale cof p.kas q.kas)
2318
2226
2319−This mutates a silk by putting the values of other silks at particular
2227+This mutates a silk by putting the values of other silks at particular axes.
2320−axes. This is useful in, for example, replacing the sample of the door
2228+This is useful in, for example, replacing the sample of the door in a mark
2321−in a mark definition.
2229+definition.
2322
2230
2323 ++ kale :: mutate
2231 ++ kale :: mutate
2324 |= [cof=cafe kas=silk muy=(list (pair wing silk))]
2232 |= [cof=cafe kas=silk muy=(list (pair wing silk))]
@@ -2338,10 +2246,10 @@ in a mark definition.
2338 |= [cof=cafe vax=vase]
2246 |= [cof=cafe vax=vase]
2339 (fine cof p.cay vax)
2247 (fine cof p.cay vax)
2340
2248
2341−First, we process each of the silks by calling `++make` on them. We pass
2249+First, we process each of the silks by calling `++make` on them. We pass the
2342−the resultant vase and list of pairs of wings and silks to `++keel` to
2250+resultant vase and list of pairs of wings and silks to `++keel` to do the actual
2343−do the actual mutation. We assume the mutation doesn't change the mark
2251+mutation. We assume the mutation doesn't change the mark of the main silk, so we
2344−of the main silk, so we mark the produced vase with the original mark.
2252+mark the produced vase with the original mark.
2345
2253
2346 ++ keel :: apply mutations
2254 ++ keel :: apply mutations
2347 |= [cof=cafe suh=vase yom=(list (pair wing vase))]
2255 |= [cof=cafe suh=vase yom=(list (pair wing vase))]
@@ -2359,19 +2267,16 @@ of the main silk, so we mark the produced vase with the original mark.
2359 :- [p.i.yom [%$ (peg axe 2)]]
2267 :- [p.i.yom [%$ (peg axe 2)]]
2360 $(yom t.yom, axe (peg axe 3))
2268 $(yom t.yom, axe (peg axe 3))
2361
2269
2362−We first put the vases together in one big tuple starting with the
2270+We first put the vases together in one big tuple starting with the subject and
2363−subject and going through the mutations. We slap against this tuple a
2271+going through the mutations. We slap against this tuple a `%_` twig we directly
2364−`%_` twig we directly construct. Since a `%_` twig takes a list of pairs
2272+construct. Since a `%_` twig takes a list of pairs of wings and twigs, we simply
2365−of wings and twigs, we simply have to generate twigs referring to the
2273+have to generate twigs referring to the correct axes in the subject. This is
2366−correct axes in the subject. This is very easy since we just recur on
2274+very easy since we just recur on axis 3 of whatever axis we were already at.
2367−axis 3 of whatever axis we were already at.
2368
2275
2369−Note the use of `%_` instead of `%=` enforces that our mutations don't
2276+Note the use of `%_` instead of `%=` enforces that our mutations don't change
2370−change the type of the subject, which justifies our use of the original
2277+the type of the subject, which justifies our use of the original mark.
2371−mark.
2372
2278
2373−Lifecycle of a `%pact`
2279+## Lifecycle of a `%pact`
2374−----------------------
2375
2280
2376 %pact
2281 %pact
2377 %+ cool |.(leaf/"ford: pact {<`@p`(mug p.kas)>} {<`@p`(mug q.kas)>}")
2282 %+ cool |.(leaf/"ford: pact {<`@p`(mug p.kas)>} {<`@p`(mug q.kas)>}")
@@ -2408,12 +2313,12 @@ We pull in the relevant mark's definition.
2408 (flaw cof leaf/"bad ++sted:grad" ~)
2313 (flaw cof leaf/"bad ++sted:grad" ~)
2409 (make cof %cast p.cay %pact [%cast u.for kas] kos)
2314 (make cof %cast p.cay %pact [%cast u.for kas] kos)
2410
2315
2411−If there's no `++grad`, we complain. If there's no `++form:grad`, then
2316+If there's no `++grad`, we complain. If there's no `++form:grad`, then we look
2412−we look for a `++sted:grad`. If we can't find either, or if
2317+for a `++sted:grad`. If we can't find either, or if `++sted:grad` isn't a term,
2413−`++sted:grad` isn't a term, then we complain. If `++sted:grad` exists
2318+then we complain. If `++sted:grad` exists and is a term, then it represents the
2414−and is a term, then it represents the mark we should use as a proxy to
2319+mark we should use as a proxy to get our diff. So, we cast the first argument to
2415−get our diff. So, we cast the first argument to the new mark, then try
2320+the new mark, then try to patch. Afterward, we cast the result back to the
2416−to patch. Afterward, we cast the result back to the original mark.
2321+original mark.
2417
2322
2418 =+ for=((soft ,@tas) q:(slap gar [%cnzy %form]))
2323 =+ for=((soft ,@tas) q:(slap gar [%cnzy %form]))
2419 ?~ for
2324 ?~ for
@@ -2423,8 +2328,8 @@ to patch. Afterward, we cast the result back to the original mark.
2423 =< leaf/"pact on data with wrong form: {-} {+<} {+>}"
2328 =< leaf/"pact on data with wrong form: {-} {+<} {+>}"
2424 [(trip p.cay) (trip u.for) (trip p.coy)]
2329 [(trip p.cay) (trip u.for) (trip p.coy)]
2425
2330
2426−If `++form:grad` isn't a term, or else our second argument isn't of that
2331+If `++form:grad` isn't a term, or else our second argument isn't of that mark,
2427−mark, we complain.
2332+we complain.
2428
2333
2429 ?. (slab %pact p.gar)
2334 ?. (slab %pact p.gar)
2430 (flaw cof leaf/"no ++pact:grad" ~)
2335 (flaw cof leaf/"no ++pact:grad" ~)
@@ -2434,8 +2339,7 @@ If we don't have a `++pact:grad`, we complain.
2434 %+ cope (keel cof pro [[%& 6]~ q.cay]~)
2339 %+ cope (keel cof pro [[%& 6]~ q.cay]~)
2435 |= [cof=cafe pox=vase]
2340 |= [cof=cafe pox=vase]
2436
2341
2437−We put the first cage's data into the sample of the given mark's
2342+We put the first cage's data into the sample of the given mark's definition.
2438−definition.
2439
2343
2440 %+ cope
2344 %+ cope
2441 %^ maul cof
2345 %^ maul cof
@@ -2443,36 +2347,32 @@ definition.
2443 q.coy
2347 q.coy
2444 |= [cof=cafe pat=vase]
2348 |= [cof=cafe pat=vase]
2445
2349
2446−We run `++pact:grad` with a sample of the second cage's data, which is
2350+We run `++pact:grad` with a sample of the second cage's data, which is the diff.
2447−the diff.
2448
2351
2449 (fine cof p.cay pat)
2352 (fine cof p.cay pat)
2450 ==
2353 ==
2451
2354
2452We tag the result with the mark of our first argument.
2355We tag the result with the mark of our first argument.
2453
2356
2454−Lifecycle of a `%plan`
2357+## Lifecycle of a `%plan`
2455−----------------------
2456
2358
2457 %plan
2359 %plan
2458 %+ cope (abut:(meow p.kas q.kas) cof r.kas)
2360 %+ cope (abut:(meow p.kas q.kas) cof r.kas)
2459 |= [cof=cafe vax=vase]
2361 |= [cof=cafe vax=vase]
2460 (fine cof %noun vax)
2362 (fine cof %noun vax)
2461
2363
2462−This is a direct request to compile a hood at a given beam with a heel
2364+This is a direct request to compile a hood at a given beam with a heel of the
2463−of the given path. We comply by calling `++abut` with the given
2365+given path. We comply by calling `++abut` with the given arguments and producing
2464−arguments and producing the vase with a mark of `%noun`.
2366+the vase with a mark of `%noun`.
2465
2367
2466−Lifecycle of a `%reef`
2368+## Lifecycle of a `%reef`
2467−----------------------
2468
2369
2469 %reef (fine cof %noun pit)
2370 %reef (fine cof %noun pit)
2470
2371
2471−This is one of the simplest silks. We simply produce our context, which
2372+This is one of the simplest silks. We simply produce our context, which is zuse
2472−is zuse compiled against hoon. The mark is a `%noun`.
2373+compiled against hoon. The mark is a `%noun`.
2473
2374
2474−Lifcycle of a `%ride`
2375+## Lifcycle of a `%ride`
2475−---------------------
2476
2376
2477 %ride
2377 %ride
2478 %+ cool |.(leaf/"ford: ride {<`@p`(mug kas)>}")
2378 %+ cool |.(leaf/"ford: ride {<`@p`(mug kas)>}")
@@ -2482,12 +2382,11 @@ Lifcycle of a `%ride`
2482 |= [cof=cafe vax=vase]
2382 |= [cof=cafe vax=vase]
2483 (fine cof %noun vax)
2383 (fine cof %noun vax)
2484
2384
2485−This slaps evaluates the given silk, then it slaps the result against
2385+This slaps evaluates the given silk, then it slaps the result against the given
2486−the given twig. Since we don't know what of what mark (if any) is the
2386+twig. Since we don't know what of what mark (if any) is the result, we give it a
2487−result, we give it a mark of `%noun`.
2387+mark of `%noun`.
2488
2388
2489−Lifecycle of a `%vale`
2389+## Lifecycle of a `%vale`
2490−----------------------
2491
2390
2492 %vale
2391 %vale
2493 %+ cool |.(leaf/"ford: vale {<p.kas>} {<q.kas>} {<`@p`(mug r.kas)>}")
2392 %+ cool |.(leaf/"ford: vale {<p.kas>} {<q.kas>} {<`@p`(mug r.kas)>}")
@@ -2495,20 +2394,20 @@ Lifecycle of a `%vale`
2495 |= [cof=cafe vax=vase]
2394 |= [cof=cafe vax=vase]
2496 (fine cof `cage`[p.kas vax])
2395 (fine cof `cage`[p.kas vax])
2497
2396
2498−This checks whether given data is of the given mark. If we don't have
2397+This checks whether given data is of the given mark. If we don't have the
2499−the definition of the mark, we check the given ship for it.
2398+definition of the mark, we check the given ship for it.
2500
2399
2501−We call `++lave` to perform the check, producing a vase. We produce this
2400+We call `++lave` to perform the check, producing a vase. We produce this vase
2502−vase tagged with the given mark.
2401+tagged with the given mark.
2503
2402
2504 ++ lave :: validate
2403 ++ lave :: validate
2505 |= [cof=cafe for=mark his=ship som=*]
2404 |= [cof=cafe for=mark his=ship som=*]
2506 ^- (bolt vase)
2405 ^- (bolt vase)
2507 ((lake for [our %main [%da now]]) cof [%noun som])
2406 ((lake for [our %main [%da now]]) cof [%noun som])
2508
2407
2509−This is a thinly-veiled wrapper over `++lake`. Note that, contrary to
2408+This is a thinly-veiled wrapper over `++lake`. Note that, contrary to documented
2510−documented opinion, we do not in fact check the other ship's definition
2409+opinion, we do not in fact check the other ship's definition of a mark. This is
2511−of a mark. This is likely a bug.
2410+likely a bug.
2512
2411
2513At any rate, `++lake` coerces a noun into the correct type for a mark.
2412At any rate, `++lake` coerces a noun into the correct type for a mark.
2514
2413
@@ -2536,27 +2435,24 @@ At any rate, `++lake` coerces a noun into the correct type for a mark.
2536 (fine cof (slot 6 pro))
2435 (fine cof (slot 6 pro))
2537 (flaw cof [%leaf "ford: invalid content: {<[for bek]>}"]~)
2436 (flaw cof [%leaf "ford: invalid content: {<[for bek]>}"]~)
2538
2437
2539−This is going to coerce the sample into the correct type for the mark.
2438+This is going to coerce the sample into the correct type for the mark. First, we
2540−First, we push a line onto the stack trace saying that we're checking
2439+push a line onto the stack trace saying that we're checking the type. If the
2541−the type. If the requested mark is a gate, core, door, hoon, or hook,
2440+requested mark is a gate, core, door, hoon, or hook, then we don't do any more
2542−then we don't do any more type information than just saying it's a noun,
2441+type information than just saying it's a noun, so we're done.
2543−so we're done.
2544
2442
2545−Otherwise, we get the mark definition from our `/=main=/mar` directory
2443+Otherwise, we get the mark definition from our `/=main=/mar` directory with
2546−with `++fang`, which we'll describe below.
2444+`++fang`, which we'll describe below.
2547
2445
2548−We check to see if our sample type nests within the type of the sample
2446+We check to see if our sample type nests within the type of the sample to the
2549−to the door. If so, then we're already of the correct type, so we're
2447+door. If so, then we're already of the correct type, so we're done.
2550−done.
2551
2448
2552−Otherwise, we check to see if there's a `++grab` in the door, and a
2449+Otherwise, we check to see if there's a `++grab` in the door, and a `++noun` in
2553−`++noun` in the `++grab`. If not, there's no way we can translate to
2450+the `++grab`. If not, there's no way we can translate to this mark, so we fail.
2554−this mark, so we fail.
2555
2451
2556−If we have everything we need, we slam our sample (typed as a noun)
2452+If we have everything we need, we slam our sample (typed as a noun) against the
2557−against the `++noun` in `++grab`. If the sample of the door is the same
2453+`++noun` in `++grab`. If the sample of the door is the same as our sample, then
2558−as our sample, then the check succeeded, so we produce the well-typed
2454+the check succeeded, so we produce the well-typed sample of the door. Otherwise,
2559−sample of the door. Otherwise, we fail.
2455+we fail.
2560
2456
2561 ++ fang :: protocol door
2457 ++ fang :: protocol door
2562 |= [cof=cafe for=mark bek=beak]
2458 |= [cof=cafe for=mark bek=beak]
@@ -2571,6 +2467,6 @@ sample of the door. Otherwise, we fail.
2571 (cope (fade cof %hook bem) abut:(meow bem ~))
2467 (cope (fade cof %hook bem) abut:(meow bem ~))
2572
2468
2573A mark's definition is generally in
2469A mark's definition is generally in
2574−`/=main=/mar/[mark-name]/door/hook'. If we don't find it there, we look in`/[given-beak]/mar/[mark-name]/door/hook'.
2470+`/=main=/mar/[mark-name]/door/hook'. If we don't find it there, we look in`/\[given-beak\]/mar/\[mark-name\]/door/hook'.
2575−We parse the mark definition with `++fade` and assemble it with
2471+We parse the mark definition with `++fade` and assemble it with `++abut:meow`.
2576−`++abut:meow`. `++fade` is defined under the `%bake` silk.
2472+`++fade` is defined under the `%bake` silk.
docs/arvo/internals/ford/runes.md+228−225
@@ -2,22 +2,33 @@
2
2
3## ford runes
3## ford runes
4
4
5−One of the most common ways to use ford is through ford's runes, all of which begin with `/` ("fas"). A ford rune runs a step of a ford build and places the result in the subject. There are various kinds of build steps, some of which take other build steps as parameters. The most common pattern is to have a sequence of ford runes at the top of a hoon source file that import the results of evaluating other hoon files. This is how we "import a library" in urbit: we add the result of compiling another hoon file into the current subject, possibly renaming it by wrapping a face around it.
5+One of the most common ways to use ford is through ford's runes, all of which
6−
6+begin with `/` ("fas"). A ford rune runs a step of a ford build and places the
7−Another common use case is assembling and rendering markdown templates into a single HTML file, to be sent as an HTTP response in urbit's web server `%eyre`. It's important to keep in mind that %ford is quite flexible and generic, and is used in other parts of urbit -- such as the dojo -- that would not traditionally fall under the purview of a build system.
7+result in the subject. There are various kinds of build steps, some of which
8+take other build steps as parameters. The most common pattern is to have a
9+sequence of ford runes at the top of a hoon source file that import the results
10+of evaluating other hoon files. This is how we "import a library" in urbit: we
11+add the result of compiling another hoon file into the current subject, possibly
12+renaming it by wrapping a face around it.
13+
14+Another common use case is assembling and rendering markdown templates into a
15+single HTML file, to be sent as an HTTP response in urbit's web server `%eyre`.
16+It's important to keep in mind that %ford is quite flexible and generic, and is
17+used in other parts of urbit -- such as the dojo -- that would not traditionally
18+fall under the purview of a build system.
8
19
9### `/+` import from lib/
20### `/+` import from lib/
10
21
11−The `/+` rune accepts a filename as an argument. It interprets that filename
22+The `/+` rune accepts a filename as an argument. It interprets that filename as
12−as a hoon source file within the `lib` directory. This is how we import a shared
23+a hoon source file within the `lib` directory. This is how we import a shared
13library in urbit.
24library in urbit.
14
25
15Example:
26Example:
16−```
27+
17−/+ time-to-id
28+ /+ time-to-id
18−::
29+ ::
19−(time-to-id now)
30+ (time-to-id now)
20−```
31+
21produces: `"c.314d"` (or something similar depending on when you run it)
32produces: `"c.314d"` (or something similar depending on when you run it)
22
33
23### `/-` import from sur/
34### `/-` import from sur/
@@ -28,11 +39,11 @@ shared structures that can be used by other parts of urbit. This is somewhat
28similar to including a header file in C.
39similar to including a header file in C.
29
40
30Example:
41Example:
31−```
42+
32−/- talk
43+ /- talk
33−::
44+ ::
34−*serial:talk
45+ *serial:talk
35−```
46+
36produces: `0v0`
47produces: `0v0`
37
48
38### `/~` twig by hand
49### `/~` twig by hand
@@ -43,53 +54,52 @@ the subject. Arbitrary hoon can be in the twig. A
43structure representing a `%ford` static resource.
54structure representing a `%ford` static resource.
44
55
45Example:
56Example:
46−```
47−/~ [a=0 b=1]
48−[a b]
49−```
50−produces:
51−`[0 1]`
52
57
53−In wide-form, `/~` always takes a tuple (which may be a degenerate tuple of one element), and produces it.
58+ /~ [a=0 b=1]
59+ [a b]
60+
61+produces: `[0 1]`
62+
63+In wide-form, `/~` always takes a tuple (which may be a degenerate tuple of one
64+element), and produces it.
54
65
55Example:
66Example:
56−```
67+
57−/~[%something]
68+ /~[%something]
58−```
69+
59produces: `%something`
70produces: `%something`
60
71
61−```
72+ /~[%a %b]
62−/~[%a %b]
73+
63−```
64produces: `[%a %b]`
74produces: `[%a %b]`
65
75
66### `//` include a file by relative path
76### `//` include a file by relative path
67
77
68`// <rel-path>` parses `rel-path` as a hoon twig, and then adds the resulting
78`// <rel-path>` parses `rel-path` as a hoon twig, and then adds the resulting
69twig to the subject. Note that the result type of this rune is not a horn, but
79twig to the subject. Note that the result type of this rune is not a horn, but
70−just a hoon twig, so its result can't be used as an argument to other runes
80+just a hoon twig, so its result can't be used as an argument to other runes that
71−that expect a horn. There is no wide-form for this rune.
81+expect a horn. There is no wide-form for this rune.
72
82
73Example:
83Example:
74−```
84+
75−// %/file-to-include
85+ // %/file-to-include
76−::
86+ ::
77−(frobnicate:file-to-include %.n)
87+ (frobnicate:file-to-include %.n)
78−
88+
79−:: contents of file-to-include.hoon
89+ :: contents of file-to-include.hoon
80−|%
90+ |%
81−++ frobnicate
91+ ++ frobnicate
82− |= a/? ^+ a
92+ |= a/? ^+ a
83− !a
93+ !a
84−--
94+ --
85−```
95+
86produces: `%.y`
96produces: `%.y`
87
97
88−_Note: `%` means current directory, which in a hoon file will resolve to the
98+*Note: `%` means current directory, which in a hoon file will resolve to the
89"directory" containing that file. Also, %clay doesn't have first-class
99"directory" containing that file. Also, %clay doesn't have first-class
90directories the way unix filesystems do. Whereas in unix, a directory is a
100directories the way unix filesystems do. Whereas in unix, a directory is a
91special kind of file, in %clay it's just a path prefix, and there is no file
101special kind of file, in %clay it's just a path prefix, and there is no file
92−stored at that path._
102+stored at that path.*
93
103
94### `/=` wrap a face around an included horn
104### `/=` wrap a face around an included horn
95
105
@@ -97,44 +107,42 @@ stored at that path._
97that horn, and wraps a face around it.
107that horn, and wraps a face around it.
98
108
99Example:
109Example:
100−```
110+
101−/= foo /~ [a=0 b=1]
111+ /= foo /~ [a=0 b=1]
102−::
112+ ::
103−[a.foo b.foo]
113+ [a.foo b.foo]
104−```
114+
105produces: `[0 1]`
115produces: `[0 1]`
106
116
107In wide-form, `/=` uses `=` as a delimiter: `/=foo=/~[a=0 b=1]`
117In wide-form, `/=` uses `=` as a delimiter: `/=foo=/~[a=0 b=1]`
108
118
109### `/:` evaluate at path, and `/<mark>/` render mark at current path
119### `/:` evaluate at path, and `/<mark>/` render mark at current path
110
120
111−`/:` takes a path and a horn, and evaluates the horn with the current path set to the supplied path.
121+`/:` takes a path and a horn, and evaluates the horn with the current path set
112−`/mar/` renders the mark 'mar' at the current path.
122+to the supplied path. `/mar/` renders the mark 'mar' at the current path.
113
123
114Example:
124Example:
115−```
125+
116−/= hoo-source /: /path/to/hoon-file /hoon/
126+ /= hoo-source /: /path/to/hoon-file /hoon/
117−::
127+ ::
118−`@t`hoo-source
128+ `@t`hoo-source
119−```
120
129
121produces the text of the hoon file at "/path/to/hoon-file/hoon".
130produces the text of the hoon file at "/path/to/hoon-file/hoon".
122
131
123`/hoon/` renders the current path using the `%hoon` mark, which in this case
132`/hoon/` renders the current path using the `%hoon` mark, which in this case
124−passes the contents through unchanged. In general, rendering a file with a
133+passes the contents through unchanged. In general, rendering a file with a mark
125−mark will potentially run the contents through a series of conversion
134+will potentially run the contents through a series of conversion operations. For
126−operations. For details on marks, see the
135+details on marks, see the [marks docs](https://urbit.org/docs/arvo/marks).
127−[marks docs](https://urbit.org/docs/arvo/marks).
128
136
129Here's an example that includes a mark conversion:
137Here's an example that includes a mark conversion:
130−```
138+
131−/= page /: /path/to/html/file /mime/
139+ /= page /: /path/to/html/file /mime/
132−::
140+ ::
133−page
141+ page
134−
142+
135−:: contents of /path/to/html/file:
143+ :: contents of /path/to/html/file:
136−<html><head><title>My Fascinating Blog</title></head></html>
144+ <html><head><title>My Fascinating Blog</title></head></html>
137−```
145+
138produces:
146produces:
139`[[%text %html ~] 60 '<html><head><title>My Fascinating Blog</title></head></html>']`
147`[[%text %html ~] 60 '<html><head><title>My Fascinating Blog</title></head></html>']`
140
148
@@ -146,51 +154,49 @@ In wide-form, `/:` uses `:` as a delimiter: `/=page=/:/path/to/html/file:/mime/`
146### `/!mark/` evaluate as hoon, then pass through mark
154### `/!mark/` evaluate as hoon, then pass through mark
147
155
148Example:
156Example:
149−```
157+
150−/= mime /: /%/some-hoon-file /!mime/
158+ /= mime /: /%/some-hoon-file /!mime/
151−::
159+ ::
152−mime
160+ mime
153−
161+
154−:: contents of /%/some-hoon-file:
162+ :: contents of /%/some-hoon-file:
155−%produces-a-cord
163+ %produces-a-cord
156−```
164+
157produces: `[[%text %plain ~] 15 'produces-a-cord']`
165produces: `[[%text %plain ~] 15 'produces-a-cord']`
158
166
159### `/&` pass through a series of marks
167### `/&` pass through a series of marks
160
168
161`/&` passes a horn through multiple marks, right-to-left. It has both a
169`/&` passes a horn through multiple marks, right-to-left. It has both a
162wide-form and a tall-form syntax. In wide-form, it takes a series of mark
170wide-form and a tall-form syntax. In wide-form, it takes a series of mark
163−arguments followed by a horn. In tall-form, it takes a single mark followed by
171+arguments followed by a horn. In tall-form, it takes a single mark followed by a
164−a horn.
172+horn.
165−
173+
166−```
174+ /= some-text /: /%/text-file /&mime&/txt/
167−/= some-text /: /%/text-file /&mime&/txt/
175+ ::
168−::
176+ some-text
169−some-text
177+
170−```
171produces: `[[%text %plain ~] p=17 q='Hi I\'m some text\0a']`
178produces: `[[%text %plain ~] p=17 q='Hi I\'m some text\0a']`
172
179
173−This example shows two of the ways marks are used. The first way is what
180+This example shows two of the ways marks are used. The first way is what happens
174−happens with the `/txt/` mark: we use it to find a file in clay with that
181+with the `/txt/` mark: we use it to find a file in clay with that extension,
175−extension, without performing any conversions. Since this file is stored with
182+without performing any conversions. Since this file is stored with the `%txt`
176−the `%txt` mark in `%clay`, its type is a `wain`: a list of cords, where each
183+mark in `%clay`, its type is a `wain`: a list of cords, where each cord is a
177−cord is a single line. Once we've read the file, the `%mime` mark converts the
184+single line. Once we've read the file, the `%mime` mark converts the `wain` to a
178−`wain` to a triple that includes the MIME type ("text/plain"), the content
185+triple that includes the MIME type ("text/plain"), the content length in bytes,
179−length in bytes, and the content itself as a cord.
186+and the content itself as a cord.
180−
187+
181−```
188+ /= page /&html&elem&/~[;div.foo;]
182−/= page /&html&elem&/~[;div.foo;]
189+ ::
183−::
190+ page
184−page
191+
185−```
186produces: `'<html><head></head><body><div class="foo"></div></body></html>'`
192produces: `'<html><head></head><body><div class="foo"></div></body></html>'`
187
193
188−This runs the sail expression `;div.foo;` through the `%elem` mark, then
194+This runs the sail expression `;div.foo;` through the `%elem` mark, then through
189−through the `%html` mark. The `/~` rune produces an item of mark `noun`. The
195+the `%html` mark. The `/~` rune produces an item of mark `noun`. The `%elem`
190−`%elem` mark converts the mark of the expression from `noun` to `elem` by
196+mark converts the mark of the expression from `noun` to `elem` by checking that
191−checking that the type fits in `manx` (a hoon/sail type indicating an XML
197+the type fits in `manx` (a hoon/sail type indicating an XML element). The
192−element). The `%html` mark recognizes the `%elem` and converts it to an HTML
198+`%html` mark recognizes the `%elem` and converts it to an HTML string with
193−string with enclosing `<html>`, `<head>`, and `<body>` tags.
199+enclosing `<html>`, `<head>`, and `<body>` tags.
194
200
195It's possible to use wide-form `/&` with more than two marks, by using `&` as a
201It's possible to use wide-form `/&` with more than two marks, by using `&` as a
196delimiter between marks and adding a `/` before the last mark, like:
202delimiter between marks and adding a `/` before the last mark, like:
@@ -206,41 +212,40 @@ need to be enclosed in `/`'s: `/& html /elem/`
206succeeds. If none succeed, it produces stack traces from all of its arguments.
212succeeds. If none succeed, it produces stack traces from all of its arguments.
207
213
208Example:
214Example:
209−```
215+
210−/= res /: /%/path/to/file
216+ /= res /: /%/path/to/file
211−/|
217+ /|
212− /!elem/
218+ /!elem/
213− /elem/
219+ /elem/
214−==
220+ ==
215−::
221+ ::
216−res
222+ res
217−```
223+
218tries to parse the file at `/%/path/to/file` as a hoon expression and evaluate
224tries to parse the file at `/%/path/to/file` as a hoon expression and evaluate
219it (`/!elem/`), and then if that fails, tries to convert the file to an HTML
225it (`/!elem/`), and then if that fails, tries to convert the file to an HTML
220−element directly (`/elem/`). This is used in urbit's `tree` publishing system
226+element directly (`/elem/`). This is used in urbit's `tree` publishing system to
221−to enable a user to place either a hoon file or a static file in a directory,
227+enable a user to place either a hoon file or a static file in a directory, and
222−and have them both result in webpages, with preference given to treating the
228+have them both result in webpages, with preference given to treating the file as
223−file as hoon to be evaluated.
229+hoon to be evaluated.
224
230
225−Wide-form `/|` encloses its arguments in parentheses, with a single space as a delimiter:
231+Wide-form `/|` encloses its arguments in parentheses, with a single space as a
226−`/|(/!elem/ /elem/)`
232+delimiter: `/|(/!elem/ /elem/)`
227
233
228### `/_` unfiltered: run a horn on each file in the current directory
234### `/_` unfiltered: run a horn on each file in the current directory
229
235
230`/_` can be used in two ways: filtered and unfiltered.
236`/_` can be used in two ways: filtered and unfiltered.
231
237
232−Unfiltered `/_` takes a horn as an argument. It produces a new horn representing the
238+Unfiltered `/_` takes a horn as an argument. It produces a new horn representing
233−result of mapping the supplied horn over the list of files in the current
239+the result of mapping the supplied horn over the list of files in the current
234directory. The keys in the resulting map are the basenames of the files in the
240directory. The keys in the resulting map are the basenames of the files in the
235directory, and each value is the result of running that horn on the contents of
241directory, and each value is the result of running that horn on the contents of
236the file.
242the file.
237
243
238Example:
244Example:
239−```
245+
240−/= kids /_ /hoon/
246+ /= kids /_ /hoon/
241−::
247+ ::
242−`(map term cord)`kids
248+ `(map term cord)`kids
243−```
244
249
245produces a value of type `(map term cord)`, where each key is the basename (the
250produces a value of type `(map term cord)`, where each key is the basename (the
246filename without the prefix), and each value is the result of running the
251filename without the prefix), and each value is the result of running the
@@ -255,14 +260,14 @@ Wide-form unfiltered `/_` doesn't need a delimiter: `/= kids /_/hoon/`
255Filtered `/_` takes an aura and a horn, and filters the list of files in the
260Filtered `/_` takes an aura and a horn, and filters the list of files in the
256current directory by whether their filenames can be parsed to an atom of that
261current directory by whether their filenames can be parsed to an atom of that
257aura. It then produces a map where each key is the filename after parsing into
262aura. It then produces a map where each key is the filename after parsing into
258−that atom type, and each value is the result of running the horn on the
263+that atom type, and each value is the result of running the horn on the contents
259−contents of that file.
264+of that file.
260
265
261Example:
266Example:
262−```
267+
263−/= timed-posts /_ @da /md/
268+ /= timed-posts /_ @da /md/
264−`(map @da @t)`timed-posts
269+ `(map @da @t)`timed-posts
265−```
270+
266produces a map from dates to cords. This product will only contain files whose
271produces a map from dates to cords. This product will only contain files whose
267names are parsable into `@da` dates. The values are the file contents converted
272names are parsable into `@da` dates. The values are the file contents converted
268to a cord of markdown-formatted text.
273to a cord of markdown-formatted text.
@@ -271,89 +276,86 @@ Wide-form filtered `/_` uses `_` as a delimiter: `/= timed-posts /_@da_/md/`
271
276
272### `/;` operate on
277### `/;` operate on
273
278
274−`/;` takes a twig and a horn. The twig should evaluate to a gate, which is then slammed
279+`/;` takes a twig and a horn. The twig should evaluate to a gate, which is then
275−with the result of the horn as its sample.
280+slammed with the result of the horn as its sample.
276
281
277Example:
282Example:
278−```
283+
279−/= goo
284+ /= goo
280− /; |=({a/@ b/@} +(b))
285+ /; |=({a/@ b/@} +(b))
281− /~ [a=0 b=1]
286+ /~ [a=0 b=1]
282−::
287+ ::
283−goo
288+ goo
284−```
285
289
286produces: `2`
290produces: `2`
287
291
288Here's a slightly more complex example with runes that use the filesystem:
292Here's a slightly more complex example with runes that use the filesystem:
289−```
293+
290−/= file-length
294+ /= file-length
291− /; |=(a/@t (lent (crip a)))
295+ /; |=(a/@t (lent (crip a)))
292− /: /%/path/to/hoon/file /hoon/
296+ /: /%/path/to/hoon/file /hoon/
293−::
297+ ::
294−file-length
298+ file-length
295−```
299+
296produces the number of bytes in the file "/%/path/to/hoon/file."
300produces the number of bytes in the file "/%/path/to/hoon/file."
297
301
298### `/,` switch by path
302### `/,` switch by path
299
303
300−`/,` is a switch statement, which picks a branch to evaluate based on
304+`/,` is a switch statement, which picks a branch to evaluate based on whether
301−whether the current path matches the path in the switch statement.
305+the current path matches the path in the switch statement. Takes a sequence of
302−Takes a sequence of pairs of (path, horn) terminated by a `==`. No wide-form.
306+pairs of (path, horn) terminated by a `==`. No wide-form.
303
307
304Example:
308Example:
305−```
309+
306−/= just-right
310+ /= just-right
307− /: /===/right-path :: set path to /%/right-path
311+ /: /===/right-path :: set path to /%/right-path
308− /,
312+ /,
309− /wrong-path /~ ~
313+ /wrong-path /~ ~
310− /another-wrong-path /~ ~
314+ /another-wrong-path /~ ~
311− /right-path /~ %evaluate-me :: only evaluate this horn
315+ /right-path /~ %evaluate-me :: only evaluate this horn
312− ==
316+ ==
313−::
317+ ::
314−`@t`just-right
318+ `@t`just-right
315−```
316
319
317produces: `'evaluate-me'`
320produces: `'evaluate-me'`
318
321
319### `/.` list
322### `/.` list
320
323
321−Produce a null-terminated list from a sequence of horns, terminated by a `==`. No wide-form.
324+Produce a null-terminated list from a sequence of horns, terminated by a `==`.
325+No wide-form.
322
326
323Example:
327Example:
324−```
328+
325−/= vanes
329+ /= vanes
326− /.
330+ /.
327− /~ %ames
331+ /~ %ames
328− /~ %behn
332+ /~ %behn
329− /~ %clay
333+ /~ %clay
330− /~ %dill
334+ /~ %dill
331− /~ %eyre
335+ /~ %eyre
332− /~ %ford
336+ /~ %ford
333− /~ %gall
337+ /~ %gall
334− ==
338+ ==
335−::
339+ ::
336−vanes
340+ vanes
337−```
341+
338produces: `[%ames %behn %clay %dill %eyre %ford %gall ~]`.
342produces: `[%ames %behn %clay %dill %eyre %ford %gall ~]`.
339
343
340### `/*` heterogeneous map
344### `/*` heterogeneous map
341
345
342−_DEPRECATED_
346+*DEPRECATED* TODO: remove
343−TODO: remove
344
347
345### `/^` cast
348### `/^` cast
346
349
347`/^` takes a mold and a horn, and casts the result of the horn to the mold.
350`/^` takes a mold and a horn, and casts the result of the horn to the mold.
348
351
349Example:
352Example:
350−```
353+
351−/= liz /^ (list @) :: cast to real list
354+ /= liz /^ (list @) :: cast to real list
352− /~ ~[1 2 3] :: no 'i' or 't' faces
355+ /~ ~[1 2 3] :: no 'i' or 't' faces
353−::
356+ ::
354−?< ?=($~ liz) :: prevent find-fork
357+ ?< ?=($~ liz) :: prevent find-fork
355−i.liz
358+ i.liz
356−```
357
359
358produces: `1`
360produces: `1`
359
361
@@ -363,60 +365,61 @@ Wide-form `/^` uses `^` as a delimiter: `/^(list @)^/~[1 2 3 ~]`
363
365
364### `/#` insert dephash
366### `/#` insert dephash
365
367
366−`/#` takes a horn and produces a cell of the dependency
368+`/#` takes a horn and produces a cell of the dependency hash of the result of
367−hash of the result of the horn, and the result itself.
369+the horn, and the result itself.
368
370
369Example:
371Example:
370−```
372+
371−/= inline /^ {dep/@uvH txt/@t} /# /: /%/my-script /js/
373+ /= inline /^ {dep/@uvH txt/@t} /# /: /%/my-script /js/
372−::
374+ ::
373−;= ;script: (trip txt.inline) :: set script contents
375+ ;= ;script: (trip txt.inline) :: set script contents
374− ;script@"/~/on/{<dep.inline>}.js"; :: set script src
376+ ;script@"/~/on/{<dep.inline>}.js"; :: set script src
375−==
377+ ==
376−```
378+
377−produces two `<script>` tags. The first has its contents set to the contents
379+produces two `<script>` tags. The first has its contents set to the contents of
378−of the JavaScript file we loaded from clay. The second is an auto-update
380+the JavaScript file we loaded from clay. The second is an auto-update script
379−script that polls the server to check whether the dependency hash has changed.
381+that polls the server to check whether the dependency hash has changed. This
380−This pattern is used in urbit's `tree` web publishing system.
382+pattern is used in urbit's `tree` web publishing system.
381
383
382Wide-form `/#` does not need a delimiter:
384Wide-form `/#` does not need a delimiter:
383`/= inline /^ {dep/@uvH txt/@t} /#/:/%/my-script:/js/`
385`/= inline /^ {dep/@uvH txt/@t} /#/:/%/my-script:/js/`
384
386
385### `/$` process extra arguments
387### `/$` process extra arguments
386
388
387−`/$` will slam a gate on whatever extra arguments have been supplied to this build.
389+`/$` will slam a gate on whatever extra arguments have been supplied to this
388−At the moment, only HTTP requests forwarded by `%eyre` contain any extra arguments,
390+build. At the moment, only HTTP requests forwarded by `%eyre` contain any extra
389−and using this rune outside of that context will cause an error. Requests from `%eyre`
391+arguments, and using this rune outside of that context will cause an error.
390−contain an argument representing the query string, which can be parsed using the
392+Requests from `%eyre` contain an argument representing the query string, which
391−standard library gate `++fuel`.
393+can be parsed using the standard library gate `++fuel`.
392
394
393Example:
395Example:
394−```
396+
395−/= gas /$ fuel
397+ /= gas /$ fuel
396−::
398+ ::
397−:*
399+ :*
398− %who {<(~(get ju aut.ced.gas) 0)>}
400+ %who {<(~(get ju aut.ced.gas) 0)>}
399− %where {(spud s.bem.gas)}
401+ %where {(spud s.bem.gas)}
400− %case {(scow %ud p.r.bem.gas)}
402+ %case {(scow %ud p.r.bem.gas)}
401−==
403+ ==
402−```
404+
403−produces: [%who '~zod' %where /ford/pages/web %case 60]
405+produces: \[%who '~zod' %where /ford/pages/web %case 60\]
404−
406+
405−Wide-form `/$` takes its arguments inside brackets, like `/~`:
407+Wide-form `/$` takes its arguments inside brackets, like `/~`: `/=gas=/$[fuel]`
406−`/=gas=/$[fuel]`
407
408
408### `/%` propagate extra arguments into renderers
409### `/%` propagate extra arguments into renderers
409
410
410−`/%` will forward extra arguments (usually from `%eyre`) on to any enclosed `/renderer/`'s. Without this,
411+`/%` will forward extra arguments (usually from `%eyre`) on to any enclosed
411−renderers that use `/$` to read the extra arguments will crash.
412+`/renderer/`'s. Without this, renderers that use `/$` to read the extra
413+arguments will crash.
412
414
413Example:
415Example:
414−```
416+
415−/= dat /% /tree-json/
417+ /= dat /% /tree-json/
416−::
418+ ::
417−dat
419+ dat
418−```
420+
419−produces the results of running `ren/tree-json` on the current path. This renderer needs the query string,
421+produces the results of running `ren/tree-json` on the current path. This
420−which is one of the extra arguments passed in from `%eyre`.
422+renderer needs the query string, which is one of the extra arguments passed in
423+from `%eyre`.
421
424
422Wide-form `/%` has no delimiter: `/%/tree-json/`
425Wide-form `/%` has no delimiter: `/%/tree-json/`
docs/arvo/internals/gall.md+57−71
@@ -1,116 +1,102 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 7
3sort: 7
4---
4---
5
5
6−
6+# `%gall`
7−`%gall`
8−=======
9
7
10Our application server manager.
8Our application server manager.
11
9
12−It allows applications and vanes to send messages to applications and
10+It allows applications and vanes to send messages to applications and subscribe
13−subscribe to data streams. This requires `%gall` to be a sort of a
11+to data streams. This requires `%gall` to be a sort of a hypervisor. Messages
14−hypervisor. Messages coming into `%gall` are routed to the intended
12+coming into `%gall` are routed to the intended application, and the response
15−application, and the response comes back along the same route. If the
13+comes back along the same route. If the intended target is on another ship,
16−intended target is on another ship, `%gall` will behind-the-scenes route
14+`%gall` will behind-the-scenes route it through ames to the other ship to run.
17−it through ames to the other ship to run. This provides an abstraction
15+This provides an abstraction where all apps on all ships are communicated with
18−where all apps on all ships are communicated with over the same
16+over the same interface.
19−interface.
20
17
21`%gall` neither accepts events from unix nor produces effects. It exists
18`%gall` neither accepts events from unix nor produces effects. It exists
22−entirely for the benefit of other vanes and, in particular,
19+entirely for the benefit of other vanes and, in particular, applications. Eyre
23−applications. Eyre exposes `%gall`'s interface over http, and ames does
20+exposes `%gall`'s interface over http, and ames does the same over the ames
24−the same over the ames network. `%gall` uses ford to compile and run the
21+network. `%gall` uses ford to compile and run the applications.
25−applications.
26
22
27−Cards
23+## Cards
28−-----
29
24
30−`%gall` accepts the following cards. The first three are the most
25+`%gall` accepts the following cards. The first three are the most commonly used,
31−commonly used, while the others are primarily used internally.
26+while the others are primarily used internally.
32
27
33### `%mess`
28### `%mess`
34
29
35−Sends a message to an app. This will result in a call to the app's
30+Sends a message to an app. This will result in a call to the app's `++poke` arm.
36−`++poke` arm. The response is exactly one of a `%nice` if the action
31+The response is exactly one of a `%nice` if the action succeeded or a `%mean` if
37−succeeded or a `%mean` if not.
32+not.
38
33
39### `%show`
34### `%show`
40
35
41−Subscribes to a stream from an app. This will result in a call to the
36+Subscribes to a stream from an app. This will result in a call to the app's
42−app's either `++peek` or `++peer` arm. The first response will always be
37+either `++peek` or `++peer` arm. The first response will always be either a
43−either a `%nice` or a `%mean`, indicating whether or not the
38+`%nice` or a `%mean`, indicating whether or not the subscription was successful.
44−subscription was successful. After the first response, there will be
39+After the first response, there will be zero or more responses of either `%rush`
45−zero or more responses of either `%rush` or `%rust`, which communicate
40+or `%rust`, which communicate either a differntial or full update to the data
46−either a differntial or full update to the data stream. There may be a
41+stream. There may be a `%mean`, which indicates that the subscription has been
47−`%mean`, which indicates that the subscription has been canceled and no
42+canceled and no more responses will be received along this stream.
48−more responses will be received along this stream.
49−
50
43
51### `%nuke`
44### `%nuke`
52
45
53−Unsubscribes the current duct from its stream. This receives a response
46+Unsubscribes the current duct from its stream. This receives a response of
54−of either a `%nice` or a `%mean`. Note that a response of `%nice` does
47+either a `%nice` or a `%mean`. Note that a response of `%nice` does not imply
55−not imply that the current duct was in fact subscribed to any stream.
48+that the current duct was in fact subscribed to any stream.
56
49
57### `%init`
50### `%init`
58
51
59−Initializes a ship's apps. This should be called exactly once for each
52+Initializes a ship's apps. This should be called exactly once for each ship on
60−ship on the pier. This produces no moves in response.
53+the pier. This produces no moves in response.
61
54
62### `%sire`
55### `%sire`
63
56
64−Instantiates a child app. The app will be at path `[p parent-path]`, and
57+Instantiates a child app. The app will be at path `[p parent-path]`, and it will
65−it will be an instance of the `q` app. The only response will be `%gone`
58+be an instance of the `q` app. The only response will be `%gone` when the child
66−when the child dies.
59+dies.
67−
68
60
69### `%rote`
61### `%rote`
70
62
71−Signifies a remote request from ames. `r` should be of type `rook`. This
63+Signifies a remote request from ames. `r` should be of type `rook`. This how an
72−how an app on a foreign ship may send a `%mess`, `%show`, or `%nuke`
64+app on a foreign ship may send a `%mess`, `%show`, or `%nuke` card. Note that
73−card. Note that `%gall` automatically converts `%mess`, `%show`, and
65+`%gall` automatically converts `%mess`, `%show`, and `%nuke` into ames messages
74−`%nuke` into ames messages behind the scenes, so the only entity that
66+behind the scenes, so the only entity that should use `%rote` and `%roth` is
75−should use `%rote` and `%roth` is ames. Formally, the response is either
67+ames. Formally, the response is either a `%nice` or a `%mean`, which ames uses
76−a `%nice` or a `%mean`, which ames uses to give a positive or negative
68+to give a positive or negative ack. A logical response comes by passing a
77−ack. A logical response comes by passing a `%roth` card.
69+`%roth` card.
78−
79
70
80### `%roth`
71### `%roth`
81
72
82−Gives the response received from a remote request. `r` should be of type
73+Gives the response received from a remote request. `r` should be of type `roon`.
83−`roon`. This is how an app responds to a foreign request with a `%rush`,
74+This is how an app responds to a foreign request with a `%rush`, `%rust`,
84−`%rust`, `%nice`, or `%mean`. The response is either a `%nice` or a
75+`%nice`, or `%mean`. The response is either a `%nice` or a `%mean`. Even though
85−`%mean`. Even though we, as the proverb goes, "never ack an ack", we do
76+we, as the proverb goes, "never ack an ack", we do need to acknowledge these
86−need to acknowledge these responses since they're really independent
77+responses since they're really independent one-way messages.
87−one-way messages.
88−
89
78
90### `%wipe`
79### `%wipe`
91
80
92−Wipes the given app from memory. This is generally considered a hack,
81+Wipes the given app from memory. This is generally considered a hack, but it is
93−but it is sometimes useful during development to wipe the state of an
82+sometimes useful during development to wipe the state of an app. We don't
94−app. We don't guarantee that this actually completely wipes the app.
83+guarantee that this actually completely wipes the app. Generally, you want to
95−Generally, you want to use a `%cide` card if you actually want to kill
84+use a `%cide` card if you actually want to kill an app. This gives no response.
96−an app. This gives no response.
97−
98
85
99### `%cide`
86### `%cide`
100
87
101−Kills an app and all its children. Even though it's not technically a
88+Kills an app and all its children. Even though it's not technically a part of
102−part of `%gall`'s interface since it's not in `++kiss` and can't be
89+`%gall`'s interface since it's not in `++kiss` and can't be called from the
103−called from the outside, it's worth mentioning `%cide`, which may be
90+outside, it's worth mentioning `%cide`, which may be called from within `%gall`
104−called from within `%gall` apps. It should call `++part` to allow the
91+apps. It should call `++part` to allow the app any last words. This gives no
105−app any last words. This gives no response.
92+response.
106
93
107−Service Gates
94+## Service Gates
108−-------------
109
95
110### `++poke`
96### `++poke`
111
97
112−Handles incoming messages. Most commonly with an associated `%logo`. For
98+Handles incoming messages. Most commonly with an associated `%logo`. For example
113−example `++poke-json` handles an incoming JSON request from `%eyre`.
99+`++poke-json` handles an incoming JSON request from `%eyre`.
114
100
115### `++peer`
101### `++peer`
116
102
docs/arvo/marks.md+82−102
@@ -1,131 +1,111 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: true
3+next: True
4sort: 16
4sort: 16
5title: Marks
5title: Marks
6---
6---
7
7
8# Marks
8# Marks
9
9
10−We've used predefined marks already, but we haven't yet created
10+We've used predefined marks already, but we haven't yet created our own marks.
11−our own marks. Let's write a sample mark of our own, then chain
11+Let's write a sample mark of our own, then chain it together with some
12−it together with some preexisting ones to have fun with type
12+preexisting ones to have fun with type conversions.
13−conversions.
14
13
15−Let's make a small "examples-cord" mark. "Cord" is a name we use for `@t`
14+Let's make a small "examples-cord" mark. "Cord" is a name we use for `@t` text,
16−text, but there's no predefined mark for it. Let's put the following code in:
15+but there's no predefined mark for it. Let's put the following code in:
17`/mar/examples/cord.hoon`:
16`/mar/examples/cord.hoon`:
18
17
19−```
18+ /? 314
20−/? 314
19+ |_ cod/@t
21−|_ cod/@t
20+ ++ grab
22−++ grab
21+ |%
23− |%
22+ ++ atom |=(arg/@ `@t`(scot %ud arg))
24− ++ atom |=(arg/@ `@t`(scot %ud arg))
23+ --
25− --
24+ ++ grow
26−++ grow
25+ |%
27− |%
26+ ++ md cod
28− ++ md cod
27+ --
29− --
28+ --
30−--
29+
31−```
30+`/? 314` is the required version number, just like in apps. After that
32−
31+everything's in a `|_`, which is a `|%` core, but with input arguments. In our
33−`/? 314` is the required
34−version number, just like in apps. After that everything's in a
35−`|_`, which is a `|%` core, but with input arguments. In our
36case, the argument is the marked data.
32case, the argument is the marked data.
37
33
38−There are three possible top-level arms in the `|_` core,
34+There are three possible top-level arms in the `|_` core, `++grab`, `++grow`,
39−`++grab`, `++grow`, and `++grad` (which is used for revision
35+and `++grad` (which is used for revision control, covered elsewhere). `++grab`
40−control, covered elsewhere). `++grab` specifies
36+specifies functions to convert from another mark to the current mark. `++grow`
41−functions to convert from another mark to the current mark.
37+specifies how to convert from the current mark to another one.
42−`++grow` specifies how to convert from the current mark to
38+
43−another one.
39+In our case, we only define one arm in `++grab`, namely `++atom`. This allows us
44−
40+to convert any atom to a cord. `++scot` is a standard library function in
45−In our case, we only define one arm in `++grab`, namely `++atom`.
41+`hoon.hoon` which pretty-prints atoms of various types. We want to treat the
46−This allows us to convert any atom to a cord. `++scot` is a
42+atom as an unsigned decimal number, so we give `%ud` and the given argument to
47−standard library function in `hoon.hoon` which pretty-prints
43+`++scot`. Thus, the form of an arm in `++grab` is
48−atoms of various types. We want to treat the atom as an
49−unsigned decimal number, so we give `%ud` and the given argument
50−to `++scot`. Thus, the form of an arm in `++grab` is
51`|=(other-mark-type this-mark-value)`.
44`|=(other-mark-type this-mark-value)`.
52
45
53−In `++grow`, we just convert to the `md` mark, which is just a
46+In `++grow`, we just convert to the `md` mark, which is just a `@t` internally,
54−`@t` internally, but it has fancy conversion functions to things
47+but it has fancy conversion functions to things like html. `++md` isn't a
55−like html. `++md` isn't a function because the "argument" is
48+function because the "argument" is already a `@t`, so we can just produce the
56−already a `@t`, so we can just produce the new value directly.
49+new value directly.
57
50
58−Let's play around a bit with this mark. First, let's take a
51+Let's play around a bit with this mark. First, let's take a marked atom and
59−marked atom and convert it to our new mark.
52+convert it to our new mark.
60
53
61−```
54+ ~fintud-macrep:dojo> &atom 9
62−~fintud-macrep:dojo> &atom 9
55+ 9
63−9
56+ ~fintud-macrep:dojo> &examples-cord &atom 9
64−~fintud-macrep:dojo> &examples-cord &atom 9
57+ '9'
65−'9'
58+ -~fintud-macrep:dojo> &examples-cord &atom &examples-cord &atom 9
66−-~fintud-macrep:dojo> &examples-cord &atom &examples-cord &atom 9
59+ -'57'
67− -'57'
68−```
69
60
70−ASCII 9 is 57. There's no requirement, implicit or otherwise,
61+ASCII 9 is 57. There's no requirement, implicit or otherwise, that conversions
71−that conversions back and forth between marks be inverses of each
62+back and forth between marks be inverses of each other. These are semantic
72−other. These are semantic conversions, in that they refer to the
63+conversions, in that they refer to the same concept. They're not isomorphisms.
73−same concept. They're not isomorphisms.
74
64
75Let's play around a little more:
65Let's play around a little more:
76
66
77−```
67+ ~fintud-macrep:dojo> &md &examples-cord &atom 17
78−~fintud-macrep:dojo> &md &examples-cord &atom 17
68+ '17'
79−'17'
69+ ~fintud-macrep:dojo> &hymn &md &examples-cord &atom 17
80−~fintud-macrep:dojo> &hymn &md &examples-cord &atom 17
70+ [ [%html ~]
81−[ [%html ~]
71+ [[%head ~] [[%title ~] [[%~. [%~. "Untitled"] ~] ~] ~] ~]
82− [[%head ~] [[%title ~] [[%~. [%~. "Untitled"] ~] ~] ~] ~]
72+ [ [%body ~]
83− [ [%body ~]
73+ [ i=[g=[n=%meta a=~[[n=%value v="{}"] [n=%name v="frontmatter"] [n=[%urb %front] v=""]]] c=~]
84− [ i=[g=[n=%meta a=~[[n=%value v="{}"] [n=%name v="frontmatter"] [n=[%urb %front] v=""]]] c=~]
74+ t=[i=[g=[n=%p a=~] c=~[[g=[n=%$ a=~[[n=%$ v="17"]]] c=~]]] t=~]
85− t=[i=[g=[n=%p a=~] c=~[[g=[n=%$ a=~[[n=%$ v="17"]]] c=~]]] t=~]
75+ ]
76+ ]
77+ ~
86 ]
78 ]
87− ]
79+
88− ~
80+That was exciting. If you squint at that, it looks an awful lot like html. Our
89−]
81+value `17` is still in there, in the body. `urb` is the mark we render most web
90−```
82+pages to. It makes sure you have a complete skeleton of a web page. Of course,
91−
83+by the time this gets to the web page, it's plain html. Let's do the final step
92−That was exciting. If you squint at that, it looks an
84+in the conversion.
93−awful lot like html. Our value `17` is still in there, in the
85+
94−body. `urb` is the mark we render most web pages to. It makes
86+ ~fintud-macrep:dojo> &mime &hymn &md &examples-cord &atom 17
95−sure you have a complete skeleton of a web page. Of course, by
87+ [ [%text %html ~]
96−the time this gets to the web page, it's plain html. Let's do
88+ p=121
97−the final step in the conversion.
89+ q='<html><head><title>Untitled</title></head><body><meta value="{}" name="frontmatter" urb:front="" /><p>17</p></body></html>'
98−
90+ ]
99−```
91+
100−~fintud-macrep:dojo> &mime &hymn &md &examples-cord &atom 17
92+This is a mime-typed octet stream with type `/text/html`, length 121 bytes, and
101−[ [%text %html ~]
93+our lowly number `17` rendered to a web page. `examples-cord` was just one step
102− p=121
94+in the chain.
103− q='<html><head><title>Untitled</title></head><body><meta value="{}" name="frontmatter" urb:front="" /><p>17</p></body></html>'
104−]
105−```
106−
107−This is a mime-typed octet stream with type `/text/html`, length
108−121 bytes, and our lowly number `17` rendered to a web page.
109−`examples-cord` was just one step in the chain.
110
95
111Of course, arvo can infer some parts of this chain:
96Of course, arvo can infer some parts of this chain:
112
97
113−```
98+ ~fintud-macrep:dojo> &examples-cord 17
114−~fintud-macrep:dojo> &examples-cord 17
99+ '17'
115−'17'
116−```
117
100
118−Likewise, `md` is the only way to get from `examples-cord` to `hymn`, so
101+Likewise, `md` is the only way to get from `examples-cord` to `hymn`, so that
119−that can be omitted. If we omit the `hymn`, though, we get:
102+can be omitted. If we omit the `hymn`, though, we get:
120
103
121−```
104+ ~fintud-macrep:dojo> &mime &examples-cord 17
122−~fintud-macrep:dojo> &mime &examples-cord 17
105+ [[%text %x-markdown ~] p=1 q='7']
123−[[%text %x-markdown ~] p=1 q='7']
124−```
125
106
126−Here, we converted straight from `examples-cord` to `md` to `mime`. Arvo
107+Here, we converted straight from `examples-cord` to `md` to `mime`. Arvo decided
127−decided that was the most straightforward conversion, but it
108+that was the most straightforward conversion, but it gives a different result
128−gives a different result than passing it through the `hymn`
109+than passing it through the `hymn` mark.
129−mark.
130
110
131The minimal correct input, then, is `&mime &hymn &examples-cord &atom 17`.
111The minimal correct input, then, is `&mime &hymn &examples-cord &atom 17`.
docs/arvo/network.md+183−202
@@ -1,19 +1,18 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: false
3+next: False
4sort: 13
4sort: 13
5title: Network messages
5title: Network messages
6---
6---
7
7
8−Network messages
8+# Network messages
9−================
10
9
11−Now we've learned enough Hoon to do more interesting things. Let's get
10+Now we've learned enough Hoon to do more interesting things. Let's get our
12−our planets to talk to each other.
11+planets to talk to each other.
13
12
14−All we've written up until now are just shell commands that produce a
13+All we've written up until now are just shell commands that produce a value and
15−value and then disappear. To listen for and receive messages from other
14+then disappear. To listen for and receive messages from other planets, we'll
16−planets, we'll need an app. Let's look at a very simple one:
15+need an app. Let's look at a very simple one:
17
16
18 :: Accepts any noun from dojo and prints it out
17 :: Accepts any noun from dojo and prints it out
19 ::
18 ::
@@ -29,10 +28,9 @@ planets, we'll need an app. Let's look at a very simple one:
29 [~ +>.$]
28 [~ +>.$]
30 --
29 --
31
30
32−This is a very simple app that does only one thing. If you poke it with
31+This is a very simple app that does only one thing. If you poke it with a value,
33−a value, it prints that value out. To try this out, you have to start
32+it prints that value out. To try this out, you have to start the app, then you
34−the app, then you can poke it from the command line with the following
33+can poke it from the command line with the following commands:
35−commands:
36
34
37 ~fintud-macrep:dojo> |start %examples-echo
35 ~fintud-macrep:dojo> |start %examples-echo
38 'prep'
36 'prep'
@@ -44,46 +42,44 @@ commands:
44 [%argument [1 2]]
42 [%argument [1 2]]
45 >=
43 >=
46
44
47−> There is currently a bug where the `%argument` lines are printed
45+> There is currently a bug where the `%argument` lines are printed *above* the
48−> *above* the line you entered, so your output may not look exactly like
46+> line you entered, so your output may not look exactly like this. `>=` means
49−> this. `>=` means that a command was successfully received and
47+> that a command was successfully received and executed.
50−> executed.
51
48
52Most of the app code should be simple enough to guess its function. The
49Most of the app code should be simple enough to guess its function. The
53important part of this code is the definition of `++poke-noun`.
50important part of this code is the definition of `++poke-noun`.
54
51
55−Once an app starts, it's always running in the background and you
52+Once an app starts, it's always running in the background and you interact with
56−interact with it by sending it messages. The most straightforward way to
53+it by sending it messages. The most straightforward way to do that is to poke it
57−do that is to poke it from the command line, which we we did with
54+from the command line, which we we did with `:echo 5`
58−`:echo 5` (`:[app-name] [argument(s)]`).
55+(`:[app-name] [argument(s)]`).
59−
56+
60−In this case, `++poke-noun` takes an argument `arg` and prints it out
57+In this case, `++poke-noun` takes an argument `arg` and prints it out with `~&`
61−with `~&` ([sigpam](../../hoon/twig/sig-hint/pam-dump/)). This is an unusual
58+([sigpam](../../hoon/twig/sig-hint/pam-dump/)). This is an unusual rune that
62−rune that formally "does nothing", but the interpreter detects it and
59+formally "does nothing", but the interpreter detects it and printfs the first
63−printfs the first child, before executing the second as if the first
60+child, before executing the second as if the first didn't exist. This is a
64−didn't exist. This is a slightly hacky way of printing to the console,
61+slightly hacky way of printing to the console, but we'll get to the correct way
65−but we'll get to the correct way later on.
62+later on.
66−
63+
67−But what does `++poke-noun` produce? Recall that `^-` casts to a type.
64+But what does `++poke-noun` produce? Recall that `^-` casts to a type. In this
68−In this case, it's declaring that the end result of the function will be of
65+case, it's declaring that the end result of the function will be of type
69−type `{(list) _+>.$}`. But what does this mean?
66+`{(list) _+>.$}`. But what does this mean?
70−
67+
71−The phrase to remember is "a list of moves and our state". Urbit is a
68+The phrase to remember is "a list of moves and our state". Urbit is a message
72−message passing system, so whenever we want to do something that
69+passing system, so whenever we want to do something that interacts with the rest
73−interacts with the rest of the system, we send a message. Thus, a move
70+of the system, we send a message. Thus, a move is Arvo's equivalent of a
74−is Arvo's equivalent of a syscall. The first thing that `++poke-noun`
71+syscall. The first thing that `++poke-noun` produces is a list of messages,
75−produces is a list of messages, called "moves". In this case, we don't
72+called "moves". In this case, we don't actually want the system to do anything,
76−actually want the system to do anything, so we produce the empty list,
73+so we produce the empty list, `~` (in the `[~ +>.$]` line).
77−`~` (in the `[~ +>.$]` line).
78
74
79The second thing `++poke-noun` produces is our state. `+>.$` refers to a
75The second thing `++poke-noun` produces is our state. `+>.$` refers to a
80−particular address in our subject where our formal app state is stored.
76+particular address in our subject where our formal app state is stored. It'll
81−It'll become clear why this is later on, but for now pretend that `+>.$`
77+become clear why this is later on, but for now pretend that `+>.$` is a magic
82−is a magic invocation that means "app state".
78+invocation that means "app state".
83
79
84−Let's look at another example (edit your code in
80+Let's look at another example (edit your code in `/app/examples/echo/hoon` to
85−`/app/examples/echo/hoon` to reflect the code below). Say we want to
81+reflect the code below). Say we want to only accept a number, and then print out
86−only accept a number, and then print out the square of that number.
82+the square of that number.
87
83
88 /? 314
84 /? 314
89 !:
85 !:
@@ -96,31 +92,27 @@ only accept a number, and then print out the square of that number.
96 [~ +>.$]
92 [~ +>.$]
97 --
93 --
98
94
99−A few things have changed. Firstly, we no longer accept arbitrary nouns
95+A few things have changed. Firstly, we no longer accept arbitrary nouns because
100−because we can only square atoms (integers, in this case an unsigned
96+we can only square atoms (integers, in this case an unsigned one). Thus, our
101−one). Thus, our argument is now `arg/@`. Secondly, it's `++poke-atom`
97+argument is now `arg/@`. Secondly, it's `++poke-atom` rather than `++poke-noun`.
102−rather than `++poke-noun`.
103
98
104−Intro to marks
99+# Intro to marks
105−==============
106
100
107−Are there other `++poke`s? Yes. In fact, `noun` and `atom` are just two
101+Are there other `++poke`s? Yes. In fact, `noun` and `atom` are just two of
108−of arbitrarily many "marks". A mark is fundamentally a type definition,
102+arbitrarily many "marks". A mark is fundamentally a type definition, but
109−but accessible at the Arvo level. Each mark is defined in the `/mar`
103+accessible at the Arvo level. Each mark is defined in the `/mar` directory. Some
110−directory. Some marks have conversion routines to other marks, and some
104+marks have conversion routines to other marks, and some have diff, patch, and
111−have diff, patch, and merge algorithms. None of these are required for a
105+merge algorithms. None of these are required for a mark to exist, however.
112−mark to exist, however.
113
106
114−`noun` and `atom` are two predefined marks. In your `/mar` directory,
107+`noun` and `atom` are two predefined marks. In your `/mar` directory, there are
115−there are already many more, and you may add more at will. The type
108+already many more, and you may add more at will. The type associated with `noun`
116−associated with `noun` is `*`, and the type associated with `atom` is
109+is `*`, and the type associated with `atom` is `@`.
117−`@`.
118
110
119−When we poke an app from anywhere, we do so with a mark that searches
111+When we poke an app from anywhere, we do so with a mark that searches for the
120−for the corresponding (`++poke-[mark]`) arm. Data constructed on the
112+corresponding (`++poke-[mark]`) arm. Data constructed on the command line is by
121−command line is by default marked with `noun`. In this case, the app is
113+default marked with `noun`. In this case, the app is expecting an atom, so we
122−expecting an atom, so we have to explicitly mark the data with `atom`
114+have to explicitly mark the data with `atom` using `&[mark]`. Try the following
123−using `&[mark]`. Try the following commands:
115+commands:
124
116
125 ~fintud-macrep:dojo> |start %square
117 ~fintud-macrep:dojo> |start %square
126 >=
118 >=
@@ -132,16 +124,15 @@ using `&[mark]`. Try the following commands:
132
124
133> Recall the bug where `%square` may get printed above the input line.
125> Recall the bug where `%square` may get printed above the input line.
134
126
135−Marks are powerful, and they're the backbone of Urbit's data pipeline,
127+Marks are powerful, and they're the backbone of Urbit's data pipeline, so we'll
136−so we'll be getting quite used to them.
128+be getting quite used to them.
137
129
138**Exercise**:
130**Exercise**:
139
131
140- Write an app that computes fizzbuzz on its input (as in the previous
132- Write an app that computes fizzbuzz on its input (as in the previous
141 section).
133 section).
142
134
143−Sending a message to another urbit
135+# Sending a message to another urbit
144−==================================
145
136
146Let's write our first network message! Here's `/app/examples/pong.hoon`:
137Let's write our first network message! Here's `/app/examples/pong.hoon`:
147
138
@@ -173,36 +164,34 @@ Run it with these commands:
173 ~fintud-macrep:dojo> :examples-pong &urbit ~sampel-sipnym
164 ~fintud-macrep:dojo> :examples-pong &urbit ~sampel-sipnym
174 >=
165 >=
175
166
176−Replace `~sampel-sipnym` with another urbit. The easiest thing to do is
167+Replace `~sampel-sipnym` with another urbit. The easiest thing to do is to start
177−to start a comet, a free and disposable Urbit identity. If you don't know
168+a comet, a free and disposable Urbit identity. If you don't know how to start a
178−how to start a comet, see [the user setup section](/docs/using/setup/).
169+comet, see [the user setup section](/docs/using/setup/). Don't forget to start
179−Don't forget to start the `%examples-pong` app on that urbit, too. You
170+the `%examples-pong` app on that urbit, too. You should see, on the foreign
180−should see, on the foreign urbit, this output:
171+urbit, this output:
181
172
182 [%receiving 'howdy']
173 [%receiving 'howdy']
183
174
184−Most of the code should be straightforward. In `++poke-atom`, the only
175+Most of the code should be straightforward. In `++poke-atom`, the only new thing
185−new thing is the expression `(@t arg)`, which is the type `@t` being
176+is the expression `(@t arg)`, which is the type `@t` being called as a function
186−called as a function with argument `arg`. As we already know, `@t` is
177+with argument `arg`. As we already know, `@t` is the type of "cord" text
187−the type of "cord" text strings. In Hoon, when types are called as
178+strings. In Hoon, when types are called as functions, they serve as a validator
188−functions, they serve as a validator function called a "clam" -- that
179+function called a "clam" -- that is, a function whose domain is all nouns, and
189−is, a function whose domain is all nouns, and range is the given type
180+range is the given type (in this case, `@t`).
190−(in this case, `@t`).
181+
191−
182+In simpler terms, if a clam is passed a value of its own type, it produces that
192−In simpler terms, if a clam is passed a value of its own type, it
183+value. Otherwise, it produces the default value (aka the "bunt") of its type.
193−produces that value. Otherwise, it produces the default value (aka the
184+Here we call this `@t` function on the argument. This coerces the argument to
194−"bunt") of its type. Here we call this `@t` function on the argument.
185+text, so that we can print it out prettily no matter what we're passed.
195−This coerces the argument to text, so that we can print it out prettily
186+
196−no matter what we're passed.
187+The more interesting part is in `++poke-urbit`. The `urbit` mark is an urbit
197−
188+identity, and the Hoon type associated with it is `@p` (the "p" stands for
198−The more interesting part is in `++poke-urbit`. The `urbit` mark is an
189+"phonetic base").
199−urbit identity, and the Hoon type associated with it is `@p` (the "p"
190+
200−stands for "phonetic base").
191+Recall that in a `++poke` arm we produce "a list of moves and our state". Until
201−
192+now, we've left the list of moves empty, since we haven't wanted to tell Arvo to
202−Recall that in a `++poke` arm we produce "a list of moves and our
193+do anything in particular. Now we want to send a message to another urbit. Thus,
203−state". Until now, we've left the list of moves empty, since we haven't
194+we produce a list with one element:
204−wanted to tell Arvo to do anything in particular. Now we want to send a
205−message to another urbit. Thus, we produce a list with one element:
206
195
207 [ost %poke /sending [to-urbit-address %pong] %atom 'howdy']
196 [ost %poke /sending [to-urbit-address %pong] %atom 'howdy']
208
197
@@ -218,126 +207,118 @@ Let's walk through each of these elements step by step.
218
207
219#### Bones ("cause")
208#### Bones ("cause")
220
209
221−If you look up `++bone` in `hoon.hoon`, you'll see that it's a number
210+If you look up `++bone` in `hoon.hoon`, you'll see that it's a number (`@ud`),
222−(`@ud`), and that it's an opaque reference to a duct. `++duct` in
211+and that it's an opaque reference to a duct. `++duct` in hoon.hoon is a list of
223−hoon.hoon is a list of `wire`s, where `++wire` is an alias for `++path`.
212+`wire`s, where `++wire` is an alias for `++path`. `++path` is a list of
224−`++path` is a list of `++knot`s, which are ASCII text. Thus, a duct is a
213+`++knot`s, which are ASCII text. Thus, a duct is a list of paths, and a bone is
225−list of paths, and a bone is an opaque reference to it (in the same way
214+an opaque reference to it (in the same way that a Unix file descriptor is an
226−that a Unix file descriptor is an opaque reference to a file structure).
215+opaque reference to a file structure). Thus, to truly understand bones, we must
227−Thus, to truly understand bones, we must understand ducts.
216+understand ducts.
228−
217+
229−A duct is stack of causes, again, represented as paths, which are called
218+A duct is stack of causes, again, represented as paths, which are called wires.
230−wires. At the bottom of every duct is a unix event, such as a keystroke,
219+At the bottom of every duct is a unix event, such as a keystroke, network
231−network packet, file change, or timer event. When Arvo is given this
220+packet, file change, or timer event. When Arvo is given this event, it routes
232−event, it routes the event to appropriate kernel module for handling.
221+the event to appropriate kernel module for handling.
233−
222+
234−Sometimes, the module can immediately handle the event and produce any
223+Sometimes, the module can immediately handle the event and produce any necessary
235−necessary results. For example, when we poked the `++poke-atom` arm
224+results. For example, when we poked the `++poke-atom` arm above, we poked %gall,
236−above, we poked %gall, our application server, which was able to respond
225+our application server, which was able to respond to our poke directly. When the
237−to our poke directly. When the module cannot service the request itself,
226+module cannot service the request itself, it sends instructions to another
238−it sends instructions to another kernel module or application (through the
227+kernel module or application (through the %gall module) to do a specified
239−%gall module) to do a specified action, and produces the result from
228+action, and produces the result from that.
240−that.
241
229
242Furthermore, when one module sends a message to another kernel module or
230Furthermore, when one module sends a message to another kernel module or
243−application, it also sends along the duct it was given with its new wire
231+application, it also sends along the duct it was given with its new wire tacked
244−tacked onto the end. Now the duct has two entries, with the unix event on
232+onto the end. Now the duct has two entries, with the unix event on the bottom
245−the bottom and the kernel module that handled it next. This process can
233+and the kernel module that handled it next. This process can continue
246−continue indefinitely, adding more and more layers onto the duct. When
234+indefinitely, adding more and more layers onto the duct. When an entity finally
247−an entity finally produces a result, a layer is popped off the duct, and
235+produces a result, a layer is popped off the duct, and the result is passed all
248−the result is passed all the way back down.
236+the way back down.
249−
237+
250−In effect, a duct is an Arvo-level call stack. It's worth noting that
238+In effect, a duct is an Arvo-level call stack. It's worth noting that while in
251−while in traditional call stacks a function call happens synchronously
239+traditional call stacks a function call happens synchronously and returns
252−and returns exactly once. In Arvo, multiple moves can be sent at once,
240+exactly once. In Arvo, multiple moves can be sent at once, are evaluated
253−are evaluated asynchronously, and each one may be responded to zero or
241+asynchronously, and each one may be responded to zero or more times.
254−more times.
242+
255−
243+The point to take home is that whatever caused `++poke-urbit` to be called is
256−The point to take home is that whatever caused `++poke-urbit` to be
244+also the root cause for the network message we're trying to send. Thus, we say
257−called is also the root cause for the network message we're trying to
245+to send the network message along the given bone `ost`.
258−send. Thus, we say to send the network message along the given bone
259−`ost`.
260
246
261##### Wire ('tack on new layer to duct')
247##### Wire ('tack on new layer to duct')
262
248
263−Of course, we have to push a new layer onto our duct before passing it
249+Of course, we have to push a new layer onto our duct before passing it along (or
264−along (or responding to it directly) anywhere. This layer can have any
250+responding to it directly) anywhere. This layer can have any data we want in it,
265−data we want in it, but we don't need anything specific here, so we just
251+but we don't need anything specific here, so we just use the wire `/sending`
266−use the wire `/sending` (`/elem1/elem2/elemN` is one syntax used to
252+(`/elem1/elem2/elemN` is one syntax used to create `++path`s and `++wire`s of N
267−create `++path`s and `++wire`s of N elements). If we were expecting a
253+elements). If we were expecting a response (which we're not), it would come back
268−response (which we're not), it would come back along the `/sending`
254+along the `/sending` wire, meaning that the path `/sending` will be passed back
269−wire, meaning that the path `/sending` will be passed back to the
255+to the response handler as an argument. Although it's not required, it's
270−response handler as an argument. Although it's not required, it's
256+generally a good idea to make the wire human-readable for bug-handling purposes.
271−generally a good idea to make the wire human-readable for bug-handling
272−purposes.
273
257
274##### Term (sys-call)
258##### Term (sys-call)
275
259
276−Each move also has a `term`, composed of lowercase ASCII and/or `-`.
260+Each move also has a `term`, composed of lowercase ASCII and/or `-`. This `term`
277−This `term` has the sign `@tas`. In this case, out `term` is `%poke`,
261+has the sign `@tas`. In this case, out `term` is `%poke`, which is the name of
278−which is the name of the particular kind of move we're sending. You can
262+the particular kind of move we're sending. You can always use `%poke` to message
279−always use `%poke` to message an app. Other common names include
263+an app. Other common names include `%warp`, to read from the filesystem;
280−`%warp`, to read from the filesystem; `%wait`, to set a timer; and
264+`%wait`, to set a timer; and `%them`, to send an http request.
281−`%them`, to send an http request.
282
265
283The move ends with `*` (that is, any noun) since each type of move takes
266The move ends with `*` (that is, any noun) since each type of move takes
284−different data. In our case, a `%poke` move takes a target (urbit and
267+different data. In our case, a `%poke` move takes a target (urbit and app) and
285−app) and marked data, then pokes the arm of the corresponding mark on
268+marked data, then pokes the arm of the corresponding mark on that app on that
286−that app on that urbit with that data. `[to-urbit-address %pong]` is the
269+urbit with that data. `[to-urbit-address %pong]` is the target urbit and app,
287−target urbit and app, `%atom` is the `mark`, and`'howdy'` is the data.
270+`%atom` is the `mark`, and`'howdy'` is the data.
288
271
289−When Arvo receives a `%poke` move, it calls the appropriate `++poke`.
272+When Arvo receives a `%poke` move, it calls the appropriate `++poke`. The same
290−The same mechanism is used for sending messages between apps on the same
273+mechanism is used for sending messages between apps on the same urbit as for
291−urbit as for sending messages between apps on different urbits.
274+sending messages between apps on different urbits.
292
275
293−> We said earlier that we're not expecting a response. This is not
276+> We said earlier that we're not expecting a response. This is not entirely
294−> entirely true: the `++coup` is called when we receive acknowledgment
277+> true: the `++coup` is called when we receive acknowledgment that the `++poke`
295−> that the `++poke` was called. We don't do anything with this
278+> was called. We don't do anything with this information right now, but we
296−> information right now, but we could.
279+> could.
297
280
298**Exercises**:
281**Exercises**:
299
282
300−- Extend either of the apps in the first two exercises to accept input
283+- Extend either of the apps in the first two exercises to accept input over
301− over the network in the same way as `pong`.
284+ the network in the same way as `pong`.
302
285
303- Modify `examples-pong` to print out a message when it receives
286- Modify `examples-pong` to print out a message when it receives
304 acknowledgement.
287 acknowledgement.
305
288
306−- Write two apps, `even` and `odd`. When you pass an atom to `even`,
289+- Write two apps, `even` and `odd`. When you pass an atom to `even`, check
307− check whether it's even. If so, divide it by two and recurse;
290+ whether it's even. If so, divide it by two and recurse; otherwise, poke
308− otherwise, poke `odd` with it. When `odd` receives an atom, check
291+ `odd` with it. When `odd` receives an atom, check whether it's equal to one.
309− whether it's equal to one. If so, terminate, printing "%success".
292+ If so, terminate, printing "%success". Otherwise, check whether it's odd. If
310− Otherwise, check whether it's odd. If so, multiply it by three, add
293+ so, multiply it by three, add one, and recurse; otherwise, poke `even` with
311− one, and recurse; otherwise, poke `even` with it. When either app
294+ it. When either app receives a number, print it out along with the name of
312− receives a number, print it out along with the name of the app. In
295+ the app. In the end, you should be able to watch Collatz's conjecture play
313− the end, you should be able to watch Collatz's conjecture play out
296+ out between the two apps. Sample output:
314− between the two apps. Sample output:
297+
315−
298+<!-- -->
316−```
299+
317−~fintud-macrep:dojo> :even &atom 18
300+ ~fintud-macrep:dojo> :even &atom 18
318−[%even 18]
301+ [%even 18]
319−[%odd 9]
302+ [%odd 9]
320−[%even 28]
303+ [%even 28]
321−[%even 14]
304+ [%even 14]
322−[%odd 7]
305+ [%odd 7]
323−[%even 22]
306+ [%even 22]
324−[%odd 11]
307+ [%odd 11]
325−[%even 34]
308+ [%even 34]
326−[%odd 17]
309+ [%odd 17]
327−[%even 52]
310+ [%even 52]
328−[%even 26]
311+ [%even 26]
329−[%odd 13]
312+ [%odd 13]
330−[%even 40]
313+ [%even 40]
331−[%even 20]
314+ [%even 20]
332−[%even 10]
315+ [%even 10]
333−[%odd 5]
316+ [%odd 5]
334−[%even 16]
317+ [%even 16]
335−[%even 8]
318+ [%even 8]
336−[%even 4]
319+ [%even 4]
337−[%even 2]
320+ [%even 2]
338−%success
321+ %success
339−```
322+
340−
323+- Put `even` and `odd` on two separate urbits and pass the messages over the
341−- Put `even` and `odd` on two separate urbits and pass the messages
324+ network. Post a link to a working solution in :talk to receive a cookie.
342− over the network. Post a link to a working solution in :talk to
343− receive a cookie.
docs/arvo/security-driver.md+215−255
@@ -1,302 +1,262 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: false
3+next: False
4sort: 20
4sort: 20
5title: Security drivers
5title: Security drivers
6---
6---
7
7
8# Security drivers
8# Security drivers
9
9
10−A security driver is a file in `/=home=/sec/<tld>/<domain>/hoon`
10+A security driver is a file in `/=home=/sec/<tld>/<domain>/hoon` that handles
11−that handles the authentication for all HTTP requests to
11+the authentication for all HTTP requests to `https://<domain>.<tld>`. When
12−`https://<domain>.<tld>`. When anything in Urbit makes an HTTP
12+anything in Urbit makes an HTTP request through `%eyre` (our web server), it
13−request through `%eyre` (our web server), it checks to see if we
13+checks to see if we have a security driver for the requested domain, and if so
14−have a security driver for the requested domain, and if so
14+filters the request through the driver. The security driver will usually either
15−filters the request through the driver. The security driver will
15+decorate the request with the needed credentials if it has them, or else it will
16−usually either decorate the request with the needed credentials
16+guide the user through the process of authenticating Urbit with the service.
17−if it has them, or else it will guide the user through the
17+
18−process of authenticating Urbit with the service.
18+Each web service needs its own security driver, but most of them are pretty
19−
19+standard. We recommend starting with an existing security driver based on your
20−Each web service needs its own security driver, but most of them
20+needed authentication method (e.g. basic auth, OAuth1, OAuth2), and changing
21−are pretty standard. We recommend starting with an existing
21+whatever is specific to your service.
22−security driver based on your needed authentication method (e.g.
22+
23−basic auth, OAuth1, OAuth2), and changing whatever is specific to
23+The best strategy for building security drivers is to copy a similar one and
24−your service.
24+tweak it until it works for you. The best representatives are Github for Basic
25−
25+Authentication, Twitter for OAuth1, Slack for OAuth2 when access tokens don't
26−The best strategy for building security drivers is to copy a
26+expire, and Google APIs for OAuth2 when access tokens do expire. In most cases
27−similar one and tweak it until it works for you. The best
27+you'll need to make very few changes to one of these models.
28−representatives are Github for Basic Authentication, Twitter for
28+
29−OAuth1, Slack for OAuth2 when access tokens don't expire, and
29+Still, it's worth seeing one or two built from the ground up. Here, we'll build
30−Google APIs for OAuth2 when access tokens do expire. In most
30+a connector for the Github API v3. The simplest way to interact with the Github
31−cases you'll need to make very few changes to one of these
31+API is to just fetch https://api.github.com from the dojo:
32−models.
33−
34−Still, it's worth seeing one or two built from the ground up.
35−Here, we'll build a connector for the Github API v3. The
36−simplest way to interact with the Github API is to just fetch
37−https://api.github.com from the dojo:
38
32
39 ~your-urbit:dojo> +https://api.github.com
33 ~your-urbit:dojo> +https://api.github.com
40
34
41−Github exposes a few endpoints to the general web, and the root
35+Github exposes a few endpoints to the general web, and the root endpoint is one
42−endpoint is one of them. This gives you a textual representation
36+of them. This gives you a textual representation of a JSON object that contains
43−of a JSON object that contains a bunch of urls to other parts of
37+a bunch of urls to other parts of Github's API. This is exactly the same
44−Github's API. This is exactly the same response you would get
38+response you would get from just running `curl https://api.github.com` on UNIX.
45−from just running `curl https://api.github.com` on UNIX.
46
39
47−If we don't have a security driver for Github yet, many of the
40+If we don't have a security driver for Github yet, many of the endpoints won't
48−endpoints won't be accessible, or they will only have publicly
41+be accessible, or they will only have publicly accessible information. Most of
49−accessible information. Most of what we care about requires us
42+what we care about requires us to be authenticated.
50−to be authenticated.
51
43
52## Basic auth
44## Basic auth
53
45
54Here's a simple security driver:
46Here's a simple security driver:
55
47
56−```
48+ :: Test url +https://api.github.com/user
57−:: Test url +https://api.github.com/user
49+ ::
58−::
50+ :::: /hoon/github/com/sec
59−:::: /hoon/github/com/sec
51+ ::
60− ::
52+ /+ basic-auth
61−/+ basic-auth
53+ !:
62−!:
54+ |_ {bal/(bale keys:basic-auth) $~}
63−|_ {bal/(bale keys:basic-auth) $~}
55+ ++ aut ~(standard basic-auth bal ~)
64−++ aut ~(standard basic-auth bal ~)
56+ ++ filter-request out-adding-header:aut
65−++ filter-request out-adding-header:aut
57+ --
66−--
58+
67−```
59+Github supports authentication through either Basic Authentication or OAuth2.
68−
60+We'll show the basic auth example first, but in general we'd prefer OAuth2.
69−Github supports authentication through either Basic
61+
70−Authentication or OAuth2. We'll show the basic auth example
62+Since this driver is for Github, put it in `/=home=/sec/com/github/hoon`.
71−first, but in general we'd prefer OAuth2.
72−
73−Since this driver is for Github, put it in
74−`/=home=/sec/com/github/hoon`.
75
63
76To try this out, we first have to initialize our credentials with
64To try this out, we first have to initialize our credentials with
77−`|init-auth-basic`, which prompts for the url of the service
65+`|init-auth-basic`, which prompts for the url of the service (api.github.com),
78−(api.github.com), username, and password. It stores this
66+username, and password. It stores this information in a manner that `%eyre`
79−information in a manner that `%eyre` knows how to decode and send
67+knows how to decode and send put into your `bale`. Note that `|init-auth-basic`
80−put into your `bale`. Note that `|init-auth-basic` is standard for
68+is standard for basic auth, but other auth schemes (like OAuth) are initialized
81−basic auth, but other auth schemes (like OAuth) are initialized
82in other ways.
69in other ways.
83
70
84After you've run `|init-auth-basic`, you should be able to run
71After you've run `|init-auth-basic`, you should be able to run
85−`+https://api.github.com/user` and get a response indicating who
72+`+https://api.github.com/user` and get a response indicating who you're logged
86−you're logged in as.
73+in as.
87
74
88You can similarly POST:
75You can similarly POST:
89
76
90 ~your-urbit:dojo> +https://api.github.com/gists &json _(cork poja need) '{"files":{"file1.txt":{"content":"can\'t stop the signal"}}}'
77 ~your-urbit:dojo> +https://api.github.com/gists &json _(cork poja need) '{"files":{"file1.txt":{"content":"can\'t stop the signal"}}}'
91
78
92−This creates a gist, go to `https://gist.github.com/<username>` to
79+This creates a gist, go to `https://gist.github.com/<username>` to see it.
93−see it.
94
80
95−The `+url` syntax used in "source" position makes a GET request,
81+The `+url` syntax used in "source" position makes a GET request, but if you pass
96−but if you pass it data it will make a POST request with that
82+it data it will make a POST request with that data as the body. This request is
97−data as the body. This request is authenticated seamlessly. You
83+authenticated seamlessly. You can send data of any mark as long as it has a
98−can send data of any mark as long as it has a conversion path to
84+conversion path to `%mime`.
99−`%mime`.
100
85
101−Let's take a look at the code and see if we can figure out what's
86+Let's take a look at the code and see if we can figure out what's happening
102−happening here:
87+here:
103
88
104 /+ basic-auth
89 /+ basic-auth
105
90
106−First, we load a library called `basic-auth`. This library
91+First, we load a library called `basic-auth`. This library exposes two items.
107−exposes two items. `keys` is the type of the basic auth key,
92+`keys` is the type of the basic auth key, which is essentially a base64'd
108−which is essentially a base64'd username and password.
93+username and password. `standard` accepts a `bale` and produces a core with
109−`standard` accepts a `bale` and produces a core with everything
94+everything we'll need to implement basic auth.
110−we'll need to implement basic auth.
95+
111−
96+Every security driver needs some amount of state, which is stored in `bale`.
112−Every security driver needs some amount of state, which is stored
97+`bale` contains:
113−in `bale`. `bale` contains:
98+
114−
99+- `our`, our urbit name
115−- `our`, our urbit name
100+- `now`, the current time
116−- `now`, the current time
101+- `eny`, 256 bits of entropy
117−- `eny`, 256 bits of entropy
102+- `byk`, the urbit, desk and case which the security driver is running from
118−- `byk`, the urbit, desk and case which the security driver is
103+- `usr`, the particular identity we're logging in with
119− running from
104+- `dom`, the site we're accessing
120−- `usr`, the particular identity we're logging in with
105+- `key`, the secrets required to authenticate requests. The type for this
121−- `dom`, the site we're accessing
106+ entry is supplied by the programmer as the argument to `++bale`. Thus, in
122−- `key`, the secrets required to authenticate requests. The type
107+ our case, `key` in our bale is of type `keys:basic-auth`.
123− for this entry is supplied by the programmer as the argument to
108+
124− `++bale`. Thus, in our case, `key` in our bale is of type
109+Additionally, a security driver contains at least one out of the five "special"
125− `keys:basic-auth`.
110+arms:
126−
111+
127−Additionally, a security driver contains at least one out of the
112+- `++filter-request` is a function which takes a hiss (http request) and
128−five "special" arms:
113+ produces a `sec-move` (defined in zuse), which is one of:
129−
114+
130−- `++filter-request` is a function which takes a hiss (http
115+- `[%send hiss]`, which sends the new hiss. This is the case in, for example,
131− request) and produces a `sec-move` (defined in zuse), which is
116+ basic auth, where all we need to do is add an extra header to the request.
132− one of:
117+- `[%give httr]`, where httr is an http response. This immediately returns an
133−
118+ http response to the sender.
134− * `[%send hiss]`, which sends the new hiss. This is the
119+- `[%show purl]`, where a purl is a parsed url. This displays a message asking
135− case in, for example, basic auth, where all we need to do is
120+ the user to visit the given url to continue the authentication process.
136− add an extra header to the request.
121+- `[%redo ~]`, which redoes the request.
137− * `[%give httr]`, where httr is an http response. This
122+
138− immediately returns an http response to the sender.
123+`%eyre` calls `++filter-request` just before sending an HTTP request to the
139− * `[%show purl]`, where a purl is a parsed url. This displays
124+specified domain. This allows the security driver to filter the requests,
140− a message asking the user to visit the given url to continue
125+decorating them with authentication data.
141− the authentication process.
126+
142− * `[%redo ~]`, which redoes the request.
127+- `++filter-response` is a function which takes an httr and produces a
143−
128+ `sec-move`. `%eyre` calls it after receiving an HTTP response from the
144− `%eyre` calls `++filter-request` just before sending an HTTP
129+ specified domain. This allows the security driver to handle authentication
145− request to the specified domain. This allows the security
130+ errors, commonly caused by expired tokens, and retry the request.
146− driver to filter the requests, decorating them with
131+
147− authentication data.
132+- `++receive-auth-query-string` is a function that takes a `quay` (list of
148−
133+ query parameters) and produces a `sec-move`. `%eyre` calls it when it
149−- `++filter-response` is a function which takes an httr and
134+ receives a request on the callback url. Generally, this happens in OAuth
150− produces a `sec-move`. `%eyre` calls it after receiving an
135+ after the user has granted access to the urbit app, and the service makes a
151− HTTP response from the specified domain. This allows the
136+ request to the callback url with the code. The security driver should then
152− security driver to handle authentication errors, commonly
137+ make a request to convert the code into an access token.
153− caused by expired tokens, and retry the request.
138+
154−
139+- `++receive-auth-response` is a function that takes an httr and produces a
155−- `++receive-auth-query-string` is a function that takes a `quay`
140+ `sec-move`. `%eyre` calls it when it gets a response to the request made in
156− (list of query parameters) and produces a `sec-move`. `%eyre`
141+ `++receive-auth-query-string`.
157− calls it when it receives a request on the callback url.
142+
158− Generally, this happens in OAuth after the user has granted
143+- `++update` is a function which converts old state to a new format when the
159− access to the urbit app, and the service makes a request to the
144+ security driver gets updated. If you want to just get rid of the old state,
160− callback url with the code. The security driver should then
145+ define `++discard-state` as `~`.
161− make a request to convert the code into an access token.
146+
162−
147+For basic auth, we only have to worry about `++filter-request`, since all we
163−- `++receive-auth-response` is a function that takes an httr and
148+need to do is add the correct header to each request. Our `++aut` core contains
164− produces a `sec-move`. `%eyre` calls it when it gets a
149+a function `++out-addding-header`, which does exactly what we want.
165− response to the request made in `++receive-auth-query-string`.
150+
166−
151+Where do the keys come from, though? Remember we ran `|init-auth-basic` to input
167−- `++update` is a function which converts old state to a new
152+them. This just prompts you for the service url, username, and password, and it
168− format when the security driver gets updated. If you want to
153+stores them next to the driver. In our case, that's
169− just get rid of the old state, define `++discard-state` as `~`.
154+`/=home=/sec/com/gitub/api/atom`. These keys are stored encrypted, and you don't
170−
155+want to edit them directly. `%eyre` loads them directly into your `bale`.
171−For basic auth, we only have to worry about `++filter-request`,
172−since all we need to do is add the correct header to each
173−request. Our `++aut` core contains a function
174−`++out-addding-header`, which does exactly what we want.
175−
176−Where do the keys come from, though? Remember we ran
177−`|init-auth-basic` to input them. This just prompts you for the
178−service url, username, and password, and it stores them next to
179−the driver. In our case, that's
180−`/=home=/sec/com/gitub/api/atom`. These keys are stored
181−encrypted, and you don't want to edit them directly. `%eyre`
182−loads them directly into your `bale`.
183
156
184## OAuth2
157## OAuth2
185
158
186−Most services are better accessed through some form of OAuth.
159+Most services are better accessed through some form of OAuth. Github can be
187−Github can be accessed with OAuth2, which is a little more
160+accessed with OAuth2, which is a little more complicated than basic auth, but
188−complicated than basic auth, but not hugely so.
161+not hugely so.
189
162
190The basic steps for OAuth2 are these:
163The basic steps for OAuth2 are these:
191
164
192−- create an app and get its client id and client secret (on
165+- create an app and get its client id and client secret (on Github)
193− Github)
166+- store these in Urbit (with `|init-oauth2`)
194−- store these in Urbit (with `|init-oauth2`)
167+- try to make a request that requires authentication. The security driver
195−- try to make a request that requires authentication. The
168+ should prompt you to visit a particular url.
196− security driver should prompt you to visit a particular url.
169+- visit the url and click "authorize", which gives the app access to your
197−- visit the url and click "authorize", which gives the app access
170+ account
198− to your account
171+- the security driver will turn the code it receives into an access token
199−- the security driver will turn the code it receives into an
172+- on every request, include that access token
200− access token
201−- on every request, include that access token
202
173
203Creating an app is easy and
174Creating an app is easy and
204−[well-documented](https://github.com/settings/applications/new).
175+[well-documented](https://github.com/settings/applications/new). If necessary,
205−If necessary, set the callback url to
176+set the callback url to `/~/ac/github.com/_state/in`, and note its client id and
206−`/~/ac/github.com/_state/in`, and note its client id and client
207−secret.
208−
209−Run `|init-oauth2`, which will prompt for the hostname
210−(api.github.com), client id, and client secret. This will store
211−the information, encrypted, in `/=home=/sec/com/github.atom`,
212−just as with Basic Authentication.
213−
214−Now we need a security driver. Use this:
215−
216−```
217−:: Test url +https://api.github.com/user
218−::
219−:::: /hoon/github/com/sec
220− ::
221−/+ oauth2
222−!:
223−::::
224− ::
225−|_ {bal/(bale keys:oauth2) tok/token:oauth2}
226−++ scopes :: comment out scopes to taste
227− :~ 'user' 'user:email' 'user:follow' 'public_repo' 'repo'
228− 'repo_deployment' 'repo:status' 'delete_repo' 'notifications'
229− 'gist' 'read:repo_hook' 'write:repo_hook' 'admin:repo_hook'
230− 'admin:org_hook' 'read:org' 'write:org' 'admin:org'
231− 'read:public_key' 'write:public_key' 'admin:public_key'
232− ==
233−:: ++aut is a "standard oauth2" core, which implements the
234−:: most common handling of oauth2 semantics. see lib/oauth2 for more details,
235−:: and examples at the bottom of the file.
236−++ aut (~(standard oauth2 bal tok) . |=(tok/token:oauth2 +>(tok tok)))
237−++ filter-request
238− %^ out-add-query-param:aut 'access_token'
239− scope=~[%client %admin]
240− oauth-dialog='https://github.com/login/oauth/authorize'
241−::
242−++ receive-auth-query-string
243− %- in-code-to-token:aut
244− url='https://github.com/login/oauth/api/access_token'
245−++ receive-auth-response bak-save-token:aut
246−--
247−```
248−
249−The oauth2 library provides the main "engine" in `standard`, just
250−like in basic auth, except that we also have to specify a
251−function to save the access token when we get it. It's worth
252−noting that this library is well documented in the source,
253−including examples: `/=home=/lib/oauth2/hoon`.
254−
255−Running `+http://api.github.com/user` tries to make a request to
256−Github with authentication. This loads into the bale the keys,
257−which are of type `keys:oauth2`, defined in the oauth2 library.
258−We don't have to worry about the specifics of these keys -- the
259−library handles them -- but they include the client id and the
260client secret.
177client secret.
261
178
262−Before the request is sent, `%eyre` calls `++filter-request` to
179+Run `|init-oauth2`, which will prompt for the hostname (api.github.com), client
263−decorate it with authentication headers. Since we don't yet have
180+id, and client secret. This will store the information, encrypted, in
264−an access token, we need to prompt the user to visit the "dialog"
181+`/=home=/sec/com/github.atom`, just as with Basic Authentication.
265−url. `++out-add-query-param` checks whether we have a token, and
182+
266−if we don't it produces `%show` and a message telling the user to
183+Now we need a security driver. Use this:
267−visit the `oauth-dialog` url we provide, with the extra query
184+
268−parameters added (client id, scopes, and state).
185+ :: Test url +https://api.github.com/user
269−
186+ ::
270−When the user visits that url, Github will ask them to log in and
187+ :::: /hoon/github/com/sec
271−authorize the application. When they do so, Github will post a
188+ ::
272−request to `/~/ac/github.com/_state/in` with a code in the query
189+ /+ oauth2
273−string. `%eyre` sends the query string to
190+ !:
274−`++receive-auth-query-string`, which is a function that takes a
191+ ::::
275−quay and produces a `sec-move`. `++in-code-to-token` from our
192+ ::
276−`aut` takes the `code` parameter from the quay and uses it to
193+ |_ {bal/(bale keys:oauth2) tok/token:oauth2}
277−create a request to the given url. This request includes the
194+ ++ scopes :: comment out scopes to taste
278−client id, the client secret, and the code we just got.
195+ :~ 'user' 'user:email' 'user:follow' 'public_repo' 'repo'
279−
196+ 'repo_deployment' 'repo:status' 'delete_repo' 'notifications'
280−Github's response to that request includes an access token in its
197+ 'gist' 'read:repo_hook' 'write:repo_hook' 'admin:repo_hook'
281−body. This is handled in `++receive-auth-response`.
198+ 'admin:org_hook' 'read:org' 'write:org' 'admin:org'
282−`++bak-save-token` extracts that token and gives it to the
199+ 'read:public_key' 'write:public_key' 'admin:public_key'
283−function that we passed in the definition of `++aut`. In our
200+ ==
201+ :: ++aut is a "standard oauth2" core, which implements the
202+ :: most common handling of oauth2 semantics. see lib/oauth2 for more details,
203+ :: and examples at the bottom of the file.
204+ ++ aut (~(standard oauth2 bal tok) . |=(tok/token:oauth2 +>(tok tok)))
205+ ++ filter-request
206+ %^ out-add-query-param:aut 'access_token'
207+ scope=~[%client %admin]
208+ oauth-dialog='https://github.com/login/oauth/authorize'
209+ ::
210+ ++ receive-auth-query-string
211+ %- in-code-to-token:aut
212+ url='https://github.com/login/oauth/api/access_token'
213+ ++ receive-auth-response bak-save-token:aut
214+ --
215+
216+The oauth2 library provides the main "engine" in `standard`, just like in basic
217+auth, except that we also have to specify a function to save the access token
218+when we get it. It's worth noting that this library is well documented in the
219+source, including examples: `/=home=/lib/oauth2/hoon`.
220+
221+Running `+http://api.github.com/user` tries to make a request to Github with
222+authentication. This loads into the bale the keys, which are of type
223+`keys:oauth2`, defined in the oauth2 library. We don't have to worry about the
224+specifics of these keys -- the library handles them -- but they include the
225+client id and the client secret.
226+
227+Before the request is sent, `%eyre` calls `++filter-request` to decorate it with
228+authentication headers. Since we don't yet have an access token, we need to
229+prompt the user to visit the "dialog" url. `++out-add-query-param` checks
230+whether we have a token, and if we don't it produces `%show` and a message
231+telling the user to visit the `oauth-dialog` url we provide, with the extra
232+query parameters added (client id, scopes, and state).
233+
234+When the user visits that url, Github will ask them to log in and authorize the
235+application. When they do so, Github will post a request to
236+`/~/ac/github.com/_state/in` with a code in the query string. `%eyre` sends the
237+query string to `++receive-auth-query-string`, which is a function that takes a
238+quay and produces a `sec-move`. `++in-code-to-token` from our `aut` takes the
239+`code` parameter from the quay and uses it to create a request to the given url.
240+This request includes the client id, the client secret, and the code we just
241+got.
242+
243+Github's response to that request includes an access token in its body. This is
244+handled in `++receive-auth-response`. `++bak-save-token` extracts that token and
245+gives it to the function that we passed in the definition of `++aut`. In our
284case, we just save that token into our state for later usage.
246case, we just save that token into our state for later usage.
285
247
286−Now we have the credentials we need, so `++filter-request` is
248+Now we have the credentials we need, so `++filter-request` is called with the
287−called with the original request, and `++out-add-query-param`
249+original request, and `++out-add-query-param` passes the request through with
288−passes the request through with the addition of the query
250+the addition of the query parameter `access_token`.
289−parameter `access_token`.
290
251
291Just like with Basic Authentication, you should be able to run
252Just like with Basic Authentication, you should be able to run
292−`+https://api.github.com/user` and get a response indicating who
253+`+https://api.github.com/user` and get a response indicating who you're logged
293−you're logged in as.
254+in as.
294
255
295−Some drivers a slightly more complicated. For example, Github's
256+Some drivers a slightly more complicated. For example, Github's access token's
296−access token's don't expire, but that's not the case for all
257+don't expire, but that's not the case for all service. The googleapis driver
297−service. The googleapis driver shows the flow when the access
258+shows the flow when the access token can expire. Twitter uses OAuth1, for which
298−token can expire. Twitter uses OAuth1, for which we have another
259+we have another library.
299−library.
300
260
301−Remember, of course, that the best strategy for building security
261+Remember, of course, that the best strategy for building security drivers is to
302−drivers is to copy a similar one and tweak it to taste.
262+copy a similar one and tweak it to taste.
docs/arvo/state.md+62−69
@@ -1,18 +1,16 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: true
3+next: True
4sort: 14
4sort: 14
5title: State
5title: State
6---
6---
7
7
8−State
8+# State
9−=====
10
9
11−In the last section we built a few small apps that sent moves. These
10+In the last section we built a few small apps that sent moves. These apps were
12−apps were entirely stateless, however. Most useful apps require some
11+entirely stateless, however. Most useful apps require some amount of state.
13−amount of state. Let's build a trivial stateful app. It'll keep a
12+Let's build a trivial stateful app. It'll keep a running the sum of all the
14−running the sum of all the atoms we poke it with. Here's
13+atoms we poke it with. Here's `app/examples/sum.hoon`:
15−`app/examples/sum.hoon`:
16
14
17 /? 314
15 /? 314
18 !:
16 !:
@@ -38,72 +36,67 @@ We can start it with `|start %examples-sum`, and then run it:
38
36
39We can see that app state is being saved, but when, where, and how?
37We can see that app state is being saved, but when, where, and how?
40
38
41−The state is stored as the second thing in the `|_` line. In our case,
39+The state is stored as the second thing in the `|_` line. In our case, it's
42−it's simply an atom named `state`. We change it by producing our
40+simply an atom named `state`. We change it by producing our state not with
43−state not with `+>.$` (as before), but with
41+`+>.$` (as before), but with `+>.$(state (add state arg))`. We've seen all these
44−`+>.$(state (add state arg))`. We've seen all these parts before, but
42+parts before, but you might not recognize them.
45−you might not recognize them.
46
43
47Recall in the first chapter that we recursed with the expression
44Recall in the first chapter that we recursed with the expression
48−`$(b (add 3 b))`. This meant "produce `$` with `b` changed to
45+`$(b (add 3 b))`. This meant "produce `$` with `b` changed to `(add 3 b)`.
49−`(add 3 b)`. Similarly, `+>.$(state (add state arg))` means "produce
46+Similarly, `+>.$(state (add state arg))` means "produce `+>.$` (i.e. our
50−`+>.$` (i.e. our context, which contains our state) with `state` changed
47+context, which contains our state) with `state` changed to `(add state arg)`.
51−to `(add state arg)`.
48+
52−
49+At a high level, then, when we handle state, we do it explicitly. It's passed in
53−At a high level, then, when we handle state, we do it explicitly. It's
50+and produced explicitly. In Unix systems, application state is just a block of
54−passed in and produced explicitly. In Unix systems, application state is
51+memory, which you need to serialize to disk if you want to keep it around for
55−just a block of memory, which you need to serialize to disk if you want
52+very long.
56−to keep it around for very long.
53+
57−
54+In Urbit, app state is a single (usually complex) value. In our example, we have
58−In Urbit, app state is a single (usually complex) value. In our example,
55+very simple state, so we defined `state/@`, meaning that our state is an atom.
59−we have very simple state, so we defined `state/@`, meaning that our
56+Of course, `state` is just a name, and you're free to name your state whatever
60−state is an atom. Of course, `state` is just a name, and you're free to
57+you like. But let's clarify a couple other things before we continue.
61−name your state whatever you like. But let's clarify a couple other
58+
62−things before we continue.
59+First, `bowl` is a set of general global states. This set is managed by the
63−
60+system. It includes things like `now` (the current time), `our` (our urbit
64−First, `bowl` is a set of general global states. This set is managed by
61+identity), and `eny` (256 bits of guaranteed-fresh entropy). For the full list
65−the system. It includes things like `now` (the current time), `our` (our
62+of things in `++bowl`, search for `++ bowl` (note: two spaces) in
66−urbit identity), and `eny` (256 bits of guaranteed-fresh entropy). For
63+`/arvo/zuse.hoon`.
67−the full list of things in `++bowl`, search for `++ bowl` (note: two
64+
68−spaces) in `/arvo/zuse.hoon`.
65+> This is, perhaps, the most common way to learn Hoon. The easiest way to learn
69−
66+> about an identifier you see in code is to search in `/arvo/zuse.hoon` and
70−> This is, perhaps, the most common way to learn Hoon. The easiest way
67+> `/arvo/hoon.hoon` for it.\\ These "standard libraries" are usually simple to
71−> to learn about an identifier you see in code is to search in
68+> read. Since Urbit's codebase is less than 15000 lines of code combined,
72−> `/arvo/zuse.hoon` and `/arvo/hoon.hoon` for it.\
69+> including the hoon parser, the compiler, and the `/arvo` microkernel, you can
73−> These "standard libraries" are usually simple to read. Since Urbit's
70+> usually use the code and its comments as a reference doc.
74−> codebase is less than 15000 lines of code combined, including the hoon
75−> parser, the compiler, and the `/arvo` microkernel, you can usually use
76−> the code and its comments as a reference doc.
77>
71>
78> You can also read
72> You can also read
79−> [zuse.hoon](https://github.com/urbit/arvo/blob/master/arvo/zuse.hoon)
73+> [zuse.hoon](https://github.com/urbit/arvo/blob/master/arvo/zuse.hoon) and
80−> and
74+> [hoon.hoon](https://github.com/urbit/arvo/blob/master/arvo/hoon.hoon) in your
81−> [hoon.hoon](https://github.com/urbit/arvo/blob/master/arvo/hoon.hoon)
75+> browser.
82−> in your browser.
76+
83−
77+The second thing we should clear up is this: Urbit needs no "serialize to disk"
84−The second thing we should clear up is this: Urbit needs no "serialize
78+step. Everything you produce in the app state is persistent across calls to the
85−to disk" step. Everything you produce in the app state is persistent
79+app, restarts of the urbit, and even power failure. If you want to write to the
86−across calls to the app, restarts of the urbit, and even power failure.
80+Unix filesystem, you can, but it's not needed for persistence. Urbit has
87−If you want to write to the Unix filesystem, you can, but it's not
81+transactional events, which makes it an [ACID operating
88−needed for persistence. Urbit has transactional events, which makes it
82+system](https://en.wikipedia.org/wiki/ACID). Thus, you don't have to worry about
89−an [ACID operating system](https://en.wikipedia.org/wiki/ACID). Thus,
83+persistence when programming in Urbit, or ever go through the hassle of having
90−you don't have to worry about persistence when programming in Urbit, or
84+to set up and write to a database.
91−ever go through the hassle of having to set up and write to a database.
92
85
93**Exercises**:
86**Exercises**:
94
87
95- Modify `:examples-sum` to reset the counter when you poke it with 0.
88- Modify `:examples-sum` to reset the counter when you poke it with 0.
96
89
97−- Write an app that prints out the previous value you poked it with.
90+- Write an app that prints out the previous value you poked it with. Sample
98− Sample output:
91+ output:
99−
92+
100−```
93+<!-- -->
101−~fintud-macrep:dojo> :examples-last 7
94+
102−[%last 0]
95+ ~fintud-macrep:dojo> :examples-last 7
103−>=
96+ [%last 0]
104−~fintud-macrep:dojo> :examples-last [1 2 3]
97+ >=
105−[%last 7]
98+ ~fintud-macrep:dojo> :examples-last [1 2 3]
106−>=
99+ [%last 7]
107−~fintud-macrep:dojo> :examples-last 'howdy'
100+ >=
108−[%last [1 2 3]]
101+ ~fintud-macrep:dojo> :examples-last 'howdy'
109−```
102+ [%last [1 2 3]]
docs/arvo/subject.md+36−43
@@ -1,12 +1,11 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3+next: True
3sort: 12
4sort: 12
4−next: true
5title: The subject
5title: The subject
6---
6---
7
7
8−The subject
8+# The subject
9−===========
10
9
11Now we're going to cover the boiler plate that we skimmed over earlier.
10Now we're going to cover the boiler plate that we skimmed over earlier.
12
11
@@ -14,26 +13,24 @@ Now we're going to cover the boiler plate that we skimmed over earlier.
14 :- %noun
13 :- %noun
15 =< (sum [1.000 2.000])
14 =< (sum [1.000 2.000])
16
15
17−The first rune, `:-` (colhep, aka
16+The first rune, `:-` (colhep, aka [:cons](../../hoon/twig/col-cell/hep-cons/)),
18−[:cons](../../hoon/twig/col-cell/hep-cons/)), constructs the 2-element cell
17+constructs the 2-element cell that will be our program. The first element,
19−that will be our program. The first element, `%say`, tells the
18+`%say`, tells the interpreter what to produce--in this case a value.
20−interpreter what to produce--in this case a value.
21
19
22−The second element is `|=`, which we know produces a function. `|=`'s
20+The second element is `|=`, which we know produces a function. `|=`'s first
23−first child is its argument(s), which in this case is any noun (`*`).
21+child is its argument(s), which in this case is any noun (`*`). Its second child
24−Its second child is the remainder of the program.
22+is the remainder of the program.
25
23
26−Similarly, the rest of the program (which we construct with another
24+Similarly, the rest of the program (which we construct with another `:-`) is a
27−`:-`) is a cell of the literal `%noun`, which tells the shell that we're
25+cell of the literal `%noun`, which tells the shell that we're producing a value
28−producing a value of type `noun`, and the code that we run to actually
26+of type `noun`, and the code that we run to actually produce our value of the
29−produce our value of the type `noun`.
27+type `noun`.
30
28
31`=<` ([tisgal](../../hoon/twig/tis-flow/gal-rap/)) is a rune that takes two
29`=<` ([tisgal](../../hoon/twig/tis-flow/gal-rap/)) is a rune that takes two
32−children. The second child is the context against which we run the first
30+children. The second child is the context against which we run the first child.
33−child. So in this case, we are running the expression
31+So in this case, we are running the expression `(sum [1.000 2.000])` against
34−`(sum [1.000 2.000])` against everything contained within the `|%`. In
32+everything contained within the `|%`. In Hoon, we call the code executed the
35−Hoon, we call the code executed the "formula" and its context the
33+"formula" and its context the "subject".
36−"subject".
37
34
38 ::::::::::::::::::::::::::::::
35 ::::::::::::::::::::::::::::::
39 =< (sum [1.000 2.000]) :: formula
36 =< (sum [1.000 2.000]) :: formula
@@ -63,39 +60,35 @@ Hoon, we call the code executed the "formula" and its context the
63 -- ::
60 -- ::
64 ::::::::::::::::::::::::::::::
61 ::::::::::::::::::::::::::::::
65
62
66−In nearly every language there is a similar concept of a "context" in
63+In nearly every language there is a similar concept of a "context" in which
67−which expressions are executed. For example, in C this includes things
64+expressions are executed. For example, in C this includes things like the call
68−like the call stack, stack variables, and so on.
65+stack, stack variables, and so on.
69
66
70−Hoon is unique in that this context is a first-class value. Scheme
67+Hoon is unique in that this context is a first-class value. Scheme allows a sort
71−allows a sort of reification of the context through continuations, and
68+of reification of the context through continuations, and some may see a parallel
72−some may see a parallel to Forth's stack, but Hoon takes the
69+to Forth's stack, but Hoon takes the concept one step further.
73−concept one step further.
74
70
75Our starting subject is the [standard library](../../hoon/library), which is
71Our starting subject is the [standard library](../../hoon/library), which is
76−defined in `/arvo/hoon.hoon` and `/arvo/zuse.hoon`. This is where
72+defined in `/arvo/hoon.hoon` and `/arvo/zuse.hoon`. This is where functions like
77−functions like `add` are defined. When we define a core with `|%`, we
73+`add` are defined. When we define a core with `|%`, we don't throw away the
78−don't throw away the subject (i.e. the standard library); rather, we
74+subject (i.e. the standard library); rather, we stack the new core on top of the
79−stack the new core on top of the old subject so that both are
75+old subject so that both are accessible.
80−accessible.
81
76
82**Exercises**:
77**Exercises**:
83
78
84−- Pass `++sum` its arguments (`2.000` and `3.000`) from the
79+- Pass `++sum` its arguments (`2.000` and `3.000`) from the command line.
85− command line.
86
80
87−- Comment out all of the arms of the `|%`. Now add another arm and
81+- Comment out all of the arms of the `|%`. Now add another arm and call it
88− call it `++add`, have it accept two arguments and produce 42
82+ `++add`, have it accept two arguments and produce 42 (regardless of input).
89− (regardless of input). Change the `=<` line to
83+ Change the `=<` line to `[(add 5 7) (^add 5 7)]`. Can you recognize what's
90− `[(add 5 7) (^add 5 7)]`. Can you recognize what's happening?
84+ happening?
91
85
92Cheatsheet:
86Cheatsheet:
93
87
94−- To pass arguments from the command line to a program, you replace
88+- To pass arguments from the command line to a program, you replace the `*` in
95− the `*` in the first line of the boiler plate to
89+ the first line of the boiler plate to `{^ {{arg/TYPE $~} $~}}` where `TYPE`
96− `{^ {{arg/TYPE $~} $~}}` where `TYPE` is replaced with the type of
90+ is replaced with the type of argument you're expecting. Then `+euler1 a`
97− argument you're expecting. Then `+euler1 a` from the dojo sets `arg`
91+ from the dojo sets `arg` to `a`.
98− to `a`.
99- The empty list is `~`
92- The empty list is `~`
100- Lisp-style cons (construct a cell/prepend an element) is
93- Lisp-style cons (construct a cell/prepend an element) is
101 `[new-element list]`
94 `[new-element list]`
docs/arvo/subscriptions.md+116−129
@@ -1,20 +1,19 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: true
3+next: True
4sort: 15
4sort: 15
5title: Subscriptions
5title: Subscriptions
6---
6---
7
7
8# Subscriptions
8# Subscriptions
9
9
10−We've dealt fairly extensively with "poke" messages to an app, but these
10+We've dealt fairly extensively with "poke" messages to an app, but these are
11−are somewhat limited. A poke is a one-way message, but more often we
11+somewhat limited. A poke is a one-way message, but more often we want to
12−want to subscribe to updates from another app. You could build a
12+subscribe to updates from another app. You could build a subscription model out
13−subscription model out of one-way pokes, but it's such a common pattern
13+of one-way pokes, but it's such a common pattern that it's built into arvo.
14−that it's built into arvo.
15
14
16−Let's take a look at two apps, `:examples-source` and `:examples-sink`.
15+Let's take a look at two apps, `:examples-source` and `:examples-sink`. First,
17−First, `:examples-source`:
16+`:examples-source`:
18
17
19 /? 314
18 /? 314
20 !:
19 !:
@@ -73,54 +72,50 @@ And secondly, `:examples-sink`:
73
72
74Cheat sheet:
73Cheat sheet:
75
74
76−- `&` (pam) can either be the boolean true (as can `%.y`, `0`), or
75+- `&` (pam) can either be the boolean true (as can `%.y`, `0`), or the
77− the irregular wide form of the `?&`
76+ irregular wide form of the `?&` ([wutpam](../../hoon/twig/wut-test/pam-and))
78− ([wutpam](../../hoon/twig/wut-test/pam-and)) rune, which computes
77+ rune, which computes logical `AND` on its two children.
79− logical `AND` on its two children.
80
78
81−- Similar to `&`,`|` is either the boolean false (along with `%.n` and
79+- Similar to `&`,`|` is either the boolean false (along with `%.n` and `1`),
82− `1`), or the irregular short for of `?|`
80+ or the irregular short for of `?|`
83− ([wutbar](../../hoon/twig/wut-test/bar-or)), which computes logical `OR`
81+ ([wutbar](../../hoon/twig/wut-test/bar-or)), which computes logical `OR` on
84− on its two children.
82+ its two children.
85
83
86- `!` is the irregular wide form of `?!`
84- `!` is the irregular wide form of `?!`
87− ([wutzap](../../hoon/twig/wut-test/zap-not/)), which computes logical
85+ ([wutzap](../../hoon/twig/wut-test/zap-not/)), which computes logical `NOT`
88− `NOT` on its child.
86+ on its child.
89
87
90- `?~` ([wutsig](../../hoon/twig/wut-test/sig-ifno/)) is basically an
88- `?~` ([wutsig](../../hoon/twig/wut-test/sig-ifno/)) is basically an
91− if-then-else that checks whether condition `p` is `~` (null). `?~`
89+ if-then-else that checks whether condition `p` is `~` (null). `?~` is
92− is slightly different from `?:(~ %tru %fal)` in that `?~` reduces to
90+ slightly different from `?:(~ %tru %fal)` in that `?~` reduces to
93 `?:($=(%type value) %tru %false)`. `$=`
91 `?:($=(%type value) %tru %false)`. `$=`
94− ([buctis](../../hoon/twig/buc-mold/tis-coat/)) tests whether value `q` is
92+ ([buctis](../../hoon/twig/buc-mold/tis-coat/)) tests whether value `q` is of
95− of type `p`.
93+ type `p`. <!-- One thing to watch out for in hoon: if you do `?~`, it
96−<!-- One thing to watch out for in hoon: if you do `?~`, it
94+ affects the type of the conditional value: XXexample -->
97− affects the type of the conditional value: XXexample -->
98
95
99- `:_` ([colcab](../../hoon/twig/col-cell/cab-scon/)) is inverted `:-`: it
96- `:_` ([colcab](../../hoon/twig/col-cell/cab-scon/)) is inverted `:-`: it
100 accepts `p` and `q`, and produces `[q p]`.
97 accepts `p` and `q`, and produces `[q p]`.
101
98
102−- `++bowl` is the type of the system state within our app. For
99+- `++bowl` is the type of the system state within our app. For example, it
103− example, it includes things like `our`, the name of the host urbit,
100+ includes things like `our`, the name of the host urbit, and `now`, the
104− and `now`, the current time.
101+ current time.
105
102
106−- `$%` ([buccen](../../hoon/twig/buc-mold/cen-book/)) is a type
103+- `$%` ([buccen](../../hoon/twig/buc-mold/cen-book/)) is a type constructor:
107− constructor: it defines a new type, composed of `n` types that it is
104+ it defines a new type, composed of `n` types that it is passed. For example
108− passed. For example `$% @ * ^ ==` is the type of either `@`,
105+ `$% @ * ^ ==` is the type of either `@`, `*`, or a cell `^`.
109− `*`, or a cell `^`.
106+ <!-- XX this is a union, right? -->
110−<!-- XX this is a union, right? -->
111
107
112- You may have noticed the separate `|%`
108- You may have noticed the separate `|%`
113 ([barcen](../../hoon/twig/bar-core/cen-core/)) above the application core
109 ([barcen](../../hoon/twig/bar-core/cen-core/)) above the application core
114− `|_` ([barcab](../../hoon/twig/bar-core/cab-door/). We usually put our
110+ `|_` ([barcab](../../hoon/twig/bar-core/cab-door/). We usually put our types
115− types in another core on top of the application core. We can access
111+ in another core on top of the application core. We can access these type
116− these type from our `|_` because in `hoon.hoon` files, all cores are
112+ from our `|_` because in `hoon.hoon` files, all cores are called against
117− called against each other. (The shorthand for 'called' is `=>`.)
113+ each other. (The shorthand for 'called' is `=>`.) Thus, the `|%` with the
118− Thus, the `|%` with the types is in the context of the `|_`, as it
114+ types is in the context of the `|_`, as it lies above it: `hoon.hoon`
119− lies above it: `hoon.hoon` `=> |% w types => |_`
115+ `=> |% w types => |_`
120
116
121Here's some sample output of the two working together:
117Here's some sample output of the two working together:
122
118
123−
124 ~fintud-macrep:dojo> |start %examples-source
119 ~fintud-macrep:dojo> |start %examples-source
125 >=
120 >=
126 ~fintud-macrep:dojo> |start %examples-sink
121 ~fintud-macrep:dojo> |start %examples-sink
@@ -147,123 +142,115 @@ Here's some sample output of the two working together:
147### :examples-source
142### :examples-source
148
143
149Hopefully you can get a sense for what's happening here. When we poke
144Hopefully you can get a sense for what's happening here. When we poke
150−`:examples-sink` with `%on`, `:examples-sink` subscribes to
145+`:examples-sink` with `%on`, `:examples-sink` subscribes to `:examples-source`,
151−`:examples-source`, and so whenever we poke `:examples-source`,
146+and so whenever we poke `:examples-source`, `:examples-sink` gets the update and
152−`:examples-sink` gets the update and prints it out. Then we unsubscribe
147+prints it out. Then we unsubscribe by poking `:examples-sink` with `%off`, and
153−by poking `:examples-sink` with `%off`, and `:examples-sink` stops
148+`:examples-sink` stops getting updates. We then resubscribe.
154−getting updates. We then resubscribe.
155
149
156−There's a fair bit going on in this code. Let's look at
150+There's a fair bit going on in this code. Let's look at `:examples-source`
157−`:examples-source` first.
151+first.
158
152
159−Our definition of `move` is fairly specific, since we're only going to
153+Our definition of `move` is fairly specific, since we're only going to sending
160−sending one kind of move. The `%diff` move is a subscription update, and
154+one kind of move. The `%diff` move is a subscription update, and its content is
161−its content is marked data which gall routes to our subscribers.
155+marked data which gall routes to our subscribers.
162
156
163This is a slightly different kind of move than [we've dealt with so
157This is a slightly different kind of move than [we've dealt with so
164−far](/arvo/network). It's producing a result rather than calling other
158+far](/arvo/network). It's producing a result rather than calling other code
165−code (i.e. it's a return rather than a function call), so if you recall
159+(i.e. it's a return rather than a function call), so if you recall the
166−the discussion of ducts, a layer gets popped off the duct rather than
160+discussion of ducts, a layer gets popped off the duct rather than added to it.
167−added to it. This is why no wire is needed for the move -- we won't
161+This is why no wire is needed for the move -- we won't receive anything in
168−receive anything in response to it.
162+response to it.
169
163
170Anyways, there're two functions inside the `|_`. We already know when
164Anyways, there're two functions inside the `|_`. We already know when
171−`++poke-noun` is called. `++peer` is called when someone tries to
165+`++poke-noun` is called. `++peer` is called when someone tries to subscribe to
172−subscribe to our app. Of course, you don't just subscribe to an app; you
166+our app. Of course, you don't just subscribe to an app; you subscribe to a path
173−subscribe to a path on that app. This path comes in as the argument to
167+on that app. This path comes in as the argument to `++peer`.
174−`++peer`.
168+
175−
169+In our case, we don't care what path you subscribed on, and all we do is print
176−In our case, we don't care what path you subscribed on, and all we do is
170+out that you subscribed. Arvo keeps track of your subscriptions, so you don't
177−print out that you subscribed. Arvo keeps track of your subscriptions,
171+have to. You can access your subscribers by looking at `sup` in the bowl that's
178−so you don't have to. You can access your subscribers by looking at
172+passed in. `sup` is of type `(map bone {@p path})`, which associates bones with
179−`sup` in the bowl that's passed in. `sup` is of type
173+the urbit who subscribed, and which path they subscribed on. If you want to
180−`(map bone {@p path})`, which associates bones with the urbit who
181−subscribed, and which path they subscribed on. If you want to
182communicate with your subscribers, send them messages along their bone.
174communicate with your subscribers, send them messages along their bone.
183
175
184−`++poke-noun` "spams" the given argument to all our subscribers.
176+`++poke-noun` "spams" the given argument to all our subscribers. There's a few
185−There's a few things we haven't seen before. Firstly, `:_(a b)` is the
177+things we haven't seen before. Firstly, `:_(a b)` is the same as `[b a]`. It's
186−same as `[b a]`. It's just a convenient way of formatting things when
178+just a convenient way of formatting things when the first thing in a cell is
187−the first thing in a cell is much more complicated than the second.
179+much more complicated than the second. Thus, we're producing our state
188−Thus, we're producing our state unchanged.
180+unchanged.
189
181
190−Our list of moves is the result of a call to `++turn`. `++turn` is what
182+Our list of moves is the result of a call to `++turn`. `++turn` is what many
191−many languages call "map" -- it runs a function on every item in a list
183+languages call "map" -- it runs a function on every item in a list and collects
192−and collects the results in a list. The list is `(prey /example-path hid)`
184+the results in a list. The list is `(prey /example-path hid)` and the function
193−and the function is the `|=` line right after it.
185+is the `|=` line right after it.
194
186
195−`++prey` is a standard library function defined in `zuse`. It takes a
187+`++prey` is a standard library function defined in `zuse`. It takes a path and a
196−path and a bowl and gives you a list of the subscribers who are
188+bowl and gives you a list of the subscribers who are subscribed on a path that
197−subscribed on a path that begins with the given path. "Prey" is short
189+begins with the given path. "Prey" is short for "prefix".
198−for "prefix".
199
190
200Now we have the list of relevant subscribers. This a list of triples,
191Now we have the list of relevant subscribers. This a list of triples,
201−`{bone @p path}`, where the only thing we really need is the bone,
192+`{bone @p path}`, where the only thing we really need is the bone, because we
202−because we don't need to know their urbit or what exact path they
193+don't need to know their urbit or what exact path they subscribed on. Thus, our
203−subscribed on. Thus, our transformer function takes `{o/bone *}` and
194+transformer function takes `{o/bone *}` and produces `[o %diff %noun arg]`,
204−produces `[o %diff %noun arg]`, which is a move that provides bone `o`
195+which is a move that provides bone `o` with this subscription update:
205−with this subscription update: `[%noun arg]`". This is fairly dense
196+`[%noun arg]`". This is fairly dense code, but what it's doing is
206−code, but what it's doing is straightforward.
197+straightforward.
207
198
208### :examples-sink
199### :examples-sink
209
200
210−`:examples-source` should now make sense. `:examples-sink` is a little
201+`:examples-source` should now make sense. `:examples-sink` is a little longer,
211−longer, but not much more complicated.
202+but not much more complicated.
212−
203+
213−In `:examples-sink`, our definition of of `++move` is different. All
204+In `:examples-sink`, our definition of of `++move` is different. All moves start
214−moves start with a `bone`, and we conventionally refer to the second
205+with a `bone`, and we conventionally refer to the second half as the "card", so
215−half as the "card", so that we can say a move is an action that sends a
206+that we can say a move is an action that sends a card along a bone.
216−card along a bone.
207+
217−
208+We have two kinds of cards here: we `%peer` to start a subscription, and we
218−We have two kinds of cards here: we `%peer` to start a subscription, and
209+`%pull` to stop it. Both of these are "forward" moves that may receive a
219−we `%pull` to stop it. Both of these are "forward" moves that may
210+response, so they need a wire to tack onto the duct before they pass it on. They
220−receive a response, so they need a wire to tack onto the duct before
211+also need a target, which is a pair of an urbit and an app name. Additionally,
221−they pass it on. They also need a target, which is a pair of an urbit
212+`%peer` needs a path on that app to subscribe too. `%pull` doesn't need this,
222−and an app name. Additionally, `%peer` needs a path on that app to
213+because its semantics are to cancel any subscriptions coming over this duct. If
223−subscribe too. `%pull` doesn't need this, because its semantics are to
214+your bone and wire are the same as when you subscribed, then the cancellation
224−cancel any subscriptions coming over this duct. If your bone and wire
215+will happen correctly.
225−are the same as when you subscribed, then the cancellation will happen
216+
226−correctly.
217+The only state we need for `:examples-sink` is a boolean to indicate whether
227−
218+we're already subscribed to `:examples-source`. We use `available/?`, where `?`
228−The only state we need for `:examples-sink` is a boolean to indicate
219+is the sign of type boolean (similar to `*`, `@`), which defaults to true (that
229−whether we're already subscribed to `:examples-source`. We use
220+is, `0`).
230−`available/?`, where `?` is the sign of type boolean (similar to `*`,
231−`@`), which defaults to true (that is, `0`).
232
221
233In `++poke-noun` we check our input to see both if it's `%on` and we're
222In `++poke-noun` we check our input to see both if it's `%on` and we're
234−available. If so, we produce the move to subscribe to
223+available. If so, we produce the move to subscribe to `:examples-source`:
235−`:examples-source`:
236
224
237 [ost %peer /subscribe [our %source] /example-path]
225 [ost %peer /subscribe [our %source] /example-path]
238
226
239Also, we set available to false (`|`) with `+>.$(available |)`.
227Also, we set available to false (`|`) with `+>.$(available |)`.
240
228
241−Otherwise, if our input is `%off` and we're already subscribed (i.e.
229+Otherwise, if our input is `%off` and we're already subscribed (i.e. `available`
242−`available` is false), then we unsubscribe from `:examples-source`:
230+is false), then we unsubscribe from `:examples-source`:
243
231
244 [ost %pull /subscribe [our %source] ~]
232 [ost %pull /subscribe [our %source] ~]
245
233
246−It's important to send over the same bone and wire (`/subscribe`) as the
234+It's important to send over the same bone and wire (`/subscribe`) as the one we
247−one we originally subscribed on.
235+originally subscribed on.
248
236
249We also set `available` to true (`&`).
237We also set `available` to true (`&`).
250
238
251−If neither of these cases are true, then we print our current
239+If neither of these cases are true, then we print our current subscription
252−subscription state.
240+state.
253
241
254−`++diff-noun` is called when we get a `%diff` update along a
242+`++diff-noun` is called when we get a `%diff` update along a subscription with a
255−subscription with a mark of `noun`. `++diff-noun` is given the wire that
243+mark of `noun`. `++diff-noun` is given the wire that we originally passed with
256−we originally passed with the `%peer` subscription request along and the
244+the `%peer` subscription request along and the data we got back. In our case we
257−data we got back. In our case we just print out the data.
245+just print out the data.
258
246
259`++reap` is called when we receive an acknowledgment as to whether the
247`++reap` is called when we receive an acknowledgment as to whether the
260−subscription was handled successfully. You can remember that `++reap` is
248+subscription was handled successfully. You can remember that `++reap` is the
261−the counterpart to `++peer` as it's pronounced like 'peer' backwards.
249+counterpart to `++peer` as it's pronounced like 'peer' backwards. Similarly,
262−Similarly, `coup` is similar to 'poke' backwards.
250+`coup` is similar to 'poke' backwards.
263−
251+
264−Moving forward, `++reap` is given the wire we attempted to subscribe
252+Moving forward, `++reap` is given the wire we attempted to subscribe over,
265−over, possibly along with an error message in cases of failure.
253+possibly along with an error message in cases of failure. `(unit type)` means
266−`(unit type)` means "either `~` or `[~ type]`, which means it's used
254+"either `~` or `[~ type]`, which means it's used like Haskell's "maybe" or C's
267−like Haskell's "maybe" or C's nullability. If `error` is `~`, then the
255+nullability. If `error` is `~`, then the subscription was successful and we tell
268−subscription was successful and we tell that to the user. Otherwise, we
256+that to the user. Otherwise, we print out the error message.
269−print out the error message.
docs/arvo/web-apps.md+212−229
@@ -1,265 +1,248 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−next: true
3+next: True
4sort: 17
4sort: 17
5title: Web apps
5title: Web apps
6---
6---
7
7
8# Backend
8# Backend
9
9
10−We've done purely functional web pages as hook files, and we've
10+We've done purely functional web pages as hook files, and we've done command
11−done command line apps, so now let's interact with an app over
11+line apps, so now let's interact with an app over the web. The fundamental
12−the web. The fundamental concepts are no different than from the
12+concepts are no different than from the command line -- you send "poke" messages
13−command line -- you send "poke" messages to apps and subscribe to
13+to apps and subscribe to data streams. The only difference is that you do it
14−data streams. The only difference is that you do it with
14+with javascript.
15−javascript.
16
15
17−Let's create a small web app that counts the number of times you
16+Let's create a small web app that counts the number of times you poke it.
18−poke it. There'll be a button to poke the app, and line of text
17+There'll be a button to poke the app, and line of text below it saying how many
19−below it saying how many times the app has been poked. We can
18+times the app has been poked. We can update this by subscribing our page to the
20−update this by subscribing our page to the relevant path on its app.
19+relevant path on its app.
21
20
22Let's first checkout the app `/app/examples/click.hoon`:
21Let's first checkout the app `/app/examples/click.hoon`:
23
22
24−```
23+ /? 314
25−/? 314
24+ !:
26−!:
25+ |%
27−|%
26+ ++ move {bone $diff $mark *}
28−++ move {bone $diff $mark *}
27+ --
29−--
28+ !:
30−!:
29+ |_ {hid/bowl clicks/@}
31−|_ {hid/bowl clicks/@}
30+ ++ poke-examples-click-click
32−++ poke-examples-click-click
31+ |= click/$click
33− |= click/$click
32+ ~& [%poked +(clicks)]
34− ~& [%poked +(clicks)]
33+ :_ +>.$(clicks +(clicks))
35− :_ +>.$(clicks +(clicks))
34+ %+ turn (prey /the-path hid)
36− %+ turn (prey /the-path hid)
35+ |=({o/bone *} [o %diff %examples-click-clicks +(clicks)])
37− |=({o/bone *} [o %diff %examples-click-clicks +(clicks)])
36+ ++ peer-the-path
38−++ peer-the-path
37+ |= pax/path
39− |= pax/path
38+ [[[ost.hid %diff %examples-click-clicks clicks] ~] +>.$]
40− [[[ost.hid %diff %examples-click-clicks clicks] ~] +>.$]
39+ --
41−--
40+
42−```
41+There's nothing really new here, except that we use a couple of new marks,
43−
42+`click` and `clicks`. When we get poked with a `click`, we increment the
44−There's nothing really new here, except that we use a couple of
43+variable `clicks` in our state and tell all our subcribers the new value. When
45−new marks, `click` and `clicks`. When we get poked with a
44+someone subscribes to us on the path `/the-path`, we immediately give them the
46−`click`, we increment the variable `clicks` in our state and tell
45+current number of clicks.
47−all our subcribers the new value. When someone subscribes to us
46+
48−on the path `/the-path`, we immediately give them the current
47+Let's take a look at the new marks. Here's `/mar/examples/click/click.hoon`:
49−number of clicks.
48+
50−
49+ |_ click/$click
51−Let's take a look at the new marks. Here's `/mar/examples/click/click.hoon`:
50+ ++ grab
52−
51+ |%
53−```
52+ ++ noun |=(* %click)
54−|_ click/$click
53+ ++ json
55−++ grab
54+ |= jon/^json
56− |%
55+ ?> =('click' (need (so:jo jon)))
57− ++ noun |=(* %click)
56+ %click
58− ++ json
57+ --
59− |= jon/^json
58+ --
60− ?> =('click' (need (so:jo jon)))
61− %click
62− --
63−--
64−```
65
59
66The mark `examples-click-click` has hoon type `%click`, which means the only
60The mark `examples-click-click` has hoon type `%click`, which means the only
67−valid value is `%click`. We can convert from `noun` by just
61+valid value is `%click`. We can convert from `noun` by just producing `%click`.
68−producing `%click`.
62+
69−
63+We also convert from json by parsing a json string with `so:jo`, asserting the
70−We also convert from json by parsing a json string with `so:jo`,
64+parsing succeeded with `need`, and asserting the result was 'click' with `?>`,
71−asserting the parsing succeeded with `need`, and asserting the
65+which asserts that its first child is true.
72−result was 'click' with `?>`, which asserts that its first child
66+
73−is true.
67+> Note the argument to `++json`is `jon/^json`. Why `^json`? `++json` shadows the
74−
68+> type definition, so if we want to refer to the type, we have to prepend a `^`.
75−> Note the argument to `++json`is `jon/^json`. Why `^json`?
69+> This extends to multiple levels: `^^^foo` means the fourth most innermost
76−> `++json` shadows the type definition, so if we want to refer to
70+> instance of `foo`.
77−> the type, we have to prepend a `^`. This extends to multiple
71+
78−> levels: `^^^foo` means the fourth most innermost instance of `foo`.
72+> `++jo` in `zuse` is a useful library for parsing complex json into hoon
79−
73+> structures. In this case, the `:` between `so` and `jo` means 'inside of',
80−> `++jo` in `zuse` is a useful library for parsing complex json
74+> because `so` is an arm contained within the core `jo`. Our case is actually
81−> into hoon structures. In this case, the `:` between `so` and
75+> simple enough that the `?>` line could have been `?> =([%s 'click'] jon)`.
82−> `jo` means 'inside of', because `so` is an arm contained within
76+
83−> the core `jo`. Our case is actually simple enough that the
77+We can test this mark from the command line (don't forget to start your app with
84−> `?>` line could have been `?> =([%s 'click'] jon)`.
78+`|start %examples-click`)
85−
79+
86−We can test this mark from the command line (don't forget to start your app with `|start %examples-click`)
80+ ~fintud-macrep:dojo> &examples-click-click %click
87−
81+ %click
88−```
82+ ~fintud-macrep:dojo> &examples-click-click &json [%s 'click']
89−~fintud-macrep:dojo> &examples-click-click %click
83+ %click
90−%click
84+ ~fintud-macrep:dojo> &examples-click-click &json &mime [/text/json (taco '"click"')]
91−~fintud-macrep:dojo> &examples-click-click &json [%s 'click']
85+ %click
92−%click
86+ ~fintud-macrep:dojo> &examples-click-click &json &mime [/text/json (taco '"clickety"')]
93−~fintud-macrep:dojo> &examples-click-click &json &mime [/text/json (taco '"click"')]
87+ /~fintud-macrep/home/0/mar/click:<[7 5].[8 11]>
94−%click
88+ ford: casting %json to %click
95−~fintud-macrep:dojo> &examples-click-click &json &mime [/text/json (taco '"clickety"')]
89+ ford: cast %click
96−/~fintud-macrep/home/0/mar/click:<[7 5].[8 11]>
97−ford: casting %json to %click
98−ford: cast %click
99−```
100
90
101And `/mar/examples/click/clicks.hoon`:
91And `/mar/examples/click/clicks.hoon`:
102
92
103−```
93+ |_ clicks/@
104−|_ clicks/@
94+ ++ grow
105−++ grow
95+ |%
106− |%
96+ ++ json
107− ++ json
97+ (joba %clicks (jone clicks))
108− (joba %clicks (jone clicks))
98+ --
109− --
99+ --
110−--
100+
111−```
101+`examples-clicks-clicks` is just an atom. We convert to json by creating an
112−
113−`examples-clicks-clicks` is just an atom. We convert to json by creating an
114object with a single field "clicks" with our value.
102object with a single field "clicks" with our value.
115
103
116−> Be sure to checkout section 3bD, JSON and XML, in zuse.hoon
104+> Be sure to checkout section 3bD, JSON and XML, in zuse.hoon `++joba` is just a
117−> `++joba` is just a function that takes a key-value pair and
105+> function that takes a key-value pair and produces a JSON object with one
118−> produces a JSON object with one element.
106+> element.
119
107
120−```
108+ ~fintud-macrep:dojo> &json &examples-click-clicks 6
121−~fintud-macrep:dojo> &json &examples-click-clicks 6
109+ [%o p={[p='clicks' q=[%n p=~.6]]}]
122−[%o p={[p='clicks' q=[%n p=~.6]]}]
123−```
124
110
125−Now that we know the marks involved, take another look at the app
111+Now that we know the marks involved, take another look at the app above.
126−above. Everything should be pretty straightforward. Let's poke
112+Everything should be pretty straightforward. Let's poke the app from the command
127−the app from the command line.
113+line.
128
114
129−```
115+ ~fintud-macrep:dojo> |start %examples-click
130−~fintud-macrep:dojo> |start %examples-click
116+ >=
131−>=
117+ ~fintud-macrep:dojo> :examples-click &examples-click-click %click
132−~fintud-macrep:dojo> :examples-click &examples-click-click %click
118+ [%poked 1]
133−[%poked 1]
119+ >=
134−>=
120+ ~fintud-macrep:dojo> :examples-click &examples-click-click %click
135−~fintud-macrep:dojo> :examples-click &examples-click-click %click
121+ [%poked 2]
136−[%poked 2]
122+ >=
137−>=
138−```
139
123
140−Please pardon the awkwardness of the name 'examples-click-click', but we used it to stay true to our examples naming convention.
124+Please pardon the awkwardness of the name 'examples-click-click', but we used it
125+to stay true to our examples naming convention.
141
126
142**Exercise**:
127**Exercise**:
143
128
144−- Modify `:sink` from the subcriptions chapter to listen to
129+- Modify `:sink` from the subcriptions chapter to listen to `:click` and print
145− `:click` and print out the subscription updates on the command
130+ out the subscription updates on the command line.
146− line.
147
131
148# Frontend
132# Frontend
149
133
150−That's all that's needed for the back end. The front end is just
134+That's all that's needed for the back end. The front end is just some "sail"
151−some "sail" html (Hoon markup for XML) and javascript. Here's `/web/pages/examples/click.hoon`:
135+html (Hoon markup for XML) and javascript. Here's
152−
136+`/web/pages/examples/click.hoon`:
153−```
137+
154−;html
138+ ;html
155− ;head
139+ ;head
156− ;script(type "text/javascript", src "//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.min.js");
140+ ;script(type "text/javascript", src "//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.min.js");
157− ;script(type "text/javascript", src "/~/at/lib/js/urb.js");
141+ ;script(type "text/javascript", src "/~/at/lib/js/urb.js");
158− ;title: Clickety!
142+ ;title: Clickety!
159− ==
143+ ==
160− ;body
144+ ;body
161− ;div#cont
145+ ;div#cont
162− ;input#go(type "button", value "Poke!");
146+ ;input#go(type "button", value "Poke!");
163− ;div#err(class "disabled");
147+ ;div#err(class "disabled");
164− ;div#clicks;
148+ ;div#clicks;
149+ ==
150+ ;script(type "text/javascript", src "/pages/examples/click/click.js");
151+ ==
165 ==
152 ==
166− ;script(type "text/javascript", src "/pages/examples/click/click.js");
153+
167− ==
154+You should recognize the sail syntax from an earlier chapter. Aside from jquery,
168−==
155+we also include `urb.js`, which is a framework for interacting with urbit.
169−```
156+
170−
157+To view the frontend, point your browser at
171−You should recognize the sail syntax from an earlier chapter.
158+`ship-name.urbit.org/~~/pages/examples/click` if you're on the live network. If
172−Aside from jquery, we also include `urb.js`, which is a framework
159+you're running a fake galaxy, navigate to whatever port it's running on, which
173−for interacting with urbit.
160+is usually: `https://localhost:8443/~~/pages/examples/click`.
174−
161+
175−To view the frontend, point your browser at `ship-name.urbit.org/~~/pages/examples/click` if you're on the live network. If you're running a fake galaxy, navigate to whatever port it's running on, which is usually: `https://localhost:8443/~~/pages/examples/click`.
162+We have a button labeled "Poke!" and a div with id `clicks` where we'll put the
176−
163+number of clicks. We also include a small javascript file where the client-side
177−We have a button labeled "Poke!" and a div with id `clicks` where
164+application logic can be found. It's in `/web/pages/examples/click/click.js`:
178−we'll put the number of clicks. We also include a small
165+
179−javascript file where the client-side application logic can be
166+
180−found. It's in `/web/pages/examples/click/click.js`:
167+ $(function() {
181−
168+ var clicks, $go, $clicks, $err
182−```
169+
183−
170+ $go = $('#go')
184−$(function() {
171+ $clicks = $('#clicks')
185− var clicks, $go, $clicks, $err
172+ $err = $('#err')
186−
173+
187− $go = $('#go')
174+ $go.on("click",
188− $clicks = $('#clicks')
175+ function() {
189− $err = $('#err')
176+ window.urb.send(
190−
177+ "click", {mark: "examples-click-click"}
191− $go.on("click",
178+ ,function(err,res) {
192− function() {
179+ if(err)
193− window.urb.send(
180+ return $err.text("There was an error. Sorry!")
194− "click", {mark: "examples-click-click"}
181+ if(res.data !== undefined &&
195− ,function(err,res) {
182+ res.data.ok !== undefined &&
196− if(err)
183+ res.data.ok !== true)
197− return $err.text("There was an error. Sorry!")
184+ $err.text(res.data.res)
198− if(res.data !== undefined &&
185+ else
199− res.data.ok !== undefined &&
186+ console.log("poke succeeded");
200− res.data.ok !== true)
187+ })
201− $err.text(res.data.res)
202− else
203− console.log("poke succeeded");
204 })
188 })
205− })
189+
206−
190+ window.urb.appl = "examples-click"
207− window.urb.appl = "examples-click"
191+ window.urb.bind('/the-path',
208− window.urb.bind('/the-path',
192+ function(err,dat) {
209− function(err,dat) {
193+ clicks = dat.data.clicks
210− clicks = dat.data.clicks
194+ $clicks.text(clicks)
211− $clicks.text(clicks)
195+ }
212− }
196+ )
213− )
197+ })
214−})
198+
215−```
199+We set up two event handlers. When we click the button, we run
216−
200+`window.urb.send`, which sends a poke to the app specified as the first
217−We set up two event handlers. When we click the button, we run
201+argument, in this case our :click app. The arguments are data, parameters, and
218−`window.urb.send`, which sends a poke to the app specified as the first argument, in this case our :click app. The arguments
202+the callback. The data is the specific data we want to send. The parameters are
219−are data, parameters, and the callback. The data is the specific
203+all optional, but here's a list of the available ones:
220−data we want to send. The parameters are all optional, but here's a list
204+
221−of the available ones:
205+- `ship`: target urbit. Defaults to `window.urb.ship`, which defaults to the
222−
206+ urbit which served the page.
223−- `ship`: target urbit. Defaults to `window.urb.ship`, which
207+- `appl`: target app. Defaults to `window.urb.appl`, which defaults to null.
224− defaults to the urbit which served the page.
208+- `mark`: mark of the data. Defaults to `window.urb.send.mark`, which defaults
225−- `appl`: target app. Defaults to `window.urb.appl`, which
209+ to "json".
226− defaults to null.
210+- `wire`: wire of the poke. Defaults to `/`.
227−- `mark`: mark of the data. Defaults to `window.urb.send.mark`,
228− which defaults to "json".
229−- `wire`: wire of the poke. Defaults to `/`.
230
211
231In our case, we specify only that the mark is "click".
212In our case, we specify only that the mark is "click".
232
213
233−The callback funciton is called when we receive an
214+The callback funciton is called when we receive an acknowledgment. If there was
234−acknowledgment. If there was an error, we put it in the `err`
215+an error, we put it in the `err` div that we defined above. Otherwise, we printf
235−div that we defined above. Otherwise, we printf to the console a
216+to the console a message saying the poke succeeded. As is common, we gray out
236−message saying the poke succeeded. As is common, we gray
217+the button when it is clicked. It is only reenabled when positive
237−out the button when it is clicked. It is only reenabled when positive acknowledgement is received.
218+acknowledgement is received.
238−
219+
239−Our second event handler is `window.urb.bind`, which is called immediately when the page is loaded, subscribing the page to the specified data stream on the app set with `urb.appl` (which is, in this case, :click). It also takes three arguments: path, parameters, and callback. The path is the path on the app to subscribe to. The parameters are all optional (indeed, we omit them entirely here), but are similar to those for `send`:
220+Our second event handler is `window.urb.bind`, which is called immediately when
240−
221+the page is loaded, subscribing the page to the specified data stream on the app
241−- `ship`: target urbit. Defaults to `window.urb.ship`, which
222+set with `urb.appl` (which is, in this case, :click). It also takes three
242− defaults to the urbit which served the page.
223+arguments: path, parameters, and callback. The path is the path on the app to
243−- `appl`: target app. Defaults to `window.urb.appl`, which
224+subscribe to. The parameters are all optional (indeed, we omit them entirely
244− defaults to null.
225+here), but are similar to those for `send`:
245−- `mark`: mark of expected data. Data of other marks is
226+
246− converted to this mark on the server before coming to the web
227+- `ship`: target urbit. Defaults to `window.urb.ship`, which defaults to the
247− front end. Defaults to `window.urb.bind.mark`, which defaults
228+ urbit which served the page.
248− to "json".
229+- `appl`: target app. Defaults to `window.urb.appl`, which defaults to null.
249−- `wire`: wire of the subscription. Defaults to the given path.
230+- `mark`: mark of expected data. Data of other marks is converted to this mark
250−
231+ on the server before coming to the web front end. Defaults to
251−The callback function here just updates the data in the `clicks`
232+ `window.urb.bind.mark`, which defaults to "json".
252−div. A truly robust app would intelligently handle errors, of
233+- `wire`: wire of the subscription. Defaults to the given path.
253−course.
234+
254−
235+The callback function here just updates the data in the `clicks` div. A truly
255−Note that the app doesn't have anything web-specific in itself.
236+robust app would intelligently handle errors, of course.
256−As far as it knows, it's just receiving pokes and subscriptions.
237+
257−The javascript is fairly pure as well, sending and receiving json
238+Note that the app doesn't have anything web-specific in itself. As far as it
258−everywhere. The marks are the translation layer, and they're the
239+knows, it's just receiving pokes and subscriptions. The javascript is fairly
259−only things that need to know how the hoon types map to json.
240+pure as well, sending and receiving json everywhere. The marks are the
241+translation layer, and they're the only things that need to know how the hoon
242+types map to json.
260
243
261**Exercise**:
244**Exercise**:
262
245
263−- Open the app in multiple tabs, click the button, and verify
246+- Open the app in multiple tabs, click the button, and verify that all the
264− that all the tabs stay in sink. Poke it manually from the
247+ tabs stay in sink. Poke it manually from the command line and verify the
265− command line and verify the tabs are updated as well.
248+ tabs are updated as well.
docs/byte.md+3−2
@@ -1,10 +1,11 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 2
3sort: 2
4---
4---
5
5
6# Learn
6# Learn
7
7
8−Learn Hoon in short, self-contained, bottom-up tutorials through our in-house Urbyte series. A work in progress. Still ideal for the complete newbie.
8+Learn Hoon in short, self-contained, bottom-up tutorials through our in-house
9+Urbyte series. A work in progress. Still ideal for the complete newbie.
9
10
10<list/>
11<list/>
docs/byte/0.md+157−199
@@ -1,7 +1,7 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3+next: True
3sort: 1
4sort: 1
4−next: true
5title: Setup and nouns
5title: Setup and nouns
6---
6---
7
7
@@ -9,276 +9,234 @@ title: Setup and nouns
9
9
10This is the first post in our new Urbyte series.
10This is the first post in our new Urbyte series.
11
11
12−The Urbyte concept is that you, an ordinary programmer, not a
12+The Urbyte concept is that you, an ordinary programmer, not a kernel hacker or a
13−kernel hacker or a Haskell magician (not that there's anything
13+Haskell magician (not that there's anything wrong with kernel hackers or Haskell
14−wrong with kernel hackers or Haskell magicians) can learn Hoon,
14+magicians) can learn Hoon, Urbit's programming language.
15−Urbit's programming language.
16
15
17−We're going to test this daring idea by teaching you Hoon, in
16+We're going to test this daring idea by teaching you Hoon, in easily digestible,
18−easily digestible, byte-size chunks. Each post in the "Urbytes"
17+byte-size chunks. Each post in the "Urbytes" series is designed to take about 15
19−series is designed to take about 15 minutes to consume. (Maybe
18+minutes to consume. (Maybe 10, if you're a kernel hacker or a Haskell magician.)
20−10, if you're a kernel hacker or a Haskell magician.)
21
19
22−We'll assume no knowledge of Urbit except for previous lessons.
20+We'll assume no knowledge of Urbit except for previous lessons. Ideally you have
23−Ideally you have some basic familiarity with programming and can
21+some basic familiarity with programming and can operate the Unix command line.
24−operate the Unix command line.
25
22
26−We strongly encourage you to type in all the examples in each
23+We strongly encourage you to type in all the examples in each Urbyte. Urbytes
27−Urbyte. Urbytes are short, but not meant to be skimmed.
24+are short, but not meant to be skimmed.
28
25
29## Install
26## Install
30
27
31−First, [install Urbit](http://urbit.org/docs/using/install/) on any
28+First, [install Urbit](http://urbit.org/docs/using/install/) on any Mac or Unix
32−Mac or Unix machine. On Windows, make a virtual Linux machine
29+machine. On Windows, make a virtual Linux machine using VirtualBox or a similar
33−using VirtualBox or a similar tool.
30+tool.
34
31
35While you're downloading, read about Urbit ships:
32While you're downloading, read about Urbit ships:
36
33
37## Ships
34## Ships
38
35
39−In the Urbyte exercises you'll be running what we call a "comet". A string starting with `~`, like `~zod`, is what Urbit
36+In the Urbyte exercises you'll be running what we call a "comet". A string
40−calls a *ship*.
37+starting with `~`, like `~zod`, is what Urbit calls a *ship*.
41
38
42−A *ship* is one number which means four things: a network address,
39+A *ship* is one number which means four things: a network address, a
43−a cryptographic identity, a memorable name, and a single virtual
40+cryptographic identity, a memorable name, and a single virtual computer.
44−computer.
45
41
46−We write this number in a base-256 format, where each digit is a
42+We write this number in a base-256 format, where each digit is a syllable.
47−syllable. Imagine if your phone number was a pronounceable
43+Imagine if your phone number was a pronounceable string which sounded like a
48−string which sounded like a name in a foreign language. `~zod`
44+name in a foreign language. `~zod` is ship zero (making it a very important ship
49−is ship zero (making it a very important ship indeed!). An ordinary
45+indeed!). An ordinary user-level ship is a "planet", a 32-bit number which
50−user-level ship is a "planet", a 32-bit number which becomes
46+becomes a four-syllable string, like `~talsur-todres`.
51−a four-syllable string, like `~talsur-todres`.
52
47
53−*Comets*, at 128-bits or 16 syllables, are
48+*Comets*, at 128-bits or 16 syllables, are disposable identities that anyone can
54−disposable identities that anyone can make for free to
49+make for free to join the live network. Thus, comets make the ideal ship for
55−join the live network. Thus, comets make the ideal ship for
50+playing around with Hoon, asking for help from other Urbit devs in
56−playing around with Hoon, asking for help from other Urbit devs in [`:talk`](http://urbit.org/docs/using/messaging/) if you
51+[`:talk`](http://urbit.org/docs/using/messaging/) if you need it, and showing
57−need it, and showing off Urbit to your hacker friends.
52+off Urbit to your hacker friends.
58
53
59## Create and restart
54## Create and restart
60
55
61To create your comet, run:
56To create your comet, run:
62
57
63−```
58+ $ cd ~
64−$ cd ~
59+ $ mkdir urbit
65−$ mkdir urbit
60+ $ cd urbit
66−$ cd urbit
61+ $ urbit -c mycomet
67−$ urbit -c mycomet
68−```
69−
70−This will take a few minutes and will spin out a bunch of boot
71−messages. At the end, you'll see something like:
72−
73−```
74−ames: on localhost, UDP 31337.
75−http: live (insecure, public) on 8080
76−http: live ("secure", public) on 8443
77−http: live (insecure, loopback) on 12321
78−~palnul_nocser:dojo>
79−```
80−
81−To make sure everything is working, type `(add 2 2)` at the
82−prompt, then hit return. Your screen now shows:
83−
84−```
85−ames: on localhost, UDP 31337.
86−http: live (insecure, public) on 8080
87−http: live ("secure", public) on 8443
88−http: live (insecure, loopback) on 12321
89−> (add 2 2)
90−4
91−```
92−~palnul_nocser:dojo>
93−
94−That's super awesome! You made your first noun! We'll make a
95−few more in a second. First, quit Urbit with `ctrl-d`:
96−
97−```
98−> (add 2 2)
99−4
100−~palnul_nocser:dojo>
101−$
102−```
103−
104−You now have an Urbit image, or *pier*, in the `mycomet` directory.
105−Right now the only thing in your pier is Urbit's system files:
106−
107−```
108−$ ls -a /path/to/mycomet
109−./ ../ .urb/
110−```
111−
112−Restarting uses almost the same command, but without `-c` (which
113−means "create"). There are also fewer startup messages:
114−
115−```
116−$ urbit mycomet
117−[...]
118−ames: on localhost, UDP 31337.
119−http: live (insecure, public) on 8080
120−http: live ("secure", public) on 8443
121−http: live (insecure, loopback) on 12321
122−~palnul_nocser:dojo>
123−```
124−
125−Note also that Urbit is an ACID database. You can stop it
126−politely with `ctrl-d`, abruptly with `ctrl-z`, even violently with `kill
127−-9`. None of these should cause Urbit to lose data. If you do
128−manage to corrupt a pier, it's a bug; please let us know.
129
62
130−## Let's make a noun
63+This will take a few minutes and will spin out a bunch of boot messages. At the
64+end, you'll see something like:
65+
66+ ames: on localhost, UDP 31337.
67+ http: live (insecure, public) on 8080
68+ http: live ("secure", public) on 8443
69+ http: live (insecure, loopback) on 12321
70+ ~palnul_nocser:dojo>
71+
72+To make sure everything is working, type `(add 2 2)` at the prompt, then hit
73+return. Your screen now shows:
74+
75+ ames: on localhost, UDP 31337.
76+ http: live (insecure, public) on 8080
77+ http: live ("secure", public) on 8443
78+ http: live (insecure, loopback) on 12321
79+ > (add 2 2)
80+ 4
81+
82+~palnul\_nocser:dojo&gt;
83+
84+That's super awesome! You made your first noun! We'll make a few more in a
85+second. First, quit Urbit with `ctrl-d`:
86+
87+ > (add 2 2)
88+ 4
89+ ~palnul_nocser:dojo>
90+ $
131
91
132−You've already made a noun. Let's make another!
92+You now have an Urbit image, or *pier*, in the `mycomet` directory. Right now
93+the only thing in your pier is Urbit's system files:
94+
95+ $ ls -a /path/to/mycomet
96+ ./ ../ .urb/
97+
98+Restarting uses almost the same command, but without `-c` (which means
99+"create"). There are also fewer startup messages:
100+
101+ $ urbit mycomet
102+ [...]
103+ ames: on localhost, UDP 31337.
104+ http: live (insecure, public) on 8080
105+ http: live ("secure", public) on 8443
106+ http: live (insecure, loopback) on 12321
107+ ~palnul_nocser:dojo>
108+
109+Note also that Urbit is an ACID database. You can stop it politely with
110+`ctrl-d`, abruptly with `ctrl-z`, even violently with `kill -9`. None of these
111+should cause Urbit to lose data. If you do manage to corrupt a pier, it's a bug;
112+please let us know.
113+
114+## Let's make a noun
133
115
134−*We won't show the `~palnul_nocser:dojo> ` prompt from here on out.
116+You've already made a noun. Let's make another!
135−We'll just show the echoed command.*
136
117
118+*We won't show the `~palnul_nocser:dojo>` prompt from here on out. We'll just
119+show the echoed command.*
137
120
138−```
121+ ~palnul_nocser:dojo> 42
139−~palnul_nocser:dojo> 42
122+ 42
140−42
141−```
142
123
143You'll see:
124You'll see:
144
125
145−```
126+ > 42
146−> 42
127+ 42
147−42
148−```
149
128
150## Understanding nouns
129## Understanding nouns
151
130
152−A value in Urbit is called a *noun*. A noun is either an *atom*
131+A value in Urbit is called a *noun*. A noun is either an *atom* (an unsigned
153−(an unsigned integer of any size) or a *cell* (an ordered pair of
132+integer of any size) or a *cell* (an ordered pair of any two nouns).
154−any two nouns).
155
133
156−In a sentence, a noun is a binary tree whose leaves are numbers.
134+In a sentence, a noun is a binary tree whose leaves are numbers. If you know
157−If you know Lisp, a noun is a minimalist S-expression. A noun
135+Lisp, a noun is a minimalist S-expression. A noun can't have cycles (pointer
158−can't have cycles (pointer loops). An implementation can use
136+loops). An implementation can use dags (shared subtrees), but the programmer
159−dags (shared subtrees), but the programmer can't detect them
137+can't detect them (there's no pointer equality test).
160−(there's no pointer equality test).
161
138
162−The absence of graph cycles makes nouns much easier to save in
139+The absence of graph cycles makes nouns much easier to save in persistent
163−persistent storage, send over the network, manage memory for
140+storage, send over the network, manage memory for (Urbit has no tracing garbage
164−(Urbit has no tracing garbage collector), etc.
141+collector), etc.
165
142
166## Let's make some atoms
143## Let's make some atoms
167
144
168−So one kind of atom is a number. And the syntax (or at least one
145+So one kind of atom is a number. And the syntax (or at least one syntax) for a
169−syntax) for a number is a decimal, like `42`. Seems legit.
146+number is a decimal, like `42`. Seems legit.
170
147
171−But just to be sure we understand this decimal thing, let's make
148+But just to be sure we understand this decimal thing, let's make some atoms:
172−some atoms:
173
149
174−```
150+ > 4
175−> 4
151+ 4
176−4
177−```
178
152
179Seems pretty straightforward.
153Seems pretty straightforward.
180
154
181−```
155+ > 42
182−> 42
156+ 42
183−42
184−```
185
157
186Why not try it again to make sure?
158Why not try it again to make sure?
187
159
188−```
160+ > 420
189−> 420
161+ 420
190−420
191−```
192
162
193Can we add another zero?
163Can we add another zero?
194
164
195−```
165+ > 420
196−> 420
197−```
198
166
199−Wat? When you tried to type that next zero and turn `420` into
167+Wat? When you tried to type that next zero and turn `420` into `4200`, Urbit
200−`4200`, Urbit actually *deleted the `0` and beeped at you.*
168+actually *deleted the `0` and beeped at you.*
201
169
202−But isn't an atom an unsigned integer of any size? It is.
170+But isn't an atom an unsigned integer of any size? It is. But the syntax is
203−But the syntax is actually `4.200`:
171+actually `4.200`:
204
172
205−```
173+ > 4.200
206−> 4.200
174+ 4.200
207−4.200
175+ > (mul 42 100)
208−> (mul 42 100)
176+ 4.200
209−4.200
210−```
211
177
212Just think of it as "4,200", written the German way with a dot.
178Just think of it as "4,200", written the German way with a dot.
213
179
214−Why? Long decimals are unreadable. Human beings know this, so
180+Why? Long decimals are unreadable. Human beings know this, so we group them in
215−we group them in threes. For some historical reason, programming
181+threes. For some historical reason, programming languages never copied this
216−languages never copied this marvelous usability innovation.
182+marvelous usability innovation.
217
183
218−English notation for decimals is more common than German
184+English notation for decimals is more common than German notation.
219−notation. Unfortunately, dot is URL-safe and comma isn't, and
185+Unfortunately, dot is URL-safe and comma isn't, and it's nice to have a regular
220−it's nice to have a regular syntax for atoms that's URL-safe.
186+syntax for atoms that's URL-safe.
221
187
222−As for why Urbit deleted your zero: it parses your command line
188+As for why Urbit deleted your zero: it parses your command line as you type, and
223−as you type, and rejects any characters after the parser stops.
189+rejects any characters after the parser stops.
224
190
225## Let's make some cells
191## Let's make some cells
226
192
227−There's not much mystery about cells. A cell has a head and a
193+There's not much mystery about cells. A cell has a head and a tail, and puts
228−tail, and puts brackets around them:
194+brackets around them:
229
195
230−```
196+ > [42 420]
231−> [42 420]
197+ [42 420]
232−[42 420]
233−```
234
198
235−`42` is the *head* and `420` is the *tail* of the cell. Cell
199+`42` is the *head* and `420` is the *tail* of the cell. Cell syntax nests to the
236−syntax nests to the right:
200+right:
237
201
238−```
202+ > [4 [42 420]]
239−> [4 [42 420]]
203+ [4 42 420]
240−[4 42 420]
241
204
242−> [4 42 420]
205+ > [4 42 420]
243−[4 42 420]
206+ [4 42 420]
244
207
245−> [[4 42] 420]
208+ > [[4 42] 420]
246−[[4 42] 420]
209+ [[4 42] 420]
247−```
248
210
249If you look at these nouns as binary trees, you see
211If you look at these nouns as binary trees, you see
250
212
251−```
213+ [4 [42 420]] [[4 42] 420]
252− [4 [42 420]] [[4 42] 420]
214+ . .
253− . .
215+ / \ / \
254− / \ / \
216+ 4 . . 420
255− 4 . . 420
217+ / \ / \
256− / \ / \
218+ 42 420 4 42
257− 42 420 4 42
258−```
259
219
260Each number is an atom; each dot is a cell.
220Each number is an atom; each dot is a cell.
261
221
262## And we're done
222## And we're done
263
223
264−And we're done! Type `|exit` to exit:
224+And we're done! Type `|exit` to exit:
265
225
266−```
226+ ~palnul_nocser:dojo> |exit
267−~palnul_nocser:dojo> |exit
227+ %drum-exit
268−%drum-exit
228+ $
269−$
270−```
271
229
272## Questions and/or exercises
230## Questions and/or exercises
273
231
274−And we're done with our very first Urbyte! Here are some questions to
232+And we're done with our very first Urbyte! Here are some questions to think
275−think about. These exercises are never mandatory, but they may
233+about. These exercises are never mandatory, but they may be fun and/or
276−be fun and/or interesting. Skip them if they're obvious.
234+interesting. Skip them if they're obvious.
277
235
278−* What is the right way to represent common
236+- What is the right way to represent common data structures as nouns?
279−data structures as nouns?
237+- A linked list?
280−* A linked list?
238+- A string? A *signed* integer?
281−* A string? A *signed* integer?
239+- A table or associative array?
282−* A table or associative array?
283
240
284−Just try to picture these structures in your head, or draw them on a piece of paper.
241+Just try to picture these structures in your head, or draw them on a piece of
242+paper.
docs/byte/1.md+222−252
@@ -1,369 +1,339 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3+next: True
3sort: 2
4sort: 2
4−next: true
5+title: 'Atoms, auras, types'
5−title: Atoms, auras, types
6---
6---
7
7
8# Urbyte 1: Atoms, auras, types
8# Urbyte 1: Atoms, auras, types
9
9
10−Thanks for reading Urbyte 0! In Urbyte 1, we'll look a little
10+Thanks for reading Urbyte 0! In Urbyte 1, we'll look a little more deeply at
11−more deeply at atoms and other simple noun types.
11+atoms and other simple noun types.
12
12
13Let's restart our engines:
13Let's restart our engines:
14
14
15−```
15+ $ urbit mycomet
16−$ urbit mycomet
16+ [...]
17−[...]
17+ ames: on localhost, UDP 31337.
18−ames: on localhost, UDP 31337.
18+ http: live (insecure, public) on 8080
19−http: live (insecure, public) on 8080
19+ http: live ("secure", public) on 8443
20−http: live ("secure", public) on 8443
20+ http: live (insecure, loopback) on 12321
21−http: live (insecure, loopback) on 12321
21+ ~palnul_nocser:dojo>
22−~palnul_nocser:dojo>
23−```
24
22
25## What's in an atom?
23## What's in an atom?
26
24
27−It seems limiting to have only one kind of atom, the unsigned
25+It seems limiting to have only one kind of atom, the unsigned decimal. How are
28−decimal. How are signed integers represented? Symbols?
26+signed integers represented? Symbols? Floating-point values? Dates? IP
29−Floating-point values? Dates? IP addresses? Even just hex?
27+addresses? Even just hex?
30
28
31−Lisp's S-expression, with its own cells and atoms, is very
29+Lisp's S-expression, with its own cells and atoms, is very similar to Hoon's
32−similar to Hoon's nouns. But languages in the Lisp family mark
30+nouns. But languages in the Lisp family mark each atom with a dynamic tag that
33−each atom with a dynamic tag that defines the atom's meaning.
31+defines the atom's meaning. These tags are effectively dynamic types.
34−These tags are effectively dynamic types.
35
32
36−Hoon also tracks the meaning of atoms. But Hoon is statically
33+Hoon also tracks the meaning of atoms. But Hoon is statically typed, so an atom
37−typed, so an atom description is obviously a type. An atomic type
34+description is obviously a type. An atomic type in Hoon is called an *aura*. The
38−in Hoon is called an *aura*. The aura syntax is just an ASCII
35+aura syntax is just an ASCII string starting with `@`, like `@ud` for unsigned
39−string starting with `@`, like `@ud` for unsigned decimal.
36+decimal.
40
37
41−Hoon has a wide (but not extensible) variety of atom syntaxes.
38+Hoon has a wide (but not extensible) variety of atom syntaxes. Each syntax for
42−Each syntax for an atom produces an appropriate predefined aura.
39+an atom produces an appropriate predefined aura. And Hoon's printer can
43−And Hoon's printer can invertibly print any aura Hoon can parse.
40+invertibly print any aura Hoon can parse. Because atoms make great path nodes
44−Because atoms make great path nodes and paths make great URLs,
41+and paths make great URLs, all regular atom syntaxes use only URL-safe
45−all regular atom syntaxes use only URL-safe characters.
42+characters.
46
43
47## Examples
44## Examples
48
45
49−With the dojo command `? expression`, you tell the dojo to print
46+With the dojo command `? expression`, you tell the dojo to print both the
50−both the product of the expression, and the product's type.
47+product of the expression, and the product's type.
51
48
52Let's type in some aura examples:
49Let's type in some aura examples:
53
50
54### Unsigned integers
51### Unsigned integers
55
52
56−```
53+ > 42
57−> 42
54+ 42
58−42
59
55
60−> ? 42
56+ > ? 42
61− @ud
57+ @ud
62−42
58+ 42
63
59
64−> ? 0x42
60+ > ? 0x42
65− @ux
61+ @ux
66−0x42
62+ 0x42
67
63
68−> ? (add 0x42 42)
64+ > ? (add 0x42 42)
69− @
65+ @
70−108
66+ 108
71
67
72−> ? `@ud`0x42
68+ > ? `@ud`0x42
73− @ud
69+ @ud
74−66
70+ 66
75
71
76−```
72+So decimal is aura `@ud`, hexadecimal is `@ux`. Note the cast syntax, which lets
73+us change the type of an atom, converting hex `0x42` to a decimal `66`.
77
74
78−So decimal is aura `@ud`, hexadecimal is `@ux`. Note the cast
75+An aura is soft type. The programmer can always insist on how to interpret an
79−syntax, which lets us change the type of an atom, converting hex
76+atom. Hoon doesn't have "dependent types"; it can't enforce any constraints the
80−`0x42` to a decimal `66`.
77+aura puts on the atom's value.
81−
82−An aura is soft type. The programmer can always insist on how
83−to interpret an atom. Hoon doesn't have "dependent types"; it
84−can't enforce any constraints the aura puts on the atom's value.
85
78
86### Signed integers
79### Signed integers
87
80
88−```
81+ > -7
89−> -7
82+ -7
90−-7
91
83
92−> ? -7
84+ > ? -7
93− @sd
85+ @sd
94
86
95−> --7
87+ > --7
96−--7
88+ --7
97
89
98−> ? --7
90+ > ? --7
99− @sd
91+ @sd
100−--7
92+ --7
101
93
102−> ? (sum:si -7 --7)
94+ > ? (sum:si -7 --7)
103− @s
95+ @s
104−--0
96+ --0
105
97
106−> `@ud`-7
98+ > `@ud`-7
107−13
99+ 13
108
100
109−> `@ud`--7
101+ > `@ud`--7
110−14
102+ 14
111−```
112
103
113−`--7` means "positive 7." `+7` might have been better, but `+`
104+`--7` means "positive 7." `+7` might have been better, but `+` is not URL-safe.
114−is not URL-safe.
115
105
116−Hoon needs `--` to distinguish positive signed numbers because
106+Hoon needs `--` to distinguish positive signed numbers because Hoon's type
117−Hoon's type system doesn't feed back into its parser. The
107+system doesn't feed back into its parser. The expression itself needs to control
118−expression itself needs to control whether it's `@sd` or `@ud`.
108+whether it's `@sd` or `@ud`.
119
109
120−With unlimited atom width, traditional sign extension makes no
110+With unlimited atom width, traditional sign extension makes no sense. The sign
121−sense. The sign bit is the low bit. Even atoms are positive,
111+bit is the low bit. Even atoms are positive, odd atoms are negative.
122−odd atoms are negative.
123
112
124−And Hoon has no overloading or type detection; we need to use a
113+And Hoon has no overloading or type detection; we need to use a different
125−different function to add signed integers. We'll always choose
114+function to add signed integers. We'll always choose slightly clunkier syntax in
126−slightly clunkier syntax in exchange for much simpler semantics.
115+exchange for much simpler semantics.
127
116
128### Symbols
117### Symbols
129
118
130−```
119+ > ? 'Ürbit'
131−> ? 'Ürbit'
120+ @t
132− @t
121+ 'Ürbit'
133−'Ürbit'
134
122
135−> ? `@ux`'Ürbit'
123+ > ? `@ux`'Ürbit'
136− @ux
124+ @ux
137−0x7469.6272.9cc3
125+ 0x7469.6272.9cc3
138
126
139−> ? ~~Ürbit
127+ > ? ~~Ürbit
140− @t
128+ @t
141−'Ürbit'
129+ 'Ürbit'
142
130
143−> ? ~.urbit
131+ > ? ~.urbit
144− @ta
132+ @ta
145−~.urbit
133+ ~.urbit
146
134
147−> ? `@ux`'foo'
135+ > ? `@ux`'foo'
148− @ux
136+ @ux
149−0x6f.6f66
137+ 0x6f.6f66
150−```
151
138
152−The `@t` aura is UTF-8 text, with the first byte low. Again,
139+The `@t` aura is UTF-8 text, with the first byte low. Again, Hoon doesn't
153−Hoon doesn't actually have the power to check statically that
140+actually have the power to check statically that your UTF-8 is valid. Auras
154−your UTF-8 is valid. Auras don't tell you what an atom is, just
141+don't tell you what an atom is, just what you think it is.
155−what you think it is.
156
142
157−The familiar single-quote syntax for `@t` is an irregular form of
143+The familiar single-quote syntax for `@t` is an irregular form of the more
158−the more unusual `~~` prefix. The `~.` prefix, aura `@ta`, is
144+unusual `~~` prefix. The `~.` prefix, aura `@ta`, is restricted to lowercase
159−restricted to lowercase ASCII (a great common denominator). Of
145+ASCII (a great common denominator). Of course, `~` is URL-safe and `'` is not.
160−course, `~` is URL-safe and `'` is not.
161
146
162### Dates, floats, IP addresses...
147### Dates, floats, IP addresses...
163
148
164−```
149+ > ? now
165−> ? now
150+ @da
166− @da
151+ ~2016.7.11..01.32.04..2c0d
167−~2016.7.11..01.32.04..2c0d
168
152
169−> ? ~h6
153+ > ? ~h6
170− @dr
154+ @dr
171−~h6
155+ ~h6
172
156
173−> `@ux`~h6
157+ > `@ux`~h6
174− @ux
158+ @ux
175−0x5460.0000.0000.0000.0000
159+ 0x5460.0000.0000.0000.0000
176
160
177−> ? (add ~h6 now)
161+ > ? (add ~h6 now)
178− @
162+ @
179−170.141.184.502.236.089.414.812.327.632.537.911.296
163+ 170.141.184.502.236.089.414.812.327.632.537.911.296
180
164
181−> `@da`(add ~h6 now)
165+ > `@da`(add ~h6 now)
182−~2016.7.11..07.32.28..842f
166+ ~2016.7.11..07.32.28..842f
183
167
184−> ? .3.14
168+ > ? .3.14
185− @rs
169+ @rs
186−.3.14
170+ .3.14
187
171
188−> ? .127.0.0.1
172+ > ? .127.0.0.1
189− @if
173+ @if
190−.127.0.0.1
174+ .127.0.0.1
191−```
192
175
193−`@da` is a 128-bit absolute date (64 bits for fractions of a
176+`@da` is a 128-bit absolute date (64 bits for fractions of a second, 64 bits for
194−second, 64 bits for seconds); `@dr` is a relative date; `@rs` is
177+seconds); `@dr` is a relative date; `@rs` is a single-precision IEEE real; `@if`
195−a single-precision IEEE real; `@if` is an IPv4 address. This is
178+is an IPv4 address. This is not all Hoon's aura syntax, just a good range of
196−not all Hoon's aura syntax, just a good range of examples.
179+examples.
197
180
198−`(add ~h6 now)` is a good example of the use and limits of
181+`(add ~h6 now)` is a good example of the use and limits of the aura system.
199−the aura system. Unlike the signed-integer sum, offsetting a
182+Unlike the signed-integer sum, offsetting a date can use plain atomic addition.
200−date can use plain atomic addition. But the product of `add` is
183+But the product of `add` is a flavorless atom with no aura, `@`. So we have to
201−a flavorless atom with no aura, `@`. So we have to cast it back
184+cast it back to an absolute date `@da`.
202−to an absolute date `@da`.
203
185
204## Auras explained
186## Auras explained
205
187
206−Does an aura represent the units of the atom, its syntax, its
188+Does an aura represent the units of the atom, its syntax, its semantics, its
207−semantics, its constraints?
189+constraints?
208
190
209−Any or all of the above. Programmers can even use their own aura
191+Any or all of the above. Programmers can even use their own aura strings, which
210−strings, which nothing in Hoon recognizes. The set of syntaxes
192+nothing in Hoon recognizes. The set of syntaxes that Hoon knows how to parse
211−that Hoon knows how to parse and/or print is fixed. But there
193+and/or print is fixed. But there are plenty of good reasons to use your own
212−are plenty of good reasons to use your own user-defined auras.
194+user-defined auras.
213
195
214−If your program uses both fortnights and furlongs, Hoon will not
196+If your program uses both fortnights and furlongs, Hoon will not parse a
215−parse a user-defined fortnight syntax. It will not print furlongs
197+user-defined fortnight syntax. It will not print furlongs in the dojo. But the
216−in the dojo. But the type system will keep you from accidentally
198+type system will keep you from accidentally passing a furlong to a function that
217−passing a furlong to a function that expects a fortnight.
199+expects a fortnight.
218
200
219−Hoon does interpret the aura string in one way: auras specialize
201+Hoon does interpret the aura string in one way: auras specialize to the right.
220−to the right. For example, `@u` is any unsigned integer; `@ux`
202+For example, `@u` is any unsigned integer; `@ux` is an unsigned integer that
221−is an unsigned integer that likes to be printed as hex.
203+likes to be printed as hex.
222
204
223−Hoon's typechecker lets you go up or down the specialization
205+Hoon's typechecker lets you go up or down the specialization ladder. It
224−ladder. It complains if you try to move across. For example,
206+complains if you try to move across. For example, `@u` can silently convert into
225−`@u` can silently convert into `@ux` or `@ux` into `@u`, but
207+`@ux` or `@ux` into `@u`, but turning `@ud` into `@ux`, `@ux` into `@da`, etc,
226−turning `@ud` into `@ux`, `@ux` into `@da`, etc, requires an
208+requires an explicit cast.
227−explicit cast.
228
209
229## Constants, symbols, loobeans and null
210## Constants, symbols, loobeans and null
230
211
231−An atom type isn't just an aura. There's another choice: the
212+An atom type isn't just an aura. There's another choice: the type can be
232−type can be *general* and contain all atoms (like the types we've
213+*general* and contain all atoms (like the types we've seen above), or *constant*
233−seen above), or *constant* and contain just one atom.
214+and contain just one atom.
234
215
235−A type describes a set of values. While auras may informally
216+A type describes a set of values. While auras may informally describe a format
236−describe a format for which not all atoms are valid, any atom can
217+for which not all atoms are valid, any atom can be cast into the aura whether
237−be cast into the aura whether valid or invalid. But in a
218+valid or invalid. But in a constant, the type contains just one atom.
238−constant, the type contains just one atom.
239
219
240−The regular syntax for an atomic constant is just to put `%` in
220+The regular syntax for an atomic constant is just to put `%` in front of the
241−front of the constant. Like:
221+constant. Like:
242
222
243−```
223+ > ? 42
244−> ? 42
224+ @ud
245− @ud
225+ 42
246−42
247
226
248−> ? %42
227+ > ? %42
249− $42
228+ $42
250−%42
229+ %42
251
230
252−> ? %~h6
231+ > ? %~h6
253− $~h6
232+ $~h6
254−%~h6
233+ %~h6
255−```
256
234
257−The `$` on the type indicates the constant. Note that constants
235+The `$` on the type indicates the constant. Note that constants still have
258−still have auras; `~h6` (6 hours) is still a `@dr`.
236+auras; `~h6` (6 hours) is still a `@dr`.
259
237
260## Common irregulars
238## Common irregulars
261
239
262−There are a few special forms worth learning early: symbols,
240+There are a few special forms worth learning early: symbols, null, loobeans, and
263−null, loobeans, and of course ships:
241+of course ships:
264
242
265−```
243+ > ? %foo
266−> ? %foo
244+ $foo
267− $foo
245+ %foo
268−%foo
269
246
270−> `@ux`%foo
247+ > `@ux`%foo
271−0x6f.6f66
248+ 0x6f.6f66
272
249
273−> .n
250+ > .n
274−%.n
251+ %.n
275
252
276−> |
253+ > |
277−%.n
254+ %.n
278
255
279−> .y
256+ > .y
280−%.y
257+ %.y
281
258
282−> &
259+ > &
283−%.y
260+ %.y
284
261
285−> `@ud`&
262+ > `@ud`&
286−0
263+ 0
287
264
288−> ~
265+ > ~
289−~
266+ ~
290
267
291−> `@ud`~
268+ > `@ud`~
292−0
269+ 0
293
270
294−> ? ~zod
271+ > ? ~zod
295− @p
272+ @p
296−~zod
273+ ~zod
297
274
298−> `@ux`~forseg-bolbyn
275+ > `@ux`~forseg-bolbyn
299−0x885e.8af9
276+ 0x885e.8af9
300−```
301
277
302−`%symbol` is a constant with aura `@tas` (text, ASCII, symbol).
278+`%symbol` is a constant with aura `@tas` (text, ASCII, symbol). The symbol
303−The symbol rules, which are enforced by the Hoon parser but not
279+rules, which are enforced by the Hoon parser but not (as we've discussed) the
304−(as we've discussed) the type system: a symbol is lowercase with
280+type system: a symbol is lowercase with infix hyphens ("kebab-case").
305−infix hyphens ("kebab-case").
306
281
307−Loobeans are not true and false, but yes (`.y` or `&`) and no
282+Loobeans are not true and false, but yes (`.y` or `&`) and no (`.n` or `|`).
308−(`.n` or `|`). This is because *yes is zero and no is one*.
283+This is because *yes is zero and no is one*. This makes sense at a certain
309−This makes sense at a certain mathematical level, though it
284+mathematical level, though it was probably a mistake from a practical
310−was probably a mistake from a practical perspective.
285+perspective.
311
286
312`~` is Hoon's nil: constant `0`, aura `@n`.
287`~` is Hoon's nil: constant `0`, aura `@n`.
313
288
314−And of course, `@p` is the phonemic base used for ship names.
289+And of course, `@p` is the phonemic base used for ship names. (It's useful for
315−(It's useful for any kind of memorable number.)
290+any kind of memorable number.)
316
291
317## Cells and types
292## Cells and types
318
293
319−Naturally, all these atoms fit neatly into cells, and retain
294+Naturally, all these atoms fit neatly into cells, and retain their types within
320−their types within the cells.
295+the cells.
321
296
322−```
297+ > ? [~h6 [.3.14 %foo] 0xdead.beef]
323−> ? [~h6 [.3.14 %foo] 0xdead.beef]
298+ {@dr {@rs $foo} @ux}
324− {@dr {@rs $foo} @ux}
299+ [~h6 [.3.14 %foo] 0xdead.beef]
325−[~h6 [.3.14 %foo] 0xdead.beef]
326
300
327−> `*`[~h6 [.3.14 %foo] 0xdead.beef]
301+ > `*`[~h6 [.3.14 %foo] 0xdead.beef]
328−[398.449.671.992.126.314.905.600 [1.078.523.331 7.303.014] 3.735.928.559]
302+ [398.449.671.992.126.314.905.600 [1.078.523.331 7.303.014] 3.735.928.559]
329−```
330
303
331−Another fun trick is to cast any noun to the `*` type, ie,
304+Another fun trick is to cast any noun to the `*` type, ie, generic noun. We then
332−generic noun. We then know nothing about the noun, so we print
305+know nothing about the noun, so we print all its atoms as decimals.
333−all its atoms as decimals.
334
306
335## And we're done
307## And we're done
336
308
337−And we're done! Type `|exit` to exit:
309+And we're done! Type `|exit` to exit:
338
310
339−```
311+ ~palnul_nocser:dojo> |exit
340−~palnul_nocser:dojo> |exit
312+ %drum-exit
341−%drum-exit
313+ $
342−$
343−```
344
314
345## Questions and/or exercises
315## Questions and/or exercises
346
316
347As always, these questions are optional and only for fun.
317As always, these questions are optional and only for fun.
348
318
349−We didn't include anywhere near all Hoon's auras or syntaxes in
319+We didn't include anywhere near all Hoon's auras or syntaxes in this lesson.
350−this lesson. It's just a set of examples.
320+It's just a set of examples.
351
321
352−* Try using the dojo's magic error erasure to stumble around the
322+- Try using the dojo's magic error erasure to stumble around the rest of
353−rest of Hoon's atom syntax. Any string that can't be extended
323+ Hoon's atom syntax. Any string that can't be extended into a valid
354−into a valid expression will be trimmed back, with a beep, until
324+ expression will be trimmed back, with a beep, until it can. Return will beep
355−it can. Return will beep if the expression is not complete.
325+ if the expression is not complete. This feedback is harmless and does not
356−This feedback is harmless and does not use an electric shock.
326+ use an electric shock. It enables stochastic reinforcement learning of Hoon
357−It enables stochastic reinforcement learning of Hoon syntax.
327+ syntax.
358
328
359−* If `~h6` is six hours, how do you write forty-five minutes? Two
329+- If `~h6` is six hours, how do you write forty-five minutes? Two days, six
360−days, six hours and forty-five minutes?
330+ hours and forty-five minutes?
361
331
362−* What would you expect the syntax for double-precision float to
332+- What would you expect the syntax for double-precision float to be? What
363−be? What about base64?
333+ about base64?
364
334
365−* Does an absolute date have to include every zillisecond, or can
335+- Does an absolute date have to include every zillisecond, or can it be pruned
366−it be pruned to a day or a year?
336+ to a day or a year?
367
337
368−* If there's an IP syntax, what about bitcoin addresses? What
338+- If there's an IP syntax, what about bitcoin addresses? What other auras do
369−other auras do you think should be supported?
339+ you think should be supported?
docs/byte/2.md+287−342
@@ -1,495 +1,440 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 3
3sort: 3
4−title: Subject-oriented programming
4+title: 'Subject-oriented programming'
5---
5---
6
6
7# Urbyte 2: Subject-oriented programming
7# Urbyte 2: Subject-oriented programming
8
8
9−Thanks for reading Urbytes 0 and 1! In Urbyte 2, we'll take
9+Thanks for reading Urbytes 0 and 1! In Urbyte 2, we'll take nouns for granted
10−nouns for granted and have some real fun with them.
10+and have some real fun with them.
11
11
12Let's restart our engines:
12Let's restart our engines:
13
13
14−```
14+ $ urbit mycomet
15−$ urbit mycomet
15+ [...]
16−[...]
16+ ames: on localhost, UDP 31337.
17−ames: on localhost, UDP 31337.
17+ http: live (insecure, public) on 8080
18−http: live (insecure, public) on 8080
18+ http: live ("secure", public) on 8443
19−http: live ("secure", public) on 8443
19+ http: live (insecure, loopback) on 12321
20−http: live (insecure, loopback) on 12321
20+ ~palnul_nocser:dojo>
21−~palnul_nocser:dojo>
22−```
23
21
24## Welcome to the dojo
22## Welcome to the dojo
25
23
26Wait, where are we?
24Wait, where are we?
27
25
28−We're playing in the Urbit `:dojo`. The dojo is sort of like a
26+We're playing in the Urbit `:dojo`. The dojo is sort of like a cross between a
29−cross between a Lisp REPL and the Unix shell. (If you know the
27+Lisp REPL and the Unix shell. (If you know the Unix shell, the Urbit command
30−Unix shell, the Urbit command line has the same history model and
28+line has the same history model and Emacs editing commands.)
31−Emacs editing commands.)
32
29
33−The dojo has its own command language, which is a superset of
30+The dojo has its own command language, which is a superset of Hoon. Any Hoon
34−Hoon. Any Hoon expression is a command. But not all commands
31+expression is a command. But not all commands are expressions.
35−are expressions.
36
32
37−(The dojo is not the only command-line app on your Urbit. By
33+(The dojo is not the only command-line app on your Urbit. By default there is
38−default there is one other, `:talk`. Since you're on a fakezod,
34+one other, `:talk`. Since you're on a fakezod, you can only talk to yourself. If
39−you can only talk to yourself. If you accidentally press `ctrl-x`,
35+you accidentally press `ctrl-x`, it will put you into `:talk`; press `ctrl-x`
40−it will put you into `:talk`; press `ctrl-x` again to switch back.)
36+again to switch back.)
41
37
42### Dojo variables
38### Dojo variables
43
39
44−Like the Unix shell, the dojo has command variables. Type
40+Like the Unix shell, the dojo has command variables. Type
45
41
46−```
42+ > =a 42
47−> =a 42
48−```
49
43
50−You've set the dojo variable `a` to the value `42`. The dojo
44+You've set the dojo variable `a` to the value `42`. The dojo puts this binding
51−puts this binding in the Hoon subject for expressions to use.
45+in the Hoon subject for expressions to use. So you can write:
52−So you can write:
53
46
54−```
47+ > a
55−> a
48+ 42
56−42
57
49
58−> [4 a]
50+ > [4 a]
59−[4 42]
51+ [4 42]
60
52
61−> ? [4 a]
53+ > ? [4 a]
62− {@ud @ud}
54+ {@ud @ud}
63−[4 42]
55+ [4 42]
64−```
65
56
66To unbind `a`, just omit the value:
57To unbind `a`, just omit the value:
67
58
68−```
59+ > =a
69−> =a
70−```
71
60
72−You removed `a` from the subject. (We can bind it again later, or
61+You removed `a` from the subject. (We can bind it again later, or bind over it
73−bind over it without removing it.)
62+without removing it.)
74
63
75−```
64+ > [4 a]
76−> [4 a]
65+ -find.a
77−-find.a
78−```
79
66
80Now trying to use `a` is an error.
67Now trying to use `a` is an error.
81
68
82## Subject-oriented programming
69## Subject-oriented programming
83
70
84−What is this *subject* thing, anyway? We put `a` in what?
71+What is this *subject* thing, anyway? We put `a` in what?
85
72
86−Most programming languages have something called a *scope* or an
73+Most programming languages have something called a *scope* or an *environment*
87−*environment* or a *context*. This is a data structure that
74+or a *context*. This is a data structure that contains the variables and
88−contains the variables and functions an expression can use.
75+functions an expression can use.
89
76
90−In a normal language, the environment is not a first-class value.
77+In a normal language, the environment is not a first-class value. You cannot
91−You cannot save an environment in a variable, compute an
78+save an environment in a variable, compute an expression against the environment
92−expression against the environment in the variable, etc, etc.
79+in the variable, etc, etc.
93
80
94−In Hoon or any "subject-oriented" language, an expression is
81+In Hoon or any "subject-oriented" language, an expression is executed against
95−executed against one normal value, the *subject*. In Hoon, of
82+one normal value, the *subject*. In Hoon, of course, the subject is a noun.
96−course, the subject is a noun.
97
83
98## First experiments
84## First experiments
99
85
100−Up to now, we've only built constants. By definition, a constant
86+Up to now, we've only built constants. By definition, a constant doesn't use the
101−doesn't use the subject.
87+subject.
102
88
103−It would be nice to start with the subject every dojo expression
89+It would be nice to start with the subject every dojo expression gets. But that
104−gets. But that subject is pretty complex. Instead, let's make a
90+subject is pretty complex. Instead, let's make a simple noun from the kinds of
105−simple noun from the kinds of atoms we've already learned about:
91+atoms we've already learned about:
106
92
107−```
93+ > =a [42 [%foo .6.28 ~m45] 0xdead.beef]
108−> =a [42 [%foo .6.28 ~m45] 0xdead.beef]
109
94
110−> ? a
95+ > ? a
111− {@ud {$foo @rs @dr} @ux}
96+ {@ud {$foo @rs @dr} @ux}
112−[42 [%foo .3.14 ~m45] 0xdead.beef]
97+ [42 [%foo .3.14 ~m45] 0xdead.beef]
113
98
114−> 420:a
99+ > 420:a
115−420
100+ 420
116−```
117
101
118−The syntax `expression:expression` uses the product of the second
102+The syntax `expression:expression` uses the product of the second expression as
119−expression as the subject of the first. So we're using `a` as
103+the subject of the first. So we're using `a` as the subject of the expression
120−the subject of the expression `420`.
104+`420`.
121
105
122−Of course, `420` is a constant expression. It doesn't care what
106+Of course, `420` is a constant expression. It doesn't care what the subject is.
123−the subject is. But:
107+But:
124
108
125−```
109+ > (add 2 2):a
126−> (add 2 2):a
110+ -find.add
127−-find.add
128−```
129
111
130−We can't use `a` as the subject of even a simple addition. How
112+We can't use `a` as the subject of even a simple addition. How could we? There's
131−could we? There's no `add` function in our `a`. It sometimes
113+no `add` function in our `a`. It sometimes takes new Hoon programmers a little
132−takes new Hoon programmers a little while before they realize
114+while before they realize that there really isn't a scope or environment in
133−that there really isn't a scope or environment in Hoon.
115+Hoon.
134
116
135## Tree geometry
117## Tree geometry
136
118
137−So what can we do with this subject? How can we get data out of
119+So what can we do with this subject? How can we get data out of it? Let's keep
138−it? Let's keep working with our little test subject `a`.
120+working with our little test subject `a`.
139
121
140−The most basic way to access the subject is *tree geometry*.
122+The most basic way to access the subject is *tree geometry*. Every noun is a
141−Every noun is a binary tree; every noun in the tree has a number
123+binary tree; every noun in the tree has a number which is its address. The root
142−which is its address. The root of the tree is `1`; if the
124+of the tree is `1`; if the address of a cell is `n`, its head is `2n` and its
143−address of a cell is `n`, its head is `2n` and its tail is
125+tail is `2n+1`.
144−`2n+1`.
145
126
146A picture might help:
127A picture might help:
147−```
128+
148− 1
129+ 1
149− / \
130+ / \
150− 2 3
131+ 2 3
151− / \ / \
132+ / \ / \
152− 4 5 6 7
133+ 4 5 6 7
153− / \
134+ / \
154− 14 15
135+ 14 15
155−```
136+
156−
137+The syntax for a fragment of the subject at a tree address, or *axis*, is just
157−The syntax for a fragment of the subject at a tree address, or
138+`+axis`. So:
158−*axis*, is just `+axis`. So:
139+
159−
140+ > +3:a
160−```
141+ [[%foo .3.14 ~m45] 0xdead.beef]
161−> +3:a
142+
162−[[%foo .3.14 ~m45] 0xdead.beef]
143+ > ? +7:a
163−
144+ @ux
164−> ? +7:a
145+ 0xdead.beef
165− @ux
146+
166−0xdead.beef
147+ > ? +13:a
167−
148+ {@rs @dr}
168−> ? +13:a
149+ [.3.14 ~m45]
169− {@rs @dr}
170−[.3.14 ~m45]
171−```
172
150
173Of course, the fragment retains the type it had when we built it.
151Of course, the fragment retains the type it had when we built it.
174
152
175## Labeled surfaces
153## Labeled surfaces
176
154
177−We can obviously get anything we want out of the subject by tree
155+We can obviously get anything we want out of the subject by tree geometry. But
178−geometry. But that's not really how we'd like to program. We'd
156+that's not really how we'd like to program. We'd much rather *label* subtrees
179−much rather *label* subtrees with convenient, memorable symbols.
157+with convenient, memorable symbols.
180
158
181For example, suppose our tree looked like:
159For example, suppose our tree looked like:
182
160
183−```
161+ 1
184− 1
162+ / \
185− / \
163+ x=2 3
186− x=2 3
164+ / \
187− / \
165+ y=6 z=7
188− y=6 z=7
166+ / \
189− / \
167+ p=14 15
190− p=14 15
168+ / \
191− / \
169+ q=30 r=31
192− q=30 r=31
193−```
194
170
195Then instead of writing `+2`, we could write `x`, and so on.
171Then instead of writing `+2`, we could write `x`, and so on.
196
172
197−Let's replace our test subject `a` with a new noun built around
173+Let's replace our test subject `a` with a new noun built around this tree of
198−this tree of *labeled surfaces*.
174+*labeled surfaces*.
199
175
200−```
176+ > =a [x=42 y=[p=%foo q=.6.28 r=~m45] z=0xdead.beef]
201−> =a [x=42 y=[p=%foo q=.6.28 r=~m45] z=0xdead.beef]
202
177
203−> ? x:a
178+ > ? x:a
204− @ud
179+ @ud
205−42
180+ 42
206
181
207−> ? y:a
182+ > ? y:a
208− {p/$foo q/@rs r/@dr}
183+ {p/$foo q/@rs r/@dr}
209−[p=%foo q=.6.28 r=~m45]
184+ [p=%foo q=.6.28 r=~m45]
210
185
211−> ? q.y:a
186+ > ? q.y:a
212− @rs
187+ @rs
213−.6.28
188+ .6.28
214−```
215
189
216−Surfaces might remind you of attributes in an object. But again,
190+Surfaces might remind you of attributes in an object. But again, attributes in
217−attributes in an object are a symbol table. There is no symbol
191+an object are a symbol table. There is no symbol table here -- just a tree
218−table here -- just a tree search.
192+search.
219
193
220−An expression like the label `x` or the axis `+2`, which
194+An expression like the label `x` or the axis `+2`, which designates a fragment
221−designates a fragment or feature of the subject, is called a
195+or feature of the subject, is called a *limb*. A list of limbs connected with
222−*limb*. A list of limbs connected with `.` is a *wing*.
196+`.` is a *wing*.
223
197
224−To remind you that you're not in Kansas anymore, the syntax for a
198+To remind you that you're not in Kansas anymore, the syntax for a wing, like
225−wing, like `q.y` above, means means "q inside y." In a classic
199+`q.y` above, means means "q inside y." In a classic object-oriented system, of
226−object-oriented system, of course, this would be written `y.q`,
200+course, this would be written `y.q`, that is, attribute `q` within object `y`.
227−that is, attribute `q` within object `y`. In Hoon, `q.y` means
201+In Hoon, `q.y` means "find limb `y`, then find limb `q` inside that."
228−"find limb `y`, then find limb `q` inside that."
229
202
230## Surfaces versus fragments
203## Surfaces versus fragments
231
204
232−Once again, Hoon has no symbol tables. When we put a surface
205+Once again, Hoon has no symbol tables. When we put a surface on a noun, the
233−on a noun, the surface label is part of that noun's type. It's
206+surface label is part of that noun's type. It's not part of the cell or other
234−not part of the cell or other structure that contains the noun.
207+structure that contains the noun. This will take a little getting used to:
235−This will take a little getting used to:
236
208
237−```
209+ > ? x:a
238−> ? x:a
210+ @ud
239− @ud
211+ 42
240−42
241
212
242−> ? +2:a
213+ > ? +2:a
243− x/@ud
214+ x/@ud
244−x=42
215+ x=42
245
216
246−? x.+2:a
217+ ? x.+2:a
247− @ud
218+ @ud
248−42
219+ 42
249−```
250
220
251−When we take fragment 2 of the labeled subject, the *surface
221+When we take fragment 2 of the labeled subject, the *surface comes with it*.
252−comes with it*. `+2` is not `42` but rather `x=42`. We can
222+`+2` is not `42` but rather `x=42`. We can take the surface off with `x.+2`.
253−take the surface off with `x.+2`.
254
223
255## The search algorithm
224## The search algorithm
256
225
257−In resolving a label to a fragment axis, we search the subject
226+In resolving a label to a fragment axis, we search the subject type in a
258−type in a depth-first traverse.
227+depth-first traverse.
259
228
260−If we find a surface that matches the limb, we succeed. If we
229+If we find a surface that matches the limb, we succeed. If we find a surface
261−find a surface that doesn't match the limb, we fail. If we find
230+that doesn't match the limb, we fail. If we find a cell, we search first head,
262−a cell, we search first head, then tail.
231+then tail.
263
232
264## Skipping
233## Skipping
265
234
266−What if the same label appears twice in one noun? Let's modify
235+What if the same label appears twice in one noun? Let's modify our subject
267−our subject slightly as a test case:
236+slightly as a test case:
268
237
269−```
238+ > ? +7:a
270−> ? +7:a
239+ z/@ux
271− z/@ux
240+ z=0xdead.beef
272−z=0xdead.beef
273
241
274−> =a a(+7 x=z:a)
242+ > =a a(+7 x=z:a)
275
243
276−> ? a
244+ > ? a
277− {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}
245+ {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}
278−[x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
246+ [x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
279
247
280−> ? x:a
248+ > ? x:a
281− @ud
249+ @ud
282−42
250+ 42
283
251
284−> ? ^x:a
252+ > ? ^x:a
285− @ux
253+ @ux
286−0xdead.beef
254+ 0xdead.beef
287−```
288
255
289−We don't want to hide any part of the noun from an expression.
256+We don't want to hide any part of the noun from an expression. So the modified
290−So the modified limb `^x` *skips* the first match with a surface
257+limb `^x` *skips* the first match with a surface `x`, getting us to our
291−`x`, getting us to our `0xdead.beef`. This is a unary notation:
258+`0xdead.beef`. This is a unary notation: `^^x` would skip the first two matches.
292−`^^x` would skip the first two matches.
293
259
294## Editing
260## Editing
295
261
296Above, we mutated our first noun with the flashy `a(+7 x=z:a)`.
262Above, we mutated our first noun with the flashy `a(+7 x=z:a)`.
297
263
298−Of course we are not really "modifying" a noun. Hoon is a pure
264+Of course we are not really "modifying" a noun. Hoon is a pure language. We are
299−language. We are making an edited copy. `a(+7 x=z:a)` means "a
265+making an edited copy. `a(+7 x=z:a)` means "a copy of `a`, with limb `+7`
300−copy of `a`, with limb `+7` replaced by the product of `x=z:a`."
266+replaced by the product of `x=z:a`."
301
267
302We can edit multiple (disjoint) wings in one expression:
268We can edit multiple (disjoint) wings in one expression:
303
269
304−```
270+ > ? a(+7 [%hello %world], q.y .3.14)
305−> ? a(+7 [%hello %world], q.y .3.14)
271+ {x/@ud y/{p/$foo q/@rs r/@dr} $hello $world}
306− {x/@ud y/{p/$foo q/@rs r/@dr} $hello $world}
272+ [x=42 y=[p=%foo q=.3.14 r=~m45] %hello %world]
307−[x=42 y=[p=%foo q=.3.14 r=~m45] %hello %world]
308−```
309
273
310−Notice that nothing stopped us from changing `+7` from
274+Notice that nothing stopped us from changing `+7` from `x=0xdead.beef` to
311−`x=0xdead.beef` to `[%hello %world]`. Since the mutated
275+`[%hello %world]`. Since the mutated copy is a different noun, it has a
312−copy is a different noun, it has a different type.
276+different type.
313
277
314−(And remember that since `[x y z]` means `[x [y z]]`,
278+(And remember that since `[x y z]` means `[x [y z]]`, `[x y [%hello %world]]`
315−`[x y [%hello %world]]` means `[x y %hello %world]`.)
279+means `[x y %hello %world]`.)
316
280
317## Cores and arms
281## Cores and arms
318
282
319−It should be reasonably clear how Hoon's system of fragments and
283+It should be reasonably clear how Hoon's system of fragments and surfaces
320−surfaces parallels variables, records or structures in a
284+parallels variables, records or structures in a classical language. But where is
321−classical language. But where is Hoon's equivalent of objects:
285+Hoon's equivalent of objects: values which bind code to data?
322−values which bind code to data?
323
286
324−The Hoon equivalent of an object is a *core*. As usual in Hoon,
287+The Hoon equivalent of an object is a *core*. As usual in Hoon, the resemblance
325−the resemblance is inexact. A core has a set of computed
288+is inexact. A core has a set of computed attributes, or *arms*. Each arm is an
326−attributes, or *arms*. Each arm is an expression whose subject
289+expression whose subject is the core. There is no argument, so an arm is not a
327−is the core. There is no argument, so an arm is not a method.
290+method.
328
291
329−Physically, a core is always a cell `[battery payload]`, where
292+Physically, a core is always a cell `[battery payload]`, where the *payload* is
330−the *payload* is any noun, and the *battery* is a tree of
293+any noun, and the *battery* is a tree of formulas in Nock (Hoon's functional
331−formulas in Nock (Hoon's functional assembly language) that
294+assembly language) that implements the arms. (A battery is like a C++ vtable,
332−implements the arms. (A battery is like a C++ vtable, sort of.)
295+sort of.)
333
296
334## The search algorithm
297## The search algorithm
335
298
336−As you'd expect with a computed attribute, there is only one
299+As you'd expect with a computed attribute, there is only one namespace and one
337−namespace and one search algorithm. We just extend the algorithm
300+search algorithm. We just extend the algorithm described above. And an
338−described above. And an expression which uses a label can't know
301+expression which uses a label can't know whether the subject will resolve it to
339−whether the subject will resolve it to a surface fragment (a
302+a surface fragment (a *leg*) or to an arm.
340−*leg*) or to an arm.
341
303
342−If we're searching a core and the label matches one of the core's
304+If we're searching a core and the label matches one of the core's arms, we
343−arms, we compute the arm against the core. If the label is not
305+compute the arm against the core. If the label is not found, we search into the
344−found, we search into the payload of the core.
306+payload of the core.
345
307
346## Some core examples
308## Some core examples
347
309
348−When you type these examples, make sure you get the number of
310+When you type these examples, make sure you get the number of spaces exactly
349−spaces exactly right. Hoon syntax is not whitespace-sensitive
311+right. Hoon syntax is not whitespace-sensitive in general, except that "one
350−in general, except that "one space" is always a different token
312+space" is always a different token from "two or more spaces."
351−from "two or more spaces."
352
313
353−```
314+ > ? x:a
354−> ? x:a
315+ @ud
355− @ud
316+ 42
356−42
357
317
358−> ? :per a x
318+ > ? :per a x
359− @ud
319+ @ud
360−42
320+ 42
361
321
362−> =b :per a :core ++ foo x ++ bar [q.y foo] --
322+ > =b :per a :core ++ foo x ++ bar [q.y foo] --
363−```
364
323
365−We're seeing some Hoon regular forms for the first time. `x:a`
324+We're seeing some Hoon regular forms for the first time. `x:a` turns out to be
366−turns out to be an irregular relative of the regular `:per` form.
325+an irregular relative of the regular `:per` form.
367
326
368−We'll get more into syntax in a later Urbyte. If some of these
327+We'll get more into syntax in a later Urbyte. If some of these examples seem
369−examples seem syntactically a little funky, this isn't usually
328+syntactically a little funky, this isn't usually way we'd do them in "real
370−way we'd do them in "real life." We don't normally write cores
329+life." We don't normally write cores on a single command line. In a file this
371−on a single command line. In a file this would be:
330+would be:
372
331
373−```
332+ :per a
374− :per a
333+ :core
375− :core
334+ ++ foo x
376− ++ foo x
335+ ++ bar [q.y foo]
377− ++ bar [q.y foo]
336+ --
378− --
379−```
380
337
381−which is much more readable, at least once you get used to it.
338+which is much more readable, at least once you get used to it. Let's look at the
382−Let's look at the core itself:
339+core itself:
383
340
384−```
341+ > b
385−> b
342+ <2.spy {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}>
386−<2.spy {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}>
387
343
388−> ? +2:b
344+ > ? +2:b
389− *
345+ *
390−[[0 6] [0 58] 9 4 0 1]
346+ [[0 6] [0 58] 9 4 0 1]
391
347
392−> ? +3:b
348+ > ? +3:b
393− {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}
349+ {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}
394−[x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
350+ [x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
395−```
396
351
397−When we print `b`, or any core, we see that typed noun printing
352+When we print `b`, or any core, we see that typed noun printing can't be an
398−can't be an invertible function in Hoon.
353+invertible function in Hoon.
399
354
400−We actually can read the battery (with `+2:b` above), but it's
355+We actually can read the battery (with `+2:b` above), but it's just Nock code,
401−just Nock code, not typed data. `<2.spy>` tells us the battery
356+not typed data. `<2.spy>` tells us the battery has two arms and hashes to the
402−has two arms and hashes to the three-letter code `spy`.
357+three-letter code `spy`.
403
358
404−When printing a full core, we don't even print the payload data,
359+When printing a full core, we don't even print the payload data, just its type.
405−just its type. But our `+3:b` confirms the `[battery payload]`
360+But our `+3:b` confirms the `[battery payload]` structure of the core. Remember,
406−structure of the core. Remember, `+2` is the head and `+3` is
361+`+2` is the head and `+3` is the tail.
407−the tail.
408
362
409Finally, let's use this thing:
363Finally, let's use this thing:
410
364
411−```
365+ > ? foo:b
412−> ? foo:b
366+ @ud
413− @ud
367+ 42
414−42
415
368
416−> ? bar:b
369+ > ? bar:b
417− {@rs @ud}
370+ {@rs @ud}
418−[.6.28 42]
371+ [.6.28 42]
419
372
420−? x:b
373+ ? x:b
421− @ud
374+ @ud
422−42
375+ 42
423−```
424
376
425Seems legit.
377Seems legit.
426
378
427## The dojo subject
379## The dojo subject
428
380
429−Now that we've made a test subject which is a core, we can look
381+Now that we've made a test subject which is a core, we can look directly at the
430−directly at the real dojo subject:
382+real dojo subject:
431−
383+
432−```
384+ > +1
433−> +1
385+ [ [ [ a=[x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
434−[ [ [ a=[x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
386+ b=<2.spy {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}>
435− b=<2.spy {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}>
387+ ]
388+ now=~2016.7.18..08.05.15..0b5c
389+ eny=0vl.hsfrh.tg642.g5o21.1n80b.0h141.0vuan.pm8o4.77ovg.96rk0.n26nn
390+ our=~palnul-wolmev-bonfex-danred--tilwen-midlux-bolfus-nocser
391+ ]
392+ ~
393+ <283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>
436 ]
394 ]
437− now=~2016.7.18..08.05.15..0b5c
395+
438− eny=0vl.hsfrh.tg642.g5o21.1n80b.0h141.0vuan.pm8o4.77ovg.96rk0.n26nn
396+We see a bunch of surfaces: our custom `a` and `b`; standard surfaces for
439− our=~palnul-wolmev-bonfex-danred--tilwen-midlux-bolfus-nocser
397+identity, date, and entropy; and a ginormous core stack with *over eight
440− ]
398+hundred* total arms.
441− ~
399+
442− <283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>
400+To be exact, the syntax `<283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>` denotes a
443−]
401+stack of six cores, each of whose payload is another core, until the innermost
444−```
402+payload `%151` (the Hoon version number). This stack is our standard library,
445−
403+from which comes `add` and other fine functions. (We don't know how to write a
446−We see a bunch of surfaces: our custom `a` and `b`; standard
404+Hoon function yet, but we're getting way closer.)
447−surfaces for identity, date, and entropy; and a ginormous core
448−stack with *over eight hundred* total arms.
449−
450−To be exact, the syntax `<283.xzp 42.ajz 407.xtr 110.xht 1.ztu
451−$151>` denotes a stack of six cores, each of whose payload is
452−another core, until the innermost payload `%151` (the Hoon
453−version number). This stack is our standard library, from which
454−comes `add` and other fine functions. (We don't know how to
455−write a Hoon function yet, but we're getting way closer.)
456
405
457Let's clean up our subject, just to be hygienic:
406Let's clean up our subject, just to be hygienic:
458
407
459−```
408+ > =a
460−> =a
409+ > =b
461−> =b
462
410
463−> +1
411+ > +1
464− now=~2016.7.18..08.10.42..36a0
412+ now=~2016.7.18..08.10.42..36a0
465− eny=0vf.8j71j.hlqim.k340u.q0ual.v2cd8.cfa65.vkbte.e733g.ehihg.e8aog
413+ eny=0vf.8j71j.hlqim.k340u.q0ual.v2cd8.cfa65.vkbte.e733g.ehihg.e8aog
466−[ [ our=~palnul-wolmev-bonfex-danred--tilwen-midlux-bolfus-nocser
414+ [ [ our=~palnul-wolmev-bonfex-danred--tilwen-midlux-bolfus-nocser
467− ]
415+ ]
468− ~
416+ ~
469− <283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>
417+ <283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>
470−]
418+ ]
471−```
472
419
473−Again, remember that `=a` is not Hoon syntax, but `:dojo` syntax.
420+Again, remember that `=a` is not Hoon syntax, but `:dojo` syntax. How the dojo
474−How the dojo assembles this subject remains a mystery. But it's
421+assembles this subject remains a mystery. But it's not a particularly
475−not a particularly interesting mystery.
422+interesting mystery.
476
423
477## And we're done
424## And we're done
478
425
479−And we're done! Type `|exit` to exit:
426+And we're done! Type `|exit` to exit:
480
427
481−```
428+ ~palnul_nocser:dojo> |exit
482−~palnul_nocser:dojo> |exit
429+ %drum-exit
483−%drum-exit
430+ $
484−$
485−```
486
431
487## Questions and/or exercises
432## Questions and/or exercises
488
433
489As always, these questions are optional and only for fun.
434As always, these questions are optional and only for fun.
490
435
491−What's the best way to construct a core which acts as a function?
436+What's the best way to construct a core which acts as a function? What do you do
492−What do you do to call the function?
437+to call the function?
493
438
494−Why don't any other languages use subject-oriented programming?
439+Why don't any other languages use subject-oriented programming? Which languages
495−Which languages are most similar to it?
440+are most similar to it?
docs/community-docs.md+20−21
@@ -1,31 +1,30 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 7
3sort: 7
4---
4---
5
5
6−
7# Community docs
6# Community docs
8
7
9<div>
8<div>
10−
11<ul>
9<ul>
12−
10+<li>
13−<li><code>~master-morzod</code>
11+<code>~master-morzod</code> <br />
14−<br />
12+<a href="https://mardev.urbit.org/docs/">docs</a>, <code>fora</code>
15−<a href="https://mardev.urbit.org/docs/">docs</a>, <code>fora</code> <a href="https://urbit.org/fora/posts/~2016.12.25..06.35.44..a1ec~/">post</a></li>
13+<a href="https://urbit.org/fora/posts/~2016.12.25..06.35.44..a1ec~/">post</a>
16−
14+</li>
17−<li><code>~hidduc-posmeg</code>
15+<li>
18−<br />
16+<code>~hidduc-posmeg</code> <br />
19−<a href="https://fosnut-dandut.urbit.org/pages/hidducs-notes/tutorial/">docs</a>, <code>fora</code> <a href="https://urbit.org/fora/posts/~2017.2.12..21.54.40..6fde~/">post</a></li>
17+<a href="https://fosnut-dandut.urbit.org/pages/hidducs-notes/tutorial/">docs</a>,
20−
18+<code>fora</code>
21−<li><code>~palfun-foslup</code>
19+<a href="https://urbit.org/fora/posts/~2017.2.12..21.54.40..6fde~/">post</a>
22−<br />
20+</li>
23−<a href="https://github.com/Fang-/Urbit-By-Doing">Urbit by Doing</a></li>
21+<li>
24−
22+<code>~palfun-foslup</code> <br />
25−<li><code>@knubie</code>
23+<a href="https://github.com/Fang-/Urbit-By-Doing">Urbit by Doing</a>
26−<br />
24+</li>
27−<a href="https://github.com/knubie/learning-hoon">Learning Hoon</a></li>
25+<li>
28−
26+<code>@knubie</code> <br />
27+<a href="https://github.com/knubie/learning-hoon">Learning Hoon</a>
28+</li>
29</ul>
29</ul>
30−
31</div>
30</div>
docs/community-projects.md+15−19
@@ -1,30 +1,26 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3sort: 8
3sort: 8
4---
4---
5
5
6−
7# Community projects
6# Community projects
8
7
9<div>
8<div>
10−
11<ul>
9<ul>
12−
10+<li>
13−<li><a href="https://github.com/ponnys-podfer/yint">Yint: A Tiny MUD</a>
11+<a href="https://github.com/ponnys-podfer/yint">Yint: A Tiny MUD</a> <br />
14−<br />
12+<code>~ponnys-podfer</code>
15−<code>~ponnys-podfer</code></li>
13+</li>
16−
14+<li>
17−<li><a href="https://github.com/Fang-/talkbot">~talkbot</a>
15+<a href="https://github.com/Fang-/talkbot">~talkbot</a> <br />
18−<br />
16+<code>~palfun-foslup</code>
19−<code>~palfun-foslup</code></li>
17+</li>
20−
18+<li>
21−<li><a href="https://github.com/Fang-/urbit-string">String processing</a>
19+<a href="https://github.com/Fang-/urbit-string">String processing</a> <br />
22−<br />
20+<code>~palfun-foslup</code>
23−<code>~palfun-foslup</code></li>
21+</li>
24−
22+<li>
25−<li><a href="community-projects/nock-implementations">Nock implementations</a>
23+<a href="community-projects/nock-implementations">Nock implementations</a>
26</li>
24</li>
27−
28</ul>
25</ul>
29−
30</div>
26</div>
docs/community-projects/nock-implementations.md+2−2
@@ -1,8 +1,8 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3+next: True
3sort: 4
4sort: 4
4title: Nock implementations
5title: Nock implementations
5−next: true
6---
6---
7
7
8# Community Nock implementations
8# Community Nock implementations
docs/community-projects/nock-implementations/clojure.md+117−118
@@ -1,125 +1,124 @@
1---
1---
2−navhome: /docs/
2+navhome: '/docs/'
3−title: Clojure
3+next: True
4sort: 5
4sort: 5
5−next: true
5+title: Clojure
6---
6---
7
7
8# Clojure
8# Clojure
9
9
10−From [Matt Earnshaw](https://github.com/mattearnshaw/anock/blob/master/src/anock/core.clj):
10+From [Matt
11−
11+Earnshaw](https://github.com/mattearnshaw/anock/blob/master/src/anock/core.clj):
12−```
12+
13−(ns anock.core
13+ (ns anock.core
14− (:import anock.NockException))
14+ (:import anock.NockException))
15−
15+
16−(declare atom? cell? cell)
16+ (declare atom? cell? cell)
17−
17+
18−(defn noun?
18+ (defn noun?
19− "A noun is an atom or a cell."
19+ "A noun is an atom or a cell."
20− [noun & ns]
20+ [noun & ns]
21− (if ns false
21+ (if ns false
22− (or (atom? noun) (cell? noun))))
22+ (or (atom? noun) (cell? noun))))
23−
23+
24−(defn atom?
24+ (defn atom?
25− "An atom is a natural number."
25+ "An atom is a natural number."
26− [noun & ns]
26+ [noun & ns]
27− (if ns false
27+ (if ns false
28− (and (integer? noun) (>= noun 0))))
28+ (and (integer? noun) (>= noun 0))))
29−
29+
30−(defn cell?
30+ (defn cell?
31− "A cell is an ordered pair of nouns."
31+ "A cell is an ordered pair of nouns."
32− [noun]
32+ [noun]
33− (cond
33+ (cond
34− (atom? noun) false
34+ (atom? noun) false
35− (nil? noun) false
35+ (nil? noun) false
36− (not= 2 (count noun)) false
36+ (not= 2 (count noun)) false
37− :else (and (noun? (first noun))
37+ :else (and (noun? (first noun))
38− (noun? (second noun)))))
38+ (noun? (second noun)))))
39−
39+
40−(defn tis
40+ (defn tis
41− "= (pronounced 'tis') tests a cell for equality."
41+ "= (pronounced 'tis') tests a cell for equality."
42− [noun]
42+ [noun]
43− (if (atom? noun) (throw (anock.NockException. "Cannot tis an atom."))
43+ (if (atom? noun) (throw (anock.NockException. "Cannot tis an atom."))
44− (let [[a b] noun]
44+ (let [[a b] noun]
45− (if (= a b) 0 1))))
45+ (if (= a b) 0 1))))
46−
46+
47−(defn wut
47+ (defn wut
48− "? (pronounced 'wut') tests whether a noun is a cell."
48+ "? (pronounced 'wut') tests whether a noun is a cell."
49− [noun]
49+ [noun]
50− (cond
50+ (cond
51− (atom? noun) 1
51+ (atom? noun) 1
52− (cell? noun) 0
52+ (cell? noun) 0
53− :else (throw (anock.NockException. "Invalid noun."))))
53+ :else (throw (anock.NockException. "Invalid noun."))))
54−
54+
55−(defn lus
55+ (defn lus
56− "+ (pronounced 'lus') adds 1 to an atom."
56+ "+ (pronounced 'lus') adds 1 to an atom."
57− [noun]
57+ [noun]
58− (if (atom? noun) (inc noun)
58+ (if (atom? noun) (inc noun)
59− (throw (anock.NockException. "Can only lus atoms."))))
59+ (throw (anock.NockException. "Can only lus atoms."))))
60−
60+
61−(defn fas
61+ (defn fas
62− "/ (pronounced 'fas') is a tree address function."
62+ "/ (pronounced 'fas') is a tree address function."
63− [noun]
63+ [noun]
64− (if (atom? noun) (throw (anock.NockException. "Cannot fas an atom."))
64+ (if (atom? noun) (throw (anock.NockException. "Cannot fas an atom."))
65− (let [[a b] (cell noun)]
65+ (let [[a b] (cell noun)]
66− (assert (and (pos? a) (atom? a)) "Subject of fas must be a positive atom.")
66+ (assert (and (pos? a) (atom? a)) "Subject of fas must be a positive atom.")
67− (if (and (not (coll? b)) (or (= 2 a) (= 3 a)))
67+ (if (and (not (coll? b)) (or (= 2 a) (= 3 a)))
68− (throw (anock.NockException. (str "Cannot fas noun: " noun))))
68+ (throw (anock.NockException. (str "Cannot fas noun: " noun))))
69+ (cond
70+ (= 1 a) b
71+ (= 2 a) (first b)
72+ (= 3 a) (second b)
73+ (even? a) (fas [2 (fas [(/ a 2) b])])
74+ (odd? a) (fas [3 (fas [(/ (dec a) 2) b])])))))
75+
76+ (defn tar
77+ "* (pronounced 'tar') means Nock"
78+ [noun]
79+ (if (atom? noun) (throw (anock.NockException. "Cannot tar an atom."))
80+ (try
81+ (let [noun (cell noun) [x [y z]] noun]
82+ (cond
83+ (cell? y) (cell (tar [x y]) (tar [x z]))
84+ (zero? y) (fas [z x])
85+ (= 1 y) z
86+ (= 3 y) (wut (tar [x z]))
87+ (= 4 y) (lus (tar [x z]))
88+ (= 5 y) (tis (tar [x z]))
89+ :else (let [[p q] z]
90+ (cond
91+ (= 2 y) (tar [(tar [x p]) (tar [x q])])
92+ (= 6 y) (tar [x 2 [0 1] 2 [1 (first q) (second q)]
93+ [1 0] 2 [1 2 3] [1 0] 4 4 p])
94+ (= 7 y) (tar [x 2 p 1 q])
95+ (= 8 y) (tar [x 7 [[7 [0 1] p] 0 1] q])
96+ (= 9 y) (tar [x 7 q 2 [0 1] 0 p])
97+ (= 10 y) (if (cell? p)
98+ (tar [x 8 (second p) 7 [0 3] q])
99+ (tar [x q]))))))
100+ (catch RuntimeException e
101+ (throw (anock.NockException. (str "Cannot tar the noun " noun)))))))
102+
103+ (def nock tar)
104+
105+ ; Some convenience functions
106+ (defn apply* [f x]
107+ (if (and (= 1 (count x)) (coll? (first x)))
108+ (apply f x)
109+ (f x)))
110+
111+ (defn bracket
112+ "[a b c] -> [a [b c]]"
113+ [[a & b :as c]]
114+ (let [b (vec b)]
69 (cond
115 (cond
70− (= 1 a) b
116+ (and (noun? a) (apply noun? b)) (vec c)
71− (= 2 a) (first b)
117+ (apply noun? b) (apply vector (bracket a) b)
72− (= 3 a) (second b)
118+ (noun? a) [a (apply* bracket b)]
73− (even? a) (fas [2 (fas [(/ a 2) b])])
119+ :else [(bracket a) (apply* bracket b)])))
74− (odd? a) (fas [3 (fas [(/ (dec a) 2) b])])))))
120+
75−
121+ (defn cell [& nouns]
76−(defn tar
122+ (if (apply atom? nouns)
77− "* (pronounced 'tar') means Nock"
123+ (throw (anock.NockException. "Cannot convert atom to cell."))
78− [noun]
124+ (apply* bracket nouns)))
79− (if (atom? noun) (throw (anock.NockException. "Cannot tar an atom."))
80− (try
81− (let [noun (cell noun) [x [y z]] noun]
82− (cond
83− (cell? y) (cell (tar [x y]) (tar [x z]))
84− (zero? y) (fas [z x])
85− (= 1 y) z
86− (= 3 y) (wut (tar [x z]))
87− (= 4 y) (lus (tar [x z]))
88− (= 5 y) (tis (tar [x z]))
89− :else (let [[p q] z]
90− (cond
91− (= 2 y) (tar [(tar [x p]) (tar [x q])])
92− (= 6 y) (tar [x 2 [0 1] 2 [1 (first q) (second q)]
93− [1 0] 2 [1 2 3] [1 0] 4 4 p])
94− (= 7 y) (tar [x 2 p 1 q])
95− (= 8 y) (tar [x 7 [[7 [0 1] p] 0 1] q])
96− (= 9 y) (tar [x 7 q 2 [0 1] 0 p])
97− (= 10 y) (if (cell? p)
98− (tar [x 8 (second p) 7 [0 3] q])
99− (tar [x q]))))))
100− (catch RuntimeException e
101− (throw (anock.NockException. (str "Cannot tar the noun " noun)))))))
102−
103−(def nock tar)
104−
105−; Some convenience functions
106−(defn apply* [f x]
107− (if (and (= 1 (count x)) (coll? (first x)))
108− (apply f x)
109− (f x)))
110−
111−(defn bracket
112− "[a b c] -> [a [b c]]"
113− [[a & b :as c]]
114− (let [b (vec b)]
115− (cond
116− (and (noun? a) (apply noun? b)) (vec c)
117− (apply noun? b) (apply vector (bracket a) b)
118− (noun? a) [a (apply* bracket b)]
119− :else [(bracket a) (apply* bracket b)])))
120−
121−(defn cell [& nouns]
122− (if (apply atom? nouns)
123− (throw (anock.NockException. "Cannot convert atom to cell."))
124− (apply* bracket nouns)))
125−```
docs/community-projects/nock-implementations/csharp.md+377−379

This diff is too big to show here.

docs/community-projects/nock-implementations/groovy.md+177−178

This diff is too big to show here.

docs/community-projects/nock-implementations/haskell.md+56−57

This diff is too big to show here.

docs/community-projects/nock-implementations/hoon.md+47−49

This diff is too big to show here.

docs/community-projects/nock-implementations/javascript.md+355−357

This diff is too big to show here.

docs/community-projects/nock-implementations/python.md+75−77

This diff is too big to show here.

docs/community-projects/nock-implementations/ruby.md+130−131

This diff is too big to show here.

docs/community-projects/nock-implementations/scala.md+68−69

This diff is too big to show here.

docs/community-projects/nock-implementations/scheme.md+112−113

This diff is too big to show here.

docs/community-projects/nock-implementations/swift.md+317−319

This diff is too big to show here.

docs/community-projects/string-processing.md+6−8

This diff is too big to show here.

docs/community-projects/talkbot.md+4−13

This diff is too big to show here.

docs/community-projects/yint.md+4−13

This diff is too big to show here.

docs/hoon.md+3−3

This diff is too big to show here.

docs/hoon/advanced.md+140−159

This diff is too big to show here.

docs/hoon/basic.md+123−139

This diff is too big to show here.

docs/hoon/concepts.md+134−154

This diff is too big to show here.

docs/hoon/demo.md+43−47

This diff is too big to show here.

docs/hoon/examples.md+453−512

This diff is too big to show here.

docs/hoon/exercises.md+6−7

This diff is too big to show here.

docs/hoon/exercises/data-structures.md+30−40

This diff is too big to show here.

docs/hoon/exercises/fizzbuzz.md+25−28

This diff is too big to show here.

docs/hoon/exercises/life.md+94−110

This diff is too big to show here.

docs/hoon/exercises/lists.md+22−27

This diff is too big to show here.

docs/hoon/exercises/math.md+12−17

This diff is too big to show here.

docs/hoon/irregular.md+114−95

This diff is too big to show here.

docs/hoon/library.md+1−1

This diff is too big to show here.

docs/hoon/library/1a.md+65−123

This diff is too big to show here.

docs/hoon/library/1b.md+21−37

This diff is too big to show here.

docs/hoon/library/1c.md+51−103

This diff is too big to show here.

docs/hoon/library/2a.md+75−141

This diff is too big to show here.

docs/hoon/library/2b.md+191−347

This diff is too big to show here.

docs/hoon/library/2c.md+74−150

This diff is too big to show here.

docs/hoon/library/2d.md+25−49

This diff is too big to show here.

docs/hoon/library/2e.md+22−41

This diff is too big to show here.

docs/hoon/library/2f.md+29−63

This diff is too big to show here.

docs/hoon/library/2g.md+1−2

This diff is too big to show here.

docs/hoon/library/2h.md+77−152

This diff is too big to show here.

docs/hoon/library/2i.md+46−55

This diff is too big to show here.

docs/hoon/library/2j.md+17−24

This diff is too big to show here.

docs/hoon/library/2k.md+46−84

This diff is too big to show here.

docs/hoon/library/2l.md+1−2

This diff is too big to show here.

docs/hoon/library/2m.md+1−2

This diff is too big to show here.

docs/hoon/library/2n.md+26−54

This diff is too big to show here.

docs/hoon/library/2o.md+21−44

This diff is too big to show here.

docs/hoon/library/2p.md+30−52

This diff is too big to show here.

docs/hoon/library/2q.md+49−85

This diff is too big to show here.

docs/hoon/library/3a.md+116−205

This diff is too big to show here.

docs/hoon/library/3b.md+1−2

This diff is too big to show here.

docs/hoon/library/3c.md+99−167

This diff is too big to show here.

docs/hoon/library/3d.md+13−31

This diff is too big to show here.

docs/hoon/library/3e.md+4−8

This diff is too big to show here.

docs/hoon/library/3f.md+108−182

This diff is too big to show here.

docs/hoon/library/3g.md+49−96

This diff is too big to show here.

docs/hoon/library/4a.md+30−52

This diff is too big to show here.

docs/hoon/library/4b.md+97−190

This diff is too big to show here.

docs/hoon/library/4c.md+33−66

This diff is too big to show here.

docs/hoon/library/4d.md+14−27

This diff is too big to show here.

docs/hoon/library/4e.md+70−136

This diff is too big to show here.

docs/hoon/library/4f.md+148−286

This diff is too big to show here.

docs/hoon/library/4g.md+24−47

This diff is too big to show here.

docs/hoon/library/4h.md+110−210

This diff is too big to show here.

docs/hoon/library/4i.md+1−2

This diff is too big to show here.

docs/hoon/library/4j.md+310−545

This diff is too big to show here.

docs/hoon/library/4k.md+138−267

This diff is too big to show here.

docs/hoon/library/4l.md+75−157

This diff is too big to show here.

docs/hoon/library/4m.md+57−111

This diff is too big to show here.

docs/hoon/library/4n.md+54−97

This diff is too big to show here.

docs/hoon/library/4o.md+107−182

This diff is too big to show here.

docs/hoon/library/5a.md+1−2

This diff is too big to show here.

docs/hoon/library/5b.md+1−2

This diff is too big to show here.

docs/hoon/library/5c.md+1−2

This diff is too big to show here.

docs/hoon/library/5d.md+1−2

This diff is too big to show here.

docs/hoon/library/5e.md+1−2

This diff is too big to show here.

docs/hoon/library/5f.md+31−61

This diff is too big to show here.

docs/hoon/library/5g.md+1−1

This diff is too big to show here.

docs/hoon/library/zuse.md+1−1

This diff is too big to show here.

docs/hoon/library/zuse/core/crua.md+1−6

This diff is too big to show here.

docs/hoon/library/zuse/core/epur.md+107−213

This diff is too big to show here.

docs/hoon/library/zuse/core/fu.md+2−5

This diff is too big to show here.

docs/hoon/library/zuse/core/jo.md+131−239

This diff is too big to show here.

docs/hoon/library/zuse/core/poja.md+114−209

This diff is too big to show here.

docs/hoon/library/zuse/core/poxa.md+50−99

This diff is too big to show here.

docs/hoon/library/zuse/core/poxo.md+25−52

This diff is too big to show here.

docs/hoon/library/zuse/core/yu.md+23−35

This diff is too big to show here.

docs/hoon/library/zuse/diff.md+82−143

This diff is too big to show here.

docs/hoon/library/zuse/gate/clan.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/dawn.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/daws.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/deal.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/deft.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/dust.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/earl.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/earn.md+20−38

This diff is too big to show here.

docs/hoon/library/zuse/gate/fain.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/feel.md+6−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/file.md+6−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/foal.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/fray.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/fuel.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/furl.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/glam.md+4−10

This diff is too big to show here.

docs/hoon/library/zuse/gate/glon.md+4−10

This diff is too big to show here.

docs/hoon/library/zuse/gate/gnom.md+4−10

This diff is too big to show here.

docs/hoon/library/zuse/gate/jape.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/jesc.md+7−13

This diff is too big to show here.

docs/hoon/library/zuse/gate/joba.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/jobe.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/jone.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/lead.md+6−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/mang.md+7−15

This diff is too big to show here.

docs/hoon/library/zuse/gate/meat.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/moon.md+6−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/mung.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/perk.md+16−23

This diff is too big to show here.

docs/hoon/library/zuse/gate/pojo.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/saxo.md+6−13

This diff is too big to show here.

docs/hoon/library/zuse/gate/scanf.md+17−19

This diff is too big to show here.

docs/hoon/library/zuse/gate/sein.md+5−10

This diff is too big to show here.

docs/hoon/library/zuse/gate/sifo.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/stud.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/taco.md+4−10

This diff is too big to show here.

docs/hoon/library/zuse/gate/tact.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/tame.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/tell.md+7−14

This diff is too big to show here.

docs/hoon/library/zuse/gate/tome.md+6−13

This diff is too big to show here.

docs/hoon/library/zuse/gate/tope.md+3−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/txml.md+5−11

This diff is too big to show here.

docs/hoon/library/zuse/gate/unt.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/urld.md+5−12

This diff is too big to show here.

docs/hoon/library/zuse/gate/urle.md+7−15

This diff is too big to show here.

docs/hoon/library/zuse/mold/udal.md+3−7

This diff is too big to show here.

docs/hoon/library/zuse/mold/udon.md+1−4

This diff is too big to show here.

docs/hoon/library/zuse/mold/umph.md+1−4

This diff is too big to show here.

docs/hoon/library/zuse/mold/unce.md+1−2

This diff is too big to show here.

docs/hoon/library/zuse/mold/upas.md+1−4

This diff is too big to show here.

docs/hoon/library/zuse/mold/urge.md+1−4

This diff is too big to show here.

docs/hoon/library/zuse/rsa.md+24−26

This diff is too big to show here.

docs/hoon/mission.md+28−34

This diff is too big to show here.

docs/hoon/reference.md+11−12

This diff is too big to show here.

docs/hoon/syntax.md+228−262

This diff is too big to show here.

docs/hoon/troubleshooting.md+97−120

This diff is too big to show here.

docs/hoon/twig.md+44−50

This diff is too big to show here.

docs/hoon/twig/atom.md+3−6

This diff is too big to show here.

docs/hoon/twig/atom/knit.md+9−17

This diff is too big to show here.

docs/hoon/twig/atom/path.md+9−11

This diff is too big to show here.

docs/hoon/twig/atom/rock.md+12−16

This diff is too big to show here.

docs/hoon/twig/atom/sand.md+40−43

This diff is too big to show here.

docs/hoon/twig/bar-core.md+14−16

This diff is too big to show here.

docs/hoon/twig/bar-core/cab-door.md+46−60

This diff is too big to show here.

docs/hoon/twig/bar-core/cen-core.md+26−32

This diff is too big to show here.

docs/hoon/twig/bar-core/col-gasp.md+7−12

This diff is too big to show here.

docs/hoon/twig/bar-core/dot-trap.md+41−54

This diff is too big to show here.

docs/hoon/twig/bar-core/hep-loop.md+27−40

This diff is too big to show here.

docs/hoon/twig/bar-core/ket-cork.md+26−36

This diff is too big to show here.

docs/hoon/twig/bar-core/sig-port.md+6−15

This diff is too big to show here.

docs/hoon/twig/bar-core/tar-gill.md+38−48

This diff is too big to show here.

docs/hoon/twig/bar-core/tis-gate.md+29−43

This diff is too big to show here.

docs/hoon/twig/bar-core/wut-tray.md+12−20

This diff is too big to show here.

docs/hoon/twig/buc-mold.md+34−39

This diff is too big to show here.

docs/hoon/twig/buc-mold/base.md+15−18

This diff is too big to show here.

docs/hoon/twig/buc-mold/cab-shoe.md+20−28

This diff is too big to show here.

docs/hoon/twig/buc-mold/cen-book.md+21−25

This diff is too big to show here.

docs/hoon/twig/buc-mold/col-bank.md+18−23

This diff is too big to show here.

docs/hoon/twig/buc-mold/hep-lamb.md+16−25

This diff is too big to show here.

docs/hoon/twig/buc-mold/ket-bush.md+20−26

This diff is too big to show here.

docs/hoon/twig/buc-mold/pat-claw.md+21−25

This diff is too big to show here.

docs/hoon/twig/buc-mold/tis-coat.md+14−23

This diff is too big to show here.

docs/hoon/twig/buc-mold/wut-pick.md+20−26

This diff is too big to show here.

docs/hoon/twig/cen-call.md+3−5

This diff is too big to show here.

docs/hoon/twig/cen-call/cab-keep.md+26−35

This diff is too big to show here.

docs/hoon/twig/cen-call/dot-lace.md+10−19

This diff is too big to show here.

docs/hoon/twig/cen-call/hep-call.md+22−33

This diff is too big to show here.

docs/hoon/twig/cen-call/ket-calq.md+11−19

This diff is too big to show here.

docs/hoon/twig/cen-call/lus-calt.md+11−19

This diff is too big to show here.

docs/hoon/twig/cen-call/sig-open.md+11−18

This diff is too big to show here.

docs/hoon/twig/cen-call/tar-bake.md+20−27

This diff is too big to show here.

docs/hoon/twig/cen-call/tis-make.md+22−26

This diff is too big to show here.

docs/hoon/twig/col-cell.md+3−5

This diff is too big to show here.

docs/hoon/twig/col-cell/cab-scon.md+7−15

This diff is too big to show here.

docs/hoon/twig/col-cell/hep-cons.md+15−17

This diff is too big to show here.

docs/hoon/twig/col-cell/ket-conq.md+18−26

This diff is too big to show here.

docs/hoon/twig/col-cell/lus-cont.md+16−24

This diff is too big to show here.

docs/hoon/twig/col-cell/sig-conl.md+35−45

This diff is too big to show here.

docs/hoon/twig/col-cell/tar-conp.md+48−56

This diff is too big to show here.

docs/hoon/twig/dot-nock.md+1−2

This diff is too big to show here.

docs/hoon/twig/dot-nock/ket-wish.md+6−6

This diff is too big to show here.

docs/hoon/twig/dot-nock/lus-bump.md+11−13

This diff is too big to show here.

docs/hoon/twig/dot-nock/tar-nock.md+29−33

This diff is too big to show here.

docs/hoon/twig/dot-nock/tis-same.md+15−19

This diff is too big to show here.

docs/hoon/twig/dot-nock/wut-deep.md+11−15

This diff is too big to show here.

docs/hoon/twig/ket-cast.md+5−6

This diff is too big to show here.

docs/hoon/twig/ket-cast/bar-iron.md+14−20

This diff is too big to show here.

docs/hoon/twig/ket-cast/hep-cast.md+31−40

This diff is too big to show here.

docs/hoon/twig/ket-cast/lus-like.md+7−11

This diff is too big to show here.

docs/hoon/twig/ket-cast/sig-burn.md+10−14

This diff is too big to show here.

docs/hoon/twig/ket-cast/tis-name.md+15−17

This diff is too big to show here.

docs/hoon/twig/ket-cast/wut-lead.md+13−20

This diff is too big to show here.

docs/hoon/twig/limb.md+10−11

This diff is too big to show here.

docs/hoon/twig/limb/limb.md+31−41

This diff is too big to show here.

docs/hoon/twig/limb/wing.md+18−21

This diff is too big to show here.

docs/hoon/twig/sem-make.md+1−2

This diff is too big to show here.

docs/hoon/twig/sem-make/col-wad.md+26−36

This diff is too big to show here.

docs/hoon/twig/sem-make/fas-nub.md+6−11

This diff is too big to show here.

docs/hoon/twig/sem-make/sem-fry.md+23−41

This diff is too big to show here.

docs/hoon/twig/sem-make/sig-dip.md+29−36

This diff is too big to show here.

docs/hoon/twig/sig-hint.md+2−4

This diff is too big to show here.

docs/hoon/twig/sig-hint/bar-show.md+8−12

This diff is too big to show here.

docs/hoon/twig/sig-hint/buc-poll.md+5−9

This diff is too big to show here.

docs/hoon/twig/sig-hint/cab-lurk.md+8−12

This diff is too big to show here.

docs/hoon/twig/sig-hint/cen-fast.md+38−45

This diff is too big to show here.

docs/hoon/twig/sig-hint/fas-funk.md+12−18

This diff is too big to show here.

docs/hoon/twig/sig-hint/gal-thin.md+16−21

This diff is too big to show here.

docs/hoon/twig/sig-hint/gar-hint.md+18−27

This diff is too big to show here.

docs/hoon/twig/sig-hint/lus-memo.md+9−15

This diff is too big to show here.

docs/hoon/twig/sig-hint/pam-dump.md+9−13

This diff is too big to show here.

docs/hoon/twig/sig-hint/tis-ddup.md+10−13

This diff is too big to show here.

docs/hoon/twig/sig-hint/wut-warn.md+13−17

This diff is too big to show here.

docs/hoon/twig/sig-hint/zap-peep.md+13−17

This diff is too big to show here.

docs/hoon/twig/tis-flow.md+15−17

This diff is too big to show here.

docs/hoon/twig/tis-flow/bar-new.md+21−30

This diff is too big to show here.

docs/hoon/twig/tis-flow/col-fix.md+17−25

This diff is too big to show here.

docs/hoon/twig/tis-flow/dot-set.md+22−30

This diff is too big to show here.

docs/hoon/twig/tis-flow/fas-var.md+35−52

This diff is too big to show here.

docs/hoon/twig/tis-flow/gal-rap.md+9−15

This diff is too big to show here.

docs/hoon/twig/tis-flow/gar-per.md+11−15

This diff is too big to show here.

docs/hoon/twig/tis-flow/hep-nip.md+19−27

This diff is too big to show here.

docs/hoon/twig/tis-flow/ket-sip.md+22−32

This diff is too big to show here.

docs/hoon/twig/tis-flow/lus-pin.md+17−25

This diff is too big to show here.

docs/hoon/twig/tis-flow/sem-rev.md+16−24

This diff is too big to show here.

docs/hoon/twig/tis-flow/sig-tow.md+27−31

This diff is too big to show here.

docs/hoon/twig/tis-flow/tar-aka.md+27−31

This diff is too big to show here.

docs/hoon/twig/wut-test.md+13−15

This diff is too big to show here.

docs/hoon/twig/wut-test/bar-or.md+20−30

This diff is too big to show here.

docs/hoon/twig/wut-test/col-if.md+17−23

This diff is too big to show here.

docs/hoon/twig/wut-test/dot-lest.md+29−37

This diff is too big to show here.

docs/hoon/twig/wut-test/gal-deny.md+9−18

This diff is too big to show here.

docs/hoon/twig/wut-test/gar-sure.md+9−17

This diff is too big to show here.

docs/hoon/twig/wut-test/hep-case.md+50−63

This diff is too big to show here.

docs/hoon/twig/wut-test/ket-ifcl.md+22−30

This diff is too big to show here.

docs/hoon/twig/wut-test/lus-deft.md+43−53

This diff is too big to show here.

docs/hoon/twig/wut-test/pam-and.md+19−29

This diff is too big to show here.

docs/hoon/twig/wut-test/pat-ifat.md+22−30

This diff is too big to show here.

docs/hoon/twig/wut-test/sig-ifno.md+17−26

This diff is too big to show here.

docs/hoon/twig/wut-test/tis-fits.md+17−23

This diff is too big to show here.

docs/hoon/twig/wut-test/zap-not.md+21−29

This diff is too big to show here.

docs/hoon/twig/zap-wild.md+1−2

This diff is too big to show here.

docs/hoon/twig/zap-wild/gar-wrap.md+7−11

This diff is too big to show here.

docs/hoon/twig/zap-wild/tis-code.md+14−18

This diff is too big to show here.

docs/hoon/twig/zap-wild/wut-need.md+16−20

This diff is too big to show here.

docs/hoon/twig/zap-wild/zap-fail.md+6−10

This diff is too big to show here.

docs/nock.md+3−2

This diff is too big to show here.

docs/nock/definition.md+105−119

This diff is too big to show here.

docs/nock/example.md+159−243

This diff is too big to show here.

docs/nock/explanation.md+265−386

This diff is too big to show here.

docs/nock/implementations.md+179−179

This diff is too big to show here.

docs/nock/implementations/c.md+176−178

This diff is too big to show here.

docs/nock/implementations/clojure.md+117−118

This diff is too big to show here.

docs/nock/implementations/csharp.md+377−379

This diff is too big to show here.

docs/nock/implementations/groovy.md+177−178

This diff is too big to show here.

docs/nock/implementations/haskell.md+56−57

This diff is too big to show here.

docs/nock/implementations/hoon.md+47−49

This diff is too big to show here.

docs/nock/implementations/javascript.md+355−357

This diff is too big to show here.

docs/nock/implementations/python.md+75−77

This diff is too big to show here.

docs/nock/implementations/ruby.md+130−131

This diff is too big to show here.

docs/nock/implementations/scala.md+68−69

This diff is too big to show here.

docs/nock/implementations/scheme.md+112−113

This diff is too big to show here.

docs/nock/implementations/swift.md+317−319

This diff is too big to show here.

docs/using.md+3−2

This diff is too big to show here.

docs/using/admin.md+49−21

This diff is too big to show here.

docs/using/filesystem.md+70−86

This diff is too big to show here.

docs/using/install.md+38−34

This diff is too big to show here.

docs/using/layout.md+32−19

This diff is too big to show here.

docs/using/messaging.md+61−72

This diff is too big to show here.

docs/using/setup.md+31−37

This diff is too big to show here.

docs/using/shell.md+38−48

This diff is too big to show here.

docs/using/urb.md+255−282

This diff is too big to show here.

docs/using/web.md+188−118

This diff is too big to show here.

docs/using/web/blog.md+27−32

This diff is too big to show here.

docs/using/web/comments.md+13−15

This diff is too big to show here.

docs/using/web/static.md+23−26

This diff is too big to show here.