on master help sign in
Merge branch 'master' into ford-runes-fix
b4643aaTed Blackman
docs.md+34−55
@@ -1,88 +1,67 @@
1---
1---
2anchor: none
2anchor: none
3−footer: True
3+footer: true
4−sort: 4
5title: Docs
4title: Docs
5+sort: 4
6---
6---
7
7
8<div class="lead">
8<div class="lead">
9 <div class="logo inverse"></div>
9 <div class="logo inverse"></div>
10 <h1 class="blue i-b">Urbit docs</h1>
10 <h1 class="blue i-b">Urbit docs</h1>
11−
12</div>
11</div>
12+
13<div class="col-md-12 head">
13<div class="col-md-12 head">
14+
14<div class="col-md-10">
15<div class="col-md-10">
15−**Urbit** is a secure peer-to-peer network of personal servers, built on a
16−clean-slate system software stack.
17
16
18−Check out the [technical overview](about/overview) for a high-level architecture
17+**Urbit** is a secure peer-to-peer network of personal servers, built on a clean-slate system software stack.
19−description. To install and run Urbit, head to the Using section, starting with
20−the [installation guide](using/install) and continuing forward.
21
18
22−To begin developing for Urbit, start by walking through
19+Check out the [technical overview](about/overview) for a high-level
23−[Urbytes](https://urbit.org/docs/byte/0). After that, delve into Hoon
20+architecture description. To install and run Urbit, head to the Using section, starting with the [installation guide](using/install) and continuing forward.
24−[syntax](hoon/syntax) directly, then walk through the [Arvo tutorials](arvo).
25−The examples and exercises in Arvo will teach you to build high-level programs
26−and interact with the network.
27
21
28−Dig into the mechanics of Urbit with [Core Hoon](hoon). Refer back there for
22+To begin developing for Urbit, start by walking through [Urbytes](https://urbit.org/docs/byte/0). After that, delve into Hoon [syntax](hoon/syntax) directly, then walk through the [Arvo tutorials](arvo). The examples and exercises in Arvo will teach you to build high-level programs and interact with the network.
29−pure-code exercises and functions in the [standard library](hoon/library).
30
23
24+Dig into the mechanics of Urbit with [Core Hoon](hoon). Refer back
25+there for pure-code exercises and functions in the [standard library](hoon/library).
26+
31−Anyone obsessed with math, theory, fundamentals of computation, etc, should
27+Anyone obsessed with math, theory, fundamentals of computation,
32−start with the [Nock definition](nock/definition) and work upward from there.
28+etc, should start with the [Nock definition](nock/definition) and work upward from there.
33
29
34</div>
30</div>
31+
35</div>
32</div>
33+
36<div class="col-md-12 head">
34<div class="col-md-12 head">
35+
37<div class="col-md-4">
36<div class="col-md-4">
37+
38<b>Community & contact</b>
38<b>Community & contact</b>
39<ul>
39<ul>
40−<li>
41−Chat with us on <a href="https://urbit.org/stream"><code>:talk</code></a>
40+<li>Chat with us on <a href="https://urbit.org/stream"><code>:talk</code></a></li>
42−</li>
43−<li>
44−Our discussion forum <a href="http://urbit.org/fora"><code>fora</code></a>
41+<li>Our discussion forum <a href="http://urbit.org/fora"><code>fora</code></a></li>
45−</li>
46−<li>
47−On Twitter <a href="https://twitter.com/urbit">@urbit</a>
42+<li>On Twitter <a href="https://twitter.com/urbit">@urbit</a></li>
48−</li>
49−<li>
50−Or email us at <a href="mailto:urbit@urbit.org">urbit@urbit.org</a>
43+<li>Or email us at <a href="mailto:urbit@urbit.org">urbit@urbit.org</a></li>
51−</li>
52</ul>
44</ul>
45+
53</div>
46</div>
47+
54<div class="col-md-4">
48<div class="col-md-4">
49+
55<b>Source code on <a href="https://github.com/urbit">GitHub</a></b>
50<b>Source code on <a href="https://github.com/urbit">GitHub</a></b>
56<ul>
51<ul>
57−<li>
58−<a href="https://github.com/urbit/urbit"><code>vere</code></a> - Urbit's VM
52+<li><a href="https://github.com/urbit/urbit"><code>vere</code></a> - Urbit's VM</li>
59−</li>
60−<li>
61−<a href="https://github.com/urbit/arvo"><code>arvo</code></a> - Urbit's OS
53+<li><a href="https://github.com/urbit/arvo"><code>arvo</code></a> - Urbit's OS</li>
62−</li>
63−<li>
64−<a href="https://github.com/urbit/tree"><code>tree</code></a> - Urbit's web
54+<li><a href="https://github.com/urbit/tree"><code>tree</code></a> - Urbit's web publishing UI</li>
65−publishing UI
66−</li>
67−<li>
68−<a href="https://github.com/urbit/talk"><code>talk</code></a> - Urbit's web chat
55+<li><a href="https://github.com/urbit/talk"><code>talk</code></a> - Urbit's web chat</li>
69−</li>
70−<li>
71−<a href="https://github.com/urbit/sole"><code>sole</code></a> - Urbit's web
56+<li><a href="https://github.com/urbit/sole"><code>sole</code></a> - Urbit's web console</li>
72−console
73−</li>
74−<li>
75−<a href="https://github.com/urbit/examples"><code>examples</code></a> - Hoon,
57+<li><a href="https://github.com/urbit/examples"><code>examples</code></a> - Hoon, Arvo and Tree</li>
76−Arvo and Tree
77−</li>
78−<li>
79−<a href="https://github.com/urbit/docs"><code>docs</code></a> - The repo for
58+<li><a href="https://github.com/urbit/docs"><code>docs</code></a> - The repo for these docs</li>
80−these docs
81−</li>
82</ul>
59</ul>
60+
83</div>
61</div>
62+
84</div>
63</div>
64+
85<div class="sections">
65<div class="sections">
86 <kids grid="true"></kids>
66 <kids grid="true"></kids>
87−
88</div>
67</div>
docs/about.md+1−1
@@ -1,5 +1,5 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 3
3sort: 3
4---
4---
5
5
docs/about/contributing.md+14−15
@@ -7,25 +7,24 @@ title: Contributing to Urbit
7
7
8# Contributing to Urbit
8# Contributing to Urbit
9
9
10−It's always a good day for pull requests! We gladly accept submissions for
10+It's always a good day for pull requests! We gladly accept submissions
11−improving our [code](https://github.com/urbit/urbit/) and
11+for improving our [code](https://github.com/urbit/urbit/) and
12[documentation](https://github.com/urbit/docs/) alike.
12[documentation](https://github.com/urbit/docs/) alike.
13
13
14## Code guidelines
14## Code guidelines
15
15
16Our official guide to contributing can be found
16Our official guide to contributing can be found
17−[here](https://github.com/urbit/urbit/blob/master/CONTRIBUTING.md), which is
17+[here](https://github.com/urbit/urbit/blob/master/CONTRIBUTING.md),
18−most useful for large and low-level changes. For simpler revisions, refer to
18+which is most useful for large and low-level changes. For simpler
19−[Hoon Syntax](../../hoon/syntax) for an in-depth guide to good Hoon.
19+revisions, refer to [Hoon Syntax](../../hoon/syntax) for an in-depth guide to
20−Alternatively, [browse the
20+good Hoon. Alternatively, [browse the
21source](https://github.com/urbit/arvo/blob/master/arvo/hoon.hoon).
21source](https://github.com/urbit/arvo/blob/master/arvo/hoon.hoon).
22
22
23## Docs guidelines
23## Docs guidelines
24
24
25For documentation pull requests, if you are making a brand new page, you'll want
25For documentation pull requests, if you are making a brand new page, you'll want
26−it to conform to our general documentation standards. Acquire
26+it to conform to our general documentation standards. Acquire [pandoc](http://pandoc.org)
27−[pandoc](http://pandoc.org) and make sure to format your edited files with the
27+and make sure to format your edited files with the following flags:
28−following flags:
29
28
30 $ pandoc --standalone \
29 $ pandoc --standalone \
31 --from=markdown+yaml_metadata_block \
30 --from=markdown+yaml_metadata_block \
@@ -34,13 +33,13 @@ following flags:
34 input.md
33 input.md
35
34
36If you are editing an existing page, reformatting the entire page with pandoc
35If you are editing an existing page, reformatting the entire page with pandoc
37−will make the diff hard to read. Instead, just make sure that your edits
36+will make the diff hard to read. Instead, just make sure that your edits generally
38−generally conform to the style of the existing docs around it. For example,
37+conform to the style of the existing docs around it. For example, you'll notice
39−you'll notice that the paragraphs mostly wrap at column 80.
38+that the paragraphs mostly wrap at column 80.
40
39
41−**Note:** If you have an old version of Pandoc, it may convert the terminating
40+**Note:** If you have an old version of Pandoc, it may convert the
42−`---` in YAML metadata to `...`, which is grounds for termination. Run the
41+terminating `---` in YAML metadata to `...`, which is grounds for termination.
43−following one-liner to atone.
42+Run the following one-liner to atone.
44
43
45 $ sed '1,/^$/ s/^\.\.\./---/g' input.md > output.md
44 $ sed '1,/^$/ s/^\.\.\./---/g' input.md > output.md
46
45
docs/about/glossary.md+121−108
@@ -7,17 +7,17 @@ title: Urbit glossary
7
7
8# Urbit Glossary
8# Urbit Glossary
9
9
10−Urbit is renowned for its exotic terminology. Here's a simple overview from the
10+Urbit is renowned for its exotic terminology. Here's a simple overview
11−strange words in.
11+from the strange words in.
12
12
13−As Dijkstra put it: "The purpose of abstraction is not to be vague, but to
13+As Dijkstra put it: "The purpose of abstraction is not to be vague, but
14−create a new semantic level in which one can be absolutely precise."
14+to create a new semantic level in which one can be absolutely precise."
15
15
16## Ships
16## Ships
17
17
18−An Urbit *ship* is a cryptographic title on a *will* signed by private key, a
18+An Urbit *ship* is a cryptographic title on a *will* signed by private
19−human-memorable name, and a packet routing address. Ships are classed by the
19+key, a human-memorable name, and a packet routing address. Ships are
20−number of bits in their address:
20+classed by the number of bits in their address:
21
21
22 Size Name Parent Object Example
22 Size Name Parent Object Example
23 ----- ------ ------ ------ -------
23 ----- ------ ------ ------ -------
@@ -31,29 +31,35 @@ Any ship can be called an "urbit."
31
31
32> A summary [here](http://urbit.org/posts/address-space/).
32> A summary [here](http://urbit.org/posts/address-space/).
33
33
34−An Urbit identity is a string like `~firbyr-napbes`. It means nothing, but it's
34+An Urbit identity is a string like `~firbyr-napbes`. It means nothing,
35−easy to remember and say out loud. `~firbyr-napbes` is actually just a 32-bit
35+but it's easy to remember and say out loud. `~firbyr-napbes` is actually
36−number, like an IP address, that we turn into a human-memorable string.
36+just a 32-bit number, like an IP address, that we turn into a
37+human-memorable string.
37
38
38−Technically, an urbit is a secure digital identity that you own and control with
39+Technically, an urbit is a secure digital identity that you own and
39−a cryptographic key, like a Bitcoin wallet. As in Bitcoin, the supply of urbits
40+control with a cryptographic key, like a Bitcoin wallet. As in Bitcoin,
40−is mathematically limited. This keeps the network friendly, by making spam and
41+the supply of urbits is mathematically limited. This keeps the network
41−abuse expensive.
42+friendly, by making spam and abuse expensive.
42
43
43−An urbit name is just a number; smaller numbers make shorter names. Shorter
44+An urbit name is just a number; smaller numbers make shorter names.
44−names are easier to remember, so they're more valuable. So urbits are classified
45+Shorter names are easier to remember, so they're more valuable. So
45−by the number of bits in their name. (A ship name is just a scrambled base-256
46+urbits are classified by the number of bits in their name. (A ship name
46−representation of the number.)
47+is just a scrambled base-256 representation of the number.)
47
48
48−A 32-bit urbit (like `~firbyr-napbes`) is called a *planet.* A 16-bit urbit
49+A 32-bit urbit (like `~firbyr-napbes`) is called a *planet.* A 16-bit
49−(like `~pollev)` is a *star.* An 8-bit ship (like `~mun`) is a *galaxy.* A
50+urbit (like `~pollev)` is a *star.* An 8-bit ship (like `~mun`) is a
50−planet is an identity for an independent, adult human. Stars and galaxies are
51+*galaxy.* A planet is an identity for an independent, adult human. Stars
51−network infrastructure.
52+and galaxies are network infrastructure.
52
53
53−Each planet or star is launched by its *parent,* the star or galaxy whose number
54+Each planet or star is launched by its *parent,* the star or galaxy
54−is its bottom half. So the planet `~firbyr-napbes`, `0xdead.beef` or
55+whose number is its bottom half. So the planet `~firbyr-napbes`,
55−`3.735.928.559`, is the child of `~pollev`, `0xbeef` or `48.879`, whose parent
56+`0xdead.beef` or `3.735.928.559`, is the child of `~pollev`, `0xbeef` or
57+`48.879`, whose parent is `~mun`, `0xef`, `239`. The parent of `~mun`
56−is `~mun`, `0xef`, `239`. The parent of `~mun` and all galaxies is `~zod`, `0`.
58+and all galaxies is `~zod`, `0`.
59+
60+##### <h5 id="pier">Pier</h5>
61+
62+A ship's *pier* is its Unix directory. For planets the name of the pier is usually the planet name.
57
63
58### Nock
64### Nock
59
65
@@ -62,59 +68,63 @@ is `~mun`, `0xef`, `239`. The parent of `~mun` and all galaxies is `~zod`, `0`.
62- *noun*: an *atom* or a *cell*
68- *noun*: an *atom* or a *cell*
63- *atom*: any natural number
69- *atom*: any natural number
64- *cell*: any ordered pair of nouns
70- *cell*: any ordered pair of nouns
65−- *subject*: a noun - the data against which a `formula` is evaluated
71+- *subject*: a noun - the data against which a `formula` is
72+ evaluated
66- *formula*: a noun - a function at the Nock level
73- *formula*: a noun - a function at the Nock level
67−- *product*: a noun - the result of evaluating a formula against a subject
74+- *product*: a noun - the result of evaluating a formula against a
75+ subject
68
76
69*See [Nock definition](../../nock/definition/).*
77*See [Nock definition](../../nock/definition/).*
70
78
71### Hoon
79### Hoon
72
80
73−> Hoon is a strict, higher-order typed functional language that compiles itself
81+> Hoon is a strict, higher-order typed functional language that compiles
74−> to Nock.
82+> itself to Nock.
75
83
76−###### span: an inferred type
84+###### <h6 id="span">span:</h6> an inferred type
77
85
78−A *span* defines a set (finite or infinite) of nouns and ascribes some semantics
86+A *span* defines a set (finite or infinite) of nouns and ascribes some
79−to it. There is no Hoon syntax for a span; it is always produced as the inferred
87+semantics to it. There is no Hoon syntax for a span; it is always
80−range of an expression (*twig*).
88+produced as the inferred range of an expression (*twig*).
81
89
82*See [basic types](../../hoon/basic/#-type-span-and-mold).*
90*See [basic types](../../hoon/basic/#-type-span-and-mold).*
83
91
84−###### core: a code-data cell
92+###### <h6 id="core">core:</h6> a code-data cell
85
93
86The code (*battery*) is the head, the data (*payload*) is the tail. All
94The code (*battery*) is the head, the data (*payload*) is the tail. All
87−code-data structures in normal languages (functions, objects, modules, etc)
95+code-data structures in normal languages (functions, objects, modules,
88−become *core*s in Hoon.
96+etc) become *core*s in Hoon.
89
97
90−- *battery*: the code of a core, a tree of *arms*
98+- <h6 id="battery">*battery*:</h6> the code of a core, a tree of *arms*
91−- *payload*: the data in a core
99+- <h6 id="payload">*payload*:</h6> the data in a core
92
100
93*See [basic types](../../hoon/basic)*.
101*See [basic types](../../hoon/basic)*.
94
102
95−###### arm: a named, functionally-computed attribute of a core
103+###### <h6 id="arm">arm:</h6> a named, functionally-computed attribute of a core
96
104
97−The *twig* of each arm is compiled to a Nock formula, with the enclosing `core`
105+The *twig* of each arm is compiled to a Nock formula, with the
98−itself as the subject.
106+enclosing `core` itself as the subject.
99
107
100There are two kinds of arms: *dry* and *wet*. Most arms are dry.
108There are two kinds of arms: *dry* and *wet*. Most arms are dry.
101
109
102- *dry*: normal, `++`, polymorphic by means of *variance*
110- *dry*: normal, `++`, polymorphic by means of *variance*
103
111
104−For a dry arm, we ask, is the new payload compatible with the old payload
112+For a dry arm, we ask, is the new payload compatible with the old
105−(against which the core was compiled)?
113+payload (against which the core was compiled)?
106
114
107- *wet*: unusual, `+-`, polymorphic by means of *genericity*
115- *wet*: unusual, `+-`, polymorphic by means of *genericity*
108
116
109−A wet arm uses the twig as a macro. We create a new type analysis path, which
117+A wet arm uses the twig as a macro. We create a new type analysis
110−works as if we expanded the callee with the caller's context.
118+path, which works as if we expanded the callee with the caller's
119+context.
111
120
112*See [advanced types](/hoon/advanced)*.
121*See [advanced types](/hoon/advanced)*.
113
122
114###### metal: a variance model
123###### metal: a variance model
115
124
116−Every core has a *metal* which defines its variance model (ie, the properties of
125+Every core has a *metal* which defines its variance model (ie, the
117−the span of a compatible core). The default is `gold` (invariant).
126+properties of the span of a compatible core). The default is `gold`
127+(invariant).
118
128
119- `gold`: *invariant*
129- `gold`: *invariant*
120- `lead`: *bivariant*
130- `lead`: *bivariant*
@@ -123,10 +133,11 @@ the span of a compatible core). The default is `gold` (invariant).
123
133
124*See [advanced types](/hoon/advanced)*.
134*See [advanced types](/hoon/advanced)*.
125
135
126−###### gate: a function/lambda/closure
136+###### <h6 id="gate">gate:</h6> a function/lambda/closure
127
137
128−A *gate* is a core with one arm. To call a gate on an argument, replace the
138+A *gate* is a core with one arm. To call a gate on an argument,
139+replace the sample (at [tree
129−sample (at [tree address](../../hoon/twig/limb/limb/) `+6` in the core) with the
140+address](../../hoon/twig/limb/limb/) `+6` in the core) with the
130argument, and then compute the arm.
141argument, and then compute the arm.
131
142
132The payload of a gate has a shape of `{sample context}`.
143The payload of a gate has a shape of `{sample context}`.
@@ -134,114 +145,116 @@ The payload of a gate has a shape of `{sample context}`.
134- *sample*: the argument tuple
145- *sample*: the argument tuple
135- *context*: the subject in which the gate was defined
146- *context*: the subject in which the gate was defined
136
147
137−*See [basic types](../../hoon/basic/#-core-p-span-q-map-term-span), [`%-` or
148+*See [basic types](../../hoon/basic/#-core-p-span-q-map-term-span),
138−`:call`](../../hoon/twig/cen-call/hep-call/) (the `twig` for calling a `gate`).*
149+[`%-` or `:call`](../../hoon/twig/cen-call/hep-call/) (the `twig` for
150+calling a `gate`).*
139
151
140−###### mold: a type constructor / validator
152+###### <h6 id="mold">mold:</h6> a type constructor / validator
141
153
142−A *mold* is an idempotent gate (function), accepting any noun, and producing a
154+A *mold* is an idempotent gate (function), accepting any noun, and
143−range with a useful span.
155+producing a range with a useful span.
144
156
145−- *bunt*: the value a mold produces when normalizing its default sample
157+- *bunt*: the value a mold produces when normalizing its default
158+ sample
146- *icon*: the span of the mold's range
159- *icon*: the span of the mold's range
147
160
148*See [mold twigs](/hoon/twig/buc-mold/).*
161*See [mold twigs](/hoon/twig/buc-mold/).*
149
162
150###### atom
163###### atom
151
164
152−A Hoon *atom* span describes a Nock atom, with two additional pieces of
165+A Hoon *atom* span describes a Nock atom, with two additional pieces
153−metadata: an *aura*, and an optional constant.
166+of metadata: an *aura*, and an optional constant.
154
167
155An atom span is *warm* or *cold* based on whether the constant exists.
168An atom span is *warm* or *cold* based on whether the constant exists.
156
169
157- *warm*: if the constant is `~` (null), any atom is in the span
170- *warm*: if the constant is `~` (null), any atom is in the span
158−- *cold*: if the constant is `[~ atom]`, its only legal value is `atom`.
171+- *cold*: if the constant is `[~ atom]`, its only legal value is
172+ `atom`.
159
173
160*See [basic types](../../hoon/basic/#-atom-p-term-q-unit-atom)*
174*See [basic types](../../hoon/basic/#-atom-p-term-q-unit-atom)*
161
175
162###### aura: a soft atom type
176###### aura: a soft atom type
163
177
164−*aura* is a name for an atom type. It represents the structure of an atom in a
178+*aura* is a name for an atom type. It represents the structure of an
165−string beginning with `@`. An aura may represent units, print format, or other
179+atom in a string beginning with `@`. An aura may represent units,
166−semantics. Its constraints on the value of an atom aren't enforced in any way.
180+print format, or other semantics. Its constraints on the value of an
181+atom aren't enforced in any way.
167
182
168−Two auras are compatible if one is a prefix of the other. You can change any
183+Two auras are compatible if one is a prefix of the other. You can
169−aura into any other aura by casting through the empty aura, `@`. The standard
184+change any aura into any other aura by casting through the empty
170−library has molds which alias many common auras.
185+aura, `@`. The standard library has molds which alias many common
186+auras.
171
187
172Some common auras and their aliases:
188Some common auras and their aliases:
173
189
174−- *term* (`@tas`): a symbol - an atomic ASCII string which obeys symbol rules:
190+- *term* (`@tas`): a symbol - an atomic ASCII string which obeys
175− lowercase and digit only, infix hyphen ("kebab-case"), first character must
191+ symbol rules: lowercase and digit only, infix hyphen ("kebab-case"),
176− be lowercase alphabetic.
192+ first character must be lowercase alphabetic.
177- *cord* (`@t`): UTF-8 text, least-significant-byte first
193- *cord* (`@t`): UTF-8 text, least-significant-byte first
178- *char* (`@tD`): a character, a single unicode byte (for multi-byte
194- *char* (`@tD`): a character, a single unicode byte (for multi-byte
179 characters and codepoints, see `@c`)
195 characters and codepoints, see `@c`)
180
196
181*See [basic types](../../hoon/basic/#-atom-p-term-q-unit-atom)*.
197*See [basic types](../../hoon/basic/#-atom-p-term-q-unit-atom)*.
182
198
183−###### twig: a Hoon expression
199+###### <h6 id="twig">twig:</h6> a Hoon expression
184−
185−A *twig* is a Hoon expression. Specifically, twig is the mold for the noun that
186−a Hoon source expression compiles to. A twig is always a cell.
187
200
188−A twig has one of two forms: `{twig twig}`, which produces the ordered pair of
201+A *twig* is a Hoon expression. Specifically, twig is the mold for the
189−the two twigs, or a cell `{stem bulb}`, where `stem` is an atomic symbol.
202+noun that a Hoon source expression compiles to. A twig is always a cell.
190
203
204+A twig has one of two forms: `{twig twig}`, which produces the ordered
205+pair of the two twigs, or a cell `{stem bulb}`, where `stem` is an
191−- *stem*: an atomic symbol (`@tas`) - the name of a twig.
206+atomic symbol.
192−- *bulb*: The mold of the twig's contents.
193
207
194−Most twigs have a *regular form*, beginning with a *sigil* which is either a
208+- <h6 id="stem">*stem*:</h6> an atomic symbol (`@tas`) - the name of a twig.
195−*keyword* or a *rune*. Some twigs also have a syntactic *irregular form*; a few
209+- <h6 id="bulb">*bulb*:</h6> The mold of the twig's contents.
196−have *only* an *irregular form*.
197
210
198−- *sigil*: a *keyword* or *rune* used to begin a twig
211+Most twigs have a *regular form*, beginning with a *rune*. Some twigs
199−- *keyword*: the value of a stem, prefixed with `:`.
212+also have a syntactic *irregular form*; a few have *only* an *irregular form*.
200−- *rune*: a pair of ASCII symbols used to begin a twig.
201
213
202−For example, the stem `%if` becomes the keyword
214+- <h6 id="rune">*rune*:</h6> a pair of ASCII symbols used to begin a twig.
203−[`:if`](../../hoon/twig/wut-test/col-if/), with the rune `?:`. The first symbol
204−in a rune represents a family of related `twig`s. For example, the [`?`
205−family](../../hoon/twig/wut-test/) are all conditionals.
206
215
207−The keyword or rune forms are up to the programmer's choice. Most existing code
216+For example, the rune [`?:`](../../hoon/twig/wut-test/col-if/) is
208−uses runes, but the keyword form makes the learning path easier.
217+Hoon's most common conditional, a branch on a boolean test. The first symbol
218+in a rune represents a family of related `twig`s. For example, the
219+[`?` family](../../hoon/twig/wut-test/) are all conditionals.
209
220
210A regular twig has two syntactic forms, *tall* and *flat*:
221A regular twig has two syntactic forms, *tall* and *flat*:
211
222
212−- *tall*: multiple lines, no parentheses, two or more spaces between tokens
223+- *tall*: multiple lines, no parentheses, two or more spaces between
224+ tokens
213- *flat*: one line, parentheses, one space between tokens
225- *flat*: one line, parentheses, one space between tokens
214
226
215−Tall twigs can contain flat ones, but not vice versa. All irregular forms are
227+Tall twigs can contain flat ones, but not vice versa. All irregular
216−flat.
228+forms are flat.
217
229
218*See [twig concept](../../hoon/concepts/#-twig-expression),
230*See [twig concept](../../hoon/concepts/#-twig-expression),
219[expressions](../../hoon/twig/), and [syntax](../../hoon/syntax/)*.
231[expressions](../../hoon/twig/), and [syntax](../../hoon/syntax/)*.
220
232
221###### limb: attribute or variable reference
233###### limb: attribute or variable reference
222
234
223−To resolve a *limb* named "foo", the subject is searched depth-first, head-first
235+To resolve a *limb* named "foo", the subject is searched depth-first,
224−for either a face named "foo" or a core with an arm of "foo". If a face is
236+head-first for either a face named "foo" or a core with an arm of
237+"foo". If a face is found, the result is a leg, if a core is
225−found, the result is a leg, if a core is found, the result is the product of the
238+found, the result is the product of the arm.
226−arm.
227
239
228- *leg*: a subexpression, or subtree of the subject.
240- *leg*: a subexpression, or subtree of the subject.
229−- *wing*: a list of limbs, searched from right to left (`a.b` means `b` within
241+- *wing*: a list of limbs, searched from right to left (`a.b` means
230− `a`).
242+ `b` within `a`).
231
243
232*See [Limbs and wings](../../hoon/twig/limb)*
244*See [Limbs and wings](../../hoon/twig/limb)*
233
245
234###### slot: a tree addressing scheme
246###### slot: a tree addressing scheme
235
247
236−Every cell has a head and a tail, each of which may be either an atom or a cell.
248+Every cell has a head and a tail, each of which may be either an atom or
237−Therefore every noun is a binary tree. `+1` or `.` resolves to the whole cell
249+a cell. Therefore every noun is a binary tree. `+1` or `.` resolves to
238−(technically this would work against an atom as well). The head of `+n` is
250+the whole cell (technically this would work against an atom as well).
239−`+2n`, the tail is `+(2n+1)`.
251+The head of `+n` is `+2n`, the tail is `+(2n+1)`.
240
252
241###### face: a labeled subtree
253###### face: a labeled subtree
242
254
243−Hoon has no scope or symbol-table; there is only the subject. To "declare" a
255+Hoon has no scope or symbol-table; there is only the subject. To
244−"variable" is to construct a new subject: `[name=value old-subject]`.
256+"declare" a "variable" is to construct a new subject:
257+`[name=value old-subject]`.
245
258
246A *face* is a span that wraps a name (`@tas`) around another span.
259A *face* is a span that wraps a name (`@tas`) around another span.
247
260
@@ -249,8 +262,8 @@ A *face* is a span that wraps a name (`@tas`) around another span.
249
262
250###### nest: type compatibility test
263###### nest: type compatibility test
251
264
252−*nest* is an internal function on two spans, which produces yes if the set of
265+*nest* is an internal function on two spans, which produces yes if the
253−nouns in the second span is provably a subset of the first.
266+set of nouns in the second span is provably a subset of the first.
254
267
255*See [advanced types](../../hoon/advanced/)*.
268*See [advanced types](../../hoon/advanced/)*.
256
269
@@ -258,5 +271,5 @@ nouns in the second span is provably a subset of the first.
258
271
259`0` (`%.y`) is *yes*, `1` (`%.n`) is *no*.
272`0` (`%.y`) is *yes*, `1` (`%.n`) is *no*.
260
273
261−> Why? It's fresh, it's different, it's new. And it's annoying. And it keeps you
274+> Why? It's fresh, it's different, it's new. And it's annoying. And it
262−> on your toes. And it's also just intuitively right.
275+> keeps you on your toes. And it's also just intuitively right.
docs/about/objections.md+170−453
@@ -7,545 +7,262 @@ title: Common objections to Urbit
7
7
8# Common objections to Urbit
8# Common objections to Urbit
9
9
10−"We choose to go to the moon, not because it is easy, but because it is hard."
10+"We choose to go to the moon, not because it is easy, but because it is hard." -- John F. Kennedy
11−-- John F. Kennedy
11+
12−
13−We're strong believers in crowdsourced criticism. We've heard a lot of great
12+We're strong believers in crowdsourced criticism. We've heard a lot of great objections from all kinds of directions. Surely there's more out there -- if you think of a reason Urbit can't work, and it's not listed here, please add it in a comment at the bottom, post it somewhere and send us a link, etc, etc.
14−objections from all kinds of directions. Surely there's more out there -- if you
13+
15−think of a reason Urbit can't work, and it's not listed here, please add it in a
16−comment at the bottom, post it somewhere and send us a link, etc, etc.
17−
18−## Urbit is a total scamcoin. It's 100% premined.
14+## Urbit is a total scamcoin. It's 100% premined.
19−
15+
20−Like a cryptocurrency, Urbit is a digital title. But Urbit is digital land, not
16+Like a cryptocurrency, Urbit is a digital title. But Urbit is digital land, not digital money. If you assess it as a currency, it's a lousy currency. If you assess a cat as a dog, it's a lousy dog.
21−digital money. If you assess it as a currency, it's a lousy currency. If you
17+
22−assess a cat as a dog, it's a lousy dog.
23−
24−In real-world economics, the crucial distinction between land and money is that
18+In real-world economics, the crucial distinction between land and money is that land (a) has intrinsic utility and (b) is non-fungible.
25−land (a) has intrinsic utility and (b) is non-fungible.
19+
26−
27−Because of (b) alone, frictionless transactions in land are impossible. A
28−blockchain is only needed to ensure zero-cost high-frequency transactions. So
20+Because of (b) alone, frictionless transactions in land are impossible. A blockchain is only needed to ensure zero-cost high-frequency transactions. So digital land doesn't need a blockchain. Which means it doesn't need mining.
29−digital land doesn't need a blockchain. Which means it doesn't need mining.
21+
30−
31−Of course, just because a digital title system doesn't need mining, doesn't mean
22+Of course, just because a digital title system doesn't need mining, doesn't mean it can't distribute an initial allocation by proof-of-work -- or by any other arbitrary method which is careful not to benefit the system's own developers. Could we give away the Urbit address space randomly? Sure. But why?
32−it can't distribute an initial allocation by proof-of-work -- or by any other
23+
33−arbitrary method which is careful not to benefit the system's own developers.
34−Could we give away the Urbit address space randomly? Sure. But why?
35−
36−Probably the purest design is the "proof-of-burn" initial allocation used by
37−some coins. By any standard, proof-of-burn is technically beautiful and results
24+Probably the purest design is the "proof-of-burn" initial allocation used by some coins. By any standard, proof-of-burn is technically beautiful and results in an extraordinary level of ethical independence. We can't help but admire it.
38−in an extraordinary level of ethical independence. We can't help but admire it.
25+
39−
40−But any altcoin or metacoin is just way less work than Urbit. An altcoin can
26+But any altcoin or metacoin is just way less work than Urbit. An altcoin can afford to burn even its creation energy, capturing no value at all from its seigniorage. Your altcoin doesn't have a clean-slate OS to fund, and no one started building it in 2002.
41−afford to burn even its creation energy, capturing no value at all from its
27+
42−seigniorage. Your altcoin doesn't have a clean-slate OS to fund, and no one
43−started building it in 2002.
44−
45−Ethically, what makes "premining" feel wrong is that the creators of the
46−currency are rewarded disproportionately for their effort. The scamcoin is a
47−get-rich-quick scheme, vaguely pyramid-shaped. Nothing about Urbit matches this
28+Ethically, what makes "premining" feel wrong is that the creators of the currency are rewarded disproportionately for their effort. The scamcoin is a get-rich-quick scheme, vaguely pyramid-shaped. Nothing about Urbit matches this pattern at all.
48−pattern at all.
29+
49−
50−To put it bluntly, Galen drives a 2004 Saab and Curtis drives a moldy '99 Cougar
30+To put it bluntly, Galen drives a 2004 Saab and Curtis drives a moldy '99 Cougar covered with raven shit. And we both love cars. We also have kids. And our spouses are more skeptical of Urbit than anyone else could ever be. (Our next startup definitely *will* be a get-rich-quick scheme, though.)
51−covered with raven shit. And we both love cars. We also have kids. And our
31+
52−spouses are more skeptical of Urbit than anyone else could ever be. (Our next
32+Curtis created Urbit. So by the [homesteading principle](https://en.wikipedia.org/wiki/Homestead_principle), he started out owning all of it. But by now, Urbit's 8-bit galaxies have already been distributed among [57 separate individuals](). Curtis owns 4 -- or 24, counting his whole family. (Some of whom he still owes money.)
53−startup definitely *will* be a get-rich-quick scheme, though.)
33+
54−
55−Curtis created Urbit. So by the [homesteading
56−principle](https://en.wikipedia.org/wiki/Homestead_principle), he started out
57−owning all of it. But by now, Urbit's 8-bit galaxies have already been
58−distributed among [57 separate individuals](). Curtis owns 4 -- or 24, counting
59−his whole family. (Some of whom he still owes money.)
60−
61−Tlon, the corporation which still runs Urbit development, has 95. (Without Tlon,
34+Tlon, the corporation which still runs Urbit development, has 95. (Without Tlon, there is basically no Arvo.) 50 are reserved for a hypothetical Urbit Foundation. Tlon employees have another 16.
62−there is basically no Arvo.) 50 are reserved for a hypothetical Urbit
35+
63−Foundation. Tlon employees have another 16.
36+The rest of the galaxies belong to donors, investors, contributors, contest winners, and just random people who were in the right place at the right time. One is a Silicon Valley tycoon. Another is an Orthodox nun in a monastery in Greece. A couple are ex-Tlon employees who might totally hate us. There is nothing they all have in common, which is sort of the point.
64−
37+
65−The rest of the galaxies belong to donors, investors, contributors, contest
38+This property distribution looks a lot like land ownership in the real world -- complex and asymmetrical. That's no accident. If you have a problem living in New York because New York's real estate is not distributed equally among its residents, you may have a similar problem with Urbit. The reality is that Urbit, like New York, wasn't born yesterday and comes with a complex ownership history which we couldn't possibly change, even if we wanted to.
66−winners, and just random people who were in the right place at the right time.
39+
67−One is a Silicon Valley tycoon. Another is an Orthodox nun in a monastery in
68−Greece. A couple are ex-Tlon employees who might totally hate us. There is
69−nothing they all have in common, which is sort of the point.
70−
71−This property distribution looks a lot like land ownership in the real world --
72−complex and asymmetrical. That's no accident. If you have a problem living in
73−New York because New York's real estate is not distributed equally among its
74−residents, you may have a similar problem with Urbit. The reality is that Urbit,
75−like New York, wasn't born yesterday and comes with a complex ownership history
76−which we couldn't possibly change, even if we wanted to.
77−
78−Also, even in Manhattan you can just sleep in your van, and in Urbit you can
79−always make your own comet. But people can't be prevented from thinking this
40+Also, even in Manhattan you can just sleep in your van, and in Urbit you can always make your own comet. But people can't be prevented from thinking this means you're a weirdo.
80−means you're a weirdo.
41+
81−
82−## Urbit isn't even really decentralized! It has a government!
42+## Urbit isn't even really decentralized! It has a government!
83−
43+
84−TLDR: yeah. But at least it's not the DAO.
44+TLDR: yeah. But at least it's not the DAO.
85−
45+
86−Real TLDR: it's technically impossible for Urbit to be decentralized at this
46+Real TLDR: it's technically impossible for Urbit to be decentralized at this stage of its life. Urbit is designed to achieve decentralization, not be born with it.
87−stage of its life. Urbit is designed to achieve decentralization, not be born
47+
88−with it.
89−
90−A young network can't afford to be decentralized. It has to act as a unit.
91−Because it has no network effect, it needs every scrap of efficiency it can get.
48+A young network can't afford to be decentralized. It has to act as a unit. Because it has no network effect, it needs every scrap of efficiency it can get. Right now, Urbit needs a central government about ten times over.
92−Right now, Urbit needs a central government about ten times over.
49+
93−
50+As we found out with the DAO, if a network isn't ready to be decentralized, its developers shouldn't even try. If Ethereum is hard-forked to roll back the DAO (which looks like the most probable result), it acknowledges that it both has a central government, and needs a central government. Decentralization is good. Pretend decentralization -- "decentralization theater" -- is harmful, and not a effective path to actual decentralization.
94−As we found out with the DAO, if a network isn't ready to be decentralized, its
51+
95−developers shouldn't even try. If Ethereum is hard-forked to roll back the DAO
52+Urbit did not have to get hacked to realize that a decentralized network can only be bootstrapped by a central government. It's part of Urbit's basic design. For example, by default your urbit upgrades its OS automatically (like an "evergreen" browser) with signed network updates. Whoever signs these updates is a central government by definition.
96−(which looks like the most probable result), it acknowledges that it both has a
53+
97−central government, and needs a central government. Decentralization is good.
98−Pretend decentralization -- "decentralization theater" -- is harmful, and not a
99−effective path to actual decentralization.
100−
101−Urbit did not have to get hacked to realize that a decentralized network can
102−only be bootstrapped by a central government. It's part of Urbit's basic design.
103−For example, by default your urbit upgrades its OS automatically (like an
104−"evergreen" browser) with signed network updates. Whoever signs these updates is
105−a central government by definition.
106−
107−But the test of a free software project is whether it can fork. So long as the
108−unity of the PKI and the protocol is maintained, Urbit development can fork
109−easily without disrupting the network. Any stress will fragment this central
54+But the test of a free software project is whether it can fork. So long as the unity of the PKI and the protocol is maintained, Urbit development can fork easily without disrupting the network. Any stress will fragment this central government, and as it fragments it grows more limited and stable.
110−government, and as it fragments it grows more limited and stable.
55+
111−
56+Urbit is designed to govern itself collectively by address space ownership -- effectively, "proof of stake." Any arbitrary power, will tend to fracture along these property lines. For instance, our hot updates are sent through the galaxy hierarchy; in a development fork, active galaxies would pick sides. This would actually be awesome for Urbit.
112−Urbit is designed to govern itself collectively by address space ownership --
57+
113−effectively, "proof of stake." Any arbitrary power, will tend to fracture along
114−these property lines. For instance, our hot updates are sent through the galaxy
115−hierarchy; in a development fork, active galaxies would pick sides. This would
116−actually be awesome for Urbit.
117−
118−By fragmenting the galaxy space into 57 separate partitions, we've created a
119−global property map which is much more likely to splinter further, than reunite.
58+By fragmenting the galaxy space into 57 separate partitions, we've created a global property map which is much more likely to splinter further, than reunite.
120−
59+
121−In the long run, every galaxy should have a separate owner, who is a human being
122−rather than a corporation. Combining galaxy holdings is regarded as creepy, and
60+In the long run, every galaxy should have a separate owner, who is a human being rather than a corporation. Combining galaxy holdings is regarded as creepy, and damages the value of both galaxies. Splitting up any remaining clumps is meritorious.
123−damages the value of both galaxies. Splitting up any remaining clumps is
61+
124−meritorious.
125−
126−The technical goal of Urbit is to create a network which, *if it actually takes
127−over the world*, inevitably becomes decentralized. Even if Urbit takes over the
62+The technical goal of Urbit is to create a network which, *if it actually takes over the world*, inevitably becomes decentralized. Even if Urbit takes over the world, nobody takes over the world. World domination must be technically implausible.
128−world, nobody takes over the world. World domination must be technically
63+
129−implausible.
130−
131−This is the goal of all the new "web3" decentralized systems. If IPFS, Sandstorm
64+This is the goal of all the new "web3" decentralized systems. If IPFS, Sandstorm or Ethereum takes over the world, nobody takes over the world. (The same is almost true of the Internet -- though it wouldn't be that surprising if a time traveler from 2116 revealed that ICANN had become a planetary government.)
132−or Ethereum takes over the world, nobody takes over the world. (The same is
65+
133−almost true of the Internet -- though it wouldn't be that surprising if a time
134−traveler from 2116 revealed that ICANN had become a planetary government.)
135−
136−Urbit, by starting out as a centralized network, just reduces the technical
66+Urbit, by starting out as a centralized network, just reduces the technical problem to match the actual goal. Only long-term decentralization matters.
137−problem to match the actual goal. Only long-term decentralization matters.
67+
138−
139−It's very important that Urbit in 2116 be fully decentralized. For Urbit in
68+It's very important that Urbit in 2116 be fully decentralized. For Urbit in 2016, pretending to be decentralized would be a joke at best, and a hoax at worst. And of course, no one can DAO us -- since the property system is still just a central registry.
140−2016, pretending to be decentralized would be a joke at best, and a hoax at
141−worst. And of course, no one can DAO us -- since the property system is still
142−just a central registry.
143
69
144## {Sandstorm, IPFS, Ethereum, ...} is way ahead of Urbit.
70## {Sandstorm, IPFS, Ethereum, ...} is way ahead of Urbit.
145
71
146−In general, web3 (next distributed web) solutions don't compete with each other.
72+In general, web3 (next distributed web) solutions don't compete with each other. They support each other. Concretely, they are all part of each others' network effect.
147−They support each other. Concretely, they are all part of each others' network
148−effect.
149
73
150−IPFS, Sandstorm, Ethereum, ZeroTier, and Urbit can all take over the world at
151−the same time. None of them competes with the others. They enable each other.
152−For instance, IPFS is not a computer; Urbit will be a great computer to use IPFS
74+IPFS, Sandstorm, Ethereum, ZeroTier, and Urbit can all take over the world at the same time. None of them competes with the others. They enable each other. For instance, IPFS is not a computer; Urbit will be a great computer to use IPFS from; IPFS is a great CDN for Urbit. Moar IPFS now!
153−from; IPFS is a great CDN for Urbit. Moar IPFS now!
154
75
155−The closest thing to a competitor that Urbit has is Sandstorm. Sandstorm, which
76+The closest thing to a competitor that Urbit has is Sandstorm. Sandstorm, which by the way rocks, is also a personal server.
156−by the way rocks, is also a personal server.
157
77
158−But Sandstorm and Urbit compete in the way that, on the PC, the OS and the
159−browser compete. In theory, the OS could solve the problems that the browser
78+But Sandstorm and Urbit compete in the way that, on the PC, the OS and the browser compete. In theory, the OS could solve the problems that the browser solves; or the browser could be the OS. In practice, Windows is a great OS to run a browser on.
160−solves; or the browser could be the OS. In practice, Windows is a great OS to
161−run a browser on.
162
79
163−Similarly, a world in which Sandstorm takes off is an awesome world for Urbit,
164−because everyone who has a Sandstorm instance has a great platform for running
80+Similarly, a world in which Sandstorm takes off is an awesome world for Urbit, because everyone who has a Sandstorm instance has a great platform for running Urbit on. (Although Sandstorm could still work on its UDP capabilities.)
165−Urbit on. (Although Sandstorm could still work on its UDP capabilities.)
166
81
167−Urbit on Sandstorm will be a sort of "double-hulled" personal server, even safer
82+Urbit on Sandstorm will be a sort of "double-hulled" personal server, even safer and more friendly than either system alone. And Sandstorm is not its own P2P network, like Urbit.
168−and more friendly than either system alone. And Sandstorm is not its own P2P
169−network, like Urbit.
170
83
171## Urbit doesn't have enough planets for every human being.
84## Urbit doesn't have enough planets for every human being.
172
85
173−A planet is a tool, not a toy. Like a car, it's a device for a responsible and
86+A planet is a tool, not a toy. Like a car, it's a device for a responsible and civilized adult. There aren't 4 billion cars in the world, nor 4 billion responsible adults.
174−civilized adult. There aren't 4 billion cars in the world, nor 4 billion
175−responsible adults.
176
87
177−And limiting the supply of anything doesn't create a shortage. It sets a price.
88+And limiting the supply of anything doesn't create a shortage. It sets a price. Ideally, in the long run, we think a planet should cost about $10. This could be off by an order of magnitude either way, and it wouldn't be the end of the world.
178−Ideally, in the long run, we think a planet should cost about $10. This could be
179−off by an order of magnitude either way, and it wouldn't be the end of the
180−world.
181
89
182−If a planet costs $10, that means you paid $10 to get a planet. A planet with no
183−reputation has a positive reputation, because someone paid $10 for it and hasn't
184−yet done anything wrong. Positive default reputation is the single quality that
90+If a planet costs $10, that means you paid $10 to get a planet. A planet with no reputation has a positive reputation, because someone paid $10 for it and hasn't yet done anything wrong. Positive default reputation is the single quality that makes a social network social.
185−makes a social network social.
186
91
187−There is always a tradeoff between free speech and free beer. Urbit will always
92+There is always a tradeoff between free speech and free beer. Urbit will always choose free speech.
188−choose free speech.
189
93
190## Cloud computing is inherently non-private.
94## Cloud computing is inherently non-private.
191
95
192−TLDR: a general-purpose cloud computer is much, much more private than a
193−special-purpose cloud computer. In general, a VM is secure against corporations
96+TLDR: a general-purpose cloud computer is much, much more private than a special-purpose cloud computer. In general, a VM is secure against corporations but not governments.
194−but not governments.
97+
195−
196−One reason we want the decentralized Internet is that we're tired of giant
98+One reason we want the decentralized Internet is that we're tired of giant corporations spying on us. Think about it this way: Amazon is Netflix's main competitor. And yet, Netflix runs its whole operation on AWS. Why is Netflix sure that Amazon won't spy on it?
197−corporations spying on us. Think about it this way: Amazon is Netflix's main
99+
198−competitor. And yet, Netflix runs its whole operation on AWS. Why is Netflix
100+A general-purpose virtual machine is special from a privacy standpoint, just because it's commercially realistic to design terms of service that prevent the host from peeking and poking at the customer's machine. Crossing this moat is technically nontrivial and can't possibly happen by any reasonable accident. No hosting contract could contemplate it.
199−sure that Amazon won't spy on it?
101+
200−
201−A general-purpose virtual machine is special from a privacy standpoint, just
202−because it's commercially realistic to design terms of service that prevent the
203−host from peeking and poking at the customer's machine. Crossing this moat is
204−technically nontrivial and can't possibly happen by any reasonable accident. No
205−hosting contract could contemplate it.
206−
207−Whereas when your cloud computer is a special-purpose machine -- just a database
102+Whereas when your cloud computer is a special-purpose machine -- just a database row on some corporation's mainframe -- there is no way these services can promise not to handle your data. Handling your data is *what they do*. On a special-purpose server, the concept of privacy is not even defined.
208−row on some corporation's mainframe -- there is no way these services can
103+
209−promise not to handle your data. Handling your data is *what they do*. On a
104+Where even a general-purpose server falls down is where the user has a sovereign adversary, which can secretly violate explicit or implicit hosting contracts. Users with a sovereign adversary must rely only on their own physical security. However, most users don't have a sovereign adversary. And those who do maximize their "herd immunity" if they can use the same protocols as those who don't.
210−special-purpose server, the concept of privacy is not even defined.
105+
211−
212−Where even a general-purpose server falls down is where the user has a sovereign
213−adversary, which can secretly violate explicit or implicit hosting contracts.
214−Users with a sovereign adversary must rely only on their own physical security.
215−However, most users don't have a sovereign adversary. And those who do maximize
216−their "herd immunity" if they can use the same protocols as those who don't.
217−
218−Also, the portability of a cloud server image is critical. Of course, since
219−Urbit is formally specified, it's trivially portable. In fact, one strategy
220−against sovereign adversaries may be continuous rotation between hosts, like an
106+Also, the portability of a cloud server image is critical. Of course, since Urbit is formally specified, it's trivially portable. In fact, one strategy against sovereign adversaries may be continuous rotation between hosts, like an MX missile.
221−MX missile.
222
107
223## "Jets" are, like, a total hoax.
108## "Jets" are, like, a total hoax.
224
109
225−TLDR: like, all of computing is a hoax. You can't compare a hoax to nothing. You
110+TLDR: like, all of computing is a hoax. You can't compare a hoax to nothing. You can only compare it to all the other hoaxes.
226−can only compare it to all the other hoaxes.
111+
227−
112+A "jet" is a sort of software device driver in Urbit. Urbit is precisely defined in terms of an interpreter that fits on a T-shirt, Nock. But a naive implementation of Nock is nowhere near practical, because Nock has only one integer operation: increment. For example, to decrement a number in Nock, you increment a counter up to the number under it.
228−A "jet" is a sort of software device driver in Urbit. Urbit is precisely defined
113+
229−in terms of an interpreter that fits on a T-shirt, Nock. But a naive
114+The only way an efficient Nock interpreter can work is to recognize algorithms like decrement, and implement them non-naively. By far the easiest way to do this is not to analyze arbitrary code as it's being executed, but to register and match well-known implementations in the standard library. No one should be writing their own decrement.
230−implementation of Nock is nowhere near practical, because Nock has only one
115+
231−integer operation: increment. For example, to decrement a number in Nock, you
232−increment a counter up to the number under it.
233−
234−The only way an efficient Nock interpreter can work is to recognize algorithms
235−like decrement, and implement them non-naively. By far the easiest way to do
236−this is not to analyze arbitrary code as it's being executed, but to register
237−and match well-known implementations in the standard library. No one should be
238−writing their own decrement.
239−
240−But the semantics of the interpreter, *so long as the jet and the jet harness
116+But the semantics of the interpreter, *so long as the jet and the jet harness are defined correctly*, are exactly those of Nock. We are not extending or changing Nock in any way. We are just taking a shortcut in certain cases we understand perfectly.
241−are defined correctly*, are exactly those of Nock. We are not extending or
117+
242−changing Nock in any way. We are just taking a shortcut in certain cases we
118+It's not at all unprecedented to have implementations much more complex than the corresponding specification. The x86 instruction set is relatively simple; the complexity of its decoding and execution path in a modern CPU is enormous. This path may well have "peephole optimizations" that replace matched algorithms, and if it doesn't the modern C compiler does.
243−understand perfectly.
119+
244−
245−It's not at all unprecedented to have implementations much more complex than the
246−corresponding specification. The x86 instruction set is relatively simple; the
247−complexity of its decoding and execution path in a modern CPU is enormous. This
248−path may well have "peephole optimizations" that replace matched algorithms, and
249−if it doesn't the modern C compiler does.
250−
251−So jets are *morally* defensible. The fact remains, however, that a Nock
252−interpreter without jets is completely useless. And there is not even an
253−informal document that tells you what jets you need for a reasonably efficient
120+So jets are *morally* defensible. The fact remains, however, that a Nock interpreter without jets is completely useless. And there is not even an informal document that tells you what jets you need for a reasonably efficient Urbit. (This last, we can agree, should be corrected.)
254−Urbit. (This last, we can agree, should be corrected.)
121+
255−
256−The correct way to think about jets is not to compare them to a perfect
257−interpreter implemented by unicorns. Jets are a combined replacement for two
258−other techniques in language implementation: intrinsics and a foreign-function
122+The correct way to think about jets is not to compare them to a perfect interpreter implemented by unicorns. Jets are a combined replacement for two other techniques in language implementation: intrinsics and a foreign-function interface.
259−interface.
123+
260−
124+Compared to both intrinsics and an FFI, your jets have one key disadvantage: they need to be specified in pure code. You may not have this executable specification, which means you need to write it. Is this really so bad for your development process? An executable specification? Which can be trivially tested against the efficient implementation?
261−Compared to both intrinsics and an FFI, your jets have one key disadvantage:
125+
262−they need to be specified in pure code. You may not have this executable
263−specification, which means you need to write it. Is this really so bad for your
264−development process? An executable specification? Which can be trivially tested
265−against the efficient implementation?
266−
267−Compared to intrinsics, jets give you true separation between mechanism and
126+Compared to intrinsics, jets give you true separation between mechanism and policy; only the interpreter, not the programmer and not the compiler, needs to know what's accelerated.
268−policy; only the interpreter, not the programmer and not the compiler, needs to
127+
269−know what's accelerated.
270−
271−Compared to an FFI, jets preclude I/O, so can be trivially sandboxed. Except
272−that the programmer needs to mark routines which may benefit from optimization
128+Compared to an FFI, jets preclude I/O, so can be trivially sandboxed. Except that the programmer needs to mark routines which may benefit from optimization with a hint, here too we separate mechanism (efficient implementation) from policy.
273−with a hint, here too we separate mechanism (efficient implementation) from
129+
274−policy.
275−
276−Yes, formal verification of jet equivalence is desirable. However, in the real
130+Yes, formal verification of jet equivalence is desirable. However, in the real world, many specifications, including quite complex ones, are tested into equivalence by brute force.
277−world, many specifications, including quite complex ones, are tested into
131+
278−equivalence by brute force.
279−
280−Urbit has some jet abuses. The equivalence between our markdown parser in Hoon
132+Urbit has some jet abuses. The equivalence between our markdown parser in Hoon and the CommonMark C library is super dubious. It should be taken out and shot.
281−and the CommonMark C library is super dubious. It should be taken out and shot.
133+
282−
134+However, the miscomputation resulting from a minor jet discrepancy is not a security hole or even fatal. It effectively means the event history does not match the current state. The way to resolve this conflict is for the current state to win, and the event history to be discarded -- not abnormal anyway. See the next question.
283−However, the miscomputation resulting from a minor jet discrepancy is not a
284−security hole or even fatal. It effectively means the event history does not
285−match the current state. The way to resolve this conflict is for the current
286−state to win, and the event history to be discarded -- not abnormal anyway. See
287−the next question.
288
135
289## Urbit needs an infinitely extending packet/event log.
136## Urbit needs an infinitely extending packet/event log.
290
137
291TLDR: no, it doesn't.
138TLDR: no, it doesn't.
292
139
293−We can prune the event log because the Urbit transition function doesn't have
140+We can prune the event log because the Urbit transition function doesn't have any access to the event history -- just the current state and the current event.
294−any access to the event history -- just the current state and the current event.
295
141
296−Urbit defines the semantics of a complete event log. It doesn't and can't
297−require an implementation to maintain the whole log. It can maintain a
298−checkpoint and the log since that checkpoint. The current implementation of
142+Urbit defines the semantics of a complete event log. It doesn't and can't require an implementation to maintain the whole log. It can maintain a checkpoint and the log since that checkpoint. The current implementation of Urbit keeps the whole log, but the next release will optionally prune it.
299−Urbit keeps the whole log, but the next release will optionally prune it.
300
143
301## Urbit will never be able to call any existing code.
144## Urbit will never be able to call any existing code.
302
145
303−TLDR: neither will Javascript. And dear Unix, the browser does not miss
146+TLDR: neither will Javascript. And dear Unix, the browser does not miss interacting with your `$LD_LIBRARY_PATH` at all.
304−interacting with your `$LD_LIBRARY_PATH` at all.
305
147
306−You can't talk to the OS or use system libraries from Javascript. Therefore,
307−there is no point in having any kind of gateway, compatibility bridge, native
148+You can't talk to the OS or use system libraries from Javascript. Therefore, there is no point in having any kind of gateway, compatibility bridge, native methods, or other interface between Javascript and C.
308−methods, or other interface between Javascript and C.
309
149
310−When in doubt, remember that Urbit is "the browser for the server side." It
150+When in doubt, remember that Urbit is "the browser for the server side." It plays by the same general rules. Isolation means isolation.
311−plays by the same general rules. Isolation means isolation.
312
151
313−## There's no way Urbit will ever be adopted. It's too weird.
152+## There's no way Urbit will ever be adopted. It's too weird.
314
153
315−TLDR: Urbit adoption is designed to be driven by ordinary end users. Urbit isn't
154+TLDR: Urbit adoption is designed to be driven by ordinary end users. Urbit isn't ready for ordinary end users yet. We're working on it -- come help.
316−ready for ordinary end users yet. We're working on it -- come help.
317
155
318−Right now, Urbit is a developer tool. Or rather, a developer toy -- since the
319−only thing it's useful for is developing Urbit. It also could be a much better
156+Right now, Urbit is a developer tool. Or rather, a developer toy -- since the only thing it's useful for is developing Urbit. It also could be a much better toy. We're working on that too.
320−toy. We're working on that too.
321
157
322−But it's absolutely wrong to think of Urbit being adopted as a developer tool in
158+But it's absolutely wrong to think of Urbit being adopted as a developer tool in the conventional sense. If you think this is almost impossible, you're right.
323−the conventional sense. If you think this is almost impossible, you're right.
324
159
325−If we look at the way most successful platforms are adopted, they're not adopted
160+If we look at the way most successful platforms are adopted, they're not adopted for their own virtues. They're adopted because they're part of a larger solution. It's very hard to displace a native component of an existing platform. Urbit is not trying to do this.
326−for their own virtues. They're adopted because they're part of a larger
327−solution. It's very hard to displace a native component of an existing platform.
328−Urbit is not trying to do this.
329
161
330−C is a better language than Pascal, but it didn't beat Pascal because it's a
162+C is a better language than Pascal, but it didn't beat Pascal because it's a better language. C beat Pascal because C was the native language of Unix, and people wanted to program in Unix. People don't use Javascript because Javascript is a great language, but because it's the native language of the browser. Replacing the native language of a platform may be doable -- but it's incredibly hard.
331−better language. C beat Pascal because C was the native language of Unix, and
332−people wanted to program in Unix. People don't use Javascript because Javascript
333−is a great language, but because it's the native language of the browser.
334−Replacing the native language of a platform may be doable -- but it's incredibly
335−hard.
336
163
337−Hoon is not competing with C, or even Haskell, as a Unix programming language.
164+Hoon is not competing with C, or even Haskell, as a Unix programming language. The only reason to program in Hoon is that it's the native language of Urbit. Urbit is not competing with node.js or Rails as a Web backend. The only reason to run Urbit is to run Urbit.
338−The only reason to program in Hoon is that it's the native language of Urbit.
339−Urbit is not competing with node.js or Rails as a Web backend. The only reason
340−to run Urbit is to run Urbit.
341
165
342−Nock is cool, but it only gets adopted if Hoon gets adopted. Hoon is cool, but
166+Nock is cool, but it only gets adopted if Hoon gets adopted. Hoon is cool, but it only gets adopted if Urbit gets adopted. Urbit only gets adopted if it's useful as a personal server for human beings -- most of whom won't be Hoon programmers, or programmers at all.
343−it only gets adopted if Urbit gets adopted. Urbit only gets adopted if it's
344−useful as a personal server for human beings -- most of whom won't be Hoon
345−programmers, or programmers at all.
346
167
347−## Hoon is too weird. No one will be able to program in it.
168+## Hoon is too weird. No one will be able to program in it.
348
169
349−TLDR: Hoon is ridiculously simple. Our documentation and tooling are just
170+TLDR: Hoon is ridiculously simple. Our documentation and tooling are just immature.
350−immature.
351
171
352−It's pretty fair to assume that (a) functional programming is hard and weird,
172+It's pretty fair to assume that (a) functional programming is hard and weird, (b) Hoon is weird even for a functional language, so (c) Hoon must be hard-squared.
353−(b) Hoon is weird even for a functional language, so (c) Hoon must be
354−hard-squared.
355
173
356−Actually, Hoon is weird even for a functional language because it's an attempt
357−to remove the hardest things about functional programming: category theory, lazy
174+Actually, Hoon is weird even for a functional language because it's an attempt to remove the hardest things about functional programming: category theory, lazy evaluation, and gratuitous abstraction. To know a language is to know what the compiler is doing; all the semantics of the Hoon compiler fit in 2000 lines of code.
358−evaluation, and gratuitous abstraction. To know a language is to know what the
359−compiler is doing; all the semantics of the Hoon compiler fit in 2000 lines of
360−code.
361
175
362−It's too easy for us to wind up living in Isaac Asimov's Trantor, a world in
363−which everything got invented thousands of years ago and the idea of new
364−technology is just weird and scary. History isn't over yet, and nor is technical
176+It's too easy for us to wind up living in Isaac Asimov's Trantor, a world in which everything got invented thousands of years ago and the idea of new technology is just weird and scary. History isn't over yet, and nor is technical history.
365−history.
366
177
367−New things are harder to learn than old things. We ask smart high-school
368−students to learn calculus. Think about how much harder it was for Newton and
178+New things are harder to learn than old things. We ask smart high-school students to learn calculus. Think about how much harder it was for Newton and Leibniz to invent calculus, or for their first students to pick it up.
369−Leibniz to invent calculus, or for their first students to pick it up.
370
179
371−Most of the difference is just the collective certainty that our society conveys
180+Most of the difference is just the collective certainty that our society conveys to a high-school student that calculus is learnable. When you're learning Urbit, you don't have this collective knowledge; so you don't have the confidence that the problem is solvable. Confidence makes a huge difference. Moreover, any bug or glitch in the developer experience destroys this confidence and takes a long time to heal.
372−to a high-school student that calculus is learnable. When you're learning Urbit,
373−you don't have this collective knowledge; so you don't have the confidence that
374−the problem is solvable. Confidence makes a huge difference. Moreover, any bug
375−or glitch in the developer experience destroys this confidence and takes a long
376−time to heal.
377
181
378−Yes, we know: Hoon remains quite hard to learn. Again, we have no reason to
379−think this problem isn't solvable. Public enlightenment remains our main
182+Yes, we know: Hoon remains quite hard to learn. Again, we have no reason to think this problem isn't solvable. Public enlightenment remains our main priority.
380−priority.
381
183
382## Urbit programmers are stuck with Hoon.
184## Urbit programmers are stuck with Hoon.
383
185
384TLDR: patience, grasshopper.
186TLDR: patience, grasshopper.
385
187
386−There's quite a straightforward way to implement any perfectly specified
387−language with a reliable interpreter in Hoon. First, write an interpreter for
188+There's quite a straightforward way to implement any perfectly specified language with a reliable interpreter in Hoon. First, write an interpreter for that language in Hoon. Second, jet that intepreter function with the existing implementation.
388−that language in Hoon. Second, jet that intepreter function with the existing
389−implementation.
390
189
391−It is not clear that Javascript is *quite* well-defined enough for this
190+It is not clear that Javascript is *quite* well-defined enough for this treatment. But Lua probably is.
392−treatment. But Lua probably is.
393
191
394## Ordinary users will never be able to manage an Urbit server.
192## Ordinary users will never be able to manage an Urbit server.
395
193
396−TLDR: Urbit's technical simplicity has always been designed to drive human
194+TLDR: Urbit's technical simplicity has always been designed to drive human usability. But the polish is the last thing that goes on. So we can't evaluate this claim right now.
397−usability. But the polish is the last thing that goes on. So we can't evaluate
398−this claim right now.
399
195
400−Right now, Urbit is only for geeks. You already need to be able to wrangle a
196+Right now, Urbit is only for geeks. You already need to be able to wrangle a Unix machine to run it. What's the point?
401−Unix machine to run it. What's the point?
402
197
403−This Urbit isn't the real Urbit. It's an early milestone on the way to the real
198+This Urbit isn't the real Urbit. It's an early milestone on the way to the real Urbit.
404−Urbit.
405
199
406−The real Urbit -- for normal human beings -- is a hosted service. Getting a
407−planet is as easy as signing up for Facebook. Probably many users will think of
200+The real Urbit -- for normal human beings -- is a hosted service. Getting a planet is as easy as signing up for Facebook. Probably many users will think of it as another web account. They might not even know it can run more apps than the ones they use.
408−it as another web account. They might not even know it can run more apps than
409−the ones they use.
410
201
411−A deterministic, persistent cloud computer is something different from a
412−traditional server. Normally when we think of a server in the cloud, we think of
202+A deterministic, persistent cloud computer is something different from a traditional server. Normally when we think of a server in the cloud, we think of it like a live animal -- a creature that needs to be cared for, fed, and herded. A file in the cloud is more like a rock -- you can forget about it for ten years.
413−it like a live animal -- a creature that needs to be cared for, fed, and herded.
414−A file in the cloud is more like a rock -- you can forget about it for ten
415−years.
416
203
417−An urbit is a general-purpose computer, but it doesn't even have the concept of
204+An urbit is a general-purpose computer, but it doesn't even have the concept of a reboot. It upgrades itself. If you don't feed it any packets for ten years, then turn it back on, it will spend a day or two upgrading itself, then be back to normal. Maybe it's something in between a rock and an animal -- a cactus?
418−a reboot. It upgrades itself. If you don't feed it any packets for ten years,
419−then turn it back on, it will spend a day or two upgrading itself, then be back
420−to normal. Maybe it's something in between a rock and an animal -- a cactus?
421
205
422−Another huge difference versus an Internet server is that we're confident that
206+Another huge difference versus an Internet server is that we're confident that Urbit's limited supply of identities will make it easy to keep abusers off the network. So far, so good -- but of course, no one abuses a young network.
423−Urbit's limited supply of identities will make it easy to keep abusers off the
424−network. So far, so good -- but of course, no one abuses a young network.
425
207
426−The bottom line is that managing your urbit has to be as easy as managing your
208+The bottom line is that managing your urbit has to be as easy as managing your iPhone. Normally, it consists of deciding what apps you want to be running, and how much you trust them. Resources can't be free, so sometimes you have to play memory police. Humans have shown that they can handle these tasks.
427−iPhone. Normally, it consists of deciding what apps you want to be running, and
428−how much you trust them. Resources can't be free, so sometimes you have to play
429−memory police. Humans have shown that they can handle these tasks.
430
209
431−Our '70s OSes and networks, which were great designs in their day and have
210+Our '70s OSes and networks, which were great designs in their day and have matured into excellent industrial tools, give us a totally misleading impression of how hard it should be to manage a general-purpose computer. Computing isn't scary or dangerous. Unix and the Internet are scary and dangerous. That's fine for an industrial tool, but not a personal tool.
432−matured into excellent industrial tools, give us a totally misleading impression
433−of how hard it should be to manage a general-purpose computer. Computing isn't
434−scary or dangerous. Unix and the Internet are scary and dangerous. That's fine
435−for an industrial tool, but not a personal tool.
436
211
437## Urbit has no killer app.
212## Urbit has no killer app.
438
213
439TLDR: the killer app is personal API aggregation.
214TLDR: the killer app is personal API aggregation.
440
215
441−It's a truism that any new platform has to be fundamentally much better at
216+It's a truism that any new platform has to be fundamentally much better at solving some problem than the mature platform it seeks to replace.
442−solving some problem than the mature platform it seeks to replace.
217+
443−
444−We can look at Urbit's killer app from two perspectives: as a developer, and as
218+We can look at Urbit's killer app from two perspectives: as a developer, and as a user.
445−a user.
219+
446−
220+From the developer's perspective, Urbit's advantage is that it's a network stack which provides exactly-once delivery of typed, validated, transactional, encrypted and authenticated messages. The difference between an Urbit API and a REST API is like the difference between a paved road and a dirt road. On the other hand, you do need to learn a new functional language.
447−From the developer's perspective, Urbit's advantage is that it's a network stack
221+
448−which provides exactly-once delivery of typed, validated, transactional,
449−encrypted and authenticated messages. The difference between an Urbit API and a
450−REST API is like the difference between a paved road and a dirt road. On the
451−other hand, you do need to learn a new functional language.
452−
453−From the user's perspective, there's one task that any personal server can
222+From the user's perspective, there's one task that any personal server can perform, that can't be performed by any Web service: unrestricted API aggregation.
454−perform, that can't be performed by any Web service: unrestricted API
223+
455−aggregation.
456−
457−Well before Urbit is ready to *replace* any conventional web services, it should
224+Well before Urbit is ready to *replace* any conventional web services, it should be ready to *command* them in place by API. (Or if needed, by scraping.) If all you care about is being in control of your own data, it's fine to leave it in the hands of these services, as you have the API secrets to get it out.
458−be ready to *command* them in place by API. (Or if needed, by scraping.) If all
225+
459−you care about is being in control of your own data, it's fine to leave it in
460−the hands of these services, as you have the API secrets to get it out.
461−
462−Managing a set of security connections to your existing services is one of the
463−first important tasks of a modern personal server. This is a stateful task that
464−typically requires both a durable API key and a temporary OAuth token. APIs
465−change all the time, so connection maintenance also requires evergreen updates.
226+Managing a set of security connections to your existing services is one of the first important tasks of a modern personal server. This is a stateful task that typically requires both a durable API key and a temporary OAuth token. APIs change all the time, so connection maintenance also requires evergreen updates.
466−
227+
467−The origin of this API key will vary, but usually the best way to set it is
228+The origin of this API key will vary, but usually the best way to set it is BYOK: the user registers their own urbit as an "app." And the neat thing about having your own API keys, for multiple services, on a single general-purpose computer, is that none of these services really has any practical way to control what you do with your own data.
468−BYOK: the user registers their own urbit as an "app." And the neat thing about
229+
469−having your own API keys, for multiple services, on a single general-purpose
470−computer, is that none of these services really has any practical way to control
471−what you do with your own data.
472−
473−Terms of service which constrain the use of data (as opposed to the abuse of the
474−service) simply aren't realistic and will no longer apply. This enables a
475−variety of novel user experiences which aren't implementable as traditional web
230+Terms of service which constrain the use of data (as opposed to the abuse of the service) simply aren't realistic and will no longer apply. This enables a variety of novel user experiences which aren't implementable as traditional web services, and also aren't available via conventional third-party API aggregators.
476−services, and also aren't available via conventional third-party API
231+
477−aggregators.
232+Also, since Urbit is after all quite immature, it doesn't make much sense to restrict this security state to code within Urbit. The Urbit interpreter `vere` runs as a Unix daemon that implements the FUSE user-level filesystem protocol, and maps the data on your services into the Unix filesystem. For example, one of our demos lets you `ls` your Github issues.
478−
233+
479−Also, since Urbit is after all quite immature, it doesn't make much sense to
480−restrict this security state to code within Urbit. The Urbit interpreter `vere`
481−runs as a Unix daemon that implements the FUSE user-level filesystem protocol,
482−and maps the data on your services into the Unix filesystem. For example, one of
483−our demos lets you `ls` your Github issues.
484−
485−So it's straightforward to build, for instance, a Python app that uses Urbit as
234+So it's straightforward to build, for instance, a Python app that uses Urbit as a gateway to mash up a bunch of web services. As a Unix programmer, you might even just regard Urbit as a generic Web API gateway daemon, without worrying about its internals.
486−a gateway to mash up a bunch of web services. As a Unix programmer, you might
235+
487−even just regard Urbit as a generic Web API gateway daemon, without worrying
488−about its internals.
489−
490−We have other ideas for initial killer apps. But this, we feel, is the best one.
491−API libraries for `npm` et al abound. Stateful, self-updating, generic API
492−aggregators with FUSE integration, also integrated with a reactive web server,
236+We have other ideas for initial killer apps. But this, we feel, is the best one. API libraries for `npm` et al abound. Stateful, self-updating, generic API aggregators with FUSE integration, also integrated with a reactive web server, are more rare. So perhaps there's a small niche here.
493−are more rare. So perhaps there's a small niche here.
494
237
495## Urbit is sneered at by brilliant talking horses.
238## Urbit is sneered at by brilliant talking horses.
496
239
497−It's difficult to categorize this [brilliant essay of
498−Fare's](http://ngnghm.github.io/blog/2016/06/11/chapter-10-houyhnhnms-vs-martians/)
240+It's difficult to categorize this [brilliant essay of Fare's](http://ngnghm.github.io/blog/2016/06/11/chapter-10-houyhnhnms-vs-martians/) in a single bullet point. So I've saved it for last.
499−in a single bullet point. So I've saved it for last.
500
241
501−We've already addressed the point most commonly made by others: that jets are a
242+We've already addressed the point most commonly made by others: that jets are a sham. It's not always trivial to decode this critique, which is quite abstract. But Fare' makes a few other points.
502−sham. It's not always trivial to decode this critique, which is quite abstract.
503−But Fare' makes a few other points.
504
243
505### Event-sourcing and/or caching is a pattern, not a layer
244### Event-sourcing and/or caching is a pattern, not a layer
506
245
507−Fare' is perfectly right that "one size does not fit all" in persistence.
246+Fare' is perfectly right that "one size does not fit all" in persistence. Generally, Urbit uses event-sourced persistence.
508−Generally, Urbit uses event-sourced persistence.
509
247
510−But it has multiple layers of event-sourcing: in the definition of Urbit itself;
248+But it has multiple layers of event-sourcing: in the definition of Urbit itself; in the application sandbox vane (Gall), and in the application itself. Urbit also has a completely orthogonal revision control system (Clay).
511−in the application sandbox vane (Gall), and in the application itself. Urbit
512−also has a completely orthogonal revision control system (Clay).
513
249
514−And there is not a single cache across the whole system, though arguably there
250+And there is not a single cache across the whole system, though arguably there should be; for instance, the functional build system (Ford, which Fare' the master of build systems might appreciate) has its own internal cache.
515−should be; for instance, the functional build system (Ford, which Fare' the
516−master of build systems might appreciate) has its own internal cache.
517
251
518−It is a common mistake to confuse a design pattern with a layer or feature, and
252+It is a common mistake to confuse a design pattern with a layer or feature, and equally fatal whichever direction you fail.
519−equally fatal whichever direction you fail.
520
253
521### Let's be clear about why DSLs suck
254### Let's be clear about why DSLs suck
522
255
523−DSLs suck not because they are difficult to newbies in the base language. DSLs
524−suck because every time a programmer, even one *experienced* in the base
525−language, comes into a new codebase or project, she has to learn a new custom
256+DSLs suck not because they are difficult to newbies in the base language. DSLs suck because every time a programmer, even one *experienced* in the base language, comes into a new codebase or project, she has to learn a new custom language.
526−language.
527
257
528−You need enormous wins to justify this cost. It's the goal of the base language
258+You need enormous wins to justify this cost. It's the goal of the base language designer to preempt these wins, creating a language which is almost never so tiresome and full of boilerplate that it requires a DSL or template layer.
529−designer to preempt these wins, creating a language which is almost never so
530−tiresome and full of boilerplate that it requires a DSL or template layer.
531
259
532−Hoon is almost entirely made of macros -- but they are internal, fixed, built-in
260+Hoon is almost entirely made of macros -- but they are internal, fixed, built-in macros (with no gensyms). This fixed set of macros is essentially enough. Occasionally we want to add one or two. But we never, ever pine for context-dependent macros.
533−macros (with no gensyms). This fixed set of macros is essentially enough.
534−Occasionally we want to add one or two. But we never, ever pine for
535−context-dependent macros.
536
261
537### Determinism is not consensus, and is not constraining
262### Determinism is not consensus, and is not constraining
538
263
539−Fare' is totally right that an urbit's data is meant for itself alone. In fact,
540−an urbit has the right to compute incorrectly if it wants. We go to considerable
264+Fare' is totally right that an urbit's data is meant for itself alone. In fact, an urbit has the right to compute incorrectly if it wants. We go to considerable lengths to ensure that you can't gain any particular advantage by incorrect computing.
541−lengths to ensure that you can't gain any particular advantage by incorrect
542−computing.
543
265
544−The whole point of Nock is that it's so low-level that there is no conceivable
545−advantage in upgrading it. Divergence is just pointless. This is not a question
266+The whole point of Nock is that it's so low-level that there is no conceivable advantage in upgrading it. Divergence is just pointless. This is not a question of network integrity, just of practical standardization.
546−of network integrity, just of practical standardization.
547
267
548−This consistency stops at Nock. Without anyone violating the correctness of
549−their lifecycle functions, anyone or everyone in Urbit could replace Hoon with
268+This consistency stops at Nock. Without anyone violating the correctness of their lifecycle functions, anyone or everyone in Urbit could replace Hoon with Javascript. Determinism does not cramp Urbit's semantic style in any way, shape or form.
550−Javascript. Determinism does not cramp Urbit's semantic style in any way, shape
551−or form.
docs/about/overview.md+58−47
@@ -1,77 +1,88 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
3+next: true
4sort: 1
4sort: 1
5title: Technical overview
5title: Technical overview
6---
6---
7
7
8# Technical overview
8# Technical overview
9
9
10−Urbit is a clean-slate system software stack defined as a deterministic
10+Urbit is a clean-slate system software stack defined as a
11−computer. An encrypted P2P network, `%ames`, runs on a functional operating
11+deterministic computer. An encrypted P2P network, `%ames`, runs
12+on a functional operating system, Arvo, written in a strict,
12−system, Arvo, written in a strict, typed functional language, Hoon, which
13+typed functional language, Hoon, which compiles itself to a
13−compiles itself to a combinator interpreter, Nock, whose spec gzips to 340
14+combinator interpreter, Nock, whose spec gzips to 340 bytes.
14−bytes.
15
15
16## Architectural overview
16## Architectural overview
17
17
18−A deterministic computer? Urbit's state is a pure function of its event history.
18+A deterministic computer? Urbit's state is a pure function of
19−In practice it uses a memory checkpoint and an append-only log. Every event is a
19+its event history. In practice it uses a memory checkpoint and
20+an append-only log. Every event is a transaction; Urbit is an
20−transaction; Urbit is an ACID database and a single-level store. Urbit runs on
21+ACID database and a single-level store. Urbit runs on Unix now,
21−Unix now, but it's easy to imagine on a hypervisor or even bare metal.
22+but it's easy to imagine on a hypervisor or even bare metal.
22
23
23−A purely functional OS? Urbit is pure -- no code inside it can make system calls
24+A purely functional OS? Urbit is pure -- no code inside it can
24−or otherwise affect the underlying platform. Instead, the top-level event
25+make system calls or otherwise affect the underlying platform.
26+Instead, the top-level event function defines an I/O protocol.
25−function defines an I/O protocol. It maps an input event and the current state
27+It maps an input event and the current state to a list of output
26−to a list of output actions and the subsequent state. In Hoon:
28+actions and the subsequent state. In Hoon:
27
29
30+```
28− $-([event state] [(list action) state])
31+$-([event state] [(list action) state])
32+```
29
33
30### Nock
34### Nock
31
35
32−Nock is a sort of nano-Lisp without syntax, symbols or lambdas. Most Lisps are
36+Nock is a sort of nano-Lisp without syntax, symbols or lambdas.
33−one-layer: they create a practical language by extending a theoretically simple
37+Most Lisps are one-layer: they create a practical language by
38+extending a theoretically simple interpreter. The abstraction is
34−interpreter. The abstraction is simple and the implementation is practical;
39+simple and the implementation is practical; there is no actual
35−there is no actual codebase both simple and practical. Hoon and Nock are two
40+codebase both simple and practical. Hoon and Nock are two
36−layers: Hoon compiles itself to pure Nock. Since Urbit is defined in Nock, not
41+layers: Hoon compiles itself to pure Nock. Since Urbit is
37−Hoon, we can upgrade Hoon over the air.
42+defined in Nock, not Hoon, we can upgrade Hoon over the air.
38
43
39−The Nock data model is especially trivial. A *noun* is an atom or a cell. An
44+The Nock data model is especially trivial. A *noun* is an atom
40−atom is any unsigned integer. A cell is an ordered pair of nouns. Nouns are
45+or a cell. An atom is any unsigned integer. A cell is an
41−acyclic and expose no pointer equality test.
46+ordered pair of nouns. Nouns are acyclic and expose no pointer
47+equality test.
42
48
43### Hoon
49### Hoon
44
50
45−Hoon is a strict combinator language that avoids mathematical theory and
51+Hoon is a strict combinator language that avoids mathematical
46−notation. It aims at a mechanical, imperative feel. Hoon uses ASCII digraphs
52+theory and notation. It aims at a mechanical, imperative feel.
53+Hoon uses ASCII digraphs instead of keywords; there are no
47−instead of keywords; there are no user-level macros. The type system infers only
54+user-level macros. The type system infers only forward and does
48−forward and does not use unification, but is not much weaker than Haskell's. The
55+not use unification, but is not much weaker than Haskell's. The
49compiler and inference engine is about 2000 lines of Hoon.
56compiler and inference engine is about 2000 lines of Hoon.
50
57
51### Arvo
58### Arvo
52
59
53−Arvo is an event-driven OS written in Hoon. It can upgrade itself and everything
60+Arvo is an event-driven OS written in Hoon. It can upgrade
61+itself and everything inside it over the network. The Arvo
54−inside it over the network. The Arvo kernel proper is 500 lines of Hoon, which
62+kernel proper is 500 lines of Hoon, which implements a typed
55−implements a typed event system with explicit call-stack structure. Arvo ships
63+event system with explicit call-stack structure. Arvo ships
56−with modules that provide P2P networking (`%ames`), a revision- control system
64+with modules that provide P2P networking (`%ames`), a revision-
57−(`%clay`), a web client/server (`%eyre`), a functional build system (`%ford`),
65+control system (`%clay`), a web client/server (`%eyre`), a
58−and an application engine `(%gall)`.
66+functional build system (`%ford`), and an application engine
67+`(%gall)`.
59
68
60### Ames
69### Ames
61
70
62−Ames, the Urbit network, is an encrypted P2P protocol over UDP. Its address
71+Ames, the Urbit network, is an encrypted P2P protocol over UDP.
63−space is semi-decentralized; 64-bit addresses are hierarchically distributed,
72+Its address space is semi-decentralized; 64-bit addresses are
73+hierarchically distributed, 128-bit addresses are self-created.
64−128-bit addresses are self-created. Addresses (or *plots*) are rendered in a
74+Addresses (or *plots*) are rendered in a phonemic syntax for
65−phonemic syntax for memorability. The scarcity of short plots helps control spam
75+memorability. The scarcity of short plots helps control spam and
66−and other Sybil attacks. The short plot hierarchy is also reused as a supernode
76+other Sybil attacks. The short plot hierarchy is also reused as
67−routing system for NAT traversal.
77+a supernode routing system for NAT traversal.
68
78
69### Dojo and talk
79### Dojo and talk
70
80
71−Urbit ships with two default applications: a REPL or shell `:dojo`, and a
81+Urbit ships with two default applications: a REPL or shell
72−distributed user-level message-bus `:talk`. `:talk` under the hood resembles
82+`:dojo`, and a distributed user-level message-bus `:talk`.
83+`:talk` under the hood resembles NNTP; to the user, it looks like
73−NNTP; to the user, it looks like a self-hosted Slack or persistent IRC.
84+a self-hosted Slack or persistent IRC.
74
85
75−The full Urbit stack (compiler, standard library, kernel, modules, and
86+The full Urbit stack (compiler, standard library, kernel,
76−applications) is about 25,000 lines of Hoon. Urbit is patent-free and MIT
87+modules, and applications) is about 25,000 lines of Hoon.
77−licensed.
88+Urbit is patent-free and MIT licensed.
docs/about/runtime.md+923−794
@@ -1,60 +1,67 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 2
3sort: 2
4title: C runtime system
4title: C runtime system
5---
5---
6
6
7# C runtime system
7# C runtime system
8
8
9−The Urbit interpreter is built on a Nock runtime system written in C, `u3`. This
9+The Urbit interpreter is built on a Nock runtime system written
10−document is a relatively complete description
10+in C, `u3`. This document is a relatively complete description
11
11
12−You should keep reading if (a) you're planning to work on the Urbit interpreter;
12+You should keep reading if (a) you're planning to work on the
13−(b) you're a language implementation geek; or (c) you don't really understand
13+Urbit interpreter; (b) you're a language implementation geek; or
14−anything until you've seen the actual structs.
14+(c) you don't really understand anything until you've seen the
15+actual structs.
15
16
16## u3: noun processing in C
17## u3: noun processing in C
17
18
18−`u3` is the C library that makes Urbit work. If it wasn't called `u3`, it might
19+`u3` is the C library that makes Urbit work. If it wasn't called
19−be called `libnoun` - it's a library for making and storing nouns.
20+`u3`, it might be called `libnoun` - it's a library for making
21+and storing nouns.
20
22
21−What's a noun? A noun is either a cell or an atom. A cell is an ordered pair of
23+What's a noun? A noun is either a cell or an atom. A cell is an
22−any two nouns. An atom is an unsigned integer of any size.
24+ordered pair of any two nouns. An atom is an unsigned integer of
25+any size.
23
26
24−To the C programmer, this is not a terribly complicated data structure, so why
27+To the C programmer, this is not a terribly complicated data
25−do you need a library for it?
28+structure, so why do you need a library for it?
26
29
27−One: nouns have a well-defined computation kernel, Nock, whose spec fits on a
30+One: nouns have a well-defined computation kernel, Nock, whose
28−page and gzips to 340 bytes. But the only arithmetic operation in Nock is
31+spec fits on a page and gzips to 340 bytes. But the only
32+arithmetic operation in Nock is increment. So it's nontrivial
29−increment. So it's nontrivial to compute both efficiently and correctly.
33+to compute both efficiently and correctly.
30
34
31−Two: `u3` is designed to be a "solid-state interpreter," ie, a single-level
35+Two: `u3` is designed to be a "solid-state interpreter," ie, a
32−store which is transparently snapshotted. This implies a specialized
36+single-level store which is transparently snapshotted. This
33−memory-management model, etc, etc.
37+implies a specialized memory-management model, etc, etc.
34
38
35−(Does `u3` depend on the higher levels of Urbit, Arvo and Hoon? Yes and no. `u3`
39+(Does `u3` depend on the higher levels of Urbit, Arvo and Hoon?
36−expects you to load something shaped like an Arvo kernel, and use it as an
40+Yes and no. `u3` expects you to load something shaped like an
41+Arvo kernel, and use it as an event-processing function. But you
37−event-processing function. But you don't need to use this feature if you don't
42+don't need to use this feature if you don't want, and your kernel
38−want, and your kernel doesn't have to be Arvo proper - just Arvo-compatible.
43+doesn't have to be Arvo proper - just Arvo-compatible. Think of
39−Think of `u3` as the BIOS and Arvo as the boot kernel. And there are no
44+`u3` as the BIOS and Arvo as the boot kernel. And there are no
40dependencies at all between Hoon the language and `u3`.)
45dependencies at all between Hoon the language and `u3`.)
41
46
42## c3: C in Urbit
47## c3: C in Urbit
43
48
44−Under `u3` is the simple `c3` layer, which is just how we write C in Urbit.
49+Under `u3` is the simple `c3` layer, which is just how we write C
50+in Urbit.
45
51
46−When writing C in u3, please of course follow the conventions of the code around
52+When writing C in u3, please of course follow the conventions of
47−you as regards indentation, etc. It's especially important that every function
53+the code around you as regards indentation, etc. It's especially
54+important that every function have a header comment, even if it
48−have a header comment, even if it says nothing interesting.
55+says nothing interesting.
49
56
50−But some of our idiosyncrasies go beyond convention. Yes, we've done awful
57+But some of our idiosyncrasies go beyond convention. Yes, we've
51−things to C. Here's what we did and why we did.
58+done awful things to C. Here's what we did and why we did.
52
59
53### c3: integer types
60### c3: integer types
54
61
55−First, it's generally acknowledged that underspecified integer types are C's
62+First, it's generally acknowledged that underspecified integer
56−worst disaster. C99 fixed this, but the `stdint` types are wordy and annoying.
63+types are C's worst disaster. C99 fixed this, but the `stdint`
57−We've replaced them with:
64+types are wordy and annoying. We've replaced them with:
58
65
59 /* Good integers.
66 /* Good integers.
60 */
67 */
@@ -81,47 +88,50 @@ We've replaced them with:
81 typedef uintptr_t c3_p; // pointer-length uint - really really bad
88 typedef uintptr_t c3_p; // pointer-length uint - really really bad
82 typedef intptr_t c3_ps; // pointer-length int - really really bad
89 typedef intptr_t c3_ps; // pointer-length int - really really bad
83
90
84−Some of these need explanation. A loobean is a Nock boolean - Nock, for
91+Some of these need explanation. A loobean is a Nock boolean -
85−mysterious reasons, uses 0 as true (always say "yes") and 1 as false (always say
92+Nock, for mysterious reasons, uses 0 as true (always say "yes")
86−"no").
93+and 1 as false (always say "no").
87
94
88−Nock and/or Hoon cannot tell the difference between a short atom and a long one,
95+Nock and/or Hoon cannot tell the difference between a short atom
89−but at the `u3` level every atom under `2^31` is direct. The `c3_l` type is
96+and a long one, but at the `u3` level every atom under `2^31` is
97+direct. The `c3_l` type is useful to annotate this. A `c3_m` is
90−useful to annotate this. A `c3_m` is a *mote* - a string of up to 4 characters
98+a *mote* - a string of up to 4 characters in a `c3_l`, least
91−in a `c3_l`, least significant byte first. A `c3_g` should be a 5-bit atom. Of
99+significant byte first. A `c3_g` should be a 5-bit atom. Of
92course, C cannot enforce these constraints, only document them.
100course, C cannot enforce these constraints, only document them.
93
101
94−Use the "bad" - ie, poorly specified - integer types only when interfacing with
102+Use the "bad" - ie, poorly specified - integer types only when
95−external code that expects them.
103+interfacing with external code that expects them.
96
104
97−An enormous number of motes are defined in `i/c/motes.h`. There is no reason to
105+An enormous number of motes are defined in `i/c/motes.h`. There
98−delete motes that aren't being used, or even to modularize the definitions. Keep
106+is no reason to delete motes that aren't being used, or even to
99−them alphabetical, though.
107+modularize the definitions. Keep them alphabetical, though.
100
108
101### c3: variables and variable naming
109### c3: variables and variable naming
102
110
103−The C3 style uses Hoon style TLV variable names, with a quasi Hungarian syntax.
111+The C3 style uses Hoon style TLV variable names, with a quasi
104−This is weird, but works really well, as long as what you're doing isn't
112+Hungarian syntax. This is weird, but works really well, as long
113+as what you're doing isn't hideously complicated. (Then it works
105−hideously complicated. (Then it works badly, but we shouldn't need anything
114+badly, but we shouldn't need anything hideous in u3.)
106−hideous in u3.)
107
115
108−A TLV variable name is a random pronounceable three-letter string, sometimes
116+A TLV variable name is a random pronounceable three-letter
109−with some vague relationship to its meaning, but usually not. Usually CVC
117+string, sometimes with some vague relationship to its meaning,
110−(consonant-vowel-consonant) is a good choice.
118+but usually not. Usually CVC (consonant-vowel-consonant) is a
119+good choice.
111
120
112−You should use TLVs much the way math people use Greek letters. The same concept
121+You should use TLVs much the way math people use Greek letters.
113−should in general get the same name across different contexts. When you're
122+The same concept should in general get the same name across
123+different contexts. When you're working in a given area, you'll
114−working in a given area, you'll tend to remember the binding from TLV to concept
124+tend to remember the binding from TLV to concept by sheer power
115−by sheer power of associative memory. When you come back to it, it's not that
125+of associative memory. When you come back to it, it's not that
116−hard to relearn. And of course, when in doubt, comment it.
126+hard to relearn. And of course, when in doubt, comment it.
117
127
118−Variables take pseudo-Hungarian suffixes, matching in general the suffix of the
128+Variables take pseudo-Hungarian suffixes, matching in general the
119−integer type:
129+suffix of the integer type:
120
130
121 c3_w wor_w; // 32-bit word
131 c3_w wor_w; // 32-bit word
122
132
123−Unlike in standard Hungarian, there is no change for pointer variables. C
133+Unlike in standard Hungarian, there is no change for pointer
124−structure variables take a `_u` suffix.
134+variables. C structure variables take a `_u` suffix.
125
135
126### c3: loobeans
136### c3: loobeans
127
137
@@ -135,19 +145,20 @@ The code (from `defs.h`) tells the story:
135 # define c3a(x, y) __(_(x) && _(y))
145 # define c3a(x, y) __(_(x) && _(y))
136 # define c3o(x, y) __(_(x) || _(y))
146 # define c3o(x, y) __(_(x) || _(y))
137
147
138−In short, use `_()` to turn a loobean into a boolean, `__` to go the other way.
148+In short, use `_()` to turn a loobean into a boolean, `__` to go
139−Use `!` as usual, `c3y` for yes and `c3n` for no, `c3a` for and and `c3o` for
149+the other way. Use `!` as usual, `c3y` for yes and `c3n` for no,
140−or.
150+`c3a` for and and `c3o` for or.
141
151
142## u3: land of nouns
152## u3: land of nouns
143
153
144−The division between `c3` and `u3` is that you could theoretically imagine using
154+The division between `c3` and `u3` is that you could theoretically
145−`c3` as just a generic C environment. Anything to do with nouns is in `u3`.
155+imagine using `c3` as just a generic C environment. Anything to do
156+with nouns is in `u3`.
146
157
147### u3: a map of the system
158### u3: a map of the system
148
159
149−There are two kinds of symbols in `u3`: regular and irregular. Regular symbols
160+There are two kinds of symbols in `u3`: regular and irregular.
150−follow this pattern:
161+Regular symbols follow this pattern:
151
162
152 prefix purpose .h .c
163 prefix purpose .h .c
153 -------------------------------------------------------
164 -------------------------------------------------------
@@ -167,20 +178,23 @@ follow this pattern:
167 u3q[a-g] jets (retain, C args) i/j/q.h j/[a-g]/*.c
178 u3q[a-g] jets (retain, C args) i/j/q.h j/[a-g]/*.c
168 u3w[a-g] jets (retain, nock core) i/j/w.h j/[a-g]/*.c
179 u3w[a-g] jets (retain, nock core) i/j/w.h j/[a-g]/*.c
169
180
170−Irregular symbols always start with `u3` and obey no other rules. They're
181+Irregular symbols always start with `u3` and obey no other rules.
171−defined in `i/n/u.h`. Finally, `i/all.h` includes all these headers (fast
182+They're defined in `i/n/u.h`. Finally, `i/all.h` includes all
172−compilers, yay) and is all you need to program in `u3`.
183+these headers (fast compilers, yay) and is all you need to
184+program in `u3`.
173
185
174### u3: noun internals
186### u3: noun internals
175
187
176A noun is a `u3_noun` - currently defined as a 32-bit `c3_w`.
188A noun is a `u3_noun` - currently defined as a 32-bit `c3_w`.
177
189
178−If your `u3_noun` is less than `(1 << 31)`, it's a direct atom. Every unsigned
190+If your `u3_noun` is less than `(1 << 31)`, it's a direct atom.
179−integer between `0` and `0x7fffffff` inclusive is its own noun.
191+Every unsigned integer between `0` and `0x7fffffff` inclusive is
192+its own noun.
180
193
181−If bit `31` is set in a `u3_noun`, bit `30` is always set - this bit is
194+If bit `31` is set in a `u3_noun`, bit `30` is always set - this
182−reserved. Bit `29` is `1` if the noun is a cell, `0` if it's an atom. Bits `28`
195+bit is reserved. Bit `29` is `1` if the noun is a cell, `0` if
196+it's an atom. Bits `28` through `0` are a word pointer into the
183−through `0` are a word pointer into the loom - see below. The structures are:
197+loom - see below. The structures are:
184
198
185 typedef struct {
199 typedef struct {
186 c3_w mug_w;
200 c3_w mug_w;
@@ -194,53 +208,57 @@ through `0` are a word pointer into the loom - see below. The structures are:
194 u3_noun tel;
208 u3_noun tel;
195 } u3a_cell;
209 } u3a_cell;
196
210
197−The only thing that should be mysterious here is `mug_w`, which is a 31-bit
211+The only thing that should be mysterious here is `mug_w`, which
198−lazily computed nonzero short hash (FNV currently, soon Murmur3). If `mug_w` is
212+is a 31-bit lazily computed nonzero short hash (FNV currently,
213+soon Murmur3). If `mug_w` is 0, the hash is not yet computed.
199−0, the hash is not yet computed. We also hijack this field for various hacks,
214+We also hijack this field for various hacks, such as saving the
200−such as saving the new address of a noun when copying over.
215+new address of a noun when copying over.
201
216
202−Also, the value `0xffffffff` is `u3_none`, which is never a valid noun. Use the
217+Also, the value `0xffffffff` is `u3_none`, which is never a valid
203−type `u3_weak` to express that a noun variable may be `u3_none`.
218+noun. Use the type `u3_weak` to express that a noun variable may
219+be `u3_none`.
204
220
205### u3: reference counts
221### u3: reference counts
206
222
207−The only really essential thing you need to know about `u3` is how to handle
223+The only really essential thing you need to know about `u3` is
208−reference counts. Everything else, you can skip and just get to work.
224+how to handle reference counts. Everything else, you can skip
225+and just get to work.
209
226
210−u3 deals with reference-counted, immutable, acyclic nouns. Unfortunately, we are
227+u3 deals with reference-counted, immutable, acyclic nouns.
228+Unfortunately, we are not Apple and can't build reference
211−not Apple and can't build reference counting into your C compiler, so you need
229+counting into your C compiler, so you need to count by hand.
212−to count by hand.
213
230
214−Every allocated noun (or any allocation object, because our allocator is
231+Every allocated noun (or any allocation object, because our
215−general-purpose) contains a counter which counts the number of references to it
232+allocator is general-purpose) contains a counter which counts the
233+number of references to it - typically variables with type
216−- typically variables with type `u3_noun`. When this counter goes to 0, the noun
234+`u3_noun`. When this counter goes to 0, the noun is freed.
217−is freed.
218
235
219−To tell `u3` that you've added a reference to a noun, call the function
236+To tell `u3` that you've added a reference to a noun, call the
220−`u3a_gain()` or its shorthand `u3k()`. (For your convenience, this function
237+function `u3a_gain()` or its shorthand `u3k()`. (For your
238+convenience, this function returns its argument.) To tell `u3`
221−returns its argument.) To tell `u3` that you've destroyed a reference, call
239+that you've destroyed a reference, call `u3a_lose()` or `u3z()`.
222−`u3a_lose()` or `u3z()`.
223
240
224−(If you screw up by decrementing the counter too much, `u3` will dump core in
241+(If you screw up by decrementing the counter too much, `u3` will
225−horrible ways. If you screw up by incrementing it too much, `u3` will leak
242+dump core in horrible ways. If you screw up by incrementing it
243+too much, `u3` will leak memory. To check for memory leaks,
226−memory. To check for memory leaks, set the `bug_o` flag in `u3e_boot()` - eg,
244+set the `bug_o` flag in `u3e_boot()` - eg, run `vere` with `-g`.
227−run `vere` with `-g`. Memory leaks are difficult to debug - the best way to
245+Memory leaks are difficult to debug - the best way to handle
228−handle leaks is just to revert to a version that didn't have them, and look over
246+leaks is just to revert to a version that didn't have them, and
229−your code again.)
247+look over your code again.)
230
248
231−(You can gain or lose a direct atom. It does nothing.)
249+(You can gain or lose a direct atom. It does nothing.)
232
250
233### u3: reference protocols
251### u3: reference protocols
234
252
235*THIS IS THE MOST CRITICAL SECTION IN THE `u3` DOCUMENTATION.*
253*THIS IS THE MOST CRITICAL SECTION IN THE `u3` DOCUMENTATION.*
236
254
237−The key question when calling a C function in a refcounted world is what the
255+The key question when calling a C function in a refcounted world
238−function will do to the noun refcounts - and, if the function returns a noun,
256+is what the function will do to the noun refcounts - and, if the
239−what it does to the return.
257+function returns a noun, what it does to the return.
240
258
241−There are two semantic patterns, `transfer` and `retain`. In `transfer`
259+There are two semantic patterns, `transfer` and `retain`. In
242−semantics, the caller "gives" a use count to the callee, which "gives back" any
260+`transfer` semantics, the caller "gives" a use count to the
243−return. For instance, if I have
261+callee, which "gives back" any return. For instance, if I have
244
262
245 {
263 {
246 u3_noun foo = u3i_string("foobar");
264 u3_noun foo = u3i_string("foobar");
@@ -251,12 +269,14 @@ return. For instance, if I have
251 u3z(bar);
269 u3z(bar);
252 }
270 }
253
271
254−Suppose `u3f_futz()` has `transfer` semantics. At `[...]`, my code holds one
272+Suppose `u3f_futz()` has `transfer` semantics. At `[...]`, my
255−reference to `bar` and zero references to `foo` - which has been freed, unless
273+code holds one reference to `bar` and zero references to `foo` -
274+which has been freed, unless it's part of `bar`. My code now
256−it's part of `bar`. My code now owns `bar` and gets to work with it until it's
275+owns `bar` and gets to work with it until it's done, at which
257−done, at which point a `u3z()` is required.
276+point a `u3z()` is required.
258
277
259−On the other hand, if `u3f_futz()` has `retain` semantics, we need to write
278+On the other hand, if `u3f_futz()` has `retain` semantics, we
279+need to write
260
280
261 {
281 {
262 u3_noun foo = u3i_string("foobar");
282 u3_noun foo = u3i_string("foobar");
@@ -267,71 +287,75 @@ On the other hand, if `u3f_futz()` has `retain` semantics, we need to write
267 u3z(foo);
287 u3z(foo);
268 }
288 }
269
289
270−because calling `u3f_futz()` does not release our ownership of `foo`, which we
290+because calling `u3f_futz()` does not release our ownership of
271−have to free ourselves.
291+`foo`, which we have to free ourselves.
272
292
273−But if we free `bar`, we are making a great mistake, because our reference to it
293+But if we free `bar`, we are making a great mistake, because our
274−is not in any way registered in the memory manager (which cannot track
294+reference to it is not in any way registered in the memory
295+manager (which cannot track references in C variables, of
275−references in C variables, of course). It is normal and healthy to have these
296+course). It is normal and healthy to have these uncounted
276−uncounted C references, but they must be treated with care.
297+C references, but they must be treated with care.
277
298
278−The bottom line is that it's essential for the caller to know the refcount
299+The bottom line is that it's essential for the caller to know
279−semantics of any function which takes or returns a noun. (In some unusual
300+the refcount semantics of any function which takes or returns a
301+noun. (In some unusual circumstances, different arguments or
280−circumstances, different arguments or returns in one function may be handled
302+returns in one function may be handled differently.)
281−differently.)
282
303
283−Broadly speaking, as a design question, retain semantics are more appropriate
304+Broadly speaking, as a design question, retain semantics are more
284−for functions which inspect or query nouns. For instance, `u3h()` (which takes
305+appropriate for functions which inspect or query nouns. For
306+instance, `u3h()` (which takes the head of a noun) retains, so
285−the head of a noun) retains, so that we can traverse a noun tree without
307+that we can traverse a noun tree without constantly incrementing
286−constantly incrementing and decrementing.
308+and decrementing.
287
309
288−Transfer semantics are more appropriate for functions which make nouns, which is
310+Transfer semantics are more appropriate for functions which make
289−obviously what most functions do.
311+nouns, which is obviously what most functions do.
290
312
291−In general, though, in most places it's not worth thinking about what your
313+In general, though, in most places it's not worth thinking about
292−function does. There is a convention for it, which depends on where it is, not
314+what your function does. There is a convention for it, which
293−what it does. Follow the convention.
315+depends on where it is, not what it does. Follow the convention.
294
316
295### u3: reference conventions
317### u3: reference conventions
318+
319+The `u3` convention is that, unless otherwise specified, *all
320+functions have transfer semantics* - with the exception of the
321+prefixes: `u3r`, `u3x`, `u3z`, `u3q` and `u3w`. Also, within
322+jet directories `a` through `f` (but not `g`), internal functions
323+retain (for historical reasons).
296
324
297−The `u3` convention is that, unless otherwise specified, *all functions have
298−transfer semantics* - with the exception of the prefixes: `u3r`, `u3x`, `u3z`,
299−`u3q` and `u3w`. Also, within jet directories `a` through `f` (but not `g`),
300−internal functions retain (for historical reasons).
301−
302−If functions outside this set have retain semantics, they need to be commented,
325+If functions outside this set have retain semantics, they need to
303−both in the `.h` and `.c` file, with `RETAIN` in all caps. Yes, it's this
326+be commented, both in the `.h` and `.c` file, with `RETAIN` in
304−important.
327+all caps. Yes, it's this important.
305
328
306### u3: system architecture
329### u3: system architecture
307
330
308−If you just want to tinker with some existing code, it might be enough to
331+If you just want to tinker with some existing code, it might be
309−understand the above. If not, it's probably worth taking the time to look at
332+enough to understand the above. If not, it's probably worth
310−`u3` as a whole.
333+taking the time to look at `u3` as a whole.
311
334
312−`u3` is designed to work as a persistent event processor. Logically, it computes
335+`u3` is designed to work as a persistent event processor.
313−a function of the form
336+Logically, it computes a function of the form
314
337
315 f(event, old state) -> (actions, new state)
338 f(event, old state) -> (actions, new state)
316
339
317−Obviously almost any computing model - including, but not limited to, Urbit -
340+Obviously almost any computing model - including, but not limited
318−can be defined in this form. To create the illusion of a computer that never
341+to, Urbit - can be defined in this form. To create the illusion
319−loses state and never fails, we:
342+of a computer that never loses state and never fails, we:
320
343
321−- log every event externally before it goes into u3
344+- log every event externally before it goes into u3
322−- keep a single reference to a permanent state noun.
345+- keep a single reference to a permanent state noun.
323−- can abort any event without damaging the permanent state.
346+- can abort any event without damaging the permanent state.
324−- snapshot the permanent state periodically, and/or prune logs.
347+- snapshot the permanent state periodically, and/or prune logs.
325
348
326### u3: the road model
349### u3: the road model
327
350
328−`u3` uses a memory design which I'm sure someone has invented somewhere before,
351+`u3` uses a memory design which I'm sure someone has invented
329−because it's not very clever, but I've never seen it anywhere in particular.
352+somewhere before, because it's not very clever, but I've never
353+seen it anywhere in particular.
330
354
331−Every allocation starts with a solid block of memory, which `u3` calls the
355+Every allocation starts with a solid block of memory, which `u3`
332−`loom`. How do we allocate on the loom? You're probably familiar with the Unix
356+calls the `loom`. How do we allocate on the loom? You're
357+probably familiar with the Unix heap-stack design, in which the
333−heap-stack design, in which the stack grows downward and the heap (malloc arena)
358+stack grows downward and the heap (malloc arena) grows upward:
334−grows upward:
335
359
336 0 brk ffff
360 0 brk ffff
337 | heap | stack |
361 | heap | stack |
@@ -339,35 +363,40 @@ grows upward:
339 | | |
363 | | |
340 0 sp ffff
364 0 sp ffff
341
365
342−A road is a normal heap-stack system, except that the heap and stack can point
366+A road is a normal heap-stack system, except that the heap
343−in *either direction*. Therefore, inside a road, we can nest another road in the
367+and stack can point in *either direction*. Therefore, inside
344−*opposite direction*.
368+a road, we can nest another road in the *opposite direction*.
345
369
346−When the opposite road completes, its heap is left on top of the opposite heap's
370+When the opposite road completes, its heap is left on top of
347−stack. It's no more than the normal behavior of a stack machine for all
371+the opposite heap's stack. It's no more than the normal
372+behavior of a stack machine for all subcomputations to push
348−subcomputations to push their results on the stack.
373+their results on the stack.
349
374
350−The performance tradeoff of "leaping" - reversing directions in the road - is
375+The performance tradeoff of "leaping" - reversing directions in
351−that if the outer computation wants to preserve the results of the inner one,
376+the road - is that if the outer computation wants to preserve the
377+results of the inner one, not just use them for temporary
352−not just use them for temporary purposes, it has to *copy them*.
378+purposes, it has to *copy them*.
353
379
354−This is a trivial cost in some cases, a prohibitive cost in others. The upside,
380+This is a trivial cost in some cases, a prohibitive cost in
381+others. The upside, of course, is that all garbage accrued
355−of course, is that all garbage accrued in the inner computation is discarded at
382+in the inner computation is discarded at zero cost.
356−zero cost.
357
383
358−The goal of the road system is the ability to *layer* memory models. If you are
384+The goal of the road system is the ability to *layer* memory
385+models. If you are allocating on a road, you have no idea
359−allocating on a road, you have no idea how deep within a nested road system you
386+how deep within a nested road system you are - in other words,
360−are - in other words, you have no idea exactly how durable your result may be.
387+you have no idea exactly how durable your result may be.
361But free space is never fragmented within a road.
388But free space is never fragmented within a road.
362
389
363−Roads do not reduce the generality or performance of a memory system, since even
390+Roads do not reduce the generality or performance of a memory
364−the most complex GC system can be nested within a road at no particular loss of
391+system, since even the most complex GC system can be nested
392+within a road at no particular loss of performance - a road
365−performance - a road is just a block of memory.
393+is just a block of memory.
366
394
367−Each road (`u3a_road` to be exact) uses four pointers: `rut` is the bottom of
395+Each road (`u3a_road` to be exact) uses four pointers: `rut` is
368−the arena, `hat` the top of the arena, `mat` the bottom of the stack, `cap` the
396+the bottom of the arena, `hat` the top of the arena, `mat` the
397+bottom of the stack, `cap` the top of the stack. (Bear in mind
369−top of the stack. (Bear in mind that the road "stack" is not actually used as
398+that the road "stack" is not actually used as the C function-call
370−the C function-call stack, though it probably should be.)
399+stack, though it probably should be.)
371
400
372A "north" road has the stack high and the heap low:
401A "north" road has the stack high and the heap low:
373
402
@@ -385,112 +414,125 @@ A "south" road is the other way around:
385 | | | |
414 | | | |
386 0 hat rut ffff
415 0 hat rut ffff
387
416
388−Legend: `-` is durable storage (heap); `+` is temporary storage (stack); `~` is
417+Legend: `-` is durable storage (heap); `+` is temporary storage
389−deep storage (immutable); `$` is the allocation frame `#` is free memory.
418+(stack); `~` is deep storage (immutable); `$` is the allocation
419+frame `#` is free memory.
390
420
391−Pointer restrictions: pointers stored in `+` can point anywhere. Pointers in `-`
421+Pointer restrictions: pointers stored in `+` can point anywhere.
392−can only point to `-` or `~`; pointers in `~` only point to `~`.
422+Pointers in `-` can only point to `-` or `~`; pointers in `~`
423+only point to `~`.
393
424
394−To "leap" is to create a new inner road in the `###` free space. but in the
425+To "leap" is to create a new inner road in the `###` free space.
395−reverse direction, so that when the inner road "falls" (terminates), its durable
426+but in the reverse direction, so that when the inner road
427+"falls" (terminates), its durable storage is left on the
396−storage is left on the temporary storage of the outer road.
428+temporary storage of the outer road.
397
429
398−`u3` keeps a global variable, `u3_Road` or its alias `u3R`, which points to the
430+`u3` keeps a global variable, `u3_Road` or its alias `u3R`, which
399−current road. (If we ever run threads in inner roads - see below - this will
431+points to the current road. (If we ever run threads in inner
432+roads - see below - this will become a thread-local variable.)
400−become a thread-local variable.) Relative to `u3R`, `+` memory is called
433+Relative to `u3R`, `+` memory is called `junior` memory; `-`
401−`junior` memory; `-` memory is `normal` memory; `~` is `senior` memory.
434+memory is `normal` memory; `~` is `senior` memory.
402
435
403### u3: explaining the road model
436### u3: explaining the road model
404
437
405But... why?
438But... why?
406
439
407−We're now ready to understand why the road system works so logically with the
440+We're now ready to understand why the road system works so
408−event and persistence model.
441+logically with the event and persistence model.
409−
442+
410−The key is that *we don't update refcounts in senior memory.* A pointer from an
443+The key is that *we don't update refcounts in senior memory.*
411−inner road to an outer road is not counted. Also, the outmost, or `surface`
444+A pointer from an inner road to an outer road is not counted.
445+Also, the outmost, or `surface` road, is the only part of the
412−road, is the only part of the image that gets checkpointed.
446+image that gets checkpointed.
413−
447+
414−So the surface road contains the entire durable state of `u3`. When we process
448+So the surface road contains the entire durable state of `u3`.
415−an event, or perform any kind of complicated or interesting calculation, *we
449+When we process an event, or perform any kind of complicated or
450+interesting calculation, *we process it in an inner road*. If
416−process it in an inner road*. If its results are saved, they need to be copied.
451+its results are saved, they need to be copied.
417−
452+
418−Since processing in an inner road does not touch surface memory, (a) we can
453+Since processing in an inner road does not touch surface memory,
419−leave the surface road in a read-only state and not mark its pages dirty; (b) we
454+(a) we can leave the surface road in a read-only state and not
455+mark its pages dirty; (b) we can abort an inner calculation
420−can abort an inner calculation without screwing up the surface; and (c) because
456+without screwing up the surface; and (c) because inner results
421−inner results are copied onto the surface, the surface doesn't get fragmented.
457+are copied onto the surface, the surface doesn't get fragmented.
422−
458+
423−All of (a), (b) and (c) are needed for checkpointing to be easy. It might be
459+All of (a), (b) and (c) are needed for checkpointing to be easy.
424−tractable otherwise, but easy is even better.
460+It might be tractable otherwise, but easy is even better.
425−
461+
426−Moreover, while the surface is most definitely single-threaded, we could easily
462+Moreover, while the surface is most definitely single-threaded,
427−run multiple threads in multiple inner roads (as long as the threads don't have
463+we could easily run multiple threads in multiple inner roads
464+(as long as the threads don't have pointers into each others'
428−pointers into each others' memory, which they obviously shouldn't).
465+memory, which they obviously shouldn't).
429−
466+
430−Moreover, in future, we'll experiment more with adding road control hints to the
467+Moreover, in future, we'll experiment more with adding road
431−programmer's toolbox. Reference counting is expensive. We hypothesize that in
468+control hints to the programmer's toolbox. Reference counting is
469+expensive. We hypothesize that in many - if not most - cases,
432−many - if not most - cases, the programmer can identify procedural structures
470+the programmer can identify procedural structures whose garbage
433−whose garbage should be discarded in one step by copying the results. Then,
471+should be discarded in one step by copying the results. Then,
434−within the procedure, we can switch the allocator into `sand` mode, and stop
472+within the procedure, we can switch the allocator into `sand`
435−tracking references at all.
473+mode, and stop tracking references at all.
436
474
437### u3: rules for C programming
475### u3: rules for C programming
438
476
439−There are two levels at which we program in C: (1) above the interpreter; (2)
477+There are two levels at which we program in C: (1) above the
440−within the interpreter or jets. These have separate rules which need to be
478+interpreter; (2) within the interpreter or jets. These have
441−respected.
479+separate rules which need to be respected.
442
480
443### u3: rules above the interpreter
481### u3: rules above the interpreter
444
482
445−In its relations with Unix, Urbit follows a strict rule of "call me, I won't
483+In its relations with Unix, Urbit follows a strict rule of "call
446−call you." We do of course call Unix system calls, but only for the purpose of
484+me, I won't call you." We do of course call Unix system calls,
447−actually computing.
485+but only for the purpose of actually computing.
448
486
449−Above Urbit, you are in a normal C/Unix programming environment and can call
487+Above Urbit, you are in a normal C/Unix programming environment
450−anything in or out of Urbit. Note that when using `u3`, you're always on the
488+and can call anything in or out of Urbit. Note that when using
451−surface road, which is not thread-safe by default. Generally speaking, `u3` is
489+`u3`, you're always on the surface road, which is not thread-safe
490+by default. Generally speaking, `u3` is designed to support
452−designed to support event-oriented, single-threaded programming.
491+event-oriented, single-threaded programming.
453
492
454−If you need threads which create nouns, you could use `u3m_hate()` and
493+If you need threads which create nouns, you could use
455−`u3m_love()` to run these threads in subroads. You'd need to make the global
494+`u3m_hate()` and `u3m_love()` to run these threads in subroads.
495+You'd need to make the global road pointer, `u3R`, a thread-local
456−road pointer, `u3R`, a thread-local variable instead. This seems perfectly
496+variable instead. This seems perfectly practical, but we haven't
457−practical, but we haven't done it because we haven't needed to.
497+done it because we haven't needed to.
458
498
459### u3: rules within the interpreter
499### u3: rules within the interpreter
460
500
461−Within the interpreter, your code can run either in the surface road or in a
501+Within the interpreter, your code can run either in the surface
462−deep road. You can test this by testing
502+road or in a deep road. You can test this by testing
463
503
464 (&u3H->rod_u == u3R)
504 (&u3H->rod_u == u3R)
465
505
466ie: does the pier's home road equal the current road pointer?
506ie: does the pier's home road equal the current road pointer?
467
507
468−Normally in this context you assume you're obeying the rules of running on an
508+Normally in this context you assume you're obeying the rules of
469−inner road, ie, "deep memory." Remember, however, that the interpreter *can* run
509+running on an inner road, ie, "deep memory." Remember, however,
510+that the interpreter *can* run on surface memory - but anything
470−on surface memory - but anything you can do deep, you can do on the surface. The
511+you can do deep, you can do on the surface. The converse is by
471−converse is by no means the case.
512+no means the case.
472
513
473−In deep memory, think of yourself as if in a signal handler. Your execution
514+In deep memory, think of yourself as if in a signal handler.
474−context is extremely fragile and may be terminated without warning or cleanup at
515+Your execution context is extremely fragile and may be terminated
475−any time (for instance, by `ctrl-c`).
516+without warning or cleanup at any time (for instance, by `ctrl-c`).
476
517
477−For instance, you can't call `malloc` (or C++ `new`) in your C code, because you
518+For instance, you can't call `malloc` (or C++ `new`) in your C
478−don't have the right to modify data structures at the global level, and will
519+code, because you don't have the right to modify data structures
520+at the global level, and will leave them in an inconsistent state
479−leave them in an inconsistent state if your inner road gets terminated.
521+if your inner road gets terminated. (Instead, use our drop-in
480−(Instead, use our drop-in replacements, `u3a_malloc()`, `u3a_free()`,
522+replacements, `u3a_malloc()`, `u3a_free()`, `u3a_realloc()`.)
481−`u3a_realloc()`.)
482
523
483−A good example is the different meaning of `c3_assert()` inside and outside the
524+A good example is the different meaning of `c3_assert()` inside
484−interpreter. At either layer, you can use regular assert(), which will just kill
525+and outside the interpreter. At either layer, you can use
526+regular assert(), which will just kill your process. On the
485−your process. On the surface, `c3_assert()` will just... kill your process.
527+surface, `c3_assert()` will just... kill your process.
486
528
487−In deep execution, `c3_assert()` will issue an exception that queues an error
529+In deep execution, `c3_assert()` will issue an exception that
488−event, complete with trace stack, on the Arvo event queue. Let's see how this
530+queues an error event, complete with trace stack, on the Arvo
489−happens.
531+event queue. Let's see how this happens.
490
532
491### u3: exceptions
533### u3: exceptions
492
534
493−You produce an exception with
535+You produce an exception with
494
536
495 /* u3m_bail(): bail out. Does not return.
537 /* u3m_bail(): bail out. Does not return.
496 **
538 **
@@ -509,75 +551,82 @@ You produce an exception with
509 c3_i
551 c3_i
510 u3m_bail(c3_m how_m);
552 u3m_bail(c3_m how_m);
511
553
512−Broadly speaking, there are two classes of exception: internal and external. An
554+Broadly speaking, there are two classes of exception: internal
513−external exception begins in a Unix signal handler. An internal exception begins
555+and external. An external exception begins in a Unix signal
556+handler. An internal exception begins with a call to longjmp()
514−with a call to longjmp() on the main thread.
557+on the main thread.
515
558
516−There are also two kinds of exception: mild and severe. An external exception is
559+There are also two kinds of exception: mild and severe. An
517−always severe. An internal exception is normally mild, but some (like
560+external exception is always severe. An internal exception is
561+normally mild, but some (like `c3__oops`, generated by
518−`c3__oops`, generated by `c3_assert()`) are severe.
562+`c3_assert()`) are severe.
519
563
520−Either way, exceptions come with a stack trace. The `u3` nock interpreter is
564+Either way, exceptions come with a stack trace. The `u3` nock
521−instrumented to retain stack trace hints and produce them as a printable
565+interpreter is instrumented to retain stack trace hints and
522−`(list tank)`.
566+produce them as a printable `(list tank)`.
523
567
524−Mild exceptions are caught by the first virtualization layer and returned to the
568+Mild exceptions are caught by the first virtualization layer and
525−caller, following the behavior of the Nock virtualizer `++mock` (in `hoon.hoon`)
569+returned to the caller, following the behavior of the Nock
570+virtualizer `++mock` (in `hoon.hoon`)
526
571
527−Severe exceptions, or mild exceptions at the surface, terminate the entire
572+Severe exceptions, or mild exceptions at the surface, terminate
528−execution stack at any depth and send the cumulative trace back to the `u3`
573+the entire execution stack at any depth and send the cumulative
529−caller.
574+trace back to the `u3` caller.
530
575
531−For instance, `vere` uses this trace to construct a `%crud` event, which conveys
576+For instance, `vere` uses this trace to construct a `%crud`
532−our trace back toward the Arvo context where it crashed. This lets any UI
577+event, which conveys our trace back toward the Arvo context where
578+it crashed. This lets any UI component anywhere, even on a
533−component anywhere, even on a remote node, render the stacktrace as a
579+remote node, render the stacktrace as a consequence of the user's
534−consequence of the user's action - even if its its direct cause was (for
580+action - even if its its direct cause was (for instance) a Unix
535−instance) a Unix SIGINT or SIGALRM.
581+SIGINT or SIGALRM.
536
582
537### u3: C structures on the loom
583### u3: C structures on the loom
538
584
539−Normally, all data on the loom is nouns. Sometimes we break this rule just a
585+Normally, all data on the loom is nouns. Sometimes we break this
540−little, though - eg, in the `u3h` hashtables.
586+rule just a little, though - eg, in the `u3h` hashtables.
541
587
542−To point to non-noun C structs on the loom, we use a `u3_post`, which is just a
588+To point to non-noun C structs on the loom, we use a `u3_post`,
543−loom word offset. A macro lets us declare this as if it was a pointer:
589+which is just a loom word offset. A macro lets us declare this
590+as if it was a pointer:
544
591
545 typedef c3_w u3_post;
592 typedef c3_w u3_post;
546 #define u3p(type) u3_post
593 #define u3p(type) u3_post
547
594
548−Some may regard this as clever, others as pointless. Anyway, use `u3to()` and
595+Some may regard this as clever, others as pointless. Anyway, use
549−`u3of()` to convert to and from pointers.
596+`u3to()` and `u3of()` to convert to and from pointers.
550
597
551−When using C structs on the loom - generally a bad idea - make sure anything
598+When using C structs on the loom - generally a bad idea - make
552−which could be on the surface road is structurally portable, eg, won't change
599+sure anything which could be on the surface road is structurally
600+portable, eg, won't change size when the pointer size changes.
553−size when the pointer size changes. (Note also: we consider little-endian,
601+(Note also: we consider little-endian, rightly or wrongly, to
554−rightly or wrongly, to have won the endian wars.)
602+have won the endian wars.)
555
603
556## u3: API overview by prefix
604## u3: API overview by prefix
557
605
558−Let's run through the `u3` modules one by one. All public functions are
606+Let's run through the `u3` modules one by one. All public
559−commented, but the comments may be cryptic.
607+functions are commented, but the comments may be cryptic.
560
608
561### u3m: main control
609### u3m: main control
562
610
563−To start `u3`, run
611+To start `u3`, run
564
612
565 /* u3m_boot(): start the u3 system.
613 /* u3m_boot(): start the u3 system.
566 */
614 */
567 void
615 void
568 u3m_boot(c3_o nuu_o, c3_o bug_o, c3_c* dir_c);
616 u3m_boot(c3_o nuu_o, c3_o bug_o, c3_c* dir_c);
569
617
570−`nuu_o` is `c3y` (yes, `0`) if you're creating a new pier, `c3n` (no, `1`) if
618+`nuu_o` is `c3y` (yes, `0`) if you're creating a new pier,
571−you're loading an existing one. `bug_o` is `c3y` if you want to test the
619+`c3n` (no, `1`) if you're loading an existing one. `bug_o`
620+is `c3y` if you want to test the garbage-collector, `c3n`
572−garbage-collector, `c3n` otherwise. `dir_c` is the directory for the pier files.
621+otherwise. `dir_c` is the directory for the pier files.
573
622
574−`u3m_boot()` expects an `urbit.pill` file to load the kernel from. It will try
623+`u3m_boot()` expects an `urbit.pill` file to load the kernel
575−first `$dir/.urb.urbit.pill`, then `U3_LIB`.
624+from. It will try first `$dir/.urb.urbit.pill`, then `U3_LIB`.
576
625
577−Any significant computation with nouns, certainly anything Turing complete,
626+Any significant computation with nouns, certainly anything Turing
578−should be run (a) virtualized and (b) in an inner road. These are slightly
627+complete, should be run (a) virtualized and (b) in an inner road.
579−different things, but at the highest level we bundle them together for your
628+These are slightly different things, but at the highest level we
580−convenience, in `u3m_soft()`:
629+bundle them together for your convenience, in `u3m_soft()`:
581
630
582 /* u3m_soft(): system soft wrapper. unifies unix and nock errors.
631 /* u3m_soft(): system soft wrapper. unifies unix and nock errors.
583 **
632 **
@@ -586,160 +635,175 @@ convenience, in `u3m_soft()`:
586 u3_noun
635 u3_noun
587 u3m_soft(c3_w sec_w, u3_funk fun_f, u3_noun arg);
636 u3m_soft(c3_w sec_w, u3_funk fun_f, u3_noun arg);
588
637
589−`sec_w` is the number of seconds to time out the computation. `fun_f` is a C
638+`sec_w` is the number of seconds to time out the computation.
590−function accepting `arg`.
639+`fun_f` is a C function accepting `arg`.
591−
640+
592−The result of `u3m_soft()` is a cell whose head is an atom. If the head is `%$`
641+The result of `u3m_soft()` is a cell whose head is an atom. If
642+the head is `%$` - ie, `0` - the tail is the result of
593−- ie, `0` - the tail is the result of `fun_f(arg)`. Otherwise, the head is a
643+`fun_f(arg)`. Otherwise, the head is a `term` (an atom which is
594−`term` (an atom which is an LSB first string), and the tail is a `(list tank)`
644+an LSB first string), and the tail is a `(list tank)` (a list of
595−(a list of `tank` printables - see `++tank` in `hoon.hoon`). Error terms should
645+`tank` printables - see `++tank` in `hoon.hoon`). Error terms
596−be the same as the exception terms above.
646+should be the same as the exception terms above.
597−
647+
598−If you're confident that your computation won't fail, you can use
648+If you're confident that your computation won't fail, you can
599−`u3m_soft_sure()`, `u3m_soft_slam()`, or `u3m_soft_nock()` for C functions, Hoon
649+use `u3m_soft_sure()`, `u3m_soft_slam()`, or `u3m_soft_nock()`
600−function calls, and Nock invocations. Caution - this returns just the result,
650+for C functions, Hoon function calls, and Nock invocations.
601−and asserts globally.
651+Caution - this returns just the result, and asserts globally.
602−
652+
603−All the `u3m_soft` functions above work *only on the surface*. Within the
653+All the `u3m_soft` functions above work *only on the surface*.
604−surface, virtualize with `u3m_soft_run()`. Note that this takes a `fly` (a
654+Within the surface, virtualize with `u3m_soft_run()`. Note that
605−namespace gate), thus activating the `11` super-operator in the nock
655+this takes a `fly` (a namespace gate), thus activating the `11`
656+super-operator in the nock virtualizer, `++mock`. When actually
606−virtualizer, `++mock`. When actually using the `fly`, call `u3m_soft_esc()`.
657+using the `fly`, call `u3m_soft_esc()`. Don't do either unless
607−Don't do either unless you know what you're doing!
658+you know what you're doing!
608−
659+
609−For descending into a subroad *without* Nock virtualization, use `u3m_hate()`
660+For descending into a subroad *without* Nock virtualization,
610−and `u3m_love` respectively. Hating enters a subroad; loving leaves it, copying
661+use `u3m_hate()` and `u3m_love` respectively. Hating enters
611−out a product noun.
662+a subroad; loving leaves it, copying out a product noun.
612−
663+
613−Other miscellaneous tools in `u3m`: `u3m_file()` loads a Unix file as a Nock
664+Other miscellaneous tools in `u3m`: `u3m_file()` loads a Unix
614−atom; `u3m_water()` measures the boundaries of the loom in current use (ie,
665+file as a Nock atom; `u3m_water()` measures the boundaries of
666+the loom in current use (ie, watermarks); and a variety of
615−watermarks); and a variety of prettyprinting routines, none perfect, are
667+prettyprinting routines, none perfect, are available, mainly for
616−available, mainly for debugging printfs: `u3m_pretty()`, `u3m_p()`, `u3m_tape()`
668+debugging printfs: `u3m_pretty()`, `u3m_p()`, `u3m_tape()` and
617−and `u3m_wall()`.
669+`u3m_wall()`.
618−
670+
619−It's sometimes nice to run a mark-and-sweep garbage collector, `u3m_grab()`,
671+It's sometimes nice to run a mark-and-sweep garbage collector,
620−which collects the world from a list of roots, and asserts if it finds any leaks
672+`u3m_grab()`, which collects the world from a list of roots,
673+and asserts if it finds any leaks or incorrect refcounts. This
621−or incorrect refcounts. This tool is for debugging and long-term maintenance
674+tool is for debugging and long-term maintenance only; refcounts
622−only; refcounts should never err.
675+should never err.
623
676
624### u3j: jets
677### u3j: jets
625
678
626−The jet system, `u3j`, is what makes `u3` and `nock` in any sense a useful
679+The jet system, `u3j`, is what makes `u3` and `nock` in any sense
627−computing environment. Except perhaps `u3a` (there is really no such thing as a
680+a useful computing environment. Except perhaps `u3a` (there is
628−trivial allocator, though `u3a` is dumber than most) - `u3j` is the most
681+really no such thing as a trivial allocator, though `u3a` is
629−interesting code in `u3`.
682+dumber than most) - `u3j` is the most interesting code in `u3`.
630
683
631−Let's consider the minor miracle of driver-to-battery binding which lets `u3j`
684+Let's consider the minor miracle of driver-to-battery binding
632−work - and decrement not be `O(n)` - without violating the precisely defined
685+which lets `u3j` work - and decrement not be `O(n)` - without
633−semantics of pure Nock, *ever*.
686+violating the precisely defined semantics of pure Nock, *ever*.
634
687
635−It's easy to assume that jets represent an architectural coupling between Hoon
688+It's easy to assume that jets represent an architectural coupling
636−language semantics and Nock interpreter internals. Indeed such a coupling would
689+between Hoon language semantics and Nock interpreter internals.
690+Indeed such a coupling would be wholly wrongtious and un-Urbit.
637−be wholly wrongtious and un-Urbit. But the jet system is not Hoon-specific. It
691+But the jet system is not Hoon-specific. It is specific to nock
638−is specific to nock runtime systems that use a design pattern we call a `core`.
692+runtime systems that use a design pattern we call a `core`.
639
693
640#### u3j: core structure
694#### u3j: core structure
641
695
642−A core is no more than a cell `[code data]`, in which a `code` is either a Nock
696+A core is no more than a cell `[code data]`, in which a `code` is
643−formula or a cell of `code`s, and `data` is anything. In a proper core, the
697+either a Nock formula or a cell of `code`s, and `data` is anything.
644−subject each formula expects is the core itself.
698+In a proper core, the subject each formula expects is the core
645−
699+itself.
700+
646−Except for the arbitrary decision to make a core `[code data]`, (or as we
701+Except for the arbitrary decision to make a core `[code data]`,
647−sometimes say, `[battery payload]`), instead of `[data code]`, any high-level
702+(or as we sometimes say, `[battery payload]`), instead of `[data
648−language transforming itself to Nock would use this design.
703+code]`, any high-level language transforming itself to Nock would
649−
704+use this design.
705+
650−So jets are in fact fully general. Broadly speaking, the jet system works by
706+So jets are in fact fully general. Broadly speaking, the jet
651−matching a C *driver* to a battery. When the battery is invoked with Nock
707+system works by matching a C *driver* to a battery. When the
708+battery is invoked with Nock operator `9`, it must be found in
652−operator `9`, it must be found in associative memory and linked to its driver.
709+associative memory and linked to its driver. Then we link the
653−Then we link the formula axis of the operation (`a` in `[9 a b]`) to a specific
710+formula axis of the operation (`a` in `[9 a b]`) to a specific
654function in the driver.
711function in the driver.
655
712
656−To validate this jet binding, we need to know two things. One, we need to know
713+To validate this jet binding, we need to know two things. One,
657−the C function actually is a perfect semantic match for the Nock formula. This
714+we need to know the C function actually is a perfect semantic
715+match for the Nock formula. This can be developed with driver
658−can be developed with driver test flags, which work, and locked down with a
716+test flags, which work, and locked down with a secure formula
659−secure formula hash in the driver, which we haven't bothered with just yet. (You
717+hash in the driver, which we haven't bothered with just yet.
660−could also try to develop a formal method for verifying that C functions and
718+(You could also try to develop a formal method for verifying
719+that C functions and Nock formulas are equivalent, but this is
661−Nock formulas are equivalent, but this is a research problem for the future.)
720+a research problem for the future.)
662−
721+
663−Two, we need to validate that the payload is appropriate for the battery. We
722+Two, we need to validate that the payload is appropriate for the
664−should note that jets are a Nock feature and have no reference to Hoon. A driver
723+battery. We should note that jets are a Nock feature and have no
724+reference to Hoon. A driver which relies on the Hoon type system
665−which relies on the Hoon type system to only pair it with valid payloads is a
725+to only pair it with valid payloads is a broken driver, and
666−broken driver, and breaks the Nock compliance of the system as a whole. So
726+breaks the Nock compliance of the system as a whole. So don't.
667−don't.
727+
668−
669−Now, a casual observer might look at `[battery payload]` and expect the simplest
728+Now, a casual observer might look at `[battery payload]` and
670−case of it to be `[formula subject]`. That is: to execute a simple core whose
729+expect the simplest case of it to be `[formula subject]`. That
671−battery is a single formula, we compute
730+is: to execute a simple core whose battery is a single formula,
731+we compute
672
732
673 nock(+.a -.a)
733 nock(+.a -.a)
674
734
675−Then, naturally, when we go from Hoon or a high-level language containing
735+Then, naturally, when we go from Hoon or a high-level language
676−functions down to Nock, `[function arguments]` turns into `[formula subject]`.
736+containing functions down to Nock, `[function arguments]` turns
677−This seems like an obvious design, and we mention it only because it is
737+into `[formula subject]`. This seems like an obvious design, and
678−*completely wrong*.
738+we mention it only because it is *completely wrong*.
679
739
680Rather, to execute a one-armed core like the above, we run
740Rather, to execute a one-armed core like the above, we run
681
741
682 nock(a -.a)
742 nock(a -.a)
683
743
684−and the normal structure of a `gate`, which is simply Urbitese for "function,"
744+and the normal structure of a `gate`, which is simply Urbitese
685−is:
745+for "function," is:
686
746
687 [formula [sample context]]
747 [formula [sample context]]
688
748
689−where `sample` is Urbitese for "arguments" - and `context`, any Lisper will at
749+where `sample` is Urbitese for "arguments" - and `context`, any
690−once recognize, is Urbitese for "environment."
750+Lisper will at once recognize, is Urbitese for "environment."
691
751
692−To `slam` or call the gate, we simply replace the default sample with the
752+To `slam` or call the gate, we simply replace the default sample
693−caller's data, then nock the formula on the entire gate.
753+with the caller's data, then nock the formula on the entire gate.
694
754
695−What's in the context? Unlike in most dynamic languages, it is not some secret
755+What's in the context? Unlike in most dynamic languages, it is
696−system-level bag of tricks. Almost always it is another core. This onion
756+not some secret system-level bag of tricks. Almost always it is
697−continues until at the bottom, there is an atomic constant, conventionally is
757+another core. This onion continues until at the bottom, there is
698−the kernel version number.
758+an atomic constant, conventionally is the kernel version number.
699
759
700Thus a (highly desirable) `static` core is one of the form
760Thus a (highly desirable) `static` core is one of the form
701
761
702 [battery constant]
762 [battery constant]
703 [battery static-core]
763 [battery static-core]
704
764
705−ie, a solid stack of nested libraries without any dynamic data. The typical gate
765+ie, a solid stack of nested libraries without any dynamic data.
706−will thus be, for example,
766+The typical gate will thus be, for example,
707
767
708 [formula [sample [battery battery battery constant]]]
768 [formula [sample [battery battery battery constant]]]
709
769
710−but we would be most foolish to restrict the jet mechanism to cores of this
770+but we would be most foolish to restrict the jet mechanism to
711−particular structure. We cannot constrain a payload to be
771+cores of this particular structure. We cannot constrain a
712−`[sample static-core]`, or even `[sample core]`. Any such constraint would not
772+payload to be `[sample static-core]`, or even `[sample core]`.
773+Any such constraint would not be rich enough to handle Hoon,
713−be rich enough to handle Hoon, let alone other languages.
774+let alone other languages.
714
775
715#### u3j: jet state
776#### u3j: jet state
716
777
717−There are two fundamental rules of computer science: (1) every system is best
778+There are two fundamental rules of computer science: (1) every
718−understood through its state; (2) less state is better than more state. Sadly, a
779+system is best understood through its state; (2) less state is
780+better than more state. Sadly, a pier has three different jet
719−pier has three different jet state systems: `cold`, `warm` and `hot`. It needs
781+state systems: `cold`, `warm` and `hot`. It needs all of them.
720−all of them.
721
782
722−Hot state is associated with this particular Unix process. The persistent pier
783+Hot state is associated with this particular Unix process. The
723−is portable not just between process and process, but machine and machine or OS
784+persistent pier is portable not just between process and process,
785+but machine and machine or OS and OS. The set of jets loaded
724−and OS. The set of jets loaded into a pier may itself change (in theory, though
786+into a pier may itself change (in theory, though not in the
725−not in the present implementation) during the lifetime of the process. Hot state
787+present implementation) during the lifetime of the process. Hot
726−is a pure C data structure.
788+state is a pure C data structure.
727
789
728−Cold state is associated with the logical execution history of the pier. It
790+Cold state is associated with the logical execution history of
729−consists entirely of nouns and ignores restarts.
791+the pier. It consists entirely of nouns and ignores restarts.
730
792
731−Warm state contains all dependencies between cold and hot state. It consists of
793+Warm state contains all dependencies between cold and hot
732−C structures allocated on the loom.
794+state. It consists of C structures allocated on the loom.
733
795
734−Warm state is purely a function of cold and hot states, and we can wipe and
796+Warm state is purely a function of cold and hot states, and
735−regenerate it at any time. On any restart where the hot state might have
797+we can wipe and regenerate it at any time. On any restart where
736−changed, we clear the warm state with `u3j_ream()`.
798+the hot state might have changed, we clear the warm state
799+with `u3j_ream()`.
737
800
738−There is only one hot state, the global jet dashboard `u3j_Dash` or `u3D` for
801+There is only one hot state, the global jet dashboard
802+`u3j_Dash` or `u3D` for short. In the present implementation,
739−short. In the present implementation, u3D is a static structure not modified at
803+u3D is a static structure not modified at runtime, except for
740−runtime, except for numbering itself on process initialization. This structure -
804+numbering itself on process initialization. This structure -
741−which embeds function pointers to all the jets - is defined in `j/tree.c`. The
805+which embeds function pointers to all the jets - is defined
742−data structures:
806+in `j/tree.c`. The data structures:
743
807
744 /* u3j_harm: driver arm.
808 /* u3j_harm: driver arm.
745 */
809 */
@@ -770,16 +834,16 @@ data structures:
770 u3j_core* ray_u; // dynamic driver array
834 u3j_core* ray_u; // dynamic driver array
771 } u3j_dash;
835 } u3j_dash;
772
836
773−Warm and cold state is *per road*. In other words, as we nest roads, we also
837+Warm and cold state is *per road*. In other words, as we nest
774−nest jet state. The jet state in the road is:
838+roads, we also nest jet state. The jet state in the road is:
775
839
776 struct { // jet dashboard
840 struct { // jet dashboard
777 u3p(u3h_root) har_p; // warm state
841 u3p(u3h_root) har_p; // warm state
778 u3_noun das; // cold state
842 u3_noun das; // cold state
779 } jed;
843 } jed;
780
844
781−In case you understand Hoon, `das` (cold state) is a `++dash`, and `har_p` (warm
845+In case you understand Hoon, `das` (cold state) is a `++dash`,
782−state) is a map from battery to `++calx`:
846+and `har_p` (warm state) is a map from battery to `++calx`:
783
847
784 ++ bane ,@tas :: battery name
848 ++ bane ,@tas :: battery name
785 ++ bash ,@uvH :: label hash
849 ++ bash ,@uvH :: label hash
@@ -799,288 +863,323 @@ state) is a map from battery to `++calx`:
799 ++ corp (each core batt) :: parent or static
863 ++ corp (each core batt) :: parent or static
800 ++ dash (map bash clog) :: jet system
864 ++ dash (map bash clog) :: jet system
801
865
802−The driver index `jax` in a `++calx` is an index into `ray_u` in the dashboard -
866+The driver index `jax` in a `++calx` is an index into `ray_u` in the
803−ie, a pointer into hot state. This is why the warm state has to be reset when we
867+dashboard - ie, a pointer into hot state. This is why the warm
804−reload the pier in a new process.
868+state has to be reset when we reload the pier in a new process.
805
869
806−Why is jet state nested? Nock of course is a functional system, so as we compute
870+Why is jet state nested? Nock of course is a functional system,
807−we don't explicitly create state. Jet state is an exception to this principle
871+so as we compute we don't explicitly create state. Jet state is
872+an exception to this principle (which works only because it can't
808−(which works only because it can't be semantically detected from Nock/Hoon) -
873+be semantically detected from Nock/Hoon) - but it can't violate
809−but it can't violate the fundamental rules of the allocation system.
874+the fundamental rules of the allocation system.
810
875
811−For instance, when we're on an inner road, we can't allocate on an outer road,
876+For instance, when we're on an inner road, we can't allocate on
812−or point from an outer road to an inner. So if we learn something - like a
877+an outer road, or point from an outer road to an inner. So if we
878+learn something - like a mapping from battery to jet - in the
813−mapping from battery to jet - in the inner road, we have to keep it in the inner
879+inner road, we have to keep it in the inner road.
814−road.
815
880
816−Mitigating this problem, when we leave an inner road (with `u3m_love()`), we
881+Mitigating this problem, when we leave an inner road (with
817−call `u3j_reap()` to promote jet information in the dying road. Reaping promotes
882+`u3m_love()`), we call `u3j_reap()` to promote jet information in
883+the dying road. Reaping promotes anything we've learned about
818−anything we've learned about any battery that either (a) already existed in the
884+any battery that either (a) already existed in the outer road, or
819−outer road, or (b) is being saved to the outer road.
885+(b) is being saved to the outer road.
820
886
821#### u3j: jet binding
887#### u3j: jet binding
822
888
823−Jet binding starts with a `%fast` hint. (In Hoon, this is produced by the runes
889+Jet binding starts with a `%fast` hint. (In Hoon, this is
890+produced by the runes `~%`, for the general case, or `~/`
824−`~%`, for the general case, or `~/` for simple functions.) To bind a jet,
891+for simple functions.) To bind a jet, execute a formula of the
825−execute a formula of the form:
892+form:
826
893
827 [10 [%fast clue-formula] core-formula]
894 [10 [%fast clue-formula] core-formula]
828
895
829−`core-formula` assembles the core to be jet-propelled. `clue-formula` produces
896+`core-formula` assembles the core to be jet-propelled.
897+`clue-formula` produces the hint information, or `++clue`
830−the hint information, or `++clue` above, which we want to annotate it with.
898+above, which we want to annotate it with.
831
899
832A clue is a triple of name, parent, and hooks:
900A clue is a triple of name, parent, and hooks:
833
901
834 ++ clue (trel chum nock (list (pair term nock)))
902 ++ clue (trel chum nock (list (pair term nock)))
835
903
836−The name, or `++chum`, has a bunch of historical structure which we don't need
904+The name, or `++chum`, has a bunch of historical structure which
837−(cleaning these things up is tricky), but just gets flattened into a term.
905+we don't need (cleaning these things up is tricky), but just gets
906+flattened into a term.
838
907
839−The parent axis is a nock formula, but always reduces to a simple axis, which is
908+The parent axis is a nock formula, but always reduces to a simple
840−the address of this core's *parent*. Consider again an ordinary gate
909+axis, which is the address of this core's *parent*. Consider
910+again an ordinary gate
841
911
842 [formula [sample context]]
912 [formula [sample context]]
843
913
844−Typically the `context` is itself a library core, which itself has a jet
914+Typically the `context` is itself a library core, which itself
845−binding. If so, the parent axis of this gate is `7`.
915+has a jet binding. If so, the parent axis of this gate is `7`.
846
916
847−If the parent is already bound - and the parent *must* be already bound, in this
917+If the parent is already bound - and the parent *must* be already
848−road or a road containing it - we can hook this core bottom-up into a tree
918+bound, in this road or a road containing it - we can hook this core
919+bottom-up into a tree hierarchy. Normally the child core is
849−hierarchy. Normally the child core is produced by an arm of the parent core, so
920+produced by an arm of the parent core, so this is not a problem -
850−this is not a problem - we wouldn't have the child if we hadn't already made the
921+we wouldn't have the child if we hadn't already made the parent.
851−parent.
852
922
853−The clue also contains a list of *hooks*, named nock formulas on the core.
923+The clue also contains a list of *hooks*, named nock formulas on
854−Usually these are arms, but they need not be. The point is that we often want to
924+the core. Usually these are arms, but they need not be. The
925+point is that we often want to call a core from C, in a situation
855−call a core from C, in a situation where we have no type or other source
926+where we have no type or other source information. A common case
856−information. A common case of this is a complex system in which we're mixing
927+of this is a complex system in which we're mixing functions which
857−functions which are jet-propelled with functions that aren't.
928+are jet-propelled with functions that aren't.
858
929
859−In any case, all the information in the `%fast` hint goes to `u3j_mine()`, which
930+In any case, all the information in the `%fast` hint goes to
860−registers the battery in cold state (`das` in `jed` in `u3R`), then warm state
931+`u3j_mine()`, which registers the battery in cold state (`das` in
861−(`har_p` in `jed`).
932+`jed` in `u3R`), then warm state (`har_p` in `jed`).
862
933
863−It's essential to understand that the `%fast` hint has to be, well, fast -
934+It's essential to understand that the `%fast` hint has to be,
864−because we apply it whenever we build a core. For instance, if the core is a
935+well, fast - because we apply it whenever we build a core. For
936+instance, if the core is a Hoon gate - a function - we will call
865−Hoon gate - a function - we will call `u3j_mine` every time the function is
937+`u3j_mine` every time the function is called.
866−called.
867
938
868### u3j: the cold jet dashboard
939### u3j: the cold jet dashboard
869
940
870−For even more fun, the jet tree is not actually a tree of batteries. It's a tree
941+For even more fun, the jet tree is not actually a tree of
942+batteries. It's a tree of battery *labels*, where a label is
871−of battery *labels*, where a label is an \[axis term\] path from the root of the
943+an [axis term] path from the root of the tree. (At the root,
872−tree. (At the root, if the core pattern is always followed properly, is a core
944+if the core pattern is always followed properly, is a core whose
873−whose payload is an atomic constant, conventionally the Hoon version.)
945+payload is an atomic constant, conventionally the Hoon version.)
874
946
875−Under each of these labels, it's normal to have an arbitrary number of different
947+Under each of these labels, it's normal to have an arbitrary
876−Nock batteries (not just multiple copies of the same noun, a situation we *do*
948+number of different Nock batteries (not just multiple copies
949+of the same noun, a situation we *do* strive to avoid). For
877−strive to avoid). For instance, one might be compiled with debugging hints, one
950+instance, one might be compiled with debugging hints, one not.
878−not.
879
951
880−We might even have changed the semantics of the battery without changing the
952+We might even have changed the semantics of the battery without
881−label - so long as those semantics don't invalidate any attached driver.
953+changing the label - so long as those semantics don't invalidate
954+any attached driver.
882
955
883−For instance, it's normal to have two equivalent Nock batteries at the same time
956+For instance, it's normal to have two equivalent Nock batteries
884−in one pier: one battery compiled with debugging hints, one not.
957+at the same time in one pier: one battery compiled with
958+debugging hints, one not.
885
959
886−Rather, the jet tree is a semantic hierarchy. The root of the hierarchy is a
960+Rather, the jet tree is a semantic hierarchy. The root of the
887−constant, by convention the Hoon kernel version because any normal jet-propelled
961+hierarchy is a constant, by convention the Hoon kernel version
962+because any normal jet-propelled core has, at the bottom of its
888−core has, at the bottom of its onion of libraries, the standard kernel. Thus if
963+onion of libraries, the standard kernel. Thus if the core is
889−the core is
890
964
891 [foo-battery [bar-battery [moo-battery 164]]]
965 [foo-battery [bar-battery [moo-battery 164]]]
892
966
893−we can reverse the nesting to construct a hierarchical core path. The static
967+we can reverse the nesting to construct a hierarchical core
894−core
968+path. The static core
895
969
896 164/moo/bar/foo
970 164/moo/bar/foo
897
971
898−extends the static core `164/moo/bar` by wrapping the `foo` battery (ie, in
972+extends the static core `164/moo/bar` by wrapping the `foo`
899−Hoon, `|%`) around it. With the core above, you can compute `foo` stuff, `bar`
973+battery (ie, in Hoon, `|%`) around it. With the core above,
974+you can compute `foo` stuff, `bar` stuff, and `moo` stuff.
900−stuff, and `moo` stuff. Rocket science, not.
975+Rocket science, not.
901−
976+
902−Not all cores are static, of course - they may contain live data, like the
977+Not all cores are static, of course - they may contain live data,
903−sample in a gate (ie, argument to a function). Once again, it's important to
978+like the sample in a gate (ie, argument to a function). Once
904−remember that we track jet bindings not by the core, which may not be static,
979+again, it's important to remember that we track jet bindings not
980+by the core, which may not be static, but by the battery, which
905−but by the battery, which is always static.
981+is always static.
906−
982+
907−(And if you're wondering how we can use a deep noun like a Nock formula or
983+(And if you're wondering how we can use a deep noun like a Nock
908−battery as a key in a key-value table, remember `mug_w`, the lazily computed
984+formula or battery as a key in a key-value table, remember
909−short hash, in all boxed nouns.)
985+`mug_w`, the lazily computed short hash, in all boxed nouns.)
910−
986+
911−In any case, `das`, the dashboard, is a map from `bash` to jet location record
987+In any case, `das`, the dashboard, is a map from `bash` to jet
912−(`++clog`). A `clog` in turn contains two kinds of information: the `++cope`, or
988+location record (`++clog`). A `clog` in turn contains two kinds
989+of information: the `++cope`, or per-location noun; and a map of
913−per-location noun; and a map of batteries to a per-battery `++club`.
990+batteries to a per-battery `++club`.
914−
991+
915−The `cope` is a triple of `++bane` (battery name, right now just a `term`);
992+The `cope` is a triple of `++bane` (battery name, right now just
916−`++axis`, the axis, within *this* core, of the parent; and `(each bash noun)`,
993+a `term`); `++axis`, the axis, within *this* core, of the parent;
917−which is either `[0 bash]` if the parent is another core, or `[1 noun]`, for the
994+and `(each bash noun)`, which is either `[0 bash]` if the parent
995+is another core, or `[1 noun]`, for the constant noun (like
918−constant noun (like `164`) if there is no parent core.
996+`164`) if there is no parent core.
919−
997+
920−A `bash` is just the noun hash (`++sham`) of a `cope`, which uniquely expresses
998+A `bash` is just the noun hash (`++sham`) of a `cope`, which
921−the battery's hierarchical location without depending on the actual formulas.
999+uniquely expresses the battery's hierarchical location without
922−
1000+depending on the actual formulas.
1001+
923−The `club` contains a `++corp`, which we use to actually validate the core.
1002+The `club` contains a `++corp`, which we use to actually validate
924−Obviously jet execution has to be perfectly compatible with Nock. We search on
1003+the core. Obviously jet execution has to be perfectly compatible
1004+with Nock. We search on the battery, but getting the battery
925−the battery, but getting the battery right is not enough - a typical battery is
1005+right is not enough - a typical battery is dependent on its
926−dependent on its context. For example, your jet-propelled library function is
1006+context. For example, your jet-propelled library function is
927−very likely to call `++dec` or other advanced kernel technology. If you've
1007+very likely to call `++dec` or other advanced kernel technology.
928−replaced the kernel in your context with something else, we need to detect this
1008+If you've replaced the kernel in your context with something
929−and not run the jet.
1009+else, we need to detect this and not run the jet.
930−
1010+
931−There are two cases for a jet-propelled core - either the entire core is a
1011+There are two cases for a jet-propelled core - either the entire
932−static constant, or it isn't. Hence the definition of `corp`:
1012+core is a static constant, or it isn't. Hence the definition
1013+of `corp`:
933
1014
934 ++ corp (each core batt) :: parent or static
1015 ++ corp (each core batt) :: parent or static
935
1016
936−Ie, a `corp` is `[0 core]` or `[1 batt]`. If it's static - meaning that the jet
1017+Ie, a `corp` is `[0 core]` or `[1 batt]`. If it's static -
937−only works with one specific core, ie, the parent axis of each location in the
1018+meaning that the jet only works with one specific core, ie, the
1019+parent axis of each location in the hierarchy is `3` - we can
938−hierarchy is `3` - we can validate with a single comparison. Otherwise, we have
1020+validate with a single comparison. Otherwise, we have to recurse
939−to recurse upward by checking the parent.
1021+upward by checking the parent.
940
1022
941−Note that there is at present no way to force a jet to depend on static *data*.
1023+Note that there is at present no way to force a jet to depend on
1024+static *data*.
942
1025
943### u3j: the warm jet dashboard
1026### u3j: the warm jet dashboard
944
1027
945−We don't use the cold state to match jets as we call them. We use the cold state
1028+We don't use the cold state to match jets as we call them. We
946−to register jets as we find them, and also to rebuild the warm state after the
1029+use the cold state to register jets as we find them, and also to
947−hot state is reset.
1030+rebuild the warm state after the hot state is reset.
948
1031
949−What we actually use at runtime is the warm state, `jed->har_p`, which is a
1032+What we actually use at runtime is the warm state, `jed->har_p`,
950−`u3h` (built-in hashtable), allocated on the loom, from battery to `++calx`.
1033+which is a `u3h` (built-in hashtable), allocated on the loom,
1034+from battery to `++calx`.
951
1035
952−A `calx` is a triple of a `++calf`, a `[bash cope]` cell, and a `club`. The
1036+A `calx` is a triple of a `++calf`, a `[bash cope]` cell, and a
953−latter two are all straight from cold state.
1037+`club`. The latter two are all straight from cold state.
954
1038
955−The `calf` contains warm data dependent on hot state. It's a quadruple: of
1039+The `calf` contains warm data dependent on hot state. It's a
956−`jax`, the hot driver index (in `ray_u` in `u3j_dash`); `hap`, a table from arm
1040+quadruple: of `jax`, the hot driver index (in `ray_u` in
1041+`u3j_dash`); `hap`, a table from arm axis (ie, the axis of each
957−axis (ie, the axis of each formula within the battery) to driver arm index (into
1042+formula within the battery) to driver arm index (into `arm_u` in
958−`arm_u` in `u3j_core`); `lab`, the complete label path; and `jit`, any other
1043+`u3j_core`); `lab`, the complete label path; and `jit`, any
959−dynamic data that may speed up execution.
1044+other dynamic data that may speed up execution.
960
1045
961−We construct `hap`, when we create the calx, by iterating through the arms
1046+We construct `hap`, when we create the calx, by iterating through
962−registered in the `u3j_core`. Note the way a `u3j_harm` declares itself, with
1047+the arms registered in the `u3j_core`. Note the way a `u3j_harm`
963−the string `fcs_c` which can contain either an axis or a name. Most jetted cores
1048+declares itself, with the string `fcs_c` which can contain either
1049+an axis or a name. Most jetted cores are of course gates, which
964−are of course gates, which have one formula at one axis within the core: `fcs_c`
1050+have one formula at one axis within the core: `fcs_c` is `".3"`.
965−is `".3"`.
966
1051
967−But we do often have fast cores with more complex arm structure, and it would be
1052+But we do often have fast cores with more complex arm structure,
968−sad to have to manage their axes by hand. To use an `fcs_c` with a named arm,
1053+and it would be sad to have to manage their axes by hand. To use
1054+an `fcs_c` with a named arm, it's sufficient to make sure the
969−it's sufficient to make sure the name is bound to a formula `[0 axis]` in the
1055+name is bound to a formula `[0 axis]` in the hook table.
970−hook table.
971
1056
972−`jit`, as its name suggests, is a stub where any sort of optimization data
1057+`jit`, as its name suggests, is a stub where any sort of
973−computed on battery registration might go. To use it, fill in the `_cj_jit()`
1058+optimization data computed on battery registration might go. To
974−function.
1059+use it, fill in the `_cj_jit()` function.
975
1060
976### u3j: the hot dashboard
1061### u3j: the hot dashboard
977
1062
978−Now it should be easy to see how we actually invoke jets. Every time we run a
1063+Now it should be easy to see how we actually invoke jets. Every
979−nock `9` instruction (pretty often, obviously), we have a core and an axis. We
1064+time we run a nock `9` instruction (pretty often, obviously), we
1065+have a core and an axis. We pass these to `u3j_kick()`, which
980−pass these to `u3j_kick()`, which will try to execute them.
1066+will try to execute them.
981
1067
982−Because nouns with a reference count of 1 are precious, `u3j_kick()` has a
1068+Because nouns with a reference count of 1 are precious,
983−tricky reference control definition. It reserves the right to return `u3_none`
1069+`u3j_kick()` has a tricky reference control definition. It
1070+reserves the right to return `u3_none` in the case where there is
984−in the case where there is no driver, or the driver does not apply for this
1071+no driver, or the driver does not apply for this case; in this
985−case; in this case, it retains argument `cor`. If it succeeds, though, it
1072+case, it retains argument `cor`. If it succeeds, though, it
986transfers `cor`.
1073transfers `cor`.
987
1074
988−`u3j_kick()` searches for the battery (always the head of the core, of course)
1075+`u3j_kick()` searches for the battery (always the head of the
1076+core, of course) in the hot dashboard. If the battery is
989−in the hot dashboard. If the battery is registered, it searches for the axis in
1077+registered, it searches for the axis in `hap` in the `calx`.
990−`hap` in the `calx`. If it exists, the core matches a driver and the driver jets
1078+If it exists, the core matches a driver and the driver jets this
991−this arm. If not, we return `u3_none`.
1079+arm. If not, we return `u3_none`.
992
1080
993−Otherwise, we call `fun_f` in our `u3j_harm`. This obeys the same protocol as
1081+Otherwise, we call `fun_f` in our `u3j_harm`. This obeys the
1082+same protocol as `u3j_kick()`; it can refuse to function by
994−`u3j_kick()`; it can refuse to function by returning `u3_none`, or consume the
1083+returning `u3_none`, or consume the noun.
995−noun.
996
1084
997−Besides the actual function pointer `fun_f`, we have some flags in the
1085+Besides the actual function pointer `fun_f`, we have some flags
998−`u3j_harm` which tell us how to call the arm function.
1086+in the `u3j_harm` which tell us how to call the arm function.
999
1087
1000−If `ice` is yes (`&`, `0`), the jet is known to be perfect and we can just trust
1088+If `ice` is yes (`&`, `0`), the jet is known to be perfect and we
1001−the product of `fun_f`. Otherwise, we need to run *both* the Nock arm and
1089+can just trust the product of `fun_f`. Otherwise, we need to run
1002−`fun_f`, and compare their results.
1090+*both* the Nock arm and `fun_f`, and compare their results.
1003
1091
1004−(Note that while executing the C side of this test, we have to set `ice` to yes;
1092+(Note that while executing the C side of this test, we have to
1005−on the Nock side, we have to set `liv` to no. Otherwise, many non-exponential
1093+set `ice` to yes; on the Nock side, we have to set `liv` to no.
1094+Otherwise, many non-exponential functions become exponential.
1006−functions become exponential. When auto-testing jets in this way, the principle
1095+When auto-testing jets in this way, the principle is that the
1007−is that the test is on the outermost layer of recursion.)
1096+test is on the outermost layer of recursion.)
1008
1097
1009−(Note also that anyone who multi-threads this execution environment has a slight
1098+(Note also that anyone who multi-threads this execution
1010−locking problem with these flags if arm testing is multi-threaded.)
1099+environment has a slight locking problem with these flags if arm
1100+testing is multi-threaded.)
1011
1101
1012−If `tot` is yes, (`&`, `0`), the arm function is *total* and has to return
1102+If `tot` is yes, (`&`, `0`), the arm function is *total* and has
1013−properly (though it can still return \*u3\_none*). Otherwise, it is*partial\*
1103+to return properly (though it can still return *u3_none*).
1104+Otherwise, it is *partial* and can `u3_cm_bail()` out with
1014−and can `u3_cm_bail()` out with c3\_\_punt. This feature has a cost: the jet
1105+c3__punt. This feature has a cost: the jet runs in a subroad.
1015−runs in a subroad.
1016
1106
1017Finally, if `liv` is no (`|`, 1), the jet is off and doesn't run.
1107Finally, if `liv` is no (`|`, 1), the jet is off and doesn't run.
1018
1108
1019−It should be easy to see how the tree of cores gets declared - precisely, in
1109+It should be easy to see how the tree of cores gets declared -
1020−`j/dash.c`. We declare the hierarchy as a tree of `u3j_core` structures, each of
1110+precisely, in `j/dash.c`. We declare the hierarchy as a tree
1111+of `u3j_core` structures, each of which comes with a static list
1021−which comes with a static list of arms `arm_u` and sub-cores `dev_u`.
1112+of arms `arm_u` and sub-cores `dev_u`.
1022
1113
1023−In `u3j_boot()`, we traverse the hierarchy, fill in parent pointers `par_u`, and
1114+In `u3j_boot()`, we traverse the hierarchy, fill in parent
1115+pointers `par_u`, and enumerate all `u3j_core` structures
1024−enumerate all `u3j_core` structures into a single flat array `u3j_dash.ray_u`.
1116+into a single flat array `u3j_dash.ray_u`. Our hot state
1025−Our hot state then appears ready for action.
1117+then appears ready for action.
1026
1118
1027### u3j: jet functions
1119### u3j: jet functions
1028
1120
1029−At present, all drivers are compiled statically into `u3`. This is not a
1121+At present, all drivers are compiled statically into `u3`. This is
1030−long-term permanent solution or anything. However, it will always be the case
1122+not a long-term permanent solution or anything. However, it will
1031−with a certain amount of core functionality.
1123+always be the case with a certain amount of core functionality.
1032−
1124+
1033−For instance, there are some jet functions that we need to call as part of
1125+For instance, there are some jet functions that we need to call
1034−loading the Arvo kernel - like `++cue` to unpack a noun from an atom. And
1126+as part of loading the Arvo kernel - like `++cue` to unpack a
1035−obviously it makes sense, when jets are significant enough to compile into `u3`,
1127+noun from an atom. And obviously it makes sense, when jets are
1128+significant enough to compile into `u3`, to export their symbols
1036−to export their symbols in headers and the linker.
1129+in headers and the linker.
1037−
1130+
1038−There are three interface prefixes for standard jet functions: `u3k`, `u3q`, and
1131+There are three interface prefixes for standard jet functions:
1039−`u3w`. All jets have `u3w` interfaces; most have `u3q`; some have `u3k`. Of
1132+`u3k`, `u3q`, and `u3w`. All jets have `u3w` interfaces; most
1040−course the actual logic is shared.
1133+have `u3q`; some have `u3k`. Of course the actual logic is
1041−
1134+shared.
1135+
1042−`u3w` interfaces use the same protocol as `fun_f` above: the caller passes the
1136+`u3w` interfaces use the same protocol as `fun_f` above: the
1043−entire core, which is retained if the function returns `u3_none`, transferred
1137+caller passes the entire core, which is retained if the function
1138+returns `u3_none`, transferred otherwise. Why? Again, use
1044−otherwise. Why? Again, use counts of 1 are special and precious for performance
1139+counts of 1 are special and precious for performance hackers.
1045−hackers.
1140+
1046−
1047−`u3q` interfaces break the core into C arguments, *retain* noun arguments, and
1141+`u3q` interfaces break the core into C arguments, *retain* noun
1048−*transfer* noun returns. `u3k` interfaces are the same, except with more use of
1142+arguments, and *transfer* noun returns. `u3k` interfaces are the
1143+same, except with more use of `u3_none` and other simple C
1049−`u3_none` and other simple C variations on the Hoon original, but *transfer*
1144+variations on the Hoon original, but *transfer* both arguments
1050−both arguments and returns. Generally, `u3k` are most convenient for new code.
1145+and returns. Generally, `u3k` are most convenient for new code.
1051−
1146+
1052−Following `u3k/q/w` is `[a-f]`, corresponding to the 6 logical tiers of the
1147+Following `u3k/q/w` is `[a-f]`, corresponding to the 6 logical
1053−kernel, or `g` for user-level jets. Another letter is added for functions within
1148+tiers of the kernel, or `g` for user-level jets. Another letter
1149+is added for functions within subcores. The filename, under
1054−subcores. The filename, under `j/`, follows the tier and the function name.
1150+`j/`, follows the tier and the function name.
1055−
1151+
1056−For instance, `++add` is `u3wa_add(cor)`, `u3qa_add(a, b)`, or `u3ka_add(a, b)`,
1152+For instance, `++add` is `u3wa_add(cor)`, `u3qa_add(a, b)`, or
1153+`u3ka_add(a, b)`, in `j/a/add.c`. `++get` in `++by` is
1057−in `j/a/add.c`. `++get` in `++by` is `u3wdb_get(cor)`, `u3kdb_get(a, b)`, etc,
1154+`u3wdb_get(cor)`, `u3kdb_get(a, b)`, etc, in `j/d/by_get.c`.
1058−in `j/d/by_get.c`.
1155+
1059−
1060−For historical reasons, all internal jet code in `j/[a-f]` *retains* noun
1156+For historical reasons, all internal jet code in `j/[a-f]`
1061−arguments, and *transfers* noun results. Please do not do this in new `g` jets!
1157+*retains* noun arguments, and *transfers* noun results. Please
1158+do not do this in new `g` jets! The new standard protocol is to
1062−The new standard protocol is to transfer both arguments and results.
1159+transfer both arguments and results.
1063
1160
1064### u3a: allocation functions
1161### u3a: allocation functions
1065
1162
1066−`u3a` allocates on the current road (u3R). Its internal structures are
1163+`u3a` allocates on the current road (u3R). Its internal
1067−uninteresting and typical of a naive allocator.
1164+structures are uninteresting and typical of a naive allocator.
1068
1165
1069−The two most-used `u3a` functions are `u3a_gain()` to add a reference count, and
1166+The two most-used `u3a` functions are `u3a_gain()` to add a
1070−`u3a_lose()` to release one (and free the noun, if the use count is zero). For
1167+reference count, and `u3a_lose()` to release one (and free the
1168+noun, if the use count is zero). For convenience, `u3a_gain()`
1071−convenience, `u3a_gain()` returns its argument. The pair are generally
1169+returns its argument. The pair are generally abbreviated with
1072−abbreviated with the macros `u3k()` and `u3z()` respectively.
1170+the macros `u3k()` and `u3z()` respectively.
1073
1171
1074−Normally we create nouns through `u3i` functions, and don't call the `u3a`
1172+Normally we create nouns through `u3i` functions, and don't call
1075−allocators directly. But if you do:
1173+the `u3a` allocators directly. But if you do:
1076
1174
1077−One, there are *two* sets of allocators: the word-aligned allocators and the
1175+One, there are *two* sets of allocators: the word-aligned
1176+allocators and the fully-aligned (ie, malloc compatible)
1078−fully-aligned (ie, malloc compatible) allocators. For instance, on a typical OS
1177+allocators. For instance, on a typical OS X setup, malloc
1079−X setup, malloc produces 16-byte aligned results - needed for some SSE
1178+produces 16-byte aligned results - needed for some SSE
1080instructions.
1179instructions.
1081
1180
1082−These allocators are *not compatible*. For 32-bit alignment as used in nouns,
1181+These allocators are *not compatible*. For 32-bit alignment
1083−call
1182+as used in nouns, call
1084
1183
1085 /* u3a_walloc(): allocate storage measured in words.
1184 /* u3a_walloc(): allocate storage measured in words.
1086 */
1185 */
@@ -1124,41 +1223,48 @@ For full alignment, call:
1124 void
1223 void
1125 u3a_free2(void* tox_v, size_t siz_i);
1224 u3a_free2(void* tox_v, size_t siz_i);
1126
1225
1127−There are also a set of special-purpose allocators for building atoms. When
1226+There are also a set of special-purpose allocators for building
1128−building atoms, please remember that it's incorrect to have a high 0 word - the
1227+atoms. When building atoms, please remember that it's incorrect
1228+to have a high 0 word - the word length in the atom structure
1129−word length in the atom structure must be strictly correct.
1229+must be strictly correct.
1130−
1230+
1131−Of course, we don't always know how large our atom will be. Therefore, the
1231+Of course, we don't always know how large our atom will be.
1132−standard way of building large atoms is to allocate a block of raw space with
1232+Therefore, the standard way of building large atoms is to
1233+allocate a block of raw space with `u3a_slab()`, then chop off
1133−`u3a_slab()`, then chop off the end with `u3a_malt()` (which does the measuring
1234+the end with `u3a_malt()` (which does the measuring itself)
1134−itself) or `u3a_mint()` in case you've measured it yourself.
1235+or `u3a_mint()` in case you've measured it yourself.
1135−
1236+
1136−Once again, *do not call `malloc()`* (or C++ `new`) within any code that may be
1237+Once again, *do not call `malloc()`* (or C++ `new`) within any
1137−run within a jet. This will cause rare sporadic corruption when we interrupt
1238+code that may be run within a jet. This will cause rare sporadic
1239+corruption when we interrupt execution within a `malloc()`. We'd
1138−execution within a `malloc()`. We'd just override the symbol, but `libuv` uses
1240+just override the symbol, but `libuv` uses `malloc()` across
1139−`malloc()` across threads within its own synchronization primitives - for this
1241+threads within its own synchronization primitives - for this to
1140−to work with `u3a_malloc()`, we'd have to introduce our own locks on the
1242+work with `u3a_malloc()`, we'd have to introduce our own locks on
1141−surface-level road (which might be a viable solution).
1243+the surface-level road (which might be a viable solution).
1142
1244
1143### u3n: nock execution
1245### u3n: nock execution
1144
1246
1145−The `u3n` routines execute Nock itself. On the inside, they have a surprising
1247+The `u3n` routines execute Nock itself. On the inside, they have
1146−resemblance to the spec proper (the only interesting detail is how we handle
1248+a surprising resemblance to the spec proper (the only interesting
1147−tail-call elimination) and are, as one would expect, quite slow. (There is no
1249+detail is how we handle tail-call elimination) and are, as one
1250+would expect, quite slow. (There is no such thing as a fast tree
1148−such thing as a fast tree interpreter.)
1251+interpreter.)
1149−
1252+
1150−There is only one Nock, but there are lots of ways to call it. (Remember that
1253+There is only one Nock, but there are lots of ways to call it.
1151−all `u3n` functions *transfer* C arguments and returns.)
1254+(Remember that all `u3n` functions *transfer* C arguments and
1152−
1255+returns.)
1256+
1153−The simplest interpreter, `u3n_nock_on(u3_noun bus, u3_noun fol)` invokes Nock
1257+The simplest interpreter, `u3n_nock_on(u3_noun bus, u3_noun fol)`
1154−on `bus` (the subject) and `fol` (the formula). (Why is it`[subject formula]`,
1258+invokes Nock on `bus` (the subject) and `fol` (the formula).
1259+(Why is it`[subject formula]`, not `[formula subject]`? The same
1155−not `[formula subject]`? The same reason `0` is true and `1` is false.)
1260+reason `0` is true and `1` is false.)
1156−
1261+
1157−A close relative is `u3n_slam_on(u3_noun gat, u3_noun sam)`, which slams a
1262+A close relative is `u3n_slam_on(u3_noun gat, u3_noun sam)`,
1158−*gate* (`gat`) on a sample (`sam`). (In a normal programming language which
1263+which slams a *gate* (`gat`) on a sample (`sam`). (In a normal
1264+programming language which didn't talk funny and was retarded,
1159−didn't talk funny and was retarded, `u3n_slam_on()` would call a function on an
1265+`u3n_slam_on()` would call a function on an argument.) We could
1160−argument.) We could write it most simply as:
1266+write it most simply as:
1161−
1267+
1162 u3_noun
1268 u3_noun
1163 u3n_slam_on(u3_noun gat, u3_noun sam)
1269 u3n_slam_on(u3_noun gat, u3_noun sam)
1164 {
1270 {
@@ -1170,9 +1276,9 @@ argument.) We could write it most simply as:
1170 return pro;
1276 return pro;
1171 }
1277 }
1172
1278
1173−Simpler is `u3n_kick_on(u3_noun gat)`, which slams a gate (or, more generally, a
1279+Simpler is `u3n_kick_on(u3_noun gat)`, which slams a gate (or,
1174−*trap* - because sample structure is not even needed here) without changing its
1280+more generally, a *trap* - because sample structure is not even
1175−sample:
1281+needed here) without changing its sample:
1176
1282
1177 u3_noun
1283 u3_noun
1178 u3n_kick_on(u3_noun gat, u3_noun sam)
1284 u3n_kick_on(u3_noun gat, u3_noun sam)
@@ -1180,34 +1286,37 @@ sample:
1180 return u3n_nock_on(gat, u3k(u3h(gat)));
1286 return u3n_nock_on(gat, u3k(u3h(gat)));
1181 }
1287 }
1182
1288
1183−The `_on` functions in `u3n` are all defined as pure Nock. But actually, even
1289+The `_on` functions in `u3n` are all defined as pure Nock. But
1184−though we say we don't extend Nock, we do. But we don't. But we do.
1290+actually, even though we say we don't extend Nock, we do. But we
1291+don't. But we do.
1185
1292
1186−Note that `u3` has a well-developed error handling system - `u3m_bail()` to
1293+Note that `u3` has a well-developed error handling system -
1187−throw an exception, `u3m_soft_*` to catch one. But Nock has no exception model
1294+`u3m_bail()` to throw an exception, `u3m_soft_*` to catch one.
1295+But Nock has no exception model at all. That's okay - all it
1188−at all. That's okay - all it means if that if an `_on` function bails, the
1296+means if that if an `_on` function bails, the exception is an
1189−exception is an exception in the caller.
1297+exception in the caller.
1190
1298
1191−However, `u3`'s exception handling happens to match a convenient virtual
1299+However, `u3`'s exception handling happens to match a convenient
1192−super-Nock in `hoon.hoon`, the infamous `++mock`. Of course, Nock is slow, and
1300+virtual super-Nock in `hoon.hoon`, the infamous `++mock`. Of
1301+course, Nock is slow, and `mock` is Nock in Nock, so it is
1193−`mock` is Nock in Nock, so it is (logically) super-slow. Then again, so is
1302+(logically) super-slow. Then again, so is decrement.
1194−decrement.
1195
1303
1196−With the power of `u3`, we nest arbitrary layers of `mock` without any
1304+With the power of `u3`, we nest arbitrary layers of `mock`
1197−particular performance cost. Moreover, we simply treat Nock proper as a special
1305+without any particular performance cost. Moreover, we simply
1306+treat Nock proper as a special case of `mock`. (More precisely,
1198−case of `mock`. (More precisely, the internal VM loop is `++mink` and the error
1307+the internal VM loop is `++mink` and the error compiler is
1199−compiler is `++mook`. But we call the whole sandbox system `mock`.)
1308+`++mook`. But we call the whole sandbox system `mock`.)
1200
1309
1201−The nice thing about `mock` functions is that (by executing within
1310+The nice thing about `mock` functions is that (by executing
1202−`u3m_soft_run()`, which as you may recall uses a nested road) they provide both
1311+within `u3m_soft_run()`, which as you may recall uses a nested
1203−exceptions and the namespace operator - `.^` in Hoon, which becomes operator
1312+road) they provide both exceptions and the namespace operator -
1204−`11` in `mock`.
1313+`.^` in Hoon, which becomes operator `11` in `mock`.
1205
1314
1206−`11` requires a namespace function, or `fly`, which produces a `++unit` - `~`
1315+`11` requires a namespace function, or `fly`, which produces a
1207−(`0`) for no binding, or `[0 value]`. The sample to a `fly` is a `++path`, just
1316+`++unit` - `~` (`0`) for no binding, or `[0 value]`. The sample
1208−a list of text `span`.
1317+to a `fly` is a `++path`, just a list of text `span`.
1209
1318
1210−`mock` functions produce a `++toon`. Fully elaborated:
1319+`mock` functions produce a `++toon`. Fully elaborated:
1211
1320
1212 ++ noun ,* :: any noun
1321 ++ noun ,* :: any noun
1213 ++ path (list ,@ta) :: namespace path
1322 ++ path (list ,@ta) :: namespace path
@@ -1230,18 +1339,20 @@ a list of text `span`.
1230
1339
1231(Note that `tank` is overdesigned and due for replacement.)
1340(Note that `tank` is overdesigned and due for replacement.)
1232
1341
1233−What does a `toon` mean? Either your computation succeded (`[0 noun]`, or could
1342+What does a `toon` mean? Either your computation succeded (`[0
1234−not finish because it blocked on one or more global paths (`[1 (list path)]`),
1343+noun]`, or could not finish because it blocked on one or more
1235−or it exited with a stack trace (`[2 (list tank)]`).
1344+global paths (`[1 (list path)]`), or it exited with a stack trace
1345+(`[2 (list tank)]`).
1236
1346
1237−Note that of all the `u3` exceptions, only `%exit` is produced deterministically
1347+Note that of all the `u3` exceptions, only `%exit` is produced
1238−by the Nock definition. Therefore, only `%exit` produces a `2` result. Any other
1348+deterministically by the Nock definition. Therefore, only
1349+`%exit` produces a `2` result. Any other argument to
1239−argument to `u3m_bail()` will unwind the virtualization stack all the way to the
1350+`u3m_bail()` will unwind the virtualization stack all the way to
1240−top - or to be more exact, to `u3m_soft_top()`.
1351+the top - or to be more exact, to `u3m_soft_top()`.
1241
1352
1242−In any case, the simplest `mock` functions are `u3n_nock_un()` and
1353+In any case, the simplest `mock` functions are `u3n_nock_un()`
1243−`u3n_slam_un()`. These provide exception control without any namespace change,
1354+and `u3n_slam_un()`. These provide exception control without
1244−as you can see by the code:
1355+any namespace change, as you can see by the code:
1245
1356
1246 /* u3n_nock_un(): produce .*(bus fol), as ++toon.
1357 /* u3n_nock_un(): produce .*(bus fol), as ++toon.
1247 */
1358 */
@@ -1263,92 +1374,104 @@ as you can see by the code:
1263 return u3n_slam_in(fly, gat, sam);
1374 return u3n_slam_in(fly, gat, sam);
1264 }
1375 }
1265
1376
1266−The `fly` is added as the first argument to `u3n_nock_in()` and `u3n_slam_in()`.
1377+The `fly` is added as the first argument to `u3n_nock_in()` and
1267−Of course, logically, `fly` executes in the caller's exception layer.
1378+`u3n_slam_in()`. Of course, logically, `fly` executes in the
1268−(Maintaining this illusion is slightly nontrivial.) Finally, `u3n_nock_an()` is
1379+caller's exception layer. (Maintaining this illusion is slightly
1380+nontrivial.) Finally, `u3n_nock_an()` is a sandbox with a null
1269−a sandbox with a null namespace.
1381+namespace.
1270
1382
1271### u3e: persistence
1383### u3e: persistence
1272
1384
1273−The only `u3e` function you should need to call is `u3e_save()`, which saves the
1385+The only `u3e` function you should need to call is `u3e_save()`,
1274−loom. As it can be restored on any platform, please make sure you don't have any
1386+which saves the loom. As it can be restored on any platform,
1387+please make sure you don't have any state in the loom that is
1275−state in the loom that is bound to your process or architecture - except for
1388+bound to your process or architecture - except for exceptions
1276−exceptions like the warm jet state, which is actively purged on reboot.
1389+like the warm jet state, which is actively purged on reboot.
1277
1390
1278### u3r: reading nouns (weak)
1391### u3r: reading nouns (weak)
1279
1392
1280−As befits accessors they don't make anything, `u3r` noun reading functions
1393+As befits accessors they don't make anything, `u3r` noun reading
1281−always retain their arguments and their returns. They never bail; rather, when
1394+functions always retain their arguments and their returns. They
1282−they don't work, they return a `u3_weak` result.
1395+never bail; rather, when they don't work, they return a `u3_weak`
1283−
1396+result.
1397+
1284−Most of these functions are straightforward and do only what their comments say.
1398+Most of these functions are straightforward and do only what
1285−A few are interesting enough to discuss.
1399+their comments say. A few are interesting enough to discuss.
1286−
1400+
1287−`u3r_at()` is the familiar tree fragment function, `/` from the Nock spec. For
1401+`u3r_at()` is the familiar tree fragment function, `/` from the
1288−taking complex nouns apart, `u3r_mean()` is a relatively funky way of
1402+Nock spec. For taking complex nouns apart, `u3r_mean()` is a
1289−deconstructing nouns with a varargs list of `axis`, `u3_noun *`. For cells,
1403+relatively funky way of deconstructing nouns with a varargs list
1404+of `axis`, `u3_noun *`. For cells, triples, etc, decompose with
1290−triples, etc, decompose with `u3r_cell()`, `u3r_trel()`, etc. For the tagged
1405+`u3r_cell()`, `u3r_trel()`, etc. For the tagged equivalents, use
1291−equivalents, use `u3r_pq()` and friends.
1406+`u3r_pq()` and friends.
1292−
1407+
1293−`u3r_sing(u3_noun a, u3_noun b)` (true if `a` and `b` are a *single* noun) are
1408+`u3r_sing(u3_noun a, u3_noun b)` (true if `a` and `b` are a
1294−interesting because it uses mugs to help it out. Clearly, different nouns may
1409+*single* noun) are interesting because it uses mugs to help it
1410+out. Clearly, different nouns may have the same mug, but the
1295−have the same mug, but the same nouns cannot have a different mug. It's
1411+same nouns cannot have a different mug. It's important to
1296−important to understand the performance characteristics of `u3r_sing()`: the
1412+understand the performance characteristics of `u3r_sing()`:
1297−worst possible case is a comparison of duplicate nouns, which have the same
1413+the worst possible case is a comparison of duplicate nouns,
1298−value but were created separately. In this case, the tree is traversed
1414+which have the same value but were created separately. In this
1299−
1415+case, the tree is traversed
1416+
1300−`u3r_sung()` is a deeply funky and frightening version of `u3r_sing()` that
1417+`u3r_sung()` is a deeply funky and frightening version of
1301−unifies pointers to the duplicate nouns it finds, freeing the second copy.
1418+`u3r_sing()` that unifies pointers to the duplicate nouns it
1419+finds, freeing the second copy. Obviously, do not use
1302−Obviously, do not use `u3r_sung()` when you have live, but not reference
1420+`u3r_sung()` when you have live, but not reference counted, noun
1303−counted, noun references from C - if they match a noun with a refcount of 1 that
1421+references from C - if they match a noun with a refcount of 1
1304−gets freed, bad things happen.
1422+that gets freed, bad things happen.
1305−
1423+
1306−It's important to remember that `u3r_mug()`, which produces a 31-bit, nonzero
1424+It's important to remember that `u3r_mug()`, which produces a
1307−insecure hash, uses the `mug_w` slot in any boxed noun as a lazy cache. There
1425+31-bit, nonzero insecure hash, uses the `mug_w` slot in any boxed
1426+noun as a lazy cache. There are a number of variants of
1308−are a number of variants of `u3r_mug()` that can get you out of building
1427+`u3r_mug()` that can get you out of building unneeded nouns.
1309−unneeded nouns.
1310
1428
1311### u3x: reading nouns (bail)
1429### u3x: reading nouns (bail)
1312
1430
1313−`u3x` functions are like `u3r` functions, but instead of returning `u3_none`
1431+`u3x` functions are like `u3r` functions, but instead of
1314−when (for instance) we try to take the head of an atom, they bail with `%exit`.
1432+returning `u3_none` when (for instance) we try to take the head
1433+of an atom, they bail with `%exit`. In other words, they do what
1315−In other words, they do what the same operation would do in Nock.
1434+the same operation would do in Nock.
1316
1435
1317### u3h: hash tables.
1436### u3h: hash tables.
1318
1437
1319−We can of course use the Hoon `map` structure as an associative array. This is a
1438+We can of course use the Hoon `map` structure as an associative
1320−balanced treap and reasonably fast. However, it's considerably inferior to a
1439+array. This is a balanced treap and reasonably fast. However,
1440+it's considerably inferior to a custom structure like an HAMT
1321−custom structure like an HAMT (hash array-mapped trie). We use `u3_post` to
1441+(hash array-mapped trie). We use `u3_post` to allocate HAMT
1322−allocate HAMT structures on the loom.
1442+structures on the loom.
1323
1443
1324−(Our HAMT implements the classic Bagwell algorithm which depends on the `gcc`
1444+(Our HAMT implements the classic Bagwell algorithm which depends
1325−standard directive `__builtin_popcount()`. On a CPU which doesn't support
1445+on the `gcc` standard directive `__builtin_popcount()`. On a CPU
1326−popcount or an equivalent instruction, some other design would probably be
1446+which doesn't support popcount or an equivalent instruction, some
1327−preferable.)
1447+other design would probably be preferable.)
1328
1448
1329−There's no particular rocket science in the API. `u3h_new()` creates a
1449+There's no particular rocket science in the API. `u3h_new()`
1330−hashtable; `u3h_free()` destroys one; `u3h_put()` inserts, `u3h_get()`
1450+creates a hashtable; `u3h_free()` destroys one; `u3h_put()`
1331−retrieves. You can transform values in a hashtable with `u3h_walk()`.
1451+inserts, `u3h_get()` retrieves. You can transform values in a
1452+hashtable with `u3h_walk()`.
1332
1453
1333−The only funky function is `u3h_gut()`, which unifies keys with `u3r_sung()`. As
1454+The only funky function is `u3h_gut()`, which unifies keys with
1334−with all cases of `u3r_sung()`, this must be used with extreme caution.
1455+`u3r_sung()`. As with all cases of `u3r_sung()`, this must be
1456+used with extreme caution.
1335
1457
1336### u3z: memoization
1458### u3z: memoization
1337
1459
1338−Connected to the `~+` rune in Hoon, via the Nock `%memo` hint, the memoization
1460+Connected to the `~+` rune in Hoon, via the Nock `%memo` hint,
1339−facility is a general-purpose cache.
1461+the memoization facility is a general-purpose cache.
1340
1462
1341−(It's also used for partial memoization - a feature that'll probably be removed,
1463+(It's also used for partial memoization - a feature that'll
1342−in which conservative worklist algorithms (which would otherwise be exponential)
1464+probably be removed, in which conservative worklist algorithms
1465+(which would otherwise be exponential) memoize everything in the
1343−memoize everything in the subject *except* the worklist. This is used heavily in
1466+subject *except* the worklist. This is used heavily in the Hoon
1344−the Hoon compiler jets (j/f/\*.c). Unfortunately, it's probably not possible to
1467+compiler jets (j/f/*.c). Unfortunately, it's probably not
1345−make this work perfectly in that it can't be abused to violate Nock, so we'll
1468+possible to make this work perfectly in that it can't be abused
1469+to violate Nock, so we'll probably remove it at a later date,
1346−probably remove it at a later date, instead making `++ut` keep its own monadic
1470+instead making `++ut` keep its own monadic cache.)
1347−cache.)
1348
1471
1349−Each `u3z` function comes with a `c3_m` mote which disambiguates the function
1472+Each `u3z` function comes with a `c3_m` mote which disambiguates
1350−mapping key to value. For Nock itself, use 0. For extra speed, small tuples are
1473+the function mapping key to value. For Nock itself, use 0. For
1351−split out in C; thus, find with
1474+extra speed, small tuples are split out in C; thus, find with
1352
1475
1353 u3_weak u3z_find(c3_m, u3_noun);
1476 u3_weak u3z_find(c3_m, u3_noun);
1354 u3_weak u3z_find_2(c3_m, u3_noun, u3_noun);
1477 u3_weak u3z_find_2(c3_m, u3_noun, u3_noun);
@@ -1362,18 +1485,18 @@ and save with
1362 u3_noun u3z_save_3(c3_m, u3_noun, u3_noun, u3_noun, u3_noun);
1485 u3_noun u3z_save_3(c3_m, u3_noun, u3_noun, u3_noun, u3_noun);
1363 u3_noun u3z_save_4(c3_m, u3_noun, u3_noun, u3_noun, u3_noun, u3_noun);
1486 u3_noun u3z_save_4(c3_m, u3_noun, u3_noun, u3_noun, u3_noun, u3_noun);
1364
1487
1365−where the value is the last argument. To eliminate duplicate nouns, there is
1488+where the value is the last argument. To eliminate duplicate
1366−also
1489+nouns, there is also
1367
1490
1368 u3_noun
1491 u3_noun
1369 u3z_uniq(u3_noun);
1492 u3z_uniq(u3_noun);
1370
1493
1371`u3z` functions retain keys and transfer values.
1494`u3z` functions retain keys and transfer values.
1372−
1495+
1373−The `u3z` cache, built on `u3h` hashes, is part of the current road, and goes
1496+The `u3z` cache, built on `u3h` hashes, is part of the current
1374−away when it goes away. (In future, we may wish to promote keys/values which
1497+road, and goes away when it goes away. (In future, we may wish
1498+to promote keys/values which outlive the road, as we do with jet
1375−outlive the road, as we do with jet state.) There is no cache reclamation at
1499+state.) There is no cache reclamation at present, so be careful.
1376−present, so be careful.
1377
1500
1378### u3t: tracing and profiling.
1501### u3t: tracing and profiling.
1379
1502
@@ -1381,8 +1504,9 @@ TBD.
1381
1504
1382### u3v: the Arvo kernel
1505### u3v: the Arvo kernel
1383
1506
1384−An Arvo kernel - or at least, a core that compiles with the Arvo interface - is
1507+An Arvo kernel - or at least, a core that compiles with the Arvo
1385−part of the global `u3` state. What is an Arvo core? Slightly pseudocoded:
1508+interface - is part of the global `u3` state. What is an Arvo
1509+core? Slightly pseudocoded:
1386
1510
1387 ++ arvo
1511 ++ arvo
1388 |%
1512 |%
@@ -1409,49 +1533,54 @@ part of the global `u3` state. What is an Arvo core? Slightly pseudocoded:
1409 ++ ovum {p/wire q/card} :: Arvo event
1533 ++ ovum {p/wire q/card} :: Arvo event
1410 ++ wire path :: event cause
1534 ++ wire path :: event cause
1411
1535
1412−This is the Arvo ABI in a very real sense. Arvo is a core with these six arms.
1536+This is the Arvo ABI in a very real sense. Arvo is a core with
1413−To use these arms, we hardcode the axis of the formula (`11`, `4`, `86`, etc)
1537+these six arms. To use these arms, we hardcode the axis of the
1538+formula (`11`, `4`, `86`, etc) into the C code that calls Arvo,
1414−into the C code that calls Arvo, because otherwise we'd need type metadata -
1539+because otherwise we'd need type metadata - which we can get, by
1415−which we can get, by calling Arvo.
1540+calling Arvo.
1416
1541
1417−It's important to understand the Arvo event/action structure, or `++ovum`. An
1542+It's important to understand the Arvo event/action structure, or
1418−`ovum` is a `card`, which is any `[term noun]` cell, and a `++wire`, a `path`
1543+`++ovum`. An `ovum` is a `card`, which is any `[term noun]`
1544+cell, and a `++wire`, a `path` which indicates the location of
1419−which indicates the location of the event. At the Unix level, the `wire`
1545+the event. At the Unix level, the `wire` corresponds to a system
1420−corresponds to a system module or context. For input events, this is the module
1546+module or context. For input events, this is the module that
1421−that caused the event; for output actions, it's the module that performs the
1547+caused the event; for output actions, it's the module that
1422−action.
1548+performs the action.
1423
1549
1424−`++poke` sends Arvo an event `ovum`, producing a cell of action ova and a new
1550+`++poke` sends Arvo an event `ovum`, producing a cell of action
1425−Arvo core.
1551+ova and a new Arvo core.
1426
1552
1427−`++peek` dereferences the Arvo namespace. It takes a date and a key, and
1553+`++peek` dereferences the Arvo namespace. It takes a date and a
1428−produces `~` (`0`) or `[~ value]`.
1554+key, and produces `~` (`0`) or `[~ value]`.
1429
1555
1430−`++keep` asks Arvo the next time it wants to be woken up, for the given `wire`.
1556+`++keep` asks Arvo the next time it wants to be woken up, for the
1431−(This input will probably be eliminated in favor of a single global timer.)
1557+given `wire`. (This input will probably be eliminated in favor
1558+of a single global timer.)
1432
1559
1433−`++wish` compiles a string of Hoon source. While just a convenience, it's a very
1560+`++wish` compiles a string of Hoon source. While just a
1434−convenient convenience.
1561+convenience, it's a very convenient convenience.
1435
1562
1436−`++come` and `++load` are used by Arvo to reset itself (more precisely, to shift
1563+`++come` and `++load` are used by Arvo to reset itself (more
1437−the Arvo state from an old kernel to a new one); there is no need to call them
1564+precisely, to shift the Arvo state from an old kernel to a new
1438−from C.
1565+one); there is no need to call them from C.
1439
1566
1440−Now that we understand the Arvo kernel interface, let's look at the `u3v` API.
1567+Now that we understand the Arvo kernel interface, let's look at
1441−As usual, all the functions in `u3v` are commented, but unfortunately it's hard
1568+the `u3v` API. As usual, all the functions in `u3v` are
1569+commented, but unfortunately it's hard to describe this API as
1442−to describe this API as clean at present. The problem is that `u3v` remains
1570+clean at present. The problem is that `u3v` remains design
1443−design coupled to the old `vere` event handling code written for `u2`. But let's
1571+coupled to the old `vere` event handling code written for `u2`.
1444−describe the functions you should be calling, assuming you're not writing the
1572+But let's describe the functions you should be calling, assuming
1445−next event system. There are only two.
1573+you're not writing the next event system. There are only two.
1446
1574
1447−`u3v_wish(str_c)` wraps the `++wish` functionality in a cache (which is
1575+`u3v_wish(str_c)` wraps the `++wish` functionality in a cache
1448−read-only unless you're on the surface road).
1576+(which is read-only unless you're on the surface road).
1449
1577
1450−`u3v_do()` uses `wish` to provide a convenient interface for calling Hoon kernel
1578+`u3v_do()` uses `wish` to provide a convenient interface for
1451−functions by name. Even more conveniently, we tend to call `u3v_do()` with these
1579+calling Hoon kernel functions by name. Even more conveniently,
1452−convenient aliases:
1580+we tend to call `u3v_do()` with these convenient aliases:
1453
1581
1454 #define u3do(txt_c, arg) u3v_do(txt_c, arg)
1582 #define u3do(txt_c, arg) u3v_do(txt_c, arg)
1455 #define u3dc(txt_c, a, b) u3v_do(txt_c, u3nc(a, b))
1583 #define u3dc(txt_c, a, b) u3v_do(txt_c, u3nc(a, b))
1456 #define u3dt(txt_c, a, b, c) u3v_do(txt_c, u3nt(a, b, c))
1584 #define u3dt(txt_c, a, b, c) u3v_do(txt_c, u3nt(a, b, c))
1457 #define u3dq(txt_c, a, b, c, d) u3v_do(txt_c, u3nt(a, b, c, d))
1585 #define u3dq(txt_c, a, b, c, d) u3v_do(txt_c, u3nt(a, b, c, d))
1586+
docs/arvo.md+2−3
@@ -1,12 +1,11 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 4
3sort: 4
4---
4---
5
5
6# Arvo
6# Arvo
7
7
8−Fast, pragmatic tutorials which show you how to build simple programs in Hoon
8+Fast, pragmatic tutorials which show you how to build simple programs in Hoon and Arvo.
9−and Arvo.
10
9
11If you learn by getting things done, not by exploring fundamentals, start here.
10If you learn by getting things done, not by exploring fundamentals, start here.
12
11
docs/arvo/api.md+456−425
@@ -1,34 +1,38 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
3+next: true
4sort: 21
4sort: 21
5title: API Connectors
5title: API Connectors
6---
6---
7
7
8# API Connectors
8# API Connectors
9
9
10−Most people have lots of data stored in online services, many of which have
10+Most people have lots of data stored in online services, many of
11−APIs. API connectors allow the user to access this data from within their Urbit.
11+which have APIs. API connectors allow the user to access this
12+data from within their Urbit.
12
13
13−A security driver allows the user to make authenticated requests to the
14+A security driver allows the user to make authenticated requests
14−service's API, but the user still needs to know the API and send and receive
15+to the service's API, but the user still needs to know the API
15−JSON. An API connector puts a layer of porcelain over this to allow the user
16+and send and receive JSON. An API connector puts a layer of
16−easier control over their data.
17+porcelain over this to allow the user easier control over their
18+data.
17
19
18−Each connector should perform two basic functions. First, it should expose a
20+Each connector should perform two basic functions. First, it
19−tree of the data in the service that's accessible through `.^` and FUSE. Second,
21+should expose a tree of the data in the service that's accessible
22+through `.^` and FUSE. Second, it should expose event streams as
20−it should expose event streams as actions occur in the service. Let's take a
23+actions occur in the service. Let's take a look at how the `%gh`
21−look at how the `%gh` app accomplishes both for Github.
24+app accomplishes both for Github.
22
25
23−After starting `%gh` (`|start %gh`), let's look at the root of the tree that
26+After starting `%gh` (`|start %gh`), let's look at the root of
24−`%gh` exposes:
27+the tree that `%gh` exposes:
25
28
26 ~your-urbit:dojo> .^(arch %gy /=gh=)
29 ~your-urbit:dojo> .^(arch %gy /=gh=)
27 [fil=~ dir={[p=~.issues q=~]}]
30 [fil=~ dir={[p=~.issues q=~]}]
28
31
29−`%gh` is currently a skeleton -- it contains examples of all the functionality
32+`%gh` is currently a skeleton -- it contains examples of all the
30−necessary for a connector, but many endpoints aren't implemented. As we can see
33+functionality necessary for a connector, but many endpoints
34+aren't implemented. As we can see here, only issues are
31−here, only issues are implemented. We can explore this tree:
35+implemented. We can explore this tree:
32
36
33 ~your-urbit:dojo> .^(arch %gy /=gh=/issues)
37 ~your-urbit:dojo> .^(arch %gy /=gh=/issues)
34 [fil=~ dir={[p=~.by-repo q=~] [p=~.mine q=~]}]
38 [fil=~ dir={[p=~.by-repo q=~] [p=~.mine q=~]}]
@@ -41,50 +45,58 @@ And eventually:
41 login.user:.^(issue:gh %gx /=gh=/issues/by-repo/philipcmonktest/testing/7)
45 login.user:.^(issue:gh %gx /=gh=/issues/by-repo/philipcmonktest/testing/7)
42 'philipcmonktest'
46 'philipcmonktest'
43
47
44−The other thing an API connector needs to do is expose event streams. One
48+The other thing an API connector needs to do is expose event
45−straightforward way to use this capacity is with the `:pipe` app:
49+streams. One straightforward way to use this capacity is with
50+the `:pipe` app:
46
51
47 :pipe|connect %gh /listen/philipcmonktest/testing/issues/'issue_comment' %public
52 :pipe|connect %gh /listen/philipcmonktest/testing/issues/'issue_comment' %public
48
53
49−Now creating an issue results in the following message in your `%public` `:talk`
54+Now creating an issue results in the following message in your
50−channel:
55+`%public` `:talk` channel:
51
56
52 ~your-urbit[philipcmonktest@github]: opened issue #11: i found a bug!
57 ~your-urbit[philipcmonktest@github]: opened issue #11: i found a bug!
53
58
54−You'll also get notifications of issue status changes and comments.
59+You'll also get notifications of issue status changes and
60+comments.
55
61
56−As is often the case, the best way to write an API connector is to copy an
62+As is often the case, the best way to write an API connector is
57−existing one and modify it. We'll dissect the Github connector here. This code
63+to copy an existing one and modify it. We'll dissect the Github
64+connector here. This code is in `/=home=/app/gh/hoon`, and it's
58−is in `/=home=/app/gh/hoon`, and it's also reproduced at the bottom of this page
65+also reproduced at the bottom of this page (in case the code
59−(in case the code drifts out of sync with this doc).
66+drifts out of sync with this doc).
60
67
61We'll go over the two parts to an API connector (one-time reading vs listening
68We'll go over the two parts to an API connector (one-time reading vs listening
62for events) separately.
69for events) separately.
63
70
64## Reading
71## Reading
65
72
66−A connector exposes a tree of data. Every read request has a `care`, which is
73+A connector exposes a tree of data. Every read request has a
67−generally either `%x` or
74+`care`, which is generally either `%x` or `%y. `%x` is a request
68−`%y.`%x`is a request for a particular piece of data, while`%y`is a request for a directory listing. Roughly,`%x`means Unix`cat`and`%y`means Unix`ls\`.
75+for a particular piece of data, while `%y` is a request for a
69−
76+directory listing. Roughly, `%x` means Unix `cat` and `%y` means
77+Unix `ls`.
78+
70−> Sometimes you wish to expose a tree where a part of the path can't be
79+> Sometimes you wish to expose a tree where a part of the path
71−> enumerated. For example, a `%y` of `/issues/by-repo` "should" produce a list
80+> can't be enumerated. For example, a `%y` of `/issues/by-repo`
81+> "should" produce a list of all Github users, but we don't do
72−> of all Github users, but we don't do that because it's too long. Instead, we
82+> that because it's too long. Instead, we just produce our own
73−> just produce our own username (from the `web.plan` file). You can still access
83+> username (from the `web.plan` file). You can still access
74−> repos from other users, you just don't see them in the directory listing.
84+> repos from other users, you just don't see them in the
75−
85+> directory listing.
86+
76−The usual flow for implementing this tree of data makes heavy use of the
87+The usual flow for implementing this tree of data makes heavy use
77−`connector` library. This library is well documented in the source, so check out
88+of the `connector` library. This library is well documented in
78−`/=home=/lib/connector/hoon`.
89+the source, so check out `/=home=/lib/connector/hoon`.
79−
90+
80−The `connector` library needs to be initialized with definitions of `move` and
91+The `connector` library needs to be initialized with definitions
81−`sub-result` (all the types of data that can be returned by a place). This can
92+of `move` and `sub-result` (all the types of data that can be
82−just be a line at the top of the main core:
93+returned by a place). This can just be a line at the top of the
94+main core:
83
95
84 =+ connector=(connector move sub-result) :: Set up connector library
96 =+ connector=(connector move sub-result) :: Set up connector library
85
97
86−Most of the Github-specific logic is in `++places`, which is a list of all the
98+Most of the Github-specific logic is in `++places`, which is a
87−places we can request. A place consists of:
99+list of all the places we can request. A place consists of:
88
100
89 ++ place
101 ++ place
90 $: guard/mold
102 $: guard/mold
@@ -94,402 +106,421 @@ places we can request. A place consists of:
94 sigh-y/$-(jon/json (unit arch))
106 sigh-y/$-(jon/json (unit arch))
95 ==
107 ==
96
108
97−- `guard`, the type of the paths we should match. For example, to match
109+- `guard`, the type of the paths we should match. For example,
98− `/issues/<user>/<repo>` use `{$issues @t @t $~}`.
110+ to match `/issues/<user>/<repo>` use `{$issues @t @t $~}`.
99−
111+
100−- `read-x`, called when someone tries to read the place with `care` `%x`.
112+- `read-x`, called when someone tries to read the place with
101− Should produce a single move, usually either a `%diff` response if we can
113+ `care` `%x`. Should produce a single move, usually either a
114+ `%diff` response if we can immediately answer or a `%hiss`
102− immediately answer or a `%hiss` http request if we need to make a request to
115+ http request if we need to make a request to the api. See the
103− the api. See the `++read-*` functions in `++helpers` for some common
116+ `++read-*` functions in `++helpers` for some common handlers.
104− handlers.
117+
105−
106−- `read-y`, same as `read-x` except with care `%y`.
118+- `read-y`, same as `read-x` except with care `%y`.
107−
119+
108−- `sigh-x`, called when an http response comes back on this place. You're
120+- `sigh-x`, called when an http response comes back on this
109− given the json of the result, and you should produce either a result or
121+ place. You're given the json of the result, and you should
122+ produce either a result or null. Null represents an error.
110− null. Null represents an error. If you didn't create an http request in
123+ If you didn't create an http request in `read-x`, then this
111− `read-x`, then this should never be called. Use `++sigh-strange` from
124+ should never be called. Use `++sigh-strange` from `++helpers`
112− `++helpers` to unconditionally signal an error.
125+ to unconditionally signal an error.
113−
126+
114−- `sigh-y`, same as `sigh-x` except with care `%y`. Note that a `%y` request
127+- `sigh-y`, same as `sigh-x` except with care `%y`. Note that a
115− must produce an arch, unlike a `%x` request, which may produce data of any
128+ `%y` request must produce an arch, unlike a `%x` request,
116− mark.
129+ which may produce data of any mark.
117−
130+
118−Filling out the list of places is a lot of grunt work, but most places are
131+Filling out the list of places is a lot of grunt work, but most
119−fairly straightforward, and the `++read-*` helper functions are useful. Check
132+places are fairly straightforward, and the `++read-*` helper
133+functions are useful. Check out the library source for more
120−out the library source for more information on those.
134+information on those.
121−
135+
122−Besides the list of places, we just need to handle the flow of control. `++peek`
136+Besides the list of places, we just need to handle the flow of
123−and `++peer-scry` are the interface we expose to the rest of the system while
137+control. `++peek` and `++peer-scry` are the interface we expose
138+to the rest of the system while `++sigh-httr` and `++sigh-tang`
124−`++sigh-httr` and `++sigh-tang` are used to handle responses when we make HTTP
139+are used to handle responses when we make HTTP requests.
125−requests.
140+
126−
127−- `++peek` should usually just produce `~`. If there are cases where we can
141+- `++peek` should usually just produce `~`. If there are cases
128− respond directly to a `.^` request without blocking on anything, we could do
142+ where we can respond directly to a `.^` request without
143+ blocking on anything, we could do it here, but it's generally
129− it here, but it's generally not worth the hassle since the same logic should
144+ not worth the hassle since the same logic should be duplicated
130− be duplicated in `++peer-scry`.
145+ in `++peer-scry`.
131−
146+
132−- `++peer-scry` is where the actual handling for a read request goes. In
147+- `++peer-scry` is where the actual handling for a read request
133− general, we just need to call `++read` from the `connector` library with the
148+ goes. In general, we just need to call `++read` from the
149+ `connector` library with the bone, list of places, care, and
134− bone, list of places, care, and path. This will match the path to the
150+ path. This will match the path to the appropriate place and
135− appropriate place and run either `++read-x` or `++read-y`, depending on the
151+ run either `++read-x` or `++read-y`, depending on the care.
136− care.
152+
137−
138−- If `++read-x` or `++read-y` made a successful API request, then the response
153+- If `++read-x` or `++read-y` made a successful API request, then
139− will come back on `++sigh-httr`. Here, we just need to parse out the wire
154+ the response will come back on `++sigh-httr`. Here, we just
155+ need to parse out the wire and call `++sigh` from the
140− and call `++sigh` from the `connector` library with the list of places,
156+ `connector` library with the list of places, care, path, and
141− care, path, and HTTP result. This will match the path to the appropriate
157+ HTTP result. This will match the path to the appropriate place
142− place and run either `++sigh-x` or `++sigh-y`, depending on the care.
158+ and run either `++sigh-x` or `++sigh-y`, depending on the care.
143−
159+
144−- If `++read-x` or `++read-y` made an API request that failed, then we'll get
160+- If `++read-x` or `++read-y` made an API request that failed,
145− a stack trace in `++sigh-tang`. Here, we just print it out and move on.
161+ then we'll get a stack trace in `++sigh-tang`. Here, we just
146−
162+ print it out and move on.
163+
147−That's really all there is to the reading portion of API connectors.
164+That's really all there is to the reading portion of API
148−
165+connectors.
166+
149−One of the most accessbile ways to jump into Arvo programming is to just add
167+One of the most accessbile ways to jump into Arvo programming is
150−more places to an existing API connector. It's useful, small in scope, and comes
168+to just add more places to an existing API connector. It's
169+useful, small in scope, and comes in bite-sized chunks since most
151−in bite-sized chunks since most places are less than ten lines of code.
170+places are less than ten lines of code.
152
171
153## Listening
172## Listening
154
173
155−Listening for events is fairly service-specific. In some services, we poll for
174+Listening for events is fairly service-specific. In some
175+services, we poll for changes. The Twitter connector has an
156−changes. The Twitter connector has an example of this, but note that it predates
176+example of this, but note that it predates the `connector`
157−the `connector` library and is thus more complicated than it needs to be, and
177+library and is thus more complicated than it needs to be, and the
158−the interface it exposes isn't standard. In Github, we power our event streams
178+interface it exposes isn't standard. In Github, we power our
159−with webhooks.
179+event streams with webhooks.
160−
180+
161−For Github, when someone subscribes to `/listen/<user>/<repo>/<events...>`, we
181+For Github, when someone subscribes to
182+`/listen/<user>/<repo>/<events...>`, we want to produce
162−want to produce well-typed results when they occur.
183+well-typed results when they occur.
163−
184+
164−We only want to create one webhook per event, so in our state we have `hook`,
185+We only want to create one webhook per event, so in our state we
165−which is a map of event names to the set of bones which are subscribed to that
186+have `hook`, which is a map of event names to the set of bones
166−event. We must always keep this up-to-date.
187+which are subscribed to that event. We must always keep this
167−
188+up-to-date.
189+
168−Flow of control starts in `++peer-listen`, where we just call `++listen`. In
190+Flow of control starts in `++peer-listen`, where we just call
169−`++listen`, for each event in the list of events, we check to see if we have
191+`++listen`. In `++listen`, for each event in the list of events,
170−that hook set up (by checking whether it exists in `hook`). If so, we call
192+we check to see if we have that hook set up (by checking whether
193+it exists in `hook`). If so, we call `++update-hook` to add the
171−`++update-hook` to add the current bone to the set of listeners. Otherwise, we
194+current bone to the set of listeners. Otherwise, we call
172−call `++create-hook`, which sends a request to Github to set up the new webhook.
195+`++create-hook`, which sends a request to Github to set up the
173−We also create an entry in `hook` with the current bone.
196+new webhook. We also create an entry in `hook` with the current
197+bone.
174
198
175When we created the webhook, we told Github to send the event to
199When we created the webhook, we told Github to send the event to
176−`/~/to/gh/gh-<event>.json?anon&wire=/`. This turns into a poke. Let's parse out
200+`/~/to/gh/gh-<event>.json?anon&wire=/`. This turns into a poke.
177−this url. The first `gh` is the app name to poke. The next portion, `gh-<event>`
201+Let's parse out this url. The first `gh` is the app name to
202+poke. The next portion, `gh-<event>` is the mark to convert the
178−is the mark to convert the JSON to. For example, when the "issues" event fires,
203+JSON to. For example, when the "issues" event fires, it'll get
179−it'll get converted to mark `gh-issues`. This mark has a conversion routine from
204+converted to mark `gh-issues`. This mark has a conversion
180−`json` to the `issues` type, which is defined in the `gh` structure
205+routine from `json` to the `issues` type, which is defined in the
181−(`/=home=/sur/gh/hoon`). Once this conversion happens, the `gh` app will get
206+`gh` structure (`/=home=/sur/gh/hoon`). Once this conversion
182−poked with the correctly marked data in `++poke`.
207+happens, the `gh` app will get poked with the correctly marked
208+data in `++poke`.
183
209
184−In `++poke`, we check `hook` to find the subscribers to the event that was
210+In `++poke`, we check `hook` to find the subscribers to the event
185−fired, and we update them.
211+that was fired, and we update them.
186
212
187−That's really all there is to it. Webhook flow isn't well standardized, so even
213+That's really all there is to it. Webhook flow isn't well
214+standardized, so even if your chosen service is powered by
188−if your chosen service is powered by webhooks, your listening code might look
215+webhooks, your listening code might look rather different. The
189−rather different. The point is to expose the correct interface.
216+point is to expose the correct interface.
190
217
191−## Github API Connector Code
192
218
193−In case the code in `/=home=/app/gh/hoon` drifts out of sync with this doc:
219+## Github API Connector Code
194
220
221+In case the code in `/=home=/app/gh/hoon` drifts out of sync with
222+this doc:
223+
224+```
195− :: This is a connector for the Github API v3.
225+:: This is a connector for the Github API v3.
196− ::
226+::
197− :: You can interact with this in a few different ways:
227+:: You can interact with this in a few different ways:
198− ::
228+::
199− :: - .^({type} %gx /=gh={/endpoint}) to read data or
229+:: - .^({type} %gx /=gh={/endpoint}) to read data or
200− :: .^(arch %gy /=gh={/endpoint}) to explore the possible
230+:: .^(arch %gy /=gh={/endpoint}) to explore the possible
201− :: endpoints.
231+:: endpoints.
232+::
233+:: - subscribe to /listen/{owner}/{repo}/{events...} for
234+:: webhook-powered event notifications. For event list, see
235+:: https://developer.github.com/webhooks/.
236+::
237+:: This is written with the standard structure for api
238+:: connectors, as described in lib/connector.hoon.
239+::
240+/? 314
241+/- gh, plan-acct
242+/+ gh-parse, connector
243+::
244+!:
245+=> |%
246+ ++ move (pair bone card)
247+ ++ card
248+ $% {$diff sub-result}
249+ {$them wire (unit hiss)}
250+ {$hiss wire {$~ $~} $httr {$hiss hiss}}
251+ ==
202 ::
252 ::
203− :: - subscribe to /listen/{owner}/{repo}/{events...} for
253+ :: Types of results we produce to subscribers.
204− :: webhook-powered event notifications. For event list, see
205− :: https://developer.github.com/webhooks/.
206 ::
254 ::
207− :: This is written with the standard structure for api
255+ ++ sub-result
208− :: connectors, as described in lib/connector.hoon.
256+ $% {$arch arch}
257+ {$gh-issue issue:gh}
258+ {$gh-list-issues (list issue:gh)}
259+ {$gh-issues issues:gh}
260+ {$gh-issue-comment issue-comment:gh}
261+ {$json json}
262+ {$null $~}
263+ ==
209 ::
264 ::
210− /? 314
265+ :: Types of webhooks we expect.
211− /- gh, plan-acct
212− /+ gh-parse, connector
213 ::
266 ::
214− !:
267+ ++ hook-response
215− => |%
268+ $% {$gh-issues issues:gh}
216− ++ move (pair bone card)
269+ {$gh-issue-comment issue-comment:gh}
217− ++ card
270+ ==
218− $% {$diff sub-result}
271+ --
219− {$them wire (unit hiss)}
272+=+ connector=(connector move sub-result) :: Set up connector library
220− {$hiss wire {$~ $~} $httr {$hiss hiss}}
273+::
274+|_ $: hid/bowl
275+ hook/(map @t {id/@t listeners/(set bone)}) :: map events to listeners
221− ==
276+ ==
222− ::
277+:: ++ prep _`. :: Clear state when code changes
223− :: Types of results we produce to subscribers.
278+::
279+:: List of endpoints
280+::
281+++ places
282+ |= wir/wire
283+ ^- (list place:connector)
284+ =+ (helpers:connector ost.hid wir "https://api.github.com")
285+ => |% :: gh-specific helpers
286+ ++ sigh-list-issues-x
287+ |= jon/json
288+ %+ bind ((ar:jo issue:gh-parse) jon)
289+ |= issues/(list issue:gh)
290+ gh-list-issues+issues
291+ ::
292+ ++ sigh-list-issues-y
293+ |= jon/json
294+ %+ bind ((ar:jo issue:gh-parse) jon)
295+ |= issues/(list issue:gh)
296+ :- `(shax (jam issues))
297+ (malt (turn issues |=(issue:gh [(rsh 3 2 (scot %ui number)) ~])))
298+ --
299+ :~ ^- place :: /
300+ :* guard=$~
301+ read-x=read-null
302+ read-y=(read-static %issues ~)
303+ sigh-x=sigh-strange
304+ sigh-y=sigh-strange
305+ ==
306+ ^- place :: /issues
307+ :* guard={$issues $~}
308+ read-x=read-null
309+ read-y=(read-static %mine %by-repo ~)
310+ sigh-x=sigh-strange
311+ sigh-y=sigh-strange
312+ ==
313+ ^- place :: /issues/mine
314+ :* guard={$issues $mine $~}
315+ read-x=(read-get /issues)
316+ read-y=(read-get /issues)
317+ sigh-x=sigh-list-issues-x
318+ sigh-y=sigh-list-issues-y
319+ ==
320+ ^- place :: /issues/by-repo
321+ :* guard={$issues $by-repo $~}
322+ read-x=read-null
323+ ^= read-y
324+ |= pax/path
325+ =+ /(scot %p our.hid)/home/(scot %da now.hid)/web/plan
326+ =+ .^({* acc/(map knot plan-acct)} %cx -)
224 ::
327 ::
225− ++ sub-result
328+ ((read-static usr:(~(got by acc) %github) ~) pax)
226− $% {$arch arch}
329+ sigh-x=sigh-strange
227− {$gh-issue issue:gh}
330+ sigh-y=sigh-strange
228− {$gh-list-issues (list issue:gh)}
331+ ==
229− {$gh-issues issues:gh}
332+ ^- place :: /issues/by-repo/<user>
230− {$gh-issue-comment issue-comment:gh}
333+ :* guard={$issues $by-repo @t $~}
231− {$json json}
232− {$null $~}
334+ read-x=read-null
335+ read-y=|=(pax/path (get /users/[-.+>.pax]/repos))
336+ sigh-x=sigh-strange
337+ ^= sigh-y
338+ |= jon/json
339+ %+ bind ((ar:jo repository:gh-parse) jon)
340+ |= repos/(list repository:gh)
341+ [~ (malt (turn repos |=(repository:gh [name ~])))]
342+ ==
343+ ^- place :: /issues/by-repo/<user>/<repo>
344+ :* guard={$issues $by-repo @t @t $~}
345+ read-x=|=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
346+ read-y=|=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
347+ sigh-x=sigh-list-issues-x
348+ sigh-y=sigh-list-issues-y
233− ==
349+ ==
350+ ^- place :: /issues/by-repo/<user>/<repo>
351+ :* guard={$issues $by-repo @t @t @t $~}
352+ ^= read-x
353+ |=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues/[-.+>+>.pax]))
234 ::
354 ::
235− :: Types of webhooks we expect.
355+ read-y=(read-static ~)
356+ ^= sigh-x
357+ |= jon/json
358+ %+ bind (issue:gh-parse jon)
359+ |= issue/issue:gh
360+ gh-issue+issue
236 ::
361 ::
237− ++ hook-response
238− $% {$gh-issues issues:gh}
239− {$gh-issue-comment issue-comment:gh}
240− ==
241− --
242− =+ connector=(connector move sub-result) :: Set up connector library
243− ::
244− |_ $: hid/bowl
245− hook/(map @t {id/@t listeners/(set bone)}) :: map events to listeners
246− ==
247− :: ++ prep _`. :: Clear state when code changes
248− ::
249− :: List of endpoints
250− ::
251− ++ places
252− |= wir/wire
253− ^- (list place:connector)
254− =+ (helpers:connector ost.hid wir "https://api.github.com")
255− => |% :: gh-specific helpers
256− ++ sigh-list-issues-x
257− |= jon/json
258− %+ bind ((ar:jo issue:gh-parse) jon)
259− |= issues/(list issue:gh)
260− gh-list-issues+issues
261− ::
262− ++ sigh-list-issues-y
263− |= jon/json
264− %+ bind ((ar:jo issue:gh-parse) jon)
265− |= issues/(list issue:gh)
266− :- `(shax (jam issues))
267− (malt (turn issues |=(issue:gh [(rsh 3 2 (scot %ui number)) ~])))
268− --
269− :~ ^- place :: /
270− :* guard=$~
271− read-x=read-null
272− read-y=(read-static %issues ~)
273− sigh-x=sigh-strange
362+ sigh-y=sigh-strange
274− sigh-y=sigh-strange
275− ==
276− ^- place :: /issues
277− :* guard={$issues $~}
278− read-x=read-null
279− read-y=(read-static %mine %by-repo ~)
280− sigh-x=sigh-strange
281− sigh-y=sigh-strange
282− ==
283− ^- place :: /issues/mine
284− :* guard={$issues $mine $~}
285− read-x=(read-get /issues)
286− read-y=(read-get /issues)
287− sigh-x=sigh-list-issues-x
288− sigh-y=sigh-list-issues-y
289− ==
290− ^- place :: /issues/by-repo
291− :* guard={$issues $by-repo $~}
292− read-x=read-null
293− ^= read-y
294− |= pax/path
295− =+ /(scot %p our.hid)/home/(scot %da now.hid)/web/plan
296− =+ .^({* acc/(map knot plan-acct)} %cx -)
297− ::
298− ((read-static usr:(~(got by acc) %github) ~) pax)
299− sigh-x=sigh-strange
300− sigh-y=sigh-strange
301− ==
302− ^- place :: /issues/by-repo/<user>
303− :* guard={$issues $by-repo @t $~}
304− read-x=read-null
305− read-y=|=(pax/path (get /users/[-.+>.pax]/repos))
306− sigh-x=sigh-strange
307− ^= sigh-y
308− |= jon/json
309− %+ bind ((ar:jo repository:gh-parse) jon)
310− |= repos/(list repository:gh)
311− [~ (malt (turn repos |=(repository:gh [name ~])))]
312− ==
313− ^- place :: /issues/by-repo/<user>/<repo>
314− :* guard={$issues $by-repo @t @t $~}
315− read-x=|=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
316− read-y=|=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues))
317− sigh-x=sigh-list-issues-x
318− sigh-y=sigh-list-issues-y
319− ==
320− ^- place :: /issues/by-repo/<user>/<repo>
321− :* guard={$issues $by-repo @t @t @t $~}
322− ^= read-x
323− |=(pax/path (get /repos/[-.+>.pax]/[-.+>+.pax]/issues/[-.+>+>.pax]))
324− ::
325− read-y=(read-static ~)
326− ^= sigh-x
327− |= jon/json
328− %+ bind (issue:gh-parse jon)
329− |= issue/issue:gh
330− gh-issue+issue
331− ::
332− sigh-y=sigh-strange
333− ==
334 ==
363 ==
364+ ==
335− ::
365+::
336− :: When a peek on a path blocks, ford turns it into a peer on
366+:: When a peek on a path blocks, ford turns it into a peer on
337− :: /scry/{care}/{path}. You can also just peer to this
367+:: /scry/{care}/{path}. You can also just peer to this
338− :: directly.
368+:: directly.
339− ::
369+::
340− :: We hand control to ++scry.
370+:: We hand control to ++scry.
341− ::
371+::
342− ++ peer-scry
372+++ peer-scry
343− |= pax/path
373+ |= pax/path
344− ^- {(list move) _+>.$}
374+ ^- {(list move) _+>.$}
345− ?> ?=({care *} pax)
375+ ?> ?=({care *} pax)
346− :_ +>.$ :_ ~
376+ :_ +>.$ :_ ~
347− (read:connector ost.hid (places %read pax) i.pax t.pax)
377+ (read:connector ost.hid (places %read pax) i.pax t.pax)
348− ::
378+::
349− :: HTTP response. We make sure the response is good, then
379+:: HTTP response. We make sure the response is good, then
350− :: produce the result (as JSON) to whoever sent the request.
380+:: produce the result (as JSON) to whoever sent the request.
351− ::
381+::
352− ++ sigh-httr
382+++ sigh-httr
353− |= {way/wire res/httr}
383+ |= {way/wire res/httr}
354− ^- {(list move) _+>.$}
384+ ^- {(list move) _+>.$}
355− ?. ?=({$read care @ *} way)
385+ ?. ?=({$read care @ *} way)
356− ~& res=res
386+ ~& res=res
357− [~ +>.$]
387+ [~ +>.$]
358− =* style i.way
388+ =* style i.way
359− =* ren i.t.way
389+ =* ren i.t.way
360− =* pax t.t.way
390+ =* pax t.t.way
361− :_ +>.$ :_ ~
391+ :_ +>.$ :_ ~
362− :+ ost.hid %diff
392+ :+ ost.hid %diff
363− (sigh:connector (places ren style pax) ren pax res)
393+ (sigh:connector (places ren style pax) ren pax res)
364− ::
394+::
365− :: HTTP error. We just print it out, though maybe we should
395+:: HTTP error. We just print it out, though maybe we should
366− :: also produce a result so that the request doesn't hang?
396+:: also produce a result so that the request doesn't hang?
367− ::
397+::
368− ++ sigh-tang
398+++ sigh-tang
369− |= {way/wire tan/tang}
399+ |= {way/wire tan/tang}
370− ^- {(list move) _+>.$}
400+ ^- {(list move) _+>.$}
371− %- (slog >%gh-sigh-tang< tan)
401+ %- (slog >%gh-sigh-tang< tan)
372− [[ost.hid %diff null+~]~ +>.$]
402+ [[ost.hid %diff null+~]~ +>.$]
373− ::
403+::
374− :: We can't actually give the response to pretty much anything
404+:: We can't actually give the response to pretty much anything
375− :: without blocking, so we just block unconditionally.
405+:: without blocking, so we just block unconditionally.
376− ::
406+::
377− ++ peek
407+++ peek
378− |= {ren/@tas tyl/path}
408+ |= {ren/@tas tyl/path}
379− ^- (unit (unit (pair mark *)))
409+ ^- (unit (unit (pair mark *)))
380− ~ ::``noun/[ren tyl]
410+ ~ ::``noun/[ren tyl]
381− ::
411+::
382− ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
412+::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
383− :: Webhook-powered event streams (/listen) ::
413+:: Webhook-powered event streams (/listen) ::
384− ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
414+::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
385− ::
415+::
386− :: To listen to a webhook-powered stream of events, subscribe
416+:: To listen to a webhook-powered stream of events, subscribe
387− :: to /listen/<user>/<repo>/<events...>
417+:: to /listen/<user>/<repo>/<events...>
388− ::
418+::
389− :: We hand control to ++listen.
419+:: We hand control to ++listen.
390− ::
420+::
391− ++ peer-listen
421+++ peer-listen
392− |= pax/path
422+ |= pax/path
393− ^- {(list move) _+>.$}
423+ ^- {(list move) _+>.$}
394− ?. ?=({@ @ *} pax)
424+ ?. ?=({@ @ *} pax)
395− ~& [%bad-listen-path pax]
425+ ~& [%bad-listen-path pax]
396− [~ +>.$]
426+ [~ +>.$]
397− (listen pax)
427+ (listen pax)
398− ::
428+::
399− :: This core handles event subscription requests by starting or
429+:: This core handles event subscription requests by starting or
400− :: updating the webhook flow for each event.
430+:: updating the webhook flow for each event.
401− ::
431+::
402− ++ listen
432+++ listen
403− |= pax/path
433+ |= pax/path
404− =| mow/(list move)
434+ =| mow/(list move)
405− =< abet:listen
435+ =< abet:listen
406− |%
436+ |%
407− ++ abet :: Resolve core.
437+ ++ abet :: Resolve core.
408− ^- {(list move) _+>.$}
438+ ^- {(list move) _+>.$}
409− [(flop mow) +>.$]
439+ [(flop mow) +>.$]
410− ::
440+ ::
411− ++ send-hiss :: Send a hiss
441+ ++ send-hiss :: Send a hiss
412− |= hiz/hiss
442+ |= hiz/hiss
413− ^+ +>
443+ ^+ +>
414− =+ wir=`wire`[%x %listen pax]
444+ =+ wir=`wire`[%x %listen pax]
415− +>.$(mow [[ost.hid %hiss wir `~ %httr [%hiss hiz]] mow])
445+ +>.$(mow [[ost.hid %hiss wir `~ %httr [%hiss hiz]] mow])
416− ::
446+ ::
417− :: Create or update a webhook to listen for a set of events.
447+ :: Create or update a webhook to listen for a set of events.
418− ::
448+ ::
419− ++ listen
449+ ++ listen
420− ^+ .
450+ ^+ .
421− =+ pax=pax :: TMI-proofing
451+ =+ pax=pax :: TMI-proofing
422− ?> ?=({@ @ *} pax)
452+ ?> ?=({@ @ *} pax)
423− =+ events=t.t.pax
453+ =+ events=t.t.pax
424− |- ^+ +>+.$
454+ |- ^+ +>+.$
425− ?~ events
455+ ?~ events
426− +>+.$
456+ +>+.$
427− ?: (~(has by hook) i.events)
457+ ?: (~(has by hook) i.events)
428− $(+>+ (update-hook i.events), events t.events)
458+ $(+>+ (update-hook i.events), events t.events)
429− $(+>+ (create-hook i.events), events t.events)
459+ $(+>+ (create-hook i.events), events t.events)
430− ::
460+ ::
431− :: Set up a webhook.
461+ :: Set up a webhook.
432− ::
462+ ::
433− ++ create-hook
463+ ++ create-hook
434− |= event/@t
464+ |= event/@t
435− ^+ +>
465+ ^+ +>
436− ?> ?=({@ @ *} pax)
466+ ?> ?=({@ @ *} pax)
437− =+ clean-event=`tape`(turn (trip event) |=(a/@tD ?:(=('_' a) '-' a)))
467+ =+ clean-event=`tape`(turn (trip event) |=(a/@tD ?:(=('_' a) '-' a)))
438− =. hook
468+ =. hook
439− %+ ~(put by hook) (crip clean-event)
469+ %+ ~(put by hook) (crip clean-event)
440− =+ %+ fall
470+ =+ %+ fall
441− (~(get by hook) (crip clean-event))
471+ (~(get by hook) (crip clean-event))
442− *{id/@t listeners/(set bone)}
472+ *{id/@t listeners/(set bone)}
443− [id (~(put in listeners) ost.hid)]
473+ [id (~(put in listeners) ost.hid)]
444− %- send-hiss
474+ %- send-hiss
445− :* %+ scan
475+ :* %+ scan
446− =+ [(trip i.pax) (trip i.t.pax)]
476+ =+ [(trip i.pax) (trip i.t.pax)]
447− "https://api.github.com/repos/{-<}/{->}/hooks"
477+ "https://api.github.com/repos/{-<}/{->}/hooks"
448− auri:epur
478+ auri:epur
449− %post ~ ~
479+ %post ~ ~
450− %- taco %- crip %- pojo %- jobe :~
480+ %- taco %- crip %- pojo %- jobe :~
451− name+s+%web
481+ name+s+%web
452− active+b+&
482+ active+b+&
453− events+a+~[s+event] ::(turn `(list ,@t)`t.t.pax |=(a=@t s/a))
483+ events+a+~[s+event] ::(turn `(list ,@t)`t.t.pax |=(a=@t s/a))
454− :- %config
484+ :- %config
455− %- jobe :~
485+ %- jobe :~
456− =+ =+ clean-event
486+ =+ =+ clean-event
457− "http://107.170.195.5:8443/~/to/gh/gh-{-}.json?anon&wire=/"
487+ "http://107.170.195.5:8443/~/to/gh/gh-{-}.json?anon&wire=/"
458− [%url s+(crip -)]
488+ [%url s+(crip -)]
459− [%'content_type' s+%json]
489+ [%'content_type' s+%json]
460− ==
461− ==
462− ==
490+ ==
463− ::
464− :: Add current bone to the list of subscribers for this event.
465− ::
466− ++ update-hook
467− |= event/@t
468− ^+ +>
469− =+ hok=(~(got by hook) event)
470− %_ +>.$
471− hook
472− %+ ~(put by hook) event
473− hok(listeners (~(put in listeners.hok) ost.hid))
474 ==
491 ==
475− --
492+ ==
493+ ::
494+ :: Add current bone to the list of subscribers for this event.
495+ ::
496+ ++ update-hook
497+ |= event/@t
498+ ^+ +>
499+ =+ hok=(~(got by hook) event)
500+ %_ +>.$
501+ hook
502+ %+ ~(put by hook) event
503+ hok(listeners (~(put in listeners.hok) ost.hid))
504+ ==
505+ --
476− ::
506+::
477− :: Pokes that aren't caught in more specific arms are handled
507+:: Pokes that aren't caught in more specific arms are handled
478− :: here. These should be only from webhooks firing, so if we
508+:: here. These should be only from webhooks firing, so if we
479− :: get any mark that we shouldn't get from a webhook, we reject
509+:: get any mark that we shouldn't get from a webhook, we reject
480− :: it. Otherwise, we spam out the event to everyone who's
510+:: it. Otherwise, we spam out the event to everyone who's
481− :: listening for that event.
511+:: listening for that event.
482− ::
512+::
483− ++ poke
513+++ poke
484− |= response/hook-response
514+ |= response/hook-response
485− ^- {(list move) _+>.$}
515+ ^- {(list move) _+>.$}
486− =+ hook-data=(~(get by hook) (rsh 3 3 -.response))
516+ =+ hook-data=(~(get by hook) (rsh 3 3 -.response))
487− ?~ hook-data
517+ ?~ hook-data
488− ~& [%strange-hook hook response]
518+ ~& [%strange-hook hook response]
489− [~ +>.$]
519+ [~ +>.$]
490− :: ~& response=response
520+ :: ~& response=response
491− :_ +>.$
521+ :_ +>.$
492− %+ turn (~(tap in listeners.u.hook-data))
522+ %+ turn (~(tap in listeners.u.hook-data))
493− |= ost/bone
523+ |= ost/bone
494− [ost %diff response]
524+ [ost %diff response]
495− --
525+--
526+```
docs/arvo/basic.md+113−103
@@ -1,43 +1,49 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
4sort: 11
3sort: 11
4+next: true
5title: Basic Hoon
5title: Basic Hoon
6---
6---
7
7
8−# Basic Hoon
8+Basic Hoon
9+==========
9
10
10−Our goal is to get you programming interesting and useful things as soon as
11+Our goal is to get you programming interesting and useful things as soon
11−possible. To get there, we have to quickly cover some of the fundamentals of
12+as possible. To get there, we have to quickly cover some of the
12−[Hoon](/docs/hoon/). To do this, we'll walk through two simple programs: the
13+fundamentals of [Hoon](/docs/hoon/). To do this, we'll walk through two simple
13−first [Project Euler](https://projecteuler.net/) problem and
14+programs: the first [Project Euler](https://projecteuler.net/) problem
14−[fizzbuzz](https://en.wikipedia.org/wiki/Fizz_buzz).
15+and [fizzbuzz](https://en.wikipedia.org/wiki/Fizz_buzz).
15
16
16−To run this code, you'll need an urbit, and will have to copy the [examples
17+To run this code, you'll need an urbit, and will have to copy the
17−repo](https://github.com/urbit/examples) into it.
18+[examples repo](https://github.com/urbit/examples) into it.
18
19
19If you haven't installed urbit yet, check out the [installation
20If you haven't installed urbit yet, check out the [installation
20−instructions](/docs/using/install). Once urbit is installed, take a look at the
21+instructions](/docs/using/install). Once urbit is installed, take a look
21−[basic operation](/docs/using/admin) of your urbit.
22+at the [basic operation](/docs/using/admin) of your urbit.
22
23
23−If you haven't copied in the examples repo, running the following commands from
24+If you haven't copied in the examples repo, running the following
24−your urbit directory should do the trick:
25+commands from your urbit directory should do the trick:
25
26
26Make sure you've mounted your `%home` desk:
27Make sure you've mounted your `%home` desk:
27
28
29+```
28− ~fintud-macrep:dojo> |mount %
30+~fintud-macrep:dojo> |mount %
31+```
29
32
30Then, copy the example into it
33Then, copy the example into it
31
34
35+```
32− $ cp -r {urbit-examples}/gall/*/* {your-pier}/home/
36+$ cp -r {urbit-examples}/gall/*/* {your-pier}/home/
33− $ cp -r {urbit-examples}/dojo/*/* {your-pier}/home/
37+$ cp -r {urbit-examples}/dojo/*/* {your-pier}/home/
38+```
34
39
35Run an example to ensure it worked:
40Run an example to ensure it worked:
36
41
37 ~fintud-macrep:dojo> +examples-euler1
42 ~fintud-macrep:dojo> +examples-euler1
38 233.168
43 233.168
39
44
40−## Euler 1
45+Euler 1
46+-------
41
47
42Let's check out the code for Euler 1. First, the problem:
48Let's check out the code for Euler 1. First, the problem:
43
49
@@ -85,11 +91,11 @@ Here is the hoon solution (which should be in your pier directory under
85 (add (five a) (three a))
91 (add (five a) (three a))
86 --
92 --
87
93
88−> Hoon is not generally whitespace sensitive, but we do have two different kinds
94+> Hoon is not generally whitespace sensitive, but we do have two
89−> of whitespace: a single space and a gap, which is two or more spaces or a
95+> different kinds of whitespace: a single space and a gap, which is two
90−> linebreak. Tabs are taboo. Do not use them. Really. For a more detailed
96+> or more spaces or a linebreak. Tabs are taboo. Do not use them.
97+> Really. For a more detailed explanation of when to use spaces vs.
91−> explanation of when to use spaces vs. gaps, see the syntax section before the
98+> gaps, see the syntax section before the first exercises.
92−> first exercises.
93
99
94### Lines 1-11:
100### Lines 1-11:
95
101
@@ -100,29 +106,31 @@ Any line that begins with `::` is a comment.
100 =< (sum 1.000)
106 =< (sum 1.000)
101
107
102All you need to know about the lines above is that they call the `++sum`
108All you need to know about the lines above is that they call the `++sum`
103−function with an argument of `1.000`. We'll cover them in more detail later.
109+function with an argument of `1.000`. We'll cover them in more detail
110+later.
104
111
105### How to form expressions
112### How to form expressions
106
113
107Hoon does not typically use reserved words to form expressions. Instead,
114Hoon does not typically use reserved words to form expressions. Instead,
108−expressions are formed with *runes*: digraphs of two ASCII symbols. Each rune
115+expressions are formed with *runes*: digraphs of two ASCII symbols. Each
109−takes a specific number of children, either expressions formed by other runes or
116+rune takes a specific number of children, either expressions formed by
110−literals that produce their own value (some runes take *N* children, and are
117+other runes or literals that produce their own value (some runes take
111−usually closed with `==`).
118+*N* children, and are usually closed with `==`).
112
119
113−For example, the rune `?:` from line 18 is the classic 'if-then-else' statement,
120+For example, the rune `?:` from line 18 is the classic 'if-then-else'
114−and thus takes three children:
121+statement, and thus takes three children:
115
122
116 ?: (lth a b) :: if first child evals to true
123 ?: (lth a b) :: if first child evals to true
117 0 :: then produce result of second
124 0 :: then produce result of second
118 (add b $(b (add 3 b))) :: else, produce result of third
125 (add b $(b (add 3 b))) :: else, produce result of third
119
126
120Since runes are such a fundamental structure in Hoon, we found ourselves
127Since runes are such a fundamental structure in Hoon, we found ourselves
121−speaking them out loud frequently. It quickly grew cumbersome to have to say
128+speaking them out loud frequently. It quickly grew cumbersome to have to
122−"question mark, colon" to describe `?:`. To alleviate this problem, we came up
129+say "question mark, colon" to describe `?:`. To alleviate this problem,
123−with our own naming scheme: each ASCII glyph has a single-syllable pronunciation
130+we came up with our own naming scheme: each ASCII glyph has a
124−phonetically designed to be both easily remembered and easily pronounced in
131+single-syllable pronunciation phonetically designed to be both easily
125−conjunction with the other glyphs (when forming a rune).
132+remembered and easily pronounced in conjunction with the other glyphs
133+(when forming a rune).
126
134
127See the entire naming scheme below.
135See the entire naming scheme below.
128
136
@@ -142,8 +150,9 @@ See the entire naming scheme below.
142Using this scheme, we would pronounce `?:` as 'wutcol'.
150Using this scheme, we would pronounce `?:` as 'wutcol'.
143
151
144For those who prefer to use reserved words to form expressions, you can
152For those who prefer to use reserved words to form expressions, you can
145−substitute any rune with that rune's keyword, which you can find in `++twig` in
153+substitute any rune with that rune's keyword, which you can find in
146−`/arvo/hoon.hoon`. For example, here's the code for `?:` in `++twig`:
154+`++twig` in `/arvo/hoon.hoon`. For example, here's the code for `?:` in
155+`++twig`:
147
156
148`{$if p/twig q/twig r/twig} :: ?: if/then/else`
157`{$if p/twig q/twig r/twig} :: ?: if/then/else`
149
158
@@ -159,16 +168,16 @@ You can write:
159
168
160### Lines 13-35
169### Lines 13-35
161
170
162−Now let's quickly walk through this code line-by-line. Lines 13-35 are wrapped
171+Now let's quickly walk through this code line-by-line. Lines 13-35 are
163−in a `|%` ('[barcen](../../hoon/twig/bar-core/cen-core/)'), which produces a
172+wrapped in a `|%` ('[barcen](../../hoon/twig/bar-core/cen-core/)'), which produces a core. Cores are a
164−core. Cores are a fundamental datatype in Hoon, similar to a struct, class, or
173+fundamental datatype in Hoon, similar to a struct, class, or object. A
165−object. A core is just a map of names to any kind of code, whether it be
174+core is just a map of names to any kind of code, whether it be functions
166−functions or data. Each element in this map begins with a `++` followed by the
175+or data. Each element in this map begins with a `++` followed by the
167name and the corresponding code. Since `|%` takes an arbitrary number of
176name and the corresponding code. Since `|%` takes an arbitrary number of
168children, it needs to be closed with a `--`.
177children, it needs to be closed with a `--`.
169
178
170−> `++` ('luslus') is not technically a rune, since it is only used in core
179+> `++` ('luslus') is not technically a rune, since it is only used in
171−> syntax, as shown above.
180+> core syntax, as shown above.
172
181
173Let's step into each of the three arms within our core.
182Let's step into each of the three arms within our core.
174
183
@@ -179,16 +188,17 @@ Let's step into each of the three arms within our core.
179 (add (five a) (three a))
188 (add (five a) (three a))
180 --
189 --
181
190
182−`|=` ('[bartis](../../hoon/twig/bar-core/tis-gate/)') produces a function, much
191+`|=` ('[bartis](../../hoon/twig/bar-core/tis-gate/)') produces a function, much like a lambda in lisp. It
183−like a lambda in lisp. It takes two children:
192+takes two children:
184
193
185−1. A set of argument(s). In this case our argument set only contains one: `a`,
194+1. A set of argument(s). In this case our argument set only contains
186− which is required to be an atom or natural number, denoted by `@`.
195+ one: `a`, which is required to be an atom or natural number, denoted
196+ by `@`.
187
197
188−2. The body of the function itself, which is executed when the function is
198+2. The body of the function itself, which is executed when the function
189− called (in this case, with `(sum 1.000)`). This particular function adds the
199+ is called (in this case, with `(sum 1.000)`). This particular
190− results of evaluating the gates `++ five` and `++three` with each of their
200+ function adds the results of evaluating the gates `++ five` and
191− respective input parameters set to `a`.
201+ `++three` with each of their respective input parameters set to `a`.
192
202
193### ++ three
203### ++ three
194
204
@@ -200,17 +210,17 @@ like a lambda in lisp. It takes two children:
200 0
210 0
201 (add b $(b (add 3 b)))
211 (add b $(b (add 3 b)))
202
212
203−As above, `++three` takes an integer argument, `a`, and then executes the
213+As above, `++three` takes an integer argument, `a`, and then executes
204−remainder of the code with `a` set to the actual arguments.
214+the remainder of the code with `a` set to the actual arguments.
205
215
206−Similarly, `=|` ('[tisbar](../../hoon/twig/tis-flow/bar-new/)') pushes its first
216+Similarly, `=|` ('[tisbar](../../hoon/twig/tis-flow/bar-new/)') pushes its first child, `b` into our context
207−child, `b` into our context (in other words, it declares a variable `b`) and
217+(in other words, it declares a variable `b`) and executes the remainder
208−executes the remainder of the code. However, `b` is not an argument; `=|` sets
218+of the code. However, `b` is not an argument; `=|` sets `b` to the
209−`b` to the default value of whatever type it is declared as. Since the default
219+default value of whatever type it is declared as. Since the default
210value of an atom is `0`, b is set to `0`.
220value of an atom is `0`, b is set to `0`.
211
221
212−> To produce the default value of any given type, use `*` ('tar') followed by
222+> To produce the default value of any given type, use `*` ('tar')
213−> the type. This operaction is called *bunt*.
223+> followed by the type. This operaction is called *bunt*.
214>
224>
215> > *@ :: produce the default value of atom @
225> > *@ :: produce the default value of atom @
216> 0
226> 0
@@ -218,24 +228,25 @@ value of an atom is `0`, b is set to `0`.
218> > *@p :: produce the phonemic base of atom @
228> > *@p :: produce the phonemic base of atom @
219> ~zod
229> ~zod
220
230
221−So now we have two variables: `a` is set to our input, and `b` is initialized to
231+So now we have two variables: `a` is set to our input, and `b` is
222−`0`.
232+initialized to `0`.
223
233
224−One way to think about `|-` ('[barhep](../../hoon/twig/bar-core/hep-loop/)') is
234+One way to think about `|-` ('[barhep](../../hoon/twig/bar-core/hep-loop/)') is that it lays down a recursion
225−that it lays down a recursion point. More on this later.
235+point. More on this later.
226
236
227−`^-` ('[kethep](../../hoon/core/ket-cast/hep-cast/)') is just a cast that sets
237+`^-` ('[kethep](../../hoon/core/ket-cast/hep-cast/)') is just a cast that sets the result of the remainder of
228−the result of the remainder of the code to an unsigned integer, `@u`.
238+the code to an unsigned integer, `@u`.
229
239
230−In pseudocode, the last three lines read like this: if `a` is less than `b`,
240+In pseudocode, the last three lines read like this: if `a` is less than
231−produce zero. Else, add `b` to the result of rerunning the segment of the
241+`b`, produce zero. Else, add `b` to the result of rerunning the segment
232−function following the `|-` with the value of `b` changed to `b` plus three.
242+of the function following the `|-` with the value of `b` changed to `b`
243+plus three.
233
244
234The only thing that should look completely unfamiliar to you here is the
245The only thing that should look completely unfamiliar to you here is the
235−`$(b (add 3 b))`, which causes us to recurse back to our last recursion point
246+`$(b (add 3 b))`, which causes us to recurse back to our last recursion
236−with the value of `b` set to `(add 3 b)`. Note that we only specify what changes
247+point with the value of `b` set to `(add 3 b)`. Note that we only
237−(`b` in this case). If you recurse by an actual function call, then you have to
248+specify what changes (`b` in this case). If you recurse by an actual
238−specify every argument.
249+function call, then you have to specify every argument.
239
250
240> If you're familiar with Clojure, `|-` is `loop` and `$()` is `recur`.
251> If you're familiar with Clojure, `|-` is `loop` and `$()` is `recur`.
241
252
@@ -243,18 +254,18 @@ specify every argument.
243
254
244Tweak your code to complete the following exercises.
255Tweak your code to complete the following exercises.
245
256
246−There are a few runes and some syntax that we have yet to cover that you will
257+There are a few runes and some syntax that we have yet to cover that you
247−need to complete the exercises below. For these, please refer to our cheatsheat
258+will need to complete the exercises below. For these, please refer to
248−at the bottom.
259+our cheatsheat at the bottom.
249
260
2501. Read and understand `++five` line by line.
2611. Read and understand `++five` line by line.
251
262
252−2. Change `++sum` to accept two variables, `a` and `b`. Pass `a` to `three` and
263+2. Change `++sum` to accept two variables, `a` and `b`. Pass `a` to
253− `b` to `five`. Then run the code with `a` set to `1.000` and `b` set to
264+ `three` and `b` to `five`. Then run the code with `a` set to `1.000`
254− `2.000`.
265+ and `b` set to `2.000`.
255
266
256−3. Check if this new result is under one thousand. If it is, return the string
267+3. Check if this new result is under one thousand. If it is, return the
257− `'result < 1.000'`. If not, return `'result >= 1.000'`.
268+ string `'result < 1.000'`. If not, return `'result >= 1.000'`.
258
269
259### Review
270### Review
260
271
@@ -267,40 +278,39 @@ at the bottom.
267 =(a b) test equality
278 =(a b) test equality
268 (function args ...) call function with args
279 (function args ...) call function with args
269
280
270−Lookup each of these expressions (and all others!) in the [Twig
281+Lookup each of these expressions (and all others!) in the [Twig Expressions](../../hoon/twig/) definition.
271−Expressions](../../hoon/twig/) definition.
272
282
273### New material
283### New material
274
284
2751. There are two syntaxes for writing Hoon: tallform and wideform.
2851. There are two syntaxes for writing Hoon: tallform and wideform.
276
286
277− In tallform, expressions are formed with either two spaces or a line break
287+ In tallform, expressions are formed with either two spaces or a line
278− separating both a rune from its children and each of its children from one
288+ break separating both a rune from its children and each of its children
279− another. We use tallform when writing multiline expressions.
289+ from one another. We use tallform when writing multiline expressions.
280
290
281 For more concise expressions, we use wideform, which is always a single
291 For more concise expressions, we use wideform, which is always a single
282− line. Wideform can be used inside tallform expressions, but not vice versa.
292+ line. Wideform can be used inside tallform expressions, but not vice
293+ versa.
283
294
284− Wideform expressions are formed with a rune followed by `()` containing its
295+ Wideform expressions are formed with a rune followed by `()` containing
285− children, all of which are separated by a single space. For example, to make
296+ its children, all of which are separated by a single space. For example,
286− a cell of two elements, use `:-(a b)`.
297+ to make a cell of two elements, use `:-(a b)`.
287
298
288− We've already seen wideform in action, for example with `=((mod b 3) 0)`. In
299+ We've already seen wideform in action, for example with
289− this case, `=` is actually an irregular form of `.=`, which tests its two
300+ `=((mod b 3) 0)`. In this case, `=` is actually an irregular form of
290− children for equality.
301+ `.=`, which tests its two children for equality.
291
302
292 Surrounding a function with `()` is an irregular wide form syntax for
303 Surrounding a function with `()` is an irregular wide form syntax for
293 calling a function with *N* arguments. More on this later.
304 calling a function with *N* arguments. More on this later.
294
305
295−2. For a set of multiple arguments following `|=`
296− ('[bartis](../../hoon/twig/bar-core/tis-gate/)'), use `{` 'kel' and `}`
306+2. For a set of multiple arguments following `|=` ('[bartis](../../hoon/twig/bar-core/tis-gate/)'), use `{` 'kel' and `}` 'ker.' For example: `{a/@u b/@u}`.
297− 'ker.' For example: `{a/@u b/@u}`.
298
307
299−3. `:-` makes a cell of values. The irregular wide form of this is `[a b]`,
308+3. `:-` makes a cell of values. The irregular wide form of this is
300− with two expressions separated by a single space. While the regular form of
309+ `[a b]`, with two expressions separated by a single space. While the
301− this rune takes a fixed number of children (two), its irregular wide form
310+ regular form of this rune takes a fixed number of children (two),
302− can accept *N* expressions: `[a1..an]`.
311+ its irregular wide form can accept *N* expressions: `[a1..an]`.
303
312
304−4. Cords are one datatype for text in Hoon. They're just a big atom formed from
313+4. Cords are one datatype for text in Hoon. They're just a big atom
305− adjacent unicode bytes -- a "c string". To produce a cord, enclose text
314+ formed from adjacent unicode bytes -- a "c string". To produce a
306− within single quotes. To set the type of an argument to a cord, use `@t`.
315+ cord, enclose text within single quotes. To set the type of an
316+ argument to a cord, use `@t`.
docs/arvo/generators.md+86−76
@@ -7,60 +7,66 @@ title: Using generators with apps
7
7
8# Generators
8# Generators
9
9
10−Up until now we've poked apps directly. This requires the user to specify the
10+Up until now we've poked apps directly. This requires the user to
11−mark, and it requires the app to accept the arguments in a way that's convenient
11+specify the mark, and it requires the app to accept the arguments in a
12−for users to input. This is the "plumbing" way to interact with apps. Generators
12+way that's convenient for users to input. This is the "plumbing" way to
13−are the "porcelain" layer. This is why when you run a command like `+ls` or
13+interact with apps. Generators are the "porcelain" layer. This is why
14−`|merge`, there are no marks in sight.
14+when you run a command like `+ls` or `|merge`, there are no marks in
15−
15+sight.
16+
16−We've used generators before, back in [Basic Operation](/docs/using/admin). At
17+We've used generators before, back in [Basic
18+Operation](/docs/using/admin). At that point, we just used the
17−that point, we just used the generators to produce values -- we didn't pipe
19+generators to produce values -- we didn't pipe their results into apps.
18−their results into apps. In the dojo cast, the role of a generator is to take a
20+In the dojo cast, the role of a generator is to take a list of arguments
19−list of arguments and produce a value, which is often, though not always, piped
21+and produce a value, which is often, though not always, piped into an
20−into an app. Generators are pure, stateless functions, and they cannot send
22+app. Generators are pure, stateless functions, and they cannot send
21moves.
23moves.
22
24
23−`+ls` and `+cat` are commonly-used generators that produce a directory listing
25+`+ls` and `+cat` are commonly-used generators that produce a directory
24−and print out a file, respectively. These are useful in themselves, so we
26+listing and print out a file, respectively. These are useful in
25−generally don't pipe the results into an app.
27+themselves, so we generally don't pipe the results into an app.
26
28
27−Another generator is `+hood/merge`. Try running `+hood/merge` with the normal
29+Another generator is `+hood/merge`. Try running `+hood/merge` with the
28−arguments for `|merge`:
30+normal arguments for `|merge`:
29
31
30 ~fintud-macrep:dojo> +hood/merge %home our %they
32 ~fintud-macrep:dojo> +hood/merge %home our %they
31 [syd=%home her=~fintud-macrep sud=%they gem=%auto]
33 [syd=%home her=~fintud-macrep sud=%they gem=%auto]
32 ~fintud-macrep:dojo> +hood/merge %home our %they, =gem %this
34 ~fintud-macrep:dojo> +hood/merge %home our %they, =gem %this
33 [syd=%home her=~fintud-macrep sud=%they gem=%auto]
35 [syd=%home her=~fintud-macrep sud=%they gem=%auto]
34
36
35−This didn't run any merge, it just constructed the command that, if sent to
37+This didn't run any merge, it just constructed the command that, if sent
36−`:hood`, would run it. Note first that, even though it's not printed, this value
38+to `:hood`, would run it. Note first that, even though it's not printed,
37−has mark `kiln-merge`. Also, the merge strategy `gem=%auto` was added
39+this value has mark `kiln-merge`. Also, the merge strategy `gem=%auto`
38−automatically. Optional arguments are straightforward with generators.
40+was added automatically. Optional arguments are straightforward with
39−
41+generators.
42+
40−In general, generators are prepended by either `+` or `|`. The general form is
43+In general, generators are prepended by either `+` or `|`. The general
41−`+generator`, but often generators are created specifically to work with a
44+form is `+generator`, but often generators are created specifically to
42−single app, they're usually placed in a subdirectory of gen and run with `|`.
45+work with a single app, they're usually placed in a subdirectory of gen
43−Thus, if `my-generator` generator is made for use with `:my-app`, then
46+and run with `|`. Thus, if `my-generator` generator is made for use with
47+`:my-app`, then `my-generator` is put in
44−`my-generator` is put in `/gen/my-app/my-generator.hoon`, and can be run
48+`/gen/my-app/my-generator.hoon`, and can be run directly with
45−directly with `+my-app/my-generator <args>`. If you want to pipe the result into
49+`+my-app/my-generator <args>`. If you want to pipe the result into
46−`:my-app`, then run `:my-app +my-app/my-generator <args>`. Because this pattern
50+`:my-app`, then run `:my-app +my-app/my-generator <args>`. Because this
47−is so common, this can be abbreviated to `:my-app|my-generator <args>`. Because
51+pattern is so common, this can be abbreviated to
52+`:my-app|my-generator <args>`. Because most built-in commands are
48−most built-in commands are generators for the `:hood` app,
53+generators for the `:hood` app, `:hood|generator <args>` can be
49−`:hood|generator <args>` can be shortened to `|generator <args>`.
54+shortened to `|generator <args>`.
50−
55+
51−Let's write a generator for a modified version of `:examples-pong` from the
56+Let's write a generator for a modified version of `:examples-pong` from
52−chapter on [Network Messages](/docs/arvo/system/network). Recall that
57+the chapter on [Network Messages](/docs/arvo/system/network). Recall
53−`:examples-pong` takes an urbit address, which is of mark `urbit`, and sends
58+that `:examples-pong` takes an urbit address, which is of mark `urbit`,
54−that urbit the message `'howdy'`. First--without a generator--let's make
59+and sends that urbit the message `'howdy'`. First--without a
55−`:examples-ping` that does the same, except that it lets the user optionally
60+generator--let's make `:examples-ping` that does the same, except that
56−specify the message as well.
61+it lets the user optionally specify the message as well.
57−
62+
58−We'll need a new mark for our arguments. Let's call it `examples-ping-message`.
63+We'll need a new mark for our arguments. Let's call it
59−
64+`examples-ping-message`.
65+
60−> For app-specific marks, it's good style to prefix the name of the mark with
66+> For app-specific marks, it's good style to prefix the name of the mark
61−> the name of the app. Since many apps have several such marks, subdirectories
67+> with the name of the app. Since many apps have several such marks,
62−> in `/mar` are rendered as `-`, so that `ping-message` is written in
68+> subdirectories in `/mar` are rendered as `-`, so that `ping-message`
63−> (/mar/examples/ping/message.hoon).
69+> is written in (/mar/examples/ping/message.hoon).
64
70
65 ::
71 ::
66 :::: /hoon/message/ping/examples/app
72 :::: /hoon/message/ping/examples/app
@@ -113,40 +119,43 @@ Now we can run this with:
113 ~fintud-macrep:dojo> :examples-ping &examples-ping-message [~sampel-sipnym 'heyya']
119 ~fintud-macrep:dojo> :examples-ping &examples-ping-message [~sampel-sipnym 'heyya']
114 >=
120 >=
115
121
116−And on `~sampel-sipnym`, assuming it's running `:examples-ping` as well, we see
122+And on `~sampel-sipnym`, assuming it's running `:examples-ping` as well,
117−`[%receiving 'heyya']`.
123+we see `[%receiving 'heyya']`.
118
124
119−This is an annoying way to invoke the app, though. The `[]` are mandatory, and
125+This is an annoying way to invoke the app, though. The `[]` are
120−optional arguments are hard. Let's make a generator, `+examples-send` to make
126+mandatory, and optional arguments are hard. Let's make a generator,
121−everything nicer. Since it's specific to `:examples-ping`, let's put it in
127+`+examples-send` to make everything nicer. Since it's specific to
122−`/gen/examples/ping/send.hoon`:
128+`:examples-ping`, let's put it in `/gen/examples/ping/send.hoon`:
123
129
124 :- %say
130 :- %say
125 |= {^ {to/@p message/?($~ {text/@t $~})} $~}
131 |= {^ {to/@p message/?($~ {text/@t $~})} $~}
126 [%examples-ping-message to ?~(message 'howdy' text.message)]
132 [%examples-ping-message to ?~(message 'howdy' text.message)]
127
133
128−A couple of new things here. Firstly, `message/?($~ {text/@t $~})` should be
134+A couple of new things here. Firstly, `message/?($~ {text/@t $~})`
129−read as "the message is either null or a pair of text and null". Generator
135+should be read as "the message is either null or a pair of text and
130−argument lists are always null-terminated, which makes it convenient to accept
136+null". Generator argument lists are always null-terminated, which makes
131−lists in tail position (which are particularly annoying without generators).
137+it convenient to accept lists in tail position (which are particularly
138+annoying without generators). `?(a b)` is the irregular form of
132−`?(a b)` is the irregular form of `$?(a b)`, which is a union type. Thus,
139+`$?(a b)`, which is a union type. Thus, `?(a b)` means the type of
133−`?(a b)` means the type of anything that's in either `a` or `b`. Thus, in our
140+anything that's in either `a` or `b`. Thus, in our case,
134−case, `?(~ [text=@t ~])` is either null or a pair of text and null.
141+`?(~ [text=@t ~])` is either null or a pair of text and null.
135−
142+
136−Secondly, `?~(a b c)` is a rune which means "if the `a` is null, do `b`, else
143+Secondly, `?~(a b c)` is a rune which means "if the `a` is null, do `b`,
137−`c`". It is roughly equivalent to `?:(=(~ a) b c)`.
144+else `c`". It is roughly equivalent to `?:(=(~ a) b c)`.
138−
145+
139−> `?~(a b c)` is actually equivalent to `?:=(?=(~ a) b c)`, which, although
146+> `?~(a b c)` is actually equivalent to `?:=(?=(~ a) b c)`, which,
140−> identical at run time, is subtly different at compile time. Specifically,
147+> although identical at run time, is subtly different at compile time.
141−> using `?=` rather than `=` means that we're checking whether `a` is in the
148+> Specifically, using `?=` rather than `=` means that we're checking
142−> *type* of `~`, and so the compiler knows that in the `b` case `a` is null, and
149+> whether `a` is in the *type* of `~`, and so the compiler knows that in
143−> in the `c` case `a` is not null. Since `=` is purely a runtime value check
150+> the `b` case `a` is null, and in the `c` case `a` is not null. Since
151+> `=` is purely a runtime value check with no type implications, the
144−> with no type implications, the compiler doesn't gain any information.
152+> compiler doesn't gain any information.
145>
153>
146−> At any rate, this is the reason why we can refer to `text.message` in the `c`
154+> At any rate, this is the reason why we can refer to `text.message` in
147−> clause. The compiler knows that `message` is not null, so it must have `text`
155+> the `c` clause. The compiler knows that `message` is not null, so it
156+> must have `text` within it. If you used
148−> within it. If you used `?:(=(~ message) 'howdy' text.message)` the compiler
157+> `?:(=(~ message) 'howdy' text.message)` the compiler would complain
149−> would complain that it doesn't know whether `message` has `text` within it.
158+> that it doesn't know whether `message` has `text` within it.
150
159
151This is run as follows:
160This is run as follows:
152
161
@@ -160,9 +169,10 @@ Which causes `~sampel-sipnym` to print `[%receiving 'howdy']` and
160
169
161**Exercises**:
170**Exercises**:
162
171
163−- Create a generator for `:examples-sum` from [State](/docs/arvo/state) so
172+- Create a generator for `:examples-sum` from
164− that you can run `:examples-sum|add 5` to add numbers to it.
173+ [State](/docs/arvo/state) so that you can run `:examples-sum|add 5`
174+ to add numbers to it.
165
175
166- Create a generator for `:examples-click` from [Web
176- Create a generator for `:examples-click` from [Web
167− Apps](/docs/arvo/web-apps) so that you can run `:examples-click|poke` to
177+ Apps](/docs/arvo/web-apps) so that you can run
168− poke it.
178+ `:examples-click|poke` to poke it.
docs/arvo/http.md+208−192
@@ -1,214 +1,230 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: False
3+next: false
4sort: 19
4sort: 19
5title: HTTP requests and timers
5title: HTTP requests and timers
6---
6---
7
7
8# Writing an HTTP request
8# Writing an HTTP request
9
9
10−There's a variety of arvo services we haven't touched on yet. Let's figure out
10+There's a variety of arvo services we haven't touched on yet.
11−how to make HTTP requests and set timers.
11+Let's figure out how to make HTTP requests and set timers.
12−
12+
13−Let's build an uptime-monitoring system. It'll ping a web service periodically
13+Let's build an uptime-monitoring system. It'll ping a web
14−and print out an error when the response code isn't 2xx. Here's `app/up.hoon`:
14+service periodically and print out an error when the response
15−
15+code isn't 2xx. Here's `app/up.hoon`:
16+
17+```
16− :: Up-ness monitor. Accepts atom url, 'on', or 'off'
18+:: Up-ness monitor. Accepts atom url, 'on', or 'off'
17− ::
19+::
18− :::: /hoon/up/examples/app
20+:::: /hoon/up/examples/app
19− ::
21+ ::
20− /? 314
22+/? 314
21− |%
23+|%
22− ++ move {bone card}
24+++ move {bone card}
23− ++ card
25+++ card
24− $% {$hiss wire unit-iden/$~ mark/$httr cage/{mark/$purl vase/purl}}
26+ $% {$hiss wire unit-iden/$~ mark/$httr cage/{mark/$purl vase/purl}}
25− {$wait wire @da}
27+ {$wait wire @da}
26− ==
28+ ==
27− ++ action
29+++ action
28− $% {$on $~} :: enable polling('on')
30+ $% {$on $~} :: enable polling('on')
29− {$off $~} :: disable polling('off)
31+ {$off $~} :: disable polling('off)
30− {$target p/cord} :: set poll target('http://...')
32+ {$target p/cord} :: set poll target('http://...')
31− ==
33+ ==
32− --
34+--
33− |_ {hid/bowl on/_| in-progress/_| target/@t}
35+|_ {hid/bowl on/_| in-progress/_| target/@t}
34− ++ poke-atom
36+++ poke-atom
35− |= url-or-command/@t ^- (quip move +>)
37+ |= url-or-command/@t ^- (quip move +>)
36− =+ ^- act/action
38+ =+ ^- act/action
37− ?: ?=($on url-or-command) [%on ~]
39+ ?: ?=($on url-or-command) [%on ~]
38− ?: ?=($off url-or-command) [%off ~]
40+ ?: ?=($off url-or-command) [%off ~]
39− [%target url-or-command]
41+ [%target url-or-command]
40− ?- -.act
42+ ?- -.act
41− $target [~ +>.$(target p.act)]
43+ $target [~ +>.$(target p.act)]
42− $off [~ +>.$(on |)]
44+ $off [~ +>.$(on |)]
43− $on
45+ $on
44− :- ?: |(on in-progress) ~
46+ :- ?: |(on in-progress) ~
45− [ost.hid %hiss /request ~ %httr %purl (need (epur target))]~
47+ [ost.hid %hiss /request ~ %httr %purl (need (epur target))]~
46− +>.$(on &, in-progress &)
48+ +>.$(on &, in-progress &)
47− ==
49+ ==
48− ++ sigh-httr
50+++ sigh-httr
49− |= {wir/wire code/@ud headers/mess body/(unit octs)}
51+ |= {wir/wire code/@ud headers/mess body/(unit octs)}
50− ~& 'arrive here'
52+ ~& 'arrive here'
51− ^- {(list move) _+>.$}
53+ ^- {(list move) _+>.$}
52− ?: &((gte code 200) (lth code 300))
54+ ?: &((gte code 200) (lth code 300))
53− ~& [%all-is-well code]
55+ ~& [%all-is-well code]
54− :_ +>.$
56+ :_ +>.$
55− [ost.hid %wait /timer (add ~s10 now.hid)]~
57+ [ost.hid %wait /timer (add ~s10 now.hid)]~
56− ~& [%we-have-a-problem code]
58+ ~& [%we-have-a-problem code]
57− ~& [%headers headers]
59+ ~& [%headers headers]
58− ~& [%body body]
60+ ~& [%body body]
59− :_ +>.$
61+ :_ +>.$
60− [ost.hid %wait /timer (add ~s10 now.hid)]~
62+ [ost.hid %wait /timer (add ~s10 now.hid)]~
61− ++ wake-timer
63+++ wake-timer
62− |= {wir/wire $~} ^- (quip move +>)
64+ |= {wir/wire $~} ^- (quip move +>)
63− ?: on
65+ ?: on
64− :_ +>.$
66+ :_ +>.$
65− [ost.hid %hiss /request ~ %httr %purl (need (epur target))]~
67+ [ost.hid %hiss /request ~ %httr %purl (need (epur target))]~
66− [~ +>.$(in-progress |)]
68+ [~ +>.$(in-progress |)]
67− ::
69+::
68− ++ prep ~& target _`. :: computed when the source file changes;
70+++ prep ~& target _`. :: computed when the source file changes;
69− -- :: here it prints target then resets our state
71+-- :: here it prints target then resets our state
70−
72+```
73+
71−There's some fancy stuff going on here. We'll go through it line by line,
74+There's some fancy stuff going on here. We'll go through it line by
72−though.
75+line, though.
73−
76+
74−There's two kinds of cards we're sending to arvo. A `%hiss` move tells `%eyre`
77+There's two kinds of cards we're sending to arvo. A `%hiss` move tells
75−to make an HTTP request. It expects a `(unit iden)`, which will be null here
78+`%eyre` to make an HTTP request. It expects a `(unit iden)`, which will
76−because we aren't doing any authentication; a mark, in this case the http
79+be null here because we aren't doing any authentication; a mark, in this
77−request mark `&httr`; and some data in the form of a `cage`, which is a
80+case the http request mark `&httr`; and some data in the form of a
78−`{mark vase}`. This is confusing, so the mold in the code quoted above includes
81+`cage`, which is a `{mark vase}`. This is confusing, so the mold in the
79−a bunch of otherwise unnecessary faces for the purpose of illustration. Compare:
82+code quoted above includes a bunch of otherwise unnecessary faces for
83+the purpose of illustration. Compare: `{$hiss p/(unit iden) q/mark
80−`{$hiss p/(unit iden) q/mark r/cage}` from `zuse`.
84+r/cage}` from `zuse`.
81−
85+
82−> You can grep zuse.hoon and arvo.hoon for most of these definitions. However,
86+> You can grep zuse.hoon and arvo.hoon for most of these definitions.
83−> beware the ambiguity surrounding "hiss": there's ++hiss in section 3bI of
87+> However, beware the ambiguity surrounding "hiss": there's ++hiss in
84−> `zuse` ("Arvo structures"), and there's `%hiss` the move that is sent to %eyre
88+> section 3bI of `zuse` ("Arvo structures"), and there's `%hiss` the
85−> to make our HTTP request happen. Here we're talking about the latter. See
89+> move that is sent to %eyre to make our HTTP request happen. Here we're
86−> `++kiss-eyre` in `zuse`.
90+> talking about the latter. See `++kiss-eyre` in `zuse`.
87−
91+
88−> Recall that `++unit` means "maybe". Formally, "`(unit a)` is either null or a
92+> Recall that `++unit` means "maybe". Formally, "`(unit a)` is
89−> pair of null and a value in `a`". Also recall that to pull a value out of a
93+> either null or a pair of null and a value in `a`". Also recall
94+> that to pull a value out of a unit (`u.unit`), you must first
90−> unit (`u.unit`), you must first verify that the unit is not null (for example
95+> verify that the unit is not null (for example with `?~`). See
91−> with `?~`). See `++need`.
96+> `++need`.
92−
97+
93−A `purl` is a parsed url structure, which can be created with `++epur`, which is
98+A `purl` is a parsed url structure, which can be created with `++epur`,
94−a function that takes a url as text and parses it into a `(unit purl)`. Thus,
99+which is a function that takes a url as text and parses it into a `(unit
95−the result is null if and only if the url is malformed. 'Purl' is also the mark
100+purl)`. Thus, the result is null if and only if the url is malformed.
96−which will be applied to this result.
101+'Purl' is also the mark which will be applied to this result.
97−
102+
98−When you send this request, you can expect a `%sigh` with the response, which we
103+When you send this request, you can expect a `%sigh` with the
99−handle later on in `++sigh-httr`.
104+response, which we handle later on in `++sigh-httr`.
100−
105+
101−For `%wait`, you just pass a [`@da`]() (absolute date), and arvo will produce a
106+For `%wait`, you just pass a [`@da`]() (absolute date), and arvo will
102−`%wake` when the time comes.
107+produce a `%wake` when the time comes.
103−
108+
104−> A timer is guaranteed to not be triggered before the given time, but it's
109+> A timer is guaranteed to not be triggered before the given
105−> currently impossible to guarantee the timer will be triggered at exactly the
110+> time, but it's currently impossible to guarantee the timer will be
106−> requested time.
111+> triggered at exactly the requested time.
107
112
108Let's take a look at our state:
113Let's take a look at our state:
109
114
115+```
110− |_ {hid/bowl on/_| in-progress=_| target/@t}
116+|_ {hid/bowl on/_| in-progress=_| target/@t}
117+```
111
118
112−We have three pieces of app-specific state. `target` is the url we're
119+We have three pieces of app-specific state. `target` is the url
113−monitoring. `on` and `in-progress` are booleans representing, respectively,
120+we're monitoring. `on` and `in-progress` are booleans
121+representing, respectively, whether or not we're supposed to keep
114−whether or not we're supposed to keep monitoring and whether or not we're in the
122+monitoring and whether or not we're in the middle of a request.
115−middle of a request.
116
123
117−The type of booleans is usually written as `?`, which is a union of `&` (true)
124+The type of booleans is usually written as `?`, which is a union
118−and `|` false. This type defaults to `&` (true). In our case, we want both of
125+of `&` (true) and `|` false. This type defaults to `&` (true). In our
119−these to default to `|` (false). Because of that, we use `_|` as the type.
126+case, we want both of these to default to `|` (false). Because
127+of that, we use `_|` as the type. `_value` means "take the type
120−`_value` means "take the type of `value`, but make the default value be
128+of `value`, but make the default value be `value`". Thus, our
121−`value`". Thus, our type is still a boolean, just like `?`, but the default
129+type is still a boolean, just like `?`, but the default value is
122−value is `|` (false).
130+`|` (false).
123
131
124−> This is the same `_` used in `_+>.$`, which means "the same type as the value
132+> This is the same `_` used in `_+>.$`, which means "the same
125−> "+&gt;.$". In other words, the same type as our current context and state.
133+> type as the value "+>.$". In other words, the same type as our
134+> current context and state.
126
135
127−Let's take a look at `++poke-atom`. When we're poked with an atom, we first
136+Let's take a look at `++poke-atom`. When we're poked with an atom, we
128−check whether the atom is `'off'`. If so, we set our state variable `on` to
137+first check whether the atom is `'off'`. If so, we set our state
129−false.
138+variable `on` to false.
130
139
131−If not, we check whether the atom is `'on'`. If so, we set `on` and
140+If not, we check whether the atom is `'on'`. If so, we set `on` and
132−`in-progress` to true. If it was already either on or in progress, then we don't
141+`in-progress` to true. If it was already either on or in progress, then
133−take any other immediate action. If it was both off and not in progress, then we
142+we don't take any other immediate action. If it was both off and not in
134−send an HTTP request.
143+progress, then we send an HTTP request.
135
144
136−If the argument is neither 'off' nor 'on', then we assume it's an actual url, so
145+If the argument is neither 'off' nor 'on', then we assume it's an
137−we save it in `target`.
146+actual url, so we save it in `target`.
138
147
139Here's the move that sends the HTTP request:
148Here's the move that sends the HTTP request:
140−
149+```
141− [ost.hid %hiss /request ~ %httr %purl (need (epur target))]
150+[ost.hid %hiss /request ~ %httr %purl (need (epur target))]
142−
151+```
152+
143−> Remember, we are expected to produce a *list* of moves. Note the `~` after the
153+> Remember, we are expected to produce a *list* of moves. Note the `~`
144−> move in the full example. This is a convenient shortcut for creating a list of
154+> after the move in the full example. This is a convenient shortcut for
145−> a single element. It's part of a small family of such shortcuts. `~[a b c]` is
155+> creating a list of a single element. It's part of a small family of
146−> `[a b c ~]`, `[a b c]~` is `[[a b c] ~]` and `\`\[a b c\]`is`\[~ a b c\]\`.
156+> such shortcuts. `~[a b c]` is `[a b c ~]`, `[a b c]~` is `[[a b c]
157+> ~]` and `\`[a b c]` is `[~ a b c]`. These may be mixed and matched to
147−> These may be mixed and matched to create various convoluted structures and
158+> create various convoluted structures and emojis.
148−> emojis.
159+
149−
150−The correspondence between this move and `{bone card}` can be hard to visualize
160+The correspondence between this move and `{bone card}` can be hard to
151−on one line. Here it is more pedantically:
161+visualize on one line. Here it is more pedantically:
152−
162+
163+```
153− :* bone=ost.hid :: the move
164+:* bone=ost.hid :: the move
154− term=%hiss
165+ term=%hiss
155− wire=/request
166+ wire=/request
156− unit-iden=~
167+ unit-iden=~
157− mark=%httr
168+ mark=%httr
158− cage=[%purl (need (epur target))]
169+ cage=[%purl (need (epur target))]
159− ==
170+==
160−
171+```
172+
161−When the HTTP response comes back, we handle it with `++sigh-httr`, which, along
173+When the HTTP response comes back, we handle it with
174+`++sigh-httr`, which, along with the wire the request was sent
162−with the wire the request was sent on, takes the status code, the response
175+on, takes the status code, the response headers, and the response
163−headers, and the response body.
176+body.
164−
177+
165−We check whether the status code is between 200 and 300. If so, all is well, and
178+We check whether the status code is between 200 and 300. If so,
166−we print a message saying so. We start the timer for ten seconds after the
179+all is well, and we print a message saying so. We start the
167−present time.
180+timer for ten seconds after the present time.
168−
181+
169−If we got a bad status code, then we print out the entire response and start the
182+If we got a bad status code, then we print out the entire
170−timer again.
183+response and start the timer again.
171−
184+
172−After ten seconds, arvo will give us a `%wake` event, which will be handled in
185+After ten seconds, arvo will give us a `%wake` event, which will
173−`++wake-timer`. If we're still supposed to keep monitoring, we send the same
186+be handled in `++wake-timer`. If we're still supposed to keep
187+monitoring, we send the same HTTP request as before. Otherwise,
174−HTTP request as before. Otherwise, we set `in-progress` to false.
188+we set `in-progress` to false.
175
189
176Let's try it out:
190Let's try it out:
177
191
192+```
178− ~fintud-macrep:dojo> |start %up
193+~fintud-macrep:dojo> |start %up
179− >=
194+>=
180− ~fintud-macrep:dojo> :examples-up &atom 'http://www.google.com'
195+~fintud-macrep:dojo> :examples-up &atom 'http://www.google.com'
181− >=
196+>=
182− [%all-is-well 200]
197+[%all-is-well 200]
183− [%all-is-well 200]
198+[%all-is-well 200]
184− ~fintud-macrep:dojo> :examples-up &atom 'http://example.com'
199+~fintud-macrep:dojo> :examples-up &atom 'http://example.com'
185− >=
200+>=
186− [%all-is-well 200]
201+[%all-is-well 200]
187− ~fintud-macrep:dojo> :examples-up &atom 'http://google.com'
202+~fintud-macrep:dojo> :examples-up &atom 'http://google.com'
188− >=
203+>=
189− [%we-have-a-problem 301]
204+[%we-have-a-problem 301]
190− [ %headers
205+[ %headers
191− ~[
206+ ~[
192− [p='X-Frame-Options' q='SAMEORIGIN']
207+ [p='X-Frame-Options' q='SAMEORIGIN']
193− [p='X-XSS-Protection' q='1; mode=block']
208+ [p='X-XSS-Protection' q='1; mode=block']
194− [p='Content-Length' q='219']
209+ [p='Content-Length' q='219']
195− [p='Server' q='gws']
210+ [p='Server' q='gws']
196− [p='Cache-Control' q='public, max-age=2592000']
211+ [p='Cache-Control' q='public, max-age=2592000']
197− [p='Expires' q='Sun, 10 Jan 2016 23:45:05 GMT']
212+ [p='Expires' q='Sun, 10 Jan 2016 23:45:05 GMT']
198− [p='Date' q='Fri, 11 Dec 2015 23:45:05 GMT']
213+ [p='Date' q='Fri, 11 Dec 2015 23:45:05 GMT']
199− [p='Content-Type' q='text/html; charset=UTF-8']
214+ [p='Content-Type' q='text/html; charset=UTF-8']
200− [p='Location' q='http://www.google.com/']
215+ [p='Location' q='http://www.google.com/']
201− ]
216+ ]
202− ]
217+]
203− [ %body
218+[ %body
204− [ ~
219+ [ ~
205− [ p=219
220+ [ p=219
206− q
221+ q
207− \/'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\0a<TITLE>301 Moved</TI\/
222+ \/'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\0a<TITLE>301 Moved</TI\/
208− TLE></HEAD><BODY>\0a<H1>301 Moved</H1>\0aThe document has moved\0a<A HREF="http://www.google.com/">her
223+ TLE></HEAD><BODY>\0a<H1>301 Moved</H1>\0aThe document has moved\0a<A HREF="http://www.google.com/">her
209− e</A>.\0d\0a</BODY></HTML>\0d\0a'
224+ e</A>.\0d\0a</BODY></HTML>\0d\0a'
210− \/ \/
225+ \/ \/
211− ]
212− ]
213 ]
226 ]
227+ ]
228+]
214− ~fintud-macrep:dojo> :up &atom 'off'
229+~fintud-macrep:dojo> :up &atom 'off'
230+```
docs/arvo/internals.md+104−92
@@ -1,5 +1,5 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 22
3sort: 22
4---
4---
5
5
@@ -9,25 +9,27 @@ Arvo is composed of modules called vanes:
9
9
10<list dataPreview="true"></list>
10<list dataPreview="true"></list>
11
11
12−<hr>
12+<hr></hr>
13−</hr>
14−At a high level `%arvo` takes a mess of unix io events and turns them into
15−something clean and structured for the programmer.
16
13
17−`%arvo` is designed to avoid the usual state of complex event networks: event
14+At a high level `%arvo` takes a mess of unix io events and turns them
18−spaghetti. We keep track of every event's cause so that we have a clear causal
15+into something clean and structured for the programmer.
19−chain for every computation. At the bottom of every chain is a unix io event,
20−such as a network request, terminal input, file sync, or timer event. We push
21−every step in the path the request takes onto the chain until we get to the
22−terminal cause of the computation. Then we use this causal stack to route
23−results back to the caller.
24
16
25−## `++ducts`
17+`%arvo` is designed to avoid the usual state of complex event networks:
18+event spaghetti. We keep track of every event's cause so that we have a
19+clear causal chain for every computation. At the bottom of every chain
20+is a unix io event, such as a network request, terminal input, file
21+sync, or timer event. We push every step in the path the request takes
22+onto the chain until we get to the terminal cause of the computation.
23+Then we use this causal stack to route results back to the caller.
26
24
25+`++ducts`
26+---------
27+
27−The `%arvo` causal stack is called a `++duct`. This is represented simply as a
28+The `%arvo` causal stack is called a `++duct`. This is represented
28−list of paths, where each path represents a step in the causal chain. The first
29+simply as a list of paths, where each path represents a step in the
29−element in the path is the first letter of whichever vane handled that step in
30+causal chain. The first element in the path is the first letter of
30−the computation, or the empty span for unix.
31+whichever vane handled that step in the computation, or the empty span
32+for unix.
31
33
32Here's a duct that was recently observed in the wild:
34Here's a duct that was recently observed in the wild:
33
35
@@ -39,98 +41,108 @@ Here's a duct that was recently observed in the wild:
39 //term/1
41 //term/1
40 ]
42 ]
41
43
42−This is the duct the timer vane receives when "timer" sample app asks the timer
44+This is the duct the timer vane receives when "timer" sample app asks
43−vane to set a timer. This is also the duct over which the response is produced
45+the timer vane to set a timer. This is also the duct over which the
44−at the specified time. Unix sent a terminal keystroke event (enter), and arvo
46+response is produced at the specified time. Unix sent a terminal
47+keystroke event (enter), and arvo routed it to %dill(our terminal),
45−routed it to %dill(our terminal), which passed it on to the %gall app terminal,
48+which passed it on to the %gall app terminal, which sent it to shell,
46−which sent it to shell, its child, which created a new child (with process id
49+its child, which created a new child (with process id 4), which on
47−4), which on startup asked the timer vane to set a timer.
50+startup asked the timer vane to set a timer.
48−
51+
49−The timer vane saves this duct, so that when the specified time arrives and unix
52+The timer vane saves this duct, so that when the specified time arrives
50−sends a wakeup event to the timer vane, it can produce the response on the same
53+and unix sends a wakeup event to the timer vane, it can produce the
51−duct. This response is routed to the place we popped off the top of the duct,
54+response on the same duct. This response is routed to the place we
52−i.e. the time app. This app produces the text "ding", which falls down to the
55+popped off the top of the duct, i.e. the time app. This app produces the
56+text "ding", which falls down to the shell, which drops it through to
53−shell, which drops it through to the terminal. Terminal drops this down to dill,
57+the terminal. Terminal drops this down to dill, which converts it into
54−which converts it into an effect that unix will recognize as a request to print
58+an effect that unix will recognize as a request to print "ding" to the
55−"ding" to the screen. When dill produces this, the last path in the duct has an
59+screen. When dill produces this, the last path in the duct has an
56−initial element of the empty span, so this is routed to unix, which applies the
60+initial element of the empty span, so this is routed to unix, which
57−effects.
61+applies the effects.
58
62
59This is a call stack, with a crucial feature: the stack is a first-class
63This is a call stack, with a crucial feature: the stack is a first-class
60−citizen. You can respond over a duct zero, one, or many times. You can save
64+citizen. You can respond over a duct zero, one, or many times. You can
61−ducts for later use. There are definitely parallels to Scheme-style
65+save ducts for later use. There are definitely parallels to Scheme-style
62continuations, but simpler and with more structure.
66continuations, but simpler and with more structure.
63
67
64−## Making Moves
68+Making Moves
69+------------
65
70
66−If ducts are a call stack, then how do we make calls and produce results? Arvo
71+If ducts are a call stack, then how do we make calls and produce
67−processes "moves" which are a combination of message data and metadata. There
72+results? Arvo processes "moves" which are a combination of message data
68−are two types of moves. A `%pass` move is analogous to a call:
73+and metadata. There are two types of moves. A `%pass` move is analogous
74+to a call:
69
75
70 [duct %pass return-path=path vane-name=@tD data=card]
76 [duct %pass return-path=path vane-name=@tD data=card]
71
77
72−Arvo pushes the return path (preceded by the first letter of the vane name) onto
78+Arvo pushes the return path (preceded by the first letter of the vane
73−the duct and sends the given data, a card, to the vane we specified. Any
79+name) onto the duct and sends the given data, a card, to the vane we
74−response will come along the same duct with the path `return-path`.
80+specified. Any response will come along the same duct with the path
81+`return-path`.
75
82
76A `%give` move is analogous to a return:
83A `%give` move is analogous to a return:
77
84
78 [duct %give data=card]
85 [duct %give data=card]
79
86
80−Arvo pops the top path off the duct and sends the given card back to the caller.
87+Arvo pops the top path off the duct and sends the given card back to the
88+caller.
81
89
82−## Vanes
90+Vanes
91+-----
83
92
84−As shown above, we use arvo proper to route and control the flow of moves.
93+As shown above, we use arvo proper to route and control the flow of
85−However, arvo proper is rarely directly responsible for processing the event
94+moves. However, arvo proper is rarely directly responsible for
86−data that directly causes the desired outcome of a move. This event data is
95+processing the event data that directly causes the desired outcome of a
87−contained within a card, which is simply a `(pair term noun)`. Instead, arvo
96+move. This event data is contained within a card, which is simply a
97+`(pair term noun)`. Instead, arvo proper passes the card off to one of
88−proper passes the card off to one of its vanes, which each present an interface
98+its vanes, which each present an interface to clients for a particular
89−to clients for a particular well-defined, stable, and general-purpose piece of
99+well-defined, stable, and general-purpose piece of functionality.
90−functionality.
91
100
92−As of this writing, we have seven vanes, which each provide the following
101+As of this writing, we have seven vanes, which each provide the
93−services:
102+following services:
94
103
95−- `%ames` name of both our network and the vane that communicates over it
104+- `%ames` name of both our network and the vane that communicates over
105+ it
106+- `%behn` a simple timer
96- `%clay` version-controlled, referentially- transparent, and global
107- `%clay` version-controlled, referentially- transparent, and global
97 filesystem
108 filesystem
98−- `%dill` terminal driver. Unix sends keyboard events to `%dill` from either
109+- `%dill` terminal driver. Unix sends keyboard events to `%dill` from
99− the console or telnet, and `%dill` produces terminal output.
110+ either the console or telnet, and `%dill` produces terminal output.
100−- `%eyre` http server. Unix sends http messages to `%eyre`, and `%eyre`
111+- `%eyre` http server. Unix sends http messages to `%eyre`, and
101− produces http messages in response
112+ `%eyre` produces http messages in response
102- `%ford` handles resources and publishing
113- `%ford` handles resources and publishing
103−- `%gall` manages our userspace applications.. `%gall` keeps state and manages
114+- `%gall` manages our userspace applications. `%gall` keeps state and
104− subscribers
115+ manages subscribers
105−- `%time` a simple timer
116+
106−
107−## Cards
117+Cards
108−
118+-----
119+
109−Cards are the vane-specific portion of a move. Each vane defines a protocol for
120+Cards are the vane-specific portion of a move. Each vane defines a
110−interacting with other vanes (via arvo) by defining four types of cards: kisses,
121+protocol for interacting with other vanes (via arvo) by defining four
111−gifts, notes, and signs.
122+types of cards: kisses, gifts, notes, and signs.
112−
123+
113−When one vane is `%pass`ed a card in its `++kiss`, arvo activates the `++call`
124+When one vane is `%pass`ed a card in its `++kiss`, arvo activates the
114−gate with the card as its argument. To produce a result, the vane `%give`s one
125+`++call` gate with the card as its argument. To produce a result, the
115−of the cards defined in its `++gift`. If the vane needs to request something of
126+vane `%give`s one of the cards defined in its `++gift`. If the vane
127+needs to request something of another vane, it `%pass`es it a `++note`
116−another vane, it `%pass`es it a `++note` card. When that other vane returns a
128+card. When that other vane returns a result, arvo activates the `++take`
117−result, arvo activates the `++take` gate of the initial vane with one of the
129+gate of the initial vane with one of the cards defined in its `++sign`.
118−cards defined in its `++sign`.
130+
119−
120−In other words, there are only four ways of seeing a move: (1) as a request seen
131+In other words, there are only four ways of seeing a move: (1) as a
121−by the caller, which is a ++note. (2) that same request as seen by the callee, a
132+request seen by the caller, which is a ++note. (2) that same request as
122−`++kiss`. (3) the response to that first request as seen by the callee, a
133+seen by the callee, a `++kiss`. (3) the response to that first request
123−`++gift`. (4) the response to the first request as seen by the caller, a
134+as seen by the callee, a `++gift`. (4) the response to the first request
124−`++sign`.
135+as seen by the caller, a `++sign`.
125−
136+
126−When a `++kiss` card is passed to a vane, arvo calls its `++call` gate, passing
137+When a `++kiss` card is passed to a vane, arvo calls its `++call` gate,
127−it both the card and its duct. This gate must be defined in every vane. It
138+passing it both the card and its duct. This gate must be defined in
128−produces two things in the following order: a list of moves and a
139+every vane. It produces two things in the following order: a list of
129−possibly-modified copy of its context. The moves are used to interact with other
140+moves and a possibly-modified copy of its context. The moves are used to
130−vanes, while the new context allows the vane to save its state. The next time
141+interact with other vanes, while the new context allows the vane to save
131−arvo activates the vane it will have this context as its subject.
142+its state. The next time arvo activates the vane it will have this
132−
143+context as its subject.
144+
133−This overview has detailed how to pass a card to a particular vane. To see the
145+This overview has detailed how to pass a card to a particular vane. To
134−cards each vane can be `%pass`ed as a `++kiss` or return as a `++gift` (as well
146+see the cards each vane can be `%pass`ed as a `++kiss` or return as a
135−as the semantics tied to them), each vane's public interface is explained in
147+`++gift` (as well as the semantics tied to them), each vane's public
136−detail in its respective overview.
148+interface is explained in detail in its respective overview.
docs/arvo/internals/ames.md+18−15
@@ -1,25 +1,28 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 1
3sort: 1
4---
4---
5
5
6−# `%ames`
6+`%ames`
7+=======
7
8
8Our networking protocol.
9Our networking protocol.
9
10
10−`%ames` is the name of both our network and the vane that communicates over it.
11+`%ames` is the name of both our network and the vane that communicates
11−When Unix receives a packet over the correct UDP port, it pipes it straight into
12+over it. When Unix receives a packet over the correct UDP port, it pipes
12−`%ames` for handling. Also, all packets sent over the `%ames` network are sent
13+it straight into `%ames` for handling. Also, all packets sent over the
13−by the `%ames` vane. Apps and vanes may use `%ames` to directly send messages to
14+`%ames` network are sent by the `%ames` vane. Apps and vanes may use
15+`%ames` to directly send messages to other ships. In general, apps use
14−other ships. In general, apps use gall and clay to communicate with other ships
16+gall and clay to communicate with other ships rather than using `%ames`
15−rather than using `%ames` directly, but this isn't a requirement. Of course,
17+directly, but this isn't a requirement. Of course, gall and clay use
16−gall and clay use `%ames` behind the scenes to communicate across the network.
18+`%ames` behind the scenes to communicate across the network. These are
17−These are the only two vanes that use `%ames`.
19+the only two vanes that use `%ames`.
18
20
19−`%ames` includes several significant components. Although the actual crypto
21+`%ames` includes several significant components. Although the actual
20−algorithms are defined in zuse, they're used extensively in `%ames` for
22+crypto algorithms are defined in zuse, they're used extensively in
21−encrypting and decrypting packets. Congestion control and routing is handled
23+`%ames` for encrypting and decrypting packets. Congestion control and
22−entirely in `%ames`. Finally, the actual `%ames` protocol itself, including how
24+routing is handled entirely in `%ames`. Finally, the actual `%ames`
25+protocol itself, including how to route incoming packets to the correct
23−to route incoming packets to the correct vane or app, is defined in `%ames`.
26+vane or app, is defined in `%ames`.
24
27
25<list></list>
28<list></list>
docs/arvo/internals/ames/commentary.md+769−690
@@ -1,69 +1,80 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3---
3---
4
4
5+
5−# `%ames` commentary
6+`%ames` commentary
7+==================
6
8
7`%ames` is our networking protocol.
9`%ames` is our networking protocol.
8
10
9−First we give commentary on the code, the algorithms involved, and the protocol.
11+First we give commentary on the code, the algorithms involved, and the
10−We trace through the code touched when a packet is sent, received, acknowledged,
12+protocol. We trace through the code touched when a packet is sent,
11−and that acknowledgment applied. This is fairly comprehensive, and contains many
13+received, acknowledged, and that acknowledgment applied. This is fairly
14+comprehensive, and contains many implementation details, but if you
12−implementation details, but if you understand this, then you understand `%ames`.
15+understand this, then you understand `%ames`.
13−
16+
14−If you've scrolled down this page, you may be intimidated by the amount of Hoon
17+If you've scrolled down this page, you may be intimidated by the amount
15−code, especially if you are new to the language. Don't be afraid of it, you
18+of Hoon code, especially if you are new to the language. Don't be afraid
16−don't have to read any of it if you don't want to -- every interesting action
19+of it, you don't have to read any of it if you don't want to -- every
17−the code takes is explained in plain English. In fact, if you are new to the
20+interesting action the code takes is explained in plain English. In
21+fact, if you are new to the language, this may be a good learning
18−language, this may be a good learning opportunity. Even if you don't understand
22+opportunity. Even if you don't understand every line of Hoon code,
19−every line of Hoon code, you'll hopefully be able to follow most lines. By the
23+you'll hopefully be able to follow most lines. By the time you've worked
20−time you've worked through this, you'll have seen many common patterns and best
24+through this, you'll have seen many common patterns and best practices.
21−practices. Hoon, much more than other languages, is best learned by reading and
25+Hoon, much more than other languages, is best learned by reading and
22−understanding large quantities of existing code. In this way, it is similar to
26+understanding large quantities of existing code. In this way, it is
23−learning a natural language. All of this code is in `arvo/ames.hoon`.
27+similar to learning a natural language. All of this code is in
28+`arvo/ames.hoon`.
24
29
25After the commentary, we have reference documentation for all the data
30After the commentary, we have reference documentation for all the data
26−structures that are specific to `%ames`. If you see a data structure or a
31+structures that are specific to `%ames`. If you see a data structure or
27−variable used that you don't recognize, search for it in the code, and it's very
32+a variable used that you don't recognize, search for it in the code, and
28−likely defined in one of these data structures. We recommend that another tab is
33+it's very likely defined in one of these data structures. We recommend
29−kept open for easy access to the data structure reference documentation. The
34+that another tab is kept open for easy access to the data structure
35+reference documentation. The code for these is split between
30−code for these is split between `arvo/ames.hoon` and `arvo/zuse.hoon`.
36+`arvo/ames.hoon` and `arvo/zuse.hoon`.
31
37
32−## The Lifecycle of a Packet (or, How a Packet Becomes Law)
38+The Lifecycle of a Packet (or, How a Packet Becomes Law)
39+--------------------------------------------------------
33
40
34Here, we will trace a packet as it makes its way through ames. There are
41Here, we will trace a packet as it makes its way through ames. There are
35−actually two pathways through ames: the legacy path through `%want`, and the
42+actually two pathways through ames: the legacy path through `%want`, and
36−modern way, entered through `%wont`, with full end-to-end acknowledgments. Here
43+the modern way, entered through `%wont`, with full end-to-end
37−we will only trace the modern way, though much of the path is the same for both.
44+acknowledgments. Here we will only trace the modern way, though much of
45+the path is the same for both.
38
46
39−When an app (or a vane) wishes to send a packet to another ship, it must send a
47+When an app (or a vane) wishes to send a packet to another ship, it must
40−`%wont` card:
48+send a `%wont` card:
41
49
42 [%wont p=sock q=path r=*] :: e2e send message
50 [%wont p=sock q=path r=*] :: e2e send message
43
51
44−This card takes three arguments. The `p` is a `sock`, that is, a pair of two
52+This card takes three arguments. The `p` is a `sock`, that is, a pair of
45−ships, the first of which is the sender and the second is the receiver. But
53+two ships, the first of which is the sender and the second is the
46−wait, you ask, why do I get to decide who is the sender? Can I fake like I'm
54+receiver. But wait, you ask, why do I get to decide who is the sender?
47−someone else? The reason is that there are potentially multiple ships on the
55+Can I fake like I'm someone else? The reason is that there are
56+potentially multiple ships on the same pier, and the kernel can send a
48−same pier, and the kernel can send a message from any of them. If you attempt to
57+message from any of them. If you attempt to send a message from a ship
49−send a message from a ship not on your pier, then ames will refuse to send it.
58+not on your pier, then ames will refuse to send it. If you hack around
50−If you hack around in your own copy of ames to go ahead and send it anyway, then
59+in your own copy of ames to go ahead and send it anyway, then the other
51−the other ship will reject it because your key is bad. Only send messages from
60+ship will reject it because your key is bad. Only send messages from
52yourself.
61yourself.
53
62
54−The `q` is a path, representing the place on the other side that you want to
63+The `q` is a path, representing the place on the other side that you
55−receive your message. It is approximately equivalent to a port number. Messages
64+want to receive your message. It is approximately equivalent to a port
56−on the same path are guaranteed to arrive in the same order as they were sent.
65+number. Messages on the same path are guaranteed to arrive in the same
57−No such guarantees are made across paths.
66+order as they were sent. No such guarantees are made across paths.
58
67
59−The `r` is the actual data that you are sending. As the type implies, this can
68+The `r` is the actual data that you are sending. As the type implies,
60−be an arbitrary noun, and it will be transferred to the receiver exactly as-is,
69+this can be an arbitrary noun, and it will be transferred to the
61−in a well-typed way. Of course, this is data that is sent over the wire, so be
70+receiver exactly as-is, in a well-typed way. Of course, this is data
71+that is sent over the wire, so be careful not to send anything too
62−careful not to send anything too massive unless you're willing to wait.
72+massive unless you're willing to wait.
63
73
64−But enough about the interface. Grepping in ames.hoon for `%wont`, we find that
74+But enough about the interface. Grepping in ames.hoon for `%wont`, we
65−it appears in exactly two places: at its definition in `++kiss`, and in
75+find that it appears in exactly two places: at its definition in
66−`++knob`, where it is handled. We see that we go directly into `++wise:am`.
76+`++kiss`, and in `++knob`, where it is handled. We see that we go
77+directly into `++wise:am`.
67
78
68 ++ wise :: wise:am
79 ++ wise :: wise:am
69 |= [soq=sock hen=duct cha=path val=* ete=?] :: send a statement
80 |= [soq=sock hen=duct cha=path val=* ete=?] :: send a statement
@@ -71,19 +82,20 @@ it appears in exactly two places: at its definition in `++kiss`, and in
71 zork:zank:(wool:(ho:(um p.soq) q.soq) hen cha val ete)
82 zork:zank:(wool:(ho:(um p.soq) q.soq) hen cha val ete)
72
83
73The inputs to this gate are exactly the sort of thing you'd expect. In
84The inputs to this gate are exactly the sort of thing you'd expect. In
74−particular, everything in the `%wont` gate is here plus the calling duct so that
85+particular, everything in the `%wont` gate is here plus the calling duct
75−we know where to send the acknowledgment and `ete` to determine if we're going
86+so that we know where to send the acknowledgment and `ete` to determine
76−to do the modern end-to-end acknowledgments.
87+if we're going to do the modern end-to-end acknowledgments.
77
88
78−The actual line of code looks intimidating, but it's really not all that bad.
89+The actual line of code looks intimidating, but it's really not all that
79−Working from the inside out, the call to `++um` sets up our domestic server, and
90+bad. Working from the inside out, the call to `++um` sets up our
80−the call to `++ho` sets up our knowledge about the neighbor we're sending to.
91+domestic server, and the call to `++ho` sets up our knowledge about the
81−From the outside, `++zork` and `++zank` just apply the changes made to our
92+neighbor we're sending to. From the outside, `++zork` and `++zank` just
93+apply the changes made to our `++um` and `++am` cores, respectively. If
82−`++um` and `++am` cores, respectively. If you're familiar with the common idiom
94+you're familiar with the common idiom of `++abet`, that's all this is.
83−of `++abet`, that's all this is. The code predates the widespread usage of that
95+The code predates the widespread usage of that name.
84−name.
85
96
86−The interesting part, then, is in `++wool:ho:um:am`. Let's look at the code.
97+The interesting part, then, is in `++wool:ho:um:am`. Let's look at the
98+code.
87
99
88 ++ wool :: wool:ho:um:am
100 ++ wool :: wool:ho:um:am
89 |= [hen=duct cha=path val=* ete=?] :: send a statement
101 |= [hen=duct cha=path val=* ete=?] :: send a statement
@@ -102,33 +114,36 @@ The interesting part, then, is in `++wool:ho:um:am`. Let's look at the code.
102 [%bund q.cov cha sex val]
114 [%bund q.cov cha sex val]
103 [%bond q.cov cha sex val]
115 [%bond q.cov cha sex val]
104
116
105−This is slightly more complicated, but it's still not all that bad. Our inputs,
117+This is slightly more complicated, but it's still not all that bad. Our
106−at least, are fairly obvious.
118+inputs, at least, are fairly obvious.
107
119
108−If you glance at the code for a second, you'll see that `++wind:ho:um:am` seems
120+If you glance at the code for a second, you'll see that
109−to be able to send a message, or `++meal`, given a `++soup`. This gate, then,
121+`++wind:ho:um:am` seems to be able to send a message, or `++meal`, given
110−just sets up the things we need to for `++wind` to do its job.
122+a `++soup`. This gate, then, just sets up the things we need to for
123+`++wind` to do its job.
111
124
112−We first get `rol`, which is a `++rill`, that is, a particular outbound stream.
125+We first get `rol`, which is a `++rill`, that is, a particular outbound
113−This stream is specific to the path on which we're sending. If the path hasn't
126+stream. This stream is specific to the path on which we're sending. If
114−been used before, then we create it. We let `sex` be the number of messages
127+the path hasn't been used before, then we create it. We let `sex` be the
115−we've already sent on this path.
128+number of messages we've already sent on this path.
116
129
117−Then, we update the outbound stream by incrementing the number of messages sent
130+Then, we update the outbound stream by incrementing the number of
118−and placing an entry in `san.rol` that associates the message number with the
131+messages sent and placing an entry in `san.rol` that associates the
119−`duct` that sent the message. This allows us to give the acknowledgment to the
132+message number with the `duct` that sent the message. This allows us to
120−one who sent the message.
133+give the acknowledgment to the one who sent the message.
121
134
122−We let `cov` be the current life of our crypto and our neighbor's crypto. At the
135+We let `cov` be the current life of our crypto and our neighbor's
123−moment, we only need our neighbor's life, which we put into the meal.
136+crypto. At the moment, we only need our neighbor's life, which we put
137+into the meal.
124
138
125−Finally, we call `++wind:ho:um:am` with the `++soup` of the path and message
139+Finally, we call `++wind:ho:um:am` with the `++soup` of the path and
126−number and the `++meal` of the payload itself. For end-to-end acknowledged
140+message number and the `++meal` of the payload itself. For end-to-end
127−messages, we use `%bund`.
141+acknowledged messages, we use `%bund`.
128
142
129 [%bund p=life q=path r=@ud s=*] :: e2e message
143 [%bund p=life q=path r=@ud s=*] :: e2e message
130
144
131−Looking at how we create the `%bund`, we can easily see what each field is for.
145+Looking at how we create the `%bund`, we can easily see what each field
146+is for.
132
147
133Following the trail a little further, we go to `++wind:ho:um:am`.
148Following the trail a little further, we go to `++wind:ho:um:am`.
134
149
@@ -141,13 +156,13 @@ Following the trail a little further, we go to `++wind:ho:um:am`.
141 (busk xong:diz feh)
156 (busk xong:diz feh)
142
157
143`++wind` does three things: it (1) encodes the message into a list of
158`++wind` does three things: it (1) encodes the message into a list of
144−possibly-encrypted packets, (2) puts the message into the packet pump, and (3)
159+possibly-encrypted packets, (2) puts the message into the packet pump,
145−sends any packets that are ready to be sent. Yes, our nice little linear run of
160+and (3) sends any packets that are ready to be sent. Yes, our nice
146−each gate calling exactly one other interesting gate is over. We'll go in order
161+little linear run of each gate calling exactly one other interesting
147−here.
162+gate is over. We'll go in order here.
148
163
149−`++zuul:lax:as:go` is the what converts a `++meal` into a list of actual, 1KB
164+`++zuul:lax:as:go` is the what converts a `++meal` into a list of
150−packets.
165+actual, 1KB packets.
151
166
152 ++ zuul :: zuul:lax:as:go
167 ++ zuul :: zuul:lax:as:go
153 |= [now=@da ham=meal] :: encode message
168 |= [now=@da ham=meal] :: encode message
@@ -157,32 +172,33 @@ packets.
157 ++ weft :: fragment message
172 ++ weft :: fragment message
158 ++ wisp :: generate message
173 ++ wisp :: generate message
159
174
160−For organizational purposes, `++zuul` constructs an internal core with three
175+For organizational purposes, `++zuul` constructs an internal core with
161−arms. `++wasp` encodes the meal into an atom with no encryption. `++wisp`
176+three arms. `++wasp` encodes the meal into an atom with no encryption.
162−encodes a meal with possible encryption (else it simply calls `++wasp`).
177+`++wisp` encodes a meal with possible encryption (else it simply calls
163−`++weft` takes the result of `++wisp` and splits it into actual packets.
178+`++wasp`). `++weft` takes the result of `++wisp` and splits it into
179+actual packets.
164
180
165 ++ wasp :: null security
181 ++ wasp :: null security
166 ^-([p=skin q=@] [%none (jam ham)])
182 ^-([p=skin q=@] [%none (jam ham)])
167
183
168−This simply jams the meal, wrapping it with the `skin` of `%none`, meaning no
184+This simply jams the meal, wrapping it with the `skin` of `%none`,
169−encryption.
185+meaning no encryption.
170
186
171Since `++wisp` is a little long, we'll go through it line-by-line.
187Since `++wisp` is a little long, we'll go through it line-by-line.
172
188
173 ++ wisp :: generate message
189 ++ wisp :: generate message
174 ^- [[p=skin q=@] q=_..wisp]
190 ^- [[p=skin q=@] q=_..wisp]
175
191
176−`++wisp` produces a pair of a `skin` and an atom, which is the meal encoded as a
192+`++wisp` produces a pair of a `skin` and an atom, which is the meal
177−single atom and possibly encrypted.
193+encoded as a single atom and possibly encrypted.
178
194
179 ?: =(%carp -.ham)
195 ?: =(%carp -.ham)
180 [wasp ..wisp]
196 [wasp ..wisp]
181
197
182−If the meal that we're encoding is a `%carp`, then we don't encrypt it. A
198+If the meal that we're encoding is a `%carp`, then we don't encrypt it.
183−`%carp` meal is a partial meal, used when a message is more than 1KB. Since the
199+A `%carp` meal is a partial meal, used when a message is more than 1KB.
184−entire message is already encrypted, we don't need to encrypt each packet
200+Since the entire message is already encrypted, we don't need to encrypt
185−individually again.
201+each packet individually again.
186
202
187 ?: !=(~ yed.caq.dur)
203 ?: !=(~ yed.caq.dur)
188 ?> ?=(^ yed.caq.dur)
204 ?> ?=(^ yed.caq.dur)
@@ -192,29 +208,31 @@ individually again.
192 p.u.yed.caq.dur
208 p.u.yed.caq.dur
193 (en:r:cluy q.u.yed.caq.dur (jam ham))
209 (en:r:cluy q.u.yed.caq.dur (jam ham))
194
210
195−If we have a symmetric key set up with this neighbor, then we simply use it. The
211+If we have a symmetric key set up with this neighbor, then we simply use
196−skin `%fast` is used to indicate a symmetric key.
212+it. The skin `%fast` is used to indicate a symmetric key.
197
213
198 ?: &(=(~ lew.wod.dur) |(=(%back -.ham) =(%buck -.ham)))
214 ?: &(=(~ lew.wod.dur) |(=(%back -.ham) =(%buck -.ham)))
199 [wasp ..wisp]
215 [wasp ..wisp]
200
216
201−If we do not yet have our neighbor's will, then there is no way that we can seal
217+If we do not yet have our neighbor's will, then there is no way that we
202−the message so that only they may read it. If what we're sending is an
218+can seal the message so that only they may read it. If what we're
203−acknowledgment, then we go ahead and just send it in the clear.
219+sending is an acknowledgment, then we go ahead and just send it in the
220+clear.
204
221
205 =^ tuy +>.$
222 =^ tuy +>.$
206 ?:(=(~ lew.wod.dur) [*code +>.$] (griz now))
223 ?:(=(~ lew.wod.dur) [*code +>.$] (griz now))
207
224
208−If we don't have our neighbor's will, then we "encrypt" with a key of 0. If we
225+If we don't have our neighbor's will, then we "encrypt" with a key of 0.
209−do have their will, then we generate a new symmetric key that we will propose.
226+If we do have their will, then we generate a new symmetric key that we
227+will propose.
210
228
211 :_ ..wisp
229 :_ ..wisp
212 =+ yig=sen
230 =+ yig=sen
213 =+ bil=law.saf :: XX send whole will
231 =+ bil=law.saf :: XX send whole will
214 =+ hom=(jam ham)
232 =+ hom=(jam ham)
215
233
216−`yig` will be the life and engine for our current crypto. `bil` is our will.
234+`yig` will be the life and engine for our current crypto. `bil` is our
217−`hom` is the meal encoded as a single atom.
235+will. `hom` is the meal encoded as a single atom.
218
236
219 ?: =(~ lew.wod.dur)
237 ?: =(~ lew.wod.dur)
220 :- %open
238 :- %open
@@ -223,8 +241,9 @@ do have their will, then we generate a new symmetric key that we will propose.
223 bil
241 bil
224 (sign:as:q.yig tuy hom)
242 (sign:as:q.yig tuy hom)
225
243
226−If we do not have our neighbor's will, then we send our current life along with
244+If we do not have our neighbor's will, then we send our current life
227−our will and the message. The message itself is "signed" with a key of 0.
245+along with our will and the message. The message itself is "signed" with
246+a key of 0.
228
247
229 :- %full
248 :- %full
230 =+ cay=cluy
249 =+ cay=cluy
@@ -234,9 +253,9 @@ our will and the message. The message itself is "signed" with a key of 0.
234 (seal:as:q.yig pub:ex:r.cay tuy hom)
253 (seal:as:q.yig pub:ex:r.cay tuy hom)
235 -- :: --zuul:lax:as:go
254 -- :: --zuul:lax:as:go
236
255
237−If we do have our neighbor's will, then we send our perception of their current
256+If we do have our neighbor's will, then we send our perception of their
238−life, our current life, our will, and the message. The message is sealed with
257+current life, our current life, our will, and the message. The message
239−their public key so that only they can read our message.
258+is sealed with their public key so that only they can read our message.
240
259
241Once we have the message encoded as an atom, `++weft` goes to work.
260Once we have the message encoded as an atom, `++weft` goes to work.
242
261
@@ -252,23 +271,23 @@ aforementioned `++wisp` to get the message as an atom.
252 =+ wit=(met 13 q.gim)
271 =+ wit=(met 13 q.gim)
253 ?< =(0 wit)
272 ?< =(0 wit)
254
273
255−`wit` is the number of 1KB (2\\^13 bit) blocks in the message. We assert that
274+`wit` is the number of 1KB (2\^13 bit) blocks in the message. We assert
256−there is at least one block.
275+that there is at least one block.
257
276
258 ?: =(1 wit)
277 ?: =(1 wit)
259 =+ yup=(spit [our her] p.gim q.gim)
278 =+ yup=(spit [our her] p.gim q.gim)
260 [yup ~]
279 [yup ~]
261
280
262−If there is exactly one block, then we just call `++spit` to turn the message
281+If there is exactly one block, then we just call `++spit` to turn the
263−into a packet. We'll explain what `++spit` does momentarily.
282+message into a packet. We'll explain what `++spit` does momentarily.
264
283
265 =+ ruv=(rip 13 q.gim)
284 =+ ruv=(rip 13 q.gim)
266 =+ gom=(shaf %thug q.gim)
285 =+ gom=(shaf %thug q.gim)
267 =+ inx=0
286 =+ inx=0
268
287
269−If there is more than one block, then we rip it into blocks in `ruv`. `gom` is a
288+If there is more than one block, then we rip it into blocks in `ruv`.
270−hash of the message, used as an id. `inx` is the number of packets we've already
289+`gom` is a hash of the message, used as an id. `inx` is the number of
271−made.
290+packets we've already made.
272
291
273 |- ^- (list rock)
292 |- ^- (list rock)
274 ?~ ruv ~
293 ?~ ruv ~
@@ -279,8 +298,8 @@ made.
279 :- vie
298 :- vie
280 $(ruv t.ruv, inx +(inx))
299 $(ruv t.ruv, inx +(inx))
281
300
282−Here we package each block into a packet with `++spit` and produce the list of
301+Here we package each block into a packet with `++spit` and produce the
283−packets.
302+list of packets.
284
303
285 ++ spit :: cake to packet
304 ++ spit :: cake to packet
286 |= kec=cake ^- @
305 |= kec=cake ^- @
@@ -302,30 +321,32 @@ packets.
302 ==
321 ==
303 (lsh 5 1 bod)
322 (lsh 5 1 bod)
304
323
305−This is how we turn a message into a real packet. This has the definition of the
324+This is how we turn a message into a real packet. This has the
306−packet format.
325+definition of the packet format.
307
326
308`wim` is the length of the sending ship, and `dum` is the length of the
327`wim` is the length of the sending ship, and `dum` is the length of the
309receiving ship. There are only five possibilities for each of those,
328receiving ship. There are only five possibilities for each of those,
310−corresponding to carriers, cruisers, destroyers, yachts, and submarines. These
329+corresponding to carriers, cruisers, destroyers, yachts, and submarines.
311−are encoded in `yax` and `qax` as 0, 0, 1, 2, and 3, respectively. Thus, `wix`
330+These are encoded in `yax` and `qax` as 0, 0, 1, 2, and 3, respectively.
312−and `vix` are the number of bytes that must be reserved for the ship names in a
331+Thus, `wix` and `vix` are the number of bytes that must be reserved for
313−packet.
332+the ship names in a packet.
314−
333+
315−Next, we construct `bod` by simply concatenating the sending ship, the receiving
334+Next, we construct `bod` by simply concatenating the sending ship, the
316−ship, and the body of the message. Then, we get the encryption mechanism from
335+receiving ship, and the body of the message. Then, we get the encryption
317−`++skin`, which may be a 0, 1, 2, or 3, and put it in `tay`.
336+mechanism from `++skin`, which may be a 0, 1, 2, or 3, and put it in
318−
337+`tay`.
338+
319−Next, we concatenate together, bit by bit, some final metadata. We use three
339+Next, we concatenate together, bit by bit, some final metadata. We use
320−bits for our protocol number, which is incremented modulo eight when there is a
340+three bits for our protocol number, which is incremented modulo eight
321−continuity breach or the protocol changes. We use the final twenty bits of a
341+when there is a continuity breach or the protocol changes. We use the
322−hash of the body (which, we suppose, makes it a twenty bit hash) for
342+final twenty bits of a hash of the body (which, we suppose, makes it a
323−error-checking. We use two bits to tell how much room is used in the body for
343+twenty bit hash) for error-checking. We use two bits to tell how much
324−the sending ship, and another two bits for the receiving ship. Finally, we use
344+room is used in the body for the sending ship, and another two bits for
345+the receiving ship. Finally, we use five bits to store the encryption
325−five bits to store the encryption type. Note that since there are only two bits
346+type. Note that since there are only two bits worth of encryption types,
326−worth of encryption types, there are three unused bits here. This adds up to 32
347+there are three unused bits here. This adds up to 32 bits of header
327−bits of header data. Finally, we concatenate this onto the front of the packet.
348+data. Finally, we concatenate this onto the front of the packet. Thus,
328−Thus, we can summarize the packet header format as follows.
349+we can summarize the packet header format as follows.
329
350
330 0 1 2 3
351 0 1 2 3
331 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
352 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
@@ -333,14 +354,15 @@ Thus, we can summarize the packet header format as follows.
333 |Proto| Hash of Body |yax|qax| Crypto |
354 |Proto| Hash of Body |yax|qax| Crypto |
334 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
355 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
335
356
336−After this, there are `yax` bits of the sender name, `qax` bits of the receiver
357+After this, there are `yax` bits of the sender name, `qax` bits of the
337−name, and up to 8192 bits of data. Thus, the maximum size of a packet is
358+receiver name, and up to 8192 bits of data. Thus, the maximum size of a
338−achieved in a message between two submarines with 8192 bits of data. This will
359+packet is achieved in a message between two submarines with 8192 bits of
339−require 32+128+128+8192 = 8480 bits, or 1060 bytes.
360+data. This will require 32+128+128+8192 = 8480 bits, or 1060 bytes.
340
361
341This concludes our discussion of `++zuul:lax:as:go`. If you recall from
362This concludes our discussion of `++zuul:lax:as:go`. If you recall from
342−`++wind:ho:um:am`, the list of packets from `++zuul` is passed into `++whap:pu`
363+`++wind:ho:um:am`, the list of packets from `++zuul` is passed into
343−to update the packet pump and get any packets that can be sent immediately.
364+`++whap:pu` to update the packet pump and get any packets that can be
365+sent immediately.
344
366
345 ++ whap :: whap:pu
367 ++ whap :: whap:pu
346 |= [now=@da gom=soup wyv=(list rock)] :: send a message
368 |= [now=@da gom=soup wyv=(list rock)] :: send a message
@@ -357,15 +379,16 @@ to update the packet pump and get any packets that can be sent immediately.
357 ==
379 ==
358 (harv now)
380 (harv now)
359
381
360−First, we put into `pyz` the id for this message and the number of its packets
382+First, we put into `pyz` the id for this message and the number of its
361−that have not yet been acknowledged, which is of course the total number of
383+packets that have not yet been acknowledged, which is of course the
362−packets since we haven't even sent the packets.
384+total number of packets since we haven't even sent the packets.
363
385
364−For every packet, we change three things in the state (`++shed`) of our packet
386+For every packet, we change three things in the state (`++shed`) of our
365−pump: (1) we increment `nus`, the number of packets sent; (2) we put the packet
387+packet pump: (1) we increment `nus`, the number of packets sent; (2) we
366−number into `diq` keyed by a hash of the packet; and (3) we put the packet into
388+put the packet number into `diq` keyed by a hash of the packet; and (3)
367−the packet queue, with the basic metadata of its id `gom`, 0 transmissions, not
389+we put the packet into the packet queue, with the basic metadata of its
368−live yet, last sent in the year 2000, and the packet itself.
390+id `gom`, 0 transmissions, not live yet, last sent in the year 2000, and
391+the packet itself.
369
392
370Finally, we harvest the packet pump.
393Finally, we harvest the packet pump.
371
394
@@ -379,17 +402,17 @@ Finally, we harvest the packet pump.
379 =< abet =< apse
402 =< abet =< apse
380 |%
403 |%
381
404
382−`++harv` contains a core for most of its work. The meat is in `++apse`. First,
405+`++harv` contains a core for most of its work. The meat is in `++apse`.
383−though, it sets itself up. If there aren't any packets in the queue, then we
406+First, though, it sets itself up. If there aren't any packets in the
384−simply do nothing except set `rtn`, our next timeout, to nil because we don't
407+queue, then we simply do nothing except set `rtn`, our next timeout, to
385−have any packets that may need to be retransmitted. If we have more live (that
408+nil because we don't have any packets that may need to be retransmitted.
386−is, sent and unacknowledged) packets than our window size, then we don't do
409+If we have more live (that is, sent and unacknowledged) packets than our
387−anything.
410+window size, then we don't do anything.
388
411
389−Otherwise, we let `wid` be the width of our remaining packet window, and we
412+Otherwise, we let `wid` be the width of our remaining packet window, and
390−initialize `rub` to nil. `rub` will be the list of packets that are ready to be
413+we initialize `rub` to nil. `rub` will be the list of packets that are
391−sent. We then call `++apse` and pass the result to `++abet`. `++apse` decides
414+ready to be sent. We then call `++apse` and pass the result to `++abet`.
392−which packets are ready to be sent.
415+`++apse` decides which packets are ready to be sent.
393
416
394 ++ apse
417 ++ apse
395 ^+ .
418 ^+ .
@@ -409,19 +432,20 @@ which packets are ready to be sent.
409 lys.q.n.puq now
432 lys.q.n.puq now
410 ==
433 ==
411
434
412−If there are no remaining packets to send, or if we've filled the packet window,
435+If there are no remaining packets to send, or if we've filled the packet
413−do nothing. We call `++rigt` and `++left` to process the left and right branches
436+window, do nothing. We call `++rigt` and `++left` to process the left
414−of the packet queue.
437+and right branches of the packet queue.
415
438
416−Now we assert that the queue is not empty, and we again check that we haven't
439+Now we assert that the queue is not empty, and we again check that we
417−filled the packet window. We will operate on the head of the queue. If the
440+haven't filled the packet window. We will operate on the head of the
418−packet is live, then do nothing. Otherwise, we go ahead and send it.
441+queue. If the packet is live, then do nothing. Otherwise, we go ahead
442+and send it.
419
443
420−To send, we (1) decrement `wid`, our packet window width; (2) cons the packet
444+To send, we (1) decrement `wid`, our packet window width; (2) cons the
421−onto the `rub`, which will be returned as the list of packets to send; (3)
445+packet onto the `rub`, which will be returned as the list of packets to
422−increment `nif`, the number of live packets; (4) set the packet to be live; (5)
446+send; (3) increment `nif`, the number of live packets; (4) set the
423−increment the number of transmissions of the packet; and (6) set the last sent
447+packet to be live; (5) increment the number of transmissions of the
424−time of the packet to now.
448+packet; and (6) set the last sent time of the packet to now.
425
449
426 ++ left
450 ++ left
427 ?> ?=(^ puq)
451 ?> ?=(^ puq)
@@ -430,8 +454,8 @@ time of the packet to now.
430 ?> ?=(^ puq)
454 ?> ?=(^ puq)
431 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
455 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
432
456
433−These do exactly what you would expect: they traverse the packet queue so that
457+These do exactly what you would expect: they traverse the packet queue
434−`++apse` gets called recursively through it.
458+so that `++apse` gets called recursively through it.
435
459
436Finally, `++abet` gets called, which resolves the changes.
460Finally, `++abet` gets called, which resolves the changes.
437
461
@@ -439,13 +463,14 @@ Finally, `++abet` gets called, which resolves the changes.
439 ?~ rub [~ +>.$]
463 ?~ rub [~ +>.$]
440 [(flop rub) +>.$(rtn [~ (add rto now)])]
464 [(flop rub) +>.$(rtn [~ (add rto now)])]
441
465
442−This returns the packets that we wish to send, and it updates the timeout so
466+This returns the packets that we wish to send, and it updates the
443−that we know when to try resending unacknowledged packets.
467+timeout so that we know when to try resending unacknowledged packets.
444
468
445−This concludes our discussion of `++whap:pu`. To finish `++wind:ho:um:am`, we
469+This concludes our discussion of `++whap:pu`. To finish
446−just need to delve into `++busk:ho:um:am`. But wait, in the call to `++busk`,
470+`++wind:ho:um:am`, we just need to delve into `++busk:ho:um:am`. But
447−the first argument is `xong:diz`. What is this? This, my dear reader, is one
471+wait, in the call to `++busk`, the first argument is `xong:diz`. What is
448−more detour, this time into `++xong:lax:as:go`.
472+this? This, my dear reader, is one more detour, this time into
473+`++xong:lax:as:go`.
449
474
450 ++ xong :: xong:lax:as:go
475 ++ xong :: xong:lax:as:go
451 ^- (list ship) :: route unto
476 ^- (list ship) :: route unto
@@ -459,24 +484,26 @@ more detour, this time into `++xong:lax:as:go`.
459 :: ~& [%xong-to [our her] (weld oot ?>(?=(^ fro) t.fro))]
484 :: ~& [%xong-to [our her] (weld oot ?>(?=(^ fro) t.fro))]
460 (weld oot ?>(?=(^ fro) t.fro))
485 (weld oot ?>(?=(^ fro) t.fro))
461
486
462−This gets the list of intermediate ships needed to get a packet from us to our
487+This gets the list of intermediate ships needed to get a packet from us
463−neighbor. First, we get `fro` and `too`, the "canons" of ourself and our
488+to our neighbor. First, we get `fro` and `too`, the "canons" of ourself
464−neighbor, respectively.
489+and our neighbor, respectively.
465
490
466−What is this "canon", you ask? A canon is simply a ship plus its "ancestors", as
491+What is this "canon", you ask? A canon is simply a ship plus its
467−defined by `++sein`. For example, the canon of `~hoclur-bicrel` is:
492+"ancestors", as defined by `++sein`. For example, the canon of
493+`~hoclur-bicrel` is:
468
494
469 ~hoclur-bicrel/try=> (saxo ~hoclur-bicrel)
495 ~hoclur-bicrel/try=> (saxo ~hoclur-bicrel)
470 ~[~hoclur-bicrel ~tasruc ~tug]
496 ~[~hoclur-bicrel ~tasruc ~tug]
471
497
472−If we follow the algorithm in `++xong`, we see that we are simply creating a
498+If we follow the algorithm in `++xong`, we see that we are simply
473−list of ships that form a path from our neighbor to ourself. Essentially, we
499+creating a list of ships that form a path from our neighbor to ourself.
474−look through the canon of our neighbor until we find something in our own cannon
500+Essentially, we look through the canon of our neighbor until we find
475−-- a common ancestor. Or, if we are from different carriers, then there is no
501+something in our own cannon -- a common ancestor. Or, if we are from
502+different carriers, then there is no common ancestor. We then weld this
476−common ancestor. We then weld this onto the tail of our own canon. In the end,
503+onto the tail of our own canon. In the end, this is simply a list of
477−this is simply a list of possible ships to try to route via to get to our
504+possible ships to try to route via to get to our neighbor, ordered by
478−neighbor, ordered by preferability (that is, closeness to our neighbor). We will
505+preferability (that is, closeness to our neighbor). We will end up
479−end up trying, in order, to find a lane to these.
506+trying, in order, to find a lane to these.
480
507
481Now, we can finally get to `++busk:ho:um:am`.
508Now, we can finally get to `++busk:ho:um:am`.
482
509
@@ -489,9 +516,10 @@ Now, we can finally get to `++busk:ho:um:am`.
489 $(pax t.pax, bin (weld (flop (wist:diz now waz ~ i.pax)) bin))
516 $(pax t.pax, bin (weld (flop (wist:diz now waz ~ i.pax)) bin))
490 ==
517 ==
491
518
492−Thankfully, `++busk` is fairly simple. We go through the list of packets and
519+Thankfully, `++busk` is fairly simple. We go through the list of packets
493−convert them to `++boon`s with `++wist:lax:as:go`. These boons are placed into
520+and convert them to `++boon`s with `++wist:lax:as:go`. These boons are
494−`bin`, and they end up getting processed by `++clop` (this happens in `++knob`).
521+placed into `bin`, and they end up getting processed by `++clop` (this
522+happens in `++knob`).
495
523
496 ++ wist :: wist:lax:as:go
524 ++ wist :: wist:lax:as:go
497 |= $: now=@da :: route via
525 |= $: now=@da :: route via
@@ -526,50 +554,53 @@ convert them to `++boon`s with `++wist:lax:as:go`. These boons are placed into
526This takes a sample of the current time, the list of ships that we just
554This takes a sample of the current time, the list of ships that we just
527generated, a lane if we already know it, and the packet itself.
555generated, a lane if we already know it, and the packet itself.
528
556
529−First, if we are sending a message to ourself, then we simply create a `%ouzo`
557+First, if we are sending a message to ourself, then we simply create a
530−boon with a bunted lane. Otherwise, if there are no routing candidates, there is
558+`%ouzo` boon with a bunted lane. Otherwise, if there are no routing
531−nothing we can do, so we return nil.
559+candidates, there is nothing we can do, so we return nil.
532−
560+
533−Next, we get the `dore` of the first routing candidate. If we're looking at the
561+Next, we get the `dore` of the first routing candidate. If we're looking
534−neighbor to whom we're trying to send the message, then we simply use the `dore`
562+at the neighbor to whom we're trying to send the message, then we simply
535−that we already have. Otherwise, we get a default `dore`.
563+use the `dore` that we already have. Otherwise, we get a default `dore`.
536−
564+
537−If we're the first routing candidate, or if we have don't have a lane to this
565+If we're the first routing candidate, or if we have don't have a lane to
538−candidate, then we skip this candidate and move on to the next one.
566+this candidate, then we skip this candidate and move on to the next one.
539−
567+
540−If we have only a provisional ip address, then we try to send on it, but we also
568+If we have only a provisional ip address, then we try to send on it, but
541−try to send on later routing candidates as well. Otherwise, we only send on this
569+we also try to send on later routing candidates as well. Otherwise, we
542−one candidate.
570+only send on this one candidate.
543−
571+
544−Finally, we create the actual `%ouzo` boon. The lane is the one from our `dore`.
572+Finally, we create the actual `%ouzo` boon. The lane is the one from our
545−If we're sending it directly to our intended recipient, and we haven't been told
573+`dore`. If we're sending it directly to our intended recipient, and we
546−to use a specific lane, then we just send the packet directly. Otherwise, we
574+haven't been told to use a specific lane, then we just send the packet
547−wrap it in a little `%fore` meal, telling the intermediary to whom we wish it to
575+directly. Otherwise, we wrap it in a little `%fore` meal, telling the
576+intermediary to whom we wish it to be sent. If we have already set up a
548−be sent. If we have already set up a symmetric key with the intermediary, then
577+symmetric key with the intermediary, then we encrypt it with that.
549−we encrypt it with that. Otherwise, we send it in the clear.
578+Otherwise, we send it in the clear.
550−
579+
551−Now, if you recall, we have traced all the way through from the beginning when,
580+Now, if you recall, we have traced all the way through from the
552−in `++knob`, the `%wont` card was handled by a call to `++wise`. There is only
581+beginning when, in `++knob`, the `%wont` card was handled by a call to
553−one more step before the packet is finally sent. Looking in `++knob`, we see
582+`++wise`. There is only one more step before the packet is finally sent.
554−that the resultant list of boons is passed into `++clop`, which will execute the
583+Looking in `++knob`, we see that the resultant list of boons is passed
584+into `++clop`, which will execute the correct actions and return a list
555−correct actions and return a list of moves. In `++clop`, we see the handling of
585+of moves. In `++clop`, we see the handling of each specific boon. The
556−each specific boon. The one we are interested in is `%ouzo`, since that is the
586+one we are interested in is `%ouzo`, since that is the only one we have
557−only one we have sent thus far.
587+sent thus far.
558
588
559 %ouzo
589 %ouzo
560 :: ~& [%send now p.bon `@p`(mug (shaf %flap q.bon))]
590 :: ~& [%send now p.bon `@p`(mug (shaf %flap q.bon))]
561 :_ fox
591 :_ fox
562 [[gad.fox [%give %send p.bon q.bon]] ~]
592 [[gad.fox [%give %send p.bon q.bon]] ~]
563
593
564−Very simply, we give a `%send` gift along the special duct that goes straight
594+Very simply, we give a `%send` gift along the special duct that goes
565−into the bowels of unix. This is the last stop before we drop into vere, and
595+straight into the bowels of unix. This is the last stop before we drop
566−later libuv. And then... the world.
596+into vere, and later libuv. And then... the world.
567
597
568−The packet, after its creation, embarks on a journey across physical time and
598+The packet, after its creation, embarks on a journey across physical
569−space into the great unknown. Hurtling through fiber-optic cables at hundreds of
599+time and space into the great unknown. Hurtling through fiber-optic
570−thousands of kilometers per second, it finally arrives at our neighbor's network
600+cables at hundreds of thousands of kilometers per second, it finally
601+arrives at our neighbor's network adapter. The adapter tells unix, unix
571−adapter. The adapter tells unix, unix tells libuv, libuv tells vere, and vere
602+tells libuv, libuv tells vere, and vere sends a `%hear` kiss to ames.
572−sends a `%hear` kiss to ames. And now we reenter the kernel.
603+And now we reenter the kernel.
573
604
574The `%hear` kiss goes straight to `++knob`, just as did the `%wont` kiss
605The `%hear` kiss goes straight to `++knob`, just as did the `%wont` kiss
575earlier.
606earlier.
@@ -577,13 +608,14 @@ earlier.
577 %hear
608 %hear
578 (~(gnaw am [now fox]) %good p.kyz q.kyz)
609 (~(gnaw am [now fox]) %good p.kyz q.kyz)
579
610
580−Here, though, we call `++gnaw:am` to process the packet. The arguments to
611+Here, though, we call `++gnaw:am` to process the packet. The arguments
581−`++gnaw` are the same as those to the `%hear` kiss: the lane on which the packet
612+to `++gnaw` are the same as those to the `%hear` kiss: the lane on which
582−was received and the packet itself. The other argument is just `%good`, which is
613+the packet was received and the packet itself. The other argument is
583−a `++cape` saying that we expect the packet to succeed. If a formal error
614+just `%good`, which is a `++cape` saying that we expect the packet to
584−occurs, then since we have a transactional event system, the `%hear` event will
615+succeed. If a formal error occurs, then since we have a transactional
616+event system, the `%hear` event will never be considered to have
585−never be considered to have actually happened, and unix will send a `%hole` kiss
617+actually happened, and unix will send a `%hole` kiss so that we may send
586−so that we may send a negative acknowledgment.
618+a negative acknowledgment.
587
619
588 ++ gnaw :: gnaw:am
620 ++ gnaw :: gnaw:am
589 |= [kay=cape ryn=lane pac=rock] :: process packet
621 |= [kay=cape ryn=lane pac=rock] :: process packet
@@ -598,10 +630,11 @@ so that we may send a negative acknowledgment.
598 %- ~(chew la:(ho:(um q.p.kec) p.p.kec) kay ryn %none (shaf %flap pac))
630 %- ~(chew la:(ho:(um q.p.kec) p.p.kec) kay ryn %none (shaf %flap pac))
599 [q.kec r.kec]
631 [q.kec r.kec]
600
632
601−First, we check the protocol number. If it is not correct, then we simply ignore
633+First, we check the protocol number. If it is not correct, then we
602−the packet entirely. Otherwise, we parse the packet with `++bite`, which
634+simply ignore the packet entirely. Otherwise, we parse the packet with
603−converts a packet atom into a `cake`, that is, a triple of the `sock` (pair of
635+`++bite`, which converts a packet atom into a `cake`, that is, a triple
604−sender and receiver), the `skin` (encryption type), and the data.
636+of the `sock` (pair of sender and receiver), the `skin` (encryption
637+type), and the data.
605
638
606 ++ bite :: packet to cake
639 ++ bite :: packet to cake
607 |= pac=rock ^- cake
640 |= pac=rock ^- cake
@@ -618,24 +651,26 @@ sender and receiver), the `skin` (encryption type), and the data.
618 (kins tay)
651 (kins tay)
619 (rsh 3 (add wix vix) bod)
652 (rsh 3 (add wix vix) bod)
620
653
621−This is exactly the inverse of `++spit`. Note that here we check both the
654+This is exactly the inverse of `++spit`. Note that here we check both
622−protocol number and the hash, crashing on error. Remember that a crash will
655+the protocol number and the hash, crashing on error. Remember that a
623−result in a negative acknowledgment being sent.
656+crash will result in a negative acknowledgment being sent.
624
657
625−Continuing in `++gnaw`, we see that if the intended recipient is not on our
658+Continuing in `++gnaw`, we see that if the intended recipient is not on
626−pier, then we drop the packet.
659+our pier, then we drop the packet.
627
660
628−If we've gotten this far, then we wish to process the packet. Recall that `++ho`
661+If we've gotten this far, then we wish to process the packet. Recall
629−and `++um` set up the domestic server and foreign client cores, respectively,
662+that `++ho` and `++um` set up the domestic server and foreign client
630−and that `++zork` and `++zank` resolve any changes to these cores.
663+cores, respectively, and that `++zork` and `++zank` resolve any changes
664+to these cores.
631
665
632−The new stuff here, then, is the `++la` core and the `++chew` arm. The `++la`
666+The new stuff here, then, is the `++la` core and the `++chew` arm. The
633−sets up a core for this particular packet, containing the current
667+`++la` sets up a core for this particular packet, containing the current
634−success/failure `cape`, the lane it was sent on, the encryption type, and a hash
668+success/failure `cape`, the lane it was sent on, the encryption type,
635−of the packet, used as an id.
669+and a hash of the packet, used as an id.
636
670
637−`++chew` is called with the encryption type and the message itself. It contains
671+`++chew` is called with the encryption type and the message itself. It
638−a little helper core inside of it, which starts immediately with `++apse`.
672+contains a little helper core inside of it, which starts immediately
673+with `++apse`.
639
674
640 ++ apse
675 ++ apse
641 ^+ +>.$
676 ^+ +>.$
@@ -652,23 +687,23 @@ a little helper core inside of it, which starts immediately with `++apse`.
652 :_(bin [%wine [our her] " is your neighbor"])
687 :_(bin [%wine [our her] " is your neighbor"])
653 +>.$
688 +>.$
654
689
655−First, we let `oub` be true if our neighbor hasn't been responding to us for
690+First, we let `oub` be true if our neighbor hasn't been responding to us
656−more than sixteen seconds. Let `neg` be true if we haven't yet proposed a
691+for more than sixteen seconds. Let `neg` be true if we haven't yet
657−symmetric key, meaning that we haven't yet corresponded with this ship, so they
692+proposed a symmetric key, meaning that we haven't yet corresponded with
658−are not our neighbor. Next, we run `++east`, which we'll go into in just a
693+this ship, so they are not our neighbor. Next, we run `++east`, which
659−minute.
694+we'll go into in just a minute.
660
695
661−We now do the same two checks and store the results in `eng` and `bou`. If our
696+We now do the same two checks and store the results in `eng` and `bou`.
662−neighbor has, like the prodigal son, returned after an extended absense, then we
697+If our neighbor has, like the prodigal son, returned after an extended
663−send a `%wine` boon as the proverbial fatted calf, which is simply printed out
698+absense, then we send a `%wine` boon as the proverbial fatted calf,
664−to the console. Likewise, if we are meeting one with whom we have never had the
699+which is simply printed out to the console. Likewise, if we are meeting
700+one with whom we have never had the pleasure of acquainting ourselves,
665−pleasure of acquainting ourselves, we send a message to the console to that
701+we send a message to the console to that effect.
666−effect.
667
702
668−We skipped over `++east`, which contains the meat of the processing. It first
703+We skipped over `++east`, which contains the meat of the processing. It
669−decrypts the message, then calls `++chow:la:ho:um:am` with the resultant meal.
704+first decrypts the message, then calls `++chow:la:ho:um:am` with the
670−We'll go through each of the four cases in turn, but first since each one calls
705+resultant meal. We'll go through each of the four cases in turn, but
671−`++bilk:pu`, we'll take a brief detour.
706+first since each one calls `++bilk:pu`, we'll take a brief detour.
672
707
673 ++ bilk :: bilk:pu
708 ++ bilk :: bilk:pu
674 |= now=@da :: inbound packet
709 |= now=@da :: inbound packet
@@ -680,10 +715,11 @@ We'll go through each of the four cases in turn, but first since each one calls
680 rtn ?~(puq ~ [~ (add now trt)])
715 rtn ?~(puq ~ [~ (add now trt)])
681 ==
716 ==
682
717
683−This updates the timing information in our packet pump. `rue`, the last time we
718+This updates the timing information in our packet pump. `rue`, the last
684−have heard from this neighbor, is set to now. `rto`, the retransmit timeout is
719+time we have heard from this neighbor, is set to now. `rto`, the
685−set to twice the current ping time, and if there is anything in the packet
720+retransmit timeout is set to twice the current ping time, and if there
686−queue, then we reset the next timeout, since we've just heard a message.
721+is anything in the packet queue, then we reset the next timeout, since
722+we've just heard a message.
687
723
688Back to `++east`.
724Back to `++east`.
689
725
@@ -691,11 +727,11 @@ Back to `++east`.
691 =. puz (bilk:puz now)
727 =. puz (bilk:puz now)
692 (chow ((hard meal) (cue msg)))
728 (chow ((hard meal) (cue msg)))
693
729
694−The simplest case is when the encryption type is `%none`. We first call `++bilk`
730+The simplest case is when the encryption type is `%none`. We first call
695−to update the packet pump, then we cue (unjam) the message into a meal. We hard
731+`++bilk` to update the packet pump, then we cue (unjam) the message into
696−cast it into a meal -- if the cast fails, then we do want to crash since someone
732+a meal. We hard cast it into a meal -- if the cast fails, then we do
697−is sending us malformed data. Finally, we send the result to `++chow` for
733+want to crash since someone is sending us malformed data. Finally, we
698−interpretation and handling.
734+send the result to `++chow` for interpretation and handling.
699
735
700 %fast
736 %fast
701 =+ [mag=`hand`(end 7 1 msg) bod=(rsh 7 1 msg)]
737 =+ [mag=`hand`(end 7 1 msg) bod=(rsh 7 1 msg)]
@@ -707,14 +743,15 @@ interpretation and handling.
707 =^ key diz u.dey
743 =^ key diz u.dey
708 (chow(aut sin) ((hard meal) (cue (dy:q:sen:gus key bod))))
744 (chow(aut sin) ((hard meal) (cue (dy:q:sen:gus key bod))))
709
745
710−For symmetric encryption, we first get the `hand`, which is the hash of the
746+For symmetric encryption, we first get the `hand`, which is the hash of
711−symmetric key. We pass it to `++kuch:lax:as:go`, which returns the key if we
747+the symmetric key. We pass it to `++kuch:lax:as:go`, which returns the
712−either have used it before or we have proposed it. If we have proposed it, then
748+key if we either have used it before or we have proposed it. If we have
713−we change its status from proposed to real. If `++kuch` fails, then we drop the
749+proposed it, then we change its status from proposed to real. If
714−packet and print out a `%bad-key` message.
750+`++kuch` fails, then we drop the packet and print out a `%bad-key`
751+message.
715
752
716−Otherwise, we call `++bilk` as before to update the packet pump and pass into
753+Otherwise, we call `++bilk` as before to update the packet pump and pass
717−`++chow` the decrypted data.
754+into `++chow` the decrypted data.
718
755
719 %full
756 %full
720 =+ mex=((hard ,[p=[p=life q=life] q=will r=@]) (cue msg))
757 =+ mex=((hard ,[p=[p=life q=life] q=will r=@]) (cue msg))
@@ -727,16 +764,17 @@ Otherwise, we call `++bilk` as before to update the packet pump and pass into
727 =. puz (bilk:puz now)
764 =. puz (bilk:puz now)
728 (west(msg q.mes))
765 (west(msg q.mes))
729
766
730−For sealed asymmetric encryption, we first take off the the layer of data that
767+For sealed asymmetric encryption, we first take off the the layer of
731−gives us the life and will of our neighbor, and we apply try to extend their
768+data that gives us the life and will of our neighbor, and we apply try
732−former will with the new data. `++deng` will fail if this is impossible.
769+to extend their former will with the new data. `++deng` will fail if
770+this is impossible.
733
771
734−Next, we get our most current understanding of our neighbor's crypto, and we
772+Next, we get our most current understanding of our neighbor's crypto,
735−verify that it's the same life as what they're sending. Then, we get our own
773+and we verify that it's the same life as what they're sending. Then, we
736−crypto from `++sev` and decrypt the message with the public key from our
774+get our own crypto from `++sev` and decrypt the message with the public
737−neighbor's crypto. We register the proposed symmetric key, update the packet
775+key from our neighbor's crypto. We register the proposed symmetric key,
738−pump, and call `++west`, which simply casts the message to a meal and calls
776+update the packet pump, and call `++west`, which simply casts the
739−`++chow`, reporting any error.
777+message to a meal and calls `++chow`, reporting any error.
740
778
741 %open
779 %open
742 =+ mex=((hard ,[p=[~ q=life] q=will r=@]) (cue msg))
780 =+ mex=((hard ,[p=[~ q=life] q=will r=@]) (cue msg))
@@ -747,13 +785,14 @@ pump, and call `++west`, which simply casts the message to a meal and calls
747 =. puz (bilk:puz now)
785 =. puz (bilk:puz now)
748 (west(msg mes))
786 (west(msg mes))
749
787
750−Finally, for signed asymmetric encryption, we, as before, take off the layer of
788+Finally, for signed asymmetric encryption, we, as before, take off the
751−data that gives us the life and will of our neighbor. This time, of course, we
789+layer of data that gives us the life and will of our neighbor. This
752−do not get our own crypto -- only that of our neighbor.
790+time, of course, we do not get our own crypto -- only that of our
791+neighbor.
753
792
754−The rest you have seen. We call `++deng` to extend the will, we verify that
793+The rest you have seen. We call `++deng` to extend the will, we verify
755−their crypto life is what we think it ought to be, we "decrypt" the data, we
794+that their crypto life is what we think it ought to be, we "decrypt" the
756−update the packet pump, and we call `++west` to call `++chow`.
795+data, we update the packet pump, and we call `++west` to call `++chow`.
757
796
758 ++ chow :: chow:la:ho:um:am
797 ++ chow :: chow:la:ho:um:am
759 |= fud=meal :: interpret meal
798 |= fud=meal :: interpret meal
@@ -761,14 +800,15 @@ update the packet pump, and we call `++west` to call `++chow`.
761 =. diz ?:(=(%none aut) diz (wast:diz ryn))
800 =. diz ?:(=(%none aut) diz (wast:diz ryn))
762 (dine fud)
801 (dine fud)
763
802
764−Here, if the message was encrypted at all, then we call `++wast:lax:as:go`,
803+Here, if the message was encrypted at all, then we call
765−which simply updates the lane (route) to our neighbor (unless we're given a
804+`++wast:lax:as:go`, which simply updates the lane (route) to our
805+neighbor (unless we're given a provisional route). This ensures that we
766−provisional route). This ensures that we always have the most direct possible
806+always have the most direct possible path to them.
767−path to them.
768
807
769−We've been handling this meal for so long, we've almost forgotten what we want
808+We've been handling this meal for so long, we've almost forgotten what
770−to do with it. The telos is of any meal to be dined on. We will choose out the
809+we want to do with it. The telos is of any meal to be dined on. We will
771−cases here that are important to our current investigation.
810+choose out the cases here that are important to our current
811+investigation.
772
812
773 %fore
813 %fore
774 =+ ^= lyn ^- lane
814 =+ ^= lyn ^- lane
@@ -781,12 +821,12 @@ cases here that are important to our current investigation.
781 =+ zid=(myx:gus p.fud)
821 =+ zid=(myx:gus p.fud)
782 (emir (wist:zid now xong:zid [~ lyn] r.fud))
822 (emir (wist:zid now xong:zid [~ lyn] r.fud))
783
823
784−Forwarding is the simplest case, since we've seen all the arms before, except
824+Forwarding is the simplest case, since we've seen all the arms before,
785−perhaps `++emit` and `++emir`, which simply take a boon or list of boons
825+except perhaps `++emit` and `++emir`, which simply take a boon or list
786−respectively and queue them up to be handled when the core resolves. If we're
826+of boons respectively and queue them up to be handled when the core
787−told to forward a packet to ourselves, then we emit a `%mead` boon which simply
827+resolves. If we're told to forward a packet to ourselves, then we emit a
788−sends another `%hear` kiss to ourselves with the data. Otherwise, we try to find
828+`%mead` boon which simply sends another `%hear` kiss to ourselves with
789−a route to the recipient, as before.
829+the data. Otherwise, we try to find a route to the recipient, as before.
790
830
791 %carp
831 %carp
792 =+ zol=(~(get by olz.weg) s.fud)
832 =+ zol=(~(get by olz.weg) s.fud)
@@ -815,30 +855,31 @@ a route to the recipient, as before.
815 +>.$(nys.weg (~(put by nys.weg) s.fud neb))
855 +>.$(nys.weg (~(put by nys.weg) s.fud neb))
816
856
817Here, we have received a partial message, and we're just assembling the
857Here, we have received a partial message, and we're just assembling the
818−individual packets into a message. Most of this code is fairly algorithmic, so
858+individual packets into a message. Most of this code is fairly
819−we'll just hit the high points. In the beginning, we check if we've already
859+algorithmic, so we'll just hit the high points. In the beginning, we
820−received this message, and if so, we resend the acknowledgment. Remember,
860+check if we've already received this message, and if so, we resend the
821−"always ack a dupe, never ack an ack".
861+acknowledgment. Remember, "always ack a dupe, never ack an ack".
822−
862+
823−In `nys.weg` we keep track of an incoming set of partial packets, indexed by the
863+In `nys.weg` we keep track of an incoming set of partial packets,
824−`flap` hash that comes with every packet. We check to see if we have already
864+indexed by the `flap` hash that comes with every packet. We check to see
825−received this partial message, and if so we acknowledge it. Otherwise, we put it
865+if we have already received this partial message, and if so we
866+acknowledge it. Otherwise, we put it in `nys.weg` unless this is the
826−in `nys.weg` unless this is the last message, in which case we ack the last
867+last message, in which case we ack the last partial message, move the
827−partial message, move the complete message into `olz.weg`, and call `++golf`,
868+complete message into `olz.weg`, and call `++golf`, which assembles the
828−which assembles the message and calls `++chew`, to start the dance again with
869+message and calls `++chew`, to start the dance again with the complete
829−the complete message.
870+message.
830
871
831 %bund
872 %bund
832 :: ~& [%bund q.fud r.fud]
873 :: ~& [%bund q.fud r.fud]
833 ?> =(p:sen:gus p.fud)
874 ?> =(p:sen:gus p.fud)
834 (deer q.fud r.fud ?-(kay %dead ~, %good [~ s.fud]))
875 (deer q.fud r.fud ?-(kay %dead ~, %good [~ s.fud]))
835
876
836−What if we're just receiving a regular old, garden variety message? We call
877+What if we're just receiving a regular old, garden variety message? We
837−`++deer` with the data from the message. If we already know that the message
878+call `++deer` with the data from the message. If we already know that
838−processing will fail (that is, if we got a `%hole` card from unix rather than a
879+the message processing will fail (that is, if we got a `%hole` card from
839−`%hear` card), then we don't even send the data at all. Remember, if a packet
880+unix rather than a `%hear` card), then we don't even send the data at
840−fails to process, it's as if it never even arrived, except that we send a
881+all. Remember, if a packet fails to process, it's as if it never even
841−negative acknowledgment.
882+arrived, except that we send a negative acknowledgment.
842
883
843 ++ deer :: deer:la:ho:um:am
884 ++ deer :: deer:la:ho:um:am
844 |= [cha=path num=@ud dut=(unit)] :: interpret message
885 |= [cha=path num=@ud dut=(unit)] :: interpret message
@@ -857,22 +898,24 @@ negative acknowledgment.
857 ==
898 ==
858 ==
899 ==
859
900
860−First, we get the race for this particular triple of sender, receiver, and path,
901+First, we get the race for this particular triple of sender, receiver,
861−creating it if it doesn't exist. If we've already acked the message, then we
902+and path, creating it if it doesn't exist. If we've already acked the
862−resend the ack. Note that `did.rum` is the number of packets we acknowledged,
903+message, then we resend the ack. Note that `did.rum` is the number of
863−positively or negatively while `bum.rum` is a map of message numbers to negative
904+packets we acknowledged, positively or negatively while `bum.rum` is a
905+map of message numbers to negative acknowledgments. Thus, if a message
864−acknowledgments. Thus, if a message number is less than `did.rum`, then if it's
906+number is less than `did.rum`, then if it's in `bum.rum` then it was
865−in `bum.rum` then it was negatively acknowledged, otherwise it's postively
907+negatively acknowledged, otherwise it's postively acknowledged. Thus, we
866−acknowledged. Thus, we are constant in space with the number of successful
908+are constant in space with the number of successful messages and linear
867−messages and linear in the number of failed messages. We'll document `++cook`
909+in the number of failed messages. We'll document `++cook` later on, but
868−later on, but suffice it to say that it sends an acknowledgment. It is to
910+suffice it to say that it sends an acknowledgment. It is to end-to-end
869−end-to-end acknowledgments what `++cock` is to packet-level acknowledgments.
911+acknowledgments what `++cock` is to packet-level acknowledgments.
870−
912+
871−If we are still processing a message (that is, `dod.rum` is false), then we
913+If we are still processing a message (that is, `dod.rum` is false), then
872−simply put this message in the map of misordered packets to be processed when
914+we simply put this message in the map of misordered packets to be
873−their time comes. "Processing a message" in this case means that we've received
915+processed when their time comes. "Processing a message" in this case
916+means that we've received the message and notified the correct
874−the message and notified the correct application, but we're still waiting for
917+application, but we're still waiting for the application-level
875−the application-level acknowledgment.
918+acknowledgment.
876
919
877Otherwise, we're ready for a packet, so we process it.
920Otherwise, we're ready for a packet, so we process it.
878
921
@@ -894,18 +937,20 @@ Otherwise, we're ready for a packet, so we process it.
894 u.s.u.cun
937 u.s.u.cun
895 ==
938 ==
896
939
897−First, we grab the message we want to process and store it in `cun`. If it's a
940+First, we grab the message we want to process and store it in `cun`. If
898−good packet, then we change `dod.rum` to false, meaning that we're in the middle
941+it's a good packet, then we change `dod.rum` to false, meaning that
899−of processing a packet and should not start processing another one. We also put
942+we're in the middle of processing a packet and should not start
943+processing another one. We also put a `%mulk` boon into the queue so
900−a `%mulk` boon into the queue so that, when it all resolves, we send a mesage to
944+that, when it all resolves, we send a mesage to the intended recipient
901−the intended recipient application. The boon contains the sender, the receiver,
945+application. The boon contains the sender, the receiver, the identity of
902−the identity of the message, and the message itself.
946+the message, and the message itself.
903−
947+
904−This bubbles up all the way back to `++knob`, where we were handling the `%hear`
948+This bubbles up all the way back to `++knob`, where we were handling the
905−card. Following the logic in `++knob`, we can see that the boons get sent into
949+`%hear` card. Following the logic in `++knob`, we can see that the boons
906−`++clop` to be turned into actual arvo-level moves. We've been here before, if
950+get sent into `++clop` to be turned into actual arvo-level moves. We've
907−you recall, when we handled the `%cake` boon to send a message. Now, we're
951+been here before, if you recall, when we handled the `%cake` boon to
952+send a message. Now, we're handling the `%mulk` boon, which is
908−handling the `%mulk` boon, which is unfortunately slightly more complicated.
953+unfortunately slightly more complicated.
909
954
910 %mulk
955 %mulk
911 :: ~& [%mulk p.bon q.bon]
956 :: ~& [%mulk p.bon q.bon]
@@ -935,48 +980,51 @@ handling the `%mulk` boon, which is unfortunately slightly more complicated.
935 :_ fox [hen %pass pax %g %roth p.bon app r.bon]~
980 :_ fox [hen %pass pax %g %roth p.bon app r.bon]~
936 ==
981 ==
937
982
938−We're dispatching messages based on the prefix of their path. Since only `%gall`
983+We're dispatching messages based on the prefix of their path. Since only
939−apps use end-to-end acknowledgments at the moment, every path must have at least
984+`%gall` apps use end-to-end acknowledgments at the moment, every path
940−two elements, and the first one must be `%q`. Beyond that, we handle the `/q/ge`
985+must have at least two elements, and the first one must be `%q`. Beyond
941−and `/q/gh` cases for gall requests and responses, respectively.
986+that, we handle the `/q/ge` and `/q/gh` cases for gall requests and
942−
987+responses, respectively.
988+
943−In both cases, we require the next term in the path to be the name of the
989+In both cases, we require the next term in the path to be the name of
944−intended recipient `%gall` app. Thus, a message to `/q/ge/talk` for example,
990+the intended recipient `%gall` app. Thus, a message to `/q/ge/talk` for
945−will send a message to the talk app.
991+example, will send a message to the talk app.
946−
992+
947−We then send a message to the app itself. The message is either a `%rote` or a
993+We then send a message to the app itself. The message is either a
948−`%roth` for a request and a response, respectively. The content is the `rook` or
994+`%rote` or a `%roth` for a request and a response, respectively. The
949−`roon` that was sent (stored in `r.bon`), but we don't actually handle that at
995+content is the `rook` or `roon` that was sent (stored in `r.bon`), but
996+we don't actually handle that at all here. That's completely a
950−all here. That's completely a `%gall`-level thing. We're just the messenger.
997+`%gall`-level thing. We're just the messenger.
951−
998+
952−Notice the path we send this over. We encode the sender, the receiver, and the
999+Notice the path we send this over. We encode the sender, the receiver,
953−path over which it was sent. This fully specifies the `race` so that when the
1000+and the path over which it was sent. This fully specifies the `race` so
954−app gives us the acknowledgment we know where to send it.
1001+that when the app gives us the acknowledgment we know where to send it.
955−
1002+
956−We now have another interlude. We have entrusted our precious data, so carefully
1003+We now have another interlude. We have entrusted our precious data, so
957−guarded and guided from the app on that far-away ship, to our local app. It has
1004+carefully guarded and guided from the app on that far-away ship, to our
958−the ability to do whatever it pleases with it. It may take a significant amount
1005+local app. It has the ability to do whatever it pleases with it. It may
959−of time to process. When the message has been handled by this app, though, it
1006+take a significant amount of time to process. When the message has been
1007+handled by this app, though, it must produce an acknowledgment. Our
960−must produce an acknowledgment. Our final task is to deliver this acknowledgment
1008+final task is to deliver this acknowledgment to the sending app.
961−to the sending app.
962
1009
963We should describe here what exactly these oft-mentioned acknowledgments
1010We should describe here what exactly these oft-mentioned acknowledgments
964−actually consist of. There are two kinds of acknowledgments: positive and
1011+actually consist of. There are two kinds of acknowledgments: positive
965−negative. A positive acknowledgment contains no data other than its existence. A
1012+and negative. A positive acknowledgment contains no data other than its
966−negative acknowledgment may optionally include a reason for said negativity.
1013+existence. A negative acknowledgment may optionally include a reason for
967−Formally, a negative acknowledgment is an `ares`, which is a unit pair of a term
1014+said negativity. Formally, a negative acknowledgment is an `ares`, which
968−and a list of tanks. If this is null, this is simply a failure with no
1015+is a unit pair of a term and a list of tanks. If this is null, this is
969−associated information. If the pair exists, the term is a short error code that
1016+simply a failure with no associated information. If the pair exists, the
1017+term is a short error code that is usually both human and computer
970−is usually both human and computer readable. For example, if you try to send a
1018+readable. For example, if you try to send a message to a valid `%gall`
971−message to a valid `%gall` app that doesn't have any `++poke` to handle it, then
1019+app that doesn't have any `++poke` to handle it, then `%gall` will give
972−`%gall` will give a negative acknowledgment with error term `%poke-find-fail`.
1020+a negative acknowledgment with error term `%poke-find-fail`. The list of
973−The list of tanks is a human-readable description of the error. This often
1021+tanks is a human-readable description of the error. This often contains
974−contains a stack trace. At any rate, all this information is returned to the
1022+a stack trace. At any rate, all this information is returned to the
975sending app on the other end of the wire.
1023sending app on the other end of the wire.
976
1024
977−After this brief interlude, our story resumes in `++knap`, where we receive
1025+After this brief interlude, our story resumes in `++knap`, where we
978−responses. In particular, a `%mean` indicates a negative acknowledgment while a
1026+receive responses. In particular, a `%mean` indicates a negative
979−`%nice` indicates a positive acknowledgment.
1027+acknowledgment while a `%nice` indicates a positive acknowledgment.
980
1028
981 ?(%mean %nice)
1029 ?(%mean %nice)
982 ?> ?=([@ @ @ *] tea)
1030 ?> ?=([@ @ @ *] tea)
@@ -994,10 +1042,11 @@ responses. In particular, a `%mean` indicates a negative acknowledgment while a
994 =^ toe fox (clop now hen i.p.fuy)
1042 =^ toe fox (clop now hen i.p.fuy)
995 $(p.fuy t.p.fuy, out (weld (flop toe) out))
1043 $(p.fuy t.p.fuy, out (weld (flop toe) out))
996
1044
997−Recall the format of the path we sent the message on, and you'll understand why
1045+Recall the format of the path we sent the message on, and you'll
998−`soq` and `pax` are the sender/receiver pair and path on which the message was
1046+understand why `soq` and `pax` are the sender/receiver pair and path on
999−sent. The rest of this is structured much like `++knob`, so we call `++rack:am`
1047+which the message was sent. The rest of this is structured much like
1048+`++knob`, so we call `++rack:am` and send the resulting boons to
1000−and send the resulting boons to `++clop`. Business as usual.
1049+`++clop`. Business as usual.
1001
1050
1002 ++ rack :: rack:am
1051 ++ rack :: rack:am
1003 |= [soq=sock cha=path cop=coop] :: e2e ack
1052 |= [soq=sock cha=path cop=coop] :: e2e ack
@@ -1005,10 +1054,11 @@ and send the resulting boons to `++clop`. Business as usual.
1005 =. oh (cook:oh cop cha ~)
1054 =. oh (cook:oh cop cha ~)
1006 (cans:oh cha)
1055 (cans:oh cha)
1007
1056
1008−First, we set up `++um` and `++ho`, as we've done twice before, for our domestic
1057+First, we set up `++um` and `++ho`, as we've done twice before, for our
1009−and foreign servers, respectively. The other two things are new, though. Well,
1058+domestic and foreign servers, respectively. The other two things are
1010−`++cook` is not actually new, but we delayed the explanation saying only that it
1059+new, though. Well, `++cook` is not actually new, but we delayed the
1011−sends an acknowledgment. The time has come.
1060+explanation saying only that it sends an acknowledgment. The time has
1061+come.
1012
1062
1013 ++ cook :: cook:ho:um:am
1063 ++ cook :: cook:ho:um:am
1014 |= [cop=coop cha=path ram=(unit ,[ryn=lane dam=flap])]
1064 |= [cop=coop cha=path ram=(unit ,[ryn=lane dam=flap])]
@@ -1027,58 +1077,61 @@ sends an acknowledgment. The time has come.
1027 =^ roc diz (zuul:diz now [%buck cop dam ~s0])
1077 =^ roc diz (zuul:diz now [%buck cop dam ~s0])
1028 (busk(diz (wast:diz ryn)) xong:diz roc)
1078 (busk(diz (wast:diz ryn)) xong:diz roc)
1029
1079
1030−If we are acknowledging a message that we have already acked, the `ram` will
1080+If we are acknowledging a message that we have already acked, the `ram`
1031−contain the new lane and flap to send the duplicate ack to. This happens if we
1081+will contain the new lane and flap to send the duplicate ack to. This
1032−call `++cook` in `++deer`, but it doesn't happen from `++rack`. If there is no
1082+happens if we call `++cook` in `++deer`, but it doesn't happen from
1033−message waiting to be acknowledged and we're not given an explicit lane and flap
1083+`++rack`. If there is no message waiting to be acknowledged and we're
1084+not given an explicit lane and flap (that is, we're not sending a
1034−(that is, we're not sending a duplicate ack), then the app must have sent us
1085+duplicate ack), then the app must have sent us multiple acknowledgments.
1035−multiple acknowledgments. We do the only sensible thing we can do and drop all
1086+We do the only sensible thing we can do and drop all acknowledgments
1036−acknowledgments after the first, printing a message. This is, in fact, an error,
1087+after the first, printing a message. This is, in fact, an error, so it
1037−so it could be argued that we ought to crash. Whatever you do, don't depend on
1088+could be argued that we ought to crash. Whatever you do, don't depend on
1038this not crashing.
1089this not crashing.
1039
1090
1040−First, we grab the race specified by the given path, and we get the most recent
1091+First, we grab the race specified by the given path, and we get the most
1041−in-order message, which must be the one which is being acknowledged.
1092+recent in-order message, which must be the one which is being
1042−
1093+acknowledged.
1094+
1043−Then, we decide which lane/flap to respond on/to. Basically, in the usual case
1095+Then, we decide which lane/flap to respond on/to. Basically, in the
1044−we respond on the lane through which the initial message was sent, which is
1096+usual case we respond on the lane through which the initial message was
1045−stored along with the other packet information in `mis.rum`, since it has to be
1097+sent, which is stored along with the other packet information in
1046−remembered across calls to ames. However, if we receive a duplicate message,
1098+`mis.rum`, since it has to be remembered across calls to ames. However,
1099+if we receive a duplicate message, then we must respond to the new
1047−then we must respond to the new message. It's quite possible the reason the
1100+message. It's quite possible the reason the other acknowledgment didn't
1048−other acknowledgment didn't get returned was that the lane between the ships was
1101+get returned was that the lane between the ships was broken.
1049−broken.
1102+
1050−
1051−At any rate, we update the race by saying that we've finished processing this
1103+At any rate, we update the race by saying that we've finished processing
1052−packet (unless we're sending a duplicate ack) and, if we're sending a negative
1104+this packet (unless we're sending a duplicate ack) and, if we're sending
1053−acknowledgment, putting the negative ack into `bum.rum` so that we can resend it
1105+a negative acknowledgment, putting the negative ack into `bum.rum` so
1054−if necessary.
1106+that we can resend it if necessary.
1055−
1107+
1056−We encode our new message, updating the packet pump, with `++zuul`, as before,
1108+We encode our new message, updating the packet pump, with `++zuul`, as
1057−and we send it off with `++busk`, routed via `++wast` to one of the ships in
1109+before, and we send it off with `++busk`, routed via `++wast` to one of
1058−`++xong`. Of course, in practice, we don't even look at the ships in `++xong`
1110+the ships in `++xong`. Of course, in practice, we don't even look at the
1059−because we already have a lane directly to our neighbor (the one over which they
1111+ships in `++xong` because we already have a lane directly to our
1060−sent their message to us).
1112+neighbor (the one over which they sent their message to us).
1061−
1113+
1062−We glossed over the actual message we're sending back. We're sending a `%buck`
1114+We glossed over the actual message we're sending back. We're sending a
1063−meal, which is an acknowledgment. The `cop` specifies whether this is a positive
1115+`%buck` meal, which is an acknowledgment. The `cop` specifies whether
1064−or a negative ack, `dam` specifies the message we're acknowledging, and the
1116+this is a positive or a negative ack, `dam` specifies the message we're
1065−`~s0` is a placeholder for the processing time required. This time is neither
1117+acknowledging, and the `~s0` is a placeholder for the processing time
1118+required. This time is neither calculated (though it is hopefully
1066−calculated (though it is hopefully obvious how to do so) nor used at present,
1119+obvious how to do so) nor used at present, but this information may be
1067−but this information may be used in the future for improved congestion control.
1120+used in the future for improved congestion control. Since the round-trip
1068−Since the round-trip time for an end-to-end acknowledged packet includes the
1121+time for an end-to-end acknowledged packet includes the processing time
1069−processing time on the other end, most common congestion control algorithms will
1122+on the other end, most common congestion control algorithms will stumble
1070−stumble when some messages take much longer to process than others. As noted,
1123+when some messages take much longer to process than others. As noted,
1071−though, this is simply an opportunity for improvement -- our congestion control
1124+though, this is simply an opportunity for improvement -- our congestion
1072−algorithms are relatively naive at the moment.
1125+control algorithms are relatively naive at the moment.
1073−
1126+
1074−Recall that `++busk` calls `++wist` to put the actual `%ouzo` boon in the queue,
1127+Recall that `++busk` calls `++wist` to put the actual `%ouzo` boon in
1075−which gets handled by `++clop` to actually send the message. This is the same
1128+the queue, which gets handled by `++clop` to actually send the message.
1076−pipeline as sending any other message, so we'll refer you to the explanation
1129+This is the same pipeline as sending any other message, so we'll refer
1077−above if you've forgotten it.
1130+you to the explanation above if you've forgotten it.
1078−
1131+
1079−The last thing we need to do on this ship is move on to the next packet in the
1132+The last thing we need to do on this ship is move on to the next packet
1080−queue if there is one. If you recall, in `++rack` after the call to `++cook`
1133+in the queue if there is one. If you recall, in `++rack` after the call
1081−there was a call to `++cans:ho:um:am`.
1134+to `++cook` there was a call to `++cans:ho:um:am`.
1082
1135
1083 ++ cans :: cans:ho:um:am
1136 ++ cans :: cans:ho:um:am
1084 |= cha=path
1137 |= cha=path
@@ -1090,37 +1143,38 @@ there was a call to `++cans:ho:um:am`.
1090 ==
1143 ==
1091 (coat cha rum)
1144 (coat cha rum)
1092
1145
1093−This is very simple. We increment the number of packets that we've acknowledged
1146+This is very simple. We increment the number of packets that we've
1094−on this race and we delete the packet that we just acknowledged from the set of
1147+acknowledged on this race and we delete the packet that we just
1095−misordered packets.
1148+acknowledged from the set of misordered packets.
1096
1149
1097Then, we call `++coat` again to process the next packet if we've already
1150Then, we call `++coat` again to process the next packet if we've already
1098received it. And that's it for this.
1151received it. And that's it for this.
1099
1152
1100−The acknowledgment now travels the same path that its forebearer, the original
1153+The acknowledgment now travels the same path that its forebearer, the
1101−message, once tread, but this time not into the great unknown. The weary
1154+original message, once tread, but this time not into the great unknown.
1102−traveler is seeking out its familial roots, finding the app from whom sprung
1155+The weary traveler is seeking out its familial roots, finding the app
1103−forth the original message way back in paragraph three. When it arrives at the
1156+from whom sprung forth the original message way back in paragraph three.
1104−network adapter of its ancestors, the adapter tells unix, unix tells libuv,
1157+When it arrives at the network adapter of its ancestors, the adapter
1158+tells unix, unix tells libuv, libuv tells vere, and vere sends a `%hear`
1105−libuv tells vere, and vere sends a `%hear` kiss to ames. Once more into the
1159+kiss to ames. Once more into the kernel.
1106−kernel.
1107
1160
1108−The `%hear` kiss is handled in `++knob` as before, leading to `++gnaw`, going
1161+The `%hear` kiss is handled in `++knob` as before, leading to `++gnaw`,
1109−over to `++chew`, `++apse`, `++chow`, and eventualy to `++dine`. We've seen most
1162+going over to `++chew`, `++apse`, `++chow`, and eventualy to `++dine`.
1110−of the cases in `++dine`, but we haven't yet looked at the handling of this
1163+We've seen most of the cases in `++dine`, but we haven't yet looked at
1111−`%buck` meal.
1164+the handling of this `%buck` meal.
1112
1165
1113 %buck
1166 %buck
1114 =. +> ?.(=(%full aut) +> cock) :: finish key exch
1167 =. +> ?.(=(%full aut) +> cock) :: finish key exch
1115 +>(..la (tock p.fud q.fud r.fud))
1168 +>(..la (tock p.fud q.fud r.fud))
1116
1169
1117−We send a packet level acknowledgment if we're finishing a key exchange, else we
1170+We send a packet level acknowledgment if we're finishing a key exchange,
1118−call `++tock` to process the acknowledgment.
1171+else we call `++tock` to process the acknowledgment.
1119
1172
1120−This will get a little involved, so if you don't much care about how exactly an
1173+This will get a little involved, so if you don't much care about how
1121−acknowledgment happens, just know that the result gets gifted as a `%woot` card
1174+exactly an acknowledgment happens, just know that the result gets gifted
1122−back to the app who sent it. For those brave souls who wish to see this thing
1175+as a `%woot` card back to the app who sent it. For those brave souls who
1123−through to the end, it's once more into the breach.
1176+wish to see this thing through to the end, it's once more into the
1177+breach.
1124
1178
1125 ++ tock :: tock:ho:um:am
1179 ++ tock :: tock:ho:um:am
1126 |= [cop=coop fap=flap cot=@dr] :: e2e ack by hash
1180 |= [cop=coop fap=flap cot=@dr] :: e2e ack by hash
@@ -1138,9 +1192,9 @@ through to the end, it's once more into the breach.
1138 ==
1192 ==
1139 (busk xong:diz q.yoh)
1193 (busk xong:diz q.yoh)
1140
1194
1141−We're going to work through this one a little backwards since it's mostly fairly
1195+We're going to work through this one a little backwards since it's
1142−simple except the call to `++bick:pu`. In fact, we'll just skip `++bick` for the
1196+mostly fairly simple except the call to `++bick:pu`. In fact, we'll just
1143−moment and finish the rest.
1197+skip `++bick` for the moment and finish the rest.
1144
1198
1145If `++bick` succesfully acks the message, then we call `++done`.
1199If `++bick` succesfully acks the message, then we call `++done`.
1146
1200
@@ -1156,35 +1210,37 @@ If `++bick` succesfully acks the message, then we call `++done`.
1156 (~(put by ryl.bah) cha rol(san (~(del by san.rol) num)))
1210 (~(put by ryl.bah) cha rol(san (~(del by san.rol) num)))
1157 ==
1211 ==
1158
1212
1159−This very simply gets the rill (the outgoing counterpart to a race, if you
1213+This very simply gets the rill (the outgoing counterpart to a race, if
1160−recall), pulls out of the map of outstanding messages the duct over which the
1214+you recall), pulls out of the map of outstanding messages the duct over
1161−original message was sent, and produces this duct while deleting that entry from
1215+which the original message was sent, and produces this duct while
1162−the map of outstanding messages.
1216+deleting that entry from the map of outstanding messages.
1163
1217
1164−Going back to `++tock`, we now have the duct we need to return the result over.
1218+Going back to `++tock`, we now have the duct we need to return the
1165−We do the very sensible thing and put a `%cake` boon in the queue to be
1219+result over. We do the very sensible thing and put a `%cake` boon in the
1166−processed later by `++clop`.
1220+queue to be processed later by `++clop`.
1167
1221
1168−In `q.yoh` we have a list of messages that may need to be sent, which we pass to
1222+In `q.yoh` we have a list of messages that may need to be sent, which we
1169−`++busk` to send, as usual. When an acknowledgment arrives, that may trigger
1223+pass to `++busk` to send, as usual. When an acknowledgment arrives, that
1170−other messages immediately. This often happens when sending more messages than
1224+may trigger other messages immediately. This often happens when sending
1171−the width of the logical window since for congestion control reasons another
1225+more messages than the width of the logical window since for congestion
1172−message cannot be sent until some of the earlier ones have been acknowledged.
1226+control reasons another message cannot be sent until some of the earlier
1227+ones have been acknowledged.
1173
1228
1174−We'll look at the processing of the `%cake` boon in `++clop` before we get back
1229+We'll look at the processing of the `%cake` boon in `++clop` before we
1175−to talking about `++bick`.
1230+get back to talking about `++bick`.
1176
1231
1177 %cake
1232 %cake
1178 :_ fox
1233 :_ fox
1179 :~ [s.bon %give %woot q.p.bon r.bon]
1234 :~ [s.bon %give %woot q.p.bon r.bon]
1180 ==
1235 ==
1181
1236
1182−We very simply give, along the duct we found above, a `%woot` card with the ship
1237+We very simply give, along the duct we found above, a `%woot` card with
1183−who sent us the ack and the ack itself. This allows the application to decide
1238+the ship who sent us the ack and the ack itself. This allows the
1184−what to do about the result. In case of a failure, we usually either resend the
1239+application to decide what to do about the result. In case of a failure,
1240+we usually either resend the message or display it to the user.
1185−message or display it to the user. Sometimes, we recognize the error term and
1241+Sometimes, we recognize the error term and handle it internally. In any
1186−handle it internally. In any case, the decision of how to handle the
1242+case, the decision of how to handle the acknowledgment is entirely up to
1187−acknowledgment is entirely up to the application. Our job is done.
1243+the application. Our job is done.
1188
1244
1189Well, except that we skipped `++bick:pu`. Let's go back to that.
1245Well, except that we skipped `++bick:pu`. Let's go back to that.
1190
1246
@@ -1199,18 +1255,19 @@ Well, except that we skipped `++bick:pu`. Let's go back to that.
1199 =^ yop +>.$ (harv now)
1255 =^ yop +>.$ (harv now)
1200 [[gub yop] +>.$]
1256 [[gub yop] +>.$]
1201
1257
1202−If you recall, in `++whap:pu` we created the packet pump's representation of the
1258+If you recall, in `++whap:pu` we created the packet pump's
1203−message, which included putting the message into `diq`, which maps from packet
1259+representation of the message, which included putting the message into
1204−hashes to packet sequence numbers. Thus, `u.sun` is the sequence number of this
1260+`diq`, which maps from packet hashes to packet sequence numbers. Thus,
1205−particular message.
1261+`u.sun` is the sequence number of this particular message.
1206
1262
1207−We delete this message from `diq` since we have now received an ack for it. We
1263+We delete this message from `diq` since we have now received an ack for
1208−call `++bock` to perform the ack by sequence number. We call `++harv` to harvest
1264+it. We call `++bock` to perform the ack by sequence number. We call
1209−the packet queue, sending any messages that are now able to be sent.
1265+`++harv` to harvest the packet queue, sending any messages that are now
1266+able to be sent.
1210
1267
1211−In `++bock`, there are three arms we haven't seen before: `++bine`, `+wept`, and
1268+In `++bock`, there are three arms we haven't seen before: `++bine`,
1212−`++beet`. We'll describe each of these before we get to `++bock`. `++bine` looks
1269+`+wept`, and `++beet`. We'll describe each of these before we get to
1213−scariest.
1270+`++bock`. `++bine` looks scariest.
1214
1271
1215 ++ bine :: bine:pu
1272 ++ bine :: bine:pu
1216 |= [now=@da num=@ud] :: apply ack
1273 |= [now=@da num=@ud] :: apply ack
@@ -1236,34 +1293,36 @@ scariest.
1236 :- gub
1293 :- gub
1237 +>.$(puq ~(nap to puq))
1294 +>.$(puq ~(nap to puq))
1238
1295
1239−The first few lines are simply looking through the packet queue until we find
1296+The first few lines are simply looking through the packet queue until we
1240−the correct packet to ack. This is basic queue manipulation that operates
1297+find the correct packet to ack. This is basic queue manipulation that
1241−directly on the treap structure of the queue. If you understand treap queues,
1298+operates directly on the treap structure of the queue. If you understand
1242−the logic is easy to follow. Otherwise, just trust us that by the time we get to
1299+treap queues, the logic is easy to follow. Otherwise, just trust us that
1243−the `=:`, the packet with sequence number `num` is on the top of the packet
1300+by the time we get to the `=:`, the packet with sequence number `num` is
1244−queue (that is, at `n.puq`).
1301+on the top of the packet queue (that is, at `n.puq`).
1245−
1302+
1246−We first update the round-trip time. If the packet is either not alive or had to
1303+We first update the round-trip time. If the packet is either not alive
1247−be transmitted more than once, then we don't have any reliable way of
1304+or had to be transmitted more than once, then we don't have any reliable
1248−calculating the round-trip time since we're unsure of exactly which transmission
1305+way of calculating the round-trip time since we're unsure of exactly
1249−was acknowledged. Otherwise, the round-trip time is the difference between now
1306+which transmission was acknowledged. Otherwise, the round-trip time is
1250−and when the packet was last sent. We set `rtt` by a little weighted average
1307+the difference between now and when the packet was last sent. We set
1308+`rtt` by a little weighted average where the previous smoothed RTT is
1251−where the previous smoothed RTT is weighted twice as much as the RTT of the
1309+weighted twice as much as the RTT of the current packet. Thus,
1252−current packet. Thus, `(2*rtt+gap)/3`. This gives us a nice smooth RTT that is
1310+`(2*rtt+gap)/3`. This gives us a nice smooth RTT that is somewhat
1253−somewhat resilient to outlier data while still being responsive to our
1311+resilient to outlier data while still being responsive to our
1254ever-changing world.
1312ever-changing world.
1255
1313
1256−If the packet wasn't already dead, then we decrement the number of live packets,
1314+If the packet wasn't already dead, then we decrement the number of live
1257−which may allow more packets to be sent.
1315+packets, which may allow more packets to be sent.
1258
1316
1259We decrement the number of unacknowledged packets in our `pyz` for this
1317We decrement the number of unacknowledged packets in our `pyz` for this
1260−particular message. If you recall, this was set in `++whap` to the number of
1318+particular message. If you recall, this was set in `++whap` to the
1261−packets required to send a message.
1319+number of packets required to send a message.
1262
1320
1263−If that was the last packet in the messge that needed to be acked, then we
1321+If that was the last packet in the messge that needed to be acked, then
1264−delete the messgae reference from `pyz` and produce the id of the message.
1322+we delete the messgae reference from `pyz` and produce the id of the
1265−Otherwise, we simply update `pyz` with the new number of unacked messages. In
1323+message. Otherwise, we simply update `pyz` with the new number of
1266−either case, we remove the packet from the packet queue.
1324+unacked messages. In either case, we remove the packet from the packet
1325+queue.
1267
1326
1268 ++ wept :: wept:pu
1327 ++ wept :: wept:pu
1269 |= [fip=@ud lap=@ud] :: fip thru lap-1
1328 |= [fip=@ud lap=@ud] :: fip thru lap-1
@@ -1287,12 +1346,12 @@ either case, we remove the packet from the packet queue.
1287 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
1346 ^+(. =+(rig=apse(puq r.puq) rig(puq [n.puq l.puq puq.rig])))
1288 --
1347 --
1289
1348
1290−The algorithm is a simple case of traversing the packet queue. Essentialy, we
1349+The algorithm is a simple case of traversing the packet queue.
1291−mark as dead all packets in the queue between `fip` and `(dec lap)`. We also
1350+Essentialy, we mark as dead all packets in the queue between `fip` and
1292−update `nif`, the number of live packets. Lest you mourn too much the passing of
1351+`(dec lap)`. We also update `nif`, the number of live packets. Lest you
1293−these packets, know that they shall soon rise again. Recall that in `++bick`
1352+mourn too much the passing of these packets, know that they shall soon
1294−after the call to `++bock` we call `++harv`. This will resend the packets that
1353+rise again. Recall that in `++bick` after the call to `++bock` we call
1295−have just been labeled dead.
1354+`++harv`. This will resend the packets that have just been labeled dead.
1296
1355
1297 ++ beet :: beet:pu
1356 ++ beet :: beet:pu
1298 ^+ . :: advance unacked
1357 ^+ . :: advance unacked
@@ -1304,9 +1363,10 @@ have just been labeled dead.
1304 =+ rig=$(puq r.puq)
1363 =+ rig=$(puq r.puq)
1305 ?^(rig rig [~ p.n.puq])
1364 ?^(rig rig [~ p.n.puq])
1306
1365
1307−Here we search for the next expected packet number. Basically, we search the
1366+Here we search for the next expected packet number. Basically, we search
1308−queue for the leftmost packet whose number is greater than the current `nep`. If
1367+the queue for the leftmost packet whose number is greater than the
1309−we don't find any such packet, we just use the total number of packets sent.
1368+current `nep`. If we don't find any such packet, we just use the total
1369+number of packets sent.
1310
1370
1311We can now dive into `++bock`, our last arm.
1371We can now dive into `++bock`, our last arm.
1312
1372
@@ -1327,29 +1387,32 @@ We can now dive into `++bock`, our last arm.
1327 :: ~& [%bock-fill num nif caw cag]
1387 :: ~& [%bock-fill num nif caw cag]
1328 +>.$
1388 +>.$
1329
1389
1330−First, we call `++bine` to apply the ack to the packet pump information. We
1390+First, we call `++bine` to apply the ack to the packet pump information.
1331−produce `gym`, which, if it exists, is the id of the packet that was acked. If
1391+We produce `gym`, which, if it exists, is the id of the packet that was
1332−we received an ack for a packet later than the one we expected, then we halve
1392+acked. If we received an ack for a packet later than the one we
1333−the logical packet window and kill all the earlier packets so that they may be
1393+expected, then we halve the logical packet window and kill all the
1334−resent.
1394+earlier packets so that they may be resent.
1335
1395
1336−Otherwise, we possibly increase the congestion window. If the window is less
1396+Otherwise, we possibly increase the congestion window. If the window is
1337−than the congestion threshold, then we increment the size of the window.
1397+less than the congestion threshold, then we increment the size of the
1338−Otherwise, we only increment one out of every `caw` times.
1398+window. Otherwise, we only increment one out of every `caw` times.
1339
1399
1340−If we received an ack for the packet we expected, then we simply advance `nep`
1400+If we received an ack for the packet we expected, then we simply advance
1341−with `++beet`. If we received an ack for a packet earlier than we expected, we
1401+`nep` with `++beet`. If we received an ack for a packet earlier than we
1342−do nothing.
1402+expected, we do nothing.
1343
1403
1344−It may be hard to believe, but we are, in fact, done. The message has been sent,
1404+It may be hard to believe, but we are, in fact, done. The message has
1345−received, acknowledged, and the acknowledgment has been returned to the original
1405+been sent, received, acknowledged, and the acknowledgment has been
1346−sender. We hope it's clear that, while the process has been somewhat involved,
1406+returned to the original sender. We hope it's clear that, while the
1407+process has been somewhat involved, the algorithms are not all that
1347−the algorithms are not all that complicated. If you've read this far, you know
1408+complicated. If you've read this far, you know `%ames`. The only other
1348−`%ames`. The only other code involves initialization, timeouts, and the like.
1409+code involves initialization, timeouts, and the like.
1349
1410
1350−Below, we give detailed reference documentation for the data models involved.
1411+Below, we give detailed reference documentation for the data models
1412+involved.
1351
1413
1352−## Data Models
1414+Data Models
1415+-----------
1353
1416
1354### `++fort`, formal state
1417### `++fort`, formal state
1355
1418
@@ -1361,27 +1424,28 @@ Below, we give detailed reference documentation for the data models involved.
1361 zac=(map ship corn) :: flows by server
1424 zac=(map ship corn) :: flows by server
1362 == ::
1425 == ::
1363
1426
1364−This is the state of our vane. Anything that must be remembered between calls to
1427+This is the state of our vane. Anything that must be remembered between
1365−ames must be stored in this state.
1428+calls to ames must be stored in this state.
1366
1429
1367−`%0` is the version of the ames state model itself. If the data model `++fort`
1430+`%0` is the version of the ames state model itself. If the data model
1368−changes, then this number needs to be incremented, and an adapter must be
1431+`++fort` changes, then this number needs to be incremented, and an
1369−written to upgrade the old state into the new state. Note that this is the
1432+adapter must be written to upgrade the old state into the new state.
1370−version number of the model itself, not the contents. When the data changes,
1433+Note that this is the version number of the model itself, not the
1371−there is of course no need to change this.
1434+contents. When the data changes, there is of course no need to change
1435+this.
1372
1436
1373`gad` is a `duct` over which we send `%send` cards to unix. This card is
1437`gad` is a `duct` over which we send `%send` cards to unix. This card is
1374−initialized when unix sends a `%barn` card as vere starts up. Vere treats this
1438+initialized when unix sends a `%barn` card as vere starts up. Vere
1375−duct specially -- don't send anything weird over it.
1439+treats this duct specially -- don't send anything weird over it.
1376
1440
1377−`hop` is the network boot date. This is set when the `%kick` card is sent by
1441+`hop` is the network boot date. This is set when the `%kick` card is
1378−vere on start up.
1442+sent by vere on start up.
1379
1443
1380−`ton` is a `++town`, where we store all of our security/encryption state. Note
1444+`ton` is a `++town`, where we store all of our security/encryption
1381−that this is shared across all ships on a pier.
1445+state. Note that this is shared across all ships on a pier.
1382
1446
1383−`zac` is a map of ships to `++corn`. This stores all the per-ship state. The
1447+`zac` is a map of ships to `++corn`. This stores all the per-ship state.
1384−keys to this map are the ships on the current pier.
1448+The keys to this map are the ships on the current pier.
1385
1449
1386### `++town`, all security state
1450### `++town`, all security state
1387
1451
@@ -1396,17 +1460,20 @@ This is the security state of our pier.
1396
1460
1397`lit` is unused.
1461`lit` is unused.
1398
1462
1399−`any` is 256 bits of entropy. This entropy is used and updated in exactly two
1463+`any` is 256 bits of entropy. This entropy is used and updated in
1400−places: when we send a `%junk` card, and when we generate a new symmetric key in
1464+exactly two places: when we send a `%junk` card, and when we generate a
1401−`++griz:lax:as:go`. When it is updated, it is updated by a SHA-256 hash of the
1465+new symmetric key in `++griz:lax:as:go`. When it is updated, it is
1402−current time and the old value of the entropy.
1466+updated by a SHA-256 hash of the current time and the old value of the
1467+entropy.
1403
1468
1404−`urb` is a map of ships to `++sufi`. This is where we store all the per-ship
1469+`urb` is a map of ships to `++sufi`. This is where we store all the
1405−state for the pier. The keys to this map are the ships on the current pier.
1470+per-ship state for the pier. The keys to this map are the ships on the
1471+current pier.
1406
1472
1407−`fak` is true if we are on a fake network. This disables certain security checks
1473+`fak` is true if we are on a fake network. This disables certain
1408−so that anyone may run a fake `~zod`. This is used only for development. To use,
1474+security checks so that anyone may run a fake `~zod`. This is used only
1475+for development. To use, run vere with the `-F` option (and the
1409−run vere with the `-F` option (and the `-I ~zod` option for a fake `~zod`).
1476+`-I ~zod` option for a fake `~zod`).
1410
1477
1411### `++sufi`, domestic host
1478### `++sufi`, domestic host
1412
1479
@@ -1420,8 +1487,9 @@ run vere with the `-F` option (and the `-I ~zod` option for a fake `~zod`).
1420
1487
1421This is the security state of a domestic server.
1488This is the security state of a domestic server.
1422
1489
1423−`hoy` is a list of the ships directly above us in the hierarchy of ships. For
1490+`hoy` is a list of the ships directly above us in the hierarchy of
1424−example, for `~hoclur-bicrel`, this would be `~tasruc` and `~tug`. See `++sein`.
1491+ships. For example, for `~hoclur-bicrel`, this would be `~tasruc` and
1492+`~tug`. See `++sein`.
1425
1493
1426`val` is a list of our private keys.
1494`val` is a list of our private keys.
1427
1495
@@ -1429,51 +1497,51 @@ example, for `~hoclur-bicrel`, this would be `~tasruc` and `~tug`. See `++sein`.
1429
1497
1430`seh`
1498`seh`
1431
1499
1432−`hoc` is a map of ships to `++dore`. The stores all the security informatoin
1500+`hoc` is a map of ships to `++dore`. The stores all the security
1433−about foreign ships. The keys to this map are the neighbors (ships we have been
1501+informatoin about foreign ships. The keys to this map are the neighbors
1434−in contact with) of this domestic server.
1502+(ships we have been in contact with) of this domestic server.
1435
1503
1436### `++wund`, private keys
1504### `++wund`, private keys
1437
1505
1438 ++ wund (list ,[p=life q=ring r=acru]) :: mace in action
1506 ++ wund (list ,[p=life q=ring r=acru]) :: mace in action
1439
1507
1440−This is a list of our own private keys, indexed by life. The key itself is the
1508+This is a list of our own private keys, indexed by life. The key itself
1441−`++ring`, and the `++acru` is the encryption engine. We generate the `++acru`
1509+is the `++ring`, and the `++acru` is the encryption engine. We generate
1442−from the private key by calling `++weur`. Thus, we can at any time regenerate
1510+the `++acru` from the private key by calling `++weur`. Thus, we can at
1443−our `++wund` from a `++mace`. The current crypto is at the head of the list and
1511+any time regenerate our `++wund` from a `++mace`. The current crypto is
1444−can be accessed with `++sen:as:go`.
1512+at the head of the list and can be accessed with `++sen:as:go`.
1445
1513
1446### `++ring`, private key
1514### `++ring`, private key
1447
1515
1448 ++ ring ,@ :: private key
1516 ++ ring ,@ :: private key
1449
1517
1450−This is a private key. The first byte is reserved to identify the type of
1518+This is a private key. The first byte is reserved to identify the type
1451−cryptography. Lower-case means public key, upper-case means public key, and the
1519+of cryptography. Lower-case means public key, upper-case means public
1452−letter identifies which `++acru` to use.
1520+key, and the letter identifies which `++acru` to use.
1453
1521
1454### `++pass`, public key
1522### `++pass`, public key
1455
1523
1456 ++ pass ,@ :: public key
1524 ++ pass ,@ :: public key
1457
1525
1458This is a public key. The first byte is reserved to identify the type of
1526This is a public key. The first byte is reserved to identify the type of
1459−cryptography. Lower-case means public key, upper-case means public key, and the
1527+cryptography. Lower-case means public key, upper-case means public key,
1460−letter identifies which `++acru` to use.
1528+and the letter identifies which `++acru` to use.
1461
1529
1462### `++mace`, private secrets
1530### `++mace`, private secrets
1463
1531
1464 ++ mace (list ,[p=life q=ring]) :: private secrets
1532 ++ mace (list ,[p=life q=ring]) :: private secrets
1465
1533
1466−This is a list of the our private keys, indexed by life. From this we can
1534+This is a list of the our private keys, indexed by life. From this we
1467−generate a `++wund` for actual use.
1535+can generate a `++wund` for actual use.
1468
1536
1469### `++skin`, encoding stem
1537### `++skin`, encoding stem
1470
1538
1471 ++ skin ?(%none %open %fast %full) :: encoding stem
1539 ++ skin ?(%none %open %fast %full) :: encoding stem
1472
1540
1473−This defines the type of encryption used for each message. `%none` refers to
1541+This defines the type of encryption used for each message. `%none`
1474−messages sent in the clear, `%open` refers to signed messages, `%full` refers to
1542+refers to messages sent in the clear, `%open` refers to signed messages,
1475−sealed messages, and `%fast` refers to symmetrically encrypted messages. See
1543+`%full` refers to sealed messages, and `%fast` refers to symmetrically
1476−`++acru` for details.
1544+encrypted messages. See `++acru` for details.
1477
1545
1478### `++acru`, asymmetric cryptosuite
1546### `++acru`, asymmetric cryptosuite
1479
1547
@@ -1502,11 +1570,11 @@ sealed messages, and `%fast` refers to symmetrically encrypted messages. See
1502 --
1570 --
1503 --
1571 --
1504
1572
1505−This is an opaque interface for a general asymmetric cryptosuite. Any form of
1573+This is an opaque interface for a general asymmetric cryptosuite. Any
1506−asymmetric cryptography can be dropped in to be used instead of the default.
1574+form of asymmetric cryptography can be dropped in to be used instead of
1507−Right now, there are two cryptosuites, `++crua`, which is your standard RSA, and
1575+the default. Right now, there are two cryptosuites, `++crua`, which is
1508−`++crub`, which is elliptic curve crypto but is mostly stubbed out at the
1576+your standard RSA, and `++crub`, which is elliptic curve crypto but is
1509−moment.
1577+mostly stubbed out at the moment.
1510
1578
1511#### `++as:acru`, asymmetric operations
1579#### `++as:acru`, asymmetric operations
1512
1580
@@ -1518,28 +1586,31 @@ moment.
1518 *(unit ,[p=@ q=@]) ::
1586 *(unit ,[p=@ q=@]) ::
1519 -- ::
1587 -- ::
1520
1588
1521−This is the core that defines the standard asymmetric cryptography operations.
1589+This is the core that defines the standard asymmetric cryptography
1522−
1590+operations.
1591+
1523−`++seal:as:acru` allows us to send a message encrypted with someone's public key
1592+`++seal:as:acru` allows us to send a message encrypted with someone's
1524−so that only they may read it. If Alice seals a message with Bob's public key,
1593+public key so that only they may read it. If Alice seals a message with
1525−then she can be sure that Bob is the only one who can read it. This is
1594+Bob's public key, then she can be sure that Bob is the only one who can
1595+read it. This is associated with the `++skin` `%full`.
1596+
1597+`++sign:as:acru` allows us to sign a message with our private key so
1598+that others can verify that we sent the message. If Alice signs a
1599+message with her private key, then Bob can verify with her public key
1600+that it was indeed Alice who sent it. This is associated with the
1601+`++skin` `%open`.
1602+
1603+`++sure:as:acru` is the dual to `++sign:as:acru`. It allows us to verify
1604+that a message we have received is indeed from the claimed sender. If
1605+Alice sends a message with her private key, then Bob can use this arm to
1606+verify that it was indeed Alice who sent it. This is associated with the
1607+`++skin` `%open`.
1608+
1609+`++tear:as:acru` is the dual to `++seal:as:acru`. It allows us to read a
1610+message that we can be sure is only read by us. If Alice seals a message
1611+with Bob's public key, then Bob can use this arm to read it. This is
1526associated with the `++skin` `%full`.
1612associated with the `++skin` `%full`.
1527
1613
1528−`++sign:as:acru` allows us to sign a message with our private key so that others
1529−can verify that we sent the message. If Alice signs a message with her private
1530−key, then Bob can verify with her public key that it was indeed Alice who sent
1531−it. This is associated with the `++skin` `%open`.
1532−
1533−`++sure:as:acru` is the dual to `++sign:as:acru`. It allows us to verify that a
1534−message we have received is indeed from the claimed sender. If Alice sends a
1535−message with her private key, then Bob can use this arm to verify that it was
1536−indeed Alice who sent it. This is associated with the `++skin` `%open`.
1537−
1538−`++tear:as:acru` is the dual to `++seal:as:acru`. It allows us to read a message
1539−that we can be sure is only read by us. If Alice seals a message with Bob's
1540−public key, then Bob can use this arm to read it. This is associated with the
1541−`++skin` `%full`.
1542−
1543#### `++de:acru`, `++dy:acru`, and `++en:acru`, symmetric encryption/decryption
1614#### `++de:acru`, `++dy:acru`, and `++en:acru`, symmetric encryption/decryption
1544
1615
1545 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
1616 ++ de |+([a=@ b=@] *(unit ,@)) :: symmetric de, soft
@@ -1548,12 +1619,12 @@ public key, then Bob can use this arm to read it. This is associated with the
1548
1619
1549Symmetric encryption is associated with the `++skin` `%fast`.
1620Symmetric encryption is associated with the `++skin` `%fast`.
1550
1621
1551−`++de:acru` decrypts a message with a symmetric key, returning `~` on failure
1622+`++de:acru` decrypts a message with a symmetric key, returning `~` on
1552−and `[~ u=data]` on success.
1623+failure and `[~ u=data]` on success.
1553
1624
1554−`++dy:acru` decrypts a message with a symmetric key, crashing on failure. This
1625+`++dy:acru` decrypts a message with a symmetric key, crashing on
1555−should almost always be defined as, and should always be semantically equivalent
1626+failure. This should almost always be defined as, and should always be
1556−to, `(need (de a b))`.
1627+semantically equivalent to, `(need (de a b))`.
1557
1628
1558`++en:acru` encrypts a message with a symmetric key.
1629`++en:acru` encrypts a message with a symmetric key.
1559
1630
@@ -1566,9 +1637,10 @@ to, `(need (de a b))`.
1566 ++ sec *ring :: private key
1637 ++ sec *ring :: private key
1567 --
1638 --
1568
1639
1569−`++fig:ex:acru` is our fingerprint, usually a hash of our public key. This is
1640+`++fig:ex:acru` is our fingerprint, usually a hash of our public key.
1570−used, for example, in `++zeno`, where every carrier owner's fingerprint is
1641+This is used, for example, in `++zeno`, where every carrier owner's
1571−stored so that we can ensure that carriers are indeed owned by their owners
1642+fingerprint is stored so that we can ensure that carriers are indeed
1643+owned by their owners
1572
1644
1573`++pac:ex:acru` is our default passcode, which is unused at present.
1645`++pac:ex:acru` is our default passcode, which is unused at present.
1574
1646
@@ -1586,45 +1658,51 @@ stored so that we can ensure that carriers are indeed owned by their owners
1586
1658
1587These arms allow us to reconstruct a `++acru` from basic data.
1659These arms allow us to reconstruct a `++acru` from basic data.
1588
1660
1589−`++pit:nu:acru` constructs a `++acru` from the width of our intended key and
1661+`++pit:nu:acru` constructs a `++acru` from the width of our intended key
1590−seed entropy. This is usually used in the initial construction of the `++acru`.
1662+and seed entropy. This is usually used in the initial construction of
1663+the `++acru`.
1591
1664
1592−`++nol:nu:acru` constructs a `++acru` from a "naked ring", meaning a `++ring`
1665+`++nol:nu:acru` constructs a `++acru` from a "naked ring", meaning a
1593−without the initial byte identifying the type of crypto. There is often a helper
1666+`++ring` without the initial byte identifying the type of crypto. There
1594−arm that that wraps this; see `++weur` for `++crua` and `++wear` for `++crub`.
1667+is often a helper arm that that wraps this; see `++weur` for `++crua`
1668+and `++wear` for `++crub`.
1595
1669
1596−`++com:nu:acru` constructs a `++acru` from a "naked pass", meaning a `++ring`
1670+`++com:nu:acru` constructs a `++acru` from a "naked pass", meaning a
1597−without the initial byte identifying the type of crypto. There is often a helper
1671+`++ring` without the initial byte identifying the type of crypto. There
1598−arm that that wraps this; see `++haul` for `++crua` and `++hail` for `++crub`.
1672+is often a helper arm that that wraps this; see `++haul` for `++crua`
1673+and `++hail` for `++crub`.
1599
1674
1600### `++will`, certificate
1675### `++will`, certificate
1601
1676
1602 ++ will (list deed) :: certificate
1677 ++ will (list deed) :: certificate
1603
1678
1604−This is a list of deeds associated with the current ship. There should be an
1679+This is a list of deeds associated with the current ship. There should
1605−item in this list for every ship from this point up in the hierarchy times the
1680+be an item in this list for every ship from this point up in the
1606−number of lives that each ship has had. For example, ~hoclur-bicrel may have a
1681+hierarchy times the number of lives that each ship has had. For example,
1607−will with three items: one for itself, one for ~tasruc (who issued
1682+\~hoclur-bicrel may have a will with three items: one for itself, one
1608−~hoclur-bicrel's deed) and one for ~tug (who issued ~tasruc's deed).
1683+for \~tasruc (who issued \~hoclur-bicrel's deed) and one for \~tug (who
1684+issued \~tasruc's deed).
1609
1685
1610### `++deed`, identity
1686### `++deed`, identity
1611
1687
1612 ++ deed ,[p=@ q=step r=?] :: sig, stage, fake?
1688 ++ deed ,[p=@ q=step r=?] :: sig, stage, fake?
1613
1689
1614−`p` is the signature of a particular deed, which is a signed copy of `q`.
1690+`p` is the signature of a particular deed, which is a signed copy of
1691+`q`.
1615
1692
1616`q` is the stage in the identity.
1693`q` is the stage in the identity.
1617
1694
1618−`r` is true if we're working on a fake network, where we don't check that the
1695+`r` is true if we're working on a fake network, where we don't check
1619−carrier fingerprints are correct. This allows us to create fake networks for
1696+that the carrier fingerprints are correct. This allows us to create fake
1620−development without interfering with the real network.
1697+networks for development without interfering with the real network.
1621
1698
1622### `++step`, identity stage
1699### `++step`, identity stage
1623
1700
1624 ++ step ,[p=bray q=gens r=pass] :: identity stage
1701 ++ step ,[p=bray q=gens r=pass] :: identity stage
1625
1702
1626−This is a single stage in our identity. Thus, this is specific to a single life
1703+This is a single stage in our identity. Thus, this is specific to a
1627−in a single ship. Everything in here may change between lives.
1704+single life in a single ship. Everything in here may change between
1705+lives.
1628
1706
1629`p`
1707`p`
1630
1708
@@ -1642,9 +1720,9 @@ XXX
1642
1720
1643 ++ gens ,[p=lang q=gcos] :: general identity
1721 ++ gens ,[p=lang q=gcos] :: general identity
1644
1722
1645−`p` is the IETF language code for the preferred language of this identity. This
1723+`p` is the IETF language code for the preferred language of this
1646−is unused at the moment, but in the future text should be localized based on
1724+identity. This is unused at the moment, but in the future text should be
1647−this.
1725+localized based on this.
1648
1726
1649`q` is the description of the ship.
1727`q` is the description of the ship.
1650
1728
@@ -1658,31 +1736,32 @@ this.
1658 [%pawn p=(unit ,@t)] :: 128-bit ship
1736 [%pawn p=(unit ,@t)] :: 128-bit ship
1659 == ::
1737 == ::
1660
1738
1661−This is the description of the identity of a ship. Most types of identity have a
1739+This is the description of the identity of a ship. Most types of
1662−`@t` field, which is their human-readable name. The identity of a `%duke` is
1740+identity have a `@t` field, which is their human-readable name. The
1663−more involved.
1741+identity of a `%duke` is more involved.
1664
1742
1665−A `%czar`, a carrier, is a ship with an 8-bit address. Thus, there are only 256
1743+A `%czar`, a carrier, is a ship with an 8-bit address. Thus, there are
1666−carriers. These are at the top of the namespace hierarchy, and the fingerprint
1744+only 256 carriers. These are at the top of the namespace hierarchy, and
1667−of each carrier is stored in `++zeno`. These are the "senators" of Urbit.
1745+the fingerprint of each carrier is stored in `++zeno`. These are the
1746+"senators" of Urbit.
1668
1747
1669−A `%king`, a cruiser, is a ship with a 16-bit address. Thus, there are 65,536
1748+A `%king`, a cruiser, is a ship with a 16-bit address. Thus, there are
1670−cruisers. Each carrier may issue 256 cruisers. These are the infrastructure of
1749+65,536 cruisers. Each carrier may issue 256 cruisers. These are the
1671−Urbit.
1750+infrastructure of Urbit.
1672
1751
1673A `%duke`, a destroyer, is a ship with a 32-bit address. Thus, there are
1752A `%duke`, a destroyer, is a ship with a 32-bit address. Thus, there are
1674−4,294,967,296 destroyers. Each cruiser may issue 65,536 cruisers. These are the
1753+4,294,967,296 destroyers. Each cruiser may issue 65,536 cruisers. These
1675−individuals of Urbit.
1754+are the individuals of Urbit.
1676
1755
1677A `%earl`, a yacht, is a ship with a 64-bit address. Thus, there are
1756A `%earl`, a yacht, is a ship with a 64-bit address. Thus, there are
1678−18,446,744,073,709,551,616 yachts. Each destroyer may issue 4,294,967,296
1757+18,446,744,073,709,551,616 yachts. Each destroyer may issue
1679−yachts. These are the devices of Urbit.
1758+4,294,967,296 yachts. These are the devices of Urbit.
1680
1759
1681−A `%pawn`, a submarine, is a ship with a 128-bit address. Thus, there are a lot
1760+A `%pawn`, a submarine, is a ship with a 128-bit address. Thus, there
1682−of submarines. The chance of random name collision is negligible, so submarines
1761+are a lot of submarines. The chance of random name collision is
1683−are not issued by any ship. They must simply assert their presence, and they are
1762+negligible, so submarines are not issued by any ship. They must simply
1684−all considered children of ~zod. This is the underworld of Urbit, where
1763+assert their presence, and they are all considered children of \~zod.
1685−anonymity reigns supreme.
1764+This is the underworld of Urbit, where anonymity reigns supreme.
1686
1765
1687### `++what`, logical destroyer identity
1766### `++what`, logical destroyer identity
1688
1767
@@ -1695,10 +1774,10 @@ anonymity reigns supreme.
1695
1774
1696This is the logical identity of a destroyer.
1775This is the logical identity of a destroyer.
1697
1776
1698−A `%anon` is a completely anonymous destroyer. The difference between this and a
1777+A `%anon` is a completely anonymous destroyer. The difference between
1699−submarine is that a submarine is ephemeral while a `%anon` destroyer is not.
1778+this and a submarine is that a submarine is ephemeral while a `%anon`
1700−Thus, we may not know who ~hoclur-bicrel is, but we do know that it's always the
1779+destroyer is not. Thus, we may not know who \~hoclur-bicrel is, but we
1701−same person.
1780+do know that it's always the same person.
1702
1781
1703A `%lady` is a female person. The name used here should be a real name.
1782A `%lady` is a female person. The name used here should be a real name.
1704
1783
@@ -1710,8 +1789,8 @@ A `%punk` is a person who is identified only by a handle.
1710
1789
1711 ++ whom ,[p=@ud q=govt r=sect s=name] :: year/govt/id
1790 ++ whom ,[p=@ud q=govt r=sect s=name] :: year/govt/id
1712
1791
1713−Ths is the information associated with a real person. It is mostly information
1792+Ths is the information associated with a real person. It is mostly
1714−that could be observed with the briefest of interactions.
1793+information that could be observed with the briefest of interactions.
1715
1794
1716`p` is the birth year.
1795`p` is the birth year.
1717
1796
@@ -1737,5 +1816,5 @@ XXX
1737
1816
1738 ++ name ,[p=@t q=(unit ,@t) r=(unit ,@t) s=@t] :: first mid/nick last
1817 ++ name ,[p=@t q=(unit ,@t) r=(unit ,@t) s=@t] :: first mid/nick last
1739
1818
1740−This is the given name, possible middle name/initial, possible nickname, and
1819+This is the given name, possible middle name/initial, possible nickname,
1741−surname of a user.
1820+and surname of a user.
docs/arvo/internals/behn.md+14−12
@@ -1,20 +1,22 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 2
3sort: 2
4---
4---
5
5
6−# `%behn`
6+`%behn`
7+=======
7
8
8Our simple timer.
9Our simple timer.
9
10
10−It allows vanes and applications to set and timer events, which are managed in a
11+It allows vanes and applications to set and timer events, which are
11−simple priority queue. `%behn` produces effects to start the unix timer, and
12+managed in a simple priority queue. `%behn` produces effects to start
12−when the requested `%behn` passes, unix sends wake events to `%behn`, which time
13+the unix timer, and when the requested `%behn` passes, unix sends wake
13−routes back to original sender. We don't guarantee that a timer event will
14+events to `%behn`, which time routes back to original sender. We don't
15+guarantee that a timer event will happen at exactly the `%behn` it was
14−happen at exactly the `%behn` it was set for, or even that it'll be particularly
16+set for, or even that it'll be particularly close. A timer event is a
15−close. A timer event is a request to not be woken until after the given time.
17+request to not be woken until after the given time.
16
18
17−`%eyre` uses `%behn` for timing out sessions, and `%clay` uses `%behn` for
19+`%eyre` uses `%behn` for timing out sessions, and `%clay` uses `%behn`
18−keeping track of time-specified file requests. `%ames` should probably use
20+for keeping track of time-specified file requests. `%ames` should
19−`%behn` to keep track of things like network timeouts and retry timing, but it
21+probably use `%behn` to keep track of things like network timeouts and
20−currently uses its own alarm system.
22+retry timing, but it currently uses its own alarm system.
docs/arvo/internals/clay.md+16−13
@@ -1,23 +1,26 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 3
3sort: 3
4---
4---
5
5
6−# `%clay`
6+`%clay`
7+=======
7
8
8Our filesystem.
9Our filesystem.
9
10
10−`%clay` is version-controlled, referentially-transparent, and global. While this
11+`%clay` is version-controlled, referentially-transparent, and global.
11−filesystem is stored in `%clay`, it is mirrored to Unix for convenience. Unix
12+While this filesystem is stored in `%clay`, it is mirrored to Unix for
12−tells `%clay`s whenever a file changes in the Unix copy of the filesystem so
13+convenience. Unix tells `%clay`s whenever a file changes in the Unix
13−that the change may be applied. `%clay` tells unix whenever an app or vane
14+copy of the filesystem so that the change may be applied. `%clay` tells
14−changes the filesystem so that the change can be effected in Unix. Apps and
15+unix whenever an app or vane changes the filesystem so that the change
16+can be effected in Unix. Apps and vanes may use `%clay` to write to the
15−vanes may use `%clay` to write to the filesystem, query it, and subscribe to
17+filesystem, query it, and subscribe to changes in it. Ford and gall use
16−changes in it. Ford and gall use `%clay` to serve up apps and web pages.
18+`%clay` to serve up apps and web pages.
17
19
18−`%clay` includes three components. First is the filesystem/version control
20+`%clay` includes three components. First is the filesystem/version
19−algorithms, which are mostly defined in `++ze` and `++zu` in zuse. Second is the
21+control algorithms, which are mostly defined in `++ze` and `++zu` in
20−write, query, and subscription logic. Finally, there is the logic for
22+zuse. Second is the write, query, and subscription logic. Finally, there
21−communicating requests to, and receiving requests from, foreign ships.
23+is the logic for communicating requests to, and receiving requests from,
24+foreign ships.
22
25
23<list></list>
26<list></list>
docs/arvo/internals/clay/architecture.md+349−295
@@ -1,176 +1,201 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3---
3---
4
4
5+
5# clay
6# clay
6
7
7## high-level
8## high-level
8
9
9−clay is the primary filesystem for the arvo operating system, which is the core
10+clay is the primary filesystem for the arvo operating system,
10−of an urbit. The architecture of clay is intrinsically connected with arvo, but
11+which is the core of an urbit. The architecture of clay is
12+intrinsically connected with arvo, but we assume no knowledge of
11−we assume no knowledge of either arvo or urbit. We will point out only those
13+either arvo or urbit. We will point out only those features of
12−features of arvo that are necessary for an understanding of clay, and we will do
14+arvo that are necessary for an understanding of clay, and we will
13−so only when they arise.
15+do so only when they arise.
14−
16+
15−The first relevant feature of arvo is that it is a deterministic system where
17+The first relevant feature of arvo is that it is a deterministic
16−input and output are defined as a series of events and effects. The state of
18+system where input and output are defined as a series of events
17−arvo is simply a function of its event log. None of the effects from an event
19+and effects. The state of arvo is simply a function of its event
20+log. None of the effects from an event are emitted until the
18−are emitted until the event is entered in the log and persisted, either to disk
21+event is entered in the log and persisted, either to disk or
19−or another trusted source of persistence, such as a Kafka cluster. Consequently,
22+another trusted source of persistence, such as a Kafka cluster.
20−arvo is a single-level store: everything in its state is persistent.
23+Consequently, arvo is a single-level store: everything in its
21−
24+state is persistent.
25+
22−In a more traditional OS, everything in RAM can be erased at any time by power
26+In a more traditional OS, everything in RAM can be erased at any
23−failure, and is always erased on reboot. Thus, a primary purpose of a filesystem
27+time by power failure, and is always erased on reboot. Thus, a
28+primary purpose of a filesystem is to ensure files persist across
24−is to ensure files persist across power failures and reboots. In arvo, both
29+power failures and reboots. In arvo, both power failures and
25−power failures and reboots are special cases of suspending computation, which is
30+reboots are special cases of suspending computation, which is
26−done safely since our event log is already persistent. Therefore, clay is not
31+done safely since our event log is already persistent. Therefore,
27−needed in arvo for persistence. Why, then, do we have a filesystem? There are
32+clay is not needed in arvo for persistence. Why, then, do we have a
28−two answers to this question.
33+filesystem? There are two answers to this question.
29−
34+
30−First, clay provides a filesystem tree, which is a convenient user interface for
35+First, clay provides a filesystem tree, which is a convenient
31−some applications. Unix has the useful concept of virtual filesystems, which are
36+user interface for some applications. Unix has the useful concept
37+of virtual filesystems, which are used for everything from direct
32−used for everything from direct access to devices, to random number generators,
38+access to devices, to random number generators, to the /proc
33−to the /proc tree. It is easy and intuitive to read from and write to a
39+tree. It is easy and intuitive to read from and write to a
34filesystem tree.
40filesystem tree.
35
41
36−Second, clay has a distributed revision control system baked into it.
42+Second, clay has a distributed revision control system baked into
37−Traditional filesystems are not revision controlled, so userspace software --
43+it. Traditional filesystems are not revision controlled, so
38−such as git -- is written on top of them to do so. clay natively provides the
44+userspace software -- such as git -- is written on top of them to
39−same functionality as modern DVCSes, and more.
45+do so. clay natively provides the same functionality as modern
46+DVCSes, and more.
40
47
41−clay has two other unique properties that we'll cover later on: it supports
48+clay has two other unique properties that we'll cover later on:
42−typed data and is referentially transparent.
49+it supports typed data and is referentially transparent.
43
50
44### Revision Control
51### Revision Control
45
52
46−Every urbit has one or more "desks", which are independently revision-controlled
53+Every urbit has one or more "desks", which are independently
54+revision-controlled branches. Each desk contains its own mark
47−branches. Each desk contains its own mark definitions, apps, doc, and so forth.
55+definitions, apps, doc, and so forth.
48−
56+
49−Traditionally, an urbit has at least a base and a home desk. The base desk has
57+Traditionally, an urbit has at least a base and a home desk. The
50−all the system software from the distribution. the home desk is a fork of base
58+base desk has all the system software from the distribution. the
51−with all the stuff specific to the user of the urbit.
59+home desk is a fork of base with all the stuff specific to the
52−
60+user of the urbit.
61+
53−A desk is a series of numbered commits, the most recent of which represents the
62+A desk is a series of numbered commits, the most recent of which
54−current state of the desk. A commit is composed of (1) an absolute time when it
63+represents the current state of the desk. A commit is composed of
64+(1) an absolute time when it was created, (2) a list of zero or
55−was created, (2) a list of zero or more parents, and (3) a map from paths to
65+more parents, and (3) a map from paths to data.
56−data.
66+
57−
58−Most commits have exactly one parent, but the initial commit on a desk may have
67+Most commits have exactly one parent, but the initial commit on a
59−zero parents, and merge commits have more than one parent.
68+desk may have zero parents, and merge commits have more than one
60−
69+parent.
70+
61−The non-meta data is stored in the map of paths to data. It's worth noting that
71+The non-meta data is stored in the map of paths to data. It's
62−no constraints are put on this map, so, for example, both /a/b and /a/b/c could
72+worth noting that no constraints are put on this map, so, for
73+example, both /a/b and /a/b/c could have data. This is impossible
63−have data. This is impossible in a traditional Unix filesystem since it means
74+in a traditional Unix filesystem since it means that /a/b is both
64−that /a/b is both a file and a directory. Conventionally, the final element in
75+a file and a directory. Conventionally, the final element in the
65−the path is its mark -- much like a filename extension in Unix. Thus,
76+path is its mark -- much like a filename extension in Unix. Thus,
66/doc/readme.md in Unix is stored as /doc/readme/md in urbit.
77/doc/readme.md in Unix is stored as /doc/readme/md in urbit.
67
78
68−The data is not stored directly in the map; rather, a hash of the data is
79+The data is not stored directly in the map; rather, a hash of the
69−stored, and we maintain a master blob store. Thus, if the same data is referred
80+data is stored, and we maintain a master blob store. Thus, if the
70−to in multiple commits (as, for example, when a file doesn't change between
81+same data is referred to in multiple commits (as, for example,
71−commits), only the hash is duplicated.
82+when a file doesn't change between commits), only the hash is
72−
83+duplicated.
84+
73−In the master blob store, we either store the data directly, or else we store a
85+In the master blob store, we either store the data directly, or
74−diff against another blob. The hash is dependent only on the data within and not
86+else we store a diff against another blob. The hash is dependent
87+only on the data within and not on whether or not it's stored
75−on whether or not it's stored directly, so we may on occasion rearrange the
88+directly, so we may on occasion rearrange the contents of the
76−contents of the blob store for performance reasons.
89+blob store for performance reasons.
77−
90+
78−Recall that a desk is a series of numbered commits. Not every commit in a desk
91+Recall that a desk is a series of numbered commits. Not every
79−must be numbered. For example, if the base desk has had 50 commits since home
92+commit in a desk must be numbered. For example, if the base desk
93+has had 50 commits since home was forked from it, then a merge
80−was forked from it, then a merge from base to home will only add a single
94+from base to home will only add a single revision number to home,
81−revision number to home, although the full commit history will be accessible by
95+although the full commit history will be accessible by traversing
82−traversing the parentage of the individual commits.
96+the parentage of the individual commits.
83−
97+
84−We do guarantee that the first commit is numbered 1, commits are numbered
98+We do guarantee that the first commit is numbered 1, commits are
85−consecutively after that (i.e. there are no "holes"), the topmost commit is
99+numbered consecutively after that (i.e. there are no "holes"),
86−always numbered, and every numbered commit is an ancestor of every later
100+the topmost commit is always numbered, and every numbered commit
87−numbered commit.
101+is an ancestor of every later numbered commit.
88−
102+
89−There are three ways to refer to particular commits in the revision history.
103+There are three ways to refer to particular commits in the
90−Firstly, one can use the revision number. Secondly, one can use any absolute
104+revision history. Firstly, one can use the revision number.
105+Secondly, one can use any absolute time between the one numbered
91−time between the one numbered commit and the next (inclusive of the first,
106+commit and the next (inclusive of the first, exclusive of the
92−exclusive of the second). Thirdly, every desk has a map of labels to revision
107+second). Thirdly, every desk has a map of labels to revision
93numbers. These labels may be used to refer to specific commits.
108numbers. These labels may be used to refer to specific commits.
94
109
95−Additionally, clay is a global filesystem, so data on other urbit is easily
110+Additionally, clay is a global filesystem, so data on other urbit
96−accessible the same way as data on our local urbit. In general, the path to a
111+is easily accessible the same way as data on our local urbit. In
112+general, the path to a particular revision of a desk is
97−particular revision of a desk is /~urbit-name/desk-name/revision. Thus, to get
113+/~urbit-name/desk-name/revision. Thus, to get /try/readme/md
98−/try/readme/md from revision 5 of the home desk on ~sampel-sipnym, we refer to
114+from revision 5 of the home desk on ~sampel-sipnym, we refer to
99−/~sampel-sipnym/home/5/try/readme/md. Clay's namespace is thus global and
115+/~sampel-sipnym/home/5/try/readme/md. Clay's namespace is thus
100−referentially transparent.
116+global and referentially transparent.
101
117
102XXX reactivity here?
118XXX reactivity here?
103
119
104### A Typed Filesystem
120### A Typed Filesystem
105
121
106−Since clay is a general filesystem for storing data of arbitrary types, in order
122+Since clay is a general filesystem for storing data of arbitrary
107−to revision control correctly it needs to be aware of types all the way through.
123+types, in order to revision control correctly it needs to be
124+aware of types all the way through. Traditional revision control
108−Traditional revision control does an excellent job of handling source code, so
125+does an excellent job of handling source code, so for source code
109−for source code we act very similar to traditional revision control. The
126+we act very similar to traditional revision control. The
110challenge is to handle other data similarly well.
127challenge is to handle other data similarly well.
111
128
112−For example, modern VCSs generally support "binary files", which are files for
129+For example, modern VCSs generally support "binary files", which
113−which the standard textual diffing, patching, and merging algorithms are not
130+are files for which the standard textual diffing, patching, and
131+merging algorithms are not helpful. A "diff" of two binary files
114−helpful. A "diff" of two binary files is just a pair of the files, "patching"
132+is just a pair of the files, "patching" this diff is just
115−this diff is just replacing the old file with the new one, and "merging"
133+replacing the old file with the new one, and "merging"
116−non-identical diffs is always a conflict, which can't even be helpfully
134+non-identical diffs is always a conflict, which can't even be
117−annotated. Without knowing anything about the structure of a blob of data, this
135+helpfully annotated. Without knowing anything about the structure
118−is the best we can do.
136+of a blob of data, this is the best we can do.
119−
137+
120−Often, though, "binary" files have some internal structure, and it is possible
138+Often, though, "binary" files have some internal structure, and
121−to create diff, patch, and merge algorithms that take advantage of this
139+it is possible to create diff, patch, and merge algorithms that
122−structure. An image may be the result of a base image with some set of
140+take advantage of this structure. An image may be the result of a
141+base image with some set of operations applied. With algorithms
123−operations applied. With algorithms aware of this set of operations, not only
142+aware of this set of operations, not only can revision control
124−can revision control software save space by not having to save every revision of
143+software save space by not having to save every revision of the
125−the image individually, these transformations can be made on parallel branches
144+image individually, these transformations can be made on parallel
126−and merged at will.
145+branches and merged at will.
127−
146+
128−Suppose Alice is tasked with touching up a picture, improving the color balance,
147+Suppose Alice is tasked with touching up a picture, improving the
129−adjusting the contrast, and so forth, while Bob has the job of cropping the
148+color balance, adjusting the contrast, and so forth, while Bob
149+has the job of cropping the picture to fit where it's needed and
130−picture to fit where it's needed and adding textual overlay. Without type-aware
150+adding textual overlay. Without type-aware revision control,
131−revision control, these changes must be made serially, requiring Alice and Bob
151+these changes must be made serially, requiring Alice and Bob to
132−to explicitly coordinate their efforts. With type-aware revision control, these
152+explicitly coordinate their efforts. With type-aware revision
133−operations may be performed in parallel, and then the two changesets can be
153+control, these operations may be performed in parallel, and then
134−merged programmatically.
154+the two changesets can be merged programmatically.
135−
155+
136−Of course, even some kinds of text files may be better served by diff, patch,
156+Of course, even some kinds of text files may be better served by
137−and merge algorithms aware of the structure of the files. Consider a file
157+diff, patch, and merge algorithms aware of the structure of the
138−containing a pretty-printed JSON object. Small changes in the JSON object may
158+files. Consider a file containing a pretty-printed JSON object.
159+Small changes in the JSON object may result in rather significant
139−result in rather significant changes in how the object is pretty-printed (for
160+changes in how the object is pretty-printed (for example, by
140−example, by addding an indentation level, splitting a single line into multiple
161+addding an indentation level, splitting a single line into
141−lines).
162+multiple lines).
142−
163+
143−A text file wrapped at 80 columns also reacts suboptimally with unadorned
164+A text file wrapped at 80 columns also reacts suboptimally with
144−Hunt-McIlroy diffs. A single word inserted in a paragraph may push the final
165+unadorned Hunt-McIlroy diffs. A single word inserted in a
166+paragraph may push the final word or two of the line onto the
145−word or two of the line onto the next line, and the entire rest of the paragraph
167+next line, and the entire rest of the paragraph may be flagged as
146−may be flagged as a change. Two diffs consisting of a single added word to
168+a change. Two diffs consisting of a single added word to
147−different sentences may be flagged as a conflict. In general, prose should be
169+different sentences may be flagged as a conflict. In general,
148−diffed by sentence, not by line.
170+prose should be diffed by sentence, not by line.
149−
171+
150−As far as the author is aware, clay is the first generalized, type-aware
172+As far as the author is aware, clay is the first generalized,
151−revision control system. We'll go into the workings of this system in some
173+type-aware revision control system. We'll go into the workings
152−detail.
174+of this system in some detail.
153
175
154### Marks
176### Marks
155
177
156−Central to a typed filesystem is the idea of types. In clay, we call these
178+Central to a typed filesystem is the idea of types. In clay, we
157−"marks". A mark is a file that defines a type, conversion routines to and from
179+call these "marks". A mark is a file that defines a type,
180+conversion routines to and from the mark, and diff, patch, and
158−the mark, and diff, patch, and merge routines.
181+merge routines.
159
182
160−For example, a `%txt` mark may be a list of lines of text, and it may include
183+For example, a `%txt` mark may be a list of lines of text, and it
161−conversions to `%mime` to allow it to be serialized and sent to a browswer or to
184+may include conversions to `%mime` to allow it to be serialized
185+and sent to a browswer or to the unix filesystem. It will also
162−the unix filesystem. It will also include Hunt-McIlroy diff, patch, and merge
186+include Hunt-McIlroy diff, patch, and merge algorithms.
163−algorithms.
164
187
165−A `%json` mark would be defined as a json object in the code, and it would have
188+A `%json` mark would be defined as a json object in the code, and
166−a parser to convert from `%txt` and a printer to convert back to `%txt`. The
189+it would have a parser to convert from `%txt` and a printer to
190+convert back to `%txt`. The diff, patch, and merge algorithms are
167−diff, patch, and merge algorithms are fairly straightforward for json, though
191+fairly straightforward for json, though they're very different
168−they're very different from the text ones.
192+from the text ones.
169
193
170−More formally, a mark is a core with three arms, `++grab`, `++grow`, and
194+More formally, a mark is a core with three arms, `++grab`,
171−`++grad`. In `++grab` is a series of functions to convert from other marks to
195+`++grow`, and `++grad`. In `++grab` is a series of functions to
196+convert from other marks to the given mark. In `++grow` is a
172−the given mark. In `++grow` is a series of functions to convert from the given
197+series of functions to convert from the given mark to other
173−mark to other marks. In `++grad` is `++diff`, `++pact`, `++join`, and `++mash`.
198+marks. In `++grad` is `++diff`, `++pact`, `++join`, and `++mash`.
174
199
175The types are as follows, in an informal pseudocode:
200The types are as follows, in an informal pseudocode:
176
201
@@ -188,179 +213,208 @@ The types are as follows, in an informal pseudocode:
188 ++ join: (<diff-type>, <diff-type>) -> <diff-type> or NULL
213 ++ join: (<diff-type>, <diff-type>) -> <diff-type> or NULL
189 ++ mash: (<diff-type>, <diff-type>) -> <diff-type>
214 ++ mash: (<diff-type>, <diff-type>) -> <diff-type>
190
215
191−These types are basically what you would expect. Not every mark has each of
216+These types are basically what you would expect. Not every mark
192−these functions defined -- all of them are optional in the general case.
217+has each of these functions defined -- all of them are optional
218+in the general case.
193
219
194−In general, for a particular mark, the `++grab` and `++grow` entries (if they
220+In general, for a particular mark, the `++grab` and `++grow` entries
195−exist) should be inverses of each other.
221+(if they exist) should be inverses of each other.
196
222
197−In `++grad`, `++diff` takes two instances of a mark and produces a diff of them.
223+In `++grad`, `++diff` takes two instances of a mark and produces
198−`++pact` takes an instance of a mark and patches it with the given diff.
224+a diff of them. `++pact` takes an instance of a mark and patches
225+it with the given diff. `++join` takes two diffs and attempts to
199−`++join` takes two diffs and attempts to merge them into a single diff. If there
226+merge them into a single diff. If there are conflicts, it
200−are conflicts, it produces null. `++mash` takes two diffs and forces a merge,
227+produces null. `++mash` takes two diffs and forces a merge,
201annotating any conflicts.
228annotating any conflicts.
202
229
203−In general, if `++diff` called with A and B produces diff D, then `++pact`
230+In general, if `++diff` called with A and B produces diff D, then
204−called with A and D should produce B. Also, if `++join` of two diffs does not
231+`++pact` called with A and D should produce B. Also, if `++join`
232+of two diffs does not produce null, then `++mash` of the same
205−produce null, then `++mash` of the same diffs should produce the same result.
233+diffs should produce the same result.
206
234
207−Alternately, instead of `++diff`, `++pact`, `++join`, and `++mash`, a mark can
235+Alternately, instead of `++diff`, `++pact`, `++join`, and
208−provide the same functionality by defining `++sted` to be the name of another
236+`++mash`, a mark can provide the same functionality by defining
237+`++sted` to be the name of another mark to which we wish to
209−mark to which we wish to delegate the revision control responsibilities. Then,
238+delegate the revision control responsibilities. Then, before
210−before running any of those functions, clay will convert to the other mark, and
239+running any of those functions, clay will convert to the other
211−convert back afterward. For example, the `%hoon` mark is revision-controlled in
240+mark, and convert back afterward. For example, the `%hoon` mark
241+is revision-controlled in the same way as `%txt`, so its `++grad`
212−the same way as `%txt`, so its `++grad` is simply `++sted %txt`. Of course,
242+is simply `++sted %txt`. Of course, `++txt` must be defined in
213−`++txt` must be defined in `++grow` and `++grab` as well.
243+`++grow` and `++grab` as well.
214
244
215−Every file in clay has a mark, and that mark must have a fully-functioning
245+Every file in clay has a mark, and that mark must have a
216−`++grad`. Marks are used for more than just clay, and other marks don't need a
246+fully-functioning `++grad`. Marks are used for more than just
247+clay, and other marks don't need a `++grad`, but if a piece of
217−`++grad`, but if a piece of data is to be saved to clay, we must know how to
248+data is to be saved to clay, we must know how to revision-control
218−revision-control it.
249+it.
219
250
220−Additionally, if a file is to be synced out to unix, then it must have
251+Additionally, if a file is to be synced out to unix, then it must
221−conversion routines to and from the `%mime` mark.
252+have conversion routines to and from the `%mime` mark.
222
253
223−## Using clay
254+##Using clay
224
255
225### Reading and Subscribing
256### Reading and Subscribing
226
257
227−When reading from Clay, there are three types of requests. A `%sing` request
258+When reading from Clay, there are three types of requests. A
228−asks for data at single revsion. A `%next` request asks to be notified the next
259+`%sing` request asks for data at single revsion. A `%next`
260+request asks to be notified the next time there's a change to
229−time there's a change to given file. A `%many` request asks to be notified on
261+given file. A `%many` request asks to be notified on every
230−every change in a desk for a range of changes.
262+change in a desk for a range of changes.
231−
263+
232−For `%sing` and `%next`, there are generally three things to be queried. A `%u`
264+For `%sing` and `%next`, there are generally three things to be
233−request simply checks for the existence of a file at a path. A `%x` request gets
265+queried. A `%u` request simply checks for the existence of a
266+file at a path. A `%x` request gets the data in the file at a
234−the data in the file at a path. A `%y` request gets a hash of the data in the
267+path. A `%y` request gets a hash of the data in the file at the
235−file at the path combined with all its children and their data. Thus, `%y` of a
268+path combined with all its children and their data. Thus, `%y`
236−node changes if it or any of its children change.
269+of a node changes if it or any of its children change.
237−
270+
238−A `%sing` request is fulfilled immediately if possible. If the requested
271+A `%sing` request is fulfilled immediately if possible. If the
239−revision is in the future, or is on another ship for which we don't have the
272+requested revision is in the future, or is on another ship for
273+which we don't have the result cached, we don't respond
240−result cached, we don't respond immediately. If the requested revision is in the
274+immediately. If the requested revision is in the future, we wait
241−future, we wait until the revision happens before we respond to the request. If
275+until the revision happens before we respond to the request. If
242−the request is for data on another ship, we pass on the request to the other
276+the request is for data on another ship, we pass on the request
243−ship. In general, Clay subscriptions, like most things in Urbit, aren't
277+to the other ship. In general, Clay subscriptions, like most
278+things in Urbit, aren't guaranteed to return immediately.
244−guaranteed to return immediately. They'll return when they can, and they'll do
279+They'll return when they can, and they'll do so in a
245−so in a referentially transparent manner.
280+referentially transparent manner.
246−
281+
247−A `%next` request checks query at the given revision, and it produces the result
282+A `%next` request checks query at the given revision, and it
248−of the query the next time it changes, along with the revsion number when it
283+produces the result of the query the next time it changes, along
284+with the revsion number when it changes. Thus, a `%next` of a
249−changes. Thus, a `%next` of a `%u` is triggered when a file is added or deleted,
285+`%u` is triggered when a file is added or deleted, a `%next` of a
250−a `%next` of a `%x` is triggered when a file is added, deleted, or changed, and
286+`%x` is triggered when a file is added, deleted, or changed, and
251−a `%next` of a `%y` is triggered when a file or any of its children is added,
287+a `%next` of a `%y` is triggered when a file or any of its
252−deleted, or changed.
288+children is added, deleted, or changed.
253−
289+
254−A `%many` request is triggered every time the given desk has a new revision.
290+A `%many` request is triggered every time the given desk has a
255−Unlike a `%next`, a `%many` has both a start and an end revsion, after which it
291+new revision. Unlike a `%next`, a `%many` has both a start and
292+an end revsion, after which it stops returning. For `%next`, a
256−stops returning. For `%next`, a single change is reported, and if the caller
293+single change is reported, and if the caller wishes to hear of
257−wishes to hear of the next change, it must resubscribe. For `%many`, every
294+the next change, it must resubscribe. For `%many`, every revsion
258−revsion from the start to the end triggers a response. Since a `%many` request
295+from the start to the end triggers a response. Since a `%many`
259−doesn't ask for any particular data, there aren't `%u`, `%x`, and `%y` versions
296+request doesn't ask for any particular data, there aren't `%u`,
260−for it.
297+`%x`, and `%y` versions for it.
261
298
262### Unix sync
299### Unix sync
263
300
264−One of the primary functions of clay is as a convenient user interface. While
301+One of the primary functions of clay is as a convenient user
265−tools exist to use clay from within urbit, it's often useful to be able to treat
302+interface. While tools exist to use clay from within urbit, it's
303+often useful to be able to treat clay like any other filesystem
266−clay like any other filesystem from the Unix perspective -- to "mount" it, as it
304+from the Unix perspective -- to "mount" it, as it were.
267−were.
268
305
269−From urbit, you can run `|mount /path/to/directory %mount-point`, and this will
306+From urbit, you can run `|mount /path/to/directory %mount-point`,
270−mount the given clay directory to the mount-point directory in Unix. Every file
307+and this will mount the given clay directory to the mount-point
308+directory in Unix. Every file is converted to `%mime` before it's
271−is converted to `%mime` before it's written to Unix, and converted back when
309+written to Unix, and converted back when read from Unix. The
272−read from Unix. The entire directory is watched (a la Dropbox), and every change
310+entire directory is watched (a la Dropbox), and every change is
273−is auto-committed to clay.
311+auto-committed to clay.
274
312
275### Merging
313### Merging
276
314
277−Merging is a fundamental operation for a distributed revision control system. At
315+Merging is a fundamental operation for a distributed revision
278−their root, clay's merges are similar to git's, but with some additions to
316+control system. At their root, clay's merges are similar to
317+git's, but with some additions to accomodate typed data. There
279−accomodate typed data. There are seven different merge strategies.
318+are seven different merge strategies.
280−
319+
281−Throughout our discussion, we'll say that the merge is from Alice's desk to
320+Throughout our discussion, we'll say that the merge is from
282−Bob's. Recall that a commit is a date (for all new commits this will be the
321+Alice's desk to Bob's. Recall that a commit is a date (for all
283−current date), a list of parents, and the data itself.
322+new commits this will be the current date), a list of parents,
284−
323+and the data itself.
324+
285−A `%init` merge should be used iff it's the first commit to a desk. The head of
325+A `%init` merge should be used iff it's the first commit to a
286−Alice's desk is used as the number 1 commit to Bob's desk. Obviously, the
326+desk. The head of Alice's desk is used as the number 1 commit to
327+Bob's desk. Obviously, the ancestry remains intact through
287−ancestry remains intact through traversing the parentage of the commit even
328+traversing the parentage of the commit even though previous
288−though previous commits are not numbered for Bob's desk.
329+commits are not numbered for Bob's desk.
289−
330+
290−A `%this` merge means to keep what's in Bob's desk, but join the ancestry. Thus,
331+A `%this` merge means to keep what's in Bob's desk, but join the
291−the new commit has the head of each desk as parents, but the data is exactly
332+ancestry. Thus, the new commit has the head of each desk as
333+parents, but the data is exactly what's in Bob's desk. For those
292−what's in Bob's desk. For those following along in git, this is the 'ours' merge
334+following along in git, this is the 'ours' merge strategy, not
293−strategy, not the '--ours' option to the 'recursive' merge strategy. In other
335+the '--ours' option to the 'recursive' merge strategy. In other
294−words, even if Alice makes a change that does not conflict with Bob, we throw it
336+words, even if Alice makes a change that does not conflict with
295−away. It's Bob's way or the highway.
337+Bob, we throw it away. It's Bob's way or the highway.
296−
338+
297−A `%that` merge means to take what's in Alice's desk, but join the ancestry.
339+A `%that` merge means to take what's in Alice's desk, but join
298−This is the reverse of `%this`.
340+the ancestry. This is the reverse of `%this`.
299−
341+
300−A `%fine` merge is a "fast-forward" merge. This succeeds iff one head is in the
342+A `%fine` merge is a "fast-forward" merge. This succeeds iff one
301−ancestry of the other. In this case, we use the descendant as our new head.
343+head is in the ancestry of the other. In this case, we use the
302−
344+descendant as our new head.
345+
303−For `%meet`, `%mate`, and `%meld` merges, we first find the most recent common
346+For `%meet`, `%mate`, and `%meld` merges, we first find the most
304−ancestor to use as our merge base. If we have no common ancestors, then we fail.
347+recent common ancestor to use as our merge base. If we have no
348+common ancestors, then we fail. If we have more than one most
305−If we have more than one most recent common ancestor, then we have a criss-cross
349+recent common ancestor, then we have a criss-cross situation,
306−situation, which should be handled delicately. At present, we delicately throw
350+which should be handled delicately. At present, we delicately
307−up our hands and give up, but something akin to git's 'recursive' strategy
351+throw up our hands and give up, but something akin to git's
308−should be implemented in the future.
352+'recursive' strategy should be implemented in the future.
309−
353+
310−There's a functional inclusion ordering on `%fine`, `%meet`, `%mate`, and
354+There's a functional inclusion ordering on `%fine`, `%meet`,
311−`%meld` such that if an earlier strategy would have succeeded, then every later
355+`%mate`, and `%meld` such that if an earlier strategy would have
356+succeeded, then every later strategy will produce the same
312−strategy will produce the same result. Put another way, every earlier strategy
357+result. Put another way, every earlier strategy is the same as
313−is the same as every later strategy except with a restricted domain.
358+every later strategy except with a restricted domain.
314−
359+
315−A `%meet` merge only succeeds if the changes from the merge base to Alice's head
360+A `%meet` merge only succeeds if the changes from the merge base
316−(hereafter, "Alice's changes") are in different files than Bob's changes. In
361+to Alice's head (hereafter, "Alice's changes") are in different
362+files than Bob's changes. In this case, the parents are both
317−this case, the parents are both Alice's and Bob's heads, and the data is the
363+Alice's and Bob's heads, and the data is the merge base plus
318−merge base plus Alice's changed files plus Bob's changed files.
364+Alice's changed files plus Bob's changed files.
319−
365+
320−A `%mate` merge attempts to merge changes to the same file when both Alice and
366+A `%mate` merge attempts to merge changes to the same file when
321−bob change it. If the merge is clean, we use it; otherwise, we fail. A merge
367+both Alice and bob change it. If the merge is clean, we use it;
368+otherwise, we fail. A merge between different types of changes --
322−between different types of changes -- for example, deleting a file vs changing
369+for example, deleting a file vs changing it -- is always a
323−it -- is always a conflict. If we succeed, the parents are both Alice's and
370+conflict. If we succeed, the parents are both Alice's and Bob's
324−Bob's heads, and the data is the merge base plus Alice's changed files plus
371+heads, and the data is the merge base plus Alice's changed files
325−Bob's changed files plus the merged files.
372+plus Bob's changed files plus the merged files.
326−
373+
327−A `%meld` merge will succeed even if there are conflicts. If there are conflicts
374+A `%meld` merge will succeed even if there are conflicts. If
375+there are conflicts in a file, then we use the merge base's
328−in a file, then we use the merge base's version of that file, and we produce a
376+version of that file, and we produce a set of files with
329−set of files with conflicts. The parents are both Alice's and Bob's heads, and
377+conflicts. The parents are both Alice's and Bob's heads, and the
330−the data is the merge base plus Alice's changed files plus Bob's changed files
378+data is the merge base plus Alice's changed files plus Bob's
331−plus the successfully merged files plus the merge base's version of the
379+changed files plus the successfully merged files plus the merge
332−conflicting files.
380+base's version of the conflicting files.
333−
381+
334−That's the extent of the merge options in clay proper. In userspace there's a
382+That's the extent of the merge options in clay proper. In
335−final option `%auto`, which is the most common. `%auto` checks to see if Bob's
383+userspace there's a final option `%auto`, which is the most
384+common. `%auto` checks to see if Bob's desk exists, and if it
336−desk exists, and if it doesn't we use a `%init` merge. Otherwise, we
385+doesn't we use a `%init` merge. Otherwise, we progressively try
337−progressively try `%fine`, `%meet`, and `%mate` until one succeeds.
386+`%fine`, `%meet`, and `%mate` until one succeeds.
338−
387+
339−If none succeed, we merge Bob's desk into a scratch desk. Then, we merge Alice's
388+If none succeed, we merge Bob's desk into a scratch desk. Then,
340−desk into the scratch desk with the `%meld` option to force the merge. For each
389+we merge Alice's desk into the scratch desk with the `%meld`
390+option to force the merge. For each file in the produced set of
341−file in the produced set of conflicting files, we call the `++mash` function for
391+conflicting files, we call the `++mash` function for the
342−the appropriate mark, which annotates the conflicts if we know how.
392+appropriate mark, which annotates the conflicts if we know how.
343−
393+
344−Finally, we display a message to the user informing them of the scratch desk's
394+Finally, we display a message to the user informing them of the
345−existence, which files have annotated conflicts, and which files have
395+scratch desk's existence, which files have annotated conflicts,
346−unannotated conflicts. When the user has resolved the conflicts, they can merge
396+and which files have unannotated conflicts. When the user has
397+resolved the conflicts, they can merge the scratch desk back into
347−the scratch desk back into Bob's desk. This will be a `%fine` merge since Bob's
398+Bob's desk. This will be a `%fine` merge since Bob's head is in
348−head is in the ancestry of the scratch desk.
399+the ancestry of the scratch desk.
349
400
350### Autosync
401### Autosync
351
402
352−Tracking and staying in sync with another desk is another fundamental operation.
403+Tracking and staying in sync with another desk is another
353−We call this "autosync". This doesn't mean simply mirroring a desk, since that
404+fundamental operation. We call this "autosync". This doesn't mean
405+simply mirroring a desk, since that wouldn't allow local changes.
354−wouldn't allow local changes. We simply want to apply changes as they are made
406+We simply want to apply changes as they are made upstream, as
355−upstream, as long as there are no conflicts with local changes.
407+long as there are no conflicts with local changes.
356
408
357−This is implemented by watching the other desk, and, when it has changes,
409+This is implemented by watching the other desk, and, when it has
358−merging these changes into our desk with the usual merge strategies.
410+changes, merging these changes into our desk with the usual merge
411+strategies.
359
412
360−Note that it's quite reasonable for two desks to be autosynced to each other.
413+Note that it's quite reasonable for two desks to be autosynced to
361−This results in any change on one desk being mirrored to the other and vice
414+each other. This results in any change on one desk being mirrored
362−versa.
415+to the other and vice versa.
363
416
364−Additionally, it's fine to set up an autosync even if one desk, the other desk,
417+Additionally, it's fine to set up an autosync even if one desk,
365−or both desks do not exist. The sync will be activated when the upstream desk
418+the other desk, or both desks do not exist. The sync will be
419+activated when the upstream desk comes into existence and will
366−comes into existence and will create the downstream desk if needed.
420+create the downstream desk if needed.
docs/arvo/internals/clay/commentary.md+811−724
@@ -1,39 +1,46 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3---
3---
4
4
5−# `%clay` commentary
6
5
7−`%clay` is our filesystem.
6+`%clay` commentary
8−
7+==================
9−The first part of this will be reference documentation for the data types used
10−by our filesystem. In fact, as a general guide, we recommend reading and
11−attempting to understand the data structures used in any Hoon code before you
12−try to read the code itself. Although complete understanding of the data
13−structures is impossible without seeing them used in the code, an 80%
14−understanding greatly clarifies the code. As another general guide, when reading
15−Hoon, it rarely pays off to understand every line of code when it appears. Try
16−to get the gist of it, and then move on. The next time you come back to it,
17−it'll likely make a lot more sense.
18−
19−After a description of the data models, we'll give an overview of the interface
20−that vanes and applications can use to interact with the filesystem.
21
8
22−Finally, we'll dive into the code and the algorithms themselves. You know, the
9+`%clay` is our filesystem.
23−fun part.
24−
25−## Data Models
26−
27−As you're reading through this section, remember you can always come back to
28−this when you run into these types later on. You're not going to remember
29−everything the first time through, but it is worth reading, or at least
30−skimming, this so that you get a rough idea of how our state is organized.
31
10
11+The first part of this will be reference documentation for the data
12+types used by our filesystem. In fact, as a general guide, we recommend
13+reading and attempting to understand the data structures used in any
14+Hoon code before you try to read the code itself. Although complete
15+understanding of the data structures is impossible without seeing them
16+used in the code, an 80% understanding greatly clarifies the code. As
17+another general guide, when reading Hoon, it rarely pays off to
18+understand every line of code when it appears. Try to get the gist of
19+it, and then move on. The next time you come back to it, it'll likely
20+make a lot more sense.
21+
22+After a description of the data models, we'll give an overview of the
23+interface that vanes and applications can use to interact with the
24+filesystem.
25+
26+Finally, we'll dive into the code and the algorithms themselves. You
27+know, the fun part.
28+
29+Data Models
30+-----------
31+
32+As you're reading through this section, remember you can always come
33+back to this when you run into these types later on. You're not going to
34+remember everything the first time through, but it is worth reading, or
35+at least skimming, this so that you get a rough idea of how our state is
36+organized.
37+
32−The types that are certainly worth reading are `++raft`, `++room`, `++dome`,
38+The types that are certainly worth reading are `++raft`, `++room`,
33−`++ankh`, `++rung`, `++rang`, `++blob`, `++yaki`, and `++nori` (possibly in that
39+`++dome`, `++ankh`, `++rung`, `++rang`, `++blob`, `++yaki`, and `++nori`
34−order). All in all, though, this section isn't too long, so many readers may
40+(possibly in that order). All in all, though, this section isn't too
35−wish to quickly read through all of it. If you get bored, though, just skip to
41+long, so many readers may wish to quickly read through all of it. If you
36−the next section. You can always come back when you need to.
42+get bored, though, just skip to the next section. You can always come
43+back when you need to.
37
44
38### `++raft`, formal state
45### `++raft`, formal state
39
46
@@ -43,17 +50,18 @@ the next section. You can always come back when you need to.
43 ran=rang :: hashes
50 ran=rang :: hashes
44 == ::
51 == ::
45
52
46−This is the state of our vane. Anything that must be remembered between calls to
53+This is the state of our vane. Anything that must be remembered between
47−clay is stored in this state.
54+calls to clay is stored in this state.
48
55
49−`fat` is the set of domestic servers. This stores all the information that is
56+`fat` is the set of domestic servers. This stores all the information
50−specfic to a particular ship on this pier. The keys to this map are the ships on
57+that is specfic to a particular ship on this pier. The keys to this map
51−the current pier. all the information that is specific to a particular foreign
58+are the ships on the current pier. all the information that is specific
52−ship. The keys to this map are all the ships whose filesystems we have attempted
59+to a particular foreign ship. The keys to this map are all the ships
53−to access through clay.
60+whose filesystems we have attempted to access through clay.
54
61
55−`ran` is the store of all commits and deltas, keyed by hash. The is where all
62+`ran` is the store of all commits and deltas, keyed by hash. The is
56−the "real" data we know is stored; the rest is "just bookkeeping".
63+where all the "real" data we know is stored; the rest is "just
64+bookkeeping".
57
65
58### `++room`, filesystem per domestic ship
66### `++room`, filesystem per domestic ship
59
67
@@ -65,40 +73,42 @@ the "real" data we know is stored; the rest is "just bookkeeping".
65
73
66This is the representation of the filesystem of a ship on our pier.
74This is the representation of the filesystem of a ship on our pier.
67
75
68−`hun` is the duct we use to send messages to dill to display notifications of
76+`hun` is the duct we use to send messages to dill to display
69−filesystem changes. Only `%note` gifts should be produced along this duct. This
77+notifications of filesystem changes. Only `%note` gifts should be
70−is set by the `%init` kiss.
78+produced along this duct. This is set by the `%init` kiss.
71
79
72−`hez`, if present, is the duct we use to send sync messages to unix so that they
80+`hez`, if present, is the duct we use to send sync messages to unix so
73−end up in the pier unix directory. Only `%ergo` gifts should be producd along
81+that they end up in the pier unix directory. Only `%ergo` gifts should
74−this duct. This is set by `%into` and `%invo` kisses.
82+be producd along this duct. This is set by `%into` and `%invo` kisses.
75
83
76−`dos` is a well-known operating system released in 1981. It is also the set of
84+`dos` is a well-known operating system released in 1981. It is also the
77−desks on this ship, mapped to their data.
85+set of desks on this ship, mapped to their data.
78
86
79### `++desk`, filesystem branch
87### `++desk`, filesystem branch
80
88
81 ++ desk ,@tas :: ship desk case spur
89 ++ desk ,@tas :: ship desk case spur
82
90
83−This is the name of a branch of the filesystem. The default desks are "arvo",
91+This is the name of a branch of the filesystem. The default desks are
84−"main", and "try". More may be created by simply referencing them. Desks have
92+"arvo", "main", and "try". More may be created by simply referencing
85−independent histories and states, and they may be merged into each other.
93+them. Desks have independent histories and states, and they may be
94+merged into each other.
86
95
87### `++dojo`, domestic desk state
96### `++dojo`, domestic desk state
88
97
89 ++ dojo ,[p=cult q=dome] :: domestic desk state
98 ++ dojo ,[p=cult q=dome] :: domestic desk state
90
99
91−This is the all the data that is specific to a particular desk on a domestic
100+This is the all the data that is specific to a particular desk on a
92−ship. `p` is the set of subscribers to this desk and `q` is the data in the
101+domestic ship. `p` is the set of subscribers to this desk and `q` is the
93−desk.
102+data in the desk.
94
103
95### `++cult`, subscriptions
104### `++cult`, subscriptions
96
105
97 ++ cult (map duct rave) :: subscriptions
106 ++ cult (map duct rave) :: subscriptions
98
107
99−This is the set of subscriptions to a particular desk. The keys are the ducts
108+This is the set of subscriptions to a particular desk. The keys are the
100−from where the subscriptions requests came. The results will be produced along
109+ducts from where the subscriptions requests came. The results will be
101−these ducts. The values are a description of the requested information.
110+produced along these ducts. The values are a description of the
111+requested information.
102
112
103### `++rave`, general subscription request
113### `++rave`, general subscription request
104
114
@@ -107,60 +117,61 @@ these ducts. The values are a description of the requested information.
107 [| p=moat] :: change range
117 [| p=moat] :: change range
108 == ::
118 == ::
109
119
110−This represents a subscription request for a desk. The request can be for either
120+This represents a subscription request for a desk. The request can be
111−a single item in the desk or else for a range of changes on the desk.
121+for either a single item in the desk or else for a range of changes on
122+the desk.
112
123
113### `++rove`, stored general subscription request
124### `++rove`, stored general subscription request
114
125
115 ++ rove (each mood moot) :: stored request
126 ++ rove (each mood moot) :: stored request
116
127
117−When we store a request, we store subscriptions with a little extra information
128+When we store a request, we store subscriptions with a little extra
118−so that we can determine whether new versions actually affect the path we're
129+information so that we can determine whether new versions actually
119−subscribed to.
130+affect the path we're subscribed to.
120
131
121### `++mood`, single subscription request
132### `++mood`, single subscription request
122
133
123 ++ mood ,[p=care q=case r=path] :: request in desk
134 ++ mood ,[p=care q=case r=path] :: request in desk
124
135
125−This represents a request for the state of the desk at a particular commit,
136+This represents a request for the state of the desk at a particular
126−specfied by `q`. `p` specifies what kind of information is desired, and `r`
137+commit, specfied by `q`. `p` specifies what kind of information is
127−specifies the path we are requesting.
138+desired, and `r` specifies the path we are requesting.
128
139
129### `++moat`, range subscription request
140### `++moat`, range subscription request
130
141
131 ++ moat ,[p=case q=case r=path] :: change range
142 ++ moat ,[p=case q=case r=path] :: change range
132
143
133−This represents a request for all changes between `p` and `q` on path `r`. You
144+This represents a request for all changes between `p` and `q` on path
134−will be notified when a change is made to the node referenced by the path or to
145+`r`. You will be notified when a change is made to the node referenced
135−any of its children.
146+by the path or to any of its children.
136
147
137### `++moot`, stored range subscription request
148### `++moot`, stored range subscription request
138
149
139 ++ moot ,[p=case q=case r=path s=(map path lobe)] ::
150 ++ moot ,[p=case q=case r=path s=(map path lobe)] ::
140
151
141−This is just a `++moat` plus a map of paths to lobes. This map represents the
152+This is just a `++moat` plus a map of paths to lobes. This map
142−data at the node referenced by the path at case `p`, if we've gotten to that
153+represents the data at the node referenced by the path at case `p`, if
143−case (else null). We only send a notification along the subscription if the data
154+we've gotten to that case (else null). We only send a notification along
144−at a new revision is different than it was.
155+the subscription if the data at a new revision is different than it was.
145
156
146### `++care`, clay submode
157### `++care`, clay submode
147
158
148 ++ care ?(%u %v %w %x %y %z) :: clay submode
159 ++ care ?(%u %v %w %x %y %z) :: clay submode
149
160
150−This specifies what type of information is requested in a subscription or a
161+This specifies what type of information is requested in a subscription
151−scry.
162+or a scry.
152
163
153−`%u` requests the `++rang` at the current moment. Because this information is
164+`%u` requests the `++rang` at the current moment. Because this
154−not stored for any moment other than the present, we crash if the `++case` is
165+information is not stored for any moment other than the present, we
155−not a `%da` for now.
166+crash if the `++case` is not a `%da` for now.
156
167
157`%v` requests the `++dome` at the specified commit.
168`%v` requests the `++dome` at the specified commit.
158
169
159`%w` requests the revsion number of the desk.
170`%w` requests the revsion number of the desk.
160
171
161−`%x` requests the file at a specified path at the specified commit. If there is
172+`%x` requests the file at a specified path at the specified commit. If
162−no node at that path or if the node has no contents (that is, if `q:ankh` is
173+there is no node at that path or if the node has no contents (that is,
163−null), then this produces null.
174+if `q:ankh` is null), then this produces null.
164
175
165`%y` requests a `++arch` of the specfied commit at the specified path.
176`%y` requests a `++arch` of the specfied commit at the specified path.
166
177
@@ -170,12 +181,13 @@ null), then this produces null.
170
181
171 ++ arch ,[p=@uvI q=(unit ,@uvI) r=(map ,@ta ,~)] :: fundamental node
182 ++ arch ,[p=@uvI q=(unit ,@uvI) r=(map ,@ta ,~)] :: fundamental node
172
183
173−This is analogous to `++ankh` except that the we have neither our contents nor
184+This is analogous to `++ankh` except that the we have neither our
174−the ankhs of our children. The other fields are exactly the same, so `p` is a
185+contents nor the ankhs of our children. The other fields are exactly the
175−hash of the associated ankh, `u.q`, if it exists, is a hash of the contents of
186+same, so `p` is a hash of the associated ankh, `u.q`, if it exists, is a
176−this node, and the keys of `r` are the names of our children. `r` is a map to
187+hash of the contents of this node, and the keys of `r` are the names of
188+our children. `r` is a map to null rather than a set so that the
177−null rather than a set so that the ordering of the map will be equivalent to
189+ordering of the map will be equivalent to that of `r:ankh`, allowing
178−that of `r:ankh`, allowing efficient conversion.
190+efficient conversion.
179
191
180### `++case`, specifying a commit
192### `++case`, specifying a commit
181
193
@@ -185,10 +197,11 @@ that of `r:ankh`, allowing efficient conversion.
185 [%ud p=@ud] :: number
197 [%ud p=@ud] :: number
186 == ::
198 == ::
187
199
188−A commit can be referred to in three ways: `%da` refers to the commit that was
200+A commit can be referred to in three ways: `%da` refers to the commit
189−at the head on date `p`, `%tas` refers to the commit labeled `p`, and `%ud`
201+that was at the head on date `p`, `%tas` refers to the commit labeled
190−refers to the commit numbered `p`. Note that since these all can be reduced down
202+`p`, and `%ud` refers to the commit numbered `p`. Note that since these
191−to a `%ud`, only numbered commits may be referenced with a `++case`.
203+all can be reduced down to a `%ud`, only numbered commits may be
204+referenced with a `++case`.
192
205
193### `++dome`, desk data
206### `++dome`, desk data
194
207
@@ -204,26 +217,29 @@ This is the data that is actually stored in a desk.
204
217
205`ang` is unused and should be removed.
218`ang` is unused and should be removed.
206
219
207−`ank` is the current state of the desk. Thus, it is the state of the filesystem
220+`ank` is the current state of the desk. Thus, it is the state of the
208−at revison `let`. The head of a desk is always a numbered commit.
221+filesystem at revison `let`. The head of a desk is always a numbered
209−
222+commit.
223+
210−`let` is the number of the most recently numbered commit. This is also the total
224+`let` is the number of the most recently numbered commit. This is also
211−number of numbered commits.
225+the total number of numbered commits.
212−
226+
213−`hit` is a map of numerical ids to hashes of commits. These hashes are mapped
227+`hit` is a map of numerical ids to hashes of commits. These hashes are
214−into their associated commits in `hut:rang`. In general, the keys of this map
228+mapped into their associated commits in `hut:rang`. In general, the keys
215−are exactly the numbers from 1 to `let`, with no gaps. Of course, when there are
229+of this map are exactly the numbers from 1 to `let`, with no gaps. Of
216−no numbered commits, `let` is 0, so `hit` is null. Additionally, each of the
230+course, when there are no numbered commits, `let` is 0, so `hit` is
217−commits is an ancestor of every commit numbered greater than this one. Thus,
231+null. Additionally, each of the commits is an ancestor of every commit
232+numbered greater than this one. Thus, each is a descendant of every
218−each is a descendant of every commit numbered less than this one. Since it is
233+commit numbered less than this one. Since it is true that the date in
219−true that the date in each commit (`t:yaki`) is no earlier than that of each of
234+each commit (`t:yaki`) is no earlier than that of each of its parents,
220−its parents, the numbered commits are totally ordered in the same way by both
235+the numbered commits are totally ordered in the same way by both
221−pedigree and date. Of course, not every commit is numbered. If that sounds too
236+pedigree and date. Of course, not every commit is numbered. If that
222−complicated to you, don't worry about it. It basically behaves exactly as you
237+sounds too complicated to you, don't worry about it. It basically
223−would expect.
238+behaves exactly as you would expect.
224−
239+
225−`lab` is a map of textual labels to numbered commits. Note that labels can only
240+`lab` is a map of textual labels to numbered commits. Note that labels
226−be applied to numbered commits. Labels must be unique across a desk.
241+can only be applied to numbered commits. Labels must be unique across a
242+desk.
227
243
228### `++ankh`, filesystem node
244### `++ankh`, filesystem node
229
245
@@ -233,34 +249,35 @@ be applied to numbered commits. Labels must be unique across a desk.
233 r=(map ,@ta ankh) :: folders
249 r=(map ,@ta ankh) :: folders
234 == ::
250 == ::
235
251
236−This is a single node in the filesystem. This may be file or a directory or
252+This is a single node in the filesystem. This may be file or a directory
237−both. In earth filesystems, a node is a file xor a directory. On mars, we're
253+or both. In earth filesystems, a node is a file xor a directory. On
238−inclusive, so a node is a file ior a directory.
254+mars, we're inclusive, so a node is a file ior a directory.
239
255
240`p` is a recursive hash that depends on the contents of the this file or
256`p` is a recursive hash that depends on the contents of the this file or
241directory and on any children.
257directory and on any children.
242
258
243−`q` is the contents of this file, if any. `p.q` is a hash of the contents while
259+`q` is the contents of this file, if any. `p.q` is a hash of the
244−`q.q` is the data itself.
260+contents while `q.q` is the data itself.
245
261
246−`r` is the set of children of this node. In the case of a pure file, this is
262+`r` is the set of children of this node. In the case of a pure file,
247−empty. The keys are the names of the children and the values are, recursively,
263+this is empty. The keys are the names of the children and the values
248−the nodes themselves.
264+are, recursively, the nodes themselves.
249
265
250### `++cash`, ankh hash
266### `++cash`, ankh hash
251
267
252 ++ cash ,@uvH :: ankh hash
268 ++ cash ,@uvH :: ankh hash
253
269
254This is a 128-bit hash of an ankh. These are mostly stored within ankhs
270This is a 128-bit hash of an ankh. These are mostly stored within ankhs
255−themselves, and they are used to check for changes in possibly-deep hierarchies.
271+themselves, and they are used to check for changes in possibly-deep
272+hierarchies.
256
273
257### `++rung`, filesystem per neighbor ship
274### `++rung`, filesystem per neighbor ship
258
275
259 ++ rung $: rus=(map desk rede) :: neighbor desks
276 ++ rung $: rus=(map desk rede) :: neighbor desks
260 == ::
277 == ::
261
278
262−This is the filesystem of a neighbor ship. The keys to this map are all the
279+This is the filesystem of a neighbor ship. The keys to this map are all
263−desks we know about on their ship.
280+the desks we know about on their ship.
264
281
265### `++rede`, desk state
282### `++rede`, desk state
266
283
@@ -271,17 +288,18 @@ desks we know about on their ship.
271 dom=dome :: revision state
288 dom=dome :: revision state
272 == ::
289 == ::
273
290
274−This is our knowledge of the state of a desk, either foreign or domestic.
291+This is our knowledge of the state of a desk, either foreign or
292+domestic.
275
293
276−`lim` is the date of the last full update. We only respond to requests for stuff
294+`lim` is the date of the last full update. We only respond to requests
277−before this time.
295+for stuff before this time.
278
296
279−`qyx` is the list of subscribers to this desk. For domestic desks, this is
297+`qyx` is the list of subscribers to this desk. For domestic desks, this
280−simply `p:dojo`, all subscribers to the desk, while in foreign desks this is all
298+is simply `p:dojo`, all subscribers to the desk, while in foreign desks
281−the subscribers from our ship to the foreign desk.
299+this is all the subscribers from our ship to the foreign desk.
282
300
283−`ref` is the request manager for the desk. For domestic desks, this is null
301+`ref` is the request manager for the desk. For domestic desks, this is
284−since we handle requests ourselves.
302+null since we handle requests ourselves.
285
303
286`dom` is the actual data in the desk.
304`dom` is the actual data in the desk.
287
305
@@ -296,22 +314,23 @@ since we handle requests ourselves.
296
314
297This is the request manager for a foreign desk.
315This is the request manager for a foreign desk.
298
316
299−`nix` is one more than the index of the most recent request. Thus, it is the
317+`nix` is one more than the index of the most recent request. Thus, it is
300−next available request number.
318+the next available request number.
301
319
302−`bom` is the set of outstanding requests. The keys of this map are some subset
320+`bom` is the set of outstanding requests. The keys of this map are some
303−of the numbers between 0 and one less than `nix`. The members of the map are
321+subset of the numbers between 0 and one less than `nix`. The members of
304−exactly those requests that have not yet been fully satisfied.
322+the map are exactly those requests that have not yet been fully
323+satisfied.
305
324
306−`fod` is the same set as `bom`, but from a different perspective. In particular,
325+`fod` is the same set as `bom`, but from a different perspective. In
307−the values of `fod` are the same as the values of `bom`, and the `p` out of the
326+particular, the values of `fod` are the same as the values of `bom`, and
308−values of `bom` are the same as the keys of `fod`. Thus, we can map ducts to
327+the `p` out of the values of `bom` are the same as the keys of `fod`.
309−their associated request number and `++rave`, and we can map numbers to their
328+Thus, we can map ducts to their associated request number and `++rave`,
310−associated duct and `++rave`.
329+and we can map numbers to their associated duct and `++rave`.
311
330
312−`haw` is a map from simple requests to their values. This acts as a cache for
331+`haw` is a map from simple requests to their values. This acts as a
313−requests that have already been made. Thus, the second request for a particular
332+cache for requests that have already been made. Thus, the second request
314−`++mood` is nearly instantaneous.
333+for a particular `++mood` is nearly instantaneous.
315
334
316### `++rang`, data store
335### `++rang`, data store
317
336
@@ -319,23 +338,26 @@ requests that have already been made. Thus, the second request for a particular
319 lat=(map lobe blob) ::
338 lat=(map lobe blob) ::
320 == ::
339 == ::
321
340
322−This is a set of data keyed by hash. Thus, this is where the "real" data is
341+This is a set of data keyed by hash. Thus, this is where the "real" data
323−stored, but it is only meaningful if we know the hash of what we're looking for.
342+is stored, but it is only meaningful if we know the hash of what we're
343+looking for.
324
344
325−`hut` is a map from hashes to commits. We often get the hashes from `hit:dome`,
345+`hut` is a map from hashes to commits. We often get the hashes from
326−which keys them by logical id. Not every commit has an id.
346+`hit:dome`, which keys them by logical id. Not every commit has an id.
327
347
328−`lat` is a map from hashes to the actual data. We often get the hashes from a
348+`lat` is a map from hashes to the actual data. We often get the hashes
329−`++yaki`, a commit, which references this map to get the data. There is no
349+from a `++yaki`, a commit, which references this map to get the data.
330−`++blob` in any `++yaki`. They are only accessible through this map.
350+There is no `++blob` in any `++yaki`. They are only accessible through
351+this map.
331
352
332### `++tako`, commit reference
353### `++tako`, commit reference
333
354
334 ++ tako ,@ :: yaki ref
355 ++ tako ,@ :: yaki ref
335
356
336−This is a hash of a `++yaki`, a commit. These are most notably used as the keys
357+This is a hash of a `++yaki`, a commit. These are most notably used as
337−in `hut:rang`, where they are associated with the actual `++yaki`, and as the
358+the keys in `hut:rang`, where they are associated with the actual
338−values in `hit:dome`, where sequential ids are associated with these.
359+`++yaki`, and as the values in `hit:dome`, where sequential ids are
360+associated with these.
339
361
340### `++yaki`, commit
362### `++yaki`, commit
341
363
@@ -343,14 +365,15 @@ values in `hit:dome`, where sequential ids are associated with these.
343
365
344This is a single commit.
366This is a single commit.
345
367
346−`p` is a list of the hashes of the parents of this commit. In most cases, this
368+`p` is a list of the hashes of the parents of this commit. In most
347−will be a single commit, but in a merge there may be more parents. In theory,
369+cases, this will be a single commit, but in a merge there may be more
348−there may be an arbitrary number of parents, but in practice merges have exactly
370+parents. In theory, there may be an arbitrary number of parents, but in
371+practice merges have exactly two parents. This may change in the future.
349−two parents. This may change in the future. For commit 1, there is no parent.
372+For commit 1, there is no parent.
350
373
351`q` is a map of the paths on a desk to the data at that location. If you
374`q` is a map of the paths on a desk to the data at that location. If you
352−understand what a `++lobe` and a `++blob` is, then the type signature here tells
375+understand what a `++lobe` and a `++blob` is, then the type signature
353−the whole story.
376+here tells the whole story.
354
377
355`r` is the hash associated with this commit.
378`r` is the hash associated with this commit.
356
379
@@ -360,9 +383,9 @@ the whole story.
360
383
361 ++ lobe ,@ :: blob ref
384 ++ lobe ,@ :: blob ref
362
385
363−This is a hash of a `++blob`. These are most notably used in `lat:rang`, where
386+This is a hash of a `++blob`. These are most notably used in `lat:rang`,
364−they are associated with the actual `++blob`, and as the values in `q:yaki`,
387+where they are associated with the actual `++blob`, and as the values in
365−where paths are associated with their data in a commit.
388+`q:yaki`, where paths are associated with their data in a commit.
366
389
367### `++blob`, data
390### `++blob`, data
368
391
@@ -373,21 +396,22 @@ where paths are associated with their data in a commit.
373
396
374This is a node of data. In every case, `p` is the hash of the blob.
397This is a node of data. In every case, `p` is the hash of the blob.
375
398
376−`%delta` is the case where we define the data by a delta on other data. In
399+`%delta` is the case where we define the data by a delta on other data.
377−practice, the other data is always the previous commit, but nothing depends on
400+In practice, the other data is always the previous commit, but nothing
378−this. `q` is the hash of the parent blob, and `r` is the delta.
401+depends on this. `q` is the hash of the parent blob, and `r` is the
402+delta.
379
403
380−`%direct` is the case where we simply have the data directly. `q` is the data
404+`%direct` is the case where we simply have the data directly. `q` is the
381−itself, and `r` is any preprocessing instructions. These almost always come from
405+data itself, and `r` is any preprocessing instructions. These almost
382−the creation of a file.
406+always come from the creation of a file.
383
407
384−`%indirect` is both of the preceding cases at once. `q` is the direct data, `r`
408+`%indirect` is both of the preceding cases at once. `q` is the direct
385−is the delta, and `s` is the parent blob. It should always be the case that
409+data, `r` is the delta, and `s` is the parent blob. It should always be
386−applying `r` to `s` gives the same data as `q` directly (with the
410+the case that applying `r` to `s` gives the same data as `q` directly
387−prepreprocessor instructions in `p.r`). This exists purely for performance
411+(with the prepreprocessor instructions in `p.r`). This exists purely for
388−reasons. This is unused, at the moment, but in general these should be created
412+performance reasons. This is unused, at the moment, but in general these
389−when there are a long line of changes so that we do not have to traverse the
413+should be created when there are a long line of changes so that we do
390−delta chain back to the creation of the file.
414+not have to traverse the delta chain back to the creation of the file.
391
415
392### `++udon`, abstract delta
416### `++udon`, abstract delta
393
417
@@ -401,31 +425,33 @@ delta chain back to the creation of the file.
401 == ::
425 == ::
402 == ::
426 == ::
403
427
404−This is an abstract change to a file. This is a superset of what would normally
428+This is an abstract change to a file. This is a superset of what would
405−be called diffs. Diffs usually refer to changes in lines of text while we have
429+normally be called diffs. Diffs usually refer to changes in lines of
406−the ability to do more interesting deltas on arbitrary data structures.
430+text while we have the ability to do more interesting deltas on
431+arbitrary data structures.
407
432
408`p` is any preprocessor instructions.
433`p` is any preprocessor instructions.
409
434
410−`%a` refers to the trival delta of a complete replace of old data with new data.
435+`%a` refers to the trival delta of a complete replace of old data with
436+new data.
411
437
412−`%b` refers to changes in an opaque atom on the block level. This has very
438+`%b` refers to changes in an opaque atom on the block level. This has
413−limited usefulness, and is not used at the moment.
439+very limited usefulness, and is not used at the moment.
414
440
415−`%c` refers to changes in a list of data. This is often lines of text, which is
441+`%c` refers to changes in a list of data. This is often lines of text,
416−your classic diff. We, however, will work on any list of data.
442+which is your classic diff. We, however, will work on any list of data.
417
443
418−`%d` refers to changes in a tree of data. This is general enough to describe
444+`%d` refers to changes in a tree of data. This is general enough to
419−changes to any hoon noun, but often more special-purpose delta should be created
445+describe changes to any hoon noun, but often more special-purpose delta
420−for different content types. This is not used at the moment, and may in fact be
446+should be created for different content types. This is not used at the
421−unimplemented.
447+moment, and may in fact be unimplemented.
422
448
423### `++urge`, list change
449### `++urge`, list change
424
450
425 ++ urge |*(a=_,* (list (unce a))) :: list change
451 ++ urge |*(a=_,* (list (unce a))) :: list change
426
452
427−This is a parametrized type for list changes. For example, `(urge ,@t)` is a
453+This is a parametrized type for list changes. For example, `(urge ,@t)`
428−list change for lines of text.
454+is a list change for lines of text.
429
455
430### `++unce`, change part of a list.
456### `++unce`, change part of a list.
431
457
@@ -451,8 +477,8 @@ This is a single change in a list of elements of type `a`. For example,
451 $% [%d p=@ud] :: blocklist
477 $% [%d p=@ud] :: blocklist
452 == ::
478 == ::
453
479
454−This space intentionally left undocumented. This stuff will change once we get a
480+This space intentionally left undocumented. This stuff will change once
455−well-typed clay.
481+we get a well-typed clay.
456
482
457### `++upas`, tree change
483### `++upas`, tree change
458
484
@@ -463,9 +489,9 @@ well-typed clay.
463 [%2 p=axis q=udon] :: mutate!
489 [%2 p=axis q=udon] :: mutate!
464 == ::
490 == ::
465
491
466−This space intentionally left undocumented. This stuff is not known to work, and
492+This space intentionally left undocumented. This stuff is not known to
467−will likely change when we get a well-typed clay. Also, this is not a
493+work, and will likely change when we get a well-typed clay. Also, this
468−complicated type; it is not difficult to work out the meaning.
494+is not a complicated type; it is not difficult to work out the meaning.
469
495
470### `++nori`, repository action
496### `++nori`, repository action
471
497
@@ -474,21 +500,22 @@ complicated type; it is not difficult to work out the meaning.
474 [| p=@tas] :: label
500 [| p=@tas] :: label
475 == ::
501 == ::
476
502
477−This describes a change that we are asking clay to make to the desk. There are
503+This describes a change that we are asking clay to make to the desk.
478−two kinds of changes that may be made: we can modify files or we can apply a
504+There are two kinds of changes that may be made: we can modify files or
479−label to a commit.
505+we can apply a label to a commit.
480
506
481−In the `|` case, we will simply label the current commit with the given label.
507+In the `|` case, we will simply label the current commit with the given
482−In the `&` case, we will apply the given changes.
508+label. In the `&` case, we will apply the given changes.
483
509
484### `++soba`, delta
510### `++soba`, delta
485
511
486 ++ soba ,[p=cart q=(list ,[p=path q=miso])] :: delta
512 ++ soba ,[p=cart q=(list ,[p=path q=miso])] :: delta
487
513
488−This describes a set of changes to make to a desk. The `cart` is simply a pair
514+This describes a set of changes to make to a desk. The `cart` is simply
489−of the old hash and the new hash of the desk. The list is a list of changes
515+a pair of the old hash and the new hash of the desk. The list is a list
490−keyed by the file they're changing. Thus, the paths are paths to files to be
516+of changes keyed by the file they're changing. Thus, the paths are paths
491−changed while `miso` is a description of the change itself.
517+to files to be changed while `miso` is a description of the change
518+itself.
492
519
493### `++miso`, ankh delta
520### `++miso`, ankh delta
494
521
@@ -498,38 +525,40 @@ changed while `miso` is a description of the change itself.
498 [%mut p=udon] :: mutate
525 [%mut p=udon] :: mutate
499 == ::
526 == ::
500
527
501−There are three kinds of changes that may be made to a node in a desk. We can
528+There are three kinds of changes that may be made to a node in a desk.
502−insert a file, in which case `p` is the contents of the new file. We can delete
529+We can insert a file, in which case `p` is the contents of the new file.
503−a file, in which case `p` is the contents of the old file. Finally, we can
530+We can delete a file, in which case `p` is the contents of the old file.
504−mutate that file, in which case the `udon` describes the changes we are applying
531+Finally, we can mutate that file, in which case the `udon` describes the
505−to the file.
532+changes we are applying to the file.
506
533
507### `++mizu`, merged state
534### `++mizu`, merged state
508
535
509 ++ mizu ,[p=@u q=(map ,@ud tako) r=rang] :: new state
536 ++ mizu ,[p=@u q=(map ,@ud tako) r=rang] :: new state
510
537
511−This is the input to the `%merg` kiss, which allows us to perform a merge. The
538+This is the input to the `%merg` kiss, which allows us to perform a
512−`p` is the number of the new head commit. The `q` is a map from numbers to
539+merge. The `p` is the number of the new head commit. The `q` is a map
513−commit hashes. This is all the new numbered commits that are to be inserted. The
540+from numbers to commit hashes. This is all the new numbered commits that
514−keys to this should always be the numbers from `let.dom` plus one to `p`,
541+are to be inserted. The keys to this should always be the numbers from
515−inclusive. The `r` is the maps of all the new commits and data. Since these are
542+`let.dom` plus one to `p`, inclusive. The `r` is the maps of all the new
543+commits and data. Since these are merged into the current state, no old
516−merged into the current state, no old commits or data need be here.
544+commits or data need be here.
517
545
518### `++riff`, request/desist
546### `++riff`, request/desist
519
547
520 ++ riff ,[p=desk q=(unit rave)] :: request/desist
548 ++ riff ,[p=desk q=(unit rave)] :: request/desist
521
549
522−This represents a request for data about a particular desk. If `q` contains a
550+This represents a request for data about a particular desk. If `q`
523−`rave`, then this opens a subscription to the desk for that data. If `q` is
551+contains a `rave`, then this opens a subscription to the desk for that
524−null, then this tells clay to cancel the subscription along this duct.
552+data. If `q` is null, then this tells clay to cancel the subscription
553+along this duct.
525
554
526### `++riot`, response
555### `++riot`, response
527
556
528 ++ riot (unit rant) :: response/complete
557 ++ riot (unit rant) :: response/complete
529
558
530−A riot is a response to a subscription. If null, the subscription has been
559+A riot is a response to a subscription. If null, the subscription has
531−completed, and no more responses will be sent. Otherwise, the `rant` is the
560+been completed, and no more responses will be sent. Otherwise, the
532−produced data.
561+`rant` is the produced data.
533
562
534### `++rant`, response data
563### `++rant`, response data
535
564
@@ -539,11 +568,11 @@ produced data.
539 r=* :: data
568 r=* :: data
540 == ::
569 == ::
541
570
542−This is the data at a particular node in the filesystem. `p.p` specifies the
571+This is the data at a particular node in the filesystem. `p.p` specifies
543−type of data that was requested (and is produced). `q.p` gives the specific
572+the type of data that was requested (and is produced). `q.p` gives the
544−version reported (since a range of versions may be requested in a subscription).
573+specific version reported (since a range of versions may be requested in
545−`r.p` is the desk. `q` is the path to the filesystem node. `r` is the data
574+a subscription). `r.p` is the desk. `q` is the path to the filesystem
546−itself (in the format specified by `p.p`).
575+node. `r` is the data itself (in the format specified by `p.p`).
547
576
548### `++nako`, subscription response data
577### `++nako`, subscription response data
549
578
@@ -553,24 +582,26 @@ itself (in the format specified by `p.p`).
553 bar=(set blob) :: new content
582 bar=(set blob) :: new content
554 == ::
583 == ::
555
584
556−This is the data that is produced by a request for a range of revisions of a
585+This is the data that is produced by a request for a range of revisions
557−desk. This allows us to easily keep track of a remote repository -- all the new
586+of a desk. This allows us to easily keep track of a remote repository --
558−information we need is contained in the `nako`.
587+all the new information we need is contained in the `nako`.
559
588
560−`gar` is a map of the revisions in the range to the hash of the commit at that
589+`gar` is a map of the revisions in the range to the hash of the commit
561−revision. These hashes can be used with `hut:rang` to find the commit itself.
590+at that revision. These hashes can be used with `hut:rang` to find the
591+commit itself.
562
592
563`let` is either the last revision number in the range or the most recent
593`let` is either the last revision number in the range or the most recent
564revision number, whichever is smaller.
594revision number, whichever is smaller.
565
595
566`lar` is the set of new commits, and `bar` is the set of new content.
596`lar` is the set of new commits, and `bar` is the set of new content.
567
597
568−## Public Interface
598+Public Interface
599+----------------
569
600
570−As with all vanes, there are exactly two ways to interact with clay. `%clay`
601+As with all vanes, there are exactly two ways to interact with clay.
571−exports a namespace accessible through `.^`, which is described above under
602+`%clay` exports a namespace accessible through `.^`, which is described
572−`++care`. The primary way of interacting with clay, though, is by sending kisses
603+above under `++care`. The primary way of interacting with clay, though,
573−and receiving gifts.
604+is by sending kisses and receiving gifts.
574
605
575 ++ gift :: out result <-$
606 ++ gift :: out result <-$
576 $% [%ergo p=@p q=@tas r=@ud] :: version update
607 $% [%ergo p=@p q=@tas r=@ud] :: version update
@@ -588,82 +619,91 @@ and receiving gifts.
588 [%warp p=sock q=riff] :: file request
619 [%warp p=sock q=riff] :: file request
589 == ::
620 == ::
590
621
591−There are only a small number of possible kisses, so it behooves us to describe
622+There are only a small number of possible kisses, so it behooves us to
592−each in detail.
623+describe each in detail.
593
624
594 $% [%info p=@p q=@tas r=nori] :: internal edit
625 $% [%info p=@p q=@tas r=nori] :: internal edit
595
626
596 [%into p=@p q=@tas r=nori] :: external edit
627 [%into p=@p q=@tas r=nori] :: external edit
597
628
598−These two kisses are nearly identical. At a high level, they apply changes to
629+These two kisses are nearly identical. At a high level, they apply
599−the filesystem. Whenever we add, remove, or edit a file, one of these cards is
630+changes to the filesystem. Whenever we add, remove, or edit a file, one
600−sent. The `p` is the ship whose filesystem we're trying to change, the `q` is
631+of these cards is sent. The `p` is the ship whose filesystem we're
601−the desk we're changing, and the `r` is the request change. For the format of
632+trying to change, the `q` is the desk we're changing, and the `r` is the
633+request change. For the format of the requested change, see the
602−the requested change, see the documentation for `++nori` above.
634+documentation for `++nori` above.
603
635
604−When a file is changed in the unix filesystem, vere will send a `%into` kiss.
636+When a file is changed in the unix filesystem, vere will send a `%into`
605−This tells clay that the duct over which the kiss was sent is the duct that unix
637+kiss. This tells clay that the duct over which the kiss was sent is the
606−is listening on for changes. From within Arvo, though, we should never send a
638+duct that unix is listening on for changes. From within Arvo, though, we
607−`%into` kiss. The `%info` kiss is exactly identical except it does not reset the
639+should never send a `%into` kiss. The `%info` kiss is exactly identical
608−duct.
640+except it does not reset the duct.
609
641
610 [%ingo p=@p q=@tas r=nori] :: internal noun edit
642 [%ingo p=@p q=@tas r=nori] :: internal noun edit
611
643
612 [%invo p=@p q=@tas r=nori] :: external noun edit
644 [%invo p=@p q=@tas r=nori] :: external noun edit
613
645
614−These kisses are currently identical to `%info` and `%into`, though this will
646+These kisses are currently identical to `%info` and `%into`, though this
615−not always be the case. The intent is for these kisses to allow typed changes to
647+will not always be the case. The intent is for these kisses to allow
616−clay so that we may store typed data. This is currently unimplemented.
648+typed changes to clay so that we may store typed data. This is currently
649+unimplemented.
617
650
618 [%init p=@p] :: report install
651 [%init p=@p] :: report install
619
652
620−Init is called when a ship is started on our pier. This simply creates a default
653+Init is called when a ship is started on our pier. This simply creates a
621−`room` to go into our `raft`. Essentially, this initializes the filesystem for a
654+default `room` to go into our `raft`. Essentially, this initializes the
622−ship.
655+filesystem for a ship.
623
656
624 [%merg p=@p q=@tas r=mizu] :: internal change
657 [%merg p=@p q=@tas r=mizu] :: internal change
625
658
626−This is called to perform a merge. This is most visibly called by :update to
659+This is called to perform a merge. This is most visibly called by
627−update the filesystem of the current ship to that of its sein. The `p` and `q`
660+:update to update the filesystem of the current ship to that of its
628−are as in `%info`, and the `r` is the description of the merge. See `++mizu`
661+sein. The `p` and `q` are as in `%info`, and the `r` is the description
629−above.
662+of the merge. See `++mizu` above.
630
663
631XX
664XX
632`XX [%wake ~] :: timer activate XX`
665`XX [%wake ~] :: timer activate XX`
666+XX\
633−XX\\ XX This card is sent by unix at the time specified by `++doze`. This time
667+XX This card is sent by unix at the time specified by `++doze`. This
634−is XX usually the closest time specified in a subscription request. When `%wake`
668+time is XX usually the closest time specified in a subscription request.
635−is XX called, we update our subscribers if there have been any changes.
669+When `%wake` is XX called, we update our subscribers if there have been
670+any changes.
636
671
637 [%wart p=sock q=@tas r=path s=*] :: network request
672 [%wart p=sock q=@tas r=path s=*] :: network request
638
673
639−This is a request that has come across the network for a particular file. When
674+This is a request that has come across the network for a particular
640−another ship asks for a file from us, that request comes to us in the form of a
675+file. When another ship asks for a file from us, that request comes to
641−`%wart` kiss. This is handled by trivially turning it into a `%warp`.
676+us in the form of a `%wart` kiss. This is handled by trivially turning
677+it into a `%warp`.
642
678
643 [%warp p=sock q=riff] :: file request
679 [%warp p=sock q=riff] :: file request
644
680
645−This is a request for information about a particular desk. This is, in its most
681+This is a request for information about a particular desk. This is, in
646−general form, a subscription, though in many cases it is the trivial case of a
682+its most general form, a subscription, though in many cases it is the
647−subscription -- a read. See `++riff` for the format of the request.
683+trivial case of a subscription -- a read. See `++riff` for the format of
684+the request.
648
685
649−## Lifecycle of a Local Read
686+Lifecycle of a Local Read
687+-------------------------
650
688
651−There are two real types of interaction with a filesystem: you can read, and you
689+There are two real types of interaction with a filesystem: you can read,
652−can write. We'll describe each process, detailing both the flow of control
690+and you can write. We'll describe each process, detailing both the flow
653−followed by the kernel and the algorithms involved. The simpler case is that of
691+of control followed by the kernel and the algorithms involved. The
654−the read, so we'll begin with that.
692+simpler case is that of the read, so we'll begin with that.
655
693
656−When a vane or an application wishes to read a file from the filesystem, it
694+When a vane or an application wishes to read a file from the filesystem,
657−sends a `%warp` kiss, as described above. Of course, you may request a file on
695+it sends a `%warp` kiss, as described above. Of course, you may request
658−another ship and, being a global filesystem, clay will happily produce it for
696+a file on another ship and, being a global filesystem, clay will happily
659−you. That code pathway will be described in another section; here, we will
697+produce it for you. That code pathway will be described in another
660−restrict ourselves to examining the case of a read from a ship on our own pier.
698+section; here, we will restrict ourselves to examining the case of a
699+read from a ship on our own pier.
661
700
662−The kiss can request either a single version of a file node or a range of
701+The kiss can request either a single version of a file node or a range
663−versions of a desk. Here, we'll deal only with a request for a single version.
702+of versions of a desk. Here, we'll deal only with a request for a single
703+version.
664
704
665−As in all vanes, a kiss enters clay via a call to `++call`. Scanning through the
705+As in all vanes, a kiss enters clay via a call to `++call`. Scanning
666−arm, we quickly see where `%warp` is handled.
706+through the arm, we quickly see where `%warp` is handled.
667
707
668 ?: =(p.p.q.hic q.p.q.hic)
708 ?: =(p.p.q.hic q.p.q.hic)
669 =+ une=(un p.p.q.hic now ruf)
709 =+ une=(un p.p.q.hic now ruf)
@@ -676,31 +716,33 @@ arm, we quickly see where `%warp` is handled.
676 abet:wao
716 abet:wao
677 [-.woo abet:(pish:une p.q.q.hic +.woo ran.wao)]
717 [-.woo abet:(pish:une p.q.q.hic +.woo ran.wao)]
678
718
679−We're following the familar patern of producing a list of moves and an updated
719+We're following the familar patern of producing a list of moves and an
680−state. In this case, the state is `++raft`.
720+updated state. In this case, the state is `++raft`.
681
721
682−We first check to see if the sending and receiving ships are the same. If
722+We first check to see if the sending and receiving ships are the same.
683−they're not, then this is a request for data on another ship. We describe that
723+If they're not, then this is a request for data on another ship. We
684−process later. Here, we discuss only the case of a local read.
724+describe that process later. Here, we discuss only the case of a local
725+read.
685
726
686−At a high level, the call to `++un` sets up the core for the domestic ship that
727+At a high level, the call to `++un` sets up the core for the domestic
687−contains the files we're looking for. The call to `++di` sets up the core for
728+ship that contains the files we're looking for. The call to `++di` sets
688−the particular desk we're referring to.
729+up the core for the particular desk we're referring to.
689
730
690−After this, we perform the actual request. If there is no rave in the riff, then
731+After this, we perform the actual request. If there is no rave in the
691−that means we are cancelling a request, so we call `++ease:de`. Otherwise, we
732+riff, then that means we are cancelling a request, so we call
692−start a subscription with `++eave:de`. We call `++abet:de` to resolve our
733+`++ease:de`. Otherwise, we start a subscription with `++eave:de`. We
734+call `++abet:de` to resolve our various types of output into actual
693−various types of output into actual moves. We produce the moves we found above
735+moves. We produce the moves we found above and the `++un` core resolved
694−and the `++un` core resolved with `++pish:un` (putting the modified desk in the
736+with `++pish:un` (putting the modified desk in the room) and `++abet:un`
695−room) and `++abet:un` (putting the modified room in the raft).
737+(putting the modified room in the raft).
696
738
697Much of this is fairly straightforward, so we'll only describe `++ease`,
739Much of this is fairly straightforward, so we'll only describe `++ease`,
698−`++eave`, and `++abet:de`. Feel free to look up the code to the other steps --
740+`++eave`, and `++abet:de`. Feel free to look up the code to the other
699−it should be easy to follow.
741+steps -- it should be easy to follow.
700
742
701−Although it's called last, it's usually worth examining `++abet` first, since it
743+Although it's called last, it's usually worth examining `++abet` first,
702−defines in what ways we can cause side effects. Let's do that, and also a few of
744+since it defines in what ways we can cause side effects. Let's do that,
703−the lines at the beginning of `++de`.
745+and also a few of the lines at the beginning of `++de`.
704
746
705 =| yel=(list ,[p=duct q=gift])
747 =| yel=(list ,[p=duct q=gift])
706 =| byn=(list ,[p=duct q=riot])
748 =| byn=(list ,[p=duct q=riot])
@@ -730,31 +772,31 @@ the lines at the beginning of `++de`.
730 |=([a=duct b=path c=note] [a %pass b c])
772 |=([a=duct b=path c=note] [a %pass b c])
731 ==
773 ==
732
774
733−This is very simple code. We see there are exactly five different kinds of side
775+This is very simple code. We see there are exactly five different kinds
734−effects we can generate.
776+of side effects we can generate.
735
777
736−In `yel` we put gifts that we wish to be sent along the `hun:room` duct to dill.
778+In `yel` we put gifts that we wish to be sent along the `hun:room` duct
737−See the documentation for `++room` above. This is how we display messages to the
779+to dill. See the documentation for `++room` above. This is how we
738−terminal.
780+display messages to the terminal.
739
781
740−In `byn` we put riots that we wish returned to subscribers. Recall that a riot
782+In `byn` we put riots that we wish returned to subscribers. Recall that
741−is a response to a subscription. These are returned to our subscribers in the
783+a riot is a response to a subscription. These are returned to our
742−form of a `%writ` gift.
784+subscribers in the form of a `%writ` gift.
743
785
744−In `vag` we put gifts along with the ducts on which to send them. This allows us
786+In `vag` we put gifts along with the ducts on which to send them. This
745−to produce arbitrary gifts, but in practice this is only used to produce `%ergo`
787+allows us to produce arbitrary gifts, but in practice this is only used
746−gifts.
788+to produce `%ergo` gifts.
747
789
748−In `say` we put messages we wish to pass to ames. These messages are used to
790+In `say` we put messages we wish to pass to ames. These messages are
749−request information from clay on other piers. We must provide not only the duct
791+used to request information from clay on other piers. We must provide
750−and the request (the riff), but also the return path, the other ship to talk to,
792+not only the duct and the request (the riff), but also the return path,
751−and the sequence number of the request.
793+the other ship to talk to, and the sequence number of the request.
752
794
753−In `tag` we put arbitrary notes we wish to pass to other vanes. For now, the
795+In `tag` we put arbitrary notes we wish to pass to other vanes. For now,
754−only notes we pass here are `%wait` and `%rest` to the timer vane.
796+the only notes we pass here are `%wait` and `%rest` to the timer vane.
755
797
756−Now that we know what kinds of side effects we may have, we can jump into the
798+Now that we know what kinds of side effects we may have, we can jump
757−handling of requests.
799+into the handling of requests.
758
800
759 ++ ease :: release request
801 ++ ease :: release request
760 |= hen=duct
802 |= hen=duct
@@ -773,24 +815,24 @@ handling of requests.
773 bom.u.ref (~(del by bom.u.ref) u.nux)
815 bom.u.ref (~(del by bom.u.ref) u.nux)
774 ==
816 ==
775
817
776−This is called when we're cancelling a subscription. For domestic desks, `ref`
818+This is called when we're cancelling a subscription. For domestic desks,
777−is null, so we're going to cancel any timer we might have created. We first
819+`ref` is null, so we're going to cancel any timer we might have created.
778−delete the duct from our map of requests, and then we call `++mabe` with
820+We first delete the duct from our map of requests, and then we call
779−`++best` to send a `%rest` kiss to the timer vane if we have started a timer.
821+`++mabe` with `++best` to send a `%rest` kiss to the timer vane if we
780−We'll describe `++best` and `++mabe` momentarily.
822+have started a timer. We'll describe `++best` and `++mabe` momentarily.
781
823
782−Although we said we're not going to talk about foreign requests yet, it's easy
824+Although we said we're not going to talk about foreign requests yet,
783−to see that for foreign desks, we cancel any outstanding requests for this duct
825+it's easy to see that for foreign desks, we cancel any outstanding
784−and send a message over ames to the other ship telling them to cancel the
826+requests for this duct and send a message over ames to the other ship
785−subscription.
827+telling them to cancel the subscription.
786
828
787 ++ best
829 ++ best
788 |= [hen=duct tym=@da]
830 |= [hen=duct tym=@da]
789 %_(+> tag :_(tag [hen /tyme %t %rest tym]))
831 %_(+> tag :_(tag [hen /tyme %t %rest tym]))
790
832
791−This simply pushes a `%rest` note onto `tag`, from where it will be passed back
833+This simply pushes a `%rest` note onto `tag`, from where it will be
792−to arvo to be handled. This cancels the timer at the given duct (with the given
834+passed back to arvo to be handled. This cancels the timer at the given
793−time).
835+duct (with the given time).
794
836
795 ++ mabe :: maybe fire function
837 ++ mabe :: maybe fire function
796 |* [rov=rove fun=$+(@da _+>.^$)]
838 |* [rov=rove fun=$+(@da _+>.^$)]
@@ -811,15 +853,15 @@ time).
811 ?.((lth now p.q.mot) [~ now] [~ p.q.mot])
853 ?.((lth now p.q.mot) [~ now] [~ p.q.mot])
812 ==
854 ==
813
855
814−This decides whether the given request can only be satsified in the future. In
856+This decides whether the given request can only be satsified in the
815−that case, we call the given function with the time in the future when we expect
857+future. In that case, we call the given function with the time in the
816−to have an update to give to this request. This is called with `++best` to
858+future when we expect to have an update to give to this request. This is
817−cancel timers and with `++bait` to start them.
859+called with `++best` to cancel timers and with `++bait` to start them.
818
860
819−For single requests, we have a time if the request is for a particular time
861+For single requests, we have a time if the request is for a particular
820−(which is assumed to be in the future). For ranges of requests, we check both
862+time (which is assumed to be in the future). For ranges of requests, we
821−the start and end cases to see if they are time cases. If so, we choose the
863+check both the start and end cases to see if they are time cases. If so,
822−earlier time.
864+we choose the earlier time.
823
865
824If any of those give us a time, then we call the given funciton with the
866If any of those give us a time, then we call the given funciton with the
825smallest time.
867smallest time.
@@ -842,22 +884,22 @@ subscription but starting one.
842 (blub hen)
884 (blub hen)
843 (blab hen p.rav u.u.ver)
885 (blab hen p.rav u.u.ver)
844
886
845−There are two types of subscriptions -- either we're requesting a single file or
887+There are two types of subscriptions -- either we're requesting a single
846−we're requesting a range of versions of a desk. We'll dicuss the simpler case
888+file or we're requesting a range of versions of a desk. We'll dicuss the
847−first.
889+simpler case first.
848
890
849−First, we check that we're not requesting the `rang` from any time other than
891+First, we check that we're not requesting the `rang` from any time other
850−the present. Since we don't store that information for any other time, we can't
892+than the present. Since we don't store that information for any other
851−produce it in a referentially transparent manner for any time other than the
893+time, we can't produce it in a referentially transparent manner for any
852−present.
894+time other than the present.
853
895
854−Then, we try to read the requested `mood` `p.rav`. If we can't access the
896+Then, we try to read the requested `mood` `p.rav`. If we can't access
855−request data right now, we call `++duce` to put the request in our queue to be
897+the request data right now, we call `++duce` to put the request in our
856−satisfied when the information becomes available.
898+queue to be satisfied when the information becomes available.
857
899
858−This case occurs when we make a request for a case whose (1) date is after the
900+This case occurs when we make a request for a case whose (1) date is
859−current date, (2) number is after the current number, or (3) label is not yet
901+after the current date, (2) number is after the current number, or (3)
860−used.
902+label is not yet used.
861
903
862 ++ duce :: produce request
904 ++ duce :: produce request
863 |= [hen=duct rov=rove]
905 |= [hen=duct rov=rove]
@@ -878,41 +920,43 @@ used.
878 fod.u.ref (~(put by fod.u.ref) hen inx)
920 fod.u.ref (~(put by fod.u.ref) hen inx)
879 ==
921 ==
880
922
881−The code for `++duce` is nearly the exact inverse of `++ease`, which in the case
923+The code for `++duce` is nearly the exact inverse of `++ease`, which in
882−of a domestic desk is very simple -- we simply put the duct and rave into `qyx`
924+the case of a domestic desk is very simple -- we simply put the duct and
883−and possibly start a timer with `++mabe` and `++bait`. Recall that `ref` is null
925+rave into `qyx` and possibly start a timer with `++mabe` and `++bait`.
884−for domestic desks and that `++mabe` fires the given function with the time we
926+Recall that `ref` is null for domestic desks and that `++mabe` fires the
927+given function with the time we need to be woken up at, if we need to be
885−need to be woken up at, if we need to be woken up at a particular time.
928+woken up at a particular time.
886
929
887 ++ bait
930 ++ bait
888 |= [hen=duct tym=@da]
931 |= [hen=duct tym=@da]
889 %_(+> tag :_(tag [hen /tyme %t %wait tym]))
932 %_(+> tag :_(tag [hen /tyme %t %wait tym]))
890
933
891−This sets an alarm by sending a `%wait` card with the given time to the timer
934+This sets an alarm by sending a `%wait` card with the given time to the
892−vane.
935+timer vane.
893
936
894−Back in `++eave`, if `++aver` returned `[~ ~]`, then we cancel the subscription.
937+Back in `++eave`, if `++aver` returned `[~ ~]`, then we cancel the
895−This occurs when we make (1) a `%x` request for a file that does not exist, (2)
938+subscription. This occurs when we make (1) a `%x` request for a file
896−a `%w` request with a case that is not a number, or (3) a `%w` request with a
939+that does not exist, (2) a `%w` request with a case that is not a
940+number, or (3) a `%w` request with a nonempty path. The `++blub` is
897−nonempty path. The `++blub` is exactly what you would expect it to be.
941+exactly what you would expect it to be.
898
942
899 ++ blub :: ship stop
943 ++ blub :: ship stop
900 |= hen=duct
944 |= hen=duct
901 %_(+> byn [[hen ~] byn])
945 %_(+> byn [[hen ~] byn])
902
946
903−We notify the duct that we're cancelling their subscription since it isn't
947+We notify the duct that we're cancelling their subscription since it
904−satisfiable.
948+isn't satisfiable.
905
949
906−Otherwise, we have received the desired information, so we send it on to the
950+Otherwise, we have received the desired information, so we send it on to
907−subscriber with `++blab`.
951+the subscriber with `++blab`.
908
952
909 ++ blab :: ship result
953 ++ blab :: ship result
910 |= [hen=duct mun=mood dat=*]
954 |= [hen=duct mun=mood dat=*]
911 ^+ +>
955 ^+ +>
912 +>(byn [[hen ~ [p.mun q.mun syd] r.mun dat] byn])
956 +>(byn [[hen ~ [p.mun q.mun syd] r.mun dat] byn])
913
957
914−The most interesting arm called in `++eave` is, of course, `++aver`, where we
958+The most interesting arm called in `++eave` is, of course, `++aver`,
915−actually try to read the data.
959+where we actually try to read the data.
916
960
917 ++ aver :: read
961 ++ aver :: read
918 |= mun=mood
962 |= mun=mood
@@ -925,19 +969,19 @@ actually try to read the data.
925 =+ nao=(~(case-to-aeon ze lim dom ran) q.mun)
969 =+ nao=(~(case-to-aeon ze lim dom ran) q.mun)
926 ?~(nao ~ [~ (~(read-at-aeon ze lim dom ran) u.nao mun)])
970 ?~(nao ~ [~ (~(read-at-aeon ze lim dom ran) u.nao mun)])
927
971
928−We check immediately that we're not requesting the `rang` for any time other
972+We check immediately that we're not requesting the `rang` for any time
929−than the present.
973+other than the present.
930
974
931−If this is a foreign desk, then we check our cache for the specific request. If
975+If this is a foreign desk, then we check our cache for the specific
932−either this is a domestic desk or we don't have the request in our cache, then
976+request. If either this is a domestic desk or we don't have the request
933−we have to actually go read the data from our dome.
977+in our cache, then we have to actually go read the data from our dome.
934
978
935We need to do two things. First, we try to find the number of the commit
979We need to do two things. First, we try to find the number of the commit
936specified by the given case, and then we try to get the data there.
980specified by the given case, and then we try to get the data there.
937
981
938−Here, we jump into `arvo/zuse.hoon`, which is where much of the algorithmic code
982+Here, we jump into `arvo/zuse.hoon`, which is where much of the
939−is stored, as opposed to the clay interface, which is stored in
983+algorithmic code is stored, as opposed to the clay interface, which is
940−`arvo/clay.hoon`. We examine `++case-to-aeon:ze`.
984+stored in `arvo/clay.hoon`. We examine `++case-to-aeon:ze`.
941
985
942 ++ case-to-aeon :: case-to-aeon:ze
986 ++ case-to-aeon :: case-to-aeon:ze
943 |= lok=case :: act count through
987 |= lok=case :: act count through
@@ -959,20 +1003,22 @@ is stored, as opposed to the clay interface, which is stored in
959 %ud ?:((gth p.lok let) ~ [~ p.lok])
1003 %ud ?:((gth p.lok let) ~ [~ p.lok])
960 ==
1004 ==
961
1005
962−We handle each type of `case` differently. The latter two types are easy.
1006+We handle each type of `case` differently. The latter two types are
1007+easy.
963
1008
964−If we're requesting a revision by label, then we simply look up the requested
1009+If we're requesting a revision by label, then we simply look up the
965−label in `lab` from the given dome. If it exists, that is our aeon; else we
1010+requested label in `lab` from the given dome. If it exists, that is our
966−produce null, indicating the requested revision does not yet exist.
1011+aeon; else we produce null, indicating the requested revision does not
1012+yet exist.
967
1013
968−If we're requesting a revision by number, we check if we've yet reached that
1014+If we're requesting a revision by number, we check if we've yet reached
969−number. If so, we produce the number; else we produce null.
1015+that number. If so, we produce the number; else we produce null.
970
1016
971−If we're requesting a revision by date, we check first if the date is in the
1017+If we're requesting a revision by date, we check first if the date is in
972−future, returning null if so. Else we start from the most recent revision and
1018+the future, returning null if so. Else we start from the most recent
973−scan backwards until we find the first revision committed before that date, and
1019+revision and scan backwards until we find the first revision committed
974−we produce that. If we requested a date before any revisions were committed, we
1020+before that date, and we produce that. If we requested a date before any
975−produce `0`.
1021+revisions were committed, we produce `0`.
976
1022
977The definitions of `++aeon-to-tako` and `++tako-to-yaki` are trivial.
1023The definitions of `++aeon-to-tako` and `++tako-to-yaki` are trivial.
978
1024
@@ -980,10 +1026,12 @@ The definitions of `++aeon-to-tako` and `++tako-to-yaki` are trivial.
980
1026
981 ++ tako-to-yaki ~(got by hut) :: grab yaki
1027 ++ tako-to-yaki ~(got by hut) :: grab yaki
982
1028
983−We simply look up the aeon or tako in their respective maps (`hit` and `hut`).
1029+We simply look up the aeon or tako in their respective maps (`hit` and
1030+`hut`).
984
1031
985−Assuming we got a valid version number, `++aver` calls `++read-at-aeon:ze`,
1032+Assuming we got a valid version number, `++aver` calls
986−which reads the requested data at the given revision.
1033+`++read-at-aeon:ze`, which reads the requested data at the given
1034+revision.
987
1035
988 ++ read-at-aeon :: read-at-aeon:ze
1036 ++ read-at-aeon :: read-at-aeon:ze
989 |= [oan=aeon mun=mood] :: seek and read
1037 |= [oan=aeon mun=mood] :: seek and read
@@ -992,10 +1040,10 @@ which reads the requested data at the given revision.
992 ?^(r.mun ~ [~ oan])
1040 ?^(r.mun ~ [~ oan])
993 (read:(rewind oan) mun)
1041 (read:(rewind oan) mun)
994
1042
995−If we're requesting the revision number with a case other than by number, then
1043+If we're requesting the revision number with a case other than by
996−we go ahead and just produce the number we were given. Otherwise, we call
1044+number, then we go ahead and just produce the number we were given.
997−`++rewind` to rewind our state to the given revision, and then we call `++read`
1045+Otherwise, we call `++rewind` to rewind our state to the given revision,
998−to get the requested information.
1046+and then we call `++read` to get the requested information.
999
1047
1000 ++ rewind :: rewind:ze
1048 ++ rewind :: rewind:ze
1001 |= oan=aeon :: rewind to aeon
1049 |= oan=aeon :: rewind to aeon
@@ -1004,13 +1052,14 @@ to get the requested information.
1004 ?: (gth oan let) !! :: don't have version
1052 ?: (gth oan let) !! :: don't have version
1005 +>(ank (checkout-ankh q:(tako-to-yaki (aeon-to-tako oan))), let oan)
1053 +>(ank (checkout-ankh q:(tako-to-yaki (aeon-to-tako oan))), let oan)
1006
1054
1007−If we're already at the requested version, we do nothing. If we're requesting a
1055+If we're already at the requested version, we do nothing. If we're
1008−version later than our head, we are unable to comply.
1056+requesting a version later than our head, we are unable to comply.
1009
1057
1010−Otherwise, we get the hash of the commit at the number, and from that we get the
1058+Otherwise, we get the hash of the commit at the number, and from that we
1011−commit itself (the yaki), which has the map of path to lobe that represents a
1059+get the commit itself (the yaki), which has the map of path to lobe that
1012−version of the filesystem. We call `++checkout-ankh` to checkout the commit, and
1060+represents a version of the filesystem. We call `++checkout-ankh` to
1013−we replace `ank` in our context with the result.
1061+checkout the commit, and we replace `ank` in our context with the
1062+result.
1014
1063
1015 ++ checkout-ankh :: checkout-ankh:ze
1064 ++ checkout-ankh :: checkout-ankh:ze
1016 |= hat=(map path lobe) :: checkout commit
1065 |= hat=(map path lobe) :: checkout commit
@@ -1029,8 +1078,9 @@ we replace `ank` in our context with the result.
1029 $(pat t.pat, ank (fall nak _ankh))
1078 $(pat t.pat, ank (fall nak _ankh))
1030 ==
1079 ==
1031
1080
1032−Twice we call `++cosh`, which hashes a commit, updating `p` in an `ankh`. Let's
1081+Twice we call `++cosh`, which hashes a commit, updating `p` in an
1033−jump into that algorithm before we describe `++checkout-ankh`.
1082+`ankh`. Let's jump into that algorithm before we describe
1083+`++checkout-ankh`.
1034
1084
1035 ++ cosh :: locally rehash
1085 ++ cosh :: locally rehash
1036 |= ank=ankh :: NB v/unix.c
1086 |= ank=ankh :: NB v/unix.c
@@ -1056,30 +1106,33 @@ We simply replace `p` in the hash with the `cash` we get from a call to
1056 $(r.ank r.r.ank, axe (peg axe 3))
1106 $(r.ank r.r.ank, axe (peg axe 3))
1057 ==
1107 ==
1058
1108
1059−`++zu` is a core we set up with a particular filesystem node to traverse a
1109+`++zu` is a core we set up with a particular filesystem node to traverse
1060−checkout of the filesystem and access the actual data inside it. One of the
1110+a checkout of the filesystem and access the actual data inside it. One
1061−things we can do with it is to create a recursive hash of the node.
1111+of the things we can do with it is to create a recursive hash of the
1112+node.
1062
1113
1063−In `++rehash`, if this node is a file, then we xor the remainder of the hash
1114+In `++rehash`, if this node is a file, then we xor the remainder of the
1064−with the hash of the contents of the file. The remainder of the hash is `0` if
1115+hash with the hash of the contents of the file. The remainder of the
1065−we have no children, else we descend into our children. Basically, we do a half
1116+hash is `0` if we have no children, else we descend into our children.
1066−SHA-256 of the xor of the axis of this child and the half SHA-256 of the xor of
1117+Basically, we do a half SHA-256 of the xor of the axis of this child and
1067−the name of the child and the hash of the child. This is done for each child and
1118+the half SHA-256 of the xor of the name of the child and the hash of the
1068−all the results are xored together.
1119+child. This is done for each child and all the results are xored
1120+together.
1069
1121
1070Now we return to our discussion of `++checkout-ankh`.
1122Now we return to our discussion of `++checkout-ankh`.
1071
1123
1072−We fold over every path in this version of the filesystem and create a great
1124+We fold over every path in this version of the filesystem and create a
1073−ankh out of them. First, we call `++lobe-to-noun` to get the raw data referred
1125+great ankh out of them. First, we call `++lobe-to-noun` to get the raw
1074−to be each lobe.
1126+data referred to be each lobe.
1075
1127
1076 ++ lobe-to-noun :: grab blob
1128 ++ lobe-to-noun :: grab blob
1077 |= p=lobe :: ^- *
1129 |= p=lobe :: ^- *
1078 %- blob-to-noun
1130 %- blob-to-noun
1079 (lobe-to-blob p)
1131 (lobe-to-blob p)
1080
1132
1081−This converts a lobe into the raw data it refers to by first getting the blob
1133+This converts a lobe into the raw data it refers to by first getting the
1082−with `++lobe-to-blob` and converting that into data with `++blob-to-noun`.
1134+blob with `++lobe-to-blob` and converting that into data with
1135+`++blob-to-noun`.
1083
1136
1084 ++ lobe-to-blob ~(got by lat) :: grab blob
1137 ++ lobe-to-blob ~(got by lat) :: grab blob
1085
1138
@@ -1093,10 +1146,10 @@ This just grabs the blob that the lobe refers to.
1093 %indirect q.p
1146 %indirect q.p
1094 ==
1147 ==
1095
1148
1096−If we have either a direct or an indirect blob, then the data is stored right in
1149+If we have either a direct or an indirect blob, then the data is stored
1097−the blob. Otherwise, we have to reconstruct it from the diffs. We do this by
1150+right in the blob. Otherwise, we have to reconstruct it from the diffs.
1098−calling `++lump` on the diff in the blob with the data obtained by recursively
1151+We do this by calling `++lump` on the diff in the blob with the data
1099−calling the parent of this blob.
1152+obtained by recursively calling the parent of this blob.
1100
1153
1101 ++ lump :: apply patch
1154 ++ lump :: apply patch
1102 |= [don=udon src=*]
1155 |= [don=udon src=*]
@@ -1118,12 +1171,13 @@ calling the parent of this blob.
1118 ==
1171 ==
1119 ==
1172 ==
1120
1173
1121−This is defined in `arvo/hoon.hoon` for historical reasons which are likely no
1174+This is defined in `arvo/hoon.hoon` for historical reasons which are
1122−longer applicable. Since the `++umph` structure will likely change we convert
1175+likely no longer applicable. Since the `++umph` structure will likely
1123−clay to be a typed filesystem, we'll only give a high-level description of this
1176+change we convert clay to be a typed filesystem, we'll only give a
1177+high-level description of this process. If we have a `%a` udon, then
1124−process. If we have a `%a` udon, then we're performing a trivial replace, so we
1178+we're performing a trivial replace, so we produce simply `q.q.don`. If
1125−produce simply `q.q.don`. If we have a `%c` udon, then we're performing a list
1179+we have a `%c` udon, then we're performing a list merge (as in, for
1126−merge (as in, for example, lines of text). The merge is performed by `++lurk`.
1180+example, lines of text). The merge is performed by `++lurk`.
1127
1181
1128 ++ lurk :: apply list patch
1182 ++ lurk :: apply list patch
1129 |* [hel=(list) rug=(urge)]
1183 |* [hel=(list) rug=(urge)]
@@ -1150,16 +1204,17 @@ merge (as in, for example, lines of text). The merge is performed by `++lurk`.
1150 ==
1204 ==
1151 ==
1205 ==
1152
1206
1153−We accumulate our final result in `war`. If there's nothing more in our list of
1207+We accumulate our final result in `war`. If there's nothing more in our
1154−merge instructions (unces), we just reverse `war` and produce it. Otherwise, we
1208+list of merge instructions (unces), we just reverse `war` and produce
1155−process another unce. If the unce is of type `&`, then we have `p.i.rug` lines
1209+it. Otherwise, we process another unce. If the unce is of type `&`, then
1156−of no changes, so we just copy them over from `hel` to `war`. If the unice is of
1210+we have `p.i.rug` lines of no changes, so we just copy them over from
1211+`hel` to `war`. If the unice is of type `|`, then we verify that the
1157−type `|`, then we verify that the source lines (in `hel`) are what we expect
1212+source lines (in `hel`) are what we expect them to be (`p.i.rug`),
1158−them to be (`p.i.rug`), crashing on failure. If they're good, then we append the
1213+crashing on failure. If they're good, then we append the new lines in
1159−new lines in `q.i.rug` onto `war`.
1214+`q.i.rug` onto `war`.
1160
1215
1161−And that's really it. List merges are pretty easy. Anyway, if you recall, we
1216+And that's really it. List merges are pretty easy. Anyway, if you
1162−were discussing `++checkout-ankh`.
1217+recall, we were discussing `++checkout-ankh`.
1163
1218
1164 ++ checkout-ankh :: checkout-ankh:ze
1219 ++ checkout-ankh :: checkout-ankh:ze
1165 |= hat=(map path lobe) :: checkout commit
1220 |= hat=(map path lobe) :: checkout commit
@@ -1178,16 +1233,16 @@ were discussing `++checkout-ankh`.
1178 $(pat t.pat, ank (fall nak _ankh))
1233 $(pat t.pat, ank (fall nak _ankh))
1179 ==
1234 ==
1180
1235
1181−If the path is null, then we calculate `zar`, the raw data at the path `pat` in
1236+If the path is null, then we calculate `zar`, the raw data at the path
1182−this version. We produce the given ankh with the correct data.
1237+`pat` in this version. We produce the given ankh with the correct data.
1183
1238
1184−Otherwise, we try to get the child we're looking at from our parent ankh. If
1239+Otherwise, we try to get the child we're looking at from our parent
1185−it's already been created, this succeeds; otherwise, we simply create a default
1240+ankh. If it's already been created, this succeeds; otherwise, we simply
1186−blank ankh. We place ourselves in our parent after recursively computing our
1241+create a default blank ankh. We place ourselves in our parent after
1187−children.
1242+recursively computing our children.
1188
1243
1189−This algorithm really isn't that complicated, but it may not be immediately
1244+This algorithm really isn't that complicated, but it may not be
1190−obvious why it works. An example should clear everything up.
1245+immediately obvious why it works. An example should clear everything up.
1191
1246
1192Suppose `hat` is a map of the following information.
1247Suppose `hat` is a map of the following information.
1193
1248
@@ -1207,60 +1262,63 @@ Furthermore, let's say that we process them in this order:
1207 /greeting/spanish
1262 /greeting/spanish
1208 /farewell/russian
1263 /farewell/russian
1209
1264
1210−Then, the first path we process is `/greeting/english` . Since our path is not
1265+Then, the first path we process is `/greeting/english` . Since our path
1211−null, we try to get `nak`, but because our ankh is blank at this point it
1266+is not null, we try to get `nak`, but because our ankh is blank at this
1212−doesn't find anything. Thus, update our blank top-level ankh with a child
1267+point it doesn't find anything. Thus, update our blank top-level ankh
1213−`%greeting`. and recurse with the blank `nak` to create the ankh of the new
1268+with a child `%greeting`. and recurse with the blank `nak` to create the
1214−child.
1269+ankh of the new child.
1215−
1270+
1216−In the recursion, we our path is `/english` and our ankh is again blank. We try
1271+In the recursion, we our path is `/english` and our ankh is again blank.
1217−to get the `english` child of our ankh, but this of course fails. Thus, we
1272+We try to get the `english` child of our ankh, but this of course fails.
1218−update our blank `/greeting` ankh with a child `english` produced by recursing.
1273+Thus, we update our blank `/greeting` ankh with a child `english`
1219−
1274+produced by recursing.
1275+
1220−Now our path is null, so we call `++lobe-to-noun` to get the actual data, and we
1276+Now our path is null, so we call `++lobe-to-noun` to get the actual
1221−place it in the brand-new ankh.
1277+data, and we place it in the brand-new ankh.
1222−
1278+
1223−Next, we process `/greeting/russian/short`. Since our path is not null, we try
1279+Next, we process `/greeting/russian/short`. Since our path is not null,
1224−to get the child named `%greeting`, which does exist since we created it
1280+we try to get the child named `%greeting`, which does exist since we
1225−earlier. We put modify this child by recursing on it. Our path is now
1281+created it earlier. We put modify this child by recursing on it. Our
1226−`/russian/short`, so we look for a `%russian` child in our `/greeting` ankh.
1282+path is now `/russian/short`, so we look for a `%russian` child in our
1227−This doesn't exist, so we add it by recursing. Our path is now `/short`, so we
1283+`/greeting` ankh. This doesn't exist, so we add it by recursing. Our
1284+path is now `/short`, so we look for a `%short` child in our
1228−look for a `%short` child in our `/greeting/russian` ankh. This doesn't exist,
1285+`/greeting/russian` ankh. This doesn't exist, so we add it by recursing.
1229−so we add it by recursing. Now our path is null, so we set the contents of this
1286+Now our path is null, so we set the contents of this node to `"привет"`,
1230−node to `"привет"`, and we're done processing this path.
1287+and we're done processing this path.
1231−
1288+
1232−Next, we process `/greeting/russian/long`. This proceeds similarly to the
1289+Next, we process `/greeting/russian/long`. This proceeds similarly to
1233−previous except that now the ankh for `/greeting/russian` already exists, so we
1290+the previous except that now the ankh for `/greeting/russian` already
1234−simply reuse it rather than creating a new one. Of course, we still must create
1291+exists, so we simply reuse it rather than creating a new one. Of course,
1235−a new `/greeting/russian/long` ankh.
1292+we still must create a new `/greeting/russian/long` ankh.
1236−
1293+
1237−Next, we process `/greeting`. This ankh already exists, so after we've recursed
1294+Next, we process `/greeting`. This ankh already exists, so after we've
1238−once, our path is null, and our ankh is not blank -- it already has two children
1295+recursed once, our path is null, and our ankh is not blank -- it already
1239−(and two grandchildren). We don't touch those, though, since a node may be both
1296+has two children (and two grandchildren). We don't touch those, though,
1240−a file and a directory. We just add the contents of the file -- "customary upon
1297+since a node may be both a file and a directory. We just add the
1241−meeting" -- to the existing ankh.
1298+contents of the file -- "customary upon meeting" -- to the existing
1242−
1299+ankh.
1300+
1243−Next, we process `/greeting/spanish`. Of course, the `/greeting` ankh already
1301+Next, we process `/greeting/spanish`. Of course, the `/greeting` ankh
1244−exists, but it doesn't have a `%spanish` child, so we create that, taking care
1302+already exists, but it doesn't have a `%spanish` child, so we create
1245−not to disturb the contents of the `/greeting` file. We put "hola" into the ankh
1303+that, taking care not to disturb the contents of the `/greeting` file.
1246−and call it good.
1304+We put "hola" into the ankh and call it good.
1247−
1305+
1248−Finally, we process `/farewell/russian`. Here, the `/farewell` ankh doesn't
1306+Finally, we process `/farewell/russian`. Here, the `/farewell` ankh
1249−exist, so we create it. Clearly the newly-created ankh doesn't have any
1307+doesn't exist, so we create it. Clearly the newly-created ankh doesn't
1250−children, so we have to add a `%russian` child, and in this child we put our
1308+have any children, so we have to add a `%russian` child, and in this
1251−last content -- "до свидания".
1309+child we put our last content -- "до свидания".
1252−
1310+
1253−We hope it's fairly obvious that the order we process the paths doesn't affect
1311+We hope it's fairly obvious that the order we process the paths doesn't
1254−the final ankh tree. The tree will be constructed in a very different order
1312+affect the final ankh tree. The tree will be constructed in a very
1255−depending on what order the paths come in, but the resulting tree is independent
1313+different order depending on what order the paths come in, but the
1256−of order.
1314+resulting tree is independent of order.
1257−
1315+
1258−At any rate, we were talking about something important, weren't we? If you
1316+At any rate, we were talking about something important, weren't we? If
1259−recall, that concludes our discussion of `++rewind`, which was called from
1317+you recall, that concludes our discussion of `++rewind`, which was
1260−`++read-at-aeon`. In summary, `++rewind` returns a context in which our current
1318+called from `++read-at-aeon`. In summary, `++rewind` returns a context
1261−state is (very nearly) as it was when the specified version of the desk was the
1319+in which our current state is (very nearly) as it was when the specified
1262−head. This allows `++read-at-aeon` to call `++read` to read the requested
1320+version of the desk was the head. This allows `++read-at-aeon` to call
1263−information.
1321+`++read` to read the requested information.
1264
1322
1265 ++ read :: read:ze
1323 ++ read :: read:ze
1266 |= mun=mood :: read at point
1324 |= mun=mood :: read at point
@@ -1280,13 +1338,14 @@ information.
1280
1338
1281If we're requesting the dome, then we just return that immediately.
1339If we're requesting the dome, then we just return that immediately.
1282
1340
1283−If we're requesting the revision number of the desk and we're not requesting it
1341+If we're requesting the revision number of the desk and we're not
1284−by number, then we just return the current number of this desk. Note of course
1342+requesting it by number, then we just return the current number of this
1285−that this was really already handled in `++read-at-aeon`.
1343+desk. Note of course that this was really already handled in
1344+`++read-at-aeon`.
1286
1345
1287−If we're requesting a `%w` with a specific revision number, then we do something
1346+If we're requesting a `%w` with a specific revision number, then we do
1288−or other with the commit there. It's kind of weird, and it doesn't seem to work,
1347+something or other with the commit there. It's kind of weird, and it
1289−so we'll ignore this case.
1348+doesn't seem to work, so we'll ignore this case.
1290
1349
1291Otherwise, we descend into the ankh tree to the given path with
1350Otherwise, we descend into the ankh tree to the given path with
1292`++descend-path:zu`, and then we handle specific request in `++query`.
1351`++descend-path:zu`, and then we handle specific request in `++query`.
@@ -1296,8 +1355,8 @@ Otherwise, we descend into the ankh tree to the given path with
1296 ^+ +>
1355 ^+ +>
1297 ?~(way +> $(way t.way, +> (descend i.way)))
1356 ?~(way +> $(way t.way, +> (descend i.way)))
1298
1357
1299−This is simple recursion down into the ankh tree. `++descend` descends one
1358+This is simple recursion down into the ankh tree. `++descend` descends
1300−level, so this will eventually get us down to the path we want.
1359+one level, so this will eventually get us down to the path we want.
1301
1360
1302 ++ descend :: descend
1361 ++ descend :: descend
1303 |= lol=@ta
1362 |= lol=@ta
@@ -1305,12 +1364,12 @@ level, so this will eventually get us down to the path we want.
1305 =+ you=(~(get by r.ank) lol)
1364 =+ you=(~(get by r.ank) lol)
1306 +>.$(ram [lol ram], ank ?~(you [*cash ~ ~] u.you))
1365 +>.$(ram [lol ram], ank ?~(you [*cash ~ ~] u.you))
1307
1366
1308−`ram` is the path that we're at, so to descend one level we push the name of
1367+`ram` is the path that we're at, so to descend one level we push the
1309−this level onto that path. We update the ankh with the correct one at that path
1368+name of this level onto that path. We update the ankh with the correct
1310−if it exists; else we create a blank one.
1369+one at that path if it exists; else we create a blank one.
1311
1370
1312−Once we've decscended to the correct level, we need to actually deal with the
1371+Once we've decscended to the correct level, we need to actually deal
1313−request.
1372+with the request.
1314
1373
1315 ++ query :: query:ze
1374 ++ query :: query:ze
1316 |= ren=?(%u %v %x %y %z) :: endpoint query
1375 |= ren=?(%u %v %x %y %z) :: endpoint query
@@ -1323,9 +1382,10 @@ request.
1323 %z [~ ank]
1382 %z [~ ank]
1324 ==
1383 ==
1325
1384
1326−Now that everything's set up, it's really easy. If they're requesting the rang,
1385+Now that everything's set up, it's really easy. If they're requesting
1327−dome, or ankh, we give it to them. If the contents of a file, we give it to them
1386+the rang, dome, or ankh, we give it to them. If the contents of a file,
1328−if it is in fact a file. If the `arch`, then we calculate it with `++as-arch`.
1387+we give it to them if it is in fact a file. If the `arch`, then we
1388+calculate it with `++as-arch`.
1329
1389
1330 ++ as-arch :: as-arch:ze
1390 ++ as-arch :: as-arch:ze
1331 ^- arch :: arch report
1391 ^- arch :: arch report
@@ -1335,16 +1395,17 @@ if it is in fact a file. If the `arch`, then we calculate it with `++as-arch`.
1335 ?~ r.ank ~
1395 ?~ r.ank ~
1336 [[p.n.r.ank ~] $(r.ank l.r.ank) $(r.ank r.r.ank)]
1396 [[p.n.r.ank ~] $(r.ank l.r.ank) $(r.ank r.r.ank)]
1337
1397
1338−This very simply strips out all the "real" data and returns just our own hash,
1398+This very simply strips out all the "real" data and returns just our own
1339−the hash of the file contents (if we're a file), and a map of the names of our
1399+hash, the hash of the file contents (if we're a file), and a map of the
1340−immediate children.
1400+names of our immediate children.
1341
1401
1342−## Lifecycle of a Local Subscription
1402+Lifecycle of a Local Subscription
1403+---------------------------------
1343
1404
1344−A subscription to a range of revisions of a desk initially follows the same path
1405+A subscription to a range of revisions of a desk initially follows the
1345−that a single read does. In `++aver`, we checked the head of the given rave. If
1406+same path that a single read does. In `++aver`, we checked the head of
1346−the head was `&`, then it was a single request, so we handled it above. If `|`,
1407+the given rave. If the head was `&`, then it was a single request, so we
1347−then we handle it with the following code.
1408+handled it above. If `|`, then we handle it with the following code.
1348
1409
1349 =+ nab=(~(case-to-aeon ze lim dom ran) p.p.rav)
1410 =+ nab=(~(case-to-aeon ze lim dom ran) p.p.rav)
1350 ?~ nab
1411 ?~ nab
@@ -1368,46 +1429,51 @@ then we handle it with the following code.
1368 ==
1429 ==
1369
1430
1370Recall that `++case-to-aeon:ze` produces the revision number that a case
1431Recall that `++case-to-aeon:ze` produces the revision number that a case
1371−corresponds to, if it corresponds to any. If it doesn't yet correspond to a
1432+corresponds to, if it corresponds to any. If it doesn't yet correspond
1372−revision, then it produces null.
1433+to a revision, then it produces null.
1373−
1434+
1374−Thus, we first check to see if we've even gotten to the beginning of the range
1435+Thus, we first check to see if we've even gotten to the beginning of the
1375−of revisions requested. If not, then we assert that we haven't yet gotten to the
1436+range of revisions requested. If not, then we assert that we haven't yet
1376−end of the range either, because that would be really strange. If not, then we
1437+gotten to the end of the range either, because that would be really
1377−immediately call `++duce`, which, if you recall, for a local request, simply
1438+strange. If not, then we immediately call `++duce`, which, if you
1439+recall, for a local request, simply puts this duct and rove into our
1378−puts this duct and rove into our cult `qyx`, so that we know who to respond to
1440+cult `qyx`, so that we know who to respond to when the revision does
1379−when the revision does appear.
1441+appear.
1380−
1442+
1381−If we've already gotten to the first revision, then we can produce some content
1443+If we've already gotten to the first revision, then we can produce some
1382−immediately. If we've also gotten to the final revision, and that revision is
1444+content immediately. If we've also gotten to the final revision, and
1383−earlier than the start revision, then it's a bad request and we call `++blub`,
1445+that revision is earlier than the start revision, then it's a bad
1446+request and we call `++blub`, which tells the subscriber that his
1384−which tells the subscriber that his subscription will not be satisfied.
1447+subscription will not be satisfied.
1385
1448
1386Otherwise, we find the data at the given path at the beginning of the
1449Otherwise, we find the data at the given path at the beginning of the
1387−subscription and at the last available revision in the subscription. If they're
1450+subscription and at the last available revision in the subscription. If
1388−the same, then we don't send a notification. Otherwise, we call `++gack`, which
1451+they're the same, then we don't send a notification. Otherwise, we call
1389−creates the `++nako` we need to produce. We call `++bleb` to actually produce
1452+`++gack`, which creates the `++nako` we need to produce. We call
1390−the information.
1453+`++bleb` to actually produce the information.
1391
1454
1392−If we already have the last requested revision, then we also tell the subscriber
1455+If we already have the last requested revision, then we also tell the
1393−with `++blub` that the subscription will receive no further updates.
1456+subscriber with `++blub` that the subscription will receive no further
1457+updates.
1394
1458
1395−If there will be more revisions in the subscription, then we call `++duce`,
1459+If there will be more revisions in the subscription, then we call
1396−adding the duct to our subscribers. We modify the rove to start at the next
1460+`++duce`, adding the duct to our subscribers. We modify the rove to
1397−revision since we've already handled all the revisions up to the present.
1461+start at the next revision since we've already handled all the revisions
1462+up to the present.
1398
1463
1399−We glossed over the calls to `++lobes-at-path`, `++make-nako`, and `++bleb`, so
1464+We glossed over the calls to `++lobes-at-path`, `++make-nako`, and
1400−we'll get back to those right now. `++bleb` is simple, so we'll start with that.
1465+`++bleb`, so we'll get back to those right now. `++bleb` is simple, so
1466+we'll start with that.
1401
1467
1402 ++ bleb :: ship sequence
1468 ++ bleb :: ship sequence
1403 |= [hen=duct ins=@ud hip=nako]
1469 |= [hen=duct ins=@ud hip=nako]
1404 ^+ +>
1470 ^+ +>
1405 (blab hen [%w [%ud ins] ~] hip)
1471 (blab hen [%w [%ud ins] ~] hip)
1406
1472
1407−We're given a duct, the beginning revision number, and the nako that contains
1473+We're given a duct, the beginning revision number, and the nako that
1408−the updates since that revision. We use `++blab` to produce this result to the
1474+contains the updates since that revision. We use `++blab` to produce
1409−subscriber. The case is `%w` with a revision number of the beginning of the
1475+this result to the subscriber. The case is `%w` with a revision number
1410−subscription, and the data is the nako itself.
1476+of the beginning of the subscription, and the data is the nako itself.
1411
1477
1412We call `++lobes-at-path:ze` to get the data at the particular path.
1478We call `++lobes-at-path:ze` to get the data at the particular path.
1413
1479
@@ -1430,14 +1496,15 @@ We call `++lobes-at-path:ze` to get the data at the particular path.
1430 $(p +.p, pax +.pax)
1496 $(p +.p, pax +.pax)
1431 == ==
1497 == ==
1432
1498
1433−At revision zero, the theoretical common revision between all repositories,
1499+At revision zero, the theoretical common revision between all
1434−there is no data, so we produce null.
1500+repositories, there is no data, so we produce null.
1435
1501
1436−We get the list of paths (paired with their lobe) in the revision referred to by
1502+We get the list of paths (paired with their lobe) in the revision
1437−the given number and we keep only those paths which begin with `pax`. Converting
1503+referred to by the given number and we keep only those paths which begin
1438−to a map, we now have a map from the subpaths at the given path to the hash of
1504+with `pax`. Converting to a map, we now have a map from the subpaths at
1439−their data. This is simple and efficient to calculate and compare to later
1505+the given path to the hash of their data. This is simple and efficient
1506+to calculate and compare to later revisions. This allows us to easily
1440−revisions. This allows us to easily tell if a node or its children have changed.
1507+tell if a node or its children have changed.
1441
1508
1442Finally, we will describe `++make-nako:ze`.
1509Finally, we will describe `++make-nako:ze`.
1443
1510
@@ -1454,17 +1521,19 @@ Finally, we will describe `++make-nako:ze`.
1454 |= [p=aeon *]
1521 |= [p=aeon *]
1455 &((gth p a) (lte p b))
1522 &((gth p a) (lte p b))
1456
1523
1457−We need to produce four things -- the numbers of the new commits, the number of
1524+We need to produce four things -- the numbers of the new commits, the
1458−the latest commit, the new commits themselves, and the new data itself.
1525+number of the latest commit, the new commits themselves, and the new
1526+data itself.
1459
1527
1460−The first is fairly easy to produce. We simply go over our map of numbered
1528+The first is fairly easy to produce. We simply go over our map of
1461−commits and produce all those numbered greater than `a` and not greater than
1529+numbered commits and produce all those numbered greater than `a` and not
1462−`b`.
1530+greater than `b`.
1463
1531
1464−The second is even easier to produce -- `b` is clearly our most recent commit.
1532+The second is even easier to produce -- `b` is clearly our most recent
1533+commit.
1465
1534
1466−The third and fourth are slightly more interesting, though not too terribly
1535+The third and fourth are slightly more interesting, though not too
1467−difficult. First, we call `++reachable-between-takos`.
1536+terribly difficult. First, we call `++reachable-between-takos`.
1468
1537
1469 ++ reachable-between-takos
1538 ++ reachable-between-takos
1470 |= [a=(unit tako) b=tako] :: pack a through b
1539 |= [a=(unit tako) b=tako] :: pack a through b
@@ -1488,12 +1557,12 @@ difficult. First, we call `++reachable-between-takos`.
1488 %- ~(uni in bar)
1557 %- ~(uni in bar)
1489 ^$(yak (tako-to-yaki yek))
1558 ^$(yak (tako-to-yaki yek))
1490
1559
1491−We take a possible starting commit and a definite ending commit, and we produce
1560+We take a possible starting commit and a definite ending commit, and we
1492−the set of commits and the set of data between them.
1561+produce the set of commits and the set of data between them.
1493
1562
1494−We let `sar` be the set of commits reachable from `a`. If `a` is null, then
1563+We let `sar` be the set of commits reachable from `a`. If `a` is null,
1495−obviously no commits are reachable. Otherwise, we call `++reachable-takos` to
1564+then obviously no commits are reachable. Otherwise, we call
1496−calculate this.
1565+`++reachable-takos` to calculate this.
1497
1566
1498 ++ reachable-takos :: reachable
1567 ++ reachable-takos :: reachable
1499 |= p=tako :: XX slow
1568 |= p=tako :: XX slow
@@ -1506,16 +1575,17 @@ calculate this.
1506 s :: hence skip
1575 s :: hence skip
1507 (~(uni in s) ^$(p q)) :: otherwise traverse
1576 (~(uni in s) ^$(p q)) :: otherwise traverse
1508
1577
1509−We very simply produce the set of the given tako plus its parents, recursively.
1578+We very simply produce the set of the given tako plus its parents,
1579+recursively.
1510
1580
1511−Back in `++reachable-between-takos`, we let `yak` be the yaki of `b`, the ending
1581+Back in `++reachable-between-takos`, we let `yak` be the yaki of `b`,
1512−commit. With this, we create a set that is the union of `sar` and all takos
1582+the ending commit. With this, we create a set that is the union of `sar`
1513−reachable from `b`.
1583+and all takos reachable from `b`.
1514
1584
1515−We pass `sar` into `++new-lobes` to get all the lobes referenced by any tako
1585+We pass `sar` into `++new-lobes` to get all the lobes referenced by any
1516−referenced by `a`. The result is passed into `++new-lobes-takos` to do the same,
1586+tako referenced by `a`. The result is passed into `++new-lobes-takos` to
1517−but not recomputing those in already calculated last sentence. This produces the
1587+do the same, but not recomputing those in already calculated last
1518−sets of takos and lobes we need.
1588+sentence. This produces the sets of takos and lobes we need.
1519
1589
1520 ++ new-lobes :: object hash set
1590 ++ new-lobes :: object hash set
1521 |= [b=(set lobe) a=(set tako)] :: that aren't in b
1591 |= [b=(set lobe) a=(set tako)] :: that aren't in b
@@ -1536,34 +1606,37 @@ sets of takos and lobes we need.
1536 %indirect (~(put in $(lob s.gar)) lob)
1606 %indirect (~(put in $(lob s.gar)) lob)
1537 ==
1607 ==
1538
1608
1539−Here, we're creating a set of lobes referenced in a commit in `a`. We start out
1609+Here, we're creating a set of lobes referenced in a commit in `a`. We
1540−with `b` as the initial set of lobes, so we don't need to recompute any of the
1610+start out with `b` as the initial set of lobes, so we don't need to
1541−lobes referenced in there.
1611+recompute any of the lobes referenced in there.
1542
1612
1543−The algorithm is pretty simple, so we won't bore you with the details. We simply
1613+The algorithm is pretty simple, so we won't bore you with the details.
1544−traverse every commit in `a`, looking at every blob referenced there, and, if
1614+We simply traverse every commit in `a`, looking at every blob referenced
1545−it's not already in `b`, we add it to `b`. In the case of a direct blob, we're
1615+there, and, if it's not already in `b`, we add it to `b`. In the case of
1616+a direct blob, we're done. For a delta or an indirect blob, we
1546−done. For a delta or an indirect blob, we recursively add every blob referenced
1617+recursively add every blob referenced within the blob.
1547−within the blob.
1548
1618
1549 ++ new-lobes-takos :: garg & repack
1619 ++ new-lobes-takos :: garg & repack
1550 |= [b=(set lobe) a=(set tako)]
1620 |= [b=(set lobe) a=(set tako)]
1551 ^- [(set tako) (set lobe)]
1621 ^- [(set tako) (set lobe)]
1552 [a (new-lobes b a)]
1622 [a (new-lobes b a)]
1553
1623
1554−Here, we just update the set of lobes we're given with the commits we're given
1624+Here, we just update the set of lobes we're given with the commits we're
1555−and produce both sets.
1625+given and produce both sets.
1556
1626
1557This concludes our discussion of a local subscription.
1627This concludes our discussion of a local subscription.
1558
1628
1559−## Lifecycle of a Foreign Read or Subscription
1629+Lifecycle of a Foreign Read or Subscription
1630+-------------------------------------------
1560
1631
1561−Foreign reads and subscriptions are handled in much the same way as local ones.
1632+Foreign reads and subscriptions are handled in much the same way as
1562−The interface is the same -- a vane or app sends a `%warp` kiss with the
1633+local ones. The interface is the same -- a vane or app sends a `%warp`
1563−request. The difference is simply that the `sock` refers to the foreign ship.
1634+kiss with the request. The difference is simply that the `sock` refers
1635+to the foreign ship.
1564
1636
1565−Thus, we start in the same place -- in `++call`, handling `%warp`. However,
1637+Thus, we start in the same place -- in `++call`, handling `%warp`.
1566−since the two side of the `sock` are different, we follow a different path.
1638+However, since the two side of the `sock` are different, we follow a
1639+different path.
1567
1640
1568 =+ wex=(do now p.q.hic p.q.q.hic ruf)
1641 =+ wex=(do now p.q.hic p.q.q.hic ruf)
1569 =+ ^= woo
1642 =+ ^= woo
@@ -1572,13 +1645,14 @@ since the two side of the `sock` are different, we follow a different path.
1572 abet:(eave:wex hen u.q.q.q.hic)
1645 abet:(eave:wex hen u.q.q.q.hic)
1573 [-.woo (posh q.p.q.hic p.q.q.hic +.woo ruf)]
1646 [-.woo (posh q.p.q.hic p.q.q.hic +.woo ruf)]
1574
1647
1575−If we compare this to how the local case was handled, we see that it's not all
1648+If we compare this to how the local case was handled, we see that it's
1576−that different. We use `++do` rather than `++un` and `++de` to set up the core
1649+not all that different. We use `++do` rather than `++un` and `++de` to
1577−for the foreign ship. This gives us a `++de` core, so we either cancel or begin
1650+set up the core for the foreign ship. This gives us a `++de` core, so we
1578−the request by calling `++ease` or `++eave`, exactly as in the local case. In
1651+either cancel or begin the request by calling `++ease` or `++eave`,
1652+exactly as in the local case. In either case, we call `++abet:de` to
1579−either case, we call `++abet:de` to resolve our various types of output into
1653+resolve our various types of output into actual moves, as described in
1580−actual moves, as described in the local case. Finally, we call `++posh` to
1654+the local case. Finally, we call `++posh` to update our raft, putting
1581−update our raft, putting the modified rung into the raft.
1655+the modified rung into the raft.
1582
1656
1583We'll first trace through `++do`.
1657We'll first trace through `++do`.
1584
1658
@@ -1592,16 +1666,17 @@ We'll first trace through `++do`.
1592 ?^(yit u.yit `rede`[~2000.1.1 ~ [~ *rind] *dome])
1666 ?^(yit u.yit `rede`[~2000.1.1 ~ [~ *rind] *dome])
1593 ((de now ~ ~) [who him] syd red ran.ruf)
1667 ((de now ~ ~) [who him] syd red ran.ruf)
1594
1668
1595−If we already have a rung for this foreign ship, then we use that. Otherwise, we
1669+If we already have a rung for this foreign ship, then we use that.
1596−create a new blank one. If we already have a rede in this rung, then we use
1670+Otherwise, we create a new blank one. If we already have a rede in this
1597−that, otherwise we create a blank one. An important point to note here is that
1671+rung, then we use that, otherwise we create a blank one. An important
1672+point to note here is that we let `ref` in the rede be `[~ *rind]`.
1598−we let `ref` in the rede be `[~ *rind]`. Recall, for domestic desks `ref` is
1673+Recall, for domestic desks `ref` is null. We use this to distinguish
1599−null. We use this to distinguish between foreign and domestic desks in `++de`.
1674+between foreign and domestic desks in `++de`.
1600
1675
1601With this information, we create a `++de` core as usual.
1676With this information, we create a `++de` core as usual.
1602
1677
1603−Although we've already covered `++ease` and `++eave`, we'll go through them
1678+Although we've already covered `++ease` and `++eave`, we'll go through
1604−quickly again, highlighting the case of foreign request.
1679+them quickly again, highlighting the case of foreign request.
1605
1680
1606 ++ ease :: release request
1681 ++ ease :: release request
1607 |= hen=duct
1682 |= hen=duct
@@ -1620,19 +1695,20 @@ quickly again, highlighting the case of foreign request.
1620 bom.u.ref (~(del by bom.u.ref) u.nux)
1695 bom.u.ref (~(del by bom.u.ref) u.nux)
1621 ==
1696 ==
1622
1697
1623−Here, we still remove the duct from our cult (we maintain a cult even for
1698+Here, we still remove the duct from our cult (we maintain a cult even
1624−foreign desks), but we also need to tell the foreign desk to cancel our
1699+for foreign desks), but we also need to tell the foreign desk to cancel
1625−subscription. We do this by sending a request (by appending to `say`, which gets
1700+our subscription. We do this by sending a request (by appending to
1626−resolved in `++abet:de` to a `%want` kiss to ames) to the foreign ship to cancel
1701+`say`, which gets resolved in `++abet:de` to a `%want` kiss to ames) to
1627−the subscription. Since we don't anymore expect a response on this duct, we
1702+the foreign ship to cancel the subscription. Since we don't anymore
1703+expect a response on this duct, we remove it from `fod` and `bom`, which
1628−remove it from `fod` and `bom`, which are the maps between ducts, raves, and
1704+are the maps between ducts, raves, and request sequence numbers.
1629−request sequence numbers. Basically, we remove every trace of the subscription
1705+Basically, we remove every trace of the subscription from our request
1630−from our request manager.
1706+manager.
1631−
1707+
1632−In the case of `++eave`, where we're creating a new request, everything is
1708+In the case of `++eave`, where we're creating a new request, everything
1633−exactly identical to the case of the local request except `++duce`. We said that
1709+is exactly identical to the case of the local request except `++duce`.
1634−`++duce` simply puts the request into our cult. This is true for a domestic
1710+We said that `++duce` simply puts the request into our cult. This is
1635−request, but distinctly untrue for foreign requests.
1711+true for a domestic request, but distinctly untrue for foreign requests.
1636
1712
1637 ++ duce :: produce request
1713 ++ duce :: produce request
1638 |= [hen=duct rov=rove]
1714 |= [hen=duct rov=rove]
@@ -1652,21 +1728,22 @@ request, but distinctly untrue for foreign requests.
1652 fod.u.ref (~(put by fod.u.ref) hen inx)
1728 fod.u.ref (~(put by fod.u.ref) hen inx)
1653 ==
1729 ==
1654
1730
1655−If we have a request manager (i.e. this is a foreign desk), then we do the
1731+If we have a request manager (i.e. this is a foreign desk), then we do
1656−approximate inverse of `++ease`. We create a rave out of the given request and
1732+the approximate inverse of `++ease`. We create a rave out of the given
1657−send it off to the foreign desk by putting it in `say`. Note that the rave is
1733+request and send it off to the foreign desk by putting it in `say`. Note
1658−created to request the information starting at the next revision number. Since
1734+that the rave is created to request the information starting at the next
1659−this is a new request, we put it into `fod` and `bom` to associate the request
1735+revision number. Since this is a new request, we put it into `fod` and
1660−with its duct and its sequence number. Since we're using another sequence
1736+`bom` to associate the request with its duct and its sequence number.
1737+Since we're using another sequence number, we must increment `nix`,
1661−number, we must increment `nix`, which represents the next available sequence
1738+which represents the next available sequence number.
1662−number.
1663
1739
1664And that's really it for this side of the request. Requesting foreign
1740And that's really it for this side of the request. Requesting foreign
1665−information isn't that hard. Let's see what it looks like on the other side.
1741+information isn't that hard. Let's see what it looks like on the other
1666−When we get a request from another ship for information on our ship, that comes
1742+side. When we get a request from another ship for information on our
1667−to us in the form of a `%wart` from ames.
1743+ship, that comes to us in the form of a `%wart` from ames.
1668
1744
1669−We handle a `%wart` in `++call`, right next to where we handle the `%warp` case.
1745+We handle a `%wart` in `++call`, right next to where we handle the
1746+`%warp` case.
1670
1747
1671 %wart
1748 %wart
1672 ?> ?=(%re q.q.hic)
1749 ?> ?=(%re q.q.hic)
@@ -1678,11 +1755,11 @@ We handle a `%wart` in `++call`, right next to where we handle the `%warp` case.
1678 [%warp [p.p.q.hic p.p.q.hic] ryf]
1755 [%warp [p.p.q.hic p.p.q.hic] ryf]
1679 ==
1756 ==
1680
1757
1681−Every request we receive should be of type `riff`, so we coerce it into that
1758+Every request we receive should be of type `riff`, so we coerce it into
1682−type. We just convert this into a new `%warp` kiss that we pass to ourself. This
1759+that type. We just convert this into a new `%warp` kiss that we pass to
1683−gets handled like normal, as a local request. When the request produces a value,
1760+ourself. This gets handled like normal, as a local request. When the
1684−it does so like normal as a `%writ`, which is returned to `++take` along the
1761+request produces a value, it does so like normal as a `%writ`, which is
1685−path we just sent it on.
1762+returned to `++take` along the path we just sent it on.
1686
1763
1687 %writ
1764 %writ
1688 ?> ?=([@ @ *] tea)
1765 ?> ?=([@ @ *] tea)
@@ -1693,12 +1770,12 @@ path we just sent it on.
1693 [%pass ~ %a [%want [our him] [%r %re %c t.t.tea] p.+.q.hin]]
1770 [%pass ~ %a [%want [our him] [%r %re %c t.t.tea] p.+.q.hin]]
1694 ==
1771 ==
1695
1772
1696−Since we encoded the ship we need to respond to in the path, we can just pass
1773+Since we encoded the ship we need to respond to in the path, we can just
1697−our `%want` back to ames, so that we tell the requesting ship about the new
1774+pass our `%want` back to ames, so that we tell the requesting ship about
1698−data.
1775+the new data.
1699
1776
1700−This comes back to the original ship as a `%waft` from ames, which comes into
1777+This comes back to the original ship as a `%waft` from ames, which comes
1701−`++take`, right next to where we handled `%writ`.
1778+into `++take`, right next to where we handled `%writ`.
1702
1779
1703 %waft
1780 %waft
1704 ?> ?=([@ @ ~] tea)
1781 ?> ?=([@ @ ~] tea)
@@ -1712,10 +1789,11 @@ This comes back to the original ship as a `%waft` from ames, which comes into
1712 [-.zot (posh q.p.+.q.hin syd +.zot ruf)]
1789 [-.zot (posh q.p.+.q.hin syd +.zot ruf)]
1713 [mos ..^$(ran.ruf ran.zat)] :: merge in new obj
1790 [mos ..^$(ran.ruf ran.zat)] :: merge in new obj
1714
1791
1715−This gets the desk and sequence number from the path the request was sent over.
1792+This gets the desk and sequence number from the path the request was
1716−This determines exactly which request is being responded to. We call `++knit:de`
1793+sent over. This determines exactly which request is being responded to.
1717−to apply the changes to our local desk, and we call `++wake` to update our
1794+We call `++knit:de` to apply the changes to our local desk, and we call
1718−subscribers. Then we call `++abet:de` and `++posh` as normal (like in `++eave`).
1795+`++wake` to update our subscribers. Then we call `++abet:de` and
1796+`++posh` as normal (like in `++eave`).
1719
1797
1720We'll examine `++knit` and `++wake`, in that order.
1798We'll examine `++knit` and `++wake`, in that order.
1721
1799
@@ -1758,33 +1836,36 @@ We'll examine `++knit` and `++wake`, in that order.
1758
1836
1759This is kind of a long gate, but don't worry, it's not bad at all.
1837This is kind of a long gate, but don't worry, it's not bad at all.
1760
1838
1761−First, we assert that we're not a domestic desk. That wouldn't make any sense at
1839+First, we assert that we're not a domestic desk. That wouldn't make any
1762−all.
1840+sense at all.
1763−
1841+
1764−Since we have the sequence number of the request, we can get the duct and rave
1842+Since we have the sequence number of the request, we can get the duct
1765−from `bom` in our request manager. If we didn't actually request this data (or
1843+and rave from `bom` in our request manager. If we didn't actually
1766−the request was canceled before we got it), then we do nothing.
1844+request this data (or the request was canceled before we got it), then
1767−
1845+we do nothing.
1846+
1768−Else, we remove the request from `bom` and `fod` unless this was a subscription
1847+Else, we remove the request from `bom` and `fod` unless this was a
1769−request and we didn't receive a null riot (which would indicate the last message
1848+subscription request and we didn't receive a null riot (which would
1770−on the subscription).
1849+indicate the last message on the subscription).
1771−
1850+
1772−Now, if we received a null riot, then if this was a subscription request by
1851+Now, if we received a null riot, then if this was a subscription request
1773−date, then we update `lim` in our request manager (representing the latest time
1852+by date, then we update `lim` in our request manager (representing the
1774−at which we have complete information for this desk) to the date that was
1853+latest time at which we have complete information for this desk) to the
1775−requested. If this was a single read request, then we put the result in our
1854+date that was requested. If this was a single read request, then we put
1776−simple cache `haw` to make future requests faster. Then we're done.
1855+the result in our simple cache `haw` to make future requests faster.
1777−
1856+Then we're done.
1857+
1778−If we received actual data, then we put it into our cache `haw`. Unless it's a
1858+If we received actual data, then we put it into our cache `haw`. Unless
1779−`%w` request, we're done.
1859+it's a `%w` request, we're done.
1780−
1860+
1781−If it is a `%w` request, then we try to get the `%w` at our current head from
1861+If it is a `%w` request, then we try to get the `%w` at our current head
1782−the cache. In general, that should be the thing we just put in a moment ago, but
1862+from the cache. In general, that should be the thing we just put in a
1783−that is not guaranteed. The most common case where this is different is when we
1863+moment ago, but that is not guaranteed. The most common case where this
1784−receive desk updates out of order. At any rate, since we now have new
1864+is different is when we receive desk updates out of order. At any rate,
1785−information, we need to apply it to our local copy of the desk. We do so in
1865+since we now have new information, we need to apply it to our local copy
1786−`++edis`, and then we remove the stuff we just applied from the cache, since
1866+of the desk. We do so in `++edis`, and then we remove the stuff we just
1867+applied from the cache, since it's not really a true "single read", like
1787−it's not really a true "single read", like what should really be in the cache.
1868+what should really be in the cache.
1788
1869
1789 ++ edis :: apply subscription
1870 ++ edis :: apply subscription
1790 |= nak=nako
1871 |= nak=nako
@@ -1803,21 +1884,21 @@ it's not really a true "single read", like what should really be in the cache.
1803 %+ ~(put by yeb) r.sar sar
1884 %+ ~(put by yeb) r.sar sar
1804 ==
1885 ==
1805
1886
1806−This shows, of course, exactly why nako is defined the way it is. To become
1887+This shows, of course, exactly why nako is defined the way it is. To
1807−completely up to date with the foreign desk, we need to merge `hit` with the
1888+become completely up to date with the foreign desk, we need to merge
1808−foreign one so that we have all the revision numbers. We update `let` so that we
1889+`hit` with the foreign one so that we have all the revision numbers. We
1809−know which revision is the head.
1890+update `let` so that we know which revision is the head.
1810
1891
1811−We merge the new blobs in `lat`, keying them by their hash, which we get from a
1892+We merge the new blobs in `lat`, keying them by their hash, which we get
1812−call to `++blob-to-lobe`. Recall that the hash is stored in the actual blob
1893+from a call to `++blob-to-lobe`. Recall that the hash is stored in the
1813−itself. We do the same thing to the new yakis, putting them in `hut`, keyed by
1894+actual blob itself. We do the same thing to the new yakis, putting them
1814−their hash.
1895+in `hut`, keyed by their hash.
1815
1896
1816Now, our local dome should be exactly the same as the foreign one.
1897Now, our local dome should be exactly the same as the foreign one.
1817
1898
1818−This concludes our discussion of `++knit`. Now the changes have been applied to
1899+This concludes our discussion of `++knit`. Now the changes have been
1819−our local copy of the desk, and we just need to update our subscribers. We do so
1900+applied to our local copy of the desk, and we just need to update our
1820−in `++wake:de`.
1901+subscribers. We do so in `++wake:de`.
1821
1902
1822 ++ wake :: update subscribers
1903 ++ wake :: update subscribers
1823 ^+ .
1904 ^+ .
@@ -1868,24 +1949,27 @@ in `++wake:de`.
1868 ==
1949 ==
1869 --
1950 --
1870
1951
1871−This is even longer than `++knit`, but it's pretty similar to `++eave`. We loop
1952+This is even longer than `++knit`, but it's pretty similar to `++eave`.
1872−through each of our subscribers `xiq`, processing each in turn. When we're done,
1953+We loop through each of our subscribers `xiq`, processing each in turn.
1873−we just put the remaining subscribers back in our subscriber list.
1954+When we're done, we just put the remaining subscribers back in our
1874−
1955+subscriber list.
1956+
1875−If the subscriber is a single read, then, if this is a foreign desk (note that
1957+If the subscriber is a single read, then, if this is a foreign desk
1876−`++wake` is called from other arms, and not only on foreign desks). Obviously,
1958+(note that `++wake` is called from other arms, and not only on foreign
1877−if we find an identical request there, then we can produce the result
1959+desks). Obviously, if we find an identical request there, then we can
1878−immediately. Referential transparency for the win. We produce the result with a
1960+produce the result immediately. Referential transparency for the win. We
1879−call to `++blab`. If this is a foreign desk but the result is not in the cache,
1961+produce the result with a call to `++blab`. If this is a foreign desk
1962+but the result is not in the cache, then we produce `++blub` (canceled
1880−then we produce `++blub` (canceled subscription with no data) for reasons
1963+subscription with no data) for reasons entirely opaque to me. Seriously,
1881−entirely opaque to me. Seriously, it seems like we should wait until we get an
1964+it seems like we should wait until we get an actual response to the
1882−actual response to the request. If someone figures out why this is, let me know.
1965+request. If someone figures out why this is, let me know. At any rate,
1883−At any rate, it seems to work.
1966+it seems to work.
1884−
1967+
1885−If this is a domestic desk, then we check to see if the case exists yet. If it
1968+If this is a domestic desk, then we check to see if the case exists yet.
1886−doesn't, then we simply move on to the next subscriber, consing this one onto
1969+If it doesn't, then we simply move on to the next subscriber, consing
1887−`xaq` so that we can check again the next time around. If it does exist, then we
1970+this one onto `xaq` so that we can check again the next time around. If
1888−call `++balk` to fulfill the request and produce it.
1971+it does exist, then we call `++balk` to fulfill the request and produce
1972+it.
1889
1973
1890`++balk` is very simple, so we'll describe it here before we get to the
1974`++balk` is very simple, so we'll describe it here before we get to the
1891subscription case.
1975subscription case.
@@ -1896,30 +1980,33 @@ subscription case.
1896 =+ vid=(~(read-at-aeon ze lim dom ran) oan mun)
1980 =+ vid=(~(read-at-aeon ze lim dom ran) oan mun)
1897 ?~ vid (blub hen) (blab hen mun u.vid)
1981 ?~ vid (blub hen) (blab hen mun u.vid)
1898
1982
1899−We call `++read-at-aeon` on the given request and aeon. If you recall, this
1983+We call `++read-at-aeon` on the given request and aeon. If you recall,
1900−processes a `mood` at a particular aeon and produces the result, if there is
1984+this processes a `mood` at a particular aeon and produces the result, if
1901−one. If there is data at the requested location, then we produce it with
1985+there is one. If there is data at the requested location, then we
1902−`++blab`. Else, we call `++blub` to notify the subscriber that no data can ever
1986+produce it with `++blab`. Else, we call `++blub` to notify the
1987+subscriber that no data can ever come over this subscriptioin since it
1903−come over this subscriptioin since it is now impossible for there to ever be
1988+is now impossible for there to ever be data for the given request.
1904−data for the given request. Because referential transparency.
1989+Because referential transparency.
1905−
1990+
1906−At any rate, back to `++wake`. If the given rave is a subscription request, then
1991+At any rate, back to `++wake`. If the given rave is a subscription
1907−we proceed similarly to how we do in `++eave`. We first try to get the aeon
1992+request, then we proceed similarly to how we do in `++eave`. We first
1908−referred to by the starting case. If it doesn't exist yet, then we can't do
1993+try to get the aeon referred to by the starting case. If it doesn't
1909−anything interesting with this subscription, so we move on to the next one.
1994+exist yet, then we can't do anything interesting with this subscription,
1910−
1995+so we move on to the next one.
1996+
1911−Otherwise, we try to get the aeon referred to by the ending case. If it doesn't
1997+Otherwise, we try to get the aeon referred to by the ending case. If it
1912−exist yet, then we produce all the information we can. We call `++lobes-at-path`
1998+doesn't exist yet, then we produce all the information we can. We call
1913−at the given aeon and path to see if the requested path has actually changed. If
1999+`++lobes-at-path` at the given aeon and path to see if the requested
1914−it hasn't, then we don't produce anything; else, we produce the correct nako by
2000+path has actually changed. If it hasn't, then we don't produce anything;
2001+else, we produce the correct nako by calling `++bleb` on the result of
1915−calling `++bleb` on the result of `++make-nako`, as in `++eave`. At any rate, we
2002+`++make-nako`, as in `++eave`. At any rate, we move on to the next
1916−move on to the next subscription, putting back into our cult the current
2003+subscription, putting back into our cult the current subscription with a
1917−subscription with a new start case of the next aeon after the present.
2004+new start case of the next aeon after the present.
1918
2005
1919If the aeon referred to by the ending case does exist, then we drop this
2006If the aeon referred to by the ending case does exist, then we drop this
1920−subscriber from our cult and satisfy its request immediately. This is the same
2007+subscriber from our cult and satisfy its request immediately. This is
1921−as before -- we check to see if the data at the path has actually changed,
2008+the same as before -- we check to see if the data at the path has
1922−producing it if it has; else, we call `++blub` since no more data can be
2009+actually changed, producing it if it has; else, we call `++blub` since
1923−produced over this subscription.
2010+no more data can be produced over this subscription.
1924
2011
1925This concludes our discussion of foreign requests.
2012This concludes our discussion of foreign requests.
docs/arvo/internals/dill.md+18−16
@@ -1,24 +1,26 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 4
3sort: 4
4---
4---
5
5
6−# `%dill`
6+`%dill`
7+=======
7
8
8Our terminal driver.
9Our terminal driver.
9
10
10−Unix sends keyboard events to `%dill` from either the console or telnet, and
11+Unix sends keyboard events to `%dill` from either the console or telnet,
11−`%dill` produces terminal output. The only app that should directly talk to
12+and `%dill` produces terminal output. The only app that should directly
12−`%dill` is the terminal app. Command-line apps are run by, receive input from,
13+talk to `%dill` is the terminal app. Command-line apps are run by,
13−and produce output to, the `%shell` app, which is controlled by `%terminal`,
14+receive input from, and produce output to, the `%shell` app, which is
14−which talks to `%dill`, which talks to unix. Clay also uses `%dill` directly to
15+controlled by `%terminal`, which talks to `%dill`, which talks to unix.
16+Clay also uses `%dill` directly to print out the filesystem change
15−print out the filesystem change events, but this is questionable behavior.
17+events, but this is questionable behavior.
16
18
17−`%dill` has two main components. First, it controls the terminal on a very
19+`%dill` has two main components. First, it controls the terminal on a
18−basic, event-by-event level. This includes keeping track of things like the
20+very basic, event-by-event level. This includes keeping track of things
19−dimensions of the terminal, the prompt type (plain text or password), which duct
21+like the dimensions of the terminal, the prompt type (plain text or
20−to produce effects on, and so forth. Second, it handles terminal events,
22+password), which duct to produce effects on, and so forth. Second, it
21−keystroke by keystroke. Most characters are simply pushed onto the buffer and
23+handles terminal events, keystroke by keystroke. Most characters are
24+simply pushed onto the buffer and blitted to the screen, but some
22−blitted to the screen, but some characters, including control-modified keys,
25+characters, including control-modified keys, arrow keys, etc. require
23−arrow keys, etc. require special handling. Most of the readline functionality is
26+special handling. Most of the readline functionality is in `%dill`.
24−in `%dill`.
docs/arvo/internals/eyre.md+13−14
@@ -1,23 +1,22 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 5
3sort: 5
4---
4---
5
5
6−# `%eyre`
6+`%eyre`
7+=======
7
8
8Our http server.
9Our http server.
9
10
10−Unix sends http messages to `%eyre`, and `%eyre` produces http messages in
11+Unix sends http messages to `%eyre`, and `%eyre` produces http messages
11−response. In general, apps and vanes do not call `%eyre`; rather, `%eyre` calls
12+in response. In general, apps and vanes do not call `%eyre`; rather,
12−apps and vanes. `%eyre` uses `%ford` and `%gall` to functionally publish pages
13+`%eyre` calls apps and vanes. `%eyre` uses `%ford` and `%gall` to
13−and facilitate communication with apps.
14+functionally publish pages and facilitate communication with apps.
14
15
15−`%eyre` primarily parses web requests and handles them in a variety of ways,
16+`%eyre` primarily parses web requests and handles them in a variety of
16−depending on the control string. Nearly all of these are essentially stateless,
17+ways, depending on the control string. Nearly all of these are
17−like functional publishing with `%ford`. Additionally, there's a fairly
18+essentially stateless, like functional publishing with `%ford`.
19+Additionally, there's a fairly significant component that handles
18−significant component that handles `%gall` messaging and subscriptions, which
20+`%gall` messaging and subscriptions, which must be stateful.
19−must be stateful.
20
21
21−<div>
22−<list/>
22+<div><list/></div>
23−</div>
docs/arvo/internals/eyre/commentary.md+45−50
@@ -1,42 +1,43 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3---
3---
4
4
5−# Eyre: Reference
5+Eyre: Reference
6+===============
6
7
7## State
8## State
8
9
9Stored
10Stored
10
11
11−- `++bolo`
12+- `++bolo`
12−- `++cyst`
13+- `++cyst`
13−- `++stem`
14+- `++stem`
14
15
15Runtime
16Runtime
16
17
17−- `perk`
18+- `perk`
18−- `perk-auth`
19+ + `perk-auth`
19−- `pest`
20+- `pest`
20−- `even`
21+- `even`
21
22
22## Cores
23## Cores
23
24
24−- `++handle`
25+- `++handle`
25−- `++parse`
26+ + `++parse`
26− - `++as...`
27+ - `++as...`
27−- `++ya`
28+- `++ya`
28−- `++ix`
29+- `++ix`
29
30
30−# Eyre: Commentary
31+Eyre: Commentary
32+================
31
33
32−Let us follow the loading of a simple cli app, as it bounces from browser to
34+Let us follow the loading of a simple cli app, as it bounces from
33−server to browser and back.
35+browser to server to browser and back.
34
36
35## Initial request<a id="init"/>
37## Initial request<a id="init"/>
36
38
37−An http request for `http://sampel-sipnym.urbit.org/cli` will be
39+An http request for `http://sampel-sipnym.urbit.org/cli` will be [redirected](dns)
38−[redirected](dns) to the `%eyre` on ~sampel-sipnym, and come in as a `%this`
40+to the `%eyre` on ~sampel-sipnym, and come in as a `%this` kiss.
39−kiss.
40
41
41From arvo, requests enter `++call`, which after some type reification are passed
42From arvo, requests enter `++call`, which after some type reification are passed
42along to `++apex:ye`. In the case of a `%this` kiss, its components are
43along to `++apex:ye`. In the case of a `%this` kiss, its components are
@@ -45,8 +46,7 @@ in `++emule` to produce a `++fail` page in case of error. `++apex:handle` will
45`++process` the request to a `pest` or a `++done` core, and in the former case
46`++process` the request to a `pest` or a `++done` core, and in the former case
46`++resolve` the pest into an outgoing card.
47`++resolve` the pest into an outgoing card.
47
48
48−XX it also seems to affect the current ship, test that serving ship name is
49+XX it also seems to affect the current ship, test that serving ship name is consistently correct
49−consistently correct
50
50
51The pest is produced by `++process`, which will first further `++parse` the
51The pest is produced by `++process`, which will first further `++parse` the
52request, and if this does not make the response immediately obvious,
52request, and if this does not make the response immediately obvious,
@@ -68,8 +68,8 @@ the perk it into a `%boil` `++ford-req`, adding an `++fcgi` path-segment
68containing query string and `++fcgi-cred:for-client` auth information.
68containing query string and `++fcgi-cred:for-client` auth information.
69
69
70`%ford`s translation of `/=cli=/hymn/hook` to a self-refreshing `%urb` html page
70`%ford`s translation of `/=cli=/hymn/hook` to a self-refreshing `%urb` html page
71−[deserves its own commentary](../ford/commentary), but we resume in `%eyre` when
71+[deserves its own commentary](../ford/commentary), but we resume in `%eyre`
72−the `%made` sign arrives in `++take`, and soon after `++axon:ye`. There the
72+when the `%made` sign arrives in `++take`, and soon after `++axon:ye`. There the
73`wire`, or rather the `whir` it has been verified to be, determines that the
73`wire`, or rather the `whir` it has been verified to be, determines that the
74response should be served immediately. However, as the mark is not `%mime`,
74response should be served immediately. However, as the mark is not `%mime`,
75another trip to `%ford` is required to encode it, on the same wire; afterwards,
75another trip to `%ford` is required to encode it, on the same wire; afterwards,
@@ -145,7 +145,7 @@ To understand `/~/at`, there will first be a brief diversion to `~/auth.json`.
145`auth.json`, perk `[%auth %json]`, in `++process-auth` serves `++stat-json:ya`,
145`auth.json`, perk `[%auth %json]`, in `++process-auth` serves `++stat-json:ya`,
146containing such information as the serving ship, which identities are associated
146containing such information as the serving ship, which identities are associated
147with this session, and `oryx`, a CSRF token. An oryx must be present on all
147with this session, and `oryx`, a CSRF token. An oryx must be present on all
148−stateful requests, in this case executing a log in. It also saves the new/old
148+stateful requests, in this case executing a log in. It also saves the new/old
149session using `abet`.
149session using `abet`.
150
150
151XX explain `ixor` here and not [later](#ixor)?
151XX explain `ixor` here and not [later](#ixor)?
@@ -153,22 +153,21 @@ XX explain `ixor` here and not [later](#ixor)?
153`/~/at` is an alternate interface, which injects `auth.json` data into the
153`/~/at` is an alternate interface, which injects `auth.json` data into the
154requested file. `/~/at/auth.js`, then, is a request for the built-in `auth:js`
154requested file. `/~/at/auth.js`, then, is a request for the built-in `auth:js`
155(parsed to and processed from an `[%auth %js ~]` perk), with session data added
155(parsed to and processed from an `[%auth %js ~]` perk), with session data added
156−as `window.urb`. And indeed, `` [`%js /~/at/auth] `` is parsed to
156+as `window.urb`. And indeed, ``[`%js /~/at/auth]`` is parsed to
157−`` [%auth at [`%js /auth] ``, which in `++process-auth` is re-`process`ed to
157+``[%auth at [`%js /auth]``, which in `++process-auth` is re-`process`ed to
158−`[%js ++auth:js]`, which is `++resolve`d after an `++add-json` of the relevant
158+`[%js ++auth:js]`, which is `++resolve`d after an `++add-json` of
159−data. The yac cookies are also passed to `resolve`, which `++add-cookies`
159+the relevant data. The yac cookies are also passed to `resolve`, which
160−injects into the `httr`.
160+`++add-cookies` injects into the `httr`.
161
161
162−It is at this point that there is first occasion for user input, namely the
162+It is at this point that there is first occasion for user input, namely the password.
163−password.
164
163
165The `auth:js` script sends a `PUT` request, also to `/~/auth.json`. In `parse`,
164The `auth:js` script sends a `PUT` request, also to `/~/auth.json`. In `parse`,
166the first nontrivial `++check-oryx` occurs, `++grab-body` the request oryx and
165the first nontrivial `++check-oryx` occurs, `++grab-body` the request oryx and
167ensuring it is recorded for the session. The request parsed with `++need-body`
166ensuring it is recorded for the session. The request parsed with `++need-body`
168to a `[%auth %try {password}]` perk. `%get:process-auth` checks it against
167to a `[%auth %try {password}]` perk. `%get:process-auth` checks it against
169−`++load-secret`, upon success updates the session with `++logon:ya`, and serves
168+`++load-secret`, upon success updates the session with `++logon:ya`, and
170−a fresh `auth.json` which reflects the changed `user`. Upon receiving this, the
169+serves a fresh `auth.json` which reflects the changed `user`. Upon receiving
171−page is refreshed to retry the original request.
170+this, the page is refreshed to retry the original request.
172
171
173## Post-authentication: app communication. <a id="auth-ok"/>
172## Post-authentication: app communication. <a id="auth-ok"/>
174
173
@@ -183,7 +182,7 @@ know to serve the window.urb necessary to make requests, and the `urb.js`
183standard library which extends it with a number of wrappers to them and other
182standard library which extends it with a number of wrappers to them and other
184useful functions.
183useful functions.
185
184
186−--------------------------------------------------------------------------------
185+---
187
186
188One of those functions is `urb.bind`, which is used to subscribe to application
187One of those functions is `urb.bind`, which is used to subscribe to application
189data. Userspace javascript sets `urb.appl` to `/tic`, and binds `lines` to a
188data. Userspace javascript sets `urb.appl` to `/tic`, and binds `lines` to a
@@ -213,19 +212,18 @@ is started using `urb.poll`. At `/~/of/{ixor}`, perk
213`[%view ixor ~ {sequence-number}]`, it is `process`ed by `++poll:ix` (the cyst
212`[%view ixor ~ {sequence-number}]`, it is `process`ed by `++poll:ix` (the cyst
214is retrieved by `++ire-ix` form global state, using the perk `ixor`): the
213is retrieved by `++ire-ix` form global state, using the perk `ixor`): the
215sequence number is in the past, so the previously received `%rush` is
214sequence number is in the past, so the previously received `%rush` is
216−`++give-even`. After deleting the previous message in the queue and invoking
215+`++give-even`. After deleting the previous message in the queue and invoking
217`++pass-took` to signal `%gall` of this occurrence, the data is annotated with
216`++pass-took` to signal `%gall` of this occurrence, the data is annotated with
218the source app+path `++subs-to-json`, and returned to the polling duct.
217the source app+path `++subs-to-json`, and returned to the polling duct.
219
218
220On the client, the user callback receives the `/cli` history, and displays it on
219On the client, the user callback receives the `/cli` history, and displays it on
221the page. The `/~/of` long poll is continued, this time reaching `++poll:ix`
220the page. The `/~/of` long poll is continued, this time reaching `++poll:ix`
222−with the "pending" sequence number, and being stored in the `cyst` for its
221+with the "pending" sequence number, and being stored in the `cyst` for its troubles.
223−troubles.
224
222
225−--------------------------------------------------------------------------------
223+---
226
224
227Its next update proceeds idenitcally, but first it must be triggered, which
225Its next update proceeds idenitcally, but first it must be triggered, which
228−happens when the user enters "(add 2 2)\\n", firing an `urb.send` from the event
226+happens when the user enters "(add 2 2)\n", firing an `urb.send` from the event
229handler. This sends a `POST` request to `/~/to/cli/json.json`, perk `%mess`,
227handler. This sends a `POST` request to `/~/to/cli/json.json`, perk `%mess`,
230`++process`ed to a `%g %mess`. Were the mark not `%json`, a `%ford` conversion
228`++process`ed to a `%g %mess`. Were the mark not `%json`, a `%ford` conversion
231would occur first, and `%made:axon` would send the gall message proper. In
229would occur first, and `%made:axon` would send the gall message proper. In
@@ -235,7 +233,7 @@ sent to the client callback.
235## A path not taken: magic filenames <a id="mage"/>
233## A path not taken: magic filenames <a id="mage"/>
236
234
237The `/robots.txt` and `/favicon.(ico|png)` files are static, and served
235The `/robots.txt` and `/favicon.(ico|png)` files are static, and served
238−immediately when caught by a `++parse`.
236+immediately when caught by a `++parse`.
239
237
240XX index.html?
238XX index.html?
241
239
@@ -252,6 +250,7 @@ normal login page but later sends the client back. XX expand, basically the
252status quo is you're logged in and `/~/as/foo` is ignored, just setting your
250status quo is you're logged in and `/~/as/foo` is ignored, just setting your
253`urb.user` XX
251`urb.user` XX
254
252
253+
255## A path not taken: deauthentication
254## A path not taken: deauthentication
256
255
257`/~/away`, perk `[%away ~]`, produces a static `++logout-page:xml`, which also
256`/~/away`, perk `[%away ~]`, produces a static `++logout-page:xml`, which also
@@ -265,12 +264,8 @@ from memory.
265`++del-subs:ix` acting as reverse of `++add-subs` by deleting the duct binding
264`++del-subs:ix` acting as reverse of `++add-subs` by deleting the duct binding
266and sending `%g %nuke`.
265and sending `%g %nuke`.
267
266
268−XX unmentioned arms: abet, add-poll, adit, ames-gram, anon, ares-to-json, bolo,
267+
269−cyst, doze, even, ford-kill, get-mean, gift, give-json, give-thou, gram, hapt,
268+XX unmentioned arms: abet, add-poll, adit, ames-gram, anon, ares-to-json, bolo, cyst, doze, even, ford-kill, get-mean, gift, give-json, give-thou, gram, hapt, hasp, host-to-ship, ix, ixor, js, kiss, load, mean-json, move, note, pass-note, perk, perk-auth, pest, poke-test, print-subs, render-tang, resp, root-beak, scry, ses-authed, ses-ya, sign, silk, sine, stay, stem, teba, titl, to-oryx, urb, wait-era, wake, whir, wush, xml, ya, ye
270−hasp, host-to-ship, ix, ixor, js, kiss, load, mean-json, move, note, pass-note,
271−perk, perk-auth, pest, poke-test, print-subs, render-tang, resp, root-beak,
272−scry, ses-authed, ses-ya, sign, silk, sine, stay, stem, teba, titl, to-oryx,
273−urb, wait-era, wake, whir, wush, xml, ya, ye
274
269
275## Appendix A: DNS <a id="dns"/>
270## Appendix A: DNS <a id="dns"/>
276
271
@@ -278,5 +273,5 @@ The `*.urbit.org` domain can be used to access destroyers and cruisers. In the
278common case oh hosted ships, this is done by dynamic DNS directly to the hosting
273common case oh hosted ships, this is done by dynamic DNS directly to the hosting
279instance. We do not speak of the uncommon case. When ports are blocked and
274instance. We do not speak of the uncommon case. When ports are blocked and
280infrastructure crumbles around you, only imported martian networking can be
275infrastructure crumbles around you, only imported martian networking can be
281−trusted: the `%get` and `%got` [gram]()s are used to proxy [`%this` requests]()
276+trusted: the `%get` and `%got` [gram]()s are used to proxy [`%this` requests]() and
282−and [`%thou` responses]() respectively.
277+[`%thou` responses]() respectively.
docs/arvo/internals/eyre/specification.md+189−355
@@ -1,186 +1,135 @@
1# Specification
1# Specification
2
2
3−The `%eyre` vane serves one purpose: receiving inbound HTTP requests from
3+The `%eyre` vane serves one purpose: receiving inbound HTTP requests from browsers and clients pretending to be browsers. To facilitate this communication, `%eyre` does two things: 1. provide an API for traditional clients to talk to; and 2. provide a mechanism for saving state that allows the server to keep track of identities/sessions, subscription data, and active connections.
4−browsers and clients pretending to be browsers. To facilitate this
5−communication, `%eyre` does two things: 1. provide an API for traditional
6−clients to talk to; and 2. provide a mechanism for saving state that allows the
7−server to keep track of identities/sessions, subscription data, and active
8−connections.
9
4
10This document outlines the following:
5This document outlines the following:
11
6
12−1. [`perk`](#perk), a `pork` subset that defines what URLs the client can send
7+1. [`perk`](#perk), a `pork` subset that defines what URLs the client can send to `%eyre`.
13− to `%eyre`.
14−2. [the values stored on the client](#client), both statically in cookies and
15− dynamically as the `window.urb` object, which facilitate this transaction.
8+2. [the values stored on the client](#client), both statically in cookies and dynamically as the `window.urb` object, which facilitate this transaction.
16
9
17−3. [`kiss`](#kiss), the requests `%eyre` will accept.
10+3. [`kiss`](#kiss), the requests `%eyre` will accept.
18−4. [`gift`](#gift), the responses it will give to those requests.
11+4. [`gift`](#gift), the responses it will give to those requests.
19
12
20−5. [`note`](#note), the set of arvo requests that %eyre makes to the other
13+5. [`note`](#note), the set of arvo requests that %eyre makes to the other vanes.
21− vanes.
22−6. [`whir`](#whir), a `wire` subset, the associated dynamic state
14+6. [`whir`](#whir), a `wire` subset, the associated dynamic state
23−7. [`sign`](#sign), the responses `%eyre` accepts
15+7. [`sign`](#sign), the responses `%eyre` accepts
24−8. [`bolo`](#bolo), the state stored between separate events.
16+8. [`bolo`](#bolo), the state stored between separate events.
25
17
26# 1. URL structure <a id="perk"/>
18# 1. URL structure <a id="perk"/>
27−
28−There exist several URL forms accepted by `%eyre`.
19+There exist several URL forms accepted by `%eyre`.
29
20
30## 1.1 Functional request
21## 1.1 Functional request
31−
32The simplest of them is a static `%f` functional publish request:
22The simplest of them is a static `%f` functional publish request:
33
23
24+ ```hoon
34−`hoon GET /[ship]/[desk]/[case]/path/to/file.[mark]` e.g.
25+ GET /[ship]/[desk]/[case]/path/to/file.[mark]
26+ ```
27+e.g.
35
28
29+ ```hoon
36−`hoon /~pittyp-pittyp/main/5/lib/urb.js` A `case` of 0 signifies the most
30+ /~pittyp-pittyp/main/5/lib/urb.js
31+ ```
37−recent version.
32+A `case` of 0 signifies the most recent version.
38
33
39−If no file extension is provided, the default `mark` is `%urb`, which consists
34+If no file extension is provided, the default `mark` is `%urb`, which consists of `%html` injected with an auto-refresh script.
40−of `%html` injected with an auto-refresh script.
41
35
42A shorter form:
36A shorter form:
43
37
38+ ```hoon
39+ /path/to.[mark]
40+ ```
44−`hoon /path/to.[mark]` is prepended with the [default request path](#prefix).
41+is prepended with the [default request path](#prefix). For example, if `%eyre`
45−For example, if `%eyre` is configured with a root of `/<our>/main/0/pub`, then
42+is configured with a root of `/<our>/main/0/pub`, then `/doc/arvo` is
46−`/doc/arvo` is interpreted as a request for `/=main=/pub/doc/arvo.urb`.
43+interpreted as a request for `/=main=/pub/doc/arvo.urb`.
47
44
48Auxiliary paths start with `/~/`, and provide other functionality, such as:
45Auxiliary paths start with `/~/`, and provide other functionality, such as:
49
46
50## 1.2 Auto-reload
47## 1.2 Auto-reload
51−
52−When a new version of a page becomes available, it is useful to propagate it to
48+When a new version of a page becomes available, it is useful to propagate it to the client. The `%f` vane manages the particulars, and provides an update token along with all published pages. The `/~/on` long-polling endpoint makes use of that token, which is a hash of the server-stored dependency set.
53−the client. The `%f` vane manages the particulars, and provides an update token
54−along with all published pages. The `/~/on` long-polling endpoint makes use of
55−that token, which is a hash of the server-stored dependency set. -
56−`/~/on/[hash].json` retuns `true` when a page may require updating. -
49+- `/~/on/[hash].json` retuns `true` when a page may require updating.
57−`/~/on/[hash].js` is a lightweight script that polls for the above. It is
50+- `/~/on/[hash].js` is a lightweight script that polls for the above. It is primarily used in error messages.
58−primarily used in error messages.
59
51
60## 1.3 Authentication <a id="auth"/>
52## 1.3 Authentication <a id="auth"/>
61−
62−Authenticated requests are accomplished through sessions, tracked with cookies
53+Authenticated requests are accomplished through sessions, tracked with cookies which are set on the first such request.
63−which are set on the first such request. - `/~/as/[user]/...` is an
64−authenticated request to `...`, where `user` can be a `@p` ship, `'own'`,
54+- `/~/as/[user]/...` is an authenticated request to `...`, where `user` can be a `@p` ship, `'own'`, `'any'`, or `'anon'`.
65−`'any'`, or `'anon'`. + `'anon'` is a randomly-generated per-session submarine,
55+ + `'anon'` is a randomly-generated per-session submarine, and requires no further authentication.
66−and requires no further authentication. + `'own'` represents the serving ship.
56+ + `'own'` represents the serving ship. The short form of `/~/as/own/` is `/~~/`
67−The short form of `/~/as/own/` is `/~~/` + `'any'` requires a network ship,
68−chosen with a prompt if unknown. The short form of `/~/as/any/` is `/_/` + Auth
57+ + `'any'` requires a network ship, chosen with a prompt if unknown. The short form of `/~/as/any/` is `/_/`
69−requests are handled in three ways: + I. If the session is associated with the
58+ + Auth requests are handled in three ways:
70−`ship`, serve `...`, storing the ship as the session's default ship. For
59+ + I. If the session is associated with the `ship`, serve `...`, storing the ship as the session's default ship.
60+ For example,
71−example, /~/as/sivtyv-barnel/main/pub/test.html is an authenticated request of
61+ /~/as/sivtyv-barnel/main/pub/test.html
62+ is an authenticated request of
63+ /main/pub/test.html
72−/main/pub/test.html with the identity of `~sivtyv-barnel`. + II. If the session
64+ with the identity of `~sivtyv-barnel`.
73−is not associated with the `ship` in the path, the client is redirected to a
74−login page, where a password can be entered. Successful authentication results
65+ + II. If the session is not associated with the `ship` in the path, the client is redirected to a login page, where a password can be entered. Successful authentication results in a page refresh to the requested path.
75−in a page refresh to the requested path. + III. otherwise, [send a
76−message](#foreign) to the claimed ship, and redirect the user to
66+ + III. otherwise, [send a message](#foreign) to the claimed ship, and redirect the user to `https://[foreign-host]/~/am/[ses]/...#[current-host]&?code=[code]`
77−`https://[foreign-host]/~/am/[ses]/...#[current-host]&?code=[code]`
67+
78−
68+
79−- `/~/am/[ses]/...#original-host` similarly serves a login page. Upon success,
69+- `/~/am/[ses]/...#original-host` similarly serves a login page. Upon success, inform the `ship` of the success and redirect it back to `http://[original-host]/~/as/<this-ship>/...`
80− inform the `ship` of the success and redirect it back to
81− `http://[original-host]/~/as/<this-ship>/...`
82−- `/~/away.html` is a log-out page.
70+- `/~/away.html` is a log-out page.
83−- `/~/auth.json` returns `{ship,oryx,user,auth}`: the serving ship, a CSRF
71+- `/~/auth.json` returns `{ship,oryx,user,auth}`: the serving ship, a CSRF token, active user, and all allowed users respectively.
84− token, active user, and all allowed users respectively.
85−- `/~/at/{...}.js` is an alternate form of `/~/auth.json`, which injects a
86− line into the served js file that sets `{ship, oryx, user, auth}` on the
72+- `/~/at/{...}.js` is an alternate form of `/~/auth.json`, which injects a line into the served js file that sets `{ship, oryx, user, auth}` on the `window.urb` object.
87− `window.urb` object.
73+
88−
89−So far, all the paths specified have been GET requests. Authentication, however,
74+So far, all the paths specified have been GET requests. Authentication, however, requires sending data. This is also done at the `/~/auth.json` endpoint.
90−requires sending data. This is also done at the `/~/auth.json` endpoint. -
91−`POST {oryx,ship,code} /~/auth.json?PUT` is used by the login script to
92−authorize a user, and returns the same result as `GET`, additionally containing
75+- `POST {oryx,ship,code} /~/auth.json?PUT` is used by the login script to authorize a user, and returns the same result as `GET`, additionally containing either`{ok:true}` or `{fail:'type',mess:"Error message"}`
93−either`{ok:true}` or `{fail:'type',mess:"Error message"}` -
94−`POST {oryx,ship?} /~/auth.json?DELETE` revokes authorization. Response as
76+- `POST {oryx,ship?} /~/auth.json?DELETE` revokes authorization. Response as above.
95−above.
96
77
97## 1.4 Messaging
78## 1.4 Messaging
98−
99−- `POST {oryx, wire, xyro} /~/to/[app]/[mark].json`, where `xyro` is data that
100− will be converted to the `mark`. In the simplest case,
101− `/~/to/hello/json.json` will pass `xyro` through verbatim, sending
79+- `POST {oryx, wire, xyro} /~/to/[app]/[mark].json`, where `xyro` is data that will be converted to the `mark`. In the simplest case, `/~/to/hello/json.json` will pass `xyro` through verbatim, sending `[%json xyro]` to the `++poke` arm of app `%hello`.
102− `[%json xyro]` to the `++poke` arm of app `%hello`.
103−- `POST {oryx, wire, xyro} /~/to/<ship>/[app]/[mark].json` is a foreign
80+- `POST {oryx, wire, xyro} /~/to/<ship>/[app]/[mark].json` is a foreign message send, as above.
104− message send, as above.
105
81
106## 1.5 Subscriptions <a id="subs"/>
82## 1.5 Subscriptions <a id="subs"/>
83+- `/~/of/[ixor]`<a id="of-ixor"/> is an `EventSource`: a conceptually infinite file containing a stream of events. By default, it sends a newline every 30 seconds serving to signal that the connection is alive to both the server and IP middleware.
84+ + `/~/of/[ixor]?poll={n}` is a long-polling fallback interface. Produces event number {n}, blocking until it occurs.
107
85
108−- `/~/of/[ixor]`<a id="of-ixor"/> is an `EventSource`: a conceptually infinite
109− file containing a stream of events. By default, it sends a newline every 30
110− seconds serving to signal that the connection is alive to both the server
111− and IP middleware.
112−- `/~/of/[ixor]?poll={n}` is a long-polling fallback interface. Produces event
113− number {n}, blocking until it occurs.
114−
115−Its contexts can be affected by `POST` requests with a body of `{oryx, wire}`,
86+Its contexts can be affected by `POST` requests with a body of `{oryx, wire}`, and a query string of `PUT` or `DELETE`. They return either `{mark}`(which may be `null`), or [`{fail, mess}`](#mean-json)
116−and a query string of `PUT` or `DELETE`. They return either `{mark}`(which may
117−be `null`), or [`{fail, mess}`](#mean-json)
118
87
119−- `/~/in/[hash].json` is a dual to [`/~/on`](#on-change). It binds `mod`
88+- `/~/in/[hash].json` is a dual to [`/~/on`](#on-change). It binds `mod` events, which echo the requesting token.
120− events, which echo the requesting token.
121−- `/~/is/[app]/path/to.[mark]` binds `rush` events, formatted with a first
122− line of `[app] /path/to`, and the rest containing subscription data.
123− Additionally, `mean` events formatted similarly may arrive, also converted
89+- `/~/is/[app]/path/to.[mark]` binds `rush` events, formatted with a first line of `[app] /path/to`, and the rest containing subscription data. Additionally, `mean` events formatted similarly may arrive, also converted to the relevant mark.
124− to the relevant mark.
125
90
126## 1.6 Ablative <a id="temp"/>
91## 1.6 Ablative <a id="temp"/>
127−
128−These interfaces will temporarily exist to aid development, and are to be
92+These interfaces will temporarily exist to aid development, and are to be considered unstable.
129−considered unstable. - `/~/debug/...` access normally inaccessible pages. For
130−example, `/~/debug/as.html` will present the login page, regardless of current
93+- `/~/debug/...` access normally inaccessible pages. For example, `/~/debug/as.html` will present the login page, regardless of current session status.
131−session status.
132
94
133# 2. Client state <a id="client"/>
95# 2. Client state <a id="client"/>
134−
135Some information is stored on, and provided to, browser clients
96Some information is stored on, and provided to, browser clients
136
97
137## 2.1 Cookies
98## 2.1 Cookies
138−
139−Authenticated users receive a cookie on the domain of `*.urbit.org`. The cookie
99+Authenticated users receive a cookie on the domain of `*.urbit.org`. The cookie contains a client session token, keyed by the serving ship.
140−contains a client session token, keyed by the serving ship.
141
100
142## 2.2 Authentication <a id="auth-json"/>
101## 2.2 Authentication <a id="auth-json"/>
143−
144−It is common(e.g. by the `%urb` mark) to set window.urb to the contents of
102+It is common(e.g. by the `%urb` mark) to set window.urb to the contents of `/~/auth.json`:
145−`/~/auth.json`: - `ship` is the serving ship - `oryx` is a unique CSRF token
103+- `ship` is the serving ship
146−that identifies this view, bound to the session cookie - `ixor` is a hash of
104+- `oryx` is a unique CSRF token that identifies this view, bound to the session cookie
147−`oryx`, used as an insecure view identifier - `user` is the
105+- `ixor` is a hash of `oryx`, used as an insecure view identifier
148−authenticated/generated ship name - `auth` is a list of identities usable by
106+- `user` is the authenticated/generated ship name
149−this session
107+- `auth` is a list of identities usable by this session
150
108
151## 2.3 Client library
109## 2.3 Client library
152−
153−To this object, `/main/lib/urb.js` adds helpers: - `app` is an optional app name
110+To this object, `/main/lib/urb.js` adds helpers:
154−to send messages to by default -
111+- `app` is an optional app name to send messages to by default
155−`send({data,mark?='json',app?=urb.app,ship?}, cb?)` delivers messages -
112+- `send({data,mark?='json',app?=urb.app,ship?}, cb?)` delivers messages
156−`bind({path,app?=urb.app}, cb)` subscribes by path -
113+- `bind({path,app?=urb.app}, cb)` subscribes by path
157−`drop({path,app?=urb.app}, cb?)` pulls the subscription - `util` is an object
114+- `drop({path,app?=urb.app}, cb?)` pulls the subscription
158−containing methods for converting between JavaScript types and Hoon atom odors.
115+- `util` is an object containing methods for converting between JavaScript types and Hoon atom odors.
159
116
160# 3. Requests <a id="kiss"/>
117# 3. Requests <a id="kiss"/>
161
118
162## `[%born port=@ud]`, unix init
119## `[%born port=@ud]`, unix init
163−
164−When the `vere` process is re/started, a TCP port is bound, HTTP requests on
120+When the `vere` process is re/started, a TCP port is bound, HTTP requests on which are the scope of this document. The `%born` gift informs eyre of this occurrence, which in turn updates [its state.](#bolo)
165−which are the scope of this document. The `%born` gift informs eyre of this
166−occurrence, which in turn updates [its state.](#bolo)
167
121
168**Gifts given in response:** none.
122**Gifts given in response:** none.
169
123
170## `[%hiss p=mark q=cage]`, outbound http
124## `[%hiss p=mark q=cage]`, outbound http
171−
172Userspace http request, converted by `%ford` into an actual `hiss`.
125Userspace http request, converted by `%ford` into an actual `hiss`.
173
126
174**Gifts given in response:** a [`%sigh`]()
127**Gifts given in response:** a [`%sigh`]()
175
128
176## 3.1 `[%this httq]`, inbound http
129## 3.1 `[%this httq]`, inbound http
130+The primary `%eyre` request type comes from the host system, and constitutes an inbound http request. It consists of five parts:
177
131
178−The primary `%eyre` request type comes from the host system, and constitutes an
179−inbound http request. It consists of five parts:
180−
181−I. `p=[? @if]`: a loobean determining whether the request is being made over
182−https, and the client source, an IPv6 address possibly encoding an IPv4 one as
132+I. `p=[? @if]`: a loobean determining whether the request is being made over https, and the client source, an IPv6 address possibly encoding an IPv4 one as `.0.0.0.0.0.ffff.wwxx.yyzz`
183−`.0.0.0.0.0.ffff.wwxx.yyzz`
184
133
185II. `q=meth`, an http method
134II. `q=meth`, an http method
186
135
@@ -190,342 +139,225 @@ IV. `s=mess`, an associative list of header keys/values
190
139
191V. `t=(unit octs)`, the request body, if any.
140V. `t=(unit octs)`, the request body, if any.
192
141
193−After `r` is parsed to a `purl`, the `pork`(relative path) determines the
194−resource being served. The first step in handling an http request is
142+After `r` is parsed to a `purl`, the `pork`(relative path) determines the resource being served. The first step in handling an http request is [classifying its intent](#perk).
195−[classifying its intent](#perk).
196−
197−- A ford request results in a [`%boil` note](#boil).
198−- `/~/[name].js` is generated from [session](#sink) or [view](#stem) state,
199− and [served](#mime) as `%js`.
200−- `/~/am` generates a login page.
201−- `/~/as` is resolved further if authenticated, generates a similar login page
202− if the claimed ship is local, and otherwise is saved to [session
203− state](#sink) and sends a ["login" `%a` message](#lon) to the foreign ship.
204−- `/~/at` and `/~/auth.json` generate a new `oryx` and store it in the
205− [session state](#sink).
206−- `/~/in` and `/~/is` affect [view state](#stem), possibly sending
207− subscriptions or unsubscriptions to `%f` and `%g` respectively, and are
208− generated from view state.
209−- `/~/on` is saved to the `bolo` by `hash`, and translated to a [`%wasp`
210− note](#wasp).
211−- `/~/of` fails if the `ixor` is unknown, and otherwise is stored in the
212− `ixor`, receiving a partial response and all events after its [last-seen
213− header](#). If it has a `poll` parameter, the response is full, and only
214− occurs if the body is non-empty. Otherwise the oryx is saved as a
215− [live](#live) request in case of cancellation.
216−- `/~/to` transforms into a [`%g` message](#mess).
217−
218−**Gifts given in response:** a [`%thou`](), or a [`%that`](#that) followed by
219−multiple [`%thar`]()s.
220
143
221−## 3.2 `[%thud ~]`, inbound close
144+- A ford request results in a [`%boil` note](#boil).
145+- `/~/[name].js` is generated from [session](#sink) or [view](#stem) state, and [served](#mime) as `%js`.
146+- `/~/am` generates a login page.
147+- `/~/as` is resolved further if authenticated, generates a similar login page if the claimed ship is local, and otherwise is saved to [session state](#sink) and sends a ["login" `%a` message](#lon) to the foreign ship.
148+- `/~/at` and `/~/auth.json` generate a new `oryx` and store it in the [session state](#sink).
149+- `/~/in` and `/~/is` affect [view state](#stem), possibly sending subscriptions or unsubscriptions to `%f` and `%g` respectively, and are generated from view state.
150+- `/~/on` is saved to the `bolo` by `hash`, and translated to a [`%wasp` note](#wasp).
151+- `/~/of` fails if the `ixor` is unknown, and otherwise is stored in the `ixor`, receiving a partial response and all events after its [last-seen header](#). If it has a `poll` parameter, the response is full, and only occurs if the body is non-empty. Otherwise the oryx is saved as a [live](#live) request in case of cancellation.
152+- `/~/to` transforms into a [`%g` message](#mess).
222
153
223−Sent when an unresponded-to request is cancelled. The [live](#live) effect of
154+**Gifts given in response:** a [`%thou`](), or a [`%that`](#that) followed by multiple [`%thar`]()s.
224−the request is looked up, and - `%exec` ford boil requests are cancelled with
225−`[%exec ~]` - `%wasp` ford deps requests are cancelled with `[%wasp |]` -
226−`%poll` session polls `%rest` the session heartbeat timeout
227
155
228−## 3.3 `[%thin p=@ud]`, delivery failure
156+## 3.2 `[%thud ~]`, inbound close
157+Sent when an unresponded-to request is cancelled. The [live](#live) effect of the request is looked up, and
158+- `%exec` ford boil requests are cancelled with `[%exec ~]`
159+- `%wasp` ford deps requests are cancelled with `[%wasp |]`
160+- `%poll` session polls `%rest` the session heartbeat timeout
229
161
230−Sent when a partial [response](#that) or [body](#thar) fails to arrive. Its
162+## 3.3 `[%thin p=@ud]`, delivery failure
231−`duct` is the same as that of the original request. This is handled by
232−[waiting](#wait) for the client to reconnect, and if this not occur, unwinding
163+Sent when a partial [response](#that) or [body](#thar) fails to arrive. Its `duct` is the same as that of the original request. This is handled by [waiting](#wait) for the client to reconnect, and if this not occur, unwinding all subscription state.
233−all subscription state.
234
164
235**Gifts given in response:** None.
165**Gifts given in response:** None.
236
166
237−## 3.4 `[%wart sock [path *]]`, inbound message
167+## 3.4 ``[%wart sock [path *]]``, inbound message
238−
239−Sometimes, messages are received from other ships. They are expected to take the
168+Sometimes, messages are received from other ships. They are expected to take the from of a [`gram`]().
240−from of a [`gram`]().
241
169
242**Gifts given in response:** [`%nice` or `%mean`](#ack).
170**Gifts given in response:** [`%nice` or `%mean`](#ack).
243
171
244### 3.4.1 `gram` <a id="gram"/>
172### 3.4.1 `gram` <a id="gram"/>
245
173
246−There are three messages of note, all concerning authentication
174+There are three messages of note, all concerning authentication <a id="foreign"/>
247−<a id="foreign"/> - `[/lon ses]` <a id="lon"/> is a login request, which
248−contains the session wishing to authenticate. - if the session is already
175+- `[/lon ses]` <a id="lon"/> is a login request, which contains the session wishing to authenticate.
249−authorized, a reply of `[/aut ses]` from (%eyre on) a foreign ship prescribes
176+- if the session is already authorized, a reply of `[/aut ses]` from (%eyre on) a foreign ship prescribes that the session is henceforth allowed to act on its behalf.
250−that the session is henceforth allowed to act on its behalf. All waiting `/~/as`
251−are resolved as succesful. - otherwise, `[/hat ses hart]` contains the ship's
177+All waiting `/~/as` are resolved as succesful.
252−preferred hostname. This can then be used to redirect the client. All waiting
178+- otherwise, `[/hat ses hart]` contains the ship's preferred hostname. This can then be used to redirect the client.
253−`/~/as` are given `307 Redirect`s to `/~/am` on the provided host.
179+All waiting `/~/as` are given `307 Redirect`s to `/~/am` on the provided host.
254
180
255**Gifts given in response:** nice?
181**Gifts given in response:** nice?
256
182
257# 4. Responses <a id="gift"/>
183# 4. Responses <a id="gift"/>
258−
259## 4.1 `[%nice ~]`, `[%mean ares]`, network acknowledgement <a id="ack"/>
184## 4.1 `[%nice ~]`, `[%mean ares]`, network acknowledgement <a id="ack"/>
260−
261−A `%nice` is given upon receiving an [ames message](#gram), indicating succesful
185+A `%nice` is given upon receiving an [ames message](#gram), indicating succesful receipt. `%mean` is currently unused directly, but reserved for error conditions.
262−receipt. `%mean` is currently unused directly, but reserved for error
263−conditions.
264
186
265## `[%sigh p=cage]`, inbound http response
187## `[%sigh p=cage]`, inbound http response
266−
267−An HTTP response gets converted to the requested mark, or `%tang` in case of
188+An HTTP response gets converted to the requested mark, or `%tang` in case of error.
268−error.
269
189
270## 4.2 `[%thou httr]`, full HTTP response
190## 4.2 `[%thou httr]`, full HTTP response
271−
272−Most requests are served with one coherent response, consisting of - `p=@ud`, an
191+Most requests are served with one coherent response, consisting of
192+- `p=@ud`, an HTTP status code
273−HTTP status code - `q=mess`, response headers - `r=(unit octs)`, an optional
193+- `q=mess`, response headers
274−body
194+- `r=(unit octs)`, an optional body
275
195
276## 4.3 `[%that httr]`, partial HTTP response <a id="that"/>
196## 4.3 `[%that httr]`, partial HTTP response <a id="that"/>
277−
278−[EventStream responses](#of-ixor) consist of multiple sequential chunks. Treated
197+[EventStream responses](#of-ixor) consist of multiple sequential chunks. Treated as `%thou`, except the request is kept open.
279−as `%thou`, except the request is kept open.
280
198
281## 4.4 `[%thar (unit octs)]`, partial HTTP body <a id="thar"/>
199## 4.4 `[%thar (unit octs)]`, partial HTTP body <a id="thar"/>
200+Complementing `%that`, `%thar` is a body chunk, containing an event. An empty `%thar` signals for the connection to be closed.
282
201
283−Complementing `%that`, `%thar` is a body chunk, containing an event. An empty
284−`%thar` signals for the connection to be closed.
285−
286−Body chunks, besides `[1 '\0a']`(a heartbeat newline), are encoded from `even`
287−events. The stem becomes the [`event` field](#eventsource), and the content,
202+Body chunks, besides `[1 '\0a']`(a heartbeat newline), are encoded from `even` events. The stem becomes the [`event` field](#eventsource), and the content, `data` lines.
288−`data` lines.
289
203
290### 4.4.1 `even`, event types <a id="even"/>
204### 4.4.1 `even`, event types <a id="even"/>
291−
292There are three events that a [client subscription](#subs) will be given.
205There are three events that a [client subscription](#subs) will be given.
293
206
294−- `[%news hash]`<a id="even-news"/> is a `%f` update, and contains the
207+- `[%news hash]`<a id="even-news"/> is a `%f` update, and contains the relevant dependency token.
295− relevant dependency token.
296−- `[%rush [term path] wain]`<a id="even-rush"/> is sourced subscription data.
208+- `[%rush [term path] wain]`<a id="even-rush"/> is sourced subscription data.
297−- `[%mean [term path] ares]` is a sourced subscription error.
209+- `[%mean [term path] ares]` is a sourced subscription error.
298
210
299# 5. Vane requests <a id="note"/>
211# 5. Vane requests <a id="note"/>
300−
301−## 5.1 `` [%a %wont sock `[path *]`gram] ``, outbound message
212+## 5.1 ``[%a %wont sock `[path *]`gram]``, outbound message
302−
303The `%a` interface provides conveyance of [messages](#gram) over UDP.
213The `%a` interface provides conveyance of [messages](#gram) over UDP.
304
214
305Signs a `%woot` upon message arrival.
215Signs a `%woot` upon message arrival.
306
216
307## 5.2 `[%b ?(%wait %rest) time]`, timeout set/unset <a id="wait"/>
217## 5.2 `[%b ?(%wait %rest) time]`, timeout set/unset <a id="wait"/>
308−
309−All open [subscriptions](#of-ixor) require a "heartbeat" newline every `~s30`.
310−When this fails to arrive, a complementary timer is set for `~m1`, after which
218+All open [subscriptions](#of-ixor) require a "heartbeat" newline every `~s30`. When this fails to arrive, a complementary timer is set for `~m1`, after which the client is considered to have departed.
311−the client is considered to have departed. The `%b` timer interface is used to
312−schedule the next such event, or cancel past scheduled ones when a connection
219+The `%b` timer interface is used to schedule the next such event, or cancel past scheduled ones when a connection closes.
313−closes.
314
220
315Signs a `%wake` upon timer activation.
221Signs a `%wake` upon timer activation.
316
222
317## 5.3 `[%f %wasp @uvI]`, dependency listen <a id="wasp"/>
223## 5.3 `[%f %wasp @uvI]`, dependency listen <a id="wasp"/>
224+Knowledge of filesystem changes, requested by `/~/on` and `/~/in`, is requested the `%wasp` note. It contains the hash token which idenitifes a set of dependencies to query. Saved as a (live[live](#live)#live] request in case of cancellation when caused by `/~/on`.
318
225
319−Knowledge of filesystem changes, requested by `/~/on` and `/~/in`, is requested
320−the `%wasp` note. It contains the hash token which idenitifes a set of
321−dependencies to query. Saved as a (live[live](#live)\#live\] request in case of
322−cancellation when caused by `/~/on`.
323−
324−Signs a `%writ` upon change.
226+Signs a `%writ` upon change.
325
227
326## 5.4 `[%f %exec (unit silk)]`, functional transformations
228## 5.4 `[%f %exec (unit silk)]`, functional transformations
327−
328−The `%f` interface converts nouns from one form to another. An empty unit
229+The `%f` interface converts nouns from one form to another. An empty unit represents cancellation.
329−represents cancellation.
330
230
331Signs a `%made` containing the computation result.
231Signs a `%made` containing the computation result.
332
232
333−The `silk`s used are as follows:
233+The `silk`s used are as follows:
334
234
335### `[%cast %mime %boil mark beam [%web span ~]]`, file load
235### `[%cast %mime %boil mark beam [%web span ~]]`, file load
336−
337−The simplest functional request is the construction of a page. Saved as a
236+The simplest functional request is the construction of a page. Saved as a [live](#live) request in case of cancellation.
338−[live](#live) request in case of cancellation. - The `mark` is the request
237+- The `mark` is the request extension, defaulting to `%html`.
339−extension, defaulting to `%html`. - A `beam`(path in `%clay`) is decoded from
238+- A `beam`(path in `%clay`) is decoded from the request path.
340−the request path. - A `/web/<nyp ced quy>` virtual path is appended,
341−span-encoding method, auth, and query string.
239+- A `/web/<nyp ced quy>` virtual path is appended, span-encoding method, auth, and query string.
342
240
343### `[%cast mark %done ~ cage]`, convert <a id="done"/>
241### `[%cast mark %done ~ cage]`, convert <a id="done"/>
344−
345−This is used when communicating with apps, in both directions. - Client messages
242+This is used when communicating with apps, in both directions.
346−are encoded as JSON objects, but can contain many different marks. Ones sent to
243+- Client messages are encoded as JSON objects, but can contain many different marks. Ones sent to non-`json.json` endpoints are `%cast` to he relevant mark, with a `cage` of `[%json p:!>(*json) jon]`, prior to being sent onwards to applications with `[%g %mess]`.
347−non-`json.json` endpoints are `%cast` to he relevant mark, with a `cage` of
348−`[%json p:!>(*json) jon]`, prior to being sent onwards to applications with
349−`[%g %mess]`. - Subscription responses also come in arbitrary marks, but are
350−required to be sent as whichever mark an endpoint is subscribed to. They get
244+- Subscription responses also come in arbitrary marks, but are required to be sent as whichever mark an endpoint is subscribed to. They get `%cast` prior to being sent back as a `%that` EventSource segment.
351−`%cast` prior to being sent back as a `%that` EventSource segment.
352
245
353## 5.5 `%g`, app actions
246## 5.5 `%g`, app actions
354−
355−The end goal of many a userspace `hymn.hook` is to provide UI for a `%gall` app.
356−To accommodate this, various functionality needs to be interfaced with.
247+The end goal of many a userspace `hymn.hook` is to provide UI for a `%gall` app. To accommodate this, various functionality needs to be interfaced with.
357
248
358### `[%mess hapt ship cage]`, app message <a id="mess"/>
249### `[%mess hapt ship cage]`, app message <a id="mess"/>
359−
360−After a `/~/to` POST has been received, and possibly converted to the correct
250+After a `/~/to` POST has been received, and possibly converted to the correct mark, it is sent to `%g` for processing. The `hapt` is the destination, the `ship` is the source, the `cage` is the marked message data.
361−mark, it is sent to `%g` for processing. The `hapt` is the destination, the
362−`ship` is the source, the `cage` is the marked message data.
363
251
364Signs a `%nice` or `%mean`, in userspace or upon crashing.
252Signs a `%nice` or `%mean`, in userspace or upon crashing.
365
253
366### `[%show hapt ship path]`, app subscription<a id="show"/>
254### `[%show hapt ship path]`, app subscription<a id="show"/>
255+An `/~/is` PUT, if not already registered, results in a new subscription. A request for such contains the destination `hapt`, requesting `ship`, and app-internal `path`.
367
256
368−An `/~/is` PUT, if not already registered, results in a new subscription. A
369−request for such contains the destination `hapt`, requesting `ship`, and
370−app-internal `path`.
371−
372−Signs a `%nice` on init, `%rush` on data, and a `%mean` in subscription
257+Signs a `%nice` on init, `%rush` on data, and a `%mean` in subscription termination.
373−termination.
374
258
375### `[%took hapt ship]`, %rush confirmation
259### `[%took hapt ship]`, %rush confirmation
376−
377−A `%took` must be sent to acknowledge the receipt of subscription data, which in
260+A `%took` must be sent to acknowledge the receipt of subscription data, which in this case occurs automatically.
378−this case occurs automatically.
379
261
380Causes no signs.
262Causes no signs.
381
263
382### `[%nuke hapt $|(ship [ship path])]`, subscription cancel
264### `[%nuke hapt $|(ship [ship path])]`, subscription cancel
383−
384−An `/~/is` DELETE is used to remove a subscription, and is converted
385−straightforwardly to a `%nuke`. Additionally, when a client disconnects, all of
265+An `/~/is` DELETE is used to remove a subscription, and is converted straightforwardly to a `%nuke`. Additionally, when a client disconnects, all of its subscriptions are `%nuke`d to reflect their experiation.
386−its subscriptions are `%nuke`d to reflect their experiation.
387
266
388Signs an empty `%mean` for each open subscription that is closed.
267Signs an empty `%mean` for each open subscription that is closed.
389
268
390# 6. In-flight metadata <a id="whir"/>
269# 6. In-flight metadata <a id="whir"/>
391−
392−Various state can be associated with requests, but not necessarily be returned
270+Various state can be associated with requests, but not necessarily be returned in responses to them.
393−in responses to them.
394
271
395## 6.1 `~`, dropthrough <a id="wire-drop"/>
272## 6.1 `~`, dropthrough <a id="wire-drop"/>
396−
397−If the response should be sent statelessly further up the duct, the `wire` is
273+If the response should be sent statelessly further up the duct, the `wire` is empty. This is used in `%f` pure functional page generation.
398−empty. This is used in `%f` pure functional page generation.
399
274
400## 6.2 `[?(%y %n) ...]`, stability <a id="wire-live"/>
275## 6.2 `[?(%y %n) ...]`, stability <a id="wire-live"/>
401−
402−The first element of a [`%b` timer](#wait) path distinguishes between live(%y)
276+The first element of a [`%b` timer](#wait) path distinguishes between live(%y) and dying(%n) channels.
403−and dying(%n) channels.
404
277
405## 6.3 `/of/[ixor]`, view
278## 6.3 `/of/[ixor]`, view
406−
407Designates which stream to act upon. Present on `%b` timer cards.
279Designates which stream to act upon. Present on `%b` timer cards.
408
280
409## 6.4 `/on/<deps>`
281## 6.4 `/on/<deps>`
410−
411−Present on `%f` dependency news. Looked up in [the state](#bolo) to see if any
282+Present on `%f` dependency news. Looked up in [the state](#bolo) to see if any response is necessary.
412−response is necessary.
413
283
414## 6.5 `/to/<hasp>/<ship>`, app <a id="wire-to"/>
284## 6.5 `/to/<hasp>/<ship>`, app <a id="wire-to"/>
415−
416Present on `%f` translations of message marks
285Present on `%f` translations of message marks
417
286
418## 6.6 `/is/[ixor]/{hasp path}`, subscription <a id="wire-is"/>
287## 6.6 `/is/[ixor]/{hasp path}`, subscription <a id="wire-is"/>
419−
420−Subscription requests. Present on `%g` subscription requests, and `%f`
288+Subscription requests. Present on `%g` subscription requests, and `%f` translations of [`%rush` subscription data](#show).
421−translations of [`%rush` subscription data](#show).
422
289
423# 7. Vane responses <a id="sign"/>
290# 7. Vane responses <a id="sign"/>
424−
425−The goal of requests is to cause some manner of result. Specifically,
291+The goal of requests is to cause some manner of result. Specifically,
426
292
427## 7.1 `[%a %woot ship coop]`, acknowledgement
293## 7.1 `[%a %woot ship coop]`, acknowledgement
428−
429A [message](#mess) which has been acknowledged remotely.
294A [message](#mess) which has been acknowledged remotely.
430
295
431## 7.2 `[%b %wake ~]`, timer
296## 7.2 `[%b %wake ~]`, timer
432−
433−Timer activate. If this is a `%y` [timer](#wire-live), send a newline to each
297+Timer activate. If this is a `%y` [timer](#wire-live), send a newline to each `/~/of/[oryx]`; if `%n` and the `oryx` still has no listeners, wipe its state/subscriptions.
434−`/~/of/[oryx]`; if `%n` and the `oryx` still has no listeners, wipe its
435−state/subscriptions.
436
298
437## 7.3 `[%f %news ~]`, dependency update
299## 7.3 `[%f %news ~]`, dependency update
438−
439−This message signifies a `hash` has changed, and listeners to it should be
300+This message signifies a `hash` has changed, and listeners to it should be updated. Results in [`%news` events](#even-news) to concerned streams, and [full responses](#thou) to relevant long-polls.
440−updated. Results in [`%news` events](#even-news) to concerned streams, and [full
441−responses](#thou) to relevant long-polls.
442
301
443## 7.4 `[%f %made (each bead (list tank))]`, compute result
302## 7.4 `[%f %made (each bead (list tank))]`, compute result
444−
445−Further action depends on the [`wire`](#whir). It can: - Be empty, designating
303+Further action depends on the [`wire`](#whir). It can:
446−direct request data, which is [served](#mime) back up the `duct`. - Contain [app
304+- Be empty, designating direct request data, which is [served](#mime) back up the `duct`.
447−and requester identities](#wire-to), which designates an app message has been
305+- Contain [app and requester identities](#wire-to), which designates an app message has been converted to the correct `mark`, and can now be [sent to the app](#mess) with an empty `wire`.
448−converted to the correct `mark`, and can now be [sent to the app](#mess) with an
449−empty `wire`. - Contain an `oryx`, app, and `path`, designating subscription
450−data. If the subscription is active in the relevant `stem`, it is acknowledged
306+- Contain an `oryx`, app, and `path`, designating subscription data. If the subscription is active in the relevant `stem`, it is acknowledged with a `%took` and given as a [`%rush` event](#even-rush) to all listeners. Otherwise, the subscription is `%nuke`d.
451−with a `%took` and given as a [`%rush` event](#even-rush) to all listeners.
452−Otherwise, the subscription is `%nuke`d.
453
307
454## 7.5 `%g`, applications
308## 7.5 `%g`, applications
455−
456The `%g` interface sends several responses.
309The `%g` interface sends several responses.
457
310
458### `[%nice ~]` and `[%mean ares]`, acknowledgment
311### `[%nice ~]` and `[%mean ares]`, acknowledgment
459−
460−These are given as a response to `%mess`, with an empty wire, and `%show`s, with
312+These are given as a response to `%mess`, with an empty wire, and `%show`s, with a [subscription wire](#wire-is).
461−a [subscription wire](#wire-is). The former is [served](#mime) back to the
462−requester(i.e. the remaining duct), and the latter returned to the client.
313+The former is [served](#mime) back to the requester(i.e. the remaining duct), and the latter returned to the client.
463
314
464### 7.5 `[%rush cage]`, subscription data
315### 7.5 `[%rush cage]`, subscription data
465−
466−A `%rush` arrives on a [subscription wire](#wire-is), and is then sent [for
316+A `%rush` arrives on a [subscription wire](#wire-is), and is then sent [for conversion](#done) to the correct mark. An unexpected `%rush` is `%nuke`d.
467−conversion](#done) to the correct mark. An unexpected `%rush` is `%nuke`d.
468
317
469# 8. Server state <a id="bolo"/>
318# 8. Server state <a id="bolo"/>
470−
471There is a quantity of data that persists between events.
319There is a quantity of data that persists between events.
472
320
473## 8.1 Global
321## 8.1 Global
474−
475−- `path` <a id="prefix"/> the default path which is prepended to relative
476− plain requests. Initialized to `/<our>`, the serving ship; this interprets
322+- `path` <a id="prefix"/> the default path which is prepended to relative plain requests. Initialized to `/<our>`, the serving ship; this interprets the first element of requests as a desk, and injects a `case` of `0`.
477− the first element of requests as a desk, and injects a `case` of `0`.
478−- `(jar ship hart)` remembered hostnames in order of preference, made
323+- `(jar ship hart)` remembered hostnames in order of preference, made accessible through `.^`
479− accessible through `.^`
480−- The default values(for self) are `http://<ship>.urbit.org:80` if not on a
324+ + The default values(for self) are `http://<ship>.urbit.org:80` if not on a fake network, followed by `https://0.0.0.0:[port]`.
481− fake network, followed by `https://0.0.0.0:[port]`.
482−- `(jug deps (each duct oryx))`, `/~/on` and `/~/in` endpoints listening to
325+- `(jug deps (each duct oryx))`, `/~/on` and `/~/in` endpoints listening to `%c` updates.
483− `%c` updates.
484−- `(map duct live)`, inbound request state
326+- `(map duct live)`, inbound request state
485−- `(map ixor oryx)`, open `/~/of` streams.
327+- `(map ixor oryx)`, open `/~/of` streams.
486−- `(map oryx stem)`, open view state.
328+- `(map oryx stem)`, open view state.
487−- `(map hole sink)`, session state.
329+- `(map hole sink)`, session state.
488−- `(map hole ,[ship ?])`, foreign session names, along with their origin ships
330+- `(map hole ,[ship ?])`, foreign session names, along with their origin ships and whether they are authorized to act on our behalf.
489− and whether they are authorized to act on our behalf.
490
331
491## 8.2 `live`, per-request state <a id="live"/>
332## 8.2 `live`, per-request state <a id="live"/>
492−
493−To honor request cancellations, each unserved request must track which effect it
333+To honor request cancellations, each unserved request must track which effect it is causing.
494−is causing. + `[%exec whir]` if it is a ford data request +
334++ `[%exec whir]` if it is a ford data request
495−`[%wasp (list @uvH)]` if it is a ford dependency poll + `[%xeno ship]` if this
335++ `[%wasp (list @uvH)]` if it is a ford dependency poll
336++ `[%xeno ship]` if this is a proxied request
496−is a proxied request + `[%poll ixor]` if this is a session long-poll
337++ `[%poll ixor]` if this is a session long-poll
497
338
498## 8.3 `sink`, per-session state <a id="sink"/>
339## 8.3 `sink`, per-session state <a id="sink"/>
499−
500−Each `hole` has associated authentication state. - `priv`, a random secret that
340+Each `hole` has associated authentication state.
501−verifies the session, stored in cookie. - `[ship (set ship)]`, the `ship` a
341+- `priv`, a random secret that verifies the session, stored in cookie.
502−session is acting as and any others it has been authorized with. -
342+- `[ship (set ship)]`, the `ship` a session is acting as and any others it has been authorized with.
503−`(jug ship ,[path duct])`, authentication `/~/as` requests waiting on foreign
343+- `(jug ship ,[path duct])`, authentication `/~/as` requests waiting on foreign ships.
504−ships. - `(set oryx)`, views associated with this session. Dual of `hole` in
344+- `(set oryx)`, views associated with this session. Dual of `hole` in `stem`.
505−`stem`.
506
345
507## 8.4 `stem`, per-view state <a id="stem"/>
346## 8.4 `stem`, per-view state <a id="stem"/>
508−
509−Each `oryx` has active subscription state. - `hole`, session in which this view
347+Each `oryx` has active subscription state.
510−resides. Dual of `(set oryx)` in `sink`. - `ixor`<a id="ixor"/>, a cached hash
348+- `hole`, session in which this view resides. Dual of `(set oryx)` in `sink`.
511−of the `oryx` that is used as a subscription id, by the `/~/of` EventStream. -
349+- `ixor`<a id="ixor"/>, a cached hash of the `oryx` that is used as a subscription id, by the `/~/of` EventStream.
512−`[,@u (map ,@u even)]`, queued [events](#even). Has a maximum size. -
350+- `[,@u (map ,@u even)]`, queued [events](#even). Has a maximum size.
513−`(unit ,[duct @u ?])`, http connection, its last received event, and whether it
351+- `(unit ,[duct @u ?])`, http connection, its last received event, and whether it is a long-poll request.
514−is a long-poll request. - `(set deps)`, subscribed `/~/in` points. Mirrored in
352+- `(set deps)`, subscribed `/~/in` points. Mirrored in `bolo`.
515−`bolo`. - `(map ,[hasp path] mark)`, subscribed `/~/is` points - `(que duct)`,
353+- `(map ,[hasp path] mark)`, subscribed `/~/is` points
516−where any `%nice`/\`%mean
354+- `(que duct)`, where any `%nice`/`%mean
517
355
518# Appendix A: Glossary
356# Appendix A: Glossary
519−
520−- An `oryx` is a CSRF token used for [authenticated requests](#auth)
357+- An `oryx` is a CSRF token used for [authenticated requests](#auth)
521−- `mean.json`<a id="mean-json"/> is a rendering of hoon `ares`: an error
358+- `mean.json`<a id="mean-json"/> is a rendering of hoon `ares`: an error message formatted as `{fail:'type',mess:"Error message"}`
522− message formatted as `{fail:'type',mess:"Error message"}`
523−- `urb.js`<a id="urb-js"/> is the standard client-side implementation of the
359+- `urb.js`<a id="urb-js"/> is the standard client-side implementation of the `%eyre` protocols, normally found in `/=main=/lib`
524− `%eyre` protocols, normally found in `/=main=/lib`
525−- The act of "serving"<a id="mime"/>, refers to the wrapping of a `%mime` cage
526− in an HTTP `200 Success` response, and errors or other marks being [sent to
360+- The act of "serving"<a id="mime"/>, refers to the wrapping of a `%mime` cage in an HTTP `200 Success` response, and errors or other marks being [sent to ford](#done) for conversion. Inside a `sign`, this conversion occurs along the same [`wire`](#wire); otherwise, the `wire` is [empty](#wire-drop).
527− ford](#done) for conversion. Inside a `sign`, this conversion occurs along
528− the same [`wire`](#wire); otherwise, the `wire` is [empty](#wire-drop).
529
361
530# Appendix B: EventSource <a id="eventsource"/>
362# Appendix B: EventSource <a id="eventsource"/>
531
363
@@ -533,10 +365,12 @@ UNIMPLEMENTED
533
365
534For more information, see <http://dev.w3.org/html5/eventsource/>
366For more information, see <http://dev.w3.org/html5/eventsource/>
535
367
536−An EventSource is a stream of events, encoded in a conceptually infinite `GET`
368+An EventSource is a stream of events, encoded in a conceptually infinite `GET` request. Each event is formatted as
537−request. Each event is formatted as
369+ ```http
370+ id: {number}
371+ event: {type}
372+ data: {contents}
538−`http id: {number} event: {type} data: {contents} data: {more-contents}`
373+ data: {more-contents}
374+ ```
539
375
540−Data lines are joined by ASCII `10` newline characters; multiple newlines
376+Data lines are joined by ASCII `10` newline characters; multiple newlines signify separate events. The `id` is used in a `Last-Event-ID` HTTP request header upon reconnection.
541−signify separate events. The `id` is used in a `Last-Event-ID` HTTP request
542−header upon reconnection.
docs/arvo/internals/ford.md+55−47
@@ -1,47 +1,53 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 6
3sort: 6
4---
4---
5
5
6+
6−# `%ford`
7+`%ford`
8+=======
7
9
8Our typed and marked computation engine.
10Our typed and marked computation engine.
9
11
10−A variety of different services are provided by `%ford`, but they mostly involve
12+A variety of different services are provided by `%ford`, but they mostly
11−compiling hook files, slapping/slammming code with marked data, and converting
13+involve compiling hook files, slapping/slammming code with marked data,
12−data between marks, including validating data to a mark. Throughout every
14+and converting data between marks, including validating data to a mark.
13−computation, `%ford` keeps track of which resources are dependencies so that the
15+Throughout every computation, `%ford` keeps track of which resources are
14−client may be aware when one or more dependencies are updated.
16+dependencies so that the client may be aware when one or more
17+dependencies are updated.
15
18
16−`%ford` neither accepts unix events nor produces effects. It exists entirely for
19+`%ford` neither accepts unix events nor produces effects. It exists
17−the benefit of applications and other vanes, in particular `%gall`. `%eyre`
20+entirely for the benefit of applications and other vanes, in particular
18−exposes the functional publishing aspects of `%ford` while `%gall` uses `%ford`
21+`%gall`. `%eyre` exposes the functional publishing aspects of `%ford`
19−to control the execution of applications. `%clay` is intended to use `%ford` to
22+while `%gall` uses `%ford` to control the execution of applications.
20−managed marked data, but this is not yet reality.
23+`%clay` is intended to use `%ford` to managed marked data, but this is
24+not yet reality.
21
25
22<list></list>
26<list></list>
23
27
24−## Cards
28+Cards
29+-----
25
30
26−`%ford` accepts just one card, `%exec`. This is misleading, however, since there
31+`%ford` accepts just one card, `%exec`. This is misleading, however,
27−are fourteen different `silk`s that may be used with it. In every case, the
32+since there are fourteen different `silk`s that may be used with it. In
28−expected response to a `%exec` card is a `%made` gift with either an error or
33+every case, the expected response to a `%exec` card is a `%made` gift
29−the produced result along with its set of dependencies.
34+with either an error or the produced result along with its set of
35+dependencies.
30
36
31−Silks may autocons, so that the product of a cell of silks is a cell of the
37+Silks may autocons, so that the product of a cell of silks is a cell of
32−product of the two silks.
38+the product of the two silks.
33
39
34### `%bake`
40### `%bake`
35
41
36−Tries to functionally produce the file at a given beam with the given mark and
42+Tries to functionally produce the file at a given beam with the given
37−heel. It fails if there is no way to translate at this level.
43+mark and heel. It fails if there is no way to translate at this level.
38
44
39### `%boil`
45### `%boil`
40
46
41−Functionally produces the file at a given beam with the given mark and heel. If
47+Functionally produces the file at a given beam with the given mark and
42−there is no way to translate at this beam, we pop levels off the stack and
48+heel. If there is no way to translate at this beam, we pop levels off
43−attempt to bake there until we find a level we can bake. This should almost
49+the stack and attempt to bake there until we find a level we can bake.
44−always be called instead of `%bake`.
50+This should almost always be called instead of `%bake`.
45
51
46### `%call`
52### `%call`
47
53
@@ -49,50 +55,52 @@ Slams the result of one silk against the result of another.
49
55
50### `%cast`
56### `%cast`
51
57
52−Translates the given silk to the given mark, if possible. This is one of the
58+Translates the given silk to the given mark, if possible. This is one of
53−critical and fundamental operations of ford.
59+the critical and fundamental operations of ford.
54
60
55### `%diff`
61### `%diff`
56
62
57−Diffs the two given silks (which must be of the same mark), producing a cage of
63+Diffs the two given silks (which must be of the same mark), producing a
58−the mark specified in `++mark` in `++grad` for the mark of the two silks.
64+cage of the mark specified in `++mark` in `++grad` for the mark of the
65+two silks.
59
66
60### `%done`
67### `%done`
61
68
62−Produces exactly its input. This is rarely used on its own, but many silks are
69+Produces exactly its input. This is rarely used on its own, but many
63−recursively defined in terms of other silks, so we often need a silk that simply
70+silks are recursively defined in terms of other silks, so we often need
64−produces its input. A monadic return, if you will.
71+a silk that simply produces its input. A monadic return, if you will.
65
72
66### `%dude`
73### `%dude`
67
74
68−Computes the given silk with the given tank as part of the stack trace if there
75+Computes the given silk with the given tank as part of the stack trace
69−is an error.
76+if there is an error.
70
77
71### `%dune`
78### `%dune`
72
79
73−Produces an error if the cage is empty. Otherwise, it produces the value in the
80+Produces an error if the cage is empty. Otherwise, it produces the value
74−unit.
81+in the unit.
75
82
76### `%mute`
83### `%mute`
77
84
78−Takes a silk and a list of changes to make to the silk. At each wing in the list
85+Takes a silk and a list of changes to make to the silk. At each wing in
79−we put the value of the associated silk.
86+the list we put the value of the associated silk.
80
87
81### `%pact`
88### `%pact`
82
89
83−Applies the second silk as a patch to the first silk. The second silk must be of
90+Applies the second silk as a patch to the first silk. The second silk
84−the mark specified in `++mark` in `++grad` for the mark of the first silk.
91+must be of the mark specified in `++mark` in `++grad` for the mark of
92+the first silk.
85
93
86### `%plan`
94### `%plan`
87
95
88−Performs a structured assembly directly. This is not generally directly useful
96+Performs a structured assembly directly. This is not generally directly
89−because several other silks perform supersets of this functionality. We don't
97+useful because several other silks perform supersets of this
90−usually have naked hoods outside ford.
98+functionality. We don't usually have naked hoods outside ford.
91
99
92### `%reef`
100### `%reef`
93
101
94−Produces a core containing the entirety of zuse and hoon, suitable for running
102+Produces a core containing the entirety of zuse and hoon, suitable for
95−arbitrary code against. The mark is `%noun`.
103+running arbitrary code against. The mark is `%noun`.
96
104
97### `%ride`
105### `%ride`
98
106
@@ -100,5 +108,5 @@ Slaps a twig against a subject silk. The mark of the result is `%noun`.
100
108
101### `%vale`
109### `%vale`
102
110
103−Validates untyped data from a ship against a given mark. This is an extremely
111+Validates untyped data from a ship against a given mark. This is an
104−useful function.
112+extremely useful function.
docs/arvo/internals/ford/commentary.md+956−852
@@ -1,10 +1,13 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3---
3---
4
4
5−# Reference
6
5
7−## Data Models
6+Reference
7+=========
8+
9+Data Models
10+-----------
8
11
9### `++axle`, formal state
12### `++axle`, formal state
10
13
@@ -13,13 +16,13 @@ navhome: '/docs/'
13 pol=(map ship baby) ::
16 pol=(map ship baby) ::
14 == ::
17 == ::
15
18
16−This is the formal state of our vane. Anything that must be remembered between
19+This is the formal state of our vane. Anything that must be remembered
17−calls to ford must be stored here. The number `%1` is a version number for our
20+between calls to ford must be stored here. The number `%1` is a version
18−state that allows us to upgrade the structure of our state in the future if we
21+number for our state that allows us to upgrade the structure of our
19−wish.
22+state in the future if we wish.
20
23
21−`pol` is the a map from every ship on our pier to their individual ford state.
24+`pol` is the a map from every ship on our pier to their individual ford
22−There is no shared ford state -- every ship is entirely separate.
25+state. There is no shared ford state -- every ship is entirely separate.
23
26
24### `++baby`, state by ship
27### `++baby`, state by ship
25
28
@@ -31,18 +34,19 @@ There is no shared ford state -- every ship is entirely separate.
31
34
32This is the state specific to each ship.
35This is the state specific to each ship.
33
36
34−`tad` and `dym` keep track of the tasks we're currently working on. `dym` is a
37+`tad` and `dym` keep track of the tasks we're currently working on.
35−map from ducts to task numbers, and `q.tad` is a map from task number to the
38+`dym` is a map from ducts to task numbers, and `q.tad` is a map from
36−task itself. `p.tad` is the next available task number. Thus, the keys of
39+task number to the task itself. `p.tad` is the next available task
37−`q.tad` are a subset of the numbers less than `p.tad`, and ford has attempted
40+number. Thus, the keys of `q.tad` are a subset of the numbers less than
38−exactly `p.tad` tasks so far.
41+`p.tad`, and ford has attempted exactly `p.tad` tasks so far.
39
42
40−`jav` is the cache of previously-solved problems. The keys are a pair of a term
43+`jav` is the cache of previously-solved problems. The keys are a pair of
41−(either `%hood`, `%slap`, or `%slam`) and a noun that represents the exact
44+a term (either `%hood`, `%slap`, or `%slam`) and a noun that represents
42−problem solved. In the case of a `%hood`, then, the key is of the form
45+the exact problem solved. In the case of a `%hood`, then, the key is of
43−`[%hood beam cage]`. For `%slap`, there is `[%slap vase twig]`. For `%slam`,
46+the form `[%hood beam cage]`. For `%slap`, there is `[%slap vase twig]`.
44−there is `[%slam vase vase]`. The values are the result of solving the problem.
47+For `%slam`, there is `[%slam vase vase]`. The values are the result of
45−Note that this cache is wiped in `++stay` when ford is reloaded.
48+solving the problem. Note that this cache is wiped in `++stay` when ford
49+is reloaded.
46
50
47### `++task`, problem in progress
51### `++task`, problem in progress
48
52
@@ -52,12 +56,12 @@ Note that this cache is wiped in `++stay` when ford is reloaded.
52 kig=[p=@ud q=(map ,@ud beam)] :: blocks
56 kig=[p=@ud q=(map ,@ud beam)] :: blocks
53 == ::
57 == ::
54
58
55−This is all the state we keep regarding a particular task. `nah` is the duct
59+This is all the state we keep regarding a particular task. `nah` is the
56−which asked us to solve the problem, and `kas` is the most recent statement of
60+duct which asked us to solve the problem, and `kas` is the most recent
57−the problem.
61+statement of the problem.
58
62
59−`kig` keeps track of which resources we are blocked on. Our blocks are stored by
63+`kig` keeps track of which resources we are blocked on. Our blocks are
60−index in `q.kig`, and the next available index is `p.kig`.
64+stored by index in `q.kig`, and the next available index is `p.kig`.
61
65
62### `++silk`, problem
66### `++silk`, problem
63
67
@@ -79,12 +83,13 @@ index in `q.kig`, and the next available index is `p.kig`.
79 [%vale p=mark q=ship r=*] :: validate [our his]
83 [%vale p=mark q=ship r=*] :: validate [our his]
80 == ::
84 == ::
81
85
82−This is the every type of problem we can solve. Every `%exec` kiss that requests
86+This is the every type of problem we can solve. Every `%exec` kiss that
83−us to solve a problem must choose one of these problems to solve.
87+requests us to solve a problem must choose one of these problems to
88+solve.
84
89
85−Because this is both an internal structure used in ford and the public interface
90+Because this is both an internal structure used in ford and the public
86−to ford, we choose to document this structure in our discussion of the public
91+interface to ford, we choose to document this structure in our
87−interface to ford below.
92+discussion of the public interface to ford below.
88
93
89### `++calx`, cache line
94### `++calx`, cache line
90
95
@@ -94,19 +99,20 @@ interface to ford below.
94 [%slam p=calm q=[p=vase q=vase] r=vase] :: compute
99 [%slam p=calm q=[p=vase q=vase] r=vase] :: compute
95 == ::
100 == ::
96
101
97−There are three kinds of cache entries. Every entry includes some metadata in
102+There are three kinds of cache entries. Every entry includes some
98−`p` and is the combination of an input and its output.
103+metadata in `p` and is the combination of an input and its output.
99
104
100−The input to a `%hood` is the location of the resource and a cage representing
105+The input to a `%hood` is the location of the resource and a cage
101−the data at that location. The output is the hood found by compiling the given
106+representing the data at that location. The output is the hood found by
102−cage at the given location.
107+compiling the given cage at the given location.
103
108
104−The input to a `%slap` is the vase of the subject and the twig of the formula
109+The input to a `%slap` is the vase of the subject and the twig of the
105−against which we are slapping the subject. The output is the vase produced by
110+formula against which we are slapping the subject. The output is the
106−slapping them.
111+vase produced by slapping them.
107
112
108−The input to a `%slam` is the vase of the subject and the vase of the gate which
113+The input to a `%slam` is the vase of the subject and the vase of the
109−we are slapping. The output is the vase produced by slamming them.
114+gate which we are slapping. The output is the vase produced by slamming
115+them.
110
116
111### `++calm`, cache line metadata
117### `++calm`, cache line metadata
112
118
@@ -115,9 +121,9 @@ we are slapping. The output is the vase produced by slamming them.
115 dep=(set beam) :: dependencies
121 dep=(set beam) :: dependencies
116 == ::
122 == ::
117
123
118−Every line in the cache needs to have two pieces of metadata. We must know the
124+Every line in the cache needs to have two pieces of metadata. We must
119−last time this line in the cache was accessed, and we must know what are the
125+know the last time this line in the cache was accessed, and we must know
120−dependencies of this line.
126+what are the dependencies of this line.
121
127
122### `++hood`, assembly components
128### `++hood`, assembly components
123
129
@@ -131,43 +137,46 @@ dependencies of this line.
131
137
132When assembling a hook file, we split it into several sections.
138When assembling a hook file, we split it into several sections.
133
139
134−`zus` is the kelvin version of the required zuse. In general, we assume that any
140+`zus` is the kelvin version of the required zuse. In general, we assume
135−newer (lower-numbered) zuse will retain backward compatibility, so any newer
141+that any newer (lower-numbered) zuse will retain backward compatibility,
136−zuse is also permissible. This number is set with a `/?` at the beginning of the
142+so any newer zuse is also permissible. This number is set with a `/?` at
137−file.
143+the beginning of the file.
138
144
139−`sur` is the set of structures included. These are included with the `/-` rune.
145+`sur` is the set of structures included. These are included with the
140−When a structure is included, we look in `/=main=/sur` for the given structure
146+`/-` rune. When a structure is included, we look in `/=main=/sur` for
141−and we load the gate there. When compiling, all the included structures are
147+the given structure and we load the gate there. When compiling, all the
142−collected into a single core placed in the subject of the body with a `=>`.
148+included structures are collected into a single core placed in the
149+subject of the body with a `=>`.
143
150
144−`lib` is the set of librarires included. These are included with the `/+` rune.
151+`lib` is the set of librarires included. These are included with the
145−When a library is included, we look in `/=main=/lib` for the given library and
152+`/+` rune. When a library is included, we look in `/=main=/lib` for the
146−we load the library there. As with structures, all the included libraries are
153+given library and we load the library there. As with structures, all the
147−collected into a single core placed in the subject of the body with a `=>`.
154+included libraries are collected into a single core placed in the
155+subject of the body with a `=>`.
148
156
149−`fan` is the set of resources included. These are loaded in many different ways
157+`fan` is the set of resources included. These are loaded in many
150−and may load resources from any location. These are placed in the subject of the
158+different ways and may load resources from any location. These are
151−body with a `=~`.
159+placed in the subject of the body with a `=~`.
152
160
153−`src` is the set of twigs or references to twigs in the body of the program.
161+`src` is the set of twigs or references to twigs in the body of the
154−Generally, each of these will represent a core, but this is not required. When
162+program. Generally, each of these will represent a core, but this is not
155−compiling, these are strung together in a `=~`.
163+required. When compiling, these are strung together in a `=~`.
156
164
157### `++hoot`
165### `++hoot`
158
166
159 ++ hoot (pair bean hoof) :: structure gate/core
167 ++ hoot (pair bean hoof) :: structure gate/core
160
168
161−A structures may be either a direct gate or a core. These are syntactically
169+A structures may be either a direct gate or a core. These are
162−distinguished by adding a `*` to the beginning of the structure name for a core.
170+syntactically distinguished by adding a `*` to the beginning of the
163−The structure itself is a `hoof`.
171+structure name for a core. The structure itself is a `hoof`.
164
172
165### `++hoof`
173### `++hoof`
166
174
167 ++ hoof (pair term (unit (pair case ship))) :: resource reference
175 ++ hoof (pair term (unit (pair case ship))) :: resource reference
168
176
169−A hoof, which is either a structure or a library, has a name, and it may also
177+A hoof, which is either a structure or a library, has a name, and it may
170−specify which version of the resource to use and which ship to retrieve it from.
178+also specify which version of the resource to use and which ship to
179+retrieve it from.
171
180
172### `++horn`
181### `++horn`
173
182
@@ -193,42 +202,44 @@ discussion of their use from a user's perspective is documented in the [runes
193documentation](https://urbit.org/docs/arvo/internals/runes), so we will here
202documentation](https://urbit.org/docs/arvo/internals/runes), so we will here
194only give a description of the data structure itself.
203only give a description of the data structure itself.
195
204
196−A `%ape` horn is simply a twig that gets evaluated and placed in the subject.
205+A `%ape` horn is simply a twig that gets evaluated and placed in the
206+subject.
197
207
198−A `%arg` is a gate that gets evaluated with a sample of our location and our
208+A `%arg` is a gate that gets evaluated with a sample of our location and
199−heel.
209+our heel.
200
210
201−A `%day` is a horn that applies to each of a list of `@dr`-named files in the
211+A `%day` is a horn that applies to each of a list of `@dr`-named files
202−directory.
212+in the directory.
203
213
204−A `%dub` is a term and horn, where the result of a horn is given the face of the
214+A `%dub` is a term and horn, where the result of a horn is given the
205−term.
215+face of the term.
206
216
207A `%fan` is a list of horns, all at the current directory level.
217A `%fan` is a list of horns, all at the current directory level.
208
218
209−A `%for` is a path and a horn, where the horn is evaluated relative to the given
219+A `%for` is a path and a horn, where the horn is evaluated relative to
210−path, where the given path is relative to the current location.
220+the given path, where the given path is relative to the current
221+location.
211
222
212−A `%hub` is a horn that applies to each of a list of `@ud`-named files in the
223+A `%hub` is a horn that applies to each of a list of `@ud`-named files
213−directory.
224+in the directory.
214
225
215−A `%man` is a map of spans to horns where the result is a set of each horn
226+A `%man` is a map of spans to horns where the result is a set of each
216−applied to the current directory given the associated face.
227+horn applied to the current directory given the associated face.
217
228
218−A `%nap` is a homogenous map where each entry in a directory is handled with the
229+A `%nap` is a homogenous map where each entry in a directory is handled
219−same horn and is given a face according to its name.
230+with the same horn and is given a face according to its name.
220
231
221−A `%now` is a horn that applies to each of a list of `@da`-named files in the
232+A `%now` is a horn that applies to each of a list of `@da`-named files
222−directory.
233+in the directory.
223
234
224−A `%saw` is a twig and a horn, where the twig operates on the result of the
235+A `%saw` is a twig and a horn, where the twig operates on the result of
225−horn.
236+the horn.
226
237
227−A `%see` is a beam and a horn, where the horn is evaluated at a location of the
238+A `%see` is a beam and a horn, where the horn is evaluated at a location
228−given beam.
239+of the given beam.
229
240
230−A `%sic` is a tile and a horn, where the horn is evaluated and cast to the type
241+A `%sic` is a tile and a horn, where the horn is evaluated and cast to
231−associated with the tile.
242+the type associated with the tile.
232
243
233A `%toy` is simply a mark to be baked.
244A `%toy` is simply a mark to be baked.
234
245
@@ -239,9 +250,9 @@ A `%toy` is simply a mark to be baked.
239 [%| p=beam] :: resource location
250 [%| p=beam] :: resource location
240 == ::
251 == ::
241
252
242−This is an entry in the body of the hook file. The hoop can either be defined
253+This is an entry in the body of the hook file. The hoop can either be
243−directly in the given file or it can be a reference to another file. The second
254+defined directly in the given file or it can be a reference to another
244−is specified with a `//` rune.
255+file. The second is specified with a `//` rune.
245
256
246### `++bolt`, monadic edge
257### `++bolt`, monadic edge
247
258
@@ -255,27 +266,28 @@ is specified with a `//` rune.
255 == ::
266 == ::
256 == ::
267 == ::
257
268
258−Throughout our computation, we let our result flow through with the set of
269+Throughout our computation, we let our result flow through with the set
259−dependencies of the value. At various times, we may wish to either throw an
270+of dependencies of the value. At various times, we may wish to either
260−error or declare that the actual result cannot be found until a particular
271+throw an error or declare that the actual result cannot be found until a
261−resource is retrieved. This is a perfect case for a monad, so here we define a
272+particular resource is retrieved. This is a perfect case for a monad, so
262−data structure for it.
273+here we define a data structure for it.
263
274
264−At every step, we have a cache, so we store that in `p`. In `q` we store the
275+At every step, we have a cache, so we store that in `p`. In `q` we store
265−data.
276+the data.
266
277
267−In the case of `%0`, we have the result in `q` and the set of dependencies in
278+In the case of `%0`, we have the result in `q` and the set of
268−`p`.
279+dependencies in `p`.
269
280
270−In the case of `%1`, we have a set of dependencies on which we are blocking.
281+In the case of `%1`, we have a set of dependencies on which we are
271−When this happens, we make a call to clay to get the dependencies, and we
282+blocking. When this happens, we make a call to clay to get the
272−proceed with the computation when we receive them. Technically, we restart the
283+dependencies, and we proceed with the computation when we receive them.
273−computation, but since every expensive step is cached, there is no significant
284+Technically, we restart the computation, but since every expensive step
274−performance penalty to doing this. Referential transparency has its uses.
285+is cached, there is no significant performance penalty to doing this.
286+Referential transparency has its uses.
275
287
276−In the case of `%2`, we have a hit an error. This gets passed all the way
288+In the case of `%2`, we have a hit an error. This gets passed all the
277−through to the calling duct. The list of tanks is some description of what went
289+way through to the calling duct. The list of tanks is some description
278−wrong, often including a stack trace.
290+of what went wrong, often including a stack trace.
279
291
280### `++burg`, monadic rule
292### `++burg`, monadic rule
281
293
@@ -284,49 +296,53 @@ wrong, often including a stack trace.
284 $+([c=cafe d=a] (bolt b)) ::
296 $+([c=cafe d=a] (bolt b)) ::
285 :: ::
297 :: ::
286
298
287−To operate on bolts, we use `++cope` as our bind operator, and the functions it
299+To operate on bolts, we use `++cope` as our bind operator, and the
288−works on are of type `burg`. Our functions that operate on bolts should have a
300+functions it works on are of type `burg`. Our functions that operate on
289−sample of the cache and a value. Their output should be a bolt of the output
301+bolts should have a sample of the cache and a value. Their output should
290−value. Then, `++cope` will only call the function when necessary (in the `%0`
302+be a bolt of the output value. Then, `++cope` will only call the
291−case), and it will do so without the wrapping of a bolt.
303+function when necessary (in the `%0` case), and it will do so without
304+the wrapping of a bolt.
292
305
293−If you understand monads, this is probably fairly obvious. Otherwise, see the
306+If you understand monads, this is probably fairly obvious. Otherwise,
294−discussion on `++cope` (link).
307+see the discussion on `++cope` (link).
295
308
296−## Public Interface
309+Public Interface
310+----------------
297
311
298−Ford does not export a scry interface, so the only way to interact with ford is
312+Ford does not export a scry interface, so the only way to interact with
299−by sending kisses and receiving gifts. In fact, ford only sends accepts one kiss
313+ford is by sending kisses and receiving gifts. In fact, ford only sends
300−and gives one gift. This is, of course, misleading because ford actually does
314+accepts one kiss and gives one gift. This is, of course, misleading
301−many different things. It does, however, only produce one type of thing -- a
315+because ford actually does many different things. It does, however, only
302−result of a computation, which is either an error or the value produced along
316+produce one type of thing -- a result of a computation, which is either
303−with the set of dependencies referenced by it.
317+an error or the value produced along with the set of dependencies
318+referenced by it.
304
319
305 ++ kiss :: in request ->$
320 ++ kiss :: in request ->$
306 $% [%exec p=@p q=(unit silk)] :: make / kill
321 $% [%exec p=@p q=(unit silk)] :: make / kill
307 == ::
322 == ::
308
323
309The `%exec` gift requests ford to perform a computation on behalf of a
324The `%exec` gift requests ford to perform a computation on behalf of a
310−particular ship. `p` is the ship, and `q` is the computation. If `q` is null,
325+particular ship. `p` is the ship, and `q` is the computation. If `q` is
311−then we are requesting that ford cancel the computation that it is currently
326+null, then we are requesting that ford cancel the computation that it is
312−being run along this duct. Thus, if you wish to cancel a computation, you must
327+currently being run along this duct. Thus, if you wish to cancel a
313−send the kiss along the same duct as the original request.
328+computation, you must send the kiss along the same duct as the original
329+request.
314
330
315−Otherwise, we ask ford to perform a certain computation, as defined in `++silk`.
331+Otherwise, we ask ford to perform a certain computation, as defined in
316−Since all computations produce the same type of result, we will discuss that
332+`++silk`. Since all computations produce the same type of result, we
317−result before we jump into `++silk`.
333+will discuss that result before we jump into `++silk`.
318
334
319 ++ gift :: out result <-$
335 ++ gift :: out result <-$
320 $% [%made p=(each bead (list tank))] :: computed result
336 $% [%made p=(each bead (list tank))] :: computed result
321 == ::
337 == ::
322
338
323−We give either a `bead`, which is a result, or a list of tanks, which is an
339+We give either a `bead`, which is a result, or a list of tanks, which is
324−error messge, often including a stack trace.
340+an error messge, often including a stack trace.
325
341
326 ++ bead ,[p=(set beam) q=cage] :: computed result
342 ++ bead ,[p=(set beam) q=cage] :: computed result
327
343
328−This is a set of dependencies required to compute this value and a cage of the
344+This is a set of dependencies required to compute this value and a cage
329−result with its associated mark.
345+of the result with its associated mark.
330
346
331There are twelve possible computations defined in `++silk`.
347There are twelve possible computations defined in `++silk`.
332
348
@@ -346,77 +362,83 @@ There are twelve possible computations defined in `++silk`.
346 [%vale p=mark q=ship r=*] :: validate [our his]
362 [%vale p=mark q=ship r=*] :: validate [our his]
347 == ::
363 == ::
348
364
349−First, we allow silks to autocons. A cell of silks is also a silk, and the
365+First, we allow silks to autocons. A cell of silks is also a silk, and
350−product vase is a cell of the two silks. This obviously extends to an arbitrary
366+the product vase is a cell of the two silks. This obviously extends to
351−number of silks.
367+an arbitrary number of silks.
352
368
353−`%bake` tries to functionally produce the file at a given beam with the given
369+`%bake` tries to functionally produce the file at a given beam with the
354−mark and heel. It fails if there is no way to translate at this level.
370+given mark and heel. It fails if there is no way to translate at this
371+level.
355
372
356−`%boil` functionally produces the file at a given beam with the given mark and
373+`%boil` functionally produces the file at a given beam with the given
357−heel. If there is no way to translate at this beam, we pop levels off the stack
374+mark and heel. If there is no way to translate at this beam, we pop
358−and attempt to bake there until we find a level we can bake. This should almost
375+levels off the stack and attempt to bake there until we find a level we
359−always be called instead of `%bake`.
376+can bake. This should almost always be called instead of `%bake`.
360
377
361`%call` slams the result of one silk against the result of another.
378`%call` slams the result of one silk against the result of another.
362
379
363−`%cast` translates the given silk to the given mark, if possible. This is one of
380+`%cast` translates the given silk to the given mark, if possible. This
364−the critical and fundamental operations of ford.
381+is one of the critical and fundamental operations of ford.
365
382
366−`%done` produces exactly its input. This is rarely used on its own, but many
383+`%done` produces exactly its input. This is rarely used on its own, but
367−silks are recursively defined in terms of other silks, so we often need a silk
384+many silks are recursively defined in terms of other silks, so we often
368−that simply produces its input. A monadic return, if you will.
385+need a silk that simply produces its input. A monadic return, if you
386+will.
369
387
370−`%diff` diffs the two given silks (which must be of the same mark), producing a
388+`%diff` diffs the two given silks (which must be of the same mark),
371−cage of the mark specified in `++mark` in `++grad` for the mark of the two
389+producing a cage of the mark specified in `++mark` in `++grad` for the
372−silks.
390+mark of the two silks.
373
391
374−`%dude` computes the given silk with the given tank as part of the stack trace
392+`%dude` computes the given silk with the given tank as part of the stack
375−if there is an error.
393+trace if there is an error.
376
394
377−`%dune` produces an error if the cage is empty. Otherwise, it produces the value
395+`%dune` produces an error if the cage is empty. Otherwise, it produces
378−in the unit.
396+the value in the unit.
379
397
380−`%mute` takes a silk and a list of changes to make to the silk. At each wing in
398+`%mute` takes a silk and a list of changes to make to the silk. At each
381−the list we put the value of the associated silk.
399+wing in the list we put the value of the associated silk.
382
400
383−`%pact` applies the second silk as a patch to the first silk. The second silk
401+`%pact` applies the second silk as a patch to the first silk. The second
384−must be of the mark specified in `++mark` in `++grad` for the mark of the first
402+silk must be of the mark specified in `++mark` in `++grad` for the mark
385−silk.
403+of the first silk.
386
404
387−`%plan` performs a structured assembly directly. This is not generally directly
405+`%plan` performs a structured assembly directly. This is not generally
388−useful because several other silks perform supersets of this functionality. We
406+directly useful because several other silks perform supersets of this
389−don't usually have naked hoods outside ford.
407+functionality. We don't usually have naked hoods outside ford.
390
408
391−`%reef` produces a core containing the entirety of zuse and hoon, suitable for
409+`%reef` produces a core containing the entirety of zuse and hoon,
392−running arbitrary code against. The mark is `%noun`.
410+suitable for running arbitrary code against. The mark is `%noun`.
393
411
394−`%ride` slaps a twig against a subject silk. The mark of the result is `%noun`.
412+`%ride` slaps a twig against a subject silk. The mark of the result is
413+`%noun`.
395
414
396−`%vale` validates untyped data from a ship against a given mark. This is an
415+`%vale` validates untyped data from a ship against a given mark. This is
397−extremely useful function.
416+an extremely useful function.
398
417
399−# Commentary
418+Commentary
419+==========
400
420
401−## Parsing Hook Files
421+Parsing Hook Files
422+------------------
402
423
403−In the commentary on other vanes, we have traced through the lifecycle of
424+In the commentary on other vanes, we have traced through the lifecycle
404−various external requests. This is generally a very reasonable order to examine
425+of various external requests. This is generally a very reasonable order
405−vanes since it will eventually cover the entire vane, and we are never left
426+to examine vanes since it will eventually cover the entire vane, and we
406−wondering why we are doing something.
427+are never left wondering why we are doing something.
407
428
408−For ford, however, it makes more sense to begin by discussing the parsing and
429+For ford, however, it makes more sense to begin by discussing the
409−assembliing of hook files. Many of the possible requests require us to assemble
430+parsing and assembliing of hook files. Many of the possible requests
410−hook files, so we may as well examine this immediately.
431+require us to assemble hook files, so we may as well examine this
432+immediately.
411
433
412−First, we will examine the parsing. We parse a file at a beam to a hood in
434+First, we will examine the parsing. We parse a file at a beam to a hood
413−`++fade:zo:za`. The top-level parsing rule is `++fair`, which takes a beam and
435+in `++fade:zo:za`. The top-level parsing rule is `++fair`, which takes a
414−produces a rule to parse an entire hood file.
436+beam and produces a rule to parse an entire hood file.
415
437
416−A note on the naming scheme: the parsing the combinators that parse into a
438+A note on the naming scheme: the parsing the combinators that parse into
417−particular structure are conventionally given the same name as the structure.
439+a particular structure are conventionally given the same name as the
418−Although this locally clobbers the type names, this pattern makes obvious the
440+structure. Although this locally clobbers the type names, this pattern
419−intent of the parsing combinators.
441+makes obvious the intent of the parsing combinators.
420
442
421We kick off with `++hood:fair`.
443We kick off with `++hood:fair`.
422
444
@@ -442,23 +464,24 @@ We kick off with `++hood:fair`.
442 (most gap hoop)
464 (most gap hoop)
443 ==
465 ==
444
466
445−There are five sections to a hood: system version, structures, libraries,
467+There are five sections to a hood: system version, structures,
446−resources, and body.
468+libraries, resources, and body.
447
469
448−First, we parse the requested version number of the system. This is specified
470+First, we parse the requested version number of the system. This is
449−with a unary `/?` rune. If not present, then we default to the current version.
471+specified with a unary `/?` rune. If not present, then we default to the
472+current version.
450
473
451−Second, we may have zero or more `/-` runes followed by a parsing of a `++hoot`,
474+Second, we may have zero or more `/-` runes followed by a parsing of a
452−which represents a shared structure.
475+`++hoot`, which represents a shared structure.
453
476
454−Third, we may have zero or more `/+` runes followed by a parsing of a `++hoof`,
477+Third, we may have zero or more `/+` runes followed by a parsing of a
455−which represents a shared library.
478+`++hoof`, which represents a shared library.
456
479
457−Fourth, we may have zero or more other `/` runes (as described in `++horn`),
480+Fourth, we may have zero or more other `/` runes (as described in
458−which represent program-specific resources to be loaded.
481+`++horn`), which represent program-specific resources to be loaded.
459
482
460−Fifth and finally, we must have one or more body statements (hoops), which are
483+Fifth and finally, we must have one or more body statements (hoops),
461−either direct twigs or `//` runes.
484+which are either direct twigs or `//` runes.
462
485
463 ++ hoot
486 ++ hoot
464 ;~ pose
487 ;~ pose
@@ -466,9 +489,9 @@ either direct twigs or `//` runes.
466 (stag %& hoof)
489 (stag %& hoof)
467 ==
490 ==
468
491
469−A structure can either be a direct gate, or it can be a simple core. Either one
492+A structure can either be a direct gate, or it can be a simple core.
470−is parsed with `++hoof`, so we distinguish the two cases by requireing core
493+Either one is parsed with `++hoof`, so we distinguish the two cases by
471−references to be prefixed by a `*`.
494+requireing core references to be prefixed by a `*`.
472
495
473 ++ hoof
496 ++ hoof
474 %+ cook |=(a=^hoof a)
497 %+ cook |=(a=^hoof a)
@@ -481,11 +504,11 @@ references to be prefixed by a `*`.
481 ==
504 ==
482 ==
505 ==
483
506
484−A hoof must have a name, which is a term. Optionally, we also include a case and
507+A hoof must have a name, which is a term. Optionally, we also include a
485−a ship. This is marked by appending a `/` followed by a case to denote the
508+case and a ship. This is marked by appending a `/` followed by a case to
486−requested version of the resource and a `/` followed by a ship name to denote
509+denote the requested version of the resource and a `/` followed by a
487−the requested source of the resource. For example, `resource/1/~zod` requests
510+ship name to denote the requested source of the resource. For example,
488−the first version of `resource` on `~zod`.
511+`resource/1/~zod` requests the first version of `resource` on `~zod`.
489
512
490 ++ case
513 ++ case
491 %- sear
514 %- sear
@@ -494,11 +517,11 @@ the first version of `resource` on `~zod`.
494 ?. ?=([%$ ?(%da %ud %tas) *] a) ~
517 ?. ?=([%$ ?(%da %ud %tas) *] a) ~
495 [~ u=(^case a)]
518 [~ u=(^case a)]
496
519
497−Here, we parse a literal with `++nuck:so`, and we accept the input if it is
520+Here, we parse a literal with `++nuck:so`, and we accept the input if it
498−either an absolute date, an unsigned decimal, or a label.
521+is either an absolute date, an unsigned decimal, or a label.
499
522
500−This leaves only horns and hoops to parse. Hoops are much simple to parse, so
523+This leaves only horns and hoops to parse. Hoops are much simple to
501−we'll discuss those first.
524+parse, so we'll discuss those first.
502
525
503 ++ hoop
526 ++ hoop
504 ;~ pose
527 ;~ pose
@@ -506,20 +529,20 @@ we'll discuss those first.
506 (stag %& tall:vez)
529 (stag %& tall:vez)
507 ==
530 ==
508
531
509−There are two types of hoops. Direct twigs are parsed with `++tall:vast`, which
532+There are two types of hoops. Direct twigs are parsed with
510−is the just the hoon parser for a tall-form twig.
533+`++tall:vast`, which is the just the hoon parser for a tall-form twig.
511
534
512−References to external twigs are marked with a `//` rune followed by a beam,
535+References to external twigs are marked with a `//` rune followed by a
513−which is parsed with `++have`.
536+beam, which is parsed with `++have`.
514
537
515 ++ hath (sear plex:voz (stag %clsg poor:voz)) :: hood path
538 ++ hath (sear plex:voz (stag %clsg poor:voz)) :: hood path
516 ++ have (sear tome ;~(pfix fas hath)) :: hood beam
539 ++ have (sear tome ;~(pfix fas hath)) :: hood beam
517
540
518−`++have` parses a path with `++hath`, and then it converts the path into a beam
541+`++have` parses a path with `++hath`, and then it converts the path into
519−with `++tome`.
542+a beam with `++tome`.
520
543
521−`++hath` parses a `/`-separated list with `++poor:vast`, then converts it to an
544+`++hath` parses a `/`-separated list with `++poor:vast`, then converts
522−actual path with `++plex:vast`.
545+it to an actual path with `++plex:vast`.
523
546
524This leaves only horns to parse.
547This leaves only horns to parse.
525
548
@@ -548,17 +571,18 @@ This leaves only horns to parse.
548 ==
571 ==
549 ==
572 ==
550
573
551−Horn parsing is slightly complex, so we create an internal core to organize our
574+Horn parsing is slightly complex, so we create an internal core to
552−code. Our core has a global variable of `tol`, which is true if tall form is
575+organize our code. Our core has a global variable of `tol`, which is
553−permissible and false if we're already in wide form. We kick off the parsing
576+true if tall form is permissible and false if we're already in wide
554−with `++apex`.
577+form. We kick off the parsing with `++apex`.
555
578
556−`++apex` specifies how each rune is parsed. This allows us to offload the
579+`++apex` specifies how each rune is parsed. This allows us to offload
557−different ways of parsing the arguments to these runes into separate arms. The
580+the different ways of parsing the arguments to these runes into separate
558−exception here is that the `%toy` horn is simply of the form `/mark/`.
581+arms. The exception here is that the `%toy` horn is simply of the form
582+`/mark/`.
559
583
560−We'll examine each of the horn parsing arms right after we discuss `++rail`,
584+We'll examine each of the horn parsing arms right after we discuss
561−which is used in each one.
585+`++rail`, which is used in each one.
562
586
563 ++ rail
587 ++ rail
564 |* [wid=_rule tal=_rule]
588 |* [wid=_rule tal=_rule]
@@ -566,8 +590,8 @@ which is used in each one.
566 ;~(pose wid tal)
590 ;~(pose wid tal)
567
591
568This takes a wide-form and a tall-form parsing rule. If tall form is
592This takes a wide-form and a tall-form parsing rule. If tall form is
569−permissible, then it allows either rule to match; else, it allows only the wide
593+permissible, then it allows either rule to match; else, it allows only
570−form rule.
594+the wide form rule.
571
595
572 ++ read
596 ++ read
573 |% ++ ape
597 |% ++ ape
@@ -575,47 +599,49 @@ form rule.
575 (ifix [sel ser] (stag %cltr (most ace wide:vez)))
599 (ifix [sel ser] (stag %cltr (most ace wide:vez)))
576 ;~(pfix gap tall:vez)
600 ;~(pfix gap tall:vez)
577
601
578−`++ape:read` parses for both the `/~` and the `/$` runes. It produces a twig.
602+`++ape:read` parses for both the `/~` and the `/$` runes. It produces a
579−The wide form is a tuple of one or more ace-separated wide-form twigs parsed
603+twig. The wide form is a tuple of one or more ace-separated wide-form
580−with `++wide:vast` and surrounded by `[` and `]`. The tall form is a single tall
604+twigs parsed with `++wide:vast` and surrounded by `[` and `]`. The tall
581−form twig parsed by `++tall:vast`
605+form is a single tall form twig parsed by `++tall:vast`
582
606
583 ++ day
607 ++ day
584 %+ rail
608 %+ rail
585 apex(tol |)
609 apex(tol |)
586 ;~(pfix gap apex)
610 ;~(pfix gap apex)
587
611
588−This parses for the `/|`, `/@`, `/%`, and `/&` runes. It produces a horn. The
612+This parses for the `/|`, `/@`, `/%`, and `/&` runes. It produces a
589−wide form is, recursively, the entire horn parser with tall form disabled. The
613+horn. The wide form is, recursively, the entire horn parser with tall
590−tall form is a gap followed by, recursively, the entire horn parser.
614+form disabled. The tall form is a gap followed by, recursively, the
615+entire horn parser.
591
616
592 ++ dub
617 ++ dub
593 %+ rail
618 %+ rail
594 ;~(plug sym ;~(pfix tis apex(tol |)))
619 ;~(plug sym ;~(pfix tis apex(tol |)))
595 ;~(pfix gap ;~(plug sym ;~(pfix gap apex)))
620 ;~(pfix gap ;~(plug sym ;~(pfix gap apex)))
596
621
597−This parses for the `/=` rune. It produces a term followed by a horn. The wide
622+This parses for the `/=` rune. It produces a term followed by a horn.
598−form is a symbol name followed by a `=` and, recursively, the entire horn parser
623+The wide form is a symbol name followed by a `=` and, recursively, the
599−with tall form disabled. The tall form is a gap followed by a symbol name,
624+entire horn parser with tall form disabled. The tall form is a gap
600−another gap, and, recursively, the entire horn parser.
625+followed by a symbol name, another gap, and, recursively, the entire
626+horn parser.
601
627
602 ++ fan
628 ++ fan
603 %+ rail fail
629 %+ rail fail
604 ;~(sfix (star ;~(pfix gap apex)) ;~(plug gap duz))
630 ;~(sfix (star ;~(pfix gap apex)) ;~(plug gap duz))
605
631
606−This parses for the `/.` rune. It produces a list of horns. There is no wide
632+This parses for the `/.` rune. It produces a list of horns. There is no
607−form. The tall form is a stet-terminated series of gap-separated recursive calls
633+wide form. The tall form is a stet-terminated series of gap-separated
608−to the entire horn parser.
634+recursive calls to the entire horn parser.
609
635
610 ++ for
636 ++ for
611 %+ rail
637 %+ rail
612 ;~(plug (ifix [sel ser] hath) apex(tol |))
638 ;~(plug (ifix [sel ser] hath) apex(tol |))
613 ;~(pfix gap ;~(plug hath ;~(pfix gap apex)))
639 ;~(pfix gap ;~(plug hath ;~(pfix gap apex)))
614
640
615−This parses for the `/,` rune. It produces a path and a horn. The wide form is a
641+This parses for the `/,` rune. It produces a path and a horn. The wide
616−`[`-`]`-surrounded path followed by, recursively, the entire horn parser with
642+form is a `[`-`]`-surrounded path followed by, recursively, the entire
617−tall form disabled. The tall form is a gap followed by a path, another gap, and,
643+horn parser with tall form disabled. The tall form is a gap followed by
618−recursively, the entire horn parser.
644+a path, another gap, and, recursively, the entire horn parser.
619
645
620 ++ man
646 ++ man
621 %+ rail fail
647 %+ rail fail
@@ -628,31 +654,31 @@ recursively, the entire horn parser.
628 ?. ?=(%dub -.i.fan) ~
654 ?. ?=(%dub -.i.fan) ~
629 $(fan t.fan, naf [[p.i.fan q.i.fan] naf])
655 $(fan t.fan, naf [[p.i.fan q.i.fan] naf])
630
656
631−This parses for the `/*` rune. It produces a map of spans to horns. There is no
657+This parses for the `/*` rune. It produces a map of spans to horns.
632−wide form. The tall form is a stet-terminated series of gap-separated recursive
658+There is no wide form. The tall form is a stet-terminated series of
633−calls to the entire horn parser. All produced horns are expected to be from `/=`
659+gap-separated recursive calls to the entire horn parser. All produced
634−runes. The term and horn in each `/=` horn is inserted into the produced map as
660+horns are expected to be from `/=` runes. The term and horn in each `/=`
635−a key-value pair.
661+horn is inserted into the produced map as a key-value pair.
636
662
637 ++ saw
663 ++ saw
638 %+ rail
664 %+ rail
639 ;~(plug ;~(sfix wide:vez sem) apex(tol |))
665 ;~(plug ;~(sfix wide:vez sem) apex(tol |))
640 ;~(pfix gap ;~(plug tall:vez ;~(pfix gap apex)))
666 ;~(pfix gap ;~(plug tall:vez ;~(pfix gap apex)))
641
667
642−This parses for the `/;` rune. It produces a twig and a horn. The wide form is a
668+This parses for the `/;` rune. It produces a twig and a horn. The wide
643−wide-form twig followed by a `;` and, recursively, the entire horn parser with
669+form is a wide-form twig followed by a `;` and, recursively, the entire
644−tall form disabled. The tall form is a gap followed by a tall-form twig, another
670+horn parser with tall form disabled. The tall form is a gap followed by
645−gap, and, recursively, the entire horn parser.
671+a tall-form twig, another gap, and, recursively, the entire horn parser.
646
672
647 ++ see
673 ++ see
648 %+ rail
674 %+ rail
649 ;~(plug ;~(sfix have col) apex(tol |))
675 ;~(plug ;~(sfix have col) apex(tol |))
650 ;~(pfix gap ;~(plug have ;~(pfix gap apex)))
676 ;~(pfix gap ;~(plug have ;~(pfix gap apex)))
651
677
652−This parses for the `/:` rune. It produces a beam and a horn. The wide form is a
678+This parses for the `/:` rune. It produces a beam and a horn. The wide
653−beam followed by a `;` and, recursively, the entire horn parser with tall form
679+form is a beam followed by a `;` and, recursively, the entire horn
654−disabled. The tall form is a gap followed by a beam, another gap, and,
680+parser with tall form disabled. The tall form is a gap followed by a
655−recursively, the entire horn parser.
681+beam, another gap, and, recursively, the entire horn parser.
656
682
657 ++ sic
683 ++ sic
658 %+ rail
684 %+ rail
@@ -660,17 +686,18 @@ recursively, the entire horn parser.
660 ;~(pfix gap ;~(plug howl:vez ;~(pfix gap apex)))
686 ;~(pfix gap ;~(plug howl:vez ;~(pfix gap apex)))
661 --
687 --
662
688
663−This parses for the `/^` rune. It produces a tile and a horn. The wide form is a
689+This parses for the `/^` rune. It produces a tile and a horn. The wide
664−wide-form tile, parsed with `++toil:vast`, followed by a `^` and, recursively,
690+form is a wide-form tile, parsed with `++toil:vast`, followed by a `^`
665−the entire horn parser with tall form disabled. The tall form is a gap followed
691+and, recursively, the entire horn parser with tall form disabled. The
666−by a tall-form tile, parsed with `++howl:vast`, another gap, and, recursively,
692+tall form is a gap followed by a tall-form tile, parsed with
667−the entire horn parser.
693+`++howl:vast`, another gap, and, recursively, the entire horn parser.
668
694
669−## Assembling Hook Files
695+Assembling Hook Files
696+---------------------
670
697
671−At this point, we've parsed a hook file into a hood. We will now describe
698+At this point, we've parsed a hook file into a hood. We will now
672−exactly how this hood is assembled into a vase. The problem of assembling is
699+describe exactly how this hood is assembled into a vase. The problem of
673−handled entirely within the `++meow:zo:za` core.
700+assembling is handled entirely within the `++meow:zo:za` core.
674
701
675 ++ meow :: assemble
702 ++ meow :: assemble
676 |= [how=beam arg=heel]
703 |= [how=beam arg=heel]
@@ -684,45 +711,47 @@ handled entirely within the `++meow:zo:za` core.
684 ==
711 ==
685 |%
712 |%
686
713
687−We take two arguments and keep seven pieces of state. `how` is the location of
714+We take two arguments and keep seven pieces of state. `how` is the
688−the hook file we're assembling, and `arg` is the heel, or virtual path
715+location of the hook file we're assembling, and `arg` is the heel, or
689−extension, of the file.
716+virtual path extension, of the file.
690
717
691−In `rop`, we maintain a map of terms to pairs of hooves and twigs to represent
718+In `rop`, we maintain a map of terms to pairs of hooves and twigs to
692−the structures we've encountered that we will put together in a core at the top
719+represent the structures we've encountered that we will put together in
693−of the file.
720+a core at the top of the file.
694
721
695−In `zog`, we maintain the set of structures we're in the middle of loading. If
722+In `zog`, we maintain the set of structures we're in the middle of
696−we try to load a structure already in our dependency ancestry, then we fail
723+loading. If we try to load a structure already in our dependency
697−because we do not allow circular dependencies. This enforces that our structure
724+ancestry, then we fail because we do not allow circular dependencies.
698−dependency graph is a DAG.
725+This enforces that our structure dependency graph is a DAG.
699
726
700−In `bil`, we maintain a map of terms to pairs of hooves and twigs to represent
727+In `bil`, we maintain a map of terms to pairs of hooves and twigs to
701−the libraries we've encountered that we will put together in a series of cores
728+represent the libraries we've encountered that we will put together in a
702−after the structure core.
729+series of cores after the structure core.
703
730
704−In `lot`, we maintain a stack of library names as they are encountered during a
731+In `lot`, we maintain a stack of library names as they are encountered
705−depth-first search. More precisely, we push a library onto the stack after we've
732+during a depth-first search. More precisely, we push a library onto the
706−processed all its children. Thus, every library depends only on things deeper in
733+stack after we've processed all its children. Thus, every library
707−the list. The libraries must be loaded in the reverse of this order. Concisely,
734+depends only on things deeper in the list. The libraries must be loaded
708−this is a topological sort of the library dependency partial ordering.
735+in the reverse of this order. Concisely, this is a topological sort of
736+the library dependency partial ordering.
709
737
710−In `zeg`, we maintain the set of libraries we're in the middle of loading. If we
738+In `zeg`, we maintain the set of libraries we're in the middle of
711−try to load a library already in our dependency ancestry, then we fail because
739+loading. If we try to load a library already in our dependency ancestry,
712−we do not allow circular dependencies. This enforces that our library dependency
740+then we fail because we do not allow circular dependencies. This
713−graph is a DAG.
741+enforces that our library dependency graph is a DAG.
714
742
715−In `boy`, we maintain a stack of body twigs, which we'll put together in a
743+In `boy`, we maintain a stack of body twigs, which we'll put together in
716−series of cores at the end of the file.
744+a series of cores at the end of the file.
717
745
718−In `hol`, we decide if we're allowed to contain horns. Libraries and structures
746+In `hol`, we decide if we're allowed to contain horns. Libraries and
719−are not allowed to contain horns.
747+structures are not allowed to contain horns.
720
748
721−We in every case enter `++meow` through `++abut`. You'll notice that there are
749+We in every case enter `++meow` through `++abut`. You'll notice that
722−four (count 'em, four!) calls to `++cope` in `++abut`. If you've glanced at the
750+there are four (count 'em, four!) calls to `++cope` in `++abut`. If
723−ford code in general, you've probably seen cope over and over. It is called in
751+you've glanced at the ford code in general, you've probably seen cope
724−79 different places. We need to discuss the use of this critical function in
752+over and over. It is called in 79 different places. We need to discuss
725−detail, so we may as well do it here.
753+the use of this critical function in detail, so we may as well do it
754+here.
726
755
727 ++ cope :: bolt along
756 ++ cope :: bolt along
728 |* [hoc=(bolt) fun=(burg)]
757 |* [hoc=(bolt) fun=(burg)]
@@ -738,36 +767,40 @@ detail, so we may as well do it here.
738 %0 [%0 p=(grom `_p.q.nuf`p.q.hoc p.q.nuf) q=q.q.nuf]
767 %0 [%0 p=(grom `_p.q.nuf`p.q.hoc p.q.nuf) q=q.q.nuf]
739 == ==
768 == ==
740
769
741−In monad-speak, this is the bind operator for the bolt monad. If monads aren't
770+In monad-speak, this is the bind operator for the bolt monad. If monads
742−your thing, don't worry, we're going to explain the use of cope without further
771+aren't your thing, don't worry, we're going to explain the use of cope
743−reference to them.
772+without further reference to them.
744−
773+
745−Recall that there are three different types of bolt. A `%2` error bolt contains
774+Recall that there are three different types of bolt. A `%2` error bolt
746−a list of tanks describing the error, a `%1` block bolt contains a set of
775+contains a list of tanks describing the error, a `%1` block bolt
747−resources we're blocked on, and a `%0` value bolt contains an actual value and
776+contains a set of resources we're blocked on, and a `%0` value bolt
748−the set of its dependencies.
777+contains an actual value and the set of its dependencies.
749−
778+
750−We most commonly want to perform an operation on the value in a bolt if it is a
779+We most commonly want to perform an operation on the value in a bolt if
751−`%0` bolt. If it's not a `%0` bolt, we want to leave it alone. This requires us
780+it is a `%0` bolt. If it's not a `%0` bolt, we want to leave it alone.
752−to write a certain amount of boilerplate between each of our operations to see
781+This requires us to write a certain amount of boilerplate between each
753−if any of them produced a `%1` or a `%2` bolt. This gets tiresome, so we pull it
782+of our operations to see if any of them produced a `%1` or a `%2` bolt.
754−out into a separate arm and call it `++cope`.
783+This gets tiresome, so we pull it out into a separate arm and call it
755−
784+`++cope`.
756−Intuitively, we're calling the function `fun` with the value in `hoc`, where
785+
757−`fun` takes an argument of type whatever is the value in a `%0` case of `hoc`,
786+Intuitively, we're calling the function `fun` with the value in `hoc`,
758−and it produces a bolt of some (possibly different) type. For brevity, we will
787+where `fun` takes an argument of type whatever is the value in a `%0`
759−refer to the type of the of the value in the `%0` case of a bolt as the "type of
788+case of `hoc`, and it produces a bolt of some (possibly different) type.
760−the bolt".
789+For brevity, we will refer to the type of the of the value in the `%0`
761−
790+case of a bolt as the "type of the bolt".
762−If the `hoc` bolt we're given as input to `fun` is already a `%1` or a `%2`
791+
763−bolt, then we simply produce that. We don't even try to run `fun` on it.
792+If the `hoc` bolt we're given as input to `fun` is already a `%1` or a
764−
793+`%2` bolt, then we simply produce that. We don't even try to run `fun`
765−Otherwise, we run `fun` with the arguments from the bolt and, if it produces a
794+on it.
766−`%1` or a `%2` bolt, we simply produce that. If it produces a `%0` bolt, then we
795+
767−produce that with the old set of dependencies merged in with the new set.
796+Otherwise, we run `fun` with the arguments from the bolt and, if it
768−
797+produces a `%1` or a `%2` bolt, we simply produce that. If it produces a
769−We'll see more about how the bolt monad works as we run into more interesting
798+`%0` bolt, then we produce that with the old set of dependencies merged
770−uses of it. For now, this is sufficient to move on with `++abut`.
799+in with the new set.
800+
801+We'll see more about how the bolt monad works as we run into more
802+interesting uses of it. For now, this is sufficient to move on with
803+`++abut`.
771
804
772 ++ abut :: generate
805 ++ abut :: generate
773 |= [cof=cafe hyd=hood]
806 |= [cof=cafe hyd=hood]
@@ -783,17 +816,18 @@ uses of it. For now, this is sufficient to move on with `++abut`.
783 |= [cof=cafe fin=vase]
816 |= [cof=cafe fin=vase]
784 (fine cof fin)
817 (fine cof fin)
785
818
786−Our job is simple: we must assemble a hood file into a vase. Hopefully, the
819+Our job is simple: we must assemble a hood file into a vase. Hopefully,
787−usage of `++cope` is fairly understandable. The correct way to read this is that
820+the usage of `++cope` is fairly understandable. The correct way to read
788−it does essentially five things.
821+this is that it does essentially five things.
789
822
790−First, we call `++apex` to process the structures, libraries, and body. This
823+First, we call `++apex` to process the structures, libraries, and body.
791−changes our state, so we set our context to the produced context. Second, we
824+This changes our state, so we set our context to the produced context.
792−call `++able` to assemble the strucutres and libraries into a twig, which we
825+Second, we call `++able` to assemble the strucutres and libraries into a
793−slap against zuse with `++maim`. Third, we call `++chap` to process the
826+twig, which we slap against zuse with `++maim`. Third, we call `++chap`
794−resources in the context of the already-loaded structures and libraries. Fourth,
827+to process the resources in the context of the already-loaded structures
795−we slap the body against the structures, libraries, and resources. Fifth and
828+and libraries. Fourth, we slap the body against the structures,
796−finally, we produce the resultant vase.
829+libraries, and resources. Fifth and finally, we produce the resultant
830+vase.
797
831
798 ++ apex :: build to body
832 ++ apex :: build to body
799 |= [cof=cafe hyd=hood]
833 |= [cof=cafe hyd=hood]
@@ -818,9 +852,9 @@ Otherwise, we produce and error with `++flaw`.
818
852
819This produces a `%2` error bolt from a list of tanks. Fairly trivial.
853This produces a `%2` error bolt from a list of tanks. Fairly trivial.
820
854
821−We should be starting to get used to the cope syntax, so we can see that we
855+We should be starting to get used to the cope syntax, so we can see that
822−really only do three things here. We process the body with `++body`, the
856+we really only do three things here. We process the body with `++body`,
823−libraries with `++neck`, and the structures with `++head`.
857+the libraries with `++neck`, and the structures with `++head`.
824
858
825 ++ body :: produce functions
859 ++ body :: produce functions
826 |= [cof=cafe src=(list hoop)]
860 |= [cof=cafe src=(list hoop)]
@@ -830,19 +864,19 @@ libraries with `++neck`, and the structures with `++head`.
830 |= [cof=cafe sel=_..body]
864 |= [cof=cafe sel=_..body]
831 ^$(cof cof, src t.src, ..body sel)
865 ^$(cof cof, src t.src, ..body sel)
832
866
833−We must process a list of hoops that represent our body. If there are no more
867+We must process a list of hoops that represent our body. If there are no
834−hoops, we just produce our context in a `%0` bolt with `++fine`.
868+more hoops, we just produce our context in a `%0` bolt with `++fine`.
835
869
836 ++ fine |* [a=cafe b=*] :: bolt from data
870 ++ fine |* [a=cafe b=*] :: bolt from data
837 [p=`cafe`a q=[%0 p=*(set beam) q=b]] ::
871 [p=`cafe`a q=[%0 p=*(set beam) q=b]] ::
838
872
839−In monad-speak, this is the return operator. For us, this just means that we're
873+In monad-speak, this is the return operator. For us, this just means
840−producing a `%0` bolt, which contains a path and a set of dependencies. We
874+that we're producing a `%0` bolt, which contains a path and a set of
841−assume there are no dependencies for the given data, or that they will be added
875+dependencies. We assume there are no dependencies for the given data, or
842−later.
876+that they will be added later.
843
877
844−If there are more hoops in `++body`, we call `++wilt` to process an individual
878+If there are more hoops in `++body`, we call `++wilt` to process an
845−hoop and recurse.
879+individual hoop and recurse.
846
880
847 ++ wilt :: process body entry
881 ++ wilt :: process body entry
848 |= [cof=cafe hop=hoop]
882 |= [cof=cafe hop=hoop]
@@ -882,22 +916,24 @@ hoop and recurse.
882 (fine cof sel(boy [[%brcn mav] boy]))
916 (fine cof sel(boy [[%brcn mav] boy]))
883 ==
917 ==
884
918
885−In the case of a direct twig hoop, we just push it onto `boy` and we're done. In
919+In the case of a direct twig hoop, we just push it onto `boy` and we're
886−the case of an indirect hoop, we must compile the referenced file.
920+done. In the case of an indirect hoop, we must compile the referenced
921+file.
887
922
888−First, we push onto the stack trace a message indicating which file exactly
923+First, we push onto the stack trace a message indicating which file
889−we're compiling at the moment with `++cool`.
924+exactly we're compiling at the moment with `++cool`.
890
925
891 ++ cool :: error caption
926 ++ cool :: error caption
892 |* [cyt=trap hoc=(bolt)]
927 |* [cyt=trap hoc=(bolt)]
893 ?. ?=(%2 -.q.hoc) hoc
928 ?. ?=(%2 -.q.hoc) hoc
894 [p.hoc [%2 *cyt p.q.hoc]]
929 [p.hoc [%2 *cyt p.q.hoc]]
895
930
896−If an error occurred in computing `hoc`, we put the bunt of `cyt` onto the stack
931+If an error occurred in computing `hoc`, we put the bunt of `cyt` onto
897−trace. Thus, `cyt` is not evaluated at all unless an error occurred.
932+the stack trace. Thus, `cyt` is not evaluated at all unless an error
933+occurred.
898
934
899−Next in `++wilt`, we load the information about the filesystem node referenced
935+Next in `++wilt`, we load the information about the filesystem node
900−by the hoop with `++lend`.
936+referenced by the hoop with `++lend`.
901
937
902 ++ lend :: load arch
938 ++ lend :: load arch
903 |= [cof=cafe bem=beam]
939 |= [cof=cafe bem=beam]
@@ -906,27 +942,28 @@ by the hoop with `++lend`.
906 ?~ von [p=cof q=[%1 [bem ~] ~ ~]]
942 ?~ von [p=cof q=[%1 [bem ~] ~ ~]]
907 (fine cof ((hard arch) (need u.von)))
943 (fine cof ((hard arch) (need u.von)))
908
944
909−This is a simple call to the namespace. If the resource does not yet exist, we
945+This is a simple call to the namespace. If the resource does not yet
910−block on it by producing a `%1` bolt. Otherwise, we cast it to an arch and
946+exist, we block on it by producing a `%1` bolt. Otherwise, we cast it to
911−produce this.
947+an arch and produce this.
912
948
913Continuing in `++wilt`, we examine the produced arch. If the referenced
949Continuing in `++wilt`, we examine the produced arch. If the referenced
914−filesystem node has a `hoon` child node, then we've found the required source,
950+filesystem node has a `hoon` child node, then we've found the required
915−so we parse it with `++fade`. Recall that we referred earlier to `++fade`. The
951+source, so we parse it with `++fade`. Recall that we referred earlier to
916−salient point there is that it takes a beam, reads in the hook file there, and
952+`++fade`. The salient point there is that it takes a beam, reads in the
917−parses it into a hood file with `++fair`.
953+hook file there, and parses it into a hood file with `++fair`.
918
954
919−Now, we simply recurse on `++apex` to compile the new hood. Note that, while we
955+Now, we simply recurse on `++apex` to compile the new hood. Note that,
920−do clear the `boy` list, we do not clear the other lists. Thus, we are
956+while we do clear the `boy` list, we do not clear the other lists. Thus,
921−accumulating all the structures and libraries referenced in all the referenced
957+we are accumulating all the structures and libraries referenced in all
922−hook files in one group, which we will put at the top of the product.
958+the referenced hook files in one group, which we will put at the top of
959+the product.
923
960
924−After this, we put the new list of body twigs into a `=~`, push this onto our
961+After this, we put the new list of body twigs into a `=~`, push this
925−old list of body twigs, and produce the result.
962+onto our old list of body twigs, and produce the result.
926
963
927−If there is no hoon file here, then we descend into each of our children until
964+If there is no hoon file here, then we descend into each of our children
928−we find a hoon file. First, we produce a list of all our children whose names
965+until we find a hoon file. First, we produce a list of all our children
929−are terms with `++lark`.
966+whose names are terms with `++lark`.
930
967
931 ++ lark :: filter arch names
968 ++ lark :: filter arch names
932 |= [wox=$+(span (unit ,@)) arc=arch]
969 |= [wox=$+(span (unit ,@)) arc=arch]
@@ -939,39 +976,43 @@ are terms with `++lark`.
939 =+ gib=(wox p.n.r.arc)
976 =+ gib=(wox p.n.r.arc)
940 ?~(gib rac [[u.gib p.n.r.arc] rac])
977 ?~(gib rac [[u.gib p.n.r.arc] rac])
941
978
942−We traverse the children map of `arc` to filter out those children whose names
979+We traverse the children map of `arc` to filter out those children whose
943−aren't accepted by `wox` and produce a map from the product of `wox` to the
980+names aren't accepted by `wox` and produce a map from the product of
944−original name. `++lark` is used in many cases to parse names into other types,
981+`wox` to the original name. `++lark` is used in many cases to parse
945−like numbers or dates, ignoring those which do not fit the format. In `++wilt`,
982+names into other types, like numbers or dates, ignoring those which do
946−though, we simply want to filter out those children whose names are not terms.
983+not fit the format. In `++wilt`, though, we simply want to filter out
984+those children whose names are not terms.
947
985
948−Next, we will produce a map from terms to feet. Each of these feet will be
986+Next, we will produce a map from terms to feet. Each of these feet will
949−placed in a core named by the child name, and it will contain arms according to
987+be placed in a core named by the child name, and it will contain arms
950−its children. Thus, if the indirect hoop references `/path`, then to access the
988+according to its children. Thus, if the indirect hoop references
951−twig defined in `/path/to/twig/hoon`, our body must refer to `twig:to`.
989+`/path`, then to access the twig defined in `/path/to/twig/hoon`, our
990+body must refer to `twig:to`.
952
991
953−If there are no more children, then we are done, so we produce our current
992+If there are no more children, then we are done, so we produce our
954−context.
993+current context.
955
994
956−Else, we recurse into the left and right sides of our map. Finally, we process
995+Else, we recurse into the left and right sides of our map. Finally, we
957−our current entry in the map. We first recurse by calling `++wilt` one level
996+process our current entry in the map. We first recurse by calling
958−down. Thus, in the previous example, the first time we get to this point we are
997+`++wilt` one level down. Thus, in the previous example, the first time
959−processing `/path`, so we recurse on `++wilt` with path `/path/to`. We also
998+we get to this point we are processing `/path`, so we recurse on
960−remove our current body from the recursion, so that we may add it back in later
999+`++wilt` with path `/path/to`. We also remove our current body from the
961−the way we want to.
1000+recursion, so that we may add it back in later the way we want to.
962
1001
963−After recursing, we push the new body onto our map, keyed by its name. We also
1002+After recursing, we push the new body onto our map, keyed by its name.
964−produce the new context so that all external structures, libraries, and
1003+We also produce the new context so that all external structures,
965−resources are collected into the same place.
1004+libraries, and resources are collected into the same place.
966
1005
967−Finally, we have a map of names to feet. If this map is empty, then there were
1006+Finally, we have a map of names to feet. If this map is empty, then
968−no twigs at the requested path, so we give an error with `++flaw`.
1007+there were no twigs at the requested path, so we give an error with
1008+`++flaw`.
969
1009
970−If the map is nonempty, then we finally produce our context with with one thing
1010+If the map is nonempty, then we finally produce our context with with
971−pushed onto the front: a core made out of the map we just produced.
1011+one thing pushed onto the front: a core made out of the map we just
1012+produced.
972
1013
973−This concludes our discussion of `++wilt` and `++body`. Thus, it remains in
1014+This concludes our discussion of `++wilt` and `++body`. Thus, it remains
974−`++apex` to discuss `++neck` and `++head`.
1015+in `++apex` to discuss `++neck` and `++head`.
975
1016
976 ++ neck :: consume libraries
1017 ++ neck :: consume libraries
977 |= [cof=cafe bir=(list hoof)]
1018 |= [cof=cafe bir=(list hoof)]
@@ -999,24 +1040,26 @@ This concludes our discussion of `++wilt` and `++body`. Thus, it remains in
999 ==
1040 ==
1000 ^^$(cof cof, bir t.bir)
1041 ^^$(cof cof, bir t.bir)
1001
1042
1002−Here, we're going to consume the list of libraries and place them in `bil`. If
1043+Here, we're going to consume the list of libraries and place them in
1003−there are no more libraries, we're done, so we just produce our current context.
1044+`bil`. If there are no more libraries, we're done, so we just produce
1045+our current context.
1004
1046
1005−Otherwise, we check to see if the next library in the list is in `zeg`. If so,
1047+Otherwise, we check to see if the next library in the list is in `zeg`.
1006−then this library is one of the libraries that we're already in the middle of
1048+If so, then this library is one of the libraries that we're already in
1007−compiling. There is a circular dependency, so we fail.
1049+the middle of compiling. There is a circular dependency, so we fail.
1008
1050
1009Otherwise, we let `gez` be `zeg` plus the current library so that while
1051Otherwise, we let `gez` be `zeg` plus the current library so that while
1010−compiling the dependencies of this library we don't later create a circular
1052+compiling the dependencies of this library we don't later create a
1011−dependency. We check next to see if this library is alredy in `bil`. If so, then
1053+circular dependency. We check next to see if this library is alredy in
1012−we have already included this library earlier, so we check to see if this is the
1054+`bil`. If so, then we have already included this library earlier, so we
1013−same version of the library as we included earlier. If so, we skip it. Else, we
1055+check to see if this is the same version of the library as we included
1014−fail since we can't include two different versions of a library. We really
1056+earlier. If so, we skip it. Else, we fail since we can't include two
1015−should allow for newer versions of a library since in kelvin versioning we
1057+different versions of a library. We really should allow for newer
1016−assume backwards compatibility, but for now we require an exact match.
1058+versions of a library since in kelvin versioning we assume backwards
1017−
1059+compatibility, but for now we require an exact match.
1018−If we haven't already included this library, then we're going to do that. First,
1060+
1019−we get the location of the library with `++hone`.
1061+If we haven't already included this library, then we're going to do
1062+that. First, we get the location of the library with `++hone`.
1020
1063
1021 ++ hone :: plant hoof
1064 ++ hone :: plant hoof
1022 |= [for=@tas way=@tas huf=hoof]
1065 |= [for=@tas way=@tas huf=hoof]
@@ -1025,27 +1068,28 @@ we get the location of the library with `++hone`.
1025 how(s ~[for p.huf way])
1068 how(s ~[for p.huf way])
1026 [[q.u.q.huf %main p.u.q.huf] ~[for p.huf way]]
1069 [[q.u.q.huf %main p.u.q.huf] ~[for p.huf way]]
1027
1070
1028−If we haven't specified the version of the library, we use the current ship,
1071+If we haven't specified the version of the library, we use the current
1029−desk, and case. Otherwise, we use the given ship and case on desk `%main`. In
1072+ship, desk, and case. Otherwise, we use the given ship and case on desk
1030−either case, the path is `/way/p.huf/for`. In the case of `++neck`, this means
1073+`%main`. In either case, the path is `/way/p.huf/for`. In the case of
1031−`/lib/core/[library name]`.
1074+`++neck`, this means `/lib/core/[library name]`.
1032−
1075+
1033−In `++neck`, we next compile the hook file at that location with `++fade`.
1076+In `++neck`, we next compile the hook file at that location with
1034−Again, we will delay the discussion of `++fade`, noting only that it takes a
1077+`++fade`. Again, we will delay the discussion of `++fade`, noting only
1035−beam and parses the hook file there into a hood.
1078+that it takes a beam and parses the hook file there into a hood.
1036−
1079+
1037−We recurse on this to compile the library. During the compilation, we let `zeg`
1080+We recurse on this to compile the library. During the compilation, we
1038−be `gez` to avoid circular dependencies, we let `hol` be false since we don't
1081+let `zeg` be `gez` to avoid circular dependencies, we let `hol` be false
1039−allow horns in libraries, and we let `boy` be null so that we can isolate the
1082+since we don't allow horns in libraries, and we let `boy` be null so
1040−new body twigs.
1083+that we can isolate the new body twigs.
1041−
1084+
1042−Next, we reintegrate the new data into our context. We use the context created
1085+Next, we reintegrate the new data into our context. We use the context
1043−by the recursion with four changes. First, we reset `zeg` to our old `zeg`.
1086+created by the recursion with four changes. First, we reset `zeg` to our
1044−Second, we reset `hol` to our old `hol`. Third, we put the name of our library
1087+old `zeg`. Second, we reset `hol` to our old `hol`. Third, we put the
1045−onto the stack of libraries. This means all of a libraries dependencies will be
1088+name of our library onto the stack of libraries. This means all of a
1046−earlier in `lot` than the library itself, making `lot` a topological ordering on
1089+libraries dependencies will be earlier in `lot` than the library itself,
1047−the dependency graph. Fourth, we put in `bil` the library hoof and body (with
1090+making `lot` a topological ordering on the dependency graph. Fourth, we
1048−all body twigs collected in a `=~`), keyed by the library name.
1091+put in `bil` the library hoof and body (with all body twigs collected in
1092+a `=~`), keyed by the library name.
1049
1093
1050Finally, we recurse, processing the next library in our list.
1094Finally, we recurse, processing the next library in our list.
1051
1095
@@ -1084,36 +1128,38 @@ To complete our disucssion of `++apex`, we must process our structures.
1084 ==
1128 ==
1085 ^^$(cof cof, bir t.bir)
1129 ^^$(cof cof, bir t.bir)
1086
1130
1087−The processing of our structures is very similar to that of our libraries. For
1131+The processing of our structures is very similar to that of our
1088−clarity, we'll use many of the same phrases in describing the parallel natures.
1132+libraries. For clarity, we'll use many of the same phrases in describing
1089−First, we check to see if there are more structures to process. If not, we're
1133+the parallel natures. First, we check to see if there are more
1090−done, so we produce our context.
1134+structures to process. If not, we're done, so we produce our context.
1091−
1135+
1092−Otherwise, we let `goz` be `zog` plus the current structure so that while
1136+Otherwise, we let `goz` be `zog` plus the current structure so that
1093−compiling the dependencies of this structure we don't later create a circular
1137+while compiling the dependencies of this structure we don't later create
1094−dependency. We check next to see if this structure is alredy in `rop`. If so,
1138+a circular dependency. We check next to see if this structure is alredy
1095−then we have already included this structure earlier, so we check to see if this
1139+in `rop`. If so, then we have already included this structure earlier,
1096−is the same version of the structure as we included earlier. If so, we skip it.
1140+so we check to see if this is the same version of the structure as we
1097−Else, we fail since we can't include two different versions of a structure.
1141+included earlier. If so, we skip it. Else, we fail since we can't
1098−
1142+include two different versions of a structure.
1099−If we haven't loaded this structure, then we call `++hone` to get the beam where
1143+
1100−the file structure should be. If the loobean in the hoot is true, then we're
1144+If we haven't loaded this structure, then we call `++hone` to get the
1101−looking for a gate; otherwise, we're looking for a core. We parse this file with
1145+beam where the file structure should be. If the loobean in the hoot is
1102−`++fade`.
1146+true, then we're looking for a gate; otherwise, we're looking for a
1103−
1147+core. We parse this file with `++fade`.
1104−Now, we recurse on this to compile the structure. During the recursion, there we
1148+
1105−have threee changes. Frist, we let `zog` be `goz` so that we don't create a
1149+Now, we recurse on this to compile the structure. During the recursion,
1106−circular dependency. Second, we let `hol` be false since we do not allow horns
1150+there we have threee changes. Frist, we let `zog` be `goz` so that we
1107−in structures. Third, we let `boy` be null so that we can isolate the new body
1151+don't create a circular dependency. Second, we let `hol` be false since
1108−twigs.
1152+we do not allow horns in structures. Third, we let `boy` be null so that
1109−
1153+we can isolate the new body twigs.
1110−Next, we reintegrate the new data into our context. We use the context cretaed
1154+
1111−by the recursion with four changes. First, if we're including a gate structure,
1155+Next, we reintegrate the new data into our context. We use the context
1112−then we reset the body to its original body. Else we put on the top of our list
1156+cretaed by the recursion with four changes. First, if we're including a
1113−of body twigs what is essentially a `=+ structure-name` to take off the face of
1157+gate structure, then we reset the body to its original body. Else we put
1114−the structure. Second, we reset `zog` to our old `zog`. Third, we reset `hol` to
1158+on the top of our list of body twigs what is essentially a
1115−our old `hol`. Finally, we put in `rop` the structure hoof and body (with all
1159+`=+ structure-name` to take off the face of the structure. Second, we
1116−body twiggs collected in a `=~`), keyed by the structure name.
1160+reset `zog` to our old `zog`. Third, we reset `hol` to our old `hol`.
1161+Finally, we put in `rop` the structure hoof and body (with all body
1162+twiggs collected in a `=~`), keyed by the structure name.
1117
1163
1118Finally, we recurse, processing the next structure in our list.
1164Finally, we recurse, processing the next structure in our list.
1119
1165
@@ -1133,9 +1179,9 @@ This concludes our discussion of `++apex`.
1133 |= [cof=cafe fin=vase]
1179 |= [cof=cafe fin=vase]
1134 (fine cof fin)
1180 (fine cof fin)
1135
1181
1136−Returning to `++abut`, we have now processed the structures, libraries and body
1182+Returning to `++abut`, we have now processed the structures, libraries
1137−twigs. Next, we slap our preamble (structures and libraries) against zuse.
1183+and body twigs. Next, we slap our preamble (structures and libraries)
1138−First, we construct our preamble in `++able`.
1184+against zuse. First, we construct our preamble in `++able`.
1139
1185
1140 ++ able :: assemble preamble
1186 ++ able :: assemble preamble
1141 ^- twig
1187 ^- twig
@@ -1143,10 +1189,10 @@ First, we construct our preamble in `++able`.
1143 ?:(=(~ rop) [%$ 1] [%brcn (~(run by rop) |=([* a=twig] [%ash a]))])
1189 ?:(=(~ rop) [%$ 1] [%brcn (~(run by rop) |=([* a=twig] [%ash a]))])
1144 [%tssg (turn (flop lot) |=(a=term q:(need (~(get by bil) a))))]
1190 [%tssg (turn (flop lot) |=(a=term q:(need (~(get by bil) a))))]
1145
1191
1146−We first put the structures in `rop` into a single `|%` at the top and `=>` it
1192+We first put the structures in `rop` into a single `|%` at the top and
1147−onto a `=~` of our libraries, in the reverse order that they appear in `lot`.
1193+`=>` it onto a `=~` of our libraries, in the reverse order that they
1148−Thus, the structures are in a single core while the libraries are in consecutive
1194+appear in `lot`. Thus, the structures are in a single core while the
1149−cores.
1195+libraries are in consecutive cores.
1150
1196
1151We slap the preamble against zuse with `++maim`.
1197We slap the preamble against zuse with `++maim`.
1152
1198
@@ -1163,47 +1209,49 @@ We slap the preamble against zuse with `++maim`.
1163 `vase`[p.p.puz val]
1209 `vase`[p.p.puz val]
1164 ==
1210 ==
1165
1211
1166−Here we start to get into ford's caching system. We wrap our computation in a
1212+Here we start to get into ford's caching system. We wrap our computation
1167−call to `++clef` so that we only actually compute it if the result is not
1213+in a call to `++clef` so that we only actually compute it if the result
1168−already in our cache. First we'll discuss the computation, then we'll discuss
1214+is not already in our cache. First we'll discuss the computation, then
1169−the caching system.
1215+we'll discuss the caching system.
1170
1216
1171−We call `++mule` with a call to `++mint:ut` on the type of our subject vase
1217+We call `++mule` with a call to `++mint:ut` on the type of our subject
1172−against the given twig. In other words, we're compiling the twig with against
1218+vase against the given twig. In other words, we're compiling the twig
1173−the subject type in the given subject vase.
1219+with against the subject type in the given subject vase.
1174
1220
1175−If compilation fails, then we produce an error bolt with the produced stack
1221+If compilation fails, then we produce an error bolt with the produced
1176−trace. Otherwise, we run the produced nock with `++mock` and our sky function.
1222+stack trace. Otherwise, we run the produced nock with `++mock` and our
1177−We convert the produced toon to a bolt with `++coup` and use the type from `puz`
1223+sky function. We convert the produced toon to a bolt with `++coup` and
1178−combined with the value from `mock` to produce our vase.
1224+use the type from `puz` combined with the value from `mock` to produce
1225+our vase.
1179
1226
1180−If this process seems harder than just calling `++slap`, it's because it is. We
1227+If this process seems harder than just calling `++slap`, it's because it
1181−have two requirements that `++slap` doesn't satisfy. First, we want the to use
1228+is. We have two requirements that `++slap` doesn't satisfy. First, we
1182−an explicit sky function for use with `.^`. With `++slap`, you get whatever sky
1229+want the to use an explicit sky function for use with `.^`. With
1183−function is available in the calling context, which in ford is none. Second, we
1230+`++slap`, you get whatever sky function is available in the calling
1184−want to explicitly handle the stack trace on failure. `++slap` would cause crash
1231+context, which in ford is none. Second, we want to explicitly handle the
1185−on failure.
1232+stack trace on failure. `++slap` would cause crash on failure.
1186
1233
1187−We haven't yet discussed either `++clef` or `++coup`. We'll start with `++coup`
1234+We haven't yet discussed either `++clef` or `++coup`. We'll start with
1188−to finish the discussion of the computation.
1235+`++coup` to finish the discussion of the computation.
1189
1236
1190\``++ coup :: toon to bolt |= cof=cafe |* [ton=toon fun=$+(* *)] :- p=cof ^= q ?- -.ton %2 [%2 p=p.ton] %0 [%0 p=*(set beam) q=(fun p.ton)] %1 ~& [%coup-need ((list path) p.ton)] =- ?- -.faw & [%1 p=(sa (turn p.faw |=(a=beam [a *(list tank)])))] | [%2 p=p.faw] == ^= faw |- ^- (each (list beam) (list tank)) ?~ p.ton [%& ~] =+ nex=$(p.ton t.p.ton) =+ pax=(path i.p.ton) ?~ pax [%| (smyt pax) ?:(?=(& -.nex) ~ p.nex)] =+ zis=(tome t.pax) ?~ zis [%| (smyt pax) ?:(?=(& -.nex) ~ p.nex)] ?- -.nex & [%& u.zis p.nex] | nex == ==`
1237\``++ coup :: toon to bolt |= cof=cafe |* [ton=toon fun=$+(* *)] :- p=cof ^= q ?- -.ton %2 [%2 p=p.ton] %0 [%0 p=*(set beam) q=(fun p.ton)] %1 ~& [%coup-need ((list path) p.ton)] =- ?- -.faw & [%1 p=(sa (turn p.faw |=(a=beam [a *(list tank)])))] | [%2 p=p.faw] == ^= faw |- ^- (each (list beam) (list tank)) ?~ p.ton [%& ~] =+ nex=$(p.ton t.p.ton) =+ pax=(path i.p.ton) ?~ pax [%| (smyt pax) ?:(?=(& -.nex) ~ p.nex)] =+ zis=(tome t.pax) ?~ zis [%| (smyt pax) ?:(?=(& -.nex) ~ p.nex)] ?- -.nex & [%& u.zis p.nex] | nex == ==`
1191
1238
1192−Recall that a toon is either a `%0` value, a `%1` block, or a `%2` failure.
1239+Recall that a toon is either a `%0` value, a `%1` block, or a `%2`
1193−Converting a `%2` toon failure into a `%2` bolt failure is trivial. Converting a
1240+failure. Converting a `%2` toon failure into a `%2` bolt failure is
1194−`%0` toon value into a `%0` bolt value is easy since we assume there were no
1241+trivial. Converting a `%0` toon value into a `%0` bolt value is easy
1195−dependencies. Converting the blocks is rather more difficult.
1242+since we assume there were no dependencies. Converting the blocks is
1243+rather more difficult.
1196
1244
1197−To compute `faw`, we recurse through the list of paths in the `%1` toon. At each
1245+To compute `faw`, we recurse through the list of paths in the `%1` toon.
1198−one, we make sure with `++tome` that it is, in fact, a beam. If so, then we
1246+At each one, we make sure with `++tome` that it is, in fact, a beam. If
1199−check to see if the later paths succeed as well. If so, we append the current
1247+so, then we check to see if the later paths succeed as well. If so, we
1200−path to the list of other paths. If not, we produce the error message we got
1248+append the current path to the list of other paths. If not, we produce
1201−from processing the rest of the paths. If this path is not a beam, then we fail,
1249+the error message we got from processing the rest of the paths. If this
1202−producing a list of tanks including this path and, if later paths fail too,
1250+path is not a beam, then we fail, producing a list of tanks including
1203−those paths as well.
1251+this path and, if later paths fail too, those paths as well.
1204
1252
1205−If some paths were not beams, then we produce a `%2` error bolt. If all paths
1253+If some paths were not beams, then we produce a `%2` error bolt. If all
1206−were correct, then we produce a `%1` blocking bolt.
1254+paths were correct, then we produce a `%1` blocking bolt.
1207
1255
1208We will now discuss `++clef`. This is where the cache magic happens.
1256We will now discuss `++clef`. This is where the cache magic happens.
1209
1257
@@ -1238,28 +1286,29 @@ Otherwise, we look up the request in the cache with `++calk`.
1238 ?~ d [~ a] ::
1286 ?~ d [~ a] ::
1239 [d a(p (~(put in p.a) u.d))] ::
1287 [d a(p (~(put in p.a) u.d))] ::
1240
1288
1241−When looking up something in the cache, we mark it if we find it. This way, we
1289+When looking up something in the cache, we mark it if we find it. This
1242−have in our cache the set of all cache entries that have been referenced. While
1290+way, we have in our cache the set of all cache entries that have been
1243−we do not at present do anything with this data, it should be used to clear out
1291+referenced. While we do not at present do anything with this data, it
1244−old and unused entries in the cache.
1292+should be used to clear out old and unused entries in the cache.
1245
1293
1246−Moving on in `++clef`, we check to see if we actually found anything. If we
1294+Moving on in `++clef`, we check to see if we actually found anything. If
1247−didn't find a cache entry, then we run the computation in `fun`, and examine its
1295+we didn't find a cache entry, then we run the computation in `fun`, and
1248−result. If it produced a `%2` error or `%1` block bolt, we just pass that
1296+examine its result. If it produced a `%2` error or `%1` block bolt, we
1249−through. Otherwise, we produce both the value and an updated cache with this new
1297+just pass that through. Otherwise, we produce both the value and an
1250−entry. We add the entry with `++came`.
1298+updated cache with this new entry. We add the entry with `++came`.
1251
1299
1252 ++ came ::
1300 ++ came ::
1253 |= [a=cafe b=calx] :: cache install
1301 |= [a=cafe b=calx] :: cache install
1254 ^- cafe ::
1302 ^- cafe ::
1255 a(q (~(put by q.a) [-.b q.b] b)) ::
1303 a(q (~(put by q.a) [-.b q.b] b)) ::
1256
1304
1257−We key cache entries by the type of computation (`-:calx`) and the inputs to the
1305+We key cache entries by the type of computation (`-:calx`) and the
1258−computation (`q:calc`). This just puts the cache line in the cache at the
1306+inputs to the computation (`q:calc`). This just puts the cache line in
1259−correct key.
1307+the cache at the correct key.
1260
1308
1261−Back in `++clef`, if we did find a cache entry, then we just produce the value
1309+Back in `++clef`, if we did find a cache entry, then we just produce the
1262−at that cache line. We convert the cache line into a value with `++calf`.
1310+value at that cache line. We convert the cache line into a value with
1311+`++calf`.
1263
1312
1264 ++ calf :: reduce calx
1313 ++ calf :: reduce calx
1265 |* sem=* :: a typesystem hack
1314 |* sem=* :: a typesystem hack
@@ -1270,22 +1319,23 @@ at that cache line. We convert the cache line into a value with `++calf`.
1270 %slam ?>(?=(%slam -.cax) r.cax)
1319 %slam ?>(?=(%slam -.cax) r.cax)
1271 ==
1320 ==
1272
1321
1273−This is simply a typesystem hack. Because the `sem` is passed in through a wet
1322+This is simply a typesystem hack. Because the `sem` is passed in through
1274−gate, we know at type time which of the three cases will be chosen. Thus, the
1323+a wet gate, we know at type time which of the three cases will be
1275−correct type of the value in the cache line gets passed through to the caller.
1324+chosen. Thus, the correct type of the value in the cache line gets
1276−This also depends on the fact that `++clef` is wet. The type stuff here is
1325+passed through to the caller. This also depends on the fact that
1277−mathematically interesting, but the action is simple: we get the value from the
1326+`++clef` is wet. The type stuff here is mathematically interesting, but
1278−cache line.
1327+the action is simple: we get the value from the cache line.
1279
1328
1280This concludes our discussion of `++clef` and `++maim`.
1329This concludes our discussion of `++clef` and `++maim`.
1281
1330
1282−Back in `++abut`, recall that we processed the structures, libraries, and body
1331+Back in `++abut`, recall that we processed the structures, libraries,
1283−with `++apex`. Then, we slapped our preamble (structures and libraries) against
1332+and body with `++apex`. Then, we slapped our preamble (structures and
1284−zuse with `++maim`. Next, we process our resources with `++chap`. Note that we
1333+libraries) against zuse with `++maim`. Next, we process our resources
1285−pass in the preamble so that we may refer to anything in there in our resources.
1334+with `++chap`. Note that we pass in the preamble so that we may refer to
1335+anything in there in our resources.
1286
1336
1287−`++chap` is broken up into a different case for each horn. We'll go through them
1337+`++chap` is broken up into a different case for each horn. We'll go
1288−one by one.
1338+through them one by one.
1289
1339
1290 ++ chap :: produce resources
1340 ++ chap :: produce resources
1291 |= [cof=cafe bax=vase hon=horn]
1341 |= [cof=cafe bax=vase hon=horn]
@@ -1300,9 +1350,9 @@ This is `/~`. We slap the twig against our context.
1300 |= [cof=cafe gat=vase]
1350 |= [cof=cafe gat=vase]
1301 (maul cof gat !>([how arg]))
1351 (maul cof gat !>([how arg]))
1302
1352
1303−This is `/$`. We slap the twig against our context, which we expect to produce a
1353+This is `/$`. We slap the twig against our context, which we expect to
1304−gate. We slam this gate with a sample of `how` and `arg`, which is our location
1354+produce a gate. We slam this gate with a sample of `how` and `arg`,
1305−and the heel (virtual path extension).
1355+which is our location and the heel (virtual path extension).
1306
1356
1307`++maul` is similar to `++maim`, but it slams instead of slaps.
1357`++maul` is similar to `++maim`, but it slams instead of slaps.
1308
1358
@@ -1319,19 +1369,20 @@ and the heel (virtual path extension).
1319 `vase`[p.top val]
1369 `vase`[p.top val]
1320 ==
1370 ==
1321
1371
1322−We cache slams exactly as we cache slaps. We use `++slit` to find the type of
1372+We cache slams exactly as we cache slaps. We use `++slit` to find the
1323−the product of the slam given the types of the gate and the sample.
1373+type of the product of the slam given the types of the gate and the
1374+sample.
1324
1375
1325−If this type fails, we produce the given stack trace as a `%2` error bolt.
1376+If this type fails, we produce the given stack trace as a `%2` error
1326−Otherwise, we produce the top produced above combined with the value we get from
1377+bolt. Otherwise, we produce the top produced above combined with the
1327−slamming the values in the vases with `++mong`.
1378+value we get from slamming the values in the vases with `++mong`.
1328
1379
1329Back to `++chap`.
1380Back to `++chap`.
1330
1381
1331 %day (chad cof bax %dr p.hon)
1382 %day (chad cof bax %dr p.hon)
1332
1383
1333−This is `/|`. We call `++chad` to convert textual names to relative dates and
1384+This is `/|`. We call `++chad` to convert textual names to relative
1334−process the next horn against each of the discovered paths.
1385+dates and process the next horn against each of the discovered paths.
1335
1386
1336 ++ chad :: atomic list
1387 ++ chad :: atomic list
1337 |= [cof=cafe bax=vase doe=term hon=horn]
1388 |= [cof=cafe bax=vase doe=term hon=horn]
@@ -1355,8 +1406,8 @@ process the next horn against each of the discovered paths.
1355 ?~ yal [[%cube 0 [%atom %n]] 0]
1406 ?~ yal [[%cube 0 [%atom %n]] 0]
1356 (slop (slop [[%atom doe] p.i.yal] q.i.yal) $(yal t.yal))
1407 (slop (slop [[%atom doe] p.i.yal] q.i.yal) $(yal t.yal))
1357
1408
1358−First, we call `++lash` to parse the children of the current beam and pick out
1409+First, we call `++lash` to parse the children of the current beam and
1359−those ones that are of the requested format.
1410+pick out those ones that are of the requested format.
1360
1411
1361 ++ lash :: atomic sequence
1412 ++ lash :: atomic sequence
1362 |= wox=$+(span (unit ,@))
1413 |= wox=$+(span (unit ,@))
@@ -1366,22 +1417,26 @@ those ones that are of the requested format.
1366 |= [cof=cafe arc=arch]
1417 |= [cof=cafe arc=arch]
1367 (fine cof (lark wox arc))
1418 (fine cof (lark wox arc))
1368
1419
1369−First, we get the arch with `++lend`, as described above. We filter and parse
1420+First, we get the arch with `++lend`, as described above. We filter and
1370−the child names with `++lark` according to the given parser function.
1421+parse the child names with `++lark` according to the given parser
1422+function.
1371
1423
1372−In `++chad`, this parser function is `(slat doe)`, which will parse a cord into
1424+In `++chad`, this parser function is `(slat doe)`, which will parse a
1373−an atom of the requested odor. For `%day` the odor is for relative dates.
1425+cord into an atom of the requested odor. For `%day` the odor is for
1426+relative dates.
1374
1427
1375−Thus, we now have a map from atoms of the given odor to the actual child names.
1428+Thus, we now have a map from atoms of the given odor to the actual child
1376−We next turn this map into a list and sort it in increasing order by the atom.
1429+names. We next turn this map into a list and sort it in increasing order
1430+by the atom.
1377
1431
1378−We next convert this list of pairs of atoms and spans to a list of pairs of
1432+We next convert this list of pairs of atoms and spans to a list of pairs
1379−atoms and vases. We process the given horn once at every child beam, producing
1433+of atoms and vases. We process the given horn once at every child beam,
1380−the resource at that location.
1434+producing the resource at that location.
1381
1435
1382−Finally, we convert this list of pairs of atoms and vases to a vase of a list of
1436+Finally, we convert this list of pairs of atoms and vases to a vase of a
1383−pairs of atoms to (well-typed) values. Each entry in the list is of type atom
1437+list of pairs of atoms to (well-typed) values. Each entry in the list is
1384−with the given odor combined with the type of the produced vase.
1438+of type atom with the given odor combined with the type of the produced
1439+vase.
1385
1440
1386Back in `++chap`, we continue parsing resources.
1441Back in `++chap`, we continue parsing resources.
1387
1442
@@ -1390,8 +1445,8 @@ Back in `++chap`, we continue parsing resources.
1390 |= [cof=cafe vax=vase]
1445 |= [cof=cafe vax=vase]
1391 (fine cof [[%face p.hon p.vax] q.vax])
1446 (fine cof [[%face p.hon p.vax] q.vax])
1392
1447
1393−This is `/=`. We process the given horn, giving us a vase. We put as a face on
1448+This is `/=`. We process the given horn, giving us a vase. We put as a
1394−the vase so that it may be referred to later by name.
1449+face on the vase so that it may be referred to later by name.
1395
1450
1396 %fan
1451 %fan
1397 %+ cope
1452 %+ cope
@@ -1408,21 +1463,21 @@ the vase so that it may be referred to later by name.
1408 ?~ tev [[%cube 0 [%atom %n]] 0]
1463 ?~ tev [[%cube 0 [%atom %n]] 0]
1409 (slop i.tev $(tev t.tev))
1464 (slop i.tev $(tev t.tev))
1410
1465
1411−This is `/.`. We first process each of the child horns, producing a list of
1466+This is `/.`. We first process each of the child horns, producing a list
1412−vases. This is done by just recursing on `++chap`. Then, we simply fold over
1467+of vases. This is done by just recursing on `++chap`. Then, we simply
1413−this list to create a vase of the list of values.
1468+fold over this list to create a vase of the list of values.
1414
1469
1415 %for $(hon q.hon, s.how (weld (flop p.hon) s.how))
1470 %for $(hon q.hon, s.how (weld (flop p.hon) s.how))
1416
1471
1417−This is `/,`. We simply recurse on the horn with the given path welded onto our
1472+This is `/,`. We simply recurse on the horn with the given path welded
1418−current beam.
1473+onto our current beam.
1419
1474
1420 %hub (chad cof bax %ud p.hon)
1475 %hub (chad cof bax %ud p.hon)
1421
1476
1422−This is `/@`. This is exactly like the processing of `%day` except we expect the
1477+This is `/@`. This is exactly like the processing of `%day` except we
1423−children to be named as unsigned integers rather than relative dates. We process
1478+expect the children to be named as unsigned integers rather than
1424−the horn at each of the children's locations and produce a list of pairs of
1479+relative dates. We process the horn at each of the children's locations
1425−absolute dates and values.
1480+and produce a list of pairs of absolute dates and values.
1426
1481
1427 %man
1482 %man
1428 |- ^- (bolt vase)
1483 |- ^- (bolt vase)
@@ -1438,20 +1493,21 @@ absolute dates and values.
1438 (slop [[%atom %tas] p.n.p.hon] vax)
1493 (slop [[%atom %tas] p.n.p.hon] vax)
1439 (slop lef rig)
1494 (slop lef rig)
1440
1495
1441−This is `/*`. We process each of the horns in the given map by recursion through
1496+This is `/*`. We process each of the horns in the given map by recursion
1442−`++chap`. Once we have these vases, we create a vase of a map from the given
1497+through `++chap`. Once we have these vases, we create a vase of a map
1443−textual names to the produced values.
1498+from the given textual names to the produced values.
1444
1499
1445 %now (chad cof bax %da p.hon)
1500 %now (chad cof bax %da p.hon)
1446
1501
1447−This is `/&`. This is exactly like the processing of `%now` except we expect the
1502+This is `/&`. This is exactly like the processing of `%now` except we
1448−children to be names as absolute dates rather than relative dates. We process
1503+expect the children to be names as absolute dates rather than relative
1449−the horn at each of the children's locations and produce a list of pairs of
1504+dates. We process the horn at each of the children's locations and
1450−absolute dates and values.
1505+produce a list of pairs of absolute dates and values.
1451
1506
1452 %nap (chai cof bax p.hon)
1507 %nap (chai cof bax p.hon)
1453
1508
1454−This is `/%`. Here, we process the horn at each of our children with `++chai`.
1509+This is `/%`. Here, we process the horn at each of our children with
1510+`++chai`.
1455
1511
1456 ++ chai :: atomic map
1512 ++ chai :: atomic map
1457 |= [cof=cafe bax=vase hon=horn]
1513 |= [cof=cafe bax=vase hon=horn]
@@ -1476,10 +1532,10 @@ This is `/%`. Here, we process the horn at each of our children with `++chai`.
1476 (slop [[%atom %a] p.n.doy] q.n.doy)
1532 (slop [[%atom %a] p.n.doy] q.n.doy)
1477 (slop $(doy l.doy) $(doy r.doy))
1533 (slop $(doy l.doy) $(doy r.doy))
1478
1534
1479−We get the arch at our current beam with `++lend`. Then, we process the horn at
1535+We get the arch at our current beam with `++lend`. Then, we process the
1480−each of our children to give us a map of atoms to vases. Finally, we convert
1536+horn at each of our children to give us a map of atoms to vases.
1481−that into a vase of a map of these atoms to the values. This is very similar to
1537+Finally, we convert that into a vase of a map of these atoms to the
1482−`++chad` and the handling of `%man`.
1538+values. This is very similar to `++chad` and the handling of `%man`.
1483
1539
1484 %see $(hon q.hon, how p.hon)
1540 %see $(hon q.hon, how p.hon)
1485
1541
@@ -1492,9 +1548,9 @@ This is `/:`. We process the given horn at the given beam.
1492 |= [cof=cafe gat=vase]
1548 |= [cof=cafe gat=vase]
1493 (maul cof gat sam)
1549 (maul cof gat sam)
1494
1550
1495−This is `/;`. First, we process the given horn. Then, we slap the given twig
1551+This is `/;`. First, we process the given horn. Then, we slap the given
1496−against our context to produce (hopefully) a gate. Finally, we slam the vase we
1552+twig against our context to produce (hopefully) a gate. Finally, we slam
1497−got from processing the horn against the gate.
1553+the vase we got from processing the horn against the gate.
1498
1554
1499 %sic
1555 %sic
1500 %+ cope $(hon q.hon)
1556 %+ cope $(hon q.hon)
@@ -1505,26 +1561,27 @@ got from processing the horn against the gate.
1505 (flaw cof [%leaf "type error: {<p.hon>} {<q.hon>}"]~)
1561 (flaw cof [%leaf "type error: {<p.hon>} {<q.hon>}"]~)
1506 (fine cof [p.tug q.vax])
1562 (fine cof [p.tug q.vax])
1507
1563
1508−This is `/^`. First, we process the given horn. Then, we slap the the bunt of
1564+This is `/^`. First, we process the given horn. Then, we slap the the
1509−the given tile against our context. This will produce a vase with the correct
1565+bunt of the given tile against our context. This will produce a vase
1510−type. We test to see if this type nests within the type of the vase we got from
1566+with the correct type. We test to see if this type nests within the type
1511−processing the horn. If so, we produce the value from the horn along with the
1567+of the vase we got from processing the horn. If so, we produce the value
1512−type from the tile. Otherwise, we produce a `%2` error bolt.
1568+from the horn along with the type from the tile. Otherwise, we produce a
1569+`%2` error bolt.
1513
1570
1514 %toy (cope (make cof %bake p.hon how ~) feel)
1571 %toy (cope (make cof %bake p.hon how ~) feel)
1515 ==
1572 ==
1516
1573
1517−This is `/mark/`. Here, we simply run the `%bake` silk on the given mark,
1574+This is `/mark/`. Here, we simply run the `%bake` silk on the given
1518−producing a cage. We convert this cage into a vase with `++feel`, which is
1575+mark, producing a cage. We convert this cage into a vase with `++feel`,
1519−exactly as simple as it sounds like it should be.
1576+which is exactly as simple as it sounds like it should be.
1520
1577
1521 ++ feel |=([a=cafe b=cage] (fine a q.b)) :: cage to vase
1578 ++ feel |=([a=cafe b=cage] (fine a q.b)) :: cage to vase
1522
1579
1523This is trivial.
1580This is trivial.
1524
1581
1525−We will discuss later `++make` and how `%bake` is processed. Suffice it to say
1582+We will discuss later `++make` and how `%bake` is processed. Suffice it
1526−that baking a resource with a given mark gets the resource and converts it, if
1583+to say that baking a resource with a given mark gets the resource and
1527−necessary, to the requested mark.
1584+converts it, if necessary, to the requested mark.
1528
1585
1529This concludes our discussion of `++chap`.
1586This concludes our discussion of `++chap`.
1530
1587
@@ -1544,24 +1601,26 @@ We return once more to `++abut`.
1544 |= [cof=cafe fin=vase]
1601 |= [cof=cafe fin=vase]
1545 (fine cof fin)
1602 (fine cof fin)
1546
1603
1547−Recall that we processed our structures, libraries and body with `++apex`. We
1604+Recall that we processed our structures, libraries and body with
1548−slapped our structures and libraries against zuse with `++maim`. We processed
1605+`++apex`. We slapped our structures and libraries against zuse with
1549−our resources with `++chap`. Now, all our body twigs are collected in a `=~` and
1606+`++maim`. We processed our resources with `++chap`. Now, all our body
1550−slapped against our structures, libraries, and resources. This produces our
1607+twigs are collected in a `=~` and slapped against our structures,
1551−final result.
1608+libraries, and resources. This produces our final result.
1552
1609
1553The hook file has been assembled. And there was great rejoicing.
1610The hook file has been assembled. And there was great rejoicing.
1554
1611
1555−## Lifecycle of a Kiss
1612+Lifecycle of a Kiss
1613+-------------------
1556
1614
1557−We're now going to go through a series of lifecycle descriptions. When a user of
1615+We're now going to go through a series of lifecycle descriptions. When a
1558−ford sends a kiss, it is one of a dozen different types of silk. We'll go
1616+user of ford sends a kiss, it is one of a dozen different types of silk.
1559−through each one, tracing through the flow of control of each of these.
1617+We'll go through each one, tracing through the flow of control of each
1618+of these.
1560
1619
1561First, though, we'll describe the common handling to all kisses.
1620First, though, we'll describe the common handling to all kisses.
1562
1621
1563−The silk in a `%exec` kiss to ford ends up in `++apex`, so we'll enter the
1622+The silk in a `%exec` kiss to ford ends up in `++apex`, so we'll enter
1564−narrative here.
1623+the narrative here.
1565
1624
1566 ++ apex :: call
1625 ++ apex :: call
1567 |= kus=(unit silk)
1626 |= kus=(unit silk)
@@ -1580,13 +1639,13 @@ narrative here.
1580 ==
1639 ==
1581 ~(exec zo [num `task`[hen u.kus 0 ~]])
1640 ~(exec zo [num `task`[hen u.kus 0 ~]])
1582
1641
1583−Recall that a `%exec` kiss actually sends a unit silk. If it's null, we're
1642+Recall that a `%exec` kiss actually sends a unit silk. If it's null,
1584−trying to cancel the request. We first look up the task number keyed by duct. If
1643+we're trying to cancel the request. We first look up the task number
1585−we don't find it, then we're trying to cancel a request that either was never
1644+keyed by duct. If we don't find it, then we're trying to cancel a
1586−started or has already completed. We print out `%ford-mystery` and do nothing.
1645+request that either was never started or has already completed. We print
1587−If we do find the task number, then we look up the task from it, call
1646+out `%ford-mystery` and do nothing. If we do find the task number, then
1588−`++camo:zo` to cancel pending requests, and call `++amok:zo` to remove the task
1647+we look up the task from it, call `++camo:zo` to cancel pending
1589−from our task lists.
1648+requests, and call `++amok:zo` to remove the task from our task lists.
1590
1649
1591 ++ camo :: stop requests
1650 ++ camo :: stop requests
1592 ^+ .
1651 ^+ .
@@ -1602,9 +1661,9 @@ from our task lists.
1602 [%warp [our p.q.i.kiz] q.q.i.kiz ~]
1661 [%warp [our p.q.i.kiz] q.q.i.kiz ~]
1603 ==
1662 ==
1604
1663
1605−Our list of blocks is in `q.kig`, so we iterate over it, cancelling our pending
1664+Our list of blocks is in `q.kig`, so we iterate over it, cancelling our
1606−requests for each block. Our requests are all to clay, so we need only to send
1665+pending requests for each block. Our requests are all to clay, so we
1607−`%warp` kisses with a null instead of a rave.
1666+need only to send `%warp` kisses with a null instead of a rave.
1608
1667
1609 ++ amok
1668 ++ amok
1610 %_ ..zo
1669 %_ ..zo
@@ -1612,12 +1671,12 @@ requests for each block. Our requests are all to clay, so we need only to send
1612 dym.bay (~(del by dym.bay) nah)
1671 dym.bay (~(del by dym.bay) nah)
1613 ==
1672 ==
1614
1673
1615−We remove the task number from the map of numbers to tasks and the duct from the
1674+We remove the task number from the map of numbers to tasks and the duct
1616−map of ducts to task numbers.
1675+from the map of ducts to task numbers.
1617
1676
1618−Back in `++apex`, if we were given a silk, we need to process it. We add the
1677+Back in `++apex`, if we were given a silk, we need to process it. We add
1619−task to our maps, increment the next task number, and call `++exec:zo` on the
1678+the task to our maps, increment the next task number, and call
1620−new task.
1679+`++exec:zo` on the new task.
1621
1680
1622 ++ exec :: execute app
1681 ++ exec :: execute app
1623 ^+ ..zo
1682 ^+ ..zo
@@ -1638,9 +1697,9 @@ new task.
1638
1697
1639If we're still blocked on something in `q.kig`, we don't do anything.
1698If we're still blocked on something in `q.kig`, we don't do anything.
1640
1699
1641−Otherwise, we try to process the silk with `++make`. `++make` handles each
1700+Otherwise, we try to process the silk with `++make`. `++make` handles
1642−individual request and will be the entire focus of the remainder of this doc
1701+each individual request and will be the entire focus of the remainder of
1643−after this section. It produces a bolt of a cage.
1702+this doc after this section. It produces a bolt of a cage.
1644
1703
1645We put the new cache in our state with `++dash`.
1704We put the new cache in our state with `++dash`.
1646
1705
@@ -1649,12 +1708,13 @@ We put the new cache in our state with `++dash`.
1649 ^+ +>
1708 ^+ +>
1650 %_(+> jav.bay q.cof)
1709 %_(+> jav.bay q.cof)
1651
1710
1652−The cache is put in the baby so that it gets stored across calls to ford.
1711+The cache is put in the baby so that it gets stored across calls to
1712+ford.
1653
1713
1654−In `++exec`, we process the bolt in three different ways according to the type
1714+In `++exec`, we process the bolt in three different ways according to
1655−of bolt produced. If we produced a `%0` value bolt, we use `++expo` to give the
1715+the type of bolt produced. If we produced a `%0` value bolt, we use
1656−produced value and set of dependencies as a `%made` gift, and we remove
1716+`++expo` to give the produced value and set of dependencies as a `%made`
1657−ourselves from the task list with `++amok`.
1717+gift, and we remove ourselves from the task list with `++amok`.
1658
1718
1659 ++ expo :: return gift
1719 ++ expo :: return gift
1660 |= gef=gift
1720 |= gef=gift
@@ -1662,11 +1722,11 @@ ourselves from the task list with `++amok`.
1662
1722
1663We simply push the gift onto our list of moves.
1723We simply push the gift onto our list of moves.
1664
1724
1665−In `++exec`, if we produced a `%2` error bolt, we produce a `%made` gift with
1725+In `++exec`, if we produced a `%2` error bolt, we produce a `%made` gift
1666−the stack trace.
1726+with the stack trace.
1667
1727
1668−If we produced a `%1` block bolt, we iterate through each of the blocks and call
1728+If we produced a `%1` block bolt, we iterate through each of the blocks
1669−`++camp` to produce a clay request for the resource.
1729+and call `++camp` to produce a clay request for the resource.
1670
1730
1671 ++ camp :: request a file
1731 ++ camp :: request a file
1672 |= [ren=care bem=beam]
1732 |= [ren=care bem=beam]
@@ -1680,13 +1740,16 @@ If we produced a `%1` block bolt, we iterate through each of the blocks and call
1680 [%warp [our p.bem] q.bem [~ %& %x r.bem s.bem]]
1740 [%warp [our p.bem] q.bem [~ %& %x r.bem s.bem]]
1681 ==
1741 ==
1682
1742
1683−We put the resource in our block list in `q.kig` so that we save the fact that
1743+We put the resource in our block list in `q.kig` so that we save the
1684−we're blocked. We then produce the `%warp` request to clay for the resource. Our
1744+fact that we're blocked. We then produce the `%warp` request to clay for
1685−request path has the format \`/\[our-ship\]/\[task-number\]/\[block-number\]'.
1745+the resource. Our request path has the format
1746+\`/[our-ship]/[task-number]/[block-number]'.
1686
1747
1687−We'll now describe how each of the individual silks are processed in `++make`.
1748+We'll now describe how each of the individual silks are processed in
1749+`++make`.
1688
1750
1689−## Lifecycle of a Cell
1751+Lifecycle of a Cell
1752+-------------------
1690
1753
1691 ^
1754 ^
1692 %. [cof p.kas q.kas]
1755 %. [cof p.kas q.kas]
@@ -1700,10 +1763,10 @@ We'll now describe how each of the individual silks are processed in `++make`.
1700 [p=cof q=[%0 ~ [%$ (slop q.bor q.heg)]]]
1763 [p=cof q=[%0 ~ [%$ (slop q.bor q.heg)]]]
1701 ==
1764 ==
1702
1765
1703−Silks autocons. The product of a cell of silks is a cell of the products of the
1766+Silks autocons. The product of a cell of silks is a cell of the products
1704−silks, so we evaluate the two silks in parallel with `++coax` and slop together
1767+of the silks, so we evaluate the two silks in parallel with `++coax` and
1705−the results in a cell vase. We mark the product with `%$`, which means we know
1768+slop together the results in a cell vase. We mark the product with `%$`,
1706−no more mark information than that it is a noun.
1769+which means we know no more mark information than that it is a noun.
1707
1770
1708 ++ coax :: bolt across
1771 ++ coax :: bolt across
1709 |* [hoc=(bolt) fun=(burg)]
1772 |* [hoc=(bolt) fun=(burg)]
@@ -1727,19 +1790,23 @@ no more mark information than that it is a noun.
1727 %2 hoc
1790 %2 hoc
1728 ==
1791 ==
1729
1792
1730−If the first bolt is a value, we evaluate the burg to get the next bolt. If that
1793+If the first bolt is a value, we evaluate the burg to get the next bolt.
1731−also produces a value, we merge the dependency sets and produce a cell of the
1794+If that also produces a value, we merge the dependency sets and produce
1732−two values. Otherwise, we produce the block or error of the second bolt.
1795+a cell of the two values. Otherwise, we produce the block or error of
1796+the second bolt.
1733
1797
1734−If the first bolt is a block, we evaluate the burg to get the next bolt. If that
1798+If the first bolt is a block, we evaluate the burg to get the next bolt.
1735−produces a value, we just produce the block. If it produces a block, we merge
1799+If that produces a value, we just produce the block. If it produces a
1736−the two block sets. If it produces an error, we produce that error.
1800+block, we merge the two block sets. If it produces an error, we produce
1801+that error.
1737
1802
1738If the first bolt is already an error, we just pass that through.
1803If the first bolt is already an error, we just pass that through.
1739
1804
1740−Note that `++coax` (and, indeed, `++cope`) is reasonable to use with `;~`.
1805+Note that `++coax` (and, indeed, `++cope`) is reasonable to use with
1806+`;~`.
1741
1807
1742−## Lifecycle of a `%bake`
1808+Lifecycle of a `%bake`
1809+----------------------
1743
1810
1744 %bake
1811 %bake
1745 %+ cool |.(leaf/"ford: bake {<p.kas>} {<(tope q.kas)>}")
1812 %+ cool |.(leaf/"ford: bake {<p.kas>} {<(tope q.kas)>}")
@@ -1749,11 +1816,12 @@ Note that `++coax` (and, indeed, `++cope`) is reasonable to use with `;~`.
1749 (flaw cof (smyt (tope q.kas)) ~)
1816 (flaw cof (smyt (tope q.kas)) ~)
1750 (fine cof [p.kas u.vux])
1817 (fine cof [p.kas u.vux])
1751
1818
1752−This is one of the most critical silks. We are going to functionally produce the
1819+This is one of the most critical silks. We are going to functionally
1753−hook file at the given beam with the given heel. The result will be of the
1820+produce the hook file at the given beam with the given heel. The result
1754−correct mark, even if we need to run conversion functions. The functionality is
1821+will be of the correct mark, even if we need to run conversion
1755−encapsulated in `++lime`. If it produces null, then we produce an error.
1822+functions. The functionality is encapsulated in `++lime`. If it produces
1756−Otherwise, we take the vase produced and give it the correct mark.
1823+null, then we produce an error. Otherwise, we take the vase produced and
1824+give it the correct mark.
1757
1825
1758 ++ lima :: load at depth
1826 ++ lima :: load at depth
1759 |= [cof=cafe for=mark bem=beam arg=heel]
1827 |= [cof=cafe for=mark bem=beam arg=heel]
@@ -1775,19 +1843,19 @@ Otherwise, we take the vase produced and give it the correct mark.
1775 |= [cof=cafe vax=vase]
1843 |= [cof=cafe vax=vase]
1776 (fine cof ~ vax)
1844 (fine cof ~ vax)
1777
1845
1778−First, we load the arch at the given beam with `++lend`. If we have a child
1846+First, we load the arch at the given beam with `++lend`. If we have a
1779−named the mark, our job is straightforward, so we go ahead and load that with
1847+child named the mark, our job is straightforward, so we go ahead and
1780−`++lace`.
1848+load that with `++lace`.
1781
1849
1782−Otherwise, we iterate through our children. If we have no children, we produce
1850+Otherwise, we iterate through our children. If we have no children, we
1783−null, signifying that we didn't find any way to convert to the requested mark.
1851+produce null, signifying that we didn't find any way to convert to the
1784−Otherwise, we call `++lion` to find a translation path from one of the available
1852+requested mark. Otherwise, we call `++lion` to find a translation path
1785−marks into the target mark. We recursively bake the child that has a path to the
1853+from one of the available marks into the target mark. We recursively
1786−target mark, and then we call `++lope` to translate this mark into the target
1854+bake the child that has a path to the target mark, and then we call
1787−mark.
1855+`++lope` to translate this mark into the target mark.
1788
1856
1789−We'll first discuss the direct case of when one of our children is of the
1857+We'll first discuss the direct case of when one of our children is of
1790−correct mark.
1858+the correct mark.
1791
1859
1792 ++ lace :: load and check
1860 ++ lace :: load and check
1793 |= [cof=cafe for=mark bem=beam arg=heel]
1861 |= [cof=cafe for=mark bem=beam arg=heel]
@@ -1803,10 +1871,11 @@ correct mark.
1803 (cope (cope (abut:(meow bem arg) cof hyd) (lake for bek)) fest)
1871 (cope (cope (abut:(meow bem arg) cof hyd) (lake for bek)) fest)
1804 (fine cof ~)
1872 (fine cof ~)
1805
1873
1806−First, we get the arch at the given beam with `++lend`. If this is a file, we
1874+First, we get the arch at the given beam with `++lend`. If this is a
1807−load the file with `++liar` and coerce the type with `++lake`. Otherwise, we
1875+file, we load the file with `++liar` and coerce the type with `++lake`.
1808−check to see if we have a hook file here. If so, we parse it with `++fade`,
1876+Otherwise, we check to see if we have a hook file here. If so, we parse
1809−compile it with `++abut:meow`, and coerce the type with `++lake`.
1877+it with `++fade`, compile it with `++abut:meow`, and coerce the type
1878+with `++lake`.
1810
1879
1811Otherwise, there is no way to translate this, so we produce null.
1880Otherwise, there is no way to translate this, so we produce null.
1812
1881
@@ -1817,8 +1886,9 @@ Otherwise, there is no way to translate this, so we produce null.
1817This is just `++some` for bolts.
1886This is just `++some` for bolts.
1818
1887
1819We've delayed the discussion of `++fade` far too many times. It's not
1888We've delayed the discussion of `++fade` far too many times. It's not
1820−complicated, we just wanted to spare a premature discussion of `++make` and the
1889+complicated, we just wanted to spare a premature discussion of `++make`
1821−`%bake` silk. We 're now able to discuss everything in `++fade` with ease.
1890+and the `%bake` silk. We 're now able to discuss everything in `++fade`
1891+with ease.
1822
1892
1823 ++ fade :: compile to hood
1893 ++ fade :: compile to hood
1824 |= [cof=cafe for=mark bem=beam]
1894 |= [cof=cafe for=mark bem=beam]
@@ -1837,17 +1907,18 @@ complicated, we just wanted to spare a premature discussion of `++make` and the
1837 (flaw cof [%leaf "syntax error: {<p.p.vex>} {<q.p.vex>}"] ~)
1907 (flaw cof [%leaf "syntax error: {<p.p.vex>} {<q.p.vex>}"] ~)
1838 (fine cof p.u.q.vex)
1908 (fine cof p.u.q.vex)
1839
1909
1840−We first push a line onto a stack trace to say that we're parsing into a hood
1910+We first push a line onto a stack trace to say that we're parsing into a
1841−file.
1911+hood file.
1842
1912
1843−We bake the given beam with the given mark and no heel. Recall that baking gate,
1913+We bake the given beam with the given mark and no heel. Recall that
1844−core, door, hoon, and hook files produces simply an atom of the text. We check
1914+baking gate, core, door, hoon, and hook files produces simply an atom of
1845−to make sure that our value is an atom, failing otherwise.
1915+the text. We check to make sure that our value is an atom, failing
1916+otherwise.
1846
1917
1847−The parsing step is run within `++clef` so that the result is cached. We call
1918+The parsing step is run within `++clef` so that the result is cached. We
1848−`++fair` with the current beam to generate the parsing rule, and we parse the
1919+call `++fair` with the current beam to generate the parsing rule, and we
1849−file. If parsing fails, we fail giving a syntax error with the line and column
1920+parse the file. If parsing fails, we fail giving a syntax error with the
1850−number. Otherwise, we produce the value.
1921+line and column number. Otherwise, we produce the value.
1851
1922
1852 ++ liar :: load vase
1923 ++ liar :: load vase
1853 |= [cof=cafe bem=beam]
1924 |= [cof=cafe bem=beam]
@@ -1859,14 +1930,15 @@ number. Otherwise, we produce the value.
1859 (flaw cof (smyt (tope bem)) ~)
1930 (flaw cof (smyt (tope bem)) ~)
1860 (fine cof ?^(u.u.von [%cell %noun %noun] [%atom %$]) u.u.von)
1931 (fine cof ?^(u.u.von [%cell %noun %noun] [%atom %$]) u.u.von)
1861
1932
1862−This takes a beam and loads the file at that location. If our sky function
1933+This takes a beam and loads the file at that location. If our sky
1863−produces null, that means the resource is currently unavailable, so we block on
1934+function produces null, that means the resource is currently
1864−it. If it produces `[~ ~]`, that means our resource is permanently unavailable,
1935+unavailable, so we block on it. If it produces `[~ ~]`, that means our
1865−so we produce an error. Otherwise, we produce the value there with a type of
1936+resource is permanently unavailable, so we produce an error. Otherwise,
1866−either a cell of two nouns or an atom, depending on whether the value is a cell
1937+we produce the value there with a type of either a cell of two nouns or
1867−or not.
1938+an atom, depending on whether the value is a cell or not.
1868
1939
1869−Back in `++lima`, recall that we call `++lion` to find a translation path.
1940+Back in `++lima`, recall that we call `++lion` to find a translation
1941+path.
1870
1942
1871 ++ lion :: translation search
1943 ++ lion :: translation search
1872 |= [cof=cafe too=@tas bek=beak fro=(list ,@tas)]
1944 |= [cof=cafe too=@tas bek=beak fro=(list ,@tas)]
@@ -1901,38 +1973,41 @@ Back in `++lima`, recall that we call `++lion` to find a translation path.
1901 (fine cof ?~(ked ~ [~ for u.ked]))
1973 (fine cof ?~(ked ~ [~ for u.ked]))
1902 --
1974 --
1903
1975
1904−At a high level, we have `++apex` and `++apse`. `++apex` takes a list of marks
1976+At a high level, we have `++apex` and `++apse`. `++apex` takes a list of
1905−to try in succession until we find one that can be translated into the target
1977+marks to try in succession until we find one that can be translated into
1906−mark. On each one, it calls `++apse`, which takes a single mark and tries to
1978+the target mark. On each one, it calls `++apse`, which takes a single
1907−find a translation path from this mark to the target. To do this, it sees which
1979+mark and tries to find a translation path from this mark to the target.
1908−marks we know how to directly translate to, and calls `++apex` on this list. The
1980+To do this, it sees which marks we know how to directly translate to,
1909−result of this mututal recursion is a depth-first search of the translation
1981+and calls `++apex` on this list. The result of this mututal recursion is
1910−graph to find the target mark. Since the translation graph is not necessarily
1982+a depth-first search of the translation graph to find the target mark.
1911−acyclic, we maintain a set of marks that we've already tried.
1983+Since the translation graph is not necessarily acyclic, we maintain a
1984+set of marks that we've already tried.
1912
1985
1913−We kick off our search in `++apex`, starting with the given initial list of
1986+We kick off our search in `++apex`, starting with the given initial list
1914−marks that we know how to get to.
1987+of marks that we know how to get to.
1915
1988
1916−If `++apex` is called with a bolt other than a `%0` value bolt, we simply
1989+If `++apex` is called with a bolt other than a `%0` value bolt, we
1917−produce it. Otherwise, we check to see if the list of available marks to
1990+simply produce it. Otherwise, we check to see if the list of available
1918−investigate is null. If so, then we're done, so we produce a `%0` bolt with a
1991+marks to investigate is null. If so, then we're done, so we produce a
1919−null list of accessible marks.
1992+`%0` bolt with a null list of accessible marks.
1920
1993
1921Otherwise, we process this next mark with `++apse`, which will produce a
1994Otherwise, we process this next mark with `++apse`, which will produce a
1922−possible list of marks from this one to the target mark. If it fails to produce
1995+possible list of marks from this one to the target mark. If it fails to
1923−a `%0` bolt, we just produce that. Otherwise, if it produces null, we can't get
1996+produce a `%0` bolt, we just produce that. Otherwise, if it produces
1924−to our target through this mark, so we move on to the next one.
1997+null, we can't get to our target through this mark, so we move on to the
1998+next one.
1925
1999
1926−If it doesn't produce null, then we have successfully found a translation path,
2000+If it doesn't produce null, then we have successfully found a
1927−so we produce it.
2001+translation path, so we produce it.
1928
2002
1929−In `++apse`, we first test to see if we've arrived at the target path. If so,
2003+In `++apse`, we first test to see if we've arrived at the target path.
1930−we're done, so we produce a list including just ourself. Otherwise, we check to
2004+If so, we're done, so we produce a list including just ourself.
1931−see if we've already tried this mark. If so, we know we can't succeed here, so
2005+Otherwise, we check to see if we've already tried this mark. If so, we
1932−we produce null. Otherwise, we put ourselves in the set of already-tried marks,
2006+know we can't succeed here, so we produce null. Otherwise, we put
1933−and we move on.
2007+ourselves in the set of already-tried marks, and we move on.
1934
2008
1935−We call `++lily` to get the list of marks we can translate this one into.
2009+We call `++lily` to get the list of marks we can translate this one
2010+into.
1936
2011
1937 ++ lily :: translation targets
2012 ++ lily :: translation targets
1938 |= [cof=cafe for=mark bek=beak]
2013 |= [cof=cafe for=mark bek=beak]
@@ -1951,27 +2026,29 @@ We call `++lily` to get the list of marks we can translate this one into.
1951 =+ gow=(slap vax [%cnzy %grow])
2026 =+ gow=(slap vax [%cnzy %grow])
1952 (sloe p.gow)
2027 (sloe p.gow)
1953
2028
1954−We call `++fang` to get the mark definition door. This is documented under
2029+We call `++fang` to get the mark definition door. This is documented
1955−`%vale`. If getting the mark fails, we produce null because we can't translate a
2030+under `%vale`. If getting the mark fails, we produce null because we
1956−non-existent mark into anything.
2031+can't translate a non-existent mark into anything.
1957
2032
1958−Otherwise, we examine the door. The door may have a `++garb`, which is simply a
2033+Otherwise, we examine the door. The door may have a `++garb`, which is
1959−list of marks which know how to translate from the current one. There must be a
2034+simply a list of marks which know how to translate from the current one.
1960−corresponding `++grab` in the definition of the other mark, though we don't
2035+There must be a corresponding `++grab` in the definition of the other
1961−check that here.
2036+mark, though we don't check that here.
1962
2037
1963−The door may also have a `++grow`, which defines how to translate this mark into
2038+The door may also have a `++grow`, which defines how to translate this
1964−another one. Each arm in `++grow` is the name of a mark we can translate into.
2039+mark into another one. Each arm in `++grow` is the name of a mark we can
1965−The call to `++sloe` simply produces a list of arm names in `++grow`.
2040+translate into. The call to `++sloe` simply produces a list of arm names
2041+in `++grow`.
1966
2042
1967−Back in `++apse:lion`, we take the list of translation targets we just found and
2043+Back in `++apse:lion`, we take the list of translation targets we just
1968−call `++apex` on it. If we got back a null, we produce a null; otherwise, we
2044+found and call `++apex` on it. If we got back a null, we produce a null;
1969−produce the list of marks we got back plus the current mark.
2045+otherwise, we produce the list of marks we got back plus the current
2046+mark.
1970
2047
1971This concludes our discussion of `++lion`.
2048This concludes our discussion of `++lion`.
1972
2049
1973−The final piece of `++lima` is `++lope`, which performs the actual translation
2050+The final piece of `++lima` is `++lope`, which performs the actual
1974−along the path we just computed.
2051+translation along the path we just computed.
1975
2052
1976 ++ lope :: translation pipe
2053 ++ lope :: translation pipe
1977 |= [cof=cafe for=mark yaw=(list mark) bek=beak vax=vase]
2054 |= [cof=cafe for=mark yaw=(list mark) bek=beak vax=vase]
@@ -1981,12 +2058,14 @@ along the path we just computed.
1981 |= [cof=cafe yed=vase]
2058 |= [cof=cafe yed=vase]
1982 ^$(cof cof, for i.yaw, yaw t.yaw, vax yed)
2059 ^$(cof cof, for i.yaw, yaw t.yaw, vax yed)
1983
2060
1984−We iterate through our list, calling `++link` on every adjacent pair of marks,
2061+We iterate through our list, calling `++link` on every adjacent pair of
1985−translating from one mark to the next until we finish the list of marks. A call
2062+marks, translating from one mark to the next until we finish the list of
1986−to `++link` is equivalent to a `%cast` silk, so we document it there. After
2063+marks. A call to `++link` is equivalent to a `%cast` silk, so we
1987−we've called performed every step in the translation pipeline, we're done.
2064+document it there. After we've called performed every step in the
2065+translation pipeline, we're done.
1988
2066
1989−## Lifecycle of a `%boil`
2067+Lifecycle of a `%boil`
2068+----------------------
1990
2069
1991 %boil
2070 %boil
1992 %+ cool |.(leaf/"ford: boil {<p.kas>} {<(tope q.kas)>} {<r.kas>}")
2071 %+ cool |.(leaf/"ford: boil {<p.kas>} {<(tope q.kas)>} {<r.kas>}")
@@ -1996,13 +2075,13 @@ we've called performed every step in the translation pipeline, we're done.
1996 |= [cof=cafe vax=vase]
2075 |= [cof=cafe vax=vase]
1997 (fine cof `cage`[p.kas vax])
2076 (fine cof `cage`[p.kas vax])
1998
2077
1999−At a high level, we try to bake at the given beam, and if it fails, we go up a
2078+At a high level, we try to bake at the given beam, and if it fails, we
2000−level and try again. This is the usual semantics of ford, and this should nearly
2079+go up a level and try again. This is the usual semantics of ford, and
2001−always be preferred over directly baking.
2080+this should nearly always be preferred over directly baking.
2002
2081
2003−First, we normalize the version case to a number with `++lamp`. This allows
2082+First, we normalize the version case to a number with `++lamp`. This
2004−caching to be based on revision number rather than something more ephemeral like
2083+allows caching to be based on revision number rather than something more
2005−a particular time.
2084+ephemeral like a particular time.
2006
2085
2007 ++ lamp :: normalize version
2086 ++ lamp :: normalize version
2008 |= [cof=cafe bem=beam]
2087 |= [cof=cafe bem=beam]
@@ -2011,10 +2090,11 @@ a particular time.
2011 ?~ von [p=cof q=[%1 [bem ~] ~ ~]]
2090 ?~ von [p=cof q=[%1 [bem ~] ~ ~]]
2012 (fine cof bem(r [%ud ((hard ,@) (need u.von))]))
2091 (fine cof bem(r [%ud ((hard ,@) (need u.von))]))
2013
2092
2014−We call the sky function with `%cw`, asking clay for the revision number at this
2093+We call the sky function with `%cw`, asking clay for the revision number
2015−case. If the case refers to a revision that isn't there yet, we produce a `%1`
2094+at this case. If the case refers to a revision that isn't there yet, we
2016−blocking bolt. Otherwise, we require that the value exist and that it's a
2095+produce a `%1` blocking bolt. Otherwise, we require that the value exist
2017−number, both of which are guaranteed by clay. We produce this number.
2096+and that it's a number, both of which are guaranteed by clay. We produce
2097+this number.
2018
2098
2019Next for `%boil` we call `++lime` to try to load the beam.
2099Next for `%boil` we call `++lime` to try to load the beam.
2020
2100
@@ -2029,13 +2109,14 @@ Next for `%boil` we call `++lime` to try to load the beam.
2029 (flaw cof (smyt (tope bem)) ~)
2109 (flaw cof (smyt (tope bem)) ~)
2030 ^$(s.mob t.s.mob, mer [i.s.mob mer])
2110 ^$(s.mob t.s.mob, mer [i.s.mob mer])
2031
2111
2032−We start at the given beam and try to bake it. If it succeeds, we're good.
2112+We start at the given beam and try to bake it. If it succeeds, we're
2033−Otherwise, we pop off the top level of the path and put it in our heel (virtual
2113+good. Otherwise, we pop off the top level of the path and put it in our
2034−path extension). We do this recursively until either we find something we can
2114+heel (virtual path extension). We do this recursively until either we
2035−bake or we've gone all the way up to the root path of the desk, in which case we
2115+find something we can bake or we've gone all the way up to the root path
2036−fail.
2116+of the desk, in which case we fail.
2037
2117
2038−## Lifecycle of a `%call`
2118+Lifecycle of a `%call`
2119+----------------------
2039
2120
2040 %call
2121 %call
2041 %+ cool |.(leaf/"ford: call {<`@p`(mug kas)>}")
2122 %+ cool |.(leaf/"ford: call {<`@p`(mug kas)>}")
@@ -2053,13 +2134,15 @@ fail.
2053 (fine cof %noun vax)
2134 (fine cof %noun vax)
2054 ==
2135 ==
2055
2136
2056−This is slam for silks. We process both of the given silks in parallel with
2137+This is slam for silks. We process both of the given silks in parallel
2057−`++coax`. We then slam the two produced vases together with `++maul` and mark
2138+with `++coax`. We then slam the two produced vases together with
2058−the produced vase with `%noun` since we don't know any more specific mark.
2139+`++maul` and mark the produced vase with `%noun` since we don't know any
2140+more specific mark.
2059
2141
2060`++coax` is documented under Lifecycle of a Cell.
2142`++coax` is documented under Lifecycle of a Cell.
2061
2143
2062−## Lifecycle of a `%cast`
2144+Lifecycle of a `%cast`
2145+----------------------
2063
2146
2064 %cast
2147 %cast
2065 %+ cool |.(leaf/"ford: cast {<p.kas>}")
2148 %+ cool |.(leaf/"ford: cast {<p.kas>}")
@@ -2069,11 +2152,11 @@ the produced vase with `%noun` since we don't know any more specific mark.
2069 |= [cof=cafe vax=vase]
2152 |= [cof=cafe vax=vase]
2070 (fine cof [p.kas vax])
2153 (fine cof [p.kas vax])
2071
2154
2072−This is a request to convert data of one mark to another mark directly. We
2155+This is a request to convert data of one mark to another mark directly.
2073−evaluate the given silk and pass the result into `++link`, which performs the
2156+We evaluate the given silk and pass the result into `++link`, which
2074−actual translation. Note that this will not search for indirect conversion
2157+performs the actual translation. Note that this will not search for
2075−paths, so the conversion must be defined either in the `++grow` of the given
2158+indirect conversion paths, so the conversion must be defined either in
2076−mark or the `++grab` of the target mark.
2159+the `++grow` of the given mark or the `++grab` of the target mark.
2077
2160
2078 ++ link :: translate
2161 ++ link :: translate
2079 |= [cof=cafe too=mark for=mark bek=beak vax=vase]
2162 |= [cof=cafe too=mark for=mark bek=beak vax=vase]
@@ -2098,28 +2181,29 @@ mark or the `++grab` of the target mark.
2098 (flaw cof [%leaf "ford: no link: {<[for too]>}"]~)
2181 (flaw cof [%leaf "ford: no link: {<[for too]>}"]~)
2099 (maul cof u.zat vax)
2182 (maul cof u.zat vax)
2100
2183
2101−This performs one step in the translation pipeline. If the given and target
2184+This performs one step in the translation pipeline. If the given and
2102−marks are the same, we're done. If we're translating from a noun or the empty
2185+target marks are the same, we're done. If we're translating from a noun
2103−mark, we coerce with `++lake` (documented in `%vale`). Otherwise, we're
2186+or the empty mark, we coerce with `++lake` (documented in `%vale`).
2104−translating from a user-defined mark.
2187+Otherwise, we're translating from a user-defined mark.
2105
2188
2106−We load the definition of the given mark with `++fang`, and we check to see if
2189+We load the definition of the given mark with `++fang`, and we check to
2107−it has an arm in `++grow` named the target mark. If so, we place our data in the
2190+see if it has an arm in `++grow` named the target mark. If so, we place
2108−sample of the door with `++keel` and slap the arm. `++keel` is equivalent to a
2191+our data in the sample of the door with `++keel` and slap the arm.
2109−`%mute` silk, so we document it there.
2192+`++keel` is equivalent to a `%mute` silk, so we document it there.
2110
2193
2111−If there is no arm in `++grow` of the given mark named the target mark, we
2194+If there is no arm in `++grow` of the given mark named the target mark,
2112−suppose there must be an arm in `++grab` of the target mark named the given
2195+we suppose there must be an arm in `++grab` of the target mark named the
2113−mark. We get the definition of the target mark and check to see if it has the
2196+given mark. We get the definition of the target mark and check to see if
2114−required arm, failing if it doesn't. Finally, we slam the data against the
2197+it has the required arm, failing if it doesn't. Finally, we slam the
2115−correct arm, producing the translated data.
2198+data against the correct arm, producing the translated data.
2116
2199
2117If you're confused as to why the handling of `++grow` and `++grab` look
2200If you're confused as to why the handling of `++grow` and `++grab` look
2118−superficially so different, remember that the correct arm in `++grow` does not
2201+superficially so different, remember that the correct arm in `++grow`
2119−have a sample while the one in `++grab` does. This means they must be called
2202+does not have a sample while the one in `++grab` does. This means they
2120−rather differently.
2203+must be called rather differently.
2121
2204
2122−## Lifecycle of a `%diff`
2205+Lifecycle of a `%diff`
2206+----------------------
2123
2207
2124 %diff
2208 %diff
2125 %+ cool |.(leaf/"ford: diff {<`@p`(mug p.kas)>} {<`@p`(mug q.kas)>}")
2209 %+ cool |.(leaf/"ford: diff {<`@p`(mug p.kas)>} {<`@p`(mug q.kas)>}")
@@ -2144,7 +2228,8 @@ First, we process the two given silks to get our arguments.
2144 %+ flaw cof :_ ~
2228 %+ flaw cof :_ ~
2145 leaf/"diff on data of different marks: {(trip p.cay)} {(trip p.coy)}"
2229 leaf/"diff on data of different marks: {(trip p.cay)} {(trip p.coy)}"
2146
2230
2147−If the two cages have different marks, then we can't diff them, so we complain.
2231+If the two cages have different marks, then we can't diff them, so we
2232+complain.
2148
2233
2149 %+ cope (fang cof p.cay [our %main %da now])
2234 %+ cope (fang cof p.cay [our %main %da now])
2150 |= [cof=cafe pro=vase]
2235 |= [cof=cafe pro=vase]
@@ -2162,11 +2247,12 @@ We pull in the relevant mark's definition.
2162 (flaw cof leaf/"bad ++sted:grad" ~)
2247 (flaw cof leaf/"bad ++sted:grad" ~)
2163 (make cof %diff [%cast u.for kas] [%cast u.for kos])
2248 (make cof %diff [%cast u.for kas] [%cast u.for kos])
2164
2249
2165−If there's no `++grad`, we complain. If there's no `++form:grad`, then we look
2250+If there's no `++grad`, we complain. If there's no `++form:grad`, then
2166−for a `++sted:grad`. If we can't find either, or if `++sted:grad` isn't a term,
2251+we look for a `++sted:grad`. If we can't find either, or if
2167−then we complain. If `++sted:grad` exists and is a term, then it represents the
2252+`++sted:grad` isn't a term, then we complain. If `++sted:grad` exists
2168−mark we should use as a proxy to get our diff. So, we cast both our given cages
2253+and is a term, then it represents the mark we should use as a proxy to
2169−to the new mark and start the dance again.
2254+get our diff. So, we cast both our given cages to the new mark and start
2255+the dance again.
2170
2256
2171 ?. (slab %diff p.gar)
2257 ?. (slab %diff p.gar)
2172 (flaw cof leaf/"no ++diff:grad" ~)
2258 (flaw cof leaf/"no ++diff:grad" ~)
@@ -2176,7 +2262,8 @@ Otherwise, we expect a `++diff:grad`.
2176 %+ cope (keel cof pro [[%& 6]~ q.cay]~)
2262 %+ cope (keel cof pro [[%& 6]~ q.cay]~)
2177 |= [cof=cafe pox=vase]
2263 |= [cof=cafe pox=vase]
2178
2264
2179−We put the first cage's data into the sample of the given mark's definition.
2265+We put the first cage's data into the sample of the given mark's
2266+definition.
2180
2267
2181 %+ cope
2268 %+ cope
2182 %^ maul cof
2269 %^ maul cof
@@ -2192,41 +2279,46 @@ We run `++diff:grad` with a sample of the second cage's data.
2192 (fine cof u.for dif)
2279 (fine cof u.for dif)
2193 ==
2280 ==
2194
2281
2195−We check that `++form:grad` exists, and we tag the result with it to give the
2282+We check that `++form:grad` exists, and we tag the result with it to
2196−final cage.
2283+give the final cage.
2197
2284
2198−## Lifecycle of a `%done`
2285+Lifecycle of a `%done`
2286+----------------------
2199
2287
2200 %done [cof %0 p.kas q.kas]
2288 %done [cof %0 p.kas q.kas]
2201
2289
2202This is trivial. We simply produce the given cage with the given set of
2290This is trivial. We simply produce the given cage with the given set of
2203−dependencies. This is used when we already have a cage that we want to insert
2291+dependencies. This is used when we already have a cage that we want to
2204−into another silk that requires a silk argument. It's analogous to the return
2292+insert into another silk that requires a silk argument. It's analogous
2205−operator in a monad -- which makes it sound way more complicated than it is.
2293+to the return operator in a monad -- which makes it sound way more
2294+complicated than it is.
2206
2295
2207−## Lifecycle of a `%dude`
2296+Lifecycle of a `%dude`
2297+----------------------
2208
2298
2209 %dude (cool |.(p.kas) $(kas q.kas))
2299 %dude (cool |.(p.kas) $(kas q.kas))
2210
2300
2211−This simply puts a given tank on the stack trace if the given silk produces an
2301+This simply puts a given tank on the stack trace if the given silk
2212−error. This is implemented as a simple call to `++cool`.
2302+produces an error. This is implemented as a simple call to `++cool`.
2213
2303
2214−## Lifecycle of a `%dune`
2304+Lifecycle of a `%dune`
2305+----------------------
2215
2306
2216 %dune
2307 %dune
2217 ?~ q.kas [cof [%2 [%leaf "no data"]~]]
2308 ?~ q.kas [cof [%2 [%leaf "no data"]~]]
2218 $(kas [%done p.kas u.q.kas])
2309 $(kas [%done p.kas u.q.kas])
2219
2310
2220−This is a sort of a `++need` for silks. If there is no data in the unit cage, we
2311+This is a sort of a `++need` for silks. If there is no data in the unit
2221−produce an error. Else, we simply produce the data in the cage.
2312+cage, we produce an error. Else, we simply produce the data in the cage.
2222
2313
2223−## Lifcycle of a `%mute`
2314+Lifcycle of a `%mute`
2315+---------------------
2224
2316
2225 %mute (kale cof p.kas q.kas)
2317 %mute (kale cof p.kas q.kas)
2226
2318
2227−This mutates a silk by putting the values of other silks at particular axes.
2319+This mutates a silk by putting the values of other silks at particular
2228−This is useful in, for example, replacing the sample of the door in a mark
2320+axes. This is useful in, for example, replacing the sample of the door
2229−definition.
2321+in a mark definition.
2230
2322
2231 ++ kale :: mutate
2323 ++ kale :: mutate
2232 |= [cof=cafe kas=silk muy=(list (pair wing silk))]
2324 |= [cof=cafe kas=silk muy=(list (pair wing silk))]
@@ -2246,10 +2338,10 @@ definition.
2246 |= [cof=cafe vax=vase]
2338 |= [cof=cafe vax=vase]
2247 (fine cof p.cay vax)
2339 (fine cof p.cay vax)
2248
2340
2249−First, we process each of the silks by calling `++make` on them. We pass the
2341+First, we process each of the silks by calling `++make` on them. We pass
2250−resultant vase and list of pairs of wings and silks to `++keel` to do the actual
2342+the resultant vase and list of pairs of wings and silks to `++keel` to
2251−mutation. We assume the mutation doesn't change the mark of the main silk, so we
2343+do the actual mutation. We assume the mutation doesn't change the mark
2252−mark the produced vase with the original mark.
2344+of the main silk, so we mark the produced vase with the original mark.
2253
2345
2254 ++ keel :: apply mutations
2346 ++ keel :: apply mutations
2255 |= [cof=cafe suh=vase yom=(list (pair wing vase))]
2347 |= [cof=cafe suh=vase yom=(list (pair wing vase))]
@@ -2267,16 +2359,19 @@ mark the produced vase with the original mark.
2267 :- [p.i.yom [%$ (peg axe 2)]]
2359 :- [p.i.yom [%$ (peg axe 2)]]
2268 $(yom t.yom, axe (peg axe 3))
2360 $(yom t.yom, axe (peg axe 3))
2269
2361
2270−We first put the vases together in one big tuple starting with the subject and
2362+We first put the vases together in one big tuple starting with the
2271−going through the mutations. We slap against this tuple a `%_` twig we directly
2363+subject and going through the mutations. We slap against this tuple a
2272−construct. Since a `%_` twig takes a list of pairs of wings and twigs, we simply
2364+`%_` twig we directly construct. Since a `%_` twig takes a list of pairs
2273−have to generate twigs referring to the correct axes in the subject. This is
2365+of wings and twigs, we simply have to generate twigs referring to the
2274−very easy since we just recur on axis 3 of whatever axis we were already at.
2366+correct axes in the subject. This is very easy since we just recur on
2367+axis 3 of whatever axis we were already at.
2275
2368
2276−Note the use of `%_` instead of `%=` enforces that our mutations don't change
2369+Note the use of `%_` instead of `%=` enforces that our mutations don't
2277−the type of the subject, which justifies our use of the original mark.
2370+change the type of the subject, which justifies our use of the original
2371+mark.
2278
2372
2279−## Lifecycle of a `%pact`
2373+Lifecycle of a `%pact`
2374+----------------------
2280
2375
2281 %pact
2376 %pact
2282 %+ cool |.(leaf/"ford: pact {<`@p`(mug p.kas)>} {<`@p`(mug q.kas)>}")
2377 %+ cool |.(leaf/"ford: pact {<`@p`(mug p.kas)>} {<`@p`(mug q.kas)>}")
@@ -2313,12 +2408,12 @@ We pull in the relevant mark's definition.
2313 (flaw cof leaf/"bad ++sted:grad" ~)
2408 (flaw cof leaf/"bad ++sted:grad" ~)
2314 (make cof %cast p.cay %pact [%cast u.for kas] kos)
2409 (make cof %cast p.cay %pact [%cast u.for kas] kos)
2315
2410
2316−If there's no `++grad`, we complain. If there's no `++form:grad`, then we look
2411+If there's no `++grad`, we complain. If there's no `++form:grad`, then
2317−for a `++sted:grad`. If we can't find either, or if `++sted:grad` isn't a term,
2412+we look for a `++sted:grad`. If we can't find either, or if
2318−then we complain. If `++sted:grad` exists and is a term, then it represents the
2413+`++sted:grad` isn't a term, then we complain. If `++sted:grad` exists
2319−mark we should use as a proxy to get our diff. So, we cast the first argument to
2414+and is a term, then it represents the mark we should use as a proxy to
2320−the new mark, then try to patch. Afterward, we cast the result back to the
2415+get our diff. So, we cast the first argument to the new mark, then try
2321−original mark.
2416+to patch. Afterward, we cast the result back to the original mark.
2322
2417
2323 =+ for=((soft ,@tas) q:(slap gar [%cnzy %form]))
2418 =+ for=((soft ,@tas) q:(slap gar [%cnzy %form]))
2324 ?~ for
2419 ?~ for
@@ -2328,8 +2423,8 @@ original mark.
2328 =< leaf/"pact on data with wrong form: {-} {+<} {+>}"
2423 =< leaf/"pact on data with wrong form: {-} {+<} {+>}"
2329 [(trip p.cay) (trip u.for) (trip p.coy)]
2424 [(trip p.cay) (trip u.for) (trip p.coy)]
2330
2425
2331−If `++form:grad` isn't a term, or else our second argument isn't of that mark,
2426+If `++form:grad` isn't a term, or else our second argument isn't of that
2332−we complain.
2427+mark, we complain.
2333
2428
2334 ?. (slab %pact p.gar)
2429 ?. (slab %pact p.gar)
2335 (flaw cof leaf/"no ++pact:grad" ~)
2430 (flaw cof leaf/"no ++pact:grad" ~)
@@ -2339,7 +2434,8 @@ If we don't have a `++pact:grad`, we complain.
2339 %+ cope (keel cof pro [[%& 6]~ q.cay]~)
2434 %+ cope (keel cof pro [[%& 6]~ q.cay]~)
2340 |= [cof=cafe pox=vase]
2435 |= [cof=cafe pox=vase]
2341
2436
2342−We put the first cage's data into the sample of the given mark's definition.
2437+We put the first cage's data into the sample of the given mark's
2438+definition.
2343
2439
2344 %+ cope
2440 %+ cope
2345 %^ maul cof
2441 %^ maul cof
@@ -2347,32 +2443,36 @@ We put the first cage's data into the sample of the given mark's definition.
2347 q.coy
2443 q.coy
2348 |= [cof=cafe pat=vase]
2444 |= [cof=cafe pat=vase]
2349
2445
2350−We run `++pact:grad` with a sample of the second cage's data, which is the diff.
2446+We run `++pact:grad` with a sample of the second cage's data, which is
2447+the diff.
2351
2448
2352 (fine cof p.cay pat)
2449 (fine cof p.cay pat)
2353 ==
2450 ==
2354
2451
2355We tag the result with the mark of our first argument.
2452We tag the result with the mark of our first argument.
2356
2453
2357−## Lifecycle of a `%plan`
2454+Lifecycle of a `%plan`
2455+----------------------
2358
2456
2359 %plan
2457 %plan
2360 %+ cope (abut:(meow p.kas q.kas) cof r.kas)
2458 %+ cope (abut:(meow p.kas q.kas) cof r.kas)
2361 |= [cof=cafe vax=vase]
2459 |= [cof=cafe vax=vase]
2362 (fine cof %noun vax)
2460 (fine cof %noun vax)
2363
2461
2364−This is a direct request to compile a hood at a given beam with a heel of the
2462+This is a direct request to compile a hood at a given beam with a heel
2365−given path. We comply by calling `++abut` with the given arguments and producing
2463+of the given path. We comply by calling `++abut` with the given
2366−the vase with a mark of `%noun`.
2464+arguments and producing the vase with a mark of `%noun`.
2367
2465
2368−## Lifecycle of a `%reef`
2466+Lifecycle of a `%reef`
2467+----------------------
2369
2468
2370 %reef (fine cof %noun pit)
2469 %reef (fine cof %noun pit)
2371
2470
2372−This is one of the simplest silks. We simply produce our context, which is zuse
2471+This is one of the simplest silks. We simply produce our context, which
2373−compiled against hoon. The mark is a `%noun`.
2472+is zuse compiled against hoon. The mark is a `%noun`.
2374
2473
2375−## Lifcycle of a `%ride`
2474+Lifcycle of a `%ride`
2475+---------------------
2376
2476
2377 %ride
2477 %ride
2378 %+ cool |.(leaf/"ford: ride {<`@p`(mug kas)>}")
2478 %+ cool |.(leaf/"ford: ride {<`@p`(mug kas)>}")
@@ -2382,11 +2482,12 @@ compiled against hoon. The mark is a `%noun`.
2382 |= [cof=cafe vax=vase]
2482 |= [cof=cafe vax=vase]
2383 (fine cof %noun vax)
2483 (fine cof %noun vax)
2384
2484
2385−This slaps evaluates the given silk, then it slaps the result against the given
2485+This slaps evaluates the given silk, then it slaps the result against
2386−twig. Since we don't know what of what mark (if any) is the result, we give it a
2486+the given twig. Since we don't know what of what mark (if any) is the
2387−mark of `%noun`.
2487+result, we give it a mark of `%noun`.
2388
2488
2389−## Lifecycle of a `%vale`
2489+Lifecycle of a `%vale`
2490+----------------------
2390
2491
2391 %vale
2492 %vale
2392 %+ cool |.(leaf/"ford: vale {<p.kas>} {<q.kas>} {<`@p`(mug r.kas)>}")
2493 %+ cool |.(leaf/"ford: vale {<p.kas>} {<q.kas>} {<`@p`(mug r.kas)>}")
@@ -2394,20 +2495,20 @@ mark of `%noun`.
2394 |= [cof=cafe vax=vase]
2495 |= [cof=cafe vax=vase]
2395 (fine cof `cage`[p.kas vax])
2496 (fine cof `cage`[p.kas vax])
2396
2497
2397−This checks whether given data is of the given mark. If we don't have the
2498+This checks whether given data is of the given mark. If we don't have
2398−definition of the mark, we check the given ship for it.
2499+the definition of the mark, we check the given ship for it.
2399
2500
2400−We call `++lave` to perform the check, producing a vase. We produce this vase
2501+We call `++lave` to perform the check, producing a vase. We produce this
2401−tagged with the given mark.
2502+vase tagged with the given mark.
2402
2503
2403 ++ lave :: validate
2504 ++ lave :: validate
2404 |= [cof=cafe for=mark his=ship som=*]
2505 |= [cof=cafe for=mark his=ship som=*]
2405 ^- (bolt vase)
2506 ^- (bolt vase)
2406 ((lake for [our %main [%da now]]) cof [%noun som])
2507 ((lake for [our %main [%da now]]) cof [%noun som])
2407
2508
2408−This is a thinly-veiled wrapper over `++lake`. Note that, contrary to documented
2509+This is a thinly-veiled wrapper over `++lake`. Note that, contrary to
2409−opinion, we do not in fact check the other ship's definition of a mark. This is
2510+documented opinion, we do not in fact check the other ship's definition
2410−likely a bug.
2511+of a mark. This is likely a bug.
2411
2512
2412At any rate, `++lake` coerces a noun into the correct type for a mark.
2513At any rate, `++lake` coerces a noun into the correct type for a mark.
2413
2514
@@ -2435,24 +2536,27 @@ At any rate, `++lake` coerces a noun into the correct type for a mark.
2435 (fine cof (slot 6 pro))
2536 (fine cof (slot 6 pro))
2436 (flaw cof [%leaf "ford: invalid content: {<[for bek]>}"]~)
2537 (flaw cof [%leaf "ford: invalid content: {<[for bek]>}"]~)
2437
2538
2438−This is going to coerce the sample into the correct type for the mark. First, we
2539+This is going to coerce the sample into the correct type for the mark.
2439−push a line onto the stack trace saying that we're checking the type. If the
2540+First, we push a line onto the stack trace saying that we're checking
2440−requested mark is a gate, core, door, hoon, or hook, then we don't do any more
2541+the type. If the requested mark is a gate, core, door, hoon, or hook,
2441−type information than just saying it's a noun, so we're done.
2542+then we don't do any more type information than just saying it's a noun,
2543+so we're done.
2442
2544
2443−Otherwise, we get the mark definition from our `/=main=/mar` directory with
2545+Otherwise, we get the mark definition from our `/=main=/mar` directory
2444−`++fang`, which we'll describe below.
2546+with `++fang`, which we'll describe below.
2445
2547
2446−We check to see if our sample type nests within the type of the sample to the
2548+We check to see if our sample type nests within the type of the sample
2447−door. If so, then we're already of the correct type, so we're done.
2549+to the door. If so, then we're already of the correct type, so we're
2550+done.
2448
2551
2449−Otherwise, we check to see if there's a `++grab` in the door, and a `++noun` in
2552+Otherwise, we check to see if there's a `++grab` in the door, and a
2450−the `++grab`. If not, there's no way we can translate to this mark, so we fail.
2553+`++noun` in the `++grab`. If not, there's no way we can translate to
2554+this mark, so we fail.
2451
2555
2452−If we have everything we need, we slam our sample (typed as a noun) against the
2556+If we have everything we need, we slam our sample (typed as a noun)
2453−`++noun` in `++grab`. If the sample of the door is the same as our sample, then
2557+against the `++noun` in `++grab`. If the sample of the door is the same
2454−the check succeeded, so we produce the well-typed sample of the door. Otherwise,
2558+as our sample, then the check succeeded, so we produce the well-typed
2455−we fail.
2559+sample of the door. Otherwise, we fail.
2456
2560
2457 ++ fang :: protocol door
2561 ++ fang :: protocol door
2458 |= [cof=cafe for=mark bek=beak]
2562 |= [cof=cafe for=mark bek=beak]
@@ -2467,6 +2571,6 @@ we fail.
2467 (cope (fade cof %hook bem) abut:(meow bem ~))
2571 (cope (fade cof %hook bem) abut:(meow bem ~))
2468
2572
2469A mark's definition is generally in
2573A mark's definition is generally in
2470−`/=main=/mar/[mark-name]/door/hook'. If we don't find it there, we look in`/\[given-beak\]/mar/\[mark-name\]/door/hook'.
2574+`/=main=/mar/[mark-name]/door/hook'. If we don't find it there, we look in`/[given-beak]/mar/[mark-name]/door/hook'.
2471−We parse the mark definition with `++fade` and assemble it with `++abut:meow`.
2575+We parse the mark definition with `++fade` and assemble it with
2472−`++fade` is defined under the `%bake` silk.
2576+`++abut:meow`. `++fade` is defined under the `%bake` silk.
docs/arvo/internals/ford/runes.md+241−238
@@ -2,37 +2,28 @@
2
2
3## ford runes
3## ford runes
4
4
5−One of the most common ways to use ford is through ford's runes, all of which
5+One of the most common ways to use ford is through ford's runes, all of which begin with `/` ("fas"). A ford rune runs a step of a ford build and places the result in the subject. There are various kinds of build steps, some of which take other build steps as parameters. The most common pattern is to have a sequence of ford runes at the top of a hoon source file that import the results of evaluating other hoon files. This is how we "import a library" in urbit: we add the result of compiling another hoon file into the current subject, possibly renaming it by wrapping a face around it.
6−begin with `/` ("fas"). A ford rune runs a step of a ford build and places the
6+
7−result in the subject. There are various kinds of build steps, some of which
7+Another common use case is assembling and rendering markdown templates into a single HTML file, to be sent as an HTTP response in urbit's web server `%eyre`. It's important to keep in mind that %ford is quite flexible and generic, and is used in other parts of urbit -- such as the dojo -- that would not traditionally fall under the purview of a build system.
8−take other build steps as parameters. The most common pattern is to have a
9−sequence of ford runes at the top of a hoon source file that import the results
10−of evaluating other hoon files. This is how we "import a library" in urbit: we
11−add the result of compiling another hoon file into the current subject, possibly
12−renaming it by wrapping a face around it.
13−
14−Another common use case is assembling and rendering markdown templates into a
15−single HTML file, to be sent as an HTTP response in urbit's web server `%eyre`.
16−It's important to keep in mind that %ford is quite flexible and generic, and is
17−used in other parts of urbit -- such as the dojo -- that would not traditionally
18−fall under the purview of a build system.
19
8
20### `/+` import from lib/
9### `/+` import from lib/
21
10
22−The `/+` rune accepts a filename as an argument. It interprets that filename as
11+The `/+` rune accepts a filename as an argument. It interprets that filename
23−a hoon source file within the `lib` directory. This is how we import a shared
12+as a hoon source file within the `lib` directory. This is how we import a shared
24library in urbit.
13library in urbit.
25
14
26To run this example, put this code in your desk at `gen/faslus.hoon` and run
15To run this example, put this code in your desk at `gen/faslus.hoon` and run
27`+faslus` in your dojo. This example is a generator. For more information on
16`+faslus` in your dojo. This example is a generator. For more information on
28generators, see the [generator docs](https://urbit.org/docs/arvo/generators).
17generators, see the [generator docs](https://urbit.org/docs/arvo/generators).
29
18
30− /+ time-to-id
19+```
31− ::
20+/+ time-to-id
32− :- %say
21+::
33− |= {now/@da * *} $~ $~}
22+:- %say
34− :- %noun
23+|= {now/@da * *} $~ $~}
35− (time-to-id now)
24+:- %noun
25+(time-to-id now)
26+```
36
27
37produces: `"c.314d"` (or something similar depending on when you run it)
28produces: `"c.314d"` (or something similar depending on when you run it)
38
29
@@ -41,14 +32,16 @@ with commas.
41
32
42Replace the code in `gen/faslus.hoon` with the following:
33Replace the code in `gen/faslus.hoon` with the following:
43
34
44− /+ time-to-id, hep-to-cab
35+```
45− ::
36+/+ time-to-id, hep-to-cab
46− :- %say
37+::
47− |= {{now/@da * *} $~ $~}
38+:- %say
48− :- %noun
39+|= {{now/@da * *} $~ $~}
49− =/ id (time-to-id now)
40+:- %noun
50− =/ str "my-id-is-{id}"
41+=/ id (time-to-id now)
51− (hep-to-cab (crip str))
42+=/ str "my-id-is-{id}"
43+(hep-to-cab (crip str))
44+```
52
45
53This should print something like `my_id_is_c.3588`.
46This should print something like `my_id_is_c.3588`.
54
47
@@ -58,7 +51,9 @@ load the library.
58
51
59Example:
52Example:
60
53
61− /+ time-to-id, hep-to-cab/4/~zod
54+```
55+/+ time-to-id, hep-to-cab/4/~zod
56+```
62
57
63will load the `hep-to-cab` library from `~zod` at `%clay` revision `4`.
58will load the `hep-to-cab` library from `~zod` at `%clay` revision `4`.
64
59
@@ -70,11 +65,11 @@ shared structures that can be used by other parts of urbit. This is somewhat
70similar to including a header file in C.
65similar to including a header file in C.
71
66
72Example:
67Example:
73−
68+```
74− /- talk
69+/- talk
75− ::
70+::
76− *serial:talk
71+*serial:talk
77−
72+```
78produces: `0v0`
73produces: `0v0`
79
74
80`/-` can also take multiple files as arguments, and the ship and case of those
75`/-` can also take multiple files as arguments, and the ship and case of those
@@ -89,52 +84,53 @@ the subject. Arbitrary hoon can be in the twig. A
89structure representing a `%ford` static resource.
84structure representing a `%ford` static resource.
90
85
91Example:
86Example:
87+```
88+/~ [a=0 b=1]
89+[a b]
90+```
91+produces:
92+`[0 1]`
92
93
93− /~ [a=0 b=1]
94+In wide-form, `/~` always takes a tuple (which may be a degenerate tuple of one element), and produces it.
94− [a b]
95−
96−produces: `[0 1]`
97−
98−In wide-form, `/~` always takes a tuple (which may be a degenerate tuple of one
99−element), and produces it.
100
95
101Example:
96Example:
102−
97+```
103− /~[%something]
98+/~[%something]
104−
99+```
105produces: `%something`
100produces: `%something`
106
101
107− /~[%a %b]
102+```
108−
103+/~[%a %b]
104+```
109produces: `[%a %b]`
105produces: `[%a %b]`
110
106
111### `//` include a file by relative path
107### `//` include a file by relative path
112
108
113`// <rel-path>` parses `rel-path` as a hoon twig, and then adds the resulting
109`// <rel-path>` parses `rel-path` as a hoon twig, and then adds the resulting
114twig to the subject. Note that the result type of this rune is not a horn, but
110twig to the subject. Note that the result type of this rune is not a horn, but
115−just a hoon twig, so its result can't be used as an argument to other runes that
111+just a hoon twig, so its result can't be used as an argument to other runes
116−expect a horn. There is no wide-form for this rune.
112+that expect a horn. There is no wide-form for this rune.
117
113
118Example:
114Example:
119−
115+```
120− // %/file-to-include
116+// %/file-to-include
121− ::
117+::
122− (frobnicate:file-to-include %.n)
118+(frobnicate:file-to-include %.n)
123−
119+
124− :: contents of file-to-include.hoon
120+:: contents of file-to-include.hoon
125− |%
121+|%
126− ++ frobnicate
122+++ frobnicate
127− |= a/? ^+ a
123+ |= a/? ^+ a
128− !a
124+ !a
129− --
125+--
130−
126+```
131produces: `%.y`
127produces: `%.y`
132
128
133−*Note: `%` means current directory, which in a hoon file will resolve to the
129+_Note: `%` means current directory, which in a hoon file will resolve to the
134"directory" containing that file. Also, %clay doesn't have first-class
130"directory" containing that file. Also, %clay doesn't have first-class
135directories the way unix filesystems do. Whereas in unix, a directory is a
131directories the way unix filesystems do. Whereas in unix, a directory is a
136special kind of file, in %clay it's just a path prefix, and there is no file
132special kind of file, in %clay it's just a path prefix, and there is no file
137−stored at that path.*
133+stored at that path._
138
134
139### `/=` wrap a face around an included horn
135### `/=` wrap a face around an included horn
140
136
@@ -142,42 +138,44 @@ stored at that path.*
142that horn, and wraps a face around it.
138that horn, and wraps a face around it.
143
139
144Example:
140Example:
145−
141+```
146− /= foo /~ [a=0 b=1]
142+/= foo /~ [a=0 b=1]
147− ::
143+::
148− [a.foo b.foo]
144+[a.foo b.foo]
149−
145+```
150produces: `[0 1]`
146produces: `[0 1]`
151
147
152In wide-form, `/=` uses `=` as a delimiter: `/=foo=/~[a=0 b=1]`
148In wide-form, `/=` uses `=` as a delimiter: `/=foo=/~[a=0 b=1]`
153
149
154### `/:` evaluate at path, and `/<mark>/` render mark at current path
150### `/:` evaluate at path, and `/<mark>/` render mark at current path
155
151
156−`/:` takes a path and a horn, and evaluates the horn with the current path set
152+`/:` takes a path and a horn, and evaluates the horn with the current path set to the supplied path.
157−to the supplied path. `/mar/` renders the mark 'mar' at the current path.
153+`/mar/` renders the mark 'mar' at the current path.
158
154
159Example:
155Example:
160−
156+```
161− /= hoo-source /: /path/to/hoon-file /hoon/
157+/= hoo-source /: /path/to/hoon-file /hoon/
162− ::
158+::
163− `@t`hoo-source
159+`@t`hoo-source
160+```
164
161
165produces the text of the hoon file at "/path/to/hoon-file/hoon".
162produces the text of the hoon file at "/path/to/hoon-file/hoon".
166
163
167`/hoon/` renders the current path using the `%hoon` mark, which in this case
164`/hoon/` renders the current path using the `%hoon` mark, which in this case
168−passes the contents through unchanged. In general, rendering a file with a mark
165+passes the contents through unchanged. In general, rendering a file with a
169−will potentially run the contents through a series of conversion operations. For
166+mark will potentially run the contents through a series of conversion
170−details on marks, see the [marks docs](https://urbit.org/docs/arvo/marks).
167+operations. For details on marks, see the
168+[marks docs](https://urbit.org/docs/arvo/marks).
171
169
172Here's an example that includes a mark conversion:
170Here's an example that includes a mark conversion:
173−
171+```
174− /= page /: /path/to/html/file /mime/
172+/= page /: /path/to/html/file /mime/
175− ::
173+::
176− page
174+page
177−
175+
178− :: contents of /path/to/html/file:
176+:: contents of /path/to/html/file:
179− <html><head><title>My Fascinating Blog</title></head></html>
177+<html><head><title>My Fascinating Blog</title></head></html>
180−
178+```
181produces:
179produces:
182`[[%text %html ~] 60 '<html><head><title>My Fascinating Blog</title></head></html>']`
180`[[%text %html ~] 60 '<html><head><title>My Fascinating Blog</title></head></html>']`
183
181
@@ -189,49 +187,51 @@ In wide-form, `/:` uses `:` as a delimiter: `/=page=/:/path/to/html/file:/mime/`
189### `/!mark/` evaluate as hoon, then pass through mark
187### `/!mark/` evaluate as hoon, then pass through mark
190
188
191Example:
189Example:
192−
190+```
193− /= mime /: /%/some-hoon-file /!mime/
191+/= mime /: /%/some-hoon-file /!mime/
194− ::
192+::
195− mime
193+mime
196−
194+
197− :: contents of /%/some-hoon-file:
195+:: contents of /%/some-hoon-file:
198− %produces-a-cord
196+%produces-a-cord
199−
197+```
200produces: `[[%text %plain ~] 15 'produces-a-cord']`
198produces: `[[%text %plain ~] 15 'produces-a-cord']`
201
199
202### `/&` pass through a series of marks
200### `/&` pass through a series of marks
203
201
204`/&` passes a horn through multiple marks, right-to-left. It has both a
202`/&` passes a horn through multiple marks, right-to-left. It has both a
205wide-form and a tall-form syntax. In wide-form, it takes a series of mark
203wide-form and a tall-form syntax. In wide-form, it takes a series of mark
206−arguments followed by a horn. In tall-form, it takes a single mark followed by a
204+arguments followed by a horn. In tall-form, it takes a single mark followed by
207−horn.
205+a horn.
208−
206+
209− /= some-text /: /%/text-file /&mime&/txt/
207+```
210− ::
208+/= some-text /: /%/text-file /&mime&/txt/
211− some-text
209+::
212−
210+some-text
211+```
213produces: `[[%text %plain ~] p=17 q='Hi I\'m some text\0a']`
212produces: `[[%text %plain ~] p=17 q='Hi I\'m some text\0a']`
214
213
215−This example shows two of the ways marks are used. The first way is what happens
214+This example shows two of the ways marks are used. The first way is what
216−with the `/txt/` mark: we use it to find a file in clay with that extension,
215+happens with the `/txt/` mark: we use it to find a file in clay with that
217−without performing any conversions. Since this file is stored with the `%txt`
216+extension, without performing any conversions. Since this file is stored with
218−mark in `%clay`, its type is a `wain`: a list of cords, where each cord is a
217+the `%txt` mark in `%clay`, its type is a `wain`: a list of cords, where each
219−single line. Once we've read the file, the `%mime` mark converts the `wain` to a
218+cord is a single line. Once we've read the file, the `%mime` mark converts the
220−triple that includes the MIME type ("text/plain"), the content length in bytes,
219+`wain` to a triple that includes the MIME type ("text/plain"), the content
221−and the content itself as a cord.
220+length in bytes, and the content itself as a cord.
222−
221+
223− /= page /&html&elem&/~[;div.foo;]
222+```
224− ::
223+/= page /&html&elem&/~[;div.foo;]
225− page
224+::
226−
225+page
226+```
227produces: `'<html><head></head><body><div class="foo"></div></body></html>'`
227produces: `'<html><head></head><body><div class="foo"></div></body></html>'`
228
228
229−This runs the sail expression `;div.foo;` through the `%elem` mark, then through
229+This runs the sail expression `;div.foo;` through the `%elem` mark, then
230−the `%html` mark. The `/~` rune produces an item of mark `noun`. The `%elem`
230+through the `%html` mark. The `/~` rune produces an item of mark `noun`. The
231−mark converts the mark of the expression from `noun` to `elem` by checking that
231+`%elem` mark converts the mark of the expression from `noun` to `elem` by
232−the type fits in `manx` (a hoon/sail type indicating an XML element). The
232+checking that the type fits in `manx` (a hoon/sail type indicating an XML
233−`%html` mark recognizes the `%elem` and converts it to an HTML string with
233+element). The `%html` mark recognizes the `%elem` and converts it to an HTML
234−enclosing `<html>`, `<head>`, and `<body>` tags.
234+string with enclosing `<html>`, `<head>`, and `<body>` tags.
235
235
236It's possible to use wide-form `/&` with more than two marks, by using `&` as a
236It's possible to use wide-form `/&` with more than two marks, by using `&` as a
237delimiter between marks and adding a `/` before the last mark, like:
237delimiter between marks and adding a `/` before the last mark, like:
@@ -247,40 +247,41 @@ need to be enclosed in `/`'s: `/& html /elem/`
247succeeds. If none succeed, it produces stack traces from all of its arguments.
247succeeds. If none succeed, it produces stack traces from all of its arguments.
248
248
249Example:
249Example:
250−
250+```
251− /= res /: /%/path/to/file
251+/= res /: /%/path/to/file
252− /|
252+/|
253− /!elem/
253+ /!elem/
254− /elem/
254+ /elem/
255− ==
255+==
256− ::
256+::
257− res
257+res
258−
258+```
259tries to parse the file at `/%/path/to/file` as a hoon expression and evaluate
259tries to parse the file at `/%/path/to/file` as a hoon expression and evaluate
260it (`/!elem/`), and then if that fails, tries to convert the file to an HTML
260it (`/!elem/`), and then if that fails, tries to convert the file to an HTML
261−element directly (`/elem/`). This is used in urbit's `tree` publishing system to
261+element directly (`/elem/`). This is used in urbit's `tree` publishing system
262−enable a user to place either a hoon file or a static file in a directory, and
262+to enable a user to place either a hoon file or a static file in a directory,
263−have them both result in webpages, with preference given to treating the file as
263+and have them both result in webpages, with preference given to treating the
264−hoon to be evaluated.
264+file as hoon to be evaluated.
265
265
266−Wide-form `/|` encloses its arguments in parentheses, with a single space as a
266+Wide-form `/|` encloses its arguments in parentheses, with a single space as a delimiter:
267−delimiter: `/|(/!elem/ /elem/)`
267+`/|(/!elem/ /elem/)`
268
268
269### `/_` unfiltered: run a horn on each file in the current directory
269### `/_` unfiltered: run a horn on each file in the current directory
270
270
271`/_` can be used in two ways: filtered and unfiltered.
271`/_` can be used in two ways: filtered and unfiltered.
272
272
273−Unfiltered `/_` takes a horn as an argument. It produces a new horn representing
273+Unfiltered `/_` takes a horn as an argument. It produces a new horn representing the
274−the result of mapping the supplied horn over the list of files in the current
274+result of mapping the supplied horn over the list of files in the current
275directory. The keys in the resulting map are the basenames of the files in the
275directory. The keys in the resulting map are the basenames of the files in the
276directory, and each value is the result of running that horn on the contents of
276directory, and each value is the result of running that horn on the contents of
277the file.
277the file.
278
278
279Example:
279Example:
280−
280+```
281− /= kids /_ /hoon/
281+/= kids /_ /hoon/
282− ::
282+::
283− `(map term cord)`kids
283+`(map term cord)`kids
284+```
284
285
285produces a value of type `(map term cord)`, where each key is the basename (the
286produces a value of type `(map term cord)`, where each key is the basename (the
286filename without the prefix), and each value is the result of running the
287filename without the prefix), and each value is the result of running the
@@ -295,14 +296,14 @@ Wide-form unfiltered `/_` doesn't need a delimiter: `/= kids /_/hoon/`
295Filtered `/_` takes an aura and a horn, and filters the list of files in the
296Filtered `/_` takes an aura and a horn, and filters the list of files in the
296current directory by whether their filenames can be parsed to an atom of that
297current directory by whether their filenames can be parsed to an atom of that
297aura. It then produces a map where each key is the filename after parsing into
298aura. It then produces a map where each key is the filename after parsing into
298−that atom type, and each value is the result of running the horn on the contents
299+that atom type, and each value is the result of running the horn on the
299−of that file.
300+contents of that file.
300
301
301Example:
302Example:
302−
303+```
303− /= timed-posts /_ @da /md/
304+/= timed-posts /_ @da /md/
304− `(map @da @t)`timed-posts
305+`(map @da @t)`timed-posts
305−
306+```
306produces a map from dates to cords. This product will only contain files whose
307produces a map from dates to cords. This product will only contain files whose
307names are parsable into `@da` dates. The values are the file contents converted
308names are parsable into `@da` dates. The values are the file contents converted
308to a cord of markdown-formatted text.
309to a cord of markdown-formatted text.
@@ -311,86 +312,89 @@ Wide-form filtered `/_` uses `_` as a delimiter: `/= timed-posts /_@da_/md/`
311
312
312### `/;` operate on
313### `/;` operate on
313
314
314−`/;` takes a twig and a horn. The twig should evaluate to a gate, which is then
315+`/;` takes a twig and a horn. The twig should evaluate to a gate, which is then slammed
315−slammed with the result of the horn as its sample.
316+with the result of the horn as its sample.
316
317
317Example:
318Example:
318−
319+```
319− /= goo
320+/= goo
320− /; |=({a/@ b/@} +(b))
321+ /; |=({a/@ b/@} +(b))
321− /~ [a=0 b=1]
322+ /~ [a=0 b=1]
322− ::
323+::
323− goo
324+goo
325+```
324
326
325produces: `2`
327produces: `2`
326
328
327Here's a slightly more complex example with runes that use the filesystem:
329Here's a slightly more complex example with runes that use the filesystem:
328−
330+```
329− /= file-length
331+/= file-length
330− /; |=(a/@t (lent (crip a)))
332+ /; |=(a/@t (lent (crip a)))
331− /: /%/path/to/hoon/file /hoon/
333+ /: /%/path/to/hoon/file /hoon/
332− ::
334+::
333− file-length
335+file-length
334−
336+```
335produces the number of bytes in the file "/%/path/to/hoon/file."
337produces the number of bytes in the file "/%/path/to/hoon/file."
336
338
337### `/,` switch by path
339### `/,` switch by path
338
340
339−`/,` is a switch statement, which picks a branch to evaluate based on whether
341+`/,` is a switch statement, which picks a branch to evaluate based on
340−the current path matches the path in the switch statement. Takes a sequence of
342+whether the current path matches the path in the switch statement.
341−pairs of (path, horn) terminated by a `==`. No wide-form.
343+Takes a sequence of pairs of (path, horn) terminated by a `==`. No wide-form.
342
344
343Example:
345Example:
344−
346+```
345− /= just-right
347+/= just-right
346− /: /===/right-path :: set path to /%/right-path
348+ /: /===/right-path :: set path to /%/right-path
347− /,
349+ /,
348− /wrong-path /~ ~
350+ /wrong-path /~ ~
349− /another-wrong-path /~ ~
351+ /another-wrong-path /~ ~
350− /right-path /~ %evaluate-me :: only evaluate this horn
352+ /right-path /~ %evaluate-me :: only evaluate this horn
351− ==
353+ ==
352− ::
354+::
353− `@t`just-right
355+`@t`just-right
356+```
354
357
355produces: `'evaluate-me'`
358produces: `'evaluate-me'`
356
359
357### `/.` list
360### `/.` list
358
361
359−Produce a null-terminated list from a sequence of horns, terminated by a `==`.
362+Produce a null-terminated list from a sequence of horns, terminated by a `==`. No wide-form.
360−No wide-form.
361
363
362Example:
364Example:
363−
365+```
364− /= vanes
366+/= vanes
365− /.
367+ /.
366− /~ %ames
368+ /~ %ames
367− /~ %behn
369+ /~ %behn
368− /~ %clay
370+ /~ %clay
369− /~ %dill
371+ /~ %dill
370− /~ %eyre
372+ /~ %eyre
371− /~ %ford
373+ /~ %ford
372− /~ %gall
374+ /~ %gall
373− ==
375+ ==
374− ::
376+::
375− vanes
377+vanes
376−
378+```
377produces: `[%ames %behn %clay %dill %eyre %ford %gall ~]`.
379produces: `[%ames %behn %clay %dill %eyre %ford %gall ~]`.
378
380
379### `/*` heterogeneous map
381### `/*` heterogeneous map
380
382
381−*DEPRECATED* TODO: remove
383+_DEPRECATED_
384+TODO: remove
382
385
383### `/^` cast
386### `/^` cast
384
387
385`/^` takes a mold and a horn, and casts the result of the horn to the mold.
388`/^` takes a mold and a horn, and casts the result of the horn to the mold.
386
389
387Example:
390Example:
388−
391+```
389− /= liz /^ (list @) :: cast to real list
392+/= liz /^ (list @) :: cast to real list
390− /~ ~[1 2 3] :: no 'i' or 't' faces
393+ /~ ~[1 2 3] :: no 'i' or 't' faces
391− ::
394+::
392− ?< ?=($~ liz) :: prevent find-fork
395+?< ?=($~ liz) :: prevent find-fork
393− i.liz
396+i.liz
397+```
394
398
395produces: `1`
399produces: `1`
396
400
@@ -400,61 +404,60 @@ Wide-form `/^` uses `^` as a delimiter: `/^(list @)^/~[1 2 3 ~]`
400
404
401### `/#` insert dephash
405### `/#` insert dephash
402
406
403−`/#` takes a horn and produces a cell of the dependency hash of the result of
407+`/#` takes a horn and produces a cell of the dependency
404−the horn, and the result itself.
408+hash of the result of the horn, and the result itself.
405
409
406Example:
410Example:
407−
411+```
408− /= inline /^ {dep/@uvH txt/@t} /# /: /%/my-script /js/
412+/= inline /^ {dep/@uvH txt/@t} /# /: /%/my-script /js/
409− ::
413+::
410− ;= ;script: (trip txt.inline) :: set script contents
414+;= ;script: (trip txt.inline) :: set script contents
411− ;script@"/~/on/{<dep.inline>}.js"; :: set script src
415+ ;script@"/~/on/{<dep.inline>}.js"; :: set script src
412− ==
416+==
413−
417+```
414−produces two `<script>` tags. The first has its contents set to the contents of
418+produces two `<script>` tags. The first has its contents set to the contents
415−the JavaScript file we loaded from clay. The second is an auto-update script
419+of the JavaScript file we loaded from clay. The second is an auto-update
416−that polls the server to check whether the dependency hash has changed. This
420+script that polls the server to check whether the dependency hash has changed.
417−pattern is used in urbit's `tree` web publishing system.
421+This pattern is used in urbit's `tree` web publishing system.
418
422
419Wide-form `/#` does not need a delimiter:
423Wide-form `/#` does not need a delimiter:
420`/= inline /^ {dep/@uvH txt/@t} /#/:/%/my-script:/js/`
424`/= inline /^ {dep/@uvH txt/@t} /#/:/%/my-script:/js/`
421
425
422### `/$` process extra arguments
426### `/$` process extra arguments
423
427
424−`/$` will slam a gate on whatever extra arguments have been supplied to this
428+`/$` will slam a gate on whatever extra arguments have been supplied to this build.
425−build. At the moment, only HTTP requests forwarded by `%eyre` contain any extra
429+At the moment, only HTTP requests forwarded by `%eyre` contain any extra arguments,
426−arguments, and using this rune outside of that context will cause an error.
430+and using this rune outside of that context will cause an error. Requests from `%eyre`
427−Requests from `%eyre` contain an argument representing the query string, which
431+contain an argument representing the query string, which can be parsed using the
428−can be parsed using the standard library gate `++fuel`.
432+standard library gate `++fuel`.
429
433
430Example:
434Example:
431−
435+```
432− /= gas /$ fuel
436+/= gas /$ fuel
433− ::
437+::
434− :*
438+:*
435− %who {<(~(get ju aut.ced.gas) 0)>}
439+ %who {<(~(get ju aut.ced.gas) 0)>}
436− %where {(spud s.bem.gas)}
440+ %where {(spud s.bem.gas)}
437− %case {(scow %ud p.r.bem.gas)}
441+ %case {(scow %ud p.r.bem.gas)}
438− ==
442+==
439−
443+```
440−produces: \[%who '~zod' %where /ford/pages/web %case 60\]
444+produces: [%who '~zod' %where /ford/pages/web %case 60]
441−
445+
442−Wide-form `/$` takes its arguments inside brackets, like `/~`: `/=gas=/$[fuel]`
446+Wide-form `/$` takes its arguments inside brackets, like `/~`:
447+`/=gas=/$[fuel]`
443
448
444### `/%` propagate extra arguments into renderers
449### `/%` propagate extra arguments into renderers
445
450
446−`/%` will forward extra arguments (usually from `%eyre`) on to any enclosed
451+`/%` will forward extra arguments (usually from `%eyre`) on to any enclosed `/renderer/`'s. Without this,
447−`/renderer/`'s. Without this, renderers that use `/$` to read the extra
452+renderers that use `/$` to read the extra arguments will crash.
448−arguments will crash.
449
453
450Example:
454Example:
451−
455+```
452− /= dat /% /tree-json/
456+/= dat /% /tree-json/
453− ::
457+::
454− dat
458+dat
455−
459+```
456−produces the results of running `ren/tree-json` on the current path. This
460+produces the results of running `ren/tree-json` on the current path. This renderer needs the query string,
457−renderer needs the query string, which is one of the extra arguments passed in
461+which is one of the extra arguments passed in from `%eyre`.
458−from `%eyre`.
459
462
460Wide-form `/%` has no delimiter: `/%/tree-json/`
463Wide-form `/%` has no delimiter: `/%/tree-json/`
docs/arvo/internals/gall.md+71−57
@@ -1,102 +1,116 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 7
3sort: 7
4---
4---
5
5
6−# `%gall`
6+
7+`%gall`
8+=======
7
9
8Our application server manager.
10Our application server manager.
9
11
10−It allows applications and vanes to send messages to applications and subscribe
12+It allows applications and vanes to send messages to applications and
11−to data streams. This requires `%gall` to be a sort of a hypervisor. Messages
13+subscribe to data streams. This requires `%gall` to be a sort of a
12−coming into `%gall` are routed to the intended application, and the response
14+hypervisor. Messages coming into `%gall` are routed to the intended
13−comes back along the same route. If the intended target is on another ship,
15+application, and the response comes back along the same route. If the
14−`%gall` will behind-the-scenes route it through ames to the other ship to run.
16+intended target is on another ship, `%gall` will behind-the-scenes route
15−This provides an abstraction where all apps on all ships are communicated with
17+it through ames to the other ship to run. This provides an abstraction
16−over the same interface.
18+where all apps on all ships are communicated with over the same
19+interface.
17
20
18`%gall` neither accepts events from unix nor produces effects. It exists
21`%gall` neither accepts events from unix nor produces effects. It exists
19−entirely for the benefit of other vanes and, in particular, applications. Eyre
22+entirely for the benefit of other vanes and, in particular,
20−exposes `%gall`'s interface over http, and ames does the same over the ames
23+applications. Eyre exposes `%gall`'s interface over http, and ames does
21−network. `%gall` uses ford to compile and run the applications.
24+the same over the ames network. `%gall` uses ford to compile and run the
25+applications.
22
26
23−## Cards
27+Cards
28+-----
24
29
25−`%gall` accepts the following cards. The first three are the most commonly used,
30+`%gall` accepts the following cards. The first three are the most
26−while the others are primarily used internally.
31+commonly used, while the others are primarily used internally.
27
32
28### `%mess`
33### `%mess`
29
34
30−Sends a message to an app. This will result in a call to the app's `++poke` arm.
35+Sends a message to an app. This will result in a call to the app's
31−The response is exactly one of a `%nice` if the action succeeded or a `%mean` if
36+`++poke` arm. The response is exactly one of a `%nice` if the action
32−not.
37+succeeded or a `%mean` if not.
33
38
34### `%show`
39### `%show`
35
40
36−Subscribes to a stream from an app. This will result in a call to the app's
41+Subscribes to a stream from an app. This will result in a call to the
37−either `++peek` or `++peer` arm. The first response will always be either a
42+app's either `++peek` or `++peer` arm. The first response will always be
38−`%nice` or a `%mean`, indicating whether or not the subscription was successful.
43+either a `%nice` or a `%mean`, indicating whether or not the
39−After the first response, there will be zero or more responses of either `%rush`
44+subscription was successful. After the first response, there will be
40−or `%rust`, which communicate either a differntial or full update to the data
45+zero or more responses of either `%rush` or `%rust`, which communicate
41−stream. There may be a `%mean`, which indicates that the subscription has been
46+either a differential or full update to the data stream. There may be a
42−canceled and no more responses will be received along this stream.
47+`%mean`, which indicates that the subscription has been canceled and no
48+more responses will be received along this stream.
49+
43
50
44### `%nuke`
51### `%nuke`
45
52
46−Unsubscribes the current duct from its stream. This receives a response of
53+Unsubscribes the current duct from its stream. This receives a response
47−either a `%nice` or a `%mean`. Note that a response of `%nice` does not imply
54+of either a `%nice` or a `%mean`. Note that a response of `%nice` does
48−that the current duct was in fact subscribed to any stream.
55+not imply that the current duct was in fact subscribed to any stream.
49
56
50### `%init`
57### `%init`
51
58
52−Initializes a ship's apps. This should be called exactly once for each ship on
59+Initializes a ship's apps. This should be called exactly once for each
53−the pier. This produces no moves in response.
60+ship on the pier. This produces no moves in response.
54
61
55### `%sire`
62### `%sire`
56
63
57−Instantiates a child app. The app will be at path `[p parent-path]`, and it will
64+Instantiates a child app. The app will be at path `[p parent-path]`, and
58−be an instance of the `q` app. The only response will be `%gone` when the child
65+it will be an instance of the `q` app. The only response will be `%gone`
59−dies.
66+when the child dies.
67+
60
68
61### `%rote`
69### `%rote`
62
70
63−Signifies a remote request from ames. `r` should be of type `rook`. This how an
71+Signifies a remote request from ames. `r` should be of type `rook`. This
64−app on a foreign ship may send a `%mess`, `%show`, or `%nuke` card. Note that
72+how an app on a foreign ship may send a `%mess`, `%show`, or `%nuke`
65−`%gall` automatically converts `%mess`, `%show`, and `%nuke` into ames messages
73+card. Note that `%gall` automatically converts `%mess`, `%show`, and
66−behind the scenes, so the only entity that should use `%rote` and `%roth` is
74+`%nuke` into ames messages behind the scenes, so the only entity that
67−ames. Formally, the response is either a `%nice` or a `%mean`, which ames uses
75+should use `%rote` and `%roth` is ames. Formally, the response is either
68−to give a positive or negative ack. A logical response comes by passing a
76+a `%nice` or a `%mean`, which ames uses to give a positive or negative
69−`%roth` card.
77+ack. A logical response comes by passing a `%roth` card.
78+
70
79
71### `%roth`
80### `%roth`
72
81
73−Gives the response received from a remote request. `r` should be of type `roon`.
82+Gives the response received from a remote request. `r` should be of type
74−This is how an app responds to a foreign request with a `%rush`, `%rust`,
83+`roon`. This is how an app responds to a foreign request with a `%rush`,
75−`%nice`, or `%mean`. The response is either a `%nice` or a `%mean`. Even though
84+`%rust`, `%nice`, or `%mean`. The response is either a `%nice` or a
76−we, as the proverb goes, "never ack an ack", we do need to acknowledge these
85+`%mean`. Even though we, as the proverb goes, "never ack an ack", we do
77−responses since they're really independent one-way messages.
86+need to acknowledge these responses since they're really independent
87+one-way messages.
88+
78
89
79### `%wipe`
90### `%wipe`
80
91
81−Wipes the given app from memory. This is generally considered a hack, but it is
92+Wipes the given app from memory. This is generally considered a hack,
82−sometimes useful during development to wipe the state of an app. We don't
93+but it is sometimes useful during development to wipe the state of an
83−guarantee that this actually completely wipes the app. Generally, you want to
94+app. We don't guarantee that this actually completely wipes the app.
84−use a `%cide` card if you actually want to kill an app. This gives no response.
95+Generally, you want to use a `%cide` card if you actually want to kill
96+an app. This gives no response.
97+
85
98
86### `%cide`
99### `%cide`
87
100
88−Kills an app and all its children. Even though it's not technically a part of
101+Kills an app and all its children. Even though it's not technically a
89−`%gall`'s interface since it's not in `++kiss` and can't be called from the
102+part of `%gall`'s interface since it's not in `++kiss` and can't be
90−outside, it's worth mentioning `%cide`, which may be called from within `%gall`
103+called from the outside, it's worth mentioning `%cide`, which may be
91−apps. It should call `++part` to allow the app any last words. This gives no
104+called from within `%gall` apps. It should call `++part` to allow the
92−response.
105+app any last words. This gives no response.
93
106
94−## Service Gates
107+Service Gates
108+-------------
95
109
96### `++poke`
110### `++poke`
97
111
98−Handles incoming messages. Most commonly with an associated `%logo`. For example
112+Handles incoming messages. Most commonly with an associated `%logo`. For
99−`++poke-json` handles an incoming JSON request from `%eyre`.
113+example `++poke-json` handles an incoming JSON request from `%eyre`.
100
114
101### `++peer`
115### `++peer`
102
116
docs/arvo/marks.md+102−82
@@ -1,111 +1,131 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
3+next: true
4sort: 16
4sort: 16
5title: Marks
5title: Marks
6---
6---
7
7
8# Marks
8# Marks
9
9
10−We've used predefined marks already, but we haven't yet created our own marks.
10+We've used predefined marks already, but we haven't yet created
11−Let's write a sample mark of our own, then chain it together with some
11+our own marks. Let's write a sample mark of our own, then chain
12−preexisting ones to have fun with type conversions.
12+it together with some preexisting ones to have fun with type
13+conversions.
13
14
14−Let's make a small "examples-cord" mark. "Cord" is a name we use for `@t` text,
15+Let's make a small "examples-cord" mark. "Cord" is a name we use for `@t`
15−but there's no predefined mark for it. Let's put the following code in:
16+text, but there's no predefined mark for it. Let's put the following code in:
16`/mar/examples/cord.hoon`:
17`/mar/examples/cord.hoon`:
17
18
18− /? 314
19+```
19− |_ cod/@t
20+/? 314
20− ++ grab
21+|_ cod/@t
21− |%
22+++ grab
22− ++ atom |=(arg/@ `@t`(scot %ud arg))
23+ |%
23− --
24+ ++ atom |=(arg/@ `@t`(scot %ud arg))
24− ++ grow
25+ --
25− |%
26+++ grow
26− ++ md cod
27+ |%
27− --
28+ ++ md cod
28− --
29+ --
29−
30+--
30−`/? 314` is the required version number, just like in apps. After that
31+```
31−everything's in a `|_`, which is a `|%` core, but with input arguments. In our
32+
33+`/? 314` is the required
34+version number, just like in apps. After that everything's in a
35+`|_`, which is a `|%` core, but with input arguments. In our
32case, the argument is the marked data.
36case, the argument is the marked data.
33
37
34−There are three possible top-level arms in the `|_` core, `++grab`, `++grow`,
38+There are three possible top-level arms in the `|_` core,
35−and `++grad` (which is used for revision control, covered elsewhere). `++grab`
39+`++grab`, `++grow`, and `++grad` (which is used for revision
36−specifies functions to convert from another mark to the current mark. `++grow`
40+control, covered elsewhere). `++grab` specifies
37−specifies how to convert from the current mark to another one.
41+functions to convert from another mark to the current mark.
38−
42+`++grow` specifies how to convert from the current mark to
39−In our case, we only define one arm in `++grab`, namely `++atom`. This allows us
43+another one.
40−to convert any atom to a cord. `++scot` is a standard library function in
44+
41−`hoon.hoon` which pretty-prints atoms of various types. We want to treat the
45+In our case, we only define one arm in `++grab`, namely `++atom`.
42−atom as an unsigned decimal number, so we give `%ud` and the given argument to
46+This allows us to convert any atom to a cord. `++scot` is a
43−`++scot`. Thus, the form of an arm in `++grab` is
47+standard library function in `hoon.hoon` which pretty-prints
48+atoms of various types. We want to treat the atom as an
49+unsigned decimal number, so we give `%ud` and the given argument
50+to `++scot`. Thus, the form of an arm in `++grab` is
44`|=(other-mark-type this-mark-value)`.
51`|=(other-mark-type this-mark-value)`.
45
52
46−In `++grow`, we just convert to the `md` mark, which is just a `@t` internally,
53+In `++grow`, we just convert to the `md` mark, which is just a
47−but it has fancy conversion functions to things like html. `++md` isn't a
54+`@t` internally, but it has fancy conversion functions to things
48−function because the "argument" is already a `@t`, so we can just produce the
55+like html. `++md` isn't a function because the "argument" is
49−new value directly.
56+already a `@t`, so we can just produce the new value directly.
50
57
51−Let's play around a bit with this mark. First, let's take a marked atom and
58+Let's play around a bit with this mark. First, let's take a
52−convert it to our new mark.
59+marked atom and convert it to our new mark.
53
60
54− ~fintud-macrep:dojo> &atom 9
61+```
55− 9
62+~fintud-macrep:dojo> &atom 9
56− ~fintud-macrep:dojo> &examples-cord &atom 9
63+9
57− '9'
64+~fintud-macrep:dojo> &examples-cord &atom 9
58− -~fintud-macrep:dojo> &examples-cord &atom &examples-cord &atom 9
65+'9'
59− -'57'
66+-~fintud-macrep:dojo> &examples-cord &atom &examples-cord &atom 9
67+ -'57'
68+```
60
69
61−ASCII 9 is 57. There's no requirement, implicit or otherwise, that conversions
70+ASCII 9 is 57. There's no requirement, implicit or otherwise,
62−back and forth between marks be inverses of each other. These are semantic
71+that conversions back and forth between marks be inverses of each
63−conversions, in that they refer to the same concept. They're not isomorphisms.
72+other. These are semantic conversions, in that they refer to the
73+same concept. They're not isomorphisms.
64
74
65Let's play around a little more:
75Let's play around a little more:
66
76
67− ~fintud-macrep:dojo> &md &examples-cord &atom 17
77+```
68− '17'
78+~fintud-macrep:dojo> &md &examples-cord &atom 17
69− ~fintud-macrep:dojo> &hymn &md &examples-cord &atom 17
79+'17'
70− [ [%html ~]
80+~fintud-macrep:dojo> &hymn &md &examples-cord &atom 17
71− [[%head ~] [[%title ~] [[%~. [%~. "Untitled"] ~] ~] ~] ~]
81+[ [%html ~]
72− [ [%body ~]
82+ [[%head ~] [[%title ~] [[%~. [%~. "Untitled"] ~] ~] ~] ~]
73− [ i=[g=[n=%meta a=~[[n=%value v="{}"] [n=%name v="frontmatter"] [n=[%urb %front] v=""]]] c=~]
83+ [ [%body ~]
74− t=[i=[g=[n=%p a=~] c=~[[g=[n=%$ a=~[[n=%$ v="17"]]] c=~]]] t=~]
84+ [ i=[g=[n=%meta a=~[[n=%value v="{}"] [n=%name v="frontmatter"] [n=[%urb %front] v=""]]] c=~]
75− ]
85+ t=[i=[g=[n=%p a=~] c=~[[g=[n=%$ a=~[[n=%$ v="17"]]] c=~]]] t=~]
76− ]
77− ~
78 ]
86 ]
79−
87+ ]
80−That was exciting. If you squint at that, it looks an awful lot like html. Our
88+ ~
81−value `17` is still in there, in the body. `urb` is the mark we render most web
89+]
82−pages to. It makes sure you have a complete skeleton of a web page. Of course,
90+```
83−by the time this gets to the web page, it's plain html. Let's do the final step
91+
84−in the conversion.
92+That was exciting. If you squint at that, it looks an
85−
93+awful lot like html. Our value `17` is still in there, in the
86− ~fintud-macrep:dojo> &mime &hymn &md &examples-cord &atom 17
94+body. `urb` is the mark we render most web pages to. It makes
87− [ [%text %html ~]
95+sure you have a complete skeleton of a web page. Of course, by
88− p=121
96+the time this gets to the web page, it's plain html. Let's do
89− q='<html><head><title>Untitled</title></head><body><meta value="{}" name="frontmatter" urb:front="" /><p>17</p></body></html>'
97+the final step in the conversion.
90− ]
98+
91−
99+```
92−This is a mime-typed octet stream with type `/text/html`, length 121 bytes, and
100+~fintud-macrep:dojo> &mime &hymn &md &examples-cord &atom 17
93−our lowly number `17` rendered to a web page. `examples-cord` was just one step
101+[ [%text %html ~]
94−in the chain.
102+ p=121
103+ q='<html><head><title>Untitled</title></head><body><meta value="{}" name="frontmatter" urb:front="" /><p>17</p></body></html>'
104+]
105+```
106+
107+This is a mime-typed octet stream with type `/text/html`, length
108+121 bytes, and our lowly number `17` rendered to a web page.
109+`examples-cord` was just one step in the chain.
95
110
96Of course, arvo can infer some parts of this chain:
111Of course, arvo can infer some parts of this chain:
97
112
98− ~fintud-macrep:dojo> &examples-cord 17
113+```
99− '17'
114+~fintud-macrep:dojo> &examples-cord 17
115+'17'
116+```
100
117
101−Likewise, `md` is the only way to get from `examples-cord` to `hymn`, so that
118+Likewise, `md` is the only way to get from `examples-cord` to `hymn`, so
102−can be omitted. If we omit the `hymn`, though, we get:
119+that can be omitted. If we omit the `hymn`, though, we get:
103
120
104− ~fintud-macrep:dojo> &mime &examples-cord 17
121+```
105− [[%text %x-markdown ~] p=1 q='7']
122+~fintud-macrep:dojo> &mime &examples-cord 17
123+[[%text %x-markdown ~] p=1 q='7']
124+```
106
125
107−Here, we converted straight from `examples-cord` to `md` to `mime`. Arvo decided
126+Here, we converted straight from `examples-cord` to `md` to `mime`. Arvo
108−that was the most straightforward conversion, but it gives a different result
127+decided that was the most straightforward conversion, but it
109−than passing it through the `hymn` mark.
128+gives a different result than passing it through the `hymn`
129+mark.
110
130
111The minimal correct input, then, is `&mime &hymn &examples-cord &atom 17`.
131The minimal correct input, then, is `&mime &hymn &examples-cord &atom 17`.
docs/arvo/network.md+202−183
@@ -1,18 +1,19 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: False
3+next: false
4sort: 13
4sort: 13
5title: Network messages
5title: Network messages
6---
6---
7
7
8−# Network messages
8+Network messages
9+================
9
10
10−Now we've learned enough Hoon to do more interesting things. Let's get our
11+Now we've learned enough Hoon to do more interesting things. Let's get
11−planets to talk to each other.
12+our planets to talk to each other.
12
13
13−All we've written up until now are just shell commands that produce a value and
14+All we've written up until now are just shell commands that produce a
14−then disappear. To listen for and receive messages from other planets, we'll
15+value and then disappear. To listen for and receive messages from other
15−need an app. Let's look at a very simple one:
16+planets, we'll need an app. Let's look at a very simple one:
16
17
17 :: Accepts any noun from dojo and prints it out
18 :: Accepts any noun from dojo and prints it out
18 ::
19 ::
@@ -28,9 +29,10 @@ need an app. Let's look at a very simple one:
28 [~ +>.$]
29 [~ +>.$]
29 --
30 --
30
31
31−This is a very simple app that does only one thing. If you poke it with a value,
32+This is a very simple app that does only one thing. If you poke it with
32−it prints that value out. To try this out, you have to start the app, then you
33+a value, it prints that value out. To try this out, you have to start
33−can poke it from the command line with the following commands:
34+the app, then you can poke it from the command line with the following
35+commands:
34
36
35 ~fintud-macrep:dojo> |start %examples-echo
37 ~fintud-macrep:dojo> |start %examples-echo
36 'prep'
38 'prep'
@@ -42,44 +44,46 @@ can poke it from the command line with the following commands:
42 [%argument [1 2]]
44 [%argument [1 2]]
43 >=
45 >=
44
46
45−> There is currently a bug where the `%argument` lines are printed *above* the
47+> There is currently a bug where the `%argument` lines are printed
46−> line you entered, so your output may not look exactly like this. `>=` means
48+> *above* the line you entered, so your output may not look exactly like
47−> that a command was successfully received and executed.
49+> this. `>=` means that a command was successfully received and
50+> executed.
48
51
49Most of the app code should be simple enough to guess its function. The
52Most of the app code should be simple enough to guess its function. The
50important part of this code is the definition of `++poke-noun`.
53important part of this code is the definition of `++poke-noun`.
51
54
52−Once an app starts, it's always running in the background and you interact with
55+Once an app starts, it's always running in the background and you
53−it by sending it messages. The most straightforward way to do that is to poke it
56+interact with it by sending it messages. The most straightforward way to
54−from the command line, which we we did with `:echo 5`
57+do that is to poke it from the command line, which we we did with
55−(`:[app-name] [argument(s)]`).
58+`:echo 5` (`:[app-name] [argument(s)]`).
56−
59+
57−In this case, `++poke-noun` takes an argument `arg` and prints it out with `~&`
60+In this case, `++poke-noun` takes an argument `arg` and prints it out
58−([sigpam](../../hoon/twig/sig-hint/pam-dump/)). This is an unusual rune that
61+with `~&` ([sigpam](../../hoon/twig/sig-hint/pam-dump/)). This is an unusual
59−formally "does nothing", but the interpreter detects it and printfs the first
62+rune that formally "does nothing", but the interpreter detects it and
60−child, before executing the second as if the first didn't exist. This is a
63+printfs the first child, before executing the second as if the first
61−slightly hacky way of printing to the console, but we'll get to the correct way
64+didn't exist. This is a slightly hacky way of printing to the console,
62−later on.
65+but we'll get to the correct way later on.
63−
66+
64−But what does `++poke-noun` produce? Recall that `^-` casts to a type. In this
67+But what does `++poke-noun` produce? Recall that `^-` casts to a type.
65−case, it's declaring that the end result of the function will be of type
68+In this case, it's declaring that the end result of the function will be of
66−`{(list) _+>.$}`. But what does this mean?
69+type `{(list) _+>.$}`. But what does this mean?
67−
70+
68−The phrase to remember is "a list of moves and our state". Urbit is a message
71+The phrase to remember is "a list of moves and our state". Urbit is a
69−passing system, so whenever we want to do something that interacts with the rest
72+message passing system, so whenever we want to do something that
70−of the system, we send a message. Thus, a move is Arvo's equivalent of a
73+interacts with the rest of the system, we send a message. Thus, a move
71−syscall. The first thing that `++poke-noun` produces is a list of messages,
74+is Arvo's equivalent of a syscall. The first thing that `++poke-noun`
72−called "moves". In this case, we don't actually want the system to do anything,
75+produces is a list of messages, called "moves". In this case, we don't
73−so we produce the empty list, `~` (in the `[~ +>.$]` line).
76+actually want the system to do anything, so we produce the empty list,
77+`~` (in the `[~ +>.$]` line).
74
78
75The second thing `++poke-noun` produces is our state. `+>.$` refers to a
79The second thing `++poke-noun` produces is our state. `+>.$` refers to a
76−particular address in our subject where our formal app state is stored. It'll
80+particular address in our subject where our formal app state is stored.
77−become clear why this is later on, but for now pretend that `+>.$` is a magic
81+It'll become clear why this is later on, but for now pretend that `+>.$`
78−invocation that means "app state".
82+is a magic invocation that means "app state".
79
83
80−Let's look at another example (edit your code in `/app/examples/echo/hoon` to
84+Let's look at another example (edit your code in
81−reflect the code below). Say we want to only accept a number, and then print out
85+`/app/examples/echo/hoon` to reflect the code below). Say we want to
82−the square of that number.
86+only accept a number, and then print out the square of that number.
83
87
84 /? 314
88 /? 314
85 !:
89 !:
@@ -92,27 +96,31 @@ the square of that number.
92 [~ +>.$]
96 [~ +>.$]
93 --
97 --
94
98
95−A few things have changed. Firstly, we no longer accept arbitrary nouns because
99+A few things have changed. Firstly, we no longer accept arbitrary nouns
96−we can only square atoms (integers, in this case an unsigned one). Thus, our
100+because we can only square atoms (integers, in this case an unsigned
97−argument is now `arg/@`. Secondly, it's `++poke-atom` rather than `++poke-noun`.
101+one). Thus, our argument is now `arg/@`. Secondly, it's `++poke-atom`
102+rather than `++poke-noun`.
98
103
99−# Intro to marks
104+Intro to marks
105+==============
100
106
101−Are there other `++poke`s? Yes. In fact, `noun` and `atom` are just two of
107+Are there other `++poke`s? Yes. In fact, `noun` and `atom` are just two
102−arbitrarily many "marks". A mark is fundamentally a type definition, but
108+of arbitrarily many "marks". A mark is fundamentally a type definition,
103−accessible at the Arvo level. Each mark is defined in the `/mar` directory. Some
109+but accessible at the Arvo level. Each mark is defined in the `/mar`
104−marks have conversion routines to other marks, and some have diff, patch, and
110+directory. Some marks have conversion routines to other marks, and some
105−merge algorithms. None of these are required for a mark to exist, however.
111+have diff, patch, and merge algorithms. None of these are required for a
112+mark to exist, however.
106
113
107−`noun` and `atom` are two predefined marks. In your `/mar` directory, there are
114+`noun` and `atom` are two predefined marks. In your `/mar` directory,
108−already many more, and you may add more at will. The type associated with `noun`
115+there are already many more, and you may add more at will. The type
109−is `*`, and the type associated with `atom` is `@`.
116+associated with `noun` is `*`, and the type associated with `atom` is
117+`@`.
110
118
111−When we poke an app from anywhere, we do so with a mark that searches for the
119+When we poke an app from anywhere, we do so with a mark that searches
112−corresponding (`++poke-[mark]`) arm. Data constructed on the command line is by
120+for the corresponding (`++poke-[mark]`) arm. Data constructed on the
113−default marked with `noun`. In this case, the app is expecting an atom, so we
121+command line is by default marked with `noun`. In this case, the app is
114−have to explicitly mark the data with `atom` using `&[mark]`. Try the following
122+expecting an atom, so we have to explicitly mark the data with `atom`
115−commands:
123+using `&[mark]`. Try the following commands:
116
124
117 ~fintud-macrep:dojo> |start %square
125 ~fintud-macrep:dojo> |start %square
118 >=
126 >=
@@ -124,15 +132,16 @@ commands:
124
132
125> Recall the bug where `%square` may get printed above the input line.
133> Recall the bug where `%square` may get printed above the input line.
126
134
127−Marks are powerful, and they're the backbone of Urbit's data pipeline, so we'll
135+Marks are powerful, and they're the backbone of Urbit's data pipeline,
128−be getting quite used to them.
136+so we'll be getting quite used to them.
129
137
130**Exercise**:
138**Exercise**:
131
139
132- Write an app that computes fizzbuzz on its input (as in the previous
140- Write an app that computes fizzbuzz on its input (as in the previous
133 section).
141 section).
134
142
135−# Sending a message to another urbit
143+Sending a message to another urbit
144+==================================
136
145
137Let's write our first network message! Here's `/app/examples/pong.hoon`:
146Let's write our first network message! Here's `/app/examples/pong.hoon`:
138
147
@@ -164,34 +173,36 @@ Run it with these commands:
164 ~fintud-macrep:dojo> :examples-pong &urbit ~sampel-sipnym
173 ~fintud-macrep:dojo> :examples-pong &urbit ~sampel-sipnym
165 >=
174 >=
166
175
167−Replace `~sampel-sipnym` with another urbit. The easiest thing to do is to start
176+Replace `~sampel-sipnym` with another urbit. The easiest thing to do is
168−a comet, a free and disposable Urbit identity. If you don't know how to start a
177+to start a comet, a free and disposable Urbit identity. If you don't know
169−comet, see [the user setup section](/docs/using/setup/). Don't forget to start
178+how to start a comet, see [the user setup section](/docs/using/setup/).
170−the `%examples-pong` app on that urbit, too. You should see, on the foreign
179+Don't forget to start the `%examples-pong` app on that urbit, too. You
171−urbit, this output:
180+should see, on the foreign urbit, this output:
172
181
173 [%receiving 'howdy']
182 [%receiving 'howdy']
174
183
175−Most of the code should be straightforward. In `++poke-atom`, the only new thing
184+Most of the code should be straightforward. In `++poke-atom`, the only
176−is the expression `(@t arg)`, which is the type `@t` being called as a function
185+new thing is the expression `(@t arg)`, which is the type `@t` being
177−with argument `arg`. As we already know, `@t` is the type of "cord" text
186+called as a function with argument `arg`. As we already know, `@t` is
178−strings. In Hoon, when types are called as functions, they serve as a validator
187+the type of "cord" text strings. In Hoon, when types are called as
179−function called a "clam" -- that is, a function whose domain is all nouns, and
188+functions, they serve as a validator function called a "clam" -- that
180−range is the given type (in this case, `@t`).
189+is, a function whose domain is all nouns, and range is the given type
181−
190+(in this case, `@t`).
182−In simpler terms, if a clam is passed a value of its own type, it produces that
191+
183−value. Otherwise, it produces the default value (aka the "bunt") of its type.
192+In simpler terms, if a clam is passed a value of its own type, it
184−Here we call this `@t` function on the argument. This coerces the argument to
193+produces that value. Otherwise, it produces the default value (aka the
185−text, so that we can print it out prettily no matter what we're passed.
194+"bunt") of its type. Here we call this `@t` function on the argument.
186−
195+This coerces the argument to text, so that we can print it out prettily
187−The more interesting part is in `++poke-urbit`. The `urbit` mark is an urbit
196+no matter what we're passed.
188−identity, and the Hoon type associated with it is `@p` (the "p" stands for
197+
189−"phonetic base").
198+The more interesting part is in `++poke-urbit`. The `urbit` mark is an
190−
199+urbit identity, and the Hoon type associated with it is `@p` (the "p"
191−Recall that in a `++poke` arm we produce "a list of moves and our state". Until
200+stands for "phonetic base").
192−now, we've left the list of moves empty, since we haven't wanted to tell Arvo to
201+
193−do anything in particular. Now we want to send a message to another urbit. Thus,
202+Recall that in a `++poke` arm we produce "a list of moves and our
194−we produce a list with one element:
203+state". Until now, we've left the list of moves empty, since we haven't
204+wanted to tell Arvo to do anything in particular. Now we want to send a
205+message to another urbit. Thus, we produce a list with one element:
195
206
196 [ost %poke /sending [to-urbit-address %pong] %atom 'howdy']
207 [ost %poke /sending [to-urbit-address %pong] %atom 'howdy']
197
208
@@ -207,118 +218,126 @@ Let's walk through each of these elements step by step.
207
218
208#### Bones ("cause")
219#### Bones ("cause")
209
220
210−If you look up `++bone` in `hoon.hoon`, you'll see that it's a number (`@ud`),
221+If you look up `++bone` in `hoon.hoon`, you'll see that it's a number
211−and that it's an opaque reference to a duct. `++duct` in hoon.hoon is a list of
222+(`@ud`), and that it's an opaque reference to a duct. `++duct` in
212−`wire`s, where `++wire` is an alias for `++path`. `++path` is a list of
223+hoon.hoon is a list of `wire`s, where `++wire` is an alias for `++path`.
213−`++knot`s, which are ASCII text. Thus, a duct is a list of paths, and a bone is
224+`++path` is a list of `++knot`s, which are ASCII text. Thus, a duct is a
214−an opaque reference to it (in the same way that a Unix file descriptor is an
225+list of paths, and a bone is an opaque reference to it (in the same way
215−opaque reference to a file structure). Thus, to truly understand bones, we must
226+that a Unix file descriptor is an opaque reference to a file structure).
216−understand ducts.
227+Thus, to truly understand bones, we must understand ducts.
217−
228+
218−A duct is stack of causes, again, represented as paths, which are called wires.
229+A duct is stack of causes, again, represented as paths, which are called
219−At the bottom of every duct is a unix event, such as a keystroke, network
230+wires. At the bottom of every duct is a unix event, such as a keystroke,
220−packet, file change, or timer event. When Arvo is given this event, it routes
231+network packet, file change, or timer event. When Arvo is given this
221−the event to appropriate kernel module for handling.
232+event, it routes the event to appropriate kernel module for handling.
222−
233+
223−Sometimes, the module can immediately handle the event and produce any necessary
234+Sometimes, the module can immediately handle the event and produce any
224−results. For example, when we poked the `++poke-atom` arm above, we poked %gall,
235+necessary results. For example, when we poked the `++poke-atom` arm
225−our application server, which was able to respond to our poke directly. When the
236+above, we poked %gall, our application server, which was able to respond
226−module cannot service the request itself, it sends instructions to another
237+to our poke directly. When the module cannot service the request itself,
227−kernel module or application (through the %gall module) to do a specified
238+it sends instructions to another kernel module or application (through the
228−action, and produces the result from that.
239+%gall module) to do a specified action, and produces the result from
240+that.
229
241
230Furthermore, when one module sends a message to another kernel module or
242Furthermore, when one module sends a message to another kernel module or
231−application, it also sends along the duct it was given with its new wire tacked
243+application, it also sends along the duct it was given with its new wire
232−onto the end. Now the duct has two entries, with the unix event on the bottom
244+tacked onto the end. Now the duct has two entries, with the unix event on
233−and the kernel module that handled it next. This process can continue
245+the bottom and the kernel module that handled it next. This process can
234−indefinitely, adding more and more layers onto the duct. When an entity finally
246+continue indefinitely, adding more and more layers onto the duct. When
235−produces a result, a layer is popped off the duct, and the result is passed all
247+an entity finally produces a result, a layer is popped off the duct, and
236−the way back down.
248+the result is passed all the way back down.
237−
249+
238−In effect, a duct is an Arvo-level call stack. It's worth noting that while in
250+In effect, a duct is an Arvo-level call stack. It's worth noting that
239−traditional call stacks a function call happens synchronously and returns
251+while in traditional call stacks a function call happens synchronously
240−exactly once. In Arvo, multiple moves can be sent at once, are evaluated
252+and returns exactly once. In Arvo, multiple moves can be sent at once,
241−asynchronously, and each one may be responded to zero or more times.
253+are evaluated asynchronously, and each one may be responded to zero or
242−
254+more times.
243−The point to take home is that whatever caused `++poke-urbit` to be called is
255+
244−also the root cause for the network message we're trying to send. Thus, we say
256+The point to take home is that whatever caused `++poke-urbit` to be
245−to send the network message along the given bone `ost`.
257+called is also the root cause for the network message we're trying to
258+send. Thus, we say to send the network message along the given bone
259+`ost`.
246
260
247##### Wire ('tack on new layer to duct')
261##### Wire ('tack on new layer to duct')
248
262
249−Of course, we have to push a new layer onto our duct before passing it along (or
263+Of course, we have to push a new layer onto our duct before passing it
250−responding to it directly) anywhere. This layer can have any data we want in it,
264+along (or responding to it directly) anywhere. This layer can have any
251−but we don't need anything specific here, so we just use the wire `/sending`
265+data we want in it, but we don't need anything specific here, so we just
252−(`/elem1/elem2/elemN` is one syntax used to create `++path`s and `++wire`s of N
266+use the wire `/sending` (`/elem1/elem2/elemN` is one syntax used to
253−elements). If we were expecting a response (which we're not), it would come back
267+create `++path`s and `++wire`s of N elements). If we were expecting a
254−along the `/sending` wire, meaning that the path `/sending` will be passed back
268+response (which we're not), it would come back along the `/sending`
255−to the response handler as an argument. Although it's not required, it's
269+wire, meaning that the path `/sending` will be passed back to the
256−generally a good idea to make the wire human-readable for bug-handling purposes.
270+response handler as an argument. Although it's not required, it's
271+generally a good idea to make the wire human-readable for bug-handling
272+purposes.
257
273
258##### Term (sys-call)
274##### Term (sys-call)
259
275
260−Each move also has a `term`, composed of lowercase ASCII and/or `-`. This `term`
276+Each move also has a `term`, composed of lowercase ASCII and/or `-`.
261−has the sign `@tas`. In this case, out `term` is `%poke`, which is the name of
277+This `term` has the sign `@tas`. In this case, out `term` is `%poke`,
262−the particular kind of move we're sending. You can always use `%poke` to message
278+which is the name of the particular kind of move we're sending. You can
263−an app. Other common names include `%warp`, to read from the filesystem;
279+always use `%poke` to message an app. Other common names include
264−`%wait`, to set a timer; and `%them`, to send an http request.
280+`%warp`, to read from the filesystem; `%wait`, to set a timer; and
281+`%them`, to send an http request.
265
282
266The move ends with `*` (that is, any noun) since each type of move takes
283The move ends with `*` (that is, any noun) since each type of move takes
267−different data. In our case, a `%poke` move takes a target (urbit and app) and
284+different data. In our case, a `%poke` move takes a target (urbit and
268−marked data, then pokes the arm of the corresponding mark on that app on that
285+app) and marked data, then pokes the arm of the corresponding mark on
269−urbit with that data. `[to-urbit-address %pong]` is the target urbit and app,
286+that app on that urbit with that data. `[to-urbit-address %pong]` is the
270−`%atom` is the `mark`, and`'howdy'` is the data.
287+target urbit and app, `%atom` is the `mark`, and`'howdy'` is the data.
271
288
272−When Arvo receives a `%poke` move, it calls the appropriate `++poke`. The same
289+When Arvo receives a `%poke` move, it calls the appropriate `++poke`.
273−mechanism is used for sending messages between apps on the same urbit as for
290+The same mechanism is used for sending messages between apps on the same
274−sending messages between apps on different urbits.
291+urbit as for sending messages between apps on different urbits.
275
292
276−> We said earlier that we're not expecting a response. This is not entirely
293+> We said earlier that we're not expecting a response. This is not
277−> true: the `++coup` is called when we receive acknowledgment that the `++poke`
294+> entirely true: the `++coup` is called when we receive acknowledgment
278−> was called. We don't do anything with this information right now, but we
295+> that the `++poke` was called. We don't do anything with this
279−> could.
296+> information right now, but we could.
280
297
281**Exercises**:
298**Exercises**:
282
299
283−- Extend either of the apps in the first two exercises to accept input over
300+- Extend either of the apps in the first two exercises to accept input
284− the network in the same way as `pong`.
301+ over the network in the same way as `pong`.
285
302
286- Modify `examples-pong` to print out a message when it receives
303- Modify `examples-pong` to print out a message when it receives
287 acknowledgement.
304 acknowledgement.
288
305
289−- Write two apps, `even` and `odd`. When you pass an atom to `even`, check
306+- Write two apps, `even` and `odd`. When you pass an atom to `even`,
290− whether it's even. If so, divide it by two and recurse; otherwise, poke
307+ check whether it's even. If so, divide it by two and recurse;
291− `odd` with it. When `odd` receives an atom, check whether it's equal to one.
308+ otherwise, poke `odd` with it. When `odd` receives an atom, check
292− If so, terminate, printing "%success". Otherwise, check whether it's odd. If
309+ whether it's equal to one. If so, terminate, printing "%success".
293− so, multiply it by three, add one, and recurse; otherwise, poke `even` with
310+ Otherwise, check whether it's odd. If so, multiply it by three, add
294− it. When either app receives a number, print it out along with the name of
311+ one, and recurse; otherwise, poke `even` with it. When either app
295− the app. In the end, you should be able to watch Collatz's conjecture play
312+ receives a number, print it out along with the name of the app. In
296− out between the two apps. Sample output:
313+ the end, you should be able to watch Collatz's conjecture play out
297−
314+ between the two apps. Sample output:
298−<!-- -->
315+
299−
316+```
300− ~fintud-macrep:dojo> :even &atom 18
317+~fintud-macrep:dojo> :even &atom 18
301− [%even 18]
318+[%even 18]
302− [%odd 9]
319+[%odd 9]
303− [%even 28]
320+[%even 28]
304− [%even 14]
321+[%even 14]
305− [%odd 7]
322+[%odd 7]
306− [%even 22]
323+[%even 22]
307− [%odd 11]
324+[%odd 11]
308− [%even 34]
325+[%even 34]
309− [%odd 17]
326+[%odd 17]
310− [%even 52]
327+[%even 52]
311− [%even 26]
328+[%even 26]
312− [%odd 13]
329+[%odd 13]
313− [%even 40]
330+[%even 40]
314− [%even 20]
331+[%even 20]
315− [%even 10]
332+[%even 10]
316− [%odd 5]
333+[%odd 5]
317− [%even 16]
334+[%even 16]
318− [%even 8]
335+[%even 8]
319− [%even 4]
336+[%even 4]
320− [%even 2]
337+[%even 2]
321− %success
338+%success
322−
339+```
323−- Put `even` and `odd` on two separate urbits and pass the messages over the
340+
324− network. Post a link to a working solution in :talk to receive a cookie.
341+- Put `even` and `odd` on two separate urbits and pass the messages
342+ over the network. Post a link to a working solution in :talk to
343+ receive a cookie.
docs/arvo/security-driver.md+255−215
@@ -1,262 +1,302 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: False
3+next: false
4sort: 20
4sort: 20
5title: Security drivers
5title: Security drivers
6---
6---
7
7
8# Security drivers
8# Security drivers
9
9
10−A security driver is a file in `/=home=/sec/<tld>/<domain>/hoon` that handles
10+A security driver is a file in `/=home=/sec/<tld>/<domain>/hoon`
11−the authentication for all HTTP requests to `https://<domain>.<tld>`. When
11+that handles the authentication for all HTTP requests to
12−anything in Urbit makes an HTTP request through `%eyre` (our web server), it
12+`https://<domain>.<tld>`. When anything in Urbit makes an HTTP
13−checks to see if we have a security driver for the requested domain, and if so
13+request through `%eyre` (our web server), it checks to see if we
14−filters the request through the driver. The security driver will usually either
14+have a security driver for the requested domain, and if so
15−decorate the request with the needed credentials if it has them, or else it will
15+filters the request through the driver. The security driver will
16−guide the user through the process of authenticating Urbit with the service.
16+usually either decorate the request with the needed credentials
17−
17+if it has them, or else it will guide the user through the
18−Each web service needs its own security driver, but most of them are pretty
18+process of authenticating Urbit with the service.
19−standard. We recommend starting with an existing security driver based on your
19+
20−needed authentication method (e.g. basic auth, OAuth1, OAuth2), and changing
20+Each web service needs its own security driver, but most of them
21−whatever is specific to your service.
21+are pretty standard. We recommend starting with an existing
22−
22+security driver based on your needed authentication method (e.g.
23−The best strategy for building security drivers is to copy a similar one and
23+basic auth, OAuth1, OAuth2), and changing whatever is specific to
24−tweak it until it works for you. The best representatives are Github for Basic
24+your service.
25−Authentication, Twitter for OAuth1, Slack for OAuth2 when access tokens don't
25+
26−expire, and Google APIs for OAuth2 when access tokens do expire. In most cases
26+The best strategy for building security drivers is to copy a
27−you'll need to make very few changes to one of these models.
27+similar one and tweak it until it works for you. The best
28−
28+representatives are Github for Basic Authentication, Twitter for
29−Still, it's worth seeing one or two built from the ground up. Here, we'll build
29+OAuth1, Slack for OAuth2 when access tokens don't expire, and
30−a connector for the Github API v3. The simplest way to interact with the Github
30+Google APIs for OAuth2 when access tokens do expire. In most
31−API is to just fetch https://api.github.com from the dojo:
31+cases you'll need to make very few changes to one of these
32+models.
33+
34+Still, it's worth seeing one or two built from the ground up.
35+Here, we'll build a connector for the Github API v3. The
36+simplest way to interact with the Github API is to just fetch
37+https://api.github.com from the dojo:
32
38
33 ~your-urbit:dojo> +https://api.github.com
39 ~your-urbit:dojo> +https://api.github.com
34
40
35−Github exposes a few endpoints to the general web, and the root endpoint is one
41+Github exposes a few endpoints to the general web, and the root
36−of them. This gives you a textual representation of a JSON object that contains
42+endpoint is one of them. This gives you a textual representation
37−a bunch of urls to other parts of Github's API. This is exactly the same
43+of a JSON object that contains a bunch of urls to other parts of
38−response you would get from just running `curl https://api.github.com` on UNIX.
44+Github's API. This is exactly the same response you would get
45+from just running `curl https://api.github.com` on UNIX.
39
46
40−If we don't have a security driver for Github yet, many of the endpoints won't
47+If we don't have a security driver for Github yet, many of the
41−be accessible, or they will only have publicly accessible information. Most of
48+endpoints won't be accessible, or they will only have publicly
42−what we care about requires us to be authenticated.
49+accessible information. Most of what we care about requires us
50+to be authenticated.
43
51
44## Basic auth
52## Basic auth
45
53
46Here's a simple security driver:
54Here's a simple security driver:
47
55
48− :: Test url +https://api.github.com/user
56+```
49− ::
57+:: Test url +https://api.github.com/user
50− :::: /hoon/github/com/sec
58+::
51− ::
59+:::: /hoon/github/com/sec
52− /+ basic-auth
60+ ::
53− !:
61+/+ basic-auth
54− |_ {bal/(bale keys:basic-auth) $~}
62+!:
55− ++ aut ~(standard basic-auth bal ~)
63+|_ {bal/(bale keys:basic-auth) $~}
56− ++ filter-request out-adding-header:aut
64+++ aut ~(standard basic-auth bal ~)
57− --
65+++ filter-request out-adding-header:aut
58−
66+--
59−Github supports authentication through either Basic Authentication or OAuth2.
67+```
60−We'll show the basic auth example first, but in general we'd prefer OAuth2.
68+
61−
69+Github supports authentication through either Basic
62−Since this driver is for Github, put it in `/=home=/sec/com/github/hoon`.
70+Authentication or OAuth2. We'll show the basic auth example
71+first, but in general we'd prefer OAuth2.
72+
73+Since this driver is for Github, put it in
74+`/=home=/sec/com/github/hoon`.
63
75
64To try this out, we first have to initialize our credentials with
76To try this out, we first have to initialize our credentials with
65−`|init-auth-basic`, which prompts for the url of the service (api.github.com),
77+`|init-auth-basic`, which prompts for the url of the service
66−username, and password. It stores this information in a manner that `%eyre`
78+(api.github.com), username, and password. It stores this
67−knows how to decode and send put into your `bale`. Note that `|init-auth-basic`
79+information in a manner that `%eyre` knows how to decode and send
68−is standard for basic auth, but other auth schemes (like OAuth) are initialized
80+put into your `bale`. Note that `|init-auth-basic` is standard for
81+basic auth, but other auth schemes (like OAuth) are initialized
69in other ways.
82in other ways.
70
83
71After you've run `|init-auth-basic`, you should be able to run
84After you've run `|init-auth-basic`, you should be able to run
72−`+https://api.github.com/user` and get a response indicating who you're logged
85+`+https://api.github.com/user` and get a response indicating who
73−in as.
86+you're logged in as.
74
87
75You can similarly POST:
88You can similarly POST:
76
89
77 ~your-urbit:dojo> +https://api.github.com/gists &json _(cork poja need) '{"files":{"file1.txt":{"content":"can\'t stop the signal"}}}'
90 ~your-urbit:dojo> +https://api.github.com/gists &json _(cork poja need) '{"files":{"file1.txt":{"content":"can\'t stop the signal"}}}'
78
91
79−This creates a gist, go to `https://gist.github.com/<username>` to see it.
92+This creates a gist, go to `https://gist.github.com/<username>` to
93+see it.
80
94
81−The `+url` syntax used in "source" position makes a GET request, but if you pass
95+The `+url` syntax used in "source" position makes a GET request,
82−it data it will make a POST request with that data as the body. This request is
96+but if you pass it data it will make a POST request with that
83−authenticated seamlessly. You can send data of any mark as long as it has a
97+data as the body. This request is authenticated seamlessly. You
84−conversion path to `%mime`.
98+can send data of any mark as long as it has a conversion path to
99+`%mime`.
85
100
86−Let's take a look at the code and see if we can figure out what's happening
101+Let's take a look at the code and see if we can figure out what's
87−here:
102+happening here:
88
103
89 /+ basic-auth
104 /+ basic-auth
90
105
91−First, we load a library called `basic-auth`. This library exposes two items.
106+First, we load a library called `basic-auth`. This library
92−`keys` is the type of the basic auth key, which is essentially a base64'd
107+exposes two items. `keys` is the type of the basic auth key,
93−username and password. `standard` accepts a `bale` and produces a core with
108+which is essentially a base64'd username and password.
94−everything we'll need to implement basic auth.
109+`standard` accepts a `bale` and produces a core with everything
95−
110+we'll need to implement basic auth.
96−Every security driver needs some amount of state, which is stored in `bale`.
111+
97−`bale` contains:
112+Every security driver needs some amount of state, which is stored
98−
113+in `bale`. `bale` contains:
99−- `our`, our urbit name
114+
100−- `now`, the current time
115+- `our`, our urbit name
101−- `eny`, 256 bits of entropy
116+- `now`, the current time
102−- `byk`, the urbit, desk and case which the security driver is running from
117+- `eny`, 256 bits of entropy
103−- `usr`, the particular identity we're logging in with
118+- `byk`, the urbit, desk and case which the security driver is
104−- `dom`, the site we're accessing
119+ running from
105−- `key`, the secrets required to authenticate requests. The type for this
120+- `usr`, the particular identity we're logging in with
106− entry is supplied by the programmer as the argument to `++bale`. Thus, in
121+- `dom`, the site we're accessing
107− our case, `key` in our bale is of type `keys:basic-auth`.
122+- `key`, the secrets required to authenticate requests. The type
108−
123+ for this entry is supplied by the programmer as the argument to
109−Additionally, a security driver contains at least one out of the five "special"
124+ `++bale`. Thus, in our case, `key` in our bale is of type
110−arms:
125+ `keys:basic-auth`.
111−
126+
112−- `++filter-request` is a function which takes a hiss (http request) and
127+Additionally, a security driver contains at least one out of the
113− produces a `sec-move` (defined in zuse), which is one of:
128+five "special" arms:
114−
129+
115−- `[%send hiss]`, which sends the new hiss. This is the case in, for example,
130+- `++filter-request` is a function which takes a hiss (http
116− basic auth, where all we need to do is add an extra header to the request.
131+ request) and produces a `sec-move` (defined in zuse), which is
117−- `[%give httr]`, where httr is an http response. This immediately returns an
132+ one of:
118− http response to the sender.
133+
119−- `[%show purl]`, where a purl is a parsed url. This displays a message asking
134+ * `[%send hiss]`, which sends the new hiss. This is the
120− the user to visit the given url to continue the authentication process.
135+ case in, for example, basic auth, where all we need to do is
121−- `[%redo ~]`, which redoes the request.
136+ add an extra header to the request.
122−
137+ * `[%give httr]`, where httr is an http response. This
123−`%eyre` calls `++filter-request` just before sending an HTTP request to the
138+ immediately returns an http response to the sender.
124−specified domain. This allows the security driver to filter the requests,
139+ * `[%show purl]`, where a purl is a parsed url. This displays
125−decorating them with authentication data.
140+ a message asking the user to visit the given url to continue
126−
141+ the authentication process.
127−- `++filter-response` is a function which takes an httr and produces a
142+ * `[%redo ~]`, which redoes the request.
128− `sec-move`. `%eyre` calls it after receiving an HTTP response from the
143+
129− specified domain. This allows the security driver to handle authentication
144+ `%eyre` calls `++filter-request` just before sending an HTTP
130− errors, commonly caused by expired tokens, and retry the request.
145+ request to the specified domain. This allows the security
131−
146+ driver to filter the requests, decorating them with
132−- `++receive-auth-query-string` is a function that takes a `quay` (list of
147+ authentication data.
133− query parameters) and produces a `sec-move`. `%eyre` calls it when it
148+
134− receives a request on the callback url. Generally, this happens in OAuth
149+- `++filter-response` is a function which takes an httr and
135− after the user has granted access to the urbit app, and the service makes a
150+ produces a `sec-move`. `%eyre` calls it after receiving an
136− request to the callback url with the code. The security driver should then
151+ HTTP response from the specified domain. This allows the
137− make a request to convert the code into an access token.
152+ security driver to handle authentication errors, commonly
138−
153+ caused by expired tokens, and retry the request.
139−- `++receive-auth-response` is a function that takes an httr and produces a
154+
140− `sec-move`. `%eyre` calls it when it gets a response to the request made in
155+- `++receive-auth-query-string` is a function that takes a `quay`
141− `++receive-auth-query-string`.
156+ (list of query parameters) and produces a `sec-move`. `%eyre`
142−
157+ calls it when it receives a request on the callback url.
143−- `++update` is a function which converts old state to a new format when the
158+ Generally, this happens in OAuth after the user has granted
144− security driver gets updated. If you want to just get rid of the old state,
159+ access to the urbit app, and the service makes a request to the
145− define `++discard-state` as `~`.
160+ callback url with the code. The security driver should then
146−
161+ make a request to convert the code into an access token.
147−For basic auth, we only have to worry about `++filter-request`, since all we
162+
148−need to do is add the correct header to each request. Our `++aut` core contains
163+- `++receive-auth-response` is a function that takes an httr and
149−a function `++out-addding-header`, which does exactly what we want.
164+ produces a `sec-move`. `%eyre` calls it when it gets a
150−
165+ response to the request made in `++receive-auth-query-string`.
151−Where do the keys come from, though? Remember we ran `|init-auth-basic` to input
166+
152−them. This just prompts you for the service url, username, and password, and it
167+- `++update` is a function which converts old state to a new
153−stores them next to the driver. In our case, that's
168+ format when the security driver gets updated. If you want to
154−`/=home=/sec/com/gitub/api/atom`. These keys are stored encrypted, and you don't
169+ just get rid of the old state, define `++discard-state` as `~`.
155−want to edit them directly. `%eyre` loads them directly into your `bale`.
170+
171+For basic auth, we only have to worry about `++filter-request`,
172+since all we need to do is add the correct header to each
173+request. Our `++aut` core contains a function
174+`++out-addding-header`, which does exactly what we want.
175+
176+Where do the keys come from, though? Remember we ran
177+`|init-auth-basic` to input them. This just prompts you for the
178+service url, username, and password, and it stores them next to
179+the driver. In our case, that's
180+`/=home=/sec/com/gitub/api/atom`. These keys are stored
181+encrypted, and you don't want to edit them directly. `%eyre`
182+loads them directly into your `bale`.
156
183
157## OAuth2
184## OAuth2
158
185
159−Most services are better accessed through some form of OAuth. Github can be
186+Most services are better accessed through some form of OAuth.
160−accessed with OAuth2, which is a little more complicated than basic auth, but
187+Github can be accessed with OAuth2, which is a little more
161−not hugely so.
188+complicated than basic auth, but not hugely so.
162
189
163The basic steps for OAuth2 are these:
190The basic steps for OAuth2 are these:
164
191
165−- create an app and get its client id and client secret (on Github)
192+- create an app and get its client id and client secret (on
166−- store these in Urbit (with `|init-oauth2`)
193+ Github)
167−- try to make a request that requires authentication. The security driver
194+- store these in Urbit (with `|init-oauth2`)
168− should prompt you to visit a particular url.
195+- try to make a request that requires authentication. The
169−- visit the url and click "authorize", which gives the app access to your
196+ security driver should prompt you to visit a particular url.
170− account
197+- visit the url and click "authorize", which gives the app access
171−- the security driver will turn the code it receives into an access token
198+ to your account
172−- on every request, include that access token
199+- the security driver will turn the code it receives into an
200+ access token
201+- on every request, include that access token
173
202
174Creating an app is easy and
203Creating an app is easy and
175−[well-documented](https://github.com/settings/applications/new). If necessary,
204+[well-documented](https://github.com/settings/applications/new).
176−set the callback url to `/~/ac/github.com/_state/in`, and note its client id and
205+If necessary, set the callback url to
206+`/~/ac/github.com/_state/in`, and note its client id and client
207+secret.
208+
209+Run `|init-oauth2`, which will prompt for the hostname
210+(api.github.com), client id, and client secret. This will store
211+the information, encrypted, in `/=home=/sec/com/github.atom`,
212+just as with Basic Authentication.
213+
214+Now we need a security driver. Use this:
215+
216+```
217+:: Test url +https://api.github.com/user
218+::
219+:::: /hoon/github/com/sec
220+ ::
221+/+ oauth2
222+!:
223+::::
224+ ::
225+|_ {bal/(bale keys:oauth2) tok/token:oauth2}
226+++ scopes :: comment out scopes to taste
227+ :~ 'user' 'user:email' 'user:follow' 'public_repo' 'repo'
228+ 'repo_deployment' 'repo:status' 'delete_repo' 'notifications'
229+ 'gist' 'read:repo_hook' 'write:repo_hook' 'admin:repo_hook'
230+ 'admin:org_hook' 'read:org' 'write:org' 'admin:org'
231+ 'read:public_key' 'write:public_key' 'admin:public_key'
232+ ==
233+:: ++aut is a "standard oauth2" core, which implements the
234+:: most common handling of oauth2 semantics. see lib/oauth2 for more details,
235+:: and examples at the bottom of the file.
236+++ aut (~(standard oauth2 bal tok) . |=(tok/token:oauth2 +>(tok tok)))
237+++ filter-request
238+ %^ out-add-query-param:aut 'access_token'
239+ scope=~[%client %admin]
240+ oauth-dialog='https://github.com/login/oauth/authorize'
241+::
242+++ receive-auth-query-string
243+ %- in-code-to-token:aut
244+ url='https://github.com/login/oauth/api/access_token'
245+++ receive-auth-response bak-save-token:aut
246+--
247+```
248+
249+The oauth2 library provides the main "engine" in `standard`, just
250+like in basic auth, except that we also have to specify a
251+function to save the access token when we get it. It's worth
252+noting that this library is well documented in the source,
253+including examples: `/=home=/lib/oauth2/hoon`.
254+
255+Running `+http://api.github.com/user` tries to make a request to
256+Github with authentication. This loads into the bale the keys,
257+which are of type `keys:oauth2`, defined in the oauth2 library.
258+We don't have to worry about the specifics of these keys -- the
259+library handles them -- but they include the client id and the
177client secret.
260client secret.
178
261
179−Run `|init-oauth2`, which will prompt for the hostname (api.github.com), client
262+Before the request is sent, `%eyre` calls `++filter-request` to
180−id, and client secret. This will store the information, encrypted, in
263+decorate it with authentication headers. Since we don't yet have
181−`/=home=/sec/com/github.atom`, just as with Basic Authentication.
264+an access token, we need to prompt the user to visit the "dialog"
182−
265+url. `++out-add-query-param` checks whether we have a token, and
183−Now we need a security driver. Use this:
266+if we don't it produces `%show` and a message telling the user to
184−
267+visit the `oauth-dialog` url we provide, with the extra query
185− :: Test url +https://api.github.com/user
268+parameters added (client id, scopes, and state).
186− ::
269+
187− :::: /hoon/github/com/sec
270+When the user visits that url, Github will ask them to log in and
188− ::
271+authorize the application. When they do so, Github will post a
189− /+ oauth2
272+request to `/~/ac/github.com/_state/in` with a code in the query
190− !:
273+string. `%eyre` sends the query string to
191− ::::
274+`++receive-auth-query-string`, which is a function that takes a
192− ::
275+quay and produces a `sec-move`. `++in-code-to-token` from our
193− |_ {bal/(bale keys:oauth2) tok/token:oauth2}
276+`aut` takes the `code` parameter from the quay and uses it to
194− ++ scopes :: comment out scopes to taste
277+create a request to the given url. This request includes the
195− :~ 'user' 'user:email' 'user:follow' 'public_repo' 'repo'
278+client id, the client secret, and the code we just got.
196− 'repo_deployment' 'repo:status' 'delete_repo' 'notifications'
279+
197− 'gist' 'read:repo_hook' 'write:repo_hook' 'admin:repo_hook'
280+Github's response to that request includes an access token in its
198− 'admin:org_hook' 'read:org' 'write:org' 'admin:org'
281+body. This is handled in `++receive-auth-response`.
199− 'read:public_key' 'write:public_key' 'admin:public_key'
282+`++bak-save-token` extracts that token and gives it to the
200− ==
283+function that we passed in the definition of `++aut`. In our
201− :: ++aut is a "standard oauth2" core, which implements the
202− :: most common handling of oauth2 semantics. see lib/oauth2 for more details,
203− :: and examples at the bottom of the file.
204− ++ aut (~(standard oauth2 bal tok) . |=(tok/token:oauth2 +>(tok tok)))
205− ++ filter-request
206− %^ out-add-query-param:aut 'access_token'
207− scope=~[%client %admin]
208− oauth-dialog='https://github.com/login/oauth/authorize'
209− ::
210− ++ receive-auth-query-string
211− %- in-code-to-token:aut
212− url='https://github.com/login/oauth/api/access_token'
213− ++ receive-auth-response bak-save-token:aut
214− --
215−
216−The oauth2 library provides the main "engine" in `standard`, just like in basic
217−auth, except that we also have to specify a function to save the access token
218−when we get it. It's worth noting that this library is well documented in the
219−source, including examples: `/=home=/lib/oauth2/hoon`.
220−
221−Running `+http://api.github.com/user` tries to make a request to Github with
222−authentication. This loads into the bale the keys, which are of type
223−`keys:oauth2`, defined in the oauth2 library. We don't have to worry about the
224−specifics of these keys -- the library handles them -- but they include the
225−client id and the client secret.
226−
227−Before the request is sent, `%eyre` calls `++filter-request` to decorate it with
228−authentication headers. Since we don't yet have an access token, we need to
229−prompt the user to visit the "dialog" url. `++out-add-query-param` checks
230−whether we have a token, and if we don't it produces `%show` and a message
231−telling the user to visit the `oauth-dialog` url we provide, with the extra
232−query parameters added (client id, scopes, and state).
233−
234−When the user visits that url, Github will ask them to log in and authorize the
235−application. When they do so, Github will post a request to
236−`/~/ac/github.com/_state/in` with a code in the query string. `%eyre` sends the
237−query string to `++receive-auth-query-string`, which is a function that takes a
238−quay and produces a `sec-move`. `++in-code-to-token` from our `aut` takes the
239−`code` parameter from the quay and uses it to create a request to the given url.
240−This request includes the client id, the client secret, and the code we just
241−got.
242−
243−Github's response to that request includes an access token in its body. This is
244−handled in `++receive-auth-response`. `++bak-save-token` extracts that token and
245−gives it to the function that we passed in the definition of `++aut`. In our
246case, we just save that token into our state for later usage.
284case, we just save that token into our state for later usage.
247
285
248−Now we have the credentials we need, so `++filter-request` is called with the
286+Now we have the credentials we need, so `++filter-request` is
249−original request, and `++out-add-query-param` passes the request through with
287+called with the original request, and `++out-add-query-param`
250−the addition of the query parameter `access_token`.
288+passes the request through with the addition of the query
289+parameter `access_token`.
251
290
252Just like with Basic Authentication, you should be able to run
291Just like with Basic Authentication, you should be able to run
253−`+https://api.github.com/user` and get a response indicating who you're logged
292+`+https://api.github.com/user` and get a response indicating who
254−in as.
293+you're logged in as.
255
294
256−Some drivers a slightly more complicated. For example, Github's access token's
295+Some drivers a slightly more complicated. For example, Github's
257−don't expire, but that's not the case for all service. The googleapis driver
296+access token's don't expire, but that's not the case for all
258−shows the flow when the access token can expire. Twitter uses OAuth1, for which
297+service. The googleapis driver shows the flow when the access
259−we have another library.
298+token can expire. Twitter uses OAuth1, for which we have another
299+library.
260
300
261−Remember, of course, that the best strategy for building security drivers is to
301+Remember, of course, that the best strategy for building security
262−copy a similar one and tweak it to taste.
302+drivers is to copy a similar one and tweak it to taste.
docs/arvo/state.md+69−62
@@ -1,16 +1,18 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
3+next: true
4sort: 14
4sort: 14
5title: State
5title: State
6---
6---
7
7
8−# State
8+State
9+=====
9
10
10−In the last section we built a few small apps that sent moves. These apps were
11+In the last section we built a few small apps that sent moves. These
11−entirely stateless, however. Most useful apps require some amount of state.
12+apps were entirely stateless, however. Most useful apps require some
12−Let's build a trivial stateful app. It'll keep a running the sum of all the
13+amount of state. Let's build a trivial stateful app. It'll keep a
13−atoms we poke it with. Here's `app/examples/sum.hoon`:
14+running the sum of all the atoms we poke it with. Here's
15+`app/examples/sum.hoon`:
14
16
15 /? 314
17 /? 314
16 !:
18 !:
@@ -36,67 +38,72 @@ We can start it with `|start %examples-sum`, and then run it:
36
38
37We can see that app state is being saved, but when, where, and how?
39We can see that app state is being saved, but when, where, and how?
38
40
39−The state is stored as the second thing in the `|_` line. In our case, it's
41+The state is stored as the second thing in the `|_` line. In our case,
40−simply an atom named `state`. We change it by producing our state not with
42+it's simply an atom named `state`. We change it by producing our
41−`+>.$` (as before), but with `+>.$(state (add state arg))`. We've seen all these
43+state not with `+>.$` (as before), but with
42−parts before, but you might not recognize them.
44+`+>.$(state (add state arg))`. We've seen all these parts before, but
45+you might not recognize them.
43
46
44Recall in the first chapter that we recursed with the expression
47Recall in the first chapter that we recursed with the expression
45−`$(b (add 3 b))`. This meant "produce `$` with `b` changed to `(add 3 b)`.
48+`$(b (add 3 b))`. This meant "produce `$` with `b` changed to
46−Similarly, `+>.$(state (add state arg))` means "produce `+>.$` (i.e. our
49+`(add 3 b)`. Similarly, `+>.$(state (add state arg))` means "produce
47−context, which contains our state) with `state` changed to `(add state arg)`.
50+`+>.$` (i.e. our context, which contains our state) with `state` changed
48−
51+to `(add state arg)`.
49−At a high level, then, when we handle state, we do it explicitly. It's passed in
52+
50−and produced explicitly. In Unix systems, application state is just a block of
53+At a high level, then, when we handle state, we do it explicitly. It's
51−memory, which you need to serialize to disk if you want to keep it around for
54+passed in and produced explicitly. In Unix systems, application state is
52−very long.
55+just a block of memory, which you need to serialize to disk if you want
53−
56+to keep it around for very long.
54−In Urbit, app state is a single (usually complex) value. In our example, we have
57+
55−very simple state, so we defined `state/@`, meaning that our state is an atom.
58+In Urbit, app state is a single (usually complex) value. In our example,
56−Of course, `state` is just a name, and you're free to name your state whatever
59+we have very simple state, so we defined `state/@`, meaning that our
57−you like. But let's clarify a couple other things before we continue.
60+state is an atom. Of course, `state` is just a name, and you're free to
58−
61+name your state whatever you like. But let's clarify a couple other
59−First, `bowl` is a set of general global states. This set is managed by the
62+things before we continue.
60−system. It includes things like `now` (the current time), `our` (our urbit
63+
61−identity), and `eny` (256 bits of guaranteed-fresh entropy). For the full list
64+First, `bowl` is a set of general global states. This set is managed by
62−of things in `++bowl`, search for `++ bowl` (note: two spaces) in
65+the system. It includes things like `now` (the current time), `our` (our
63−`/arvo/zuse.hoon`.
66+urbit identity), and `eny` (256 bits of guaranteed-fresh entropy). For
64−
67+the full list of things in `++bowl`, search for `++ bowl` (note: two
65−> This is, perhaps, the most common way to learn Hoon. The easiest way to learn
68+spaces) in `/arvo/zuse.hoon`.
66−> about an identifier you see in code is to search in `/arvo/zuse.hoon` and
69+
67−> `/arvo/hoon.hoon` for it.\\ These "standard libraries" are usually simple to
70+> This is, perhaps, the most common way to learn Hoon. The easiest way
68−> read. Since Urbit's codebase is less than 15000 lines of code combined,
71+> to learn about an identifier you see in code is to search in
69−> including the hoon parser, the compiler, and the `/arvo` microkernel, you can
72+> `/arvo/zuse.hoon` and `/arvo/hoon.hoon` for it.\
70−> usually use the code and its comments as a reference doc.
73+> These "standard libraries" are usually simple to read. Since Urbit's
74+> codebase is less than 15000 lines of code combined, including the hoon
75+> parser, the compiler, and the `/arvo` microkernel, you can usually use
76+> the code and its comments as a reference doc.
71>
77>
72> You can also read
78> You can also read
73−> [zuse.hoon](https://github.com/urbit/arvo/blob/master/arvo/zuse.hoon) and
79+> [zuse.hoon](https://github.com/urbit/arvo/blob/master/arvo/zuse.hoon)
74−> [hoon.hoon](https://github.com/urbit/arvo/blob/master/arvo/hoon.hoon) in your
80+> and
75−> browser.
81+> [hoon.hoon](https://github.com/urbit/arvo/blob/master/arvo/hoon.hoon)
76−
82+> in your browser.
77−The second thing we should clear up is this: Urbit needs no "serialize to disk"
83+
78−step. Everything you produce in the app state is persistent across calls to the
84+The second thing we should clear up is this: Urbit needs no "serialize
79−app, restarts of the urbit, and even power failure. If you want to write to the
85+to disk" step. Everything you produce in the app state is persistent
80−Unix filesystem, you can, but it's not needed for persistence. Urbit has
86+across calls to the app, restarts of the urbit, and even power failure.
81−transactional events, which makes it an [ACID operating
87+If you want to write to the Unix filesystem, you can, but it's not
82−system](https://en.wikipedia.org/wiki/ACID). Thus, you don't have to worry about
88+needed for persistence. Urbit has transactional events, which makes it
83−persistence when programming in Urbit, or ever go through the hassle of having
89+an [ACID operating system](https://en.wikipedia.org/wiki/ACID). Thus,
84−to set up and write to a database.
90+you don't have to worry about persistence when programming in Urbit, or
91+ever go through the hassle of having to set up and write to a database.
85
92
86**Exercises**:
93**Exercises**:
87
94
88- Modify `:examples-sum` to reset the counter when you poke it with 0.
95- Modify `:examples-sum` to reset the counter when you poke it with 0.
89
96
90−- Write an app that prints out the previous value you poked it with. Sample
97+- Write an app that prints out the previous value you poked it with.
91− output:
98+ Sample output:
92−
99+
93−<!-- -->
100+```
94−
101+~fintud-macrep:dojo> :examples-last 7
95− ~fintud-macrep:dojo> :examples-last 7
102+[%last 0]
96− [%last 0]
103+>=
97− >=
104+~fintud-macrep:dojo> :examples-last [1 2 3]
98− ~fintud-macrep:dojo> :examples-last [1 2 3]
105+[%last 7]
99− [%last 7]
106+>=
100− >=
107+~fintud-macrep:dojo> :examples-last 'howdy'
101− ~fintud-macrep:dojo> :examples-last 'howdy'
108+[%last [1 2 3]]
102− [%last [1 2 3]]
109+```
docs/arvo/subject.md+43−36
@@ -1,11 +1,12 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
4sort: 12
3sort: 12
4+next: true
5title: The subject
5title: The subject
6---
6---
7
7
8−# The subject
8+The subject
9+===========
9
10
10Now we're going to cover the boiler plate that we skimmed over earlier.
11Now we're going to cover the boiler plate that we skimmed over earlier.
11
12
@@ -13,24 +14,26 @@ Now we're going to cover the boiler plate that we skimmed over earlier.
13 :- %noun
14 :- %noun
14 =< (sum [1.000 2.000])
15 =< (sum [1.000 2.000])
15
16
16−The first rune, `:-` (colhep, aka [:cons](../../hoon/twig/col-cell/hep-cons/)),
17+The first rune, `:-` (colhep, aka
17−constructs the 2-element cell that will be our program. The first element,
18+[:cons](../../hoon/twig/col-cell/hep-cons/)), constructs the 2-element cell
18−`%say`, tells the interpreter what to produce--in this case a value.
19+that will be our program. The first element, `%say`, tells the
20+interpreter what to produce--in this case a value.
19
21
20−The second element is `|=`, which we know produces a function. `|=`'s first
22+The second element is `|=`, which we know produces a function. `|=`'s
21−child is its argument(s), which in this case is any noun (`*`). Its second child
23+first child is its argument(s), which in this case is any noun (`*`).
22−is the remainder of the program.
24+Its second child is the remainder of the program.
23
25
24−Similarly, the rest of the program (which we construct with another `:-`) is a
26+Similarly, the rest of the program (which we construct with another
25−cell of the literal `%noun`, which tells the shell that we're producing a value
27+`:-`) is a cell of the literal `%noun`, which tells the shell that we're
26−of type `noun`, and the code that we run to actually produce our value of the
28+producing a value of type `noun`, and the code that we run to actually
27−type `noun`.
29+produce our value of the type `noun`.
28
30
29`=<` ([tisgal](../../hoon/twig/tis-flow/gal-rap/)) is a rune that takes two
31`=<` ([tisgal](../../hoon/twig/tis-flow/gal-rap/)) is a rune that takes two
30−children. The second child is the context against which we run the first child.
32+children. The second child is the context against which we run the first
31−So in this case, we are running the expression `(sum [1.000 2.000])` against
33+child. So in this case, we are running the expression
32−everything contained within the `|%`. In Hoon, we call the code executed the
34+`(sum [1.000 2.000])` against everything contained within the `|%`. In
33−"formula" and its context the "subject".
35+Hoon, we call the code executed the "formula" and its context the
36+"subject".
34
37
35 ::::::::::::::::::::::::::::::
38 ::::::::::::::::::::::::::::::
36 =< (sum [1.000 2.000]) :: formula
39 =< (sum [1.000 2.000]) :: formula
@@ -60,35 +63,39 @@ everything contained within the `|%`. In Hoon, we call the code executed the
60 -- ::
63 -- ::
61 ::::::::::::::::::::::::::::::
64 ::::::::::::::::::::::::::::::
62
65
63−In nearly every language there is a similar concept of a "context" in which
66+In nearly every language there is a similar concept of a "context" in
64−expressions are executed. For example, in C this includes things like the call
67+which expressions are executed. For example, in C this includes things
65−stack, stack variables, and so on.
68+like the call stack, stack variables, and so on.
66
69
67−Hoon is unique in that this context is a first-class value. Scheme allows a sort
70+Hoon is unique in that this context is a first-class value. Scheme
68−of reification of the context through continuations, and some may see a parallel
71+allows a sort of reification of the context through continuations, and
69−to Forth's stack, but Hoon takes the concept one step further.
72+some may see a parallel to Forth's stack, but Hoon takes the
73+concept one step further.
70
74
71Our starting subject is the [standard library](../../hoon/library), which is
75Our starting subject is the [standard library](../../hoon/library), which is
72−defined in `/arvo/hoon.hoon` and `/arvo/zuse.hoon`. This is where functions like
76+defined in `/arvo/hoon.hoon` and `/arvo/zuse.hoon`. This is where
73−`add` are defined. When we define a core with `|%`, we don't throw away the
77+functions like `add` are defined. When we define a core with `|%`, we
74−subject (i.e. the standard library); rather, we stack the new core on top of the
78+don't throw away the subject (i.e. the standard library); rather, we
75−old subject so that both are accessible.
79+stack the new core on top of the old subject so that both are
80+accessible.
76
81
77**Exercises**:
82**Exercises**:
78
83
79−- Pass `++sum` its arguments (`2.000` and `3.000`) from the command line.
84+- Pass `++sum` its arguments (`2.000` and `3.000`) from the
85+ command line.
80
86
81−- Comment out all of the arms of the `|%`. Now add another arm and call it
87+- Comment out all of the arms of the `|%`. Now add another arm and
82− `++add`, have it accept two arguments and produce 42 (regardless of input).
88+ call it `++add`, have it accept two arguments and produce 42
83− Change the `=<` line to `[(add 5 7) (^add 5 7)]`. Can you recognize what's
89+ (regardless of input). Change the `=<` line to
84− happening?
90+ `[(add 5 7) (^add 5 7)]`. Can you recognize what's happening?
85
91
86Cheatsheet:
92Cheatsheet:
87
93
88−- To pass arguments from the command line to a program, you replace the `*` in
94+- To pass arguments from the command line to a program, you replace
89− the first line of the boiler plate to `{^ {{arg/TYPE $~} $~}}` where `TYPE`
95+ the `*` in the first line of the boiler plate to
90− is replaced with the type of argument you're expecting. Then `+euler1 a`
96+ `{^ {{arg/TYPE $~} $~}}` where `TYPE` is replaced with the type of
91− from the dojo sets `arg` to `a`.
97+ argument you're expecting. Then `+euler1 a` from the dojo sets `arg`
98+ to `a`.
92- The empty list is `~`
99- The empty list is `~`
93- Lisp-style cons (construct a cell/prepend an element) is
100- Lisp-style cons (construct a cell/prepend an element) is
94 `[new-element list]`
101 `[new-element list]`
docs/arvo/subscriptions.md+129−116
@@ -1,19 +1,20 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
3+next: true
4sort: 15
4sort: 15
5title: Subscriptions
5title: Subscriptions
6---
6---
7
7
8# Subscriptions
8# Subscriptions
9
9
10−We've dealt fairly extensively with "poke" messages to an app, but these are
10+We've dealt fairly extensively with "poke" messages to an app, but these
11−somewhat limited. A poke is a one-way message, but more often we want to
11+are somewhat limited. A poke is a one-way message, but more often we
12−subscribe to updates from another app. You could build a subscription model out
12+want to subscribe to updates from another app. You could build a
13−of one-way pokes, but it's such a common pattern that it's built into arvo.
13+subscription model out of one-way pokes, but it's such a common pattern
14+that it's built into arvo.
14
15
15−Let's take a look at two apps, `:examples-source` and `:examples-sink`. First,
16+Let's take a look at two apps, `:examples-source` and `:examples-sink`.
16−`:examples-source`:
17+First, `:examples-source`:
17
18
18 /? 314
19 /? 314
19 !:
20 !:
@@ -72,50 +73,54 @@ And secondly, `:examples-sink`:
72
73
73Cheat sheet:
74Cheat sheet:
74
75
75−- `&` (pam) can either be the boolean true (as can `%.y`, `0`), or the
76+- `&` (pam) can either be the boolean true (as can `%.y`, `0`), or
76− irregular wide form of the `?&` ([wutpam](../../hoon/twig/wut-test/pam-and))
77+ the irregular wide form of the `?&`
77− rune, which computes logical `AND` on its two children.
78+ ([wutpam](../../hoon/twig/wut-test/pam-and)) rune, which computes
79+ logical `AND` on its two children.
78
80
79−- Similar to `&`,`|` is either the boolean false (along with `%.n` and `1`),
81+- Similar to `&`,`|` is either the boolean false (along with `%.n` and
80− or the irregular short for of `?|`
82+ `1`), or the irregular short for of `?|`
81− ([wutbar](../../hoon/twig/wut-test/bar-or)), which computes logical `OR` on
83+ ([wutbar](../../hoon/twig/wut-test/bar-or)), which computes logical `OR`
82− its two children.
84+ on its two children.
83
85
84- `!` is the irregular wide form of `?!`
86- `!` is the irregular wide form of `?!`
85− ([wutzap](../../hoon/twig/wut-test/zap-not/)), which computes logical `NOT`
87+ ([wutzap](../../hoon/twig/wut-test/zap-not/)), which computes logical
86− on its child.
88+ `NOT` on its child.
87
89
88- `?~` ([wutsig](../../hoon/twig/wut-test/sig-ifno/)) is basically an
90- `?~` ([wutsig](../../hoon/twig/wut-test/sig-ifno/)) is basically an
89− if-then-else that checks whether condition `p` is `~` (null). `?~` is
91+ if-then-else that checks whether condition `p` is `~` (null). `?~`
90− slightly different from `?:(~ %tru %fal)` in that `?~` reduces to
92+ is slightly different from `?:(~ %tru %fal)` in that `?~` reduces to
91 `?:($=(%type value) %tru %false)`. `$=`
93 `?:($=(%type value) %tru %false)`. `$=`
92− ([buctis](../../hoon/twig/buc-mold/tis-coat/)) tests whether value `q` is of
94+ ([buctis](../../hoon/twig/buc-mold/tis-coat/)) tests whether value `q` is
93− type `p`. <!-- One thing to watch out for in hoon: if you do `?~`, it
95+ of type `p`.
94− affects the type of the conditional value: XXexample -->
96+<!-- One thing to watch out for in hoon: if you do `?~`, it
97+ affects the type of the conditional value: XXexample -->
95
98
96- `:_` ([colcab](../../hoon/twig/col-cell/cab-scon/)) is inverted `:-`: it
99- `:_` ([colcab](../../hoon/twig/col-cell/cab-scon/)) is inverted `:-`: it
97 accepts `p` and `q`, and produces `[q p]`.
100 accepts `p` and `q`, and produces `[q p]`.
98
101
99−- `++bowl` is the type of the system state within our app. For example, it
102+- `++bowl` is the type of the system state within our app. For
100− includes things like `our`, the name of the host urbit, and `now`, the
103+ example, it includes things like `our`, the name of the host urbit,
101− current time.
104+ and `now`, the current time.
102
105
103−- `$%` ([buccen](../../hoon/twig/buc-mold/cen-book/)) is a type constructor:
106+- `$%` ([buccen](../../hoon/twig/buc-mold/cen-book/)) is a type
104− it defines a new type, composed of `n` types that it is passed. For example
107+ constructor: it defines a new type, composed of `n` types that it is
105− `$% @ * ^ ==` is the type of either `@`, `*`, or a cell `^`.
108+ passed. For example `$% @ * ^ ==` is the type of either `@`,
106− <!-- XX this is a union, right? -->
109+ `*`, or a cell `^`.
110+<!-- XX this is a union, right? -->
107
111
108- You may have noticed the separate `|%`
112- You may have noticed the separate `|%`
109 ([barcen](../../hoon/twig/bar-core/cen-core/)) above the application core
113 ([barcen](../../hoon/twig/bar-core/cen-core/)) above the application core
110− `|_` ([barcab](../../hoon/twig/bar-core/cab-door/). We usually put our types
114+ `|_` ([barcab](../../hoon/twig/bar-core/cab-door/). We usually put our
111− in another core on top of the application core. We can access these type
115+ types in another core on top of the application core. We can access
112− from our `|_` because in `hoon.hoon` files, all cores are called against
116+ these type from our `|_` because in `hoon.hoon` files, all cores are
113− each other. (The shorthand for 'called' is `=>`.) Thus, the `|%` with the
117+ called against each other. (The shorthand for 'called' is `=>`.)
114− types is in the context of the `|_`, as it lies above it: `hoon.hoon`
118+ Thus, the `|%` with the types is in the context of the `|_`, as it
115− `=> |% w types => |_`
119+ lies above it: `hoon.hoon` `=> |% w types => |_`
116
120
117Here's some sample output of the two working together:
121Here's some sample output of the two working together:
118
122
123+
119 ~fintud-macrep:dojo> |start %examples-source
124 ~fintud-macrep:dojo> |start %examples-source
120 >=
125 >=
121 ~fintud-macrep:dojo> |start %examples-sink
126 ~fintud-macrep:dojo> |start %examples-sink
@@ -142,115 +147,123 @@ Here's some sample output of the two working together:
142### :examples-source
147### :examples-source
143
148
144Hopefully you can get a sense for what's happening here. When we poke
149Hopefully you can get a sense for what's happening here. When we poke
145−`:examples-sink` with `%on`, `:examples-sink` subscribes to `:examples-source`,
150+`:examples-sink` with `%on`, `:examples-sink` subscribes to
146−and so whenever we poke `:examples-source`, `:examples-sink` gets the update and
151+`:examples-source`, and so whenever we poke `:examples-source`,
147−prints it out. Then we unsubscribe by poking `:examples-sink` with `%off`, and
152+`:examples-sink` gets the update and prints it out. Then we unsubscribe
148−`:examples-sink` stops getting updates. We then resubscribe.
153+by poking `:examples-sink` with `%off`, and `:examples-sink` stops
154+getting updates. We then resubscribe.
149
155
150−There's a fair bit going on in this code. Let's look at `:examples-source`
156+There's a fair bit going on in this code. Let's look at
151−first.
157+`:examples-source` first.
152
158
153−Our definition of `move` is fairly specific, since we're only going to sending
159+Our definition of `move` is fairly specific, since we're only going to
154−one kind of move. The `%diff` move is a subscription update, and its content is
160+sending one kind of move. The `%diff` move is a subscription update, and
155−marked data which gall routes to our subscribers.
161+its content is marked data which gall routes to our subscribers.
156
162
157This is a slightly different kind of move than [we've dealt with so
163This is a slightly different kind of move than [we've dealt with so
158−far](/arvo/network). It's producing a result rather than calling other code
164+far](/arvo/network). It's producing a result rather than calling other
159−(i.e. it's a return rather than a function call), so if you recall the
165+code (i.e. it's a return rather than a function call), so if you recall
160−discussion of ducts, a layer gets popped off the duct rather than added to it.
166+the discussion of ducts, a layer gets popped off the duct rather than
161−This is why no wire is needed for the move -- we won't receive anything in
167+added to it. This is why no wire is needed for the move -- we won't
162−response to it.
168+receive anything in response to it.
163
169
164Anyways, there're two functions inside the `|_`. We already know when
170Anyways, there're two functions inside the `|_`. We already know when
165−`++poke-noun` is called. `++peer` is called when someone tries to subscribe to
171+`++poke-noun` is called. `++peer` is called when someone tries to
166−our app. Of course, you don't just subscribe to an app; you subscribe to a path
172+subscribe to our app. Of course, you don't just subscribe to an app; you
167−on that app. This path comes in as the argument to `++peer`.
173+subscribe to a path on that app. This path comes in as the argument to
168−
174+`++peer`.
169−In our case, we don't care what path you subscribed on, and all we do is print
175+
170−out that you subscribed. Arvo keeps track of your subscriptions, so you don't
176+In our case, we don't care what path you subscribed on, and all we do is
171−have to. You can access your subscribers by looking at `sup` in the bowl that's
177+print out that you subscribed. Arvo keeps track of your subscriptions,
172−passed in. `sup` is of type `(map bone {@p path})`, which associates bones with
178+so you don't have to. You can access your subscribers by looking at
173−the urbit who subscribed, and which path they subscribed on. If you want to
179+`sup` in the bowl that's passed in. `sup` is of type
180+`(map bone {@p path})`, which associates bones with the urbit who
181+subscribed, and which path they subscribed on. If you want to
174communicate with your subscribers, send them messages along their bone.
182communicate with your subscribers, send them messages along their bone.
175
183
176−`++poke-noun` "spams" the given argument to all our subscribers. There's a few
184+`++poke-noun` "spams" the given argument to all our subscribers.
177−things we haven't seen before. Firstly, `:_(a b)` is the same as `[b a]`. It's
185+There's a few things we haven't seen before. Firstly, `:_(a b)` is the
178−just a convenient way of formatting things when the first thing in a cell is
186+same as `[b a]`. It's just a convenient way of formatting things when
179−much more complicated than the second. Thus, we're producing our state
187+the first thing in a cell is much more complicated than the second.
180−unchanged.
188+Thus, we're producing our state unchanged.
181
189
182−Our list of moves is the result of a call to `++turn`. `++turn` is what many
190+Our list of moves is the result of a call to `++turn`. `++turn` is what
183−languages call "map" -- it runs a function on every item in a list and collects
191+many languages call "map" -- it runs a function on every item in a list
184−the results in a list. The list is `(prey /example-path hid)` and the function
192+and collects the results in a list. The list is `(prey /example-path hid)`
185−is the `|=` line right after it.
193+and the function is the `|=` line right after it.
186
194
187−`++prey` is a standard library function defined in `zuse`. It takes a path and a
195+`++prey` is a standard library function defined in `zuse`. It takes a
188−bowl and gives you a list of the subscribers who are subscribed on a path that
196+path and a bowl and gives you a list of the subscribers who are
189−begins with the given path. "Prey" is short for "prefix".
197+subscribed on a path that begins with the given path. "Prey" is short
198+for "prefix".
190
199
191Now we have the list of relevant subscribers. This a list of triples,
200Now we have the list of relevant subscribers. This a list of triples,
192−`{bone @p path}`, where the only thing we really need is the bone, because we
201+`{bone @p path}`, where the only thing we really need is the bone,
193−don't need to know their urbit or what exact path they subscribed on. Thus, our
202+because we don't need to know their urbit or what exact path they
194−transformer function takes `{o/bone *}` and produces `[o %diff %noun arg]`,
203+subscribed on. Thus, our transformer function takes `{o/bone *}` and
195−which is a move that provides bone `o` with this subscription update:
204+produces `[o %diff %noun arg]`, which is a move that provides bone `o`
196−`[%noun arg]`". This is fairly dense code, but what it's doing is
205+with this subscription update: `[%noun arg]`". This is fairly dense
197−straightforward.
206+code, but what it's doing is straightforward.
198
207
199### :examples-sink
208### :examples-sink
200
209
201−`:examples-source` should now make sense. `:examples-sink` is a little longer,
210+`:examples-source` should now make sense. `:examples-sink` is a little
202−but not much more complicated.
211+longer, but not much more complicated.
203−
212+
204−In `:examples-sink`, our definition of of `++move` is different. All moves start
213+In `:examples-sink`, our definition of of `++move` is different. All
205−with a `bone`, and we conventionally refer to the second half as the "card", so
214+moves start with a `bone`, and we conventionally refer to the second
206−that we can say a move is an action that sends a card along a bone.
215+half as the "card", so that we can say a move is an action that sends a
207−
216+card along a bone.
208−We have two kinds of cards here: we `%peer` to start a subscription, and we
217+
209−`%pull` to stop it. Both of these are "forward" moves that may receive a
218+We have two kinds of cards here: we `%peer` to start a subscription, and
210−response, so they need a wire to tack onto the duct before they pass it on. They
219+we `%pull` to stop it. Both of these are "forward" moves that may
211−also need a target, which is a pair of an urbit and an app name. Additionally,
220+receive a response, so they need a wire to tack onto the duct before
212−`%peer` needs a path on that app to subscribe too. `%pull` doesn't need this,
221+they pass it on. They also need a target, which is a pair of an urbit
213−because its semantics are to cancel any subscriptions coming over this duct. If
222+and an app name. Additionally, `%peer` needs a path on that app to
214−your bone and wire are the same as when you subscribed, then the cancellation
223+subscribe too. `%pull` doesn't need this, because its semantics are to
215−will happen correctly.
224+cancel any subscriptions coming over this duct. If your bone and wire
216−
225+are the same as when you subscribed, then the cancellation will happen
217−The only state we need for `:examples-sink` is a boolean to indicate whether
226+correctly.
218−we're already subscribed to `:examples-source`. We use `available/?`, where `?`
227+
219−is the sign of type boolean (similar to `*`, `@`), which defaults to true (that
228+The only state we need for `:examples-sink` is a boolean to indicate
220−is, `0`).
229+whether we're already subscribed to `:examples-source`. We use
230+`available/?`, where `?` is the sign of type boolean (similar to `*`,
231+`@`), which defaults to true (that is, `0`).
221
232
222In `++poke-noun` we check our input to see both if it's `%on` and we're
233In `++poke-noun` we check our input to see both if it's `%on` and we're
223−available. If so, we produce the move to subscribe to `:examples-source`:
234+available. If so, we produce the move to subscribe to
235+`:examples-source`:
224
236
225 [ost %peer /subscribe [our %source] /example-path]
237 [ost %peer /subscribe [our %source] /example-path]
226
238
227Also, we set available to false (`|`) with `+>.$(available |)`.
239Also, we set available to false (`|`) with `+>.$(available |)`.
228
240
229−Otherwise, if our input is `%off` and we're already subscribed (i.e. `available`
241+Otherwise, if our input is `%off` and we're already subscribed (i.e.
230−is false), then we unsubscribe from `:examples-source`:
242+`available` is false), then we unsubscribe from `:examples-source`:
231
243
232 [ost %pull /subscribe [our %source] ~]
244 [ost %pull /subscribe [our %source] ~]
233
245
234−It's important to send over the same bone and wire (`/subscribe`) as the one we
246+It's important to send over the same bone and wire (`/subscribe`) as the
235−originally subscribed on.
247+one we originally subscribed on.
236
248
237We also set `available` to true (`&`).
249We also set `available` to true (`&`).
238
250
239−If neither of these cases are true, then we print our current subscription
251+If neither of these cases are true, then we print our current
240−state.
252+subscription state.
241
253
242−`++diff-noun` is called when we get a `%diff` update along a subscription with a
254+`++diff-noun` is called when we get a `%diff` update along a
243−mark of `noun`. `++diff-noun` is given the wire that we originally passed with
255+subscription with a mark of `noun`. `++diff-noun` is given the wire that
244−the `%peer` subscription request along and the data we got back. In our case we
256+we originally passed with the `%peer` subscription request along and the
245−just print out the data.
257+data we got back. In our case we just print out the data.
246
258
247`++reap` is called when we receive an acknowledgment as to whether the
259`++reap` is called when we receive an acknowledgment as to whether the
248−subscription was handled successfully. You can remember that `++reap` is the
260+subscription was handled successfully. You can remember that `++reap` is
249−counterpart to `++peer` as it's pronounced like 'peer' backwards. Similarly,
261+the counterpart to `++peer` as it's pronounced like 'peer' backwards.
250−`coup` is similar to 'poke' backwards.
262+Similarly, `coup` is similar to 'poke' backwards.
251−
263+
252−Moving forward, `++reap` is given the wire we attempted to subscribe over,
264+Moving forward, `++reap` is given the wire we attempted to subscribe
253−possibly along with an error message in cases of failure. `(unit type)` means
265+over, possibly along with an error message in cases of failure.
254−"either `~` or `[~ type]`, which means it's used like Haskell's "maybe" or C's
266+`(unit type)` means "either `~` or `[~ type]`, which means it's used
255−nullability. If `error` is `~`, then the subscription was successful and we tell
267+like Haskell's "maybe" or C's nullability. If `error` is `~`, then the
256−that to the user. Otherwise, we print out the error message.
268+subscription was successful and we tell that to the user. Otherwise, we
269+print out the error message.
docs/arvo/web-apps.md+229−212
@@ -1,248 +1,265 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
3+next: true
4sort: 17
4sort: 17
5title: Web apps
5title: Web apps
6---
6---
7
7
8# Backend
8# Backend
9
9
10−We've done purely functional web pages as hook files, and we've done command
10+We've done purely functional web pages as hook files, and we've
11−line apps, so now let's interact with an app over the web. The fundamental
11+done command line apps, so now let's interact with an app over
12−concepts are no different than from the command line -- you send "poke" messages
12+the web. The fundamental concepts are no different than from the
13−to apps and subscribe to data streams. The only difference is that you do it
13+command line -- you send "poke" messages to apps and subscribe to
14−with javascript.
14+data streams. The only difference is that you do it with
15+javascript.
15
16
16−Let's create a small web app that counts the number of times you poke it.
17+Let's create a small web app that counts the number of times you
17−There'll be a button to poke the app, and line of text below it saying how many
18+poke it. There'll be a button to poke the app, and line of text
18−times the app has been poked. We can update this by subscribing our page to the
19+below it saying how many times the app has been poked. We can
19−relevant path on its app.
20+update this by subscribing our page to the relevant path on its app.
20
21
21Let's first checkout the app `/app/examples/click.hoon`:
22Let's first checkout the app `/app/examples/click.hoon`:
22
23
23− /? 314
24+```
24− !:
25+/? 314
25− |%
26+!:
26− ++ move {bone $diff $mark *}
27+|%
27− --
28+++ move {bone $diff $mark *}
28− !:
29+--
29− |_ {hid/bowl clicks/@}
30+!:
30− ++ poke-examples-click-click
31+|_ {hid/bowl clicks/@}
31− |= click/$click
32+++ poke-examples-click-click
32− ~& [%poked +(clicks)]
33+ |= click/$click
33− :_ +>.$(clicks +(clicks))
34+ ~& [%poked +(clicks)]
34− %+ turn (prey /the-path hid)
35+ :_ +>.$(clicks +(clicks))
35− |=({o/bone *} [o %diff %examples-click-clicks +(clicks)])
36+ %+ turn (prey /the-path hid)
36− ++ peer-the-path
37+ |=({o/bone *} [o %diff %examples-click-clicks +(clicks)])
37− |= pax/path
38+++ peer-the-path
38− [[[ost.hid %diff %examples-click-clicks clicks] ~] +>.$]
39+ |= pax/path
39− --
40+ [[[ost.hid %diff %examples-click-clicks clicks] ~] +>.$]
40−
41+--
41−There's nothing really new here, except that we use a couple of new marks,
42+```
42−`click` and `clicks`. When we get poked with a `click`, we increment the
43+
43−variable `clicks` in our state and tell all our subcribers the new value. When
44+There's nothing really new here, except that we use a couple of
44−someone subscribes to us on the path `/the-path`, we immediately give them the
45+new marks, `click` and `clicks`. When we get poked with a
45−current number of clicks.
46+`click`, we increment the variable `clicks` in our state and tell
46−
47+all our subcribers the new value. When someone subscribes to us
47−Let's take a look at the new marks. Here's `/mar/examples/click/click.hoon`:
48+on the path `/the-path`, we immediately give them the current
48−
49+number of clicks.
49− |_ click/$click
50+
50− ++ grab
51+Let's take a look at the new marks. Here's `/mar/examples/click/click.hoon`:
51− |%
52+
52− ++ noun |=(* %click)
53+```
53− ++ json
54+|_ click/$click
54− |= jon/^json
55+++ grab
55− ?> =('click' (need (so:jo jon)))
56+ |%
56− %click
57+ ++ noun |=(* %click)
57− --
58+ ++ json
58− --
59+ |= jon/^json
60+ ?> =('click' (need (so:jo jon)))
61+ %click
62+ --
63+--
64+```
59
65
60The mark `examples-click-click` has hoon type `%click`, which means the only
66The mark `examples-click-click` has hoon type `%click`, which means the only
61−valid value is `%click`. We can convert from `noun` by just producing `%click`.
67+valid value is `%click`. We can convert from `noun` by just
62−
68+producing `%click`.
63−We also convert from json by parsing a json string with `so:jo`, asserting the
69+
64−parsing succeeded with `need`, and asserting the result was 'click' with `?>`,
70+We also convert from json by parsing a json string with `so:jo`,
65−which asserts that its first child is true.
71+asserting the parsing succeeded with `need`, and asserting the
66−
72+result was 'click' with `?>`, which asserts that its first child
67−> Note the argument to `++json`is `jon/^json`. Why `^json`? `++json` shadows the
73+is true.
68−> type definition, so if we want to refer to the type, we have to prepend a `^`.
74+
69−> This extends to multiple levels: `^^^foo` means the fourth most innermost
75+> Note the argument to `++json`is `jon/^json`. Why `^json`?
70−> instance of `foo`.
76+> `++json` shadows the type definition, so if we want to refer to
71−
77+> the type, we have to prepend a `^`. This extends to multiple
72−> `++jo` in `zuse` is a useful library for parsing complex json into hoon
78+> levels: `^^^foo` means the fourth most innermost instance of `foo`.
73−> structures. In this case, the `:` between `so` and `jo` means 'inside of',
79+
74−> because `so` is an arm contained within the core `jo`. Our case is actually
80+> `++jo` in `zuse` is a useful library for parsing complex json
75−> simple enough that the `?>` line could have been `?> =([%s 'click'] jon)`.
81+> into hoon structures. In this case, the `:` between `so` and
76−
82+> `jo` means 'inside of', because `so` is an arm contained within
77−We can test this mark from the command line (don't forget to start your app with
83+> the core `jo`. Our case is actually simple enough that the
78−`|start %examples-click`)
84+> `?>` line could have been `?> =([%s 'click'] jon)`.
79−
85+
80− ~fintud-macrep:dojo> &examples-click-click %click
86+We can test this mark from the command line (don't forget to start your app with `|start %examples-click`)
81− %click
87+
82− ~fintud-macrep:dojo> &examples-click-click &json [%s 'click']
88+```
83− %click
89+~fintud-macrep:dojo> &examples-click-click %click
84− ~fintud-macrep:dojo> &examples-click-click &json &mime [/text/json (taco '"click"')]
90+%click
85− %click
91+~fintud-macrep:dojo> &examples-click-click &json [%s 'click']
86− ~fintud-macrep:dojo> &examples-click-click &json &mime [/text/json (taco '"clickety"')]
92+%click
87− /~fintud-macrep/home/0/mar/click:<[7 5].[8 11]>
93+~fintud-macrep:dojo> &examples-click-click &json &mime [/text/json (taco '"click"')]
88− ford: casting %json to %click
94+%click
89− ford: cast %click
95+~fintud-macrep:dojo> &examples-click-click &json &mime [/text/json (taco '"clickety"')]
96+/~fintud-macrep/home/0/mar/click:<[7 5].[8 11]>
97+ford: casting %json to %click
98+ford: cast %click
99+```
90
100
91And `/mar/examples/click/clicks.hoon`:
101And `/mar/examples/click/clicks.hoon`:
92
102
93− |_ clicks/@
103+```
94− ++ grow
104+|_ clicks/@
95− |%
105+++ grow
96− ++ json
106+ |%
97− (joba %clicks (jone clicks))
107+ ++ json
98− --
108+ (joba %clicks (jone clicks))
99− --
109+ --
100−
110+--
101−`examples-clicks-clicks` is just an atom. We convert to json by creating an
111+```
112+
113+`examples-clicks-clicks` is just an atom. We convert to json by creating an
102object with a single field "clicks" with our value.
114object with a single field "clicks" with our value.
103
115
104−> Be sure to checkout section 3bD, JSON and XML, in zuse.hoon `++joba` is just a
116+> Be sure to checkout section 3bD, JSON and XML, in zuse.hoon
105−> function that takes a key-value pair and produces a JSON object with one
117+> `++joba` is just a function that takes a key-value pair and
106−> element.
118+> produces a JSON object with one element.
107
119
108− ~fintud-macrep:dojo> &json &examples-click-clicks 6
120+```
109− [%o p={[p='clicks' q=[%n p=~.6]]}]
121+~fintud-macrep:dojo> &json &examples-click-clicks 6
122+[%o p={[p='clicks' q=[%n p=~.6]]}]
123+```
110
124
111−Now that we know the marks involved, take another look at the app above.
125+Now that we know the marks involved, take another look at the app
112−Everything should be pretty straightforward. Let's poke the app from the command
126+above. Everything should be pretty straightforward. Let's poke
113−line.
127+the app from the command line.
114
128
115− ~fintud-macrep:dojo> |start %examples-click
129+```
116− >=
130+~fintud-macrep:dojo> |start %examples-click
117− ~fintud-macrep:dojo> :examples-click &examples-click-click %click
131+>=
118− [%poked 1]
132+~fintud-macrep:dojo> :examples-click &examples-click-click %click
119− >=
133+[%poked 1]
120− ~fintud-macrep:dojo> :examples-click &examples-click-click %click
134+>=
121− [%poked 2]
135+~fintud-macrep:dojo> :examples-click &examples-click-click %click
122− >=
136+[%poked 2]
137+>=
138+```
123
139
124−Please pardon the awkwardness of the name 'examples-click-click', but we used it
140+Please pardon the awkwardness of the name 'examples-click-click', but we used it to stay true to our examples naming convention.
125−to stay true to our examples naming convention.
126
141
127**Exercise**:
142**Exercise**:
128
143
129−- Modify `:sink` from the subcriptions chapter to listen to `:click` and print
144+- Modify `:sink` from the subcriptions chapter to listen to
130− out the subscription updates on the command line.
145+ `:click` and print out the subscription updates on the command
146+ line.
131
147
132# Frontend
148# Frontend
133
149
134−That's all that's needed for the back end. The front end is just some "sail"
150+That's all that's needed for the back end. The front end is just
135−html (Hoon markup for XML) and javascript. Here's
151+some "sail" html (Hoon markup for XML) and javascript. Here's `/web/pages/examples/click.hoon`:
136−`/web/pages/examples/click.hoon`:
152+
137−
153+```
138− ;html
154+;html
139− ;head
155+ ;head
140− ;script(type "text/javascript", src "//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.min.js");
156+ ;script(type "text/javascript", src "//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.min.js");
141− ;script(type "text/javascript", src "/~/at/lib/js/urb.js");
157+ ;script(type "text/javascript", src "/~/at/lib/js/urb.js");
142− ;title: Clickety!
158+ ;title: Clickety!
143− ==
159+ ==
144− ;body
160+ ;body
145− ;div#cont
161+ ;div#cont
146− ;input#go(type "button", value "Poke!");
162+ ;input#go(type "button", value "Poke!");
147− ;div#err(class "disabled");
163+ ;div#err(class "disabled");
148− ;div#clicks;
164+ ;div#clicks;
149− ==
150− ;script(type "text/javascript", src "/pages/examples/click/click.js");
151− ==
152 ==
165 ==
153−
166+ ;script(type "text/javascript", src "/pages/examples/click/click.js");
154−You should recognize the sail syntax from an earlier chapter. Aside from jquery,
167+ ==
155−we also include `urb.js`, which is a framework for interacting with urbit.
168+==
156−
169+```
157−To view the frontend, point your browser at
170+
158−`ship-name.urbit.org/~~/pages/examples/click` if you're on the live network. If
171+You should recognize the sail syntax from an earlier chapter.
159−you're running a fake galaxy, navigate to whatever port it's running on, which
172+Aside from jquery, we also include `urb.js`, which is a framework
160−is usually: `https://localhost:8443/~~/pages/examples/click`.
173+for interacting with urbit.
161−
174+
162−We have a button labeled "Poke!" and a div with id `clicks` where we'll put the
175+To view the frontend, point your browser at `ship-name.urbit.org/~~/pages/examples/click` if you're on the live network. If you're running a fake galaxy, navigate to whatever port it's running on, which is usually: `https://localhost:8443/~~/pages/examples/click`.
163−number of clicks. We also include a small javascript file where the client-side
176+
164−application logic can be found. It's in `/web/pages/examples/click/click.js`:
177+We have a button labeled "Poke!" and a div with id `clicks` where
165−
178+we'll put the number of clicks. We also include a small
166−
179+javascript file where the client-side application logic can be
167− $(function() {
180+found. It's in `/web/pages/examples/click/click.js`:
168− var clicks, $go, $clicks, $err
181+
169−
182+```
170− $go = $('#go')
183+
171− $clicks = $('#clicks')
184+$(function() {
172− $err = $('#err')
185+ var clicks, $go, $clicks, $err
173−
186+
174− $go.on("click",
187+ $go = $('#go')
175− function() {
188+ $clicks = $('#clicks')
176− window.urb.send(
189+ $err = $('#err')
177− "click", {mark: "examples-click-click"}
190+
178− ,function(err,res) {
191+ $go.on("click",
179− if(err)
192+ function() {
180− return $err.text("There was an error. Sorry!")
193+ window.urb.send(
181− if(res.data !== undefined &&
194+ "click", {mark: "examples-click-click"}
182− res.data.ok !== undefined &&
195+ ,function(err,res) {
183− res.data.ok !== true)
196+ if(err)
184− $err.text(res.data.res)
197+ return $err.text("There was an error. Sorry!")
185− else
198+ if(res.data !== undefined &&
186− console.log("poke succeeded");
199+ res.data.ok !== undefined &&
187− })
200+ res.data.ok !== true)
201+ $err.text(res.data.res)
202+ else
203+ console.log("poke succeeded");
188 })
204 })
189−
205+ })
190− window.urb.appl = "examples-click"
206+
191− window.urb.bind('/the-path',
207+ window.urb.appl = "examples-click"
192− function(err,dat) {
208+ window.urb.bind('/the-path',
193− clicks = dat.data.clicks
209+ function(err,dat) {
194− $clicks.text(clicks)
210+ clicks = dat.data.clicks
195− }
211+ $clicks.text(clicks)
196− )
212+ }
197− })
213+ )
198−
214+})
199−We set up two event handlers. When we click the button, we run
215+```
200−`window.urb.send`, which sends a poke to the app specified as the first
216+
201−argument, in this case our :click app. The arguments are data, parameters, and
217+We set up two event handlers. When we click the button, we run
202−the callback. The data is the specific data we want to send. The parameters are
218+`window.urb.send`, which sends a poke to the app specified as the first argument, in this case our :click app. The arguments
203−all optional, but here's a list of the available ones:
219+are data, parameters, and the callback. The data is the specific
204−
220+data we want to send. The parameters are all optional, but here's a list
205−- `ship`: target urbit. Defaults to `window.urb.ship`, which defaults to the
221+of the available ones:
206− urbit which served the page.
222+
207−- `appl`: target app. Defaults to `window.urb.appl`, which defaults to null.
223+- `ship`: target urbit. Defaults to `window.urb.ship`, which
208−- `mark`: mark of the data. Defaults to `window.urb.send.mark`, which defaults
224+ defaults to the urbit which served the page.
209− to "json".
225+- `appl`: target app. Defaults to `window.urb.appl`, which
210−- `wire`: wire of the poke. Defaults to `/`.
226+ defaults to null.
227+- `mark`: mark of the data. Defaults to `window.urb.send.mark`,
228+ which defaults to "json".
229+- `wire`: wire of the poke. Defaults to `/`.
211
230
212In our case, we specify only that the mark is "click".
231In our case, we specify only that the mark is "click".
213
232
214−The callback funciton is called when we receive an acknowledgment. If there was
233+The callback funciton is called when we receive an
215−an error, we put it in the `err` div that we defined above. Otherwise, we printf
234+acknowledgment. If there was an error, we put it in the `err`
216−to the console a message saying the poke succeeded. As is common, we gray out
235+div that we defined above. Otherwise, we printf to the console a
217−the button when it is clicked. It is only reenabled when positive
236+message saying the poke succeeded. As is common, we gray
218−acknowledgement is received.
237+out the button when it is clicked. It is only reenabled when positive acknowledgement is received.
219−
238+
220−Our second event handler is `window.urb.bind`, which is called immediately when
239+Our second event handler is `window.urb.bind`, which is called immediately when the page is loaded, subscribing the page to the specified data stream on the app set with `urb.appl` (which is, in this case, :click). It also takes three arguments: path, parameters, and callback. The path is the path on the app to subscribe to. The parameters are all optional (indeed, we omit them entirely here), but are similar to those for `send`:
221−the page is loaded, subscribing the page to the specified data stream on the app
240+
222−set with `urb.appl` (which is, in this case, :click). It also takes three
241+- `ship`: target urbit. Defaults to `window.urb.ship`, which
223−arguments: path, parameters, and callback. The path is the path on the app to
242+ defaults to the urbit which served the page.
224−subscribe to. The parameters are all optional (indeed, we omit them entirely
243+- `appl`: target app. Defaults to `window.urb.appl`, which
225−here), but are similar to those for `send`:
244+ defaults to null.
226−
245+- `mark`: mark of expected data. Data of other marks is
227−- `ship`: target urbit. Defaults to `window.urb.ship`, which defaults to the
246+ converted to this mark on the server before coming to the web
228− urbit which served the page.
247+ front end. Defaults to `window.urb.bind.mark`, which defaults
229−- `appl`: target app. Defaults to `window.urb.appl`, which defaults to null.
248+ to "json".
230−- `mark`: mark of expected data. Data of other marks is converted to this mark
249+- `wire`: wire of the subscription. Defaults to the given path.
231− on the server before coming to the web front end. Defaults to
250+
232− `window.urb.bind.mark`, which defaults to "json".
251+The callback function here just updates the data in the `clicks`
233−- `wire`: wire of the subscription. Defaults to the given path.
252+div. A truly robust app would intelligently handle errors, of
234−
253+course.
235−The callback function here just updates the data in the `clicks` div. A truly
254+
236−robust app would intelligently handle errors, of course.
255+Note that the app doesn't have anything web-specific in itself.
237−
256+As far as it knows, it's just receiving pokes and subscriptions.
238−Note that the app doesn't have anything web-specific in itself. As far as it
257+The javascript is fairly pure as well, sending and receiving json
239−knows, it's just receiving pokes and subscriptions. The javascript is fairly
258+everywhere. The marks are the translation layer, and they're the
240−pure as well, sending and receiving json everywhere. The marks are the
259+only things that need to know how the hoon types map to json.
241−translation layer, and they're the only things that need to know how the hoon
242−types map to json.
243
260
244**Exercise**:
261**Exercise**:
245
262
246−- Open the app in multiple tabs, click the button, and verify that all the
263+- Open the app in multiple tabs, click the button, and verify
247− tabs stay in sink. Poke it manually from the command line and verify the
264+ that all the tabs stay in sink. Poke it manually from the
248− tabs are updated as well.
265+ command line and verify the tabs are updated as well.
docs/byte.md+2−3
@@ -1,11 +1,10 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 2
3sort: 2
4---
4---
5
5
6# Learn
6# Learn
7
7
8−Learn Hoon in short, self-contained, bottom-up tutorials through our in-house
8+Learn Hoon in short, self-contained, bottom-up tutorials through our in-house Urbyte series. A work in progress. Still ideal for the complete newbie.
9−Urbyte series. A work in progress. Still ideal for the complete newbie.
10
9
11<list/>
10<list/>
docs/byte/0.md+199−157
@@ -1,7 +1,7 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
4sort: 1
3sort: 1
4+next: true
5title: Setup and nouns
5title: Setup and nouns
6---
6---
7
7
@@ -9,234 +9,276 @@ title: Setup and nouns
9
9
10This is the first post in our new Urbyte series.
10This is the first post in our new Urbyte series.
11
11
12−The Urbyte concept is that you, an ordinary programmer, not a kernel hacker or a
12+The Urbyte concept is that you, an ordinary programmer, not a
13−Haskell magician (not that there's anything wrong with kernel hackers or Haskell
13+kernel hacker or a Haskell magician (not that there's anything
14−magicians) can learn Hoon, Urbit's programming language.
14+wrong with kernel hackers or Haskell magicians) can learn Hoon,
15+Urbit's programming language.
15
16
16−We're going to test this daring idea by teaching you Hoon, in easily digestible,
17+We're going to test this daring idea by teaching you Hoon, in
17−byte-size chunks. Each post in the "Urbytes" series is designed to take about 15
18+easily digestible, byte-size chunks. Each post in the "Urbytes"
18−minutes to consume. (Maybe 10, if you're a kernel hacker or a Haskell magician.)
19+series is designed to take about 15 minutes to consume. (Maybe
20+10, if you're a kernel hacker or a Haskell magician.)
19
21
20−We'll assume no knowledge of Urbit except for previous lessons. Ideally you have
22+We'll assume no knowledge of Urbit except for previous lessons.
21−some basic familiarity with programming and can operate the Unix command line.
23+Ideally you have some basic familiarity with programming and can
24+operate the Unix command line.
22
25
23−We strongly encourage you to type in all the examples in each Urbyte. Urbytes
26+We strongly encourage you to type in all the examples in each
24−are short, but not meant to be skimmed.
27+Urbyte. Urbytes are short, but not meant to be skimmed.
25
28
26## Install
29## Install
27
30
28−First, [install Urbit](http://urbit.org/docs/using/install/) on any Mac or Unix
31+First, [install Urbit](http://urbit.org/docs/using/install/) on any
29−machine. On Windows, make a virtual Linux machine using VirtualBox or a similar
32+Mac or Unix machine. On Windows, make a virtual Linux machine
30−tool.
33+using VirtualBox or a similar tool.
31
34
32While you're downloading, read about Urbit ships:
35While you're downloading, read about Urbit ships:
33
36
34## Ships
37## Ships
35
38
36−In the Urbyte exercises you'll be running what we call a "comet". A string
39+In the Urbyte exercises you'll be running what we call a "comet". A string starting with `~`, like `~zod`, is what Urbit
37−starting with `~`, like `~zod`, is what Urbit calls a *ship*.
40+calls a *ship*.
38
41
39−A *ship* is one number which means four things: a network address, a
42+A *ship* is one number which means four things: a network address,
40−cryptographic identity, a memorable name, and a single virtual computer.
43+a cryptographic identity, a memorable name, and a single virtual
44+computer.
41
45
42−We write this number in a base-256 format, where each digit is a syllable.
46+We write this number in a base-256 format, where each digit is a
43−Imagine if your phone number was a pronounceable string which sounded like a
47+syllable. Imagine if your phone number was a pronounceable
44−name in a foreign language. `~zod` is ship zero (making it a very important ship
48+string which sounded like a name in a foreign language. `~zod`
45−indeed!). An ordinary user-level ship is a "planet", a 32-bit number which
49+is ship zero (making it a very important ship indeed!). An ordinary
46−becomes a four-syllable string, like `~talsur-todres`.
50+user-level ship is a "planet", a 32-bit number which becomes
51+a four-syllable string, like `~talsur-todres`.
47
52
48−*Comets*, at 128-bits or 16 syllables, are disposable identities that anyone can
53+*Comets*, at 128-bits or 16 syllables, are
49−make for free to join the live network. Thus, comets make the ideal ship for
54+disposable identities that anyone can make for free to
50−playing around with Hoon, asking for help from other Urbit devs in
55+join the live network. Thus, comets make the ideal ship for
51−[`:talk`](http://urbit.org/docs/using/messaging/) if you need it, and showing
56+playing around with Hoon, asking for help from other Urbit devs in [`:talk`](http://urbit.org/docs/using/messaging/) if you
52−off Urbit to your hacker friends.
57+need it, and showing off Urbit to your hacker friends.
53
58
54## Create and restart
59## Create and restart
55
60
56To create your comet, run:
61To create your comet, run:
57
62
58− $ cd ~
63+```
59− $ mkdir urbit
64+$ cd ~
60− $ cd urbit
65+$ mkdir urbit
61− $ urbit -c mycomet
66+$ cd urbit
62−
67+$ urbit -c mycomet
63−This will take a few minutes and will spin out a bunch of boot messages. At the
68+```
64−end, you'll see something like:
69+
65−
70+This will take a few minutes and will spin out a bunch of boot
66− ames: on localhost, UDP 31337.
71+messages. At the end, you'll see something like:
67− http: live (insecure, public) on 8080
72+
68− http: live ("secure", public) on 8443
73+```
69− http: live (insecure, loopback) on 12321
74+ames: on localhost, UDP 31337.
70− ~palnul_nocser:dojo>
75+http: live (insecure, public) on 8080
71−
76+http: live ("secure", public) on 8443
72−To make sure everything is working, type `(add 2 2)` at the prompt, then hit
77+http: live (insecure, loopback) on 12321
73−return. Your screen now shows:
78+~palnul_nocser:dojo>
74−
79+```
75− ames: on localhost, UDP 31337.
80+
76− http: live (insecure, public) on 8080
81+To make sure everything is working, type `(add 2 2)` at the
77− http: live ("secure", public) on 8443
82+prompt, then hit return. Your screen now shows:
78− http: live (insecure, loopback) on 12321
83+
79− > (add 2 2)
84+```
80− 4
85+ames: on localhost, UDP 31337.
81−
86+http: live (insecure, public) on 8080
82−~palnul\_nocser:dojo&gt;
87+http: live ("secure", public) on 8443
83−
88+http: live (insecure, loopback) on 12321
84−That's super awesome! You made your first noun! We'll make a few more in a
89+> (add 2 2)
85−second. First, quit Urbit with `ctrl-d`:
90+4
86−
91+```
87− > (add 2 2)
92+~palnul_nocser:dojo>
88− 4
93+
89− ~palnul_nocser:dojo>
94+That's super awesome! You made your first noun! We'll make a
90− $
95+few more in a second. First, quit Urbit with `ctrl-d`:
91−
96+
92−You now have an Urbit image, or *pier*, in the `mycomet` directory. Right now
97+```
93−the only thing in your pier is Urbit's system files:
98+> (add 2 2)
94−
99+4
95− $ ls -a /path/to/mycomet
100+~palnul_nocser:dojo>
96− ./ ../ .urb/
101+$
97−
102+```
98−Restarting uses almost the same command, but without `-c` (which means
103+
99−"create"). There are also fewer startup messages:
104+You now have an Urbit image, or *pier*, in the `mycomet` directory.
100−
105+Right now the only thing in your pier is Urbit's system files:
101− $ urbit mycomet
106+
102− [...]
107+```
103− ames: on localhost, UDP 31337.
108+$ ls -a /path/to/mycomet
104− http: live (insecure, public) on 8080
109+./ ../ .urb/
105− http: live ("secure", public) on 8443
110+```
106− http: live (insecure, loopback) on 12321
111+
107− ~palnul_nocser:dojo>
112+Restarting uses almost the same command, but without `-c` (which
108−
113+means "create"). There are also fewer startup messages:
109−Note also that Urbit is an ACID database. You can stop it politely with
114+
110−`ctrl-d`, abruptly with `ctrl-z`, even violently with `kill -9`. None of these
115+```
111−should cause Urbit to lose data. If you do manage to corrupt a pier, it's a bug;
116+$ urbit mycomet
112−please let us know.
117+[...]
118+ames: on localhost, UDP 31337.
119+http: live (insecure, public) on 8080
120+http: live ("secure", public) on 8443
121+http: live (insecure, loopback) on 12321
122+~palnul_nocser:dojo>
123+```
124+
125+Note also that Urbit is an ACID database. You can stop it
126+politely with `ctrl-d`, abruptly with `ctrl-z`, even violently with `kill
127+-9`. None of these should cause Urbit to lose data. If you do
128+manage to corrupt a pier, it's a bug; please let us know.
113
129
114## Let's make a noun
130## Let's make a noun
115
131
116−You've already made a noun. Let's make another!
132+You've already made a noun. Let's make another!
133+
134+*We won't show the `~palnul_nocser:dojo> ` prompt from here on out.
135+We'll just show the echoed command.*
117
136
118−*We won't show the `~palnul_nocser:dojo>` prompt from here on out. We'll just
119−show the echoed command.*
120
137
121− ~palnul_nocser:dojo> 42
138+```
122− 42
139+~palnul_nocser:dojo> 42
140+42
141+```
123
142
124You'll see:
143You'll see:
125
144
126− > 42
145+```
127− 42
146+> 42
147+42
148+```
128
149
129## Understanding nouns
150## Understanding nouns
130
151
131−A value in Urbit is called a *noun*. A noun is either an *atom* (an unsigned
152+A value in Urbit is called a *noun*. A noun is either an *atom*
132−integer of any size) or a *cell* (an ordered pair of any two nouns).
153+(an unsigned integer of any size) or a *cell* (an ordered pair of
154+any two nouns).
133
155
134−In a sentence, a noun is a binary tree whose leaves are numbers. If you know
156+In a sentence, a noun is a binary tree whose leaves are numbers.
135−Lisp, a noun is a minimalist S-expression. A noun can't have cycles (pointer
157+If you know Lisp, a noun is a minimalist S-expression. A noun
136−loops). An implementation can use dags (shared subtrees), but the programmer
158+can't have cycles (pointer loops). An implementation can use
137−can't detect them (there's no pointer equality test).
159+dags (shared subtrees), but the programmer can't detect them
160+(there's no pointer equality test).
138
161
139−The absence of graph cycles makes nouns much easier to save in persistent
162+The absence of graph cycles makes nouns much easier to save in
140−storage, send over the network, manage memory for (Urbit has no tracing garbage
163+persistent storage, send over the network, manage memory for
141−collector), etc.
164+(Urbit has no tracing garbage collector), etc.
142
165
143## Let's make some atoms
166## Let's make some atoms
144
167
145−So one kind of atom is a number. And the syntax (or at least one syntax) for a
168+So one kind of atom is a number. And the syntax (or at least one
146−number is a decimal, like `42`. Seems legit.
169+syntax) for a number is a decimal, like `42`. Seems legit.
147
170
148−But just to be sure we understand this decimal thing, let's make some atoms:
171+But just to be sure we understand this decimal thing, let's make
172+some atoms:
149
173
150− > 4
174+```
151− 4
175+> 4
176+4
177+```
152
178
153Seems pretty straightforward.
179Seems pretty straightforward.
154
180
155− > 42
181+```
156− 42
182+> 42
183+42
184+```
157
185
158Why not try it again to make sure?
186Why not try it again to make sure?
159
187
160− > 420
188+```
161− 420
189+> 420
190+420
191+```
162
192
163Can we add another zero?
193Can we add another zero?
164
194
165− > 420
195+```
196+> 420
197+```
166
198
167−Wat? When you tried to type that next zero and turn `420` into `4200`, Urbit
199+Wat? When you tried to type that next zero and turn `420` into
168−actually *deleted the `0` and beeped at you.*
200+`4200`, Urbit actually *deleted the `0` and beeped at you.*
169
201
170−But isn't an atom an unsigned integer of any size? It is. But the syntax is
202+But isn't an atom an unsigned integer of any size? It is.
171−actually `4.200`:
203+But the syntax is actually `4.200`:
172
204
173− > 4.200
205+```
174− 4.200
206+> 4.200
175− > (mul 42 100)
207+4.200
176− 4.200
208+> (mul 42 100)
209+4.200
210+```
177
211
178Just think of it as "4,200", written the German way with a dot.
212Just think of it as "4,200", written the German way with a dot.
179
213
180−Why? Long decimals are unreadable. Human beings know this, so we group them in
214+Why? Long decimals are unreadable. Human beings know this, so
181−threes. For some historical reason, programming languages never copied this
215+we group them in threes. For some historical reason, programming
182−marvelous usability innovation.
216+languages never copied this marvelous usability innovation.
183
217
184−English notation for decimals is more common than German notation.
218+English notation for decimals is more common than German
185−Unfortunately, dot is URL-safe and comma isn't, and it's nice to have a regular
219+notation. Unfortunately, dot is URL-safe and comma isn't, and
186−syntax for atoms that's URL-safe.
220+it's nice to have a regular syntax for atoms that's URL-safe.
187
221
188−As for why Urbit deleted your zero: it parses your command line as you type, and
222+As for why Urbit deleted your zero: it parses your command line
189−rejects any characters after the parser stops.
223+as you type, and rejects any characters after the parser stops.
190
224
191## Let's make some cells
225## Let's make some cells
192
226
193−There's not much mystery about cells. A cell has a head and a tail, and puts
227+There's not much mystery about cells. A cell has a head and a
194−brackets around them:
228+tail, and puts brackets around them:
195
229
196− > [42 420]
230+```
197− [42 420]
231+> [42 420]
232+[42 420]
233+```
198
234
199−`42` is the *head* and `420` is the *tail* of the cell. Cell syntax nests to the
235+`42` is the *head* and `420` is the *tail* of the cell. Cell
200−right:
236+syntax nests to the right:
201
237
202− > [4 [42 420]]
238+```
203− [4 42 420]
239+> [4 [42 420]]
240+[4 42 420]
204
241
205− > [4 42 420]
242+> [4 42 420]
206− [4 42 420]
243+[4 42 420]
207
244
208− > [[4 42] 420]
245+> [[4 42] 420]
209− [[4 42] 420]
246+[[4 42] 420]
247+```
210
248
211If you look at these nouns as binary trees, you see
249If you look at these nouns as binary trees, you see
212
250
213− [4 [42 420]] [[4 42] 420]
251+```
214− . .
252+ [4 [42 420]] [[4 42] 420]
215− / \ / \
253+ . .
216− 4 . . 420
254+ / \ / \
217− / \ / \
255+ 4 . . 420
218− 42 420 4 42
256+ / \ / \
257+ 42 420 4 42
258+```
219
259
220Each number is an atom; each dot is a cell.
260Each number is an atom; each dot is a cell.
221
261
222## And we're done
262## And we're done
223
263
224−And we're done! Type `|exit` to exit:
264+And we're done! Type `|exit` to exit:
225
265
226− ~palnul_nocser:dojo> |exit
266+```
227− %drum-exit
267+~palnul_nocser:dojo> |exit
228− $
268+%drum-exit
269+$
270+```
229
271
230## Questions and/or exercises
272## Questions and/or exercises
231
273
232−And we're done with our very first Urbyte! Here are some questions to think
274+And we're done with our very first Urbyte! Here are some questions to
233−about. These exercises are never mandatory, but they may be fun and/or
275+think about. These exercises are never mandatory, but they may
234−interesting. Skip them if they're obvious.
276+be fun and/or interesting. Skip them if they're obvious.
235
277
236−- What is the right way to represent common data structures as nouns?
278+* What is the right way to represent common
237−- A linked list?
279+data structures as nouns?
238−- A string? A *signed* integer?
280+* A linked list?
239−- A table or associative array?
281+* A string? A *signed* integer?
282+* A table or associative array?
240
283
241−Just try to picture these structures in your head, or draw them on a piece of
284+Just try to picture these structures in your head, or draw them on a piece of paper.
242−paper.
docs/byte/1.md+249−223
@@ -1,339 +1,365 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
4sort: 2
3sort: 2
5−title: 'Atoms, auras, types'
4+next: true
5+title: Atoms, auras, types
6---
6---
7
7
8# Urbyte 1: Atoms, auras, types
8# Urbyte 1: Atoms, auras, types
9
9
10−Thanks for reading Urbyte 0! In Urbyte 1, we'll look a little more deeply at
10+Thanks for reading Urbyte 0! In Urbyte 1, we'll look a little
11−atoms and other simple noun types.
11+more deeply at atoms and other simple noun types.
12
12
13Let's restart our engines:
13Let's restart our engines:
14
14
15− $ urbit mycomet
15+```
16− [...]
16+$ urbit mycomet
17− ames: on localhost, UDP 31337.
17+[...]
18− http: live (insecure, public) on 8080
18+ames: on localhost, UDP 31337.
19− http: live ("secure", public) on 8443
19+http: live (insecure, public) on 8080
20− http: live (insecure, loopback) on 12321
20+http: live ("secure", public) on 8443
21− ~palnul_nocser:dojo>
21+http: live (insecure, loopback) on 12321
22+~palnul_nocser:dojo>
23+```
22
24
23## What's in an atom?
25## What's in an atom?
24
26
25−It seems limiting to have only one kind of atom, the unsigned decimal. How are
27+It seems limiting to have only one kind of atom, the unsigned
26−signed integers represented? Symbols? Floating-point values? Dates? IP
28+decimal. How are signed integers represented? Symbols?
27−addresses? Even just hex?
29+Floating-point values? Dates? IP addresses? Even just hex?
28
30
29−Lisp's S-expression, with its own cells and atoms, is very similar to Hoon's
31+Lisp's S-expression, with its own cells and atoms, is very
30−nouns. But languages in the Lisp family mark each atom with a dynamic tag that
32+similar to Hoon's nouns. But languages in the Lisp family mark
31−defines the atom's meaning. These tags are effectively dynamic types.
33+each atom with a dynamic tag that defines the atom's meaning.
34+These tags are effectively dynamic types.
32
35
33−Hoon also tracks the meaning of atoms. But Hoon is statically typed, so an atom
36+Hoon also tracks the meaning of atoms. But Hoon is statically
34−description is obviously a type. An atomic type in Hoon is called an *aura*. The
37+typed, so an atom description is obviously a type. An atomic type
35−aura syntax is just an ASCII string starting with `@`, like `@ud` for unsigned
38+in Hoon is called an *aura*. The aura syntax is just an ASCII
36−decimal.
39+string starting with `@`, like `@ud` for unsigned decimal.
37
40
38−Hoon has a wide (but not extensible) variety of atom syntaxes. Each syntax for
41+Hoon has a wide (but not extensible) variety of atom syntaxes.
39−an atom produces an appropriate predefined aura. And Hoon's printer can
42+Each syntax for an atom produces an appropriate predefined aura.
40−invertibly print any aura Hoon can parse. Because atoms make great path nodes
43+And Hoon's printer can invertibly print any aura Hoon can parse.
41−and paths make great URLs, all regular atom syntaxes use only URL-safe
44+Because atoms make great path nodes and paths make great URLs,
42−characters.
45+all regular atom syntaxes use only URL-safe characters.
43
46
44## Examples
47## Examples
45
48
46−With the dojo command `? expression`, you tell the dojo to print both the
49+With the dojo command `? expression`, you tell the dojo to print
47−product of the expression, and the product's type.
50+both the product of the expression, and the product's type.
48
51
49Let's type in some aura examples:
52Let's type in some aura examples:
50
53
51### Unsigned integers
54### Unsigned integers
52
55
53− > 42
56+```
54− 42
57+> 42
58+42
55
59
56− > ? 42
60+> ? 42
57− @ud
61+ @ud
58− 42
62+42
59
63
60− > ? 0x42
64+> ? 0x42
61− @ux
65+ @ux
62− 0x42
66+0x42
63
67
64− > ? (add 0x42 42)
68+> ? (add 0x42 42)
65− @
69+ @
66− 108
70+108
67
71
68− > ? `@ud`0x42
72+> ? `@ud`0x42
69− @ud
73+ @ud
70− 66
74+66
71
75
72−So decimal is aura `@ud`, hexadecimal is `@ux`. Note the cast syntax, which lets
76+```
73−us change the type of an atom, converting hex `0x42` to a decimal `66`.
74
77
75−An aura is soft type. The programmer can always insist on how to interpret an
78+So decimal is aura `@ud`, hexadecimal is `@ux`. The cast
76−atom. Hoon doesn't have "dependent types"; it can't enforce any constraints the
79+syntax&mdash;`` `@ud` ``&mdash;lets us change the type of an
77−aura puts on the atom's value.
80+atom, converting hex `0x42` to a decimal `66`.
81+
82+An aura is soft type. The programmer can always insist on how
83+to interpret an atom. Hoon doesn't have "dependent types"; it
84+can't enforce any constraints the aura puts on the atom's value.
78
85
79### Signed integers
86### Signed integers
80
87
81− > -7
88+```
82− -7
89+> -7
90+-7
83
91
84− > ? -7
92+> ? -7
85− @sd
93+ @sd
86
94
87− > --7
95+> --7
88− --7
96+--7
89
97
90− > ? --7
98+> ? --7
91− @sd
99+ @sd
92− --7
100+--7
93
101
94− > ? (sum:si -7 --7)
102+> ? (sum:si -7 --7)
95− @s
103+ @s
96− --0
104+--0
97
105
98− > `@ud`-7
106+> `@ud`-7
99− 13
107+13
100
108
101− > `@ud`--7
109+> `@ud`--7
102− 14
110+14
111+```
103
112
104−`--7` means "positive 7." `+7` might have been better, but `+` is not URL-safe.
113+`--7` means "positive 7." `+7` might have been better, but `+`
114+is not URL-safe.
105
115
106−Hoon needs `--` to distinguish positive signed numbers because Hoon's type
116+Hoon needs `--` to distinguish positive signed numbers because
107−system doesn't feed back into its parser. The expression itself needs to control
117+Hoon's type system doesn't feed back into its parser. The
108−whether it's `@sd` or `@ud`.
118+expression itself needs to control whether it's `@sd` or `@ud`.
109
119
110−With unlimited atom width, traditional sign extension makes no sense. The sign
120+With unlimited atom width, traditional sign extension makes no
111−bit is the low bit. Even atoms are positive, odd atoms are negative.
121+sense. The sign bit is the low bit. Even atoms are positive,
122+odd atoms are negative.
112
123
113−And Hoon has no overloading or type detection; we need to use a different
124+And Hoon has no overloading or type detection; we need to use a
114−function to add signed integers. We'll always choose slightly clunkier syntax in
125+different function to add signed integers. We'll always choose
115−exchange for much simpler semantics.
126+slightly clunkier syntax in exchange for much simpler semantics.
116
127
117### Symbols
128### Symbols
118
129
119− > ? 'Ürbit'
130+```
120− @t
131+> ? 'Ürbit'
121− 'Ürbit'
132+ @t
122−
133+'Ürbit'
123− > ? `@ux`'Ürbit'
124− @ux
125− 0x7469.6272.9cc3
126
134
127− > ? ~~Ürbit
135+> ? `@ux`'Ürbit'
128− @t
136+ @ux
129− 'Ürbit'
137+0x7469.6272.9cc3
130
138
131− > ? ~.urbit
139+> ? ~.urbit
132− @ta
140+ @ta
133− ~.urbit
141+~.urbit
134
142
135− > ? `@ux`'foo'
143+> ? `@ux`'foo'
136− @ux
144+ @ux
137− 0x6f.6f66
145+0x6f.6f66
146+```
138
147
139−The `@t` aura is UTF-8 text, with the first byte low. Again, Hoon doesn't
148+The `@t` aura is UTF-8 text, with the first byte low. Again,
140−actually have the power to check statically that your UTF-8 is valid. Auras
149+Hoon doesn't actually have the power to check statically that
141−don't tell you what an atom is, just what you think it is.
150+your UTF-8 is valid. Auras don't tell you what an atom is, just
151+what you think it is.
142
152
143−The familiar single-quote syntax for `@t` is an irregular form of the more
153+The familiar single-quote syntax for `@t` is an irregular form of
144−unusual `~~` prefix. The `~.` prefix, aura `@ta`, is restricted to lowercase
154+the more unusual `~~` prefix. The `~.` prefix, aura `@ta`, is
145−ASCII (a great common denominator). Of course, `~` is URL-safe and `'` is not.
155+restricted to lowercase ASCII (a great common denominator). Of
156+course, `~` is URL-safe and `'` is not.
146
157
147### Dates, floats, IP addresses...
158### Dates, floats, IP addresses...
148
159
149− > ? now
160+```
150− @da
161+> ? now
151− ~2016.7.11..01.32.04..2c0d
162+ @da
163+~2016.7.11..01.32.04..2c0d
152
164
153− > ? ~h6
165+> ? ~h6
154− @dr
166+ @dr
155− ~h6
167+~h6
156
168
157− > `@ux`~h6
169+> `@ux`~h6
158− @ux
170+ @ux
159− 0x5460.0000.0000.0000.0000
171+0x5460.0000.0000.0000.0000
160
172
161− > ? (add ~h6 now)
173+> ? (add ~h6 now)
162− @
174+ @
163− 170.141.184.502.236.089.414.812.327.632.537.911.296
175+170.141.184.502.236.089.414.812.327.632.537.911.296
164
176
165− > `@da`(add ~h6 now)
177+> `@da`(add ~h6 now)
166− ~2016.7.11..07.32.28..842f
178+~2016.7.11..07.32.28..842f
167
179
168− > ? .3.14
180+> ? .3.14
169− @rs
181+ @rs
170− .3.14
182+.3.14
171
183
172− > ? .127.0.0.1
184+> ? .127.0.0.1
173− @if
185+ @if
174− .127.0.0.1
186+.127.0.0.1
187+```
175
188
176−`@da` is a 128-bit absolute date (64 bits for fractions of a second, 64 bits for
189+`@da` is a 128-bit absolute date (64 bits for fractions of a
177−seconds); `@dr` is a relative date; `@rs` is a single-precision IEEE real; `@if`
190+second, 64 bits for seconds); `@dr` is a relative date; `@rs` is
178−is an IPv4 address. This is not all Hoon's aura syntax, just a good range of
191+a single-precision IEEE real; `@if` is an IPv4 address. This is
179−examples.
192+not all Hoon's aura syntax, just a good range of examples.
180
193
181−`(add ~h6 now)` is a good example of the use and limits of the aura system.
194+`(add ~h6 now)` is a good example of the use and limits of
182−Unlike the signed-integer sum, offsetting a date can use plain atomic addition.
195+the aura system. Unlike the signed-integer sum, offsetting a
183−But the product of `add` is a flavorless atom with no aura, `@`. So we have to
196+date can use plain atomic addition. But the product of `add` is
184−cast it back to an absolute date `@da`.
197+a flavorless atom with no aura, `@`. So we have to cast it back
198+to an absolute date `@da`.
185
199
186## Auras explained
200## Auras explained
187
201
188−Does an aura represent the units of the atom, its syntax, its semantics, its
202+Does an aura represent the units of the atom, its syntax, its
189−constraints?
203+semantics, its constraints?
190
204
191−Any or all of the above. Programmers can even use their own aura strings, which
205+Any or all of the above. Programmers can even use their own aura
192−nothing in Hoon recognizes. The set of syntaxes that Hoon knows how to parse
206+strings, which nothing in Hoon recognizes. The set of syntaxes
193−and/or print is fixed. But there are plenty of good reasons to use your own
207+that Hoon knows how to parse and/or print is fixed. But there
194−user-defined auras.
208+are plenty of good reasons to use your own user-defined auras.
195
209
196−If your program uses both fortnights and furlongs, Hoon will not parse a
210+If your program uses both fortnights and furlongs, Hoon will not
197−user-defined fortnight syntax. It will not print furlongs in the dojo. But the
211+parse a user-defined fortnight syntax. It will not print furlongs
198−type system will keep you from accidentally passing a furlong to a function that
212+in the dojo. But the type system will keep you from accidentally
199−expects a fortnight.
213+passing a furlong to a function that expects a fortnight.
200
214
201−Hoon does interpret the aura string in one way: auras specialize to the right.
215+Hoon does interpret the aura string in one way: auras specialize
202−For example, `@u` is any unsigned integer; `@ux` is an unsigned integer that
216+to the right. For example, `@u` is any unsigned integer; `@ux`
203−likes to be printed as hex.
217+is an unsigned integer that likes to be printed as hex.
204
218
205−Hoon's typechecker lets you go up or down the specialization ladder. It
219+Hoon's typechecker lets you go up or down the specialization
206−complains if you try to move across. For example, `@u` can silently convert into
220+ladder. It complains if you try to move across. For example,
207−`@ux` or `@ux` into `@u`, but turning `@ud` into `@ux`, `@ux` into `@da`, etc,
221+`@u` can silently convert into `@ux` or `@ux` into `@u`, but
208−requires an explicit cast.
222+turning `@ud` into `@ux`, `@ux` into `@da`, etc, requires an
223+explicit cast.
209
224
210## Constants, symbols, loobeans and null
225## Constants, symbols, loobeans and null
211
226
212−An atom type isn't just an aura. There's another choice: the type can be
227+An atom type isn't just an aura. There's another choice: the
213−*general* and contain all atoms (like the types we've seen above), or *constant*
228+type can be *general* and contain all atoms (like the types we've
214−and contain just one atom.
229+seen above), or *constant* and contain just one atom.
215
230
216−A type describes a set of values. While auras may informally describe a format
231+A type describes a set of values. While auras may informally
217−for which not all atoms are valid, any atom can be cast into the aura whether
232+describe a format for which not all atoms are valid, any atom can
218−valid or invalid. But in a constant, the type contains just one atom.
233+be cast into the aura whether valid or invalid. But in a
234+constant, the type contains just one atom.
219
235
220−The regular syntax for an atomic constant is just to put `%` in front of the
236+The regular syntax for an atomic constant is just to put `%` in
221−constant. Like:
237+front of the constant. Like:
222
238
223− > ? 42
239+```
224− @ud
240+> ? 42
225− 42
241+ @ud
242+42
226
243
227− > ? %42
244+> ? %42
228− $42
245+ $42
229− %42
246+%42
230
247
231− > ? %~h6
248+> ? %~h6
232− $~h6
249+ $~h6
233− %~h6
250+%~h6
251+```
234
252
235−The `$` on the type indicates the constant. Note that constants still have
253+The `$` on the type indicates the constant. Note that constants
236−auras; `~h6` (6 hours) is still a `@dr`.
254+still have auras; `~h6` (6 hours) is still a `@dr`.
237
255
238## Common irregulars
256## Common irregulars
239
257
240−There are a few special forms worth learning early: symbols, null, loobeans, and
258+There are a few special forms worth learning early: symbols,
241−of course ships:
259+null, loobeans, and of course ships:
242
260
243− > ? %foo
261+```
244− $foo
262+> ? %foo
245− %foo
263+ $foo
264+%foo
246
265
247− > `@ux`%foo
266+> `@ux`%foo
248− 0x6f.6f66
267+0x6f.6f66
249
268
250− > .n
269+> .n
251− %.n
270+%.n
252
271
253− > |
272+> |
254− %.n
273+%.n
255
274
256− > .y
275+> .y
257− %.y
276+%.y
258
277
259− > &
278+> &
260− %.y
279+%.y
261
280
262− > `@ud`&
281+> `@ud`&
263− 0
282+0
264
283
265− > ~
284+> ~
266− ~
285+~
267
286
268− > `@ud`~
287+> `@ud`~
269− 0
288+0
270
289
271− > ? ~zod
290+> ? ~zod
272− @p
291+ @p
273− ~zod
292+~zod
274
293
275− > `@ux`~forseg-bolbyn
294+> `@ux`~forseg-bolbyn
276− 0x885e.8af9
295+0x885e.8af9
296+```
277
297
278−`%symbol` is a constant with aura `@tas` (text, ASCII, symbol). The symbol
298+`%symbol` is a constant with aura `@tas` (text, ASCII, symbol).
279−rules, which are enforced by the Hoon parser but not (as we've discussed) the
299+The symbol rules, which are enforced by the Hoon parser but not
280−type system: a symbol is lowercase with infix hyphens ("kebab-case").
300+(as we've discussed) the type system: a symbol is lowercase with
301+infix hyphens ("kebab-case").
281
302
282−Loobeans are not true and false, but yes (`.y` or `&`) and no (`.n` or `|`).
303+Loobeans are not true and false, but yes (`%.y` or `&`) and no
283−This is because *yes is zero and no is one*. This makes sense at a certain
304+(`%.n` or `|`). This is because *yes is zero and no is one*.
284−mathematical level, though it was probably a mistake from a practical
305+This makes sense at a certain mathematical level, though it
285−perspective.
306+was probably a mistake from a practical perspective.
286
307
287`~` is Hoon's nil: constant `0`, aura `@n`.
308`~` is Hoon's nil: constant `0`, aura `@n`.
288
309
289−And of course, `@p` is the phonemic base used for ship names. (It's useful for
310+And of course, `@p` is the phonemic base used for ship names.
290−any kind of memorable number.)
311+(It's useful for any kind of memorable number.)
291
312
292## Cells and types
313## Cells and types
293
314
294−Naturally, all these atoms fit neatly into cells, and retain their types within
315+Naturally, all these atoms fit neatly into cells, and retain
295−the cells.
316+their types within the cells.
296
317
297− > ? [~h6 [.3.14 %foo] 0xdead.beef]
318+```
298− {@dr {@rs $foo} @ux}
319+> ? [~h6 [.3.14 %foo] 0xdead.beef]
299− [~h6 [.3.14 %foo] 0xdead.beef]
320+ {@dr {@rs $foo} @ux}
321+[~h6 [.3.14 %foo] 0xdead.beef]
300
322
301− > `*`[~h6 [.3.14 %foo] 0xdead.beef]
323+> `*`[~h6 [.3.14 %foo] 0xdead.beef]
302− [398.449.671.992.126.314.905.600 [1.078.523.331 7.303.014] 3.735.928.559]
324+[398.449.671.992.126.314.905.600 [1.078.523.331 7.303.014] 3.735.928.559]
325+```
303
326
304−Another fun trick is to cast any noun to the `*` type, ie, generic noun. We then
327+Another fun trick is to cast any noun to the `*` type, ie,
305−know nothing about the noun, so we print all its atoms as decimals.
328+generic noun. We then know nothing about the noun, so we print
329+all its atoms as decimals.
306
330
307## And we're done
331## And we're done
308
332
309−And we're done! Type `|exit` to exit:
333+And we're done! Type `|exit` to exit:
310
334
311− ~palnul_nocser:dojo> |exit
335+```
312− %drum-exit
336+~palnul_nocser:dojo> |exit
313− $
337+%drum-exit
338+$
339+```
314
340
315## Questions and/or exercises
341## Questions and/or exercises
316
342
317As always, these questions are optional and only for fun.
343As always, these questions are optional and only for fun.
318
344
319−We didn't include anywhere near all Hoon's auras or syntaxes in this lesson.
345+We didn't include anywhere near all Hoon's auras or syntaxes in
320−It's just a set of examples.
346+this lesson. It's just a set of examples.
321
347
322−- Try using the dojo's magic error erasure to stumble around the rest of
348+* Try using the dojo's magic error erasure to stumble around the
323− Hoon's atom syntax. Any string that can't be extended into a valid
349+rest of Hoon's atom syntax. Any string that can't be extended
324− expression will be trimmed back, with a beep, until it can. Return will beep
350+into a valid expression will be trimmed back, with a beep, until
325− if the expression is not complete. This feedback is harmless and does not
351+it can. Return will beep if the expression is not complete.
326− use an electric shock. It enables stochastic reinforcement learning of Hoon
352+This feedback is harmless and does not use an electric shock.
327− syntax.
353+It enables stochastic reinforcement learning of Hoon syntax.
328
354
329−- If `~h6` is six hours, how do you write forty-five minutes? Two days, six
355+* If `~h6` is six hours, how do you write forty-five minutes? Two
330− hours and forty-five minutes?
356+days, six hours and forty-five minutes?
331
357
332−- What would you expect the syntax for double-precision float to be? What
358+* What would you expect the syntax for double-precision float to
333− about base64?
359+be? What about base64?
334
360
335−- Does an absolute date have to include every zillisecond, or can it be pruned
361+* Does an absolute date have to include every zillisecond, or can
336− to a day or a year?
362+it be pruned to a day or a year?
337
363
338−- If there's an IP syntax, what about bitcoin addresses? What other auras do
364+* If there's an IP syntax, what about bitcoin addresses? What
339− you think should be supported?
365+other auras do you think should be supported?
docs/byte/2.md+352−289
@@ -1,440 +1,503 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 3
3sort: 3
4−title: 'Subject-oriented programming'
4+title: Subject-oriented programming
5---
5---
6
6
7# Urbyte 2: Subject-oriented programming
7# Urbyte 2: Subject-oriented programming
8
8
9−Thanks for reading Urbytes 0 and 1! In Urbyte 2, we'll take nouns for granted
9+Thanks for reading Urbytes 0 and 1! In Urbyte 2, we'll take
10−and have some real fun with them.
10+nouns for granted and have some real fun with them.
11
11
12Let's restart our engines:
12Let's restart our engines:
13
13
14− $ urbit mycomet
14+```
15− [...]
15+$ urbit mycomet
16− ames: on localhost, UDP 31337.
16+[...]
17− http: live (insecure, public) on 8080
17+ames: on localhost, UDP 31337.
18− http: live ("secure", public) on 8443
18+http: live (insecure, public) on 8080
19− http: live (insecure, loopback) on 12321
19+http: live ("secure", public) on 8443
20− ~palnul_nocser:dojo>
20+http: live (insecure, loopback) on 12321
21+~palnul_nocser:dojo>
22+```
21
23
22## Welcome to the dojo
24## Welcome to the dojo
23
25
24Wait, where are we?
26Wait, where are we?
25
27
26−We're playing in the Urbit `:dojo`. The dojo is sort of like a cross between a
28+We're playing in the Urbit `:dojo`. The dojo is sort of like a
27−Lisp REPL and the Unix shell. (If you know the Unix shell, the Urbit command
29+cross between a Lisp REPL and the Unix shell. (If you know the
28−line has the same history model and Emacs editing commands.)
30+Unix shell, the Urbit command line has the same history model and
31+Emacs editing commands.)
29
32
30−The dojo has its own command language, which is a superset of Hoon. Any Hoon
33+The dojo has its own command language, which is a superset of
31−expression is a command. But not all commands are expressions.
34+Hoon. Any Hoon expression is a command. But not all commands
35+are expressions.
32
36
33−(The dojo is not the only command-line app on your Urbit. By default there is
37+(The dojo is not the only command-line app on your Urbit. By
34−one other, `:talk`. Since you're on a fakezod, you can only talk to yourself. If
38+default there is one other, `:talk`. Since you're on a fakezod,
35−you accidentally press `ctrl-x`, it will put you into `:talk`; press `ctrl-x`
39+you can only talk to yourself. If you accidentally press `ctrl-x`,
36−again to switch back.)
40+it will put you into `:talk`; press `ctrl-x` again to switch back.)
37
41
38### Dojo variables
42### Dojo variables
39
43
40−Like the Unix shell, the dojo has command variables. Type
44+Like the Unix shell, the dojo has command variables. Type
41
45
42− > =a 42
46+```
47+> =a 42
48+```
43
49
44−You've set the dojo variable `a` to the value `42`. The dojo puts this binding
50+You've set the dojo variable `a` to the value `42`. The dojo
45−in the Hoon subject for expressions to use. So you can write:
51+puts this binding in the Hoon subject for expressions to use.
52+So you can write:
46
53
47− > a
54+```
48− 42
55+> a
56+42
49
57
50− > [4 a]
58+> [4 a]
51− [4 42]
59+[4 42]
52
60
53− > ? [4 a]
61+> ? [4 a]
54− {@ud @ud}
62+ {@ud @ud}
55− [4 42]
63+[4 42]
64+```
56
65
57To unbind `a`, just omit the value:
66To unbind `a`, just omit the value:
58
67
59− > =a
68+```
69+> =a
70+```
60
71
61−You removed `a` from the subject. (We can bind it again later, or bind over it
72+You removed `a` from the subject. (We can bind it again later, or
62−without removing it.)
73+bind over it without removing it.)
63
74
64− > [4 a]
75+```
65− -find.a
76+> [4 a]
77+-find.a
78+```
66
79
67Now trying to use `a` is an error.
80Now trying to use `a` is an error.
68
81
69## Subject-oriented programming
82## Subject-oriented programming
70
83
71−What is this *subject* thing, anyway? We put `a` in what?
84+What is this *subject* thing, anyway? We put `a` in what?
72
85
73−Most programming languages have something called a *scope* or an *environment*
86+Most programming languages have something called a *scope* or an
74−or a *context*. This is a data structure that contains the variables and
87+*environment* or a *context*. This is a data structure that
75−functions an expression can use.
88+contains the variables and functions an expression can use.
76
89
77−In a normal language, the environment is not a first-class value. You cannot
90+In a normal language, the environment is not a first-class value.
78−save an environment in a variable, compute an expression against the environment
91+You cannot save an environment in a variable, compute an
79−in the variable, etc, etc.
92+expression against the environment in the variable, etc, etc.
80
93
81−In Hoon or any "subject-oriented" language, an expression is executed against
94+In Hoon or any "subject-oriented" language, an expression is
82−one normal value, the *subject*. In Hoon, of course, the subject is a noun.
95+executed against one normal value, the *subject*. In Hoon, of
96+course, the subject is a noun.
83
97
84## First experiments
98## First experiments
85
99
86−Up to now, we've only built constants. By definition, a constant doesn't use the
100+Up to now, we've only built constants. By definition, a constant
87−subject.
101+doesn't use the subject.
88
102
89−It would be nice to start with the subject every dojo expression gets. But that
103+It would be nice to start with the subject every dojo expression
90−subject is pretty complex. Instead, let's make a simple noun from the kinds of
104+gets. But that subject is pretty complex. Instead, let's make a
91−atoms we've already learned about:
105+simple noun from the kinds of atoms we've already learned about:
92
106
93− > =a [42 [%foo .6.28 ~m45] 0xdead.beef]
107+```
108+> =a [42 [%foo .6.28 ~m45] 0xdead.beef]
94
109
95− > ? a
110+> ? a
96− {@ud {$foo @rs @dr} @ux}
111+ {@ud {$foo @rs @dr} @ux}
97− [42 [%foo .3.14 ~m45] 0xdead.beef]
112+[42 [%foo .3.14 ~m45] 0xdead.beef]
98
113
99− > 420:a
114+> 420:a
100− 420
115+420
116+```
101
117
102−The syntax `expression:expression` uses the product of the second expression as
118+The syntax `expression:expression` uses the product of the second
103−the subject of the first. So we're using `a` as the subject of the expression
119+expression as the subject of the first. So we're using `a` as
104−`420`.
120+the subject of the expression `420`.
105
121
106−Of course, `420` is a constant expression. It doesn't care what the subject is.
122+Of course, `420` is a constant expression. It doesn't care what
107−But:
123+the subject is. But:
108
124
109− > (add 2 2):a
125+```
110− -find.add
126+> (add 2 2):a
127+-find.add
128+```
111
129
112−We can't use `a` as the subject of even a simple addition. How could we? There's
130+We can't use `a` as the subject of even a simple addition. How
113−no `add` function in our `a`. It sometimes takes new Hoon programmers a little
131+could we? There's no `add` function in our `a`. It sometimes
114−while before they realize that there really isn't a scope or environment in
132+takes new Hoon programmers a little while before they realize
115−Hoon.
133+that there really isn't a scope or environment in Hoon.
116
134
117## Tree geometry
135## Tree geometry
118
136
119−So what can we do with this subject? How can we get data out of it? Let's keep
137+So what can we do with this subject? How can we get data out of
120−working with our little test subject `a`.
138+it? Let's keep working with our little test subject `a`.
121
139
122−The most basic way to access the subject is *tree geometry*. Every noun is a
140+The most basic way to access the subject is *tree geometry*.
123−binary tree; every noun in the tree has a number which is its address. The root
141+Every noun is a binary tree; every noun in the tree has a number
124−of the tree is `1`; if the address of a cell is `n`, its head is `2n` and its
142+which is its address. The root of the tree is `1`. If the
125−tail is `2n+1`.
143+address of a cell is `n`, the address of its head is `2n` and the
144+address of its tail is `2n+1`.
126
145
127A picture might help:
146A picture might help:
128−
147+```
129− 1
148+ 1
130− / \
149+ / \
131− 2 3
150+ 2 3
132− / \ / \
151+ / \ / \
133− 4 5 6 7
152+ 4 5 6 7
134− / \
153+ / \
135− 14 15
154+ 14 15
136−
155+```
137−The syntax for a fragment of the subject at a tree address, or *axis*, is just
156+
138−`+axis`. So:
157+The syntax for a fragment of the subject at a tree address, or
139−
158+*axis*, is just `+axis`. So:
140− > +3:a
159+
141− [[%foo .3.14 ~m45] 0xdead.beef]
160+```
142−
161+> +3:a
143− > ? +7:a
162+[[%foo .3.14 ~m45] 0xdead.beef]
144− @ux
163+
145− 0xdead.beef
164+> ? +7:a
146−
165+ @ux
147− > ? +13:a
166+0xdead.beef
148− {@rs @dr}
167+
149− [.3.14 ~m45]
168+> ? +13:a
150−
169+ {@rs @dr}
151−Of course, the fragment retains the type it had when we built it.
170+[.3.14 ~m45]
171+```
172+
173+The fragment retains the type it had when we built it.
174+
175+Another (equivalent) way to think of tree addressing involves
176+binary. As before, the root of the tree is at address `1`. For
177+the node of a tree at some address `b`, where `b` is binary,
178+you get the address of the head by concatenating a `0` to
179+the end of `b`; and to get the tail, concatenate a `1` at the
180+end. For example, the head of the node at binary address `111`
181+is at `1110`, and the tail is at `1111`.
152
182
153## Labeled surfaces
183## Labeled surfaces
154
184
155−We can obviously get anything we want out of the subject by tree geometry. But
185+We can obviously get anything we want out of the subject by tree
156−that's not really how we'd like to program. We'd much rather *label* subtrees
186+geometry. But that's not really how we'd like to program. We'd
157−with convenient, memorable symbols.
187+much rather *label* subtrees with convenient, memorable symbols.
158
188
159For example, suppose our tree looked like:
189For example, suppose our tree looked like:
160
190
161− 1
191+```
162− / \
192+ 1
163− x=2 3
193+ / \
164− / \
194+ x=2 3
165− y=6 z=7
195+ / \
166− / \
196+ y=6 z=7
167− p=14 15
197+ / \
168− / \
198+ p=14 15
169− q=30 r=31
199+ / \
200+ q=30 r=31
201+```
170
202
171Then instead of writing `+2`, we could write `x`, and so on.
203Then instead of writing `+2`, we could write `x`, and so on.
172
204
173−Let's replace our test subject `a` with a new noun built around this tree of
205+Let's replace our test subject `a` with a new noun built around
174−*labeled surfaces*.
206+this tree of *labeled surfaces*.
175
207
176− > =a [x=42 y=[p=%foo q=.6.28 r=~m45] z=0xdead.beef]
208+```
209+> =a [x=42 y=[p=%foo q=.6.28 r=~m45] z=0xdead.beef]
177
210
178− > ? x:a
211+> ? x:a
179− @ud
212+ @ud
180− 42
213+42
181
214
182− > ? y:a
215+> ? y:a
183− {p/$foo q/@rs r/@dr}
216+ {p/$foo q/@rs r/@dr}
184− [p=%foo q=.6.28 r=~m45]
217+[p=%foo q=.6.28 r=~m45]
185
218
186− > ? q.y:a
219+> ? q.y:a
187− @rs
220+ @rs
188− .6.28
221+.6.28
222+```
189
223
190−Surfaces might remind you of attributes in an object. But again, attributes in
224+Surfaces might remind you of attributes in an object. But again,
191−an object are a symbol table. There is no symbol table here -- just a tree
225+attributes in an object are a symbol table. There is no symbol
192−search.
226+table here -- just a tree search.
193
227
194−An expression like the label `x` or the axis `+2`, which designates a fragment
228+An expression like the label `x` or the axis `+2`, which
195−or feature of the subject, is called a *limb*. A list of limbs connected with
229+designates a fragment or feature of the subject, is called a
196−`.` is a *wing*.
230+*limb*. A list of limbs connected with `.` is a *wing*.
197
231
198−To remind you that you're not in Kansas anymore, the syntax for a wing, like
232+To remind you that you're not in Kansas anymore, the syntax for a
199−`q.y` above, means means "q inside y." In a classic object-oriented system, of
233+wing, like `q.y` above, means means "q inside y." In a classic
200−course, this would be written `y.q`, that is, attribute `q` within object `y`.
234+object-oriented system, of course, this would be written `y.q`,
201−In Hoon, `q.y` means "find limb `y`, then find limb `q` inside that."
235+that is, attribute `q` within object `y`. In Hoon, `q.y` means
236+"find limb `y`, then find limb `q` inside that."
202
237
203## Surfaces versus fragments
238## Surfaces versus fragments
204
239
205−Once again, Hoon has no symbol tables. When we put a surface on a noun, the
240+Once again, Hoon has no symbol tables. When we put a surface
206−surface label is part of that noun's type. It's not part of the cell or other
241+on a noun, the surface label is part of that noun's type. It's
207−structure that contains the noun. This will take a little getting used to:
242+not part of the cell or other structure that contains the noun.
243+This will take a little getting used to:
208
244
209− > ? x:a
245+```
210− @ud
246+> ? x:a
211− 42
247+ @ud
248+42
212
249
213− > ? +2:a
250+> ? +2:a
214− x/@ud
251+ x/@ud
215− x=42
252+x=42
216
253
217− ? x.+2:a
254+? x.+2:a
218− @ud
255+ @ud
219− 42
256+42
257+```
220
258
221−When we take fragment 2 of the labeled subject, the *surface comes with it*.
259+When we take fragment 2 of the labeled subject, the *surface
222−`+2` is not `42` but rather `x=42`. We can take the surface off with `x.+2`.
260+comes with it*. `+2` is not `42` but rather `x=42`. We can
261+take the surface off with `x.+2`.
223
262
224## The search algorithm
263## The search algorithm
225
264
226−In resolving a label to a fragment axis, we search the subject type in a
265+In resolving a label to a fragment axis, we search the subject
227−depth-first traverse.
266+type in a depth-first traverse.
228
267
229−If we find a surface that matches the limb, we succeed. If we find a surface
268+If we find a surface that matches the limb, we succeed. If we
230−that doesn't match the limb, we fail. If we find a cell, we search first head,
269+find a surface that doesn't match the limb, we fail. If we find
231−then tail.
270+a cell, we search first head, then tail.
232
271
233## Skipping
272## Skipping
234
273
235−What if the same label appears twice in one noun? Let's modify our subject
274+What if the same label appears twice in one noun? Let's modify
236−slightly as a test case:
275+our subject slightly as a test case:
237
276
238− > ? +7:a
277+```
239− z/@ux
278+> ? +7:a
240− z=0xdead.beef
279+ z/@ux
280+z=0xdead.beef
241
281
242− > =a a(+7 x=z:a)
282+> =a a(+7 x=z:a)
243
283
244− > ? a
284+> ? a
245− {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}
285+ {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}
246− [x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
286+[x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
247
287
248− > ? x:a
288+> ? x:a
249− @ud
289+ @ud
250− 42
290+42
251
291
252− > ? ^x:a
292+> ? ^x:a
253− @ux
293+ @ux
254− 0xdead.beef
294+0xdead.beef
295+```
255
296
256−We don't want to hide any part of the noun from an expression. So the modified
297+We don't want to hide any part of the noun from an expression.
257−limb `^x` *skips* the first match with a surface `x`, getting us to our
298+So the modified limb `^x` *skips* the first match with a surface
258−`0xdead.beef`. This is a unary notation: `^^x` would skip the first two matches.
299+`x`, getting us to our `0xdead.beef`. This is a unary notation:
300+`^^x` would skip the first two matches.
259
301
260## Editing
302## Editing
261
303
262Above, we mutated our first noun with the flashy `a(+7 x=z:a)`.
304Above, we mutated our first noun with the flashy `a(+7 x=z:a)`.
263
305
264−Of course we are not really "modifying" a noun. Hoon is a pure language. We are
306+Of course we are not really "modifying" a noun. Hoon is a pure
265−making an edited copy. `a(+7 x=z:a)` means "a copy of `a`, with limb `+7`
307+language. We are making an edited copy. `a(+7 x=z:a)` means "a
266−replaced by the product of `x=z:a`."
308+copy of `a`, with limb `+7` replaced by the product of `x=z:a`."
267
309
268We can edit multiple (disjoint) wings in one expression:
310We can edit multiple (disjoint) wings in one expression:
269
311
270− > ? a(+7 [%hello %world], q.y .3.14)
312+```
271− {x/@ud y/{p/$foo q/@rs r/@dr} $hello $world}
313+> ? a(+7 [%hello %world], q.y .3.14)
272− [x=42 y=[p=%foo q=.3.14 r=~m45] %hello %world]
314+ {x/@ud y/{p/$foo q/@rs r/@dr} $hello $world}
315+[x=42 y=[p=%foo q=.3.14 r=~m45] %hello %world]
316+```
273
317
274−Notice that nothing stopped us from changing `+7` from `x=0xdead.beef` to
318+Notice that nothing stopped us from changing `+7` from
275−`[%hello %world]`. Since the mutated copy is a different noun, it has a
319+`x=0xdead.beef` to `[%hello %world]`. Since the mutated
276−different type.
320+copy is a different noun, it has a different type.
277
321
278−(And remember that since `[x y z]` means `[x [y z]]`, `[x y [%hello %world]]`
322+(And remember that since `[x y z]` means `[x [y z]]`,
279−means `[x y %hello %world]`.)
323+`[x y [%hello %world]]` means `[x y %hello %world]`.)
280
324
281## Cores and arms
325## Cores and arms
282
326
283−It should be reasonably clear how Hoon's system of fragments and surfaces
327+It should be reasonably clear how Hoon's system of fragments and
284−parallels variables, records or structures in a classical language. But where is
328+surfaces parallels variables, records or structures in a
285−Hoon's equivalent of objects: values which bind code to data?
329+classical language. But where is Hoon's equivalent of objects:
330+values which bind code to data?
286
331
287−The Hoon equivalent of an object is a *core*. As usual in Hoon, the resemblance
332+The Hoon equivalent of an object is a *core*. As usual in Hoon,
288−is inexact. A core has a set of computed attributes, or *arms*. Each arm is an
333+the resemblance is inexact. A core has a set of computed
289−expression whose subject is the core. There is no argument, so an arm is not a
334+attributes, or *arms*. Each arm is an expression whose subject
290−method.
335+is the core. There is no argument, so an arm is not a method.
291
336
292−Physically, a core is always a cell `[battery payload]`, where the *payload* is
337+Physically, a core is always a cell `[battery payload]`, where
293−any noun, and the *battery* is a tree of formulas in Nock (Hoon's functional
338+the *payload* is any noun, and the *battery* is a tree of
294−assembly language) that implements the arms. (A battery is like a C++ vtable,
339+formulas in Nock (Hoon's functional assembly language) that
295−sort of.)
340+implements the arms. (A battery is like a C++ vtable, sort of.)
296
341
297## The search algorithm
342## The search algorithm
298
343
299−As you'd expect with a computed attribute, there is only one namespace and one
344+As you'd expect with a computed attribute, there is only one
300−search algorithm. We just extend the algorithm described above. And an
345+namespace and one search algorithm. We just extend the algorithm
301−expression which uses a label can't know whether the subject will resolve it to
346+described above. And an expression which uses a label can't know
302−a surface fragment (a *leg*) or to an arm.
347+whether the subject will resolve it to a surface fragment (a
348+*leg*) or to an arm.
303
349
304−If we're searching a core and the label matches one of the core's arms, we
350+If we're searching a core and the label matches one of the core's
305−compute the arm against the core. If the label is not found, we search into the
351+arms, we compute the arm against the core. If the label is not
306−payload of the core.
352+found, we search into the payload of the core.
307
353
308## Some core examples
354## Some core examples
309
355
310−When you type these examples, make sure you get the number of spaces exactly
356+When you type these examples, make sure you get the number of
311−right. Hoon syntax is not whitespace-sensitive in general, except that "one
357+spaces exactly right. Hoon syntax is not whitespace-sensitive
312−space" is always a different token from "two or more spaces."
358+in general, except that "one space" is always a different token
359+from "two or more spaces."
313
360
314− > ? x:a
361+```
315− @ud
362+> ? x:a
316− 42
363+ @ud
364+42
317
365
318− > ? :per a x
366+> ? :per a x
319− @ud
367+ @ud
320− 42
368+42
321
369
322− > =b :per a :core ++ foo x ++ bar [q.y foo] --
370+> =b :per a :core ++ foo x ++ bar [q.y foo] --
371+```
323
372
324−We're seeing some Hoon regular forms for the first time. `x:a` turns out to be
373+We're seeing some Hoon regular forms for the first time. `x:a`
325−an irregular relative of the regular `:per` form.
374+turns out to be an irregular relative of the regular `:per` form.
326
375
327−We'll get more into syntax in a later Urbyte. If some of these examples seem
376+We'll get more into syntax in a later Urbyte. If some of these
328−syntactically a little funky, this isn't usually way we'd do them in "real
377+examples seem syntactically a little funky, this isn't usually
329−life." We don't normally write cores on a single command line. In a file this
378+way we'd do them in "real life." We don't normally write cores
330−would be:
379+on a single command line. In a file this would be:
331
380
332− :per a
381+```
333− :core
382+ :per a
334− ++ foo x
383+ :core
335− ++ bar [q.y foo]
384+ ++ foo x
336− --
385+ ++ bar [q.y foo]
386+ --
387+```
337
388
338−which is much more readable, at least once you get used to it. Let's look at the
389+which is much more readable, at least once you get used to it.
339−core itself:
390+Let's look at the core itself:
340
391
341− > b
392+```
342− <2.spy {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}>
393+> b
394+<2.spy {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}>
343
395
344− > ? +2:b
396+> ? +2:b
345− *
397+ *
346− [[0 6] [0 58] 9 4 0 1]
398+[[0 6] [0 58] 9 4 0 1]
347
399
348− > ? +3:b
400+> ? +3:b
349− {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}
401+ {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}
350− [x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
402+[x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
403+```
351
404
352−When we print `b`, or any core, we see that typed noun printing can't be an
405+When we print `b`, or any core, we see that typed noun printing
353−invertible function in Hoon.
406+can't be an invertible function in Hoon.
354
407
355−We actually can read the battery (with `+2:b` above), but it's just Nock code,
408+We actually can read the battery (with `+2:b` above), but it's
356−not typed data. `<2.spy>` tells us the battery has two arms and hashes to the
409+just Nock code, not typed data. `<2.spy>` tells us the battery
357−three-letter code `spy`.
410+has two arms and hashes to the three-letter code `spy`.
358
411
359−When printing a full core, we don't even print the payload data, just its type.
412+When printing a full core, we don't even print the payload data,
360−But our `+3:b` confirms the `[battery payload]` structure of the core. Remember,
413+just its type. But our `+3:b` confirms the `[battery payload]`
361−`+2` is the head and `+3` is the tail.
414+structure of the core. Remember, `+2` is the head and `+3` is
415+the tail.
362
416
363Finally, let's use this thing:
417Finally, let's use this thing:
364
418
365− > ? foo:b
419+```
366− @ud
420+> ? foo:b
367− 42
421+ @ud
422+42
368
423
369− > ? bar:b
424+> ? bar:b
370− {@rs @ud}
425+ {@rs @ud}
371− [.6.28 42]
426+[.6.28 42]
372
427
373− ? x:b
428+? x:b
374− @ud
429+ @ud
375− 42
430+42
431+```
376
432
377Seems legit.
433Seems legit.
378
434
379## The dojo subject
435## The dojo subject
380
436
381−Now that we've made a test subject which is a core, we can look directly at the
437+Now that we've made a test subject which is a core, we can look
382−real dojo subject:
438+directly at the real dojo subject:
383−
384− > +1
385− [ [ [ a=[x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
386− b=<2.spy {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}>
387− ]
388− now=~2016.7.18..08.05.15..0b5c
389− eny=0vl.hsfrh.tg642.g5o21.1n80b.0h141.0vuan.pm8o4.77ovg.96rk0.n26nn
390− our=~palnul-wolmev-bonfex-danred--tilwen-midlux-bolfus-nocser
391− ]
392− ~
393− <283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>
394− ]
395−
396−We see a bunch of surfaces: our custom `a` and `b`; standard surfaces for
397−identity, date, and entropy; and a ginormous core stack with *over eight
398−hundred* total arms.
399
439
400−To be exact, the syntax `<283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>` denotes a
440+```
401−stack of six cores, each of whose payload is another core, until the innermost
441+> +1
402−payload `%151` (the Hoon version number). This stack is our standard library,
442+[ [ [ a=[x=42 y=[p=%foo q=.6.28 r=~m45] x=0xdead.beef]
403−from which comes `add` and other fine functions. (We don't know how to write a
443+ b=<2.spy {x/@ud y/{p/$foo q/@rs r/@dr} x/@ux}>
404−Hoon function yet, but we're getting way closer.)
444+ ]
445+ now=~2016.7.18..08.05.15..0b5c
446+ eny=0vl.hsfrh.tg642.g5o21.1n80b.0h141.0vuan.pm8o4.77ovg.96rk0.n26nn
447+ our=~palnul-wolmev-bonfex-danred--tilwen-midlux-bolfus-nocser
448+ ]
449+ ~
450+ <283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>
451+]
452+```
453+
454+We see a bunch of surfaces: our custom `a` and `b`; standard
455+surfaces for identity, date, and entropy; and a ginormous core
456+stack with *over eight hundred* total arms.
457+
458+To be exact, the syntax `<283.xzp 42.ajz 407.xtr 110.xht 1.ztu
459+$151>` denotes a stack of six cores, each of whose payload is
460+another core, until the innermost payload `%151` (the Hoon
461+version number). This stack is our standard library, from which
462+comes `add` and other fine functions. (We don't know how to
463+write a Hoon function yet, but we're getting way closer.)
405
464
406Let's clean up our subject, just to be hygienic:
465Let's clean up our subject, just to be hygienic:
407
466
408− > =a
467+```
409− > =b
468+> =a
469+> =b
410
470
411− > +1
471+> +1
412− now=~2016.7.18..08.10.42..36a0
472+ now=~2016.7.18..08.10.42..36a0
413− eny=0vf.8j71j.hlqim.k340u.q0ual.v2cd8.cfa65.vkbte.e733g.ehihg.e8aog
473+ eny=0vf.8j71j.hlqim.k340u.q0ual.v2cd8.cfa65.vkbte.e733g.ehihg.e8aog
414− [ [ our=~palnul-wolmev-bonfex-danred--tilwen-midlux-bolfus-nocser
474+[ [ our=~palnul-wolmev-bonfex-danred--tilwen-midlux-bolfus-nocser
415− ]
475+ ]
416− ~
476+ ~
417− <283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>
477+ <283.xzp 42.ajz 407.xtr 110.xht 1.ztu $151>
418− ]
478+]
479+```
419
480
420−Again, remember that `=a` is not Hoon syntax, but `:dojo` syntax. How the dojo
481+Again, remember that `=a` is not Hoon syntax, but `:dojo` syntax.
421−assembles this subject remains a mystery. But it's not a particularly
482+How the dojo assembles this subject remains a mystery. But it's
422−interesting mystery.
483+not a particularly interesting mystery.
423
484
424## And we're done
485## And we're done
425
486
426−And we're done! Type `|exit` to exit:
487+And we're done! Type `|exit` to exit:
427
488
428− ~palnul_nocser:dojo> |exit
489+```
429− %drum-exit
490+~palnul_nocser:dojo> |exit
430− $
491+%drum-exit
492+$
493+```
431
494
432## Questions and/or exercises
495## Questions and/or exercises
433
496
434As always, these questions are optional and only for fun.
497As always, these questions are optional and only for fun.
435
498
436−What's the best way to construct a core which acts as a function? What do you do
499+What's the best way to construct a core which acts as a function?
437−to call the function?
500+What do you do to call the function?
438
501
439−Why don't any other languages use subject-oriented programming? Which languages
502+Why don't any other languages use subject-oriented programming?
440−are most similar to it?
503+Which languages are most similar to it?
docs/community-docs.md+21−20
@@ -1,30 +1,31 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 7
3sort: 7
4---
4---
5
5
6+
6# Community docs
7# Community docs
7
8
8<div>
9<div>
10+
9<ul>
11<ul>
10−<li>
12+
11−<code>~master-morzod</code> <br />
13+<li><code>~master-morzod</code>
12−<a href="https://mardev.urbit.org/docs/">docs</a>, <code>fora</code>
14+<br />
13−<a href="https://urbit.org/fora/posts/~2016.12.25..06.35.44..a1ec~/">post</a>
15+<a href="https://mardev.urbit.org/docs/">docs</a>, <code>fora</code> <a href="https://urbit.org/fora/posts/~2016.12.25..06.35.44..a1ec~/">post</a></li>
14−</li>
16+
15−<li>
17+<li><code>~hidduc-posmeg</code>
16−<code>~hidduc-posmeg</code> <br />
18+<br />
17−<a href="https://fosnut-dandut.urbit.org/pages/hidducs-notes/tutorial/">docs</a>,
19+<a href="https://fosnut-dandut.urbit.org/pages/hidducs-notes/tutorial/">docs</a>, <code>fora</code> <a href="https://urbit.org/fora/posts/~2017.2.12..21.54.40..6fde~/">post</a></li>
18−<code>fora</code>
20+
19−<a href="https://urbit.org/fora/posts/~2017.2.12..21.54.40..6fde~/">post</a>
21+<li><code>~palfun-foslup</code>
20−</li>
22+<br />
21−<li>
23+<a href="https://github.com/Fang-/Urbit-By-Doing">Urbit by Doing</a></li>
22−<code>~palfun-foslup</code> <br />
24+
23−<a href="https://github.com/Fang-/Urbit-By-Doing">Urbit by Doing</a>
25+<li><code>@knubie</code>
24−</li>
26+<br />
25−<li>
27+<a href="https://github.com/knubie/learning-hoon">Learning Hoon</a></li>
26−<code>@knubie</code> <br />
28+
27−<a href="https://github.com/knubie/learning-hoon">Learning Hoon</a>
28−</li>
29</ul>
29</ul>
30+
30</div>
31</div>
docs/community-projects.md+19−15
@@ -1,26 +1,30 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3sort: 8
3sort: 8
4---
4---
5
5
6+
6# Community projects
7# Community projects
7
8
8<div>
9<div>
10+
9<ul>
11<ul>
10−<li>
12+
11−<a href="https://github.com/ponnys-podfer/yint">Yint: A Tiny MUD</a> <br />
13+<li><a href="https://github.com/ponnys-podfer/yint">Yint: A Tiny MUD</a>
12−<code>~ponnys-podfer</code>
14+<br />
13−</li>
15+<code>~ponnys-podfer</code></li>
14−<li>
16+
15−<a href="https://github.com/Fang-/talkbot">~talkbot</a> <br />
17+<li><a href="https://github.com/Fang-/talkbot">~talkbot</a>
16−<code>~palfun-foslup</code>
18+<br />
17−</li>
19+<code>~palfun-foslup</code></li>
18−<li>
20+
19−<a href="https://github.com/Fang-/urbit-string">String processing</a> <br />
21+<li><a href="https://github.com/Fang-/urbit-string">String processing</a>
20−<code>~palfun-foslup</code>
22+<br />
21−</li>
23+<code>~palfun-foslup</code></li>
22−<li>
24+
23−<a href="community-projects/nock-implementations">Nock implementations</a>
25+<li><a href="community-projects/nock-implementations">Nock implementations</a>
24</li>
26</li>
27+
25</ul>
28</ul>
29+
26</div>
30</div>
docs/community-projects/nock-implementations.md+2−2
@@ -1,8 +1,8 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
4sort: 4
3sort: 4
5title: Nock implementations
4title: Nock implementations
5+next: true
6---
6---
7
7
8# Community Nock implementations
8# Community Nock implementations
docs/community-projects/nock-implementations/clojure.md+118−117
@@ -1,124 +1,125 @@
1---
1---
2−navhome: '/docs/'
2+navhome: /docs/
3−next: True
4−sort: 5
5title: Clojure
3title: Clojure
4+sort: 5
5+next: true
6---
6---
7
7
8# Clojure
8# Clojure
9
9
10−From [Matt
10+From [Matt Earnshaw](https://github.com/mattearnshaw/anock/blob/master/src/anock/core.clj):
11−Earnshaw](https://github.com/mattearnshaw/anock/blob/master/src/anock/core.clj):
11+
12−
12+```
13− (ns anock.core
13+(ns anock.core
14− (:import anock.NockException))
14+ (:import anock.NockException))
15−
15+
16− (declare atom? cell? cell)
16+(declare atom? cell? cell)
17−
17+
18− (defn noun?
18+(defn noun?
19− "A noun is an atom or a cell."
19+ "A noun is an atom or a cell."
20− [noun & ns]
20+ [noun & ns]
21− (if ns false
21+ (if ns false
22− (or (atom? noun) (cell? noun))))
22+ (or (atom? noun) (cell? noun))))
23−
23+
24− (defn atom?
24+(defn atom?
25− "An atom is a natural number."
25+ "An atom is a natural number."
26− [noun & ns]
26+ [noun & ns]
27− (if ns false
27+ (if ns false
28− (and (integer? noun) (>= noun 0))))
28+ (and (integer? noun) (>= noun 0))))
29−
29+
30− (defn cell?
30+(defn cell?
31− "A cell is an ordered pair of nouns."
31+ "A cell is an ordered pair of nouns."
32− [noun]
32+ [noun]
33− (cond
33+ (cond
34− (atom? noun) false
34+ (atom? noun) false
35− (nil? noun) false
35+ (nil? noun) false
36− (not= 2 (count noun)) false
36+ (not= 2 (count noun)) false
37− :else (and (noun? (first noun))
37+ :else (and (noun? (first noun))
38− (noun? (second noun)))))
38+ (noun? (second noun)))))
39−
39+
40− (defn tis
40+(defn tis
41− "= (pronounced 'tis') tests a cell for equality."
41+ "= (pronounced 'tis') tests a cell for equality."
42− [noun]
42+ [noun]
43− (if (atom? noun) (throw (anock.NockException. "Cannot tis an atom."))
43+ (if (atom? noun) (throw (anock.NockException. "Cannot tis an atom."))
44− (let [[a b] noun]
44+ (let [[a b] noun]
45− (if (= a b) 0 1))))
45+ (if (= a b) 0 1))))
46−
46+
47− (defn wut
47+(defn wut
48− "? (pronounced 'wut') tests whether a noun is a cell."
48+ "? (pronounced 'wut') tests whether a noun is a cell."
49− [noun]
49+ [noun]
50− (cond
50+ (cond
51− (atom? noun) 1
51+ (atom? noun) 1
52− (cell? noun) 0
52+ (cell? noun) 0
53− :else (throw (anock.NockException. "Invalid noun."))))
53+ :else (throw (anock.NockException. "Invalid noun."))))
54−
54+
55− (defn lus
55+(defn lus
56− "+ (pronounced 'lus') adds 1 to an atom."
56+ "+ (pronounced 'lus') adds 1 to an atom."
57− [noun]
57+ [noun]
58− (if (atom? noun) (inc noun)
58+ (if (atom? noun) (inc noun)
59− (throw (anock.NockException. "Can only lus atoms."))))
59+ (throw (anock.NockException. "Can only lus atoms."))))
60−
60+
61− (defn fas
61+(defn fas
62− "/ (pronounced 'fas') is a tree address function."
62+ "/ (pronounced 'fas') is a tree address function."
63− [noun]
63+ [noun]
64− (if (atom? noun) (throw (anock.NockException. "Cannot fas an atom."))
64+ (if (atom? noun) (throw (anock.NockException. "Cannot fas an atom."))
65− (let [[a b] (cell noun)]
65+ (let [[a b] (cell noun)]
66− (assert (and (pos? a) (atom? a)) "Subject of fas must be a positive atom.")
66+ (assert (and (pos? a) (atom? a)) "Subject of fas must be a positive atom.")
67− (if (and (not (coll? b)) (or (= 2 a) (= 3 a)))
67+ (if (and (not (coll? b)) (or (= 2 a) (= 3 a)))
68− (throw (anock.NockException. (str "Cannot fas noun: " noun))))
68+ (throw (anock.NockException. (str "Cannot fas noun: " noun))))
69− (cond
70− (= 1 a) b
71− (= 2 a) (first b)
72− (= 3 a) (second b)
73− (even? a) (fas [2 (fas [(/ a 2) b])])
74− (odd? a) (fas [3 (fas [(/ (dec a) 2) b])])))))
75−
76− (defn tar
77− "* (pronounced 'tar') means Nock"
78− [noun]
79− (if (atom? noun) (throw (anock.NockException. "Cannot tar an atom."))
80− (try
81− (let [noun (cell noun) [x [y z]] noun]
82− (cond
83− (cell? y) (cell (tar [x y]) (tar [x z]))
84− (zero? y) (fas [z x])
85− (= 1 y) z
86− (= 3 y) (wut (tar [x z]))
87− (= 4 y) (lus (tar [x z]))
88− (= 5 y) (tis (tar [x z]))
89− :else (let [[p q] z]
90− (cond
91− (= 2 y) (tar [(tar [x p]) (tar [x q])])
92− (= 6 y) (tar [x 2 [0 1] 2 [1 (first q) (second q)]
93− [1 0] 2 [1 2 3] [1 0] 4 4 p])
94− (= 7 y) (tar [x 2 p 1 q])
95− (= 8 y) (tar [x 7 [[7 [0 1] p] 0 1] q])
96− (= 9 y) (tar [x 7 q 2 [0 1] 0 p])
97− (= 10 y) (if (cell? p)
98− (tar [x 8 (second p) 7 [0 3] q])
99− (tar [x q]))))))
100− (catch RuntimeException e
101− (throw (anock.NockException. (str "Cannot tar the noun " noun)))))))
102−
103− (def nock tar)
104−
105− ; Some convenience functions
106− (defn apply* [f x]
107− (if (and (= 1 (count x)) (coll? (first x)))
108− (apply f x)
109− (f x)))
110−
111− (defn bracket
112− "[a b c] -> [a [b c]]"
113− [[a & b :as c]]
114− (let [b (vec b)]
115 (cond
69 (cond
116− (and (noun? a) (apply noun? b)) (vec c)
70+ (= 1 a) b
117− (apply noun? b) (apply vector (bracket a) b)
71+ (= 2 a) (first b)
118− (noun? a) [a (apply* bracket b)]
72+ (= 3 a) (second b)
119− :else [(bracket a) (apply* bracket b)])))
73+ (even? a) (fas [2 (fas [(/ a 2) b])])
120−
74+ (odd? a) (fas [3 (fas [(/ (dec a) 2) b])])))))
121− (defn cell [& nouns]
75+
122− (if (apply atom? nouns)
76+(defn tar
123− (throw (anock.NockException. "Cannot convert atom to cell."))
77+ "* (pronounced 'tar') means Nock"
124− (apply* bracket nouns)))
78+ [noun]
79+ (if (atom? noun) (throw (anock.NockException. "Cannot tar an atom."))
80+ (try
81+ (let [noun (cell noun) [x [y z]] noun]
82+ (cond
83+ (cell? y) (cell (tar [x y]) (tar [x z]))
84+ (zero? y) (fas [z x])
85+ (= 1 y) z
86+ (= 3 y) (wut (tar [x z]))
87+ (= 4 y) (lus (tar [x z]))
88+ (= 5 y) (tis (tar [x z]))
89+ :else (let [[p q] z]
90+ (cond
91+ (= 2 y) (tar [(tar [x p]) (tar [x q])])
92+ (= 6 y) (tar [x 2 [0 1] 2 [1 (first q) (second q)]
93+ [1 0] 2 [1 2 3] [1 0] 4 4 p])
94+ (= 7 y) (tar [x 2 p 1 q])
95+ (= 8 y) (tar [x 7 [[7 [0 1] p] 0 1] q])
96+ (= 9 y) (tar [x 7 q 2 [0 1] 0 p])
97+ (= 10 y) (if (cell? p)
98+ (tar [x 8 (second p) 7 [0 3] q])
99+ (tar [x q]))))))
100+ (catch RuntimeException e
101+ (throw (anock.NockException. (str "Cannot tar the noun " noun)))))))
102+
103+(def nock tar)
104+
105+; Some convenience functions
106+(defn apply* [f x]
107+ (if (and (= 1 (count x)) (coll? (first x)))
108+ (apply f x)
109+ (f x)))
110+
111+(defn bracket
112+ "[a b c] -> [a [b c]]"
113+ [[a & b :as c]]
114+ (let [b (vec b)]
115+ (cond
116+ (and (noun? a) (apply noun? b)) (vec c)
117+ (apply noun? b) (apply vector (bracket a) b)
118+ (noun? a) [a (apply* bracket b)]
119+ :else [(bracket a) (apply* bracket b)])))
120+
121+(defn cell [& nouns]
122+ (if (apply atom? nouns)
123+ (throw (anock.NockException. "Cannot convert atom to cell."))
124+ (apply* bracket nouns)))
125+```
docs/community-projects/nock-implementations/csharp.md+379−377

This diff is too big to show here.

docs/community-projects/nock-implementations/groovy.md+178−177

This diff is too big to show here.

docs/community-projects/nock-implementations/haskell.md+57−56

This diff is too big to show here.

docs/community-projects/nock-implementations/hoon.md+49−47

This diff is too big to show here.

docs/community-projects/nock-implementations/javascript.md+357−355

This diff is too big to show here.

docs/community-projects/nock-implementations/python.md+77−75

This diff is too big to show here.

docs/community-projects/nock-implementations/ruby.md+131−130

This diff is too big to show here.

docs/community-projects/nock-implementations/scala.md+69−68

This diff is too big to show here.

docs/community-projects/nock-implementations/scheme.md+113−112

This diff is too big to show here.

docs/community-projects/nock-implementations/swift.md+319−317

This diff is too big to show here.

docs/community-projects/string-processing.md+8−6

This diff is too big to show here.

docs/community-projects/talkbot.md+13−4

This diff is too big to show here.

docs/community-projects/yint.md+13−4

This diff is too big to show here.

docs/hoon.md+3−3

This diff is too big to show here.

docs/hoon/advanced.md+159−140

This diff is too big to show here.

docs/hoon/basic.md+139−123

This diff is too big to show here.

docs/hoon/concepts.md+154−134

This diff is too big to show here.

docs/hoon/demo.md+62−43

This diff is too big to show here.

docs/hoon/examples.md+512−453

This diff is too big to show here.

docs/hoon/exercises.md+7−6

This diff is too big to show here.

docs/hoon/exercises/data-structures.md+40−30

This diff is too big to show here.

docs/hoon/exercises/fizzbuzz.md+28−25

This diff is too big to show here.

docs/hoon/exercises/life.md+110−94

This diff is too big to show here.

docs/hoon/exercises/lists.md+27−22

This diff is too big to show here.

docs/hoon/exercises/math.md+17−12

This diff is too big to show here.

docs/hoon/irregular.md+95−114

This diff is too big to show here.

docs/hoon/library.md+1−1

This diff is too big to show here.

docs/hoon/library/1a.md+123−65

This diff is too big to show here.

docs/hoon/library/1b.md+37−21

This diff is too big to show here.

docs/hoon/library/1c.md+103−51

This diff is too big to show here.

docs/hoon/library/2a.md+141−75

This diff is too big to show here.

docs/hoon/library/2b.md+347−191

This diff is too big to show here.

docs/hoon/library/2c.md+150−74

This diff is too big to show here.

docs/hoon/library/2d.md+49−25

This diff is too big to show here.

docs/hoon/library/2e.md+41−22

This diff is too big to show here.

docs/hoon/library/2f.md+63−29

This diff is too big to show here.

docs/hoon/library/2g.md+2−1

This diff is too big to show here.

docs/hoon/library/2h.md+152−77

This diff is too big to show here.

docs/hoon/library/2i.md+55−46

This diff is too big to show here.

docs/hoon/library/2j.md+24−17

This diff is too big to show here.

docs/hoon/library/2k.md+84−46

This diff is too big to show here.

docs/hoon/library/2l.md+2−1

This diff is too big to show here.

docs/hoon/library/2m.md+2−1

This diff is too big to show here.

docs/hoon/library/2n.md+54−26

This diff is too big to show here.

docs/hoon/library/2o.md+44−21

This diff is too big to show here.

docs/hoon/library/2p.md+52−30

This diff is too big to show here.

docs/hoon/library/2q.md+85−49

This diff is too big to show here.

docs/hoon/library/3a.md+205−116

This diff is too big to show here.

docs/hoon/library/3b.md+2−1

This diff is too big to show here.

docs/hoon/library/3c.md+167−99

This diff is too big to show here.

docs/hoon/library/3d.md+31−13

This diff is too big to show here.

docs/hoon/library/3e.md+8−4

This diff is too big to show here.

docs/hoon/library/3f.md+182−108

This diff is too big to show here.

docs/hoon/library/3g.md+96−49

This diff is too big to show here.

docs/hoon/library/4a.md+52−30

This diff is too big to show here.

docs/hoon/library/4b.md+190−97

This diff is too big to show here.

docs/hoon/library/4c.md+66−33

This diff is too big to show here.

docs/hoon/library/4d.md+27−14

This diff is too big to show here.

docs/hoon/library/4e.md+136−70

This diff is too big to show here.

docs/hoon/library/4f.md+286−148

This diff is too big to show here.

docs/hoon/library/4g.md+47−24

This diff is too big to show here.

docs/hoon/library/4h.md+210−110

This diff is too big to show here.

docs/hoon/library/4i.md+2−1

This diff is too big to show here.

docs/hoon/library/4j.md+545−310

This diff is too big to show here.

docs/hoon/library/4k.md+267−138

This diff is too big to show here.

docs/hoon/library/4l.md+157−75

This diff is too big to show here.

docs/hoon/library/4m.md+111−57

This diff is too big to show here.

docs/hoon/library/4n.md+97−54

This diff is too big to show here.

docs/hoon/library/4o.md+182−107

This diff is too big to show here.

docs/hoon/library/5a.md+2−1

This diff is too big to show here.

docs/hoon/library/5b.md+2−1

This diff is too big to show here.

docs/hoon/library/5c.md+2−1

This diff is too big to show here.

docs/hoon/library/5d.md+2−1

This diff is too big to show here.

docs/hoon/library/5e.md+2−1

This diff is too big to show here.

docs/hoon/library/5f.md+61−31

This diff is too big to show here.

docs/hoon/library/5g.md+1−1

This diff is too big to show here.

docs/hoon/library/zuse.md+1−1

This diff is too big to show here.

docs/hoon/library/zuse/core/crua.md+6−1

This diff is too big to show here.

docs/hoon/library/zuse/core/epur.md+213−107

This diff is too big to show here.

docs/hoon/library/zuse/core/fu.md+5−2

This diff is too big to show here.

docs/hoon/library/zuse/core/jo.md+239−131

This diff is too big to show here.

docs/hoon/library/zuse/core/poja.md+209−114

This diff is too big to show here.

docs/hoon/library/zuse/core/poxa.md+99−50

This diff is too big to show here.

docs/hoon/library/zuse/core/poxo.md+52−25

This diff is too big to show here.

docs/hoon/library/zuse/core/yu.md+35−23

This diff is too big to show here.

docs/hoon/library/zuse/diff.md+143−82

This diff is too big to show here.

docs/hoon/library/zuse/gate/clan.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/dawn.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/daws.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/deal.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/deft.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/dust.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/earl.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/earn.md+38−20

This diff is too big to show here.

docs/hoon/library/zuse/gate/fain.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/feel.md+14−6

This diff is too big to show here.

docs/hoon/library/zuse/gate/file.md+14−6

This diff is too big to show here.

docs/hoon/library/zuse/gate/foal.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/fray.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/fuel.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/furl.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/glam.md+10−4

This diff is too big to show here.

docs/hoon/library/zuse/gate/glon.md+10−4

This diff is too big to show here.

docs/hoon/library/zuse/gate/gnom.md+10−4

This diff is too big to show here.

docs/hoon/library/zuse/gate/jape.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/jesc.md+13−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/joba.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/jobe.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/jone.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/lead.md+14−6

This diff is too big to show here.

docs/hoon/library/zuse/gate/mang.md+15−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/meat.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/moon.md+14−6

This diff is too big to show here.

docs/hoon/library/zuse/gate/mung.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/perk.md+23−16

This diff is too big to show here.

docs/hoon/library/zuse/gate/pojo.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/saxo.md+13−6

This diff is too big to show here.

docs/hoon/library/zuse/gate/scanf.md+19−17

This diff is too big to show here.

docs/hoon/library/zuse/gate/sein.md+10−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/sifo.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/stud.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/taco.md+10−4

This diff is too big to show here.

docs/hoon/library/zuse/gate/tact.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/tame.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/tell.md+14−7

This diff is too big to show here.

docs/hoon/library/zuse/gate/tome.md+13−6

This diff is too big to show here.

docs/hoon/library/zuse/gate/tope.md+7−3

This diff is too big to show here.

docs/hoon/library/zuse/gate/txml.md+11−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/unt.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/urld.md+12−5

This diff is too big to show here.

docs/hoon/library/zuse/gate/urle.md+15−7

This diff is too big to show here.

docs/hoon/library/zuse/mold/udal.md+7−3

This diff is too big to show here.

docs/hoon/library/zuse/mold/udon.md+4−1

This diff is too big to show here.

docs/hoon/library/zuse/mold/umph.md+4−1

This diff is too big to show here.

docs/hoon/library/zuse/mold/unce.md+2−1

This diff is too big to show here.

docs/hoon/library/zuse/mold/upas.md+4−1

This diff is too big to show here.

docs/hoon/library/zuse/mold/urge.md+4−1

This diff is too big to show here.

docs/hoon/library/zuse/rsa.md+26−24

This diff is too big to show here.

docs/hoon/mission.md+34−28

This diff is too big to show here.

docs/hoon/reference.md+12−11

This diff is too big to show here.

docs/hoon/syntax.md+276−233

This diff is too big to show here.

docs/hoon/troubleshooting.md+120−97

This diff is too big to show here.

docs/hoon/twig.md+50−44

This diff is too big to show here.

docs/hoon/twig/atom.md+6−3

This diff is too big to show here.

docs/hoon/twig/atom/knit.md+17−9

This diff is too big to show here.

docs/hoon/twig/atom/path.md+11−9

This diff is too big to show here.

docs/hoon/twig/atom/rock.md+16−12

This diff is too big to show here.

docs/hoon/twig/atom/sand.md+43−40

This diff is too big to show here.

docs/hoon/twig/bar-core.md+16−14

This diff is too big to show here.

docs/hoon/twig/bar-core/cab-door.md+60−46

This diff is too big to show here.

docs/hoon/twig/bar-core/cen-core.md+32−26

This diff is too big to show here.

docs/hoon/twig/bar-core/col-gasp.md+12−7

This diff is too big to show here.

docs/hoon/twig/bar-core/dot-trap.md+54−41

This diff is too big to show here.

docs/hoon/twig/bar-core/hep-loop.md+40−27

This diff is too big to show here.

docs/hoon/twig/bar-core/ket-cork.md+36−26

This diff is too big to show here.

docs/hoon/twig/bar-core/sig-port.md+15−6

This diff is too big to show here.

docs/hoon/twig/bar-core/tar-gill.md+48−38

This diff is too big to show here.

docs/hoon/twig/bar-core/tis-gate.md+43−29

This diff is too big to show here.

docs/hoon/twig/bar-core/wut-tray.md+20−12

This diff is too big to show here.

docs/hoon/twig/buc-mold.md+39−34

This diff is too big to show here.

docs/hoon/twig/buc-mold/base.md+18−15

This diff is too big to show here.

docs/hoon/twig/buc-mold/cab-shoe.md+28−20

This diff is too big to show here.

docs/hoon/twig/buc-mold/cen-book.md+25−21

This diff is too big to show here.

docs/hoon/twig/buc-mold/col-bank.md+23−18

This diff is too big to show here.

docs/hoon/twig/buc-mold/hep-lamb.md+25−16

This diff is too big to show here.

docs/hoon/twig/buc-mold/ket-bush.md+26−20

This diff is too big to show here.

docs/hoon/twig/buc-mold/pat-claw.md+25−21

This diff is too big to show here.

docs/hoon/twig/buc-mold/tis-coat.md+23−14

This diff is too big to show here.

docs/hoon/twig/buc-mold/wut-pick.md+26−20

This diff is too big to show here.

docs/hoon/twig/cen-call.md+5−3

This diff is too big to show here.

docs/hoon/twig/cen-call/cab-keep.md+35−26

This diff is too big to show here.

docs/hoon/twig/cen-call/dot-lace.md+19−10

This diff is too big to show here.

docs/hoon/twig/cen-call/hep-call.md+33−22

This diff is too big to show here.

docs/hoon/twig/cen-call/ket-calq.md+19−11

This diff is too big to show here.

docs/hoon/twig/cen-call/lus-calt.md+19−11

This diff is too big to show here.

docs/hoon/twig/cen-call/sig-open.md+18−11

This diff is too big to show here.

docs/hoon/twig/cen-call/tar-bake.md+27−20

This diff is too big to show here.

docs/hoon/twig/cen-call/tis-make.md+26−22

This diff is too big to show here.

docs/hoon/twig/col-cell.md+5−3

This diff is too big to show here.

docs/hoon/twig/col-cell/cab-scon.md+15−7

This diff is too big to show here.

docs/hoon/twig/col-cell/hep-cons.md+17−15

This diff is too big to show here.

docs/hoon/twig/col-cell/ket-conq.md+26−18

This diff is too big to show here.

docs/hoon/twig/col-cell/lus-cont.md+24−16

This diff is too big to show here.

docs/hoon/twig/col-cell/sig-conl.md+45−35

This diff is too big to show here.

docs/hoon/twig/col-cell/tar-conp.md+56−48

This diff is too big to show here.

docs/hoon/twig/dot-nock.md+2−1

This diff is too big to show here.

docs/hoon/twig/dot-nock/ket-wish.md+6−6

This diff is too big to show here.

docs/hoon/twig/dot-nock/lus-bump.md+13−11

This diff is too big to show here.

docs/hoon/twig/dot-nock/tar-nock.md+33−29

This diff is too big to show here.

docs/hoon/twig/dot-nock/tis-same.md+19−15

This diff is too big to show here.

docs/hoon/twig/dot-nock/wut-deep.md+15−11

This diff is too big to show here.

docs/hoon/twig/ket-cast.md+6−5

This diff is too big to show here.

docs/hoon/twig/ket-cast/bar-iron.md+20−14

This diff is too big to show here.

docs/hoon/twig/ket-cast/hep-cast.md+40−31

This diff is too big to show here.

docs/hoon/twig/ket-cast/lus-like.md+11−7

This diff is too big to show here.

docs/hoon/twig/ket-cast/sig-burn.md+14−10

This diff is too big to show here.

docs/hoon/twig/ket-cast/tis-name.md+17−15

This diff is too big to show here.

docs/hoon/twig/ket-cast/wut-lead.md+20−13

This diff is too big to show here.

docs/hoon/twig/limb.md+11−10

This diff is too big to show here.

docs/hoon/twig/limb/limb.md+41−31

This diff is too big to show here.

docs/hoon/twig/limb/wing.md+21−18

This diff is too big to show here.

docs/hoon/twig/sem-make.md+2−1

This diff is too big to show here.

docs/hoon/twig/sem-make/col-wad.md+36−26

This diff is too big to show here.

docs/hoon/twig/sem-make/fas-nub.md+11−6

This diff is too big to show here.

docs/hoon/twig/sem-make/sem-fry.md+41−23

This diff is too big to show here.

docs/hoon/twig/sem-make/sig-dip.md+75−29

This diff is too big to show here.

docs/hoon/twig/sig-hint.md+4−2

This diff is too big to show here.

docs/hoon/twig/sig-hint/bar-show.md+12−8

This diff is too big to show here.

docs/hoon/twig/sig-hint/buc-poll.md+9−5

This diff is too big to show here.

docs/hoon/twig/sig-hint/cab-lurk.md+12−8

This diff is too big to show here.

docs/hoon/twig/sig-hint/cen-fast.md+45−38

This diff is too big to show here.

docs/hoon/twig/sig-hint/fas-funk.md+18−12

This diff is too big to show here.

docs/hoon/twig/sig-hint/gal-thin.md+21−16

This diff is too big to show here.

docs/hoon/twig/sig-hint/gar-hint.md+27−18

This diff is too big to show here.

docs/hoon/twig/sig-hint/lus-memo.md+15−9

This diff is too big to show here.

docs/hoon/twig/sig-hint/pam-dump.md+13−9

This diff is too big to show here.

docs/hoon/twig/sig-hint/tis-ddup.md+13−10

This diff is too big to show here.

docs/hoon/twig/sig-hint/wut-warn.md+17−13

This diff is too big to show here.

docs/hoon/twig/sig-hint/zap-peep.md+17−13

This diff is too big to show here.

docs/hoon/twig/tis-flow.md+17−15

This diff is too big to show here.

docs/hoon/twig/tis-flow/bar-new.md+30−21

This diff is too big to show here.

docs/hoon/twig/tis-flow/col-fix.md+25−17

This diff is too big to show here.

docs/hoon/twig/tis-flow/dot-set.md+30−22

This diff is too big to show here.

docs/hoon/twig/tis-flow/fas-var.md+52−35

This diff is too big to show here.

docs/hoon/twig/tis-flow/gal-rap.md+15−9

This diff is too big to show here.

docs/hoon/twig/tis-flow/gar-per.md+15−11

This diff is too big to show here.

docs/hoon/twig/tis-flow/hep-nip.md+27−19

This diff is too big to show here.

docs/hoon/twig/tis-flow/ket-sip.md+32−22

This diff is too big to show here.

docs/hoon/twig/tis-flow/lus-pin.md+25−17

This diff is too big to show here.

docs/hoon/twig/tis-flow/sem-rev.md+24−16

This diff is too big to show here.

docs/hoon/twig/tis-flow/sig-tow.md+31−27

This diff is too big to show here.

docs/hoon/twig/tis-flow/tar-aka.md+31−27

This diff is too big to show here.

docs/hoon/twig/wut-test.md+15−13

This diff is too big to show here.

docs/hoon/twig/wut-test/bar-or.md+30−20

This diff is too big to show here.

docs/hoon/twig/wut-test/col-if.md+23−17

This diff is too big to show here.

docs/hoon/twig/wut-test/dot-lest.md+37−29

This diff is too big to show here.

docs/hoon/twig/wut-test/gal-deny.md+18−9

This diff is too big to show here.

docs/hoon/twig/wut-test/gar-sure.md+17−9

This diff is too big to show here.

docs/hoon/twig/wut-test/hep-case.md+63−50

This diff is too big to show here.

docs/hoon/twig/wut-test/ket-ifcl.md+30−22

This diff is too big to show here.

docs/hoon/twig/wut-test/lus-deft.md+53−43

This diff is too big to show here.

docs/hoon/twig/wut-test/pam-and.md+29−19

This diff is too big to show here.

docs/hoon/twig/wut-test/pat-ifat.md+30−22

This diff is too big to show here.

docs/hoon/twig/wut-test/sig-ifno.md+26−17

This diff is too big to show here.

docs/hoon/twig/wut-test/tis-fits.md+23−17

This diff is too big to show here.

docs/hoon/twig/wut-test/zap-not.md+29−21

This diff is too big to show here.

docs/hoon/twig/zap-wild.md+2−1

This diff is too big to show here.

docs/hoon/twig/zap-wild/gar-wrap.md+11−7

This diff is too big to show here.

docs/hoon/twig/zap-wild/tis-code.md+18−14

This diff is too big to show here.

docs/hoon/twig/zap-wild/wut-need.md+20−16

This diff is too big to show here.

docs/hoon/twig/zap-wild/zap-fail.md+10−6

This diff is too big to show here.

docs/nock.md+2−3

This diff is too big to show here.

docs/nock/definition.md+119−105

This diff is too big to show here.

docs/nock/example.md+243−159

This diff is too big to show here.

docs/nock/explanation.md+386−265

This diff is too big to show here.

docs/nock/implementations.md+179−179

This diff is too big to show here.

docs/nock/implementations/c.md+178−176

This diff is too big to show here.

docs/nock/implementations/clojure.md+118−117

This diff is too big to show here.

docs/nock/implementations/csharp.md+379−377

This diff is too big to show here.

docs/nock/implementations/groovy.md+178−177

This diff is too big to show here.

docs/nock/implementations/haskell.md+57−56

This diff is too big to show here.

docs/nock/implementations/hoon.md+49−47

This diff is too big to show here.

docs/nock/implementations/javascript.md+357−355

This diff is too big to show here.

docs/nock/implementations/python.md+77−75

This diff is too big to show here.

docs/nock/implementations/ruby.md+131−130

This diff is too big to show here.

docs/nock/implementations/scala.md+69−68

This diff is too big to show here.

docs/nock/implementations/scheme.md+113−112

This diff is too big to show here.

docs/nock/implementations/swift.md+319−317

This diff is too big to show here.

docs/using.md+2−3

This diff is too big to show here.

docs/using/admin.md+21−49

This diff is too big to show here.

docs/using/filesystem.md+86−70

This diff is too big to show here.

docs/using/install.md+34−38

This diff is too big to show here.

docs/using/layout.md+19−32

This diff is too big to show here.

docs/using/messaging.md+72−61

This diff is too big to show here.

docs/using/setup.md+37−31

This diff is too big to show here.

docs/using/shell.md+48−38

This diff is too big to show here.

docs/using/urb.md+282−255

This diff is too big to show here.

docs/using/web.md+118−188

This diff is too big to show here.

docs/using/web/blog.md+32−27

This diff is too big to show here.

docs/using/web/comments.md+15−13

This diff is too big to show here.

docs/using/web/static.md+26−23

This diff is too big to show here.