on master help sign in
Merge pull request #1179 from tinnus-napbus/add-dist-docs

Add 3rd party app distribution docs

commit3bd0fc8188b404ecc09e6cbafe680e37cfa6f8ed authortinnus-napbus <77721746+tinnus-napbus@users.noreply.github.com> dateTue 5 Oct 2021, 23:14 (5 years ago) parentf36fdddabacd50
7 files changed+1163
content/docs/userspace/_index.md+3 content/docs/userspace/dist/_index.md+29 content/docs/userspace/dist/dist.md+105 content/docs/userspace/dist/docket.md+296 content/docs/userspace/dist/glob.md+102 content/docs/userspace/dist/guide.md+286 content/docs/userspace/dist/tools.md+342
content/docs/userspace/_index.md+3
@@ -28,3 +28,6 @@ Resources on threads - monadic functions useful for complex IO.
2828
2929Resources on Graph Store, a non-relational database specialized for social media applications.
3030
31+## [Distribution](/docs/userspace/dist/dist)
32+
33+Resources on distribution and management of desks/apps.
content/docs/userspace/dist/_index.mdnew file+29
@@ -0,0 +1,29 @@
1++++
2+title = "Distribution"
3+weight = 900
4+sort_by = "weight"
5+template = "sections/docs/chapters.html"
6+insert_anchor_links = "right"
7++++
8+
9+Developer documentation for desk/app distribution and management.
10+
11+## [Overview](/docs/userspace/dist/dist)
12+
13+An overview of desk/app distribution and management.
14+
15+## [Docket Files](/docs/userspace/dist/docket)
16+
17+Documentation of `desk.docket` files.
18+
19+## [Glob](/docs/userspace/dist/glob)
20+
21+Documentation of `glob`s (client bundles).
22+
23+## [Guide](/docs/userspace/dist/guide)
24+
25+A walkthrough of creating, installing and publishing a new desk with a tile and front-end.
26+
27+## [Dojo Tools](/docs/userspace/dist/tools)
28+
29+Documentation of useful generators for managing and distributing desks.
content/docs/userspace/dist/dist.mdnew file+105
@@ -0,0 +1,105 @@
1++++
2+title = "Overview"
3+weight = 1
4+template = "doc.html"
5++++
6+
7+Urbit allows peer-to-peer distribution and installation of applications. A user can click on a link to an app hosted by another ship to install that app. The homescreen interface lets users manage their installed apps and launch their interfaces in new tabs.
8+
9+This document describes the architecture of Urbit's app distribution system. For a walk-through of creating and distributing an app, see the [`Guide`](/docs/userspace/dist/guide) document.
10+
11+## Architecture
12+
13+The unit of software distribution is the desk. A desk is a lot like a git branch, but full of typed files, and designed to work with the Arvo kernel. In addition to files full of source code, a desk specifies the Kelvin version of the kernel that it's expecting to interact with, and it includes a manifest file describing which of the Gall agents it defines should be run by default.
14+
15+Every desk is self-contained: the result of validating its files and building its agents is a pure function of its contents and the code in the specified Kelvin version of the kernel. A desk on one ship will build into the same files and programs, noun for noun, as on any other ship.
16+
17+This symmetry is broken during agent installation, which can emit effects that might trigger other actions that cause the Arvo event to fail and be rolled back. An agent can ask the kernel to kill the Arvo event by using the new `%pyre` effect. Best practice, though, is for no desk to have a hard dependency on another desk.
18+
19+If you're publishing an app that expects another app to be installed in order to function, the best practice is to check in `+on-init` for the presence of that other app's desk. If it's not installed, your app should display a message to the user and a link to the app that they should install in order to support your app. App-install links are well-supported in Tlon's Landscape, a suite of user-facing applications developed by Tlon.
20+
21+For the moment, every live desk must have the same Kelvin version as the kernel. Future kernels that know how to maintain backward compatibility with older kernels will also allow older desks, but no commitment has yet been made to maintain backward compatibility across kernel versions, so for the time being, app developers should expect to update their apps accordingly.
22+
23+Each desk defines its own filetypes (called `mark`s), in its `/mar` folder. There are no longer shared system marks that all userspace code knows, nor common libraries in `/lib` or `/sur` — each desk is completely self-contained.
24+
25+It's common for a desk to want to use files that were originally defined in another desk, so that it can interact with agents on that desk. The convention is that if I'm publishing an app that I expect other devs to build client apps for (on other desks), I split out a "dev desk" containing just the external interface to my desk. Typically, both my app desk and clients' app desks will sync from this dev desk.
26+
27+Tlon has done this internally. Most desks will want to sync the `%base-dev` desk so they can easily interact with the kernel and system apps in the `%base` desk. The `%base` desk includes agents such as `%dojo` and `%hood` (with Kiln as an informal sub-agent of `%hood` that manages desk installations).
28+
29+A "landscape app", i.e. a desk that defines a tile that the user can launch from the home screen, should also sync from the `%garden-dev` desk. This desk includes the versioned `%docket-0` mark, which the app needs in order to include a `/desk/docket-0` file.
30+
31+The `%docket` agent reads the `/desk/docket-0` file to display an app tile on the home screen and hook up other front-end functionality, such as downloading the app's client bundle ([glob](/docs/userspace/dist/glob)). Docket is a new agent, in the `%garden` desk, that manages app installations. Docket serves the home screen, downloads client bundles, and communicates with Kiln to configure the apps on your system.
32+
33+For those of you familiar with the old `%glob` and `%file-server` agents, they have now been replaced by Docket.
34+
35+### Anatomy of a Desk
36+
37+Desks still contain helper files in `/lib` and `/sur`, generators in `/gen`, marks in `/mar`, threads in `/ted`, tests in `/tests`, and agents in `/app`. In addition, desks now also contain these files:
38+
39+```
40+/sys/kelvin :: Kernel kelvin, e.g. [%zuse 420]
41+/desk/bill :: (optional, read by Kiln) list of agents to run
42+/desk/docket-0 :: (optional, read by Docket) app metadata
43+/desk/ship :: (optional, read by Docket) ship of original desk publisher, e.g. ~zod
44+```
45+
46+Only the `%base` desk contains a `/sys` directory with the standard library, zuse, Arvo code and vanes. All other desks simply specify the kernel version with which they're compatible in the `/sys/kelvin` file.
47+
48+### Updates
49+
50+The main idea is that an app should only ever be run by a kernel that knows how to run it. For now, since there are not yet kernels that know how to run apps designed for an older kernel, this constraint boils down to ensuring that all live desks have the same kernel Kelvin version as the running kernel itself.
51+
52+To upgrade your kernel to a new version, you need to make a commit to the `%base` desk. Clay will then check if any files in `/sys` changed in this commit. If so, Clay sends the new commit to Arvo, which decides if it needs to upgrade (or upgrade parts of itself, such as a vane). After Arvo upgrades (or decides not to), it wakes up Clay, which finalizes the commit to the `%base` desk and notify the rest of the system.
53+
54+That's the basic flow for upgrading the kernel. However, some kernel updates also change the Kelvin version. If the user has also installed apps, those apps are designed to work with the old Kelvin, so they won't work with the new Kelvin — at least, not at the commit that's currently running.
55+
56+Kiln, part of the system app `%hood` in the `%base` desk, manages desk installations, including the `%base` desk. It can install an app in two ways: a local install, sourced from a desk on the user's machine, or a remote install, which downloads a desk from another ship. Both are performed using the same generator, `|install`.
57+
58+A remote install syncs an upstream desk into a local desk by performing a merge into the local desk whenever the upstream desk changes.
59+
60+The Kelvin update problem is especially thorny for remote installs, which are the most common. By default, a planet has its `%base` desk synced from its sponsor's `%kids` desk, and it will typically have app desks synced from their publishers' ships.
61+
62+Kiln listens (through Clay, which knows how to query remote Clays) for new commits on a remote-installed app's upstream ship and desk. When Clay hears about a new commit, it downloads the files and stores them as a "foreign desk", without validating or building them. It also tells Kiln.
63+
64+When Kiln learns of these new foreign files, it reads the new `/sys/kelvin`. If it's the same as the live kernel's, Kiln asks Clay to merge the new files into the local desk where the app is installed. If the new foreign Kelvin is further ahead (closer to zero) than the kernel's, Kiln does not merge it into the local desk yet. Instead, it enqueues it.
65+
66+Later, when Kiln hears of a new kernel Kelvin version on the upstream `%base` desk, it checks whether all the other live desks have a commit enqueued at that Kelvin. If so, it updates `%base` and then all the other desks, in one big Arvo event. This brings the system from fully at the old Kelvin, to fully at the new Kelvin, atomically — if any part of that fails, the Arvo event will abort and be rolled back, leaving the system back fully at the old Kelvin.
67+
68+If not all live desks have an enqueued commit at the new kernel Kelvin, then Kiln notifies its clients that a kernel update is blocked on a set of desks. Docket, listening to Kiln, presents the user with a choice: either dismiss the notification and keep the old kernel, or suspend the blocking desks and apply the kernel update.
69+
70+Suspending a desk turns off all its agents, saving their states in Gall. If there are no agents running from a desk, that desk doesn't force the kernel to be at the same Kelvin version. It's just inert data. If a later upstream update allows this desk to be run with a newer kernel, the user can revive the desk, and the system will migrate the old state into the new agent.
71+
72+### Managing Apps and Desks in Kiln
73+
74+Turning agents on and off is managed declaratively, rather than imperatively. Kiln maintains state for each desk about which agents should be forced on and which should be forced off. The set of running agents is now a function of the desk's `/desk/bill` manifest file and that user configuration state in Kiln. This means starting or stopping an agent is idempotent, both in Kiln and Gall.
75+
76+For details of the generators for managing desks and agents in Kiln, see the [`Dojo Tools`](/docs/userspace/dist/tools) document.
77+
78+### Landscape apps
79+
80+It's possible to create and distribute desks without a front-end, but typically you'll want to distribute an app with a user interface. Such an app has two primary components:
81+
82+- Gall agents and associated backend code which reside in the desk.
83+- A client bundle called a [`glob`](/docs/userspace/dist/glob), which contains the front-end files like HTML, CSS, JS, images, and so forth.
84+
85+When a desk is installed, Kiln will start up the Gall agents in the `desk.bill` manifest, and the `%docket` agent will read the `desk.docket-0` file. This file will specify the name of the app, various metadata, the appearance of the app's tile in the homescreen, and the source of the `glob` so it can serve the interface. For more details of the docket file, see the [Docket File](/docs/userspace/dist/docket) document.
86+
87+### Globs
88+
89+The reason to separate a glob from Clay is that Clay is a revision-controlled system. Like in most revision control systems, deleting data from it is nontrivial due to newer commits referencing old commits. If Clay grows the ability to delete data, perhaps glob data could be moved into it. Until then, since client bundles tend to be updated frequently, it's best practice not to put your glob in your app host ship's Clay at all to make sure it doesn't fill up your ship's "loom" memory arena.
90+
91+If the glob is to be served over Ames, there is an HTTP-based glob uploader that allows you to use a web form to upload a folder into your ship, which will convert the folder to a glob and link to it in your app desk's docket manifest file.
92+
93+Note that serving a glob over Ames might increase the install time for your app, since Ames is currently pretty slow compared to HTTP — but being able to serve a glob from your ship allows you to serve your whole app, both server-side and client-side, without setting up a CDN or any other external web tooling. Your ship can do it all on its own.
94+
95+For further details of globs, see the [Glob](/docs/userspace/dist/glob) document.
96+
97+## Sections
98+
99+- [Glob](/docs/userspace/dist/glob) - Documentation of `glob`s (client bundles).
100+
101+- [Docket Files](/docs/userspace/dist/docket) - Documentation of docket files.
102+
103+- [Guide](/docs/userspace/dist/guide) - A walkthrough of creating, installing and publishing a new desk with a tile and front-end.
104+
105+- [Dojo Tools](/docs/userspace/dist/tools) - Documentation of useful generators for managing and distributing desks.
content/docs/userspace/dist/docket.mdnew file+296
@@ -0,0 +1,296 @@
1++++
2+title = "Docket File"
3+weight = 2
4+template = "doc.html"
5++++
6+
7+The docket file sets various options for desks with a tile and (usually) a browser-based front-end of some kind. Mainly it configures the appearance of an app's tile, the source of its [glob](/docs/userspace/dist/glob), and some additional metadata.
8+
9+The docket file is read by the `%docket` agent when a desk is `|install`ed. The `%docket` agent will fetch the glob if applicable and create the tile as specified on the homescreen. If the desk is published with `:treaty|publish`, the information specified in the docket file will also be displayed for others who are browsing apps to install on your ship.
10+
11+The docket file is _optional_ in the general case. If it is omitted, however, the app cannot have a tile in the homescreen, nor can it be published with the `%treaty` agent, so others will not be able to browse for it from their homescreens.
12+
13+The docket file must be named `desk.docket-0`. The `%docket` `mark` is versioned to facilitate changes down the line, so the `-0` suffix may be incremented in the future.
14+
15+The file must contain a `hoon` list with a series of clauses. The clauses are defined in `/sur/docket.hoon` as:
16+
17+```hoon
18++$ clause
19+ $% [%title title=@t]
20+ [%info info=@t]
21+ [%color color=@ux]
22+ [%glob-http url=cord hash=@uvH]
23+ [%glob-ames =ship hash=@uvH]
24+ [%image =url]
25+ [%site =path]
26+ [%base base=term]
27+ [%version =version]
28+ [%website website=url]
29+ [%license license=cord]
30+ ==
31+```
32+
33+The `%image` clause is optional. It is mandatory to have exactly one of either `%site`, `%glob-http` or `%glob-ames`. All other clauses are mandatory.
34+
35+Here's what a typical docket file might look like:
36+
37+```hoon
38+:~
39+ title+'Foo'
40+ info+'An app that does a thing.'
41+ color+0xf9.8e40
42+ glob-ames+[~zod 0v0]
43+ image+'https://example.com/tile.svg'
44+ base+'foo'
45+ version+[0 0 1]
46+ license+'MIT'
47+ website+'https://example.com'
48+==
49+```
50+
51+Details of each clause and their purpose are described below.
52+
53+---
54+
55+## `%title`
56+
57+_required_
58+
59+The `%title` field specifies the name of the app. The title will be the name shown on the app's tile, as well as the name of the app when others search for it.
60+
61+#### Type
62+
63+```hoon
64+[%title title=@t]
65+```
66+
67+#### Example
68+
69+```hoon
70+title+'Bitcoin'
71+```
72+
73+---
74+
75+## `%info`
76+
77+_required_
78+
79+The `%info` field is a brief summary of what the app does. It will be shown as the subtitle in _App Info_.
80+
81+#### Type
82+
83+```hoon
84+[%info info=@t]
85+```
86+
87+#### Example
88+
89+```hoon
90+info+'A Bitcoin Wallet that lets you send and receive Bitcoin directly to and from other Urbit users'
91+```
92+
93+---
94+
95+## `%color`
96+
97+_required_
98+
99+The `%color` field specifies the color of the app tile as an `@ux`-formatted hex value.
100+
101+#### Type
102+
103+```hoon
104+[%color color=@ux]
105+```
106+
107+#### Example
108+
109+```hoon
110+color+0xf9.8e40
111+```
112+
113+---
114+
115+## `%glob-http`
116+
117+_exactly one of either this, [glob-ames](#glob-ames) or [site](#site) is required_
118+
119+The `%glob-http` field specifies the URL and hash of the app's [glob](/docs/userspace/distribution/glob) if it is distributed via HTTP.
120+
121+#### Type
122+
123+```hoon
124+[%glob-http url=cord hash=@uvH]
125+```
126+
127+#### Example
128+
129+```hoon
130+glob-http+['https://example.com/glob-0v1.s0me.h4sh.glob' 0v1.s0me.h4sh]
131+```
132+
133+---
134+
135+## `%glob-ames`
136+
137+_exactly one of either this, [glob-http](#glob-http) or [site](#site) is required_
138+
139+The `%glob-ames` field specifies the ship and hash of the app's [glob](/docs/userspace/distribution/glob) if it is distributed from a ship over Ames. If the glob will be distributed from our ship, the hash can initially be `0v0` as it will be overwritten with the hash produced by the [Globulator](/docs/userspace/distribution/glob#globulator).
140+
141+#### Type
142+
143+```hoon
144+[%glob-ames =ship hash=@uvH]
145+```
146+
147+#### Example
148+
149+```hoon
150+glob-ames+[~zod 0v0]
151+```
152+
153+---
154+
155+## `%site`
156+
157+_exactly one of either this, [glob-ames](#glob-ames) or [glob-http](#glob-http) is required_
158+
159+It's possible for an app to handle HTTP requests from the client directly rather than with a separate [glob](/docs/userspace/distribution/glob). In that case, the `%site` field specifies the `path` of the Eyre endpoint the app will bind. If `%site` is used, clicking the app's tile will simply open a new tab with a GET request to the specified Eyre endpoint.
160+
161+For more information on direct HTTP handling with a Gall agent or generator, see the [Eyre Internal API Reference](/docs/arvo/eyre/tasks) documentation.
162+
163+#### Type
164+
165+```hoon
166+[%site =path]
167+```
168+
169+#### Example
170+
171+```hoon
172+site+/foo/bar
173+```
174+
175+---
176+
177+## `%image`
178+
179+_optional_
180+
181+The `%image` field specifies the URL of an image to be displayed on the app's tile. This field is optional and may be omitted entirely.
182+
183+The given image will be displayed on top of the [color](#color)ed tile. The app [title](#title) (and hamburger menu upon hover) will be displayed on top of the given image, in small rounded boxes with the same background color as the main tile. The given image will be displayed at 100% of the width of the tile. The image's corners will be hidden by the rounded corners of the tile, so the image itself needn't have rounded corners. The tile is a perfect square, so if the image should occupy the whole tile, it should also be a perfect square. If the image should be a smaller icon in the center of the tile (like the bitcoin tile), it should just have a square of transparent negative space around it.
184+
185+It may be tempting to set the image URL as a root-relative path like `/apps/myapp/img/tile.svg` and bundle it in the glob. While this would work locally, it means the image would fail to load for those browsing apps to install. Therefore, the image should be hosted somewhere globally available.
186+
187+#### Type
188+
189+```hoon
190+[%image =url]
191+```
192+
193+The `url` type is a simple `cord`:
194+
195+```hoon
196++$ url cord
197+```
198+
199+#### Example
200+
201+```hoon
202+image+'http://example.com/icon.svg'
203+```
204+
205+---
206+
207+## `%base`
208+
209+_required_
210+
211+The `%base` field specifies the base of the URL path of the glob resources. In the browser, the path will begin with `/apps`, then the specified base, then the rest of the path to the particular glob resource like `http://localhost:8080/apps/my-base/index.html`. Note the `path`s of the glob contents themselves should not include this base element.
212+
213+#### Type
214+
215+```hoon
216+[%base base=term]
217+```
218+
219+#### Example
220+
221+```hoon
222+base+'bitcoin'
223+```
224+
225+---
226+
227+## `%version`
228+
229+_required_
230+
231+The `%version` field specifies the current version of the app. It's a triple of three `@ud` numbers representing the major version, minor version and patch version. In the client, `[1 2 3]` will be rendered as `1.2.3`. You would typically increase the appropriate number each time you published a change to the app.
232+
233+#### Type
234+
235+```hoon
236+[%version =version]
237+```
238+
239+The `version` type is just a triple of three numbers:
240+
241+```hoon
242++$ version
243+ [major=@ud minor=@ud patch=@ud]
244+```
245+
246+#### Example
247+
248+```hoon
249+version+[0 0 1]
250+```
251+
252+---
253+
254+## `%website`
255+
256+_required_
257+
258+The `%website` field is for a link to a relevant website. This might be a link to the app's github repo, company website, or whatever is appropriate. This field will be displayed when people are browsing apps to install.
259+
260+#### Type
261+
262+```hoon
263+[%website website=url]
264+```
265+
266+The `url` type is a simple `cord`:
267+
268+```hoon
269++$ url cord
270+```
271+
272+#### Example
273+
274+```hoon
275+website+'https://example.com'
276+```
277+
278+---
279+
280+## `%license`
281+
282+_required_
283+
284+The `%license` field specifies the license for the app in question. It would typically be a short name like `MIT`, `GPLv2`, or what have you. The field just takes a `cord` so any license can be specified.
285+
286+#### Type
287+
288+```hoon
289+[%license license=cord]
290+```
291+
292+#### Example
293+
294+```hoon
295+license+'MIT'
296+```
content/docs/userspace/dist/glob.mdnew file+102
@@ -0,0 +1,102 @@
1++++
2+title = "Glob"
3+weight = 3
4+template = "doc.html"
5++++
6+
7+A `glob` contains the client bundle—client-side resources like HTML, JS, and CSS files—for a landscape app distributed in a desk. Globs are managed separately from other files in desks because they often contain large files that frequently change, and would therefore bloat a ship's state if they were subject to Clay's revision control mechanisms.
8+
9+The hash and source of an app's glob is defined in a desk's [docket file](/docs/userspace/dist/docket). The `%docket` agent reads the docket file, obtains the glob from the specified source, and makes its contents available to the browser client. On a desk publisher's ship, if the glob is to be distributed over Ames, the glob is also made available to desk subscribers.
10+
11+## The `glob` type
12+
13+The `%docket`agent defines the type of a `glob` as:
14+
15+```hoon
16++$ glob (map path mime)
17+```
18+
19+Given the following file heirarchy:
20+
21+```
22+foo
23+├── css
24+│ └── style.css
25+├── img
26+│ ├── favicon.png
27+│ ├── foo.svg
28+│ └── bar.svg
29+├── index.html
30+└── js
31+ └── baz.js
32+```
33+
34+...its `$glob` form would look like:
35+
36+```hoon
37+{ [p=/img/foo/svg q=[p=/image/svg+xml q=[p=0 q=0]]]
38+ [p=/css/style/css q=[p=/text/css q=[p=0 q=0]]]
39+ [p=/img/favicon/png q=[p=/image/png q=[p=0 q=0]]]
40+ [p=/js/baz/js q=[p=/application/javascript q=[p=0 q=0]]]
41+ [p=/img/bar/svg q=[p=/image/svg+xml q=[p=0 q=0]]]
42+ [p=/index/html q=[p=/text/html q=[p=0 q=0]]]
43+}
44+```
45+
46+<sup>Note: The mime byte-length and data are 0 in this example because it was made with empty dummy files.</sup>
47+
48+A glob may contain any number of files and folders in any kind of heirarchy. The one important thing is that an `index.html` file is present in its root. The `index.html` file is automatically served when the app is opened in the browser and will fail if it is missing.
49+
50+In addition to the `$glob` type, a glob can also be output to Unix with a `.glob` file extension for distribution over HTTP. This file simply contains a [`jam`](/docs/hoon/reference/stdlib/2p#jam)med `$glob` structure.
51+
52+## Docket file clause
53+
54+The `desk.docket-0` file must include exactly one of the following clauses:
55+
56+#### `site+/some/path`
57+
58+If an app binds an Eyre endpoint and handles HTTP directly, for example with a [`%connect` task:eyre](/docs/arvo/eyre/tasks#connect), the `%site` clause is used, specifying the Eyre binding. In this case a glob is omitted entirely.
59+
60+#### `glob-ames+[~zod 0vs0me.h4sh]`
61+
62+If the glob is to be distributed over Ames, the `%glob-ames` clause is used, with a cell of the `ship` which has the glob and the `@uv` hash of the glob. If it's our ship, the hash can just be `0v0` and the glob can instead be created with the [Globulator](#globulator).
63+
64+#### `glob-http+['https://example.com/some.glob' 0vs0me.h4sh]`
65+
66+If the glob is to be distributed over HTTP, for example from an s3 instance, the `%glob-http` clause is used. It takes a cell of a `cord` with the URL serving the glob and the `@uv` hash of the glob.
67+
68+## Making a glob
69+
70+There are a couple of different methods depending on whether the glob will be distributed over HTTP or Ames.
71+
72+### Globulator
73+
74+For globs distributed over Ames from our ship, the client bundle can be uploaded directly with `%docket`'s Globulator tool, which is available in the browser at `http[s]://[host]/docket/upload`. It looks like this:
75+
76+![Globulator](https://media.urbit.org/docs/userspace/dist/globulator.png)
77+
78+Simply select the target desk, select the folder to be globulated, and hit `glob!`.
79+
80+Note the target desk must have been `|install`ed before uploading its glob. When installed, `%docket` will print `docket: awaiting manual glob for %desk-name desk` in the terminal and wait for the upload. The hash in the `%ames-glob` clause of the docket file will be overwritten by the hash of the new glob. As a result, there's no need to specify the actual glob hash in `desk.docket` - you can just use any `@uv` like `0v0`. Once uploaded, the desk can then be published with `:treaty|publish %desk-name` and the glob will become available for download by subscribers.
81+
82+### `-make-glob`
83+
84+There's a different process for globs to be distributed over HTTP from a webserver rather than over Ames from a ship. For this purpose, the `%garden` desk includes a `%make-glob` thread. The thread takes a folder in a desk and produces a glob of the files it contains, which it then saves to Unix in a [`jam`](/docs/hoon/reference/stdlib/2p#jam)file with a `.glob` extension.
85+
86+To begin, you'll need to spin up a ship (typically a fake ship) and `|mount` a desk for which to add the files. In order for Clay to add the files, the desk must contain `mark` files in its `/mar` directory for all file extensions your folder contains. The `%garden` desk is a good bet because it includes `mark` files for `.js`, `.html`, `.png`, `.svg`, `.woff2` and a couple of others. If there's no desk with a mark for a particular file type you want included in your glob, you may need to add a new mark file. A very rudimentary mark file like the `png.hoon` mark will suffice.
87+
88+With the desk mounted, add the folder to be globbed to the root of the desk in Unix. It's imporant it's in the root because the `%make-glob` thread will only strip the first level of the folder heirarchy.
89+
90+Next, `|commit` the files to the desk and change directory in the dojo to the `%garden` desk with `=dir /=garden=`. Then, run `-make-glob %the-desk /folder-name`, where `%the-desk` is the desk containing the folder to be globbed and `/folder-name` is its name.
91+
92+On Unix, if you look in `/path/to/pier/.urb/put`, you'll now see a file which looks like:
93+
94+```
95+glob-0v1.7vpqa.r8pn5.6t0s1.rhc7r.5e9vo.glob
96+```
97+
98+This file can be uploaded to your webserver and the `desk.docket-0` file of the desk you're publishing can be updated with:
99+
100+```hoon
101+glob-http+['https://s3.example.com/glob-0v1.7vpqa.r8pn5.6t0s1.rhc7r.5e9vo.glob' 0v1.7vpqa.r8pn5.6t0s1.rhc7r.5e9vo]
102+```
content/docs/userspace/dist/guide.mdnew file+286
@@ -0,0 +1,286 @@
1++++
2+title = "Guide"
3+weight = 4
4+template = "doc.html"
5++++
6+
7+In this document we'll walk through an example of creating and publishing a desk that others can install. We'll create a simple "Hello World!" front-end with a "Hello" tile to launch it. For simplicity, the desk won't include an actual Gall agent, but we'll note everything necessary if there were one.
8+
9+## Create desk
10+
11+To begin, we'll need to clone the [urbit git repo](https://github.com/urbit/urbit) from the Unix terminal:
12+
13+```
14+[user@host ~]$ git clone https://github.com/urbit/urbit urbit-git
15+```
16+
17+Once that's done, we can navigate to the `pkg` directory in our cloned repo:
18+
19+```
20+[user@host ~]$ cd urbit-git/pkg
21+[user@host pkg]$ ls .
22+arvo btc-wallet garden grid interface npm webterm
23+base-dev docker-image garden-dev herb landscape symbolic-merge.sh
24+bitcoin ent ge-additions hs libaes_siv urbit
25+```
26+
27+Each desk defines its own `mark`s, in its `/mar` folder. There are no longer shared system marks that all userspace code knows, nor common libraries in `/lib` or `/sur`. Each desk is completely self-contained. This means any new desk will need a number of base files.
28+
29+To make the creation of a new desk easier, `base-dev` and `garden-dev` contain symlinks to all `/sur`, `/lib` and `/mar` files necessary for interacting with the `%base` and `%garden` desks respectively. These dev desks can be copied and merged with the `symbolic-merge.sh` included.
30+
31+Let's create a new `hello` desk:
32+
33+```
34+[user@host pkg]$ mkdir hello
35+[user@host pkg]$ ./symbolic-merge.sh base-dev hello
36+[user@host pkg]$ ./symbolic-merge.sh garden-dev hello
37+[user@host pkg]$ cd hello
38+[user@host hello]$ ls
39+lib mar sur
40+```
41+
42+### `sys.kelvin`
43+
44+Our desk must include a `sys.kelvin` file which specifies the kernel version it's compatible with. Let's create that:
45+
46+```
47+[user@host hello]$ echo "[%zuse 420]" > sys.kelvin
48+[user@host hello]$ cat sys.kelvin
49+[%zuse 420]
50+```
51+
52+### `desk.ship`
53+
54+We can also add a `desk.ship` file to specify the original publisher of this desk. We'll try this on a fakezod so let's just add `~zod` as the publisher:
55+
56+```
57+[user@host hello]$ echo "~zod" > desk.ship
58+[user@host hello]$ cat desk.ship
59+~zod
60+```
61+
62+### `desk.bill`
63+
64+If we had Gall agents in this desk which should be automatically started when the desk is installed, we'd add them to a `hoon` list in the `desk.bill` file. It would look something like this:
65+
66+```hoon
67+:~ %some-app
68+ %another
69+==
70+```
71+
72+In this example we're not adding any agents, so we'll simply omit the `desk.bill` file.
73+
74+### `desk.docket-0`
75+
76+The final file we need is `desk.docket-0`. This one's more complicated, so we'll open it in our preferred text editor:
77+
78+```
79+[user@host hello]$ nano desk.docket-0
80+```
81+
82+In the text editor, we'll add the following:
83+
84+```hoon
85+:~ title+'Hello'
86+ info+'A simple hello world app.'
87+ color+0x81.88c9
88+ image+'https://media.urbit.org/docs/userspace/dist/wut.svg'
89+ base+'hello'
90+ glob-ames+[~zod 0v0]
91+ version+[0 0 1]
92+ website+'https://urbit.org/docs/userspace/dist/guide'
93+ license+'MIT'
94+==
95+```
96+
97+You can refer to the [Docket File](/docs/userspace/dist/docket) documentation for more details of what is required. In brief, the `desk.docket-0` file contains a `hoon` list of [clauses](/docs/userspace/dist/docket) which configure the appearance of the app tile, the source of the [glob](/docs/userspace/dist/glob), and some other metadata.
98+
99+We've given the app a [`%title`](/docs/userspace/dist/docket#title) of "Hello", which will be displayed on the app tile and will be the name of the app when others browse to install it. We've given the app tile a [`%color`](/docs/userspace/dist/docket#color) of `#8188C9`, and also specified the URL of an [`%image`](/docs/userspace/dist/docket#image) to display on the tile.
100+
101+The [`%base`](/docs/userspace/dist/docket#base) clause specifies the base URL path for the app. We've specified "hello" so it'll be `http://localhost:8080/apps/hello/...` in the browser. For the [glob](/docs/userspace/dist/glob), we've used a clause of [`%glob-ames`](/docs/userspace/dist/docket#glob-ames), which means the glob will be served from a ship over Ames, as opposed to being served over HTTP with a [`%glob-http`](/docs/userspace/dist/docket#glob-http) clause or having an Eyre binding with a [`%site`](/docs/userspace/dist/docket#site) clause. You can refer to the [glob](/docs/userspace/dist/glob) documentation for more details of the glob options. In our case we've specified `[~zod 0v0]`. Since `~zod` is the fakeship we'll install it on, the `%docket` agent will await a separate upload of the `glob`, so we can just specify `0v0` here as it'll get overwritten later.
102+
103+The [`%version`](/docs/userspace/dist/docket#version) clause specifies the version as a triple of major version, minor version and patch version. The rest is just some additional informative metadata which will be displayed in _App Info_.
104+
105+So let's save that to the `desk.docket-0` file and have a look at our desk:
106+
107+```
108+[user@host hello]$ ls
109+desk.docket-0 desk.ship lib mar sur sys.kelvin
110+```
111+
112+That's everything we need for now.
113+
114+## Install
115+
116+Let's spin up a fakezod in which we can install our desk:
117+
118+```
119+[user@host hello]$ cd ~/piers/fake
120+[user@host fake]$ urbit -F zod
121+```
122+
123+Once our fakezod is booted, we'll need to create a new `%hello` desk for our app and mount it. We can do this in the dojo like so:
124+
125+```
126+> |merge %hello our %base
127+>=
128+> |mount %hello
129+>=
130+```
131+
132+Now, back in the Unix terminal, we should see the new desk mounted:
133+
134+```
135+[user@host fake]$ cd zod
136+[user@host zod]$ ls
137+hello
138+```
139+
140+Currently it's just a clone of the `%base` desk, so let's delete its contents:
141+
142+```
143+[user@host zod]$ rm -r hello/*
144+```
145+
146+Next, we'll copy in the contents of the `hello` desk we created earlier. We must use `cp -LR` to resolve all the symlinks:
147+
148+```
149+[user@host zod]$ cp -LR ~/urbit-git/pkg/hello/* hello/
150+```
151+
152+Back in the dojo we can commit the changes and install the desk:
153+
154+```
155+> |commit %hello
156+> |install our %hello
157+kiln: installing %hello locally
158+docket: awaiting manual glob for %hello desk
159+```
160+
161+The `docket: awaiting manual glob for %hello desk` message is because our `desk.docket-0` file includes a [`%glob-ames`](/docs/userspace/dist/docket#glob-ames) clause which specifies our ship as the source, so it's waiting for us to upload the glob.
162+
163+If we open a browser now, navigate to `http://localhost:8080` and login with the default fakezod code `lidlut-tabwed-pillex-ridrup`, we'll see our tile's appeared but it says "installing" with a spinner due to the missing glob:
164+
165+![Installing Tile](https://media.urbit.org/docs/userspace/dist/local-install-1.png)
166+
167+## Create files for glob
168+
169+We'll now create the files for the glob. We'll use a very simple static HTML page that just displayes "Hello World!" and an image. Typically we'd have a more complex JS web app that talked to apps on our ship through Eyre's channel system, but for the sake of simplicity we'll forgo that. Let's hop back in the Unix terminal:
170+
171+```
172+[user@host zod]$ cd ~
173+[user@host ~]$ mkdir hello-glob
174+[user@host ~]$ cd hello-glob
175+[user@host hello-glob]$ mkdir img
176+[user@host hello-glob]$ wget -P img https://media.urbit.org/docs/userspace/dist/pot.svg
177+[user@host hello-glob]$ tree
178+.
179+└── img
180+ └── pot.svg
181+
182+1 directory, 1 file
183+```
184+
185+We've grabbed an image to use in our "Hello world!" page. The next thing we need to add is an `index.html` file in the root of the folder. The `index.html` file is mandatory; it's what will be loaded when the app's tile is clicked. Let's open our preferred editor and create it:
186+
187+```
188+[user@host hello-glob]$ nano index.html
189+```
190+
191+In the editor, paste in the following HTML and save it:
192+
193+```html
194+<!DOCTYPE html>
195+<html>
196+ <head>
197+ <style>
198+ div {
199+ text-align: center;
200+ }
201+ </style>
202+ </head>
203+ <title>Hello World</title>
204+ <body>
205+ <div>
206+ <h1>Hello World!</h1>
207+ <img src="img/pot.svg" alt="pot" width="219" height="196" />
208+ </div>
209+ </body>
210+</html>
211+```
212+
213+Our `hello-glob` folder should now look like this:
214+
215+```
216+[user@host hello-glob]$ tree
217+.
218+├── img
219+│ └── pot.svg
220+└── index.html
221+
222+1 directory, 2 files
223+```
224+
225+## Upload to glob
226+
227+We can now create a glob from the directory. To do so, navigate to `http://localhost:8080/docket/upload` in the browser. This will bring up the `%docket` app's [Globulator](/docs/userspace/dist/glob#globulator) tool:
228+
229+![Globulator](https://media.urbit.org/docs/userspace/dist/globulator.png)
230+
231+Simply select the `hello` desk from the drop-down, click `Choose file` and select the `hello-glob` folder in the the file browser, then hit `glob!`.
232+
233+Now if we return to our ship's homescreen, we should see the tile looks as we specified in the docket file:
234+
235+![Installed Tile](https://media.urbit.org/docs/userspace/dist/local-install-2.png)
236+
237+And if we click on the tile, it'll load the `index.html` in our glob:
238+
239+![Hello World!](https://media.urbit.org/docs/userspace/dist/local-install-3.png)
240+
241+Our app is working!
242+
243+## Publish
244+
245+The final step is publishing our desk with the `%treaty` agent so others can install it. To do this, there's a simple command in the dojo:
246+
247+```
248+> :treaty|publish %hello
249+>=
250+```
251+
252+Note: For desks without a docket file (and therefore without a tile and glob), treaty can't be used. Instead you can make the desk public with `|public %desk-name`.
253+
254+## Remote install
255+
256+Let's spin up another fake ship so we can try install it:
257+
258+```
259+[user@host hello-glob]$ cd ~/piers/fake
260+[user@host fake]$ urbit -F bus
261+```
262+
263+At the time of writing, we must manually request the published apps on a given ship from the dojo. This will likely change in the future but for now we must run the following on `~bus` before we can search for apps on `~zod` from the homescreen:
264+
265+```
266+> :treaty|ally ~zod
267+>=
268+```
269+
270+Note: For desks without a docket files (and therefore without a tile and glob), treaty can't be used. Instead remote users can install it from the dojo with `|install ~our-ship %desk-name`.
271+
272+Now in the browser, navigate to `http://localhost:8081` and login with `~bus`'s code `riddec-bicrym-ridlev-pocsef`. Next, type `~zod/` in the search bar, and it should pop up a list of `~zod`'s published apps, which in this case is our `Hello` app:
273+
274+![Remote install search](https://media.urbit.org/docs/userspace/dist/remote-install-1.png)
275+
276+When we click on the app, it'll show some of the information from the clauses in the docket file:
277+
278+![Remote app info](https://media.urbit.org/docs/userspace/dist/remote-install-2.png)
279+
280+Click `Get App` and it'll ask as if we want to install it:
281+
282+![Remote app install](https://media.urbit.org/docs/userspace/dist/remote-install-3.png)
283+
284+Finally, click `Get "Hello"` and it'll be installed as a tile on `~bus` which can then be opened:
285+
286+![Remote app finished](https://media.urbit.org/docs/userspace/dist/remote-install-4.png)
content/docs/userspace/dist/tools.mdnew file+342
@@ -0,0 +1,342 @@
1++++
2+title = "Dojo Tools"
3+weight = 5
4+template = "doc.html"
5++++
6+
7+A number of generators are included for managing and viewing the status of desks and their agents. Each of these are documented below.
8+
9+Note that some old generators have been replaced:
10+
11+- `|start` - Replaced with [`|rein`](#rein).
12+- `|ota` - Replaced with [`|install`](#install), [`|uninstall`](#uninstall), [`|pause`](#pause) and [`|resume`](#resume).
13+- `|fade` - Replaced with [`|nuke`](#nuke).
14+- `|doze` - Replaced with [`|rein`](#rein) or [`|suspend`](#suspend).
15+- `|wake` - Replaced with [`|rein`](#rein) or [`|revive`](#revive).
16+- `+trouble` - This still exists but is deprecated, and is now an alias for [`+vats`](#vats).
17+
18+---
19+
20+## `+vats`
21+
22+Print out the status of each installed desk.
23+
24+Desks in Clay which aren't installed will be omitted.
25+
26+Fields:
27+
28+- `/sys/kelvin` - The version of `%zuse` the desk is compatible with.
29+- `base hash` - The merge base (common ancestor) between the desk and its upstream source.
30+- `%cz hash` - The hash of the desk.
31+- `app status` - May be `suspended` or `running`.
32+- `force on` - The set of agents on the desk which have been manually started despite not being on the `desk.bill` manifest.
33+- `force off` - The set of agents on the desk which have been manually stopped despite being on the `desk.bill` manifest.
34+- `publishing ship` - The original publisher if the source ship is republishing the desk.
35+- `updates` - May be `local`, `tracking` or `paused`. Local means it will receive updates via commits on the local ship. Tracking means it will receive updates from the `source ship`. Paused means it will not receive updates.
36+- `source desk` - The desk on the `source ship`.
37+- `source aeon` - The revision number of the desk on the `source ship`.
38+- `pending updates` - Updates waiting to be applied due to incompatibility.
39+
40+#### Arguments
41+
42+None.
43+
44+#### Examples
45+
46+```
47+> +vats
48+%base
49+ /sys/kelvin: [%zuse 420]
50+ base hash: ~
51+ %cz hash: 0v6.2nqmu.oqm24.ighl6.n0gp9.s8res.feql1.dl8ap.isli3.jk0hu.acrd2
52+ app status: running
53+ force on: ~
54+ force off: ~
55+ publishing ship: ~
56+ updates: tracking
57+ source ship: ~zod
58+ source desk: %base
59+ source aeon: 3
60+ pending updates: ~
61+::
62+%garden
63+ /sys/kelvin: [%zuse 420]
64+ base hash: ~
65+ %cz hash: 0v1e.2h7hs.elq3g.1sdt7.qfga6.ganga.7p95j.aog44.8p5fe.kpr6v.7ai82
66+ app status: running
67+ force on: ~
68+ force off: ~
69+ publishing ship: ~
70+ updates: tracking
71+ source ship: ~zod
72+ source desk: %garden
73+ source aeon: 3
74+ pending updates: ~
75+```
76+
77+---
78+
79+## `+agents`
80+
81+Print out the status of Gall agents on a desk.
82+
83+Agents may either be `archived` or `running`. Nuked or unstarted agents which are not on the manifest are omitted.
84+
85+#### Arguments
86+
87+```
88+desk
89+```
90+
91+#### Example
92+
93+```
94+> +agents %garden
95+status: running %hark-system-hook
96+status: running %treaty
97+status: running %docket
98+status: running %settings-store
99+status: running %hark-store
100+```
101+
102+---
103+
104+## `|suspend`
105+
106+Shut down all agents on a desk, archiving their states.
107+
108+The tile in the homescreen (if it has one) will turn gray and say "Suspended" in the top-left corner. This generator does the same thing as selecting "Suspend" from an app tile's hamburger menu.
109+
110+#### Arguments
111+
112+```
113+desk
114+```
115+
116+#### Examples
117+
118+```
119+|suspend %bitcoin
120+```
121+
122+---
123+
124+## `|revive`
125+
126+Revive all agents on a desk, migrating archived states.
127+
128+All agents specified in `desk.bill` which are suspended will be restarted. If updates to the agents have occurred since their states were archived, they'll be migrated with the state transition functions in the agent. This generator does the same thing as selecting "Resume App" from the app tile's hamburger menu.
129+
130+#### Arguments
131+
132+```
133+desk
134+```
135+
136+#### Examples
137+
138+```
139+|revive %bitcoin
140+```
141+
142+---
143+
144+## `|install`
145+
146+Install a desk, starting its agents and listening for updates.
147+
148+If it's a remote desk we don't already have, it will be fetched. The agents started will be those specified in the `desk.bill` manifest. If it has a docket file, its tile will be added to the homescreen and its glob fetched. If we already have the desk, the source for updates will be switched to the ship specified.
149+
150+It takes a `ship` and `desk` as its mandatory arguments. The desk may be installed with a different name specified in the optional `local` argument.
151+
152+#### Arguments
153+
154+```
155+ship desk, =local desk
156+```
157+
158+#### Examples
159+
160+```
161+|install ~zod %bitcoin
162+```
163+
164+```
165+|install our %webterm
166+```
167+
168+```
169+|install ~zod %bitcoin, =local %foo
170+```
171+
172+---
173+
174+## `|uninstall`
175+
176+Uninstall a desk, suspending its agents and ignoring updates.
177+
178+The specified desk will be retained in Clay, but its agents will all be stopped and have their states archived like [`|suspend`](#suspend). This is the reverse of [`|install`](#install).
179+
180+Note that this will not remove the tile or glob from the homescreen, so if the desk has a tile it should be uninstalled with the "Remove App" button in the tile's hamburger menu.
181+
182+#### Arguments
183+
184+```
185+desk
186+```
187+
188+#### Examples
189+
190+```
191+|uninstall %bitcoin
192+```
193+
194+---
195+
196+## `|pause`
197+
198+Pause updates on a desk.
199+
200+The specified desk will stop tracking updates from its upstream source.
201+
202+#### Arguments
203+
204+```
205+desk
206+```
207+
208+#### Examples
209+
210+```
211+|pause %bitcoin
212+```
213+
214+---
215+
216+## `|resume`
217+
218+Resume updates on a desk.
219+
220+Start tracking previously [`|pause`](#pause)d updates from a desk's upstream source.
221+
222+#### Arguments
223+
224+```
225+desk
226+```
227+
228+#### Examples
229+
230+```
231+|resume %bitcoin
232+```
233+
234+---
235+
236+## `|nuke`
237+
238+Shut down an agent and permanently delete its state.
239+
240+The default behaviour is to shut down the specified Gall agent and discard its state, similar to the now-deprecated `|fade` generator. **Note this irreversibly wipes the app's state**. Additionally, if the optional `desk` argument is `%.y`, it takes a desk rather than an agent name and nukes every agent on the specified desk.
241+
242+#### Arguments
243+
244+```
245+@tas, =desk ?
246+```
247+
248+#### Examples
249+
250+Nuke a single agent:
251+
252+```
253+|nuke %btc-wallet
254+```
255+
256+Nuke every agent on a desk:
257+
258+```
259+|nuke %bitcoin, =desk &
260+```
261+
262+---
263+
264+## `|bump`
265+
266+Try to apply a kernel update.
267+
268+If `%zuse`'s kelvin version has decreased, try to apply the update. If any desks are incompatible with the new `%zuse` version, the update will fail. The update can be forced with the optional `force` argument, in which case incompatible desks will be `|suspend`ed. The `set` of desks to omit (and therefore suspend) can also be explicitly specified with the optional `except` argument.
269+
270+#### Arguments
271+
272+All optional:
273+
274+```
275+, =force ?, =except (set desk)
276+```
277+
278+#### Examples
279+
280+Try to apply update, failing if blocked by any desks:
281+
282+```
283+|bump
284+```
285+
286+Force the update, suspending incompatible desks:
287+
288+```
289+|bump, =force &
290+```
291+
292+Apply kernel update, omitting and suspending the specified desks:
293+
294+```
295+|bump, =except (silt ~[%bitcoin %webterm])
296+```
297+
298+---
299+
300+## `|rein`
301+
302+Adjust the state of a desk.
303+
304+This allows you to stop and start agents on a desk, regardless of whether they're on the manifest. Stopped agents have their states archived. It can also suspend and revive the whole desk with the optional `liv` argument.
305+
306+#### Arguments
307+
308+```
309+desk (list [? dude:gall]), =liv ?
310+```
311+
312+#### Examples
313+
314+Start an agent:
315+
316+```
317+|rein %bitcoin [& %btc-provider]
318+```
319+
320+Stop an agent, archiving its state:
321+
322+```
323+|rein %bitcoin [| %btc-wallet]
324+```
325+
326+Stop one agent, start another:
327+
328+```
329+|rein %bitcoin [| %btc-wallet] [& %btc-provider]
330+```
331+
332+Suspend a desk:
333+
334+```
335+|rein %bitcoin, =liv |
336+```
337+
338+Revive a desk:
339+
340+```
341+|rein %bitcoin
342+```