@@ -1,3 +1,5 @@
1+<img src="desk/web/shelf-icon.svg" width="96" height="96" alt="Shelf logo">
2+
13# %shelf
24
35A public web page that lists the apps your Urbit ship publishes, so anyone can
@@ -7,29 +9,42 @@ The page shows your ship's `%treaty` alliance: your own published desks plus
79any desks you republish from other ships. Each app has an Install button that
810opens it in the visitor's own Landscape, and the Dojo `|install` command as a
911fallback. The page follows the visitor's light or dark preference. It has no
10−JavaScript except for the Copy buttons.
12+JavaScript except for the Copy buttons. The same URL also returns the list as
13+JSON.
1114
1215%shelf answers on exactly one path, `/shelf`. Run it behind a reverse proxy
1316that serves it at the root of a dedicated domain (see
14−[Serving it on its own domain](#serving-it-on-its-own-domain)). All of the
15−page's links point at `/`.
17+[Serving it on its own domain](#serving-it-on-its-own-domain)). The page's
18+links and its install form all point at `/`.
1619
1720## Requirements
1821
1922- Kernel `[%zuse 408]`.
20−- `%landscape`, which provides `%treaty`. Fresh ships have it.
23+- `%landscape`, which provides `%treaty` and `%docket`. Fresh ships have it.
24+- Nix, for the one-command build, if you install from source. Without it,
25+ copy `desk/` from a checkout.
2126
2227## Install
2328
24−Build the desk with Nix. `result` is then the desk folder:
29+Install %shelf from ~donmec-sallud, which publishes it:
30+
31+```
32+~sampel-palnet:dojo> |install ~donmec-sallud %shelf
33+```
34+
35+%shelf then gets its updates from ~donmec-sallud.
36+[shelf.iko.soy](https://shelf.iko.soy) is %shelf running on that ship. It
37+lists all the apps ~donmec-sallud offers, %shelf among them.
38+
39+To install from source instead, build the desk:
2540
2641```sh
2742nix build github:iko-soy/shelf
2843```
2944
30−From a checkout, `nix build` does the same, or you can use `desk/` directly.
31−
32−Then, with your ship running:
45+`result` is now the desk folder (see [Building with Nix](#building-with-nix)).
46+Without Nix, use `desk/` from a checkout in its place. Then, with your ship
47+running:
3348
3449```
3550~sampel-palnet:dojo> |new-desk %shelf
@@ -46,18 +61,59 @@ chmod -R u+w /path/to/pier/shelf # files from the Nix store are read-only
4661~sampel-palnet:dojo> |install our %shelf
4762```
4863
49−The page is now at `https://<your-ship>/shelf`, and Landscape shows a Shelf
50−tile that opens it. It is meant to be opened through your own domain, though:
51−on the ship's address the page renders, but its links lead to `/`, which
52−belongs to Landscape.
64+Either way, once %shelf runs, the page is at `https://<your-ship>/shelf`, and
65+Landscape shows a Shelf tile that opens it. It is meant to be opened through
66+your own domain, though: on the ship's address the page renders, but its links
67+lead to `/`, which belongs to Landscape.
68+
69+## Building with Nix
70+
71+`flake.nix` has one output: the desk, `packages.<system>.desk`, which is also
72+the default package. Building it copies `desk/` into the Nix store; nothing is
73+compiled, because Clay compiles the Hoon on your ship when you commit the desk.
74+
75+| Command | Builds |
76+|---|---|
77+| `nix build github:iko-soy/shelf` | the latest commit on the default branch |
78+| `nix build github:iko-soy/shelf/<commit>` | a specific commit |
79+| `nix build` | your checkout, from the repo root |
80+
81+Each one leaves a `result` symlink to the desk in the store:
82+
83+```
84+result/
85+├── desk.bill desk.docket-0 sys.kelvin
86+├── app/ gen/ lib/ mar/ sur/ tests/
87+└── web/
88+```
89+
90+- Files in the store are read-only, so `chmod -R u+w` the mount after copying
91+ them in: the ship has to be able to write to its own mount.
92+- `nix flake check --all-systems` checks the flake on every system it
93+ supports: `x86_64-linux`, `aarch64-linux` and `aarch64-darwin`. Plain
94+ `nix flake check` checks only your own. Current nixpkgs no longer supports
95+ Intel Macs.
96+- nixpkgs is only used for `runCommandLocal`. `nix flake update` moves its
97+ pin in `flake.lock`.
98+- From another flake, for example a NixOS config that sets up a ship, the desk
99+ is:
100+
101+ ```nix
102+ inputs.shelf.url = "github:iko-soy/shelf";
103+ # …
104+ shelf.packages.${system}.desk
105+ ```
53106
54107## Update
55108
56−Build and copy the desk into the mount again, as above, and run
57−`|commit %shelf`. Then check that `.^(cass:clay %cw /=shelf=)` moved: if the
58−new version doesn't build, Clay keeps the old one and logs nothing.
109+A %shelf installed from ~donmec-sallud gets its updates from that ship.
59110
60−%shelf is installed from your own ship, so it gets no over-the-air updates.
111+To update a source install, build and copy the desk into the mount again, as
112+above, and run `|commit %shelf`. Then check that `.^(cass:clay %cw /=shelf=)`
113+moved: if the new version doesn't build, Clay keeps the old one and logs
114+nothing.
115+
116+A source install comes from your own ship, so it gets no over-the-air updates.
61117After a kernel upgrade, Clay holds it until `desk/sys.kelvin` lists the new
62118`%zuse` version; update that file (or pull a release that has) and commit.
63119`+vats %shelf` shows whether it is running or held.
@@ -76,7 +132,9 @@ before you can publish it.
76132 `:treaty|unally ~other-ship`.
77133
78134%shelf has a docket of its own, so `:treaty|publish %shelf` lists it on its
79−own page, where others can install it.
135+own page, where others can install it. The docket carries the app icon
136+inline, as a URL-encoded `data:` URL, so the icon shows in Landscape and on
137+other ships' pages with nothing to host.
80138
81139A republished app appears once your ship has received its docket from the
82140publisher, which happens a few seconds after you ally them. Apps are sorted by
@@ -103,7 +161,7 @@ apps.example.com {
103161- `redir` sends every path except `/` back to `/`, so nothing else on your
104162 ship can be reached through this domain.
105163- `rewrite` replaces only the path, so the query string the page relies on
106− (`/?install=…`) reaches the ship intact.
164+ (`/?install=…`, `/?asset=…`) reaches the ship intact.
107165- `Forwarded` tells Eyre the visitor came over HTTPS. Eyre trusts it only
108166 from `127.0.0.1`, hence the IP instead of `localhost`. Without it, a ship
109167 that also serves its own TLS redirects every proxied request to its HTTPS
@@ -113,6 +171,10 @@ apps.example.com {
113171 another proxy, pass `Host` through (nginx: `proxy_set_header Host $host;`)
114172 and send `Forwarded` (or `X-Forwarded-Proto`) the same way.
115173- Your ship's own domain is unaffected: `/` still goes to Landscape.
174+- %shelf serves everything the page needs from the desk, at the same URL: the
175+ CSS and JavaScript inline, and the favicon at `/?asset=favicon.svg`. It
176+ also serves the app icon at `/?asset=shelf-icon.svg`, though the docket
177+ carries that inline. Nothing else needs to be reachable.
116178
117179This config was tested with Caddy 2.11 in front of a fake ship.
118180
@@ -155,12 +217,15 @@ curl -H 'Accept: application/json' https://apps.example.com/
155217- `image` is `null` when the docket has none, and `website` is `""`.
156218- `image` and `website` are the docket's values as they are. Dockets can come
157219 from other ships, and unlike the page, the JSON doesn't filter these, so
158− check for `http(s)` before using them.
220+ check them before using them. `image` may also be an inline
221+ `data:image/…` URL.
222+- Text that isn't valid UTF-8 comes out with U+FFFD in its place, so the
223+ listing is always valid JSON.
159224- Entries are sorted like the page. Key order within an object is not fixed.
160225- JSON wins when the header ranks `application/json` above `text/html`. When
161226 the two rank equally, the more specific match wins, so
162227 `application/json, */*` gets JSON. A full tie, a bare `*/*` or a missing
163− header gets HTML.
228+ header gets HTML. Repeated `Accept` lines count as one list.
164229- The response has `Access-Control-Allow-Origin: *`, varies on `Accept` and
165230 `Origin`, and is `private, max-age=60`. It is private because Eyre attaches
166231 its session cookie to every response, and on your own logged-in requests
@@ -172,10 +237,10 @@ curl -H 'Accept: application/json' https://apps.example.com/
172237 URL (for example `sampel-palnet.tlon.network`) and shows the Dojo command.
1732382. The form is POSTed. %shelf checks the URL and keeps only its scheme, host
174239 and port. It remembers that origin in a cookie for a year, then redirects
175− to
176− `<ship>/apps/landscape/search/direct/apps/~publisher/desk`. That is
240+ to `<ship>/apps/landscape/search/direct/apps/~publisher/desk`. That is
177241 Landscape's own "Get App" dialog. The visitor's ship fetches the app's
178− details directly from the publisher.
242+ details directly from the publisher, and a visitor who isn't logged in
243+ lands there after their ship's login page.
1792443. On later visits, Install links straight there. The button reads
180245 `Install on ~sampel-palnet` when the URL's first label is a planet name
181246 (or a moon or comet), or any ship name on `arvo.network`, `tlon.network`
@@ -191,8 +256,9 @@ Guards:
191256 docket image URL pointing at your domain, can plant a URL in a visitor's
192257 cookie or bounce them elsewhere through your domain.
193258- Only apps actually on the shelf can be installed through the form.
194−- On the page, image and website URLs from dockets are used only if they are
195− `http(s)`. Dockets can come from other ships.
259+- On the page, docket images are used only if they are `http(s)` or
260+ `data:image/…` URLs, and website links only if they are `http(s)`.
261+ Dockets can come from other ships.
196262
197263## Traffic
198264
@@ -223,6 +289,9 @@ after %shelf stops. Until you remove it, `/shelf` answers 503 at once.
223289 Landscape. This removes the tile and suspends the agent; a plain
224290 `|uninstall %shelf` would leave a suspended tile behind. Your `head`/`body`
225291 HTML is kept, and `|install our %shelf` brings the page straight back.
292+ Uninstalling a %shelf installed from ~donmec-sallud also stops its updates.
293+ To turn them back on, run `|install ~donmec-sallud %shelf` after
294+ `|install our %shelf`.
2262953. To delete that state too: `|nuke %shelf, =desk &` (answer `y`).
2272964. Optionally, free the path. Only the binding's owner can remove it, so take
228297 it over from the dojo, then remove it:
@@ -235,22 +304,116 @@ after %shelf stops. Until you remove it, `/shelf` answers 503 at once.
235304 Installing %shelf again binds `/shelf` again. %shelf re-binds every time
236305 it loads, so it also restores the binding if someone removed it by hand.
237306
307+## Changes from the design handoffs
308+
309+%shelf follows two design handoffs: the page itself (layout, copy, behaviour,
310+JSON, a skeleton agent) and a later icon update. Rendered with the mock's sample
311+data, the page matches the design mock pixel for pixel in light and dark mode,
312+apart from the icons and names. Everything below differs from the handoffs,
313+and why.
314+
315+**Serving**
316+- **One path, `/shelf`, instead of `/`.** On most ships Landscape's `%docket`
317+ owns `/`, and Eyre doesn't refuse a second claim: it silently hands `/`
318+ over, which would take away the owner's login page. A dedicated domain
319+ mapped onto `/shelf` gives visitors a clean root URL instead.
320+- **One URL for everything.** Because only `/` reaches the ship through that
321+ domain, the CSS and JavaScript are inlined (not `/shelf.css`, `/copy.js`),
322+ the icons are at `/?asset=…` (not `/favicon.svg`, `/shelf-icon.svg`), the
323+ install form posts to `/` (not `/install`), and Change ship is `/?forget`
324+ (not `/forget`). Other paths are the proxy's job; the ship answers them
325+ with 404.
326+- **JSON by `Accept` header at the same URL, not `/apps.json`, and
327+ `private`.** The single URL rules out `/apps.json`, and `public` caching
328+ could hand an owner's login cookie to a shared cache.
329+- **HTML varies on `Cookie` as well as `Accept`.** Otherwise a browser shows
330+ its cached copy for up to a minute after Change ship or a save.
331+- **Bodies go out in 16 KiB pieces**, because a single large response gets
332+ quadratically slower on its way out of the ship.
333+
334+**Installing**
335+- **The install form is a POST, accepted only from the page itself, and only
336+ for apps on the shelf.** A GET could be triggered by a link or by another
337+ publisher's docket image, planting a phishing URL in visitors' cookies.
338+- **Deep link:** `<ship>/apps/landscape/search/direct/apps/~pub/desk`,
339+ Landscape's own "show install" route, tested end to end. The handoff's
340+ `…/leap/search/…` also works, but only through a redirect.
341+- **Saved ship URLs keep only scheme, host and port**, so a pasted
342+ `…/apps/groups` doesn't break the deep link. `localhost` and `127.*`
343+ default to `http://`, and ports must be 1–65535.
344+- **The cookie holds the URL as is (not URL-encoded), `HttpOnly`, and gets
345+ `Secure` when the visitor came over HTTPS**, as `Forwarded` or
346+ `X-Forwarded-Proto` says behind a proxy. The normalized URL only contains
347+ cookie-safe characters.
348+- **"Install on ~ship" needs a planet or longer name, or a known ship host.**
349+ The handoff's rule (any first DNS label that parses as an `@p`) turned
350+ `dev.example.com` into "Install on ~dev".
351+- **Returning visitors see "Installs open on ~ship. Change ship"** in the
352+ masthead, a muted link that clears the cookie. The handoff left this open.
353+
354+**Reading the alliance**
355+- **Own desks' treaties are built from Clay**, from each desk's
356+ `desk.docket-0`, as `%treaty` does. `%treaty`'s `/treaty/<ship>/<desk>`
357+ scry only covers other ships' desks and crashes for your own.
358+- **Only scries that can't fail.** The handoff wrapped scries in `mule`, but
359+ inside an agent `mule` can't catch a failed scry: the request just fails.
360+- **Visitors arrive under an Eyre guest identity**, so the owner check
361+ applies only to the settings pokes. Checking every request would 500 for
362+ all visitors.
363+- **Docket text from other ships is handled defensively:** images only
364+ `http(s)` or `data:image/…`, links only `http(s)`, invalid UTF-8 replaced
365+ in the JSON, and a title that starts with a control character still
366+ renders.
367+
368+**Page**
369+- **Header and rows share one grid (CSS subgrid)**, so columns stay aligned
370+ when Install labels get wider for a saved ship.
371+- **Narrow screens:** with a saved ship the stacked layout starts at 1040px
372+ instead of 860px, and the install button's text wraps; the URL field gets
373+ its own line; long words wrap instead of overflowing.
374+- **Keyboard focus inside a row tints it**, as the handoff's interaction table
375+ says (its CSS only did this on hover).
376+- **"1 app"**, not "1 apps".
377+- **Republished apps say "from ~publisher"**, not "by ~publisher".
378+- **The sigil is drawn in Hoon** by a vendored renderer, in `currentColor` on
379+ `var(--ink)` so it inverts with the theme, at sigil-js's 56px geometry.
380+ Moons and comets get the plain square, as in Landscape.
381+- **New: owner-set raw HTML in `<head>` and `<body>`**, for analytics.
382+
383+**Icons and docket** (the icon update)
384+- **The docket's `image` is the icon inline, as a URL-encoded `data:` URL**,
385+ instead of an absolute URL on the ship, which would need filling in per
386+ ship and would break wherever that ship can't be reached.
387+- **The tile opens `/shelf`** (`site+/shelf`, not `site+/`), and the docket's
388+ website is this repo.
389+- **The docket's description drops "with one-click install"**: first-time
390+ visitors enter their ship's URL before the Install button takes them there.
391+- **`mar/svg` is the upstream mark from `%base`**, not the handoff's new one.
392+- **Icons are served `private`** for the same login-cookie reason as the
393+ JSON.
394+
395+**Other**
396+- **Kernel `[%zuse 408]`**, the current release, replacing the handoff's
397+ placeholder 409.
398+
238399## Development
239400
240401The desk is self-contained. `desk/` holds everything Clay needs:
241402
242403| Path | What |
243404|---|---|
244−| `desk.docket-0` | What Landscape shows: title, description, color, version, website, license |
405+| `desk.docket-0` | What Landscape shows: title, description, color, icon, version, website, license. Its `image` is `web/shelf-icon.svg` URL-encoded; a test checks they match |
245406| `app/shelf.hoon` | The agent: Eyre binding, routing, scries, responses |
246407| `lib/shelf.hoon` | Pure helpers and the page (Sail) |
247408| `sur/shelf.hoon`, `mar/shelf/action.hoon`, `gen/shelf/` | The `head`/`body` settings |
248409| `web/shelf.css`, `web/copy.js` | Inlined into the page at build time with `/*` |
410+| `web/shelf-icon.svg`, `web/favicon.svg` | The app icon (also inlined in the docket) and the favicon, served at `?asset=` |
249411| `tests/lib/shelf.hoon` | Unit tests, including the rendered page |
250412| `lib/sigil.hoon`, `lib/sigil/symbols.hoon` | Vendored [sigil renderer](https://github.com/Fang-/suite) (MIT, ~paldev; shapes from `@urbit/sigil-js`, MIT, Tlon) |
251−| other `lib/`, `mar/`, `sur/` files | Copied unmodified from `%base` (urbit/urbit `5a187fe`) and `%landscape` (`sur/treaty`, `sur/docket`, `lib/docket`, `mar/docket-0`; tloncorp/landscape `46c6ed8`) |
413+| other `lib/`, `mar/`, `sur/` files | Copied unmodified from `%base` (urbit/urbit `5a187fe`, including `mar/svg`) and `%landscape` (`sur/treaty`, `sur/docket`, `lib/docket`, `mar/docket-0`; tloncorp/landscape `46c6ed8`) |
252414
253−`flake.nix` makes `nix build` produce the desk as `result`.
415+Outside the desk, `flake.nix` and `flake.lock` define the Nix build (see
416+[Building with Nix](#building-with-nix)).
254417
255418Run the tests on a ship with the desk installed:
256419