@@ -1,3 +1,5 @@
1+<img src="desk/web/shelf-icon.svg" width="96" height="96" alt="Shelf logo">
2+
13# %shelf
24
35A public web page that lists the apps your Urbit ship publishes, so anyone can
@@ -7,21 +9,42 @@ The page shows your ship's `%treaty` alliance: your own published desks plus
79any desks you republish from other ships. Each app has an Install button that
810opens it in the visitor's own Landscape, and the Dojo `|install` command as a
911fallback. The page follows the visitor's light or dark preference. It has no
10−JavaScript except for the Copy buttons.
12+JavaScript except for the Copy buttons. The same URL also returns the list as
13+JSON.
1114
1215%shelf answers on exactly one path, `/shelf`. Run it behind a reverse proxy
1316that serves it at the root of a dedicated domain (see
14−[Serving it on its own domain](#serving-it-on-its-own-domain)). All of the
15−page's links point at `/`.
17+[Serving it on its own domain](#serving-it-on-its-own-domain)). The page's
18+links and its install form all point at `/`.
1619
1720## Requirements
1821
1922- Kernel `[%zuse 408]`.
20−- `%landscape`, which provides `%treaty`. Fresh ships have it.
23+- `%landscape`, which provides `%treaty` and `%docket`. Fresh ships have it.
24+- Nix, for the one-command build, if you install from source. Without it,
25+ copy `desk/` from a checkout.
2126
2227## Install
2328
24−From the repo root, with your ship running:
29+Install %shelf from ~donmec-sallud, which publishes it:
30+
31+```
32+~sampel-palnet:dojo> |install ~donmec-sallud %shelf
33+```
34+
35+%shelf then gets its updates from ~donmec-sallud.
36+[shelf.iko.soy](https://shelf.iko.soy) is %shelf running on that ship. It
37+lists all the apps ~donmec-sallud offers, %shelf among them.
38+
39+To install from source instead, build the desk:
40+
41+```sh
42+nix build github:iko-soy/shelf
43+```
44+
45+`result` is now the desk folder (see [Building with Nix](#building-with-nix)).
46+Without Nix, use `desk/` from a checkout in its place. Then, with your ship
47+running:
2548
2649```
2750~sampel-palnet:dojo> |new-desk %shelf
@@ -29,7 +52,8 @@ From the repo root, with your ship running:
2952```
3053
3154```sh
32−cp -R desk/* /path/to/pier/shelf/
55+cp -R result/* /path/to/pier/shelf/
56+chmod -R u+w /path/to/pier/shelf # files from the Nix store are read-only
3357```
3458
3559```
@@ -37,17 +61,59 @@ cp -R desk/* /path/to/pier/shelf/
3761~sampel-palnet:dojo> |install our %shelf
3862```
3963
40−The page is now at `https://<your-ship>/shelf`. It is meant to be opened
41−through your own domain, though: on the ship's address the page renders, but
42−its links lead to `/`, which belongs to Landscape.
64+Either way, once %shelf runs, the page is at `https://<your-ship>/shelf`, and
65+Landscape shows a Shelf tile that opens it. It is meant to be opened through
66+your own domain, though: on the ship's address the page renders, but its links
67+lead to `/`, which belongs to Landscape.
68+
69+## Building with Nix
70+
71+`flake.nix` has one output: the desk, `packages.<system>.desk`, which is also
72+the default package. Building it copies `desk/` into the Nix store; nothing is
73+compiled, because Clay compiles the Hoon on your ship when you commit the desk.
74+
75+| Command | Builds |
76+|---|---|
77+| `nix build github:iko-soy/shelf` | the latest commit on the default branch |
78+| `nix build github:iko-soy/shelf/<commit>` | a specific commit |
79+| `nix build` | your checkout, from the repo root |
80+
81+Each one leaves a `result` symlink to the desk in the store:
82+
83+```
84+result/
85+├── desk.bill desk.docket-0 sys.kelvin
86+├── app/ gen/ lib/ mar/ sur/ tests/
87+└── web/
88+```
89+
90+- Files in the store are read-only, so `chmod -R u+w` the mount after copying
91+ them in: the ship has to be able to write to its own mount.
92+- `nix flake check --all-systems` checks the flake on every system it
93+ supports: `x86_64-linux`, `aarch64-linux` and `aarch64-darwin`. Plain
94+ `nix flake check` checks only your own. Current nixpkgs no longer supports
95+ Intel Macs.
96+- nixpkgs is only used for `runCommandLocal`. `nix flake update` moves its
97+ pin in `flake.lock`.
98+- From another flake, for example a NixOS config that sets up a ship, the desk
99+ is:
100+
101+ ```nix
102+ inputs.shelf.url = "github:iko-soy/shelf";
103+ # …
104+ shelf.packages.${system}.desk
105+ ```
43106
44107## Update
45108
46−Copy `desk/*` into the mount again and run `|commit %shelf`. Then check that
47−`.^(cass:clay %cw /=shelf=)` moved: if the new version doesn't build, Clay
48−keeps the old one and logs nothing.
109+A %shelf installed from ~donmec-sallud gets its updates from that ship.
49110
50−%shelf is installed from your own ship, so it gets no over-the-air updates.
111+To update a source install, build and copy the desk into the mount again, as
112+above, and run `|commit %shelf`. Then check that `.^(cass:clay %cw /=shelf=)`
113+moved: if the new version doesn't build, Clay keeps the old one and logs
114+nothing.
115+
116+A source install comes from your own ship, so it gets no over-the-air updates.
51117After a kernel upgrade, Clay holds it until `desk/sys.kelvin` lists the new
52118`%zuse` version; update that file (or pull a release that has) and commit.
53119`+vats %shelf` shows whether it is running or held.
@@ -65,6 +131,11 @@ before you can publish it.
65131 `:treaty|remove-desk ~other-ship %app`, and stop following with
66132 `:treaty|unally ~other-ship`.
67133
134+%shelf has a docket of its own, so `:treaty|publish %shelf` lists it on its
135+own page, where others can install it. The docket carries the app icon
136+inline, as a URL-encoded `data:` URL, so the icon shows in Landscape and on
137+other ships' pages with nothing to host.
138+
68139A republished app appears once your ship has received its docket from the
69140publisher, which happens a few seconds after you ally them. Apps are sorted by
70141title. %shelf reads the alliance on every request, but browsers may show a
@@ -90,7 +161,7 @@ apps.example.com {
90161- `redir` sends every path except `/` back to `/`, so nothing else on your
91162 ship can be reached through this domain.
92163- `rewrite` replaces only the path, so the query string the page relies on
93− (`/?install=…`) reaches the ship intact.
164+ (`/?install=…`, `/?asset=…`) reaches the ship intact.
94165- `Forwarded` tells Eyre the visitor came over HTTPS. Eyre trusts it only
95166 from `127.0.0.1`, hence the IP instead of `localhost`. Without it, a ship
96167 that also serves its own TLS redirects every proxied request to its HTTPS
@@ -100,6 +171,10 @@ apps.example.com {
100171 another proxy, pass `Host` through (nginx: `proxy_set_header Host $host;`)
101172 and send `Forwarded` (or `X-Forwarded-Proto`) the same way.
102173- Your ship's own domain is unaffected: `/` still goes to Landscape.
174+- %shelf serves everything the page needs from the desk, at the same URL: the
175+ CSS and JavaScript inline, and the favicon at `/?asset=favicon.svg`. It
176+ also serves the app icon at `/?asset=shelf-icon.svg`, though the docket
177+ carries that inline. Nothing else needs to be reachable.
103178
104179This config was tested with Caddy 2.11 in front of a fake ship.
105180
@@ -142,12 +217,15 @@ curl -H 'Accept: application/json' https://apps.example.com/
142217- `image` is `null` when the docket has none, and `website` is `""`.
143218- `image` and `website` are the docket's values as they are. Dockets can come
144219 from other ships, and unlike the page, the JSON doesn't filter these, so
145− check for `http(s)` before using them.
220+ check them before using them. `image` may also be an inline
221+ `data:image/…` URL.
222+- Text that isn't valid UTF-8 comes out with U+FFFD in its place, so the
223+ listing is always valid JSON.
146224- Entries are sorted like the page. Key order within an object is not fixed.
147225- JSON wins when the header ranks `application/json` above `text/html`. When
148226 the two rank equally, the more specific match wins, so
149227 `application/json, */*` gets JSON. A full tie, a bare `*/*` or a missing
150− header gets HTML.
228+ header gets HTML. Repeated `Accept` lines count as one list.
151229- The response has `Access-Control-Allow-Origin: *`, varies on `Accept` and
152230 `Origin`, and is `private, max-age=60`. It is private because Eyre attaches
153231 its session cookie to every response, and on your own logged-in requests
@@ -159,10 +237,10 @@ curl -H 'Accept: application/json' https://apps.example.com/
159237 URL (for example `sampel-palnet.tlon.network`) and shows the Dojo command.
1602382. The form is POSTed. %shelf checks the URL and keeps only its scheme, host
161239 and port. It remembers that origin in a cookie for a year, then redirects
162− to
163− `<ship>/apps/landscape/search/direct/apps/~publisher/desk`. That is
240+ to `<ship>/apps/landscape/search/direct/apps/~publisher/desk`. That is
164241 Landscape's own "Get App" dialog. The visitor's ship fetches the app's
165− details directly from the publisher.
242+ details directly from the publisher, and a visitor who isn't logged in
243+ lands there after their ship's login page.
1662443. On later visits, Install links straight there. The button reads
167245 `Install on ~sampel-palnet` when the URL's first label is a planet name
168246 (or a moon or comet), or any ship name on `arvo.network`, `tlon.network`
@@ -178,8 +256,9 @@ Guards:
178256 docket image URL pointing at your domain, can plant a URL in a visitor's
179257 cookie or bounce them elsewhere through your domain.
180258- Only apps actually on the shelf can be installed through the form.
181−- On the page, image and website URLs from dockets are used only if they are
182− `http(s)`. Dockets can come from other ships.
259+- On the page, docket images are used only if they are `http(s)` or
260+ `data:image/…` URLs, and website links only if they are `http(s)`.
261+ Dockets can come from other ships.
183262
184263## Traffic
185264
@@ -206,8 +285,13 @@ after %shelf stops. Until you remove it, `/shelf` answers 503 at once.
2062851. Remove the site from your reverse proxy first. Otherwise visitors get that
207286 503 page, or, once the binding is gone, an endless redirect loop between
208287 Landscape's login page and `/`.
209−2. `|uninstall %shelf`. Your `head`/`body` HTML is kept, and installing again
210− brings the page straight back.
288+2. `:docket|uninstall %shelf`, or Uninstall from the tile's menu in
289+ Landscape. This removes the tile and suspends the agent; a plain
290+ `|uninstall %shelf` would leave a suspended tile behind. Your `head`/`body`
291+ HTML is kept, and `|install our %shelf` brings the page straight back.
292+ Uninstalling a %shelf installed from ~donmec-sallud also stops its updates.
293+ To turn them back on, run `|install ~donmec-sallud %shelf` after
294+ `|install our %shelf`.
2112953. To delete that state too: `|nuke %shelf, =desk &` (answer `y`).
2122964. Optionally, free the path. Only the binding's owner can remove it, so take
213297 it over from the dojo, then remove it:
@@ -220,19 +304,116 @@ after %shelf stops. Until you remove it, `/shelf` answers 503 at once.
220304 Installing %shelf again binds `/shelf` again. %shelf re-binds every time
221305 it loads, so it also restores the binding if someone removed it by hand.
222306
307+## Changes from the design handoffs
308+
309+%shelf follows two design handoffs: the page itself (layout, copy, behaviour,
310+JSON, a skeleton agent) and a later icon update. Rendered with the mock's sample
311+data, the page matches the design mock pixel for pixel in light and dark mode,
312+apart from the icons and names. Everything below differs from the handoffs,
313+and why.
314+
315+**Serving**
316+- **One path, `/shelf`, instead of `/`.** On most ships Landscape's `%docket`
317+ owns `/`, and Eyre doesn't refuse a second claim: it silently hands `/`
318+ over, which would take away the owner's login page. A dedicated domain
319+ mapped onto `/shelf` gives visitors a clean root URL instead.
320+- **One URL for everything.** Because only `/` reaches the ship through that
321+ domain, the CSS and JavaScript are inlined (not `/shelf.css`, `/copy.js`),
322+ the icons are at `/?asset=…` (not `/favicon.svg`, `/shelf-icon.svg`), the
323+ install form posts to `/` (not `/install`), and Change ship is `/?forget`
324+ (not `/forget`). Other paths are the proxy's job; the ship answers them
325+ with 404.
326+- **JSON by `Accept` header at the same URL, not `/apps.json`, and
327+ `private`.** The single URL rules out `/apps.json`, and `public` caching
328+ could hand an owner's login cookie to a shared cache.
329+- **HTML varies on `Cookie` as well as `Accept`.** Otherwise a browser shows
330+ its cached copy for up to a minute after Change ship or a save.
331+- **Bodies go out in 16 KiB pieces**, because a single large response gets
332+ quadratically slower on its way out of the ship.
333+
334+**Installing**
335+- **The install form is a POST, accepted only from the page itself, and only
336+ for apps on the shelf.** A GET could be triggered by a link or by another
337+ publisher's docket image, planting a phishing URL in visitors' cookies.
338+- **Deep link:** `<ship>/apps/landscape/search/direct/apps/~pub/desk`,
339+ Landscape's own "show install" route, tested end to end. The handoff's
340+ `…/leap/search/…` also works, but only through a redirect.
341+- **Saved ship URLs keep only scheme, host and port**, so a pasted
342+ `…/apps/groups` doesn't break the deep link. `localhost` and `127.*`
343+ default to `http://`, and ports must be 1–65535.
344+- **The cookie holds the URL as is (not URL-encoded), `HttpOnly`, and gets
345+ `Secure` when the visitor came over HTTPS**, as `Forwarded` or
346+ `X-Forwarded-Proto` says behind a proxy. The normalized URL only contains
347+ cookie-safe characters.
348+- **"Install on ~ship" needs a planet or longer name, or a known ship host.**
349+ The handoff's rule (any first DNS label that parses as an `@p`) turned
350+ `dev.example.com` into "Install on ~dev".
351+- **Returning visitors see "Installs open on ~ship. Change ship"** in the
352+ masthead, a muted link that clears the cookie. The handoff left this open.
353+
354+**Reading the alliance**
355+- **Own desks' treaties are built from Clay**, from each desk's
356+ `desk.docket-0`, as `%treaty` does. `%treaty`'s `/treaty/<ship>/<desk>`
357+ scry only covers other ships' desks and crashes for your own.
358+- **Only scries that can't fail.** The handoff wrapped scries in `mule`, but
359+ inside an agent `mule` can't catch a failed scry: the request just fails.
360+- **Visitors arrive under an Eyre guest identity**, so the owner check
361+ applies only to the settings pokes. Checking every request would 500 for
362+ all visitors.
363+- **Docket text from other ships is handled defensively:** images only
364+ `http(s)` or `data:image/…`, links only `http(s)`, invalid UTF-8 replaced
365+ in the JSON, and a title that starts with a control character still
366+ renders.
367+
368+**Page**
369+- **Header and rows share one grid (CSS subgrid)**, so columns stay aligned
370+ when Install labels get wider for a saved ship.
371+- **Narrow screens:** with a saved ship the stacked layout starts at 1040px
372+ instead of 860px, and the install button's text wraps; the URL field gets
373+ its own line; long words wrap instead of overflowing.
374+- **Keyboard focus inside a row tints it**, as the handoff's interaction table
375+ says (its CSS only did this on hover).
376+- **"1 app"**, not "1 apps".
377+- **Republished apps say "from ~publisher"**, not "by ~publisher".
378+- **The sigil is drawn in Hoon** by a vendored renderer, in `currentColor` on
379+ `var(--ink)` so it inverts with the theme, at sigil-js's 56px geometry.
380+ Moons and comets get the plain square, as in Landscape.
381+- **New: owner-set raw HTML in `<head>` and `<body>`**, for analytics.
382+
383+**Icons and docket** (the icon update)
384+- **The docket's `image` is the icon inline, as a URL-encoded `data:` URL**,
385+ instead of an absolute URL on the ship, which would need filling in per
386+ ship and would break wherever that ship can't be reached.
387+- **The tile opens `/shelf`** (`site+/shelf`, not `site+/`), and the docket's
388+ website is this repo.
389+- **The docket's description drops "with one-click install"**: first-time
390+ visitors enter their ship's URL before the Install button takes them there.
391+- **`mar/svg` is the upstream mark from `%base`**, not the handoff's new one.
392+- **Icons are served `private`** for the same login-cookie reason as the
393+ JSON.
394+
395+**Other**
396+- **Kernel `[%zuse 408]`**, the current release, replacing the handoff's
397+ placeholder 409.
398+
223399## Development
224400
225401The desk is self-contained. `desk/` holds everything Clay needs:
226402
227403| Path | What |
228404|---|---|
405+| `desk.docket-0` | What Landscape shows: title, description, color, icon, version, website, license. Its `image` is `web/shelf-icon.svg` URL-encoded; a test checks they match |
229406| `app/shelf.hoon` | The agent: Eyre binding, routing, scries, responses |
230407| `lib/shelf.hoon` | Pure helpers and the page (Sail) |
231408| `sur/shelf.hoon`, `mar/shelf/action.hoon`, `gen/shelf/` | The `head`/`body` settings |
232409| `web/shelf.css`, `web/copy.js` | Inlined into the page at build time with `/*` |
410+| `web/shelf-icon.svg`, `web/favicon.svg` | The app icon (also inlined in the docket) and the favicon, served at `?asset=` |
233411| `tests/lib/shelf.hoon` | Unit tests, including the rendered page |
234412| `lib/sigil.hoon`, `lib/sigil/symbols.hoon` | Vendored [sigil renderer](https://github.com/Fang-/suite) (MIT, ~paldev; shapes from `@urbit/sigil-js`, MIT, Tlon) |
235−| other `lib/`, `mar/`, `sur/` files | Copied unmodified from `%base` (urbit/urbit `5a187fe`) and `%landscape` (`sur/treaty`, `sur/docket`; tloncorp/landscape `46c6ed8`) |
413+| other `lib/`, `mar/`, `sur/` files | Copied unmodified from `%base` (urbit/urbit `5a187fe`, including `mar/svg`) and `%landscape` (`sur/treaty`, `sur/docket`, `lib/docket`, `mar/docket-0`; tloncorp/landscape `46c6ed8`) |
414+
415+Outside the desk, `flake.nix` and `flake.lock` define the Nix build (see
416+[Building with Nix](#building-with-nix)).
236417
237418Run the tests on a ship with the desk installed:
238419
@@ -254,3 +435,10 @@ Some details that are easy to get wrong:
254435- **Clay can refuse a commit silently.** It refuses commits that would stop a
255436 running agent from building, and logs nothing. After `|commit`, check that
256437 `.^(cass:clay %cw /=shelf=)` moved.
438+
439+## License
440+
441+MIT; see [LICENSE](LICENSE). The vendored files keep their own notices:
442+`lib/sigil.hoon` and `lib/sigil/symbols.hoon` are MIT (Fang; shapes from
443+`@urbit/sigil-js`, Tlon), and the files from `%base` and `%landscape` come
444+from urbit/urbit (MIT) and tloncorp/landscape.