@@ -4,22 +4,24 @@
5`urbit-apps` is a Nix flake that builds Urbit apps' desks into `.deskjam`
5`urbit-apps` is a Nix flake that builds Urbit apps' desks into `.deskjam`
6files, the upload format of [deskjam](https://github.com/iko-soy/deskjam),
6files, the upload format of [deskjam](https://github.com/iko-soy/deskjam),
7−with each app's frontend glob next to it. `jammer/` is the Rust CLI that packs
7+with its glob next to each one, for an app whose frontend is one. `jammer/`
8−them. `README.md` documents the apps for users: read it before changing what
8+is the Rust CLI that packs them. `README.md` documents the apps for users:
9−the flake builds.
9+read it before changing what the flake builds.
11## Code map
11## Code map
13−- `flake.nix`: one input per app, plus any dependencies an app's own build
13+- `flake.nix`: nixpkgs, flake-utils, one input per app, and the commits
14− pins. Each app is an `<app>-desk` built with `mkDeskjam` and exposed as
14+ harness' build pins. Each app is an `<app>-desk` built with `mkDeskjam` and
15− `packages.<system>.<app>`; frontends are `packages.<system>.<app>-glob`.
15+ exposed as `packages.<system>.<app>`. Globs are `packages.<system>.<app>-glob`,
16+ for hits, silo and vita. jammer is `packages.<system>.jammer` and `default`.
16 - `mkDeskjam { desk, src, glob ? null }` packs the desk folder `src` into
17 - `mkDeskjam { desk, src, glob ? null }` packs the desk folder `src` into
17 `$out/<desk>.deskjam` and copies `glob` next to it as `$out/<desk>-glob`.
18 `$out/<desk>.deskjam` and copies `glob` next to it as `$out/<desk>-glob`.
18 - `names-glob` reads a `desk.docket-0` with `urbit eval`, as the docket mark
19 - `names-glob` reads a `desk.docket-0` with `urbit eval`, as the docket mark
19 does. `mkDeskjam` fails the build when the docket names a glob (a
20 does. `mkDeskjam` fails the build when the docket names a glob (a
20 `glob-http` or `glob-ames` clause, and no `site` clause) but has no
21 `glob-http` or `glob-ames` clause, and no `site` clause) but has no
21− `glob`, when it has a `glob` for a docket that names none, and when the
22+ `glob`, when it has a `glob` for a docket that names none, and when Hoon
22− docket isn't valid Hoon.
23+ can't read the docket as a list of clauses. It doesn't check that the
24+ docket has the clauses the mark needs, such as `title`.
23 - `mkGlob` builds a frontend with `buildNpmPackage` (`npm run build`, then
25 - `mkGlob` builds a frontend with `buildNpmPackage` (`npm run build`, then
24 its `dist/`), for apps whose own flakes don't build their globs.
26 its `dist/`), for apps whose own flakes don't build their globs.
25 - `harness-git` and `harness-src` run harness' own `zig build` offline; see
27 - `harness-git` and `harness-src` run harness' own `zig build` offline; see
@@ -33,24 +35,33 @@ the flake builds.
33## Working conventions
35## Working conventions
35- Use jj, not git, and pass `--git` to `jj diff` and `jj show`. Commit or push
37- Use jj, not git, and pass `--git` to `jj diff` and `jj show`. Commit or push
36− only when asked. Nix only sees files jj tracks. Never write to `/nix` by
38+ only when asked. Never write to `/nix` by hand.
37− hand.
39+- Nix only sees files jj tracks, and jj tracks a new file at its next
40+ command. After adding a file, run `jj status` before building, not
42+- jammer writes `<desk>.deskjam` or `<desk>/` in the current directory, and jj
43+ would track them in this repo. Run it elsewhere, or give `jam` and `unjam`
44+ an `-o` outside the repo; `nix shell .` puts it on your PATH.
38- Don't change the app repos from here. When one needs a change, write a
45- Don't change the app repos from here. When one needs a change, write a
39 prompt the user can give an agent in that repo: the problem, with evidence,
46 prompt the user can give an agent in that repo: the problem, with evidence,
40 the change, and how to check it.
47 the change, and how to check it.
41- Add an app by the first route its repo allows:
48- Add an app by the first route its repo allows:
42− 1. Its flake follows the convention: `packages.<system>.desk` is the full
49+ 1. Its flake builds the full desk, as `packages.<system>.desk`, or as
43− desk, `.glob` is the built frontend if its docket names a glob, and
44− `default` is `desk`. Add it as a flake input, with
50+ `default` if it has no `desk`, like vita's. Add it as a flake input, with
45− `src = app.packages.${system}.desk` and
51+ `src = app.packages.${system}.desk`, or `.default`. If its docket names a
46− `glob = app.packages.${system}.glob`. Have its `nixpkgs` follow this
52+ glob and its flake builds it, use `glob = app.packages.${system}.glob`;
53+ if not, build it here with `mkGlob`, as for hits. Have its `nixpkgs`
47− flake's only if its flake builds no frontend; otherwise keep the nixpkgs
54+ follow this flake's only if its flake builds no frontend; otherwise keep
48− it pins, so its glob is the same build as in its own repo.
55+ the nixpkgs it pins, so its glob is the same build as in its own repo.
49 2. It has no flake, and its `desk/` is the whole desk: a `flake = false`
56 2. It has no flake, and its `desk/` is the whole desk: a `flake = false`
50− input, with `src = "${app}/desk"`.
57+ input, with `src = "${app}/desk"`. If its docket names a glob, build it
58+ with `mkGlob`, starting its `npmDepsHash` at `pkgs.lib.fakeHash`.
51 3. Its own build tool assembles the desk: run that build in Nix, with its
59 3. Its own build tool assembles the desk: run that build in Nix, with its
52− fetches pinned as inputs, as for harness. Don't reimplement its steps,
60+ fetches pinned as inputs, as for harness. Don't reimplement its steps.
61+ Replace only a step that can't run in the sandbox, as `flake.nix` does
62+ for harness' npm steps, with `--replace-fail`, so a change upstream
53− and make the build fail loudly when upstream pins something new.
63+ stops the build. Make the build fail loudly when upstream pins
54- Before adding an app, check:
65- Before adding an app, check:
55 - that its `sys.kelvin` lists the current kernel, `[%zuse 408]`. Kernel
66 - that its `sys.kelvin` lists the current kernel, `[%zuse 408]`. Kernel
56 numbers count down, so 411 is older than 408. Don't package an app that
67 numbers count down, so 411 is older than 408. Don't package an app that
@@ -58,23 +69,35 @@ the flake builds.
58 - its desk name, from its README or its docket's `base`.
69 - its desk name, from its README or its docket's `base`.
59 - whether its docket names a glob, and where else its frontend could come
70 - whether its docket names a glob, and where else its frontend could come
60 from, such as files its own agent serves from the desk.
71 from, such as files its own agent serves from the desk.
61− - that `jammer jam -v` reports no missing marks, and that the desk has the
72+ - that `jammer jam -v` reports no missing marks, and that deskjam's confirm
62− libraries its Hoon imports.
73+ page, on a fake ship, finds nothing missing that its agents and marks
74+ need. Its warning that other files import files the desk lacks, as for
75+ silo's base-dev libraries, is fine if nothing uses them.
63- The flake supports `x86_64-linux`, `aarch64-linux` and `aarch64-darwin`;
76- The flake supports `x86_64-linux`, `aarch64-linux` and `aarch64-darwin`;
64 nixpkgs 26.11 dropped `x86_64-darwin`. An app's flake must provide its
77 nixpkgs 26.11 dropped `x86_64-darwin`. An app's flake must provide its
65 outputs on all three.
78 outputs on all three.
66−- Keep `README.md` in step with the flake: the app's row in the Apps table
79+- Keep `README.md` in step with the flake. Give the app its row in the Apps
80+ table (description, source link,
67− (description, source link, `nix build github:iko-soy/urbit-apps#<app>`), the
81+ `nix build github:iko-soy/urbit-apps#<app>`), and add it wherever the README
82+ lists apps like it: the glob and build-all commands and their counts, the
68− glob and build-all commands, the Frontends section for an app with a glob,
83+ Frontends section, which names every app, with a glob or without, and
69− and its bullet in "How the desks are built". Write in its voice: short,
84+ "How the desks are built", including its npm hashes. Search the README for
85+ a similar app's name to find them all, and keep the lists in alphabetical
86+ order. Write in the README's voice: short, plain sentences.
70− plain sentences. In a table cell, escape `|` as `\|`, even inside backticks.
87+- In a README table cell, escape `|` as `\|`, even inside backticks. GitHub
88+ drops the cells an unescaped one adds, so a rendered table won't show it.
89+- The README says `~donmec-sallud` publishes every app in its table, and
90+ [shelf.iko.soy](https://shelf.iko.soy) lists the apps it offers. Only the
91+ user can publish an app there: when you add one that shelf.iko.soy doesn't
92+ list, say so in your report.
71- Format `flake.nix` with `nix run --inputs-from . nixpkgs#nixfmt -- flake.nix`.
93- Format `flake.nix` with `nix run --inputs-from . nixpkgs#nixfmt -- flake.nix`.
72- `nix flake update` moves every input except those pinned by a commit in
94- `nix flake update` moves every input except those pinned by a commit in
73 their URL, such as `harness-grubbery` and `harness-tlon`; move those by
95 their URL, such as `harness-grubbery` and `harness-tlon`; move those by
74 editing the URL.
96 editing the URL.
75−- When a frontend's `package-lock.json` changes, its build stops on a hash
97+- When a frontend's `package-lock.json` changes, its build stops with a hash
76− mismatch. Set its hash (`npmDepsHash`, or the `hash` of harness' `npmDeps`)
98+ mismatch, or `npmDepsHash is out of date`. Set its hash (`npmDepsHash`, or
99+ the `hash` of harness' `npmDeps`) to `pkgs.lib.fakeHash`, build, and use
77− to `pkgs.lib.fakeHash`, build, and use the hash shown after `got:`.
100+ the hash shown after `got:`.
79## Validation
102## Validation
@@ -84,27 +107,44 @@ Run from the repository root:
84nix run --inputs-from . nixpkgs#nixfmt -- --check flake.nix
107nix run --inputs-from . nixpkgs#nixfmt -- --check flake.nix
85nix flake check --all-systems --no-build
108nix flake check --all-systems --no-build
86nix build --no-link --print-out-paths .#<app>
109nix build --no-link --print-out-paths .#<app>
110+grep '^|' README.md | sed 's/\\|//g' | awk -F'|' 'NF != 6'
113+The last command prints any README table row with a stray `|`; it should
89−For a change that shouldn't change a desk, `cmp` each app's new `.deskjam`
116+For a change that shouldn't change a desk, build every app before and after
90−with its old one. For one that should, `jammer unjam` both, `diff -r` them,
117+it, and `cmp` each `.deskjam`. Store paths don't tell: they change whenever
91−and check that only the expected files changed. `nix build --rebuild` of a
118+jammer is rebuilt. For a change that should, unpack both with
92−glob checks that it's reproducible.
119+`jammer unjam OLD.deskjam -o old` and `jammer unjam NEW.deskjam -o new`, and
93−
120+check that `diff -r old new` shows only the expected changes. To check that a
121+glob is reproducible, build it, then run `nix build --rebuild` on it a few
122+times; `--rebuild` fails on a glob that isn't built yet.
94−For a new app, also compare its deskjam with what the app's own procedure
124+For a new app, check that its deskjam has what the app's install steps make.
95−makes. deskjam's `tests/fakezod/pack.sh` packs a desk folder with
125+`|new-desk` adds `mar/hoon`, `mar/kelvin`, `mar/noun`, `mar/txt` and
96−`urbit eval`, byte for byte as jammer does. Then render the README with
126+`sys.kelvin` from `%base` to what you copy in. Make that folder, pack it with
97−`nix run --inputs-from . nixpkgs#pandoc -- -f gfm -t html README.md`, and
127+deskjam's `tests/fakezod/pack.sh`, which makes the same bytes as jammer with
98−check that every table row has four cells.
128+`urbit eval`, and `cmp` the two. pack.sh stops on a file jammer skips, such
99−
129+as hits' `LICENSE`, which has no extension: pack a copy without such files.
100−To test an app on a ship, use deskjam's fake-ship harness, `tests/fakezod` in
130+
131+To test an app on a ship, clone iko-soy/deskjam outside this repo and use its
132+fake-ship harness, `tests/fakezod`, which keeps its pier in its
133+`tests/fakezod/.work`:
101−iko-soy/deskjam. `bootstrap.sh` boots a fake `~zod` with deskjam, and
135+- `bootstrap.sh` boots a fake `~zod` with deskjam, and a desk `%harness` whose
102−`smoke-test.sh` shows how to drive deskjam's pages with curl. Set `HTTP_PORT`
136+ `%logtap` agent copies the ship's output into the log. This repo's harness
103−and `AMES_PORT`, and don't pick 31337 plus a galaxy's number as the Ames
137+ app is also `%harness`, and replacing that desk deletes logtap: move
104−port: the fake ship sends that galaxy's packets there, so they loop back.
138+ `%logtap` to a desk of its own before installing it.
139+- Set `HTTP_PORT`, and an `AMES_PORT` outside 31337 to 31592. A fake ship
140+ sends galaxy N's packets to port 31337 + N, so a ship on one of those ports
141+ loops them back to itself. The harness' default, 31483, is in that range.
142+- `smoke-test.sh` shows how to drive deskjam's pages with curl. It has no
143+ glob: for an app with one, post the confirm page's **set** form first, as
144+ the README's Frontends section says.
105−Upload a glob at `/docket/upload` as a browser does: a `desk` part, then one
145+- Upload a glob at `/docket/upload` as a browser does: a `desk` part, then one
106−`glob` part per file, named `<folder>/<path>`, with its content type. Stop
146+ `glob` part per file, named `<folder>/<path>`, with its content type.
107−the ship with `stop.sh` when done.
147+- Stop the ship with `stop.sh` when done.