@@ -4,22 +4,24 @@
44
55`urbit-apps` is a Nix flake that builds Urbit apps' desks into `.deskjam`
66files, the upload format of [deskjam](https://github.com/iko-soy/deskjam),
7−with each app's frontend glob next to it. `jammer/` is the Rust CLI that packs
8−them. `README.md` documents the apps for users: read it before changing what
9−the flake builds.
7+with its glob next to each one, for an app whose frontend is one. `jammer/`
8+is the Rust CLI that packs them. `README.md` documents the apps for users:
9+read it before changing what the flake builds.
1010
1111## Code map
1212
13−- `flake.nix`: one input per app, plus any dependencies an app's own build
14− pins. Each app is an `<app>-desk` built with `mkDeskjam` and exposed as
15− `packages.<system>.<app>`; frontends are `packages.<system>.<app>-glob`.
13+- `flake.nix`: nixpkgs, flake-utils, one input per app, and the commits
14+ harness' build pins. Each app is an `<app>-desk` built with `mkDeskjam` and
15+ exposed as `packages.<system>.<app>`. Globs are `packages.<system>.<app>-glob`,
16+ for hits, silo and vita. jammer is `packages.<system>.jammer` and `default`.
1617 - `mkDeskjam { desk, src, glob ? null }` packs the desk folder `src` into
1718 `$out/<desk>.deskjam` and copies `glob` next to it as `$out/<desk>-glob`.
1819 - `names-glob` reads a `desk.docket-0` with `urbit eval`, as the docket mark
1920 does. `mkDeskjam` fails the build when the docket names a glob (a
2021 `glob-http` or `glob-ames` clause, and no `site` clause) but has no
21− `glob`, when it has a `glob` for a docket that names none, and when the
22− docket isn't valid Hoon.
22+ `glob`, when it has a `glob` for a docket that names none, and when Hoon
23+ can't read the docket as a list of clauses. It doesn't check that the
24+ docket has the clauses the mark needs, such as `title`.
2325 - `mkGlob` builds a frontend with `buildNpmPackage` (`npm run build`, then
2426 its `dist/`), for apps whose own flakes don't build their globs.
2527 - `harness-git` and `harness-src` run harness' own `zig build` offline; see
@@ -33,24 +35,33 @@ the flake builds.
3335## Working conventions
3436
3537- Use jj, not git, and pass `--git` to `jj diff` and `jj show`. Commit or push
36− only when asked. Nix only sees files jj tracks. Never write to `/nix` by
37− hand.
38+ only when asked. Never write to `/nix` by hand.
39+- Nix only sees files jj tracks, and jj tracks a new file at its next
40+ command. After adding a file, run `jj status` before building, not
41+ `git add`.
42+- jammer writes `<desk>.deskjam` or `<desk>/` in the current directory, and jj
43+ would track them in this repo. Run it elsewhere, or give `jam` and `unjam`
44+ an `-o` outside the repo; `nix shell .` puts it on your PATH.
3845- Don't change the app repos from here. When one needs a change, write a
3946 prompt the user can give an agent in that repo: the problem, with evidence,
4047 the change, and how to check it.
4148- Add an app by the first route its repo allows:
42− 1. Its flake follows the convention: `packages.<system>.desk` is the full
43− desk, `.glob` is the built frontend if its docket names a glob, and
44− `default` is `desk`. Add it as a flake input, with
45− `src = app.packages.${system}.desk` and
46− `glob = app.packages.${system}.glob`. Have its `nixpkgs` follow this
47− flake's only if its flake builds no frontend; otherwise keep the nixpkgs
48− it pins, so its glob is the same build as in its own repo.
49+ 1. Its flake builds the full desk, as `packages.<system>.desk`, or as
50+ `default` if it has no `desk`, like vita's. Add it as a flake input, with
51+ `src = app.packages.${system}.desk`, or `.default`. If its docket names a
52+ glob and its flake builds it, use `glob = app.packages.${system}.glob`;
53+ if not, build it here with `mkGlob`, as for hits. Have its `nixpkgs`
54+ follow this flake's only if its flake builds no frontend; otherwise keep
55+ the nixpkgs it pins, so its glob is the same build as in its own repo.
4956 2. It has no flake, and its `desk/` is the whole desk: a `flake = false`
50− input, with `src = "${app}/desk"`.
57+ input, with `src = "${app}/desk"`. If its docket names a glob, build it
58+ with `mkGlob`, starting its `npmDepsHash` at `pkgs.lib.fakeHash`.
5159 3. Its own build tool assembles the desk: run that build in Nix, with its
52− fetches pinned as inputs, as for harness. Don't reimplement its steps,
53− and make the build fail loudly when upstream pins something new.
60+ fetches pinned as inputs, as for harness. Don't reimplement its steps.
61+ Replace only a step that can't run in the sandbox, as `flake.nix` does
62+ for harness' npm steps, with `--replace-fail`, so a change upstream
63+ stops the build. Make the build fail loudly when upstream pins
64+ something new.
5465- Before adding an app, check:
5566 - that its `sys.kelvin` lists the current kernel, `[%zuse 408]`. Kernel
5667 numbers count down, so 411 is older than 408. Don't package an app that
@@ -58,23 +69,35 @@ the flake builds.
5869 - its desk name, from its README or its docket's `base`.
5970 - whether its docket names a glob, and where else its frontend could come
6071 from, such as files its own agent serves from the desk.
61− - that `jammer jam -v` reports no missing marks, and that the desk has the
62− libraries its Hoon imports.
72+ - that `jammer jam -v` reports no missing marks, and that deskjam's confirm
73+ page, on a fake ship, finds nothing missing that its agents and marks
74+ need. Its warning that other files import files the desk lacks, as for
75+ silo's base-dev libraries, is fine if nothing uses them.
6376- The flake supports `x86_64-linux`, `aarch64-linux` and `aarch64-darwin`;
6477 nixpkgs 26.11 dropped `x86_64-darwin`. An app's flake must provide its
6578 outputs on all three.
66−- Keep `README.md` in step with the flake: the app's row in the Apps table
67− (description, source link, `nix build github:iko-soy/urbit-apps#<app>`), the
68− glob and build-all commands, the Frontends section for an app with a glob,
69− and its bullet in "How the desks are built". Write in its voice: short,
70− plain sentences. In a table cell, escape `|` as `\|`, even inside backticks.
79+- Keep `README.md` in step with the flake. Give the app its row in the Apps
80+ table (description, source link,
81+ `nix build github:iko-soy/urbit-apps#<app>`), and add it wherever the README
82+ lists apps like it: the glob and build-all commands and their counts, the
83+ Frontends section, which names every app, with a glob or without, and
84+ "How the desks are built", including its npm hashes. Search the README for
85+ a similar app's name to find them all, and keep the lists in alphabetical
86+ order. Write in the README's voice: short, plain sentences.
87+- In a README table cell, escape `|` as `\|`, even inside backticks. GitHub
88+ drops the cells an unescaped one adds, so a rendered table won't show it.
89+- The README says `~donmec-sallud` publishes every app in its table, and
90+ [shelf.iko.soy](https://shelf.iko.soy) lists the apps it offers. Only the
91+ user can publish an app there: when you add one that shelf.iko.soy doesn't
92+ list, say so in your report.
7193- Format `flake.nix` with `nix run --inputs-from . nixpkgs#nixfmt -- flake.nix`.
7294- `nix flake update` moves every input except those pinned by a commit in
7395 their URL, such as `harness-grubbery` and `harness-tlon`; move those by
7496 editing the URL.
75−- When a frontend's `package-lock.json` changes, its build stops on a hash
76− mismatch. Set its hash (`npmDepsHash`, or the `hash` of harness' `npmDeps`)
77− to `pkgs.lib.fakeHash`, build, and use the hash shown after `got:`.
97+- When a frontend's `package-lock.json` changes, its build stops with a hash
98+ mismatch, or `npmDepsHash is out of date`. Set its hash (`npmDepsHash`, or
99+ the `hash` of harness' `npmDeps`) to `pkgs.lib.fakeHash`, build, and use
100+ the hash shown after `got:`.
78101
79102## Validation
80103
@@ -84,27 +107,44 @@ Run from the repository root:
84107nix run --inputs-from . nixpkgs#nixfmt -- --check flake.nix
85108nix flake check --all-systems --no-build
86109nix build --no-link --print-out-paths .#<app>
110+grep '^|' README.md | sed 's/\\|//g' | awk -F'|' 'NF != 6'
87111```
88112
89−For a change that shouldn't change a desk, `cmp` each app's new `.deskjam`
90−with its old one. For one that should, `jammer unjam` both, `diff -r` them,
91−and check that only the expected files changed. `nix build --rebuild` of a
92−glob checks that it's reproducible.
93−
94−For a new app, also compare its deskjam with what the app's own procedure
95−makes. deskjam's `tests/fakezod/pack.sh` packs a desk folder with
96−`urbit eval`, byte for byte as jammer does. Then render the README with
97−`nix run --inputs-from . nixpkgs#pandoc -- -f gfm -t html README.md`, and
98−check that every table row has four cells.
99−
100−To test an app on a ship, use deskjam's fake-ship harness, `tests/fakezod` in
101−iko-soy/deskjam. `bootstrap.sh` boots a fake `~zod` with deskjam, and
102−`smoke-test.sh` shows how to drive deskjam's pages with curl. Set `HTTP_PORT`
103−and `AMES_PORT`, and don't pick 31337 plus a galaxy's number as the Ames
104−port: the fake ship sends that galaxy's packets there, so they loop back.
105−Upload a glob at `/docket/upload` as a browser does: a `desk` part, then one
106−`glob` part per file, named `<folder>/<path>`, with its content type. Stop
107−the ship with `stop.sh` when done.
113+The last command prints any README table row with a stray `|`; it should
114+print nothing.
115+
116+For a change that shouldn't change a desk, build every app before and after
117+it, and `cmp` each `.deskjam`. Store paths don't tell: they change whenever
118+jammer is rebuilt. For a change that should, unpack both with
119+`jammer unjam OLD.deskjam -o old` and `jammer unjam NEW.deskjam -o new`, and
120+check that `diff -r old new` shows only the expected changes. To check that a
121+glob is reproducible, build it, then run `nix build --rebuild` on it a few
122+times; `--rebuild` fails on a glob that isn't built yet.
123+
124+For a new app, check that its deskjam has what the app's install steps make.
125+`|new-desk` adds `mar/hoon`, `mar/kelvin`, `mar/noun`, `mar/txt` and
126+`sys.kelvin` from `%base` to what you copy in. Make that folder, pack it with
127+deskjam's `tests/fakezod/pack.sh`, which makes the same bytes as jammer with
128+`urbit eval`, and `cmp` the two. pack.sh stops on a file jammer skips, such
129+as hits' `LICENSE`, which has no extension: pack a copy without such files.
130+
131+To test an app on a ship, clone iko-soy/deskjam outside this repo and use its
132+fake-ship harness, `tests/fakezod`, which keeps its pier in its
133+`tests/fakezod/.work`:
134+
135+- `bootstrap.sh` boots a fake `~zod` with deskjam, and a desk `%harness` whose
136+ `%logtap` agent copies the ship's output into the log. This repo's harness
137+ app is also `%harness`, and replacing that desk deletes logtap: move
138+ `%logtap` to a desk of its own before installing it.
139+- Set `HTTP_PORT`, and an `AMES_PORT` outside 31337 to 31592. A fake ship
140+ sends galaxy N's packets to port 31337 + N, so a ship on one of those ports
141+ loops them back to itself. The harness' default, 31483, is in that range.
142+- `smoke-test.sh` shows how to drive deskjam's pages with curl. It has no
143+ glob: for an app with one, post the confirm page's **set** form first, as
144+ the README's Frontends section says.
145+- Upload a glob at `/docket/upload` as a browser does: a `desk` part, then one
146+ `glob` part per file, named `<folder>/<path>`, with its content type.
147+- Stop the ship with `stop.sh` when done.
108148
109149Quirks:
110150