on f43d2ee help sign in
jibb/docs/development.md
238 lines 26 kB history raw
1# Development
3How to work on Jibb: the rules the plan sets, how to set up a machine, and what to run before pushing. The repo's [README](../README.md) describes each tool in full.
5## Ground rules
7From the plan's Setup and ground rules, and from @iko:
9- **The doc wins.** Where code and doc disagree, the doc stands until it changes. [spec/](spec/) is a snapshot; the doc itself is the authority.
10- **Language changes are agreed first** and recorded in the Decisions tab, before the code changes.
11- **Questions go to @iko**, as comments on the doc, on the words they're about. While one is open, follow the most likely reading, mark it "(doc question)" in the library's notes, and list it in [open-questions.md](open-questions.md). Where an error points is not a doc question: the library's section 2.0 settles it.
12- **Every Jibb code block in the doc is a golden test**, and each stage's output is printed and compared. The doc labels its Jibb blocks `jibb`; `tools/doc-blocks` checks that each has its case.
13- **Each milestone ends with its checks passing on a fake ship**, not only under `urbit eval`.
14- **Push straight to `master`.** @iko reviews work in the project's chat, not in pull requests, so there are none. Run the checks below first.
15- **Keep docs/ current**: [status.md](status.md) when a milestone moves, [open-questions.md](open-questions.md) when a question is settled or found.
17The source-backed generic-union guard can be run under eval with `tools/type-union-source-test`; its `test-union-source` arm is also assembled for the fake ship. It checks a bounded partial inference slice, not general union completeness. `tools/type-union-alias-overlap-test` exercises the separate positive overlap witness and its shared ship test arm. `tools/type-full-overlap-test` checks structured full-module overlap evidence and the legacy error text under eval; both of its test arms are assembled for the fake ship.
19## Setting up a machine
21The tools need Python 3 and `urbit` (vere) on `PATH`; the ship tools also need a fake `~zod` and click. Everything so far ran with vere 4.6 on Linux.
23On a usual machine:
251. Install vere as [docs.urbit.org](https://docs.urbit.org) says, and get a brass pill.
262. Boot a fake ship where `tools/ship` expects it, or point `--pier` or `$JIBB_PIER` elsewhere. Booting takes a few minutes:
27 ```
28 mkdir -p /root/urbit && cd /root/urbit
29 urbit -F zod -B brass.pill -t -x -c zod
30 ```
313. Get click, the command-line client for a ship's `conn.sock`, from [urbit/tools](https://github.com/urbit/tools), at `/root/src/tools/pkg/click/click` or wherever `--click` or `$JIBB_CLICK` says.
324. Optionally, clone [urbit/urbit](https://github.com/urbit/urbit) to read Arvo's source. `vendor/SOURCES.txt` names the commit the vendored files come from.
34In a Claude cloud container, the network blocks urbit.org, bootstrap.urbit.org, nixos.org's installer and GitHub's release downloads. This worked on 2026-10-29, and has to be redone in each new container:
36```
37# Nix, single-user
38curl -sSfL -o install https://releases.nixos.org/nix/nix-2.32.4/install
39mkdir -p /etc/nix && printf 'build-users-group =\nsandbox = false\n' > /etc/nix/nix.conf
40sh ./install --no-daemon --no-channel-add --yes
41. /root/.nix-profile/etc/profile.d/nix.sh
43# vere, from nixpkgs
44nix build -I nixpkgs=https://channels.nixos.org/nixos-unstable/nixexprs.tar.xz \
45 --impure --expr 'with import <nixpkgs> {}; urbit'
46# then put result/bin/urbit on PATH
48# the brass pill, from Git LFS
49mkdir -p /root/urbit && cd /root/urbit
50curl -sSfL -o brass.pill https://media.githubusercontent.com/media/urbit/urbit/master/bin/brass.pill
51urbit -F zod -B brass.pill -t -x -c zod
53# click
54git clone https://github.com/urbit/tools /root/src/tools
55```
57`echo '(add 2 2)' | urbit eval` checks that vere works without a ship.
59## The tools
61| Tool | What it does |
62| --- | --- |
63| `tools/golden` | Runs the golden cases under `urbit eval`: every case, or those matching `--case`; `--update` rewrites the expected outputs; `--show` prints them |
64| `tools/importer-test` | Compares the partial Hoon importer's diagnostic trees and errors with `tests/importer/tree.txt` under `urbit eval` |
65| `tools/type-import-default-test` | Compares default-wrapped union source proofs and adversarial cases under eval; the same golden runs on the ship |
66| `tools/type-import-test` | Composes the Hoon importer with the partial checker and compares `tests/type-import/expected.txt` under eval |
67| `tools/type-import-overlap-test` | Checks source-aware Hoon union overlap, provenance and proof-budget controls under eval; the same fixture runs on the ship |
68| `tools/type-import-structural-test` | Checks multi-case and nested Hoon union overlap witnesses under eval and on the ship |
69| `tools/type-union-source-test` | Checks bounded local generic-alias disjointness and state guards under eval and on the ship |
70| `tools/type-union-alias-overlap-test` | Checks source-backed generic-alias overlap witnesses under eval and on the ship |
71| `tools/type-full-overlap-test` | Checks structured full-module overlap evidence and imported-source attribution |
72| `tools/type-tlon-test` | Runs the real Tlon presence source through the importer and partial checker, comparing `tests/type-tlon/expected.txt` under eval |
73| `tools/type-import-diagnostic-test` | Compares the structured reader diagnostic bridge report under eval |
74| `tools/type-mark-test` | Checks the partial checker’s mark import boundary against `tests/type-mark/expected.txt` under eval |
75| `tools/type-poke-test` | Checks source-backed prelude and poke-only API reports against `tests/type-poke/expected.txt` under eval |
76| `tools/type-api-paths-test` | Checks partial API path analysis with real todo mark samples against `tests/type-api-paths/expected.txt` under eval |
77| `tools/type-api-evidence-test` | Runs source, scope, reader-result and declaration-watchability evidence guards under eval and on the ship |
78| `tools/type-hole-guards-test` | Runs the shared hole-defaulting and API interface guards under eval; the same checks run on the ship |
79| `tools/type-list-alias-test` | Runs the shared prelude list-alias and provenance guards under eval and on the ship |
80| `tools/type-generic-recursion-test` | Checks source-backed generic recursive identity and provenance under eval and on the ship |
81| `tools/type-generic-construction-test` | Checks source-backed one-layer recursive construction, provenance and fuel under eval and on the ship |
82| `tools/type-generic-pattern-test` | Checks source-backed recursive patterns, catch-all coverage, state and per-attempt fuel/arm limits under eval and on the ship |
83| `tools/type-deferred-test` | Checks the bounded source-owned deferred inference transaction under eval and on the ship |
84| `tools/type-deferred-resolver-test` | Checks deferred resolution, report consistency and atomic failure under eval and on the ship |
85| `tools/type-deferred-context-test` | Checks deferred source order and literal branch contexts under eval and on the ship |
86| `tools/type-deferred-prepared-test` | Checks prepared-prelude tuple initializer recovery, exact partial reports and fail-closed guards under eval and on the ship |
87| `tools/type-cell-branch-test` | Checks closed-cell branch context, rollback and bounded retry screens under eval and on the ship |
88| `tools/type-generic-alias-test` | Runs public/private generic union alias and provenance guards under eval and on the ship |
89| `tools/type-positional-face-guards-test` | Runs positional-call and nested-face guards under eval and on the ship |
90| `tools/type-qath-guards-test` | Runs source-provenance, recursive unfolding and face guards under eval and on the ship |
91| `tools/type-todo-partial-test` | Reports partial inference of the unchanged todo island through the real API and readers under eval and on the ship |
92| `tools/type-todo-api-test` | Exercises the actual todo API through the real Hoon readers and partial checker |
93| `tools/type-direct-gate-test` | Checks a synthetic source-backed program report and direct gate provenance under eval and on the ship |
94| `tools/type-page-main-test` | Fully checks byte-identical static and direct data pages through real readers, exact main types, downstream imports and rejection guards under eval and on the ship |
95| `tools/type-buttons-test` | Checks the documented valid button helpers and the unchanged unresolved-`%add` module through detailed and legacy inference under eval and on the ship |
96| `tools/type-doc-examples-test` | Fully checks the import-only, view-day, step and cmd/sub doc fragments in their intended contexts through detailed and legacy inference under eval and on the ship |
97| `tools/type-theme-test` | Fully checks the byte-identical real theme, its exact exported type, structural downstream import, dark absent/present forms and rejection guards under eval and on the ship |
98| `tools/type-program-main-test` | Checks the narrow closed subscription-free program-main interface, downstream import and rejection guards under eval and on the ship |
99| `tools/type-program-init-test` | Checks transactional direct-program initializer recovery, exact exported/imported types and fail-closed boundaries under eval and on the ship |
100| `tools/type-program-update-test` | Checks transactional direct-program update result recovery, union/boolean context, exact exported/imported types and fail-closed boundaries under eval and on the ship |
101| `tools/type-program-watch-test` | Checks direct singleton request-value watchability against merged real-todo evidence, exact types/imports and rejection boundaries under eval and on the ship |
102| `tools/type-store-test` | Checks the unchanged cart/basket through detailed and legacy inference, exact store/derived schemes, malformed-prelude guards, structural imports, source-bound state watch evidence and rejection guards under eval and on the ship |
103| `tools/type-todo-full-test` | Fully checks the byte-identical todo island through real readers, exact main type, structural downstream import and rejection controls under eval and on the ship |
104| `tools/type-closed-gate-guards-test` | Checks closed gate sample labels and face uncertainty under eval and on the ship |
105| `tools/type-constructor-fit-guards-test` | Checks constructor-case uncertainty and known mismatches under eval and on the ship |
106| `tools/type-nested-union-guards-test` | Checks nested tagged-union case selection and uncertainty under eval and on the ship |
107| `tools/type-empty-test` | Checks source-backed empty molds, inhabitedness, coverage and proof budgets under eval and on the ship |
108| `tools/type-gate-free-guards-test` | Checks bounded gate-free verdicts, persistent constraints, substitution safeguards, collection keys and witness paths under eval and on the ship |
109| `tools/type-gate-free-source-test` | Checks source-backed todo mold analysis through the real readers under eval and on the ship |
110| `tools/type-sub-guards-test` | Checks source-backed subscription mold dependencies under eval and on the ship |
111| `tools/type-ask-guards-test` | Runs ask origin and canonical list/unit bootstrap guards under eval and on the ship |
112| `tools/type-request-guards-test` | Runs request evidence, source consistency and cache provenance guards under eval and on the ship |
113| `tools/type-proj-guards-test` | Runs projection evidence, name-scope and dependency guards under eval and on the ship |
114| `tools/type-arithmetic-guards-test` | Runs restricted atom-variable, unification and projection guards under eval and on the ship |
115| `tools/type-module-scope-test` | Checks the partial module checker’s prelude boundary and duplicate-source rejection against `tests/type-module-scope/expected.txt` under eval |
116| `tools/clay-fixtures` | Syncs `my-app`, builds its example `/sur` files and marks through Clay, checks selected mold and mark projections and the partial checker, and records Ford and mark subject shapes on the fake ship |
117| `tools/hoon` | Evaluates Hoon with the library in scope, as `tools/hoon '(golden /a/b/jibb (crip "++ main view"))'`; `--cord name=file` binds a file's text, `-f` reads the expression from a file |
118| `tools/doc-blocks` | Checks the doc's `jibb` blocks against the `doc-*` golden cases: `tools/doc-blocks docs/spec` |
119| `tools/ship` | Runs the fake ship: `start`, `stop`, `status`, `log`, `sync <desk>`, `test`, `gen-tests`, `check-commit`, `scry`, `hoon`, `run`, `poke`, `commit`, `mount`, `unmount` |
120| `tools/testdata/selftest` | Tests `tools/golden` and `tools/hoon` against a stub library, and `tools/doc-blocks` against Markdown it writes |
121| `tools/testdata/ship-selftest` | Tests `tools/ship`'s test generator and runner against a stub library on the ship |
122| `tools/testdata/clay-fixtures-selftest` | Tests temporary Clay fixture cleanup and signal handling without a ship |
124`tools/golden` starts `urbit eval` once and builds the library, which takes about 8 s, then runs every case under `mule`, so a crash fails only its own case.
126## Before pushing
128```
129tools/golden # all cases pass
130tools/golden --case types-prelude-ui # exact UI/program/page schemes and composition
131tools/importer-test # Hoon importer fixture projections match
132tools/type-import-default-test # default-wrapped source proofs match
133tools/type-import-test # importer/checker bridge golden report matches
134tools/type-import-overlap-test # source-aware Hoon overlap and provenance controls
135tools/type-import-structural-test # nested and multi-case Hoon overlap controls
136tools/type-union-source-test # local generic-alias union guards
137tools/type-union-alias-overlap-test # local generic-alias overlap diagnostics
138tools/type-full-overlap-test # full-module overlap evidence
139tools/type-tlon-test # real Tlon importer/checker report matches
140tools/type-import-diagnostic-test # direct structured reader diagnostics match
141tools/type-mark-test # mark/checker bridge golden report matches
142tools/type-poke-test # source-backed poke/API bridge report matches
143tools/type-api-paths-test # partial API path analysis report matches
144tools/type-api-evidence-test # API declaration evidence guards pass
145tools/type-hole-guards-test # hole defaulting and API guards pass
146tools/type-list-alias-test # prelude list-alias guards pass
147tools/type-generic-alias-test # generic union alias guards pass
148tools/type-positional-face-guards-test # positional sample face guards pass
149tools/type-qath-guards-test # qath source and unfolding guards pass
150tools/type-todo-partial-test # unchanged todo partial report and reader failure controls
151tools/type-todo-api-test # real todo API partial inference and reader guards pass
152tools/type-direct-gate-test # direct gate source and provenance guards pass
153tools/type-page-main-test # static/data pages, exact types, imports and rejection guards pass
154tools/type-buttons-test # documented buttons schemes and exact pending-tag error
155tools/type-doc-examples-test # remaining doc fragments fully check in context
156tools/type-theme-test # real theme, exact type, structural import and rejection guards pass
157tools/type-program-main-test # closed program-main interface and rejection guards pass
158tools/type-program-init-test # transactional direct-program initializer context guards pass
159tools/type-program-update-test # transactional direct-program update context guards pass
160tools/type-program-watch-test # direct request-value watchability guards pass
161tools/type-store-test # unchanged cart/basket, store evidence and rejection guards pass
162tools/type-todo-full-test # byte-identical full todo, downstream import and rejection guards pass
163tools/type-generic-recursion-test # source-backed recursive identities and provenance
164tools/type-generic-construction-test # source-backed recursive construction and bounds
165tools/type-generic-pattern-test # source-backed recursive patterns and bounds
166tools/type-deferred-test # source-owned deferred inference guards
167tools/type-deferred-resolver-test # resolver consistency and failure guards
168tools/type-deferred-context-test # deferred source-order and branch guards
169tools/type-deferred-prepared-test # prepared initializer reports and failure guards
170tools/type-cell-branch-test # contextual branch rollback and bounds
171tools/type-closed-gate-guards-test # closed gate fit guards pass
172tools/type-constructor-fit-guards-test # constructor fit guards pass
173tools/type-nested-union-guards-test # nested union selector guards pass
174tools/type-empty-test # empty mold and uninhabited-case guards pass
175tools/type-gate-free-guards-test # bounded gate-free classifier guards pass
176tools/type-gate-free-source-test # real todo mold analysis and source guards pass
177tools/type-sub-guards-test # subscription source guards pass
178tools/type-ask-guards-test # ask source and bootstrap guards pass
179tools/type-request-guards-test # request evidence and cache guards pass
180tools/type-proj-guards-test # projection evidence and scope guards pass
181tools/type-arithmetic-guards-test # arithmetic constraint guards pass
182tools/type-module-scope-test # module/prelude scope golden report matches
183tools/hoon check-spans-test # prints <<>>
184tools/hoon itype-unify-test # type-unification controls pass
185tools/hoon -f tools/testdata/zing-hoon-reference.hoon # Hoon translation agrees on four inputs
186tools/hoon -f tools/testdata/lead-late-hoon-reference.hoon # Hoon translation agrees on selected inputs
187tools/hoon -f tools/testdata/succ-hoon-reference.hoon # Hoon translation agrees on selected inputs
188tools/testdata/selftest # 22 checks, 0 failed
189tools/doc-blocks docs/spec # every jibb block has its case
190tools/ship gen-tests --check # tests/lib/jibb.hoon is up to date
191tools/ship test # the cases pass as -test arms on the ship
192tools/testdata/ship-selftest # when tools/ship changed
193tools/testdata/clay-fixtures-selftest # scratch fixture cleanup without a ship
194tools/ship check-commit # when the mark or the desks changed
195tools/clay-fixtures # builds example fixtures and real Tlon notes through Clay
196```
198`tools/ship test` regenerates `tests/lib/jibb.hoon` and syncs the `jibb-dev` desk before it runs, so commit the regenerated file. Sync preflights and splits large deletion lists into bounded click requests, waiting for each Clay revision; a failure stops later batches. The ship self-test exercises this when switching from the full desk to a smaller fixture desk. A run with `--stub`, `--lib`, or a custom `--golden` isolates its generated golden tests from the repository's other test programs; run the default command afterward to restore and test those programs. It also runs the independent `tests/lib/jibb-import.hoon` test against the same importer fixtures and expected diagnostic projection as `tools/importer-test`. The projection is an internal tree, not the Jibb type printer. `tests/lib/jibb-bridge.hoon` also runs the importer/checker bridge fixture on the ship; `tools/type-import-test` runs the same report under eval. `tests/lib/jibb-tlon-bridge.hoon` and `tools/type-tlon-test` run the real Tlon presence reader/checker fixture on both. The structured reader diagnostics have their own `tests/lib/jibb-import-diagnostic.hoon` ship test and `tools/type-import-diagnostic-test` eval test. The mark bridge has its own `tests/lib/jibb-mark-bridge.hoon` ship test and `tools/type-mark-test` eval test; both compare the same fixture report. `tests/lib/jibb-poke-bridge.hoon` and `tools/type-poke-test` share the source-backed prelude and poke-only API fixture. `tests/lib/jibb-api-paths.hoon` and `tools/type-api-paths-test` share partial path analysis over the real todo marks; this does not generate request interfaces. `tests/lib/jibb-module-scope.hoon` and `tools/type-module-scope-test` compare the prelude scope report on the ship and under eval.
200## Golden cases
202A case is a folder `tests/golden/<name>/` holding:
204- `source.jibb`, the input, ending in one newline;
205- `about.txt`: a `source:` line with the doc's words or block it comes from, a `tests:` line saying what it exercises, and an `expect:` line, either `parses` or `error at <line>:<col>` and why (one case, `inline-leftout-signed-float-auras`, says `unclear`: see grammar question 13 in [open-questions.md](open-questions.md));
206- the expected outputs, `tokens.txt` and `tree.txt`, each the output plus one newline.
207- optionally, `path.txt`, the logical `/ui/.../*.jibb` path to pass to the compiler; the source remains `source.jibb`.
208- optionally, an `expect-types:` line in `about.txt` opts the case into `+golden-types`; `types.txt` holds its output plus one newline and `--update` creates it. A `types.txt` also opts in when the line is absent. This interface is provisional while the checker is under construction.
209- optionally, `imports.txt` selects Hoon `/sur` molds, one `module mold` pair per line. Each module's source goes in `sur/<module>.txt` and is compiled against zuse by both runners. `marks.txt` selects marks, one name per line, with source in `mar/<name>.txt`; both kinds may be supplied together. The reader results go to the partial checker, and a type output is required. Both runners reconstruct fixture text from `%txt` lines and identify the fixture if its compilation crashes; a crash fails the case without discarding its successful parse output. These fixtures currently handle only sources without Ford imports.
210- optionally, `modules.txt` lists logical `/ui/.../*.jibb` paths, one per line. The corresponding sources live below the case's `modules/ui/` directory. Both runners parse each source at its logical path and compare the combined `module-trees.txt`; `--update` writes that output. A case with type output also supplies these sources and the vendored prelude to the partial module checker. The file opts in even when it lists no sources; use a comment such as `# no module fixtures` for an empty bundle. Blank lines and comment lines are ignored. Cases without `modules.txt` keep their existing no-prelude type interface.
212The runners prepare the trusted vendored public and internal prelude sources once for their generated test core and share the partial interface. Preparation reserves all names present in the public source and supplies the molds and values the fragment can infer; an unsupported declaration remains reserved but unavailable. The source and checker are incomplete, so the output still says the module has not been checked and does not claim complete shadowing coverage. `+golden-types-with-prelude-and-internal` accepts both sources directly; callers using `+golden-types-with-prepared-prelude` pass the unchanged result of `+prepare-prelude-with-internal`. The older source-only entry points remain available for preludes without imports. Missing or malformed sources fail closed.
214Names say what a case is for:
216| Name | For |
217| --- | --- |
218| `lex-<rule>-ok-<n>`, `lex-<rule>-bad-<n>` | A lexical rule, kept or broken |
219| `inline-<topic>` | Any other rule, often from one sentence of the Grammar tab |
220| `doc-<tab>-<n>-<what>` | The doc's `<n>`th code block of the tab, counting every block, labelled or not |
222Write `source.jibb` and `about.txt` by hand, run `tools/golden --update --case <name>`, and read the outputs against `about.txt` before committing. A changed output on purpose is reviewed with `git diff tests/golden`.
224## Code conventions
226`jibb-dev/lib/jibb.hoon` is one core with no Ford runes, so `urbit eval` can run it and Clay can build it. Its design notes open the file in numbered sections; keep them true when the code changes, since they are where the reasoning lives. [compiler.md](compiler.md) maps the file.
228- An arm's comment is `:: +name: what it is`, then details indented by four spaces.
229- Types avoid the standard library's names (`mold`, `wing`, `gate`), hence `+shape` for a mold as written. Every type in a recursive cycle has a `$~` default that is a leaf, or Hoon can't build its default.
230- Two arms do shadow the standard library: `+in`, a bit test, hides the set door `in`, and `+fail` hides Hoon's parser rule `fail`. Inside the library, and under `tools/hoon`, write `~(has in:..zuse s)`, or rename `+in` before code there uses sets.
231- The parser's errors read "expected <what>, found <token>", with an optional "; <hint>"; the layout pass's state the rule broken, as "a tab: Jibb uses spaces only". No final period, lowercase except for names, and the code the error is about in backticks; example code in a hint, as m(done=%.y), isn't quoted so far. Their wording is fixed by the golden tests once written.
232- Loops over many items are tail calls, so long inputs grow no stack; check new passes on a long module.
234## Commits
236A short subject in the imperative, then a body that says what changed and why, wrapped at 72 columns, as in `git log`. Each commit leaves every check passing.
238`tools/type-deferred-test` exercises the bounded, source-owned constructor transaction and low-level freshness/replay guards. `tools/type-deferred-resolver-test` checks the experimental import-free library resolver, report consistency, atomic failure, and that both source orders of a late face constraint decline. `tools/type-deferred-prepared-test` checks the separate partial-report-only prepared-prelude route, direct tuple-literal obligations, exact report types and its wrong-field, unrelated-tag, mold-collision and no-final-cast controls. These checks do not establish island-kind or watchability support.