on master help sign in
Run the eval checks in CI

Montelot chose a CI job for the eval checks now rather than at milestone 7 (review question 30, A92, A96). On every push to master and every pull request, .github/workflows/check.yml installs Nix and builds the flake's desks and eval checks, so it runs `tools/check --group eval` in the Nix sandbox with vere and Python pinned by flake.lock. The ship group still runs before pushing; development.md says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U4FzmSdubYxmZGDp2VJRAQ

commit84b8527891da92f8fb27e8b39ade52ac16643be4 authorClaude <noreply@anthropic.com> dateMon 2 Nov 2026, 20:21 (17 hours ago) parent7bc7d54
2 files changed+24
.github/workflows/check.yml+22 docs/development.md+2
.github/workflows/check.ymlnew file+22
@@ -0,0 +1,22 @@
1+# The eval checks on every push to master and every pull request:
2+# the flake builds the desks and runs `tools/check --group eval` in the
3+# Nix sandbox (see flake.nix). The ship checks need a fake ~zod and run
4+# before pushing, as docs/development.md says.
5+name: check
6+
7+on:
8+ push:
9+ branches: [master]
10+ pull_request:
11+ workflow_dispatch:
12+
13+jobs:
14+ eval:
15+ name: desks and eval checks
16+ runs-on: ubuntu-latest
17+ timeout-minutes: 120
18+ steps:
19+ - uses: actions/checkout@v4
20+ - uses: cachix/install-nix-action@v31
21+ - name: Build the desks and run the eval checks
22+ run: nix build --print-build-logs .#checks.x86_64-linux.desks .#checks.x86_64-linux.eval
docs/development.md+2
@@ -144,6 +144,8 @@ nix flake check # the same checks in the Nix sandbox, on a fresh fa
144144
145145`tools/check` runs every check the repo has. Their one list is `CHECKS` in the script, so a new check goes there, as well as in the table above; `tools/check --list` prints it with each check's command. The eval group, the 62 checks that need only `urbit` and Python, runs several at a time (`-j`, by default one per CPU), the slowest first. The ship group, `ship-selftest`, `ship-test`, `check-commit` and `clay-fixtures`, then runs one check at a time on the ship `tools/ship` talks to; `tools/check` starts it if it isn't running and stops it again afterwards. Name checks, or globs, to run only those: `tools/check golden 'type-todo-*'`. A failing check's output follows its FAIL line; `-v` prints every check's output, and `--logs DIR` keeps it all.
146146
147+CI, [.github/workflows/check.yml](../.github/workflows/check.yml), builds the desks and runs the eval group through the flake on every push to `master` and every pull request. It doesn't run the ship group, so run that before pushing.
148+
147149`ship-selftest` matters most when `tools/ship` changed, and `check-commit` when the mark or the desks changed, but `tools/check` runs both every time. The self-test leaves fixture files on the `jibb-dev` desk, and `ship-test`, which runs after it, puts the real ones back.
148150
149151`tools/ship test` regenerates `tests/lib/jibb.hoon` and syncs the `jibb-dev` desk before it runs, so commit the regenerated file. Sync preflights and splits large deletion lists into bounded click requests, waiting for each Clay revision; a failure stops later batches. The ship self-test exercises this when switching from the full desk to a smaller fixture desk. A run with `--stub`, `--lib`, or a custom `--golden` isolates its generated golden tests from the repository's other test programs; run the default command afterward to restore and test those programs. It also runs the independent `tests/lib/jibb-import.hoon` test against the same importer fixtures and expected diagnostic projection as `tools/importer-test`. The projection is an internal tree, not the Jibb type printer. `tests/lib/jibb-bridge.hoon` also runs the importer/checker bridge fixture on the ship; `tools/type-import-test` runs the same report under eval. `tests/lib/jibb-tlon-bridge.hoon` and `tools/type-tlon-test` run the real Tlon presence reader/checker fixture on both. The structured reader diagnostics have their own `tests/lib/jibb-import-diagnostic.hoon` ship test and `tools/type-import-diagnostic-test` eval test. The mark bridge has its own `tests/lib/jibb-mark-bridge.hoon` ship test and `tools/type-mark-test` eval test; both compare the same fixture report. `tests/lib/jibb-poke-bridge.hoon` and `tools/type-poke-test` share the source-backed prelude and poke-only API fixture. `tests/lib/jibb-api-paths.hoon` and `tools/type-api-paths-test` share partial path analysis over the real todo marks; this does not generate request interfaces. `tests/lib/jibb-module-scope.hoon` and `tools/type-module-scope-test` compare the prelude scope report on the ship and under eval.