Commit Graph

30705 Commits

Author SHA1 Message Date
Joe Bryan
a7022e75a0 u3: enforce snapshot invariants on segment truncation 2022-12-02 10:27:38 -05:00
Joe Bryan
c111129a5e u3: adds new, batch page tracking implementations + tests 2022-12-02 10:27:35 -05:00
Joe Bryan
d1d1860ac7 u3: cleanup comments about snapshot system limitations 2022-12-02 10:27:32 -05:00
Joe Bryan
fee50375c9 u3: document snapshot system invariants 2022-12-02 10:27:29 -05:00
Joe Bryan
d94d6d3dd2 Merge branch 'jb/release/vere' into jb/restage/demand 2022-11-30 18:28:30 -05:00
Joe Bryan
50243ed0d4 Merge branch 'jb/restage/pread-all' into jb/release/vere 2022-11-30 18:16:16 -05:00
Joe Bryan
ea6f239811 Merge tag 'urbit-v1.13' into jb/restage/demand
urbit-v1.13

Arvo 417K
Vere 1.13

This release makes the loom variably sized, up to a new maximum of 4GB.
The default remains 2GB -- opt for the max with `--loom 32`.
(NB: larger sizes may require adding swap space.)

Release Notes

- supports setting the loom size on startup. sizes are specified in
  exponents of 2, from 1MB (20) to 4GB (32); can differ between processes
  (`--loom` and `--urth-loom`); and can be set for relevant subcommands
  (`meld`, `pack`, `next`, &c).

- improves bit-slice performance (for noun de/serialization and many atom jets)
  by ensuring that implementation inner loops can be vectorized.

- adds the `vile` command, for exporting the keyfile from a ship

- adds the `%xray` hint, for printing the bytecode of a given expression
  (ty ~topfet-parmed!)

- adds the `eval` command, for running hoon (from stdin) without booting a ship
  (ty ~mopfel-winrux!)

- adds a better error message when a ship is already running as root
  (ty ~dinleb-rambep!)

Contributions:

Joe Bryan (48):
      u3: use size_t in u3m_init()
      u3: normalize home-road stack after snapshot restoration
      u3: refactors custom allocator init for libraries
      bump
      vere: make `prep` and `next` commands non-interactive (no tty required)
      Merge pull request #5928 from urbit/jb/upgrade-no-tty
      u3: remove references to u3a_bits/u3a_bytes/words/pages
      u3: makes loom length configurable
      u3: improves image-related error messages in events
      Merge branch 'master' into next/vere
      u3: suppress bail:evil printf
      Merge pull request #5959 from urbit/jb/silence-evil
      ur: optimizes bitstream bytes-writer implementation
      ur: optimizes bitstream bytes-reader implementation
      Revert "test: move _test_lily into noun_tests"
      u3: rewrites u3r_chop(), optimizing bloq sizes < 5
      u3: removes loom byte-length in favor of recalculation
      Merge pull request #5957 from mopfel-winrux/eval
      vere: style clean for eval cmd
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge tag 'urbit-v1.12' into jb/fix
      Merge commit 'f94875f9ce^2' into jb/release/vere
      Merge commit '264e63330f^2' into jb/release/vere
      u3: backports u3v_soft_peek()
      vere: adds %vile command for extracting keys
      Merge commit '330db284e8^2' into jb/release/vere
      Merge branch 'master' into jb/release/vere
      test: bump loom to 16MB in jam tests
      u3: restages #6001 for release
      u3: adds loom size to u3m_boot() / u3m_boot_lite()
      vere: adds args to set mars/urth looms by binary exponent
      Merge branch 'jb/restage/loom-set' into jb/release/vere
      vere: supports --loom for all relevant subcommands
      Merge branch 'jb/loom-everywhere' into jb/release/vere
      Merge branch 'jb/release/vere' into jb/restage/chop-fast
      u3: bumps maximum reference bits on all platforms
      vere: preserve 2GB defaults for --loom and --loom-urth
      Merge branch 'jb/restage/chop-fast' into jb/release/vere
      Merge branch 'jb/restage/bump' into jb/release/vere
      Merge branch 'jb/fix' into jb/release/vere
      vere: bumps version
      vere: bumps benchmarks to 16MB loom, initializes guard page
      vere: supports --loom in eval cmd
      vere: bumps version

DaneBettis (22):
      Adds %meme and %xray hints.
      adds xray test, reorders tests, fixes sanity checks
      more notes and debugging for xray
      more xray debugs via ~master-morzod
      more xray debugs thanks to ~master-morzod
      partial fix for indexed bytecode args; more to come
      silenced the broken hoon tests
      cleaner expression and docs
      added macro to clarify _slog_bytecode and cleaned hoon hint tests
      first cleanup from meeting with master-morzod
      this branch PR is now for xray only. meme later.
      minimize the diff
      taming int declarations
      docs, tammed ints, better var names, removed redundant funcs
      fixes suggested by master-morzod, and finished int taming
      make _is_indexed() more obvious and less cute
      remove dev-time modifications to manage.c
      WIP stuff re master-morzod
      more refactoring as suggested by master-morzod
      corrected private function names and block comments
      fixed names of vars to follow cvc_TypeLetter style
      simplified xray callsites and rendering

mopfel-winrux (15):
      Added option to run the eval it checks for and prints whatever you pass it as an arguement
      Added basic nock increment evaulation to eval function
      Got basic hoon command compiling and executing
      Fixed a double boot_lite
      Added vase and stack trace wrapper to input, run hoo in virutalization
      added ability to read from stdin rather than being passed an arguement
      changed up how we are executing hoon. This fixes the issue we see with rpevious versions
      cleaned up code
      merged to master fixed conflict in motes.h
      fixed a few of the issues ~master-morzod pointed out
      changed error messages when booting the ivory pill
      changed how the stdin is concatenated. this version should have less memory leaks
      fixed memory leak and cleaned up style
      removed unneeded checks
      Merge branch 'urbit:master' into eval

pkova (1):
      vere: check for permission error on old process kill
2022-11-23 11:24:49 -05:00
Joe Bryan
2e57625e26 Merge tag 'urbit-v1.13' into jb/restage/pread-all
urbit-v1.13

Arvo 417K
Vere 1.13

This release makes the loom variably sized, up to a new maximum of 4GB.
The default remains 2GB -- opt for the max with `--loom 32`.
(NB: larger sizes may require adding swap space.)

Release Notes

- supports setting the loom size on startup. sizes are specified in
  exponents of 2, from 1MB (20) to 4GB (32); can differ between processes
  (`--loom` and `--urth-loom`); and can be set for relevant subcommands
  (`meld`, `pack`, `next`, &c).

- improves bit-slice performance (for noun de/serialization and many atom jets)
  by ensuring that implementation inner loops can be vectorized.

- adds the `vile` command, for exporting the keyfile from a ship

- adds the `%xray` hint, for printing the bytecode of a given expression
  (ty ~topfet-parmed!)

- adds the `eval` command, for running hoon (from stdin) without booting a ship
  (ty ~mopfel-winrux!)

- adds a better error message when a ship is already running as root
  (ty ~dinleb-rambep!)

Contributions:

Joe Bryan (48):
      u3: use size_t in u3m_init()
      u3: normalize home-road stack after snapshot restoration
      u3: refactors custom allocator init for libraries
      bump
      vere: make `prep` and `next` commands non-interactive (no tty required)
      Merge pull request #5928 from urbit/jb/upgrade-no-tty
      u3: remove references to u3a_bits/u3a_bytes/words/pages
      u3: makes loom length configurable
      u3: improves image-related error messages in events
      Merge branch 'master' into next/vere
      u3: suppress bail:evil printf
      Merge pull request #5959 from urbit/jb/silence-evil
      ur: optimizes bitstream bytes-writer implementation
      ur: optimizes bitstream bytes-reader implementation
      Revert "test: move _test_lily into noun_tests"
      u3: rewrites u3r_chop(), optimizing bloq sizes < 5
      u3: removes loom byte-length in favor of recalculation
      Merge pull request #5957 from mopfel-winrux/eval
      vere: style clean for eval cmd
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge tag 'urbit-v1.12' into jb/fix
      Merge commit 'f94875f9ce^2' into jb/release/vere
      Merge commit '264e63330f^2' into jb/release/vere
      u3: backports u3v_soft_peek()
      vere: adds %vile command for extracting keys
      Merge commit '330db284e8^2' into jb/release/vere
      Merge branch 'master' into jb/release/vere
      test: bump loom to 16MB in jam tests
      u3: restages #6001 for release
      u3: adds loom size to u3m_boot() / u3m_boot_lite()
      vere: adds args to set mars/urth looms by binary exponent
      Merge branch 'jb/restage/loom-set' into jb/release/vere
      vere: supports --loom for all relevant subcommands
      Merge branch 'jb/loom-everywhere' into jb/release/vere
      Merge branch 'jb/release/vere' into jb/restage/chop-fast
      u3: bumps maximum reference bits on all platforms
      vere: preserve 2GB defaults for --loom and --loom-urth
      Merge branch 'jb/restage/chop-fast' into jb/release/vere
      Merge branch 'jb/restage/bump' into jb/release/vere
      Merge branch 'jb/fix' into jb/release/vere
      vere: bumps version
      vere: bumps benchmarks to 16MB loom, initializes guard page
      vere: supports --loom in eval cmd
      vere: bumps version

DaneBettis (22):
      Adds %meme and %xray hints.
      adds xray test, reorders tests, fixes sanity checks
      more notes and debugging for xray
      more xray debugs via ~master-morzod
      more xray debugs thanks to ~master-morzod
      partial fix for indexed bytecode args; more to come
      silenced the broken hoon tests
      cleaner expression and docs
      added macro to clarify _slog_bytecode and cleaned hoon hint tests
      first cleanup from meeting with master-morzod
      this branch PR is now for xray only. meme later.
      minimize the diff
      taming int declarations
      docs, tammed ints, better var names, removed redundant funcs
      fixes suggested by master-morzod, and finished int taming
      make _is_indexed() more obvious and less cute
      remove dev-time modifications to manage.c
      WIP stuff re master-morzod
      more refactoring as suggested by master-morzod
      corrected private function names and block comments
      fixed names of vars to follow cvc_TypeLetter style
      simplified xray callsites and rendering

mopfel-winrux (15):
      Added option to run the eval it checks for and prints whatever you pass it as an arguement
      Added basic nock increment evaulation to eval function
      Got basic hoon command compiling and executing
      Fixed a double boot_lite
      Added vase and stack trace wrapper to input, run hoo in virutalization
      added ability to read from stdin rather than being passed an arguement
      changed up how we are executing hoon. This fixes the issue we see with rpevious versions
      cleaned up code
      merged to master fixed conflict in motes.h
      fixed a few of the issues ~master-morzod pointed out
      changed error messages when booting the ivory pill
      changed how the stdin is concatenated. this version should have less memory leaks
      fixed memory leak and cleaned up style
      removed unneeded checks
      Merge branch 'urbit:master' into eval

pkova (1):
      vere: check for permission error on old process kill
2022-11-23 11:24:27 -05:00
Joe Bryan
dac240bdfa vere: bumps version 2022-11-23 10:38:02 -05:00
Joe Bryan
2e861dafcf vere: supports --loom in eval cmd 2022-11-22 15:38:15 -05:00
Joe Bryan
670ebc8f8c vere: bumps benchmarks to 16MB loom, initializes guard page 2022-11-22 15:23:29 -05:00
Joe Bryan
6c4b102815 vere: supports --no-demand for all relevant subcommands 2022-11-18 18:38:09 -05:00
Joe Bryan
f40db733ba vere: bumps version 2022-11-18 17:44:46 -05:00
Joe Bryan
39ca2e5fe2 Merge branch 'jb/fix' into jb/release/vere 2022-11-18 17:44:02 -05:00
Joe Bryan
efd49ed338 Merge branch 'jb/restage/bump' into jb/release/vere 2022-11-18 17:43:57 -05:00
Joe Bryan
1ec977aede Merge branch 'jb/restage/chop-fast' into jb/release/vere 2022-11-18 17:43:46 -05:00
Joe Bryan
320740c5b9 vere: preserve 2GB defaults for --loom and --loom-urth 2022-11-18 17:41:54 -05:00
Joe Bryan
fd7e84e26d u3: bumps maximum reference bits on all platforms 2022-11-18 17:41:50 -05:00
Joe Bryan
f78e6a5447 Merge branch 'jb/release/vere' into jb/restage/chop-fast 2022-11-18 10:41:23 -05:00
Joe Bryan
0f7feef5c7 vere: adds --no-demand argument to disable demand paging 2022-11-18 00:11:21 -05:00
Joe Bryan
b102cbb908 u3: adds printf ("live: mapped ...") for demand paging 2022-11-18 00:08:55 -05:00
Joe Bryan
1b5e808ec5 u3: implements full demand paging for the home-road heap 2022-11-18 00:07:23 -05:00
Joe Bryan
a04521585c u3: coalesces memory protection when saving a snapshot 2022-11-18 00:03:13 -05:00
Joe Bryan
0682cc2864 u3: refactors _ce_loom_blit_north(), replacing loop with a single read 2022-11-18 00:00:27 -05:00
Joe Bryan
0959bb60f3 u3: coalesces memory protection when loading a snapshot 2022-11-17 23:58:39 -05:00
Joe Bryan
622f86e714 u3: misc cleanup of snapshot system
corrects comments
removes dead code
makes page and offset calculations more consistent
improves error messages
2022-11-17 23:46:52 -05:00
Joe Bryan
97e7a14123 u3: restages #6062 for release 2022-11-17 23:44:12 -05:00
Joe Bryan
5c3daa265d Merge branch 'jb/loom-everywhere' into jb/release/vere 2022-11-17 22:41:25 -05:00
pkova
2300d4fbcb vere: check for permission error on old process kill 2022-11-17 22:30:53 -05:00
Joe Bryan
255c08f53d vere: supports --loom for all relevant subcommands 2022-11-17 20:42:48 -05:00
Joe Bryan
a7de786d01 Merge branch 'jb/restage/loom-set' into jb/release/vere 2022-11-17 19:54:47 -05:00
Joe Bryan
12a7fae0b5 vere: adds args to set mars/urth looms by binary exponent 2022-11-17 19:54:21 -05:00
Joe Bryan
7ae6005f87 u3: adds loom size to u3m_boot() / u3m_boot_lite() 2022-11-17 17:44:16 -05:00
Joe Bryan
0f0ab948f1 u3: restages #6001 for release 2022-11-17 17:39:09 -05:00
Joe Bryan
3fe55a9f84 test: bump loom to 16MB in jam tests 2022-11-17 15:46:29 -05:00
Joe Bryan
1c6aba94f2 Merge branch 'master' into jb/release/vere 2022-11-17 15:38:11 -05:00
Joe Bryan
5841d97832 Merge commit '330db284e8^2' into jb/release/vere 2022-11-17 14:57:59 -05:00
Joe Bryan
31b9a4b470 vere: adds %vile command for extracting keys 2022-11-17 14:57:05 -05:00
Joe Bryan
4d57c3c9a0 u3: backports u3v_soft_peek() 2022-11-17 14:57:04 -05:00
Joe Bryan
d0dc3e7710 Merge commit '264e63330f^2' into jb/release/vere 2022-11-17 14:36:01 -05:00
Joe Bryan
bda40ac64b Merge commit 'f94875f9ce^2' into jb/release/vere 2022-11-17 14:34:55 -05:00
Joe Bryan
4b70f0a895 Merge tag 'urbit-v1.12' into jb/fix
urbit-v1.12

Arvo 417K
Vere 1.12

This is a hotfix release, fixing another memory corruption vulnerability
introduced in v1.10 and improving snapshot durability.

Release Notes

- restores the guard page (preventing road stack overflow) after |meld
- improves snapshot error handling and messaging
- fixes an overflow bug affecting jammed scry output (-X)

Contributions:

Joe Bryan (16):
      u3: protect guard page in u3e_yolo()
      vere: bumps version
      u3: assert guard page invariants when saving snapshot
      u3: initialize guard page in u3m_boot_lite()
      vere: bumps version
      nix: update linux-aarch64 overlay to exclude macos m1
      u3: handle partial reads in snapshot system
      u3: handle partial writes in snapshot system
      u3: print mprotect errors in snapshot system
      u3: print error msg if system page size is incompatible
      u3: detect snapshots from a larger loom, print and exit
      u3: normalize home-road stack after snapshot restoration
      u3: handle and print lseek errors in snapshot system
      u3: print errors while deleting snapshot patch
      vere: bumps version
      vere: bumps version

Liam Fitzgerald (1):
      pier: fix jamming archive
2022-11-15 14:55:02 -05:00
Joe Bryan
12196d1113 Merge tag 'urbit-v1.12'
urbit-v1.12

Arvo 417K
Vere 1.12

This is a hotfix release, fixing another memory corruption vulnerability
introduced in v1.10 and improving snapshot durability.

Release Notes

- restores the guard page (preventing road stack overflow) after |meld
- improves snapshot error handling and messaging
- fixes an overflow bug affecting jammed scry output (-X)

Contributions:

Joe Bryan (16):
      u3: protect guard page in u3e_yolo()
      vere: bumps version
      u3: assert guard page invariants when saving snapshot
      u3: initialize guard page in u3m_boot_lite()
      vere: bumps version
      nix: update linux-aarch64 overlay to exclude macos m1
      u3: handle partial reads in snapshot system
      u3: handle partial writes in snapshot system
      u3: print mprotect errors in snapshot system
      u3: print error msg if system page size is incompatible
      u3: detect snapshots from a larger loom, print and exit
      u3: normalize home-road stack after snapshot restoration
      u3: handle and print lseek errors in snapshot system
      u3: print errors while deleting snapshot patch
      vere: bumps version
      vere: bumps version

Liam Fitzgerald (1):
      pier: fix jamming archive
2022-11-15 14:43:12 -05:00
Joe Bryan
d07e31ac60 vere: bumps version 2022-11-15 14:07:27 -05:00
Joe Bryan
885eb2fea1 Merge branch 'jb/hotfix' into jb/fix 2022-11-14 23:36:29 -05:00
Joe Bryan
215f6ce650 vere: bumps version 2022-11-14 23:30:29 -05:00
Joe Bryan
58805b8cef u3: print errors while deleting snapshot patch 2022-11-14 23:18:43 -05:00
Joe Bryan
284addf159 u3: handle and print lseek errors in snapshot system 2022-11-14 23:15:11 -05:00
Joe Bryan
4f5f7e2804 u3: normalize home-road stack after snapshot restoration 2022-11-14 23:15:10 -05:00
Joe Bryan
2a1a707900 u3: detect snapshots from a larger loom, print and exit 2022-11-14 23:15:10 -05:00