Instead of potentially waiting ten minutes in the problematic case, we
scry out the next timer from behn and set to that (if we haven't set
a new timer while we were waiting for the scry).
No longer abuse the desk field, instead making use of the path. Reject
any scries outside of the local ship, empty desk and current time as
invalid.
Expose ducts only under a debug endpoint, nothing else should care about
being able to inspect them.
Add scry endpoints for the very next timer (if any), and all timers up
to and including a specified timestamp.
Tries to get a scry result from a /vanecare/desk/path formatted path,
and jams the result to disk (.urb/put/) if it succeeds.
Optionally use -Y to specify a name for the resulting file.
This is a convenient way to count memory usage of noun by simplying
running `(jam 1.337 noun-1 noun-2 ... ~)`. This should
be a hint, but for debugging this is sufficient.
With these changes, about 90% less memory and 15% less time is needed to
compile hoon.hoon. The produced noun is within 3% of the same size,
which suggests this results in little if any duplication.
These are three of the four most commonly hit +ut jets. The other is
+nest, which cannot be un-memoized without taking much longer to compile
(it didn't finish in my test). These four jets combined for 2.3 million
out of the 2.4 million cache entries, the other +ut jets combine for
less than 100k, and literal ~+ accounted for about 50k entries.
This also caps the memo cache at 50k entries. Even with these jets not
memoized, the memo cache grows to 357k entries and 122 MB. Capping at
50k entries has no effect on time and reduces memory usage of the hash
table to about 25MB. Entries are reclaimed with the clock algorithm,
which seems to be sufficient for this use.
Adds a few functions to count the size of nouns in the current road.
Since this marks the nouns (high bit of refcount), you need to
"discount" them immediately after to unmark them. Parallel functions
exist for the counting the size of a hashtable.
It would nice to hook this up to a hint, but these are useful to have
available to run in the debugger or by inserting callsites as necessary.
It's also possible to hook them up to the +jam jet gated on a special
value.
* origin/jb/aes-siv-fix:
tests: updates aes-siv regression test comment
pill: updates solid
zuse: propagates fix to aes-128-siv and aes-192-siv as well
Revert "test: disable aes-siv jets to demonstrate test failure"
pill: updates solid
zuse: fixes bug in aes-256-siv iv calculation (+s2vc:aes:crypto)
test: disable aes-siv jets to demonstrate test failure
test: add test case for aes-256-siv jet mismatch, observed in the wild
Signed-off-by: Philip Monk <phil@pcmonk.me>
Many ships have been observed bumping up against the existing mapsize
limits. This results in a Vere crash via LMDB, which necessitates
compiling a new binary with a higher mapsize if one wants to relaunch.
There doesn't seem to be any serious penalty to setting this
somewhere in the terabyte range, though. [1] In cases where the mapsize
exceeds the size of the disk, I infer from the LMDB docs that the
database may simply be permitted to grow until it runs up against the
disk limitations, which feels acceptable.
I've tested this on macOS and Linux and the binary runs without issue,
despite the mapsize being set far in excess of the disks I'm running on.
[1]: https://lmdb.readthedocs.io/en/release/
Adds +mure to run a trap in a separate road. This should eventually be
just a hint.
Vega was running inside a mule, but since +load was called within vega,
the new kernel was all run within the same mule, so it didn't actually
get to reclaim the space after hoon compiled.
We verified this with printfs in u3m_fall. On the test ship (from
mainnet) which had 800MB used, vega was taking interior free space from
950MB to 450 over the course of compiling hoon, then each vane would go
from about 450 to 350 and then back to 450 once it finished (which
proves they were correctly isolated). With this change, after hoon
compiles the free space goes back up to 950MB. This gives us a lot more
space to compile OTAs.
We had to slightly refactor the logic for doubly-recompiling hoon, since
+mure as written produces a ?(!! _trap), and you can't find faces in the
result of the trap. We could bake mure, but that's rather awkward. I
wonder if there's a way to fix this as a wet gate.
