We were unconditionally adding equal items to the hashtable of
already-compared pointers. This table exists so that if we've already
determined two non-pointer-equal nouns are value-equal, we don't have to
check them again. However, atoms (especially direct) ended up in this
hashtable even though we don't look those up. This makes sure we don't
add items to the hashtable if they're "pointer"-equal (which for direct
atoms is value-equal).
The impact of this inefficiency was greatly magnified by the issue with
+mug where (mug a a) = (mug b b) for all a, b. For this reason, these
identical pairs added to the hashtable always had the same mug, so they
were added to the same bucket, which meant adding to that list required
linearly traversing the entire bucket.
This was the first barrier that was causing |pack to take a long time on
ships which distribute OTAs, but it isn't a complete solution for |pack.
shipResults would remove the ability to search through contacts, but it
would still produce a ship if it was a valid patp with no results --
even for group-only search contexts.
Fixes#3118. Because we nest the data in a weather key and save that
in app state, we don't need to nest the data in weather twice over when
giving initial data to new subscribers.
Fixes#3096. During the migration to global store, the API became
structured such that you must specify the local (app-specific) API.
Some cases were missed. This includes them.
In Ford Fusion, Clay builds generators but Dojo and Eyre run them. Dojo
is already virtualized with a scry function, so +mule is fine, but Eyre
is not, so Eyre needs to use +mock and explicitly supply the scry
function. This does that. Fortunately, the produced result is simple
and easily clammable.
Fixes#3089
* ford-fusion: (259 commits)
clay: remove scaffolding
kiln: make otas continue even if they failed to apply
metadata-store: add cleanup utility, use on-poke:def instead of no-op
clay: flop syntax error trace
landscape/img/codeeval.png: typo in filename
invite-view: reinstate as potato
goad: don't crash on pre-OTA sign
chat-store: responded to comments, cleaned up
-test: support other desks (full beams in args)
arvo: use date instead of kelvin
hoon: re-fix +slab; /tests: fix clay tests
kiln: don't implicitly create syncs
gall: don't make large stack trace
ames: use +cut in +encrypt
pill: solid
dojo: print generator type errors
Squashed commit of the following:
ci: fix herb tests; update pills
ford,kiln: nicer errors
dojo: too many nouns
...
Signed-off-by: Philip Monk <phil@pcmonk.me>
* master:
vere: bump version to 0.10.7
libsigsegv: disable stack vma check
vere: bump version to 0.10.6
ci: add travis as trusted user
jets: use appropriate macro
noun: add -C to control memo cache size
jets: restore fond/play/peek hooks
jam: add commented-out functionality to count size of atom
jets: cap memo cache and remove peek, play, and fond jets
noun: add functions to count size of noun
* master:
vere: bump version to 0.10.7
libsigsegv: disable stack vma check
vere: bump version to 0.10.6
ci: add travis as trusted user
jets: use appropriate macro
noun: add -C to control memo cache size
jets: restore fond/play/peek hooks
jam: add commented-out functionality to count size of atom
jets: cap memo cache and remove peek, play, and fond jets
noun: add functions to count size of noun
No longer abuse the desk field, instead making use of the path. Reject
any scries outside of the local ship, empty desk and current time as
invalid.
Expose ducts only under a debug endpoint, nothing else should care about
being able to inspect them.
Add scry endpoints for the very next timer (if any), and all timers up
to and including a specified timestamp.
* master: (147 commits)
vere: bump version to 0.10.7
libsigsegv: disable stack vma check
vere: bump version to 0.10.6
ci: add travis as trusted user
jets: use appropriate macro
noun: add -C to control memo cache size
jets: restore fond/play/peek hooks
jam: add commented-out functionality to count size of atom
jets: cap memo cache and remove peek, play, and fond jets
noun: add functions to count size of noun
release: urbit-os-v1.0.23
interface/config: fix production build
soto: run +on-load migration once
publish, links: restore full height
sh/build-interface: amend for SPA
interface/CONTRIBUTING: amend for SPA / webpack
solid: update pill
hood + apps: fix OTA process for feat/SPA
hood: add version %6 for %file-server upgrade
chat: equally size both code + s3 buttons
...
* ipc-redux:
behn: optimize bounded timers scry
vere: support saving scry jam to directory
vere: u3_nul in place of c3__null
vere: if behn scry fails, don't try again
vere: rename behn.c's alm -> alm_o
vere: scry out next behn timer for backstop
vere: warn on invalid behn doze
behn: improve scry interface
arvo: allow the empty desk (%$) in scries
vere: add -X flag for running a scry
Instead of potentially waiting ten minutes in the problematic case, we
scry out the next timer from behn and set to that (if we haven't set
a new timer while we were waiting for the scry).
No longer abuse the desk field, instead making use of the path. Reject
any scries outside of the local ship, empty desk and current time as
invalid.
Expose ducts only under a debug endpoint, nothing else should care about
being able to inspect them.
Add scry endpoints for the very next timer (if any), and all timers up
to and including a specified timestamp.
Tries to get a scry result from a /vanecare/desk/path formatted path,
and jams the result to disk (.urb/put/) if it succeeds.
Optionally use -Y to specify a name for the resulting file.
The component for editing a post was tied up in checking for API
instantiation -- we want the check for a blank body to be independent
of that.
Fixes#3040.
This is a convenient way to count memory usage of noun by simplying
running `(jam 1.337 noun-1 noun-2 ... ~)`. This should
be a hint, but for debugging this is sufficient.
With these changes, about 90% less memory and 15% less time is needed to
compile hoon.hoon. The produced noun is within 3% of the same size,
which suggests this results in little if any duplication.
These are three of the four most commonly hit +ut jets. The other is
+nest, which cannot be un-memoized without taking much longer to compile
(it didn't finish in my test). These four jets combined for 2.3 million
out of the 2.4 million cache entries, the other +ut jets combine for
less than 100k, and literal ~+ accounted for about 50k entries.
This also caps the memo cache at 50k entries. Even with these jets not
memoized, the memo cache grows to 357k entries and 122 MB. Capping at
50k entries has no effect on time and reduces memory usage of the hash
table to about 25MB. Entries are reclaimed with the clock algorithm,
which seems to be sufficient for this use.
Adds a few functions to count the size of nouns in the current road.
Since this marks the nouns (high bit of refcount), you need to
"discount" them immediately after to unmark them. Parallel functions
exist for the counting the size of a hashtable.
It would nice to hook this up to a hint, but these are useful to have
available to run in the debugger or by inserting callsites as necessary.
It's also possible to hook them up to the +jam jet gated on a special
value.
