tid was accidentally getting set to the name of the output mark. As we
don't currently support cancelling threads, there is no reason to
maintain the originally-intended correspondence between tid and conn
request-id.
Take the opportunity to clean up indentation somewhat.
Also strips out `$` from khan top-level comment.
There are arguments for keeping $crag in lull, and on the other side for
moving $cast to arvo. This seemed like the most reasonable approach.
%fyrd is now implemented in terms of %fard, and likewise %avow in terms
of %arow. State is tracked via wire rather than in a global map.
Unit tests adjusted to match.
These take and produce vases, and assign random tids (rather than
deducing them from the input duct.)
Since %fard does not require mark conversion, we make the mark/beak on
$thread-state optional (and use this to decide whether to send %avow or
%arow.) Provide a state adapter since it's possible that people have
been experimenting with this vane.
This makes the negative case of %avow/%arow kind of clunky, since there
is no content difference, but the following does not seem possible
within the Hoon type system:
=/ gif
?~ p.tad
%arow %avow
[hen %give gif %| p.cag tang]~
- use desk parameter instead of %base everywhere
- formatting clean up
- make |story-remove take a case instead of an aeon
- make desk param optional for story-set and story-log
- only store metadata in the persistent map. just enough to support
(eventual) thread cancellation and output mark lookup.
- try to delete thread state at other failure points not covered by
%kick.
- reflect back the passed output mark rather than form.dais. not sure
about this one yet.
This change greatly improves the ergonomics of working with channel JSON
in statically typed languages, as the polymorphism is moved out of the
actual diff and into the event framing.
+wake had accumulated several layers of abstractions which were later
rendered unnecessary. This removes those abstractions and should have
no semantic effect.
This adds support for tombstoned files to clay. It does not include any
way to actually tombstone them; that is left for later.
This allows tombstoning at the level of a file. Precisely, this expands
+blob:clay by adding a %dead case:
+$ blob :: fs blob
$% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q
[%direct p=lobe q=page] :: immediate
[%dead p=lobe ~] :: tombstone
== ::
Thus, we maintain the invariant that every lobe corresponds to a blob,
but now a blob may be an explicit tombstone.
Details:
- This has not been tested at all, except that it compiles and boots.
- This does not have a state adapter from master. The only state change
is the definition of +cach.
- Additionally, out-of-date ships may unexpectedly receive a %dead blob
from a foreign clay which would interfere with their ability to download
that desk. No code changes necessary, but sponsors should avoid
tombstoning files in %base for a while so their children can get the
update.
- A merge will only fail if the tombstoned file conflicts with another
change. Note that as written, merging from a past desk *can* bring a
tombstoned file to the head of a desk. Possibly this shouldn't be
allowed.
This also includes a couple refactors that were made possible by ford
fusion (since everything is synchronous now) but never got done. In
both cases we get to remove a monad, which simplifies the code
considerably.
- refactor +merge's error handling to use !!/mule instead of threading
through errors
- refactor all +read-* functions and related parts of +try-fill-sub to
eagerly convert lobes to cages.
We also add support reading %a/b/c/e/f/r/x from past and foreign desks,
when possible. Apologies that all of these are in one commit, it was
all a single chunk of work.
This is a draft until we have a way to tombstone. I suspect we'll want
to have a mechanism of keeping track of gc roots and trace to remove,
but this PR doesn't suggest any particular strategy.
Jael needs to be reconfigured to listen to the new aagent for azimuth
events, and the old app needs to be shut down. We do this in
/app/azimuth's +on-init.
Additionally, we make sure that jael doesn't crash when it (as expected)
loses its subscription to the old agent.
When you loaded an app with an error, then fixed the error, it would
create the main gall %mult subscription at a time in the past. Then,
clay would never fill the subscription since it couldn't get the old %a
entries for the apps.
This fixes the issue in two ways: first, don't subscribe in the past.
Second, if clay can't get the old versions, just fire the subscription
anyway.
Previously, if trying to bind to an endpoint that was already bound to,
eyre would reject it. This doesn't play very nicely in a softdist world
where uninstalled apps might not get a chance to clean up, and apps
might re-bind simply for being re-installed.
Here we change eyre to overwrite an existing binding if it conflicts
with the new one to be added.
As SSE are unidirectional, the client always realises that the
connection has failed faster than the server does. Hence, resuming a
subscription is useless, because channels can only be bound to one duct
at a time. Now, instead of failing a request for a channel
that is already bound to a duct, we replace the duct and continue
normally.
Start with |start %desk %app-name
Everywhere in the kernel that we deal with marks, we infer the app it's
connected to and use the marks from that desk.
Also some light renaming in gall, especially path->wire and
current-agent->yoke.
Subsequent tasks:
- Dojo needs a syntax to run generators and threads from other desks
- The home desk should be split into at least a minimal base desk and
big "userspace" desk. Dill's initialization logic should be updated
to handle
- |show-package, |install, and |uninstall should to be written
- Clay should have smarter handling of system versions instead of just
ignoring what's on each desk. It's not clear that this will work
correctly when sys updates right now.
Improves the multikeyfile format by taking a single ship and a list of
life+key pairs, instead of a list of full seeds.
Also decouples these changes from the dawn event, once again putting a
single seed into it. In the multikeyfile case, keys are injected as
%rekey events to jael near the end of the boot sequence.
Haskell-side changes may or may not be incomplete, boot presently fails
at some unknown point with what looks like a noun conversion error.
Allows booting with a keyfile containing multiple keys, as long as one
of them matches current PKI pubkeys for the specified ship.
All relevant keys are loaded into jael and will automatically be put to
use when they match PKI state.
Four changes:
- implement +validate-u to allow %u requests over the network
- make +validate-x use our local marks to make %x requests generally
work over the network
- in +start-request, if a foreign ship is making a request that we
shouldn't send over the network, ignore it. This closes a DOS vector.
- in +duce, if we're about to make a request to a foreign ship which
they won't be able to answer, crash the event.
Combined, these fix many of the common cases of weirdness around foreign
clay requests. Notably absent is a fix for reading `%a` across the
network, which I still maintain should happen against the foreign
hoon/zuse.
fixes#4834
see also #4307
Also switches everything to ropsten by default, including ivory pill.
Batches work on ropsten now.
Also adds +tx as a hacky development tool to create text for metamask to
sign and then turn that into a batch. A useful reference for bridge and
aggregator work.
By factoring their shared logic out into +build-dependency, which gets
passed the relevant details about how to track the file being built in
the dependency stack.
Hoon files may want to import nouns from all files in a given directory.
/~ lets you do so, importing as a (map @ta *) (but with typed values).
Note the description as "directories" here, instead of "path prefix".
The behavior, as implemented, will not include /path/hoon for /~ /path,
instead only including /path/more/hoon and more deeply nested files.
This seems to be, generally, the behavior you want, for example when
importing from /app/myapp/* for /app/myapp/hoon.
Actually using the resulting map requires some manual casting, which is
not ideal. Some code style improvement work remains to be done as well.
Avoid allocating hundreds of thousands of cells when giving large
requests. This took the footprint of this function on initial landscape
load from 1 second to 100 ms.
We can't molt until clay has gotten its pork or else we'll build the old
app against the new kernel. This ignores vegas, since we should get a
notification from clay on /sys/lyv.
When we changed wires from /a/foo to /ames/foo, our sorting function
started sorting by last character instead of first character, so breach
notifications were given to gall before ames. This made gall try to
resubscribe before ames cleared its state, so the message would be lost.
Fixes#4177
This had regressed during some breach-related merge. Multiple commits/branches
had touched this codepath recently, eating the code step change introduced in
#3217.
Fixes#4126.
* jb/motion:
pill: solid
zuse: remove %crud from vane-task
arvo: full vane names in $sign
aqua: build again (still broken)
arvo: reform of the scry reform
When you first boot, if you try talk to someone before your azimuth is
up-to-date (for example by import), then if they've ever breached
(twice) then you'll get breach notification, cancelling your message.
This changes is it so that if we haven't heard anything about this ship,
we don't signal a breach.
The implementation complexity is primarily because we need
eth-watcher/azimuth-tracker to produce an update of a list instead of a
list of updates. This way, Jael can keep a "state as of the beginning
of this move" variable to check when deciding whether to signal a
breach.
* na-release/candidate:
kh: use Word8 for Tint true color values
arvo: remove unused app files, libraries, and imports
webterm: improve line-spacing in certain browsers
vere: avoid +scot call for color value rendering
kh: support 24-bit %klr colors
vere: support 24-bit %klr colors
webterm: update mar and js to support 24-bit color
tests: fix ames tests
pill: update ivory pill
dojo: correct mark conversion scry path
pill: solid
aqua/ph: fix comet test
ames: flat packet format
hoon, dill: Add 24-bit true color
+riff-any is all clay requests except "backfill" requests. Change to
`$%` from `$^`, which was used to distinguish originally non-versioned
requests.
+fill is backfill requests and had no version number, so we add one.
We do not have version numbers on responses since those are implied by
the request. If someone requests at version `n` and you're at `n+1`,
you must respond in the format of `n`.
If someone requests at version `n+1` and you're at `n`, you crash;
though possibly you should be able to respond with message "I only know
up to `n`", in which case they may be able to re-request at `n`. In
either case, the version of the response is dictated by the request.
Unflops the spur in +en-beam, +de-beam, and everything that calls either
of those, or works with the consequences of their output.
This includes clay's interface for mounting and unmounting, which now
no longer expects the arguments to contain an old-style spur.
* master: (390 commits)
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
interface: links and publish comments both work
...
* na-release/next-vere: (943 commits)
pill: solid
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
...
* master: (390 commits)
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
interface: links and publish comments both work
...
* release/next-vere: (1369 commits)
nix: fixes `shellFor` nix-shell helper
vere: print error and exit if stdin is not a tty
build: silence service account activation output
build: minor refactoring of haskell-nix overlays
build: move darwin install_name_tool fixup from vere to king haskell
u3: fixes incorrect double ref-counting in |ff jets
u3: removes unused `Exit` variable
u3: removes obsolete bail:need assertion
u3: refactors fatal exception handling in u3m_bail()
build: remove {sha256,md5} output for push-storage-object effects
build: add log message when destination object already exists
build: force google-cloud-sdk to use python3
build: adding support for hercules ci effects
build: remove push-to-storage for ivory, brass, and solid pills
pill: rebuild solid pill with %lens included in lite boot apps
arvo: run %lens when lite boot (-l) is specified
build: expose configurable arguments when booting/testing fake ships
build: ensure urbit tests are run with the -g argument
vere: ensure debug symbols aren't stripped by default (by nix)
build: remove from-scratch ropsten pill builds on ci
...
Eyre's clog logic was a tad inconsistent about "only facts" vs "not poke-acks".
This makes it consistently say "only facts" when it comes to clog-related logic.
Yes, in theory this means %watch-acks and %kicks can build up endlessly, but
those should take up negligible space compared to %facts.
Should fix any oddball cases of crashes here that #3835 didn't already catch.
This was a little bit too crummy. Instead, we put in a placeholder of ~,
which should be forwards-compatible with atomic session identifiers,
where ~ identifies the default session.
Additionally touches up the herm wires/paths to stick to the above more
closely.
This lets us support the "random userspace app sending dill belts".
Ultimately, we'll want to be able to specify a session identifier
alongside the belt, instead of strictly relying on the duct.
Adds a %view task, which opens a subscription on the output sent to the
specified session. %flee closes the same.
Whenever dill sends a blit to the session, any subscribers get the
output also.
The structures here will become more reasonable once we replace ducts
with proper dill session identifiers.
People using older runtimes might not support the %klr blit. It's not
uncommon for prompts without style to get passed in as %pom though, so
here we catch that case and turn it into a %pro, which gets rendered as
a traditional %lin.
Pretty-printing is expensive, yet we do it whenever we construct the cookie
string, at least once (but usually twice) per authenticated request.
Here we call out the the specific to-tape functions we need, instead of relying
on the pretty-printer for converting... tapes to tapes, among other things.
The primary gains come from the cookie-related instances, we update the others
mostly for good style.
For the "receive request and immediately send response" case, that is processed
synchronously within eyre (ie, client sends channel ack), speeds thing up by
roughly 55%.
Motivation for the change is performance improvements on the un-`^~`d uses of
ream. Parsing turns out to be slow, making ream slow in turn. So we construct
the hoon ast manually instead.
!, is arguably better style than ream, since it doesn't require a ^~ for static
input, and lets syntax highlighting function properly.
For the investigated case, in +get-cast's +grow flow, improves performance by
over 80%.
If the Forwarded header specifies the original connection is secure,
update the flag to reflect that, regardless of whether the connection
directly to the urbit was made securely.
When an application would send multiple facts during a single event, it
was possible for the first fact to trigger a clog, removing the
subscription and sending a quit, but then the second fact still getting
sent out at normal.
Here, we drop any facts for subscriptions we don't have registered in
state, which should only happen in the described case.
Because storing in reverse order means producing in reverse reverse
order.
The tests didn't catch this because they, too, were infected with the
"reverse moves" meme.
In order to curb event queue growth when a client for whatever reason
isn't acking the events we send out, we implement a mechanism for
detecting such "clogging", and proactively kick subscriptions which are
adding too many events to the queue.
If the client hasn't sent an ack for ~s30, any subscription that accrues
more than 50 unacked %facts gets closed to prevent further buildup.
Upon reconnecting, the client will see %kick for the relevant
subscriptions and can open a new subscription as appropriate.
Includes a simple test for this behavior, and updates /app/dbug to be
able to display the newly tracked statistics.
