Previously we were dropping events that used old
wires that lacked a rift in them. This seems a
bad behavior because we don't want to destroy a
flow that has not been processed by both ends.
Note: pending a fix to test-old-ames-wire
This converts the blob store from having deltas, directs, and
tombstones, to just having direct pages. This simplifies a lot of code,
since we don't have to constantly ensure that deltas always have their
parent available.
This removes the hardcoded text diff logic from clay, which was
previously required for bootstrapping.
Over the wire, we handle both old and new requests and responses
transparently, so communication is normal in both directions across
ships which do or do not have this change.
this was wiping out some comments buried that weren't written at the top
of an arm. not sure if this is used in another fashion that will create
issues, so I just commented it out to help remember that it used to be
there in case it needs further changes, like if I should actually skip
%know and %made notes but not %help notes
We had trie operations independently implemented in +de in arvo,
+an:cloy in zuse, +zu in clay, lib/trie, and app/spider. This unifies
them all into +de in arvo, aggregating the used operations.
docs written above an arm are now distinguishable in the AST from
docs written above the product of the arm, by tagging docs written
above the arm with a %funk link
This crashed at runtime when `a` is null because it tried to instantiate
`meg`, and that means bunting its argument, and that calls `node`, where
the assert failed.
Wet gates use the bunt of their formal argument, so we use that.
%rez has always used "width & height". Certainly, "x & y" is more
standard than "row & column". As such, we settle on making %hop and %hit
respect the more natural ordering. This change is safe because these
interfaces haven't made it to livenet yet.
This fixes a space leak where the entire ford/clay core would be
included in the ford cache. Heavily reduces memory usage by clay,
reducing total usage from 100-200MB to around 10MB.
Instead of reporting a single memory size for built files, marks and
conversions, we now report memory size per path, mark name and mark
pair, respectively.
This changes the parser for +tall so that it looks before and after a
hoon for doccords, and then extracts a label for %brcn if it exists.
+wrap will be used to annotating most hoons, but this commit only covers
%brcn
Too often when dealing with big types the compiler traces and other such
outputs become hard to read. Wrapping a type as $+(shorthand big-type)
will now print #shorthand in place of the type.
Too often when dealing with big types the compiler traces and other such
outputs become hard to read. Wrapping a type as $+(shorthand big-type)
will now print #shorthand in place of the type.
instead make it a rolling 128-bit integer. 128 bits is the same size as
the +sham space, so is one natural choice for "big enough to never have
to think about this." 64, 32, 16, even 8 bits would probably be fine.
Threads should eventually take and produce $cage instead of $vase. Since
%khan is likely to be used by third parties, we write to the eventual
intended API. We ignore the mark on the input $cage (it is safe to
always specify %noun), and we always use %noun as the output mark.
%fyrd now makes more sense. It was previously discarding the type of the
output %arow and re-encoding the raw noun as a vase of the output mark;
it is now performing mark conversion from the mark of the output $cage
to the originally requested output mark.
tid was accidentally getting set to the name of the output mark. As we
don't currently support cancelling threads, there is no reason to
maintain the originally-intended correspondence between tid and conn
request-id.
Take the opportunity to clean up indentation somewhat.
Also strips out `$` from khan top-level comment.
There are arguments for keeping $crag in lull, and on the other side for
moving $cast to arvo. This seemed like the most reasonable approach.
%fyrd is now implemented in terms of %fard, and likewise %avow in terms
of %arow. State is tracked via wire rather than in a global map.
Unit tests adjusted to match.
These take and produce vases, and assign random tids (rather than
deducing them from the input duct.)
Since %fard does not require mark conversion, we make the mark/beak on
$thread-state optional (and use this to decide whether to send %avow or
%arow.) Provide a state adapter since it's possible that people have
been experimenting with this vane.
This makes the negative case of %avow/%arow kind of clunky, since there
is no content difference, but the following does not seem possible
within the Hoon type system:
=/ gif
?~ p.tad
%arow %avow
[hen %give gif %| p.cag tang]~
- use desk parameter instead of %base everywhere
- formatting clean up
- make |story-remove take a case instead of an aeon
- make desk param optional for story-set and story-log
+sign:schnorr crashes on `=(0 sk)`, so the bounds checking code is not
exercised for sk=0. It also crashes on `(gte sk n.domain.c)`, which is
redundant with the size check on sk, so we remove that.
- only store metadata in the persistent map. just enough to support
(eventual) thread cancellation and output mark lookup.
- try to delete thread state at other failure points not covered by
%kick.
- reflect back the passed output mark rather than form.dais. not sure
about this one yet.
Resolves a good number of conflicts. Most notably, re-propagates removal
of gall's %onto, confirms new /app/herm behavior, coerces hood/drum
state adapters back into place, and updates webterm to use the latest
api.
This change greatly improves the ergonomics of working with channel JSON
in statically typed languages, as the polymorphism is moved out of the
actual diff and into the event framing.
de-xml parser fails when xml content node contains doublequotes (`doq` rule), this PR proposes to remove this restriction as high-level javascript APIs that operate on DOM don't entitize/encode doublequotes by default.
+wake had accumulated several layers of abstractions which were later
rendered unnecessary. This removes those abstractions and should have
no semantic effect.
This adds support for tombstoned files to clay. It does not include any
way to actually tombstone them; that is left for later.
This allows tombstoning at the level of a file. Precisely, this expands
+blob:clay by adding a %dead case:
+$ blob :: fs blob
$% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q
[%direct p=lobe q=page] :: immediate
[%dead p=lobe ~] :: tombstone
== ::
Thus, we maintain the invariant that every lobe corresponds to a blob,
but now a blob may be an explicit tombstone.
Details:
- This has not been tested at all, except that it compiles and boots.
- This does not have a state adapter from master. The only state change
is the definition of +cach.
- Additionally, out-of-date ships may unexpectedly receive a %dead blob
from a foreign clay which would interfere with their ability to download
that desk. No code changes necessary, but sponsors should avoid
tombstoning files in %base for a while so their children can get the
update.
- A merge will only fail if the tombstoned file conflicts with another
change. Note that as written, merging from a past desk *can* bring a
tombstoned file to the head of a desk. Possibly this shouldn't be
allowed.
This also includes a couple refactors that were made possible by ford
fusion (since everything is synchronous now) but never got done. In
both cases we get to remove a monad, which simplifies the code
considerably.
- refactor +merge's error handling to use !!/mule instead of threading
through errors
- refactor all +read-* functions and related parts of +try-fill-sub to
eagerly convert lobes to cages.
We also add support reading %a/b/c/e/f/r/x from past and foreign desks,
when possible. Apologies that all of these are in one commit, it was
all a single chunk of work.
This is a draft until we have a way to tombstone. I suspect we'll want
to have a mechanism of keeping track of gc roots and trace to remove,
but this PR doesn't suggest any particular strategy.
Jael needs to be reconfigured to listen to the new aagent for azimuth
events, and the old app needs to be shut down. We do this in
/app/azimuth's +on-init.
Additionally, we make sure that jael doesn't crash when it (as expected)
loses its subscription to the old agent.
Render `@p` shorthands correctly for short moon names. Fixes#5318.
This also changes galaxy and star moons to render as `~parent^` instead of some
longer variation.
When you loaded an app with an error, then fixed the error, it would
create the main gall %mult subscription at a time in the past. Then,
clay would never fill the subscription since it couldn't get the old %a
entries for the apps.
This fixes the issue in two ways: first, don't subscribe in the past.
Second, if clay can't get the old versions, just fire the subscription
anyway.
Previously, if trying to bind to an endpoint that was already bound to,
eyre would reject it. This doesn't play very nicely in a softdist world
where uninstalled apps might not get a chance to clean up, and apps
might re-bind simply for being re-installed.
Here we change eyre to overwrite an existing binding if it conflicts
with the new one to be added.
And reject paths ending in empty segments.
The following cases were being parsed incorrectly:
- `/` represents the empty path, `~`. This was being parsed into `[~. ~]`
- `/x/` is not valid. This was being parsed into `[~.x ~. ~]`
This happens because `urs:ab` has no problem parsing the empty string.
For some supported cases, like `//x` (`[~. ~.x ~]`), this is actually desired
behavior, but it results in trailing empty segments for paths ending in `/`.
Here we apply a `+sear` on top of the existing parser, that transform the `/`
case to produce `~`, and ensures the absence of a trailing empty segment in
all other cases.
Note that we change `(more fas urs:ab)` to `(most fas urs:ab)`. Since `urs:ab`
parses the empty string, this doesn't actually make a difference, but it does
make it more obvious that the `+rear` call will never crash.
Alternative approaches I attempted all resulted in much more complicated
parser, so the dumb `+sear` seems preferable.
We do eat the performance cost of an additional list traversal (in `+rear`)
with this change, but that is probably not the end of the world.
Fixes#1501.
This reverts d96d50199 because +ad is incredibly opaque, and +ergo's
sitting right there anyway. It looks like it was intended to abstract
over +endo, +elbo, and +ergo, but only +ergo was every implemented. I
don't doubt the others could be as well, but then they would be just as
inscrutable.
As SSE are unidirectional, the client always realises that the
connection has failed faster than the server does. Hence, resuming a
subscription is useless, because channels can only be bound to one duct
at a time. Now, instead of failing a request for a channel
that is already bound to a duct, we replace the duct and continue
normally.
Start with |start %desk %app-name
Everywhere in the kernel that we deal with marks, we infer the app it's
connected to and use the marks from that desk.
Also some light renaming in gall, especially path->wire and
current-agent->yoke.
Subsequent tasks:
- Dojo needs a syntax to run generators and threads from other desks
- The home desk should be split into at least a minimal base desk and
big "userspace" desk. Dill's initialization logic should be updated
to handle
- |show-package, |install, and |uninstall should to be written
- Clay should have smarter handling of system versions instead of just
ignoring what's on each desk. It's not clear that this will work
correctly when sys updates right now.
Improves the multikeyfile format by taking a single ship and a list of
life+key pairs, instead of a list of full seeds.
Also decouples these changes from the dawn event, once again putting a
single seed into it. In the multikeyfile case, keys are injected as
%rekey events to jael near the end of the boot sequence.
Haskell-side changes may or may not be incomplete, boot presently fails
at some unknown point with what looks like a noun conversion error.
Allows booting with a keyfile containing multiple keys, as long as one
of them matches current PKI pubkeys for the specified ship.
All relevant keys are loaded into jael and will automatically be put to
use when they match PKI state.
Four changes:
- implement +validate-u to allow %u requests over the network
- make +validate-x use our local marks to make %x requests generally
work over the network
- in +start-request, if a foreign ship is making a request that we
shouldn't send over the network, ignore it. This closes a DOS vector.
- in +duce, if we're about to make a request to a foreign ship which
they won't be able to answer, crash the event.
Combined, these fix many of the common cases of weirdness around foreign
clay requests. Notably absent is a fix for reading `%a` across the
network, which I still maintain should happen against the foreign
hoon/zuse.
fixes#4834
see also #4307
Also switches everything to ropsten by default, including ivory pill.
Batches work on ropsten now.
Also adds +tx as a hacky development tool to create text for metamask to
sign and then turn that into a batch. A useful reference for bridge and
aggregator work.
By factoring their shared logic out into +build-dependency, which gets
passed the relevant details about how to track the file being built in
the dependency stack.
Hoon files may want to import nouns from all files in a given directory.
/~ lets you do so, importing as a (map @ta *) (but with typed values).
Note the description as "directories" here, instead of "path prefix".
The behavior, as implemented, will not include /path/hoon for /~ /path,
instead only including /path/more/hoon and more deeply nested files.
This seems to be, generally, the behavior you want, for example when
importing from /app/myapp/* for /app/myapp/hoon.
Actually using the resulting map requires some manual casting, which is
not ideal. Some code style improvement work remains to be done as well.
In +en-json, the vast majority of our time is in +jesc (json string
escaping). Since ships will always be string-safe, we pretend they're
numbers to bypass the escaper. This saves about a second on initial
landscape load.
Avoid allocating hundreds of thousands of cells when giving large
requests. This took the footprint of this function on initial landscape
load from 1 second to 100 ms.
We commonly print many names in a row, often the same ones. For
example, on landscape's initial load, we send all the members of all the
groups we're in, and there's substantial overlap.
At least half the cost is in +fein, which is not currently jetted, but I
believe there's an old jet in the git history.
Fixes#4598.
#4474 made the JSON time conversion no longer invertible, which caused
problems for chat, which uses message timestamp in milliseconds as a key
-- so chat would send a message with ms timestamp x, it would get
encoded as @da x, but then when it went back through the conversion to
milliseconds, it would often (not always) get encoded as x-1.
I still do not fully understand why this is -- and why it doesn't seem
to be a problem with seconds based on cursory testing -- but integer
multiplication and division generally do not invert. And adding a half a
millisecond to the input date before converting it resolves the issue
and makes the functions invertible.
I added a regression test, so hopefully the next courageous adventurer
who winds up here after wondering why +unm looks funny will have a
safeguard against some of the mistakes I made.
State before: in chrono:userlib, there were second-resolution
@da-to-unix and unix-to-@da functions. In en/dejs:format, there were
millisecond-resolution @da-to-unix and unix-to-@da functions. The
@da-to-unix path in time:enjs confusingly rounded to the nearest
millisecond, meaning millisecond n was a label for [n-0.5, n+0.5) rather
than [n, n+1).
This adds a millisecond-resolution @da-to-unix and unix-to-@da to
chrono:userlib, and a second-resolution conversion to en/dejs:format.
It makes use of the chrono:userlib functions in en/dejs, and doesn't do
any rounding.
Backwards-incompatible changes:
- made unt:chrono:userlib take a @da rather than @.
We can't molt until clay has gotten its pork or else we'll build the old
app against the new kernel. This ignores vegas, since we should get a
notification from clay on /sys/lyv.
When we changed wires from /a/foo to /ames/foo, our sorting function
started sorting by last character instead of first character, so breach
notifications were given to gall before ames. This made gall try to
resubscribe before ames cleared its state, so the message would be lost.
Fixes#4177
This had regressed during some breach-related merge. Multiple commits/branches
had touched this codepath recently, eating the code step change introduced in
#3217.
Fixes#4126.
* jb/motion:
pill: solid
zuse: remove %crud from vane-task
arvo: full vane names in $sign
aqua: build again (still broken)
arvo: reform of the scry reform
