This makes it so during `run`, you can inject a series of Events
into your Urbit. The `-I` injection was implemented in Vere, and
has been ported to KH, with the extended `--inject-event-list`
flag which handles a jammed list of events.
Also included is a sample generator which builds a list of events.
This commit adds 24bit true color capabilities to `sole-effect` for
those terminal supporting it (which most modern terminal does). It adds
a RGB type squashed into `$tint`, which will get converted to escape
sequences in `dill` for the moment.
As Urbit does not do `termcap` detection, this also does not attempt
that. But on terminals that doesn't support true color (e.g. linux
console), the color would be truncated to the nearest achievable
approximate.
Previously, if a subscriber received the OTA migration before the host,
then it would attempt to subscribe to a non-existent resource, and
receive a nack, thus unsubscribing the ship from the notebook. To
rectify this, the tombstoned publish has been changed to resubscribe in
a loop with exponential back off. Once it receives a positive watch-ack
(i.e. the host completed the OTA), it will then migrate the notebook to
graph-store by adding it to graph-pull-hook
In order to curb event queue growth when a client for whatever reason
isn't acking the events we send out, we implement a mechanism for
detecting such "clogging", and proactively kick subscriptions which are
adding too many events to the queue.
If the client hasn't sent an ack for ~s30, any subscription that accrues
more than 50 unacked %facts gets closed to prevent further buildup.
Upon reconnecting, the client will see %kick for the relevant
subscriptions and can open a new subscription as appropriate.
Includes a simple test for this behavior, and updates /app/dbug to be
able to display the newly tracked statistics.
By doing a %watch instead of %watch-as %json for channel subscriptions,
we can hopefully make better use of noun deduplication, when storing
events in a channel's event queue until they get acked.
Store the gall events from channel subscriptions as (vaseless) signs,
instead of serialized events. This should be smaller in memory, and
makes it more likely for noun deduplication to happen.
The cost is needing to reserialize upon channel reconnect, but this is
the less common case, and we don't expect it to be particularly slow.
In certain cases +find-merge-points was very slow. Specifically, the
`done` set was meant to avoid checking the same commit repeatedly, but
it didn't catch the case where a commit was added to the worklist that
was already in that worklist.
Secondly, the worklist was stored as a list but used as a queue, which
resulted in a lot of unnecessary welding. We change it to a qeu.
Fixes#3735
* release/next-vere: (1707 commits)
king: fix zig-zag in stderr logging
u3: refactors +murn/+turn, removing unused variable
u3: rewrites +skim jet with u3i_defcons()
u3: rewrites +skip jet with u3i_defcons()
u3: rewrites +skid jet with u3i_defcons()
vere: updates ames to only print network send failures once
u3: cleans up testing protocol, enables gc in mug tests
u3: refactors and enables gc in jam tests
u3: cleans up testing protocol, enables gc in hashtable tests
u3: enables gc in ames and newt tests
u3: initializes head/tail in u3i_defcons() (under U3_MEMORY_DEBUG)
king: actually try shutting down the piers
king: --serf="" is a host option, not a per ship option.
u3: optimizes +wyt:in jet, gated by compile-time assertion
u3: further optimizes +lent jet, gated by compile-time assertion
u3: refactors allocator constants, adds u3a_cells and u3a_maximum
u3: optimizes +lent jet, avoiding u3i_vint() while possible
u3: moves cell allocation counter into u3a_celloc()
u3: fixes memory leak introduced in +murn jet
u3: fixes mismatches in +div and +dvr jets
...
Instead of always providing a wildcard for the allowed methods and
headers, now echoes back the method and headers that the client asked
for, if any.
Fixes#3676.
Disallows registering bindings (through %connect and %serve) that would capture
traffic on paths starting with /~ (Eyre's) or /~_~ (runtime's, as of cc389c5).
Note that we don't touch +insert-binding, which is used by Eyre internally to
set up bindings in its own namespace.
Adds a cors-registry to Eyre's state that tracks allowed and rejected
origins for the purposes of CORS request handling.
For preflight requests, generates a response in-line.
For simple requests, adds CORS headers onto whatever response is given.
See also:
https://groups.google.com/a/urbit.org/g/dev/c/bb82dwEJGzM/m/q2JjNSx5BwAJ
Lets you check whether a specific Cookie header value string constitutes an
authenticated request.
/ex/=//=/authenticated/cookie/(scot %t 'cookie-string')
Intended for use in the runtime, for example with #3557.
Adds a cors-registry to Eyre's state that tracks allowed and rejected
origins for the purposes of CORS request handling.
For preflight requests, generates a response in-line.
For simple requests, adds CORS headers onto whatever response is given.
See also:
https://groups.google.com/a/urbit.org/g/dev/c/bb82dwEJGzM/m/q2JjNSx5BwAJ
Previously, up-arrowing into (or otherwise retrieving) a command from history
that automatically runs on-input would directly run the command again,
preventing the user from up-arrowing past the auto-run command into further
history.
With this change, shoe detects discontinuous inputs (sole's `%set` edit), and
refuses to auto-run the parsed command in those cases.
The OTA deployment of urbit-os-v1.0.50 unintentionally wiped the indices
during the state migration. This caused widespread permissions issues,
as agents were unable to tell which group is associated with what channel.
Fortunately, the indices are a simple transform of the associations noun,
which was kept intact. This commit introduces a state migration to rebuild
the indices, in order to restore connectivity on the network.
* origin/release/link: (72 commits)
graph: handle already joined case
graph: subscribe to keys globally
leap: fix react-routing to links
interface: fixing syntax error in metadata settings
interface: last tweaks
leap: refer to module when indexing graph data
link-store: fix migration bugs
links: fix s3 detection
links: all inputs use gray2
links: link-item retains hostname detection
links: pass remoteContentPolicy to pending comment
link-listen-hook: silently discard %behn %wake
link: iterate over OrderedMap correctly
interface: added oembeds and drag and drop back
interface: ported in Liam's graph-update reducer and threaded through some display parameters into LinkItem
apps: fix issues with mark and remove unnecessary comments
interface: fix tab bar to display and link properly
graph-view: en-path resource for md-hook
push-hook: Proxy local pokes and fix kick path
interface: removed unused api and store files
...
Signed-off-by: Matilde Park <matilde@tlon.io>
This was originally introduced by me in #1814 to address #1811. Eyre was not
canceling heartbeat timers on all relevant events making it easy to end
up with an infinite behn loop. This check allowed ships that entered an infinite
loop to recover, as per my comment at
https://github.com/urbit/urbit/pull/1814#discussion_r333477482. Otherwise it's
not necessary.
@t further indicates to the caller (although loosely, because auras
are loosely enforced types) that the input should not contain embedded
NUL bytes, which are not valid @t.
Separately, a minor improvement has been made to the performance of
the raw hoon by pinning the gates used in the inner loop.
Prior to this commit, there was a jet mismatch in to-wain (formerly
called lore, and still jetted under that name). 0 bytes in the middle of
a cord caused the jet to crash, whereas the hoon simply treated them as
the end of cord and truncated the output. The history of this behavior
is fraught with controversy. This commit rectifies the current mess with
the following rationale: Null bytes are valid ASCII/UTF-8, and \n\n in
the input will cause null list items in the output, so nulls are (for
the purposes of to-wain) allowed in cords. Trailing nulls cannot be
represented because of the nature of atoms, but that is outside the
scope of to-wain's concern. Therefore to-wain should simply measure the
cord and split on newlines, and do nothing fancy at all with nulls.
In addition, the hoon for to-wain was written in an inefficient style
that produced a lot of intermediate garbage atoms via rsh and cat. This
commit's implementation measures once and cuts once, so to speak, and so
avoids the intermediate garbage. Quick benchmarks suggest it is about
20x faster than the old hoon, but still orders of magnitude slower than
the jetted code. to-wain is the workhorse for the txt mark, so we should
still prefer to have a jet.
The old jet is left wired up under %lore, and should be removed when
support for the old, unupgraded zuse is no longer necessary. A new jet
with matching null handling has been wired up under the name %leer.
Well-behaved secp jets can now plausibly exit if they are given inputs
of the wrong size/range. Previously this was either not checked or the inputs
were silently truncated.
The secp core had some flaws: in particular, the logic for signing/recovery
did not match libsecbp256k1 w.r.t. the enigmatic "recid" (v) value. The jet
hints were also subtly wrong, in that the curve parameters were in a sample
(not an arm) and thus not matched by the jet matching scheme. Consequently,
the jets would be used (but incorrect) for other curve parameters.
Tests were also added to exercise the recovery id cases thoroughly.
Depending on the additions to term.c made in 467d8d239 allows dill to
forget about ansi escape codes, and pass styled text nouns straight on
to vere.
Also removes a bit of logic from drum, which assumed things about the
rendering of escape codes to adjust cursor positioning. Now it simply
states the semantic cursor position, letting the runtime deal with the
potential influence of styling.
If the underlying app implements a /x/dbug/state endpoint in +on-peek,
dbug will now use that for %state evaluation. Falls back to the vase
provided by +on-save if the peek fails.
This allows apps and (perhaps more usefully) wrapper agents to provide
customized vases to /lib/dbug.
Updates /lib/shoe to make use of this, properly prepending the wrapped
app's vase to shoe's own, instead of including it as-is.
If both sides changed a file in the same way, %mate used the version in
the mergebase, which is incorrect. This changes it to use the version
in the destination desk.
An example of this issue:
> +cat %/test/hoon
/~zod/home/~2020.9.3..21.41.24..61ed/test/hoon
first
> |merge %scratch our %home
>=
merged with strategy %fine
+ /~zod/scratch/2/test/hoon
> +cat /=scratch=/test
/~zod/scratch/~2020.9.3..21.41.32..408c/test/hoon
first
> *%/test/hoon 'second'
: /~zod/home/3/test/hoon
> *%%%/scratch=/test/hoon 'second'
: /~zod/scratch/3/test/hoon
> |merge %scratch our %home
>=
%fine merge failed, trying %meet
%meet merge failed, trying %mate
merged with strategy %mate
: /~zod/scratch/4/test/hoon
> +cat /=scratch=/test
/~zod/scratch/~2020.9.3..21.42.25..9e8b/test/hoon
first
For rendering a data table or just a single table row respectively.
Table data is a list of header names, a list of width restrictions, and a list
of rows, defined as lists of dimes.
Row contents are rendered as per the aura in the dime. This also determines
their alignment (left/right), and how they break to fit smaller-width columns.
%row was added because %table necessitated implementing it. Whether it's a good
fit for the shoe "protocol" remains to be seen.
Barebones 2-way diff thread. Further development should include:
- recognize at least %txt and %hoon diff types as (urge:clay cord) and
print them prettily
- check if they're directories and recurse through them, noting files
that exist in one or the other and diffing ones that exist in both
- support 3-way diffs. Since we know the desks they're on, we can find
a mergebase (see how +trouble gets the base-hash in #3378) and do a
3-way diff.
- extend the above to support all the useful diffs, including
diff3(a,b), diff2(mergebase(a,b),a) and any other useful ones.
Ordinarily, eyre cleans up the relevant gall subscriptions whenever a
channel disappears. In yet unresolved erroneous behavior though, it may
leave a gall subscription open, despite wiping the channel from state.
Attempting to pass the response onto the deleted channel anyway results
in an %eyre-no-channel error later in the event. The volume of these
errors can degrade the user experience, as per #3196.
To resolve the annoyance (but not the underlying issue) we detect the
"subscription has no channel" case, and issue a %leave. Doing so
requires additional information in the wire, so we add that in,
refactoring the relevant wire building along the way.
Note that due to the wire requirements, this cannot resolve existing
cases. For that, we depend on bc929ba6d.
As part of the solution to #3196, we need to clean up any gall
subscriptions that eyre didn't properly clean up.
Since detecting that is hard, we opt to just wipe _all_ eyre-originating
subscriptions from gall. We inspect the duct, which isn't good, but it's
only just this once.
* origin/la/file-server-fix:
file-server: fix %file-server by adding SPA flag and threading is-file through request
Signed-off-by: Matilde Park <matilde.park@sunshinegardens.org>
The main thing here is that we aggressively check whether we're in
ancestry of another mergebase candidate. This means we don't have to do
a 2nd pass to eliminate redundant candidates.
Change the definition of base-hash to be the mergebase of %home with the
OTA source. This means it's the most recent successfully-applied
update, which is usually the most important information.
Add sour-hash, which is the hash of the most recently *downloaded*
update, regardless of whether it applied successfuly (ie the old
base-hash).
Add a summary of the various hashes at the top of gen/trouble.
Only no-op if the incoming commit's parent is the old head of the desk.
Also move the printing near the end so we can know exactly if anything
changed.
If we receive an %add poke for a resource we are already pulling, no-op
instead of crashing. This should prevent crashes upon repulling a
resource after a breach of the host.
The contact-hook was attempting to join a group upon it's invite being
accepted. However, the join poke is also sent from the frontend, causing
a potential race condition. Changes contact-hook to no-op on an
%invite-update.
Jael now stores a `step` that is combined with the original salt to
produce a new code. A `%step` card is used to increment that value,
and effectively resetting the keys. Because the first `step` is zero,
the first code is the same as before.
Eyre was changed to be notified with `%code-changed` so it can forget
old cookies, sessions and discard all the existing channels.
A new generator was added |code, that does both querying and
resetting the code
|code :: shows current code, step and help
|code %reset :: changes the code
The old +code generator still works correctly.
