+riff-any is all clay requests except "backfill" requests. Change to
`$%` from `$^`, which was used to distinguish originally non-versioned
requests.
+fill is backfill requests and had no version number, so we add one.
We do not have version numbers on responses since those are implied by
the request. If someone requests at version `n` and you're at `n+1`,
you must respond in the format of `n`.
If someone requests at version `n+1` and you're at `n`, you crash;
though possibly you should be able to respond with message "I only know
up to `n`", in which case they may be able to re-request at `n`. In
either case, the version of the response is dictated by the request.
* jb/re-mug-pre:
arvo: temporary, build +brass out of /not-sys
hoon: switches to new +mug
u3: adds new +mug (as u3r_gum_*)
hoon: adds new +mug (as +gum)
u3: updates +muk to truncate seed and removes spurious assertion
hoon: updates +muk to truncate seed and removes spurious assertion
Virtualises the kick handler for the inner door in lib/pull-hook, so
that crashes in the handler do not cause a dangling resource with no
subscription. Additionally, failed kicks now cause the sync to be
dropped into a failed-kicks map in the state, where we attempt to
recover from the failure in the on-load.
If the %kick handler of the inner-core crashes, then we never get to
resubscribe, thus leaving a dangling entry in the state with no
corresponding subscription. Updates the pull-hook-action pokes to not
crash when given a dangling entry.
* na-release/candidate: (461 commits)
pill: update
zuse: make octs value @ again
zuse, clay, various: unflop the spur in beams
hoon: add +snip and +rear
arvo: unflop the spur in scry
tests: checks bip32 for xpub/xprv/pif/address
bip32: adds network option for bitcoin addresses
bip32: adds network type option for xpub/priv
pill: all
pill: change to ropsten
zuse: change to ropsten
pill: all
u3: fix accidental neologism in comments
pill: solid
pill: solid
nix: trims trailing whitespace from solid pill builder
hoon: adds $cord to $tank
zuse: modernizes syntax of top-level structures
zuse: removes obselete structures $ares, $coop, and $disc
hoon: moves $json and styled-text molds back to %zuse
...
If a notification came in that merged with a notification that had
already been read, then it would not be added to the cache properly and
therefore would not correctly send %read updates when %read-index was
called.
Fixes#3977
Unflops the spur in +en-beam, +de-beam, and everything that calls either
of those, or works with the consequences of their output.
This includes clay's interface for mounting and unmounting, which now
no longer expects the arguments to contain an old-style spur.
* na-release/secp:
update solid pill
fix secp test (new-secp staging name is gone)
hoon: ensure lib/bip32 is working with new secp, remove old secp
vere: wire up jets for new secp
zuse: make it clear where the secp jet should make explicit size checks
update solid pill
zuse: add refactored secp core (unjetted)
* na-release/runes:
pill: solid
pill: solid.pill
hoon: bscl to bccl, etc
* jb/co-norm:
pill: solid
pill: solid
hoon: modernizes syntax in the rest of |co
hoon: updates @p rendering to avoid intermediate right-shifts
hoon: refactors @da/@dr coin printing
hoon: comments, modernizes syntax for all +*-co:co gates
hoon: refactors +r-co:co, modernizing syntax
hoon: comments, modernizes syntax for all +**-co gates
hoon: refactors +ro-co:co, modernizing syntax
hoon: use +pow instead of handrolled loop in +ox-co:co
hoon: use +dvr instead of +div/+mod in |co helpers
* na-release/next-vere: (1459 commits)
u3: fix accidental neologism in comments
nix: run tests against the latest arvo source
test: fixes +to-wain (no more trailing empty lines)
pill: solid
u3: refactors dynamic hint bytecodes, documents protocol
u3: adds dynamic hint bytecodes and implementation
u3: moves _n_swap() next to other stack ops
u3: adds ghetto +mook replacement
u3: moves |ut battery to the rightmost cache key position
u3: adds |ut battery to jet cache keys
build: update gcloud to use non-deprecated action
pill: solid
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
...
Makes chat-cli listen to graph-store rather than chat-store.
Along the way, cuts some features. No longer supports channel creation,
deletion, permission management or inviting. No longer lets you send
messages to multiple targets at once.
No longer displays messages from all joined chats, and no longer lets
you join or leave chats. Instead, chat-cli solely concerns itself with
viewing chat content you're already subscribed to (through use of
Landscape or other rich clients). ;view ~host/chat to start printing
messages for that chat when they come in, ;flee to toggle off. ;view to
see what chats are enabled, ;chats to see all available chats.
The removal of features acknowledges chat-cli's lack of real support for
the modern, group-centric state of userspace.
The intent of ;view ux is to prepare chat-cli for a multi-session
terminal world.
* na-release/next-vere: (1601 commits)
nix: run tests against the latest arvo source
test: fixes +to-wain (no more trailing empty lines)
pill: solid
build: update gcloud to use non-deprecated action
pill: solid
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
...
a tape is just a list of utf8 bytes, it's never save to split one
at an arbitrary position. this is exactly what \/ windowing did,
so i had hacked in to/from utf32 conversions to prevent splitting
multi-byte characters. that is dumb and slow, so it's now gone.
* master: (390 commits)
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
interface: links and publish comments both work
...
* release/next-vere:
build: update gcloud to use non-deprecated action
vere: manage memory properly in _fore_import
build: update GoogleCloudPlatform/github-actions/setup-gcloud to 0.2.0
king: try undoing warnings in eventlog-lmdb
king: diswarn tests
king: more warnery
king: fix warnings; more consistent warn policy
vere: add -i and -o options for import and export
herb/lens: add utilites for import/export
vere: refactors pier initialization to fix -X
king: fix test (daaad)
king: fix warnings in Tree (thanks dad)
king: this is a holdup!
* release/next-vere:
build: update gcloud to use non-deprecated action
vere: manage memory properly in _fore_import
build: update GoogleCloudPlatform/github-actions/setup-gcloud to 0.2.0
king: try undoing warnings in eventlog-lmdb
king: diswarn tests
king: more warnery
king: fix warnings; more consistent warn policy
vere: add -i and -o options for import and export
herb/lens: add utilites for import/export
vere: refactors pier initialization to fix -X
king: fix test (daaad)
king: fix warnings in Tree (thanks dad)
king: this is a holdup!
* na-release/next-vere: (943 commits)
pill: solid
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
...
* master: (390 commits)
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
interface: links and publish comments both work
...
hark-chat-hook scries for the group in order to determine whether the
chat should be automatically watched for notifications. Therefore, if we
are joining a chat, we should acquire the metadata before joining the
chat. This is to ensure that the scry works correctly.
Fixes#3946
* release/next-vere: (1369 commits)
nix: fixes `shellFor` nix-shell helper
vere: print error and exit if stdin is not a tty
build: silence service account activation output
build: minor refactoring of haskell-nix overlays
build: move darwin install_name_tool fixup from vere to king haskell
u3: fixes incorrect double ref-counting in |ff jets
u3: removes unused `Exit` variable
u3: removes obsolete bail:need assertion
u3: refactors fatal exception handling in u3m_bail()
build: remove {sha256,md5} output for push-storage-object effects
build: add log message when destination object already exists
build: force google-cloud-sdk to use python3
build: adding support for hercules ci effects
build: remove push-to-storage for ivory, brass, and solid pills
pill: rebuild solid pill with %lens included in lite boot apps
arvo: run %lens when lite boot (-l) is specified
build: expose configurable arguments when booting/testing fake ships
build: ensure urbit tests are run with the -g argument
vere: ensure debug symbols aren't stripped by default (by nix)
build: remove from-scratch ropsten pill builds on ci
...
Eyre's clog logic was a tad inconsistent about "only facts" vs "not poke-acks".
This makes it consistently say "only facts" when it comes to clog-related logic.
Yes, in theory this means %watch-acks and %kicks can build up endlessly, but
those should take up negligible space compared to %facts.
Should fix any oddball cases of crashes here that #3835 didn't already catch.
When an application would send multiple facts during a single event, it
was possible for the first fact to trigger a clog, removing the
subscription and sending a quit, but then the second fact still getting
sent out at normal.
Here, we drop any facts for subscriptions we don't have registered in
state, which should only happen in the described case.
Eyre's clog logic was a tad inconsistent about "only facts" vs "not poke-acks".
This makes it consistently say "only facts" when it comes to clog-related logic.
Yes, in theory this means %watch-acks and %kicks can build up endlessly, but
those should take up negligible space compared to %facts.
Should fix any oddball cases of crashes here that #3835 didn't already catch.
Make the marks produce json arrays where the (list @c) type is used.
On the frontend, transform the semantic (character-based) cursor
position into a byte-based one, in order to ensure the cursor gets
rendered at the correct location, even if the presence of multi-byte
characters like emoji.
This was a little bit too crummy. Instead, we put in a placeholder of ~,
which should be forwards-compatible with atomic session identifiers,
where ~ identifies the default session.
Additionally touches up the herm wires/paths to stick to the above more
closely.
Listens to the default dill session and passes its %blits on to
subscribers. Passes any %belt pokes it gets into dill.
Updates webdojo to make use of it, which is the primary motivation for
herm's existence.
This lets us support the "random userspace app sending dill belts".
Ultimately, we'll want to be able to specify a session identifier
alongside the belt, instead of strictly relying on the duct.
Adds a %view task, which opens a subscription on the output sent to the
specified session. %flee closes the same.
Whenever dill sends a blit to the session, any subscribers get the
output also.
The structures here will become more reasonable once we replace ducts
with proper dill session identifiers.
People using older runtimes might not support the %klr blit. It's not
uncommon for prompts without style to get passed in as %pom though, so
here we catch that case and turn it into a %pro, which gets rendered as
a traditional %lin.
Pretty-printing is expensive, yet we do it whenever we construct the cookie
string, at least once (but usually twice) per authenticated request.
Here we call out the the specific to-tape functions we need, instead of relying
on the pretty-printer for converting... tapes to tapes, among other things.
The primary gains come from the cookie-related instances, we update the others
mostly for good style.
For the "receive request and immediately send response" case, that is processed
synchronously within eyre (ie, client sends channel ack), speeds thing up by
roughly 55%.
Instead of passing in a tape to print in case we need in, pass in a trap that
can be called to generate the tape on-demand.
Eagerly rendering printouts was costing us a lot of time when we didn't actually
need them, because the pretty-printer is slow.
Motivation for the change is performance improvements on the un-`^~`d uses of
ream. Parsing turns out to be slow, making ream slow in turn. So we construct
the hoon ast manually instead.
!, is arguably better style than ream, since it doesn't require a ^~ for static
input, and lets syntax highlighting function properly.
For the investigated case, in +get-cast's +grow flow, improves performance by
over 80%.
If the Forwarded header specifies the original connection is secure,
update the flag to reflect that, regardless of whether the connection
directly to the urbit was made securely.
As links is the only application currently using graph-store, we migrate
all indices to @da instead of a @da or a unix timestamp. Rolls over
current graphs and update log to rectify this. Additionally, applies the
link validator mark to graphs, as this was missing. Fixes a latent bug
in graph-store where the removal of a graph that triggered a mark build
would prevent the mark from being rebuilt. Fixes mark input and index
formatting from the frontend as well.
When an application would send multiple facts during a single event, it
was possible for the first fact to trigger a clog, removing the
subscription and sending a quit, but then the second fact still getting
sent out at normal.
Here, we drop any facts for subscriptions we don't have registered in
state, which should only happen in the described case.
Because storing in reverse order means producing in reverse reverse
order.
The tests didn't catch this because they, too, were infected with the
"reverse moves" meme.
This makes it so during `run`, you can inject a series of Events
into your Urbit. The `-I` injection was implemented in Vere, and
has been ported to KH, with the extended `--inject-event-list`
flag which handles a jammed list of events.
Also included is a sample generator which builds a list of events.
This commit adds 24bit true color capabilities to `sole-effect` for
those terminal supporting it (which most modern terminal does). It adds
a RGB type squashed into `$tint`, which will get converted to escape
sequences in `dill` for the moment.
As Urbit does not do `termcap` detection, this also does not attempt
that. But on terminals that doesn't support true color (e.g. linux
console), the color would be truncated to the nearest achievable
approximate.
Previously, if a subscriber received the OTA migration before the host,
then it would attempt to subscribe to a non-existent resource, and
receive a nack, thus unsubscribing the ship from the notebook. To
rectify this, the tombstoned publish has been changed to resubscribe in
a loop with exponential back off. Once it receives a positive watch-ack
(i.e. the host completed the OTA), it will then migrate the notebook to
graph-store by adding it to graph-pull-hook
In order to curb event queue growth when a client for whatever reason
isn't acking the events we send out, we implement a mechanism for
detecting such "clogging", and proactively kick subscriptions which are
adding too many events to the queue.
If the client hasn't sent an ack for ~s30, any subscription that accrues
more than 50 unacked %facts gets closed to prevent further buildup.
Upon reconnecting, the client will see %kick for the relevant
subscriptions and can open a new subscription as appropriate.
Includes a simple test for this behavior, and updates /app/dbug to be
able to display the newly tracked statistics.
By doing a %watch instead of %watch-as %json for channel subscriptions,
we can hopefully make better use of noun deduplication, when storing
events in a channel's event queue until they get acked.
Store the gall events from channel subscriptions as (vaseless) signs,
instead of serialized events. This should be smaller in memory, and
makes it more likely for noun deduplication to happen.
The cost is needing to reserialize upon channel reconnect, but this is
the less common case, and we don't expect it to be particularly slow.
In certain cases +find-merge-points was very slow. Specifically, the
`done` set was meant to avoid checking the same commit repeatedly, but
it didn't catch the case where a commit was added to the worklist that
was already in that worklist.
Secondly, the worklist was stored as a list but used as a queue, which
resulted in a lot of unnecessary welding. We change it to a qeu.
Fixes#3735
* release/next-vere: (1707 commits)
king: fix zig-zag in stderr logging
u3: refactors +murn/+turn, removing unused variable
u3: rewrites +skim jet with u3i_defcons()
u3: rewrites +skip jet with u3i_defcons()
u3: rewrites +skid jet with u3i_defcons()
vere: updates ames to only print network send failures once
u3: cleans up testing protocol, enables gc in mug tests
u3: refactors and enables gc in jam tests
u3: cleans up testing protocol, enables gc in hashtable tests
u3: enables gc in ames and newt tests
u3: initializes head/tail in u3i_defcons() (under U3_MEMORY_DEBUG)
king: actually try shutting down the piers
king: --serf="" is a host option, not a per ship option.
u3: optimizes +wyt:in jet, gated by compile-time assertion
u3: further optimizes +lent jet, gated by compile-time assertion
u3: refactors allocator constants, adds u3a_cells and u3a_maximum
u3: optimizes +lent jet, avoiding u3i_vint() while possible
u3: moves cell allocation counter into u3a_celloc()
u3: fixes memory leak introduced in +murn jet
u3: fixes mismatches in +div and +dvr jets
...
Instead of always providing a wildcard for the allowed methods and
headers, now echoes back the method and headers that the client asked
for, if any.
Fixes#3676.
Disallows registering bindings (through %connect and %serve) that would capture
traffic on paths starting with /~ (Eyre's) or /~_~ (runtime's, as of cc389c5).
Note that we don't touch +insert-binding, which is used by Eyre internally to
set up bindings in its own namespace.
Adds a cors-registry to Eyre's state that tracks allowed and rejected
origins for the purposes of CORS request handling.
For preflight requests, generates a response in-line.
For simple requests, adds CORS headers onto whatever response is given.
See also:
https://groups.google.com/a/urbit.org/g/dev/c/bb82dwEJGzM/m/q2JjNSx5BwAJ
Lets you check whether a specific Cookie header value string constitutes an
authenticated request.
/ex/=//=/authenticated/cookie/(scot %t 'cookie-string')
Intended for use in the runtime, for example with #3557.
Adds a cors-registry to Eyre's state that tracks allowed and rejected
origins for the purposes of CORS request handling.
For preflight requests, generates a response in-line.
For simple requests, adds CORS headers onto whatever response is given.
See also:
https://groups.google.com/a/urbit.org/g/dev/c/bb82dwEJGzM/m/q2JjNSx5BwAJ
Previously, up-arrowing into (or otherwise retrieving) a command from history
that automatically runs on-input would directly run the command again,
preventing the user from up-arrowing past the auto-run command into further
history.
With this change, shoe detects discontinuous inputs (sole's `%set` edit), and
refuses to auto-run the parsed command in those cases.
The OTA deployment of urbit-os-v1.0.50 unintentionally wiped the indices
during the state migration. This caused widespread permissions issues,
as agents were unable to tell which group is associated with what channel.
Fortunately, the indices are a simple transform of the associations noun,
which was kept intact. This commit introduces a state migration to rebuild
the indices, in order to restore connectivity on the network.
