Also switches everything to ropsten by default, including ivory pill.
Batches work on ropsten now.
Also adds +tx as a hacky development tool to create text for metamask to
sign and then turn that into a batch. A useful reference for bridge and
aggregator work.
By factoring their shared logic out into +build-dependency, which gets
passed the relevant details about how to track the file being built in
the dependency stack.
Hoon files may want to import nouns from all files in a given directory.
/~ lets you do so, importing as a (map @ta *) (but with typed values).
Note the description as "directories" here, instead of "path prefix".
The behavior, as implemented, will not include /path/hoon for /~ /path,
instead only including /path/more/hoon and more deeply nested files.
This seems to be, generally, the behavior you want, for example when
importing from /app/myapp/* for /app/myapp/hoon.
Actually using the resulting map requires some manual casting, which is
not ideal. Some code style improvement work remains to be done as well.
In +en-json, the vast majority of our time is in +jesc (json string
escaping). Since ships will always be string-safe, we pretend they're
numbers to bypass the escaper. This saves about a second on initial
landscape load.
Avoid allocating hundreds of thousands of cells when giving large
requests. This took the footprint of this function on initial landscape
load from 1 second to 100 ms.
We commonly print many names in a row, often the same ones. For
example, on landscape's initial load, we send all the members of all the
groups we're in, and there's substantial overlap.
At least half the cost is in +fein, which is not currently jetted, but I
believe there's an old jet in the git history.
Fixes#4598.
#4474 made the JSON time conversion no longer invertible, which caused
problems for chat, which uses message timestamp in milliseconds as a key
-- so chat would send a message with ms timestamp x, it would get
encoded as @da x, but then when it went back through the conversion to
milliseconds, it would often (not always) get encoded as x-1.
I still do not fully understand why this is -- and why it doesn't seem
to be a problem with seconds based on cursory testing -- but integer
multiplication and division generally do not invert. And adding a half a
millisecond to the input date before converting it resolves the issue
and makes the functions invertible.
I added a regression test, so hopefully the next courageous adventurer
who winds up here after wondering why +unm looks funny will have a
safeguard against some of the mistakes I made.
State before: in chrono:userlib, there were second-resolution
@da-to-unix and unix-to-@da functions. In en/dejs:format, there were
millisecond-resolution @da-to-unix and unix-to-@da functions. The
@da-to-unix path in time:enjs confusingly rounded to the nearest
millisecond, meaning millisecond n was a label for [n-0.5, n+0.5) rather
than [n, n+1).
This adds a millisecond-resolution @da-to-unix and unix-to-@da to
chrono:userlib, and a second-resolution conversion to en/dejs:format.
It makes use of the chrono:userlib functions in en/dejs, and doesn't do
any rounding.
Backwards-incompatible changes:
- made unt:chrono:userlib take a @da rather than @.
We can't molt until clay has gotten its pork or else we'll build the old
app against the new kernel. This ignores vegas, since we should get a
notification from clay on /sys/lyv.
When we changed wires from /a/foo to /ames/foo, our sorting function
started sorting by last character instead of first character, so breach
notifications were given to gall before ames. This made gall try to
resubscribe before ames cleared its state, so the message would be lost.
Fixes#4177
This had regressed during some breach-related merge. Multiple commits/branches
had touched this codepath recently, eating the code step change introduced in
#3217.
Fixes#4126.
* jb/motion:
pill: solid
zuse: remove %crud from vane-task
arvo: full vane names in $sign
aqua: build again (still broken)
arvo: reform of the scry reform
When you first boot, if you try talk to someone before your azimuth is
up-to-date (for example by import), then if they've ever breached
(twice) then you'll get breach notification, cancelling your message.
This changes is it so that if we haven't heard anything about this ship,
we don't signal a breach.
The implementation complexity is primarily because we need
eth-watcher/azimuth-tracker to produce an update of a list instead of a
list of updates. This way, Jael can keep a "state as of the beginning
of this move" variable to check when deciding whether to signal a
breach.
* na-release/candidate:
kh: use Word8 for Tint true color values
arvo: remove unused app files, libraries, and imports
webterm: improve line-spacing in certain browsers
vere: avoid +scot call for color value rendering
kh: support 24-bit %klr colors
vere: support 24-bit %klr colors
webterm: update mar and js to support 24-bit color
tests: fix ames tests
pill: update ivory pill
dojo: correct mark conversion scry path
pill: solid
aqua/ph: fix comet test
ames: flat packet format
hoon, dill: Add 24-bit true color
+riff-any is all clay requests except "backfill" requests. Change to
`$%` from `$^`, which was used to distinguish originally non-versioned
requests.
+fill is backfill requests and had no version number, so we add one.
We do not have version numbers on responses since those are implied by
the request. If someone requests at version `n` and you're at `n+1`,
you must respond in the format of `n`.
If someone requests at version `n+1` and you're at `n`, you crash;
though possibly you should be able to respond with message "I only know
up to `n`", in which case they may be able to re-request at `n`. In
either case, the version of the response is dictated by the request.
* jb/re-mug-pre:
arvo: temporary, build +brass out of /not-sys
hoon: switches to new +mug
u3: adds new +mug (as u3r_gum_*)
hoon: adds new +mug (as +gum)
u3: updates +muk to truncate seed and removes spurious assertion
hoon: updates +muk to truncate seed and removes spurious assertion
* na-release/candidate: (461 commits)
pill: update
zuse: make octs value @ again
zuse, clay, various: unflop the spur in beams
hoon: add +snip and +rear
arvo: unflop the spur in scry
tests: checks bip32 for xpub/xprv/pif/address
bip32: adds network option for bitcoin addresses
bip32: adds network type option for xpub/priv
pill: all
pill: change to ropsten
zuse: change to ropsten
pill: all
u3: fix accidental neologism in comments
pill: solid
pill: solid
nix: trims trailing whitespace from solid pill builder
hoon: adds $cord to $tank
zuse: modernizes syntax of top-level structures
zuse: removes obselete structures $ares, $coop, and $disc
hoon: moves $json and styled-text molds back to %zuse
...
Unflops the spur in +en-beam, +de-beam, and everything that calls either
of those, or works with the consequences of their output.
This includes clay's interface for mounting and unmounting, which now
no longer expects the arguments to contain an old-style spur.
* na-release/secp:
update solid pill
fix secp test (new-secp staging name is gone)
hoon: ensure lib/bip32 is working with new secp, remove old secp
vere: wire up jets for new secp
zuse: make it clear where the secp jet should make explicit size checks
update solid pill
zuse: add refactored secp core (unjetted)
* na-release/runes:
pill: solid
pill: solid.pill
hoon: bscl to bccl, etc
* jb/co-norm:
pill: solid
pill: solid
hoon: modernizes syntax in the rest of |co
hoon: updates @p rendering to avoid intermediate right-shifts
hoon: refactors @da/@dr coin printing
hoon: comments, modernizes syntax for all +*-co:co gates
hoon: refactors +r-co:co, modernizing syntax
hoon: comments, modernizes syntax for all +**-co gates
hoon: refactors +ro-co:co, modernizing syntax
hoon: use +pow instead of handrolled loop in +ox-co:co
hoon: use +dvr instead of +div/+mod in |co helpers
* na-release/next-vere: (1459 commits)
u3: fix accidental neologism in comments
nix: run tests against the latest arvo source
test: fixes +to-wain (no more trailing empty lines)
pill: solid
u3: refactors dynamic hint bytecodes, documents protocol
u3: adds dynamic hint bytecodes and implementation
u3: moves _n_swap() next to other stack ops
u3: adds ghetto +mook replacement
u3: moves |ut battery to the rightmost cache key position
u3: adds |ut battery to jet cache keys
build: update gcloud to use non-deprecated action
pill: solid
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
...
* na-release/next-vere: (1601 commits)
nix: run tests against the latest arvo source
test: fixes +to-wain (no more trailing empty lines)
pill: solid
build: update gcloud to use non-deprecated action
pill: solid
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
...
a tape is just a list of utf8 bytes, it's never save to split one
at an arbitrary position. this is exactly what \/ windowing did,
so i had hacked in to/from utf32 conversions to prevent splitting
multi-byte characters. that is dumb and slow, so it's now gone.
* master: (390 commits)
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
interface: links and publish comments both work
...
* na-release/next-vere: (943 commits)
pill: solid
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
...
* master: (390 commits)
glob: update to 0v4.fpa4r.s6dtc.h8tps.62jv0.qn0fj
notifications: prevent safari shrinkage
glob: update to 0v5.91i1u.1g535.t3de3.6c3ih.fanmv
Sidebar: loosen property access
launch: loosen property access in unread count
notifications: fix scroll to load
glob: update to 0v1.pak02.pfla3.gh56f.qhc6h.3h881
inbox: fix graph resource redirects
inbox: fix link routing and rendering
glob: update to 0v4.3fbh4.p7j6i.2pi9g.d1ltq.5u7uu
hark-fe: fix crash
hark: update graph marks for editable comments
graph-store: change atom to %1 for all migrated comments
glob: update to 0v5.67obv.15auf.c2rc7.jpcu2.iain3
inbox: correct notification order
inbox: redirect invites correctly
publish: Restore basic 'add writers' form
interface: show currently editing comment as pending
landscape: preclude dropdown duplicates on exact match
interface: links and publish comments both work
...
* release/next-vere: (1369 commits)
nix: fixes `shellFor` nix-shell helper
vere: print error and exit if stdin is not a tty
build: silence service account activation output
build: minor refactoring of haskell-nix overlays
build: move darwin install_name_tool fixup from vere to king haskell
u3: fixes incorrect double ref-counting in |ff jets
u3: removes unused `Exit` variable
u3: removes obsolete bail:need assertion
u3: refactors fatal exception handling in u3m_bail()
build: remove {sha256,md5} output for push-storage-object effects
build: add log message when destination object already exists
build: force google-cloud-sdk to use python3
build: adding support for hercules ci effects
build: remove push-to-storage for ivory, brass, and solid pills
pill: rebuild solid pill with %lens included in lite boot apps
arvo: run %lens when lite boot (-l) is specified
build: expose configurable arguments when booting/testing fake ships
build: ensure urbit tests are run with the -g argument
vere: ensure debug symbols aren't stripped by default (by nix)
build: remove from-scratch ropsten pill builds on ci
...
Eyre's clog logic was a tad inconsistent about "only facts" vs "not poke-acks".
This makes it consistently say "only facts" when it comes to clog-related logic.
Yes, in theory this means %watch-acks and %kicks can build up endlessly, but
those should take up negligible space compared to %facts.
Should fix any oddball cases of crashes here that #3835 didn't already catch.
This was a little bit too crummy. Instead, we put in a placeholder of ~,
which should be forwards-compatible with atomic session identifiers,
where ~ identifies the default session.
Additionally touches up the herm wires/paths to stick to the above more
closely.
This lets us support the "random userspace app sending dill belts".
Ultimately, we'll want to be able to specify a session identifier
alongside the belt, instead of strictly relying on the duct.
Adds a %view task, which opens a subscription on the output sent to the
specified session. %flee closes the same.
Whenever dill sends a blit to the session, any subscribers get the
output also.
The structures here will become more reasonable once we replace ducts
with proper dill session identifiers.
People using older runtimes might not support the %klr blit. It's not
uncommon for prompts without style to get passed in as %pom though, so
here we catch that case and turn it into a %pro, which gets rendered as
a traditional %lin.
Pretty-printing is expensive, yet we do it whenever we construct the cookie
string, at least once (but usually twice) per authenticated request.
Here we call out the the specific to-tape functions we need, instead of relying
on the pretty-printer for converting... tapes to tapes, among other things.
The primary gains come from the cookie-related instances, we update the others
mostly for good style.
For the "receive request and immediately send response" case, that is processed
synchronously within eyre (ie, client sends channel ack), speeds thing up by
roughly 55%.
Motivation for the change is performance improvements on the un-`^~`d uses of
ream. Parsing turns out to be slow, making ream slow in turn. So we construct
the hoon ast manually instead.
!, is arguably better style than ream, since it doesn't require a ^~ for static
input, and lets syntax highlighting function properly.
For the investigated case, in +get-cast's +grow flow, improves performance by
over 80%.
If the Forwarded header specifies the original connection is secure,
update the flag to reflect that, regardless of whether the connection
directly to the urbit was made securely.
When an application would send multiple facts during a single event, it
was possible for the first fact to trigger a clog, removing the
subscription and sending a quit, but then the second fact still getting
sent out at normal.
Here, we drop any facts for subscriptions we don't have registered in
state, which should only happen in the described case.
Because storing in reverse order means producing in reverse reverse
order.
The tests didn't catch this because they, too, were infected with the
"reverse moves" meme.
This commit adds 24bit true color capabilities to `sole-effect` for
those terminal supporting it (which most modern terminal does). It adds
a RGB type squashed into `$tint`, which will get converted to escape
sequences in `dill` for the moment.
As Urbit does not do `termcap` detection, this also does not attempt
that. But on terminals that doesn't support true color (e.g. linux
console), the color would be truncated to the nearest achievable
approximate.
In order to curb event queue growth when a client for whatever reason
isn't acking the events we send out, we implement a mechanism for
detecting such "clogging", and proactively kick subscriptions which are
adding too many events to the queue.
If the client hasn't sent an ack for ~s30, any subscription that accrues
more than 50 unacked %facts gets closed to prevent further buildup.
Upon reconnecting, the client will see %kick for the relevant
subscriptions and can open a new subscription as appropriate.
Includes a simple test for this behavior, and updates /app/dbug to be
able to display the newly tracked statistics.
By doing a %watch instead of %watch-as %json for channel subscriptions,
we can hopefully make better use of noun deduplication, when storing
events in a channel's event queue until they get acked.
Store the gall events from channel subscriptions as (vaseless) signs,
instead of serialized events. This should be smaller in memory, and
makes it more likely for noun deduplication to happen.
The cost is needing to reserialize upon channel reconnect, but this is
the less common case, and we don't expect it to be particularly slow.
In certain cases +find-merge-points was very slow. Specifically, the
`done` set was meant to avoid checking the same commit repeatedly, but
it didn't catch the case where a commit was added to the worklist that
was already in that worklist.
Secondly, the worklist was stored as a list but used as a queue, which
resulted in a lot of unnecessary welding. We change it to a qeu.
Fixes#3735
* release/next-vere: (1707 commits)
king: fix zig-zag in stderr logging
u3: refactors +murn/+turn, removing unused variable
u3: rewrites +skim jet with u3i_defcons()
u3: rewrites +skip jet with u3i_defcons()
u3: rewrites +skid jet with u3i_defcons()
vere: updates ames to only print network send failures once
u3: cleans up testing protocol, enables gc in mug tests
u3: refactors and enables gc in jam tests
u3: cleans up testing protocol, enables gc in hashtable tests
u3: enables gc in ames and newt tests
u3: initializes head/tail in u3i_defcons() (under U3_MEMORY_DEBUG)
king: actually try shutting down the piers
king: --serf="" is a host option, not a per ship option.
u3: optimizes +wyt:in jet, gated by compile-time assertion
u3: further optimizes +lent jet, gated by compile-time assertion
u3: refactors allocator constants, adds u3a_cells and u3a_maximum
u3: optimizes +lent jet, avoiding u3i_vint() while possible
u3: moves cell allocation counter into u3a_celloc()
u3: fixes memory leak introduced in +murn jet
u3: fixes mismatches in +div and +dvr jets
...
Instead of always providing a wildcard for the allowed methods and
headers, now echoes back the method and headers that the client asked
for, if any.
Fixes#3676.
Disallows registering bindings (through %connect and %serve) that would capture
traffic on paths starting with /~ (Eyre's) or /~_~ (runtime's, as of cc389c5).
Note that we don't touch +insert-binding, which is used by Eyre internally to
set up bindings in its own namespace.
Lets you check whether a specific Cookie header value string constitutes an
authenticated request.
/ex/=//=/authenticated/cookie/(scot %t 'cookie-string')
Intended for use in the runtime, for example with #3557.
Adds a cors-registry to Eyre's state that tracks allowed and rejected
origins for the purposes of CORS request handling.
For preflight requests, generates a response in-line.
For simple requests, adds CORS headers onto whatever response is given.
See also:
https://groups.google.com/a/urbit.org/g/dev/c/bb82dwEJGzM/m/q2JjNSx5BwAJ
This was originally introduced by me in #1814 to address #1811. Eyre was not
canceling heartbeat timers on all relevant events making it easy to end
up with an infinite behn loop. This check allowed ships that entered an infinite
loop to recover, as per my comment at
https://github.com/urbit/urbit/pull/1814#discussion_r333477482. Otherwise it's
not necessary.
@t further indicates to the caller (although loosely, because auras
are loosely enforced types) that the input should not contain embedded
NUL bytes, which are not valid @t.
Separately, a minor improvement has been made to the performance of
the raw hoon by pinning the gates used in the inner loop.
Prior to this commit, there was a jet mismatch in to-wain (formerly
called lore, and still jetted under that name). 0 bytes in the middle of
a cord caused the jet to crash, whereas the hoon simply treated them as
the end of cord and truncated the output. The history of this behavior
is fraught with controversy. This commit rectifies the current mess with
the following rationale: Null bytes are valid ASCII/UTF-8, and \n\n in
the input will cause null list items in the output, so nulls are (for
the purposes of to-wain) allowed in cords. Trailing nulls cannot be
represented because of the nature of atoms, but that is outside the
scope of to-wain's concern. Therefore to-wain should simply measure the
cord and split on newlines, and do nothing fancy at all with nulls.
In addition, the hoon for to-wain was written in an inefficient style
that produced a lot of intermediate garbage atoms via rsh and cat. This
commit's implementation measures once and cuts once, so to speak, and so
avoids the intermediate garbage. Quick benchmarks suggest it is about
20x faster than the old hoon, but still orders of magnitude slower than
the jetted code. to-wain is the workhorse for the txt mark, so we should
still prefer to have a jet.
The old jet is left wired up under %lore, and should be removed when
support for the old, unupgraded zuse is no longer necessary. A new jet
with matching null handling has been wired up under the name %leer.
Well-behaved secp jets can now plausibly exit if they are given inputs
of the wrong size/range. Previously this was either not checked or the inputs
were silently truncated.
The secp core had some flaws: in particular, the logic for signing/recovery
did not match libsecbp256k1 w.r.t. the enigmatic "recid" (v) value. The jet
hints were also subtly wrong, in that the curve parameters were in a sample
(not an arm) and thus not matched by the jet matching scheme. Consequently,
the jets would be used (but incorrect) for other curve parameters.
Tests were also added to exercise the recovery id cases thoroughly.
Depending on the additions to term.c made in 467d8d239 allows dill to
forget about ansi escape codes, and pass styled text nouns straight on
to vere.
Also removes a bit of logic from drum, which assumed things about the
rendering of escape codes to adjust cursor positioning. Now it simply
states the semantic cursor position, letting the runtime deal with the
potential influence of styling.
If both sides changed a file in the same way, %mate used the version in
the mergebase, which is incorrect. This changes it to use the version
in the destination desk.
An example of this issue:
> +cat %/test/hoon
/~zod/home/~2020.9.3..21.41.24..61ed/test/hoon
first
> |merge %scratch our %home
>=
merged with strategy %fine
+ /~zod/scratch/2/test/hoon
> +cat /=scratch=/test
/~zod/scratch/~2020.9.3..21.41.32..408c/test/hoon
first
> *%/test/hoon 'second'
: /~zod/home/3/test/hoon
> *%%%/scratch=/test/hoon 'second'
: /~zod/scratch/3/test/hoon
> |merge %scratch our %home
>=
%fine merge failed, trying %meet
%meet merge failed, trying %mate
merged with strategy %mate
: /~zod/scratch/4/test/hoon
> +cat /=scratch=/test
/~zod/scratch/~2020.9.3..21.42.25..9e8b/test/hoon
first
Ordinarily, eyre cleans up the relevant gall subscriptions whenever a
channel disappears. In yet unresolved erroneous behavior though, it may
leave a gall subscription open, despite wiping the channel from state.
Attempting to pass the response onto the deleted channel anyway results
in an %eyre-no-channel error later in the event. The volume of these
errors can degrade the user experience, as per #3196.
To resolve the annoyance (but not the underlying issue) we detect the
"subscription has no channel" case, and issue a %leave. Doing so
requires additional information in the wire, so we add that in,
refactoring the relevant wire building along the way.
Note that due to the wire requirements, this cannot resolve existing
cases. For that, we depend on bc929ba6d.
As part of the solution to #3196, we need to clean up any gall
subscriptions that eyre didn't properly clean up.
Since detecting that is hard, we opt to just wipe _all_ eyre-originating
subscriptions from gall. We inspect the duct, which isn't good, but it's
only just this once.
The main thing here is that we aggressively check whether we're in
ancestry of another mergebase candidate. This means we don't have to do
a 2nd pass to eliminate redundant candidates.
Change the definition of base-hash to be the mergebase of %home with the
OTA source. This means it's the most recent successfully-applied
update, which is usually the most important information.
Add sour-hash, which is the hash of the most recently *downloaded*
update, regardless of whether it applied successfuly (ie the old
base-hash).
Add a summary of the various hashes at the top of gen/trouble.
Only no-op if the incoming commit's parent is the old head of the desk.
Also move the printing near the end so we can know exactly if anything
changed.
Jael now stores a `step` that is combined with the original salt to
produce a new code. A `%step` card is used to increment that value,
and effectively resetting the keys. Because the first `step` is zero,
the first code is the same as before.
Eyre was changed to be notified with `%code-changed` so it can forget
old cookies, sessions and discard all the existing channels.
A new generator was added |code, that does both querying and
resetting the code
|code :: shows current code, step and help
|code %reset :: changes the code
The old +code generator still works correctly.
* master: (915 commits)
vere: bumps urbit version to v0.10.8
pill: updates all
king: fix ames tests
contact-store: restore /~/default contacts
contact-hook: resubscribe on correct paths
u3: note that u3a_rewrit* doesn't yet support south roads
king: it was too clever of me to use stateTVar; compiler can't help
king: fix comment about ames q behavior
king: ames bounded q, now with logging and fifo
serf: tweaks |pack and |mass printfs
u3: moves u3a_compact to u3m_pack, refactors internals
metadata: handle OTA correctly
u3: refactors u3m_reclaim() into noun modules, works on any road
release: urbit-os-v1.0.30
group-store: remove scries from OTA logic
release: urbit-os-v1.0.30
MAINTAINERS: amend for post-fusion
ames: add scry endpoint for forward lanes
ames: improve scry interface
chat, publish, contacts: fix OTA bugs
...
We used to not accept new indirect lanes if we already have a direct
lane. This means that if Bob, with a publicly-accessible lane, changes
lanes (eg by restarting the process and getting a new port or changing
ip addresses), tries to talk to Alice, who is behind a NAT, then Bob
will try directly but fail (because Alice is behind a NAT), so he will
route the message through her galaxy. This is good -- the message gets
to Alice. However, Alice had a direct route to Bob's old lane, so she
will try to ack on that lane, which fails. She will not time out this
lane because she doesn't know that Bob isn't getting the acks (acks
don't have their own acks).
The solution is that if Alice receives an indirect lane for Bob when she
already has a direct lane, she shouldn't ignore it. If the lane is the
same as what she has, she shouldn't change anything (in particular, she
shouldn't mark it as indirect). But if it's a new lane, she should
discard her old direct lane and use the new indirect lane.
RFC2396 defines[1] unreserved characters as alphanumerics and nine "mark"
characters. We were only parsing for four of those, leading to parsing failure
for valid URLs.
[1]: https://tools.ietf.org/html/rfc2396#appendix-A
In Ford Fusion, Clay builds generators but Dojo and Eyre run them. Dojo
is already virtualized with a scry function, so +mule is fine, but Eyre
is not, so Eyre needs to use +mock and explicitly supply the scry
function. This does that. Fortunately, the produced result is simple
and easily clammable.
Fixes#3089
No longer abuse the desk field, instead making use of the path. Reject
any scries outside of the local ship, empty desk and current time as
invalid.
Expose ducts only under a debug endpoint, nothing else should care about
being able to inspect them.
Add scry endpoints for the very next timer (if any), and all timers up
to and including a specified timestamp.
When merging, +reachable-takos is called roughly once per merge commit
in the ancestry of the new commit. +reachable-takos was exponential in
the number of merge commits in the ancestry of the commit it's looking
at, due to mishandling of the accumulator. This makes it linear.
Of course, linear x linear is still quadratic, which is not great. I
doubt +reachable-takos can be made asymptotically better, but
+reduce-merge-points/+find-merge-points probably can. 50 merge commits
already gives about 14.000 iterations through the loop in
+reachable-takos. Another option is to try to memoize this somehow, but
a simple ~+ is insufficient since `s` is usually different.
In local tests on macOS with a -L copy of ~wicdev-wisryt, this speeds up
OTAs significantly. The majority of time was spent on this.
* origin/jb/aes-siv-fix:
tests: updates aes-siv regression test comment
pill: updates solid
zuse: propagates fix to aes-128-siv and aes-192-siv as well
Revert "test: disable aes-siv jets to demonstrate test failure"
pill: updates solid
zuse: fixes bug in aes-256-siv iv calculation (+s2vc:aes:crypto)
test: disable aes-siv jets to demonstrate test failure
test: add test case for aes-256-siv jet mismatch, observed in the wild
Signed-off-by: Philip Monk <phil@pcmonk.me>
Adds +mure to run a trap in a separate road. This should eventually be
just a hint.
Vega was running inside a mule, but since +load was called within vega,
the new kernel was all run within the same mule, so it didn't actually
get to reclaim the space after hoon compiled.
We verified this with printfs in u3m_fall. On the test ship (from
mainnet) which had 800MB used, vega was taking interior free space from
950MB to 450 over the course of compiling hoon, then each vane would go
from about 450 to 350 and then back to 450 once it finished (which
proves they were correctly isolated). With this change, after hoon
compiles the free space goes back up to 950MB. This gives us a lot more
space to compile OTAs.
We had to slightly refactor the logic for doubly-recompiling hoon, since
+mure as written produces a ?(!! _trap), and you can't find faces in the
result of the trap. We could bake mure, but that's rather awkward. I
wonder if there's a way to fix this as a wet gate.
Attempt to convert the scry result to the mark that was asked for,
failing the scry (with ~) if the conversion fails.
Eyre's scry logic, then, can pass the requested mark directly into gall.
Exposes a scry endpoint. Any requests made to the /app/scry.mark url
under the endpoint will scry into %app using a %gx scry, at the
/scry/noun path, and attempt to convert the scry result into the %mark,
before converting that into the %mime mark, and sending that as an http
response.
In addition to producing the action bound for a given request, now also
produces the subset of the request url that comes _after_ the path at
which the binding has been established.
Will allow some bindings to more easily dispatch off the relevant part
of the url.
If we failed the password check, the login page served to us would never
include any redirect details, even if they were there in the original request.
Now we simply (attempt to) parse out the redirect field a little earlier.
Associates channels with the authentication sessions that opened them,
and deletes the channel when its associated session expires.
Also updates the debug dashboard to display channel counts per session.
Turns +on-channel-timeout into +discard-channel, which cleans up the
entirety of the channel, based on its current state. This allows us to
simplify the %delete channel request into a simple function call.
Changes the HTTP status code of the redirect that occurs upon a
successful login from 307 to 303. 307 preserves the method of the
original request, so the redirected request is a POST. With the new SPA,
this causes a 404 as app/file-server validates the method of the
request, something that did not happen in earlier versions of landscape.
303 instead changes the method to always produce a GET request.
Set up, by default, on /~/logout.
Sending a POST request to this expires the current session and redirects
to the login page. If the "all" key is set in the request body, expires
all open sessions.
We build a reef for each desk but use the compiler from our kernel. At
some point we should use the compiler from the desk, but then we need to
validate any results we get from it.
For request transparency, HTTP proxies may set the Forwarded header to
specify who the original requester is.
For requests from localhost only, we make Eyre respect the Forwarded
header, and adjust the handled ip address accordingly.
Note that we do not support X-Forwarded or other non-standard variants.
The header remains in the request, so server applications can handle
them as desired.
Fixes#2723.
When sending a response to an authenticated request, update the session
to last for +session-timeout again, and send an updated cookie to match.
Assuming the user makes an actual HTTP request at least once a week,
this will make sure they don't get logged out automatically. Simply
keeping a channel open, unfortunately, doesn't count.
Instead of setting a timer for every session, we set a single expiry
timer when the first session is created. On the subsequent wake event,
we clear all cookies that have expired at that time, then set a timer
for when the next session expires.
This approach gives us flexibility wrt sessions going forward, allowing
extending or early deleting of sessions without having to care about the
related timers.
Note that in +load, we clear all existing sessions. We would start the
expiry timer flow there, but can't. Forcing the user to login again
post-ota once isn't the end of the world.
We inspect the wire of our subscriber to see if we need to produce the
result as a %public-keys or a %boon. This is bad -- we should proxy the
subscription to avoid this need, but this doesn't make that change yet.
%pubs is an old name that doesn't exist anymore (last existed around
September 2019). The new version is /public-keys, but it's worked so
far because /public-keys has only one item in the path, so it missed the
conditional. This commit makes the intent more clear.
The [%a @ @ *] could be just [%a @ *], but I leave it to reduce the
chance of breaking stuff.
Somehow we ended up with flows which expected to awaken but did not wake
up. This was likely caused by the error in r920j OTA, urbit-os-v1.0.18.
This adds a command which ensures that every flow has an active timer.
I expect this to be needed only once, but it's a pretty general tool, so
it's worth keeping.
I've included an unused @t parameter to more easily add simple debug
commands to ames without having to add a new task
The subscription changes in drum broke existing subscriptions. This
worked alright (though loud) for dojo, but it left chat-cli "frozen"
unless you manually unlinked/relinked. This does that automatically.
It also includes a refactoring of +on-load in drum, to avoid vain
repetition.
We need to get updates directly into %home in case the marks depend on
changes to hoon.hoon. %base has no reason to exist.
Our ota strategy is now to merge from parent/kids to home, then
parent/kids to kids.
* origin/release/link-dojo:
chat-cli: allow sending • character
chat-cli: always talk to local ship only
chat-cli: single-target sole effects as needed
chat-cli: don't allow excessively small cli widths
chat-cli: pull in sole-sur namespace where relevant
chat-cli: remove unused entropy from state
chat-cli: print newlines correctly
chat-cli: support multiple sole connections
chat-cli: don't crash on %bad-text
dojo: rename remote access generators
gall: fix handling of empty path list
dojo: remove unused %json poke
dojo: add remote access controls
drum: switch to per-ship /sole/drum duct
Signed-off-by: Philip Monk <phil@pcmonk.me>
At some point this should be more properly styled similar to +by, +in,
and +to, but for now this reduces duplication and makes the ordered map
available to everyone.
