urbit-v1.11
Arvo 417K (zuse+417, lull+328)
Vere 1.11
This is a hotfix release, fixing a memory corruption vulnerability
introduced in v1.10, and including official aarch64 binaries.
Release Notes
- repositions the guard page (preventing road stack overflow) on inner roads
- decrements %lull and %zuse kelvins, in preparation for a pending arvo release
- adds support for static aarch64 binaries
- cleans up the terminal on exit when booting a fake ship
Contributions:
Bradley (2):
vere: changed to use u3_king_bail instead of manual term clean up
vere: removed unnecessary exit(1) call
Bradley Ray (1):
vere: clean up terminal on invalid fake ship name
Joe Bryan (11):
u3: reposition guard page if needed for new roads
test: initialize guard page in jam-tests
vere: decrement %lull and %zuse kelvins
Merge commit 'ac5842fd6e' into jb/hotfix
vere: bumps version
build: add aarch64 via buildjet
build: run unit tests early on linux
build: run unit tests on windows
build: updated gcp configuration for buildjet runner
build: renames "nightly" pace from "often" to "edge"
vere: bumps version
~botter-nidnul (1):
nix: enable aarch64-linux platform string
Also, remove the conditional rendering logic for AppPrefs. With the new logic (and before this change) if the User disables OTAs, the toggle would disappear, which feels like an antipattern.
- fix `fragment-num` and `num-fragments` having duplicate faces
- fix faces being wrapped around wrong things in various places
- fix `bone` not being printed in "hear last in-progess" message
- make pretty tape interpolation style more uniform
in the past, +team meant "our / our moon", but
it has been primarly used to represent "our"
moons as having the full permissions of their
parents doesn't make a lot of sense anymore
this looks like the more elegant solution
instead of changing each instance of +team
I've combed through the uses of +team throughout
urbit/urbit and I'm quite sure that each instance
is better off as just "our"
The +on-cork handler asserts that the peer is known to us. This is the
incorrect behaviour, because it will crash when corking a flow to a peer
that is still an %alien. This can happen, for instance, when making a
gall subscription for the first time and then corking it before the
alien naturalises.
`+story-list` produced janky indentation because the `$-(story
wain)` functions encoded linefeeds in the cords of the wain and the
printer doesn't like this.
Story printing functions have been changed to produce pure wains without
linefeeds.
Any change to the session object was triggering this. But that now includes
an "unacked keystrokes" counter, which updates frequently, and we
definitely don't want to send resize notifications in that case.
Instead of forcing people to connect over http://, fall back gracefully to http if no protocol is given.
This fixes an issue where external clients can't use this method, since SameSite cookies need to be secure as per https://web.dev/samesite-cookies-explained/#samesite=none-must-be-secure
PR #5840 mostly fixed#1559, but introduced a new bug. before, you could safely `=dir` into a desk without a case, and it would use the nonexistent case `ud+0` as the beam for dojo state, and switch that out for da+now whenever it tries to resolve the current path. but this check causes it to fail, because `ud+0` is a nonexistent case. this uses he-beam to transform the beam in the conditional to see if the case is 0, and if it is, changes the case to da+now before it scries
if a cert is configured and a secure port is live it will set the
redirect flag in http-config.state.
When it gets a ++request it will return a 301 redirect to
https://[host]/[path] if:
1. not already secure
2. redirect flag set
3. secure port live
4. is not requesting /.well-known/acme-challenge/...
5. the host is in domains.state
It will not happen if forwarded-secured, localhost, local loopback, ip
addresses or domains not in domains.state.
in ++load it checks the secure port is live and a cert is set and
enables it if so (for people who already use in-urbit letencrypt)
%rule %cert tasks also toggle it (only turning it on if secure port
live)
%live tasks also toggle it (only turning it on if cert set)
Have tested with a couple of ships and seems to work fine.
This is useful in combination with pyry's auto arvo.network dns config
system - can finally get rid of reverse proxies entirely.
Eyre always gets passed request headers in lowercase, so we should search for
the lowercased version of the header.
Arguably `+get-header` should lowercase keys before comparing them, but that's
a more serious behavioral change.
This allows you to pass a thread directly into khan, instead of passing
a filename. This has several implications:
- The friction for using threads from an app is significantly lower.
Consider:
=/ shed
=/ m (strand ,vase)
;< ~ bind:m (poke:strandio [our %hood] %helm-hi !>('hi'))
;< ~ bind:m (poke:strandio [our %hood] %helm-hi !>('there'))
(pure:m !>('product'))
[%pass /wire %arvo %k %lard %base shed]
- These threads close over their subject, so you don't need to parse
arguments out from a vase -- you can just refer to them. The produced
value must still be a vase.
++ hi-ship
|= [=ship msg1=@t msg2=@t]
=/ shed
=/ m (strand ,vase)
;< ~ bind:m (poke:strandio [ship %hood] %helm-hi !>(msg1))
;< ~ bind:m (poke:strandio [ship %hood] %helm-hi !>(msg2))
(pure:m !>('product'))
[%pass /wire %arvo %k %lard %base shed]
- Inline threads can be added to the dojo, though this PR does not add
any sugar for this.
=strandio -build-file %/lib/strandio/hoon
=sh |= message=@t
=/ m (strand:rand ,vase)
;< ~ bind:m (poke:strandio [our %hood] %helm-hi !>('hi'))
;< ~ bind:m (poke:strandio [our %hood] %helm-hi !>(message))
(pure:m !>('product'))
|pass [%k %lard %base (sh 'the message')]
Implementation notes:
- Review the commits separately: the first is small and implements the
real feature. The second moves the strand types into lull so khan can
refer to them.
- In lull, I wanted to put +rand inside +khan, but this fails to that
issue that puts the compiler in a loop. +rand depends on +gall, which
depends on +sign-arvo, which depends on +khan. If +rand is in +khan,
this spins the compiler. The usual solution is to either move
everything into the same battery (very ugly here) or break the
recursion (which we do here).
Before this, the %watch to eth-watcher was happening before the %poke,
and so eth-watcher was responding with its entire history immediately.
This is bad because it takes a lot of memory to process that many logs,
and also because those logs are stale.
Now, the %poke happens first, which clears the history.
%kick is supposed to start back from the snapshot and move forward.
Without this, we would only fetch logs that we hadn't already fetched.
Thus, if you were up-to-date when you kicked, you would miss anything
that happened between the time the snapshot was taken and the present,
though you would see things after the present.
Also reverted lull change to make this a safer upgrade.
Previously, when the larva got to processing enqueued events, it was
doing so without loading state into the adult beforehand, resulting in
incorrect processing of events.
Here, we make the larva call +molt more eagerly, ensuring that the adult
always has its state available when we use it.
Yes, there is a global timer for closing flows, but all that does is
enqueue a cork message. +on-stir needs to set _pump_ timers for all
flows that might still have messages to send, which includes closing
flows.
When ames notifies us that our subscription has been kicked, we enqueue
a cork to clean up the flow. Unlike the %leave case, however, we were
not registering the cork in the queue of outstanding comms. We would
eventually get an ack, but not know what for, and erroneously inject
%poke-acks and %watch-acks.
Here we simply add a %cork entry to the queue before sending it.
This is sufficient to bring the normal (non-prerelease-bugged) cases
into the new world.
For the prerelease ships that ran a buggier version of the new gall
subscription logic, we note that the conditional may trigger for the
nonce=1 case where it had already triggered for their
(shouldn't-be-possible) nonce=0 case. This results in a %leave on a wire
that wasn't in use. This no-ops on the publisher side though, and the
flow gets corked right away, so this is considered harmless.
* master:
bitcoin: v0.0.2
bitcoin-wallet: set state as default case for handle-provider-update
%bitcoin: Implement additional RPC calls from btc-provider. %histogram, %block-headers, %tx-from-pos, %fee, %psbt are now all callable from the btc-provider agent. These actions are necessary in order to get the lightning network working within Urbit.
%bitcoin: Added %regtest to arms using the network type definition.
%bitcoin: add regtest type to network
In response to clog notification from remote ames, we were sending a
%cork to clean up the flow. However, the wire we were using had the /sys
prefix already stripped off. Here, we put it back in.
Start by killing subscription nonce 0, then work our way up instead of
down. We enhance the printf with a "total nonces" indicator so we can
still easily see the progress being made.
Previous +ap-doff kicked the agent repeatedly. We needed to kick
it only once. Now publisher agents clear their incoming subscription
state without the subscriber making lots of new subscriptions because
of repeated kicking.
+on-plea gets called in two very different ways:
1) handling request from local vane to send %plea to peer
2) handling %cork request from another ship, which our local ames has %pass'ed
to ourselves
In the second case, we shouldn't print misleadingly, or bind a duct in the ossuary.
+ap-nuke was not including the nonce, but should.
+ap-handle-peers was potentially including a zero nonce.
(The latter shouldn't have been possible, but there's a bug in +load
where sub-nonce.yoke gets initialized as 0 instead of 1.)
Gall tells ames to %cork flows for subscriptions it has closed.
Receiving a kick also closes a subscription, but gall wasn't issuing a
%cork in that case. We correct that here.
Inlines +mo-handle-ames-response's logic at its only callsite.
seems that this structure has been unused since
e75ab631a4 and confuses
newbies trying to figure out exactly what the commit
structure is (which is how I came across this)
Without this, a ship would send a cork on a max of one flow per
recork timer, which could take years to clear for some ships.
This starts a hot loop of trying the next cork once one gets
positively acked.
The previous recork timer queued up %cork messages without sending them.
It also relied on making sure pump timers didn't get set for recork bones.
This was fragile.
The new design enqueues up to one new %cork message per ship during each
recork timer, based on the state of the flow. If the flow is closing but
there are no outstanding messages in it, then it needs to be recorked.
Flows will be recorked in ascending numerical order by bone.
The condition got butchered during refactor: instead of avoiding the creation
of pump timers during recork wake, it was setting them _exclusively_ during
recork wake.
Currently when creating a fake ship, if an invalid ship name is given,
then the program exits without ever cleaning up the terminal. This
results in a bugged termianal that requires closing and repopening
or using the `reset` cmd.
This commits adds a call to `u3_term_log_exit()` and `fflush(stdout)`
before calling `exit(1)` to ensure proper cleanup.
Address issue #5914
* next/vere: (49 commits)
vere: bumps version
vere: bumps version
ci: enable release version mgmt
Revert "ci: reenable release mgmt"
ci: reenable release mgmt
ames: track/log bad ciphertext crashes separately
u3: %evil leaves no trace
vere: drop bail:evil events without error notifications
jets: bail:evil in ae-siv decryption jets
vere: make uv_cancel return code check explicit
u3: check for overflow in interpreter if guard page not present
build: default to using guard page
u3: return 0 from u3e_fault() if guard page cannot be protected
u3: control presence of guard page using macro
Revert "u3: check for road stack overflow on every nock %2 and %9"
build: make bench is phony
u3: failure to mprotect() the new guard page is fatal
u3: removes/disables obsolete road stack overflow checks
u3: account for guard page in cellblock allocation conditional
u3: bump road heap offset before allocating cellblock
...
This test started failing presumably somewhere during #5886. Testing
with a comet on the network, the test seems inaccurate: the comet can
communicate and be communicated to just fine.
Before this change, `term/lib` was importing the Poke type from the `http-api` package. This was causing the rollup build output to place the `term` exported types in a separate path (`dist/api/term`). By switching to the relative import, it is now exported at the expected path (`dist/term`). This fixes imports in consuming projects (e.g., `import { Belt, pokeTask, pokeBelt } from '@urbit/api/term';`).
Also, remove the extraneous `Scry` import.
this refactors the parser for %brcn and %brpt to separate the optional
argument(s) from the required argument(s).
also adds +blab, which allows for a minor refactor of a couple other
arms as well as being used for %brcn and %brpt
When the first byte is greater or equal to 0xfd, (bex len) bytes are consumed to
form the csiz atom, but only one byte is dropped from the 'rest' of the input.
The parser should consume all bytes of the CompactSize.
In reconnect scenarios (or wonky network situations) we may receive
events we had already heard. Here we make sure to drop those.
Also simplifies the getEventId() logic.
- Fixes an issue where behn would fail to report the next timer to vere
correctly, resulting in timers only firing once every ten minutes.
- Updates |rein to be additive instead of fully replacing existing
configuration. Specifying a single agent to start will no longer stop
other explicitly-started agents.
- Various QoL improvements to dojo.
- Updates DNS requests to go to ~deg instead of ~zod. The Foundation
will take over DNS request handling.
Contributions:
David Farrell (5):
dojo: have dojo check =dir exists before switching
dojo: simplify dir not exist error message build
dojo: setting eny/now/our shouldn't crash dojo
dojo: say how to beat the %dy-edit-busy escape room
dojo: remove unnecessary debug output
Joe Bryan (8):
behn: adds (failing) tests
behn: unconditionally clear runtime timer state on %wake
behn: don't compare pending timers to now
behn: refactor to use +abet pattern
behn: emit %doze on any rescheduling of the next timer
behn: cleans up comments
Merge pull request #5858 from urbit/jb/behn-fix2
test: removes debugging comments from %behn unit tests
Sidnym Ladrut (1):
hood: fix issue w/ rein diff application
Yaseen (1):
eyre: Modify landing title from "OS1" to "Urbit"
fang (16):
clay: render syntax errors at end of file
Merge pull request #5811 from ynx0/patch-1
Merge pull request #5812 from urbit/m/eof-syntax-error
Merge pull request #5837 from dnmfarrell/dnmfarrell/arvo-dy-edit-press-bksp-to-abort
Merge pull request #5840 from dnmfarrell/dnmfarrell/arvo-dojo-check-dir-exists-before-switch
Merge branch 'master' into next/arvo
gall: exclude from traces
Merge pull request #5841 from dnmfarrell/dnmfarrell/arvo-dojo-dont-crash-set-beak
Merge pull request #5863 from sidnym-ladrut/sl/fix-rein-toggle-behaviori
Merge pull request #5868 from urbit/pkova/dns
Merge branch 'master' into next/arvo
Merge pull request #5857 from urbit/jb/behn-fix1
Merge pull request #5855 from urbit/m/gall-quieter
hood: re-patch |rein
Revert "clay: render syntax errors at end of file"
Revert "gall: exclude from traces"
pkova (1):
dns: transfer ship.arvo.network domains from ~zod to ~deg
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEB0bRQARxix0iuhs56wNXYMG7qXIFAmLPF9wACgkQ6wNXYMG7
qXLO5w/7BSc5aajTaA/o8ma4SoySX/PXH1dnnoJO9A2fiYR3sIpcdY12xNmnCA/e
y0OyQeSqmRzO90bVJI3AVxN0qEDcz0fR9ZoC8YeYWRrAWy7xRTm808NRW1rovAEN
LIlOemUpJODlqkkiD0x8hwxNSix9+tkcr6ehilzIuORaBcTI586rekT+FfXGuD9Z
IURbKkKS443M3kIyL9MUXu8lopMpfwZg9VYwo3a8fnUvUmVRleHBpUI71GXd41w4
eQv63197mcwMq/od9JUXBdX1yek/QyVV0SqZhPCksUOipdCbH0oJpzBmh/4i0CGb
Ij2gyGlth4iErXx+TBBpHk0FKQysHoPQyBI3/ljtZG4xpVToKHyuw7hODHR4STUL
hYV+iWbOJ+reh0Zkrk/SjZHZMYBKcuBonIXwNXDE5cC50q+dqqaj8+0oPOmdjhDy
rbzN377Ijags+ivLagpra1KE+lVLtKbOae16FqzBZJqs7xuj9ZsoabG+OxoXDH7j
i1oXIskYuIzHfbSbDk2eaCPKONFs19Oi8ULg5/zxaJsQLks9WMuckBPFr4paCrdo
eBFe139f87xBi8WBZ7rzYyrUFtGhTLo1Bzapdbvh18WhYfK95sTo+PCUJNHB29+H
qmkE3bMZ+v2VOPLFtRfrb1bolpsoR/m5Z6SWElMB2Ig2WKkDqvE=
=CxGA
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEB0bRQARxix0iuhs56wNXYMG7qXIFAmLPJAQACgkQ6wNXYMG7
qXLiyxAAmOlm72h+UC34dnmtiIKjRyuq+XXl8MdyPu0XZPKS1XzewFAP+iQE1uVG
X5HJ04xl6SI/npa99p95K7f2T69OkXWvwax8fhu/+hhIVq8bQAOfzMIW5jk2Bfzl
j1wMjcjzDZquqNVaYHsYAmnUZFl2Tp3mv3j3VgxsyUfdXFKQLz9g7lhBellIZ1F+
HTfK8YBWGhzfUoJ/3pfXp7Q/6w60pW4AC/6dbfbmKW5rcZpyATWhIbYWullPsJWC
pjf5KYjwiiOVQU/0cZ5alXZX0ofV4Tf9er5I4Np0fF3j/5O1vdmBJ9E4RGJXJMrR
sQ6xt7yuX5wrpbUTqgvxA7sSjEB4+lk8sZJvC5KpLIvNF6s4o1aUnHYSODeIr0r3
vkx7YYfhbDv84xHA3lIejh7vcM5182dq0cNhHD1uLGt81IeE+YCl0G1cvUV5QxrQ
WGt3uLanIpBAJjseMc4N3mUcG343G0vs8058gZtwqsGXy7byF3tFKmt1kHPh2pjP
W9HJ0+MN7VSGqNidMv74r4ZpsDPdERkUqdhzb30tvBBp6xraXflVjvUT5Ln+501p
m2PO5aLbeVclIViwsC7wFLYooHDz5O7hZx5Sf1uUjNsaGYSPotPGH8UsXSiyKZ5/
p067gSrcyqHuLRYR/7izDCe7xTHImpP3doL6xmW4br2gl5wKX1s=
=BGu/
-----END PGP SIGNATURE-----
Merge tag 'urbit-os-v2.124'
urbit-os-v2.124
- Fixes an issue where behn would fail to report the next timer to vere
correctly, resulting in timers only firing once every ten minutes.
- Updates |rein to be additive instead of fully replacing existing
configuration. Specifying a single agent to start will no longer stop
other explicitly-started agents.
- Various QoL improvements to dojo.
- Updates DNS requests to go to ~deg instead of ~zod. The Foundation
will take over DNS request handling.
Contributions:
David Farrell (5):
dojo: have dojo check =dir exists before switching
dojo: simplify dir not exist error message build
dojo: setting eny/now/our shouldn't crash dojo
dojo: say how to beat the %dy-edit-busy escape room
dojo: remove unnecessary debug output
Joe Bryan (8):
behn: adds (failing) tests
behn: unconditionally clear runtime timer state on %wake
behn: don't compare pending timers to now
behn: refactor to use +abet pattern
behn: emit %doze on any rescheduling of the next timer
behn: cleans up comments
Merge pull request #5858 from urbit/jb/behn-fix2
test: removes debugging comments from %behn unit tests
Sidnym Ladrut (1):
hood: fix issue w/ rein diff application
Yaseen (1):
eyre: Modify landing title from "OS1" to "Urbit"
fang (16):
clay: render syntax errors at end of file
Merge pull request #5811 from ynx0/patch-1
Merge pull request #5812 from urbit/m/eof-syntax-error
Merge pull request #5837 from dnmfarrell/dnmfarrell/arvo-dy-edit-press-bksp-to-abort
Merge pull request #5840 from dnmfarrell/dnmfarrell/arvo-dojo-check-dir-exists-before-switch
Merge branch 'master' into next/arvo
gall: exclude from traces
Merge pull request #5841 from dnmfarrell/dnmfarrell/arvo-dojo-dont-crash-set-beak
Merge pull request #5863 from sidnym-ladrut/sl/fix-rein-toggle-behaviori
Merge pull request #5868 from urbit/pkova/dns
Merge branch 'master' into next/arvo
Merge pull request #5857 from urbit/jb/behn-fix1
Merge pull request #5855 from urbit/m/gall-quieter
hood: re-patch |rein
Revert "clay: render syntax errors at end of file"
Revert "gall: exclude from traces"
pkova (1):
dns: transfer ship.arvo.network domains from ~zod to ~deg
Problem:
by-channel has its own copy of server-state from line 2182. discard-channel returns an altered state, with one channel removed from the state of by-channel.
but the state of by-channel isn't changing with each iteration, so |trim is only removing one channel per invocation.
Solution:
update by-channel on each iteration.
xterm.js seems to have a bug where it doesn't register ctrl+uppercase
keypresses correctly. To work around this, we catch such keypresses
explicitly and handle them inline.
%histogram, %block-headers, %tx-from-pos, %fee, %psbt are now all callable from the btc-provider agent. These actions are necessary in order to get the lightning network working within Urbit.
this commit replaces the previous intermediate parsing structure, $whit,
with a new one better suited for batch comments and taking into account
that {# %label} syntax is no longer being used anywhere. basically,
this makes it so that all doccords are batch comments, where if they are
preceded by a (list link) then they will try to attach to the given
link (only utilizes first link for now), and a blank link means it will
try to attach to the following hoon or spec
This is a temporary fix, and first part of the gall-request-queue-fix
release in two stages. This gives a publisher ship the ability to
understand a %cork and handle it properly, but no subscriber will
be sending %corks at this stage when leaving a subscription.
We still add a nonce to all subscription wires but it doesn't
increment it when resubscribing, allowing flows to be reused.
Tested locally with toy pub/sub agents and Group join/leaving
* master: (61 commits)
rich-text: removing broken plugin, reverting react-md to 5.0.3 for compat
graphcontent: blockquotes now correctly break paragraphs off
md-editor: fixing background in darkmode
landscape: [skip actions] update glob (0v5.kgrq2.gp725.bo5bk.dmr7d.h41qk)
ops: fixing build
meta: version bump for new changes from next/landscape
meta: package bump:
eventsource: prevent resetting event id counter
interface: fixing react-codemirror2 dep
groups: add webp to list of image types to check links for in chat
groups: fix issue where URL would be shown along with image in chat
groups: fix issue with chanel perms caused by use of Set()
groups: update glob (0v4.2se6m.fvv67.nn5e8.vfrv9.mmi88)[skip actions]
groups: updating package lock
groups: updating nvmrc
Fix webpack, build dependency issues
compat: fixing react-codemirror2 deps
compat: updating to latest sigil-js for react
compat: updating indigo, and removing unused deps
meta: fixing react version issue
...
Previously, the initial Azimuth snapshot was stored in Clay and shipped
in the pill. This causes several problems:
- It bloats the pill
- Updating the snapshot added large blobs to Clay's state. Even now
that tombstoning is possible, you don't want to have to do that
regularly.
- As a result, the snapshot was never updated.
- Even if you did tombstone those files, it could only be updated as
often as the pill
- And those updates would be sent over the network to people who didn't
need them
This moves the snapshot out of the pill and refactors Azimuth's
initialization process. On boot, when app/azimuth starts up, it first
downloads a snapshot from bootstrap.urbit.org and uses that to
initialize its state. As before, updates after this initial snapshot
come from an Ethereum node directly and are verified locally.
Relevant commands are:
- `-azimuth-snap-state %filename` creates a snapshot file
- `-azimuth-load "url"` downloads and inits from a snapshot, with url
defaulting to https://bootstrap.urbit.org/mainnet.azimuth-snapshot
- `:azimuth &azimuth-poke-data %load snap-state` takes a snap-state any
way you have it
Note the snapshot is downloaded from the same place as the pill, so this
doesn't introduce additional trust beyond what was already required.
When remote scry is released, we should consider allowing downloading
the snapshot in that way.
using col as a seperate made it look like a bunch of =<, which doesn't
make sense for e.g. pritning chapters. paths aren't quite right either,
so we don't use +stap, and just want identifiers separated by fas
without any leading fas
changes $whit to have a (unit link) instead of (unit term). this holds
the identifier for where a comment is supposed to go. changes to parsers
in docs:vast to accomodate this.
this only allows for batch comments written for arms within a given
core. someday, the feature should allow you to write comments
virtually anywhere. the (unit link) in $whit should become a (unit (list
link)) to accommodate this
Because the publisher will send the cork plea back to the subscriber on
the next bone, we are not able to know the bone for the original cork.
To handle it, we add the cork bone to the plea path
still wip: it keeps resending the cork plea faster than its ~h1 timer
