Most of the memory stays in gall anyway, and this means you need to
recompile everything the next time anything changes, which could be
counterproductive. It's important that %trim not make things worse.
The functionality is moved to the debug %stir task.
These were originally added because they reduced memory usage, primarily
by clearing the memoization cache. Now that the memoization cache is
no longer used, we use less memory without them. On ~wicdev-wisryt with
~30 apps, updating Clay now takes ~320MB.
(addendum to +team change)
address feedback from ~rovnys-ricfer, ~master-morzod,
~ritpub-sipsyl, ~tacryt-socryp, ~wicdev-wisryt, and others.
the original functionality of +team has been split out
between +team:title and +moon:title.
also:
fixes "middle core" and "surface core" comments in title
urbit-v1.13
Arvo 417K
Vere 1.13
This release makes the loom variably sized, up to a new maximum of 4GB.
The default remains 2GB -- opt for the max with `--loom 32`.
(NB: larger sizes may require adding swap space.)
Release Notes
- supports setting the loom size on startup. sizes are specified in
exponents of 2, from 1MB (20) to 4GB (32); can differ between processes
(`--loom` and `--urth-loom`); and can be set for relevant subcommands
(`meld`, `pack`, `next`, &c).
- improves bit-slice performance (for noun de/serialization and many atom jets)
by ensuring that implementation inner loops can be vectorized.
- adds the `vile` command, for exporting the keyfile from a ship
- adds the `%xray` hint, for printing the bytecode of a given expression
(ty ~topfet-parmed!)
- adds the `eval` command, for running hoon (from stdin) without booting a ship
(ty ~mopfel-winrux!)
- adds a better error message when a ship is already running as root
(ty ~dinleb-rambep!)
Contributions:
Joe Bryan (48):
u3: use size_t in u3m_init()
u3: normalize home-road stack after snapshot restoration
u3: refactors custom allocator init for libraries
bump
vere: make `prep` and `next` commands non-interactive (no tty required)
Merge pull request #5928 from urbit/jb/upgrade-no-tty
u3: remove references to u3a_bits/u3a_bytes/words/pages
u3: makes loom length configurable
u3: improves image-related error messages in events
Merge branch 'master' into next/vere
u3: suppress bail:evil printf
Merge pull request #5959 from urbit/jb/silence-evil
ur: optimizes bitstream bytes-writer implementation
ur: optimizes bitstream bytes-reader implementation
Revert "test: move _test_lily into noun_tests"
u3: rewrites u3r_chop(), optimizing bloq sizes < 5
u3: removes loom byte-length in favor of recalculation
Merge pull request #5957 from mopfel-winrux/eval
vere: style clean for eval cmd
Merge branch 'jb/hotfix' into jb/fix
Merge branch 'jb/hotfix' into jb/fix
Merge branch 'jb/hotfix' into jb/fix
Merge branch 'jb/hotfix' into jb/fix
Merge branch 'jb/hotfix' into jb/fix
Merge tag 'urbit-v1.12' into jb/fix
Merge commit 'f94875f9ce^2' into jb/release/vere
Merge commit '264e63330f^2' into jb/release/vere
u3: backports u3v_soft_peek()
vere: adds %vile command for extracting keys
Merge commit '330db284e8^2' into jb/release/vere
Merge branch 'master' into jb/release/vere
test: bump loom to 16MB in jam tests
u3: restages #6001 for release
u3: adds loom size to u3m_boot() / u3m_boot_lite()
vere: adds args to set mars/urth looms by binary exponent
Merge branch 'jb/restage/loom-set' into jb/release/vere
vere: supports --loom for all relevant subcommands
Merge branch 'jb/loom-everywhere' into jb/release/vere
Merge branch 'jb/release/vere' into jb/restage/chop-fast
u3: bumps maximum reference bits on all platforms
vere: preserve 2GB defaults for --loom and --loom-urth
Merge branch 'jb/restage/chop-fast' into jb/release/vere
Merge branch 'jb/restage/bump' into jb/release/vere
Merge branch 'jb/fix' into jb/release/vere
vere: bumps version
vere: bumps benchmarks to 16MB loom, initializes guard page
vere: supports --loom in eval cmd
vere: bumps version
DaneBettis (22):
Adds %meme and %xray hints.
adds xray test, reorders tests, fixes sanity checks
more notes and debugging for xray
more xray debugs via ~master-morzod
more xray debugs thanks to ~master-morzod
partial fix for indexed bytecode args; more to come
silenced the broken hoon tests
cleaner expression and docs
added macro to clarify _slog_bytecode and cleaned hoon hint tests
first cleanup from meeting with master-morzod
this branch PR is now for xray only. meme later.
minimize the diff
taming int declarations
docs, tammed ints, better var names, removed redundant funcs
fixes suggested by master-morzod, and finished int taming
make _is_indexed() more obvious and less cute
remove dev-time modifications to manage.c
WIP stuff re master-morzod
more refactoring as suggested by master-morzod
corrected private function names and block comments
fixed names of vars to follow cvc_TypeLetter style
simplified xray callsites and rendering
mopfel-winrux (15):
Added option to run the eval it checks for and prints whatever you pass it as an arguement
Added basic nock increment evaulation to eval function
Got basic hoon command compiling and executing
Fixed a double boot_lite
Added vase and stack trace wrapper to input, run hoo in virutalization
added ability to read from stdin rather than being passed an arguement
changed up how we are executing hoon. This fixes the issue we see with rpevious versions
cleaned up code
merged to master fixed conflict in motes.h
fixed a few of the issues ~master-morzod pointed out
changed error messages when booting the ivory pill
changed how the stdin is concatenated. this version should have less memory leaks
fixed memory leak and cleaned up style
removed unneeded checks
Merge branch 'urbit:master' into eval
pkova (1):
vere: check for permission error on old process kill
urbit-v1.13
Arvo 417K
Vere 1.13
This release makes the loom variably sized, up to a new maximum of 4GB.
The default remains 2GB -- opt for the max with `--loom 32`.
(NB: larger sizes may require adding swap space.)
Release Notes
- supports setting the loom size on startup. sizes are specified in
exponents of 2, from 1MB (20) to 4GB (32); can differ between processes
(`--loom` and `--urth-loom`); and can be set for relevant subcommands
(`meld`, `pack`, `next`, &c).
- improves bit-slice performance (for noun de/serialization and many atom jets)
by ensuring that implementation inner loops can be vectorized.
- adds the `vile` command, for exporting the keyfile from a ship
- adds the `%xray` hint, for printing the bytecode of a given expression
(ty ~topfet-parmed!)
- adds the `eval` command, for running hoon (from stdin) without booting a ship
(ty ~mopfel-winrux!)
- adds a better error message when a ship is already running as root
(ty ~dinleb-rambep!)
Contributions:
Joe Bryan (48):
u3: use size_t in u3m_init()
u3: normalize home-road stack after snapshot restoration
u3: refactors custom allocator init for libraries
bump
vere: make `prep` and `next` commands non-interactive (no tty required)
Merge pull request #5928 from urbit/jb/upgrade-no-tty
u3: remove references to u3a_bits/u3a_bytes/words/pages
u3: makes loom length configurable
u3: improves image-related error messages in events
Merge branch 'master' into next/vere
u3: suppress bail:evil printf
Merge pull request #5959 from urbit/jb/silence-evil
ur: optimizes bitstream bytes-writer implementation
ur: optimizes bitstream bytes-reader implementation
Revert "test: move _test_lily into noun_tests"
u3: rewrites u3r_chop(), optimizing bloq sizes < 5
u3: removes loom byte-length in favor of recalculation
Merge pull request #5957 from mopfel-winrux/eval
vere: style clean for eval cmd
Merge branch 'jb/hotfix' into jb/fix
Merge branch 'jb/hotfix' into jb/fix
Merge branch 'jb/hotfix' into jb/fix
Merge branch 'jb/hotfix' into jb/fix
Merge branch 'jb/hotfix' into jb/fix
Merge tag 'urbit-v1.12' into jb/fix
Merge commit 'f94875f9ce^2' into jb/release/vere
Merge commit '264e63330f^2' into jb/release/vere
u3: backports u3v_soft_peek()
vere: adds %vile command for extracting keys
Merge commit '330db284e8^2' into jb/release/vere
Merge branch 'master' into jb/release/vere
test: bump loom to 16MB in jam tests
u3: restages #6001 for release
u3: adds loom size to u3m_boot() / u3m_boot_lite()
vere: adds args to set mars/urth looms by binary exponent
Merge branch 'jb/restage/loom-set' into jb/release/vere
vere: supports --loom for all relevant subcommands
Merge branch 'jb/loom-everywhere' into jb/release/vere
Merge branch 'jb/release/vere' into jb/restage/chop-fast
u3: bumps maximum reference bits on all platforms
vere: preserve 2GB defaults for --loom and --loom-urth
Merge branch 'jb/restage/chop-fast' into jb/release/vere
Merge branch 'jb/restage/bump' into jb/release/vere
Merge branch 'jb/fix' into jb/release/vere
vere: bumps version
vere: bumps benchmarks to 16MB loom, initializes guard page
vere: supports --loom in eval cmd
vere: bumps version
DaneBettis (22):
Adds %meme and %xray hints.
adds xray test, reorders tests, fixes sanity checks
more notes and debugging for xray
more xray debugs via ~master-morzod
more xray debugs thanks to ~master-morzod
partial fix for indexed bytecode args; more to come
silenced the broken hoon tests
cleaner expression and docs
added macro to clarify _slog_bytecode and cleaned hoon hint tests
first cleanup from meeting with master-morzod
this branch PR is now for xray only. meme later.
minimize the diff
taming int declarations
docs, tammed ints, better var names, removed redundant funcs
fixes suggested by master-morzod, and finished int taming
make _is_indexed() more obvious and less cute
remove dev-time modifications to manage.c
WIP stuff re master-morzod
more refactoring as suggested by master-morzod
corrected private function names and block comments
fixed names of vars to follow cvc_TypeLetter style
simplified xray callsites and rendering
mopfel-winrux (15):
Added option to run the eval it checks for and prints whatever you pass it as an arguement
Added basic nock increment evaulation to eval function
Got basic hoon command compiling and executing
Fixed a double boot_lite
Added vase and stack trace wrapper to input, run hoo in virutalization
added ability to read from stdin rather than being passed an arguement
changed up how we are executing hoon. This fixes the issue we see with rpevious versions
cleaned up code
merged to master fixed conflict in motes.h
fixed a few of the issues ~master-morzod pointed out
changed error messages when booting the ivory pill
changed how the stdin is concatenated. this version should have less memory leaks
fixed memory leak and cleaned up style
removed unneeded checks
Merge branch 'urbit:master' into eval
pkova (1):
vere: check for permission error on old process kill
+find-item-in-type and everything it called was pretty bad spaghetti and
it wasn't clear that it was doing this right thing besides that it
passed tests. this refactors most of that functionality into a door that
has the type and search terms as the sample, and should be much easier
to follow.
the remaining functionality related to arm docs ought to be refactored
as well
By using `%kiln-install` instead of `%kiln-sync` for System Updates, this ensures that the `%kids` desk is also updated.
Also, address UX feedback: render the entire source ship's patp to avoid ambiguity. (as opposed to truncating a moon's name).
this makes it so that when an arm matches a search query in
+find-item-in-type, it checks to see if the arm builds a core with an
arm other than %$. if so, a %core item is returned rather than an %arm
item.
the name of the core will then be considered to be the name of the arm
that builds it
since cores can't be given names in v0 doccords, this changes the main
search function in lib/dprint to check if the summary of a core
description matches the search term, along with other appropriate
changes downstream of this.
still to be done: use .name in a %core item for either this summary name
or the name of the arm that built the core, if appropriate
this is to handle potential future cases where doccords might be kinds
of notes other than %help notes. example: #6085 to document invariants
in clay.
$whit (used for apex:docs/batch comments) also ought to be changed but
im still thinking about what that should look like.
The faliing test was expecting ames to give a %done to %gall
but https://github.com/urbit/urbit/pull/5977, changed that,
so %ames doesnt give %done for %cork acks.
partial revert of 3d3ea61d53, which introduced core names by completing
an unimplemented feature that was already present in hoon.hoon. we've
decided to remove this for the initial launch since it violates the
principle of least surprise for the name of a core to end up in its
$garb and yet only be used for doccords, as opposed to something like a
wing resolution. it was also confusing that this only worked for |% and
|@.
this breaks two of the tests for the dprint library, which have been
commented out. these tests ought to be restored once dprint is rewritten
in order to implement a different way to refer to cores not built by arms
this constitutes a pretty major rework of how whitespace is handled in
hoon in order to change the doccords syntax from :> and :< to ::.
in summary: throughout the hoon parser (+vast) many instances of +gap
have been replaced by +jump, which first tries to remove whitespace (+leap)
until it arrives at something that can be parsed as a prefix
doccord (+apex:docs:vast). if it does not encounter a doccord, it
instead uses +gap as normal.
if you follow along with the parser, you will notice that every time
jump is called, it then tries to call +apex:docs via +scye or +seam. if
apex:docs succeeds, it will end up consuming a newline at the end,
hiding the fact that there was valid whitespace from the parser. thus
+apex:docs then inserts a newline after successfully parsing a prefix
doccord, which will then be consumed by a subsequent invocation of +gap,
ensuring that there was proper whitespace if the doccord would have been
consumed by +gap instead of +leap.
there are a few other changes:
+hint in the compiler throws out doccords attached to %noun types. this
was already the behavior before doccords, and the change was made before
i understood what i was doing.
similarly for commenting out the %note case in +open:ap - this was an
earlier mistake
postfix comments for chapters are now enabled.
+expx was unused and removed in order to be rid of the
convention-defying +exp1. other unused +ex(p/q)* were commented out.
arms that handle batch comments (+glow and +whap) were refactored
+toad, which was used to change between tall and wide form, tries to
+jump before +gap. since +jump is ;~(pose leap:docs gap), i would have
thought that just using +jump there would be fine, but it fails for some
reason.
some parsers built with |@ were rewritten to use |*
|$ was made so that any doccords put on the spec are converted into hoon
doccords on the $ arm. it wouldn't compile otherwise. there's probably a
more principled way to do this but it works fine for now.
urbit-v1.12
Arvo 417K
Vere 1.12
This is a hotfix release, fixing another memory corruption vulnerability
introduced in v1.10 and improving snapshot durability.
Release Notes
- restores the guard page (preventing road stack overflow) after |meld
- improves snapshot error handling and messaging
- fixes an overflow bug affecting jammed scry output (-X)
Contributions:
Joe Bryan (16):
u3: protect guard page in u3e_yolo()
vere: bumps version
u3: assert guard page invariants when saving snapshot
u3: initialize guard page in u3m_boot_lite()
vere: bumps version
nix: update linux-aarch64 overlay to exclude macos m1
u3: handle partial reads in snapshot system
u3: handle partial writes in snapshot system
u3: print mprotect errors in snapshot system
u3: print error msg if system page size is incompatible
u3: detect snapshots from a larger loom, print and exit
u3: normalize home-road stack after snapshot restoration
u3: handle and print lseek errors in snapshot system
u3: print errors while deleting snapshot patch
vere: bumps version
vere: bumps version
Liam Fitzgerald (1):
pier: fix jamming archive
urbit-v1.11
Arvo 417K (zuse+417, lull+328)
Vere 1.11
This is a hotfix release, fixing a memory corruption vulnerability
introduced in v1.10, and including official aarch64 binaries.
Release Notes
- repositions the guard page (preventing road stack overflow) on inner roads
- decrements %lull and %zuse kelvins, in preparation for a pending arvo release
- adds support for static aarch64 binaries
- cleans up the terminal on exit when booting a fake ship
Contributions:
Bradley (2):
vere: changed to use u3_king_bail instead of manual term clean up
vere: removed unnecessary exit(1) call
Bradley Ray (1):
vere: clean up terminal on invalid fake ship name
Joe Bryan (11):
u3: reposition guard page if needed for new roads
test: initialize guard page in jam-tests
vere: decrement %lull and %zuse kelvins
Merge commit 'ac5842fd6e' into jb/hotfix
vere: bumps version
build: add aarch64 via buildjet
build: run unit tests early on linux
build: run unit tests on windows
build: updated gcp configuration for buildjet runner
build: renames "nightly" pace from "often" to "edge"
vere: bumps version
~botter-nidnul (1):
nix: enable aarch64-linux platform string
Also, remove the conditional rendering logic for AppPrefs. With the new logic (and before this change) if the User disables OTAs, the toggle would disappear, which feels like an antipattern.
- fix `fragment-num` and `num-fragments` having duplicate faces
- fix faces being wrapped around wrong things in various places
- fix `bone` not being printed in "hear last in-progess" message
- make pretty tape interpolation style more uniform
in the past, +team meant "our / our moon", but
it has been primarly used to represent "our"
moons as having the full permissions of their
parents doesn't make a lot of sense anymore
this looks like the more elegant solution
instead of changing each instance of +team
I've combed through the uses of +team throughout
urbit/urbit and I'm quite sure that each instance
is better off as just "our"
The +on-cork handler asserts that the peer is known to us. This is the
incorrect behaviour, because it will crash when corking a flow to a peer
that is still an %alien. This can happen, for instance, when making a
gall subscription for the first time and then corking it before the
alien naturalises.
`+story-list` produced janky indentation because the `$-(story
wain)` functions encoded linefeeds in the cords of the wain and the
printer doesn't like this.
Story printing functions have been changed to produce pure wains without
linefeeds.
Any change to the session object was triggering this. But that now includes
an "unacked keystrokes" counter, which updates frequently, and we
definitely don't want to send resize notifications in that case.
Instead of forcing people to connect over http://, fall back gracefully to http if no protocol is given.
This fixes an issue where external clients can't use this method, since SameSite cookies need to be secure as per https://web.dev/samesite-cookies-explained/#samesite=none-must-be-secure
PR #5840 mostly fixed#1559, but introduced a new bug. before, you could safely `=dir` into a desk without a case, and it would use the nonexistent case `ud+0` as the beam for dojo state, and switch that out for da+now whenever it tries to resolve the current path. but this check causes it to fail, because `ud+0` is a nonexistent case. this uses he-beam to transform the beam in the conditional to see if the case is 0, and if it is, changes the case to da+now before it scries
if a cert is configured and a secure port is live it will set the
redirect flag in http-config.state.
When it gets a ++request it will return a 301 redirect to
https://[host]/[path] if:
1. not already secure
2. redirect flag set
3. secure port live
4. is not requesting /.well-known/acme-challenge/...
5. the host is in domains.state
It will not happen if forwarded-secured, localhost, local loopback, ip
addresses or domains not in domains.state.
in ++load it checks the secure port is live and a cert is set and
enables it if so (for people who already use in-urbit letencrypt)
%rule %cert tasks also toggle it (only turning it on if secure port
live)
%live tasks also toggle it (only turning it on if cert set)
Have tested with a couple of ships and seems to work fine.
This is useful in combination with pyry's auto arvo.network dns config
system - can finally get rid of reverse proxies entirely.
Eyre always gets passed request headers in lowercase, so we should search for
the lowercased version of the header.
Arguably `+get-header` should lowercase keys before comparing them, but that's
a more serious behavioral change.
This allows you to pass a thread directly into khan, instead of passing
a filename. This has several implications:
- The friction for using threads from an app is significantly lower.
Consider:
=/ shed
=/ m (strand ,vase)
;< ~ bind:m (poke:strandio [our %hood] %helm-hi !>('hi'))
;< ~ bind:m (poke:strandio [our %hood] %helm-hi !>('there'))
(pure:m !>('product'))
[%pass /wire %arvo %k %lard %base shed]
- These threads close over their subject, so you don't need to parse
arguments out from a vase -- you can just refer to them. The produced
value must still be a vase.
++ hi-ship
|= [=ship msg1=@t msg2=@t]
=/ shed
=/ m (strand ,vase)
;< ~ bind:m (poke:strandio [ship %hood] %helm-hi !>(msg1))
;< ~ bind:m (poke:strandio [ship %hood] %helm-hi !>(msg2))
(pure:m !>('product'))
[%pass /wire %arvo %k %lard %base shed]
- Inline threads can be added to the dojo, though this PR does not add
any sugar for this.
=strandio -build-file %/lib/strandio/hoon
=sh |= message=@t
=/ m (strand:rand ,vase)
;< ~ bind:m (poke:strandio [our %hood] %helm-hi !>('hi'))
;< ~ bind:m (poke:strandio [our %hood] %helm-hi !>(message))
(pure:m !>('product'))
|pass [%k %lard %base (sh 'the message')]
Implementation notes:
- Review the commits separately: the first is small and implements the
real feature. The second moves the strand types into lull so khan can
refer to them.
- In lull, I wanted to put +rand inside +khan, but this fails to that
issue that puts the compiler in a loop. +rand depends on +gall, which
depends on +sign-arvo, which depends on +khan. If +rand is in +khan,
this spins the compiler. The usual solution is to either move
everything into the same battery (very ugly here) or break the
recursion (which we do here).
Also adds a dedicated %born test to check if it sends a %sessions gift,
and refactors %init and %born running into separate arms to cut down on
repetition.
Accept the newly added %session effect. Store the tokens received in it,
and refer back to them when checking requests for authentication.
Adds logic for checking requests for authentication based on the
presence of a valid urbauth cookie for the host ship, and updated the
slogstream endpoint to use that instead of the scry-based authentication
check.
Whenever a session gets created or removed, send the set of valid auth
tokens to the runtime, so that it may use them in determining whether
incoming requests are authenticated or not.
Before this, the %watch to eth-watcher was happening before the %poke,
and so eth-watcher was responding with its entire history immediately.
This is bad because it takes a lot of memory to process that many logs,
and also because those logs are stale.
Now, the %poke happens first, which clears the history.
%kick is supposed to start back from the snapshot and move forward.
Without this, we would only fetch logs that we hadn't already fetched.
Thus, if you were up-to-date when you kicked, you would miss anything
that happened between the time the snapshot was taken and the present,
though you would see things after the present.
Also reverted lull change to make this a safer upgrade.
