Commit Graph

1223 Commits

Author SHA1 Message Date
fang
dd4a115e51
clay: avoid %v scries, cancel scry timer correctly
%v requests are already handled specially for the ames case. We should
continue to respect that, only doing those kinds of requests over ames.

Also cancels the scry timer after receiving a response, instead of not
doing that.
2022-02-17 15:40:44 +01:00
Liam Fitzgerald
6bd6029f57 ames: use finger trees for packet storage 2022-02-16 15:35:59 -06:00
Jōshin
470ae80d4b
khan: cleanup 2022-02-15 22:32:52 -08:00
fang
fb4b3c7196
clay: update straggler state version number 2022-02-15 22:35:53 +01:00
Liam Fitzgerald
a36dc3aaff ames: per-lane congestion control 2022-02-15 08:47:38 -06:00
fang
0375e0a53f
clay: remove dist-upgraded flag
Ancient version upgraders beware! If you're coming from a home-based
arvo, you must first upgrade to the version prior to this, or else
you'll be in trouble!
2022-02-15 01:02:19 +01:00
fang
c02274dfdd
clay: sync files using remote scry
When clay wants to download blobs from a foreign ship, it attempts doing
this using the new remote scry protocol.

If it doesn't receive a response within ~m1, it falls back to using the
old ames-based syncing instead. We remember this "prefer ames" state for
the specific ship for an hour, after which we'll begin trying the scry
flow again.

Compiles, but untested. Some TODOs and REVIEWs remaining herein.

We intentionally leave the dist-upgraded flag in state to avoid
cluttering the diff here. The next commit will remove it.
2022-02-15 00:51:54 +01:00
Jōshin
6aa72069e3
arvo: introduces $crag
Also strips out `$` from khan top-level comment.

There are arguments for keeping $crag in lull, and on the other side for
moving $cast to arvo. This seemed like the most reasonable approach.
2022-02-14 12:46:31 -06:00
Ted Blackman
b2051fcc2b fine: WIP second list for performance 2022-02-14 10:34:26 -06:00
Jōshin
6d275fe9f3
khan: move =/ down 2022-02-14 09:09:55 -06:00
Jōshin
1d487a758b
test: full run, happy path 2022-02-13 07:33:03 -08:00
Jōshin
d7608019f2
khan: cleanup, namespace 2022-02-11 22:18:15 -08:00
Jōshin
c0423adea6
khan: parameterized types, nested implementation
%fyrd is now implemented in terms of %fard, and likewise %avow in terms
of %arow. State is tracked via wire rather than in a global map.

Unit tests adjusted to match.
2022-02-11 21:58:09 -08:00
Liam Fitzgerald
e97d99275b ames: congestion control for remote scry 2022-02-11 19:30:40 -06:00
Jōshin
5ff5082a5d
khan: docs, lift arg vase into (unit) 2022-02-08 12:03:00 -06:00
Jōshin
8123255a62
khan: add %fard/%arow for in-arvo threads
These take and produce vases, and assign random tids (rather than
deducing them from the input duct.)

Since %fard does not require mark conversion, we make the mark/beak on
$thread-state optional (and use this to decide whether to send %avow or
%arow.) Provide a state adapter since it's possible that people have
been experimenting with this vane.

This makes the negative case of %avow/%arow kind of clunky, since there
is no content difference, but the following does not seem possible
within the Hoon type system:

    =/  gif
      ?~  p.tad
      %arow  %avow
    [hen %give gif %| p.cag tang]~
2022-02-07 17:51:05 -06:00
Yaseen
ce0fa85e18 story: generators improvements
- use desk parameter instead of %base everywhere
- formatting clean up
- make |story-remove take a case instead of an aeon
- make desk param optional for story-set and story-log
2022-02-05 22:50:10 -05:00
fang
a5b16b1673
fine: consolidate duplicate logic
Further merge cleanup.
2022-02-04 13:36:32 +01:00
fang
47feec3ecc
Merge branch 'lf/ames-notifs' into m/scry-vane 2022-02-03 22:00:24 +01:00
fang
a8574df401
ames: retrieve pk from old crypto core on load
Since we cannot scry the key out of jael during load, and the
alternative is defering this logic through the larval core in some way,
we simply include the type for the old acru interface, letting us reuse
the old core to initialize the new one.
2022-02-03 13:49:38 +01:00
fang
39959942d2
ames: use +safe:as:crub instead of +veri:ed
+safe now exists to wrap that, so we should use it instead of deriving
the relevant key and calling +veri:ed directly.
2022-02-02 22:16:16 +01:00
fang
5776226a6c
lull: add acru arms for simply signing & authing
Instead of exporting keys so that caller can do this themselves, we
expose arms for signing and authenticating that produce and operate on
just the signature, without mangling it into the message.
2022-02-02 21:52:48 +01:00
fang
83e315cccd
lull: add arms to ex:acru for separate keys
We may want _just_ the encryption key, or _just_ the signing key.
2022-02-02 20:10:30 +01:00
fang
1302c5fa0b
fine: fix broken +full:keys callsite
Straggler from last commit about data type consistency.
2022-02-02 19:55:20 +01:00
Liam Fitzgerald
9dad96332b ames: remove logging 2022-02-01 20:50:02 -06:00
Liam Fitzgerald
6ee2e1626f ames: unify path format 2022-02-01 20:48:04 -06:00
Liam Fitzgerald
1eaad36d4d ames: working notifications 2022-02-01 20:30:11 -06:00
Liam Fitzgerald
7205625abf wip 2022-02-01 13:25:28 -06:00
fang
c72c187edd
fine: use $@(~ (cask)) consistently
It's what ends up in the packet, probably better than passing units and
converting them all over.
2022-02-01 12:02:47 +01:00
fang
4bea69bc7d
fine: verify life & signatures of packets & msgs
In addition to doing signing & verification consistently with each
other, we now handle the "no data" case sanely.

Still uses the wrong keys though.
2022-02-01 11:55:04 +01:00
Yaseen
b7e0b28220
Merge branch 'urbit:master' into story 2022-01-30 16:03:48 -05:00
fang
b42dc477c5
fine: move into per-event core
This is an obviously more ergonomic factoring.
2022-01-28 23:54:01 +01:00
fang
fb72b7df7d
fine: set receiver life in request packet 2022-01-28 23:11:27 +01:00
fang
709dd75d74
fine: track pending requests for unknown peers 2022-01-28 23:09:50 +01:00
fang
769d10fdd4
fine: use keys from ames state instead of jael 2022-01-28 14:30:59 +01:00
Jōshin
21ea32e8ad
khan: +bear allows a full +beak 2022-01-27 22:28:30 -08:00
Jōshin
72981203c8
khan: PR feedback, wip tests 2022-01-27 22:20:31 -08:00
fang
4c045a9c4f
ames, fine: minimal compilable unification 2022-01-27 14:32:49 +01:00
Jōshin
05a11d57b3
khan: cleanup
- only store metadata in the persistent map. just enough to support
  (eventual) thread cancellation and output mark lookup.

- try to delete thread state at other failure points not covered by
  %kick.

- reflect back the passed output mark rather than form.dais. not sure
  about this one yet.
2022-01-26 20:09:36 -08:00
Jōshin
aba5c0303d
khan: del in %kick 2022-01-26 17:45:25 -08:00
Jōshin
1281e1cdbe
khan: functionally done 2022-01-26 17:41:01 -08:00
Jōshin
425bf6b011
khan: accepts thread arguments with marks 2022-01-26 16:26:30 -08:00
fang
d4a0f1d901
fine: correctly calculate request body size
This had missed getting updated for byte-logic.
2022-01-26 20:03:03 +01:00
Jōshin
8ad57d8bd2
khan: cleanup 2022-01-25 21:28:45 -08:00
Jōshin
12b4c80cf1
khan: wip threads v0
super ghetto first pass with a bunch of stuff hard-coded to prevent
sending giant vases over the wire.
2022-01-25 20:08:48 -08:00
Jōshin
fd7d315e3b
khan: wip fyrd 2022-01-25 15:37:48 -08:00
fang
af8fd67472
fine: scry using the roof, not .^ 2022-01-25 21:27:52 +01:00
fang
793c31d7f9
fine: properly sign individual packets 2022-01-24 23:51:22 +01:00
fang
ff53ae5e66
fine: supply lane where needed
Still somewhat wip, this might not be the lane behavior we want for
real.
2022-01-24 23:26:02 +01:00
fang
03b73b85bf
fine: set and handle the response bit correctly 2022-01-24 23:06:06 +01:00
fang
2279764eff
fine: include header & prelude with %hoot 2022-01-24 21:01:46 +01:00
fang
b606dea970
fine: work on bytes instead of bits 2022-01-24 21:01:09 +01:00
Jōshin
30df250c7d
khan: closer to starting threads 2022-01-22 01:46:59 +00:00
Jōshin
3cf803129d
khan: basic +call, types 2022-01-21 22:09:01 +00:00
Jōshin
575ff61fa6
khan: vane skeleton 2022-01-21 18:48:05 +00:00
fang
92aa604be0
fine: factor out the fetching of scry results
And use it to implement task-ways request handling, which we'll probably
remove in favor of peek-ways handling.
2022-01-21 14:59:54 +01:00
fang
2d0a6f160b
fine: reuse packet en/decoding logic from ames
Also adds some more of the core logic, but still wip.
2022-01-20 20:36:38 +01:00
fang
dd4eb87f42
ames: move packet en/decoding into lull
By moving this behavior for packets with ames-style headers into lull,
other vanes that want to do protocols in similar style (like the upcoming
%fine vane) can reuse the logic here.

Note that we parameterize the main en- and decoding functions, so that
we may pass in the "is ames" flag and protocol versions as appropriate
to the context.
2022-01-20 01:59:43 +01:00
fang
207dc91c8c
fine: wip, add packet deserializing logic
Also stub our some flows, add todos, etc. Bit of a mess, follow-up soon.
2022-01-19 12:35:54 +01:00
yosoyubik
3d943ec201 ames: ignore missing peer-state on-publ-sponsor 2022-01-18 20:34:32 +01:00
fang
fa9c4016aa
fine: work around compiler bug 2022-01-14 13:51:02 +01:00
fang
355f05753f
fine: remote scry
Implements a WIP version of %fine, the remote scry vane. Plenty of work
remains to be done here, marked by TODOs and REVIEWs.
2022-01-14 00:22:56 +01:00
Yaseen
b37fd09fab clay: implement %base-tako scry under the %s care 2022-01-13 15:56:20 -05:00
yosoyubik
432d9674f5 jael: don't put conflicting this-su in the subject 2022-01-12 11:01:06 +01:00
yosoyubik
fccf99e0a6 jael: refactor new-event for %keys diff
this also adds a %rerun %poke to /app/azimuth
2022-01-08 19:28:11 +01:00
yosoyubik
8535563de6 jael: give private key to subscribers on keys diff
Triggering this event has been added to /app/azimuth's +on-load
2022-01-08 09:48:28 +01:00
Yaseen
163ed030d4 clay: add new %s scry which gives the tako for a given beak 2021-11-25 12:48:24 -05:00
Philip Monk
a2a89d3532
Merge remote-tracking branch 'origin/next/arvo' into philip/roller 2021-11-16 13:58:57 -08:00
Philip Monk
4656372b09
Merge pull request #5426 from ynx0/claybel
clay: support labels
2021-11-15 08:49:31 -08:00
Yaseen
c080eb8336 fix comment styling 2021-11-15 09:07:24 -05:00
Yaseen
ae7cf81c29
clay: keep info in stacktrace when labeling fails
Co-authored-by: Philip Monk <phil@pcmonk.me>
2021-11-15 05:32:59 -05:00
Philip Monk
8af1dd3acc
clay: refactor +wake
+wake had accumulated several layers of abstractions which were later
rendered unnecessary.  This removes those abstractions and should have
no semantic effect.
2021-11-14 19:35:32 -08:00
Yaseen
70a85ccdb0 remove unnecessary comments 2021-11-14 20:41:30 -05:00
Yaseen
17751e7890 clay: enforce referential transparency on labels 2021-11-14 16:49:17 -05:00
Yaseen
25ce97cade clay: add support for labeling commits
- add helper arm `label:de`
- update relevant code to take in arbitrary aeons
2021-11-12 15:35:06 -05:00
Philip Monk
08ab77893e
azimuth: don't start on fake ships 2021-11-09 22:36:05 -08:00
Philip Monk
e4a069a4f6
jael: remove vestigial boq and nod 2021-11-09 16:41:23 -08:00
Philip Monk
bf7f4ba453
Merge remote-tracking branch 'origin/m/naive-ugprade-path' into philip/roller 2021-11-09 12:52:25 -08:00
Philip Monk
45c1693085
jael: use default network id 2021-11-08 22:59:06 -08:00
Philip Monk
91b38d4cea
ivory: update preboot validation format
string<->numbers didn't match the json serialization.  This makes
preboot validation succeed.
2021-11-08 20:34:32 -08:00
Philip Monk
d9276cd51c
clay: Add support for representing tombstones
This adds support for tombstoned files to clay.  It does not include any
way to actually tombstone them; that is left for later.

This allows tombstoning at the level of a file.  Precisely, this expands
+blob:clay by adding a %dead case:

    +$  blob                                              ::  fs blob
      $%  [%delta p=lobe q=[p=mark q=lobe] r=page]        ::  delta on q
          [%direct p=lobe q=page]                         ::  immediate
          [%dead p=lobe ~]                                ::  tombstone
      ==                                                  ::

Thus, we maintain the invariant that every lobe corresponds to a blob,
but now a blob may be an explicit tombstone.

Details:
- This has not been tested at all, except that it compiles and boots.
- This does not have a state adapter from master.  The only state change
  is the definition of +cach.
- Additionally, out-of-date ships may unexpectedly receive a %dead blob
  from a foreign clay which would interfere with their ability to download
  that desk.  No code changes necessary, but sponsors should avoid
  tombstoning files in %base for a while so their children can get the
  update.
- A merge will only fail if the tombstoned file conflicts with another
  change.  Note that as written, merging from a past desk *can* bring a
  tombstoned file to the head of a desk.  Possibly this shouldn't be
  allowed.

This also includes a couple refactors that were made possible by ford
fusion (since everything is synchronous now) but never got done.  In
both cases we get to remove a monad, which simplifies the code
considerably.

- refactor +merge's error handling to use !!/mule instead of threading
  through errors
- refactor all +read-* functions and related parts of +try-fill-sub to
  eagerly convert lobes to cages.

We also add support reading %a/b/c/e/f/r/x from past and foreign desks,
when possible.  Apologies that all of these are in one commit, it was
all a single chunk of work.

This is a draft until we have a way to tombstone.  I suspect we'll want
to have a mechanism of keeping track of gc roots and trace to remove,
but this PR doesn't suggest any particular strategy.
2021-11-07 22:17:02 -08:00
fang
fecb809573
jael, azimuth: reconfigure jael for L2
Jael needs to be reconfigured to listen to the new aagent for azimuth
events, and the old app needs to be shut down. We do this in
/app/azimuth's +on-init.

Additionally, we make sure that jael doesn't crash when it (as expected)
loses its subscription to the old agent.
2021-11-06 01:10:33 +01:00
fang
0ec3d5111d
Merge branch 'master' into naive/roller 2021-10-26 20:46:32 +02:00
fang
36cd69b6d0
Merge tag 'urbit-os-v2.100' into naive/roller 2021-10-26 17:08:03 +02:00
fang
f91c4dec08
Merge branch 'master' into release/next-sys 2021-10-08 17:39:03 +04:00
fang
bf4aef198c
clay: give %u answer for non-existent desks
Instead of considering it "in the future", an existence check for paths
on non-existent desks now always produce false.
2021-10-07 21:57:56 +04:00
fang
2a1c47ddd2
jael: add %ruin task for pretend-breaching others
The initial use case here is clearing state relating to long-gone
comets, but this could prove to be a useful general-purpose rescue tool.
2021-10-01 01:54:33 +04:00
Philip Monk
facdf4dbc1
jael: on rekey, update life if moon 2021-09-29 20:51:57 -05:00
Ted Blackman
5cce66523c Revert "clay: print cult"
This reverts commit d4bc38f774.
2021-09-29 20:31:57 -04:00
Ted Blackman
86a2715e67 clay: purge %next and %sing on migration 2021-09-29 20:10:53 -04:00
Ted Blackman
d4bc38f774 clay: print cult 2021-09-29 20:01:43 -04:00
Ted Blackman
5e1778ea84 clay: compile failure 2021-09-29 19:43:26 -04:00
Ted Blackman
2effd3cec0 Merge branch 'dist' of github.com:urbit/urbit into dist 2021-09-29 19:37:18 -04:00
Ted Blackman
b16d176863 clay: delete /app/publish from subscription state on migration 2021-09-29 19:36:29 -04:00
Philip Monk
3c658b17fa
Revert "gall: don't crash on suspended agent build failures"
This reverts commit d446418d53.
2021-09-29 18:30:01 -05:00
Ted Blackman
d446418d53 gall: don't crash on suspended agent build failures 2021-09-29 18:49:20 -04:00
Ted Blackman
bd46a367a3 clay: always notify if new read says nonexistent 2021-09-29 17:57:55 -04:00
Ted Blackman
7e5feb4cb4 gall: implement pupal +stay 2021-09-29 16:24:12 -04:00
Ted Blackman
fd54c96a3f gall: %jolt restart agent from new desk 2021-09-29 16:03:14 -04:00
Ted Blackman
9560fa43b2 gall,clay: fix migration file build dates; add %home as parent desk in jams 2021-09-29 15:00:42 -04:00
Fang
f6bb541269
gall: also watch changes to /sys/kelvin 2021-09-29 22:13:39 +04:00
Fang
27fcc9b4c3
various: %home -> %base 2021-09-29 22:13:08 +04:00
Fang
526e60b04c
strandio: fail when http request gets cancelled
Instead of waiting for a proper response. Iris doesn't currently retry!

Also updates docket to detect this case and retry appropriately.
2021-09-28 20:51:51 +04:00
Fang
1867209a12
Merge branch 'jm/spider-gall' into dist 2021-09-28 17:12:17 +04:00
Fang
059c2daed3
clay: make distribution desks public on-migrate
To ensure our sponsees (and other ships) can install from them.
2021-09-28 15:59:36 +04:00
ryjm
472f16fd2e garden spider: address comments 2021-09-27 21:23:46 -04:00
ryjm
47a6aa2753 spider garden: remove sigpams and move glob mark 2021-09-27 21:23:46 -04:00
ryjm
c630b560ee gall: don't verify spider marks 2021-09-27 21:23:46 -04:00
Fang
18bad9c423
gall: give correct %gu results
Give false for suspended agents, instead of always giving true if the
agent exists.
2021-09-28 00:22:52 +04:00
Ted Blackman
9aa8566a6e -jam-all-desks 2021-09-26 22:03:13 -04:00
Philip Monk
a94e17915f clay: fix failure to load apps
When you loaded an app with an error, then fixed the error, it would
create the main gall %mult subscription at a time in the past.  Then,
clay would never fill the subscription since it couldn't get the old %a
entries for the apps.

This fixes the issue in two ways:  first, don't subscribe in the past.
Second, if clay can't get the old versions, just fire the subscription
anyway.
2021-09-26 00:26:37 -04:00
Ted Blackman
d81a4a7cba dill,kiln: fixes for boot and same-kelvin OTAs 2021-09-26 00:20:37 -04:00
Ted Blackman
8f115bf059 clay: remote-install desks during migration 2021-09-25 16:53:52 -04:00
Ted Blackman
bbc80f9d45 zuse,ames: ^^sein:title 2021-09-25 16:52:53 -04:00
ryjm
154deb3281 clay: print stacktrace on build failure 2021-09-25 15:36:00 -04:00
Fang
4ed2a5914b
eyre: bind most specific endpoint first
This sticks to the old behavior, which was briefly lost in recent
changes.
2021-09-25 17:44:31 +04:00
Fang
369a6b7a47
eyre: allow overwriting existing bindings
Previously, if trying to bind to an endpoint that was already bound to,
eyre would reject it. This doesn't play very nicely in a softdist world
where uninstalled apps might not get a chance to clean up, and apps
might re-bind simply for being re-installed.

Here we change eyre to overwrite an existing binding if it conflicts
with the new one to be added.
2021-09-25 00:54:24 +04:00
Fang
ce09681b0a
clay: avoid migrating during boot
We set the "have we migrated yet" flag to true during boot, because
we're starting out in a post-migration world.

Also fixes a state version number.
2021-09-24 20:55:28 +04:00
Ted Blackman
c80fb82318 clay,kiln,landscape/notify: migration runs 2021-09-23 18:07:13 -04:00
Ted Blackman
16abd02a66 gall: set %hood to %base during dist upgrade 2021-09-22 23:11:41 -04:00
Fang
6812d15480
gall, kiln: better migration logic 2021-09-22 22:34:20 +02:00
Fang
eb1ec5b749
clay: clean up migration error case printfs 2021-09-22 15:45:04 +02:00
Fang
904165f83a
clay: don't forget the extra q
+load still nest-fails though.
2021-09-22 01:39:29 +02:00
Fang
f588ea9feb
clay: grab tmp desk jamfile correctly 2021-09-22 01:16:21 +02:00
Ted Blackman
30364ab29b clay: wip migration 2021-09-21 19:13:54 -04:00
Fang
c0bfab9bf5
clay: wip migrate from desk jams
To work around mark availability issues.
2021-09-21 00:28:31 +02:00
Fang
e9aa3821ef
zuse, clay: further wip migration 2021-09-20 23:13:04 +02:00
Ted Blackman
fb8c73e39a zuse,clay: WIP +migrate-dist 2021-09-20 13:01:48 -04:00
Ted Blackman
04e010e6c8 Merge branch 'dist' into ted/dist-migrate-wip 2021-09-20 07:43:43 -04:00
Liam Fitzgerald
1ac0465da7 gall: ignore reload for dead agents
Fixes a bug where even if an agent was dead, if its source was modified,
then gall would still attempt a scry for its cage.
2021-09-13 08:43:57 +10:00
Ted Blackman
84c043d921 lull,clay: WIP migration 2021-08-31 02:25:36 +03:00
Ted Blackman
91714f5579 lull,dill,gall,kiln: move %pyre to :agent:gall 2021-08-27 18:06:53 +03:00
Ted Blackman
dda7f13e0e clay: WIP migrate from master 2021-08-26 19:32:52 +03:00
Ted Blackman
f9e1812a14 sys,gen/kiln: kelvin checking seems to work 2021-08-20 10:41:16 +03:00
Ted Blackman
4ea0c56849 dill: don't sync %kids on galaxies 2021-08-18 00:57:38 +03:00
Ted Blackman
616ab66f26 merge master; loosen :hood's updates 2021-08-11 19:23:36 +03:00
Ted Blackman
b8af4835cf dill: sync to %kids on galaxies 2021-08-09 02:31:53 +03:00
Philip Monk
eb383baba9
dill: no-op on strange fact
Otherwise, if you accidentally give on the system duct, you crash the
event, as in #5138
2021-07-29 14:56:26 -04:00
Ted Blackman
a27697a45d dill: revert change to ota setup 2021-07-23 14:14:23 +03:00
Ted Blackman
6bf7d3316e dill: |install %base from sponsor on boot 2021-07-23 14:08:30 +03:00
Ted Blackman
d9c9740b8d clay: print desk in +read-a 2021-07-23 14:03:36 +03:00
Ted Blackman
891a295366 gall: fix /clear-huck handling 2021-07-21 11:55:21 +03:00
Philip Monk
1d06b77a46
jael, dawn: support multikeyfile
taken from bdaad4cf84
2021-07-20 13:28:33 -04:00
Philip Monk
564c1216e1
Merge remote-tracking branch 'origin/lf/replace-channel' into release/next-sys 2021-07-20 13:09:41 -04:00
Philip Monk
5240b66347
Merge branch 'pr' into release/next-sys 2021-07-20 13:07:57 -04:00
Ted Blackman
88c15bcca2 gall: make %jolt idempotent, remove dead +mo-reboot 2021-07-20 17:59:20 +03:00
Ted Blackman
38d6d0f4d4 gall,kiln: clean up agent stopping, starting, linking 2021-07-19 16:58:18 +03:00
Ted Blackman
4f5739879e clay: more printing adjustment 2021-07-19 14:01:34 +03:00
Ted Blackman
f521881212 clay: better print 2021-07-18 23:11:13 +03:00
Ted Blackman
4f3e99f797 clay,hood: seems to boot for real now 2021-07-17 02:54:20 +03:00