Commit Graph

206 Commits

Author SHA1 Message Date
Philip Monk
e3a8cab943 khan: support inline threads
This allows you to pass a thread directly into khan, instead of passing
a filename.  This has several implications:

- The friction for using threads from an app is significantly lower.
  Consider:

    =/  shed
      =/  m  (strand ,vase)
      ;<  ~  bind:m  (poke:strandio [our %hood] %helm-hi !>('hi'))
      ;<  ~  bind:m  (poke:strandio [our %hood] %helm-hi !>('there'))
      (pure:m !>('product'))
    [%pass /wire %arvo %k %lard %base shed]

- These threads close over their subject, so you don't need to parse
  arguments out from a vase -- you can just refer to them.  The produced
  value must still be a vase.

    ++  hi-ship
      |=  [=ship msg1=@t msg2=@t]
      =/  shed
        =/  m  (strand ,vase)
        ;<  ~  bind:m  (poke:strandio [ship %hood] %helm-hi !>(msg1))
        ;<  ~  bind:m  (poke:strandio [ship %hood] %helm-hi !>(msg2))
        (pure:m !>('product'))
      [%pass /wire %arvo %k %lard %base shed]

- Inline threads can be added to the dojo, though this PR does not add
  any sugar for this.

    =strandio -build-file %/lib/strandio/hoon
    =sh |=  message=@t
        =/  m  (strand:rand ,vase)
        ;<  ~  bind:m  (poke:strandio [our %hood] %helm-hi !>('hi'))
        ;<  ~  bind:m  (poke:strandio [our %hood] %helm-hi !>(message))
        (pure:m !>('product'))
    |pass [%k %lard %base (sh 'the message')]

Implementation notes:
- Review the commits separately: the first is small and implements the
  real feature.  The second moves the strand types into lull so khan can
  refer to them.

- In lull, I wanted to put +rand inside +khan, but this fails to that
  issue that puts the compiler in a loop.  +rand depends on +gall, which
  depends on +sign-arvo, which depends on +khan.  If +rand is in +khan,
  this spins the compiler.  The usual solution is to either move
  everything into the same battery (very ugly here) or break the
  recursion (which we do here).
2022-08-29 21:04:50 -08:00
Philip Monk
fa569cf7f3 kiln: wip 2022-08-29 10:53:36 -08:00
Philip Monk
9a4d754429 clay: add +tire subscription for app state 2022-08-25 19:13:51 -08:00
fang
9c3c2b13c7
eyre: send valid auth tokens to the runtime
Whenever a session gets created or removed, send the set of valid auth
tokens to the runtime, so that it may use them in determining whether
incoming requests are authenticated or not.
2022-08-24 00:34:08 +02:00
Ted Blackman
d8df2787a0 gall: %rake task to clean up dead %leave's 2022-08-18 20:34:42 +03:00
Philip Monk
50c22343b6 clay: wip 2022-08-16 20:33:25 -08:00
fang
04b7b86cae
gall: support %doff-ing specific apps and/or ships
This gives us more control over exactly which subscriptions get culled,
which is useful in debugging and targetted recovery scenarios.
2022-08-10 20:33:07 +02:00
Ted Blackman
543efac183 gall: kill old subscriptions on %doff (still has bug)
Squashed commit of the following:

commit 1e31ab96c14e7506d5cb746d90153cb5658df6e0
Author: Ted Blackman <ted@tlon.io>
Date:   Wed Aug 10 19:08:39 2022 +0300

    ames: revert prints on bad memos

commit c47f9c6053c05da0177f0c64210a88161e8adc98
Author: Ted Blackman <ted@tlon.io>
Date:   Wed Aug 10 18:44:56 2022 +0300

    Merge fix to +ap-kill-down and use that pattern in +mo-doff.

    This doesn't seem to work properly, but I'm not sure why so far.

commit e696816b8d3ab81534de6c2910d0fd4bebcfb68a
Merge: 4edb247cd7 a5f08643ff
Author: Ted Blackman <ted@tlon.io>
Date:   Wed Aug 10 18:29:42 2022 +0300

    Merge branch 'next/arvo' into ted/doff-revival

commit 4edb247cd77ac988e5b11806d0d9b5e7cec4a39e
Author: Ted Blackman <ted@tlon.io>
Date:   Sat Aug 6 00:09:04 2022 +0300

    ames,gall: fix lopsided bitt/boar state from old doffs

commit b21ffd9cfab0987297f3ae4cd16447ed0c3a467e
Merge: 7d532afaf8 8eb8a1da29
Author: Ted Blackman <ted@tlon.io>
Date:   Fri Aug 5 17:53:15 2022 +0300

    Merge branch 'next/arvo' into ted/kill-subs

commit 7d532afaf80b773139b66d787c4a446e3d6a9a33
Author: Ted Blackman <ted@tlon.io>
Date:   Fri Aug 5 17:52:16 2022 +0300

    gall: add subscription nonce to %doff

commit f750de9aacdb19f796bff527e220df9c10e41e71
Merge: 38540658b5 aad5fa6fae
Author: Ted Blackman <ted@tlon.io>
Date:   Fri Aug 5 11:24:55 2022 +0300

    Merge branch 'next/arvo' into ted/kill-subs

commit 38540658b59d703b0fa162af4f958948b7ee2031
Author: fang <git@fang.io>
Date:   Thu Jul 28 21:10:57 2022 +0200

    gall: account for renaming of subscription state

commit c015dc8446fc02729ad492e24635c9c521f09965
Merge: 7c222e4c60 731e27d5a1
Author: Zach Alberico <git@zalberico.com>
Date:   Thu Jul 28 10:14:24 2022 -0700

    Merge branch 'next/arvo' into ted/kill-subs

commit 7c222e4c60d946042fd06d858086a883bf8a0f7b
Author: Ted Blackman <ted@tlon.io>
Date:   Thu Jul 28 14:01:00 2022 +0200

    hood: |doff generator

commit 8d00cf1bd1b304f30431614b68002827e498d97e
Author: Ted Blackman <ted@tlon.io>
Date:   Thu Jul 28 13:21:45 2022 +0200

    gall: fix compile errors

commit 6a1fd360ffc65d8c6f90e491bb21fa7799a98c31
Author: Ted Blackman <ted@tlon.io>
Date:   Thu Jul 28 13:01:16 2022 +0200

    gall: add %doff task to kill subscriptions
2022-08-10 19:09:47 +03:00
yosoyubik
c599e60c5b ames: add recork set 2022-07-15 16:45:32 +02:00
Philip Monk
a4aa835fe1 clay: add %rein move to config agents 2022-07-06 23:09:43 -07:00
Philip Monk
d4f02e0a57 clay: add global %goad logic 2022-07-06 03:32:15 -07:00
Philip Monk
6d321c60cb clay, gall: move agent build management to clay 2022-07-06 00:51:11 -07:00
Philip Monk
89b9977ac8 azimuth: fetch snapshot on boot
Previously, the initial Azimuth snapshot was stored in Clay and shipped
in the pill.  This causes several problems:

- It bloats the pill
- Updating the snapshot added large blobs to Clay's state.  Even now
  that tombstoning is possible, you don't want to have to do that
  regularly.
- As a result, the snapshot was never updated.
- Even if you did tombstone those files, it could only be updated as
  often as the pill
- And those updates would be sent over the network to people who didn't
  need them

This moves the snapshot out of the pill and refactors Azimuth's
initialization process.  On boot, when app/azimuth starts up, it first
downloads a snapshot from bootstrap.urbit.org and uses that to
initialize its state.  As before, updates after this initial snapshot
come from an Ethereum node directly and are verified locally.

Relevant commands are:

- `-azimuth-snap-state %filename` creates a snapshot file
- `-azimuth-load "url"` downloads and inits from a snapshot, with url
  defaulting to https://bootstrap.urbit.org/mainnet.azimuth-snapshot
- `:azimuth &azimuth-poke-data %load snap-state` takes a snap-state any
  way you have it

Note the snapshot is downloaded from the same place as the pill, so this
doesn't introduce additional trust beyond what was already required.
When remote scry is released, we should consider allowing downloading
the snapshot in that way.
2022-06-30 23:22:50 -07:00
fang
292a65f64c
Merge branch 'next/arvo' into yu/content-dist-merged
Includes patched versions of ames' and clay's +load arms.

In clay, we do a dumb ;; hack to get the state to adapt properly. This
shouldn't be needed ($case had an extra... case added to it, old ones
should still nest), and so we should revisit the logic there to make it
cleaner/better before release.
2022-06-30 16:27:02 +02:00
yosoyubik
2e92331a0b Merge branch 'next/arvo' into yu/gall-rq-wire-ames-flow-kill 2022-06-16 08:54:24 +02:00
Ted Blackman
4825f8968c lull,ames: renames, more packet verification
Regularized arm names to +etch-* and +sift-* to match Vere.  Renamed
$packet to $shot.  Used $meow, $purr, and $keen to match Vere's naming.
Reorganized packet decoding arms and moved some to Lull for later use in
Aqua.
2022-06-03 22:58:50 -05:00
Ted Blackman
b8ff5f42e3 lull: fix (duplicate) 2022-05-28 22:18:27 -05:00
Ted Blackman
062fbc71d7 ames: merge driver rework 2022-05-27 14:52:30 -05:00
Ted Blackman
aaffad63b0 ames: fix up packet encoding 2022-05-27 13:14:53 -05:00
Ted Blackman
302d406a94 ames: WIP runs and ames works, but scrying kills ships 2022-05-26 21:23:09 -05:00
Ted Blackman
a1f202deb0 fine: refactor driver and arvo/vere interface 2022-05-25 17:30:35 -05:00
Philip Monk
b914fbaedc clay: add logging commands 2022-05-20 17:01:50 -07:00
fang
a0f7891f50
Merge branch 'philip/tomb' into lf/content-dist-rc
Beware, the ames conflict was particularly nasty. State migrations as of
yet untested.

Includes patched-up fine tests.
2022-05-19 17:07:35 +02:00
yosoyubik
fe412b1ca7 gall: remove $watches from egg/yoke
Also renames $beat to $boar so it signifies more its closeness to $boat
2022-05-18 15:27:17 +02:00
Philip Monk
61c1e19db5 clay: progress download when receiving tombstone 2022-05-18 00:11:16 -07:00
yosoyubik
83356f02ca gall: add $beat map for subscription nonces
Previously we stored the nonce in $boat, which changed the $bowl of each
agent. This compiles and all agents reload, but more testing is needed.
It also renames inbound/outbound watches to $bitt/$boat.
2022-05-16 09:13:40 +02:00
yosoyubik
a1dcc5a8cd gall: print trace logs by agent
This uses the same patterns found in %ames. We add an %odd flag
for logs related to the gall-rq PR, and leave it open for other flags
2022-05-15 17:16:18 +02:00
Philip Monk
2a38467f04 clay: re-export page to not break apps 2022-05-15 02:54:44 -07:00
Philip Monk
c091ec92eb various: make all files build correctly 2022-05-14 22:49:10 -07:00
Philip Monk
8a2eac7c70 Merge remote-tracking branch 'origin/ted/ames-prod-2' into philip/tomb 2022-05-14 21:17:10 -07:00
Philip Monk
d50d3915b8 Merge remote-tracking branch 'origin/yu/ames-fixes' into philip/tomb 2022-05-14 19:28:30 -07:00
fang
bb7e74c1e9
arvo: add %uv hash case to scry paths
Clay desparately wants something like this. It helps answer questions
around hash-based content fetching.
2022-05-11 16:16:03 +02:00
Philip Monk
32ff1417a7 clay: only maintain mime cache if mounted 2022-05-10 20:33:01 -07:00
fang
8bef14d0a7
Merge branch 'philip/tomb' into lf/content-dist-rc 2022-05-05 00:33:37 +02:00
Philip Monk
591bdf458c Merge branch 'next/arvo' into philip/tomb 2022-05-04 00:45:57 -07:00
fang
b42833a568
Merge branch 'm/scry-vane-next' into lf/content-dist-rc 2022-05-03 15:29:16 +02:00
yosoyubik
53e1c86833 Merge remote-tracking branch 'origin/yu/ames-fixes' into yu/gall-rq-wire-ames-flow-kill 2022-05-03 13:57:39 +02:00
h5gq3
c95cce4404 style fixes 2022-05-03 12:49:09 +02:00
h5gq3
5583fc1320 ames-flow-kill 2022-05-03 12:49:09 +02:00
Ted Blackman
a4ae79e794 gall: compiles with nonce in wire 2022-05-03 12:49:08 +02:00
Philip Monk
a7eaddfdc7 clay: remove ankh
ankh was a cache of the current validated blobs, which is no longer
necessary
2022-04-29 17:29:50 -07:00
yosoyubik
1b01455fe5 ames: fix dangling-bone error 2022-04-22 12:25:55 +02:00
fang
12ca0f45e5
ames: move various types and gates into lull
So that userspace may also en- and decode (scry) packets.
2022-04-20 00:09:00 +02:00
Liam Fitzgerald
8a3ec4a8d8 urbit: lull@328K, zuse@417K 2022-04-15 16:54:43 -05:00
Liam Fitzgerald
4ae93c8bd0 ames: strip out notifications system 2022-04-15 16:42:33 -05:00
Philip Monk
ce5ef26202 Merge branch 'philip/direct' into philip/tomb 2022-04-13 01:00:23 -07:00
Philip Monk
8805e0ee85 clay: convert all blobs to pages
This converts the blob store from having deltas, directs, and
tombstones, to just having direct pages.  This simplifies a lot of code,
since we don't have to constantly ensure that deltas always have their
parent available.

This removes the hardcoded text diff logic from clay, which was
previously required for bootstrapping.

Over the wire, we handle both old and new requests and responses
transparently, so communication is normal in both directions across
ships which do or do not have this change.
2022-04-13 00:35:58 -07:00
Philip Monk
bb14c495da clay: seek wip 2022-04-12 21:02:50 -07:00
Philip Monk
72fa850912 clay: don't download unneeded blobs 2022-04-12 18:31:55 -07:00
Philip Monk
ed00d38166 clay: add |tomb 2022-04-08 21:20:02 -07:00
Philip Monk
07d8187fcc clay: allow to set norm for past revision 2022-04-08 15:50:35 -07:00
fang
af4de87ff2
Merge branch 'm/aqua-revival' into m/scry-vane-next 2022-04-07 12:59:14 +02:00
Philip Monk
7482717c02 clay: make +pick fast 2022-04-06 23:16:26 -07:00
Philip Monk
8084200f81 clay: add norm state, remove mark from %delta blobs 2022-04-06 21:44:50 -07:00
Philip Monk
4131821b4f clay: unify all trie implementations
We had trie operations independently implemented in +de in arvo,
+an:cloy in zuse, +zu in clay, lib/trie, and app/spider.  This unifies
them all into +de in arvo, aggregating the used operations.
2022-04-06 00:19:38 -07:00
Philip Monk
0f6f2d6321 clay: bump kelvin, refactor 2022-04-06 00:18:23 -07:00
Philip Monk
90098067f2 clay: extend %tomb to delete all unneeded blobs 2022-03-26 14:37:46 -07:00
Philip Monk
c5fda496ac clay: add basic %tomb 2022-03-25 13:13:09 -07:00
Philip Monk
fb05255a83 clay: align comments 2022-03-25 12:38:51 -07:00
Philip Monk
0c3eee07b5 Merge remote-tracking branch 'origin/master' into philip/tomb 2022-03-25 12:21:56 -07:00
fang
01de5a06b0
term: consistently use x/y coordinate ordering
%rez has always used "width & height". Certainly, "x & y" is more
standard than "row & column". As such, we settle on making %hop and %hit
respect the more natural ordering. This change is safe because these
interfaces haven't made it to livenet yet.
2022-03-25 13:45:29 +01:00
Liam Fitzgerald
92e52c9f7e ames: fix path handling in %bide 2022-03-15 10:40:34 -06:00
Liam Fitzgerald
0587e0e836 ames: add %pine task
%pine automatically scries for the latest case of a path
2022-03-14 15:04:14 -06:00
Liam Fitzgerald
49e53290a1 Merge remote-tracking branch 'origin/yu/dangling-bone' into lf/content-dist-rc 2022-03-11 11:07:51 -06:00
yosoyubik
ab116653e8 wip fix dangling bone 2022-03-11 10:21:12 +01:00
fang
8906d1c17d
dill: move %mor case into $blit
This lets us send a single blit around, instead of sending facts for
every individual blit in a draw event.
2022-03-05 18:17:48 -06:00
Jōshin
9d50957861
arvo: remove $crag, promote $page 2022-03-02 17:59:56 -06:00
Jōshin
340f4d5b6c
khan: %fard/%arow use $cage, not $vase
Threads should eventually take and produce $cage instead of $vase. Since
%khan is likely to be used by third parties, we write to the eventual
intended API. We ignore the mark on the input $cage (it is safe to
always specify %noun), and we always use %noun as the output mark.

%fyrd now makes more sense. It was previously discarding the type of the
output %arow and re-encoding the raw noun as a vase of the output mark;
it is now performing mark conversion from the mark of the output $cage
to the originally requested output mark.
2022-03-02 13:07:52 -06:00
Liam Fitzgerald
f1c24c07dd ames: refactor to remove unused state 2022-02-23 16:00:56 -06:00
Liam Fitzgerald
6bd6029f57 ames: use finger trees for packet storage 2022-02-16 15:35:59 -06:00
Liam Fitzgerald
a36dc3aaff ames: per-lane congestion control 2022-02-15 08:47:38 -06:00
Jōshin
6aa72069e3
arvo: introduces $crag
Also strips out `$` from khan top-level comment.

There are arguments for keeping $crag in lull, and on the other side for
moving $cast to arvo. This seemed like the most reasonable approach.
2022-02-14 12:46:31 -06:00
Ted Blackman
b2051fcc2b fine: WIP second list for performance 2022-02-14 10:34:26 -06:00
Jōshin
6c443191a7
khan: name $cast type 2022-02-13 21:13:48 -06:00
Jōshin
c0423adea6
khan: parameterized types, nested implementation
%fyrd is now implemented in terms of %fard, and likewise %avow in terms
of %arow. State is tracked via wire rather than in a global map.

Unit tests adjusted to match.
2022-02-11 21:58:09 -08:00
Liam Fitzgerald
e97d99275b ames: congestion control for remote scry 2022-02-11 19:30:40 -06:00
Liam Fitzgerald
67105a854b lull: move ordered-map from zuse 2022-02-10 11:52:37 -06:00
Ted Blackman
a86664076d lull: fine congestion control data structures 2022-02-10 11:39:31 -06:00
Jōshin
8123255a62
khan: add %fard/%arow for in-arvo threads
These take and produce vases, and assign random tids (rather than
deducing them from the input duct.)

Since %fard does not require mark conversion, we make the mark/beak on
$thread-state optional (and use this to decide whether to send %avow or
%arow.) Provide a state adapter since it's possible that people have
been experimenting with this vane.

This makes the negative case of %avow/%arow kind of clunky, since there
is no content difference, but the following does not seem possible
within the Hoon type system:

    =/  gif
      ?~  p.tad
      %arow  %avow
    [hen %give gif %| p.cag tang]~
2022-02-07 17:51:05 -06:00
fang
5776226a6c
lull: add acru arms for simply signing & authing
Instead of exporting keys so that caller can do this themselves, we
expose arms for signing and authenticating that produce and operate on
just the signature, without mangling it into the message.
2022-02-02 21:52:48 +01:00
fang
83e315cccd
lull: add arms to ex:acru for separate keys
We may want _just_ the encryption key, or _just_ the signing key.
2022-02-02 20:10:30 +01:00
fang
709dd75d74
fine: track pending requests for unknown peers 2022-01-28 23:09:50 +01:00
Jōshin
21ea32e8ad
khan: +bear allows a full +beak 2022-01-27 22:28:30 -08:00
fang
4c045a9c4f
ames, fine: minimal compilable unification 2022-01-27 14:32:49 +01:00
Jōshin
7f6868a06e
khan: strip inner faces from +bear 2022-01-26 21:50:08 -08:00
Jōshin
1281e1cdbe
khan: functionally done 2022-01-26 17:41:01 -08:00
Jōshin
425bf6b011
khan: accepts thread arguments with marks 2022-01-26 16:26:30 -08:00
Jōshin
8ad57d8bd2
khan: cleanup 2022-01-25 21:28:45 -08:00
Jōshin
12b4c80cf1
khan: wip threads v0
super ghetto first pass with a bunch of stuff hard-coded to prevent
sending giant vases over the wire.
2022-01-25 20:08:48 -08:00
fang
176aa3a45e
lull: stop hardcoding ames bit in +encode-packet 2022-01-24 21:38:55 +01:00
fang
2c5510db64
Merge branch 'jb/next-gen-term' into next/dill
Resolves a good number of conflicts. Most notably, re-propagates removal
of gall's %onto, confirms new /app/herm behavior, coerces hood/drum
state adapters back into place, and updates webterm to use the latest
api.
2022-01-24 14:50:16 +01:00
Jōshin
a247113c57
khan: %done task 2022-01-22 01:46:12 +00:00
Jōshin
3cf803129d
khan: basic +call, types 2022-01-21 22:09:01 +00:00
Jōshin
575ff61fa6
khan: vane skeleton 2022-01-21 18:48:05 +00:00
fang
2d0a6f160b
fine: reuse packet en/decoding logic from ames
Also adds some more of the core logic, but still wip.
2022-01-20 20:36:38 +01:00
fang
dd4eb87f42
ames: move packet en/decoding into lull
By moving this behavior for packets with ames-style headers into lull,
other vanes that want to do protocols in similar style (like the upcoming
%fine vane) can reuse the logic here.

Note that we parameterize the main en- and decoding functions, so that
we may pass in the "is ames" flag and protocol versions as appropriate
to the context.
2022-01-20 01:59:43 +01:00
fang
355f05753f
fine: remote scry
Implements a WIP version of %fine, the remote scry vane. Plenty of work
remains to be done here, marked by TODOs and REVIEWs.
2022-01-14 00:22:56 +01:00
yosoyubik
8535563de6 jael: give private key to subscribers on keys diff
Triggering this event has been added to /app/azimuth's +on-load
2022-01-08 09:48:28 +01:00
yosoyubik
d08ab43f3a lull: skip rift/life printf when booting 2021-12-06 13:47:59 +01:00
Philip Monk
4656372b09
Merge pull request #5426 from ynx0/claybel
clay: support labels
2021-11-15 08:49:31 -08:00