Commit Graph

1832 Commits

Author SHA1 Message Date
yosoyubik
91435a92ca ames: don't drop events with old wires
Previously we were dropping events that used old
wires that lacked a rift in them. This seems a
bad behavior because we don't want to destroy a
flow that has not been processed by both ends.

Note: pending a fix to test-old-ames-wire
2022-04-22 12:26:14 +02:00
yosoyubik
6cdebea1af ames: address review 2022-04-22 12:26:08 +02:00
yosoyubik
60cc44e961 ames: fix cached state in larval upgrade 2022-04-22 12:26:02 +02:00
yosoyubik
1b01455fe5 ames: fix dangling-bone error 2022-04-22 12:25:55 +02:00
Philip Monk
dfd4fc1028 clay: wip 2022-04-19 20:03:32 -07:00
fang
12ca0f45e5
ames: move various types and gates into lull
So that userspace may also en- and decode (scry) packets.
2022-04-20 00:09:00 +02:00
Liam Fitzgerald
8a3ec4a8d8 urbit: lull@328K, zuse@417K 2022-04-15 16:54:43 -05:00
Liam Fitzgerald
4ae93c8bd0 ames: strip out notifications system 2022-04-15 16:42:33 -05:00
Philip Monk
3b337a14ce Merge branch 'philip/direct' into philip/tomb 2022-04-13 15:01:25 -07:00
Philip Monk
3a04dc3160 clay: address review comments 2022-04-13 15:00:59 -07:00
Philip Monk
10a2acdcc1 clay: use old requests for now when backfilling 2022-04-13 01:12:05 -07:00
Philip Monk
ce5ef26202 Merge branch 'philip/direct' into philip/tomb 2022-04-13 01:00:23 -07:00
Philip Monk
8805e0ee85 clay: convert all blobs to pages
This converts the blob store from having deltas, directs, and
tombstones, to just having direct pages.  This simplifies a lot of code,
since we don't have to constantly ensure that deltas always have their
parent available.

This removes the hardcoded text diff logic from clay, which was
previously required for bootstrapping.

Over the wire, we handle both old and new requests and responses
transparently, so communication is normal in both directions across
ships which do or do not have this change.
2022-04-13 00:35:58 -07:00
Philip Monk
bb14c495da clay: seek wip 2022-04-12 21:02:50 -07:00
Philip Monk
72fa850912 clay: don't download unneeded blobs 2022-04-12 18:31:55 -07:00
Philip Monk
ed00d38166 clay: add |tomb 2022-04-08 21:20:02 -07:00
Philip Monk
78d2d12cca clay; fix overly cautios tombstoning 2022-04-08 17:28:42 -07:00
Philip Monk
39b7b0a96b clay: add %tako scry 2022-04-08 16:21:19 -07:00
Philip Monk
07d8187fcc clay: allow to set norm for past revision 2022-04-08 15:50:35 -07:00
Philip Monk
d71a685b91 clay: add norm to +pick cache 2022-04-07 18:10:00 -07:00
Philip Monk
85fb3502c7 Merge branch 'philip/trie' into philip/tomb 2022-04-07 17:16:14 -07:00
Philip Monk
d46dc282c4 arvo: revert axal to use $ 2022-04-07 17:15:59 -07:00
Philip Monk
3096bc8991 Merge branch 'philip/trie' into philip/tomb 2022-04-07 16:31:27 -07:00
Philip Monk
6765a03cac clay: address review comments 2022-04-07 16:31:12 -07:00
fang
af4de87ff2
Merge branch 'm/aqua-revival' into m/scry-vane-next 2022-04-07 12:59:14 +02:00
Philip Monk
6dc20bfa7f clay: trying to find %dead bug 2022-04-07 00:44:49 -07:00
Philip Monk
7482717c02 clay: make +pick fast 2022-04-06 23:16:26 -07:00
Philip Monk
84ccf60b19 clay: add gc routines 2022-04-06 21:45:30 -07:00
Philip Monk
8084200f81 clay: add norm state, remove mark from %delta blobs 2022-04-06 21:44:50 -07:00
fang
13d1c28927
hoon: make manual shorthand type printing prettier 2022-04-06 23:24:43 +02:00
Philip Monk
a51152d1a4 clay: address review comments 2022-04-06 12:16:22 -07:00
Philip Monk
4131821b4f clay: unify all trie implementations
We had trie operations independently implemented in +de in arvo,
+an:cloy in zuse, +zu in clay, lib/trie, and app/spider.  This unifies
them all into +de in arvo, aggregating the used operations.
2022-04-06 00:19:38 -07:00
Philip Monk
0f6f2d6321 clay: bump kelvin, refactor 2022-04-06 00:18:23 -07:00
Philip Monk
90a2625016 clay: style 2022-04-05 20:30:52 -07:00
lukechampine
8399b7b96c ames: add resend timer for attestation requests 2022-04-04 00:19:55 -04:00
Joe Bryan
9a1b3eaf15
Merge pull request #5664 from urbit/jb/type-leak
hoon: plug type leak in compiler bootstrap
2022-03-28 09:08:59 -04:00
Philip Monk
90098067f2 clay: extend %tomb to delete all unneeded blobs 2022-03-26 14:37:46 -07:00
Jōshin
66655a76b8
arvo: $card is (cask) 2022-03-26 09:17:39 -06:00
Jōshin
29a6692784
khan: minor whitespace cleanup 2022-03-26 09:17:38 -06:00
Jōshin
83baaaafa0
lull, zuse: cool 2022-03-26 09:17:28 -06:00
Philip Monk
ee3951e2df clay: require non-dead files at head of remote 2022-03-25 22:15:53 -07:00
Philip Monk
4278d4c8b3
Merge pull request #5673 from urbit/philip/fix-uno
hoon: fix +uno:by when map is null
2022-03-25 18:23:32 -07:00
Philip Monk
07d01a0071 Merge branch 'philip/fix-uno' into philip/tomb 2022-03-25 18:11:34 -07:00
Philip Monk
50fb239d2f hoon: fix +uno:by when map is null
This crashed at runtime when `a` is null because it tried to instantiate
`meg`, and that means bunting its argument, and that calls `node`, where
the assert failed.

Wet gates use the bunt of their formal argument, so we use that.
2022-03-25 18:08:13 -07:00
Philip Monk
6e641005a0 clay: add upgrade to %11 2022-03-25 18:02:22 -07:00
Philip Monk
fb3f74f639 clay: prefer life 2022-03-25 15:32:38 -07:00
Philip Monk
ef9521272a Merge remote-tracking branch 'origin/lf/ford-memory-shit' into philip/tomb 2022-03-25 13:13:24 -07:00
Philip Monk
c5fda496ac clay: add basic %tomb 2022-03-25 13:13:09 -07:00
Philip Monk
fb05255a83 clay: align comments 2022-03-25 12:38:51 -07:00
Philip Monk
99a10aa7a8 clay: use XX for things that really need to change 2022-03-25 12:33:48 -07:00
Philip Monk
0c3eee07b5 Merge remote-tracking branch 'origin/master' into philip/tomb 2022-03-25 12:21:56 -07:00
Joe Bryan
d937e131a4 hoon: preserve structural sharing by testing subject/product equality in +burp 2022-03-25 10:28:05 -04:00
Joe Bryan
15e570a622 hoon: plug type leak in vase literals by properly burping %hint 2022-03-25 10:27:32 -04:00
Philip Monk
5ff4cc7f66
Merge pull request #5612 from ynx0/story
clay: add `story`, a set of tools to support commit messages
2022-03-22 21:49:16 -07:00
Liam Fitzgerald
dd70856188 clay: be careful with subject in ford results
This fixes a space leak where the entire ford/clay core would be
included in the ford cache. Heavily reduces memory usage by clay,
reducing total usage from 100-200MB to around 10MB.
2022-03-21 13:02:36 -06:00
Liam Fitzgerald
2def946ded clay: break out ford memory usage in more detail
Instead of reporting a single memory size for built files, marks and
conversions, we now report memory size per path, mark name and mark
pair, respectively.
2022-03-21 12:21:38 -06:00
Jōshin
4be43d5e28
hoon: fix comment whitespace 2022-03-21 10:24:27 -06:00
Liam Fitzgerald
92e52c9f7e ames: fix path handling in %bide 2022-03-15 10:40:34 -06:00
Liam Fitzgerald
0587e0e836 ames: add %pine task
%pine automatically scries for the latest case of a path
2022-03-14 15:04:14 -06:00
Liam Fitzgerald
49e53290a1 Merge remote-tracking branch 'origin/yu/dangling-bone' into lf/content-dist-rc 2022-03-11 11:07:51 -06:00
yosoyubik
5ecf87cf52 ames: fix cached state in larval upgrade 2022-03-11 10:22:50 +01:00
yosoyubik
ab116653e8 wip fix dangling bone 2022-03-11 10:21:12 +01:00
fang
3df227760f
hoon: hook up $+ for shorthand type rendering
Too often when dealing with big types the compiler traces and other such
outputs become hard to read. Wrapping a type as $+(shorthand big-type)
will now print #shorthand in place of the type.
2022-03-10 01:45:18 +01:00
Liam Fitzgerald
e621744169 zuse: add +balk core for remote scry path format
+balk defines a datastructure for a parsed remote scry path format in
addition to conversion gates to and from paths.

The new scry path format is /~ship/rift/life/vane/care/spur. Note that
desk no longer appears in the scry path format and should instead be
encoded as the first item of the spur.
2022-03-08 15:11:25 -06:00
ryjm
155729ec8b %home -> %base
Mostly cosmetic, but also fixes `language-server` and parts of sundry
neglected components (the nix build for solid pills, aqua, gaze exports,
+help)
2022-03-06 11:55:55 -05:00
Jōshin
06fe78ef6e
Merge branch 'next/arvo' into jo/khan 2022-03-02 20:28:44 -06:00
Jōshin
0139550c79
khan: saner tic rollover, remove our 2022-03-02 20:15:44 -06:00
Jōshin
e54e92cfa9
khan: docs 2022-03-02 18:02:20 -06:00
Jōshin
9d50957861
arvo: remove $crag, promote $page 2022-03-02 17:59:56 -06:00
Jōshin
8afea3c2ba
arvo: $card is not a cask 2022-03-02 17:38:45 -06:00
Jōshin
0b507cf498
khan: don't reset tic
instead make it a rolling 128-bit integer. 128 bits is the same size as
the +sham space, so is one natural choice for "big enough to never have
to think about this." 64, 32, 16, even 8 bits would probably be fine.
2022-03-02 17:28:10 -06:00
Jōshin
18274090b2
khan: handle multiple %fards in one event 2022-03-02 16:37:46 -06:00
Jōshin
5d7321385f
khan: separate arms by blank comments 2022-03-02 15:09:01 -06:00
Jōshin
1ef958ea25
khan: use +en-beam / +de-beam, wide form 2022-03-02 15:07:38 -06:00
Jōshin
02d796cddb
arvo: remove redundant * 2022-03-02 15:07:10 -06:00
Jōshin
91d390d529
khan: update docs 2022-03-02 13:23:06 -06:00
Jōshin
340f4d5b6c
khan: %fard/%arow use $cage, not $vase
Threads should eventually take and produce $cage instead of $vase. Since
%khan is likely to be used by third parties, we write to the eventual
intended API. We ignore the mark on the input $cage (it is safe to
always specify %noun), and we always use %noun as the output mark.

%fyrd now makes more sense. It was previously discarding the type of the
output %arow and re-encoding the raw noun as a vase of the output mark;
it is now performing mark conversion from the mark of the output $cage
to the originally requested output mark.
2022-03-02 13:07:52 -06:00
Yaseen
a57cb822a3
Merge branch 'urbit:master' into story 2022-03-02 12:03:44 -05:00
lukechampine
53a0c87cdb ames: add sendkeys-packet 2022-03-02 00:56:57 -05:00
lukechampine
df6318acd0 ames: fix comet self-attestation handling 2022-03-02 00:52:48 -05:00
lukechampine
1044d42a15 ames: don't enqueue blobs sent to unknown comets 2022-03-02 00:46:18 -05:00
Jōshin
f5fb214939
khan: initial state version is %0 2022-03-01 23:38:55 -06:00
Jōshin
6ce6e4f504
khan: structural slap, not general slam 2022-03-01 19:34:56 -06:00
Jōshin
2e9b4f5a94
khan: better wire production
runs afoul of a likely hoon bug and/or use case for dependent hoon.
2022-03-01 19:33:54 -06:00
Yaseen
70afcbd642 clay: fix bug in fuse
Co-authored-by: Philip Monk <phil@pcmonk.me>
2022-02-25 16:34:44 -05:00
Jōshin
a9442040be
khan: alignment 2022-02-25 11:16:47 -08:00
Liam Fitzgerald
b4da246f01 strandio: add +keen gate 2022-02-24 12:03:07 -06:00
fang
027d80b90c
Merge branch 'lf/content-dist-rc' into m/scry-vane 2022-02-23 16:18:17 -06:00
Liam Fitzgerald
f1c24c07dd ames: refactor to remove unused state 2022-02-23 16:00:56 -06:00
Liam Fitzgerald
c16aaae201 ames: correct behaviour for %yawn 2022-02-23 16:00:56 -06:00
Liam Fitzgerald
479e060e32 ames: add +load for remote scry 2022-02-23 16:00:56 -06:00
Liam Fitzgerald
5baa1b8318 ames: add fast retransmit 2022-02-23 16:00:55 -06:00
Liam Fitzgerald
4be4e9a7a0 ames: fix keen ducts 2022-02-23 16:00:55 -06:00
fang
aaa759268e clay: clean up REVIEW comments 2022-02-23 16:00:55 -06:00
fang
cdff1d91fa clay: scry files at aeons, not timestamps
Turns out we can simply find the corresponding aeon in the nako.
2022-02-23 16:00:55 -06:00
fang
2adc52e03a clay: retry with ames on empty scry response 2022-02-23 16:00:55 -06:00
Liam Fitzgerald
5f5092168f ames: fix sig + data catenation 2022-02-23 16:00:55 -06:00
Liam Fitzgerald
22f3e49823 ames: add |yawn, fix duct wrangling 2022-02-23 16:00:31 -06:00
fang
6b85022a61
clay: properly cancel scry timers and/or requests
Use the correct wires when cancelling the scry request and/or its
timers.

Note that we may produce more %rests and %yawns than strictly necessary,
but these no-op cleanly in those cases.
2022-02-23 15:41:19 -06:00
fang
d8dca44bd2
clay: refactor request wire construction 2022-02-23 15:40:16 -06:00
Joe Bryan
ff3bb495d8 hoon: updates %mcgl with an hygienic expansion 2022-02-22 20:39:30 -05:00
Jōshin
d771a4b9ef
khan: randomize tids
tid was accidentally getting set to the name of the output mark. As we
don't currently support cancelling threads, there is no reason to
maintain the originally-intended correspondence between tid and conn
request-id.

Take the opportunity to clean up indentation somewhat.
2022-02-19 02:18:49 -08:00
~nallux-dozryl
ab35dd4028
Merge branch 'urbit:master' into master 2022-02-19 09:04:02 +08:00
Yaseen
3d006d60ad clay: add scry for case existence check 2022-02-18 17:43:38 -05:00
fang
2969b1aae1
clay: clean up REVIEW comments 2022-02-17 21:17:45 +01:00
fang
02dec3ef5b
clay: scry files at aeons, not timestamps
Turns out we can simply find the corresponding aeon in the nako.
2022-02-17 21:14:52 +01:00
fang
b91e3f425e
clay: retry with ames on empty scry response 2022-02-17 20:10:38 +01:00
Liam Fitzgerald
ecb2de63b0 Merge remote-tracking branch 'origin/m/scry-vane' into lf/content-dist-rc 2022-02-17 12:46:09 -06:00
fang
7c0f05f8c3
clay: hook up /back-index %wake
And make sure we handle the timer error case safely.
2022-02-17 16:08:48 +01:00
fang
d6b5633c99
clay: allow for lobe-only need
This can occur after upgrade, or when receiving a %delta blob. We do not
know the path or commit timestamp of a file that resolves to the blob,
so we simply fall back to old ames-style blob-by-lobe fetching.
2022-02-17 16:02:57 +01:00
fang
3473033492
clay: scry when doing file requests outside %sing
In +work, we simply want to fetch file data, regardless of what the
overarching request is. (In fact, %sing would likely never hit this path
anyway.)

In order to be able to make the scry request, we track the timestamp of
the commit that contains a blob we're missing, and scry for that %da
revision.

There's two edge cases where we cannot immediately know the timestamp
that are currently assumed broken. Fix soon.
2022-02-17 15:44:16 +01:00
fang
dd4a115e51
clay: avoid %v scries, cancel scry timer correctly
%v requests are already handled specially for the ames case. We should
continue to respect that, only doing those kinds of requests over ames.

Also cancels the scry timer after receiving a response, instead of not
doing that.
2022-02-17 15:40:44 +01:00
Liam Fitzgerald
6bd6029f57 ames: use finger trees for packet storage 2022-02-16 15:35:59 -06:00
Jōshin
470ae80d4b
khan: cleanup 2022-02-15 22:32:52 -08:00
fang
fb4b3c7196
clay: update straggler state version number 2022-02-15 22:35:53 +01:00
Liam Fitzgerald
a36dc3aaff ames: per-lane congestion control 2022-02-15 08:47:38 -06:00
fang
0375e0a53f
clay: remove dist-upgraded flag
Ancient version upgraders beware! If you're coming from a home-based
arvo, you must first upgrade to the version prior to this, or else
you'll be in trouble!
2022-02-15 01:02:19 +01:00
fang
c02274dfdd
clay: sync files using remote scry
When clay wants to download blobs from a foreign ship, it attempts doing
this using the new remote scry protocol.

If it doesn't receive a response within ~m1, it falls back to using the
old ames-based syncing instead. We remember this "prefer ames" state for
the specific ship for an hour, after which we'll begin trying the scry
flow again.

Compiles, but untested. Some TODOs and REVIEWs remaining herein.

We intentionally leave the dist-upgraded flag in state to avoid
cluttering the diff here. The next commit will remove it.
2022-02-15 00:51:54 +01:00
Jōshin
6aa72069e3
arvo: introduces $crag
Also strips out `$` from khan top-level comment.

There are arguments for keeping $crag in lull, and on the other side for
moving $cast to arvo. This seemed like the most reasonable approach.
2022-02-14 12:46:31 -06:00
Ted Blackman
b2051fcc2b fine: WIP second list for performance 2022-02-14 10:34:26 -06:00
Jōshin
6d275fe9f3
khan: move =/ down 2022-02-14 09:09:55 -06:00
Jōshin
6c443191a7
khan: name $cast type 2022-02-13 21:13:48 -06:00
Jōshin
044e9015af
Merge branch 'next/arvo' into jo/khan
Conflicts:
	bin/brass.pill
	bin/solid.pill

Resolved by regenerating pills.
2022-02-13 21:00:10 -06:00
Jōshin
1d487a758b
test: full run, happy path 2022-02-13 07:33:03 -08:00
Jōshin
d7608019f2
khan: cleanup, namespace 2022-02-11 22:18:15 -08:00
Jōshin
c0423adea6
khan: parameterized types, nested implementation
%fyrd is now implemented in terms of %fard, and likewise %avow in terms
of %arow. State is tracked via wire rather than in a global map.

Unit tests adjusted to match.
2022-02-11 21:58:09 -08:00
Liam Fitzgerald
e97d99275b ames: congestion control for remote scry 2022-02-11 19:30:40 -06:00
Liam Fitzgerald
67105a854b lull: move ordered-map from zuse 2022-02-10 11:52:37 -06:00
Ted Blackman
a86664076d lull: fine congestion control data structures 2022-02-10 11:39:31 -06:00
Jōshin
ca0a861c5e
zuse: add missing assertions 2022-02-09 14:14:12 -06:00
Liam Fitzgerald
b813580b93 zuse: add ++ns:dejs:format
Adds a signed integer JSON parser, i.e. a gate of type $-(json @s)
2022-02-09 13:16:24 -06:00
Jōshin
5ff5082a5d
khan: docs, lift arg vase into (unit) 2022-02-08 12:03:00 -06:00
Jōshin
8123255a62
khan: add %fard/%arow for in-arvo threads
These take and produce vases, and assign random tids (rather than
deducing them from the input duct.)

Since %fard does not require mark conversion, we make the mark/beak on
$thread-state optional (and use this to decide whether to send %avow or
%arow.) Provide a state adapter since it's possible that people have
been experimenting with this vane.

This makes the negative case of %avow/%arow kind of clunky, since there
is no content difference, but the following does not seem possible
within the Hoon type system:

    =/  gif
      ?~  p.tad
      %arow  %avow
    [hen %give gif %| p.cag tang]~
2022-02-07 17:51:05 -06:00
Yaseen
ce0fa85e18 story: generators improvements
- use desk parameter instead of %base everywhere
- formatting clean up
- make |story-remove take a case instead of an aeon
- make desk param optional for story-set and story-log
2022-02-05 22:50:10 -05:00
fang
a5b16b1673
fine: consolidate duplicate logic
Further merge cleanup.
2022-02-04 13:36:32 +01:00
fang
47feec3ecc
Merge branch 'lf/ames-notifs' into m/scry-vane 2022-02-03 22:00:24 +01:00
Jōshin
64fcd685af
zuse: comment clarifying sk bounds check 2022-02-03 13:52:09 -06:00
Jōshin
4cef7dc38b
bounds-check against sk=1
+sign:schnorr crashes on `=(0 sk)`, so the bounds checking code is not
exercised for sk=0. It also crashes on `(gte sk n.domain.c)`, which is
redundant with the size check on sk, so we remove that.
2022-02-03 13:21:21 -06:00
Jōshin
f07f759253
zuse: style cleanup, use +rep/+end 2022-02-03 12:54:27 -06:00
fang
a8574df401
ames: retrieve pk from old crypto core on load
Since we cannot scry the key out of jael during load, and the
alternative is defering this logic through the larval core in some way,
we simply include the type for the old acru interface, letting us reuse
the old core to initialize the new one.
2022-02-03 13:49:38 +01:00
Jōshin
822eb6ea08
Merge branch 'next/arvo' into jo/secp-arvo 2022-02-02 20:34:11 -06:00
fang
39959942d2
ames: use +safe:as:crub instead of +veri:ed
+safe now exists to wrap that, so we should use it instead of deriving
the relevant key and calling +veri:ed directly.
2022-02-02 22:16:16 +01:00
fang
5776226a6c
lull: add acru arms for simply signing & authing
Instead of exporting keys so that caller can do this themselves, we
expose arms for signing and authenticating that produce and operate on
just the signature, without mangling it into the message.
2022-02-02 21:52:48 +01:00
fang
83e315cccd
lull: add arms to ex:acru for separate keys
We may want _just_ the encryption key, or _just_ the signing key.
2022-02-02 20:10:30 +01:00
fang
1302c5fa0b
fine: fix broken +full:keys callsite
Straggler from last commit about data type consistency.
2022-02-02 19:55:20 +01:00
Liam Fitzgerald
9dad96332b ames: remove logging 2022-02-01 20:50:02 -06:00
Liam Fitzgerald
6ee2e1626f ames: unify path format 2022-02-01 20:48:04 -06:00
Liam Fitzgerald
1eaad36d4d ames: working notifications 2022-02-01 20:30:11 -06:00
Liam Fitzgerald
7205625abf wip 2022-02-01 13:25:28 -06:00
fang
c72c187edd
fine: use $@(~ (cask)) consistently
It's what ends up in the packet, probably better than passing units and
converting them all over.
2022-02-01 12:02:47 +01:00