Commit Graph

515 Commits

Author SHA1 Message Date
Ted Blackman
09a0da4c84 clay: refuse %cs subs scry unless at now 2023-02-03 12:33:27 -06:00
bacwyls
4c2e5460bd
clay: revise /subs %cs scry again
return an empty set instead of crashing if desk doesn't exist
2023-02-03 11:46:54 -06:00
bacwyls
3c7ff0c1f2
Merge branch 'develop' into bwyl/clay-subs 2023-02-03 09:47:04 -06:00
bacwyls
f75e52b9ff clay: revise scry for subscribers to a desk
Address PR6136 comments to improve the interface to this scry.
Now it looks like .^((set ship) %cs /=landscape=/subs)
  instead of .^((set ship) %cs %/subs/landscape)
2023-02-02 17:20:21 -06:00
tadad
a5e0356fd6
Merge branch 'develop' into da/flow-scry 2023-01-26 17:10:11 -06:00
dachus
9e9fdeb2c6 flow scry added, moved to lull 2023-01-26 17:04:08 -06:00
fang
c61a9f8bd7
clay: update %prep task for pages instead of blobs 2023-01-23 21:10:28 +01:00
fang
f2d3a64d9c
clay: add /cs/===/bloc scry for all desk blobs
Lets you retrieve all blobs from the blob store that are in use by the
desk, both presently and in all its history.
2023-01-23 20:39:18 +01:00
fang
3fa5856f94
clay: add %prep task for priming the blob store 2023-01-23 20:36:34 +01:00
fang
50cfdb0f2a
clay: allow scrying out of pending requests
Implements a /cx/[our]//[now]/cult/[desk] endpoint, for getting a set of
pending requests for any given desk. We don't give the $cult for the
desk as-is, but instead slim the $roves back down into $raves, remove
clay protocol version metadata, and make sure to put our @p in place of
empty "for" fields.
2023-01-22 21:42:57 +01:00
Philip Monk
9d56637916 clay: Drop foreign %wris responses
This flow is not supported, and it was causing issues releasing
416.  This change just drops the responses to avoid crashing, but at
some point we should either support this flow or reject the request in
the first place.
2023-01-18 17:28:51 -07:00
fang
721fa05519
Merge branch 'next/arvo' into next/dill 2023-01-03 18:56:33 +01:00
bacwyls
94519cbfbf clay: add scry for subscribers of a desk
This data is valuable for app developers and %base OTA providers
2022-12-07 17:16:11 -06:00
Philip Monk
4e95a45dab clay: don't clear ford cache on %trim
Most of the memory stays in gall anyway, and this means you need to
recompile everything the next time anything changes, which could be
counterproductive.  It's important that %trim not make things worse.

The functionality is moved to the debug %stir task.
2022-11-28 16:17:41 -07:00
Philip Monk
cb0bbf1e32 clay: don't eagerly build marks
Building the marks adds 50% to the memory requirements for reloading
clay on ~wicdev, from 320MB to 470MB.  This isn't worth it for now.
2022-11-28 16:11:34 -07:00
Philip Monk
38a26041c2 clay: don't add road transitions in clay
These were originally added because they reduced memory usage, primarily
by clearing the memoization cache.  Now that the memoization cache is
no longer used, we use less memory without them.  On ~wicdev-wisryt with
~30 apps, updating Clay now takes ~320MB.
2022-11-28 16:08:41 -07:00
Philip Monk
280c1cb19a clay: crash if conflicting agent names 2022-11-21 11:05:55 -07:00
Philip Monk
a600ca0663 clay: don't re-commit %base during kelvin upgrade 2022-11-20 22:06:29 -07:00
Philip Monk
d5ecc3c053 lull: bump to 328 2022-11-20 20:58:39 -07:00
Philip Monk
8376a759ae clay: make compile 2022-11-20 18:41:00 -07:00
Philip Monk
45b5257c04 clay: document [tare] invariant 2022-11-17 19:32:40 -07:00
Philip Monk
8255dcd4bd clay: document [goad] invariant 2022-11-17 17:45:19 -07:00
Philip Monk
54c2b652e5 clay: document [wick] invariant 2022-11-17 17:30:06 -07:00
Philip Monk
3ec3516ccb clay: document [ergo] invariant 2022-11-17 17:19:02 -07:00
Philip Monk
df130fc015 clay: document [wake] invariant 2022-11-17 16:47:56 -07:00
Philip Monk
57be9e28bc clay: light cleanup 2022-11-14 16:12:20 -07:00
Philip Monk
c382eb14f9 clay: call +tare when tire might have changed 2022-11-14 14:59:44 -07:00
Philip Monk
ebf09bd248 clay: defer all calls to %wick 2022-10-26 22:09:53 -07:00
Philip Monk
301158c40a clay: use single-weft wafts when possible 2022-10-23 14:52:32 -07:00
Philip Monk
4490e98470 clay: move goads into wick 2022-10-17 21:16:54 -07:00
Philip Monk
5f5878cf6f clay: add %stay, which removes commit-in-waiting 2022-10-17 20:20:22 -07:00
Philip Monk
4f5600df15 clay: don't cast in +stay 2022-10-13 21:42:36 -07:00
Philip Monk
fc70193ef6 Merge remote-tracking branch 'origin/next/arvo' into philip/agent-clay 2022-10-13 20:54:00 -07:00
Philip Monk
478da3b77b clay: support ranges in sys.kelvin 2022-10-13 20:36:33 -07:00
Philip Monk
ca8249a0be clay: rename %next -> %held 2022-10-13 19:27:50 -07:00
Philip Monk
f01f046af3 clay: add verbose output to goad 2022-10-08 22:50:44 -07:00
Philip Monk
2829f5b354 clay: allow old kelvins on best-effort basis
also add back |bump, with no arguments
2022-10-04 23:57:37 -07:00
fang
bdf9f9d044
Merge pull request #5929 from urbit/jon/remove-dork
clay: remove $dork
2022-09-15 22:23:56 +02:00
Edward Amsden
105710d54e
clay: remove stack tracing 2022-09-14 19:28:17 -05:00
Edward Amsden
4f8c851ef7
Don't print a stack trace for read-at-aeon-fail
We shouldn't get a clay stack trace for read-at-aeon-fail because that gives us miles of clay stack trace whenever hoon compilation fails
2022-09-14 19:26:21 -05:00
Philip Monk
77319d20c4 clay: fix kelvin upgrade issues 2022-09-14 01:04:13 -08:00
Philip Monk
c3f5fe923a docket: update to use tire instead of kiln 2022-09-13 16:14:43 -08:00
Philip Monk
c46fb686ff kiln: remove resolve XX's 2022-09-12 20:51:39 -08:00
Philip Monk
8884e7dfbc kiln: make compile 2022-09-02 16:51:39 -08:00
Philip Monk
fa569cf7f3 kiln: wip 2022-08-29 10:53:36 -08:00
Philip Monk
9a4d754429 clay: add +tire subscription for app state 2022-08-25 19:13:51 -08:00
Philip Monk
50c22343b6 clay: wip 2022-08-16 20:33:25 -08:00
drbeefsupreme
2b7b3266ac clay: remove $dork
seems that this structure has been unused since
e75ab631a4 and confuses
newbies trying to figure out exactly what the commit
structure is (which is how I came across this)
2022-08-07 13:08:55 -04:00
fang
a1b3348cdf
Revert "clay: render syntax errors at end of file"
This reverts commit 7a110db60d.
2022-07-07 16:21:43 +02:00
Philip Monk
1bc0d0cd8e gall: don't reload if code is unchanged 2022-07-07 00:12:27 -07:00
Philip Monk
a4aa835fe1 clay: add %rein move to config agents 2022-07-06 23:09:43 -07:00
Philip Monk
56ed49578f clay: respect rein state 2022-07-06 22:26:34 -07:00
Philip Monk
71ecde32e1 clay: keep marks warm 2022-07-06 18:12:59 -07:00
Philip Monk
eb1e7dd173 clay: enable goad on every commit 2022-07-06 16:56:38 -07:00
Philip Monk
ddf13b3fb8 clay: make upgrade compile 2022-07-06 11:51:06 -07:00
Ted Blackman
0d4ce15780 clay: add .liv and .ren state to dome 2022-07-06 16:41:20 +01:00
Philip Monk
d4f02e0a57 clay: add global %goad logic 2022-07-06 03:32:15 -07:00
Philip Monk
6d321c60cb clay, gall: move agent build management to clay 2022-07-06 00:51:11 -07:00
Philip Monk
ba702d5f76 clay: comments 2022-06-30 18:12:19 -07:00
Philip Monk
172af42def clay: clear ford cache on +load instead of +stay
Best practice is clean up state on +load, when you know more about what
the migration should be.

fixes #5849
2022-06-30 11:55:44 -07:00
fang
abd008e453
clay: patch state migration logic
We had overlooked changes to the $update-state type. Accounting for
those makes +load work properly again.
2022-06-30 20:48:24 +02:00
fang
292a65f64c
Merge branch 'next/arvo' into yu/content-dist-merged
Includes patched versions of ames' and clay's +load arms.

In clay, we do a dumb ;; hack to get the state to adapt properly. This
shouldn't be needed ($case had an extra... case added to it, old ones
should still nest), and so we should revisit the logic there to make it
cleaner/better before release.
2022-06-30 16:27:02 +02:00
yosoyubik
20ad05eabc clay: update scry retry/timeout 2022-06-24 09:33:29 +02:00
fang
7a110db60d
clay: render syntax errors at end of file
Previously, if the pointer for a syntax error pointed to the end of the file
(and the file ended in a newline) the code snippet rendering would try to
display a line _beyond_ the end of the file, causing a crash.

Here, we detect that case, and display `<<end of file>>` instead.
2022-05-30 16:08:59 +02:00
Ted Blackman
f159f38f16 ames: (debug build) fix yawn 2022-05-27 15:54:38 -05:00
fang
6eba8053f8
various: further tweaks and assets for testnet 2022-05-24 23:55:32 +02:00
Philip Monk
b914fbaedc clay: add logging commands 2022-05-20 17:01:50 -07:00
fang
a0f7891f50
Merge branch 'philip/tomb' into lf/content-dist-rc
Beware, the ames conflict was particularly nasty. State migrations as of
yet untested.

Includes patched-up fine tests.
2022-05-19 17:07:35 +02:00
Philip Monk
7f63f548f5 clay: don't add lobe to wire to avoid flow leak 2022-05-18 20:54:10 -07:00
fang
c1ce8bcec0
clay: don't clear busy flag unnecessarily
Not every +foreign-update call means we got a backfill response. Only
+take-backfill does, and that already clears the busy flag.
2022-05-18 16:17:19 +02:00
fang
055f31dada
clay: set saner scry timeout & retry timers 2022-05-18 15:24:02 +02:00
fang
2e6a7cfec8
clay: track pending (scry) requests better
Instead of having a separate "busy" flag and pending scry request state,
we now have a single "busy" unit that, if there is a pending request,
contains details about the pending request. In the ames case, this is
simply %ames. But in the scry case, it contains all the details we need
to cancel the request, timer, or both when needed.

Additionally, we now make sure to always cancel the scry timeout timer
whenever we get a scry response.
2022-05-18 15:21:26 +02:00
Philip Monk
201cb7290f clay: add tomb scry 2022-05-18 02:12:35 -07:00
Philip Monk
6cae525464 clay: make trim clear ford cache 2022-05-18 00:35:23 -07:00
Philip Monk
61c1e19db5 clay: progress download when receiving tombstone 2022-05-18 00:11:16 -07:00
Philip Monk
4051d182ee clay: fetch entire latest foreign commit 2022-05-17 22:07:53 -07:00
Philip Monk
9548f8333a clay: don't require head of foreign desks 2022-05-17 17:06:03 -07:00
Philip Monk
9f0e8294db clay: fixes 2022-05-17 16:02:15 -07:00
fang
4848d58410
clay: properly resume work when +retry-with-ames
Previously, when the scry timeout fired and we retried the request with
ames, we would always start over from the very first request (ie,
fetching the ""manifest"" containing the files we need to download).
This behavior is not correct in the case where we had already received
that initial response, and were now fetching individual files. So here,
if that's the case, we simply call into +work after marking this peer as
having known-broken scry, which will resume work as appropriate.

(The actual bug here that broke the fallback behavior entirely was the
busy flag not getting unset. We now clean it up properly.)

We also move setting of the sad timer into +retry-with-ames, instead of
doing it at each individual callsite. (In fact, one of the callsites was
missing this behavior.)
2022-05-18 00:30:31 +02:00
Philip Monk
c440a333d3 clay: tweaks from walkthrough 2022-05-17 14:17:53 -07:00
Philip Monk
9c7c776fd8 clay: fix stack while building directories 2022-05-15 02:27:49 -07:00
Philip Monk
d25194c7a3 arvo: reduce priority of lifecycle printfs 2022-05-14 23:59:21 -07:00
Philip Monk
c091ec92eb various: make all files build correctly 2022-05-14 22:49:10 -07:00
fang
0251b127e1
clay: request files over scry by tako, not aeon 2022-05-12 17:14:55 +02:00
fang
70aeb2a7c1
clay: treat %d scry on empty desk specially
Similar to how %x on the empty desk was already treated specially. We
continue supporting %d on non-empty desks, but do add a print marking it
as deprecated.
2022-05-12 00:41:58 +02:00
fang
9286fc93b1
clay: give tako 0v0 for aeon 0
The previous commit already started treating the 0v0 commit as
equivalent to the 0 aeon (that is, never-real root), so we should give
the 0v0 commit for all 0 aeons.
2022-05-12 00:39:02 +02:00
fang
5549071649
clay: do reads by tako, not by aeon
This corresponds more closely to how clay actually works/stores data.
2022-05-11 20:58:24 +02:00
fang
bb7e74c1e9
arvo: add %uv hash case to scry paths
Clay desparately wants something like this. It helps answer questions
around hash-based content fetching.
2022-05-11 16:16:03 +02:00
Philip Monk
32ff1417a7 clay: only maintain mime cache if mounted 2022-05-10 20:33:01 -07:00
Philip Monk
3b0c2dac9c clay: validate files in separate roads 2022-05-10 20:32:11 -07:00
Philip Monk
7f573cb357 clay: tweak |mass output 2022-05-10 03:24:58 -07:00
fang
80958c70a0
Revert "clay: scry for files by lobe"
This reverts commit d5dca99ba8.
2022-05-09 17:20:02 +02:00
fang
d5dca99ba8
clay: scry for files by lobe
Instead of scrying for revisions of files at paths in desks, we now scry
for data corresponding to a given lobe. This removes concerns of aeons,
paths, and other such frivolities, and lets us ask for the specific data
we need regardless of where it may live.

How we're supposed to answer permissioning questions around data this
way remains to be seen.
2022-05-05 16:59:09 +02:00
fang
8bef14d0a7
Merge branch 'philip/tomb' into lf/content-dist-rc 2022-05-05 00:33:37 +02:00
Philip Monk
591bdf458c Merge branch 'next/arvo' into philip/tomb 2022-05-04 00:45:57 -07:00
Philip Monk
4760e4e158 clay: fix |pick 2022-05-04 00:22:00 -07:00
Philip Monk
c995f04474 clay: comment 2022-05-03 22:09:36 -07:00
Philip Monk
53353cde11 clay: add %sweep scry to verify refcounts 2022-05-03 21:59:56 -07:00
Philip Monk
d59fb3a82b clay: address review comments 2022-05-03 19:17:39 -07:00
Philip Monk
a3a2934d53 clay: add dependencies to stack on sprig hit 2022-05-03 16:47:48 -07:00