Commit Graph

2164 Commits

Author SHA1 Message Date
yosoyubik
dd812283ac ames: optimize memory measurement
Co-authored-by: Joe Bryan <joemfb@gmail.com>
2023-02-09 10:47:49 +01:00
yosoyubik
4bac6ea642 Merge branch 'develop' into ted/ames-clog-less 2023-02-09 10:14:02 +01:00
jose
7de9c45c38
Merge pull request #6129 from urbit/yu/clean-flows
ames: add |close-flows
2023-02-08 16:50:16 +01:00
Josh Lehman
88e853cf69
Merge branch 'develop' into i/6236/clay-cult-inspection 2023-02-08 06:42:32 -08:00
ryjm
a5512fe9bc add bad desk name to all traces 2023-02-08 07:38:44 -05:00
Jake Miller
972588e573
add bad desk name to stack trace
Co-authored-by: fang <github@fang.io>
2023-02-08 07:11:50 -05:00
ryjm
62b6e8c8f8 clay: desk name sanity check
Ensures no desk with an illegal name can be created by sanity checking
in `%fuse`, `%park`, and `%merge`.
2023-02-07 21:24:33 -05:00
fang
c5aece980e
Merge pull request #6116 from urbit/fix-host-scry
eyre: fix %host scry so it says secure even if no domains configured
2023-02-07 16:28:47 +01:00
fang
92066f3a33
Merge branch 'develop' into m/http-auth-tokens 2023-02-06 19:44:01 +01:00
yosoyubik
46c27d33cd ames: fix check for stale resubscriptions 2023-02-06 16:48:32 +01:00
yosoyubik
fbb95b0d9f ames: check if we have a flow for a nax ack bone 2023-02-06 16:39:00 +01:00
yosoyubik
1522e4ab96 ames: don't scry dudes for every peer 2023-02-06 16:39:00 +01:00
yosoyubik
108e0cd023 ames: fix syntax error 2023-02-06 16:39:00 +01:00
yosoyubik
5f9d6b23d1 ames: improve logging in dry mode for on-kroc 2023-02-06 16:39:00 +01:00
yosoyubik
24140d5f12 ames: clarify forward/backward bones 2023-02-06 16:38:59 +01:00
yosoyubik
e32c18b677 ames: style fixes and comments 2023-02-06 16:38:59 +01:00
yosoyubik
1d8214530a ames: fix |close-flows generator 2023-02-06 16:38:50 +01:00
yosoyubik
1e04e9498e ames: clean all stale %watches in on-kroc
This removes the logic from cleaning up stale subscriptions in %gall,
leaving +ap-rake as it was, and moves it to the +on-kroc arm in %ames.

Failed subscriptions from nacking a %watch plea that were
not properly corked (fixed in https://github.com/urbit/urbit/pull/6102)
are a subset of the more general "stale re-subscription" issue, so
we take care of all stale flows at the same time, by focusing on the
current  subscription—leaving all others to be corked automatically—and
checking if it received a nack, to subsequently cork it.
2023-02-06 16:38:04 +01:00
yosoyubik
ad712caccf ames, gall: move |close-flows logic to vanes
This modifies the %rake task in %gall, to select what kind of
subscriptions we try to close:

=mode %o: kill old pre-nonce subscriptions
=mode %z: kill old pre-nonce subscriptions, including sub-nonce = 0
=mode %r: kills all stale resubscription flows

It also adds a dry-run option to both tasks (%kroc in ames, %rake in gall)
2023-02-06 16:36:32 +01:00
yosoyubik
872db0efcc ames, +close-flows: combine stale flows
This combines two types of flows that can be closed:

- nacking %watch pleas
- resubscriptions that were not properly corked
2023-02-06 16:36:31 +01:00
yosoyubik
7d4959db2e ames: don't send %kroc cards in |close-flows 2023-02-06 16:36:30 +01:00
yosoyubik
7c26716bab ames: remove stale comment 2023-02-06 16:36:29 +01:00
yosoyubik
05582d12c2 ames: refactor +on-krock 2023-02-06 16:36:28 +01:00
yosoyubik
3a7bd3a67f ames: add tool for removing failed %watch flows 2023-02-06 16:36:27 +01:00
Ted Blackman
2bae8127bf
Merge branch 'develop' into i/6236/clay-cult-inspection 2023-02-03 15:31:08 -06:00
Ted Blackman
09a0da4c84 clay: refuse %cs subs scry unless at now 2023-02-03 12:33:27 -06:00
bacwyls
4c2e5460bd
clay: revise /subs %cs scry again
return an empty set instead of crashing if desk doesn't exist
2023-02-03 11:46:54 -06:00
Josh Lehman
3ec1172c5b
Merge branch 'develop' into fix-host-scry 2023-02-03 07:50:27 -08:00
bacwyls
3c7ff0c1f2
Merge branch 'develop' into bwyl/clay-subs 2023-02-03 09:47:04 -06:00
bacwyls
f75e52b9ff clay: revise scry for subscribers to a desk
Address PR6136 comments to improve the interface to this scry.
Now it looks like .^((set ship) %cs /=landscape=/subs)
  instead of .^((set ship) %cs %/subs/landscape)
2023-02-02 17:20:21 -06:00
tadad
ae6187dd06
Merge branch 'develop' into da/flow-scry 2023-02-01 15:35:15 -06:00
Philip Monk
fa451613ec
Merge pull request #6246 from urbit/i/6245/is-ok
ames: put "is ok"/"not responding" behind |ames-verb
2023-02-01 14:07:38 -07:00
fang
faa16a1811
eyre: provide proper secure flag in %host scry 2023-01-27 14:37:56 +01:00
fang
eab40abcc9
Merge branch 'develop' into fix-host-scry 2023-01-27 11:54:28 +01:00
tadad
a5e0356fd6
Merge branch 'develop' into da/flow-scry 2023-01-26 17:10:11 -06:00
dachus
9e9fdeb2c6 flow scry added, moved to lull 2023-01-26 17:04:08 -06:00
Philip Monk
8609599035 ames: put "is ok"/"not responding" behind |ames-verb
Fixes #6245
2023-01-24 13:07:19 -07:00
fang
c61a9f8bd7
clay: update %prep task for pages instead of blobs 2023-01-23 21:10:28 +01:00
fang
903351f86f
pill: support including clay blob store
During pill and install prop generation.

In autopill, we stop tracking a copy of the blob store, instead just
having the pill lib scry it out on-demand.
2023-01-23 20:42:13 +01:00
fang
f2d3a64d9c
clay: add /cs/===/bloc scry for all desk blobs
Lets you retrieve all blobs from the blob store that are in use by the
desk, both presently and in all its history.
2023-01-23 20:39:18 +01:00
fang
3fa5856f94
clay: add %prep task for priming the blob store 2023-01-23 20:36:34 +01:00
fang
b3b5400cec
boot: partial prop support
Doesn't yet support pre-userspace props, and the code might need to be
shuffled around a bit, but what's here works already.
2023-01-23 20:34:56 +01:00
fang
50cfdb0f2a
clay: allow scrying out of pending requests
Implements a /cx/[our]//[now]/cult/[desk] endpoint, for getting a set of
pending requests for any given desk. We don't give the $cult for the
desk as-is, but instead slim the $roves back down into $raves, remove
clay protocol version metadata, and make sure to put our @p in place of
empty "for" fields.
2023-01-22 21:42:57 +01:00
Philip Monk
9d56637916 clay: Drop foreign %wris responses
This flow is not supported, and it was causing issues releasing
416.  This change just drops the responses to avoid crashing, but at
some point we should either support this flow or reject the request in
the first place.
2023-01-18 17:28:51 -07:00
Philip Monk
8a013d147d Merge remote-tracking branch 'origin/master' into next/arvo 2023-01-18 16:13:44 -07:00
Philip Monk
d9e5ebe806
Merge pull request #6117 from urbit/lf/lure
hoon: add +lure, for turning a singleton into a list
2023-01-17 15:22:58 -07:00
Philip Monk
77616520c3 ames: add logging for snubbing 2023-01-17 15:16:35 -07:00
Philip Monk
ecdd16dcf3 Merge remote-tracking branch 'origin/next/arvo' into lf/ames-snub 2023-01-17 15:06:11 -07:00
Philip Monk
899712383a
Merge pull request #6160 from urbit/jon/remove-plat
hoon: remove $plat
2023-01-17 15:00:27 -07:00
fang
0556e30e08
dill: clean up old-style userspace subscriptions
As of version %5, dill uses a new wire format for its userspace
subscriptions. Its existing subscriptions (read: the one subscription
into %hood for the default session) was never updated to use this new
style.

We observed a bug on one ship, where it had both old-style and new-style
subscriptions into hood, resulting in output being rendered twice. How
exactly this happened remains as of yet unclear.

Here, we forcefully clean up the old-style subscription, and
(re)establish the equivalent new-style subscription. This will prevent
issues like this from reoccurring.
2023-01-17 19:07:57 +01:00
Philip Monk
86cf63986b Merge remote-tracking branch 'origin/jon/bartar-crusade' into next/415 2023-01-13 17:58:12 -07:00
Philip Monk
0791c455a6 hoon: put doccords behind parser flag 2023-01-13 17:41:07 -07:00
Philip Monk
40451fa727 Revert "Revert "Merge pull request #6039 from tadad/da/apt-in""
This reverts commit e306d32c74.
2023-01-13 16:13:03 -07:00
Philip Monk
a7a3790346 kelvin: bump to 139 2023-01-13 16:12:35 -07:00
Philip Monk
0119eef28d Merge branch 'next/arvo' into jon/doccords 2023-01-12 13:14:06 -07:00
Philip Monk
e306d32c74 Revert "Merge pull request #6039 from tadad/da/apt-in"
This reverts commit 31bb93846c, reversing
changes made to 7940dd442b.

Reverting because we can't upgrade jetted code without ensuring the jets
change in lockstep.
2023-01-11 15:12:11 -07:00
Philip Monk
7bbcceebaf hoon: typo 2023-01-11 14:47:13 -07:00
Philip Monk
f36591493c kelvin: landscape is only compatible with 416 2023-01-11 14:44:55 -07:00
fang
b78902c30e
Merge branch 'next/dill' into next/arvo 2023-01-10 20:22:29 +01:00
fang
5b160f0c74
zuse: bump kelvin to 416
And lull to 327.
2023-01-06 23:31:16 +01:00
Philip Monk
9031128759
Merge pull request #6047 from bacwyls/team_patch
zuse: change team:title to return false for moons
2023-01-05 15:23:03 -07:00
fang
db3fcdf4d5
Merge branch 'next/arvo' into fix-host-scry 2023-01-05 13:35:22 +01:00
drbeefsupreme
5b99f74c94 hoon: remove deprecated lustar parser
See #6052. This is completely different from the +* used at the top
of doors, and has almost entirely been replaced by |$. The exception is
the use of the `%made` spec, not present in `|$`. I do not see an
obvious way to change `|$` to use `%made` since this `+*` parser uses
the name of the arm in the `%made` structure, unless we change the
AST of |$.
2023-01-04 18:33:47 -05:00
drbeefsupreme
d5bf903f56 Merge branch 'next/arvo' into jon/doccords 2023-01-04 10:22:17 -05:00
fang
a15111a35c
Merge branch 'next/arvo' into m/drum-sessions 2023-01-03 17:19:32 +01:00
drbeefsupreme
d7edbd63da hoon: doccordsify hoon.hoon comments
making hoon.hoon more legible to doccords. also moving some things
around that seemed to be in the wrong place
2022-12-16 15:47:09 -05:00
drbeefsupreme
3da062594e hoon: sweeten =+ foo |@ ++ $ bar -- patterns
|*  foo  bar is sugar for =+  foo  |@  ++  $  bar  --, and newbies find
the old style confusing. this switches out the |@ pattern for the |*
one, at least in layer <=4. the only ones remaining are +toad, +rune,
and +runo, which are already tweaked in #5873 so we omit them here.
2022-12-16 12:46:13 -05:00
Zach Alberico
34e8ff7616
Merge branch 'next/arvo' into ted/ames-clog-less 2022-12-15 19:17:45 -08:00
Zach Alberico
a2173f1eb3
Merge branch 'next/arvo' into lf/ames-snub 2022-12-15 17:40:04 -08:00
Liam Fitzgerald
14baf6f3d6 ames: add %snub functionality
Adds .snub to ames-state, a global blocklist for ships. If a packet is
received from a ship that is in the .snub set, it is immediately
dropped. Adds %snub to ames' $task, to allow manipulating this list
2022-12-15 13:37:14 -06:00
drbeefsupreme
e04262ed2d hoon: swap ?. for ?: on +loaf and +loan
no idea how this ended up happening, but apparently it was my fault.
2022-12-15 14:24:38 -05:00
drbeefsupreme
517945c582 hoon: remove $plat
$plat was a type of $plot added in
bc6ad0456d

plots are no longer a thing, so
neither should be plats.
2022-12-14 20:43:39 -05:00
drbeefsupreme
44aa90c9ad hoon: remove +hint:ax
all this did was set .nut. while it could be used with doccords, it is
currently unused, and none of the other values in the sample of _ax are
set this way (bug, def, cox, hay, dom). i experimented a little bit with
trying to make use of this but it made things overall more unreadable,
and it wouldn't make sense to do it without doing the same for other
values of the sample. im guessing this is just an old style.
2022-12-14 20:27:08 -05:00
drbeefsupreme
68c2ab5e25 hoon: fix .nut comment
notes aren't just for doccords, of course
2022-12-14 16:25:04 -05:00
drbeefsupreme
25dba7e672 hoon: remove extra hoon casts in +decorate
everything appears to work fine without them and I don't think it is any
less unclear what this gate is doing.
2022-12-14 16:02:41 -05:00
fang
94dd77bb0e
jael: also send %ruin breaches to catch-all subs
Previously, fake breaches triggered by a %ruin task would only get sent to
subscribers watching for the affected ship specifically. Now, we send them to
both those subscribers, and the ones watching for pubkey changes on all ships.
2022-12-13 23:52:10 +01:00
bacwyls
94519cbfbf clay: add scry for subscribers of a desk
This data is valuable for app developers and %base OTA providers
2022-12-07 17:16:11 -06:00
drbeefsupreme
d508c8c507 hoon: add comment about .nut in +ax 2022-12-07 16:17:57 -05:00
drbeefsupreme
627827f029 hoon: revise leap:docs and apse:docs
when +apse sees a link, it presume that the following a batch comment,
and stops parsing so that it can be picked up by apex:docs next

this required a change to +leap, which has been rewritten to pretty much
look like +gap but stop parsing when encountering doccords.
2022-12-06 14:30:31 -05:00
drbeefsupreme
b76134fb1d hoon: attach all untagged arm docs
previously we just threw them out and wasn't sure whether it was the
right answer. this violates the principle of least surprise - even
though it hard to see the value of attaching multiple empty $cuff notes
to an arm, we shouldn't stop the programmer from doing it without any
indication or explanation as to why. its the behaviour you'd expect
given how doccords is structured.
2022-12-06 13:16:45 -05:00
drbeefsupreme
75cf230d54 hoon: revise doccord intermediate parse structures
it is desirable for both apex:docs and apse:docs to parse into an
intermediate representation that never ends up in an AST so that it is
clear that these parsed representations may be altered in the future
without worrying about old types nesting with new types. this was
already the case for $whit, but apse:docs parsed directly as a $help,
which is used in ASTs. so apse:docs now parses as a $whiz, which is
simply a cord. in the future, if postfix comments are used for something
like invariants, or allow $links, we may want to change this.

this also changes $whit to remove .use, which was unused. similarly,
+glom is removed since its not used anywhere.
2022-12-06 12:12:45 -05:00
drbeefsupreme
3b6f0f5808 hoon: remove comment on batch arm docs above chap
this might actually be undesirable, don't want to leave this as a trap
for somebody in the future thinking we knew it was definitely the right
answer. having batch comments follow the chapter declaration does make a
certain amount of sense, stylistically
2022-12-05 18:13:42 -05:00
drbeefsupreme
ce209fc2c7 hoon: %gist spec doccords are tagged
future-proofing %gist specs by putting a %help tag on the $help. this
looks pointless at first glance, but it allows the opportunity for %gist
specs to have a $% in the future in a way such that the old type nests
with the new one, eliding the need for a typo->type migration
2022-12-05 18:03:22 -05:00
drbeefsupreme
4afdfdb689 hoon: remove unused $links
done for future-proofing purposes - don't want to presume how they will
get used in the future
2022-12-05 18:00:06 -05:00
drbeefsupreme
0ed484a6ef hoon: +seam and +scye doccords edit 2022-12-05 17:36:38 -05:00
drbeefsupreme
3c323783ca hoon: make some comments into doccords 2022-12-05 17:14:51 -05:00
drbeefsupreme
baa48948cf hoon: rune parser whitespace changes for docs
makes =; etc not eat doccords
2022-12-05 17:13:38 -05:00
drbeefsupreme
ca3140a8b3 hoon: doccord parsing fixes
addressing @Fang- comments on the parser
2022-12-05 17:13:03 -05:00
Zach Alberico
c84fc5d5d0
Merge branch 'next/arvo' into lf/lure 2022-12-02 11:21:48 -08:00
fang
f8e297c3fc
Merge pull request #6106 from rabsef-bicrym/rb/fix-comment-of-core-in-arvo
Fix Comment: `arvo` comment for `|of` reads `|de`
2022-12-01 23:51:11 +01:00
rabsef bicrym
93ef3e8043 arvo: fix |of door comment. 2022-12-01 13:27:36 -08:00
Liam Fitzgerald
02944af669 hoon: add +lure, for turning a singleton into a list 2022-12-01 15:47:04 +10:00
Tinnus Napbus
3f60cd9f90 eyre: fix %host scry so it still says secure if localhost 2022-12-01 14:17:23 +13:00
drbeefsupreme
f519b4caeb hoon: fix sig rune whitespace to allow doccords
without this, e.g.

~&  %foo
::    dox
1

wouldn't attach dox to 1 as a doccord
2022-11-30 13:39:50 -05:00
Philip Monk
c23fec3e70 Merge remote-tracking branch 'origin/next/417' into next/arvo 2022-11-28 18:32:05 -07:00
Philip Monk
4e95a45dab clay: don't clear ford cache on %trim
Most of the memory stays in gall anyway, and this means you need to
recompile everything the next time anything changes, which could be
counterproductive.  It's important that %trim not make things worse.

The functionality is moved to the debug %stir task.
2022-11-28 16:17:41 -07:00
Philip Monk
cb0bbf1e32 clay: don't eagerly build marks
Building the marks adds 50% to the memory requirements for reloading
clay on ~wicdev, from 320MB to 470MB.  This isn't worth it for now.
2022-11-28 16:11:34 -07:00
Philip Monk
38a26041c2 clay: don't add road transitions in clay
These were originally added because they reduced memory usage, primarily
by clearing the memoization cache.  Now that the memoization cache is
no longer used, we use less memory without them.  On ~wicdev-wisryt with
~30 apps, updating Clay now takes ~320MB.
2022-11-28 16:08:41 -07:00
Ted Blackman
35194a6a08
Merge pull request #6102 from urbit/yu/kill-nack-flows
ames, gall: send a %cork if getting a %nack for a %watch
2022-11-28 11:57:07 -06:00
Ted Blackman
da4046a859
Merge pull request #6076 from urbit/yu/remove-num-live
ames: remove num-live from pump-metrics
2022-11-28 11:50:01 -06:00