Commit Graph

455 Commits

Author SHA1 Message Date
yosoyubik
dec8252d4a Merge branch 'develop' into i/6103/ames-refactor 2023-02-23 13:51:35 +01:00
yosoyubik
cecd7b4835 ames: remove unused pump/sink/pack gifts 2023-02-23 13:39:41 +01:00
yosoyubik
3963b0a951 ames: fix test-nack
previously we were acking the nack-trace message (adding a %send move)
before notifying the message-pump with a %near task. Now, due to the
refactoring we invert the order of those moves. This seems safe but will
determine with livenet testing
2023-02-22 14:45:44 +01:00
yosoyubik
458b40318d ames: remove unused message pump timer relay 2023-02-22 11:06:09 +01:00
yosoyubik
8d9e9c20f5 ames: refactor message-sink 2023-02-22 10:50:20 +01:00
yosoyubik
9c8080cb0b ames: move peer-state lookups to helper chapter 2023-02-22 09:11:22 +01:00
yosoyubik
eaae5b1161 ames: nest pump-gauge in packet-pump 2023-02-22 09:06:05 +01:00
yosoyubik
91be2b8bf1 ames: refactor message and packet pumps 2023-02-22 08:50:55 +01:00
yosoyubik
f508a569f8 ames: add core chapters to peer-core 2023-02-21 13:38:03 +01:00
yosoyubik
6d6f1b956f ames: add core chapters to +ev 2023-02-21 12:56:50 +01:00
yosoyubik
f310b5cb3a ames: rename make-peer-core -> pe 2023-02-21 12:46:20 +01:00
yosoyubik
07af14ecc3 ames: rename per-event -> ev 2023-02-21 12:32:33 +01:00
yosoyubik
311fa0efbb ames: move formal interface core to the bottom 2023-02-21 12:22:53 +01:00
dachus
6f5f4e2100 done 2023-02-17 14:02:11 -06:00
yosoyubik
c503647eb3 ames: no-op on %kroc task 2023-02-14 06:07:59 +01:00
yosoyubik
25dc16addb Merge branch 'develop' into i/6065/clean-flows 2023-02-14 05:02:40 +01:00
yosoyubik
b8b7e1951f ames: add %cong task to adjust clog 2023-02-13 12:04:41 +01:00
yosoyubik
c92c2c9279 ames: log stale flows in closing state 2023-02-10 16:19:18 +01:00
yosoyubik
abbd95f2ce ames: fix typo 2023-02-09 13:12:50 +01:00
yosoyubik
dd812283ac ames: optimize memory measurement
Co-authored-by: Joe Bryan <joemfb@gmail.com>
2023-02-09 10:47:49 +01:00
yosoyubik
4bac6ea642 Merge branch 'develop' into ted/ames-clog-less 2023-02-09 10:14:02 +01:00
yosoyubik
46c27d33cd ames: fix check for stale resubscriptions 2023-02-06 16:48:32 +01:00
yosoyubik
fbb95b0d9f ames: check if we have a flow for a nax ack bone 2023-02-06 16:39:00 +01:00
yosoyubik
1522e4ab96 ames: don't scry dudes for every peer 2023-02-06 16:39:00 +01:00
yosoyubik
108e0cd023 ames: fix syntax error 2023-02-06 16:39:00 +01:00
yosoyubik
5f9d6b23d1 ames: improve logging in dry mode for on-kroc 2023-02-06 16:39:00 +01:00
yosoyubik
24140d5f12 ames: clarify forward/backward bones 2023-02-06 16:38:59 +01:00
yosoyubik
e32c18b677 ames: style fixes and comments 2023-02-06 16:38:59 +01:00
yosoyubik
1d8214530a ames: fix |close-flows generator 2023-02-06 16:38:50 +01:00
yosoyubik
1e04e9498e ames: clean all stale %watches in on-kroc
This removes the logic from cleaning up stale subscriptions in %gall,
leaving +ap-rake as it was, and moves it to the +on-kroc arm in %ames.

Failed subscriptions from nacking a %watch plea that were
not properly corked (fixed in https://github.com/urbit/urbit/pull/6102)
are a subset of the more general "stale re-subscription" issue, so
we take care of all stale flows at the same time, by focusing on the
current  subscription—leaving all others to be corked automatically—and
checking if it received a nack, to subsequently cork it.
2023-02-06 16:38:04 +01:00
yosoyubik
ad712caccf ames, gall: move |close-flows logic to vanes
This modifies the %rake task in %gall, to select what kind of
subscriptions we try to close:

=mode %o: kill old pre-nonce subscriptions
=mode %z: kill old pre-nonce subscriptions, including sub-nonce = 0
=mode %r: kills all stale resubscription flows

It also adds a dry-run option to both tasks (%kroc in ames, %rake in gall)
2023-02-06 16:36:32 +01:00
yosoyubik
7d4959db2e ames: don't send %kroc cards in |close-flows 2023-02-06 16:36:30 +01:00
yosoyubik
7c26716bab ames: remove stale comment 2023-02-06 16:36:29 +01:00
yosoyubik
05582d12c2 ames: refactor +on-krock 2023-02-06 16:36:28 +01:00
yosoyubik
3a7bd3a67f ames: add tool for removing failed %watch flows 2023-02-06 16:36:27 +01:00
Philip Monk
8609599035 ames: put "is ok"/"not responding" behind |ames-verb
Fixes #6245
2023-01-24 13:07:19 -07:00
Philip Monk
77616520c3 ames: add logging for snubbing 2023-01-17 15:16:35 -07:00
Zach Alberico
34e8ff7616
Merge branch 'next/arvo' into ted/ames-clog-less 2022-12-15 19:17:45 -08:00
Zach Alberico
a2173f1eb3
Merge branch 'next/arvo' into lf/ames-snub 2022-12-15 17:40:04 -08:00
Liam Fitzgerald
14baf6f3d6 ames: add %snub functionality
Adds .snub to ames-state, a global blocklist for ships. If a packet is
received from a ship that is in the .snub set, it is immediately
dropped. Adds %snub to ames' $task, to allow manipulating this list
2022-12-15 13:37:14 -06:00
Ted Blackman
35194a6a08
Merge pull request #6102 from urbit/yu/kill-nack-flows
ames, gall: send a %cork if getting a %nack for a %watch
2022-11-28 11:57:07 -06:00
Ted Blackman
da4046a859
Merge pull request #6076 from urbit/yu/remove-num-live
ames: remove num-live from pump-metrics
2022-11-28 11:50:01 -06:00
yosoyubik
a7eee8ae6b Merge branch 'next/arvo' into yu/kill-nack-flows 2022-11-28 17:22:15 +01:00
yosoyubik
403b3eef82 ames, gall: address feedback 2022-11-28 17:21:41 +01:00
fang
3ef5260efc
Merge branch 'next/arvo' into pkova/ruin-comets 2022-11-25 12:20:46 +01:00
Ted Blackman
fc32dde010
Merge pull request #6059 from urbit/ames-verb-fixes
ames: debug print msg formatting fixes
2022-11-23 17:23:16 -06:00
Philip Monk
15c5447a05 Merge branch 'philip/agent-clay' into next/arvo 2022-11-22 17:24:15 -07:00
yosoyubik
430233a3de ames, gall: cork flow if getting %nack for %watch 2022-11-21 14:23:19 +01:00
yosoyubik
012c1e19bf ames: don't use num-live from pump-metrics 2022-11-17 08:38:50 -08:00
yosoyubik
8bb969c2ec Revert "ames: remove num-live from pump-metrics"
This reverts commit b2a0f2fd304f961e4db89499aedefe0be95e6239.
2022-11-17 08:38:50 -08:00
yosoyubik
a3e8607cb1 ames: remove num-live from pump-metrics 2022-11-17 08:38:50 -08:00
yosoyubik
4b0c1fab3b ames: count in flight packets in packet pump
num-live is still used—the state update will be addressed later, as well as the update in %lull
2022-11-17 08:38:50 -08:00
fang
2027907db5
Merge branch 'master' into next/arvo 2022-11-16 16:11:53 +01:00
yosoyubik
3b5110ed1d ames: enqueue %cork in alien-agenda 2022-11-09 16:05:45 +01:00
Tinnus Napbus
d3c2c734e3 ames: debug print msg formatting fixes
- fix `fragment-num` and `num-fragments` having duplicate faces
- fix faces being wrapped around wrong things in various places
- fix `bone` not being printed in "hear last in-progess" message
- make pretty tape interpolation style more uniform
2022-11-07 21:46:15 +13:00
Liam Fitzgerald
26aac7581a ames: prevent crashes when corking an alien flow
The +on-cork handler asserts that the peer is known to us. This is the
incorrect behaviour, because it will crash when corking a flow to a peer
that is still an %alien. This can happen, for instance, when making a
gall subscription for the first time and then corking it before the
alien naturalises.
2022-11-01 15:50:10 +10:00
pkova
708c95b6b2 ames: correct %ruin comet logic 2022-10-17 23:05:49 +03:00
Philip Monk
846dda88b4 Merge remote-tracking branch 'origin/m/ames-no-stale-forwards' into philip/agent-clay 2022-10-13 21:42:58 -07:00
pkova
1d73558163 ames: ruin comets not seen for six months 2022-10-03 17:03:49 +03:00
Joe Bryan
9df08c5e72 ames: ignore pki nponsorship loss 2022-09-22 10:38:39 -04:00
fang
f46d8dd557
Merge pull request #5974 from urbit/m/recork-slower
ames: bump recork timer back up to a day
2022-09-06 22:18:14 +02:00
fang
7a4034f48a
Merge pull request #5977 from urbit/m/cork-undone
ames, gall: no %done on corks
2022-09-06 22:17:34 +02:00
fang
9c931f844b
Merge pull request #5976 from urbit/m/ames-restir
ames: clean & re-set recork timer during %stir
2022-09-06 22:17:18 +02:00
fang
36dd2f8ace
Merge branch 'next/arvo' into m/recork-slower 2022-09-06 22:16:41 +02:00
fang
fec3c45d30
Merge pull request #5971 from urbit/jb/recork-error
ames: handle failed recork timers
2022-09-06 22:15:12 +02:00
fang
0e5b3b4107
ames: don't give %done on cork
Locally, %cork is guaranteed to succeed instantly. The requesting vane
doesn't need an ack to safely consider the flow "closed".
2022-08-25 18:13:29 +02:00
fang
81462da7aa
ames: clean & re-set recork timer during %stir
Pre-release ships may have multiple recork timers. This ensures that,
after stir, we always have exactly one.
2022-08-25 14:42:16 +02:00
fang
0871c24e24
ames: bump recork timer back up to a day
We had lowered this initially for easier testing, but a day should be
sufficient for real-world use.
2022-08-25 14:11:45 +02:00
fang
b2422030f7
ames: correctly identify stale routes 2022-08-25 13:41:55 +02:00
Joe Bryan
76cc8939d3 ames: fix error slog on failed attestation timer 2022-08-23 14:34:08 -04:00
Joe Bryan
5b8f26558e ames: handle failed recork timers 2022-08-23 14:33:53 -04:00
Joe Bryan
679b38138d ames: prevent larval stage from possibly clobbering adult state 2022-08-23 10:37:01 -04:00
Ted Blackman
5a96f3bdbe ames: drop %hear as larva; check duct before metamorpohse 2022-08-23 16:45:17 +03:00
Ted Blackman
07601c6bee ames: metamorphose on first +take if possible 2022-08-23 16:09:35 +03:00
Ted Blackman
9abe49489d ames: +on-stir clean up comments 2022-08-23 13:31:10 +03:00
Joe Bryan
9439841ce8 ames: refactor %stir, %rest stale/wrong timers 2022-08-22 20:14:02 -04:00
fang
3a7938f057
ames: say "metamorphosis" in all relevant cases 2022-08-20 00:05:20 +02:00
fang
53b0e7ecaa
ames: ensure larva initializes adult's state
Previously, when the larva got to processing enqueued events, it was
doing so without loading state into the adult beforehand, resulting in
incorrect processing of events.

Here, we make the larva call +molt more eagerly, ensuring that the adult
always has its state available when we use it.
2022-08-19 23:58:10 +02:00
fang
ce2e433a8a
ames: don't skip closing flows in +on-stir
Yes, there is a global timer for closing flows, but all that does is
enqueue a cork message. +on-stir needs to set _pump_ timers for all
flows that might still have messages to send, which includes closing
flows.
2022-08-19 20:26:08 +02:00
Ted Blackman
6b836bb56d ames: fix no-op for messages on closing bones 2022-08-10 20:18:57 +03:00
Ted Blackman
20ccac22a2 ames: clean up closing bone check and printfs 2022-08-10 19:55:07 +03:00
Ted Blackman
0baad53fb4 ames: drop pleas on dead bones 2022-08-10 19:47:59 +03:00
Ted Blackman
ce9f692ecd ames: clean up %cork plea handling
+on-plea gets called in two very different ways:
1) handling request from local vane to send %plea to peer
2) handling %cork request from another ship, which our local ames has %pass'ed
   to ourselves

In the second case, we shouldn't print misleadingly, or bind a duct in the ossuary.
2022-08-10 00:47:38 +03:00
Ted Blackman
2dc7171dfa ames: no-op on acks on corked bones 2022-08-10 00:44:54 +03:00
Ted Blackman
da591a4078 ames: invert busted conditional in recork timer 2022-08-05 02:37:03 +03:00
Ted Blackman
56d4906a13 ames: try next recork on cork ack
Without this, a ship would send a cork on a max of one flow per
recork timer, which could take years to clear for some ships.
This starts a hot loop of trying the next cork once one gets
positively acked.
2022-08-05 01:35:34 +03:00
Ted Blackman
54cd1a5eca ames: simplify recork timer
The previous recork timer queued up %cork messages without sending them.
It also relied on making sure pump timers didn't get set for recork bones.
This was fragile.

The new design enqueues up to one new %cork message per ship during each
recork timer, based on the state of the flow.  If the flow is closing but
there are no outstanding messages in it, then it needs to be recorked.
Flows will be recorked in ascending numerical order by bone.
2022-08-05 01:13:21 +03:00
fang
9433d3354d
ames: unsent-fragments is a list, not a tree
Measure accordingly.
2022-08-03 21:03:49 +02:00
fang
7af527e605
ames: correctly measure queue size 2022-08-03 20:52:03 +02:00
fang
e6453bae3c
ames: more detailed "future msg" trace
This additional information may come in handy when debugging this crash
we shouldn't be seeing.
2022-08-03 20:46:54 +02:00
fang
e16c499724
ames: make recork timer faster
This will aid visibility and debugging. We'll probably want to make this
slower again later.
2022-08-03 20:33:04 +02:00
fang
7c01ee9b35
ames: account for corked flows during %stir
Also patches the behn scry in %stir handling to be correct for latest behn.
2022-08-03 19:37:53 +02:00
fang
b860bef066
ames: correctly avoid setting pump timer
The condition got butchered during refactor: instead of avoiding the creation
of pump timers during recork wake, it was setting them _exclusively_ during
recork wake.
2022-08-02 23:23:48 +02:00
Ted Blackman
e5e4c8602d ames: properly set recork flag in +on-wake 2022-08-02 23:15:43 +03:00
Ted Blackman
67efb2ad7c ames: don't start pump timers on recork 2022-08-02 23:03:34 +03:00
yosoyubik
a5aedaccc5 ames: delete naxplanation flows if bone is corked 2022-07-18 10:27:36 +02:00
yosoyubik
35c6ac416e ames: fix predicate for detecting %cork acks 2022-07-16 07:40:05 +02:00
yosoyubik
c599e60c5b ames: add recork set 2022-07-15 16:45:32 +02:00
yosoyubik
a3518ef4d6 ames: add trace for %cork plea 2022-07-13 09:17:42 +02:00
yosoyubik
34a7496720 ames: clean up nack flows after a cork 2022-07-08 07:29:04 +02:00
yosoyubik
ca1b586a46 ames: init cork timer on larval +take 2022-07-07 11:35:28 +02:00
yosoyubik
d4d443db13 ames: remove cork wire from state 2022-07-07 09:20:57 +02:00
yosoyubik
e366e55dbc ames: add global daily timer for pending corks
WIP compiles and upgrades, and the timer gets properly reset
but untested for actual re-subscriptions with old publishers
2022-07-06 17:00:35 +02:00
yosoyubik
76919ce07c ames: resend %cork after a naxplanation in one day 2022-07-06 10:52:19 +02:00
yosoyubik
ea4a31558f ames: ack last fragment from closing bone 2022-07-01 09:59:24 +02:00
fang
f2d6104e9c
ames: remove debug printfs
Accidentally included.
2022-06-30 20:38:32 +02:00
yosoyubik
a967b42c30 ames: print bone if "hear last in-progress" 2022-06-30 17:12:07 +02:00
fang
292a65f64c
Merge branch 'next/arvo' into yu/content-dist-merged
Includes patched versions of ames' and clay's +load arms.

In clay, we do a dumb ;; hack to get the state to adapt properly. This
shouldn't be needed ($case had an extra... case added to it, old ones
should still nest), and so we should revisit the logic there to make it
cleaner/better before release.
2022-06-30 16:27:02 +02:00
yosoyubik
b0766c2f53 ames: remove misleading trace printf 2022-06-30 10:46:02 +02:00
yosoyubik
27ac0b601e ames: use %$ as recipient vane for %corks 2022-06-30 10:18:37 +02:00
yosoyubik
17d52b8535 ames: handle bounce %cork from old publisher
Because the publisher will send the cork plea back to the subscriber on
the next bone, we are not able to know the bone for the original cork.
To handle it, we add the cork bone to the plea path

still wip: it keeps resending the cork plea faster than its ~h1 timer
2022-06-28 12:48:07 +02:00
yosoyubik
7de9c72f36 ames: add rift when inserting new peer state 2022-06-27 13:55:07 +02:00
yosoyubik
2e92331a0b Merge branch 'next/arvo' into yu/gall-rq-wire-ames-flow-kill 2022-06-16 08:54:24 +02:00
Ted Blackman
0190a477f5 ames: adjust comments 2022-06-15 15:35:41 -06:00
Ted Blackman
1380e83b55 ames: clog on 5 messages and 100kB 2022-06-15 15:28:34 -06:00
yosoyubik
d7544e9bde ames: fix parsing rift/life in ke-rcv 2022-06-06 15:50:02 +02:00
Ted Blackman
d2ea33ddfe
ames: condense message size calculation
Co-authored-by: fang <git@fang.io>
2022-06-05 08:49:47 -05:00
Ted Blackman
953273004a ames: clog at 100kB instead of 5 messages 2022-06-04 17:18:21 -05:00
Ted Blackman
4825f8968c lull,ames: renames, more packet verification
Regularized arm names to +etch-* and +sift-* to match Vere.  Renamed
$packet to $shot.  Used $meow, $purr, and $keen to match Vere's naming.
Reorganized packet decoding arms and moved some to Lull for later use in
Aqua.
2022-06-03 22:58:50 -05:00
Ted Blackman
3466f9854b ames: fix forwarding, use +send-blob in fine 2022-06-03 18:22:55 -05:00
Ted Blackman
32cc092704 ames: ;; scry result instead of \!< 2022-06-03 11:24:03 -05:00
Ted Blackman
0a3a793161 ames: don't forward to ourselves 2022-06-02 23:43:22 -05:00
Ted Blackman
44954dec7b ames: fix packet forwarding 2022-06-02 23:26:42 -05:00
Ted Blackman
d6428a9023 ames: cleaned up printing, signature bug appears fixed 2022-06-01 17:08:12 -05:00
fang
43dff9c279
ames: do not use old lanes during stateful forward
As 999f0e8, but for the stateful code path in +send-blob.
2022-06-01 21:47:15 +02:00
fang
999f0e84e9
ames: do not produce stale forward lanes
Stale lanes may cause forwarding loops. Imagine the following:
1) Planet A is live. Galaxy B, its indirect sponsor, learns of its route.
2) A goes offline. Another ship, C, is started in its place, at the same route.
3) B receives a packet for A, forwards it to the known route.
4) C received the packet, forwards it to B.
5) Repeat from 3.

Here, we update the forward lane(s) scry used by the runtime to not produce a
peer's lane if they haven't communicated with us in the last hour. Everyone's
supposed to ping their sponsorship chain every 30 seconds. If those aren't
going through, you shouldn't expect to be reachable anyway.

We may or may not want to update +send-blob to match.
2022-06-01 17:17:37 +02:00
Ted Blackman
6a13048cf1 ames: WIP modernized tests pass, still haven't found signature bug 2022-05-31 18:31:38 -05:00
Ted Blackman
930ce64343 ames: fix some meow and mug encoding issues; still signature problems 2022-05-28 20:23:37 -05:00
Ted Blackman
f159f38f16 ames: (debug build) fix yawn 2022-05-27 15:54:38 -05:00
Ted Blackman
062fbc71d7 ames: merge driver rework 2022-05-27 14:52:30 -05:00
Ted Blackman
aaffad63b0 ames: fix up packet encoding 2022-05-27 13:14:53 -05:00
Ted Blackman
a596322202 ames: remote scry works again 2022-05-27 10:35:03 -05:00
fang
984d2845a4
ames: correctly scry into jael for sponsors
+saxo:title uses .^, which doesn't work from kernelspace. We manually
call the scry function instead.

Also, we want the end of the list, not the head.
2022-05-27 13:06:26 +02:00
Ted Blackman
302d406a94 ames: WIP runs and ames works, but scrying kills ships 2022-05-26 21:23:09 -05:00
Ted Blackman
a1f202deb0 fine: refactor driver and arvo/vere interface 2022-05-25 17:30:35 -05:00
fang
830ee80325
ames: fall back to galaxy if no scry lane known
Will need to be updated alongside other routing logic eventually.
2022-05-25 20:58:14 +02:00
Philip Monk
5b773723b3 Merge remote-tracking branch 'origin/yu/ames-fixes' into next/arvo 2022-05-20 22:52:00 -07:00
Philip Monk
ac64f44538 clay: update tests 2022-05-20 20:03:34 -07:00
Philip Monk
f16abbd992 ames: fix comet communication with breached ship 2022-05-19 20:20:00 -07:00
fang
a0f7891f50
Merge branch 'philip/tomb' into lf/content-dist-rc
Beware, the ames conflict was particularly nasty. State migrations as of
yet untested.

Includes patched-up fine tests.
2022-05-19 17:07:35 +02:00
fang
e7f9686b1c
fine: re-enable scry permission check
Only expose fully public clay data over scry. Other vanes and permission
modes to follow at a later date.
2022-05-18 20:25:57 +02:00
yosoyubik
16920a894e Merge branch 'yu/ames-fixes' into yu/gall-rq-wire-ames-flow-kill 2022-05-18 17:37:24 +02:00
yosoyubik
c2576cdc46 ames: handle +call moves if re-evolving from larva 2022-05-18 17:36:40 +02:00
yosoyubik
fe412b1ca7 gall: remove $watches from egg/yoke
Also renames $beat to $boar so it signifies more its closeness to $boat
2022-05-18 15:27:17 +02:00
Philip Monk
7e4ad62a51 Merge remote-tracking branch 'origin/yu/ames-fixes' into philip/tomb 2022-05-17 14:07:05 -07:00
yosoyubik
4dae3ae3e2 ames: indent 2022-05-17 07:49:15 +02:00
yosoyubik
f81061fe40 Merge branch 'yu/ames-fixes' into yu/gall-cork-wip 2022-05-17 07:47:58 +02:00
yosoyubik
2553573be0 ames: start drainage timer if regressed from adult
This accounts for a possible race condition where ames expects a
response, but regresses into the larval state. Upon receiving the
$sign on +take, we would remain stuck as a larva. Now we check
that we have enough information to re-evolve and then start a
/larval timer to begin draining the queue.
2022-05-17 07:11:09 +02:00
fang
9e3bcf14c1
fine: more straightforwardly cancel keen timer
When +ke-abet-gone gets called, we're going to remove this keen
entirely. +ke-set-wake does a whole song-and-dance and may even set a
new timer. So instead, we simply call +ke-rest, if we have a known
outstanding timer.
2022-05-16 22:52:07 +02:00
Philip Monk
d25194c7a3 arvo: reduce priority of lifecycle printfs 2022-05-14 23:59:21 -07:00