Commit Graph

376 Commits

Author SHA1 Message Date
Philip Monk
6215137ed5
Merge remote-tracking branch 'origin/ted/ordered-map-fix' into release/next-sys 2021-03-03 18:20:33 -08:00
Philip Monk
6e3d1680a7
Merge remote-tracking branch 'origin/master' into release/next-sys 2021-03-03 17:54:57 -08:00
Philip Monk
413413b70c
zuse: traverse always delete 2021-02-26 21:03:20 -08:00
Philip Monk
5ac70ec974
ames: reset num-live to actual number of live packets 2021-02-26 20:53:27 -08:00
Ted Blackman
04ea4c78b2
ames: +on-hear-shut remove dead code 2021-02-26 16:55:26 +10:00
Ted Blackman
ccea39f865
ames: fix comet handshake 2021-02-26 16:55:24 +10:00
Ted Blackman
eb56fbd3f4 ames: shorten error printing; update pill 2021-02-19 11:54:34 -05:00
Ted Blackman
473a520b27 ames: more informative printing 2021-02-18 21:47:02 -05:00
Ted Blackman
6234b56ac1 ames: cache message +jam in +on-memo 2021-01-20 15:29:45 -05:00
Philip Monk
e5d4230f6b
ames: print on crashing %done
This is guaranteed to produce a stuck flow if the error isn't transient,
so this will be good info to get.
2020-12-11 16:49:55 -08:00
Philip Monk
175ce28bb7
ames: tag verb printfs 2020-12-11 16:45:13 -08:00
Philip Monk
595efabf6c
ames: recover from %strange-current 2020-12-11 16:43:13 -08:00
Joe Bryan
2b8d83e255 ames: remove "larva: drain" printf (too noisy) 2020-12-08 03:32:29 -08:00
Joe Bryan
aaf506ce8a Merge branch 'jb/motion' into jb/motion-merge
* jb/motion:
  pill: solid
  zuse: remove %crud from vane-task
  arvo: full vane names in $sign
  aqua: build again (still broken)
  arvo: reform of the scry reform
2020-12-07 18:52:51 -08:00
Joe Bryan
5dd3e148b8 zuse: remove %crud from vane-task 2020-12-07 18:01:48 -08:00
fang
d98c59e73a
zuse: lift moves out of +able 2020-12-08 01:47:06 +01:00
Joe Bryan
17736990ed arvo: full vane names in $sign 2020-12-07 16:30:22 -08:00
Joe Bryan
98f2b8c6b3 arvo: reform of the scry reform 2020-12-07 13:52:12 -08:00
Joe Bryan
5a82f1d4c0 Merge branch 'na-release/candidate' into jb/motion
* na-release/candidate: (36 commits)
  ames: don't crash on forward-lane scry
  jael: cleanup per review
  jael: don't signal breach if first hearing of ship
  pill: update
  vere: compile
  lens: don't de-base64
  vere: make import flow not use base64 encoding
  version: strip trailing newline
  pill: mainnet
  Revert "zuse: change to ropsten"
  chat-hook: fix another poke-import issue
  chat-hook: fix poke-import issue
  apps: reorder maps, sets, and jugs
  apps: reorder maps, sets, and jugs
  kiln: add base-hash and our scries
  lens: fix change lost in merge
  graph-store: leave try-rejoin wire when subscription succeeds
  graphs: import-export
  chat: import/export
  invites: import/export
  ...
2020-12-07 00:27:58 -08:00
Joe Bryan
ff06ccd2b9 arvo: completes scry reform 2020-12-06 03:33:43 -08:00
Joe Bryan
5bc6d38437 arvo: updates vane interface 2020-12-06 03:33:43 -08:00
Philip Monk
4b8be294b2
ames: don't crash on forward-lane scry 2020-12-05 20:02:31 -08:00
Joe Bryan
958bb5acf7 arvo: removes +is and all references to it 2020-12-05 00:51:49 -08:00
Joe Bryan
5111831a03 ames: use +rep to assemble message fragments 2020-12-03 14:26:55 -08:00
Joe Bryan
9bb790e3e9 ames: updates +end, +lsh, and +rsh call sites 2020-12-03 14:25:31 -08:00
Ted Blackman
c7b8ffbf4e ames: flat packet format 2020-12-01 09:51:14 -05:00
Joe Bryan
072e2c8101 Merge branch 'm/vane-unversion' into m/modern-hoon
* m/vane-unversion:
  ames: remove old state versions & conversions
  gall: rename fading styles
  vanes: remove old load types and logic
  gall: refresh clay subscription on fade
  gall: add nonce to agent wires
  gall: add %doze-style fade
  gall: fade initial commit
2020-11-30 21:04:16 -08:00
fang
122123c11a
ames: remove old state versions & conversions 2020-11-27 13:55:21 +01:00
Joe Bryan
f95e1f48c2 arvo: scry reform 2020-11-25 16:02:10 -08:00
fang
7956adedee
vanes: remove old load types and logic
Maintains state version numbers to avoid tripping up migration tools.
2020-11-25 16:24:16 +01:00
Ted Blackman
56db7cdcda ames/zuse: require >64-bit comet addresses 2020-10-27 17:28:07 -04:00
Ted Blackman
fcf9cfbcdb ames/zuse: require >64-bit comet addresses 2020-10-27 17:20:25 -04:00
Joe Bryan
da928bed10 ames: updates forward-lane scry-handling to match +send-blob 2020-10-22 17:37:46 -07:00
Ted Blackman
f2be97d315 ames: print failed scrys 2020-09-18 16:06:13 -04:00
Philip Monk
1d0409c4c4
ames: fix lane discovery during some lane changes
We used to not accept new indirect lanes if we already have a direct
lane.  This means that if Bob, with a publicly-accessible lane, changes
lanes (eg by restarting the process and getting a new port or changing
ip addresses), tries to talk to Alice, who is behind a NAT, then Bob
will try directly but fail (because Alice is behind a NAT), so he will
route the message through her galaxy.  This is good -- the message gets
to Alice.  However, Alice had a direct route to Bob's old lane, so she
will try to ack on that lane, which fails.  She will not time out this
lane because she doesn't know that Bob isn't getting the acks (acks
don't have their own acks).

The solution is that if Alice receives an indirect lane for Bob when she
already has a direct lane, she shouldn't ignore it.  If the lane is the
same as what she has, she shouldn't change anything (in particular, she
shouldn't mark it as indirect).  But if it's a new lane, she should
discard her old direct lane and use the new indirect lane.
2020-07-24 20:41:48 -07:00
Fang
3955d1092b
ames: add scry endpoint for forward lanes
Finds the lane for that peer, or their galaxy. Intended for use in the
runtime, to enable stateless forwarding.
2020-07-22 16:33:34 +02:00
Fang
4ea98e569f
ames: improve scry interface
No longer abuse the desk field, instead making use of the path. Reject
any scries outside of the local ship, empty desk and current time as
invalid.
2020-07-22 16:30:47 +02:00
Ted Blackman
ef138bb007 ames: use +cut in +encrypt 2020-06-25 22:54:37 -04:00
Philip Monk
05f0e47f68
ames: fix larval +load for fusion 2020-06-11 22:55:08 -07:00
Philip Monk
8b78f04dd3
Merge remote-tracking branch 'origin/master' into ford-fusion 2020-06-02 21:50:20 -07:00
Philip Monk
818d2231f6
ames: remove jumbo packets 2020-06-02 14:13:28 -07:00
Philip Monk
18e396cf53
release: urbit-os-v1.0.20
Previous urbit-os-v1.0.20 was not released
2020-05-28 18:43:20 -07:00
Philip Monk
c20cd29932
jael: look for correct wire
We inspect the wire of our subscriber to see if we need to produce the
result as a %public-keys or a %boon.  This is bad -- we should proxy the
subscription to avoid this need, but this doesn't make that change yet.

%pubs is an old name that doesn't exist anymore (last existed around
September 2019).  The new version is /public-keys, but it's worked so
far because /public-keys has only one item in the path, so it missed the
conditional.  This commit makes the intent more clear.

The [%a @ @ *] could be just [%a @ *], but I leave it to reduce the
chance of breaking stuff.
2020-05-28 14:27:08 -07:00
Philip Monk
0642cc5861
ames: add |ames-wake
Somehow we ended up with flows which expected to awaken but did not wake
up.  This was likely caused by the error in r920j OTA, urbit-os-v1.0.18.

This adds a command which ensures that every flow has an active timer.
I expect this to be needed only once, but it's a pretty general tool, so
it's worth keeping.

I've included an unused @t parameter to more easily add simple debug
commands to ames without having to add a new task
2020-05-28 10:28:31 -07:00
Philip Monk
ff01a1ea04
ames: don't set new timer if we woke up too early
Otherwise if we end up having multple outstanding timers, they never
coalesce to a single timer.
2020-05-27 17:38:18 -07:00
Fang
58e85fa661
Merge branch 'release/next-sys' into m/debug-dashboard 2020-05-23 01:00:45 +02:00
Philip Monk
a4a44f1e21
kiln: cleanup 2020-05-22 14:32:32 -07:00
Philip Monk
4fcd9f23c0
Merge branch 'release/behn-fast' (#2915)
* release/behn-fast:
  zuse: add +ordered-map
  behn: switch to ordered-map
  chat: virtualize message-to-json conversion
2020-05-21 22:43:53 -07:00
Philip Monk
274215ec6b
zuse: add +ordered-map
At some point this should be more properly styled similar to +by, +in,
and +to, but for now this reduces duplication and makes the ordered map
available to everyone.
2020-05-21 22:28:16 -07:00
Fang
4c9d18e5c0
Merge branch 'release/next-sys' into m/debug-dashboard 2020-05-21 20:09:58 +02:00
Philip Monk
8c4c0f31fc
Merge pull request #2920 from urbit/ted/ames-mass-alien
ames: fix %alien |mass over-reporting
2020-05-21 01:40:30 -07:00
Ted Blackman
a0eafa8aa3 zuse,ames: handle %trim task 2020-05-21 03:45:08 -04:00
Ted Blackman
0f7b213f38 ames: fix %alien |mass over-reporting 2020-05-21 03:19:20 -04:00
Ted Blackman
7e69e66280 ames/zuse: handle %trim event 2020-05-20 03:29:54 -04:00
Fang
be087a8c92
Merge branch 'release/next-sys' of https://github.com/urbit/urbit into m/debug-dashboard 2020-05-11 20:39:00 +02:00
Fang
14831f4864
ames: expose state through scries
Support /=peers= and /=peer=/~ship scries for getting at all peers and
a specific peer's connection state, respectively.

Moves some internal types into zuse for easier external use.
2020-05-08 01:37:07 +02:00
Ted Blackman
a064afbd89 /sys: |mass works again 2020-05-07 04:51:08 -04:00
Philip Monk
4cc3fea534
clay: don't send peers to message pump 2020-05-05 16:39:50 -07:00
Philip Monk
490b615b6a
ames: only dedupe long messages 2020-05-04 21:39:59 -07:00
Philip Monk
081c8a864d
ames: don't split messages until ready to send
Trying to reduce the size of ames queues.  This deduplicates incoming
message-blobs by comparing with existing message-blobs in other queues.

It also stops splitting into fragments in +feed-packets.  Instead, it
splits into fragments at the last moment, in +encrypt.  This means we
don't have to store a large number of packets in our home road.
2020-05-02 03:35:44 -07:00
Philip Monk
c50c34d8be
ames: dedup new messages and fragments 2020-05-01 22:55:14 -07:00
Philip Monk
3b9f7249d3
ames: print num-fragments in packet verbage
Gives you a poor man's progress bar.  For example, to determine how much
of an OTA you've downloaded from your sponsor, run:

|ames-sift (sein:title our now our)
|ames-verb %rcv

and then to turn it off:

|ames-verb
2020-03-17 14:03:48 -07:00
Joe Bryan
6322639436 vane: jet-hints all vanes for profiling 2020-03-03 14:17:42 -08:00
Joe Bryan
53d9798cda vane: prints error notifications where not handled 2020-02-26 16:56:17 -08:00
Joe Bryan
93eaff7401 ames: downcast %hear error notification to %hole 2020-02-26 16:56:17 -08:00
Joe Bryan
4cae84d9ac vane: downcast all error notifications to %crud 2020-02-26 16:56:17 -08:00
Joe Bryan
df970ed417 arvo: passes errors to all vanes 2020-02-26 16:56:17 -08:00
Philip Monk
41fd367bff
ames: make routing simpler 2020-02-10 17:49:18 -08:00
Ted Blackman
0bee77ce8e
/sys: use +harden on vane tasks
Uses Zuse's previously unused +harden helper function to streamline
+task unwrapping in vanes.

(Arguably, in landlocked vanes like Ford, we should crash if we get a
%soft task, since no events should be coming in directly from the
outside.)
2020-01-27 09:53:53 +04:00
Jared Tobin
c182672b54
Merge branch 'ames-goof' (#2166)
* origin/ames-goof:
  ames: adjust route update logic

Signed-off-by: Jared Tobin <jared@tlon.io>
2020-01-22 13:14:39 +04:00
Ted Blackman
11c92e691d
ames: adjust route update logic
There was a typo in the routing logic that was comparing equality
against a value where it should have been doing a pattern match. The
value compared against contained the literal * gate, which would never
match route.peer-state, so this condition was always true, meaning the
fix that had added this extra condition (5406f06) did not actually
change the behavior from what it been previously.
2020-01-22 12:50:18 +04:00
Philip Monk
d578159791
ames: fix assertion bug and add debug info
If we receive the naxplanation before the nack, the assertion in the gte
direction fails.  The intent of the assertion is to make sure top of the
live queue never falls behind current.state, so it was simply in the
wrong direction.
2020-01-14 08:34:12 -08:00
Ted Blackman
895f1c069d ames: update comment docs 2019-12-21 01:56:51 -05:00
Jared Tobin
9b0582323c
Merge branch 'philip/eth-watcher' (#2113)
* philip/eth-watcher:
  ph: fix tests by spamming blocks regularly
  gaze: reflect changes to eth-watcher
  ames: better printfs
  jael: only advance lifes
  jael: stop ship-to-ship
  jael: add "eager" mode to avoid hitting nodes as much
  jael: properly store ship sources
  gen: add +azimuth-sources
  jael: re-enable ship-to-ship communication
  eth-watcher: actually stop pending thread when restarting

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-18 12:13:27 -03:30
Philip Monk
0e876b3cd4
ames: better printfs 2019-12-18 11:31:17 -03:30
Philip Monk
7ca3d9624e
ames: handle misordered crashing boons
Two bugs fixed here: first, if the %done reentrancy triggered another
%boon, that wasn't getting translated to a %lost, even though it could
have been the reason the event crashed in the first place.

Second, the %done reentrancy needs to happen after we emit our move, so
that we don't invert the order of the %boon's we produce.
2019-12-17 20:58:30 -08:00
Jared Tobin
9ba4505086
Merge branch 'ames-sift' (#2081)
* ames-sift:
  ames: refactor +load
  ames: +send-blob better ship printing
  hood: |ames-sift generator to trace by ship
  ames: add %sift  to trace by ship

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-12 16:06:32 +08:00
Ted Blackman
35596ca7de
ames: refactor +load 2019-12-12 15:55:37 +08:00
Ted Blackman
d4574b5da4
ames: +send-blob better ship printing 2019-12-12 15:55:36 +08:00
Ted Blackman
d77fb0f685
ames: add %sift to trace by ship 2019-12-12 15:55:32 +08:00
Jared Tobin
85d447f173
Merge branch 'philip/gall-noop' (#2073)
* origin/philip/gall-noop:
  gall: no-op on duplicate watch-ack

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-12 15:50:19 +08:00
Jared Tobin
2aa86e3121
Merge branch 'philip/stuck-flow' (#2071)
* origin/philip/stuck-flow:
  ames: recover from mismatched message nums

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-12 15:49:53 +08:00
Philip Monk
3b41a8be15
gall: no-op on duplicate watch-ack
fixes #2070
2019-12-10 18:49:50 -08:00
Philip Monk
29f078bb14
ames: don't forward up the sponsorship chain
This is *actually* why the galaxies are under so much load.  They're in
a forwarding loop with their stars, and this breaks the loop.
2019-12-10 16:20:12 -08:00
Philip Monk
e7c8a44e11
ames: recover from mismatched message nums
We've seen issues where the message-num of the head of live.state is
less than current.state.  When this happens, we continually try to
resend message n-1, but we throw away any acknowledgment for n-1 because
current.state is already n.  This halts progress on that flow.

We don't know what causes us to get in this bad state, so this adds an
assert to the packet pump that we're in a good state, run every time
the packet pump is run.  When this crashes, we can turn on |ames-verb
and hopefully identify the cause.

This also adds logic to +on-wake in the packet pump to not try to resend
any messages that have already been acknowledged.  This is just to
rescue ships that currently have these stuck flows.

(Incidentally, I'd love to have a rr-style debugger for stuff like this.
Just run a command that says "replay my event log watching for this
specific condition and then stop and let me poke around".)
2019-12-09 23:31:18 -08:00
Philip Monk
abde1d8aa9
ames: reduce load by increasing timer delays 2019-12-06 12:11:06 -08:00
Ted Blackman
bee0b5803a
ames: don't crash on missing queued larval event 2019-12-05 17:04:24 +08:00
Jared Tobin
41b64feb16
Merge branch 'philip/p2p' (#2025)
* philip/p2p:
  ames: don't overwrite lane if already direct

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-05 16:08:01 +08:00
Philip Monk
5406f06092
ames: don't overwrite lane if already direct
This is why basically all packets are going through the galaxies right
now.  Most of the time, the flow right now is:

* talking to ~dopzod but don't know where it is, so ask ~zod to forward,
  which it does

* ~dopzod responds both directly (on the origin lane) and through ~zod

* (if NAT, the direct response doesn't get back, but the one through
  ~zod does. Then you respond directly to ~dopzod because their lane
  piggybacked on the response. ~dopzod responds both directly and
  through ~zod, and the story picks up the same as if you weren't behind a
  NAT)

* now you have a direct lane to ~dopzod, so all is well.

* now the duplicate response from ~dopzod through ~zod comes in (takes a
  little longer because it's bouncing off ~zod), resetting your lane to
  "provisional"

* since your lane is provisional, you send your next packet both
  directly and through ~zod

* GOTO 2

This change says "if I already have a direct lane, don't overwrite it
with a provisional one". This way, the only way the direct lane can be
overwritten is if they stop responding on it (cleared on "not
responding; still trying").

I also added |- to +send-blob to make |ames-verb %rot less confusing.
2019-12-05 16:05:06 +08:00
Jared Tobin
75ca54ca24
Merge branch 'ames-sponsor-scry-2' (#2021)
* ames-sponsor-scry-2:
  ames: scry for sponsor and don't crash on jael response

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-05 15:43:00 +08:00
Ted Blackman
a7e638ebab ames: scry for sponsor and don't crash on jael response 2019-12-04 17:18:39 -05:00
Ted Blackman
b3f757d88b
ames: send larval crashes to dill 2019-12-05 02:23:13 +08:00
Ted Blackman
4c9cc1542a
ames: dequeue failed larval timer 2019-12-05 02:23:13 +08:00
Ted Blackman
c20f2391e1
ames: print and retry larval crashes 2019-12-05 02:22:27 +08:00
Philip Monk
9bc6ccb7fc
ames: don't say not responding if we haven't been talking 2019-12-03 20:21:43 -08:00
Philip Monk
702dd2c07a
verb: add +verb %bowl to print bowl on every event 2019-12-03 15:05:42 -08:00
Philip Monk
8c2c52c01c
ames: make life printf helpful 2019-12-03 13:06:04 -08:00
Philip Monk
5f1c4805fe
ames: printfs 2019-12-02 23:13:48 -08:00
Philip Monk
c90107659b
Merge remote-tracking branch 'origin/rc-ames-verb' into rc 2019-12-02 20:22:04 -08:00
Philip Monk
93d3edbf73
pill 2019-12-02 20:09:36 -08:00
Ted Blackman
d0d45ed8f2
ames: fix message pump to complete queueing fix 2019-12-02 20:09:35 -08:00
Ted Blackman
6dcb6622fa
ames: fix ack queueing 2019-12-02 20:09:35 -08:00
Ted Blackman
0cb6464e9d ames: %spew to set verbosity 2019-12-02 18:46:40 -05:00
Philip Monk
e3005eaffa
ames: clear out-of-order messages from packet queue 2019-12-02 00:41:50 -08:00
Philip Monk
aaf7b3b42e
ames: don't crash in +on-take-wake 2019-12-01 16:00:32 -08:00
Ted Blackman
0a8b12c882 ames: state adapter 2019-12-01 02:49:46 -05:00
Ted Blackman
900d923ccc ames: fix aggressive lane timeout (still needs migration) 2019-12-01 02:49:46 -05:00
Philip Monk
d6c1ff4e20
ames: add routing diagnostics 2019-11-30 14:44:57 -08:00
Ted Blackman
071b1a4bbe ames: ~s30 max timeout instead of ~m2 2019-11-28 01:17:34 -05:00
Ted Blackman
93604c2f29 Merge branch 'rc' of github.com:urbit/urbit into rc 2019-11-27 23:06:53 -05:00
Ted Blackman
3779cca5a9 ames: try sponsors above .our 2019-11-27 23:06:39 -05:00
Philip Monk
23cc21c383
ames: remove printf 2019-11-27 19:39:12 -08:00
Ted Blackman
e9ba500ee4 Merge branch 'rc' of github.com:urbit/urbit into rc 2019-11-27 22:31:18 -05:00
Philip Monk
26c5be2948
ames: remove printf 2019-11-27 18:40:33 -08:00
Ted Blackman
9af7b3954a ames: ignore encrypted packets from alien comets 2019-11-27 20:58:18 -05:00
Philip Monk
138cbb5d2e
ames: clean up printfs 2019-11-27 16:58:26 -08:00
Philip Monk
fdb1069b33
ames: printfs 2019-11-27 16:43:09 -08:00
Philip Monk
fc74ab2dbd
ames: count unsent messages for backpressure 2019-11-27 15:58:38 -08:00
Philip Monk
74b0f66850
ames: continue processing memos after %done 2019-11-27 15:13:17 -08:00
Philip Monk
f035955a36
ames: rename alef -> ames 2019-11-27 00:46:02 -08:00
Ted Blackman
043f508f27 deleted old ames 2019-07-22 19:15:16 -07:00
Joe Bryan
dc2483f1f8 fixes jet-registration hint in %ames +turf-scry (for profiling) 2019-07-11 23:41:54 -07:00
Fang
12b8134c33
Merge branch 'v0.8.0rc' into gut-by 2019-07-10 01:49:07 +02:00
Joe Bryan
47aaef7904 disables spurious condition in %ames ping flow 2019-07-02 18:08:06 -07:00
Fang
eb6c8a45ce
Replace (fall (~(get by calls with (~(gut by 2019-06-30 18:13:34 +02:00
Jared Tobin
b3901ab42f Add 'pkg/arvo/' from commit 'c20e2a185f131ff3f5d3961829bd7a3fe0f227f8'
git-subtree-dir: pkg/arvo
git-subtree-mainline: 9c8f40bf6c
git-subtree-split: c20e2a185f
2019-06-28 12:48:05 +08:00