Commit Graph

19298 Commits

Author SHA1 Message Date
Fang
57a0b6f654
vere: add ames counters for forwarded packets
Track the amount of packets we forward, and the amount we drop due to
forward queue pressure. Once every 1000 packets, printf the total.
2020-07-08 02:04:55 +02:00
Fang
c58ac8e6f4
vere: clean up packets forwarded to galaxies
In the forward-to-galaxy case, we don't go through _ames_lane_scry_cb,
instead calling _ames_forward directly, meaning _ames_panc_free wouldn't
get called on those packets.

Now, we move the _ames_panc_free call into _ames_forward, and only keep
it in the "scry failed" section of _ames_lane_scry_cb.

By moving that call into the
2020-07-08 01:31:41 +02:00
Fang
1eab1660a7
vere: limit ames.c forwarding queue to 1k
Keeps a counter, increments and decrements when starting and completing
forwarding logic respectively. Once the counter hits 1000, further
packets are dropped instead of forwarded.

Ideally you want to drop the oldest packets first, but this'd imply
removing scry events from the event log, which ames.c shouldn't know
how to do. Perhaps once u3_lord_peek_cancel or similar gets implemented,
we can do this sanely. Until then, this isn't the end of the world.
2020-07-08 01:15:29 +02:00
Fang
b864d30685
vere: use pier data directly in ames.c
Instead of copying it into the local struct.

Arguably the port should still be present in the ames struct, since it's
written to. There's a comment for removing it from the _pier_ struct
though, which seems like the better change, but out of scope for here.
2020-07-06 22:48:02 +02:00
Fang
95d90a362e
vere: initial stateless forwarding in ames.c
When receiving a packet for which we are not the recipient, attempt to
forward it statelessly. This means scrying a lane for the recipient out
of ames, then (if necessary) updating the origin lane in the packet and
sending the updates packet over the lane resulting from the scry.

Remaining work listed among the TODOs:
- implement a cap on the amount of pending to-forward packets we hold,
- handle version mismatches in the sponsee case,
- track counters around filtering & forwarding packets,
- test for memory leaks more aggressively.
2020-07-03 01:14:03 +02:00
Fang
70bdc4a675
vere: account for scry callback changes 2020-07-03 01:08:10 +02:00
Fang
92beeca9f0
ames: add scry endpoint for forward lane
Finds the lane for that peer, or their galaxy. Intended for use in the
runtime, to enable stateless forwarding.
2020-07-03 00:51:54 +02:00
Fang
31d776c50b
arvo: allow the empty desk (%$) in scries
+slaw fails to parse empty strings as %tas, so we special-case it here.
2020-07-01 19:54:33 +02:00
Fang
1e1bfb81b0
Merge branch 'ipc-redux' into m/stateless-forward 2020-07-01 17:08:36 +02:00
Joe Bryan
64a9e80cf3 Merge branch 'release/next-vere' into ipc-redux
* release/next-vere:
2020-06-29 01:48:27 -07:00
Joe Bryan
04353f95cf Merge branch 'master' into release/next-vere
* master:
  vere: bump version to 0.10.7
  libsigsegv: disable stack vma check
  vere: bump version to 0.10.6
  ci: add travis as trusted user
  jets: use appropriate macro
  noun: add -C to control memo cache size
  jets: restore fond/play/peek hooks
  jam: add commented-out functionality to count size of atom
  jets: cap memo cache and remove peek, play, and fond jets
  noun: add functions to count size of noun
2020-06-29 01:44:42 -07:00
Joe Bryan
6e8a914791 vere: refactor %behn's retries of failed events 2020-06-29 01:41:00 -07:00
Joe Bryan
a5a148e66d vere: remove stale function declarations 2020-06-27 22:32:52 -07:00
Joe Bryan
3969e78612 vere: retry behn timer failures 2020-06-27 01:15:32 -07:00
Joe Bryan
49444e86c5 vere: refactors ovum and callbacks to support retry 2020-06-27 01:13:06 -07:00
Joe Bryan
b6a0487d7a vere: corrects all uses of [vod_p] to [ptr_v] 2020-06-27 01:04:03 -07:00
Joe Bryan
3965b261d1 vere: refactors ovum scheduling api 2020-06-27 00:59:31 -07:00
Joe Bryan
b4ff9888b5 vere: refactors %ames packet failure callback 2020-06-27 00:50:11 -07:00
Joe Bryan
7c245aa85c vere: retry %behn %born event failures 2020-06-27 00:49:43 -07:00
Fang
b3cb7453cc vere: rename behn.c's alm -> alm_o
To adhere to the naming conventions.
2020-06-27 00:33:40 -07:00
Joe Bryan
a53f688d01 vere/serf/king: support timeouts and error response in %peek 2020-06-27 00:12:53 -07:00
Joe Bryan
14faa22e0f nix: add explicit shutdown to solid-pill derivation 2020-06-26 22:34:20 -07:00
Joe Bryan
1d7c361b06 vere: adds/improves error messages for term syscall retry loops 2020-06-26 16:24:43 -07:00
Joe Bryan
47796f812e pill: solid 2020-06-26 16:24:43 -07:00
Joe Bryan
7f61b5a491 Revert "Merge pull request #3050 from urbit/m/behn-improvements"
This reverts commit ed808614aa, reversing
changes made to a6db3add83.
2020-06-26 16:24:43 -07:00
Joe Bryan
6fa194d527 Merge branch 'master' into jb/ipc-redux
* master: (147 commits)
  vere: bump version to 0.10.7
  libsigsegv: disable stack vma check
  vere: bump version to 0.10.6
  ci: add travis as trusted user
  jets: use appropriate macro
  noun: add -C to control memo cache size
  jets: restore fond/play/peek hooks
  jam: add commented-out functionality to count size of atom
  jets: cap memo cache and remove peek, play, and fond jets
  noun: add functions to count size of noun
  release: urbit-os-v1.0.23
  interface/config: fix production build
  soto: run +on-load migration once
  publish, links: restore full height
  sh/build-interface: amend for SPA
  interface/CONTRIBUTING: amend for SPA / webpack
  solid: update pill
  hood + apps: fix OTA process for feat/SPA
  hood: add version %6 for %file-server upgrade
  chat: equally size both code + s3 buttons
  ...
2020-06-26 13:33:00 -07:00
Joe Bryan
db9472dbe2 vere/king: tweaks serf startup args (unconditionally send all) 2020-06-26 12:29:18 -07:00
Joe Bryan
ac42ce09c5 Merge branch 'ipc-redux' into jb/ipc-redux
* ipc-redux:
  behn: optimize bounded timers scry
  vere: support saving scry jam to directory
  vere: u3_nul in place of c3__null
  vere: if behn scry fails, don't try again
  vere: rename behn.c's alm -> alm_o
  vere: scry out next behn timer for backstop
  vere: warn on invalid behn doze
  behn: improve scry interface
  arvo: allow the empty desk (%$) in scries
  vere: add -X flag for running a scry
2020-06-26 11:56:56 -07:00
Fang
02450fe380
Merge pull request #3049 from urbit/m/scry-command
vere: add -X flag for running a scry
2020-06-26 20:47:32 +02:00
Joe Bryan
55dd1e2cf7 vere: fixes re-entrancy bug in term.c write() wrapper 2020-06-26 11:32:04 -07:00
Joe Bryan
6148c87525 vere: restores arbitrary event injection with -I 2020-06-26 11:31:42 -07:00
Fang
5544784591
vere: scry ames protocol version out of arvo
Instead of assuming 0. We still use 0 in the fallback case for now.
2020-06-26 20:27:53 +02:00
Fang
b224ac0878
ames: improve scry interface
No longer abuse the desk field, instead making use of the path. Reject
any scries outside of the local ship, empty desk and current time as
invalid.
2020-06-26 20:27:51 +02:00
Joe Bryan
49bea37049 vere: adds pending ipc message to lord status printfs 2020-06-26 11:20:08 -07:00
Philip Monk
271abcd3e7
vere: bump version to 0.10.7 2020-06-26 11:17:02 -07:00
Philip Monk
c6d536a676
Merge branch 'philip/linux' (#3054)
* origin/philip/linux:
  libsigsegv: disable stack vma check

Signed-off-by: Philip Monk <phil@pcmonk.me>
2020-06-26 11:15:31 -07:00
Philip Monk
2713083386
libsigsegv: disable stack vma check
This patches libsigsegv to not check the stack vma on Linux, since that
involves reading procfs, and we make very heavy use of sigsegv.  This
eliminates most of urbit's performance discrepancy between Linux and
MacOS.  These are the benchmarks used; note this is a local MBP vs a
cloud Linux server, and the MBP is almost certainly faster hardware.

We take two benchmarks, one of which decrements 10 million times and the
other simply allocates 125MB of memory.  These are the results:

cpu-heavy ==  =/ n 10.000.000 |-(?~(n n $(n (dec n))))
mem-heavy == =a (bex 1.000.000.008)

macos, cpu-heavy: 6 seconds
macos, mem-heavy: 1 second

linux-before, cpu-heavy: 30 seconds
linux-before, mem-heavy: 160 seconds

linux-after, cpu-heavy 9 seconds
linux-after, mem-heavy 1.3 seconds

This represents a 3x speedup for the cpu-heavy operation and a 120x
speedup for the memory-heavy operation.

This check was used to try to distinguish stack overflow from other
forms of segmentation fault.  In the comments in src/handler-unix.c, it
describes three heuristics it uses, depending on what's available from
the OS.  In the linux-i386 case, all three are availble, so we simply
disable the slow one.  This correctly recognizes stack overflow if you
simply alloca(10000000000).
2020-06-26 11:15:01 -07:00
Joe Bryan
57967b35fb vere: tweaks pier/lord info printfs 2020-06-26 10:25:24 -07:00
Joe Bryan
0f54d25d8e vere: silence ames errors unless -v, prints stats every 1k 2020-06-26 00:54:29 -07:00
Joe Bryan
5a3dbde218 vere: factors common _mcut functions into ward 2020-06-26 00:54:29 -07:00
Joe Bryan
9177b3ea87 vere: initialize king timer early (for reliable early-exit) 2020-06-26 00:54:29 -07:00
Joe Bryan
1762412162 vere: handle partial writes and retry in the terminal 2020-06-26 00:54:29 -07:00
Joe Bryan
48add92ca5 vere: skip SIGINFO on linux (unavailable) 2020-06-26 00:54:29 -07:00
Joe Bryan
6e9f8ec34b king: fixes --serf worker-path cli parser 2020-06-26 00:54:29 -07:00
Joe Bryan
81ff98c5d4 vere/king/serf: adds and enables event timeouts 2020-06-26 00:54:29 -07:00
Joe Bryan
078b274491 vere: completes daemon->king re-renaming 2020-06-26 00:54:29 -07:00
Joe Bryan
b860e8cd63 vere: adds basic info printfs on SIGINFO/SIGUSR1 2020-06-26 00:54:29 -07:00
Joe Bryan
d96a05e387 vere: fix use-after-free on exit in cttp 2020-06-26 00:54:24 -07:00
Fang
ed808614aa
Merge pull request #3050 from urbit/m/behn-improvements
behn, vere: some improvements
2020-06-25 22:38:02 +02:00
Fang
7cc1b4f3ca
behn: optimize bounded timers scry 2020-06-25 19:25:44 +02:00