Commit Graph

750 Commits

Author SHA1 Message Date
Joe Bryan
aa200cf25a
Merge pull request #3786 from urbit/jb/ames-peek-forward-lane
ames: updates forward-lane scry-handling to prevent routing loops
2020-10-26 11:56:23 -07:00
fang
b92cfdb242
eyre: produce moves in correct order
Because storing in reverse order means producing in reverse reverse
order.

The tests didn't catch this because they, too, were infected with the
"reverse moves" meme.
2020-10-24 01:48:51 +02:00
Joe Bryan
da928bed10 ames: updates forward-lane scry-handling to match +send-blob 2020-10-22 17:37:46 -07:00
fang
5bebff3c38
eyre: kick subscriptions if fact conversion fails
This matches the behavior that gall uses for %watch-as subscriptions,
which eyre was using previously.
2020-10-19 19:17:58 +02:00
fang
1d4ee5a7b5
eyre: kick busy subscriptions if client not acking
In order to curb event queue growth when a client for whatever reason
isn't acking the events we send out, we implement a mechanism for
detecting such "clogging", and proactively kick subscriptions which are
adding too many events to the queue.

If the client hasn't sent an ack for ~s30, any subscription that accrues
more than 50 unacked %facts gets closed to prevent further buildup.

Upon reconnecting, the client will see %kick for the relevant
subscriptions and can open a new subscription as appropriate.

Includes a simple test for this behavior, and updates /app/dbug to be
able to display the newly tracked statistics.
2020-10-19 15:56:05 +02:00
fang
63b4fb3e19
eyre: simplify channel subscription storage
Instead of storing by "channel wire", store by request-id instead.
The channel wire was just the channel-id, request-id, and some cruft.
2020-10-19 00:48:18 +02:00
fang
7e5f29cfd2
eyre: convert facts to json manually
By doing a %watch instead of %watch-as %json for channel subscriptions,
we can hopefully make better use of noun deduplication, when storing
events in a channel's event queue until they get acked.
2020-10-18 16:31:35 +02:00
fang
8def1dbea8
eyre: store unacked events unserialized
Store the gall events from channel subscriptions as (vaseless) signs,
instead of serialized events. This should be smaller in memory, and
makes it more likely for noun deduplication to happen.

The cost is needing to reserialize upon channel reconnect, but this is
the less common case, and we don't expect it to be particularly slow.
2020-10-18 16:04:19 +02:00
Philip Monk
8cd72daf20
clay: speed up +find-merge-points
In certain cases +find-merge-points was very slow.  Specifically, the
`done` set was meant to avoid checking the same commit repeatedly, but
it didn't catch the case where a commit was added to the worklist that
was already in that worklist.

Secondly, the worklist was stored as a list but used as a queue, which
resulted in a lot of unnecessary welding.  We change it to a qeu.

Fixes #3735
2020-10-16 11:21:02 -07:00
Joe Bryan
45038ad7b1
Merge pull request #3689 from urbit/jb/clay-cancel-correct
clay: refactors +cancel-request, consistently accounting for TMI
2020-10-14 22:56:49 -07:00
Joe Bryan
1402f763f3 clay: refactors +cancel-request, consistently accounting for TMI 2020-10-09 10:34:18 -07:00
Fang
c444806c3d
eyre: explicitly permit proposed request in cors
Instead of always providing a wildcard for the allowed methods and
headers, now echoes back the method and headers that the client asked
for, if any.

Fixes #3676.
2020-10-09 14:07:05 +02:00
fang
a1e43e02a0
eyre: prevent binding in reserved namespaces
Disallows registering bindings (through %connect and %serve) that would capture
traffic on paths starting with /~ (Eyre's) or /~_~ (runtime's, as of cc389c5).

Note that we don't touch +insert-binding, which is used by Eyre internally to
set up bindings in its own namespace.
2020-10-06 16:50:49 +02:00
Joe Bryan
a208387a72
Merge pull request #3520 from urbit/ted/ames-scry-print
ames: print failed scrys
2020-10-02 15:30:27 -07:00
fang
be1f4a5f6b
eyre: add authentication checker scry endpoint
Lets you check whether a specific Cookie header value string constitutes an
authenticated request.

/ex/=//=/authenticated/cookie/(scot %t 'cookie-string')

Intended for use in the runtime, for example with #3557.
2020-10-01 19:55:16 +02:00
fang
a37b728b86
Merge pull request #3565 from tylershuster/eyre-response-code
eyre: send 'no content' status with no content
2020-10-01 16:06:44 +02:00
Fang
0866d99c73
eyre: minimal CORS support
Adds a cors-registry to Eyre's state that tracks allowed and rejected
origins for the purposes of CORS request handling.

For preflight requests, generates a response in-line.
For simple requests, adds CORS headers onto whatever response is given.

See also:
https://groups.google.com/a/urbit.org/g/dev/c/bb82dwEJGzM/m/q2JjNSx5BwAJ
2020-09-30 15:54:15 +02:00
Tyler Brown Cifu Shuster
8a9534e0ac eyre: send 'no content' headers with no content 2020-09-24 21:07:07 -07:00
pkova
5902ef9a53 eyre: remove superfluous connection-state check
This was originally introduced by me in #1814 to address #1811. Eyre was not
canceling heartbeat timers on all relevant events making it easy to end
up with an infinite behn loop. This check allowed ships that entered an infinite
loop to recover, as per my comment at
https://github.com/urbit/urbit/pull/1814#discussion_r333477482. Otherwise it's
not necessary.
2020-09-24 17:50:42 +03:00
Ted Blackman
f2be97d315 ames: print failed scrys 2020-09-18 16:06:13 -04:00
Joe Bryan
31fe240632 arvo: adds |meld, triggering memory unification 2020-09-09 22:50:43 -07:00
Joe Bryan
7f0b500778 arvo: adds |trim, triggering memory reclamation 2020-09-09 22:49:37 -07:00
Philip Monk
f9665c0cbd
Merge pull request #3378 from urbit/philip/base-hash
clay: expose mergebase as base-hash
2020-09-08 20:11:05 -07:00
Philip Monk
c7480c9b07
clay: fix bug in %mate
If both sides changed a file in the same way, %mate used the version in
the mergebase, which is incorrect.  This changes it to use the version
in the destination desk.

An example of this issue:

    > +cat %/test/hoon
    /~zod/home/~2020.9.3..21.41.24..61ed/test/hoon
    first

    > |merge %scratch our %home
    >=
    merged with strategy %fine
    + /~zod/scratch/2/test/hoon

    > +cat /=scratch=/test
    /~zod/scratch/~2020.9.3..21.41.32..408c/test/hoon
    first

    > *%/test/hoon 'second'
    : /~zod/home/3/test/hoon

    > *%%%/scratch=/test/hoon 'second'
    : /~zod/scratch/3/test/hoon

    > |merge %scratch our %home
    >=
    %fine merge failed, trying %meet
    %meet merge failed, trying %mate
    merged with strategy %mate
    : /~zod/scratch/4/test/hoon

    > +cat /=scratch=/test
    /~zod/scratch/~2020.9.3..21.42.25..9e8b/test/hoon
    first
2020-09-03 14:34:30 -07:00
Philip Monk
67c5caada8
clay: add safe merge options
As described in gen/hood/merge/help.txt
2020-08-28 16:36:00 -07:00
Fang
2385670631
Merge pull request #3387 from urbit/m/eyre-wack-gall
eyre: prevent %eyre-no-channel errors
2020-08-27 22:26:19 +02:00
Fang
b06f8a0f9e
eyre: send %leave on %fact/ack for deleted channel
Ordinarily, eyre cleans up the relevant gall subscriptions whenever a
channel disappears. In yet unresolved erroneous behavior though, it may
leave a gall subscription open, despite wiping the channel from state.

Attempting to pass the response onto the deleted channel anyway results
in an %eyre-no-channel error later in the event. The volume of these
errors can degrade the user experience, as per #3196.

To resolve the annoyance (but not the underlying issue) we detect the
"subscription has no channel" case, and issue a %leave. Doing so
requires additional information in the wire, so we add that in,
refactoring the relevant wire building along the way.

Note that due to the wire requirements, this cannot resolve existing
cases. For that, we depend on bc929ba6d.
2020-08-27 14:16:46 +02:00
Fang
bc929ba6db
gall: forget eyre subscriptions once
As part of the solution to #3196, we need to clean up any gall
subscriptions that eyre didn't properly clean up.

Since detecting that is hard, we opt to just wipe _all_ eyre-originating
subscriptions from gall. We inspect the duct, which isn't good, but it's
only just this once.
2020-08-27 13:23:36 +02:00
Philip Monk
769b43d73a
clay: add comments and cleanup +find-merge-points 2020-08-26 18:32:20 -07:00
Philip Monk
9cd826f8f9
clay: make +find-merge-points faster
The main thing here is that we aggressively check whether we're in
ancestry of another mergebase candidate.  This means we don't have to do
a 2nd pass to eliminate redundant candidates.
2020-08-25 22:54:48 -07:00
Philip Monk
4468d79509
clay: expose mergebase as base-hash
Change the definition of base-hash to be the mergebase of %home with the
OTA source.  This means it's the most recent successfully-applied
update, which is usually the most important information.

Add sour-hash, which is the hash of the most recently *downloaded*
update, regardless of whether it applied successfuly (ie the old
base-hash).

Add a summary of the various hashes at the top of gen/trouble.
2020-08-25 20:25:39 -07:00
Philip Monk
f606d2e3d6
clay: cleaner no-op detection
Only no-op if the incoming commit's parent is the old head of the desk.
Also move the printing near the end so we can know exactly if anything
changed.
2020-08-25 18:15:06 -07:00
Philip Monk
d5596eb1a3
clay: if commit would be a no-op, don't commit it
fixes #3317
2020-08-25 16:24:36 -07:00
Caio Marcelo de Oliveira Filho
92cf9d7c33 Merge branch 'master' into code-reset 2020-08-17 14:55:07 -07:00
Caio Marcelo de Oliveira Filho
72b164ade2 eyre: Make %code-changed handling less chatty 2020-07-31 13:01:19 -07:00
Caio Marcelo de Oliveira Filho
df868e2c4f jael, eyre: Add a way to reset the web login code
Jael now stores a `step` that is combined with the original salt to
produce a new code.  A `%step` card is used to increment that value,
and effectively resetting the keys.  Because the first `step` is zero,
the first code is the same as before.

Eyre was changed to be notified with `%code-changed` so it can forget
old cookies, sessions and discard all the existing channels.

A new generator was added |code, that does both querying and
resetting the code

    |code             :: shows current code, step and help
    |code %reset      :: changes the code

The old +code generator still works correctly.
2020-07-31 11:35:48 -07:00
Philip Monk
48745e945a
Merge branch 'philip/ames-lane' (#3186)
* origin/philip/ames-lane:
  ames: fix lane discovery during some lane changes

Signed-off-by: Philip Monk <phil@pcmonk.me>
2020-07-30 13:33:29 -07:00
Philip Monk
3d1b7f2b35
clay: fix ph tests 2020-07-28 21:44:45 -07:00
Philip Monk
495a6cf004
clay: remove debugging printfs 2020-07-24 22:05:49 -07:00
Philip Monk
1d0409c4c4
ames: fix lane discovery during some lane changes
We used to not accept new indirect lanes if we already have a direct
lane.  This means that if Bob, with a publicly-accessible lane, changes
lanes (eg by restarting the process and getting a new port or changing
ip addresses), tries to talk to Alice, who is behind a NAT, then Bob
will try directly but fail (because Alice is behind a NAT), so he will
route the message through her galaxy.  This is good -- the message gets
to Alice.  However, Alice had a direct route to Bob's old lane, so she
will try to ack on that lane, which fails.  She will not time out this
lane because she doesn't know that Bob isn't getting the acks (acks
don't have their own acks).

The solution is that if Alice receives an indirect lane for Bob when she
already has a direct lane, she shouldn't ignore it.  If the lane is the
same as what she has, she shouldn't change anything (in particular, she
shouldn't mark it as indirect).  But if it's a new lane, she should
discard her old direct lane and use the new indirect lane.
2020-07-24 20:41:48 -07:00
Philip Monk
51983e5480
clay: rewrite new protocol
Use an explicit state machine to carefully manage pending downloads.
2020-07-24 00:20:22 -07:00
Philip Monk
553a9db843
clay: refactor over-the-wire protocol
Don't send blobs initially; instead, wait for the recipient to ask for
any they need.  This should significantly reduce network and memory usage.
2020-07-23 00:12:33 -07:00
Matilde Park
0d68f3f901 Merge branch 'release/next-sys' 2020-07-22 14:31:17 -04:00
Fang
3955d1092b
ames: add scry endpoint for forward lanes
Finds the lane for that peer, or their galaxy. Intended for use in the
runtime, to enable stateless forwarding.
2020-07-22 16:33:34 +02:00
Fang
4ea98e569f
ames: improve scry interface
No longer abuse the desk field, instead making use of the path. Reject
any scries outside of the local ship, empty desk and current time as
invalid.
2020-07-22 16:30:47 +02:00
Philip Monk
753ccf1b7e
Merge remote-tracking branch 'origin/ted/fix-huck' into release/next-sys 2020-07-21 17:08:01 -07:00
Fang
3eab480de7
Merge branch 'release/next-sys' into jb/m/behn-scry 2020-07-22 02:02:05 +02:00
Philip Monk
1169d6b443
clay: ignore extraneous /sys/lyv results 2020-07-21 15:42:46 -07:00
Philip Monk
f4c34c7598
release: urbit-os-v1.0.29 2020-07-21 12:59:12 -07:00
matildepark
2ea019850e
Merge pull request #3105 from urbit/mp/destub-ver
eyre: remove stubbed version text
2020-07-09 16:53:12 -04:00