RFC2396 defines[1] unreserved characters as alphanumerics and nine "mark"
characters. We were only parsing for four of those, leading to parsing failure
for valid URLs.
[1]: https://tools.ietf.org/html/rfc2396#appendix-A
In Ford Fusion, Clay builds generators but Dojo and Eyre run them. Dojo
is already virtualized with a scry function, so +mule is fine, but Eyre
is not, so Eyre needs to use +mock and explicitly supply the scry
function. This does that. Fortunately, the produced result is simple
and easily clammable.
Fixes#3089
No longer abuse the desk field, instead making use of the path. Reject
any scries outside of the local ship, empty desk and current time as
invalid.
Expose ducts only under a debug endpoint, nothing else should care about
being able to inspect them.
Add scry endpoints for the very next timer (if any), and all timers up
to and including a specified timestamp.
When merging, +reachable-takos is called roughly once per merge commit
in the ancestry of the new commit. +reachable-takos was exponential in
the number of merge commits in the ancestry of the commit it's looking
at, due to mishandling of the accumulator. This makes it linear.
Of course, linear x linear is still quadratic, which is not great. I
doubt +reachable-takos can be made asymptotically better, but
+reduce-merge-points/+find-merge-points probably can. 50 merge commits
already gives about 14.000 iterations through the loop in
+reachable-takos. Another option is to try to memoize this somehow, but
a simple ~+ is insufficient since `s` is usually different.
In local tests on macOS with a -L copy of ~wicdev-wisryt, this speeds up
OTAs significantly. The majority of time was spent on this.
* origin/jb/aes-siv-fix:
tests: updates aes-siv regression test comment
pill: updates solid
zuse: propagates fix to aes-128-siv and aes-192-siv as well
Revert "test: disable aes-siv jets to demonstrate test failure"
pill: updates solid
zuse: fixes bug in aes-256-siv iv calculation (+s2vc:aes:crypto)
test: disable aes-siv jets to demonstrate test failure
test: add test case for aes-256-siv jet mismatch, observed in the wild
Signed-off-by: Philip Monk <phil@pcmonk.me>
Adds +mure to run a trap in a separate road. This should eventually be
just a hint.
Vega was running inside a mule, but since +load was called within vega,
the new kernel was all run within the same mule, so it didn't actually
get to reclaim the space after hoon compiled.
We verified this with printfs in u3m_fall. On the test ship (from
mainnet) which had 800MB used, vega was taking interior free space from
950MB to 450 over the course of compiling hoon, then each vane would go
from about 450 to 350 and then back to 450 once it finished (which
proves they were correctly isolated). With this change, after hoon
compiles the free space goes back up to 950MB. This gives us a lot more
space to compile OTAs.
We had to slightly refactor the logic for doubly-recompiling hoon, since
+mure as written produces a ?(!! _trap), and you can't find faces in the
result of the trap. We could bake mure, but that's rather awkward. I
wonder if there's a way to fix this as a wet gate.
Attempt to convert the scry result to the mark that was asked for,
failing the scry (with ~) if the conversion fails.
Eyre's scry logic, then, can pass the requested mark directly into gall.
Exposes a scry endpoint. Any requests made to the /app/scry.mark url
under the endpoint will scry into %app using a %gx scry, at the
/scry/noun path, and attempt to convert the scry result into the %mark,
before converting that into the %mime mark, and sending that as an http
response.
In addition to producing the action bound for a given request, now also
produces the subset of the request url that comes _after_ the path at
which the binding has been established.
Will allow some bindings to more easily dispatch off the relevant part
of the url.
If we failed the password check, the login page served to us would never
include any redirect details, even if they were there in the original request.
Now we simply (attempt to) parse out the redirect field a little earlier.
Associates channels with the authentication sessions that opened them,
and deletes the channel when its associated session expires.
Also updates the debug dashboard to display channel counts per session.
Turns +on-channel-timeout into +discard-channel, which cleans up the
entirety of the channel, based on its current state. This allows us to
simplify the %delete channel request into a simple function call.
Changes the HTTP status code of the redirect that occurs upon a
successful login from 307 to 303. 307 preserves the method of the
original request, so the redirected request is a POST. With the new SPA,
this causes a 404 as app/file-server validates the method of the
request, something that did not happen in earlier versions of landscape.
303 instead changes the method to always produce a GET request.
Set up, by default, on /~/logout.
Sending a POST request to this expires the current session and redirects
to the login page. If the "all" key is set in the request body, expires
all open sessions.
We build a reef for each desk but use the compiler from our kernel. At
some point we should use the compiler from the desk, but then we need to
validate any results we get from it.
For request transparency, HTTP proxies may set the Forwarded header to
specify who the original requester is.
For requests from localhost only, we make Eyre respect the Forwarded
header, and adjust the handled ip address accordingly.
Note that we do not support X-Forwarded or other non-standard variants.
The header remains in the request, so server applications can handle
them as desired.
Fixes#2723.
When sending a response to an authenticated request, update the session
to last for +session-timeout again, and send an updated cookie to match.
Assuming the user makes an actual HTTP request at least once a week,
this will make sure they don't get logged out automatically. Simply
keeping a channel open, unfortunately, doesn't count.
Instead of setting a timer for every session, we set a single expiry
timer when the first session is created. On the subsequent wake event,
we clear all cookies that have expired at that time, then set a timer
for when the next session expires.
This approach gives us flexibility wrt sessions going forward, allowing
extending or early deleting of sessions without having to care about the
related timers.
Note that in +load, we clear all existing sessions. We would start the
expiry timer flow there, but can't. Forcing the user to login again
post-ota once isn't the end of the world.
We inspect the wire of our subscriber to see if we need to produce the
result as a %public-keys or a %boon. This is bad -- we should proxy the
subscription to avoid this need, but this doesn't make that change yet.
%pubs is an old name that doesn't exist anymore (last existed around
September 2019). The new version is /public-keys, but it's worked so
far because /public-keys has only one item in the path, so it missed the
conditional. This commit makes the intent more clear.
The [%a @ @ *] could be just [%a @ *], but I leave it to reduce the
chance of breaking stuff.
Somehow we ended up with flows which expected to awaken but did not wake
up. This was likely caused by the error in r920j OTA, urbit-os-v1.0.18.
This adds a command which ensures that every flow has an active timer.
I expect this to be needed only once, but it's a pretty general tool, so
it's worth keeping.
I've included an unused @t parameter to more easily add simple debug
commands to ames without having to add a new task
The subscription changes in drum broke existing subscriptions. This
worked alright (though loud) for dojo, but it left chat-cli "frozen"
unless you manually unlinked/relinked. This does that automatically.
It also includes a refactoring of +on-load in drum, to avoid vain
repetition.
We need to get updates directly into %home in case the marks depend on
changes to hoon.hoon. %base has no reason to exist.
Our ota strategy is now to merge from parent/kids to home, then
parent/kids to kids.
* origin/release/link-dojo:
chat-cli: allow sending • character
chat-cli: always talk to local ship only
chat-cli: single-target sole effects as needed
chat-cli: don't allow excessively small cli widths
chat-cli: pull in sole-sur namespace where relevant
chat-cli: remove unused entropy from state
chat-cli: print newlines correctly
chat-cli: support multiple sole connections
chat-cli: don't crash on %bad-text
dojo: rename remote access generators
gall: fix handling of empty path list
dojo: remove unused %json poke
dojo: add remote access controls
drum: switch to per-ship /sole/drum duct
Signed-off-by: Philip Monk <phil@pcmonk.me>
At some point this should be more properly styled similar to +by, +in,
and +to, but for now this reduces duplication and makes the ordered map
available to everyone.
a %hunk is the error-stack frame for a failed ([~ ~]) scry.
this changes the frame type from tank to *, avoiding
coordination overhead between +mink and the interpreter.
Allow one or more whitespace characters before and/or after the equals sign in
name attribute pairs, such as `<hello a = "yo" />` or `<hello a= "yo" />`.
Following the spec at https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-Eq.
* origin/philip/ames-dedup:
clay: don't send peers to message pump
ames: only dedupe long messages
ames: don't split messages until ready to send
ames: dedup new messages and fragments
This will fix the issue described in #2867 for ducts that have already
triggered the bug. This will also send spurious acks for any messages
that are outstanding at the time of the upgrade, but I don't believe
this will cause a serious problem.
Support /=peers= and /=peer=/~ship scries for getting at all peers and
a specific peer's connection state, respectively.
Moves some internal types into zuse for easier external use.
I'm not going to be able to debug the memory leaks in the +ob core
jets, so remove those new jets and hooks to +fein and +fynd so they
can be called from the @p parsing jets. This moves +fein adn +fynd
to the toplevel because there appear to be issues with hooks
referencing things that aren't in the directly jetted core.
Trying to reduce the size of ames queues. This deduplicates incoming
message-blobs by comparing with existing message-blobs in other queues.
It also stops splitting into fragments in +feed-packets. Instead, it
splits into fragments at the last moment, in +encrypt. This means we
don't have to store a large number of packets in our home road.
`at` is for when you expect an array of a certain exact structure. If it
has extra elements, that indicates you were mistaken about the strucutre,
so it should fail to match.
RFC 2396 specifies that segments must be zero or more pchars.[1] We were
deviating from this by requiring at least one pchar per segment.
With this change, we support /some//path, and no longer lose the
trailing slash in /some/path/.
[1]: https://tools.ietf.org/html/rfc2396#section-3.3
-merge will replace |merge so that. Once they reach feature parity and
%info is rewritten to forward to -commit, we can rip out about half of
clay.hoon
Makes it so that |cancel %force skips the next thing in the queue if
you're not in the middle of something. If you are in the middle of
something, it skips the thing you're in the middle of (just like naked
|cancel).
This should resolve issues where |cancel doesn't drain the queue.
Considering some of the options here were atoms, not cells, $% wasn't
appropriate, and led to *etyp:abi:ethereum resulting in ford %ride execution
failure. Simply using $? instead would result in a fish-loop, so here we split
the atom cases from the tagged union ones with a $@.
%park is a plumbing commit task. It guarantees completion in a single
event, so you have to do much of the work before calling it. -commit
is an example of how to do this.
When a ship breaches, we remove all messages that have yet to be
delivered to an app (eg if it's not yet started). We also add
|gall-sear to do this manually, but this shouldn't be needed in normal
operation.
Finally, to unblock ~zod and ~bus on mainnet, we sear one particular
ship automatically on loading hood. It cannot be done manually because
no userpace changes can be made until it's unblocked.
We have three stacks: the hoon stack, bar stack, and duct stack. This
turns the bar stack to a list of ducts and adds it to the hoon stack.
This tells you the ducts of the moves that caused the move where you
crashed.
See:
recover: dig: intr
crud: %belt event failed
bail: intr
bar-stack
~[
~[/g/use/spider/~zod/build/~.dojo_0v5ogno.5anji.vn3f6.4gs7t.6r2ft /d //term/1]
~[/d //term/1]
~[/g/use/spider/~zod/find/~.dojo_0v5ogno.5anji.vn3f6.4gs7t.6r2ft /d //term/1]
~[/g/use/dojo/~zod/out/~zod/spider/drum/wool /d //term/1]
~[/d //term/1]
~[/g/use/dojo/~zod/drum/hand /d //term/1]
~[/g/use/hood/~zod/out/~zod/dojo/drum/phat/~zod/dojo /d //term/1]
~[/d //term/1]
~[//term/1]
]
call: failed
/~zod/home/~2020.3.17..23.14.11..50e0/sys/vane/ford:<[6.128 3].[6.220 5]>
/~zod/home/~2020.3.17..23.14.11..50e0/sys/vane/ford:<[6.129 3].[6.220 5]>
/~zod/home/~2020.3.17..23.14.11..50e0/sys/vane/ford:<[6.132 3].[6.220 5]>
...
Gives you a poor man's progress bar. For example, to determine how much
of an OTA you've downloaded from your sponsor, run:
|ames-sift (sein:title our now our)
|ames-verb %rcv
and then to turn it off:
|ames-verb
This reverts commit 046506f9d4, reversing
changes made to 6ef08962ef.
I'm reverting this as we're moving to a new branch/release model in
which breaching changes (as this one is) will live on a long-running
'next' branch, rather than alongside non-breaching changes in master.
This revert should itself be reverted on the 'next' branch.
* master: (484 commits)
king: Slight CLI cleanup and fix test build.
king: Add command-line flags to configure HTTP and HTTPS ports.
groups: reduce metadata updates, removal
chat: reducer handles metadata removal
groups: exclude group metadata from channels list
groups: set and surface group name metadata
groups: remove dummy 'share' flow, 'default' group
contacts: rename, migrate '~contacts' to '~groups'
sh/release: rename vere release tarballs
vere: patch version bump (v0.10.3 -> v0.10.4.rc1) [ci skip]
pills: updated brass and solid
chat: pull room contacts from associated group
chat: spell 'permanent' correctly
eyre: remove padding from 'access' input
chat: only delete metadata for a chat if you created it
chat: settings inputs add borders on focus
vere: disables gc on |mass in the daemon process
chat: remove console.log from metadataAction
chat: style fixes during review, use metadata-hook
chat: edit description, color settings
...
* origin/os1-rc: (439 commits)
pills: updated brass and solid
chat: pull room contacts from associated group
chat: spell 'permanent' correctly
eyre: remove padding from 'access' input
chat: only delete metadata for a chat if you created it
chat: settings inputs add borders on focus
chat: remove console.log from metadataAction
chat: style fixes during review, use metadata-hook
chat: edit description, color settings
chat: add update-metadata to metadata reducer
chat: revise api.js to match data structures
metadata-json: add json to action parsers
chat: construct settings page for metadata
chat: correct bottom border on join links
chat: copy shortcodes
chat: linkify unmanaged chats
metadata-hook: support group members other than host creating shared resources
contacts: add bg-gray0 to root page
chat + contact views: updated for style and to assert that group-path must be equal to app-path if there are ships in the members set
contacts: changed color + copy of "add to group" button
...
* origin/ted/ford-no-pit:
pills: update solid
http.c: revert timeout to original ~m10
tests: prime ford %reef cache
http.c: bump timeout from ~m20 to ~m30
http.c: bump timeout from ~m10 to ~m20
tests: fix ford tests for no %reef short-circuit
ford: remove pit short-circuit
