Commit Graph

269 Commits

Author SHA1 Message Date
Joe Bryan
30befdf4a9 arvo: updates +of-wain:format sample
and consistently uses $cord and $wain in of/to-wain
2020-09-18 16:15:13 -07:00
Paul Driver
2da36aa234 arvo: change sample type of ++to-wain:format to @t
@t further indicates to the caller (although loosely, because auras
are loosely enforced types) that the input should not contain embedded
NUL bytes, which are not valid @t.

Separately, a minor improvement has been made to the performance of
the raw hoon by pinning the gates used in the inner loop.
2020-09-18 16:15:12 -07:00
Paul Driver
3aaf82de5e arvo: accept embedded nulls in to-wain:format
Prior to this commit, there was a jet mismatch in to-wain (formerly
called lore, and still jetted under that name). 0 bytes in the middle of
a cord caused the jet to crash, whereas the hoon simply treated them as
the end of cord and truncated the output. The history of this behavior
is fraught with controversy. This commit rectifies the current mess with
the following rationale: Null bytes are valid ASCII/UTF-8, and \n\n in
the input will cause null list items in the output, so nulls are (for
the purposes of to-wain) allowed in cords. Trailing nulls cannot be
represented because of the nature of atoms, but that is outside the
scope of to-wain's concern. Therefore to-wain should simply measure the
cord and split on newlines, and do nothing fancy at all with nulls.

In addition, the hoon for to-wain was written in an inefficient style
that produced a lot of intermediate garbage atoms via rsh and cat. This
commit's implementation measures once and cuts once, so to speak, and so
avoids the intermediate garbage.  Quick benchmarks suggest it is about
20x faster than the old hoon, but still orders of magnitude slower than
the jetted code. to-wain is the workhorse for the txt mark, so we should
still prefer to have a jet.

The old jet is left wired up under %lore, and should be removed when
support for the old, unupgraded zuse is no longer necessary. A new jet
with matching null handling has been wired up under the name %leer.
2020-09-18 16:15:10 -07:00
Paul Driver
44b7d5de6b zuse: make it clear where the secp jet should make explicit size checks
Well-behaved secp jets can now plausibly exit if they are given inputs
of the wrong size/range. Previously this was either not checked or the inputs
were silently truncated.
2020-09-18 15:59:00 -07:00
Paul Driver
9be3318ae9 zuse: add refactored secp core (unjetted)
The secp core had some flaws: in particular, the logic for signing/recovery
did not match libsecbp256k1 w.r.t. the enigmatic "recid" (v) value. The jet
hints were also subtly wrong, in that the curve parameters were in a sample
(not an arm) and thus not matched by the jet matching scheme. Consequently,
the jets would be used (but incorrect) for other curve parameters.

Tests were also added to exercise the recovery id cases thoroughly.
2020-09-18 14:47:22 -07:00
Joe Bryan
31fe240632 arvo: adds |meld, triggering memory unification 2020-09-09 22:50:43 -07:00
Joe Bryan
7f0b500778 arvo: adds |trim, triggering memory reclamation 2020-09-09 22:49:37 -07:00
Fang
ef4b56f8c7
dill: send styled text to runtime as-is
Depending on the additions to term.c made in 467d8d239 allows dill to
forget about ansi escape codes, and pass styled text nouns straight on
to vere.

Also removes a bit of logic from drum, which assumed things about the
rendering of escape codes to adjust cursor positioning. Now it simply
states the semantic cursor position, letting the runtime deal with the
potential influence of styling.
2020-09-09 19:09:29 +02:00
matildepark
c45691d214
Merge pull request #3408 from urbit/lf/xml-ver
zuse: allow xml declaration in +de-xml:html
2020-09-05 16:17:31 -04:00
Liam Fitzgerald
5020223ff2 zuse: de-xml:html handles declarations as in spec 2020-09-01 02:15:19 +10:00
Liam Fitzgerald
b5f64715f4 zuse: allow xml declaration in +de-xml:html
Allows for parsing of an xml declaration. Throws it away, as the
datastructure doesn't have anywhere to put the result
2020-08-31 12:24:41 +10:00
Philip Monk
67c5caada8
clay: add safe merge options
As described in gen/hood/merge/help.txt
2020-08-28 16:36:00 -07:00
Caio Marcelo de Oliveira Filho
92cf9d7c33 Merge branch 'master' into code-reset 2020-08-17 14:55:07 -07:00
Caio Marcelo de Oliveira Filho
df868e2c4f jael, eyre: Add a way to reset the web login code
Jael now stores a `step` that is combined with the original salt to
produce a new code.  A `%step` card is used to increment that value,
and effectively resetting the keys.  Because the first `step` is zero,
the first code is the same as before.

Eyre was changed to be notified with `%code-changed` so it can forget
old cookies, sessions and discard all the existing channels.

A new generator was added |code, that does both querying and
resetting the code

    |code             :: shows current code, step and help
    |code %reset      :: changes the code

The old +code generator still works correctly.
2020-07-31 11:35:48 -07:00
Philip Monk
553a9db843
clay: refactor over-the-wire protocol
Don't send blobs initially; instead, wait for the recipient to ask for
any they need.  This should significantly reduce network and memory usage.
2020-07-23 00:12:33 -07:00
Matilde Park
0d68f3f901 Merge branch 'release/next-sys' 2020-07-22 14:31:17 -04:00
Philip Monk
452b668c5f
Merge pull request #2972 from urbit/la/ordered-map-subset
zuse: add subset arm to ordered-map
2020-07-21 18:09:22 -07:00
Philip Monk
753ccf1b7e
Merge remote-tracking branch 'origin/ted/fix-huck' into release/next-sys 2020-07-21 17:08:01 -07:00
Logan Allen
f76759b9e0 zuse: added docstring and warning 2020-07-21 14:39:42 -07:00
Fang
d65148ea37
zuse: parse rfc2396 unreserved chars correctly
RFC2396 defines[1] unreserved characters as alphanumerics and nine "mark"
characters. We were only parsing for four of those, leading to parsing failure
for valid URLs.

[1]: https://tools.ietf.org/html/rfc2396#appendix-A
2020-07-19 17:58:19 +02:00
Fang
76b3ce8ef1
Merge pull request #3063 from urbit/m/drop-pole
zuse: make +drop-pole produce a unit
2020-07-09 18:06:15 +02:00
Logan Allen
64a30fea62 ordered-map: got %zuse to compile and all tests to pass 2020-07-08 15:07:14 -04:00
Ted Blackman
11a8e911be zuse,behn,gall: fix %huck and |reset 2020-07-08 02:08:27 -04:00
Logan Allen
cd5840770e zuse: ordered-map subset style fixes 2020-07-07 19:19:47 -04:00
Logan Allen
4168344c3b zuse: added mop, a mold builder for ordered-maps 2020-07-07 19:19:47 -04:00
Logan Allen
04232e0ebe zuse: add subset arm to ordered-map 2020-07-07 19:19:47 -04:00
Fang
0ecd108f98
zuse: make +drop-pole produce a unit
Both +drop-list and +drop-map produce units instead of crashing. This makes
+drop-pole match that behavior, producing a unit of the resulting tuple.
2020-06-28 21:33:17 +02:00
Philip Monk
0301838f25
Merge remote-tracking branch 'origin/release/next-sys' into ford-fusion 2020-06-26 17:46:25 -07:00
Philip Monk
6ff9f75bb7
Merge remote-tracking branch 'origin/master' into ford-fusion 2020-06-19 14:52:18 -07:00
Fang
b2600f553c
Merge branch 'master' into m/eyre-kill-channels 2020-06-18 22:23:47 +02:00
Joe Bryan
faa4956969 zuse: propagates fix to aes-128-siv and aes-192-siv as well 2020-06-12 22:21:19 -07:00
Joe Bryan
0ddeb6526b zuse: fixes bug in aes-256-siv iv calculation (+s2vc:aes:crypto) 2020-06-12 21:34:04 -07:00
Fang
d20877e414
eyre: support %gx scries
Exposes a scry endpoint. Any requests made to the /app/scry.mark url
under the endpoint will scry into %app using a %gx scry, at the
/scry/noun path, and attempt to convert the scry result into the %mark,
before converting that into the %mime mark, and sending that as an http
response.
2020-06-11 01:45:05 +02:00
Fang
0a32bcda35
Merge branch 'release/next-sys' into ford-fusion 2020-06-09 20:10:28 +02:00
Fang
90ef268a32
Merge branch 'release/next-sys' into m/eyre-kill-channels 2020-06-05 22:25:18 +02:00
Philip Monk
80ead610ea
Merge remote-tracking branch 'origin/master' into release/next-sys 2020-06-04 17:52:42 -07:00
Fang
b54dc7cd34
eyre, zuse: expire channels with their sessions
Associates channels with the authentication sessions that opened them,
and deletes the channel when its associated session expires.

Also updates the debug dashboard to display channel counts per session.
2020-06-05 00:22:39 +02:00
Philip Monk
8b78f04dd3
Merge remote-tracking branch 'origin/master' into ford-fusion 2020-06-02 21:50:20 -07:00
Fang
4d93349402
eyre: provide logout endpoint
Set up, by default, on /~/logout.

Sending a POST request to this expires the current session and redirects
to the login page. If the "all" key is set in the request body, expires
all open sessions.
2020-06-03 01:40:32 +02:00
Fang
a87000149d
zuse: add parsers for http header field values
Multi-value format as used by some headers.

Also updates pill with recent zuse changes.
2020-05-31 17:45:20 +02:00
Fang
b1daaec7bf
zuse: add parsers for ip addresses
Parses into +address:eyre.
2020-05-31 17:45:18 +02:00
Philip Monk
0642cc5861
ames: add |ames-wake
Somehow we ended up with flows which expected to awaken but did not wake
up.  This was likely caused by the error in r920j OTA, urbit-os-v1.0.18.

This adds a command which ensures that every flow has an active timer.
I expect this to be needed only once, but it's a pretty general tool, so
it's worth keeping.

I've included an unused @t parameter to more easily add simple debug
commands to ames without having to add a new task
2020-05-28 10:28:31 -07:00
John Franklin
9d8730a833 hoon: leb/reb to kel/ker 2020-05-28 07:20:04 -05:00
John Franklin
510fb6b394 hoon: bat/net to bas/fas 2020-05-28 06:58:18 -05:00
John Franklin
250f4e3130 hoon: lac/rac to sel/ser 2020-05-28 06:14:23 -05:00
John Franklin
413c5682b6 hoon: lit/rit to pal/par 2020-05-28 05:31:50 -05:00
John Franklin
fdb03c1f7d hoon: led/ban to gal/gar 2020-05-28 04:46:13 -05:00
John Franklin
a75cd8c85c hoon: yel to doq 2020-05-28 04:01:25 -05:00
John Franklin
51bd4a95f8 hoon: say to soq 2020-05-28 03:17:41 -05:00
John Franklin
6c8e3b1c92 hoon: vat to pat 2020-05-28 02:45:04 -05:00
John Franklin
bc46e26a5d hoon: pad to pam 2020-05-28 02:06:00 -05:00
John Franklin
ba51e2391c hoon: tec to tic 2020-05-28 01:44:36 -05:00
John Franklin
1d8624cd74 hoon: bus to buc 2020-05-28 00:50:45 -05:00
Fang
36b7fc2e4f
Merge branch 'release/next-sys' into m/debug-dashboard 2020-05-26 20:36:27 +02:00
pkova
28cd252ce7 zuse: improve json to rn parsing
Allow the exponent token to be 'e' or 'E', allow the explicit sign of '+'.
Following the spec at https://json.org. Fixes #2935.
2020-05-26 19:29:24 +03:00
Fang
58e85fa661
Merge branch 'release/next-sys' into m/debug-dashboard 2020-05-23 01:00:45 +02:00
Fang
52ef23ccca
eyre, zuse: add scry interfaces for eyre state
Scries for getting out open connections, cookie sessions, and existing
channels.

Moves the involved types from eyre into zuse.
2020-05-22 23:55:17 +02:00
Philip Monk
4fcd9f23c0
Merge branch 'release/behn-fast' (#2915)
* release/behn-fast:
  zuse: add +ordered-map
  behn: switch to ordered-map
  chat: virtualize message-to-json conversion
2020-05-21 22:43:53 -07:00
Philip Monk
274215ec6b
zuse: add +ordered-map
At some point this should be more properly styled similar to +by, +in,
and +to, but for now this reduces duplication and makes the ordered map
available to everyone.
2020-05-21 22:28:16 -07:00
Fang
4c9d18e5c0
Merge branch 'release/next-sys' into m/debug-dashboard 2020-05-21 20:09:58 +02:00
Ted Blackman
a0eafa8aa3 zuse,ames: handle %trim task 2020-05-21 03:45:08 -04:00
Ted Blackman
7e69e66280 ames/zuse: handle %trim event 2020-05-20 03:29:54 -04:00
Philip Monk
a874ab4355
zuse: remove +ford 2020-05-14 00:22:39 -07:00
Philip Monk
2c2e16df7a
language-server: convert to fusion 2020-05-13 23:30:52 -07:00
Philip Monk
fd26b2184e
kiln: convert to fusion 2020-05-13 20:13:53 -07:00
Ted Blackman
4dde2081f1 Merge branch 'ted/leap-arvo' into ford-fusion 2020-05-13 19:44:44 -04:00
Ted Blackman
778f6e9d05 /sys: make hoon optional in %lyra 2020-05-13 18:27:14 -04:00
Philip Monk
4e1bac8946
clay: convert |mount to fusion 2020-05-12 01:27:19 -07:00
Fang
be087a8c92
Merge branch 'release/next-sys' of https://github.com/urbit/urbit into m/debug-dashboard 2020-05-11 20:39:00 +02:00
pkova
9d3c997ecf zuse: allow whitespace around = in attr:de-xml:html
Allow one or more whitespace characters before and/or after the equals sign in
name attribute pairs, such as `<hello a = "yo" />` or `<hello a=    "yo" />`.
Following the spec at https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-Eq.
2020-05-11 21:09:23 +03:00
Fang
664275c9f1
eyre: expose bindings through scry
Allows you to scry out all bound endpoints at /=bindings=.

Moves an internal type into zuse for easier external use.
2020-05-08 01:39:56 +02:00
Fang
14831f4864
ames: expose state through scries
Support /=peers= and /=peer=/~ship scries for getting at all peers and
a specific peer's connection state, respectively.

Moves some internal types into zuse for easier external use.
2020-05-08 01:37:07 +02:00
Ted Blackman
a064afbd89 /sys: |mass works again 2020-05-07 04:51:08 -04:00
Philip Monk
65766a73e2
clay: port -merge to fusion 2020-04-29 17:38:24 -07:00
Ted Blackman
80541b39aa gall: live update agents 2020-04-27 23:03:17 -04:00
Philip Monk
dca81a2ee2
clay: prepare for kernel updates 2020-04-24 17:44:29 -07:00
Philip Monk
fa0e019ddf
clay: change %park interface to not take caches
Now a full commit is synchronous.  -commit will be assimilated back into
clay.
2020-04-23 13:54:57 -07:00
Ted Blackman
627bb565c2 boots off gall using fusion to build apps; some app errors on boot, but usable 2020-04-22 05:27:36 -04:00
Ted Blackman
9b0cb85a02 gall: clean up API, remove empty caches from state. 2020-04-21 18:50:47 -04:00
Ted Blackman
8b4ad9bfc9 :spider2 works with threads in subfolders 2020-04-21 00:22:54 -04:00
Ted Blackman
f36dc49ede -build-file, -build-mark, -build-cast 2020-04-17 22:50:03 -04:00
Philip Monk
310f5d19ef
clay: add stubs for %a, %b, and %c cares 2020-04-15 17:47:45 -07:00
Aaron Sikes
eb0ce2daba Change at:dejs-soft:format to reject too-long json arrays
`at` is for when you expect an array of a certain exact structure. If it
has extra elements, that indicates you were mistaken about the strucutre,
so it should fail to match.
2020-04-06 14:05:35 -04:00
Isaac Visintainer
2ad8264c52 Merge branch 'philip/gall-clear' 2020-04-02 15:32:43 -07:00
Fang
d1894cdbe6
zuse: allow empty segments in de-purl url parsing
RFC 2396 specifies that segments must be zero or more pchars.[1] We were
deviating from this by requiring at least one pchar per segment.

With this change, we support /some//path, and no longer lose the
trailing slash in /some/path/.

[1]: https://tools.ietf.org/html/rfc2396#section-3.3
2020-04-02 14:44:45 +02:00
Philip Monk
3f2a7b08a5
clay: add -merge except meet/mate/meld
-merge will replace |merge so that.  Once they reach feature parity and
%info is rewritten to forward to -commit, we can rip out about half of
clay.hoon
2020-03-25 20:54:19 -07:00
ixv
375cb9ba1b
Merge pull request #2591 from urbit/m/eth-abi-etyp
zuse: better ethereum abi data type
2020-03-24 15:02:17 -07:00
Fang
303b4b2756
zuse: better ethereum abi data type
Considering some of the options here were atoms, not cells, $% wasn't
appropriate, and led to *etyp:abi:ethereum resulting in ford %ride execution
failure. Simply using $? instead would result in a fish-loop, so here we split
the atom cases from the tagged union ones with a $@.
2020-03-23 12:43:40 +01:00
Fang
87dd953136
zuse: allow com in de-purl query parsing 2020-03-22 01:21:03 +01:00
Philip Monk
531630e93a
clay: add %park task and -commit
%park is a plumbing commit task.  It guarantees completion in a single
event, so you have to do much of the work before calling it.  -commit
is an example of how to do this.
2020-03-20 21:30:56 -07:00
Philip Monk
cf9ef764f1
gall: remove pending messages from breached ship
When a ship breaches, we remove all messages that have yet to be
delivered to an app (eg if it's not yet started).  We also add
|gall-sear to do this manually, but this shouldn't be needed in normal
operation.

Finally, to unblock ~zod and ~bus on mainnet, we sear one particular
ship automatically on loading hood.  It cannot be done manually because
no userpace changes can be made until it's unblocked.
2020-03-18 21:05:06 -07:00
lukechampine
225fee0258 zuse: split sivb keys at correct offset 2020-02-04 00:17:43 -05:00
Jared Tobin
47b1b0a9bd
Revert "zuse: allow non-trailing null bytes in to-wain:format"
This reverts commit db1b88b21b.

See discussion in #2195, #1755.
2020-01-29 14:53:09 +04:00
pkova
db1b88b21b zuse: allow non-trailing null bytes in to-wain:format 2020-01-25 20:54:49 +02:00
Fang
ae8a57ca25
gall: (list path) in %fact and %kick
Instead of providing a (unit path), allows for (list path), which better
supports the "update to path and subpath cases".

For example, if /things wants updates about everything, and
/things/specific wants updates about the specific thing, they'll both
need to receive a %fact when the specific thing changes.
Previously, these would have been two separate moves. Now, gall handles
the multi-targeting for you.
2019-12-23 13:37:32 +01:00
Fang
ea7c1db61c
various: use =/ in place of =+ ^-
Also faceless =; where appropriate.
2019-12-21 14:29:14 -03:30
Jared Tobin
eee651042c
Merge branch 'm/eth-sending' (#2094)
* origin/m/eth-sending:
  eth-sender: do proper nonce reading
  zuse: implement eth rlp decoding
  zuse: correctly encode leading zeroes in rlp data
  zuse: lightly restructure encode:rlp:ethereum
  static gall: update send-txs
  ethio: implement +request-batch-rpc-loose

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-19 11:20:06 -03:30
Philip Monk
3d30b3b73c
jael: only advance lifes
This matches the semantics of rifts and makes it more forgiving of
misordered updates.
2019-12-18 11:31:14 -03:30
Fang
1647d760d1
zuse: implement eth rlp decoding
Also updates existing tests to check both en- and decoding.
2019-12-14 21:39:54 +01:00
Fang
7755bc9718
zuse: correctly encode leading zeroes in rlp data 2019-12-14 21:25:20 +01:00