Commit Graph

3016 Commits

Author SHA1 Message Date
Philip Monk
093f0ae9de Merge branch 'philip/inline-thread' into philip/agent-clay-wip 2022-09-01 19:03:17 -08:00
Philip Monk
e3a8cab943 khan: support inline threads
This allows you to pass a thread directly into khan, instead of passing
a filename.  This has several implications:

- The friction for using threads from an app is significantly lower.
  Consider:

    =/  shed
      =/  m  (strand ,vase)
      ;<  ~  bind:m  (poke:strandio [our %hood] %helm-hi !>('hi'))
      ;<  ~  bind:m  (poke:strandio [our %hood] %helm-hi !>('there'))
      (pure:m !>('product'))
    [%pass /wire %arvo %k %lard %base shed]

- These threads close over their subject, so you don't need to parse
  arguments out from a vase -- you can just refer to them.  The produced
  value must still be a vase.

    ++  hi-ship
      |=  [=ship msg1=@t msg2=@t]
      =/  shed
        =/  m  (strand ,vase)
        ;<  ~  bind:m  (poke:strandio [ship %hood] %helm-hi !>(msg1))
        ;<  ~  bind:m  (poke:strandio [ship %hood] %helm-hi !>(msg2))
        (pure:m !>('product'))
      [%pass /wire %arvo %k %lard %base shed]

- Inline threads can be added to the dojo, though this PR does not add
  any sugar for this.

    =strandio -build-file %/lib/strandio/hoon
    =sh |=  message=@t
        =/  m  (strand:rand ,vase)
        ;<  ~  bind:m  (poke:strandio [our %hood] %helm-hi !>('hi'))
        ;<  ~  bind:m  (poke:strandio [our %hood] %helm-hi !>(message))
        (pure:m !>('product'))
    |pass [%k %lard %base (sh 'the message')]

Implementation notes:
- Review the commits separately: the first is small and implements the
  real feature.  The second moves the strand types into lull so khan can
  refer to them.

- In lull, I wanted to put +rand inside +khan, but this fails to that
  issue that puts the compiler in a loop.  +rand depends on +gall, which
  depends on +sign-arvo, which depends on +khan.  If +rand is in +khan,
  this spins the compiler.  The usual solution is to either move
  everything into the same battery (very ugly here) or break the
  recursion (which we do here).
2022-08-29 21:04:50 -08:00
Philip Monk
0e14eedb50 spider: accept inline threads 2022-08-29 20:09:08 -08:00
Philip Monk
fa569cf7f3 kiln: wip 2022-08-29 10:53:36 -08:00
Philip Monk
43ad38cd82 azimuth: reorder %kick flow
Before this, the %watch to eth-watcher was happening before the %poke,
and so eth-watcher was responding with its entire history immediately.
This is bad because it takes a lot of memory to process that many logs,
and also because those logs are stale.

Now, the %poke happens first, which clears the history.
2022-08-19 20:22:20 -08:00
Philip Monk
c6024c2661 azimuth: clear logs on %kick
%kick is supposed to start back from the snapshot and move forward.
Without this, we would only fetch logs that we hadn't already fetched.
Thus, if you were up-to-date when you kicked, you would miss anything
that happened between the time the snapshot was taken and the present,
though you would see things after the present.

Also reverted lull change to make this a safer upgrade.
2022-08-19 15:04:30 -08:00
Philip Monk
ca0a00cb3c Merge remote-tracking branch 'origin/master' into philip/snap 2022-08-19 12:37:25 -08:00
drbeefsupreme
938dc8cfe4 dojo: reverse doccords query ordering 2022-07-06 17:19:44 -04:00
Sidnym Ladrut
13b8b337a0 dojo: re-add support for /=, /% runes 2022-07-06 00:29:23 -05:00
fang
6b18e73b0b
Revert "various: further tweaks and assets for testnet"
This reverts commit 6eba8053f8.
2022-07-05 16:34:04 +02:00
Sidnym Ladrut
3e22ee68e9 dojo: improve ford rune support for cable expressions 2022-07-02 18:04:54 -05:00
Sidnym Ladrut
8a017ce308 dojo: re-add support for /+, /- ford runes 2022-07-01 23:39:37 -05:00
Philip Monk
89b9977ac8 azimuth: fetch snapshot on boot
Previously, the initial Azimuth snapshot was stored in Clay and shipped
in the pill.  This causes several problems:

- It bloats the pill
- Updating the snapshot added large blobs to Clay's state.  Even now
  that tombstoning is possible, you don't want to have to do that
  regularly.
- As a result, the snapshot was never updated.
- Even if you did tombstone those files, it could only be updated as
  often as the pill
- And those updates would be sent over the network to people who didn't
  need them

This moves the snapshot out of the pill and refactors Azimuth's
initialization process.  On boot, when app/azimuth starts up, it first
downloads a snapshot from bootstrap.urbit.org and uses that to
initialize its state.  As before, updates after this initial snapshot
come from an Ethereum node directly and are verified locally.

Relevant commands are:

- `-azimuth-snap-state %filename` creates a snapshot file
- `-azimuth-load "url"` downloads and inits from a snapshot, with url
  defaulting to https://bootstrap.urbit.org/mainnet.azimuth-snapshot
- `:azimuth &azimuth-poke-data %load snap-state` takes a snap-state any
  way you have it

Note the snapshot is downloaded from the same place as the pill, so this
doesn't introduce additional trust beyond what was already required.
When remote scry is released, we should consider allowing downloading
the snapshot in that way.
2022-06-30 23:22:50 -07:00
drbeefsupreme
b3e6f82499 dojo: change doccords syntax to be path-like
using col as a seperate made it look like a bunch of =<, which doesn't
make sense for e.g. pritning chapters. paths aren't quite right either,
so we don't use +stap, and just want identifiers separated by fas
without any leading fas
2022-06-30 17:13:07 -04:00
drbeefsupreme
0f8f90c1fd dprint: simplify type to item interface
shouldn't need to surface recursion to the developer
2022-06-30 16:56:04 -04:00
drbeefsupreme
4b8cdd6f83 dprint: easier to read style changes 2022-06-30 15:14:56 -04:00
drbeefsupreme
5b65288268 dprint: add styled text first run
mostly just to get something working, next i will clean up syntax
2022-06-30 11:39:06 -04:00
fang
292a65f64c
Merge branch 'next/arvo' into yu/content-dist-merged
Includes patched versions of ames' and clay's +load arms.

In clay, we do a dumb ;; hack to get the state to adapt properly. This
shouldn't be needed ($case had an extra... case added to it, old ones
should still nest), and so we should revisit the logic there to make it
cleaner/better before release.
2022-06-30 16:27:02 +02:00
drbeefsupreme
8cd919b936 dojo: ?? pattern matches on %hint types 2022-06-28 11:31:22 -04:00
drbeefsupreme
8f946ed414 dojo: print %help hints with ?? 2022-06-24 13:55:09 -04:00
fang
981bc796d7
Merge pull request #5841 from dnmfarrell/dnmfarrell/arvo-dojo-dont-crash-set-beak
dojo: setting eny/now/our shouldn't crash dojo
2022-06-18 14:25:04 +02:00
fang
df82167111
Merge pull request #5840 from dnmfarrell/dnmfarrell/arvo-dojo-check-dir-exists-before-switch
dojo: check =dir exists before switching
2022-06-14 16:26:01 +02:00
David Farrell
d8f34e8930 dojo: remove unnecessary debug output
The tip already communicates to the user that dojo is busy, so the
sigpam output isn't needed anymore.
2022-06-13 21:00:46 -04:00
David Farrell
d439b33cc9 dojo: say how to beat the %dy-edit-busy escape room
It's too easy to get trapped in the dojo %dy-edit-busy escape room. Just
type something like:

  -build-file /=base/gen/ls/hoon

This modifies the dojo output to tell the user how to get out.

Fixes #1462.
2022-06-13 21:00:21 -04:00
David Farrell
04672fbe1d dojo: setting eny/now/our shouldn't crash dojo
Changes dojo to emit a usage message and ignore the request.

Fixes #1519
2022-06-13 20:51:38 -04:00
David Farrell
3a10b4d8f3 dojo: simplify dir not exist error message build
Incorporates @Fang- suggested changes (thanks!).

Drops the path serialization as it will print on two separate lines,
and it is already displayed in dojo immediately above the error message:

    > =dir /=base=/ge
    dojo: dir does not exist
2022-06-13 20:37:55 -04:00
David Farrell
73c1c94ea8 dojo: have dojo check =dir exists before switching
Scries clay for a list of files beneath the provided path - if the response
is nil then the dir must not exist (clay abhors a vacuum).

Fixes #1559
2022-06-13 20:37:49 -04:00
drbeefsupreme
39ac7cb820 Merge branch 'master' into jon/doccords 2022-06-07 11:10:20 -04:00
fang
6eba8053f8
various: further tweaks and assets for testnet 2022-05-24 23:55:32 +02:00
fang
a0f7891f50
Merge branch 'philip/tomb' into lf/content-dist-rc
Beware, the ames conflict was particularly nasty. State migrations as of
yet untested.

Includes patched-up fine tests.
2022-05-19 17:07:35 +02:00
Philip Monk
d25194c7a3 arvo: reduce priority of lifecycle printfs 2022-05-14 23:59:21 -07:00
Philip Monk
e457a4c15e Merge remote-tracking branch 'origin/jm/homebase' into philip/tomb 2022-05-14 20:46:09 -07:00
Philip Monk
915dc733f5 Merge remote-tracking branch 'origin/m/buclus' into philip/tomb 2022-05-14 20:39:51 -07:00
Philip Monk
0ef9570f06 eth-watcher: fix upgrade 2022-05-14 20:28:56 -07:00
Philip Monk
d50d3915b8 Merge remote-tracking branch 'origin/yu/ames-fixes' into philip/tomb 2022-05-14 19:28:30 -07:00
Philip Monk
6f46e483d0 Merge remote-tracking branch 'origin/yu/azimuth-kick' into philip/tomb 2022-05-14 19:27:45 -07:00
yosoyubik
7fc5177602 azimuth: restart %eth-watcher if not running 2022-05-12 07:14:26 +02:00
Philip Monk
8e2720cd0e Merge remote-tracking branch 'origin/yu/azimuth-kick' into philip/tomb 2022-05-09 22:36:28 -07:00
Philip Monk
c76214b226 Merge remote-tracking branch 'origin/next/arvo' into philip/tomb 2022-05-09 22:36:19 -07:00
yosoyubik
da4de1579d eth-watcher: discard future pend-logs in watchdog
If there were pending-logs in an existing watchdog that was not fully
restarted, and the number of the starting block is newer than the first
in pending, when starting a new thread, those logs will be carried over
to the new thread, which will then be re-downloaded and will fail to be
verified in /lib/naive
2022-05-08 09:43:30 +02:00
yosoyubik
79d0129401 azimuth: use %kick poke to restart subscriptions 2022-05-07 16:27:04 +02:00
fang
8bef14d0a7
Merge branch 'philip/tomb' into lf/content-dist-rc 2022-05-05 00:33:37 +02:00
Philip Monk
591bdf458c Merge branch 'next/arvo' into philip/tomb 2022-05-04 00:45:57 -07:00
Philip Monk
71d4f72ce1
Merge pull request #5647 from urbit/roller/batch-split
roller: add batch-trimming functionality
2022-05-03 14:34:12 -07:00
fang
b42833a568
Merge branch 'm/scry-vane-next' into lf/content-dist-rc 2022-05-03 15:29:16 +02:00
yosoyubik
79af26fc3c azimuth: revert azimuth change in 60cc44e 2022-04-26 18:36:48 +02:00
Philip Monk
ac6e793aa2 arvo: rename +de to +of 2022-04-22 17:55:42 -07:00
yosoyubik
60cc44e961 ames: fix cached state in larval upgrade 2022-04-22 12:26:02 +02:00
yosoyubik
1b01455fe5 ames: fix dangling-bone error 2022-04-22 12:25:55 +02:00
fang
4415aa781e
Merge branch 'next/arvo' into m/aqua-revival 2022-04-20 18:47:59 +02:00
fang
a2cab52742
aqua: minimum viable remote scry support
Includes a dumb test for -keen.
2022-04-20 00:12:32 +02:00
yosoyubik
7bf759adb0 roller: fix parsing nonces with dot separator 2022-04-15 15:27:08 +02:00
yosoyubik
2eec24ee27 roller: make output less loud 2022-04-15 13:19:10 +02:00
Pyry Kovanen
148c7880da
eth-watcher: remove obsolete zoom-margin handling 2022-04-11 22:15:14 +03:00
drbeefsupreme
24a175a803
dojo: flop order of topics for dy-inspect
you had to search for core:arm instead of arm:core before
2022-04-07 13:05:15 -04:00
fang
132592e32e
Merge pull request #5695 from pkova/master
eth-watcher: implement optional to-block parameter
2022-04-07 16:15:21 +02:00
fang
8731ac191d
Merge branch 'm/aqua-revival' into m/scry-vane-next 2022-04-07 14:55:21 +02:00
fang
c4f3825f3a
aqua: revert erroneously committed changes
These changes, part of optimization experiments, had snuck in somewhere.
2022-04-07 14:39:52 +02:00
pkova
32ac7434f4 eth-watcher: implement optional to-block parameter 2022-04-07 14:24:28 +03:00
fang
af4de87ff2
Merge branch 'm/aqua-revival' into m/scry-vane-next 2022-04-07 12:59:14 +02:00
drbeefsupreme
7a94319af3
dojo, doccords: dprint library can take list term
this allows you to search for foo:bar instead of just foo
2022-04-06 17:37:55 -04:00
drbeefsupreme
33b2761b39
doccords, dojo: move flop print-arm -> dy-inspect
this looks more sensible
2022-04-06 15:50:23 -04:00
drbeefsupreme
877fe0059e
dojo: add command to lookup doccords
syntax: `# foo` to look up foo. this is not the final form - it does not
support `# foo:bar` yet. the dprint library needs some changes first.
2022-04-06 14:02:25 -04:00
fang
9bbf6dfad0
aqua: disable cache loading for real ships for now
It breaks some of the azimuth-related tests in a seemingly inconsistent
way, and I cannot find a way to make it work.
2022-04-06 14:52:21 +02:00
fang
fa1a04ea6e
aqua: support using real moons
Only "shortmoons" though, due to some ames lane size limitation which
makes encoding longer ships difficult.

Notably the -ph-moon-az test does not pass, the moon cannot talk to a
non-sponsor galaxy.
2022-04-06 14:51:02 +02:00
Philip Monk
4131821b4f clay: unify all trie implementations
We had trie operations independently implemented in +de in arvo,
+an:cloy in zuse, +zu in clay, lib/trie, and app/spider.  This unifies
them all into +de in arvo, aggregating the used operations.
2022-04-06 00:19:38 -07:00
Philip Monk
73180a0d94
Merge pull request #5670 from urbit/jon/thread-marks
spider: fix child threads
2022-04-01 16:48:15 -07:00
fang
69c3976460
aqua: remove todo list note
Was committed accidentally.
2022-03-31 23:39:20 +02:00
fang
30be6ef9c8
aqua: stub out azimuth snapshot in pills
It would interfere with aqua's simulation of azimuth.
2022-03-30 17:38:09 +02:00
fang
3d639f325e
aqua: cache real ships as well
And inject their latest keys as soon as we pull them from cache. This
way, we avoid having to do the whole boot sequence again just for a
modified dawn event.
2022-03-30 16:38:48 +02:00
fang
2f3527921a
aqua: increase azimuth refresh rate for real ships
This reduces the time it takes to run tests that wait on azimuth events.
For example, -ph-breach-hi runs ~30% faster.
2022-03-30 13:59:10 +02:00
fang
dad82436be
aqua: support injecting arbitrary tasks 2022-03-30 12:51:46 +02:00
fang
c55975e8de
azimuth: allow configuring of the refresh rate
Pulls a constant out of the code, into state, and adds a poke for
changing it.
2022-03-30 12:19:57 +02:00
drbeefsupreme
656a86479f
spider: fix child threads
addresses #5442 by adding %thread-done and %thread-fail marks. also
fixes await-thread:strandio and removes some blank lines from
app/spider.hoon

%thread-done loses the type of the result, so you'll need to use ;; to
get it back. the real way to fix this is to have threads produce cages
instead of vases
2022-03-25 18:01:48 -04:00
fang
8da6c20f70
herm: stop sending %hail on-connect
Client will probably want to send a %blew first anyway. By not doing any
screen refreshed in herm, we avoid doing unnecessary redraws on-connect.
2022-03-25 15:08:50 +01:00
fang
3259f1f588
aqua: fix most tests
Small touch-ups to simulation behavior and ph tests. Most of them pass
now, even if they're still really slow at times.

The breach ones don't pass, but also complain of dangling bone, so might
work once the fix for that is in.
2022-03-23 22:19:35 +01:00
fang
804c8a8bb7
aqua: use cache for fake ships only
Boot sequence for real ships contains %dawn, which is affected by
azimuth state and may not be consistent.
2022-03-21 20:29:23 +01:00
yosoyubik
cff562b4c6 Merge branch 'next/arvo' into roller/batch-split 2022-03-20 14:27:50 +01:00
yosoyubik
cf838fd1d7 roller: add batch-trimming functionality
If a batch gets bigger than a max size defined by the ethereum node
the raw transaction is sent to, the /ted/roller/send thread will crash and
the batch will be blocked, stopping any subsequent batches to be sent.

This detects when the current batch reaches a certain threshold and only
includes transactions up to that point, moving the ones that are not sent
back to the pending queue, adjusting their history and finding status.
2022-03-20 14:07:27 +01:00
fang
3a4993f58c
aqua: cache results of booting ships from pill
This gives us much higher cycle times when retrying tests over and over,
since we can just fast-forward past the boot sequence, which never
changes unless we change the pill.
2022-03-19 00:24:23 +01:00
fang
250e750a89
spider: factor spider-arg type out into interface
And fix a callsite that wasn't conforming.
2022-03-16 21:10:31 +01:00
fang
41a796d2d1
hoon: hook up $+ for shorthand type rendering
Too often when dealing with big types the compiler traces and other such
outputs become hard to read. Wrapping a type as $+(shorthand big-type)
will now print #shorthand in place of the type.
2022-03-14 13:58:48 -04:00
Liam Fitzgerald
49e53290a1 Merge remote-tracking branch 'origin/yu/dangling-bone' into lf/content-dist-rc 2022-03-11 11:07:51 -06:00
yosoyubik
5ecf87cf52 ames: fix cached state in larval upgrade 2022-03-11 10:22:50 +01:00
yosoyubik
ab116653e8 wip fix dangling bone 2022-03-11 10:21:12 +01:00
fang
3df227760f
hoon: hook up $+ for shorthand type rendering
Too often when dealing with big types the compiler traces and other such
outputs become hard to read. Wrapping a type as $+(shorthand big-type)
will now print #shorthand in place of the type.
2022-03-10 01:45:18 +01:00
ryjm
20fdae85a4 fix typo and beak 2022-03-08 10:13:04 -05:00
ryjm
dc03d866f8 arvo: language-server desk support 2022-03-06 15:45:47 -05:00
ryjm
155729ec8b %home -> %base
Mostly cosmetic, but also fixes `language-server` and parts of sundry
neglected components (the nix build for solid pills, aqua, gaze exports,
+help)
2022-03-06 11:55:55 -05:00
fang
e41263c653
dojo: add missing nu-sole-id compatability
Forgotten code.
2022-03-02 22:49:31 -06:00
Jōshin
06fe78ef6e
Merge branch 'next/arvo' into jo/khan 2022-03-02 20:28:44 -06:00
fang
41a063867b
dojo: auto-fill generator's named drum-session arg
If a generator has a named argument named "drum-session", and the caller
doesn't specify a value for it, dojo auto-fills it with the current drum
session identifier.
2022-02-28 16:41:49 -06:00
Liam Fitzgerald
b4da246f01 strandio: add +keen gate 2022-02-24 12:03:07 -06:00
fang
3120681b2b
sole: properly support multiple sessions
We update the sole protocol to more cleanly support multiple sessions.

Primarily, the "sole id" is updated to be a [@p @ta] instead of a @ta,
and it is now generated based off the connected dill session, rather
than statically.

This change ripples out to applications that support the sole protocol:
the subscription path becomes /sole/[ship]/[session] (as opposed to
/sole/[per-ship-constant]), and %sole-action pokes include the new id as
well.
For shoe agents, this means (at the very least) updating the function
signatures of the shoe arms.

/lib/sole has been updated to include helper functions for parsing a
sole-id from a subscription path, and turning a sole-id into its
corresponding path. It also has a function to aid in migrating old
sole-ids.

Existing sole agents are made to kick any known open sessions, forcing a
resubscribe by drum, so that they may use exclusively the new format
going forward. Third-party agents are recommended to do the same.

Note that some functionality, such as |link, still operates exclusively
on the default session. Improvements in this area to follow soon.
2022-02-20 17:19:03 -06:00
yosoyubik
1dd5fca11c roller: bug fix in looking for sending diff 2022-02-12 14:36:40 +01:00
yosoyubik
72794870c8 roller: include signing address in sending txs 2022-02-12 14:02:50 +01:00
Liam Fitzgerald
e97d99275b ames: congestion control for remote scry 2022-02-11 19:30:40 -06:00
yosoyubik
403c8af519 roller: check for existence in resend nonce 2022-02-11 15:46:13 +01:00
yosoyubik
5c9fc52f47 roller: keep force flag in sending transactions 2022-02-11 14:28:27 +01:00
fang
cb706ee8ae
dbug: update for ames' new alien-agenda 2022-02-03 00:45:16 +01:00