urbit/developers.urbit.org
1
1
Fork 0
mirror of https://github.com/urbit/developers.urbit.org.git synced 2024-09-11 13:55:59 +03:00
Code Issues Packages Projects Releases Wiki Activity

document remote scries & add gall scry reference

Browse Source
This commit is contained in:
Tinnus Napbus 2023-05-04 01:20:32 +12:00
parent a8d90d6100
commit fa04307343
7 changed files with 586 additions and 63 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

192
content/guides/additional/remote-scry.md Normal file
View File

@ -0,0 +1,192 @@
+++
title = "Remote Scry"
description = "Learn about scrying over the network"
weight = 5
+++
To [scry](/reference/glossary/scry) is to perform a *read* from Urbit's
referentially transparent namespace. In other words, it's a function from a
`path` to a `noun` (although in some cases, the resulting type may be more
constrained). Previously we only supported scrying within the same ship, but
from Kernel version `[%zuse 413]`, it is possible to scry from *other* ships.
## Lifecycle of a scry
When you think of scry, you probably think of [`.^`
(dotket)](/reference/hoon/rune/dot#-dotket). However, since networking is
asynchronous, this is not a suitable interface for remote scry. Instead, a ship
that wants to read from a remote part of the namespace will have to pass a
`%keen` task to its Ames, which then cooperates with Vere to produce the
desired data. In some future event when the result is available, Ames gives it
back as a `%tune` gift. From the requester's perspective, this is the entire
default lifecycle of a remote scry request.
Of course, you need to know how `%keen` and `%tune` look to be able to use
them. There are also a few exceptions to this default lifecycle. We'll go
through all of this in a moment, but first, let's look at what kind of data is
possible to scry.
## Publishing
At the moment, there are two vanes that can handle remote scry requests:
[Clay](/reference/arvo/clay/clay) and [Gall](/reference/arvo/gall/gall). Clay
uses it to distribute source code in a more efficient manner than is possible
with conventional Ames, but conceptually it only extends its [local
scries](/reference/arvo/clay/scry) over the network, with the notable
difference that you can't scry at the *current* time, since the requester
doesn't know when the request reaches the publisher. Additionally, the paths
are modified so that the vane and care are specified separately, like so:
`/c/x/1/base/sys/hoon/hoon`.
Gall is more interesting. First, let's clear up a possible misunderstanding
that could easily come up: remote scry does *not* involve calling an agent's
`+on-peek` arm. `+on-peek` scries always happen at the current time, and since
the requester can't know at which time the publisher handles the request, these
aren't possible to reliably serve.
Instead, agents *ask* Gall to `%grow` paths in the namespace on their behalf.
Gall will take care of incrementing version numbers, so that the same path
never maps to different nouns. The agent can also ask Gall to delete data,
either at a specific version number, or everything up to and including a
version number. Concretely, we've extended `$note:agent:gall` to include the
following cases:
```hoon
+$ note
$% ...
[%grow =path =page] :: publish
[%tomb =case =path] :: delete one
[%cull =case =path] :: delete up to
==
```
Here's an example sequence of cards that use these:
```hoon
[%pass /call/back/path %grow /foo atom+'lorem'] :: /foo version 0
[%pass /call/back/path %grow /foo atom+'ipsum'] :: /foo version 1
[%pass /call/back/path %grow /foo atom+'dolor'] :: /foo version 2
[%pass /call/back/path %grow /foo atom+'sit'] :: /foo version 3
[%pass /call/back/path %tomb ud+3 /foo] :: delete /foo version 3
[%pass /call/back/path %cull ud+1 /foo] :: delete /foo 0 through 1
[%pass /call/back/path %grow /foo atom+'amet'] :: /foo version 4
[%pass /call/back/path %grow /foo/bar atom+123] :: /foo/bar version 0
```
After this sequence of cards we would have the following mappings (assuming the
agent that emits them is named `%test`):
```hoon
/g/x/2/test//foo -> [%atom 'dolor']
/g/x/4/test//foo -> [%atom 'amet']
/g/x/0/test//foo/bar -> [%atom 123]
```
Let's pick apart the first one of these paths.
```hoon
/g :: g for Gall
/x :: a care of %x generally means "normal read"
/2 :: version number
/test :: the agent that published the data
/ :: ???
/foo :: the path that the data is published on
```
What's that lone `/` before the path? It signifies that this data is published
by *Gall* itself, instead of the `+on-peek` arm in the `%test` agent. As part
of the remote scry release, we will *reserve* part of the scry namespace for
Gall, effectively *preventing* any agents from directly publishing at those
paths. Though as we've seen, they can do it indirectly, by asking Gall to do it
for them using `%grow`.
As long as the extra `/` is included, Gall will serve scries with care `%x` at
both specific revision numbers and at arbitrary times. If the extra `/` is not
included, the scry has to happen at the current time, since we don't cache old
results of calling `+on-peek`.
### Additional Gall cares
Apart from supporting reads using the `%x` care, Gall now also supports three new cares:
- `%t` lists all subpaths that are bound under a path (only supported at the
current time, i.e. not remotely!).
- `%w` gives the latest revision number for a path (only supported at the
current time, i.e. not remotely!).
- `%z` gives the hash identifier of the value bound at the path (supported at
any time and at specific revisions, but not remotely).
All of these require the extra `/` to be present in the path, just as with `%x`.
## Scrying tasks
With this, we're ready to look at all the new tasks to, and gifts from, Ames:
```hoon
+$ task
$% ...
[%keen =ship =path] :: peek [ship /vane/care/case/spur]
[%yawn =ship =path] :: cancel request from arvo
[%wham =ship =path] :: cancels all scry requests from any vane
...
==
::
+$ gift
$% ...
[%tune =path sign=@ux data=(unit page)] :: peek result
[%miss =path] :: peek dropped
...
==
```
At this point, most of these should be very clear, but briefly:
- We pass `[%keen =ship =path]` to Ames to request to read from `path` on
`ship`. Example:
```hoon
[%pass /call/back/path %arvo %a %keen ~sampel /c/x/4/base/sys/hoon/hoon]
```
- We pass `[%yawn =ship =path]` to tell Ames that we're no longer interested in
a response. Example:
```hoon
[%pass /call/back/path %arvo %a %yawn ~sampel /g/x/4/test//foo]
```
- We pass `[%wham =ship =path]` to tell Ames that *no-one* on this ship is
interested in a response. Example:
```hoon
[%pass /call/back/path %arvo %a %wham ~sampel /g/x/4/test//foo]
```
- Ames gives `[%tune =path sign=@ux data=(unit page)]` to the original
requester(s) when it knows the value on `path`. It includes a signature
`sign`. If `data` is `~`, then the path will *never* have a value.
- Ames gives `[%miss =path]` if it wasn't able to produce the value on `path`.
This can happen if the publisher doesn't know the answer, or if the signature
verification fails. This does *not* imply that the same request will fail in
the future!
## `-keen`
In addition to the above interface offered to agents, there is also support for
making scry requests from threads using `+keen` in `lib/strandio`. It accepts a
`[=ship =path]` and returns a `(unit page)`. There is also a [thread `ted/keen`
that demonstrates
this](https://github.com/urbit/urbit/blob/i/5788/remote-scry/pkg/arvo/ted/keen.hoon).
You can run it from the dojo using `-keen [ship path]`. For example, this reads
the thread's own source code out of `~sampel`'s `%kids` desk, try it!
```
-keen [~sampel /c/x/1/kids/ted/keen/hoon]
```
## Additional reading
- [Gall scry reference](/reference/arvo/gall/scry): Reference documentation of
Gall's vane-level and agent-level scry interface.
- [Ames API reference](/reference/arvo/ames/tasks): Reference documentation of `task`s that can be passed to Ames, including those for remote scries.

82
content/guides/core/app-school/10-scry.md
View File

@ -15,71 +15,29 @@ define _scry endpoints_ which allow data to be requested from their states. The
endpoints can process the data in any way before returning it, but they cannot
alter the actual state - scries can only read, not modify.
Most of the time, scry requests are handled by Arvo, which routes the request to
the appropriate vane. When you scry a Gall agent you actually scry Gall itself.
Gall interprets the request, runs it on the specified agent, and then returns
the result. Scries are performed with the
[dotket](/reference/hoon/rune/dot#-dotket) (`.^`) rune. Here's a summary of
their format:
Gall itself defines some special vane-level endpoints [as described in its scry
reference](/reference/arvo/gall/scry), but most scries to Gall are routed to
particular agents and handled by them instead. Agent scries are what we'll
focus on here.
Scries are performed with the [dotket](/reference/hoon/rune/dot#-dotket) (`.^`)
rune. Here's a summary of their format:
![scry summary diagram](https://storage.googleapis.com/media.urbit.org/docs/arvo/scry-diagram-v2.svg)
A note on `care`s: Cares are most carefully implemented by Clay, where they specify
submodules and have tightly defined behaviors. For Gall agents, most of these
don't have any special behavior, and are just used to indicate the general kind
of data produced by the endpoint. There are a handful of exceptions to this:
`%d`, `%e`, `%u` and `%x`.
#### `%d`
A scry to Gall with a `%d` `care` and no `path` will produce the `desk` in which
the specified agent resides. For example:
```
> .^(desk %gd /=hark-store=)
%garden
> .^(desk %gd /=hood=)
%base
```
#### `%e`
A scry to Gall with a `%e` `care`, a `desk` rather than agent in the `desk`
field of the above diagram, and no path, will produce a set of all installed
agents on that desk and their status. For example:
```
> .^((set [=dude:gall live=?]) %ge /=garden=)
{ [dude=%hark-system-hook live=%.y]
[dude=%treaty live=%.y]
[dude=%docket live=%.y]
[dude=%settings-store live=%.y]
[dude=%hark-store live=%.y]
}
```
#### `%u`
A scry to Gall with a `%u` `care` and no `path` will check whether or not the
specified agent is installed and running:
```
> .^(? %gu /=btc-wallet=)
%.y
> .^(? %gu /=btc-provider=)
%.n
> .^(? %gu /=foobar=)
%.n
```
A note on `care`s: Cares are most carefully implemented by Clay, where they
specify submodules and have tightly defined behaviors. For Gall agents, most of
these don't have any special behavior, and are just used to indicate the
general kind of data produced by the endpoint, with the exception of the `%x`
care:
#### `%x`
A scry to Gall with a `%x` `care` will be passed to the agent for handling. Gall
handles `%x` specially, and expects an extra field at the end of the `path` that
specifies the `mark` to return. Gall will take the data produced by the
specified endpoint and try to convert it to the given mark, crashing if the mark
conversion fails. The extra field specifying the mark is not passed through to
the agent itself. Here's a couple of examples:
Gall handles `%x` specially, and expects an extra field at the end of the
`path` that specifies the `mark` to return. Gall will take the data produced by
the specified endpoint and try to convert it to the given mark, crashing if the
mark conversion fails. The extra field specifying the mark is not passed
through to the agent itself. Here's a couple of examples:
```
> =store -build-file /=landscape=/sur/graph-store/hoon
@ -338,8 +296,9 @@ crash!
- Scries will fail if the scry endpoint does not exist, the requested data does
not exist, or the data does not nest in the return type specified.
- Scries can only be performed on the local ship, not on remote ships.
- Gall scries with an agent name in the `desk` field will be passed to that
agent's `on-peek` arm for handling.
- Gall scries with an agent name in the `desk` field and without an extra empty
element at the beginning of the path will be passed to that agent's `on-peek`
arm for handling.
- Gall scries with a `%x` `care` take a `mark` at the end of the scry `path`,
telling Gall to convert the data returned by the scry endpoint to the mark
specified.
@ -352,6 +311,7 @@ crash!
## Exercises
- Have a read through the [Scry Guide](/reference/arvo/concepts/scry).
- Have a look at Gall's [scry reference](/reference/arvo/gall/scry).
- Have a read through the [dotket rune
documentation](/reference/hoon/rune/dot#-dotket).
- Run through the [Example](#example) yourself if you've not done so already.

102
content/reference/arvo/ames/tasks.md
View File

@ -10,8 +10,8 @@ Some `task`s appear to have more than one arm associated to them, e.g. there are
four `+on-hear` arms. We denote this where it occurs, but always refer to the
`+on-hear:event-core` arm.
Ames `task`s can be naturally divided into two categories: messaging tasks and
system/lifecycle tasks.
Ames `task`s can be naturally divided into three categories: messaging tasks,
system/lifecycle tasks, and remote scry tasks.
## Messaging Tasks
@ -33,6 +33,8 @@ There are multiple `+on-hear` arms in `ames.hoon`. Here we refer to `+on-hear:ev
`%hear` can trigger a number of possible returns. It can trigger the release of zero or more additional packets via `%send` `gift`s. It may also trigger a `%boon` or `%plea` `gift` (collectively referred to as a `%memo` within Ames) to a local vane in the case of a completed message.
---
### `%heed`
```hoon
@ -51,6 +53,8 @@ If the `ship` is indeed being unresponsive, as measured by backed up `%boon`s,
Ames will `give` a `%clog` `gift` to the requesting vane containing the
unresponsive peer's urbit address.
---
### `%jilt`
```hoon
@ -68,6 +72,8 @@ The `ship` field specifies the peer we want to stop tracking.
This `task` returns no `gift`s.
---
### `%plea`
```hoon
@ -93,6 +99,8 @@ A `%plea` `task` takes in the `ship` the `plea` is addressed to, and a [$plea](/
This `task` returns no `gift`s.
---
## System Tasks
### `%born`
@ -111,6 +119,8 @@ The `duct` along which `%born` comes is Ames' only duct to Unix, so `%send`
`gift`s (which are instructions for Unix to send a packet) are also returned in
response to `%born`.
---
### `%init`
```hoon
@ -134,6 +144,8 @@ contained by Jael.
`%init` sends two moves that subscribe to `%turf` and `%private-keys` in Jael.
---
### `%sift`
```hoon
@ -148,6 +160,8 @@ The `ships` field specifies the ships for which debug output is desired.
This `task` returns no `gift`s.
---
### `%snub`
```hoon
@ -175,6 +189,12 @@ whole modified list and form in a new `%snub` `task`.
{% /callout %}
#### Returns
This `task` returns no `gift`s.
---
### `%spew`
```hoon
@ -191,6 +211,8 @@ Sets verbosity toggles on debug output. This `task` is used internally when the
This `task` returns no `gift`s.
---
### `%stir`
```hoon
@ -205,6 +227,8 @@ The `arg` field is unused.
This `task` returns no `gift`s.
---
### `%vega`
```hoon
@ -217,3 +241,77 @@ anything in response to this.
#### Returns
This `task` returns no `gift`s.
---
## Remote scry tasks
### `%keen`
```hoon
[%keen =ship =path]
```
A `%keen` `task` asks Ames to perform a remote scry, retrieving the value of
`path` on the given `ship`. The `path` has the general format of
`/[vane-letter]/[care]/[revision]/[rest-of-path]`. For a regular read into
Gall, it's `/g/x/[revision]/[agent]//[rest-of-path]`. Note the empty element in
between the agent and the rest of the path.
#### Returns
Either a `%tune` or `%miss` gift. A `%tune` gift looks like:
```hoon
[%tune =path sign=@ux data=(unit page)]
```
It represents a *result*. The `sign` is a signature from the publisher and the
`data` is the result itself, which will be null if the requested path at the
given revision doesn't exist and will never exist (equivalent to the `[~ ~]`
case of an ordinary scry).
A `%miss` gift looks like:
```hoon
[%miss =path]
```
It represents a failure to produce the value at the given path. This can
happen if the publisher doesn't know the answer, or if the signature
verification fails. This does *not* imply the same request will fail in the
future, unlike a `%tune` with null `data`. This is equivalent to the `~` case
of an ordinary scry.
---
### `%yawn`
```hoon
[%keen =ship =path]
```
A `%yawn` task asks Ames to cancel an existing remote scry request to the given
`path` on the given `ship`.
#### Returns
This `task` returns no `gift`s.
---
### `%wham`
```hoon
[%wham =ship =path]
```
A `%wham` task asks Ames to cancel all existing remote scry requests from all
vanes on all ducts for the given `path` on the given `ship`.
#### Returns
A `%tune` gift with a null `data` is given to all listeners. See the
[`%keen`](#keen) entry for more details of the `%tune` gift.
---

4
content/reference/arvo/concepts/scry.md
View File

@ -3,6 +3,10 @@ title = "Scries"
weight = 40
+++
{% callout %}
This document mostly covers local scries. Remote scries have recently been introduced, and are documented in a [separate guide](/guides/additional/remote-scry).
## What is a scry?
A scry is a read-only request to Arvo's global namespace.

7
content/reference/arvo/gall/data-types.md
View File

@ -218,6 +218,10 @@ types.
$% [%agent [=ship name=term] =task]
[%arvo note-arvo]
[%pyre =tang]
::
[%grow =spur =page]
[%tomb =case =spur]
[%cull =case =spur]
==
```
@ -251,6 +255,9 @@ vanes, or for sending out updates to subscribers. The three cases are:
documentation](/reference/arvo/overview)
- `%pyre`: This is for aborting side-effects initiated during agent
installation. The `tang` is an error message.
- `%grow`/`%tomb`/`%cull`: These are used for publishing and managing data
available for remote scries. For more information, see the [remote scries
guide](/guides/additional/remote-scry).
A `note:agent` is always wrapped in a `%pass` [`card:agent`](#cardagent).

230
content/reference/arvo/gall/scry.md Normal file
View File

@ -0,0 +1,230 @@
+++
title = "Scry Reference"
weight = 4
+++
Gall's scry interface is mainly used to scry into the individual agents it's
running. The vane itself does have its own interface, however. Both [agent
scries](#agent-scries) and [vane scries](#vane-scries) are documented below.
Note that for all agent scries and most vane scries, `q.beak`, where there'd
usually be a `desk`, will be the agent name instead, like:
```
.^(some-type %gx /=agent-name-here=/some/path/noun)
```
{% callout %}
**Important:** Vane scries are differentiated from agent scries by an extra
empty (`%$`) element at the beginning of the `spur` (the path after the
`beak`), like: `/=agent-name/$` or `/=agent-name=//some/more/fields`. Without
that empty element, Gall will try route the scry to an agent instead.
{% /callout %}
## Agent scries
In order to hit the `+on-peek` arm of a Gall agent, you need to:
1. Put the agent in `q.beak` of the scry path (where the `desk` usually goes), like `/=some-agent=`.
2. Make sure the beginning of the `spur` is *not* an empty `%$` element, as
that will route the scry to the [vane endpoints](#vane-scries) instead. An
agent scry must be `/=some-agent=/some/path` not `/=some-agent=//some/path`.
Any `care` can be used (dependent on what the agent accepts, of course). The
most common is `%x`.
{% callout %}
Note that `%x` cares alone must include an extra `mark` field at the end of the
`spur`. This mark field lets Gall perform any necessary mark conversions before
returning your data. For plain unmarked data, you can just use the `%noun`
mark. As an example, if an agent specifies an endpoint `/x/some/path` and just
returns ordinary data, you'd do `.^(some-type %gx
/=some-agent=/some/path/noun)`. If the endpoint returns a `%json` mark or
whatever (and that's what you want), you'd put that at the end instead.
{% /callout %}
---
## Vane scries
Gall itself provides the special vane-level endpoints listed below. They are
organized by the `care`. In order to hit the vane-level endpoints, the
beginning of the the `spur` (e.g. the `path` after the `beak`) *must* be a `%$`
empty element. For example:
```hoon
.^(desk %gd /=acme=/$)
.^((set [=dude:gall live=?]) %ge /=base=/$)
.^((list path) %gt /=acme=//foo)
```
Note you can use `$` to make the last element empty since it won't allow a
trailing `/`. Note how in the third example, the empty element is at the
*beginning* of the `spur` and *after* the `beak`. If you fail to include this
empty element, Gall will try route the scry to an agent for handling instead.
### `%d`: get desk of app
A scry with a `%d` care and an agent in `q.beak` will
give you the desk that agent is on.
#### Produces
A `desk`
#### Example
```
> .^(desk %gd /=acme=/$)
%base
```
---
### `%e`: running apps
A scry with an `%e` care will give you the agents on the desk given in
`q.beak`.
#### Produces
A `(set [=dude live=?])`, where `live` is true if running and false if not.
#### Examples
```
> .^((set [=dude:gall live=?]) %ge /=base=/$)
{ [dude=%acme live=%.y]
[dude=%hood live=%.y]
[dude=%lens live=%.y]
[dude=%dbug live=%.y]
[dude=%azimuth live=%.y]
[dude=%ping live=%.y]
[dude=%dojo live=%.y]
[dude=%eth-watcher live=%.y]
[dude=%spider live=%.y]
[dude=%herm live=%.y]
}
```
---
### `%f`: nonces of apps
A scry with a care of `%f` and anything in `q.beak` will produce the
subscription nonces of all apps. You are unlikely to use this, it's
mostly for kernel debugging.
#### Produces
A `(map dude @)` where the `@` is the nonce.
#### Examples
```
> .^((map dude:gall @) %gf /=//=/$)
[ n=[p=%treaty q=2]
l
[ n=[p=%metadata-hook q=1]
l={[p=%contacts q=1] [p=%notify q=2] [p=%groups q=1] [p=%dm-hook q=1] [p=%spider q=1]}
r
{ [p=%docket q=9]
[p=%bait q=1]
[p=%hood q=15]
[p=%hark-graph-hook q=2]
[p=%s3-store q=1]
[p=%hark-system-hook q=1]
......(truncated for brevity)......
```
---
### `%n`: get nonce of subscription
A scry with a care of `%n`, an agent in `q.beak` and a path of
`//[ship]/[agent]/[wire]` will produce the nonce for that subscription.
You are unlikely to use this, it's mostly for kernel debugging.
#### Produces
A `@`.
---
### `%t`: remote scry subpaths
A scry with a `%t` care, an agent in `q.beak` and a path of `//some/path`
will give you the list of remote scry subpaths bound under the given
path.
See the [remote scry guide](/guides/additional/remote-scry) for more
details.
#### Produces
A `(list path)`
#### Examples
```
> .^((list path) %gt /=acme=//foo)
~
```
---
### `%u`: check if installed
A scry with a `%u` care will check whether the given agent is installed and
running.
#### Produces
A `?`
#### Examples
```
> .^(? %gu /=acme=/$)
%.y
```
```
> .^(? %gu /=doesnt-exist=/$)
%.n
```
---
### `%w`: latest revision of path
A scry with a `%w` care and an agent in `q.beak` will get the latest revision
number of the bound remote scry path given in the `spur`.
See the [remote scry guide](/guides/additional/remote-scry) for more
details.
#### Produces
A `cass:clay`, specifically the `%ud` kind.
---
### `%z`: hash of value at path
A scry with a `%z` care and an agent in `q.beak` will get the hash identifier
of the value bound at the remote scry path given in the `spur`.
See the [remote scry guide](/guides/additional/remote-scry) for more
details.
#### Produces
A `@uvI`.
---

32
content/reference/glossary/remote-scry.md Normal file
View File

@ -0,0 +1,32 @@
+++
title = "Scry"
[extra]
category = "hoon-nock"
[glossaryEntry."Remote scry"]
name = "remote scry"
symbol = ""
usage = "hoon-nock"
desc = "A remote scry is a read-only request to the namespace of a remote ship."
+++
A **remote scry** is a read-only request to the namespace of a remote
[vane](/reference/glossary/vane) or [agent](/reference/glossary/agent). These
differ from ordinary [local scries](/reference/glossary/scry) and are not
performed with the `.^` [rune](/reference/glossary/rune).
Remote scries reduce event log bloat on the publishing ship, allow the
publisher's runtime to cache data, and especially improve performance when
publishing the same data for many ships to retrieve.
At the the time of writing, Gall allows agents to bind data to remote scry
paths and perform remote scries with `task`s to
[Ames](/reference/glossary/ames). Additionally, Clay uses remote scries
internally to sync remote desks.
#### Further Reading
- [Remote scry guide](/guides/additional/remote-scry): developer documentation
of how remote scries work and how to use them.