on master help sign in
Merge pull request #1391 from urbit/add-full-stack-walkthrough

add full-stack walk-through

commitc0c5945c00ec5afdf1099ef7a269f4e0e5c1cc6f authortinnus-napbus <77721746+tinnus-napbus@users.noreply.github.com> dateWed 13 Apr 2022, 09:43 (5 years ago) parent1e0f50c1333c46
17 files changed+2218−18
content/docs/arvo/clay/marks/_index.md+1−1 content/docs/arvo/clay/marks/marks.md+1−1 content/docs/userspace/dist/dist.md+1−1 content/docs/userspace/full-stack/1-intro.md+89 content/docs/userspace/full-stack/10-final.md+118 content/docs/userspace/full-stack/2-types.md+215 content/docs/userspace/full-stack/3-agent.md+338 content/docs/userspace/full-stack/4-json.md+357 content/docs/userspace/full-stack/5-marks.md+97 content/docs/userspace/full-stack/6-eyre.md+79 content/docs/userspace/full-stack/7-react-setup.md+214 content/docs/userspace/full-stack/8-app-logic.md+406 content/docs/userspace/full-stack/9-desk.md+226 content/docs/userspace/full-stack/_index.md+52 content/docs/userspace/gall-guide/10-scry.md+5−4 content/docs/userspace/gall-guide/12-next-steps.md+11−3 content/docs/userspace/gall-guide/7-sur-and-marks.md+8−8
content/docs/arvo/clay/marks/_index.md+1−1
@@ -12,7 +12,7 @@ Overview of `mark`s.
1212
1313[Writing Marks](/docs/arvo/clay/marks/writing-marks)
1414
15−A practical walk-through of writing a `mark` file.
15+A practical walkthrough of writing a `mark` file.
1616
1717[Using Marks](/docs/arvo/clay/marks/using-marks)
1818
content/docs/arvo/clay/marks/marks.md+1−1
@@ -52,7 +52,7 @@ The second case of interest is the `+json` arm for converting to and from a `%js
5252
5353## Sections
5454
55−[Writing Marks](/docs/arvo/clay/marks/writing-marks) - A practical walk-through of writing a `mark` file.
55+[Writing Marks](/docs/arvo/clay/marks/writing-marks) - A practical walkthrough of writing a `mark` file.
5656
5757[Using Marks](/docs/arvo/clay/marks/using-marks) - Details of using `mark` conversion gates and `mark` cores in your own code.
5858
content/docs/userspace/dist/dist.md+1−1
@@ -6,7 +6,7 @@ template = "doc.html"
66
77Urbit allows peer-to-peer distribution and installation of applications. A user can click on a link to an app hosted by another ship to install that app. The homescreen interface lets users manage their installed apps and launch their interfaces in new tabs.
88
9−This document describes the architecture of Urbit's app distribution system. For a walk-through of creating and distributing an app, see the [`Guide`](/docs/userspace/dist/guide) document.
9+This document describes the architecture of Urbit's app distribution system. For a walkthrough of creating and distributing an app, see the [`Guide`](/docs/userspace/dist/guide) document.
1010
1111## Architecture
1212
content/docs/userspace/full-stack/1-intro.mdnew file+89
@@ -0,0 +1,89 @@
1++++
2+title = "1. Introduction"
3+weight = 1
4+template = "doc.html"
5++++
6+
7+This series walks through the writing of a full Gall agent, and then the process
8+of integrating it with a React front-end. This series follows on from the
9+previous [Gall Guide](/docs/userspace/gall-guide/intro). If you haven't
10+completed that, or otherwise aren't familiar with the basics of writing Gall
11+agents, it's strongly recommended to work through that guide first.
12+
13+The app we'll be looking at is a simple journal with an agent called `%journal`.
14+In the browser, users will be able to add plain text journal entries organized
15+by date. Entries may be scrolled through in ascending date order, with more
16+entries loaded each time the bottom of the list is reached. Old entries will be
17+able to be edited and deleted, and users will be able to search through entries
18+by specifying a date range.
19+
20+The `Journal` app we'll be looking at can be installed on a live ship from
21+`~pocwet/journal`, and its source code is available [here](https://github.com/urbit/docs-examples/tree/main/journal-app).
22+
23+![journal ui screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/entries.png)
24+
25+This walkthrough does not contain exercises, nor does it completely cover every
26+aspect of building the app in full depth. Rather, its purpose is to demonstrate
27+the process of creating a full-stack Urbit app, showing how everything fits
28+together, and how concepts you've previously learned are applied in practice.
29+
30+The primary focus of the walkthrough is to show how a Javascript front-end is
31+integrated with a Gall agent and distributed as a complete app. Consequently,
32+the example app is fairly simple and runs on a local ship only, rather than one
33+with more complex inter-ship networking.
34+
35+Each section of this walkthrough will list additional resources and learning
36+material at the bottom of the page, which will cover the concepts discussed in a
37+more comprehensive manner.
38+
39+Here is the basic structure of the app we'll be building:
40+
41+![journal app
42+diagram](https://media.urbit.org/docs/userspace/full-stack-guide/journal-app-diagram.svg)
43+
44+## Sections
45+
46+#### [1. Introduction](/docs/userspace/gall-2/1-intro)
47+
48+An overview of the guide and table of contents.
49+
50+#### [2. Types](/docs/userspace/gall-2/2-types)
51+
52+Creating the `/sur` structure file for our `%journal` agent.
53+
54+#### [3. Agent](/docs/userspace/gall-2/3-agent)
55+
56+Creating the `%journal` agent itself.
57+
58+#### [4. JSON](/docs/userspace/gall-2/5-json)
59+
60+Writing a library to convert between our agent's marks and JSON. This lets our
61+React front-end poke our agent, and our agent send updates back to it.
62+
63+#### [5. Marks](/docs/userspace/gall-2/4-marks)
64+
65+Creating the mark files for the pokes our agent takes and updates it sends out.
66+
67+#### [6. Eyre](/docs/userspace/gall-2/6-eyre)
68+
69+A brief overview of how the webserver vane Eyre works.
70+
71+#### [7. React App Setup](/docs/userspace/gall-2/7-react-setup)
72+
73+Creating a new React app, installing the required packages, and setting up some
74+basic things for our front-end.
75+
76+#### [8. React App Logic](/docs/userspace/gall-2/8-http-api)
77+
78+Analyzing the core logic of our React app, with particular focus on using
79+methods of the `Urbit` class from `@urbit/http-api` to communicate with our
80+agent.
81+
82+#### [9. Desk and Glob](/docs/userspace/gall-2/9-web-scries)
83+
84+Building and "globbing" our front-end, and putting together a desk for
85+distribution.
86+
87+#### [10. Summary](/docs/userspace/gall-2/10-final)
88+
89+Some final comments and additional resources.
content/docs/userspace/full-stack/10-final.mdnew file+118
@@ -0,0 +1,118 @@
1++++
2+title = "10. Summary"
3+weight = 10
4+template = "doc.html"
5++++
6+
7+That's it! We've built our agent and React front-end, put together a desk and
8+published it. We hope this walkthrough has helped you see how all the pieces
9+for together for building and distributing an app in Urbit.
10+
11+The reference material for each section of this walkthrough is listed
12+[below](#reference-material), the source code for our app is available
13+[here](https://github.com/urbit/docs-examples/tree/main/journal-app), and it can
14+be installed from `~pocwet/journal`.
15+
16+In this guide we've built a separate React app for the front-end, but Hoon also
17+has a native domain-specific language for composing HTML structures called Sail.
18+Sail allows you to compose a front-end inside a Gall agent and serve it
19+directly. See the [Sail guide](/docs/hoon/guides/sail) for details.
20+
21+Along with `@urbit/http-api`, there's also the `@urbit/api` NPM package, which
22+contains a large number of helpful functions for dealing with Hoon data types
23+and interacting with a number of agents - particularly those used by the Groups
24+app. Its source code is [available
25+here](https://github.com/urbit/urbit/tree/master/pkg/npm/api).
26+
27+## Reference material
28+
29+Here is the reference material for each section of this walkthrough.
30+
31+#### Types
32+
33+- [Gall Guide /sur section](/docs/userspace/gall-guide/7-sur-and-marks#sur) -
34+ This section of the Gall Guide covers writing a `/sur` structure library for
35+ an agent.
36+
37+- [Ordered map functions in
38+ `zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284-L5688) -
39+ This section of `zuse.hoon` contains all the functions for working with
40+ `mop`s, and is well commented.
41+
42+#### Agent
43+
44+- [The Gall Guide](/docs/userspace/gall-guide/intro) - The Gall Guide covers all
45+ aspects of writing Gall agents in detail.
46+
47+- [Ordered map functions in
48+ `zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284-L5688) -
49+ This section of `zuse.hoon` contains all the functions for working with
50+ `mop`s, and is well commented.
51+
52+- [`/lib/agentio.hoon`](https://github.com/urbit/urbit/blob/master/pkg/base-dev/lib/agentio.hoon) -
53+ The `agentio` library in the `%base` desk contains a large number of useful
54+ functions which making writing Gall agents easier.
55+
56+#### JSON
57+
58+- [The JSON Guide](/docs/hoon/guides/json-guide) - The stand-alone JSON guide
59+ covers JSON encoding/decoding in great detail.
60+- [The Zuse Reference](/docs/hoon/reference/zuse/table-of-contents) - The
61+ `zuse.hoon` reference documents all JSON-related functions in detail.
62+
63+- [`++enjs:format` reference](/docs/hoon/reference/zuse/2d_1-5#enjsformat) -
64+ This section of the `zuse.hoon` documentation covers all JSON encoding
65+ functions.
66+
67+- [`++dejs:format` reference](/docs/hoon/reference/zuse/2d_6) - This section of
68+ the `zuse.hoon` documentation covers all JSON _decoding_ functions.
69+
70+- [Eyre Overview](/docs/arvo/eyre/eyre) - This section of the Eyre vane
71+ documentation goes over the basic features of the Eyre vane.
72+
73+#### Marks
74+
75+- [The Marks section of the Clay documentation](/docs/arvo/clay/marks/marks) -
76+ This section of the Clay vane documentation covers mark files comprehensively.
77+- [The mark file section of the Gall
78+ Guide](/docs/userspace/gall-guide/7-sur-and-marks#mark-files) - This part of
79+ the Gall Guide goes through the basics of mark files.
80+
81+- [The JSON Guide](/docs/hoon/guides/json-guide) - This also covers writing mark
82+ files to convert to/from JSON.
83+
84+#### Eyre
85+
86+- [The Eyre vane documentation](/docs/arvo/eyre/eyre) - This section of the vane
87+ docs covers all aspects of Eyre.
88+- [Eyre External API Reference](/docs/arvo/eyre/external-api-ref) - This section
89+ of the Eyre documentation contains reference material for Eyre's external API.
90+
91+- [The Eyre Guide](/docs/arvo/eyre/guide) - This section of the Eyre
92+ documentation walks through using Eyre's external API at a low level (using
93+ `curl`).
94+
95+#### React App Setup and Logic
96+
97+- [HTTP API Guide](/docs/userspace/http-api-guide) - Reference documentation for
98+ `@urbit/http-api`.
99+
100+- [React app source
101+ code](https://github.com/urbit/docs-examples/tree/main/journal-app/ui) - The
102+ source code for the Journal app UI.
103+
104+- [`@urbit/http-api` source
105+ code](https://github.com/urbit/urbit/tree/master/pkg/npm/http-api) - The
106+ source code for the `@urbit/http-api` NPM package.
107+
108+#### Desk and Glob
109+
110+- [App publishing/distribution docs](/docs/userspace/dist/dist) -
111+ Documentation covering third party desk composition, publishing and
112+ distribution.
113+
114+- [Glob documentation](/docs/userspace/dist/glob) - Comprehensive documentation
115+ of handling front-end files.
116+
117+- [Desk publishing guide](/docs/userspace/dist/guide) - A step-by-step guide to
118+ creating and publishing a desk.
content/docs/userspace/full-stack/2-types.mdnew file+215
@@ -0,0 +1,215 @@
1++++
2+title = "2. Types"
3+weight = 2
4+template = "doc.html"
5++++
6+
7+The best place to start when building a new agent is its type definitions in its
8+`/sur` structure file. The main things to think through are:
9+
10+1. What basic types of data does my agent deal with?
11+2. What actions/commands does my agent need to handle?
12+3. What updates/events will my agent need to send out to subscribers?
13+4. What does my agent need to store in its state?
14+
15+Let's look at each of these questions in turn, and put together our agent's
16+`/sur` file, which we'll call `/sur/journal.hoon`.
17+
18+### 1. Basic types
19+
20+Our journal entries will just be plain text, so a simple `@t` will work fine to
21+store their contents. Entries will be organized by date, so we'll also need to
22+decide a format for that.
23+
24+One option would be to use an `@da`, and then use the date functions included
25+in the `@urbit/api` NPM package on the front-end to convert them to ordinary
26+Javascript `Date` objects. In this case, to keep it simple, we'll just use the
27+number of milliseconds since the Unix Epoch as an `atom`, since it's natively
28+supported by the Javascript `Date` object.
29+
30+The structure for a journal entry can therefore be:
31+
32+```hoon
33++$ id @
34++$ txt @t
35++$ entry [=id =txt]
36+```
37+
38+### 2. Actions
39+
40+Now that we know what a journal entry looks like, we can think about what kind
41+of actions/commands our agent will handle in its `++on-poke` arm. For our
42+journal app, there are three basic things we might do:
43+
44+1. Add a new journal entry.
45+2. Edit an existing journal entry.
46+3. Delete an existing journal entry.
47+
48+We can create a tagged union structure for these actions, like so:
49+
50+```hoon
51++$ action
52+ $% [%add =id =txt]
53+ [%edit =id =txt]
54+ [%del =id]
55+ ==
56+```
57+
58+### 3. Updates
59+
60+Updates are a little more complicated than our actions. Firstly, our front-end
61+needs to be able to retrieve an initial list of journal entries to display. Once
62+it has that, it also needs to be notified of any changes. For example, if a new
63+entry is added, it needs to know so it can add it to the list it's displaying.
64+If an entry gets deleted, it needs to remove it from the list. Etc.
65+
66+The simplest approach to the initial entries is just a `(list entry)`. Then, for
67+the subsequent updates, we could send out the `$action`. Since an `$action` is a
68+tagged union, it's simpler to have all updates be a tagged union, so when we get
69+to doing mark conversions we can just switch on the head tag. Therefore, we can
70+define an `$update` structure like so:
71+
72+```hoon
73++$ update
74+ $% action
75+ [%jrnl list=(list entry)]
76+ ==
77+```
78+
79+There's one drawback to this structure. Suppose either an agent on a remote ship
80+or an instance of the front-end client is subscribed for updates, and the
81+network connection is disrupted. In the remote ship case, Gall will only allow
82+so many undelivered messages to accumulate in Ames before it automatically kicks
83+the unresponsive subscriber. In the front-end case, the subscription will also
84+be ended if enough unacknowledged messages accumulate, and additionally the
85+client may sometimes need to establish an entirely new connection with the ship,
86+discarding existing subscriptions. When this happens, the remote ship or web
87+client has no way to know how many (if any) updates they've missed.
88+
89+The only way to resynchronize their state with ours is to discard their existing
90+state, refetch the entire initial state once again, and then resubscribe for
91+updates. This might be fine if the state of our agent is small, but it becomes a
92+problem if it's very large. For example, if our agent holds tens of thousands of
93+chat messages, having to resend them all every time anyone has connectivity
94+issues is quite inefficient.
95+
96+One solution to this is to keep an _update log_. Each update can be tagged with
97+the time it occurred, and stored in our agent's state, separately to the
98+entries. If an agent or web client needs to resynchronize with our agent, it can
99+just request all updates since the last one it received. This approach is used
100+by the `%graph-store` agent, for example. Our agent is local-only and doesn't
101+have a huge state so it might not be strictly necessary, but we'll use it to
102+demonstrate the approach.
103+
104+We can define a logged update like so, where the `@` is the update timestamp in
105+milliseconds since the Unix Epoch:
106+
107+```hoon
108++$ logged (pair @ action)
109++$ update
110+ %+ pair @
111+ $% action
112+ [%jrnl list=(list entry)]
113+ [%logs list=(list logged)]
114+ ==
115+```
116+
117+### 4. State
118+
119+We need to store two things in our state: the journal entries and the update
120+log. We could just use a couple of `map`s like so:
121+
122+```hoon
123++$ journal (map id txt)
124++$ log (map @ action)
125+```
126+
127+Ordinary `map`s are fine if we just want to access one value at a time, but we
128+want to be able to:
129+
130+1. Retrieve only some of the journal entries at a time, so we can have "lazy
131+ loading" in the front-end, loading more entries each time the user scrolls to
132+ the bottom of the list.
133+2. Retrieve only logged updates newer than a certain time, in the case where the
134+ subscription is interrupted due to connectivity issues.
135+3. Retrieve journal entries between two dates.
136+
137+Maps are ordered by the hash of their key, so if we convert them to a list
138+they'll come out in seemingly random order. That means we'd have to convert the
139+map to a list, sort the list, and then iterate over it again to pull out the
140+items we want. We could alternatively store things in a list directly, but
141+retrieving or modifying arbitrary items would be less efficient.
142+
143+To solve this, rather than using a `map` or a `list`, we can use an _ordered
144+map_. The mold builder for an ordered map is a `mop`, and it's included in the
145+[`zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284)
146+utility library rather than the standard library.
147+
148+A `mop` is defined similarly to a `map`, but it takes an extra argument in the
149+following manner:
150+
151+```hoon
152+((mop key-mold val-mold) comparator-gate)
153+```
154+
155+The gate is a binary gate which takes two keys and produces a `?`. The
156+comparator is used to decide how to order the items in the mop. In our case,
157+we'll create a `$journal` and `$log` `mop` like so:
158+
159+```hoon
160++$ journal ((mop id txt) gth)
161++$ log ((mop @ action) lth)
162+```
163+
164+The entries in `$journal` are arranged in ascending time order using `++gth`, so
165+the right-most item is the newest. The `$log` `mop` contains the update log, and
166+is arranged in descending time order, so the right-most item is the oldest.
167+
168+We'll look at how to use ordered maps later when we get to writing the agent
169+itself.
170+
171+## Conclusion
172+
173+When we put each of these parts together, we have our complete
174+`/sur/journal.hoon` file:
175+
176+```hoon
177+|%
178+:: Basic types of the data we're dealing with
179+::
180++$ id @
181++$ txt @t
182++$ entry [=id =txt]
183+:: Poke actions
184+::
185++$ action
186+ $% [%add =id =txt]
187+ [%edit =id =txt]
188+ [%del =id]
189+ ==
190+:: Types for updates to subscribers or returned via scries
191+::
192++$ logged (pair @ action)
193++$ update
194+ %+ pair @
195+ $% action
196+ [%jrnl list=(list entry)]
197+ [%logs list=(list logged)]
198+ ==
199+:: Types for our agent's state
200+::
201++$ journal ((mop id txt) gth)
202++$ log ((mop @ action) lth)
203+--
204+```
205+
206+## Resources
207+
208+- [Gall Guide /sur section](/docs/userspace/gall-guide/7-sur-and-marks#sur) -
209+ This section of the Gall Guide covers writing a `/sur` structure library for
210+ an agent.
211+
212+- [Ordered map functions in
213+ `zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284-L5688) -
214+ This section of `zuse.hoon` contains all the functions for working with
215+ `mop`s, and is well commented.
content/docs/userspace/full-stack/3-agent.mdnew file+338
@@ -0,0 +1,338 @@
1++++
2+title = "3. Agent"
3+weight = 3
4+template = "doc.html"
5++++
6+
7+Now that we have our agent's types defined and have thought through its
8+behavior, we can write the `%journal` agent itself.
9+
10+## Imports
11+
12+```hoon
13+/- *journal
14+/+ default-agent, dbug, agentio
15+```
16+
17+We first import the `/sur/journal.hoon` file we previously created and expose
18+its structures. We import the standard `default-agent` and `dbug`, and also an additional library called `agentio`.
19+
20+Agentio contains a number of convenience functions to make common agent tasks
21+simpler. For example, rather than writing out the full `$card`s when sending
22+`%fact`s to subscribers, we can call `++fact` in `agentio` with the `cage` and
23+`path`s and it will compose them for us. There are many more functions in
24+`agentio` than we'll use here - you can have a look through the library in
25+[`/base/lib/agentio.hoon`](https://github.com/urbit/urbit/blob/master/pkg/base-dev/lib/agentio.hoon)
26+to see what else it can do.
27+
28+## State and type core
29+
30+```hoon
31+|%
32++$ versioned-state
33+ $% state-0
34+ ==
35++$ state-0 [%0 =journal =log]
36++$ card card:agent:gall
37+++ j-orm ((on id txt) gth)
38+++ log-orm ((on @ action) lth)
39+++ unique-time
40+ |= [=time =log]
41+ ^- @
42+ =/ unix-ms=@
43+ (unm:chrono:userlib time)
44+ |-
45+ ?. (has:log-orm log unix-ms)
46+ unix-ms
47+ $(time (add unix-ms 1))
48+--
49+```
50+
51+As we discussed in the previous section, our state will contain a `$journal`
52+structure containing all our journal entries, and a `$log` structure containing
53+the update log. These are both _ordered maps_, defined as `((mop id txt) gth)`
54+and `((mop @ action) lth)` respectively. We can therefore define our _versioned
55+state_ as `[%0 =journal =log]`, in the usual manner.
56+
57+We've define `$card` for convenience as usual, and we've also added three more
58+arms. The first two relate to our two ordered maps. If you'll recall, an
59+ordinary `map` is called with the `++by` door in the standard library, like so:
60+
61+```hoon
62+(~(get by foo) %bar)
63+```
64+
65+An ordered map uses the `++on` gate in `zuse.hoon` rather than `++by`, and its
66+invocation is slightly different. It must first be setup in a similar manner to
67+the `mop` type, by providing it the key/value molds and comparator gates. Once
68+that's done, its individual functions can be called with the `mop` and
69+arguments, like:
70+
71+```hoon
72+(get:((on @ud @ud) gth) foo %bar)
73+```
74+
75+This is quite a cumbersome expression to use every time we want to interact with
76+our `mop`. To make it easier, we can store the `((on @ud @ud) gth)` part in an
77+arm, and then when we need to use it we can just do `(get:arm-name foo %bar)`.
78+In this case, we've done one each of our ordered maps like so:
79+
80+```hoon
81+++ j-orm ((on id txt) gth)
82+++ log-orm ((on @ action) lth)
83+```
84+
85+The last arm in our state definition core is `++unique-time`. Since we'll use
86+`now.bowl` to derive the timestamp for updates, we run into an issue if multiple
87+pokes arrive in a single Arvo event. In that case, `now.bowl` would be the same
88+for each poke, so they'd be given the same key and override each other in the
89+`mop`. To avoid this, `++unique-time` is just a simple recursive function that
90+will increment the timestamp by one millisecond if the key already exists in the
91+`$log` `mop`, ensuring all updates get unique timestamps and there are no
92+collisions.
93+
94+## Agent core setup
95+
96+```hoon
97+%- agent:dbug
98+=| state-0
99+=* state -
100+^- agent:gall
101+|_ =bowl:gall
102++* this .
103+ def ~(. (default-agent this %|) bowl)
104+ io ~(. agentio bowl)
105+++ on-init on-init:def
106+++ on-save
107+ ^- vase
108+ !>(state)
109+::
110+++ on-load
111+ |= old-vase=vase
112+ ^- (quip card _this)
113+ `this(state !<(versioned-state old-vase))
114+::
115+```
116+
117+Here we setup our agent core and define the three lifecycle arms. Since we only
118+have a single state version at present, these are very simple functions. You'll
119+notice in our `+*` arm, along with the usual `this` and `def`, we've also setup
120+the `agentio` library we imported, giving it the bowl and an alias of `io`.
121+
122+## Pokes
123+
124+```hoon
125+++ on-poke
126+ |= [=mark =vase]
127+ ^- (quip card _this)
128+ |^
129+ ?> (team:title our.bowl src.bowl)
130+ ?. ?=(%journal-action mark) (on-poke:def mark vase)
131+ =/ now=@ (unique-time now.bowl log)
132+ =/ act !<(action vase)
133+ =. state (poke-action act)
134+ :_ this(log (put:log-orm log now act))
135+ ~[(fact:io journal-update+!>(`update`[now act]) ~[/updates])]
136+ ::
137+ ++ poke-action
138+ |= act=action
139+ ^- _state
140+ ?- -.act
141+ %add
142+ ?< (has:j-orm journal id.act)
143+ state(journal (put:j-orm journal id.act txt.act))
144+ ::
145+ %edit
146+ ?> (has:j-orm journal id.act)
147+ state(journal (put:j-orm journal id.act txt.act))
148+ ::
149+ %del
150+ ?> (has:j-orm journal id.act)
151+ state(journal +:(del:j-orm journal id.act))
152+ ==
153+ --
154+::
155+```
156+
157+Here we have our `++on-poke` arm, where we handle `$action`s. Since our
158+`%journal` agent is intended for local use only, we make sure only our ship or
159+our moons may perform actions with:
160+
161+```hoon
162+?> (team:title our.bowl src.bowl)
163+```
164+
165+We haven't yet written our mark files, but our mark for `$action`s will be
166+`%journal-action`, so we make sure that's what we've received and if not, call
167+`++on-poke:def` to crash with an error message. We make sure the the timestamps
168+are unique with our `++unique-time` function described earlier, and then we
169+extract the poke's vase to an `$action` structure and call `++poke-action` to
170+handle it. We've made `++on-poke` a door with a separate `++poke-action` arm to
171+make the logic a little simpler, but in principle we could have had it all
172+directly inside the main `++poke-action` gate, or even separated it out into a
173+helper core below.
174+
175+The logic in `++poke-action` is very simple, with three cases for each of the possible `$action`s:
176+
177+- `%add` - Add a new journal entry. We check it doesn't already exist with
178+ `++has:j-orm`, and then add it to our `$journal` with `++put:j-orm`.
179+
180+- `%edit` - Edit an existing journal entry. We make sure it _does_ exist with
181+ `++has:j-orm`, and then override the old entry with the new one using
182+ `++put:j-orm` again.
183+
184+- `%del` - Delete an existing journal entry. We make sure it exists again with
185+ `++has:j-orm`, and then use `++del:j-orm` to delete it from our `$journal`
186+ `mop`.
187+
188+Back in the main part of `++on-poke`, `++poke-action` updates the state with the
189+new `$journal`, then we proceed to:
190+
191+```hoon
192+:_ this(log (put:log-orm log now act))
193+~[(fact:io journal-update+!>(`update`[now act]) ~[/updates])]
194+```
195+
196+We add the timestamp to the action, converting it to a logged update. We add it
197+to the `$log` update log using `++put:log-orm`, and also send the logged update
198+out to subscribers on the `/updates` subscription path. We haven't written our
199+mark files yet, but `%journal-update` is the mark we'll use for `$update`s, so
200+we pack the `$update` in a vase and add the mark to make it a `$cage`. Notice
201+we're using the `++fact` function in `agentio` (which we aliased as `io`) rather
202+than manually composing the `%fact`.
203+
204+## Subscriptions
205+
206+```hoon
207+++ on-watch
208+ |= =path
209+ ^- (quip card _this)
210+ ?> (team:title our.bowl src.bowl)
211+ ?+ path (on-watch:def path)
212+ [%updates ~] `this
213+ ==
214+::
215+```
216+
217+Our subscription logic is extremely simple - we just have a single `/updates`
218+path, which the front-end or other local agents may subscribe to. All updates
219+get sent out on this path. We enforce local-only with the `team:title` check.
220+
221+We could have had our `++on-watch` arm send out some initial state to new
222+subscribers, but for our front-end we'll instead fetch the initial state
223+separately with a scry. This just makes it slightly easier if our front-end
224+needs to resubscribe at some point - it'll already have some state in that case
225+so we don't want it to get sent again.
226+
227+## Scry Endpoints
228+
229+```hoon
230+++ on-peek
231+ |= =path
232+ ^- (unit (unit cage))
233+ ?> (team:title our.bowl src.bowl)
234+ =/ now=@ (unm:chrono:userlib now.bowl)
235+ ?+ path (on-peek:def path)
236+ [%x %entries *]
237+ ?+ t.t.path (on-peek:def path)
238+ [%all ~]
239+ :^ ~ ~ %journal-update
240+ !> ^- update
241+ [now %jrnl (tap:j-orm journal)]
242+ ::
243+ [%before @ @ ~]
244+ =/ before=@ (rash i.t.t.t.path dem)
245+ =/ max=@ (rash i.t.t.t.t.path dem)
246+ :^ ~ ~ %journal-update
247+ !> ^- update
248+ [now %jrnl (tab:j-orm journal `before max)]
249+ ::
250+ [%between @ @ ~]
251+ =/ start=@
252+ =+ (rash i.t.t.t.path dem)
253+ ?:(=(0 -) - (sub - 1))
254+ =/ end=@ (add 1 (rash i.t.t.t.t.path dem))
255+ :^ ~ ~ %journal-update
256+ !> ^- update
257+ [now %jrnl (tap:j-orm (lot:j-orm journal `end `start))]
258+ ==
259+ ::
260+ [%x %updates *]
261+ ?+ t.t.path (on-peek:def path)
262+ [%all ~]
263+ :^ ~ ~ %journal-update
264+ !> ^- update
265+ [now %logs (tap:log-orm log)]
266+ ::
267+ [%since @ ~]
268+ =/ since=@ (rash i.t.t.t.path dem)
269+ :^ ~ ~ %journal-update
270+ !> ^- update
271+ [now %logs (tap:log-orm (lot:log-orm log `since ~))]
272+ ==
273+ ==
274+::
275+```
276+
277+Here we have our `++on-scry` arm. The scry endpoints we've defined are divided
278+into two parts: querying the update `$log` and retrieving entries from the
279+`$journal`. Each end-point is as follows:
280+
281+- `/x/entries/all` - Retrieve all entries in the `$journal`. Our front-end will
282+ use lazy-loading and only get a few at a time, so it won't use this. It's nice
283+ to have it though, in case other agents want to get that data.
284+
285+- `/x/entries/before/[before]/[max]` - Retrieve at most `[max]` entries older
286+ than the entry on `[before]` date. This is so our lazy-loading front-end can
287+ progressively load more as the user scrolls down the page. The Javascript
288+ front-end will format numbers without dot separators, so the path will look
289+ like `/x/entries/before/1648051573109/10`. We therefore have to use the
290+ [`++dem`](docs/hoon/reference/stdlib/4i#dem) parsing `rule` in a
291+ [`++rash`](/docs/hoon/reference/stdlib/4g#rash) parser to convert it to an
292+ ordinary atom. We then use the `++tap:log-orm` `mop` function to retrieve the
293+ requested range as a list and return it as an `$update` with a
294+ `%journal-update` mark.
295+
296+- `/x/entries/between/[start]/[end]` - Retrieve all journal entries between two
297+ dates. This is so our front-end can have a search function, where the user can
298+ enter a start and end date and get all the entries in between. The
299+ `++lot:j-orm` `mop` function returns the subset of a `mop` between the two
300+ given keys as a `mop`, and then we call `++tap:j-orm` to convert it to a list.
301+ The `++lot:j-orm function` excludes the start and end values, so we subtract 1
302+ from the start and add 1 to the end to make sure it includes the full range.
303+
304+- `/x/updates/all` - Retrieve the entire update `$log`. Our front-end won't use
305+ this but it might be useful for other agents, so we've included it here.
306+
307+- `/x/updates/since/[since]` - Retrieve all `$update`s that have happened since
308+ the specified timestamp, if any. This is so our front-end (or another agent)
309+ can resynchronize its state in the event its subscription is interrupted,
310+ without having to fetch everything from scratch again.
311+
312+We don't use any of the other agent arms, so the remainder have all been passed
313+to `default-agent` for handling:
314+
315+```hoon
316+++ on-leave on-leave:def
317+++ on-agent on-agent:def
318+++ on-arvo on-arvo:def
319+++ on-fail on-fail:def
320+--
321+```
322+
323+The full agent source can be viewed
324+[here](https://github.com/urbit/docs-examples/blob/main/journal-app/desk/app/journal.hoon).
325+
326+## Resources
327+
328+- [The Gall Guide](/docs/userspace/gall-guide/intro) - The Gall Guide covers all
329+ aspects of writing Gall agents in detail.
330+
331+- [Ordered map functions in
332+ `zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284-L5688) -
333+ This section of `zuse.hoon` contains all the functions for working with
334+ `mop`s, and is well commented.
335+
336+- [`/lib/agentio.hoon`](https://github.com/urbit/urbit/blob/master/pkg/base-dev/lib/agentio.hoon) -
337+ The `agentio` library in the `%base` desk contains a large number of useful
338+ functions which making writing Gall agents easier.
content/docs/userspace/full-stack/4-json.mdnew file+357
@@ -0,0 +1,357 @@
1++++
2+title = "4. JSON"
3+weight = 4
4+template = "doc.html"
5++++
6+
7+Data sent between our agent and our front-end will all be encoded as JSON. In
8+this section, we'll briefly look at how JSON works in Urbit, and write a library
9+to convert our agent's structures to and from JSON for our front-end.
10+
11+JSON data comes into Eyre as a string, and Eyre parses it with the
12+[`++de-json:html`](/docs/hoon/reference/zuse/2e_2-3#de-jsonhtml) function in
13+[`zuse.hoon`](/docs/hoon/reference/zuse/table-of-contents). The
14+hoon type it's parsed to is `$json`, which is defined as:
15+
16+```hoon
17++$ json :: normal json value
18+ $@ ~ :: null
19+ $% [%a p=(list json)] :: array
20+ [%b p=?] :: boolean
21+ [%o p=(map @t json)] :: object
22+ [%n p=@ta] :: number
23+ [%s p=@t] :: string
24+ == ::
25+```
26+
27+Once Eyre has converted the raw JSON string to a `$json` structure, it will be
28+converted to the mark the web client specified and then delivered to the target
29+agent (unless the mark specified is already `%json`, in which case it will be
30+delivered directly). Outbound facts will go through the same process in
31+reverse - converted from the agent's native mark to `$json`, then encoded in a
32+string by Eyre using
33+[`++en-json:html`](<(/docs/hoon/reference/zuse/2e_2-3#en-jsonhtml)>) and delivered
34+to the web client. The basic flow for both inbound messages (pokes) and outbound
35+messages (facts and scry results) looks like this:
36+
37+![eyre mark flow diagram](https://media.urbit.org/docs/userspace/full-stack-guide/eyre-mark-flow-diagram.svg)
38+
39+The mark conversion will be done by the corresponding mark file in `/mar` on the
40+agent's desk. In our case it would be `/mar/journal/action.hoon` and
41+`/mar/journal/update.hoon` in the `%journal` desk for our `%journal-action` and
42+`%journal-update` marks, which are for the `$action` and `$update` structures we
43+defined previously.
44+
45+Mark conversion functions can be included directly in the mark file, or they can
46+be written in a separate library, then imported and called by the mark file. We
47+will do the latter in this case, so before we create the mark files themselves,
48+we'll write a library called `/lib/journal.hoon` with the conversion functions.
49+
50+## `$json` utilities
51+
52+[`zuse.hoon`](/docs/hoon/reference/zuse/table-of-contents) contains three main
53+cores for converting to and from `$json`:
54+
55+- [`++enjs:format`](/docs/hoon/reference/zuse/2d_1-5#enjsformat) - Functions to
56+ help encode data structures as `$json`.
57+- [`++dejs:format`](/docs/hoon/reference/zuse/2d_6#dejsformat) - Functions to
58+ decode `$json` to other data structures.
59+- [`++dejs-soft:format`](/docs/hoon/reference/zuse/2d_7#dejs-softformat) -
60+ Mostly the same as `++dejs:format` except the functions produce units which
61+ are null if decoding fails, rather than just crashing.
62+
63+### `++enjs:format`
64+
65+This contains ten functions for encoding `$json`. Most of them are for specific
66+hoon data types, such as `++tape:enjs:format`, `++ship:enjs:format`,
67+`++path:enjs:format`, etc. We'll just have a look at the two most general and
68+useful ones: `++frond:enjs:format` and `++pairs:enjs:format`.
69+
70+#### `++frond`
71+
72+This function is for forming a JSON object from a single key-value pair. For
73+example:
74+
75+```
76+> (frond:enjs:format 'foo' s+'bar')
77+[%o p={[p='foo' q=[%s p='bar']]}]
78+```
79+
80+When stringified by Eyre, this will look like:
81+
82+```json
83+{ "foo": "bar" }
84+```
85+
86+#### `++pairs`
87+
88+This is similar to `++frond` and also forms a JSON object, but it takes multiple
89+key-value pairs rather than just one:
90+
91+```
92+> (pairs:enjs:format ~[['foo' n+~.123] ['bar' s+'abc'] ['baz' b+&]])
93+[%o p={[p='bar' q=[%s p='abc']] [p='baz' q=[%b p=%.y]] [p='foo' q=[%n p=~.123]]}]
94+```
95+
96+When stringified by Eyre, this will look like:
97+
98+```json
99+{
100+ "foo": 123,
101+ "baz": true,
102+ "bar": "abc"
103+}
104+```
105+
106+Notice that we used a knot for the value of `foo` (`n+~.123`). Numbers in JSON
107+can be signed or unsigned and integers or floating point values. The `$json`
108+structure uses a knot so that you can decide whether a particular number should
109+be treated as `@ud`, `@sd`, `@rs`, etc.
110+
111+### `++dejs:format`
112+
113+This core contains many functions for decoding `$json`. We'll touch on some
114+useful families of `++dejs` functions in brief, but because there's so many, in
115+practice you'll need to look through the [`++dejs`
116+reference](/docs/hoon/reference/zuse/2d_6) to find the correct functions for
117+your use case.
118+
119+#### Number functions
120+
121+- `++ne` - decode a number to a `@rd`.
122+- `++ni` - decode a number to a `@ud`.
123+- `++no` - decode a number to a `@ta`.
124+- `++nu` - decode a hexadecimal string to a `@ux`.
125+
126+For example:
127+
128+```
129+> (ni:dejs:format n+'123')
130+123
131+```
132+
133+#### String functions
134+
135+- `++sa` - decode a string to a `tape`.
136+- `++sd` - decode a string containing a `@da` aura date value to a `@da`.
137+- `++se` - decode a string containing the specified aura to that aura.
138+- `++so` - decode a string to a `@t`.
139+- `++su` - decode a string by parsing it with the given [parsing
140+ rule](/docs/hoon/reference/stdlib/4f).
141+
142+#### Array functions
143+
144+`++ar`, `++as`, and `++at` decode a `$json` array to a `list`, `set`, and
145+n-tuple respectively. These gates take other `++dejs` functions as an argument,
146+producing a new gate that will then take the `$json` array. For example:
147+
148+```
149+> ((ar so):dejs:format a+[s+'foo' s+'bar' s+'baz' ~])
150+<|foo bar baz|>
151+```
152+
153+Notice that `++so` is given as the argument to `++ar`. `++so` is a `++dejs`
154+function that decodes a `$json` string to a `cord`. The gate resulting from `(ar so)` is then called with a `$json` array as its argument, and its product is a
155+`(list @t)` of the elements of the array.
156+
157+Many `++dejs` functions take other `++dejs` functions as their arguments. A
158+complex nested `$json` decoding function can be built up in this manner.
159+
160+#### Object functions
161+
162+- `++of` - decode an object containing a single key-value pair to a head-tagged
163+ cell.
164+- `++ot` - decode an object to a n-tuple.
165+- `++ou` - decode an object to an n-tuple, replacing optional missing values
166+ with a given value.
167+- `++oj` - decode an object of arrays to a `jug`.
168+- `++om` - decode an object to a `map`.
169+- `++op` - decode an object to a `map`, and also parse the object keys with a
170+ [parsing rule](/docs/hoon/reference/stdlib/4f).
171+
172+For example:
173+
174+```
175+> =js %- need %- de-json:html
176+ '''
177+ {
178+ "foo": "hello",
179+ "baz": true,
180+ "bar": 123
181+ }
182+ '''
183+
184+> %- (ot ~[foo+so bar+ni]):dejs:format js
185+['hello' 123]
186+```
187+
188+## Our types as JSON
189+
190+We need to decide how our `$action` and `$update` types will be represented as
191+JSON in order to write our conversion functions. There are many ways to do this,
192+but in this case we'll do it as follows:
193+
194+### Actions
195+
196+| JSON | Noun |
197+| ------------------------------------------------- | ---------------------------------------------- |
198+| `{"add":{"id":1648366311070,"txt":"some text"}}` | `[%add id=1.648.366.034.844 txt='some text']` |
199+| `{"edit":{"id":1648366311070,"txt":"some text"}}` | `[%edit id=1.648.366.034.844 txt='some text']` |
200+| `{"del":{"id":1648366311070}}` | `[%del id=1.648.366.034.844]` |
201+
202+### Updates
203+
204+| Noun | JSON |
205+| ----------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
206+| `[1.648.366.492.459 %add id=1.648.366.034.844 txt='some text']` | `{time:1648366481425,"add":{"id":1648366311070,"txt":"some text"}}` |
207+| `[1.648.366.492.459 %edit id=1.648.366.034.844 txt='some text']` | `{time:1648366481425,"edit":{"id":1648366311070,"txt":"some text"}}` |
208+| `[1.648.366.492.459 %del id=1.648.366.034.844]` | `{time:1648366481425,"del":{"id":1648366311070}}` |
209+| `[1.648.366.492.459 %jrnl ~[[id=1.648.366.034.844 txt='some text'] ...]` | `{time:1648366481425,"entries":[{"id":1648366311070,"txt":"some text"},...]}` |
210+| `[1.648.366.492.459 %logs ~[[1.648.366.492.459 %add id=1.648.366.034.844 txt='some text'] ...]` | `{time:1648366481425,"logs":[{time:1648366481425,"add":{id":1648366311070,"txt":"some text"}},...]}` |
211+
212+Now let's write our library of encoding/decoding functions.
213+
214+## `/lib/journal.hoon`
215+
216+```hoon
217+/- *journal
218+|%
219+```
220+
221+First, we'll import the `/sur/journal.hoon` structures we previously created.
222+Next, we'll create two arms in our core, `++dejs-action` and `++enjs-update`, to
223+handle incoming poke `$action`s and outgoing facts or scry result `$update`s.
224+
225+### `$json` to `$action`
226+
227+```hoon
228+++ dejs-action
229+ =, dejs:format
230+ |= jon=json
231+ ^- action
232+ %. jon
233+ %- of
234+ :~ [%add (ot ~[id+ni txt+so])]
235+ [%edit (ot ~[id+ni txt+so])]
236+ [%del (ot ~[id+ni])]
237+ ==
238+```
239+
240+The first thing we do is use the [`=,`
241+rune](/docs/hoon/reference/rune/tis#-tiscom) to expose the `++dejs:format`
242+namespace. This allows us to reference `ot`, `ni`, etc rather than having to
243+write `ot:dejs:format` every time. Note that you should be careful using `=,`
244+generally as the exposed wings can shadow previous wings if they have the same
245+name.
246+
247+We then create a gate that takes `$json` and returns a `$action` structure.
248+Since we'll only take one action at a time, we can use the `++of` function,
249+which takes a single key-value pair. `++of` takes a list of all possible `$json`
250+objects it will receive, tagged by key.
251+
252+For each key, we specify a function to handle its value. Ours will be objects,
253+so we use `++ot` and specify the pairs of the key and `+dejs` function to decode
254+it. We then cast the output to our `$action` structure.
255+
256+You'll notice the nesting of these `++dejs` functions approximately reflects the
257+nested structure of the `$json` it's decoding.
258+
259+### `$update` to `$json`
260+
261+```hoon
262+++ enjs-update
263+ =, enjs:format
264+ |= upd=update
265+ ^- json
266+ |^
267+ ?+ -.q.upd (logged upd)
268+ %jrnl
269+ %- pairs
270+ :~ ['time' (numb p.upd)]
271+ ['entries' a+(turn list.q.upd entry)]
272+ ==
273+ ::
274+ %logs
275+ %- pairs
276+ :~ ['time' (numb p.upd)]
277+ ['logs' a+(turn list.q.upd logged)]
278+ ==
279+ ==
280+ ++ entry
281+ |= ent=^entry
282+ ^- json
283+ %- pairs
284+ :~ ['id' (numb id.ent)]
285+ ['txt' s+txt.ent]
286+ ==
287+ ++ logged
288+ |= lgd=^logged
289+ ^- json
290+ ?- -.q.lgd
291+ %add
292+ %- pairs
293+ :~ ['time' (numb p.lgd)]
294+ :- 'add'
295+ %- pairs
296+ :~ ['id' (numb id.q.lgd)]
297+ ['txt' s+txt.q.lgd]
298+ == ==
299+ %edit
300+ %- pairs
301+ :~ ['time' (numb p.lgd)]
302+ :- 'edit'
303+ %- pairs
304+ :~ ['id' (numb id.q.lgd)]
305+ ['txt' s+txt.q.lgd]
306+ == ==
307+ %del
308+ %- pairs
309+ :~ ['time' (numb p.lgd)]
310+ :- 'del'
311+ (frond 'id' (numb id.q.lgd))
312+ ==
313+ ==
314+ --
315+--
316+```
317+
318+Our `$update` encoding function's a little more complex than our `$action`
319+decoding function, since our `$update` structure is more complex.
320+
321+Like the previous one, we use `=,` to expose the namespace of `++enjs:format`.
322+
323+Our gate takes an `$update` and returns a `$json` structure. We use `|^` so we
324+can separate out the encoding functions for individual entries (`++entry`) and
325+individual logged actions (`++logged`).
326+
327+We first test the head of the `$update`, and if it's `%jrnl` (a list of
328+entries), we `turn` over the entries and call `++entry` to encode each one. If
329+it's `%logs`, we do the same, but call `++logged` for each item in the list.
330+Otherwise, if it's just a single update, we encode it with `++logged`.
331+
332+We primarily use `++pairs` to form the object, though sometimes `++frond` if it
333+only contains a single key-value pair. We also use `++numb` to encode numerical
334+values.
335+
336+You'll notice more of our encoding function is done manually than our previous
337+decoding function. For example, we form arrays by tagging an ordinary `list`
338+with `%a`, and strings by tagging an ordinary `cord` with `%s`. This is typical
339+when you write `$json` encoding functions, and is the reason there are far fewer
340+`+enjs` functions than `+dejs` functions.
341+
342+## Resources
343+
344+- [The JSON Guide](/docs/hoon/guides/json-guide) - The stand-alone JSON guide
345+ covers JSON encoding/decoding in great detail.
346+- [The Zuse reference](/docs/hoon/reference/zuse/table-of-contents) - The
347+ `zuse.hoon` reference documents all JSON-related functions in detail.
348+
349+- [`++enjs:format` reference](/docs/hoon/reference/zuse/2d_1-5#enjsformat) -
350+ This section of the `zuse.hoon` documentation covers all JSON encoding
351+ functions.
352+
353+- [`++dejs:format` reference](/docs/hoon/reference/zuse/2d_6) - This section of
354+ the `zuse.hoon` documentation covers all JSON _decoding_ functions.
355+
356+- [Eyre overview](/docs/arvo/eyre/eyre) - This section of the Eyre vane
357+ documentation goes over the basic features of the Eyre vane.
content/docs/userspace/full-stack/5-marks.mdnew file+97
@@ -0,0 +1,97 @@
1++++
2+title = "5. Marks"
3+weight = 5
4+template = "doc.html"
5++++
6+
7+In this section we'll write the mark files for our agent. We'll need two marks,
8+one for poke `$action`s and one for subscription updates and scry results, both
9+of which are `$update`s. Our `$action` mark will be called `%journal-action` and
10+our `$update` mark will be called `%journal-update`. These will be located at
11+`/mar/journal/action.hoon` and `/mar/journal/update.hoon`.
12+
13+Note that a mark called `%foo-bar` will first be looked for in
14+`/mar/foo-bar.hoon`, and if it's not there it will be looked for in
15+`/mar/foo/bar.hoon`. That's why we can have a single name like `%journal-action`
16+but have it in `/mar/journal/action.hoon`
17+
18+## `%journal-action`
19+
20+```hoon
21+/- *journal
22+/+ *journal
23+|_ act=action
24+++ grow
25+ |%
26+ ++ noun act
27+ --
28+++ grab
29+ |%
30+ ++ noun action
31+ ++ json dejs-action
32+ --
33+++ grad %noun
34+--
35+```
36+
37+First we import our `/sur/journal.hoon` structure file and also our
38+`/lib/journal.hoon` library (containing our `$json` conversion functions). The
39+sample of our mark door is just our `$action` structure. The `++grow` arm of a
40+mark core, if you recall, contains methods for converting _from_ our mark _to_
41+another mark. Actions only ever come inwards in pokes, so we don't need to worry
42+about converting an `$action` _to_ `$json`. The `++grow` arm can therefore just
43+handle the generic `%noun` case, simply returning our mark door's sample without
44+doing anything.
45+
46+`++grab`, conversely, defines methods for converting _to_ our mark _from_
47+another mark. Since `$action`s will come in from the front-end as `$json`, we
48+need to be able to convert `$json` data to our `$action` structure. Our
49+`/lib/journal.hoon` library contains the `++dejs-action` function for performing
50+this conversion, so we can just specify that function for the `%json` case.
51+We'll also define a standard `%noun` method, which will just "clam" (or "mold")
52+the incoming noun with the `$action` mold. Clamming/molding coerces a noun to a
53+type and is done by calling a mold as a function.
54+
55+Lastly, `++grad` defines revision control methods, but can be delegated to
56+another mark. Since this mark will never be used for actually storing files in
57+Clay, we can just delegate it to the generic `%noun` mark rather than writing a
58+proper set of `++grad` methods.
59+
60+## `%journal-update`
61+
62+```hoon
63+/- *journal
64+/+ *journal
65+|_ upd=update
66+++ grow
67+ |%
68+ ++ noun upd
69+ ++ json (enjs-update upd)
70+ --
71+++ grab
72+ |%
73+ ++ noun update
74+ --
75+++ grad %noun
76+--
77+```
78+
79+Next we have our `%journal-update` mark file. The sample of our mark door is our
80+`$update` structure. Our `$update`s are always outbound, never inbound, so we
81+only need to define a method for converting our `$update` structure to `$json`
82+in the `++grow` arm, and not the opposite direction in `++grad`. Our
83+`/lib/journal.hoon` library contains the `++enjs-update` function for performing
84+this conversion, so we can call it with the sample `$update` as its argument. We
85+can add `%noun` conversion methods and delegate revision control to the `%noun`
86+mark in the same manner as our `%journal-action` mark above.
87+
88+## Resources
89+
90+- [The Marks section of the Clay documentation](/docs/arvo/clay/marks/marks) -
91+ This section of the Clay vane documentation covers mark files comprehensively.
92+- [The mark file section of the Gall
93+ Guide](/docs/userspace/gall-guide/7-sur-and-marks#mark-files) - This part of
94+ the Gall Guide goes through the basics of mark files.
95+
96+- [The JSON Guide](/docs/hoon/guides/json-guide) - This also covers writing mark
97+ files to convert to/from JSON.
content/docs/userspace/full-stack/6-eyre.mdnew file+79
@@ -0,0 +1,79 @@
1++++
2+title = "6. Eyre"
3+weight = 6
4+template = "doc.html"
5++++
6+
7+Now that we have our structure file, agent, `$json` conversion library and mark
8+file, our back-end is complete. Before we start writing our front-end, though,
9+we should give a brief overview of how Eyre works.
10+
11+[Eyre](/docs/arvo/eyre/eyre) is the HTTP server [vane](/docs/glossary/vane) of
12+Arvo. Eyre has a handful of different subsystems, but the main two are the
13+channel system and the scry interface. These two are what we'll focus on here.
14+
15+In order to use the channel system or perform scries, a web client must have
16+authenticated with the ship's web login code (e.g.
17+`lidlut-tabwed-pillex-ridrup`) and obtained a session cookie. Our front-end will
18+be served directly from the ship by the `%docket` agent, so we can assume a
19+session cookie was already obtained when the user logged into landscape, and
20+skip over authentication.
21+
22+## Channels
23+
24+Eyre's channel system is the main way to interact with agents from a web client.
25+It provides a JSON interface to the ordinary poke and subscription system for
26+Gall agents.
27+
28+First, a unique channel ID is generated by the web client (`@urbit/http-api`
29+uses the current Unix time concatenated with a random hex string). The client
30+then sends a poke or subscription request for the channel, and Eyre
31+automatically opens a new channel with that ID. Once open, the client can then
32+connect to the channel and receive any events such as poke acks, watch acks,
33+facts from subscriptions, etc.
34+
35+The new channel is an SSE ([Server Sent
36+Event](https://html.spec.whatwg.org/#server-sent-events)) stream, and can be
37+handled by an `EventSource` object in Javascript. The `@urbit/http-api` library
38+we'll use abstracts this for us, so we won't need to deal with an `EventSource`
39+object directly. The channel can handle multiple concurrent subscriptions to
40+different agent subscription paths, and different agents can be poked through
41+the one channel. This means a client only needs to open a single channel for all
42+of its interactions with the ship. Each subscription is given a different ID, so
43+they can be individually unsubscribed later.
44+
45+A channel will timeout after 12 hours of inactivity, and the timeout is reset
46+every time Eyre receives a message of any kind from the client. Additionally,
47+each subscription on the channel may only accumulate 50 unacknowledged facts
48+before it's considered "clogged", in which case the individual clogged
49+subscription will be closed by Eyre after a short delay. All events of any kind
50+which Eyre sends out on the channel must be ack'd by the client. Ack'ing one
51+event will also ack all previous events too. The `@urbit/http-api` library we'll
52+use automatically acks events for us, so we don't need to worry about clogged
53+subscriptions or manually ack'ing events.
54+
55+Eyre expects a particular JSON object structure for each of these different
56+requests, but the `@urbit/http-api` library we'll use includes functions to send
57+pokes, subscription requests, etc, so we won't need to manually construct the
58+JSON objects in our front-end.
59+
60+## Scries
61+
62+Eyre's scry interface is separate to the channel system. Scries are performed by
63+a simple GET request to a path with a format of `/~/scry/{agent}{path}.{mark}`.
64+If successful, the HTTP response will contain the result with the mark
65+specified. If unsuccessful, an HTTP error will be thrown in response.
66+
67+The `@urbit/http-api` library we'll use includes a function for performing
68+scries, so we'll not need to manually send GET requests to the ship.
69+
70+## Resources
71+
72+- [The Eyre vane documentation](/docs/arvo/eyre/eyre) - This section of the vane
73+ docs covers all aspects of Eyre.
74+- [Eyre External API Reference](/docs/arvo/eyre/external-api-ref) - This section
75+ of the Eyre documentation contains reference material for Eyre's external API.
76+
77+- [The Eyre Guide](/docs/arvo/eyre/guide) - This section of the Eyre
78+ documentation walks through using Eyre's external API at a low level (using
79+ `curl`).
content/docs/userspace/full-stack/7-react-setup.mdnew file+214
@@ -0,0 +1,214 @@
1++++
2+title = "7. React app setup"
3+weight = 7
4+template = "doc.html"
5++++
6+
7+Now that we have a basic idea of how Eyre works, we can begin working on our
8+React app front-end.
9+
10+## Create React app
11+
12+Node.js must be installed, and can be downloaded from their
13+[website](https://nodejs.org/en/download). With that installed, we'll have the
14+`npm` package manager available. The first thing we'll do is globally install
15+the `create-react-app` package with the following command:
16+
17+```sh
18+npm install -g create-react-app
19+```
20+
21+Once installed, we can use it to create a new `journal-ui` directory and setup a
22+new React app in it with the following command:
23+
24+```sh
25+create-react-app journal-ui
26+```
27+
28+We can then open our new directory:
29+
30+```sh
31+cd journal-ui
32+```
33+
34+Its contents should look something like this:
35+
36+```
37+journal-ui
38+├── node_modules
39+├── package.json
40+├── package-lock.json
41+├── public
42+├── README.md
43+└── src
44+```
45+
46+## Install `http-api`
47+
48+Inside our React app directory, let's install the `@urbit/http-api` NPM package:
49+
50+```sh
51+npm i @urbit/http-api
52+```
53+
54+We also install a handful of other packages for the UI components (`bootstrap react-bootstrap react-textarea-autosize date-fns react-bottom-scroll-listener react-day-picker`), but that's not important to our purposes here.
55+
56+## Additional tweaks
57+
58+Our front-end will be served directly from the ship by the `%docket` app, where
59+a user will open it by clicking on its homescreen tile. Docket serves such
60+front-ends with a base URL path of `/apps/[desk]/`, so in our case it will be
61+`/apps/journal`. In order for our app to be built with correct resource paths,
62+we must add the following line to `package.json`:
63+
64+```json
65+"homepage": "/apps/journal/",
66+```
67+
68+Our app also needs to know the name of the ship it's being served from in order
69+to talk with it. The `%docket` agent serves a small file for this purpose at
70+`[host]/session.js`. This file is very simple and just contains:
71+
72+```js
73+window.ship = "sampel-palnet";
74+```
75+
76+`sampel-palnet` will of course be replaced by the actual name of the ship. We
77+include this script by adding the following line to the `&lt;head>` section of
78+`public/index.html`:
79+
80+```
81+<script src="/session.js"></script>
82+```
83+
84+## Basic API setup
85+
86+With everything now setup, we can begin work on the app itself. In this case
87+we'll just edit the existing `App.js` file in the `/src` directory. The first thing is to import the `Urbit` class from `@urbit/http-api`:
88+
89+```js
90+import Urbit from "@urbit/http-api";
91+```
92+
93+We also need to import a few other things, mostly relating to UI components (but
94+these aren't important for our purposes here):
95+
96+```js
97+import React, { Component } from "react";
98+import "bootstrap/dist/css/bootstrap.min.css";
99+import "react-day-picker/lib/style.css";
100+import TextareaAutosize from "react-textarea-autosize";
101+import Button from "react-bootstrap/Button";
102+import Card from "react-bootstrap/Card";
103+import Stack from "react-bootstrap/Stack";
104+import Tab from "react-bootstrap/Tab";
105+import Tabs from "react-bootstrap/Tabs";
106+import ToastContainer from "react-bootstrap/ToastContainer";
107+import Toast from "react-bootstrap/Toast";
108+import Spinner from "react-bootstrap/Spinner";
109+import CloseButton from "react-bootstrap/CloseButton";
110+import Modal from "react-bootstrap/Modal";
111+import DayPickerInput from "react-day-picker/DayPickerInput";
112+import endOfDay from "date-fns/endOfDay";
113+import startOfDay from "date-fns/startOfDay";
114+import { BottomScrollListener } from "react-bottom-scroll-listener";
115+```
116+
117+Inside the existing `App` class:
118+
119+```js
120+class App extends Component {
121+```
122+
123+...we'll clear out the existing demo code and start adding ours. The first thing
124+is to define our app's state. We'll look at most of the state entries in the
125+next section. For now, we'll just consider `status`.
126+
127+```js
128+state = {
129+ // .....
130+ status: null,
131+ // .....
132+};
133+```
134+
135+Next, we'll setup the `Urbit` API object in `componentDidMount`. We could do
136+this outside the `App` class since we're adding it to `window`, but we'll do it
137+this way so it's all in one place:
138+
139+```js
140+componentDidMount() {
141+ window.urbit = new Urbit("");
142+ window.urbit.ship = window.ship;
143+ window.urbit.onOpen = () => this.setState({status: "con"});
144+ window.urbit.onRetry = () => this.setState({status: "try"});
145+ window.urbit.onError = (err) => this.setState({status: "err"});
146+ this.init();
147+};
148+```
149+
150+The first thing we do is create a new instance of the `Urbit` class we imported
151+from `@urbit/http-api`, and save it to `window.urbit`. The `Urbit` class
152+constructor takes three arguments: `url`, `desk` and `code`, of which only `url`
153+is mandatory.
154+
155+- `url` is the URL of the ship we want to talk to. Since our React app will be
156+ served by the ship, we can just leave it as an empty `""` string and let
157+ `Urbit` use root-relative paths.
158+- `desk` is only necessary if we want to run threads through Eyre, and since
159+ we're not going to do that, we can exclude it.
160+- `code` is the web login code for authentication, but since the user will
161+ already have logged in, we can also exclude that.
162+
163+Therefore, we call the class contructor with just the empty `url` string:
164+
165+```js
166+window.urbit = new Urbit("");
167+```
168+
169+Next, we need to set the ship name in our `Urbit` instance. Eyre requires the
170+ship name be specified in all requests, so if we don't set it, Eyre will reject
171+all the messages we send. We previously included `session.js` which sets
172+`window.ship` to the ship name, so we just set `window.urbit.ship` as that:
173+
174+```js
175+window.urbit.ship = window.ship;
176+```
177+
178+Next, we set three callbacks: `onOpen`, `onRetry`, and `onError`. These
179+callbacks are fired when the state of our channel connection changes:
180+
181+- `onOpen` is called when a connection is established.
182+- `onRetry` is called when a channel connection has been interrupted (such as by
183+ network issues) and the `Urbit` object is trying to reconnect. Reconnection
184+ will be attempted up to three times: immediately, after 750ms, and after
185+ 3000ms.
186+- `onError` is called with an `Error` message once all retries have failed, or
187+ otherwise when a fatal error occurs.
188+
189+We'll look at how we handle these cases in the next section. For now, we'll just
190+set the `status` entry in the state to either `"con"`, `"try"`, or `"err"` as
191+the case may be. Note that it's not mandatory to set these callbacks, but
192+leaving connection problems unhandled is usually a bad idea.
193+
194+The last thing we do is call:
195+
196+```js
197+this.init();
198+```
199+
200+This function will fetch initial entries and subscribe for updates. We'll look
201+at it in the next section.
202+
203+## Resources
204+
205+- [HTTP API Guide](/docs/userspace/http-api-guide) - Reference documentation for
206+ `@urbit/http-api`.
207+
208+- [React app source
209+ code](https://github.com/urbit/docs-examples/tree/main/journal-app/ui) - The
210+ source code for the Journal app UI.
211+
212+- [`@urbit/http-api` source
213+ code](https://github.com/urbit/urbit/tree/master/pkg/npm/http-api) - The
214+ source code for the `@urbit/http-api` NPM package.
content/docs/userspace/full-stack/8-app-logic.mdnew file+406
@@ -0,0 +1,406 @@
1++++
2+title = "8. React app logic"
3+weight = 8
4+template = "doc.html"
5++++
6+
7+With the basic things setup, we can now go over the logic of our app. We'll just
8+focus on functions that are related to ship communications using the `Urbit`
9+object we previously setup, and ignore UI components and other helper functions.
10+
11+## State
12+
13+In the previous section we just mentioned the connection `status` field of our
14+state. Here's the full state of our App:
15+
16+```js
17+state = {
18+ entries: [], // list of journal entries for display
19+ drafts: {}, // edits which haven't been submitted yet
20+ newDraft: {}, // new entry which hasn't been submitted yet
21+ results: [], // search results
22+ searchStart: null, // search query start date
23+ searchEnd: null, // search query end date
24+ resultStart: null, // search results start date
25+ resultEnd: null, // search results end date
26+ searchTime: null, // time of last search
27+ latestUpdate: null, // most recent update we've received
28+ entryToDelete: null, // deletion target for confirmation modal
29+ status: null, // connection status (con, try, err)
30+ errorCount: 0, // number of errors so far
31+ errors: new Map(), // list of error messages for display
32+};
33+```
34+
35+We'll see how these are used subsequently.
36+
37+## Initialize
38+
39+The first thing our app does is call `init()`:
40+
41+```js
42+init = () => {
43+ this.getEntries().then(
44+ (result) => {
45+ this.handleUpdate(result);
46+ this.setState({ latestUpdate: result.time });
47+ this.subscribe();
48+ },
49+ (err) => {
50+ this.setErrorMsg("Connection failed");
51+ this.setState({ status: "err" });
52+ }
53+ );
54+};
55+```
56+
57+This function just calls `getEntries()` to retrieve the initial list of journal
58+entries then, if that succeeded, it calls `subscribe()` to subscribe for new
59+updates. If the initial entry retrieval failed, we set the connection `status`
60+and save an error message in the `errors` map. We'll look at what we do with
61+errors later.
62+
63+## Getting entries
64+
65+![entries screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/entries.png)
66+
67+The `getEntries` function scries our `%journal` agent for up to 10 entries
68+before the oldest we currently have. We call this initially, and then each time
69+the user scrolls to the bottom of the list.
70+
71+```js
72+getEntries = async () => {
73+ const { entries: e } = this.state;
74+ const before = e.length === 0 ? Date.now() : e[e.length - 1].id;
75+ const max = 10;
76+ const path = `/entries/before/${before}/${max}`;
77+ return window.urbit.scry({
78+ app: "journal",
79+ path: path,
80+ });
81+};
82+```
83+
84+The scry is done with the `Urbit.scry` method. This function takes two arguments
85+in an object:
86+
87+- `app` - the agent to scry.
88+- `path` - the scry path. Note the `care` is not included - all scries through
89+ Eyre are `%x` scries.
90+
91+The `Urbit.scry` method only allows JSON results, but note that scries done via
92+direct GET requests allow other marks too.
93+
94+The `Urbit.scry` method returns a Promise which will contain an HTTP error
95+message if the scry failed. We handle it with a `.then` expression back in the
96+function that called it, either [`init()`](#initialize) or `moreEntries()`. If
97+the Promise is successfuly, the results are passed to the
98+[`handleUpdate`](#updates) function which appends the new entries to the
99+existing ones in state.
100+
101+## Subscription
102+
103+A subscription to the `/updates` path of our `%journal` agent is opened with our
104+`subscribe()` function:
105+
106+```js
107+subscribe = () => {
108+ try {
109+ window.urbit.subscribe({
110+ app: "journal",
111+ path: "/updates",
112+ event: this.handleUpdate,
113+ err: () => this.setErrorMsg("Subscription rejected"),
114+ quit: () => this.setErrorMsg("Kicked from subscription"),
115+ });
116+ } catch {
117+ this.setErrorMsg("Subscription failed");
118+ }
119+};
120+```
121+
122+We use the `Urbit.subscribe` method for this, which takes five arguments in an
123+object:
124+
125+- `app` - the target agent.
126+- `path` - the `%watch` path we're subscribing to.
127+- `event` - a function to handle each fact the agent sends out. We call our
128+ `handleUpdate` function, which we'll describe below.
129+- `err` - a function to call if the subscription request is rejected (nacked).
130+ We just display an error in this case.
131+- `quit` - a function to call if we get kicked from the subscription. We also
132+ just display an error in this case.
133+
134+Note that the `Urbit.subscribe` method returns a subscription ID number. Since
135+we only have one subscription in our app which we never close, we don't bother
136+to record it. If your app has multiple subscriptions to manage, you may wish to
137+keep track of these IDs in your app's state.
138+
139+## Updates
140+
141+This `handleUpdate` function handles all updates we receive. It's called
142+whenever an event comes in for our subscription, and it's also called with the
143+results of [`getEntries`](#getting-entries) and [`getUpdates`](#error-handling)
144+(described later).
145+
146+It's a bit complex, but basically it just checks whether the JSON object is
147+`add`, `edit`, `delete`, or `entries`, and then updates the state appropriately.
148+The object it's receiving is just the `$update` structure converted to JSON by
149+the mark conversion functions we wrote previously.
150+
151+```js
152+handleUpdate = (upd) => {
153+ const { entries, drafts, results, latestUpdate } = this.state;
154+ if (upd.time !== latestUpdate) {
155+ if ("entries" in upd) {
156+ this.setState({ entries: entries.concat(upd.entries) });
157+ } else if ("add" in upd) {
158+ const { time, add } = upd;
159+ const eInd = this.spot(add.id, entries);
160+ const rInd = this.spot(add.id, results);
161+ const toE =
162+ entries.length === 0 || add.id > entries[entries.length - 1].id;
163+ const toR = this.inSearch(add.id, time);
164+ toE && entries.splice(eInd, 0, add);
165+ toR && results.splice(rInd, 0, add);
166+ this.setState({
167+ ...(toE && { entries: entries }),
168+ ...(toR && { results: results }),
169+ latestUpdate: time,
170+ });
171+ } else if ("edit" in upd) {
172+ const { time, edit } = upd;
173+ const eInd = entries.findIndex((e) => e.id === edit.id);
174+ const rInd = results.findIndex((e) => e.id === edit.id);
175+ const toE = eInd !== -1;
176+ const toR = rInd !== -1 && this.inSearch(edit.id, time);
177+ if (toE) entries[eInd] = edit;
178+ if (toR) results[rInd] = edit;
179+ (toE || toR) && delete drafts[edit.id];
180+ this.setState({
181+ ...(toE && { entries: entries }),
182+ ...(toR && { results: results }),
183+ ...((toE || toR) && { drafts: drafts }),
184+ latestUpdate: time,
185+ });
186+ } else if ("del" in upd) {
187+ const { time, del } = upd;
188+ const eInd = entries.findIndex((e) => e.id === del.id);
189+ const rInd = results.findIndex((e) => e.id === del.id);
190+ const toE = eInd !== -1;
191+ const toR = this.inSearch(del.id, time) && rInd !== -1;
192+ toE && entries.splice(eInd, 1);
193+ toR && results.splice(rInd, 1);
194+ (toE || toR) && delete drafts[del.id];
195+ this.setState({
196+ ...(toE && { entries: entries }),
197+ ...(toR && { results: results }),
198+ ...((toE || toR) && { drafts: drafts }),
199+ latestUpdate: time,
200+ });
201+ }
202+ }
203+};
204+```
205+
206+## Add, edit, delete
207+
208+![add screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/add.png)
209+
210+When a user writes a new journal entry and hits submit, the `submitNew` function
211+is called. It uses the `Urbit.poke` method to poke our `%journal` agent.
212+
213+```js
214+submitNew = (id, txt) => {
215+ window.urbit.poke({
216+ app: "journal",
217+ mark: "journal-action",
218+ json: { add: { id: id, txt: txt } },
219+ onSuccess: () => this.setState({ newDraft: {} }),
220+ onError: () => this.setErrorMsg("New entry rejected"),
221+ });
222+};
223+```
224+
225+The `Urbit.poke` method takes five arguments:
226+
227+- `app` is the agent to poke.
228+- `mark` is the mark of the data we're sending. We specify `"journal-action"`,
229+ so Eyre will use the `/mar/journal/action.hoon` mark we created to convert it
230+ to a `$action` structure with a `%journal-action` mark before it's delivered
231+ to our agent.
232+- `json` is the actual data we're poking our agent with. In this case it's the
233+ JSON form of the `%add` `$action`.
234+- `onSuccess` is a callback that fires if we get a positive ack in response. In
235+ this case we just clear the draft.
236+- `onError` is a callback that fires if we get a negative ack (nack) in
237+ response, meaning the poke failed. In this case we just set an error message
238+ to be displayed.
239+
240+`onSuccess` and `onError` are optional, but it's usually desirable to handle
241+these cases.
242+
243+The `delete` and `submitEdit` functions are similar to `submitNew`, but for the
244+`%del` and `%edit` actions rather than `%add`:
245+
246+![edit screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/edit.png)
247+
248+```js
249+submitEdit = (id, txt) => {
250+ if (txt !== null) {
251+ window.urbit.poke({
252+ app: "journal",
253+ mark: "journal-action",
254+ json: { edit: { id: id, txt: txt } },
255+ onError: () => this.setErrorMsg("Edit rejected"),
256+ });
257+ } else this.cancelEdit(id);
258+};
259+```
260+
261+![delete screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/delete.png)
262+
263+```js
264+delete = (id) => {
265+ window.urbit.poke({
266+ app: "journal",
267+ mark: "journal-action",
268+ json: {"del": {"id": id}},
269+ onError: ()=>this.setErrorMsg("Deletion rejected")
270+ })
271+ this.setState({rmModalShow: false, entryToDelete: null})
272+};
273+```
274+
275+Note that whether we're adding, editing or deleting entries, we update our state
276+when we receive the update back on the `/updates` subscription, not when we poke
277+our agent.
278+
279+## Search
280+
281+![search screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/search.png)
282+
283+When searching for entries between two dates, the `getSearch` function is
284+called, which uses the `Urbit.scry` method to scry for the results in a similar
285+fashion to [`getEntries`](#getting-entries), but using the
286+`/x/entries/between/[start]/[end]` endpoint.
287+
288+```js
289+getSearch = async () => {
290+ const { searchStart: ss, searchEnd: se, latestUpdate: lu } = this.state;
291+ if (lu !== null && ss !== null && se !== null) {
292+ let start = ss.getTime();
293+ let end = se.getTime();
294+ if (start < 0) start = 0;
295+ if (end < 0) end = 0;
296+ const path = `/entries/between/${start}/${end}`;
297+ window.urbit
298+ .scry({
299+ app: "journal",
300+ path: path,
301+ })
302+ .then(
303+ (result) => {
304+ this.setState({
305+ searchTime: result.time,
306+ searchStart: null,
307+ searchEnd: null,
308+ resultStart: ss,
309+ resultEnd: se,
310+ results: result.entries,
311+ });
312+ },
313+ (err) => {
314+ this.setErrorMsg("Search failed");
315+ }
316+ );
317+ } else {
318+ lu !== null && this.setErrorMsg("Searh failed");
319+ }
320+};
321+```
322+
323+## Error handling
324+
325+When the channel connection is interrupted, the `Urbit` object will begin trying to reconnect. On each attempt, it sets the connection `status` to `"try"`, as we specified for the `onRetry` callback. When this is set, a "reconnecting" message is displayed at the bottom of the screen:
326+
327+![reconnecting screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/reconnecting.png)
328+
329+If all three reconnection attempts fail, the `onError` callback is fired and we replace the "reconnecting" message with a "reconnect" button:
330+
331+![reconnect screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/reconnect.png)
332+
333+When clicked, the following function is called:
334+
335+```js
336+reconnect = () => {
337+ window.urbit.reset();
338+ const latest = this.state.latestUpdate;
339+ if (latest === null) {
340+ this.init();
341+ } else {
342+ this.getUpdates().then(
343+ (result) => {
344+ result.logs.map((e) => this.handleUpdate(e));
345+ this.subscribe();
346+ },
347+ (err) => {
348+ this.setErrorMsg("Connection failed");
349+ this.setState({ status: "err" });
350+ }
351+ );
352+ }
353+};
354+```
355+
356+Our `reconnect()` function first calls the `Urbit.reset` method. This closes the
357+channel connection, wipes event counts and subscriptions, and generates a new
358+channel ID. We could have tried reconnecting without resetting the connection,
359+but we don't know whether the channel still exists. We could time how long the
360+connection has been down and estimate whether it still exists, but it's easier
361+to just start fresh in this case.
362+
363+Since we've reset the channel, we don't know if we've missed any updates. Rather
364+than having to refresh our whole state, we can use the `getUpdates()` function
365+to get any missing update:
366+
367+```js
368+getUpdates = async () => {
369+ const { latestUpdate: latest } = this.state;
370+ const since = latest === null ? Date.now() : latest;
371+ const path = `/updates/since/${since}`;
372+ return window.urbit.scry({
373+ app: "journal",
374+ path: path,
375+ });
376+};
377+```
378+
379+This function uses the `Urbit.scry` method to scry the
380+`/x/updates/since/[since]` path, querying the update `$log` for entries more
381+recent than `latestUpdate`, which is always set to the last logged action we
382+received. The `getUpdates` function returns a Promise to the `reconnect`
383+function above which called it. The `reconnect` function handles it in a `.then`
384+expression, where the success case passes each update retrieved to the
385+[`handleUpdate`](#updates) function, updating our state.
386+
387+Lastly, as well as handling channel connection errors, we also handle errors
388+such as poke nacks or failed scries by printing error messages added to the
389+`error` map by the `setErrorMsg` function. You could of course handle nacks,
390+kicks, scry failures, etc differently than just printing an error, it depends on
391+the needs of your app.
392+
393+![search failed screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/search-failed.png)
394+
395+## Resources
396+
397+- [HTTP API Guide](/docs/userspace/http-api-guide) - Reference documentation for
398+ `@urbit/http-api`.
399+
400+- [React app source
401+ code](https://github.com/urbit/docs-examples/tree/main/journal-app/ui) - The
402+ source code for the Journal app UI.
403+
404+- [`@urbit/http-api` source
405+ code](https://github.com/urbit/urbit/tree/master/pkg/npm/http-api) - The
406+ source code for the `@urbit/http-api` NPM package.
content/docs/userspace/full-stack/9-desk.mdnew file+226
@@ -0,0 +1,226 @@
1++++
2+title = "9. Desk and glob"
3+weight = 9
4+template = "doc.html"
5++++
6+
7+With our React app now complete, we can put together the final desk and publish
8+it.
9+
10+## Config files
11+
12+So far we've written the following files for the back-end:
13+
14+```
15+ourfiles
16+├── app
17+│ └── journal.hoon
18+├── lib
19+│ └── journal.hoon
20+├── mar
21+│ └── journal
22+│ ├── action.hoon
23+│ └── update.hoon
24+└── sur
25+ └── journal.hoon
26+```
27+
28+There's a handful of extra files we need in the root of our desk:
29+
30+- `desk.bill` - the list of agents that should be started when our app is
31+ installed.
32+- `sys.kelvin` - the kernel version our app is compatible with.
33+- `desk.docket-0` - configuration of our app tile, front-end glob and other
34+ metadata.
35+
36+We only have one agent to start, so `desk.bill` is very simple:
37+
38+```
39+:~ %journal
40+==
41+```
42+
43+Likewise, `sys.kelvin` just contains:
44+
45+```
46+[%zuse 419]
47+```
48+
49+The `desk.docket-0` file is slightly more complicated:
50+
51+```
52+:~
53+ title+'Journal'
54+ info+'Dear diary...'
55+ color+0xd9.b06d
56+ version+[0 1 0]
57+ website+'https://urbit.org'
58+ license+'MIT'
59+ base+'journal'
60+ glob-ames+[~zod 0v0]
61+==
62+```
63+
64+The fields are as follows:
65+
66+- `title` is the name of the app - this will be displayed on the tile and when
67+ people search for the app to install it.
68+- `info` is a brief description of the app.
69+- `color` - the RGB hex color of the tile.
70+- `version` - the version number of the app. The fields represent major, minor
71+ and patch version.
72+- `website` - a link to a website for the app. This would often be its Github repo.
73+- `license` - the license of for the app.
74+- `base` - the desk name of the app.
75+- `glob-ames` - the ship to retrieve the front-end files from, and the hash of
76+ those files. We've put `~zod` here but this would be the actual ship
77+ distributing the app when it's live on the network. The hash is `0v0`
78+ initially, but once we upload the front-end files it will be updated to the
79+ hash of those files automatically. Note that it's also possible to distribute
80+ front-end files from a separate web server. In that case, you'd use
81+ `glob-http` rather than `glob-ames`. The [Glob section of the distribution
82+ guide](/docs/userspace/dist/glob) covers this alternative approach in more
83+ detail.
84+
85+Our files should now look like this:
86+
87+```
88+ourfiles
89+├── app
90+│ └── journal.hoon
91+├── desk.bill
92+├── desk.docket-0
93+├── lib
94+│ └── journal.hoon
95+├── mar
96+│ └── journal
97+│ ├── action.hoon
98+│ └── update.hoon
99+├── sur
100+│ └── journal.hoon
101+└── sys.kelvin
102+```
103+
104+## New desk
105+
106+Next, we'll create a new `%journal` desk on our ship by forking an existing one.
107+Once created, we can mount it to the unix filesystem.
108+
109+In the dojo of a fake ship:
110+
111+```
112+> |merge %journal our %webterm
113+>=
114+> |mount %journal
115+>=
116+```
117+
118+Now we can browse to it in the unix terminal:
119+
120+```sh
121+cd ~/zod/journal
122+```
123+
124+Currently it has the same files as the `%webterm` desk, so we need to delete
125+those:
126+
127+```sh
128+rm -r .
129+```
130+
131+Apart from the kernel and standard library, desks need to be totally
132+self-contained, including all mark files and libraries necessary to build them.
133+For example, since our app contains a number of `.hoon` files, we need the
134+`hoon.hoon` mark, and its dependencies. The easiest way to ensure our desk has
135+everything it needs is to copy in the "dev" versions of the `%base` and
136+`%garden` desks. To do this, we first clone the Urbit git repository:
137+
138+```sh
139+git clone https://github.com/urbit/urbit.git urbit-git
140+```
141+
142+If we navigate to the `pkg` directory in the cloned repo:
143+
144+```sh
145+cd ~/urbit-git/pkg
146+```
147+
148+...we can combine the `base-dev` and `garden-dev` desks with the included
149+`symbolic-merge.sh` script:
150+
151+```sh
152+./symbolic-merge.sh base-dev journal
153+./symbolic-merge.sh garden-dev journal
154+```
155+
156+Now, we copy the contents of the new `journal` folder into our empty desk:
157+
158+```sh
159+cp -rL journal/* ~/zod/journal/
160+```
161+
162+Note we've used the `L` flag to resolve symbolic links, because the dev-desks
163+contain symlinks to files in the actual `arvo` and `garden` folders.
164+
165+We can copy across all of our own files too:
166+
167+```sh
168+cp -r ~/ourfiles/* ~/zod/journal/
169+```
170+
171+Finally, in the dojo, we can commit the whole lot:
172+
173+```
174+|commit %journal
175+```
176+
177+## Glob
178+
179+The next step is to build our front-end and upload the files to our ship. In the
180+`journal-ui` folder containing our React app, we can run:
181+
182+```sh
183+npm run build
184+```
185+
186+This will create a `build` directory containing the compiled front-end files. To
187+upload it to our ship, we need to first install the `%journal` desk. In the
188+dojo:
189+
190+```
191+|install our %journal
192+```
193+
194+Next, in the browser, we navigate to the `%docket` globulator at
195+`http://localhost:8080/docket/upload` (replacing localhost with the actual host):
196+
197+![globulator screenshot](https://m.tinnus-napbus.xyz/pub/globulator.png)
198+
199+We select our `journal` desk, then we hit `Choose file`, and select the whole
200+`build` directory which was created when we build our React app. Finally, we hit
201+`glob!` to upload it.
202+
203+If we now return to the homescreen of our ship, we'll see our tile displayed, and we can open our app by clicking on it:
204+
205+![tiles screenshot](https://m.tinnus-napbus.xyz/pub/tiles.png)
206+
207+## Publishing
208+
209+The last thing we need to do is publish our app, so other users can install it
210+from our ship. To do that, we just run the following command in the dojo:
211+
212+```
213+:treaty|publish %journal
214+```
215+
216+## Resources
217+
218+- [App publishing/distribution documentation](/docs/userspace/dist/dist) -
219+ Documentation covering third party desk composition, publishing and
220+ distribution.
221+
222+- [Glob documentation](/docs/userspace/dist/glob) - Comprehensive documentation
223+ of handling front-end files.
224+
225+- [Desk publishing guide](/docs/userspace/dist/guide) - A step-by-step guide to
226+ creating and publishing a desk.
content/docs/userspace/full-stack/_index.mdnew file+52
@@ -0,0 +1,52 @@
1++++
2+title = "Full-Stack Walkthrough"
3+weight = 10
4+sort_by = "weight"
5+template = "sections/docs/chapters.html"
6+insert_anchor_links = "right"
7++++
8+
9+#### [1. Introduction](/docs/userspace/gall-2/1-intro)
10+
11+An overview of the guide and table of contents.
12+
13+#### [2. Types](/docs/userspace/gall-2/2-types)
14+
15+Creating the `/sur` structure file for our `%journal` agent.
16+
17+#### [3. Agent](/docs/userspace/gall-2/3-agent)
18+
19+Creating the `%journal` agent itself.
20+
21+#### [4. JSON](/docs/userspace/gall-2/5-json)
22+
23+Writing a library to convert between our agent's marks and JSON. This lets our
24+React front-end poke our agent, and our agent send updates back to it.
25+
26+#### [5. Marks](/docs/userspace/gall-2/4-marks)
27+
28+Creating the mark files for the pokes our agent takes and updates it sends out.
29+
30+#### [6. Eyre](/docs/userspace/gall-2/6-eyre)
31+
32+A brief overview of how the webserver vane Eyre works.
33+
34+#### [7. React App Setup](/docs/userspace/gall-2/7-react-setup)
35+
36+Creating a new React app, installing the required packages, and setting up some
37+basic things for our front-end.
38+
39+#### [8. React App Logic](/docs/userspace/gall-2/8-http-api)
40+
41+Analysing the core logic of our React app, with particular focus on using
42+methods of the `Urbit` class from `@urbit/http-api` to communicate with our
43+agent.
44+
45+#### [9. Desk and Glob](/docs/userspace/gall-2/9-web-scries)
46+
47+Building and "globbing" our front-end, and putting together a desk for
48+distribution.
49+
50+#### [10. Summary](/docs/userspace/gall-2/10-final)
51+
52+Some final comments and additional resources.
content/docs/userspace/gall-guide/10-scry.md+5−4
@@ -154,10 +154,11 @@ can do something like:
154154
155155Previously we discussed custom `mark` files. Such mark files are most commonly
156156used when the data might be accessed through Eyre's HTTP API, and therefore
157−required JSON conversion methods. We'll cover such things in part two of the
158−Gall Guide, but note that if that's the case for your agent, you may wish to
159−also have your scry endpoints return data with your custom `mark` so it can
160−easily be converted to JSON when accessed from the web.
157+required JSON conversion methods. We cover such things separately in the
158+[Full-Stack Walkthrough](/docs/userspace/full-stack/1-intro), but note that if
159+that's the case for your agent, you may wish to also have your scry endpoints
160+return data with your custom `mark` so it can easily be converted to JSON when
161+accessed from the web.
161162
162163In some cases, typically with scry `path`s that contain wildcards like the `[%x %blah @ ~]` example above, your agent may not always be able to find the
163164requested data. In such cases, you can just produce a cell of `[~ ~]` for the
content/docs/userspace/gall-guide/12-next-steps.md+11−3
@@ -9,9 +9,11 @@ to start writing your own agent.
99
1010The things haven't touched on yet are front-end development and integration,
1111Eyre's HTTP API for communicating with agents from the web, and dealing with
12−JSON data. A second part to the Gall Guide is coming soon, and will focus
13−exclusively on those aspects. In the mean time, you can refer to the following
14−documents for help writing a web front-end for your app:
12+JSON data. The [Full-stack Walkthrough](/docs/userspace/full-stack/1-intro)
13+covers these aspects of Urbit app development, and it also puts into practice
14+many of the concepts we've discussed in this guide, so you might like to work
15+through that next. In addition to that walkthrough, you can refer to the
16+following documents for help writing a web front-end for your app:
1517
1618- [Eyre's external API reference](/docs/arvo/eyre/external-api-ref) - This
1719 explains Eyre's HTTP API, through which a browser or other HTTP client can
@@ -27,6 +29,12 @@ documents for help writing a web front-end for your app:
2729- [The software distribution guide](/docs/userspace/dist/dist) - This covers
2830 everything you need to know to distribute apps to other ships. It includes
2931 details of bundling a web front-end and serving it to the user in the browser.
32+- [The HTTP API guide](/docs/userspace/http-api-guide) - This is a reference
33+ and guide to using the `@urbit/http-api` NPM module.
34+- [The Sail guide](/docs/hoon/guides/sail) - Sail is a domain-specific language
35+ for composing XML structure in Hoon. It can be used to compose front-ends for
36+ Urbit apps directly in agents, as an alternative approach to having a
37+ separate Javascript app.
3038
3139In addition to these documents about creating a web-based user interface for
3240your app, there are some other guides you might like to have a look at:
content/docs/userspace/gall-guide/7-sur-and-marks.md+8−8
@@ -282,12 +282,13 @@ arm in the previous example could test for it instead of `%noun` like so:
282282Note how `%todo-action` will be resolved to `/mar/todo/action.hoon` - the hyphen
283283will be interpreted as `/` if there's not already a `/mar/todo-action.hoon`.
284284
285−This simple mark file isn't all that useful. Typically, you'd add `json` arms to
286−`grow` and `grab`, which allow your data to be converted to and from JSON, and
287−therefore allow your agent to communicate with a web front-end. Front-ends,
288−JSON, and Eyre's APIs which facilitate such communications will be covered in a
289−forthcoming part two of the Gall guide. For now though, it's still useful to use
290−marks and understand how they work.
285+This simple mark file isn't all that useful. Typically, you'd add `json` arms
286+to `grow` and `grab`, which allow your data to be converted to and from JSON,
287+and therefore allow your agent to communicate with a web front-end. Front-ends,
288+JSON, and Eyre's APIs which facilitate such communications will be covered in
289+the separate [Full-Stack Walkthrough](/docs/userspace/full-stack/1-intro),
290+which you might like to work through after completing this guide. For now
291+though, it's still useful to use marks and understand how they work.
291292
292293One further note on marks - while data from remote ships must have a matching
293294mark file in `/mar`, it's possible to exchange data between local agents with
@@ -384,8 +385,7 @@ Mark files:
384385- Messages passed between agents on a local ship don't necessarily need mark
385386 files in `/mar`.
386387- Mark files are most commonly used for converting an agent's native types to
387− JSON, in order to interact with a web front-end (which we'll cover in part two
388− of the Gall guide).
388+ JSON, in order to interact with a web front-end.
389389
390390Permissions:
391391