@@ -0,0 +1,369 @@
1+# Specification
2+
3+The `%eyre` vane serves one purpose: receiving inbound HTTP requests from browsers and clients pretending to be browsers. To facilitate this communication, `%eyre` does two things: 1. provide an API for traditional clients to talk to; and 2. provide a mechanism for saving state that allows the server to keep track of identities/sessions, subscription data, and active connections.
4+
5+This document outlines the following:
6+
7+1. [`perk`](#perk), a `pork` subset that defines what URLs the client can send to `%eyre`.
8+2. [the values stored on the client](#client), both statically in cookies and dynamically as the `window.urb` object, which facilitate this transaction.
9+
10+3. [`kiss`](#kiss), the requests `%eyre` will accept.
11+4. [`gift`](#gift), the responses it will give to those requests.
12+
13+5. [`note`](#note), the set of arvo requests that %eyre makes to the other vanes.
14+6. [`whir`](#whir), a `wire` subset, the associated dynamic state
15+7. [`sign`](#sign), the responses `%eyre` accepts
16+8. [`bolo`](#bolo), the state stored between separate events.
17+
18+# 1. URL structure [#perk]
19+There exist several URL forms accepted by `%eyre`.
20+
21+## 1.1 Functional request
22+The simplest of them is a static `%f` functional publish request:
23+
24+ ```hoon
25+ GET /[ship]/[desk]/[case]/path/to/file.[mark]
26+ ```
27+e.g.
28+
29+ ```hoon
30+ /~pittyp-pittyp/main/5/lib/urb.js
31+ ```
32+A `case` of 0 signifies the most recent version.
33+
34+If no file extension is provided, the default `mark` is `%urb`, which consists of `%html` injected with an auto-refresh script.
35+
36+A shorter form:
37+
38+ ```hoon
39+ /path/to.[mark]
40+ ```
41+is prepended with the [default request path](#prefix). For example, if `%eyre`
42+is configured with a root of `/<our>/main/0/pub`, then `/doc/arvo` is
43+interpreted as a request for `/=main=/pub/doc/arvo.urb`.
44+
45+Auxiliary paths start with `/~/`, and provide other functionality, such as:
46+
47+## 1.2 Auto-reload
48+When a new version of a page becomes available, it is useful to propagate it to the client. The `%f` vane manages the particulars, and provides an update token along with all published pages. The `/~/on` long-polling endpoint makes use of that token, which is a hash of the server-stored dependency set.
49+- `/~/on/[hash].json` retuns `true` when a page may require updating.
50+- `/~/on/[hash].js` is a lightweight script that polls for the above. It is primarily used in error messages.
51+
52+## 1.3 Authentication [#auth]
53+Authenticated requests are accomplished through sessions, tracked with cookies which are set on the first such request.
54+- `/~/as/[user]/...` is an authenticated request to `...`, where `user` can be a `@p` ship, `'own'`, `'any'`, or `'anon'`.
55+ + `'anon'` is a randomly-generated per-session submarine, and requires no further authentication.
56+ + `'own'` represents the serving ship. The short form of `/~/as/own/` is `/~~/`
57+ + `'any'` requires a network ship, chosen with a prompt if unknown. The short form of `/~/as/any/` is `/_/`
58+ + Auth requests are handled in three ways:
59+ + I. If the session is associated with the `ship`, serve `...`, storing the ship as the session's default ship.
60+ For example,
61+ /~/as/sivtyv-barnel/main/pub/test.html
62+ is an authenticated request of
63+ /main/pub/test.html
64+ with the identity of `~sivtyv-barnel`.
65+ + II. If the session is not associated with the `ship` in the path, the client is redirected to a login page, where a password can be entered. Successful authentication results in a page refresh to the requested path.
66+ + III. otherwise, [send a message](#foreign) to the claimed ship, and redirect the user to `https://[foreign-host]/~/am/[ses]/...#[current-host]&?code=[code]`
67+
68+
69+- `/~/am/[ses]/...#original-host` similarly serves a login page. Upon success, inform the `ship` of the success and redirect it back to `http://[original-host]/~/as/<this-ship>/...`
70+- `/~/away.html` is a log-out page.
71+- `/~/auth.json` returns `{ship,oryx,user,auth}`: the serving ship, a CSRF token, active user, and all allowed users respectively.
72+- `/~/at/{...}.js` is an alternate form of `/~/auth.json`, which injects a line into the served js file that sets `{ship, oryx, user, auth}` on the `window.urb` object.
73+
74+So far, all the paths specified have been GET requests. Authentication, however, requires sending data. This is also done at the `/~/auth.json` endpoint.
75+- `POST {oryx,ship,code} /~/auth.json?PUT` is used by the login script to authorize a user, and returns the same result as `GET`, additionally containing either`{ok:true}` or `{fail:'type',mess:"Error message"}`
76+- `POST {oryx,ship?} /~/auth.json?DELETE` revokes authorization. Response as above.
77+
78+## 1.4 Messaging
79+- `POST {oryx, xyro} /~/to/[app]/[mark].json`, where `xyro` is data that will be converted to the `mark`. In the simplest case, `/~/do/hello/json.json` will pass `xyro` through verbatim, sending `[%json xyro]` to the `++poke` arm of app `%hello`.
80+- `POST {oryx, xyro} /~/to/<ship>/[app]/[mark].json` is a foreign message send, as above.
81+
82+## 1.5 Subscriptions [#subs]
83+- `/~/of/[ixor]`[#of-ixor] is an `EventSource`: a conceptually infinite file containing a stream of events. By default, it sends a newline every 30 seconds serving to signal that the connection is alive to both the server and IP middleware.
84+ + `/~/of/[ixor]?poll={n}` is a long-polling fallback interface. Produces event number {n}, blocking until it occurs.
85+
86+Its contexts can be affected by `POST` requests with a body of `{oryx, wire}`, and a query string of `PUT` or `DELETE`. They return either `{mark}`(which may be `null`), or [`{fail, mess}`](#mean-json)
87+
88+- `/~/in/[hash].json` is a dual to [`/~/on`](#on-change). It binds `mod` events, which echo the requesting token.
89+- `/~/is/[app]/path/to.[mark]` binds `rush` events, formatted with a first line of `[app] /path/to`, and the rest containing subscription data. Additionally, `mean` events formatted similarly may arrive, also converted to the relevant mark.
90+
91+## 1.6 Ablative [#temp]
92+These interfaces will temporarily exist to aid development, and are to be considered unstable.
93+- `/~/debug/...` access normally inaccessible pages. For example, `/~/debug/as.html` will present the login page, regardless of current session status.
94+
95+# 2. Client state [#client]
96+Some information is stored on, and provided to, browser clients
97+
98+## 2.1 Cookies
99+Authenticated users receive a cookie on the domain of `*.urbit.org`. The cookie contains a client session token, keyed by the serving ship.
100+
101+## 2.2 Authentication [#auth-json]
102+It is common(e.g. by the `%urb` mark) to set window.urb to the contents of `/~/auth.json`:
103+- `ship` is the serving ship
104+- `oryx` is a unique CSRF token that identifies this view, bound to the session cookie
105+- `ixor` is a hash of `oryx`, used as an insecure view identifier
106+- `user` is the authenticated/generated ship name
107+- `auth` is a list of identities usable by this session
108+
109+## 2.3 Client library
110+To this object, `/main/lib/urb.js` adds helpers:
111+- `app` is an optional app name to send messages to by default
112+- `send({data,mark?='json',app?=urb.app,ship?}, cb?)` delivers messages
113+- `bind({path,app?=urb.app}, cb)` subscribes by path
114+- `drop({path,app?=urb.app}, cb?)` pulls the subscription
115+- `util` is an object containing methods for converting between JavaScript types and Hoon atom odors.
116+
117+# 3. Requests [#kiss]
118+
119+## `[%born port=@ud]`, unix init
120+When the `vere` process is re/started, a TCP port is bound, HTTP requests on which are the scope of this document. The `%born` gift informs eyre of this occurrence, which in turn updates [its state.](#bolo)
121+
122+**Gifts given in response:** none.
123+
124+## `[%hiss p=mark q=cage]`, outbound http
125+Userspace http request, converted by `%ford` into an actual `hiss`.
126+
127+**Gifts given in response:** a [`%sigh`]()
128+
129+## 3.1 `[%this httq]`, inbound http
130+The primary `%eyre` request type comes from the host system, and constitutes an inbound http request. It consists of five parts:
131+
132+I. `p=[? @if]`: a loobean determining whether the request is being made over https, and the client source, an IPv6 address possibly encoding an IPv4 one as `.0.0.0.0.0.ffff.wwxx.yyzz`
133+II. `q=meth`, an http method
134+III. `r=@t`, the unparsed URL
135+IV. `s=mess`, an associative list of header keys/values
136+V. `t=(unit octs)`, the request body, if any.
137+
138+After `r` is parsed to a `purl`, the `pork`(relative path) determines the resource being served. The first step in handling an http request is [classifying its intent](#perk).
139+
140+- A ford request results in a [`%boil` note](#boil).
141+- `/~/[name].js` is generated from [session](#sink) or [view](#stem) state, and [served](#mime) as `%js`.
142+- `/~/am` generates a login page.
143+- `/~/as` is resolved further if authenticated, generates a similar login page if the claimed ship is local, and otherwise is saved to [session state](#sink) and sends a ["login" `%a` message](#lon) to the foreign ship.
144+- `/~/at` and `/~/auth.json` generate a new `oryx` and store it in the [session state](#sink).
145+- `/~/in` and `/~/is` affect [view state](#stem), possibly sending subscriptions or unsubscriptions to `%f` and `%g` respectively, and are generated from view state.
146+- `/~/on` is saved to the `bolo` by `hash`, and translated to a [`%wasp` note](#wasp).
147+- `/~/of` fails if the `ixor` is unknown, and otherwise is stored in the `ixor`, receiving a partial response and all events after its [last-seen header](#). If it has a `poll` parameter, the response is full, and only occurs if the body is non-empty. Otherwise the oryx is saved as a (live)[#live] request in case of cancellation.
148+- `/~/to` transforms into a [`%g` message](#mess).
149+
150+**Gifts given in response:** a [`%thou`](), or a [`%that`](#that) followed by multiple [`%thar`]()s.
151+
152+## 3.2 `[%thud ~]`, inbound close
153+Sent when an unresponded-to request is cancelled. The [live](#live) effect of the request is looked up, and
154+- `%exec` ford boil requests are cancelled with `[%exec ~]`
155+- `%wasp` ford deps requests are cancelled with `[%wasp |]`
156+- `%poll` session polls `%rest` the session heartbeat timeout
157+
158+## 3.3 `[%thin p=@ud]`, delivery failure
159+Sent when a partial [response](#that) or [body](#thar) fails to arrive. Its `duct` is the same as that of the original request. This is handled by [waiting](#wait) for the client to reconnect, and if this not occur, unwinding all subscription state.
160+
161+**Gifts given in response:** None.
162+
163+## 3.4 ``[%wart sock [path *]]``, inbound message
164+Sometimes, messages are received from other ships. They are expected to take the from of a [`gram`]().
165+
166+**Gifts given in response:** [`%nice` or `%mean`](#ack).
167+
168+### 3.4.1 `gram` [#gram]
169+
170+There are three messages of note, all concerning authentication [#foreign]
171+- `[/lon ses]` [#lon] is a login request, which contains the session wishing to authenticate.
172+- if the session is already authorized, a reply of `[/aut ses]` from (%eyre on) a foreign ship prescribes that the session is henceforth allowed to act on its behalf.
173+All waiting `/~/as` are resolved as succesful.
174+- otherwise, `[/hat ses hart]` contains the ship's preferred hostname. This can then be used to redirect the client.
175+All waiting `/~/as` are given `307 Redirect`s to `/~/am` on the provided host.
176+
177+**Gifts given in response:** nice?
178+
179+# 4. Responses [#gift]
180+## 4.1 `[%nice ~]`, `[%mean ares]`, network acknowledgement [#ack]
181+A `%nice` is given upon receiving an [ames message](#gram), indicating succesful receipt. `%mean` is currently unused directly, but reserved for error conditions.
182+
183+## `[%sigh p=cage]`, inbound http response
184+An HTTP response gets converted to the requested mark, or `%tang` in case of error.
185+
186+## 4.2 `[%thou httr]`, full HTTP response
187+Most requests are served with one coherent response, consisting of
188+- `p=@ud`, an HTTP status code
189+- `q=mess`, response headers
190+- `r=(unit octs)`, an optional body
191+
192+## 4.3 `[%that httr]`, partial HTTP response [#that]
193+[EventStream responses](#of-ixor) consist of multiple sequential chunks. Treated as `%thou`, except the request is kept open.
194+
195+## 4.4 `[%thar (unit octs)]`, partial HTTP body [#thar]
196+Complementing `%that`, `%thar` is a body chunk, containing an event. An empty `%thar` signals for the connection to be closed.
197+
198+Body chunks, besides `[1 '\0a']`(a heartbeat newline), are encoded from `even` events. The stem becomes the [`event` field](#eventsource), and the content, `data` lines.
199+
200+### 4.4.1 `even`, event types [#even]
201+There are three events that a [client subscription](#subs) will be given.
202+
203+- `[%news hash]`[#even-news] is a `%f` update, and contains the relevant dependency token.
204+- `[%rush [term path] wain]`[#even-rush] is sourced subscription data.
205+- `[%mean [term path] ares]` is a sourced subscription error.
206+
207+# 5. Vane requests [#note]
208+## 5.1 ``[%a %wont sock `[path *]`gram]``, outbound message
209+The `%a` interface provides conveyance of [messages](#gram) over UDP.
210+
211+Signs a `%woot` upon message arrival.
212+
213+## 5.2 `[%b ?(%wait %rest) time]`, timeout set/unset [#wait]
214+All open [subscriptions](#of-ixor) require a "heartbeat" newline every `~s30`. When this fails to arrive, a complementary timer is set for `~m1`, after which the client is considered to have departed.
215+The `%b` timer interface is used to schedule the next such event, or cancel past scheduled ones when a connection closes.
216+
217+Signs a `%wake` upon timer activation.
218+
219+## 5.3 `[%f %wasp @uvI]`, dependency listen [#wasp]
220+Knowledge of filesystem changes, requested by `/~/on` and `/~/in`, is requested the `%wasp` note. It contains the hash token which idenitifes a set of dependencies to query. Saved as a (live)[#live] request in case of cancellation when caused by `/~/on`.
221+
222+Signs a `%writ` upon change.
223+
224+## 5.4 `[%f %exec (unit silk)]`, functional transformations
225+The `%f` interface converts nouns from one form to another. An empty unit represents cancellation.
226+
227+Signs a `%made` containing the computation result.
228+
229+The `silk`s used are as follows:
230+
231+### `[%cast %mime %boil mark beam [%web span ~]]`, file load
232+The simplest functional request is the construction of a page. Saved as a (live)[#live] request in case of cancellation.
233+- The `mark` is the request extension, defaulting to `%html`.
234+- A `beam`(path in `%clay`) is decoded from the request path.
235+- A `/web/<nyp ced quy>` virtual path is appended, span-encoding method, auth, and query string.
236+
237+### `[%cast mark %done ~ cage]`, convert [#done]
238+This is used when communicating with apps, in both directions.
239+- Client messages are encoded as JSON objects, but can contain many different marks. Ones sent to non-`json.json` endpoints are `%cast` to he relevant mark, with a `cage` of `[%json p:!>(*json) jon]`, prior to being sent onwards to applications with `[%g %mess]`.
240+- Subscription responses also come in arbitrary marks, but are required to be sent as whichever mark an endpoint is subscribed to. They get `%cast` prior to being sent back as a `%that` EventSource segment.
241+
242+## 5.5 `%g`, app actions
243+The end goal of many a userspace `hymn.hook` is to provide UI for a `%gall` app. To accommodate this, various functionality needs to be interfaced with.
244+
245+### `[%mess hapt ship cage]`, app message [#mess]
246+After a `/~/to` POST has been received, and possibly converted to the correct mark, it is sent to `%g` for processing. The `hapt` is the destination, the `ship` is the source, the `cage` is the marked message data.
247+
248+Signs a `%nice` or `%mean`, in userspace or upon crashing.
249+
250+### `[%show hapt ship path]`, app subscription[#show]
251+An `/~/is` PUT, if not already registered, results in a new subscription. A request for such contains the destination `hapt`, requesting `ship`, and app-internal `path`.
252+
253+Signs a `%nice` on init, `%rush` on data, and a `%mean` in subscription termination.
254+
255+### `[%took hapt ship]`, %rush confirmation
256+A `%took` must be sent to acknowledge the receipt of subscription data, which in this case occurs automatically.
257+
258+Causes no signs.
259+
260+### `[%nuke hapt $|(ship [ship path])]`, subscription cancel
261+An `/~/is` DELETE is used to remove a subscription, and is converted straightforwardly to a `%nuke`. Additionally, when a client disconnects, all of its subscriptions are `%nuke`d to reflect their experiation.
262+
263+Signs an empty `%mean` for each open subscription that is closed.
264+
265+# 6. In-flight metadata [#whir]
266+Various state can be associated with requests, but not necessarily be returned in responses to them.
267+
268+## 6.1 `~`, dropthrough [#wire-drop]
269+If the response should be sent statelessly further up the duct, the `wire` is empty. This is used in `%f` pure functional page generation.
270+
271+## 6.2 `[?(%y %n) ...]`, stability [#wire-live]
272+The first element of a [`%b` timer](#wait) path distinguishes between live(%y) and dying(%n) channels.
273+
274+## 6.3 `/of/[ixor]`, view
275+Designates which stream to act upon. Present on `%b` timer cards.
276+
277+## 6.4 `/on/<deps>`
278+Present on `%f` dependency news. Looked up in [the state](#bolo) to see if any response is necessary.
279+
280+## 6.5 `/to/<hasp>/<ship>`, app [#wire-to]
281+Present on `%f` translations of message marks
282+
283+## 6.6 `/is/[ixor]/{hasp path}`, subscription [#wire-is]
284+Subscription requests. Present on `%g` subscription requests, and `%f` translations of [`%rush` subscription data](#show).
285+
286+# 7. Vane responses [#sign]
287+The goal of requests is to cause some manner of result. Specifically,
288+
289+## 7.1 `[%a %woot ship coop]`, acknowledgement
290+A [message](#mess) which has been acknowledged remotely.
291+
292+## 7.2 `[%b %wake ~]`, timer
293+Timer activate. If this is a `%y` [timer](#wire-live), send a newline to each `/~/of/[oryx]`; if `%n` and the `oryx` still has no listeners, wipe its state/subscriptions.
294+
295+## 7.3 `[%f %news ~]`, dependency update
296+This message signifies a `hash` has changed, and listeners to it should be updated. Results in [`%news` events](#even-news) to concerned streams, and [full responses](#thou) to relevant long-polls.
297+
298+## 7.4 `[%f %made (each bead (list tank))]`, compute result
299+Further action depends on the [`wire`](#whir). It can:
300+- Be empty, designating direct request data, which is [served](#mime) back up the `duct`.
301+- Contain [app and requester identities](#wire-to), which designates an app message has been converted to the correct `mark`, and can now be [sent to the app](#mess) with an empty `wire`.
302+- Contain an `oryx`, app, and `path`, designating subscription data. If the subscription is active in the relevant `stem`, it is acknowledged with a `%took` and given as a [`%rush` event](#even-rush) to all listeners. Otherwise, the subscription is `%nuke`d.
303+
304+## 7.5 `%g`, applications
305+The `%g` interface sends several responses.
306+
307+### `[%nice ~]` and `[%mean ares]`, acknowledgment
308+These are given as a response to `%mess`, with an empty wire, and `%show`s, with a [subscription wire](#wire-is).
309+The former is [served](#mime) back to the requester(i.e. the remaining duct), and the latter returned to the client.
310+
311+### 7.5 `[%rush cage]`, subscription data
312+A `%rush` arrives on a [subscription wire](#wire-is), and is then sent [for conversion](#done) to the correct mark. An unexpected `%rush` is `%nuke`d.
313+
314+# 8. Server state [#bolo]
315+There is a quantity of data that persists between events.
316+
317+## 8.1 Global
318+- `path` [#prefix] the default path which is prepended to relative plain requests. Initialized to `/<our>`, the serving ship; this interprets the first element of requests as a desk, and injects a `case` of `0`.
319+- `(jar ship hart)` remembered hostnames in order of preference, made accessible through `.^`
320+ + The default values(for self) are `http://<ship>.urbit.org:80` if not on a fake network, followed by `https://0.0.0.0:[port]`.
321+- `(jug deps (each duct oryx))`, `/~/on` and `/~/in` endpoints listening to `%c` updates.
322+- `(map duct live)`, inbound request state
323+- `(map ixor oryx)`, open `/~/of` streams.
324+- `(map oryx stem)`, open view state.
325+- `(map hole sink)`, session state.
326+- `(map hole ,[ship ?])`, foreign session names, along with their origin ships and whether they are authorized to act on our behalf.
327+
328+## 8.2 `live`, per-request state [#live]
329+To honor request cancellations, each unserved request must track which effect it is causing.
330++ `[%exec whir]` if it is a ford data request
331++ `[%wasp (list @uvH)]` if it is a ford dependency poll
332++ `[%xeno ship]` if this is a proxied request
333++ `[%poll ixor]` if this is a session long-poll
334+
335+## 8.3 `sink`, per-session state [#sink]
336+Each `hole` has associated authentication state.
337+- `priv`, a random secret that verifies the session, stored in cookie.
338+- `[ship (set ship)]`, the `ship` a session is acting as and any others it has been authorized with.
339+- `(jug ship ,[path duct])`, authentication `/~/as` requests waiting on foreign ships.
340+- `(set oryx)`, views associated with this session. Dual of `hole` in `stem`.
341+
342+## 8.4 `stem`, per-view state [#stem]
343+Each `oryx` has active subscription state.
344+- `hole`, session in which this view resides. Dual of `(set oryx)` in `sink`.
345+- `ixor`[#ixor], a cached hash of the `oryx` that is used as a subscription id, by the `/~/of` EventStream.
346+- `[,@u (map ,@u even)]`, queued [events](#even). Has a maximum size.
347+- `(unit ,[duct @u ?])`, http connection, its last received event, and whether it is a long-poll request.
348+- `(set deps)`, subscribed `/~/in` points. Mirrored in `bolo`.
349+- `(map ,[hasp path] mark)`, subscribed `/~/is` points
350+- `(que duct)`, where any `%nice`/`%mean
351+
352+# Appendix A: Glossary
353+- An `oryx` is a CSRF token used for [authenticated requests](#auth)
354+- `mean.json`[#mean-json] is a rendering of hoon `ares`: an error message formatted as `{fail:'type',mess:"Error message"}`
355+- `urb.js`[#urb-js] is the standard client-side implementation of the `%eyre` protocols, normally found in `/=main=/lib`
356+- The act of "serving" \[#mime], refers to the wrapping of a `%mime` cage in an HTTP `200 Success` response, and errors or other marks being [sent to ford](#done) for conversion. Inside a `sign`, this conversion occurs along the same [`wire`](#wire); otherwise, the `wire` is [empty](#wire-drop).
357+
358+# Appendix B: EventSource [#eventsource]
359+For more information, see <http://dev.w3.org/html5/eventsource/>
360+
361+An EventSource is a stream of events, encoded in a conceptually infinite `GET` request. Each event is formatted as
362+ ```http
363+ id: {number}
364+ event: {type}
365+ data: {contents}
366+ data: {more-contents}
367+ ```
368+
369+Data lines are joined by ASCII `10` newline characters; multiple newlines signify separate events. The `id` is used in a `Last-Event-ID` HTTP request header upon reconnection.