content/docs/userspace/http-api-guide.md · urbit.org

git.iko.soy/urbit.org

on 6a0cb5f help sign in

urbit.org/content/docs/userspace/http-api-guide.md

685 lines 33 kB history raw

1+++

2title = "HTTP API Guide"

3weight = 4

4template = "doc.html"

5+++

7[Eyre] is the web-server [vane] (kernel module) of [Arvo], an Urbit ship's

8kernel and operating system. In this guide, we'll look at interacting with a

9ship through Eyre's web API using the [@urbit/http-api] Javascript module.

11## Background

13Before we dig into the details of Eyre's API, we'll first give a quick run-down

14of some concepts in Arvo. Eyre's API is a fairly thin overlay on some of Arvo's

15internal systems, so there's some basic things to understand.

17### Clay

19[Clay] is the filesystem vane. It's typed, and it's revision-controlled in a

20similar way to git. Clay is comprised of a number of [desk]s, which are a bit

21like git repositories. Each app on your ship's home screen corresponds to a desk

22in Clay. That desk contains the source code and resources for that app.

24#### Marks

26Most of Clay's workings aren't relevant to front-end development, but there's

27one important concept to understand: [mark]s. Clay is a typed filesystem, and

28marks are the filetypes. There's a mark for `.hoon` files, a mark for `.txt`

29files, and so on. The mark specifies the hoon datatype for those files, and it

30also specifies conversion methods between different marks. Marks aren't just

31used for files saved in Clay, but also for data that goes to and from the web

32through Eyre.

34When you send a [poke](#pokes) or run a [thread](#threads) through Eyre, it will

35look at the mark specified (for example `graph-action-3`, `settings-event`, etc)

36and use the corresponding mark file in the desk to convert the given JSON to

37that datatype before passing it to the target [agent](#gall-agents) or thread.

38The same conversion will happen in reverse for responses.

40Note that Eyre makes a best effort attempt to convert data to and from JSON. If

41the marks in question do not contain appropriate JSON conversion functions, it

42will fail. Not all [scry endpoints](#scry-endpoints), [subscription

43paths](#subscriptions), pokes, etc, are intended to be used from a front-end, so

44not all use marks which can be converted to and from JSON (the `noun` mark for

45example). If documentation is available for the agent or thread in question, it

46should note in some fashion whether it can take or produce JSON. The majority of

47things you'll want to interact with through Eyre _will_ work with JSON.

49### Gall agents

51An _agent_ is a userspace application managed by the [Gall] vane. A desk may

52contain multiple agents that do different things. The Groups app, for example,

53has `graph-store`, `group-store`, `contact-store`, and others in its desk.

54Agents are the main thing you'll interact with through Eyre. They have a simple

55interface with three main parts:

57| Interface | Description |

58| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

59| Pokes | One-off message to an agent. Pokes often represent actions, commands, requests, etc. |

60| Subscriptions | An agent may have a number of different paths to which you may subscribe. If a subscription request is accepted, you'll continue to receive update/events the agent sends out on that path until you either cancel the subscription or are kicked by the agent. |

61| Scries | Scries are one-off requests for data. Like subscriptions, they are organized by path. Scry requests will be fulfilled immediately. Scries are "read-only", they cannot alter the state of an agent. |

63#### Pokes

65Pokes are single, stand-alone messages to agents. Pokes are how you send data

66and requests to agents. Agents will send back either a positive acknowledgement

67(ack) or a negative acknowledgement (nack). The agent can't send actual data

68back in the acks. If they have any response to give back, it will be sent out to

69subscribers on a subscription path instead.

71The pokes an agent accepts will be defined in the `on-poke` section of its

72source in the desk's `/app` directory, and typically in its type definition file

73in the `/sur` directory too. If the agent has documentation, the data structures

74and marks of its pokes should be listed there.

76`http-api` includes a `poke` function which allows you to perform pokes through

77Eyre, and is [detailed below](#poke).

79#### Subscriptions

81Agents define subscription paths which you can subscribe to through Eyre. A path

82might be simple and fixed like `/foo/bar`, or it might have variable elements

83where you can specify a date, ship, or what have you. Each agent will define its

84subscription paths in the `on-watch` section of its source, and if it has

85documentation the paths should also be listed there.

87You can subscribe by sending a watch request to the agent with the desired path

88specifed. The agent will apply some logic (such as checking permissions) to the

89request and then ack or nack it. If acked, you'll be subscribed and begin

90receiving any updates the agent sends out on that path. What you'll receive on a

91given path depends entirely on the agent.

93Agents can kick subscribers, and you can also unsubscribe at any time.

95`http-api` includes a `subscribe` function which allows you to perform pokes through

96Eyre, and is [detailed below](#subscribe).

98#### Scry Endpoints

100Pokes and subscriptions can modify the state of the agent. A third kind of

101interaction called a _scry_ does not - it simply retrieves data from the agent

102without any side-effects. Agents can define _scry endpoints_ which (like

103subscriptions) are paths. A scry to one of these endpoints will retrieve some

104data (as determined by the agent's design) from the agent's state. Like

105subscription paths, scry paths can be simple like `/foo/bar` or contain variable

106elements. Unlike subscriptions, a scry is a one-off request and the data will

107come back immediately.

108

109Scry endpoints are defined in the `on-peek` section of an agent's source, and

110will likely be listed in the agent's documentation if it has any. Scry endpoints

111will often be written with a leading letter like `/x/foo/bar`. All scries

112through Eyre are `x` scries, so that letter (called a `care`) needn't be

113specified.

114

115`http-api` includes a `scry` function which allows you to perform scries through

116Eyre, and is [detailed below](#scry).

117

118### Threads

119

120A thread is a monadic function in Arvo that takes arguments and produces a

121result. Threads are conceptually similar to Javascript promises - they can

122perform a series of asynchronous I/O operations and may succeed or fail. Threads

123are often used to handle complex I/O operations for agents. Threads live in the

124`/ted` directory of a desk.

125

126`http-api` includes a `thread` function which allows you to run threads through

127Eyre, and is [detailed below](#thread).

128

129## `http-api` basics

130

131The `http-api` module provides convenient functions to interact with a ship

132through Eyre. In this section we'll discuss the basics of how it works.

133

134### Importing the module

135

136The `http-api` module is available in npm as `@urbit/http-api`, and can be

137installed with:

138

139```

140npm i @urbit/http-api

141```

142

143Once installed, you can import it into your app with:

144

145```

146import Urbit from '@urbit/http-api';

147```

148

149Note that the examples in this guide are simple HTML documents with vanilla

150Javascript in `<script>` tags, so they use [unpkg.com](https://unpkg.com) to

151import `@urbit/http-api`. This is not typical, and is just done here for

152purposes of simplicity.

153

154### `Urbit`

155

156All functionality is contained within the `Urbit` class. There are two ways to

157instantiate it, depending on whether your web app is served directly from the

158ship or whether it's served externally. The reason for the difference is that

159you require a session cookie to talk to the ship. If your app is served from the

160ship, the user will already have a cookie. If it's not, they'll need to

161authenticate, which is [detailed separately below](#authentication).

162

163In the case of a front-end served from the ship, the `Urbit` class contains a `constructor` which takes three arguments:

164

166| -------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |

167| `url` | `string` | The host of the ship without the protocol. This is mandatory but would typically be left blank as requests will still work if they're root-relative paths. | `"example.com"`, `"localhost:8080"`, `""` |

168| `code` | `string` | The web login code of the ship. You'll typically use this constructor when it's served directly from the ship and therefore you'll already have a session cookie, so this field is optional and is typically omitted or left empty. | `""`, `"lidlut-tabwed-pillex-ridrup"` |

169| `desk` | `string` | The desk on which you want to run threads. This field is optional and is only used if you want to run threads, no other function uses it. If you're not running threads, you can omit it or leave it empty. | `"landscape"`, `""` |

170

171To create an `Urbit` instance, you can simply do:

172

173```javascript

174const api = new Urbit("");

175```

176

177If you want to specify a desk, you can do:

178

179```javascript

180const api = new Urbit("", "", "landscape");

181```

182

183### `/session.js`

184

185Most functions of the `Urbit` class need to know the ship name or they will

186fail. This is given explicitly with the external authentication method [detailed

187below](#authentication), but not when using the `Urbit` constructor in a web app

188served directly from the ship. Rather than having the user manually enter it,

189Urbit ships serve a JS library at `/session.js` that contains the following:

190

191```javascript

192window.ship = "zod";

193```

194

195`"zod"` will be replaced with the actual name of the ship in question. You can

196include this file like:

197

198```

199<script src="/session.js"></script>

200```

201

202Then you need to set the `ship` field in the `Urbit` object. You would typically

203do it immediately after instantiating it:

204

205```javascript

206const api = new Urbit("");

207api.ship = window.ship;

208```

209

210### Channels

211

212With the exception of scries and threads, all communication with Eyre happens

213through its channel system.

214

215The `Urbit` object will generate a random channel ID that looks like

216`1646295453-e1bdfd`, and use a path of `/~/channel/1646295453-e1bdfd` for

217channel communications. Pokes will and subscription requests will be sent to

218that channel. Responses and subscription updates will be sent out to you on that

219channel too.

220

221Eyre sends out updates and responses on an [SSE] (Server Sent Event) stream for

222the channel. The `Urbit` object handles this internally with an `eventSource`

223object, but you won't deal with it directly. Eyre requires all events it sends

224out be acknowledged by the client, and will eventually close the channel if

225enough unacknowledged events accumulate. The `Urbit` object handles event

226acknowledgement automatically, so you don't need to worry about this.

227

228Eyre automatically creates a channel when a poke or subscription request is

229first sent to `/~/channel/[a new channel ID]`. If your web app is served

230external to the ship and you use the `authenticate` function [described

231below](#authentication), the function will automatically send a poke and open

232the channel. If your web app is served directly from the ship and you use

233`Urbit` class constructor, it won't open the channel right away. Instead, the

234channel will be opened whenever you first send a poke or subscription request.

235

236### Connection state

237

238`Urbit` class includes three optional callbacks that fire when the SSE

239connection state changes:

240

241| Callback | Description |

242| `Urbit.onOpen` | Called when an SSE channel connection is successfully established. |

243| `Urbit.onRetry` | This is called whenever a reconnection attempt is made due to an interruption, for example if there are network problems. |

244| `Urbit.onError` | This is called when there is an unrecoverable error, for example after enough reconnection attemps have failed. |

245

246As mentioned in the previous section, typically a channel will be opened and an

247SSE connection established after you first poke the ship or make a subscription

248request. If successful, whatever function you provided to `onOpen` will be

249called. If at some point the connection is interrupted, a reconnection attempt will be made three times:

250

2511. Instantly.

2522. 750ms after the first.

2533. 3000ms after the second.

254

255Each attempt will call the function you provided to `onRetry`, if any. If all

256three reconnection attempts failed, or if a fatal error occurred, the function

257you provided `onError` will be called with an `Error` object containing an error

258message as its argument.

259

260How you use these (if you do at all) is up to you and depends on the nature of

261your app. If you want to try reconnecting when `onError` fires, note that Eyre

262will delete a channel if it's had no messages from the client in the last 12

263hours. The timeout is reset whenever it receives a message, including the acks

264that are automatically sent by the `Urbit` object in response to subscription

265updates.

266

267If you don't want to account for the possibility of the channel having been

268deleted in your app's logic, you can also just call the [`reset()`](#reset)

269function before you try reconnecting and consequently open a brand new channel.

270

271### Authentication

272

273**If your front-end is served directly from the Urbit ship, this step can be

274skipped.** If your web app is served externally to the ship, you must

275authenticate and obtain a session cookie before commencing communications with

276the ship.

277

278The `Urbit` class in `http-api` includes an `authenticate` function which does

279the following:

280

2811. Login to the user's ship with their `code` and obtain a session cookie.

2822. Generate a random channel ID for the connection.

2833. Poke the user's ship and print "opening airlock" in the dojo to initialize

284 the channel.

285

286The `authenticate` function takes four arguments in an object: `ship`, `url`,

287`code` and `verbose`:

288

290| --------- | --------- | -------------------------------------------------------------------------------------- | --------------------------------- |

295

296This function returns a promise that if successful, produces an `Urbit` object

297which can then be used for communications with the ship.

298

299**Note:** An Urbit ship will deny CORS requests from external URLs by default.

300In order to allow CORS for an external web app, the user will need to run the

301following in the dojo (replacing `https://example.com` with the address in

302question):

303

304```

305|cors-approve 'https://example.com'

306```

307

308#### Example

309

310Here's a simple example that will run `authenticate` for a fakezod when the

311_Connect_ button is pressed:

312

313```

314<html>

315 <head>

316 <script src="https://unpkg.com/@urbit/http-api"></script>

317 </head>

318 <body>

319 <button id="start" type="button" onClick="connect()" >Connect</button>

320 </body>

321 <script>

322 async function connect() {

323 window.api = await UrbitHttpApi.Urbit.authenticate({

324 ship: "zod",

325 url: "localhost:8080",

326 code: "lidlut-tabwed-pillex-ridrup",

327 verbose: true

328 });

329 document.body.innerHTML = "Connected!";

330 };

331 </script>

332</html>

333```

334

335`window.api` can then be used for further communications.

336

337## Functions

338

339The various functions you can use in the `Urbit` object are described below.

340

341### Poke

342

343For poking a ship, the `Urbit` class in `http-api` includes a `poke` function.

344The `poke` function takes six arguments in a object:

345

347| ----------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------- | ----------------------- |

350| `json` | any JSON | The data to poke the agent with. | `{foo: "bar", baz: 42}` |

354

355#### Example

356

357This example lets you "hi" your ship with a given message, which will be printed

358in the terminal. It pokes the `hood` agent (the system agent) with a `helm-hi`

359mark. The `json` argument just contains the message in a string. If for some

360reason the `hood` agent rejects the poke, "Poke failed!" will be displayed on

361the page.

362

363```

364<html>

365 <head>

366 <script src="https://unpkg.com/@urbit/http-api"></script>

367 <script src="/session.js"></script>

368 </head>

369 <body>

370 <input id="msg" type="text" placeholder="Message for ship" />

371 <button id="submit" type="button" onClick="doPoke()" >Submit</button>

372 <p id="err"></p>

373 </body>

374 <script>

375 document.getElementById("msg")

376 .addEventListener("keyup", function(event) {

377 if (event.keyCode === 13) {

378 document.getElementById("submit").click();

379 }

380 })

381 const api = new UrbitHttpApi.Urbit("");

382 api.ship = window.ship;

383 function doPoke() {

384 const msg = document.getElementById("msg").value;

385 api.poke({

386 app: "hood",

387 mark: "helm-hi",

388 json: msg,

389 onSuccess: success,

390 onError: error

391 });

392 }

393 function success() {

394 document.getElementById("msg").value = "";

395 document.getElementById("err").innerHTML = "";

396 }

397 function error() {

398 document.getElementById("msg").value = "";

399 document.getElementById("err").innerHTML = "Poke failed!";

400 }

401 </script>

402</html>

403```

404

405### Subscribe and Unsubscribe {#subscribe}

406

407For subscribing to a particular path in an agent, the `Urbit` class in `http-api` includes a `subscribe` function.

408The `subscribe` function takes six arguments in a object:

409

411| -------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------- |

418

419The `subscribe` function returns a subscription ID, which is just a number. This

420ID can be used to unsubscribe down the line.

421

422If the subscription request is successful, you'll continue to receive updates

423until you either unsubscribe or are kicked by the agent. You may subscribe to

424multiple different agents and subscription paths by calling the `subscribe`

425function for each one.

426

427If you wish to unsubscribe from a particular subscription, the `Urbit` class in

428`http-api` includes an `unsubscribe` function. This function just takes a single

429argument: the subscription ID number of an existing subscription. Once

430unsubscribed, you'll stop receiving updates for the specified subscription.

431

432#### Example

433

434This example will, upon clicking the "Subscribe" button, subscribe to the

435`/updates` path of the `graph-store` agent, and print the raw JSON of each

436update it receives. You can test it by doing something like posting a message in

437a chat. You can also unsubscribe by clicking the "Unsubscribe" button.

438

439```

440<html>

441 <head>

442 <script src="https://unpkg.com/@urbit/http-api"></script>

443 <script src="/session.js"></script>

444 </head>

445 <body>

446 <button id="toggle" type="button" onClick="doSub()" >Subscribe</button>

447 <pre id="event"></pre>

448 </body>

449 <script>

450 const api = new UrbitHttpApi.Urbit("");

451 api.ship = window.ship;

452 const toggle = document.getElementById("toggle");

453 const event = document.getElementById("event");

454 function doSub() {

455 window.id = api.subscribe({

456 app: "graph-store",

457 path: "/updates",

458 event: printEvent,

459 quit: doSub,

460 err: subFail

461 });

462 toggle.innerHTML = "Unsubscribe";

463 toggle.onclick = doUnsub;

464 event.innerHTML = "Awaiting event";

465 };

466 function doUnsub() {

467 api.unsubscribe(window.id);

468 toggle.innerHTML = "Subscribe";

469 toggle.onclick = doSub;

470 event.innerHTML = "Subscription closed";

471 };

472 function printEvent(update) {

473 event.innerHTML = JSON.stringify(update, null, 2);

474 };

475 function subFail() {

476 event.innerHTML = "Subscription failed!";

477 };

478 </script>

479</html>

480```

481

482### Subscribe Once

483

484The `subscribeOnce` function in the `Urbit` class of `http-api` is a variation

485on the ordinary [`subscribe`](#subscribe) function. Rather than keeping the

486subscription going and receiving an arbitrary number of updates, instead it

487waits to receive a single update and then closes the subscription. This is

488useful if, for example, you send a poke and just want a response to that one

489poke.

490

491The `subscribeOnce` function also takes an optional `timeout` argument, which

492specifies the number of milliseconds to wait for an update before closing the

493subscription. If omitted, `subscribeOnce` will wait indefinitely.

494

495`subscribeOnce` takes three arguments (these can't be in an object like most

496other `Urbit` functions):

497

499| --------- | -------- | ------------------------------------------------------------------------------------------------------------------------ | --------------- |

503

504`subscribeOnce` returns a Promise. If successful, the Promise produces the JSON

505data of the update it received. If it failed due to either timing out or getting

506kicked from the subscription, it will return an error message of either

507`"timeout"` or `"quit"`.

508

509#### Example

510

511Here's a variation on the ordinary [subscribe](#subscribe) function. Rather than

512printing every `graph-store` update it receives, it will instead just print the

513first one it receives and close the subscription. Additionally, it sets a five

514second timeout, and prints an error message if it times out.

515

516```

517<html>

518 <head>

519 <script src="https://unpkg.com/@urbit/http-api"></script>

520 <script src="/session.js"></script>

521 </head>

522 <body>

523 <button type="button" onClick="doSub()" >Subscribe Once</button>

524 <pre id="event"></pre>

525 </body>

526 <script>

527 const api = new UrbitHttpApi.Urbit("");

528 api.ship = window.ship;

529 const event = document.getElementById("event");

530 function doSub() {

531 event.innerHTML = "";

532 api.subscribeOnce("graph-store","/updates",5000)

533 .then(printEvent, noEvent);

534 };

535 function printEvent(update) {

536 event.innerHTML = JSON.stringify(update, null, 2);

537 };

538 function noEvent(error) {

539 event.innerHTML = error;

540 };

541 </script>

542</html>

543```

544

545### Scries

546

547To scry agents on the ship, the `Urbit` class in `http-api` includes a `scry` function.

548The `scry` function takes two arguments in a object:

549

551| -------- | -------- | ---------------------------------- | --------------- |

554

555The `scry` function returns a promise that, if successful, contains the

556requested data as JSON.

557

558#### Example

559

560Upon pressing the "Scry Graphs" button, this example will scry the `graph-store`

561agent's `/keys` endpoint for the list of graphs, and print the resulting JSON

562data.

563

564```

565<html>

566 <head>

567 <script src="https://unpkg.com/@urbit/http-api"></script>

568 <script src="/session.js"></script>

569 </head>

570 <body>

571 <button id="scry" type="button" onClick="doScry()" >Scry Graphs</button>

572 <pre id="result"></pre>

573 </body>

574 <script>

575 const api = new UrbitHttpApi.Urbit("");

576 api.ship = window.ship;

577 async function doScry() {

578 var groups = await api.scry({app: "graph-store", path: "/keys"});

579 document.getElementById("result")

580 .innerHTML = JSON.stringify(groups, null, 2);

581 }

582 </script>

583</html>

584```

585

586### Thread

587

588To run a thread, the `Urbit` class in `http-api` includes a `thread` function.

589The `thread` function takes five arguments in an object:

590

592| ------------ | -------- | ------------------------------------------------------------------------------------------------------------- | ----------------------- |

596| `body` | any JSON | The data to give to the thread as its argument. | `{foo: "bar", baz: 42}` |

598

599The `thread` function will produce a promise that, if successful, contains the

600JSON result of the thread.

601

602#### Example

603

604This example takes a hoon expression (such as `(add 1 1)`), evalutes it with the

605`graph-eval` thread in the `landscape` desk, and prints the result.

606

607```

608<html>

609 <head>

610 <script src="https://unpkg.com/@urbit/http-api"></script>

611 <script src="/session.js"></script>

612 </head>

613 <body>

614 <input id="hoon" type="text" placeholder="Hoon to evaluate" />

615 <button id="submit" type="button" onClick="runThread()" >Submit</button>

616 <pre id="expr"></pre>

617 <pre id="result"></pre>

618 </body>

619 <script>

620 document.getElementById("hoon")

621 .addEventListener("keyup", function(event) {

622 if (event.keyCode === 13) {

623 document.getElementById("submit").click();

624 }

625 });

626 const hoon = document.getElementById("hoon");

627 const expr = document.getElementById("expr");

628 const result = document.getElementById("result");

629 const desk = "landscape";

630 const api = new UrbitHttpApi.Urbit("", "", desk);

631 api.ship = window.ship;

632 async function runThread() {

633 const threadResult = await api.thread({

634 inputMark: "graph-view-action",

635 outputMark: "tang",

636 threadName: "graph-eval",

637 body: {eval: hoon.value}

638 });

639 result.innerHTML = threadResult[0].join("\n");

640 expr.innerHTML = hoon.value;

641 hoon.value = "";

642 };

643 </script>

644</html>

645```

646

647### Delete Channel

648

649Rather than just closing individual subscriptions, the entire channel can be

650closed with the `delete` function in the `Urbit` class of `http-api`. The

651function takes no arguments, and can be called like `api.cancel()` (replacing

652`api` with whatever you've named the `Urbit` object you previously instantiated).

653When a channel is closed, all subscriptions will be cancelled and all pending

654updates will be discarded.

655

656### Reset

657

658An existing instance of `http-api`'s `Urbit` class can be reset with its

659`reset` function. This function takes no arguments, and can be called like

660`api.reset()` (replacing `api` with whatever you've named the `Urbit` object you

661previously instantiated). When called, the channel will be closed and all

662subscriptions cancelled. Additionally, all outstanding outbound pokes will be

663discarded and a fresh channel ID will be generated.

664

665## Further reading

666

667- [Eyre External API Reference][eyre-ext-ref] - Lower-level documentation of

668 Eyre's external API.

669- [Eyre Guide][eyre-guide] - Lower-level examples of using Eyre's external API

670 with `curl`.

671- [`http-api` on Github][http-api-src] - The source code for `@urbit/http-api`.

672

673[eyre-ext-ref]: /docs/arvo/eyre/external-api-ref

674[eyre-guide]: /docs/arvo/eyre/guide

675[http-api-src]: https://github.com/urbit/urbit/tree/master/pkg/npm/http-api

676[eyre]: /docs/glossary/eyre

677[vane]: /docs/glossary/vane

678[arvo]: /docs/glossary/arvo

679[hoon]: /docs/glossary/hoon

680[gall]: /docs/glossary/gall

681[clay]: /docs/glossary/clay

682[desk]: /docs/glossary/desk

683[mark]: /docs/glossary/mark

684[sse]: https://html.spec.whatwg.org/#server-sent-events

685[@urbit/http-api]: https://github.com/urbit/urbit/tree/master/pkg/npm/http-api