@@ -6,550 +6,59 @@ title: Shell
7# `:dojo` (Shell)
7# `:dojo` (Shell)
9−The dojo is a typed functional shell. Assuming our default
9+The dojo is a typed functional shell.
10−plot `~fintud-macrep`,
12−Its prompt is:
11+(Goals of a REPL+IFTTT, hood)
16## Quickstart
13## Quickstart
18−To print a Hoon expression or other recipe:
15+Evaluate a hoon expression:
20− ~fintud-macrep:dojo> (add 2 2)
22−To save a recipe as a variable `foo`:
24− ~fintud-macrep:dojo> =foo (add 2 2)
26−To save as a unix file (`$pier/.urb/put/foo/bar.baz`):
28− ~fintud-macrep:dojo> .foo/bar/baz (add 2 2)
30−To save as an urbit file (`/===/foo/bar/baz`):
32− ~fintud-macrep:dojo> *foo/bar/baz (add 2 2)
34−A noun generator with ordered and named arguments:
36− ~fintud-macrep:dojo> +make one two three, =foo (add 2 2), =bar 42
38−A poke message to an urbit daemon:
40− ~fintud-macrep:dojo> :~fintud-macrep/talk (add 2 2)
42−A system command to `:hood`:
44− ~fintud-macrep:dojo> |reload %vane
48−An Urbit value is called a "noun." A noun is either an unsigned
49−integer ("atom") or an ordered pair of nouns ("cell"). Nouns
50−are just values, with no cyclic structure or pointer identity.
52−The dojo is your safe space for hand-to-hand combat with nouns.
53−Every dojo command builds a "product" noun functionally, then
54−applies this product in a side effect -- show, save, or send.
58−In the quickstart we learned a crude interpretation of the dojo
59−in terms of "expressions, generators and operations." While
60−nothing in the quickstart section is inaccurate, it's not the way
61−the system works internally.
63−*All* dojo lines are commands. An operation uses a *recipe*
64−to create a noun, which the command uses in its side effect.
65−Just printing the noun is a trivial case of a command.
67−Recipes are *conceptually* functional, but often use concrete,
68−stateful action sequences. A simple Hoon expression (*twig*) is
69−purely functional, but it's only one kind of recipe.
71−A recipe can get data from an HTTP GET request or an interactive
72−input dialog. It can also query, even block on, the Urbit
73−namespace. These operations are *conceptually* functional, but
74−not *actually* functional. They don't belong in a pure Hoon
75−expression, but they do belong in a dojo recipe. A recipe is "FP
76−in the large," more like Unix pipes than Haskell monads.
78−The dojo is "single-threaded" in each session. One session can
79−work on one command at a time. The session does not accept user
80−input while processing a command, even when it blocks over the
81−network. And each session's state is independent. (If you want
82−to work on two things at a time, connect two console sessions to
85−Once you've built the product of your recipe, you show, save,
88−You can pretty-print the product to the console. You can save it
89−-- as a dojo variable, as a revision to the Urbit filesystem, or
90−as an export to a file in the Unix filesystem. Or you can
91−send it -- staying native with an Urbit poke, or going retro
92−with an HTTP PUT/POST.
94−All these operations are typed. Hoon is a statically typed
95−language, but the dojo is a dynamic interpreter. The nouns you
96−build in the dojo are dynamically typed nouns, or "cages".
98−A cage actually has two layers of type: "mark," a network label
99−(like a MIME type), and "span," a Hoon language type. When a
100−cage is sent across the Urbit network, the receiving daemon
101−validates the noun against its own version of the mark, and
102−regenerates the span.
104−Of course, sometimes a recipe produces a noun with mark `%noun`,
105−meaning "any noun," and span `*`, the set of all nouns. We have
106−no choice but to do the best we can with mystery nouns, but we
107−prefer a formal description.
109−A mark is also called a "format." Marks let us perform a variety
110−of formal typed operations on nouns: validation, translation,
111−even patch and diff for revision control.
115−An excellent way to understand `:dojo` is to read the source,
116−which is in `/===/app/dojo/hoon` (or `home/app/dojo.hoon` if your `home` desk is mounted to Unix).
118−You may or may not know Hoon. We'll use some Hoon
119−snippets here for defining structures and grammars. Just think
120−of it as pseudocode -- the meaning should be clear from context.
122−### Syntax and semantics
124−To use the dojo, type a complete command at the dojo prompt.
125−The simplest command just prints a Hoon expression:
127 ~fintud-macrep:dojo> (add 2 2)
17 ~fintud-macrep:dojo> (add 2 2)
129−Hit return. You'll see:
133− ~fintud-macrep:dojo>
135−Similarly in tall form,
137 ~fintud-macrep:dojo> %+ add 2 2
18 ~fintud-macrep:dojo> %+ add 2 2
140− ~fintud-macrep:dojo>
142−An incomplete command goes into a multiline input buffer. Use
143−the up-arrow (see the console history section) to get the last
144−command back, edit it so it's just `%+ add 2`, and press return.
148− ~fintud-macrep/dojo<
150−Enter `2`. You'll see:
152− > %+ add 2
20+Tall form Hoon will be prompted for:
155− ~fintud-macrep/dojo>
157−The full command that parses and runs is the concatenation of all
158−the partial lines, with a space inserted between them. To clear
159−all multiline input, just hit return on an empty prompt.
161−### Command structure
163−Every finished line is parsed into one `++dojo-command`:
166− $% [%edit p=path q=dojo-recipe] :: modify clay file
167− [%http p=? q=purl r=dojo-recipe] :: http post or put
168− [%poke p=goal q=dojo-recipe] :: send request
169− [%save p=path q=dojo-recipe] :: replace clay file
170− [%show p=dojo-recipe] :: print to console
171− [%unix p=path q=dojo-recipe] :: export to unix
172− [%verb p=term q=dojo-recipe] :: store variable
175−Each kind of `++dojo-command` is an action that depends on one
176−noun production, a `++dojo-recipe`. We describe first the
177−commands, then the recipes.
179−##### `[%show p=dojo-recipe]`
181−To print the product, the command is just the recipe:
183− ~fintud-macrep:dojo> (add 2 2)
185−##### `[%verb p=term q=dojo-recipe]`
22+ ~fintud-macrep:dojo> %+ add
23+ ~fintud-macrep:dojo< 2
24+ ~fintud-macrep:dojo< 2
187−To save the product to a variable `foo`:
26+Use `=var` to save a shell variable:
189 ~fintud-macrep:dojo> =foo (add 2 2)
28 ~fintud-macrep:dojo> =foo (add 2 2)
191−`foo` goes into your Hoon subject (scope) and is available to all
30+Generators (files in `/gen`) are run with `+`:
196− ~fintud-macrep:dojo> =foo
198−The dojo has a set of special variables, some read-write and some
208−The read-write specials are `dir`, `lib` and `arc`. `dir` is the
209−current working beak (`%clay` branch and revision) and directory. The
210−current beak is often the invalid "version 0", which is shorthand for
211−the most recent revision. If you need a valid `%clay` path, say, as an
212−argument to `+ls`, use the special read-only variable `%`, which
213−automatically expands this to the current time. If `dir` is unset,
214−then the current working path is defaulted to the current revision of
215−the `home` desk on your urbit.
217−`lib` is a set of libraries, and `arc` a set of structures, to put in
218−the Hoon subject. Add libraries with the `%ford` rune `/+`, and
219−structures with `/-`. Libraries will be loaded from the `lib`
220−directory in the current beak, and structures from the `sur`
223−Read-only specials are `now`, the current (128-bit `@da`) time, `our`,
224−the current urbit, and `eny`, 256 bits of fresh entropy.
226−##### `[%edit p=path q=dojo-recipe]`
227−##### `[%save p=path q=dojo-recipe]`
229−The product is either a new version of, or a modification to,
230−the Urbit file at the given path. (See the discussion of Urbit
235− ~fintud-macrep:dojo> *%/numbers/four (add 2 2)
239− ~fintud-macrep:dojo> -%/numbers/four (add 2 2)
241−A save (`*`) overwrites the current (if any) version of the file
242−with a new version of any mark. The save command above will work
243−(if you want `/numbers/four` at your current path).
245−An edit (`-`) applies a diff whose mark has to match the diff
246−mark for the current version of the file. The edit command above
247−will not work, because evaluating a Hoon expression like `(add 2
248−2)` just produces a `%noun` mark, ie, an arbitrary noun.
250−For either saves or edits, the current version of the file must
251−be the same version specified in the write -- in other words,
252−we can only write to HEAD. If someone else has sneaked in a
253−change since the version specified, the command will fail.
255−##### `[%unix p=path q=dojo-recipe]`
257− ~fintud-macrep:dojo> ./numbers/four (add 2 2)
259−The product is saved as a Unix file (its mark is translated
32+ ~fintud-macrep:dojo> +hello 'world'
260−to MIME, and the MIME type is mapped as the extension).
262−##### `[%poke p=goal q=dojo-recipe]`
34+Save output to a file in `%clay` with `*`:
264−A poke or *order* is a one-way transactional request. It either
36+ ~fintud-macrep:dojo> *foo/bar/hoon 'hello world'
265−succeeds and returns no information, or fails and produces an
268−Every order is sent to one daemon on one urbit. The default
38+A system command to `:hood` using `|`:
269−urbit is your urbit. The default daemon is the system daemon,
270−`:hood`. The following syntactic forms are equivalent:
272− ~fintud-macrep:dojo> :~fintud-macrep/hood (add 2 2)
40+ ~fintud-macrep:dojo> |reload %eyre
273− ~fintud-macrep:dojo> :hood (add 2 2)
274− ~fintud-macrep:dojo> :~fintud-macrep (add 2 2)
275− ~fintud-macrep:dojo> : (add 2 2)
277−Urbit pokes do not have a separate verb. The mark of the message
42+# dir
278−defines the semantics of the operation. You don't call a method
279−`foo` whose argument is a noun in mark `bar` -- you poke a noun
280−in mark `bar`. The mark is the protocol is the method.
282−If the order succeeds, you'll see an `>=` line. If not, you'll
44+## Manual
283−see an error report, typically with a stack trace.
285−It's common (but not necessary) to use a custom generator for the
286−daemon you're talking to. (For generators, see below.) Hence
288− ~fintud-macrep:dojo> :~fintud-macrep/fish +fish/catch (add 2 2)
290−It's irritating to type "fish" twice, just because we're using a
291−fish generator to talk to a fish daemon. Hence a shortcut:
293− ~fintud-macrep:dojo> :~fintud-macrep/fish|catch (add 2 2)
295−If we combine all these defaults, we get the "system command"
298− ~fintud-macrep:dojo> :~fintud-macrep/hood +hood/reload %ames
299− ~fintud-macrep:dojo> |reload %ames
301−This is the most common poke, a generated message to your own
304−##### `[%http p=? q=purl r=dojo-recipe]`
306−The Web has its own poke, unfortunately in two flavors. To POST,
308− ~fintud-macrep:dojo> +http://website.com (add 2 2)
312− ~fintud-macrep:dojo> -http://website.com (add 2 2)
314−As with a poke, you'll get a >= for success, or an error report.
316−#### Recipes, models and filters
318−But wait, what's a recipe? Simplifying the actual code slightly:
320− ++ dojo-recipe :: functional build
321− $% [%ex p=twig] :: hoon expression
322− [%as p=mark q=dojo-recipe] :: format conversion
323− [%do p=twig q=dojo-recipe] :: apply gate
324− [%ge p=dojo-script] :: generator
325− [%ur p=purl] :: get url
326− [%tu p=(list dojo-recipe)] :: tuple
328− ++ dojo-script :: script
329− $: p=path :: core recipe
330− q=dojo-config :: configuration
332− ++ dojo-config :: configuration
333− $: p=(list dojo-recipe) :: by order
334− q=(map term (unit dojo-recipe)) :: by keyword
339−The twig in an `%ex` recipe is a Hoon expression. The recipe
340−syntax is just the Hoon syntax.
342−The subject of the twig is a core stack: first the Hoon kernel,
343−then the Arvo standard library, then the structures and libraries
344−in `lib` and `arc`. On the very top are the dojo variables.
346−A twig produces the trivial mark `%noun`, except in two cases
347−where the dojo can do better. The dojo analyzes the twig to
348−detect two trivial cases where direct evaluation gives us a mark:
349−a variable reference like `foo` that matches a dojo variable, or
350−an urbitspace dereference like `.^(/cx/~fintud-macrep/main/1/foo)`.
351−In either case, if we executed these through Hoon, we'd get the
352−same noun with the same span.
354−##### `[%tu p=(list dojo-recipe)]`
356−A is just a tuple of recipes, using the normal Hoon syntax for
357−a tuple. `[a]` is `a`, `[a b]` the cell `[a b]`, `[a b c]` the
360−A tuple, unless it's a trivial 1-tuple, is always marked `%noun`.
362−##### `[%ge p=dojo-script]`
364−A `%ge` is a generator, a configurable script loaded from the
367−The script recipe `++dojo-script` specifies a script path, a list
368−of ordered arguments, and a list of keyword arguments. All the
369−arguments are recipes. The path specifies a Hoon source file in
372−For the path `/fun/make`, the ordered arguments `1`, `2` and `3`,
373−and the named arguments `foo` and `bar`, the syntax is:
375− ~fintud-macrep:dojo> +fun/make 1 2 3, =foo (add 2 2), =bar 42
377−Unless this non-closed form is the end of a command, it needs to
378−be surrounded by `[]` to make it play well with others.
380−Generator programming is covered in the dojo developer's guide.
381−The user doesn't need to know or notice how the generator gets
382−its input (if any), except in one case: a dialog.
384−A dialog generator will take over the prompt and ask you
385−questions. If this seems terrifying, ^D will abort the dialog,
386−the recipe, and the command, and take you back to the dojo.
388−##### `[%as p=mark q=dojo-recipe]`
390−`%as` is a mark conversion. Since the input to it is another
391−recipe, we can chain them to make a conversion pipeline.
393−To convert a recipe, just precede it with the converison form, `&mark`:
395− ~fintud-macrep:dojo> &noun (add 2 2)
396− ~fintud-macrep:dojo> &md (add 50 7)
398−##### `[%do p=twig q=dojo-recipe]`
400−`%do` is a Hoon functino (gate) application. It can also be in a pipeline.
402−Its syntax is a hoon expression preceeded by `_`:
404− ~fintud-macrep:dojo> _lore 'hello\0aworld'
405− ~fintud-macrep:dojo> _|=(a=@ (mul 3 a))} (add 2 2)
409−A simple HTTP get produces the result as a `%httr` noun.
413−Developing dojo generators is the easiest form of Hoon programming.
414−Generator scripts are found in the `gen` folder.
418−All generator scripts are configured with the same configuration gate:
420− |= $: [now=@da eny=@ bec=beak]
421− [arg=my-arguments opt=my-options]
424−We try to minimize boilerplate, but you can't get out of this
425−one. The dojo will slam this configuration gate to create your
428−The head of the sample is a system context. `now` is the date of
429−the call; `eny` is 256 bits of entropy; `bec` is a triple
430−`[p=ship q=desk r=case]` (ie, the root of a filesystem path).
431−This beak is the path to the script, not the current path within
432−the dojo (dojo variables are not, unlike in Unix, visible to
435−`arg` and `opt` are whatever you want them to be. (Use `~` if
436−you have no arguments or options.) The dojo will replace `arg`
437−with the user's ordered arguments, and replace any options in
438−`opt` specified by the user's named arguments. (More exactly,
439−if the user specifies `=foo 42`, your `opt` is replaced with
442−Bear in mind that dojo syntax is list-centric, so your `arg` will
443−always end with a `~`. For instance,
445− ~fintud-macrep/dojo> +fun/make 1 2 3
447−will generate an `arg` of `[1 2 3 ~]`. Yes, this is the only
448−place in Urbit where we do list-centric arguments.
450−Note also that script configuration is typed. The user's command
451−will fail if there's a type mismatch. But `arg` does not have to
452−be a homogeneous list -- just a tuple with `~` on the end. Also,
453−you can use `arg=*` and sort out the nouns by hand. Any value
454−you don't care about can simply be `*`.
458−There are four kinds of generators: builders (with no special
459−I/O), dialogs (which read console input), scrapers (which pull
460−data from the webs), and synthesizers (which produce another
461−generator). Any generator can use `.^` to both read from and
462−block (wait for remote or delayed results) on the Urbit global
465−A generator produces a cell whose tail is the configuration gate,
466−and whose head is `%say` for a builder, `%ask` for a dialog,
467−`%get` for a scraper, and `%con` for a constructor.
471−A builder just produces a cask (mark-value cell) directly from
472−the configuration gate. Here's the simplest builder, with a
481−A dialog is a console input form. We recommend using the helpful
482−`sole` structures, with
486−(If you're interested in building your own dialogs without `sole`
487−(not that complicated at all), it's easiest to start by
488−reverse-engineering `sole`.)
490−Otherwise, a dialog boilerplate (with blank configuration), which
491−generates a `my-result` result with mark `%my-result-mark`:
494− ^- (sole-result (cask my-result))
495− %+ sole-so %my-result-mark
498−Internally, a `++sole-result` is either a final result or a new
499−dialog core with a new prompt. The dojo keeps running the dialog
500−until it produces a final result.
502−A dialog step can do one of three things: print a message, prompt
503−for a value, or finish with a result. These are done with
504−`sole-yo`, `sole-lo`, and `sole-so` respectively. Here's a
505−simple dialog which uses all of them:
508− ^- (sole-result (cask ,@ud))
509− %+ sole-yo leaf/"let's multiply two numbers..."
510− %+ sole-lo [%& %number "number one: "]
513− %+ sole-lo [%& %number "number two: "]
519−`++sole-yo` prints a tank (prettyprint structure). See `++tank`
522−`++sole-lo` takes a prompt and a new dialog. In the example,
46+# special vars
523−`[%& %number "your number: "]` is a `++sole-prompt`. `&` as
524−opposed to `|` means input is echoed (not a password).
525−`%number` is a history label; all inputs with the same label
526−share the same history buffer.
528−The `++sole-dialog` is generally built with `++sole-go`, as used
48+# command prefixes
529−above. This takes a parsing `++rule` (here `dim:ag`, which
530−parses a decimal), and a gate whose sample is the parsed value,
531−producing a new dialog.
533−#### Scrapers
50+### `:` send to app
535−Most stuff on the internets is crap, but there's exceptions.
52+Send a poke message to `:square` as an `atom` mark:
536−Sometimes it's nice to get it and compute functions on it.
538−A scraper is much like a dialog, except instead of `sole-lo` and
54+ ~fintud-macrep:dojo> :~fintud-macrep/square &atom (add 2 2)
539−`sole-go` it uses `sole-at`.
541− :- %get |= *
56+### `.` download to Unix
542− %+ sole-yo leaf/"Fetching example.com"
543− %+ sole-at [[& ~ `/com/example] `/ ~]
545− %+ sole-yo leaf/"Fetched."
549−`++sole-at` takes a `purl` request url, and a gate through
58+Download a noun to Unix with `.`:
550−which to slam the result `httr`.
552−#### Synthesizer
60+ ~fintud-macrep:dojo> .foo/bar/baz (add 2 2)
554−A synthesizer simply produces another recipe. Its
62+# generators