@@ -6,550 +6,59 @@ title: Shell
66
77# `:dojo` (Shell)
88
9−The dojo is a typed functional shell. Assuming our default
10−plot `~fintud-macrep`,
9+The dojo is a typed functional shell.
1110
12−Its prompt is:
13−
14− ~fintud-macrep:dojo>
11+(Goals of a REPL+IFTTT, hood)
1512
1613## Quickstart
1714
18−To print a Hoon expression or other recipe:
19−
20− ~fintud-macrep:dojo> (add 2 2)
21−
22−To save a recipe as a variable `foo`:
23−
24− ~fintud-macrep:dojo> =foo (add 2 2)
25−
26−To save as a unix file (`$pier/.urb/put/foo/bar.baz`):
27−
28− ~fintud-macrep:dojo> .foo/bar/baz (add 2 2)
29−
30−To save as an urbit file (`/===/foo/bar/baz`):
31−
32− ~fintud-macrep:dojo> *foo/bar/baz (add 2 2)
33−
34−A noun generator with ordered and named arguments:
35−
36− ~fintud-macrep:dojo> +make one two three, =foo (add 2 2), =bar 42
37−
38−A poke message to an urbit daemon:
39−
40− ~fintud-macrep:dojo> :~fintud-macrep/talk (add 2 2)
41−
42−A system command to `:hood`:
43−
44− ~fintud-macrep:dojo> |reload %vane
45−
46−## Manual
47−
48−An Urbit value is called a "noun." A noun is either an unsigned
49−integer ("atom") or an ordered pair of nouns ("cell"). Nouns
50−are just values, with no cyclic structure or pointer identity.
51−
52−The dojo is your safe space for hand-to-hand combat with nouns.
53−Every dojo command builds a "product" noun functionally, then
54−applies this product in a side effect -- show, save, or send.
55−
56−### Theory
57−
58−In the quickstart we learned a crude interpretation of the dojo
59−in terms of "expressions, generators and operations." While
60−nothing in the quickstart section is inaccurate, it's not the way
61−the system works internally.
62−
63−*All* dojo lines are commands. An operation uses a *recipe*
64−to create a noun, which the command uses in its side effect.
65−Just printing the noun is a trivial case of a command.
66−
67−Recipes are *conceptually* functional, but often use concrete,
68−stateful action sequences. A simple Hoon expression (*twig*) is
69−purely functional, but it's only one kind of recipe.
70−
71−A recipe can get data from an HTTP GET request or an interactive
72−input dialog. It can also query, even block on, the Urbit
73−namespace. These operations are *conceptually* functional, but
74−not *actually* functional. They don't belong in a pure Hoon
75−expression, but they do belong in a dojo recipe. A recipe is "FP
76−in the large," more like Unix pipes than Haskell monads.
77−
78−The dojo is "single-threaded" in each session. One session can
79−work on one command at a time. The session does not accept user
80−input while processing a command, even when it blocks over the
81−network. And each session's state is independent. (If you want
82−to work on two things at a time, connect two console sessions to
83−your dojo.)
84−
85−Once you've built the product of your recipe, you show, save,
86−or send it.
87−
88−You can pretty-print the product to the console. You can save it
89−-- as a dojo variable, as a revision to the Urbit filesystem, or
90−as an export to a file in the Unix filesystem. Or you can
91−send it -- staying native with an Urbit poke, or going retro
92−with an HTTP PUT/POST.
93−
94−All these operations are typed. Hoon is a statically typed
95−language, but the dojo is a dynamic interpreter. The nouns you
96−build in the dojo are dynamically typed nouns, or "cages".
97−
98−A cage actually has two layers of type: "mark," a network label
99−(like a MIME type), and "span," a Hoon language type. When a
100−cage is sent across the Urbit network, the receiving daemon
101−validates the noun against its own version of the mark, and
102−regenerates the span.
103−
104−Of course, sometimes a recipe produces a noun with mark `%noun`,
105−meaning "any noun," and span `*`, the set of all nouns. We have
106−no choice but to do the best we can with mystery nouns, but we
107−prefer a formal description.
108−
109−A mark is also called a "format." Marks let us perform a variety
110−of formal typed operations on nouns: validation, translation,
111−even patch and diff for revision control.
112−
113−### Other resources
114−
115−An excellent way to understand `:dojo` is to read the source,
116−which is in `/===/app/dojo/hoon` (or `home/app/dojo.hoon` if your `home` desk is mounted to Unix).
117−
118−You may or may not know Hoon. We'll use some Hoon
119−snippets here for defining structures and grammars. Just think
120−of it as pseudocode -- the meaning should be clear from context.
121−
122−### Syntax and semantics
123−
124−To use the dojo, type a complete command at the dojo prompt.
125−The simplest command just prints a Hoon expression:
15+Evaluate a hoon expression:
12616
12717 ~fintud-macrep:dojo> (add 2 2)
128−
129−Hit return. You'll see:
130−
131− > (add 2 2)
132− 4
133− ~fintud-macrep:dojo>
134−
135−Similarly in tall form,
136−
13718 ~fintud-macrep:dojo> %+ add 2 2
138− > %+ add 2 2
139− 4
140− ~fintud-macrep:dojo>
141−
142−An incomplete command goes into a multiline input buffer. Use
143−the up-arrow (see the console history section) to get the last
144−command back, edit it so it's just `%+ add 2`, and press return.
145−You'll see:
146−
147− > %+ add 2
148− ~fintud-macrep/dojo<
149−
150−Enter `2`. You'll see:
15119
152− > %+ add 2
153− 2
154− 4
155− ~fintud-macrep/dojo>
156−
157−The full command that parses and runs is the concatenation of all
158−the partial lines, with a space inserted between them. To clear
159−all multiline input, just hit return on an empty prompt.
160−
161−### Command structure
162−
163−Every finished line is parsed into one `++dojo-command`:
164−
165− ++ dojo-command ::
166− $% [%edit p=path q=dojo-recipe] :: modify clay file
167− [%http p=? q=purl r=dojo-recipe] :: http post or put
168− [%poke p=goal q=dojo-recipe] :: send request
169− [%save p=path q=dojo-recipe] :: replace clay file
170− [%show p=dojo-recipe] :: print to console
171− [%unix p=path q=dojo-recipe] :: export to unix
172− [%verb p=term q=dojo-recipe] :: store variable
173− == ::
174−
175−Each kind of `++dojo-command` is an action that depends on one
176−noun production, a `++dojo-recipe`. We describe first the
177−commands, then the recipes.
178−
179−##### `[%show p=dojo-recipe]`
180−
181−To print the product, the command is just the recipe:
182−
183− ~fintud-macrep:dojo> (add 2 2)
20+Tall form Hoon will be prompted for:
18421
185−##### `[%verb p=term q=dojo-recipe]`
22+ ~fintud-macrep:dojo> %+ add
23+ ~fintud-macrep:dojo< 2
24+ ~fintud-macrep:dojo< 2
18625
187−To save the product to a variable `foo`:
26+Use `=var` to save a shell variable:
18827
18928 ~fintud-macrep:dojo> =foo (add 2 2)
19029
191−`foo` goes into your Hoon subject (scope) and is available to all
192−expressions.
193−
194−To unbind `foo`:
195−
196− ~fintud-macrep:dojo> =foo
197−
198−The dojo has a set of special variables, some read-write and some
199−read-only:
200−
201− dir
202− lib
203− arc
204− now
205− our
206− eny
207−
208−The read-write specials are `dir`, `lib` and `arc`. `dir` is the
209−current working beak (`%clay` branch and revision) and directory. The
210−current beak is often the invalid "version 0", which is shorthand for
211−the most recent revision. If you need a valid `%clay` path, say, as an
212−argument to `+ls`, use the special read-only variable `%`, which
213−automatically expands this to the current time. If `dir` is unset,
214−then the current working path is defaulted to the current revision of
215−the `home` desk on your urbit.
216−
217−`lib` is a set of libraries, and `arc` a set of structures, to put in
218−the Hoon subject. Add libraries with the `%ford` rune `/+`, and
219−structures with `/-`. Libraries will be loaded from the `lib`
220−directory in the current beak, and structures from the `sur`
221−directory.
222−
223−Read-only specials are `now`, the current (128-bit `@da`) time, `our`,
224−the current urbit, and `eny`, 256 bits of fresh entropy.
225−
226−##### `[%edit p=path q=dojo-recipe]`
227−##### `[%save p=path q=dojo-recipe]`
228−
229−The product is either a new version of, or a modification to,
230−the Urbit file at the given path. (See the discussion of Urbit
231−filesystem paths.)
232−
233−To save:
234−
235− ~fintud-macrep:dojo> *%/numbers/four (add 2 2)
236−
237−To edit:
238−
239− ~fintud-macrep:dojo> -%/numbers/four (add 2 2)
240−
241−A save (`*`) overwrites the current (if any) version of the file
242−with a new version of any mark. The save command above will work
243−(if you want `/numbers/four` at your current path).
244−
245−An edit (`-`) applies a diff whose mark has to match the diff
246−mark for the current version of the file. The edit command above
247−will not work, because evaluating a Hoon expression like `(add 2
248−2)` just produces a `%noun` mark, ie, an arbitrary noun.
249−
250−For either saves or edits, the current version of the file must
251−be the same version specified in the write -- in other words,
252−we can only write to HEAD. If someone else has sneaked in a
253−change since the version specified, the command will fail.
254−
255−##### `[%unix p=path q=dojo-recipe]`
256−
257− ~fintud-macrep:dojo> ./numbers/four (add 2 2)
30+Generators (files in `/gen`) are run with `+`:
25831
259−The product is saved as a Unix file (its mark is translated
260−to MIME, and the MIME type is mapped as the extension).
32+ ~fintud-macrep:dojo> +hello 'world'
26133
262−##### `[%poke p=goal q=dojo-recipe]`
34+Save output to a file in `%clay` with `*`:
26335
264−A poke or *order* is a one-way transactional request. It either
265−succeeds and returns no information, or fails and produces an
266−error dump.
36+ ~fintud-macrep:dojo> *foo/bar/hoon 'hello world'
26737
268−Every order is sent to one daemon on one urbit. The default
269−urbit is your urbit. The default daemon is the system daemon,
270−`:hood`. The following syntactic forms are equivalent:
38+A system command to `:hood` using `|`:
27139
272− ~fintud-macrep:dojo> :~fintud-macrep/hood (add 2 2)
273− ~fintud-macrep:dojo> :hood (add 2 2)
274− ~fintud-macrep:dojo> :~fintud-macrep (add 2 2)
275− ~fintud-macrep:dojo> : (add 2 2)
40+ ~fintud-macrep:dojo> |reload %eyre
27641
277−Urbit pokes do not have a separate verb. The mark of the message
278−defines the semantics of the operation. You don't call a method
279−`foo` whose argument is a noun in mark `bar` -- you poke a noun
280−in mark `bar`. The mark is the protocol is the method.
42+# dir
28143
282−If the order succeeds, you'll see an `>=` line. If not, you'll
283−see an error report, typically with a stack trace.
284−
285−It's common (but not necessary) to use a custom generator for the
286−daemon you're talking to. (For generators, see below.) Hence
287−
288− ~fintud-macrep:dojo> :~fintud-macrep/fish +fish/catch (add 2 2)
289−
290−It's irritating to type "fish" twice, just because we're using a
291−fish generator to talk to a fish daemon. Hence a shortcut:
292−
293− ~fintud-macrep:dojo> :~fintud-macrep/fish|catch (add 2 2)
294−
295−If we combine all these defaults, we get the "system command"
296−shortcut:
297−
298− ~fintud-macrep:dojo> :~fintud-macrep/hood +hood/reload %ames
299− ~fintud-macrep:dojo> |reload %ames
300−
301−This is the most common poke, a generated message to your own
302−hood.
303−
304−##### `[%http p=? q=purl r=dojo-recipe]`
305−
306−The Web has its own poke, unfortunately in two flavors. To POST,
307−
308− ~fintud-macrep:dojo> +http://website.com (add 2 2)
309−
310−To PUT:
311−
312− ~fintud-macrep:dojo> -http://website.com (add 2 2)
313−
314−As with a poke, you'll get a >= for success, or an error report.
315−
316−#### Recipes, models and filters
317−
318−But wait, what's a recipe? Simplifying the actual code slightly:
319−
320− ++ dojo-recipe :: functional build
321− $% [%ex p=twig] :: hoon expression
322− [%as p=mark q=dojo-recipe] :: format conversion
323− [%do p=twig q=dojo-recipe] :: apply gate
324− [%ge p=dojo-script] :: generator
325− [%ur p=purl] :: get url
326− [%tu p=(list dojo-recipe)] :: tuple
327− == ::
328− ++ dojo-script :: script
329− $: p=path :: core recipe
330− q=dojo-config :: configuration
331− == ::
332− ++ dojo-config :: configuration
333− $: p=(list dojo-recipe) :: by order
334− q=(map term (unit dojo-recipe)) :: by keyword
335− == ::
336−
337−##### `[%ex p=twig]`
338−
339−The twig in an `%ex` recipe is a Hoon expression. The recipe
340−syntax is just the Hoon syntax.
341−
342−The subject of the twig is a core stack: first the Hoon kernel,
343−then the Arvo standard library, then the structures and libraries
344−in `lib` and `arc`. On the very top are the dojo variables.
345−
346−A twig produces the trivial mark `%noun`, except in two cases
347−where the dojo can do better. The dojo analyzes the twig to
348−detect two trivial cases where direct evaluation gives us a mark:
349−a variable reference like `foo` that matches a dojo variable, or
350−an urbitspace dereference like `.^(/cx/~fintud-macrep/main/1/foo)`.
351−In either case, if we executed these through Hoon, we'd get the
352−same noun with the same span.
353−
354−##### `[%tu p=(list dojo-recipe)]`
355−
356−A is just a tuple of recipes, using the normal Hoon syntax for
357−a tuple. `[a]` is `a`, `[a b]` the cell `[a b]`, `[a b c]` the
358−cell `[a [b c]]`.
359−
360−A tuple, unless it's a trivial 1-tuple, is always marked `%noun`.
361−
362−##### `[%ge p=dojo-script]`
363−
364−A `%ge` is a generator, a configurable script loaded from the
365−filesystem.
366−
367−The script recipe `++dojo-script` specifies a script path, a list
368−of ordered arguments, and a list of keyword arguments. All the
369−arguments are recipes. The path specifies a Hoon source file in
370−`/===/gen/[path]`.
371−
372−For the path `/fun/make`, the ordered arguments `1`, `2` and `3`,
373−and the named arguments `foo` and `bar`, the syntax is:
374−
375− ~fintud-macrep:dojo> +fun/make 1 2 3, =foo (add 2 2), =bar 42
376−
377−Unless this non-closed form is the end of a command, it needs to
378−be surrounded by `[]` to make it play well with others.
379−
380−Generator programming is covered in the dojo developer's guide.
381−The user doesn't need to know or notice how the generator gets
382−its input (if any), except in one case: a dialog.
383−
384−A dialog generator will take over the prompt and ask you
385−questions. If this seems terrifying, ^D will abort the dialog,
386−the recipe, and the command, and take you back to the dojo.
387−
388−##### `[%as p=mark q=dojo-recipe]`
389−
390−`%as` is a mark conversion. Since the input to it is another
391−recipe, we can chain them to make a conversion pipeline.
392−
393−To convert a recipe, just precede it with the converison form, `&mark`:
394−
395− ~fintud-macrep:dojo> &noun (add 2 2)
396− ~fintud-macrep:dojo> &md (add 50 7)
397−
398−##### `[%do p=twig q=dojo-recipe]`
399−
400−`%do` is a Hoon functino (gate) application. It can also be in a pipeline.
401−
402−Its syntax is a hoon expression preceeded by `_`:
403−
404− ~fintud-macrep:dojo> _lore 'hello\0aworld'
405− ~fintud-macrep:dojo> _|=(a=@ (mul 3 a))} (add 2 2)
406−
407−##### `[%ur p=purl]`
408−
409−A simple HTTP get produces the result as a `%httr` noun.
410−
411−### Development
412−
413−Developing dojo generators is the easiest form of Hoon programming.
414−Generator scripts are found in the `gen` folder.
415−
416−#### Configuration
417−
418−All generator scripts are configured with the same configuration gate:
419−
420− |= $: [now=@da eny=@ bec=beak]
421− [arg=my-arguments opt=my-options]
422− ==
423−
424−We try to minimize boilerplate, but you can't get out of this
425−one. The dojo will slam this configuration gate to create your
426−generator.
427−
428−The head of the sample is a system context. `now` is the date of
429−the call; `eny` is 256 bits of entropy; `bec` is a triple
430−`[p=ship q=desk r=case]` (ie, the root of a filesystem path).
431−This beak is the path to the script, not the current path within
432−the dojo (dojo variables are not, unlike in Unix, visible to
433−generator scripts).
434−
435−`arg` and `opt` are whatever you want them to be. (Use `~` if
436−you have no arguments or options.) The dojo will replace `arg`
437−with the user's ordered arguments, and replace any options in
438−`opt` specified by the user's named arguments. (More exactly,
439−if the user specifies `=foo 42`, your `opt` is replaced with
440−`opt(foo 42)`.)
441−
442−Bear in mind that dojo syntax is list-centric, so your `arg` will
443−always end with a `~`. For instance,
444−
445− ~fintud-macrep/dojo> +fun/make 1 2 3
446−
447−will generate an `arg` of `[1 2 3 ~]`. Yes, this is the only
448−place in Urbit where we do list-centric arguments.
449−
450−Note also that script configuration is typed. The user's command
451−will fail if there's a type mismatch. But `arg` does not have to
452−be a homogeneous list -- just a tuple with `~` on the end. Also,
453−you can use `arg=*` and sort out the nouns by hand. Any value
454−you don't care about can simply be `*`.
455−
456−#### Generators
457−
458−There are four kinds of generators: builders (with no special
459−I/O), dialogs (which read console input), scrapers (which pull
460−data from the webs), and synthesizers (which produce another
461−generator). Any generator can use `.^` to both read from and
462−block (wait for remote or delayed results) on the Urbit global
463−namespace.
464−
465−A generator produces a cell whose tail is the configuration gate,
466−and whose head is `%say` for a builder, `%ask` for a dialog,
467−`%get` for a scraper, and `%con` for a constructor.
468−
469−#### Builders
470−
471−A builder just produces a cask (mark-value cell) directly from
472−the configuration gate. Here's the simplest builder, with a
473−blank configuration:
474−
475− :- %say |= *
476− :- %noun
477− "hello, world."
478−
479−#### Dialogs
480−
481−A dialog is a console input form. We recommend using the helpful
482−`sole` structures, with
483−
484− /- *sole
485−
486−(If you're interested in building your own dialogs without `sole`
487−(not that complicated at all), it's easiest to start by
488−reverse-engineering `sole`.)
489−
490−Otherwise, a dialog boilerplate (with blank configuration), which
491−generates a `my-result` result with mark `%my-result-mark`:
492−
493− :- %ask |= *
494− ^- (sole-result (cask my-result))
495− %+ sole-so %my-result-mark
496− *my-result
497−
498−Internally, a `++sole-result` is either a final result or a new
499−dialog core with a new prompt. The dojo keeps running the dialog
500−until it produces a final result.
501−
502−A dialog step can do one of three things: print a message, prompt
503−for a value, or finish with a result. These are done with
504−`sole-yo`, `sole-lo`, and `sole-so` respectively. Here's a
505−simple dialog which uses all of them:
506−
507− :- %ask |= *
508− ^- (sole-result (cask ,@ud))
509− %+ sole-yo leaf/"let's multiply two numbers..."
510− %+ sole-lo [%& %number "number one: "]
511− %+ sole-go dim:ag
512− |= one=@ud
513− %+ sole-lo [%& %number "number two: "]
514− %+ sole-go dim:ag
515− |= two=@ud
516− %+ sole-so %noun
517− (mul one two)
518−
519−`++sole-yo` prints a tank (prettyprint structure). See `++tank`
520−in hoon.hoon.
44+## Manual
52145
522−`++sole-lo` takes a prompt and a new dialog. In the example,
523−`[%& %number "your number: "]` is a `++sole-prompt`. `&` as
524−opposed to `|` means input is echoed (not a password).
525−`%number` is a history label; all inputs with the same label
526−share the same history buffer.
46+# special vars
52747
528−The `++sole-dialog` is generally built with `++sole-go`, as used
529−above. This takes a parsing `++rule` (here `dim:ag`, which
530−parses a decimal), and a gate whose sample is the parsed value,
531−producing a new dialog.
48+# command prefixes
53249
533−#### Scrapers
50+### `:` send to app
53451
535−Most stuff on the internets is crap, but there's exceptions.
536−Sometimes it's nice to get it and compute functions on it.
52+Send a poke message to `:square` as an `atom` mark:
53753
538−A scraper is much like a dialog, except instead of `sole-lo` and
539−`sole-go` it uses `sole-at`.
54+ ~fintud-macrep:dojo> :~fintud-macrep/square &atom (add 2 2)
54055
541− :- %get |= *
542− %+ sole-yo leaf/"Fetching example.com"
543− %+ sole-at [[& ~ `/com/example] `/ ~]
544− |= hit=httr
545− %+ sole-yo leaf/"Fetched."
546− %+ sole-so %httr
547− hit
56+### `.` download to Unix
54857
549−`++sole-at` takes a `purl` request url, and a gate through
550−which to slam the result `httr`.
58+Download a noun to Unix with `.`:
55159
552−#### Synthesizer
60+ ~fintud-macrep:dojo> .foo/bar/baz (add 2 2)
55361
554−A synthesizer simply produces another recipe. Its
62+# generators
55563
64+# hood