git.iko.soy/urbit.org
on 42e22ee help sign in
urbit.org/tutorials/hoon/generators.md
604 lines 21 kB history raw
1+++
2title = "1.9 Generators"
3weight = 19
4template = "doc.html"
5aliases = ["/docs/learn/hoon/hoon-tutorial/generators/"]
6+++
8Generators are the most straightforward way to write programs for Urbit. They
9are used for doing computations that do not require persistence: they take an
10input and produce an output, then disappear. Generators might make sense for
11listing directory contents, or running unit tests, or fetching the contents of a
12URL.
14There are four kinds of generators: naked, `%say`, `%get`, and `%ask`.
16### Naked Generators
18A naked generator is simply a `gate`; that is, it is an anonymous function that
19takes a `sample` (argument) and produces a [noun](/docs/glossary/noun/). All you need to do is write a
20`gate` and put it into a file in the `/gen` directory. Let's take a look at a
21very simple one:
23```hoon
24|= a=*
26```
28This generator takes one argument of any noun and produces it without any
29changes. Once you put this into a file named `echo.hoon` in the `/gen` directory, you must make your ship recognize the change by inputting `|commit %home` in the dojo. You can then run it from the dojo:
31```
32> +echo 42
3342
34```
36This command just passes in 42 and gets 42 back. But what about when we pass in
37`"asdf"`?
39```
40> +echo "asdf"
41[97 115 100 102 0]
42```
44We just get a pile of numbers back in the form of a raw `noun`. The numbers that
45compose that noun are called `atoms`, and how they are interpreted is a matter
46of what `aura` is being applied to them. For the purposes of this lesson, an
47`aura` tells the pretty-printer how to display an `atom`, but keep in mind that
48an `aura` is type metadata and does other things, too.
50In the example above, we didn't specify an `aura`, leaving the printer to fend
51for itself. `"asdf"` is a `tape`, a type that is simply a `list` of `cords`. A
52`cord` is itself an [atom](/docs/glossary/atom/) represented as a string of UTF-8 characters. When used
53as part of a `tape`, the `cord` is only a single character. So each atom in the
54`[97 115 100 102 0]` output corresponds to a component of the `tape`: 97 is
55`a`, 115 is `s`, 100 is `d`, 102 is `f`, and `0` is the "null" that every list
56is terminated with.
58We can tell the Dojo to cast -- apply a specific type to -- the output of our
59generator to see something more familiar:
61```
62> _tape +echo "asdf"
63"asdf"
64```
66Now let's create a generator with two arguments instead of one. Save the code
67below as `add.hoon` in the `/gen` directory.
69```hoon
70|= [a=@ud b=@ud]
71(add a b)
72```
74Now, run the generator:
76```
77> +add [3 4]
79```
81You may notice our generator takes a cell containing two `@ud`. This is
82actually one of the limitations of naked generators. We can only pass one
83argument. We can get around this, of course, by passing a cell, but it's less
84than ideal. Also, a naked generator cannot be called without an argument.
85This brings us to `%say` generators.
87### `%say` Generators
89We use `%say` generators when we want to provide something else in Arvo, the
90Urbit operating system, with metadata about the generator's output. This is
91useful when a generator is needed to pipe data to another program, a frequent
92occurrence.
94To that end, `%say` generators use `mark`s to make it clear, to other Arvo
95computations, exactly what kind of data their output is. A `mark` is akin
96to a MIME type on the Arvo level. A `mark` describes the data in
97some way, indicating that it's an `%atom`, or that it's a standard such as
98`%json`, or even that it's an application-specific data structure like
99`%talk-command`. `mark`s are not specific to `%say` generators; whenever data
100moves between programs in Arvo, that data is marked.
102So, more formally, a `%say` generator is a `cell`. The head of that cell is the
103`%say` tag, and the tail is a `gate` that produces a `cask` -- a pair of the
104output data and the `mark` describing that data.
106Below is an example of a `%say` generator. Save it to `add.hoon` in the `/gen`
107directory.
109```hoon
110:- %say
111|= *
112:- %noun
113(add 40 2)
114```
116Now run the generator as below:
118```
119> +add
12042
121```
123Notice that we used no argument, something that is possible with `%say`
124generators but impossible with naked generators. We'll explain that in a moment.
125For now, let's focus on the code that is necessary to make something a `%say`
126generator.
128```hoon
129:- %say
130```
132Recall that the rune `:-` produces a cell, with the first following expression
133as its head and the second following expression as its tail.
135The expression above creates a cell with `%say` as the head. The tail is
136the `|= *` expression on the line that follows.
138```hoon
139|= *
140:- %noun
141(add 40 2)
142```
144`|= *` constructs a [gate](/docs/glossary/gate/) that takes a noun. This [gate](/docs/glossary/gate/) will itself produce a
145`cask`, which is cell formed by the prepending `:-`. The head of that `cask` is
146`%noun` and the tail is the rest of the program, `(add 40 2)`. The tail of the
147`cask` will be our actual data produced by the body of the program: in this
148case, just adding 40 and 2 together.
150#### `%say` generators with arguments
152We can modify the boilerplate code to allow arguments to be passed into a `%say`
153generator, but in a way that gives us more power than we would have if we just
154used a naked generator.
156Naked generators are limited because they have no way of accessing data that
157exists in Arvo, such as the date and time or pieces of fresh entropy. In `%say`
158generators, however, we can access that kind of subject by identifying them
159in the gate's sample, which we only specified as `*` in the previous few
160examples. But we can do more with `%say` generators if we do more with that
161sample. Any valid sample will follow this 3-tuple scheme:
163`[[now, eny, beak] [list of unnamed arguments] [list of named arguments]]`
165This entire structure is a noun, which is why `*` is a valid sample if we
166wish to not use any of the information here in a generator. But let's look at
167each of these three elements, piece by piece.
169The **first part** of the above 3-tuple is a noun that is composed of three
170atoms:
172`now` is the current time.
173`eny` is 512 bits of entropy for seeding random number generators.
174`beak` contains the current ship, desk, and case.
176You can access each of those pieces of data by typing their names into the Dojo.
177But in a generator, we need to put faces (variable names) onto that data so we
178can easily use it in the program. We can do so like this:
180```hoon
181|= [[now=@da eny=@uvJ bec=beak] ~ ~]
182```
184Any of those pieces of data could be omitted by replacing part of the noun with
185`*` rather than giving them faces. For example, `[now=@da * bec=beak]` if we
186didn't want `eny`, or `[* * bec=beak]` if we only wanted `beak`.
188The **second part** of the sample is a `list` of arguments that _must_ be passed
189to the generator as it is run. Because it's a `list`, this element needs to be
190terminated with a `~`. In the example above, we used `~`, the empty list, to
191represent this second part. But that would mean we don't want to use any new
192faces for our generator. So a sample where we want to declare a single required
193argument would look like this:
195```hoon
196|= [* [n=@ud ~] ~]
197```
199In the above code, we use a `*` to ignore any of the information that would go
200in the first part of the sample, somewhat similar to how we use a `~` to say
201"nothing here" for the second or third parts of the sample.
203But to use both parts together, our gate and sample would look like this:
205```hoon
206|= [[now=@da eny=@uvJ bec=beak] [n=@ud ~] ~]
207```
209The **third part** of the sample is a list of _optional_ arguments. These
210arguments may be passed to the program as it's being run, but the program doesn't
211require them. The syntax for the third part is just like the syntax for the
212second part, besides its position:
214```hoon
215|= [* ~ [bet=@ud ~]]
216```
218Let's look at an example that uses all three parts. Save the code below in a
219file called `dice.hoon` in your `/gen` directory.
221```hoon
222:- %say
223|= [[now=@da eny=@uvJ bec=beak] [n=@ud ~] [bet=@ud ~]]
224:- %noun
225[(~(rad og eny) n) bet]
226```
228This is a very simple dice program with an optional betting functionality.
229In the code, our sample specifies faces on all of the Arvo data, meaning that we
230can easily access them. We also require the argument `[n=@ud ~]`, and allow the
231_optional_ argument `[bet=@ud ~]`.
233But there's something new: `(~(rad og eny) n)`. This code pulls the `rad` [arm](/docs/glossary/arm/)
234out of the `og` [core](/docs/glossary/core/) with the subject of `eny`. Recall that `eny` is our entropy
235value, so this is used to seed the generator. The `rad` arm will give us a
236pseudorandom number between 0 and `n`. Then we form a cell with the result and
237`bet`, the optional named argument specified previously.
239We can run this generator like so:
241```
242> +dice 6, =bet 2
243[4 2]
245> +dice 6
246[5 0]
248> +dice 6
249[2 0]
251> +dice 6, =bet 200
252[0 200]
254> +dice
255nest-fail
256```
258We get a different value from the same generator between runs, something that
259isn't possible with a naked generator. Another novelty is the ability to choose
260to not use the second argument.
262##### Arguments without a cell
264Also unlike a naked generator, we don't need to put our arguments together into
265a cell. Swap the code below into your `add.hoon` file.
267```hoon
268:- %say
269|= [* [a=@ud b=@ud ~] ~]
270:- %noun
271(add a b)
272```
274In the above code we're again creating a `%say` generator, but the sample of our
275gate is a little different than before. We are using `*` for the first part of
276the sample, because we don't use any Arvo data in this program. We are using a
277`~` for the third part, because we aren't allowing optional arguments -- the
278list of optional arguments is _empty_.
280Run a command like the one below in the Dojo. Notice that you can use two
281arguments that aren't in the cells.
283```
284> +add 40 2
28542
286```
288### `%ask` generators
290We use the `%ask` generator when we want to create an interactive program
291that prompts for inputs as it runs, rather than expecting arguments to be passed
292in at the time of initiation.
294Code-wise, two things are needed to create a useful `%ask` generator.
296The first is **`sole-result`**, which lives in `/sur/sole.hoon`. It's a
297mold-builder that's required by any part of Arvo, such as the Dojo, that wants
298to understand the output of the `%ask` generator. Because it's the only type of
299`%ask` data that is intelligible to the system, the ultimate output of an `%ask`
300generator must be `sole-result`. You can think of it as a function that takes
301molds and digests them into a type for the operating system. The simplest way to
302use `sole-result` is by calling the `produce` gate from `/lib/generators.hoon`.
304The second is **`prompt`**, a gate that lives in `lib/generators.hoon`. `prompt`
305allows you to get information back from the user, which is essential to the
306typical desired operation of an `%ask` generator.
308Since this isn't easy to digest in the abstract, we'll use an example generator
309to explain how this kind of generator works.
311##### `%ask` example
313The code below is an `%ask` generator that checks if the user inputs "blue" when
314prompted. Save it as `axe.hoon` in `/gen`.
316```hoon
317/- sole
318/+ generators
319=, [sole generators]
320:- %ask
321|= *
322^- (sole-result (cask tang))
323%+ print leaf+"What is your favorite color?"
324%+ prompt [%& %prompt "color: "]
325|= t=tape
326%+ produce %tang
327?: =(t "blue")
328 :~ leaf+"Oh. Thank you very much."
329 leaf+"Right. Off you go then."
330 ==
331:~ leaf+"Aaaaagh!"
332 leaf+"Into the Gorge of Eternal Peril with you!"
333==
334```
336Run the generator from the Dojo:
338```
339> +axe
340What is your favorite color?
341: color:
342```
344Something new happened. Instead of simply returning something, your Dojo's
345prompt changed from `~your-urbit:dojo>` to `~your-urbit:dojo: color:`, and
346now expects additional input. Let's give it an input:
348```
349: color: red
350Into the Gorge of Eternal Peril with you!
351Aaaaagh!
352```
354Let's go over what exactly is happening in this code.
356```hoon
357/- sole
358/+ generators
359=, [sole generators]
360```
362Here we bring in some of the types we are going to need from `/sur/sole` and
363gates we will use from `/lib/generators`. We use some special runes for this.
365`/-` is a Ford rune used to import types from `sur`.
367`/+` is a Ford rune used to import libraries from `lib`.
369`=,` is a rune that allows us to expose a namespace. We do this to avoid having
370to write `sole-result:sole` instead of `sole-result` or `print:generators`
371instead of `print`.
373```hoon
374:- %ask
375|= *
376```
378This code might be familiar. Just as with their `%say` cousins, `%ask`
379generators need to produce a `cell`, the head of which specifies what kind of
380generator we are running.
382With `|= *`, we create a gate and ignore the standard arguments we are given,
383because we're not using them.
385```hoon
386^- (sole-result (cask tang))
387```
389`%ask` generators need to have the second half of the cell be a gate that
390produces a `sole-result`, one that in this case contains a `cask` of `tang`.
391We use the `^-` rune to constrain the generator's output to such a
392`sole-result`.
394A `cask` is a pair of a `mark` name and a noun. Recall that a `mark` can be
395thought of as an Arvo-level MIME type for data.
397A `tang` is a `list` of `tank`, and a `tank` is a structure for printing data.
398There are three types of `tank`: `leaf`, `palm`, and `rose`. A `leaf` is for
399printing a single noun, a `rose` is for printing rows of data, and a `palm` is
400for printing backstep-indented lists.
402```hoon
403%+ print leaf+"What is your favorite color?"
404%+ prompt [%& %prompt "color: "]
405|= t=tape
406%+ produce %tang
407```
409Because we imported `generators`, we can access its contained gates, three of
410which we use in `axe.hoon`: `print`, `prompt`, and `produce`.
412**`print`** is used for printing a `tank` to the console.
414In our example, `%+` is the rune to call a gate, and our gate `print` takes one
415argument which is a `tank` to print. The `+` here is syntactic sugar for
416`[leaf "What is your favorite color?"]` that just makes it easier to write.
418**`prompt`** is used to construct a prompt for the user to provide input.
419It takes a single argument that is a tuple. Most `%ask` generators will want to
420use the `prompt` gate.
422The first element of the `prompt` sample is a flag that indicates whether what
423the user typed should be echoed out to them or hidden. `%&` will produce echoed
424output and `%|` will hide the output (for use in passwords or other secret
425text).
427The second element of the `prompt` sample is intended to be information for use
428in creating autocomplete options for the prompt. This functionality is not yet
429implemented.
431The third element of the `prompt` sample is the `tape` that we would like to
432use to prompt the user. In the case of our example, we use `"color: "`.
434**`produce`** is used to construct the output of the generator. In our example,
435we produce a `tang`.
438```hoon
439|= t=tape
440```
442Our gate here takes a `tape` that was produced by `prompt`. If we needed
443another type of data we could use `parse` to obtain it.
445The rest of this generator should be intelligible to those with Hoon knowledge
446at this point.
448One quirk that you should be aware of, though, is that `tang` prints in reverse
449order from how it is created. The reason for this is that `tang` was originally
450created to display stack trace information, which should be produced in reverse
451order. This leads to an annoyance: we either have to specify our messages
452backwards or construct them in the order we want and then `flop` the `list`.
453This is a [known issue](https://github.com/urbit/arvo/issues/840) to be resolved.
455### `%get` generators
457`%get` generators are used for making HTTP requests through `eyre`. For this
458section we'll assume knowledge of generators covered in previous sections, so
459we can jump in with an example:
461```hoon
462/- sole
463/+ generators
464=, generators
465:- %get
466|= [* [url=tape ~] ~]
467^- (sole-request:sole (cask json))
468%+ curl (scan url auri:de-purl:html)
469|= hit/httr:eyre
470?~ r.hit !!
471=/ my-json (de-json:html q.u.r.hit)
472?~ my-json !!
473=, dejs:format
474%+ produce %json
475%. %title
476%~ got by
477%- (om same)
478u.my-json
479```
481The above generator was written with the
482[Studio Ghibli API](https://ghibliapi.herokuapp.com/). It uses this API to
483print the title of a Studio Ghibli film.
485```hoon
486/- sole
487/+ generators
488=, generators
489```
491Here we import types from `/sur/sole.hoon` and gates from
492`/lib/generators.hoon`. We also use `=,` to expose the namespace of
493`generators` that we just imported.
495```hoon
496:- %get
497|= [* [url=tape ~] ~]
498```
500Like with `%say` and `%ask` generators, we must include the "boilerplate":
501code that produces a generator-specific `cell`. The head of that cell is `%get`,
502and the tail is a gate that takes a `tape` of the URL that we are trying
503to access.
505```hoon
506^- (sole-request:sole (cask json))
507```
509Here we make sure our gate is producing a `sole-request` which contains a `cask`
510of the type `json`.
513```hoon
514%+ curl (scan url auri:de-purl:html)
515|= hit/httr:eyre
516```
518Here we can see the call to `curl`, the sample of which will parse our url and
519make sure it's in the proper format.
521The second part of the `curl` call is the `gate` that will get called with the
522result provided by the actual HTTP request.
524```hoon
525?~ r.hit !!
526```
528Here we verify that the `r` face of our response is not null before
529continuing. If it is null, the generator will crash, as there is nothing left to
530do.
532```hoon
533=/ my-json (de-json:html q.u.r.hit)
534```
536Here we verify that data was returned to us in the `json` Hoon type.
537`q.u.r.hit` is the actual data we care about as a `cord`. `hit` is actually an
538entire response object, including headers, which we don't care about for this
539example. `de-json` out of `html` will parse the `cord` and produce a `json` type
540for us that we can then use to inspect the data.
542```hoon
543?~ my-json !!
544=, dejs:format
545```
547After parsing the JSON structure we want to make sure we got back a valid `json`
548type. To do that, we test for null and crash if that test passes. If it doesn't
549pass, we use `=,` on the `dejs:format` core so that we can more easily pull arms
550out of it.
552```hoon
553%+ produce %json
554%. %title
555%~ got by
556```
558Now we come to our `produce` call. Recall that `produce` is used in all the
559non-naked generators to output the final result. Here we want to produce a
560`cask` of `json`, so we use the mark `%json` to indicate that.
562Next we have what may be an unfamiliar rune: `%.`. This rune is simply `%-` with
563the argument order reversed. `%title` is the argument passed to the next gate
564that is called. `by` is the core that is the `map` engine. A Hoon `map` is a
565key-value pair structure, sometimes referred to as a dictionary in other
566languages. `got` is the arm in `by` that will produce a gate that we can call to
567access members of a given `map`.
569> The idiomatic way to write Hoon is to have the
570 "heaviest" code paths lowest in the program, which is why `%.`
571 is preferred in this case over `%-`. Use of this style is recommended, as it
572 results in code that is much easier to read.
574At the moment, however, we do not have a `map`, but instead have a `json` type.
576```hoon
577%- (om same)
578u.my-json
579```
581Here we use `om`, which is an arm in `dejs:format` that will take a `json` and
582return a `map`. `same` is a wet gate from the standard library that takes an
583argument of any type and returns it as that type. We could use any other gate to
584modify the data if that were useful for the particular data we are going to be
585processing, but here we want the data exactly as it is.
587Since `my-json` is a `unit` of `json`, the actual data we are looking for is in
588`u.my-json`. We take this and give it to the gate produced by the call to `om`.
589This gate is going to transform our data by taking in the `json` structure and
590producing a `map`. Then we can look up keys in this map using `got` from
591the `by` core.
593`%.` performs the same function as `%-`, the rune used to call a function,
594except with the arguments reversed so that the function is last and
595the arguments of that function come first. It's used here to keep with the
596principle that heaviest, or most complex, expressions should go to the bottom.
598The key we are looking up in our new `map` we've created from `my-json` is
599`%title`, which is an element in the original source, to produce the final
600result of the generator.
602## Conclusion
604You've now reached the end of Chapter 1 of the Hoon tutorial. Ideally you should have a fair understanding of the fundamental concepts of subject-oriented programming: limbs, legs, faces, wings, arms, cores, gates, and [doors](/docs/glossary/door/). If you can master these concepts you should have little or no trouble learning to write substantial Hoon programs.