on master help sign in
purging or warning labels on arvo kernel and ford (#917)

* purging or warning labels on arvo kernel and ford * revised headers

commitae139c8f123207389e59ebb947a0834daed87c79 authorpoprox <51337059+syzygyzer@users.noreply.github.com> dateMon 20 Jul 2020, 15:30 (6 years ago) parent0eff0b5
4 files changed+7−944
tutorials/arvo/arvo.md+4 tutorials/arvo/ford.md+3−560 tutorials/hoon/ford.md−193 tutorials/hoon/test-sets.md−191
tutorials/arvo/arvo.md+4
@@ -4,6 +4,10 @@ weight = 1
44template = "doc.html"
55aliases = ["/docs/learn/arvo/arvo/"]
66+++
7+
8+> Parts of this document are out of date as of 2020.07.20, please use information here with
9+> caution. This message will be removed once it is up to date.
10+
711Arvo, also called Urbit OS, is our operating system.
812
913# Introduction
tutorials/arvo/ford.md+3−560
@@ -1,566 +1,9 @@
11+++
2−title = "Ford"
2+title = "Ford Fusion"
33weight = 7
44template = "doc.html"
55aliases = ["/docs/learn/arvo/ford/"]
66+++
7−Our typed and marked computation engine.
87
9−A variety of different services are provided by `%ford`, but they mostly
10−involve compiling hook files, slapping/slamming code with marked data,
11−and converting data between marks, including validating data to a mark.
12−Throughout every computation, `%ford` keeps track of which resources are
13−dependencies so that the client may be aware when one or more
14−dependencies are updated.
15−
16−`%ford` neither accepts unix events nor produces effects. It exists
17−entirely for the benefit of applications and other vanes, in particular
18−`%gall`. `%eyre` exposes the functional publishing aspects of `%ford`
19−while `%gall` uses `%ford` to control the execution of applications.
20−`%clay` is intended to use `%ford` to managed marked data, but this is
21−not yet reality.
22−
23−## Cards
24−
25−`%ford` accepts just one card, `%exec`. This is misleading, however,
26−since there are fourteen different `silk`s that may be used with it. In
27−every case, the expected response to a `%exec` card is a `%made` gift
28−with either an error or the produced result along with its set of
29−dependencies.
30−
31−Silks may autocons, so that the product of a cell of silks is a cell of
32−the product of the two silks.
33−
34−### `%bake`
35−
36−Tries to functionally produce the file at a given beam with the given
37−mark and heel. It fails if there is no way to translate at this level.
38−
39−### `%boil`
40−
41−Functionally produces the file at a given beam with the given mark and
42−heel. If there is no way to translate at this beam, we pop levels off
43−the stack and attempt to bake there until we find a level we can bake.
44−This should almost always be called instead of `%bake`.
45−
46−### `%call`
47−
48−Slams the result of one silk against the result of another.
49−
50−### `%cast`
51−
52−Translates the given silk to the given mark, if possible. This is one of
53−the critical and fundamental operations of ford.
54−
55−### `%diff`
56−
57−Diffs the two given silks (which must be of the same mark), producing a
58−cage of the mark specified in `++mark` in `++grad` for the mark of the
59−two silks.
60−
61−### `%done`
62−
63−Produces exactly its input. This is rarely used on its own, but many
64−silks are recursively defined in terms of other silks, so we often need
65−a silk that simply produces its input. A monadic return, if you will.
66−
67−### `%dude`
68−
69−Computes the given silk with the given tank as part of the stack trace
70−if there is an error.
71−
72−### `%dune`
73−
74−Produces an error if the cage is empty. Otherwise, it produces the value
75−in the unit.
76−
77−### `%mute`
78−
79−Takes a silk and a list of changes to make to the silk. At each wing in
80−the list we put the value of the associated silk.
81−
82−### `%pact`
83−
84−Applies the second silk as a patch to the first silk. The second silk
85−must be of the mark specified in `++mark` in `++grad` for the mark of
86−the first silk.
87−
88−### `%plan`
89−
90−Performs a structured assembly directly. This is not generally directly
91−useful because several other silks perform supersets of this
92−functionality. We don't usually have naked hoods outside ford.
93−
94−### `%reef`
95−
96−Produces a [core](/docs/glossary/core/) containing the entirety of Zuse and Hoon, suitable for
97−running arbitrary code against. The mark is `%noun`.
98−
99−### `%ride`
100−
101−Slaps a hoon against a subject silk. The mark of the result is `%noun`.
102−
103−### `%vale`
104−
105−Validates untyped data from a ship against a given mark. This is an
106−extremely useful function.
107−
108−## Ford runes
109−
110−One of the most common ways to use ford is through ford's runes, all of which begin with `/` ("fas"). A ford rune runs a step of a ford build and places the result in the subject. There are various kinds of build steps, some of which take other build steps as parameters. The most common pattern is to have a sequence of ford runes at the top of a hoon source file that import the results of evaluating other hoon files. This is how we "import a library" in urbit: we add the result of compiling another hoon file into the current subject, possibly renaming it by wrapping a face around it.
111−
112−Another common use case is assembling and rendering markdown templates into a single HTML file, to be sent as an HTTP response in urbit's web server `%eyre`. It's important to keep in mind that %ford is quite flexible and generic, and is used in other parts of urbit -- such as the dojo -- that would not traditionally fall under the purview of a build system.
113−
114−### `/+` "faslus"
115−
116−The `/+` rune accepts a filename as an argument. It interprets that filename
117−as a hoon source file within the `lib` directory. This is how we import a shared
118−library in urbit.
119−
120−To run this example, put this code in your desk at `gen/faslus.hoon` and run
121−`+faslus` in your dojo. This example is a generator. For more information on
122−generators, see the [generator docs](@/docs/tutorials/hoon/generators.md).
123−
124−```hoon
125−/+ time-to-id
126−::
127−:- %say
128−|= {{now/@da * *} $~ $~}
129−:- %noun
130−(time-to-id now)
131−```
132−
133−produces: `"c.314d"` (or something similar depending on when you run it)
134−
135−You can import multiple libraries with a single `/+` rune by separating them
136−with commas.
137−
138−Replace the code in `gen/faslus.hoon` with the following:
139−
140−```hoon
141−/+ time-to-id, hep-to-cab
142−::
143−:- %say
144−|= {{now/@da * *} $~ $~}
145−:- %noun
146−=/ id (time-to-id now)
147−=/ str "my-id-is-{id}"
148−(hep-to-cab (crip str))
149−```
150−
151−This should print something like `my_id_is_c.3588`.
152−
153−Another feature of the
154−`/+` and `/-` runes is the ability to specify the ship and case from which to
155−load the library.
156−
157−Example:
158−
159−```
160−/+ time-to-id, hep-to-cab/4/~zod
161−```
162−
163−will load the `hep-to-cab` library from `~zod` at `%clay` revision `4`.
164−
165−### `/-` "fashep"
166−
167−The `/-` rune accepts a filename as an argument. It interprets that filename as
168−a hoon source file within the `sur` directory. The `sur` directory contains
169−shared structures that can be used by other parts of urbit. This is somewhat
170−similar to including a header file in C.
171−
172−Example:
173−
174−```hoon
175−/- talk
176−::
177−*serial:talk
178−```
179−
180−produces: `0v0`
181−
182−`/-` can also take multiple files as arguments, and the ship and case of those
183−arguments can be specified. See the `/+` docs for more details about the syntax
184−for those features.
185−
186−### `/~` "fassig"
187−
188−`/~ <twig>` produces a "horn" that evaluates a twig and places the product in
189−the subject. Arbitrary hoon can be in the twig. A
190−horn is a data
191−structure representing a `%ford` static resource.
192−
193−Example:
194−
195−```hoon
196−/~ [a=0 b=1]
197−[a b]
198−```
199−
200−produces: `[0 1]`
201−
202−In wide-form, `/~` always takes a tuple (which may be a degenerate tuple of one element), and produces it.
203−
204−Example:
205−```hoon
206−/~[%something]
207−```
208−produces: `%something`
209−
210−```hoon
211−/~[%a %b]
212−```
213−produces: `[%a %b]`
214−
215−### `//` "fasfas"
216−
217−`// <rel-path>` parses `rel-path` as a hoon twig, and then adds the resulting
218−twig to the subject. Note that the result type of this rune is not a horn, but
219−just a hoon twig, so its result can't be used as an argument to other runes
220−that expect a horn. There is no wide-form for this rune.
221−
222−Example:
223−```hoon
224−// %/file-to-include
225−::
226−(frobnicate:file-to-include %.n)
227−
228−:: contents of file-to-include.hoon
229−|%
230−++ frobnicate
231− |= a/? ^+ a
232− !a
233−--
234−```
235−produces: `%.y`
236−
237−_Note: `%` means current directory, which in a hoon file will resolve to the
238−"directory" containing that file. Also, %clay doesn't have first-class
239−directories the way unix filesystems do. Whereas in unix, a directory is a
240−special kind of file, in %clay it's just a path prefix, and there is no file
241−stored at that path._
242−
243−### `/=` "fastis"
244−
245−`/=` runs a horn (usually produced by another ford rune), takes the result of
246−that horn, and wraps a face around it.
247−
248−Example:
249−```hoon
250−/= foo /~ [a=0 b=1]
251−::
252−[a.foo b.foo]
253−```
254−produces: `[0 1]`
255−
256−In wide-form, `/=` uses `=` as a delimiter: `/=foo=/~[a=0 b=1]`
257−
258−### `/:` "fascol"
259−
260−`/:` takes a path and a horn, and evaluates the horn with the current path set to the supplied path.
261−`/mar/` renders the mark 'mar' at the current path.
262−
263−Example:
264−```hoon
265−/= hoo-source /: /path/to/hoon-file /hoon/
266−::
267−`@t`hoo-source
268−```
269−
270−produces the text of the hoon file at "/path/to/hoon-file/hoon".
271−
272−`/hoon/` renders the current path using the `%hoon` mark, which in this case
273−passes the contents through unchanged. In general, rendering a file with a
274−mark will potentially run the contents through a series of conversion
275−operations. For details on marks, see the
276−[Gall docs](gall).
277−
278−Here's an example that includes a mark conversion:
279−```hoon
280−/= page /: /path/to/html/file /mime/
281−::
282−page
283−
284−:: contents of /path/to/html/file:
285−<html><head><title>My Fascinating Blog</title></head></html>
286−```
287−produces:
288−`[[%text %html ~] 60 '<html><head><title>My Fascinating Blog</title></head></html>']`
289−
290−This result includes the MIME type ('text/html'), the content length in bytes,
291−and the HTML itself.
292−
293−In wide-form, `/:` uses `:` as a delimiter: `/=page=/:/path/to/html/file:/mime/`
294−
295−### `/!mark/` "faszap"
296−
297−Example:
298−```hoon
299−/= mime /: /%/some-hoon-file /!mime/
300−::
301−mime
302−
303−:: contents of /%/some-hoon-file:
304−%produces-a-cord
305−```
306−produces: `[[%text %plain ~] 15 'produces-a-cord']`
307−
308−### `/&` "faspam"
309−
310−`/&` passes a horn through multiple marks, right-to-left. It has both a
311−wide-form and a tall-form syntax. In wide-form, it takes a series of mark
312−arguments followed by a horn. In tall-form, it takes a single mark followed by
313−a horn.
314−
315−```hoon
316−/= some-text /: /%/text-file /&mime&/txt/
317−::
318−some-text
319−```
320−produces: `[[%text %plain ~] p=17 q='Hi I\'m some text\0a']`
321−
322−This example shows two of the ways marks are used. The first way is what
323−happens with the `/txt/` mark: we use it to find a file in clay with that
324−extension, without performing any conversions. Since this file is stored with
325−the `%txt` mark in `%clay`, its type is a `wain`: a list of cords, where each
326−cord is a single line. Once we've read the file, the `%mime` mark converts the
327−`wain` to a triple that includes the MIME type ("text/plain"), the content
328−length in bytes, and the content itself as a cord.
329−
330−```hoon
331−/= page /&html&elem&/~[;div.foo;]
332−::
333−page
334−```
335−produces: `'<html><head></head><body><div class="foo"></div></body></html>'`
336−
337−This runs the sail expression `;div.foo;` through the `%elem` mark, then
338−through the `%html` mark. The `/~` rune produces an item of mark `noun`. The
339−`%elem` mark converts the mark of the expression from `noun` to `elem` by
340−checking that the type fits in `manx` (a hoon/sail type indicating an XML
341−element). The `%html` mark recognizes the `%elem` and converts it to an HTML
342−string with enclosing `<html>`, `<head>`, and `<body>` tags.
343−
344−It's possible to use wide-form `/&` with more than two marks, by using `&` as a
345−delimiter between marks and adding a `/` before the last mark, like:
346−`/&d&c&b&/a/`. The last argument here can actually be any arbitrary horn, not
347−just a mark: `/&c&b&/:path:/mark/`
348−
349−Tall-form `/&` takes only two arguments: a mark and a horn. The mark does not
350−need to be enclosed in `/`'s: `/& html /elem/`
351−
352−### `/|` "fasbar"
353−
354−`/|` takes a series of horns and produces the first one (left-to-right) that
355−succeeds. If none succeed, it produces stack traces from all of its arguments.
356−
357−Example:
358−```hoon
359−/= res /: /%/path/to/file
360−/|
361− /!elem/
362− /elem/
363−==
364−::
365−res
366−```
367−tries to parse the file at `/%/path/to/file` as a hoon expression and evaluate
368−it (`/!elem/`), and then if that fails, tries to convert the file to an HTML
369−element directly (`/elem/`). This is used in urbit's `tree` publishing system
370−to enable a user to place either a hoon file or a static file in a directory,
371−and have them both result in webpages, with preference given to treating the
372−file as hoon to be evaluated.
373−
374−Wide-form `/|` encloses its arguments in parentheses, with a single space as a delimiter:
375−`/|(/!elem/ /elem/)`
376−
377−### `/_` "fascab"
378−
379−`/_` can be used in two ways: filtered and unfiltered.
380−
381−Unfiltered `/_` takes a horn as an argument. It produces a new horn representing the
382−result of mapping the supplied horn over the list of files in the current
383−directory. The keys in the resulting map are the basenames of the files in the
384−directory, and each value is the result of running that horn on the contents of
385−the file.
386−
387−Example:
388−```hoon
389−/= kids /_ /hoon/
390−::
391−`(map term cord)`kids
392−```
393−
394−produces a value of type `(map term cord)`, where each key is the basename (the
395−filename without the prefix), and each value is the result of running the
396−contents of the file through the `%hoon` mark, which validates the
397−hoon code and returns it unmodified. So, the resulting map associates basenames
398−with file contents.
399−
400−Wide-form unfiltered `/_` doesn't need a delimiter: `/= kids /_/hoon/`
401−
402−### `/_` filtered: run a horn on each file in the directory matching an aura
403−
404−Filtered `/_` takes an aura and a horn, and filters the list of files in the
405−current directory by whether their filenames can be parsed to an [atom](/docs/glossary/atom/) of that
406−aura. It then produces a map where each key is the filename after parsing into
407−that atom type, and each value is the result of running the horn on the
408−contents of that file.
409−
410−Example:
411−```hoon
412−/= timed-posts /_ @da /md/
413−`(map @da @t)`timed-posts
414−```
415−produces a map from dates to cords. This product will only contain files whose
416−names are parsable into `@da` dates. The values are the file contents converted
417−to a cord of markdown-formatted text.
418−
419−Wide-form filtered `/_` uses `_` as a delimiter: `/= timed-posts /_@da_/md/`
420−
421−### `/;` "fasmic"
422−
423−`/;` takes a twig and a horn. The twig should evaluate to a [gate](/docs/glossary/gate/), which is then slammed
424−with the result of the horn as its sample.
425−
426−Example:
427−```hoon
428−/= goo
429− /; |=({a/@ b/@} +(b))
430− /~ [a=0 b=1]
431−::
432−goo
433−```
434−
435−produces: `2`
436−
437−Here's a slightly more complex example with runes that use the filesystem:
438−```hoon
439−/= file-length
440− /; |=(a/@t (lent (crip a)))
441− /: /%/path/to/hoon/file /hoon/
442−::
443−file-length
444−```
445−produces the number of bytes in the file "/%/path/to/hoon/file."
446−
447−### `/,` "fascom"
448−
449−`/,` is a switch statement, which picks a branch to evaluate based on
450−whether the current path matches the path in the switch statement.
451−Takes a sequence of pairs of (path, horn) terminated by a `==`. No wide-form.
452−
453−Example:
454−```hoon
455−/= just-right
456− /: /===/right-path :: set path to /%/right-path
457− /,
458− /wrong-path /~ ~
459− /another-wrong-path /~ ~
460− /right-path /~ %evaluate-me :: only evaluate this horn
461− ==
462−::
463−`@t`just-right
464−```
465−
466−produces: `'evaluate-me'`
467−
468−### `/.` "fasdot"
469−
470−Produce a null-terminated list from a sequence of horns, terminated by a `==`. No wide-form.
471−
472−Example:
473−```hoon
474−/= vanes
475− /.
476− /~ %ames
477− /~ %behn
478− /~ %clay
479− /~ %dill
480− /~ %eyre
481− /~ %ford
482− /~ %gall
483− ==
484−::
485−vanes
486−```
487−produces: `[%ames %behn %clay %dill %eyre %ford %gall ~]`.
488−
489−### `/^` "fasket"
490−
491−`/^` takes a mold and a horn, and casts the result of the horn to the mold.
492−
493−Example:
494−```hoon
495−/= liz /^ (list @) :: cast to real list
496− /~ ~[1 2 3] :: no 'i' or 't' faces
497−::
498−?< ?=($~ liz) :: prevent find-fork
499−i.liz
500−```
501−
502−produces: `1`
503−
504−Without the cast, we wouldn't be able to access the 'i' face of the list.
505−
506−Wide-form `/^` uses `^` as a delimiter: `/^(list @)^/~[1 2 3 ~]`
507−
508−### `/#` "fashax"
509−
510−`/#` takes a horn and produces a cell of the dependency
511−hash of the result of the horn, and the result itself.
512−
513−Example:
514−```hoon
515−/= inline /^ {dep/@uvH txt/@t} /# /: /%/my-script /js/
516−::
517−;= ;script: (trip txt.inline) :: set script contents
518− ;script@"/~/on/{<dep.inline>}.js"; :: set script src
519−==
520−```
521−produces two `<script>` tags. The first has its contents set to the contents
522−of the JavaScript file we loaded from clay. The second is an auto-update
523−script that polls the server to check whether the dependency hash has changed.
524−This pattern is used in urbit's `tree` web publishing system.
525−
526−Wide-form `/#` does not need a delimiter:
527−`/= inline /^ {dep/@uvH txt/@t} /#/:/%/my-script:/js/`
528−
529−### `/$` "fasbuc"
530−
531−`/$` will slam a gate on whatever extra arguments have been supplied to this build.
532−At the moment, only HTTP requests forwarded by `%eyre` contain any extra arguments,
533−and using this rune outside of that context will cause an error. Requests from `%eyre`
534−contain an argument representing the query string, which can be parsed using the
535−standard library gate `++fuel:html`.
536−
537−Example:
538−```hoon
539−/= gas /$ fuel:html
540−::
541−:*
542− %who {<(~(get ju aut.ced.gas) 0)>}
543− %where {(spud s.bem.gas)}
544− %case {(scow %ud p.r.bem.gas)}
545−==
546−```
547−produces: [%who '~zod' %where /ford/pages/web %case 60]
548−
549−Wide-form `/$` takes its arguments inside brackets, like `/~`:
550−`/=gas=/$[fuel:html]`
551−
552−### `/%` "fascen"
553−
554−`/%` will forward extra arguments (usually from `%eyre`) on to any enclosed `/renderer/`'s. Without this,
555−renderers that use `/$` to read the extra arguments will crash.
556−
557−Example:
558−```hoon
559−/= dat /% /tree-json/
560−::
561−dat
562−```
563−produces the results of running `ren/tree-json` on the current path. This renderer needs the query string,
564−which is one of the extra arguments passed in from `%eyre`.
565−
566−Wide-form `/%` has no delimiter: `/%/tree-json/`
8+> This document is out of date as of 2020.07.20. In the meantime, see the [blog
9+> post](https://urbit.org/blog/ford-fusion/) about Ford Fusion.
tutorials/hoon/ford.mddeleted−193
@@ -1,193 +0,0 @@
1−+++
2−title = "2.8 Ford"
3−weight = 38
4−template = "doc.html"
5−+++
6−
7−This lesson introduces the basics of the Ford vane, Arvo's build system and computation engine.
8−
9−In terms of sheer size, Ford is Arvo's largest vane, weighing in at over 6000 lines of code at the time of writing. As a result, we will only be scratching the surface of Ford in this lesson. We will mostly focus on describing how Ford works in a broad sense and not dig too deeply into the details.
10−
11−## What are build systems?
12−
13−You may safely skip this section if you are familiar with build systems, as there is no Ford-specific information.
14−
15−At a high level, build systems are software that builds other software from source files. How this is accomplished varies from build system to build system, but there are some general properties shared by almost everything in this category.
16−
17−The primary output of a build system, the final binary known as a _build_, comes from two inputs: (1) source files, and (2) build instructions. Analogous to building a house, you could think of these as corresponding to raw materials and architectural blueprints. The build system calls the compiler (typically a separate piece of software) on each source file, which you may think of as shaping the raw materials something suitable for the final product. The compiled source files are then put together according to the build instructions.
18−
19−The number of languages a build system is designed to work with varies, from specialized build systems that focus on a single language (like Ford) to general purpose ones that could conceivably work for any language and frequently rely on other build systems to function (such as [make](https://en.wikipedia.org/wiki/Make_(software))).
20−
21−Source files frequently have _dependencies_, which are other source files, outside libraries, or tools needed to build a software program. These dependencies can have dependencies themselves, and in general one can draw a dependency graph of all of the different libraries and tools that go into building a project.
22−
23−It is frequently the case that only specific versions of a given library or tool will work. This can create contradictory version requirements, an issue that falls under the umbrella of "dependency hell". Most build systems suffer from this problem, but functional programming provides workarounds. [Nix](https://nixos.org/nix/) is a popular build system that avoids this issue, and Ford does as well.
24−
25−Besides the primary task of building software outlined above, build systems also typically handle some or all of the following responsibilities:
26− - Resolving dependencies
27− - Downloading dependencies
28− - Managing dependencies (updating, garbage collection, etc.)
29− - Making modular sub-builds
30− - Running tests on compiled code
31− - Performing version control
32−
33−While the inner workings of a build system may be quite complex, for most smaller projects they are very simple to use.
34−
35−## Ford Features
36−
37−In this section we cover the main features of the Ford build system that distinguish it from other build systems. We do not yet speak of how they are implemented, which will be covered to some extent later in the lesson. We also remind you that Ford has additional capabilities that are not covered in this lesson.
38−
39−The main features of Ford that distinguish it from most build systems that we wish to emphasize in this section are the following:
40−
41− - Ford is strongly and dynamically typed
42− - Ford is monadic
43− - Ford is referentially transparent
44− - Ford can do live builds
45− - Ford keeps a cache of previous builds
46−
47− To some extent these features overlap depending on how you split hairs. If you are already familiar with build systems, know that the capabilities of Ford are similar to those of [Shake](https://shakebuild.com/) and [Nix](https://nixos.org/nix/), but still differs from them in some key aspects.
48−
49− We will also give an overview of how to use Ford for unit testing, as that is what we will be doing in the following lesson.
50−
51−
52−### Strongly and Dynamically Typed
53−
54−Ford is a typed build system. The short version of what this means is that Ford cares about Arvo marks and Hoon types, and will fail to build if the marks and types do not satisfy certain restrictions. This guarantees to some level that the built software will work, though of course you can never completely guarantee that software will work as expected due to things such as bugs.
55−
56−Recall that a mark such as `%foo` acts as metadata which tells Arvo vanes what kind of file it is working with. In the context of Ford, they may be thought of as being analogous to file types in other operating systems. When an Arvo vane sees marked data, it expects that data to take a certain shape, and verifying that that data is in the expected shape is a task performed by Ford.
57−
58−In fact, it is important to emphasize that the Arvo kernel does not actually know what marks are. Ford is the vane that knows what marks are, and whenever the kernel or another vane needs to know what a mark means, it asks Ford.
59−
60−Ford is also responsible for converting marked data to another mark. This is a task that is frequently performed during a build, but also is something that other vanes may request Ford to do.
61−
62−Ford is dynamically typed in the sense that types of builds are not known a priori and are computed dynamically in the course of the build. It is strongly typed in the sense that it is typesafe and the result of a build always has a known type, as do all of the sub-builds that went into it.
63−
64−### Monadic
65−
66−That Ford is a monadic build system essentially means that dependencies are generated dynamically as the build proceeds, as a opposed to the list of dependencies being a static value that is input manually, such as with `make`. There are no downsides to this over the alternative, which are known as applicative build systems.
67−
68−### Referentially transparent
69−
70−Ford is [referentially transparent](https://en.wikipedia.org/wiki/Referential_transparency). The exact meaning of this depends on who you ask, but for us it means that one may replace a reference to a build (which in Ford's case is something known as a schematic, see below) with the result of that build without changing the output of the overall build. What this ultimately means is that builds are deterministic, and so the same build of the same source files will produce the same output every time. Put another way, the output is a pure function of the input. If you've never dealt with build systems before, the idea that things could work any other way may sound absurd!
71−
72−For a typical applicative build system, such as `make`, the build instructions contain a static list of dependencies. These dependencies may or may not be written with a restriction on a particular version, and even for a fixed version number there may still be multiple builds for any number of reasons (like poor version control practices). For example, you may run a build with a package with `make` that has `gizmo` named as a dependency, with no version restriction. On one system, you may have `gizmo` v1.0 installed, while on another you may have `gizmo` 1.1 installed. The build system makes no distinction between these versions - all it sees is that it needs `gizmo`, and pays no attention to the version. Thus, building the same source files with the same build instructions on two different system may result in two slightly different outputs.
73−
74−One might instead call Ford purely functional instead of (or in addition to) referentially transparent. The exact meaning of this terminology differs from source to source and how many hairs you want to split, but the general notion that one may replace Hoon expressions by their product without changing the output of the build is true.
75−
76−### Cache
77−
78−Ford saves much of what it builds in a cache. Whenever Ford generates a dependency for a build, it checks the cache to see if it has already built that dependency before. If so, because Ford is referentially transparent, the result of that build can be put directly into the current build. No rebuilding necessary.
79−
80−The primary reason for this feature is that it saves time - building large projects can take hours or more.
81−
82−The cache is somewhat analogous to the Nix store in functionality, but is quite different in terms of implementation. How the cache works is by far the most complex part of Ford, but this internal complexity ultimately results in simplicity for the software engineer.
83−
84−Ford does not save _everything_ it has ever built forever, but instead applies a heuristic to determine what build results to hold onto. An example of such a heuristic is "LRU caching", and Ford does something similar to this.
85−
86−### Live Builds
87−
88−Ford is capable of live builds. This means that Ford can subscribe to a Clay desk containing some source code for something Ford has built before, and whenever that source code is updated Ford will automatically rebuild the project.
89−
90−It is important for us to note that Ford is _not_ used to build vanes. Vanes are compiled from the raw source code by the Arvo kernel.
91−
92−## Using Ford
93−
94−Having covered the distinctive features of Ford, we now dive into some of the specifics on how these features are implemented. Let us first cover some of most important types of data we will run into, and then discuss the most frequently used [arms](/docs/glossary/arm/).
95−
96−### Data types
97−
98−#### Vase
99−
100−From `hoon.hoon`:
101−```hoon
102−+$ vase [p=type q=*]
103−```
104−A `vase` is simply typed data, and it is used wherever typed data is being explicitly worked with.
105−
106−#### Cage
107−
108−From `arvo.hoon`, a `cage` is a `[mark vase]` - so its just typed data that is also marked for usage by Arvo.
109−
110−#### Schematics
111−
112−A `schematic`, found in `zuse.hoon`, is a set of build instruction for Ford. It is a recursive data structure, in that a schematic may itself consist of additional schematics. Schematics have 25 subtypes corresponding to possible basic build instructions. The product of a `schematic` is a `cage`.
113−
114−Some examples of subtypes include:
115− - `%list` - a list of schematics to build.
116− - `%[core](/docs/glossary/core/)` - a Clay path to a Hoon source file to build.
117− - `%call` - a schematic consisting of two sub-schematics, one of which produces a [gate](/docs/glossary/gate/) and a second that produces a sample. It then runs the [gate](/docs/glossary/gate/) on the sample, producing a vase of the result, tagged with the mark `%noun`.
118− - `%scry` - look up a value from the Urbit namespace.
119−
120− Schematics are only ever sent to Ford by another vane. Userspace apps do not directly create schematics.
121−
122−### Ford runes
123−
124−Ford runes, which all begin with `/`, are instructions for Ford typically written at the top of a source file. Ford runes instruct Ford to perform actions such as scraping through directories and grabbing files from Clay.
125−
126−The most common pattern you are likely to see is a sequence of Ford runes at the top of a Hoon source file that imports the results of evaluating other Hoon files. This is like importing a library in any other build system. The way this works in Hoon is that the result of compiling a named Hoon source file is added to the current subject, possibly with a face.
127−
128−The rune interpreter is most complicated part of `ford.hoon` after the cache. But again, thankfully for us, most of the commonly used runes are straightforward. We cover a few of them here.
129−
130−#### `/+` import from `lib/`
131−
132−The `/+` rune accepts a filename as an argument. It interprets that filename as a hoon source file within the `lib` directory. This is how we import a shared library in urbit.
133−
134−To run this example, put this code in your desk at `gen/faslus.hoon` and run `+faslus` in your dojo.
135−
136−```hoon
137−/+ time-to-id
138−::
139−:- %say
140−|= {{now/@da * *} $~ $~}
141−:- %noun
142−(time-to-id now)
143−```
144−
145−produces: `"c.314d"` (or something similar depending on when you run it).
146−
147−You can import multiple libraries with a single `/+` rune by separating them with commas.
148−
149−Replace the code in `gen/faslus.hoon` with the following:
150−
151−```hoon
152−/+ time-to-id, hep-to-cab
153−::
154−:- %say
155−|= {{now/@da * *} $~ $~}
156−:- %noun
157−=/ id (time-to-id now)
158−=/ str "my-id-is-{id}"
159−(hep-to-cab (crip str))
160−```
161−
162−This should print something like `my_id_is_c.3588`.
163−
164−Another feature of the `/+` rune (and `/-` below) is the ability to specify the ship and case from which to load the library.
165−
166−Example:
167−
168−```hoon
169−/+ time-to-id, hep-to-cab/4/~zod
170−```
171−will load the `hep-to-cab` library from `~zod` at `%clay` revision `4`.
172−
173−#### `/-` import from `sur/`
174−
175−The `/-` rune accepts a filename as an argument. It interprets that filename as a hoon source file within the `sur` directory. The `sur` directory contains shared structures that can be used by other parts of urbit. This is somewhat similar to including a header file in C.
176−
177−Example:
178−
179−```hoon
180−/- talk
181−::
182−*serial:talk
183−```
184−
185−produces: `0v0`
186−
187−`/-` can also take multiple files as arguments, and the ship and case of those arguments can be specified. See the `/+` docs for more details about the syntax for those features.
188−
189−### Testing
190−
191−Ford is the vane utilized for unit testing, which are the the simplest sort of test one may perform on software that is one step removed from manual testing. Ford itself does not have testing capabilities built in, rather Ford has a more general capability known as _rendering_ which is utilized by the `+test` generator to perform the tests. The walkthrough which follows this lesson goes into more detail.
192−
193−One typically tests software by feeding it inputs and seeing if the outputs matches what is expected (something that is determined manually by the engineer). Unit tests work with only a single "module" at a time. What exactly is meant by this is shown by example in the following walkthrough, but for now it suffices to say that unit tests are in contrast with larger scale tests variously known as end-to-end tests, system tests, and integration tests.
tutorials/hoon/test-sets.mddeleted−191
@@ -1,191 +0,0 @@
1−+++
2−title = "2.8.1 Unit Testing with Ford"
3−weight = 39
4−template = "doc.html"
5−+++
6−
7−In this walkthrough we will show how to use Ford as a testing suite.
8−
9−First we will cover the essential molds, [arms](/docs/glossary/arm/), and generators used for testing, followed by walking through the code that implements a series of tests for standard library sets, as described [here](@/docs/tutorials/hoon/trees-sets-and-maps.md).
10−
11−## Running our first test
12−
13−To start with, lets try running the unit tests on sets by entering the following into the Dojo, and then we will work backwards to see how we obtained this output:
14−```
15−> +test /sys/hoon/set
16−OK /sys/hoon/set/test-set-all
17−OK /sys/hoon/set/test-set-any
18−OK /sys/hoon/set/test-set-apt
19−OK /sys/hoon/set/test-set-bif
20−OK /sys/hoon/set/test-set-del
21−OK /sys/hoon/set/test-set-dif
22−OK /sys/hoon/set/test-set-dig
23−OK /sys/hoon/set/test-set-gas
24−OK /sys/hoon/set/test-set-has
25−OK /sys/hoon/set/test-set-int
26−OK /sys/hoon/set/test-set-put
27−OK /sys/hoon/set/test-set-rep
28−OK /sys/hoon/set/test-set-run
29−OK /sys/hoon/set/test-set-tap
30−OK /sys/hoon/set/test-set-uni
31−OK /sys/hoon/set/test-set-wyt
32−```
33−Here, we ran a generator called `+test` with a path argument `/sys/hoon/set`. The output tells us that all tests ran successfully. Running this generator automatically appends `/tests/` to the front of the path entered, so lets look at the beginning of `/tests/sys/hoon/set.hoon` to see what the `+test` generator took as an input:
34−
35−```hoon
36−:: Tests for +in (set logic)
37−::
38−/+ *test
39−::
40−:: Testing arms
41−::
42−|%
43−```
44−We note that the first line that isn't a comment is `/+ *test`, which from [the previous lesson](@/docs/tutorials/hoon/ford.md) you know means that we are importing the shared library `lib/test.hoon`. This is followed by a [core](/docs/glossary/core/) which will contain arms that perform our tests, which we will see an example of shortly.
45−
46−In `lib/test.hoon`, we find a core with a few [gates](/docs/glossary/gate/): `expect-eq`, `expect-fail`, and `category`. The source for `expect-eq` is:
47−```hoon
48−++ expect-eq
49− |= [expected=vase actual=vase]
50− ^- tang
51− ::
52− =| result=tang
53− ::
54− =? result !=(q.expected q.actual)
55− %+ weld result
56− ^- tang
57− :~ [%palm [": " ~ ~ ~] [leaf+"expected" (sell expected) ~]]
58− [%palm [": " ~ ~ ~] [leaf+"actual" (sell actual) ~]]
59− ==
60− ::
61− =? result !(~(nest ut p.actual) | p.expected)
62− %+ weld result
63− ^- tang
64− :~ :+ %palm [": " ~ ~ ~]
65− :~ [%leaf "failed to nest"]
66− (~(dunk ut p.actual) %actual)
67− (~(dunk ut p.expected) %expected)
68− == ==
69− result
70−```
71−This is the most frequently used gate in testing, so we may conclude that any testing suite will likely have the line `/+ *test` at the top so that we may utilize this gate. What does this gate do? Recall that a `vase` consists of `[p=type q=*]`. Then what `expect-eq` does is check to see if two `vase`s are equal, and pretty-prints the result of that test.
72−
73−## Testing the functionality of set intersection
74−
75−Here's one of the testing arms in the set testing core:
76−```hoon
77−++ test-set-int ^- tang
78− =/ s-nul=(set @) *(set @)
79− =/ s-asc=(set @) (sy (gulf 1 7))
80− =/ s-des=(set @) (sy (flop (gulf 1 7)))
81− =/ s-dos=(set @) (sy (gulf 8 9))
82− =/ s-dup (sy ~[1 1 4 1 3 5 9 4])
83− ;: weld
84− :: Checks with empty set
85− ::
86− %+ expect-eq
87− !> ~
88− !> (~(int in s-nul) s-asc)
89− %+ expect-eq
90− !> ~
91− !> (~(int in s-asc) s-nul)
92− :: Checks with all elements different
93− ::
94− %+ expect-eq
95− !> ~
96− !> (~(int in s-dos) s-asc)
97− :: Checks success (total intersection)
98− ::
99− %+ expect-eq
100− !> s-asc
101− !> (~(int in s-asc) s-des)
102− :: Checks success (partial intersection)
103− ::
104− %+ expect-eq
105− !> (sy ~[9])
106− !> (~(int in s-dos) s-dup)
107− ==
108−```
109−This arm is used to test whether the set intersection functionality works as expected. Let's walk through it line by line.
110−```hoon
111−++ test-set-int ^- tang
112− =/ s-nul=(set @) *(set @)
113− =/ s-asc=(set @) (sy (gulf 1 7))
114− =/ s-des=(set @) (sy (flop (gulf 1 7)))
115− =/ s-dos=(set @) (sy (gulf 8 9))
116− =/ s-dup (sy ~[1 1 4 1 3 5 9 4])
117−```
118−We see that this arm returns a `tang`, i.e. a pretty-printed message, and adds several sets to the subject which will be utilized in our tests.
119−```hoon
120−;: weld
121−```
122−`;: weld` calls `weld`, which is typically a binary function, as an n-ary function. This will allow us to concatenate all of `tang`s that are to be generated into a single `tang`.
123−```hoon
124− :: Checks with empty set
125− ::
126− %+ expect-eq
127− !> ~
128− !> (~(int in s-nul) s-asc)
129−```
130−`%+` is used to call a gate with a cell sample. `expect-eq` is the gate, `!> ~` is the head of the sample, and `!> (~(int in s-nul) s-asc)` is the tail of the sample. `!>` is a rune used to wrap a [noun](/docs/glossary/noun/) in its type - in other words, it produces a `vase`. We've seen this rune before when we learned about the type spear `-:!>`. Thus, what this block of code is doing is checking whether the expected `vase` that is the product of `!> ~` is equal to the `vase` that is the product of `!> (~(int in s-nul) s-asc)`. `s-nul` is simply the empty set, and here we are taking the intersection of the empty set with `s-asc`, a non-empty set. This ought to produce the empty set of course, and so when this block of code runs, we compare the expected result `!> ~` (which is something we determined by hand) with the actual result `!> (~(int in s-nul) s-asc)`.
131−```hoon
132− %+ expect-eq
133− !> ~
134− !> (~(int in s-asc) s-nul)
135−```
136−This is another test to check if intersection works with the empty set, but we have swapped where `s-asc` and `s-nul` are. We do not know a priori whether this ordering matters (and of course it shouldn't), which is why it is important to perform both tests.
137−
138−
139−```hoon
140− :: Checks with all elements different
141− ::
142− %+ expect-eq
143− !> ~
144− !> (~(int in s-dos) s-asc)
145−```
146−Now we are checking if intersection works correctly when two non-empty sets share no common elements. We expect the intersection to be empty, so our expected `vase` is `!> ~`, and we then compare it with what the set library produces when it takes the intersection of `s-dos` and `s-asc`.
147−
148−```hoon
149− :: Checks success (total intersection)
150− ::
151− %+ expect-eq
152− !> s-asc
153− !> (~(int in s-asc) s-des)
154−```
155−`s-asc` and `s-des` both ought to be the set containing the integers 1 through 7, but `s-asc` was generated with the list `~[1,2,3,4,5,6,7]` while `s-des` was generated with the list `~[7,6,5,4,3,2,1]`. Sets have no ordering, so they ought to be the same. Thus, when we take their intersection, we should get the same set back. So, our expected `vase` is `!> s-asc` and our actual `vase` is `!> (~(int in s-asc) s-des)`.
156−
157−```hoon
158− :: Checks success (partial intersection)
159− ::
160− %+ expect-eq
161− !> (sy ~[9])
162− !> (~(int in s-dos) s-dup)
163−```
164−Our last test in this arm is to check whether partial intersection works. Here, we compute the intersection of `s-dos` and `s-dup`. Our expectation is that this should return the set whose only element is 9, and so our expected vase is `!> (sy ~[9])`. Then our actual vase is given by actually computing this intersection: `!> (~(int in s-dos) s-dup)`.
165−
166−We invite you to look further into the source of `/tests/sys/hoon/set.hoon` to see how other tests are written.
167−
168−## Running many tests
169−
170−When we ran the command `+test /sys/hoon/set` in the Dojo, what ultimately happened is that Ford looked inside of `/tests/sys/hoon/set.hoon` and found each of the tests in there and performed them in sequence. However, it is not necessary to put all tests you would like to run in a single source file, or even in a single folder. What `+test` actually does is take in a path and then performs all tests located in that folder (if it is a folder) and then looks for additional folders inside of the given path to find additional tests.
171−
172−In other words, if you want to run all of the tests located in `/tests/sys/hoon/`, you would run `+test /sys/hoon`:
173−
174−```
175−> +test /sys/hoon
176−OK /sys/hoon/auras/test-parse-p
177−OK /sys/hoon/auras/test-parse-q
178−OK /sys/hoon/auras/test-render-p
179−OK /sys/hoon/auras/test-render-q
180−OK /sys/hoon/bits/test-bits
181−OK /sys/hoon/differ/test-berk
182−OK /sys/hoon/differ/test-loss
183−OK /sys/hoon/differ/test-lurk
184−OK /sys/hoon/differ/test-lusk
185−OK /sys/hoon/hashes/test-mug
186−OK /sys/hoon/hashes/test-muk
187−OK /sys/hoon/map/test-map-all
188−OK /sys/hoon/map/test-map-any
189−...
190−```
191−We have truncated the results here, but you can already see that this command is performing tests found in subfolders of `/tests/sys/hoon`. This command runs quicky, but if you ran e.g. `+test /sys` you may be waiting quite awhile.