urbit/pub/docs/dev/hoon/tutorial/2-syntax.md

# Hoon 2: serious syntax

We've done a bunch of fun stuff on the command line.  We know our
nouns.  It's time to actually write some serious code -- in a
real source file.

## Building a simple generator

In Urbit there's a variety of source file roles, distinguished by
the magic paths they're loaded from: `/gen` for generators,
`/ape` for appliances, `/fab` for renderers, etc.

We'll start with a generator, the simplest kind of Urbit program.

### Create a sandbox desk

A desk is the Urbit equivalent of a `git` branch.  We're just
playing around here and don't intend to soil our `%home` desk with
test files, so let's make a sandbox:
```
|merge %sandbox our %home
```
### Mount the sandbox

Your Urbit pier is in `~/tasfyn-partyv`, or at least mine is.
So we can get our code into Urbit, run the command 
```
~tasfyn-partyv:dojo> |mount /=sandbox=/gen %gen
```
mounts the `/gen` folder from the `%sandbox` desk in your Unix
directory  `~/tasfyn-partyv/gen`.  The mount is a two-way sync,
like your Dropbox.  When you edit a Unix file and save, your edit
is automatically committed as a change to `%sandbox`.

### Execute from the sandbox

The `%sandbox` desk obviously is merged from `%home`, so it
contains find all the default facilities you'd expect there.
Bear in mind, we didn't set it to auto-update when `%home` 
is updated (that would be `|sync` instead of `|merge`).

So we're not roughing it when we set the dojo to load from
`%sandbox`:
```
[switch to %home]
```

### Write your builder

Let's build the simplest possible kind of generator, a builder.
With your favorite Unix text editor (there are Hoon modes for vim
and emacs), create the file `~/tasfyn-partyv/gen/test.hoon`.
Edit it into this:
```
:-  %say  |=  *  :-  %noun
[%hello %world]
```
Get the spaces exactly right, please.  Hoon is not in general a
whitespace-sensitive language, but the difference between one
space and two-or-more matters.  And for the moment, think of
```
:-  %say  |=  *  :-  %noun
```
as gibberish boilerplate at the start of a file, like `#include
"stdio.h"` at the start of a C program.   Any of our old Hoon
constants would work in place of `[%hello %world`].

Now, run your builder:
```
~tasfyn-partyv:dojo/sandbox> +test
[%hello %world]
```
Obviously this is your first Hoon *program* per se.  

## Hoon syntax 101

But what's up with this syntax?

### A syntactic apology

The relationship between ASCII and human programming languages
is like the relationship between the electric guitar and
rock-and-roll.  If it doesn't have a guitar, it's not rock.
Some great rockers play three chords, like Johnny Ramone; some
shred it up, like Jimmy Page.

The two major families of ASCII-shredding languages are Perl and
the even more spectacular APL.  (Using non-ASCII characters is
just a fail, but APL successors like J fixed this.)  No one
has any right to rag on Larry Wall or Ken Iverson, but Hoon, 
though it shreds, shreds very differently.

The philosophical case for a "metalhead" language is threefold.
One, human beings are much better at associating meaning with
symbols than they think they are.  Two, a programming language is
a professional tool and not a plastic shovel for three-year-olds.

And three, the alternative to heavy metal is keywords.  When you
use a keyword language, not only are you forcing the programmer
to tiptoe around a ridiculous maze of restricted words used and
reserved, you're expressing your program through two translation
steps: symbol->English and English->computation.  When you shred,
you are going direct: symbol->computation.  Especially in a pure
language, this creates a sense of "seeing the function" which no
keyword language can quite duplicate.

But any metalhead language you don't yet know is line noise.
Let's get you up to speed as fast as possible.

### A glyphic bestiary

A programming language needs to be not just read but said.  But
no one wants to say "ampersand."  Therefore, we've taken the
liberty of assigning three-letter names to all ASCII glyphs.

Some of these bindings are obvious and some aren't.  You'll be
genuinely surprised at how easy they are to remember:
```
    ace [1 space]  dot .               pan ]
    bar |          fas /               pel )
    bis \          gap [>1 space, nl]  pid }
    buc $          hax #               ran >
    cab _          ket ^               rep '
    cen %          lep (               sac ;
    col :          lit <               tar *
    com ,          lus +               tec `
    das -          mat @               tis =
    den "          med &               wut ?
    dip {          nap [               zap !
```
It's fun to confuse people by using these outside Urbit.  A few
digraphs also have irregular sounds:
```
==  stet
--  shed
++  slus
->  dart
-<  dusk
+>  lark
+<  lush
```

### The shape of a twig

A twig, of course, is a noun.  As usual, the easiest way to
explain both the syntax that compiles into that noun, and the
semantic meaning of the noun, is the noun's physical structure.

#### Autocons

A twig is always a cell, and any cell of twigs is a twig
producing a cell.  As an homage to Lisp, we call this
"autocons."  Where you'd write `(cons a b)` in Lisp, you write
`[a b]` in Hoon, and the shape of the twig follows.

The `???` prefix prints a twig as a noun instead of running it.
Let's see autocons in action:
```
~tasfyn-partyv:dojo/sandbox> ??? 42
[%dtzy %ud 42]
~tasfyn-partyv:dojo/sandbox> ??? 0x2a
[%dtzy %ux 42]
~tasfyn-partyv:dojo/sandbox> ??? [42 0xa]
[[%dtzy %ud 42] %dtzy %ux 42]
```
(As always, it may confuse *you* that this is the same noun as
`[[%dtzy %ud 42] [%dtzy %ux 42]]`, but it doesn't confuse Hoon.)

#### The stem-bulb pattern

If the head of your twig is a cell, it's an autocons.  If the
head is an atom, it's an unpronounceable four-letter symbol like
the `%dtzy` above.

This is the same pattern as we see in the `span` mold -- a
variant record, essentially, in nouns.  The head of one of these
cells is called the "stem."  The tail is the "bulb."  The shape
of the bulb is totally dependent on the value of the stem.

#### Runes and stems

A "rune" (a word intentionally chosen to annoy Go programmers) is
a digraph - a sequence of two ASCII glyphs.  If you know C, you
know digraphs like `->` and `?:` and are used to reading them as
single characters.

In Hoon you can *say* them as words: "dasran" and "wattis"
respectively.  In a metalhead language, if we had to say 
"minus greater-than" and "question-colon", we'd just die.

Most twig stems are made from runes, by concatenating the glyph
names and removing the vowels.  For example, the rune `=+`,
pronounced "tislus," becomes the stem `%tsls`.  (Note that in
many noun implementations, this is a 31-bit direct value.)

(Some stems (like `%dtzy`) are not runes, simply because they
don't have regular-form syntax and don't need to use precious
ASCII real estate.  They are otherwise no different.)

An important point to note about runes: they're organized.  The
first glyph in the rune defines a category.  For instance, runes
starting with `.` compute intrinsics; runes starting with `|`
produce cores; etc.

Another important point about runes: they come in two flavors,
"natural" (stems interpreted directly by the compiler) and
"synthetic" (macros, essentially).

(Language food fight warning: one advantage of Hoon over Lisp is
that all Hoon macros are inherently hygienic.  Another advantage
is that Hoon has no (user-level) macros.  In Hoon terms, nobody 
gets to invent their own runes.  A DSL is always and everywhere
a write-only language.  Hoon shreds its ASCII pretty hard, but
the same squiggles mean the same things in everyone's code.)

#### Wide and tall regular forms

A good rune example is the simple rune `=+`, pronounced "tislus",
which becomes the stem `%tsls`.  A `%tsls` twig has the shape
`[%tsls twig twig]`.

The very elegance of functional languages creates a visual
problem that imperative languages lack.  An imperative language
has distinct statements (with side effects) and (usually pure)
expressions; it's natural that in most well-formatted code,
statements flow vertically down the screen, and expressions grow
horizontally across this.  This interplay creates a natural and
relaxing shape on your screen.

In a functional language, there's no difference.  The trivial
functional syntax is Lisp's, which has two major problems.  One:
piles of expression terminators build up at the bottom of complex
functions.  Two: the natural shape of code is diagonal.  The more
complex a function, the more it wants to besiege the right
margin.  The children of a node have to start to the right of its
parent, so the right margin bounds the tree depth.

Hoon does not completely solve these problems, but alleviates
them.  In Hoon, there are actually two regular syntax forms for
most twig cases: "tall" and "wide" form.  Tall twigs can contain
wide twigs, but not vice versa, so the visual shape of a program
is very like that of a statements-and-expressions language.
 
Also, in tall mode, most runes don't need terminators.  Take
`=+`, for example.  Since the parser knows to expect exactly 
two twigs after the `=+` rune, it doesn't need any extra syntax
to tell it that it's done.

Let's try a wide `=+` in the dojo:
```
~tasfyn-partyv:dojo/sandbox> =+(planet=%world [%hello planet])
[%hello %world]
```
(`=+` seems to be some sort of variable declaration?  Let's not
worry about it right now.  We're on syntax.)

The wide syntax for a `=+` twig, or any binary rune: `(`, the 
first subtwig, one space, the second subtwig, and `)`).  To read
this twig out loud, you'd say:
```
tislus lap planet is cen world ace nep cen hello ace planet pen
pal
```
("tis" not in a rune gets contracted to "is".)

Let's try a tall `=+` in `test.hoon`: 
```
:-  %say  |=  *  :-  %noun
=+  planet=%world
[%hello planet]
```
The tall syntax for a `=+` twig, or any binary rune: the rune, at 
least two spaces or one newline, the first subtwig, at least two 
spaces or one newline, the second subtwig.  Again, tall subtwigs
can be tall or wide; wide subtwigs have to be wide.

(Note that our boilerplate line is a bunch of tall runes on one
line, with two-space gaps.  This is unusual but quite legal, and
not to be confused with the actual wide form.)

To read this twig out loud, you'd say:
```
tislus gap planet is cen world gap nep cen hello ace planet pen
```
#### Layout conventions

Should you use wide twigs or tall twigs?  When?  How?  What
should your code look like?  You're the artist.  Except for the
difference between one space (`ace`) and more space (`gap`), the
parser doesn't care how you format your code.  Hoon is not Go --
there are no fixed rules for doing it right.

However, the universal convention is to keep lines under 80
characters.  Also, hard tab characters are illegal.  And when in
doubt, make your code look like the kernel code.

##### Backstep indentation

Note that the "variable declaration" concept of `=+` (which is no
more a variable declaration than a Tasmanian tiger is a tiger)
works perfectly here.  Because `[%hello planet]` -- despite being
a subtree of the the `=+` twig -- is at the same indent level.
So our code flows down the screen, not down and to the right, and
of course there are no superfluous terminators.  It looks good,
and creates fewer hard-to-find syntax errors than you'd think.

This is called "backstep" indentation.  Another example, using a
ternary rune that has a strange resemblance to C:
```
:-  %say  |=  *  :-  %noun
=+  planet=%world
?:  =(%world planet)
  [%hello planet]
[%goodbye planet]
```
It's not always the case when backstepping that the largest
subtwig is at the bottom and loses no margin, but it often is.
And not all runes have tuple structure; some are n-ary, and use
the `==` terminator (again, pronounced "stet"):
```
:-  %say  |=  *  :-  %noun
=+  planet=%world
?+  planet
          [%unknown planet]
  %world  [%hello planet]
  %ocean  [%goodbye planet]
==
```
So we occasionally lose right-margin as we descend a deep twig.
But we can keep this lossage low with good layout design.  The
goal is to keep the heavy twigs on the right, and Hoon tries as
hard as possible to help you with this.

For instance, `=+` ("tislus") is a binary rune: `=+(a b)`.  In
most cases of `=+` the heavy twig is `b`, but sometimes it's `a`.
So we can use its friend the `=-` rune ("tisdas") to get the same
semantics with the right shape: `=-(b a)`.

#### Irregular forms

There are more regular forms than we've shown above, but not a
lot more.  Hoon would be quite easy to learn if it was only its
regular forms.  It wouldn't be as easy to read or use, though.
The learning curve is important, but not all-important.

Some stems (like the `%dtzy` constants above) obviously don't and
can't have any kind of regular form (which is why `%dtzy` is not
a real digraph rune).  Many of the true runes have only regular
forms.  But some have irregular forms.  Irregular forms are 
always wide, but there is no other constraint on their syntax.

We've already encountered one of the irregular forms: `foo=42`
from the last chapter, and `planet=%world` here.  Let's unpack
this twig:
```
~tasfyn-partyv:dojo/sandbox> ?? %world
  [%cube 431.316.168.567 %atom %tas]
%world

~tasfyn-partyv:dojo/sandbox> ??? %world
[%dtzz %tas 431.316.168.567]
```
Clearly, `%dtzz` is one of our non-regulars.  But we can wrap it
with our irregular form:
```
~tasfyn-partyv:dojo/sandbox> ?? planet=%world
  [%face %planet [%cube 431.316.168.567 %atom %tas]]
planet=%world

~tasfyn-partyv:dojo/sandbox> ??? planet=%world
[%ktts %planet %dtzz %tas 431.316.168.567]
```
Since `%ktts` is "kettis", ie, `^=`, this has to be the irregular
form of
```
~tasfyn-partyv:dojo/sandbox> ^=(planet %world)
planet=world
```
So if we wrote our example without this irregular form, it'd be
```
:-  %say  |=  *  :-  %noun
=+  ^=(planet %world)
[%hello planet]
```
Or with a gratuitous use of tall form:
```
:-  %say  |=  *  :-  %noun
=+  ^=  planet  %world
[%hello planet]
```
Now you know how to read Hoon!  For fun, try to pronounce more of
the code on this page.  Please don't laugh too hard at yourself.