@@ -24,37 +24,38 @@ Parsing a string is rarely done all at once. Instead, it is usually done
2424character-by-character, and the return contains the data structure representing
2525what has been parsed thus far as well as the remainder of the string to be
2626parsed. They also need to be able to fail in case the input is improperly
27−formed. We will see each of these standard practices implemented in Hoon below.
27+formed. We will see each of these standard practices implemented in [Hoon below](#parsing-in-hoon).
2828
2929## Functional parsers
3030
3131How parsers are built varies substantially depending on what sort of programming
3232language it is written in. As Hoon is a functional programming language, we will
3333be focused on understanding _functional parsers_, also known as _combinator
34−parsers_.
34+parsers_. In this section we will make light use of pseudocode, as introducing
35+the Hoon to describe functional parsers here creates a chicken-and-egg problem.
3536
3637Complex functional parsers are built piece by piece from simpler parsers
3738that are plugged into one another in various ways to perform the desired task.
3839
3940The basic building blocks, or primitives, are parsers that read only a
4041single character. There are frequently a few types of possible input characters,
41−such as letters, numbers, and symbols. For example, `parse(integer, "1")` calls
42+such as letters, numbers, and symbols. For example, `parse("1", integer)` calls
4243the parsing routine on the string `"1"` and looks for an integer, and so it
4344returns the integer `1`. However, taking into account what was said above about
4445parsers returning the unparsed portion of the string as well, we should
4546represent this return as a tuple. So we should expect something like this:
4647```
47−> parse(integer, "1")
48+> parse("1", integer)
4849(1, "")
49−> parse(integer, "123")
50+> parse("123", integer)
5051(1, "23")
5152```
5253What if we wish to parse the rest of the string? We would need to apply the
53−`parse(integer, -)` function again:
54+`parse` function again:
5455```
55−> parse(integer, parse(integer, "123"))
56+> parse(parse("123", integer), integer)
5657(12, "3")
57−> parse(integer, parse(integer, parse(integer, "123")))
58+> parse(parse(parse("123", integer), integer), integer)
5859(123, "")
5960```
6061So we see that we can parse strings larger than one character by stringing
@@ -76,8 +77,6 @@ are quite a few), but ought to be
7677sufficient to get started with parsing in Hoon and be equipped to discover the
7778remainder yourself.
7879
79−We encourage you to follow along with this section by inputting the relevant
80−commands in dojo as they are presented.
8180
8281## Basic types
8382
@@ -94,7 +93,10 @@ In this section we discuss the types most commonly used for Hoon parsers. In sho
9493++ hair [p=@ud q=@ud]
9594```
9695
97−A `hair` is a pair of `@ud` used to keep track of what has already been parsed.
96+A `hair` is a pair of `@ud` used to keep track of what has already been parsed
97+for stack tracing purposes. This allows the parser to reveal where the problem
98+is in case it hits something unexpected during parsing.
99+
98100`p` represents the column and `q` represents the line.
99101
100102### `nail`
@@ -111,20 +113,23 @@ the original input string (i.e. everything after the location indicated by the
111113
112114For example, if you wish to feed the entire `tape` `"abc"` into a parser, you
113115would pass it as the `nail` `[[1 1] "abc"]`. If the parser successfully parses the first
114−character, the `nail` it returns will be `[[2 1] "bc"]` (though we note that
116+character, the `nail` it returns will be `[[1 2] "bc"]` (though we note that
115117parser outputs are actually `edge`s which contain a `nail`, see the following).
118+The `nail` only matters for book-keeping reasons - it could be any value here
119+since it doesn't refer to a specific portion of the string being input, but only
120+what has theoretically already been parsed up to that point.
116121
117122### `edge`
118123
119124```hoon
120125++ edge [p=hair q=(unit [p=* q=nail])]
121126```
122−An `edge` is the output of a parser. If the parse succeeded, `p` is the location
123−of the original input `tape `up to which the text has been parsed. If the parse
124−failed, `p` will be the first `hair` at which the parse failed.
127+An `edge` is the output of a parser. If parsing succeeded, `p` is the location
128+of the original input `tape `up to which the text has been parsed. If parsing
129+failed, `p` will be the first `hair` at which parsing failed.
125130
126−`q` may be `~`, indicating that the parse has failed .
127−If the parse did not fail, `p.q` is the data structure that is the result of the
131+`q` may be `~`, indicating that parsing has failed .
132+If parsing did not fail, `p.q` is the data structure that is the result of the
128133parse up to this point, while `q.q` is the `nail` which contains the remainder
129134of what is to be parsed. If `q` is not null, `p` and `p.q.q` are identical.
130135
@@ -134,13 +139,7 @@ of what is to be parsed. If `q` is not null, `p` and `p.q.q` are identical.
134139++ rule _|:($:nail $:edge)
135140```
136141A `rule` is a gate which takes in a `nail` and returns an `edge` - in other
137−words, a parser. Not totally sure why its written like this, though.
138−
139−Maybe mention this from `/app/shoe.hoon` as well?
140−```hoon
141−|~(nail *(like [? command]))
142−```
143−
142+words, a parser.
144143
145144## Parser builders
146145
@@ -164,29 +163,31 @@ of the input `nail`.
164163[p=[p=1 q=2] q=[~ [p='a' q=[p=[p=1 q=2] q="bc"]]]]
165164```
166165We note that `p.edg` is `[p=1 q=2]`, indicating that the next character to be
167−parsed is in line 1, column 2. `q.edg` is not null, indicating that the parse
166+parsed is in line 1, column 2. `q.edg` is not null, indicating that parsing
168167succeeded. `p.q.edg` is `'a'`, which is the result of the parse. `p.q.q.edg` is the same as `p.edg`, which is always the case for
169−`rule`s built using standard library functions when the parse succeeds. Lastly,
168+`rule`s built using standard library functions when parsing succeeds. Lastly,
170169`q.q.edg` is `"bc"`, which is the part of the input `tape` that has yet to be parsed.
171170
172−Now let's see what happens when the parse fails.
171+Now let's see what happens when parsing fails.
173172```
174173> =edg ((just 'b') [[1 1] "abc"])
175174> edg
176175[p=[p=1 q=1] q=~]
177176```
178177Now we have that `p.edg` is the same as the input `hair`, `[1 1]`, meaning the
179−parser has not advanced since the parse failed. `q.edg` is null, indicating that
180−the parse has failed.
178+parser has not advanced since parsing failed. `q.edg` is null, indicating that
179+parsing has failed.
181180
182181Later we will use [+star](#star) to string together a sequence of `+just`s in
183−order to parse multiple characters at once.
182+order to parse multiple characters at once. However, it should be noted that
183+`+jest`, which is covered next, ought to be used in favor of `+just`, which is
184+mostly included here for pedagogical reasons.
184185
185186### `+jest`
186187
187188`+jest` is a `rule` builder used to match a `cord`. It takes an input `cord` and
188189produces a `rule` that attempts to match that `cord` against the beginning of
189−the `tape` in an input `nail`.
190+the input.
190191
191192Let's see what happens when we successfully parse the entire input `tape`.
192193```
@@ -236,7 +237,7 @@ and returns a `rule`.
236237
237238`+cold` is a `rule` builder that takes in a constant noun we'll call `cus` and a
238239`rule` we'll call `sef` and returns a `rule` identical to the `sef` except it
239−returns `cus` when it parses successfully.
240+replaces the parsing result with `cus`.
240241
241242Here we see that `p.q` of the `edge` returned by the `rule` created with `+cold`
242243is `%foo`.
@@ -268,13 +269,15 @@ themselves. Thus we have several functions for exactly this purpose.
268269
269270These functions take in either a `tape` or an `@` (typically representing a
270271`cord`) with a `rule` and attempts to parse the input with the `rule`. If the
271−parse succeeds, it returns the result, otherwise it crashes with a stack trace or returns null.
272+parse succeeds, it returns the result. There are crashing and unitized versions
273+of each caller, corresponding to what happens when a parse fails.
272274
273−Here we cover the basics, for complete information including additional examples see [4g: Parsing (Outside Caller)](@/docs/reference/library/4g.md).
275+For additional information including additional examples see [4g: Parsing (Outside Caller)](@/docs/reference/library/4g.md).
274276
275277### Parsing `tape`s
276278
277−`+scan` takes in a `tape` and a `rule` and attempts to parse the `tape` with the `rule`.
279+`+scan` takes in a `tape` and a `rule` and attempts to parse the `tape` with the
280+`rule`.
278281
279282```
280283> (scan "hello" (jest 'hello'))
@@ -284,8 +287,7 @@ Here we cover the basics, for complete information including additional examples
284287'syntax-error'
285288```
286289
287−`+rust` works the same way as `+scan`, except it returns a `unit` of the result,
288−which is null if the parse failed.
290+`+rust` is the unitized version of `+scan`.
289291
290292```
291293> (rust "a" (just 'a'))
@@ -297,12 +299,38 @@ which is null if the parse failed.
297299For the remainder of this tutorial we will make use of `+scan` so that we do not
298300need to deal directly with `nail`s except where it is ilustrative to do so.
299301
302+
303+### Parsing atoms
304+
305+[Recall](@/docs/tutorials/hoon/lists.md) that `cord`s are atoms with the aura
306+`@t` and are typically used to represent strings internally as data, as atoms
307+are faster for the computer to work with than `tape`s, which are `list`s of
308+`@tD` atoms. `+rash` and `+rush` are for parsing atoms, with `+rash` being
309+analogous to `+scan` and `+rush` being analogous to `rust`. Under the hood, `+rash`
310+calls `+scan` after converting the input atom to a `tape`, and `+rush` does
311+similary for `+rust`.
312+
300313## Parser modifiers
301314
302315The standard library provides a number of gates that take a `rule` and
303316produce a new modified `rule` according to some process. We call these _parser modifiers_. These are
304317documented among the [parser builders](@/docs/reference/library/4f.md).
305318
319+
320+### `+ifix`
321+
322+`+ifix` modifies a `rule` so that it matches that `rule` only when it is
323+surrounded on both sides by text that matches a pair of `rule`s, which is discarded.
324+
325+```
326+> (scan "(42)" (ifix [lit rit] (jest '42')))
327+'42'
328+```
329+`+lit` and `+rit` are shorthand for `(just '(')` and `(just ')')`, respectively. All
330+ASCII glyphs have counterparts of this sort, documented
331+[here](@/docs/reference/library/4h.md).
332+
333+
306334### `+star` {#star}
307335
308336`+star` is used to apply a `rule` repeatedly. Recall that `+just` only parses
@@ -325,20 +353,6 @@ and we note that the parsing ceases when it fails.
325353[p=[p=1 q=4] q=[~ [p=[i='a' t=<|a a|>] q=[p=[p=1 q=4] q="b"]]]]
326354```
327355
328−### `+ifix`
329−
330−`+ifix` modifies a `rule` so that it matches that `rule` only when it is
331−surrounded on both sides by text that matches a pair of `rule`s, which is discarded.
332−
333−```
334−> (scan "(42)" (ifix [lit rit] (jest '42')))
335−'42'
336−```
337−`+lit` and `+rit` are shorthand for `(just '(')` and `(just ')')`, respectively. All
338−ASCII glyphs have counterparts of this sort, documented
339−[here](@/docs/reference/library/4h.md).
340−
341−
342356## Parser combinators
343357
344358Building complex parsers from simpler parsers is accomplished in Hoon with the
@@ -349,7 +363,7 @@ combinators, then we examine more closely how `;~` is used to chain them togethe
349363
350364The syntax to combine `rule`s is
351365```hoon
352−;~(combinator [list rule])
366+;~(combinator rule1 rule2 ... ruleN)
353367```
354368The `rule`s are composed together using the combinator as an
355369intermediate function, which takes product of a `rule` (an `edge`) and a `rule` and turns
@@ -433,15 +447,7 @@ is more than one `rule`.
433447syntax error
434448```
435449
436−## Parsing atoms
437450
438−[Recall](@/docs/tutorials/hoon/lists.md) that `cord`s are atoms with the aura
439−`@t` and are typically used to represent strings internally as data, as atoms
440−are faster for the computer to work with than `tape`s, which are `list`s of
441−`@tD` atoms. `+rash` and `+rush` are for parsing atoms, with `+rash` being
442−analogous to `+scan` and `+rush` being analogous to `rust`. Under the hood, `+rash`
443−calls `+scan` after converting the input atom to a `tape`, and `+rush` does
444−similary for `+rust`.
445451
446452## Parsing numbers
447453
@@ -462,7 +468,7 @@ Naively attempting to write a recursive `rule`, i.e. like
462468```
463469> |-(;~(pose den ;~(pose $ (easy ~))))
464470```
465−results in an error, as the compiler will attempt to build an infinite function.
471+results in an error.
466472```
467473-find.,.+6
468474-find.,.+6
@@ -475,12 +481,17 @@ as the bunt value of the type that the `rule` produces, as well as a gate that
475481accepts a `rule`. `+knee` produces a `rule` that implements any recursive calls
476482in the `rule` in a manner acceptable to the compiler. Thus the preferred manner
477483to write the above `rule` is as follows:
484+
478485```hoon
479−|-(;~(plug prn ;~(pose (knee *tape |.(^$)) (easy ~))))
486+++ pars
487+ |-
488+ ;~ plug prn
489+ ;~ pose knee *tape
490+ |. ^$ (easy ~)
480491```
481492
482493You may want to utilize the `~+` rune when writing recursive parsers to cache
483−the results as you go to improve performance. In [parsing arithmetic
494+the parser to improve performance. In [parsing arithmetic
484495expressions](#parsing-arithmetic-expressions) we will be writing a recursive
485496parser making use of `+knee` and `~+` throughout.
486497
@@ -492,7 +503,7 @@ arithmetic expressions in Hoon. That is, we will make a `rule` that takes in
492503`tape`s of the form `"(2+3)*4"` and returns `20` as a `@ud`.
493504
494505We call a `tape` consisting of some consistent arithmetic string of numbers,
495−`+,*`, `(`, and `)` an _expression_. We wish to build a `rule` that takes in an
506+`+`, `*`, `(`, and `)` an _expression_. We wish to build a `rule` that takes in an
496507expression and returns the result of the arithmetic computation described by the
497508expression as a `@ud`.
498509
@@ -505,31 +516,28 @@ First lets look at the code we're going to use, and then dive into explaining
505516it. If you'd like to follow along, save the following as `expr-parse.hoon` in
506517your `gen/` folder.
507518```hoon
508−:: This generator parses arithmetic expressions
519+:: expr-parse: parse arithmetic expressions
509520::
510−|= [xprs=tape]
511−=<
512−(print xprs)
513−|%
514−++ int @ud
515−++ print
516− |= [message=tape]
517− (scan message expr)
521+|= xprs=tape
522+|^ (scan xprs tape)
518523++ factor
519− %+ knee *int
520− |. ~+ ;~ pose
524+ %+ knee @ud
525+ |. ~+
526+ ;~ pose
521527 dem
522528 (ifix [lit rit] expr)
523529 ==
524530++ turm
525− %+ knee *int
526− |. ~+ ;~ pose
531+ %+ knee @ud
532+ |. ~+
533+ ;~ pose
527534 ((slug mul) tar ;~(pose factor turm))
528535 factor
529536 ==
530537++ expr
531− %+ knee *int
532− |. ~+ ;~ pose
538+ %+ knee @ud
539+ |. ~+
540+ ;~ pose
533541 ((slug add) lus ;~(pose turm expr))
534542 turm
535543 ==
@@ -545,7 +553,7 @@ for expressions.
545553
546554```hoon
547555++ factor
548− %+ knee *int
556+ %+ knee @ud
549557 |. ~+ ;~ pose
550558 dem
551559 (ifix [lit rit] expr)
@@ -562,26 +570,24 @@ with `+pose`, which says to try each rule in succession until one of them works.
562570
563571Since expressions ultimately reduce to factors, we are actually building a
564572recursive rule. Thus we need to make use of `+knee`. The first argument for
565−`+knee` is `*int`, since our final answer should be a `@ud`, to which we have
566−assigned the macro `int`.
573+`+knee` is `@ud`, since our final answer should be a `@ud`.
567574
568575Then follows the definition of the gate utilized by `+knee`:
569576```hoon
570− |. ~+ ;~ pose
577+ |. ~+
578+ ;~ pose
571579 dem
572580 (ifix [lit rit] expr)
573581 ==
574582```
575−`|.` is used instead of `|-` since... `~+` is used to cache results as we go -
576−there is no point in computing `2+2` more than once. Then follows the `rule` we
577−described above.
583+`~+` is used to cache the parser, so that it does not need to be computed over
584+and over again. Then it follows the `rule` we described above.
578585
579586#### Parsing expressions
580587
581588An _expression_ is either a term plus an expression or a term.
582589
583−In the case of a
584−term plus an expression, we actually must compute what that equals. Thus we will
590+In the case of a term plus an expression, we actually must compute what that equals. Thus we will
585591make use of [`+slug`](@/docs/reference/library/4f.md#slug), which parses a
586592delimited list into `tape`s separated by a given delimiter and then composes
587593them by folding with a binary gate. In this case, our delimiter is `+` and our
@@ -590,15 +596,15 @@ and expressions separated by luses, parse each term and expression until they
590596reduce to a `@ud`, and then add them together. This is accomplished with the
591597rule `((slug add) lus ;~(pose turm expr))`.
592598
593−If the above `rule` does not parse the expression, it must be a `turm`, so we
594−pass the `tape` to `+turm` to be evaluated. Again we use `;~` and `pose` to
599+If the above `rule` does not parse the expression, it must be a `turm`, so the
600+`tape` is automatically passed to `+turm` to be evaluated. Again we use `;~` and `pose` to
595601accomplish this:
596602
597603```hoon
598604;~ pose
599− ((slug add) lus ;~(pose turm expr))
600− turm
601− ==
605+ ((slug add) lus ;~(pose turm expr))
606+ turm
607+==
602608```
603609
604610The rest of the `+expr` arm is structured just like how `+factor` is, and for
@@ -609,7 +615,7 @@ the same reasons.
609615
610616A _term_ is either a factor times a term or a factor. This is handled similarly
611617for expressions, we just need to swap `lus` for `tar`, `add` for `mul`, and
612−`;~(pose factor turm)` instead of `;(pose turm expr)`.
618+`;~(pose factor turm)` instead of `;~(pose turm expr)`.
613619
614620```hoon
615621++ expr