on master help sign in
checkpoint
fffb204JONATHAN PAPROCKI
tutorials/arvo/parsing.md+88−19
@@ -165,9 +165,9 @@ Glyphs)](@/docs/reference/library/4h.md), [4i: Parsing (Useful
165Idioms)](@/docs/reference/library/4i.md), [4j: Parsing (Bases and Base
165Idioms)](@/docs/reference/library/4i.md), [4j: Parsing (Bases and Base
166Digits)](@/docs/reference/library/4j.md), [4l: Atom Parsing](@/docs/reference/library/4l.md).
166Digits)](@/docs/reference/library/4j.md), [4l: Atom Parsing](@/docs/reference/library/4l.md).
167
167
168−### `just`
168+### `+just`
169
169
170−The most basic rule builder, `just` takes in a single `char` and produces a
170+The most basic rule builder, `+just` takes in a single `char` and produces a
171`rule` that attempts to match that `char` to the first character in the `tape`
171`rule` that attempts to match that `char` to the first character in the `tape`
172of the input `nail`.
172of the input `nail`.
173
173
@@ -192,13 +192,11 @@ Now we have that `p.edg` is the same as the input `hair`, `[1 1]`, meaning the
192parser has not advanced since the parse failed. `q.edg` is null, indicating that
192parser has not advanced since the parse failed. `q.edg` is null, indicating that
193the parse has failed.
193the parse has failed.
194
194
195−In [parsing arithmetic expressions](#parsing-arithmetic-expressions) we will see
196−how to string together a sequence of `just`s in order to parse an entire string.
195+Later we will use [+star](#star) to string together a sequence of `+just`s in order to parse an entire string.
197−(maybe in the next section instead? seems like a good example for `;~`)
198
196
199−### `jest`
197+### `+jest`
200
198
201−`jest` is a `rule` builder used to match a `cord`. It takes an input `cord` and
199+`+jest` is a `rule` builder used to match a `cord`. It takes an input `cord` and
202produces a `rule` that attempts to match that `cord` against the beginning of
200produces a `rule` that attempts to match that `cord` against the beginning of
203the `tape` in an input `nail`.
201the `tape` in an input `nail`.
204
202
@@ -222,23 +220,30 @@ What happens if we only match some of the input `tape`?
222[p=[p=1 q=3] q=[~ [p='ab' q=[p=[p=1 q=3] q="c"]]]]
220[p=[p=1 q=3] q=[~ [p='ab' q=[p=[p=1 q=3] q="c"]]]]
223```
221```
224Now we have that the result, `p.q.edg`, is `'ab'`, while the remainder `q.q.q.edg`
222Now we have that the result, `p.q.edg`, is `'ab'`, while the remainder `q.q.q.edg`
225−is `"c"`. So `jest` has successfully parsed the first two characters, while the
223+is `"c"`. So `+jest` has successfully parsed the first two characters, while the
226last character remains. Furthermore, we still have the information that the
224last character remains. Furthermore, we still have the information that the
227remaining character was in line 1 column 3 from `p.edg` and `p.q.q.edg`.
225remaining character was in line 1 column 3 from `p.edg` and `p.q.q.edg`.
228
226
229−What happens when `jest` fails?
227+What happens when `+jest` fails?
230```
228```
231> ((jest 'bc') [[1 1] "abc"])
229> ((jest 'bc') [[1 1] "abc"])
232[p=[p=1 q=1] q=~]
230[p=[p=1 q=1] q=~]
233```
231```
234Despite the fact that `'bc'` appears in `"abc"`, because it was not at the
232Despite the fact that `'bc'` appears in `"abc"`, because it was not at the
235−beginning the parse failed. We will see in [basic parser
233+beginning the parse failed. We will see in [parser
236−combinators](#basic-parser-combinators) how to modify this `rule` so that it
234+combinators](#parser-combinators) how to modify this `rule` so that it
237finds `bc` successfully.
235finds `bc` successfully.
238
236
239−## Basic parser combinators
237+### `+shim`
238+
239+`+shim` is used to parse characters within a given range. It takes in two atoms
240+and returns a `rule`.
241+
242+```
243+> ((shim 'a' 'z') [[1 1] "abc"])
244+[p=[p=1 q=2] q=[~ [p='a' q=[p=[p=1 q=2] q="bc"]]]]
245+```
240
246
241−Something about `;~`
242
247
243## Outside callers
248## Outside callers
244
249
@@ -256,7 +261,7 @@ Here we cover the basics, for complete information including additional examples
256
261
257### Parsing `tape`s
262### Parsing `tape`s
258
263
259−`scan` takes in a `tape` and a `rule` and attempts to parse the `tape` with the rule.
264+`+scan` takes in a `tape` and a `rule` and attempts to parse the `tape` with the rule.
260
265
261```
266```
262> (scan "hello" (jest 'hello'))
267> (scan "hello" (jest 'hello'))
@@ -266,7 +271,7 @@ Here we cover the basics, for complete information including additional examples
266'syntax-error'
271'syntax-error'
267```
272```
268
273
269−`rust` works the same way as `scan`, except it returns a `unit` of the result,
274+`+rust` works the same way as `+scan`, except it returns a `unit` of the result,
270which is null if the parse failed.
275which is null if the parse failed.
271
276
272```
277```
@@ -276,17 +281,81 @@ which is null if the parse failed.
276~
281~
277```
282```
278
283
284+For the remainder of this tutorial we will make use of `+scan` so that we do not
285+need to deal directly with `nail`s unles it would be ilustrative to do so.
286+
287+## Parser modifiers
288+
289+The standard library provides a number of gates that take a `rule` and
290+produce a new modified `rule` according to some process. We call these _parser modifiers_. These are
291+documented among the [parser builders](@/docs/reference/library/4f.md).
292+
293+### `+star` {#star}
294+
295+`+star` is used to apply a `rule` repeatedly. Recall that `+just` only parses
296+the first character in the input `tape.`
297+
298+```
299+> ((just 'a') [[1 1] "aaaa"])
300+[p=[p=1 q=2] q=[~ [p='a' q=[p=[p=1 q=2] q="aaa"]]]]
301+```
302+
303+We can use `+star` to get the rest of the `tape`:
304+
305+```
306+> ((star (just 'a')) [[1 1] "aaa"])
307+[p=[p=1 q=4] q=[~ [p=[i='a' t=<|a a|>] q=[p=[p=1 q=4] q=""]]]]
308+```
309+and we note that the parsing ceases when it fails.
310+```
311+> ((star (just 'a')) [[1 1] "aaab"])
312+[p=[p=1 q=4] q=[~ [p=[i='a' t=<|a a|>] q=[p=[p=1 q=4] q="b"]]]]
313+```
314+
315+### `+ifix`
316+
317+`+ifix` modifies a `rule` so that it matches that `rule` only when it is
318+surrounded on both sides by text that matches a pair of `rule`s, which is discarded.
319+
320+```
321+> (scan "(42)" (ifix [lit rit] (jest '42')))
322+'42'
323+```
324+`+lit` and `+rit` are shorthand for `(just '(')` and `(just ')')`, respectively. All
325+ASCII glyphs have counterparts of this sort, documented
326+[here](@/docs/reference/library/4h.md).
327+
328+
329+## Parser combinators
330+
331+Building complex parsers from simpler parsers is accomplished in Hoon with the
332+use of two tools: the monadic applicator rune
333+[`;~`](@/docs/reference/hoon-expressions/rune/mic.md#micsig) and [parsing
334+combinators](@/docs/reference/library/4e.md).
335+
336+The syntax to combine parsers (chain them together) is
337+```hoon
338+;~(combinator rule-1 rule-2 ... rule-n)
339+```
340+The `rule`s are composed together using the combinator as an
341+intermediate function, which takes product of a `rule` (an `edge`) and a `rule` and turns
342+it into a sample (a `nail`) for the next `rule`.
343+
344+
279### Parsing atoms
345### Parsing atoms
280
346
281[Recall](@/docs/tutorials/hoon/lists.md) that `cord`s are atoms with the aura
347[Recall](@/docs/tutorials/hoon/lists.md) that `cord`s are atoms with the aura
282`@t` and are typically used to represent strings internally as data, as atoms
348`@t` and are typically used to represent strings internally as data, as atoms
283are faster for the computer to work with than `tape`s, which are `list`s of
349are faster for the computer to work with than `tape`s, which are `list`s of
284−`@tD` atoms. `rash` and `rush` are for parsing atoms, with `rash` being
350+`@tD` atoms. `+rash` and `+rush` are for parsing atoms, with `+rash` being
285−analogous to `scan` and `rush` being analogous to `rust`. Under the hood, `rash`
351+analogous to `+scan` and `+rush` being analogous to `rust`. Under the hood, `+rash`
286−calls `scan` after converting the input atom to a `tape`, and `rush` does
352+calls `+scan` after converting the input atom to a `tape`, and `+rush` does
287−similary for `rust`.
353+similary for `+rust`.
288
354
289
355
290## Parsing arithmetic expressions
356## Parsing arithmetic expressions
291
357
358+Functions for parsing numbers are documented in [4j: Parsing (Bases and Base
359+Digits)](@/docs/reference/library/4j.md). `dim:ag` is for parsing decimal
360+numbers.
292
361