@@ -7,7 +7,7 @@
7:: ship builds it as /lib/jibb.hoon.
7:: ship builds it as /lib/jibb.hoon.
10−:: +layout [file=path src=@t] (each (list token) error)
10+:: +layout [file=path src=@t] (list token)
11:: +parse [file=path src=@t] (each module error)
11:: +parse [file=path src=@t] (each module error)
12:: +parse-tokens [file=path toks=(list token)] (each module error)
12:: +parse-tokens [file=path toks=(list token)] (each module error)
13:: +golden [file=path src=@t] [tokens=@t tree=@t]
13:: +golden [file=path src=@t] [tokens=@t tree=@t]
@@ -15,13 +15,18 @@
15:: +show-module module @t
15:: +show-module module @t
16:: +show-error [src=@t error] @t
16:: +show-error [src=@t error] @t
17:: A file is a Clay path whose last segment is the extension, as the
17:: A file is a Clay path whose last segment is the extension, as the
18−:: golden runners pass /tests/golden/<case>/source/jibb.
18+:: golden runners pass /tests/golden/<case>/source/jibb. +layout never
19+:: fails: an error it finds is its last token (2.0), and +layout-error
20−:: Status: the types, the printers and +golden work. +layout and
22+:: Status: the types, the printers, +golden and the layout pass
21−:: +parse-tokens are stubs that return an error; they are implemented
23+:: (+layout, +lexer) work. +parse-tokens is a stub that returns an
22−:: next, against the design below. Choices the doc leaves open are marked
24+:: error; it is implemented next, against the design below. Choices the
23−:: "(doc question)": they follow the most likely reading and are listed,
25+:: doc leaves open are marked "(doc question)": they follow the most
24−:: with the doc's words, for the doc's author to settle.
26+:: likely reading and are listed, with the doc's words, for the doc's
27+:: author to settle. The golden corpus's "expect:" lines give each
28+:: error's position. The layout pass's errors sit there, except where
29+:: the parser finds an earlier error first, as in `++ 2nd 5` (2.0, 2.8).
26:: ===================================================================
31:: ===================================================================
27:: 1. Positions
32:: 1. Positions
@@ -38,9 +43,15 @@
38:: has beg = end). Nodes keep their ends because the checker's errors in
43:: has beg = end). Nodes keep their ends because the checker's errors in
39:: milestone 2 name both conflicting sites with excerpts, and an excerpt
44:: milestone 2 name both conflicting sites with excerpts, and an excerpt
40:: that underlines a whole expression, as Elm and Rust do, needs its end;
45:: that underlines a whole expression, as Elm and Rust do, needs its end;
41−:: so does editor tooling later. Ends cost the parser nothing: a node ends
46+:: so does editor tooling later. A node starts at its first token and
42−:: where its last child or closing token ends. The printed tree shows
47+:: ends where its last token ends: its closing token if it has one, else
48+:: its last child's last token, never a layout token. So an open
49+:: bracket, a body, ?-'s arms and $?'s cases end with their last line's
50+:: code, not at the line that closes them. Every node's span holds its
51+:: children's spans, in source order. The printed tree shows starts
43−:: starts only, and milestone 1's errors use starts only.
52+:: only, and milestone 1's errors use starts only, so ends are first
53+:: tested by milestone 2's excerpts; the parser's own tests should check
54+:: the two rules above on every golden tree.
45:: ===================================================================
56:: ===================================================================
46:: 2. The layout pass (+layout)
57:: 2. The layout pass (+layout)
@@ -49,16 +60,30 @@
49:: +layout turns the text into tokens one line at a time: no token
60:: +layout turns the text into tokens one line at a time: no token
50:: crosses a line break, and only layout tokens join lines.
61:: crosses a line break, and only layout tokens join lines.
63+:: 2.0 Errors. +layout never fails. Where the text breaks a rule of this
64+:: section, it emits one error token, [%error msg], at the first
65+:: character that breaks it, and stops reading. The parser accepts no
66+:: error token, so it reports whichever comes first, its own error or
67+:: this one (section 3, Errors). Every error, from either stage, points
68+:: at the first character where the text stops being the start of any
69+:: valid module: everything before it could still be right. Hoon's
70+:: parser reports errors the same way, and the golden corpus's expect
71+:: lines assume it. It also puts `++ x ]` on line 1 before a bad string
72+:: on line 9, which a lexer that failed first would report instead. The
73+:: positions below follow from it. Messages quote code in backticks, as
74+:: the parser's do (section 3, Errors).
52:: 2.1 Lines. The text is split at every \0a. A final \0a ends the last
76:: 2.1 Lines. The text is split at every \0a. A final \0a ends the last
53:: line rather than starting an empty one. Lines are numbered from 1.
77:: line rather than starting an empty one. Lines are numbered from 1.
55:: 2.2 Characters. Anywhere in the text, strings and comments included,
79:: 2.2 Characters. Anywhere in the text, strings and comments included,
56−:: these are errors: tab (9), carriage return (13), every other byte
80+:: these are errors at the character: tab (9), "a tab: Jibb uses spaces
81+:: only"; carriage return (13), "a carriage return: Jibb lines end with
57−:: below 32 except \0a, and 127. Jibb, like Hoon, spaces with spaces
82+:: a line feed alone"; every other byte below 32 except \0a, and 127, "a
58−:: only ("tab: Jibb uses spaces only"). Bytes from 128 up (UTF-8) are
83+:: control character". Bytes from 128 up (UTF-8) are allowed only inside
59−:: allowed only inside strings and comments ("non-ASCII text outside a
84+:: strings and comments ("non-ASCII text outside a string or comment").
60−:: string or comment"). Outside strings and comments, see 2.6 for which
85+:: Outside strings and comments, see 2.6 for which printable characters
61−:: printable characters start a token.
86+:: start a token.
63:: 2.3 Blank and comment-only lines. A line of spaces only, or whose first
88:: 2.3 Blank and comment-only lines. A line of spaces only, or whose first
64:: non-space characters are ::, emits nothing and changes no level. Its
89:: non-space characters are ::, emits nothing and changes no level. Its
@@ -66,27 +91,31 @@
67:: 2.4 Levels. The stack holds the columns of the open levels, innermost
92:: 2.4 Levels. The stack holds the columns of the open levels, innermost
68:: first, and starts as ~[1]. For every other line let c be the column of
93:: first, and starts as ~[1]. For every other line let c be the column of
69−:: its first non-space character; then:
94+:: its first non-space character. The first such line emits no layout
70−:: - The first such line needs c = 1, else the error "the first line is
95+:: token and needs c = 1, else the error "the first line is indented:
71−:: indented: top-level lines start at column 1".
96+:: top-level lines start at column 1". Each later one emits:
72:: - c > top: push c, emit INDENT.
97:: - c > top: push c, emit INDENT.
73:: - c = top: emit NEWLINE.
98:: - c = top: emit NEWLINE.
74:: - c < top: while c < top, pop and emit DEDENT. Then c must equal the
99:: - c < top: while c < top, pop and emit DEDENT. Then c must equal the
75:: new top, else the error "this line's indentation matches no
100:: new top, else the error "this line's indentation matches no
76−:: enclosing line: indent it to column 1 or 5" (the stack's columns,
101+:: enclosing line: indent it to column 1, 5 or 7" (the stack's
77−:: outermost first); emit NEWLINE.
102+:: columns before the pops, outermost first); emit NEWLINE.
103+:: Then the line's tokens (2.6). These errors, and a bad character (2.2)
104+:: at column c, sit at column c and come before the line's layout
78−:: Then the line's tokens (2.6). A layout token's span runs from just past
105+:: tokens. A layout token's span runs from just past the previous token,
79−:: the previous token, on an earlier line, to column c of this line:
106+:: on an earlier line, to column c of this line. Errors and the token
80−:: its start is where the previous line's code ended, where "found the
107+:: printer place a layout token at its span's end, the start of the line
81−:: end of the line" errors point, and its end is where the new line
108+:: it begins (+where): the line's column is what made it INDENT, NEWLINE
82−:: starts, where "found a deeper line" errors point (see +where).
109+:: or DEDENT, and until that line, a deeper one could still have come.
84−:: 2.5 End of file. If any token was emitted: pop every level above 1,
111+:: 2.5 End of file. If any token was emitted and no error token: pop
85−:: emitting DEDENT for each, then emit one NEWLINE, all with empty spans
112+:: every level above 1, emitting DEDENT for each, then emit one NEWLINE,
113+:: all with empty spans just past the last token. They are the only
114+:: layout tokens with empty spans, which is how errors name "the end of
86−:: just past the last token. So every import and definition, the last
115+:: the file". So every import and definition, the last one included, is
87−:: one included, is followed by NEWLINE, as `module = { import NEWLINE }
116+:: followed by NEWLINE, as `module = { import NEWLINE } { definition
88−:: { definition NEWLINE }` needs. A file without tokens (empty, or only
117+:: NEWLINE }` needs. A file without tokens (empty, or only blank and
89−:: blank and comment lines) gives no tokens: an empty module.
118+:: comment lines) gives no tokens: an empty module.
91:: The layout pass knows nothing of brackets: it emits INDENT, NEWLINE and
120:: The layout pass knows nothing of brackets: it emits INDENT, NEWLINE and
92:: DEDENT by column alone. The parser decides what takes an indented
121:: DEDENT by column alone. The parser decides what takes an indented
@@ -95,60 +124,102 @@
95:: closed on one line is just tokens.
124:: closed on one line is just tokens.
97:: 2.6 Tokens. After the indentation, tokens are read left to right;
126:: 2.6 Tokens. After the indentation, tokens are read left to right;
98−:: spaces separate them and are dropped. A token is tight when it
127+:: spaces separate them and are dropped, spaces at the end of a line too.
99−:: follows the previous token on the same line with no space between;
128+:: A token is tight when it follows the previous token on the same line
100−:: the first token of a line never is. Tightness is how the parser tells
129+:: with no space between; the first token of a line never is. Tightness
101−:: the tight forms apart: m( from m (, a+b from a + b, `x from ` x,
130+:: is how the parser tells the tight forms apart: m( from m (, a+b from
102−:: |=%got from |= %got, name=value from name = value. What a token is
131+:: a + b, `x from ` x, |=%got from |= %got, name=value from name = value.
103−:: depends on its first characters:
132+:: What a token is depends on its first characters. An error "at 2nd" is
133+:: at the token's second character, the first that no Jibb token allows
105:: :: a comment: the rest of the line is skipped
136:: :: a comment: the rest of the line is skipped
106:: ( ) [ ] ` = * glyphs
137:: ( ) [ ] ` = * glyphs
107:: ~[ ~& glyphs
138:: ~[ ~& glyphs
108−:: ~ a ~ literal: ~ and the run of [a-z0-9._~-] after it;
139+:: ~ a ~ literal: ~ and the run of [a-z0-9._~-] after it,
109−:: a bare ~ is null (2.8)
140+:: read as 2.8 says; a bare ~ is null, [%null %n 0]
110:: %/ glyph, then a path segment may follow (2.7)
141:: %/ glyph, then a path segment may follow (2.7)
111−:: %.y %.n loobean literals, [%f 0] and [%f 1]
142+:: %.y %.n loobean literals, [%f 0] and [%f 1]; %. then anything
143+:: else is an error at 3rd, "the loobeans are `%.y` and
112:: %& %| term literals, [%f 0] and [%f 1]
145:: %& %| term literals, [%f 0] and [%f 1]
113−:: %a... term literal: % and a name [a-z][a-z0-9-]*, [%tas name]
146+:: %a... term literal: % and a name [a-z][a-z0-9-]*, [%tas name];
147+:: a capital or _ right after it is an error there,
148+:: "names are lowercase letters, digits and -"
114:: %0... term literal: % and digits [0-9]+, [%ud value]
149:: %0... term literal: % and digits [0-9]+, [%ud value]
115:: (doc question: digits only, as the EBNF says, so
150:: (doc question: digits only, as the EBNF says, so
116−:: %1000 is a term and %1.000 an error, unlike Hoon)
151+:: %1000 is a term and %1.000 an error at the dot); a
117−:: /- /% /= import glyphs, when followed by a space or the end
152+:: letter, _, - or . right after it is an error there,
118−:: of the line
153+:: "unexpected `a`: a numeric term is `%` and digits
155+:: % anything else after it: error at 2nd, "`%` begins a
156+:: term, a loobean or `%/`"
157+:: /- /% /= import glyphs at column 1, where imports go; anywhere
158+:: else / is a path's, so ++ x /- is the path whose
159+:: one segment is -, as the EBNF's knot allows
119:: / glyph, then a path segment may follow (2.7)
160:: / glyph, then a path segment may follow (2.7)
120:: ++ +$ + glyphs
161:: ++ +$ + glyphs
121:: |=( |= |$ glyphs
162:: |=( |= |$ glyphs
122−:: $?( $? $-( glyphs; $- without ( is the error "$- has only the
163+:: $?( $? $-( glyphs; $- followed by anything but ( is an error
123−:: wide form $-(sample product)"
164+:: at 3rd, "`$-` has only the wide form `$-(sample
124:: ^- ?- !! ? glyphs (? alone is the loobean mold)
166:: ^- ?- !! ? glyphs (? alone is the loobean mold)
125:: @ aura: @, then [a-z]*, then at most one [A-Z]; the
167:: @ aura: @, then [a-z]*, then at most one [A-Z]; the
126−:: token holds the text after @. Auras starting @s or @r
168+:: token holds the text after @. An aura starting @s or
127−:: are errors: "signed and floating-point auras are left
169+:: @r is an error at 2nd: "signed and floating-point
128−:: out of Jibb"
170+:: auras are left out of Jibb"
129−:: A-Z type variable: one capital; a letter or digit right
171+:: A-Z type variable: one capital. A letter or digit right
130−:: after it is the error "a type variable is one capital
172+:: after it is an error there, "a type variable is one
131−:: letter"
173+:: capital letter", after the type variable's token
132:: a-z name [a-z][a-z0-9-]*, or a wing: two or more names
174:: a-z name [a-z][a-z0-9-]*, or a wing: two or more names
133−:: joined by dots. A . must be followed by [a-z] ("a . in
175+:: joined by dots. Right after the name or wing, which
134−:: a wing is followed by a name"); a capital or _ right
176+:: is emitted first, a capital or _ is an error there,
135−:: after a name is the error "names are lowercase letters,
177+:: "names are lowercase letters, digits and -", and so
136−:: digits and -"
178+:: is a . not followed by [a-z], at the character after
179+:: the dot, "a `.` in a wing is followed by a name"
137:: 0-9 number literal: the run of [0-9a-zA-Z.] (2.8)
180:: 0-9 number literal: the run of [0-9a-zA-Z.] (2.8)
138:: " string (2.9)
181:: " string (2.9)
139:: } ends the innermost interpolation (2.9), else an error
182:: } ends the innermost interpolation (2.9), else an error
183+:: at it, "this `}` closes no `{`"
184+:: | $ ^ followed by nothing above or in the table below:
185+:: an error at 2nd, "`|` begins `|=`, `|=(` or `|$`",
186+:: "`$` begins `$?`, `$?(` or `$-(`", "`^` begins `^-`"
187+:: : followed by anything but :, an error at 2nd: the
188+:: table's :~ :* and a:b, else "a comment starts with
140−:: anything else an error naming the character: "unexpected `,`"
190+:: anything else an error at it, naming it: "unexpected `,`", with a
191+:: hint where one helps (+unexpected): for , . - and <,
192+:: what Jibb writes instead
142−:: Hoon runes the doc leaves out are errors with the doc's reason, found
194+:: Hoon forms the doc leaves out are errors with the doc's reason, as
195+:: "`|%` is left out: modules are records". Each points where the text
196+:: stops being Jibb: at its second character when its first can begin
197+:: Jibb there, else at its first.
143−:: where the token would start: |% |_ |* "modules are records"; $% "$?
199+:: |% |_ |* 2nd modules are records
200+:: |- 2nd gates recurse by name
201+:: $( 1st gates recurse by name; no expression starts $
144−:: covers unions"; ?+ ?: "?- is the branch"; .= ?! and !x "eq and not
202+:: $% 2nd $? covers unions
145−:: are gates"; =/ "++ defines values"; |- $( "gates recurse by name"; ~|
203+:: ?+ ?: 2nd ?- is the branch (?:: is ? and a comment)
146−:: "~& covers debugging"; => =< .* and a:b "names never shadow, and .
204+:: ?! 2nd eq and not are gates
205+:: .= 1st eq and not are gates
206+:: !x 2nd not is a gate: ! then anything but ! or >
147−:: reads imports too"; !> /+ "code also runs in the browser"; ; "layout
207+:: !> /+ 2nd code also runs in the browser
208+:: =< => .* 1st names never shadow, and . reads imports too
209+:: a:b 2nd of the :, right after a name: names never
210+:: shadow, and . reads imports too; write a.b
211+:: :~ :* 2nd indentation covers them; write ~[…] or […]
212+:: %: %= 2nd indentation covers them
148−:: is Jibb's elements"; :~ :* %: %= == -- "indentation covers them"; '
213+:: == -- 1st the closers are left out: indentation covers
215+:: ~| 2nd ~& covers debugging
216+:: ; 1st Sail's ; runes are left out: layout is Jibb's
149−:: "there is no '...' form: strings are \"...\"". Wide forms the doc
218+:: ' 1st there is no '…' form: strings are "…"
150−:: leaves out, ^-(...) and ?-(...), lex fine and fail in the parser: a
219+::
151−:: rune is followed by a space.
220+:: The parser finds the doc's other left-out forms (section 3, Errors):
221+:: =(a b), =/, *mold, ^-(…) and ?-(…). The lexer can't, since =( =/ and *
222+:: begin valid Jibb too: open=(not open.s), scry=/todos, a * mold.
153:: 2.7 Paths. Right after a / or %/ glyph, a run of knot characters
224:: 2.7 Paths. Right after a / or %/ glyph, a run of knot characters
154:: [a-z0-9._~-] is one knot token, tight. So /posts/~zod is / knot / knot;
225:: [a-z0-9._~-] is one knot token, tight. So /posts/~zod is / knot / knot;
@@ -158,50 +229,85 @@
158:: lexer tracks: segments such as logo.png, 1 or ~zod would otherwise lex
229:: lexer tracks: segments such as logo.png, 1 or ~zod would otherwise lex
159:: as a wing, a number or a ship. A / followed by anything else is just a
230:: as a wing, a number or a ship. A / followed by anything else is just a
160:: glyph: / alone is the empty path, and the parser rejects empty
231:: glyph: / alone is the empty path, and the parser rejects empty
161−:: segments such as //.
232+:: segments such as //. After the glyph and its knot, if any, comes a /,
233+:: ( or [, a closer ) ] or }, a : (of a comment) or a space: any other
234+:: printable character is an error there, "unexpected `P`: a path
235+:: segment holds lowercase letters, digits and - . ~ _", which for ?
236+:: and # also says a path carries no query or fragment. No Jibb token
237+:: goes on a path that way, so /Posts and %/blog?page=2 fail at the P
238+:: and the ?. A ~. knot literal (2.8) gets the same check, "a knot
239+:: holds lowercase letters, digits and - . ~ _".
163−:: 2.8 Literals. The lexer takes the literal's characters first, then
241+:: 2.8 Literals. The lexer takes a literal's run (2.6), then reads it
164−:: reads them with Hoon's own parser, which must consume all of them:
242+:: with Hoon's own parser, (nuck:so [[line col] run]), whose edge says
165−:: 0-9 run (rush run nuck:so) must give [%$ %ud|%ux|%uv val]: 1.000,
243+:: how much it read and how far it looked (p.edge):
166−:: 0x1f, 0v5.fdf99. So 1000 is an error, as in Hoon (groups
244+:: - It read the whole run, as an aura Jibb takes: the token. Numbers
167−:: of three), and 0b1, 0w1, 0i1, 0c1 are errors. Hoon reads
245+:: take %ud, %ux and %uv: 1.000, 0x1f, 0v5.fdf99 (Hoon reads 0x1F
169−:: ~ run a bare ~ is null, [%n 0]. Otherwise (rush run nuck:so) must
170−:: give [%$ %p|%da|%dr|%ta val]: ~zod, ~sampel-palnet,
246+:: too). ~ runs take %p, %da, %dr and %ta: ~zod, ~sampel-palnet,
171−:: ~2026.10.23..14.30.00, ~h1.m30, ~.my-post. ~~a, ~-a and
247+:: ~2026.10.23..14.30.00, ~h1.m30, ~.my-post.
172−:: ~0v... are errors.
248+:: - It read the run or part of it as an aura Jibb doesn't take: 0w1,
173−:: terms and loobeans as in 2.6.
249+:: 0i1, ~~a (a cord), ~-a (a @c). An error at 2nd, "Jibb writes no
250+:: @uw literals". A run starting 0b, 0c, 0i or 0w gets that error
251+:: before nuck:so reads it, since no Jibb literal goes on that way:
252+:: Hoon's parser would read 0b, or a 0c1 whose checksum fails, as
253+:: the number 0 and an error past the run.
254+:: - It read only part of the run, as an aura Jibb takes: that part is
255+:: a token, then an error token at p.edge, the furthest character
256+:: Hoon's parser looked at, where Hoon reports it too. 1000 is the
257+:: token 100 and an error at its fourth digit ("a decimal is digits
258+:: in groups of three joined by dots, as 1.000": each kind of literal
259+:: has its message, +literal-msg); 0v5.fdfz9 is 0v5 and an error at the
260+:: z; ~2026.10.29..14.30 is ~2026.10.29 and an error just past the
261+:: run. The token lets the parser report an earlier error first: in
262+:: `++ 2nd 5` it finds the number 2 where a name goes, at 1:4.
263+:: When it read a ship, or only the ~ of a run that goes on with a
264+:: letter, the error goes instead at the first letter that no
265+:: syllable continues (+ship-stop), or just past a duration's unit,
266+:: d h m or s, if that is later: Hoon's parser reads a syllable's
267+:: three letters before it looks them up, so it looks past that
268+:: letter. `++ x ~x` fails at the x, 1:7, as no syllable starts with
269+:: x, and ~sampel-palnez at the z.
270+:: A run that starts as a date BC does, ~ and a year then -, fails at the
271+:: - if the year is 0 or past 292.277.024.401, before nuck:so sees it:
272+:: Hoon's +year crashes on those years, as in ~0-.1.1 (+year-bc).
174:: The token is [%lit [form aura val]]: form names the literal rule that
273:: The token is [%lit [form aura val]]: form names the literal rule that
175−:: matched, aura and val are the dime Hoon's parser gives.
274+:: matched, aura and val are the dime Hoon's parser gives. Terms,
275+:: loobeans and the bare ~ are read by hand (2.6).
177:: 2.9 Strings. A " starts a string, which ends on the same line. Up to
277:: 2.9 Strings. A " starts a string, which ends on the same line. Up to
178:: the closing ":
278:: the closing ":
179:: \\ \" \{ and \ with two lowercase hex digits are escapes, as in
279:: \\ \" \{ and \ with two lowercase hex digits are escapes, as in
180−:: Hoon's tapes (doc question: the doc names only \{ and \"); any
280+:: Hoon's tapes (doc question: the doc names only \{ and \"). Any
181−:: other \ is an error at the \
281+:: other \ is an error at the first character that can't continue
282+:: the escape: the one after the \, or a bad second hex digit (2.2's
283+:: error if it is a tab or another control character)
182−:: { starts an interpolation: the text so far is emitted as str-head
284+:: { starts an interpolation. The text so far is emitted as str-head,
285+:: spanning from the opening " to just past the {, or as str-mid,
183−:: (at the opening ") or str-mid (at the } that ended the previous
286+:: from the } that ended the previous interpolation to just past the
184−:: interpolation); then tokens are lexed as usual, strings included,
287+:: {. Then tokens are lexed as usual, strings included, up to the }
185−:: up to the } that ends this interpolation, where the string resumes
288+:: that ends this interpolation, where the string resumes
186−:: the closing " emits str when there was no interpolation (at the
289+:: the closing " emits str when there was no interpolation, spanning
187−:: opening ") or str-tail (at the last })
290+:: the whole string, or str-tail, from the last } to just past the "
291+:: any other character is text, } included, as in Hoon
188:: So "Since {day}!" is str-head "Since ", name day, str-tail "!", and the
292:: So "Since {day}!" is str-head "Since ", name day, str-tail "!", and the
189:: parser reads head expr { mid expr } tail. The lexer keeps a stack of
293:: parser reads head expr { mid expr } tail. The lexer keeps a stack of
190:: open interpolations, since strings nest inside them. Inside an
294:: open interpolations, since strings nest inside them. Inside an
191−:: interpolation, :: is the error "a comment can't start inside {...}"
295+:: interpolation a : is an error at it, "a comment can't start inside
192−:: and { is an error; inside a string, :: is text. The end of the line
296+:: `{…}`", and so is a {; inside a string, :: is text. A line that ends
193−:: inside a string or an interpolation is an error at its " ("this
297+:: inside a string or an interpolation is an error at its end, "this
194−:: string has no closing \" on its line") or { ("this { has no }").
298+:: string has no closing `\"` on its line" or "this `{` has no `}` on its
195−:: Token texts hold the decoded text. Spaces after { and before } are
299+:: line". Token texts hold the decoded text. Spaces after { and before }
196−:: allowed, as after any opener.
300+:: are allowed, as after any opener.
198:: 2.10 What fails here. Bad characters (2.2), an indented first line and
302:: 2.10 What fails here. Bad characters (2.2), an indented first line and
199:: a dedent to no open level (2.4), characters that start no token or a
303:: a dedent to no open level (2.4), characters that start no token or a
200−:: left-out rune (2.6), malformed literals, auras, type variables and
304+:: left-out form (2.6), malformed literals, auras, type variables and
305+:: wings (2.6, 2.8), a character that no token lets follow a path
306+:: segment, a ~. knot or a numeric term (2.6, 2.7), and unclosed strings
201−:: wings (2.6, 2.8), and unclosed strings and interpolations, bad escapes
307+:: and interpolations, bad escapes and stray { or } (2.9). Everything
202−:: and stray { or } (2.9). Everything else is the parser's: brackets that
308+:: else is the parser's: brackets that don't match or don't close, a
203−:: don't match or don't close, a missing or extra space, a deeper line
309+:: missing or extra space, a deeper line that nothing takes, a token
204−:: that nothing takes, a token where it can't go.
310+:: where it can't go.
206:: ===================================================================
312:: ===================================================================
207:: 3. The parser (+parse-tokens)
313:: 3. The parser (+parse-tokens)
@@ -221,7 +327,8 @@
221:: name, then = tight a face (items, lines, fields, samples)
327:: name, then = tight a face (items, lines, fields, samples)
222:: name, then + tight a pair
328:: name, then + tight a pair
223:: name or wing, then ( tight a change
329:: name or wing, then ( tight a change
224−:: |=, then a term tight a constructor; else the tall gate
330+:: |=, then a term tight a constructor; else, where an
331+:: expression starts, the tall gate
225:: ++ or +$ a definition; else a body's final expr
332:: ++ or +$ a definition; else a body's final expr
226:: INDENT a body, an open bracket's lines, arms
333:: INDENT a body, an open bracket's lines, arms
227:: or cases; else the expression goes on
334:: or cases; else the expression goes on
@@ -231,11 +338,18 @@
231:: their failures carry a position but no expectation; over tokens they
338:: their failures carry a position but no expectation; over tokens they
232:: would need a token "tape" and an expectation set bolted on. Recursive
339:: would need a token "tape" and an expectation set bolted on. Recursive
233:: descent is shorter, and each error names what its construct expected.
340:: descent is shorter, and each error names what its construct expected.
341+:: An operand that isn't where an expression starts (a call's argument,
342+:: an item, a pair's or unit's operand, the wide gate's body) can't be a
343+:: rune, so there |= is a constructor and needs its term at once:
344+:: (state.cart |= %got) is an error at the space after |=, 1:20. Loops
345+:: over many items (definitions, arms, lines, items) may recurse: vere's
346+:: stack takes ten million plain frames, so no Jibb file nests or runs
347+:: long enough to overflow it.
235:: The EBNF and its nodes (see the types below):
349:: The EBNF and its nodes (see the types below):
236:: module { import NEWLINE } { definition NEWLINE }, then the end
350:: module { import NEWLINE } { definition NEWLINE }, then the end
237:: of the tokens; an import after a definition is the error
351:: of the tokens; an import after a definition is the error
238−:: "imports come first"
352+:: "expected a definition, found `/-`; imports come first"
239:: import /- name -> %sur; /% term -> %mark; /= name pathlit ->
353:: import /- name -> %sur; /% term -> %mark; /= name pathlit ->
240:: %jibb. /% takes % and a name only: no mark is called %0,
354:: %jibb. /% takes % and a name only: no mark is called %0,
241:: %& or %| (doc question)
355:: %& or %| (doc question)
@@ -252,15 +366,19 @@
252:: group ( expr ) -> %group of the expr. The open form ( {operand}
366:: group ( expr ) -> %group of the expr. The open form ( {operand}
253:: open -> %group of the call whose elements are the first
367:: open -> %group of the call whose elements are the first
254:: line's operands, then one per line; one element alone is
368:: line's operands, then one per line; one element alone is
369+:: just grouped. A ( followed by a rune is the closed form.
255−:: just grouped. Lines of an open group or list take no
370+:: Lines of an open group or list take no name= (doc
256−:: name= (doc question)
371+:: question)
257:: change every item and line has name= (the EBNF's "items named")
372:: change every item and line has name= (the EBNF's "items named")
258:: rune |= -> %gate; ^- -> %cast; ~& -> %trace; ?- -> %branch;
373:: rune |= -> %gate; ^- -> %cast; ~& -> %trace; ?- -> %branch;
259−:: !! -> %crash. A bare name as the whole sample binds, in
260−:: |=(…) and in the tall |= alike; inside a cell sample a
261−:: bare name is a mold, as field says (doc question). !!
262−:: is not an operand, so [a !!] needs [a (!!)] (doc
374+:: !! -> %crash. !! is not an operand, so [a !!] needs
263−:: question)
375+:: [a (!!)] (doc question)
376+:: sample the sample of |= and |=(…): a field, except that a bare
377+:: name binds (%bind) wherever it stands in the sample: the
378+:: whole sample, as |= x and |=(p title.p), or a field of a
379+:: cell sample at any depth, as b in |= [a=@ b], which is
380+:: a %cell of samples (doc question). A name inside a mold,
381+:: as a in (list a), $-(a b) or $?(a b), is a mold
264:: mold aura -> %aura; * -> %noun; !! -> %void; ? -> %flag;
382:: mold aura -> %aura; * -> %noun; !! -> %void; ? -> %flag;
265:: ~ term loobean -> %const; ref -> %ref; typevar -> %tvar;
383:: ~ term loobean -> %const; ref -> %ref; typevar -> %tvar;
266:: pathmold -> %path; ( ref mold … ) -> %apply; [ field … ]
384:: pathmold -> %path; ( ref mold … ) -> %apply; [ field … ]
@@ -280,7 +398,7 @@
280:: expr } str-tail, where each expr is a full expression.
398:: expr } str-tail, where each expr is a full expression.
281:: - Spaces, comments and blank lines: the layout pass, and Spacing
399:: - Spaces, comments and blank lines: the layout pass, and Spacing
283−:: - pathlit as an operand (above).
401+:: - pathlit as an operand (above), with / alone the empty path.
284:: - knot may be empty: only / alone uses that, as the empty path; //
402:: - knot may be empty: only / alone uses that, as the empty path; //
285:: and a trailing / are errors (doc question).
403:: and a trailing / are errors (doc question).
286:: - Open brackets in molds: mold has no open form, so a mold's [ or (
404:: - Open brackets in molds: mold has no open form, so a mold's [ or (
@@ -292,36 +410,67 @@
292:: ^-, ~& and arms), the Runes prose says "inside a gate"; the parser
410:: ^-, ~& and arms), the Runes prose says "inside a gate"; the parser
293:: takes them in every body and leaves the rest to the checker (doc
411:: takes them in every body and leaves the rest to the checker (doc
294:: question).
412:: question).
413+:: - Where errors point: the doc says only "line, column and an
414+:: excerpt"; 2.0 and Errors below say where.
296:: Spacing. Two tokens on one line are separated by spaces, except:
416:: Spacing. Two tokens on one line are separated by spaces, except:
297:: - tight, always: name+operand (pair), `operand (unit), |=term
417:: - tight, always: name+operand (pair), `operand (unit), |=term
298:: (constructor), wing( (change), name= and = value (faces, in items,
418:: (constructor), wing( (change), name= and = value (faces, in items,
299−:: lines, fields and samples), =name (pun), and every token of a path
419+:: lines, fields and samples), =name (pun); in a path or application
300−:: or application path after its first / or %/, typed segments
420+:: path, each / and the segment after it, and all of a typed segment,
301−:: ([id=@ud]) included
421+:: [id=@ud]
302:: - either way: right after an opener ( [ ~[ |=( $?( $-( or {, and
422:: - either way: right after an opener ( [ ~[ |=( $?( $-( or {, and
303−:: right before a closer ) ] or } (the theme example writes `[ space=`)
423+:: right before a closer ) ] or } (the theme example writes `[ space=`),
424+:: the ( and ) of an application path's segment included; inside
425+:: them the expression is spaced as anywhere else, as %/x/(f a)
304−:: A missing space is an error ("expected a space between items, found
426+:: A missing space is an error at the tight token, and so is a space
305−:: `[`"), and so is a space inside a tight form ("a pair has no spaces,
306−:: as in gap+%s2"). A rune with children, and an import glyph, is
427+:: inside a tight form, at the space (Errors). A rune with children, and
307−:: followed by a space (or, for $? with its cases below, the end of the
428+:: an import glyph, is followed by a space (or, for $? with its cases
429+:: below, the end of the line). Items are separated by one or more
308−:: line). This is how the parser makes "one space between items; more for
430+:: spaces; how many isn't checked. This is how the parser makes "one
309−:: alignment" precise (doc question).
431+:: space between items; more for alignment" precise (doc question).
433+:: Errors. The parser stops at the first token it can't accept and
311−:: Errors. "expected <what>, found <token>", with an optional "; <hint>",
434+:: reports "expected <what>, found <token>", with an optional "; <hint>",
312:: lowercase, no final period. <what> uses the doc's words: "an
435:: lowercase, no final period. <what> uses the doc's words: "an
313−:: expression", "a mold", "a pattern", "a name", "`]`", "the end of the
436+:: expression", "a mold", "a pattern", "a name", "`]`", "a space between
314−:: line". <token> is +show-found: the token's text in backticks, "a
437+:: items". <token> is +show-found: the token's text in backticks, "a
315−:: string", "a deeper line" for INDENT, "the end of the line" for NEWLINE
438+:: string", and for layout tokens "a deeper line" (INDENT), "a line at
316−:: and DEDENT. The error sits where +where says: at the token's start, but
439+:: the same depth" (NEWLINE), "a shallower line" (DEDENT), or "the end of
317−:: at the end of the previous line's code for NEWLINE and DEDENT, and at
440+:: the file" (an empty span, 2.5). Each error points at the first
318−:: the start of the deeper line for INDENT. Examples:
441+:: character that can't begin a valid module (2.0):
319−:: expected `]` to close the `[` at 3:5, found the end of the line
442+:: - At the token's start; for a layout token, at the start of the
443+:: line it begins, or just past the last token at the end of the
444+:: file (+where). So `++ view` above `++ main view` is an error at
445+:: 2:1, the line that isn't deeper.
446+:: - Where a tight form needs its next token at once (after name+, `,
447+:: |= in an operand, name=, the = of a pun, a path's / and %/, a
448+:: typed segment's parts) and finds a space: at the end of the
449+:: previous token, where the space starts, "expected a term right
450+:: after `|=`, found a space" (+fail-space). A wing and a ( after a
451+:: space are no error: that is a call.
452+:: - At an error token: the lexer's position and message, unchanged.
453+:: - Three pattern errors point inside their token, where it stops
454+:: being a pattern: a date, duration or knot at its second character
455+:: (~ alone is null, and no ship goes on so), ~[ at its [, and a
456+:: string with {…} at its {.
457+:: In a bracket, a closer that doesn't match is "expected `]` to close
458+:: the `~[` at 1:6, found `)`", checked before spacing; after an item, a
459+:: tight token that isn't a closer is "expected a space between items,
460+:: found `M`". Hints name the left-out forms the lexer can't see (2.6):
461+:: where an expression starts, = followed tight by ( is "=(a b) is left
462+:: out: eq is a gate", = followed tight by / is "=/ is left out: ++
463+:: defines values", and * is "*mold is left out: no defaults"; ^- or ?-
464+:: followed tight by ( is "the wide form ^-(…) is left out: indentation
465+:: covers it". Examples:
466+:: expected `]` or an item on a deeper line, found a line at the same
320:: expected the end of the line, found a deeper line; only a bracket
468:: expected the end of the line, found a deeper line; only a bracket
321:: open at the end of a line, or a rune's body, takes deeper lines
469:: open at the end of a line, or a rune's body, takes deeper lines
322:: expected an expression, found `^-`; inside a call, a rune goes in
470:: expected an expression, found `^-`; inside a call, a rune goes in
323:: parentheses
471:: parentheses
324−:: a change's items are named, as in m(done=%.y)
472+:: expected a named item, found `x`; a change's items are named, as in
325:: Messages of milestone 1 are golden-tested, so their wording is fixed
474:: Messages of milestone 1 are golden-tested, so their wording is fixed
326:: by the tests once written.
475:: by the tests once written.
@@ -331,30 +480,51 @@
332:: Each printer returns a cord: lines joined by \0a, no final newline.
481:: Each printer returns a cord: lines joined by \0a, no final newline.
334−:: 4.1 Tokens (+show-tokens): one token per line, "<line>:<col>", then
483+:: 4.1 Tokens (+show-tokens): one token per line, "<line>:<col>" at
335−:: "tight " if the token is tight, then the kind and the text. Kinds:
484+:: +where, then "tight " if the token is tight, then the kind and the
336−:: glyph (with its text), INDENT, DEDENT, NEWLINE, name, wing, lit (the
485+:: text. Kinds: glyph (with its text), INDENT, DEDENT, NEWLINE, name, wing,
337−:: literal as Hoon prints its dime: 1.000, 0x1f, ~zod, ~h1.m30, ~.knot;
486+:: lit (the literal as Hoon prints its dime: 1.000, 0x1f, ~zod, ~h1.m30,
338−:: terms as %add, %0, %& and %|; loobeans %.y and %.n; null ~), str,
487+:: ~.knot; terms as %add, %0, %& and %|; loobeans %.y and %.n; null ~),
339−:: str-head, str-mid and str-tail (quoted, with \\ \" \{ and \XX
488+:: str, str-head, str-mid and str-tail (quoted, with \\ \" \{ and \XX
340−:: escapes), knot, aura (@ud), tvar. Layout tokens print their span's
489+:: escapes), knot, aura (@ud), tvar, and ERROR with its message (+golden
490+:: prints the rendered error instead). A layout token prints at the
491+:: start of the line it begins, beside that line's first token; at the
341−:: start: the end of the previous line's code. For `++ main view`:
492+:: end of the file, just past the last token. For `++ main view`:
343:: 1:1 glyph ++
494:: 1:1 glyph ++
344:: 1:4 name main
495:: 1:4 name main
345:: 1:9 name view
496:: 1:9 name view
346:: 1:13 NEWLINE
497:: 1:13 NEWLINE
499+:: and for `++ init |= ~` with ` [[~ %.n] ""]` below it:
510+:: 2:10 tight glyph ]
512+:: 2:14 tight glyph ]
348:: 4.2 Trees (+show-module): S-expressions, (kind line:col fields kids),
516:: 4.2 Trees (+show-module): S-expressions, (kind line:col fields kids),
349:: with the node's start. A leaf is a node whose kids, if any, are atoms
517:: with the node's start. A leaf is a node whose kids, if any, are atoms
350:: (a string's quoted text, an application path's knots). A node whose
518:: (a string's quoted text, an application path's knots). A node whose
351:: kids are all leaves prints on one line; otherwise each kid goes on its
519:: kids are all leaves prints on one line; otherwise each kid goes on its
352:: own line, indented two more spaces, and the closing parens end the
520:: own line, indented two more spaces, and the closing parens end the
353:: last kid's line. (module …) always breaks. Bare fields and samples
521:: last kid's line. (module …) always breaks. Bare fields and samples
522+:: print as their mold, and a cell sample as (cell …) of its samples, a
354−:: print as their mold, unnamed items as their expr; strings print text
523+:: bare name there as (bind …); unnamed items print as their expr;
355−:: pieces quoted, between their interpolated exprs; application paths
524+:: strings print text pieces quoted, between their interpolated exprs;
356−:: print knots bare between their (expr) segments. The cart store from
525+:: application paths print knots bare between their (expr) segments. The
357−:: the Grammar tab, its code block's first line being line 1:
526+:: cart store from the Grammar tab, its code block's first line being
359:: (module
529:: (module
360:: (mold 2:1 state
530:: (mold 2:1 state
@@ -400,7 +570,10 @@
400:: source line, then a caret under the column (col - 1 spaces, then ^).
570:: source line, then a caret under the column (col - 1 spaces, then ^).
401:: The file prints as a Clay path with its last segment as the extension:
571:: The file prints as a Clay path with its last segment as the extension:
402:: /tests/golden/x/source/jibb is /tests/golden/x/source.jibb. A line
572:: /tests/golden/x/source/jibb is /tests/golden/x/source.jibb. A line
403−:: past the end of the text prints as an empty line.
573+:: past the end of the text prints as an empty line. In the excerpt every
574+:: control character, a tab or carriage return being the likely ones,
575+:: prints as a space, so the caret stays under its column and golden
576+:: files hold no raw control bytes.
405:: /ui/basket.jibb:22:5: msg has no case %submitt
578:: /ui/basket.jibb:22:5: msg has no case %submitt
406:: %submitt [m(note="") ~]
579:: %submitt [m(note="") ~]
@@ -430,6 +603,9 @@
431:: tight is %.y when the token follows the previous one on the same
604:: tight is %.y when the token follows the previous one on the same
432:: line with no space between them; layout tokens are never tight.
605:: line with no space between them; layout tokens are never tight.
606+:: A layout token's span runs from just past the previous token to the
607+:: start of the line it begins, or is empty at the end of the file
608+:: (2.4, 2.5). An error token is the last token, if any (2.0).
434+$ token [at=span tight=? lex=lexeme]
610+$ token [at=span tight=? lex=lexeme]
@@ -445,6 +621,7 @@
445 [%knot p=@ta] :: path segment
621 [%knot p=@ta] :: path segment
446 [%aura p=@ta] :: @ud as 'ud', @ as ''
622 [%aura p=@ta] :: @ud as 'ud', @ as ''
447 [%tvar p=@t] :: M
623 [%tvar p=@t] :: M
624+ [%error p=tape] :: 2.0: what broke
450:: +glyph: a token that is its text, and the layout tokens
627:: +glyph: a token that is its text, and the layout tokens
@@ -481,7 +658,10 @@
483:: The syntax tree. Every node is [%tag at=span …] with at least one
660:: The syntax tree. Every node is [%tag at=span …] with at least one
484−:: field after at, so +span-of finds the span of any node at +<.
661+:: field after at, so +span-of finds its span at +<, except +item, +edit
662+:: and +arm, which have no tag and start with at=span; and a path's
663+:: knots (+pseg, +qseg) and a string's text (+piece), which have no
664+:: span: their node's span covers them.
485:: Types avoid the standard library's names (mold, wing, gate, sym…),
665:: Types avoid the standard library's names (mold, wing, gate, sym…),
486:: hence +shape for a mold as written. Every type in a recursive cycle
666:: hence +shape for a mold as written. Every type in a recursive cycle
487:: has a $~ default that is a leaf: Hoon can't build the default of
667:: has a $~ default that is a leaf: Hoon can't build the default of
@@ -549,14 +729,18 @@
550+$ arm $~([*span *pattern *expr] [at=span pat=pattern val=expr])
730+$ arm $~([*span *pattern *expr] [at=span pat=pattern val=expr])
552−:: +sample: a gate's sample; a bare name binds (%bind)
732+:: +sample: a gate's sample, or a field of a cell sample
734+:: A bare name binds (%bind) wherever it stands in a sample, so a cell
735+:: sample is a %cell of samples, never a %bare cell (section 3).
555 $~ [%bind *span %$]
738 $~ [%bind *span %$]
556 $% [%bind at=span name=term] :: p
739 $% [%bind at=span name=term] :: p
557 [%face at=span name=term =shape] :: m=model
740 [%face at=span name=term =shape] :: m=model
558 [%pun at=span name=term] :: =msg, for msg=msg
741 [%pun at=span name=term] :: =msg, for msg=msg
742+ [%cell at=span items=(list sample)] :: [a=@ b] [=msg m=model]
559− [%bare at=span =shape] :: ~ [a=@ b=@]
743+ [%bare at=span =shape] :: ~ @t (list @t) %add
562:: +field: a field of a cell mold; nests in +sample
746:: +field: a field of a cell mold; nests in +sample
@@ -604,46 +788,713 @@
605:: +layout: source to tokens (design: section 2)
789:: +layout: source to tokens (design: section 2)
791+:: Never fails: an error it finds is its last token, [%error msg], at
792+:: the error's position (2.0). +lexer does the work.
608 |= [file=path src=@t]
795 |= [file=path src=@t]
609− ^- (each (list token) error)
796+ ^- (list token)
610− [%| file 1 1 "layout pass not implemented"]
797+ ~(run lexer [(trip src) 1 1 ~ ~[1] [0 0] 0 &])
799+:: +layout-error: the error of a token list's error token, if any
802+ |= [file=path toks=(list token)]
805+ =/ last=token (rear toks)
806+ ?. ?=([%error *] lex.last) ~
807+ `[file line.beg.at.last col.beg.at.last p.lex.last]
612:: +parse: source to tree
809:: +parse: source to tree
615 |= [file=path src=@t]
812 |= [file=path src=@t]
616 ^- (each module error)
813 ^- (each module error)
617− =/ lay (layout file src)
619− (parse-tokens file p.lay)
814+ (parse-tokens file (layout file src))
621:: +parse-tokens: tokens to tree (design: section 3)
816:: +parse-tokens: tokens to tree (design: section 3)
818+:: It accepts no error token, so it fails at the first of its own
819+:: error and the layout pass's (2.0). The stub reports an error token
820+:: that comes first, as the parser will, and fails otherwise.
623++ parse-tokens
822++ parse-tokens
624 |= [file=path toks=(list token)]
823 |= [file=path toks=(list token)]
625 ^- (each module error)
824 ^- (each module error)
825+ ?: ?=([[* * [%error *]] *] toks)
826+ (fail file toks "a module")
626 =/ at=loc ?~(toks [1 1] (where i.toks))
827 =/ at=loc ?~(toks [1 1] (where i.toks))
627 [%| file line.at col.at "parser not implemented"]
828 [%| file line.at col.at "parser not implemented"]
629:: +golden: both stages printed, or the rendered error
830:: +golden: both stages printed, or the rendered error
631−:: tokens: the layout pass's output, or its error
832+:: tokens: the layout pass's tokens, or its error if it found one
632−:: tree: the parser's output, or the layout pass's or parser's error
833+:: tree: the parser's tree, or the first error of either stage, which
834+:: the parser reports since it accepts no error token (2.0)
635 |= [file=path src=@t]
837 |= [file=path src=@t]
636 ^- [tokens=@t tree=@t]
838 ^- [tokens=@t tree=@t]
637− =/ lay (layout file src)
839+ =/ toks=(list token) (layout file src)
638− ?: ?=(%| -.lay)
840+ =/ bad=(unit error) (layout-error file toks)
639− =/ msg=@t (show-error src p.lay)
841+ :- ?^ bad (show-error src u.bad)
640− [msg msg]
842+ (show-tokens toks)
641− :- (show-tokens p.lay)
642− =/ par (parse-tokens file p.lay)
843+ =/ par (parse-tokens file toks)
644− (show-error src p.par)
844+ ?: ?=(%| -.par) (show-error src p.par)
845+ :: a parser that accepted an error token would be wrong; say so
846+ ?^ bad (show-error src u.bad)
645 (show-module p.par)
847 (show-module p.par)
851+:: +lexer: the layout pass (section 2), a door over its state
853+:: It reads the text as one tape, a line at a time, keeps its tokens
854+:: last first and flops them at the end, so its time is linear in the
855+:: text. Each +step reads a line's indentation, one token or the end of
856+:: a line, and returns the door; +run steps until the text ends or an
857+:: error token stops it (2.0). Every loop is a trap whose recursive
858+:: call is a tail call, so a long file grows no stack.
861+ |_ $: cs=tape :: the text not yet read
862+ lin=@ud :: the line it starts on
863+ col=@ud :: the column it starts at
864+ out=(list token) :: the tokens so far, last first
865+ lev=(list @ud) :: open levels, innermost first
866+ pend=loc :: just past the last token
867+ dep=@ud :: open interpolations (2.9)
868+ bol=? :: at the start of a line
872+ :: +run: the tokens, to an error token or the end of the file
878+ ?: stopped:lx (flop out.lx)
879+ ?: &(bol.lx ?=(~ cs.lx)) (flop out:eof:lx)
882+ :: +stopped: whether an error token was emitted (2.0)
884+ ++ stopped ?=([[* * [%error *]] *] out)
886+ :: +eof: the end of the file (2.5): a DEDENT for each open level but
887+ :: the first, then NEWLINE, all with empty spans just past the last
888+ :: token; nothing in a file without tokens
893+ =/ at=span [pend pend]
894+ =/ n=@ud (dec (max 1 (lent lev)))
895+ cor(out [[at | %newline] (weld (reap n `token`[at | %dedent]) out)])
897+ :: +step: a line's indentation, or the line's next token or its end
901+ ?: bol indent:spaces
904+ :: +spaces: past the spaces at the head of the text
909+ ?. &(?=(^ cs) =(' ' i.cs)) cor
910+ $(cs t.cs, col +(col))
912+ :: +next: past the end of the line, to the start of the next one
917+ cor(cs t.cs, lin +(lin), col 1, bol &)
919+ :: +indent: a line, from its first character that isn't a space. A
920+ :: blank or comment-only line gives nothing (2.3); another line gets
921+ :: that character checked (2.2), then its layout tokens (2.4).
925+ ?: |(?=(~ cs) =('\0a' i.cs)) next
926+ ?: &(=(':' i.cs) ?=(^ t.cs) =(':' i.t.cs)) next:comment
927+ =/ bad=(unit tape) (bad-char i.cs)
928+ ?^ bad (err col u.bad)
929+ ?: (gte i.cs 128) (err col non-ascii)
933+ "the first line is indented: top-level lines start at column 1"
935+ =/ top=@ud ?~(lev 1 i.lev)
936+ ?: (gth col top) (lay(lev [col lev], bol |) 1 %indent)
937+ ?: =(col top) (lay(bol |) 1 %newline)
939+ =/ rest=(list @ud) lev
941+ ?: &(?=(^ rest) (lth col i.rest)) $(n +(n), rest t.rest)
942+ ?. &(?=(^ rest) =(col i.rest)) (err col (no-level lev))
943+ (lay:(lay(lev rest, bol |) n %dedent) 1 %newline)
945+ :: +lay: n layout tokens g, spanning from just past the last token to
946+ :: this line's first character (2.4); they leave pend as it is
949+ |= [n=@ud g=?(%indent %dedent %newline)]
951+ cor(out (weld (reap n `token`[[pend [lin col]] | g]) out))
953+ :: +err: the error token, at column c of this line (2.0)
958+ cor(out [[[[lin c] [lin c]] | [%error msg]] out])
960+ :: +put: a token from beg to end on this line, with rest the text
961+ :: after it, which starts at end
964+ |= [beg=loc end=loc lex=lexeme rest=tape]
967+ out [[[beg end] =(beg pend) lex] out]
973+ :: +emit: a token of the next n characters, which are ASCII
976+ |= [n=@ud lex=lexeme]
978+ (put [lin col] [lin (add col n)] lex (slag n cs))
980+ :: +comment: past a comment to the end of its line, checking its
986+ ?: |(?=(~ cs) =('\0a' i.cs)) cor
987+ ?: (in i.cs bad-mask) (err col (need (bad-char i.cs)))
988+ $(cs t.cs, col (next-col col i.cs))
990+ :: +read: the line's next token, or its end (2.6)
992+ :: A token is told by its first characters a, b and c, where b and c
993+ :: are 0 past the end of the text. Names, the most common, go first.
997+ ?: |(?=(~ cs) =('\0a' i.cs))
998+ ?. =(0 dep) (err col "this `\{` has no `}` on its line")
1001+ ?: (is-lower a) name
1002+ =/ b=@ ?~(t.cs 0 i.t.cs)
1003+ =/ c=@ ?~(t.cs 0 ?~(t.t.cs 0 i.t.t.cs))
1004+ ?: =('(' a) (emit 1 %'(')
1005+ ?: =(')' a) (emit 1 %')')
1006+ ?: =('[' a) (emit 1 %'[')
1007+ ?: =(']' a) (emit 1 %']')
1008+ ?: =('"' a) (text |)
1010+ ?: =(0 dep) (err col "this `}` closes no `\{`")
1011+ (text(dep (dec dep)) &)
1012+ ?: (is-digit a) (literal (run-len cs number-mask))
1013+ ?: =('%' a) (percent b c)
1015+ ?: |(=('<' b) =('>' b))
1017+ "`={[b ~]}` is left out: names never shadow, and `.` reads imports too"
1019+ (err col "the closer `==` is left out: indentation covers it")
1021+ ?: =('`' a) (emit 1 %'`')
1022+ ?: =('*' a) (emit 1 %'*')
1024+ ?: =('[' b) (emit 2 %'~[')
1025+ ?: =('&' b) (emit 2 %'~&')
1026+ ?: =('|' b) (err +(col) "`~|` is left out: `~&` covers debugging")
1027+ =/ n=@ud (run-len t.cs knot-mask)
1028+ ?: =(0 n) (emit 1 [%lit %null %n 0])
1031+ ?: &(=(1 col) |(=('-' b) =('%' b) =('=' b)))
1032+ (emit 2 ?:(=('-' b) %'/-' ?:(=('%' b) %'/%' %'/=')))
1034+ (err +(col) "`/+` is left out: code also runs in the browser")
1035+ segment:(emit 1 %'/')
1037+ ?: =('+' b) (emit 2 %'++')
1038+ ?: =('$' b) (emit 2 %'+$')
1041+ ?: &(=('=' b) =('(' c)) (emit 3 %'|=(')
1042+ ?: =('=' b) (emit 2 %'|=')
1043+ ?: =('$' b) (emit 2 %'|$')
1044+ ?: |(=('%' b) =('_' b) =('*' b))
1045+ (err +(col) "`|{[b ~]}` is left out: modules are records")
1046+ ?: =('-' b) (err +(col) "`|-` is left out: gates recurse by name")
1047+ (err +(col) "`|` begins `|=`, `|=(` or `|$`")
1049+ ?: &(=('?' b) =('(' c)) (emit 3 %'$?(')
1050+ ?: =('?' b) (emit 2 %'$?')
1051+ ?: &(=('-' b) =('(' c)) (emit 3 %'$-(')
1053+ (err (add 2 col) "`$-` has only the wide form `$-(sample product)`")
1054+ ?: =('(' b) (err col "`$(…)` is left out: gates recurse by name")
1055+ ?: =('%' b) (err +(col) "`$%` is left out: `$?` covers unions")
1056+ (err +(col) "`$` begins `$?`, `$?(` or `$-(`")
1058+ ?: =('-' b) (emit 2 %'^-')
1059+ (err +(col) "`^` begins `^-`")
1061+ ?: =('-' b) (emit 2 %'?-')
1062+ ?: =('+' b) (err +(col) "`?+` is left out: `?-` is the branch")
1063+ ?: &(=(':' b) !=(':' c))
1064+ (err +(col) "`?:` is left out: `?-` is the branch")
1065+ ?: =('!' b) (err +(col) "`?!` is left out: eq and not are gates")
1068+ ?: =('!' b) (emit 2 %'!!')
1070+ (err +(col) "`!>` is left out: code also runs in the browser")
1071+ (err +(col) "`!` begins only `!!`: `!x` is left out, as not is a gate")
1073+ ?: |(=('s' b) =('r' b))
1074+ (err +(col) "signed and floating-point auras are left out of Jibb")
1075+ =/ n=@ud (run-len t.cs lower-mask)
1076+ =/ rest=tape (slag n t.cs)
1077+ =? n &(?=(^ rest) (is-upper i.rest)) +(n)
1078+ (emit +(n) [%aura (@ta (crip (scag n t.cs)))])
1080+ => (emit 1 [%tvar a])
1081+ ?. &(?=(^ cs) |((is-lower i.cs) (is-upper i.cs) (is-digit i.cs)))
1083+ (err col "a type variable is one capital letter")
1085+ ?. =(0 dep) (err col "a comment can't start inside `\{…}`")
1086+ ?: =(':' b) comment
1088+ (err +(col) "`:~` is left out: indentation covers it; write ~[…]")
1090+ (err +(col) "`:*` is left out: indentation covers it; write […]")
1091+ ?: &(=(pend [lin col]) ?=([[* * ?([%name *] [%wing *])] *] out))
1093+ %+ weld "`a:b` is left out: names never shadow, "
1094+ "and `.` reads imports too; write a.b"
1095+ (err +(col) "a comment starts with `::`")
1097+ ?: =('=' b) (err col "`.=` is left out: eq and not are gates")
1100+ "`.*` is left out: names never shadow, and `.` reads imports too"
1101+ (err col (unexpected a))
1104+ (err col "the closer `--` is left out: indentation covers it")
1105+ (err col (unexpected a))
1107+ (err col "Sail's `;` runes are left out: layout is Jibb's elements")
1108+ ?: =('\'' a) (err col "there is no '…' form: strings are \"…\"")
1110+ (err col "unexpected `\{`: `\{…}` interpolates only inside a string")
1111+ ?: (gte a 128) (err col non-ascii)
1112+ =/ bad=(unit tape) (bad-char a)
1113+ ?^ bad (err col u.bad)
1114+ (err col (unexpected a))
1116+ :: +unexpected: the error for a character that starts no token (2.6),
1117+ :: with a hint where one helps
1122+ =/ what=tape "unexpected `{[a ~]}`"
1123+ ?: =('.' a) "{what}: a `.` joins the names of a wing, as title.p"
1124+ ?: =(',' a) "{what}: items are separated by spaces"
1126+ "{what}: a name starts with a letter, and Jibb has no operators"
1128+ "{what}: there is no `<…>`, since `\{…}` prints a value by its aura"
1131+ :: +percent: a token that starts with %, whose next characters are b
1132+ :: and c: %/, a loobean or a term (2.6)
1137+ ?: =('/' b) segment:(emit 2 %'%/')
1139+ ?: =('y' c) (emit 3 [%lit %loobean %f 0])
1140+ ?: =('n' c) (emit 3 [%lit %loobean %f 1])
1141+ (err (add 2 col) "the loobeans are `%.y` and `%.n`")
1142+ ?: =('&' b) (emit 2 [%lit %term %f 0])
1143+ ?: =('|' b) (emit 2 [%lit %term %f 1])
1144+ ?: =(':' b) (err +(col) "`%:` is left out: indentation covers it")
1147+ "`%=` is left out: indentation covers it; write a change, as m(a=b)"
1148+ =/ rest=tape ?~(cs ~ t.cs)
1150+ =/ n=@ud (run-len rest name-mask)
1151+ => (emit +(n) [%lit %term %tas (crip (scag n rest))])
1152+ ?. &(?=(^ cs) |((is-upper i.cs) =('_' i.cs))) cor
1153+ (err col "names are lowercase letters, digits and -")
1155+ =/ n=@ud (run-len rest digit-mask)
1156+ => (emit +(n) [%lit %term %ud (decimal (scag n rest))])
1157+ ?. &(?=(^ cs) (in i.cs term-tail-mask)) cor
1159+ "unexpected `{[i.cs ~]}`: a numeric term is `%` and digits only, as %1000"
1160+ (err +(col) "`%` begins a term, a loobean or `%/`")
1162+ :: +name: a name, or a wing, names joined by dots (2.6)
1170+ =/ n=@ud (run-len rest name-mask)
1171+ =/ nam=term (@tas (crip (scag n rest)))
1172+ =. rest (slag n rest)
1173+ ?: &(?=([@ @ *] rest) =('.' i.rest) (is-lower i.t.rest))
1174+ $(ns [nam ns], len (add +(len) n), rest t.rest)
1175+ =/ lex=lexeme ?~(ns [%name nam] [%wing (flop [nam ns])])
1176+ => (emit (add len n) lex)
1178+ ?: |((is-upper i.cs) =('_' i.cs))
1179+ (err col "names are lowercase letters, digits and -")
1181+ (err +(col) "a `.` in a wing is followed by a name")
1183+ :: +segment: after / or %/, a run of knot characters as one knot
1184+ :: token, tight (2.7)
1188+ =/ n=@ud (run-len cs knot-mask)
1189+ => ?:(=(0 n) cor (emit n [%knot (@ta (crip (scag n cs)))]))
1190+ :: what follows a segment: / ( [, a closer, a comment or a space
1191+ ?. &(?=(^ cs) (in i.cs seg-bad-mask)) cor
1194+ "unexpected `{[i.cs ~]}`: "
1195+ ?: =('?' i.cs) "a path carries no query, and a "
1196+ ?: =('#' i.cs) "a path carries no fragment, and a "
1198+ "segment holds lowercase letters, digits and - . ~ _"
1201+ :: +literal: a number or a ~ literal, the next n characters, read by
1202+ :: Hoon's own parser, nuck:so (2.8)
1207+ =/ run=tape (scag n cs)
1208+ =/ a=@ ?~(run 0 i.run)
1209+ =/ b=@ ?~(run 0 ?~(t.run 0 i.t.run))
1210+ :: 0b 0c 0i 0w begin auras Jibb doesn't write, whatever follows
1211+ ?: &(=('0' a) |(=('b' b) =('c' b) =('i' b) =('w' b)))
1212+ (err +(col) "Jibb writes no @u{[b ~]} literals")
1213+ :: a year BC that Hoon's dates can't hold, which its date code
1214+ :: crashes on: an error at the - that makes it BC
1215+ =/ bc=(unit [at=@ud y=@ud]) (year-bc col run)
1216+ ?: &(?=(^ bc) =(0 y.u.bc))
1217+ (err at.u.bc "there is no year 0 BC: 1 BC is ~1-, as ~1-.1.1")
1218+ ?: &(?=(^ bc) (gth y.u.bc 292.277.024.401))
1219+ (err at.u.bc "Hoon's dates begin in 292.277.024.401 BC")
1220+ =/ ed (nuck:so [[lin col] run])
1221+ ?~ q.ed (err q.p.ed (literal-msg run %$))
1222+ =/ used=@ud (sub n (lent q.q.u.q.ed))
1223+ =/ con=coin p.u.q.ed
1224+ ?: =(0 used) (err col (literal-msg run %$))
1225+ ?. ?=([%$ *] con) (err +(col) "Jibb writes no such literal")
1226+ =/ form=(unit lit-form) (lit-form-of p.p.con)
1229+ %+ weld "Jibb writes no @{(trip p.p.con)} literals"
1230+ ?:(=(%t p.p.con) "; strings are \"…\"" "")
1231+ =/ lex=lexeme [%lit u.form p.p.con q.p.con]
1233+ ?. =(%ta p.p.con) (emit n lex)
1234+ :: a ~. knot is followed as a path segment is (2.7)
1236+ ?. &(?=(^ cs) (in i.cs seg-bad-mask)) cor
1238+ %+ weld "unexpected `{[i.cs ~]}`: "
1239+ "a knot holds lowercase letters, digits and - . ~ _"
1240+ :: the error goes where Hoon's parser stopped looking; after a ship,
1241+ :: or a bare ~ that a letter follows, at the first letter that no
1242+ :: syllable continues (+ship-stop), or past a duration's unit, d h m
1243+ :: or s, if that is later (2.8)
1245+ =? at &(=('~' a) (is-lower b) ?=(?(%n %p) p.p.con))
1246+ =/ dur=@ud ?:(|(=('d' b) =('h' b) =('m' b) =('s' b)) 2 1)
1247+ (min at (add col (max dur (ship-stop run))))
1248+ (err:(emit used lex) at (literal-msg run p.p.con))
1250+ :: +text: a string's text, from its opening " or the } that closes
1251+ :: an interpolation, through the " or { that ends it (2.9). mid: it
1252+ :: resumes after a }, so it is str-mid or str-tail, not str-head or str.
1257+ =/ beg=loc [lin col]
1258+ =/ rest=tape ?~(cs ~ t.cs)
1262+ ?: |(?=(~ rest) =('\0a' i.rest))
1263+ (err at "this string has no closing `\"` on its line")
1266+ =/ txt=@t (crip (flop acc))
1267+ (put beg [lin +(at)] ?:(mid [%str-tail txt] [%str txt]) t.rest)
1269+ =/ txt=@t (crip (flop acc))
1270+ => (put beg [lin +(at)] ?:(mid [%str-mid txt] [%str-head txt]) t.rest)
1273+ ?~ t.rest (err +(at) bad-escape)
1275+ ?: |(=('\\' d) =('"' d) =('{' d))
1276+ $(rest t.t.rest, at (add 2 at), acc [d acc])
1277+ =/ bd=(unit tape) (bad-char d)
1278+ ?^ bd (err +(at) u.bd)
1279+ ?. (is-hex d) (err +(at) bad-escape)
1280+ ?~ t.t.rest (err (add 2 at) bad-hex)
1282+ =/ be=(unit tape) (bad-char e)
1283+ ?^ be (err (add 2 at) u.be)
1284+ ?. (is-hex e) (err (add 2 at) bad-hex)
1288+ acc [(add (mul 16 (hex-val d)) (hex-val e)) acc]
1290+ ?: =(1 (cut 0 [ch 1] bad-mask)) (err at (need (bad-char ch)))
1293+ at ?:(=(1 (cut 0 [ch 1] cont-mask)) at +(at))
1298+:: +bad-char: the error for a character that no text may hold (2.2)
1303+ ?: =(9 c) `"a tab: Jibb uses spaces only"
1304+ ?: =(13 c) `"a carriage return: Jibb lines end with a line feed alone"
1305+ ?: |((lth c 32) =(127 c)) `"a control character"
1308+++ non-ascii "non-ASCII text outside a string or comment"
1309+++ bad-escape "a `\\` begins `\\\\`, `\\\"`, `\\\{` or two hex digits"
1310+++ bad-hex "a `\\` escape takes two lowercase hex digits, as `\\0a`"
1312+:: Character classes are masks, bit c set for each character c in the
1313+:: class, so a test is one (cut 0 [c 1] mask) rather than a chain of
1316+++ lower-mask 0x7ff.fffe.0000.0000.0000.0000.0000.0000 :: a-z
1317+++ upper-mask 0x7ff.fffe.0000.0000.0000.0000 :: A-Z
1318+++ digit-mask 0x3ff.0000.0000.0000 :: 0-9
1319+++ hex-mask 0x7e.0000.0000.03ff.0000.0000.0000 :: 0-9 a-f
1320+++ name-mask 0x7ff.fffe.0000.0000.03ff.2000.0000.0000 :: a-z 0-9 -
1321+++ knot-mask 0x47ff.fffe.8000.0000.03ff.6000.0000.0000 :: and . ~ _
1322+++ number-mask 0x7ff.fffe.07ff.fffe.03ff.4000.0000.0000 :: a-z A-Z 0-9 .
1323+++ bad-mask 0x8000.0000.0000.0000.0000.0000.ffff.fbff :: 2.2
1324+:: after a path segment (2.7): printable, but not a knot character,
1326+++ seg-bad-mask 0x1800.0001.57ff.ffff.f800.1cfe.0000.0000
1327+:: after a numeric term: a-z A-Z _ - .
1328+++ term-tail-mask 0x7ff.fffe.87ff.fffe.0000.6000.0000.0000
1330+:: +cont-mask: UTF-8 continuation bytes, 10xx.xxxx, which start no
1334+ 0xffff.ffff.ffff.ffff.0000.0000.0000.0000.0000.0000.0000.0000
1336+++ in |=([c=@ m=@] =(1 (cut 0 [c 1] m)))
1337+++ is-lower |=(c=@ (in c lower-mask))
1338+++ is-upper |=(c=@ (in c upper-mask))
1339+++ is-digit |=(c=@ (in c digit-mask))
1340+++ is-hex |=(c=@ (in c hex-mask))
1341+++ hex-val |=(c=@ ^-(@ ?:((is-digit c) (sub c '0') (sub c 87))))
1343+:: +next-col: the column after character c at column col
1348+ ?:(=(1 (cut 0 [c 1] cont-mask)) col +(col))
1350+:: +run-len: how many characters at the head of cs are in class m
1358+ ?. =(1 (cut 0 [`@`i.cs 1] m)) n
1361+:: +decimal: digits as a number
1366+ (roll ds |=([d=@ a=@ud] (add (mul 10 a) (sub d '0'))))
1368+:: +lit-form-of: the literal rule of a dime's aura, if Jibb takes it (2.8)
1384+:: +literal-msg: why a literal stops where nuck:so stops (2.8), told by
1385+:: its first characters and the aura of what nuck:so read
1388+ |= [run=tape aura=@ta]
1390+ =/ a=@ ?~(run 0 i.run)
1391+ =/ b=@ ?~(run 0 ?~(t.run 0 i.t.run))
1393+ ?: &(=('0' a) =('x' b))
1394+ %+ weld "a hexadecimal number is 0x and digits 0-9 and a-f, "
1395+ "in groups of four joined by dots, as 0x1.ffff"
1396+ ?: &(=('0' a) =('v' b))
1397+ %+ weld "a base-32 number is 0v and digits 0-9 and a-v, "
1398+ "in groups of five joined by dots, as 0v5.fdf99"
1399+ "a decimal is digits in groups of three joined by dots, as 1.000"
1400+ ?: |(=(%da aura) &(=(%n aura) (is-digit b)))
1401+ %+ weld "a date is ~year.month.day, then optionally "
1402+ "..hours.minutes.seconds, as ~2026.10.23..14.30.00"
1404+ "a duration is units of d, h, m and s joined by dots, as ~h1.m30"
1406+ %+ weld "a ship is syllables in pairs joined by hyphens, "
1407+ "as ~zod, ~marzod or ~sampel-palnet"
1408+ %+ weld "`~` alone is null, and nothing here reads as a ship, "
1409+ "date, duration or knot"
1411+:: +year-bc: a ~ literal run's year and the column of its -, if it
1412+:: starts as a date BC does, ~ then digits Hoon reads as a year, then -
1414+:: Hoon's +year crashes on a year BC of 0 or past 292.277.024.401, as
1415+:: in ~0-.1.1, so these never reach nuck:so (2.8).
1418+ |= [col=@ud run=tape]
1419+ ^- (unit [at=@ud y=@ud])
1420+ ?. ?=([%'~' @ *] run) ~
1422+ =/ ds=tape (scag (run-len tr digit-mask) tr)
1424+ :: a year is 0 or has no leading 0, as dim:ag reads it
1425+ ?: &(=('0' i.ds) ?=(^ t.ds)) ~
1426+ =/ rest=tape (slag (lent ds) tr)
1427+ ?. ?=([%'-' *] rest) ~
1428+ `[(add col +((lent ds))) (decimal ds)]
1430+:: +ship-stop: how many characters of a ~ literal's run, from its ~,
1431+:: could still begin a ship (2.8)
1433+:: Hoon's parser reads a syllable's three letters before it looks
1434+:: them up, so it stops past the first letter that no syllable
1435+:: continues; this stops at that letter. It takes a galaxy's suffix,
1436+:: or words of a prefix and a suffix joined by - or --, as many as
1437+:: come. That is a few more texts than Hoon reads, so it never stops
1438+:: before Hoon's parser could have gone on.
1443+ =/ cs=tape ?~(run ~ t.run)
1444+ =/ pre=@ud (syllable cs sis:po)
1445+ :: a galaxy's suffix, or a word's prefix
1446+ ?. =(3 pre) +((max pre (syllable cs dex:po)))
1450+ :: the word's suffix
1451+ =/ suf=@ud (syllable cs dex:po)
1452+ ?. =(3 suf) (add i suf)
1455+ :: then - or --, and the next word's prefix
1456+ ?. &(?=(^ cs) =('-' i.cs)) i
1457+ =/ h=@ud ?:(&(?=([@ @ *] cs) =('-' i.t.cs)) 2 1)
1458+ =/ word=tape (slag h `tape`cs)
1459+ =/ nex=@ud (syllable word sis:po)
1460+ ?. =(3 nex) :(add i h nex)
1461+ $(i :(add i h 3), cs (slag 3 word))
1463+:: +syllable: how many of cs's first three characters begin one of the
1464+:: 256 syllables in syls, the prefixes or suffixes of +po
1467+ |= [cs=tape syls=@t]
1469+ =/ l=tape (scag 3 cs)
1473+ =/ head=@ (crip (scag +(k) l))
1474+ ?. (lien (gulf 0 255) |=(i=@ =(head (cut 3 [(mul 3 i) +(k)] syls))))
1478+:: +no-level: the error for a line that returns to no open level (2.4),
1479+:: naming the open levels' columns, outermost first
1485+ "this line's indentation matches no enclosing line: indent it to column "
1486+ (join-or (turn (flop lev) |=(c=@ud (a-co:co c))))
1488+:: +join-or: words as "a", "a or b", "a, b or c"
1495+ ?~ t.t.ws "{i.ws} or {i.t.ws}"
1496+ "{i.ws}, {$(ws t.ws)}"
647+| %printers
1498+| %printers
649:: +show-tokens: one token per line (format: section 4.1)
1500:: +show-tokens: one token per line (format: section 4.1)
@@ -657,7 +1508,7 @@
657 |= t=token
1508 |= t=token
660− (show-loc beg.at.t)
1511+ (show-loc (where t))
662 ?:(tight.t "tight " "")
1513 ?:(tight.t "tight " "")
663 (show-lexeme lex.t)
1514 (show-lexeme lex.t)
@@ -683,6 +1534,7 @@
683 %knot "knot {(trip p.x)}"
1534 %knot "knot {(trip p.x)}"
684 %aura "aura @{(trip p.x)}"
1535 %aura "aura @{(trip p.x)}"
685 %tvar "tvar {(trip p.x)}"
1536 %tvar "tvar {(trip p.x)}"
1537+ %error "ERROR {p.x}"
688:: +show-module: the tree as S-expressions (format: section 4.2)
1540:: +show-module: the tree as S-expressions (format: section 4.2)
@@ -700,6 +1552,8 @@
701:: +show-error: "<file>:<line>:<col>: <message>", the line, a caret
1553:: +show-error: "<file>:<line>:<col>: <message>", the line, a caret
1555+:: In the excerpt a control character prints as a space (4.3).
703++ show-error
1557++ show-error
704 |= [src=@t err=error]
1558 |= [src=@t err=error]
@@ -707,6 +1561,9 @@
707 =/ text=@t
1561 =/ text=@t
708 ?: |(=(0 line.err) (gth line.err (lent lines))) ''
1562 ?: |(=(0 line.err) (gth line.err (lent lines))) ''
709 (snag (dec line.err) lines)
1563 (snag (dec line.err) lines)
1565+ %+ turn (trip text)
1566+ |=(c=@tD ?:(|((lth c 32) =(127 c)) ' ' c))
712 (show-file file.err)
1569 (show-file file.err)
@@ -717,7 +1574,7 @@
720− (trip text)
1577+ excerpt
722 (reap (dec (max 1 col.err)) ' ')
1579 (reap (dec (max 1 col.err)) ' ')
@@ -725,15 +1582,19 @@
726:: +show-found: a token as error messages name it
1583:: +show-found: a token as error messages name it
1585+:: A layout token with an empty span is at the end of the file (2.5).
728++ show-found
1587++ show-found
729 |= t=token
1588 |= t=token
731 =/ x=lexeme lex.t
1590 =/ x=lexeme lex.t
1592+ ?: &(?=(?(%indent %dedent %newline) x) =(beg.at.t end.at.t))
1593+ "the end of the file"
733 ?+ x "`{(trip x)}`"
1594 ?+ x "`{(trip x)}`"
734 %indent "a deeper line"
1595 %indent "a deeper line"
735− %dedent "the end of the line"
1596+ %dedent "a shallower line"
736− %newline "the end of the line"
1597+ %newline "a line at the same depth"
739 %name "`{(trip p.x)}`"
1600 %name "`{(trip p.x)}`"
@@ -746,17 +1607,19 @@
746 %knot "`{(trip p.x)}`"
1607 %knot "`{(trip p.x)}`"
747 %aura "`@{(trip p.x)}`"
1608 %aura "`@{(trip p.x)}`"
748 %tvar "`{(trip p.x)}`"
1609 %tvar "`{(trip p.x)}`"
751−:: +where: where an error at a token points
1613+:: +where: where a token is, for errors and the token printer
753−:: At the token's start; for NEWLINE and DEDENT that is the end of the
1615+:: At the token's start; for a layout token, at the end of its span:
754−:: previous line's code. For INDENT, the start of the deeper line.
1616+:: the start of the line it begins, or at the end of the file just
1617+:: past the last token (2.4, 2.5).
757 |= t=token
1620 |= t=token
759− ?: ?=(%indent lex.t) end.at.t
1622+ ?: ?=(?(%indent %dedent %newline) lex.t) end.at.t
760 beg.at.t
1623 beg.at.t
762++ show-loc
1625++ show-loc
@@ -984,6 +1847,7 @@
984 %bind (nod "bind" at.s (trip name.s) ~)
1847 %bind (nod "bind" at.s (trip name.s) ~)
985 %face (nod "face" at.s (trip name.s) ~[(sexp-shape shape.s)])
1848 %face (nod "face" at.s (trip name.s) ~[(sexp-shape shape.s)])
986 %pun (nod "pun" at.s (trip name.s) ~)
1849 %pun (nod "pun" at.s (trip name.s) ~)
1850+ %cell (nod "cell" at.s "" (turn items.s sexp-sample))
987 %bare (sexp-shape shape.s)
1851 %bare (sexp-shape shape.s)
@@ -1042,7 +1906,8 @@
1042 %& (next p.res)
1906 %& (next p.res)
1045−:: +span-of: the span of any node: every node is [%tag at=span …]
1909+:: +span-of: the span of a tagged node, [%tag at=span …]; an +item,
1910+:: +edit or +arm has its span at its head, at.node
1047++ span-of
1912++ span-of
1048 |* node=[@ span *]
1913 |* node=[@ span *]
@@ -1051,10 +1916,35 @@
1052:: +fail: the error "expected <want>, found <the next token>"
1917:: +fail: the error "expected <want>, found <the next token>"
1919+:: At an error token, the layout pass's error instead (2.0). The token
1920+:: list always ends with NEWLINE or an error token (2.5), and the
1921+:: parser stops at either, so it never runs out; if it did, the error
1922+:: would sit at 1:1.
1055 |= [file=path toks=(list token) want=tape]
1925 |= [file=path toks=(list token) want=tape]
1056 ^- [%| p=error]
1926 ^- [%| p=error]
1057 ?~ toks [%| file 1 1 "expected {want}, found the end of the tokens"]
1927 ?~ toks [%| file 1 1 "expected {want}, found the end of the tokens"]
1928+ ?: ?=([%error *] lex.i.toks)
1929+ [%| file line.beg.at.i.toks col.beg.at.i.toks p.lex.i.toks]
1058 =/ l=loc (where i.toks)
1930 =/ l=loc (where i.toks)
1059 [%| file line.l col.l "expected {want}, found {(show-found i.toks)}"]
1931 [%| file line.l col.l "expected {want}, found {(show-found i.toks)}"]
1933+:: +fail-space: a tight form found a space before its next token
1935+:: The error sits where the space starts, just past prev, the token
1936+:: that needs its next one at once (section 3, Errors). If the next
1937+:: token is a layout or error token, it is +fail's error instead.
1940+ |= [file=path prev=token toks=(list token) want=tape]
1943+ ?=(?(%indent %dedent %newline) lex.i.toks)
1944+ ?=([%error *] lex.i.toks)
1946+ (fail file toks want)
1948+ :+ line.end.at.prev col.end.at.prev
1949+ "expected {want} right after {(show-found prev)}, found a space"