@@ -7,7 +7,7 @@
77:: ship builds it as /lib/jibb.hoon.
88::
99:: API
10−:: +layout [file=path src=@t] (each (list token) error)
10+:: +layout [file=path src=@t] (list token)
1111:: +parse [file=path src=@t] (each module error)
1212:: +parse-tokens [file=path toks=(list token)] (each module error)
1313:: +golden [file=path src=@t] [tokens=@t tree=@t]
@@ -15,13 +15,18 @@
1515:: +show-module module @t
1616:: +show-error [src=@t error] @t
1717:: A file is a Clay path whose last segment is the extension, as the
18−:: golden runners pass /tests/golden/<case>/source/jibb.
19−::
20−:: Status: the types, the printers and +golden work. +layout and
21−:: +parse-tokens are stubs that return an error; they are implemented
22−:: next, against the design below. Choices the doc leaves open are marked
23−:: "(doc question)": they follow the most likely reading and are listed,
24−:: with the doc's words, for the doc's author to settle.
18+:: golden runners pass /tests/golden/<case>/source/jibb. +layout never
19+:: fails: an error it finds is its last token (2.0), and +layout-error
20+:: takes it out.
21+::
22+:: Status: the types, the printers, +golden and the layout pass
23+:: (+layout, +lexer) work. +parse-tokens is a stub that returns an
24+:: error; it is implemented next, against the design below. Choices the
25+:: doc leaves open are marked "(doc question)": they follow the most
26+:: likely reading and are listed, with the doc's words, for the doc's
27+:: author to settle. The golden corpus's "expect:" lines give each
28+:: error's position. The layout pass's errors sit there, except where
29+:: the parser finds an earlier error first, as in `++ 2nd 5` (2.0, 2.8).
2530::
2631:: ===================================================================
2732:: 1. Positions
@@ -38,9 +43,15 @@
3843:: has beg = end). Nodes keep their ends because the checker's errors in
3944:: milestone 2 name both conflicting sites with excerpts, and an excerpt
4045:: that underlines a whole expression, as Elm and Rust do, needs its end;
41−:: so does editor tooling later. Ends cost the parser nothing: a node ends
42−:: where its last child or closing token ends. The printed tree shows
43−:: starts only, and milestone 1's errors use starts only.
46+:: so does editor tooling later. A node starts at its first token and
47+:: ends where its last token ends: its closing token if it has one, else
48+:: its last child's last token, never a layout token. So an open
49+:: bracket, a body, ?-'s arms and $?'s cases end with their last line's
50+:: code, not at the line that closes them. Every node's span holds its
51+:: children's spans, in source order. The printed tree shows starts
52+:: only, and milestone 1's errors use starts only, so ends are first
53+:: tested by milestone 2's excerpts; the parser's own tests should check
54+:: the two rules above on every golden tree.
4455::
4556:: ===================================================================
4657:: 2. The layout pass (+layout)
@@ -49,16 +60,30 @@
4960:: +layout turns the text into tokens one line at a time: no token
5061:: crosses a line break, and only layout tokens join lines.
5162::
63+:: 2.0 Errors. +layout never fails. Where the text breaks a rule of this
64+:: section, it emits one error token, [%error msg], at the first
65+:: character that breaks it, and stops reading. The parser accepts no
66+:: error token, so it reports whichever comes first, its own error or
67+:: this one (section 3, Errors). Every error, from either stage, points
68+:: at the first character where the text stops being the start of any
69+:: valid module: everything before it could still be right. Hoon's
70+:: parser reports errors the same way, and the golden corpus's expect
71+:: lines assume it. It also puts `++ x ]` on line 1 before a bad string
72+:: on line 9, which a lexer that failed first would report instead. The
73+:: positions below follow from it. Messages quote code in backticks, as
74+:: the parser's do (section 3, Errors).
75+::
5276:: 2.1 Lines. The text is split at every \0a. A final \0a ends the last
5377:: line rather than starting an empty one. Lines are numbered from 1.
5478::
5579:: 2.2 Characters. Anywhere in the text, strings and comments included,
56−:: these are errors: tab (9), carriage return (13), every other byte
57−:: below 32 except \0a, and 127. Jibb, like Hoon, spaces with spaces
58−:: only ("tab: Jibb uses spaces only"). Bytes from 128 up (UTF-8) are
59−:: allowed only inside strings and comments ("non-ASCII text outside a
60−:: string or comment"). Outside strings and comments, see 2.6 for which
61−:: printable characters start a token.
80+:: these are errors at the character: tab (9), "a tab: Jibb uses spaces
81+:: only"; carriage return (13), "a carriage return: Jibb lines end with
82+:: a line feed alone"; every other byte below 32 except \0a, and 127, "a
83+:: control character". Bytes from 128 up (UTF-8) are allowed only inside
84+:: strings and comments ("non-ASCII text outside a string or comment").
85+:: Outside strings and comments, see 2.6 for which printable characters
86+:: start a token.
6287::
6388:: 2.3 Blank and comment-only lines. A line of spaces only, or whose first
6489:: non-space characters are ::, emits nothing and changes no level. Its
@@ -66,27 +91,31 @@
6691::
6792:: 2.4 Levels. The stack holds the columns of the open levels, innermost
6893:: first, and starts as ~[1]. For every other line let c be the column of
69−:: its first non-space character; then:
70−:: - The first such line needs c = 1, else the error "the first line is
71−:: indented: top-level lines start at column 1".
94+:: its first non-space character. The first such line emits no layout
95+:: token and needs c = 1, else the error "the first line is indented:
96+:: top-level lines start at column 1". Each later one emits:
7297:: - c > top: push c, emit INDENT.
7398:: - c = top: emit NEWLINE.
7499:: - c < top: while c < top, pop and emit DEDENT. Then c must equal the
75100:: new top, else the error "this line's indentation matches no
76−:: enclosing line: indent it to column 1 or 5" (the stack's columns,
77−:: outermost first); emit NEWLINE.
78−:: Then the line's tokens (2.6). A layout token's span runs from just past
79−:: the previous token, on an earlier line, to column c of this line:
80−:: its start is where the previous line's code ended, where "found the
81−:: end of the line" errors point, and its end is where the new line
82−:: starts, where "found a deeper line" errors point (see +where).
83−::
84−:: 2.5 End of file. If any token was emitted: pop every level above 1,
85−:: emitting DEDENT for each, then emit one NEWLINE, all with empty spans
86−:: just past the last token. So every import and definition, the last
87−:: one included, is followed by NEWLINE, as `module = { import NEWLINE }
88−:: { definition NEWLINE }` needs. A file without tokens (empty, or only
89−:: blank and comment lines) gives no tokens: an empty module.
101+:: enclosing line: indent it to column 1, 5 or 7" (the stack's
102+:: columns before the pops, outermost first); emit NEWLINE.
103+:: Then the line's tokens (2.6). These errors, and a bad character (2.2)
104+:: at column c, sit at column c and come before the line's layout
105+:: tokens. A layout token's span runs from just past the previous token,
106+:: on an earlier line, to column c of this line. Errors and the token
107+:: printer place a layout token at its span's end, the start of the line
108+:: it begins (+where): the line's column is what made it INDENT, NEWLINE
109+:: or DEDENT, and until that line, a deeper one could still have come.
110+::
111+:: 2.5 End of file. If any token was emitted and no error token: pop
112+:: every level above 1, emitting DEDENT for each, then emit one NEWLINE,
113+:: all with empty spans just past the last token. They are the only
114+:: layout tokens with empty spans, which is how errors name "the end of
115+:: the file". So every import and definition, the last one included, is
116+:: followed by NEWLINE, as `module = { import NEWLINE } { definition
117+:: NEWLINE }` needs. A file without tokens (empty, or only blank and
118+:: comment lines) gives no tokens: an empty module.
90119::
91120:: The layout pass knows nothing of brackets: it emits INDENT, NEWLINE and
92121:: DEDENT by column alone. The parser decides what takes an indented
@@ -95,60 +124,102 @@
95124:: closed on one line is just tokens.
96125::
97126:: 2.6 Tokens. After the indentation, tokens are read left to right;
98−:: spaces separate them and are dropped. A token is tight when it
99−:: follows the previous token on the same line with no space between;
100−:: the first token of a line never is. Tightness is how the parser tells
101−:: the tight forms apart: m( from m (, a+b from a + b, `x from ` x,
102−:: |=%got from |= %got, name=value from name = value. What a token is
103−:: depends on its first characters:
127+:: spaces separate them and are dropped, spaces at the end of a line too.
128+:: A token is tight when it follows the previous token on the same line
129+:: with no space between; the first token of a line never is. Tightness
130+:: is how the parser tells the tight forms apart: m( from m (, a+b from
131+:: a + b, `x from ` x, |=%got from |= %got, name=value from name = value.
132+:: What a token is depends on its first characters. An error "at 2nd" is
133+:: at the token's second character, the first that no Jibb token allows
134+:: after its first:
104135::
105136:: :: a comment: the rest of the line is skipped
106137:: ( ) [ ] ` = * glyphs
107138:: ~[ ~& glyphs
108−:: ~ a ~ literal: ~ and the run of [a-z0-9._~-] after it;
109−:: a bare ~ is null (2.8)
139+:: ~ a ~ literal: ~ and the run of [a-z0-9._~-] after it,
140+:: read as 2.8 says; a bare ~ is null, [%null %n 0]
110141:: %/ glyph, then a path segment may follow (2.7)
111−:: %.y %.n loobean literals, [%f 0] and [%f 1]
142+:: %.y %.n loobean literals, [%f 0] and [%f 1]; %. then anything
143+:: else is an error at 3rd, "the loobeans are `%.y` and
144+:: `%.n`"
112145:: %& %| term literals, [%f 0] and [%f 1]
113−:: %a... term literal: % and a name [a-z][a-z0-9-]*, [%tas name]
146+:: %a... term literal: % and a name [a-z][a-z0-9-]*, [%tas name];
147+:: a capital or _ right after it is an error there,
148+:: "names are lowercase letters, digits and -"
114149:: %0... term literal: % and digits [0-9]+, [%ud value]
115150:: (doc question: digits only, as the EBNF says, so
116−:: %1000 is a term and %1.000 an error, unlike Hoon)
117−:: /- /% /= import glyphs, when followed by a space or the end
118−:: of the line
151+:: %1000 is a term and %1.000 an error at the dot); a
152+:: letter, _, - or . right after it is an error there,
153+:: "unexpected `a`: a numeric term is `%` and digits
154+:: only, as %1000"
155+:: % anything else after it: error at 2nd, "`%` begins a
156+:: term, a loobean or `%/`"
157+:: /- /% /= import glyphs at column 1, where imports go; anywhere
158+:: else / is a path's, so ++ x /- is the path whose
159+:: one segment is -, as the EBNF's knot allows
119160:: / glyph, then a path segment may follow (2.7)
120161:: ++ +$ + glyphs
121162:: |=( |= |$ glyphs
122−:: $?( $? $-( glyphs; $- without ( is the error "$- has only the
123−:: wide form $-(sample product)"
163+:: $?( $? $-( glyphs; $- followed by anything but ( is an error
164+:: at 3rd, "`$-` has only the wide form `$-(sample
165+:: product)`"
124166:: ^- ?- !! ? glyphs (? alone is the loobean mold)
125167:: @ aura: @, then [a-z]*, then at most one [A-Z]; the
126−:: token holds the text after @. Auras starting @s or @r
127−:: are errors: "signed and floating-point auras are left
128−:: out of Jibb"
129−:: A-Z type variable: one capital; a letter or digit right
130−:: after it is the error "a type variable is one capital
131−:: letter"
168+:: token holds the text after @. An aura starting @s or
169+:: @r is an error at 2nd: "signed and floating-point
170+:: auras are left out of Jibb"
171+:: A-Z type variable: one capital. A letter or digit right
172+:: after it is an error there, "a type variable is one
173+:: capital letter", after the type variable's token
132174:: a-z name [a-z][a-z0-9-]*, or a wing: two or more names
133−:: joined by dots. A . must be followed by [a-z] ("a . in
134−:: a wing is followed by a name"); a capital or _ right
135−:: after a name is the error "names are lowercase letters,
136−:: digits and -"
175+:: joined by dots. Right after the name or wing, which
176+:: is emitted first, a capital or _ is an error there,
177+:: "names are lowercase letters, digits and -", and so
178+:: is a . not followed by [a-z], at the character after
179+:: the dot, "a `.` in a wing is followed by a name"
137180:: 0-9 number literal: the run of [0-9a-zA-Z.] (2.8)
138181:: " string (2.9)
139182:: } ends the innermost interpolation (2.9), else an error
140−:: anything else an error naming the character: "unexpected `,`"
141−::
142−:: Hoon runes the doc leaves out are errors with the doc's reason, found
143−:: where the token would start: |% |_ |* "modules are records"; $% "$?
144−:: covers unions"; ?+ ?: "?- is the branch"; .= ?! and !x "eq and not
145−:: are gates"; =/ "++ defines values"; |- $( "gates recurse by name"; ~|
146−:: "~& covers debugging"; => =< .* and a:b "names never shadow, and .
147−:: reads imports too"; !> /+ "code also runs in the browser"; ; "layout
148−:: is Jibb's elements"; :~ :* %: %= == -- "indentation covers them"; '
149−:: "there is no '...' form: strings are \"...\"". Wide forms the doc
150−:: leaves out, ^-(...) and ?-(...), lex fine and fail in the parser: a
151−:: rune is followed by a space.
183+:: at it, "this `}` closes no `{`"
184+:: | $ ^ followed by nothing above or in the table below:
185+:: an error at 2nd, "`|` begins `|=`, `|=(` or `|$`",
186+:: "`$` begins `$?`, `$?(` or `$-(`", "`^` begins `^-`"
187+:: : followed by anything but :, an error at 2nd: the
188+:: table's :~ :* and a:b, else "a comment starts with
189+:: `::`"
190+:: anything else an error at it, naming it: "unexpected `,`", with a
191+:: hint where one helps (+unexpected): for , . - and <,
192+:: what Jibb writes instead
193+::
194+:: Hoon forms the doc leaves out are errors with the doc's reason, as
195+:: "`|%` is left out: modules are records". Each points where the text
196+:: stops being Jibb: at its second character when its first can begin
197+:: Jibb there, else at its first.
198+::
199+:: |% |_ |* 2nd modules are records
200+:: |- 2nd gates recurse by name
201+:: $( 1st gates recurse by name; no expression starts $
202+:: $% 2nd $? covers unions
203+:: ?+ ?: 2nd ?- is the branch (?:: is ? and a comment)
204+:: ?! 2nd eq and not are gates
205+:: .= 1st eq and not are gates
206+:: !x 2nd not is a gate: ! then anything but ! or >
207+:: !> /+ 2nd code also runs in the browser
208+:: =< => .* 1st names never shadow, and . reads imports too
209+:: a:b 2nd of the :, right after a name: names never
210+:: shadow, and . reads imports too; write a.b
211+:: :~ :* 2nd indentation covers them; write ~[…] or […]
212+:: %: %= 2nd indentation covers them
213+:: == -- 1st the closers are left out: indentation covers
214+:: them
215+:: ~| 2nd ~& covers debugging
216+:: ; 1st Sail's ; runes are left out: layout is Jibb's
217+:: elements
218+:: ' 1st there is no '…' form: strings are "…"
219+::
220+:: The parser finds the doc's other left-out forms (section 3, Errors):
221+:: =(a b), =/, *mold, ^-(…) and ?-(…). The lexer can't, since =( =/ and *
222+:: begin valid Jibb too: open=(not open.s), scry=/todos, a * mold.
152223::
153224:: 2.7 Paths. Right after a / or %/ glyph, a run of knot characters
154225:: [a-z0-9._~-] is one knot token, tight. So /posts/~zod is / knot / knot;
@@ -158,50 +229,85 @@
158229:: lexer tracks: segments such as logo.png, 1 or ~zod would otherwise lex
159230:: as a wing, a number or a ship. A / followed by anything else is just a
160231:: glyph: / alone is the empty path, and the parser rejects empty
161−:: segments such as //.
162−::
163−:: 2.8 Literals. The lexer takes the literal's characters first, then
164−:: reads them with Hoon's own parser, which must consume all of them:
165−:: 0-9 run (rush run nuck:so) must give [%$ %ud|%ux|%uv val]: 1.000,
166−:: 0x1f, 0v5.fdf99. So 1000 is an error, as in Hoon (groups
167−:: of three), and 0b1, 0w1, 0i1, 0c1 are errors. Hoon reads
168−:: 0x1F too.
169−:: ~ run a bare ~ is null, [%n 0]. Otherwise (rush run nuck:so) must
170−:: give [%$ %p|%da|%dr|%ta val]: ~zod, ~sampel-palnet,
171−:: ~2026.10.23..14.30.00, ~h1.m30, ~.my-post. ~~a, ~-a and
172−:: ~0v... are errors.
173−:: terms and loobeans as in 2.6.
232+:: segments such as //. After the glyph and its knot, if any, comes a /,
233+:: ( or [, a closer ) ] or }, a : (of a comment) or a space: any other
234+:: printable character is an error there, "unexpected `P`: a path
235+:: segment holds lowercase letters, digits and - . ~ _", which for ?
236+:: and # also says a path carries no query or fragment. No Jibb token
237+:: goes on a path that way, so /Posts and %/blog?page=2 fail at the P
238+:: and the ?. A ~. knot literal (2.8) gets the same check, "a knot
239+:: holds lowercase letters, digits and - . ~ _".
240+::
241+:: 2.8 Literals. The lexer takes a literal's run (2.6), then reads it
242+:: with Hoon's own parser, (nuck:so [[line col] run]), whose edge says
243+:: how much it read and how far it looked (p.edge):
244+:: - It read the whole run, as an aura Jibb takes: the token. Numbers
245+:: take %ud, %ux and %uv: 1.000, 0x1f, 0v5.fdf99 (Hoon reads 0x1F
246+:: too). ~ runs take %p, %da, %dr and %ta: ~zod, ~sampel-palnet,
247+:: ~2026.10.23..14.30.00, ~h1.m30, ~.my-post.
248+:: - It read the run or part of it as an aura Jibb doesn't take: 0w1,
249+:: 0i1, ~~a (a cord), ~-a (a @c). An error at 2nd, "Jibb writes no
250+:: @uw literals". A run starting 0b, 0c, 0i or 0w gets that error
251+:: before nuck:so reads it, since no Jibb literal goes on that way:
252+:: Hoon's parser would read 0b, or a 0c1 whose checksum fails, as
253+:: the number 0 and an error past the run.
254+:: - It read only part of the run, as an aura Jibb takes: that part is
255+:: a token, then an error token at p.edge, the furthest character
256+:: Hoon's parser looked at, where Hoon reports it too. 1000 is the
257+:: token 100 and an error at its fourth digit ("a decimal is digits
258+:: in groups of three joined by dots, as 1.000": each kind of literal
259+:: has its message, +literal-msg); 0v5.fdfz9 is 0v5 and an error at the
260+:: z; ~2026.10.29..14.30 is ~2026.10.29 and an error just past the
261+:: run. The token lets the parser report an earlier error first: in
262+:: `++ 2nd 5` it finds the number 2 where a name goes, at 1:4.
263+:: When it read a ship, or only the ~ of a run that goes on with a
264+:: letter, the error goes instead at the first letter that no
265+:: syllable continues (+ship-stop), or just past a duration's unit,
266+:: d h m or s, if that is later: Hoon's parser reads a syllable's
267+:: three letters before it looks them up, so it looks past that
268+:: letter. `++ x ~x` fails at the x, 1:7, as no syllable starts with
269+:: x, and ~sampel-palnez at the z.
270+:: A run that starts as a date BC does, ~ and a year then -, fails at the
271+:: - if the year is 0 or past 292.277.024.401, before nuck:so sees it:
272+:: Hoon's +year crashes on those years, as in ~0-.1.1 (+year-bc).
174273:: The token is [%lit [form aura val]]: form names the literal rule that
175−:: matched, aura and val are the dime Hoon's parser gives.
274+:: matched, aura and val are the dime Hoon's parser gives. Terms,
275+:: loobeans and the bare ~ are read by hand (2.6).
176276::
177277:: 2.9 Strings. A " starts a string, which ends on the same line. Up to
178278:: the closing ":
179279:: \\ \" \{ and \ with two lowercase hex digits are escapes, as in
180−:: Hoon's tapes (doc question: the doc names only \{ and \"); any
181−:: other \ is an error at the \
182−:: { starts an interpolation: the text so far is emitted as str-head
183−:: (at the opening ") or str-mid (at the } that ended the previous
184−:: interpolation); then tokens are lexed as usual, strings included,
185−:: up to the } that ends this interpolation, where the string resumes
186−:: the closing " emits str when there was no interpolation (at the
187−:: opening ") or str-tail (at the last })
280+:: Hoon's tapes (doc question: the doc names only \{ and \"). Any
281+:: other \ is an error at the first character that can't continue
282+:: the escape: the one after the \, or a bad second hex digit (2.2's
283+:: error if it is a tab or another control character)
284+:: { starts an interpolation. The text so far is emitted as str-head,
285+:: spanning from the opening " to just past the {, or as str-mid,
286+:: from the } that ended the previous interpolation to just past the
287+:: {. Then tokens are lexed as usual, strings included, up to the }
288+:: that ends this interpolation, where the string resumes
289+:: the closing " emits str when there was no interpolation, spanning
290+:: the whole string, or str-tail, from the last } to just past the "
291+:: any other character is text, } included, as in Hoon
188292:: So "Since {day}!" is str-head "Since ", name day, str-tail "!", and the
189293:: parser reads head expr { mid expr } tail. The lexer keeps a stack of
190294:: open interpolations, since strings nest inside them. Inside an
191−:: interpolation, :: is the error "a comment can't start inside {...}"
192−:: and { is an error; inside a string, :: is text. The end of the line
193−:: inside a string or an interpolation is an error at its " ("this
194−:: string has no closing \" on its line") or { ("this { has no }").
195−:: Token texts hold the decoded text. Spaces after { and before } are
196−:: allowed, as after any opener.
295+:: interpolation a : is an error at it, "a comment can't start inside
296+:: `{…}`", and so is a {; inside a string, :: is text. A line that ends
297+:: inside a string or an interpolation is an error at its end, "this
298+:: string has no closing `\"` on its line" or "this `{` has no `}` on its
299+:: line". Token texts hold the decoded text. Spaces after { and before }
300+:: are allowed, as after any opener.
197301::
198302:: 2.10 What fails here. Bad characters (2.2), an indented first line and
199303:: a dedent to no open level (2.4), characters that start no token or a
200−:: left-out rune (2.6), malformed literals, auras, type variables and
201−:: wings (2.6, 2.8), and unclosed strings and interpolations, bad escapes
202−:: and stray { or } (2.9). Everything else is the parser's: brackets that
203−:: don't match or don't close, a missing or extra space, a deeper line
204−:: that nothing takes, a token where it can't go.
304+:: left-out form (2.6), malformed literals, auras, type variables and
305+:: wings (2.6, 2.8), a character that no token lets follow a path
306+:: segment, a ~. knot or a numeric term (2.6, 2.7), and unclosed strings
307+:: and interpolations, bad escapes and stray { or } (2.9). Everything
308+:: else is the parser's: brackets that don't match or don't close, a
309+:: missing or extra space, a deeper line that nothing takes, a token
310+:: where it can't go.
205311::
206312:: ===================================================================
207313:: 3. The parser (+parse-tokens)
@@ -221,7 +327,8 @@
221327:: name, then = tight a face (items, lines, fields, samples)
222328:: name, then + tight a pair
223329:: name or wing, then ( tight a change
224−:: |=, then a term tight a constructor; else the tall gate
330+:: |=, then a term tight a constructor; else, where an
331+:: expression starts, the tall gate
225332:: ++ or +$ a definition; else a body's final expr
226333:: INDENT a body, an open bracket's lines, arms
227334:: or cases; else the expression goes on
@@ -231,11 +338,18 @@
231338:: their failures carry a position but no expectation; over tokens they
232339:: would need a token "tape" and an expectation set bolted on. Recursive
233340:: descent is shorter, and each error names what its construct expected.
341+:: An operand that isn't where an expression starts (a call's argument,
342+:: an item, a pair's or unit's operand, the wide gate's body) can't be a
343+:: rune, so there |= is a constructor and needs its term at once:
344+:: (state.cart |= %got) is an error at the space after |=, 1:20. Loops
345+:: over many items (definitions, arms, lines, items) may recurse: vere's
346+:: stack takes ten million plain frames, so no Jibb file nests or runs
347+:: long enough to overflow it.
234348::
235349:: The EBNF and its nodes (see the types below):
236350:: module { import NEWLINE } { definition NEWLINE }, then the end
237351:: of the tokens; an import after a definition is the error
238−:: "imports come first"
352+:: "expected a definition, found `/-`; imports come first"
239353:: import /- name -> %sur; /% term -> %mark; /= name pathlit ->
240354:: %jibb. /% takes % and a name only: no mark is called %0,
241355:: %& or %| (doc question)
@@ -252,15 +366,19 @@
252366:: group ( expr ) -> %group of the expr. The open form ( {operand}
253367:: open -> %group of the call whose elements are the first
254368:: line's operands, then one per line; one element alone is
255−:: just grouped. Lines of an open group or list take no
256−:: name= (doc question)
369+:: just grouped. A ( followed by a rune is the closed form.
370+:: Lines of an open group or list take no name= (doc
371+:: question)
257372:: change every item and line has name= (the EBNF's "items named")
258373:: rune |= -> %gate; ^- -> %cast; ~& -> %trace; ?- -> %branch;
259−:: !! -> %crash. A bare name as the whole sample binds, in
260−:: |=(…) and in the tall |= alike; inside a cell sample a
261−:: bare name is a mold, as field says (doc question). !!
262−:: is not an operand, so [a !!] needs [a (!!)] (doc
263−:: question)
374+:: !! -> %crash. !! is not an operand, so [a !!] needs
375+:: [a (!!)] (doc question)
376+:: sample the sample of |= and |=(…): a field, except that a bare
377+:: name binds (%bind) wherever it stands in the sample: the
378+:: whole sample, as |= x and |=(p title.p), or a field of a
379+:: cell sample at any depth, as b in |= [a=@ b], which is
380+:: a %cell of samples (doc question). A name inside a mold,
381+:: as a in (list a), $-(a b) or $?(a b), is a mold
264382:: mold aura -> %aura; * -> %noun; !! -> %void; ? -> %flag;
265383:: ~ term loobean -> %const; ref -> %ref; typevar -> %tvar;
266384:: pathmold -> %path; ( ref mold … ) -> %apply; [ field … ]
@@ -280,7 +398,7 @@
280398:: expr } str-tail, where each expr is a full expression.
281399:: - Spaces, comments and blank lines: the layout pass, and Spacing
282400:: below.
283−:: - pathlit as an operand (above).
401+:: - pathlit as an operand (above), with / alone the empty path.
284402:: - knot may be empty: only / alone uses that, as the empty path; //
285403:: and a trailing / are errors (doc question).
286404:: - Open brackets in molds: mold has no open form, so a mold's [ or (
@@ -292,36 +410,67 @@
292410:: ^-, ~& and arms), the Runes prose says "inside a gate"; the parser
293411:: takes them in every body and leaves the rest to the checker (doc
294412:: question).
413+:: - Where errors point: the doc says only "line, column and an
414+:: excerpt"; 2.0 and Errors below say where.
295415::
296416:: Spacing. Two tokens on one line are separated by spaces, except:
297417:: - tight, always: name+operand (pair), `operand (unit), |=term
298418:: (constructor), wing( (change), name= and = value (faces, in items,
299−:: lines, fields and samples), =name (pun), and every token of a path
300−:: or application path after its first / or %/, typed segments
301−:: ([id=@ud]) included
419+:: lines, fields and samples), =name (pun); in a path or application
420+:: path, each / and the segment after it, and all of a typed segment,
421+:: [id=@ud]
302422:: - either way: right after an opener ( [ ~[ |=( $?( $-( or {, and
303−:: right before a closer ) ] or } (the theme example writes `[ space=`)
304−:: A missing space is an error ("expected a space between items, found
305−:: `[`"), and so is a space inside a tight form ("a pair has no spaces,
306−:: as in gap+%s2"). A rune with children, and an import glyph, is
307−:: followed by a space (or, for $? with its cases below, the end of the
308−:: line). This is how the parser makes "one space between items; more for
309−:: alignment" precise (doc question).
310−::
311−:: Errors. "expected <what>, found <token>", with an optional "; <hint>",
423+:: right before a closer ) ] or } (the theme example writes `[ space=`),
424+:: the ( and ) of an application path's segment included; inside
425+:: them the expression is spaced as anywhere else, as %/x/(f a)
426+:: A missing space is an error at the tight token, and so is a space
427+:: inside a tight form, at the space (Errors). A rune with children, and
428+:: an import glyph, is followed by a space (or, for $? with its cases
429+:: below, the end of the line). Items are separated by one or more
430+:: spaces; how many isn't checked. This is how the parser makes "one
431+:: space between items; more for alignment" precise (doc question).
432+::
433+:: Errors. The parser stops at the first token it can't accept and
434+:: reports "expected <what>, found <token>", with an optional "; <hint>",
312435:: lowercase, no final period. <what> uses the doc's words: "an
313−:: expression", "a mold", "a pattern", "a name", "`]`", "the end of the
314−:: line". <token> is +show-found: the token's text in backticks, "a
315−:: string", "a deeper line" for INDENT, "the end of the line" for NEWLINE
316−:: and DEDENT. The error sits where +where says: at the token's start, but
317−:: at the end of the previous line's code for NEWLINE and DEDENT, and at
318−:: the start of the deeper line for INDENT. Examples:
319−:: expected `]` to close the `[` at 3:5, found the end of the line
436+:: expression", "a mold", "a pattern", "a name", "`]`", "a space between
437+:: items". <token> is +show-found: the token's text in backticks, "a
438+:: string", and for layout tokens "a deeper line" (INDENT), "a line at
439+:: the same depth" (NEWLINE), "a shallower line" (DEDENT), or "the end of
440+:: the file" (an empty span, 2.5). Each error points at the first
441+:: character that can't begin a valid module (2.0):
442+:: - At the token's start; for a layout token, at the start of the
443+:: line it begins, or just past the last token at the end of the
444+:: file (+where). So `++ view` above `++ main view` is an error at
445+:: 2:1, the line that isn't deeper.
446+:: - Where a tight form needs its next token at once (after name+, `,
447+:: |= in an operand, name=, the = of a pun, a path's / and %/, a
448+:: typed segment's parts) and finds a space: at the end of the
449+:: previous token, where the space starts, "expected a term right
450+:: after `|=`, found a space" (+fail-space). A wing and a ( after a
451+:: space are no error: that is a call.
452+:: - At an error token: the lexer's position and message, unchanged.
453+:: - Three pattern errors point inside their token, where it stops
454+:: being a pattern: a date, duration or knot at its second character
455+:: (~ alone is null, and no ship goes on so), ~[ at its [, and a
456+:: string with {…} at its {.
457+:: In a bracket, a closer that doesn't match is "expected `]` to close
458+:: the `~[` at 1:6, found `)`", checked before spacing; after an item, a
459+:: tight token that isn't a closer is "expected a space between items,
460+:: found `M`". Hints name the left-out forms the lexer can't see (2.6):
461+:: where an expression starts, = followed tight by ( is "=(a b) is left
462+:: out: eq is a gate", = followed tight by / is "=/ is left out: ++
463+:: defines values", and * is "*mold is left out: no defaults"; ^- or ?-
464+:: followed tight by ( is "the wide form ^-(…) is left out: indentation
465+:: covers it". Examples:
466+:: expected `]` or an item on a deeper line, found a line at the same
467+:: depth
320468:: expected the end of the line, found a deeper line; only a bracket
321469:: open at the end of a line, or a rune's body, takes deeper lines
322470:: expected an expression, found `^-`; inside a call, a rune goes in
323471:: parentheses
324−:: a change's items are named, as in m(done=%.y)
472+:: expected a named item, found `x`; a change's items are named, as in
473+:: m(done=%.y)
325474:: Messages of milestone 1 are golden-tested, so their wording is fixed
326475:: by the tests once written.
327476::
@@ -331,30 +480,51 @@
331480::
332481:: Each printer returns a cord: lines joined by \0a, no final newline.
333482::
334−:: 4.1 Tokens (+show-tokens): one token per line, "<line>:<col>", then
335−:: "tight " if the token is tight, then the kind and the text. Kinds:
336−:: glyph (with its text), INDENT, DEDENT, NEWLINE, name, wing, lit (the
337−:: literal as Hoon prints its dime: 1.000, 0x1f, ~zod, ~h1.m30, ~.knot;
338−:: terms as %add, %0, %& and %|; loobeans %.y and %.n; null ~), str,
339−:: str-head, str-mid and str-tail (quoted, with \\ \" \{ and \XX
340−:: escapes), knot, aura (@ud), tvar. Layout tokens print their span's
341−:: start: the end of the previous line's code. For `++ main view`:
483+:: 4.1 Tokens (+show-tokens): one token per line, "<line>:<col>" at
484+:: +where, then "tight " if the token is tight, then the kind and the
485+:: text. Kinds: glyph (with its text), INDENT, DEDENT, NEWLINE, name, wing,
486+:: lit (the literal as Hoon prints its dime: 1.000, 0x1f, ~zod, ~h1.m30,
487+:: ~.knot; terms as %add, %0, %& and %|; loobeans %.y and %.n; null ~),
488+:: str, str-head, str-mid and str-tail (quoted, with \\ \" \{ and \XX
489+:: escapes), knot, aura (@ud), tvar, and ERROR with its message (+golden
490+:: prints the rendered error instead). A layout token prints at the
491+:: start of the line it begins, beside that line's first token; at the
492+:: end of the file, just past the last token. For `++ main view`:
342493::
343494:: 1:1 glyph ++
344495:: 1:4 name main
345496:: 1:9 name view
346497:: 1:13 NEWLINE
347498::
499+:: and for `++ init |= ~` with ` [[~ %.n] ""]` below it:
500+::
501+:: 1:1 glyph ++
502+:: 1:4 name init
503+:: 1:9 glyph |=
504+:: 1:12 lit ~
505+:: 2:3 INDENT
506+:: 2:3 glyph [
507+:: 2:4 tight glyph [
508+:: 2:5 tight lit ~
509+:: 2:7 lit %.n
510+:: 2:10 tight glyph ]
511+:: 2:12 str ""
512+:: 2:14 tight glyph ]
513+:: 2:15 DEDENT
514+:: 2:15 NEWLINE
515+::
348516:: 4.2 Trees (+show-module): S-expressions, (kind line:col fields kids),
349517:: with the node's start. A leaf is a node whose kids, if any, are atoms
350518:: (a string's quoted text, an application path's knots). A node whose
351519:: kids are all leaves prints on one line; otherwise each kid goes on its
352520:: own line, indented two more spaces, and the closing parens end the
353521:: last kid's line. (module …) always breaks. Bare fields and samples
354−:: print as their mold, unnamed items as their expr; strings print text
355−:: pieces quoted, between their interpolated exprs; application paths
356−:: print knots bare between their (expr) segments. The cart store from
357−:: the Grammar tab, its code block's first line being line 1:
522+:: print as their mold, and a cell sample as (cell …) of its samples, a
523+:: bare name there as (bind …); unnamed items print as their expr;
524+:: strings print text pieces quoted, between their interpolated exprs;
525+:: application paths print knots bare between their (expr) segments. The
526+:: cart store from the Grammar tab, its code block's first line being
527+:: line 1:
358528::
359529:: (module
360530:: (mold 2:1 state
@@ -400,7 +570,10 @@
400570:: source line, then a caret under the column (col - 1 spaces, then ^).
401571:: The file prints as a Clay path with its last segment as the extension:
402572:: /tests/golden/x/source/jibb is /tests/golden/x/source.jibb. A line
403−:: past the end of the text prints as an empty line.
573+:: past the end of the text prints as an empty line. In the excerpt every
574+:: control character, a tab or carriage return being the likely ones,
575+:: prints as a space, so the caret stays under its column and golden
576+:: files hold no raw control bytes.
404577::
405578:: /ui/basket.jibb:22:5: msg has no case %submitt
406579:: %submitt [m(note="") ~]
@@ -430,6 +603,9 @@
430603::
431604:: tight is %.y when the token follows the previous one on the same
432605:: line with no space between them; layout tokens are never tight.
606+:: A layout token's span runs from just past the previous token to the
607+:: start of the line it begins, or is empty at the end of the file
608+:: (2.4, 2.5). An error token is the last token, if any (2.0).
433609::
434610+$ token [at=span tight=? lex=lexeme]
435611::
@@ -445,6 +621,7 @@
445621 [%knot p=@ta] :: path segment
446622 [%aura p=@ta] :: @ud as 'ud', @ as ''
447623 [%tvar p=@t] :: M
624+ [%error p=tape] :: 2.0: what broke
448625 ==
449626::
450627:: +glyph: a token that is its text, and the layout tokens
@@ -481,7 +658,10 @@
481658+| %tree
482659::
483660:: The syntax tree. Every node is [%tag at=span …] with at least one
484−:: field after at, so +span-of finds the span of any node at +<.
661+:: field after at, so +span-of finds its span at +<, except +item, +edit
662+:: and +arm, which have no tag and start with at=span; and a path's
663+:: knots (+pseg, +qseg) and a string's text (+piece), which have no
664+:: span: their node's span covers them.
485665:: Types avoid the standard library's names (mold, wing, gate, sym…),
486666:: hence +shape for a mold as written. Every type in a recursive cycle
487667:: has a $~ default that is a leaf: Hoon can't build the default of
@@ -549,14 +729,18 @@
549729::
550730+$ arm $~([*span *pattern *expr] [at=span pat=pattern val=expr])
551731::
552−:: +sample: a gate's sample; a bare name binds (%bind)
732+:: +sample: a gate's sample, or a field of a cell sample
733+::
734+:: A bare name binds (%bind) wherever it stands in a sample, so a cell
735+:: sample is a %cell of samples, never a %bare cell (section 3).
553736::
554737+$ sample
555738 $~ [%bind *span %$]
556739 $% [%bind at=span name=term] :: p
557740 [%face at=span name=term =shape] :: m=model
558741 [%pun at=span name=term] :: =msg, for msg=msg
559− [%bare at=span =shape] :: ~ [a=@ b=@]
742+ [%cell at=span items=(list sample)] :: [a=@ b] [=msg m=model]
743+ [%bare at=span =shape] :: ~ @t (list @t) %add
560744 ==
561745::
562746:: +field: a field of a cell mold; nests in +sample
@@ -604,46 +788,713 @@
604788::
605789:: +layout: source to tokens (design: section 2)
606790::
791+:: Never fails: an error it finds is its last token, [%error msg], at
792+:: the error's position (2.0). +lexer does the work.
793+::
607794++ layout
608795 |= [file=path src=@t]
609− ^- (each (list token) error)
610− [%| file 1 1 "layout pass not implemented"]
796+ ^- (list token)
797+ ~(run lexer [(trip src) 1 1 ~ ~[1] [0 0] 0 &])
798+::
799+:: +layout-error: the error of a token list's error token, if any
800+::
801+++ layout-error
802+ |= [file=path toks=(list token)]
803+ ^- (unit error)
804+ ?~ toks ~
805+ =/ last=token (rear toks)
806+ ?. ?=([%error *] lex.last) ~
807+ `[file line.beg.at.last col.beg.at.last p.lex.last]
611808::
612809:: +parse: source to tree
613810::
614811++ parse
615812 |= [file=path src=@t]
616813 ^- (each module error)
617− =/ lay (layout file src)
618− ?: ?=(%| -.lay) lay
619− (parse-tokens file p.lay)
814+ (parse-tokens file (layout file src))
620815::
621816:: +parse-tokens: tokens to tree (design: section 3)
622817::
818+:: It accepts no error token, so it fails at the first of its own
819+:: error and the layout pass's (2.0). The stub reports an error token
820+:: that comes first, as the parser will, and fails otherwise.
821+::
623822++ parse-tokens
624823 |= [file=path toks=(list token)]
625824 ^- (each module error)
825+ ?: ?=([[* * [%error *]] *] toks)
826+ (fail file toks "a module")
626827 =/ at=loc ?~(toks [1 1] (where i.toks))
627828 [%| file line.at col.at "parser not implemented"]
628829::
629830:: +golden: both stages printed, or the rendered error
630831::
631−:: tokens: the layout pass's output, or its error
632−:: tree: the parser's output, or the layout pass's or parser's error
832+:: tokens: the layout pass's tokens, or its error if it found one
833+:: tree: the parser's tree, or the first error of either stage, which
834+:: the parser reports since it accepts no error token (2.0)
633835::
634836++ golden
635837 |= [file=path src=@t]
636838 ^- [tokens=@t tree=@t]
637− =/ lay (layout file src)
638− ?: ?=(%| -.lay)
639− =/ msg=@t (show-error src p.lay)
640− [msg msg]
641− :- (show-tokens p.lay)
642− =/ par (parse-tokens file p.lay)
643− ?: ?=(%| -.par)
644− (show-error src p.par)
839+ =/ toks=(list token) (layout file src)
840+ =/ bad=(unit error) (layout-error file toks)
841+ :- ?^ bad (show-error src u.bad)
842+ (show-tokens toks)
843+ =/ par (parse-tokens file toks)
844+ ?: ?=(%| -.par) (show-error src p.par)
845+ :: a parser that accepted an error token would be wrong; say so
846+ ?^ bad (show-error src u.bad)
645847 (show-module p.par)
646848::
849++| %lexer
850+::
851+:: +lexer: the layout pass (section 2), a door over its state
852+::
853+:: It reads the text as one tape, a line at a time, keeps its tokens
854+:: last first and flops them at the end, so its time is linear in the
855+:: text. Each +step reads a line's indentation, one token or the end of
856+:: a line, and returns the door; +run steps until the text ends or an
857+:: error token stops it (2.0). Every loop is a trap whose recursive
858+:: call is a tail call, so a long file grows no stack.
859+::
860+++ lexer
861+ |_ $: cs=tape :: the text not yet read
862+ lin=@ud :: the line it starts on
863+ col=@ud :: the column it starts at
864+ out=(list token) :: the tokens so far, last first
865+ lev=(list @ud) :: open levels, innermost first
866+ pend=loc :: just past the last token
867+ dep=@ud :: open interpolations (2.9)
868+ bol=? :: at the start of a line
869+ ==
870+ ++ cor .
871+ ::
872+ :: +run: the tokens, to an error token or the end of the file
873+ ::
874+ ++ run
875+ ^- (list token)
876+ =/ lx cor
877+ |- ^- (list token)
878+ ?: stopped:lx (flop out.lx)
879+ ?: &(bol.lx ?=(~ cs.lx)) (flop out:eof:lx)
880+ $(lx step:lx)
881+ ::
882+ :: +stopped: whether an error token was emitted (2.0)
883+ ::
884+ ++ stopped ?=([[* * [%error *]] *] out)
885+ ::
886+ :: +eof: the end of the file (2.5): a DEDENT for each open level but
887+ :: the first, then NEWLINE, all with empty spans just past the last
888+ :: token; nothing in a file without tokens
889+ ::
890+ ++ eof
891+ ^+ cor
892+ ?~ out cor
893+ =/ at=span [pend pend]
894+ =/ n=@ud (dec (max 1 (lent lev)))
895+ cor(out [[at | %newline] (weld (reap n `token`[at | %dedent]) out)])
896+ ::
897+ :: +step: a line's indentation, or the line's next token or its end
898+ ::
899+ ++ step
900+ ^+ cor
901+ ?: bol indent:spaces
902+ read:spaces
903+ ::
904+ :: +spaces: past the spaces at the head of the text
905+ ::
906+ ++ spaces
907+ ^+ cor
908+ |- ^+ cor
909+ ?. &(?=(^ cs) =(' ' i.cs)) cor
910+ $(cs t.cs, col +(col))
911+ ::
912+ :: +next: past the end of the line, to the start of the next one
913+ ::
914+ ++ next
915+ ^+ cor
916+ ?~ cs cor(bol &)
917+ cor(cs t.cs, lin +(lin), col 1, bol &)
918+ ::
919+ :: +indent: a line, from its first character that isn't a space. A
920+ :: blank or comment-only line gives nothing (2.3); another line gets
921+ :: that character checked (2.2), then its layout tokens (2.4).
922+ ::
923+ ++ indent
924+ ^+ cor
925+ ?: |(?=(~ cs) =('\0a' i.cs)) next
926+ ?: &(=(':' i.cs) ?=(^ t.cs) =(':' i.t.cs)) next:comment
927+ =/ bad=(unit tape) (bad-char i.cs)
928+ ?^ bad (err col u.bad)
929+ ?: (gte i.cs 128) (err col non-ascii)
930+ ?~ out
931+ ?. =(1 col)
932+ %+ err col
933+ "the first line is indented: top-level lines start at column 1"
934+ cor(bol |)
935+ =/ top=@ud ?~(lev 1 i.lev)
936+ ?: (gth col top) (lay(lev [col lev], bol |) 1 %indent)
937+ ?: =(col top) (lay(bol |) 1 %newline)
938+ =/ n=@ud 0
939+ =/ rest=(list @ud) lev
940+ |- ^+ cor
941+ ?: &(?=(^ rest) (lth col i.rest)) $(n +(n), rest t.rest)
942+ ?. &(?=(^ rest) =(col i.rest)) (err col (no-level lev))
943+ (lay:(lay(lev rest, bol |) n %dedent) 1 %newline)
944+ ::
945+ :: +lay: n layout tokens g, spanning from just past the last token to
946+ :: this line's first character (2.4); they leave pend as it is
947+ ::
948+ ++ lay
949+ |= [n=@ud g=?(%indent %dedent %newline)]
950+ ^+ cor
951+ cor(out (weld (reap n `token`[[pend [lin col]] | g]) out))
952+ ::
953+ :: +err: the error token, at column c of this line (2.0)
954+ ::
955+ ++ err
956+ |= [c=@ud msg=tape]
957+ ^+ cor
958+ cor(out [[[[lin c] [lin c]] | [%error msg]] out])
959+ ::
960+ :: +put: a token from beg to end on this line, with rest the text
961+ :: after it, which starts at end
962+ ::
963+ ++ put
964+ |= [beg=loc end=loc lex=lexeme rest=tape]
965+ ^+ cor
966+ %= cor
967+ out [[[beg end] =(beg pend) lex] out]
968+ pend end
969+ cs rest
970+ col col.end
971+ ==
972+ ::
973+ :: +emit: a token of the next n characters, which are ASCII
974+ ::
975+ ++ emit
976+ |= [n=@ud lex=lexeme]
977+ ^+ cor
978+ (put [lin col] [lin (add col n)] lex (slag n cs))
979+ ::
980+ :: +comment: past a comment to the end of its line, checking its
981+ :: characters (2.2)
982+ ::
983+ ++ comment
984+ ^+ cor
985+ |- ^+ cor
986+ ?: |(?=(~ cs) =('\0a' i.cs)) cor
987+ ?: (in i.cs bad-mask) (err col (need (bad-char i.cs)))
988+ $(cs t.cs, col (next-col col i.cs))
989+ ::
990+ :: +read: the line's next token, or its end (2.6)
991+ ::
992+ :: A token is told by its first characters a, b and c, where b and c
993+ :: are 0 past the end of the text. Names, the most common, go first.
994+ ::
995+ ++ read
996+ ^+ cor
997+ ?: |(?=(~ cs) =('\0a' i.cs))
998+ ?. =(0 dep) (err col "this `\{` has no `}` on its line")
999+ next
1000+ =/ a=@ i.cs
1001+ ?: (is-lower a) name
1002+ =/ b=@ ?~(t.cs 0 i.t.cs)
1003+ =/ c=@ ?~(t.cs 0 ?~(t.t.cs 0 i.t.t.cs))
1004+ ?: =('(' a) (emit 1 %'(')
1005+ ?: =(')' a) (emit 1 %')')
1006+ ?: =('[' a) (emit 1 %'[')
1007+ ?: =(']' a) (emit 1 %']')
1008+ ?: =('"' a) (text |)
1009+ ?: =('}' a)
1010+ ?: =(0 dep) (err col "this `}` closes no `\{`")
1011+ (text(dep (dec dep)) &)
1012+ ?: (is-digit a) (literal (run-len cs number-mask))
1013+ ?: =('%' a) (percent b c)
1014+ ?: =('=' a)
1015+ ?: |(=('<' b) =('>' b))
1016+ %+ err col
1017+ "`={[b ~]}` is left out: names never shadow, and `.` reads imports too"
1018+ ?: =('=' b)
1019+ (err col "the closer `==` is left out: indentation covers it")
1020+ (emit 1 %'=')
1021+ ?: =('`' a) (emit 1 %'`')
1022+ ?: =('*' a) (emit 1 %'*')
1023+ ?: =('~' a)
1024+ ?: =('[' b) (emit 2 %'~[')
1025+ ?: =('&' b) (emit 2 %'~&')
1026+ ?: =('|' b) (err +(col) "`~|` is left out: `~&` covers debugging")
1027+ =/ n=@ud (run-len t.cs knot-mask)
1028+ ?: =(0 n) (emit 1 [%lit %null %n 0])
1029+ (literal +(n))
1030+ ?: =('/' a)
1031+ ?: &(=(1 col) |(=('-' b) =('%' b) =('=' b)))
1032+ (emit 2 ?:(=('-' b) %'/-' ?:(=('%' b) %'/%' %'/=')))
1033+ ?: =('+' b)
1034+ (err +(col) "`/+` is left out: code also runs in the browser")
1035+ segment:(emit 1 %'/')
1036+ ?: =('+' a)
1037+ ?: =('+' b) (emit 2 %'++')
1038+ ?: =('$' b) (emit 2 %'+$')
1039+ (emit 1 %'+')
1040+ ?: =('|' a)
1041+ ?: &(=('=' b) =('(' c)) (emit 3 %'|=(')
1042+ ?: =('=' b) (emit 2 %'|=')
1043+ ?: =('$' b) (emit 2 %'|$')
1044+ ?: |(=('%' b) =('_' b) =('*' b))
1045+ (err +(col) "`|{[b ~]}` is left out: modules are records")
1046+ ?: =('-' b) (err +(col) "`|-` is left out: gates recurse by name")
1047+ (err +(col) "`|` begins `|=`, `|=(` or `|$`")
1048+ ?: =('$' a)
1049+ ?: &(=('?' b) =('(' c)) (emit 3 %'$?(')
1050+ ?: =('?' b) (emit 2 %'$?')
1051+ ?: &(=('-' b) =('(' c)) (emit 3 %'$-(')
1052+ ?: =('-' b)
1053+ (err (add 2 col) "`$-` has only the wide form `$-(sample product)`")
1054+ ?: =('(' b) (err col "`$(…)` is left out: gates recurse by name")
1055+ ?: =('%' b) (err +(col) "`$%` is left out: `$?` covers unions")
1056+ (err +(col) "`$` begins `$?`, `$?(` or `$-(`")
1057+ ?: =('^' a)
1058+ ?: =('-' b) (emit 2 %'^-')
1059+ (err +(col) "`^` begins `^-`")
1060+ ?: =('?' a)
1061+ ?: =('-' b) (emit 2 %'?-')
1062+ ?: =('+' b) (err +(col) "`?+` is left out: `?-` is the branch")
1063+ ?: &(=(':' b) !=(':' c))
1064+ (err +(col) "`?:` is left out: `?-` is the branch")
1065+ ?: =('!' b) (err +(col) "`?!` is left out: eq and not are gates")
1066+ (emit 1 %'?')
1067+ ?: =('!' a)
1068+ ?: =('!' b) (emit 2 %'!!')
1069+ ?: =('>' b)
1070+ (err +(col) "`!>` is left out: code also runs in the browser")
1071+ (err +(col) "`!` begins only `!!`: `!x` is left out, as not is a gate")
1072+ ?: =('@' a)
1073+ ?: |(=('s' b) =('r' b))
1074+ (err +(col) "signed and floating-point auras are left out of Jibb")
1075+ =/ n=@ud (run-len t.cs lower-mask)
1076+ =/ rest=tape (slag n t.cs)
1077+ =? n &(?=(^ rest) (is-upper i.rest)) +(n)
1078+ (emit +(n) [%aura (@ta (crip (scag n t.cs)))])
1079+ ?: (is-upper a)
1080+ => (emit 1 [%tvar a])
1081+ ?. &(?=(^ cs) |((is-lower i.cs) (is-upper i.cs) (is-digit i.cs)))
1082+ cor
1083+ (err col "a type variable is one capital letter")
1084+ ?: =(':' a)
1085+ ?. =(0 dep) (err col "a comment can't start inside `\{…}`")
1086+ ?: =(':' b) comment
1087+ ?: =('~' b)
1088+ (err +(col) "`:~` is left out: indentation covers it; write ~[…]")
1089+ ?: =('*' b)
1090+ (err +(col) "`:*` is left out: indentation covers it; write […]")
1091+ ?: &(=(pend [lin col]) ?=([[* * ?([%name *] [%wing *])] *] out))
1092+ %+ err +(col)
1093+ %+ weld "`a:b` is left out: names never shadow, "
1094+ "and `.` reads imports too; write a.b"
1095+ (err +(col) "a comment starts with `::`")
1096+ ?: =('.' a)
1097+ ?: =('=' b) (err col "`.=` is left out: eq and not are gates")
1098+ ?: =('*' b)
1099+ %+ err col
1100+ "`.*` is left out: names never shadow, and `.` reads imports too"
1101+ (err col (unexpected a))
1102+ ?: =('-' a)
1103+ ?: =('-' b)
1104+ (err col "the closer `--` is left out: indentation covers it")
1105+ (err col (unexpected a))
1106+ ?: =(';' a)
1107+ (err col "Sail's `;` runes are left out: layout is Jibb's elements")
1108+ ?: =('\'' a) (err col "there is no '…' form: strings are \"…\"")
1109+ ?: =('{' a)
1110+ (err col "unexpected `\{`: `\{…}` interpolates only inside a string")
1111+ ?: (gte a 128) (err col non-ascii)
1112+ =/ bad=(unit tape) (bad-char a)
1113+ ?^ bad (err col u.bad)
1114+ (err col (unexpected a))
1115+ ::
1116+ :: +unexpected: the error for a character that starts no token (2.6),
1117+ :: with a hint where one helps
1118+ ::
1119+ ++ unexpected
1120+ |= a=@
1121+ ^- tape
1122+ =/ what=tape "unexpected `{[a ~]}`"
1123+ ?: =('.' a) "{what}: a `.` joins the names of a wing, as title.p"
1124+ ?: =(',' a) "{what}: items are separated by spaces"
1125+ ?: =('-' a)
1126+ "{what}: a name starts with a letter, and Jibb has no operators"
1127+ ?: =('<' a)
1128+ "{what}: there is no `<…>`, since `\{…}` prints a value by its aura"
1129+ what
1130+ ::
1131+ :: +percent: a token that starts with %, whose next characters are b
1132+ :: and c: %/, a loobean or a term (2.6)
1133+ ::
1134+ ++ percent
1135+ |= [b=@ c=@]
1136+ ^+ cor
1137+ ?: =('/' b) segment:(emit 2 %'%/')
1138+ ?: =('.' b)
1139+ ?: =('y' c) (emit 3 [%lit %loobean %f 0])
1140+ ?: =('n' c) (emit 3 [%lit %loobean %f 1])
1141+ (err (add 2 col) "the loobeans are `%.y` and `%.n`")
1142+ ?: =('&' b) (emit 2 [%lit %term %f 0])
1143+ ?: =('|' b) (emit 2 [%lit %term %f 1])
1144+ ?: =(':' b) (err +(col) "`%:` is left out: indentation covers it")
1145+ ?: =('=' b)
1146+ %+ err +(col)
1147+ "`%=` is left out: indentation covers it; write a change, as m(a=b)"
1148+ =/ rest=tape ?~(cs ~ t.cs)
1149+ ?: (is-lower b)
1150+ =/ n=@ud (run-len rest name-mask)
1151+ => (emit +(n) [%lit %term %tas (crip (scag n rest))])
1152+ ?. &(?=(^ cs) |((is-upper i.cs) =('_' i.cs))) cor
1153+ (err col "names are lowercase letters, digits and -")
1154+ ?: (is-digit b)
1155+ =/ n=@ud (run-len rest digit-mask)
1156+ => (emit +(n) [%lit %term %ud (decimal (scag n rest))])
1157+ ?. &(?=(^ cs) (in i.cs term-tail-mask)) cor
1158+ %+ err col
1159+ "unexpected `{[i.cs ~]}`: a numeric term is `%` and digits only, as %1000"
1160+ (err +(col) "`%` begins a term, a loobean or `%/`")
1161+ ::
1162+ :: +name: a name, or a wing, names joined by dots (2.6)
1163+ ::
1164+ ++ name
1165+ ^+ cor
1166+ =| ns=(list term)
1167+ =| len=@ud
1168+ =/ rest=tape cs
1169+ |- ^+ cor
1170+ =/ n=@ud (run-len rest name-mask)
1171+ =/ nam=term (@tas (crip (scag n rest)))
1172+ =. rest (slag n rest)
1173+ ?: &(?=([@ @ *] rest) =('.' i.rest) (is-lower i.t.rest))
1174+ $(ns [nam ns], len (add +(len) n), rest t.rest)
1175+ =/ lex=lexeme ?~(ns [%name nam] [%wing (flop [nam ns])])
1176+ => (emit (add len n) lex)
1177+ ?~ cs cor
1178+ ?: |((is-upper i.cs) =('_' i.cs))
1179+ (err col "names are lowercase letters, digits and -")
1180+ ?. =('.' i.cs) cor
1181+ (err +(col) "a `.` in a wing is followed by a name")
1182+ ::
1183+ :: +segment: after / or %/, a run of knot characters as one knot
1184+ :: token, tight (2.7)
1185+ ::
1186+ ++ segment
1187+ ^+ cor
1188+ =/ n=@ud (run-len cs knot-mask)
1189+ => ?:(=(0 n) cor (emit n [%knot (@ta (crip (scag n cs)))]))
1190+ :: what follows a segment: / ( [, a closer, a comment or a space
1191+ ?. &(?=(^ cs) (in i.cs seg-bad-mask)) cor
1192+ %+ err col
1193+ ;: weld
1194+ "unexpected `{[i.cs ~]}`: "
1195+ ?: =('?' i.cs) "a path carries no query, and a "
1196+ ?: =('#' i.cs) "a path carries no fragment, and a "
1197+ "a path "
1198+ "segment holds lowercase letters, digits and - . ~ _"
1199+ ==
1200+ ::
1201+ :: +literal: a number or a ~ literal, the next n characters, read by
1202+ :: Hoon's own parser, nuck:so (2.8)
1203+ ::
1204+ ++ literal
1205+ |= n=@ud
1206+ ^+ cor
1207+ =/ run=tape (scag n cs)
1208+ =/ a=@ ?~(run 0 i.run)
1209+ =/ b=@ ?~(run 0 ?~(t.run 0 i.t.run))
1210+ :: 0b 0c 0i 0w begin auras Jibb doesn't write, whatever follows
1211+ ?: &(=('0' a) |(=('b' b) =('c' b) =('i' b) =('w' b)))
1212+ (err +(col) "Jibb writes no @u{[b ~]} literals")
1213+ :: a year BC that Hoon's dates can't hold, which its date code
1214+ :: crashes on: an error at the - that makes it BC
1215+ =/ bc=(unit [at=@ud y=@ud]) (year-bc col run)
1216+ ?: &(?=(^ bc) =(0 y.u.bc))
1217+ (err at.u.bc "there is no year 0 BC: 1 BC is ~1-, as ~1-.1.1")
1218+ ?: &(?=(^ bc) (gth y.u.bc 292.277.024.401))
1219+ (err at.u.bc "Hoon's dates begin in 292.277.024.401 BC")
1220+ =/ ed (nuck:so [[lin col] run])
1221+ ?~ q.ed (err q.p.ed (literal-msg run %$))
1222+ =/ used=@ud (sub n (lent q.q.u.q.ed))
1223+ =/ con=coin p.u.q.ed
1224+ ?: =(0 used) (err col (literal-msg run %$))
1225+ ?. ?=([%$ *] con) (err +(col) "Jibb writes no such literal")
1226+ =/ form=(unit lit-form) (lit-form-of p.p.con)
1227+ ?~ form
1228+ %+ err +(col)
1229+ %+ weld "Jibb writes no @{(trip p.p.con)} literals"
1230+ ?:(=(%t p.p.con) "; strings are \"…\"" "")
1231+ =/ lex=lexeme [%lit u.form p.p.con q.p.con]
1232+ ?: =(used n)
1233+ ?. =(%ta p.p.con) (emit n lex)
1234+ :: a ~. knot is followed as a path segment is (2.7)
1235+ => (emit n lex)
1236+ ?. &(?=(^ cs) (in i.cs seg-bad-mask)) cor
1237+ %+ err col
1238+ %+ weld "unexpected `{[i.cs ~]}`: "
1239+ "a knot holds lowercase letters, digits and - . ~ _"
1240+ :: the error goes where Hoon's parser stopped looking; after a ship,
1241+ :: or a bare ~ that a letter follows, at the first letter that no
1242+ :: syllable continues (+ship-stop), or past a duration's unit, d h m
1243+ :: or s, if that is later (2.8)
1244+ =/ at=@ud q.p.ed
1245+ =? at &(=('~' a) (is-lower b) ?=(?(%n %p) p.p.con))
1246+ =/ dur=@ud ?:(|(=('d' b) =('h' b) =('m' b) =('s' b)) 2 1)
1247+ (min at (add col (max dur (ship-stop run))))
1248+ (err:(emit used lex) at (literal-msg run p.p.con))
1249+ ::
1250+ :: +text: a string's text, from its opening " or the } that closes
1251+ :: an interpolation, through the " or { that ends it (2.9). mid: it
1252+ :: resumes after a }, so it is str-mid or str-tail, not str-head or str.
1253+ ::
1254+ ++ text
1255+ |= mid=?
1256+ ^+ cor
1257+ =/ beg=loc [lin col]
1258+ =/ rest=tape ?~(cs ~ t.cs)
1259+ =/ at=@ud +(col)
1260+ =| acc=tape
1261+ |- ^+ cor
1262+ ?: |(?=(~ rest) =('\0a' i.rest))
1263+ (err at "this string has no closing `\"` on its line")
1264+ =/ ch=@ i.rest
1265+ ?: =('"' ch)
1266+ =/ txt=@t (crip (flop acc))
1267+ (put beg [lin +(at)] ?:(mid [%str-tail txt] [%str txt]) t.rest)
1268+ ?: =('{' ch)
1269+ =/ txt=@t (crip (flop acc))
1270+ => (put beg [lin +(at)] ?:(mid [%str-mid txt] [%str-head txt]) t.rest)
1271+ cor(dep +(dep))
1272+ ?: =('\\' ch)
1273+ ?~ t.rest (err +(at) bad-escape)
1274+ =/ d=@ i.t.rest
1275+ ?: |(=('\\' d) =('"' d) =('{' d))
1276+ $(rest t.t.rest, at (add 2 at), acc [d acc])
1277+ =/ bd=(unit tape) (bad-char d)
1278+ ?^ bd (err +(at) u.bd)
1279+ ?. (is-hex d) (err +(at) bad-escape)
1280+ ?~ t.t.rest (err (add 2 at) bad-hex)
1281+ =/ e=@ i.t.t.rest
1282+ =/ be=(unit tape) (bad-char e)
1283+ ?^ be (err (add 2 at) u.be)
1284+ ?. (is-hex e) (err (add 2 at) bad-hex)
1285+ %= $
1286+ rest t.t.t.rest
1287+ at (add 3 at)
1288+ acc [(add (mul 16 (hex-val d)) (hex-val e)) acc]
1289+ ==
1290+ ?: =(1 (cut 0 [ch 1] bad-mask)) (err at (need (bad-char ch)))
1291+ %= $
1292+ rest t.rest
1293+ at ?:(=(1 (cut 0 [ch 1] cont-mask)) at +(at))
1294+ acc [ch acc]
1295+ ==
1296+ --
1297+::
1298+:: +bad-char: the error for a character that no text may hold (2.2)
1299+::
1300+++ bad-char
1301+ |= c=@
1302+ ^- (unit tape)
1303+ ?: =(9 c) `"a tab: Jibb uses spaces only"
1304+ ?: =(13 c) `"a carriage return: Jibb lines end with a line feed alone"
1305+ ?: |((lth c 32) =(127 c)) `"a control character"
1306+ ~
1307+::
1308+++ non-ascii "non-ASCII text outside a string or comment"
1309+++ bad-escape "a `\\` begins `\\\\`, `\\\"`, `\\\{` or two hex digits"
1310+++ bad-hex "a `\\` escape takes two lowercase hex digits, as `\\0a`"
1311+::
1312+:: Character classes are masks, bit c set for each character c in the
1313+:: class, so a test is one (cut 0 [c 1] mask) rather than a chain of
1314+:: comparisons.
1315+::
1316+++ lower-mask 0x7ff.fffe.0000.0000.0000.0000.0000.0000 :: a-z
1317+++ upper-mask 0x7ff.fffe.0000.0000.0000.0000 :: A-Z
1318+++ digit-mask 0x3ff.0000.0000.0000 :: 0-9
1319+++ hex-mask 0x7e.0000.0000.03ff.0000.0000.0000 :: 0-9 a-f
1320+++ name-mask 0x7ff.fffe.0000.0000.03ff.2000.0000.0000 :: a-z 0-9 -
1321+++ knot-mask 0x47ff.fffe.8000.0000.03ff.6000.0000.0000 :: and . ~ _
1322+++ number-mask 0x7ff.fffe.07ff.fffe.03ff.4000.0000.0000 :: a-z A-Z 0-9 .
1323+++ bad-mask 0x8000.0000.0000.0000.0000.0000.ffff.fbff :: 2.2
1324+:: after a path segment (2.7): printable, but not a knot character,
1325+:: / ( [ ) ] } or :
1326+++ seg-bad-mask 0x1800.0001.57ff.ffff.f800.1cfe.0000.0000
1327+:: after a numeric term: a-z A-Z _ - .
1328+++ term-tail-mask 0x7ff.fffe.87ff.fffe.0000.6000.0000.0000
1329+::
1330+:: +cont-mask: UTF-8 continuation bytes, 10xx.xxxx, which start no
1331+:: column (1)
1332+::
1333+++ cont-mask
1334+ 0xffff.ffff.ffff.ffff.0000.0000.0000.0000.0000.0000.0000.0000
1335+::
1336+++ in |=([c=@ m=@] =(1 (cut 0 [c 1] m)))
1337+++ is-lower |=(c=@ (in c lower-mask))
1338+++ is-upper |=(c=@ (in c upper-mask))
1339+++ is-digit |=(c=@ (in c digit-mask))
1340+++ is-hex |=(c=@ (in c hex-mask))
1341+++ hex-val |=(c=@ ^-(@ ?:((is-digit c) (sub c '0') (sub c 87))))
1342+::
1343+:: +next-col: the column after character c at column col
1344+::
1345+++ next-col
1346+ |= [col=@ud c=@]
1347+ ^- @ud
1348+ ?:(=(1 (cut 0 [c 1] cont-mask)) col +(col))
1349+::
1350+:: +run-len: how many characters at the head of cs are in class m
1351+::
1352+++ run-len
1353+ |= [cs=tape m=@]
1354+ ^- @ud
1355+ =| n=@ud
1356+ |- ^- @ud
1357+ ?~ cs n
1358+ ?. =(1 (cut 0 [`@`i.cs 1] m)) n
1359+ $(cs t.cs, n +(n))
1360+::
1361+:: +decimal: digits as a number
1362+::
1363+++ decimal
1364+ |= ds=tape
1365+ ^- @ud
1366+ (roll ds |=([d=@ a=@ud] (add (mul 10 a) (sub d '0'))))
1367+::
1368+:: +lit-form-of: the literal rule of a dime's aura, if Jibb takes it (2.8)
1369+::
1370+++ lit-form-of
1371+ |= aura=@ta
1372+ ^- (unit lit-form)
1373+ ?+ aura ~
1374+ %ud `%number
1375+ %ux `%number
1376+ %uv `%number
1377+ %p `%ship
1378+ %da `%date
1379+ %dr `%duration
1380+ %ta `%knot
1381+ %n `%null
1382+ ==
1383+::
1384+:: +literal-msg: why a literal stops where nuck:so stops (2.8), told by
1385+:: its first characters and the aura of what nuck:so read
1386+::
1387+++ literal-msg
1388+ |= [run=tape aura=@ta]
1389+ ^- tape
1390+ =/ a=@ ?~(run 0 i.run)
1391+ =/ b=@ ?~(run 0 ?~(t.run 0 i.t.run))
1392+ ?. =('~' a)
1393+ ?: &(=('0' a) =('x' b))
1394+ %+ weld "a hexadecimal number is 0x and digits 0-9 and a-f, "
1395+ "in groups of four joined by dots, as 0x1.ffff"
1396+ ?: &(=('0' a) =('v' b))
1397+ %+ weld "a base-32 number is 0v and digits 0-9 and a-v, "
1398+ "in groups of five joined by dots, as 0v5.fdf99"
1399+ "a decimal is digits in groups of three joined by dots, as 1.000"
1400+ ?: |(=(%da aura) &(=(%n aura) (is-digit b)))
1401+ %+ weld "a date is ~year.month.day, then optionally "
1402+ "..hours.minutes.seconds, as ~2026.10.23..14.30.00"
1403+ ?: =(%dr aura)
1404+ "a duration is units of d, h, m and s joined by dots, as ~h1.m30"
1405+ ?: =(%p aura)
1406+ %+ weld "a ship is syllables in pairs joined by hyphens, "
1407+ "as ~zod, ~marzod or ~sampel-palnet"
1408+ %+ weld "`~` alone is null, and nothing here reads as a ship, "
1409+ "date, duration or knot"
1410+::
1411+:: +year-bc: a ~ literal run's year and the column of its -, if it
1412+:: starts as a date BC does, ~ then digits Hoon reads as a year, then -
1413+::
1414+:: Hoon's +year crashes on a year BC of 0 or past 292.277.024.401, as
1415+:: in ~0-.1.1, so these never reach nuck:so (2.8).
1416+::
1417+++ year-bc
1418+ |= [col=@ud run=tape]
1419+ ^- (unit [at=@ud y=@ud])
1420+ ?. ?=([%'~' @ *] run) ~
1421+ =/ tr=tape t.run
1422+ =/ ds=tape (scag (run-len tr digit-mask) tr)
1423+ ?~ ds ~
1424+ :: a year is 0 or has no leading 0, as dim:ag reads it
1425+ ?: &(=('0' i.ds) ?=(^ t.ds)) ~
1426+ =/ rest=tape (slag (lent ds) tr)
1427+ ?. ?=([%'-' *] rest) ~
1428+ `[(add col +((lent ds))) (decimal ds)]
1429+::
1430+:: +ship-stop: how many characters of a ~ literal's run, from its ~,
1431+:: could still begin a ship (2.8)
1432+::
1433+:: Hoon's parser reads a syllable's three letters before it looks
1434+:: them up, so it stops past the first letter that no syllable
1435+:: continues; this stops at that letter. It takes a galaxy's suffix,
1436+:: or words of a prefix and a suffix joined by - or --, as many as
1437+:: come. That is a few more texts than Hoon reads, so it never stops
1438+:: before Hoon's parser could have gone on.
1439+::
1440+++ ship-stop
1441+ |= run=tape
1442+ ^- @ud
1443+ =/ cs=tape ?~(run ~ t.run)
1444+ =/ pre=@ud (syllable cs sis:po)
1445+ :: a galaxy's suffix, or a word's prefix
1446+ ?. =(3 pre) +((max pre (syllable cs dex:po)))
1447+ =/ i=@ud 4
1448+ =. cs (slag 3 cs)
1449+ |- ^- @ud
1450+ :: the word's suffix
1451+ =/ suf=@ud (syllable cs dex:po)
1452+ ?. =(3 suf) (add i suf)
1453+ =. i (add i 3)
1454+ =. cs (slag 3 cs)
1455+ :: then - or --, and the next word's prefix
1456+ ?. &(?=(^ cs) =('-' i.cs)) i
1457+ =/ h=@ud ?:(&(?=([@ @ *] cs) =('-' i.t.cs)) 2 1)
1458+ =/ word=tape (slag h `tape`cs)
1459+ =/ nex=@ud (syllable word sis:po)
1460+ ?. =(3 nex) :(add i h nex)
1461+ $(i :(add i h 3), cs (slag 3 word))
1462+::
1463+:: +syllable: how many of cs's first three characters begin one of the
1464+:: 256 syllables in syls, the prefixes or suffixes of +po
1465+::
1466+++ syllable
1467+ |= [cs=tape syls=@t]
1468+ ^- @ud
1469+ =/ l=tape (scag 3 cs)
1470+ =/ k=@ud 0
1471+ |- ^- @ud
1472+ ?: =(k (lent l)) k
1473+ =/ head=@ (crip (scag +(k) l))
1474+ ?. (lien (gulf 0 255) |=(i=@ =(head (cut 3 [(mul 3 i) +(k)] syls))))
1475+ k
1476+ $(k +(k))
1477+::
1478+:: +no-level: the error for a line that returns to no open level (2.4),
1479+:: naming the open levels' columns, outermost first
1480+::
1481+++ no-level
1482+ |= lev=(list @ud)
1483+ ^- tape
1484+ %+ weld
1485+ "this line's indentation matches no enclosing line: indent it to column "
1486+ (join-or (turn (flop lev) |=(c=@ud (a-co:co c))))
1487+::
1488+:: +join-or: words as "a", "a or b", "a, b or c"
1489+::
1490+++ join-or
1491+ |= ws=(list tape)
1492+ ^- tape
1493+ ?~ ws ""
1494+ ?~ t.ws i.ws
1495+ ?~ t.t.ws "{i.ws} or {i.t.ws}"
1496+ "{i.ws}, {$(ws t.ws)}"
1497+::
6471498+| %printers
6481499::
6491500:: +show-tokens: one token per line (format: section 4.1)
@@ -657,7 +1508,7 @@
6571508 |= t=token
6581509 ^- tape
6591510 ;: weld
660− (show-loc beg.at.t)
1511+ (show-loc (where t))
6611512 " "
6621513 ?:(tight.t "tight " "")
6631514 (show-lexeme lex.t)
@@ -683,6 +1534,7 @@
6831534 %knot "knot {(trip p.x)}"
6841535 %aura "aura @{(trip p.x)}"
6851536 %tvar "tvar {(trip p.x)}"
1537+ %error "ERROR {p.x}"
6861538 ==
6871539::
6881540:: +show-module: the tree as S-expressions (format: section 4.2)
@@ -700,6 +1552,8 @@
7001552::
7011553:: +show-error: "<file>:<line>:<col>: <message>", the line, a caret
7021554::
1555+:: In the excerpt a control character prints as a space (4.3).
1556+::
7031557++ show-error
7041558 |= [src=@t err=error]
7051559 ^- @t
@@ -707,6 +1561,9 @@
7071561 =/ text=@t
7081562 ?: |(=(0 line.err) (gth line.err (lent lines))) ''
7091563 (snag (dec line.err) lines)
1564+ =/ excerpt=tape
1565+ %+ turn (trip text)
1566+ |=(c=@tD ?:(|((lth c 32) =(127 c)) ' ' c))
7101567 %- crip
7111568 ;: weld
7121569 (show-file file.err)
@@ -717,7 +1574,7 @@
7171574 ": "
7181575 msg.err
7191576 "\0a"
720− (trip text)
1577+ excerpt
7211578 "\0a"
7221579 (reap (dec (max 1 col.err)) ' ')
7231580 "^"
@@ -725,15 +1582,19 @@
7251582::
7261583:: +show-found: a token as error messages name it
7271584::
1585+:: A layout token with an empty span is at the end of the file (2.5).
1586+::
7281587++ show-found
7291588 |= t=token
7301589 ^- tape
7311590 =/ x=lexeme lex.t
7321591 ?@ x
1592+ ?: &(?=(?(%indent %dedent %newline) x) =(beg.at.t end.at.t))
1593+ "the end of the file"
7331594 ?+ x "`{(trip x)}`"
7341595 %indent "a deeper line"
735− %dedent "the end of the line"
736− %newline "the end of the line"
1596+ %dedent "a shallower line"
1597+ %newline "a line at the same depth"
7371598 ==
7381599 ?- -.x
7391600 %name "`{(trip p.x)}`"
@@ -746,17 +1607,19 @@
7461607 %knot "`{(trip p.x)}`"
7471608 %aura "`@{(trip p.x)}`"
7481609 %tvar "`{(trip p.x)}`"
1610+ %error "an error"
7491611 ==
7501612::
751−:: +where: where an error at a token points
1613+:: +where: where a token is, for errors and the token printer
7521614::
753−:: At the token's start; for NEWLINE and DEDENT that is the end of the
754−:: previous line's code. For INDENT, the start of the deeper line.
1615+:: At the token's start; for a layout token, at the end of its span:
1616+:: the start of the line it begins, or at the end of the file just
1617+:: past the last token (2.4, 2.5).
7551618::
7561619++ where
7571620 |= t=token
7581621 ^- loc
759− ?: ?=(%indent lex.t) end.at.t
1622+ ?: ?=(?(%indent %dedent %newline) lex.t) end.at.t
7601623 beg.at.t
7611624::
7621625++ show-loc
@@ -984,6 +1847,7 @@
9841847 %bind (nod "bind" at.s (trip name.s) ~)
9851848 %face (nod "face" at.s (trip name.s) ~[(sexp-shape shape.s)])
9861849 %pun (nod "pun" at.s (trip name.s) ~)
1850+ %cell (nod "cell" at.s "" (turn items.s sexp-sample))
9871851 %bare (sexp-shape shape.s)
9881852 ==
9891853::
@@ -1042,7 +1906,8 @@
10421906 %& (next p.res)
10431907 ==
10441908::
1045−:: +span-of: the span of any node: every node is [%tag at=span …]
1909+:: +span-of: the span of a tagged node, [%tag at=span …]; an +item,
1910+:: +edit or +arm has its span at its head, at.node
10461911::
10471912++ span-of
10481913 |* node=[@ span *]
@@ -1051,10 +1916,35 @@
10511916::
10521917:: +fail: the error "expected <want>, found <the next token>"
10531918::
1919+:: At an error token, the layout pass's error instead (2.0). The token
1920+:: list always ends with NEWLINE or an error token (2.5), and the
1921+:: parser stops at either, so it never runs out; if it did, the error
1922+:: would sit at 1:1.
1923+::
10541924++ fail
10551925 |= [file=path toks=(list token) want=tape]
10561926 ^- [%| p=error]
10571927 ?~ toks [%| file 1 1 "expected {want}, found the end of the tokens"]
1928+ ?: ?=([%error *] lex.i.toks)
1929+ [%| file line.beg.at.i.toks col.beg.at.i.toks p.lex.i.toks]
10581930 =/ l=loc (where i.toks)
10591931 [%| file line.l col.l "expected {want}, found {(show-found i.toks)}"]
1932+::
1933+:: +fail-space: a tight form found a space before its next token
1934+::
1935+:: The error sits where the space starts, just past prev, the token
1936+:: that needs its next one at once (section 3, Errors). If the next
1937+:: token is a layout or error token, it is +fail's error instead.
1938+::
1939+++ fail-space
1940+ |= [file=path prev=token toks=(list token) want=tape]
1941+ ^- [%| p=error]
1942+ ?: ?| ?=(~ toks)
1943+ ?=(?(%indent %dedent %newline) lex.i.toks)
1944+ ?=([%error *] lex.i.toks)
1945+ ==
1946+ (fail file toks want)
1947+ :+ %| file
1948+ :+ line.end.at.prev col.end.at.prev
1949+ "expected {want} right after {(show-found prev)}, found a space"
10601950--