@@ -1,76 +1,293 @@
11# Adding indent queries
22
3−Helix uses tree-sitter to correctly indent new lines. This requires
4−a tree-sitter grammar and an `indent.scm` query file placed in
5−`runtime/queries/{language}/indents.scm`. The indentation for a line
6−is calculated by traversing the syntax tree from the lowest node at the
7−beginning of the new line. Each of these nodes contributes to the total
8−indent when it is captured by the query (in what way depends on the name
9−of the capture).
3+Helix uses tree-sitter to correctly indent new lines. This requires a tree-
4+sitter grammar and an `indent.scm` query file placed in `runtime/queries/
5+{language}/indents.scm`. The indentation for a line is calculated by traversing
6+the syntax tree from the lowest node at the beginning of the new line (see
7+[Indent queries](#indent-queries)). Each of these nodes contributes to the total
8+indent when it is captured by the query (in what way depends on the name of
9+the capture.
1010
1111Note that it matters where these added indents begin. For example,
1212multiple indent level increases that start on the same line only increase
13−the total indent level by 1.
13+the total indent level by 1. See [Capture types](#capture-types).
1414
15−## Scopes
15+## Indent queries
1616
17−Added indents don't always apply to the whole node. For example, in most
18−cases when a node should be indented, we actually only want everything
19−except for its first line to be indented. For this, there are several
20−scopes (more scopes may be added in the future if required):
17+When Helix is inserting a new line through `o`, `O`, or `<ret>`, to determine
18+the indent level for the new line, the query in `indents.scm` is run on the
19+document. The starting position of the query is the end of the line above where
20+a new line will be inserted.
2121
22−- `all`:
23−This scope applies to the whole captured node. This is only different from
24−`tail` when the captured node is the first node on its line.
22+For `o`, the inserted line is the line below the cursor, so that starting
23+position of the query is the end of the current line.
2524
26−- `tail`:
27−This scope applies to everything except for the first line of the
28−captured node.
25+```rust
26+fn need_hero(some_hero: Hero, life: Life) -> {
27+ matches!(some_hero, Hero { // ←─────────────────╮
28+ strong: true,//←╮ ↑ ↑ │
29+ fast: true, // │ │ ╰── query start │
30+ sure: true, // │ ╰───── cursor ├─ traversal
31+ soon: true, // ╰──────── new line inserted │ start node
32+ }) && // │
33+// ↑ │
34+// ╰───────────────────────────────────────────────╯
35+ some_hero > life
36+}
37+```
2938
30−Every capture type has a default scope which should do the right thing
31−in most situations. When a different scope is required, this can be
32−changed by using a `#set!` declaration anywhere in the pattern:
33−```scm
34−(assignment_expression
35− right: (_) @indent
36− (#set! "scope" "all"))
39+For `O`, the newly inserted line is the *current* line, so the starting position
40+of the query is the end of the line above the cursor.
41+
42+```rust
43+fn need_hero(some_hero: Hero, life: Life) -> { // ←─╮
44+ matches!(some_hero, Hero { // ←╮ ↑ │
45+ strong: true,// ↑ ╭───╯ │ │
46+ fast: true, // │ │ query start ─╯ │
47+ sure: true, // ╰───┼ cursor ├─ traversal
48+ soon: true, // ╰ new line inserted │ start node
49+ }) && // │
50+ some_hero > life // │
51+} // ←──────────────────────────────────────────────╯
3752```
3853
39−## Capture types
54+From this starting node, the syntax tree is traversed up until the root node.
55+Each indent capture is collected along the way, and then combined according to
56+their [capture types](#capture-types) and [scopes](#scopes) to a final indent
57+level for the line.
4058
41−- `@indent` (default scope `tail`):
42−Increase the indent level by 1. Multiple occurrences in the same line
43−don't stack. If there is at least one `@indent` and one `@outdent`
44−capture on the same line, the indent level isn't changed at all.
59+### Capture types
4560
61+- `@indent` (default scope `tail`):
62+ Increase the indent level by 1. Multiple occurrences in the same line *do not*
63+ stack. If there is at least one `@indent` and one `@outdent` capture on the
64+ same line, the indent level isn't changed at all.
4665- `@outdent` (default scope `all`):
47−Decrease the indent level by 1. The same rules as for `@indent` apply.
48−
66+ Decrease the indent level by 1. The same rules as for `@indent` apply.
67+- `@indent.always` (default scope `tail`):
68+ Increase the indent level by 1. Multiple occurrences on the same line *do*
69+ stack. The final indent level is `@indent.always` – `@outdent.always`. If
70+ an `@indent` and an `@indent.always` are on the same line, the `@indent` is
71+ ignored.
72+- `@outdent.always` (default scope `all`):
73+ Decrease the indent level by 1. The same rules as for `@indent.always` apply.
4974- `@extend`:
50−Extend the range of this node to the end of the line and to lines that
51−are indented more than the line that this node starts on. This is useful
52−for languages like Python, where for the purpose of indentation some nodes
53−(like functions or classes) should also contain indented lines that follow them.
54−
75+ Extend the range of this node to the end of the line and to lines that are
76+ indented more than the line that this node starts on. This is useful for
77+ languages like Python, where for the purpose of indentation some nodes (like
78+ functions or classes) should also contain indented lines that follow them.
5579- `@extend.prevent-once`:
56−Prevents the first extension of an ancestor of this node. For example, in Python
57−a return expression always ends the block that it is in. Note that this only stops the
58−extension of the next `@extend` capture. If multiple ancestors are captured,
59−only the extension of the innermost one is prevented. All other ancestors are unaffected
60−(regardless of whether the innermost ancestor would actually have been extended).
80+ Prevents the first extension of an ancestor of this node. For example, in Python
81+ a return expression always ends the block that it is in. Note that this only
82+ stops the extension of the next `@extend` capture. If multiple ancestors are
83+ captured, only the extension of the innermost one is prevented. All other
84+ ancestors are unaffected (regardless of whether the innermost ancestor would
85+ actually have been extended).
86+
87+#### `@indent` / `@outdent`
88+
89+Consider this example:
90+
91+```rust
92+fn shout(things: Vec<Thing>) {
93+ // ↑
94+ // ├───────────────────────╮ indent level
95+ // @indent ├┄┄┄┄┄┄┄┄┄┄┄┄┄┄
96+ // │
97+ let it_all = |out| { things.filter(|thing| { // │ 1
98+ // ↑ ↑ │
99+ // ├───────────────────────┼─────┼┄┄┄┄┄┄┄┄┄┄┄┄┄┄
100+ // @indent @indent │
101+ // │ 2
102+ thing.can_do_with(out) // │
103+ })}; // ├┄┄┄┄┄┄┄┄┄┄┄┄┄┄
104+ //↑↑↑ │ 1
105+} //╰┼┴──────────────────────────────────────────────┴┄┄┄┄┄┄┄┄┄┄┄┄┄┄
106+// 3x @outdent
107+```
108+
109+```scm
110+((block) @indent)
111+["}" ")"] @outdent
112+```
113+
114+Note how on the second line, we have two blocks begin on the same line. In this
115+case, since both captures occur on the same line, they are combined and only
116+result in a net increase of 1. Also note that the closing `}`s are part of the
117+`@indent` captures, but the 3 `@outdent`s also combine into 1 and result in that
118+line losing one indent level.
119+
120+#### `@extend` / `@extend.prevent-once`
121+
122+For an example of where `@extend` can be useful, consider Python, which is
123+whitespace-sensitive.
124+
125+```scm
126+]
127+ (parenthesized_expression)
128+ (function_definition)
129+ (class_definition)
130+] @indent
131+
132+```
133+
134+```python
135+class Hero:
136+ def __init__(self, strong, fast, sure, soon):# ←─╮
137+ self.is_strong = strong # │
138+ self.is_fast = fast # ╭─── query start │
139+ self.is_sure = sure # │ ╭─ cursor │
140+ self.is_soon = soon # │ │ │
141+ # ↑ ↑ │ │ │
142+ # │ ╰──────╯ │ │
143+ # ╰─────────────────────╯ │
144+ # ├─ traversal
145+ def need_hero(self, life): # │ start node
146+ return ( # │
147+ self.is_strong # │
148+ and self.is_fast # │
149+ and self.is_sure # │
150+ and self.is_soon # │
151+ and self > life # │
152+ ) # ←─────────────────────────────────────────╯
153+```
154+
155+Without braces to catch the scope of the function, the smallest descendant of
156+the cursor on a line feed ends up being the entire inside of the class. Because
157+of this, it will miss the entire function node and its indent capture, leading
158+to an indent level one too small.
159+
160+To address this case, `@extend` tells helix to "extend" the captured node's span
161+to the line feed and every consecutive line that has a greater indent level than
162+the line of the node.
163+
164+```scm
165+(parenthesized_expression) @indent
166+
167+]
168+ (function_definition)
169+ (class_definition)
170+] @indent @extend
171+
172+```
173+
174+```python
175+class Hero:
176+ def __init__(self, strong, fast, sure, soon):# ←─╮
177+ self.is_strong = strong # │
178+ self.is_fast = fast # ╭─── query start ├─ traversal
179+ self.is_sure = sure # │ ╭─ cursor │ start node
180+ self.is_soon = soon # │ │ ←───────────────╯
181+ # ↑ ↑ │ │
182+ # │ ╰──────╯ │
183+ # ╰─────────────────────╯
184+ def need_hero(self, life):
185+ return (
186+ self.is_strong
187+ and self.is_fast
188+ and self.is_sure
189+ and self.is_soon
190+ and self > life
191+ )
192+```
193+
194+Furthermore, there are some cases where extending to everything with a greater
195+indent level may not be desirable. Consider the `need_hero` function above. If
196+our cursor is on the last line of the returned expression.
61197
198+```python
199+class Hero:
200+ def __init__(self, strong, fast, sure, soon):
201+ self.is_strong = strong
202+ self.is_fast = fast
203+ self.is_sure = sure
204+ self.is_soon = soon
205+
206+ def need_hero(self, life):
207+ return (
208+ self.is_strong
209+ and self.is_fast
210+ and self.is_sure
211+ and self.is_soon
212+ and self > life
213+ ) # ←─── cursor
214+ #←────────── where cursor should go on new line
215+```
216+
217+In Python, the are a few tokens that will always end a scope, such as a return
218+statement. Since the scope ends, so should the indent level. But because the
219+function span is extended to every line with a greater indent level, a new line
220+would just continue on the same level. And an `@outdent` would not help us here
221+either, since it would cause everything in the parentheses to become outdented
222+as well.
223+
224+To help, we need to signal an end to the extension. We can do this with
225+`@extend.prevent-once`.
226+
227+```scm
228+(parenthesized_expression) @indent
229+
230+]
231+ (function_definition)
232+ (class_definition)
233+] @indent @extend
234+
235+(return_statement) @extend.prevent-once
236+```
237+
238+#### `@indent.always` / `@outdent.always`
239+
240+As mentioned before, normally if there is more than one `@indent` or `@outdent`
241+capture on the same line, they are combined.
242+
243+Sometimes, there are cases when you may want to ensure that every indent capture
244+is additive, regardless of how many occur on the same line. Consider this
245+example in YAML.
246+
247+```yaml
248+ - foo: bar
249+# ↑ ↑
250+# │ ╰─────────────── start of map
251+# ╰───────────────── start of list element
252+ baz: quux # ←─── cursor
253+ # ←───────────── where the cursor should go on a new line
254+ garply: waldo
255+ - quux:
256+ bar: baz
257+ xyzzy: thud
258+ fred: plugh
259+```
260+
261+In YAML, you often have lists of maps. In these cases, the syntax is such that
262+the list element and the map both start on the same line. But we really do want
263+to start an indentation for each of these so that subsequent keys in the map
264+hang over the list and align properly. This is where `@indent.always` helps.
265+
266+```scm
267+((block_sequence_item) @item @indent.always @extend
268+ (#not-one-line? @item))
269+
270+((block_mapping_pair
271+ key: (_) @key
272+ value: (_) @val
273+ (#not-same-line? @key @val)
274+ ) @indent.always @extend
275+)
276+```
62277
63278## Predicates
64279
65280In some cases, an S-expression cannot express exactly what pattern should be matched.
66281For that, tree-sitter allows for predicates to appear anywhere within a pattern,
67282similar to how `#set!` declarations work:
283+
68284```scm
69285(some_kind
70286 (child_kind) @indent
71287 (#predicate? arg1 arg2 ...)
72288)
73289```
290+
74291The number of arguments depends on the predicate that's used.
75292Each argument is either a capture (`@name`) or a string (`"some string"`).
76293The following predicates are supported by tree-sitter:
@@ -91,3 +308,47 @@ argument (a string).
91308
92309- `#same-line?`/`#not-same-line?`:
93310The captures given by the 2 arguments must/must not start on the same line.
311+
312+- `#one-line?`/`#not-one-line?`:
313+The captures given by the fist argument must/must span a total of one line.
314+
315+### Scopes
316+
317+Added indents don't always apply to the whole node. For example, in most
318+cases when a node should be indented, we actually only want everything
319+except for its first line to be indented. For this, there are several
320+scopes (more scopes may be added in the future if required):
321+
322+- `tail`:
323+This scope applies to everything except for the first line of the
324+captured node.
325+- `all`:
326+This scope applies to the whole captured node. This is only different from
327+`tail` when the captured node is the first node on its line.
328+
329+For example, imagine we have the following function
330+
331+```rust
332+fn aha() { // ←─────────────────────────────────────╮
333+ let take = "on me"; // ←──────────────╮ scope: │
334+ let take = "me on"; // ├─ "tail" ├─ (block) @indent
335+ let ill = be_gone_days(1 || 2); // │ │
336+} // ←───────────────────────────────────┴──────────┴─ "}" @outdent
337+ // scope: "all"
338+```
339+
340+We can write the following query with the `#set!` declaration:
341+
342+ ```scm
343+ ((block) @indent
344+ (#set! "scope" "tail"))
345+ ("}" @outdent
346+ (#set! "scope" "all"))
347+ ```
348+
349+As we can see, the "tail" scope covers the node, except for the first line.
350+Everything up to and including the closing brace gets an indent level of 1.
351+Then, on the closing brace, we encounter an outdent with a scope of "all", which
352+means the first line is included, and the indent level is cancelled out on this
353+line. (Note these scopes are the defaults for `@indent` and `@outdent`—they are
354+written explicitly for demonstration.)