@@ -1,45 +1,68 @@
11# Adding languages
22
3−## Submodules
3+## Language configuration
44
5−To add a new language, you should first add a tree-sitter submodule. To do this,
6−you can run the command
7−```sh
8−git submodule add -f <repository> helix-syntax/languages/tree-sitter-<name>
9−```
10−For example, to add tree-sitter-ocaml you would run
11−```sh
12−git submodule add -f https://github.com/tree-sitter/tree-sitter-ocaml helix-syntax/languages/tree-sitter-ocaml
5+To add a new language, you need to add a `language` entry to the
6+[`languages.toml`][languages.toml] found in the root of the repository;
7+this `languages.toml` file is included at compilation time, and is
8+distinct from the `languages.toml` file in the user's [configuration
9+directory](../configuration.md).
10+
11+```toml
12+[[language]]
13+name = "mylang"
14+scope = "scope.mylang"
15+injection-regex = "^mylang$"
16+file-types = ["mylang", "myl"]
17+comment-token = "#"
18+indent = { tab-width = 2, unit = " " }
1319```
14−Make sure the submodule is shallow by doing
15−```sh
16−git config -f .gitmodules submodule.helix-syntax/languages/tree-sitter-<name>.shallow true
20+
21+These are the available keys and descriptions for the file.
22+
23+| Key | Description |
24+| ---- | ----------- |
25+| `name` | The name of the language |
26+| `scope` | A string like `source.js` that identifies the language. Currently, we strive to match the scope names used by popular TextMate grammars and by the Linguist library. Usually `source.<name>` or `text.<name>` in case of markup languages |
27+| `injection-regex` | regex pattern that will be tested against a language name in order to determine whether this language should be used for a potential [language injection][treesitter-language-injection] site. |
28+| `file-types` | The filetypes of the language, for example `["yml", "yaml"]`. Extensions and full file names are supported. |
29+| `shebangs` | The interpreters from the shebang line, for example `["sh", "bash"]` |
30+| `roots` | A set of marker files to look for when trying to find the workspace root. For example `Cargo.lock`, `yarn.lock` |
31+| `auto-format` | Whether to autoformat this language when saving |
32+| `diagnostic-severity` | Minimal severity of diagnostic for it to be displayed. (Allowed values: `Error`, `Warning`, `Info`, `Hint`) |
33+| `comment-token` | The token to use as a comment-token |
34+| `indent` | The indent to use. Has sub keys `tab-width` and `unit` |
35+| `config` | Language server configuration |
36+| `grammar` | The tree-sitter grammar to use (defaults to the value of `name`) |
37+
38+## Grammar configuration
39+
40+If a tree-sitter grammar is available for the language, add a new `grammar`
41+entry to `languages.toml`.
42+
43+```toml
44+[[grammar]]
45+name = "mylang"
46+source = { git = "https://github.com/example/mylang", rev = "a250c4582510ff34767ec3b7dcdd3c24e8c8aa68" }
1747```
1848
19−or you can manually add `shallow = true` to `.gitmodules`.
49+Grammar configuration takes these keys:
2050
21−## languages.toml
51+| Key | Description |
52+| --- | ----------- |
53+| `name` | The name of the tree-sitter grammar |
54+| `path` | A path within the grammar directory which should be built. Some grammar repositories host multiple grammars (for example `tree-sitter-typescript` and `tree-sitter-ocaml`) in subdirectories. This key is used to point `hx --build-grammars` to the correct path for compilation. When ommitted, the root of the grammar directory is used |
55+| `source` | The method of fetching the grammar - a table with a schema defined below |
2256
23−Next, you need to add the language to the [`languages.toml`][languages.toml] found in the root of
24−the repository; this `languages.toml` file is included at compilation time, and
25−is distinct from the `language.toml` file in the user's [configuration
26−directory](../configuration.md).
57+Where `source` is a table with either these keys when using a grammar from a
58+git repository:
2759
28−These are the available keys and descriptions for the file.
60+| Key | Description |
61+| --- | ----------- |
62+| `git` | A git remote URL from which the grammar should be cloned |
63+| `rev` | The revision (commit hash or tag) which should be fetched |
2964
30−| Key | Description |
31−| ---- | ----------- |
32−| name | The name of the language |
33−| scope | A string like `source.js` that identifies the language. Currently, we strive to match the scope names used by popular TextMate grammars and by the Linguist library. Usually `source.<name>` or `text.<name>` in case of markup languages |
34−| injection-regex | regex pattern that will be tested against a language name in order to determine whether this language should be used for a potential [language injection][treesitter-language-injection] site. |
35−| file-types | The filetypes of the language, for example `["yml", "yaml"]` |
36−| shebangs | The interpreters from the shebang line, for example `["sh", "bash"]` |
37−| roots | A set of marker files to look for when trying to find the workspace root. For example `Cargo.lock`, `yarn.lock` |
38−| auto-format | Whether to autoformat this language when saving |
39−| diagnostic-severity | Minimal severity of diagnostic for it to be displayed. (Allowed values: `Error`, `Warning`, `Info`, `Hint`) |
40−| comment-token | The token to use as a comment-token |
41−| indent | The indent to use. Has sub keys `tab-width` and `unit` |
42−| config | Language server configuration |
65+Or a `path` key with an absolute path to a locally available grammar directory.
4366
4467## Queries
4568
@@ -51,18 +74,14 @@ gives more info on how to write queries.
5174
5275> NOTE: When evaluating queries, the first matching query takes
5376precedence, which is different from other editors like neovim where
54−the last matching query supercedes the ones before it. See
77+the last matching query supersedes the ones before it. See
5578[this issue][neovim-query-precedence] for an example.
5679
5780## Common Issues
5881
59−- If you get errors when building after switching branches, you may have to remove or update tree-sitter submodules. You can update submodules by running
60− ```sh
61− git submodule sync; git submodule update --init
62− ```
63−- Make sure to not use the `--remote` flag. To remove submodules look inside the `.gitmodules` and remove directories that are not present inside of it.
82+- If you get errors when running after switching branches, you may have to update the tree-sitter grammars. Run `hx --fetch-grammars` to fetch the grammars and `hx --build-grammars` to build any out-of-date grammars.
6483
65−- If a parser is segfaulting or you want to remove the parser, make sure to remove the submodule *and* the compiled parser in `runtime/grammar/<name>.so`
84+- If a parser is segfaulting or you want to remove the parser, make sure to remove the compiled parser in `runtime/grammar/<name>.so`
6685
6786- The indents query is `indents.toml`, *not* `indents.scm`. See [this](https://github.com/helix-editor/helix/issues/114) issue for more information.
6887