to `markup.horizontal-rule.gfm` and add styles to all eight built-in syntax themes to account for it.
All the old GFM scopes are just named however they needed to be named in order to produce syntax highlighting to the grammar author, semantics be damned. Instead, we're making sure the semantics are correct, then adapting the syntax themes accordingly.
…in which all _value types_ are scoped under `support.storage`. The goal is to avoid drawing an arbitrary distinction between, say, `int` and `SomeCustomCType` — those things are different, but not _so_ different that they belong in separate root scope namespaces.
`storage.type` can be used for core language constructs — `var`, `class`, `function`, `struct`, `enum`, `namespace`, and so on. `support.storage.type` should be used for any _recognized_ value type — certainly core language types like `int`, but also other things if they make sense. `support.other.storage.type` can be used for user-defined types, or any other types that we don't recognize. (Using the `support.other` ordering here adheres to our new convention of reserving all of `support.other` for user-defined things.)
The point of this compromise is that both the `syntax--storage` and `syntax--support` class names are present so that many themes will still highlight these tokens as though they were in the root `storage` namespace.
This commit changes the C/C++ and TypeScript `highlights.scm` files. The results appear to be satisfactory to my eye after testing on two of the builtin themes.
I’ve updated the taxonomy document to reflect this new proposal:
https://gist.github.com/savetheclocktower/c9607b97477d4817911e4f2f8db89679#file-scope-taxonomy-md
…to each PHP section in order to match the functionality of the TextMate grammar.
This was a gigantic endeavor and involved doing several things that were on my “ugh, I’ll get around to it” list:
* To make some of this stuff work, we were using two different PHP parser layers. The root one did the syntax highlighting and separated the HTML from the rest, and the deeper injection was there just so we could scope some ranges as `source.php`.
That doesn’t work because of rules we have about shallower layers acting before deeper layers when iterating through syntax highlighting boundaries. I was trying to be too clever. Now the _root_ layer is the one that does no highlighting, and the _deeper_ layer is the one that exists to apply `source.php` _and then_ perform highlighting and indentation hinting and folding and whatnot.
If we need to, we can move some of the queries to the root layer for whatever reason; maybe we come across a bug six months from now that could be fixed if we made the root layer in charge of folding. We have options.
* All boundaries for `HighlightIterator`s now _either_ open one or more scopes _or_ close one or more scopes. If one boundary can close some scopes and open others, then boundaries on different layers that share a buffer position cannot possibly close/open in the correct order because they can't intermingle. This needed to happen and this was as good of an excuse as any to do it.
* We needed to present sane PHP opening/closing ranges to the injection. Ironically, `tree-sitter-php` itself makes this nearly impossible because of how the tree is structured; it tries to hide the `<?php` and `?>` tags from the PHP layer and treat them as text, despite the fact that PHP is the thing that knows how to parse that.
The best way to do this — at least until I encounter a reason why it can’t work — is to find all the `<?php`s and `?>`s, sort them, group them, and build fake node ranges. This is fine! I mean, it’s ridiculous, but it’s fine! The ranges that we hand off to another parser are allowed to be completely arbitrary!
This lets us do what we were only approximating before: have _one_ PHP injection layer with _any number of_ content ranges, each of which begins with `<?php` and ends with `?>` without trying to claim any surrounding whitespace. This would be worth it even if we didn’t have to do any of this other stuff.
* Each content range of the injection needs _either_ `meta.embedded.line.php` _or_ `meta.embedded.block.php` depending on whether the range begins and ends on the same line. This is not something I was willing to regress on because it's _hard_ to distinguish between PHP and non-PHP unless your editor helps you out, and because I wasn't going to go into a bunch of themes and tell them to treat `source.php` like `meta.embedded`.
This also meant that we needed to be able to add _multiple_ “root” scope names per content range. But we already had a mode in which the `languageScope` injection option could be a callback, so it wasn't hard to make it a callback that could take extra parameters for the buffer and range.
This isn't a feature that I'm eager for other grammars to use, or even necessarily know about, but it was what we needed to deliver feature parity here. And none of it would have been necessary if `tree-sitter-php` made more sensible choices. (Whatever. Maybe what I want isn't possible for some strange reason.)
All existing tests pass. Tests have been written for the new `languageScope` features. Tests need to be written for PHP just like for all other language grammars, but that was on the to-do list anyway. If any of this turns out to be catastrophic, it’s easy to roll back, but tests are passing on my machine and I expect them to pass in CI.
* Properly highlight comments like `/***/`
* Add `intdiv` as a builtin function
* Change PHPDoc tags back to a `keyword` scope rather than an `entity.name.tag` scope
* Highlight more PHPDoc types
…which will make EJS and ERB much more pleasant to use, and which I should've done ages ago.
Most notably: empty tags like `<% %>` are now valid, so the highlighting won't go nuts before you've typed anything.
…and add a distinct copy for TypeScript so that it works even if `language-javascript` is disabled or replaced.
(We might want to just make `language-regex` and have it export a service.)
…and add a new style to `one-dark-syntax` and `one-light-syntax` to ensure continuity of appearance now that object properties are not all treated as `variable`s.
Also, scope private field method definitions in both JS and TS.
Choosing the other side of some dilemmas I had to resolve six months ago. Highlighting between modern and legacy Tree-sitters now appears to be identical to me (with rare exception) on the One Dark theme.
