…when a project-specific config is present.
Most people don't use a project-specific config, which is why this bug has been present for ages. Read the new spec for an explanation.
…by namespacing them as `@_IGNORE_.foo`, `@_IGNORE_.bar`, etc.
It's sometimes necessary to define a capture in a query not because you want to apply a scope name, but because you need to use it as the argument to a predicate. `@_IGNORE_` was intended for that purpose, but it was the _only_ capture name with that special effect.
Now, you can specify any number of captures that don't apply scope names. As long as it equals `@_IGNORE_` or _starts with_ `@_IGNORE_.`, it'll have the same effect. This lets you target two or more nodes and use them all in predicates in the same query without any of them applying a scope name.
…to each PHP section in order to match the functionality of the TextMate grammar.
This was a gigantic endeavor and involved doing several things that were on my “ugh, I’ll get around to it” list:
* To make some of this stuff work, we were using two different PHP parser layers. The root one did the syntax highlighting and separated the HTML from the rest, and the deeper injection was there just so we could scope some ranges as `source.php`.
That doesn’t work because of rules we have about shallower layers acting before deeper layers when iterating through syntax highlighting boundaries. I was trying to be too clever. Now the _root_ layer is the one that does no highlighting, and the _deeper_ layer is the one that exists to apply `source.php` _and then_ perform highlighting and indentation hinting and folding and whatnot.
If we need to, we can move some of the queries to the root layer for whatever reason; maybe we come across a bug six months from now that could be fixed if we made the root layer in charge of folding. We have options.
* All boundaries for `HighlightIterator`s now _either_ open one or more scopes _or_ close one or more scopes. If one boundary can close some scopes and open others, then boundaries on different layers that share a buffer position cannot possibly close/open in the correct order because they can't intermingle. This needed to happen and this was as good of an excuse as any to do it.
* We needed to present sane PHP opening/closing ranges to the injection. Ironically, `tree-sitter-php` itself makes this nearly impossible because of how the tree is structured; it tries to hide the `<?php` and `?>` tags from the PHP layer and treat them as text, despite the fact that PHP is the thing that knows how to parse that.
The best way to do this — at least until I encounter a reason why it can’t work — is to find all the `<?php`s and `?>`s, sort them, group them, and build fake node ranges. This is fine! I mean, it’s ridiculous, but it’s fine! The ranges that we hand off to another parser are allowed to be completely arbitrary!
This lets us do what we were only approximating before: have _one_ PHP injection layer with _any number of_ content ranges, each of which begins with `<?php` and ends with `?>` without trying to claim any surrounding whitespace. This would be worth it even if we didn’t have to do any of this other stuff.
* Each content range of the injection needs _either_ `meta.embedded.line.php` _or_ `meta.embedded.block.php` depending on whether the range begins and ends on the same line. This is not something I was willing to regress on because it's _hard_ to distinguish between PHP and non-PHP unless your editor helps you out, and because I wasn't going to go into a bunch of themes and tell them to treat `source.php` like `meta.embedded`.
This also meant that we needed to be able to add _multiple_ “root” scope names per content range. But we already had a mode in which the `languageScope` injection option could be a callback, so it wasn't hard to make it a callback that could take extra parameters for the buffer and range.
This isn't a feature that I'm eager for other grammars to use, or even necessarily know about, but it was what we needed to deliver feature parity here. And none of it would have been necessary if `tree-sitter-php` made more sensible choices. (Whatever. Maybe what I want isn't possible for some strange reason.)
All existing tests pass. Tests have been written for the new `languageScope` features. Tests need to be written for PHP just like for all other language grammars, but that was on the to-do list anyway. If any of this turns out to be catastrophic, it’s easy to roll back, but tests are passing on my machine and I expect them to pass in CI.
This fixes a subtle breaking change in the WASM tree-sitter modes that
affected the semanticolor package. (And probably only that package.)
Although the node text is unused in `idForScope` in the core editor, the
legacy tree-sitter mode was specifically modified to pass the node text
into `idForScope` so that the semanticolor package could make use of it
to give custom scopes (and thus colors) to different symbols. (On
activation, semanticolor monkey patches that method in relevant
grammars.)
This alone is not enough to get sematicolor working with Pulsar's new
WASM tree-sitter modes, but it's one piece of the puzzle.
Ref: 97b905a2b0
Ref: https://github.com/atom/atom/pull/20212
Ref: 1bf7fc4025/src/tree-sitter-language-mode.js (L1321-L1324)
Ref: d219ef9a1f/lib/semanticolor-grammar.js (L44-L65)
…and create `useLegacyTreeSitter` for those who want to opt into the previous default behavior.
(Legacy Tree-sitter grammars will soon be removed, but this is a step toward that future!)
Previously, we threw an error when a scope adjustment violated its bounds constraints, but that's a bit disruptive for everyday use. Instead, we throw an error in dev mode (so that the grammar's author doesn't fail to notice the problem), but downgrade it to a warning outside of dev mode so that it's recoverable.
There's a chance that the warning will be _too_ subtle, but we'll give it a shot.
We also include more diagnostic information so that it's clearer exactly _where_ the violation is happening.
…pollute the UI.
Anything that is present in a project-specific settings override should not
appear in the settings panel. The settings panel should only be able to read
from the global config file, because that's the only place it writes to.
But if we notice that a setting in the UI is affected by a project-specific
setting, we'll include a warning so that the user doesn't wonder why changes
in the settings UI don't seem to have an effect.
I realized a few days ago that the `#is?` and `#is-not?` predicates — which
puzzled me upon first investigation back in February, and which I'd ignored ever
since — are exactly what I should've been using all along for scope tests.
They are counterparts to `#set!` in that they are two additional buckets in
which to store arbitrary data. And they remove the need to have both
`test.onlyIfFoo` and `test.onlyIfNotFoo`, since the presence of `test.onlyIfFoo`
in the `refutedProperties` bucket can behave opposite to the its presence in the
`assertedProperties` bucket.
This further means that a test called `test.onlyIfFoo` can be renamed
`test.foo`. The `onlyIf` was my way of making it clear through context that this
`#set!` predicate actually enforced a criterion — but that's implicit now that
it'll be using an `#is?` or `#is-not?` predicate.
I also took the opportunity to move `test.final` and `test.shy` to their own
namespace. `final` is an oddball in the sense that it sets criteria for _other_
captures as well as its own, and they're both oddballs in the sense that they
consider the effects of other captures on the same range. And I don't want to
introduce the idea that a predicate like `#is?` can have side effects.
So they stay on `#set!` and live at `capture.final` and `capture.shy`,
respectively.
To summarize, here's a before and after:
((string "\"" @punctuation.start)
(#set! test.onlyIfFirst true)
(#set! test.final true)
((bar) @baz
(#set! test.onlyIfNotDescendantOfType thud))
becomes
((string "\"" @punctuation.start)
(#is? test.first true)
(#set! capture.final true)
((bar) @baz
(#is-not? test.descendantOfType thud))
This PR changes all the built-in grammars. I don't know if there are any
third-party modern-Tree-sitter grammars out there except for mine, but I've
added a simple compatibility layer so that all of the old predicates should
still work for now. But I'm going to remove that code before we ship this stuff
for real.
