Given that @tclem and I have found the matcher API frustrating, I've
taken a stab at improving its ergonomics, and I've found some success
in separating composition of matchers from predicate-based narrowing
thereof.
The biggest change here is the elimination of the old `match`
combinator, which proved to be clumsy in that it complected narrowing
and composition. Top-down matching combinators are now written with
the `need` combinator and the `>>>` combinator, which is more readable
and more versatile. Here's a matcher that accepts functions with
Python docstrings:
```haskell
docstringMatcher :: ( Decl.Function :< fs
, [] :< fs
, Lit.TextElement :< fs
, term ~ Term (Sum fs) ann
) => Matcher term term
docstringMatcher = target <*
(need Decl.functionBody
>>> narrow @[]
>>> mhead
>>> narrow @Lit.TextElement
>>> ensure Lit.isTripleQuoted))
```
Pretty readable, right? Each step of the tree regular expression -
choosing function bodies, ensuring said bodies are lists, examining
the first element, and choosing only TextElements containing
triple-quoted strings - is made implicit. The old way would have
looked something like this:
```haskell
docstringMatcher = target <* match Decl.functionBody
$ narrow
$ matchM listToMaybe
$ target <* ensure Lit.isTripleQuoted
```
which is a good deal more disorganized and less flexible
in the quite-common case of applying functions during a
matching pass. Separating the act of composition from
function application is a big win here.
Further comments are inline.
@tclem and I found ourselves wanting an arrow-like combinator that
promotes a given function to a Matcher. While I think an Arrow
instance is going a little overboard, there's no harm in adding a
'purely' function, the naming of which is commensurate with the
rewriting DSL.
This also renames the module, since there's not anything really
abstract about matching (indeed, it is quite concrete).
This looks like a big patch, but it's very straightforward: no
behavior has changed.
After the umpteenth time spent hitting a compile error because I
passed a `FilePath` rather than a `File` to `readBlobFromPath`, I
decided to finally make the needed refactors to Semantic.IO, and to
split off the `File` type and `Files` effect. This patch:
* adds the `MonadIO` class to `Prologue`'s export list
* moves `File` into `Data.File`
* moves `Handle` into `Data.Handle`
* moves `Files` into `Semantic.Task.Files`
* moves functions for reading blobs into `Data.Blob`
* keeps general IO helpers in Semantic.IO
* renames `readFile` to `readBlobFromFile`
* renames `readBlobFromPath` to `readBlobFromFile'`
This should have a positive effect on compile times and ease of
navigation throughout the codebase.
In the past few years, GHC has been moving to remove the `fail` method
from the definition of `Monad`, a longtime wart, and requiring monads
that fail due to an incomplete pattern match to implement `MonadFail`.
You can read about it[here](https://wiki.haskell.org/MonadFail_Proposal).
Though this move is still in progress, we can opt into it by turning
on the `-XMonadFailDesugaring` extension.
The matching and rewriting systems will both benefit from this, as the
incomplete pattern match in following rewrite rule will crash without
`-XMonadFailDesugaring`, even though the sensible and correct thing
for the rule to do is call out to its `MonadFail` instance:
```haskell
-- crashes the program without -XMonadFailDesugaring
getReceiver :: Rule a (Ruby.Send term) term
getReceiver = do
(Ruby.Send (Just rec) _ _ _) <- target
pure rec
```
In addition, turning on `MonadFailDesugaring` will warn you if you put
an incomplete pattern match in a monad that doesn't implement `MonadFail`.
This setting will become implicit in GHC 8.6, so this is a good chance
to make sure that we don't introduce any incomplete patterns going forward.
This was the longest (in terms of line count) file in the project.
Splitting it up will save on compile time, as it did for TypeScript.
I observe a ~20sec speedup from `stack clean semantic && stack build semantic:lib`
Right now, Semantic.Version is recompiled on every invocation of
`stack build`, since we marked it as `-fforce-recomp` to ensure that
all deployments are tagged appropriately for haystack. However, this
entails a good deal of wasted time during development. With some
liberal application of `CPP`, we can make this recompilation only
happen on CI, thanks to the `release` flag and passing in a compiler
flag.
To test:
* apply the patch
* `stack build semantic`, then `stack exec semantic -- -v`. It should
print `semantic version 0.4.0 (<development>)`.
* `stack clean semantic && stack build --ghc-options=-DCOMPUTE_GIT_SHA`.
`stack exec semantic -- -v` should then print out the correct SHA.
Though it's probably not strictly necessary, I've marked the
`semantic` and `semanticd` executables to compile with
-DCOMPUTE_GIT_SHA, just in case.
Because we're getting serious about benchmarking in the run-up to
Windrose, it's time to bring in the `deepseq` package to ensure that
benchmarks can fully evaluate the result of a test case.
The `deepseq` package provides an `NFData` typeclass:
```
class NFData a where
rnf :: a -> ()
```
Instances use the `seq` combinator to ensure that the argument to
`rnf` is fully evaluated, returning (). If there is a `Generic`
instance for `a`, the implementation can be omitted. This patch adds
NFData for every syntax node, graph vertex, environment data
structures, and exceptions. It is long, but the work is very
straightforward, so don't panick.
The benchmark suite (`stack bench`) now produces more accurate
results. The benchmarks previously mimicked `rnf` by calling `show` on
the result of an evaluation or graph construction; now that we have
actual `NFData` instances we can use the `nfIO` combinator from
criterion. This has sped up the evaluation benchmarks and reduced
their memory consumption, while it has slowed down the call graph
benchmarks, as those benchmarks weren't evaluating the whole of the
graph.
Unfortunately, this patch increases compile times, as we have to
derive a few more Generic instances. I wish this weren't the case, but
there's little we can do about it now. In the future I have some plans
for how to reduce compile time, and I bet that those gains will at
least nullify the speed hit from this patch.
Now that we have NFData instances for every data type, we can start
benchmarking assignments, in preparation for fixing #2205.
This patch also pulls in updates to `effects` and `fastsum` that add
appropriate NFData instances for the data they vend.
Right now, when opening a shell with `script/ghci`, you can encounter
a tremendous slowdown by turning on pretty-printing (with `pretty`)
then printing any value (`[0..3]` should work).
This stems from the fact that our `.ghci` specifies the `prettyShow`
function, defined in Semantic.Util, as its `-interactive-print`
function. While this is a good choice of a pretty-printer, the fact
that it is in Util, a file which imports many modules and uses fancy
types, is not good: pretty-printing cannot begin until Util is
recompiled and linked in. This explains why benchmarking this slowdown
revealed nothing but `dlsym` calls: we thought it was due to linking
in the tree-sitter libraries, but it was actually waiting for Util to
compile and link in.
The fix is simple: define `prettyShow` in another module. This should
provide significant speedups to our developer workflow.
Prefixes on data constructors are generally an antipattern in Haskell:
if you're concerned about name collisions, have clients use qualified
imports for whatever modules they need. As such, this removes the T-
prefixes from the `Token` and `Context` types. This also renames
Context to Scope, which is a more exact and readable name.
I figure that since we're starting to show this to other teams, it
behooves our documentation to have a more meaningful description.
Happy to change this if y'all can come up with a snappier synopsis.
Rather than relying on the `Read` instance for `Integer`, let's make
our assumptions about the format explicit. This was mostly a matter of
extracting internal functions from the `Scientific` parser.
It's always been easy to forget the `--fast` flag. With the introduction of
`semanticd`, forgetting that carries a significant speed hit, as GHC
optimizes the resulting executable very thoroughly, which entails a
significant uptick in time spent building. For our collective sanity's
sake, this patch makes `stack build` operate without optimizations. CI
and deploy jobs pass `--flag semantic:release` to `stack`, which
specifies the `-O2` level of optimizations.
Timing `stack clean && stack build` (which rebuilds the tree-sitter
packages) went from 6m17s to 3m04s on my machine. When building just
`semantic` by itself, standard builds take 1m06s, whereas builds with
`--flag semantic:release` take 4m12s. (GHC does a lot of
optimizations!)
