Instead of hard-coding the HTTP API we use for uploading errors to our
internal service, let the user provide an arbitrary function (in the IO
monad) for uploading them. The default config will create an error
reporter that logs errors to the console, just like happened before if
you didn't set the `HAYSTACK_URL` environment variable.
This is part of the critical path for repository tagging, so we should
test it to make sure it doesn't explode. This introduces a test that
creates a temporary git repo and queries it to make sure that adding
and then reading from a blob works properly.
We originally put this here because we were recompiling
haskell-tree-sitter on every ghci reload, and it sped up the process
of linking the built haskell-tree-sitter into ghci. Now that we
compile haskell-tree-sitter once, its use is no longer applicable, and
it interferes with profiling builds.
This was OSS'd because it turned out to be a little tricky and we
didn't want external consumers to have to implement it themselves. No
reason to keep it around in our tree.
Provides a `Push` type. To build that, we needed `RequestContext`,
`UserType`, `RequestMethod`, `Spamurai`, `Timestamp`, and `IPVersion`.
The schemas have all been taken from https://github.com/github/hydro-schemas/.
Now that we're on GHC 8.6, we can use `-XDerivingVia` in many cases
where we previously had to write instances by hand. If you're not
familiar with `-XDerivingVia`, the [GHC proposal][ghc] is a good place
to start.
[ghc]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0023-deriving-via.rst
Thanks to the `generic-monoid` package, we can derive a `Semigroup`
instance for any product type whose members are `Semigroups`, and the
same goes for `Monoid`. This entails an extra dependency, but it is
better than the `generic-deriving` package, which is way too much overhead.
I've also switched some trivial definitions to newtype-deriving.
Please be aware that this bumps `hlint` and `haskell-src-exts` so that
`hlint` doesn't choke on the `DerivingVia` extension. You'll need to
`stack install hlint` to get it on your `PATH`. Apologies!
Bumps most of our dependencies.
Code changes:
* algebraic-graphs-0.3 no longer provides a Foldable or Traversable
implementation for `Graph`, so now neither does our `Graph` type.
* CMark parsing now uses safe rendering of raw HTML/URLs by default,
so there is no reason to pass an `optSafe` anymore.
* algebraic-graphs now no longer requires an Eq constraint on the
output type it generates, so we can ditch our Serializing.DOT
module. (Andrey fixed this for us specifically!)
Given that @tclem and I have found the matcher API frustrating, I've
taken a stab at improving its ergonomics, and I've found some success
in separating composition of matchers from predicate-based narrowing
thereof.
The biggest change here is the elimination of the old `match`
combinator, which proved to be clumsy in that it complected narrowing
and composition. Top-down matching combinators are now written with
the `need` combinator and the `>>>` combinator, which is more readable
and more versatile. Here's a matcher that accepts functions with
Python docstrings:
```haskell
docstringMatcher :: ( Decl.Function :< fs
, [] :< fs
, Lit.TextElement :< fs
, term ~ Term (Sum fs) ann
) => Matcher term term
docstringMatcher = target <*
(need Decl.functionBody
>>> narrow @[]
>>> mhead
>>> narrow @Lit.TextElement
>>> ensure Lit.isTripleQuoted))
```
Pretty readable, right? Each step of the tree regular expression -
choosing function bodies, ensuring said bodies are lists, examining
the first element, and choosing only TextElements containing
triple-quoted strings - is made implicit. The old way would have
looked something like this:
```haskell
docstringMatcher = target <* match Decl.functionBody
$ narrow
$ matchM listToMaybe
$ target <* ensure Lit.isTripleQuoted
```
which is a good deal more disorganized and less flexible
in the quite-common case of applying functions during a
matching pass. Separating the act of composition from
function application is a big win here.
Further comments are inline.
@tclem and I found ourselves wanting an arrow-like combinator that
promotes a given function to a Matcher. While I think an Arrow
instance is going a little overboard, there's no harm in adding a
'purely' function, the naming of which is commensurate with the
rewriting DSL.
This also renames the module, since there's not anything really
abstract about matching (indeed, it is quite concrete).
This looks like a big patch, but it's very straightforward: no
behavior has changed.
After the umpteenth time spent hitting a compile error because I
passed a `FilePath` rather than a `File` to `readBlobFromPath`, I
decided to finally make the needed refactors to Semantic.IO, and to
split off the `File` type and `Files` effect. This patch:
* adds the `MonadIO` class to `Prologue`'s export list
* moves `File` into `Data.File`
* moves `Handle` into `Data.Handle`
* moves `Files` into `Semantic.Task.Files`
* moves functions for reading blobs into `Data.Blob`
* keeps general IO helpers in Semantic.IO
* renames `readFile` to `readBlobFromFile`
* renames `readBlobFromPath` to `readBlobFromFile'`
This should have a positive effect on compile times and ease of
navigation throughout the codebase.
In the past few years, GHC has been moving to remove the `fail` method
from the definition of `Monad`, a longtime wart, and requiring monads
that fail due to an incomplete pattern match to implement `MonadFail`.
You can read about it[here](https://wiki.haskell.org/MonadFail_Proposal).
Though this move is still in progress, we can opt into it by turning
on the `-XMonadFailDesugaring` extension.
The matching and rewriting systems will both benefit from this, as the
incomplete pattern match in following rewrite rule will crash without
`-XMonadFailDesugaring`, even though the sensible and correct thing
for the rule to do is call out to its `MonadFail` instance:
```haskell
-- crashes the program without -XMonadFailDesugaring
getReceiver :: Rule a (Ruby.Send term) term
getReceiver = do
(Ruby.Send (Just rec) _ _ _) <- target
pure rec
```
In addition, turning on `MonadFailDesugaring` will warn you if you put
an incomplete pattern match in a monad that doesn't implement `MonadFail`.
This setting will become implicit in GHC 8.6, so this is a good chance
to make sure that we don't introduce any incomplete patterns going forward.
This was the longest (in terms of line count) file in the project.
Splitting it up will save on compile time, as it did for TypeScript.
I observe a ~20sec speedup from `stack clean semantic && stack build semantic:lib`
Right now, Semantic.Version is recompiled on every invocation of
`stack build`, since we marked it as `-fforce-recomp` to ensure that
all deployments are tagged appropriately for haystack. However, this
entails a good deal of wasted time during development. With some
liberal application of `CPP`, we can make this recompilation only
happen on CI, thanks to the `release` flag and passing in a compiler
flag.
To test:
* apply the patch
* `stack build semantic`, then `stack exec semantic -- -v`. It should
print `semantic version 0.4.0 (<development>)`.
* `stack clean semantic && stack build --ghc-options=-DCOMPUTE_GIT_SHA`.
`stack exec semantic -- -v` should then print out the correct SHA.
Though it's probably not strictly necessary, I've marked the
`semantic` and `semanticd` executables to compile with
-DCOMPUTE_GIT_SHA, just in case.
Because we're getting serious about benchmarking in the run-up to
Windrose, it's time to bring in the `deepseq` package to ensure that
benchmarks can fully evaluate the result of a test case.
The `deepseq` package provides an `NFData` typeclass:
```
class NFData a where
rnf :: a -> ()
```
Instances use the `seq` combinator to ensure that the argument to
`rnf` is fully evaluated, returning (). If there is a `Generic`
instance for `a`, the implementation can be omitted. This patch adds
NFData for every syntax node, graph vertex, environment data
structures, and exceptions. It is long, but the work is very
straightforward, so don't panick.
The benchmark suite (`stack bench`) now produces more accurate
results. The benchmarks previously mimicked `rnf` by calling `show` on
the result of an evaluation or graph construction; now that we have
actual `NFData` instances we can use the `nfIO` combinator from
criterion. This has sped up the evaluation benchmarks and reduced
their memory consumption, while it has slowed down the call graph
benchmarks, as those benchmarks weren't evaluating the whole of the
graph.
Unfortunately, this patch increases compile times, as we have to
derive a few more Generic instances. I wish this weren't the case, but
there's little we can do about it now. In the future I have some plans
for how to reduce compile time, and I bet that those gains will at
least nullify the speed hit from this patch.
Now that we have NFData instances for every data type, we can start
benchmarking assignments, in preparation for fixing #2205.
This patch also pulls in updates to `effects` and `fastsum` that add
appropriate NFData instances for the data they vend.
Right now, when opening a shell with `script/ghci`, you can encounter
a tremendous slowdown by turning on pretty-printing (with `pretty`)
then printing any value (`[0..3]` should work).
This stems from the fact that our `.ghci` specifies the `prettyShow`
function, defined in Semantic.Util, as its `-interactive-print`
function. While this is a good choice of a pretty-printer, the fact
that it is in Util, a file which imports many modules and uses fancy
types, is not good: pretty-printing cannot begin until Util is
recompiled and linked in. This explains why benchmarking this slowdown
revealed nothing but `dlsym` calls: we thought it was due to linking
in the tree-sitter libraries, but it was actually waiting for Util to
compile and link in.
The fix is simple: define `prettyShow` in another module. This should
provide significant speedups to our developer workflow.
Prefixes on data constructors are generally an antipattern in Haskell:
if you're concerned about name collisions, have clients use qualified
imports for whatever modules they need. As such, this removes the T-
prefixes from the `Token` and `Context` types. This also renames
Context to Scope, which is a more exact and readable name.
I figure that since we're starting to show this to other teams, it
behooves our documentation to have a more meaningful description.
Happy to change this if y'all can come up with a snappier synopsis.
