1
1
mirror of https://github.com/github/semantic.git synced 2024-12-01 17:59:10 +03:00
Commit Graph

60 Commits

Author SHA1 Message Date
Patrick Thomson
54394d61c8 Move around and correct benchmarks. 2019-10-04 12:37:24 -04:00
Patrick Thomson
284fc30f1c Reorganize benchmarks. 2019-10-04 11:25:34 -04:00
Patrick Thomson
65060868bc Make benchmarks operate and switch to gauge.
I find myself wanting to benchmark some I/O code, so I decided to look
at our benchmark suite, to find that it had bitrotted somewhat. This
patch brings it back up to working status, switches to `gauge` (which
is more accurate than `criteron` and incurs fewer dependencies), and
sprinkles some typed paths on there.
2019-09-27 13:18:30 -04:00
Patrick Thomson
ac543651ee Add NFData instances to enable more accurate benchmarking.
Because we're getting serious about benchmarking in the run-up to
Windrose, it's time to bring in the `deepseq` package to ensure that
benchmarks can fully evaluate the result of a test case.

The `deepseq` package provides an `NFData` typeclass:

```
class NFData a where
  rnf :: a -> ()
```

Instances use the `seq` combinator to ensure that the argument to
`rnf` is fully evaluated, returning (). If there is a `Generic`
instance for `a`, the implementation can be omitted. This patch adds
NFData for every syntax node, graph vertex, environment data
structures, and exceptions. It is long, but the work is very
straightforward, so don't panick.

The benchmark suite (`stack bench`) now produces more accurate
results. The benchmarks previously mimicked `rnf` by calling `show` on
the result of an evaluation or graph construction; now that we have
actual `NFData` instances we can use the `nfIO` combinator from
criterion. This has sped up the evaluation benchmarks and reduced
their memory consumption, while it has slowed down the call graph
benchmarks, as those benchmarks weren't evaluating the whole of the
graph.

Unfortunately, this patch increases compile times, as we have to
derive a few more Generic instances. I wish this weren't the case, but
there's little we can do about it now. In the future I have some plans
for how to reduce compile time, and I bet that those gains will at
least nullify the speed hit from this patch.

Now that we have NFData instances for every data type, we can start
benchmarking assignments, in preparation for fixing #2205.

This patch also pulls in updates to `effects` and `fastsum` that add
appropriate NFData instances for the data they vend.
2018-10-17 14:08:47 -04:00
Rick Winfrey
00693a54ee Update bench to work again 2018-08-13 15:42:42 -07:00
Patrick Thomson
bad6e9f53a Fix benchmarks and add benchmark for call graph.
Preyy straightforward.
2018-07-18 11:38:28 -04:00
Patrick Thomson
926e81b8a0 Fix benchmarks.
The interface in Semantic.Util had changed.
Also removes the old `evalRubyFile` and `evalPythonFile` functions,
since they did not typecheck in ghci and do nothing that the project
equivalents don't.
2018-05-04 13:49:37 -04:00
Patrick Thomson
6d5561f9b9 fix misalignment 2018-03-12 10:51:07 -04:00
Patrick Thomson
4378fa5657 add comment 2018-03-08 13:18:36 -05:00
Patrick Thomson
5ffa873279 add script/bench 2018-03-08 10:13:26 -05:00
Patrick Thomson
1ea97534b6 print so that we know everything is evaluated 2018-03-07 19:56:54 -05:00
Patrick Thomson
be2d90f029 add ruby 2018-03-07 19:47:23 -05:00
Patrick Thomson
0ac705b0f5 flesh out some benchmarks 2018-03-07 19:34:20 -05:00
Timothy Clem
c862b7bcf5 Remove benchmark target 2017-04-10 15:04:45 -07:00
Rob Rix
cfbe7e2108 🔥 the generative benchmark stuff. 2017-01-10 14:46:07 -05:00
Rob Rix
a5e7f584c8 hlint insists that this is redundant. 2017-01-09 16:15:36 -05:00
Rob Rix
d2709f6cab First pass at generative testing using leancheck. 2017-01-09 16:12:25 -05:00
Timothy Clem
6cbfa91475 Try out moving project around 2016-10-05 16:17:46 -07:00
Rob Rix
114e203c4a Handle rounding error by taking exactly n measurements. 2016-06-06 17:20:00 -04:00
Rob Rix
9eff3053ec Traverse up to the default size in n steps. 2016-06-06 17:19:42 -04:00
Rob Rix
927ee4345f Benchmark SES over fixed intervals. 2016-06-06 17:17:37 -04:00
Rob Rix
a7dd94f9d7 Compute element cost as sums of string length. 2016-06-06 17:17:19 -04:00
Rob Rix
b45bc321b2 Compute SES over lists of strings. 2016-06-06 17:16:52 -04:00
Rob Rix
2648406b34 Generalize generativeBenchmark over Gen a. 2016-06-06 17:15:51 -04:00
Rob Rix
ef57f3aece Add a baseline benchmark of SES. 2016-06-06 15:32:38 -04:00
Rob Rix
95eabb8b5d Add an NFData instance for Patch. 2016-06-06 15:32:16 -04:00
Rob Rix
893f53fa08 🔥 some unused setup from the numberedRows benchmark. 2016-06-06 13:09:17 -04:00
Rob Rix
44a1599afa 🔥 redundant imports. 2016-06-06 13:06:09 -04:00
Rob Rix
de41789aab 🔥 the pointless numberedRows benchmark. 2016-06-06 13:06:04 -04:00
Rob Rix
29a725b125 Move ArbitraryDiff into semantic-diff proper. 2016-06-06 12:07:07 -04:00
Rob Rix
8780ea3f26 Move the Arbitrary instances for Join (,) and Join These into semantic-diff proper. 2016-06-06 11:50:45 -04:00
Rob Rix
4eb76ff3a3 Move the Arbitrary instance for Data.These into semantic-diff proper. 2016-06-06 11:46:30 -04:00
Rob Rix
378b577175 Move the Arbitrary instance for Patch into semantic-diff proper. 2016-06-06 11:43:38 -04:00
Rob Rix
3d4a348072 Depend on semantic-diff’s definition of ArbitraryTerm in the benchmarks. 2016-06-01 09:15:15 -04:00
Rob Rix
49898ba79e Try to generate ArbitraryTerms at fixed sizes. 2016-06-01 08:40:57 -04:00
Rob Rix
6eb99db102 Spacing. 2016-05-31 21:05:07 -04:00
Rob Rix
77e05809c6 Shrink ArbitraryDiffs. 2016-05-31 20:46:40 -04:00
Rob Rix
aa5f3f35ed Generate arbitrary Pure cases. 2016-05-31 20:46:35 -04:00
Rob Rix
bbec20d2b5 Define ArbitraryDiff over ArbitraryTerm. 2016-05-31 20:46:18 -04:00
Rob Rix
5925d3d3d4 Implement shrink over Patch. 2016-05-31 20:46:03 -04:00
Rob Rix
28b7909e06 Add an Arbitrary instance over ArbitraryTerm. 2016-05-31 20:39:52 -04:00
Rob Rix
f8443c855c Add an Arbitrary instance over Patch. 2016-05-31 20:39:39 -04:00
Rob Rix
ea16b4616c Add ArbitraryTerm to the benchmarks. 2016-05-31 20:39:24 -04:00
Rob Rix
f0f11a9f0a Simplify the definition of arbitrary over ArbitraryDiff. 2016-05-31 20:32:55 -04:00
Rob Rix
dadaa6b420 Add an Arbitrary instance for ArbitraryDiff. 2016-05-31 20:24:49 -04:00
Rob Rix
fd4ad219ae Construct Diffs from ArbitraryDiffs. 2016-05-31 20:19:51 -04:00
Rob Rix
206c3cddc3 Add an ArbitraryDiff newtype wrapper. 2016-05-31 20:19:39 -04:00
Rob Rix
64ac3703c0 Sort the benchmarks by the measured size. 2016-05-31 16:01:36 -04:00
Rob Rix
c7f51beb12 📝 generativeBenchmark. 2016-05-31 11:47:56 -04:00
Rob Rix
207c0ac24c Take the count as a parameter. 2016-05-31 11:45:17 -04:00