prepack/scripts
Caleb Meredith 108dea533b Add serializer mode for test262 execution (#2290)
Summary:
I want to better understand the bugs we have in abstract evaluation. One of my ideas to do this is to replace all the literals in test262 tests with abstract values and see what our test coverage is. The first step to do this is to serialize the test262 sources after running them in Prepack and checking if their generated JavaScript runs correctly. This PR does that by adding a `--serializer` flag.

With my methodology, all test262 harnesses are serialized every time since they are in the global scope. This unfortunately seems to be unavoidable since doing otherwise would change program semantics and break the tests.

Running `time yarn test-test262` takes about 2 minutes and has a 98% pass rate. Running `time yarn test-test262 --serializer` takes about 5 minutes and has a 95% pass rate. [Here’s a diff of the two  results.](https://gist.github.com/calebmer/1c9fe396b63ba055458c599c2be18a58)

[Here is a selection of some of the bugs](https://gist.github.com/calebmer/1ac381096a4aa7be1fc7dc2163276ab4) in the serializer caught by running test262 with the Prepack serializer. I might open issues for these, but they can be a bit pedantic. I might fix some of them depending on how important they are to the React code we want to compile.

Notably we have:

- Lots of invariants being triggered. I particularly saw [a lot of this invariant](7d355ef4c5/src/serializer/ResidualHeapVisitor.js (L531)). (An example of this is [`07.md`](https://gist.github.com/calebmer/1ac381096a4aa7be1fc7dc2163276ab4#file-07-md).)
- Lots of values that are visited, but not serialized. (An example of this is [`01.md`](https://gist.github.com/calebmer/1ac381096a4aa7be1fc7dc2163276ab4#file-01-md).)
- Some incorrect outputs only caught by executing code. Could not be caught by static analysis. Particularly around class serialization.

I’ll build off this PR to get coverage for the abstract evaluator next, but I found these results interesting on their own so decided to do this in two PRs.
Pull Request resolved: https://github.com/facebook/prepack/pull/2290

Differential Revision: D8908479

Pulled By: calebmer

fbshipit-source-id: aa57d47611fbd92af33e4647fed7bf7990fb6de1
2018-07-18 18:54:51 -07:00
..
debug-fb-www.js Upgrade Prepack to Babel 7 (#2256) 2018-07-14 09:55:18 -07:00
detect_bad_deps.js Updating Flow to .76 (#2229) 2018-07-09 17:25:15 -07:00
generate-sourcemaps-test.js Prettier 1.13.4 fixes 2018-06-06 12:49:59 -07:00
instrumentor.js Upgrade Prepack to Babel 7 (#2256) 2018-07-14 09:55:18 -07:00
lint-config.js Upgrade ESlint to support ES2018 2018-05-31 17:04:35 -07:00
multi-runner.js Fix prettier and lint failures 2017-10-23 00:00:47 -07:00
prettier.js Ignore test262 for Prettier (#2242) 2018-07-11 07:23:38 -07:00
publish-gh-pages.sh Update build command to explictly invoke yarn build 2018-05-22 04:31:58 -07:00
test262-filters.yml Use integration api to run Test262 tests 2017-12-01 17:46:29 -08:00
test262-runner.js Add serializer mode for test262 execution (#2290) 2018-07-18 18:54:51 -07:00
test262.js Prettier 1.13.4 fixes 2018-06-06 12:49:59 -07:00
test-error-handler.js Prettier 1.13.4 fixes 2018-06-06 12:49:59 -07:00
test-internal-react.js Prettier 1.13.4 fixes 2018-06-06 12:49:59 -07:00
test-internal.js Upgrade Prepack to Babel 7 (#2256) 2018-07-14 09:55:18 -07:00
test-residual.js Prettier 1.13.4 fixes 2018-06-06 12:49:59 -07:00
test-runner.js Ignore modifications of Prepack-specific intrinsic properties (#2269) 2018-07-18 10:54:59 -07:00
test-sourcemaps.sh Extract scripts from npm package (#463) 2017-04-26 12:26:18 -07:00
test-std-in.sh fix #1239 - Command-line based syntax errors now print location 2017-12-18 13:56:26 -08:00