Summary:
…pecify modules
Release Notes: None
Sometimes it'll be useful to allow the user to specify which specific modules you want to speculatively execute. This allows that by turning `--initializeMoreModules` into `--modulesToInitialize <ALL | comma separated list of modules>`
Updated tests as well.
Pull Request resolved: https://github.com/facebook/prepack/pull/2576
Differential Revision: D10092554
Pulled By: cblappert
fbshipit-source-id: bf601e14c2be59c865ae9513c914f39325521945
Summary:
Release note: none
Closes#2435Closes#1829
Join.composeWithEffects composes a forked completion with subsequent effects. When two or more forks could end normally, this could result in shallow copies of the subsequent effects. These were then joined together and applied, so it was mostly OK. The generator of the subsequent effects, however, ended up being joined with itself and thus transformed the generator tree to a DAG, which is not desirable for the serializer.
The new approach is to extract a join condition from the forked completion and using it to join the subsequent effects with a newly constructed empty effects. The condition ensures that the subsequent effects are applied only in situations where the forked completion is not abrupt.
Extracting this condition makes for complicated abstract expressions and this uncovered some existing bugs and limitations that are also addressed in this pull request. As a side effect, path conditions are now longer and the time to compile unrolled loops with conditional abrupt completions inside their bodies has gone up so much that the unroll limit had to be lowered.
Please note that the expected output React tests has changed because of re-ordering. I'm none too sure that this re-ordering is necessarily benign, so please review carefully.
Pull Request resolved: https://github.com/facebook/prepack/pull/2523
Differential Revision: D9623729
Pulled By: hermanventer
fbshipit-source-id: 737096bba54a7a2ad300dc29882ea1b7829ac745
Summary:
Release note: none
This attempts to reduce exponential blowup in the simplifier by doing more caching while computing implications and by imposing a depth limit on the number of simplify and implies calls.
It also tries to make implies and impliesNot more symmetrical, so that things are less ad-hoc.
Pull Request resolved: https://github.com/facebook/prepack/pull/2530
Differential Revision: D9664026
Pulled By: hermanventer
fbshipit-source-id: f7a9135b06298a2b77ad05bf377982a9b37e4ad1
Summary:
Release notes:
This adds a `reactFailOnUnsupportedSideEffects` flag for React internal testing with `debug-fb-www` and ignores `try/catch` PP0021 errors for internal testing too. These are intended to be temporary changes purely for internal FB testing.
Pull Request resolved: https://github.com/facebook/prepack/pull/2531
Differential Revision: D9656578
Pulled By: trueadm
fbshipit-source-id: 9339dd8245e5a03d567ca2fd8e24e90152f1e22b
Summary:
Release notes: Adding CLI options --warnaserror, --diagnosticaserror, --nodiagnostic
This resolves#2517.
- --warnAsError: Turns all warnings into errors.
- --diagnosticAsError: Must be followed by a comma-separated list of non-fatal-error PPxxxx diagnostic codes that should get turned into (recoverable) errors.
- --noDiagnostic: Must be followed by a comma-separated list of non-fatal-error PPxxxx diagnostic codes that should get suppressed.
Adding tests.
Pull Request resolved: https://github.com/facebook/prepack/pull/2521
Differential Revision: D9616473
Pulled By: NTillmann
fbshipit-source-id: c66d7396005699d4d50f801a419fa4879bd8ffc4
Summary:
Release notes: landing two previously reverted PRs
I found the issue.
Flow doesn't support optional fields in classes. This effectively becomes enforced with Babel 7 since it treats the type annotations as field initializers. Which means that these fields always gets created.
We happened to only use this newer plugin for class fields internally which broke our builds only there.
2b4546d Use `undefined` instead of missing to represent absent field in descriptors. Fixed all the callsites that checks for `hasOwnProperty` or `in` that I could find.
922d40c Fixes a Flow issue in the text printer.
I filed a [follow up issue for ObjectValue](https://github.com/facebook/prepack/issues/2510) since it has the same problem.
Pull Request resolved: https://github.com/facebook/prepack/pull/2511
Reviewed By: trueadm
Differential Revision: D9569949
Pulled By: sebmarkbage
fbshipit-source-id: f8bf84c4385de4f0ff6bcd45badacd3b8c88c533
Summary:
Release notes: None
This resolves#1944.
The functionality that Prepack provides can be seen as being done by two separate engines:
1) There's the "front-end", which does symbolic execution / abstract interpretation,
computing "effects" / "generator trees" for the global code and optimized functions.
2) There's the "back-end", which takes this intermediate representation, computes what's
reachable, and then turns it into a new executable program by performing transformations
such as breaking cycles.
Before, Prepack developers had very few tools available to understand what goes on between the
front-end and back-end. Usually, they just look at the final output, imagine what the
intermediate representation might have been, or maybe use to debugger to poke around
memory locations, or to invoke helper functions to inspect live state that way, hoping the debugger doesn't crash along the way.
This PR (and other associated PRs such as #2490, #2491) try to improve that state by turning the intermediate representation into a first-class data structure that is printable into a human-readable textual form. This will help...
- in understanding what's going on
- enabling different back-ends by ensuring that there's a first-class intermediate representation
- enabling future transformations on a well-defined intermediate data structure, ideally breaking up what the current serializer implementation does
- new ways of testing, e.g. via snapshots of the intermediate representation, or
(once there is a parser) feeding hand-written IR into the back-end
For example, for this program...
```js
let x = __abstract("number", "(x)");
if (x > 42) {
let t = Date.now();
let u = 2 + 3
console.log(t + u);
} else {
global.result = x;
}
```
... the IR would currently look like this:
```
(entry point): "main(#0)"
* value#0 = ">"(@"(x)", 42)
if value#0
then: "evaluateNodeForEffects(#4)"
path conditions value#0
_$0 := ABSTRACT_FROM_TEMPLATE<template source @"global.Date.now()">[isPure]
* value#1 = "+"(5, _$0)
CONSOLE_LOG("log", value#1)
else: "evaluateNodeForEffects(#11)"
* value#2 = "!"(value#0)
path conditions value#2
GLOBAL_ASSIGNMENT(@"(x)", "result")
```
Notes:
- Indentation reflects structure of the generator tree
- Each line encodes some information. Some of the information is atemporal, e.g.
- `* value#N = f(args)` // atemporal abstract value
- Some of the information is temporal, e.g.
- `_$N := op-type<op-data>(args)[metadata]` // generator entry that defines a temporal value
- `op-data` contains various essential data
- `args` tends to be a projection of `data` capturing all values that must be visited, but it's not consistent
- `metadata` is information that helps the visitor compute minimal reachability, but it's not semantically relevant.
- `op-type<op-data>(args)[metadata]` // generator entry that does not define a temporal value
- `if ... then ... else`.
```js
(function() {
let obj = {};
obj.p = obj;
global.result = obj;
})();
```
=>
```
(entry point): "main(#0)"
* object#14 = ObjectValue(properties [p], $Prototype @"Object.prototype")
* object#14.p = PropertyBinding(descriptor PropertyDescriptor(writable, enumerable, configurable, value object#14))
GLOBAL_ASSIGNMENT(object#14, "result")
```
Things get slightly ugly with functions.
```js
function f() { }
```
=>
```
(entry point): "main(#0)"
* declEnv#1 = DeclarativeEnvironmentRecord()
* globEnv#2 = GlobalEnvironmentRecord($DeclarativeRecord declEnv#1, $ObjectRecord declEnv#1, $VarNames [f], $GlobalThisValue global)
* lexEnv#0 = LexicalEnvironment(destroyed, environment record globEnv#2)
* func#13 = ECMAScriptSourceFunctionValue($ConstructorKind base, $ThisMode global, $FunctionKind normal, $FormalParameters 0, $Environment lexEnv#0, properties [arguments, length, caller, prototype, name], $Prototype @"Function.prototype")
* func#13.arguments = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value undefined))
* func#13.length = PropertyBinding(descriptor PropertyDescriptor(configurable, value 0))
* func#13.caller = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value undefined))
* object#14 = ObjectValue(properties [constructor], $Prototype @"Object.prototype")
* object#14.constructor = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value func#13))
* func#13.prototype = PropertyBinding(descriptor PropertyDescriptor(writable, value object#14))
* func#13.name = PropertyBinding(descriptor PropertyDescriptor(configurable, value "f"))
GLOBAL_ASSIGNMENT(func#13, "f")
```
Things get really ugly with optimized functions, but that's what it is right now:
```js
function f() { return 2 + 5; }
__optimize(f);
```
=>
```
(entry point): "main(#0)"
* declEnv#1 = DeclarativeEnvironmentRecord()
* globEnv#2 = GlobalEnvironmentRecord($DeclarativeRecord declEnv#1, $ObjectRecord declEnv#1, $VarNames [f], $GlobalThisValue global)
* lexEnv#0 = LexicalEnvironment(destroyed, environment record globEnv#2)
* func#13 = ECMAScriptSourceFunctionValue($ConstructorKind base, $ThisMode global, $FunctionKind normal, $FormalParameters 0, $Environment lexEnv#0, properties [arguments, length, caller, prototype, name], $Prototype @"Function.prototype")
* func#13.arguments = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value undefined))
* func#13.length = PropertyBinding(descriptor PropertyDescriptor(configurable, value 0))
* func#13.caller = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value undefined))
* object#15 = ObjectValue(properties [constructor], $Prototype @"Object.prototype")
* object#15.constructor = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value func#13))
* func#13.prototype = PropertyBinding(descriptor PropertyDescriptor(writable, value object#15))
* func#13.name = PropertyBinding(descriptor PropertyDescriptor(configurable, value "f"))
GLOBAL_ASSIGNMENT(func#13, "f")
=== optimized function func#13
(entry point): "AdditionalFunctionEffects(#12)"
RETURN(7)
* object#16 = ArgumentsExotic(properties [length, callee], symbols [@"Symbol.iterator"], $Prototype @"Object.prototype")
* object#16.length = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value 0))
* declEnv#1 = DeclarativeEnvironmentRecord()
* globEnv#2 = GlobalEnvironmentRecord($DeclarativeRecord declEnv#1, $ObjectRecord declEnv#1, $VarNames [f], $GlobalThisValue global)
* lexEnv#0 = LexicalEnvironment(destroyed, environment record globEnv#2)
* func#13 = ECMAScriptSourceFunctionValue($ConstructorKind base, $ThisMode global, $FunctionKind normal, $FormalParameters 0, $Environment lexEnv#0, properties [arguments, length, caller, prototype, name], $Prototype @"Function.prototype")
* func#13.arguments = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value undefined))
* func#13.length = PropertyBinding(descriptor PropertyDescriptor(configurable, value 0))
* func#13.caller = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value undefined))
* object#15 = ObjectValue(properties [constructor], $Prototype @"Object.prototype")
* object#15.constructor = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value func#13))
* func#13.prototype = PropertyBinding(descriptor PropertyDescriptor(writable, value object#15))
* func#13.name = PropertyBinding(descriptor PropertyDescriptor(configurable, value "f"))
* object#16.callee = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value func#13))
* object#16.@"Symbol.iterator" = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value @"Array.prototype.values"))
* object#16.$Prototype = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value @"Object.prototype"))
* object#16.$Extensible = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value true))
* object#16._isPartial = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#16._isLeaked = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#16._isSimple = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#16._simplicityIsTransitive = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#16._isFinal = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#17 = ObjectValue($Prototype null)
* object#17.$Prototype = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value null))
* object#17.$Extensible = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value true))
* object#17._isPartial = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#17._isLeaked = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#17._isSimple = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#17._simplicityIsTransitive = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#17._isFinal = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#18 = ObjectValue(properties [next], $Prototype @"([][Symbol.iterator]().__proto__.__proto__)")
* func#19 = NativeFunctionValue(properties [length, name], $Prototype @"Function.prototype")
* func#19.length = PropertyBinding(descriptor PropertyDescriptor(configurable, value 0))
* func#19.name = PropertyBinding(descriptor PropertyDescriptor(configurable, value "next"))
* object#18.next = PropertyBinding(descriptor PropertyDescriptor(writable, configurable, value func#19))
* object#18.$Prototype = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value @"([][Symbol.iterator]().__proto__.__proto__)"))
* object#18.$Extensible = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value true))
* object#18._isPartial = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#18._isLeaked = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#18._isSimple = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#18._simplicityIsTransitive = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#18._isFinal = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* object#18.$IteratedList = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(some array))
* func#19.$Prototype = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value @"Function.prototype"))
* func#19.$Extensible = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value true))
* func#19._isPartial = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* func#19._isLeaked = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* func#19._isSimple = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* func#19._simplicityIsTransitive = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
* func#19._isFinal = PropertyBinding(internal slot, descriptor InternalSlotDescriptor(value false))
modified property bindings: [object#16.$Prototype, object#16.$Extensible, object#16._isPartial, object#16._isLeaked, object#16._isSimple, object#16._simplicityIsTransitive, object#16._isFinal, object#17.$Prototype, object#17.$Extensible, object#17._isPartial, object#17._isLeaked, object#17._isSimple, object#17._simplicityIsTransitive, object#17._isFinal, object#16.length, object#16.@"Symbol.iterator", object#16.callee, object#18.$Prototype, object#18.$Extensible, object#18._isPartial, object#18._isLeaked, object#18._isSimple, object#18._simplicityIsTransitive, object#18._isFinal, object#18.$IteratedList, func#19.$Prototype, func#19.$Extensible, func#19._isPartial, func#19._isLeaked, func#19._isSimple, func#19._simplicityIsTransitive, func#19._isFinal, func#19.length, func#19.name, object#18.next]
created objects: [object#16, object#17, object#18, func#19]
result: SimpleNormalCompletion(value 7)
```
There are still a good number of things left to do. In particular:
- further simplify printing to make it more readable (and writable)
- further extend printed format to make it round-trippable
- some details of the current IR is really just an artefact of 2 years of hacking. It is in need of some additional
rounds of refactorings and simplifications.
- Particularly problematic / overly complicated are invariants INVARIANT, FULL_INVARIANT_ABSTRACT, FOR_IN, REACT_SSR_TEMPLATE_LITERAL
- In `TemporalOperationEntry`, there's some duplication going on with `args` and `data`. Consider eliminating `args` and deriving this information when needed from `data`.
- `OperationDescriptorData` could use some structure, or a (sub)type hierarchy.
- There are already dedicated generator entry classes for some operations, and then there is `TemporalOperationEntry` with its `data` dumping ground for everything else. This is all a bit arbitrary and should be unified.
- ...there's much more cruft.
In a way, this PR just provides yet another way of dumping values and generators. Some of the other existing ways should be consolidated or killed.
Added option --ir to test-runner to activate (and test) IR dumping.
Pull Request resolved: https://github.com/facebook/prepack/pull/2493
Differential Revision: D9560883
Pulled By: NTillmann
fbshipit-source-id: 70920c8e1b4139c69329d8f5ab6e6267a35f058b
Summary:
Release note: Speed up simplifier by using an implication cache per path branch
The realm's path conditions is now a class instances and an explicit tree, along with caches for expressions that have already been checked for true/false using Path.implies on the current set of path conditions.
The AbstractValueImplicationCounter is still there as flag, to be renamed later. It is no longer used as a global k-limit, but enables k-limits on the cost of constructing a new set of path conditions. When React is the target, path conditions are not re-specialized. This leads to a very nice performance win for the large React internal test.
A number of tweaks to the simplifier were needed to get tests to pass. Some of these were borrowed from PR #2460.
Pull Request resolved: https://github.com/facebook/prepack/pull/2494
Reviewed By: trueadm
Differential Revision: D9554820
Pulled By: hermanventer
fbshipit-source-id: 5fdc550499975fe11c0c954b9502cd4eeab2bafe
Summary:
Release notes: None
Remove dead _callRequireAndAccelerate function
Remove dead isModuleInitialized function
Remove dead --accelerateUnsupportedRequires option
Deleting dead code around checking uniqueness of previousDependencies
Add global.__eagerlyRequireModuleDependencies helper to effectively disable lazy module initialization in a code block (needed for InstantRender, to avoid having to deal with conditionally defined modules when prepacking nested callbacks)
Pull Request resolved: https://github.com/facebook/prepack/pull/2482
Differential Revision: D9492955
Pulled By: NTillmann
fbshipit-source-id: 91fe30168ffa848502982e042772462530b982a5
Summary:
Release note: Rewrote the joining logic to always do a full join at every join point
Closes#2151#2222#2279
I've spent a lot of time in the last few months trying to sort out problems that arise from effects being applied too many or too few times. Fixing these feel a bit like playing wack a mole and in the end no fix goes unpunished.
Stepping back a bit from the fray, it seems to me that the root cause of all this pain is the fact that joins of different kinds of completions get delayed.
Before we had path conditions and the simplifier this seemed like a rather good thing since exceptional paths did not contribute values to the normal paths and we thus had fewer abstract values to deal with and fewer places where Prepack would grind to a halt.
In the current state of things, however, it seems perfectly possible to join in all branches at every join point. I've had to decrease some limits, in particular the number of times we go around a loop with conditional exits. I've also had to make the test runner impose a limit on how many times the simplifier can invoke Path.implies.
Nevertheless, the tests seem to pass and hopefully this will also fix quite a lot of bugs that have been unresolved for many months already.
Pull Request resolved: https://github.com/facebook/prepack/pull/2402
Differential Revision: D9236263
Pulled By: hermanventer
fbshipit-source-id: 92a25b591591297afeba536429226c5a0291f451
Summary:
Release notes: adds a `arrayNestedOptimizedFunctionsEnabled` flag to enable nested optimized functions derived from Array.prototype methods (like `map`) and Array.from
This PR puts the existing (unstable) work for Array prototype methods behind a flag. The flag is enabled by default in React and serializer tests.
Pull Request resolved: https://github.com/facebook/prepack/pull/2404
Differential Revision: D9272747
Pulled By: trueadm
fbshipit-source-id: d7e53656a12cd6cff680a9ef0e2580a93d56e34e
Summary:
Resolves#2186. Embeds the value `__empty` in array and object literals for properties that are conditionally set, instead of handling such values via assignments and deletes. Raises an exception if there is a cycle in object or array creation (InstantRender does not support these).
Pull Request resolved: https://github.com/facebook/prepack/pull/2364
Differential Revision: D9169335
Pulled By: sb98052
fbshipit-source-id: f83d85677b30f10f3c548349e93ce792fc6c1ca0
Summary:
Release notes: Reduce memory usage of running Prepack by 3% in some scenarios
This fixes#2365.
This is realized via a stateful SourceFileCollection.
On a small internal benchmark, this saved around 3% of total node memory usage
(the parsed AST that we keep around uses an order of magnitude more memory
than the original source file and source map,
11MB of source files + source map vs 88MB of parsed AST).
Pull Request resolved: https://github.com/facebook/prepack/pull/2377
Differential Revision: D9150888
Pulled By: NTillmann
fbshipit-source-id: b38a8176c4f1e4633366bd48b7d396aec023e7c3
Summary:
Release notes: Fixing source map support with multiple source files
This fixes issue #2353: We now track multiple --srcmapIn arguments,
and match which one applies by comparing the basenames:
The convention is that sourcemaps share the same basename with an appended .map.
Pull Request resolved: https://github.com/facebook/prepack/pull/2362
Differential Revision: D9138802
Pulled By: NTillmann
fbshipit-source-id: d359ccd372b7d87445ecc1a2bdb509ba158e0200
Summary:
Release notes: none
This PR removes the `makeNotPartial` method and all its call-sites in favour of explicitly telling execution paths that we want own properties even if partial. This stops the mutation of objects where we set the partial boolean flag, which actually is a very common cause of side-effects in pure functions. Furthermore, the whole `makeNotPartial` before and after was potentially error-prone if the `try/catch` was omitted by mistake and it was generally a code smell.
I also made it test-runner log the failing test name upon error, which helps with debugging.
Pull Request resolved: https://github.com/facebook/prepack/pull/2357
Differential Revision: D9137606
Pulled By: trueadm
fbshipit-source-id: c0c59615f7a7d46836d26a3b2aecfb89274933d8
Summary:
Release Notes: None
Fixes#2266.
Looking at the `__optimize` code, what we want is for every call to `__optimize` to optimize that function, even in conditonal contexts. In this case, it makes no sense to store the functions to optimize in a special global because we don't care about the applying/reverting of effects. It simplifies the code significantly to be able to store these values in the realm instead.
The only case where this would matter is if effects containing an __optimize call get created then discarded (I am unsure if there are any legitimate cases of this in Prepack at the moment, but I created a CompilerDiagnostic just in case).
Pull Request resolved: https://github.com/facebook/prepack/pull/2303
Differential Revision: D9134968
Pulled By: cblappert
fbshipit-source-id: 040f7ccc24f928e6b8daa068521d3848caffc4d2
Summary:
Release notes: None
- Remove some dead code from test runner.
- Simplify test runner by removing option that's only used once for no good reason.
Pull Request resolved: https://github.com/facebook/prepack/pull/2350
Differential Revision: D9125266
Pulled By: NTillmann
fbshipit-source-id: 04eb9c17d8a3e7b2f93c246652691759e8a797c8
Summary:
I extended the `--serializer` command line argument I added in #2290 to now support `--serializer abstract-scalar`. What this mode does is it converts all boolean, string, number, and symbol literals into abstract values. I did not choose to extend this logic to object and array literals just yet since scalars alone showed some interesting results.
What I really want here is a review of the results.
Full suite execution results are real bad. **18%** pass rate. I dug a bit into why.
```
=== RESULTS ===
Passes: 3356 / 17780 (18%)
ES5 passes: 2276 / 12045 (18%)
ES6 passes: 1080 / 5735 (18%)
Skipped: 13375
Timeouts: 28
```
I was mostly interested in the runtime failures we see since that means Prepack is serializing invalid code. However, I found ~14k failures in the Prepack stage (more on this in a bit) and ~3k failures in the runtime stage. This means ~80% of tests _fail to compile_ with this abstract transformation applied.
Why are these tests failing? I took the first 4 items of the stack traces from errors thrown in the Prepack stage, sorted, and ranked them. [Here’s the result.](https://gist.github.com/calebmer/29e27613325fd99fa04be7ab4a9641c0) The top 5 with thousands of hits are:
```
7538 of:
at AbstractValue.throwIfNotConcrete (/Users/calebmer/prepack/src/values/AbstractValue.js:536:11)
at ToImplementation.ToStringPartial (/Users/calebmer/prepack/src/methods/to.js:717:69)
at NativeFunctionValue._index.NativeFunctionValue [as callback] (/Users/calebmer/prepack/src/intrinsics/ecma262/String.js:34:37)
at NativeFunctionValue.callCallback (/Users/calebmer/prepack/src/values/NativeFunctionValue.js:121:12)
4595 of:
at AbstractValue.throwIfNotConcrete (/Users/calebmer/prepack/src/values/AbstractValue.js:536:11)
at NativeFunctionValue.func.defineNativeMethod [as callback] (/Users/calebmer/prepack/src/intrinsics/ecma262/Object.js:328:41)
at NativeFunctionValue.callCallback (/Users/calebmer/prepack/src/values/NativeFunctionValue.js:121:12)
at functionCall (/Users/calebmer/prepack/src/methods/call.js:308:26)
1454 of:
at AbstractValue.throwIfNotConcrete (/Users/calebmer/prepack/src/values/AbstractValue.js:536:11)
at NativeFunctionValue.func.defineNativeMethod [as callback] (/Users/calebmer/prepack/src/intrinsics/ecma262/Object.js:364:41)
at NativeFunctionValue.callCallback (/Users/calebmer/prepack/src/values/NativeFunctionValue.js:121:12)
at functionCall (/Users/calebmer/prepack/src/methods/call.js:308:26)
1351 of:
at invariant (/Users/calebmer/prepack/src/invariant.js:18:15)
at EvalPropertyNamePartial (/Users/calebmer/prepack/src/evaluators/ObjectExpression.js:59:7)
at _default (/Users/calebmer/prepack/src/evaluators/ObjectExpression.js:80:21)
at LexicalEnvironment.evaluateAbstract (/Users/calebmer/prepack/src/environment.js:1368:20)
1053 of:
at AbstractValue.throwIfNotConcrete (/Users/calebmer/prepack/src/values/AbstractValue.js:536:11)
at NativeFunctionValue.obj.defineNativeMethod [as callback] (/Users/calebmer/prepack/src/intrinsics/ecma262/ObjectPrototype.js:35:39)
at NativeFunctionValue.callCallback (/Users/calebmer/prepack/src/values/NativeFunctionValue.js:121:12)
at functionCall (/Users/calebmer/prepack/src/methods/call.js:308:26)
```
This means there may be some low hanging fruit.
Here are my questions for you.
- Did you expect results like this?
- What is our ideal test262 pass rate with this transformation applied?
- What happens to React Compiler or other projects when these errors are thrown? (As I understand it, we bail out and don’t optimize the code, but do optimize the code around it.)
- Do you think my methodology is flawed?
It’s also possible that something in my methodology is wrong, but I didn’t spend much time investigating these failures as I spent investigating the failures I found in #2290.
My goal with this test suite is to build an understanding of what “correctness” for the React Compiler against all JavaScript code looks like. (Not just the few bundles we’ve selected to look at.) I don’t think these results suggest that we only safely compile 18% of the language, but it’s a data point. I’ll be looking into fixing a selection of these issues to better understand their nature or if I need to change methodologies.
Pull Request resolved: https://github.com/facebook/prepack/pull/2297
Differential Revision: D9120572
Pulled By: calebmer
fbshipit-source-id: b394f1e8da034c9985366010e3e63fd55fd94168
Summary:
Release Notes: None
The original issue here was that `nested` is defined inside of `fn2` which is a non-optimized function called by `fn` (an optimized function). That caused Prepack to not detect that `nested` was nested in `fn2`.
The fix is to use `CreatedObjects` to test for nesting instead of the environment lookup. The environment lookup fails because `nested` is evaluated with `fn2`'s effects applied but _not in `fn2`'s environment_.
This PR also adds a command I use frequently to test a single failing `test-runner` test as well as a way to skip lint because some tests can't pass lint.
Addresses the first test case of #2337.
Pull Request resolved: https://github.com/facebook/prepack/pull/2339
Differential Revision: D9074916
Pulled By: cblappert
fbshipit-source-id: 720003b965d9a9a6842d512ea41cd6402361342e
Summary:
Release notes: none
This cleans up generator.js in the following ways:
- pulls our NameGenerator and PreludeGenerator into it's own module
- takes out the Babel coupling with Generator and puts the serialization logic into the serializer instead
This is of many PRs to help clean up the serialization/generator logic.
Pull Request resolved: https://github.com/facebook/prepack/pull/2306
Differential Revision: D8961658
Pulled By: trueadm
fbshipit-source-id: 4aa3f34c2df1b26eef5719d1a49a5985b9a0e7b3
Summary:
I want to better understand the bugs we have in abstract evaluation. One of my ideas to do this is to replace all the literals in test262 tests with abstract values and see what our test coverage is. The first step to do this is to serialize the test262 sources after running them in Prepack and checking if their generated JavaScript runs correctly. This PR does that by adding a `--serializer` flag.
With my methodology, all test262 harnesses are serialized every time since they are in the global scope. This unfortunately seems to be unavoidable since doing otherwise would change program semantics and break the tests.
Running `time yarn test-test262` takes about 2 minutes and has a 98% pass rate. Running `time yarn test-test262 --serializer` takes about 5 minutes and has a 95% pass rate. [Here’s a diff of the two results.](https://gist.github.com/calebmer/1c9fe396b63ba055458c599c2be18a58)
[Here is a selection of some of the bugs](https://gist.github.com/calebmer/1ac381096a4aa7be1fc7dc2163276ab4) in the serializer caught by running test262 with the Prepack serializer. I might open issues for these, but they can be a bit pedantic. I might fix some of them depending on how important they are to the React code we want to compile.
Notably we have:
- Lots of invariants being triggered. I particularly saw [a lot of this invariant](7d355ef4c5/src/serializer/ResidualHeapVisitor.js (L531)). (An example of this is [`07.md`](https://gist.github.com/calebmer/1ac381096a4aa7be1fc7dc2163276ab4#file-07-md).)
- Lots of values that are visited, but not serialized. (An example of this is [`01.md`](https://gist.github.com/calebmer/1ac381096a4aa7be1fc7dc2163276ab4#file-01-md).)
- Some incorrect outputs only caught by executing code. Could not be caught by static analysis. Particularly around class serialization.
I’ll build off this PR to get coverage for the abstract evaluator next, but I found these results interesting on their own so decided to do this in two PRs.
Pull Request resolved: https://github.com/facebook/prepack/pull/2290
Differential Revision: D8908479
Pulled By: calebmer
fbshipit-source-id: aa57d47611fbd92af33e4647fed7bf7990fb6de1
Summary:
This change allows modeling of optimized function arguments in different way than regular environment modeling. The main point of this modeling is being able to determine what getter should be used at this particular GraphQL property access at compile time. To achieve this Prepack should know the shape of arguments used for optimizing functions and should be able to infer the shape of value when it is needed.
Shape information is attached to AbstractValue (to pass it around) and on every property access a new AbstractValue is returned with shape information specific to this property.
As an output of this process, every member access to modeled values is replaced by function call like prop_string(obj, "key").
Structure of the model can be found in `ShapeInformation.js`.
Pull Request resolved: https://github.com/facebook/prepack/pull/2215
Reviewed By: NTillmann
Differential Revision: D8874743
Pulled By: hotsnr
fbshipit-source-id: 9e1b2254ef54986229be7d1195c1586b95d9a4be
Summary:
Release note: none
The main part of this PR is to ensure that completions and the effects they complete are always 1-1.
Along the way some bugs got fixed#2241.
This also includes a little side project: limiting the console spew when doing "yarn test-serializer --fast", which is something I do all the time and I'm much happier this way.
Pull Request resolved: https://github.com/facebook/prepack/pull/2258
Differential Revision: D8864448
Pulled By: hermanventer
fbshipit-source-id: a7f257a7e07211ecd6069b84330aa3305609c5d2
Summary:
Since I'm adding a new experiment I figured I'd delete an equivalent sized one.
Last year I added an option that runs the Prepack program by invoking Node.js JS runtime which lets us prepack the whole module system and initialization. It's essentially a packager with perfect Node.js module resolution semantics. It did this by modeling Node's native environment as Prepack bindings.
This PR removes that whole option.
There's a few reasons why I don't think that worked out as a good idea.
- It's not solving a real need. It is hard to keep different module systems in tact. There is always something in the ecosystem that breaks down and using the canonical one solves that. However, in practice, if there is a need for bundling the ecosystem itself adapts to the toolchain. So it's not actually that hard to bundle up a CLI even with Webpack, even if it's strictly not 100% compatible, by tweaking a few downstream depenencies.
- Running the resulting bundle is tricky. The resulting bundle includes the JS parts of Node. This overlaps with what Node.js adds at runtime so it runs it twice. The ideal is actually to build a custom distribution of Node.js but this is generally overkill for what people want.
- Bindings change a lot. While Node.js's API notoriously doesn't change much. The internals do change a lot. By picking the API boundary in the middle of the internals of Node.js, it risks changing with any version. While technically observable changes, nobody else relies on these details. If this option was worth its weight, someone could probably maintain it but so far that has not been the case so we had to disable this option in CI to upgrade Node.
However, going forward I think there are alternative approaches we can explore.
- First class module system. This is something we really need at some point. A first class module system would be able to load Node.js module files from disk and package them up while excluding others. It doesn't have to be literally Node.js's module system. Close enough is ok. Especially as standards compliant ECMAScript modules get more popular. This lets us target compiling output that runs after Node's initialization.
- By introducing havocing and membranes in the boundaries, it becomes possible to initialize Node.js modules without actually knowing the internal of the boundaries.
- We've started optimizing residual functions which is much more interesting. However, this requires that code puts some constraints on how it works with its environment. It's not designed to be fully backwards compatible. That's probably a good thing but that also means that we can put constraints on the modules being Prepacked.
This removes the ability to prepack Prepack itself which is unfortunate but already wasn't being tested. To speed up Prepack itself, the [LLVM backend](https://github.com/facebook/prepack/pull/2264) seems much more useful if it can ever work on Prepack itself.
Pull Request resolved: https://github.com/facebook/prepack/pull/2267
Differential Revision: D8863788
Pulled By: sebmarkbage
fbshipit-source-id: d777ec9a95c8523b3386cfad553d9f691ec59074
Summary:
Release note: fix test262 to fail CircleCI test if not enough tests pass
The check for the number of tests that pass returned 1 to its caller, who just ignored it and then returned 0. Instead of that, now just call process.exit(1) when the check fails.
Also updated the expected number of ES6 tests that pass. It appears that updating Babel had a positive effect on those. Possibly it also causes one more ES5 test to fail when running locally (but not on Circle). That might be because of an ES5 test that now times out and an ES6 test that now does not. I have not investigated this as it seems of little importance right now.
Pull Request resolved: https://github.com/facebook/prepack/pull/2263
Reviewed By: trueadm
Differential Revision: D8859210
Pulled By: hermanventer
fbshipit-source-id: 724dcde05927cc914f6f9517f14dc230b8b0ad2e
Summary:
Release notes: upgrades Prepack to use Babel 7.0.0-beta.53
This is a big PR that updates all of Prepack to Babel 7. Babylon is now `babel/parser` and pretty much all of the the previous Babel packages are now located in scoped packages. I had to make a bunch of changes around Jest/Flow/Webpack to get this all working. The build times of building Prepack itself seem considerably faster (easily twice as fast locally). I followed most of the Babel 6 -> 7 upgrade guide from the Babel site in terms of changing nodes and type definitions to match the new ones.
Pull Request resolved: https://github.com/facebook/prepack/pull/2256
Differential Revision: D8850583
Pulled By: trueadm
fbshipit-source-id: 2d2aaec25c6a1ccd1ec0c08c5e7e2a71f78ac2d8
Summary:
This will fix CI on master. I forgot it's possible to land a change internally without it passing the CI checks.
Pull Request resolved: https://github.com/facebook/prepack/pull/2242
Differential Revision: D8801741
Pulled By: gaearon
fbshipit-source-id: 454fcb29e023a502557b67e0ed12a304c4e5fabc
Summary:
…local state
The logic was already there for the React Compiler's invocations of evaluatePure, this PR just generalizes it a little to be used for optmized functions as well.
Additionally, adds the ability to check for compiler info/warning logs in test-runner + updates test-error-handler.
See #1589
Pull Request resolved: https://github.com/facebook/prepack/pull/2175
Differential Revision: D8786744
Pulled By: cblappert
fbshipit-source-id: 110a4732dd6bd129b4d91047c3c9a24f5249a5e9
Summary:
This will fail CI if we forgot to run `yarn prettier` before committing.
We do the same in React repo. It prevents committing stale files that later cause unexpected changes.
Pull Request resolved: https://github.com/facebook/prepack/pull/2212
Differential Revision: D8784406
Pulled By: gaearon
fbshipit-source-id: ca948b8e088be8886c8ba865f280ba8d72750f69
Summary:
Release note: Added --expectedCounts parameter to test262-runner so that success can depend on the value of the time-out and the version of the test suite that is used.
Pull Request resolved: https://github.com/facebook/prepack/pull/2228
Differential Revision: D8775828
Pulled By: hermanventer
fbshipit-source-id: 284bdb3526467f634f41a151e3995719af751e49
Summary:
Release note: Updated the test262 submodule to latest version
Also updated test262-runner.js to deal with time out errors showing up as diagnostics.
Filter out new tests that depend on errors being generated during the parsing phase. Since we depend on Babel for parsing, such issues are out of scope for us.
Pull Request resolved: https://github.com/facebook/prepack/pull/2216
Differential Revision: D8755005
Pulled By: hermanventer
fbshipit-source-id: 0c929904984d13efccbd3ad1ca125137ca275ef0
Summary:
Release notes: adds React Native mocks to Prepack
This adds React Native mocks to Prepack and a few basic tests to demonstrate inlining of `View` and `Text`.
Closes https://github.com/facebook/prepack/pull/2096
Differential Revision: D8723932
Pulled By: NTillmann
fbshipit-source-id: 38bd265cd8935ebdf30266ec337378b4ea5b09d6
Summary:
Release note: special case expression simplification for Instant Render
When instant render is enabled turn x === undefined || x === null into __cannotBecomeObject(x) so that Instant Render can easily compile it down to a single byte code.
Closes https://github.com/facebook/prepack/pull/2188
Differential Revision: D8716594
Pulled By: hermanventer
fbshipit-source-id: bbcb25462f79852d8deac29bfed62e86745e5589
Summary:
Release notes: none
When the `objectWithoutProperties` mock was originally created, my knowledge of Prepack's internals wasn't as good as it was now. Now that I understand how AbstractObjectValues work, we can safely add the known values in `objectWithoutProperties` to the abstract backing object. The backing object was missing these values before and was an empty empty that was partial. This should give more data and value on our internal bundle, where before the values would be lost unnecessarily.
Closes https://github.com/facebook/prepack/pull/2194
Differential Revision: D8716289
Pulled By: trueadm
fbshipit-source-id: 451065473ea09943831f75c0bc15490e73c8d947
Summary:
Currently we have a single giant file with all tests, and a giant snapshot. This is both slow, and hard to work with and iterate on.
In this PR I will refactor our test setup.
- [x] Split it up into multiple files (gets the test running from 45s to 27s)
- [x] Run Prettier on test files
- [x] Split tests further for better performance
- [x] Make it possible to run one test file
- [x] Fix the issue with double test re-runs in watch mode on changes in the test file
- [x] Refactor error handling
- [x] Run Prettier on fixtures
- [x] Add a fast mode with `yarn test-react-fast <Filename>`
- [x] Fix double reruns on failure
Potential followups:
- [x] Figure out why test interruption broke (need https://github.com/facebook/jest/issues/6599 and https://github.com/facebook/jest/issues/6598 fixed)
- [x] Revisit weird things like `this['React']` assignment with a funny comment in every test
Closes https://github.com/facebook/prepack/pull/2187
Differential Revision: D8713639
Pulled By: gaearon
fbshipit-source-id: 5edbfa4e61610ecafff17c0e5e7f84d44cd51168
Summary:
Release notes: None
Tweaking some behaviors to help with Instant Render automation.
This leaves behind some TODOs in the code, but unblock me for now.
Closes https://github.com/facebook/prepack/pull/2178
Differential Revision: D8689796
Pulled By: NTillmann
fbshipit-source-id: f919a8b6bbfb63513bb9467027676e0ba7d4230e
Summary:
Release notes: none
Fixes nested `evaluatePure` calls, where before they would collide and overwrite one another. Also ensure React reconciler uses side-effect detection on nested optimized closures. This fixes a React test which should have been failing before, but wasn't.
Closes https://github.com/facebook/prepack/pull/2166
Differential Revision: D8681422
Pulled By: trueadm
fbshipit-source-id: 8941812407d1bda5af0343a09210aeff24a36cff
Summary:
Release note: Fix test runner and don't swallow invariant failures
The recent changes to make the serializer test runner deal with promises, caused it to list every test that follows a failing test as also failing.
I've also noticed that invariant failures can get swallowed. And as a result of that some test cases that should have failed with false invariants showed up as passing.
Those failures were recently introduced by PR #2134. It boils down to a single line fix, which is also included with this.
Closes https://github.com/facebook/prepack/pull/2155
Differential Revision: D8629298
Pulled By: hermanventer
fbshipit-source-id: 467f7efdf119ea9d29083d4e41054e583b38a1ff
Summary:
Release notes: none
To make https://github.com/facebook/prepack/pull/2096 easier to review, this pulls out the tidy ups from that PR.
- fix typo on function name `mergeAdjacentJSONTextNodes`
- move ReactDOM mock logic into its own file
Closes https://github.com/facebook/prepack/pull/2163
Differential Revision: D8625051
Pulled By: trueadm
fbshipit-source-id: 5345cc26f5c613b4afbb1c768213fd84caf5ebab
Summary:
Release notes: in pure mode, allow Object.assign to continue evaluation rather than throwing FatalErrors
This PR adds bail-out support of `Object.assign` sources within the `Object.assign` implementation. This allows us to continue evaluation in pure scope without bailing out of the Object.assign entirely (like we currently do) upon a FatalError occurring. Given we have snapshotting in `Object.assign`, we don't need to havoc the sources.
This allows us to inline and evaluate far more within our internal bundles. Fixes https://github.com/facebook/prepack/issues/2064
Closes https://github.com/facebook/prepack/pull/2068
Differential Revision: D8579441
Pulled By: trueadm
fbshipit-source-id: 0e97b54e0c8af63f1e3cd08fca6c7f375ee3c615
