Summary:
Release notes: this might increase code size slighty, if the simplifier isn't fully working, in the rare case of conditional objects. I haven't observed any cases of that happening.
_Reviewing the individual commits might be helpful._
The main motivation of these changes is to prepare to avoid leaking the inner object when AbstractObjectValue is wrapping a template which prevents us from modeling templates without true identity.
To do that we need to stop leaking it to the Receiver since the Receiver can be passed to user space. Currently, we unwrap abstract objects [we really need to stop that since it leaks the receiver](cf8184904e).
It used to be that we special cased simple cases in AbstractObjectValue but since OrdinarySet operations are pretty complex, sometimes it's a simple assignment and sometimes the receiver leaks to a more complex operation.
Additionally, we assume that all these operations in AbstractObjectValue can be expressed as pure operations because they're simple objects. However, that's not the case because in properties.js/get.js we emit generator entries for intrinsic objects, havoced objects, widened properties, etc. These are at the very least unnecessary operations to perform when the AbstractObjectValue is actually representing one of the other branches. However, in the case of havoced objects, we don't really know if it is a simple object so it can actually be observable side-effects.
The simplest way of dealing with all these issues is to simply deal with side-effectful operations. So I do that by evaluating each path for effects and joining the effects using the evaluateWithAbstractConditional helper. This also lets us deal with non-simple abstract objects as a bonus.
This doesn't fully get us all the way to no longer leaking receivers. In a follow up I plan to deal with defIneOwnProperty, getOwnProperty, and a few other operations that also leak the inner object template.
Pull Request resolved: https://github.com/facebook/prepack/pull/2461
Differential Revision: D9510853
Pulled By: sebmarkbage
fbshipit-source-id: 4e03b1fb5b168a68b70120630341056bbf5f0e57
Summary:
Fixes#2458.
This is a new version of my change in #2459. I closed that PR since hermanventer had doubts which I confirmed after discovering my “fix” crashed my React Native internal bundle in a different way demonstrating its incorrectness.
After thinking about how to workaround this issue (since I was blocked on this) and fixing related issue #2466 I think I have a proper fix and I’m ready to defend it.
Note that this PR is based on top of #2466. The changes in [this commit](4896087869) are the ones to look at.
The failing test case is the following. Annotated with some comments to help demonstrate the issue:
```js
if (!global.__evaluatePureFunction) global.__evaluatePureFunction = f => f();
const result = global.__evaluatePureFunction(() => {
let x, y, z;
function f() {
const getNumber = global.__abstract ? global.__abstract("function", "(() => 1)") : () => 1;
const b1 = global.__abstract ? global.__abstract("boolean", "true") : true;
const b2 = global.__abstract ? global.__abstract("boolean", "!false") : true;
const b3 = global.__abstract ? global.__abstract("boolean", "!!true") : true;
x = getNumber(); // Modify binding
if (!b1) throw new Error("abrupt"); // Compose JoinedNormalAndAbruptCompletions
y = getNumber(); // Modify binding
if (!b2) throw new Error("abrupt"); // Compose JoinedNormalAndAbruptCompletions
z = getNumber(); // Modify binding
if (!b3) throw new Error("abrupt"); // Compose JoinedNormalAndAbruptCompletions
if (global.__fatal) global.__fatal();
return x + y + z;
}
f();
return x + y + z;
});
global.inspect = () => result;
```
What is happening is that we modify three bindings in `f` before we throw a `FatalError`. `x`, `y`, and `z`. We also have three `JoinedNormalAndAbruptCompletions` since we’re using abstract booleans and throwing if the boolean is false. Our realm when we throw the `FatalError` ends up looking like:
```
Realm {
savedCompletion: JoinedNormalAndAbruptCompletions {
savedEffects: { modifiedBindings: { z } },
composedWith: JoinedNormalAndAbruptCompletions {
savedEffects: { modifiedBindings: { y } },
composedWith: JoinedNormalAndAbruptCompletions {
savedEffects: { modifiedBindings: { x } },
}
}
}
}
```
When we throw the `FatalError` we want to restore all our bindings to a state before we started the `evaluateForEffects()` call so that we can leave a residual call in pure scope. However, the way the code is currently written we _don’t look into `savedCompletion.composedWith`_ even though there may be more bindings there. Instead we only look at `savedCompletion` so we only restore the `z` binding.
My previous fix modified `stopEffectCaptureAndUndoEffects()` to chase `composedWith`, but that broke the contract of `stopEffectCaptureAndUndoEffects()` as hermanventer mentioned. So instead I updated the `finally` block in `evaluateForEffects()` which cleans up modified bindings of a `savedCompletion` (in case we throw in the middle of evaluation) to more explicitly clean up the bindings of the saved completion and all composed completions. This preserves and fixes our effect cleanup behavior without changing the meaning of `stopEffectCaptureAndUndoEffects()`.
hermanventer I know you mentioned you wanted to look at this issue, sorry if I’m stepping on your toes. Since I was blocked I thought a bit more about this issue and figured I was directionally correct, but the specific fix in #2459 was wrong. Also sorry for the PR churn. Thought I wasn’t going to come back to this issue.
Pull Request resolved: https://github.com/facebook/prepack/pull/2467
Differential Revision: D9498191
Pulled By: calebmer
fbshipit-source-id: 6b95df969b2230302d94342a392e9d8cb81329db
Summary:
Fixes#2432. I found this while trying to compile our internal React Native bundle.
Prepack was incorrectly cleaning up lexical scopes in the case of a stack overflow.
This bug is a bit difficult to reproduce. Follow these steps to reproduce:
1. Produce a stack overflow in code Prepack compiles. With a recursive loop for instance.
2. Wrap in `__evaluatePureFunction`. This should catch the stack overflow and emit a residual function call.
3. Add empty functions for `global.require` and `global.__d`. You need to do this so that we don’t crash [in tracer code trying to resolve these variables](c18bd7ee97/src/methods/function.js (L82-L90)) but we instead crash [in `PrepareForOrdinaryCall`](c18bd7ee97/src/methods/function.js (L100)).
I used a `try`/`catch` to fix this which is the obvious solution, but perhaps not the most efficient. Happy to write a slightly uglier solution for performance if we feel like it is necessary.
Pull Request resolved: https://github.com/facebook/prepack/pull/2433
Differential Revision: D9469599
Pulled By: calebmer
fbshipit-source-id: bd120c2e4bd450da3364c722550977d6566ed89a
Summary:
Release notes: None
Remove dead _callRequireAndAccelerate function
Remove dead isModuleInitialized function
Remove dead --accelerateUnsupportedRequires option
Deleting dead code around checking uniqueness of previousDependencies
Add global.__eagerlyRequireModuleDependencies helper to effectively disable lazy module initialization in a code block (needed for InstantRender, to avoid having to deal with conditionally defined modules when prepacking nested callbacks)
Pull Request resolved: https://github.com/facebook/prepack/pull/2482
Differential Revision: D9492955
Pulled By: NTillmann
fbshipit-source-id: 91fe30168ffa848502982e042772462530b982a5
Summary:
Release note: Simplify equality expressions where types are known
Fixes: #2317
If the types or the arguments are known we can simplify equality expressions involving different typed arguments to false in some cases.
Pull Request resolved: https://github.com/facebook/prepack/pull/2481
Differential Revision: D9491269
Pulled By: hermanventer
fbshipit-source-id: 4e5657e23d13fafa7d48adcf01a7626fa63a2328
Summary:
Fixes#2464.
When we bail out of a function call because of a `FatalError` we did not leak the function we called. This means modified bindings were not appropriately leaked. See issue #2464 where the result is `undefined` instead of a reference to the leaked value.
This PR also adds a utility `__fatal` function for throwing a fatal error. Allowing us to more clearly express issues involving `FatalError`s. Used in the test.
Pull Request resolved: https://github.com/facebook/prepack/pull/2466
Differential Revision: D9482612
Pulled By: calebmer
fbshipit-source-id: 792d29818245ad9f631227d923efb56228df21e0
Summary:
Fixes#2468.
We weren’t using the correct `initializerAstNodeName` in two places. Also some free small efficiency wins.
Pull Request resolved: https://github.com/facebook/prepack/pull/2469
Differential Revision: D9481902
Pulled By: calebmer
fbshipit-source-id: 07dd44ae0141a74c50296c61828706da11cd26d2
Summary:
Release notes: reduces loop iteration counter to 6 before bailing out
With our internal bundle, it takes far too long to hit the current limit of 12 with production code (gave up after 2 hours). This now takes a few minutes.
https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two
Pull Request resolved: https://github.com/facebook/prepack/pull/2476
Differential Revision: D9480198
Pulled By: trueadm
fbshipit-source-id: cff1ef3a628dccfb9a5fb20d67169bd4a5174c0e
Summary:
Release note: Make simplification more robust when complexity limits are reached
Fixes#2361
Any particular simplification can fail because Prepack has run out of a global limit for allowable implications (set by the abstractValueImpliesMax option). If Prepack then recovers from from the exception and carries on, it may later do the same simplification and succeed. If this happens in a context where the now known result is not expected, there will be an invariant failure, as for example in the issue above.
The invariant useful for finding bugs when reasoning is precise, so I'm keeping it, but only if no limit was exceeded.
Pull Request resolved: https://github.com/facebook/prepack/pull/2472
Reviewed By: trueadm
Differential Revision: D9479598
Pulled By: hermanventer
fbshipit-source-id: c38c49257df1c1e207e01a24b92171e1125fe7c2
Summary:
After #2425 I ran into _another_ issue in the React Native internal bundle I’m working on involving `React.forwardRef()` where there was some `component instanceof Component.prototype` check where `component` is an abstract `React.forwardRef()` value. This caused a far-reaching undesirable havoc.
There are a lot of components in React Native that use `React.forwardRef()`. Basically all the primitives. `Text`, `Image`, etc.
The React implementation of the `React.forwardRef()` function is also pretty simple to boot. Let’s just return concrete values from `React.forwardRef()` instead of abstract values which cause a lot of problems. This depends on the React implementation not changing, but we’re already teaching Prepack about the React internal implementation.
f9358c51c8/packages/react/src/forwardRef.js (L12-L43)
Pull Request resolved: https://github.com/facebook/prepack/pull/2465
Differential Revision: D9467945
Pulled By: calebmer
fbshipit-source-id: 93aa24a994107a77f2975b9483b59edfc87f64f5
Summary: New alpha version for master.
Reviewed By: trueadm
Differential Revision: D9457284
fbshipit-source-id: 6b67e1095dc9bcd9545e88dcfa9a7d714434a9db
Summary:
What it says on the tin
Pull Request resolved: https://github.com/facebook/prepack/pull/2463
Differential Revision: D9456926
Pulled By: gaearon
fbshipit-source-id: fef96d4e0843662ff90eb4b0b06410f3b3bac83c
Summary:
Fixes#2419#2386#2439#2447#2432#2437#2442
* Fix havoced binding not in optimized function
* Allow arrays with widened numeric properties to update index properties
* Fix nested for statement bailout with nested for-in
* Don’t record modified bindings for immutable bindings when havocing
Reviewed By: trueadm, sebmarkbage
Differential Revision: D9456957
fbshipit-source-id: f266b8cc73012b9c721f0f9eebd48347bf0e37ae
Summary:
Fixes#2442 by not marking immutable bindings as modified. Its very possible that my fix is incorrect. There’s a bunch of related logic for `binding.hasLeaked`/`binding.mutable`/`modifiedBindings` and I certainly didn’t find them all to get a holistic view of what’s happening. From the code I did look at, this fix seems to be safe since there’s no way for a havoced function to change an immutable binding.
I ran into this issue while trying to compile our internal React Native bundle with the React Compiler.
Let me walk through this repro to the invariant.
```js
const havoc = __abstract("function", "(() => {})");
global.result = __evaluatePureFunction(() => {
const b = __abstract("boolean", "true");
if (b) {
var g = function f(i) {
return i === 0 ? 0 : f(i - 1) + 1;
};
havoc(g);
} else {
return;
}
return g(5);
});
```
1. Create a function expression that refers to itself inside a construct that wraps the evaluation in `evaluateForEffects()`. (I used an if-statement.)
2. Havoc the function expression. This will add the function’s reference to itself to `modifiedBindings`. However, the original [function expression skipped (3rd arg)](3eb1e8e8ea/src/evaluators/FunctionExpression.js (L108)) adding its binding to `modifiedBindings`.
3. Outside of `evaluateForEffects()` call the function.
4. Observe the invariant firing:
3eb1e8e8ea/src/environment.js (L339)
According to `modifiedBindings`, we only initialize the function expression _after_ its definition. So if the function expression wants to recursively refer to itself, it can’t since the effects we’ve recorded/applied puts us in a state where the binding is not available.
Pull Request resolved: https://github.com/facebook/prepack/pull/2443
Reviewed By: trueadm
Differential Revision: D9447861
Pulled By: calebmer
fbshipit-source-id: de4460bd184012416e80eb65f86dae7ade9dc34e
Summary:
`typeof React.forwardRef(...)` is `object` but Prepack was returning `function`. This is particularly important since React Native’s `Animated.createAnimatedComponent` crashes with an invariant if `typeof` is `function`. We can’t compile our React Native internal bundle without this fix.
22cf5dc566/Libraries/Animated/src/createAnimatedComponent.js (L20-L25)f9358c51c8/packages/react/src/forwardRef.js (L12-L43)
Small Prepack core change in `src/evaluators/UnaryExpression.js`.
Pull Request resolved: https://github.com/facebook/prepack/pull/2425
Reviewed By: trueadm
Differential Revision: D9447593
Pulled By: calebmer
fbshipit-source-id: de1e9101f368ea52c8c17c31d04a2e4e0a38dd6b
Summary:
Fixes#2437.
The code to handle for-statement bailouts was failing on nested for-in/for-out statements since while they had a variable declaration node in their AST the variable declaration behaves very differently. Notably there must be exactly one declaration, the declaration may not have an initializer, and the declaration may only be replaced by a `LeftHandSideExpression` not a expression or statement.
This PR does two things:
1. Replace the variable declarations in for-in/for-of statements with a `LeftHandSideExpression` which will then perform `DestructuringAssignment` with loop values in the evaluator.
2. Recognize that a `LeftHandSideExpression` in a for-in/for-of expression modifies bindings in the serializer residual function visitor.
Pull Request resolved: https://github.com/facebook/prepack/pull/2438
Reviewed By: trueadm
Differential Revision: D9446786
Pulled By: calebmer
fbshipit-source-id: e7f13a3c722602270a87cf8d933de1283b8121d1
Summary:
Fixes#2432.
`evaluateWithoutEffects` sets the realm generator to `undefined`. However, I found a case in #2432 where a generator is required in `evaluateWithoutEffects`.
What happens is that we build an error stack in `evaluateWithoutEffects`.
3eb1e8e8ea/src/realm.js (L1607-L1612)
As a part of building that error stack we try to get `EventEmitter.name`. However, `EventEmitter` is havoced! So we want to create an abstract temporal value `EventEmitter.name`. But creating a temporal value requires a generator since we need to add a generator entry.
3eb1e8e8ea/src/methods/properties.js (L1361-L1367)
This PR solves the issue by adding a throw-away generator to `evaluateWithoutEffects`. Other strategies might be a bit complicated since `evaluateWithoutEffects` is a very uncommon case. Removing `evaluateWithoutEffects` is also not a great option since in this case we would add a temporal `EventEmitter.name` when we never actually use it.
Pull Request resolved: https://github.com/facebook/prepack/pull/2436
Reviewed By: trueadm
Differential Revision: D9446792
Pulled By: calebmer
fbshipit-source-id: 14b36da1fb39dc9ab2a166ecb02d0f062b2bdffe
Summary:
This test bedeviled me for quite some time because I thought it was testing for aliasing effects - it isn't. This is my fault, since I posted this in my repro. Aliasing effects are still to be dealt with in nested optimized functions. Anyhow, simplifying to prevent somebody else from having this confusion.
Pull Request resolved: https://github.com/facebook/prepack/pull/2453
Reviewed By: trueadm
Differential Revision: D9420733
Pulled By: sb98052
fbshipit-source-id: 755ccd90b1e64afdfe6ec627a3306e8baf5952ea
Summary:
Fixes#2447 by adding a separate codepath for when we encounter arrays with widened numeric properties, otherwise we hit an invariant when trying to deal the `length` property (which always remains abstract for arrays with widened numeric properties).
Pull Request resolved: https://github.com/facebook/prepack/pull/2450
Differential Revision: D9402265
Pulled By: trueadm
fbshipit-source-id: 772634bd0a69af23410e65c63b89cbfcc89d8b5b
Summary:
This PR continues the break up of #2185 into smaller pieces, and makes the use of Leaking, Havocing and Materialization consistent with agreed upon terminology.
*Leaking* transitively adds mutable locations to the environment. Accessing such leaked locations is equivalent to accessing to the environment.
*Havocing* takes on its usual meaning in static analysis: to set an abstract value to "unknown" i.e. topVal.
*Materialization* snapshots locations into runtime assignments for the purpose of outlining functions.
Presently, leaking always leads to havocing, and materialization. But in addition, it also marks a location or an object as "leaked" implying that it may be aliased and/or no longer be "well behaved" (for example, a leaked object may leak any object passed to one of its methods).
Pull Request resolved: https://github.com/facebook/prepack/pull/2444
Differential Revision: D9392730
Pulled By: sb98052
fbshipit-source-id: 0ef987be8a71bb74e2def0526caf203bbfc34d82
Summary:
Fixes#2419 and #2386. I ran into this issue while testing an internal React Native bundle. This invariant assumes that all property bindings emitted in a pure scope are also in an optimized function. This is not true for `__evaluatePureFunction` which the React Compiler wraps around initialization code so that globals outside of the closure are not havoced.
Pull Request resolved: https://github.com/facebook/prepack/pull/2420
Differential Revision: D9363247
Pulled By: calebmer
fbshipit-source-id: 4f1634165b6fe15f95b5b7332432fccacc596821
Summary:
Fixes#2151#2222#2279#2393#2399#2404#2411#2414#2415
Added a fuzz testing tool
Added test cases
Turn crash in JSON.stringify into a diagnostic
Adds a `arrayNestedOptimizedFunctionsEnabled` flag to enable nested optimized functions derived from Array.prototype methods (like `map`) and Array.from
Refactor assignment on partial or possibly deleted property
Rewrote the joining logic to always do a full join at every join point
Removed last remnants of delayUnsupportedRequires
Reviewed By: cblappert
Differential Revision: D9349841
fbshipit-source-id: 74a16dbb015777d59d23fdfde77abbe2489c292a
Summary:
Release notes: None
Did a little bit of prettying up the code in _getTarget,
renaming additional to optimized, and added a TODO #2426
annotation where issue #2426 happens in the code.
Pull Request resolved: https://github.com/facebook/prepack/pull/2427
Differential Revision: D9350285
Pulled By: NTillmann
fbshipit-source-id: 56f1c7ab1a62ca39091d06c91c0700d0cad447ad
Summary: The require method is now __r.
Reviewed By: NTillmann
Differential Revision: D9329676
fbshipit-source-id: 0c3385d593abcf93c5bb3116273b1e128b9e2950
Summary:
Release notes: None
Every native function value needs an intrinsic name,
otherwise the serializer will fail with an invariant violation.
This gives an intrinsic "name" to `ThrowTypeError`
that meets the requirements of the ECMAScript spec.
Pull Request resolved: https://github.com/facebook/prepack/pull/2424
Differential Revision: D9344248
Pulled By: NTillmann
fbshipit-source-id: c0e00652c50a555e7c43009d1394c969fbfff9b1
Summary:
The previous changes on "Do full joins" didn't remove all
uses of the delayUnsupportedRequires feature. This does it.
In particular, this fixes the internal prepack-test.
Reviewed By: hermanventer
Differential Revision: D9287509
fbshipit-source-id: f810f5e08a10cbd14272d776799a2a8f1442d1c8
Summary:
Release note: Rewrote the joining logic to always do a full join at every join point
Closes#2151#2222#2279
I've spent a lot of time in the last few months trying to sort out problems that arise from effects being applied too many or too few times. Fixing these feel a bit like playing wack a mole and in the end no fix goes unpunished.
Stepping back a bit from the fray, it seems to me that the root cause of all this pain is the fact that joins of different kinds of completions get delayed.
Before we had path conditions and the simplifier this seemed like a rather good thing since exceptional paths did not contribute values to the normal paths and we thus had fewer abstract values to deal with and fewer places where Prepack would grind to a halt.
In the current state of things, however, it seems perfectly possible to join in all branches at every join point. I've had to decrease some limits, in particular the number of times we go around a loop with conditional exits. I've also had to make the test runner impose a limit on how many times the simplifier can invoke Path.implies.
Nevertheless, the tests seem to pass and hopefully this will also fix quite a lot of bugs that have been unresolved for many months already.
Pull Request resolved: https://github.com/facebook/prepack/pull/2402
Differential Revision: D9236263
Pulled By: hermanventer
fbshipit-source-id: 92a25b591591297afeba536429226c5a0291f451
Summary:
The goal of this refactoring is to move more of the special case logic around properties as deeply as possible so that when we expand the object model, we don't have to add special cases through all the indirections. This will be clear in a follow up where I try to model the multiple identities.
This reverts the fix in #1183. That fix is no longer needed for its original purpose because those getters are now modeled as pure and the unnecessary accesses gets eliminated by the serializer.
I also remove the `Receiver.$Delete(P)` hack in `OrdinarySet` when a property might have been deleted. I'll need that because this hack doesn't work on objects that only allow weak deletions (such as object set templates). Instead, I model possibly deleted properties in ValidateAndApplyPropertyDescriptor.
It turns out that much of it doesn't matter if it has been deleted or not since you can't delete a non-configurable property.
The React environment calls Set/DefineOwnProperty on partials during initialization. These must read to get the current descriptor. However, a temporal read of the actual value can't be emitted at this point. So I emit a placeholder value that shouldn't be serialized. This is something the environment initialization must already consider. We can't have it read partial values before there is an environment to consider.
Pull Request resolved: https://github.com/facebook/prepack/pull/2389
Differential Revision: D9274895
Pulled By: sebmarkbage
fbshipit-source-id: b8e8aee697121552243f7a67b7c19c62768f4729
Summary:
Release notes: adds a `arrayNestedOptimizedFunctionsEnabled` flag to enable nested optimized functions derived from Array.prototype methods (like `map`) and Array.from
This PR puts the existing (unstable) work for Array prototype methods behind a flag. The flag is enabled by default in React and serializer tests.
Pull Request resolved: https://github.com/facebook/prepack/pull/2404
Differential Revision: D9272747
Pulled By: trueadm
fbshipit-source-id: d7e53656a12cd6cff680a9ef0e2580a93d56e34e
Summary:
Release notes: none
During React reconciliation we wrongly reset `pathConditions`. This was done originally because of a pushing false invariant, that I incorrectly thought was a bug because of React specific code.
However, today, I re-worked the original test code to remove all React logic and I found that the invariant occurs even without React specific logic. This strongly suggests that this is still an existing bug in Prepack and adding `pathConditions` logic, which gets around it, was the wrong strategy and should be reverted.
Pull Request resolved: https://github.com/facebook/prepack/pull/2415
Differential Revision: D9264730
Pulled By: trueadm
fbshipit-source-id: 2d3703e638c894eb0929b9a30a7d74ccbd998ea3
Summary:
Release notes: none
This adds a check in the React utils component function caller to ensure that the `createObjects` are correct at evaluation time.
Pull Request resolved: https://github.com/facebook/prepack/pull/2414
Differential Revision: D9264727
Pulled By: trueadm
fbshipit-source-id: 6435bf1a4ccfaf98278377ad48147a557c1608d4
Summary:
Release Notes: None
Previous PRs fixed these issues, but didn't necessarily add the test case from the issue. This PR adds the exact test cases from the issues.
Closes#2358Closes#2398
Pull Request resolved: https://github.com/facebook/prepack/pull/2410
Differential Revision: D9259118
Pulled By: cblappert
fbshipit-source-id: 3eb347044865a11e2ce5262b2338b7b6e1958c60
Summary:
Release notes: none
Fixes#2393. I created the test as a React test because I couldn't see any way of making a serializer test fail if there was a warning without making many other tests fail or without adding another `// some config` to the test (which I believe we said we would not add anymore as there's already too many). React tests will always fail on side-effect warnings, so this takes better advantage of this.
Pull Request resolved: https://github.com/facebook/prepack/pull/2394
Differential Revision: D9249714
Pulled By: trueadm
fbshipit-source-id: 0e11d13e575eca3b6bd75e437eb083b9294a3aee
Summary:
This PR adds the fuzzer I’ve been working on to the Prepack codebase so other people can contribute, provide feedback, and run it against their changes. The new commands added are:
- `yarn fuzz`: Starts generating tests in the foreground and logs progress. If it finds an error it will try and shrink it before returning the shrunken program to you with the invalid results.
- `yarn fuzz-sample`: See a selection of the programs generated by the fuzzer.
- `yarn fuzz-overnight`: Spin up a worker for each CPU and try to find failing test cases. Your computer will be basically unusable while you run this, so leave it running overnight. Failed test cases will be saved in `fuzzer/overnight.sqlite` so in the morning you can use `sqlite3` to inspect the errors the fuzzer found.
The fuzzer generates programs with an optimized function and executes them twice:
1. In Node.js
2. In Node.js after running Prepack
Then compares the results. If the results are different then the fuzzer will attempt to shrink the program to something easier to debug and return this to you. See [this gist for a sample of generated programs](https://gist.github.com/calebmer/6a727c1f4aa8c08d51940e60d29d336a). Here’s an example of a function you might see:
```js
function f3(a1, a2, a3, a4, a5, a6) {
2;
var x2;
if (0) {
return a1 ? false : a2;
} else {
var x1 = a3;
x2 = x1;
}
var x6;
if (x2) {
var x3;
if (x2) {
x3 = x2;
} else {
x3 = a4;
}
var x4 = x3;
x6 = x4;
} else {
var x5;
if (a5) {
x5 = x2;
} else {
x5 = a6;
}
x6 = f2(x5);
}
return x6;
}
```
So far I’ve reported four bugs reduced from test cases found by this version of the fuzzer. I’ve reported a couple more from old fuzzers I used, but these four from the current version. The shrinking process is not that good and it takes a while as the generated program can get large, so you’ll usually have to do some manual shrinking to get good bug reports. I only ran `yarn fuzz-overnight` for about an hour. It found 28 failures and I reduced those down to these 4.
- #2354
- #2355
- #2361
- #2363
I expect I’ll find more bugs as these get fixed and I add more JavaScript features to the fuzzer. The features I currently have are:
- Scalar primitives (number, string, boolean, null/undefined)
- Functions
- Conditional expressions
- If statements
- Variable declarations
Not too many features, but enough to catch a handful of bugs.
> **Note:** If this PR is too much to review, I’ve created [`calebmer/prepack-fuzzer`](https://github.com/calebmer/prepack-fuzzer) in which my work is broken up into commits as I made them. I then copied these files over to the Prepack repo.
The fuzzer in this PR is a rewrite from the [fuzzer I started with](https://gist.github.com/calebmer/75dd75ebe556681d3a628e75eaffc403). The main lessons I learned from that one are that I should start with a general JS fuzzer instead of a React fuzzer (since adding JS features after the fact to fuzzer designed for React is difficult) and that all nested structures need to be generated with one recursive generator from the generator library I’m using.
To generate programs I use [`leebyron/testcheck`](https://github.com/leebyron/testcheck-js) which is actually a JS compiled version of the official Clojure library [`clojure/test.check`](https://github.com/clojure/test.check). `testcheck` is designed for generative property testing at a much smaller scale then program fuzzing. So I am abusing the library a bit to use it as a fuzzer. My reasoning is that I wanted to write the fuzzer in JS (to have access to the Babel AST) and I didn’t want to write my own case generating framework. If we outgrow `testcheck` then we can keep the DSL, but rewrite the generation/shrinking logic. Although its been working fine for me so far. (Yet I am using a forked version which simply uses some unpublished features in the `testcheck` repo.)
The generator code in `fuzzer/src/gen.js` may look odd to you. It uses immutable.js and a **state monad** implemented with a JS generator so the code looks imperative. I need state since generating various program components depends on things like “what variables are declared at a given point in time,” but because I’m limited to only using a single recursive generator (based on performance lessons I learned from my first fuzzer) I can’t pass around state at the generator level and must instead maintain state at the result level. At first I tried hacking together some imperative state, but when shrinking programs `testcheck` replays some generators to get new programs. So what do you do when you need a stateful process that needs to be replayed? You use a monad.
I could try to fix the bugs I found, but I’d like to find more bugs. I need to add back support for React components and I need to add more language features.
_Which JS language features are the most interesting to fuzz?_
I ranked all the kinds of AST nodes in our internal React bundle [and got this](https://gist.github.com/calebmer/be5e2bad4b12af683522096544fc9568). I’ll be starting with that, but the Prepack team has a better intuition around what’s good for fuzzing. I know there’s been a discussion around temporals recently that I haven’t really been following. What would be good ways to trigger this behavior?
Pull Request resolved: https://github.com/facebook/prepack/pull/2374
Differential Revision: D9180836
Pulled By: calebmer
fbshipit-source-id: 59d3fb59ecc1026a865672e2833f7482ed72139a
Summary:
Release Notes: None
Previously, `tryGetOptimizedFunctionRoot` would only get the root successfully if the root was one of the optimized functions or the parent of one of the residual functions. Now we look at the optimized functions as well as any of their parents. Added 2 test cases.
Fixes#2399
Pull Request resolved: https://github.com/facebook/prepack/pull/2401
Differential Revision: D9242153
Pulled By: cblappert
fbshipit-source-id: 9bdffce23d5dd19d9ce5a6b09e1ee9647ca6e5cb
Summary:
Weekly release v0.2.47:
- Many bug fixes
- Progress in support for nested optimized functions
- Fixing source map support
- Improved support for abstract behavior in switch statements
- Reduced memory usage of Prepack
Reviewed By: cblappert
Differential Revision: D9230689
fbshipit-source-id: 2094183fe4183089dfa584b865c9997147475e45
