Run exports through the same identifier generation as imports to ensure
that everything gets a unique identifier and then just make sure all the
appropriate wires are hooked up when dealing with exports and imports.
Closes#1496
Currently the import object constructed for the `--target web` output
only ever includes the current module as an one of the modules included.
With `wasm-bindgen`'s optimization to import directly from modules,
however, it's possible to have more modules imported from in the
generated wasm file. This commit ensures that the imports are hooked up
in the `--target web` es6 emulation mode, ensuring there aren't
extraneous errors about import objects.
We don't support variadic args in front, but, luckily for us, `new Function` accepts comma-separated args as a single string as well (see updated JSON test for an example).
- `JsString::from_code_point` - allows to create JS strings using slice of codes in WASM memory.
- `JsString::from_char_code` - same as above, but also uses updated signature with `u16` instead of `u32` (partially helps with #1460 at least for the new binding).
Now that functions and strings work properly with custom typechecks, these custom methods seem obsolete, so I'd recommend to use standard dyn_ref instead.
Particularly useful in our tests, where we don't have the regular console
logging with post-facto object inspection, and instead need to provide all this
info up front.
Constructing boxed primitives was deprecated in #1447.
Some tests have been still using these methods, so this PR either updates them to use newly added {primitive}::from implementations or adds `#[allow(deprecated)]` where necessary.
This is intended to handle #1458 and #822. These issues stem from
behavior where:
wasm-pack test --node
will actually run both Node.js and browser tests! Two new env vars are
read here, `WASM_BINDGEN_TEST_ONLY_{NODE,WEB}`, which control which
tests are executed by `wasm-bindgen-test-runner`. The intention is that
these will be set by `wasm-pack` to configure which tests are run, and
if test suites are skipped due to the env vars we'll print an
informational message indicating how they can be run.
Closes#822Closes#1458
This commit enables `[NoInterfaceObject]` annotated interfaces in
`web-sys`. The `NoInterfaceObject` attribute means that there's not
actually a JS class for the object, but all of its properties and such
can still be accessed structually and invoked. This should help provide
more bindings for some more common types on the web!
Note that this builds on recent features to ensure that `dyn_into` and
friends always fail for `NoInterfaceObject` objects because they don't
actually have a class.
Closes#893Closes#1257Closes#1315
This commit switches to executing `rustfmt` by default on
`web-sys`-generated bindings. This improves situations like "view
source" in Rustdoc as well as the IDE interactive debugging experience.
This was initially disabled by default because `rustfmt` took so long to
execute, but nowadays `web-sys` is by default much smaller so there's
much less need to avoid running `rustfmt` in fear of it taking too
long.
Closes#1457
Treats any object of shape `{ next: function }` as an iterator via new `is_type_of` method. This is consistent with JavaScript iteration protocol which we already respect.
Also fixes a minor issue that `is_function` was unnecessarily called twice (once explicitly and once as part of `dyn_into` which now does the same check).
This commit adds `#[derive(PartialEq, Eq)]` to many types throughout
`js-sys`. These types are basically all based on `Object`, which means
that `Object.is` can be used for `PartialEq` and the `Eq` requirements
are upheld.
The macro has also been updated to internally store the deref target
instead of unconditionally storing `JsValue`, allowing `#[derive]` to
work a bit better in these situations.
* Ensure `PartialEq` is implemented from these types to native Rust types
* Implement `From` between these type and native Rust types
* Deprecated `Number::new` and `Boolean::new` to discourage use of the
object forms, recommending the `from` constructors instead.
Closes#1446
The last write accidentally wasn't accounted for in the returned length
of the string and we unfortunately don't have any test coverage of
`encodeInto` since it requires Firefox nightly right now (and doesn't
work in Node yet).
Closes#1436
This commit adds an intrinsics to the `wasm_bindgen` crate which
accesses the `WebAssembly.Table` which is the function table of the
module. Eventually the thinking is that a module would import its own
function table via native wasm functionality (via `anyref` and such),
but until that's implemented let's add a binding for it ourselves!
Closes#1427
This commit aims to address #1348 via a number of strategies:
* Documentation is updated to warn about UTF-16 vs UTF-8 problems
between JS and Rust. Notably documenting that `as_string` and handling
of arguments is lossy when there are lone surrogates.
* A `JsString::is_valid_utf16` method was added to test whether
`as_string` is lossless or not.
The intention is that most default behavior of `wasm-bindgen` will
remain, but where necessary bindings will use `JsString` instead of
`str`/`String` and will manually check for `is_valid_utf16` as
necessary. It's also hypothesized that this is relatively rare and not
too performance critical, so an optimized intrinsic for `is_valid_utf16`
is not yet provided.
Closes#1348
Because of some incorrect use of `js.push_str(..)`, we could sometimes emit code
before the ES modules imports, which is syntactically invalid:
const __exports = {};
import { Thing } from '...'; // Syntax error!
This has been fixed by making sure that the correct `imports` or `imports_post`
string is built up. We now also assert that the `js` string is empty at the
location where we add imports if we're using ES modules.
This commit fixes the `init` function when passed a
`WebAssembly.Module`. Upon closer reading of the [spec] we see there's
two possible return values from `WebAssembly.instantiate`. If passed a
`Module`, it will return only the `Instance`. If passed a buffer source,
though, it'll return an object with the module/instance.
The fix here is to check the result value is an `Instance`, and if so
assume the input must have been a module so it's paired up in the
output.
Closes#1418
[spec]: http://webassembly.github.io/spec/js-api/index.html#webassembly-namespace
Instead of doubling the size on each iteration, use precise upper limit (3 * JS length) if the string turned out not to be ASCII-only. This results in maximum of 1 reallocation instead of O(log N).
Some dummy examples of what this would change:
- 1000 of ASCII chars: no change, allocates 1000 bytes and bails out.
- 1000 ASCII chars + 1 '😃': before allocated 1000 bytes and reallocated to 2000; now allocates 1000 bytes and reallocates to 1006.
- 1000 of '😃' chars: before allocated 1000 bytes, reallocated to 2000, finally reallocated again to 4000; now allocates 1000 bytes and reallocates to 4000 right away.
Related issue: #1313
All numbers in WebAssembly are signed and then each operation on them
may optionally have an unsigned version. This means that when we pass
large signed numbers to JS they actually show up as large negative
numbers even though JS numbers can faithfully represent the type.
This is fixed by adding `>>>0` in a few locations in the generated
bindings to coerce the JS value into an unsigned value.
Closes#1388
Aside from visual deduplication, this actually fixes a bug in js2rust.rs where it didn't call `expose_is_like_none` but used `isLikeNone` inside of `arg.get_64()` branch.
Most of the CLI crates were already in the 2018 edition, and it turns
out that one of the macro crates was already in the 2018 edition so we
may as well move everything to the 2018 edition!
Always nice to remove those `extern crate` statements nowadays!
This commit also does a `cargo fmt --all` to make sure we're conforming
with style again.
This allows to significantly speed up iteration over small collections, where string encoding is the primary overhead.
Related to #1386, but works around only this partial case.
Node.js doesn't currently implement `TextEncoder::encodeInto`. I've raised an upstream issue to add it - https://github.com/nodejs/node/issues/26904 - but it's likely to take some time and will be available only in new releases.
In the meanwhile, it's worth noting that Node.js already has `Buffer::write` which has pretty similar semantics, but doesn't require creating an intermediate view using `.subarray` and instead accepts pointer and length directly.
Also, Node.js has `Buffer::byteLength` helper which allows to efficiently retrieve an encoded byte length of a string upfront, and so allows us to avoid a loop with reallocations.
This change takes leverage of these methods by generating an additional Buffer-based view into the WASM memory and using it for string operations.
I'm seeing up to 35% increase in performance in string-heavy library benchmarks.
We have very few tests today so this starts to add the basics of a test
suite which compiles Cargo projects on-the-fly which will hopefully help
us bolster the amount of assertions we can make about the output.
This commit implements [RFC 8], which enables transitive and transparent
dependencies on NPM. The `module` attribute, when seen and not part of a
local JS snippet, triggers detection of a `package.json` next to
`Cargo.toml`. If found it will cause the `wasm-bindgen` CLI tool to load
and parse the `package.json` within each crate and then create a merged
`package.json` at the end.
[RFC 8]: https://github.com/rustwasm/rfcs/pull/8
This commit fixes an erroneous use-after-free which can happen in
erroneous situations in JS. It's intended that if you invoke a closure
after its environment has been destroyed that you'll immediately get an
error from Rust saying so. The JS binding generation for mutable
closures, however, accidentally did not protect against this.
Each closure has an internal reference count which is incremented while
being invoked and decremented when the invocation finishes and also when
the `Closure` in Rust is dropped. That means there's two branches where
the reference count reaches zero and the internal pointer stored in JS
needs to be set to zero. Only one, however, actually set the pointer to
zero!
This means that if a closure was destroyed while it was being invoked it
would not correctly set its internal pointer to zero. A further
invocation of the closure would then pass as seemingly valid pointer
into Rust, causing a use-after-free.
A test isn't included here specifically for this because our CI has
started failing left-and-right over this test, so this commit will
hopefully just make our CI green!
We've always wanted this to be the deterministic, but usage of `HashMap`
for example can accidentally lead to non-determinism. Looks like one was
forgotten and the bindings were nondeterministic by accident as a
result!
This commit deprecates the `--web`, `--no-modules`, and `--nodejs` flags
in favor of one `--target` flag. The motivation for this commit is to be
consistent between `wasm-bindgen` and `wasm-pack` so documentation for
one is applicable for the other (so we don't have to document everywhere
what the translation is between flags). Additionally this should make it
a bit easier to add new targets (if necessary) in the future as it won't
add to the proliferation of flags.
For now the old flags (like `--web`) continue to be accepted, but
they'll be removed during the next set of breaking changes for
`wasm-bindgen`.
This commit moves our `links` annotation in the `wasm-bindgen` crate to
the `wasm-bindgen-shared` crate. The `links` annotation is used to
ensure that there's only one version of `wasm-bindgen` in a crate graph
because if there are multiple versions then a CLI surely cannot actually
process the wasm binary (as the multiple versions likely have different
formats in their custom sections).
Discovered in #1373 it looks like the usage in `wasm-bindgen` isn't
quite sufficient to cause this deduplication. It turns out that
`wasm-bindgen-shared`, a very core dependency, is actually the most
critical to be deduplicated since its the one that defines the format of
the custom section. In #1373 a case came up where `wasm-bindgen` was
deduplciated but there were two versions of `wasm-bindgen-shared` in the
crate graph, meaning that a `[patch]` for only `wasm-bindgen` wasn't
sufficient, but rather `web-sys` and/or `js-sys` also needed a `[patch]`
annotation to ensure everyone used the right dependencies.
This commit won't actually fix#1373 to the point where it "just works",
but what it does do is present a better error message than an internal
panic of `wasm-bindgen`. The hope is that by moving the `links`
annotation we can catch more errors of this crate graph duplication,
leading to more `[patch]` annotations locally.
Closes#1373
Trying to use a proc macro from a 2018 edition crate in a 2018 edition crate that reexports wasm bindgen's output failed before this commit with "could not find `wasm_bindgen` in `{{root}}`".
This commit was made with
rg " ::wasm_bindgen::" --files-with-matches | xargs sed -i 's/::wasm_bindgen::/wasm_bindgen::/g'
This allows subverting the checks and resolution performed by the
`module` attribute added as part of [RFC 6] and has been discussed in #1343.
Closes#1343
[RFC 6]: https://github.com/rustwasm/rfcs/pull/6
This commit works around Geal/nom#843 where the API of the `nom` crate
changes based on feature selection, meaning we need to be compatible
even if another crate in the crate graph enables a feature.
Ideally this'd be fixed in upstream `nom`, and it looks like it will in
the next major version! For now a local catch-all directive should help
out.
This commit reverts part of the implementation of [RFC 6]. That RFC
specified that the `--browser` flag was going to be repurposed for the
new "natively loadable as ES module output", but unfortunately the
breakage is far broader than initially expected. It turns out that
`wasm-pack` passes `--browser` by default which means that a change to
break `--browser` would break all historical versions of `wasm-pack`
which is a bit much for now.
To solve this the `--browser` flag is going back to what it represents
on the current released version of `wasm-bindgen` (optimize away some
node.js checks in a few places for bundler-style output) and a new
`--web` flag is being introduced as the new deployment strategy.
[RFC 6]: https://github.com/rustwasm/rfcs/pull/6Closes#1318
Using `unsafe` was just a little too eager there so let's use an
off-the-shelf solution for solving the actual problem we have, which is
to allocate strings with a lifetime of `Interner` rather than
deduplicating strings.
This commit starts to add some simple tests for our TypeScript output of
the wasm-bindgen CLI, currently just running `tsc` to make sure syntax
looks good and types are emitted as expected. This'll hopefully be able
to get expanded over time with bug reports as they come in as well as
ensure that we don't regress anything in egregious manners!
Closes#922
This commit is an implementation of [RFC 6] which enables crates to
inline local JS snippets into the final output artifact of
`wasm-bindgen`. This is accompanied with a few minor breaking changes
which are intended to be relatively minor in practice:
* The `module` attribute disallows paths starting with `./` and `../`.
It requires paths starting with `/` to actually exist on the filesystem.
* The `--browser` flag no longer emits bundler-compatible code, but
rather emits an ES module that can be natively loaded into a browser.
Otherwise be sure to check out [the RFC][RFC 6] for more details, and
otherwise this should implement at least the MVP version of the RFC!
Notably at this time JS snippets with `--nodejs` or `--no-modules` are
not supported and will unconditionally generate an error.
[RFC 6]: https://github.com/rustwasm/rfcs/pull/6Closes#1311
Looks like `TextEncoder#encodeInto` isn't compatible when the buffer
passed in is backed by a `SharedArrayBuffer`, so if the module has a
shared thread skip the `encodeInto` optimization entirely.
This commit adds support for the recently implemented standard of
[`TextEncoder#encodeInto`][standard]. This new function is a "bring your
own buffer" style function where we can avoid an intermediate allocation
and copy by encoding strings directly into wasm's memory.
Currently we feature-detect whether `encodeInto` exists as it is only
implemented in recent browsers and not in all browsers. Additionally
this commit emits the binding using `encodeInto` by default, but this
requires `realloc` functionality to be exposed by the wasm module.
Measured locally an empty binary which takes `&str` previously took
7.6k, but after this commit takes 8.7k due to the extra code needed for
`realloc`.
[standard]: https://encoding.spec.whatwg.org/#dom-textencoder-encodeintoCloses#1172
This commit adds experimental support to `wasm-bindgen` to emit and
leverage the `anyref` native wasm type. This native type is still in a
proposal status (the reference-types proposal). The intention of
`anyref` is to be able to directly hold JS values in wasm and pass the
to imported functions, namely to empower eventual host bindings (now
renamed WebIDL bindings) integration where we can skip JS shims
altogether for many imports.
This commit doesn't actually affect wasm-bindgen's behavior at all
as-is, but rather this support requires an opt-in env var to be
configured. Once the support is stable in browsers it's intended that
this will add a CLI switch for turning on this support, eventually
defaulting it to `true` in the far future.
The basic strategy here is to take the `stack` and `slab` globals in the
generated JS glue and move them into wasm using a table. This new table
in wasm is managed at the fringes via injected shims. At
`wasm-bindgen`-time the CLI will rewrite exports and imports with shims
that actually use `anyref` if needed, performing loads/stores inside the
wasm module instead of externally in the wasm module.
This should provide a boost over what we have today, but it's not a
fantastic strategy long term. We have a more grand vision for `anyref`
being a first-class type in the language, but that's on a much longer
horizon and this is currently thought to be the best we can do in terms
of integration in the near future.
The stack/heap JS tables are combined into one wasm table. The stack
starts at the end of the table and grows down with a stack pointer (also
injected). The heap starts at the end and grows up (state managed in
linear memory). The anyref transformation here will hook up various
intrinsics in wasm-bindgen to the runtime functionality if the anyref
supoprt is enabled.
The main tricky treatment here was applied to closures, where we need JS
to use a different function pointer than the one Rust gives it to use a
JS function pointer empowered with anyref. This works by switching up a
bit how descriptors work, embedding the shims to call inside descriptors
rather than communicated at runtime. This means that we're accessing
constant values in the generated JS and we can just update the constant
value accessed.
LLVM's mergefunc pass may mean that the same descriptor function is used
for different closure invocation sites even when the closure itself is
different. This typically only happens with LTO but in theory could
happen at any time!
The assert was tripping when we tried to delete the same function table
entry twice, so instead of a `Vec<usize>` of entries to delete this
commit switches to a `HashSet<usize>` which should do the deduplication
for us and enusre that we delete each descriptor only once.
Closes#1264
We've had a lot of bug reports with upstream webpack currently and while
webpack has a fix it may take a moment to deploy. Let's try and fix
wasm-bindgen in the meantime!
Once webpack is updated we can go back to emitting a producers section
by default and publish a new version of wasm-bindgen.
This should help handle instances like the recent Webpack bug and is
also a useful flag in its own right. For now it's set to `false`, but if
the Webpack bug persists through to tomorrow we likely want to publish a
version of `wasm-bindgen` with it default set to `true`.
This commit moves `wasm-bindgen` the CLI tool from internally using
`parity-wasm` for wasm parsing/serialization to instead use `walrus`.
The `walrus` crate is something we've been working on recently with an
aim to replace the usage of `parity-wasm` in `wasm-bindgen` to make the
current CLI tool more maintainable as well as more future-proof.
The `walrus` crate provides a much nicer AST to work with as well as a
structured `Module`, whereas `parity-wasm` provides a very raw interface
to the wasm module which isn't really appropriate for our use case. The
many transformations and tweaks that wasm-bindgen does have a huge
amount of ad-hoc index management to carefully craft a final wasm
binary, but this is all entirely taken care for us with the `walrus`
crate.
Additionally, `wasm-bindgen` will ingest and rewrite the wasm file,
often changing the binary offsets of functions. Eventually with DWARF
debug information we'll need to be sure to preserve the debug
information throughout the transformations that `wasm-bindgen` does
today. This is practically impossible to do with the `parity-wasm`
architecture, but `walrus` was designed from the get-go to solve this
problem transparently in the `walrus` crate itself. (it doesn't today,
but this is planned work)
It is the intention that this does not end up regressing any
`wasm-bindgen` use cases, neither in functionality or in speed. As a
large change and refactoring, however, it's likely that at least
something will arise! We'll want to continue to remain vigilant to any
issues that come up with this commit.
Note that the `gc` crate has been deleted as part of this change, as the
`gc` crate is no longer necessary since `walrus` does it automatically.
Additionally the `gc` crate was one of the main problems with preserving
debug information as it often deletes wasm items!
Finally, this also starts moving crates to the 2018 edition where
necessary since `walrus` requires the 2018 edition, and in general it's
more pleasant to work within the 2018 edition!
