This commit adds support for the recently implemented standard of
[`TextEncoder#encodeInto`][standard]. This new function is a "bring your
own buffer" style function where we can avoid an intermediate allocation
and copy by encoding strings directly into wasm's memory.
Currently we feature-detect whether `encodeInto` exists as it is only
implemented in recent browsers and not in all browsers. Additionally
this commit emits the binding using `encodeInto` by default, but this
requires `realloc` functionality to be exposed by the wasm module.
Measured locally an empty binary which takes `&str` previously took
7.6k, but after this commit takes 8.7k due to the extra code needed for
`realloc`.
[standard]: https://encoding.spec.whatwg.org/#dom-textencoder-encodeintoCloses#1172
This commit adds experimental support to `wasm-bindgen` to emit and
leverage the `anyref` native wasm type. This native type is still in a
proposal status (the reference-types proposal). The intention of
`anyref` is to be able to directly hold JS values in wasm and pass the
to imported functions, namely to empower eventual host bindings (now
renamed WebIDL bindings) integration where we can skip JS shims
altogether for many imports.
This commit doesn't actually affect wasm-bindgen's behavior at all
as-is, but rather this support requires an opt-in env var to be
configured. Once the support is stable in browsers it's intended that
this will add a CLI switch for turning on this support, eventually
defaulting it to `true` in the far future.
The basic strategy here is to take the `stack` and `slab` globals in the
generated JS glue and move them into wasm using a table. This new table
in wasm is managed at the fringes via injected shims. At
`wasm-bindgen`-time the CLI will rewrite exports and imports with shims
that actually use `anyref` if needed, performing loads/stores inside the
wasm module instead of externally in the wasm module.
This should provide a boost over what we have today, but it's not a
fantastic strategy long term. We have a more grand vision for `anyref`
being a first-class type in the language, but that's on a much longer
horizon and this is currently thought to be the best we can do in terms
of integration in the near future.
The stack/heap JS tables are combined into one wasm table. The stack
starts at the end of the table and grows down with a stack pointer (also
injected). The heap starts at the end and grows up (state managed in
linear memory). The anyref transformation here will hook up various
intrinsics in wasm-bindgen to the runtime functionality if the anyref
supoprt is enabled.
The main tricky treatment here was applied to closures, where we need JS
to use a different function pointer than the one Rust gives it to use a
JS function pointer empowered with anyref. This works by switching up a
bit how descriptors work, embedding the shims to call inside descriptors
rather than communicated at runtime. This means that we're accessing
constant values in the generated JS and we can just update the constant
value accessed.
LLVM's mergefunc pass may mean that the same descriptor function is used
for different closure invocation sites even when the closure itself is
different. This typically only happens with LTO but in theory could
happen at any time!
The assert was tripping when we tried to delete the same function table
entry twice, so instead of a `Vec<usize>` of entries to delete this
commit switches to a `HashSet<usize>` which should do the deduplication
for us and enusre that we delete each descriptor only once.
Closes#1264
We've had a lot of bug reports with upstream webpack currently and while
webpack has a fix it may take a moment to deploy. Let's try and fix
wasm-bindgen in the meantime!
Once webpack is updated we can go back to emitting a producers section
by default and publish a new version of wasm-bindgen.
This should help handle instances like the recent Webpack bug and is
also a useful flag in its own right. For now it's set to `false`, but if
the Webpack bug persists through to tomorrow we likely want to publish a
version of `wasm-bindgen` with it default set to `true`.
This commit moves `wasm-bindgen` the CLI tool from internally using
`parity-wasm` for wasm parsing/serialization to instead use `walrus`.
The `walrus` crate is something we've been working on recently with an
aim to replace the usage of `parity-wasm` in `wasm-bindgen` to make the
current CLI tool more maintainable as well as more future-proof.
The `walrus` crate provides a much nicer AST to work with as well as a
structured `Module`, whereas `parity-wasm` provides a very raw interface
to the wasm module which isn't really appropriate for our use case. The
many transformations and tweaks that wasm-bindgen does have a huge
amount of ad-hoc index management to carefully craft a final wasm
binary, but this is all entirely taken care for us with the `walrus`
crate.
Additionally, `wasm-bindgen` will ingest and rewrite the wasm file,
often changing the binary offsets of functions. Eventually with DWARF
debug information we'll need to be sure to preserve the debug
information throughout the transformations that `wasm-bindgen` does
today. This is practically impossible to do with the `parity-wasm`
architecture, but `walrus` was designed from the get-go to solve this
problem transparently in the `walrus` crate itself. (it doesn't today,
but this is planned work)
It is the intention that this does not end up regressing any
`wasm-bindgen` use cases, neither in functionality or in speed. As a
large change and refactoring, however, it's likely that at least
something will arise! We'll want to continue to remain vigilant to any
issues that come up with this commit.
Note that the `gc` crate has been deleted as part of this change, as the
`gc` crate is no longer necessary since `walrus` does it automatically.
Additionally the `gc` crate was one of the main problems with preserving
debug information as it often deletes wasm items!
Finally, this also starts moving crates to the 2018 edition where
necessary since `walrus` requires the 2018 edition, and in general it's
more pleasant to work within the 2018 edition!
Reported in #1191 the fix requires us to get a bit creative I think. The
general gist is that a block like this:
#[wasm_bindgen]
impl Foo {
pub fn foo() {}
}
was previously expanded all in one go. Now, however, it's expanded into:
impl Foo {
#[__wasm_bindgen_class_marker(Foo = "Foo")]
pub fn foo() {}
}
// goop generated by orginal #[wasm_bindgen]
This method of expansion takes advantage of rustc's recursive expansion
feature. It also allows us to expand `impl` blocks and allow inner items
to not be fully expanded yet, such as still having `#[cfg]` attributes
(like in the original bug report).
We use theinternal `__wasm_bindgen_class_marker` to indicate that we're
parsing an `ImplItemMethod` unconditionally, and then generation
proceeds as usual. The only final catch is that when we're expanding in
an `impl` block we have to generate tokens for the `Program`
(wasm-bindgen injected goop like the custom section) inside the body
of the function itself instead of next to it. Otherwise we'd get syntax
errors inside of impl blocks!
Closes#1191
This commit updates the `--debug` output of `wasm-bindgen` from the CLI
to catch all JS exceptions from imported functions, log such, and then
rethrow. It's hoped that this can be used when necessary to learn more
information about thrown exceptions and where an uncaught exception
could be causing issues with Rust code.
Closes#1176
Throw it in an `Option` and then `take()` it when we consume it to
ensure that future calls to insert data into it panic instead of
producing inconsistent JS.
This was removed when mozilla specific extensions were removed. It is not
mozilla specific though and currently the only way to show webcam data in
a video element that I am aware of.
For all typed arrays, this commit adds:
* `TypedArray::view(src: &[Type])`
* `TypedArray::copy_to(&self, dst: &mut [Type])`
The `view` function is unsafe because it doesn't provide any guarantees
about lifetimes or mutability. The `copy_to` function is, however, safe.
Closes#811
We might gc a table away so if we need to export it be sure to do so
before we gc! Additionally remove an extraneous gc that snuck in at some
point, no need to do more than one.
Closes#1130
Turns out `heap.fill(undefined)` is required to ensure it's a dense
array, otherwise we'll accidentally be a sparse array and much slower
than necessary!
This commit switches strategies for storing `JsValue` from a heap/stack
to just one heap. This mirrors the new strategy for `JsValue` storage
in #1002 and should make multiplexing those strategies at
`wasm-bindgen`-time much easier.
Instead of having one array which acts as a stack for borrowed values
and one array for a heap of borrowed values, only one JS array is used
for storage of JS values now. This makes `getObject` far simpler by
simply being an array access, but it means that cloning an object now
reserves a new slot instead of reference counting it. If the old
reference counting behavior is needed it's thought that `Rc<JsValue>`
can be used in Rust.
The new "heap" has an initial stack pointer which grows downwards, and a
heap which grows upwards. The heap is a singly-linked-list which is
allocated/deallocated from. The stack grows downwards to zero and
presumably starts generating errors once it underflows. An initial stack
size of 32 is chosen as that should encompass all use cases today, but
we can eventually probably add configuration for this!
Note that the heap is initialized to all `null` for the stack and then
the initial JS values (`undefined`, `null`, `true`, `false`) are pushed
onto the heap in reserved locations.
Previously `catch` and `variadic` would exopse methods in our JS shims,
but they did so earlier than necessary. Turns out `variadic` didn't
actually need to expose anything and `catch` could do so much later!
Currently closure shims are communicated to JS at runtime, although at
runtime the same constant value is always passed to JS! More pressing,
however, work in #1002 requires knowledge of closure descriptor indices
at `wasm-bindgen` time which is not currently known.
Since the closure descriptor shims and such are already constant values,
this commit moves the descriptor function indices into the *descriptor*
for a closure/function pointer. This way we can learn about these values
at `wasm-bindgen` time instead of only knowing them at runtime.
This should have no semantic change on users of `wasm-bindgen`, although
some closure invocations may be slightly speedier because there's less
arguments being transferred over the boundary. Overall though this will
help #1002 as the closure shims that the Rust compiler generates may not
be the exact ones we hand out to JS, but rather wrappers around them
which do `anyref` business things.
This commit fixes a case in `wasm2es6js` where if an imported function
was reexported it wasn't handled correctly. This doesn't have a direct
test but came up during the development of #1002
This is split out from #1002 and is intended to fix the tool's handling
of the `start` function. For the most accurate emulation of the wasm ESM
spec I believe we need to defer execution of the start function until
all our exports are wired up which should allow valid cyclical
references during instantiation.
The fix here is to remove the start function, if one is present, and
inject an invocation of it at the end of initialization (after our
exports are wired up). This fixes tests on #1002, but doesn't have any
direct analogue for tests here just yet.
Along the way because multiple files now come out of `wasm2es6js` by
default I've added an `--out-dir` argument as well as `-o` to ensure
that a folder for all outputs can be specified.
This commit adds a new attribute to `#[wasm_bindgen]`: `start`. The
`start` attribute can be used to indicate that a function should be
executed when the module is loaded, configuring the `start` function of
the wasm executable. While this doesn't necessarily literally configure
the `start` section, it does its best!
Only one crate in a crate graph may indicate `#[wasm_bindgen(start)]`,
so it's not recommended to be used in libraries but only end-user
applications. Currently this still must be used with the `crate-type =
["cdylib"]` annotation in `Cargo.toml`.
The implementation here is somewhat tricky because of the circular
dependency between our generated JS and the wasm file that we emit. This
circular dependency makes running initialization routines (like the
`start` shim) particularly fraught with complications because one may
need to run before the other but bundlers may not necessarily respect
it. Workarounds have been implemented for various emission strategies,
for example calling the start function directly after exports are wired
up with `--no-modules` and otherwise working around what appears to be
a Webpack bug with initializers running in a different order than we'd
like. In any case, this in theory doesn't show up to the end user!
Closes#74
This commit implements a system that will assert that all
`#[wasm_bindgen]` attributes are actually used during compilation. This
should help ensure that we don't sneak in stray attributes that don't
actually end up having any meaning, and hopefully make it a bit easier
to learn `#[wasm_bindgen]`!
This generates a `*.d.ts` file for the wasm file that wasm-bindgen emits
whenever typescript is enable *in addition* to the `*.d.ts` file that
already exists for the JS shim.
Closes#1040
Recently proposed in WebAssembly/tool-conventions#65 each wasm file will
now have an optional `producers` section listing the tooling that went
into producing it. Let's add `wasm-bindgen` in when it processes a wasm
file!
This commit adds an optimization to `wasm-bindgen` to directly import
and invoke other modules' functions from the wasm module, rather than
going through a shim in the imported bindings. This will be an important
optimization in the future for the host bindings proposal, but for now
it's largely just a proof-of-concept to show that we can do it and is
unlikely to bring about many performance benefits.
The implementation in this commit is largely refactoring to reorganize a
bit how functions are imported, but the implementation happens in
`generate_import_function`.
With this commit, 71/287 imports in the `tests/wasm/main.rs` suite get
hooked up directly to the ES modules, no shims needed!
When returning a ptr/length for allocations and such wasm-bindgen's
generated JS would previously return an array with two elements. It
turns out this doesn't optimize well in all engines! (See #1031). It
looks like we can optimize the array destructuring a bit more, but this
is all generated code which doesn't need to be too readable so we can
also remove the temporary allocation entirely and just pass the second
element of this array through a global instead of the return value.
Closes#1031
This fixes a mistake in allowing a `WebAssembly.Module` to be passed to
the initialization function in `--no-modules` mode by ensuring that it
resolves to a map of an instance/module instead of just resolving to an
instance.
This commit adds a `--remove-name-section` flag to the `wasm-bindgen`
command which will remove the `name` section of the wasm file, used to
indicate the names of functions typically used in debugging. This flag
is off-by-default and will primarily be controlled by wasm-pack,
typically being passed by default with `wasm-pack build --release`.
Closes#1021
This commit switches all imports of JS methods to `structural` by
default. Proposed in [RFC 5] this should increase the performance of
bindings today while also providing future-proofing for possible
confusion with the recent addition of the `Deref` trait for all imported
types by default as well.
A new attribute, `host_binding`, is introduced in this PR as well to
recover the old behavior of binding directly to an imported function
which will one day be the precise function on the prototype. Eventually
`web-sys` will switcsh over entirely to being driven via `host_binding`
methods, but for now it's been measured to be not quite as fast so we're
not making that switch yet.
Note that `host_binding` differs from the proposed name of `final` due
to the controversy, and its hoped that `host_binding` is a good
middle-ground!
[RFC 5]: https://rustwasm.github.io/rfcs/005-structural-and-deref.html
This commit removes shims, where possible, for `structural` items.
Instead of generating code that looks like:
const target = function() { this.foo(); };
exports.__wbg_thing = function(a) { target.call(getObject(a)); };
we now instead generate:
exports.__wbg_thing = function(a) { getObject(a).foo(); };
Note that this only applies to `structural` bindings, all default
bindings (as of this commit) are still using imported targets to ensure
that their binding can't change after instantiation.
This change was [detailed in RFC #5][link] as an important optimization
for `structural` bindings to ensure they've got performance parity with
today's non-`structural` default bindings.
[link]: https://rustwasm.github.io/rfcs/005-structural-and-deref.html#why-is-it-ok-to-make-structural-the-default
Previously `arguments` was used to pass around an array of arguments,
but this wasn't actually a `js_sys::Array` but rather a somewhat
esoteric internal object. When switching over `Array` methods to be
`structural` this caused issues because the inherent methods on an
`arguments` object were different than that of `js_sys::Array`.
This commit implements the first half of [RFC #5] where the `Deref`
trait is implemented for all imported types. The target of `Deref` is
either the first entry of the list of `extends` attribute or `JsValue`.
All examples using `.as_ref()` with various `web-sys` types have been
updated to the more ergonomic deref casts now. Additionally the
`web-sys` generation of the `extends` array has been fixed slightly to
explicitly list implementatoins in the hierarchy order to ensure the
correct target for `Deref` is chosen.
[RFC #5]: https://github.com/rustwasm/rfcs/blob/master/text/005-structural-and-deref.md
For example, the constructor in Response.webidl accepts multiple types. However, one of those types is `ReadableStream` which isn't defined yet, and that causes all constructors for Response to be skipped even though the other argument types could be supported.
This commit makes the `to_idl_type` infallible, returning a new enum
variant, `UnknownInterface`, in the one location that we still return
`None`. By making this infallible we can ensure that expansion of unions
which have unknown types still generate methods for all the variants
which we actually have all the methods for!
This commit adds a test harness and the beginnings of a test suite for
the crate that performs GC over a wasm module. This crate historically
has had zero tests because it was thought that it would no longer be
used once LLD landed with `--gc-sections`, but `wasm-bindgen` has come
to rely more and more on `wasm-gc` for various purposes.
The last release of `wasm-bindgen` was also released with a bug in the
recently refactored support in the `wasm-gc` crate, providing a perfect
time and motivation to start writing some tests!
All tests added here are `*.wat` files which contain the expected output
after the gc pass is executed. Tests are automatically updated with
`BLESS_TESTS=1` in the environment, which is the expected way to
generate the output for each test.
This commit updates the `wasm-gc` pass of wasm-bindgen to eliminate
duplicate types in the type section, effectively enabling a gc of the
type section itself. The main purpose here is ensure that code generated
by `wasm-bindgen` itself doesn't have to go too far out of its way to
deduplicate at generation time, but rather it can rely on the gc pass to
clean up.
Note that this currently depends on paritytech/parity-wasm#231, but this
can be updated if that ends up not landing.
This commit adds support for running a gc pass over locals in a
function. This will remove dead local declarations for a function
(completely unused) as well as compact existing entries to ensure that
we don't have two local declarations of the same type.
While this was initially intended for some future support of emitting
shims in `wasm-bindgen`, it turns out this pass is firing quite a lot
over existing functions generated by LLVM. Looks like we may see benefit
from this today with slightly smaller wasm binaries!
These don't seem to be widely used and they're not needed by
wasm-bindgen itself, so let's remove the symbols by default and
optionally in the future we can add an option to retain them.
This commit restructures some of the internals of `wasm-gc` now that
I've actually got a better grasp on the wasm format and what all the
ownership edges look like. This shouldn't actually result in any
user-facing changes, but should make us be a bit more compatible with
operations in the future.
Memories/tables/elements/segments are no longer considered automatic
roots but rather need to be rooted by something else to prevent a gc.
For example an element section is gc'd along with a table if the table
is never referenced, along with data segments as well if the memory
isn't referenced.
Additionally all index sets now don't contained offseted indices, but
rather everything is always stored relative to the "index space" to
ensure consistency.
This should make it a bit easier to add future items to gc!
Previously whenever a future readiness notification came in we would
immediately start polling a future. This ends up having two downsides,
however:
* First, the stack depth may run a risk of getting blown. There's no
recursion limit to defer execution to later, which means that if
futures are always ready we'll keep making the stack deeper.
* Second, and more worrisome in the near term, apparently future
adapaters in the `futures` crate (namely the unsync oneshot channel)
doesn't actually work if you immediately poll on readiness. This may
or may not be a bug in the `futures` crate but it's good to fix it
here anyway.
As a result whenever a future is ready to get polled again we defer its
polling to the next turn of the event loop. This should ensure that the
current call stack is always drained and we're effectively enqueueing
the future to be polled in the near future.
... and add a parallel raytracing demo!
This commit adds enough support to `wasm-bindgen` to produce a workable
wasm binary *today* with the experimental WebAssembly threads support
implemented in Firefox Nightly. I've tried to comment what's going on in
the commits and such, but at a high level the changes made here are:
* A new transformation, living in a new `wasm-bindgen-threads-xform`
crate, prepares a wasm module for parallel execution. This performs a
number of mundane tasks which I hope to detail in a blog post later on.
* The `--no-modules` output is enhanced with more support for when
shared memory is enabled, allowing passing in the module/memory to
initialize the wasm instance on multiple threads (sharing both module
and memory).
* The `wasm-bindgen` crate now offers the ability, in `--no-modules`
mode, to get a handle on the `WebAssembly.Module` instance.
* The example itself requires Xargo to recompile the standard library
with atomics and an experimental feature enabled. Afterwards it
experimentally also enables threading support in wasm-bindgen.
I've also added hopefully enough CI support to compile this example in a
builder so we can upload it and poke around live online. I hope to
detail more about the technical details here in a blog post soon as
well!
This shaves a little over 2MB off the download locally for Linux,
removing debuginfo (which no one's probably gonna use anyway) as well as
switching from jemalloc to the system allocator.
I've noticed this in a few cases where it's sometimes easy to have a
`WebAssembly.Module` on-hand for the `--no-modules` mode where you don't
want to necessarily `fetch`. This commit changes the exported
initialization function in `--no-modules` mode to support both!
