This commit is an implementation of [RFC 6] which enables crates to
inline local JS snippets into the final output artifact of
`wasm-bindgen`. This is accompanied with a few minor breaking changes
which are intended to be relatively minor in practice:
* The `module` attribute disallows paths starting with `./` and `../`.
It requires paths starting with `/` to actually exist on the filesystem.
* The `--browser` flag no longer emits bundler-compatible code, but
rather emits an ES module that can be natively loaded into a browser.
Otherwise be sure to check out [the RFC][RFC 6] for more details, and
otherwise this should implement at least the MVP version of the RFC!
Notably at this time JS snippets with `--nodejs` or `--no-modules` are
not supported and will unconditionally generate an error.
[RFC 6]: https://github.com/rustwasm/rfcs/pull/6Closes#1311
Looks like `TextEncoder#encodeInto` isn't compatible when the buffer
passed in is backed by a `SharedArrayBuffer`, so if the module has a
shared thread skip the `encodeInto` optimization entirely.
This commit adds support for the recently implemented standard of
[`TextEncoder#encodeInto`][standard]. This new function is a "bring your
own buffer" style function where we can avoid an intermediate allocation
and copy by encoding strings directly into wasm's memory.
Currently we feature-detect whether `encodeInto` exists as it is only
implemented in recent browsers and not in all browsers. Additionally
this commit emits the binding using `encodeInto` by default, but this
requires `realloc` functionality to be exposed by the wasm module.
Measured locally an empty binary which takes `&str` previously took
7.6k, but after this commit takes 8.7k due to the extra code needed for
`realloc`.
[standard]: https://encoding.spec.whatwg.org/#dom-textencoder-encodeintoCloses#1172
This commit adds experimental support to `wasm-bindgen` to emit and
leverage the `anyref` native wasm type. This native type is still in a
proposal status (the reference-types proposal). The intention of
`anyref` is to be able to directly hold JS values in wasm and pass the
to imported functions, namely to empower eventual host bindings (now
renamed WebIDL bindings) integration where we can skip JS shims
altogether for many imports.
This commit doesn't actually affect wasm-bindgen's behavior at all
as-is, but rather this support requires an opt-in env var to be
configured. Once the support is stable in browsers it's intended that
this will add a CLI switch for turning on this support, eventually
defaulting it to `true` in the far future.
The basic strategy here is to take the `stack` and `slab` globals in the
generated JS glue and move them into wasm using a table. This new table
in wasm is managed at the fringes via injected shims. At
`wasm-bindgen`-time the CLI will rewrite exports and imports with shims
that actually use `anyref` if needed, performing loads/stores inside the
wasm module instead of externally in the wasm module.
This should provide a boost over what we have today, but it's not a
fantastic strategy long term. We have a more grand vision for `anyref`
being a first-class type in the language, but that's on a much longer
horizon and this is currently thought to be the best we can do in terms
of integration in the near future.
The stack/heap JS tables are combined into one wasm table. The stack
starts at the end of the table and grows down with a stack pointer (also
injected). The heap starts at the end and grows up (state managed in
linear memory). The anyref transformation here will hook up various
intrinsics in wasm-bindgen to the runtime functionality if the anyref
supoprt is enabled.
The main tricky treatment here was applied to closures, where we need JS
to use a different function pointer than the one Rust gives it to use a
JS function pointer empowered with anyref. This works by switching up a
bit how descriptors work, embedding the shims to call inside descriptors
rather than communicated at runtime. This means that we're accessing
constant values in the generated JS and we can just update the constant
value accessed.
LLVM's mergefunc pass may mean that the same descriptor function is used
for different closure invocation sites even when the closure itself is
different. This typically only happens with LTO but in theory could
happen at any time!
The assert was tripping when we tried to delete the same function table
entry twice, so instead of a `Vec<usize>` of entries to delete this
commit switches to a `HashSet<usize>` which should do the deduplication
for us and enusre that we delete each descriptor only once.
Closes#1264
We've had a lot of bug reports with upstream webpack currently and while
webpack has a fix it may take a moment to deploy. Let's try and fix
wasm-bindgen in the meantime!
Once webpack is updated we can go back to emitting a producers section
by default and publish a new version of wasm-bindgen.
This should help handle instances like the recent Webpack bug and is
also a useful flag in its own right. For now it's set to `false`, but if
the Webpack bug persists through to tomorrow we likely want to publish a
version of `wasm-bindgen` with it default set to `true`.
This commit moves `wasm-bindgen` the CLI tool from internally using
`parity-wasm` for wasm parsing/serialization to instead use `walrus`.
The `walrus` crate is something we've been working on recently with an
aim to replace the usage of `parity-wasm` in `wasm-bindgen` to make the
current CLI tool more maintainable as well as more future-proof.
The `walrus` crate provides a much nicer AST to work with as well as a
structured `Module`, whereas `parity-wasm` provides a very raw interface
to the wasm module which isn't really appropriate for our use case. The
many transformations and tweaks that wasm-bindgen does have a huge
amount of ad-hoc index management to carefully craft a final wasm
binary, but this is all entirely taken care for us with the `walrus`
crate.
Additionally, `wasm-bindgen` will ingest and rewrite the wasm file,
often changing the binary offsets of functions. Eventually with DWARF
debug information we'll need to be sure to preserve the debug
information throughout the transformations that `wasm-bindgen` does
today. This is practically impossible to do with the `parity-wasm`
architecture, but `walrus` was designed from the get-go to solve this
problem transparently in the `walrus` crate itself. (it doesn't today,
but this is planned work)
It is the intention that this does not end up regressing any
`wasm-bindgen` use cases, neither in functionality or in speed. As a
large change and refactoring, however, it's likely that at least
something will arise! We'll want to continue to remain vigilant to any
issues that come up with this commit.
Note that the `gc` crate has been deleted as part of this change, as the
`gc` crate is no longer necessary since `walrus` does it automatically.
Additionally the `gc` crate was one of the main problems with preserving
debug information as it often deletes wasm items!
Finally, this also starts moving crates to the 2018 edition where
necessary since `walrus` requires the 2018 edition, and in general it's
more pleasant to work within the 2018 edition!
This commit updates the `--debug` output of `wasm-bindgen` from the CLI
to catch all JS exceptions from imported functions, log such, and then
rethrow. It's hoped that this can be used when necessary to learn more
information about thrown exceptions and where an uncaught exception
could be causing issues with Rust code.
Closes#1176
Throw it in an `Option` and then `take()` it when we consume it to
ensure that future calls to insert data into it panic instead of
producing inconsistent JS.
We might gc a table away so if we need to export it be sure to do so
before we gc! Additionally remove an extraneous gc that snuck in at some
point, no need to do more than one.
Closes#1130
Turns out `heap.fill(undefined)` is required to ensure it's a dense
array, otherwise we'll accidentally be a sparse array and much slower
than necessary!
This commit switches strategies for storing `JsValue` from a heap/stack
to just one heap. This mirrors the new strategy for `JsValue` storage
in #1002 and should make multiplexing those strategies at
`wasm-bindgen`-time much easier.
Instead of having one array which acts as a stack for borrowed values
and one array for a heap of borrowed values, only one JS array is used
for storage of JS values now. This makes `getObject` far simpler by
simply being an array access, but it means that cloning an object now
reserves a new slot instead of reference counting it. If the old
reference counting behavior is needed it's thought that `Rc<JsValue>`
can be used in Rust.
The new "heap" has an initial stack pointer which grows downwards, and a
heap which grows upwards. The heap is a singly-linked-list which is
allocated/deallocated from. The stack grows downwards to zero and
presumably starts generating errors once it underflows. An initial stack
size of 32 is chosen as that should encompass all use cases today, but
we can eventually probably add configuration for this!
Note that the heap is initialized to all `null` for the stack and then
the initial JS values (`undefined`, `null`, `true`, `false`) are pushed
onto the heap in reserved locations.
Previously `catch` and `variadic` would exopse methods in our JS shims,
but they did so earlier than necessary. Turns out `variadic` didn't
actually need to expose anything and `catch` could do so much later!
Currently closure shims are communicated to JS at runtime, although at
runtime the same constant value is always passed to JS! More pressing,
however, work in #1002 requires knowledge of closure descriptor indices
at `wasm-bindgen` time which is not currently known.
Since the closure descriptor shims and such are already constant values,
this commit moves the descriptor function indices into the *descriptor*
for a closure/function pointer. This way we can learn about these values
at `wasm-bindgen` time instead of only knowing them at runtime.
This should have no semantic change on users of `wasm-bindgen`, although
some closure invocations may be slightly speedier because there's less
arguments being transferred over the boundary. Overall though this will
help #1002 as the closure shims that the Rust compiler generates may not
be the exact ones we hand out to JS, but rather wrappers around them
which do `anyref` business things.
This commit fixes a case in `wasm2es6js` where if an imported function
was reexported it wasn't handled correctly. This doesn't have a direct
test but came up during the development of #1002
This is split out from #1002 and is intended to fix the tool's handling
of the `start` function. For the most accurate emulation of the wasm ESM
spec I believe we need to defer execution of the start function until
all our exports are wired up which should allow valid cyclical
references during instantiation.
The fix here is to remove the start function, if one is present, and
inject an invocation of it at the end of initialization (after our
exports are wired up). This fixes tests on #1002, but doesn't have any
direct analogue for tests here just yet.
Along the way because multiple files now come out of `wasm2es6js` by
default I've added an `--out-dir` argument as well as `-o` to ensure
that a folder for all outputs can be specified.
This commit adds a new attribute to `#[wasm_bindgen]`: `start`. The
`start` attribute can be used to indicate that a function should be
executed when the module is loaded, configuring the `start` function of
the wasm executable. While this doesn't necessarily literally configure
the `start` section, it does its best!
Only one crate in a crate graph may indicate `#[wasm_bindgen(start)]`,
so it's not recommended to be used in libraries but only end-user
applications. Currently this still must be used with the `crate-type =
["cdylib"]` annotation in `Cargo.toml`.
The implementation here is somewhat tricky because of the circular
dependency between our generated JS and the wasm file that we emit. This
circular dependency makes running initialization routines (like the
`start` shim) particularly fraught with complications because one may
need to run before the other but bundlers may not necessarily respect
it. Workarounds have been implemented for various emission strategies,
for example calling the start function directly after exports are wired
up with `--no-modules` and otherwise working around what appears to be
a Webpack bug with initializers running in a different order than we'd
like. In any case, this in theory doesn't show up to the end user!
Closes#74
This generates a `*.d.ts` file for the wasm file that wasm-bindgen emits
whenever typescript is enable *in addition* to the `*.d.ts` file that
already exists for the JS shim.
Closes#1040
Recently proposed in WebAssembly/tool-conventions#65 each wasm file will
now have an optional `producers` section listing the tooling that went
into producing it. Let's add `wasm-bindgen` in when it processes a wasm
file!
This commit adds an optimization to `wasm-bindgen` to directly import
and invoke other modules' functions from the wasm module, rather than
going through a shim in the imported bindings. This will be an important
optimization in the future for the host bindings proposal, but for now
it's largely just a proof-of-concept to show that we can do it and is
unlikely to bring about many performance benefits.
The implementation in this commit is largely refactoring to reorganize a
bit how functions are imported, but the implementation happens in
`generate_import_function`.
With this commit, 71/287 imports in the `tests/wasm/main.rs` suite get
hooked up directly to the ES modules, no shims needed!
When returning a ptr/length for allocations and such wasm-bindgen's
generated JS would previously return an array with two elements. It
turns out this doesn't optimize well in all engines! (See #1031). It
looks like we can optimize the array destructuring a bit more, but this
is all generated code which doesn't need to be too readable so we can
also remove the temporary allocation entirely and just pass the second
element of this array through a global instead of the return value.
Closes#1031
This fixes a mistake in allowing a `WebAssembly.Module` to be passed to
the initialization function in `--no-modules` mode by ensuring that it
resolves to a map of an instance/module instead of just resolving to an
instance.
This commit adds a `--remove-name-section` flag to the `wasm-bindgen`
command which will remove the `name` section of the wasm file, used to
indicate the names of functions typically used in debugging. This flag
is off-by-default and will primarily be controlled by wasm-pack,
typically being passed by default with `wasm-pack build --release`.
Closes#1021
This commit removes shims, where possible, for `structural` items.
Instead of generating code that looks like:
const target = function() { this.foo(); };
exports.__wbg_thing = function(a) { target.call(getObject(a)); };
we now instead generate:
exports.__wbg_thing = function(a) { getObject(a).foo(); };
Note that this only applies to `structural` bindings, all default
bindings (as of this commit) are still using imported targets to ensure
that their binding can't change after instantiation.
This change was [detailed in RFC #5][link] as an important optimization
for `structural` bindings to ensure they've got performance parity with
today's non-`structural` default bindings.
[link]: https://rustwasm.github.io/rfcs/005-structural-and-deref.html#why-is-it-ok-to-make-structural-the-default
... and add a parallel raytracing demo!
This commit adds enough support to `wasm-bindgen` to produce a workable
wasm binary *today* with the experimental WebAssembly threads support
implemented in Firefox Nightly. I've tried to comment what's going on in
the commits and such, but at a high level the changes made here are:
* A new transformation, living in a new `wasm-bindgen-threads-xform`
crate, prepares a wasm module for parallel execution. This performs a
number of mundane tasks which I hope to detail in a blog post later on.
* The `--no-modules` output is enhanced with more support for when
shared memory is enabled, allowing passing in the module/memory to
initialize the wasm instance on multiple threads (sharing both module
and memory).
* The `wasm-bindgen` crate now offers the ability, in `--no-modules`
mode, to get a handle on the `WebAssembly.Module` instance.
* The example itself requires Xargo to recompile the standard library
with atomics and an experimental feature enabled. Afterwards it
experimentally also enables threading support in wasm-bindgen.
I've also added hopefully enough CI support to compile this example in a
builder so we can upload it and poke around live online. I hope to
detail more about the technical details here in a blog post soon as
well!
I've noticed this in a few cases where it's sometimes easy to have a
`WebAssembly.Module` on-hand for the `--no-modules` mode where you don't
want to necessarily `fetch`. This commit changes the exported
initialization function in `--no-modules` mode to support both!
This commit migrates away from using Serde for the custom section in
wasm executables. This is a refactoring of a purely-internal data
structure to `wasm-bindgen` and should have no visible functional change
on users.
The motivation for this commit is two fold:
* First, the compile times using `serde_json` and `serde_derive` for the
syntax extension isn't the most fun.
* Second, eventually we're going to want to stablize the layout of the
custom section, and it's highly unlikely to be json!
Primarily, though, the intention of this commit is to improve the
cold-cache compile time of `wasm-bindgen` by ensuring that for new users
this project builds as quickly as possible. By removing some heavyweight
dependencies from the procedural macro, `serde`, `serde_derive`, and
`serde_json`, we're able to get a pretty nice build time improvement for
the `wasm-bindgen` crate itself:
| | single-core build | parallel build |
|-------------|-------------------|----------------|
| master | 36.5s | 17.3s |
| this commit | 20.5s | 11.8s |
These are't really end-all-be-all wins but they're much better
especially on the spectrum of weaker CPUs (in theory modeled by the
single-core case showing we have 42% less CPU work in theory).
This commit fixes instantiation of the wasm module even if some of the
improted APIs don't exist. This extends the functionality initially
added in #409 to attempt to gracefully allow importing values from the
environment which don't actually exist in all contexts. In addition to
nonexistent methods being handled now entire nonexistent types are now
also handled.
I suspect that eventually we'll add a CLI flag to `wasm-bindgen` to say
"I assert everything exists, don't check it" to trim out the extra JS
glue generated here. In the meantime though this'll pave the way for a
wasm-bindgen shim to be instantiated in both a web worker and the main
thread, while using DOM-like APIs only on the main thread.
The bindings generation for a class would accidentally omit the `__wrap`
function if it was only discovered very late in the process that
`__wrap` was needed, after we'd already passed the point where we needed
to have decided that.
This commit moves struct field generation of bindings much earlier in
the binding generation process which should ensure everything is all
hooked up by the time we generate the classes themselves.
Closes#949
The `wasm-bindgen` crate is effectively the only user of this crate now
that the `wasm-gc` tool has been deprecated. It's also much easier to
keep it in this repository as it's easier to sync changes to
`parity-wasm`. I'd also like to start refactoring out utilities for
managing a `parity_wasm::Module` to share between this crate and the
other CLI support code.
This commit does a few things, including:
* Fixing the generated JS of `wasm-bindgen` to allow polyfills to work.
(a minor tweak of the generated JS)
* All examples are updated to include a Webpack-specific polyfill for
these two types to get examples working in Edge.
* A new page has been added to the guide about supported browsers. This
mentions known caveats like IE 11 requiring `wasm2js` as well as
documenting some `TextEncoder` and `TextDecoder` workarounds for Edge.
Closes#895
This commit improves the codegen for `Closure<T>`, primarily for ZST
where the closure doesn't actually capture anything. Previously
`wasm-bindgen` would unconditionally allocate an `Rc` for a fat pointer,
meaning that it would always hit the allocator even when the `Box<T>`
didn't actually contain an allocation. Now the reference count for the
closure is stored on the JS object rather than in Rust.
Some more advanced tests were added along the way to ensure that
functionality didn't regress, and otherwise the calling convention for
`Closure` changed a good deal but should still be the same user-facing.
The primary change was that the reference count reaching zero may cause
JS to need to run the destructor. It simply returns this information in
`Drop for Closure` and otherwise when calling it now also retains a
function pointer that runs the destructor.
Closes#874
This commit implements support for binding APIs that take
`Uint8ClampedArray` in JS. This is pretty rare but comes up in a
`web-sys` binding or two, and we're now able to bind these APIs instead
of having to omit the bindings.
The `Uint8ClampedArray` type is bound by using the `Clamped` marker
struct in Rust. For example this is declaring a JS API that takes
`Uint8ClampedArray`:
use wasm_bindgen::Clamped;
#[wasm_bindgen]
extern {
fn takes_clamped(a: Clamped<&[u8]>);
}
The `Clamped` type currently only works when wrapping the `&[u8]`, `&mut
[u8]`, and `Vec<u8>` types. Everything else will produce an error at
`wasm-bindgen` time.
Closes#421
All these functions are now provided by upstream compiler-builtins, so
there's no need for us to be binding them automatically. The remaining
`Math_*` functions are also no longer needed on nightly after
https://github.com/rust-lang/rust/pull/54257 but that PR isn't on beta,
so we'll need to leave these here for awhile while beta rides the trains
This commit removes the need for an injected `ConstructorToken` type and
also cleans up the story we have for generating constructors a bit.
After this commit a `constructor()` is omitted entirely if we're in
non-debug mode and there's no actual listed constructor. Additionally we
don't deal with splat arguments and rerouting constructors, Nick was
kind enough to enlighten me about `Object.create` which is creating an
instance without running the constructor!
Instances of an exported type are now created through one of two
methods:
* If `#[wasm_bindgen(constructor)]` is present, then a `constructor` is
generated with the appropriate signature. If a constructor is not
present and we're in debug mode, a throwing constructor is generated.
If we're in release mode and there's no constructor, no constructor is
generated.
* Otherwise if a binding returns an instance of a type (or otherwise
needs to manfuacture an instance, then it will cause an internal
`__wrap` function to be generated. This function will use
`Object.create` to create an instance without running the constructor.
This should ideally clean up our generated JS for classes quite a bit,
making it much more lean-and-mean!
Previously `wasm-bindgen` would take its `breaks_if_inlined` shims and
attempt to remove them entirely, replacing calls to `breaks_if_inlined`
to the imported closure factories. This worked great in that it would
remove the `breaks_if_inlined` funtion entirely, removing the "cost" of
the `#[inline(never)]`.
Unfortunately as #864 discovered this is "too clever by half". LLVM's
aggressive optimizations won't inline `breaks_if_inlined`, but it may
still change the ABI! We can't replace calls to `breaks_if_inlined` if
the signature changes, because the function its calling has a fixed signature.
This commit cops out a bit and instead of replacing calls to
`breaks_if_inlined` to the imported closure factories, we instead
rewrite calls to `__wbindgen_describe_closure` to the closure factories.
This means that the `breaks_if_inlined` shims do not get removed. It
also means that the closure factory shims have a third and final
argument (what would be the function pointer of the descriptor function)
which is dead and unused.
This should be a functional solution for now and let us iterate on a
true fix later on (if needed). For now the cost of this
`#[inline(never)]` and the extra unused argument should be quite small.
Closes#864
This commit adds support for exporting a function defined in Rust that returns a
`Result`, translating the `Ok` variant to the actual return value and the `Err`
variant to an exception that's thrown in JS.
The support for return types and descriptors was rejiggered a bit to be a bit
more abstract and more well suited for this purpose. We no longer distinguish
between functions with a return value and those without a return value.
Additionally a new trait, `ReturnWasmAbi`, is used for converting return values.
This trait is an internal implementation detail, however, and shouldn't surface
itself to users much (if at all).
Closes#841
This is intended to address #834 where we don't actually want methods scoped
like this! Instead we'll provide one unique accessor for the `window` object
itself.
This resulted in trailing whitespace in the generated file. In addition
to wasting space in a file that gets served over the wire, this also
gets highlighted as a problem when reviewing the generated file in an
editor that highlights trailing whitespace.
The output using modules already uses string formatting that carefully
avoids emitting leading and trailing blanks; adjust the --no-modules
output to match.
This commit adds an implementation of `AsRef<JsValue>` for the `Closure<T>`
type. Previously this was not possible because the `JsValue` didn't actually
exist until the closure was passed to JS, but the implementation has been
changed to ... something a bit more unconventional. The end result, however, is
that `Closure<T>` now always contains a `JsValue`.
The end result of this work is intended to be a precursor to binding callbacks
in `web-sys` as `JsValue` everywhere but still allowing usage with `Closure<T>`.
This commit adds further support for the `Global` attribute to not only emit
structural accessors but also emit functions that don't take `&self`. All
methods on a `[Global]` interface will not require `&self` and will call
functions and/or access properties on the global scope.
This should enable things like:
Window::location() // returns `Location`
Window::fetch(...) // invokes the `fetch` function
Closes#659
We currently pass a raw view into wasm's memory for `getStringFromWasm`, but if
the memory is actually shared then `TextDecoder` rejects `SharedArrayBuffer` and
won't actually decode anything. Work around this for now with an extra copy into
a local buffer, and then pass that buffer to `getStringFromWasm` whenever memory
is shared.
In addition to closing #495 this'll be useful eventually when instantiating
multiple wasm modules from Rust as you'd now be able to acquire a reference to
the current module in Rust itself.
The default of Rust wasm binaries is to export the memory that they contain, but
LLD also supports an `--import-memory` option where memory is imported into a
module instead. It's looking like importing memory is along the lines of how
shared memory wasm modules will work (they'll all import the same memory).
This commit adds support to wasm-bindgen to support modules which import memory.
Memory accessors are tweaked to no longer always assume that the wasm module
exports its memory. Additionally JS bindings will create a `memory` option
automatically because LLD always imports memory from an `env` module which won't
actually exist.
This is a pretty heavyweight dependency which accounts for a surprising amount
of runtime for larger modules in `wasm-bindgen`. We don't need 90% of the crate
and so this commit bundles a small interpreter for instructions we know are only
going to appear in describe-related functions.
This commit adds experimental support for `WeakRef` to be used to automatically
free wasm objects instead of having to always call the `free` function manually.
Note that when enabled the `free` function for all exported objects is still
generated, it's just optionally invoked by the application.
Support isn't exposed through a CLI flag right now due to the early stages of
the `WeakRef` proposal, but the env var `WASM_BINDGEN_WEAKREF` can be used to
enable this generation. Upon doing so the output can then be edited slightly as
well to work in the SpiderMonkey shell and it looks like this is working!
Closes#704
This commit implements the `JsCast` trait automatically for all imported types
in `#[wasm_bindgen] extern { ... }` blocks. The main change here was to generate
an `instanceof` shim for all imported types in case it's needed.
All imported types now also implement `AsRef<JsValue>` and `AsMut<JsValue>`
First added in #161 this never ended up panning out, so let's remove the
experimental suport which isn't actually used by anything today and hold off on
any other changes until an RFC happens.
* Tweak the implementation of heap closures
This commit updates the implementation of the `Closure` type to internally store
an `Rc` and be suitable for dropping a `Closure` during the execution of the
closure. This is currently needed for promises but may be generally useful as
well!
* Support asynchronous tests
This commit adds support for executing tests asynchronously. This is modeled
by tests returning a `Future` instead of simply executing inline, and is
signified with `#[wasm_bindgen_test(async)]`.
Support for this is added through a new `wasm-bindgen-futures` crate which is a
binding between the `futures` crate and JS `Promise` objects.
Lots more details can be found in the details of the commit, but one of the end
results is that the `web-sys` tests are now entirely contained in the same test
suite and don't need `npm install` to be run to execute them!
* Review tweaks
* Add some bindings for `Function.call` to `js_sys`
Name them `call0`, `call1`, `call2`, ... for the number of arguments being
passed.
* Use oneshots channels with `JsFuture`
It did indeed clean up the implementation!
This commit moves the `webidl/tests` folder to a new `crates/webidl-tests` crate
(to have a test-only build script) and ports them to the `#[wasm_bindgen_test]`
attribute, which should hopefully make testing much speedier for execution!
* Fix importing the same identifier from two modules
This needed a fix in two locations:
* First the generated descriptor function needed its hash to include the module
that the import came from in order to generate unique descriptor functions.
* Second the generation of the JS shim needed to handle duplicate identifiers in
a more uniform fashion, ensuring that imported names didn't clash.
* Fix importing the same name in two modules
Previously two descriptor functions with duplicate symbols were emitted, and now
only one function is emitted by using a global table to keep track of state
across macro invocations.
Currently it generates a lot of shim functions which delegate to the wasm module
when loaded, but it turns out with `export let` we can just update the bindings!
Instead of exporting a bunch of shims this updates the export functionality to
only update the `export let` directives with the direct values from the wasm
module once the module is done loading.
In addition to being more ergonomic these are much more efficient at reading
large files as they preallocate internally. This provides a nice speed boost
locally, reducing the overhead of `wasm-bindgen-test-runner` from 0.23s to
0.19s, yay!
This commit updates the test runner to only deserialize a `Module` once and then
directly pass it to the `wasm-bindgen` config, avoiding pulling in a public
dependency with the same strategy as the `wasm-gc-api` crate for now.
This reduces the runtime of this step for `wasm-bindgen-test-runner` from ~0.23s
to ~0.19s on my machine.
Since `wasmi` already has a public dependency on `parity_wasm` let's just use
it! A `clone` is much faster than a serialize + parse, reducing a `wasm-bindgen`
invocation on my machine from 0.2s to 0.18s.
Currently the `wasm-gc-api` crate doesn't expose `parity_wasm::Module` as a
public dependency which means that whenever we want to run a GC (which is twice
per `wasm-bindgen` invocation) we have to serialize and reparse the module a
lot! The `wasm-bindgen` has to serialize, `wasm-gc` then parses, `wasm-gc` then
serializes, and `wasm-bindgen` then parses.
This commit sidesteps all of these operations by ensuring that we always use the
same `parity_wasm::Module` instance, even when multiple versions of the
`parity_wasm` crate are in use. We'll get a speed boost when they happen to
align (which they always should for `wasm-bindgen`), but it'll work even if they
aren't aligned (by going through serialization).
Concretely on my machine this takes a `wasm-bindgen` invocation from 0.5s to
0.2s, a nice win!
This is a bit of a refinement of the solution from #548 to make sure that these
statics are only present on the `wasm32-*` targets, as otherwise these
descriptors are completely inert on other platforms!
* Add a test harness to directly execute wasm tests
This commits adds a few new crates and infrastructure to enable comands like:
cargo test --target wasm32-unknown-unknown
The intention here is to make it as low-friction as possible to write wasm tests
and also have them execute in a reasonable amount of time. Eventually this is
also hopefully enough support to do things like headless testing!
For now though this is defintely MVP status rather than fully fleshed out.
There's some more information at `crates/test/README.md` about how it works and
how to use it, but for now this is mainly intended to play around with locally
in this repository for our own tests.
* Port a numbe of `js-sys` tests to the new test framework
This commit ports a number of existing tests for the `js-sys` crate over to the
new test framework created in the previous commit, showing off how they can be
executed as well as drastictlly simplifying the tests themselves! This is
intended to be a proof of concept for now which we can refine over time. This
should also show off that it's possible to incrementally move over to the new
test framework.
* Bump to 0.2.12
* Update all version numbers and deps
* Update all listed authors to `["The wasm-bindgen Developers"]`
* Update `repository` links to specific paths for each crate
* Update `homepage` links to the online book
* Update all links away from `alexcrichton/wasm-bindgen`
* Add `#[doc]` directives for HTML URLs
* Update more version requirements
* Fill out CHANGELOG
* Shard the `convert.rs` module into sub-modules
Hopefully this'll make the organization a little nicer over time!
* Start adding support for optional types
This commit starts adding support for optional types to wasm-bindgen as
arguments/return values to functions. The strategy here is to add two new
traits, `OptionIntoWasmAbi` and `OptionFromWasmAbi`. These two traits are used
as a blanket impl to implement `IntoWasmAbi` and `FromWasmAbi` for `Option<T>`.
Some consequences of this design:
* It should be possible to ensure `Option<SomeForeignType>` implements to/from
wasm traits. This is because the option-based traits can be implemented for
foreign types.
* A specialized implementation is possible for all types, so there's no need for
`Option<T>` to introduce unnecessary overhead.
* Two new traits is a bit unforutnate but I can't currently think of an
alternative design that works for the above two constraints, although it
doesn't mean one doesn't exist!
* The error messages for "can't use this type here" is actually halfway decent
because it says these new traits need to be implemented, which provides a good
place to document and talk about what's going on here!
* Nested references like `Option<&T>` can't implement `FromWasmAbi`. This means
that you can't define a function in Rust which takes `Option<&str>`. It may be
possible to do this one day but it'll likely require more trait trickery than
I'm capable of right now.
* Add support for optional slices
This commit adds support for optional slice types, things like strings and
arrays. The null representation of these has a pointer value of 0, which should
never happen in normal Rust. Otherwise the various plumbing is done throughout
the tooling to enable these types in all locations.
* Fix `takeObject` on global sentinels
These don't have a reference count as they're always expected to work, so avoid
actually dropping a reference on them.
* Remove some no longer needed bindings
* Add support for optional anyref types
This commit adds support for optional imported class types. Each type imported
with `#[wasm_bindgen]` automatically implements the relevant traits and now
supports `Option<Foo>` in various argument/return positions.
* Fix building without the `std` feature
* Actually fix the build...
* Add support for optional types to WebIDL
Closes#502
This commit adds a hack to the `wasm-bindgen` CLI tool to work around #483 which
is present on nightly Rust with the recent LLVM upgrade. Hopefully this'll carry
us forward until the [upstream bug][1] is fixed.
Closes#483
[1]: https://bugs.llvm.org/show_bug.cgi?id=38184
The changes on master Rust insert debug sections now (yay!) but this means that
wasm binaries by default pick up debug sections from the standard library, so
let's remove them by default in wasm-bindgen unless `--debug` is passed
* Create a new `web-sys` crate
This will eventually contain all the WebIDL-generated bindings to Web APIs.
* ci: Test the new `web-sys` crate in CI
* web-sys: Add a small README
* web-sys: Vendor all the WebIDL files from mozilla-central
* backend: Add a pass to remove AST items that use undefined imports
This is necessary for the WebIDL frontend, which can't translate many WebIDL
constructs into equivalent wasm-bindgen AST things yet. It lets us make
incremental progress: we can generate bindings to methods we can support right
now even though there might be methods on the same interface that we can't
support yet.
* webidl: Add a bunch of missing semicolons
* webidl: Make parsing private
It was only `pub` so that we could test it, but we ended up moving towards
integration tests rather than unit tests that assert particular ASTs are parsed
from WebIDL files.
* webidl: Remove uses of undefined import types
* test-project-builder: Build projects in "very verbose" mode
This helps for debugging failing WebIDL-related tests.
* test-project-builder: Add more profiling timers
* test-project-builder: Detect when webpack-dev-server fails
Instead of going into an infinite loop, detect when webpack-dev-server fails to
start up and early exit the test.
* webidl: Specify version for dev-dependency on wasm-bindgen-backend
Instead of only a relative path.
* guide: Add section about contributing to `web-sys`
* WIP enable Event.webidl
Still need to fix and finish the test.
* Update expected webidl output
* Start out a test's status as incomplete
That way if we don't fill it in the error message doesn't look quite so bizarre
* Fix onerror function in headless mode
Otherwise we don't see any output!
* Fix package.json/node_modules handling in project generation
Make sure these are looked up in the git project root rather than the crate root
* Avoid logging body text
This was meant for debugging and is otherwise pretty noisy
* Fix a relative path
* More expected test fixes
* Fix a typo
* test-project-builder: Allow asynchronous tests
* webidl: Convert [Unforgeable] attributes into `#[wasm_bindgen(structural)]`
Fixes#432
* test-project-builder: Print generated WebIDL bindings for debugging purposes
Helps debug bad WebIDL bindings generation inside tests.
* When we can't find a descriptor, say which one can't be found
This helps when debugging things that need to become structural.
* web-sys: Test bindings for Event
* ci: Use `--manifest-path dir` instead of `cd dir && ...`
* web-sys: Just move .webidl files isntead of symlinking to enable them
* tests: Polyfill Array.prototype.values for older browsers in CI
* test-project-builder: Don't panic on poisoned headless test mutex
We only use it to serialize headless tests so that we don't try to bind the port
concurrently. Its OK to run another headless test if an earlier one panicked.
* JsValue: Add {is,as}_{object,function} methods
Allows dynamically casting values to `js::Object` and `js::Function`.
* tidy: Fix whitespace and missing semicolons
* Allow for dynamic feature detection of methods
If we create bindings to a method that doesn't exist in this implementation,
then it shouldn't fail until if/when we actually try and invoke that missing
method.
* tests: Do feature detection in Array.prototype.values test
* Add JsValue::{is_string, as_js_string} methods
And document all the cast/convert/check methods for js value.
* eslint: allow backtick string literals
* Only generate a fallback import function for non-structural imports
* remove BindgenAttrs from other backend::ast structs
This is primarily a tool for use with the macro crate. Most of
these attributes were ignored in the actual codegen, but a few
were still being used. This is confusing when trying to add
other sources for codegen (such as webidl and typescript).
* move parsing logic to macro crate
This makes the backend crate solely concerned with having an ast
for which we can generate code.
* Reorganize Travis configuration
* Add a `JOB` env var descriptor to all matrix entries. Not used anywhere but is
useful when viewing the whole build on Travis's web interface.
* Reorganize where builds are located, moving slow builds first and fast ones
last.
* Change checking the CLI builds from `cargo build` to `cargo check`
* Use YAML references to reduce some duplication
* Print some more timing statistics for each test
* Extract `Project` helper in tests to a module
This'll help make it a bit more extensible over time. At the same time the
methods are also slightly reorganized to read more clearly from top to bottom.
* Migrate all tests away from Webpack
Wepback can take a significant amount of time to execute and when it's
multiplied by hundreds of tests that adds up really quickly! After investigating
Node's `--experimental-modules` option it looks like it's suitable for our use
so this switches all tests to using JS files (moving away from TypeScript as
well) with `--experimental-modules` with Node.
Tests will be selectively re-enabled with webpack and node.js specific output
(that doesn't require `--experimental-modules`), coming in later commits.
* Restore the node test for node.js output
Ensures it's workable as-is
* Only generate typescript with webpack
* Only read wasm files for webpack
* Skip package.json/node_modules for now
* Only generate webpack config if needed
* Start a dedicated test module for typescript
Will hopefully verify the generated Typescript compiles OK.
* Remove unneeded `node` method
* Fixup some rebase conflicts
* Don't run asmjs example on travis
* Fixup generator tests
* Attempt to fix windows
* Comment windows fix
* More test fixes
* More exclusions
* More test fixes
* Relax eslint regex
Catch mjs modules as well
* Fix eslint
* Speed up travis on examples slightly
Travis tests show hundreds of warning for `'y' is defined but never used` and
when investigating it looks like a mistake was introduced in 0938858aa
during #272, so hopefully this'll be an easy fix!
If a JS import's shim isn't actually imported that means that somewhere along
the way it was optimized out or it was never used in the first place! In that
case we can skip generation of the JS bindings for it as it's not needed.
This can happen when a nested dependency crate exports things but the root crate
doesn't use them. In these cases, it is fine to ignore the missing descriptor,
because the thing it describes was removed as dead code.
* backend comments complete
* better matching
* gen comments
* Add example
* Move test bindings gen to own fn
* move build step into build fn
* add fn to read js, refactor gen_bindings/test to allow for this
* Add comments test
* Update readmes
* add comments to travis
* fix broken tests
* +x on build.sh
* fix wbg cmd in build.sh
* Address fitzgen's comments
This commit optimizes constructing an instance of `JsValue` which is one of
`null`, `undefined`, `true`, or `false`. These are commonly created on the Rust
side of things and since there's only a limited set of values we can easily
prepopulate the global slab with a few entries and use hardcoded indices to
refer to these constants. This should avoid the need to travel into JS to insert
a `null` or and `undefined` into the global slab.
Awhile back slices switched to being raw views into wasm memory, but this
doens't work if we free the underlying memory unconditionally! Moving around a
`Vec` is already moving a lot of data, so let's copy it onto the JS heap instead
of leaving it in the wasm heap.
This commit is an implementation of mapping u64/i64 to `BigInt` in JS through
the unstable BigInt APIs. The BigInt type will ship soon in Chrome and so this
commit builds out the necessary support for wasm-bindgen to use it!
When adding support for mutable slices I was under the impression that if the
wasm memory was reallocated while we were using it then we'd have to commit the
changes from the original buffer back to the new buffer. What I didn't know,
however, is that once the wasm memory is reallocated then all views into it are
supposed to be defunkt.
It looks like node 9 didn't have this implementation quite right and it appears
fixed in node 10, causing the deleted test here to fail. While this commit does
raise the question of whether this is the right approach to interact with slices
in JS I think the answer is still "yes". The user can always initiate the copy
if need be and that seems strictly better than copying 100% of the time.
This commit updates the `Abi` associated type for all slice types to a
`WasmSlice` type, an aggregate of two `u32` integers. This translates to an ABI
where when passed as a function argument it expands to two integer arguments,
and when passed as a return value it passes a return pointer as the first
argument to get filled in.
This is hopefully more forwards-compatible with the host bindings proposal which
uses this strategy for passing string arguments at least. It's a little sketchy
what we're doing as there's not really a stable ABI yet, but hopefully this'll
all be relatively stable for awhile!
This commit adds support for mutable slices to pass the boundary between JS and
Rust. While mutable slices cannot be used as return values they can be listed as
arguments to both exported functions as well as imported functions.
When passing a mutable slice into a Rust function (aka having it as an argument
to an exported Rust function) then like before with a normal slice it's copied
into the wasm memory. Afterwards, however, the updates in the wasm memory will
be reflected back into the original slice. This does require a lot of copying
and probably isn't the most efficient, but it should at least work for the time
being.
The real nifty part happens when Rust passes a mutable slice out to JS. When
doing this it's a very cheap operation that just gets a subarray of the main
wasm memory. Now the wasm memory's buffer can change over time which can produce
surprising results where memory is modified in JS but it may not be reflected
back into Rust. To accomodate this when a JS imported function returns any
updates to the buffer are copied back to Rust if Rust's memory buffer has
changed in the meantime.
Along the way this fixes usage of `slice` to instead use `subarray` as that's
what we really want, no copying. All methods have been updated to use `subarray`
accessors instead of `slice` or constructing new arrays.
Closes#53
This commit adds an example of executing the `wasm2asm` tool to generate asm.js
output instead of WebAssembly. This is often useful when supporting older
browsers, such as IE 11, that doesn't have native support for WebAssembly.
These functions are activated with the `serde-serialization` feature of the
`wasm-bindgen` crate. When activated they will allow passing any arbitrary value
into JS that implements the `Serialize` trait and receiving any value from JS
using the `Deserialize` trait. The interchange between JS and Rust is JSON.
Closes#96
This commit adds a `#[wasm_bindgen(version = "...")]` attribute support. This
information is eventually written into a `__wasm_pack_unstable` section.
Currently this is a strawman for the proposal in ashleygwilliams/wasm-pack#101
Turns out there was a bug when passing a vector of `JsValue` instances back to
JS all objects were leaked rather than correctly removed from the global slab.
This commit fixes how the `getArrayJsValueFromWasm` function is defined to
correctly iterate over the slice by looking at the values rather than the
indices.
Closes#169
Currently errors are reported via Rust panics but there's lots more errors being
added over time so this commit starts the movement towards the `failure` crate
to more idiomatically report errors as well as provide better error messages
over time.
Add support for `#[wasm_bindgen(readonly)]` which indicates that an exported
struct field is readonly and attempting to set it in JS will throw an exception.
Closes#151
Automatically infer public struct fields as "JS wants to access this" and
generate appropriate getters/setters for the field. At this time the field is
required to implement `Copy`, but we will probably want to relax that in the
future to at least encompass `JsValue` and maybe other `Clone` values as well.
Closes#121
Currently the entire `Program` is deserialized to match schema versions but this
is likely to fail when the schema changes. Instead just deserialize the
schema/version fields, compare those, and if successful go ahead and deserialize
everything.
This commit adds support for both `#![no_std]` in the wasm-bindgen runtime
support (disabled by default with an on-by-default `std` feature). This also
adds support to work and compile in the context of `#![no_std]` crates.
Closes#146
This commit adds support for closures with arguments like strings and such. In
other words, closures passed to JS can now have the same suite of arguments as
all functions that can be exported from Rust, as one might expect!
At this time due to the way trait objects work closures still cannot use types
with references like `&str`, but bare values like `String` or `ImportedType`
should work just fine.
Closes#104
This commit overhauls the conversion traits used for types crossing the Rust/JS
boundary. Previously there were a few ad-hoc traits but now there've been
slightly reduced and decoupled.
Conversion from Rust values to JS values is now exclusively done through
`IntoWasmAbi` with no special treatment for references. Conversion from JS to
Rust is a bit trickier as we want to create references in Rust which have
implications in terms of safety. As a result there are now three traits for
this, `FromWasmAbi`, `RefFromWasmAbi`, and `RefMutFromWasmAbi`. These three
traits are implemented for various types and specially dispatched to depending
on the type of argument in the code generator.
The goal of this commit is to lay the groundwork for using these traits in
closures with straightforward-ish definitions.
This commit adds support for passing `&mut FnMut(..)` to JS via imports. These
closures cannot be invoked recursively in JS (they invalidate themselves while
they're being invoked) and otherwise work the same as `&Fn(..)` closures.
Closes#123
This commit is a complete overhaul of how the `#[wasm_bindgen]` macro
communicates type information to the CLI tool, and it's done in a somewhat...
unconventional fashion.
Today we've got a problem where the generated JS needs to understand the types
of each function exported or imported. This understanding is what enables it to
generate the appropriate JS wrappers and such. We want to, however, be quite
flexible and extensible in types that are supported across the boundary, which
means that internally we rely on the trait system to resolve what's what.
Communicating the type information historically was done by creating a four byte
"descriptor" and using associated type projections to communicate that to the
CLI tool. Unfortunately four bytes isn't a lot of space to cram information like
arguments to a generic function, tuple types, etc. In general this just wasn't
flexible enough and the way custom references were treated was also already a
bit of a hack.
This commit takes a radical step of creating a **descriptor function** for each
function imported/exported. The really crazy part is that the `wasm-bindgen` CLI
tool now embeds a wasm interpreter and executes these functions when the CLI
tool is invoked. By allowing arbitrary functions to get executed it's now *much*
easier to inform `wasm-bindgen` about complicated structures of types. Rest
assured though that all these descriptor functions are automatically unexported
and gc'd away, so this should not have any impact on binary sizes
A new internal trait, `WasmDescribe`, is added to represent a description of all
types, sort of like a serialization of the structure of a type that
`wasm-bindgen` can understand. This works by calling a special exported function
with a `u32` value a bunch of times. This means that when we run a descriptor we
effectively get a `Vec<u32>` in the `wasm-bindgen` CLI tool. This list of
integers can then be parsed into a rich `enum` for the JS generation to work
with.
This commit currently only retains feature parity with the previous
implementation. I hope to soon solve issues like #123, #104, and #111 with this
support.
* Have the global `wasm_bindgen` variable be a function which runs
initialization rather than exporting an `init` function.
* Save off the wasm object on `wasm_bindgen.wasm` so the memory can be accessed
* Tidy up the code slightly
Node's fs APIs resolve relative paths relative to the current working directory:
https://nodejs.org/api/fs.html#fs_file_paths
This creates a problem if you try to require the wasm-bindgen-generated
JavaScript from a different directory. For example, if you have
build/foo.js
build/foo_bg.js
build/foo_bg.wasm
and another script, script/index.js, that requires build/foo.js. We can instead
use __dirname to get the correct path to the file.
As soon as we've removed unneeded exports immediately run a gc pass to ensure
that we don't bind functions in JS that don't actually end up getting needed.
This commit starts wasm-bindgen down the path of supporting closures. We
discussed this at the recent Rust All-Hands but I ended up needing to pretty
significantly scale back the ambitions of what closures are supported. This
commit is just the initial support and provides only a small amount of support
but will hopefully provide a good basis for future implementations.
Specifically this commit adds support for passing `&Fn(...)` to an *imported
function*, but nothing elese. The `&Fn` type can have any lifetime and the JS
object is invalidated as soon as the import returns. The arguments and return
value of `Fn` must currently implement the `WasmAbi` trait, aka they can't
require any conversions like strings/types/etc.
I'd like to soon expand this to `&mut FnMut` as well as `'static` closures that
can be passed around for a long time in JS, but for now I'm putting that off
until later. I'm not currently sure how to implement richer argument types, but
hopefully that can be figured out at some point!
