zellij/.github/workflows/e2e.yml
har7an 11b0210de5
plugins: rework plugin loading (#1924)
* zellij: Move "populate_data_dir" to utils

and rewrite it to take a second, optional parameter. This allows
controlling whether only a specific asset should be installed. We do
this as preparation for being able to recover from a plugin version
mismatch error, where we will need to repopulate the data dir for
offending plugins.

* server/wasm_vm: Recover from PluginVersionMismatch

Adds a global plugin cache that stores, per plugin config, the wasmer
module associated with it. Make `start_plugin` take the pre-populated
module and create only the necessary modifications to the wasm env etc.

* utils: Fix formatting

* zellij: Delete non-existent module

* utils/shared: fix missing "set_permissions"

function when not on unix systems.

* server/wasm_vm: Don't populate cachedir

with serialized versions of the WASM plugins.

* utils/input/plugins: load wasm bytes from assets

for builtin plugin specifications. This foregoes any need to:

- Dump the plugin bytes to disk at all and
- subsequently read the plugin bytes from disk

* zellij: Disable default asset installation

which previously installed only the builtin plugins to disk. This is no
longer necessary because now we can load the builtin plugins directly
from the application binary.

* utils/input/plugins: Update docs

* utils/input/plugins: Add 'is_builtin' method

to `PluginConfig` that returns true if the plugin configuration refers
to a builtin plugin.

* wasm_vm: Remove plugin version mismatch handling

because a version mismatch in an internal plugin is now unfixable, with
the plugins being loaded from the running binary, and we have no control
over external plugins in the first place.

* cargo: Reintroduce feature flag

for `disable_automatic_asset_installation`

* utils/consts: Add `ASSET_MAP`

which currently contains the compiled WASM plugins.

* utils/shared: Fix clippy lint

* utils/errors: Add more `ZellijError` variants

* zellij: Make loading internal plugins optional

by reenabling the `disable_automatic_asset_installation` flag and
utilizing it for this purpose. Changes plugin search behavior to throw
better errors in case the builtin plugins cannot be found, depending on
the state of this feature.

* utils/errors: Apply rustfmt

* utils/setup: Allow dumping builtin plugins

to a specified folder on disk. This is meant to be an "escape hatch" for
users that have accidentally deleted the builtin plugins from disk (in
cases where the plugins aren't loaded from inside the zellij binary).

* utils/input/plugins: Update docs

* utils/setup: Add hint to `setup --check` output

when zellij was built without the `disable_automatic_asset_installation`
flag and will thus not read builtin plugins from the "PLUGIN DIR".

* utils/setup: Refine `setup --dump-plugins`

to dump to:

- The default "DATA DIR" when no option is provided with the argument,
  or
- The provided option, if existent

Also print a message to stdout with the destination folder that the
plugins are dumped to.

* server/wasm_vm: Ignore "NotFound" errors

when attempting to delete the non-existent plugin data directories. This
silences an error message that otherwise ends up in the logs when
quitting zellij.

* utils/errors: Extend "BuiltinPluginMissing" msg

to hint the user to the `zellij setup --dump-plugins` command to fix
their issues for them!

* utils/errors: Track caller in calls to `non_fatal`

which will hopefully, once closures can be annotated, allow us to
display the location of the call to `non_fatal` in log messages.

* utils/input/plugins: Fix plugin lookup

to prefer internal assets if available. It was previously broken because
sorting the paths vector before deduping it would bring the paths into a
wrong order, looking up in the plugin folder first.

Also print a log message when a plugin is being loaded from the internal
assets but exists on disk, too.

* Apply rustfmt

* make: build-e2e depends on wasm-opt-plugins

so it updates the assets when building the binary

* server/qwasm_vm: Remove var

* utils/consts: Add plugins from target folder

and include them in the asset map from there, too. Include plugins from
debug or release builds, depending on the build type.

* utils/consts: Take release plugins from assets

instead of the target/release folder. The latter will break
installations from crates.io, because we currently rely on including the
plugins we pre-compiled and distribute along with the binary.

* server/wasm_vm: Reintroduce .cache folder

to speedup subsequent application launches.

* cargo: Reorder workspace members

to improve behavior with `cargo make` with respect to compilation order.

* Makefile: restructure plugin tasks

* Makefile: Fix CI errors

* Makefile: More CI diagnosis

* github: Install wasm-opt in e2e test workflow

* Makefile: Build plugins for e2e-test target

* server/Wasm_vm: Reorder plugin folder creation

so no folders are created in the plugin cache when loading a plugin
fails due to not being present or similar.

* update plugins testcommit

* makefile: Change job order

* changelog: Add PR #1924
2022-11-22 20:06:02 +00:00

64 lines
2.0 KiB
YAML

name: End to End tests
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
env:
CARGO_TERM_COLOR: always
jobs:
test:
name: Build generic binary and run tests on it
runs-on: ubuntu-latest
environment: cachix
services:
ssh:
image: ghcr.io/linuxserver/openssh-server
env:
PUID: 1001
PGID: 1000
TZ: Europe/Vienna
PASSWORD_ACCESS: true
USER_PASSWORD: test
USER_NAME: test
ports:
- 2222:2222
options: -v ${{ github.workspace }}/target:/usr/src/zellij --name ssh
steps:
- uses: actions/checkout@v3
- uses: cachix/install-nix-action@v18
- uses: cachix/cachix-action@v12
with:
name: zellij
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Add WASM target
run: rustup target add wasm32-wasi
- name: Install musl-tools
run: sudo apt-get install -y --no-install-recommends musl-tools
- name: Add musl target
run: rustup target add x86_64-unknown-linux-musl
- name: Install cargo-make
run: nix profile install nixpkgs#cargo-make
- name: Install wasm-opt
run: sudo apt-get install -y --no-install-recommends binaryen
#run: cargo install --debug cargo-make
- name: Build asset
run: cargo make build-e2e
# we copy this manually into the target folder instead of mounting it because
# github actions creates the service first, and if it has a mount that is part
# of your yet unchecked out code, you cannot checkout the code after the mount
- name: Copy fixtures folder to target
run: cp -r ${{ github.workspace }}/src/tests/fixtures ${{ github.workspace }}/target
- name: Restart ssh container
# we need to do this because otherwise the volume will not be mounted
# on the docker container, since it was created before the folder existed
uses: docker://docker
with:
args: docker restart ssh
- name: Test
run: cargo make e2e-test