zellij-org/zellij
1
1
Fork 0
mirror of https://github.com/zellij-org/zellij.git synced 2024-12-23 09:11:40 +03:00
Code Issues Projects Releases Wiki Activity

plugins: rework plugin loading (#1924)

Browse Source 
* zellij: Move "populate_data_dir" to utils

and rewrite it to take a second, optional parameter. This allows
controlling whether only a specific asset should be installed. We do
this as preparation for being able to recover from a plugin version
mismatch error, where we will need to repopulate the data dir for
offending plugins.

* server/wasm_vm: Recover from PluginVersionMismatch

Adds a global plugin cache that stores, per plugin config, the wasmer
module associated with it. Make `start_plugin` take the pre-populated
module and create only the necessary modifications to the wasm env etc.

* utils: Fix formatting

* zellij: Delete non-existent module

* utils/shared: fix missing "set_permissions"

function when not on unix systems.

* server/wasm_vm: Don't populate cachedir

with serialized versions of the WASM plugins.

* utils/input/plugins: load wasm bytes from assets

for builtin plugin specifications. This foregoes any need to:

- Dump the plugin bytes to disk at all and
- subsequently read the plugin bytes from disk

* zellij: Disable default asset installation

which previously installed only the builtin plugins to disk. This is no
longer necessary because now we can load the builtin plugins directly
from the application binary.

* utils/input/plugins: Update docs

* utils/input/plugins: Add 'is_builtin' method

to `PluginConfig` that returns true if the plugin configuration refers
to a builtin plugin.

* wasm_vm: Remove plugin version mismatch handling

because a version mismatch in an internal plugin is now unfixable, with
the plugins being loaded from the running binary, and we have no control
over external plugins in the first place.

* cargo: Reintroduce feature flag

for `disable_automatic_asset_installation`

* utils/consts: Add `ASSET_MAP`

which currently contains the compiled WASM plugins.

* utils/shared: Fix clippy lint

* utils/errors: Add more `ZellijError` variants

* zellij: Make loading internal plugins optional

by reenabling the `disable_automatic_asset_installation` flag and
utilizing it for this purpose. Changes plugin search behavior to throw
better errors in case the builtin plugins cannot be found, depending on
the state of this feature.

* utils/errors: Apply rustfmt

* utils/setup: Allow dumping builtin plugins

to a specified folder on disk. This is meant to be an "escape hatch" for
users that have accidentally deleted the builtin plugins from disk (in
cases where the plugins aren't loaded from inside the zellij binary).

* utils/input/plugins: Update docs

* utils/setup: Add hint to `setup --check` output

when zellij was built without the `disable_automatic_asset_installation`
flag and will thus not read builtin plugins from the "PLUGIN DIR".

* utils/setup: Refine `setup --dump-plugins`

to dump to:

- The default "DATA DIR" when no option is provided with the argument,
  or
- The provided option, if existent

Also print a message to stdout with the destination folder that the
plugins are dumped to.

* server/wasm_vm: Ignore "NotFound" errors

when attempting to delete the non-existent plugin data directories. This
silences an error message that otherwise ends up in the logs when
quitting zellij.

* utils/errors: Extend "BuiltinPluginMissing" msg

to hint the user to the `zellij setup --dump-plugins` command to fix
their issues for them!

* utils/errors: Track caller in calls to `non_fatal`

which will hopefully, once closures can be annotated, allow us to
display the location of the call to `non_fatal` in log messages.

* utils/input/plugins: Fix plugin lookup

to prefer internal assets if available. It was previously broken because
sorting the paths vector before deduping it would bring the paths into a
wrong order, looking up in the plugin folder first.

Also print a log message when a plugin is being loaded from the internal
assets but exists on disk, too.

* Apply rustfmt

* make: build-e2e depends on wasm-opt-plugins

so it updates the assets when building the binary

* server/qwasm_vm: Remove var

* utils/consts: Add plugins from target folder

and include them in the asset map from there, too. Include plugins from
debug or release builds, depending on the build type.

* utils/consts: Take release plugins from assets

instead of the target/release folder. The latter will break
installations from crates.io, because we currently rely on including the
plugins we pre-compiled and distribute along with the binary.

* server/wasm_vm: Reintroduce .cache folder

to speedup subsequent application launches.

* cargo: Reorder workspace members

to improve behavior with `cargo make` with respect to compilation order.

* Makefile: restructure plugin tasks

* Makefile: Fix CI errors

* Makefile: More CI diagnosis

* github: Install wasm-opt in e2e test workflow

* Makefile: Build plugins for e2e-test target

* server/Wasm_vm: Reorder plugin folder creation

so no folders are created in the plugin cache when loading a plugin
fails due to not being present or similar.

* update plugins testcommit

* makefile: Change job order

* changelog: Add PR #1924
This commit is contained in:
har7an 2022-11-22 20:06:02 +00:00 committed by GitHub
parent 4921fa7cae
commit 11b0210de5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
19 changed files with 512 additions and 266 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/e2e.yml vendored
View File

@ -43,6 +43,8 @@ jobs:
run: rustup target add x86_64-unknown-linux-musl
- name: Install cargo-make
run: nix profile install nixpkgs#cargo-make
- name: Install wasm-opt
run: sudo apt-get install -y --no-install-recommends binaryen
#run: cargo install --debug cargo-make
- name: Build asset
run: cargo make build-e2e

1
CHANGELOG.md
View File

@ -19,6 +19,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
* fix(themes): black and white inverted (https://github.com/zellij-org/zellij/pull/1953)
* fix(stability): gracefully handle SSH timeouts and other client buffer overflow issues (https://github.com/zellij-org/zellij/pull/1955)
* fix: empty session name (https://github.com/zellij-org/zellij/pull/1959)
* plugins: Cache plugins, don't load builtin plugins from disk (https://github.com/zellij-org/zellij/pull/1924)
## [0.33.0] - 2022-11-10

9
Cargo.toml
View File

@ -31,15 +31,15 @@ rand = "0.8.0"
[workspace]
members = [
"default-plugins/compact-bar",
"default-plugins/status-bar",
"default-plugins/strider",
"default-plugins/tab-bar",
"zellij-client",
"zellij-server",
"zellij-utils",
"zellij-tile",
"zellij-tile-utils",
"default-plugins/compact-bar",
"default-plugins/status-bar",
"default-plugins/strider",
"default-plugins/tab-bar",
".",
]
@ -68,5 +68,6 @@ bin-dir = "{ bin }{ binary-ext }"
pkg-fmt = "tgz"
[features]
# See remarks in zellij_utils/Cargo.toml
disable_automatic_asset_installation = [ "zellij-utils/disable_automatic_asset_installation" ]
unstable = [ "zellij-client/unstable", "zellij-utils/unstable" ]

97
Makefile.toml
View File

@ -9,6 +9,7 @@ ZELLIJ_ASSETS_DIR = "${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/zellij-utils/asse
# Add clippy to the default flow
[tasks.dev-test-flow]
dependencies = [
"plugins",
"format-flow",
"format-toml-conditioned-flow",
"pre-build",
@ -25,37 +26,21 @@ args = ["test", "--target", "${CARGO_HOST_TRIPLE}", "--", "@@split(CARGO_MAKE_TA
# Running Zellij using the development data directory
[tasks.run]
workspace = false
dependencies = ["build-workspace", "build-dev-data-dir"]
dependencies = ["build-workspace"]
run_task = "launch"
[tasks.build-workspace]
run_task = { name = "build", fork = true }
[tasks.build]
env = { "CARGO_MAKE_WORKSPACE_SKIP_MEMBERS" = "default-plugins*" }
args = ["build"]
[tasks.build-release]
args = ["build", "--release"]
[tasks.build-dev-data-dir]
dependencies = ["build-plugins"]
script_runner = "@duckscript"
script = '''
target_dir = set ${CARGO_TARGET_DIR}
data_dir = set ${target_dir}/dev-data
rm -r ${data_dir}
plugins = glob_array ${target_dir}/wasm32-wasi/debug/*.wasm
mkdir ${data_dir}
mkdir ${data_dir}/plugins
for plugin in ${plugins}
plugin_name = basename ${plugin}
cp ${plugin} ${data_dir}/plugins/${plugin_name}
end
writefile ${data_dir}/VERSION ${CARGO_MAKE_CRATE_VERSION}
'''
[tasks.build-e2e-data-dir]
dependencies = ["build-plugins-release"]
dependencies = ["plugins-release"]
script_runner = "@duckscript"
script = '''
target_dir = set ${CARGO_TARGET_DIR}
@ -83,6 +68,7 @@ args = [
# Simple clippy tweak
[tasks.clippy]
dependencies = ["plugins"]
args = ["clippy", "--all-targets", "--all-features", "@@split(CARGO_MAKE_TASK_ARGS,;)"]
# Release building and installing Zellij
@ -98,23 +84,40 @@ else
end
'''
[tasks.build-plugins-release]
env = { "CARGO_MAKE_WORKSPACE_INCLUDE_MEMBERS" = [
"default-plugins/compact-bar",
"default-plugins/status-bar",
"default-plugins/strider",
"default-plugins/tab-bar",
] }
run_task = { name = "build-release", fork = true }
[tasks.wasm-opt-plugins]
alias = "plugins-release"
[tasks.build-plugins]
env = { "CARGO_MAKE_WORKSPACE_INCLUDE_MEMBERS" = [
"default-plugins/compact-bar",
"default-plugins/status-bar",
"default-plugins/strider",
"default-plugins/tab-bar",
] }
run_task = { name = "build", fork = true }
[tasks.plugins-release]
workspace = false
script_runner = "@duckscript"
script = '''
plugins = glob_array ${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/default-plugins/*
out_dir = set ${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/assets/plugins/
mkdir ${out_dir}
for plugin in ${plugins}
cd ${plugin}
exec cargo build --release
plugin_name = basename ${plugin}
plugin_in = set ${CARGO_TARGET_DIR}/wasm32-wasi/release/${plugin_name}.wasm
plugin_out = set ${out_dir}/${plugin_name}.wasm
exec wasm-opt -O ${plugin_in} -o ${plugin_out}
cd ..
end
'''
[tasks.plugins]
workspace = false
script_runner = "@duckscript"
script = '''
plugins = glob_array ${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/default-plugins/*
for plugin in ${plugins}
cd ${plugin}
exec cargo build
cd ..
end
'''
[tasks.get-host-triple]
script_runner = "@duckscript"
@ -135,20 +138,6 @@ if not is_empty ${triple}
end
'''
[tasks.wasm-opt-plugins]
dependencies = ["build-plugins-release"]
script_runner = "@duckscript"
script = '''
plugins = glob_array ${CARGO_TARGET_DIR}/wasm32-wasi/release/*.wasm
for plugin in ${plugins}
mkdir ${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/assets/plugins/
plugin_name = basename ${plugin}
plugin_out = set ${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/assets/plugins/${plugin_name}
exec wasm-opt -O ${plugin} -o ${plugin_out}
end
'''
[tasks.manpage]
workspace = false
description = "Use mandown crate to create or update man entry from docs/MANPAGES.md"
@ -177,9 +166,8 @@ cp ${ZELLIJ_ASSETS_DIR}/config/default.kdl ${ZELLIJ_EXAMPLE_DIR}/default.kdl
[tasks.ci-build-release]
workspace = false
dependencies = [
"plugins-release",
"setup-cross-compilation",
"build-plugins-release",
"wasm-opt-plugins",
"manpage",
]
command = "cross"
@ -194,7 +182,7 @@ args = [
# Build e2e asset
[tasks.build-e2e]
workspace = false
dependencies = ["build-plugins-release", "build-e2e-data-dir"]
dependencies = ["wasm-opt-plugins", "build-e2e-data-dir"]
command = "cargo"
args = [
"build",
@ -207,7 +195,7 @@ args = [
# Run e2e tests - we mark the e2e tests as "ignored" so they will not be run with the normal ones
[tasks.e2e-test]
workspace = false
dependencies = ["build-e2e"]
dependencies = ["build-e2e", "plugins"]
command = "cargo"
args = [
"test",
@ -228,9 +216,8 @@ args = ["install", "cross"]
clear = true
workspace = false
dependencies = [
"plugins-release",
"update-default-config",
"build-plugins-release",
"wasm-opt-plugins",
"release-commit",
]
run_task = "publish-zellij"

BIN
assets/plugins/compact-bar.wasm
View File

Binary file not shown.

BIN
assets/plugins/status-bar.wasm
View File

Binary file not shown.

BIN
assets/plugins/strider.wasm
View File

Binary file not shown.

BIN
assets/plugins/tab-bar.wasm
View File

Binary file not shown.

49
src/commands.rs
View File

@ -1,33 +1,28 @@
use crate::install::populate_data_dir;
use crate::sessions::kill_session as kill_session_impl;
use crate::sessions::{
assert_session, assert_session_ne, get_active_session, get_sessions,
get_sessions_sorted_by_mtime, match_session_name, print_sessions, print_sessions_with_index,
session_exists, ActiveSession, SessionNameMatch,
};
use dialoguer::Confirm;
use miette::{Report, Result};
use std::path::PathBuf;
use std::process;
use zellij_client::old_config_converter::{
config_yaml_to_config_kdl, convert_old_yaml_files, layout_yaml_to_layout_kdl,
use std::{fs::File, io::prelude::*, path::PathBuf, process};
use crate::sessions::{
assert_session, assert_session_ne, get_active_session, get_sessions,
get_sessions_sorted_by_mtime, kill_session as kill_session_impl, match_session_name,
print_sessions, print_sessions_with_index, session_exists, ActiveSession, SessionNameMatch,
};
use zellij_client::start_client as start_client_impl;
use zellij_client::{os_input_output::get_client_os_input, ClientInfo};
use zellij_server::os_input_output::get_server_os_input;
use zellij_server::start_server as start_server_impl;
use zellij_utils::input::actions::Action;
use zellij_utils::input::config::ConfigError;
use zellij_utils::input::options::Options;
use zellij_utils::nix;
use zellij_client::{
old_config_converter::{
config_yaml_to_config_kdl, convert_old_yaml_files, layout_yaml_to_layout_kdl,
},
os_input_output::get_client_os_input,
start_client as start_client_impl, ClientInfo,
};
use zellij_server::{os_input_output::get_server_os_input, start_server as start_server_impl};
use zellij_utils::{
cli::{CliArgs, Command, SessionCommand, Sessions},
envs,
setup::{get_default_data_dir, Setup},
input::{actions::Action, config::ConfigError, options::Options},
nix,
setup::Setup,
};
use std::{fs::File, io::prelude::*};
pub(crate) use crate::sessions::list_sessions;
pub(crate) fn kill_all_sessions(yes: bool) {
@ -97,11 +92,6 @@ fn create_new_client() -> ClientInfo {
ClientInfo::New(names::Generator::default().next().unwrap())
}
fn install_default_assets(opts: &CliArgs) {
let data_dir = opts.data_dir.clone().unwrap_or_else(get_default_data_dir);
populate_data_dir(&data_dir);
}
fn find_indexed_session(
sessions: Vec<String>,
config_options: Options,
@ -364,10 +354,6 @@ pub(crate) fn start_client(opts: CliArgs) {
ClientInfo::New(_) => Some(layout),
};
if create {
install_default_assets(&opts);
}
start_client_impl(
Box::new(os_input),
opts,
@ -379,7 +365,6 @@ pub(crate) fn start_client(opts: CliArgs) {
} else {
let start_client_plan = |session_name: std::string::String| {
assert_session_ne(&session_name);
install_default_assets(&opts);
};
if let Some(session_name) = opts.session.clone() {

51
src/install.rs
View File

@ -1,51 +0,0 @@
#[cfg(not(feature = "disable_automatic_asset_installation"))]
use std::fs;
use std::path::Path;
#[cfg(not(feature = "disable_automatic_asset_installation"))]
use zellij_utils::{consts::VERSION, shared::set_permissions};
#[cfg(not(feature = "disable_automatic_asset_installation"))]
macro_rules! asset_map {
($($src:literal => $dst:literal),+ $(,)?) => {
{
let mut assets = std::collections::HashMap::new();
$(
assets.insert($dst, include_bytes!(concat!(env!("CARGO_MANIFEST_DIR"), "/", $src)).to_vec());
)+
assets
}
}
}
#[cfg(not(feature = "disable_automatic_asset_installation"))]
pub(crate) fn populate_data_dir(data_dir: &Path) {
let mut assets = asset_map! {
"assets/plugins/compact-bar.wasm" => "plugins/compact-bar.wasm",
"assets/plugins/status-bar.wasm" => "plugins/status-bar.wasm",
"assets/plugins/tab-bar.wasm" => "plugins/tab-bar.wasm",
"assets/plugins/strider.wasm" => "plugins/strider.wasm",
};
assets.insert("VERSION", VERSION.as_bytes().to_vec());
let last_version = fs::read_to_string(data_dir.join("VERSION")).unwrap_or_default();
let out_of_date = VERSION != last_version;
for (path, bytes) in assets {
let path = data_dir.join(path);
// TODO: Is the [path.parent()] really necessary here?
// We already have the path and the parent through `data_dir`
if let Some(parent_path) = path.parent() {
fs::create_dir_all(parent_path).unwrap_or_else(|e| log::error!("{:?}", e));
set_permissions(parent_path, 0o700).unwrap_or_else(|e| log::error!("{:?}", e));
if out_of_date || !path.exists() {
fs::write(path, bytes)
.unwrap_or_else(|e| log::error!("Failed to install default assets! {:?}", e));
}
} else {
log::error!("The path {:?} has no parent directory", path);
}
}
}
#[cfg(feature = "disable_automatic_asset_installation")]
pub(crate) fn populate_data_dir(_data_dir: &Path) {}

1
src/main.rs
View File

@ -1,5 +1,4 @@
mod commands;
mod install;
mod sessions;
#[cfg(test)]
mod tests;

293
zellij-server/src/wasm_vm.rs
View File

@ -29,7 +29,7 @@ use crate::{
};
use zellij_utils::{
consts::{DEBUG_MODE, VERSION, ZELLIJ_CACHE_DIR, ZELLIJ_PROJ_DIR, ZELLIJ_TMP_DIR},
consts::{DEBUG_MODE, VERSION, ZELLIJ_CACHE_DIR, ZELLIJ_TMP_DIR},
data::{Event, EventType, PluginIds},
errors::{prelude::*, ContextType, PluginContext},
input::{
@ -51,26 +51,41 @@ pub struct VersionMismatchError {
zellij_version: String,
plugin_version: String,
plugin_path: PathBuf,
// true for builtin plugins
builtin: bool,
}
impl std::error::Error for VersionMismatchError {}
impl VersionMismatchError {
pub fn new(zellij_version: &str, plugin_version: &str, plugin_path: &PathBuf) -> Self {
pub fn new(
zellij_version: &str,
plugin_version: &str,
plugin_path: &PathBuf,
builtin: bool,
) -> Self {
VersionMismatchError {
zellij_version: zellij_version.to_owned(),
plugin_version: plugin_version.to_owned(),
plugin_path: plugin_path.to_owned(),
builtin,
}
}
}
impl fmt::Display for VersionMismatchError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let first_line = if self.builtin {
"It seems your version of zellij was built with outdated core plugins."
} else {
"If you're seeing this error a plugin version doesn't match the current
zellij version."
};
write!(
f,
"If you're seeing this error the plugin versions don't match the current
zellij version. Detected versions:
"{}
Detected versions:
- Plugin version: {}
- Zellij version: {}
@ -81,15 +96,13 @@ If you're a user:
to them.
If you're a developer:
Please run zellij with the updated plugins. The easiest way to achieve this
Please run zellij with updated plugins. The easiest way to achieve this
is to build zellij with `cargo make install`. Also refer to the docs:
https://github.com/zellij-org/zellij/blob/main/CONTRIBUTING.md#building
A possible fix for this error is to remove all contents of the 'PLUGIN DIR'
folder from the output of the `zellij setup --check` command.
",
self.plugin_version,
self.zellij_version,
first_line,
self.plugin_version.trim_end(),
self.zellij_version.trim_end(),
self.plugin_path.display()
)
}
@ -149,11 +162,10 @@ pub(crate) fn wasm_thread_main(
let mut connected_clients: Vec<ClientId> = vec![];
let plugin_dir = data_dir.join("plugins/");
let plugin_global_data_dir = plugin_dir.join("data");
#[cfg(not(feature = "disable_automatic_asset_installation"))]
fs::create_dir_all(&plugin_global_data_dir)
.context("failed to create plugin asset directory")
.non_fatal();
// Caches the "wasm bytes" of all plugins that have been loaded since zellij was started.
// Greatly decreases loading times of all plugins and avoids accesses to the hard-drive during
// "regular" operation.
let mut plugin_cache: HashMap<PathBuf, Module> = HashMap::new();
loop {
let (event, mut err_ctx) = bus.recv().expect("failed to receive event on channel");
@ -169,7 +181,14 @@ pub(crate) fn wasm_thread_main(
.fatal();
let (instance, plugin_env) = start_plugin(
plugin_id, client_id, &plugin, tab_index, &bus, &store, &data_dir,
plugin_id,
client_id,
&plugin,
tab_index,
&bus,
&store,
&plugin_dir,
&mut plugin_cache,
)
.with_context(err_context)?;
@ -243,7 +262,8 @@ pub(crate) fn wasm_thread_main(
anyError::new(VersionMismatchError::new(
VERSION,
"Unavailable",
&plugin_env.plugin.path
&plugin_env.plugin.path,
plugin_env.plugin.is_builtin(),
))
),
Err(e) => Err(e).with_context(err_context),
@ -353,9 +373,17 @@ pub(crate) fn wasm_thread_main(
// load headless plugins
for plugin in plugins.iter() {
if let PluginType::Headless = plugin.run {
let (instance, plugin_env) =
start_plugin(plugin_id, client_id, plugin, 0, &bus, &store, &data_dir)
.with_context(err_context)?;
let (instance, plugin_env) = start_plugin(
plugin_id,
client_id,
plugin,
0,
&bus,
&store,
&plugin_dir,
&mut plugin_cache,
)
.with_context(err_context)?;
headless_plugins.insert(plugin_id, (instance, plugin_env));
plugin_id += 1;
}
@ -368,9 +396,17 @@ pub(crate) fn wasm_thread_main(
}
}
info!("wasm main thread exits");
fs::remove_dir_all(&plugin_global_data_dir)
.context("failed to cleanup plugin data directory")?;
Ok(())
.or_else(|err| {
if err.kind() == std::io::ErrorKind::NotFound {
// I don't care...
Ok(())
} else {
Err(err)
}
})
.context("failed to cleanup plugin data directory")
}
#[allow(clippy::too_many_arguments)]
@ -381,88 +417,116 @@ fn start_plugin(
tab_index: usize,
bus: &Bus<PluginInstruction>,
store: &Store,
data_dir: &Path,
plugin_dir: &Path,
plugin_cache: &mut HashMap<PathBuf, Module>,
) -> Result<(Instance, PluginEnv)> {
let err_context = || format!("failed to start plugin {plugin:#?} for client {client_id}");
if plugin._allow_exec_host_cmd {
info!(
"Plugin({:?}) is able to run any host command, this may lead to some security issues!",
plugin.path
);
}
// The plugins blob as stored on the filesystem
let wasm_bytes = plugin
.resolve_wasm_bytes(&data_dir.join("plugins/"))
.with_context(err_context)
.fatal();
let hash: String = PortableHash::default()
.hash256(&wasm_bytes)
.iter()
.map(ToString::to_string)
.collect();
let cached_path = ZELLIJ_PROJ_DIR.cache_dir().join(&hash);
let module = unsafe {
match Module::deserialize_from_file(store, &cached_path) {
Ok(m) => m,
Err(e) => {
let inner_context = || format!("failed to recover from {e:?}");
let m = Module::new(store, &wasm_bytes)
.with_context(inner_context)
.with_context(err_context)?;
fs::create_dir_all(ZELLIJ_PROJ_DIR.cache_dir())
.with_context(inner_context)
.with_context(err_context)?;
m.serialize_to_file(&cached_path)
.with_context(inner_context)
.with_context(err_context)?;
m
},
}
};
let output = Pipe::new();
let input = Pipe::new();
let stderr = LoggingPipe::new(&plugin.location.to_string(), plugin_id);
let plugin_own_data_dir = ZELLIJ_CACHE_DIR.join(Url::from(&plugin.location).to_string());
fs::create_dir_all(&plugin_own_data_dir)
.with_context(|| format!("failed to create datadir in {plugin_own_data_dir:?}"))
.with_context(|| format!("while starting plugin {plugin:#?}"))
.non_fatal();
let cache_hit = plugin_cache.contains_key(&plugin.path);
// ensure tmp dir exists, in case it somehow was deleted (e.g systemd-tmpfiles)
fs::create_dir_all(ZELLIJ_TMP_DIR.as_path())
.with_context(|| format!("failed to create tmpdir at {:?}", &ZELLIJ_TMP_DIR.as_path()))
.with_context(|| format!("while starting plugin {plugin:#?}"))
.non_fatal();
// We remove the entry here and repopulate it at the very bottom, if everything went well.
// We must do that because a `get` will only give us a borrow of the Module. This suffices for
// the purpose of setting everything up, but we cannot return a &Module from the "None" match
// arm, because we create the Module from scratch there. Any reference passed outside would
// outlive the Module we create there. Hence, we remove the plugin here and reinsert it
// below...
let module = match plugin_cache.remove(&plugin.path) {
Some(module) => {
log::debug!(
"Loaded plugin '{}' from plugin cache",
plugin.path.display()
);
module
},
None => {
// Populate plugin module cache for this plugin!
// Is it in the cache folder already?
if plugin._allow_exec_host_cmd {
info!(
"Plugin({:?}) is able to run any host command, this may lead to some security issues!",
plugin.path
);
}
// The plugins blob as stored on the filesystem
let wasm_bytes = plugin
.resolve_wasm_bytes(plugin_dir)
.with_context(err_context)
.fatal();
fs::create_dir_all(&plugin_own_data_dir)
.with_context(|| format!("failed to create datadir in {plugin_own_data_dir:?}"))
.with_context(err_context)
.non_fatal();
// ensure tmp dir exists, in case it somehow was deleted (e.g systemd-tmpfiles)
fs::create_dir_all(ZELLIJ_TMP_DIR.as_path())
.with_context(|| {
format!("failed to create tmpdir at {:?}", &ZELLIJ_TMP_DIR.as_path())
})
.with_context(err_context)
.non_fatal();
let hash: String = PortableHash::default()
.hash256(&wasm_bytes)
.iter()
.map(ToString::to_string)
.collect();
let cached_path = ZELLIJ_CACHE_DIR.join(&hash);
unsafe {
match Module::deserialize_from_file(store, &cached_path) {
Ok(m) => {
log::debug!(
"Loaded plugin '{}' from cache folder at '{}'",
plugin.path.display(),
ZELLIJ_CACHE_DIR.display(),
);
m
},
Err(e) => {
let inner_context = || format!("failed to recover from {e:?}");
let m = Module::new(store, &wasm_bytes)
.with_context(inner_context)
.with_context(err_context)?;
fs::create_dir_all(ZELLIJ_CACHE_DIR.to_owned())
.with_context(inner_context)
.with_context(err_context)?;
m.serialize_to_file(&cached_path)
.with_context(inner_context)
.with_context(err_context)?;
m
},
}
}
},
};
let mut wasi_env = WasiState::new("Zellij")
.env("CLICOLOR_FORCE", "1")
.map_dir("/host", ".")
.with_context(err_context)?
.map_dir("/data", &plugin_own_data_dir)
.with_context(err_context)?
.map_dir("/tmp", ZELLIJ_TMP_DIR.as_path())
.with_context(err_context)?
.stdin(Box::new(input))
.stdout(Box::new(output))
.stderr(Box::new(stderr))
.finalize()
.and_then(|wasi| wasi.map_dir("/data", &plugin_own_data_dir))
.and_then(|wasi| wasi.map_dir("/tmp", ZELLIJ_TMP_DIR.as_path()))
.and_then(|wasi| {
wasi.stdin(Box::new(Pipe::new()))
.stdout(Box::new(Pipe::new()))
.stderr(Box::new(LoggingPipe::new(
&plugin.location.to_string(),
plugin_id,
)))
.finalize()
})
.with_context(err_context)?;
let wasi = wasi_env.import_object(&module).with_context(err_context)?;
let mut plugin = plugin.clone();
plugin.set_tab_index(tab_index);
let mut mut_plugin = plugin.clone();
mut_plugin.set_tab_index(tab_index);
let plugin_env = PluginEnv {
plugin_id,
client_id,
plugin,
plugin: mut_plugin,
senders: bus.senders.clone(),
wasi_env,
subscriptions: Arc::new(Mutex::new(HashSet::new())),
@ -473,17 +537,38 @@ fn start_plugin(
let zellij = zellij_exports(store, &plugin_env);
let instance = Instance::new(&module, &zellij.chain_back(wasi)).with_context(err_context)?;
// Check plugin version
if !cache_hit {
// Check plugin version
assert_plugin_version(&instance, &plugin_env).with_context(err_context)?;
}
// Only do an insert when everything went well!
let cloned_plugin = plugin.clone();
plugin_cache.insert(cloned_plugin.path, module);
Ok((instance, plugin_env))
}
// Returns `Ok` if the plugin version matches the zellij version.
// Returns an `Err` otherwise.
fn assert_plugin_version(instance: &Instance, plugin_env: &PluginEnv) -> Result<()> {
let err_context = || {
format!(
"failed to determine plugin version for plugin {}",
plugin_env.plugin.path.display()
)
};
let plugin_version_func = match instance.exports.get_function("plugin_version") {
Ok(val) => val,
Err(_) => panic!(
"{}",
anyError::new(VersionMismatchError::new(
Err(_) => {
return Err(anyError::new(VersionMismatchError::new(
VERSION,
"Unavailable",
&plugin_env.plugin.path
))
),
&plugin_env.plugin.path,
plugin_env.plugin.is_builtin(),
)))
},
};
plugin_version_func.call(&[]).with_context(err_context)?;
let plugin_version_str = wasi_read_string(&plugin_env.wasi_env);
@ -494,17 +579,15 @@ fn start_plugin(
.context("failed to parse zellij version")
.with_context(err_context)?;
if plugin_version != zellij_version {
panic!(
"{}",
anyError::new(VersionMismatchError::new(
VERSION,
&plugin_version_str,
&plugin_env.plugin.path
))
);
return Err(anyError::new(VersionMismatchError::new(
VERSION,
&plugin_version_str,
&plugin_env.plugin.path,
plugin_env.plugin.is_builtin(),
)));
}
Ok((instance, plugin_env))
Ok(())
}
fn load_plugin(instance: &mut Instance) -> Result<()> {

4
zellij-utils/Cargo.toml
View File

@ -50,5 +50,9 @@ insta = { version = "1.6.0", features = ["backtrace"] }
[features]
# If this feature is NOT set (default):
# - builtin plugins (status-bar, tab-bar, ...) are loaded directly from the application binary
# If this feature is set:
# - builtin plugins MUST be available from whatever is configured as `PLUGIN_DIR`
disable_automatic_asset_installation = []
unstable = []

45
zellij-utils/src/consts.rs
View File

@ -35,6 +35,51 @@ pub const FEATURES: &[&str] = &[
"disable_automatic_asset_installation",
];
#[cfg(not(target_family = "wasm"))]
pub use not_wasm::*;
#[cfg(not(target_family = "wasm"))]
mod not_wasm {
use lazy_static::lazy_static;
use std::collections::HashMap;
use std::path::PathBuf;
// Convenience macro to add plugins to the asset map (see `ASSET_MAP`)
macro_rules! add_plugin {
($assets:expr, $plugin:literal) => {
$assets.insert(
PathBuf::from("plugins").join($plugin),
#[cfg(debug_assertions)]
include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../target/wasm32-wasi/debug/",
$plugin
))
.to_vec(),
#[cfg(not(debug_assertions))]
include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../assets/plugins/",
$plugin
))
.to_vec(),
);
};
}
lazy_static! {
// Zellij asset map
pub static ref ASSET_MAP: HashMap<PathBuf, Vec<u8>> = {
let mut assets = std::collections::HashMap::new();
add_plugin!(assets, "compact-bar.wasm");
add_plugin!(assets, "status-bar.wasm");
add_plugin!(assets, "tab-bar.wasm");
add_plugin!(assets, "strider.wasm");
assets
};
}
}
#[cfg(unix)]
pub use unix_only::*;

59
zellij-utils/src/errors.rs
View File

@ -14,6 +14,7 @@ use colored::*;
use log::error;
use serde::{Deserialize, Serialize};
use std::fmt::{Display, Error, Formatter};
use std::path::PathBuf;
use miette::Diagnostic;
use thiserror::Error as ThisError;
@ -92,9 +93,16 @@ pub trait LoggableError<T>: Sized {
/// .print_error(|msg| println!("{msg}"))
/// .unwrap();
/// ```
#[track_caller]
fn print_error<F: Fn(&str)>(self, fun: F) -> Self;
/// Convenienve function, calls `print_error` with the closure `|msg| log::error!("{}", msg)`.
// Dev note:
// Currently this hides the location of the caller, because it will show this very line as
// "source" of the logging call. This isn't correct, because it may have been called by other
// functions, too. To track this, we need to attach `#[track_caller]` to the closure below,
// which isn't stabilized yet: https://github.com/rust-lang/rust/issues/87417
#[track_caller]
fn to_log(self) -> Self {
self.print_error(|msg| log::error!("{}", msg))
}
@ -133,6 +141,7 @@ pub trait FatalError<T> {
/// Discards the result type afterwards.
///
/// [`to_log`]: LoggableError::to_log
#[track_caller]
fn non_fatal(self);
/// Mark results as being fatal.
@ -394,6 +403,56 @@ pub enum ZellijError {
#[error("failed to start PTY")]
FailedToStartPty,
#[error(
"This version of zellij was built to load the core plugins from
the globally configured plugin directory. However, a plugin wasn't found:
Plugin name: '{plugin_path}'
Plugin directory: '{plugin_dir}'
If you're a user:
Please report this error to the distributor of your current zellij version
If you're a developer:
Either make sure to include the plugins with the application (See feature
'disable_automatic_asset_installation'), or make them available in the
plugin directory.
Possible fix for your problem:
Run `zellij setup --dump-plugins`, and optionally point it to your
'DATA DIR', visible in e.g. the output of `zellij setup --check`. Without
further arguments, it will use the default 'DATA DIR'.
"
)]
BuiltinPluginMissing {
plugin_path: PathBuf,
plugin_dir: PathBuf,
#[source]
source: anyhow::Error,
},
#[error(
"It seems you tried to load the following builtin plugin:
Plugin name: '{plugin_path}'
This is not a builtin plugin known to this version of zellij. If you were using
a custom layout, please refer to the layout documentation at:
https://zellij.dev/documentation/creating-a-layout.html#plugin
If you think this is a bug and the plugin is indeed an internal plugin, please
open an issue on GitHub:
https://github.com/zellij-org/zellij/issues
"
)]
BuiltinPluginNonexistent {
plugin_path: PathBuf,
#[source]
source: anyhow::Error,
},
#[error("an error occured")]
GenericError { source: anyhow::Error },
}

2
zellij-utils/src/input/layout.rs
View File

@ -168,7 +168,7 @@ pub struct RunPlugin {
pub location: RunPluginLocation,
}
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Eq)]
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Eq, Hash)]
pub enum RunPluginLocation {
File(PathBuf),
Zellij(PluginTag),

78
zellij-utils/src/input/plugins.rs
View File

@ -9,6 +9,8 @@ use serde::{Deserialize, Serialize};
use url::Url;
use super::layout::{RunPlugin, RunPluginLocation};
#[cfg(not(target_family = "wasm"))]
use crate::consts::ASSET_MAP;
pub use crate::data::PluginTag;
use crate::errors::prelude::*;
@ -74,7 +76,7 @@ impl Default for PluginsConfig {
}
/// Plugin metadata
#[derive(Clone, Debug, PartialEq, Deserialize, Serialize)]
#[derive(Clone, Debug, PartialEq, Eq, Hash, Deserialize, Serialize)]
pub struct PluginConfig {
/// Path of the plugin, see resolve_wasm_bytes for resolution semantics
pub path: PathBuf,
@ -87,16 +89,20 @@ pub struct PluginConfig {
}
impl PluginConfig {
/// Resolve wasm plugin bytes for the plugin path and given plugin directory. Attempts to first
/// resolve the plugin path as an absolute path, then adds a ".wasm" extension to the path and
/// resolves that, finally we use the plugin directory joined with the path with an appended
/// ".wasm" extension. So if our path is "tab-bar" and the given plugin dir is
/// "/home/bob/.zellij/plugins" the lookup chain will be this:
/// Resolve wasm plugin bytes for the plugin path and given plugin directory.
///
/// If zellij was built without the 'disable_automatic_asset_installation' feature, builtin
/// plugins (Starting with 'zellij:' in the layout file) are loaded directly from the
/// binary-internal asset map. Otherwise:
///
/// Attempts to first resolve the plugin path as an absolute path, then adds a ".wasm"
/// extension to the path and resolves that, finally we use the plugin directory joined with
/// the path with an appended ".wasm" extension. So if our path is "tab-bar" and the given
/// plugin dir is "/home/bob/.zellij/plugins" the lookup chain will be this:
///
/// ```bash
/// /tab-bar
/// /tab-bar.wasm
/// /home/bob/.zellij/plugins/tab-bar.wasm
/// ```
///
pub fn resolve_wasm_bytes(&self, plugin_dir: &Path) -> Result<Vec<u8>> {
@ -110,9 +116,8 @@ impl PluginConfig {
&plugin_dir.join(&self.path).with_extension("wasm"),
];
// Throw out dupes, because it's confusing to read that zellij checked the same plugin
// location multiple times
// location multiple times. Do NOT sort the vector here, because it will break the lookup!
let mut paths = paths_arr.to_vec();
paths.sort_unstable();
paths.dedup();
// This looks weird and usually we would handle errors like this differently, but in this
@ -122,8 +127,32 @@ impl PluginConfig {
// spell it out right here.
let mut last_err: Result<Vec<u8>> = Err(anyhow!("failed to load plugin from disk"));
for path in paths {
// Check if the plugin path matches an entry in the asset map. If so, load it directly
// from memory, don't bother with the disk.
#[cfg(not(target_family = "wasm"))]
if !cfg!(feature = "disable_automatic_asset_installation") && self.is_builtin() {
let asset_path = PathBuf::from("plugins").join(path);
if let Some(bytes) = ASSET_MAP.get(&asset_path) {
log::debug!("Loaded plugin '{}' from internal assets", path.display());
if plugin_dir.join(path).with_extension("wasm").exists() {
log::info!(
"Plugin '{}' exists in the 'PLUGIN DIR' at '{}' but is being ignored",
path.display(),
plugin_dir.display()
);
}
return Ok(bytes.to_vec());
}
}
// Try to read from disk
match fs::read(&path) {
Ok(val) => return Ok(val),
Ok(val) => {
log::debug!("Loaded plugin '{}' from disk", path.display());
return Ok(val);
},
Err(err) => {
last_err = last_err.with_context(|| err_context(err, &path));
},
@ -131,6 +160,29 @@ impl PluginConfig {
}
// Not reached if a plugin is found!
#[cfg(not(target_family = "wasm"))]
if self.is_builtin() {
// Layout requested a builtin plugin that wasn't found
let plugin_path = self.path.with_extension("wasm");
if cfg!(feature = "disable_automatic_asset_installation")
&& ASSET_MAP.contains_key(&PathBuf::from("plugins").join(&plugin_path))
{
return Err(ZellijError::BuiltinPluginMissing {
plugin_path,
plugin_dir: plugin_dir.to_owned(),
source: last_err.unwrap_err(),
})
.context("failed to load a plugin");
} else {
return Err(ZellijError::BuiltinPluginNonexistent {
plugin_path,
source: last_err.unwrap_err(),
})
.context("failed to load a plugin");
}
}
return last_err;
}
@ -143,10 +195,14 @@ impl PluginConfig {
PluginType::Headless => {},
}
}
pub fn is_builtin(&self) -> bool {
matches!(self.location, RunPluginLocation::Zellij(_))
}
}
/// Type of the plugin. Defaults to Pane.
#[derive(Clone, Debug, PartialEq, Deserialize, Serialize)]
#[derive(Clone, Debug, PartialEq, Deserialize, Serialize, Hash, Eq)]
#[serde(rename_all = "kebab-case")]
pub enum PluginType {
// TODO: A plugin with output that's cloned across every pane in a tab, or across the entire

82
zellij-utils/src/setup.rs
View File

@ -1,3 +1,5 @@
#[cfg(not(target_family = "wasm"))]
use crate::consts::ASSET_MAP;
use crate::input::theme::Themes;
use crate::{
cli::{CliArgs, Command},
@ -5,6 +7,7 @@ use crate::{
FEATURES, SYSTEM_DEFAULT_CONFIG_DIR, SYSTEM_DEFAULT_DATA_DIR_PREFIX, VERSION,
ZELLIJ_PROJ_DIR,
},
errors::prelude::*,
input::{
config::{Config, ConfigError},
layout::Layout,
@ -172,6 +175,41 @@ pub fn dump_specified_layout(layout: &str) -> std::io::Result<()> {
}
}
#[cfg(not(target_family = "wasm"))]
pub fn dump_builtin_plugins(path: &PathBuf) -> Result<()> {
for (asset_path, bytes) in ASSET_MAP.iter() {
let plugin_path = path.join(asset_path);
plugin_path
.parent()
.with_context(|| {
format!(
"failed to acquire parent path of '{}'",
plugin_path.display()
)
})
.and_then(|parent_path| {
std::fs::create_dir_all(parent_path).context("failed to create parent path")
})
.with_context(|| {
format!(
"failed to create folder '{}' to dump plugin '{}' to",
path.display(),
plugin_path.display()
)
})?;
std::fs::write(plugin_path, bytes)
.with_context(|| format!("failed to dump builtin plugin '{}'", asset_path.display()))?;
}
Ok(())
}
#[cfg(target_family = "wasm")]
pub fn dump_builtin_plugins(_path: &PathBuf) -> Result<()> {
Ok(())
}
#[derive(Debug, Default, Clone, Args, Serialize, Deserialize)]
pub struct Setup {
/// Dump the default configuration file to stdout
@ -192,6 +230,17 @@ pub struct Setup {
#[clap(long, value_parser)]
pub dump_layout: Option<String>,
/// Dump the builtin plugins to DIR or "DATA DIR" if unspecified
#[clap(
long,
value_name = "DIR",
value_parser,
exclusive = true,
min_values = 0,
max_values = 1
)]
pub dump_plugins: Option<Option<PathBuf>>,
/// Generates completion for the specified shell
#[clap(long, value_name = "SHELL", value_parser)]
pub generate_completion: Option<String>,
@ -263,7 +312,7 @@ impl Setup {
}
/// General setup helpers
pub fn from_cli(&self) -> std::io::Result<()> {
pub fn from_cli(&self) -> Result<()> {
if self.clean {
return Ok(());
}
@ -292,15 +341,24 @@ impl Setup {
}
/// Checks the merged configuration
pub fn from_cli_with_options(
&self,
opts: &CliArgs,
config_options: &Options,
) -> std::io::Result<()> {
pub fn from_cli_with_options(&self, opts: &CliArgs, config_options: &Options) -> Result<()> {
if self.check {
Setup::check_defaults_config(opts, config_options)?;
std::process::exit(0);
}
if let Some(maybe_path) = &self.dump_plugins {
let data_dir = &opts.data_dir.clone().unwrap_or_else(get_default_data_dir);
let dir = match maybe_path {
Some(path) => path,
None => data_dir,
};
println!("Dumping plugins to '{}'", dir.display());
dump_builtin_plugins(&dir)?;
std::process::exit(0);
}
Ok(())
}
@ -361,6 +419,18 @@ impl Setup {
}
writeln!(&mut message, "[DATA DIR]: {:?}", data_dir).unwrap();
message.push_str(&format!("[PLUGIN DIR]: {:?}\n", plugin_dir));
if !cfg!(feature = "disable_automatic_asset_installation") {
writeln!(
&mut message,
" Builtin, default plugins will not be loaded from disk."
)
.unwrap();
writeln!(
&mut message,
" Create a custom layout if you require this behavior."
)
.unwrap();
}
if let Some(layout_dir) = layout_dir {
writeln!(&mut message, "[LAYOUT DIR]: {:?}", layout_dir).unwrap();
} else {

5
zellij-utils/src/shared.rs
View File

@ -24,6 +24,11 @@ mod unix_only {
}
}
#[cfg(not(unix))]
pub fn set_permissions(_path: &std::path::Path, _mode: u32) -> std::io::Result<()> {
Ok(())
}
pub fn ansi_len(s: &str) -> usize {
from_utf8(&strip(s).unwrap()).unwrap().width()
}