Do it "more properly": use intrusive list linkage to manage three
indices:
* hash lookup
* recency
* frequency
eviction is frequency based, but in order to avoid things that were
super hot in the past and that are no longer hot clogging up the cache,
eviction will incrementally age out least recently used entries that
haven't been active in the span of some number of get/put operations.
Aging scales down the frequency value to reduce its strength, so an aged
item isn't necessarily immediately a candidate for removal, it just
makes it more likely to be picked up by the frequency based removal as
it goes unused for an extended period.
We were only using block_on from it, which we can call from the
underlying async_io crate.
Notably, removing async_std from the deps avoids async-global-executor
being pulled in and spawning nproc threads that are never used by
wezterm.
I spent a few hours in heap profilers. What I found was:
* Inefficient use of heap when building up runs of
`Action::Print(char)`.
-> Solve by adding `Action::PrintString(String)`
and accumulating utf8 bytes rather than u32 codepoints.
* Inefficient use of heap when building Quad buffers: the default
exponential growth of `Vec` tended to waste 40%-75% of the allocated
capacity, and since we could keep ~1024 of these in cache, there's
a lot of potential for waste.
-> Solve by bounding the growth to 64 at a time. This has similar
characteristics to exponential growth at the default 80x24 terminal
size. May need to add a config option for this step size for users
with very large terminals.
* Lazy eviction from the LFU caches. The underlying cache advisor is
somewhat probabilistic and has a minimum cache size of 256, making
it difficult to maintain low heap utilization.
-> Solve by replacing it with a very simple LFU algorithm. It doesn't
seem to hurt much at the default terminal size with the default
cache sizes. If we make the cache sizes smaller, its overhead is
reduced.
Some further experimentation is needed to adjust defaults, but this
should help reduce heap usage.
refs: https://github.com/wez/wezterm/issues/2626
I noticed from https://github.com/void-linux/void-packages/pull/36903
that 32-bit systems were failing to pass the test suite.
Running `cargo test --target i686-unknown-linux-gnu -- --nocapture
teeny_string` showed that we were faulting in the teenystring code
and digging a bit deeper showed that it was because our assumptions
about the high bits were wrong for 32-bit systems.
Fix this by making TeenyString based around a u64 rather than usize
so that we guarantee its size on all current systems.
According to its benchmarks, it's almost 2x faster than
unicode_segmentation. It doesn't appear to make a visible
difference to `time cat bigfile`, but I'll take anything
that gives more headroom for such little effort of switching.
There are caveats to determining this, but when we think
password entry is enabled, switch the cursor to the font-awesome
lock glyph instead of the normal cursor sprite.
fa_lock is used because it is monochrome and can thus be tinted
to the configured cursor color, and it respects blinking/easing.
refs: https://github.com/wez/wezterm/issues/2460
The idea here is that different kinds of panes may want to expose
additional metadata to lua scripts. It would be a bit weird to add
a Pane method for each of those and plumb it all the way through
the various APIs, so just allowing a pane impl to return a dynamic
value (likely an Object) allows a bunch of flexibility.
This commit exposes the clientpane is_tardy boolean and the time
since the last data was recevied (since_last_response_ms) from
the mux client pane implementation: these are used to show the
tardiness indicator in the client pane.
Exposing this data enables the user to add that info to their
status bar if they wish.
CTRL-SHIFT-U is a new default key assignment for this new modal.
It opens up a fuzzy searchable browser that defaults to showing
emoji/emoticons. The category can by cycled through the suggested
emoji categories using CTRL-r. Unlike the system emoji palette,
wezterm includes a category for nerdfont symbols, and another
that is a list of all unicode codepoint names, so you should be
able to browse for pretty much any codepoint you can think of.
The modal also allows fuzzy searching based on:
* The official unicode name
* The github shortcode
* codepoint value in hex
so if you know the codepoint value but not the name, you can
still find a way to input what you're looking for.
Pressing Enter will copy the selected item to the clipboard
send it to the active pane, and cancel the modal. You can therefore
repeat the insert by simply pasting.
I plan to add frecency to this in a later commit: that way the
frequently/recently used selections will show in a category of
their own and make it easier to re-input them.
refs: https://github.com/wez/wezterm/issues/2163
This breaking API change allows us to explicitly generate EOF when the
taken writer is dropped.
The examples have been updated to show how to manage read, write
and waiting without deadlock for both linux and windows.
Need to confirm that this is still good on macOS, but my
confidence is high.
I've also removed ssh2 support from this crate as part of this
change. We haven't used it directly in wezterm in a long while
and removing it from here means that there is slightly less code
to keep compiling over and over.
refs: https://github.com/wez/wezterm/discussions/2392
refs: https://github.com/wez/wezterm/issues/1396
`wezterm start -- /etc/profile` would crash on macOS because
`/etc/profile` isn't executable.
This commit checks for executable access as a prereq for both the
path search and the absolute path cases and generates a non-crashing
error:
```
$ wezterm start -- /etc/profile
17:24:03.574 ERROR wezterm_gui > Unable to spawn /etc/profile because it doesn't exist on the filesystem or is not executable (EACCES: Permission denied); terminating
```
When `parse_first_as_vec` is parsing an OSC sequence (e.g.
`SetHyperlink`) that is terminated by the escaped form of ST (`ESC \`),
ensure that the ST sequence is included in the returned vector.
This is achieved by ensuring the VTParser has returned to the "ground"
state: i.e. the stored state after the `ESC` is processed is not enough
for `parse_first_as_vec` to terminate. We must also parse the `\` to
ensure that we return a complete span to the caller.
Fixes https://github.com/markbt/streampager/issues/57
This doesn't really change any behavior, but adjusts the types
such that CSIs that set colors have the potential to track the
alpha channel and that can make it through to the GUI/render layer.
When adding sparse ranges the cartesian product of range combinations
was explored to find intersections, which is pretty awful if there
are 1 million entries to be inserted.
This commit employs binary search to reduce the complexity, at
the expense of requiring that the internal range array is sorted.
Adds the option to use an alternative clusted line storage for
the cells component of the line.
This structure is not optimal for mutation, but is better structured
for:
* matching/extracting textual content
* using less memory than the prior simple vector
For some contrast: the line "hello" occupies 5 Cells in the cell based
storage; that 5 discrete Cells each with their own tiny string
and a copy of their attributes.
The clustered version of the line stores one copy of the cell
attributes, the string "hello" and some small (almost constant size)
overhead for some metadata. For simple lines of ascii text, the
clustered version is smaller as there are fewer copies of the cell
attributes. Over the span of a large scrollback and typical terminal
display composition, this saving is anticipated to be significant.
The clustered version is also cheaper to search as it doesn't require
building a copy of the search text for each line (provided the line is
already in clustered form).
This commit introduces the capability: none of the internals request the
new form yet, and there are likely a few call sites that need to be
tweaked to avoid coersion from clustered to vector form.
Various color schemes have been duplicated as they have been added to
different scheme collections. They don't always have identical names
(eg: some remove spaces) and sometimes they have very different names
(eg: _bash vs. nightfox, or Miu vs. Blazer).
We already detected duplicates from different collections but previously
we would omit those dupes.
This commit allows us to track those duplicates by recording their
aliases.
When we write out our data, we only include "interesting" alias names;
those where the name isn't trivially identical.
Some scheme collections (eg: iterm2 color schemes) have duplicates
(eg: zenbones and zenbones_light are identical) and we have previously
shipped with both of those names, so we special case to emit dupes
for which we have prior version information in order to avoid
breaking backwards compatibility for our users.
In the doc generation we can generate links to the aliases if we
included them, but also note about the other names and how we don't
include them. That is so that someone searching the docs for say
"_bash" can discover that it is actually a duplicate of "nightfox" and
use nightfox instead.
Adjust importer to read directly from the source .itermcolors
files in the upstream repo. Extract some author information
from the comments in those files.
All data is now fetched (and cached!) via relatively minimal
http requests rather than requiring a git repo locally.
Also search for .yml files in base16 repos; found another
couple of schemes this way.
The toml files under assets/colors are no longer read by
anything in the repo. I plan to remove them, but since the
docs reference them as examples, I will first ensure that
there are docs and tooling that explains how to write and
share your own scheme files.
wezterm.color.parse() returns a color object that can be assigned in the
wezterm color config, and that can be used to adjust hue, saturation and
lightness, as well as calculate harmonizing colors (complements, triads,
squares) from the RGB/HSL color wheel.
This commit expands the toml file definition to include
metadata for the origin url, author and name.
A new sync utility fills out that metadata when it pulls from the iterm2
color schemes repo.
The utility also pulls down the scheme data json maintained by
the Gogh project: https://gogh-co.github.io/Gogh/ and converts
it to wezterm's format.
About 50% of Gogh overlaps with iterm2; we take the iterm2 versions
of those schemes by default because the iterm2 data has more info
about things like cursor and selection colors.
The sync utility is responsible for compiling the de-duplicated
set of scheme data into a form that is used by wezterm and its
docs.
Currently implemented on X11 only, this function returns information
about the geometry of the screen(s).
This is taken from the same source of information we use for the
`--position` CLI argument to `wezterm start`.
```
> wezterm.window.screens()
{
"by_name": {
"DisplayPort-1": {
"height": 2160,
"name": "DisplayPort-1",
"width": 3840,
"x": 0,
"y": 0,
},
},
"main": {
"height": 2160,
"name": "DisplayPort-1",
"width": 3840,
"x": 0,
"y": 0,
},
"origin_x": 0,
"origin_y": 0,
"virtual_height": 2160,
"virtual_width": 3840,
}
```
Using the newly exposed-to-lua mux apis, you may now run some lua code
at GUI startup and/or mux startup, just prior to any default windows
being created.
If you happen to spawn any panes as a result of this, wezterm will
skip creating the default program.
```lua
local wezterm = require 'wezterm'
local mux = wezterm.mux
-- This produces a window split horizontally into three equal parts
wezterm.on("gui-startup", function()
wezterm.log_info("doing gui startup")
local tab, pane, window = mux.spawn_window{}
mux.split_pane(pane, {size=0.3})
mux.split_pane(pane, {size=0.5})
end)
wezterm.on("mux-startup", function()
wezterm.log_info("doing mux startup")
local tab, pane, window = mux.spawn_window{}
mux.split_pane(pane, {size=0.5, direction="Top"})
end)
return {
unix_domains = {
{name="unix"}
},
}
```
refs: #674
refs: #1949
The intent is to expose Mux related functions to lua, so `wezterm.mux`
will be that module table.
In order to test this out in the debug overlay, I realized that the
overlay was running functions in a different thread and didn't have
access to the mux, so this commit also tweaks the debug overlay repl to
execute the input in the main thread.
The result is that it is now possible to do
`wezterm.mux.active_workspace()` in the debug overlay to print the
active workspace name.
More functions will follow.
refs: https://github.com/wez/wezterm/issues/225
Main thing to note here is that the open crate has deprecated
open::that_in_background, but made open::that non-blocking.
I think this is OK, but I'm a little cagey about what will
happen with this on Windows. We may need to spawn our own
thread for this if things go awry.
This allows moving past a number of outdated deps, but importantly,
allows updating miow to a version that has the correct memory layout
for SocketAddr, and satisfies a dependabot alert.
This appeases some dependabot warnings regarding the use of
an old version of parking_lot that has some unsoundness issues.
ratelimit_meter is frozen and is succeeded by governor which has
a mostly similar API.
The hope here is that the nvidia-specific resize issue might have
a workaround if it is emitting some other events that we were
previously not listening for.
This commit optionally enables the Present extension and listens
for its version of CONFIGURE_NOTIFY, routing it through the same
logic as the base CONFIGURE_NOTIFY event.
On my AMD hardware under Gnome, I see something like:
```
18:04:26.476 TRACE window::os::x11::window > Present::ConfigureNotify: width 1168 -> 1180, height 858 -> 873, dpi 124.7998046875 -> 124.7998046875
18:04:26.478 TRACE window::os::x11::window > Ignoring X::ConfigureNotify (1180x873 dpi=124.7998046875) because width,height,dpi are unchanged
```
with the Present event firing before the X event.
Let's see how this goes.
refs: #2063
refs: #1992
My windows build didn't work this morning, complaining:
; cargo build
Updating crates.io index
error: failed to select a version for the requirement `libgit2-sys = "^0.13.4"`
candidate versions found which didn't match: 0.13.2+1.4.2, 0.13.1+1.4.2, 0.13.0+1.4.1, ...
location searched: crates.io index
required by package `git2 v0.14.4`
... which satisfies dependency `git2 = "^0.14"` (locked to 0.14.4) of package `config v0.1.0 (C:\Users\wez\wez-personal\wezterm\config)`
... which satisfies path dependency `config` (locked to 0.1.0) of package `codec v0.1.0 (C:\Users\wez\wez-personal\wezterm\codec)`
... which satisfies path dependency `codec` (locked to 0.1.0) of package `wezterm v0.1.0 (C:\Users\wez\wez-personal\wezterm\wezterm)`
The cargo update downgraded the dep, but looking at crates.io the
requested revs look ok. Something is fishy.
I upgraded rust and ran cargo update again to get back on the
same git rev, so now this commit captures the other things
that changed.
Avoid using serde for mapping between Lua and Rust for the `Config`
struct.
This improves the build speed of the config crate by 2x; it goes down
from 30 seconds to 9 seconds on my 5950x.
This cleans up the `cargo audit` output on linux because the `clipboard`
crate (which hasn't been updated in 3 years) depends on xcb=0.8.2
which is flagged by cargo audit.
We don't use `clipboard` on any platform except macos
This commit switches to the `clipboard_macos` crate; that appears to
use a copy and paste of the macos specific code from the `clipboard`
crate, so this shouldn't have any change in functionality.
refs: https://github.com/wez/wezterm/issues/1952
I think I'd like to make a config option for this, but for the moment,
this first pass unconditionally updates the base environment with
data from the registry.
refs: https://github.com/wez/wezterm/issues/1848
Use some heuristics to verify the data that is about to be parsed;
this can help to detect eg: data being output to stdout prior
to us sending any encoded data to the remote mux.
In addition, add a timeout to help avoid waiting forever in
the case that we didn't detect a problem.
refs: https://github.com/wez/wezterm/issues/1860
I wanted to use the Target::Pipe feature of env_logger so that we could
log to a log file as well as stderr, but it just doesn't work
(https://github.com/env-logger-rs/env_logger/issues/208).
Since we were already composing over the top of the logger in order
to capture data for our ringlog, this commit embraces that and makes
our logger responsible for both stderr and log file printing.
Thankfully, we can use the filter parsing code from env_logger to
avoid having to get too crazy with this.
Logs are stored in the runtime directory and look something like:
/run/user/1000/wezterm/wezterm-gui-log-596324.txt
Logs are collected on all platforms.
There isn't currently a thing to clean up logs.
Go directly to the underlying env_logger crate, as pretty_env_logger
hasn't been updated in some time, and I'd like to be able to redirect
the log output to a file more directly, and that feature is in a newer
version of the env logger than pretty_env_logger was pulling in.
We were pinned on the revision where I had added dual source blending,
because I wanted that feature ahead of the crate being published.
Since then a couple of releases have been made so we can unpin.
On Windows 11, we had a report of glium complaining about the opengl
version. I can't find that error message string in the current version
of the code so it's possible that that situation has been resolved.
refs: https://github.com/wez/wezterm/issues/1813
* termwiz: add support for kitty image using shm mode
* termwiz: kitty image shm cleanup error handling
* termwiz: kitty image shm create wrapper for HANDLE and impl Drop for it
* termwiz: kitty image shm refactor windows implementation
Signed-off-by: Shreesh Adiga <16567adigashreesh@gmail.com>
There were a a couple of issues:
* `ImageData::hash` would re-hash the image on every call, and this was
called for every cell that comprised an image on the mux server side
* `SerializedLine` needed to understand how to remove the `Arc<ImageData>`
image attachments so that we didn't serialize a complete copy of the
image per cell that comprised the image.
A new RPC was introduced to attempt to fetch `ImageData` given its
content hash and pane, row and cell index as a hint to locate it.
A client side LRU of content hash to `ImageData` is used to avoid
issuing repeat calls to that new RPC.
refs: #1237
freetype can't handle a wide range of encodings for
font names and can return strings like `?????` when
the family name is only present in the font as a non-unicode encoding,
such as Chinese.
This commit improves our handling of the font name table
and prefers to use results from processing that over the
results returned for eg: font family directly from the
freetype API.
refs: https://github.com/wez/wezterm/issues/1761
The metadata is useful from a troubleshooting/diagnostic perspective.
`.cast` isn't on github's list of allowed file name suffixes, so make
it a simple text file for convenience.
These are intended to replace the `wt-record` and `wt-replay` scripts
that are primarily for me to diagnose wezterm issues.
The thinking is that this gives a cross platform implementation of
this functionality and squashes out some of the pain of dealing with
different versions and file formats.
The new subcommands use the asciicast v2 file format, meaning that
wezterm can record for asciicast/asciinema and vice-versa.
This commit only handles unix; windows support will follow in
a separate commit.
