This allows us to support the kitty style underline sequence,
or the : separated form of the true color escape sequences.
refs: https://github.com/wez/wezterm/issues/415
These aren't currently rendered, but the parser and model now support
recognizing expanded underline sequences:
```
CSI 24 m -> No underline
CSI 4 m -> Single underline
CSI 21 m -> Double underline
CSI 60 m -> Curly underline
CSI 61 m -> Dotted underline
CSI 62 m -> Dashed underline
CSI 58 ; 2 ; R ; G ; B m -> set underline color to specified true color RGB
CSI 58 ; 5 ; I m -> set underline color to palette index I (0-255)
CSI 59 -> restore underline color to default
```
The Curly, Dotted and Dashed CSI codes are a wezterm assignment in the
SGR space. This is by no means official; I just picked some numbers
that were not used based on the xterm ctrl sequences.
The color assignment codes 58 and 59 are prior art from Kitty.
refs: https://github.com/wez/wezterm/issues/415
Use the scaling factor between the font metrics for the base font
and those of the fallback font selected for a given glyph.
The scenario is this: the base font is typically the first one selected
from the font configuration. There may be multiple fallback fonts that
are different sizes; for instance, the Font Awesome font has glyphs that
are square in aspect and are thus about twice the width of a typical
textual monospace font. Similarly, Noto Color Emoji is another square
font but that has a single set of bitmap strikes at a fixed 128 px
square.
The shaper returns advance metrics in the scale of the containing font,
and the rasterizer will target the supplied size and dpi.
We need to scale these to match the base metrics.
Previously we used a crude heuristic to decide whether to scale,
and that happened to work for Noto Color Emoji but not for Font Awesome,
whose metrics were just inside the bounds of the heuristic.
This commit allows retrieving the metrics for a given font_idx so
that we can compute the correct scale factor without any heuristics,
and applies that to the rasterized glyph.
refs: https://github.com/wez/wezterm/issues/342
This is one of those massive time sinks that I almost regret...
As part of recent changes to dust-off the allsorts shaper, I noticed
that the harfbuzz shaper wasn't shaping as well as the allsorts one.
This commit:
* Adds emoji-test.txt, a text file you can `cat` to see how well
the emoji are shaped and rendered.
* Fixes (or at least, improves) the column width calculation for
combining sequences such as "deaf man" which was previously calculated
at 3 cells in width when it should have just been 2 cells wide, which
resulted in a weird "prismatic" effect during rendering where the
glyph would be rendered with an extra RHS portion of the glyph across
3 cells.
* Improved/simplified the clustering logic used to compute fallbacks.
Previously we could end up with some wonky/disjoint sequence of
undefined glyphs which wouldn't be successfully resolved from a
fallback font. We now make a better effort to consolidate runs of
undefined glyphs for fallback.
* For sequences such as "woman with veil: dark skin tone" that occupy a
single cell, the shaper may return 3 clusters with 3 glyphs in the
case that the font doesn't fully support this grapheme. At render
time we'd just take the last glyph from that sequence and render it,
resulting in eg: a female symbol in this particular case. It is
generally a bit more useful to show the first glyph in the sequence
(eg: person with veil) rather than the gender or skin tone, so the
renderer now checks for this kind of overlapping sequence and renders
only the first glyph from the sequence.
@yoichi reports that:
```bash
printf "\x1bPqh"
```
would panic wezterm; the issue was that the maximum x value was
only being updated for newlines and that sequence didn't include it.
refs: #217
Adds some supporting methods for computing the `SemanticZone`s
in the display and a key assignment that allows scrolling the
viewport to jump to the next/prev Prompt zone.
This commit allows the terminal to tag cells with their semantic
type, as defined by OSC 133 escape sequences.
The gist of it is that each cell is now semantically one of:
* Output (eg: from the activity performed by the user. This is the
default)
* Input (eg: something that the user typed as input)
* Prompt (eg: "uninteresting" chrome/UI from the shell)
The semantic type is applied almost exactly like an SGR attribute,
except that resetting SGR doesn't clear the semantic type.
Tagging the cells in this way allows for smarter UX in the future;
for example, selecting the entire input or output from the last
command without fiddling around to avoid the prompt line(s),
or "paging up" to a prior prompt rather than page.
This doc covers those escapes as used in domterm, iterm2 and other
terminals:
https://gitlab.freedesktop.org/Per_Bothner/specifications/blob/master/proposals/semantic-prompts.md
This is an example of how to configure the shell to emit these
sequences; I'll add a proper little blob of shell specifically
for wezterm in a later commit:
https://github.com/PerBothner/DomTerm/blob/master/tools/shell-integration.zsh
Moved the image and hyperlink portion of CellAttributes out
to a separate heap structure, saving 8 bytes per Cell
for the common case of no hyperlink and no image.
Replaces SmallVec with an internal TeenyString that only
occupies a single machine word and avoids heap allocation
in the common case on most architectures. This takes the
textual portion of Cell from 32 bytes to 8 bytes.
This commit introduces a new `leader` configuration setting
that acts as a modal modifier key.
If leader is specified then pressing that key combination
will enable a virtual LEADER modifier.
While LEADER is active, only defined key assignments that include
LEADER in the `mods` mask will be recognized. Other keypresses
will be swallowed and NOT passed through to the terminal.
LEADER stays active until a keypress is registered (whether it
matches a key binding or not), or until it has been active for
the duration specified by `timeout_milliseconds`, at which point
it will automatically cancel itself.
Here's an example configuration using LEADER:
```lua
local wezterm = require 'wezterm';
return {
-- timeout_milliseconds defaults to 1000 and can be omitted
leader = { key="a", mods="CTRL", timeout_milliseconds=1000 },
keys = {
{key="|", mods="LEADER|SHIFT", action=wezterm.action{SplitHorizontal={domain="CurrentPaneDomain"}}},
-- Send "CTRL-A" to the terminal when pressing CTRL-A, CTRL-A
{key="a", mods="LEADER|CTRL", action=wezterm.action{SendString="\x01"}},
}
}
```
refs: https://github.com/wez/wezterm/issues/274
The code assumed that OSC parameters were always numeric, but that isn't
the case.
In order to allow adding non-numeric OSC code we need to adjust and
allow for the codes to be any string value, and that's what this
commit does.
It shouldn't change any other behavior.
This commit teaches the terminal model about the overline attribute
and the SGR codes to enable/disable overline.
The render layer in the GUI doesn't yet understand this attribute.
Now that we're reporting a higher level from DA1, apps are asking
more exotic codes. eg: vttest now asks about the conformance level,
but doesn't have a timeout on that request and hangs if we don't
respond.
This commit adds a bit of plumbing to make it easier to consume
and parse DCS sequences that are known to be short/short-lived,
and teaches the term layer to respond to a couple of possible
DECRQSS queries.
This is an xterm sequence that adjusts how the terminal
encodes keyboard output.
This commit teaches termwiz to parse and encode the sequence,
but doesn't teach the terminal emulator to do anything with it
at this time.
I'm adding this because vim sends these sequences and I wanted
to understand what they were for.
This commit adds support for left/right margins and has been
tested against esctest, with a final status of:
```
309 tests passed, 239 known bugs
```
"known bugs" also includes unimplemented features; we have a
similar degree as iTerm2.
As of this commit, we now report as a vt520ish machine to DA1.
I confess to not having read enough of the relevant docs
to know whether this is totally righteous.
We weren't parsing `CSI 2H`; we'd decide that because it needed two
parameters it was an error instead of defaulting the omitted second
parameter to 1.
Some applications want to control the cursor visibility, without affecting
cursor shape. Provide the `CursorVisibility` change type for this purpose.
Use this when dropping `UnixTerminal` to ensure the cursor is visible
without affecting the user's cursor shape.
Previously, we'd require boxing the entire DCS representation
in the escape sequence parser. This was due to some clippy
advice that made sense at the time, but now that I'm looking
at more things using DCS it seems wasteful to Box each byte of
the dcs sequence. This commit shifts the box to the heavier
weight portion: the DCS hook/entry representation.
When processing `\033[?1002;1003;1005;1006h`, when we encountered
`1005` we would skip 1006. This was because we hit the unspecified
enum case for 1005 (we have no enum variant for that mode) and that
code path didn't properly advance only by a single position.
This commit fixes that.
refs: https://github.com/wez/wezterm/issues/211#issuecomment-641302077
This corrects an issue where the mode byte of the DCS sequence was
discarded from the DcsHook, making it impossible to know what sequence
is being activated.
So far this hasn't come up as these sequences are relatively rare,
but in looking at sixel parsing I noticed the error.
This commit doesn't do anything specific to scrollback though!
It moves the implementation of the TermWizTermTab away from
a directly manipulated Surface and over to using the term::Terminal,
making the renderer look more like the one used by the local tab
and domain implementation.
As a side effect of doing this, we get scrollback management
for free.
refs: https://github.com/wez/wezterm/issues/201
We were treating DECCKM as the sole thing to enable application
cursor reporting, but looking closely at the docs, that mode only
takes effect when both DECANM (Vt52 emulation mode) AND DECKPAM
(application keypad mode) are both active.
neovim enables DECCKM and DECKPAM but not DECANM.
refs: https://github.com/wez/wezterm/issues/203
This commit adds a simple example of nested widgets, which uncovered
an issue with respecting the layout positioning, and adds a secondary
screen buffer to make the widget render cycle more optimal:
```
cargo run --example widgets_nested --features widgets --release
```
refs: https://github.com/wez/wezterm/issues/171
* ctrl-R to find a line and then hit enter would cause the search
text rather than the match text to be returned and run!
* When exiting the editor, clear to end of screen to make sure
that we clean up any UI from the incremental search status
This helps us keep track of the extent and cursor position that
we render for the line editor, making it easier to make the editor
rendering more fancy.
This restructures the LineEditor to allow the hosting application to
override key presses and apply custom edits to the editor buffer.
Methods for performing predefined actions and for accessing the line
buffer and cursor position have been provided/exposed to support this.
One consequence of this change is that the editor instance needs to be
passed through to the host trait impl and that means that the LineEditor
can no longer be generic over `Terminal`. Instead we now take `&mut dyn
Terminal` which was how the majority of non-example code was using it in
any case. This simplifies a bit of boilerplate in wezterm but adds an
extra line to the most basic examples.
On Windows, if you run `wsl.exe` from the terminal and start zsh
(maybe bash also?) and it enables application cursor key mode,
exiting zsh doesn't clear application cursor key mode and when we
return to the shell and are using virtual terminal input rather
than the native windows console input, we'll continue to receive
application cursor key sequences instead of regular cursor key
input sequences.
This commit recognizes both flavors as arrow movement
in the line editor to make this feel less broken.
Without this, `wzsh` will keep the terminal in raw mode between
line editor invocations, resulting in staggered/stairway output
for any spawned commands.
With the revised native windows console renderer using the various
console APIs more deeply, I've seen a couple of cases where those
API calls fail inside eg: wezterm running via the new pty machinery.
Using the virtual terminal APIs and the terminfo renderer is the
right thing to do in that case.
This commit probes for virtual terminal support and uses the builtin
xterm terminfo, unless the environment has
`TERMWIZ_BYPASS_VIRTUAL_TERMINAL=1` set. This allows forcing the
use of the windows console layer.
Some windows APIs have inclusive dimensions and some exclusive;
we were off by one for the height of the display which led to some
weirdness with eg: `sp` and the line editor.
When it comes to scrolling: if the scroll request is for the entire
viewport then we simply adjust the viewport; this is desirable because
it allows data to scroll back into the history in the native console.
This fixes the math around cursor positioning for the edge case where
the width of text and the cursor position are close to the width of
the terminal.
This reduces flickering updates in the native windows console;
it works by taking a copy of the screen buffer, applying the
Change's to that buffer and then copying back to the console.
This is unfortunately a bit of a muddy commit and I'm too lazy to split
it up.
* Removed `Position::NoChange`; use `Position::Relative(0)` instead
* Added missing cursor positioning cases in the terminfo renderer
* Taught line editor about the cursor position when the line spans
multiple physical lines
* Taught the Windows input layer to process escape sequences for eg:
the arrow keys when running with virtual terminal enabled.
* Removed the hack that under-reported the terminal width; the hack
was present to make some aspects of rendering with the native windows
console logic easier, but it was getting in the way of the line
editor. This may well break something, but it fixed up the line
editor :-/
cc: @markbt
This commit changes the behavior on Windows:
* If $TERM is set and the `terminfo` crate is able to
successfully initialize and locate a terminfo database (this also
requires that $TERMINFO be set in the environment), then we'll
use the `TerminfoRenderer` instead of the `WindowsConsoleRenderer`
* If $TERM is set to `xterm-256color` and no terminfo database was
found, use our modern compiled-in copy (look in the `termwiz/data/`
directory for the source and compiled version of this) and use
the `TerminfoRenderer`.
* Otherwise use the `WindowsConsoleRenderer`.
In practice, this allows termwiz apps to opt in to features such as
true color support on Windows 10 build 1903 an later by setting their
`TERM=xterm-256color`. This happens to be the default behavior when
`ssh`ing in to a windows host via `wezterm`.
You can see the truecolor mode get applied by running this example:
```
cargo run --example widgets_basic --features widgets
```
with TERM set as above the background region that is painted by the app
will be a blueish/purplish color, but with it unset or set to something
invalid, it will fall back to black.
I'd like to eventually make termwiz assume the equivalent configuration
to `TERM=xterm-256color` by default on Windows 10 build 1903 and later,
but it's worth getting some feedback on how this works for clients such
as `streampager`.
cc: @quark-zju and @markbt
Add an update indicator to the top right of client tabs; this is
overlaid on top of the surface when the last update from the server was
more than ~3s ago and if we expected it sooner than that.
While making this work, I noticed that the exponential poll backoff
had gotten broken in an earlier refactor; instead of a series of polls
backing off slowly, we were aggressively running the backoff up to the
max 30 second interval over the span of a few ms. This commit fixes
up the backoff computation to only happen when we are ready to send
a poll.
refs: https://github.com/wez/wezterm/issues/127
derive_builder has some extra dependencies that take a while to compile.
The builder feature can be expressed via a 30-line macro. So let's do
that to make termwiz compile faster.
The palette crate has a codegen step that translates svg_colors.txt to named.rs.
That makes it hard to build using buck.
Remove the palette dependency so termwiz is easier to build using buck.
I made sure the following code:
fn main() {
use termwiz::color::RgbColor;
let r = RgbColor::from_rgb_str("#02abcd").unwrap();
let r1 = r.to_tuple_rgba();
let r2 = r.to_linear_tuple_rgba();
println!("r1 = {:?}", r1);
println!("r2 = {:?}", r2);
}
prints
r1 = (0.007843138, 0.67058825, 0.8039216, 1.0)
r2 = (0.000607054, 0.4072403, 0.6104956, 1.0)
before and after the change.
Embed rgb.txt and parse it on the fly to produce the list of colors.
This list is a superset of palette's SVG color list.
refs: https://github.com/wez/wezterm/pull/144
This makes the input behavior consistent with posix: if SHIFT is held
and a letter key is pressed, make sure that we treat that as the ascii
uppercase version of that key and that the SHIFT modifier is cleared.
Adds logic to resize handling that will consider the original logical
line length when the width of the terminal is changed.
The intent is that this will cause the text to be re-flowed as if it had
been printed into the terminal at the new width. Lines that were
wrapped due to hittin the margin will be un-wrapped and made into a
single logical line, and then split into chunks of the new width.
This can cause new lines to be generated in the scrollback when
making the terminal narrower. To avoid losing the top of the buffer
in that case, the rewrapping logic will prune blank lines off the
bottom.
This is a pretty simplistic brute force algorithm: each of the lines
will be visited and split, and for large scrollback buffers this could
be relatively costly with a busy live resize. We don't have much choice
in the current implementation.
refs: https://github.com/wez/wezterm/issues/14
`cargo run --example widgets_basic --features widgets` changes the
cursor style but wasn't changing it back when exiting.
In addition, setting the cursor to Default was only restoring visibility
and not restoring the style.
I was running `hg log --config pager.pager=sp` and pressing space without
releasing it. After about 10k lines sp appears to deadlock. It seems sp uses a
single thread for both reading terminal events and sending wake events and it
sends too many wake events without reading the events.
Failing to write to the wake pipe due to EWOULDBLOCK does not seem to be a
problem - there are enough events in the pipe to wake up the other side.
Therefore let's just make wake_pipe_write nonblocking and treat EWOULDBLOCK as
a success.
Context: The stuck thread looks like:
#0 0x00007f32671ee237 in write () from /usr/lib/libc.so.6
#1 0x000055c466022823 in std::sys::unix::fd::FileDesc::write () at src/libstd/sys/unix/fd.rs:96
#2 std::sys::unix::net::Socket::write () at src/libstd/sys/unix/net.rs:276
#3 <&std::sys::unix::ext::net::UnixStream as std::io::Write>::write ()
at src/libstd/sys/unix/ext/net.rs:597
#4 <std::sys::unix::ext::net::UnixStream as std::io::Write>::write ()
at src/libstd/sys/unix/ext/net.rs:582
#5 0x000055c465d010a6 in termwiz::terminal::unix::UnixTerminalWaker::wake (self=0x7ffe6bd32de0)
at /home/quark/.cargo/git/checkouts/wezterm-6425bab852909cc8/ef1b836/termwiz/src/terminal/unix.rs:278
#6 0x000055c465a6c64b in streampager::event::EventSender::send_unique (self=0x7ffe6bd32dd0, event=...,
unique=0x7ffe6bd32de8) at src/event.rs:66
#7 0x000055c465a7e65a in streampager::display::start (term=..., term_caps=..., events=..., files=...,
error_files=..., progress=..., mode=streampager::config::FullScreenMode::Auto) at src/display.rs:295
#8 0x000055c465a453a7 in streampager::Pager::run (self=...) at src/lib.rs:171
#9 0x000055c465a0aced in sp::open_files (args=...) at src/bin/sp/main.rs:170
#10 0x000055c465a08e4f in sp::main () at src/bin/sp/main.rs:25
This commit adds some plumbing for describing the cursor shape
(block, line, blinking etc) and visibility, and feeds that through
the mux and render layers.
The renderer now knows to omit the cursor when it is not visible.
This isn't complete but begins the process of extracting
the embedding application configuration into a trait provided
by the application rather than passing the values in at
construction.
This allows the application to change configuration at
runtime.
The first option to handle this is the scrollback size.
The defaults are pretty neutral. You can get a little more fancy
with something like this:
```
[colors.tab_bar]
background = "#0b0022"
[colors.tab_bar.active_tab]
bg_color = "#2b2042"
fg_color = "#c0c0c0"
[colors.tab_bar.inactive_tab]
bg_color = "#1b1032"
fg_color = "#808080"
[colors.tab_bar.inactive_tab_hover]
bg_color = "#3b3052"
fg_color = "#909090"
italic = true
```
This is a bit of a large commit because it needed some plumbing:
* Change mux creation to allow deferring associating any domains,
and to change the default domain later in the lifetime of the
program
* De-bounce the empty mux detection to allow for transient windows
during early startup
* Implement a bridge between the termwiz client Surface and the
frontend gui renderer so that we can render from termwiz to
the gui.
* Adjust the line editor logic so that the highlight_line method
can change the length of the output. This enables replacing
the input text with placeholders so that we can obscure password
input
I noticed while scrolling `emoji-test.txt` that some of the combined
emoji sequences rendered very poorly. This was due to the unicode
width being reported as up to 4 in some cases.
Digging into it, I discovered that the unicode width crate uses a
standard calculation that doesn't take emoji combination sequences
into account (see https://github.com/unicode-rs/unicode-width/issues/4).
This commit takes a dep on the xi-unicode crate as a lightweight way
to gain access to emoji tables and test whether a given grapheme is
part of a combining sequence of emoji.
I've noticed this off and on for a while, and thought it was something
fishy with my shell dotfiles.
Tracing through I found that the final byte in the "Face with head
bandage" emoji 🤕 U+1F915 was being interpreted as the MW control
code and causing the vt parser to jump out of the OSC state.
The solution for this is to hook up proper UTF-8 processing in the
same way that it is applied in the ground state.
Since we don't have enough bits to introduce new state values (we're
pretty tightly packed in the 16 bits available), I've introduced a
memory of the state to which the utf8 parser needs to return once
a complete sequence is detected.
I thought that I'd broken something with the DEL processing in vim with
the new frontend but it turned out that the other frontend was emitting
BS always and that I'd actuall unbroken passing DEL through and that
other layers were translating DEL into an application cursor mode output
for DEL that emits a totally different sequence.
This diff preserves DEL and disables that other sequence.
Will follow up with some explicit configuration to control this
behavior, but in the short term, the default behavior should be much
closer to what people actually want and expect!
refs: https://github.com/wez/wezterm/issues/52
This is still a bit rough because the terminal parser doesn't
understand the pixel sizes, so it relies on the hard coded
cell dimensions being accurate.
This enables using large OSC buffers in a form that we can publish
to crates.io without blocking on an external crate. Large OSC
buffers are important both for some tunnelling use cases and for
eg: iTerm2 image protocol handling.
Add a convenience function to the escape parser that, like `parse_first`,
matches only the first escape sequence, but instead collects all matching
actions.
It's taking a while for https://github.com/jwilm/vte/pull/20 to get
merged, so point to my branch directly while I build out some
tunneled mux protocol escape sequences.
I'll need to fork vte on crates.io if vte doesn't merge the PR
before the next termwiz crate bump.
The `flush_pending_attr` method does lots of unnecessary comparisons.
If the attributes have changed, then it works by resetting the
attributes and then setting new values. There's no need to emit the
codes for exiting modes.
It also doesn't support double underscore or rapid blink in the cases
where the terminfo capabilities are used, as these capabilities can't
express these attributes. Fall back to CSI sequences when these
attributes are requested.
Changing the terminal attributes (bold, underline, etc.) involves
emitting the `exit_attribute_mode` or SGR reset sequence. This also
resets the colors back to their defaults. If this happens when the
foreground or background colors haven't changed, set the colors again.
These codes are used to change the color palette, but if the `?`
string is used in place of a color spec, then we must respond with
the current color value string for that palette entry, so lets
implement that!
