Two problems:
* Need reordered_runs method to populate ranges based on
the reordered levels!
* Use reordered runs to get the *logical* bounds of those
runs and pass those to harfbuzz.
Now the text is ordered correctly, but the rendering advances
by the wrong amount for the reordered clusters and looks bad
unless experimental_pixel_positioning=true.
refs: #784
This commit is larger than it appears to due fanout from threading
through bidi parameters. The main changes are:
* When clustering cells, add an additional phase to resolve embedding
levels and further sub-divide a cluster based on the resolved bidi
runs; this is where we get the direction for a run and this needs
to be passed through to the shaper.
* When doing bidi, the forced cluster boundary hack that we use to
de-ligature when cursoring through text needs to be disabled,
otherwise the cursor appears to push/rotate the text in that
cluster when moving through it! We'll need to find a different
way to handle shading the cursor that eliminates the original
cursor/ligature/black issue.
* In the shaper, the logic for coalescing unresolved runs for font
fallback assumed LTR and needed to be adjusted to cluster RTL.
That meant also computing a little index of codepoint lengths.
* Added `experimental_bidi` boolean option that defaults to false.
When enabled, it activates the bidi processing phase in clustering
with a strong hint that the paragraph is LTR.
This implementation is incomplete and/or wrong for a number of cases:
* The config option should probably allow specifying the paragraph
direction hint to use by default.
* https://terminal-wg.pages.freedesktop.org/bidi/recommendation/paragraphs.html
recommends that bidi be applied to logical lines, not physical
lines (or really: ranges within physical lines) that we're doing
at the moment
* The paragraph direction hint should be overridden by cell attributes
and other escapes; see 85a6b178cf
and probably others.
However, as of this commit, if you `experimental_bidi=true` then
```
echo This is RTL -> عربي فارسی bidi
```
(that text was sourced from:
https://github.com/microsoft/terminal/issues/538#issuecomment-677017322)
then wezterm will display the text in the same order as the text
renders in Chrome for that github comment.
```
; ./target/debug/wezterm --config experimental_bidi=false ls-fonts --text "عربي فارسی ->"
LeftToRight
0 ع \u{639} x_adv=8 glyph=300 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
2 ر \u{631} x_adv=3.78125 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
4 ب \u{628} x_adv=4 glyph=244 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
6 ي \u{64a} x_adv=4 glyph=363 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
8 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
9 ف \u{641} x_adv=11 glyph=328 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
11 ا \u{627} x_adv=4 glyph=240 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
13 ر \u{631} x_adv=3.78125 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
15 س \u{633} x_adv=10 glyph=278 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
17 ی \u{6cc} x_adv=4 glyph=664 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
19 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
20 - \u{2d} x_adv=8 glyph=276 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
21 > \u{3e} x_adv=8 glyph=338 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
```
```
; ./target/debug/wezterm --config experimental_bidi=true ls-fonts --text "عربي فارسی ->"
RightToLeft
17 ی \u{6cc} x_adv=9 glyph=906 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
15 س \u{633} x_adv=10 glyph=277 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
13 ر \u{631} x_adv=4.78125 glyph=272 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
11 ا \u{627} x_adv=4 glyph=241 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
9 ف \u{641} x_adv=5 glyph=329 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
8 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
6 ي \u{64a} x_adv=9 glyph=904 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
4 ب \u{628} x_adv=4 glyph=243 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
2 ر \u{631} x_adv=5 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
0 ع \u{639} x_adv=6 glyph=301 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
LeftToRight
0 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
1 - \u{2d} x_adv=8 glyph=480 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
2 > \u{3e} x_adv=8 glyph=470 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
;
```
refs: https://github.com/wez/wezterm/issues/784
This commit decomposes the main get_semantic_zones method into two
parts:
* A per-line portion, where the line ranges are cached (invalidated on
change)
* The overall screen portion, where the line ranges are merged
This changes the overall complexity of computing zones from
O(width * scrollback-height)
To an incremental:
O((width * number of changed lines since last query) + scrollback-height)
You can see some samples of elapsed time below; those show the times for
running both the old and the new implementation on the same data. The
number of lines/zones in the scrollback increases with each call and you
can see that the new implementation is a bit faster anyway at low
volumes but is significantly faster as the number of lines/zones
increases, because the amount of work is reduced.
```
get_semantic_zones: 71.708µs
get_semantic_zones_new: 59.041µs
get_semantic_zones: 71.166µs
get_semantic_zones_new: 9.166µs
get_semantic_zones: 44.291µs
get_semantic_zones_new: 4.208µs
get_semantic_zones: 69.791µs
get_semantic_zones_new: 10.291µs
get_semantic_zones: 59.375µs
get_semantic_zones_new: 7.958µs
get_semantic_zones: 52.5µs
get_semantic_zones_new: 4.5µs
get_semantic_zones: 91.791µs
get_semantic_zones_new: 20.916µs
get_semantic_zones: 229.916µs
get_semantic_zones_new: 109.208µs
get_semantic_zones: 224.125µs
get_semantic_zones_new: 15.208µs
get_semantic_zones: 291.791µs
get_semantic_zones_new: 11.833µs
get_semantic_zones: 238.875µs
get_semantic_zones_new: 12.625µs
get_semantic_zones: 468.458µs
get_semantic_zones_new: 126.583µs
get_semantic_zones: 460.5µs
get_semantic_zones_new: 25.666µs
get_semantic_zones: 358.291µs
get_semantic_zones_new: 19.541µs
get_semantic_zones: 436.833µs
get_semantic_zones_new: 17.875µs
get_semantic_zones: 313.166µs
get_semantic_zones_new: 15.25µs
get_semantic_zones: 333.958µs
get_semantic_zones_new: 16.541µs
get_semantic_zones: 364.666µs
get_semantic_zones_new: 14.041µs
```
I generated nerdfonts_data.rs with this shell script; it uses `i_all.sh`
from the nerdfonts repo to get the base mapping:
```
source ./lib/i_all.sh
echo "//! Data mapping nerd font symbol names to their char codepoints"
echo "pub const NERD_FONT_GLYPHS: &[(&str, char)] = &["
for var in "${!i@}"; do
# trim 'i_' prefix
glyph_name=${var#*_}
glyph_char=${!var}
glyph_code=$(printf "%x" "'$glyph_char'")
echo "(\"$glyph_name\", '\u{$glyph_code}'), // $glyph_char"
done
echo "];"
```
Then intent is to use it in wezterm:
```
local wezterm = require 'wezterm'
wezterm.log_info(wezterm.nerdfonts.dev_mozilla)
```
We need 100% of the info for it to work correctly, so this commit:
* Exposes the keyboard encoding mode via the Pane trait
* Adds the scan code to the RawKeyEvent
* Has the GUI perform the encoding if the keyboard is set that way
* Removes the basic encoder from termwiz in favor of the gui level one
The net result is that we bypass the Pane::key_up/Pane::key_down methods
in almost all cases when the encoding mode is set to win32-input-mode.
There is now a config option: allow_win32_input_mode that can be
used to prevent using this mode.
refs: #1509
This commit causes the terminal to emit win32-input-mode encoded key up
and down events for a limited subset of keys When win32-input-mode is
enabled.
We limit them to keys where we know the VK key code equivalent,
and where those keys are either not representable (eg: modifier
only key events), or may generate ambiguous output (eg: CTRL-SPACE
in different keyboard layouts).
However, in my experiments, modifier only key presses confuse powershell
and cause it to emit `@`, so I've disabled that in the code for now.
refs: https://github.com/wez/wezterm/issues/318
refs: https://github.com/wez/wezterm/issues/1509
refs: https://github.com/wez/wezterm/issues/1510
This is a baby step that formalizes the different encoding schemes into
an enum, and hooks up the decset sequence for win32-input-mode.
It doesn't change any of the actual encoding at this time.
refs: #1509
When rendering the IME composing text, I noticed that for the Korean
input sequence: shift+'ㅅ' followed by 'ㅏ' we'd render the 'ㅆ' (the
shifted first character) in black and the composing 'ㅏ' in white
against the cursor color, and that was very difficult to read,
especially at the default font size.
To resolve this, this commit:
* Forces clustering to break around the cursor boundary, so that
we treat the cursor position as its own separately styled cluster
* Adjusts cursor/bg rendering so that we always consider the start of
the cluster for the colors of that run. We are guaranteed that a
ligatured sequence will fit in the background area anyway.
This has the effect of "breaking" programming ligatures such as '->'
when cursoring through them, and decomposing them into their individual
'-' and '>' glyphs, which is a reasonable price to pay for being able
to see things better on screen.
refs: https://github.com/wez/wezterm/issues/1504
refs: https://github.com/wez/wezterm/issues/478
Previously, we would implicitly set it to the special SEQ_ZERO
value, but since that value always flags the row as changed,
it causes some over-invalidation issues downstream in wezterm.
This commit makes that parameter required, so that the code that
is creating a new Line always passes down the seqno from that event.
refs: #1472
Not 100% sure why this only really manifested on Windows, but
the symptoms were:
* Run powershell in a tab
* Run `dir`
* Hit enter a couple of times to show a couple of prompts
* Try using the mouse to select across the prompt boundaries
The selection would get invalidated crossing the boundaries.
I traced this down to the lines around those regions having
SEQ_ZERO as their sequence, so this commit ensures that lines
that are created as part of scrolling the screen are correctly
tagged with the current seqno from the terminal display.
Why only windows? Not totally sure; perhaps it is related to
something funky happening in the conpty layer and sending us
unusual escapes (eg: scroll margins?)
permits iTerm2 images to be drawn anywhere on screen without
scrolling the cursor, including the bottom row.
Also included is a check in fcwrap.rs to_range_set(), without which
was causing a panic at runtime due to subtraction from unsigned
leading to overflow.
This helps us correctly set the size of the image cell
for the case where we have a partial cell at the right/bottom
edge of an image being mapped across cells.
refs: #1270
We need to force the codepage to UTF8 this to avoid having our UTF-8
bytestreams be misinterpreted, and to restore the original before we're
done.
refs: https://github.com/wez/wezterm/issues/1435
implement missing alt-screen support and fixup building the examples
while we're in here.
refs: #1244
This commit teaches `RgbColor::from_rgb_str` to support
colors in the form `hsl:235 100 50`, an HSL colorspace
color specification.
While banging my head on why my test wasn't passing, I realized
that this was producing 10 bpc color and the code to convert
those to RGB was incorrectly multiplying conversion terms!
refs: https://github.com/wez/wezterm/issues/1436
From esctest:
CUPTests.test_CUP_ColumnOnly
HVPTests.test_HVP_ColumnOnly
and the newly added at https://invent.kde.org/ninjalj/esctest.git:
DECSETTests.test_DECSET_DECLRMM_OnlyRight
DECSTBMTests.test_DECSTBM_OnlyBottom
Not all codepoints are valid when combined with a presentation
selector.
This commit ensures that we respect the valid sequences defined
by the current version of unicode (version 14).
refs: #1231
refs: #997
As promised in the previous commit, this one implements an escape
sequence to control the unicode version.
Unknown to me in the previous commit, iTerm2 already defines such
an escape sequence, so we simply implement it here with the same
semantics.
refs: #1231
refs: #997
This is a fairly far-reaching commit. The idea is:
* Introduce a unicode_version config that specifies the default level
of unicode conformance for each newly created Terminal (each Pane)
* The unicode_version is passed down to the `grapheme_column_width`
function which interprets the width based on the version
* `Cell` records the width so that later calculations don't need to
know the unicode version
In a subsequent diff, I will introduce an escape sequence that allows
setting/pushing/popping the unicode version so that it can be overridden
via eg: a shell alias prior to launching an application that uses a
different version of unicode from the default.
This approach allows output from multiple applications with differing
understanding of unicode to coexist on the same screen a little more
sanely.
Note that the default `unicode_version` is set to 9, which means that
emoji presentation selectors are now by-default ignored. This was
selected to better match the level of support in widely deployed
applications.
I expect to raise that default version in the future.
Also worth noting: there are a number of callers of
`unicode_column_width` in things like overlays and lua helper functions
that pass `None` for the unicode version: these will assume the latest
known-to-wezterm/termwiz version of unicode to be desired. If those
overlays do things with emoji presentation selectors, then there may be
some alignment artifacts. That can be tackled in a follow up commit.
refs: #1231
refs: #997
`use_fancy_tab_bar` switches to an alternate rendering of the tab
bar that uses the window_frame config to get a proportional
title font to use to render tabs, as well as rendering a few
additional elements to space out and make the tabs feel more
like tabs.
Computing the number of tabs doesn't respect the alternate font
at this time.
Formatted tab item foreground and background colors are also
not respected at this time.
refs: #1180
This was a bit of a PITA to run down; the essence of the problem
was that the shaper was returning an x_advance of 0 for U+3000,
which caused wezterm's shaping layer to elide that glyph.
I eventually tracked down the x_advance to be the result of
scaling by an x_scale of 0, which in turn is the result of
harfbuzz not knowing the font size.
The critical portion of this diff is the line that advises
harfbuzz that the font has changed after we've applied the
font size.
The rest is just stuff to make it easier to debug and verify.
This:
```
printf "x\u3000x."
```
Now correctly renders on screen as "x x".
fixes: #1161
I noticed while reviewing the session recording for https://github.com/wez/wezterm/issues/1102
that wezterm reported:
```
2021-09-04T22:05:46.136Z WARN wezterm_term::terminalstate::performer > unknown unspecified CSI: "58::2::87::61::38m"
2021-09-04T22:05:46.138Z WARN wezterm_term::terminalstate::performer > unknown unspecified CSI: "58::2::87::61::38m"
```
Those extra double-colons aren't valid, and I think they're coming from
this terminfo file, so fix it up!
This will enable eg: a lua helper function to serialize keycodes to
assist in some key rebinding scenarios (see:
https://github.com/wez/wezterm/pull/1091#issuecomment-910940833 for the
gist of it) but also makes it a bit easier to write unit tests for key
encoding so that situations like those in #892 are potentially less
likely to occur in the future.
If a cell contained a ligature, the math used to track where the
next quad was going to be placed could lose consistency with
the cluster/shaping information and result in offset glyphs.
This was most noticeable to me in tab titles in the tab bar;
my shell dotfiles append `-- something` to the title when a
command is launched, and the `--` is a ligature in my font.
I think I've also seen this mess up positioning in the notcurses
demo as well.
The solution is to take the cluster initial cell index rather
than trying to reverse engineer it from incomplete info.
A while back I made the line lengths lazily grown; the reduction
in memory was nice, and it helped with render performance for
really wide screens.
Unfortunately, it puts a bunch of reallocation into the hot path
of the parser and updating the terminal model when people run
the inevitable `cat giant-file.txt` benchmark.
This commit reinstates pre-allocating lines to match the physical
terminal width, and tweaks the code a bit to take advantage of
const Cell allocation and to avoid some clones (a really micro
optimization).
For simple graphemes, we can avoid subsequently calling
grapheme_column_width and cache that information in TeenyString.
Make blank TeenyString and Cell initializations const.
Terminal now maintains a sequence number that increments
for each Action that is applied to it.
Changes to lines are tagged with the current sequence number.
This makes it a bit easier to reason about when an individual
line has changed relative to some point in "time"; the consumer
of the terminal can sample the current sequence number and then
can later determine which lines have changed since that point
in time.
refs: https://github.com/wez/wezterm/issues/867
An implementation detail in wezterm is that it doesn't model
image placements as a separate entity; they are all bound to
the image cells in the terminal model.
The semantics of the kitty image protocol are that placements
are "permanent" wrt. overwriting a cell with text, except for
the explicit EraseInLine/EraseInDisplay sequences that are
used for clearing.
This commit takes a pass at implementing that semantic in
the wezterm data model.
refs: #986
This should give the shaper a better chance at using text
presentation in a run that mixes emoji with text and/or
uses presentation selectors.
It also exposes the presentation property to the shaper
so that it could potentially adjust its fallback strategy.
However, it doesn't do that here.
refs: https://github.com/wez/wezterm/issues/997
Make a distinction between default and selected presentation,
and account for that in the cell width.
Add a method to the cell that returns the effective presentation.
refs: https://github.com/wez/wezterm/issues/997
These modifiers have the effect of forcing us to consider the grapheme
as being either a single cell (VS15) or two cells (VS16) in the
terminal model.
These don't affect font choice as wezterm doesn't know whether a given
font in the fallback has a textual vs. an emoji version of a given
glyph, or whether a later font in the fallback has one or the other
because we can't know until we fall back, and that has a very high
cost--we perform fallback asynchronously in another thread because
of its high cost.
Depending on the selected glyph, it may or may not render as double
wide.
refs: #997
Since we can now mutate individual frames, we need to avoid
falsely caching across a change; switch from using (image_id, frame_idx)
to frame_hash.
refs: #986
Adds a use_image feature to termwiz that enables an optional
dep on the image crate. This in turn allows decoding of animation
formats (gif, apng) from file data, but more crucially, allows
modeling animation frames at the termwiz layer, which is a pre-req
for enabling kitty img protocol animation support.
refs: #986
I noticed when running the notcurses demo that we're spending a
decent amount of time decoding png data whenever we need to
re-do the texture atlas.
Let's avoid that by allowing for ImageData at the termwiz layer
to represent both the image file format and decoded rgba8 data.
This commit is a bit muddy and also includes some stuff to try
to delete placements from the model. It's not perfect by any
means--more expensive than I want, and there's something funky
that causes a large number of images to build up during some
phases of the demo.
refs: #986
This isn't complete; many of the placement options are not supported,
and the status reporting is missing in a number of cases, including
querying/probing, and shared memory objects are not supported yet.
However, this commit is sufficient to allow the kitty-png.py script
(that was copied from
https://sw.kovidgoyal.net/kitty/graphics-protocol/#a-minimal-example)
to render a PNG in the terminal.
This implementation routes the basic image display via the same
code that we use for iterm2 and sixel protocols, but it isn't
sufficient to support the rest of the placement options allowed
by the spec.
Notably, we'll need to add the concept of image placements to
the data model maintained by the terminal state and find a way
to efficiently manage placements both by id and by a viewport
range.
The renderer will need to manage separate quads for placements
and order them by z-index, and adjust the render phases so that
images can appear in the correct plane.
refs: #986
This teaches termwiz to recognize and encode the APC
sequences used by the kitty image protocol.
This doesn't include support for animations, just the
transmit, placement and delete requests.
refs: #986
These were parsed but swallowed. This commit expands the transitions
to be able to track the APC start, data and end and then adds
an `apc_dispatch` method to allow capturing APC sequences.
APC sequences are used in the kitty image protocol.
refs: #986
This commit hooks up DECRQM so that we can report that we implement
synchronized updates, and then refines the code that manages sending
data to the terminal model; the first cut at synchronized updates
was a bit simplistic, and now we make a point of "flushing" pending
actions when we start a sync point, and again as soon as we release
the sync point.
This smooths out the jaggies around the orca that I mentioned in
dcbbda7702
and while testing this, I realized that recent parser changes had
mangled processing bundled dec private mode sequences where multiple
modes were specified in the same overall escape sequence. I've
added the missing unit test case for this and made that work again.
refs: https://github.com/wez/wezterm/issues/955
refs: https://github.com/wez/wezterm/issues/882
This commit removes the intermediates parameter and collapses it
together with the parameters themselves.
This allows us to model DECSET (eg: `CSI ? 1 l`) correctly.
Previously this would get reported as:
```
params: [1],
intermediates: ['?'],
code: 'l'
```
but since the intermediates are logically things that precede the code,
the canonical interpretation of that would be as if we'd received
`CSI 1 ? l`.
AFAICT, DECSET isn't conforming to ECMA 48 when it comes to this
sequence.
That made things a bit of a headache in the CSI parser, so what we do
now is to treat intermediates as parameters so that it is much simpler
to reason about and match in the CSI parser; we now get:
```
params: ['?', 1],
code: 'l',
```
refs: https://github.com/wez/wezterm/issues/955
The original design of the vtparse crate was inspired by the vte
crate. There were some assumptions about the shape of CSI sequences
that were lossy and that is posing a problem when it comes to
implementing DECRQM.
This commit improves the situation by adjusting CsiParam to be capable
of capturing all of the possible parameters as well as intermediates.
This commit isn't done; I just need to push it to transfer it to another
machine.
refs: https://github.com/wez/wezterm/issues/882
refs: https://github.com/wez/wezterm/issues/955
This implementation doesn't include a timeout, but should be
recoverable via a SoftReset.
There's no query response either: I think I'm missing DECRQM
entirely at the moment.
refs: https://github.com/wez/wezterm/issues/882
4e99edf6f5 split the color into
thin/fat components, but overlooked copying the optional fat
part of the color.
That in turn broke sequences like:
* Set background color
* Erase to end of line
This commit teaches the sgr attribute cloning function to look
at the optional fat component.
refs: #962
Add the third device attributes (DA3) query+reply,
eliminating "unknown/unspecified CSI" error. Like
XTerm, simply reply with zeroes as opposed to site
codes or unique IDs.
https://vt100.net/docs/vt510-rm/DA3.html
* [terminfo] add smxx/rmxx, purge blink
Testing with WezTerm nightly indicates no blink support
available through the xterm \E[5m, and I didn't see any
support in the code -- purge it. smxx and rmxx, meanwhile,
have been added to indicate support for struck text.
* restore blink
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
The common palette indices are in the main cell attributes.
Using true color will allocate fat storage.
This allows reducing the Cell size from 32 -> 24 across the
implementation of storing 10 bpc color (it peaked at 40 in
the last couple of commits).
This adjusts the cursor position after emitting a sixel.
@dankamongmen: I don't have much of a sixel test suite to speak
of (cat snake.six :-p); I'd appreciate it if you could run
notcurses against this and confirm that it is doing something
sane!
At the very least, we shouldn't be warning about the unhandled
mode any more!
refs: https://github.com/wez/wezterm/issues/863
refs: https://github.com/dankamongmen/notcurses/issues/1743
Rust hasn't stabilized fallible allocation, so when we are presented
with an implausibly large sixel, Rust terminates the program; it's not
even a panick that we could potentially catch -> direct to termination.
This commit introduces an arbitrary constraint so that we can
avoid unconditionally terminating for this bad input case.
Thanks to @klamonte for sharing this test case!
A relatively recent commit made it possible for the cell storage
to be smaller than the viewport width.
Avoid panicking when erasing beyond the end of the cell storage.
We can simply NOP as that cell is already implictly erased.
closes: #847
ProbeHints would unconditionally load the terminfo based on $TERM
whever it was created, even if the caller had a pre-supplied
terminfo buffer.
Defer loading that until the caps instance is created.
refs: #817
There were two bugs here:
* \u8D (the utf8 encoded representation of 0x8d, aka: RI) was not
recognized as a C1 code and was instead passed through as printable
text.
* The \u8D is a zero-width sequence which means that a subsequent
set_cell call on the new empty-by-default line wouldn't allocate
any cells in the line array, and the assigment to the line would
panic.
This commit avoids the panic for the second case, and then fixes up
the vtparser to correctly recognize the sequence as a C1 control.
refs: https://github.com/wez/wezterm/issues/768
I noticed edenscm log -p (using streampager) cannot render 256 colors
if TERM is not "*-256color", despite forcing enabling true colors in
streampager (https://github.com/markbt/streampager/pull/28).
I tracked it down here. The problem is that we ask terminfo for colors
it does not claim to support. Fix it by using fallback CSI rendering
for colors exceeding the terminfo max color.
Using a boxed slice means that we hold exactly the memory required
for the file data, rather than the next-power-of-two, which can
be wasteful when a large number of images are being sent to
the terminal.
This is a API breaking change for termwiz, so bump its version.
refs: #534
```
echo -e "\033]777;notify;This is the notification title;This is the notification text\a"
```
Now pops up a notification in a similar manner to OSC 9, except
that this form allows setting both the title and the body separately.
refs: https://github.com/wez/wezterm/issues/489
I started this a while ago; it's pretty time consuming to produce
accessible and usable documentation for this sort of stuff, so
this isn't yet complete, but in the interest of avoiding additional
bit-rot, let's get this up.
refs: https://github.com/wez/wezterm/issues/257
It's been replaced with an opaque termwiz error type instead.
This is a bit of a more conservative approach than that in (refs: #407)
and has less of an impact on the surrounding code, which appeals to
me from a maintenance perspective.
refs: #406
refs: #407
You can install this into your $TERMINFO directory (default is
`$HOME/.terminfo`) by running:
`tic -x wezterm.terminfo`
from this data directory.
Once installed, you can set `TERM=wezterm`.
refs: https://github.com/wez/wezterm/issues/415
This allows us to support the kitty style underline sequence,
or the : separated form of the true color escape sequences.
refs: https://github.com/wez/wezterm/issues/415
These aren't currently rendered, but the parser and model now support
recognizing expanded underline sequences:
```
CSI 24 m -> No underline
CSI 4 m -> Single underline
CSI 21 m -> Double underline
CSI 60 m -> Curly underline
CSI 61 m -> Dotted underline
CSI 62 m -> Dashed underline
CSI 58 ; 2 ; R ; G ; B m -> set underline color to specified true color RGB
CSI 58 ; 5 ; I m -> set underline color to palette index I (0-255)
CSI 59 -> restore underline color to default
```
The Curly, Dotted and Dashed CSI codes are a wezterm assignment in the
SGR space. This is by no means official; I just picked some numbers
that were not used based on the xterm ctrl sequences.
The color assignment codes 58 and 59 are prior art from Kitty.
refs: https://github.com/wez/wezterm/issues/415
Use the scaling factor between the font metrics for the base font
and those of the fallback font selected for a given glyph.
The scenario is this: the base font is typically the first one selected
from the font configuration. There may be multiple fallback fonts that
are different sizes; for instance, the Font Awesome font has glyphs that
are square in aspect and are thus about twice the width of a typical
textual monospace font. Similarly, Noto Color Emoji is another square
font but that has a single set of bitmap strikes at a fixed 128 px
square.
The shaper returns advance metrics in the scale of the containing font,
and the rasterizer will target the supplied size and dpi.
We need to scale these to match the base metrics.
Previously we used a crude heuristic to decide whether to scale,
and that happened to work for Noto Color Emoji but not for Font Awesome,
whose metrics were just inside the bounds of the heuristic.
This commit allows retrieving the metrics for a given font_idx so
that we can compute the correct scale factor without any heuristics,
and applies that to the rasterized glyph.
refs: https://github.com/wez/wezterm/issues/342
This is one of those massive time sinks that I almost regret...
As part of recent changes to dust-off the allsorts shaper, I noticed
that the harfbuzz shaper wasn't shaping as well as the allsorts one.
This commit:
* Adds emoji-test.txt, a text file you can `cat` to see how well
the emoji are shaped and rendered.
* Fixes (or at least, improves) the column width calculation for
combining sequences such as "deaf man" which was previously calculated
at 3 cells in width when it should have just been 2 cells wide, which
resulted in a weird "prismatic" effect during rendering where the
glyph would be rendered with an extra RHS portion of the glyph across
3 cells.
* Improved/simplified the clustering logic used to compute fallbacks.
Previously we could end up with some wonky/disjoint sequence of
undefined glyphs which wouldn't be successfully resolved from a
fallback font. We now make a better effort to consolidate runs of
undefined glyphs for fallback.
* For sequences such as "woman with veil: dark skin tone" that occupy a
single cell, the shaper may return 3 clusters with 3 glyphs in the
case that the font doesn't fully support this grapheme. At render
time we'd just take the last glyph from that sequence and render it,
resulting in eg: a female symbol in this particular case. It is
generally a bit more useful to show the first glyph in the sequence
(eg: person with veil) rather than the gender or skin tone, so the
renderer now checks for this kind of overlapping sequence and renders
only the first glyph from the sequence.
@yoichi reports that:
```bash
printf "\x1bPqh"
```
would panic wezterm; the issue was that the maximum x value was
only being updated for newlines and that sequence didn't include it.
refs: #217
Adds some supporting methods for computing the `SemanticZone`s
in the display and a key assignment that allows scrolling the
viewport to jump to the next/prev Prompt zone.
This commit allows the terminal to tag cells with their semantic
type, as defined by OSC 133 escape sequences.
The gist of it is that each cell is now semantically one of:
* Output (eg: from the activity performed by the user. This is the
default)
* Input (eg: something that the user typed as input)
* Prompt (eg: "uninteresting" chrome/UI from the shell)
The semantic type is applied almost exactly like an SGR attribute,
except that resetting SGR doesn't clear the semantic type.
Tagging the cells in this way allows for smarter UX in the future;
for example, selecting the entire input or output from the last
command without fiddling around to avoid the prompt line(s),
or "paging up" to a prior prompt rather than page.
This doc covers those escapes as used in domterm, iterm2 and other
terminals:
https://gitlab.freedesktop.org/Per_Bothner/specifications/blob/master/proposals/semantic-prompts.md
This is an example of how to configure the shell to emit these
sequences; I'll add a proper little blob of shell specifically
for wezterm in a later commit:
https://github.com/PerBothner/DomTerm/blob/master/tools/shell-integration.zsh
Moved the image and hyperlink portion of CellAttributes out
to a separate heap structure, saving 8 bytes per Cell
for the common case of no hyperlink and no image.
Replaces SmallVec with an internal TeenyString that only
occupies a single machine word and avoids heap allocation
in the common case on most architectures. This takes the
textual portion of Cell from 32 bytes to 8 bytes.
This commit introduces a new `leader` configuration setting
that acts as a modal modifier key.
If leader is specified then pressing that key combination
will enable a virtual LEADER modifier.
While LEADER is active, only defined key assignments that include
LEADER in the `mods` mask will be recognized. Other keypresses
will be swallowed and NOT passed through to the terminal.
LEADER stays active until a keypress is registered (whether it
matches a key binding or not), or until it has been active for
the duration specified by `timeout_milliseconds`, at which point
it will automatically cancel itself.
Here's an example configuration using LEADER:
```lua
local wezterm = require 'wezterm';
return {
-- timeout_milliseconds defaults to 1000 and can be omitted
leader = { key="a", mods="CTRL", timeout_milliseconds=1000 },
keys = {
{key="|", mods="LEADER|SHIFT", action=wezterm.action{SplitHorizontal={domain="CurrentPaneDomain"}}},
-- Send "CTRL-A" to the terminal when pressing CTRL-A, CTRL-A
{key="a", mods="LEADER|CTRL", action=wezterm.action{SendString="\x01"}},
}
}
```
refs: https://github.com/wez/wezterm/issues/274
The code assumed that OSC parameters were always numeric, but that isn't
the case.
In order to allow adding non-numeric OSC code we need to adjust and
allow for the codes to be any string value, and that's what this
commit does.
It shouldn't change any other behavior.
This commit teaches the terminal model about the overline attribute
and the SGR codes to enable/disable overline.
The render layer in the GUI doesn't yet understand this attribute.
Now that we're reporting a higher level from DA1, apps are asking
more exotic codes. eg: vttest now asks about the conformance level,
but doesn't have a timeout on that request and hangs if we don't
respond.
This commit adds a bit of plumbing to make it easier to consume
and parse DCS sequences that are known to be short/short-lived,
and teaches the term layer to respond to a couple of possible
DECRQSS queries.
This is an xterm sequence that adjusts how the terminal
encodes keyboard output.
This commit teaches termwiz to parse and encode the sequence,
but doesn't teach the terminal emulator to do anything with it
at this time.
I'm adding this because vim sends these sequences and I wanted
to understand what they were for.
This commit adds support for left/right margins and has been
tested against esctest, with a final status of:
```
309 tests passed, 239 known bugs
```
"known bugs" also includes unimplemented features; we have a
similar degree as iTerm2.
As of this commit, we now report as a vt520ish machine to DA1.
I confess to not having read enough of the relevant docs
to know whether this is totally righteous.
We weren't parsing `CSI 2H`; we'd decide that because it needed two
parameters it was an error instead of defaulting the omitted second
parameter to 1.
Some applications want to control the cursor visibility, without affecting
cursor shape. Provide the `CursorVisibility` change type for this purpose.
Use this when dropping `UnixTerminal` to ensure the cursor is visible
without affecting the user's cursor shape.
Previously, we'd require boxing the entire DCS representation
in the escape sequence parser. This was due to some clippy
advice that made sense at the time, but now that I'm looking
at more things using DCS it seems wasteful to Box each byte of
the dcs sequence. This commit shifts the box to the heavier
weight portion: the DCS hook/entry representation.
When processing `\033[?1002;1003;1005;1006h`, when we encountered
`1005` we would skip 1006. This was because we hit the unspecified
enum case for 1005 (we have no enum variant for that mode) and that
code path didn't properly advance only by a single position.
This commit fixes that.
refs: https://github.com/wez/wezterm/issues/211#issuecomment-641302077
This corrects an issue where the mode byte of the DCS sequence was
discarded from the DcsHook, making it impossible to know what sequence
is being activated.
So far this hasn't come up as these sequences are relatively rare,
but in looking at sixel parsing I noticed the error.
This commit doesn't do anything specific to scrollback though!
It moves the implementation of the TermWizTermTab away from
a directly manipulated Surface and over to using the term::Terminal,
making the renderer look more like the one used by the local tab
and domain implementation.
As a side effect of doing this, we get scrollback management
for free.
refs: https://github.com/wez/wezterm/issues/201
We were treating DECCKM as the sole thing to enable application
cursor reporting, but looking closely at the docs, that mode only
takes effect when both DECANM (Vt52 emulation mode) AND DECKPAM
(application keypad mode) are both active.
neovim enables DECCKM and DECKPAM but not DECANM.
refs: https://github.com/wez/wezterm/issues/203
This commit adds a simple example of nested widgets, which uncovered
an issue with respecting the layout positioning, and adds a secondary
screen buffer to make the widget render cycle more optimal:
```
cargo run --example widgets_nested --features widgets --release
```
refs: https://github.com/wez/wezterm/issues/171
* ctrl-R to find a line and then hit enter would cause the search
text rather than the match text to be returned and run!
* When exiting the editor, clear to end of screen to make sure
that we clean up any UI from the incremental search status
This helps us keep track of the extent and cursor position that
we render for the line editor, making it easier to make the editor
rendering more fancy.
This restructures the LineEditor to allow the hosting application to
override key presses and apply custom edits to the editor buffer.
Methods for performing predefined actions and for accessing the line
buffer and cursor position have been provided/exposed to support this.
One consequence of this change is that the editor instance needs to be
passed through to the host trait impl and that means that the LineEditor
can no longer be generic over `Terminal`. Instead we now take `&mut dyn
Terminal` which was how the majority of non-example code was using it in
any case. This simplifies a bit of boilerplate in wezterm but adds an
extra line to the most basic examples.
On Windows, if you run `wsl.exe` from the terminal and start zsh
(maybe bash also?) and it enables application cursor key mode,
exiting zsh doesn't clear application cursor key mode and when we
return to the shell and are using virtual terminal input rather
than the native windows console input, we'll continue to receive
application cursor key sequences instead of regular cursor key
input sequences.
This commit recognizes both flavors as arrow movement
in the line editor to make this feel less broken.
Without this, `wzsh` will keep the terminal in raw mode between
line editor invocations, resulting in staggered/stairway output
for any spawned commands.
With the revised native windows console renderer using the various
console APIs more deeply, I've seen a couple of cases where those
API calls fail inside eg: wezterm running via the new pty machinery.
Using the virtual terminal APIs and the terminfo renderer is the
right thing to do in that case.
This commit probes for virtual terminal support and uses the builtin
xterm terminfo, unless the environment has
`TERMWIZ_BYPASS_VIRTUAL_TERMINAL=1` set. This allows forcing the
use of the windows console layer.
Some windows APIs have inclusive dimensions and some exclusive;
we were off by one for the height of the display which led to some
weirdness with eg: `sp` and the line editor.
When it comes to scrolling: if the scroll request is for the entire
viewport then we simply adjust the viewport; this is desirable because
it allows data to scroll back into the history in the native console.
This fixes the math around cursor positioning for the edge case where
the width of text and the cursor position are close to the width of
the terminal.
This reduces flickering updates in the native windows console;
it works by taking a copy of the screen buffer, applying the
Change's to that buffer and then copying back to the console.
This is unfortunately a bit of a muddy commit and I'm too lazy to split
it up.
* Removed `Position::NoChange`; use `Position::Relative(0)` instead
* Added missing cursor positioning cases in the terminfo renderer
* Taught line editor about the cursor position when the line spans
multiple physical lines
* Taught the Windows input layer to process escape sequences for eg:
the arrow keys when running with virtual terminal enabled.
* Removed the hack that under-reported the terminal width; the hack
was present to make some aspects of rendering with the native windows
console logic easier, but it was getting in the way of the line
editor. This may well break something, but it fixed up the line
editor :-/
cc: @markbt
This commit changes the behavior on Windows:
* If $TERM is set and the `terminfo` crate is able to
successfully initialize and locate a terminfo database (this also
requires that $TERMINFO be set in the environment), then we'll
use the `TerminfoRenderer` instead of the `WindowsConsoleRenderer`
* If $TERM is set to `xterm-256color` and no terminfo database was
found, use our modern compiled-in copy (look in the `termwiz/data/`
directory for the source and compiled version of this) and use
the `TerminfoRenderer`.
* Otherwise use the `WindowsConsoleRenderer`.
In practice, this allows termwiz apps to opt in to features such as
true color support on Windows 10 build 1903 an later by setting their
`TERM=xterm-256color`. This happens to be the default behavior when
`ssh`ing in to a windows host via `wezterm`.
You can see the truecolor mode get applied by running this example:
```
cargo run --example widgets_basic --features widgets
```
with TERM set as above the background region that is painted by the app
will be a blueish/purplish color, but with it unset or set to something
invalid, it will fall back to black.
I'd like to eventually make termwiz assume the equivalent configuration
to `TERM=xterm-256color` by default on Windows 10 build 1903 and later,
but it's worth getting some feedback on how this works for clients such
as `streampager`.
cc: @quark-zju and @markbt
Add an update indicator to the top right of client tabs; this is
overlaid on top of the surface when the last update from the server was
more than ~3s ago and if we expected it sooner than that.
While making this work, I noticed that the exponential poll backoff
had gotten broken in an earlier refactor; instead of a series of polls
backing off slowly, we were aggressively running the backoff up to the
max 30 second interval over the span of a few ms. This commit fixes
up the backoff computation to only happen when we are ready to send
a poll.
refs: https://github.com/wez/wezterm/issues/127
