We need 100% of the info for it to work correctly, so this commit:
* Exposes the keyboard encoding mode via the Pane trait
* Adds the scan code to the RawKeyEvent
* Has the GUI perform the encoding if the keyboard is set that way
* Removes the basic encoder from termwiz in favor of the gui level one
The net result is that we bypass the Pane::key_up/Pane::key_down methods
in almost all cases when the encoding mode is set to win32-input-mode.
There is now a config option: allow_win32_input_mode that can be
used to prevent using this mode.
refs: #1509
This commit causes the terminal to emit win32-input-mode encoded key up
and down events for a limited subset of keys When win32-input-mode is
enabled.
We limit them to keys where we know the VK key code equivalent,
and where those keys are either not representable (eg: modifier
only key events), or may generate ambiguous output (eg: CTRL-SPACE
in different keyboard layouts).
However, in my experiments, modifier only key presses confuse powershell
and cause it to emit `@`, so I've disabled that in the code for now.
refs: https://github.com/wez/wezterm/issues/318
refs: https://github.com/wez/wezterm/issues/1509
refs: https://github.com/wez/wezterm/issues/1510
This is a baby step that formalizes the different encoding schemes into
an enum, and hooks up the decset sequence for win32-input-mode.
It doesn't change any of the actual encoding at this time.
refs: #1509
When rendering the IME composing text, I noticed that for the Korean
input sequence: shift+'ㅅ' followed by 'ㅏ' we'd render the 'ㅆ' (the
shifted first character) in black and the composing 'ㅏ' in white
against the cursor color, and that was very difficult to read,
especially at the default font size.
To resolve this, this commit:
* Forces clustering to break around the cursor boundary, so that
we treat the cursor position as its own separately styled cluster
* Adjusts cursor/bg rendering so that we always consider the start of
the cluster for the colors of that run. We are guaranteed that a
ligatured sequence will fit in the background area anyway.
This has the effect of "breaking" programming ligatures such as '->'
when cursoring through them, and decomposing them into their individual
'-' and '>' glyphs, which is a reasonable price to pay for being able
to see things better on screen.
refs: https://github.com/wez/wezterm/issues/1504
refs: https://github.com/wez/wezterm/issues/478
Previously, we would implicitly set it to the special SEQ_ZERO
value, but since that value always flags the row as changed,
it causes some over-invalidation issues downstream in wezterm.
This commit makes that parameter required, so that the code that
is creating a new Line always passes down the seqno from that event.
refs: #1472
Not 100% sure why this only really manifested on Windows, but
the symptoms were:
* Run powershell in a tab
* Run `dir`
* Hit enter a couple of times to show a couple of prompts
* Try using the mouse to select across the prompt boundaries
The selection would get invalidated crossing the boundaries.
I traced this down to the lines around those regions having
SEQ_ZERO as their sequence, so this commit ensures that lines
that are created as part of scrolling the screen are correctly
tagged with the current seqno from the terminal display.
Why only windows? Not totally sure; perhaps it is related to
something funky happening in the conpty layer and sending us
unusual escapes (eg: scroll margins?)
permits iTerm2 images to be drawn anywhere on screen without
scrolling the cursor, including the bottom row.
Also included is a check in fcwrap.rs to_range_set(), without which
was causing a panic at runtime due to subtraction from unsigned
leading to overflow.
This helps us correctly set the size of the image cell
for the case where we have a partial cell at the right/bottom
edge of an image being mapped across cells.
refs: #1270
We need to force the codepage to UTF8 this to avoid having our UTF-8
bytestreams be misinterpreted, and to restore the original before we're
done.
refs: https://github.com/wez/wezterm/issues/1435
implement missing alt-screen support and fixup building the examples
while we're in here.
refs: #1244
This commit teaches `RgbColor::from_rgb_str` to support
colors in the form `hsl:235 100 50`, an HSL colorspace
color specification.
While banging my head on why my test wasn't passing, I realized
that this was producing 10 bpc color and the code to convert
those to RGB was incorrectly multiplying conversion terms!
refs: https://github.com/wez/wezterm/issues/1436
From esctest:
CUPTests.test_CUP_ColumnOnly
HVPTests.test_HVP_ColumnOnly
and the newly added at https://invent.kde.org/ninjalj/esctest.git:
DECSETTests.test_DECSET_DECLRMM_OnlyRight
DECSTBMTests.test_DECSTBM_OnlyBottom
Not all codepoints are valid when combined with a presentation
selector.
This commit ensures that we respect the valid sequences defined
by the current version of unicode (version 14).
refs: #1231
refs: #997
As promised in the previous commit, this one implements an escape
sequence to control the unicode version.
Unknown to me in the previous commit, iTerm2 already defines such
an escape sequence, so we simply implement it here with the same
semantics.
refs: #1231
refs: #997
This is a fairly far-reaching commit. The idea is:
* Introduce a unicode_version config that specifies the default level
of unicode conformance for each newly created Terminal (each Pane)
* The unicode_version is passed down to the `grapheme_column_width`
function which interprets the width based on the version
* `Cell` records the width so that later calculations don't need to
know the unicode version
In a subsequent diff, I will introduce an escape sequence that allows
setting/pushing/popping the unicode version so that it can be overridden
via eg: a shell alias prior to launching an application that uses a
different version of unicode from the default.
This approach allows output from multiple applications with differing
understanding of unicode to coexist on the same screen a little more
sanely.
Note that the default `unicode_version` is set to 9, which means that
emoji presentation selectors are now by-default ignored. This was
selected to better match the level of support in widely deployed
applications.
I expect to raise that default version in the future.
Also worth noting: there are a number of callers of
`unicode_column_width` in things like overlays and lua helper functions
that pass `None` for the unicode version: these will assume the latest
known-to-wezterm/termwiz version of unicode to be desired. If those
overlays do things with emoji presentation selectors, then there may be
some alignment artifacts. That can be tackled in a follow up commit.
refs: #1231
refs: #997
`use_fancy_tab_bar` switches to an alternate rendering of the tab
bar that uses the window_frame config to get a proportional
title font to use to render tabs, as well as rendering a few
additional elements to space out and make the tabs feel more
like tabs.
Computing the number of tabs doesn't respect the alternate font
at this time.
Formatted tab item foreground and background colors are also
not respected at this time.
refs: #1180
This was a bit of a PITA to run down; the essence of the problem
was that the shaper was returning an x_advance of 0 for U+3000,
which caused wezterm's shaping layer to elide that glyph.
I eventually tracked down the x_advance to be the result of
scaling by an x_scale of 0, which in turn is the result of
harfbuzz not knowing the font size.
The critical portion of this diff is the line that advises
harfbuzz that the font has changed after we've applied the
font size.
The rest is just stuff to make it easier to debug and verify.
This:
```
printf "x\u3000x."
```
Now correctly renders on screen as "x x".
fixes: #1161
I noticed while reviewing the session recording for https://github.com/wez/wezterm/issues/1102
that wezterm reported:
```
2021-09-04T22:05:46.136Z WARN wezterm_term::terminalstate::performer > unknown unspecified CSI: "58::2::87::61::38m"
2021-09-04T22:05:46.138Z WARN wezterm_term::terminalstate::performer > unknown unspecified CSI: "58::2::87::61::38m"
```
Those extra double-colons aren't valid, and I think they're coming from
this terminfo file, so fix it up!
This will enable eg: a lua helper function to serialize keycodes to
assist in some key rebinding scenarios (see:
https://github.com/wez/wezterm/pull/1091#issuecomment-910940833 for the
gist of it) but also makes it a bit easier to write unit tests for key
encoding so that situations like those in #892 are potentially less
likely to occur in the future.
If a cell contained a ligature, the math used to track where the
next quad was going to be placed could lose consistency with
the cluster/shaping information and result in offset glyphs.
This was most noticeable to me in tab titles in the tab bar;
my shell dotfiles append `-- something` to the title when a
command is launched, and the `--` is a ligature in my font.
I think I've also seen this mess up positioning in the notcurses
demo as well.
The solution is to take the cluster initial cell index rather
than trying to reverse engineer it from incomplete info.
A while back I made the line lengths lazily grown; the reduction
in memory was nice, and it helped with render performance for
really wide screens.
Unfortunately, it puts a bunch of reallocation into the hot path
of the parser and updating the terminal model when people run
the inevitable `cat giant-file.txt` benchmark.
This commit reinstates pre-allocating lines to match the physical
terminal width, and tweaks the code a bit to take advantage of
const Cell allocation and to avoid some clones (a really micro
optimization).
For simple graphemes, we can avoid subsequently calling
grapheme_column_width and cache that information in TeenyString.
Make blank TeenyString and Cell initializations const.
Terminal now maintains a sequence number that increments
for each Action that is applied to it.
Changes to lines are tagged with the current sequence number.
This makes it a bit easier to reason about when an individual
line has changed relative to some point in "time"; the consumer
of the terminal can sample the current sequence number and then
can later determine which lines have changed since that point
in time.
refs: https://github.com/wez/wezterm/issues/867
An implementation detail in wezterm is that it doesn't model
image placements as a separate entity; they are all bound to
the image cells in the terminal model.
The semantics of the kitty image protocol are that placements
are "permanent" wrt. overwriting a cell with text, except for
the explicit EraseInLine/EraseInDisplay sequences that are
used for clearing.
This commit takes a pass at implementing that semantic in
the wezterm data model.
refs: #986
This should give the shaper a better chance at using text
presentation in a run that mixes emoji with text and/or
uses presentation selectors.
It also exposes the presentation property to the shaper
so that it could potentially adjust its fallback strategy.
However, it doesn't do that here.
refs: https://github.com/wez/wezterm/issues/997
Make a distinction between default and selected presentation,
and account for that in the cell width.
Add a method to the cell that returns the effective presentation.
refs: https://github.com/wez/wezterm/issues/997
These modifiers have the effect of forcing us to consider the grapheme
as being either a single cell (VS15) or two cells (VS16) in the
terminal model.
These don't affect font choice as wezterm doesn't know whether a given
font in the fallback has a textual vs. an emoji version of a given
glyph, or whether a later font in the fallback has one or the other
because we can't know until we fall back, and that has a very high
cost--we perform fallback asynchronously in another thread because
of its high cost.
Depending on the selected glyph, it may or may not render as double
wide.
refs: #997
Since we can now mutate individual frames, we need to avoid
falsely caching across a change; switch from using (image_id, frame_idx)
to frame_hash.
refs: #986
Adds a use_image feature to termwiz that enables an optional
dep on the image crate. This in turn allows decoding of animation
formats (gif, apng) from file data, but more crucially, allows
modeling animation frames at the termwiz layer, which is a pre-req
for enabling kitty img protocol animation support.
refs: #986
I noticed when running the notcurses demo that we're spending a
decent amount of time decoding png data whenever we need to
re-do the texture atlas.
Let's avoid that by allowing for ImageData at the termwiz layer
to represent both the image file format and decoded rgba8 data.
This commit is a bit muddy and also includes some stuff to try
to delete placements from the model. It's not perfect by any
means--more expensive than I want, and there's something funky
that causes a large number of images to build up during some
phases of the demo.
refs: #986
This isn't complete; many of the placement options are not supported,
and the status reporting is missing in a number of cases, including
querying/probing, and shared memory objects are not supported yet.
However, this commit is sufficient to allow the kitty-png.py script
(that was copied from
https://sw.kovidgoyal.net/kitty/graphics-protocol/#a-minimal-example)
to render a PNG in the terminal.
This implementation routes the basic image display via the same
code that we use for iterm2 and sixel protocols, but it isn't
sufficient to support the rest of the placement options allowed
by the spec.
Notably, we'll need to add the concept of image placements to
the data model maintained by the terminal state and find a way
to efficiently manage placements both by id and by a viewport
range.
The renderer will need to manage separate quads for placements
and order them by z-index, and adjust the render phases so that
images can appear in the correct plane.
refs: #986
This teaches termwiz to recognize and encode the APC
sequences used by the kitty image protocol.
This doesn't include support for animations, just the
transmit, placement and delete requests.
refs: #986
These were parsed but swallowed. This commit expands the transitions
to be able to track the APC start, data and end and then adds
an `apc_dispatch` method to allow capturing APC sequences.
APC sequences are used in the kitty image protocol.
refs: #986
This commit hooks up DECRQM so that we can report that we implement
synchronized updates, and then refines the code that manages sending
data to the terminal model; the first cut at synchronized updates
was a bit simplistic, and now we make a point of "flushing" pending
actions when we start a sync point, and again as soon as we release
the sync point.
This smooths out the jaggies around the orca that I mentioned in
dcbbda7702
and while testing this, I realized that recent parser changes had
mangled processing bundled dec private mode sequences where multiple
modes were specified in the same overall escape sequence. I've
added the missing unit test case for this and made that work again.
refs: https://github.com/wez/wezterm/issues/955
refs: https://github.com/wez/wezterm/issues/882
This commit removes the intermediates parameter and collapses it
together with the parameters themselves.
This allows us to model DECSET (eg: `CSI ? 1 l`) correctly.
Previously this would get reported as:
```
params: [1],
intermediates: ['?'],
code: 'l'
```
but since the intermediates are logically things that precede the code,
the canonical interpretation of that would be as if we'd received
`CSI 1 ? l`.
AFAICT, DECSET isn't conforming to ECMA 48 when it comes to this
sequence.
That made things a bit of a headache in the CSI parser, so what we do
now is to treat intermediates as parameters so that it is much simpler
to reason about and match in the CSI parser; we now get:
```
params: ['?', 1],
code: 'l',
```
refs: https://github.com/wez/wezterm/issues/955
The original design of the vtparse crate was inspired by the vte
crate. There were some assumptions about the shape of CSI sequences
that were lossy and that is posing a problem when it comes to
implementing DECRQM.
This commit improves the situation by adjusting CsiParam to be capable
of capturing all of the possible parameters as well as intermediates.
This commit isn't done; I just need to push it to transfer it to another
machine.
refs: https://github.com/wez/wezterm/issues/882
refs: https://github.com/wez/wezterm/issues/955
This implementation doesn't include a timeout, but should be
recoverable via a SoftReset.
There's no query response either: I think I'm missing DECRQM
entirely at the moment.
refs: https://github.com/wez/wezterm/issues/882
4e99edf6f5 split the color into
thin/fat components, but overlooked copying the optional fat
part of the color.
That in turn broke sequences like:
* Set background color
* Erase to end of line
This commit teaches the sgr attribute cloning function to look
at the optional fat component.
refs: #962
Add the third device attributes (DA3) query+reply,
eliminating "unknown/unspecified CSI" error. Like
XTerm, simply reply with zeroes as opposed to site
codes or unique IDs.
https://vt100.net/docs/vt510-rm/DA3.html
* [terminfo] add smxx/rmxx, purge blink
Testing with WezTerm nightly indicates no blink support
available through the xterm \E[5m, and I didn't see any
support in the code -- purge it. smxx and rmxx, meanwhile,
have been added to indicate support for struck text.
* restore blink
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
The common palette indices are in the main cell attributes.
Using true color will allocate fat storage.
This allows reducing the Cell size from 32 -> 24 across the
implementation of storing 10 bpc color (it peaked at 40 in
the last couple of commits).
This adjusts the cursor position after emitting a sixel.
@dankamongmen: I don't have much of a sixel test suite to speak
of (cat snake.six :-p); I'd appreciate it if you could run
notcurses against this and confirm that it is doing something
sane!
At the very least, we shouldn't be warning about the unhandled
mode any more!
refs: https://github.com/wez/wezterm/issues/863
refs: https://github.com/dankamongmen/notcurses/issues/1743
Rust hasn't stabilized fallible allocation, so when we are presented
with an implausibly large sixel, Rust terminates the program; it's not
even a panick that we could potentially catch -> direct to termination.
This commit introduces an arbitrary constraint so that we can
avoid unconditionally terminating for this bad input case.
Thanks to @klamonte for sharing this test case!
A relatively recent commit made it possible for the cell storage
to be smaller than the viewport width.
Avoid panicking when erasing beyond the end of the cell storage.
We can simply NOP as that cell is already implictly erased.
closes: #847
ProbeHints would unconditionally load the terminfo based on $TERM
whever it was created, even if the caller had a pre-supplied
terminfo buffer.
Defer loading that until the caps instance is created.
refs: #817
There were two bugs here:
* \u8D (the utf8 encoded representation of 0x8d, aka: RI) was not
recognized as a C1 code and was instead passed through as printable
text.
* The \u8D is a zero-width sequence which means that a subsequent
set_cell call on the new empty-by-default line wouldn't allocate
any cells in the line array, and the assigment to the line would
panic.
This commit avoids the panic for the second case, and then fixes up
the vtparser to correctly recognize the sequence as a C1 control.
refs: https://github.com/wez/wezterm/issues/768
I noticed edenscm log -p (using streampager) cannot render 256 colors
if TERM is not "*-256color", despite forcing enabling true colors in
streampager (https://github.com/markbt/streampager/pull/28).
I tracked it down here. The problem is that we ask terminfo for colors
it does not claim to support. Fix it by using fallback CSI rendering
for colors exceeding the terminfo max color.
Using a boxed slice means that we hold exactly the memory required
for the file data, rather than the next-power-of-two, which can
be wasteful when a large number of images are being sent to
the terminal.
This is a API breaking change for termwiz, so bump its version.
refs: #534
```
echo -e "\033]777;notify;This is the notification title;This is the notification text\a"
```
Now pops up a notification in a similar manner to OSC 9, except
that this form allows setting both the title and the body separately.
refs: https://github.com/wez/wezterm/issues/489
I started this a while ago; it's pretty time consuming to produce
accessible and usable documentation for this sort of stuff, so
this isn't yet complete, but in the interest of avoiding additional
bit-rot, let's get this up.
refs: https://github.com/wez/wezterm/issues/257
It's been replaced with an opaque termwiz error type instead.
This is a bit of a more conservative approach than that in (refs: #407)
and has less of an impact on the surrounding code, which appeals to
me from a maintenance perspective.
refs: #406
refs: #407
You can install this into your $TERMINFO directory (default is
`$HOME/.terminfo`) by running:
`tic -x wezterm.terminfo`
from this data directory.
Once installed, you can set `TERM=wezterm`.
refs: https://github.com/wez/wezterm/issues/415
This allows us to support the kitty style underline sequence,
or the : separated form of the true color escape sequences.
refs: https://github.com/wez/wezterm/issues/415
These aren't currently rendered, but the parser and model now support
recognizing expanded underline sequences:
```
CSI 24 m -> No underline
CSI 4 m -> Single underline
CSI 21 m -> Double underline
CSI 60 m -> Curly underline
CSI 61 m -> Dotted underline
CSI 62 m -> Dashed underline
CSI 58 ; 2 ; R ; G ; B m -> set underline color to specified true color RGB
CSI 58 ; 5 ; I m -> set underline color to palette index I (0-255)
CSI 59 -> restore underline color to default
```
The Curly, Dotted and Dashed CSI codes are a wezterm assignment in the
SGR space. This is by no means official; I just picked some numbers
that were not used based on the xterm ctrl sequences.
The color assignment codes 58 and 59 are prior art from Kitty.
refs: https://github.com/wez/wezterm/issues/415
Use the scaling factor between the font metrics for the base font
and those of the fallback font selected for a given glyph.
The scenario is this: the base font is typically the first one selected
from the font configuration. There may be multiple fallback fonts that
are different sizes; for instance, the Font Awesome font has glyphs that
are square in aspect and are thus about twice the width of a typical
textual monospace font. Similarly, Noto Color Emoji is another square
font but that has a single set of bitmap strikes at a fixed 128 px
square.
The shaper returns advance metrics in the scale of the containing font,
and the rasterizer will target the supplied size and dpi.
We need to scale these to match the base metrics.
Previously we used a crude heuristic to decide whether to scale,
and that happened to work for Noto Color Emoji but not for Font Awesome,
whose metrics were just inside the bounds of the heuristic.
This commit allows retrieving the metrics for a given font_idx so
that we can compute the correct scale factor without any heuristics,
and applies that to the rasterized glyph.
refs: https://github.com/wez/wezterm/issues/342
This is one of those massive time sinks that I almost regret...
As part of recent changes to dust-off the allsorts shaper, I noticed
that the harfbuzz shaper wasn't shaping as well as the allsorts one.
This commit:
* Adds emoji-test.txt, a text file you can `cat` to see how well
the emoji are shaped and rendered.
* Fixes (or at least, improves) the column width calculation for
combining sequences such as "deaf man" which was previously calculated
at 3 cells in width when it should have just been 2 cells wide, which
resulted in a weird "prismatic" effect during rendering where the
glyph would be rendered with an extra RHS portion of the glyph across
3 cells.
* Improved/simplified the clustering logic used to compute fallbacks.
Previously we could end up with some wonky/disjoint sequence of
undefined glyphs which wouldn't be successfully resolved from a
fallback font. We now make a better effort to consolidate runs of
undefined glyphs for fallback.
* For sequences such as "woman with veil: dark skin tone" that occupy a
single cell, the shaper may return 3 clusters with 3 glyphs in the
case that the font doesn't fully support this grapheme. At render
time we'd just take the last glyph from that sequence and render it,
resulting in eg: a female symbol in this particular case. It is
generally a bit more useful to show the first glyph in the sequence
(eg: person with veil) rather than the gender or skin tone, so the
renderer now checks for this kind of overlapping sequence and renders
only the first glyph from the sequence.
@yoichi reports that:
```bash
printf "\x1bPqh"
```
would panic wezterm; the issue was that the maximum x value was
only being updated for newlines and that sequence didn't include it.
refs: #217
Adds some supporting methods for computing the `SemanticZone`s
in the display and a key assignment that allows scrolling the
viewport to jump to the next/prev Prompt zone.
This commit allows the terminal to tag cells with their semantic
type, as defined by OSC 133 escape sequences.
The gist of it is that each cell is now semantically one of:
* Output (eg: from the activity performed by the user. This is the
default)
* Input (eg: something that the user typed as input)
* Prompt (eg: "uninteresting" chrome/UI from the shell)
The semantic type is applied almost exactly like an SGR attribute,
except that resetting SGR doesn't clear the semantic type.
Tagging the cells in this way allows for smarter UX in the future;
for example, selecting the entire input or output from the last
command without fiddling around to avoid the prompt line(s),
or "paging up" to a prior prompt rather than page.
This doc covers those escapes as used in domterm, iterm2 and other
terminals:
https://gitlab.freedesktop.org/Per_Bothner/specifications/blob/master/proposals/semantic-prompts.md
This is an example of how to configure the shell to emit these
sequences; I'll add a proper little blob of shell specifically
for wezterm in a later commit:
https://github.com/PerBothner/DomTerm/blob/master/tools/shell-integration.zsh
