DiffState keeps track of a virtual cursor in order to skip emitting
cursor location changes when a previous text change has already moved
the cursor to the desired position. However, currently the code always
advances the cursor by 1 column for each added text change regardless
of the width of the text.
Fix behavior by advancing the cursor by the width of the text.
To avoid having to add a new variable, make 'self.cursor' contain
the cursor position for the next cell instead of the current cell.
In addition, add a test verifying the behavior.
The present code for creating diffs between surface regions processes
visible cells from both surfaces in lockstep. This only works when
the cells at each position have the same width in each surface.
When widths differ, the iterators go out of sync and end up comparing
cells in different positions of the surface, resulting in incorrect
diffs.
Fix this by instead iterating over the source surface ('other'), and
for each cell locating the corresponding cell in the target surface
to compare against.
To apply the fix to all three diffing functions, outline the per-line
code to a new function and call it in each function.
Also add a test that verifies the behavior in such case with differing
cell widths.
- v2.0 no longer derives a bunch of traits (Debug, Clone, Copy, PartialEq, Eq)
by default, so I manually derived them.
- It also changed the snapshot results so I updated them:
An empty bitflags struct is no longer displayed as `None` but as
`BitFlags(0x0)`.
- `bits` is no longer a field but a method so I added the missing
parenthesis.
We weren't including the invisible space cells into the model
as part of building up the logical line, resulting in the logical
line being shorter than it should have been.
That resulted in some of the components of the double wide cells
not being rendered in the correct place.
refs: https://github.com/wez/wezterm/issues/2571
refs: https://github.com/wez/wezterm/issues/2568
According to its benchmarks, it's almost 2x faster than
unicode_segmentation. It doesn't appear to make a visible
difference to `time cat bigfile`, but I'll take anything
that gives more headroom for such little effort of switching.
There were two problems:
* We weren't correctly invalidating when the hover state changed
(a recent regression caused by recent caching changes)
* We'd underline every link with the same destination on hover,
not just the one under the mouse (longstanding wart)
Recent changes allow the application layer to reference the underlying
Lines directly, so we can restore the original and expected
only-highlight-under-the-mouse by switching to those newer APIs.
Adjust the cache values so that we know to also verify the current
highlight and invalidate.
I was a little surprised to see that this also works with mux client
panes: I was expecting to need to do some follow up on those because
they return copies of Line rather than references to them. That happens
to work because the mux client updates the hyperlinks at the time where
it inserts into its cache. The effect of that is that lines in mux
client panes won't update to new hyperlink rules if they were received
prior to a change in the config.
refs: https://github.com/wez/wezterm/issues/2496
Move is_double_wide to a box; it is relatively rare to need
this and we're okay with it being a separate heap allocation
when it is needed if it reduces the size of Line in the common case.
I don't think these are really necessary any more; the implementation
cannot go out of bounds, so the worst that can happen is that we
don't return any changes.
refs: https://github.com/wez/wezterm/issues/2222
Adds Pane::for_each_logical_line_in_stable_range_mut and
Pane::with_lines_mut which allow iterating mutably over lines.
The idea is that this will allow the renderer to directly cache
data in the Line via its appdata without having to build cumbersome
external caching logic and managing cache keys.
This commit just swaps the implementation around for localpane
and sanity checks that the renderer functions.
Various overlays and the mux client don't properly implement these
yet and current warn at compile time and panic at runtime.
To follow is the logic to cache the data and make sure that it
works the way that I think before converting the other Pane
implementations.
It didn't really belong there; it was added as a bit of a hack
to propagate screen reverse video mode.
Move that to the RenderableDims struct and remove the related
bits from Line
Introduces a heap-based quad allocator that we cache on a per-line
basis, so if a line is unchanged we simply need to copy the previously
computed set of quads for it into the gpu quad buffer.
The results are encouraging wrt. constructing those quads; the
`quad_buffer_apply` is the cost of the copy operation, compare with
`render_screen_line_opengl` which is the cost of computing the quads;
it's 300x better at the p50 and >100x better at p95 for a full-screen
updating program:
full 2880x1800 screen top:
```
STAT p50 p75 p95
Key(quad_buffer_apply) 2.26µs 5.22µs 9.60µs
Key(render_screen_line_opengl) 610.30µs 905.22µs 1.33ms
Key(gui.paint.opengl) 35.39ms 37.75ms 45.88ms
```
However, the extra buffering does increase the latency of
`gui.paint.opengl` (the overall cost of painting a frame); contrast the
above with the latency in the same scenario with the current `main`
(rather than this branch):
```
Key(gui.paint.opengl) 19.14ms 21.10ms 28.18ms
```
Note that for an idle screen this latency is ~1.5ms but that is also true
of `main`.
While the overall latency in the histogram isn't a slam dunk,
running `time cat bigfile` is ~10% faster on my mac.
I'm sure there's something that can be shaved off to get a more
convincing win.
This reduces the resident memory by another ~10% because it avoids
keeping as many runs of whitespace.
Runtime for `time cat enwiki8.wiki` is still ~11-12s, resident: 530K
refs: https://github.com/wez/wezterm/issues/1626
The previous commit added the option to convert the storage to
the cluster format. That saves memory as rows are moved to scrollback,
but makes scrolling back more expensive due to that conversion.
This commit adds a fast(ish) path for the common case of simply
appending text to a new line before it gets scrolled: the default
format for lines in the screen is now the cluster format and,
provided that the cursor moves from left to right as text is
emitted, can simply append to the cluster storage in-place
and avoids a conversion when the line is moved to scrollback.
This improves the throughput of `time cat enwiki8.wiki` so
that the runtime is typically around 11-12s (compared to 9-10s
before introducing cluster storage). However, this is often
a deal of variance in the measured time and I believe that
that is due to the renderer triggering conversions back to
the vec storage and introducing slowdowns from the render side.
That's what I'll investigate next.
Previously this would create a new String because it had to, but
with the clustered storage we may be able to simply reference the
existing string as a str reference, so allow for that.
Adds the option to use an alternative clusted line storage for
the cells component of the line.
This structure is not optimal for mutation, but is better structured
for:
* matching/extracting textual content
* using less memory than the prior simple vector
For some contrast: the line "hello" occupies 5 Cells in the cell based
storage; that 5 discrete Cells each with their own tiny string
and a copy of their attributes.
The clustered version of the line stores one copy of the cell
attributes, the string "hello" and some small (almost constant size)
overhead for some metadata. For simple lines of ascii text, the
clustered version is smaller as there are fewer copies of the cell
attributes. Over the span of a large scrollback and typical terminal
display composition, this saving is anticipated to be significant.
The clustered version is also cheaper to search as it doesn't require
building a copy of the search text for each line (provided the line is
already in clustered form).
This commit introduces the capability: none of the internals request the
new form yet, and there are likely a few call sites that need to be
tweaked to avoid coersion from clustered to vector form.
Avoid using serde for mapping between Lua and Rust for the `Config`
struct.
This improves the build speed of the config crate by 2x; it goes down
from 30 seconds to 9 seconds on my 5950x.
This puts to final rest #478, wherein ligatured glyphs that span
cells would render portions of the glyph with the wrong fg color,
where wrong was usually the bg color and cause the glyph to turn
invisible when cursoring through the ligature.
The approach used here is to divide the glyph into 7 discrete strips
where each strip either intersects with the cursor, the selection, or
neither. That allows us to render each strip with the appropriate
foreground color.
This change simplifies some of the logic and allows some other code
to be removed, so that feels good!
As is tradition with these renderer changes, there's a good chance that
I overlooked something in testing and that the metrics or alignment
might be slightly off for some font/text combo. Let's see how this
goes!
refs: #784
refs: #478
refs: #1617
This commit refines bidi property handling:
* experimental_bidi has been split into two new configuration settings;
`bidi_enabled` (which controls whether the terminal performs implicit
bidi processing) and `bidi_direction` which specifies the base
direction and whether auto detection is enabled.
* The `Line` type can now store those bidi properties (they are actually
split across 3 bits representing enabled, auto-detection and
direction)
* The terminal now has a concept of active bidi properties and default
bidi properties
* The default properties are pulled from the wezterm configuration
* active bidi properties are potentially set via escape sequences,
BDSM (which sets bidi_enabled) and SCP (which sets bidi_direction).
We don't support the 2501 temporary dec private mode suggested by
the BIDI recommendation doc at this time.
* When creating new `Line`'s or clearing from the start of a `Line`, the
effective bidi properties are computed (from the active props,
falling back to default propr) and applied to the `Line`.
* When rendering the line, we now look at its bidi properties instead
of just the global config.
The default bidi properties are `bidi_enabled: false` and
`bidi_direction: LeftToRight` which corresponds to the typical
bidi-unaware mode of most terminals.
It is possible to live reload the config to change the effective
defaults, but note that they apply, by design, to new lines being
processed through the terminal. That means existing output is
left unaffected by a config reload, but subsequently printed lines
will respect it. Pressing CTRL-L or otherwise contriving to have
the running application refresh its display should cause the
refreshed display to update and apply the new bidi mode.
refs: #784
This commit is larger than it appears to due fanout from threading
through bidi parameters. The main changes are:
* When clustering cells, add an additional phase to resolve embedding
levels and further sub-divide a cluster based on the resolved bidi
runs; this is where we get the direction for a run and this needs
to be passed through to the shaper.
* When doing bidi, the forced cluster boundary hack that we use to
de-ligature when cursoring through text needs to be disabled,
otherwise the cursor appears to push/rotate the text in that
cluster when moving through it! We'll need to find a different
way to handle shading the cursor that eliminates the original
cursor/ligature/black issue.
* In the shaper, the logic for coalescing unresolved runs for font
fallback assumed LTR and needed to be adjusted to cluster RTL.
That meant also computing a little index of codepoint lengths.
* Added `experimental_bidi` boolean option that defaults to false.
When enabled, it activates the bidi processing phase in clustering
with a strong hint that the paragraph is LTR.
This implementation is incomplete and/or wrong for a number of cases:
* The config option should probably allow specifying the paragraph
direction hint to use by default.
* https://terminal-wg.pages.freedesktop.org/bidi/recommendation/paragraphs.html
recommends that bidi be applied to logical lines, not physical
lines (or really: ranges within physical lines) that we're doing
at the moment
* The paragraph direction hint should be overridden by cell attributes
and other escapes; see 85a6b178cf
and probably others.
However, as of this commit, if you `experimental_bidi=true` then
```
echo This is RTL -> عربي فارسی bidi
```
(that text was sourced from:
https://github.com/microsoft/terminal/issues/538#issuecomment-677017322)
then wezterm will display the text in the same order as the text
renders in Chrome for that github comment.
```
; ./target/debug/wezterm --config experimental_bidi=false ls-fonts --text "عربي فارسی ->"
LeftToRight
0 ع \u{639} x_adv=8 glyph=300 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
2 ر \u{631} x_adv=3.78125 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
4 ب \u{628} x_adv=4 glyph=244 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
6 ي \u{64a} x_adv=4 glyph=363 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
8 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
9 ف \u{641} x_adv=11 glyph=328 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
11 ا \u{627} x_adv=4 glyph=240 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
13 ر \u{631} x_adv=3.78125 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
15 س \u{633} x_adv=10 glyph=278 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
17 ی \u{6cc} x_adv=4 glyph=664 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
19 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
20 - \u{2d} x_adv=8 glyph=276 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
21 > \u{3e} x_adv=8 glyph=338 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
```
```
; ./target/debug/wezterm --config experimental_bidi=true ls-fonts --text "عربي فارسی ->"
RightToLeft
17 ی \u{6cc} x_adv=9 glyph=906 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
15 س \u{633} x_adv=10 glyph=277 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
13 ر \u{631} x_adv=4.78125 glyph=272 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
11 ا \u{627} x_adv=4 glyph=241 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
9 ف \u{641} x_adv=5 glyph=329 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
8 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
6 ي \u{64a} x_adv=9 glyph=904 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
4 ب \u{628} x_adv=4 glyph=243 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
2 ر \u{631} x_adv=5 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
0 ع \u{639} x_adv=6 glyph=301 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
LeftToRight
0 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
1 - \u{2d} x_adv=8 glyph=480 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
2 > \u{3e} x_adv=8 glyph=470 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
;
```
refs: https://github.com/wez/wezterm/issues/784
This commit decomposes the main get_semantic_zones method into two
parts:
* A per-line portion, where the line ranges are cached (invalidated on
change)
* The overall screen portion, where the line ranges are merged
This changes the overall complexity of computing zones from
O(width * scrollback-height)
To an incremental:
O((width * number of changed lines since last query) + scrollback-height)
You can see some samples of elapsed time below; those show the times for
running both the old and the new implementation on the same data. The
number of lines/zones in the scrollback increases with each call and you
can see that the new implementation is a bit faster anyway at low
volumes but is significantly faster as the number of lines/zones
increases, because the amount of work is reduced.
```
get_semantic_zones: 71.708µs
get_semantic_zones_new: 59.041µs
get_semantic_zones: 71.166µs
get_semantic_zones_new: 9.166µs
get_semantic_zones: 44.291µs
get_semantic_zones_new: 4.208µs
get_semantic_zones: 69.791µs
get_semantic_zones_new: 10.291µs
get_semantic_zones: 59.375µs
get_semantic_zones_new: 7.958µs
get_semantic_zones: 52.5µs
get_semantic_zones_new: 4.5µs
get_semantic_zones: 91.791µs
get_semantic_zones_new: 20.916µs
get_semantic_zones: 229.916µs
get_semantic_zones_new: 109.208µs
get_semantic_zones: 224.125µs
get_semantic_zones_new: 15.208µs
get_semantic_zones: 291.791µs
get_semantic_zones_new: 11.833µs
get_semantic_zones: 238.875µs
get_semantic_zones_new: 12.625µs
get_semantic_zones: 468.458µs
get_semantic_zones_new: 126.583µs
get_semantic_zones: 460.5µs
get_semantic_zones_new: 25.666µs
get_semantic_zones: 358.291µs
get_semantic_zones_new: 19.541µs
get_semantic_zones: 436.833µs
get_semantic_zones_new: 17.875µs
get_semantic_zones: 313.166µs
get_semantic_zones_new: 15.25µs
get_semantic_zones: 333.958µs
get_semantic_zones_new: 16.541µs
get_semantic_zones: 364.666µs
get_semantic_zones_new: 14.041µs
```
When rendering the IME composing text, I noticed that for the Korean
input sequence: shift+'ㅅ' followed by 'ㅏ' we'd render the 'ㅆ' (the
shifted first character) in black and the composing 'ㅏ' in white
against the cursor color, and that was very difficult to read,
especially at the default font size.
To resolve this, this commit:
* Forces clustering to break around the cursor boundary, so that
we treat the cursor position as its own separately styled cluster
* Adjusts cursor/bg rendering so that we always consider the start of
the cluster for the colors of that run. We are guaranteed that a
ligatured sequence will fit in the background area anyway.
This has the effect of "breaking" programming ligatures such as '->'
when cursoring through them, and decomposing them into their individual
'-' and '>' glyphs, which is a reasonable price to pay for being able
to see things better on screen.
refs: https://github.com/wez/wezterm/issues/1504
refs: https://github.com/wez/wezterm/issues/478
Previously, we would implicitly set it to the special SEQ_ZERO
value, but since that value always flags the row as changed,
it causes some over-invalidation issues downstream in wezterm.
This commit makes that parameter required, so that the code that
is creating a new Line always passes down the seqno from that event.
refs: #1472
Not 100% sure why this only really manifested on Windows, but
the symptoms were:
* Run powershell in a tab
* Run `dir`
* Hit enter a couple of times to show a couple of prompts
* Try using the mouse to select across the prompt boundaries
The selection would get invalidated crossing the boundaries.
I traced this down to the lines around those regions having
SEQ_ZERO as their sequence, so this commit ensures that lines
that are created as part of scrolling the screen are correctly
tagged with the current seqno from the terminal display.
Why only windows? Not totally sure; perhaps it is related to
something funky happening in the conpty layer and sending us
unusual escapes (eg: scroll margins?)
This is a fairly far-reaching commit. The idea is:
* Introduce a unicode_version config that specifies the default level
of unicode conformance for each newly created Terminal (each Pane)
* The unicode_version is passed down to the `grapheme_column_width`
function which interprets the width based on the version
* `Cell` records the width so that later calculations don't need to
know the unicode version
In a subsequent diff, I will introduce an escape sequence that allows
setting/pushing/popping the unicode version so that it can be overridden
via eg: a shell alias prior to launching an application that uses a
different version of unicode from the default.
This approach allows output from multiple applications with differing
understanding of unicode to coexist on the same screen a little more
sanely.
Note that the default `unicode_version` is set to 9, which means that
emoji presentation selectors are now by-default ignored. This was
selected to better match the level of support in widely deployed
applications.
I expect to raise that default version in the future.
Also worth noting: there are a number of callers of
`unicode_column_width` in things like overlays and lua helper functions
that pass `None` for the unicode version: these will assume the latest
known-to-wezterm/termwiz version of unicode to be desired. If those
overlays do things with emoji presentation selectors, then there may be
some alignment artifacts. That can be tackled in a follow up commit.
refs: #1231
refs: #997
`use_fancy_tab_bar` switches to an alternate rendering of the tab
bar that uses the window_frame config to get a proportional
title font to use to render tabs, as well as rendering a few
additional elements to space out and make the tabs feel more
like tabs.
Computing the number of tabs doesn't respect the alternate font
at this time.
Formatted tab item foreground and background colors are also
not respected at this time.
refs: #1180
A while back I made the line lengths lazily grown; the reduction
in memory was nice, and it helped with render performance for
really wide screens.
Unfortunately, it puts a bunch of reallocation into the hot path
of the parser and updating the terminal model when people run
the inevitable `cat giant-file.txt` benchmark.
This commit reinstates pre-allocating lines to match the physical
terminal width, and tweaks the code a bit to take advantage of
const Cell allocation and to avoid some clones (a really micro
optimization).
