Adopt the new blob lease layer to storing and referencing
image files.
This reduces the number of open files needed when
images are being displayed in the terminal.
refs: https://github.com/wez/wezterm/issues/3263
Input data:
\e_Ga=T,f=32,s=10,v=22,c=1,r=1,m=1\e\\e_Gm=1;/xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T//P=\e\\e_Gm=1;/xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T////E////xP///8T//P=\e\\e_Gm=0\e\
There were two issues in handling this:
* We expected there to be `;payload` in the first transmission packet,
but there wasn't one, so we ignored it as ill-formed.
* The standard base64 decoder in the rust ecosystem is super strict
and rejects the "sloppy" python base64 encoder output that isn't
strictly conformant with the RFC. We need to jump through some
hoops to get it to relax and accept the input.
refs: https://github.com/wez/wezterm/issues/2716
Continuing from the previous commit, this shifts:
* In-memory data -> temporary file
* Image decoding -> background thread
The background thread asynchronously decodes frames and
sends them to the render thread via a bounded channel.
While decoding frames, it writes them, uncompressed, to
a scratch file so that when the animation loops, it is
a very cheap operation to rewind and pull that data
from the file, without having to burn CPU to re-decode
the data from the start.
Memory usage is bounded to 4 uncompressed frames while
decoding, then 3 uncompressed frames (triple buffered)
while looping over the rest.
However, disk usage is N uncompressed frames.
refs: https://github.com/wez/wezterm/issues/3263
This makes decoding animation frames a lazy operation, but it
comes at the cost of needing to re-decode the image from scratch
when it loops, because the image crate doesn't provide a way
to rewind its frame iterator.
That initial decode can have a significant time cost; a small
webp file consistently takes 150ms to decode, which is too
much to do inline in the render thread.
Next steps will be to move that cost off the render thread.
I spent a few hours in heap profilers. What I found was:
* Inefficient use of heap when building up runs of
`Action::Print(char)`.
-> Solve by adding `Action::PrintString(String)`
and accumulating utf8 bytes rather than u32 codepoints.
* Inefficient use of heap when building Quad buffers: the default
exponential growth of `Vec` tended to waste 40%-75% of the allocated
capacity, and since we could keep ~1024 of these in cache, there's
a lot of potential for waste.
-> Solve by bounding the growth to 64 at a time. This has similar
characteristics to exponential growth at the default 80x24 terminal
size. May need to add a config option for this step size for users
with very large terminals.
* Lazy eviction from the LFU caches. The underlying cache advisor is
somewhat probabilistic and has a minimum cache size of 256, making
it difficult to maintain low heap utilization.
-> Solve by replacing it with a very simple LFU algorithm. It doesn't
seem to hurt much at the default terminal size with the default
cache sizes. If we make the cache sizes smaller, its overhead is
reduced.
Some further experimentation is needed to adjust defaults, but this
should help reduce heap usage.
refs: https://github.com/wez/wezterm/issues/2626
I've expanded the number of bits from 16->32 without impacting
the overall struct sizes and reserved 2 bits for super/subscript.
I refer to these as vertical alignment properties for conceptual
consistency with css.
SGR 73, 74, 75 are used to set super, sub and normal vertical alignment.
These are compatible with mintty.
However, mintty just added support for setting both attributes to render in
small caps in 06ac446049
(https://github.com/mintty/mintty/issues/1171)
According to its benchmarks, it's almost 2x faster than
unicode_segmentation. It doesn't appear to make a visible
difference to `time cat bigfile`, but I'll take anything
that gives more headroom for such little effort of switching.
Action is used to encode parsed terminal output and shuttle it between
the thread that does the parsing and the main gui thread that applies
it to the terminal model.
Take it down from 184 bytes to 40 bytes (on 64-bit systems). This seems
to boost `time cat bigfile` by reducing the runtime to ~40% of its prior
duration: down from 8s -> 4.5s on an M1 macbook air.
Size reductions achieved by Box'ing relatively less frequently
used enum variants. The kitty image data variant is particularly
large, and the Window variant is also pretty heavy.
Remove the faulty cache key stuff and hang references to the cached
data directly off the underlying Line. That makes the association
between the the Line and the data O(1) plus some basic cache
invalidation checks.
Adjust the shape cache portion of this to use the same ID for the
shape, so that things that invalidate just the quads (such as cursor
movement when not composing, and selection) only need to recompute
the quads without re-shaping.
Adds Pane::for_each_logical_line_in_stable_range_mut and
Pane::with_lines_mut which allow iterating mutably over lines.
The idea is that this will allow the renderer to directly cache
data in the Line via its appdata without having to build cumbersome
external caching logic and managing cache keys.
This commit just swaps the implementation around for localpane
and sanity checks that the renderer functions.
Various overlays and the mux client don't properly implement these
yet and current warn at compile time and panic at runtime.
To follow is the logic to cache the data and make sure that it
works the way that I think before converting the other Pane
implementations.
I'd forgotten that this was in here.
The recentish fixes to flush for various terminal queries
likely need this to be 100% good. Surprised this hasn't
come up as a bug already :-/
This makes those fields usable in `wezterm cli list --format json`.
This doesn't change the ABI of the mux protocol, but prior to
this commit, those fields were always 0.
refs: #2319
Just clamp it to the current physical, non-scrollback portion
of the screen.
This avoids a panic but doesn't address the screen size mismatch
in the associated issue.
refs: https://github.com/wez/wezterm/issues/2133
The recent work on the scrollback made it easier to constrain the
search region, so expose those parameters to the Pane::search
interface and to the mux protocol.
Use those new parameters to constrain quickselect search to
1000 rows above and below the current viewport by default, and
add a new parameter to QuickSelectArgs that allows overriding that
range.
A follow-up commit could make the search/copy overlay issue a series
of searches in chunks so that it avoids blocking the UI when
searching very large scrollback.
refs: https://github.com/wez/wezterm/pull/1317
This reduces the resident memory by another ~10% because it avoids
keeping as many runs of whitespace.
Runtime for `time cat enwiki8.wiki` is still ~11-12s, resident: 530K
refs: https://github.com/wez/wezterm/issues/1626
The previous commit added the option to convert the storage to
the cluster format. That saves memory as rows are moved to scrollback,
but makes scrolling back more expensive due to that conversion.
This commit adds a fast(ish) path for the common case of simply
appending text to a new line before it gets scrolled: the default
format for lines in the screen is now the cluster format and,
provided that the cursor moves from left to right as text is
emitted, can simply append to the cluster storage in-place
and avoids a conversion when the line is moved to scrollback.
This improves the throughput of `time cat enwiki8.wiki` so
that the runtime is typically around 11-12s (compared to 9-10s
before introducing cluster storage). However, this is often
a deal of variance in the measured time and I believe that
that is due to the renderer triggering conversions back to
the vec storage and introducing slowdowns from the render side.
That's what I'll investigate next.
This commit causes lines to be "compressed" (really, just translated
to the new clustered line storage variant) as they move into scrollback.
The memory savings are significant for large scrollback:
`wezterm -n --config scrollback_lines=1000000`
`time cat enwiki8.wiki`
before: ~9s, Resident: 2.1G
after: ~15s, Resident: 620K (!)
The performance impact is non-trivial, and I will dig into that
next.
refs: https://github.com/wez/wezterm/issues/1626
