When falling back to presentation-ambivalent font lookup, restart
our shaper run from font_idx=0 rather than the current font index.
For:
```
wezterm ls-fonts --text "$(printf \\U1faf0)"
```
the current font index was the last resort font so we'd never
search over anything useful in the second pass.
Looking back at 4fc8dfb374, I can't
see a good reason for starting at the later offset.
We were getting data like this back from harfbuzz:
shaped font_idx=0 "파인더에서 만든 폴더" as: [.notdef=0+448|.notdef=0+448|.notdef=6+448|.notde=6+448|.notdef=6+448|.notdef=15+448|.notdef=15+448|.notdef=21+448|.notdef=21+448|.notdef=27+448|.notdef=27+448|space=33+448|.notdef=34+448|.notdef=34+448|.notdef=34+448|.notdef=43+448|.notdef=43+448|.notdef=43+448|space=52+448|.notdef=53+448|.notdef=53+448|.notdef=53+448|.notdef=62+448|.notdef=62+448]
which had a series of discontiguous and repeated runs for the same
cluster/region, but due to a logic error, we weren't coalescing them
together and were passing somewhat nonsensical ranges to the
next font for shaping.
This commit "simply" ensures that we code the checks for codepoint==0
(the `.nodef` seen above) and the results are at least correct
for this particular case.
refs: https://github.com/wez/wezterm/issues/2482
This commit is a result of debugging a problem with a particular font
where a VS2 variation of a glyph produced incorrect shaping.
Further investigation showed that this was specific to using freetype
font functions and that switching to opentype functions produces the
correct shaping information in that case.
Further-further investigation showed that this difference in behavior
was really due to the font file being out of spec and freetype returning
unexpected data as a result.
This commit allows switching to using harfbuzz's own opentype font+face
(rather than freetype) and/or switching the freetype-based font to using
opentype font functions. These two options don't yield equivalent
results in the wezterm integration: a couple of our shaping tests
fail due to the x-advances not being the same. Those can be drastically
different in some cases (eg: I seem to get 0 for certain bitmap strikes,
and for Roboto the value is off by a factor of about 1.5).
This is incomplete and not something I want to turn on by default at
the moment, but I don't want to lose this work, hence this commit.
refs: https://github.com/wez/wezterm/issues/2475
refs: https://github.com/harfbuzz/harfbuzz/issues/3806
We've been using hb with freetype faces. This gives us the option
of using harfbuzz's own opentype based face and function set.
It doesn't change any behavior: this is just newly available code.
refs: https://github.com/wez/wezterm/issues/2475
CTRL-SHIFT-U is a new default key assignment for this new modal.
It opens up a fuzzy searchable browser that defaults to showing
emoji/emoticons. The category can by cycled through the suggested
emoji categories using CTRL-r. Unlike the system emoji palette,
wezterm includes a category for nerdfont symbols, and another
that is a list of all unicode codepoint names, so you should be
able to browse for pretty much any codepoint you can think of.
The modal also allows fuzzy searching based on:
* The official unicode name
* The github shortcode
* codepoint value in hex
so if you know the codepoint value but not the name, you can
still find a way to input what you're looking for.
Pressing Enter will copy the selected item to the clipboard
send it to the active pane, and cancel the modal. You can therefore
repeat the insert by simply pasting.
I plan to add frecency to this in a later commit: that way the
frequently/recently used selections will show in a category of
their own and make it easier to re-input them.
refs: https://github.com/wez/wezterm/issues/2163
This avoids a confusing situation where we'd not bother to resolve
a font for a given codepoint for say bold text if we had previously
resolved it for normal text.
This hashset would be more useful if we maintained a mapping of
codepoint to font at that layer, but we don't.
The simplest thing to do for this is to allow the search to continue
and succeed.
refs: https://github.com/wez/wezterm/issues/2158
refs: https://github.com/wez/wezterm/issues/1913
wezterm makes two passes over fonts when resolving emoji, based on
the presentation of the text. If the text has a defined presentation
via a variation selector, then the first pass will only consider
fonts that match that presentation.
If no fonts match, a second pass is made to locate any font with
the appropriate glyphs.
Previously, the font was assumed to have emoji presentation if it
had color glyphs. That meant that if you chose a monochrome emoji
font then it would be skipped for regular emoji presentation and
we'd end up matching against the default fallback for the Noto Color
Emoji font.
This commit changes the behavior to consider any explicitly listed
font with "Emoji" in the name as having emoji presentation.
This is also an imperfect heuristic, but perhaps it is good enough?
refs: #1959
Freetype has a compile-time feature that, when enabled, rewrites the
font names of PCF fonts to include the foundry and wide status of the
font in order to disambiguate the various versions of fonts all named
"Fixed".
That option is enabled by default in some linux distributions but not
all; it's not enabled in Fedora, for example.
When that feature is enabled it causes problems for the Terminus font as
the PCF version of the fonts are no longer resolvable via the simple
"Terminus" name but via "xos4 Terminus" instead.
wezterm builds its own version of freetype (for consistency and cross
platform support reasons), and is unaware of the choice used by the
distro.
The result of that is that fontconfig may see PCF fonts as having
different font names than how wezterm sees them.
A concrete problem is on such a system, when requesting "xos4 Terminus",
fontconfig will present a match with that name, but when wezterm opens
the font and sees that it has name "Terminus" (because of the difference
in this feature in the freetype libraries in use), wezterm will reject
that match.
This commit enables that option in the freetype library and adds a
wezterm config level option (freetype_pcf_long_family_names) that can be
used to control the underlying pcf font driver configuration.
The upshot of this is that this commit doesn't change any default
behavior, but allows users of those systems to set
`freetype_pcf_long_family_names = true` to turn that behavior on.
My personal opinion on this is that users probably shouldn't use this if
they can avoid it (and PCF fonts in general), and instead install the
OTB version of the Terminus font, which doesn't have this legacy baggage
associated with it!
refs: https://github.com/wez/wezterm/issues/2100
For fonts like Lucida Console on Windows which do not have a bold
variant, we were not synthesizing bold.
The reason was that the config-level "make bold" logic works by adding
200 to the weight which takes normal -> demibold, but the bold synthesis
logic is enabled only for bold and higher.
This commit changes the threshold for synthesis to demibold or higher.
refs: https://github.com/wez/wezterm/issues/2074
harfbuzz can return incomplete overlapping runs when it processes
text in unicode NFD. Add another check for the case where we've
accumulated the bytes in the range 0-12 and then harfbuzz returns
another range of 6-12. We coalesce the two together so that we can
pass the full unresolved sequence to the next fallback pass.
refs: https://github.com/wez/wezterm/issues/2032
This is still a bit of a WIP, but this commit:
* Introduces a new "Modal" concept to the GUI layer. The intent is
that modal intercepts key and mouse events while active, and renders
over the top of the rest of the normal display.
I think there might be a couple of cases where key events skirt
through this, but this is good enough as a first step.
Also, the render is forced into layer 1 which has some funny side
effects: if the modal choses to render transparent, it will poke
a hole in the window because all the rendering happens together:
there aren't distinct layer compositing passes.
* Add a new PaneSelect action that is implemented as a modal.
It uses quickselect style alphabet -> pane label generation and
renders the labels over ~the middle of each pane using an
enlarged version of the window frame font. Typing the label
will activate that pane. Escape will cancel the modal.
More styling and docs will follow in a later commit.
refs: #1975
Go directly to the underlying env_logger crate, as pretty_env_logger
hasn't been updated in some time, and I'd like to be able to redirect
the log output to a file more directly, and that feature is in a newer
version of the env logger than pretty_env_logger was pulling in.
Some versions of fontconfig classify some fonts as having charcell
spacing. We need to consider those as monospace as well.
refs: https://github.com/wez/wezterm/issues/1820
We were using a simple hashmap of name -> parsed file, but for
something like Terminus where there are ~10 files per weight
but for different pixel sizes, we'd end up forgetting files
beyond the first.
This commit tracks the full list in the font db.
related to #1820 in the sense that I needed this to confirm that
we do handle BDFs, but it is not the issue reported there because
that was sourcing fonts via fontconfig on linux.
The gist of the issue is that when setting eg: scale=1.2 to draw
a larger CJK glyph, it is drawn at the same descender level, which
makes it more likely to leave the top of the cell.
This commit adjusts the y position by the difference between the
original and the scaled descender so that is less likely to cause
problems.
refs: https://github.com/wez/wezterm/issues/1803
The broot icon font has glyphs with horizontal advance set to 0. That
would cause us to consider the glyph to be zero width, so handle that as
a special case. Note that it is legit for certain cells to end up with
a zero advance/width during shaping if they represent combining
characters: this is more common in Arabic scripts.
refs: https://github.com/wez/wezterm/issues/1787
This commit allows specifying a scaling factor as part of the font
attribute definition. This scaling factor is fed through to the
rasterizer and the shaper to adjust the actual font size that is
loaded.
The intent is to provide manual control for situations where the
fallback font has a different scale to the primary font and renders
either too small or too large.
The concrete example is
https://github.com/wez/wezterm/issues/1761#issuecomment-1079708207 where
the CJK fallback looks too small.
The scaling factor doesn't influence font metrics so it may also be
desirable to configure line height.
```lua
local wezterm = require 'wezterm'
return {
line_height = 1.2,
font = wezterm.font_with_fallback({
"JetBrains Mono",
{family="Microsoft YaHei", scale=1.5},
}),
}
```
freetype can't handle a wide range of encodings for
font names and can return strings like `?????` when
the family name is only present in the font as a non-unicode encoding,
such as Chinese.
This commit improves our handling of the font name table
and prefers to use results from processing that over the
results returned for eg: font family directly from the
freetype API.
refs: https://github.com/wez/wezterm/issues/1761
after reading around italic vs. oblique, I think "slant" is a misleading
way to categorize this, as slant implies something about the angle of
the font but really the difference between italic and oblique is purely
stylistic and without a suggested angle.
style more closely matches the CSS name which is well understood by
many, so we go for that.
refs: #1646
There are three slants that are broadly recognized; normal, italic and
oblique. Prior to this commit, we only considered normal or italic.
This is mostly a mechanical change to replace the boolean with the enum,
and rename the field from `italic` to `slant`.
For the sake of backwards compatibility with existing configs, the lua
helpers for working with fonts continue to accept boolean italic values
and rewrite them to the equivalent slant value.
refs: #1646
Resolves a little bit of the awkward duplication of color types
between some of the crates by factoring them a little bit better.
This is prep for allowing specifying alpha for some colors
in the config.
For a sequence like `e U+20d7` the intent is to render the `e` with
a vector arrow over the top.
This is typically implemented by fonts as an `e` followed by the
vector glyph (or vice versa), where either one of those may have
a zero advance so that the two elements are combined.
There were two problems here:
* During shaping we'd see the zero advance and assume that the entry
was useless and skip it
* During rendering, if we didn't think it had any cell width, we'd
not render it
Cursoring through that particular sequence can hide the vector
mark if the cursor is set to the default block cursor due to annoyances
in how the block cursor is rendered (it changes the fg color to match
the bg, but for elements outside where we think the cursor is, this
makes those elements invisible).
refs: https://github.com/wez/wezterm/issues/1617
This is a more robust approach; we make a separate pass to figure
out information about the (harfbuzz) cluster for a sequence of glyphs,
and then map that sequence back to the original cell sequence, and
from there compute the total cell width for the run, then distribute
the glyphs across the run.
This should yield more sane results for bidi.
Fixup the x-position math; it was still wonky despite the
efforts in 5f2c905db8 and
af92265ffb
refs: #1570
refs: #1607
refs: #1563
This allows unicode_version to be respected again when rendering.
The updated emoji-presentation.sh script now highlights this slightly
better by putting `.` characters after the emoji; unicode version 14
emoji presentation will show the `.` in the 3rd column, while earlier
versions will show it in the 2nd column for glyphs that are sensitive
to the version.
refs: #1607
refs: #1563
This commit is larger than it appears to due fanout from threading
through bidi parameters. The main changes are:
* When clustering cells, add an additional phase to resolve embedding
levels and further sub-divide a cluster based on the resolved bidi
runs; this is where we get the direction for a run and this needs
to be passed through to the shaper.
* When doing bidi, the forced cluster boundary hack that we use to
de-ligature when cursoring through text needs to be disabled,
otherwise the cursor appears to push/rotate the text in that
cluster when moving through it! We'll need to find a different
way to handle shading the cursor that eliminates the original
cursor/ligature/black issue.
* In the shaper, the logic for coalescing unresolved runs for font
fallback assumed LTR and needed to be adjusted to cluster RTL.
That meant also computing a little index of codepoint lengths.
* Added `experimental_bidi` boolean option that defaults to false.
When enabled, it activates the bidi processing phase in clustering
with a strong hint that the paragraph is LTR.
This implementation is incomplete and/or wrong for a number of cases:
* The config option should probably allow specifying the paragraph
direction hint to use by default.
* https://terminal-wg.pages.freedesktop.org/bidi/recommendation/paragraphs.html
recommends that bidi be applied to logical lines, not physical
lines (or really: ranges within physical lines) that we're doing
at the moment
* The paragraph direction hint should be overridden by cell attributes
and other escapes; see 85a6b178cf
and probably others.
However, as of this commit, if you `experimental_bidi=true` then
```
echo This is RTL -> عربي فارسی bidi
```
(that text was sourced from:
https://github.com/microsoft/terminal/issues/538#issuecomment-677017322)
then wezterm will display the text in the same order as the text
renders in Chrome for that github comment.
```
; ./target/debug/wezterm --config experimental_bidi=false ls-fonts --text "عربي فارسی ->"
LeftToRight
0 ع \u{639} x_adv=8 glyph=300 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
2 ر \u{631} x_adv=3.78125 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
4 ب \u{628} x_adv=4 glyph=244 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
6 ي \u{64a} x_adv=4 glyph=363 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
8 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
9 ف \u{641} x_adv=11 glyph=328 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
11 ا \u{627} x_adv=4 glyph=240 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
13 ر \u{631} x_adv=3.78125 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
15 س \u{633} x_adv=10 glyph=278 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
17 ی \u{6cc} x_adv=4 glyph=664 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
19 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
20 - \u{2d} x_adv=8 glyph=276 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
21 > \u{3e} x_adv=8 glyph=338 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
```
```
; ./target/debug/wezterm --config experimental_bidi=true ls-fonts --text "عربي فارسی ->"
RightToLeft
17 ی \u{6cc} x_adv=9 glyph=906 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
15 س \u{633} x_adv=10 glyph=277 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
13 ر \u{631} x_adv=4.78125 glyph=272 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
11 ا \u{627} x_adv=4 glyph=241 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
9 ف \u{641} x_adv=5 glyph=329 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
8 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
6 ي \u{64a} x_adv=9 glyph=904 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
4 ب \u{628} x_adv=4 glyph=243 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
2 ر \u{631} x_adv=5 glyph=273 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
0 ع \u{639} x_adv=6 glyph=301 wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
/System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
LeftToRight
0 \u{20} x_adv=8 glyph=2 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
1 - \u{2d} x_adv=8 glyph=480 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
2 > \u{3e} x_adv=8 glyph=470 wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
/Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
;
```
refs: https://github.com/wez/wezterm/issues/784
