1
1
mirror of https://github.com/wez/wezterm.git synced 2024-12-23 13:21:38 +03:00
Commit Graph

3 Commits

Author SHA1 Message Date
Wez Furlong
c737fb66df bidi: remove stray debug logging, add reordered_runs method 2022-01-28 17:19:36 -07:00
Wez Furlong
0324ff66f0 wezterm: add experimental_bidi config option and very basic bidi
This commit is larger than it appears to due fanout from threading
through bidi parameters.  The main changes are:

* When clustering cells, add an additional phase to resolve embedding
  levels and further sub-divide a cluster based on the resolved bidi
  runs; this is where we get the direction for a run and this needs
  to be passed through to the shaper.
* When doing bidi, the forced cluster boundary hack that we use to
  de-ligature when cursoring through text needs to be disabled,
  otherwise the cursor appears to push/rotate the text in that
  cluster when moving through it! We'll need to find a different
  way to handle shading the cursor that eliminates the original
  cursor/ligature/black issue.
* In the shaper, the logic for coalescing unresolved runs for font
  fallback assumed LTR and needed to be adjusted to cluster RTL.
  That meant also computing a little index of codepoint lengths.
* Added `experimental_bidi` boolean option that defaults to false.
  When enabled, it activates the bidi processing phase in clustering
  with a strong hint that the paragraph is LTR.

This implementation is incomplete and/or wrong for a number of cases:

* The config option should probably allow specifying the paragraph
  direction hint to use by default.
* https://terminal-wg.pages.freedesktop.org/bidi/recommendation/paragraphs.html
  recommends that bidi be applied to logical lines, not physical
  lines (or really: ranges within physical lines) that we're doing
  at the moment
* The paragraph direction hint should be overridden by cell attributes
  and other escapes; see 85a6b178cf

and probably others.

However, as of this commit, if you `experimental_bidi=true` then

```
echo This is RTL -> عربي فارسی bidi
```

(that text was sourced from:
https://github.com/microsoft/terminal/issues/538#issuecomment-677017322)

then wezterm will display the text in the same order as the text
renders in Chrome for that github comment.

```
; ./target/debug/wezterm --config experimental_bidi=false ls-fonts --text "عربي فارسی ->"
LeftToRight
 0 ع    \u{639}      x_adv=8  glyph=300  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 2 ر    \u{631}      x_adv=3.78125 glyph=273  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 4 ب    \u{628}      x_adv=4  glyph=244  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 6 ي    \u{64a}      x_adv=4  glyph=363  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 8      \u{20}       x_adv=8  glyph=2    wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
                                      /Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
 9 ف    \u{641}      x_adv=11 glyph=328  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
11 ا    \u{627}      x_adv=4  glyph=240  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
13 ر    \u{631}      x_adv=3.78125 glyph=273  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
15 س    \u{633}      x_adv=10 glyph=278  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
17 ی    \u{6cc}      x_adv=4  glyph=664  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
19      \u{20}       x_adv=8  glyph=2    wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
                                      /Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
20 -    \u{2d}       x_adv=8  glyph=276  wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
                                      /Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
21 >    \u{3e}       x_adv=8  glyph=338  wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
                                      /Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
```

```
; ./target/debug/wezterm --config experimental_bidi=true ls-fonts --text "عربي فارسی ->"
RightToLeft
17 ی    \u{6cc}      x_adv=9  glyph=906  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
15 س    \u{633}      x_adv=10 glyph=277  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
13 ر    \u{631}      x_adv=4.78125 glyph=272  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
11 ا    \u{627}      x_adv=4  glyph=241  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 9 ف    \u{641}      x_adv=5  glyph=329  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 8      \u{20}       x_adv=8  glyph=2    wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
                                      /Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
 6 ي    \u{64a}      x_adv=9  glyph=904  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 4 ب    \u{628}      x_adv=4  glyph=243  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 2 ر    \u{631}      x_adv=5  glyph=273  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
 0 ع    \u{639}      x_adv=6  glyph=301  wezterm.font(".Geeza Pro Interface", {weight="Regular", stretch="Normal", italic=false})
                                      /System/Library/Fonts/GeezaPro.ttc index=2 variation=0, CoreText
LeftToRight
 0      \u{20}       x_adv=8  glyph=2    wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
                                      /Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
 1 -    \u{2d}       x_adv=8  glyph=480  wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
                                      /Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
 2 >    \u{3e}       x_adv=8  glyph=470  wezterm.font("Operator Mono SSm Lig", {weight="DemiLight", stretch="Normal", italic=false})
                                      /Users/wez/.fonts/OperatorMonoSSmLig-Medium.otf, FontDirs
;
```

refs: https://github.com/wez/wezterm/issues/784
2022-01-25 09:00:53 -07:00
Wez Furlong
601a85e12b Add wezterm-bidi crate
In order to support RTL/BIDI, wezterm needs a bidi implementation.  I
don't think a well-conforming rust implementation exists today; what I
found were implementations that didn't pass 100% of the conformance
tests.

So I decided to port "bidiref", the reference implementation of the UBA
described in http://unicode.org/reports/tr9/ to Rust.

This implementation focuses on conformance: no special measures have
been taken to optimize it so far, with my focus having been to ensure
that all of the approx 780,000 test cases in the unicode data for
unicode 14 pass.  Having the tests passing 100% allows for making
performance improvements with confidence in the future.

The API isn't completely designed/fully baked.  Until I get to hooking
it up to wezterm's shaper, I'm not 100% sure exactly what I'll need.
There's a good discussion on API in
https://github.com/open-i18n/rust-unic/issues/273 that suggests omitting
"legacy" operations such as reordering. I suspect that wezterm may
actually need that function to support monospace text layout in some
terminal scenarios, but regardless: reordering is part of the
conformance test suite so it remains a part of the API.

That said: the API does model the major operations as separate
phases, so you should be able to pay for just what you use:

* Resolving the embedding levels from a paragraph
* Returning paragraph runs of those levels (and their directions)
* Returning the whitespace-level-reset runs for a line-slice within the
  paragraph
* Returning the reordered indices + levels for a line-slice within the
  paragraph.

refs: https://github.com/wez/wezterm/issues/784
refs: https://github.com/kas-gui/kas-text/issues/20
2022-01-24 19:29:03 -07:00