Add escape sequence to control unicode version

As promised in the previous commit, this one implements an escape
sequence to control the unicode version.

Unknown to me in the previous commit, iTerm2 already defines such
an escape sequence, so we simply implement it here with the same
semantics.

refs: #1231
refs: #997

docs/changelog.md

@@ -28,6 +28,7 @@ As features stabilize some brief notes about them will accumulate here.
 * [canonicalize_pasted_newlines](config/lua/config/canonicalize_pasted_newlines.md) option to help Windows users manage newlines in pastes [#1213](https://github.com/wez/wezterm/issues/1213)
 * SSH client now uses `libssh` by default. [ssh_backend](config/lua/config/ssh_backend.md) can be used to change this.
 * [unzoom_on_switch_pane](config/lua/config/unzoom_on_switch_pane.md) option. Thanks to [@yyogo](https://github.com/yyogo) [#1301](https://github.com/wez/wezterm/issues/1301)
+* [unicode_version](config/lua/config/unicode_version.md) option and corresponding OSC escape sequences that affects how the width of certain unicode sequences are interpreted.
 
 #### Changed
 

docs/config/lua/config/unicode_version.md

@@ -0,0 +1,105 @@
+# `unicode_version = 9`
+
+*Since nightly builds only*
+
+Specifies the version of unicode that will be used when interpreting the
+width/presentation of text.
+
+This option exists because Unicode is an evolving specification that introduces
+new features and that occasionally adjusts how existing features should be
+handled.
+
+For example, there were a number of unicode code points that had their width
+changed between Unicode version 8 and version 9. This wouldn't be an issue
+if all software was simultaneously aware of the change, but the reality is
+that there is a lot of older software out there, and that even if your local
+system is fully up to date, you might connect to a remote system vis SSH
+that is running applications that use a different version of unicode than
+your local system.
+
+The impact of mismatching expectations of unicode width for a terminal emulator
+is that text columns may no longer line up as the application author expected,
+and/or that the cursor may appear to be in the wrong place when editing lines
+or text in shells or text editors.
+
+The `unicode_version` option defaults to unicode version 9 as that is the most
+widely used version (from the perspective of width) at the time of writing,
+which means that the default experience has the lowest chance of mismatched
+expectations.
+
+| Unicode Version | Impact |
+| --------------- | ------ |
+| 8 (or lower)    | Some characters will be narrower than later versions |
+| 9-13            | Some characters will be wider than in Unicode 8 |
+| 14 (or higher)  | Explicit Emoji or Text presentation selectors will be respected and make some characters wider or narrower than earlier versions, depending on the context |
+
+If you aggressively maintain all of your software to the latest possible
+versions then you may wish to set `unicode_version = 14` to match the current
+(at the time of writing) version of Unicode.  This will enable Emoji
+Presentation selectors to affect the presentation of certain emoji characters
+and alter their width in the terminal display.
+
+If you'd like to use a higher default version but switch to a lower version
+when launching an older application, or when SSH'ing into a remote host, then
+you may be pleased to learn that wezterm also provides an escape sequence that
+allows the unicode version to be changed on the fly.
+
+## Unicode Version Escape sequence
+
+This escape sequence is was originally defined by iTerm2. It supports setting
+the value as well as pushing and popping the value on a stack, which is helpful
+when temporarily adjusting the value.
+
+```
+OSC 1337 ; UnicodeVersion=N ST
+```
+
+The above sets the unicode version to `N`, where N is the integer version number.
+
+```
+OSC 1337 ; UnicodeVersion=push ST
+```
+
+Pushes the current version onto a stack.
+
+```
+OSC 1337 ; UnicodeVersion=pop ST
+```
+
+Pops the last-pushed version from the stack and sets the unicode version to that value.
+If there were no entries on the stack, the unicode version is left unchanged.
+
+```
+OSC 1337 ; UnicodeVersion=push LABEL ST
+```
+
+Pushes the current version onto a stack, labeling it with `LABEL`.
+
+```
+OSC 1337 ; UnicodeVersion=pop LABEL ST
+```
+
+Pops entries from the stack stopping after an entry labelled with `LABEL` is popped.
+
+
+The labels are helpful when writing a wrapper alias, for example:
+
+```bash
+function run_with_unicode_version_9() {
+  local label=$(uuidgen)
+  printf "\e]1337;UnicodeVersion=push %s\e\\" $label
+  printf "\e]1337;UnicodeVersion=9\e\\"
+  eval $@
+  local result=${PIPESTATUS[0]}
+  printf "\e]1337;UnicodeVersion=pop %s\e\\" $label
+  return $result
+}
+
+# Connect to a remote machine with an older version of unicode
+run_with_unicode_version_9 ssh remote.machine
+```
+
+Will save the current version on the stack with a unique label, then set the version to `9`
+and spawn the requested command.  When the command returns it will restore the saved
+version, even if the command itself set or pushed other values.
+

term/src/terminalstate/mod.rs

@@ -345,6 +345,13 @@ pub struct TerminalState {
 
     /// The unicode version that is in effect
     unicode_version: UnicodeVersion,
+    unicode_version_stack: Vec<UnicodeVersionStackEntry>,
+}
+
+#[derive(Debug)]
+struct UnicodeVersionStackEntry {
+    vers: UnicodeVersion,
+    label: Option<String>,
 }
 
 fn default_color_map() -> HashMap<u16, RgbColor> {
@@ -474,6 +481,7 @@ impl TerminalState {
             kitty_img: Default::default(),
             seqno: 0,
             unicode_version,
+            unicode_version_stack: vec![],
         }
     }
 

term/src/terminalstate/performer.rs

@@ -1,15 +1,16 @@
 use crate::terminal::Alert;
-use crate::terminalstate::{default_color_map, CharSet, TabStop};
+use crate::terminalstate::{default_color_map, CharSet, TabStop, UnicodeVersionStackEntry};
 use crate::{ClipboardSelection, Position, TerminalState, VisibleRowIndex};
 use crate::{DCS, ST};
 use log::{debug, error};
 use num_traits::FromPrimitive;
 use std::fmt::Write;
 use std::ops::{Deref, DerefMut};
-use termwiz::cell::{grapheme_column_width, Cell, CellAttributes, SemanticType};
+use termwiz::cell::{grapheme_column_width, Cell, CellAttributes, SemanticType,UnicodeVersion};
 use termwiz::escape::csi::EraseInDisplay;
 use termwiz::escape::osc::{
     ChangeColorPair, ColorOrQuery, FinalTermSemanticPrompt, ITermProprietary, Selection,
+    ITermUnicodeVersionOp,
 };
 use termwiz::escape::{
     Action, ControlCode, DeviceControlMode, Esc, EscCode, OperatingSystemCommand, CSI,
@@ -499,6 +500,8 @@ impl<'a> Performer<'a> {
                 self.palette.take();
                 self.top_and_bottom_margins = 0..self.screen().physical_rows as VisibleRowIndex;
                 self.left_and_right_margins = 0..self.screen().physical_cols;
+                self.unicode_version = UnicodeVersion(self.config.unicode_version());
+                self.unicode_version_stack.clear();
 
                 self.screen.activate_primary_screen(seqno);
                 self.erase_in_display(EraseInDisplay::EraseScrollback);
@@ -573,6 +576,29 @@ impl<'a> Performer<'a> {
                         handler.alert(Alert::TitleMaybeChanged);
                     }
                 }
+                ITermProprietary::UnicodeVersion(ITermUnicodeVersionOp::Set(n)) => {
+                    self.unicode_version = UnicodeVersion(n);
+                }
+                ITermProprietary::UnicodeVersion(ITermUnicodeVersionOp::Push(label)) => {
+                    let vers = self.unicode_version;
+                    self.unicode_version_stack.push(UnicodeVersionStackEntry {
+                        vers,
+                        label
+                    });
+                }
+                ITermProprietary::UnicodeVersion(ITermUnicodeVersionOp::Pop(None)) => {
+                    if let Some(entry) = self.unicode_version_stack.pop() {
+                        self.unicode_version = entry.vers;
+                    }
+                }
+                ITermProprietary::UnicodeVersion(ITermUnicodeVersionOp::Pop(Some(label))) => {
+                    while let Some(entry) = self.unicode_version_stack.pop() {
+                        self.unicode_version = entry.vers;
+                        if entry.label.as_deref() == Some(&label) {
+                            break;
+                        }
+                    }
+                }
                 _ => log::warn!("unhandled iterm2: {:?}", iterm),
             },
 

termwiz/src/escape/osc.rs

@@ -826,6 +826,16 @@ pub enum ITermProprietary {
     SetBadgeFormat(String),
     /// Download file data from the application.
     File(Box<ITermFileData>),
+
+    /// Configure unicode version
+    UnicodeVersion(ITermUnicodeVersionOp),
+}
+
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum ITermUnicodeVersionOp {
+    Set(u8),
+    Push(Option<String>),
+    Pop(Option<String>),
 }
 
 #[derive(Debug, Clone, PartialEq, Eq)]
@@ -1136,6 +1146,31 @@ impl ITermProprietary {
             }
         }
 
+        if osc.len() == 2 && keyword == "UnicodeVersion" {
+            if let Some(p1) = p1 {
+                let mut iter = p1.splitn(2, ' ');
+                let keyword = iter.next();
+                let label = iter.next();
+
+                if let Some("push") = keyword {
+                    return Ok(ITermProprietary::UnicodeVersion(
+                        ITermUnicodeVersionOp::Push(label.map(|s| s.to_string())),
+                    ));
+                }
+                if let Some("pop") = keyword {
+                    return Ok(ITermProprietary::UnicodeVersion(
+                        ITermUnicodeVersionOp::Pop(label.map(|s| s.to_string())),
+                    ));
+                }
+
+                if let Ok(n) = p1.parse::<u8>() {
+                    return Ok(ITermProprietary::UnicodeVersion(
+                        ITermUnicodeVersionOp::Set(n),
+                    ));
+                }
+            }
+        }
+
         if keyword == "File" {
             return Ok(ITermProprietary::File(Box::new(ITermFileData::parse(osc)?)));
         }
@@ -1171,6 +1206,15 @@ impl Display for ITermProprietary {
             }
             SetBadgeFormat(s) => write!(f, "SetBadgeFormat={}", base64::encode(s))?,
             File(file) => file.fmt(f)?,
+            UnicodeVersion(ITermUnicodeVersionOp::Set(n)) => write!(f, "UnicodeVersion={}", n)?,
+            UnicodeVersion(ITermUnicodeVersionOp::Push(Some(label))) => {
+                write!(f, "UnicodeVersion=push {}", label)?
+            }
+            UnicodeVersion(ITermUnicodeVersionOp::Push(None)) => write!(f, "UnicodeVersion=push")?,
+            UnicodeVersion(ITermUnicodeVersionOp::Pop(Some(label))) => {
+                write!(f, "UnicodeVersion=pop {}", label)?
+            }
+            UnicodeVersion(ITermUnicodeVersionOp::Pop(None)) => write!(f, "UnicodeVersion=pop")?,
         }
         Ok(())
     }