on master help sign in
Revert "Replace stale wcwidth implementation with utf8proc library call"

This reverts commit 437b27f0524d721166f22cbe36ac268b848bec99.

commitcd534c1028b3efde9c13218a10b7e085958496bc authorJonathan Daugherty <cygnus@foobox.com> dateSat 19 Nov 2016, 11:46 (10 years ago) parentc0f919c
3 files changed+209−13
cbits/mk_wcwidth.c+198−2 src/Graphics/Text/Width.hs+9−6 vty.cabal+2−5
cbits/mk_wcwidth.c+198−2
@@ -1,8 +1,204 @@
1+/*
2+ * This is an implementation of wcwidth() and wcswidth() (defined in
3+ * IEEE Std 1002.1-2001) for Unicode.
4+ *
5+ * http://www.opengroup.org/onlinepubs/007904975/functions/wcwidth.html
6+ * http://www.opengroup.org/onlinepubs/007904975/functions/wcswidth.html
7+ *
8+ * In fixed-width output devices, Latin characters all occupy a single
9+ * "cell" position of equal width, whereas ideographic CJK characters
10+ * occupy two such cells. Interoperability between terminal-line
11+ * applications and (teletype-style) character terminals using the
12+ * UTF-8 encoding requires agreement on which character should advance
13+ * the cursor by how many cell positions. No established formal
14+ * standards exist at present on which Unicode character shall occupy
15+ * how many cell positions on character terminals. These routines are
16+ * a first attempt of defining such behavior based on simple rules
17+ * applied to data provided by the Unicode Consortium.
18+ *
19+ * For some graphical characters, the Unicode standard explicitly
20+ * defines a character-cell width via the definition of the East Asian
21+ * FullWidth (F), Wide (W), Half-width (H), and Narrow (Na) classes.
22+ * In all these cases, there is no ambiguity about which width a
23+ * terminal shall use. For characters in the East Asian Ambiguous (A)
24+ * class, the width choice depends purely on a preference of backward
25+ * compatibility with either historic CJK or Western practice.
26+ * Choosing single-width for these characters is easy to justify as
27+ * the appropriate long-term solution, as the CJK practice of
28+ * displaying these characters as double-width comes from historic
29+ * implementation simplicity (8-bit encoded characters were displayed
30+ * single-width and 16-bit ones double-width, even for Greek,
31+ * Cyrillic, etc.) and not any typographic considerations.
32+ *
33+ * Much less clear is the choice of width for the Not East Asian
34+ * (Neutral) class. Existing practice does not dictate a width for any
35+ * of these characters. It would nevertheless make sense
36+ * typographically to allocate two character cells to characters such
37+ * as for instance EM SPACE or VOLUME INTEGRAL, which cannot be
38+ * represented adequately with a single-width glyph. The following
39+ * routines at present merely assign a single-cell width to all
40+ * neutral characters, in the interest of simplicity. This is not
41+ * entirely satisfactory and should be reconsidered before
42+ * establishing a formal standard in this area. At the moment, the
43+ * decision which Not East Asian (Neutral) characters should be
44+ * represented by double-width glyphs cannot yet be answered by
45+ * applying a simple rule from the Unicode database content. Setting
46+ * up a proper standard for the behavior of UTF-8 character terminals
47+ * will require a careful analysis not only of each Unicode character,
48+ * but also of each presentation form, something the author of these
49+ * routines has avoided to do so far.
50+ *
51+ * http://www.unicode.org/unicode/reports/tr11/
52+ *
53+ * Markus Kuhn -- 2007-05-26 (Unicode 5.0)
54+ *
55+ * Permission to use, copy, modify, and distribute this software
56+ * for any purpose and without fee is hereby granted. The author
57+ * disclaims all warranties with regard to this software.
58+ *
59+ * Latest version: http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c
60+ */
161
262#include <HsFFI.h>
3−#include <utf8proc.h>
63+
64+struct interval {
65+ int first;
66+ int last;
67+};
68+
69+/* auxiliary function for binary search in interval table */
70+static int vty_bisearch(HsChar ucs, const struct interval *table, int max) {
71+ int min = 0;
72+ int mid;
73+
74+ if (ucs < table[0].first || ucs > table[max].last)
75+ return 0;
76+ while (max >= min) {
77+ mid = (min + max) / 2;
78+ if (ucs > table[mid].last)
79+ min = mid + 1;
80+ else if (ucs < table[mid].first)
81+ max = mid - 1;
82+ else
83+ return 1;
84+ }
85+
86+ return 0;
87+}
88+
89+
90+/* The following two functions define the column width of an ISO 10646
91+ * character as follows:
92+ *
93+ * - The null character (U+0000) has a column width of 0.
94+ *
95+ * - Other C0/C1 control characters and DEL will lead to a return
96+ * value of -1.
97+ *
98+ * - Non-spacing and enclosing combining characters (general
99+ * category code Mn or Me in the Unicode database) have a
100+ * column width of 0.
101+ *
102+ * - SOFT HYPHEN (U+00AD) has a column width of 1.
103+ *
104+ * - Other format characters (general category code Cf in the Unicode
105+ * database) and ZERO WIDTH SPACE (U+200B) have a column width of 0.
106+ *
107+ * - Hangul Jamo medial vowels and final consonants (U+1160-U+11FF)
108+ * have a column width of 0.
109+ *
110+ * - Spacing characters in the East Asian Wide (W) or East Asian
111+ * Full-width (F) category as defined in Unicode Technical
112+ * Report #11 have a column width of 2.
113+ *
114+ * - All remaining characters (including all printable
115+ * ISO 8859-1 and WGL4 characters, Unicode control characters,
116+ * etc.) have a column width of 1.
117+ *
118+ * This implementation assumes that wchar_t characters are encoded
119+ * in ISO 10646.
120+ */
4121
5122HsInt vty_mk_wcwidth(HsChar ucs)
6123{
7− return (HsInt) utf8proc_charwidth((utf8proc_int32_t) ucs);
124+ /* sorted list of non-overlapping intervals of non-spacing characters */
125+ /* generated by "uniset +cat=Me +cat=Mn +cat=Cf -00AD +1160-11FF +200B c" */
126+ static const struct interval combining[] = {
127+ { 0x0300, 0x036F }, { 0x0483, 0x0486 }, { 0x0488, 0x0489 },
128+ { 0x0591, 0x05BD }, { 0x05BF, 0x05BF }, { 0x05C1, 0x05C2 },
129+ { 0x05C4, 0x05C5 }, { 0x05C7, 0x05C7 }, { 0x0600, 0x0603 },
130+ { 0x0610, 0x0615 }, { 0x064B, 0x065E }, { 0x0670, 0x0670 },
131+ { 0x06D6, 0x06E4 }, { 0x06E7, 0x06E8 }, { 0x06EA, 0x06ED },
132+ { 0x070F, 0x070F }, { 0x0711, 0x0711 }, { 0x0730, 0x074A },
133+ { 0x07A6, 0x07B0 }, { 0x07EB, 0x07F3 }, { 0x0901, 0x0902 },
134+ { 0x093C, 0x093C }, { 0x0941, 0x0948 }, { 0x094D, 0x094D },
135+ { 0x0951, 0x0954 }, { 0x0962, 0x0963 }, { 0x0981, 0x0981 },
136+ { 0x09BC, 0x09BC }, { 0x09C1, 0x09C4 }, { 0x09CD, 0x09CD },
137+ { 0x09E2, 0x09E3 }, { 0x0A01, 0x0A02 }, { 0x0A3C, 0x0A3C },
138+ { 0x0A41, 0x0A42 }, { 0x0A47, 0x0A48 }, { 0x0A4B, 0x0A4D },
139+ { 0x0A70, 0x0A71 }, { 0x0A81, 0x0A82 }, { 0x0ABC, 0x0ABC },
140+ { 0x0AC1, 0x0AC5 }, { 0x0AC7, 0x0AC8 }, { 0x0ACD, 0x0ACD },
141+ { 0x0AE2, 0x0AE3 }, { 0x0B01, 0x0B01 }, { 0x0B3C, 0x0B3C },
142+ { 0x0B3F, 0x0B3F }, { 0x0B41, 0x0B43 }, { 0x0B4D, 0x0B4D },
143+ { 0x0B56, 0x0B56 }, { 0x0B82, 0x0B82 }, { 0x0BC0, 0x0BC0 },
144+ { 0x0BCD, 0x0BCD }, { 0x0C3E, 0x0C40 }, { 0x0C46, 0x0C48 },
145+ { 0x0C4A, 0x0C4D }, { 0x0C55, 0x0C56 }, { 0x0CBC, 0x0CBC },
146+ { 0x0CBF, 0x0CBF }, { 0x0CC6, 0x0CC6 }, { 0x0CCC, 0x0CCD },
147+ { 0x0CE2, 0x0CE3 }, { 0x0D41, 0x0D43 }, { 0x0D4D, 0x0D4D },
148+ { 0x0DCA, 0x0DCA }, { 0x0DD2, 0x0DD4 }, { 0x0DD6, 0x0DD6 },
149+ { 0x0E31, 0x0E31 }, { 0x0E34, 0x0E3A }, { 0x0E47, 0x0E4E },
150+ { 0x0EB1, 0x0EB1 }, { 0x0EB4, 0x0EB9 }, { 0x0EBB, 0x0EBC },
151+ { 0x0EC8, 0x0ECD }, { 0x0F18, 0x0F19 }, { 0x0F35, 0x0F35 },
152+ { 0x0F37, 0x0F37 }, { 0x0F39, 0x0F39 }, { 0x0F71, 0x0F7E },
153+ { 0x0F80, 0x0F84 }, { 0x0F86, 0x0F87 }, { 0x0F90, 0x0F97 },
154+ { 0x0F99, 0x0FBC }, { 0x0FC6, 0x0FC6 }, { 0x102D, 0x1030 },
155+ { 0x1032, 0x1032 }, { 0x1036, 0x1037 }, { 0x1039, 0x1039 },
156+ { 0x1058, 0x1059 }, { 0x1160, 0x11FF }, { 0x135F, 0x135F },
157+ { 0x1712, 0x1714 }, { 0x1732, 0x1734 }, { 0x1752, 0x1753 },
158+ { 0x1772, 0x1773 }, { 0x17B4, 0x17B5 }, { 0x17B7, 0x17BD },
159+ { 0x17C6, 0x17C6 }, { 0x17C9, 0x17D3 }, { 0x17DD, 0x17DD },
160+ { 0x180B, 0x180D }, { 0x18A9, 0x18A9 }, { 0x1920, 0x1922 },
161+ { 0x1927, 0x1928 }, { 0x1932, 0x1932 }, { 0x1939, 0x193B },
162+ { 0x1A17, 0x1A18 }, { 0x1B00, 0x1B03 }, { 0x1B34, 0x1B34 },
163+ { 0x1B36, 0x1B3A }, { 0x1B3C, 0x1B3C }, { 0x1B42, 0x1B42 },
164+ { 0x1B6B, 0x1B73 }, { 0x1DC0, 0x1DCA }, { 0x1DFE, 0x1DFF },
165+ { 0x200B, 0x200F }, { 0x202A, 0x202E }, { 0x2060, 0x2063 },
166+ { 0x206A, 0x206F }, { 0x20D0, 0x20EF }, { 0x302A, 0x302F },
167+ { 0x3099, 0x309A }, { 0xA806, 0xA806 }, { 0xA80B, 0xA80B },
168+ { 0xA825, 0xA826 }, { 0xFB1E, 0xFB1E }, { 0xFE00, 0xFE0F },
169+ { 0xFE20, 0xFE23 }, { 0xFEFF, 0xFEFF }, { 0xFFF9, 0xFFFB },
170+ { 0x10A01, 0x10A03 }, { 0x10A05, 0x10A06 }, { 0x10A0C, 0x10A0F },
171+ { 0x10A38, 0x10A3A }, { 0x10A3F, 0x10A3F }, { 0x1D167, 0x1D169 },
172+ { 0x1D173, 0x1D182 }, { 0x1D185, 0x1D18B }, { 0x1D1AA, 0x1D1AD },
173+ { 0x1D242, 0x1D244 }, { 0xE0001, 0xE0001 }, { 0xE0020, 0xE007F },
174+ { 0xE0100, 0xE01EF }
175+ };
176+
177+ /* test for 8-bit control characters */
178+ if (ucs == 0)
179+ return 0;
180+ if (ucs < 32 || (ucs >= 0x7f && ucs < 0xa0))
181+ return -1;
182+
183+ /* binary search in table of non-spacing characters */
184+ if (vty_bisearch(ucs, combining,
185+ sizeof(combining) / sizeof(struct interval) - 1))
186+ return 0;
187+
188+ /* if we arrive here, ucs is not a combining or C0/C1 control character */
189+
190+ return 1 +
191+ (ucs >= 0x1100 &&
192+ (ucs <= 0x115f || /* Hangul Jamo init. consonants */
193+ ucs == 0x2329 || ucs == 0x232a ||
194+ (ucs >= 0x2e80 && ucs <= 0xa4cf &&
195+ ucs != 0x303f) || /* CJK ... Yi */
196+ (ucs >= 0xac00 && ucs <= 0xd7a3) || /* Hangul Syllables */
197+ (ucs >= 0xf900 && ucs <= 0xfaff) || /* CJK Compatibility Ideographs */
198+ (ucs >= 0xfe10 && ucs <= 0xfe19) || /* Vertical forms */
199+ (ucs >= 0xfe30 && ucs <= 0xfe6f) || /* CJK Compatibility Forms */
200+ (ucs >= 0xff00 && ucs <= 0xff60) || /* Fullwidth Forms */
201+ (ucs >= 0xffe0 && ucs <= 0xffe6) ||
202+ (ucs >= 0x20000 && ucs <= 0x2fffd) ||
203+ (ucs >= 0x30000 && ucs <= 0x3fffd)));
8204}
src/Graphics/Text/Width.hs+9−6
@@ -13,14 +13,17 @@ foreign import ccall unsafe "vty_mk_wcwidth" wcwidth :: Char -> Int
1313wcswidth :: String -> Int
1414wcswidth = sum . map wcwidth
1515
16−-- | Returns the display width of a character. Assumes all characters
17−-- with unknown widths are 0 width. This is equivalent to wcwidth and is
18−-- provided for backwards compatibility.
16+-- XXX: Characters with unknown widths occupy 1 column?
17+--
18+-- Not sure if this is actually correct. I presume there is a replacement character that is output
19+-- by the terminal instead of the character and this replacement character is 1 column wide. If this
20+-- is not true for all terminals then a per-terminal replacement character width needs to be
21+-- implemented.
22+
23+-- | Returns the display width of a character. Assumes all characters with unknown widths are 0 width
1924safeWcwidth :: Char -> Int
2025safeWcwidth = max 0 . wcwidth
2126
22−-- | Returns the display width of a string. Assumes all characters with
23−-- unknown widths are 0 width. This is equivalent to wcwidth and is
24−-- provided for backwards compatibility.
27+-- | Returns the display width of a string. Assumes all characters with unknown widths are 0 width
2528safeWcswidth :: String -> Int
2629safeWcswidth = sum . map safeWcwidth
vty.cabal+2−5
@@ -1,5 +1,5 @@
11name: vty
2−version: 5.12
2+version: 5.11.3
33license: BSD3
44license-file: LICENSE
55author: AUTHORS
@@ -116,10 +116,7 @@ library
116116 cbits/set_term_timing.c
117117 cbits/mk_wcwidth.c
118118
119− extra-libraries: utf8proc
120−
121− include-dirs: cbits /usr/local/include
122− extra-lib-dirs: /usr/local/lib
119+ include-dirs: cbits
123120
124121 hs-source-dirs: src
125122