Change character count for AppleScript to grapheme cluster (close #1340)

CHANGELOG.md

@@ -16,6 +16,7 @@
 
 - Change the system requirement to __macOS 14 Sonoma and later__.
 - Add “Select Column Up/Down“ commands to the Edit > Select menu.
+- Change the unit of character ranges handled in CotEditor Scripting for AppleScript from UTF-16 based to the Unicode grapheme cluster-based (This is to follow the specification change in AppleScript 2.0 introduced in Mac OS X 10.5).
 - Improve VoiceOver support in the Quick Action bar.
 - Remove Solarized themes from the bundle.
 - Update all the bundled themes to have a 70% opacity in the current line highlight.

CotEditor/CotEditor.help/Contents/Resources/en.lproj/pgs/script_osascript.html

@@ -147,6 +147,8 @@
 
 <p>The <code>selection</code> property doesn’t work by itself. Use this property with others such as <code>contents</code>.</p>
 
+<p>Starting form CotEditor 5.0, characters are counted in the Unicode grapheme cluster unit. This is the same as the specification of AppleScript 2.0.</p>
+
 <p>When ‘location’ is a negative value, the selection range starts from the ‘location’-th last character.<br/>
 When ‘length’ is a positive value, the selection range becomes the ‘length’ characters starting from ‘location.’ If ‘length’ is larger than the number of the rest characters in the document, the range is from ‘location’ to the end.<br/>
 When ‘length’ is a negative value, the selection range ends at the ‘length’-th last character. If the absolute value of ‘length’ is smaller than ‘location’ (that is, the selection’s end point is before ‘location’), the caret just moves to ‘location’ (same as when {location, 0} was input).</p>

CotEditor/CotEditor.help/Contents/Resources/en.lproj/pgs/script_osascript_changes.html

@@ -16,6 +16,16 @@
 <p>This page lists up the previous specific changes on AppleScript support in CotEditor.</p>
 
 
+<article>
+<h2>Terminology change on CotEditor 5.0.0</h2>
+
+<section>
+	<h3>Change character range unit to grapheme cluster based</h3>
+	<p>Change the character unit, used in <code>selection</code> or <code>jump</code> for instance, from UTF-16 based to the Unicode grapheme cluster-based. This is to follow the specification change in <a href="https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptLangGuide/reference/ASLR_classes.html#//apple_ref/doc/uid/TP40000983-CH1g-BBCIAHJF" ref="external">AppleScript 2.0</a> introduced in Mac OS X 10.5.</p>
+</section>
+</article>
+
+
 <article>
 <h2>Terminology change on CotEditor 4.4.0</h2>
 

CotEditor/CotEditor.help/Contents/Resources/ja.lproj/pgs/script_osascript.html

@@ -147,6 +147,8 @@
 
 <p>「selection」は単独では意味を持ちません。contentsなどのプロパティとともに使用してください。</p>
 
+<p>CotEditor 5.0以降、文字はUnicode書記素クラスタ単位でカウントします。これはAppleScript 2.0の仕様と同一です。</p>
+
 <p>locationが負の場合、対象書類の文字列の後ろから数えてlocation番目から始まる範囲となります。<br/>
 lengthが正である場合、指定される範囲はlocationから数えてlength文字数分となります。また、対象書類の文字列の長さを超えてlengthが入力された場合、末尾までが範囲となります。<br/>
 lengthが負である場合、指定される範囲は対象書類の文字列の後ろから数えてlength文字までとなります。もし、lengthの絶対値がlocationよりも小さい（locationよりも前に終了位置がある）場合には、locationが優先されlocation位置に挿入ポイントが移動します（<code>{location, 0}</code>が入力されたのと同じ）。</p>

CotEditor/CotEditor.help/Contents/Resources/ja.lproj/pgs/script_osascript_changes.html

@@ -16,6 +16,16 @@
 <p>このページでは、CotEditorのAppleScript対応における今までの仕様改訂を列挙しています。</p>
 
 
+<article>
+<h2>CotEditor 5.0.0での仕様改訂</h2>
+
+<section>
+	<h3>文字範囲のカウントをUTF-16ベースから書記素クラスタベースに変更</h3>
+	<p><code>selection</code>や<code>jump</code>などで使われる文字範囲の数値指定をUTF-16ベースからUnicode書記素クラスタベースに変更しました。この変更はMac OS X 10.5で導入された<a href="https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptLangGuide/reference/ASLR_classes.html#//apple_ref/doc/uid/TP40000983-CH1g-BBCIAHJF" ref="external">AppleScript 2.0</a>での変更に追従するものです。</p>
+</section>
+</article>
+
+
 <article>
 <h2>CotEditor 4.4.0での仕様改訂</h2>
 

CotEditor/CotEditor.sdef

@@ -114,7 +114,7 @@
                 <cocoa key="lineRange"/>
                 <type type="integer" list="yes"/>
             </property>
-            <property name="range" code="cRng" access="rw" description="The range of characters in the selection. The format is “{location, length}”.">
+            <property name="range" code="cRng" access="rw" description="The range of characters in the selection in the Unicode grapheme cluster unit. The format is “{location, length}”.">
                 <type type="integer" list="yes"/>
             </property>
             <responds-to command="change case">

CotEditor/Sources/FuzzyRange.swift

@@ -8,7 +8,7 @@
 //
 //  ---------------------------------------------------------------------------
 //
-//  © 2015-2023 1024jp
+//  © 2015-2024 1024jp
 //
 //  Licensed under the Apache License, Version 2.0 (the "License");
 //  you may not use this file except in compliance with the License.
@@ -122,11 +122,11 @@ extension String {
     ///   e.g. Passing `FuzzyRange(location: 3, length: -1)` to a string that has 10 characters returns `NSRange(3..<9)`.
     ///
     /// - Parameters:
-    ///   - range: The character range that allows also negative values.
+    ///   - range: The character range using the grapheme cluster unit that allows also negative values.
     /// - Returns: A character range, or `nil` if the given value is out of range.
     func range(in range: FuzzyRange) -> NSRange? {
         
-        let wholeLength = self.length
+        let wholeLength = self.count
         let newLocation = (range.location >= 0) ? range.location : (wholeLength + range.location + 1)
         let newLength = (range.length >= 0) ? range.length : (wholeLength - newLocation + range.length)
         
@@ -136,7 +136,10 @@ extension String {
             newLocation <= wholeLength
         else { return nil }
         
-        return NSRange(newLocation..<min(newLocation + newLength, wholeLength))
+        let lowerBound = self.index(self.startIndex, offsetBy: newLocation)
+        let upperBound = self.index(lowerBound, offsetBy: newLength, limitedBy: self.endIndex) ?? self.endIndex
+        
+        return NSRange(lowerBound..<upperBound, in: self)
     }
     
     

CotEditor/Sources/TextSelection.swift

@@ -9,7 +9,7 @@
 //  ---------------------------------------------------------------------------
 //
 //  © 2004-2007 nakamuxu
-//  © 2014-2023 1024jp
+//  © 2014-2024 1024jp
 //
 //  Licensed under the Apache License, Version 2.0 (the "License");
 //  you may not use this file except in compliance with the License.
@@ -142,9 +142,14 @@ private enum OSAUnicodeNormalizationType: FourCharCode {
     @objc var range: [Int]? {
         
         get {
-            guard let range = self.textView?.selectedRange else { return nil }
+            guard
+                let textView = self.textView,
+                let string = self.textView?.string,
+                let range = Range(textView.selectedRange, in: string)
+            else { return nil }
             
-            return [range.location, range.length]
+            return [string.distance(from: string.startIndex, to: range.lowerBound),
+                    string.distance(from: range.lowerBound, to: range.upperBound)]
         }
         
         set {

Tests/FuzzyRangeTests.swift

@@ -8,7 +8,7 @@
 //
 //  ---------------------------------------------------------------------------
 //
-//  © 2020-2023 1024jp
+//  © 2020-2024 1024jp
 //
 //  Licensed under the Apache License, Version 2.0 (the "License");
 //  you may not use this file except in compliance with the License.
@@ -37,6 +37,9 @@ final class FuzzyRangeTests: XCTestCase {
         XCTAssertEqual(string.range(in: FuzzyRange(location: -2, length: 1)), NSRange(location: 9, length: 1))
         XCTAssertEqual(string.range(in: FuzzyRange(location: 3, length: -1)), NSRange(3..<9))
         XCTAssertEqual(string.range(in: FuzzyRange(location: 3, length: -2)), NSRange(location: 3, length: "45678".utf16.count))
+        
+        // grapheme cluster count
+        XCTAssertEqual("black 🐈‍⬛ cat".range(in: FuzzyRange(location: 6, length: 2)), NSRange(location: 6, length: 5))
     }