thac0/vim
1
0
Fork 0
forked from aniani/vim
Code Activity

patch 9.1.1276: inline word diff treats multibyte chars as word char

Browse Source 
Problem:  inline word diff treats multibyte chars as word char
          (after 9.1.1243)
Solution: treat all non-alphanumeric characters as non-word characters
          (Yee Cheng Chin)

Previously inline word diff simply used Vim's definition of keyword to
determine what is a word, which leads to multi-byte character classes
such as emojis and CJK (Chinese/Japanese/Korean) characters all
classifying as word characters, leading to entire sentences being
grouped as a single word which does not provide meaningful information
in a diff highlight.

Fix this by treating all non-alphanumeric characters (with class number
above 2) as non-word characters, as there is usually no benefit in using
word diff on them. These include CJK characters, emojis, and also
subscript/superscript numbers. Meanwhile, multi-byte characters like
Cyrillic and Greek letters will still continue to considered as words.

Note that this is slightly inconsistent with how words are defined
elsewhere, as Vim usually considers any character with class >=2 to be
a "word".

related: #16881 (diff inline highlight)
closes: #17050

Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
This commit is contained in:
Yee Cheng Chin
2025-04-04 19:16:21 +02:00
committed by Christian Brabandt
parent b8d5c85099
commit 9aa120f7ad
6 changed files with 43 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
src/testdir/dumps/Test_diff_inline_word_03.dump Normal file
View File

@@ -0,0 +1,20 @@
| +0#0000e05#a8a8a8255@1|🚀*0#0000000#ffd7ff255|⛵️*2&#ff404010|一*0&#ffd7ff255|二|三*2&#ff404010|ひ*0&#ffd7ff255|ら|が*0&#4040ff13|な*0&#ffd7ff255|Δ+2&#ff404010|έ|λ|τ|α| +0&#ffd7ff255|Δ+2&#ff404010|e|l|t|a| +0&#ffd7ff255|f|o@1|b|a||+1&#ffffff0| +0#0000e05#a8a8a8255@1|🚀*0#0000000#ffd7ff255|🛸*2&#ff404010|一*0&#ffd7ff255|二|四*2&#ff404010|ひ*0&#ffd7ff255|ら|な|δ+2&#ff404010|έ|λ|τ|α| +0&#ffd7ff255|δ+2&#ff404010|e|l|t|a| +0&#ffd7ff255|f|o@1|b|a|r|
|~+0#4040ff13#ffffff0| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|~| @35||+1#0000000&|~+0#4040ff13&| @35
|X+3#0000000&|d|i|f|i|l|e|1| @10|1|,|1| @11|A|l@1| |X+1&&|d|i|f|i|l|e|2| @10|1|,|1| @11|A|l@1
|:+0&&> @73

5
src/testdir/test_diffmode.vim
View File

@@ -2444,6 +2444,11 @@ func Test_diff_inline()
call term_sendkeys(buf, ":set iskeyword+=+\<CR>:wincmd w\<CR>:diffupdate\<CR>")
" Use the previous screen dump as 2nd buffer's iskeyword does not matter
call VerifyInternal(buf, "Test_diff_inline_word_01", " diffopt+=inline:word")
call term_sendkeys(buf, ":windo set iskeyword&\<CR>:1wincmd w\<CR>")
" word diff: test handling of multi-byte characters. Only alphanumeric chars
" (e.g. Greek alphabet, but not CJK/emoji) count as words.
call WriteDiffFiles(buf, ["🚀⛵️一二三ひらがなΔέλτα Δelta foobar"], ["🚀🛸一二四ひらなδέλτα δelta foobar"])
call VerifyInternal(buf, "Test_diff_inline_word_03", " diffopt+=inline:word")