patch 9.1.1476: no easy way to deduplicate text
Problem: no easy way to deduplicate text
Solution: add the :uniq ex command
(Hirohito Higashi)
closes: #17538
Signed-off-by: Hirohito Higashi <h.east.727@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
diff --git a/runtime/doc/builtin.txt b/runtime/doc/builtin.txt
index 8c79f56..8bd914b 100644
--- a/runtime/doc/builtin.txt
+++ b/runtime/doc/builtin.txt
@@ -1,4 +1,4 @@
-*builtin.txt* For Vim version 9.1. Last change: 2025 Jun 07
+*builtin.txt* For Vim version 9.1. Last change: 2025 Jun 23
VIM REFERENCE MANUAL by Bram Moolenaar
@@ -12143,6 +12143,7 @@
:let newlist = uniq(copy(mylist))
< The default compare function uses the string representation of
each item. For the use of {func} and {dict} see |sort()|.
+ For deduplicating text in the current buffer see |:uniq|.
Returns zero if {list} is not a |List|.
diff --git a/runtime/doc/change.txt b/runtime/doc/change.txt
index 6bf9e2c..65e4e72 100644
--- a/runtime/doc/change.txt
+++ b/runtime/doc/change.txt
@@ -1,4 +1,4 @@
-*change.txt* For Vim version 9.1. Last change: 2025 May 28
+*change.txt* For Vim version 9.1. Last change: 2025 Jun 23
VIM REFERENCE MANUAL by Bram Moolenaar
@@ -20,6 +20,7 @@
5. Copying and moving text |copy-move|
6. Formatting text |formatting|
7. Sorting text |sorting|
+8. Deduplicating text |deduplicating|
For inserting text see |insert.txt|.
@@ -1895,6 +1896,7 @@
Vim has a sorting function and a sorting command. The sorting function can be
found here: |sort()|, |uniq()|.
+Also see |:uniq|.
*:sor* *:sort*
:[range]sor[t][!] [b][f][i][l][n][o][r][u][x] [/{pattern}/]
@@ -1904,7 +1906,7 @@
With [!] the order is reversed.
With [i] case is ignored.
-
+ *:sort-l*
With [l] sort uses the current collation locale.
Implementation details: strcoll() is used to compare
strings. See |:language| to check or set the collation
@@ -1937,13 +1939,14 @@
With [b] sorting is done on the first binary number in
the line (after or inside a {pattern} match).
-
+ *:sort-u* *:sort-uniq*
With [u] (u stands for unique) only keep the first of
a sequence of identical lines (ignoring case when [i]
is used). Without this flag, a sequence of identical
lines will be kept in their original order.
Note that leading and trailing white space may cause
lines to be different.
+ When you just want to make things unique, use |:uniq|.
When /{pattern}/ is specified and there is no [r] flag
the text matched with {pattern} is skipped, so that
@@ -1990,4 +1993,56 @@
process you may end up with duplicated lines. This also depends on the system
library function used.
+==============================================================================
+8. Deduplicating text *deduplicating* *unique*
+
+Vim has a deduplicating function and a deduplicating command. The
+deduplicating function can be found here: |uniq()|.
+Also see |:sort-uniq|.
+
+ *:uni* *:uniq*
+:[range]uni[q][!] [i][l][r][u] [/{pattern}/]
+ Remove duplicate lines that are adjacent to each other
+ in [range]. When no range is given, all lines are
+ processed.
+
+ With [i] case is ignored when comparing lines.
+
+ With [l] comparison uses the current collation locale.
+ See |:sort-l| for more details.
+
+ With [r] comparison is done on the text that matches
+ /{pattern}/ instead of the full line.
+
+ When /{pattern}/ is specified and [r] is not used, the
+ text matched with {pattern} is skipped and comparison
+ is done on what comes after the match.
+ 'ignorecase' applies to the pattern, but 'smartcase'
+ is not used.
+
+ Instead of the slash any non-letter can be used.
+
+ For example, to remove adjacent duplicate lines based
+ on the second comma-separated field: >
+ :uniq r /[^,]*,/
+< Or to keep only unique lines ignoring the first 5
+ characters: >
+ :uniq u /.\{5}/
+< If {pattern} is empty (e.g. // is used), the last
+ search pattern is used.
+
+ With [u] only keep lines that do not repeat (i.e., are
+ not immediately followed by the same line).
+
+ With [!] only keep lines that are immediately followed
+ by a duplicate.
+
+ If both [!] and [u] are given, [u] is ignored and [!]
+ takes effect.
+
+ Note that leading and trailing white space, and lines
+ that are not adjacent, are not considered duplicates.
+ To remove all duplicates regardless of position, use
+ |:sort-u| or external tools.
+
vim:tw=78:ts=8:noet:ft=help:norl:
diff --git a/runtime/doc/index.txt b/runtime/doc/index.txt
index ff27994..11c0efd 100644
--- a/runtime/doc/index.txt
+++ b/runtime/doc/index.txt
@@ -1,4 +1,4 @@
-*index.txt* For Vim version 9.1. Last change: 2025 Jun 02
+*index.txt* For Vim version 9.1. Last change: 2025 Jun 23
VIM REFERENCE MANUAL by Bram Moolenaar
@@ -1740,6 +1740,7 @@
|:unabbreviate| :una[bbreviate] remove abbreviation
|:unhide| :unh[ide] open a window for each loaded file in the
buffer list
+|:uniq| :uni[q] uniq lines
|:unlet| :unl[et] delete variable
|:unlockvar| :unlo[ckvar] unlock variables
|:unmap| :unm[ap] remove mapping
diff --git a/runtime/doc/tags b/runtime/doc/tags
index afa4a05..a8a0804 100644
--- a/runtime/doc/tags
+++ b/runtime/doc/tags
@@ -3331,6 +3331,9 @@
:so repeat.txt /*:so*
:sor change.txt /*:sor*
:sort change.txt /*:sort*
+:sort-l change.txt /*:sort-l*
+:sort-u change.txt /*:sort-u*
+:sort-uniq change.txt /*:sort-uniq*
:source repeat.txt /*:source*
:source! repeat.txt /*:source!*
:source-range repeat.txt /*:source-range*
@@ -3565,6 +3568,8 @@
:undolist undo.txt /*:undolist*
:unh windows.txt /*:unh*
:unhide windows.txt /*:unhide*
+:uni change.txt /*:uni*
+:uniq change.txt /*:uniq*
:unl eval.txt /*:unl*
:unlet eval.txt /*:unlet*
:unlet-$ eval.txt /*:unlet-$*
@@ -6864,6 +6869,7 @@
debugger.txt debugger.txt /*debugger.txt*
dec-mouse options.txt /*dec-mouse*
decada_members ft_ada.txt /*decada_members*
+deduplicating change.txt /*deduplicating*
deepcopy() builtin.txt /*deepcopy()*
default-constructor vim9class.txt /*default-constructor*
defaults.vim starting.txt /*defaults.vim*
@@ -11015,6 +11021,7 @@
undotree() builtin.txt /*undotree()*
unicode mbyte.txt /*unicode*
uniq() builtin.txt /*uniq()*
+unique change.txt /*unique*
unix os_unix.txt /*unix*
unlisted-buffer windows.txt /*unlisted-buffer*
up-down-motions motion.txt /*up-down-motions*
diff --git a/runtime/doc/version9.txt b/runtime/doc/version9.txt
index cd72d73..606ec79 100644
--- a/runtime/doc/version9.txt
+++ b/runtime/doc/version9.txt
@@ -1,4 +1,4 @@
-*version9.txt* For Vim version 9.1. Last change: 2025 Jun 16
+*version9.txt* For Vim version 9.1. Last change: 2025 Jun 23
VIM REFERENCE MANUAL by Bram Moolenaar
@@ -41756,6 +41756,7 @@
|:pbuffer| Edit buffer [N] from the buffer list in the preview
window
|:redrawtabpanel| Force updating the 'tabpanel'.
+|:uniq| Deduplicate text in the current buffer.
Options: ~