blob: f2a7cf16c55cbddb83b4198c3c7a46f16a7e28c4 [file] [log] [blame]
Bram Moolenaar98056532019-12-12 14:18:35 +01001*testing.txt* For Vim version 8.2. Last change: 2019 Sep 08
Bram Moolenaared997ad2019-07-21 16:42:00 +02002
3
4 VIM REFERENCE MANUAL by Bram Moolenaar
5
6
7Testing Vim and Vim script *testing-support*
8
9Expression evaluation is explained in |eval.txt|. This file goes into details
10about writing tests in Vim script. This can be used for testing Vim itself
11and for testing plugins.
12
131. Testing Vim |testing|
Bram Moolenaar54775062019-07-31 21:07:14 +0200142. Test functions |test-functions-details|
153. Assert functions |assert-functions-details|
Bram Moolenaared997ad2019-07-21 16:42:00 +020016
17==============================================================================
181. Testing Vim *testing*
19
20Vim can be tested after building it, usually with "make test".
21The tests are located in the directory "src/testdir".
22
23There are several types of tests added over time:
24 test33.in oldest, don't add any of these
25 test_something.in old style tests
26 test_something.vim new style tests
27
28 *new-style-testing*
29New tests should be added as new style tests. These use functions such as
30|assert_equal()| to keep the test commands and the expected result in one
31place.
32 *old-style-testing*
33In some cases an old style test needs to be used. E.g. when testing Vim
34without the |+eval| feature.
35
36Find more information in the file src/testdir/README.txt.
37
38==============================================================================
Bram Moolenaar54775062019-07-31 21:07:14 +0200392. Test functions *test-functions-details*
Bram Moolenaared997ad2019-07-21 16:42:00 +020040
41test_alloc_fail({id}, {countdown}, {repeat}) *test_alloc_fail()*
42 This is for testing: If the memory allocation with {id} is
43 called, then decrement {countdown}, and when it reaches zero
44 let memory allocation fail {repeat} times. When {repeat} is
45 smaller than one it fails one time.
46
Bram Moolenaarce90e362019-09-08 18:58:44 +020047 Can also be used as a |method|: >
48 GetAllocId()->test_alloc_fail()
Bram Moolenaared997ad2019-07-21 16:42:00 +020049
50test_autochdir() *test_autochdir()*
51 Set a flag to enable the effect of 'autochdir' before Vim
52 startup has finished.
53
54
Bram Moolenaar07ada5f2020-02-05 20:38:22 +010055test_clear_search_pat() *test_clear_search_pat()*
56 Clears the last used search pattern (|/|) and the substitute
57 pattern (|:s|). This is useful for testing conditions where
58 these patterns are not set previously.
59
Bram Moolenaared997ad2019-07-21 16:42:00 +020060test_feedinput({string}) *test_feedinput()*
61 Characters in {string} are queued for processing as if they
62 were typed by the user. This uses a low level input buffer.
63 This function works only when with |+unix| or GUI is running.
64
Bram Moolenaarce90e362019-09-08 18:58:44 +020065 Can also be used as a |method|: >
66 GetText()->test_feedinput()
Bram Moolenaared997ad2019-07-21 16:42:00 +020067
68test_garbagecollect_now() *test_garbagecollect_now()*
69 Like garbagecollect(), but executed right away. This must
70 only be called directly to avoid any structure to exist
71 internally, and |v:testing| must have been set before calling
72 any function.
73
74
75test_garbagecollect_soon() *test_garbagecollect_soon()*
76 Set the flag to call the garbagecollector as if in the main
77 loop. Only to be used in tests.
78
79
80test_getvalue({name}) *test_getvalue()*
81 Get the value of an internal variable. These values for
82 {name} are supported:
83 need_fileinfo
84
Bram Moolenaarce90e362019-09-08 18:58:44 +020085 Can also be used as a |method|: >
86 GetName()->test_getvalue()
Bram Moolenaared997ad2019-07-21 16:42:00 +020087
88test_ignore_error({expr}) *test_ignore_error()*
89 Ignore any error containing {expr}. A normal message is given
90 instead.
91 This is only meant to be used in tests, where catching the
92 error with try/catch cannot be used (because it skips over
93 following code).
94 {expr} is used literally, not as a pattern.
95 When the {expr} is the string "RESET" then the list of ignored
96 errors is made empty.
97
Bram Moolenaarce90e362019-09-08 18:58:44 +020098 Can also be used as a |method|: >
99 GetErrorText()->test_ignore_error()
Bram Moolenaared997ad2019-07-21 16:42:00 +0200100
101test_null_blob() *test_null_blob()*
102 Return a |Blob| that is null. Only useful for testing.
103
104
105test_null_channel() *test_null_channel()*
106 Return a |Channel| that is null. Only useful for testing.
107 {only available when compiled with the +channel feature}
108
109
110test_null_dict() *test_null_dict()*
111 Return a |Dict| that is null. Only useful for testing.
112
113
114test_null_job() *test_null_job()*
115 Return a |Job| that is null. Only useful for testing.
116 {only available when compiled with the +job feature}
117
118
119test_null_list() *test_null_list()*
120 Return a |List| that is null. Only useful for testing.
121
122
123test_null_partial() *test_null_partial()*
124 Return a |Partial| that is null. Only useful for testing.
125
126
127test_null_string() *test_null_string()*
128 Return a |String| that is null. Only useful for testing.
129
130
131test_option_not_set({name}) *test_option_not_set()*
132 Reset the flag that indicates option {name} was set. Thus it
133 looks like it still has the default value. Use like this: >
134 set ambiwidth=double
135 call test_option_not_set('ambiwidth')
136< Now the 'ambiwidth' option behaves like it was never changed,
137 even though the value is "double".
138 Only to be used for testing!
139
Bram Moolenaarce90e362019-09-08 18:58:44 +0200140 Can also be used as a |method|: >
141 GetOptionName()->test_option_not_set()
142
Bram Moolenaared997ad2019-07-21 16:42:00 +0200143
144test_override({name}, {val}) *test_override()*
145 Overrides certain parts of Vim's internal processing to be able
146 to run tests. Only to be used for testing Vim!
147 The override is enabled when {val} is non-zero and removed
148 when {val} is zero.
149 Current supported values for name are:
150
151 name effect when {val} is non-zero ~
152 redraw disable the redrawing() function
153 redraw_flag ignore the RedrawingDisabled flag
154 char_avail disable the char_avail() function
155 starting reset the "starting" variable, see below
156 nfa_fail makes the NFA regexp engine fail to force a
157 fallback to the old engine
158 no_query_mouse do not query the mouse position for "dec"
159 terminals
160 no_wait_return set the "no_wait_return" flag. Not restored
161 with "ALL".
162 ALL clear all overrides ({val} is not used)
163
164 "starting" is to be used when a test should behave like
165 startup was done. Since the tests are run by sourcing a
166 script the "starting" variable is non-zero. This is usually a
167 good thing (tests run faster), but sometimes changes behavior
168 in a way that the test doesn't work properly.
169 When using: >
170 call test_override('starting', 1)
171< The value of "starting" is saved. It is restored by: >
172 call test_override('starting', 0)
173
Bram Moolenaarce90e362019-09-08 18:58:44 +0200174< Can also be used as a |method|: >
175 GetOverrideVal()-> test_override('starting')
Bram Moolenaared997ad2019-07-21 16:42:00 +0200176
177test_refcount({expr}) *test_refcount()*
178 Return the reference count of {expr}. When {expr} is of a
179 type that does not have a reference count, returns -1. Only
180 to be used for testing.
181
Bram Moolenaarce90e362019-09-08 18:58:44 +0200182 Can also be used as a |method|: >
183 GetVarname()->test_refcount()
184
Bram Moolenaared997ad2019-07-21 16:42:00 +0200185
186test_scrollbar({which}, {value}, {dragging}) *test_scrollbar()*
187 Pretend using scrollbar {which} to move it to position
188 {value}. {which} can be:
189 left Left scrollbar of the current window
190 right Right scrollbar of the current window
191 hor Horizontal scrollbar
192
193 For the vertical scrollbars {value} can be 1 to the
194 line-count of the buffer. For the horizontal scrollbar the
195 {value} can be between 1 and the maximum line length, assuming
196 'wrap' is not set.
197
198 When {dragging} is non-zero it's like dragging the scrollbar,
199 otherwise it's like clicking in the scrollbar.
200 Only works when the {which} scrollbar actually exists,
201 obviously only when using the GUI.
202
Bram Moolenaarce90e362019-09-08 18:58:44 +0200203 Can also be used as a |method|: >
204 GetValue()->test_scrollbar('right', 0)
Bram Moolenaared997ad2019-07-21 16:42:00 +0200205
206test_setmouse({row}, {col}) *test_setmouse()*
207 Set the mouse position to be used for the next mouse action.
208 {row} and {col} are one based.
209 For example: >
210 call test_setmouse(4, 20)
211 call feedkeys("\<LeftMouse>", "xt")
212
Bram Moolenaared997ad2019-07-21 16:42:00 +0200213test_settime({expr}) *test_settime()*
214 Set the time Vim uses internally. Currently only used for
215 timestamps in the history, as they are used in viminfo, and
216 for undo.
217 Using a value of 1 makes Vim not sleep after a warning or
218 error message.
219 {expr} must evaluate to a number. When the value is zero the
220 normal behavior is restored.
221
Bram Moolenaarce90e362019-09-08 18:58:44 +0200222 Can also be used as a |method|: >
223 GetTime()->test_settime()
224
Bram Moolenaar4f645c52020-02-08 16:40:39 +0100225test_srand_seed([seed]) *test_srand_seed()*
226 When [seed] is given this sets the seed value used by
227 `srand()`. When omitted the test seed is removed.
228
Bram Moolenaared997ad2019-07-21 16:42:00 +0200229==============================================================================
Bram Moolenaar54775062019-07-31 21:07:14 +02002303. Assert functions *assert-functions-details*
Bram Moolenaared997ad2019-07-21 16:42:00 +0200231
232
233assert_beeps({cmd}) *assert_beeps()*
234 Run {cmd} and add an error message to |v:errors| if it does
235 NOT produce a beep or visual bell.
236 Also see |assert_fails()| and |assert-return|.
237
Bram Moolenaar24278d22019-08-16 21:49:22 +0200238 Can also be used as a |method|: >
239 GetCmd()->assert_beeps()
240<
Bram Moolenaared997ad2019-07-21 16:42:00 +0200241 *assert_equal()*
242assert_equal({expected}, {actual} [, {msg}])
243 When {expected} and {actual} are not equal an error message is
244 added to |v:errors| and 1 is returned. Otherwise zero is
245 returned |assert-return|.
246 There is no automatic conversion, the String "4" is different
247 from the Number 4. And the number 4 is different from the
248 Float 4.0. The value of 'ignorecase' is not used here, case
249 always matters.
250 When {msg} is omitted an error in the form "Expected
251 {expected} but got {actual}" is produced.
252 Example: >
253 assert_equal('foo', 'bar')
254< Will result in a string to be added to |v:errors|:
255 test.vim line 12: Expected 'foo' but got 'bar' ~
256
Bram Moolenaar25e42232019-08-04 15:04:10 +0200257 Can also be used as a |method|: >
258 mylist->assert_equal([1, 2, 3])
259
260
261< *assert_equalfile()*
Bram Moolenaared997ad2019-07-21 16:42:00 +0200262assert_equalfile({fname-one}, {fname-two})
263 When the files {fname-one} and {fname-two} do not contain
264 exactly the same text an error message is added to |v:errors|.
265 Also see |assert-return|.
266 When {fname-one} or {fname-two} does not exist the error will
267 mention that.
268 Mainly useful with |terminal-diff|.
269
Bram Moolenaare49fbff2019-08-21 22:50:07 +0200270 Can also be used as a |method|: >
271 GetLog()->assert_equalfile('expected.log')
272
273
Bram Moolenaared997ad2019-07-21 16:42:00 +0200274assert_exception({error} [, {msg}]) *assert_exception()*
275 When v:exception does not contain the string {error} an error
276 message is added to |v:errors|. Also see |assert-return|.
277 This can be used to assert that a command throws an exception.
278 Using the error number, followed by a colon, avoids problems
279 with translations: >
280 try
281 commandthatfails
282 call assert_false(1, 'command should have failed')
283 catch
284 call assert_exception('E492:')
285 endtry
286
287assert_fails({cmd} [, {error} [, {msg}]]) *assert_fails()*
288 Run {cmd} and add an error message to |v:errors| if it does
289 NOT produce an error. Also see |assert-return|.
290 When {error} is given it must match in |v:errmsg|.
291 Note that beeping is not considered an error, and some failing
292 commands only beep. Use |assert_beeps()| for those.
293
Bram Moolenaar24278d22019-08-16 21:49:22 +0200294 Can also be used as a |method|: >
295 GetCmd()->assert_fails('E99:')
296
Bram Moolenaared997ad2019-07-21 16:42:00 +0200297assert_false({actual} [, {msg}]) *assert_false()*
298 When {actual} is not false an error message is added to
299 |v:errors|, like with |assert_equal()|.
300 Also see |assert-return|.
301 A value is false when it is zero. When {actual} is not a
302 number the assert fails.
303 When {msg} is omitted an error in the form
304 "Expected False but got {actual}" is produced.
305
Bram Moolenaar24278d22019-08-16 21:49:22 +0200306 Can also be used as a |method|: >
307 GetResult()->assert_false()
308
Bram Moolenaared997ad2019-07-21 16:42:00 +0200309assert_inrange({lower}, {upper}, {actual} [, {msg}]) *assert_inrange()*
310 This asserts number and |Float| values. When {actual} is lower
311 than {lower} or higher than {upper} an error message is added
312 to |v:errors|. Also see |assert-return|.
313 When {msg} is omitted an error in the form
314 "Expected range {lower} - {upper}, but got {actual}" is
315 produced.
316
317 *assert_match()*
318assert_match({pattern}, {actual} [, {msg}])
319 When {pattern} does not match {actual} an error message is
320 added to |v:errors|. Also see |assert-return|.
321
322 {pattern} is used as with |=~|: The matching is always done
323 like 'magic' was set and 'cpoptions' is empty, no matter what
324 the actual value of 'magic' or 'cpoptions' is.
325
326 {actual} is used as a string, automatic conversion applies.
327 Use "^" and "$" to match with the start and end of the text.
328 Use both to match the whole text.
329
330 When {msg} is omitted an error in the form
331 "Pattern {pattern} does not match {actual}" is produced.
332 Example: >
333 assert_match('^f.*o$', 'foobar')
334< Will result in a string to be added to |v:errors|:
335 test.vim line 12: Pattern '^f.*o$' does not match 'foobar' ~
336
Bram Moolenaar24278d22019-08-16 21:49:22 +0200337 Can also be used as a |method|: >
338 getFile()->assert_match('foo.*')
339<
Bram Moolenaared997ad2019-07-21 16:42:00 +0200340 *assert_notequal()*
341assert_notequal({expected}, {actual} [, {msg}])
342 The opposite of `assert_equal()`: add an error message to
343 |v:errors| when {expected} and {actual} are equal.
344 Also see |assert-return|.
345
Bram Moolenaar25e42232019-08-04 15:04:10 +0200346 Can also be used as a |method|: >
347 mylist->assert_notequal([1, 2, 3])
348
349< *assert_notmatch()*
Bram Moolenaared997ad2019-07-21 16:42:00 +0200350assert_notmatch({pattern}, {actual} [, {msg}])
351 The opposite of `assert_match()`: add an error message to
352 |v:errors| when {pattern} matches {actual}.
353 Also see |assert-return|.
354
Bram Moolenaar24278d22019-08-16 21:49:22 +0200355 Can also be used as a |method|: >
356 getFile()->assert_notmatch('bar.*')
357
Bram Moolenaare49fbff2019-08-21 22:50:07 +0200358
Bram Moolenaared997ad2019-07-21 16:42:00 +0200359assert_report({msg}) *assert_report()*
360 Report a test failure directly, using {msg}.
361 Always returns one.
362
Bram Moolenaare49fbff2019-08-21 22:50:07 +0200363 Can also be used as a |method|: >
364 GetMessage()->assert_report()
365
366
Bram Moolenaared997ad2019-07-21 16:42:00 +0200367assert_true({actual} [, {msg}]) *assert_true()*
368 When {actual} is not true an error message is added to
369 |v:errors|, like with |assert_equal()|.
370 Also see |assert-return|.
371 A value is TRUE when it is a non-zero number. When {actual}
372 is not a number the assert fails.
373 When {msg} is omitted an error in the form "Expected True but
374 got {actual}" is produced.
375
Bram Moolenaar24278d22019-08-16 21:49:22 +0200376 Can also be used as a |method|: >
377 GetResult()->assert_true()
378<
Bram Moolenaared997ad2019-07-21 16:42:00 +0200379
380 vim:tw=78:ts=8:noet:ft=help:norl: