The process of preparing and submitting syntax tests is
fraught with challenges that can turn away many aspiring
contributors from ever attempting it. (Out of 69 languages
introduced since v9.0.1627, there are only syntax tests for
Tera.)
After v9.1.1176~1, one visual clue for admitting syntax test
failures previously available with e.g. "git status" is gone
after all files under "failed/" have been made ignored for
Git and Mercurial. There isn't a single way to go about it:
some people may move files from "failed/" to "dumps/" after
each iteration; some people may only move "good" iteration
files; when a test file is refactored to a great extent,
some people may prefer deleting all test-related files under
"dumps/" before moving files from "failed/". The usability
of reporting, at any time, that there are some _untracked_
files under "failed/" cannot be overstated. Without it, the
chances are greater for pushing mismatched changesets. And
when tests fail then everyone but the author will be kept in
the dark about the cause: were some updated screendumps not
committed _or_ was a wrong version of the syntax plugin
committed?
Another file, "testdir/Xfilter" (v9.1.0763), that will be
created to establish communication from Make to Vim about
what subset of syntax tests is requested for running, should
also be not ignored but rather deleted once its contents are
read. Unless it is explicitly deleted _after test failure_,
the file may contain new *and* old test names when another
testing attempt is under way. And by virtue of it being
ignored, the reason for also running not requested tests
will be as ever puzzling.
Both Git and Mercurial support per-user configuration; such
wide-reaching settings hardly belong to clonable defaults.
Also, match literal dots in testname filters.
Also, discover and report _some_ disused screendump files
tracked under "dumps/".
References:
- https://git-scm.com/docs/gitignore
- https://www.mercurial-scm.org/help/topics/config#uicloses: #16917
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
As of patch v9.1.1176~1, there are no longer makeshift Make
targets to accommodate language names that can match phoney
targets. For example, "clean_" was previously generated for
Clean because otherwise it clashed with the "clean" target.
Additionally, enable test filtering for makeshift targets.
Reference:
https://wiki.clean.cs.ru.nl/Cleancloses: #16810
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
When a syntax file is changed, timestamps of the corresponding files are
updated.
NOTE: At the moment this script does not strictly track dependency, like
cpp on c.
Also update ignore files
closes#16548
Co-authored-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
The current lightweight synchronisation with ":redraw" needs further
reinforcement in the light of v9.1.1110. And, with v9.1.0820, make
another synchronisation point _before_ the first (or only) screenful is
dumped.
Also add a script to regenerate all screendumps.
closes: #16632
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Currently syntax tests outputs all the Vim commands to the console,
which is annoying for a local developer but also makes the CI output
impossible to parse and just printing all the terminal output comes with
a performance penalty.
So let's just simply redirect all output to /dev/null similar to what
the script tests do.
This can be turned off locally for someone diagnosing issues but it
should not be the default behavior to output all terminal control
sequences to output in CI.
closes: #16612
Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: tests: cannot run single syntax tests
Solution: Support running a subset of syntax tests
(Aliaksei Budavei)
Two methods of assembling a subset of test filenames for
selection are provided:
* Filename and filetype Make targets will be generated, and
multiple such targets can be passed before the mandated
trailing "test" target, e.g. "make html markdown test".
* Filenames and their parts can be specified as a regular
expression that is assigned to a "VIM_SYNTAX_TEST_FILTER"
environment variable, and used with the test Make target,
e.g. "VIM_SYNTAX_TEST_FILTER=html\\\|markdown make test".
(This variable will be ignored and the whole suite will be
run when Make is GNU Make and a parent Makefile is used.)
Methods can be used alone or together, with the Make targets
having the higher precedence. Neither method will influence
the order of test execution.
closes: #15670
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
The whole testdir/runtest.vim is a bit of a mess. So this is an attempt
to improving the current state. First of all, let's move most of the
code into a single RunTest() function so that it's easier to run this
manually.
Then, let's add the possibility to customize filetype specific tests by
applying filetype specific options. We will make use of it in the next
commit, where the provided java file uses tabs, which cause line
wrapping which will then cause the scrolling to miss some lines when
redrawing (see #14105)
related: #14105
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Syntax test fails when run with non C locale
Solution: Run syntax tests with C locale, clean up Xtestscript file,
strip environment variables from GetVimCommand()
(h-east)
closes: #14007
Co-authored-by: h-east <h.east.727@gmail.com>
Signed-off-by: h-east <h.east.727@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: runtime tests fail with tiny vim
Solution: check for tiny vim, run runtime tests in CI
even for tiny version
closes: #13169closes: #13170
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Dominique Pellé <dominique.pelle@tomtom.com>
Problem: Insufficient testing for syntax plugins.
Solution: Add shell file examples. (Charles Campbell) Create a messages
file for easier debugging and reporting the test results.
