1
0
forked from aniani/vim

3 Commits

Author SHA1 Message Date
zeertzjq
3e5bbb8e11
patch 9.1.0807: tests: having 'nolist' in modelines isn't always desired
Problem:  tests: having 'nolist' in modelines isn't always desired
Solution: remove 'nolist' from modeline (zeertzjq)

closes: #15910

Signed-off-by: zeertzjq <zeertzjq@outlook.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
2024-10-22 23:11:27 +02:00
Aliaksei Budavei
6bff6a2fa4
patch 9.1.0681: tests: Analyzing failed screendumps is hard
Problem:  tests: Analyzing failed screendumps is hard
Solution: Facilitate the viewing of rendered screendumps under src/
          add some documentation on how to use the viewdumps.vim
          script (Aliaksei Budavei)

With the submitted "viewdumps.vim" script, a few manual
steps in typical workflows (see below) can be automated.
The updated "README.txt" contains additional information.

============================================================

Reviewing LOCAL failed screendump tests can be arranged as
follows:

1) Run tests and generate screendumps:
------------------------------------------------------------
cd /path/to/fork/src/testdir
make
------------------------------------------------------------

2) Examine the screendumps from the "failed" directory:
------------------------------------------------------------
../vim -u NONE -S viewdumps.vim
------------------------------------------------------------

============================================================

Reviewing UPLOADED failed screendump tests can be arranged
as follows (it can be further locally scripted):

1) Fetch an artifact with failed screendumps from
"github.com/vim/vim/actions/runs/A_ID/artifacts/B_ID".

2) Extract the archived files:
------------------------------------------------------------
unzip /tmp/failed-tests.zip -d /tmp
------------------------------------------------------------

3) Set up the "dumps" directory.  Create a symlink to
"/path/to/fork/dirs/dumps" in the extracted directories so
that term_dumpdiff() can be used.  (The lookup algorithm
resolves "dumps" for every loaded filename.  So, with
"/tmp/src/testdir/failed/*.dump" files passed as script
arguments, the algorithm will make the files in
"/tmp/src/testdir/dumps" queried.)
------------------------------------------------------------
cd /path/to/fork
ln -s $(pwd)/src/testdir/dumps /tmp/src/testdir/dumps
------------------------------------------------------------

4) Examine the extracted screendumps:
------------------------------------------------------------
./src/vim -u NONE -S src/testdir/viewdumps.vim \
  /tmp/src/testdir/failed/*.dump
------------------------------------------------------------

5) Clean up:
------------------------------------------------------------
unlink /tmp/src/testdir/dumps
rm -rf /tmp/src
------------------------------------------------------------

============================================================

Reviewing SUBMITTED FOR PULL REQUEST screendump tests can be
arranged as follows (it can be further locally scripted):

1) List the fetched changeset and write the changed "dumps"
filenames to "/tmp/filelist":
------------------------------------------------------------
cd /path/to/fork
git switch prs/1234
git diff-index --relative=src/testdir/dumps/ \
  --name-only prs/1234~1 > /tmp/filelist
------------------------------------------------------------

2) Reconcile relative filepaths, and copy next-to-be-updated
"dumps" files in the "failed" directory (note the missing
new screendumps, if any):
------------------------------------------------------------
git switch master
cd src/testdir/dumps
test -d ../failed || mkdir ../failed
cp -t ../failed $(cat /tmp/filelist)
------------------------------------------------------------

3) Remember about the introduced INVERTED relation between
"dumps" and "failed", i.e. the files to be committed are in
"dumps" already and their previous versions are in "failed";
therefore, copy the missing new screendumps from "dumps" to
"failed" (otherwise these won't be shown):
------------------------------------------------------------
git switch prs/1234
cp -t ../failed foo_10.dump foo_11.dump foo_12.dump
------------------------------------------------------------

4) Examine the screendumps from the "failed" directory (new
screendumps will be shown with no difference between their
versions):
------------------------------------------------------------
cd ..
../vim -u NONE -S viewdumps.vim
------------------------------------------------------------

closes: #15515

Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
2024-08-19 21:33:26 +02:00
Aliaksei Budavei
d33afe12c6
runtime(syntax-tests): Facilitate the viewing of rendered screendumps
With the submitted "viewdumps.vim" script, a few manual
steps in typical workflows (see below) can be automated.
The updated "README.txt" contains additional information.

============================================================

Reviewing LOCAL failed syntax tests can be arranged as
follows:

1) Run tests and generate screendumps:
------------------------------------------------------------
cd /path/to/fork/runtime/syntax
make clean test
------------------------------------------------------------

2) Examine the screendumps from the "failed" directory:
------------------------------------------------------------
../../src/vim --clean -S testdir/viewdumps.vim
------------------------------------------------------------

============================================================

Reviewing UPLOADED failed syntax tests can be arranged as
follows (it can be further locally scripted):

1) Fetch an artifact with failed screendumps from
"github.com/vim/vim/actions/runs/A_ID/artifacts/B_ID".

2) Extract the archived files:
------------------------------------------------------------
unzip /tmp/failed-tests.zip -d /tmp
------------------------------------------------------------

3) Set up the "dumps" directory.  Create a symlink to
"/path/to/fork/dirs/dumps" in the extracted directories so
that term_dumpdiff() can be used.  (The lookup algorithm
resolves "dumps" for every loaded filename.  So, with
"/tmp/runtime/syntax/testdir/failed/*.dump" files passed
as script arguments, the algorithm will make the files in
"/tmp/runtime/syntax/testdir/dumps" queried.)
------------------------------------------------------------
cd /path/to/fork
ln -s $(pwd)/runtime/syntax/testdir/dumps \
  /tmp/runtime/syntax/testdir/dumps
------------------------------------------------------------

4) Examine the extracted screendumps:
------------------------------------------------------------
./src/vim --clean -S runtime/syntax/testdir/viewdumps.vim \
  /tmp/runtime/syntax/testdir/failed/*.dump
------------------------------------------------------------

5) Clean up:
------------------------------------------------------------
unlink /tmp/runtime/syntax/testdir/dumps
rm -rf /tmp/runtime
------------------------------------------------------------

============================================================

Reviewing SUBMITTED FOR PULL REQUEST syntax tests can be
arranged as follows (it can be further locally scripted):

1) List the fetched changeset and write the changed "dumps"
filenames to "/tmp/filelist":
------------------------------------------------------------
cd /path/to/fork
git switch prs/1234
git diff-index --relative=runtime/syntax/testdir/dumps/ \
  --name-only prs/1234~1 > /tmp/filelist
------------------------------------------------------------

2) Reconcile relative filepaths, and copy next-to-be-updated
"dumps" files in the "failed" directory (note the missing
new screendumps, if any):
------------------------------------------------------------
git switch master
cd runtime/syntax/testdir/dumps
cp -t ../failed $(cat /tmp/filelist)
------------------------------------------------------------

3) Remember about the introduced INVERTED relation between
"dumps" and "failed", i.e. the files to be committed are in
"dumps" already and their previous versions are in "failed";
therefore, copy the missing new screendumps from "dumps" to
"failed" (otherwise these won't be shown):
------------------------------------------------------------
git switch prs/1234
cp -t ../failed foo_10.dump foo_11.dump foo_12.dump
------------------------------------------------------------

4) Examine the screendumps from the "failed" directory (new
screendumps will be shown with no difference between their
versions):
------------------------------------------------------------
cd ..
../../../src/vim --clean -S viewdumps.vim
------------------------------------------------------------

closes: #15476

Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
2024-08-12 18:43:06 +02:00