Track the cursor forward progress through a syntax test file
so that any degenerate input can be checked and reported.
closes: #15150
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Write a batch of test files with made-up syntax to serve for
additional linewise checks to be manually performed whenever
the algorithm for screen dump file generation is modified.
Define a shell variable VIM_SYNTAX_SELF_TESTING to run these
tests:
cd runtime/syntax/
VIM_SYNTAX_SELF_TESTING=1 make clean test
related: #15150
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
The current contents of sh_01_99.dump and sh_09_99.dump
begin with ":redraw" in the ruler line whereas both the
comment and the need for the redraw command execution imply
blanks " ".
related: #15150
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
The current implementation falls short for syntax test files
on two accounts:
1. With folded lines -- some lines before folded lines are
unnecessarily repeated in generated dump files because
closed folded lines are always treated as opened for the
cursor to move _in_ instead of to move _over_ them.
2. With wrapped lines (longer than 75 columns) -- some lines
are omitted in generated dump files because calculations
for the cursor progress and its movement commands only
refer to file lines and not their layout within a 20x75
buffer (less &cmdheight).
As an alternative, we abandon deterministic (and inaccurate
at times) calculations for the cursor progress and, instead,
advance the cursor by as much as before for a single dump
file, but now rely on marking the last visible line and
additional movement to position lines at desired offsets,
carefully preserving compatibility for the &scrolloff and
&ruler values inherited from defaults.vim. The parent Vim
process will keep track of progress through a syntax test
file made by its child process ("terminal") by reading the
rightmost end of the ruler line from the terminal buffer,
looking for " All " or " Bot " for its cue to finish dump
file generation.
With these changes applied, the lossless line length limit
will be raised from 75 to 1425 (for a 19x75 view) columns.
Also, prefer "lastline" to "truncate" for &display; hiding
the content of any last _long_ line in a view goes against
the purpose of syntax file testing -- all lines should be
recorded.
related: #15150fixes: #14245
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
With the variables defined, distinctly highlight parts of
a method declaration header: its name and parameter list
parens, from its type parameters, return type, and formal
parameters; and distinctly highlight parts of a lambda
expression: its parameter list parens and the arrow, from
its formal parameters and identifiers.
closes: #15083
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Match Vim9-script comments after :def and :enddef and legacy-script
comments after :func and :endfunc, in any definition context.
Highlight incorrect comment types after these commands as errors.
fixes: #15062closes: #15072
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
In the presence of parameterised types whose names begin
with a capital letter and end with a less-than sign "<" that
introduces a type argument or a list of comma-separated type
arguments, followed by a greater-than sign ">", a variable
"g:java_highlight_generics" can be defined to have some
components of such types uniformly coloured (by picking
highlight groups for javaGenericsC{1,2}, javaWildcardBound).
For example,
------------------------------------------------------------
java.io.InputStream stream = java.io.InputStream.nullInputStream();
java.util.function.Function<String,
java.util.function.BiFunction<String, String, String>> updater =
property -> (oldValue, newValue) -> oldValue;
java.util.logging.LogManager.getLogManager()
.updateConfiguration(stream, updater);
------------------------------------------------------------
Note that the diamond form and explicit type arguments do
not qualify for this kind of recognition.
For example,
------------------------------------------------------------
new java.util.HashSet<>().<String>toArray(new String[0]);
------------------------------------------------------------
References:
https://docs.oracle.com/javase/specs/jls/se21/html/jls-4.html#jls-4.5https://docs.oracle.com/javase/specs/jls/se21/html/jls-15.html#jls-15.9https://docs.oracle.com/javase/specs/jls/se21/html/jls-15.html#jls-15.12.2.1closes: #15050
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Instances of anonymous classes can be passed as method
arguments and should be subject to line folding as well.
closes: #15048
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Also:
- Restore the capability to mark as an error braces nested
in parens with g:javaInParen.
- Try not to fold top-level-type bodies. (Defining multiple
package-private top level types in a single source file is
not recommended as it can impose order among compilation
units; so it is assumed that only one such top level type
is usually defined.)
- Compose ‘method header’ highlighting and block braces
folding.
- Do not highlight block braces whenever ‘method header’
highlighting is requested.
This bundling of ‘method headers’ and block braces for
highlighting can be traced back to Vim v5.0; however, no
comment or documentation entry conveys any justification.
For example, it is hard to discover the connection between
block braces for "while", "if", etc., statements and method
body block braces. The former behaviour can be attained in,
e.g. ~/.vim/after/syntax/java.vim:
------------------------------------------------------------
if exists("g:java_highlight_functions")
syn clear javaBlock javaInParen
syn match javaBlockOther "[{}]"
syn region javaBlock transparent matchgroup=javaBlockStart
\ start="\%(^\|^\S[^:]\+\)\@120<!{" end="}" fold
hi def link javaBlockStart javaFuncDef
hi def link javaBlockOther javaBlockStart
if exists("g:java_mark_braces_in_parens_as_errors")
syn match javaInParen contained "[{}]"
endif
endif
------------------------------------------------------------
Note: Read ‘a method header omitting a _throws_ clause’ for
every ‘method header’ appellation used above.
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Match shebang lines in Vim9 and legacy script.
Mark these as an error if they appear anywhere other than the first line
of a legacy-script file. In Vim9 script these match as normal line
comments rather than an error.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Rendered element content results in test failures on macOS and FreeBSD.
The included content for these elements was incidental to the primary
intent of the test to check element tag highlighting so set
g:html_no_rendering to disable content rendering.
FreeBSD fails for even an empty <strike> element.
See #13591 and #14215.
closes: #13595
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Match :sleep arg properly including a lone "m" with a leading count.
closes: #15003
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Make a formal definition for normal and single-element kinds
of annotations that otherwise require for their containment
to repeat each time all syntax groups that describe element
values.
Reference:
https://docs.oracle.com/javase/specs/jls/se21/html/jls-9.html#jls-9.7
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Leading whitespace is no longer matched as part of the line comment as
of #13936.
closes: #14971
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Allow highlighting of strings within comments to be disabled by setting
g:vimsyn_comment_strings to false.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Match multiline (continued) line comments.
Continued tail comments are not supported yet.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
These are no longer generated by the current test input files.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Allow whitespace between the :substitute command and its pattern
argument. Although unusual, it is supported and there are examples in
the wild.
Match Vi compatible :substitute commands like :s\/{string}/. See :help
E1270.
fixes: #14920closes: #14923
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
* runtime(java): Stop handpicking syntax groups for @javaTop
Also:
- Remove the obsolete comment for g:java_allow_cpp_keywords.
- Remove the commented out groups java\%[Debug\]StringError.
- Infer and set the preferred formatting Vim options from
the modeline.
Since vim-6-0u, non-contained syntax groups can be referred
to by using the "contains=TOP..." argument.
* Set &encoding and &termencoding to "utf-8" for test files
* Limit non-ASCII charset to [§ƒɐɘʬʭΑ-Τα-μ] for test files
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
There is a flaw in the current implementation that has been
exacerbated around v5.2. It lies in the recognition of all
three indentation styles simultaneously: a tab, two space,
and eight space character(s). With it, it is not uncommon
to misidentify various constructs as method declarations
when they belong to two-space indented members and other
blocks of a type and are offset at eight space characters or
a tab from the start of the line.
For example,
------------------------------------------------------------
class Test
{
static String hello() { return "hello"; }
public static void main(String[] args)
{
try {
if (args.length > 0) {
// FIXME: eight spaces.
System.out.println(args[0]);
} else {
// FIXME: a tab.
System.out.println(hello());
}
} catch (Exception e) {
throw new Error(e);
}
}
}
------------------------------------------------------------
------------------------------------------------------------
:let g:java_highlight_functions = 'indent'
:doautocmd Syntax
------------------------------------------------------------
A better approach is to pick an only indentation style out
of all supported styles (so either two spaces _or_ eight
spaces _or_ a tab). Note that tabs and spaces can still be
mixed, only the leading tab or the leading run of spaces
matters for the recognition. And there is no reason to not
complement the set of valid styles with any number of spaces
from 1 to 8, inclusively.
Please proceed with the necessary change as follows:
- rename from "indent" to "indent2" for a 2-space run;
- rename from "indent" to "indent8" for an 8-space run;
- continue to have "indent" for a tab run;
- define an "indent" variable with a suffix number denoting
the preferred amount of indentation for any other run of
spaces [1-8].
As before, this alternative style of recognition of method
declarations still does not prescribe naming conventions and
still cannot recognise method declarations in nested types
that are conventionally indented.
The proposed changes also follow suit of "style" in stopping
the claiming of constructor and enum constant declarations.
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
- Request the new regexp engine (v7.3.970) for [:upper:] and
[:lower:].
- Recognise declarations of in-line annotated methods.
- Recognise declarations of _strictfp_ methods.
- Establish partial order for method modifiers as shown in
the MethodModifier production; namely, _public_ and
friends should be written the leftmost, possibly followed
by _abstract_ or _default_, or possibly followed by other
modifiers.
- Stop looking for parameterisable primitive types (void<?>,
int<Object>, etc., are malformed).
- Stop looking for arrays of _void_.
- Acknowledge the prevailing convention for method names to
begin with a small letter and for class/interface names to
begin with a capital letter; and, therefore, desist from
claiming declarations of enum constants and constructors
with javaFuncDef.
Rationale:
+ Constructor is distinct from method:
* its (overloaded) name is not arbitrary;
* its return type is implicit;
* its _throws_ clause depends on indirect vagaries of
instance (variable) initialisers;
* its invocation makes other constructors of its type
hierarchy invoked one by one, concluding with the
primordial constructor;
* its explicit invocation, via _this_ or _super_, can
only appear as the first statement in a constructor
(not anymore, see JEP 447); else, its _super_ call
cannot appear in constructors of _record_ or _enum_;
and neither invocation is allowed for the primordial
constructor;
* it is not a member of its class, like initialisers,
and is never inherited;
* it is never _abstract_ or _native_.
+ Constructor declarations tend to be few in number and
merit visual recognition from method declarations.
+ Enum constants define a fixed set of type instances
and more resemble class variable initialisers.
Note that the code duplicated for @javaFuncParams is written
keeping in mind for g:java_highlight_functions a pending 3rd
variant, which would require none of the :syn-cluster added
groups.
closes: #14620
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Only match function folding start and end patterns at the start of a
line, excluding heredocs and :append/:change/:insert commands.
Fixes#14393
Signed-off-by: Christian Brabandt <cb@256bit.org>
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Allow for syntax-based folding of Vim9 script legacy header regions.
This is enabled with the "H" flag of the g:vimsyn_folding config variable.
closes: #14530
Signed-off-by: Christian Brabandt <cb@256bit.org>
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
This is a first-pass attempt to limit matching of Vim9 and legacy-script
comments to the appropriate syntactic contexts.
Vim9-script comments are highlighted at top level in a Vim9-script file,
in all :def functions, and in all :autocmd and :commmand command blocks.
Legacy-script comments are highlighted at top level in a legacy script
file, in all :func functions and in the Vim9-script preamble before the
:vim9script marker command.
Fixes#13047, #11307 and #9587.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Remove curly-brace name matching for :def functions. This is not
supported in Vim9 script.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
- Recognise a _record_ contextual keyword.
- Recognise _non-sealed_, _sealed_, and _permits_ contextual
keywords.
- Admit _$_ to keyword characters.
- Group _abstract_, _final_, _default_, _(non-)sealed_
(apart from _(non-)sealed_, the incompossibility of these
modifiers calls for attention).
- Remove another _synchronized_ keyword redefinition.
I have also replaced a function with an expression. Before
patch 8.1.0515, it should have been declared :function! to
work with repeatable script sourcing; there is less to worry
about with an expression.
References:
https://openjdk.org/jeps/395 (Records)
https://openjdk.org/jeps/409 (Sealed Classes)
https://docs.oracle.com/javase/specs/jls/se21/html/jls-3.html#jls-3.8closes: #14403
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
- Recognize block scalar style to avoid unexpected highlighting by `yamlFlowString` (fix#11517)
- Improve performance of `yamlFlowMappingKey` by allowing execution by the NFA engine (fix#10730)
- It was intentionally disabled before patterns were optimized by `s:SimplifyToAssumeAllPrintable`.
- Fix detection of flow style mapping indicators (fix#8234).
- Enable highlighting of explicit mapping value indicators and node properties in flow style.
- Add syntax highlighting tests
closes: #14354
Signed-off-by: itchyny <itchyny@cybozu.co.jp>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Currently, the overriding object method definitions are
matched as vimFunctionError (:help builtin-object-methods,
v9.1.0148).
For example:
------------------------------------------------------------
vim9script
class Test
def string(): string
return "Test"
enddef
endclass
echo string(Test.new()) == Test.new().string()
------------------------------------------------------------
Instead, let's introduce a new syntax group vimMethodName
and make these methods its members. In order to emphasise
the link between the overriding methods and the overridden
functions for highlighting, vimMethodName is linked by
default to vimFuncName.
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
- Match empty blob literals.
- Match modifier arguments to :abclear commands.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Also:
- Refine comment matching (javaComment{Error\ and,Start}).
- Continue rewriting regexps (prefer atom grouping with
non-capturing parens; factor out common prefixes in
alternations).
- Allow for relative paths with the _file_ attribute of
{@snippet}.
- Anticipate HTML in the @see tags.
- Match the nullary method parens in javaDocSeeTagParam.
- Improve the boundary patterns for summary sentences of
documentation.
> This sentence ends at ... or at the first tag (as defined
> below).
There are Java documentation tags (@) and there are HTML
tags (<?>) (with Markdown looming large; see JEP 467). With
block tags, e.g. @param, @return, @see, we begin another
documentation "sentence" whether or not the author has
terminated the summary sentence with a period; with
.<!-- -->, we may follow abbreviations, enumerations,
initials, (but instead consider @literal or ) _within_
the summary sentence. On the other hand, inline tags, e.g.
@code, @link, @literal, should not terminate the summary
sentence.
References:
https://bugs.openjdk.org/browse/JDK-8075778https://www.oracle.com/technical-resources/articles/java/javadoc-tool.html#firstsentencehttps://docs.oracle.com/en/java/javase/21/docs/specs/javadoc/doc-comment-spec.html
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Remember that ‘code fragments are typically Java source
code, but they may also be fragments of properties files,
source code in other languages, or plain text.’ Therefore,
with these changes, markup tags are highlighted in the Java
source files (as external snippets) and in the {@snippet}
tags.
Also:
- Improve matching of the multi-line {@code} documentation
tag with any contained balanced braces.
- Recognise the {@literal} documentation tag.
- Highlight stray blanks in comments.
Related to an enhancement proposal for PCRE-like callouts
discussed at https://github.com/vim/vim/issues/11217.
References:
https://openjdk.org/jeps/413https://docs.oracle.com/en/java/javase/21/docs/specs/javadoc/doc-comment-spec.html
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Improve function definition highlighting.
- Match bang and function modifiers - abort etc.
- Only match valid scope modifiers.
- Match listing commands.
- Don't match ex commands in function names.
- Split function syntax groups into :func and :def subgroups.
- Match Vim9-script parameter and return types.
- Limit legacy-script and Vim9-script comments to :func and :def
definitions, respectively.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Disallow '.' at the start of a menu item name.
This is the menu path separator character and should be escaped with a
'\' in this case.
Partially fixes#14230. "popup" is still incorrectly matched as the Ex
command.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Also:
- Include a syntax test for Markdown that takes advantage of
a sourceable setup configuration.
- Update Filelist
closes: #14215
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Not all programming languages support comments; without such
support, the TEST_SETUP functionality that offers embeddable
Vim Ex commands may no longer be applicable.
We can achieve similar functionality by storing Ex commands
in a Vim file with its basename matching the basename of the
test file, and having this Vim file sourced, and then having
the test file loaded and tested. When such a Vim file would
be used for a language that has comments and whose matching
test file has embedded TEST_SETUP lines, we will accommodate
it by letting the TEST_SETUP lines augment and/or overrule
sourced configuration.
Further details can be found in the discussion thread at
https://github.com/vim/vim/discussions/14117.
related: #14215
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Currently, the very same syntax file for which a test can
be written is the only place where global variables can be
defined so that the file parts guarded with such variables
can be read during screen dump generation. This approach
would lead to littering the syntax file with test-related
queries.
Instead, we could borrow the idea of comment-based mechanism
for test setup from the indent test runner. With it, the
first 20 lines of each test file would be ALWAYS scanned in
search of the TEST_SETUP markers and, when found, the part
between the end of the marker and the end of the line would
be treated as a Vim Ex command.
Note that with these changes, runtime/defaults.vim is no
longer sourced for screen dump generation; however, some of
its functionality is reintroduced.
Further details can be found in the discussion thread at
https://github.com/vim/vim/discussions/14117.
related: #14215
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Improve :highlight command highlighting
- Use the same highlight groups for "default link" with and without
bang.
- Match some common line-continuation use.
- Match :hi clear variants.
- Highlight color-name values.
Resync vim.vim and generator/vim.vim.base.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Fix highlighting of ":echo (expr)" (broken in commit 61887b3) by
re-enabling the original fix for #9987.
Addresses https://github.com/vim/vim/pull/14199#issuecomment-1999732062.
This will be fixed more generally when there is context-sensitive
matching for commands and functions.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Improve :echo and :execute highlighting.
- Add better line-continuation support for both commands.
- Improve the :execute command's expression argument matching.
- Remove the fix for issue #9987 as this is now handled by correctly
matching the parens in :echo (...) as operator parens.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
- Limit blob literals to an even number of hexadecimal digits and
correctly located dots.
- Match octal numbers.
The current version unsuccessfully attempts to match a leading '-' as
part of the float literal. It's actually parsed as part of the literal
for all numbers but the syntax file hasn't matched it like that for a
long time and highlights negative numbers as UNARY-MINUS NUMBER. This
will be fixed when better expression matching is implemented.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
As this is encouraged in the referenced JEPs, "to visually
distinguish a string template from a string literal, and
a text block template from a text block", the default
colours for java\%[Debug]StrTempl are made distinct from
java\%[Debug]String.
According to §3.2 Lexical Translations (JLS, c. 1996 or any
more recent version), line terminators, white space, and
comments are discarded before tokens are accepted. Since
a template expression comprises a template processor, a dot,
and a template, it may be visually appealing to break up
its head across a few lines whenever its tail already spans
multiple lines. Curiously, no allowance for it is made in
the distributed tests for OpenJDK 21; the proposed regexp
patterns take in consideration a line terminator and white
space after a dot.
References:
https://openjdk.org/jeps/430 (Preview)
https://openjdk.org/jeps/459 (Second Preview)
https://openjdk.org/jeps/465
Signed-off-by: Aliaksei Budavei <0x000c70@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Improve :menu and :menutranslate highlighting.
- Match args to :menutranslation and :popup.
- Only highlight special notation in {rhs} of :menu, like :map.
- Allow line continuations in {rhs} of :menu and between {english} and
{mylang} of :menutranslation, matching common usage.
- Bug fixes.
Signed-off-by: Doug Kearns <dougkearns@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
