Problem: Memory leak in Configure Script when checking GTK
Solution: Free the allocated memory
If the memory is not freed, GTK GUI VIM cannot be build with address
sanitizer for debugging purposes - configure script will report missing
GTK, because the testing file compilation fails due reported memory
leak.
closes: #13672
Signed-off-by: Zdenek Dohnal <zdohnal@redhat.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Perl: xsubpp may be in non-standard location
Solution: Add --with-subpp configure option
configure.ac: Add --with-xsubpp configure option
Some environments (such as flatpaks) cannot count on xsubpp being
in the common Perl directory, so a configure option should be used for
clean solution.
closes: #13470
Signed-off-by: Zdenek Dohnal <zdohnal@redhat.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: configure prints stray 6 when checking libruby
Solution: redirect stdout to dev/null
configure: Do not print "6" when checking for libruby
`expr` will print the matched string length to the standard output.
Current `configure` output looks like this:
```
checking Ruby header files... /usr/include/ruby-3.1.0
6
```
The script really only cares about `expr` exit code.
closes: #13234
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Illia Bobyr <illia.bobyr@gmail.com>
Problem: wrong auto/configure script
Solution: regenerate with autoconf 2.71
configure: sys/xattr.hs: Regenerate with autoconf 2.71
It seems that `auto/configure` update in
commit 6de4e58cf2 (tag: v9.0.1963)
Author: zeertzjq <zeertzjq@outlook.com>
Date: Sat Sep 30 14:19:14 2023 +0200
patch 9.0.1963: Configure script may not detect xattr
Problem: Configure script may not detect xattr correctly
Solution: include sys/xattr instead of attr/xattr,
make Test_write_with_xattr_support() test
xattr feature correctly
This also applies to the Smack security feature, so change the include
and configure script for it as well.
closes: #13229
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
was done manually, and missed an update to the generated variable name.
closes: #13235
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Illia Bobyr <illia.bobyr@gmail.com>
Problem: Configure script may not detect xattr correctly
Solution: include sys/xattr instead of attr/xattr,
make Test_write_with_xattr_support() test
xattr feature correctly
This also applies to the Smack security feature, so change the include
and configure script for it as well.
closes: #13229
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
Problem: No support for writing extended attributes
Solution: Add extended attribute support for linux
It's been a long standing issue, that if you write a file with extended
attributes and backupcopy is set to no, the file will loose the extended
attributes.
So this patch adds support for retrieving the extended attributes and
copying it to the new file. It currently only works on linux, mainly
because I don't know the different APIs for other systems (BSD, MacOSX and
Solaris). On linux, this should be supported since Kernel 2.4 or
something, so this should be pretty safe to use now.
Enable the extended attribute support with normal builds.
I also added it explicitly to the :version output as well as make it
able to check using `:echo has("xattr")`, to have users easily check
that this is available.
In contrast to the similar support for SELINUX and SMACK support (which
also internally uses extended attributes), I have made this a FEAT_XATTR
define, instead of the similar HAVE_XATTR.
Add a test and change CI to include relevant packages so that CI can
test that extended attributes are correctly written.
closes: #306closes: #13203
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Autoconf 2.69 too old
Solution: Migrate to Autoconf 2.71
Autoconf 2.69 is almost 10 years old. And 2.71 is also a few years old
as well. Should be pretty well tested by now. It brings a lot of
improvements and there seems to be an ongoing work on autoconf 2.72
already.
This change just addresses two minor changes `autoupdate` suggested, and
then `src/auto/configure` is regenerated by running
cd src
make AUTOCONF=autoconf2.71 autoconf
closes: #12958
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Illia Bobyr <illia.bobyr@gmail.com>
Problem: dynamically linking perl is broken
Solution: Fix all issues
This is a combination of several commits:
1) Fix if_perl.xs not being able to build on all versions of Perl (5.30)
This fixes the dynamic builds of Perl interface. The Perl interface file
previously had to manually copy and paste misc inline functions verbatim
from the Perl headers, because we defined `PERL_NO_INLINE_FUNCTIONS`
which prevents us form getting some function definitions. The original
reason we defined it was because those inline functions would reference
Perl functions that would cause linkage errors.
This is a little fragile as every time a new version of Perl comes out,
we inevitably have to copy over new versions of inline functions to our
file, and it's also easy to miss updates to existing functions.
Instead, remove the `PERL_NO_INLINE_FUNCTIONS` define, remove the manual
copy-pasted inline functions. Simply add stub implementations of the
missing linked functions like `Perl_sv_free2` and forward them to the
DLL version of the function at runtime. There are only a few functions
that need this treatment, and it's a simple stub so there is very low
upkeep compared to copying whole implementations to the file.
Also, fix the configure script so that if we are using dynamic linkage,
we don't pass `-lperl` to the build flags, to avoid accidental external
linkage while using dynamic builds. This is similar to how Python
integration works.
2) Fix GIMME_V deprecation warnings in Perl 5.38
Just use GIMME_V, and only use GIMME when using 5.30 to avoid needing to
link Perl_block_gimme. We could provide a stub like the other linked
functions like Perl_sv_free2, but simply using GIMME is the simplest and
it has always worked before.
3) Fix Perl 5.38 issues
Fix two issues:
3.1. Perl 5.38 links against more functions in their inline headers, so we
need to stub them too.
3.2. Perl 5.38 made Perl_get_context an inline function, but *only* for
non-Windows build. Fix that. Note that this was happening in Vim
currently, as it would build, but fail to run Perl code at runtime.
4) Fix Perl 5.36/5.38 when thread local is used
Perl 5.36 introduced using `_Thread_local` for the current context,
which causes inline functions to fail. Create a stub
`PL_current_context` thread local variable to satisfy the linker for
inlined functions. Note that this is going to result in a different
`PL_current_context` being used than the one used in the library, but so
far from testing it seems to work.
5) Add docs for how to build Perl for dynamic linking to work
closes: #12827closes: #12914
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
Problem: configure: sed uses non-portable regex
Solution: use '*' modifier instead of '\?' in regex
for luajit version detection
closes: #12954
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: ichizok <gclient.gaap@gmail.com>
Problem: autoconf: not correctly detecing include dirs
Solution: make use of python3 to generate includedirs
configure: Python3: Use sysconfig for -I
It seems better to use tools provided by Python for determining the
include directories, rather than construct them "manually".
Current system is broken when using virtual environments for python
3.11.4. It used to work before, but now it detects a incorrect value
for `-I`.
It would probably make sense to switch to a similar logic for lib
folders, that is for the `-l` switch. There are also
`sysconfig.get_config_h_filename()` and
`sysconfig.get_makefile_filename()`, that could replace more Python
specific logic in the current `configure{.ac,}`.
sysconfig provides the necessary tools since Python 2.7.
closes: #12889
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Illia Bobyr <illia.bobyr@gmail.com>
Problem: obsolete macros in configure script
Solution: Remove those and start moving to autoconf 2.71
src/configure.ac: Remove obsolete macros
These macros are declared obsolete in autoconf 2.69, which is almost 10
years old by now:
https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Obsolete-Macros.html
They generate warnings when in a subsequent upgrade to autoconf 2.71.
`autoupdate` from autoupdate 2.71 suggests most of these changes, except
that it also adds obsolete warnings, that where individually checked and
removed.
Regenerated `src/auto/configure` by running:
cd src
autoconf2.69 --output=auto/configure configure.ac
sed --in-place --expression='s@>config.log@>auto/config.log@g' auto/configure
closes: #12888
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Illia Bobyr <illia.bobyr@gmail.com>
Problem: Cannot build with latest luajit
Solution: adjust sed regexp and don't expect '-' in version output
closes: #12896
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: No support for stable Python 3 ABI
Solution: Support Python 3 stable ABI
Commits:
1) Support Python 3 stable ABI to allow mixed version interoperatbility
Vim currently supports embedding Python for use with plugins, and the
"dynamic" linking option allows the user to specify a locally installed
version of Python by setting `pythonthreedll`. However, one caveat is
that the Python 3 libs are not binary compatible across minor versions,
and mixing versions can potentially be dangerous (e.g. let's say Vim was
linked against the Python 3.10 SDK, but the user sets `pythonthreedll`
to a 3.11 lib). Usually, nothing bad happens, but in theory this could
lead to crashes, memory corruption, and other unpredictable behaviors.
It's also difficult for the user to tell something is wrong because Vim
has no way of reporting what Python 3 version Vim was linked with.
For Vim installed via a package manager, this usually isn't an issue
because all the dependencies would already be figured out. For prebuilt
Vim binaries like MacVim (my motivation for working on this), AppImage,
and Win32 installer this could potentially be an issue as usually a
single binary is distributed. This is more tricky when a new Python
version is released, as there's a chicken-and-egg issue with deciding
what Python version to build against and hard to keep in sync when a new
Python version just drops and we have a mix of users of different Python
versions, and a user just blindly upgrading to a new Python could lead to
bad interactions with Vim.
Python 3 does have a solution for this problem: stable ABI / limited API
(see https://docs.python.org/3/c-api/stable.html). The C SDK limits the
API to a set of functions that are promised to be stable across
versions. This pull request adds an ifdef config that allows us to turn
it on when building Vim. Vim binaries built with this option should be
safe to freely link with any Python 3 libraies without having the
constraint of having to use the same minor version.
Note: Python 2 has no such concept and this doesn't change how Python 2
integration works (not that there is going to be a new version of Python
2 that would cause compatibility issues in the future anyway).
---
Technical details:
======
The stable ABI can be accessed when we compile with the Python 3 limited
API (by defining `Py_LIMITED_API`). The Python 3 code (in `if_python3.c`
and `if_py_both.h`) would now handle this and switch to limited API
mode. Without it set, Vim will still use the full API as before so this
is an opt-in change.
The main difference is that `PyType_Object` is now an opaque struct that
we can't directly create "static types" out of, and we have to create
type objects as "heap types" instead. This is because the struct is not
stable and changes from version to version (e.g. 3.8 added a
`tp_vectorcall` field to it). I had to change all the types to be
allocated on the heap instead with just a pointer to them.
Other functions are also simply missing in limited API, or they are
introduced too late (e.g. `PyUnicode_AsUTF8AndSize` in 3.10) to it that
we need some other ways to do the same thing, so I had to abstract a few
things into macros, and sometimes re-implement functions like
`PyObject_NEW`.
One caveat is that in limited API, `OutputType` (used for replacing
`sys.stdout`) no longer inherits from `PyStdPrinter_Type` which I don't
think has any real issue other than minor differences in how they
convert to a string and missing a couple functions like `mode()` and
`fileno()`.
Also fixed an existing bug where `tp_basicsize` was set incorrectly for
`BufferObject`, `TabListObject, `WinListObject`.
Technically, there could be a small performance drop, there is a little
more indirection with accessing type objects, and some APIs like
`PyUnicode_AsUTF8AndSize` are missing, but in practice I didn't see any
difference, and any well-written Python plugin should try to avoid
excessing callbacks to the `vim` module in Python anyway.
I only tested limited API mode down to Python 3.7, which seemes to
compile and work fine. I haven't tried earlier Python versions.
2) Fix PyIter_Check on older Python vers / type##Ptr unused warning
For PyIter_Check, older versions exposed them as either macros (used in
full API), or a function (for use in limited API). A previous change
exposed PyIter_Check to the dynamic build because Python just moved it
to function-only in 3.10 anyway. Because of that, just make sure we
always grab the function in dynamic builds in earlier versions since
that's what Python eventually did anyway.
3) Move Py_LIMITED_API define to configure script
Can now use --with-python-stable-abi flag to customize what stable ABI
version to target. Can also use an env var to do so as well.
4) Show +python/dyn-stable in :version, and allow has() feature query
Not sure if the "/dyn-stable" suffix would break things, or whether we
should do it another way. Or just don't show it in version and rely on
has() feature checking.
5) Documentation first draft. Still need to implement v:python3_version
6) Fix PyIter_Check build breaks when compiling against Python 3.8
7) Add CI coverage stable ABI on Linux/Windows / make configurable on Windows
This adds configurable options for Windows make files (both MinGW and
MSVC). CI will also now exercise both traditional full API and stable
ABI for Linux and Windows in the matrix for coverage.
Also added a "dynamic" option to Linux matrix as a drive-by change to
make other scripting languages like Ruby / Perl testable under both
static and dynamic builds.
8) Fix inaccuracy in Windows docs
Python's own docs are confusing but you don't actually want to use
`python3.dll` for the dynamic linkage.
9) Add generated autoconf file
10) Add v:python3_version support
This variable indicates the version of Python3 that Vim was built
against (PY_VERSION_HEX), and will be useful to check whether the Python
library you are loading in dynamically actually fits it. When built with
stable ABI, it will be the limited ABI version instead
(`Py_LIMITED_API`), which indicates the minimum version of Python 3 the
user should have, rather than the exact match. When stable ABI is used,
we won't be exposing PY_VERSION_HEX in this var because it just doesn't
seem necessary to do so (the whole point of stable ABI is the promise
that it will work across versions), and I don't want to confuse the user
with too many variables.
Also, cleaned up some documentation, and added help tags.
11) Fix Python 3.7 compat issues
Fix a couple issues when using limited API < 3.8
- Crash on exit: In Python 3.7, if a heap-allocated type is destroyed
before all instances are, it would cause a crash later. This happens
when we destroyed `OptionsType` before calling `Py_Finalize` when
using the limited API. To make it worse, later versions changed the
semantics and now each instance has a strong reference to its own type
and the recommendation has changed to have each instance de-ref its
own type and have its type in GC traversal. To avoid dealing with
these cross-version variations, we just don't free the heap type. They
are static types in non-limited-API anyway and are designed to last
through the entirety of the app, and we also don't restart the Python
runtime and therefore do not need it to have absolutely 0 leaks.
See:
- https://docs.python.org/3/whatsnew/3.8.html#changes-in-the-c-api
- https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api
- PyIter_Check: This function is not provided in limited APIs older than
3.8. Previously I was trying to mock it out using manual
PyType_GetSlot() but it was brittle and also does not actually work
properly for static types (it will generate a Python error). Just
return false. It does mean using limited API < 3.8 is not recommended
as you lose the functionality to handle iterators, but from playing
with plugins I couldn't find it to be an issue.
- Fix loading of PyIter_Check so it will be done when limited API < 3.8.
Otherwise loading a 3.7 Python lib will fail even if limited API was
specified to use it.
12) Make sure to only load `PyUnicode_AsUTF8AndSize` in needed in limited API
We don't use this function unless limited API >= 3.10, but we were
loading it regardless. Usually it's ok in Unix-like systems where Python
just has a single lib that we load from, but in Windows where there is a
separate python3.dll this would not work as the symbol would not have
been exposed in this more limited DLL file. This makes it much clearer
under what condition is this function needed.
closes: #12032
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
Problem: Error when cross-compiling Vim
Solution: use AC_CHECK_SIZEOF to find sizeof(wchar_t)
This fixes an error when cross compiling.
closes: #12828
Bug: https://bugs.gentoo.org/889430
Signed-off-by: Mike Gilbert <floppym@gentoo.org>
Signed-off-by: Christian Brabandt <cb@256bit.org>
Problem: Configure does not handle all FORTIFY_SOURCE variants.
Solution: Also handle Fedora's default FORTIFY_SOURCE flags. (Zdenek Dohnal,
closes#11794)
Problem: There is no real need for a "big" build.
Solution: Move common features to "normal" build, less often used features
to the "huge" build. (Martin Tournoij, closes#11283)
Problem: Too many #ifdefs.
Solution: Graduate the +cmdwin feature. Now the tiny and small builds are
equal, drop the small build. (Martin Tournoij, closes#11268)
Problem: If running configure with cached results -lrt may be missing.
Solution: Use two cache variables, one without and one with -lrt.
(closes#10799) Swap checks to avoid adding -lrt unnecessarily.
Problem: Timeout implementation is not optimal.
Solution: Further improvements for timeouts. Add a test for searchpair()
timeout. (partly by Paul Ollis)
Problem: Autoconf 2.71 produces many obsolete warnings.
Solution: Replace obsolete macros with non-obsolete ones, where the
functionality does not change. (issue #10528)
Problem: The neXTaw GUI is old and does not work well.
Solution: Remove the neXTaw GUI from configure to find out who still wants
support for this GUI.
Problem: The Athena GUI is old and does not work well.
Solution: Remove the Athena GUI from configure to find out who still wants
support for this GUI.
