In particular, this makes sure we don't add a trailing slash when the
user specified a 'release-monitoring-url' property for the
'generic-html' updater.
* guix/gnu-maintenance.scm (latest-html-release): When DIRECTORY is
empty, do not append it.
This avoids the script crashing if all data is fetched from the cache.
* guix/scripts/weather.scm (report-server-coverage): Only show request
statistics when some requests have been made.
Fixes a bug whereby NetworkManager would be updated to version "rc2".
* guix/import/gnome.scm (latest-gnome-release)[even-minor-version?]:
Change catch-all case to return #f when the first part is not a digit.
Previously, the "seconds per request" and "requests per second" statistics
really reported (cache lookups + requests) per second. By looking at the
actual number of requests made within lookup-narinfos, a more representative
value can be reported.
* guix/scripts/weather.scm (let/time): Allow for multiple return values.
(report-server-coverage): Alter the reporting of request statistics.
As an additional value, in addition to the narinfos. This value is useful in
the weather script for reporting how many requests to the substitute server
were made.
* guix/substitutes.scm (lookup-narinfos): Additionally return the number of
requests made.
The call-with-connection-error-handling was added in
20c08a8a45, but that error handling was
previously inside of open-connection-for-uri/maybe, which is related
to (call-)with-cached-connection which was used in process-substitution, but
only actually used with call-with-cached-connection when used in
fetch-narinfos.
There's some handling for similar errors within with-networking, which is used
within process-substitution.
* guix/scripts/substitute.scm (process-substitution): Remove
call-with-connection-error-handling call.
In f50f5751ff, the way fetch was called within
process-substitution was changed. As call-with-cached-connection actually
includes important error handling for the opening of a HTTP request, this
change removed some error handling. This commit adds that back.
Fixes <https://bugs.gnu.org/47157>.
* guix/scripts/substitute.scm (call-with-cached-connection): New procedure.
(with-cached-connection): New syntax rule.
(process-substitution): Retry once for some errors when making HTTP requests
to fetch substitutes.
This brings total updater coverage, as reported by 'guix refresh
--list-updaters', from 78% to 88.3%. Among many other things, it covers
freedesktop.org packages.
* guix/gnu-maintenance.scm (html-updatable-package?)
(latest-html-updatable-release): New procedures.
(%generic-html-updater): New variable.
* doc/guix.texi (Invoking guix refresh): Document it.
* guix/gnu-maintenance.scm (latest-html-release): Use 'tarball->version'
rather than 'package-name->name+version' to extract the version number.
This fixes problems with packages like 'netsurf' and 'libdom' that have
"-src" in their tarball name, where "src" would be taken as the new
version number.
* guix/gnu-maintenance.scm (latest-html-release): #:file->signature
defaults to #f.
[file->signature/guess]: New procedure.
[url->release]: Use it when FILE->SIGNATURE is #f.
Introduce 'links' variable.
(url-prefix-rewrite): Check whether URL is true before calling
'string-prefix?'.
(latest-savannah-release): Adjust comment about detached signatures.
On some systems, the columns in /proc/self/mountinfo look like this:
23 28 0:21 / /proc rw,nosuid,nodev,noexec,relatime shared:11 - proc proc rw
Before this change, the mounts procedure was written with the assumption that
the type and source could always be found in columns 8 and 9, respectively.
However, the proc(5) man page explains that there can be zero or more optional
fields starting at column 7 (e.g., "shared:11" above), so this assumption is
false in some situations.
* guix/build/syscalls.scm (mounts): Update the match pattern to use ellipsis
to match zero or more optional fields followed by a single hyphen. Remove the
trailing ellipsis, since multiple ellipses are not allowed in the same level.
The proc(5) man page indicates that there are no additional columns, so it is
probably OK to match an exact number of columns at the end like this.
This fixes an issue where two packages share a common dependent.
* guix/build/cargo-build-system.scm (unpack-rust-crates): Only copy rust
crates into the target directory if there isn't one already there with
the same name.
This avoids printing the whole package table in backtraces and such.
* guix/inferior.scm (write-inferior): New procedure.
<top level>: Call 'set-record-type-printer!'.
Partly fixes <https://bugs.gnu.org/46967>.
* guix/build/download.scm (tls-wrap)[unbuffered]: New procedure.
Pass the result of 'make-custom-binary-input/output-port' to
'unbuffered'.
This is a follow-up of 6ee7e3d26b.
* guix/inferior.scm (cached-channel-instance): Introduce "symlink/safe" and
use it instead of symlink. Remove the duplicated "file-exists?" call.
This removes hydra support to use Cuirass as the only continuous integration
system.
* build-aux/hydra/gnu-system.scm: Remove it.
* build-aux/hydra/guix-modular.scm: Ditto.
* build-aux/hydra/guix.scm: Ditto.
* build-aux/cuirass/hydra-to-cuirass.scm: Ditto.
* Makefile.am (EXTRA_DIST): Update it.
(hydra-jobs.scm): Remove it.
(cuirass-jobs.scm): Update it.
* build-aux/hydra/evaluate.scm: Move it to ...
* build-aux/cuirass/evaluate.scm: ... here.
* build-aux/cuirass/guix-modular.scm: Remove it.
* build-aux/cuirass/gnu-system.scm: Ditto.
* guix/packages.scm (%hydra-supported-systems): Rename it to ...
(%cuirass-supported-systems): ... this variable.
* build-aux/check-final-inputs-self-contained: Adapt it.
* etc/release-manifest.scm: Ditto.
* gnu/ci.scm (package->alist): Remove it.
(derivation->job): New procedure.
(package-job, package-cross-job, cross-jobs, image-jobs, system-test-jobs,
tarball-jobs): Use it.
(guix-jobs): New procedure.
(hydra-jobs): Rename it to ...
(cuirass-jobs): ... this procedure.
Break cached-channel-instance into two different procedures:
channels->cached-profile and instances->cached-profile operating respectively
on channels and channels instances.
* guix/inferior.scm (cached-channel-instance): Rename it into ...
(cached-profile): ... this new procedure.
(channels->cached-profile, instances->cached-profile): New procedures.
* guix/scripts/time-machine.scm (guix-time-machine): Adapt accordingly.
This means there's a useful progress bar when running guix weather.
* guix/scripts/weather.scm (report-server-coverage): Pass
#:make-progress-reporter to lookup-narinfos.
Rather than always outputting to (current-error-port) in
lookup-narinfos (which is called from within lookup-narinfos/diverse), take a
procedure which should return a progress reporter, and defer any output to
that.
As this is now general purpose code, make the default behaviour to output
nothing. Maintain the current behaviour of the substitute script by moving the
progress reporter implementation there, and passing it in when calling
lookup-narinfos/diverse.
These changes should be generally useful, but I'm particularly looking at
getting guix weather to do progress reporting differently, with this new
flexibility.
* guix/substitutes.scm (fetch-narinfos): Take a procedure to make a
progress-reporter, and use that rather than the hardcoded behaviour.
(lookup-narinfos): Add #:make-progress-reporter keyword argument, and pass
this through to fetch-narinfos.
(lookup-narinfos/diverse): Add a #:make-progress-reporter keyword argument,
and pass this through to lookup-narinfos.
* guix/scripts/substitute.scm (process-query): Pass a progress-reporter to
lookup-narinfos/diverse.
Fixes <https://bugs.gnu.org/38884>.
* guix/scripts/system.scm (switch-to-system-generation): Load the
activate script for that generation.
squash! scripts: system: Activate system when switching generations.
Previously, test cases could fail if some process was listening
at a hard-coded port. This patch eliminates most of these potential
failures, by automatically assigning an unbound port. This should
allow for building multiple guix trees in parallel outside a build
container, though this is currently untested.
The test "home-page: Connection refused" in tests/lint.scm still
hardcodes port 9999, however.
* guix/tests/http.scm
(http-server-can-listen?): remove now unused procedure.
(%http-server-port): default to port 0, meaning the OS
will automatically choose a port.
(open-http-server-socket): remove the false statement claiming
this procedure is exported and also return the allocated port
number.
(%local-url): raise an error if the port is obviously unbound.
(call-with-http-server): set %http-server-port to the allocated
port while the thunk is called.
* tests/derivations.scm: adjust test cases to use automatically
assign a port. As there is no risk of a port conflict now,
do not make any tests conditional upon 'http-server-can-listen?'
anymore.
* tests/elpa.scm: likewise.
* tests/lint.scm: likewise, and add a TODO comment about a port
that is still hard-coded.
* tests/texlive.scm: likewise.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
This reverts commit b1248016e0, which was
a mistake: the two '%derivation-cache' are two different things. It
broke tests that use 'call-with-external-store'.
This means there's a module for working with substitutes, rather than all the
code sitting in the script. The need for this can be seen with the weather and
challenge scripts, that now don't have to use code from the substitute script,
but can instead use the substitute module.
The separation here between the actual functionality of the substitute script
and the underlying functionality used both there and elsewhere should make
maintenance easier moving forward.
This commit just moves code, none of the code should have been changed
significantly.
* guix/scripts/substitute.scm (%narinfo-cache-directory, %narinfo-ttl,
%narinfo-negative-ttl, %narinfo-transient-error-ttl, %unreachable-hosts): Move
variables to guix/substitutes.scm.
(narinfo-cache-file, cached-narinfo, cache-narinfo!, narinfo-request,
read-to-eof, call-with-connection-error-handling, fetch-narinfos,
lookup-narinfos, lookup-narinfos/diverse): Move procedures to
guix/substitutes.scm.
* guix/substitutes.scm: New file.
* Makefile.am: Add it.
* guix/narinfo.scm: Remove redundant module.
* guix/scripts/challenge.scm: Change (guix scripts substitute) to (guix
substitutes).
* guix/scripts/weather.scm: Change (guix scripts substitute) to (guix
substitutes).
Previously, transformations applied from a manifest (rather than via
"guix install") would be lost. This change fixes that and simplifies
things.
Reported by zimoun at
<https://lists.gnu.org/archive/html/guix-devel/2021-02/msg00153.html>.
* guix/profiles.scm (default-properties): New procedure.
(package->manifest-entry): Use it for #:properties.
* guix/scripts/pack.scm (guix-pack)[with-transformations]: Remove.
Remove caller.
* guix/scripts/package.scm (transaction-upgrade-entry): Remove calls to
'manifest-entry-with-transformations'.
* tests/guix-package.sh: Add test.
* tests/transformations.scm ("options->transformation + package->manifest-entry"):
New test.
Fixes a regression introduced in
c6d6aee6659acb293eb33f498fdac3b47a19a48, where #:reference-graphs would
end up referring to native inputs.
This would notably break the compilation of systems using a childhurd,
because they would attempt to build the 'hurd' package natively.
* guix/gexp.scm (lower-reference-graphs)[tuple->gexp-input]: Honor TARGET.
* tests/gexp.scm ("gexp->derivation #:references-graphs cross-compilation"):
New test.
This fixes a bug whereby the Guix importer considers files like .f.* to be
Fortran files.
The expression "\\.f(90|95)?" would match a lot of files containing ".f"
although they are not Fortran files. Instead we should only consider files
with this *suffix*.
* guix/import/cran.scm (directory-needs-fortran?): Only check for suffixes.
Co-authored-by: Ricardo Wurmus <rekado@elephly.net>
This reduces allocations and bit twiddling in the loop.
* guix/serialization.scm (write-literal-strings): New macro.
(write-file-tree): Use it in lieu of 'write-string' calls where applicable.
* guix/build/syscalls.scm (<mount>): New record type.
(option-string->mount-flags, mount-flags)
(octal-decode, mounts): New procedures.
(mount-points): Rewrite in terms of 'mount'.
* tests/syscalls.scm ("mounts"): New test.
Fixes <https://bugs.gnu.org/46362>.
Reported by Zhu Zihao <all_but_last@163.com>.
Previously, backtraces due to uncaught exceptions would always to go
file descriptor 2; the daemon would read it and error out with something
like:
error: got unexpected path `Backtrace:' from substituter
This patch fixes that by ensuring backtraces are properly displayed on
file descriptor 4.
* guix/scripts/substitute.scm (with-redirected-error-port): New macro.
(guix-substitute): Use 'with-redirected-error-port' instead of 'parameterize'.
This prevents generated launchers and desktop files from inadvertently
crashing if the directory name contains a space.
* gnu/build/renpy-build-system.scm (install, install-desktop-file): Use ~s
to format data directory.
This avoids a closure allocation when 'lookup-cached-object' is called.
* guix/store.scm (lookup-cached-object): Avoid optional/keyword
arguments and inline.
(%mcached): Adjust accordingly.
This reduces the total amount of memory allocated by 8% when running
"guix build qemu -d --no-grafts".
* guix/gexp.scm (fold/tree): New procedure.
(gexp-inputs)[interesting?]: New procedure.
[add-reference-inputs]: Change (lst ...) clause to (? pair? lst), and
use 'fold/tree' to recurse into it.
(gexp-inputs)[add-reference-output]: Likewise, and remove
plain (lst ...) clause.
Call 'fold'.
(gexp->sexp)[reference->sexp]: In the list case, avoid boxing and
recursive call when the object has a plain non-aggregate type.
This avoids double traversal of references and extra bookkeeping,
thereby further reducing memory allocations.
* guix/gexp.scm (lower-gexp): Include only one call to 'lower-inputs'.
(gexp-inputs): Remove #:native? parameter.
[set-gexp-input-native?]: New procedure.
[add-reference-inputs]: Use it.
(gexp-native-inputs): Remove.
* tests/gexp.scm (gexp-native-inputs): Remove.
(gexp-input->tuple): Include 'gexp-input-native?'.
("let-system")
("let-system, nested")
("ungexp + ungexp-native")
("ungexp + ungexp-native, nested")
("ungexp + ungexp-native, nested, special mixture")
("input list")
("input list + ungexp-native")
("input list splicing")
("input list splicing + ungexp-native-splicing")
("gexp list splicing + ungexp-splicing"): Adjust accordingly.
This slightly reduces memory allocation.
* guix/gexp.scm (lower-inputs): Expect a list of <gexp-input> rather
than a list of tuples.
(lower-reference-graphs)[tuple->gexp-input]: New procedure.
Use it.
(gexp-inputs): Return a list of <gexp-input> rather than a list of
tuples.
* tests/gexp.scm (gexp-input->tuple): New procedure.
("one input package")
("one input package, dotted list")
("one input origin")
("one local file")
("one local file, symlink")
("one plain file")
("two input packages, one derivation, one file")
("file-append")
("file-append, output")
("file-append, nested")
("let-system")
("let-system, nested")
("ungexp + ungexp-native")
("ungexp + ungexp-native, nested")
("ungexp + ungexp-native, nested, special mixture")
("input list")
("input list + ungexp-native")
("input list splicing")
("input list splicing + ungexp-native-splicing")
("gexp list splicing + ungexp-splicing"): Adjust accordingly.
Previously, 'guix install foo -r bar' would crash with a backtrace
because NAME would be #\r (a character instead of a string).
* guix/scripts.scm (parse-command-line)[parse-options-from]: Call
'option-hint' only when NAME is a string.
This is part of trying to reduce the interdependency of code within the
substitute module.
This commit addresses some of the error handling that was performed through
open-connection-for-uri/maybe. The new approach is to use
call-with-connection-error-handling, and wrap calls to http-multiple-get and
http-fetch with that procedure, which takes care of handling connection
errors.
I think this is even slightly more rigerous than the previous setup, because
this approach handles connection errors that occur when http-multiple-get
reconnects to a host.
* guix/scripts/substitute.scm (open-connection-for-uri/maybe): Transform in to
call-with-connection-error-handling.
(fetch-narinfos): Use call-with-connection-error-handling.
(process-query): Replace open-connection-for-uri/maybe with
open-connection-for-uri/cached.
(open-connection-for-uri/cached): Set a default timeout, matching the
behaviour in open-connection-for-uri/maybe.
(process-substitution): Use call-with-connection-error-handling.
At least by default. Instead, make the open-connection procedure a parameter,
and make the default guix:open-connection-for-uri. Do so similarly for
lookup-narinfos and lookup-narinfos/diverse which work towards calling
fetch-narinfos.
This means this code can be moved to a different module, without having
use/move the connection caching code.
* guix/scripts/substitute.scm (fetch-narinfos): Add #:open-connection
argument, and call http-multiple-get with it.
(lookup-narinfos) Add #:open-connection argument, and call fetch-narinfos with
it.
(lookup-narinfos/diverse): Add #:open-connection argument, and call
lookup-narinfos with it.
(process-query): Call lookup-narinfos/diverse with #:open-connection
open-connection-for-uri/maybe.
As it's only called in one place, and this should make the code easier to
read.
* guix/scripts/substitute.scm (fetch): Move procedure inside…
(process-substitution): …here.
It's just called in one place, with hardcoded argument values, so just inline
them.
* guix/scripts/substitute.scm (fetch): Remove arguments that don't vary, copy
the values from the call site in process-substitution.
(process-substitution): Remove unnecessary argument values from fetch call.
Failures now should be handled where they occur, and if there's a problem
that's symptomatic of an issue with the connection, the port should be closed.
* guix/scripts/substitute.scm (call-with-cached-connection): Remove procedure.
(with-cached-connection): Remove syntax rule.
Just pass open-connection-for-uri/maybe to http-fetch, this removes the need
for with-cached-connection and passing the port in.
* guix/scripts/substitute.scm (fetch): Don't take a port as an argument, and
pass open-connection-for-uri/maybe to http-fetch.
(process-substitution): Don't call fetch with with-cached-connection.
So that an alternative procedure can be passed in, perhaps to perform
connection caching.
* guix/http-client.scm (http-fetch): Add an #:open-connection keyword
argument.
Instead, just pass open-connection-for-uri/maybe to http-multiple-get. This
code should be functionaly similar to the previous code. The eventual aim of
this is to make the connection caching not mandatory in fetch-narinfos.
* guix/scripts/substitute.scm (fetch-narinfos): Remove use of
call-with-cached-connection.
As this is used by http-fetch and http-multiple-get when they call the
specified open connection procedure.
* guix/scripts/substitute.scm (open-connection-for-uri/maybe): Support
#:verify-certificate?.
Making sure to close the port if it looks to be unusable. This closing of the
port will allow for caching connections, without caching broken connections,
as the cache can avoid handing out closed ports.
* guix/http-client.scm (http-multiple-get): Try to catch exceptions that
happen if the port is unusable, this is a adaptation of code within the (guix
scripts substitute) module.
From (guix scripts substitute). This will make it easier to reuse this code.
* guix/scripts/substitute.scm (http-multiple-get): Remove, and move to…
* guix/http-client.scm (http-multiple-get): …here.
Fixes <https://bugs.gnu.org/46390>.
Reported by zimoun <zimon.toutoune@gmail.com>.
This is similar to the fix in d10474c38d.
* guix/packages.scm (package-field-location): Handle FILE not in %LOAD-PATH.
* tests/guix-lint.sh: Add test.
Remove the 'vm-image' command that has been superseded by the 'image'
command.
* gnu/system/vm.scm (system-qemu-image): Remove it.
* guix/scripts/system.scm (system-derivation-for-action): Mark 'vm-image'
command as deprecated and use the image API to produce the VM image.
(perform-action, show-help): Adapt accordingly.
* tests/guix-system.sh: Ditto.
* doc/guix.texi (Invoking guix system,
Running Guix in a VM): Ditto.
* etc/completion/fish/guix.fish: Ditto.
* etc/completion/zsh/_guix: Ditto.
This fixes <https://bugs.gnu.org/46481>.
* guix/ftp-client.scm (ftp-epsv, ftp-passive): New procedures.
(ftp-list, ftp-retr): Replace call to 'ftp-pasv' with 'ftp-passive'.
* guix/import/crate.scm (make-crate-sexp): Always use home-page, which is
properly set up by the caller.
(crate->guix-package): Make sure to use the repository when home-page is null.
Fixes a regression introduced in
316fc2acbb, whereby 'guix pull -l' would
always display channel information corresponding to the latest profile
generation.
Reported by Vagrant Cascadian.
* guix/scripts/describe.scm (profile-generation-channels): New
procedure.
(display-profile-content): Change default value of 'channels'.
Fixes <https://bugs.gnu.org/45895>.
When the ~/.cache/guix/authentication is empty, this change allows
authentication to start at the current commit, as shown by 'guix
describe', instead of starting from the introductory commit, which would
take more and more time (there's currently 18K commits per year).
* guix/git-authenticate.scm (authenticate-repository): Add #:authentic-commits.
[authenticated-commits]: Append it.
* guix/channels.scm (authenticate-channel)[authentic-commits]: New
variable. Pass it to 'authenticate-repository'.
Partially fixes <https://bugs.gnu.org/45896>.
* guix/config.scm.in (%channel-metadata): New variable.
* guix/describe.scm (channel-metadata): Use it.
(current-channels): New procedure.
(current-profile-entries): Clarify docstring.
* guix/self.scm (compiled-guix): Add #:channel-metadata and pass it to
'make-config.scm'.
(make-config.scm): Add #:channel-metadata and define '%channel-metadata'
in the generated file.
(guix-derivation): Add #:channel-metadata and pass it to 'compiled-guix'.
* guix/channels.scm (build-from-source): Replace 'name', 'source', and
'commit' parameters with 'instance'. Pass #:channel-metadata to BUILD.
(build-channel-instance): Adjust accordingly.
* build-aux/build-self.scm (build-program): Add #:channel-metadata
and pass it to 'guix-derivation'.
(build): Add #:channel-metadata and pass it to 'build-program'.
* guix/scripts/describe.scm (display-profile-info): Add optional
'channels' parameter. Pass it to 'display-profile-content'.
(display-profile-content): Add optional 'channels' parameter and honor
it. Iterate on CHANNELS rather than on the manifest entries of
PROFILE.
(guix-describe): When PROFILE is #f, call 'current-channels' and pass it
to 'display-profile-info', unless it returns the empty list.
The "store-path-hash-part #f", due to a SRFI-64 bug, was marked as
successful even though 'store-path-hash-part' was throwing an exception.
* guix/store.scm (store-path-hash-part): Really return #f.
As this is useful if you want to work with narinfo signatures outside of Guix,
in the Guix Data Service for example.
* guix/narinfo.scm: Export narinfo-contents.
* guix/import/gnu.scm (gnu->guix-package): Move the 'find-package' call before
the 'latest-release' call, which would fail when the package did not have an
entry on the FTP server.
* guix/channels.scm (sexp->channel, manifest-entry-channel): New
procedures.
(profile-channels): Replace lambda by 'manifest-entry-channel'.
(channel-instance->sexp): New procedure.
(channel-instances->manifest)[instance->entry]: Use
'channel-instance->sexp' instead of inline code.
* guix/scripts/package.scm (export-manifest): New procedure.
(show-help, %options): Add '--export-manifest'.
(process-query): Honor it.
* guix/build/profiles.scm (build-profile): Mention it.
* tests/guix-package.sh: Test it.
* doc/guix.texi (Invoking guix package): Document it.
* guix/import/gnu.scm (gnu->guix-package): Use 'formatted-message' and
mention the package name in error messages.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* guix/store/database.scm (assert-integer): New procedure.
(update-or-insert): Use it to validate NAR-SIZE and TIME.
* tests/store-database.scm ("sqlite-register with incorrect size"): New
test.
Fixes a regression introduced in 95f72dcd7a.
* guix/scripts/system/reconfigure.scm (upgrade-shepherd-services)[target-services]:
Add call to 'shepherd-configuration-services'.
That way a command like:
guix time-machine --commit=5aeee07cc9 -- describe
goes from 3.4s to 0.5s on a cache hit, even slightly less when passing
the full commit ID.
* guix/inferior.scm (channel-full-commit): New procedure.
(cached-channel-instance): Remove 'instances' top-level variable. Add
'commits' and use it for 'key'. Move 'latest-channel-instances' call to
the cache miss case.
* guix/ui.scm (run-guix-command): Modify order so that extensions are allowed
to override default commands.
Co-authored-by: Ricardo Wurmus <rekado@elephly.net>
Fixes a performance issue as reported by Ricardo Wurmus
in <https://bugs.gnu.org/46100>.
* guix/inferior.scm (inferior-package->manifest-entry): Remove #:parent parameter.
[cache]: New variable.
[memoized]: New macro.
[loop]: New procedure.
* guix/serialization.scm (read-string-pairs): New procedure.
* guix/store.scm (read-arg): Add support for 'string-pairs'.
(find-roots): New procedure.
* tests/store.scm ("add-indirect-root and find-roots"): New test.
This reverts commit d5a1b0e86f.
This change turned out to be misguided; it would lead to "substitute:"
lines being printed on the client side, without anything beyond the
colon sign.
The 'stack' variable could be #f when code is interpreted, which in
practice happens when running in "legacy" mode--i.e., when
'open-inferior' invokes "guile" instead of "guix repl".
* guix/repl.scm (send-repl-response)[handle-exception]: Check whether
STACK is true before passing it to 'stack->frames'.
* tests/inferior.scm ("&inferior-exception, legacy mode"): New test.
The path normalization of `--root` option of `guix environment` was
buggy as it appended full argument after normalized directory. This
patch fixes it.
* guix/scripts/environment.scm (register-gc-root): Fix gc-root path
normalization.
* tests/guix-environment.sh: Add test.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
This separation between the code for dealing with narinfos from the code doing
that for a purpose should make things clearer, and better support components
other that the substitute script in using this code.
This is just moving the code around, no code should have been significantly
changed.
* guix/scripts/substitute.scm (<narinfo>): Move record type to (guix narinfo).
(fields->alist, narinfo-hash-algorithm+value, narinfo-hash->sha256,
narinfo-signature->canonical-sexp, narinfo-maker, read-narinfo,
narinfo-sha256, valid-narinfo?, write-narinfo, narinfo->string,
string->narinfo, equivalent-narinfo?, supported-compression?,
compresses-better?, narinfo-best-uri): Move procedures to (guix narinfo).
(%compression-methods): Move variable to (guix narinfo).
* guix/narinfo.scm: New file.
* Makefile.am (MODULES): Add it.
* po/guix/POTFILES.in: Add 'guix/narinfo.scm'.
Rather than having valid-narinfo? evaluate to #t if
%allow-unauthenticated-substitutes? is set to #t, just use (const #t) for
valid-narinfo? when %allow-unauthenticated-substitutes? is set to #t. This
will allow moving valid-narinfo? in to a (guix substitutes) module.
* guix/scripts/substitute.scm (process-query, process-substitution): Change
the authorized? argument to lookup-narinfo and lookup-narinfos/diverse based
on %allow-unauthenticated-substitutes?.
(valid-narinfo?): Remove use of %allow-unauthenticated-substitutes?.
* guix/scripts/publish.scm (compress-nar)[write-compressed-file]: New
procedure.
Use it for 'gzip' and 'lzip'. Add 'zstd.
(nar-response-port, string->compression-type): Add case for 'zstd'.
* tests/publish.scm (zstd-supported?): New procedure.
("/nar/zstd/*"): New test.
* doc/guix.texi (Invoking guix publish): Document zstd compression.
(Base Services): Add cross-reference to the above node.
* guix/utils.scm (lzip-port): Return a single value.
(zstd-port): New procedure.
(decompressed-port, compressed-output-port): Add 'zstd' case.
* tests/utils.scm (test-compression/decompression): Test 'zstd' when
the (zstd) module is available.
This procedure was unused except in one test.
* guix/utils.scm (compressed-port): Remove.
* tests/utils.scm (test-compression/decompression): Rewrite to use
'compressed-output-port' instead.
This led to extra "substitute:" lines since commit
79c6614f58 or thereabouts.
* guix/scripts/substitute.scm (fetch-narinfos): Remove extra 'newline'
call, which can be traced back to
d3a652037e (2015).
This ensures ~/.cache/guix/checkouts is periodically cleaned up.
* guix/git.scm (cached-checkout-expiration)
(%checkout-cache-cleanup-period): New variables.
(delete-checkout): New procedure.
(update-cached-checkout)[cache-entries]: New procedure.
Add call to 'maybe-remove-expired-cache-entries'.
* guix/cache.scm (file-expiration-time): Add optional 'timestamp'
parameter and honor it.
Before this change, compiling the module would emit the following message:
"guix/packages.scm:213:25: warning: macro `base32' used before definition".
* guix/packages.scm (define-compile-time-decoder)
(base32, base64): Move definitions to the top of the module.
This is a small improvement awaiting a definitive fix for
<https://issues.guix.gnu.org/45656>.
* guix/import/texlive.scm (sxml->package): Produce a warning when the SVN
checkout failed. Rather than crashing on the unexpected #f value, return a
partial package definition with the source field set to #f.
Fixes <https://bugs.gnu.org/45709>.
This is a follow-up to commit cf289d7cfa.
* guix/ui.scm (run-guix-command): Match for #false first.
Signed-off-by: Ricardo Wurmus <rekado@elephly.net>
* guix/upstream.scm (lookup-updater, package-latest-release)
(package-latest-release*, package-update): Make 'updaters' an optional
parameter.
* guix/lint.scm (check-for-updates): Remove second argument to
'package-latest-release*'.
* guix/scripts.scm (%command-categories): Add extension category.
* guix/ui.scm (source-file-command): Also parse extensions files.
(command-files): Accept an optional directory argument.
(extension-directories): New procedure.
(commands): Use it.
(show-guix-help): Hide empty categories.
(run-guix-command): Try loading an extension if there is no matching Guix
command.
Reported by Christopher Baines <mail@cbaines.net>
at <https://issues.guix.gnu.org/45323#2>.
* guix/scripts/substitute.scm (call-with-cached-connection): Handle
'gnutls-error and ERROR/INVALID-SESSION.
* guix/scripts/hash.scm (guix-hash): When 'recursive? is true, use
'open-hash-port' instead of 'open-sha256-port'.
* tests/guix-hash.sh: Add test for 'guix hash -r -H sha512'.
Suggested by Philippe Swartvagher <philippe.swartvagher@inria.fr>.
* guix/transformations.scm (transform-package-patches): New procedure.
(%transformations): Add it as 'with-patch'.
(%transformation-options, show-transformation-options-help/detailed):
Add '--with-patch'.
* tests/transformations.scm ("options->transformation, with-patch"): New
test.
* doc/guix.texi (Package Transformation Options): Document it.
Fixes <https://bugs.gnu.org/45453>.
* guix/import/utils.scm (alist->package): Find plain license names in (guix
licenses) first, and only fall back to SPDX names on error.
* tests/import-utils.scm ("alist->package with SPDX license name 1/2",
"alist->package with SPDX license name 2/2"): New tests.
This simplifies setup of build machines: no need to install Guile in
addition to Guix, no need to set 'GUILE_LOAD_PATH' & co., leading to
fewer failure modes.
* guix/ssh.scm (remote-run): New procedure.
(remote-daemon-channel): Use it instead of 'open-remote-pipe*'.
(store-import-channel)[import]: Remove check for module availability.
Add call to 'primitive-exit'.
Use 'remote-run' instead of 'open-remote-pipe'.
(store-export-channel)[export]: Remove check for module availability.
Add calls to 'primitive-exit'.
Use 'remote-run' instead of 'open-remote-pipe'.
(handle-import/export-channel-error): Remove 'module-error' clause.
(report-module-error): Remove.
* guix/scripts/offload.scm (assert-node-has-guix): Replace call to
'report-module-error' by 'leave'.
* doc/guix.texi (Daemon Offload Setup): Remove mention of Guile.
This significantly speeds up things like substituting the closure of a
.drv. This is a followup to 5ff521452b.
* guix/scripts/substitute.scm (http-multiple-get): Add #:open-connection
and #:keep-alive? and honor them.
(open-connection-for-uri/maybe): Use 'open-connection-for-uri/cached'
instead of 'guix:open-connection-for-uri'. Call 'http-multiple-get'
within 'call-with-cached-connection'.
(open-connection-for-uri/cached): Add #:timeout and #:verify-certificate?
and honor them.
(call-with-cached-connection): Add 'open-connection' parameter and
honor it.
* guix/import/cran.scm (%input-style): New parameter.
(format-inputs): Use it.
* guix/scripts/import/cran.scm (guix-import-cran): Set the %input-style
parameter.
(%options): Add "--style" option.
* doc/guix.texi (Invoking guix import): Document it.
* gnu/bootloader/grub.scm (grub-configuration-file): New parameter
store-crypto-devices.
[crypto-devices]: New helper function.
[builder]: Use crypto-devices.
* gnu/machine/ssh.scm (roll-back-managed-host): Use
boot-parameters-store-crypto-devices to provide its contents to the
bootloader configuration generation process.
* gnu/tests/install.scm (%encrypted-root-not-boot-os,
%encrypted-root-not-boot-os): New os declaration.
(%encrypted-root-not-boot-installation-script): New script, whose contents
were initially taken from %encrypted-root-installation-script.
(%test-encrypted-root-not-boot-os): New test.
* gnu/system.scm (define-module): Export
operating-system-bootoader-crypto-devices and
boot-parameters-store-crypto-devices.
(<boot-parameters>): Add field store-crypto-devices.
(read-boot-parameters): Parse store-crypto-devices field.
[uuid-sexp->uuid]: New helper function extracted from
device-sexp->device.
(operating-system-bootloader-crypto-devices): New function.
(operating-system-bootcfg): Use
operating-system-bootloader-crypto-devices to provide its contents to
the bootloader configuration generation process.
(operating-system-boot-parameters): Add store-crypto-devices to the
generated boot-parameters.
(operating-system-boot-parameters-file): Likewise to the file with
the serialized structure.
* guix/scripts/system.scm (reinstall-bootloader): Use
boot-parameters-store-crypto-devices to provide its contents to the
bootloader configuration generation process.
* tests/boot-parameters.scm (%default-store-crypto-devices): New
variable.
(%grub-boot-parameters, test-read-boot-parameters): Use
%default-store-crypto-devices.
(tests store-crypto-devices): New tests.
Fixes <https://issues.guix.gnu.org/45314>.
* guix/avahi.scm (avahi-browse-service-thread): Change timeout default value
to false when no "stop-loop?" procedure is passed. Adapt "iterate-simple-poll"
call accordingly.
Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>
This removes the main source of latency between subsequent downloads.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Add a
"deduplicate" key to ENV.
(SubstitutionGoal::finished): Remove call to 'optimisePath'.
* guix/scripts/substitute.scm (process-substitution)[destination-in-store?]
[dump-file/deduplicate*]: New variables.
Pass #:dump-file to 'restore-file'.
* guix/scripts/substitute.scm (guix-substitute)[deduplicate?]: New
variable.
Pass #:deduplicate? to 'process-substitution'.
* guix/serialization.scm (dump-file): Export and augment 'dump-file'.
This way, the hash of the store item can be computed as it is restored,
thereby avoiding an additional file tree traversal ('hashPath' call)
later on in the daemon. Consequently, it should reduce latency between
subsequent substitute downloads.
This is a followup to 5ff521452b.
* guix/scripts/substitute.scm (narinfo-hash-algorithm+value): New
procedure.
(process-substitution): Wrap INPUT into a hash input port, 'hashed', and
read from it. Compare the actual and expected hashes, and print a
"hash-mismatch" status line when they differ. When they match, print
not just "success" but also the nar hash and size.
* nix/libstore/build.cc (class SubstitutionGoal)[expectedHashStr]:
Remove.
(SubstitutionGoal::finished): Tokenize 'status'. Parse it and handle
"success" and "hash-mismatch" accordingly. Call 'hashPath' only when
the returned hash is not SHA256.
(SubstitutionGoal::handleChildOutput): Remove 'expectedHashStr'
handling.
* tests/substitute.scm ("substitute, invalid hash"): Rename to...
("substitute, invalid narinfo hash"): ... this.
("substitute, invalid hash"): New test.
Fixes <https://bugs.gnu.org/44985>.
Reported by Simon Josefsson <simon@josefsson.org>.
* guix/progress.scm (display-download-progress): Add #:tty? and honor it.
* guix/status.scm (print-build-event): Pass #:tty? to
'display-download-progress'.
* guix/import/elpa.scm (default-files-spec): New variable.
(download-git-repository, package-name->melpa-recipe, file-hash, vcs-file?,
git-repository->origin, melpa-recipe->origin, melpa-recipe->maybe-arguments):
New procedures.
(elpa-package->sexp): Add optional repo argument, and use it to determine
whether to attempt to construct a source using the MELPA recipe.
(elpa->guix-package): Pass repo to elpa-package->sexp.
Signed-off-by: Christopher Baines <mail@cbaines.net>
* guix/import/cran.scm (%cran-canonical-url): New variable.
(description->package): Construct home-page using canonical URL.
Signed-off-by: Christopher Baines <mail@cbaines.net>
This avoids repeated (mkdir-p "/gnu/store/.links") calls when
deduplicating lots of files.
* guix/store/deduplication.scm (deduplicate): Remove initial call to
'mkdir-p'. Add ENOENT case in 'link' exception handler. Reindent.
* tests/store-deduplication.scm ("deduplicate, ENOSPC"): Check
for (<= links 4) to account for the initial 'link' call.
* guix/store/database.scm (timestamp): New procedure.
(sqlite-register): Use it as the default for #:time.
(register-items): Likewise for #:registeration-time.
Fixes <https://bugs.gnu.org/44760>.
Previously, the 'register-path' call would re-traverse ITEM to compute
its nar hash, even though that hash is already known in the initial
store. This patch also avoids repeated opening/closing of the
database.
* guix/store/database.scm (call-with-database): Export.
* guix/scripts/system.scm (copy-item): Add 'db' parameter. Call
'sqlite-register' instead of 'register-path'.
(copy-closure): Remove redundant call to 'references*'. Call
'call-with-database' and pass the database to 'copy-item'.
Partly fixes <https://bugs.gnu.org/44760>.
* guix/build/store-copy.scm (copy-store-item): New procedure.
(populate-store): Use it instead of the inline 'copy-recursively' call.
* guix/scripts/system.scm (copy-item): Likewise.
Pass #:reset-timestamps? and #:deduplicate? to 'register-path'.
It is now up to the caller to deduplicate store contents.
* guix/store/database.scm (register-items): Remove #:deduplicate?
parameter and call to 'deduplicate'.
(register-path): Call 'deduplicate' when #:deduplicate? is true.
* gnu/build/image.scm (register-closure): Adjust call accordingly.
* gnu/build/vm.scm (register-closure): Likewise.
* guix/nar.scm (finalize-store-file): Likewise.
* guix/scripts/pack.scm (store-database): Likewise.
Until now deduplication was performed as an additional pass after
copying files, which involve re-traversing all the files that had just
been copied.
* guix/store/deduplication.scm (copy-file/deduplicate): New procedure.
* tests/store-deduplication.scm ("copy-file/deduplicate"): New test.
* guix/build/store-copy.scm (populate-store): Add #:deduplicate?
parameter and honor it.
* tests/gexp.scm ("gexp->derivation, store copy"): Pass #:deduplicate? #f
to 'populate-store'.
* gnu/build/image.scm (initialize-root-partition): Pass #:deduplicate?
to 'populate-store'. Pass #:deduplicate? #f to 'register-closure'.
* gnu/build/vm.scm (root-partition-initializer): Likewise.
* gnu/build/install.scm (populate-single-profile-directory): Pass
#:deduplicate? #f to 'populate-store'.
* gnu/build/linux-initrd.scm (build-initrd): Likewise.
* guix/scripts/pack.scm (self-contained-tarball)[import-module?]: New
procedure.
[build]: Pass it as an argument to 'source-module-closure'.
* guix/scripts/pack.scm (squashfs-image)[build]: Wrap in
'with-extensions'.
* gnu/system/linux-initrd.scm (expression->initrd)[import-module?]: New
procedure.
[builder]: Pass it to 'source-module-closure'.
* gnu/system/install.scm (cow-store-service-type)[import-module?]: New
procedure. Pass it to 'source-module-closure'.
The assumption now is that the caller took care of resetting timestamps
and permissions.
* guix/store/database.scm (register-items): Remove #:reset-timestamps?
parameter and the call to 'reset-timestamps'.
(register-path): Adjust accordingly and add call to 'reset-timestamps'.
* gnu/build/image.scm (register-closure): Remove #:reset-timestamps?
parameter to 'register-items'.
* gnu/build/vm.scm (register-closure): Likewise.
* guix/nar.scm (finalize-store-file): Adjust accordingly.
* guix/scripts/pack.scm (store-database)[build]: Likewise.
Until now, 'populate-store' would reset permissions but not timestamps,
so callers would resort to going through an extra directory traversal to
reset timestamps.
* guix/build/store-copy.scm (reset-permissions): Remove.
(copy-recursively): New procedure.
(populate-store): Pass #:keep-permissions? to 'copy-recursively'.
Remove call to 'reset-permissions'.
* tests/gexp.scm ("gexp->derivation, store copy"): In BUILD-DRV, check
whether 'populate-store' canonicalizes permissions and timestamps.
* gnu/build/image.scm (initialize-root-partition): Pass #:reset-timestamps? #f
to 'register-closure'.
* gnu/build/vm.scm (root-partition-initializer): Likewise.
This avoids having to traverse and re-read the files that we have just
restored, thereby reducing I/O.
* guix/serialization.scm (dump-file): New procedure.
(restore-file): Add #:dump-file parameter and honor it.
* guix/store/deduplication.scm (tee, dump-file/deduplicate): New
procedures.
* guix/nar.scm (restore-one-item): Pass #:dump-file to 'restore-file'.
(finalize-store-file): Pass #:deduplicate? #f to 'register-items'.
* tests/nar.scm <top level>: Call 'setenv' to set "NIX_STORE".
* guix/serialization.scm (restore-file): Set the permissions and mtime
of FILE.
* guix/nar.scm (finalize-store-file): Pass #:reset-timestamps? #f to
'register-items'.
* tests/nar.scm (rm-rf): Add 'chmod' calls to ensure files are writable.
("write-file + restore-file with symlinks"): Ensure every file in OUTPUT
passes 'canonical-file?'.
* tests/guix-archive.sh: Run "chmod -R +w" before "rm -rf".
* guix/serialization.scm (fold-archive): Call PROC with a
'directory-complete tag when done with a directory.
(restore-file): Handle it.
* guix/scripts/archive.scm (list-contents): Likewise.
* guix/scripts/challenge.scm (archive-contents): Likewise.
* tests/nar.scm ("write-file-tree + fold-archive"): Adjust accordingly.
* guix/download.scm (%mirrors): Remove defunct ImageMagick mirrors, update
URLs for directory structure changes (/releases instead of /legacy). Move
official site last.
Previously, after a failed recursive import "guix import" would signal
success.
* guix/import/cran.scm (cran->guix-package): Raise a condition when all
repositories have been exhausted.
* guix/scripts/import/cran.scm (guix-import-cran): Handle errors.
Until now, Guile modules would first be searched for in
MODULE-DIRECTORY, then in each $GUILE_LOAD_PATH entry, and finally in
Guile itself.
* guix/self.scm (guix-command): Make GUILE the second entry in the
%LOAD-PATH and %LOAD-COMPILED-PATH.
This is a follow-up to commit 45584061a9.
* guix/import/utils.scm (package->definition): Use EQ? instead of = to compare
truthiness; use ELSE instead of (#T).
This is a followup to a8dccd4bdc, which
broke the test.
* guix/import/opam.scm (get-opam-repository): Prevent inlining.
* tests/opam.scm ("opam->guix-package"): Mock 'get-opam-repository'.
That way, when fetching a series of substitutes from the same server(s),
the connection is reused instead of being closed/opened for each
substitutes, which saves on network round trips and TLS handshakes.
* guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it.
* guix/progress.scm (progress-report-port): Add #:close? parameter and
honor it.
* guix/scripts/substitute.scm (at-most): Return the tail as a second
value.
(fetch): Add #:port and #:keep-alive? and honor them.
(%max-cached-connections): New variable.
(open-connection-for-uri/cached, call-with-cached-connection): New
procedures.
(with-cached-connection): New macro.
(process-substitution): Wrap 'fetch' call in 'with-cached-connection'.
Pass #:close? to 'progress-report-port'.
This avoids spawning one substitute process per substitution.
* nix/libstore/build.cc (class Worker)[substituter]: New field.
[outPipe, logPipe, pid]: Remove.
(class SubstitutionGoal)[expectedHashStr, status, substituter]: New fields.
(SubstitutionGoal::timedOut): Adjust to check 'substituter'.
(SubstitutionGoal::tryToRun): Remove references to 'outPipe' and
'logPipe'. Run "guix substitute --substitute" as an 'Agent'. Send the
request with 'writeLine'.
(SubstitutionGoal::finished): Likewise.
(SubstitutionGoal::handleChildOutput): Change to fill in
'expectedHashStr' and 'status'.
(SubstitutionGoal::handleEOF): Call 'wakeUp' unconditionally.
(SubstitutionGoal::~SubstitutionGoal): Adjust to check 'substituter'.
* guix/scripts/substitute.scm (process-substitution): Write "success\n"
to stdout upon success.
(%error-to-file-descriptor-4?): New variable.
(guix-substitute): Set 'current-error-port' to file descriptor 4
unless (%error-to-file-descriptor-4?) is false.
Remove "--substitute" arguments. Loop reading line from stdin.
* tests/substitute.scm <top level>: Call '%error-to-file-descriptor-4?'.
(request-substitution): New procedure.
("substitute, no signature")
("substitute, invalid hash")
("substitute, unauthorized key")
("substitute, authorized key")
("substitute, unauthorized narinfo comes first")
("substitute, unsigned narinfo comes first")
("substitute, first narinfo is unsigned and has wrong hash")
("substitute, first narinfo is unsigned and has wrong refs")
("substitute, two invalid narinfos")
("substitute, narinfo with several URLs"): Adjust to new "guix
substitute --substitute" calling convention.
* nix/libstore/local-store.hh (RunningSubstituter): Remove.
(LocalStore)[runningSubstituter]: Change to unique_ptr<Agent>.
[setSubstituterEnv, didSetSubstituterEnv]: Remove.
[getLineFromSubstituter, getIntLineFromSubstituter]: Take an 'Agent'.
* nix/libstore/local-store.cc (LocalStore::~LocalStore): Remove
reference to 'runningSubstituter'.
(LocalStore::setSubstituterEnv, LocalStore::startSubstituter): Remove.
(LocalStore::getLineFromSubstituter): Adjust to 'run' being an 'Agent'.
(LocalStore::querySubstitutablePaths): Spawn substituter agent if
needed. Adjust to 'Agent' interface.
(LocalStore::querySubstitutablePathInfos): Likewise.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Remove call to
'setSubstituterEnv' and add 'setenv' call for "_NIX_OPTIONS" instead.
(SubstitutionGoal::finished): Remove 'readLine' call for 'dummy'.
* guix/scripts/substitute.scm (%allow-unauthenticated-substitutes?):
Remove second argument to 'make-parameter'.
(process-query): Call 'warn-about-missing-authentication'
when (%allow-unauthenticated-substitutes?) is #t.
(guix-substitute): Wrap body in 'parameterize'. Set 'guix-warning-port'
too. No longer exit when 'substitute-urls' returns the empty list. No
longer print newline initially.
* tests/substitute.scm (test-quit): Parameterize 'current-error-port' to
account for the port changes in 'guix-substitute'.
* guix/import/opam.scm (get-opam-repository): Add support for coq
repositories.
(ocaml-name->guix-name): Properly name coq package.
* doc/guix.texi (Invoking guix import): Document it.
* guix/scripts/import/opam.scm (guix-import-opam): Pass --repo argument
to recursive and non-recursive importers.
* guix/import/opam.scm (get-opam-repository): Select proper repository
location depending on a new repo argument.
(opam->guix-package): Use get-opam-repository in the procedure body.
Fixes <https://bugs.gnu.org/23874>.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* guix/profiles.scm (manifest-transaction-effects): Delete duplicates in
install and remove. Let multiple upgrades and downgrades shadow previous
transactions of the same kind.
* tests/profiles.scm
("manifest-transaction-effects no double install or upgrades")
("manifest-transaction-effects no double downgrade")
("manifest-transaction-effects no double removal"): New tests.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* guix/ui.scm (call-with-paginated-output-port): Empty PAGER values
disable paging. Non-empty ones are split into command arguments.
Reported by Daniel Brooks <db48x@db48x.net>.
Reported by Simon Tournier on #guix.
Until now 'reference-available?' would always return #f when passed a
short commit ID.
* guix/git.scm (reference-available?): Call 'object-lookup-prefix' when
COMMIT is shorter than 40 characters.
* guix/import/crate.scm: Add comment for the 'module-autoload!' calls.
* tests/crate.scm (have-guile-semver?): New variable.
("crate->guix-package", "cargo-recursive-import")
("cargo-recursive-import-hoors-existing-packages"): Skip when
HAVE-GUILE-SEMVER? is false.
* gnu/packages/guile-xyz.scm (guile-semver)[inputs]: Use GUILE-3.0.
(guile3.0-semver): Define in terms of 'deprecated-package'.
(guile2.2-semver): New variable.
If a package satisfying the dependency's semver requirement already exists,
use it. Prior to this change the highest version matching the semver
requirement was used (and imported in case it was not defined as package
already).
When resolving a dependency (now done in `sort-map-dependencies`), first
search for a package matching the semver requirement and only if this fails
reach out for a crate.
* guix/import/crate.scm (crate->guix-package)[find-package-version]: New
function. [dependency-name+version]: New function.
[sort-map-dependencies]: Use it instead of lambda function.
* tests/crate.scm (test-doctool-crate, test-doctool-dependencies): New
variables.
("self-test …", "cargo-recursive-import-hoors-existing-packages"): New
tests.
This complies to how versions are matched for caret requirements in crates:
An update is allowed if the new version number does not modify the left-most
non-zero digit in the major, minor, patch grouping.
* guix/import/crate.scm (version->semver-prefix): New function.
(make-crate-sexp)[format-inputs]: Use it.
(make-crate-sexp): Use it to pass shorter version to package->definition.
* guix/import/utils.scm (package->definition): Change optional parameter
APPEND-VERSION? into APPEND-VERSION?/STRING. If it is a string, append its
value to name.
* tests/crate.scm: Adjust tests accordingly.
This remove the patch version from generated package names. For example
'rust-my-crate-1.1.2' now becomes 'rust-my-crate-1.1'.
* guix/import/utils.scm (package->definition): Trim patch version from
generated package names.
* tests/crate.scm: (cargo>guix-package, cargo-recursive-import): Likewise.
The recursive crate importer will now include development dependencies only
for the top level package, but not for any of the recursively imported
packages. Also #:skip-build will be false for the top-most package.
* guix/import/crate.scm (make-crate-sexp): Add the key BUILD?.
(crate->guix-package): Add the key INCLUDE-DEV-DEPS?.
(crate-recursive-import): Likewise.
* guix/scripts/import/crate.scm (guix-import-crate): Likewise.
* tests/crate.scm (cargo-recursive-import): Likewise.
This adds memorization to procedures that involve network lookups.
'lookup-crate*' is used on every dependency of a package to get its version
list. It is also used to lookup a package's metadata. 'crate-recursive-import'
is also memorized since creating the same package twice will trigger a lookup
on its dependencies.
* guix/import/crate.scm (lookup-crate*): New procedure.
(crate->guix-package): Memorize package metadata lookups.
(crate-recursive-import): Memorize package creation.
* guix/import/crate.scm: Add guile-semver as a soft dependency.
(make-crate-sexp): Don't allow other keys. Add '#:skip-build?' to build
system args. Pass a VERSION argument to 'cargo-inputs'.
(crate->guix-package): Use guile-semver to resolve the correct module
versions. Treat "build" dependencies as normal dependencies.
(crate-name->package-name): Reuse the procedure 'guix-name' instead of
duplicating its logic.
* guix/import/utils.scm (package-names->package-inputs): Implement
handling of (name version) pairs.
* guix/scripts/import/crate.scm (guix-import-crate): Use
crate-recursive-import instead of duplicate code.
* tests/crate.scm (recursive-import): Change test packages versions to be
distinguishable. Add version data to the test. Check created symbols, too.
Co-authored-by: Hartmut Goebel <h.goebel@crazy-compilers.com>
This adds a key VERSION to 'recursive-import' and moves the parameter REPO to
a key. This also changes all the places that rely on 'recursive-import'.
* guix/import/utils.scm (recursive-import): Add the VERSION key. Make REPO a
key.
(package->definition): Add optional 'append-version?'.
* guix/scripts/import/crate.scm (guix-import-crate): Add the VERSION key.
* guix/import/crate.scm (crate->guix-package): Add the VERSION key.
(crate-recursive-import): Pass VERSION to recursive-import, remove now
unnecessary code.
* guix/import/cran.scm (cran->guix-package, cran-recursive-import): Change the
REPO parameter to a key.
* guix/import/elpa.scm (elpa->guix-package, elpa-recursive-import): Likewise.
* guix/import/gem.scm (gem->guix-package, recursive-import): Likewise.
* guix/import/opam.scm (opam-recurive-import): Likewise.
* guix/import/pypi.scm (pypi-recursive-import): Likewise.
* guix/import/stackage.scm (stackage-recursive-import): Likewise.
* guix/scripts/import/cran.scm (guix-import-cran): Likewise.
* guix/scripts/import/elpa.scm (guix-import-elpa): Likewise.
* tests/elpa.scm (eval-test-with-elpa): Likewise.
* tests/import-utils.scm (recursive-import): Likewise.
Co-authored-by: Hartmut Goebel <h.goebel@crazy-compilers.com>
