The "narinfo-string" procedure is expensive in term of IO operations and can
take a while under IO pressure, such a GC collecting. Defer its call to a new
thread created in the http-write procedure.
Fixes: <https://issues.guix.gnu.org/48468>
Partially fixes: <https://issues.guix.gnu.org/49089>
* guix/scripts/publish.scm (render-narinfo): Defer the narinfo string creation
to the http-write procedure.
(compression->sexp, sexp->compression): New procedures.
("X-Nar-Compression"): Use them.
("X-Narinfo-Compressions"): New custom header.
(strip-headers): Add the x-nar-path header.
(http-write): Add narinfo on-the-fly creation support. It happens in a
separated thread to prevent blocking the main thread.
Previously, the max column width for the pretty-printer was 50, which caused
generated package definitions to include unnecessary newlines, e.g.,
(home-page
"https://gitlab.com/ttyperacer/terminal-typeracer")
instead of
(home-page "https://gitlab.com/ttyperacer/terminal-typeracer")
* guix/scripts/import.scm (guix-import): Set max expression width to 80 when
pretty-printing.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
This is a followup to 222f4661ed, which
was intended to improve performance of 'narinfo-string'.
* guix/scripts/publish.scm (narinfo-string): Remove 'catch' and
'read-derivation-from-file' call when rendering "Deriver".
This has been discussed here: https://issues.guix.gnu.org/50040.
* guix/scripts/publish.scm (narinfo-string): Do not render the "System" field
that is expensive to compute and currently unused.
Fixes <https://issues.guix.gnu.org/40997>.
* gnu/bootloader.scm (<bootloader-configuration>): New 'targets' field.
(%bootloader-configuration-target): New procedure.
(bootloader-configuration-target): Add deprecation warning.
(bootloader-configuration-targets): New procedure.
* guix/scripts/system.scm (install): Access targets via
bootloader-configuration-targets.
(perform-action)[bootloader-target]: Remove unused argument and update doc.
Access targets via bootloader-configuration-targets and fix indentation.
(process-action): Access targets via bootloader-configuration-targets.
Do not provide the unused BOOTLOADER-TARGET argument when applying
`perform-action'.
* guix/scripts/system/reconfigure.scm (install-bootloader-program): Rename
DEVICE argument to DEVICES. Adjust doc and comment. Apply `installer' and
`disk-installer' for every DEVICES.
(install-bootloader): Access targets via bootloader-configuration-targets and
rename variable from DEVICE to DEVICES.
* gnu/tests/install.scm: Adjust accordingly.
* tests/guix-system.sh: Likewise.
* gnu/tests/reconfigure.scm (run-install-bootloader-test): Adjust the DEVICES
argument so that it is a list.
* doc/guix.texi: Update doc.
* guix/scripts/import/opam.scm: Pass all instances of --repo as a list
to the importer.
* guix/import/opam.scm (opam-fetch): Stop expecting "expanded"
repositories and call get-opam-repository instead to keep values
"symbolic" as long as possible and factorize.
(get-opam-repository): Use the same repository source as CLI opam does
(i.e. HTTP-served index.tar.gz instead of git repositories).
(find-latest-version): Be more flexible on the repositories structure
instead of expecting packages/PACKAGE-NAME/PACKAGE-NAME.VERSION/.
* tests/opam.scm: Update the call to opam->guix-package since repo is
now expected to be a list and remove the mocked get-opam-repository
deprecated by the support for local folders by the actual
implementation.
* doc/guix.texi: Document the new semantics and valid arguments for the
--repo option.
Signed-off-by: Julien Lepiller <julien@lepiller.eu>
This allows the user of 'cached-channel-instance' to install the build
handler of its choice. In particular, it allows "guix time-machine" to
install a build notifier with the right options instead of using the
defaults that 'cached-channel-instance' would pass to
'show-what-to-build*'.
* guix/inferior.scm (cached-channel-instance): Remove call to
'show-what-to-build*'.
(inferior-for-channels): Wrap body in 'with-build-handler'.
* guix/scripts/time-machine.scm (guix-time-machine): Use
'with-build-handler'.
Thanks to Guillem Jover <guillem@debian.org> on the OFTC's #debian-dpkg
channel for helping with troubleshooting.
Letting GNU Tar recursively walk the complete files hierarchy side-steps the
risks associated with providing a list of file names:
1. Duplicated files in the archive (recorded as hard links by GNU Tar)
2. Missing parent directories.
The above would cause dpkg to malfunction, for example by aborting early and
skipping triggers when there were missing parent directories.
* guix/scripts/pack.scm (self-contained-tarball/builder): Do not call
POPULATE-SINGLE-PROFILE-DIRECTORY, which creates extraneous files such as
/root. Instead, call POPULATE-STORE and INSTALL-DATABASE-AND-GC-ROOTS
individually to more precisely generate the file system. Replace the list of
files by the current directory, "." and streamline the way options are passed.
* gnu/system/file-systems.scm (reduce-directories): Remove procedure.
* tests/file-systems.scm ("reduce-directories"): Remove test.
These fields, while optional per dpkg, are required by other tools such as
reprepro, commonly used to generate apt repositories.
* guix/scripts/pack.scm (debian-archive): Set the control file section field
to 'misc' and the priority field to 'optional'.
* guix/scripts/pack.scm (self-contained-tarball/builder)
[extra-options]: New argument.
(self-contained-tarball, squashfs-image, docker-image)
(debian-archive): Likewise. Remove two TODO comments. Document
EXTRA-OPTIONS. Use the custom control files when provided.
(%deb-format-options): New variable.
(show-deb-format-options, show-deb-format-options/detailed): New procedures.
(%options): Register new options.
(show-help): Augment with new usage.
(guix-pack): Validate and propagate new argument values.
* doc/guix.texi (Invoking guix pack)[deb]: Document how to list advanced
options. Add an example.
* tests/pack.scm (deb archive...): Provide extra-options to the debian-archive
procedure, and validate that the provided files are embedded in the pack.
* guix/scripts/package.scm (process-query): Use pretty-print-table when
listing installed and available packages.
Modified-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Signed-off-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
* .dir-locals.el (scheme-mode)[gexp->derivation]: Define indentation rule.
* guix/scripts/pack.scm (debian-archive): New procedure.
(%formats): Register the new deb format.
(show-formats): Add it to the usage string.
* tests/pack.scm (%ar-bootstrap): New variable.
(deb archive with symlinks): New test.
* doc/guix.texi (Invoking guix pack): Document it.
* NEWS: Add news entry.
Tar translate duplicate files in the archive into hard links. These can cause
problems, as not every tool support them; for example dpkg doesn't.
* gnu/system/file-systems.scm (reduce-directories): New procedure.
(file-prefix?): Lift the restriction on file prefix. The procedure can be
useful for comparing relative file names. Adjust doc.
(file-name-depth): New procedure, extracted from ...
(btrfs-store-subvolume-file-name): ... here.
* guix/scripts/pack.scm (self-contained-tarball/builder): Use
reduce-directories.
* tests/file-systems.scm ("reduce-directories"): New test.
Instead of just naming them by their pack type, add information from the
package(s) they contain to make it easier to differentiate them.
* guix/scripts/pack.scm (define-with-source): New macro.
(manifest->friendly-name): Extract procedure from ...
(docker-image): ... here, now defined via the above macro. Adjust REPOSITORY
argument value accordingly.
(guix-pack): Derive NAME using MANIFEST->FRIENDLY-NAME.
* guix/docker.scm (%tar-determinism-options): Move to a new module and rename
to `tar-base-options'. Adjust references accordingly.
* guix/build/pack.scm: New file.
* Makefile.am (MODULES): Register it.
* guix/scripts/pack.scm (self-contained-tarball/builder): Use it.
This is made to allow reusing it for the debian-archive pack format, added in
a subsequent commit.
* guix/scripts/pack.scm (self-contained-tarball/builder): New procedure,
containing the build code extracted from self-contained-tarball.
(self-contained-tarball): Use the above procedure.
This can happen if the weather information of a package
is requested for an unsupported system. For example,
try "guix weather icecat --system=aarch64-linux".
* guix/scripts/weather.scm
(report-server-coverage): Do not divide by zero when zero
store items are requested from a server.
Fixes: <https://issues.guix.gnu.org/49263>
Reported-By: Jack Hill <jackhill@jackhill.us>
Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>
This reverts commit fd62b4cf88, which
would lead 'GUIX_PACKAGE_PATH' to be ignored for instance when using
'guix build -f file.scm', as shown by 'tests/guix-build.sh'.
Previously, the error message would be displayed, followed by a
backtrace ending in &non-continuable.
* guix/scripts/deploy.scm (deploy-machine*): Call 'leave' rather than
'report-error' when C is a &message.
Note: this merge actually changes the 'curl' and 'python-attrs' derivations,
as part of solving caf4a7a277 and
12964df69a respectively.
4604d43c0e (gnu: gnutls@3.6.16: Fix cross-compilation.) was ignored because it
cannot currently be tested.
Conflicts:
gnu/local.mk
gnu/packages/aidc.scm
gnu/packages/boost.scm
gnu/packages/curl.scm
gnu/packages/nettle.scm
gnu/packages/networking.scm
gnu/packages/python-xyz.scm
gnu/packages/tls.scm
In addition to substitutes from ci.guix.gnu.org. There are more changes that
can be made in the future, but these changes seem like a good start.
* config-daemon.ac (guix_substitute_urls): Add https://bordeaux.guix.gnu.org.
* guix/scripts/substitute.scm (%default-substitute-urls): Add
http://bordeaux.guix.gnu.org.
* guix/store.scm (%default-substitute-urls): Add bordeaux.guix.gnu.org.
* doc/guix.texi: Adjust accordingly.
* doc/contributing.texi: Adjust accordingly.
Before this change, the system value would be passed as x86_64 when using a
i686-linux emulated system, e.g.:
$ guix environment --system=i686-linux --ad-hoc guile -- \
guile -c '(display (utsname:machine (uname))) (newline)'
x86_64
This change uses the Guile builtin %host-type variable, which doesn't have
this problem:
$ guix environment --system=i686-linux --ad-hoc guile -- \
guile -c '(display %host-type) (newline)'
i686-unknown-linux-gnu
* guix/scripts/pack.scm (docker-image)[#:system] Use %host-type as a fall-back
when target is not defined.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Previously, "guix package --switch-generations=4" or similar would print
"nothing to do".
* guix/scripts/package.scm (process-actions): Do not warn about "nothing
to do" when one of %ACTIONS is requested.
The process-substitution procedure is opening two distinct connections. The
first one when looking for narinfo by calling lookup-narinfo and the other one
when fetching nar files.
Cache the connection when looking for narinfos so that process-substitution
only opens one connection.
* guix/scripts/substitute.scm (lookup-narinfo): Cache connection by using
open-connection-for-uri/cached.
The default Guile web server implementation supports the keep alive
mechanism. However, in our custom http-write implementation, the connection
is unconditionally close after sending nar files.
To prevent that, when supported, add the client port to the server poll set so
that further requests can be handled without closing the connection.
* guix/scripts/publish.scm (nar-response-port): Rename it into ...
(nar-compressed-port): ... this procedure. Operate directly on a given PORT.
(http-write): Add keep-alive support when sending nar files.
* guix/scripts/substitute.scm (process-substitution): Pass the download size
to the progress-report-port procedure so that it doesn't block reading from
the input port when keep-alive is supported.
The Guile web server is reading the response connection header to decide
whether to close the connection. However, as the request connection header is
not forwarded to the response, this mechanism cannot work.
* guix/scripts/publish.scm (add-extra-headers): New procedure.
(make-request-handler): Use it to forward the request connection header to the
response.
In some cases, running "guix upgrade" can trigger this warning, and
"missing arguments" is misleading then.
Reported by flatwhatson on #guix.
* guix/scripts/package.scm (process-actions): Change warning to "nothing
to do".
This is a followup to 3f8326237d.
* guix/scripts/archive.scm (export-from-store): Warn then FILES is
empty.
* guix/scripts/build.scm (guix-build): Likewise.
* guix/scripts/copy.scm (warn-if-empty): New procedure.
(send-to-remote-host, retrieve-from-remote-host): Call it.
* guix/scripts/edit.scm (guix-edit): Warn when SPECS is empty.
* guix/scripts/environment.scm (guix-environment): Warn when MANIFEST
has zero entries.
* guix/scripts/graph.scm (guix-graph): Warn then ITEMS is empty.
* guix/scripts/package.scm (process-actions): Warn when FILES and TRANS
are both empty.
This is a follow-up of 3cde5231aa that fixes the
challenge test.
When dealing with uncompressed NAR, the file size is false. Propagate it to
progress-reporter/file as it used to be the case.
* guix/scripts/challenge.scm (call-with-nar): Accept false size.
Commit a8b927a562 Added new procedure go-module-available-versions use
'string-append' to GOPROXY, but 'string->symbol' let GOPROXY is a symbol (it
must be a string), which would lead to wrong-type-arg errors in the
'string-append' procedure.
* guix/scripts/import/go.scm (%options)[goproxy]: Remove call to
'string->symbol'.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Possibly fixes <https://bugs.gnu.org/48045>.
Reported by Bone Baboon <bone.baboon@disroot.org>.
* guix/scripts/challenge.scm (call-with-nar): Rename second value
returned by 'http-fetch' to match reality. Change second argument to
'progress-reporter/file'.
Fixes <https://bugs.gnu.org/48313>.
Previously, 'current-channels' would return the empty list when
'current-profile' is true but the profile lacks provenance info.
* guix/describe.scm (current-channels)[build-time-metadata]: New
procedure. Call it when 'manifest-entry-channel' returns #f for all of
ENTRIES.
* guix/scripts/describe.scm (guix-describe): When PROFILE is true, pass
third argument to 'display-profile-info'.
This is a fixed version of 66b14dccdd,
which was reverted in e74250c3c5.
* guix/download.scm (%disarchive-mirrors): New variable.
(%disarchive-mirror-file): New variable.
(built-in-download): Add 'disarchive-mirrors' keyword argument and
pass its value along to the 'builtin:download' derivation.
(url-fetch): Pass '%disarchive-mirror-file' to 'built-in-download'.
* guix/scripts/perform-download.scm (perform-download): Read
Disarchive mirrors from the environment and pass them to
'url-fetch'.
* guix/build/download.scm (disarchive-fetch/any): New procedure.
(url-fetch): Add 'disarchive-mirrors' keyword argument, use it to
make a list of URIs, and use the new procedure to fetch the file if
all other methods fail.
* build-aux/build-self.scm (build-program)[select?]: Exclude '(guix
build download)'.
* guix/self.scm (compiled-guix)[*core-modules*]: Add 'guile-json' to
the list of extensions.
* guix/download.scm (%disarchive-mirrors): New variable.
(%disarchive-mirror-file): New variable.
(built-in-download): Add 'disarchive-mirrors' keyword argument and
pass its value along to the 'builtin:download' derivation.
(url-fetch): Pass '%disarchive-mirror-file' to 'built-in-download'.
* guix/scripts/perform-download.scm (perform-download): Read
Disarchive mirrors from the environment and pass them to
'url-fetch'.
* guix/build/download.scm (disarchive-fetch/any): New procedure.
(url-fetch): Add 'disarchive-mirrors' keyword argument, use it to
make a list of URIs, and use the new procedure to fetch the file if
all other methods fail.
Partly fixes <https://bugs.gnu.org/47867>.
Reported by Florian Pelz <pelzflorian@pelzflorian.de>.
In GnuTLS up to 3.7.1 included, GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED
are not handled by 'write_to_session_record_port' and could be thrown at
the caller. This patch works around that by dropping connections
altogether and restarting when this happens.
* guix/http-client.scm (false-if-networking-error): Swallow ERROR/AGAIN
and ERROR/INTERRUPTED.
* guix/scripts/substitute.scm (call-with-cached-connection): Likewise.
This importer has suffered from bitrot and no longer works with current
Nix and Nixpkgs. See <https://bugs.gnu.org/32339> and
<https://bugs.gnu.org/36255>.
* guix/import/snix.scm, guix/scripts/import/nix.scm,
tests/snix.scm: Remove.
* Makefile.am (MODULES, SCM_TESTS): Remove them.
* guix/scripts/import.scm (importers): Remove "nix".
* build-aux/test-env.in: Remove NIXPKGS variable.
* configure.ac: Remove '--with-nixpkgs' option.
* doc/guix.texi (Invoking guix import): Remove bit about "guix import
nix".
* etc/completion/fish/guix.fish: Likewise.
Partly fixes <https://bugs.gnu.org/47867>.
Reported by Florian Pelz <pelzflorian@pelzflorian.de>.
In GnuTLS up to 3.7.1 included, GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED
are not handled by 'write_to_session_record_port' and could be thrown at
the caller. This patch works around that by dropping connections
altogether and restarting when this happens.
* guix/http-client.scm (false-if-networking-error): Swallow ERROR/AGAIN
and ERROR/INTERRUPTED.
* guix/scripts/substitute.scm (call-with-cached-connection): Likewise.
This importer has suffered from bitrot and no longer works with current
Nix and Nixpkgs. See <https://bugs.gnu.org/32339> and
<https://bugs.gnu.org/36255>.
* guix/import/snix.scm, guix/scripts/import/nix.scm,
tests/snix.scm: Remove.
* Makefile.am (MODULES, SCM_TESTS): Remove them.
* guix/scripts/import.scm (importers): Remove "nix".
* build-aux/test-env.in: Remove NIXPKGS variable.
* configure.ac: Remove '--with-nixpkgs' option.
* doc/guix.texi (Invoking guix import): Remove bit about "guix import
nix".
* etc/completion/fish/guix.fish: Likewise.
This allows importing packages with complicated version specific dependency
chains without the package symbol names colliding.
* doc/guix.texi (Invoking guix import): Document the --pin-versions option.
Mention that a specific version can be imported. Remove the experimental
warning.
* guix/import/go.scm (go-module->guix-package-name)[version]: Add optional
argument. Rewrite the character translation in terms of string-map.
(go-module->guix-package): Conditionally use dependencies whose symbol include
their version, based no the value of the PIN-VERSIONS? argument.
* guix/import/utils.scm (package->definition): Add a new case where the full
version string is appended to the package symbol.
* guix/scripts/import.scm (guix-import): Correctly print forms starting
with '(define-public [...]'.
* guix/scripts/import/go.scm (guix-import-go): Conditionally include the
version in the package symbols defined.
The ability to pin versions is handy when having to deal to packages that
bootstrap themselves through a chain of former versions. Not using pinned
versions in these case could introduce dependency cycles.
* guix/build-system/go.scm (guix)
(%go-version-rx): Rename to...
(%go-pseudo-version-rx): ... this. Simplify the regular expression, which in
turns makes it more robust.
* guix/build-system/go.scm (go-version->git-ref): Adjust following the above
rename.
(go-pseudo-version?): New predicate.
(go-module-latest-version): Rename to ...
(go-module-version-string): ... this. Rename goproxy-url argument to just
goproxy. Add a VERSION keyword argument, update docstring and adjust to have
it used.
(go-module-available-versions): New procedure.
(%go.mod-require-directive-rx): Document regexp.
(parse-go.mod): Harmonize the way dependencies are recorded to a list of lists
rather than a list of pairs, as done for other importers. Rewrite to directly pass
multiple values rather than a record object. Filter the replaced modules in a
functional style.
(go-module->guix-package): Add docstring.
[version, pin-versions?]: New arguments. Rename the GOPROXY-URL argument to
GOPROXY. Adjust to the new returned value of fetch-go.mod, which is a string.
Fail when the provided version doesn't exist. Return a list dependencies and
their versions when in pinned versions mode, else just the dependencies.
(go-module-recursive-import)[version, pin-versions?]: New arguments.
Honor the new arguments and guard against network errors.
* guix/scripts/import/go.scm (%default-options): Register a default value for
the goproxy argument.
(show-help): Document that a version can be specified. Remove the --version
argument and add a --pin-versions argument.
(%options)[version]: Remove option.
[pin-versions]: Add option.
(guix-import-go): Adjust so the version provided from the module name is
honored, along the new pin-versions? argument.
* tests/go.scm: Adjust and add new tests.
This avoids the situation where error messages would unintentionally go
to stderr and be wrongfully interpreted as a reply by the daemon.
Fixes <https://bugs.gnu.org/46362>.
This is a followup to ee3226e9d5.
* guix/scripts/substitute.scm (display-narinfo-data): Add 'port'
parameter and honor it.
(process-query): Likewise.
(process-substitution): Likewise.
(%error-to-file-descriptor-4?, with-redirected-error-port): Remove.
(%reply-file-descriptor): New variable.
(guix-substitute): Remove use of 'with-redirected-error-port'. Define
'reply-port' and pass it to 'process-query' and 'process-substitution'.
* nix/libstore/build.cc (SubstitutionGoal::handleChildOutput): Swap
'builderOut' and 'fromAgent'.
* nix/libstore/local-store.cc (LocalStore::getLineFromSubstituter):
Likewise.
* tests/substitute.scm <top level>: Set '%reply-file-descriptor'
rather than '%error-to-file-descriptor-4?'.
This is a followup to e45ef9a648.
* guix/scripts/archive.scm (%default-options): Change 'verbosity' to 3.
* guix/scripts/build.scm (%default-options): Likewise.
* guix/scripts/copy.scm (%default-options): Likewise.
The immediate effect is that, with '--verbosity=1', only two lines are
displayed for each substitute, instead of two lines followed by an empty
line.
* guix/scripts/substitute.scm (process-substitution): Emit a single
newline upon completion when PRINT-BUILD-TRACE? is true.
