This is a followup to 58abd58739, which
did not fully address <https://issues.guix.gnu.org/43491>.
* guix/scripts/pack.scm (wrapped-package)[build](runpath): Rewrite in
terms of 'file-needed/recursive'.
This was obtained by setting up this environment:
guix shell -D guix --with-input=guile@3.0.9=guile-next \
--with-commit=guile-next=e2ed33ef0445c867fe56c247054aa67e834861f2
-- make -j5
then adding 'unused-module' to (@@ (guix build compiler) %warnings),
building, and checking all the "unused module" warnings and removing
those that were definitely unused.
Commit 68380db4c4 moved from
'gexp->derivation', which as a side effect, would lead tests to require
a "world rebuild"--specifically, they'd have to build (default-guile).
This was mitigated by 68775338a5, but that
change introduced another regression.
* guix/scripts/pack.scm (populate-profile-root): Define 'bootstrap?'.
Pass #:guile to 'computed-file', with a value depending on 'bootstrap?'.
* tests/pack.scm ("self-contained-tarball + localstatedir")
("docker-image + localstatedir", "squashfs-image + localstatedir")
("deb archive with symlinks and control files")
("rpm archive can be installed/uninstalled"): Use a <profile> record
instead of a derivation.
It's necessary as 'generate-header' call ends up calling 'find-files', which
could fail to read file names containing non-ascii characters, as spotted in
the wild:
building /gnu/store/...-jami-rpm-pack.rpm.drv...
find-files: ./gnu/store/...-nss-certs-3.81/etc/ssl/certs/NetLock_Arany_=Class_Gold=_F??tan??s??tv??ny.pem:
No such file or directory
* guix/scripts/pack.scm (rpm-archive): Expand set-utf8-locale helper in the
builder gexp.
* guix/rpm.scm: New file.
* guix/scripts/pack.scm (rpm-archive): New procedure.
(%formats): Register it.
(show-formats): Add it.
(guix-pack): Register supported extra-options for the rpm format.
* tests/pack.scm (rpm-for-tests): New variable.
("rpm archive can be installed/uninstalled"): New test.
* tests/rpm.scm: New test.
* doc/guix.texi (Invoking guix pack): Document it.
This allows more code to be reused between the various archive writers.
* guix/scripts/pack.scm (set-utf8-locale): New top-level procedure, extracted
from...
(populate-profile-root): New procedure, extracted from...
(self-contained-tarball/builder): ... here. Add #:target argument. Call
populate-profile-root.
[LOCALSTATEDIR?]: Set db.sqlite file permissions.
(self-contained-tarball): Call self-contained-tarball/builder with the TARGET
argument, and set #:local-build? to #f for the gexp-derivation call. Remove
now extraneous #:target and #:references-graphs arguments from the
gexp->derivation call.
(debian-archive): Call self-contained-tarball/builder with the #:target
argument. Fix indentation. Remove now extraneous #:target and
#:references-graphs arguments from the gexp->derivation call.
* guix/scripts/pack.scm (%options): Extract symlink parsing logic to...
(symlink-spec-option-parser): ... here.
(self-contained-tarball/builder): Add a comment mentioning why a relative file
name is used for the link target.
* guix/scripts/environment.scm (show-environment-options-help): Document new
--symlink option.
(%default-options): Add default value for symlinks.
(%options): Register new symlink option.
(launch-environment/container): Add #:symlinks argument and extend doc, and
create symlinks using evaluate-populate-directive.
(guix-environment*): Pass symlinks arguments to launch-environment/container.
* doc/guix.texi (Invoking guix shell): Document it.
* tests/guix-shell.sh: Add a --symlink (negative) test.
* tests/guix-environment-container.sh: Add tests.
These modules would use (guix grafts) just to access '%graft?' and
related bindings, which are now in (guix store).
* gnu/ci.scm,
guix/gexp.scm,
guix/lint.scm,
guix/scripts.scm,
guix/scripts/archive.scm,
guix/scripts/build.scm,
guix/scripts/challenge.scm,
guix/scripts/deploy.scm,
guix/scripts/environment.scm,
guix/scripts/home.scm,
guix/scripts/pack.scm,
guix/scripts/package.scm,
guix/scripts/pull.scm,
guix/scripts/size.scm,
guix/scripts/system.scm,
guix/scripts/weather.scm,
tests/builders.scm,
tests/channels.scm,
tests/cpan.scm,
tests/derivations.scm,
tests/gexp.scm,
tests/graph.scm,
tests/guix-daemon.sh,
tests/monads.scm,
tests/pack.scm,
tests/packages.scm,
tests/profiles.scm,
tests/system.scm: Remove #:use-module (guix grafts).
Move the compression record to a dedicated module so that it can be used
outside (guix scripts pack) module.
* guix/scripts/pack.scm (<compressor>, %compressors, lookup-compressor): Move
it to ...
* gnu/compression.scm: ... this new file.
* gnu/ci.scm: Adapt it.
* local.mk (GNU_SYSTEM_MODULES): Add it.
Also factorize the --system and --target build options. Check that the passed
system and target arguments are known platforms.
* doc/guix.texi (Additional Build Options): Document the new --list-systems
and --list-targets options.
* guix/scripts/build.scm (show-cross-build-options-help,
show-emulated-build-options-help, list-systems, list-targets): New procedures.
(%standard-cross-build-options, %standard-emulated-build-options): New
variables.
(show-help): Remove --system and --target entries and use
show-cross-build-options-help and show-emulated-build-options-help procedures instead.
(%options): Remove --system and --target entries and use
%standard-cross-build-options and %standard-emulated-build-options variables instead.
* guix/scripts/archive.scm (show-help, %options): Adapt them.
* guix/scripts/environment.scm: Ditto.
* guix/scripts/graph.scm: Ditto.
* guix/scripts/pack.scm: Ditto.
* guix/scripts/pull.scm: Ditto.
* guix/scripts/size.scm: Ditto.
* guix/scripts/weather.scm: Ditto.
Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>
* guix/scripts/pack.scm: Remove duplicated copyright line.
(debian-archive)[single-entry]: Match one or anything, not one or zero.
Reported-by: jgart <jgart@dismail.de>
Thanks to Guillem Jover <guillem@debian.org> on the OFTC's #debian-dpkg
channel for helping with troubleshooting.
Letting GNU Tar recursively walk the complete files hierarchy side-steps the
risks associated with providing a list of file names:
1. Duplicated files in the archive (recorded as hard links by GNU Tar)
2. Missing parent directories.
The above would cause dpkg to malfunction, for example by aborting early and
skipping triggers when there were missing parent directories.
* guix/scripts/pack.scm (self-contained-tarball/builder): Do not call
POPULATE-SINGLE-PROFILE-DIRECTORY, which creates extraneous files such as
/root. Instead, call POPULATE-STORE and INSTALL-DATABASE-AND-GC-ROOTS
individually to more precisely generate the file system. Replace the list of
files by the current directory, "." and streamline the way options are passed.
* gnu/system/file-systems.scm (reduce-directories): Remove procedure.
* tests/file-systems.scm ("reduce-directories"): Remove test.
These fields, while optional per dpkg, are required by other tools such as
reprepro, commonly used to generate apt repositories.
* guix/scripts/pack.scm (debian-archive): Set the control file section field
to 'misc' and the priority field to 'optional'.
* guix/scripts/pack.scm (self-contained-tarball/builder)
[extra-options]: New argument.
(self-contained-tarball, squashfs-image, docker-image)
(debian-archive): Likewise. Remove two TODO comments. Document
EXTRA-OPTIONS. Use the custom control files when provided.
(%deb-format-options): New variable.
(show-deb-format-options, show-deb-format-options/detailed): New procedures.
(%options): Register new options.
(show-help): Augment with new usage.
(guix-pack): Validate and propagate new argument values.
* doc/guix.texi (Invoking guix pack)[deb]: Document how to list advanced
options. Add an example.
* tests/pack.scm (deb archive...): Provide extra-options to the debian-archive
procedure, and validate that the provided files are embedded in the pack.
* .dir-locals.el (scheme-mode)[gexp->derivation]: Define indentation rule.
* guix/scripts/pack.scm (debian-archive): New procedure.
(%formats): Register the new deb format.
(show-formats): Add it to the usage string.
* tests/pack.scm (%ar-bootstrap): New variable.
(deb archive with symlinks): New test.
* doc/guix.texi (Invoking guix pack): Document it.
* NEWS: Add news entry.
Tar translate duplicate files in the archive into hard links. These can cause
problems, as not every tool support them; for example dpkg doesn't.
* gnu/system/file-systems.scm (reduce-directories): New procedure.
(file-prefix?): Lift the restriction on file prefix. The procedure can be
useful for comparing relative file names. Adjust doc.
(file-name-depth): New procedure, extracted from ...
(btrfs-store-subvolume-file-name): ... here.
* guix/scripts/pack.scm (self-contained-tarball/builder): Use
reduce-directories.
* tests/file-systems.scm ("reduce-directories"): New test.
Instead of just naming them by their pack type, add information from the
package(s) they contain to make it easier to differentiate them.
* guix/scripts/pack.scm (define-with-source): New macro.
(manifest->friendly-name): Extract procedure from ...
(docker-image): ... here, now defined via the above macro. Adjust REPOSITORY
argument value accordingly.
(guix-pack): Derive NAME using MANIFEST->FRIENDLY-NAME.
* guix/docker.scm (%tar-determinism-options): Move to a new module and rename
to `tar-base-options'. Adjust references accordingly.
* guix/build/pack.scm: New file.
* Makefile.am (MODULES): Register it.
* guix/scripts/pack.scm (self-contained-tarball/builder): Use it.
This is made to allow reusing it for the debian-archive pack format, added in
a subsequent commit.
* guix/scripts/pack.scm (self-contained-tarball/builder): New procedure,
containing the build code extracted from self-contained-tarball.
(self-contained-tarball): Use the above procedure.
Note: this merge actually changes the 'curl' and 'python-attrs' derivations,
as part of solving caf4a7a277 and
12964df69a respectively.
4604d43c0e (gnu: gnutls@3.6.16: Fix cross-compilation.) was ignored because it
cannot currently be tested.
Conflicts:
gnu/local.mk
gnu/packages/aidc.scm
gnu/packages/boost.scm
gnu/packages/curl.scm
gnu/packages/nettle.scm
gnu/packages/networking.scm
gnu/packages/python-xyz.scm
gnu/packages/tls.scm
Before this change, the system value would be passed as x86_64 when using a
i686-linux emulated system, e.g.:
$ guix environment --system=i686-linux --ad-hoc guile -- \
guile -c '(display (utsname:machine (uname))) (newline)'
x86_64
This change uses the Guile builtin %host-type variable, which doesn't have
this problem:
$ guix environment --system=i686-linux --ad-hoc guile -- \
guile -c '(display %host-type) (newline)'
i686-unknown-linux-gnu
* guix/scripts/pack.scm (docker-image)[#:system] Use %host-type as a fall-back
when target is not defined.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Previously, transformations applied from a manifest (rather than via
"guix install") would be lost. This change fixes that and simplifies
things.
Reported by zimoun at
<https://lists.gnu.org/archive/html/guix-devel/2021-02/msg00153.html>.
* guix/profiles.scm (default-properties): New procedure.
(package->manifest-entry): Use it for #:properties.
* guix/scripts/pack.scm (guix-pack)[with-transformations]: Remove.
Remove caller.
* guix/scripts/package.scm (transaction-upgrade-entry): Remove calls to
'manifest-entry-with-transformations'.
* tests/guix-package.sh: Add test.
* tests/transformations.scm ("options->transformation + package->manifest-entry"):
New test.
It is now up to the caller to deduplicate store contents.
* guix/store/database.scm (register-items): Remove #:deduplicate?
parameter and call to 'deduplicate'.
(register-path): Call 'deduplicate' when #:deduplicate? is true.
* gnu/build/image.scm (register-closure): Adjust call accordingly.
* gnu/build/vm.scm (register-closure): Likewise.
* guix/nar.scm (finalize-store-file): Likewise.
* guix/scripts/pack.scm (store-database): Likewise.
Until now deduplication was performed as an additional pass after
copying files, which involve re-traversing all the files that had just
been copied.
* guix/store/deduplication.scm (copy-file/deduplicate): New procedure.
* tests/store-deduplication.scm ("copy-file/deduplicate"): New test.
* guix/build/store-copy.scm (populate-store): Add #:deduplicate?
parameter and honor it.
* tests/gexp.scm ("gexp->derivation, store copy"): Pass #:deduplicate? #f
to 'populate-store'.
* gnu/build/image.scm (initialize-root-partition): Pass #:deduplicate?
to 'populate-store'. Pass #:deduplicate? #f to 'register-closure'.
* gnu/build/vm.scm (root-partition-initializer): Likewise.
* gnu/build/install.scm (populate-single-profile-directory): Pass
#:deduplicate? #f to 'populate-store'.
* gnu/build/linux-initrd.scm (build-initrd): Likewise.
* guix/scripts/pack.scm (self-contained-tarball)[import-module?]: New
procedure.
[build]: Pass it as an argument to 'source-module-closure'.
* guix/scripts/pack.scm (squashfs-image)[build]: Wrap in
'with-extensions'.
* gnu/system/linux-initrd.scm (expression->initrd)[import-module?]: New
procedure.
[builder]: Pass it to 'source-module-closure'.
* gnu/system/install.scm (cow-store-service-type)[import-module?]: New
procedure. Pass it to 'source-module-closure'.
The assumption now is that the caller took care of resetting timestamps
and permissions.
* guix/store/database.scm (register-items): Remove #:reset-timestamps?
parameter and the call to 'reset-timestamps'.
(register-path): Adjust accordingly and add call to 'reset-timestamps'.
* gnu/build/image.scm (register-closure): Remove #:reset-timestamps?
parameter to 'register-items'.
* gnu/build/vm.scm (register-closure): Likewise.
* guix/nar.scm (finalize-store-file): Adjust accordingly.
* guix/scripts/pack.scm (store-database)[build]: Likewise.
* guix/scripts/pack.scm (wrapped-package)<build>: Build wrappers for
executable files and symlink others.
* tests/guix-pack-relocatable.sh: Test relocatable example of mixed
executable and non-executable files.
Fixes wrapping of non-package things, where the target store directory may
differ in length from the original.
* guix/scripts/pack.scm (wrapped-package)<build-wrapper>: Define
WRAPPER_PROGRAM macro with wrapper's file name.
* gnu/packages/aux-files/run-in-namespace.c (main): Offset index by len of
that file name.
xz compression is slow; using the multi-thread mode of xz can make it more
than 6 times faster, for example when compressing the large linux-libre
source.
* guix/build/utils.scm (%xz-parallel-args): New procedure.
* guix/packages.scm (patch-and-repack): Specify the required above xz
arguments by setting the XZ_DEFAULTS environment variable.
* guix/scripts/pack.scm (%compressors, bootstrap-xz): Modify the commands
Gexps so they do not need to be quoted. This allows lazily evaluating the
arguments on the builder's side. Specify the required xz arguments.
(self-contained-tarball): Do not quote the compressor command value.
(docker-image): Likewise.
* guix/utils.scm (decompressed-port, compressed-port)
(compressed-output-port): Specify the required above xz arguments.
With this change, package transformation options used while building a
manifest are saved in the metadata of the manifest entries.
* guix/scripts/build.scm (transformation-procedure): New procedure.
(options->transformation)[applicable]: Use it. Change to a list of
key/value/proc tuples instead of key/proc pairs.
[package-with-transformation-properties, tagged-object]: New
procedures. Use them.
(package-transformations, manifest-entry-with-transformations): New
procedures.
* guix/scripts/pack.scm (guix-pack)[with-transformations]: New
procedure.
Use it.
* guix/scripts/package.scm (process-actions)[transform-entry]: Use it.
* tests/guix-package-aliases.sh: Add test.
Fixes <https://bugs.gnu.org/42558>.
Until now, loading 'pack-audit.so' in a truly non-Guix environment would
usually fail because 'pack-audit.so' depends on 'libgcc_s.so' and
'libc.so', none of which could be found.
Furthermore, the test was not working as expected: the trick
unshare -mrf sh -c 'mount -t tmpfs none /gnu ; ...'
would allow the fakechroot engine to make its store available as
/gnu/store as a result of another bug.
* gnu/packages/aux-files/run-in-namespace.c (relocated_search_path): New
function.
(exec_with_loader): Pass "--library-path" to the loader.
* guix/scripts/pack.scm (wrapped-package)[build](runpath): New procedure.
(elf-loader-compile-flags): Pass "-DLOADER_AUDIT_RUNPATH".
* tests/guix-pack-relocatable.sh: Remove 'STORE_PARENT'.
(run_without_store): New function. Erase $NIX_STORE_DIR instead of
$STORE_PARENT.
Use 'run_without_store' throughout.
