Commit Graph

32 Commits

Author SHA1 Message Date
Christopher Baines
ad8d386168
svn-download: Reduce svn-fetch builder duplication.
Rather than creating a different builder in the store for every different
download (by hash), remove the hash from the builder and pass it in via an
environment variable.  This means that when svn-fetch is used by two different
package sources, the derivations will still differ but the builder will be
shared.

It used to be this way, but changed with
0e73f933b2.  I noticed the hash in the builder
script when wondering why the build coordinator on bayfront was substituting
svn-multi-download files over and over again.

To try and make the effects of introducing variance in to the builder script
more obvious, separate it out in to it's own procedure, so that it's clearer
when there's new data going in that could cause variance.

* guix/svn-download.scm (svn-fetch): Extract out builder script, include hash
in the derivation as an environment variable and update the comment to be more
directive.
(svn-fetch-builder): New procedure.

Change-Id: I256b94666296ad747f494f0b497ca209b77fbfb4
2024-06-11 11:33:22 +01:00
Christopher Baines
04ca393b38
svn-download: Reduce svn-multi-fetch builder duplication.
Rather than creating a different builder in the store for every different
download (by hash), remove the hash from the builder and pass it in via an
environment variable.  This means that when svn-multi-fetch is used by two
different package sources, the derivations will still differ but the builder
will be shared.

It used to be this way, but changed with
0e73f933b2.  I noticed the hash in the builder
script when wondering why the build coordinator on bayfront was substituting
svn-multi-download files over and over again.

To try and make the effects of introducing variance in to the builder script
more obvious, separate it out in to it's own procedure, so that it's clearer
when there's new data going in that could cause variance.

* guix/svn-download.scm (svn-multi-fetch): Extract out builder script, include
hash in the derivation as an environment variable and update comment to be
more directive.
(svn-multi-fetch-builder): New procedure.

Change-Id: I83c60140ae09e189ee5e5428038a9428ecb8e281
2024-06-11 11:33:05 +01:00
Ludovic Courtès
5d2d081af0
svn-download: Add dependency on tar+gzip for SWH retrieval.
Fixes a bug whereby ‘swh-download-directory-by-nar-hash’ would fail due
to ‘tar’ not being found in $PATH:

  $ GUIX_DOWNLOAD_METHODS=swh guix build -S texlive-geometry --check
  In execvp of tar: No such file or directory
  SWH: found directory with nar-sha256 hash 217e1c8d33aa6fd063f205453a946509b059b85fa7e8d32b7e80054d5e6e5c26 at 'swh:1:dir:ccb56476fb36b2ba0298e57f2b2e69c735ac0a2f'
  […]

* guix/svn-download.scm (svn-fetch)[tar+gzip]: New variable.
[build]: Use it.
(svn-multi-fetch): Likewise.

Change-Id: I37ba9824a5feb6d38f85743519cbd2b2c3e1a9e8
2024-05-01 17:26:18 +02:00
Ludovic Courtès
2f441fc738
download: Honor ‘GUIX_DOWNLOAD_METHODS’ environment variable.
This replaces ‘GUIX_DOWNLOAD_FALLBACK_TEST’ and allows you to test
various download methods, like so:

  GUIX_DOWNLOAD_METHODS=nar guix build guile-gcrypt -S --check
  GUIX_DOWNLOAD_METHODS=disarchive guix build hello -S --check

* guix/build/download.scm (%download-methods): New variable.
(download-method-enabled?): New procedure.
(url-fetch): Define ‘initial-uris’; honor ‘download-method-enabled?’.
Call ‘disarchive-fetch/any’ only when the 'disarchive method is enabled.
* guix/build/git.scm (git-fetch-with-fallback): Honor
‘download-method-enabled?’.
* guix/download.scm (%download-methods): New variable.
(%download-fallback-test): Remove.
(built-in-download): Add #:download-methods parameter and honor it.
(url-fetch*): Pass #:content-addressed-mirrors and #:disarchive-mirrors
unconditionally.
* guix/git-download.scm (git-fetch/in-band*): Pass “git url”
unconditionally.
(git-fetch/built-in): Likewise.  Pass “download-methods”.
* guix/bzr-download.scm (bzr-fetch)[build]: Honor ‘download-method-enabled?’.
Pass ‘GUIX_DOWNLOAD_METHODS’ to #:env-vars.
* guix/cvs-download.scm (cvs-fetch)[build]: Honor ‘download-method-enabled?’.
Pass ‘GUIX_DOWNLOAD_METHODS’ to #:env-vars.
* guix/hg-download.scm (hg-fetch): Honor ‘download-method-enabled?’.
Pass #:env-vars to ‘gexp->derivation’.
* guix/scripts/perform-download.scm (perform-download): Honor
“download-methods” from DRV.  Parameterize ‘%download-methods’ before
calling ‘url-fetch’.
(perform-git-download): Likewise.
* guix/svn-download.scm (svn-fetch): Honor ‘download-method-enabled?’.
Pass ‘GUIX_DOWNLOAD_METHODS’ to #:env-vars.
(svn-multi-fetch): Likewise.

Change-Id: Ia3402e17f0303dfa964bdc761265efe8a1dd69ab
2024-03-09 18:55:50 +01:00
Ludovic Courtès
0e73f933b2
svn-download: Use ‘swh-download-directory-by-nar-hash’.
Fixes <https://issues.guix.gnu.org/43442>.

* guix/svn-download.scm (svn-fetch)[build]: Add
‘swh-download-directory-by-nar-hash’ call as a last resort.
Import (guix swh).
* guix/svn-download.scm (svn-multi-fetch)[build]: Likewise.

Change-Id: Ifcb9be1e9c2b05ce172c44e45dcf3a3ea6df8e76
2024-03-09 18:55:49 +01:00
Christopher Baines
c1f7156d55
svn-download: Handle the single file case when downloading a nar.
Delete the output so that download-nar doesn't error when trying to restore.

* guix/svn-download.scm (svn-multi-fetch): Delete the output if it exists
prior to calling download-nar.
2023-07-18 18:16:55 +02:00
Christopher Baines
32e48b8b43
svn-download: Use download-nar in svn-multi-fetch.
This should help if there are issues fetching from the source repository.

* guix/svn-download.scm (svn-multi-fetch): Use download-nar and adjust
accordingly.
2023-07-18 18:16:54 +02:00
Christopher Baines
a8d937d1bb
svn-download: Use download-nar.
This should help if there are issues fetching from the source repository.

* guix/svn-download.scm (svn-fetch): Use download-nar and adjust accordingly.
2023-07-18 18:16:54 +02:00
Ludovic Courtès
d036e1c544
svn-download: Default to non-recursive checkouts.
As it turns out, all packages that fetch code from Subversion expect it
to be non-recursive by default.  Clarify that.

Reported by Timothy Sample <samplet@ngyro.com>.

* guix/svn-download.scm (<svn-reference>)[recursive?]: Default to #f.
(<svn-multi-reference>)[recursive?]: Likewise.
2023-04-08 23:19:13 +02:00
Ludovic Courtès
257917d08b
svn-download: Add missing exports.
* guix/svn-download.scm: Export more <svn-reference> and
<svn-multi-reference> accessors.
2022-10-17 23:15:08 +02:00
Ludovic Courtès
30ac571eac
svn-download: Pass multi-fetch parameters through environment variables.
* guix/svn-download.scm (svn-multi-fetch)[build]: Check for environment
variables instead of splicing REF fields.
Pass #:script-name and #:env-vars to 'gexp->derivation'.
2022-10-17 23:15:07 +02:00
Ludovic Courtès
8599fccef8
svn-download: Pass parameters through environment variables.
This ensures a single "svn-download" script is created in the store for
all the origins that use 'svn-fetch'.

* guix/svn-download.scm (svn-fetch)[build]: Check for environment
variables instead of splicing REF fields.
Pass #:script-name and #:env-vars to 'gexp->derivation'.
2022-10-17 23:15:07 +02:00
Ludovic Courtès
b15e543d30
Merge branch 'master' into core-updates-frozen 2021-11-23 11:29:38 +01:00
Ricardo Wurmus
3e5749fc33
guix: Add download-multi-svn-to-store.
* guix/svn-download.scm (download-multi-svn-to-store): New procedure.
2021-11-17 10:59:54 +00:00
Ludovic Courtès
f95fc73248
build-system: Use 'sexp->gexp' for plain sexps.
This reduces memory allocations and list traversals during 'gexp->sexp',
'gexp-inputs', etc.

* guix/build-system/cargo.scm (cargo-build): Use 'sexp->gexp' for sexps
known to not contain file-like objects.  Change default #:phases to a
symbol.
* guix/build-system/cmake.scm (cmake-build, cmake-cross-build): Likewise.
* guix/build-system/copy.scm (copy-build): Likewise.
* guix/build-system/font.scm (font-build): Likewise.
* guix/build-system/glib-or-gtk.scm (glib-or-gtk-build): Likewise.
* guix/build-system/gnu.scm (%strip-flags, %strip-directories): New
variables.
(gnu-build): Use them.  Use 'sexp->gexp' where appropriate.
(gnu-cross-build): Likewise.
* guix/build-system/meson.scm (meson-build): Likewise.
* guix/build-system/perl.scm (perl-build): Likewise.
* guix/build-system/python.scm (python-build): Likewise.
* guix/build-system/ruby.scm (ruby-build): Likewise.
* guix/build-system/scons.scm (scons-build): Likewise.
* guix/build-system/texlive.scm (texlive-build): Likewise.
* guix/build-system/trivial.scm (trivial-build): Likewise.
* guix/build-system/waf.scm (waf-build): Likewise.
* guix/build-system/android-ndk.scm (android-ndk-build): Likewise.
* guix/build-system/ant.scm (ant-build): Likewise.
* guix/build-system/asdf.scm (asdf-build/source, asdf-build): Likewise.
* guix/build-system/chicken.scm (chicken-build): Likewise.
* guix/build-system/clojure.scm (clojure-build): Likewise.
(source->output-path, maybe-guile->guile): Remove.
* guix/build-system/dub.scm (dub-build): Likewise.
* guix/build-system/emacs.scm (emacs-build): Likewise.
* guix/build-system/go.scm (go-build): Likewise.
* guix/build-system/haskell.scm (haskell-build): Likewise.
* guix/build-system/julia.scm (julia-build): Likewise.
* guix/build-system/linux-module.scm (linux-module-build)
(linux-module-build-cross): Likewise.
* guix/build-system/maven.scm (maven-build): Likewise.
* guix/build-system/minify.scm (minify-build): Likewise.
* guix/build-system/node.scm (node-build): Likewise.
* guix/build-system/qt.scm (qt-build, qt-cross-build): Likewise.
* guix/build-system/r.scm (r-build): Likewise.
* guix/build-system/rakudo.scm (rakudo-build): Likewise.
* guix/build-system/renpy.scm (renpy-build): Likewise.
* guix/packages.scm (patch-and-repack): Use 'sexp->gexp' when SNIPPET is
a pair.
* guix/svn-download.scm (svn-multi-fetch): Use 'sexp->gexp' for
'svn-multi-reference-locations'.
2021-03-30 22:48:44 +02:00
Paul Garlick
735808b12c
guix: Fix download-svn-to-store.
* guix/svn-download.scm (download-svn-to-store): Add a subdirectory to
the download path.  The subdirectory is used as the target for the 'svn
export' command, avoiding a 'directory exists' error when attempting to
use the parent directory directly.
2020-09-14 14:24:29 +01:00
Julien Lepiller
277ba1d4f8
guix: svn: Allow dropping externals.
* guix/build/svn.scm (svn-fetch): Allow to ignore externals.
* guix/svn-download.scm (svn-reference, svn-multi-reference): Add
recursive? field.
2019-11-13 22:49:47 +01:00
Tobias Geerinckx-Rice
8a2b231782
Revert "guix: svn: Allow dropping externals."
This reverts commit 51395c84fd,
fixing <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=37896>.

Reported by <o.rojon@posteo.net>.
2019-10-24 01:08:23 +02:00
Julien Lepiller
51395c84fd
guix: svn: Allow dropping externals.
* guix/build/svn.scm (svn-fetch): Allow to ignore externals.
* guix/svn-download.scm (svn-reference, svn-multi-reference): Add
recursive? field.
2019-10-23 23:31:08 +02:00
Ludovic Courtès
67c2db17bc
download: Pass 'http_proxy' et al. to git, hg, etc.
This allows 'git-fetch' etc. origins to honor the proxy and locale of
the daemon.

* guix/bzr-download.scm (bzr-fetch): Pass #:leaked-env-vars to
'gexp->derivation'.
* guix/cvs-download.scm (cvs-fetch): Likewise.
* guix/git-download.scm (git-fetch): Likewise.
* guix/hg-download.scm (hg-fetch): Likewise.
* guix/svn-download.scm (svn-multi-fetch): Likewise.
2019-09-05 18:33:30 +02:00
Ricardo Wurmus
f97c9e4cfb
guix: Add svn-multi-reference.
* guix/svn-download.scm (<svn-multi-reference>): New record type.
(svn-multi-reference-url, svn-multi-reference-revision,
svn-multi-reference-locations, svn-multi-reference-user-name,
svn-multi-reference-password, svn-multi-fetch): New procedures.
2019-07-23 21:43:24 +02:00
Ricardo Wurmus
bef0db3769
guix: Add download-svn-to-store.
* guix/svn-download.scm (download-svn-to-store): New procedure.
2017-06-15 17:03:14 +02:00
Ludovic Courtès
e9b046fdda
download: Use 'with-imported-modules'.
* guix/cvs-download.scm (cvs-fetch): Use 'with-imported-modules' instead
of the #:modules argument of 'gexp->derivation'.
* guix/download.scm (url-fetch): Likewise.
* guix/git-download.scm (git-fetch): Likewise.
* guix/hg-download.scm (hg-fetch): Likewise.
* guix/svn-download.scm (svn-fetch): Likewise.
2016-07-12 22:47:08 +02:00
Ricardo Wurmus
140dd8f82c
guix: Support authentication when fetching from SVN.
* guix/svn-download.scm (<svn-reference>): Add fields for optional
credentials.
(svn-fetch): Pass credentials to build-side "svn-fetch".
* guix/build/svn.scm (svn-fetch): Pass optional credentials to svn
command.
2016-07-03 18:55:30 +02:00
Ludovic Courtès
1590e8a1dd packages: Refer to the native tools when handling sources and downloads.
* guix/packages.scm (patch-and-repack)[build]: Change most #$ to #+.
* guix/cvs-download.scm (cvs-fetch)[build]: Likewise.
* guix/download.scm (url-fetch)[builder]: Likewise.
* guix/git-download.scm (git-fetch)[build]: Likewise.
* guix/svn-download.scm (svn-fetch)[build]: Likewise.
2015-04-15 10:19:47 +02:00
Mark H Weaver
613467abc5 Remove outdated comment from svn-fetch.
* guix/svn-download.scm (svn-fetch): Remove outdated comment.
2015-02-04 00:14:37 -05:00
Ludovic Courtès
f220a83848 packages: Convert source derivations to monadic style.
* guix/packages.scm (origin->derivation): Take body from
  'package-source-derivation', and change it to monadic style.  Expect
  METHOD to a monadic procedure.
  (package-source-derivation): Define in terms of 'origin->derivation'.
* guix/download.scm (url-fetch): Remove 'store' argument.  Remove
  'guile-for-build' variable.  Turn into a monadic procedure.
* guix/git-download.scm (git-fetch): Likewise.
* guix/svn-download.scm (svn-fetch): Likewise.
* tests/builders.scm (url-fetch*): New procedure.
  Change tests to call 'url-fetch*' instead of 'url-fetch'.
* tests/packages.scm ("package-source-derivation, snippet"): Remove
  'store' parameter of 'fetch' and change it to use 'interned-file'
  instead of 'add-to-store'.
* gnu/packages/bootstrap.scm (bootstrap-origin)[boot]: Remove 'store'
  parameter.
2015-01-14 14:42:10 +01:00
Ludovic Courtès
e87f0591f3 monads: Move '%store-monad' and related procedures where they belong.
This turns (guix monads) into a generic module for monads, and moves the
store monad and related monadic procedures in their corresponding
module.

* guix/monads.scm (store-return, store-bind, %store-monad, store-lift,
  text-file, interned-file, package-file, package->derivation,
  package->cross-derivation, origin->derivation, imported-modules,
  compiled, modules, built-derivations, run-with-store): Move to...
* guix/store.scm (store-return, store-bind, %store-monad, store-lift,
  text-file, interned-file): ... here.
  (%guile-for-build): New variable.
  (run-with-store): Moved from monads.scm.  Remove default value for
  #:guile-for-build.
* guix/packages.scm (default-guile): Export.
  (set-guile-for-build): New procedure.
  (package-file, package->derivation, package->cross-derivation,
  origin->derivation): Moved from monads.scm.
* guix/derivations.scm (%guile-for-build): Remove.
  (imported-modules): Rename to...
  (%imported-modules): ... this.
  (compiled-modules): Rename to...
  (%compiled-modules): ... this.
  (built-derivations, imported-modules, compiled-modules): New
  procedures.
* gnu/services/avahi.scm, gnu/services/base.scm, gnu/services/dbus.scm,
  gnu/services/dmd.scm, gnu/services/networking.scm,
  gnu/services/ssh.scm, gnu/services/xorg.scm, gnu/system/install.scm,
  gnu/system/linux-initrd.scm, gnu/system/shadow.scm, guix/download.scm,
  guix/gexp.scm, guix/git-download.scm, guix/profiles.scm,
  guix/svn-download.scm, tests/monads.scm: Adjust imports accordingly.
* guix/monad-repl.scm (default-guile-derivation): New procedure.
  (store-monad-language, run-in-store): Use it.
* build-aux/hydra/gnu-system.scm (qemu-jobs): Add explicit
  'set-guile-for-build' call.
* guix/scripts/archive.scm (derivation-from-expression): Likewise.
* guix/scripts/build.scm (options/resolve-packages): Likewise.
* guix/scripts/environment.scm (guix-environment): Likewise.
* guix/scripts/system.scm (guix-system): Likewise.
* doc/guix.texi (The Store Monad): Adjust module names accordingly.
2015-01-14 13:34:52 +01:00
Ludovic Courtès
347e17b474 download: Allow use of substitutes.
See <https://bugs.gnu.org/18747> for the original report.

* guix/download.scm (url-fetch): Comment out #:local-build? argument.
* guix/git-download.scm (git-fetch): Likewise.
* guix/svn-download.scm (svn-fetch): Likewise.
2014-10-16 23:40:10 +02:00
Ludovic Courtès
bdb36958fb gnu: Split (gnu packages base), adding (gnu packages commencement).
* gnu/packages/base.scm (gnu-make-boot0, diffutils-boot0,
  findutils-boot0, %boot0-inputs, nix-system->gnu-triplet, boot-triplet,
  binutils-boot0, gcc-boot0, perl-boot0, linux-libre-headers-boot0,
  texinfo-boot0, %boot1-inputs, glibc-final-with-bootstrap-bash,
  cross-gcc-wrapper, static-bash-for-glibc, glibc-final,
  gcc-boot0-wrapped, %boot2-inputs, binutils-final, libstdc++,
  gcc-final, ld-wrapper-boot3, %boot3-inputs, bash-final, %boot4-inputs,
  guile-final, gnu-make-final, ld-wrapper, coreutils-final, grep-final,
  %boot5-inputs, %final-inputs, canonical-package, gcc-toolchain,
  gcc-toolchain-4.8, gcc-toolchain-4.9): Move to...
* gnu/packages/commencement.scm: ... here.  New file.
* gnu-system.am (GNU_SYSTEM_MODULES): Add it.
* build-aux/check-final-inputs-self-contained.scm: Adjust accordingly.
* gnu/packages/cross-base.scm: Likewise.
* gnu/packages/make-bootstrap.scm: Likewise.
* guix/build-system/cmake.scm (cmake-build): Likewise.
* guix/build-system/gnu.scm (standard-packages, gnu-build,
  gnu-cross-build): Likewise.
* guix/build-system/perl.scm (perl-build): Likewise.
* guix/build-system/python.scm (python-build): Likewise.
* guix/build-system/trivial.scm (guile-for-build): Likewise.
* guix/download.scm (url-fetch): Likewise.
* guix/gexp.scm (default-guile): Likewise.
* guix/git-download.scm (git-fetch): Likewise.
* guix/monads.scm (run-with-store): Likewise.
* guix/packages.scm (default-guile): Likewise.
* guix/scripts/package.scm (guix-package): Likewise.
* guix/scripts/refresh.scm: Likewise.
* guix/svn-download.scm (svn-fetch): Likewise.
* tests/builders.scm (%bootstrap-inputs, %bootstrap-search-paths):
  Likewise.
* tests/packages.scm ("GNU Make, bootstrap"): Likewise.
* tests/guix-package.sh: Likewise.
* gnu/services/base.scm: Use 'canonical-package' instead of xxx-final.
* gnu/services/xorg.scm: Likewise.
* gnu/system/vm.scm: Likewise.
* guix/scripts/pull.scm (guix-pull): Likewise.
2014-08-28 09:21:54 +02:00
Ludovic Courtès
bc67269487 svn-download: Rewrite using gexps.
* guix/svn-download.scm (subversion-package): New procedure.
  (svn-fetch): Use it.  Remove 'svn-for-build'.  Use a gexp and
  'gexp->derivation'.
2014-08-23 22:58:15 +02:00
Sree Harsha Totakura
b3acf3656e Add (guix svn-download).
* guix/svn-download.scm, guix/build/svn.scm: New files.
* Makefile.am (MODULES): Add them.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2014-03-27 00:21:44 +01:00