Commit Graph

156 Commits

Author SHA1 Message Date
Leo Famulari
cfe09c7c36
download: Replace faulty kernel mirror URL with canonical upstream.
This mirror was providing truncated files. See patch ticket #66923 for more
information about this change:

<https://issues.guix.gnu.org/issue/66923>

* guix/download.scm (%mirrors): Replace kernel mirror <uio.no> with <kernel.org>.

Change-Id: I189fc4eaa2bb3fa8b6db028aa837f0d59a460327
2023-07-23 22:26:00 -04:00
Maxim Cournoyer
bdaef69556
gnu-maintenance: Allow mirror URLs to fallback to the generic HTML updater.
* guix/gnu-maintenance.scm (http-url?): Extract from html-updatable-package?,
modify to return the HTTP URL, and support the mirror:// scheme.
(%disallowed-hosting-sites): New variable, extracted from
html-updatable-package.
(html-updatable-package?): Rewrite a mirror:// URL to an HTTP or HTTPS one.
* guix/download.scm (%mirrors): Update comment.
2023-08-26 11:38:56 -04:00
Maxim Cournoyer
91c8bd01f8
download: Add mirrors for Qt.
* guix/download.scm (%mirrors): Augment with qt mirrors.

Series-changes: 2
- Move authoritative mirror last, as it's too slow.
2023-08-22 12:32:19 -04:00
Tobias Geerinckx-Rice
4920d11ec1
download: Remove stale Yandex GNOME mirror.
* guix/download.scm (%mirrors): Remove mirror.yandex.ru from gnome.
2023-07-09 02:00:01 +02:00
Tobias Geerinckx-Rice
c2c13e3225
download: Remove broken X.org mirrors.
These are still at <https://www.x.org/wiki/Releases/Download>, but
then that page hasn't been updated since 2018.

* guix/download.scm (%mirrors): Remove mirror.us.leaseweb.net and
x.cs.pu.edu.tw.
2023-07-09 02:00:01 +02:00
Tobias Geerinckx-Rice
158a218dea
download: Remove obsolete kernel.org mirrors.
* guix/download.scm (%mirrors): Remove kernel.osuosl.org and
mirror.linux.org.au.
2023-07-09 02:00:11 +02:00
Tobias Geerinckx-Rice
75c7f790b0
download: Remove obsolete Savannah subdirectory.
* guix/download.scm (%mirrors): Remove ‘releases-noredirect’ URL.
2023-07-09 02:00:02 +02:00
Christopher Baines
7d0ebe040d
download: Add bordeaux.guix.gnu.org as a content addressed mirror.
bordeaux.guix.gnu.org now provides access to some files by hash. This is done
through the nar-herder finding a nar produced by a fixed output derivation for
the requested content, and then providing the content stored inside that nar.

I've put this new entry at the start of the list, as I think it's more likely
to have content than the others. Because bordeaux.guix.gnu.org stores nars
indefinitely, my suspicion is that it's going to be able to fulfil more
requests than ci.guix.gnu.org, which relies on the file requested being in the
store (so the now frequent garbage collection is going to limit the files
available).

* guix/download.scm (%content-addressed-mirrors): Add bordeaux.guix.gnu.org.

Signed-off-by: Christopher Baines <mail@cbaines.net>
2023-02-11 20:34:22 +00:00
Tobias Geerinckx-Rice
be97c7ed11
download: Remove dead SourceForge mirrors.
* guix/download.scm (%mirrors): Remove dl.sourceforge.net subdomains
that no longer resolve.
2023-01-22 01:01:01 +01:00
Tobias Geerinckx-Rice
0081807e16
guix: Replace defective freemirror.org.
This ‘mirror’ is particularly problematic because it's the only one checked
by ‘guix refresh’.

* guix/gnu-maintenance.scm (%savannah-base): Substitute freedif.org.
* guix/download.scm (%mirrors): Likewise.

Reported by sash-kan on #guix.
2022-11-13 01:00:07 +01:00
Maxime Devos
60474b5291
download: Add a kernel.org mirror.
Add the mirror from (guix gnu-maintenance) to make the simplified
linux.org updater (of a later commit) work.

* download.scm (%mirrors)[kernel.org]: Add mirrors.edge.kernel.org mirror.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-09-26 23:29:36 +02:00
Maxime Devos
6c16de72de
download: Switch savannah mirrors to HTTPS URLs.
The URI scheme used for nongnu.freemirror.org needs to be consistent
between (guix download) and (guix gnu-maintenance) to make the simplified
savannah-updater (of a later commit) work.  While we're at it, switch the
other mirrors to https as well.

http://download.savannah.gnu.org/releases-noredirect/ is left unmodified
because it 404s

* download.scm (%mirrors)[savannah]: Switch from http to https where possible.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-09-26 23:29:36 +02:00
Tobias Geerinckx-Rice
75bc96ed8e
download: Refresh KDE mirror list.
* guix/download.scm (%mirrors)[kde]: Update list, using HTTPS wherever
possible.
2022-07-24 02:00:00 +02:00
Maxim Cournoyer
8cdad3484c
download: Add a mirror for CTAN.
* guix/download.scm (%mirrors)[ctan]: New entry.
2022-05-12 12:45:49 -04:00
Efraim Flashner
67213a2759
download: Use https for main KDE source.
* guix/download.scm (%mirrors)[kde]: Use https URIs for main KDE mirrors.
* tests/lint.scm ("mirror-url: kde suggestion"): New test.
2022-03-29 14:43:14 +03:00
Ludovic Courtès
bd61d62182
download: "GUIX_DOWNLOAD_FALLBACK_TEST=none" disables fallback mechanisms.
* guix/download.scm (url-fetch*): Handle (%download-fallback-test) set
to 'none.
2021-10-15 23:16:28 +02:00
Ludovic Courtès
c4a7aa82e2
download: Add parameter to test download fallback mechanisms.
This allows you to run, say:

  GUIX_DOWNLOAD_FALLBACK_TEST=disarchive-mirrors guix build -S r-ebimage --check

or:

  GUIX_DOWNLOAD_FALLBACK_TEST=content-addressed-mirrors ./pre-inst-env guix build -S r-ebimage --check

to check whether these fallback mechanisms work as expected.

* guix/download.scm (%no-mirrors-file, %no-disarchive-mirrors-file)
(%download-fallback-test): New variables.
(url-fetch*): Honor (%download-fallback-test).
2021-10-14 15:44:52 +02:00
Ludovic Courtès
f9a506aa6a
download: Add disarchive.guix.gnu.org.
* guix/download.scm (%disarchive-mirrors): Add disarchive.guix.gnu.org.
2021-10-14 15:44:51 +02:00
Ludovic Courtès
3cb5ae8577
download: Disarchive mirrors can be URL-returning procedures.
As discussed at <https://issues.guix.gnu.org/47336#16>.

* guix/build/download.scm (url-fetch)[disarchive-uris]: Accept MIRROR as
a procedure.
* guix/download.scm (%disarchive-mirrors): Add comment.  This change can
only be made once a 'guix perform-download' that understands procedures
is widely deployed.
2021-09-14 11:50:58 +02:00
Ludovic Courtès
67da646087
download: Remove obsolete workaround.
* guix/download.scm (%content-addressed-mirrors): Use (guix base16)
unconditionally.
2021-09-14 11:50:58 +02:00
Leo Famulari
189003c83e
download: Remove defunct GNOME mirrors.
* guix/download.scm (%mirrors): Remove defunct GNOME mirrors, and try
the canonical site first.
2021-07-27 12:34:57 -04:00
Ludovic Courtès
bc4d81d267
lint: archival: Lookup content in Disarchive database.
* guix/lint.scm (lookup-disarchive-spec): New procedure.
(check-archival): When 'lookup-content' returns #f, call
'lookup-disarchive-spec'.  Call 'lookup-directory' on the result of
'lookup-directory'.
* guix/download.scm (%disarchive-mirrors): Make public.
* tests/lint.scm ("archival: missing content"): Set
'%disarchive-mirrors'.
("archival: content unavailable but disarchive available"): New test.
2021-05-22 23:13:11 +02:00
Timothy Sample
fbc2a52a32
download: Use Disarchive as a last resort.
This is a fixed version of 66b14dccdd,
which was reverted in e74250c3c5.

* guix/download.scm (%disarchive-mirrors): New variable.
(%disarchive-mirror-file): New variable.
(built-in-download): Add 'disarchive-mirrors' keyword argument and
pass its value along to the 'builtin:download' derivation.
(url-fetch): Pass '%disarchive-mirror-file' to 'built-in-download'.
* guix/scripts/perform-download.scm (perform-download): Read
Disarchive mirrors from the environment and pass them to
'url-fetch'.
* guix/build/download.scm (disarchive-fetch/any): New procedure.
(url-fetch): Add 'disarchive-mirrors' keyword argument, use it to
make a list of URIs, and use the new procedure to fetch the file if
all other methods fail.
* build-aux/build-self.scm (build-program)[select?]: Exclude '(guix
build download)'.
* guix/self.scm (compiled-guix)[*core-modules*]: Add 'guile-json' to
the list of extensions.
2021-04-29 11:24:48 -04:00
Timothy Sample
e74250c3c5
Revert "download: Use Disarchive as a last resort."
This reverts commit 66b14dccdd, which broke
'guix pull'.
2021-04-28 00:24:28 -04:00
Timothy Sample
66b14dccdd
download: Use Disarchive as a last resort.
* guix/download.scm (%disarchive-mirrors): New variable.
(%disarchive-mirror-file): New variable.
(built-in-download): Add 'disarchive-mirrors' keyword argument and
pass its value along to the 'builtin:download' derivation.
(url-fetch): Pass '%disarchive-mirror-file' to 'built-in-download'.
* guix/scripts/perform-download.scm (perform-download): Read
Disarchive mirrors from the environment and pass them to
'url-fetch'.
* guix/build/download.scm (disarchive-fetch/any): New procedure.
(url-fetch): Add 'disarchive-mirrors' keyword argument, use it to
make a list of URIs, and use the new procedure to fetch the file if
all other methods fail.
2021-04-27 21:27:02 -04:00
Ludovic Courtès
f7008ca713
download: Autoload (guix build download).
* guix/download.scm: Autoload (guix build download).
(url-fetch): Rename to...
(url-fetch*): ... this, locally, to allow for #:autoload.
* guix/status.scm: Autoload (guix build download).
2021-03-19 13:23:27 +01:00
Ludovic Courtès
9d349afaa1
download: 'url-fetch/tarbomb' and 'url-fetch/zipbomb' refer to native tools.
* guix/download.scm (url-fetch/tarbomb, url-fetch/zipbomb): Use
'ungexp-native' instead of 'ungexp' when referring to the decompression
tools.
2021-03-05 12:49:28 +01:00
Marius Bakke
4de58a4b81
download: Update ImageMagick mirror list.
* guix/download.scm (%mirrors): Remove defunct ImageMagick mirrors, update
URLs for directory structure changes (/releases instead of /legacy).  Move
official site last.
2020-12-13 01:05:57 +01:00
Ludovic Courtès
d154462bd8
download: Update docstring.
* guix/download.scm (url-fetch): Update docstring to match what's in the
manual.
2020-10-20 16:30:15 +02:00
Ludovic Courtès
36a5efd967
download: Add Savannah mirror.
* guix/download.scm (%mirrors): Add nongnu.freemirror.org.
2020-07-20 23:25:10 +02:00
Tobias Geerinckx-Rice
5733ba7dd8
download: Replace misconfigured Apache mirror.
E.g. <http://apache.belnet.be/jakarta/oro/jakarta-oro-2.0.8.tar.gz>
redirects to a bogus 200 HTML page.

* guix/download.scm (%mirrors): Replace apache.belnet.be with
ftp.nluug.nl.
2020-07-05 08:23:35 +02:00
Tobias Geerinckx-Rice
c9ea4eecf4
download: Remove usa-mirror.go-parts.com mirror.
It incorrectly redirects 404s (at least) to the commercial home page.

* guix/download.scm (%mirrors): Remove usa-mirror.go-parts.com URLs.
2020-06-15 14:26:44 +02:00
Efraim Flashner
205a0ecd5b
download: Add more apache mirrors.
* guix/download.scm (%mirrors): Add more apache mirrors.
2020-06-12 18:28:51 +03:00
Diego Nicola Barbato
c1d81df93d
download: Use correct system and guile in 'url-fetch/tarbomb' and 'url-fetch/zipbomb'.
Fixes <https://bugs.gnu.org/40115>.

Previously the result of `guix build -s $system $package' would depend on the
system Guix was built for if $package or one of its dependencies used
'url-fetch/tarbomb' or 'url-fetch/zipbomb' as the origin method of its
source (e.g. `guix build -s i686-linux ffmpeg' on i686-linux would build a
different derivation than on x86_64-linux).

This patch fixes this by explicitly passing the correct system and guile to
'gexp->derivation'.

* guix/download.scm (url-fetch/tarbomb): Pass #:system system and
  #:guile-for-build guile to 'gexp->derivation', where guile is the derivation
  of guile for system.
  (url-fetch/zipbomb): Likewise.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2020-04-08 19:35:34 +02:00
Tobias Geerinckx-Rice
df0bb509c2
download: Remove misbehaving kvin.lv mirror.
It issues bogus redirections instead of returning 404.

* guix/download.scm (%mirrors): Remove kvin.lv from CPAN.
2020-03-09 16:01:17 +01:00
Mark H Weaver
09f9167cd4
download: Remove ramses.wh2.tu-dresden.de kernel mirror.
* guix/download.scm (%mirrors): Remove ramses.wh2.tu-dresden.de, which seems
to no longer work.
2019-12-18 23:07:38 -05:00
Ludovic Courtès
64de896a71
Merge branch 'master' into core-updates 2019-07-17 15:51:10 +02:00
Ludovic Courtès
02237f13ee
download: Use the new 'derivation' calling convention.
* guix/download.scm (built-in-download): Pass MIRRORS and
CONTENT-ADDRESSED-MIRRORS as #:sources, not #:inputs.
2019-07-15 10:01:05 +02:00
Marius Bakke
4fb9165515
Merge branch 'staging' into core-updates 2019-07-02 20:18:16 +02:00
Tobias Geerinckx-Rice
87a90486ca
download: Remove hydra from content-addressed URL list
* guix/build/download-nar.scm (urls-for-item): Remove
mirror.hydra.gnu.org.
* guix/download.scm (%content-addressed-mirrors): Likewise.
2019-06-29 14:31:23 +02:00
Ludovic Courtès
267966f911
download: Add 'url-fetch/executable'.
* guix/download.scm (built-in-download): Add #:executable? parameter.
Pass #:recursive? to 'raw-derivation' and add "executable" to the #:env-vars
alist when EXECUTABLE? is true.
(url-fetch): Add #:executable? and pass it to 'built-in-download'.
(url-fetch/executable): New procedure.
2019-06-14 15:05:43 +02:00
Ludovic Courtès
883dc11c3a
download: Update list of content-addressed mirrors.
* guix/download.scm (%content-addressed-mirrors): Change
"berlin.guixsd.org" to "ci.guix.gnu.org" and move it first.
2019-06-10 22:42:23 +02:00
Ludovic Courtès
3961edf230
store: Memoize 'built-in-builders' call directly in <store-connection>.
The caching strategy introduced in
40cc850aeb was ineffective since we
regularly start from an empty object cache.  For example, "guix build
inkscape -n" would make 241 'built-in-builders' RPCs.

* guix/store.scm (<store-connection>)[built-in-builders]: New field.
(open-connection): Adjust '%make-store-connection' call accordingly.
(port->connection): Likewise.
(built-in-builders): Rename to...
(%built-in-builders):  ... this.
(built-in-builders): New procedure.
* guix/download.scm (built-in-builders*): Remove 'mcached' call.
2019-04-16 17:30:21 +02:00
Guy Fleury Iteriteka
bc4cea6f0e
download: Add MATE mirrors.
* guix/download.scm (%mirrors): Add mirrors for the MATE Desktop.
2019-03-15 13:05:08 -04:00
Ludovic Courtès
40cc850aeb
download: 'built-in-builders*' relies on the functional cache.
The previous caching strategy, which used STORE as an 'eq?' key, would
no longer work when the functional cache is used because subsequent
store values are not 'eq?'.

* guix/download.scm (built-in-builders*): Rewrite in terms of
'mcached'.
2018-12-19 23:52:25 +01:00
Ludovic Courtès
13bcc6b45f
download: Add berlin.guixsd.org as a content-addressed mirror.
* guix/download.scm (%content-addressed-mirrors)[guix-publish]: New
procedure.
Use it for "mirror.hydra.gnu.org" and add "berlin.guixsd.org" too.
2018-11-28 22:25:34 +01:00
Ludovic Courtès
a52ae1b662
download: Make (guix base16) a soft dependency.
Fixes <https://bugs.gnu.org/33542>.
Reported by Mark H Weaver <mhw@netris.org>.

* guix/download.scm (%content-addressed-mirrors): Autoload (guix base16).
2018-11-28 22:25:34 +01:00
Ludovic Courtès
89f1fee8e7
download: Access content-addressed mirrors over HTTPS.
Bug <http://bugs.gnu.org/22774> is no longer relevant now that we use
"builtin:download" exclusively.

* guix/download.scm (%content-addressed-mirrors): Use "https", not
"http".
2018-11-14 21:34:08 +01:00
Ludovic Courtès
ee2cfdfe86
download: Add Software Heritage as a content-addressed mirror.
* guix/download.scm (%content-addressed-mirrors): Add Software
Heritage.
2018-11-14 21:34:07 +01:00
Ricardo Wurmus
b347317ae6
Merge branch 'master' into core-updates 2018-05-29 15:41:06 +02:00