This is a follow-up of 3cde5231aa that fixes the
challenge test.
When dealing with uncompressed NAR, the file size is false. Propagate it to
progress-reporter/file as it used to be the case.
* guix/scripts/challenge.scm (call-with-nar): Accept false size.
Possibly fixes <https://bugs.gnu.org/48045>.
Reported by Bone Baboon <bone.baboon@disroot.org>.
* guix/scripts/challenge.scm (call-with-nar): Rename second value
returned by 'http-fetch' to match reality. Change second argument to
'progress-reporter/file'.
This means there's a module for working with substitutes, rather than all the
code sitting in the script. The need for this can be seen with the weather and
challenge scripts, that now don't have to use code from the substitute script,
but can instead use the substitute module.
The separation here between the actual functionality of the substitute script
and the underlying functionality used both there and elsewhere should make
maintenance easier moving forward.
This commit just moves code, none of the code should have been changed
significantly.
* guix/scripts/substitute.scm (%narinfo-cache-directory, %narinfo-ttl,
%narinfo-negative-ttl, %narinfo-transient-error-ttl, %unreachable-hosts): Move
variables to guix/substitutes.scm.
(narinfo-cache-file, cached-narinfo, cache-narinfo!, narinfo-request,
read-to-eof, call-with-connection-error-handling, fetch-narinfos,
lookup-narinfos, lookup-narinfos/diverse): Move procedures to
guix/substitutes.scm.
* guix/substitutes.scm: New file.
* Makefile.am: Add it.
* guix/narinfo.scm: Remove redundant module.
* guix/scripts/challenge.scm: Change (guix scripts substitute) to (guix
substitutes).
* guix/scripts/weather.scm: Change (guix scripts substitute) to (guix
substitutes).
This separation between the code for dealing with narinfos from the code doing
that for a purpose should make things clearer, and better support components
other that the substitute script in using this code.
This is just moving the code around, no code should have been significantly
changed.
* guix/scripts/substitute.scm (<narinfo>): Move record type to (guix narinfo).
(fields->alist, narinfo-hash-algorithm+value, narinfo-hash->sha256,
narinfo-signature->canonical-sexp, narinfo-maker, read-narinfo,
narinfo-sha256, valid-narinfo?, write-narinfo, narinfo->string,
string->narinfo, equivalent-narinfo?, supported-compression?,
compresses-better?, narinfo-best-uri): Move procedures to (guix narinfo).
(%compression-methods): Move variable to (guix narinfo).
* guix/narinfo.scm: New file.
* Makefile.am (MODULES): Add it.
* po/guix/POTFILES.in: Add 'guix/narinfo.scm'.
* guix/serialization.scm (fold-archive): Call PROC with a
'directory-complete tag when done with a directory.
(restore-file): Handle it.
* guix/scripts/archive.scm (list-contents): Likewise.
* guix/scripts/challenge.scm (archive-contents): Likewise.
* tests/nar.scm ("write-file-tree + fold-archive"): Adjust accordingly.
* guix/scripts/challenge.scm (call-with-mismatches)[narinfo1]: When
LOCAL-HASH is true, call 'narinfo-hash->sha256' and use 'bytevector=?'
instead of 'string=?'.
* guix/scripts/challenge.scm (call-with-nar): New procedure.
(narinfo-contents): Express in terms of 'call-with-nar'.
(call-with-mismatches, report-differing-files/external): New
procedures.
(%diffoscope-command): New variable.
(%options): Support "diffoscope" and a string starting with "/".
* tests/challenge.scm (call-mismatch-test): New procedure.
("differing-files"): Rewrite in terms of 'call-mismatch-test'.
("call-with-mismatches"): New test.
* doc/guix.texi (Invoking guix challenge): Document it.
When a server publishes several URLs with different compression methods,
'guix substitute' can now choose the best one among the compression
methods that it supports.
* guix/scripts/substitute.scm (<narinfo>)[uri]: Replace with...
[uris]: ... this.
[compression]: Replace with...
[compressions]: ... this.
[file-size]: Replace with...
[file-sizes]: ... this.
[file-hash]: Replace with...
[file-hashes]: ... this.
(narinfo-maker): Adjust accordingly. Ensure 'file-sizes' and
'file-hashes' have the right length.
(assert-valid-signature, valid-narinfo?): Use the first element of
'narinfo-uris' in error messages.
(read-narinfo): Expect "URL", "Compression", "FileSize", and "FileHash"
to occur multiple times.
(display-narinfo-data): Call 'select-uri' to determine the file size.
(%compression-methods): New variable.
(supported-compression?, compresses-better?, select-uri): New
procedures.
(process-substitution): Call 'select-uri' to select the URI and
compression.
* guix/scripts/weather.scm (report-server-coverage): Account for all the
values returned by 'narinfo-file-sizes'.
* tests/substitute.scm ("substitute, narinfo with several URLs"): New
test.
* guix/scripts/challenge.scm (summarize-report-list): New procedure.
(guix-challenge): Use it.
* doc/guix.texi (Invoking guix challenge): Adjust command output in
example.
* guix/scripts/challenge.scm (summarize-report): Add #:verbose?
parameter.
[report-hashes]: New procedure. Use it.
Honor VERBOSE? in the 'match case.
(show-help, %options): Add '--verbose'.
(guix-challenge): Honor it.
This makes it easier to distinguish between matches, mismatches, and the
various cases of inconclusive reports.
* guix/scripts/challenge.scm (<discrepancy>): Rename to...
(<comparison-report>): ... this. Add 'result' field.
(comparison-report): New macro.
(comparison-report-predicate, comparison-report-mismatch?)
(comparison-report-match?)
(comparison-report-inconclusive?): New procedures.
(discrepancies): Rename to...
(compare-contents): ... this. Change to return a list of
<comparison-report>. Remove calls to 'warning'.
(summarize-discrepancy): Rename to...
(summarize-report): ... this. Adjust to <comparison-report>.
(guix-challenge): Likewise.
* tests/challenge.scm ("no discrepancies")
("one discrepancy"): Adjust to new API.
("inconclusive: no substitutes")
("inconclusive: no local build"): New tests.
Suggested by John Darrington <john@darrington.wattle.id.au>.
* guix/scripts/challenge.scm (guix-challenge): Exit with 2 when MISSING
is not empty.
* doc/guix.texi (Invoking guix challenge): Document it.
Reported by John Darrington.
* guix/scripts/challenge.scm (guix-challenge): Add an explicit 'exit'
call when ISSUES is empty.
* scripts/guix.in: Add comment about 'exit'.
* doc/guix.texi (Invoking guix challenge): Mention the behavior and exit
code.
* guix/scripts/substitute.scm (lookup-narinfos): Filter out #f values
from CACHED, such that the end result is exactly a list of narinfos,
not interspersed with #f.
* guix/scripts/challenge.scm (discrepancies): Assume REMOTE is a list of
narinfos.
