Commit Graph

19 Commits

Author SHA1 Message Date
Ludovic Courtès
015f17e8b9
derivations: Introduce 'read-derivation-from-file'.
This avoids the open/fstat/close syscalls upon a cache hit that we had
with the previous idiom:

  (call-with-input-file file read-derivation)

where caching happened in 'read-derivation' itself.

* guix/derivations.scm (%read-derivation): Rename to...
(read-derivation): ... this.
(read-derivation-from-file): New procedure.
(derivation-prerequisites, substitution-oracle)
(derivation-prerequisites-to-build):
(derivation-path->output-path, derivation-path->output-paths):
(derivation-path->base16-hash, map-derivation): Use
'read-derivation-from-file' instead of (call-with-input-file …
read-derivation).
* guix/grafts.scm (item->deriver): Likewise.
* guix/scripts/build.scm (log-url, options->things-to-build): Likewise.
* guix/scripts/graph.scm (file->derivation): Remove.
(derivation-dependencies, %derivation-node-type): Use
'read-derivation-from-file' instead.
* guix/scripts/offload.scm (guix-offload): Likewise.
* guix/scripts/perform-download.scm (guix-perform-download): Likewise.
* guix/scripts/publish.scm (load-derivation): Remove.
(narinfo-string): Use 'read-derivation-from-file'.
2017-06-12 17:53:51 +02:00
Ludovic Courtès
482fda2729
grafts: Do not pull derivation outputs not depended on.
Fixes <http://bugs.gnu.org/24886>.

Previously, the grafting derivation of, say, brdf-explorer would pull in
qt:doc even though brdf-explorer depends only on qt:out, not qt:doc.

* guix/grafts.scm (with-cache): Use 'vhash-assoc' and 'vhash-cons'
instead of 'vhash-assq' and 'vhash-consq'.
(cumulative-grafts): Pass #:outputs to 'graft-derivation/shallow'.  Use
OUTPUTS instead of (derivation-output-names drv).
(graft-derivation): Add #:outputs parameter; pass it to
'cumulative-grafts'.
* tests/grafts.scm (make-derivation-input): New variable.
("graft-derivation, replaced derivation has multiple outputs"): Make
sure P2:zzz is not part of the outputs of P3D.
("graft-derivation with #:outputs")
("graft-derivation, unused outputs not depended on"): New tests.
2017-01-25 11:04:25 +01:00
Ludovic Courtès
fd7d1235f1
grafts: Shallow grafting can be performed on a subset of the outputs.
* guix/grafts.scm (graft-derivation/shallow): Add #:outputs parameter.
[outputs]: Rename to...
[output-pairs]: ... this.  Adjust 'build-expression->derivation' call
accordingly.
2017-01-24 23:09:06 +01:00
Ludovic Courtès
0aeed5e310
grafts: Preserve the cache across recursive calls.
Before this commit, we'd lose the cache across recursive calls to
'cumulative-grafts', which isn't great performance-wise.

This bug was already present before
d38bc9a9f6.

* guix/grafts.scm (with-cache): In the miss case, call 'current-state'
after EXP has been evaluated.
2017-01-16 22:13:21 +01:00
Ludovic Courtès
90ad5c8836
grafts: Actually cache grafts during the derivation DAG traversal.
This fixes a regression introduced in
d38bc9a9f6 whereby the cache was
effectively disabled.

Reported by Thomas Danckaert <thomas.danckaert@gmail.com>.

* guix/grafts.scm (with-cache): In the cache miss case, wrap body in
'mbegin'.
2017-01-16 22:13:21 +01:00
Ludovic Courtès
d38bc9a9f6
grafts: Move caching to a new 'with-cache' macro.
* guix/grafts.scm (with-cache): New macro.
(cumulative-grafts)[return/cache]: Remove.
Use 'with-cache' instead.
2017-01-04 16:18:52 +01:00
Ludovic Courtès
3d47aa81ba
grafts: Apply the right grafts in the presence of multiple outputs.
Fixes <http://bugs.gnu.org/24712>.

* guix/grafts.scm (cumulative-grafts): Add grafts for all the outputs of
DRV.
* tests/grafts.scm ("graft-derivation, replaced derivation has multiple
outputs"): New test.
2016-10-17 23:59:03 +02:00
Ludovic Courtès
783ae212c2
grafts: Remove unused variables and confusing monad use.
* guix/grafts.scm (cumulative-grafts)[return/cache]: Use %STATE-MONAD,
not %STORE-MONAD.
Remove unused 'origins' variable and unnecessary inner 'cache'
variable.
2016-10-15 23:46:39 +02:00
Ludovic Courtès
b013c33f6f
grafts: 'graft-derivation' does now introduce grafts that shadow other grafts.
Partly fixes <http://bugs.gnu.org/24418>.

* guix/grafts.scm (cumulative-grafts)[graft-origin?]: New procedure.
[dependency-grafts]: Use it in new 'if' around recursive call.
* tests/grafts.scm ("graft-derivation, grafts are not shadowed"): New test.
2016-10-14 23:31:50 +02:00
Ludovic Courtès
1fd11c9259
grafts: Create only one grafted variant of each derivation.
Currently, with several grafts applicable to Inkscape, this makes:

  guix gc -R $(guix build inkscape -d) | wc -l

go from 2376 to 2266 (4.6%).

* guix/grafts.scm (cumulative-grafts): Pass 'graft-derivation/shallow'
the subset of GRAFTS that applies to DRV.
2016-05-25 23:33:56 +02:00
Ludovic Courtès
264fdedb40 grafts: Update the narinfo cache before building a derivation.
* guix/grafts.scm (references-oracle)[references*]: Add call to
'substitution-oracle'.
2016-03-14 23:34:33 +01:00
Ludovic Courtès
d4da602e4c grafts: Memoize intermediate results in 'cumulative-grafts'.
The time for:

  guix build inkscape -n --no-substitutes

goes down by 30% (in the presence of 3 replacements among all the
packages.)

* guix/grafts.scm (cumulative-grafts): Turn into a monadic procedure in
%STATE-MONAD.  Use the current state as a derivation-to-graft cache.
(graft-derivation): Call 'cumulative-grafts' within 'run-with-state'.
2016-03-05 00:19:10 +01:00
Ludovic Courtès
c90cb5c9d8 grafts: Use dependency information from substitutes when possible.
This avoids starting derivation builds just for the sake of knowing the
references of their outputs, thereby restoring the expected behavior of
--dry-run when substitutes are available.

* guix/grafts.scm (non-self-references): Remove 'store' parameter, and
add 'references'.  Use it.  Update caller.
(references-oracle): New variable.
(cumulative-grafts): Add 'references' parameter and use it.  Update
callers.
(graft-derivation): Remove 'build-derivations' call.  Add call to
'references-oracle'.
2016-03-05 00:19:10 +01:00
Ludovic Courtès
c22a1324e6 grafts: Graft recursively.
Fixes <http://bugs.gnu.org/22139>.

* guix/grafts.scm (graft-derivation): Rename to...
(graft-derivation/shallow): ... this.
(graft-origin-file-name, item->deriver, non-self-references)
(cumulative-grafts, graft-derivation): New procedures
* tests/grafts.scm ("graft-derivation, grafted item is a direct
dependency"): Clarify title.  Use 'grafted' instead of 'graft' to refer
to the grafted derivation.
("graft-derivation, grafted item is an indirect dependency")
("graft-derivation, no dependencies on grafted output"): New tests.
* guix/packages.scm (input-graft): Change to take a package instead of
an input.
(input-cross-graft): Likewise.
(fold-bag-dependencies): New procedure.
(bag-grafts): Rewrite in terms of 'fold-bag-dependencies'.
* tests/packages.scm ("package-derivation, indirect grafts"): Comment out.
* doc/guix.texi (Security Updates): Mention run-time dependencies and
recursive grafting.
2016-03-01 16:00:46 +01:00
Ludovic Courtès
f376dc3acb grafts: Consider all the outputs in the graft mapping.
Before that, outputs of a derivation could be left referring to the
ungrafted version of the derivation.

* guix/grafts.scm (graft-derivation)[outputs]: Change to a list of
name/file pairs.
* guix/grafts.scm (graft-derivation)[build]: Add 'old-outputs' variable
and use it when computing 'mapping'.  Use 'mapping' directly.
* tests/grafts.scm ("graft-derivation, multiple outputs"): New test.
2016-02-27 23:31:52 +01:00
Ludovic Courtès
cd05d38812 grafts: Slight simplification.
* guix/grafts.scm (graft-derivation)[output-names]: Use
'derivation-output-names'.
2016-02-27 23:31:52 +01:00
Ludovic Courtès
acb01e3746 grafts: Add record type printer.
* guix/grafts.scm (write-graft): New procedure.  Register it as a
printer for <graft>.
2016-02-26 23:35:29 +01:00
Ludovic Courtès
b0fef4d660 grafts: 'name' parameter of 'graft-derivation' is now optional.
* guix/grafts.scm (graft-derivation): Name 'name' a keyword parameter.
* guix/packages.scm (package-derivation, package-cross-derivation):
Adjust accordingly.
* tests/grafts.scm ("graft-derivation"): Likewise.
* tests/packages.scm ("package-derivation, indirect grafts"): Likewise.
2016-02-22 22:11:37 +01:00
Ludovic Courtès
7adf9b8469 derivations: Move grafts to (guix grafts).
* guix/derivations.scm (<graft>, graft-derivation, %graft?)
(set-grafting): Move to...
* guix/grafts.scm: ... here.  New file.
* guix/gexp.scm, guix/packages.scm, tests/packages.scm,
guix/scripts/build.scm: Use it.
* Makefile.am (MODULES): Add it.
(SCM_TESTS): Add tests/grafts.scm.
* tests/derivations.scm ("graft-derivation"): Move to...
* tests/grafts.scm: ... here.  New file.
2016-02-22 22:11:37 +01:00