Commit Graph

136 Commits

Author SHA1 Message Date
Ludovic Courtès
9b07130129
list-runtime-roots: Ignore ESRCH while reading from /proc.
Fixes <https://bugs.gnu.org/29368>.
Reported by Martin Castillo <castilma@uni-bremen.de>.

* nix/scripts/list-runtime-roots.in (referenced-files): Ignore ESRCH.
2017-11-26 15:58:33 +01:00
Ludovic Courtès
856b68bda9
list-runtime-roots: Ignore PIDs we cannot access.
This allows running as non-root.
Fixes a regression introduced in b8f59cdc20.

* nix/scripts/list-runtime-roots.in (referenced-files): Handle EACCES in
addition to ENOENT.
2017-11-12 23:53:12 +01:00
Ludovic Courtès
fb17a89912
list-runtime-roots: Canonicalize store items.
Looking at 'addAdditionalRoots' in libstore/gc.cc, it looks like it
should always have been this way.  In practice it probably doesn't make
much of a difference.

* nix/scripts/list-runtime-roots.in (canonicalize-store-item): New
procedure.
<top level>: Use it.
2017-11-12 22:42:17 +01:00
Ludovic Courtès
b8f59cdc20
list-runtime-roots: Do not use 'lsof'.
This makes things a bit faster (0.8s instead of 1.4s on my laptop).

* nix/scripts/list-runtime-roots.in (lsof-roots): Remove.
(proc-fd-roots): Return the empty list when 'scandir' returns #f.
(referenced-files): New procedure.
Use it at the top level.
2017-11-12 22:42:17 +01:00
Andy Wingo
ed464f48ef
nix: Remove unused function.
* nix/libutil/util.hh:
* nix/libutil/util.cc (decodeOctalEscaped): Remove unused and buggy function.
2017-08-18 11:31:28 +02:00
Jörg Thalheim
842e0e439a
Simplify remouting with MS_PRIVATE in sandbox build
also fix race condition if mounts are added after mountinfo is read.
2017-08-18 11:31:17 +02:00
Mark H Weaver
a1aa5dabaa
daemon: Ensure the child stack is aligned on a 16-byte boundary.
* nix/libstore/build.cc (DerivationGoal::startBuilder): When calling 'clone',
ensure that the stack is aligned on a 16-byte boundary.
2017-08-07 16:41:03 -04:00
Efraim Flashner
31ed845b70
daemon: On aarch64, use increments of 16 on the stack.
* nix/libstore/build.cc (DerivationGoal::startBuilder): When on aarch64,
when calling clone(), increment the stack by 16.
2017-08-05 22:38:48 +03:00
Mark H Weaver
7656739771
Revert "daemon: Ensure proper alignment on the stack."
This reverts commit ab4ccc8fcf.
2017-08-05 01:57:25 -04:00
Jan Nieuwenhuizen
2e009ae7cd
daemon: Flush the sink upon 'exportPath' errors.
Prior to this change, errors such as wrong permissions on
/etc/guix/signing-key.sec would give:

  guix-daemon: nix/libutil/serialise.cc:15: virtual nix::BufferedSink::~BufferedSink(): Assertion `!bufPos' failed.

This patch correctly propagates the error to the client and thus changes
that to:

  error: build failed: file `/etc/guix/signing-key.sec' should be secret (inaccessible to everybody else)!

* nix/nix-daemon/nix-daemon.cc (performOp): Wrap 'exportPath' call in
'try' block.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2017-07-17 15:13:41 +02:00
Ludovic Courtès
5c82722cac
daemon: Allow compilation with GCC 4.9.
Reported by Ricardo Wurmus.

* nix/nix-daemon/nix-daemon.cc (acceptConnection): Remove intermediate
'inaddr' variables that relied on implicit casts.  This fixes
compilation with GCC 4.9.
2017-07-10 11:55:54 +02:00
Ludovic Courtès
6efb578a14
daemon: Set TCP_NODELAY and TCP_QUICKACK on remote sockets.
* nix/nix-daemon/nix-daemon.cc (acceptConnection): Set TCP_NODELAY and
TCP_QUICKACK on REMOTE.
2017-06-22 10:59:07 +02:00
Ludovic Courtès
1071f781d9
daemon: '--listen' can be passed several times, can specify TCP endpoints.
* nix/nix-daemon/guix-daemon.cc (DEFAULT_GUIX_PORT): New macro.
(listen_options): New variable.
(parse_opt): Push back '--listen' options to LISTEN_OPTIONS.
(open_unix_domain_socket, open_inet_socket)
(listening_sockets): New functions.
(main): Use it.  Pass SOCKETS to 'run'.
* nix/nix-daemon/nix-daemon.cc (matchUser): Remove.
(SD_LISTEN_FDS_START): Remove.
(acceptConnection): New function.
(daemonLoop): Rewrite to take a vector of file descriptors, to select(2)
on them, and to call 'acceptConnection'.
(run): Change to take a vector of file descriptors.
* tests/guix-daemon.sh: Add test.
2017-06-22 10:59:07 +02:00
Ludovic Courtès
2ca9f51ec8
daemon: Add '--timeout' and '--max-silent-time'.
* nix/nix-daemon/guix-daemon.cc (GUIX_OPT_TIMEOUT)
(GUIX_OPT_MAX_SILENT_TIME): New macros.
* nix/nix-daemon/guix-daemon.cc (options): Add '--timeout' and
'--max-silent-time'.
(parse_opt): Honor them.
* tests/guix-daemon.sh: Add test.
* doc/guix.texi (Invoking guix-daemon): Document the options.
(Common Build Options): Properly describe default
timeout/max-silent-time value.  Add cross-ref to "Invoking
guix-daemon".
2017-06-04 00:35:55 +02:00
Ludovic Courtès
f0a0b58bfa
daemon: Report hash mismatches in a cleaner way.
Suggested by Hartmut Goebel.

* nix/libstore/build.cc (DerivationGoal::registerOutputs): Report hash
mismatches on 3 lines for clarity.
(SubstitutionGoal::finished): Likewise.
2017-05-30 18:08:37 +02:00
Ludovic Courtès
ac9c1dc7bd
list-runtime-roots: Bufferize the lsof pipe.
* nix/scripts/list-runtime-roots.in (lsof-roots): Add 'setvbuf' call for
PIPE.
2017-03-08 23:42:53 +01:00
Leo Famulari
613d0895b9
build: Don't embed absolute paths in .service and .conf service files.
Otherwise, users will be stuck running an old copy of guix and the guix-daemon
if they copy the service files instead of symlinking them.

* etc/guix-daemon.conf.in, etc/guix-daemon.service.in, etc/guix-publish.conf.in,
etc/guix-publish.service.in: Expand @localstatedir@ instead of @bindir@.
* nix/local.mk (etc/guix-%.service, etc/guix-%.conf): Use @localstatedir@
instead of @bindir@.
2017-03-06 13:25:51 -05:00
Efraim Flashner
ab4ccc8fcf
daemon: Ensure proper alignment on the stack.
* nix/libstore/build.cc (startBuilder): When calling 'clone', increase
the step to 16 and ensure it aligns properly on the stack.
2017-02-20 13:09:03 +02:00
Ludovic Courtès
328f7cbe43
daemon: Define 'NIX_STORE' before invoking 'guix perform-download'.
Reported by rohit yadav <rohityadav@utexas.edu>
at <https://lists.gnu.org/archive/html/guix-devel/2017-02/msg00191.html>.

This fixes a regression whereby 'guix perform-download' would always see
NIX_STORE as unset and thus use "/gnu/store", leading it to miscompute
the hydra.gnu.org content-addressed URLs when the store file name is not
"/gnu/store".

* nix/libstore/builtins.cc (builtinDownload): Add 'setenv' call for
'NIX_STORE'.
2017-02-07 14:51:17 +01:00
Ludovic Courtès
deac976d3d
daemon: Client settings no longer override daemon settings.
Fixes <http://bugs.gnu.org/20217>.

* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x161.
* nix/nix-daemon/nix-daemon.cc (performOp): "build-max-jobs",
"build-max-silent-time", and "build-cores" are no longer read upfront;
instead, read them from the key/value list at the end.
* nix/nix-daemon/guix-daemon.cc (main): Explicitly set
'settings.maxBuildJobs'.
* guix/store.scm (%protocol-version): Bump to #x161.
(set-build-options): #:max-build-jobs, #:max-silent-time, and
 #:build-cores now default to #f.  Adjust handshake to new protocol.
* tests/store.scm ("build-cores"): New test.
* tests/guix-daemon.sh: Add test for default "build-cores" value.
2017-01-15 15:43:22 +01:00
Ludovic Courtès
9b5364a3af
daemon: Allow check builds of 'builtin:download' derivations.
Fixes <http://bugs.gnu.org/25089>.
Reported by Leo Famulari <leo@famulari.name>.

* nix/libstore/build.cc (DerivationGoal::runChild): In the 'isBuiltin'
case, check whether DRV's output is in 'redirectedOutputs', and pass an
'output' argument to the built-in builder.
(DerivationGoal::addHashRewrite): Add 'printMsg' call.
* nix/libstore/builtins.hh (derivationBuilder): Add 'output' parameter.
* nix/libstore/builtins.cc (builtinDownload): Likewise.
Add OUTPUT to ARGV.
* guix/scripts/perform-download.scm (perform-download): Add 'output'
parameter.
(guix-perform-download): Adjust 'match' clauses accordingly.
* tests/derivations.scm ("'download' built-in builder, check mode"): New
test.
2017-01-11 17:06:31 +01:00
Ludovic Courtès
8ecc3c6c44
daemon: Allow fixed-output derivation builds with TMPDIR set.
Fixes <http://bugs.gnu.org/25242>.
Reported by Leo Famulari <leo@famulari.name>.
The regression was introduced in 94d92c7796.

* nix/libstore/build.cc (DerivationGoal::startBuilder): Set 'useChroot'
as a function 'of isBuiltin(drv)'.
(DerivationGoal::runChild): Use 'useChroot' instead of
'useChroot && !isBuiltin(drv)'.
2016-12-29 18:22:49 +01:00
Ludovic Courtès
b20644ed18
build: Delete all the .service and .conf files upon 'make clean'.
* nix/local.mk (CLEANFILES): Define to add $(nodist_systemdservice_DATA)
and $(nodist_upstartjob_DATA).
2016-12-20 15:14:40 +01:00
Ludovic Courtès
5a5fc61f07
build: Fix .service and .conf targets for VPATH builds.
* nix/local.mk (etc/guix-%.service, etc/guix-%.conf): Use "$<", not
"$(srcdir)/$<".
2016-12-18 17:45:44 +01:00
Ludovic Courtès
fa54f44bb7
build: Add 'guix-publish.*.in' to the distribution.
This is a followup to 332d7903f5.

* nix/local.mk (EXTRA_DIST): Add 'guix-service.publish.in' and
'guix-publish.conf.in'.
2016-12-18 17:45:44 +01:00
Hartmut Goebel
2608e40988
daemon: Set ownership of kept build directories to the calling user.
Fixes <http://bugs.gnu.org/15890>.

* nix/libstore/globals.hh (Settings) Add clientUid and clientGid.
* nix/nix-daemon/nix-daemon.cc (daemonLoop] Store UID and GID of the
  caller in settings.
* nix/libstore/build.cc (_chown): New function.
  (DerivationGoal::deleteTmpDir): Use it, change ownership of build
  directory if it is kept and the new owner is not root.
2016-12-09 15:46:09 +01:00
Ludovic Courtès
638ccde1fb
daemon: Fix invalid Boost format string.
* nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Use %3%
instead of %m, the latter being an invalid Boost format specifier.
2016-12-05 18:18:10 +01:00
Ludovic Courtès
9a8f9f84cc
daemon: Buffer data sent to clients by the 'export-path' RPC.
Before that we'd have STDERR_WRITE round trips for very small amounts of
data, ranging from a few bytes for the metadata of nars to the size of
one file being exported.

With this change, something like:

  guix archive --export /gnu/store/5rrsbaghh5ix1vjcicsl60gsxilhjnf2-coreutils-8.25 | dd of=/dev/null

reports a throughput of 35 MB/s instead of 25 MB/s before.

* nix/nix-daemon/nix-daemon.cc (TunnelSink): Inherit from 'BufferedSink'
rather than 'Sink'.  Rename 'operator ()' to 'write'.
(performOp) <wopExportPath>: Add 'sink.flush' call.
2016-12-01 23:35:10 +01:00
Hartmut Goebel
332d7903f5
Add system start-up files for "guix publish".
* .gitignore: add etc/guix-publish.conf and /etc/guix-publish.service.
* etc/guix-publish.conf.in: New file.
* etc/guix-publish.service.in: New file.
* nix/local.mk (etc/guix-%.service, etc/guix-%.conf): Generalized former
  build-rules for by using patterns.
  (nodist_systemdservice_DATA): Add etc/guix-publish.service, update
  comment.
  (nodist_upstartjob_DATA): Add etc/guix-publish.conf, update comment.
* doc/guix.texi (Invoking guix publish): Add description for enabling
  "guix publish" on host distros using the new files.
2016-11-24 09:23:11 +01:00
Ludovic Courtès
f9aefa2d5f
daemon: Add 'built-in-builders' RPC.
* nix/libstore/builtins.cc (builtinBuilderNames): New function.
* nix/libstore/builtins.hh (builtinBuilderNames): New declaration.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x160.
(WorkerOp)[wopBuiltinBuilders]: New value.
* nix/nix-daemon/nix-daemon.cc (performOp): Handle it.
* guix/store.scm (operation-id)[built-in-builders]: New value.
* guix/store.scm (read-arg): Add 'string-list'.
(built-in-builders): New procedure.
* tests/derivations.scm ("built-in-builders"): New test.
2016-11-16 18:19:54 +01:00
Ludovic Courtès
94d92c7796
daemon: Add "builtin:download" derivation builder.
This ensures that 1) the derivation doesn't change when Guix changes;
2) the derivation closure doesn't contain Guix and its dependencies; 3)
we don't have to rely on ugly chroot hacks.

Adapted from Nix commit 0a2bee307b20411f5b0dda0c662b1f9bb9e0e131.

* nix/libstore/build.cc (DerivationGoal::runChild): Add special case for
'isBuiltin(drv)'.  Disable chroot when 'isBuiltin(drv)'.
* nix/libstore/builtins.cc, nix/libstore/builtins.hh,
nix/scripts/download.in, guix/scripts/perform-download.scm: New files.
* guix/ui.scm (show-guix-help)[internal?]: Add 'perform-download'.
* nix/local.mk (libstore_a_SOURCES): Add builtins.cc.
(libstore_headers): Add builtins.hh.
(nodist_pkglibexec_SCRIPTS): Add 'scripts/download'.
* config-daemon.ac: Emit 'scripts/download'.
* Makefile.am (MODULES): Add 'guix/scripts/perform-download.scm'.
* tests/derivations.scm ("unknown built-in builder")
("'download' built-in builder")
("'download' built-in builder, invalid hash")
("'download' built-in builder, not found")
("'download' built-in builder, not fixed-output"): New tests.

Co-authored-by: Eelco Dolstra <eelco.dolstra@logicblox.com>
2016-11-16 18:19:47 +01:00
Ludovic Courtès
12b6c951cf
daemon: Do not error out when deduplication fails due to ENOSPC.
This solves a problem whereby if /gnu/store/.links had enough entries,
ext4's directory index would be full, leading to link(2) returning
ENOSPC.

* nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Upon
ENOSPC from link(2), print a message and return instead of throwing a
'SysError'.
2016-10-28 22:30:17 +02:00
Eelco Dolstra
b1fd0ab734
daemon: Improve the SQLite wrapper API.
In particular, this eliminates a bunch of boilerplate code.

Also integrates these Nix commits:

  80da7a6 Probably fix SQLITE_BUSY errors
  37a337b throwSQLiteError(): Check for SIGINT so we don't loop forever

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2016-10-28 22:30:17 +02:00
Eelco Dolstra
7bed5d91de
daemon: Factor out SQLite handling.
* nix/libstore/local-store.cc: Move SQLite code to...
* nix/libstore/sqlite.cc, nix/libstore/sqlite.hh: ... here.  New files.
* nix/local.mk (libstore_a_SOURCES): Add sqlite.cc.
(libstore_headers): Add sqlite.hh.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2016-10-28 22:30:17 +02:00
Eelco Dolstra
517ce0c15b
daemon: Turn retrying SQLite transactions into a higher-order function.
* nix/libstore/local-store.cc (retry_sqlite, end_retry_sqlite): Remove.
(retrySQLite): New template.
(LocalStore::registerFailedPath, LocalStore::hasPathFailed)
(LocalStore::queryFailedPaths, LocalStore::clearFailedPaths)
(LocalStore::queryPathInfo, LocalStore::isValidPath_)
(LocalStore::queryValidPaths, LocalStore::queryAllValidPaths)
(LocalStore::queryReferrers, LocalStore::queryValidDerivers)
(LocalStore::queryDerivationOutputs)
(LocalStore::queryDerivationOutputNames)
(LocalStore::queryPathFromHashPart, LocalStore::registerValidPaths)
(LocalStore::invalidatePathChecked): Use it.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2016-10-28 22:30:17 +02:00
David Craven
9dd674db01
daemon: Rename 'NIX_CONF_DIR' to 'GUIX_CONFIGURATION_DIRECTORY'.
Partly fixes <http://bugs.gnu.org/22459>.
Reported by Jeff Mickey <j@codemac.net> and David Craven <david@craven.ch>.

* nix/libstore/globals.cc (Settings::processEnvironment()): Change
'NIX_CONF_DIR' to 'GUIX_CONFIGURATION_DIRECTORY'.
* nix/local.mk (libstore_a_CPPFLAGS): Likewise.
* guix/config.scm.in (%config-directory): Likewise.
* build-aux/test-env.in: Likewise.
* gnu/packages/patches/hydra-automake-1.15.patch: Likewise.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2016-06-26 23:11:07 +02:00
Ludovic Courtès
151afd84dc
daemon: Substitute queries return immediately when substitutes are disabled.
Reported by Federico Beffa <beffa@ieee.org>
at <https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00928.html>.

* nix/libstore/local-store.cc (LocalStore::querySubstitutablePaths)
(LocalStore::querySubstitutablePathInfos): Return when
'settings.useSubstitutes' is false.
* tests/store.scm ("references/substitutes missing reference info"):
Make sure to return #f on failure.
* tests/store.scm ("substitutable-path-info when substitutes are turned off"):
("substitutable-paths when substitutes are turned off"): New tests.
2016-05-31 18:22:14 +02:00
Ludovic Courtès
4078fa8f72
daemon: Fix typo.
Fixes a regression/typo introduced in
e08380fb6c.

* nix/libstore/build.cc (DerivationGoal::startBuilder): Canonicalize
"/tmp", not "/tmp/guix-build".
2016-05-31 18:22:14 +02:00
Ludovic Courtès
75abbd0cbe
daemon: Remove unused XML output code.
* nix/local.mk (libutil_a_SOURCES): Remove libutil/xml-writer.cc.
(libutil_headers): Remove libutil/xml-writer.hh.
* nix/libutil/xml-writer.hh, nix/libutil/xml-writer.cc: Remove.
2016-05-31 18:22:14 +02:00
Eelco Dolstra
b4528110c6
daemon: rounds: Keep the differing output if -K is given.
Regardless of -K, we now also print which output differs.
2016-05-31 14:25:28 +02:00
Eelco Dolstra
56b8f56788
daemon: Canonicalize gids to 0.
Previously files in the Nix store were owned by root or by nixbld,
depending on whether they were created by a substituter or by a
builder. This doesn't matter much, but causes spurious diffoscope
differences. So use root everywhere.
2016-05-31 14:25:28 +02:00
Eelco Dolstra
7f3b2510d6
daemon: check: Keep the differing output if -K is given.
This makes it easier to investigate the non-determinism, e.g.

  $ nix-build pkgs/stdenv/linux -A stage1.pkgs.zlib --check -K
  error: derivation ‘/nix/store/l54i8wlw22656i4pk05c52ngv9rpl39q-zlib-1.2.8.drv’ may not be deterministic: output ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8’ differs from ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check’

  $ diffoscope /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8 /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check
  ...
  ├── lib/libz.a
  │   ├── metadata
  │   │ @@ -1,15 +1,15 @@
  │   │ -rw-r--r-- 30001/30000   3096 Jan 12 15:20 2016 adler32.o
  ...
  │   │ +rw-r--r-- 30001/30000   3096 Jan 12 15:28 2016 adler32.o
  ...

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2016-05-31 14:25:28 +02:00
Eelco Dolstra
7889e72dad
daemon: check: Fix "failed to produce output path".
This occured when sandbox building is disabled, at least one output
exists, and at least one other output does not.
2016-05-31 14:25:28 +02:00
Eelco Dolstra
f83e82cd00
daemon: check: Fix assertion failure when some outputs are missing.
E.g.

  $ nix-build pkgs/stdenv/linux/ -A stage1.pkgs.perl --check
  nix-store: src/libstore/build.cc:1323: void nix::DerivationGoal::tryToBuild(): Assertion `buildMode != bmCheck || validPaths.size() == drv->outputs.size()' failed.

when perl.out exists but perl.man doesn't. The fix is to only check
the outputs that exist. Note that "nix-build -A stage1.pkgs.all
--check" will still give a (proper) error in this case.
2016-05-31 14:25:28 +02:00
Eelco Dolstra
c56cd90cc9
daemon: When repairing, rebuild if there is no substituter. 2016-05-31 14:25:28 +02:00
Eelco Dolstra
1a9574e3a7
daemon: Fix --repair failure on multiple-output derivations.
If repair found a corrupted/missing path that depended on a
multiple-output derivation, and some of the outputs of the latter were
not present, it failed with a message like

  error: path ‘/nix/store/cnfn9d5fjys1y93cz9shld2xwaibd7nn-bash-4.3-p42-doc’ is not valid

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2016-05-31 14:25:28 +02:00
Eelco Dolstra
63a5be07e2
daemon: ~PathLocks(): Handle exceptions.
Otherwise, since the call to write a "d" character to the lock file
can fail with ENOSPC, we can get an unhandled exception resulting in a
call to terminate().
2016-05-31 14:25:28 +02:00
Eelco Dolstra
e08380fb6c
daemon: Handle /tmp being a symlink.
* nix/libstore/build.cc (DerivationGoal::startBuilder): Call 'canonPath'
on "/tmp".
2016-05-31 14:25:28 +02:00
Mathieu Lirzin
c0d2e7b197
build: Use "%D%" in Makefile fragments.
* doc/local.mk: Use "%D%" for the directory of the fragment relative to
the base 'Makefile.am'.
* emacs/local.mk: Likewise.
* gnu/local.mk: Likewise.
* nix/local.mk: Likewise.
2016-05-24 13:07:19 +02:00
Mathieu Lirzin
03d0e2d2b9 build: Move 'Makefile' fragments to subdirectories.
This follows a convention used by some other GNU packages like Autoconf,
Bison, Coreutils, and Gnulib.

* doc.am: Rename to ...
* doc/local.mk: ... this.
* emacs.am: Rename to ...
* emacs/local.mk: ... this.
* gnu-system.am: Rename to ...
* gnu/local.mk: ... this.
* daemon.am: Rename to ...
* nix/local.mk: ... this.
* Makefile.am: Adapt to them.
* doc/guix.texi (Porting to a New Platform): Adapt documentation.
* guix/config.scm.in (%state-directory, %config-directory): Adapt comments.
* emacs/guix-config.el.in (guix-config-state-directory): Likewise.
2016-04-21 18:47:50 +02:00