Commit Graph

190 Commits

Author SHA1 Message Date
espie
f7ea023751 remove distinction between local and distant running, always create
a shell object that can chdir, setenv, and exec commands.
(note that this executes stuff after fork, so permanent changes are cheap
and okay)

Also create it from "host" objects, which simplifies parameter passing.
2012-07-04 08:59:10 +00:00
espie
76fbad9f1b those have been moved to base/ 2012-06-19 16:47:06 +00:00
espie
ef74243178 adjust interface slightly 2012-05-16 08:22:22 +00:00
espie
a3e1fdee35 split the PkgPath component into a base class so that sqlports will be
able to reuse it
2012-05-15 10:10:13 +00:00
espie
7fdfa19c5a make sure stuff gets thru ssh correctly 2012-04-21 21:09:07 +00:00
espie
5db27d90a1 fix env name for wrapper program 2012-04-21 14:41:37 +00:00
espie
8dc35a6962 make it possible to use a specialized wrapper to collect rss stats 2012-04-21 11:30:53 +00:00
espie
8cb5363e94 whitespace 2012-04-10 17:06:15 +00:00
espie
0deaaa9fa4 avoid spewed warnings in case we don't care. 2012-04-10 16:50:33 +00:00
espie
2ecf172edb elide empty L= / E= lines 2012-04-07 12:16:58 +00:00
espie
a01f052d1c on accelerated boxes, one can keep the cursor visible.
make that the default, keep vis/invis as -DNO_CURSOR.
2012-04-07 12:13:36 +00:00
espie
b2a184c1df fix landry's error.
forget was in the wrong class, oops.
2012-04-02 09:21:18 +00:00
espie
5572ec286f ignore build_stats for logging distinfo (and clean-old-distfiles(1)). 2012-03-22 16:09:37 +00:00
espie
4fe1c88e80 quickpath to fullpkgpath, called a lot during adjust. 2012-03-13 16:48:47 +00:00
espie
38db8f2149 tweak weight handling so that fixing weights and scaling weights works.
add -m option as a reverse to -x, like in pkg_add
2012-03-09 12:51:38 +00:00
espie
d1754a0b78 fix things when we don't have any statistics to start with 2012-03-06 13:41:38 +00:00
espie
5327873dc1 fix -a bug: if you specify -I ..., this shouldn't negate a -a.
"permanent build log" under %f/build-stats/%a
-J250 by default
2012-03-05 19:43:09 +00:00
espie
0a05d056a4 keep track of how long a job is stuck, max. 2012-03-02 19:33:43 +00:00
espie
7222a75d55 fix detection of existing dpbs. 2012-03-02 17:14:41 +00:00
espie
243f41a146 better: if old locks, don't warn.
if no locks to possibly remove, well, don't even run ps !
2012-02-28 14:23:27 +00:00
espie
fb2feec6cf let's do error messages differently: get die to reset_cursor and die
(reverse-override reporter), and get warn to use myprint (and need for
tabs to become spaces in that case...)
2012-02-27 15:37:36 +00:00
espie
56b1b3241a allow -I/-P/-C and command-line arguments to be lists of pkgpaths, or pkgpath
(it's easy to distinguish between a file and a directory under ports).

expand sequences for those files and hosts

when restarting dpb, kill locks that don't correspond to errors, but to a dpb
running on the same host that's no longer there.

do __WARN__ like __DIE__

option -DDONT_BUILD_ONCE
option -DDONT_CLEAN_LOCKS

document some
2012-02-27 14:51:37 +00:00
espie
e7e37de86c hack to avoid stopping midway thru, need to find a better way... ;( 2012-02-23 21:42:23 +00:00
espie
b4dfd43424 also log the fullpkgpath, no need to go look in the lock proper for that. 2012-02-21 10:37:26 +00:00
espie
1a4ddc336c record master dpb pid and hostname
(will be used for automatic clean-up of locks on startup)
2012-02-20 21:18:48 +00:00
espie
81064a2b7c make sure distdir exists before writing information in there 2012-02-20 16:38:23 +00:00
espie
ac82d6c329 cosmetic: fetches come in, so <
explain waiting time better.
2012-02-17 07:36:09 +00:00
espie
9a29fe97e4 allow weights to be scaled
add longer traceback in case we die
2012-02-17 07:35:42 +00:00
espie
c422b5ae52 when you never fetched anything after the cipher change, that directory
doesn't exist... don't error out...
2012-01-31 15:45:19 +00:00
espie
7f2b8805ae finish distfiles history handling: people who don't fetch/build the whole
tree may want to weed distfiles too, so allow for a full scan of the tree
without building/fetching anything, just to update history:

dpb -DHISTORY_ONLY

(just requires making sure the right engines are created, and a very shortened
loop at end waiting for history to be updated).
2012-01-30 15:11:04 +00:00
espie
623eb8b2c5 Better name for the task that, duh, is actually updating distfiles history. 2012-01-30 14:19:30 +00:00
espie
8ddfdef47f more flexible error path while scanning, accumulate reasons why broken.
remove a few explicit (and implicit) die from Fetch: missing/out-of-sync
distinfo no longer kill dpb, instead they're properly reported as broken
paths and things still go on (note that even a missing SUPDISTFILE checksum
*will* mark a path as broken, that's totally intentional)
2012-01-29 12:02:20 +00:00
espie
cd3220153e remove a bit of noise from the log... we checked already, no need to
log it again.
2012-01-23 10:35:38 +00:00
espie
1ec4d51eba repair dpb -f0
noticed by phessler@
2012-01-18 15:26:49 +00:00
espie
f0d702b93c also scan by_cipher for old stuff. This requires a bit more logic, as
the names of files in there are not totally trivial to figure out from the
normal filename + sha.
-> filename will be stripped off DIST_SUBDIR
-> b64 checksums interfere with filesystem semantics, e.g., /u8//ffg
will become u8/ffg
2012-01-16 18:55:35 +00:00
espie
3a6eaf5081 add support to obey PERMIT_DISTFILES* 2012-01-14 12:26:21 +00:00
espie
cb500b33c7 remove files from distinfo when they no longer exist, so when you clean
up files and history, it won't recreate fantom stuff...
2012-01-10 19:35:36 +00:00
espie
f8050aba6f run "expire_old" as a separate job, since it may take some time.
find files under $DISTDIR, anything we don't know about should also
end up in history.
2012-01-09 17:56:28 +00:00
espie
c0aa7effe3 lock may fail 2012-01-09 11:54:30 +00:00
espie
fa990a1dba give dpb more knowledge about distfiles.
- keep a stash indexed by checksum, so dpb can identify duplicate files.
- in a full bulk, if the scan has no errors, write to a ${DISTDIR}/history
file  the files encountered in ${DISTDIR}/distinfo that seem to no longer
be needed (with full timestamp and checksum info).

Should be enough info to know when to expire old DISTDIR entries.
2012-01-08 20:28:37 +00:00
espie
52e58e485e make fetch_manually error directly visible in the lock 2012-01-08 14:40:58 +00:00
espie
3884c2f898 buglet with links and files with a DIST_SUBDIR, as found by naddy@.
people already running dpb -F: just remove /usr/ports/distfiles/distinfo
and rerun dpb.
(or at least all entries with a subdirectory)
2012-01-05 21:49:52 +00:00
espie
d9eef11aed prevent dpb from dying if make_link dies (also does not append to global
distinfo, of course)
2012-01-01 18:34:35 +00:00
espie
9938b28564 allow the fetcher part to work as a "true" mirror maker:
- keep a cache of checked checksums as ${DISTDIR}/distinfo
- read existing cache on startup, don't need to recheck those files.
- split checksum_task: update the cache after the checksum is done
- create the by_cipher/sha256/... entries
- in fetch_only mode, do a "long" checksum on existing files not in the cache.

If you ever have doubts on the integrity of already checked files, just
rm ${DISTDIR}/distinfo.
2011-12-31 11:20:00 +00:00
espie
1f6bab1e34 filter out lib-deps with the same pkgpath_and_flavors as what we're
building, like bsd.port.mk does.
won't change anything in the build, but makes pkg_add -a less confusing,
since mp ports won't try to add itself while building.0
2011-12-11 19:55:37 +00:00
espie
d179751a0f on SIGINFO, dump T/B hashes into info.log (and mark the display for refresh,
as the shell might muck it with its own information)
2011-12-10 14:48:40 +00:00
espie
c71b9ad066 expose interface to "refresh" display after some disturbing operation. 2011-12-10 14:47:36 +00:00
espie
3081c4ac9a remove "needed" debug log now that things work fine. 2011-12-05 21:27:53 +00:00
espie
03c4aa3410 ouch, don't forget to lock at front of prepare (okay, we can avoid it
if no-depends but that's complicated)

also, two distinct critical sections: we don't need to stay locked at
the end of show-prepare-result.

Even though that's not really important, simplify logic a wee little bit
2011-12-05 21:18:55 +00:00
espie
b10178048d move uninstall behavior to uninstall, unbreak !-J
as noticed by fgs@
2011-12-05 18:29:39 +00:00