Commit Graph

169 Commits

Author SHA1 Message Date
espie
7fdfa19c5a make sure stuff gets thru ssh correctly 2012-04-21 21:09:07 +00:00
espie
5db27d90a1 fix env name for wrapper program 2012-04-21 14:41:37 +00:00
espie
8dc35a6962 make it possible to use a specialized wrapper to collect rss stats 2012-04-21 11:30:53 +00:00
espie
8cb5363e94 whitespace 2012-04-10 17:06:15 +00:00
espie
0deaaa9fa4 avoid spewed warnings in case we don't care. 2012-04-10 16:50:33 +00:00
espie
2ecf172edb elide empty L= / E= lines 2012-04-07 12:16:58 +00:00
espie
a01f052d1c on accelerated boxes, one can keep the cursor visible.
make that the default, keep vis/invis as -DNO_CURSOR.
2012-04-07 12:13:36 +00:00
espie
b2a184c1df fix landry's error.
forget was in the wrong class, oops.
2012-04-02 09:21:18 +00:00
espie
5572ec286f ignore build_stats for logging distinfo (and clean-old-distfiles(1)). 2012-03-22 16:09:37 +00:00
espie
4fe1c88e80 quickpath to fullpkgpath, called a lot during adjust. 2012-03-13 16:48:47 +00:00
espie
38db8f2149 tweak weight handling so that fixing weights and scaling weights works.
add -m option as a reverse to -x, like in pkg_add
2012-03-09 12:51:38 +00:00
espie
d1754a0b78 fix things when we don't have any statistics to start with 2012-03-06 13:41:38 +00:00
espie
5327873dc1 fix -a bug: if you specify -I ..., this shouldn't negate a -a.
"permanent build log" under %f/build-stats/%a
-J250 by default
2012-03-05 19:43:09 +00:00
espie
0a05d056a4 keep track of how long a job is stuck, max. 2012-03-02 19:33:43 +00:00
espie
7222a75d55 fix detection of existing dpbs. 2012-03-02 17:14:41 +00:00
espie
243f41a146 better: if old locks, don't warn.
if no locks to possibly remove, well, don't even run ps !
2012-02-28 14:23:27 +00:00
espie
fb2feec6cf let's do error messages differently: get die to reset_cursor and die
(reverse-override reporter), and get warn to use myprint (and need for
tabs to become spaces in that case...)
2012-02-27 15:37:36 +00:00
espie
56b1b3241a allow -I/-P/-C and command-line arguments to be lists of pkgpaths, or pkgpath
(it's easy to distinguish between a file and a directory under ports).

expand sequences for those files and hosts

when restarting dpb, kill locks that don't correspond to errors, but to a dpb
running on the same host that's no longer there.

do __WARN__ like __DIE__

option -DDONT_BUILD_ONCE
option -DDONT_CLEAN_LOCKS

document some
2012-02-27 14:51:37 +00:00
espie
e7e37de86c hack to avoid stopping midway thru, need to find a better way... ;( 2012-02-23 21:42:23 +00:00
espie
b4dfd43424 also log the fullpkgpath, no need to go look in the lock proper for that. 2012-02-21 10:37:26 +00:00
espie
1a4ddc336c record master dpb pid and hostname
(will be used for automatic clean-up of locks on startup)
2012-02-20 21:18:48 +00:00
espie
81064a2b7c make sure distdir exists before writing information in there 2012-02-20 16:38:23 +00:00
espie
ac82d6c329 cosmetic: fetches come in, so <
explain waiting time better.
2012-02-17 07:36:09 +00:00
espie
9a29fe97e4 allow weights to be scaled
add longer traceback in case we die
2012-02-17 07:35:42 +00:00
espie
c422b5ae52 when you never fetched anything after the cipher change, that directory
doesn't exist... don't error out...
2012-01-31 15:45:19 +00:00
espie
7f2b8805ae finish distfiles history handling: people who don't fetch/build the whole
tree may want to weed distfiles too, so allow for a full scan of the tree
without building/fetching anything, just to update history:

dpb -DHISTORY_ONLY

(just requires making sure the right engines are created, and a very shortened
loop at end waiting for history to be updated).
2012-01-30 15:11:04 +00:00
espie
623eb8b2c5 Better name for the task that, duh, is actually updating distfiles history. 2012-01-30 14:19:30 +00:00
espie
8ddfdef47f more flexible error path while scanning, accumulate reasons why broken.
remove a few explicit (and implicit) die from Fetch: missing/out-of-sync
distinfo no longer kill dpb, instead they're properly reported as broken
paths and things still go on (note that even a missing SUPDISTFILE checksum
*will* mark a path as broken, that's totally intentional)
2012-01-29 12:02:20 +00:00
espie
cd3220153e remove a bit of noise from the log... we checked already, no need to
log it again.
2012-01-23 10:35:38 +00:00
espie
1ec4d51eba repair dpb -f0
noticed by phessler@
2012-01-18 15:26:49 +00:00
espie
f0d702b93c also scan by_cipher for old stuff. This requires a bit more logic, as
the names of files in there are not totally trivial to figure out from the
normal filename + sha.
-> filename will be stripped off DIST_SUBDIR
-> b64 checksums interfere with filesystem semantics, e.g., /u8//ffg
will become u8/ffg
2012-01-16 18:55:35 +00:00
espie
3a6eaf5081 add support to obey PERMIT_DISTFILES* 2012-01-14 12:26:21 +00:00
espie
cb500b33c7 remove files from distinfo when they no longer exist, so when you clean
up files and history, it won't recreate fantom stuff...
2012-01-10 19:35:36 +00:00
espie
f8050aba6f run "expire_old" as a separate job, since it may take some time.
find files under $DISTDIR, anything we don't know about should also
end up in history.
2012-01-09 17:56:28 +00:00
espie
c0aa7effe3 lock may fail 2012-01-09 11:54:30 +00:00
espie
fa990a1dba give dpb more knowledge about distfiles.
- keep a stash indexed by checksum, so dpb can identify duplicate files.
- in a full bulk, if the scan has no errors, write to a ${DISTDIR}/history
file  the files encountered in ${DISTDIR}/distinfo that seem to no longer
be needed (with full timestamp and checksum info).

Should be enough info to know when to expire old DISTDIR entries.
2012-01-08 20:28:37 +00:00
espie
52e58e485e make fetch_manually error directly visible in the lock 2012-01-08 14:40:58 +00:00
espie
3884c2f898 buglet with links and files with a DIST_SUBDIR, as found by naddy@.
people already running dpb -F: just remove /usr/ports/distfiles/distinfo
and rerun dpb.
(or at least all entries with a subdirectory)
2012-01-05 21:49:52 +00:00
espie
d9eef11aed prevent dpb from dying if make_link dies (also does not append to global
distinfo, of course)
2012-01-01 18:34:35 +00:00
espie
9938b28564 allow the fetcher part to work as a "true" mirror maker:
- keep a cache of checked checksums as ${DISTDIR}/distinfo
- read existing cache on startup, don't need to recheck those files.
- split checksum_task: update the cache after the checksum is done
- create the by_cipher/sha256/... entries
- in fetch_only mode, do a "long" checksum on existing files not in the cache.

If you ever have doubts on the integrity of already checked files, just
rm ${DISTDIR}/distinfo.
2011-12-31 11:20:00 +00:00
espie
1f6bab1e34 filter out lib-deps with the same pkgpath_and_flavors as what we're
building, like bsd.port.mk does.
won't change anything in the build, but makes pkg_add -a less confusing,
since mp ports won't try to add itself while building.0
2011-12-11 19:55:37 +00:00
espie
d179751a0f on SIGINFO, dump T/B hashes into info.log (and mark the display for refresh,
as the shell might muck it with its own information)
2011-12-10 14:48:40 +00:00
espie
c71b9ad066 expose interface to "refresh" display after some disturbing operation. 2011-12-10 14:47:36 +00:00
espie
3081c4ac9a remove "needed" debug log now that things work fine. 2011-12-05 21:27:53 +00:00
espie
03c4aa3410 ouch, don't forget to lock at front of prepare (okay, we can avoid it
if no-depends but that's complicated)

also, two distinct critical sections: we don't need to stay locked at
the end of show-prepare-result.

Even though that's not really important, simplify logic a wee little bit
2011-12-05 21:18:55 +00:00
espie
b10178048d move uninstall behavior to uninstall, unbreak !-J
as noticed by fgs@
2011-12-05 18:29:39 +00:00
espie
abb8ddf57d turn -J into an option taking a number.
always run show-prepare-results, and always serialize
depends/prepare/show-prepare-results(/junk) on a single host.
2011-12-05 16:10:00 +00:00
espie
a38035bd67 clean-up job running a bit: add methods to Grabber and PortBuilder objects
to access more stuff they need.

Remove distantshell->make, ask state instead.
Provide state with the program we run, and the parameters we pass it.

- same make for distant and local hosts
- pass BUILD_ONCE=Yes if -a.
2011-12-04 12:05:41 +00:00
espie
3858760dd5 cut up big method into smaller parts 2011-12-04 10:57:46 +00:00
espie
69df96b71c properly forget old info when rescanning... 2011-12-04 10:39:52 +00:00