Commit Graph

88 Commits

Author SHA1 Message Date
espie
99c73a5b6c fix die handler, as it should display the message as well...
grumble

make it possible to temporarily disable those handlers.
2013-04-13 10:02:11 +00:00
espie
79fa504e3a rework INFO/DIE/WARN handler to display some info in a non-opaque way. 2013-03-03 11:11:08 +00:00
espie
47036c8aec tweak the carp stuff to not recurse into itself, so that compile errors
show up correctly again.

split start_cores into a read_config/start_cores part, so that read_config
can happen right after handle_options, and so that we can put more options
in the hosts file (to be done)
2013-03-03 00:51:57 +00:00
espie
eef793741e some of args seen in trace might be undef'd. avoid triggering the WARN
handler while handling SIG{INFO}.
Also, have parse_size_info be verbose about what's going on.
2013-02-04 09:43:57 +00:00
espie
44ac9e61b7 move control of the "building in mfs" part under PortsBuilder/Heuristics.
Store previous pkgname.
do show-size only sporadically if fullpkgname didn't change
2013-02-02 13:35:17 +00:00
espie
12f4db34e8 allow for size log to contain pkgpath(pkgname) instead of just pkgpath.
make -S useful again: allow it to "prime" the log from an existing source,
but don't write back to it.
2013-02-02 10:36:31 +00:00
espie
573c499cf5 add a FETCH_TIMEOUT, similar to STUCK_TIMEOUT, but slightly simpler,
as there is no speed factor there...
2013-01-28 12:23:45 +00:00
espie
11cd4c0c77 don't bother dumping distfiles, history is quite enough 2013-01-28 10:50:46 +00:00
espie
c259d653cc tweak memory handling to make it less special: treat -M as an override_prop
as we should.
Use core methods to access the memory threshold.

UI simplification: -M can take a suffix, default is K, but you can just say
-M 520M or -M 2G now.
2013-01-28 10:14:17 +00:00
espie
cd3023d079 add debugging message that does not depend on Devel::StackTrace, may be
useful to find out where dpb gets stuck
2013-01-28 09:56:00 +00:00
espie
2f7c8bf00b bump default junk back up to 150, makes more sense with new accounting 2013-01-27 23:15:57 +00:00
espie
0a438fe138 prompted by jasper: reintroduce -S, but as a rw option, to override
the rolling journal location for size log.

make it possible to share dpb -s output from a given architecture with
similar architectures
2013-01-21 14:50:30 +00:00
espie
9ad81d2390 simplify wrkdir size handling, turn that into a rolling log. 2013-01-21 12:03:32 +00:00
espie
647327b6ba don't go thru patch/configure/build for very small ports. 2013-01-10 12:27:21 +00:00
espie
d8408ea93f fix a long-standing stupid bug. Now I know why weight changes didn't matter,
I wasn't passing them through, sigh.

Add new -X option that sets xpaths, to be used...
2013-01-07 10:57:31 +00:00
espie
8a23d3ca7e ask the ports tree to clean up stale locks, finally... 2013-01-06 11:59:40 +00:00
espie
209b83159b allow independent setup for the locks location. I strongly suggest
using a local fs.
2013-01-05 18:26:31 +00:00
espie
779bfd4ae7 fix typo 2013-01-05 15:13:48 +00:00
espie
116acde577 change to NO_HISTORY and document (since there are other NO_* thingies) 2013-01-05 13:34:28 +00:00
espie
85b554d119 make it possible to skip "UPDATING DISTFILES" if we're just running dpb
again without any change.
2013-01-05 13:32:28 +00:00
espie
9eb2e2c9df fix typo. Symptom: errors out at end of build
noticed by rpe@
2013-01-04 18:29:13 +00:00
espie
9a2e664c54 force report display to fire whenever some new ports get built. 2013-01-04 12:46:20 +00:00
espie
a1d376fbf6 take the rate-limiting concept used for the engine, and make it available
elsewhere. Also use it for the Reporter, as it makes no sense to spend
THAT much time reporting quick changes, which actually slows the build.
($factor to tweak as needed).
2013-01-04 12:06:25 +00:00
espie
c39d9434ff fix infamous "doesn't really quit when it's done" stop bug.
also move the report to the main package, and show a prominent
STOPPED in the title bar if you forget you stopped it during a
previous run ('why is my dpb not building anything ?')
2013-01-03 15:43:27 +00:00
espie
7adbbd9561 remove optimization that no longer makes sense. 2012-12-28 06:40:11 +00:00
espie
e4c671527c add affinity information to restart jobs on the right host preferentially.
- affinity info is similar to locks, but with a completely different
lifetime.
- streamline the main loop of the engine, so that it can do two passes:
first pass shuns paths with the wrong affinity. If no good path is found,
those are considered during the second pass.
- make the Core factory aware of what hosts might be running, so that
affinity info for machines removed from a config file will be ignored.

thanks to landry@ for a few tests.
2012-12-24 17:24:46 +00:00
espie
ba8f355da8 more descriptive name 2012-11-18 12:37:21 +00:00
espie
7fafb5c793 allow for -DWAIT_TIMEOUT=0 to propagate correctly
lower default for junk
2012-11-18 01:58:36 +00:00
espie
d73a931704 don't mix up wait_timeout and waiting_timeout, as noticed by naddy@.
re-enable wait_timeout on localhost temporarily (should be done in another
way, most probably by checking whether repo is on nfs, we can steal code
from VStat.pm)

pass umask through ssh. This took us long enough to figure out, and it's
considerably simpler than tweaking every login class once again.
2012-11-06 08:26:29 +00:00
espie
04618136d2 fix default property handling 2012-10-13 09:00:02 +00:00
espie
302ada6ac0 tweaks
- waiting should know when it gets interrupted! so that it doesn't zombify
- add a time limit.
2012-10-13 08:32:58 +00:00
espie
0f91b47f42 -DALWAYS_CLEAN: clean ports even if they errored out (that's a prop, so
always_clean=1  in hosts file.

refactor task code a bit. And fix a buglet related to the NFS check
(wrong finalize order)
2012-10-12 20:24:56 +00:00
espie
9b37bcd801 change things again to have command line options override host file.
and provide the possibility of a DEFAULT entry in the host file.
2012-10-11 07:40:30 +00:00
rpe
8d3efe0bb7 make out-of-date output directly usable for dbp
- ignore empty lines
- remove comments and whitespaces at the end of lines

ok espie@
2012-10-10 11:44:11 +00:00
espie
cdc017bcb0 small change for people running multiple hosts:
options on the command line now define *defaults* that host files can
override (for instance -j, stuck, -p, -J).

Add -p /n to mean "take number of jobs, if >1, divide by n, round up to 2,
and use that for parallel.

Document -p.

Make junk be 'by host' (and it's a prop, so you can tweak it).

concurrent log that records how many jobs are running each time it changes.
tag parallel builds *n in the time record.
2012-10-08 12:41:03 +00:00
espie
f8d9ead7d8 new framework to deal with the new ports cluster, to be documented 2012-09-23 18:13:32 +00:00
espie
b56361e24c always do default stats, unless we explicitly disable them with -DNO_STATS 2012-04-10 16:58:47 +00:00
espie
9cbe5aecb1 have a look at PORTSDIR_PATH as well, so that we can dpb -P dir/p
under mystuff.

problem noticed by jeremy@
2012-04-02 15:51:24 +00:00
espie
11d2036d1a duplicate -m option, resolve conflict by renaming old unused one to -M.
Document some new stuff
2012-03-09 15:16:38 +00:00
espie
38db8f2149 tweak weight handling so that fixing weights and scaling weights works.
add -m option as a reverse to -x, like in pkg_add
2012-03-09 12:51:38 +00:00
espie
bffed54177 create build-stats dir (noticed by Mark Patruck) 2012-03-06 11:22:40 +00:00
espie
5327873dc1 fix -a bug: if you specify -I ..., this shouldn't negate a -a.
"permanent build log" under %f/build-stats/%a
-J250 by default
2012-03-05 19:43:09 +00:00
espie
a366b51e34 %f for distdir (where fetched distfiles live) 2012-03-02 19:35:09 +00:00
espie
2cc9bc20ca tweak so that dpb -I misc/screen/
will work
2012-02-27 17:50:14 +00:00
espie
fb2feec6cf let's do error messages differently: get die to reset_cursor and die
(reverse-override reporter), and get warn to use myprint (and need for
tabs to become spaces in that case...)
2012-02-27 15:37:36 +00:00
espie
56b1b3241a allow -I/-P/-C and command-line arguments to be lists of pkgpaths, or pkgpath
(it's easy to distinguish between a file and a directory under ports).

expand sequences for those files and hosts

when restarting dpb, kill locks that don't correspond to errors, but to a dpb
running on the same host that's no longer there.

do __WARN__ like __DIE__

option -DDONT_BUILD_ONCE
option -DDONT_CLEAN_LOCKS

document some
2012-02-27 14:51:37 +00:00
espie
9a29fe97e4 allow weights to be scaled
add longer traceback in case we die
2012-02-17 07:35:42 +00:00
espie
7f2b8805ae finish distfiles history handling: people who don't fetch/build the whole
tree may want to weed distfiles too, so allow for a full scan of the tree
without building/fetching anything, just to update history:

dpb -DHISTORY_ONLY

(just requires making sure the right engines are created, and a very shortened
loop at end waiting for history to be updated).
2012-01-30 15:11:04 +00:00
espie
f8050aba6f run "expire_old" as a separate job, since it may take some time.
find files under $DISTDIR, anything we don't know about should also
end up in history.
2012-01-09 17:56:28 +00:00
espie
fa990a1dba give dpb more knowledge about distfiles.
- keep a stash indexed by checksum, so dpb can identify duplicate files.
- in a full bulk, if the scan has no errors, write to a ${DISTDIR}/history
file  the files encountered in ${DISTDIR}/distinfo that seem to no longer
be needed (with full timestamp and checksum info).

Should be enough info to know when to expire old DISTDIR entries.
2012-01-08 20:28:37 +00:00