Commit Graph

175 Commits

Author SHA1 Message Date
espie
9a2e664c54 force report display to fire whenever some new ports get built. 2013-01-04 12:46:20 +00:00
espie
a1d376fbf6 take the rate-limiting concept used for the engine, and make it available
elsewhere. Also use it for the Reporter, as it makes no sense to spend
THAT much time reporting quick changes, which actually slows the build.
($factor to tweak as needed).
2013-01-04 12:06:25 +00:00
espie
c39d9434ff fix infamous "doesn't really quit when it's done" stop bug.
also move the report to the main package, and show a prominent
STOPPED in the title bar if you forget you stopped it during a
previous run ('why is my dpb not building anything ?')
2013-01-03 15:43:27 +00:00
espie
7adbbd9561 remove optimization that no longer makes sense. 2012-12-28 06:40:11 +00:00
espie
e4c671527c add affinity information to restart jobs on the right host preferentially.
- affinity info is similar to locks, but with a completely different
lifetime.
- streamline the main loop of the engine, so that it can do two passes:
first pass shuns paths with the wrong affinity. If no good path is found,
those are considered during the second pass.
- make the Core factory aware of what hosts might be running, so that
affinity info for machines removed from a config file will be ignored.

thanks to landry@ for a few tests.
2012-12-24 17:24:46 +00:00
espie
cdb441ddb5 cutdate is first parameter, not second.
tighten parenthesis.
add -h history for testing.
2012-12-23 13:27:08 +00:00
espie
ba8f355da8 more descriptive name 2012-11-18 12:37:21 +00:00
espie
7fafb5c793 allow for -DWAIT_TIMEOUT=0 to propagate correctly
lower default for junk
2012-11-18 01:58:36 +00:00
espie
d73a931704 don't mix up wait_timeout and waiting_timeout, as noticed by naddy@.
re-enable wait_timeout on localhost temporarily (should be done in another
way, most probably by checking whether repo is on nfs, we can steal code
from VStat.pm)

pass umask through ssh. This took us long enough to figure out, and it's
considerably simpler than tweaking every login class once again.
2012-11-06 08:26:29 +00:00
espie
04618136d2 fix default property handling 2012-10-13 09:00:02 +00:00
espie
302ada6ac0 tweaks
- waiting should know when it gets interrupted! so that it doesn't zombify
- add a time limit.
2012-10-13 08:32:58 +00:00
espie
0f91b47f42 -DALWAYS_CLEAN: clean ports even if they errored out (that's a prop, so
always_clean=1  in hosts file.

refactor task code a bit. And fix a buglet related to the NFS check
(wrong finalize order)
2012-10-12 20:24:56 +00:00
espie
9b37bcd801 change things again to have command line options override host file.
and provide the possibility of a DEFAULT entry in the host file.
2012-10-11 07:40:30 +00:00
rpe
8d3efe0bb7 make out-of-date output directly usable for dbp
- ignore empty lines
- remove comments and whitespaces at the end of lines

ok espie@
2012-10-10 11:44:11 +00:00
espie
cdc017bcb0 small change for people running multiple hosts:
options on the command line now define *defaults* that host files can
override (for instance -j, stuck, -p, -J).

Add -p /n to mean "take number of jobs, if >1, divide by n, round up to 2,
and use that for parallel.

Document -p.

Make junk be 'by host' (and it's a prop, so you can tweak it).

concurrent log that records how many jobs are running each time it changes.
tag parallel builds *n in the time record.
2012-10-08 12:41:03 +00:00
espie
929c759b7e remove old compatibility cruft with old register-plist that used to
say "def" for version number.

check that pkgpath in dependency did not change, in which case the
dependent port should have been bumped.

problem experienced by aja@ on glib2...

patch tested and okay jasper@, sthen@
2012-09-27 14:46:37 +00:00
espie
f8d9ead7d8 new framework to deal with the new ports cluster, to be documented 2012-09-23 18:13:32 +00:00
espie
9ab27ec343 unintentional say + "\n" 2012-09-16 08:15:43 +00:00
jasper
58df29bb26 remove this script since the old readmes infrastructure has been replaced
by databases/ports-readmes.

ok espie@
2012-08-27 12:30:00 +00:00
jasper
5d931b8d90 - use mktemp -t
from rpe
ok espie@
2012-08-27 11:24:52 +00:00
espie
76fbad9f1b those have been moved to base/ 2012-06-19 16:47:06 +00:00
espie
5ce8777473 just some nits 2012-05-07 15:57:51 +00:00
espie
b56361e24c always do default stats, unless we explicitly disable them with -DNO_STATS 2012-04-10 16:58:47 +00:00
espie
9cbe5aecb1 have a look at PORTSDIR_PATH as well, so that we can dpb -P dir/p
under mystuff.

problem noticed by jeremy@
2012-04-02 15:51:24 +00:00
espie
11d2036d1a duplicate -m option, resolve conflict by renaming old unused one to -M.
Document some new stuff
2012-03-09 15:16:38 +00:00
espie
38db8f2149 tweak weight handling so that fixing weights and scaling weights works.
add -m option as a reverse to -x, like in pkg_add
2012-03-09 12:51:38 +00:00
espie
bffed54177 create build-stats dir (noticed by Mark Patruck) 2012-03-06 11:22:40 +00:00
espie
5327873dc1 fix -a bug: if you specify -I ..., this shouldn't negate a -a.
"permanent build log" under %f/build-stats/%a
-J250 by default
2012-03-05 19:43:09 +00:00
espie
a366b51e34 %f for distdir (where fetched distfiles live) 2012-03-02 19:35:09 +00:00
espie
2cc9bc20ca tweak so that dpb -I misc/screen/
will work
2012-02-27 17:50:14 +00:00
espie
fb2feec6cf let's do error messages differently: get die to reset_cursor and die
(reverse-override reporter), and get warn to use myprint (and need for
tabs to become spaces in that case...)
2012-02-27 15:37:36 +00:00
espie
56b1b3241a allow -I/-P/-C and command-line arguments to be lists of pkgpaths, or pkgpath
(it's easy to distinguish between a file and a directory under ports).

expand sequences for those files and hosts

when restarting dpb, kill locks that don't correspond to errors, but to a dpb
running on the same host that's no longer there.

do __WARN__ like __DIE__

option -DDONT_BUILD_ONCE
option -DDONT_CLEAN_LOCKS

document some
2012-02-27 14:51:37 +00:00
espie
9a29fe97e4 allow weights to be scaled
add longer traceback in case we die
2012-02-17 07:35:42 +00:00
espie
7f2b8805ae finish distfiles history handling: people who don't fetch/build the whole
tree may want to weed distfiles too, so allow for a full scan of the tree
without building/fetching anything, just to update history:

dpb -DHISTORY_ONLY

(just requires making sure the right engines are created, and a very shortened
loop at end waiting for history to be updated).
2012-01-30 15:11:04 +00:00
espie
e8ba37e682 proper usage diagnostic. 2012-01-29 14:35:40 +00:00
espie
070d4f02a9 allow a list of exceptions, for easier management of distfiles from ports not
yet linked to the build.
2012-01-29 13:03:05 +00:00
espie
1a3bfc6597 make new mirroring with dpb feature-complete, provide a small perl script that
can remove old distfiles in a smart way according to history.
2012-01-27 19:03:02 +00:00
espie
f8050aba6f run "expire_old" as a separate job, since it may take some time.
find files under $DISTDIR, anything we don't know about should also
end up in history.
2012-01-09 17:56:28 +00:00
espie
fa990a1dba give dpb more knowledge about distfiles.
- keep a stash indexed by checksum, so dpb can identify duplicate files.
- in a full bulk, if the scan has no errors, write to a ${DISTDIR}/history
file  the files encountered in ${DISTDIR}/distinfo that seem to no longer
be needed (with full timestamp and checksum info).

Should be enough info to know when to expire old DISTDIR entries.
2012-01-08 20:28:37 +00:00
espie
d179751a0f on SIGINFO, dump T/B hashes into info.log (and mark the display for refresh,
as the shell might muck it with its own information)
2011-12-10 14:48:40 +00:00
espie
c783276075 truly random mode shouldn't peek at most frequent dependencies ! 2011-12-05 21:22:35 +00:00
espie
abb8ddf57d turn -J into an option taking a number.
always run show-prepare-results, and always serialize
depends/prepare/show-prepare-results(/junk) on a single host.
2011-12-05 16:10:00 +00:00
espie
a38035bd67 clean-up job running a bit: add methods to Grabber and PortBuilder objects
to access more stuff they need.

Remove distantshell->make, ask state instead.
Provide state with the program we run, and the parameters we pass it.

- same make for distant and local hosts
- pass BUILD_ONCE=Yes if -a.
2011-12-04 12:05:41 +00:00
espie
ddebaf6da6 not ideal, but will now stop at the end of a -F run. 2011-12-04 11:24:38 +00:00
espie
91b04a0eac fix quick-start: the interface to grabber changed in a silent way.
explicitly move the main_loop near handle_non_waiting_jobs to try to
fix that mess (and finish all the time)
2011-12-03 11:01:47 +00:00
espie
2cfd4c615d new junk option (to remove packages as they're no longer needed) to pass
around thru JobBuilder (along with locks since we need to write stuff into
that file)
2011-12-02 22:33:46 +00:00
espie
5a04aafc13 zap old stuff, only read plist from stdin if -i was given 2011-12-01 11:20:19 +00:00
jasper
5c15af4428 - move to bin and adjust paths where needed, manpages will follown soon 2011-11-28 10:30:22 +00:00
espie
48b9bf74ff bug fix, don't forget to finish_scanning when reading from source ! 2011-11-27 17:37:47 +00:00
espie
34c5369453 unconfuse naming 2011-11-27 17:31:54 +00:00