options on the command line now define *defaults* that host files can
override (for instance -j, stuck, -p, -J).
Add -p /n to mean "take number of jobs, if >1, divide by n, round up to 2,
and use that for parallel.
Document -p.
Make junk be 'by host' (and it's a prop, so you can tweak it).
concurrent log that records how many jobs are running each time it changes.
tag parallel builds *n in the time record.
tree may want to weed distfiles too, so allow for a full scan of the tree
without building/fetching anything, just to update history:
dpb -DHISTORY_ONLY
(just requires making sure the right engines are created, and a very shortened
loop at end waiting for history to be updated).
stages, even if the package is already there.
This could break EXTRA depends where the package would already be there,
and thus its dependency tree not really solved, and thus depends could
easily miss packages...
problem reported by nigel@... and not reported by ajacoutot@ :)
- expose MISSING_FILES
- treat them as a special kind of IGNORE
- handle that in DPB
that way, ports that are IGNORE'd for other reasons (not correct arch)
stay ignored, and relevant ports with missing distfiles get better error
messages
- try resolving RDEPENDS on tobuild, so that known dependencie are zapped.
- use equivalence info to not reprint same thing twice.
- dump tobuild AND built, since both are usually relevant in case lots of
stuff didn't build
- instead of seen/unseen, have an actual constructor. Instead, mark pkgpath
for which we wantinfo/wantbuild.
- only mark EXTRA dependencies as wantinfo. So the devel/haddock,no_deps
temporary error should be gone.
- since we have FLAVOR and SUBPACKAGE available, construct as much info as
we can during vars scanning (see handle_equivalences). This avoids about 150
path rescans during a full bulk. Also, grab the timing and logsizes from
equivalent files, so that most stuff should know show % all the time.
- tweak subdirlist to be a hash, and correctly add pkgpath_and_flavors to it.
That way, we rescan avahi pseudo flavors just once, and not four or five times.
eventually.
- fetch all files
- ignore ignores
- specific builder that doesn't look at existing packages
currently: does not stop when fetch is finished, which is somewhat of the
remaining issue.
Also: change stats to store pid, to make sense of interleaved log files.
- zap heuristic#3, not critical
- don't rebuild the Fetch queue all the time, it's too expensive
- fix bugs in FetchQueue computation, correctly handle 1 distfile case
- create a quick path thru adjust* that handles only new paths.
- use it during LISTING while fetching files, to make more files available
quicker.
- model distfiles as FDEPENDS, DIST, so that they can get dumped.
- add -v option that dumps a bit more stuff in log files.
with these, dpb will fetch all distfiles in a not too surprising order,
and it won't consume 40% cpu while doing so...
it more obvious how much is shared (goal is to unify a large part of these).
In particular, the locking code/stuff already active is glaringly the same...
Polish the distfiles measurements. For now, 3 heuristics that kick in
at various stages of the build (a bit of black magic).
Fix pkgpath->{has} to record both "no dependencies" and "several distfiles
missing" to avoid grabbing small patchfiles which don't lead to valid
builds while heuristic#1 is active.
- clean up clock handling, jobs with timer.
- pass more parameters thru state
- tweak the engine to handle distfiles
- lots of new code in Fetch for new jobs.
Deprecate: -t and -T
use -DCONNECTION_TIMEOUT=... -DDISPLAY_TIMEOUT=... instead
New feature: -DSTUCK_TIMEOUT=
kill tasks when they don't show any progress for that long.
makes for a "quick listing" of the most important ports first (e.g., groff).
document -R, even though it's not perfect yet.
flesh out all the recent parts and update the BUGS AND LIMITATIONS section.
Somewhat inefficient (deep check of package vs port signature).
More proper approach would be to tell the engine to always build, have first
step be a check for signature if the package exist, and abort if it matches:
- here we double check everything
- propagating rebuilds means non-uptodate packages give rise to non uptodate.
- if several cores are available, recheck errors, so that we're not stuck
waiting for a core to finish to start new shit.
- $logdir/stop-<machine_name> allows you to stop a given machine while
you clean it up and (possibly) restart it (e.g., won't start new jobs)
