Commit Graph

23 Commits

Author SHA1 Message Date
espie
1ec4d51eba repair dpb -f0
noticed by phessler@
2012-01-18 15:26:49 +00:00
espie
3a6eaf5081 add support to obey PERMIT_DISTFILES* 2012-01-14 12:26:21 +00:00
espie
f8050aba6f run "expire_old" as a separate job, since it may take some time.
find files under $DISTDIR, anything we don't know about should also
end up in history.
2012-01-09 17:56:28 +00:00
espie
fa990a1dba give dpb more knowledge about distfiles.
- keep a stash indexed by checksum, so dpb can identify duplicate files.
- in a full bulk, if the scan has no errors, write to a ${DISTDIR}/history
file  the files encountered in ${DISTDIR}/distinfo that seem to no longer
be needed (with full timestamp and checksum info).

Should be enough info to know when to expire old DISTDIR entries.
2012-01-08 20:28:37 +00:00
espie
9938b28564 allow the fetcher part to work as a "true" mirror maker:
- keep a cache of checked checksums as ${DISTDIR}/distinfo
- read existing cache on startup, don't need to recheck those files.
- split checksum_task: update the cache after the checksum is done
- create the by_cipher/sha256/... entries
- in fetch_only mode, do a "long" checksum on existing files not in the cache.

If you ever have doubts on the integrity of already checked files, just
rm ${DISTDIR}/distinfo.
2011-12-31 11:20:00 +00:00
espie
a38035bd67 clean-up job running a bit: add methods to Grabber and PortBuilder objects
to access more stuff they need.

Remove distantshell->make, ask state instead.
Provide state with the program we run, and the parameters we pass it.

- same make for distant and local hosts
- pass BUILD_ONCE=Yes if -a.
2011-12-04 12:05:41 +00:00
espie
8bba5b93fa remove a bit more data 2011-11-07 13:21:46 +00:00
espie
ebe71b8762 slightly better error handling and reporting:
- log to pkgpath whenever we can
- log FETCH_MANUALLY issues as errors, so we can fix them
2011-11-06 12:23:28 +00:00
espie
234027bfe0 unfuck pkgpaths.
- instead of seen/unseen, have an actual constructor. Instead, mark pkgpath
for which we wantinfo/wantbuild.
- only mark EXTRA dependencies as wantinfo. So the devel/haddock,no_deps
temporary error should be gone.
- since we have FLAVOR and SUBPACKAGE available, construct as much info as
we can during vars scanning (see handle_equivalences). This avoids about 150
path rescans during a full bulk. Also, grab the timing and logsizes from
equivalent files, so that most stuff should know show % all the time.
- tweak subdirlist to be a hash, and correctly add pkgpath_and_flavors to it.
That way, we rescan avahi pseudo flavors just once, and not four or five times.
2011-10-10 18:56:50 +00:00
espie
1521428320 erroring once is enough... there should be some logfile for this too ;( 2011-09-25 10:40:25 +00:00
espie
91419bfb7c spaces 2011-06-04 12:58:24 +00:00
espie
8cef021866 initial support for fetching distfiles, somewhat experimental yet.
- clean up clock handling, jobs with timer.
- pass more parameters thru state
- tweak the engine to handle distfiles
- lots of new code in Fetch for new jobs.
2011-05-22 08:21:38 +00:00
espie
afdb81839a finish refactoring options.
Deprecate: -t and -T
use -DCONNECTION_TIMEOUT=... -DDISPLAY_TIMEOUT=... instead
New feature: -DSTUCK_TIMEOUT=
kill tasks when they don't show any progress for that long.
2011-04-25 11:58:46 +00:00
espie
4d8b6b0032 allow 'dpb archivers' without extra artifacts, e.g., recognize and
handle categories without classifying them as pkgpaths with errors.
2010-12-05 10:06:12 +00:00
espie
82ae79eeca in case the signature doesn't match, clean ALL packages that will be
rebuilt. Otherwise, you may suffer through 3 or 4 rebuilds of qt3...

ouchie.
2010-11-02 20:32:59 +00:00
espie
f7f23f3bdd don't cheat, we have to make clean=package to avoid side-effects with
arch-independent packages.
2010-10-30 11:36:07 +00:00
espie
f072ab3a17 proof-of-concept implementation of -R (rebuild when signature changes).
Somewhat inefficient (deep check of package vs port signature).
More proper approach would be to tell the engine to always build, have first
step be a check for signature if the package exist, and abort if it matches:
- here we double check everything
- propagating rebuilds means non-uptodate packages give rise to non uptodate.
2010-10-30 11:19:38 +00:00
espie
86be07b384 simplify parameter passing slightly 2010-10-30 10:35:09 +00:00
espie
1c010712c9 keep the output from dump-vars around, and save it in the logfile is
something bad happens.

get rid of special reporter for Vars, E= is enough, people should look
in the paths/logfile now.
2010-10-28 14:54:38 +00:00
espie
4a572ee2bd make it possible (simple) to rescan broken directories
(we should possibly log the error messages).
Forgot to reset broken, so rescanning didn't help...
2010-10-28 10:33:20 +00:00
espie
d4ee90931f tie engine with var grabber, get ready to re-do port meta on error. 2010-10-27 12:58:26 +00:00
espie
e694d95fa7 fix loop: we know we tried it, it didn't work, so don't try it AGAIN. 2010-10-23 18:02:59 +00:00
espie
4c6d1b431f move stuff here from infra/build 2010-08-20 13:40:13 +00:00