Commit Graph

77 Commits

Author SHA1 Message Date
espie
22f2dd40c8 give a bigger weight to unknown paths 2011-07-14 11:03:13 +00:00
espie
2cf4d84ffd fix a long-standing bug: add LIB_DEPENDS to the RUN_DEPENDS of a port, as
there's a safeguard against self-depends anyways.
This prevents ports from getting moved from "built" to "installable"
prematurely.

For instance, devel/libsoup,-gnome doesn't directly depend on
net/glib2-networking, but it depends on devel/libsoup,-main which has
the net/glib2-networking.

Without this patch, dependencies on devel/libsoup,-gnome would often build
prematurely and build net/glib2-networking with them.
2011-06-15 10:09:31 +00:00
espie
755907c973 parse regression info if present 2011-06-15 10:06:22 +00:00
espie
91419bfb7c spaces 2011-06-04 12:58:24 +00:00
espie
64fc6d3cf2 keep a better list of depends, so that pkg_add does add exactly what it should 2011-06-04 12:56:54 +00:00
espie
8579c8b7c3 do the fetch stuff very early, in new_path. 2011-06-03 13:38:58 +00:00
espie
9ba6fb7c5c - unify code in Fetch/Build subengines
- zap heuristic#3, not critical
- don't rebuild the Fetch queue all the time, it's too expensive
- fix bugs in FetchQueue computation, correctly handle 1 distfile case
- create a quick path thru adjust* that handles only new paths.
- use it during LISTING while fetching files, to make more files available
quicker.
- model distfiles as FDEPENDS, DIST, so that they can get dumped.
- add -v option that dumps a bit more stuff in log files.

with these, dpb will fetch all distfiles in a not too surprising order,
and it won't consume 40% cpu while doing so...
2011-06-02 17:09:25 +00:00
espie
7d68d7e282 add stuff to do quick_dump of distinfo eventually 2011-06-02 11:53:55 +00:00
espie
be6847e3c2 avoid fetching distfiles where we don't know dependencies yet 2011-06-01 15:39:47 +00:00
espie
72c6360806 optimize heuristics for fetchqueue, as they take too much time 2011-06-01 12:34:29 +00:00
espie
647950c27f typo, duplicate work 2011-06-01 12:34:09 +00:00
espie
1e22893e8e if we -f -a, then scan SUPBDISTFILES too, and use the Distfile cache
to write a list of all known distfiles, that can be used to remove extra
stuff from /usr/ports/distfiles
2011-05-29 11:06:23 +00:00
espie
84266ee8ac if we're running -f, do the checksum part internally, since we already
checksummed stuff we fetched (will allow auto-refetch eventually).
2011-05-29 09:30:13 +00:00
espie
aeae163983 oops, the Builder is one indirection down now 2011-05-28 19:32:59 +00:00
espie
3f2603c2f3 fix unlocking in case of errors 2011-05-28 19:24:15 +00:00
espie
9f989ffafe fix 2011-05-28 08:21:39 +00:00
espie
f4d6937978 nicer lockname 2011-05-27 13:13:43 +00:00
espie
2fd23899dd explain the dump_dependencies/find_best optimization for quick-restarting
dpb.
2011-05-27 11:22:13 +00:00
espie
7ef05495f5 start moving the fetch/build code into separate subengine classes, to make
it more obvious how much is shared (goal is to unify a large part of these).

In particular, the locking code/stuff already active is glaringly the same...

Polish the distfiles measurements. For now, 3 heuristics that kick in
at various stages of the build (a bit of black magic).

Fix pkgpath->{has} to record both "no dependencies" and "several distfiles
missing" to avoid grabbing small patchfiles which don't lead to valid
builds while heuristic#1 is active.
2011-05-27 10:27:50 +00:00
espie
a819e7f3e8 improved fetch: do checksum as a separate process, to avoid freezing
dpb while checksumming huge tarballs.
Use ftp -C in a systematic way, more complicated logic as to when to
remove temp files, when to keep them: if core exited okay, wrong size
is very bad. If checksum failed on a partial fetch, retry same site with
an empty file...
2011-05-23 09:44:05 +00:00
espie
0c1194b39b only check checksum if we just fetched it, for now. 2011-05-22 09:19:08 +00:00
espie
d2a3a0fa83 dyslexia 2011-05-22 09:06:49 +00:00
espie
fbfc4fba47 fix checksum logging, avoid checksumming twice 2011-05-22 09:01:27 +00:00
espie
8cef021866 initial support for fetching distfiles, somewhat experimental yet.
- clean up clock handling, jobs with timer.
- pass more parameters thru state
- tweak the engine to handle distfiles
- lots of new code in Fetch for new jobs.
2011-05-22 08:21:38 +00:00
espie
afdb81839a finish refactoring options.
Deprecate: -t and -T
use -DCONNECTION_TIMEOUT=... -DDISPLAY_TIMEOUT=... instead
New feature: -DSTUCK_TIMEOUT=
kill tasks when they don't show any progress for that long.
2011-04-25 11:58:46 +00:00
espie
e7a4ca8c8a ignore whitespace 2011-04-18 11:40:23 +00:00
espie
953361b9c4 in case we log errors, go to the next iteration correctly.
log the fullpkgpath explicitly when we can.
2011-04-05 08:24:34 +00:00
espie
d76bdca371 tweak to allow subclasses of cores that get their own separate lists
(such as local fetcher jobs)
2011-03-22 19:49:56 +00:00
espie
cabe19ad60 tidy, minor 2011-03-22 19:48:53 +00:00
espie
fc9fbe61ee add the glue that parses fetch info and build distinfo for future fetching.
also let Vars report stuff it doesn't know about to the logs file.
2011-03-22 19:48:01 +00:00
espie
c6ed94f022 not sure why it happens, but log nopkgname... 2010-12-12 21:21:03 +00:00
espie
97286a5a18 better info rebuild: make Vars responsible for creating info "just in time"
by keeping local state.
That way, when we rescan, we don't have to remove any info, and all affected
infos are replaced at the same time.
2010-12-07 10:56:26 +00:00
espie
9c270c05b6 fix fix
$build_info //= 1  indeed... ;(
2010-12-06 19:35:06 +00:00
espie
ab01800701 tag dependencies discovered as pure dependencies with their parents, allow
finding out non-sensical stuff in an easier way
2010-12-06 13:20:45 +00:00
espie
d317d925fe minor warning fix 2010-12-06 13:19:44 +00:00
espie
9ed0d9cd66 move most of libtool code around. 2010-12-05 16:37:50 +00:00
espie
4d8b6b0032 allow 'dpb archivers' without extra artifacts, e.g., recognize and
handle categories without classifying them as pkgpaths with errors.
2010-12-05 10:06:12 +00:00
espie
1d73dd9276 also add older LIB_DEPENDS to WANTLIB 2010-11-22 21:16:42 +00:00
espie
1c0a277e2c prepare to deal with new depends 2010-11-14 07:44:53 +00:00
espie
e2d1250c4c show everything as WANTLIB for now, this needs more love obviously 2010-11-11 12:43:03 +00:00
espie
47bc97ea5c fix stupid warnings 2010-11-11 12:17:20 +00:00
espie
82ae79eeca in case the signature doesn't match, clean ALL packages that will be
rebuilt. Otherwise, you may suffer through 3 or 4 rebuilds of qt3...

ouchie.
2010-11-02 20:32:59 +00:00
espie
fb3a3c80f7 much better implementation of -R. In case of rebuilds, check does not "see"
pkgfiles, unless they've been registered.
the engine queues everything to build, but normal build is only an
existence/signature check. The actual build tasks only get queued when
the package does not exist, or if its signature does NOT match the ports
tree.

Should make matthieu@ happy.
2010-11-01 10:55:26 +00:00
espie
8a26786deb new file/option: save dependencies and be able to reuse them.
makes for a "quick listing" of the most important ports first (e.g., groff).
document -R, even though it's not perfect yet.
flesh out all the recent parts and update the BUGS AND LIMITATIONS section.
2010-10-31 11:07:20 +00:00
espie
f7f23f3bdd don't cheat, we have to make clean=package to avoid side-effects with
arch-independent packages.
2010-10-30 11:36:07 +00:00
espie
f072ab3a17 proof-of-concept implementation of -R (rebuild when signature changes).
Somewhat inefficient (deep check of package vs port signature).
More proper approach would be to tell the engine to always build, have first
step be a check for signature if the package exist, and abort if it matches:
- here we double check everything
- propagating rebuilds means non-uptodate packages give rise to non uptodate.
2010-10-30 11:19:38 +00:00
espie
86be07b384 simplify parameter passing slightly 2010-10-30 10:35:09 +00:00
espie
03b44c7dac refactor to make most of it reusable to e.g., make print-package-signature 2010-10-29 12:35:00 +00:00
espie
10e6cc104f engine improvements:
- if several cores are available, recheck errors, so that we're not stuck
waiting for a core to finish to start new shit.
- $logdir/stop-<machine_name> allows you to stop a given machine while
you clean it up and (possibly) restart it (e.g., won't start new jobs)
2010-10-29 11:51:42 +00:00
landry
66327fcf42 Readd missing space before "on $hostname" 2010-10-28 22:14:54 +00:00